Pull drm updates from Dave Airlie: "Highlights are support for privacy screens found in new laptops, a bunch of nomodeset refactoring, and i915 enables ADL-P systems by default, while starting to add RPL-S support. vmwgfx adds GEM and support for OpenGL 4.3 features in userspace. Lots of internal refactorings around dma reservations, and lots of driver refactoring as well. Summary: core: - add privacy screen support - move nomodeset option into drm subsystem - clean up nomodeset handling in drivers - make drm_irq.c legacy - fix stack_depot name conflicts - remove DMA_BUF_SET_NAME ioctl restrictions - sysfs: send hotplug event - replace several DRM_* logging macros with drm_* - move hashtable to legacy code - add error return from gem_create_object - cma-helper: improve interfaces, drop CONFIG_DRM_KMS_CMA_HELPER - kernel.h related include cleanups - support XRGB2101010 source buffers ttm: - don't include drm hashtable - stop pruning fences after wait - documentation updates dma-buf: - add dma_resv selftest - add debugfs helpers - remove dma_resv_get_excl_unlocked - documentation - make fences mandatory in dma_resv_add_excl_fence dp: - add link training delay helpers gem: - link shmem/cma helpers into separate modules - use dma_resv iteratior - import dma-buf namespace into gem helper modules scheduler: - fence grab fix - lockdep fixes bridge: - switch to managed MIPI DSI helpers - register and attach during probe fixes - convert to YAML in several places. panel: - add bunch of new panesl simpledrm: - support FB_DAMAGE_CLIPS - support virtual screen sizes - add Apple M1 support amdgpu: - enable seamless boot for DCN 3.01 - runtime PM fixes - use drm_kms_helper_connector_hotplug_event - get all fences at once - use generic drm fb helpers - PSR/DPCD/LTTPR/DSC/PM/RAS/OLED/SRIOV fixes - add smart trace buffer (STB) for supported GPUs - display debugfs entries - new SMU debug option - Documentation update amdkfd: - IP discovery enumeration refactor - interface between driver fixes - SVM fixes - kfd uapi header to define some sysfs bitfields. i915: - support VESA panel backlights - enable ADL-P by default - add eDP privacy screen support - add Raptor Lake S (RPL-S) support - DG2 page table support - lots of GuC/HuC fw refactoring - refactored i915->gt interfaces - CD clock squashing support - enable 10-bit gamma support - update ADL-P DMC fw to v2.14 - enable runtime PM autosuspend by default - ADL-P DSI support - per-lane DP drive settings for ICL+ - add support for pipe C/D DMC firmware - Atomic gamma LUT updates - remove CCS FB stride restrictions on ADL-P - VRR platform support for display 11 - add support for display audio codec keepalive - lots of display refactoring - fix runtime PM handling during PXP suspend - improved eviction performance with async TTM moves - async VMA unbinding improvements - VMA locking refactoring - improved error capture robustness - use per device iommu checks - drop bits stealing from i915_sw_fence function ptr - remove dma_resv_prune - add IC cache invalidation on DG2 nouveau: - crc fixes - validate LUTs in atomic check - set HDMI AVI RGB quant to full tegra: - buffer objects reworks for dma-buf compat - NVDEC driver uAPI support - power management improvements etnaviv: - IOMMU enabled system support - fix > 4GB command buffer mapping - close a DoS vector - fix spurious GPU resets ast: - fix i2c initialization rcar-du: - DSI output support exynos: - replace legacy gpio interface - implement generic GEM object mmap msm: - dpu plane state cleanup in prep for multirect - dpu debugfs cleanups - dp support for sc7280 - a506 support - removal of struct_mutex - remove old eDP sub-driver anx7625: - support MIPI DSI input - support HDMI audio - fix reading EDID lvds: - fix bridge DT bindings megachips: - probe both bridges before registering dw-hdmi: - allow interlace on bridge ps8640: - enable runtime PM - support aux-bus tx358768: - enable reference clock - add pulse mode support ti-sn65dsi86: - use regmap bulk write - add PWM support etnaviv: - get all fences at once gma500: - gem object cleanups kmb: - enable fb console radeon: - use dma_resv_wait_timeout rockchip: - add DSP hold timeout - suspend/resume fixes - PLL clock fixes - implement mmap in GEM object functions - use generic fbdev emulation sun4i: - use CMA helpers without vmap support vc4: - fix HDMI-CEC hang with display is off - power on HDMI controller while disabling - support 4K@60Hz modes - support 10-bit YUV 4:2:0 output vmwgfx: - fix leak on probe errors - fail probing on broken hosts - new placement for MOB page tables - hide internal BOs from userspace - implement GEM support - implement GL 4.3 support virtio: - overflow fixes xen: - implement mmap as GEM object function omapdrm: - fix scatterlist export - support virtual planes mediatek: - MT8192 support - CMDQ refinement" * tag 'drm-next-2022-01-07' of git://anongit.freedesktop.org/drm/drm: (1241 commits) drm/amdgpu: no DC support for headless chips drm/amd/display: fix dereference before NULL check drm/amdgpu: always reset the asic in suspend (v2) drm/amdgpu: put SMU into proper state on runpm suspending for BOCO capable platform drm/amd/display: Fix the uninitialized variable in enable_stream_features() drm/amdgpu: fix runpm documentation amdgpu/pm: Make sysfs pm attributes as read-only for VFs drm/amdgpu: save error count in RAS poison handler drm/amdgpu: drop redundant semicolon drm/amd/display: get and restore link res map drm/amd/display: support dynamic HPO DP link encoder allocation drm/amd/display: access hpo dp link encoder only through link resource drm/amd/display: populate link res in both detection and validation drm/amd/display: define link res and make it accessible to all link interfaces drm/amd/display: 3.2.167 drm/amd/display: [FW Promotion] Release 0.0.98 drm/amd/display: Undo ODM combine drm/amd/display: Add reg defs for DCN303 drm/amd/display: Changed pipe split policy to allow for multi-display pipe split drm/amd/display: Set optimize_pwr_state for DCN31 ...
Unified · Split
| @@ -10,10 +10,12 @@ · 9 unmodified lines | |||
| 10 | # Please keep this list dictionary sorted. | 10 | # Please keep this list dictionary sorted. |
| 11 | # | 11 | # |
| 12 | Aaron Durbin <adurbin@google.com> | 12 | Aaron Durbin <adurbin@google.com> |
| 13 | Abhinav Kumar <quic_abhinavk@quicinc.com> <abhinavk@codeaurora.org> | ||
| 13 | Adam Oldham <oldhamca@gmail.com> | 14 | Adam Oldham <oldhamca@gmail.com> |
| 14 | Adam Radford <aradford@gmail.com> | 15 | Adam Radford <aradford@gmail.com> |
| 15 | Adriana Reus <adi.reus@gmail.com> <adriana.reus@intel.com> | 16 | Adriana Reus <adi.reus@gmail.com> <adriana.reus@intel.com> |
| 16 | Adrian Bunk <bunk@stusta.de> | 17 | Adrian Bunk <bunk@stusta.de> |
| 18 | Akhil P Oommen <quic_akhilpo@quicinc.com> <akhilpo@codeaurora.org> | ||
| 17 | Alan Cox <alan@lxorguk.ukuu.org.uk> | 19 | Alan Cox <alan@lxorguk.ukuu.org.uk> |
| 18 | Alan Cox <root@hraefn.swansea.linux.org.uk> | 20 | Alan Cox <root@hraefn.swansea.linux.org.uk> |
| 19 | Aleksandar Markovic <aleksandar.markovic@mips.com> <aleksandar.markovic@imgtec.com> | 21 | Aleksandar Markovic <aleksandar.markovic@mips.com> <aleksandar.markovic@imgtec.com> |
| @@ -172,6 +174,7 @@ Jeff Layton <jlayton@kernel.org> <jlayton@redhat.com> · 152 unmodified lines | |||
| 172 | Jens Axboe <axboe@suse.de> | 174 | Jens Axboe <axboe@suse.de> |
| 173 | Jens Osterkamp <Jens.Osterkamp@de.ibm.com> | 175 | Jens Osterkamp <Jens.Osterkamp@de.ibm.com> |
| 174 | Jernej Skrabec <jernej.skrabec@gmail.com> <jernej.skrabec@siol.net> | 176 | Jernej Skrabec <jernej.skrabec@gmail.com> <jernej.skrabec@siol.net> |
| 177 | Jessica Zhang <quic_jesszhan@quicinc.com> <jesszhan@codeaurora.org> | ||
| 175 | Jiri Slaby <jirislaby@kernel.org> <jirislaby@gmail.com> | 178 | Jiri Slaby <jirislaby@kernel.org> <jirislaby@gmail.com> |
| 176 | Jiri Slaby <jirislaby@kernel.org> <jslaby@novell.com> | 179 | Jiri Slaby <jirislaby@kernel.org> <jslaby@novell.com> |
| 177 | Jiri Slaby <jirislaby@kernel.org> <jslaby@suse.com> | 180 | Jiri Slaby <jirislaby@kernel.org> <jslaby@suse.com> |
| @@ -191,6 +194,7 @@ Juha Yrjola <at solidboot.com> · 13 unmodified lines | |||
| 191 | Juha Yrjola <juha.yrjola@nokia.com> | 194 | Juha Yrjola <juha.yrjola@nokia.com> |
| 192 | Juha Yrjola <juha.yrjola@solidboot.com> | 195 | Juha Yrjola <juha.yrjola@solidboot.com> |
| 193 | Julien Thierry <julien.thierry.kdev@gmail.com> <julien.thierry@arm.com> | 196 | Julien Thierry <julien.thierry.kdev@gmail.com> <julien.thierry@arm.com> |
| 197 | Kalyan Thota <quic_kalyant@quicinc.com> <kalyan_t@codeaurora.org> | ||
| 194 | Kay Sievers <kay.sievers@vrfy.org> | 198 | Kay Sievers <kay.sievers@vrfy.org> |
| 195 | Kees Cook <keescook@chromium.org> <kees.cook@canonical.com> | 199 | Kees Cook <keescook@chromium.org> <kees.cook@canonical.com> |
| 196 | Kees Cook <keescook@chromium.org> <keescook@google.com> | 200 | Kees Cook <keescook@chromium.org> <keescook@google.com> |
| @@ -202,9 +206,11 @@ Kenneth W Chen <kenneth.w.chen@intel.com> · 5 unmodified lines | |||
| 202 | Konstantin Khlebnikov <koct9i@gmail.com> <khlebnikov@yandex-team.ru> | 206 | Konstantin Khlebnikov <koct9i@gmail.com> <khlebnikov@yandex-team.ru> |
| 203 | Konstantin Khlebnikov <koct9i@gmail.com> <k.khlebnikov@samsung.com> | 207 | Konstantin Khlebnikov <koct9i@gmail.com> <k.khlebnikov@samsung.com> |
| 204 | Koushik <raghavendra.koushik@neterion.com> | 208 | Koushik <raghavendra.koushik@neterion.com> |
| 209 | Krishna Manikandan <quic_mkrishn@quicinc.com> <mkrishn@codeaurora.org> | ||
| 205 | Krzysztof Kozlowski <krzk@kernel.org> <k.kozlowski.k@gmail.com> | 210 | Krzysztof Kozlowski <krzk@kernel.org> <k.kozlowski.k@gmail.com> |
| 206 | Krzysztof Kozlowski <krzk@kernel.org> <k.kozlowski@samsung.com> | 211 | Krzysztof Kozlowski <krzk@kernel.org> <k.kozlowski@samsung.com> |
| 207 | Kuninori Morimoto <kuninori.morimoto.gx@renesas.com> | 212 | Kuninori Morimoto <kuninori.morimoto.gx@renesas.com> |
| 213 | Kuogee Hsieh <quic_khsieh@quicinc.com> <khsieh@codeaurora.org> | ||
| 208 | Leonardo Bras <leobras.c@gmail.com> <leonardo@linux.ibm.com> | 214 | Leonardo Bras <leobras.c@gmail.com> <leonardo@linux.ibm.com> |
| 209 | Leonid I Ananiev <leonid.i.ananiev@intel.com> | 215 | Leonid I Ananiev <leonid.i.ananiev@intel.com> |
| 210 | Leon Romanovsky <leon@kernel.org> <leon@leon.nu> | 216 | Leon Romanovsky <leon@kernel.org> <leon@leon.nu> |
| @@ -311,6 +317,7 @@ Qais Yousef <qsyousef@gmail.com> <qais.yousef@imgtec.com> · 100 unmodified lines | |||
| 311 | Quentin Monnet <quentin@isovalent.com> <quentin.monnet@netronome.com> | 317 | Quentin Monnet <quentin@isovalent.com> <quentin.monnet@netronome.com> |
| 312 | Quentin Perret <qperret@qperret.net> <quentin.perret@arm.com> | 318 | Quentin Perret <qperret@qperret.net> <quentin.perret@arm.com> |
| 313 | Rafael J. Wysocki <rjw@rjwysocki.net> <rjw@sisk.pl> | 319 | Rafael J. Wysocki <rjw@rjwysocki.net> <rjw@sisk.pl> |
| 320 | Rajeev Nandan <quic_rajeevny@quicinc.com> <rajeevny@codeaurora.org> | ||
| 314 | Rajesh Shah <rajesh.shah@intel.com> | 321 | Rajesh Shah <rajesh.shah@intel.com> |
| 315 | Ralf Baechle <ralf@linux-mips.org> | 322 | Ralf Baechle <ralf@linux-mips.org> |
| 316 | Ralf Wildenhues <Ralf.Wildenhues@gmx.de> | 323 | Ralf Wildenhues <Ralf.Wildenhues@gmx.de> |
| @@ -325,6 +332,7 @@ Rui Saraiva <rmps@joel.ist.utl.pt> · 8 unmodified lines | |||
| 325 | Sachin P Sant <ssant@in.ibm.com> | 332 | Sachin P Sant <ssant@in.ibm.com> |
| 326 | Sakari Ailus <sakari.ailus@linux.intel.com> <sakari.ailus@iki.fi> | 333 | Sakari Ailus <sakari.ailus@linux.intel.com> <sakari.ailus@iki.fi> |
| 327 | Sam Ravnborg <sam@mars.ravnborg.org> | 334 | Sam Ravnborg <sam@mars.ravnborg.org> |
| 335 | Sankeerth Billakanti <quic_sbillaka@quicinc.com> <sbillaka@codeaurora.org> | ||
| 328 | Santosh Shilimkar <santosh.shilimkar@oracle.org> | 336 | Santosh Shilimkar <santosh.shilimkar@oracle.org> |
| 329 | Santosh Shilimkar <ssantosh@kernel.org> | 337 | Santosh Shilimkar <ssantosh@kernel.org> |
| 330 | Sarangdhar Joshi <spjoshi@codeaurora.org> | 338 | Sarangdhar Joshi <spjoshi@codeaurora.org> |
| @@ -3551,6 +3551,13 @@ · 3550 unmodified lines | |||
| 3551 | shutdown the other cpus. Instead use the REBOOT_VECTOR | 3551 | shutdown the other cpus. Instead use the REBOOT_VECTOR |
| 3552 | irq. | 3552 | irq. |
| 3553 | 3553 | ||
| 3554 | nomodeset Disable kernel modesetting. DRM drivers will not perform | ||
| 3555 | display-mode changes or accelerated rendering. Only the | ||
| 3556 | system framebuffer will be available for use if this was | ||
| 3557 | set-up by the firmware or boot loader. | ||
| 3558 | |||
| 3559 | Useful as fallback, or for testing and debugging. | ||
| 3560 | |||
| 3554 | nomodule Disable module load | 3561 | nomodule Disable module load |
| 3555 | 3562 | ||
| 3556 | nopat [X86] Disable PAT (page attribute table extension of | 3563 | nopat [X86] Disable PAT (page attribute table extension of |
| @@ -43,14 +43,70 @@ properties: · 42 unmodified lines | |||
| 43 | vdd33-supply: | 43 | vdd33-supply: |
| 44 | description: Regulator that provides the supply 3.3V power. | 44 | description: Regulator that provides the supply 3.3V power. |
| 45 | 45 | ||
| 46 | analogix,lane0-swing: | ||
| 47 | $ref: /schemas/types.yaml#/definitions/uint8-array | ||
| 48 | minItems: 1 | ||
| 49 | maxItems: 20 | ||
| 50 | description: | ||
| 51 | an array of swing register setting for DP tx lane0 PHY. | ||
| 52 | Registers 0~9 are Swing0_Pre0, Swing1_Pre0, Swing2_Pre0, | ||
| 53 | Swing3_Pre0, Swing0_Pre1, Swing1_Pre1, Swing2_Pre1, Swing0_Pre2, | ||
| 54 | Swing1_Pre2, Swing0_Pre3, they are for [Boost control] and | ||
| 55 | [Swing control] setting. | ||
| 56 | Registers 0~9, bit 3:0 is [Boost control], these bits control | ||
| 57 | post cursor manual, increase the [Boost control] to increase | ||
| 58 | Pre-emphasis value. | ||
| 59 | Registers 0~9, bit 6:4 is [Swing control], these bits control | ||
| 60 | swing manual, increase [Swing control] setting to add Vp-p value | ||
| 61 | for each Swing, Pre. | ||
| 62 | Registers 10~19 are Swing0_Pre0, Swing1_Pre0, Swing2_Pre0, | ||
| 63 | Swing3_Pre0, Swing0_Pre1, Swing1_Pre1, Swing2_Pre1, Swing0_Pre2, | ||
| 64 | Swing1_Pre2, Swing0_Pre3, they are for [R select control] and | ||
| 65 | [R Termination control] setting. | ||
| 66 | Registers 10~19, bit 4:0 is [R select control], these bits are | ||
| 67 | compensation manual, increase it can enhance IO driven strength | ||
| 68 | and Vp-p. | ||
| 69 | Registers 10~19, bit 5:6 is [R termination control], these bits | ||
| 70 | adjust 50ohm impedance of DP tx termination. 00:55 ohm, | ||
| 71 | 01:50 ohm(default), 10:45 ohm, 11:40 ohm. | ||
| 72 | |||
| 73 | analogix,lane1-swing: | ||
| 74 | $ref: /schemas/types.yaml#/definitions/uint8-array | ||
| 75 | minItems: 1 | ||
| 76 | maxItems: 20 | ||
| 77 | description: | ||
| 78 | an array of swing register setting for DP tx lane1 PHY. | ||
| 79 | DP TX lane1 swing register setting same with lane0 | ||
| 80 | swing, please refer lane0-swing property description. | ||
| 81 | |||
| 82 | analogix,audio-enable: | ||
| 83 | type: boolean | ||
| 84 | description: let the driver enable audio HDMI codec function or not. | ||
| 85 | |||
| 46 | ports: | 86 | ports: |
| 47 | $ref: /schemas/graph.yaml#/properties/ports | 87 | $ref: /schemas/graph.yaml#/properties/ports |
| 48 | 88 | ||
| 49 | properties: | 89 | properties: |
| 50 | port@0: | 90 | port@0: |
| 51 | $ref: /schemas/graph.yaml#/properties/port | 91 | $ref: /schemas/graph.yaml#/$defs/port-base |
| 92 | unevaluatedProperties: false | ||
| 52 | description: | 93 | description: |
| 53 | Video port for MIPI DSI input. | 94 | MIPI DSI/DPI input. |
| 95 | |||
| 96 | properties: | ||
| 97 | endpoint: | ||
| 98 | $ref: /schemas/media/video-interfaces.yaml# | ||
| 99 | type: object | ||
| 100 | additionalProperties: false | ||
| 101 | |||
| 102 | properties: | ||
| 103 | remote-endpoint: true | ||
| 104 | |||
| 105 | bus-type: | ||
| 106 | enum: [1, 5] | ||
| 107 | default: 1 | ||
| 108 | |||
| 109 | data-lanes: true | ||
| 54 | 110 | ||
| 55 | port@1: | 111 | port@1: |
| 56 | $ref: /schemas/graph.yaml#/properties/port | 112 | $ref: /schemas/graph.yaml#/properties/port |
| @@ -87,6 +143,9 @@ examples: · 30 unmodified lines | |||
| 87 | vdd10-supply = <&pp1000_mipibrdg>; | 143 | vdd10-supply = <&pp1000_mipibrdg>; |
| 88 | vdd18-supply = <&pp1800_mipibrdg>; | 144 | vdd18-supply = <&pp1800_mipibrdg>; |
| 89 | vdd33-supply = <&pp3300_mipibrdg>; | 145 | vdd33-supply = <&pp3300_mipibrdg>; |
| 146 | analogix,audio-enable; | ||
| 147 | analogix,lane0-swing = /bits/ 8 <0x14 0x54 0x64 0x74>; | ||
| 148 | analogix,lane1-swing = /bits/ 8 <0x14 0x54 0x64 0x74>; | ||
| 90 | 149 | ||
| 91 | ports { | 150 | ports { |
| 92 | #address-cells = <1>; | 151 | #address-cells = <1>; |
| @@ -96,6 +155,8 @@ examples: · 3 unmodified lines | |||
| 96 | reg = <0>; | 155 | reg = <0>; |
| 97 | anx7625_in: endpoint { | 156 | anx7625_in: endpoint { |
| 98 | remote-endpoint = <&mipi_dsi>; | 157 | remote-endpoint = <&mipi_dsi>; |
| 158 | bus-type = <5>; | ||
| 159 | data-lanes = <0 1 2 3>; | ||
| 99 | }; | 160 | }; |
| 100 | }; | 161 | }; |
| 101 | 162 | ||
| @@ -79,6 +79,14 @@ properties: · 78 unmodified lines | |||
| 79 | - port@0 | 79 | - port@0 |
| 80 | - port@1 | 80 | - port@1 |
| 81 | 81 | ||
| 82 | pclk-sample: | ||
| 83 | description: | ||
| 84 | Data sampling on rising or falling edge. | ||
| 85 | enum: | ||
| 86 | - 0 # Falling edge | ||
| 87 | - 1 # Rising edge | ||
| 88 | default: 0 | ||
| 89 | |||
| 82 | powerdown-gpios: | 90 | powerdown-gpios: |
| 83 | description: | 91 | description: |
| 84 | The GPIO used to control the power down line of this device. | 92 | The GPIO used to control the power down line of this device. |
| @@ -86,21 +94,32 @@ properties: · 1 unmodified line | |||
| 86 | 94 | ||
| 87 | power-supply: true | 95 | power-supply: true |
| 88 | 96 | ||
| 89 | if: | 97 | allOf: |
| 90 | not: | 98 | - if: |
| 91 | properties: | 99 | not: |
| 92 | compatible: | 100 | properties: |
| 93 | contains: | 101 | compatible: |
| 94 | const: lvds-decoder | 102 | contains: |
| 95 | then: | 103 | const: lvds-decoder |
| 96 | properties: | 104 | then: |
| 97 | ports: | ||
| 98 | properties: | 105 | properties: |
| 99 | port@0: | 106 | ports: |
| 100 | properties: | 107 | properties: |
| 101 | endpoint: | 108 | port@0: |
| 102 | properties: | 109 | properties: |
| 103 | data-mapping: false | 110 | endpoint: |
| 111 | properties: | ||
| 112 | data-mapping: false | ||
| 113 | |||
| 114 | - if: | ||
| 115 | not: | ||
| 116 | properties: | ||
| 117 | compatible: | ||
| 118 | contains: | ||
| 119 | const: lvds-encoder | ||
| 120 | then: | ||
| 121 | properties: | ||
| 122 | pclk-sample: false | ||
| 104 | 123 | ||
| 105 | required: | 124 | required: |
| 106 | - compatible | 125 | - compatible |
| @@ -0,0 +1,106 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/bridge/nxp,ptn3460.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: NXP PTN3460 eDP to LVDS bridge | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Sean Paul <seanpaul@chromium.org> | ||
| 11 | |||
| 12 | properties: | ||
| 13 | compatible: | ||
| 14 | const: nxp,ptn3460 | ||
| 15 | |||
| 16 | reg: | ||
| 17 | description: I2C address of the bridge | ||
| 18 | maxItems: 1 | ||
| 19 | |||
| 20 | edid-emulation: | ||
| 21 | $ref: "/schemas/types.yaml#/definitions/uint32" | ||
| 22 | description: | ||
| 23 | The EDID emulation entry to use | ||
| 24 | Value Resolution Description | ||
| 25 | 0 1024x768 NXP Generic | ||
| 26 | 1 1920x1080 NXP Generic | ||
| 27 | 2 1920x1080 NXP Generic | ||
| 28 | 3 1600x900 Samsung LTM200KT | ||
| 29 | 4 1920x1080 Samsung LTM230HT | ||
| 30 | 5 1366x768 NXP Generic | ||
| 31 | 6 1600x900 ChiMei M215HGE | ||
| 32 | enum: [0, 1, 2, 3, 4, 5, 6] | ||
| 33 | |||
| 34 | powerdown-gpios: | ||
| 35 | description: GPIO connected to the PD_N signal. | ||
| 36 | maxItems: 1 | ||
| 37 | |||
| 38 | reset-gpios: | ||
| 39 | description: GPIO connected to the RST_N signal. | ||
| 40 | maxItems: 1 | ||
| 41 | |||
| 42 | ports: | ||
| 43 | $ref: /schemas/graph.yaml#/properties/ports | ||
| 44 | |||
| 45 | properties: | ||
| 46 | port@0: | ||
| 47 | $ref: /schemas/graph.yaml#/properties/port | ||
| 48 | description: | ||
| 49 | Video port for LVDS output | ||
| 50 | |||
| 51 | port@1: | ||
| 52 | $ref: /schemas/graph.yaml#/properties/port | ||
| 53 | description: | ||
| 54 | Video port for eDP input | ||
| 55 | |||
| 56 | required: | ||
| 57 | - port@0 | ||
| 58 | - port@1 | ||
| 59 | |||
| 60 | required: | ||
| 61 | - compatible | ||
| 62 | - reg | ||
| 63 | - edid-emulation | ||
| 64 | - powerdown-gpios | ||
| 65 | - reset-gpios | ||
| 66 | - ports | ||
| 67 | |||
| 68 | additionalProperties: false | ||
| 69 | |||
| 70 | examples: | ||
| 71 | - | | ||
| 72 | #include <dt-bindings/gpio/gpio.h> | ||
| 73 | |||
| 74 | i2c1 { | ||
| 75 | #address-cells = <1>; | ||
| 76 | #size-cells = <0>; | ||
| 77 | |||
| 78 | bridge@20 { | ||
| 79 | compatible = "nxp,ptn3460"; | ||
| 80 | reg = <0x20>; | ||
| 81 | edid-emulation = <5>; | ||
| 82 | powerdown-gpios = <&gpy2 5 GPIO_ACTIVE_HIGH>; | ||
| 83 | reset-gpios = <&gpx1 5 GPIO_ACTIVE_LOW>; | ||
| 84 | |||
| 85 | ports { | ||
| 86 | #address-cells = <1>; | ||
| 87 | #size-cells = <0>; | ||
| 88 | |||
| 89 | port@0 { | ||
| 90 | reg = <0>; | ||
| 91 | bridge_out: endpoint { | ||
| 92 | remote-endpoint = <&panel_in>; | ||
| 93 | }; | ||
| 94 | }; | ||
| 95 | |||
| 96 | port@1 { | ||
| 97 | reg = <1>; | ||
| 98 | bridge_in: endpoint { | ||
| 99 | remote-endpoint = <&dp_out>; | ||
| 100 | }; | ||
| 101 | }; | ||
| 102 | }; | ||
| 103 | }; | ||
| 104 | }; | ||
| 105 | |||
| 106 | ... | ||
| @@ -1,39 +0,0 @@ | |||
| 1 | ptn3460 bridge bindings | ||
| 2 | |||
| 3 | Required properties: | ||
| 4 | - compatible: "nxp,ptn3460" | ||
| 5 | - reg: i2c address of the bridge | ||
| 6 | - powerdown-gpio: OF device-tree gpio specification for PD_N pin. | ||
| 7 | - reset-gpio: OF device-tree gpio specification for RST_N pin. | ||
| 8 | - edid-emulation: The EDID emulation entry to use | ||
| 9 | +-------+------------+------------------+ | ||
| 10 | | Value | Resolution | Description | | ||
| 11 | | 0 | 1024x768 | NXP Generic | | ||
| 12 | | 1 | 1920x1080 | NXP Generic | | ||
| 13 | | 2 | 1920x1080 | NXP Generic | | ||
| 14 | | 3 | 1600x900 | Samsung LTM200KT | | ||
| 15 | | 4 | 1920x1080 | Samsung LTM230HT | | ||
| 16 | | 5 | 1366x768 | NXP Generic | | ||
| 17 | | 6 | 1600x900 | ChiMei M215HGE | | ||
| 18 | +-------+------------+------------------+ | ||
| 19 | |||
| 20 | - video interfaces: Device node can contain video interface port | ||
| 21 | nodes for panel according to [1]. | ||
| 22 | |||
| 23 | [1]: Documentation/devicetree/bindings/media/video-interfaces.txt | ||
| 24 | |||
| 25 | Example: | ||
| 26 | lvds-bridge@20 { | ||
| 27 | compatible = "nxp,ptn3460"; | ||
| 28 | reg = <0x20>; | ||
| 29 | powerdown-gpio = <&gpy2 5 1 0 0>; | ||
| 30 | reset-gpio = <&gpx1 5 1 0 0>; | ||
| 31 | edid-emulation = <5>; | ||
| 32 | ports { | ||
| 33 | port@0 { | ||
| 34 | bridge_out: endpoint { | ||
| 35 | remote-endpoint = <&panel_in>; | ||
| 36 | }; | ||
| 37 | }; | ||
| 38 | }; | ||
| 39 | }; | ||
| @@ -0,0 +1,118 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/bridge/renesas,dsi-csi2-tx.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Renesas R-Car MIPI DSI/CSI-2 Encoder | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com> | ||
| 11 | |||
| 12 | description: | | ||
| 13 | This binding describes the MIPI DSI/CSI-2 encoder embedded in the Renesas | ||
| 14 | R-Car V3U SoC. The encoder can operate in either DSI or CSI-2 mode, with up | ||
| 15 | to four data lanes. | ||
| 16 | |||
| 17 | properties: | ||
| 18 | compatible: | ||
| 19 | enum: | ||
| 20 | - renesas,r8a779a0-dsi-csi2-tx # for V3U | ||
| 21 | |||
| 22 | reg: | ||
| 23 | maxItems: 1 | ||
| 24 | |||
| 25 | clocks: | ||
| 26 | items: | ||
| 27 | - description: Functional clock | ||
| 28 | - description: DSI (and CSI-2) functional clock | ||
| 29 | - description: PLL reference clock | ||
| 30 | |||
| 31 | clock-names: | ||
| 32 | items: | ||
| 33 | - const: fck | ||
| 34 | - const: dsi | ||
| 35 | - const: pll | ||
| 36 | |||
| 37 | power-domains: | ||
| 38 | maxItems: 1 | ||
| 39 | |||
| 40 | resets: | ||
| 41 | maxItems: 1 | ||
| 42 | |||
| 43 | ports: | ||
| 44 | $ref: /schemas/graph.yaml#/properties/ports | ||
| 45 | |||
| 46 | properties: | ||
| 47 | port@0: | ||
| 48 | $ref: /schemas/graph.yaml#/properties/port | ||
| 49 | description: Parallel input port | ||
| 50 | |||
| 51 | port@1: | ||
| 52 | $ref: /schemas/graph.yaml#/$defs/port-base | ||
| 53 | unevaluatedProperties: false | ||
| 54 | description: DSI/CSI-2 output port | ||
| 55 | |||
| 56 | properties: | ||
| 57 | endpoint: | ||
| 58 | $ref: /schemas/media/video-interfaces.yaml# | ||
| 59 | unevaluatedProperties: false | ||
| 60 | |||
| 61 | properties: | ||
| 62 | data-lanes: | ||
| 63 | minItems: 1 | ||
| 64 | maxItems: 4 | ||
| 65 | |||
| 66 | required: | ||
| 67 | - data-lanes | ||
| 68 | |||
| 69 | required: | ||
| 70 | - port@0 | ||
| 71 | - port@1 | ||
| 72 | |||
| 73 | required: | ||
| 74 | - compatible | ||
| 75 | - reg | ||
| 76 | - clocks | ||
| 77 | - power-domains | ||
| 78 | - resets | ||
| 79 | - ports | ||
| 80 | |||
| 81 | additionalProperties: false | ||
| 82 | |||
| 83 | examples: | ||
| 84 | - | | ||
| 85 | #include <dt-bindings/clock/r8a779a0-cpg-mssr.h> | ||
| 86 | #include <dt-bindings/power/r8a779a0-sysc.h> | ||
| 87 | |||
| 88 | dsi0: dsi-encoder@fed80000 { | ||
| 89 | compatible = "renesas,r8a779a0-dsi-csi2-tx"; | ||
| 90 | reg = <0xfed80000 0x10000>; | ||
| 91 | power-domains = <&sysc R8A779A0_PD_ALWAYS_ON>; | ||
| 92 | clocks = <&cpg CPG_MOD 415>, | ||
| 93 | <&cpg CPG_CORE R8A779A0_CLK_DSI>, | ||
| 94 | <&cpg CPG_CORE R8A779A0_CLK_CP>; | ||
| 95 | clock-names = "fck", "dsi", "pll"; | ||
| 96 | resets = <&cpg 415>; | ||
| 97 | |||
| 98 | ports { | ||
| 99 | #address-cells = <1>; | ||
| 100 | #size-cells = <0>; | ||
| 101 | |||
| 102 | port@0 { | ||
| 103 | reg = <0>; | ||
| 104 | dsi0_in: endpoint { | ||
| 105 | remote-endpoint = <&du_out_dsi0>; | ||
| 106 | }; | ||
| 107 | }; | ||
| 108 | |||
| 109 | port@1 { | ||
| 110 | reg = <1>; | ||
| 111 | dsi0_out: endpoint { | ||
| 112 | data-lanes = <1 2>; | ||
| 113 | remote-endpoint = <&sn65dsi86_in>; | ||
| 114 | }; | ||
| 115 | }; | ||
| 116 | }; | ||
| 117 | }; | ||
| 118 | ... | ||
| @@ -1,49 +0,0 @@ | |||
| 1 | Silicon Image SiI9234 HDMI/MHL bridge bindings | ||
| 2 | |||
| 3 | Required properties: | ||
| 4 | - compatible : "sil,sii9234". | ||
| 5 | - reg : I2C address for TPI interface, use 0x39 | ||
| 6 | - avcc33-supply : MHL/USB Switch Supply Voltage (3.3V) | ||
| 7 | - iovcc18-supply : I/O Supply Voltage (1.8V) | ||
| 8 | - avcc12-supply : TMDS Analog Supply Voltage (1.2V) | ||
| 9 | - cvcc12-supply : Digital Core Supply Voltage (1.2V) | ||
| 10 | - interrupts: interrupt specifier of INT pin | ||
| 11 | - reset-gpios: gpio specifier of RESET pin (active low) | ||
| 12 | - video interfaces: Device node can contain two video interface port | ||
| 13 | nodes for HDMI encoder and connector according to [1]. | ||
| 14 | - port@0 - MHL to HDMI | ||
| 15 | - port@1 - MHL to connector | ||
| 16 | |||
| 17 | [1]: Documentation/devicetree/bindings/media/video-interfaces.txt | ||
| 18 | |||
| 19 | |||
| 20 | Example: | ||
| 21 | sii9234@39 { | ||
| 22 | compatible = "sil,sii9234"; | ||
| 23 | reg = <0x39>; | ||
| 24 | avcc33-supply = <&vcc33mhl>; | ||
| 25 | iovcc18-supply = <&vcc18mhl>; | ||
| 26 | avcc12-supply = <&vsil12>; | ||
| 27 | cvcc12-supply = <&vsil12>; | ||
| 28 | reset-gpios = <&gpf3 4 GPIO_ACTIVE_LOW>; | ||
| 29 | interrupt-parent = <&gpf3>; | ||
| 30 | interrupts = <5 IRQ_TYPE_LEVEL_HIGH>; | ||
| 31 | |||
| 32 | ports { | ||
| 33 | #address-cells = <1>; | ||
| 34 | #size-cells = <0>; | ||
| 35 | |||
| 36 | port@0 { | ||
| 37 | reg = <0>; | ||
| 38 | mhl_to_hdmi: endpoint { | ||
| 39 | remote-endpoint = <&hdmi_to_mhl>; | ||
| 40 | }; | ||
| 41 | }; | ||
| 42 | port@1 { | ||
| 43 | reg = <1>; | ||
| 44 | mhl_to_connector: endpoint { | ||
| 45 | remote-endpoint = <&connector_to_mhl>; | ||
| 46 | }; | ||
| 47 | }; | ||
| 48 | }; | ||
| 49 | }; | ||
| @@ -0,0 +1,110 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/bridge/sil,sii9234.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Silicon Image SiI9234 HDMI/MHL bridge | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Maciej Purski <m.purski@samsung.com> | ||
| 11 | |||
| 12 | properties: | ||
| 13 | compatible: | ||
| 14 | const: sil,sii9234 | ||
| 15 | |||
| 16 | reg: | ||
| 17 | description: I2C address for TPI interface | ||
| 18 | maxItems: 1 | ||
| 19 | |||
| 20 | avcc12-supply: | ||
| 21 | description: TMDS Analog Supply Voltage, 1.2V | ||
| 22 | |||
| 23 | avcc33-supply: | ||
| 24 | description: MHL/USB Switch Supply Voltage, 3.3V | ||
| 25 | |||
| 26 | cvcc12-supply: | ||
| 27 | description: Digital Core Supply Voltage, 1.2V | ||
| 28 | |||
| 29 | iovcc18-supply: | ||
| 30 | description: I/O voltage supply, 1.8V | ||
| 31 | |||
| 32 | interrupts: | ||
| 33 | maxItems: 1 | ||
| 34 | |||
| 35 | reset-gpios: | ||
| 36 | description: GPIO connected to the reset pin. | ||
| 37 | maxItems: 1 | ||
| 38 | |||
| 39 | ports: | ||
| 40 | $ref: /schemas/graph.yaml#/properties/ports | ||
| 41 | |||
| 42 | properties: | ||
| 43 | port@0: | ||
| 44 | $ref: /schemas/graph.yaml#/properties/port | ||
| 45 | description: | ||
| 46 | Video port for HDMI (encoder) input | ||
| 47 | |||
| 48 | port@1: | ||
| 49 | $ref: /schemas/graph.yaml#/properties/port | ||
| 50 | description: | ||
| 51 | MHL to connector port | ||
| 52 | |||
| 53 | required: | ||
| 54 | - port@0 | ||
| 55 | |||
| 56 | required: | ||
| 57 | - compatible | ||
| 58 | - reg | ||
| 59 | - avcc12-supply | ||
| 60 | - avcc33-supply | ||
| 61 | - cvcc12-supply | ||
| 62 | - iovcc18-supply | ||
| 63 | - interrupts | ||
| 64 | - reset-gpios | ||
| 65 | - ports | ||
| 66 | |||
| 67 | additionalProperties: false | ||
| 68 | |||
| 69 | examples: | ||
| 70 | - | | ||
| 71 | #include <dt-bindings/gpio/gpio.h> | ||
| 72 | #include <dt-bindings/interrupt-controller/irq.h> | ||
| 73 | |||
| 74 | i2c1 { | ||
| 75 | #address-cells = <1>; | ||
| 76 | #size-cells = <0>; | ||
| 77 | |||
| 78 | bridge@39 { | ||
| 79 | compatible = "sil,sii9234"; | ||
| 80 | reg = <0x39>; | ||
| 81 | avcc12-supply = <&vsil12>; | ||
| 82 | avcc33-supply = <&vcc33mhl>; | ||
| 83 | cvcc12-supply = <&vsil12>; | ||
| 84 | iovcc18-supply = <&vcc18mhl>; | ||
| 85 | interrupt-parent = <&gpf3>; | ||
| 86 | interrupts = <5 IRQ_TYPE_LEVEL_HIGH>; | ||
| 87 | reset-gpios = <&gpf3 4 GPIO_ACTIVE_LOW>; | ||
| 88 | |||
| 89 | ports { | ||
| 90 | #address-cells = <1>; | ||
| 91 | #size-cells = <0>; | ||
| 92 | |||
| 93 | port@0 { | ||
| 94 | reg = <0>; | ||
| 95 | mhl_to_hdmi: endpoint { | ||
| 96 | remote-endpoint = <&hdmi_to_mhl>; | ||
| 97 | }; | ||
| 98 | }; | ||
| 99 | |||
| 100 | port@1 { | ||
| 101 | reg = <1>; | ||
| 102 | mhl_to_connector: endpoint { | ||
| 103 | remote-endpoint = <&connector_to_mhl>; | ||
| 104 | }; | ||
| 105 | }; | ||
| 106 | }; | ||
| 107 | }; | ||
| 108 | }; | ||
| 109 | |||
| 110 | ... | ||
| @@ -17,6 +17,8 @@ properties: · 16 unmodified lines | |||
| 17 | compatible: | 17 | compatible: |
| 18 | enum: | 18 | enum: |
| 19 | - qcom,sc7180-dp | 19 | - qcom,sc7180-dp |
| 20 | - qcom,sc7280-dp | ||
| 21 | - qcom,sc7280-edp | ||
| 20 | - qcom,sc8180x-dp | 22 | - qcom,sc8180x-dp |
| 21 | - qcom,sc8180x-edp | 23 | - qcom,sc8180x-edp |
| 22 | 24 | ||
| @@ -1,56 +0,0 @@ | |||
| 1 | Qualcomm Technologies Inc. adreno/snapdragon eDP output | ||
| 2 | |||
| 3 | Required properties: | ||
| 4 | - compatible: | ||
| 5 | * "qcom,mdss-edp" | ||
| 6 | - reg: Physical base address and length of the registers of controller and PLL | ||
| 7 | - reg-names: The names of register regions. The following regions are required: | ||
| 8 | * "edp" | ||
| 9 | * "pll_base" | ||
| 10 | - interrupts: The interrupt signal from the eDP block. | ||
| 11 | - power-domains: Should be <&mmcc MDSS_GDSC>. | ||
| 12 | - clocks: device clocks | ||
| 13 | See Documentation/devicetree/bindings/clock/clock-bindings.txt for details. | ||
| 14 | - clock-names: the following clocks are required: | ||
| 15 | * "core" | ||
| 16 | * "iface" | ||
| 17 | * "mdp_core" | ||
| 18 | * "pixel" | ||
| 19 | * "link" | ||
| 20 | - #clock-cells: The value should be 1. | ||
| 21 | - vdda-supply: phandle to vdda regulator device node | ||
| 22 | - lvl-vdd-supply: phandle to regulator device node which is used to supply power | ||
| 23 | to HPD receiving chip | ||
| 24 | - panel-en-gpios: GPIO pin to supply power to panel. | ||
| 25 | - panel-hpd-gpios: GPIO pin used for eDP hpd. | ||
| 26 | |||
| 27 | |||
| 28 | Example: | ||
| 29 | mdss_edp: qcom,mdss_edp@fd923400 { | ||
| 30 | compatible = "qcom,mdss-edp"; | ||
| 31 | reg-names = | ||
| 32 | "edp", | ||
| 33 | "pll_base"; | ||
| 34 | reg = <0xfd923400 0x700>, | ||
| 35 | <0xfd923a00 0xd4>; | ||
| 36 | interrupt-parent = <&mdss_mdp>; | ||
| 37 | interrupts = <12 0>; | ||
| 38 | power-domains = <&mmcc MDSS_GDSC>; | ||
| 39 | clock-names = | ||
| 40 | "core", | ||
| 41 | "pixel", | ||
| 42 | "iface", | ||
| 43 | "link", | ||
| 44 | "mdp_core"; | ||
| 45 | clocks = | ||
| 46 | <&mmcc MDSS_EDPAUX_CLK>, | ||
| 47 | <&mmcc MDSS_EDPPIXEL_CLK>, | ||
| 48 | <&mmcc MDSS_AHB_CLK>, | ||
| 49 | <&mmcc MDSS_EDPLINK_CLK>, | ||
| 50 | <&mmcc MDSS_MDP_CLK>; | ||
| 51 | #clock-cells = <1>; | ||
| 52 | vdda-supply = <&pma8084_l12>; | ||
| 53 | lvl-vdd-supply = <&lvl_vreg>; | ||
| 54 | panel-en-gpios = <&tlmm 137 0>; | ||
| 55 | panel-hpd-gpios = <&tlmm 103 0>; | ||
| 56 | }; | ||
| @@ -0,0 +1,81 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/panel/boe,bf060y8m-aj0.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: BOE BF060Y8M-AJ0 5.99" 1080x2160 AMOLED Panel | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - AngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org> | ||
| 11 | |||
| 12 | description: | | ||
| 13 | This is a 5.99" 1080x2160 16.7M Color active matrix AMOLED | ||
| 14 | video mode panel module on MIPI-DSI 4-Lane interface, GGRB | ||
| 15 | pixel arrangement, 63 micrometers pitch, with an active | ||
| 16 | area of 68.04 x 136.08 millimeters. | ||
| 17 | Each pixel is divided into red and green dots, or blue and | ||
| 18 | green dots, and two pixels share red or blue dots which are | ||
| 19 | arranged in vertical stripe. | ||
| 20 | The DriverIC for this panel module is SW43404. | ||
| 21 | |||
| 22 | allOf: | ||
| 23 | - $ref: panel-common.yaml# | ||
| 24 | |||
| 25 | properties: | ||
| 26 | compatible: | ||
| 27 | const: boe,bf060y8m-aj0 | ||
| 28 | |||
| 29 | elvdd-supply: | ||
| 30 | description: EL Driving positive (VDD) supply (4.40-4.80V) | ||
| 31 | elvss-supply: | ||
| 32 | description: EL Driving negative (VSS) supply (-5.00V to -1.40V) | ||
| 33 | vcc-supply: | ||
| 34 | description: Core (TSP) voltage supply (2.70-3.60V) | ||
| 35 | vci-supply: | ||
| 36 | description: DriverIC Operation supply (2.60-3.60V) | ||
| 37 | vddio-supply: | ||
| 38 | description: I/O voltage supply (1.62-1.98V) | ||
| 39 | |||
| 40 | port: true | ||
| 41 | reg: true | ||
| 42 | reset-gpios: true | ||
| 43 | |||
| 44 | required: | ||
| 45 | - compatible | ||
| 46 | - elvdd-supply | ||
| 47 | - elvss-supply | ||
| 48 | - vcc-supply | ||
| 49 | - vci-supply | ||
| 50 | - vddio-supply | ||
| 51 | - reg | ||
| 52 | - reset-gpios | ||
| 53 | |||
| 54 | additionalProperties: false | ||
| 55 | |||
| 56 | examples: | ||
| 57 | - | | ||
| 58 | #include <dt-bindings/gpio/gpio.h> | ||
| 59 | |||
| 60 | dsi { | ||
| 61 | #address-cells = <1>; | ||
| 62 | #size-cells = <0>; | ||
| 63 | panel@0 { | ||
| 64 | compatible = "boe,bf060y8m-aj0"; | ||
| 65 | reg = <0>; | ||
| 66 | |||
| 67 | reset-gpios = <&tlmm 94 GPIO_ACTIVE_HIGH>; | ||
| 68 | |||
| 69 | vcc-supply = <&disp_vcc_vreg>; | ||
| 70 | vddio-supply = <&disp_vddio_vreg>; | ||
| 71 | vci-supply = <&disp_vci_vreg>; | ||
| 72 | elvdd-supply = <&disp_elvdd_vreg>; | ||
| 73 | elvss-supply = <&disp_elvss_vreg>; | ||
| 74 | |||
| 75 | port { | ||
| 76 | panel_in: endpoint { | ||
| 77 | remote-endpoint = <&dsi0_out>; | ||
| 78 | }; | ||
| 79 | }; | ||
| 80 | }; | ||
| 81 | }; | ||
| @@ -0,0 +1,69 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/panel/ilitek,ili9163.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Ilitek ILI9163 display panels device tree bindings | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Daniel Mack <daniel@zonque.org> | ||
| 11 | |||
| 12 | description: | ||
| 13 | This binding is for display panels using an Ilitek ILI9163 controller in SPI | ||
| 14 | mode. | ||
| 15 | |||
| 16 | allOf: | ||
| 17 | - $ref: panel-common.yaml# | ||
| 18 | |||
| 19 | properties: | ||
| 20 | compatible: | ||
| 21 | items: | ||
| 22 | - enum: | ||
| 23 | - newhaven,1.8-128160EF | ||
| 24 | - const: ilitek,ili9163 | ||
| 25 | |||
| 26 | spi-max-frequency: | ||
| 27 | maximum: 32000000 | ||
| 28 | |||
| 29 | dc-gpios: | ||
| 30 | maxItems: 1 | ||
| 31 | description: Display data/command selection (D/CX) | ||
| 32 | |||
| 33 | backlight: true | ||
| 34 | reg: true | ||
| 35 | reset-gpios: true | ||
| 36 | rotation: true | ||
| 37 | |||
| 38 | required: | ||
| 39 | - compatible | ||
| 40 | - reg | ||
| 41 | - dc-gpios | ||
| 42 | - reset-gpios | ||
| 43 | |||
| 44 | additionalProperties: false | ||
| 45 | |||
| 46 | examples: | ||
| 47 | - | | ||
| 48 | #include <dt-bindings/gpio/gpio.h> | ||
| 49 | |||
| 50 | backlight: backlight { | ||
| 51 | compatible = "gpio-backlight"; | ||
| 52 | gpios = <&gpio 22 GPIO_ACTIVE_HIGH>; | ||
| 53 | }; | ||
| 54 | spi { | ||
| 55 | #address-cells = <1>; | ||
| 56 | #size-cells = <0>; | ||
| 57 | |||
| 58 | display@0 { | ||
| 59 | compatible = "newhaven,1.8-128160EF", "ilitek,ili9163"; | ||
| 60 | reg = <0>; | ||
| 61 | spi-max-frequency = <32000000>; | ||
| 62 | dc-gpios = <&gpio0 24 GPIO_ACTIVE_HIGH>; | ||
| 63 | reset-gpios = <&gpio0 25 GPIO_ACTIVE_HIGH>; | ||
| 64 | rotation = <180>; | ||
| 65 | backlight = <&backlight>; | ||
| 66 | }; | ||
| 67 | }; | ||
| 68 | |||
| 69 | ... | ||
| @@ -9,24 +9,28 @@ title: Ilitek ILI9881c based MIPI-DSI panels · 8 unmodified lines | |||
| 9 | maintainers: | 9 | maintainers: |
| 10 | - Maxime Ripard <mripard@kernel.org> | 10 | - Maxime Ripard <mripard@kernel.org> |
| 11 | 11 | ||
| 12 | allOf: | ||
| 13 | - $ref: panel-common.yaml# | ||
| 14 | |||
| 12 | properties: | 15 | properties: |
| 13 | compatible: | 16 | compatible: |
| 14 | items: | 17 | items: |
| 15 | - enum: | 18 | - enum: |
| 16 | - bananapi,lhr050h41 | 19 | - bananapi,lhr050h41 |
| 17 | - feixin,k101-im2byl02 | 20 | - feixin,k101-im2byl02 |
| 21 | - wanchanglong,w552946aba | ||
| 18 | - const: ilitek,ili9881c | 22 | - const: ilitek,ili9881c |
| 19 | 23 | ||
| 20 | backlight: true | 24 | backlight: true |
| 21 | power-supply: true | 25 | power-supply: true |
| 22 | reg: true | 26 | reg: true |
| 23 | reset-gpios: true | 27 | reset-gpios: true |
| 28 | rotation: true | ||
| 24 | 29 | ||
| 25 | required: | 30 | required: |
| 26 | - compatible | 31 | - compatible |
| 27 | - power-supply | 32 | - power-supply |
| 28 | - reg | 33 | - reg |
| 29 | - reset-gpios | ||
| 30 | 34 | ||
| 31 | additionalProperties: false | 35 | additionalProperties: false |
| 32 | 36 | ||
| @@ -0,0 +1,106 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/panel/novatek,nt35950.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Novatek NT35950-based display panels | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - AngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org> | ||
| 11 | |||
| 12 | description: | | ||
| 13 | The nt35950 IC from Novatek is a Driver IC used to drive MIPI-DSI panels, | ||
| 14 | with Static RAM for content retention in command mode and also supports | ||
| 15 | video mode with VESA Frame Buffer Compression or Display Stream Compression | ||
| 16 | on single, or dual dsi port(s). | ||
| 17 | This DDIC is also capable of upscaling an input image to the panel's native | ||
| 18 | resolution, for example it can upscale a 1920x1080 input to 3840x2160 with | ||
| 19 | either bilinear interpolation or pixel duplication. | ||
| 20 | |||
| 21 | allOf: | ||
| 22 | - $ref: panel-common.yaml# | ||
| 23 | |||
| 24 | properties: | ||
| 25 | compatible: | ||
| 26 | items: | ||
| 27 | - enum: | ||
| 28 | - sharp,ls055d1sx04 | ||
| 29 | - const: novatek,nt35950 | ||
| 30 | description: This indicates the panel manufacturer of the panel | ||
| 31 | that is in turn using the NT35950 panel driver. The compatible | ||
| 32 | string determines how the NT35950 panel driver shall be configured | ||
| 33 | to work with the indicated panel. The novatek,nt35950 compatible shall | ||
| 34 | always be provided as a fallback. | ||
| 35 | |||
| 36 | reset-gpios: | ||
| 37 | maxItems: 1 | ||
| 38 | description: phandle of gpio for reset line - This should be 8mA, gpio | ||
| 39 | can be configured using mux, pinctrl, pinctrl-names (active high) | ||
| 40 | |||
| 41 | avdd-supply: | ||
| 42 | description: positive boost supply regulator | ||
| 43 | avee-supply: | ||
| 44 | description: negative boost supply regulator | ||
| 45 | dvdd-supply: | ||
| 46 | description: regulator that supplies the digital voltage | ||
| 47 | vddio-supply: | ||
| 48 | description: regulator that supplies the I/O voltage | ||
| 49 | |||
| 50 | backlight: true | ||
| 51 | ports: true | ||
| 52 | reg: true | ||
| 53 | |||
| 54 | required: | ||
| 55 | - compatible | ||
| 56 | - reg | ||
| 57 | - reset-gpios | ||
| 58 | - avdd-supply | ||
| 59 | - avee-supply | ||
| 60 | - dvdd-supply | ||
| 61 | - vddio-supply | ||
| 62 | |||
| 63 | additionalProperties: false | ||
| 64 | |||
| 65 | examples: | ||
| 66 | - | | ||
| 67 | #include <dt-bindings/gpio/gpio.h> | ||
| 68 | |||
| 69 | dsi0 { | ||
| 70 | #address-cells = <1>; | ||
| 71 | #size-cells = <0>; | ||
| 72 | |||
| 73 | panel@0 { | ||
| 74 | compatible = "sharp,ls055d1sx04", "novatek,nt35950"; | ||
| 75 | reg = <0>; | ||
| 76 | |||
| 77 | backlight = <&pmi8998_wled>; | ||
| 78 | reset-gpios = <&tlmm 94 GPIO_ACTIVE_HIGH>; | ||
| 79 | |||
| 80 | avdd-supply = <&lab>; | ||
| 81 | avee-supply = <&ibb>; | ||
| 82 | dvdd-supply = <&disp_dvdd_vreg>; | ||
| 83 | vddio-supply = <&vreg_l14a_1p85>; | ||
| 84 | |||
| 85 | ports { | ||
| 86 | #address-cells = <1>; | ||
| 87 | #size-cells = <0>; | ||
| 88 | |||
| 89 | port@0 { | ||
| 90 | reg = <0>; | ||
| 91 | panel_in0: endpoint { | ||
| 92 | remote-endpoint = <&dsi0_out>; | ||
| 93 | }; | ||
| 94 | }; | ||
| 95 | |||
| 96 | port@1 { | ||
| 97 | reg = <1>; | ||
| 98 | panel_in1: endpoint { | ||
| 99 | remote-endpoint = <&dsi1_out>; | ||
| 100 | }; | ||
| 101 | }; | ||
| 102 | }; | ||
| 103 | }; | ||
| 104 | }; | ||
| 105 | |||
| 106 | ... | ||
| @@ -35,6 +35,8 @@ properties: · 34 unmodified lines | |||
| 35 | - boe,tv080wum-nl0 | 35 | - boe,tv080wum-nl0 |
| 36 | # Innolux P079ZCA 7.85" 768x1024 TFT LCD panel | 36 | # Innolux P079ZCA 7.85" 768x1024 TFT LCD panel |
| 37 | - innolux,p079zca | 37 | - innolux,p079zca |
| 38 | # JDI FHD_R63452 1080x1920 5.2" IPS LCD Panel | ||
| 39 | - jdi,fhd-r63452 | ||
| 38 | # Khadas TS050 5" 1080x1920 LCD panel | 40 | # Khadas TS050 5" 1080x1920 LCD panel |
| 39 | - khadas,ts050 | 41 | - khadas,ts050 |
| 40 | # Kingdisplay KD097D04 9.7" 1536x2048 TFT LCD panel | 42 | # Kingdisplay KD097D04 9.7" 1536x2048 TFT LCD panel |
| @@ -290,6 +290,8 @@ properties: · 289 unmodified lines | |||
| 290 | - starry,kr070pe2t | 290 | - starry,kr070pe2t |
| 291 | # Starry 12.2" (1920x1200 pixels) TFT LCD panel | 291 | # Starry 12.2" (1920x1200 pixels) TFT LCD panel |
| 292 | - starry,kr122ea0sra | 292 | - starry,kr122ea0sra |
| 293 | # Team Source Display Technology TST043015CMHX 4.3" WQVGA TFT LCD panel | ||
| 294 | - team-source-display,tst043015cmhx | ||
| 293 | # Tianma Micro-electronics TM070JDHG30 7.0" WXGA TFT LCD panel | 295 | # Tianma Micro-electronics TM070JDHG30 7.0" WXGA TFT LCD panel |
| 294 | - tianma,tm070jdhg30 | 296 | - tianma,tm070jdhg30 |
| 295 | # Tianma Micro-electronics TM070JVHG33 7.0" WXGA TFT LCD panel | 297 | # Tianma Micro-electronics TM070JVHG33 7.0" WXGA TFT LCD panel |
| @@ -0,0 +1,72 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/panel/sony,tulip-truly-nt35521.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Sony Tulip Truly NT35521 5.24" 1280x720 MIPI-DSI Panel | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Shawn Guo <shawn.guo@linaro.org> | ||
| 11 | |||
| 12 | description: | | ||
| 13 | The Sony Tulip Truly NT35521 is a 5.24" 1280x720 MIPI-DSI panel, which | ||
| 14 | can be found no Sony Xperia M4 phone. The panel backlight is managed | ||
| 15 | through DSI link. | ||
| 16 | |||
| 17 | allOf: | ||
| 18 | - $ref: panel-common.yaml# | ||
| 19 | |||
| 20 | properties: | ||
| 21 | compatible: | ||
| 22 | const: sony,tulip-truly-nt35521 | ||
| 23 | |||
| 24 | reg: true | ||
| 25 | |||
| 26 | positive5-supply: | ||
| 27 | description: Positive 5V supply | ||
| 28 | |||
| 29 | negative5-supply: | ||
| 30 | description: Negative 5V supply | ||
| 31 | |||
| 32 | reset-gpios: true | ||
| 33 | |||
| 34 | enable-gpios: true | ||
| 35 | |||
| 36 | port: true | ||
| 37 | |||
| 38 | required: | ||
| 39 | - compatible | ||
| 40 | - reg | ||
| 41 | - positive5-supply | ||
| 42 | - negative5-supply | ||
| 43 | - reset-gpios | ||
| 44 | - enable-gpios | ||
| 45 | - port | ||
| 46 | |||
| 47 | additionalProperties: false | ||
| 48 | |||
| 49 | examples: | ||
| 50 | - | | ||
| 51 | #include <dt-bindings/gpio/gpio.h> | ||
| 52 | |||
| 53 | dsi { | ||
| 54 | #address-cells = <1>; | ||
| 55 | #size-cells = <0>; | ||
| 56 | |||
| 57 | panel@0 { | ||
| 58 | compatible = "sony,tulip-truly-nt35521"; | ||
| 59 | reg = <0>; | ||
| 60 | positive5-supply = <&positive5_reg>; | ||
| 61 | negative5-supply = <&negative5_reg>; | ||
| 62 | reset-gpios = <&msmgpio 25 GPIO_ACTIVE_LOW>; | ||
| 63 | enable-gpios = <&msmgpio 10 GPIO_ACTIVE_HIGH>; | ||
| 64 | |||
| 65 | port { | ||
| 66 | panel_in: endpoint { | ||
| 67 | remote-endpoint = <&dsi0_out>; | ||
| 68 | }; | ||
| 69 | }; | ||
| 70 | }; | ||
| 71 | }; | ||
| 72 | ... | ||
| @@ -0,0 +1,64 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/sprd/sprd,display-subsystem.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Unisoc DRM master device | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Kevin Tang <kevin.tang@unisoc.com> | ||
| 11 | |||
| 12 | description: | | ||
| 13 | The Unisoc DRM master device is a virtual device needed to list all | ||
| 14 | DPU devices or other display interface nodes that comprise the | ||
| 15 | graphics subsystem. | ||
| 16 | |||
| 17 | Unisoc's display pipeline have several components as below description, | ||
| 18 | multi display controllers and corresponding physical interfaces. | ||
| 19 | For different display scenarios, dpu0 and dpu1 maybe binding to different | ||
| 20 | encoder. | ||
| 21 | |||
| 22 | E.g: | ||
| 23 | dpu0 and dpu1 both binding to DSI for dual mipi-dsi display; | ||
| 24 | dpu0 binding to DSI for primary display, and dpu1 binding to DP for external display; | ||
| 25 | |||
| 26 | +-----------------------------------------+ | ||
| 27 | | | | ||
| 28 | | +---------+ | | ||
| 29 | +----+ | +----+ +---------+ |DPHY/CPHY| | +------+ | ||
| 30 | | +----->+dpu0+--->+MIPI|DSI +--->+Combo +----->+Panel0| | ||
| 31 | |AXI | | +----+ +---------+ +---------+ | +------+ | ||
| 32 | | | | ^ | | ||
| 33 | | | | | | | ||
| 34 | | | | +-----------+ | | ||
| 35 | | | | | | | ||
| 36 | |APB | | +--+-+ +-----------+ +---+ | +------+ | ||
| 37 | | +----->+dpu1+--->+DisplayPort+--->+PHY+--------->+Panel1| | ||
| 38 | | | | +----+ +-----------+ +---+ | +------+ | ||
| 39 | +----+ | | | ||
| 40 | +-----------------------------------------+ | ||
| 41 | |||
| 42 | properties: | ||
| 43 | compatible: | ||
| 44 | const: sprd,display-subsystem | ||
| 45 | |||
| 46 | ports: | ||
| 47 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 48 | description: | ||
| 49 | Should contain a list of phandles pointing to display interface port | ||
| 50 | of DPU devices. | ||
| 51 | |||
| 52 | required: | ||
| 53 | - compatible | ||
| 54 | - ports | ||
| 55 | |||
| 56 | additionalProperties: false | ||
| 57 | |||
| 58 | examples: | ||
| 59 | - | | ||
| 60 | display-subsystem { | ||
| 61 | compatible = "sprd,display-subsystem"; | ||
| 62 | ports = <&dpu_out>; | ||
| 63 | }; | ||
| 64 | |||
| @@ -0,0 +1,77 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/sprd/sprd,sharkl3-dpu.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Unisoc Sharkl3 Display Processor Unit (DPU) | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Kevin Tang <kevin.tang@unisoc.com> | ||
| 11 | |||
| 12 | description: | | ||
| 13 | DPU (Display Processor Unit) is the Display Controller for the Unisoc SoCs | ||
| 14 | which transfers the image data from a video memory buffer to an internal | ||
| 15 | LCD interface. | ||
| 16 | |||
| 17 | properties: | ||
| 18 | compatible: | ||
| 19 | const: sprd,sharkl3-dpu | ||
| 20 | |||
| 21 | reg: | ||
| 22 | maxItems: 1 | ||
| 23 | |||
| 24 | interrupts: | ||
| 25 | maxItems: 1 | ||
| 26 | |||
| 27 | clocks: | ||
| 28 | minItems: 2 | ||
| 29 | |||
| 30 | clock-names: | ||
| 31 | items: | ||
| 32 | - const: clk_src_128m | ||
| 33 | - const: clk_src_384m | ||
| 34 | |||
| 35 | power-domains: | ||
| 36 | maxItems: 1 | ||
| 37 | |||
| 38 | iommus: | ||
| 39 | maxItems: 1 | ||
| 40 | |||
| 41 | port: | ||
| 42 | type: object | ||
| 43 | description: | ||
| 44 | A port node with endpoint definitions as defined in | ||
| 45 | Documentation/devicetree/bindings/media/video-interfaces.txt. | ||
| 46 | That port should be the output endpoint, usually output to | ||
| 47 | the associated DSI. | ||
| 48 | |||
| 49 | required: | ||
| 50 | - compatible | ||
| 51 | - reg | ||
| 52 | - interrupts | ||
| 53 | - clocks | ||
| 54 | - clock-names | ||
| 55 | - port | ||
| 56 | |||
| 57 | additionalProperties: false | ||
| 58 | |||
| 59 | examples: | ||
| 60 | - | | ||
| 61 | #include <dt-bindings/interrupt-controller/arm-gic.h> | ||
| 62 | #include <dt-bindings/clock/sprd,sc9860-clk.h> | ||
| 63 | dpu: dpu@63000000 { | ||
| 64 | compatible = "sprd,sharkl3-dpu"; | ||
| 65 | reg = <0x63000000 0x1000>; | ||
| 66 | interrupts = <GIC_SPI 46 IRQ_TYPE_LEVEL_HIGH>; | ||
| 67 | clock-names = "clk_src_128m", "clk_src_384m"; | ||
| 68 | |||
| 69 | clocks = <&pll CLK_TWPLL_128M>, | ||
| 70 | <&pll CLK_TWPLL_384M>; | ||
| 71 | |||
| 72 | dpu_port: port { | ||
| 73 | dpu_out: endpoint { | ||
| 74 | remote-endpoint = <&dsi_in>; | ||
| 75 | }; | ||
| 76 | }; | ||
| 77 | }; | ||
| @@ -0,0 +1,88 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/sprd/sprd,sharkl3-dsi-host.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Unisoc MIPI DSI Controller | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Kevin Tang <kevin.tang@unisoc.com> | ||
| 11 | |||
| 12 | properties: | ||
| 13 | compatible: | ||
| 14 | const: sprd,sharkl3-dsi-host | ||
| 15 | |||
| 16 | reg: | ||
| 17 | maxItems: 1 | ||
| 18 | |||
| 19 | interrupts: | ||
| 20 | maxItems: 2 | ||
| 21 | |||
| 22 | clocks: | ||
| 23 | minItems: 1 | ||
| 24 | |||
| 25 | clock-names: | ||
| 26 | items: | ||
| 27 | - const: clk_src_96m | ||
| 28 | |||
| 29 | power-domains: | ||
| 30 | maxItems: 1 | ||
| 31 | |||
| 32 | ports: | ||
| 33 | type: object | ||
| 34 | |||
| 35 | properties: | ||
| 36 | "#address-cells": | ||
| 37 | const: 1 | ||
| 38 | |||
| 39 | "#size-cells": | ||
| 40 | const: 0 | ||
| 41 | |||
| 42 | port@0: | ||
| 43 | type: object | ||
| 44 | description: | ||
| 45 | A port node with endpoint definitions as defined in | ||
| 46 | Documentation/devicetree/bindings/media/video-interfaces.txt. | ||
| 47 | That port should be the input endpoint, usually coming from | ||
| 48 | the associated DPU. | ||
| 49 | |||
| 50 | required: | ||
| 51 | - "#address-cells" | ||
| 52 | - "#size-cells" | ||
| 53 | - port@0 | ||
| 54 | |||
| 55 | additionalProperties: false | ||
| 56 | |||
| 57 | required: | ||
| 58 | - compatible | ||
| 59 | - reg | ||
| 60 | - interrupts | ||
| 61 | - clocks | ||
| 62 | - clock-names | ||
| 63 | - ports | ||
| 64 | |||
| 65 | additionalProperties: false | ||
| 66 | |||
| 67 | examples: | ||
| 68 | - | | ||
| 69 | #include <dt-bindings/interrupt-controller/arm-gic.h> | ||
| 70 | #include <dt-bindings/clock/sprd,sc9860-clk.h> | ||
| 71 | dsi: dsi@63100000 { | ||
| 72 | compatible = "sprd,sharkl3-dsi-host"; | ||
| 73 | reg = <0x63100000 0x1000>; | ||
| 74 | interrupts = <GIC_SPI 48 IRQ_TYPE_LEVEL_HIGH>, | ||
| 75 | <GIC_SPI 49 IRQ_TYPE_LEVEL_HIGH>; | ||
| 76 | clock-names = "clk_src_96m"; | ||
| 77 | clocks = <&pll CLK_TWPLL_96M>; | ||
| 78 | ports { | ||
| 79 | #address-cells = <1>; | ||
| 80 | #size-cells = <0>; | ||
| 81 | port@0 { | ||
| 82 | reg = <0>; | ||
| 83 | dsi_in: endpoint { | ||
| 84 | remote-endpoint = <&dpu_out>; | ||
| 85 | }; | ||
| 86 | }; | ||
| 87 | }; | ||
| 88 | }; | ||
| @@ -1240,6 +1240,8 @@ patternProperties: · 1239 unmodified lines | |||
| 1240 | description: Truly Semiconductors Limited | 1240 | description: Truly Semiconductors Limited |
| 1241 | "^visionox,.*": | 1241 | "^visionox,.*": |
| 1242 | description: Visionox | 1242 | description: Visionox |
| 1243 | "^team-source-display,.*": | ||
| 1244 | description: Shenzhen Team Source Display Technology Co., Ltd. (TSD) | ||
| 1243 | "^tsd,.*": | 1245 | "^tsd,.*": |
| 1244 | description: Theobroma Systems Design und Consulting GmbH | 1246 | description: Theobroma Systems Design und Consulting GmbH |
| 1245 | "^tyan,.*": | 1247 | "^tyan,.*": |
| @@ -1334,6 +1336,8 @@ patternProperties: · 88 unmodified lines | |||
| 1334 | description: Wondermedia Technologies, Inc. | 1336 | description: Wondermedia Technologies, Inc. |
| 1335 | "^wobo,.*": | 1337 | "^wobo,.*": |
| 1336 | description: Wobo | 1338 | description: Wobo |
| 1339 | "^wanchanglong,.*": | ||
| 1340 | description: Wanchanglong Electronics Technology（SHENZHEN）Co.，Ltd. | ||
| 1337 | "^x-powers,.*": | 1341 | "^x-powers,.*": |
| 1338 | description: X-Powers | 1342 | description: X-Powers |
| 1339 | "^xes,.*": | 1343 | "^xes,.*": |
| @@ -1,74 +0,0 @@ | |||
| 1 | =================================== | ||
| 2 | drm/amd/display - Display Core (DC) | ||
| 3 | =================================== | ||
| 4 | |||
| 5 | *placeholder - general description of supported platforms, what dc is, etc.* | ||
| 6 | |||
| 7 | Because it is partially shared with other operating systems, the Display Core | ||
| 8 | Driver is divided in two pieces. | ||
| 9 | |||
| 10 | 1. **Display Core (DC)** contains the OS-agnostic components. Things like | ||
| 11 | hardware programming and resource management are handled here. | ||
| 12 | 2. **Display Manager (DM)** contains the OS-dependent components. Hooks to the | ||
| 13 | amdgpu base driver and DRM are implemented here. | ||
| 14 | |||
| 15 | It doesn't help that the entire package is frequently referred to as DC. But | ||
| 16 | with the context in mind, it should be clear. | ||
| 17 | |||
| 18 | When CONFIG_DRM_AMD_DC is enabled, DC will be initialized by default for | ||
| 19 | supported ASICs. To force disable, set `amdgpu.dc=0` on kernel command line. | ||
| 20 | Likewise, to force enable on unsupported ASICs, set `amdgpu.dc=1`. | ||
| 21 | |||
| 22 | To determine if DC is loaded, search dmesg for the following entry: | ||
| 23 | |||
| 24 | ``Display Core initialized with <version number here>`` | ||
| 25 | |||
| 26 | AMDgpu Display Manager | ||
| 27 | ====================== | ||
| 28 | |||
| 29 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | ||
| 30 | :doc: overview | ||
| 31 | |||
| 32 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h | ||
| 33 | :internal: | ||
| 34 | |||
| 35 | Lifecycle | ||
| 36 | --------- | ||
| 37 | |||
| 38 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | ||
| 39 | :doc: DM Lifecycle | ||
| 40 | |||
| 41 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | ||
| 42 | :functions: dm_hw_init dm_hw_fini | ||
| 43 | |||
| 44 | Interrupts | ||
| 45 | ---------- | ||
| 46 | |||
| 47 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c | ||
| 48 | :doc: overview | ||
| 49 | |||
| 50 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c | ||
| 51 | :internal: | ||
| 52 | |||
| 53 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | ||
| 54 | :functions: register_hpd_handlers dm_crtc_high_irq dm_pflip_high_irq | ||
| 55 | |||
| 56 | Atomic Implementation | ||
| 57 | --------------------- | ||
| 58 | |||
| 59 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | ||
| 60 | :doc: atomic | ||
| 61 | |||
| 62 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | ||
| 63 | :functions: amdgpu_dm_atomic_check amdgpu_dm_atomic_commit_tail | ||
| 64 | |||
| 65 | Display Core | ||
| 66 | ============ | ||
| 67 | |||
| 68 | **WIP** | ||
| 69 | |||
| 70 | FreeSync Video | ||
| 71 | -------------- | ||
| 72 | |||
| 73 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | ||
| 74 | :doc: FreeSync Video | ||
| @@ -1,324 +0,0 @@ | |||
| 1 | ========================= | ||
| 2 | drm/amdgpu AMDgpu driver | ||
| 3 | ========================= | ||
| 4 | |||
| 5 | The drm/amdgpu driver supports all AMD Radeon GPUs based on the Graphics Core | ||
| 6 | Next (GCN) architecture. | ||
| 7 | |||
| 8 | Module Parameters | ||
| 9 | ================= | ||
| 10 | |||
| 11 | The amdgpu driver supports the following module parameters: | ||
| 12 | |||
| 13 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | ||
| 14 | |||
| 15 | Core Driver Infrastructure | ||
| 16 | ========================== | ||
| 17 | |||
| 18 | This section covers core driver infrastructure. | ||
| 19 | |||
| 20 | .. _amdgpu_memory_domains: | ||
| 21 | |||
| 22 | Memory Domains | ||
| 23 | -------------- | ||
| 24 | |||
| 25 | .. kernel-doc:: include/uapi/drm/amdgpu_drm.h | ||
| 26 | :doc: memory domains | ||
| 27 | |||
| 28 | Buffer Objects | ||
| 29 | -------------- | ||
| 30 | |||
| 31 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | ||
| 32 | :doc: amdgpu_object | ||
| 33 | |||
| 34 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | ||
| 35 | :internal: | ||
| 36 | |||
| 37 | PRIME Buffer Sharing | ||
| 38 | -------------------- | ||
| 39 | |||
| 40 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | ||
| 41 | :doc: PRIME Buffer Sharing | ||
| 42 | |||
| 43 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | ||
| 44 | :internal: | ||
| 45 | |||
| 46 | MMU Notifier | ||
| 47 | ------------ | ||
| 48 | |||
| 49 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | ||
| 50 | :doc: MMU Notifier | ||
| 51 | |||
| 52 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | ||
| 53 | :internal: | ||
| 54 | |||
| 55 | AMDGPU Virtual Memory | ||
| 56 | --------------------- | ||
| 57 | |||
| 58 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | ||
| 59 | :doc: GPUVM | ||
| 60 | |||
| 61 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | ||
| 62 | :internal: | ||
| 63 | |||
| 64 | Interrupt Handling | ||
| 65 | ------------------ | ||
| 66 | |||
| 67 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | ||
| 68 | :doc: Interrupt Handling | ||
| 69 | |||
| 70 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | ||
| 71 | :internal: | ||
| 72 | |||
| 73 | IP Blocks | ||
| 74 | ------------------ | ||
| 75 | |||
| 76 | .. kernel-doc:: drivers/gpu/drm/amd/include/amd_shared.h | ||
| 77 | :doc: IP Blocks | ||
| 78 | |||
| 79 | .. kernel-doc:: drivers/gpu/drm/amd/include/amd_shared.h | ||
| 80 | :identifiers: amd_ip_block_type amd_ip_funcs | ||
| 81 | |||
| 82 | AMDGPU XGMI Support | ||
| 83 | =================== | ||
| 84 | |||
| 85 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c | ||
| 86 | |||
| 87 | AMDGPU RAS Support | ||
| 88 | ================== | ||
| 89 | |||
| 90 | The AMDGPU RAS interfaces are exposed via sysfs (for informational queries) and | ||
| 91 | debugfs (for error injection). | ||
| 92 | |||
| 93 | RAS debugfs/sysfs Control and Error Injection Interfaces | ||
| 94 | -------------------------------------------------------- | ||
| 95 | |||
| 96 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | ||
| 97 | :doc: AMDGPU RAS debugfs control interface | ||
| 98 | |||
| 99 | RAS Reboot Behavior for Unrecoverable Errors | ||
| 100 | -------------------------------------------------------- | ||
| 101 | |||
| 102 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | ||
| 103 | :doc: AMDGPU RAS Reboot Behavior for Unrecoverable Errors | ||
| 104 | |||
| 105 | RAS Error Count sysfs Interface | ||
| 106 | ------------------------------- | ||
| 107 | |||
| 108 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | ||
| 109 | :doc: AMDGPU RAS sysfs Error Count Interface | ||
| 110 | |||
| 111 | RAS EEPROM debugfs Interface | ||
| 112 | ---------------------------- | ||
| 113 | |||
| 114 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | ||
| 115 | :doc: AMDGPU RAS debugfs EEPROM table reset interface | ||
| 116 | |||
| 117 | RAS VRAM Bad Pages sysfs Interface | ||
| 118 | ---------------------------------- | ||
| 119 | |||
| 120 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | ||
| 121 | :doc: AMDGPU RAS sysfs gpu_vram_bad_pages Interface | ||
| 122 | |||
| 123 | Sample Code | ||
| 124 | ----------- | ||
| 125 | Sample code for testing error injection can be found here: | ||
| 126 | https://cgit.freedesktop.org/mesa/drm/tree/tests/amdgpu/ras_tests.c | ||
| 127 | |||
| 128 | This is part of the libdrm amdgpu unit tests which cover several areas of the GPU. | ||
| 129 | There are four sets of tests: | ||
| 130 | |||
| 131 | RAS Basic Test | ||
| 132 | |||
| 133 | The test verifies the RAS feature enabled status and makes sure the necessary sysfs and debugfs files | ||
| 134 | are present. | ||
| 135 | |||
| 136 | RAS Query Test | ||
| 137 | |||
| 138 | This test checks the RAS availability and enablement status for each supported IP block as well as | ||
| 139 | the error counts. | ||
| 140 | |||
| 141 | RAS Inject Test | ||
| 142 | |||
| 143 | This test injects errors for each IP. | ||
| 144 | |||
| 145 | RAS Disable Test | ||
| 146 | |||
| 147 | This test tests disabling of RAS features for each IP block. | ||
| 148 | |||
| 149 | |||
| 150 | GPU Power/Thermal Controls and Monitoring | ||
| 151 | ========================================= | ||
| 152 | |||
| 153 | This section covers hwmon and power/thermal controls. | ||
| 154 | |||
| 155 | HWMON Interfaces | ||
| 156 | ---------------- | ||
| 157 | |||
| 158 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 159 | :doc: hwmon | ||
| 160 | |||
| 161 | GPU sysfs Power State Interfaces | ||
| 162 | -------------------------------- | ||
| 163 | |||
| 164 | GPU power controls are exposed via sysfs files. | ||
| 165 | |||
| 166 | power_dpm_state | ||
| 167 | ~~~~~~~~~~~~~~~ | ||
| 168 | |||
| 169 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 170 | :doc: power_dpm_state | ||
| 171 | |||
| 172 | power_dpm_force_performance_level | ||
| 173 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
| 174 | |||
| 175 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 176 | :doc: power_dpm_force_performance_level | ||
| 177 | |||
| 178 | pp_table | ||
| 179 | ~~~~~~~~ | ||
| 180 | |||
| 181 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 182 | :doc: pp_table | ||
| 183 | |||
| 184 | pp_od_clk_voltage | ||
| 185 | ~~~~~~~~~~~~~~~~~ | ||
| 186 | |||
| 187 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 188 | :doc: pp_od_clk_voltage | ||
| 189 | |||
| 190 | pp_dpm_* | ||
| 191 | ~~~~~~~~ | ||
| 192 | |||
| 193 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 194 | :doc: pp_dpm_sclk pp_dpm_mclk pp_dpm_socclk pp_dpm_fclk pp_dpm_dcefclk pp_dpm_pcie | ||
| 195 | |||
| 196 | pp_power_profile_mode | ||
| 197 | ~~~~~~~~~~~~~~~~~~~~~ | ||
| 198 | |||
| 199 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 200 | :doc: pp_power_profile_mode | ||
| 201 | |||
| 202 | \*_busy_percent | ||
| 203 | ~~~~~~~~~~~~~~~ | ||
| 204 | |||
| 205 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 206 | :doc: gpu_busy_percent | ||
| 207 | |||
| 208 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 209 | :doc: mem_busy_percent | ||
| 210 | |||
| 211 | gpu_metrics | ||
| 212 | ~~~~~~~~~~~~~~~~~~~~~ | ||
| 213 | |||
| 214 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 215 | :doc: gpu_metrics | ||
| 216 | |||
| 217 | GPU Product Information | ||
| 218 | ======================= | ||
| 219 | |||
| 220 | Information about the GPU can be obtained on certain cards | ||
| 221 | via sysfs | ||
| 222 | |||
| 223 | product_name | ||
| 224 | ------------ | ||
| 225 | |||
| 226 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | ||
| 227 | :doc: product_name | ||
| 228 | |||
| 229 | product_number | ||
| 230 | -------------- | ||
| 231 | |||
| 232 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | ||
| 233 | :doc: product_name | ||
| 234 | |||
| 235 | serial_number | ||
| 236 | ------------- | ||
| 237 | |||
| 238 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | ||
| 239 | :doc: serial_number | ||
| 240 | |||
| 241 | unique_id | ||
| 242 | --------- | ||
| 243 | |||
| 244 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 245 | :doc: unique_id | ||
| 246 | |||
| 247 | GPU Memory Usage Information | ||
| 248 | ============================ | ||
| 249 | |||
| 250 | Various memory accounting can be accessed via sysfs | ||
| 251 | |||
| 252 | mem_info_vram_total | ||
| 253 | ------------------- | ||
| 254 | |||
| 255 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | ||
| 256 | :doc: mem_info_vram_total | ||
| 257 | |||
| 258 | mem_info_vram_used | ||
| 259 | ------------------ | ||
| 260 | |||
| 261 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | ||
| 262 | :doc: mem_info_vram_used | ||
| 263 | |||
| 264 | mem_info_vis_vram_total | ||
| 265 | ----------------------- | ||
| 266 | |||
| 267 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | ||
| 268 | :doc: mem_info_vis_vram_total | ||
| 269 | |||
| 270 | mem_info_vis_vram_used | ||
| 271 | ---------------------- | ||
| 272 | |||
| 273 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | ||
| 274 | :doc: mem_info_vis_vram_used | ||
| 275 | |||
| 276 | mem_info_gtt_total | ||
| 277 | ------------------ | ||
| 278 | |||
| 279 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | ||
| 280 | :doc: mem_info_gtt_total | ||
| 281 | |||
| 282 | mem_info_gtt_used | ||
| 283 | ----------------- | ||
| 284 | |||
| 285 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | ||
| 286 | :doc: mem_info_gtt_used | ||
| 287 | |||
| 288 | PCIe Accounting Information | ||
| 289 | =========================== | ||
| 290 | |||
| 291 | pcie_bw | ||
| 292 | ------- | ||
| 293 | |||
| 294 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 295 | :doc: pcie_bw | ||
| 296 | |||
| 297 | pcie_replay_count | ||
| 298 | ----------------- | ||
| 299 | |||
| 300 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | ||
| 301 | :doc: pcie_replay_count | ||
| 302 | |||
| 303 | GPU SmartShift Information | ||
| 304 | ========================== | ||
| 305 | |||
| 306 | GPU SmartShift information via sysfs | ||
| 307 | |||
| 308 | smartshift_apu_power | ||
| 309 | -------------------- | ||
| 310 | |||
| 311 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 312 | :doc: smartshift_apu_power | ||
| 313 | |||
| 314 | smartshift_dgpu_power | ||
| 315 | --------------------- | ||
| 316 | |||
| 317 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 318 | :doc: smartshift_dgpu_power | ||
| 319 | |||
| 320 | smartshift_bias | ||
| 321 | --------------- | ||
| 322 | |||
| 323 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 324 | :doc: smartshift_bias | ||
| @@ -0,0 +1,87 @@ | |||
| 1 | =============== | ||
| 2 | AMDGPU Glossary | ||
| 3 | =============== | ||
| 4 | |||
| 5 | Here you can find some generic acronyms used in the amdgpu driver. Notice that | ||
| 6 | we have a dedicated glossary for Display Core at | ||
| 7 | 'Documentation/gpu/amdgpu/display/dc-glossary.rst'. | ||
| 8 | |||
| 9 | .. glossary:: | ||
| 10 | |||
| 11 | CP | ||
| 12 | Command Processor | ||
| 13 | |||
| 14 | CPLIB | ||
| 15 | Content Protection Library | ||
| 16 | |||
| 17 | DFS | ||
| 18 | Digital Frequency Synthesizer | ||
| 19 | |||
| 20 | ECP | ||
| 21 | Enhanced Content Protection | ||
| 22 | |||
| 23 | EOP | ||
| 24 | End Of Pipe/Pipeline | ||
| 25 | |||
| 26 | GC | ||
| 27 | Graphics and Compute | ||
| 28 | |||
| 29 | GMC | ||
| 30 | Graphic Memory Controller | ||
| 31 | |||
| 32 | IH | ||
| 33 | Interrupt Handler | ||
| 34 | |||
| 35 | HQD | ||
| 36 | Hardware Queue Descriptor | ||
| 37 | |||
| 38 | IB | ||
| 39 | Indirect Buffer | ||
| 40 | |||
| 41 | IP | ||
| 42 | Intellectual Property blocks | ||
| 43 | |||
| 44 | KCQ | ||
| 45 | Kernel Compute Queue | ||
| 46 | |||
| 47 | KGQ | ||
| 48 | Kernel Graphics Queue | ||
| 49 | |||
| 50 | KIQ | ||
| 51 | Kernel Interface Queue | ||
| 52 | |||
| 53 | MEC | ||
| 54 | MicroEngine Compute | ||
| 55 | |||
| 56 | MES | ||
| 57 | MicroEngine Scheduler | ||
| 58 | |||
| 59 | MMHUB | ||
| 60 | Multi-Media HUB | ||
| 61 | |||
| 62 | MQD | ||
| 63 | Memory Queue Descriptor | ||
| 64 | |||
| 65 | PPLib | ||
| 66 | PowerPlay Library - PowerPlay is the power management component. | ||
| 67 | |||
| 68 | PSP | ||
| 69 | Platform Security Processor | ||
| 70 | |||
| 71 | RCL | ||
| 72 | RunList Controller | ||
| 73 | |||
| 74 | SDMA | ||
| 75 | System DMA | ||
| 76 | |||
| 77 | SMU | ||
| 78 | System Management Unit | ||
| 79 | |||
| 80 | SS | ||
| 81 | Spread Spectrum | ||
| 82 | |||
| 83 | VCE | ||
| 84 | Video Compression Engine | ||
| 85 | |||
| 86 | VCN | ||
| 87 | Video Codec Next | ||
| @@ -0,0 +1,414 @@ | |||
| 1 | <?xml version="1.0" encoding="UTF-8" standalone="no"?> | ||
| 2 | <!-- Created with Inkscape (http://www.inkscape.org/) --> | ||
| 3 | |||
| 4 | <svg | ||
| 5 | xmlns:dc="http://purl.org/dc/elements/1.1/" | ||
| 6 | xmlns:cc="http://creativecommons.org/ns#" | ||
| 7 | xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" | ||
| 8 | xmlns:svg="http://www.w3.org/2000/svg" | ||
| 9 | xmlns="http://www.w3.org/2000/svg" | ||
| 10 | xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" | ||
| 11 | xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" | ||
| 12 | width="144.63406mm" | ||
| 13 | height="66.596054mm" | ||
| 14 | viewBox="0 0 144.15195 66.596054" | ||
| 15 | version="1.1" | ||
| 16 | id="svg8" | ||
| 17 | inkscape:version="0.92.5 (2060ec1f9f, 2020-04-08)" | ||
| 18 | sodipodi:docname="config_example.svg"> | ||
| 19 | <defs | ||
| 20 | id="defs2"> | ||
| 21 | <marker | ||
| 22 | inkscape:stockid="Arrow1Mend" | ||
| 23 | orient="auto" | ||
| 24 | refY="0" | ||
| 25 | refX="0" | ||
| 26 | id="Arrow1Mend" | ||
| 27 | style="overflow:visible" | ||
| 28 | inkscape:isstock="true"> | ||
| 29 | <path | ||
| 30 | id="path4547" | ||
| 31 | d="M 0,0 5,-5 -12.5,0 5,5 Z" | ||
| 32 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1" | ||
| 33 | transform="matrix(-0.4,0,0,-0.4,-4,0)" | ||
| 34 | inkscape:connector-curvature="0" /> | ||
| 35 | </marker> | ||
| 36 | <marker | ||
| 37 | inkscape:stockid="Arrow1Mend" | ||
| 38 | orient="auto" | ||
| 39 | refY="0" | ||
| 40 | refX="0" | ||
| 41 | id="Arrow1Mend-3" | ||
| 42 | style="overflow:visible" | ||
| 43 | inkscape:isstock="true"> | ||
| 44 | <path | ||
| 45 | inkscape:connector-curvature="0" | ||
| 46 | id="path4547-6" | ||
| 47 | d="M 0,0 5,-5 -12.5,0 5,5 Z" | ||
| 48 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1" | ||
| 49 | transform="matrix(-0.4,0,0,-0.4,-4,0)" /> | ||
| 50 | </marker> | ||
| 51 | <marker | ||
| 52 | inkscape:stockid="Arrow1Mend" | ||
| 53 | orient="auto" | ||
| 54 | refY="0" | ||
| 55 | refX="0" | ||
| 56 | id="Arrow1Mend-3-5" | ||
| 57 | style="overflow:visible" | ||
| 58 | inkscape:isstock="true"> | ||
| 59 | <path | ||
| 60 | inkscape:connector-curvature="0" | ||
| 61 | id="path4547-6-3" | ||
| 62 | d="M 0,0 5,-5 -12.5,0 5,5 Z" | ||
| 63 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1" | ||
| 64 | transform="matrix(-0.4,0,0,-0.4,-4,0)" /> | ||
| 65 | </marker> | ||
| 66 | <marker | ||
| 67 | inkscape:stockid="Arrow1Mend" | ||
| 68 | orient="auto" | ||
| 69 | refY="0" | ||
| 70 | refX="0" | ||
| 71 | id="Arrow1Mend-3-5-0" | ||
| 72 | style="overflow:visible" | ||
| 73 | inkscape:isstock="true"> | ||
| 74 | <path | ||
| 75 | inkscape:connector-curvature="0" | ||
| 76 | id="path4547-6-3-6" | ||
| 77 | d="M 0,0 5,-5 -12.5,0 5,5 Z" | ||
| 78 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1" | ||
| 79 | transform="matrix(-0.4,0,0,-0.4,-4,0)" /> | ||
| 80 | </marker> | ||
| 81 | <marker | ||
| 82 | inkscape:stockid="Arrow1Mend" | ||
| 83 | orient="auto" | ||
| 84 | refY="0" | ||
| 85 | refX="0" | ||
| 86 | id="Arrow1Mend-3-5-7" | ||
| 87 | style="overflow:visible" | ||
| 88 | inkscape:isstock="true"> | ||
| 89 | <path | ||
| 90 | inkscape:connector-curvature="0" | ||
| 91 | id="path4547-6-3-3" | ||
| 92 | d="M 0,0 5,-5 -12.5,0 5,5 Z" | ||
| 93 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1" | ||
| 94 | transform="matrix(-0.4,0,0,-0.4,-4,0)" /> | ||
| 95 | </marker> | ||
| 96 | </defs> | ||
| 97 | <sodipodi:namedview | ||
| 98 | id="base" | ||
| 99 | pagecolor="#ffffff" | ||
| 100 | bordercolor="#666666" | ||
| 101 | borderopacity="1.0" | ||
| 102 | inkscape:pageopacity="0.0" | ||
| 103 | inkscape:pageshadow="2" | ||
| 104 | inkscape:zoom="0.98994949" | ||
| 105 | inkscape:cx="518.91791" | ||
| 106 | inkscape:cy="172.50112" | ||
| 107 | inkscape:document-units="mm" | ||
| 108 | inkscape:current-layer="layer1" | ||
| 109 | showgrid="true" | ||
| 110 | viewbox-width="209.3" | ||
| 111 | inkscape:window-width="3840" | ||
| 112 | inkscape:window-height="1136" | ||
| 113 | inkscape:window-x="0" | ||
| 114 | inkscape:window-y="27" | ||
| 115 | inkscape:window-maximized="1" | ||
| 116 | fit-margin-top="0" | ||
| 117 | fit-margin-left="0" | ||
| 118 | fit-margin-right="0" | ||
| 119 | fit-margin-bottom="0"> | ||
| 120 | <inkscape:grid | ||
| 121 | type="xygrid" | ||
| 122 | id="grid817" | ||
| 123 | originx="4.390216" | ||
| 124 | originy="-208.88856" /> | ||
| 125 | </sodipodi:namedview> | ||
| 126 | <metadata | ||
| 127 | id="metadata5"> | ||
| 128 | <rdf:RDF> | ||
| 129 | <cc:Work | ||
| 130 | rdf:about=""> | ||
| 131 | <dc:format>image/svg+xml</dc:format> | ||
| 132 | <dc:type | ||
| 133 | rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> | ||
| 134 | <dc:title></dc:title> | ||
| 135 | </cc:Work> | ||
| 136 | </rdf:RDF> | ||
| 137 | </metadata> | ||
| 138 | <g | ||
| 139 | inkscape:label="Layer 1" | ||
| 140 | inkscape:groupmode="layer" | ||
| 141 | id="layer1" | ||
| 142 | transform="translate(4.4048992,-21.515392)"> | ||
| 143 | <path | ||
| 144 | style="fill:none;stroke:#000000;stroke-width:0.26458335px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" | ||
| 145 | d="m 20.816662,35.062492 h 23.8125 v -5.291667 h 5.291667 v 5.291667 h 10.583334 v -5.291667 h 5.291667 v 5.291667 h 2.645833 v -5.291667 h 5.291667 v 5.291667 h 66.14583" | ||
| 146 | id="path4522" | ||
| 147 | inkscape:connector-curvature="0" /> | ||
| 148 | <path | ||
| 149 | style="fill:none;stroke:#000000;stroke-width:0.26458335px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" | ||
| 150 | d="m 20.816662,48.291659 h 7.9375 v -5.291667 h 5.291667 v 5.291667 h 58.208335 v -5.291667 h 5.291666 v 5.291667 h 42.33333" | ||
| 151 | id="path4524" | ||
| 152 | inkscape:connector-curvature="0" /> | ||
| 153 | <path | ||
| 154 | style="fill:none;stroke:#000000;stroke-width:0.26458335px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" | ||
| 155 | d="m 20.816662,61.520826 h 26.458334 v -5.291667 h 44.979168 v 5.291667 h 47.624996" | ||
| 156 | id="path4526" | ||
| 157 | inkscape:connector-curvature="0" /> | ||
| 158 | <path | ||
| 159 | style="fill:none;stroke:#000000;stroke-width:0.26458335px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" | ||
| 160 | d="M 20.816662,72.104159 H 139.87916" | ||
| 161 | id="path4528" | ||
| 162 | inkscape:connector-curvature="0" /> | ||
| 163 | <path | ||
| 164 | style="fill:none;stroke:#000000;stroke-width:0.26458335px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" | ||
| 165 | d="M 20.816662,77.395826 H 139.87916" | ||
| 166 | id="path4530" | ||
| 167 | inkscape:connector-curvature="0" /> | ||
| 168 | <path | ||
| 169 | style="fill:none;stroke:#000000;stroke-width:0.26458335px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" | ||
| 170 | d="M 20.816662,82.687493 H 139.87916" | ||
| 171 | id="path4532" | ||
| 172 | inkscape:connector-curvature="0" /> | ||
| 173 | <path | ||
| 174 | style="fill:none;stroke:#000000;stroke-width:0.26458335px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" | ||
| 175 | d="M 20.816662,87.97916 H 139.87916" | ||
| 176 | id="path4534" | ||
| 177 | inkscape:connector-curvature="0" /> | ||
| 178 | <path | ||
| 179 | style="fill:none;stroke:#ff0000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.52916668, 0.52916668;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend)" | ||
| 180 | d="m 47.274996,29.770826 c 3.836215,14.933158 3.472151,27.586643 0.264583,41.010418" | ||
| 181 | id="path4536" | ||
| 182 | inkscape:connector-curvature="0" | ||
| 183 | sodipodi:nodetypes="cc" /> | ||
| 184 | <path | ||
| 185 | style="fill:none;stroke:#ff0000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.52916669, 0.52916669;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend-3)" | ||
| 186 | d="m 63.149996,29.770826 c 3.836214,14.933158 5.059652,27.586642 1.852084,41.010418" | ||
| 187 | id="path4536-7" | ||
| 188 | inkscape:connector-curvature="0" | ||
| 189 | sodipodi:nodetypes="cc" /> | ||
| 190 | <path | ||
| 191 | style="fill:none;stroke:#ff0000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.5291667, 0.5291667;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend-3-5)" | ||
| 192 | d="m 71.087496,29.770825 c 3.836214,14.933158 5.059652,27.586643 1.852084,41.010419" | ||
| 193 | id="path4536-7-5" | ||
| 194 | inkscape:connector-curvature="0" | ||
| 195 | sodipodi:nodetypes="cc" /> | ||
| 196 | <text | ||
| 197 | xml:space="preserve" | ||
| 198 | style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 199 | x="59.359009" | ||
| 200 | y="24.195677" | ||
| 201 | id="text6572"><tspan | ||
| 202 | sodipodi:role="line" | ||
| 203 | x="59.359009" | ||
| 204 | y="24.195677" | ||
| 205 | style="font-size:3.52777791px;line-height:5.39999962;text-align:center;text-anchor:middle;stroke-width:0.26458335" | ||
| 206 | id="tspan6574">Configurations</tspan></text> | ||
| 207 | <text | ||
| 208 | xml:space="preserve" | ||
| 209 | style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 210 | x="46.825508" | ||
| 211 | y="28.542402" | ||
| 212 | id="text6572-6"><tspan | ||
| 213 | sodipodi:role="line" | ||
| 214 | x="46.825508" | ||
| 215 | y="28.542402" | ||
| 216 | style="font-size:3.52777815px;line-height:5.39999962;text-align:center;text-anchor:middle;fill:#008000;stroke-width:0.26458335" | ||
| 217 | id="tspan6574-2">A</tspan></text> | ||
| 218 | <text | ||
| 219 | xml:space="preserve" | ||
| 220 | style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 221 | x="62.8895" | ||
| 222 | y="28.825886" | ||
| 223 | id="text6572-6-2"><tspan | ||
| 224 | sodipodi:role="line" | ||
| 225 | x="62.8895" | ||
| 226 | y="28.825886" | ||
| 227 | style="font-size:3.52777839px;line-height:5.39999962;text-align:center;text-anchor:middle;fill:#0000ff;stroke-width:0.26458335" | ||
| 228 | id="tspan6574-2-7">B</tspan></text> | ||
| 229 | <text | ||
| 230 | xml:space="preserve" | ||
| 231 | style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 232 | x="70.827003" | ||
| 233 | y="29.109362" | ||
| 234 | id="text6572-6-2-3"><tspan | ||
| 235 | sodipodi:role="line" | ||
| 236 | x="70.827003" | ||
| 237 | y="29.109362" | ||
| 238 | style="font-size:3.52777863px;line-height:5.39999962;text-align:center;text-anchor:middle;fill:#c87137;stroke-width:0.26458335" | ||
| 239 | id="tspan6574-2-7-6">C</tspan></text> | ||
| 240 | <path | ||
| 241 | style="fill:none;stroke:#ff0000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.52916671, 0.52916671;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend-3-5-0)" | ||
| 242 | d="m 92.254164,42.999993 c 9.142136,12.745655 4.411987,28.608461 0.529167,38.364584" | ||
| 243 | id="path4536-7-5-2" | ||
| 244 | inkscape:connector-curvature="0" | ||
| 245 | sodipodi:nodetypes="cc" /> | ||
| 246 | <path | ||
| 247 | style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" | ||
| 248 | d="m 47.274996,72.104159 v 5.291667" | ||
| 249 | id="path8053" | ||
| 250 | inkscape:connector-curvature="0" /> | ||
| 251 | <path | ||
| 252 | style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" | ||
| 253 | d="m 64.472913,72.10416 v 5.291667" | ||
| 254 | id="path8053-6" | ||
| 255 | inkscape:connector-curvature="0" /> | ||
| 256 | <path | ||
| 257 | style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" | ||
| 258 | d="m 72.410413,72.10416 v 5.291667" | ||
| 259 | id="path8053-6-1" | ||
| 260 | inkscape:connector-curvature="0" /> | ||
| 261 | <path | ||
| 262 | style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" | ||
| 263 | d="m 92.254164,82.687494 v 5.291667" | ||
| 264 | id="path8053-6-1-8" | ||
| 265 | inkscape:connector-curvature="0" /> | ||
| 266 | <text | ||
| 267 | xml:space="preserve" | ||
| 268 | style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 269 | x="55.802444" | ||
| 270 | y="76.167412" | ||
| 271 | id="text6572-6-7"><tspan | ||
| 272 | sodipodi:role="line" | ||
| 273 | x="55.802444" | ||
| 274 | y="76.167412" | ||
| 275 | style="font-size:3.52777839px;line-height:5.39999962;text-align:center;text-anchor:middle;fill:#008000;stroke-width:0.26458335" | ||
| 276 | id="tspan6574-2-9">A</tspan></text> | ||
| 277 | <text | ||
| 278 | xml:space="preserve" | ||
| 279 | style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 280 | x="68.559143" | ||
| 281 | y="75.883926" | ||
| 282 | id="text6572-6-2-2"><tspan | ||
| 283 | sodipodi:role="line" | ||
| 284 | x="68.559143" | ||
| 285 | y="75.883926" | ||
| 286 | style="font-size:3.52777863px;line-height:5.39999962;text-align:center;text-anchor:middle;fill:#0000ff;stroke-width:0.26458335" | ||
| 287 | id="tspan6574-2-7-0">B</tspan></text> | ||
| 288 | <text | ||
| 289 | xml:space="preserve" | ||
| 290 | style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 291 | x="84.812119" | ||
| 292 | y="75.883911" | ||
| 293 | id="text6572-6-2-3-2"><tspan | ||
| 294 | sodipodi:role="line" | ||
| 295 | x="84.812119" | ||
| 296 | y="75.883911" | ||
| 297 | style="font-size:3.52777863px;line-height:5.39999962;text-align:center;text-anchor:middle;fill:#c87137;stroke-width:0.26458335" | ||
| 298 | id="tspan6574-2-7-6-3">C</tspan></text> | ||
| 299 | <text | ||
| 300 | xml:space="preserve" | ||
| 301 | style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 302 | x="98.513756" | ||
| 303 | y="86.845222" | ||
| 304 | id="text6572-6-2-3-2-7"><tspan | ||
| 305 | sodipodi:role="line" | ||
| 306 | x="98.513756" | ||
| 307 | y="86.845222" | ||
| 308 | style="font-size:3.52777863px;line-height:5.39999962;text-align:center;text-anchor:middle;fill:#c87137;stroke-width:0.26458335" | ||
| 309 | id="tspan6574-2-7-6-3-5">C</tspan></text> | ||
| 310 | <text | ||
| 311 | xml:space="preserve" | ||
| 312 | style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 313 | x="35.452015" | ||
| 314 | y="75.694931" | ||
| 315 | id="text6572-9"><tspan | ||
| 316 | sodipodi:role="line" | ||
| 317 | x="35.452015" | ||
| 318 | y="75.694931" | ||
| 319 | style="font-size:3.52777815px;line-height:5.39999962;text-align:center;text-anchor:middle;stroke-width:0.26458335" | ||
| 320 | id="tspan6574-22">Old config</tspan></text> | ||
| 321 | <text | ||
| 322 | xml:space="preserve" | ||
| 323 | style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 324 | x="55.484753" | ||
| 325 | y="86.656235" | ||
| 326 | id="text6572-9-8"><tspan | ||
| 327 | sodipodi:role="line" | ||
| 328 | x="55.484753" | ||
| 329 | y="86.656235" | ||
| 330 | style="font-size:3.52777839px;line-height:5.39999962;text-align:center;text-anchor:middle;stroke-width:0.26458335" | ||
| 331 | id="tspan6574-22-9">Old config</tspan></text> | ||
| 332 | <path | ||
| 333 | style="fill:none;stroke:#ff0000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.52916671, 0.52916671;stroke-dashoffset:0;stroke-opacity:1;marker-end:url(#Arrow1Mend-3-5-7)" | ||
| 334 | d="m 92.254164,42.999993 c 4.233333,4.7625 2.645833,13.229167 0.79375,17.197917" | ||
| 335 | inkscape:connector-curvature="0" | ||
| 336 | sodipodi:nodetypes="cc" /> | ||
| 337 | <text | ||
| 338 | xml:space="preserve" | ||
| 339 | style="font-style:normal;font-weight:normal;font-size:10.58333397px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 340 | x="3.7020128" | ||
| 341 | y="33.550579" | ||
| 342 | id="text6572-1"><tspan | ||
| 343 | sodipodi:role="line" | ||
| 344 | x="3.7020128" | ||
| 345 | y="42.914349" | ||
| 346 | style="font-size:3.52777815px;line-height:5.39999962;text-align:center;text-anchor:middle;stroke-width:0.26458335" | ||
| 347 | id="tspan15310" /></text> | ||
| 348 | <text | ||
| 349 | xml:space="preserve" | ||
| 350 | style="font-style:normal;font-weight:normal;font-size:3.17500019px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 351 | x="13.366468" | ||
| 352 | y="46.590767" | ||
| 353 | id="text15316"><tspan | ||
| 354 | sodipodi:role="line" | ||
| 355 | x="13.366468" | ||
| 356 | y="46.590767" | ||
| 357 | style="text-align:center;text-anchor:middle;stroke-width:0.26458335" | ||
| 358 | id="tspan15318">VUpdate</tspan></text> | ||
| 359 | <text | ||
| 360 | xml:space="preserve" | ||
| 361 | style="font-style:normal;font-weight:normal;font-size:3.17500043px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 362 | x="14.45245" | ||
| 363 | y="29.676321" | ||
| 364 | id="text15316-3"><tspan | ||
| 365 | sodipodi:role="line" | ||
| 366 | id="tspan15314-1" | ||
| 367 | x="14.45245" | ||
| 368 | y="29.676321" | ||
| 369 | style="text-align:center;text-anchor:middle;stroke-width:0.26458335">Update</tspan><tspan | ||
| 370 | sodipodi:role="line" | ||
| 371 | x="14.45245" | ||
| 372 | y="33.645073" | ||
| 373 | style="text-align:center;text-anchor:middle;stroke-width:0.26458335" | ||
| 374 | id="tspan15318-9">Lock</tspan></text> | ||
| 375 | <text | ||
| 376 | xml:space="preserve" | ||
| 377 | style="font-style:normal;font-weight:normal;font-size:3.17500043px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 378 | x="7.5676007" | ||
| 379 | y="56.985115" | ||
| 380 | id="text15316-4"><tspan | ||
| 381 | sodipodi:role="line" | ||
| 382 | x="7.5676007" | ||
| 383 | y="56.985115" | ||
| 384 | style="text-align:center;text-anchor:middle;stroke-width:0.26458335" | ||
| 385 | id="tspan15318-7">Register update</tspan><tspan | ||
| 386 | sodipodi:role="line" | ||
| 387 | x="7.5676007" | ||
| 388 | y="60.953865" | ||
| 389 | style="text-align:center;text-anchor:middle;stroke-width:0.26458335" | ||
| 390 | id="tspan15361">Pending Status</tspan></text> | ||
| 391 | <text | ||
| 392 | xml:space="preserve" | ||
| 393 | style="font-style:normal;font-weight:normal;font-size:3.17500043px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 394 | x="16.074829" | ||
| 395 | y="76.167404" | ||
| 396 | id="text15316-8"><tspan | ||
| 397 | sodipodi:role="line" | ||
| 398 | x="16.074829" | ||
| 399 | y="76.167404" | ||
| 400 | style="text-align:center;text-anchor:middle;stroke-width:0.26458335" | ||
| 401 | id="tspan15318-4">Buf 0</tspan></text> | ||
| 402 | <text | ||
| 403 | xml:space="preserve" | ||
| 404 | style="font-style:normal;font-weight:normal;font-size:3.17500067px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458335" | ||
| 405 | x="16.156994" | ||
| 406 | y="86.089279" | ||
| 407 | id="text15316-8-5"><tspan | ||
| 408 | sodipodi:role="line" | ||
| 409 | x="16.156994" | ||
| 410 | y="86.089279" | ||
| 411 | style="text-align:center;text-anchor:middle;stroke-width:0.26458335" | ||
| 412 | id="tspan15318-4-0">Buf 1</tspan></text> | ||
| 413 | </g> | ||
| 414 | </svg> | ||
| @@ -0,0 +1,77 @@ | |||
| 1 | ======================== | ||
| 2 | Display Core Debug tools | ||
| 3 | ======================== | ||
| 4 | |||
| 5 | DC Visual Confirmation | ||
| 6 | ====================== | ||
| 7 | |||
| 8 | Display core provides a feature named visual confirmation, which is a set of | ||
| 9 | bars added at the scanout time by the driver to convey some specific | ||
| 10 | information. In general, you can enable this debug option by using:: | ||
| 11 | |||
| 12 | echo <N> > /sys/kernel/debug/dri/0/amdgpu_dm_visual_confirm | ||
| 13 | |||
| 14 | Where `N` is an integer number for some specific scenarios that the developer | ||
| 15 | wants to enable, you will see some of these debug cases in the following | ||
| 16 | subsection. | ||
| 17 | |||
| 18 | Multiple Planes Debug | ||
| 19 | --------------------- | ||
| 20 | |||
| 21 | If you want to enable or debug multiple planes in a specific user-space | ||
| 22 | application, you can leverage a debug feature named visual confirm. For | ||
| 23 | enabling it, you will need:: | ||
| 24 | |||
| 25 | echo 1 > /sys/kernel/debug/dri/0/amdgpu_dm_visual_confirm | ||
| 26 | |||
| 27 | You need to reload your GUI to see the visual confirmation. When the plane | ||
| 28 | configuration changes or a full update occurs there will be a colored bar at | ||
| 29 | the bottom of each hardware plane being drawn on the screen. | ||
| 30 | |||
| 31 | * The color indicates the format - For example, red is AR24 and green is NV12 | ||
| 32 | * The height of the bar indicates the index of the plane | ||
| 33 | * Pipe split can be observed if there are two bars with a difference in height | ||
| 34 | covering the same plane | ||
| 35 | |||
| 36 | Consider the video playback case in which a video is played in a specific | ||
| 37 | plane, and the desktop is drawn in another plane. The video plane should | ||
| 38 | feature one or two green bars at the bottom of the video depending on pipe | ||
| 39 | split configuration. | ||
| 40 | |||
| 41 | * There should **not** be any visual corruption | ||
| 42 | * There should **not** be any underflow or screen flashes | ||
| 43 | * There should **not** be any black screens | ||
| 44 | * There should **not** be any cursor corruption | ||
| 45 | * Multiple plane **may** be briefly disabled during window transitions or | ||
| 46 | resizing but should come back after the action has finished | ||
| 47 | |||
| 48 | Pipe Split Debug | ||
| 49 | ---------------- | ||
| 50 | |||
| 51 | Sometimes we need to debug if DCN is splitting pipes correctly, and visual | ||
| 52 | confirmation is also handy for this case. Similar to the MPO case, you can use | ||
| 53 | the below command to enable visual confirmation:: | ||
| 54 | |||
| 55 | echo 1 > /sys/kernel/debug/dri/0/amdgpu_dm_visual_confirm | ||
| 56 | |||
| 57 | In this case, if you have a pipe split, you will see one small red bar at the | ||
| 58 | bottom of the display covering the entire display width and another bar | ||
| 59 | covering the second pipe. In other words, you will see a bit high bar in the | ||
| 60 | second pipe. | ||
| 61 | |||
| 62 | DTN Debug | ||
| 63 | ========= | ||
| 64 | |||
| 65 | DC (DCN) provides an extensive log that dumps multiple details from our | ||
| 66 | hardware configuration. Via debugfs, you can capture those status values by | ||
| 67 | using Display Test Next (DTN) log, which can be captured via debugfs by using:: | ||
| 68 | |||
| 69 | cat /sys/kernel/debug/dri/0/amdgpu_dm_dtn_log | ||
| 70 | |||
| 71 | Since this log is updated accordingly with DCN status, you can also follow the | ||
| 72 | change in real-time by using something like:: | ||
| 73 | |||
| 74 | sudo watch -d cat /sys/kernel/debug/dri/0/amdgpu_dm_dtn_log | ||
| 75 | |||
| 76 | When reporting a bug related to DC, consider attaching this log before and | ||
| 77 | after you reproduce the bug. | ||
| @@ -0,0 +1,237 @@ | |||
| 1 | =========== | ||
| 2 | DC Glossary | ||
| 3 | =========== | ||
| 4 | |||
| 5 | On this page, we try to keep track of acronyms related to the display | ||
| 6 | component. If you do not find what you are looking for, look at the | ||
| 7 | 'Documentation/gpu/amdgpu/amdgpu-glossary.rst'; if you cannot find it anywhere, | ||
| 8 | consider asking in the amdgfx and update this page. | ||
| 9 | |||
| 10 | .. glossary:: | ||
| 11 | |||
| 12 | ABM | ||
| 13 | Adaptive Backlight Modulation | ||
| 14 | |||
| 15 | APU | ||
| 16 | Accelerated Processing Unit | ||
| 17 | |||
| 18 | ASIC | ||
| 19 | Application-Specific Integrated Circuit | ||
| 20 | |||
| 21 | ASSR | ||
| 22 | Alternate Scrambler Seed Reset | ||
| 23 | |||
| 24 | AZ | ||
| 25 | Azalia (HD audio DMA engine) | ||
| 26 | |||
| 27 | BPC | ||
| 28 | Bits Per Colour/Component | ||
| 29 | |||
| 30 | BPP | ||
| 31 | Bits Per Pixel | ||
| 32 | |||
| 33 | Clocks | ||
| 34 | * PCLK: Pixel Clock | ||
| 35 | * SYMCLK: Symbol Clock | ||
| 36 | * SOCCLK: GPU Engine Clock | ||
| 37 | * DISPCLK: Display Clock | ||
| 38 | * DPPCLK: DPP Clock | ||
| 39 | * DCFCLK: Display Controller Fabric Clock | ||
| 40 | * REFCLK: Real Time Reference Clock | ||
| 41 | * PPLL: Pixel PLL | ||
| 42 | * FCLK: Fabric Clock | ||
| 43 | * MCLK: Memory Clock | ||
| 44 | |||
| 45 | CRC | ||
| 46 | Cyclic Redundancy Check | ||
| 47 | |||
| 48 | CRTC | ||
| 49 | Cathode Ray Tube Controller - commonly called "Controller" - Generates | ||
| 50 | raw stream of pixels, clocked at pixel clock | ||
| 51 | |||
| 52 | CVT | ||
| 53 | Coordinated Video Timings | ||
| 54 | |||
| 55 | DAL | ||
| 56 | Display Abstraction layer | ||
| 57 | |||
| 58 | DC (Software) | ||
| 59 | Display Core | ||
| 60 | |||
| 61 | DC (Hardware) | ||
| 62 | Display Controller | ||
| 63 | |||
| 64 | DCC | ||
| 65 | Delta Colour Compression | ||
| 66 | |||
| 67 | DCE | ||
| 68 | Display Controller Engine | ||
| 69 | |||
| 70 | DCHUB | ||
| 71 | Display Controller HUB | ||
| 72 | |||
| 73 | ARB | ||
| 74 | Arbiter | ||
| 75 | |||
| 76 | VTG | ||
| 77 | Vertical Timing Generator | ||
| 78 | |||
| 79 | DCN | ||
| 80 | Display Core Next | ||
| 81 | |||
| 82 | DCCG | ||
| 83 | Display Clock Generator block | ||
| 84 | |||
| 85 | DDC | ||
| 86 | Display Data Channel | ||
| 87 | |||
| 88 | DIO | ||
| 89 | Display IO | ||
| 90 | |||
| 91 | DPP | ||
| 92 | Display Pipes and Planes | ||
| 93 | |||
| 94 | DSC | ||
| 95 | Display Stream Compression (Reduce the amount of bits to represent pixel | ||
| 96 | count while at the same pixel clock) | ||
| 97 | |||
| 98 | dGPU | ||
| 99 | discrete GPU | ||
| 100 | |||
| 101 | DMIF | ||
| 102 | Display Memory Interface | ||
| 103 | |||
| 104 | DML | ||
| 105 | Display Mode Library | ||
| 106 | |||
| 107 | DMCU | ||
| 108 | Display Micro-Controller Unit | ||
| 109 | |||
| 110 | DMCUB | ||
| 111 | Display Micro-Controller Unit, version B | ||
| 112 | |||
| 113 | DPCD | ||
| 114 | DisplayPort Configuration Data | ||
| 115 | |||
| 116 | DPM(S) | ||
| 117 | Display Power Management (Signaling) | ||
| 118 | |||
| 119 | DRR | ||
| 120 | Dynamic Refresh Rate | ||
| 121 | |||
| 122 | DWB | ||
| 123 | Display Writeback | ||
| 124 | |||
| 125 | FB | ||
| 126 | Frame Buffer | ||
| 127 | |||
| 128 | FBC | ||
| 129 | Frame Buffer Compression | ||
| 130 | |||
| 131 | FEC | ||
| 132 | Forward Error Correction | ||
| 133 | |||
| 134 | FRL | ||
| 135 | Fixed Rate Link | ||
| 136 | |||
| 137 | GCO | ||
| 138 | Graphical Controller Object | ||
| 139 | |||
| 140 | GSL | ||
| 141 | Global Swap Lock | ||
| 142 | |||
| 143 | iGPU | ||
| 144 | integrated GPU | ||
| 145 | |||
| 146 | ISR | ||
| 147 | Interrupt Service Request | ||
| 148 | |||
| 149 | ISV | ||
| 150 | Independent Software Vendor | ||
| 151 | |||
| 152 | KMD | ||
| 153 | Kernel Mode Driver | ||
| 154 | |||
| 155 | LB | ||
| 156 | Line Buffer | ||
| 157 | |||
| 158 | LFC | ||
| 159 | Low Framerate Compensation | ||
| 160 | |||
| 161 | LTTPR | ||
| 162 | Link Training Tunable Phy Repeater | ||
| 163 | |||
| 164 | LUT | ||
| 165 | Lookup Table | ||
| 166 | |||
| 167 | MALL | ||
| 168 | Memory Access at Last Level | ||
| 169 | |||
| 170 | MC | ||
| 171 | Memory Controller | ||
| 172 | |||
| 173 | MPC | ||
| 174 | Multiple pipes and plane combine | ||
| 175 | |||
| 176 | MPO | ||
| 177 | Multi Plane Overlay | ||
| 178 | |||
| 179 | MST | ||
| 180 | Multi Stream Transport | ||
| 181 | |||
| 182 | NBP State | ||
| 183 | Northbridge Power State | ||
| 184 | |||
| 185 | NBIO | ||
| 186 | North Bridge Input/Output | ||
| 187 | |||
| 188 | ODM | ||
| 189 | Output Data Mapping | ||
| 190 | |||
| 191 | OPM | ||
| 192 | Output Protection Manager | ||
| 193 | |||
| 194 | OPP | ||
| 195 | Output Plane Processor | ||
| 196 | |||
| 197 | OPTC | ||
| 198 | Output Pipe Timing Combiner | ||
| 199 | |||
| 200 | OTG | ||
| 201 | Output Timing Generator | ||
| 202 | |||
| 203 | PCON | ||
| 204 | Power Controller | ||
| 205 | |||
| 206 | PGFSM | ||
| 207 | Power Gate Finite State Machine | ||
| 208 | |||
| 209 | PSR | ||
| 210 | Panel Self Refresh | ||
| 211 | |||
| 212 | SCL | ||
| 213 | Scaler | ||
| 214 | |||
| 215 | SDP | ||
| 216 | Scalable Data Port | ||
| 217 | |||
| 218 | SLS | ||
| 219 | Single Large Surface | ||
| 220 | |||
| 221 | SST | ||
| 222 | Single Stream Transport | ||
| 223 | |||
| 224 | TMDS | ||
| 225 | Transition-Minimized Differential Signaling | ||
| 226 | |||
| 227 | TMZ | ||
| 228 | Trusted Memory Zone | ||
| 229 | |||
| 230 | TTU | ||
| 231 | Time to Underflow | ||
| 232 | |||
| 233 | VRR | ||
| 234 | Variable Refresh Rate | ||
| 235 | |||
| 236 | UVD | ||
| 237 | Unified Video Decoder | ||
| @@ -0,0 +1,1125 @@ | |||
| 1 | <?xml version="1.0" encoding="UTF-8" standalone="no"?> | ||
| 2 | <!-- Created with Inkscape (http://www.inkscape.org/) --> | ||
| 3 | |||
| 4 | <svg | ||
| 5 | xmlns:dc="http://purl.org/dc/elements/1.1/" | ||
| 6 | xmlns:cc="http://creativecommons.org/ns#" | ||
| 7 | xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" | ||
| 8 | xmlns:svg="http://www.w3.org/2000/svg" | ||
| 9 | xmlns="http://www.w3.org/2000/svg" | ||
| 10 | xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" | ||
| 11 | xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" | ||
| 12 | width="1296.7491" | ||
| 13 | height="741.97845" | ||
| 14 | viewBox="0 0 343.0982 196.31514" | ||
| 15 | version="1.1" | ||
| 16 | id="svg8" | ||
| 17 | inkscape:version="0.92.5 (2060ec1f9f, 2020-04-08)" | ||
| 18 | sodipodi:docname="dc_pipeline_overview.svg"> | ||
| 19 | <defs | ||
| 20 | id="defs2"> | ||
| 21 | <marker | ||
| 22 | inkscape:stockid="Arrow2Mend" | ||
| 23 | orient="auto" | ||
| 24 | refY="0" | ||
| 25 | refX="0" | ||
| 26 | id="marker8858" | ||
| 27 | style="overflow:visible" | ||
| 28 | inkscape:isstock="true"> | ||
| 29 | <path | ||
| 30 | id="path8616" | ||
| 31 | style="fill:#aa00d4;fill-opacity:1;fill-rule:evenodd;stroke:#aa00d4;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 32 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 33 | transform="scale(-0.6)" | ||
| 34 | inkscape:connector-curvature="0" /> | ||
| 35 | </marker> | ||
| 36 | <marker | ||
| 37 | inkscape:stockid="Arrow2Send" | ||
| 38 | orient="auto" | ||
| 39 | refY="0" | ||
| 40 | refX="0" | ||
| 41 | id="Arrow2Send" | ||
| 42 | style="overflow:visible" | ||
| 43 | inkscape:isstock="true"> | ||
| 44 | <path | ||
| 45 | id="path8622" | ||
| 46 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 47 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 48 | transform="matrix(-0.3,0,0,-0.3,0.69,0)" | ||
| 49 | inkscape:connector-curvature="0" /> | ||
| 50 | </marker> | ||
| 51 | <marker | ||
| 52 | inkscape:stockid="Arrow1Lend" | ||
| 53 | orient="auto" | ||
| 54 | refY="0" | ||
| 55 | refX="0" | ||
| 56 | id="Arrow1Lend" | ||
| 57 | style="overflow:visible" | ||
| 58 | inkscape:isstock="true"> | ||
| 59 | <path | ||
| 60 | id="path8592" | ||
| 61 | d="M 0,0 5,-5 -12.5,0 5,5 Z" | ||
| 62 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1" | ||
| 63 | transform="matrix(-0.8,0,0,-0.8,-10,0)" | ||
| 64 | inkscape:connector-curvature="0" /> | ||
| 65 | </marker> | ||
| 66 | <marker | ||
| 67 | inkscape:stockid="Arrow2Lend" | ||
| 68 | orient="auto" | ||
| 69 | refY="0" | ||
| 70 | refX="0" | ||
| 71 | id="Arrow2Lend" | ||
| 72 | style="overflow:visible" | ||
| 73 | inkscape:isstock="true"> | ||
| 74 | <path | ||
| 75 | id="path8610" | ||
| 76 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 77 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 78 | transform="matrix(-1.1,0,0,-1.1,-1.1,0)" | ||
| 79 | inkscape:connector-curvature="0" /> | ||
| 80 | </marker> | ||
| 81 | <marker | ||
| 82 | inkscape:stockid="Arrow2Mend" | ||
| 83 | orient="auto" | ||
| 84 | refY="0" | ||
| 85 | refX="0" | ||
| 86 | id="Arrow2Mend" | ||
| 87 | style="overflow:visible" | ||
| 88 | inkscape:isstock="true"> | ||
| 89 | <path | ||
| 90 | id="path1200" | ||
| 91 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 92 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 93 | transform="scale(-0.6)" | ||
| 94 | inkscape:connector-curvature="0" /> | ||
| 95 | </marker> | ||
| 96 | <marker | ||
| 97 | inkscape:stockid="Arrow2Mend" | ||
| 98 | orient="auto" | ||
| 99 | refY="0" | ||
| 100 | refX="0" | ||
| 101 | id="Arrow2Mend-8" | ||
| 102 | style="overflow:visible" | ||
| 103 | inkscape:isstock="true"> | ||
| 104 | <path | ||
| 105 | inkscape:connector-curvature="0" | ||
| 106 | id="path1200-9" | ||
| 107 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 108 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 109 | transform="scale(-0.6)" /> | ||
| 110 | </marker> | ||
| 111 | <marker | ||
| 112 | inkscape:stockid="Arrow2Mend" | ||
| 113 | orient="auto" | ||
| 114 | refY="0" | ||
| 115 | refX="0" | ||
| 116 | id="Arrow2Mend-8-3" | ||
| 117 | style="overflow:visible" | ||
| 118 | inkscape:isstock="true"> | ||
| 119 | <path | ||
| 120 | inkscape:connector-curvature="0" | ||
| 121 | id="path1200-9-6" | ||
| 122 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 123 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 124 | transform="scale(-0.6)" /> | ||
| 125 | </marker> | ||
| 126 | <marker | ||
| 127 | inkscape:stockid="Arrow2Mend" | ||
| 128 | orient="auto" | ||
| 129 | refY="0" | ||
| 130 | refX="0" | ||
| 131 | id="Arrow2Mend-8-3-2" | ||
| 132 | style="overflow:visible" | ||
| 133 | inkscape:isstock="true"> | ||
| 134 | <path | ||
| 135 | inkscape:connector-curvature="0" | ||
| 136 | id="path1200-9-6-9" | ||
| 137 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 138 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 139 | transform="scale(-0.6)" /> | ||
| 140 | </marker> | ||
| 141 | <marker | ||
| 142 | inkscape:stockid="Arrow2Mend" | ||
| 143 | orient="auto" | ||
| 144 | refY="0" | ||
| 145 | refX="0" | ||
| 146 | id="Arrow2Mend-8-3-2-1" | ||
| 147 | style="overflow:visible" | ||
| 148 | inkscape:isstock="true"> | ||
| 149 | <path | ||
| 150 | inkscape:connector-curvature="0" | ||
| 151 | id="path1200-9-6-9-9" | ||
| 152 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 153 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 154 | transform="scale(-0.6)" /> | ||
| 155 | </marker> | ||
| 156 | <marker | ||
| 157 | inkscape:stockid="Arrow2Mend" | ||
| 158 | orient="auto" | ||
| 159 | refY="0" | ||
| 160 | refX="0" | ||
| 161 | id="Arrow2Mend-8-3-2-7" | ||
| 162 | style="overflow:visible" | ||
| 163 | inkscape:isstock="true"> | ||
| 164 | <path | ||
| 165 | inkscape:connector-curvature="0" | ||
| 166 | id="path1200-9-6-9-8" | ||
| 167 | style="fill:#008000;fill-opacity:1;fill-rule:evenodd;stroke:#008000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 168 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 169 | transform="scale(-0.6)" /> | ||
| 170 | </marker> | ||
| 171 | <marker | ||
| 172 | inkscape:stockid="Arrow2Mend" | ||
| 173 | orient="auto" | ||
| 174 | refY="0" | ||
| 175 | refX="0" | ||
| 176 | id="Arrow2Mend-8-3-4" | ||
| 177 | style="overflow:visible" | ||
| 178 | inkscape:isstock="true"> | ||
| 179 | <path | ||
| 180 | inkscape:connector-curvature="0" | ||
| 181 | id="path1200-9-6-5" | ||
| 182 | style="fill:#008000;fill-opacity:1;fill-rule:evenodd;stroke:#008000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 183 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 184 | transform="scale(-0.6)" /> | ||
| 185 | </marker> | ||
| 186 | <marker | ||
| 187 | inkscape:stockid="Arrow2Mend" | ||
| 188 | orient="auto" | ||
| 189 | refY="0" | ||
| 190 | refX="0" | ||
| 191 | id="Arrow2Mend-8-0" | ||
| 192 | style="overflow:visible" | ||
| 193 | inkscape:isstock="true"> | ||
| 194 | <path | ||
| 195 | inkscape:connector-curvature="0" | ||
| 196 | id="path1200-9-3" | ||
| 197 | style="fill:#008000;fill-opacity:1;fill-rule:evenodd;stroke:#008000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 198 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 199 | transform="scale(-0.6)" /> | ||
| 200 | </marker> | ||
| 201 | <marker | ||
| 202 | inkscape:stockid="Arrow2Mend" | ||
| 203 | orient="auto" | ||
| 204 | refY="0" | ||
| 205 | refX="0" | ||
| 206 | id="Arrow2Mend-6" | ||
| 207 | style="overflow:visible" | ||
| 208 | inkscape:isstock="true"> | ||
| 209 | <path | ||
| 210 | inkscape:connector-curvature="0" | ||
| 211 | id="path1200-1" | ||
| 212 | style="fill:#008000;fill-opacity:1;fill-rule:evenodd;stroke:#008000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 213 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 214 | transform="scale(-0.6)" /> | ||
| 215 | </marker> | ||
| 216 | <marker | ||
| 217 | inkscape:stockid="Arrow2Mend" | ||
| 218 | orient="auto" | ||
| 219 | refY="0" | ||
| 220 | refX="0" | ||
| 221 | id="Arrow2Mend-8-3-2-6" | ||
| 222 | style="overflow:visible" | ||
| 223 | inkscape:isstock="true"> | ||
| 224 | <path | ||
| 225 | inkscape:connector-curvature="0" | ||
| 226 | id="path1200-9-6-9-1" | ||
| 227 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 228 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 229 | transform="scale(-0.6)" /> | ||
| 230 | </marker> | ||
| 231 | <marker | ||
| 232 | inkscape:stockid="Arrow2Mend" | ||
| 233 | orient="auto" | ||
| 234 | refY="0" | ||
| 235 | refX="0" | ||
| 236 | id="Arrow2Mend-8-0-7" | ||
| 237 | style="overflow:visible" | ||
| 238 | inkscape:isstock="true"> | ||
| 239 | <path | ||
| 240 | inkscape:connector-curvature="0" | ||
| 241 | id="path1200-9-3-4" | ||
| 242 | style="fill:#008000;fill-opacity:1;fill-rule:evenodd;stroke:#008000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 243 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 244 | transform="scale(-0.6)" /> | ||
| 245 | </marker> | ||
| 246 | <marker | ||
| 247 | inkscape:stockid="Arrow2Mend" | ||
| 248 | orient="auto" | ||
| 249 | refY="0" | ||
| 250 | refX="0" | ||
| 251 | id="Arrow2Mend-6-3" | ||
| 252 | style="overflow:visible" | ||
| 253 | inkscape:isstock="true"> | ||
| 254 | <path | ||
| 255 | inkscape:connector-curvature="0" | ||
| 256 | id="path1200-1-0" | ||
| 257 | style="fill:#008000;fill-opacity:1;fill-rule:evenodd;stroke:#008000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 258 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 259 | transform="scale(-0.6)" /> | ||
| 260 | </marker> | ||
| 261 | <marker | ||
| 262 | inkscape:stockid="Arrow2Mend" | ||
| 263 | orient="auto" | ||
| 264 | refY="0" | ||
| 265 | refX="0" | ||
| 266 | id="Arrow2Mend-8-3-2-8" | ||
| 267 | style="overflow:visible" | ||
| 268 | inkscape:isstock="true"> | ||
| 269 | <path | ||
| 270 | inkscape:connector-curvature="0" | ||
| 271 | id="path1200-9-6-9-6" | ||
| 272 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 273 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 274 | transform="scale(-0.6)" /> | ||
| 275 | </marker> | ||
| 276 | <marker | ||
| 277 | inkscape:stockid="Arrow2Mend" | ||
| 278 | orient="auto" | ||
| 279 | refY="0" | ||
| 280 | refX="0" | ||
| 281 | id="Arrow2Mend-3" | ||
| 282 | style="overflow:visible" | ||
| 283 | inkscape:isstock="true"> | ||
| 284 | <path | ||
| 285 | id="path1200-6" | ||
| 286 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 287 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 288 | transform="scale(-0.6)" | ||
| 289 | inkscape:connector-curvature="0" /> | ||
| 290 | </marker> | ||
| 291 | <marker | ||
| 292 | inkscape:stockid="Arrow2Mend" | ||
| 293 | orient="auto" | ||
| 294 | refY="0" | ||
| 295 | refX="0" | ||
| 296 | id="marker8858-3" | ||
| 297 | style="overflow:visible" | ||
| 298 | inkscape:isstock="true"> | ||
| 299 | <path | ||
| 300 | id="path8616-5" | ||
| 301 | style="fill:#00ffcc;fill-opacity:1;fill-rule:evenodd;stroke:#00ffcc;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 302 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 303 | transform="scale(-0.6)" | ||
| 304 | inkscape:connector-curvature="0" /> | ||
| 305 | </marker> | ||
| 306 | <marker | ||
| 307 | inkscape:stockid="Arrow2Mend" | ||
| 308 | orient="auto" | ||
| 309 | refY="0" | ||
| 310 | refX="0" | ||
| 311 | id="Arrow2Mend-8-3-3" | ||
| 312 | style="overflow:visible" | ||
| 313 | inkscape:isstock="true"> | ||
| 314 | <path | ||
| 315 | inkscape:connector-curvature="0" | ||
| 316 | id="path1200-9-6-56" | ||
| 317 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 318 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 319 | transform="scale(-0.6)" /> | ||
| 320 | </marker> | ||
| 321 | <marker | ||
| 322 | inkscape:stockid="Arrow2Mend" | ||
| 323 | orient="auto" | ||
| 324 | refY="0" | ||
| 325 | refX="0" | ||
| 326 | id="Arrow2Mend-8-0-2" | ||
| 327 | style="overflow:visible" | ||
| 328 | inkscape:isstock="true"> | ||
| 329 | <path | ||
| 330 | inkscape:connector-curvature="0" | ||
| 331 | id="path1200-9-3-9" | ||
| 332 | style="fill:#008000;fill-opacity:1;fill-rule:evenodd;stroke:#008000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 333 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 334 | transform="scale(-0.6)" /> | ||
| 335 | </marker> | ||
| 336 | </defs> | ||
| 337 | <sodipodi:namedview | ||
| 338 | id="base" | ||
| 339 | pagecolor="#ffffff" | ||
| 340 | bordercolor="#666666" | ||
| 341 | borderopacity="1.0" | ||
| 342 | inkscape:pageopacity="0.0" | ||
| 343 | inkscape:pageshadow="2" | ||
| 344 | inkscape:zoom="2.8" | ||
| 345 | inkscape:cx="603.80172" | ||
| 346 | inkscape:cy="404.14319" | ||
| 347 | inkscape:document-units="mm" | ||
| 348 | inkscape:current-layer="layer1" | ||
| 349 | showgrid="false" | ||
| 350 | inkscape:window-width="3840" | ||
| 351 | inkscape:window-height="2096" | ||
| 352 | inkscape:window-x="0" | ||
| 353 | inkscape:window-y="27" | ||
| 354 | inkscape:window-maximized="1" | ||
| 355 | showguides="false" | ||
| 356 | fit-margin-top="0" | ||
| 357 | fit-margin-left="0" | ||
| 358 | fit-margin-right="0" | ||
| 359 | fit-margin-bottom="0" | ||
| 360 | units="px" | ||
| 361 | inkscape:snap-global="false" /> | ||
| 362 | <metadata | ||
| 363 | id="metadata5"> | ||
| 364 | <rdf:RDF> | ||
| 365 | <cc:Work | ||
| 366 | rdf:about=""> | ||
| 367 | <dc:format>image/svg+xml</dc:format> | ||
| 368 | <dc:type | ||
| 369 | rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> | ||
| 370 | <dc:title /> | ||
| 371 | </cc:Work> | ||
| 372 | </rdf:RDF> | ||
| 373 | </metadata> | ||
| 374 | <g | ||
| 375 | inkscape:label="Layer 1" | ||
| 376 | inkscape:groupmode="layer" | ||
| 377 | id="layer1" | ||
| 378 | transform="translate(419.79645,20.103767)"> | ||
| 379 | <path | ||
| 380 | style="fill:#008000;stroke:#008000;stroke-width:0.59275198;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3-2-7)" | ||
| 381 | d="m -340.37552,57.5332 h -14.81024" | ||
| 382 | id="path1171-7-1-3-0" | ||
| 383 | inkscape:connector-curvature="0" /> | ||
| 384 | <path | ||
| 385 | style="fill:#008000;stroke:#008000;stroke-width:0.59715915;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3-4)" | ||
| 386 | d="m -293.23443,57.5332 h -15.03129" | ||
| 387 | id="path1171-7-1-32" | ||
| 388 | inkscape:connector-curvature="0" /> | ||
| 389 | <path | ||
| 390 | style="fill:#008000;stroke:#008000;stroke-width:0.59275198;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-0)" | ||
| 391 | d="M -246.45946,57.5332 H -261.2697" | ||
| 392 | id="path1171-7-6" | ||
| 393 | inkscape:connector-curvature="0" /> | ||
| 394 | <path | ||
| 395 | style="fill:#008000;stroke:#008000;stroke-width:0.59275198;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-6)" | ||
| 396 | d="m -151.28623,57.5332 h -14.81024" | ||
| 397 | id="path1171-0" | ||
| 398 | inkscape:connector-curvature="0" /> | ||
| 399 | <path | ||
| 400 | style="fill:none;stroke:#ff0000;stroke-width:0.98222464;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3-2-6)" | ||
| 401 | d="m -310.11621,-10.988713 h -35.41856" | ||
| 402 | id="path1171-7-1-3-5" | ||
| 403 | inkscape:connector-curvature="0" | ||
| 404 | sodipodi:nodetypes="cc" /> | ||
| 405 | <path | ||
| 406 | style="fill:none;stroke:#ff0000;stroke-width:1.33745635;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3-2-1)" | ||
| 407 | d="M -174.42569,48.441117 V -10.963061 L -277.26548,-11.45916" | ||
| 408 | id="path1171-7-1-3-4" | ||
| 409 | inkscape:connector-curvature="0" | ||
| 410 | sodipodi:nodetypes="ccc" /> | ||
| 411 | <path | ||
| 412 | style="fill:none;stroke:#ff0000;stroke-width:0.95872593;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3)" | ||
| 413 | d="m -262.79442,87.935594 h 14.32069" | ||
| 414 | id="path1171-7-1" | ||
| 415 | inkscape:connector-curvature="0" /> | ||
| 416 | <path | ||
| 417 | style="fill:none;stroke:#ff0000;stroke-width:0.97006679;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8)" | ||
| 418 | d="m -309.80088,87.935594 h 14.44587" | ||
| 419 | id="path1171-7" | ||
| 420 | inkscape:connector-curvature="0" /> | ||
| 421 | <path | ||
| 422 | style="fill:none;stroke:#ff0000;stroke-width:0.96187615;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend)" | ||
| 423 | d="m -356.45657,87.935594 h 14.20296" | ||
| 424 | id="path1171" | ||
| 425 | inkscape:connector-curvature="0" /> | ||
| 426 | <path | ||
| 427 | style="fill:none;stroke:#ff0000;stroke-width:0.96061862;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3-2)" | ||
| 428 | d="m -167.44556,87.935594 h 14.16584" | ||
| 429 | id="path1171-7-1-3" | ||
| 430 | inkscape:connector-curvature="0" /> | ||
| 431 | <path | ||
| 432 | style="fill:none;stroke:#008000;stroke-width:0.87091714;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-0-7)" | ||
| 433 | d="M -193.82812,48.312503 V 14.168502 l -84.03577,-0.467726" | ||
| 434 | id="path1171-7-6-4" | ||
| 435 | inkscape:connector-curvature="0" | ||
| 436 | sodipodi:nodetypes="ccc" /> | ||
| 437 | <path | ||
| 438 | style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112339, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 439 | d="m -133.33998,42.989657 v 5.457081" | ||
| 440 | id="path7149-3-7" | ||
| 441 | inkscape:connector-curvature="0" /> | ||
| 442 | <path | ||
| 443 | style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112339, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 444 | d="m -298.69506,162.44998 v 13.31197" | ||
| 445 | id="path7149" | ||
| 446 | inkscape:connector-curvature="0" /> | ||
| 447 | <path | ||
| 448 | style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 449 | d="m -242.80131,107.00907 v 9.60171" | ||
| 450 | id="path7040-5-4-7-5-6-9" | ||
| 451 | inkscape:connector-curvature="0" /> | ||
| 452 | <path | ||
| 453 | style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 454 | d="m -300.34873,107.17445 v 9.6017" | ||
| 455 | id="path7040-5-4-7-5-6" | ||
| 456 | inkscape:connector-curvature="0" /> | ||
| 457 | <path | ||
| 458 | style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 459 | d="m -359.26293,106.99745 v 9.60171" | ||
| 460 | id="path7040-5-4-7-5" | ||
| 461 | inkscape:connector-curvature="0" /> | ||
| 462 | <path | ||
| 463 | style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 464 | d="M -369.74543,25.114933 V 37.991587" | ||
| 465 | id="path7040-5-4-7-6" | ||
| 466 | inkscape:connector-curvature="0" /> | ||
| 467 | <path | ||
| 468 | style="fill:none;stroke:#008080;stroke-width:0.91136348;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.91136346, 0.91136346;stroke-dashoffset:0;stroke-opacity:1" | ||
| 469 | d="M -135.17034,93.582486 V 107.10642 H -403.93077 V 37.882965 h 109.60575 V 25.225991" | ||
| 470 | id="path7038" | ||
| 471 | inkscape:connector-curvature="0" | ||
| 472 | sodipodi:nodetypes="cccccc" /> | ||
| 473 | <path | ||
| 474 | style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 475 | d="M -231.106,94.010086 V 106.96943" | ||
| 476 | id="path7040" | ||
| 477 | inkscape:connector-curvature="0" /> | ||
| 478 | <path | ||
| 479 | style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 480 | d="M -278.50224,93.844719 V 106.80406" | ||
| 481 | id="path7040-5" | ||
| 482 | inkscape:connector-curvature="0" /> | ||
| 483 | <path | ||
| 484 | style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 485 | d="M -325.89848,93.701083 V 106.99115" | ||
| 486 | id="path7040-5-4" | ||
| 487 | inkscape:connector-curvature="0" /> | ||
| 488 | <path | ||
| 489 | style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 490 | d="M -373.29471,93.899037 V 107.27179" | ||
| 491 | id="path7040-5-4-7" | ||
| 492 | inkscape:connector-curvature="0" /> | ||
| 493 | <g | ||
| 494 | id="g934" | ||
| 495 | transform="matrix(0.61872421,0,0,0.61872421,-154.16506,-3.5724799)"> | ||
| 496 | <rect | ||
| 497 | ry="2.1052283e-06" | ||
| 498 | y="84.280701" | ||
| 499 | x="-376.383" | ||
| 500 | height="72.786827" | ||
| 501 | width="49.352299" | ||
| 502 | id="rect834" | ||
| 503 | style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" /> | ||
| 504 | <text | ||
| 505 | id="text838" | ||
| 506 | y="95.916664" | ||
| 507 | x="-371.17261" | ||
| 508 | style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 509 | xml:space="preserve"><tspan | ||
| 510 | style="stroke-width:0.26458332" | ||
| 511 | y="95.916664" | ||
| 512 | x="-371.17261" | ||
| 513 | id="tspan836" | ||
| 514 | sodipodi:role="line">DCHUB</tspan></text> | ||
| 515 | <text | ||
| 516 | id="text846" | ||
| 517 | y="121.99702" | ||
| 518 | x="-352.74997" | ||
| 519 | style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 520 | xml:space="preserve"><tspan | ||
| 521 | style="text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 522 | y="121.99702" | ||
| 523 | x="-352.74997" | ||
| 524 | id="tspan844" | ||
| 525 | sodipodi:role="line">HUBP</tspan><tspan | ||
| 526 | id="tspan863" | ||
| 527 | style="text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 528 | y="135.22618" | ||
| 529 | x="-352.74997" | ||
| 530 | sodipodi:role="line">(n)</tspan></text> | ||
| 531 | </g> | ||
| 532 | <g | ||
| 533 | id="g942" | ||
| 534 | transform="matrix(0.61872421,0,0,0.61872421,-158.40385,-3.2216813)"> | ||
| 535 | <text | ||
| 536 | id="text838-5" | ||
| 537 | y="116.65257" | ||
| 538 | x="-269.45752" | ||
| 539 | style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 540 | xml:space="preserve"><tspan | ||
| 541 | style="text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 542 | y="116.65257" | ||
| 543 | x="-269.45752" | ||
| 544 | id="tspan836-3" | ||
| 545 | sodipodi:role="line">DPP</tspan><tspan | ||
| 546 | id="tspan936" | ||
| 547 | style="text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 548 | y="129.88174" | ||
| 549 | x="-269.45752" | ||
| 550 | sodipodi:role="line">(n)</tspan></text> | ||
| 551 | <rect | ||
| 552 | ry="2.1052283e-06" | ||
| 553 | y="83.71373" | ||
| 554 | x="-293.7952" | ||
| 555 | height="72.786827" | ||
| 556 | width="49.352303" | ||
| 557 | id="rect834-5" | ||
| 558 | style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" /> | ||
| 559 | </g> | ||
| 560 | <g | ||
| 561 | id="g1158" | ||
| 562 | transform="matrix(0.61872421,0,0,0.61872421,-154.34048,-6.2618995)"> | ||
| 563 | <text | ||
| 564 | id="text838-5-2" | ||
| 565 | y="128.87331" | ||
| 566 | x="-200.18195" | ||
| 567 | style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 568 | xml:space="preserve"><tspan | ||
| 569 | id="tspan936-1" | ||
| 570 | style="text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 571 | y="128.87331" | ||
| 572 | x="-200.18195" | ||
| 573 | sodipodi:role="line">MPC</tspan></text> | ||
| 574 | <rect | ||
| 575 | ry="2.1052283e-06" | ||
| 576 | y="88.627419" | ||
| 577 | x="-224.62555" | ||
| 578 | height="72.786827" | ||
| 579 | width="49.352303" | ||
| 580 | id="rect834-5-2" | ||
| 581 | style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" /> | ||
| 582 | </g> | ||
| 583 | <g | ||
| 584 | id="g1153" | ||
| 585 | transform="matrix(0.61872421,0,0,0.61872421,-108.51628,-6.4957668)"> | ||
| 586 | <text | ||
| 587 | id="text838-5-2-7" | ||
| 588 | y="129.2513" | ||
| 589 | x="-120.96272" | ||
| 590 | style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 591 | xml:space="preserve"><tspan | ||
| 592 | id="tspan936-1-0" | ||
| 593 | style="text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 594 | y="129.2513" | ||
| 595 | x="-120.96272" | ||
| 596 | sodipodi:role="line">OPTC</tspan></text> | ||
| 597 | <rect | ||
| 598 | ry="2.1052283e-06" | ||
| 599 | y="89.005402" | ||
| 600 | x="-145.62854" | ||
| 601 | height="72.786827" | ||
| 602 | width="49.352306" | ||
| 603 | id="rect834-5-2-9" | ||
| 604 | style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" /> | ||
| 605 | </g> | ||
| 606 | <g | ||
| 607 | id="g1148" | ||
| 608 | transform="matrix(0.61872421,0,0,0.61872421,-105.25474,-7.6650796)"> | ||
| 609 | <text | ||
| 610 | id="text838-5-2-7-3" | ||
| 611 | y="131.14117" | ||
| 612 | x="-48.981136" | ||
| 613 | style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 614 | xml:space="preserve"><tspan | ||
| 615 | id="tspan936-1-0-6" | ||
| 616 | style="text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 617 | y="131.14117" | ||
| 618 | x="-48.981136" | ||
| 619 | sodipodi:role="line">DIO</tspan></text> | ||
| 620 | <rect | ||
| 621 | ry="2.1052283e-06" | ||
| 622 | y="90.895279" | ||
| 623 | x="-73.435081" | ||
| 624 | height="72.786827" | ||
| 625 | width="49.352306" | ||
| 626 | id="rect834-5-2-9-0" | ||
| 627 | style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" /> | ||
| 628 | </g> | ||
| 629 | <g | ||
| 630 | id="g1133" | ||
| 631 | transform="matrix(0.61872421,0,0,0.61872421,-181.52704,-7.6650796)"> | ||
| 632 | <text | ||
| 633 | id="text838-5-2-6" | ||
| 634 | y="241.13223" | ||
| 635 | x="-286.96921" | ||
| 636 | style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 637 | xml:space="preserve"><tspan | ||
| 638 | id="tspan936-1-2" | ||
| 639 | style="text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 640 | y="241.13223" | ||
| 641 | x="-286.96921" | ||
| 642 | sodipodi:role="line">DCCG</tspan></text> | ||
| 643 | <rect | ||
| 644 | ry="2.1052283e-06" | ||
| 645 | y="200.88634" | ||
| 646 | x="-311.56009" | ||
| 647 | height="72.786827" | ||
| 648 | width="49.352306" | ||
| 649 | id="rect834-5-2-6" | ||
| 650 | style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" /> | ||
| 651 | </g> | ||
| 652 | <g | ||
| 653 | id="g1138" | ||
| 654 | transform="matrix(0.61872421,0,0,0.61872421,-181.52704,-7.6650796)"> | ||
| 655 | <text | ||
| 656 | id="text838-5-2-6-1" | ||
| 657 | y="241.81844" | ||
| 658 | x="-190.55942" | ||
| 659 | style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 660 | xml:space="preserve"><tspan | ||
| 661 | id="tspan936-1-2-8" | ||
| 662 | style="text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 663 | y="241.81844" | ||
| 664 | x="-190.55942" | ||
| 665 | sodipodi:role="line">DMU</tspan></text> | ||
| 666 | <rect | ||
| 667 | ry="2.1052283e-06" | ||
| 668 | y="201.6423" | ||
| 669 | x="-215.17615" | ||
| 670 | height="72.786827" | ||
| 671 | width="49.352306" | ||
| 672 | id="rect834-5-2-6-7" | ||
| 673 | style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" /> | ||
| 674 | </g> | ||
| 675 | <text | ||
| 676 | xml:space="preserve" | ||
| 677 | style="font-style:normal;font-weight:normal;font-size:6.54816437px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.1637041" | ||
| 678 | x="-297.75696" | ||
| 679 | y="109.44505" | ||
| 680 | id="text1063"><tspan | ||
| 681 | sodipodi:role="line" | ||
| 682 | id="tspan1061" | ||
| 683 | x="-297.75696" | ||
| 684 | y="115.23865" | ||
| 685 | style="stroke-width:0.1637041" /></text> | ||
| 686 | <g | ||
| 687 | id="g1143" | ||
| 688 | transform="matrix(0.61872421,0,0,0.61872421,-181.52704,-8.9747125)"> | ||
| 689 | <text | ||
| 690 | id="text838-5-2-6-1-9" | ||
| 691 | y="243.02728" | ||
| 692 | x="-99.967323" | ||
| 693 | style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 694 | xml:space="preserve"><tspan | ||
| 695 | id="tspan936-1-2-8-2" | ||
| 696 | style="text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 697 | y="243.02728" | ||
| 698 | x="-99.967323" | ||
| 699 | sodipodi:role="line">AZ</tspan></text> | ||
| 700 | <rect | ||
| 701 | ry="2.1052283e-06" | ||
| 702 | y="202.77623" | ||
| 703 | x="-124.83984" | ||
| 704 | height="72.786827" | ||
| 705 | width="49.352306" | ||
| 706 | id="rect834-5-2-6-7-0" | ||
| 707 | style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" /> | ||
| 708 | </g> | ||
| 709 | <g | ||
| 710 | id="g1169" | ||
| 711 | transform="matrix(0.61872421,0,0,0.61872421,-154.16506,1.4555785)"> | ||
| 712 | <text | ||
| 713 | id="text838-5-2-6-2" | ||
| 714 | y="5.9612885" | ||
| 715 | x="-348.74365" | ||
| 716 | style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 717 | xml:space="preserve"><tspan | ||
| 718 | id="tspan936-1-2-3" | ||
| 719 | style="text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 720 | y="5.9612885" | ||
| 721 | x="-348.74365" | ||
| 722 | sodipodi:role="line">MMHUBBUB</tspan></text> | ||
| 723 | <rect | ||
| 724 | ry="2.1010696e-06" | ||
| 725 | y="-34.142948" | ||
| 726 | x="-384.64743" | ||
| 727 | height="72.643044" | ||
| 728 | width="72.096924" | ||
| 729 | id="rect834-5-2-6-75" | ||
| 730 | style="fill:none;stroke:#000000;stroke-width:1.40378118;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" /> | ||
| 731 | </g> | ||
| 732 | <g | ||
| 733 | id="g1164" | ||
| 734 | transform="matrix(0.61872421,0,0,0.61872421,-154.16506,-7.6650796)"> | ||
| 735 | <text | ||
| 736 | id="text838-5-2-6-9" | ||
| 737 | y="13.465075" | ||
| 738 | x="-227.30836" | ||
| 739 | style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 740 | xml:space="preserve"><tspan | ||
| 741 | id="tspan936-1-2-2" | ||
| 742 | style="text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 743 | y="13.465075" | ||
| 744 | x="-227.30836" | ||
| 745 | sodipodi:role="line">DWB</tspan><tspan | ||
| 746 | id="tspan1128" | ||
| 747 | style="text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 748 | y="26.694241" | ||
| 749 | x="-227.30836" | ||
| 750 | sodipodi:role="line">(n)</tspan></text> | ||
| 751 | <rect | ||
| 752 | ry="2.1052283e-06" | ||
| 753 | y="-19.473768" | ||
| 754 | x="-251.83983" | ||
| 755 | height="72.786827" | ||
| 756 | width="49.352306" | ||
| 757 | id="rect834-5-2-6-2" | ||
| 758 | style="fill:none;stroke:#000000;stroke-width:1.16258347;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" /> | ||
| 759 | </g> | ||
| 760 | <path | ||
| 761 | style="fill:none;stroke:#000000;stroke-width:0.91371936;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:5.48231601, 0.91371934;stroke-dashoffset:0;stroke-opacity:1" | ||
| 762 | d="m -358.95963,161.63019 v 14.12431 h 250.20395 V 43.149938 H -361.845 V 25.478973" | ||
| 763 | id="path7147" | ||
| 764 | inkscape:connector-curvature="0" | ||
| 765 | sodipodi:nodetypes="cccccc" /> | ||
| 766 | <path | ||
| 767 | style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112339, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 768 | d="m -242.92533,161.58513 v 14.05612" | ||
| 769 | id="path7149-3" | ||
| 770 | inkscape:connector-curvature="0" /> | ||
| 771 | <path | ||
| 772 | style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112339, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 773 | d="m -184.37695,42.955607 v 5.457082" | ||
| 774 | id="path7149-3-7-4" | ||
| 775 | inkscape:connector-curvature="0" /> | ||
| 776 | <path | ||
| 777 | style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112339, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 778 | d="m -277.36283,43.141644 v 5.457082" | ||
| 779 | id="path7149-3-7-5" | ||
| 780 | inkscape:connector-curvature="0" /> | ||
| 781 | <path | ||
| 782 | style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112339, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 783 | d="M -325.48437,42.976278 V 48.43336" | ||
| 784 | id="path7149-3-7-2" | ||
| 785 | inkscape:connector-curvature="0" /> | ||
| 786 | <path | ||
| 787 | style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112339, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 788 | d="m -361.86492,43.141644 v 5.457083" | ||
| 789 | id="path7149-3-7-54" | ||
| 790 | inkscape:connector-curvature="0" /> | ||
| 791 | <path | ||
| 792 | style="fill:#008000;stroke:#008000;stroke-width:0.46329758;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-6-3)" | ||
| 793 | d="m -147.58542,-8.2978166 h -9.04766" | ||
| 794 | id="path1171-0-7" | ||
| 795 | inkscape:connector-curvature="0" /> | ||
| 796 | <path | ||
| 797 | style="fill:none;stroke:#ff0000;stroke-width:0.98222464;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3-2-8)" | ||
| 798 | d="m -157.13421,-1.6500501 h 8.66407" | ||
| 799 | id="path1171-7-1-3-8" | ||
| 800 | inkscape:connector-curvature="0" /> | ||
| 801 | <path | ||
| 802 | style="fill:none;stroke:#000000;stroke-width:0.74503672;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.47022031, 0.74503672;stroke-dashoffset:0;stroke-opacity:1" | ||
| 803 | d="m -148.50314,4.9845652 h -7.91265" | ||
| 804 | id="path7149-3-7-8" | ||
| 805 | inkscape:connector-curvature="0" /> | ||
| 806 | <path | ||
| 807 | style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852057, 0.81852057;stroke-dashoffset:0;stroke-opacity:1" | ||
| 808 | d="m -157.59442,11.623513 h 10.26991" | ||
| 809 | id="path7040-4" | ||
| 810 | inkscape:connector-curvature="0" /> | ||
| 811 | <text | ||
| 812 | xml:space="preserve" | ||
| 813 | style="font-style:normal;font-weight:normal;font-size:6.54816437px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.1637041" | ||
| 814 | x="-142.72867" | ||
| 815 | y="-6.9685979" | ||
| 816 | id="text12079"><tspan | ||
| 817 | sodipodi:role="line" | ||
| 818 | id="tspan12077" | ||
| 819 | x="-142.72867" | ||
| 820 | y="-6.9685979" | ||
| 821 | style="font-size:4.80198765px;stroke-width:0.1637041">Global sync</tspan></text> | ||
| 822 | <text | ||
| 823 | xml:space="preserve" | ||
| 824 | style="font-style:normal;font-weight:normal;font-size:6.54816437px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.1637041" | ||
| 825 | x="-142.93031" | ||
| 826 | y="0.13578746" | ||
| 827 | id="text12079-3"><tspan | ||
| 828 | sodipodi:role="line" | ||
| 829 | id="tspan12077-1" | ||
| 830 | x="-142.93031" | ||
| 831 | y="0.13578746" | ||
| 832 | style="font-size:4.80198765px;stroke-width:0.1637041">Pixel data</tspan></text> | ||
| 833 | <text | ||
| 834 | xml:space="preserve" | ||
| 835 | style="font-style:normal;font-weight:normal;font-size:6.54816437px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.1637041" | ||
| 836 | x="-142.77556" | ||
| 837 | y="6.3093324" | ||
| 838 | id="text12079-3-4"><tspan | ||
| 839 | sodipodi:role="line" | ||
| 840 | id="tspan12077-1-9" | ||
| 841 | x="-142.77556" | ||
| 842 | y="6.3093324" | ||
| 843 | style="font-size:4.80198765px;stroke-width:0.1637041">Sideband signal</tspan></text> | ||
| 844 | <text | ||
| 845 | xml:space="preserve" | ||
| 846 | style="font-style:normal;font-weight:normal;font-size:6.54816437px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.1637041" | ||
| 847 | x="-142.72867" | ||
| 848 | y="12.948278" | ||
| 849 | id="text12079-3-4-2"><tspan | ||
| 850 | sodipodi:role="line" | ||
| 851 | id="tspan12077-1-9-0" | ||
| 852 | x="-142.72867" | ||
| 853 | y="12.948278" | ||
| 854 | style="font-size:4.80198765px;stroke-width:0.1637041">Config. Bus</tspan></text> | ||
| 855 | <path | ||
| 856 | style="fill:none;stroke:#aa00d4;stroke-width:1.32291663;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker8858)" | ||
| 857 | d="m -406.68795,73.185276 h 14.20296" | ||
| 858 | id="path1171-75" | ||
| 859 | inkscape:connector-curvature="0" /> | ||
| 860 | <text | ||
| 861 | xml:space="preserve" | ||
| 862 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 863 | x="-420.21503" | ||
| 864 | y="75.065918" | ||
| 865 | id="text8862"><tspan | ||
| 866 | sodipodi:role="line" | ||
| 867 | id="tspan8860" | ||
| 868 | x="-420.21503" | ||
| 869 | y="75.065918" | ||
| 870 | style="font-size:6.3499999px;stroke-width:0.26458332">SDP</tspan></text> | ||
| 871 | <path | ||
| 872 | style="fill:none;stroke:#00ffcc;stroke-width:1.25980031;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#marker8858-3)" | ||
| 873 | d="m -119.19923,72.243805 h 12.88004" | ||
| 874 | id="path1171-75-6" | ||
| 875 | inkscape:connector-curvature="0" /> | ||
| 876 | <text | ||
| 877 | xml:space="preserve" | ||
| 878 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 879 | x="-104.87327" | ||
| 880 | y="74.54258" | ||
| 881 | id="text8862-2"><tspan | ||
| 882 | sodipodi:role="line" | ||
| 883 | id="tspan8860-9" | ||
| 884 | x="-104.87327" | ||
| 885 | y="74.54258" | ||
| 886 | style="font-size:6.3499999px;stroke-width:0.26458332">Monitor</tspan></text> | ||
| 887 | <g | ||
| 888 | id="g6280" | ||
| 889 | transform="translate(-133.43389,-37.35791)"> | ||
| 890 | <text | ||
| 891 | id="text838-5-2-7-6" | ||
| 892 | y="110.67171" | ||
| 893 | x="-97.4758" | ||
| 894 | style="font-style:normal;font-weight:normal;font-size:6.54816437px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.16370411" | ||
| 895 | xml:space="preserve"><tspan | ||
| 896 | id="tspan936-1-0-7" | ||
| 897 | style="text-align:center;text-anchor:middle;stroke-width:0.16370411" | ||
| 898 | y="110.67171" | ||
| 899 | x="-97.4758" | ||
| 900 | sodipodi:role="line">OPP</tspan></text> | ||
| 901 | <rect | ||
| 902 | ry="1.3025557e-06" | ||
| 903 | y="85.770599" | ||
| 904 | x="-112.73714" | ||
| 905 | height="45.034973" | ||
| 906 | width="30.535467" | ||
| 907 | id="rect834-5-2-9-5" | ||
| 908 | style="fill:none;stroke:#000000;stroke-width:0.71931857;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none" /> | ||
| 909 | </g> | ||
| 910 | <path | ||
| 911 | style="fill:#008000;stroke:#008000;stroke-width:0.59275198;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-0-2)" | ||
| 912 | d="m -199.6735,57.600919 h -14.81024" | ||
| 913 | id="path1171-7-6-1" | ||
| 914 | inkscape:connector-curvature="0" /> | ||
| 915 | <path | ||
| 916 | style="fill:none;stroke:#ff0000;stroke-width:0.95872593;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-end:url(#Arrow2Mend-8-3-3)" | ||
| 917 | d="m -214.95012,88.003315 h 14.32069" | ||
| 918 | id="path1171-7-1-2" | ||
| 919 | inkscape:connector-curvature="0" /> | ||
| 920 | <path | ||
| 921 | style="fill:none;stroke:#008080;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.81852058, 0.81852058;stroke-dashoffset:0;stroke-opacity:1" | ||
| 922 | d="M -182.99565,94.057598 V 107.01694" | ||
| 923 | id="path7040-7" | ||
| 924 | inkscape:connector-curvature="0" /> | ||
| 925 | <path | ||
| 926 | style="fill:none;stroke:#000000;stroke-width:0.81852055;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:4.91112344, 0.81852058;stroke-dashoffset:0;stroke-opacity:1" | ||
| 927 | d="m -231.7616,43.563759 v 5.457082" | ||
| 928 | id="path7149-3-7-4-0" | ||
| 929 | inkscape:connector-curvature="0" /> | ||
| 930 | <g | ||
| 931 | aria-label="[" | ||
| 932 | transform="matrix(0,-1,0.74237844,0,14.567595,39.540924)" | ||
| 933 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 934 | id="text6872"> | ||
| 935 | <path | ||
| 936 | d="m -65.936923,-548.78511 h 8.816294 v 2.79112 h -6.82247 v 176.34952 h 6.82247 v 2.41314 h -8.816294 z" | ||
| 937 | style="font-size:50.79999924px;stroke-width:0.26458332" | ||
| 938 | id="path6874" | ||
| 939 | inkscape:connector-curvature="0" | ||
| 940 | sodipodi:nodetypes="ccccccccc" /> | ||
| 941 | </g> | ||
| 942 | <g | ||
| 943 | aria-label="[" | ||
| 944 | transform="rotate(-90,182.49521,-144.01791)" | ||
| 945 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 946 | id="text6872-3"> | ||
| 947 | <path | ||
| 948 | d="m -63.825546,-623.34091 h 7.228794 v 2.26195 h -5.764137 l 0,127.08032 h 5.764137 v 1.88397 h -7.228794 z" | ||
| 949 | style="font-size:50.79999924px;stroke-width:0.26458332" | ||
| 950 | id="path6874-6" | ||
| 951 | inkscape:connector-curvature="0" | ||
| 952 | sodipodi:nodetypes="ccccccccc" /> | ||
| 953 | </g> | ||
| 954 | <text | ||
| 955 | xml:space="preserve" | ||
| 956 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 957 | x="-359.80389" | ||
| 958 | y="99.104233" | ||
| 959 | id="text6929"><tspan | ||
| 960 | sodipodi:role="line" | ||
| 961 | id="tspan6927" | ||
| 962 | x="-359.80389" | ||
| 963 | y="99.104233" | ||
| 964 | style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444447px;font-family:sans-serif;-inkscape-font-specification:'sans-serif Bold';stroke-width:0.26458332">dc_plane</tspan></text> | ||
| 965 | <text | ||
| 966 | xml:space="preserve" | ||
| 967 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 968 | x="-223.56163" | ||
| 969 | y="99.142021" | ||
| 970 | id="text6933"><tspan | ||
| 971 | sodipodi:role="line" | ||
| 972 | id="tspan6931" | ||
| 973 | x="-223.56163" | ||
| 974 | y="99.142021" | ||
| 975 | style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444447px;font-family:sans-serif;-inkscape-font-specification:'sans-serif Bold';stroke-width:0.26458332">dc_stream</tspan></text> | ||
| 976 | <g | ||
| 977 | aria-label="[" | ||
| 978 | transform="matrix(0,1,1,0,153.30551,96.566025)" | ||
| 979 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 980 | id="text6872-35"> | ||
| 981 | <path | ||
| 982 | d="m -65.936923,-545.95029 h 8.816294 v 2.79112 h -6.898066 v 271.78851 h 6.898066 v 2.41314 h -8.816294 z" | ||
| 983 | style="font-size:50.79999924px;stroke-width:0.26458332" | ||
| 984 | id="path6874-62" | ||
| 985 | inkscape:connector-curvature="0" | ||
| 986 | sodipodi:nodetypes="ccccccccc" /> | ||
| 987 | </g> | ||
| 988 | <text | ||
| 989 | xml:space="preserve" | ||
| 990 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 991 | x="-267.43958" | ||
| 992 | y="28.5028" | ||
| 993 | id="text6933-9"><tspan | ||
| 994 | sodipodi:role="line" | ||
| 995 | id="tspan6931-1" | ||
| 996 | x="-267.43958" | ||
| 997 | y="28.5028" | ||
| 998 | style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444447px;font-family:sans-serif;-inkscape-font-specification:'sans-serif Bold';stroke-width:0.26458332">dc_state</tspan></text> | ||
| 999 | <g | ||
| 1000 | aria-label="[" | ||
| 1001 | transform="matrix(0,0.98158883,-1.0187565,0,0,-7.4835468)" | ||
| 1002 | style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:6.62759447px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif Bold';letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.5522995" | ||
| 1003 | id="text6973"> | ||
| 1004 | <path | ||
| 1005 | d="m 23.679381,144.30265 h 3.028123 v 1.29445 h -1.820839 v 7.91629 h 1.820839 v 1.29445 h -3.028123 z" | ||
| 1006 | style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:11.78239059px;font-family:sans-serif;-inkscape-font-specification:'sans-serif Bold';stroke-width:0.5522995" | ||
| 1007 | id="path6975" | ||
| 1008 | inkscape:connector-curvature="0" | ||
| 1009 | sodipodi:nodetypes="ccccccccc" /> | ||
| 1010 | </g> | ||
| 1011 | <text | ||
| 1012 | xml:space="preserve" | ||
| 1013 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 1014 | x="-142.71655" | ||
| 1015 | y="18.955769" | ||
| 1016 | id="text6980"><tspan | ||
| 1017 | sodipodi:role="line" | ||
| 1018 | id="tspan6978" | ||
| 1019 | x="-142.71655" | ||
| 1020 | y="18.955769" | ||
| 1021 | style="font-size:4.58611107px;stroke-width:0.26458332">Code struct</tspan></text> | ||
| 1022 | <g | ||
| 1023 | aria-label="[" | ||
| 1024 | transform="rotate(-90,94.826273,-58.762727)" | ||
| 1025 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 1026 | id="text6872-35-2"> | ||
| 1027 | <path | ||
| 1028 | d="m -66.881863,-308.95922 h 7.115401 l 0,1.69499 h -5.197173 v 42.03568 h 5.197173 v 1.78948 h -7.115401 z" | ||
| 1029 | style="font-size:50.79999924px;stroke-width:0.26458332" | ||
| 1030 | id="path6874-62-7" | ||
| 1031 | inkscape:connector-curvature="0" | ||
| 1032 | sodipodi:nodetypes="ccccccccc" /> | ||
| 1033 | </g> | ||
| 1034 | <text | ||
| 1035 | xml:space="preserve" | ||
| 1036 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 1037 | x="-134.09625" | ||
| 1038 | y="99.354439" | ||
| 1039 | id="text6933-9-0"><tspan | ||
| 1040 | sodipodi:role="line" | ||
| 1041 | id="tspan6931-1-9" | ||
| 1042 | x="-134.09625" | ||
| 1043 | y="99.354439" | ||
| 1044 | style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:5.64444447px;font-family:sans-serif;-inkscape-font-specification:'sans-serif Bold';stroke-width:0.26458332">dc_link</tspan></text> | ||
| 1045 | <g | ||
| 1046 | aria-label="}" | ||
| 1047 | transform="rotate(90,-145.27371,-140.09832)" | ||
| 1048 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#3771c8;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 1049 | id="text1003"> | ||
| 1050 | <path | ||
| 1051 | d="m 102.58571,58.211269 h 0.86816 c 1.15755,0 3.11267,-0.177767 3.45994,-0.5333 0.35553,-0.355534 1.28925,-1.124479 1.28925,-2.306836 V -61.475482 c 0,-1.289844 0.18603,-2.228288 0.5581,-2.815332 0.37207,-0.587044 0.26105,-0.992187 1.17882,-1.215429 -0.91777,-0.206706 -0.80675,-0.603581 -1.17882,-1.190625 -0.37207,-0.587045 -0.5581,-1.529623 -0.5581,-2.827735 v -3.075781 c 0,-1.174088 -0.93372,-1.938899 -1.28925,-2.294433 -0.34727,-0.363802 -2.30239,-0.545703 -3.45994,-0.545703 h -0.86816 v -1.773536 h 0.78134 c 2.05879,0 4.63403,0.305924 5.32029,0.917774 0.69453,0.60358 1.0418,1.81901 1.0418,3.646289 v 2.976562 c 0,1.231966 0.22324,2.087728 0.66973,2.567285 0.44648,0.471289 5.80035,0.706934 6.97444,0.706934 h 0.76894 v 1.773535 h -0.76894 c -1.17409,0 -6.52796,0.239778 -6.97444,0.719336 -0.44649,0.479557 -0.66973,1.343587 -0.66973,2.59209 V 55.420742 c 0,1.827279 -0.34727,3.046842 -1.0418,3.658691 -0.68626,0.611849 -3.2615,0.917774 -5.32029,0.917774 h -0.78134 z" | ||
| 1052 | style="font-size:25.39999962px;fill:#3771c8;stroke-width:0.26458332" | ||
| 1053 | id="path1005" | ||
| 1054 | inkscape:connector-curvature="0" | ||
| 1055 | sodipodi:nodetypes="cscsscccsscsccscsscsccscsscscc" /> | ||
| 1056 | </g> | ||
| 1057 | <text | ||
| 1058 | xml:space="preserve" | ||
| 1059 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 1060 | x="-200.59984" | ||
| 1061 | y="129.60852" | ||
| 1062 | id="text1010"><tspan | ||
| 1063 | sodipodi:role="line" | ||
| 1064 | id="tspan1008" | ||
| 1065 | x="-200.59984" | ||
| 1066 | y="129.60852" | ||
| 1067 | style="font-style:italic;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:6.3499999px;font-family:sans-serif;-inkscape-font-specification:'sans-serif Bold Italic';text-align:center;text-anchor:middle;stroke-width:0.26458332">Floating point</tspan><tspan | ||
| 1068 | sodipodi:role="line" | ||
| 1069 | x="-200.59984" | ||
| 1070 | y="137.54602" | ||
| 1071 | style="font-size:6.3499999px;text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 1072 | id="tspan1059">calculation</tspan></text> | ||
| 1073 | <g | ||
| 1074 | aria-label="}" | ||
| 1075 | transform="rotate(90,-94.294068,-92.593178)" | ||
| 1076 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#3771c8;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 1077 | id="text1003-5"> | ||
| 1078 | <path | ||
| 1079 | d="m 102.58571,58.211269 h 0.86816 c 1.15755,0 3.11267,-0.177767 3.45994,-0.5333 0.35553,-0.355534 1.10026,-1.124479 1.10026,-2.306836 V -44.637502 c 0,-1.289844 0.18603,-2.228288 0.5581,-2.815332 0.37207,-0.587044 0.45004,-0.992187 1.36781,-1.215429 -0.91777,-0.206706 -0.99574,-0.603581 -1.36781,-1.190625 -0.37207,-0.587045 -0.5581,-1.529623 -0.5581,-2.827735 v -19.913761 c 0,-1.174088 -0.74473,-1.938899 -1.10026,-2.294433 -0.34727,-0.363802 -2.30239,-0.545703 -3.45994,-0.545703 h -0.86816 v -1.773536 h 0.78134 c 2.05879,0 4.63403,0.305924 5.32029,0.917774 0.69453,0.60358 1.0418,1.81901 1.0418,3.646289 v 19.814542 c 0,1.231966 0.22324,2.087728 0.66973,2.567285 0.44648,0.471289 1.25677,0.706934 2.43086,0.706934 h 0.76894 v 1.773535 h -0.76894 c -1.17409,0 -1.98438,0.239778 -2.43086,0.719336 -0.44649,0.479557 -0.66973,1.343587 -0.66973,2.59209 v 99.897013 c 0,1.827279 -0.34727,3.046842 -1.0418,3.658691 -0.68626,0.611849 -3.2615,0.917774 -5.32029,0.917774 h -0.78134 z" | ||
| 1080 | style="font-size:25.39999962px;fill:#3771c8;stroke-width:0.26458332" | ||
| 1081 | id="path1005-3" | ||
| 1082 | inkscape:connector-curvature="0" | ||
| 1083 | sodipodi:nodetypes="cscsscccsscsccscsscsccscsscscc" /> | ||
| 1084 | </g> | ||
| 1085 | <text | ||
| 1086 | xml:space="preserve" | ||
| 1087 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 1088 | x="-137.43764" | ||
| 1089 | y="122.46283" | ||
| 1090 | id="text1010-5"><tspan | ||
| 1091 | sodipodi:role="line" | ||
| 1092 | id="tspan1008-6" | ||
| 1093 | x="-137.43764" | ||
| 1094 | y="122.46283" | ||
| 1095 | style="font-size:6.3499999px;text-align:center;text-anchor:middle;stroke-width:0.26458332">bit-depth</tspan><tspan | ||
| 1096 | sodipodi:role="line" | ||
| 1097 | x="-137.43764" | ||
| 1098 | y="130.40033" | ||
| 1099 | style="font-size:6.3499999px;text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 1100 | id="tspan1057">reduction/dither</tspan></text> | ||
| 1101 | <text | ||
| 1102 | xml:space="preserve" | ||
| 1103 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#3771c8;fill-opacity:1;stroke:none;stroke-width:0.26458332;" | ||
| 1104 | x="21.087883" | ||
| 1105 | y="155.64751" | ||
| 1106 | id="text1064" | ||
| 1107 | transform="rotate(90)"><tspan | ||
| 1108 | sodipodi:role="line" | ||
| 1109 | id="tspan1062" | ||
| 1110 | x="21.087883" | ||
| 1111 | y="155.64751" | ||
| 1112 | style="font-size:9.87777805px;stroke-width:0.26458332;fill:#3771c8;">}</tspan></text> | ||
| 1113 | <text | ||
| 1114 | xml:space="preserve" | ||
| 1115 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 1116 | x="-142.71655" | ||
| 1117 | y="25.939869" | ||
| 1118 | id="text6980-9"><tspan | ||
| 1119 | sodipodi:role="line" | ||
| 1120 | id="tspan6978-1" | ||
| 1121 | x="-142.71655" | ||
| 1122 | y="25.939869" | ||
| 1123 | style="font-size:4.58611107px;stroke-width:0.26458332">Notes</tspan></text> | ||
| 1124 | </g> | ||
| 1125 | </svg> | ||
| @@ -0,0 +1,171 @@ | |||
| 1 | ======================= | ||
| 2 | Display Core Next (DCN) | ||
| 3 | ======================= | ||
| 4 | |||
| 5 | To equip our readers with the basic knowledge of how AMD Display Core Next | ||
| 6 | (DCN) works, we need to start with an overview of the hardware pipeline. Below | ||
| 7 | you can see a picture that provides a DCN overview, keep in mind that this is a | ||
| 8 | generic diagram, and we have variations per ASIC. | ||
| 9 | |||
| 10 | .. kernel-figure:: dc_pipeline_overview.svg | ||
| 11 | |||
| 12 | Based on this diagram, we can pass through each block and briefly describe | ||
| 13 | them: | ||
| 14 | |||
| 15 | * **Display Controller Hub (DCHUB)**: This is the gateway between the Scalable | ||
| 16 | Data Port (SDP) and DCN. This component has multiple features, such as memory | ||
| 17 | arbitration, rotation, and cursor manipulation. | ||
| 18 | |||
| 19 | * **Display Pipe and Plane (DPP)**: This block provides pre-blend pixel | ||
| 20 | processing such as color space conversion, linearization of pixel data, tone | ||
| 21 | mapping, and gamut mapping. | ||
| 22 | |||
| 23 | * **Multiple Pipe/Plane Combined (MPC)**: This component performs blending of | ||
| 24 | multiple planes, using global or per-pixel alpha. | ||
| 25 | |||
| 26 | * **Output Pixel Processing (OPP)**: Process and format pixels to be sent to | ||
| 27 | the display. | ||
| 28 | |||
| 29 | * **Output Pipe Timing Combiner (OPTC)**: It generates time output to combine | ||
| 30 | streams or divide capabilities. CRC values are generated in this block. | ||
| 31 | |||
| 32 | * **Display Output (DIO)**: Codify the output to the display connected to our | ||
| 33 | GPU. | ||
| 34 | |||
| 35 | * **Display Writeback (DWB)**: It provides the ability to write the output of | ||
| 36 | the display pipe back to memory as video frames. | ||
| 37 | |||
| 38 | * **Multi-Media HUB (MMHUBBUB)**: Memory controller interface for DMCUB and DWB | ||
| 39 | (Note that DWB is not hooked yet). | ||
| 40 | |||
| 41 | * **DCN Management Unit (DMU)**: It provides registers with access control and | ||
| 42 | interrupts the controller to the SOC host interrupt unit. This block includes | ||
| 43 | the Display Micro-Controller Unit - version B (DMCUB), which is handled via | ||
| 44 | firmware. | ||
| 45 | |||
| 46 | * **DCN Clock Generator Block (DCCG)**: It provides the clocks and resets | ||
| 47 | for all of the display controller clock domains. | ||
| 48 | |||
| 49 | * **Azalia (AZ)**: Audio engine. | ||
| 50 | |||
| 51 | The above diagram is an architecture generalization of DCN, which means that | ||
| 52 | every ASIC has variations around this base model. Notice that the display | ||
| 53 | pipeline is connected to the Scalable Data Port (SDP) via DCHUB; you can see | ||
| 54 | the SDP as the element from our Data Fabric that feeds the display pipe. | ||
| 55 | |||
| 56 | Always approach the DCN architecture as something flexible that can be | ||
| 57 | configured and reconfigured in multiple ways; in other words, each block can be | ||
| 58 | setup or ignored accordingly with userspace demands. For example, if we | ||
| 59 | want to drive an 8k@60Hz with a DSC enabled, our DCN may require 4 DPP and 2 | ||
| 60 | OPP. It is DC's responsibility to drive the best configuration for each | ||
| 61 | specific scenario. Orchestrate all of these components together requires a | ||
| 62 | sophisticated communication interface which is highlighted in the diagram by | ||
| 63 | the edges that connect each block; from the chart, each connection between | ||
| 64 | these blocks represents: | ||
| 65 | |||
| 66 | 1. Pixel data interface (red): Represents the pixel data flow; | ||
| 67 | 2. Global sync signals (green): It is a set of synchronization signals composed | ||
| 68 | by VStartup, VUpdate, and VReady; | ||
| 69 | 3. Config interface: Responsible to configure blocks; | ||
| 70 | 4. Sideband signals: All other signals that do not fit the previous one. | ||
| 71 | |||
| 72 | These signals are essential and play an important role in DCN. Nevertheless, | ||
| 73 | the Global Sync deserves an extra level of detail described in the next | ||
| 74 | section. | ||
| 75 | |||
| 76 | All of these components are represented by a data structure named dc_state. | ||
| 77 | From DCHUB to MPC, we have a representation called dc_plane; from MPC to OPTC, | ||
| 78 | we have dc_stream, and the output (DIO) is handled by dc_link. Keep in mind | ||
| 79 | that HUBP accesses a surface using a specific format read from memory, and our | ||
| 80 | dc_plane should work to convert all pixels in the plane to something that can | ||
| 81 | be sent to the display via dc_stream and dc_link. | ||
| 82 | |||
| 83 | Front End and Back End | ||
| 84 | ---------------------- | ||
| 85 | |||
| 86 | Display pipeline can be broken down into two components that are usually | ||
| 87 | referred as **Front End (FE)** and **Back End (BE)**, where FE consists of: | ||
| 88 | |||
| 89 | * DCHUB (Mainly referring to a subcomponent named HUBP) | ||
| 90 | * DPP | ||
| 91 | * MPC | ||
| 92 | |||
| 93 | On the other hand, BE consist of | ||
| 94 | |||
| 95 | * OPP | ||
| 96 | * OPTC | ||
| 97 | * DIO (DP/HDMI stream encoder and link encoder) | ||
| 98 | |||
| 99 | OPP and OPTC are two joining blocks between FE and BE. On a side note, this is | ||
| 100 | a one-to-one mapping of the link encoder to PHY, but we can configure the DCN | ||
| 101 | to choose which link encoder to connect to which PHY. FE's main responsibility | ||
| 102 | is to change, blend and compose pixel data, while BE's job is to frame a | ||
| 103 | generic pixel stream to a specific display's pixel stream. | ||
| 104 | |||
| 105 | Data Flow | ||
| 106 | --------- | ||
| 107 | |||
| 108 | Initially, data is passed in from VRAM through Data Fabric (DF) in native pixel | ||
| 109 | formats. Such data format stays through till HUBP in DCHUB, where HUBP unpacks | ||
| 110 | different pixel formats and outputs them to DPP in uniform streams through 4 | ||
| 111 | channels (1 for alpha + 3 for colors). | ||
| 112 | |||
| 113 | The Converter and Cursor (CNVC) in DPP would then normalize the data | ||
| 114 | representation and convert them to a DCN specific floating-point format (i.e., | ||
| 115 | different from the IEEE floating-point format). In the process, CNVC also | ||
| 116 | applies a degamma function to transform the data from non-linear to linear | ||
| 117 | space to relax the floating-point calculations following. Data would stay in | ||
| 118 | this floating-point format from DPP to OPP. | ||
| 119 | |||
| 120 | Starting OPP, because color transformation and blending have been completed | ||
| 121 | (i.e alpha can be dropped), and the end sinks do not require the precision and | ||
| 122 | dynamic range that floating points provide (i.e. all displays are in integer | ||
| 123 | depth format), bit-depth reduction/dithering would kick in. In OPP, we would | ||
| 124 | also apply a regamma function to introduce the gamma removed earlier back. | ||
| 125 | Eventually, we output data in integer format at DIO. | ||
| 126 | |||
| 127 | Global Sync | ||
| 128 | ----------- | ||
| 129 | |||
| 130 | Many DCN registers are double buffered, most importantly the surface address. | ||
| 131 | This allows us to update DCN hardware atomically for page flips, as well as | ||
| 132 | for most other updates that don't require enabling or disabling of new pipes. | ||
| 133 | |||
| 134 | (Note: There are many scenarios when DC will decide to reserve extra pipes | ||
| 135 | in order to support outputs that need a very high pixel clock, or for | ||
| 136 | power saving purposes.) | ||
| 137 | |||
| 138 | These atomic register updates are driven by global sync signals in DCN. In | ||
| 139 | order to understand how atomic updates interact with DCN hardware, and how DCN | ||
| 140 | signals page flip and vblank events it is helpful to understand how global sync | ||
| 141 | is programmed. | ||
| 142 | |||
| 143 | Global sync consists of three signals, VSTARTUP, VUPDATE, and VREADY. These are | ||
| 144 | calculated by the Display Mode Library - DML (drivers/gpu/drm/amd/display/dc/dml) | ||
| 145 | based on a large number of parameters and ensure our hardware is able to feed | ||
| 146 | the DCN pipeline without underflows or hangs in any given system configuration. | ||
| 147 | The global sync signals always happen during VBlank, are independent from the | ||
| 148 | VSync signal, and do not overlap each other. | ||
| 149 | |||
| 150 | VUPDATE is the only signal that is of interest to the rest of the driver stack | ||
| 151 | or userspace clients as it signals the point at which hardware latches to | ||
| 152 | atomically programmed (i.e. double buffered) registers. Even though it is | ||
| 153 | independent of the VSync signal we use VUPDATE to signal the VSync event as it | ||
| 154 | provides the best indication of how atomic commits and hardware interact. | ||
| 155 | |||
| 156 | Since DCN hardware is double-buffered the DC driver is able to program the | ||
| 157 | hardware at any point during the frame. | ||
| 158 | |||
| 159 | The below picture illustrates the global sync signals: | ||
| 160 | |||
| 161 | .. kernel-figure:: global_sync_vblank.svg | ||
| 162 | |||
| 163 | These signals affect core DCN behavior. Programming them incorrectly will lead | ||
| 164 | to a number of negative consequences, most of them quite catastrophic. | ||
| 165 | |||
| 166 | The following picture shows how global sync allows for a mailbox style of | ||
| 167 | updates, i.e. it allows for multiple re-configurations between VUpdate | ||
| 168 | events where only the last configuration programmed before the VUpdate signal | ||
| 169 | becomes effective. | ||
| 170 | |||
| 171 | .. kernel-figure:: config_example.svg | ||
| @@ -0,0 +1,42 @@ | |||
| 1 | ====================== | ||
| 2 | AMDgpu Display Manager | ||
| 3 | ====================== | ||
| 4 | |||
| 5 | .. contents:: Table of Contents | ||
| 6 | :depth: 3 | ||
| 7 | |||
| 8 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | ||
| 9 | :doc: overview | ||
| 10 | |||
| 11 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h | ||
| 12 | :internal: | ||
| 13 | |||
| 14 | Lifecycle | ||
| 15 | ========= | ||
| 16 | |||
| 17 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | ||
| 18 | :doc: DM Lifecycle | ||
| 19 | |||
| 20 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | ||
| 21 | :functions: dm_hw_init dm_hw_fini | ||
| 22 | |||
| 23 | Interrupts | ||
| 24 | ========== | ||
| 25 | |||
| 26 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c | ||
| 27 | :doc: overview | ||
| 28 | |||
| 29 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c | ||
| 30 | :internal: | ||
| 31 | |||
| 32 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | ||
| 33 | :functions: register_hpd_handlers dm_crtc_high_irq dm_pflip_high_irq | ||
| 34 | |||
| 35 | Atomic Implementation | ||
| 36 | ===================== | ||
| 37 | |||
| 38 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | ||
| 39 | :doc: atomic | ||
| 40 | |||
| 41 | .. kernel-doc:: drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | ||
| 42 | :functions: amdgpu_dm_atomic_check amdgpu_dm_atomic_commit_tail | ||
| @@ -0,0 +1,485 @@ | |||
| 1 | <?xml version="1.0" encoding="UTF-8" standalone="no"?> | ||
| 2 | <!-- Created with Inkscape (http://www.inkscape.org/) --> | ||
| 3 | |||
| 4 | <svg | ||
| 5 | xmlns:dc="http://purl.org/dc/elements/1.1/" | ||
| 6 | xmlns:cc="http://creativecommons.org/ns#" | ||
| 7 | xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" | ||
| 8 | xmlns:svg="http://www.w3.org/2000/svg" | ||
| 9 | xmlns="http://www.w3.org/2000/svg" | ||
| 10 | xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" | ||
| 11 | xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" | ||
| 12 | width="232.24133mm" | ||
| 13 | height="96.174995mm" | ||
| 14 | viewBox="0 0 232.24133 96.174995" | ||
| 15 | version="1.1" | ||
| 16 | id="svg8" | ||
| 17 | inkscape:version="0.92.5 (2060ec1f9f, 2020-04-08)" | ||
| 18 | sodipodi:docname="global_sync_vblank.svg"> | ||
| 19 | <defs | ||
| 20 | id="defs2"> | ||
| 21 | <marker | ||
| 22 | inkscape:stockid="Arrow2Mend" | ||
| 23 | orient="auto" | ||
| 24 | refY="0" | ||
| 25 | refX="0" | ||
| 26 | id="Arrow2Mend" | ||
| 27 | style="overflow:visible" | ||
| 28 | inkscape:isstock="true"> | ||
| 29 | <path | ||
| 30 | id="path862" | ||
| 31 | style="fill:#800080;fill-opacity:1;fill-rule:evenodd;stroke:#800080;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 32 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 33 | transform="scale(-0.6)" | ||
| 34 | inkscape:connector-curvature="0" /> | ||
| 35 | </marker> | ||
| 36 | <marker | ||
| 37 | inkscape:stockid="Arrow2Send" | ||
| 38 | orient="auto" | ||
| 39 | refY="0" | ||
| 40 | refX="0" | ||
| 41 | id="Arrow2Send" | ||
| 42 | style="overflow:visible" | ||
| 43 | inkscape:isstock="true"> | ||
| 44 | <path | ||
| 45 | id="path868" | ||
| 46 | style="fill:#ff00ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff00ff;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 47 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 48 | transform="matrix(-0.3,0,0,-0.3,0.69,0)" | ||
| 49 | inkscape:connector-curvature="0" /> | ||
| 50 | </marker> | ||
| 51 | <marker | ||
| 52 | inkscape:stockid="Arrow2Lend" | ||
| 53 | orient="auto" | ||
| 54 | refY="0" | ||
| 55 | refX="0" | ||
| 56 | id="Arrow2Lend" | ||
| 57 | style="overflow:visible" | ||
| 58 | inkscape:isstock="true"> | ||
| 59 | <path | ||
| 60 | id="path856" | ||
| 61 | style="fill:#ff00ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff00ff;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 62 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 63 | transform="matrix(-1.1,0,0,-1.1,-1.1,0)" | ||
| 64 | inkscape:connector-curvature="0" /> | ||
| 65 | </marker> | ||
| 66 | <marker | ||
| 67 | inkscape:stockid="Arrow1Lend" | ||
| 68 | orient="auto" | ||
| 69 | refY="0" | ||
| 70 | refX="0" | ||
| 71 | id="marker1719" | ||
| 72 | style="overflow:visible" | ||
| 73 | inkscape:isstock="true"> | ||
| 74 | <path | ||
| 75 | id="path1717" | ||
| 76 | d="M 0,0 5,-5 -12.5,0 5,5 Z" | ||
| 77 | style="fill:#ff00ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff00ff;stroke-width:1.00000003pt;stroke-opacity:1" | ||
| 78 | transform="matrix(-0.8,0,0,-0.8,-10,0)" | ||
| 79 | inkscape:connector-curvature="0" /> | ||
| 80 | </marker> | ||
| 81 | <marker | ||
| 82 | inkscape:stockid="Arrow1Lend" | ||
| 83 | orient="auto" | ||
| 84 | refY="0" | ||
| 85 | refX="0" | ||
| 86 | id="marker1661" | ||
| 87 | style="overflow:visible" | ||
| 88 | inkscape:isstock="true"> | ||
| 89 | <path | ||
| 90 | id="path1659" | ||
| 91 | d="M 0,0 5,-5 -12.5,0 5,5 Z" | ||
| 92 | style="fill:#ff00ff;fill-opacity:1;fill-rule:evenodd;stroke:#ff00ff;stroke-width:1.00000003pt;stroke-opacity:1" | ||
| 93 | transform="matrix(-0.8,0,0,-0.8,-10,0)" | ||
| 94 | inkscape:connector-curvature="0" /> | ||
| 95 | </marker> | ||
| 96 | <marker | ||
| 97 | inkscape:isstock="true" | ||
| 98 | style="overflow:visible" | ||
| 99 | id="marker1311" | ||
| 100 | refX="0" | ||
| 101 | refY="0" | ||
| 102 | orient="auto" | ||
| 103 | inkscape:stockid="Arrow1Lend" | ||
| 104 | inkscape:collect="always"> | ||
| 105 | <path | ||
| 106 | transform="matrix(-0.8,0,0,-0.8,-10,0)" | ||
| 107 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1" | ||
| 108 | d="M 0,0 5,-5 -12.5,0 5,5 Z" | ||
| 109 | id="path1309" | ||
| 110 | inkscape:connector-curvature="0" /> | ||
| 111 | </marker> | ||
| 112 | <marker | ||
| 113 | inkscape:isstock="true" | ||
| 114 | style="overflow:visible" | ||
| 115 | id="marker1253" | ||
| 116 | refX="0" | ||
| 117 | refY="0" | ||
| 118 | orient="auto" | ||
| 119 | inkscape:stockid="Arrow1Lstart"> | ||
| 120 | <path | ||
| 121 | transform="matrix(0.8,0,0,0.8,10,0)" | ||
| 122 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1" | ||
| 123 | d="M 0,0 5,-5 -12.5,0 5,5 Z" | ||
| 124 | id="path1251" | ||
| 125 | inkscape:connector-curvature="0" /> | ||
| 126 | </marker> | ||
| 127 | <marker | ||
| 128 | inkscape:stockid="Arrow1Lend" | ||
| 129 | orient="auto" | ||
| 130 | refY="0" | ||
| 131 | refX="0" | ||
| 132 | id="Arrow1Lend" | ||
| 133 | style="overflow:visible" | ||
| 134 | inkscape:isstock="true" | ||
| 135 | inkscape:collect="always"> | ||
| 136 | <path | ||
| 137 | id="path838" | ||
| 138 | d="M 0,0 5,-5 -12.5,0 5,5 Z" | ||
| 139 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1" | ||
| 140 | transform="matrix(-0.8,0,0,-0.8,-10,0)" | ||
| 141 | inkscape:connector-curvature="0" /> | ||
| 142 | </marker> | ||
| 143 | <marker | ||
| 144 | inkscape:stockid="Arrow1Lstart" | ||
| 145 | orient="auto" | ||
| 146 | refY="0" | ||
| 147 | refX="0" | ||
| 148 | id="Arrow1Lstart" | ||
| 149 | style="overflow:visible" | ||
| 150 | inkscape:isstock="true" | ||
| 151 | inkscape:collect="always"> | ||
| 152 | <path | ||
| 153 | id="path835" | ||
| 154 | d="M 0,0 5,-5 -12.5,0 5,5 Z" | ||
| 155 | style="fill:#ff0000;fill-opacity:1;fill-rule:evenodd;stroke:#ff0000;stroke-width:1.00000003pt;stroke-opacity:1" | ||
| 156 | transform="matrix(0.8,0,0,0.8,10,0)" | ||
| 157 | inkscape:connector-curvature="0" /> | ||
| 158 | </marker> | ||
| 159 | <marker | ||
| 160 | inkscape:stockid="Arrow1Send" | ||
| 161 | orient="auto" | ||
| 162 | refY="0" | ||
| 163 | refX="0" | ||
| 164 | id="Arrow1Send" | ||
| 165 | style="overflow:visible" | ||
| 166 | inkscape:isstock="true"> | ||
| 167 | <path | ||
| 168 | id="path850" | ||
| 169 | d="M 0,0 5,-5 -12.5,0 5,5 Z" | ||
| 170 | style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1" | ||
| 171 | transform="matrix(-0.2,0,0,-0.2,-1.2,0)" | ||
| 172 | inkscape:connector-curvature="0" /> | ||
| 173 | </marker> | ||
| 174 | <marker | ||
| 175 | inkscape:stockid="Arrow2Sstart" | ||
| 176 | orient="auto" | ||
| 177 | refY="0" | ||
| 178 | refX="0" | ||
| 179 | id="Arrow2Sstart" | ||
| 180 | style="overflow:visible" | ||
| 181 | inkscape:isstock="true"> | ||
| 182 | <path | ||
| 183 | id="path865" | ||
| 184 | style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 185 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 186 | transform="matrix(0.3,0,0,0.3,-0.69,0)" | ||
| 187 | inkscape:connector-curvature="0" /> | ||
| 188 | </marker> | ||
| 189 | <marker | ||
| 190 | inkscape:stockid="Arrow2Mend" | ||
| 191 | orient="auto" | ||
| 192 | refY="0" | ||
| 193 | refX="0" | ||
| 194 | id="Arrow2Mend-2" | ||
| 195 | style="overflow:visible" | ||
| 196 | inkscape:isstock="true"> | ||
| 197 | <path | ||
| 198 | inkscape:connector-curvature="0" | ||
| 199 | id="path862-3" | ||
| 200 | style="fill:#800080;fill-opacity:1;fill-rule:evenodd;stroke:#800080;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 201 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 202 | transform="scale(-0.6)" /> | ||
| 203 | </marker> | ||
| 204 | <marker | ||
| 205 | inkscape:stockid="Arrow2Mend" | ||
| 206 | orient="auto" | ||
| 207 | refY="0" | ||
| 208 | refX="0" | ||
| 209 | id="Arrow2Mend-2-5" | ||
| 210 | style="overflow:visible" | ||
| 211 | inkscape:isstock="true"> | ||
| 212 | <path | ||
| 213 | inkscape:connector-curvature="0" | ||
| 214 | id="path862-3-9" | ||
| 215 | style="fill:#800080;fill-opacity:1;fill-rule:evenodd;stroke:#800080;stroke-width:0.625;stroke-linejoin:round;stroke-opacity:1" | ||
| 216 | d="M 8.7185878,4.0337352 -2.2072895,0.01601326 8.7185884,-4.0017078 c -1.7454984,2.3720609 -1.7354408,5.6174519 -6e-7,8.035443 z" | ||
| 217 | transform="scale(-0.6)" /> | ||
| 218 | </marker> | ||
| 219 | </defs> | ||
| 220 | <sodipodi:namedview | ||
| 221 | id="base" | ||
| 222 | pagecolor="#ffffff" | ||
| 223 | bordercolor="#666666" | ||
| 224 | borderopacity="1.0" | ||
| 225 | inkscape:pageopacity="0.0" | ||
| 226 | inkscape:pageshadow="2" | ||
| 227 | inkscape:zoom="1.979899" | ||
| 228 | inkscape:cx="747.52324" | ||
| 229 | inkscape:cy="319.84503" | ||
| 230 | inkscape:document-units="mm" | ||
| 231 | inkscape:current-layer="layer1" | ||
| 232 | showgrid="true" | ||
| 233 | inkscape:window-width="3840" | ||
| 234 | inkscape:window-height="2096" | ||
| 235 | inkscape:window-x="0" | ||
| 236 | inkscape:window-y="27" | ||
| 237 | inkscape:window-maximized="1" | ||
| 238 | fit-margin-top="0" | ||
| 239 | fit-margin-left="0" | ||
| 240 | fit-margin-right="0" | ||
| 241 | fit-margin-bottom="0"> | ||
| 242 | <inkscape:grid | ||
| 243 | type="xygrid" | ||
| 244 | id="grid815" | ||
| 245 | originx="15.282997" | ||
| 246 | originy="-184.54792" /> | ||
| 247 | </sodipodi:namedview> | ||
| 248 | <metadata | ||
| 249 | id="metadata5"> | ||
| 250 | <rdf:RDF> | ||
| 251 | <cc:Work | ||
| 252 | rdf:about=""> | ||
| 253 | <dc:format>image/svg+xml</dc:format> | ||
| 254 | <dc:type | ||
| 255 | rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> | ||
| 256 | <dc:title /> | ||
| 257 | </cc:Work> | ||
| 258 | </rdf:RDF> | ||
| 259 | </metadata> | ||
| 260 | <g | ||
| 261 | inkscape:label="Layer 1" | ||
| 262 | inkscape:groupmode="layer" | ||
| 263 | id="layer1" | ||
| 264 | transform="translate(15.282998,-16.277083)"> | ||
| 265 | <path | ||
| 266 | style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" | ||
| 267 | d="M 15.875,27.125001 V 16.541666 H 26.458333 V 27.125001 H 177.27084 V 16.541666 h 10.58333 v 10.583335 h 29.10416" | ||
| 268 | id="path817" | ||
| 269 | inkscape:connector-curvature="0" | ||
| 270 | sodipodi:nodetypes="ccccccccc" /> | ||
| 271 | <path | ||
| 272 | style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" | ||
| 273 | d="M 15.875,37.708334 H 44.979166 V 48.291667 H 100.54167 V 37.708334 H 206.375 v 10.583333 h 10.58333" | ||
| 274 | id="path819" | ||
| 275 | inkscape:connector-curvature="0" /> | ||
| 276 | <path | ||
| 277 | style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" | ||
| 278 | d="m 15.875,66.8125 h 97.89583 V 56.229167 h 7.9375 V 66.8125 h 92.60417" | ||
| 279 | id="path821" | ||
| 280 | inkscape:connector-curvature="0" /> | ||
| 281 | <path | ||
| 282 | style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" | ||
| 283 | d="m 15.875,85.333334 c 0,0 132.29166,0 132.29166,0 V 74.75 h 15.875 v 10.583334 h 47.625" | ||
| 284 | id="path823" | ||
| 285 | inkscape:connector-curvature="0" /> | ||
| 286 | <path | ||
| 287 | style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" | ||
| 288 | d="M 15.875,101.20833 H 187.85416 V 90.625 h 10.58334 v 10.58333 h 10.58333" | ||
| 289 | id="path825" | ||
| 290 | inkscape:connector-curvature="0" /> | ||
| 291 | <path | ||
| 292 | style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.52916667, 0.52916667;stroke-dashoffset:0;stroke-opacity:1" | ||
| 293 | d="M 100.54167,48.291667 V 111.79167" | ||
| 294 | id="path827" | ||
| 295 | inkscape:connector-curvature="0" /> | ||
| 296 | <path | ||
| 297 | style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.52916667, 0.52916667;stroke-dashoffset:0;stroke-opacity:1" | ||
| 298 | d="m 113.77083,66.8125 v 44.97917" | ||
| 299 | id="path829" | ||
| 300 | inkscape:connector-curvature="0" /> | ||
| 301 | <path | ||
| 302 | style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:0.52916667, 0.52916667;stroke-dashoffset:0;stroke-opacity:1" | ||
| 303 | d="M 206.375,48.291667 V 109.14583" | ||
| 304 | id="path831" | ||
| 305 | inkscape:connector-curvature="0" /> | ||
| 306 | <path | ||
| 307 | style="fill:none;stroke:#ff0000;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:url(#Arrow1Lend)" | ||
| 308 | d="m 100.54167,106.5 h 13.22916" | ||
| 309 | id="path833" | ||
| 310 | inkscape:connector-curvature="0" /> | ||
| 311 | <path | ||
| 312 | style="fill:none;stroke:#ff0000;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#marker1253);marker-end:url(#marker1311)" | ||
| 313 | d="M 113.77083,106.5 H 206.375" | ||
| 314 | id="path1243" | ||
| 315 | inkscape:connector-curvature="0" /> | ||
| 316 | <text | ||
| 317 | xml:space="preserve" | ||
| 318 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 319 | x="105.83333" | ||
| 320 | y="111.79166" | ||
| 321 | id="text1405"><tspan | ||
| 322 | sodipodi:role="line" | ||
| 323 | id="tspan1403" | ||
| 324 | x="105.83333" | ||
| 325 | y="111.79166" | ||
| 326 | style="stroke-width:0.26458332">To</tspan></text> | ||
| 327 | <text | ||
| 328 | xml:space="preserve" | ||
| 329 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 330 | x="145.52083" | ||
| 331 | y="111.79166" | ||
| 332 | id="text1409"><tspan | ||
| 333 | sodipodi:role="line" | ||
| 334 | id="tspan1407" | ||
| 335 | x="145.52083" | ||
| 336 | y="111.79166" | ||
| 337 | style="stroke-width:0.26458332">VStartup Period</tspan></text> | ||
| 338 | <text | ||
| 339 | xml:space="preserve" | ||
| 340 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 341 | x="156.01123" | ||
| 342 | y="78.71875" | ||
| 343 | id="text1413"><tspan | ||
| 344 | sodipodi:role="line" | ||
| 345 | x="156.01123" | ||
| 346 | y="78.71875" | ||
| 347 | style="font-weight:bold;text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 348 | id="tspan1415">VUpdate</tspan><tspan | ||
| 349 | sodipodi:role="line" | ||
| 350 | x="156.01123" | ||
| 351 | y="82.6875" | ||
| 352 | style="font-weight:bold;text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 353 | id="tspan1440">Width</tspan></text> | ||
| 354 | <text | ||
| 355 | xml:space="preserve" | ||
| 356 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 357 | x="173.77611" | ||
| 358 | y="92.703873" | ||
| 359 | id="text1413-3"><tspan | ||
| 360 | sodipodi:role="line" | ||
| 361 | id="tspan1411-6" | ||
| 362 | x="173.77611" | ||
| 363 | y="92.703873" | ||
| 364 | style="font-weight:bold;text-align:center;text-anchor:middle;stroke-width:0.26458332">VReady</tspan><tspan | ||
| 365 | sodipodi:role="line" | ||
| 366 | x="173.77611" | ||
| 367 | y="96.672623" | ||
| 368 | style="font-weight:bold;text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 369 | id="tspan1415-7">Offset</tspan></text> | ||
| 370 | <text | ||
| 371 | xml:space="preserve" | ||
| 372 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 373 | x="135.78951" | ||
| 374 | y="70.78125" | ||
| 375 | id="text1413-5"><tspan | ||
| 376 | sodipodi:role="line" | ||
| 377 | x="135.78951" | ||
| 378 | y="70.78125" | ||
| 379 | style="font-weight:bold;text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 380 | id="tspan1440-5">VUpdate</tspan><tspan | ||
| 381 | sodipodi:role="line" | ||
| 382 | x="135.78951" | ||
| 383 | y="74.75" | ||
| 384 | style="font-weight:bold;text-align:center;text-anchor:middle;stroke-width:0.26458332" | ||
| 385 | id="tspan1465">Offset</tspan></text> | ||
| 386 | <text | ||
| 387 | xml:space="preserve" | ||
| 388 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 389 | x="137.39433" | ||
| 390 | y="48.291664" | ||
| 391 | id="text1479"><tspan | ||
| 392 | sodipodi:role="line" | ||
| 393 | id="tspan1477" | ||
| 394 | x="137.39433" | ||
| 395 | y="48.291664" | ||
| 396 | style="font-weight:bold;stroke-width:0.26458332">VSTARTUP_START</tspan></text> | ||
| 397 | <text | ||
| 398 | xml:space="preserve" | ||
| 399 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 400 | x="-5.4806676" | ||
| 401 | y="22.778271" | ||
| 402 | id="text1479-1"><tspan | ||
| 403 | sodipodi:role="line" | ||
| 404 | id="tspan1477-2" | ||
| 405 | x="-5.4806676" | ||
| 406 | y="22.778271" | ||
| 407 | style="font-weight:bold;font-size:4.93888903px;stroke-width:0.26458332">VSYNC</tspan></text> | ||
| 408 | <text | ||
| 409 | xml:space="preserve" | ||
| 410 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 411 | x="-9.3767252" | ||
| 412 | y="45.64584" | ||
| 413 | id="text1479-1-7"><tspan | ||
| 414 | sodipodi:role="line" | ||
| 415 | id="tspan1477-2-0" | ||
| 416 | x="-9.3767252" | ||
| 417 | y="45.64584" | ||
| 418 | style="font-weight:bold;font-size:5.64444447px;stroke-width:0.26458332">VBlank</tspan></text> | ||
| 419 | <text | ||
| 420 | xml:space="preserve" | ||
| 421 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 422 | x="-15.310558" | ||
| 423 | y="64.92263" | ||
| 424 | id="text1479-1-7-9"><tspan | ||
| 425 | sodipodi:role="line" | ||
| 426 | id="tspan1477-2-0-3" | ||
| 427 | x="-15.310558" | ||
| 428 | y="64.92263" | ||
| 429 | style="font-weight:bold;font-size:5.64444447px;stroke-width:0.26458332">VStartup</tspan></text> | ||
| 430 | <text | ||
| 431 | xml:space="preserve" | ||
| 432 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 433 | x="-14.17781" | ||
| 434 | y="85.144356" | ||
| 435 | id="text1479-1-7-9-6"><tspan | ||
| 436 | sodipodi:role="line" | ||
| 437 | id="tspan1477-2-0-3-0" | ||
| 438 | x="-14.17781" | ||
| 439 | y="85.144356" | ||
| 440 | style="font-weight:bold;font-size:5.64444447px;stroke-width:0.26458332">VUpdate</tspan></text> | ||
| 441 | <text | ||
| 442 | xml:space="preserve" | ||
| 443 | style="font-style:normal;font-weight:normal;font-size:3.17499995px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" | ||
| 444 | x="-11.052421" | ||
| 445 | y="101.39733" | ||
| 446 | id="text1479-1-7-9-6-6"><tspan | ||
| 447 | sodipodi:role="line" | ||
| 448 | id="tspan1477-2-0-3-0-2" | ||
| 449 | x="-11.052421" | ||
| 450 | y="101.39733" | ||
| 451 | style="font-weight:bold;font-size:5.64444447px;stroke-width:0.26458332">VReady</tspan></text> | ||
| 452 | <g | ||
| 453 | id="g5189" | ||
| 454 | transform="translate(269.875,-14.287499)"> | ||
| 455 | <path | ||
| 456 | sodipodi:nodetypes="cc" | ||
| 457 | inkscape:connector-curvature="0" | ||
| 458 | id="path5143" | ||
| 459 | d="m -202.40625,45.645828 3.96875,-7.9375" | ||
| 460 | style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" /> | ||
| 461 | <path | ||
| 462 | sodipodi:nodetypes="cc" | ||
| 463 | inkscape:connector-curvature="0" | ||
| 464 | id="path5143-2" | ||
| 465 | d="m -199.76042,45.645828 3.96874,-7.937499" | ||
| 466 | style="fill:none;stroke:#000000;stroke-width:0.52916676;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" /> | ||
| 467 | </g> | ||
| 468 | <g | ||
| 469 | id="g5189-3" | ||
| 470 | transform="translate(268.55209,7.9375003)"> | ||
| 471 | <path | ||
| 472 | sodipodi:nodetypes="cc" | ||
| 473 | inkscape:connector-curvature="0" | ||
| 474 | id="path5143-6" | ||
| 475 | d="m -202.40625,45.645828 3.96875,-7.9375" | ||
| 476 | style="fill:none;stroke:#000000;stroke-width:0.5291667;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" /> | ||
| 477 | <path | ||
| 478 | sodipodi:nodetypes="cc" | ||
| 479 | inkscape:connector-curvature="0" | ||
| 480 | id="path5143-2-1" | ||
| 481 | d="m -199.76042,45.645828 3.96874,-7.937499" | ||
| 482 | style="fill:none;stroke:#000000;stroke-width:0.52916676;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" /> | ||
| 483 | </g> | ||
| 484 | </g> | ||
| 485 | </svg> | ||
| @@ -0,0 +1,31 @@ | |||
| 1 | .. _amdgpu-display-core: | ||
| 2 | |||
| 3 | =================================== | ||
| 4 | drm/amd/display - Display Core (DC) | ||
| 5 | =================================== | ||
| 6 | |||
| 7 | AMD display engine is partially shared with other operating systems; for this | ||
| 8 | reason, our Display Core Driver is divided into two pieces: | ||
| 9 | |||
| 10 | 1. **Display Core (DC)** contains the OS-agnostic components. Things like | ||
| 11 | hardware programming and resource management are handled here. | ||
| 12 | 2. **Display Manager (DM)** contains the OS-dependent components. Hooks to the | ||
| 13 | amdgpu base driver and DRM are implemented here. | ||
| 14 | |||
| 15 | The display pipe is responsible for "scanning out" a rendered frame from the | ||
| 16 | GPU memory (also called VRAM, FrameBuffer, etc.) to a display. In other words, | ||
| 17 | it would: | ||
| 18 | |||
| 19 | 1. Read frame information from memory; | ||
| 20 | 2. Perform required transformation; | ||
| 21 | 3. Send pixel data to sink devices. | ||
| 22 | |||
| 23 | If you want to learn more about our driver details, take a look at the below | ||
| 24 | table of content: | ||
| 25 | |||
| 26 | .. toctree:: | ||
| 27 | |||
| 28 | display-manager.rst | ||
| 29 | dc-debug.rst | ||
| 30 | dcn-overview.rst | ||
| 31 | dc-glossary.rst | ||
| @@ -0,0 +1,182 @@ | |||
| 1 | ============================ | ||
| 2 | Core Driver Infrastructure | ||
| 3 | ============================ | ||
| 4 | |||
| 5 | GPU Hardware Structure | ||
| 6 | ====================== | ||
| 7 | |||
| 8 | Each ASIC is a collection of hardware blocks. We refer to them as | ||
| 9 | "IPs" (Intellectual Property blocks). Each IP encapsulates certain | ||
| 10 | functionality. IPs are versioned and can also be mixed and matched. | ||
| 11 | E.g., you might have two different ASICs that both have System DMA (SDMA) 5.x IPs. | ||
| 12 | The driver is arranged by IPs. There are driver components to handle | ||
| 13 | the initialization and operation of each IP. There are also a bunch | ||
| 14 | of smaller IPs that don't really need much if any driver interaction. | ||
| 15 | Those end up getting lumped into the common stuff in the soc files. | ||
| 16 | The soc files (e.g., vi.c, soc15.c nv.c) contain code for aspects of | ||
| 17 | the SoC itself rather than specific IPs. E.g., things like GPU resets | ||
| 18 | and register access functions are SoC dependent. | ||
| 19 | |||
| 20 | An APU contains more than just CPU and GPU, it also contains all of | ||
| 21 | the platform stuff (audio, usb, gpio, etc.). Also, a lot of | ||
| 22 | components are shared between the CPU, platform, and the GPU (e.g., | ||
| 23 | SMU, PSP, etc.). Specific components (CPU, GPU, etc.) usually have | ||
| 24 | their interface to interact with those common components. For things | ||
| 25 | like S0i3 there is a ton of coordination required across all the | ||
| 26 | components, but that is probably a bit beyond the scope of this | ||
| 27 | section. | ||
| 28 | |||
| 29 | With respect to the GPU, we have the following major IPs: | ||
| 30 | |||
| 31 | GMC (Graphics Memory Controller) | ||
| 32 | This was a dedicated IP on older pre-vega chips, but has since | ||
| 33 | become somewhat decentralized on vega and newer chips. They now | ||
| 34 | have dedicated memory hubs for specific IPs or groups of IPs. We | ||
| 35 | still treat it as a single component in the driver however since | ||
| 36 | the programming model is still pretty similar. This is how the | ||
| 37 | different IPs on the GPU get the memory (VRAM or system memory). | ||
| 38 | It also provides the support for per process GPU virtual address | ||
| 39 | spaces. | ||
| 40 | |||
| 41 | IH (Interrupt Handler) | ||
| 42 | This is the interrupt controller on the GPU. All of the IPs feed | ||
| 43 | their interrupts into this IP and it aggregates them into a set of | ||
| 44 | ring buffers that the driver can parse to handle interrupts from | ||
| 45 | different IPs. | ||
| 46 | |||
| 47 | PSP (Platform Security Processor) | ||
| 48 | This handles security policy for the SoC and executes trusted | ||
| 49 | applications, and validates and loads firmwares for other blocks. | ||
| 50 | |||
| 51 | SMU (System Management Unit) | ||
| 52 | This is the power management microcontroller. It manages the entire | ||
| 53 | SoC. The driver interacts with it to control power management | ||
| 54 | features like clocks, voltages, power rails, etc. | ||
| 55 | |||
| 56 | DCN (Display Controller Next) | ||
| 57 | This is the display controller. It handles the display hardware. | ||
| 58 | It is described in more details in :ref:`Display Core <amdgpu-display-core>`. | ||
| 59 | |||
| 60 | SDMA (System DMA) | ||
| 61 | This is a multi-purpose DMA engine. The kernel driver uses it for | ||
| 62 | various things including paging and GPU page table updates. It's also | ||
| 63 | exposed to userspace for use by user mode drivers (OpenGL, Vulkan, | ||
| 64 | etc.) | ||
| 65 | |||
| 66 | GC (Graphics and Compute) | ||
| 67 | This is the graphics and compute engine, i.e., the block that | ||
| 68 | encompasses the 3D pipeline and and shader blocks. This is by far the | ||
| 69 | largest block on the GPU. The 3D pipeline has tons of sub-blocks. In | ||
| 70 | addition to that, it also contains the CP microcontrollers (ME, PFP, | ||
| 71 | CE, MEC) and the RLC microcontroller. It's exposed to userspace for | ||
| 72 | user mode drivers (OpenGL, Vulkan, OpenCL, etc.) | ||
| 73 | |||
| 74 | VCN (Video Core Next) | ||
| 75 | This is the multi-media engine. It handles video and image encode and | ||
| 76 | decode. It's exposed to userspace for user mode drivers (VA-API, | ||
| 77 | OpenMAX, etc.) | ||
| 78 | |||
| 79 | Graphics and Compute Microcontrollers | ||
| 80 | ------------------------------------- | ||
| 81 | |||
| 82 | CP (Command Processor) | ||
| 83 | The name for the hardware block that encompasses the front end of the | ||
| 84 | GFX/Compute pipeline. Consists mainly of a bunch of microcontrollers | ||
| 85 | (PFP, ME, CE, MEC). The firmware that runs on these microcontrollers | ||
| 86 | provides the driver interface to interact with the GFX/Compute engine. | ||
| 87 | |||
| 88 | MEC (MicroEngine Compute) | ||
| 89 | This is the microcontroller that controls the compute queues on the | ||
| 90 | GFX/compute engine. | ||
| 91 | |||
| 92 | MES (MicroEngine Scheduler) | ||
| 93 | This is a new engine for managing queues. This is currently unused. | ||
| 94 | |||
| 95 | RLC (RunList Controller) | ||
| 96 | This is another microcontroller in the GFX/Compute engine. It handles | ||
| 97 | power management related functionality within the GFX/Compute engine. | ||
| 98 | The name is a vestige of old hardware where it was originally added | ||
| 99 | and doesn't really have much relation to what the engine does now. | ||
| 100 | |||
| 101 | Driver Structure | ||
| 102 | ================ | ||
| 103 | |||
| 104 | In general, the driver has a list of all of the IPs on a particular | ||
| 105 | SoC and for things like init/fini/suspend/resume, more or less just | ||
| 106 | walks the list and handles each IP. | ||
| 107 | |||
| 108 | Some useful constructs: | ||
| 109 | |||
| 110 | KIQ (Kernel Interface Queue) | ||
| 111 | This is a control queue used by the kernel driver to manage other gfx | ||
| 112 | and compute queues on the GFX/compute engine. You can use it to | ||
| 113 | map/unmap additional queues, etc. | ||
| 114 | |||
| 115 | IB (Indirect Buffer) | ||
| 116 | A command buffer for a particular engine. Rather than writing | ||
| 117 | commands directly to the queue, you can write the commands into a | ||
| 118 | piece of memory and then put a pointer to the memory into the queue. | ||
| 119 | The hardware will then follow the pointer and execute the commands in | ||
| 120 | the memory, then returning to the rest of the commands in the ring. | ||
| 121 | |||
| 122 | .. _amdgpu_memory_domains: | ||
| 123 | |||
| 124 | Memory Domains | ||
| 125 | ============== | ||
| 126 | |||
| 127 | .. kernel-doc:: include/uapi/drm/amdgpu_drm.h | ||
| 128 | :doc: memory domains | ||
| 129 | |||
| 130 | Buffer Objects | ||
| 131 | ============== | ||
| 132 | |||
| 133 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | ||
| 134 | :doc: amdgpu_object | ||
| 135 | |||
| 136 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | ||
| 137 | :internal: | ||
| 138 | |||
| 139 | PRIME Buffer Sharing | ||
| 140 | ==================== | ||
| 141 | |||
| 142 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | ||
| 143 | :doc: PRIME Buffer Sharing | ||
| 144 | |||
| 145 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | ||
| 146 | :internal: | ||
| 147 | |||
| 148 | MMU Notifier | ||
| 149 | ============ | ||
| 150 | |||
| 151 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | ||
| 152 | :doc: MMU Notifier | ||
| 153 | |||
| 154 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | ||
| 155 | :internal: | ||
| 156 | |||
| 157 | AMDGPU Virtual Memory | ||
| 158 | ===================== | ||
| 159 | |||
| 160 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | ||
| 161 | :doc: GPUVM | ||
| 162 | |||
| 163 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | ||
| 164 | :internal: | ||
| 165 | |||
| 166 | Interrupt Handling | ||
| 167 | ================== | ||
| 168 | |||
| 169 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | ||
| 170 | :doc: Interrupt Handling | ||
| 171 | |||
| 172 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | ||
| 173 | :internal: | ||
| 174 | |||
| 175 | IP Blocks | ||
| 176 | ========= | ||
| 177 | |||
| 178 | .. kernel-doc:: drivers/gpu/drm/amd/include/amd_shared.h | ||
| 179 | :doc: IP Blocks | ||
| 180 | |||
| 181 | .. kernel-doc:: drivers/gpu/drm/amd/include/amd_shared.h | ||
| 182 | :identifiers: amd_ip_block_type amd_ip_funcs | ||
| @@ -0,0 +1,112 @@ | |||
| 1 | ================================ | ||
| 2 | Misc AMDGPU driver information | ||
| 3 | ================================ | ||
| 4 | |||
| 5 | GPU Product Information | ||
| 6 | ======================= | ||
| 7 | |||
| 8 | Information about the GPU can be obtained on certain cards | ||
| 9 | via sysfs | ||
| 10 | |||
| 11 | product_name | ||
| 12 | ------------ | ||
| 13 | |||
| 14 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | ||
| 15 | :doc: product_name | ||
| 16 | |||
| 17 | product_number | ||
| 18 | -------------- | ||
| 19 | |||
| 20 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | ||
| 21 | :doc: product_name | ||
| 22 | |||
| 23 | serial_number | ||
| 24 | ------------- | ||
| 25 | |||
| 26 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | ||
| 27 | :doc: serial_number | ||
| 28 | |||
| 29 | unique_id | ||
| 30 | --------- | ||
| 31 | |||
| 32 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 33 | :doc: unique_id | ||
| 34 | |||
| 35 | GPU Memory Usage Information | ||
| 36 | ============================ | ||
| 37 | |||
| 38 | Various memory accounting can be accessed via sysfs | ||
| 39 | |||
| 40 | mem_info_vram_total | ||
| 41 | ------------------- | ||
| 42 | |||
| 43 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | ||
| 44 | :doc: mem_info_vram_total | ||
| 45 | |||
| 46 | mem_info_vram_used | ||
| 47 | ------------------ | ||
| 48 | |||
| 49 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | ||
| 50 | :doc: mem_info_vram_used | ||
| 51 | |||
| 52 | mem_info_vis_vram_total | ||
| 53 | ----------------------- | ||
| 54 | |||
| 55 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | ||
| 56 | :doc: mem_info_vis_vram_total | ||
| 57 | |||
| 58 | mem_info_vis_vram_used | ||
| 59 | ---------------------- | ||
| 60 | |||
| 61 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | ||
| 62 | :doc: mem_info_vis_vram_used | ||
| 63 | |||
| 64 | mem_info_gtt_total | ||
| 65 | ------------------ | ||
| 66 | |||
| 67 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | ||
| 68 | :doc: mem_info_gtt_total | ||
| 69 | |||
| 70 | mem_info_gtt_used | ||
| 71 | ----------------- | ||
| 72 | |||
| 73 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | ||
| 74 | :doc: mem_info_gtt_used | ||
| 75 | |||
| 76 | PCIe Accounting Information | ||
| 77 | =========================== | ||
| 78 | |||
| 79 | pcie_bw | ||
| 80 | ------- | ||
| 81 | |||
| 82 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 83 | :doc: pcie_bw | ||
| 84 | |||
| 85 | pcie_replay_count | ||
| 86 | ----------------- | ||
| 87 | |||
| 88 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | ||
| 89 | :doc: pcie_replay_count | ||
| 90 | |||
| 91 | GPU SmartShift Information | ||
| 92 | ========================== | ||
| 93 | |||
| 94 | GPU SmartShift information via sysfs | ||
| 95 | |||
| 96 | smartshift_apu_power | ||
| 97 | -------------------- | ||
| 98 | |||
| 99 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 100 | :doc: smartshift_apu_power | ||
| 101 | |||
| 102 | smartshift_dgpu_power | ||
| 103 | --------------------- | ||
| 104 | |||
| 105 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 106 | :doc: smartshift_dgpu_power | ||
| 107 | |||
| 108 | smartshift_bias | ||
| 109 | --------------- | ||
| 110 | |||
| 111 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 112 | :doc: smartshift_bias | ||
| @@ -0,0 +1,17 @@ | |||
| 1 | ========================== | ||
| 2 | drm/amdgpu AMDgpu driver | ||
| 3 | ========================== | ||
| 4 | |||
| 5 | The drm/amdgpu driver supports all AMD Radeon GPUs based on the Graphics Core | ||
| 6 | Next (GCN) architecture. | ||
| 7 | |||
| 8 | .. toctree:: | ||
| 9 | |||
| 10 | module-parameters | ||
| 11 | driver-core | ||
| 12 | display/index | ||
| 13 | xgmi | ||
| 14 | ras | ||
| 15 | thermal | ||
| 16 | driver-misc | ||
| 17 | amdgpu-glossary | ||
| @@ -0,0 +1,7 @@ | |||
| 1 | =================== | ||
| 2 | Module Parameters | ||
| 3 | =================== | ||
| 4 | |||
| 5 | The amdgpu driver supports the following module parameters: | ||
| 6 | |||
| 7 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | ||
| @@ -0,0 +1,62 @@ | |||
| 1 | ==================== | ||
| 2 | AMDGPU RAS Support | ||
| 3 | ==================== | ||
| 4 | |||
| 5 | The AMDGPU RAS interfaces are exposed via sysfs (for informational queries) and | ||
| 6 | debugfs (for error injection). | ||
| 7 | |||
| 8 | RAS debugfs/sysfs Control and Error Injection Interfaces | ||
| 9 | ======================================================== | ||
| 10 | |||
| 11 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | ||
| 12 | :doc: AMDGPU RAS debugfs control interface | ||
| 13 | |||
| 14 | RAS Reboot Behavior for Unrecoverable Errors | ||
| 15 | ============================================ | ||
| 16 | |||
| 17 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | ||
| 18 | :doc: AMDGPU RAS Reboot Behavior for Unrecoverable Errors | ||
| 19 | |||
| 20 | RAS Error Count sysfs Interface | ||
| 21 | =============================== | ||
| 22 | |||
| 23 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | ||
| 24 | :doc: AMDGPU RAS sysfs Error Count Interface | ||
| 25 | |||
| 26 | RAS EEPROM debugfs Interface | ||
| 27 | ============================ | ||
| 28 | |||
| 29 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | ||
| 30 | :doc: AMDGPU RAS debugfs EEPROM table reset interface | ||
| 31 | |||
| 32 | RAS VRAM Bad Pages sysfs Interface | ||
| 33 | ================================== | ||
| 34 | |||
| 35 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | ||
| 36 | :doc: AMDGPU RAS sysfs gpu_vram_bad_pages Interface | ||
| 37 | |||
| 38 | Sample Code | ||
| 39 | =========== | ||
| 40 | Sample code for testing error injection can be found here: | ||
| 41 | https://cgit.freedesktop.org/mesa/drm/tree/tests/amdgpu/ras_tests.c | ||
| 42 | |||
| 43 | This is part of the libdrm amdgpu unit tests which cover several areas of the GPU. | ||
| 44 | There are four sets of tests: | ||
| 45 | |||
| 46 | RAS Basic Test | ||
| 47 | |||
| 48 | The test verifies the RAS feature enabled status and makes sure the necessary sysfs and debugfs files | ||
| 49 | are present. | ||
| 50 | |||
| 51 | RAS Query Test | ||
| 52 | |||
| 53 | This test checks the RAS availability and enablement status for each supported IP block as well as | ||
| 54 | the error counts. | ||
| 55 | |||
| 56 | RAS Inject Test | ||
| 57 | |||
| 58 | This test injects errors for each IP. | ||
| 59 | |||
| 60 | RAS Disable Test | ||
| 61 | |||
| 62 | This test tests disabling of RAS features for each IP block. | ||
| @@ -0,0 +1,65 @@ | |||
| 1 | =========================================== | ||
| 2 | GPU Power/Thermal Controls and Monitoring | ||
| 3 | =========================================== | ||
| 4 | |||
| 5 | HWMON Interfaces | ||
| 6 | ================ | ||
| 7 | |||
| 8 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 9 | :doc: hwmon | ||
| 10 | |||
| 11 | GPU sysfs Power State Interfaces | ||
| 12 | ================================ | ||
| 13 | |||
| 14 | GPU power controls are exposed via sysfs files. | ||
| 15 | |||
| 16 | power_dpm_state | ||
| 17 | --------------- | ||
| 18 | |||
| 19 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 20 | :doc: power_dpm_state | ||
| 21 | |||
| 22 | power_dpm_force_performance_level | ||
| 23 | --------------------------------- | ||
| 24 | |||
| 25 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 26 | :doc: power_dpm_force_performance_level | ||
| 27 | |||
| 28 | pp_table | ||
| 29 | -------- | ||
| 30 | |||
| 31 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 32 | :doc: pp_table | ||
| 33 | |||
| 34 | pp_od_clk_voltage | ||
| 35 | ----------------- | ||
| 36 | |||
| 37 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 38 | :doc: pp_od_clk_voltage | ||
| 39 | |||
| 40 | pp_dpm_* | ||
| 41 | -------- | ||
| 42 | |||
| 43 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 44 | :doc: pp_dpm_sclk pp_dpm_mclk pp_dpm_socclk pp_dpm_fclk pp_dpm_dcefclk pp_dpm_pcie | ||
| 45 | |||
| 46 | pp_power_profile_mode | ||
| 47 | --------------------- | ||
| 48 | |||
| 49 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 50 | :doc: pp_power_profile_mode | ||
| 51 | |||
| 52 | \*_busy_percent | ||
| 53 | --------------- | ||
| 54 | |||
| 55 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 56 | :doc: gpu_busy_percent | ||
| 57 | |||
| 58 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 59 | :doc: mem_busy_percent | ||
| 60 | |||
| 61 | gpu_metrics | ||
| 62 | ----------- | ||
| 63 | |||
| 64 | .. kernel-doc:: drivers/gpu/drm/amd/pm/amdgpu_pm.c | ||
| 65 | :doc: gpu_metrics | ||
| @@ -0,0 +1,5 @@ | |||
| 1 | ===================== | ||
| 2 | AMDGPU XGMI Support | ||
| 3 | ===================== | ||
| 4 | |||
| 5 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c | ||
| @@ -4,8 +4,7 @@ GPU Driver Documentation · 3 unmodified lines | |||
| 4 | 4 | ||
| 5 | .. toctree:: | 5 | .. toctree:: |
| 6 | 6 | ||
| 7 | amdgpu | 7 | amdgpu/index |
| 8 | amdgpu-dc | ||
| 9 | i915 | 8 | i915 |
| 10 | mcde | 9 | mcde |
| 11 | meson | 10 | meson |
| @@ -435,3 +435,18 @@ Legacy CRTC/Modeset Helper Functions Reference · 434 unmodified lines | |||
| 435 | 435 | ||
| 436 | .. kernel-doc:: drivers/gpu/drm/drm_crtc_helper.c | 436 | .. kernel-doc:: drivers/gpu/drm/drm_crtc_helper.c |
| 437 | :export: | 437 | :export: |
| 438 | |||
| 439 | Privacy-screen class | ||
| 440 | ==================== | ||
| 441 | |||
| 442 | .. kernel-doc:: drivers/gpu/drm/drm_privacy_screen.c | ||
| 443 | :doc: overview | ||
| 444 | |||
| 445 | .. kernel-doc:: include/drm/drm_privacy_screen_driver.h | ||
| 446 | :internal: | ||
| 447 | |||
| 448 | .. kernel-doc:: include/drm/drm_privacy_screen_machine.h | ||
| 449 | :internal: | ||
| 450 | |||
| 451 | .. kernel-doc:: drivers/gpu/drm/drm_privacy_screen.c | ||
| 452 | :export: | ||
| @@ -506,6 +506,8 @@ Property Types and Blob Property Support · 505 unmodified lines | |||
| 506 | .. kernel-doc:: drivers/gpu/drm/drm_property.c | 506 | .. kernel-doc:: drivers/gpu/drm/drm_property.c |
| 507 | :export: | 507 | :export: |
| 508 | 508 | ||
| 509 | .. _standard_connector_properties: | ||
| 510 | |||
| 509 | Standard Connector Properties | 511 | Standard Connector Properties |
| 510 | ----------------------------- | 512 | ----------------------------- |
| 511 | 513 | ||
| @@ -268,17 +268,6 @@ Contact: Daniel Vetter · 267 unmodified lines | |||
| 268 | 268 | ||
| 269 | Level: Intermediate | 269 | Level: Intermediate |
| 270 | 270 | ||
| 271 | Clean up mmap forwarding | ||
| 272 | ------------------------ | ||
| 273 | |||
| 274 | A lot of drivers forward gem mmap calls to dma-buf mmap for imported buffers. | ||
| 275 | And also a lot of them forward dma-buf mmap to the gem mmap implementations. | ||
| 276 | There's drm_gem_prime_mmap() for this now, but still needs to be rolled out. | ||
| 277 | |||
| 278 | Contact: Daniel Vetter | ||
| 279 | |||
| 280 | Level: Intermediate | ||
| 281 | |||
| 282 | Generic fbdev defio support | 271 | Generic fbdev defio support |
| 283 | --------------------------- | 272 | --------------------------- |
| 284 | 273 | ||
| @@ -463,6 +452,21 @@ Contact: Thomas Zimmermann <tzimmermann@suse.de>, Christian König, Daniel Vette · 178 unmodified lines | |||
| 463 | 452 | ||
| 464 | Level: Intermediate | 453 | Level: Intermediate |
| 465 | 454 | ||
| 455 | Review all drivers for setting struct drm_mode_config.{max_width,max_height} correctly | ||
| 456 | -------------------------------------------------------------------------------------- | ||
| 457 | |||
| 458 | The values in struct drm_mode_config.{max_width,max_height} describe the | ||
| 459 | maximum supported framebuffer size. It's the virtual screen size, but many | ||
| 460 | drivers treat it like limitations of the physical resolution. | ||
| 461 | |||
| 462 | The maximum width depends on the hardware's maximum scanline pitch. The | ||
| 463 | maximum height depends on the amount of addressable video memory. Review all | ||
| 464 | drivers to initialize the fields to the correct values. | ||
| 465 | |||
| 466 | Contact: Thomas Zimmermann <tzimmermann@suse.de> | ||
| 467 | |||
| 468 | Level: Intermediate | ||
| 469 | |||
| 466 | 470 | ||
| 467 | Core refactorings | 471 | Core refactorings |
| 468 | ================= | 472 | ================= |
| @@ -642,6 +646,17 @@ See drivers/gpu/drm/amd/display/TODO for tasks. · 173 unmodified lines | |||
| 642 | 646 | ||
| 643 | Contact: Harry Wentland, Alex Deucher | 647 | Contact: Harry Wentland, Alex Deucher |
| 644 | 648 | ||
| 649 | vmwgfx: Replace hashtable with Linux' implementation | ||
| 650 | ---------------------------------------------------- | ||
| 651 | |||
| 652 | The vmwgfx driver uses its own hashtable implementation. Replace the | ||
| 653 | code with Linux' implementation and update the callers. It's mostly a | ||
| 654 | refactoring task, but the interfaces are different. | ||
| 655 | |||
| 656 | Contact: Zack Rusin, Thomas Zimmermann <tzimmermann@suse.de> | ||
| 657 | |||
| 658 | Level: Intermediate | ||
| 659 | |||
| 645 | Bootsplash | 660 | Bootsplash |
| 646 | ========== | 661 | ========== |
| 647 | 662 | ||
| @@ -966,6 +966,7 @@ F: drivers/gpu/drm/amd/include/kgd_kfd_interface.h · 965 unmodified lines | |||
| 966 | F: drivers/gpu/drm/amd/include/v9_structs.h | 966 | F: drivers/gpu/drm/amd/include/v9_structs.h |
| 967 | F: drivers/gpu/drm/amd/include/vi_structs.h | 967 | F: drivers/gpu/drm/amd/include/vi_structs.h |
| 968 | F: include/uapi/linux/kfd_ioctl.h | 968 | F: include/uapi/linux/kfd_ioctl.h |
| 969 | F: include/uapi/linux/kfd_sysfs.h | ||
| 969 | 970 | ||
| 970 | AMD SPI DRIVER | 971 | AMD SPI DRIVER |
| 971 | M: Sanjay R Mehta <sanju.mehta@amd.com> | 972 | M: Sanjay R Mehta <sanju.mehta@amd.com> |
| @@ -6057,6 +6058,7 @@ F: drivers/gpu/drm/tiny/mi0283qt.c · 5085 unmodified lines | |||
| 6057 | DRM DRIVER FOR MSM ADRENO GPU | 6058 | DRM DRIVER FOR MSM ADRENO GPU |
| 6058 | M: Rob Clark <robdclark@gmail.com> | 6059 | M: Rob Clark <robdclark@gmail.com> |
| 6059 | M: Sean Paul <sean@poorly.run> | 6060 | M: Sean Paul <sean@poorly.run> |
| 6061 | R: Abhinav Kumar <quic_abhinavk@quicinc.com> | ||
| 6060 | L: linux-arm-msm@vger.kernel.org | 6062 | L: linux-arm-msm@vger.kernel.org |
| 6061 | L: dri-devel@lists.freedesktop.org | 6063 | L: dri-devel@lists.freedesktop.org |
| 6062 | L: freedreno@lists.freedesktop.org | 6064 | L: freedreno@lists.freedesktop.org |
| @@ -6082,10 +6084,17 @@ F: drivers/gpu/drm/panel/panel-novatek-nt36672a.c · 19 unmodified lines | |||
| 6082 | 6084 | ||
| 6083 | DRM DRIVER FOR NVIDIA GEFORCE/QUADRO GPUS | 6085 | DRM DRIVER FOR NVIDIA GEFORCE/QUADRO GPUS |
| 6084 | M: Ben Skeggs <bskeggs@redhat.com> | 6086 | M: Ben Skeggs <bskeggs@redhat.com> |
| 6087 | M: Karol Herbst <kherbst@redhat.com> | ||
| 6088 | M: Lyude Paul <lyude@redhat.com> | ||
| 6085 | L: dri-devel@lists.freedesktop.org | 6089 | L: dri-devel@lists.freedesktop.org |
| 6086 | L: nouveau@lists.freedesktop.org | 6090 | L: nouveau@lists.freedesktop.org |
| 6087 | S: Supported | 6091 | S: Supported |
| 6088 | T: git git://github.com/skeggsb/linux | 6092 | W: https://nouveau.freedesktop.org/ |
| 6093 | Q: https://patchwork.freedesktop.org/project/nouveau/ | ||
| 6094 | Q: https://gitlab.freedesktop.org/drm/nouveau/-/merge_requests | ||
| 6095 | B: https://gitlab.freedesktop.org/drm/nouveau/-/issues | ||
| 6096 | C: irc://irc.oftc.net/nouveau | ||
| 6097 | T: git https://gitlab.freedesktop.org/drm/nouveau.git | ||
| 6089 | F: drivers/gpu/drm/nouveau/ | 6098 | F: drivers/gpu/drm/nouveau/ |
| 6090 | F: include/uapi/drm/nouveau_drm.h | 6099 | F: include/uapi/drm/nouveau_drm.h |
| 6091 | 6100 | ||
| @@ -6425,6 +6434,7 @@ L: dri-devel@lists.freedesktop.org · 333 unmodified lines | |||
| 6425 | L: linux-renesas-soc@vger.kernel.org | 6434 | L: linux-renesas-soc@vger.kernel.org |
| 6426 | S: Supported | 6435 | S: Supported |
| 6427 | T: git git://linuxtv.org/pinchartl/media drm/du/next | 6436 | T: git git://linuxtv.org/pinchartl/media drm/du/next |
| 6437 | F: Documentation/devicetree/bindings/display/bridge/renesas,dsi-csi2-tx.yaml | ||
| 6428 | F: Documentation/devicetree/bindings/display/bridge/renesas,dw-hdmi.yaml | 6438 | F: Documentation/devicetree/bindings/display/bridge/renesas,dw-hdmi.yaml |
| 6429 | F: Documentation/devicetree/bindings/display/bridge/renesas,lvds.yaml | 6439 | F: Documentation/devicetree/bindings/display/bridge/renesas,lvds.yaml |
| 6430 | F: Documentation/devicetree/bindings/display/renesas,du.yaml | 6440 | F: Documentation/devicetree/bindings/display/renesas,du.yaml |
| @@ -6543,6 +6553,14 @@ F: drivers/gpu/drm/drm_panel.c · 112 unmodified lines | |||
| 6543 | F: drivers/gpu/drm/panel/ | 6553 | F: drivers/gpu/drm/panel/ |
| 6544 | F: include/drm/drm_panel.h | 6554 | F: include/drm/drm_panel.h |
| 6545 | 6555 | ||
| 6556 | DRM PRIVACY-SCREEN CLASS | ||
| 6557 | M: Hans de Goede <hdegoede@redhat.com> | ||
| 6558 | L: dri-devel@lists.freedesktop.org | ||
| 6559 | S: Maintained | ||
| 6560 | T: git git://anongit.freedesktop.org/drm/drm-misc | ||
| 6561 | F: drivers/gpu/drm/drm_privacy_screen* | ||
| 6562 | F: include/drm/drm_privacy_screen* | ||
| 6563 | |||
| 6546 | DRM TTM SUBSYSTEM | 6564 | DRM TTM SUBSYSTEM |
| 6547 | M: Christian Koenig <christian.koenig@amd.com> | 6565 | M: Christian Koenig <christian.koenig@amd.com> |
| 6548 | M: Huang Rui <ray.huang@amd.com> | 6566 | M: Huang Rui <ray.huang@amd.com> |
| @@ -9499,6 +9517,7 @@ INTEL DRM DRIVERS (excluding Poulsbo, Moorestown and derivative chipsets) · 2950 unmodified lines | |||
| 9499 | M: Jani Nikula <jani.nikula@linux.intel.com> | 9517 | M: Jani Nikula <jani.nikula@linux.intel.com> |
| 9500 | M: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> | 9518 | M: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> |
| 9501 | M: Rodrigo Vivi <rodrigo.vivi@intel.com> | 9519 | M: Rodrigo Vivi <rodrigo.vivi@intel.com> |
| 9520 | M: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> | ||
| 9502 | L: intel-gfx@lists.freedesktop.org | 9521 | L: intel-gfx@lists.freedesktop.org |
| 9503 | S: Supported | 9522 | S: Supported |
| 9504 | W: https://01.org/linuxgraphics/ | 9523 | W: https://01.org/linuxgraphics/ |
| @@ -20228,6 +20247,8 @@ F: include/uapi/linux/virtio_gpio.h · 10723 unmodified lines | |||
| 20228 | VIRTIO GPU DRIVER | 20247 | VIRTIO GPU DRIVER |
| 20229 | M: David Airlie <airlied@linux.ie> | 20248 | M: David Airlie <airlied@linux.ie> |
| 20230 | M: Gerd Hoffmann <kraxel@redhat.com> | 20249 | M: Gerd Hoffmann <kraxel@redhat.com> |
| 20250 | R: Gurchetan Singh <gurchetansingh@chromium.org> | ||
| 20251 | R: Chia-I Wu <olvaffe@gmail.com> | ||
| 20231 | L: dri-devel@lists.freedesktop.org | 20252 | L: dri-devel@lists.freedesktop.org |
| 20232 | L: virtualization@lists.linux-foundation.org | 20253 | L: virtualization@lists.linux-foundation.org |
| 20233 | S: Maintained | 20254 | S: Maintained |
| @@ -554,6 +554,7 @@ static const struct pci_device_id intel_early_ids[] __initconst = { · 553 unmodified lines | |||
| 554 | INTEL_RKL_IDS(&gen11_early_ops), | 554 | INTEL_RKL_IDS(&gen11_early_ops), |
| 555 | INTEL_ADLS_IDS(&gen11_early_ops), | 555 | INTEL_ADLS_IDS(&gen11_early_ops), |
| 556 | INTEL_ADLP_IDS(&gen11_early_ops), | 556 | INTEL_ADLP_IDS(&gen11_early_ops), |
| 557 | INTEL_RPLS_IDS(&gen11_early_ops), | ||
| 557 | }; | 558 | }; |
| 558 | 559 | ||
| 559 | struct resource intel_graphics_stolen_res __ro_after_init = DEFINE_RES_MEM(0, 0); | 560 | struct resource intel_graphics_stolen_res __ro_after_init = DEFINE_RES_MEM(0, 0); |
| @@ -20,6 +20,7 @@ · 19 unmodified lines | |||
| 20 | #include <linux/kernel.h> | 20 | #include <linux/kernel.h> |
| 21 | #include <linux/pagemap.h> | 21 | #include <linux/pagemap.h> |
| 22 | #include <linux/agp_backend.h> | 22 | #include <linux/agp_backend.h> |
| 23 | #include <linux/intel-iommu.h> | ||
| 23 | #include <linux/delay.h> | 24 | #include <linux/delay.h> |
| 24 | #include <asm/smp.h> | 25 | #include <asm/smp.h> |
| 25 | #include "agp.h" | 26 | #include "agp.h" |
| @@ -932,8 +932,7 @@ static int bcm2835_clock_is_on(struct clk_hw *hw) · 931 unmodified lines | |||
| 932 | 932 | ||
| 933 | static u32 bcm2835_clock_choose_div(struct clk_hw *hw, | 933 | static u32 bcm2835_clock_choose_div(struct clk_hw *hw, |
| 934 | unsigned long rate, | 934 | unsigned long rate, |
| 935 | unsigned long parent_rate, | 935 | unsigned long parent_rate) |
| 936 | bool round_up) | ||
| 937 | { | 936 | { |
| 938 | struct bcm2835_clock *clock = bcm2835_clock_from_hw(hw); | 937 | struct bcm2835_clock *clock = bcm2835_clock_from_hw(hw); |
| 939 | const struct bcm2835_clock_data *data = clock->data; | 938 | const struct bcm2835_clock_data *data = clock->data; |
| @@ -945,10 +944,6 @@ static u32 bcm2835_clock_choose_div(struct clk_hw *hw, · 5 unmodified lines | |||
| 945 | 944 | ||
| 946 | rem = do_div(temp, rate); | 945 | rem = do_div(temp, rate); |
| 947 | div = temp; | 946 | div = temp; |
| 948 | |||
| 949 | /* Round up and mask off the unused bits */ | ||
| 950 | if (round_up && ((div & unused_frac_mask) != 0 || rem != 0)) | ||
| 951 | div += unused_frac_mask + 1; | ||
| 952 | div &= ~unused_frac_mask; | 947 | div &= ~unused_frac_mask; |
| 953 | 948 | ||
| 954 | /* different clamping limits apply for a mash clock */ | 949 | /* different clamping limits apply for a mash clock */ |
| @@ -1079,7 +1074,7 @@ static int bcm2835_clock_set_rate(struct clk_hw *hw, · 124 unmodified lines | |||
| 1079 | struct bcm2835_clock *clock = bcm2835_clock_from_hw(hw); | 1074 | struct bcm2835_clock *clock = bcm2835_clock_from_hw(hw); |
| 1080 | struct bcm2835_cprman *cprman = clock->cprman; | 1075 | struct bcm2835_cprman *cprman = clock->cprman; |
| 1081 | const struct bcm2835_clock_data *data = clock->data; | 1076 | const struct bcm2835_clock_data *data = clock->data; |
| 1082 | u32 div = bcm2835_clock_choose_div(hw, rate, parent_rate, false); | 1077 | u32 div = bcm2835_clock_choose_div(hw, rate, parent_rate); |
| 1083 | u32 ctl; | 1078 | u32 ctl; |
| 1084 | 1079 | ||
| 1085 | spin_lock(&cprman->regs_lock); | 1080 | spin_lock(&cprman->regs_lock); |
| @@ -1130,7 +1125,7 @@ static unsigned long bcm2835_clock_choose_div_and_prate(struct clk_hw *hw, · 44 unmodified lines | |||
| 1130 | 1125 | ||
| 1131 | if (!(BIT(parent_idx) & data->set_rate_parent)) { | 1126 | if (!(BIT(parent_idx) & data->set_rate_parent)) { |
| 1132 | *prate = clk_hw_get_rate(parent); | 1127 | *prate = clk_hw_get_rate(parent); |
| 1133 | *div = bcm2835_clock_choose_div(hw, rate, *prate, true); | 1128 | *div = bcm2835_clock_choose_div(hw, rate, *prate); |
| 1134 | 1129 | ||
| 1135 | *avgrate = bcm2835_clock_rate_from_divisor(clock, *prate, *div); | 1130 | *avgrate = bcm2835_clock_rate_from_divisor(clock, *prate, *div); |
| 1136 | 1131 | ||
| @@ -1216,7 +1211,7 @@ static int bcm2835_clock_determine_rate(struct clk_hw *hw, · 79 unmodified lines | |||
| 1216 | rate = bcm2835_clock_choose_div_and_prate(hw, i, req->rate, | 1211 | rate = bcm2835_clock_choose_div_and_prate(hw, i, req->rate, |
| 1217 | &div, &prate, | 1212 | &div, &prate, |
| 1218 | &avgrate); | 1213 | &avgrate); |
| 1219 | if (rate > best_rate && rate <= req->rate) { | 1214 | if (abs(req->rate - rate) < abs(req->rate - best_rate)) { |
| 1220 | best_parent = parent; | 1215 | best_parent = parent; |
| 1221 | best_prate = prate; | 1216 | best_prate = prate; |
| 1222 | best_rate = rate; | 1217 | best_rate = rate; |
| @@ -11,6 +11,7 @@ obj-$(CONFIG_DMABUF_SYSFS_STATS) += dma-buf-sysfs-stats.o · 10 unmodified lines | |||
| 11 | dmabuf_selftests-y := \ | 11 | dmabuf_selftests-y := \ |
| 12 | selftest.o \ | 12 | selftest.o \ |
| 13 | st-dma-fence.o \ | 13 | st-dma-fence.o \ |
| 14 | st-dma-fence-chain.o | 14 | st-dma-fence-chain.o \ |
| 15 | st-dma-resv.o | ||
| 15 | 16 | ||
| 16 | obj-$(CONFIG_DMABUF_SELFTESTS) += dmabuf_selftests.o | 17 | obj-$(CONFIG_DMABUF_SELFTESTS) += dmabuf_selftests.o |
| @@ -299,10 +299,8 @@ static __poll_t dma_buf_poll(struct file *file, poll_table *poll) · 298 unmodified lines | |||
| 299 | 299 | ||
| 300 | /** | 300 | /** |
| 301 | * dma_buf_set_name - Set a name to a specific dma_buf to track the usage. | 301 | * dma_buf_set_name - Set a name to a specific dma_buf to track the usage. |
| 302 | * The name of the dma-buf buffer can only be set when the dma-buf is not | 302 | * It could support changing the name of the dma-buf if the same |
| 303 | * attached to any devices. It could theoritically support changing the | 303 | * piece of memory is used for multiple purpose between different devices. |
| 304 | * name of the dma-buf if the same piece of memory is used for multiple | ||
| 305 | * purpose between different devices. | ||
| 306 | * | 304 | * |
| 307 | * @dmabuf: [in] dmabuf buffer that will be renamed. | 305 | * @dmabuf: [in] dmabuf buffer that will be renamed. |
| 308 | * @buf: [in] A piece of userspace memory that contains the name of | 306 | * @buf: [in] A piece of userspace memory that contains the name of |
| @@ -315,25 +313,16 @@ static __poll_t dma_buf_poll(struct file *file, poll_table *poll) · 6 unmodified lines | |||
| 315 | static long dma_buf_set_name(struct dma_buf *dmabuf, const char __user *buf) | 313 | static long dma_buf_set_name(struct dma_buf *dmabuf, const char __user *buf) |
| 316 | { | 314 | { |
| 317 | char *name = strndup_user(buf, DMA_BUF_NAME_LEN); | 315 | char *name = strndup_user(buf, DMA_BUF_NAME_LEN); |
| 318 | long ret = 0; | ||
| 319 | 316 | ||
| 320 | if (IS_ERR(name)) | 317 | if (IS_ERR(name)) |
| 321 | return PTR_ERR(name); | 318 | return PTR_ERR(name); |
| 322 | 319 | ||
| 323 | dma_resv_lock(dmabuf->resv, NULL); | ||
| 324 | if (!list_empty(&dmabuf->attachments)) { | ||
| 325 | ret = -EBUSY; | ||
| 326 | kfree(name); | ||
| 327 | goto out_unlock; | ||
| 328 | } | ||
| 329 | spin_lock(&dmabuf->name_lock); | 320 | spin_lock(&dmabuf->name_lock); |
| 330 | kfree(dmabuf->name); | 321 | kfree(dmabuf->name); |
| 331 | dmabuf->name = name; | 322 | dmabuf->name = name; |
| 332 | spin_unlock(&dmabuf->name_lock); | 323 | spin_unlock(&dmabuf->name_lock); |
| 333 | 324 | ||
| 334 | out_unlock: | 325 | return 0; |
| 335 | dma_resv_unlock(dmabuf->resv); | ||
| 336 | return ret; | ||
| 337 | } | 326 | } |
| 338 | 327 | ||
| 339 | static long dma_buf_ioctl(struct file *file, | 328 | static long dma_buf_ioctl(struct file *file, |
| @@ -1058,8 +1047,8 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_move_notify, DMA_BUF); · 718 unmodified lines | |||
| 1058 | * | 1047 | * |
| 1059 | * Interfaces:: | 1048 | * Interfaces:: |
| 1060 | * | 1049 | * |
| 1061 | * void \*dma_buf_vmap(struct dma_buf \*dmabuf) | 1050 | * void \*dma_buf_vmap(struct dma_buf \*dmabuf, struct dma_buf_map \*map) |
| 1062 | * void dma_buf_vunmap(struct dma_buf \*dmabuf, void \*vaddr) | 1051 | * void dma_buf_vunmap(struct dma_buf \*dmabuf, struct dma_buf_map \*map) |
| 1063 | * | 1052 | * |
| 1064 | * The vmap call can fail if there is no vmap support in the exporter, or if | 1053 | * The vmap call can fail if there is no vmap support in the exporter, or if |
| 1065 | * it runs out of vmalloc space. Note that the dma-buf layer keeps a reference | 1054 | * it runs out of vmalloc space. Note that the dma-buf layer keeps a reference |
| @@ -1338,8 +1327,6 @@ static int dma_buf_debug_show(struct seq_file *s, void *unused) · 272 unmodified lines | |||
| 1338 | { | 1327 | { |
| 1339 | struct dma_buf *buf_obj; | 1328 | struct dma_buf *buf_obj; |
| 1340 | struct dma_buf_attachment *attach_obj; | 1329 | struct dma_buf_attachment *attach_obj; |
| 1341 | struct dma_resv_iter cursor; | ||
| 1342 | struct dma_fence *fence; | ||
| 1343 | int count = 0, attach_count; | 1330 | int count = 0, attach_count; |
| 1344 | size_t size = 0; | 1331 | size_t size = 0; |
| 1345 | int ret; | 1332 | int ret; |
| @@ -1370,14 +1357,7 @@ static int dma_buf_debug_show(struct seq_file *s, void *unused) · 24 unmodified lines | |||
| 1370 | buf_obj->name ?: ""); | 1357 | buf_obj->name ?: ""); |
| 1371 | spin_unlock(&buf_obj->name_lock); | 1358 | spin_unlock(&buf_obj->name_lock); |
| 1372 | 1359 | ||
| 1373 | dma_resv_for_each_fence(&cursor, buf_obj->resv, true, fence) { | 1360 | dma_resv_describe(buf_obj->resv, s); |
| 1374 | seq_printf(s, "\t%s fence: %s %s %ssignalled\n", | ||
| 1375 | dma_resv_iter_is_exclusive(&cursor) ? | ||
| 1376 | "Exclusive" : "Shared", | ||
| 1377 | fence->ops->get_driver_name(fence), | ||
| 1378 | fence->ops->get_timeline_name(fence), | ||
| 1379 | dma_fence_is_signaled(fence) ? "" : "un"); | ||
| 1380 | } | ||
| 1381 | 1361 | ||
| 1382 | seq_puts(s, "\tAttached Devices:\n"); | 1362 | seq_puts(s, "\tAttached Devices:\n"); |
| 1383 | attach_count = 0; | 1363 | attach_count = 0; |
| @@ -104,7 +104,11 @@ static bool dma_fence_array_signaled(struct dma_fence *fence) · 103 unmodified lines | |||
| 104 | { | 104 | { |
| 105 | struct dma_fence_array *array = to_dma_fence_array(fence); | 105 | struct dma_fence_array *array = to_dma_fence_array(fence); |
| 106 | 106 | ||
| 107 | return atomic_read(&array->num_pending) <= 0; | 107 | if (atomic_read(&array->num_pending) > 0) |
| 108 | return false; | ||
| 109 | |||
| 110 | dma_fence_array_clear_pending_error(array); | ||
| 111 | return true; | ||
| 108 | } | 112 | } |
| 109 | 113 | ||
| 110 | static void dma_fence_array_release(struct dma_fence *fence) | 114 | static void dma_fence_array_release(struct dma_fence *fence) |
| @@ -15,6 +15,7 @@ · 14 unmodified lines | |||
| 15 | #include <linux/atomic.h> | 15 | #include <linux/atomic.h> |
| 16 | #include <linux/dma-fence.h> | 16 | #include <linux/dma-fence.h> |
| 17 | #include <linux/sched/signal.h> | 17 | #include <linux/sched/signal.h> |
| 18 | #include <linux/seq_file.h> | ||
| 18 | 19 | ||
| 19 | #define CREATE_TRACE_POINTS | 20 | #define CREATE_TRACE_POINTS |
| 20 | #include <trace/events/dma_fence.h> | 21 | #include <trace/events/dma_fence.h> |
| @@ -907,6 +908,22 @@ err_free_cb: · 886 unmodified lines | |||
| 907 | } | 908 | } |
| 908 | EXPORT_SYMBOL(dma_fence_wait_any_timeout); | 909 | EXPORT_SYMBOL(dma_fence_wait_any_timeout); |
| 909 | 910 | ||
| 911 | /** | ||
| 912 | * dma_fence_describe - Dump fence describtion into seq_file | ||
| 913 | * @fence: the 6fence to describe | ||
| 914 | * @seq: the seq_file to put the textual description into | ||
| 915 | * | ||
| 916 | * Dump a textual description of the fence and it's state into the seq_file. | ||
| 917 | */ | ||
| 918 | void dma_fence_describe(struct dma_fence *fence, struct seq_file *seq) | ||
| 919 | { | ||
| 920 | seq_printf(seq, "%s %s seq %llu %ssignalled\n", | ||
| 921 | fence->ops->get_driver_name(fence), | ||
| 922 | fence->ops->get_timeline_name(fence), fence->seqno, | ||
| 923 | dma_fence_is_signaled(fence) ? "" : "un"); | ||
| 924 | } | ||
| 925 | EXPORT_SYMBOL(dma_fence_describe); | ||
| 926 | |||
| 910 | /** | 927 | /** |
| 911 | * dma_fence_init - Initialize a custom fence. | 928 | * dma_fence_init - Initialize a custom fence. |
| 912 | * @fence: the fence to initialize | 929 | * @fence: the fence to initialize |
| @@ -38,6 +38,7 @@ · 37 unmodified lines | |||
| 38 | #include <linux/mm.h> | 38 | #include <linux/mm.h> |
| 39 | #include <linux/sched/mm.h> | 39 | #include <linux/sched/mm.h> |
| 40 | #include <linux/mmu_notifier.h> | 40 | #include <linux/mmu_notifier.h> |
| 41 | #include <linux/seq_file.h> | ||
| 41 | 42 | ||
| 42 | /** | 43 | /** |
| 43 | * DOC: Reservation Object Overview | 44 | * DOC: Reservation Object Overview |
| @@ -304,8 +305,7 @@ void dma_resv_add_excl_fence(struct dma_resv *obj, struct dma_fence *fence) · 260 unmodified lines | |||
| 304 | if (old) | 305 | if (old) |
| 305 | i = old->shared_count; | 306 | i = old->shared_count; |
| 306 | 307 | ||
| 307 | if (fence) | 308 | dma_fence_get(fence); |
| 308 | dma_fence_get(fence); | ||
| 309 | 309 | ||
| 310 | write_seqcount_begin(&obj->seq); | 310 | write_seqcount_begin(&obj->seq); |
| 311 | /* write_seqcount_begin provides the necessary memory barrier */ | 311 | /* write_seqcount_begin provides the necessary memory barrier */ |
| @@ -666,6 +666,28 @@ bool dma_resv_test_signaled(struct dma_resv *obj, bool test_all) · 354 unmodified lines | |||
| 666 | } | 666 | } |
| 667 | EXPORT_SYMBOL_GPL(dma_resv_test_signaled); | 667 | EXPORT_SYMBOL_GPL(dma_resv_test_signaled); |
| 668 | 668 | ||
| 669 | /** | ||
| 670 | * dma_resv_describe - Dump description of the resv object into seq_file | ||
| 671 | * @obj: the reservation object | ||
| 672 | * @seq: the seq_file to dump the description into | ||
| 673 | * | ||
| 674 | * Dump a textual description of the fences inside an dma_resv object into the | ||
| 675 | * seq_file. | ||
| 676 | */ | ||
| 677 | void dma_resv_describe(struct dma_resv *obj, struct seq_file *seq) | ||
| 678 | { | ||
| 679 | struct dma_resv_iter cursor; | ||
| 680 | struct dma_fence *fence; | ||
| 681 | |||
| 682 | dma_resv_for_each_fence(&cursor, obj, true, fence) { | ||
| 683 | seq_printf(seq, "\t%s fence:", | ||
| 684 | dma_resv_iter_is_exclusive(&cursor) ? | ||
| 685 | "Exclusive" : "Shared"); | ||
| 686 | dma_fence_describe(fence, seq); | ||
| 687 | } | ||
| 688 | } | ||
| 689 | EXPORT_SYMBOL_GPL(dma_resv_describe); | ||
| 690 | |||
| 669 | #if IS_ENABLED(CONFIG_LOCKDEP) | 691 | #if IS_ENABLED(CONFIG_LOCKDEP) |
| 670 | static int __init dma_resv_lockdep(void) | 692 | static int __init dma_resv_lockdep(void) |
| 671 | { | 693 | { |
| @@ -12,3 +12,4 @@ · 11 unmodified lines | |||
| 12 | selftest(sanitycheck, __sanitycheck__) /* keep first (igt selfcheck) */ | 12 | selftest(sanitycheck, __sanitycheck__) /* keep first (igt selfcheck) */ |
| 13 | selftest(dma_fence, dma_fence) | 13 | selftest(dma_fence, dma_fence) |
| 14 | selftest(dma_fence_chain, dma_fence_chain) | 14 | selftest(dma_fence_chain, dma_fence_chain) |
| 15 | selftest(dma_resv, dma_resv) | ||
| @@ -0,0 +1,371 @@ | |||
| 1 | /* SPDX-License-Identifier: MIT */ | ||
| 2 | |||
| 3 | /* | ||
| 4 | * Copyright © 2019 Intel Corporation | ||
| 5 | * Copyright © 2021 Advanced Micro Devices, Inc. | ||
| 6 | */ | ||
| 7 | |||
| 8 | #include <linux/slab.h> | ||
| 9 | #include <linux/spinlock.h> | ||
| 10 | #include <linux/dma-resv.h> | ||
| 11 | |||
| 12 | #include "selftest.h" | ||
| 13 | |||
| 14 | static struct spinlock fence_lock; | ||
| 15 | |||
| 16 | static const char *fence_name(struct dma_fence *f) | ||
| 17 | { | ||
| 18 | return "selftest"; | ||
| 19 | } | ||
| 20 | |||
| 21 | static const struct dma_fence_ops fence_ops = { | ||
| 22 | .get_driver_name = fence_name, | ||
| 23 | .get_timeline_name = fence_name, | ||
| 24 | }; | ||
| 25 | |||
| 26 | static struct dma_fence *alloc_fence(void) | ||
| 27 | { | ||
| 28 | struct dma_fence *f; | ||
| 29 | |||
| 30 | f = kmalloc(sizeof(*f), GFP_KERNEL); | ||
| 31 | if (!f) | ||
| 32 | return NULL; | ||
| 33 | |||
| 34 | dma_fence_init(f, &fence_ops, &fence_lock, 0, 0); | ||
| 35 | return f; | ||
| 36 | } | ||
| 37 | |||
| 38 | static int sanitycheck(void *arg) | ||
| 39 | { | ||
| 40 | struct dma_resv resv; | ||
| 41 | struct dma_fence *f; | ||
| 42 | int r; | ||
| 43 | |||
| 44 | f = alloc_fence(); | ||
| 45 | if (!f) | ||
| 46 | return -ENOMEM; | ||
| 47 | |||
| 48 | dma_fence_signal(f); | ||
| 49 | dma_fence_put(f); | ||
| 50 | |||
| 51 | dma_resv_init(&resv); | ||
| 52 | r = dma_resv_lock(&resv, NULL); | ||
| 53 | if (r) | ||
| 54 | pr_err("Resv locking failed\n"); | ||
| 55 | else | ||
| 56 | dma_resv_unlock(&resv); | ||
| 57 | dma_resv_fini(&resv); | ||
| 58 | return r; | ||
| 59 | } | ||
| 60 | |||
| 61 | static int test_signaling(void *arg, bool shared) | ||
| 62 | { | ||
| 63 | struct dma_resv resv; | ||
| 64 | struct dma_fence *f; | ||
| 65 | int r; | ||
| 66 | |||
| 67 | f = alloc_fence(); | ||
| 68 | if (!f) | ||
| 69 | return -ENOMEM; | ||
| 70 | |||
| 71 | dma_resv_init(&resv); | ||
| 72 | r = dma_resv_lock(&resv, NULL); | ||
| 73 | if (r) { | ||
| 74 | pr_err("Resv locking failed\n"); | ||
| 75 | goto err_free; | ||
| 76 | } | ||
| 77 | |||
| 78 | if (shared) { | ||
| 79 | r = dma_resv_reserve_shared(&resv, 1); | ||
| 80 | if (r) { | ||
| 81 | pr_err("Resv shared slot allocation failed\n"); | ||
| 82 | goto err_unlock; | ||
| 83 | } | ||
| 84 | |||
| 85 | dma_resv_add_shared_fence(&resv, f); | ||
| 86 | } else { | ||
| 87 | dma_resv_add_excl_fence(&resv, f); | ||
| 88 | } | ||
| 89 | |||
| 90 | if (dma_resv_test_signaled(&resv, shared)) { | ||
| 91 | pr_err("Resv unexpectedly signaled\n"); | ||
| 92 | r = -EINVAL; | ||
| 93 | goto err_unlock; | ||
| 94 | } | ||
| 95 | dma_fence_signal(f); | ||
| 96 | if (!dma_resv_test_signaled(&resv, shared)) { | ||
| 97 | pr_err("Resv not reporting signaled\n"); | ||
| 98 | r = -EINVAL; | ||
| 99 | goto err_unlock; | ||
| 100 | } | ||
| 101 | err_unlock: | ||
| 102 | dma_resv_unlock(&resv); | ||
| 103 | err_free: | ||
| 104 | dma_resv_fini(&resv); | ||
| 105 | dma_fence_put(f); | ||
| 106 | return r; | ||
| 107 | } | ||
| 108 | |||
| 109 | static int test_excl_signaling(void *arg) | ||
| 110 | { | ||
| 111 | return test_signaling(arg, false); | ||
| 112 | } | ||
| 113 | |||
| 114 | static int test_shared_signaling(void *arg) | ||
| 115 | { | ||
| 116 | return test_signaling(arg, true); | ||
| 117 | } | ||
| 118 | |||
| 119 | static int test_for_each(void *arg, bool shared) | ||
| 120 | { | ||
| 121 | struct dma_resv_iter cursor; | ||
| 122 | struct dma_fence *f, *fence; | ||
| 123 | struct dma_resv resv; | ||
| 124 | int r; | ||
| 125 | |||
| 126 | f = alloc_fence(); | ||
| 127 | if (!f) | ||
| 128 | return -ENOMEM; | ||
| 129 | |||
| 130 | dma_resv_init(&resv); | ||
| 131 | r = dma_resv_lock(&resv, NULL); | ||
| 132 | if (r) { | ||
| 133 | pr_err("Resv locking failed\n"); | ||
| 134 | goto err_free; | ||
| 135 | } | ||
| 136 | |||
| 137 | if (shared) { | ||
| 138 | r = dma_resv_reserve_shared(&resv, 1); | ||
| 139 | if (r) { | ||
| 140 | pr_err("Resv shared slot allocation failed\n"); | ||
| 141 | goto err_unlock; | ||
| 142 | } | ||
| 143 | |||
| 144 | dma_resv_add_shared_fence(&resv, f); | ||
| 145 | } else { | ||
| 146 | dma_resv_add_excl_fence(&resv, f); | ||
| 147 | } | ||
| 148 | |||
| 149 | r = -ENOENT; | ||
| 150 | dma_resv_for_each_fence(&cursor, &resv, shared, fence) { | ||
| 151 | if (!r) { | ||
| 152 | pr_err("More than one fence found\n"); | ||
| 153 | r = -EINVAL; | ||
| 154 | goto err_unlock; | ||
| 155 | } | ||
| 156 | if (f != fence) { | ||
| 157 | pr_err("Unexpected fence\n"); | ||
| 158 | r = -EINVAL; | ||
| 159 | goto err_unlock; | ||
| 160 | } | ||
| 161 | if (dma_resv_iter_is_exclusive(&cursor) != !shared) { | ||
| 162 | pr_err("Unexpected fence usage\n"); | ||
| 163 | r = -EINVAL; | ||
| 164 | goto err_unlock; | ||
| 165 | } | ||
| 166 | r = 0; | ||
| 167 | } | ||
| 168 | if (r) { | ||
| 169 | pr_err("No fence found\n"); | ||
| 170 | goto err_unlock; | ||
| 171 | } | ||
| 172 | dma_fence_signal(f); | ||
| 173 | err_unlock: | ||
| 174 | dma_resv_unlock(&resv); | ||
| 175 | err_free: | ||
| 176 | dma_resv_fini(&resv); | ||
| 177 | dma_fence_put(f); | ||
| 178 | return r; | ||
| 179 | } | ||
| 180 | |||
| 181 | static int test_excl_for_each(void *arg) | ||
| 182 | { | ||
| 183 | return test_for_each(arg, false); | ||
| 184 | } | ||
| 185 | |||
| 186 | static int test_shared_for_each(void *arg) | ||
| 187 | { | ||
| 188 | return test_for_each(arg, true); | ||
| 189 | } | ||
| 190 | |||
| 191 | static int test_for_each_unlocked(void *arg, bool shared) | ||
| 192 | { | ||
| 193 | struct dma_resv_iter cursor; | ||
| 194 | struct dma_fence *f, *fence; | ||
| 195 | struct dma_resv resv; | ||
| 196 | int r; | ||
| 197 | |||
| 198 | f = alloc_fence(); | ||
| 199 | if (!f) | ||
| 200 | return -ENOMEM; | ||
| 201 | |||
| 202 | dma_resv_init(&resv); | ||
| 203 | r = dma_resv_lock(&resv, NULL); | ||
| 204 | if (r) { | ||
| 205 | pr_err("Resv locking failed\n"); | ||
| 206 | goto err_free; | ||
| 207 | } | ||
| 208 | |||
| 209 | if (shared) { | ||
| 210 | r = dma_resv_reserve_shared(&resv, 1); | ||
| 211 | if (r) { | ||
| 212 | pr_err("Resv shared slot allocation failed\n"); | ||
| 213 | dma_resv_unlock(&resv); | ||
| 214 | goto err_free; | ||
| 215 | } | ||
| 216 | |||
| 217 | dma_resv_add_shared_fence(&resv, f); | ||
| 218 | } else { | ||
| 219 | dma_resv_add_excl_fence(&resv, f); | ||
| 220 | } | ||
| 221 | dma_resv_unlock(&resv); | ||
| 222 | |||
| 223 | r = -ENOENT; | ||
| 224 | dma_resv_iter_begin(&cursor, &resv, shared); | ||
| 225 | dma_resv_for_each_fence_unlocked(&cursor, fence) { | ||
| 226 | if (!r) { | ||
| 227 | pr_err("More than one fence found\n"); | ||
| 228 | r = -EINVAL; | ||
| 229 | goto err_iter_end; | ||
| 230 | } | ||
| 231 | if (!dma_resv_iter_is_restarted(&cursor)) { | ||
| 232 | pr_err("No restart flag\n"); | ||
| 233 | goto err_iter_end; | ||
| 234 | } | ||
| 235 | if (f != fence) { | ||
| 236 | pr_err("Unexpected fence\n"); | ||
| 237 | r = -EINVAL; | ||
| 238 | goto err_iter_end; | ||
| 239 | } | ||
| 240 | if (dma_resv_iter_is_exclusive(&cursor) != !shared) { | ||
| 241 | pr_err("Unexpected fence usage\n"); | ||
| 242 | r = -EINVAL; | ||
| 243 | goto err_iter_end; | ||
| 244 | } | ||
| 245 | |||
| 246 | /* We use r as state here */ | ||
| 247 | if (r == -ENOENT) { | ||
| 248 | r = -EINVAL; | ||
| 249 | /* That should trigger an restart */ | ||
| 250 | cursor.seq--; | ||
| 251 | } else if (r == -EINVAL) { | ||
| 252 | r = 0; | ||
| 253 | } | ||
| 254 | } | ||
| 255 | if (r) | ||
| 256 | pr_err("No fence found\n"); | ||
| 257 | err_iter_end: | ||
| 258 | dma_resv_iter_end(&cursor); | ||
| 259 | dma_fence_signal(f); | ||
| 260 | err_free: | ||
| 261 | dma_resv_fini(&resv); | ||
| 262 | dma_fence_put(f); | ||
| 263 | return r; | ||
| 264 | } | ||
| 265 | |||
| 266 | static int test_excl_for_each_unlocked(void *arg) | ||
| 267 | { | ||
| 268 | return test_for_each_unlocked(arg, false); | ||
| 269 | } | ||
| 270 | |||
| 271 | static int test_shared_for_each_unlocked(void *arg) | ||
| 272 | { | ||
| 273 | return test_for_each_unlocked(arg, true); | ||
| 274 | } | ||
| 275 | |||
| 276 | static int test_get_fences(void *arg, bool shared) | ||
| 277 | { | ||
| 278 | struct dma_fence *f, *excl = NULL, **fences = NULL; | ||
| 279 | struct dma_resv resv; | ||
| 280 | int r, i; | ||
| 281 | |||
| 282 | f = alloc_fence(); | ||
| 283 | if (!f) | ||
| 284 | return -ENOMEM; | ||
| 285 | |||
| 286 | dma_resv_init(&resv); | ||
| 287 | r = dma_resv_lock(&resv, NULL); | ||
| 288 | if (r) { | ||
| 289 | pr_err("Resv locking failed\n"); | ||
| 290 | goto err_resv; | ||
| 291 | } | ||
| 292 | |||
| 293 | if (shared) { | ||
| 294 | r = dma_resv_reserve_shared(&resv, 1); | ||
| 295 | if (r) { | ||
| 296 | pr_err("Resv shared slot allocation failed\n"); | ||
| 297 | dma_resv_unlock(&resv); | ||
| 298 | goto err_resv; | ||
| 299 | } | ||
| 300 | |||
| 301 | dma_resv_add_shared_fence(&resv, f); | ||
| 302 | } else { | ||
| 303 | dma_resv_add_excl_fence(&resv, f); | ||
| 304 | } | ||
| 305 | dma_resv_unlock(&resv); | ||
| 306 | |||
| 307 | r = dma_resv_get_fences(&resv, &excl, &i, &fences); | ||
| 308 | if (r) { | ||
| 309 | pr_err("get_fences failed\n"); | ||
| 310 | goto err_free; | ||
| 311 | } | ||
| 312 | |||
| 313 | if (shared) { | ||
| 314 | if (excl != NULL) { | ||
| 315 | pr_err("get_fences returned unexpected excl fence\n"); | ||
| 316 | goto err_free; | ||
| 317 | } | ||
| 318 | if (i != 1 || fences[0] != f) { | ||
| 319 | pr_err("get_fences returned unexpected shared fence\n"); | ||
| 320 | goto err_free; | ||
| 321 | } | ||
| 322 | } else { | ||
| 323 | if (excl != f) { | ||
| 324 | pr_err("get_fences returned unexpected excl fence\n"); | ||
| 325 | goto err_free; | ||
| 326 | } | ||
| 327 | if (i != 0) { | ||
| 328 | pr_err("get_fences returned unexpected shared fence\n"); | ||
| 329 | goto err_free; | ||
| 330 | } | ||
| 331 | } | ||
| 332 | |||
| 333 | dma_fence_signal(f); | ||
| 334 | err_free: | ||
| 335 | dma_fence_put(excl); | ||
| 336 | while (i--) | ||
| 337 | dma_fence_put(fences[i]); | ||
| 338 | kfree(fences); | ||
| 339 | err_resv: | ||
| 340 | dma_resv_fini(&resv); | ||
| 341 | dma_fence_put(f); | ||
| 342 | return r; | ||
| 343 | } | ||
| 344 | |||
| 345 | static int test_excl_get_fences(void *arg) | ||
| 346 | { | ||
| 347 | return test_get_fences(arg, false); | ||
| 348 | } | ||
| 349 | |||
| 350 | static int test_shared_get_fences(void *arg) | ||
| 351 | { | ||
| 352 | return test_get_fences(arg, true); | ||
| 353 | } | ||
| 354 | |||
| 355 | int dma_resv(void) | ||
| 356 | { | ||
| 357 | static const struct subtest tests[] = { | ||
| 358 | SUBTEST(sanitycheck), | ||
| 359 | SUBTEST(test_excl_signaling), | ||
| 360 | SUBTEST(test_shared_signaling), | ||
| 361 | SUBTEST(test_excl_for_each), | ||
| 362 | SUBTEST(test_shared_for_each), | ||
| 363 | SUBTEST(test_excl_for_each_unlocked), | ||
| 364 | SUBTEST(test_shared_for_each_unlocked), | ||
| 365 | SUBTEST(test_excl_get_fences), | ||
| 366 | SUBTEST(test_shared_get_fences), | ||
| 367 | }; | ||
| 368 | |||
| 369 | spin_lock_init(&fence_lock); | ||
| 370 | return subtests(tests, NULL); | ||
| 371 | } | ||
| @@ -8,6 +8,7 @@ · 7 unmodified lines | |||
| 8 | menuconfig DRM | 8 | menuconfig DRM |
| 9 | tristate "Direct Rendering Manager (XFree86 4.1.0 and higher DRI support)" | 9 | tristate "Direct Rendering Manager (XFree86 4.1.0 and higher DRI support)" |
| 10 | depends on (AGP || AGP=n) && !EMULATED_CMPXCHG && HAS_DMA | 10 | depends on (AGP || AGP=n) && !EMULATED_CMPXCHG && HAS_DMA |
| 11 | select DRM_NOMODESET | ||
| 11 | select DRM_PANEL_ORIENTATION_QUIRKS | 12 | select DRM_PANEL_ORIENTATION_QUIRKS |
| 12 | select HDMI | 13 | select HDMI |
| 13 | select FB_CMDLINE | 14 | select FB_CMDLINE |
| @@ -211,20 +212,13 @@ config DRM_TTM_HELPER · 197 unmodified lines | |||
| 211 | Helpers for ttm-based gem objects | 212 | Helpers for ttm-based gem objects |
| 212 | 213 | ||
| 213 | config DRM_GEM_CMA_HELPER | 214 | config DRM_GEM_CMA_HELPER |
| 214 | bool | 215 | tristate |
| 215 | depends on DRM | 216 | depends on DRM |
| 216 | help | 217 | help |
| 217 | Choose this if you need the GEM CMA helper functions | 218 | Choose this if you need the GEM CMA helper functions |
| 218 | 219 | ||
| 219 | config DRM_KMS_CMA_HELPER | ||
| 220 | bool | ||
| 221 | depends on DRM | ||
| 222 | select DRM_GEM_CMA_HELPER | ||
| 223 | help | ||
| 224 | Choose this if you need the KMS CMA helper functions | ||
| 225 | |||
| 226 | config DRM_GEM_SHMEM_HELPER | 220 | config DRM_GEM_SHMEM_HELPER |
| 227 | bool | 221 | tristate |
| 228 | depends on DRM && MMU | 222 | depends on DRM && MMU |
| 229 | help | 223 | help |
| 230 | Choose this if you need the GEM shmem helper functions | 224 | Choose this if you need the GEM shmem helper functions |
| @@ -394,6 +388,8 @@ source "drivers/gpu/drm/xlnx/Kconfig" · 163 unmodified lines | |||
| 394 | 388 | ||
| 395 | source "drivers/gpu/drm/gud/Kconfig" | 389 | source "drivers/gpu/drm/gud/Kconfig" |
| 396 | 390 | ||
| 391 | source "drivers/gpu/drm/sprd/Kconfig" | ||
| 392 | |||
| 397 | config DRM_HYPERV | 393 | config DRM_HYPERV |
| 398 | tristate "DRM Support for Hyper-V synthetic video device" | 394 | tristate "DRM Support for Hyper-V synthetic video device" |
| 399 | depends on DRM && PCI && MMU && HYPERV | 395 | depends on DRM && PCI && MMU && HYPERV |
| @@ -492,6 +488,15 @@ config DRM_EXPORT_FOR_TESTS · 92 unmodified lines | |||
| 492 | config DRM_PANEL_ORIENTATION_QUIRKS | 488 | config DRM_PANEL_ORIENTATION_QUIRKS |
| 493 | tristate | 489 | tristate |
| 494 | 490 | ||
| 491 | # Separate option because nomodeset parameter is global and expected built-in | ||
| 492 | config DRM_NOMODESET | ||
| 493 | bool | ||
| 494 | default n | ||
| 495 | |||
| 495 | config DRM_LIB_RANDOM | 496 | config DRM_LIB_RANDOM |
| 496 | bool | 497 | bool |
| 497 | default n | 498 | default n |
| 499 | |||
| 500 | config DRM_PRIVACY_SCREEN | ||
| 501 | bool | ||
| 502 | default n | ||
| @@ -4,37 +4,44 @@ · 3 unmodified lines | |||
| 4 | # Direct Rendering Infrastructure (DRI) in XFree86 4.1.0 and higher. | 4 | # Direct Rendering Infrastructure (DRI) in XFree86 4.1.0 and higher. |
| 5 | 5 | ||
| 6 | drm-y := drm_aperture.o drm_auth.o drm_cache.o \ | 6 | drm-y := drm_aperture.o drm_auth.o drm_cache.o \ |
| 7 | drm_file.o drm_gem.o drm_ioctl.o drm_irq.o \ | 7 | drm_file.o drm_gem.o drm_ioctl.o \ |
| 8 | drm_drv.o \ | 8 | drm_drv.o \ |
| 9 | drm_sysfs.o drm_hashtab.o drm_mm.o \ | 9 | drm_sysfs.o drm_mm.o \ |
| 10 | drm_crtc.o drm_fourcc.o drm_modes.o drm_edid.o drm_displayid.o \ | 10 | drm_crtc.o drm_fourcc.o drm_modes.o drm_edid.o drm_displayid.o \ |
| 11 | drm_encoder_slave.o \ | ||
| 12 | drm_trace_points.o drm_prime.o \ | 11 | drm_trace_points.o drm_prime.o \ |
| 13 | drm_rect.o drm_vma_manager.o drm_flip_work.o \ | 12 | drm_vma_manager.o \ |
| 14 | drm_modeset_lock.o drm_atomic.o drm_bridge.o \ | 13 | drm_modeset_lock.o drm_atomic.o drm_bridge.o \ |
| 15 | drm_framebuffer.o drm_connector.o drm_blend.o \ | 14 | drm_framebuffer.o drm_connector.o drm_blend.o \ |
| 16 | drm_encoder.o drm_mode_object.o drm_property.o \ | 15 | drm_encoder.o drm_mode_object.o drm_property.o \ |
| 17 | drm_plane.o drm_color_mgmt.o drm_print.o \ | 16 | drm_plane.o drm_color_mgmt.o drm_print.o \ |
| 18 | drm_dumb_buffers.o drm_mode_config.o drm_vblank.o \ | 17 | drm_dumb_buffers.o drm_mode_config.o drm_vblank.o \ |
| 19 | drm_syncobj.o drm_lease.o drm_writeback.o drm_client.o \ | 18 | drm_syncobj.o drm_lease.o drm_writeback.o drm_client.o \ |
| 20 | drm_client_modeset.o drm_atomic_uapi.o drm_hdcp.o \ | 19 | drm_client_modeset.o drm_atomic_uapi.o \ |
| 21 | drm_managed.o drm_vblank_work.o | 20 | drm_managed.o drm_vblank_work.o |
| 22 | 21 | ||
| 23 | drm-$(CONFIG_DRM_LEGACY) += drm_agpsupport.o drm_bufs.o drm_context.o drm_dma.o \ | 22 | drm-$(CONFIG_DRM_LEGACY) += drm_agpsupport.o drm_bufs.o drm_context.o drm_dma.o \ |
| 24 | drm_legacy_misc.o drm_lock.o drm_memory.o drm_scatter.o \ | 23 | drm_hashtab.o drm_irq.o drm_legacy_misc.o drm_lock.o \ |
| 25 | drm_vm.o | 24 | drm_memory.o drm_scatter.o drm_vm.o |
| 26 | drm-$(CONFIG_DRM_LIB_RANDOM) += lib/drm_random.o | 25 | drm-$(CONFIG_DRM_LIB_RANDOM) += lib/drm_random.o |
| 27 | drm-$(CONFIG_COMPAT) += drm_ioc32.o | 26 | drm-$(CONFIG_COMPAT) += drm_ioc32.o |
| 28 | drm-$(CONFIG_DRM_GEM_CMA_HELPER) += drm_gem_cma_helper.o | ||
| 29 | drm-$(CONFIG_DRM_GEM_SHMEM_HELPER) += drm_gem_shmem_helper.o | ||
| 30 | drm-$(CONFIG_DRM_PANEL) += drm_panel.o | 27 | drm-$(CONFIG_DRM_PANEL) += drm_panel.o |
| 31 | drm-$(CONFIG_OF) += drm_of.o | 28 | drm-$(CONFIG_OF) += drm_of.o |
| 32 | drm-$(CONFIG_PCI) += drm_pci.o | 29 | drm-$(CONFIG_PCI) += drm_pci.o |
| 33 | drm-$(CONFIG_DEBUG_FS) += drm_debugfs.o drm_debugfs_crc.o | 30 | drm-$(CONFIG_DEBUG_FS) += drm_debugfs.o drm_debugfs_crc.o |
| 34 | drm-$(CONFIG_DRM_LOAD_EDID_FIRMWARE) += drm_edid_load.o | 31 | drm-$(CONFIG_DRM_LOAD_EDID_FIRMWARE) += drm_edid_load.o |
| 32 | drm-$(CONFIG_DRM_PRIVACY_SCREEN) += drm_privacy_screen.o drm_privacy_screen_x86.o | ||
| 35 | 33 | ||
| 36 | obj-$(CONFIG_DRM_DP_AUX_BUS) += drm_dp_aux_bus.o | 34 | obj-$(CONFIG_DRM_DP_AUX_BUS) += drm_dp_aux_bus.o |
| 37 | 35 | ||
| 36 | obj-$(CONFIG_DRM_NOMODESET) += drm_nomodeset.o | ||
| 37 | |||
| 38 | drm_cma_helper-y := drm_gem_cma_helper.o | ||
| 39 | drm_cma_helper-$(CONFIG_DRM_KMS_HELPER) += drm_fb_cma_helper.o | ||
| 40 | obj-$(CONFIG_DRM_GEM_CMA_HELPER) += drm_cma_helper.o | ||
| 41 | |||
| 42 | drm_shmem_helper-y := drm_gem_shmem_helper.o | ||
| 43 | obj-$(CONFIG_DRM_GEM_SHMEM_HELPER) += drm_shmem_helper.o | ||
| 44 | |||
| 38 | drm_vram_helper-y := drm_gem_vram_helper.o | 45 | drm_vram_helper-y := drm_gem_vram_helper.o |
| 39 | obj-$(CONFIG_DRM_VRAM_HELPER) += drm_vram_helper.o | 46 | obj-$(CONFIG_DRM_VRAM_HELPER) += drm_vram_helper.o |
| 40 | 47 | ||
| @@ -42,18 +49,18 @@ drm_ttm_helper-y := drm_gem_ttm_helper.o · 1 unmodified line | |||
| 42 | obj-$(CONFIG_DRM_TTM_HELPER) += drm_ttm_helper.o | 49 | obj-$(CONFIG_DRM_TTM_HELPER) += drm_ttm_helper.o |
| 43 | 50 | ||
| 44 | drm_kms_helper-y := drm_bridge_connector.o drm_crtc_helper.o drm_dp_helper.o \ | 51 | drm_kms_helper-y := drm_bridge_connector.o drm_crtc_helper.o drm_dp_helper.o \ |
| 45 | drm_dsc.o drm_probe_helper.o \ | 52 | drm_dsc.o drm_encoder_slave.o drm_flip_work.o drm_hdcp.o \ |
| 53 | drm_probe_helper.o \ | ||
| 46 | drm_plane_helper.o drm_dp_mst_topology.o drm_atomic_helper.o \ | 54 | drm_plane_helper.o drm_dp_mst_topology.o drm_atomic_helper.o \ |
| 47 | drm_kms_helper_common.o drm_dp_dual_mode_helper.o \ | 55 | drm_kms_helper_common.o drm_dp_dual_mode_helper.o \ |
| 48 | drm_simple_kms_helper.o drm_modeset_helper.o \ | 56 | drm_simple_kms_helper.o drm_modeset_helper.o \ |
| 49 | drm_scdc_helper.o drm_gem_atomic_helper.o \ | 57 | drm_scdc_helper.o drm_gem_atomic_helper.o \ |
| 50 | drm_gem_framebuffer_helper.o \ | 58 | drm_gem_framebuffer_helper.o \ |
| 51 | drm_atomic_state_helper.o drm_damage_helper.o \ | 59 | drm_atomic_state_helper.o drm_damage_helper.o \ |
| 52 | drm_format_helper.o drm_self_refresh_helper.o | 60 | drm_format_helper.o drm_self_refresh_helper.o drm_rect.o |
| 53 | 61 | ||
| 54 | drm_kms_helper-$(CONFIG_DRM_PANEL_BRIDGE) += bridge/panel.o | 62 | drm_kms_helper-$(CONFIG_DRM_PANEL_BRIDGE) += bridge/panel.o |
| 55 | drm_kms_helper-$(CONFIG_DRM_FBDEV_EMULATION) += drm_fb_helper.o | 63 | drm_kms_helper-$(CONFIG_DRM_FBDEV_EMULATION) += drm_fb_helper.o |
| 56 | drm_kms_helper-$(CONFIG_DRM_KMS_CMA_HELPER) += drm_fb_cma_helper.o | ||
| 57 | drm_kms_helper-$(CONFIG_DRM_DP_AUX_CHARDEV) += drm_dp_aux_dev.o | 64 | drm_kms_helper-$(CONFIG_DRM_DP_AUX_CHARDEV) += drm_dp_aux_dev.o |
| 58 | drm_kms_helper-$(CONFIG_DRM_DP_CEC) += drm_dp_cec.o | 65 | drm_kms_helper-$(CONFIG_DRM_DP_CEC) += drm_dp_cec.o |
| 59 | 66 | ||
| @@ -127,3 +134,4 @@ obj-$(CONFIG_DRM_TIDSS) += tidss/ · 67 unmodified lines | |||
| 127 | obj-y += xlnx/ | 134 | obj-y += xlnx/ |
| 128 | obj-y += gud/ | 135 | obj-y += gud/ |
| 129 | obj-$(CONFIG_DRM_HYPERV) += hyperv/ | 136 | obj-$(CONFIG_DRM_HYPERV) += hyperv/ |
| 137 | obj-$(CONFIG_DRM_SPRD) += sprd/ | ||
| @@ -45,7 +45,7 @@ amdgpu-y += amdgpu_device.o amdgpu_kms.o \ · 44 unmodified lines | |||
| 45 | amdgpu_atombios.o atombios_crtc.o amdgpu_connectors.o \ | 45 | amdgpu_atombios.o atombios_crtc.o amdgpu_connectors.o \ |
| 46 | atom.o amdgpu_fence.o amdgpu_ttm.o amdgpu_object.o amdgpu_gart.o \ | 46 | atom.o amdgpu_fence.o amdgpu_ttm.o amdgpu_object.o amdgpu_gart.o \ |
| 47 | amdgpu_encoders.o amdgpu_display.o amdgpu_i2c.o \ | 47 | amdgpu_encoders.o amdgpu_display.o amdgpu_i2c.o \ |
| 48 | amdgpu_fb.o amdgpu_gem.o amdgpu_ring.o \ | 48 | amdgpu_gem.o amdgpu_ring.o \ |
| 49 | amdgpu_cs.o amdgpu_bios.o amdgpu_benchmark.o amdgpu_test.o \ | 49 | amdgpu_cs.o amdgpu_bios.o amdgpu_benchmark.o amdgpu_test.o \ |
| 50 | atombios_dp.o amdgpu_afmt.o amdgpu_trace_points.o \ | 50 | atombios_dp.o amdgpu_afmt.o amdgpu_trace_points.o \ |
| 51 | atombios_encoders.o amdgpu_sa.o atombios_i2c.o \ | 51 | atombios_encoders.o amdgpu_sa.o atombios_i2c.o \ |
| @@ -458,7 +458,6 @@ struct amdgpu_flip_work { · 457 unmodified lines | |||
| 458 | uint64_t base; | 458 | uint64_t base; |
| 459 | struct drm_pending_vblank_event *event; | 459 | struct drm_pending_vblank_event *event; |
| 460 | struct amdgpu_bo *old_abo; | 460 | struct amdgpu_bo *old_abo; |
| 461 | struct dma_fence *excl; | ||
| 462 | unsigned shared_count; | 461 | unsigned shared_count; |
| 463 | struct dma_fence **shared; | 462 | struct dma_fence **shared; |
| 464 | struct dma_fence_cb cb; | 463 | struct dma_fence_cb cb; |
| @@ -813,6 +812,7 @@ struct amd_powerplay { · 348 unmodified lines | |||
| 813 | 812 | ||
| 814 | #define AMDGPU_RESET_MAGIC_NUM 64 | 813 | #define AMDGPU_RESET_MAGIC_NUM 64 |
| 815 | #define AMDGPU_MAX_DF_PERFMONS 4 | 814 | #define AMDGPU_MAX_DF_PERFMONS 4 |
| 815 | #define AMDGPU_PRODUCT_NAME_LEN 64 | ||
| 816 | struct amdgpu_device { | 816 | struct amdgpu_device { |
| 817 | struct device *dev; | 817 | struct device *dev; |
| 818 | struct pci_dev *pdev; | 818 | struct pci_dev *pdev; |
| @@ -1084,7 +1084,7 @@ struct amdgpu_device { · 265 unmodified lines | |||
| 1084 | 1084 | ||
| 1085 | /* Chip product information */ | 1085 | /* Chip product information */ |
| 1086 | char product_number[16]; | 1086 | char product_number[16]; |
| 1087 | char product_name[32]; | 1087 | char product_name[AMDGPU_PRODUCT_NAME_LEN]; |
| 1088 | char serial[20]; | 1088 | char serial[20]; |
| 1089 | 1089 | ||
| 1090 | atomic_t throttling_logging_enabled; | 1090 | atomic_t throttling_logging_enabled; |
| @@ -1097,7 +1097,9 @@ struct amdgpu_device { · 6 unmodified lines | |||
| 1097 | pci_channel_state_t pci_channel_state; | 1097 | pci_channel_state_t pci_channel_state; |
| 1098 | 1098 | ||
| 1099 | struct amdgpu_reset_control *reset_cntl; | 1099 | struct amdgpu_reset_control *reset_cntl; |
| 1100 | uint32_t ip_versions[HW_ID_MAX][HWIP_MAX_INSTANCE]; | 1100 | uint32_t ip_versions[MAX_HWIP][HWIP_MAX_INSTANCE]; |
| 1101 | |||
| 1102 | bool ram_is_direct_mapped; | ||
| 1101 | }; | 1103 | }; |
| 1102 | 1104 | ||
| 1103 | static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) | 1105 | static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) |
| @@ -1318,6 +1320,8 @@ void amdgpu_device_flush_hdp(struct amdgpu_device *adev, · 214 unmodified lines | |||
| 1318 | void amdgpu_device_invalidate_hdp(struct amdgpu_device *adev, | 1320 | void amdgpu_device_invalidate_hdp(struct amdgpu_device *adev, |
| 1319 | struct amdgpu_ring *ring); | 1321 | struct amdgpu_ring *ring); |
| 1320 | 1322 | ||
| 1323 | void amdgpu_device_halt(struct amdgpu_device *adev); | ||
| 1324 | |||
| 1321 | /* atpx handler */ | 1325 | /* atpx handler */ |
| 1322 | #if defined(CONFIG_VGA_SWITCHEROO) | 1326 | #if defined(CONFIG_VGA_SWITCHEROO) |
| 1323 | void amdgpu_register_atpx_handler(void); | 1327 | void amdgpu_register_atpx_handler(void); |
| @@ -1361,8 +1365,6 @@ int amdgpu_device_resume(struct drm_device *dev, bool fbcon); · 37 unmodified lines | |||
| 1361 | u32 amdgpu_get_vblank_counter_kms(struct drm_crtc *crtc); | 1365 | u32 amdgpu_get_vblank_counter_kms(struct drm_crtc *crtc); |
| 1362 | int amdgpu_enable_vblank_kms(struct drm_crtc *crtc); | 1366 | int amdgpu_enable_vblank_kms(struct drm_crtc *crtc); |
| 1363 | void amdgpu_disable_vblank_kms(struct drm_crtc *crtc); | 1367 | void amdgpu_disable_vblank_kms(struct drm_crtc *crtc); |
| 1364 | long amdgpu_kms_compat_ioctl(struct file *filp, unsigned int cmd, | ||
| 1365 | unsigned long arg); | ||
| 1366 | int amdgpu_info_ioctl(struct drm_device *dev, void *data, | 1368 | int amdgpu_info_ioctl(struct drm_device *dev, void *data, |
| 1367 | struct drm_file *filp); | 1369 | struct drm_file *filp); |
| 1368 | 1370 | ||
| @@ -72,7 +72,7 @@ void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev) · 71 unmodified lines | |||
| 72 | if (!kfd_initialized) | 72 | if (!kfd_initialized) |
| 73 | return; | 73 | return; |
| 74 | 74 | ||
| 75 | adev->kfd.dev = kgd2kfd_probe((struct kgd_dev *)adev, vf); | 75 | adev->kfd.dev = kgd2kfd_probe(adev, vf); |
| 76 | 76 | ||
| 77 | if (adev->kfd.dev) | 77 | if (adev->kfd.dev) |
| 78 | amdgpu_amdkfd_total_mem_size += adev->gmc.real_vram_size; | 78 | amdgpu_amdkfd_total_mem_size += adev->gmc.real_vram_size; |
| @@ -233,19 +233,16 @@ int amdgpu_amdkfd_post_reset(struct amdgpu_device *adev) · 154 unmodified lines | |||
| 233 | return r; | 233 | return r; |
| 234 | } | 234 | } |
| 235 | 235 | ||
| 236 | void amdgpu_amdkfd_gpu_reset(struct kgd_dev *kgd) | 236 | void amdgpu_amdkfd_gpu_reset(struct amdgpu_device *adev) |
| 237 | { | 237 | { |
| 238 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 239 | |||
| 240 | if (amdgpu_device_should_recover_gpu(adev)) | 238 | if (amdgpu_device_should_recover_gpu(adev)) |
| 241 | amdgpu_device_gpu_recover(adev, NULL); | 239 | amdgpu_device_gpu_recover(adev, NULL); |
| 242 | } | 240 | } |
| 243 | 241 | ||
| 244 | int amdgpu_amdkfd_alloc_gtt_mem(struct kgd_dev *kgd, size_t size, | 242 | int amdgpu_amdkfd_alloc_gtt_mem(struct amdgpu_device *adev, size_t size, |
| 245 | void **mem_obj, uint64_t *gpu_addr, | 243 | void **mem_obj, uint64_t *gpu_addr, |
| 246 | void **cpu_ptr, bool cp_mqd_gfx9) | 244 | void **cpu_ptr, bool cp_mqd_gfx9) |
| 247 | { | 245 | { |
| 248 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 249 | struct amdgpu_bo *bo = NULL; | 246 | struct amdgpu_bo *bo = NULL; |
| 250 | struct amdgpu_bo_param bp; | 247 | struct amdgpu_bo_param bp; |
| 251 | int r; | 248 | int r; |
| @@ -314,7 +311,7 @@ allocate_mem_reserve_bo_failed: · 62 unmodified lines | |||
| 314 | return r; | 311 | return r; |
| 315 | } | 312 | } |
| 316 | 313 | ||
| 317 | void amdgpu_amdkfd_free_gtt_mem(struct kgd_dev *kgd, void *mem_obj) | 314 | void amdgpu_amdkfd_free_gtt_mem(struct amdgpu_device *adev, void *mem_obj) |
| 318 | { | 315 | { |
| 319 | struct amdgpu_bo *bo = (struct amdgpu_bo *) mem_obj; | 316 | struct amdgpu_bo *bo = (struct amdgpu_bo *) mem_obj; |
| 320 | 317 | ||
| @@ -325,10 +322,9 @@ void amdgpu_amdkfd_free_gtt_mem(struct kgd_dev *kgd, void *mem_obj) · 4 unmodified lines | |||
| 325 | amdgpu_bo_unref(&(bo)); | 322 | amdgpu_bo_unref(&(bo)); |
| 326 | } | 323 | } |
| 327 | 324 | ||
| 328 | int amdgpu_amdkfd_alloc_gws(struct kgd_dev *kgd, size_t size, | 325 | int amdgpu_amdkfd_alloc_gws(struct amdgpu_device *adev, size_t size, |
| 329 | void **mem_obj) | 326 | void **mem_obj) |
| 330 | { | 327 | { |
| 331 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 332 | struct amdgpu_bo *bo = NULL; | 328 | struct amdgpu_bo *bo = NULL; |
| 333 | struct amdgpu_bo_user *ubo; | 329 | struct amdgpu_bo_user *ubo; |
| 334 | struct amdgpu_bo_param bp; | 330 | struct amdgpu_bo_param bp; |
| @@ -355,18 +351,16 @@ int amdgpu_amdkfd_alloc_gws(struct kgd_dev *kgd, size_t size, · 20 unmodified lines | |||
| 355 | return 0; | 351 | return 0; |
| 356 | } | 352 | } |
| 357 | 353 | ||
| 358 | void amdgpu_amdkfd_free_gws(struct kgd_dev *kgd, void *mem_obj) | 354 | void amdgpu_amdkfd_free_gws(struct amdgpu_device *adev, void *mem_obj) |
| 359 | { | 355 | { |
| 360 | struct amdgpu_bo *bo = (struct amdgpu_bo *)mem_obj; | 356 | struct amdgpu_bo *bo = (struct amdgpu_bo *)mem_obj; |
| 361 | 357 | ||
| 362 | amdgpu_bo_unref(&bo); | 358 | amdgpu_bo_unref(&bo); |
| 363 | } | 359 | } |
| 364 | 360 | ||
| 365 | uint32_t amdgpu_amdkfd_get_fw_version(struct kgd_dev *kgd, | 361 | uint32_t amdgpu_amdkfd_get_fw_version(struct amdgpu_device *adev, |
| 366 | enum kgd_engine_type type) | 362 | enum kgd_engine_type type) |
| 367 | { | 363 | { |
| 368 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 369 | |||
| 370 | switch (type) { | 364 | switch (type) { |
| 371 | case KGD_ENGINE_PFP: | 365 | case KGD_ENGINE_PFP: |
| 372 | return adev->gfx.pfp_fw_version; | 366 | return adev->gfx.pfp_fw_version; |
| @@ -399,11 +393,9 @@ uint32_t amdgpu_amdkfd_get_fw_version(struct kgd_dev *kgd, · 26 unmodified lines | |||
| 399 | return 0; | 393 | return 0; |
| 400 | } | 394 | } |
| 401 | 395 | ||
| 402 | void amdgpu_amdkfd_get_local_mem_info(struct kgd_dev *kgd, | 396 | void amdgpu_amdkfd_get_local_mem_info(struct amdgpu_device *adev, |
| 403 | struct kfd_local_mem_info *mem_info) | 397 | struct kfd_local_mem_info *mem_info) |
| 404 | { | 398 | { |
| 405 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 406 | |||
| 407 | memset(mem_info, 0, sizeof(*mem_info)); | 399 | memset(mem_info, 0, sizeof(*mem_info)); |
| 408 | 400 | ||
| 409 | mem_info->local_mem_size_public = adev->gmc.visible_vram_size; | 401 | mem_info->local_mem_size_public = adev->gmc.visible_vram_size; |
| @@ -428,19 +420,15 @@ void amdgpu_amdkfd_get_local_mem_info(struct kgd_dev *kgd, · 18 unmodified lines | |||
| 428 | mem_info->mem_clk_max = 100; | 420 | mem_info->mem_clk_max = 100; |
| 429 | } | 421 | } |
| 430 | 422 | ||
| 431 | uint64_t amdgpu_amdkfd_get_gpu_clock_counter(struct kgd_dev *kgd) | 423 | uint64_t amdgpu_amdkfd_get_gpu_clock_counter(struct amdgpu_device *adev) |
| 432 | { | 424 | { |
| 433 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 434 | |||
| 435 | if (adev->gfx.funcs->get_gpu_clock_counter) | 425 | if (adev->gfx.funcs->get_gpu_clock_counter) |
| 436 | return adev->gfx.funcs->get_gpu_clock_counter(adev); | 426 | return adev->gfx.funcs->get_gpu_clock_counter(adev); |
| 437 | return 0; | 427 | return 0; |
| 438 | } | 428 | } |
| 439 | 429 | ||
| 440 | uint32_t amdgpu_amdkfd_get_max_engine_clock_in_mhz(struct kgd_dev *kgd) | 430 | uint32_t amdgpu_amdkfd_get_max_engine_clock_in_mhz(struct amdgpu_device *adev) |
| 441 | { | 431 | { |
| 442 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 443 | |||
| 444 | /* the sclk is in quantas of 10kHz */ | 432 | /* the sclk is in quantas of 10kHz */ |
| 445 | if (amdgpu_sriov_vf(adev)) | 433 | if (amdgpu_sriov_vf(adev)) |
| 446 | return adev->clock.default_sclk / 100; | 434 | return adev->clock.default_sclk / 100; |
| @@ -450,9 +438,8 @@ uint32_t amdgpu_amdkfd_get_max_engine_clock_in_mhz(struct kgd_dev *kgd) · 3 unmodified lines | |||
| 450 | return 100; | 438 | return 100; |
| 451 | } | 439 | } |
| 452 | 440 | ||
| 453 | void amdgpu_amdkfd_get_cu_info(struct kgd_dev *kgd, struct kfd_cu_info *cu_info) | 441 | void amdgpu_amdkfd_get_cu_info(struct amdgpu_device *adev, struct kfd_cu_info *cu_info) |
| 454 | { | 442 | { |
| 455 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 456 | struct amdgpu_cu_info acu_info = adev->gfx.cu_info; | 443 | struct amdgpu_cu_info acu_info = adev->gfx.cu_info; |
| 457 | 444 | ||
| 458 | memset(cu_info, 0, sizeof(*cu_info)); | 445 | memset(cu_info, 0, sizeof(*cu_info)); |
| @@ -473,13 +460,12 @@ void amdgpu_amdkfd_get_cu_info(struct kgd_dev *kgd, struct kfd_cu_info *cu_info) · 14 unmodified lines | |||
| 473 | cu_info->lds_size = acu_info.lds_size; | 460 | cu_info->lds_size = acu_info.lds_size; |
| 474 | } | 461 | } |
| 475 | 462 | ||
| 476 | int amdgpu_amdkfd_get_dmabuf_info(struct kgd_dev *kgd, int dma_buf_fd, | 463 | int amdgpu_amdkfd_get_dmabuf_info(struct amdgpu_device *adev, int dma_buf_fd, |
| 477 | struct kgd_dev **dma_buf_kgd, | 464 | struct amdgpu_device **dmabuf_adev, |
| 478 | uint64_t *bo_size, void *metadata_buffer, | 465 | uint64_t *bo_size, void *metadata_buffer, |
| 479 | size_t buffer_size, uint32_t *metadata_size, | 466 | size_t buffer_size, uint32_t *metadata_size, |
| 480 | uint32_t *flags) | 467 | uint32_t *flags) |
| 481 | { | 468 | { |
| 482 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 483 | struct dma_buf *dma_buf; | 469 | struct dma_buf *dma_buf; |
| 484 | struct drm_gem_object *obj; | 470 | struct drm_gem_object *obj; |
| 485 | struct amdgpu_bo *bo; | 471 | struct amdgpu_bo *bo; |
| @@ -507,8 +493,8 @@ int amdgpu_amdkfd_get_dmabuf_info(struct kgd_dev *kgd, int dma_buf_fd, · 21 unmodified lines | |||
| 507 | goto out_put; | 493 | goto out_put; |
| 508 | 494 | ||
| 509 | r = 0; | 495 | r = 0; |
| 510 | if (dma_buf_kgd) | 496 | if (dmabuf_adev) |
| 511 | *dma_buf_kgd = (struct kgd_dev *)adev; | 497 | *dmabuf_adev = adev; |
| 512 | if (bo_size) | 498 | if (bo_size) |
| 513 | *bo_size = amdgpu_bo_size(bo); | 499 | *bo_size = amdgpu_bo_size(bo); |
| 514 | if (metadata_buffer) | 500 | if (metadata_buffer) |
| @@ -528,32 +514,18 @@ out_put: · 13 unmodified lines | |||
| 528 | return r; | 514 | return r; |
| 529 | } | 515 | } |
| 530 | 516 | ||
| 531 | uint64_t amdgpu_amdkfd_get_vram_usage(struct kgd_dev *kgd) | 517 | uint64_t amdgpu_amdkfd_get_vram_usage(struct amdgpu_device *adev) |
| 532 | { | 518 | { |
| 533 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 534 | struct ttm_resource_manager *vram_man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM); | 519 | struct ttm_resource_manager *vram_man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM); |
| 535 | 520 | ||
| 536 | return amdgpu_vram_mgr_usage(vram_man); | 521 | return amdgpu_vram_mgr_usage(vram_man); |
| 537 | } | 522 | } |
| 538 | 523 | ||
| 539 | uint64_t amdgpu_amdkfd_get_hive_id(struct kgd_dev *kgd) | 524 | uint8_t amdgpu_amdkfd_get_xgmi_hops_count(struct amdgpu_device *dst, |
| 540 | { | 525 | struct amdgpu_device *src) |
| 541 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 542 | |||
| 543 | return adev->gmc.xgmi.hive_id; | ||
| 544 | } | ||
| 545 | |||
| 546 | uint64_t amdgpu_amdkfd_get_unique_id(struct kgd_dev *kgd) | ||
| 547 | { | 526 | { |
| 548 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | 527 | struct amdgpu_device *peer_adev = src; |
| 549 | 528 | struct amdgpu_device *adev = dst; | |
| 550 | return adev->unique_id; | ||
| 551 | } | ||
| 552 | |||
| 553 | uint8_t amdgpu_amdkfd_get_xgmi_hops_count(struct kgd_dev *dst, struct kgd_dev *src) | ||
| 554 | { | ||
| 555 | struct amdgpu_device *peer_adev = (struct amdgpu_device *)src; | ||
| 556 | struct amdgpu_device *adev = (struct amdgpu_device *)dst; | ||
| 557 | int ret = amdgpu_xgmi_get_hops_count(adev, peer_adev); | 529 | int ret = amdgpu_xgmi_get_hops_count(adev, peer_adev); |
| 558 | 530 | ||
| 559 | if (ret < 0) { | 531 | if (ret < 0) { |
| @@ -565,16 +537,18 @@ uint8_t amdgpu_amdkfd_get_xgmi_hops_count(struct kgd_dev *dst, struct kgd_dev *s · 5 unmodified lines | |||
| 565 | return (uint8_t)ret; | 537 | return (uint8_t)ret; |
| 566 | } | 538 | } |
| 567 | 539 | ||
| 568 | int amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(struct kgd_dev *dst, struct kgd_dev *src, bool is_min) | 540 | int amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(struct amdgpu_device *dst, |
| 541 | struct amdgpu_device *src, | ||
| 542 | bool is_min) | ||
| 569 | { | 543 | { |
| 570 | struct amdgpu_device *adev = (struct amdgpu_device *)dst, *peer_adev; | 544 | struct amdgpu_device *adev = dst, *peer_adev; |
| 571 | int num_links; | 545 | int num_links; |
| 572 | 546 | ||
| 573 | if (adev->asic_type != CHIP_ALDEBARAN) | 547 | if (adev->asic_type != CHIP_ALDEBARAN) |
| 574 | return 0; | 548 | return 0; |
| 575 | 549 | ||
| 576 | if (src) | 550 | if (src) |
| 577 | peer_adev = (struct amdgpu_device *)src; | 551 | peer_adev = src; |
| 578 | 552 | ||
| 579 | /* num links returns 0 for indirect peers since indirect route is unknown. */ | 553 | /* num links returns 0 for indirect peers since indirect route is unknown. */ |
| 580 | num_links = is_min ? 1 : amdgpu_xgmi_get_num_links(adev, peer_adev); | 554 | num_links = is_min ? 1 : amdgpu_xgmi_get_num_links(adev, peer_adev); |
| @@ -589,9 +563,8 @@ int amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(struct kgd_dev *dst, struct kgd_dev · 8 unmodified lines | |||
| 589 | return (num_links * 16 * 25000)/BITS_PER_BYTE; | 563 | return (num_links * 16 * 25000)/BITS_PER_BYTE; |
| 590 | } | 564 | } |
| 591 | 565 | ||
| 592 | int amdgpu_amdkfd_get_pcie_bandwidth_mbytes(struct kgd_dev *dev, bool is_min) | 566 | int amdgpu_amdkfd_get_pcie_bandwidth_mbytes(struct amdgpu_device *adev, bool is_min) |
| 593 | { | 567 | { |
| 594 | struct amdgpu_device *adev = (struct amdgpu_device *)dev; | ||
| 595 | int num_lanes_shift = (is_min ? ffs(adev->pm.pcie_mlw_mask) : | 568 | int num_lanes_shift = (is_min ? ffs(adev->pm.pcie_mlw_mask) : |
| 596 | fls(adev->pm.pcie_mlw_mask)) - 1; | 569 | fls(adev->pm.pcie_mlw_mask)) - 1; |
| 597 | int gen_speed_shift = (is_min ? ffs(adev->pm.pcie_gen_mask & | 570 | int gen_speed_shift = (is_min ? ffs(adev->pm.pcie_gen_mask & |
| @@ -647,39 +620,11 @@ int amdgpu_amdkfd_get_pcie_bandwidth_mbytes(struct kgd_dev *dev, bool is_min) · 49 unmodified lines | |||
| 647 | return (num_lanes_factor * gen_speed_mbits_factor)/BITS_PER_BYTE; | 620 | return (num_lanes_factor * gen_speed_mbits_factor)/BITS_PER_BYTE; |
| 648 | } | 621 | } |
| 649 | 622 | ||
| 650 | uint64_t amdgpu_amdkfd_get_mmio_remap_phys_addr(struct kgd_dev *kgd) | 623 | int amdgpu_amdkfd_submit_ib(struct amdgpu_device *adev, |
| 651 | { | 624 | enum kgd_engine_type engine, |
| 652 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 653 | |||
| 654 | return adev->rmmio_remap.bus_addr; | ||
| 655 | } | ||
| 656 | |||
| 657 | uint32_t amdgpu_amdkfd_get_num_gws(struct kgd_dev *kgd) | ||
| 658 | { | ||
| 659 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 660 | |||
| 661 | return adev->gds.gws_size; | ||
| 662 | } | ||
| 663 | |||
| 664 | uint32_t amdgpu_amdkfd_get_asic_rev_id(struct kgd_dev *kgd) | ||
| 665 | { | ||
| 666 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 667 | |||
| 668 | return adev->rev_id; | ||
| 669 | } | ||
| 670 | |||
| 671 | int amdgpu_amdkfd_get_noretry(struct kgd_dev *kgd) | ||
| 672 | { | ||
| 673 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 674 | |||
| 675 | return adev->gmc.noretry; | ||
| 676 | } | ||
| 677 | |||
| 678 | int amdgpu_amdkfd_submit_ib(struct kgd_dev *kgd, enum kgd_engine_type engine, | ||
| 679 | uint32_t vmid, uint64_t gpu_addr, | 625 | uint32_t vmid, uint64_t gpu_addr, |
| 680 | uint32_t *ib_cmd, uint32_t ib_len) | 626 | uint32_t *ib_cmd, uint32_t ib_len) |
| 681 | { | 627 | { |
| 682 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 683 | struct amdgpu_job *job; | 628 | struct amdgpu_job *job; |
| 684 | struct amdgpu_ib *ib; | 629 | struct amdgpu_ib *ib; |
| 685 | struct amdgpu_ring *ring; | 630 | struct amdgpu_ring *ring; |
| @@ -730,10 +675,8 @@ err: · 44 unmodified lines | |||
| 730 | return ret; | 675 | return ret; |
| 731 | } | 676 | } |
| 732 | 677 | ||
| 733 | void amdgpu_amdkfd_set_compute_idle(struct kgd_dev *kgd, bool idle) | 678 | void amdgpu_amdkfd_set_compute_idle(struct amdgpu_device *adev, bool idle) |
| 734 | { | 679 | { |
| 735 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 736 | |||
| 737 | amdgpu_dpm_switch_power_profile(adev, | 680 | amdgpu_dpm_switch_power_profile(adev, |
| 738 | PP_SMC_POWER_PROFILE_COMPUTE, | 681 | PP_SMC_POWER_PROFILE_COMPUTE, |
| 739 | !idle); | 682 | !idle); |
| @@ -747,10 +690,9 @@ bool amdgpu_amdkfd_is_kfd_vmid(struct amdgpu_device *adev, u32 vmid) · 7 unmodified lines | |||
| 747 | return false; | 690 | return false; |
| 748 | } | 691 | } |
| 749 | 692 | ||
| 750 | int amdgpu_amdkfd_flush_gpu_tlb_vmid(struct kgd_dev *kgd, uint16_t vmid) | 693 | int amdgpu_amdkfd_flush_gpu_tlb_vmid(struct amdgpu_device *adev, |
| 694 | uint16_t vmid) | ||
| 751 | { | 695 | { |
| 752 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 753 | |||
| 754 | if (adev->family == AMDGPU_FAMILY_AI) { | 696 | if (adev->family == AMDGPU_FAMILY_AI) { |
| 755 | int i; | 697 | int i; |
| 756 | 698 | ||
| @@ -763,10 +705,9 @@ int amdgpu_amdkfd_flush_gpu_tlb_vmid(struct kgd_dev *kgd, uint16_t vmid) · 6 unmodified lines | |||
| 763 | return 0; | 705 | return 0; |
| 764 | } | 706 | } |
| 765 | 707 | ||
| 766 | int amdgpu_amdkfd_flush_gpu_tlb_pasid(struct kgd_dev *kgd, uint16_t pasid, | 708 | int amdgpu_amdkfd_flush_gpu_tlb_pasid(struct amdgpu_device *adev, |
| 767 | enum TLB_FLUSH_TYPE flush_type) | 709 | uint16_t pasid, enum TLB_FLUSH_TYPE flush_type) |
| 768 | { | 710 | { |
| 769 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 770 | bool all_hub = false; | 711 | bool all_hub = false; |
| 771 | 712 | ||
| 772 | if (adev->family == AMDGPU_FAMILY_AI) | 713 | if (adev->family == AMDGPU_FAMILY_AI) |
| @@ -775,21 +716,18 @@ int amdgpu_amdkfd_flush_gpu_tlb_pasid(struct kgd_dev *kgd, uint16_t pasid, · 2 unmodified lines | |||
| 775 | return amdgpu_gmc_flush_gpu_tlb_pasid(adev, pasid, flush_type, all_hub); | 716 | return amdgpu_gmc_flush_gpu_tlb_pasid(adev, pasid, flush_type, all_hub); |
| 776 | } | 717 | } |
| 777 | 718 | ||
| 778 | bool amdgpu_amdkfd_have_atomics_support(struct kgd_dev *kgd) | 719 | bool amdgpu_amdkfd_have_atomics_support(struct amdgpu_device *adev) |
| 779 | { | 720 | { |
| 780 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 781 | |||
| 782 | return adev->have_atomics_support; | 721 | return adev->have_atomics_support; |
| 783 | } | 722 | } |
| 784 | 723 | ||
| 785 | void amdgpu_amdkfd_ras_poison_consumption_handler(struct kgd_dev *kgd) | 724 | void amdgpu_amdkfd_ras_poison_consumption_handler(struct amdgpu_device *adev, bool reset) |
| 786 | { | 725 | { |
| 787 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 788 | struct ras_err_data err_data = {0, 0, 0, NULL}; | 726 | struct ras_err_data err_data = {0, 0, 0, NULL}; |
| 789 | 727 | ||
| 790 | /* CPU MCA will handle page retirement if connected_to_cpu is 1 */ | 728 | /* CPU MCA will handle page retirement if connected_to_cpu is 1 */ |
| 791 | if (!adev->gmc.xgmi.connected_to_cpu) | 729 | if (!adev->gmc.xgmi.connected_to_cpu) |
| 792 | amdgpu_umc_process_ras_data_cb(adev, &err_data, NULL); | 730 | amdgpu_umc_poison_handler(adev, &err_data, reset); |
| 793 | else | 731 | else if (reset) |
| 794 | amdgpu_amdkfd_gpu_reset(kgd); | 732 | amdgpu_amdkfd_gpu_reset(adev); |
| 795 | } | 733 | } |
| @@ -144,14 +144,16 @@ void amdgpu_amdkfd_interrupt(struct amdgpu_device *adev, · 143 unmodified lines | |||
| 144 | void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev); | 144 | void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev); |
| 145 | void amdgpu_amdkfd_device_init(struct amdgpu_device *adev); | 145 | void amdgpu_amdkfd_device_init(struct amdgpu_device *adev); |
| 146 | void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev); | 146 | void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev); |
| 147 | int amdgpu_amdkfd_submit_ib(struct kgd_dev *kgd, enum kgd_engine_type engine, | 147 | int amdgpu_amdkfd_submit_ib(struct amdgpu_device *adev, |
| 148 | enum kgd_engine_type engine, | ||
| 148 | uint32_t vmid, uint64_t gpu_addr, | 149 | uint32_t vmid, uint64_t gpu_addr, |
| 149 | uint32_t *ib_cmd, uint32_t ib_len); | 150 | uint32_t *ib_cmd, uint32_t ib_len); |
| 150 | void amdgpu_amdkfd_set_compute_idle(struct kgd_dev *kgd, bool idle); | 151 | void amdgpu_amdkfd_set_compute_idle(struct amdgpu_device *adev, bool idle); |
| 151 | bool amdgpu_amdkfd_have_atomics_support(struct kgd_dev *kgd); | 152 | bool amdgpu_amdkfd_have_atomics_support(struct amdgpu_device *adev); |
| 152 | int amdgpu_amdkfd_flush_gpu_tlb_vmid(struct kgd_dev *kgd, uint16_t vmid); | 153 | int amdgpu_amdkfd_flush_gpu_tlb_vmid(struct amdgpu_device *adev, |
| 153 | int amdgpu_amdkfd_flush_gpu_tlb_pasid(struct kgd_dev *kgd, uint16_t pasid, | 154 | uint16_t vmid); |
| 154 | enum TLB_FLUSH_TYPE flush_type); | 155 | int amdgpu_amdkfd_flush_gpu_tlb_pasid(struct amdgpu_device *adev, |
| 156 | uint16_t pasid, enum TLB_FLUSH_TYPE flush_type); | ||
| 155 | 157 | ||
| 156 | bool amdgpu_amdkfd_is_kfd_vmid(struct amdgpu_device *adev, u32 vmid); | 158 | bool amdgpu_amdkfd_is_kfd_vmid(struct amdgpu_device *adev, u32 vmid); |
| 157 | 159 | ||
| @@ -159,7 +161,7 @@ int amdgpu_amdkfd_pre_reset(struct amdgpu_device *adev); · 1 unmodified line | |||
| 159 | 161 | ||
| 160 | int amdgpu_amdkfd_post_reset(struct amdgpu_device *adev); | 162 | int amdgpu_amdkfd_post_reset(struct amdgpu_device *adev); |
| 161 | 163 | ||
| 162 | void amdgpu_amdkfd_gpu_reset(struct kgd_dev *kgd); | 164 | void amdgpu_amdkfd_gpu_reset(struct amdgpu_device *adev); |
| 163 | 165 | ||
| 164 | int amdgpu_queue_mask_bit_to_set_resource_bit(struct amdgpu_device *adev, | 166 | int amdgpu_queue_mask_bit_to_set_resource_bit(struct amdgpu_device *adev, |
| 165 | int queue_bit); | 167 | int queue_bit); |
| @@ -198,37 +200,36 @@ int amdgpu_amdkfd_evict_userptr(struct kgd_mem *mem, struct mm_struct *mm) · 32 unmodified lines | |||
| 198 | } | 200 | } |
| 199 | #endif | 201 | #endif |
| 200 | /* Shared API */ | 202 | /* Shared API */ |
| 201 | int amdgpu_amdkfd_alloc_gtt_mem(struct kgd_dev *kgd, size_t size, | 203 | int amdgpu_amdkfd_alloc_gtt_mem(struct amdgpu_device *adev, size_t size, |
| 202 | void **mem_obj, uint64_t *gpu_addr, | 204 | void **mem_obj, uint64_t *gpu_addr, |
| 203 | void **cpu_ptr, bool mqd_gfx9); | 205 | void **cpu_ptr, bool mqd_gfx9); |
| 204 | void amdgpu_amdkfd_free_gtt_mem(struct kgd_dev *kgd, void *mem_obj); | 206 | void amdgpu_amdkfd_free_gtt_mem(struct amdgpu_device *adev, void *mem_obj); |
| 205 | int amdgpu_amdkfd_alloc_gws(struct kgd_dev *kgd, size_t size, void **mem_obj); | 207 | int amdgpu_amdkfd_alloc_gws(struct amdgpu_device *adev, size_t size, |
| 206 | void amdgpu_amdkfd_free_gws(struct kgd_dev *kgd, void *mem_obj); | 208 | void **mem_obj); |
| 209 | void amdgpu_amdkfd_free_gws(struct amdgpu_device *adev, void *mem_obj); | ||
| 207 | int amdgpu_amdkfd_add_gws_to_process(void *info, void *gws, struct kgd_mem **mem); | 210 | int amdgpu_amdkfd_add_gws_to_process(void *info, void *gws, struct kgd_mem **mem); |
| 208 | int amdgpu_amdkfd_remove_gws_from_process(void *info, void *mem); | 211 | int amdgpu_amdkfd_remove_gws_from_process(void *info, void *mem); |
| 209 | uint32_t amdgpu_amdkfd_get_fw_version(struct kgd_dev *kgd, | 212 | uint32_t amdgpu_amdkfd_get_fw_version(struct amdgpu_device *adev, |
| 210 | enum kgd_engine_type type); | 213 | enum kgd_engine_type type); |
| 211 | void amdgpu_amdkfd_get_local_mem_info(struct kgd_dev *kgd, | 214 | void amdgpu_amdkfd_get_local_mem_info(struct amdgpu_device *adev, |
| 212 | struct kfd_local_mem_info *mem_info); | 215 | struct kfd_local_mem_info *mem_info); |
| 213 | uint64_t amdgpu_amdkfd_get_gpu_clock_counter(struct kgd_dev *kgd); | 216 | uint64_t amdgpu_amdkfd_get_gpu_clock_counter(struct amdgpu_device *adev); |
| 214 | 217 | ||
| 215 | uint32_t amdgpu_amdkfd_get_max_engine_clock_in_mhz(struct kgd_dev *kgd); | 218 | uint32_t amdgpu_amdkfd_get_max_engine_clock_in_mhz(struct amdgpu_device *adev); |
| 216 | void amdgpu_amdkfd_get_cu_info(struct kgd_dev *kgd, struct kfd_cu_info *cu_info); | 219 | void amdgpu_amdkfd_get_cu_info(struct amdgpu_device *adev, |
| 217 | int amdgpu_amdkfd_get_dmabuf_info(struct kgd_dev *kgd, int dma_buf_fd, | 220 | struct kfd_cu_info *cu_info); |
| 218 | struct kgd_dev **dmabuf_kgd, | 221 | int amdgpu_amdkfd_get_dmabuf_info(struct amdgpu_device *adev, int dma_buf_fd, |
| 222 | struct amdgpu_device **dmabuf_adev, | ||
| 219 | uint64_t *bo_size, void *metadata_buffer, | 223 | uint64_t *bo_size, void *metadata_buffer, |
| 220 | size_t buffer_size, uint32_t *metadata_size, | 224 | size_t buffer_size, uint32_t *metadata_size, |
| 221 | uint32_t *flags); | 225 | uint32_t *flags); |
| 222 | uint64_t amdgpu_amdkfd_get_vram_usage(struct kgd_dev *kgd); | 226 | uint64_t amdgpu_amdkfd_get_vram_usage(struct amdgpu_device *adev); |
| 223 | uint64_t amdgpu_amdkfd_get_hive_id(struct kgd_dev *kgd); | 227 | uint8_t amdgpu_amdkfd_get_xgmi_hops_count(struct amdgpu_device *dst, |
| 224 | uint64_t amdgpu_amdkfd_get_unique_id(struct kgd_dev *kgd); | 228 | struct amdgpu_device *src); |
| 225 | uint64_t amdgpu_amdkfd_get_mmio_remap_phys_addr(struct kgd_dev *kgd); | 229 | int amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(struct amdgpu_device *dst, |
| 226 | uint32_t amdgpu_amdkfd_get_num_gws(struct kgd_dev *kgd); | 230 | struct amdgpu_device *src, |
| 227 | uint32_t amdgpu_amdkfd_get_asic_rev_id(struct kgd_dev *kgd); | 231 | bool is_min); |
| 228 | int amdgpu_amdkfd_get_noretry(struct kgd_dev *kgd); | 232 | int amdgpu_amdkfd_get_pcie_bandwidth_mbytes(struct amdgpu_device *adev, bool is_min); |
| 229 | uint8_t amdgpu_amdkfd_get_xgmi_hops_count(struct kgd_dev *dst, struct kgd_dev *src); | ||
| 230 | int amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(struct kgd_dev *dst, struct kgd_dev *src, bool is_min); | ||
| 231 | int amdgpu_amdkfd_get_pcie_bandwidth_mbytes(struct kgd_dev *dev, bool is_min); | ||
| 232 | 233 | ||
| 233 | /* Read user wptr from a specified user address space with page fault | 234 | /* Read user wptr from a specified user address space with page fault |
| 234 | * disabled. The memory must be pinned and mapped to the hardware when | 235 | * disabled. The memory must be pinned and mapped to the hardware when |
| @@ -258,45 +259,55 @@ int amdgpu_amdkfd_get_pcie_bandwidth_mbytes(struct kgd_dev *dev, bool is_min); · 23 unmodified lines | |||
| 258 | (&((struct amdgpu_fpriv *) \ | 259 | (&((struct amdgpu_fpriv *) \ |
| 259 | ((struct drm_file *)(drm_priv))->driver_priv)->vm) | 260 | ((struct drm_file *)(drm_priv))->driver_priv)->vm) |
| 260 | 261 | ||
| 261 | int amdgpu_amdkfd_gpuvm_acquire_process_vm(struct kgd_dev *kgd, | 262 | int amdgpu_amdkfd_gpuvm_acquire_process_vm(struct amdgpu_device *adev, |
| 262 | struct file *filp, u32 pasid, | 263 | struct file *filp, u32 pasid, |
| 263 | void **process_info, | 264 | void **process_info, |
| 264 | struct dma_fence **ef); | 265 | struct dma_fence **ef); |
| 265 | void amdgpu_amdkfd_gpuvm_release_process_vm(struct kgd_dev *kgd, void *drm_priv); | 266 | void amdgpu_amdkfd_gpuvm_release_process_vm(struct amdgpu_device *adev, |
| 267 | void *drm_priv); | ||
| 266 | uint64_t amdgpu_amdkfd_gpuvm_get_process_page_dir(void *drm_priv); | 268 | uint64_t amdgpu_amdkfd_gpuvm_get_process_page_dir(void *drm_priv); |
| 267 | int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( | 269 | int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( |
| 268 | struct kgd_dev *kgd, uint64_t va, uint64_t size, | 270 | struct amdgpu_device *adev, uint64_t va, uint64_t size, |
| 269 | void *drm_priv, struct kgd_mem **mem, | 271 | void *drm_priv, struct kgd_mem **mem, |
| 270 | uint64_t *offset, uint32_t flags); | 272 | uint64_t *offset, uint32_t flags); |
| 271 | int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( | 273 | int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( |
| 272 | struct kgd_dev *kgd, struct kgd_mem *mem, void *drm_priv, | 274 | struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv, |
| 273 | uint64_t *size); | 275 | uint64_t *size); |
| 274 | int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( | 276 | int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( |
| 275 | struct kgd_dev *kgd, struct kgd_mem *mem, void *drm_priv, bool *table_freed); | 277 | struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv, |
| 278 | bool *table_freed); | ||
| 276 | int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( | 279 | int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( |
| 277 | struct kgd_dev *kgd, struct kgd_mem *mem, void *drm_priv); | 280 | struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv); |
| 278 | int amdgpu_amdkfd_gpuvm_sync_memory( | 281 | int amdgpu_amdkfd_gpuvm_sync_memory( |
| 279 | struct kgd_dev *kgd, struct kgd_mem *mem, bool intr); | 282 | struct amdgpu_device *adev, struct kgd_mem *mem, bool intr); |
| 280 | int amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(struct kgd_dev *kgd, | 283 | int amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(struct amdgpu_device *adev, |
| 281 | struct kgd_mem *mem, void **kptr, uint64_t *size); | 284 | struct kgd_mem *mem, void **kptr, uint64_t *size); |
| 282 | void amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(struct kgd_dev *kgd, struct kgd_mem *mem); | 285 | void amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(struct amdgpu_device *adev, |
| 286 | struct kgd_mem *mem); | ||
| 283 | 287 | ||
| 284 | int amdgpu_amdkfd_gpuvm_restore_process_bos(void *process_info, | 288 | int amdgpu_amdkfd_gpuvm_restore_process_bos(void *process_info, |
| 285 | struct dma_fence **ef); | 289 | struct dma_fence **ef); |
| 286 | int amdgpu_amdkfd_gpuvm_get_vm_fault_info(struct kgd_dev *kgd, | 290 | int amdgpu_amdkfd_gpuvm_get_vm_fault_info(struct amdgpu_device *adev, |
| 287 | struct kfd_vm_fault_info *info); | 291 | struct kfd_vm_fault_info *info); |
| 288 | int amdgpu_amdkfd_gpuvm_import_dmabuf(struct kgd_dev *kgd, | 292 | int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev, |
| 289 | struct dma_buf *dmabuf, | 293 | struct dma_buf *dmabuf, |
| 290 | uint64_t va, void *drm_priv, | 294 | uint64_t va, void *drm_priv, |
| 291 | struct kgd_mem **mem, uint64_t *size, | 295 | struct kgd_mem **mem, uint64_t *size, |
| 292 | uint64_t *mmap_offset); | 296 | uint64_t *mmap_offset); |
| 293 | int amdgpu_amdkfd_get_tile_config(struct kgd_dev *kgd, | 297 | int amdgpu_amdkfd_get_tile_config(struct amdgpu_device *adev, |
| 294 | struct tile_config *config); | 298 | struct tile_config *config); |
| 295 | void amdgpu_amdkfd_ras_poison_consumption_handler(struct kgd_dev *kgd); | 299 | void amdgpu_amdkfd_ras_poison_consumption_handler(struct amdgpu_device *adev, |
| 300 | bool reset); | ||
| 296 | #if IS_ENABLED(CONFIG_HSA_AMD) | 301 | #if IS_ENABLED(CONFIG_HSA_AMD) |
| 297 | void amdgpu_amdkfd_gpuvm_init_mem_limits(void); | 302 | void amdgpu_amdkfd_gpuvm_init_mem_limits(void); |
| 298 | void amdgpu_amdkfd_gpuvm_destroy_cb(struct amdgpu_device *adev, | 303 | void amdgpu_amdkfd_gpuvm_destroy_cb(struct amdgpu_device *adev, |
| 299 | struct amdgpu_vm *vm); | 304 | struct amdgpu_vm *vm); |
| 305 | |||
| 306 | /** | ||
| 307 | * @amdgpu_amdkfd_release_notify() - Notify KFD when GEM object is released | ||
| 308 | * | ||
| 309 | * Allows KFD to release its resources associated with the GEM object. | ||
| 310 | */ | ||
| 300 | void amdgpu_amdkfd_release_notify(struct amdgpu_bo *bo); | 311 | void amdgpu_amdkfd_release_notify(struct amdgpu_bo *bo); |
| 301 | void amdgpu_amdkfd_reserve_system_mem(uint64_t size); | 312 | void amdgpu_amdkfd_reserve_system_mem(uint64_t size); |
| 302 | #else | 313 | #else |
| @@ -324,7 +335,7 @@ int kgd2kfd_schedule_evict_and_restore_process(struct mm_struct *mm, · 21 unmodified lines | |||
| 324 | #if IS_ENABLED(CONFIG_HSA_AMD) | 335 | #if IS_ENABLED(CONFIG_HSA_AMD) |
| 325 | int kgd2kfd_init(void); | 336 | int kgd2kfd_init(void); |
| 326 | void kgd2kfd_exit(void); | 337 | void kgd2kfd_exit(void); |
| 327 | struct kfd_dev *kgd2kfd_probe(struct kgd_dev *kgd, bool vf); | 338 | struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf); |
| 328 | bool kgd2kfd_device_init(struct kfd_dev *kfd, | 339 | bool kgd2kfd_device_init(struct kfd_dev *kfd, |
| 329 | struct drm_device *ddev, | 340 | struct drm_device *ddev, |
| 330 | const struct kgd2kfd_shared_resources *gpu_resources); | 341 | const struct kgd2kfd_shared_resources *gpu_resources); |
| @@ -348,7 +359,7 @@ static inline void kgd2kfd_exit(void) · 17 unmodified lines | |||
| 348 | } | 359 | } |
| 349 | 360 | ||
| 350 | static inline | 361 | static inline |
| 351 | struct kfd_dev *kgd2kfd_probe(struct kgd_dev *kgd, bool vf) | 362 | struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf) |
| 352 | { | 363 | { |
| 353 | return NULL; | 364 | return NULL; |
| 354 | } | 365 | } |
| @@ -57,11 +57,6 @@ · 56 unmodified lines | |||
| 57 | (*dump)[i++][1] = RREG32(addr); \ | 57 | (*dump)[i++][1] = RREG32(addr); \ |
| 58 | } while (0) | 58 | } while (0) |
| 59 | 59 | ||
| 60 | static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd) | ||
| 61 | { | ||
| 62 | return (struct amdgpu_device *)kgd; | ||
| 63 | } | ||
| 64 | |||
| 65 | static inline struct v9_sdma_mqd *get_sdma_mqd(void *mqd) | 60 | static inline struct v9_sdma_mqd *get_sdma_mqd(void *mqd) |
| 66 | { | 61 | { |
| 67 | return (struct v9_sdma_mqd *)mqd; | 62 | return (struct v9_sdma_mqd *)mqd; |
| @@ -123,10 +118,9 @@ static uint32_t get_sdma_rlc_reg_offset(struct amdgpu_device *adev, · 55 unmodified lines | |||
| 123 | return sdma_rlc_reg_offset; | 118 | return sdma_rlc_reg_offset; |
| 124 | } | 119 | } |
| 125 | 120 | ||
| 126 | int kgd_arcturus_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, | 121 | int kgd_arcturus_hqd_sdma_load(struct amdgpu_device *adev, void *mqd, |
| 127 | uint32_t __user *wptr, struct mm_struct *mm) | 122 | uint32_t __user *wptr, struct mm_struct *mm) |
| 128 | { | 123 | { |
| 129 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 130 | struct v9_sdma_mqd *m; | 124 | struct v9_sdma_mqd *m; |
| 131 | uint32_t sdma_rlc_reg_offset; | 125 | uint32_t sdma_rlc_reg_offset; |
| 132 | unsigned long end_jiffies; | 126 | unsigned long end_jiffies; |
| @@ -193,11 +187,10 @@ int kgd_arcturus_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, · 60 unmodified lines | |||
| 193 | return 0; | 187 | return 0; |
| 194 | } | 188 | } |
| 195 | 189 | ||
| 196 | int kgd_arcturus_hqd_sdma_dump(struct kgd_dev *kgd, | 190 | int kgd_arcturus_hqd_sdma_dump(struct amdgpu_device *adev, |
| 197 | uint32_t engine_id, uint32_t queue_id, | 191 | uint32_t engine_id, uint32_t queue_id, |
| 198 | uint32_t (**dump)[2], uint32_t *n_regs) | 192 | uint32_t (**dump)[2], uint32_t *n_regs) |
| 199 | { | 193 | { |
| 200 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 201 | uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, | 194 | uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, |
| 202 | engine_id, queue_id); | 195 | engine_id, queue_id); |
| 203 | uint32_t i = 0, reg; | 196 | uint32_t i = 0, reg; |
| @@ -225,9 +218,9 @@ int kgd_arcturus_hqd_sdma_dump(struct kgd_dev *kgd, · 21 unmodified lines | |||
| 225 | return 0; | 218 | return 0; |
| 226 | } | 219 | } |
| 227 | 220 | ||
| 228 | bool kgd_arcturus_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd) | 221 | bool kgd_arcturus_hqd_sdma_is_occupied(struct amdgpu_device *adev, |
| 222 | void *mqd) | ||
| 229 | { | 223 | { |
| 230 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 231 | struct v9_sdma_mqd *m; | 224 | struct v9_sdma_mqd *m; |
| 232 | uint32_t sdma_rlc_reg_offset; | 225 | uint32_t sdma_rlc_reg_offset; |
| 233 | uint32_t sdma_rlc_rb_cntl; | 226 | uint32_t sdma_rlc_rb_cntl; |
| @@ -244,10 +237,9 @@ bool kgd_arcturus_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd) · 10 unmodified lines | |||
| 244 | return false; | 237 | return false; |
| 245 | } | 238 | } |
| 246 | 239 | ||
| 247 | int kgd_arcturus_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, | 240 | int kgd_arcturus_hqd_sdma_destroy(struct amdgpu_device *adev, void *mqd, |
| 248 | unsigned int utimeout) | 241 | unsigned int utimeout) |
| 249 | { | 242 | { |
| 250 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 251 | struct v9_sdma_mqd *m; | 243 | struct v9_sdma_mqd *m; |
| 252 | uint32_t sdma_rlc_reg_offset; | 244 | uint32_t sdma_rlc_reg_offset; |
| 253 | uint32_t temp; | 245 | uint32_t temp; |
| @@ -20,11 +20,12 @@ · 19 unmodified lines | |||
| 20 | * OTHER DEALINGS IN THE SOFTWARE. | 20 | * OTHER DEALINGS IN THE SOFTWARE. |
| 21 | */ | 21 | */ |
| 22 | 22 | ||
| 23 | int kgd_arcturus_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, | 23 | int kgd_arcturus_hqd_sdma_load(struct amdgpu_device *adev, void *mqd, |
| 24 | uint32_t __user *wptr, struct mm_struct *mm); | 24 | uint32_t __user *wptr, struct mm_struct *mm); |
| 25 | int kgd_arcturus_hqd_sdma_dump(struct kgd_dev *kgd, | 25 | int kgd_arcturus_hqd_sdma_dump(struct amdgpu_device *adev, |
| 26 | uint32_t engine_id, uint32_t queue_id, | 26 | uint32_t engine_id, uint32_t queue_id, |
| 27 | uint32_t (**dump)[2], uint32_t *n_regs); | 27 | uint32_t (**dump)[2], uint32_t *n_regs); |
| 28 | bool kgd_arcturus_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd); | 28 | bool kgd_arcturus_hqd_sdma_is_occupied(struct amdgpu_device *adev, |
| 29 | int kgd_arcturus_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, | 29 | void *mqd); |
| 30 | int kgd_arcturus_hqd_sdma_destroy(struct amdgpu_device *adev, void *mqd, | ||
| 30 | unsigned int utimeout); | 31 | unsigned int utimeout); |
| @@ -39,37 +39,26 @@ enum hqd_dequeue_request_type { · 38 unmodified lines | |||
| 39 | SAVE_WAVES | 39 | SAVE_WAVES |
| 40 | }; | 40 | }; |
| 41 | 41 | ||
| 42 | static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd) | 42 | static void lock_srbm(struct amdgpu_device *adev, uint32_t mec, uint32_t pipe, |
| 43 | { | ||
| 44 | return (struct amdgpu_device *)kgd; | ||
| 45 | } | ||
| 46 | |||
| 47 | static void lock_srbm(struct kgd_dev *kgd, uint32_t mec, uint32_t pipe, | ||
| 48 | uint32_t queue, uint32_t vmid) | 43 | uint32_t queue, uint32_t vmid) |
| 49 | { | 44 | { |
| 50 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 51 | |||
| 52 | mutex_lock(&adev->srbm_mutex); | 45 | mutex_lock(&adev->srbm_mutex); |
| 53 | nv_grbm_select(adev, mec, pipe, queue, vmid); | 46 | nv_grbm_select(adev, mec, pipe, queue, vmid); |
| 54 | } | 47 | } |
| 55 | 48 | ||
| 56 | static void unlock_srbm(struct kgd_dev *kgd) | 49 | static void unlock_srbm(struct amdgpu_device *adev) |
| 57 | { | 50 | { |
| 58 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 59 | |||
| 60 | nv_grbm_select(adev, 0, 0, 0, 0); | 51 | nv_grbm_select(adev, 0, 0, 0, 0); |
| 61 | mutex_unlock(&adev->srbm_mutex); | 52 | mutex_unlock(&adev->srbm_mutex); |
| 62 | } | 53 | } |
| 63 | 54 | ||
| 64 | static void acquire_queue(struct kgd_dev *kgd, uint32_t pipe_id, | 55 | static void acquire_queue(struct amdgpu_device *adev, uint32_t pipe_id, |
| 65 | uint32_t queue_id) | 56 | uint32_t queue_id) |
| 66 | { | 57 | { |
| 67 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 68 | |||
| 69 | uint32_t mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; | 58 | uint32_t mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; |
| 70 | uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); | 59 | uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); |
| 71 | 60 | ||
| 72 | lock_srbm(kgd, mec, pipe, queue_id, 0); | 61 | lock_srbm(adev, mec, pipe, queue_id, 0); |
| 73 | } | 62 | } |
| 74 | 63 | ||
| 75 | static uint64_t get_queue_mask(struct amdgpu_device *adev, | 64 | static uint64_t get_queue_mask(struct amdgpu_device *adev, |
| @@ -81,33 +70,29 @@ static uint64_t get_queue_mask(struct amdgpu_device *adev, · 5 unmodified lines | |||
| 81 | return 1ull << bit; | 70 | return 1ull << bit; |
| 82 | } | 71 | } |
| 83 | 72 | ||
| 84 | static void release_queue(struct kgd_dev *kgd) | 73 | static void release_queue(struct amdgpu_device *adev) |
| 85 | { | 74 | { |
| 86 | unlock_srbm(kgd); | 75 | unlock_srbm(adev); |
| 87 | } | 76 | } |
| 88 | 77 | ||
| 89 | static void kgd_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid, | 78 | static void kgd_program_sh_mem_settings(struct amdgpu_device *adev, uint32_t vmid, |
| 90 | uint32_t sh_mem_config, | 79 | uint32_t sh_mem_config, |
| 91 | uint32_t sh_mem_ape1_base, | 80 | uint32_t sh_mem_ape1_base, |
| 92 | uint32_t sh_mem_ape1_limit, | 81 | uint32_t sh_mem_ape1_limit, |
| 93 | uint32_t sh_mem_bases) | 82 | uint32_t sh_mem_bases) |
| 94 | { | 83 | { |
| 95 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | 84 | lock_srbm(adev, 0, 0, 0, vmid); |
| 96 | |||
| 97 | lock_srbm(kgd, 0, 0, 0, vmid); | ||
| 98 | 85 | ||
| 99 | WREG32_SOC15(GC, 0, mmSH_MEM_CONFIG, sh_mem_config); | 86 | WREG32_SOC15(GC, 0, mmSH_MEM_CONFIG, sh_mem_config); |
| 100 | WREG32_SOC15(GC, 0, mmSH_MEM_BASES, sh_mem_bases); | 87 | WREG32_SOC15(GC, 0, mmSH_MEM_BASES, sh_mem_bases); |
| 101 | /* APE1 no longer exists on GFX9 */ | 88 | /* APE1 no longer exists on GFX9 */ |
| 102 | 89 | ||
| 103 | unlock_srbm(kgd); | 90 | unlock_srbm(adev); |
| 104 | } | 91 | } |
| 105 | 92 | ||
| 106 | static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid, | 93 | static int kgd_set_pasid_vmid_mapping(struct amdgpu_device *adev, u32 pasid, |
| 107 | unsigned int vmid) | 94 | unsigned int vmid) |
| 108 | { | 95 | { |
| 109 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 110 | |||
| 111 | /* | 96 | /* |
| 112 | * We have to assume that there is no outstanding mapping. | 97 | * We have to assume that there is no outstanding mapping. |
| 113 | * The ATC_VMID_PASID_MAPPING_UPDATE_STATUS bit could be 0 because | 98 | * The ATC_VMID_PASID_MAPPING_UPDATE_STATUS bit could be 0 because |
| @@ -150,22 +135,21 @@ static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid, · 36 unmodified lines | |||
| 150 | * but still works | 135 | * but still works |
| 151 | */ | 136 | */ |
| 152 | 137 | ||
| 153 | static int kgd_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id) | 138 | static int kgd_init_interrupts(struct amdgpu_device *adev, uint32_t pipe_id) |
| 154 | { | 139 | { |
| 155 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 156 | uint32_t mec; | 140 | uint32_t mec; |
| 157 | uint32_t pipe; | 141 | uint32_t pipe; |
| 158 | 142 | ||
| 159 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; | 143 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; |
| 160 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); | 144 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); |
| 161 | 145 | ||
| 162 | lock_srbm(kgd, mec, pipe, 0, 0); | 146 | lock_srbm(adev, mec, pipe, 0, 0); |
| 163 | 147 | ||
| 164 | WREG32_SOC15(GC, 0, mmCPC_INT_CNTL, | 148 | WREG32_SOC15(GC, 0, mmCPC_INT_CNTL, |
| 165 | CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK | | 149 | CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK | |
| 166 | CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK); | 150 | CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK); |
| 167 | 151 | ||
| 168 | unlock_srbm(kgd); | 152 | unlock_srbm(adev); |
| 169 | 153 | ||
| 170 | return 0; | 154 | return 0; |
| 171 | } | 155 | } |
| @@ -218,12 +202,11 @@ static inline struct v10_sdma_mqd *get_sdma_mqd(void *mqd) · 46 unmodified lines | |||
| 218 | return (struct v10_sdma_mqd *)mqd; | 202 | return (struct v10_sdma_mqd *)mqd; |
| 219 | } | 203 | } |
| 220 | 204 | ||
| 221 | static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, | 205 | static int kgd_hqd_load(struct amdgpu_device *adev, void *mqd, |
| 222 | uint32_t queue_id, uint32_t __user *wptr, | 206 | uint32_t pipe_id, uint32_t queue_id, |
| 223 | uint32_t wptr_shift, uint32_t wptr_mask, | 207 | uint32_t __user *wptr, uint32_t wptr_shift, |
| 224 | struct mm_struct *mm) | 208 | uint32_t wptr_mask, struct mm_struct *mm) |
| 225 | { | 209 | { |
| 226 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 227 | struct v10_compute_mqd *m; | 210 | struct v10_compute_mqd *m; |
| 228 | uint32_t *mqd_hqd; | 211 | uint32_t *mqd_hqd; |
| 229 | uint32_t reg, hqd_base, data; | 212 | uint32_t reg, hqd_base, data; |
| @@ -231,7 +214,7 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, · 1 unmodified line | |||
| 231 | m = get_mqd(mqd); | 214 | m = get_mqd(mqd); |
| 232 | 215 | ||
| 233 | pr_debug("Load hqd of pipe %d queue %d\n", pipe_id, queue_id); | 216 | pr_debug("Load hqd of pipe %d queue %d\n", pipe_id, queue_id); |
| 234 | acquire_queue(kgd, pipe_id, queue_id); | 217 | acquire_queue(adev, pipe_id, queue_id); |
| 235 | 218 | ||
| 236 | /* HQD registers extend from CP_MQD_BASE_ADDR to CP_HQD_EOP_WPTR_MEM. */ | 219 | /* HQD registers extend from CP_MQD_BASE_ADDR to CP_HQD_EOP_WPTR_MEM. */ |
| 237 | mqd_hqd = &m->cp_mqd_base_addr_lo; | 220 | mqd_hqd = &m->cp_mqd_base_addr_lo; |
| @@ -296,16 +279,15 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, · 58 unmodified lines | |||
| 296 | data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1); | 279 | data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1); |
| 297 | WREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE, data); | 280 | WREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE, data); |
| 298 | 281 | ||
| 299 | release_queue(kgd); | 282 | release_queue(adev); |
| 300 | 283 | ||
| 301 | return 0; | 284 | return 0; |
| 302 | } | 285 | } |
| 303 | 286 | ||
| 304 | static int kgd_hiq_mqd_load(struct kgd_dev *kgd, void *mqd, | 287 | static int kgd_hiq_mqd_load(struct amdgpu_device *adev, void *mqd, |
| 305 | uint32_t pipe_id, uint32_t queue_id, | 288 | uint32_t pipe_id, uint32_t queue_id, |
| 306 | uint32_t doorbell_off) | 289 | uint32_t doorbell_off) |
| 307 | { | 290 | { |
| 308 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 309 | struct amdgpu_ring *kiq_ring = &adev->gfx.kiq.ring; | 291 | struct amdgpu_ring *kiq_ring = &adev->gfx.kiq.ring; |
| 310 | struct v10_compute_mqd *m; | 292 | struct v10_compute_mqd *m; |
| 311 | uint32_t mec, pipe; | 293 | uint32_t mec, pipe; |
| @@ -313,7 +295,7 @@ static int kgd_hiq_mqd_load(struct kgd_dev *kgd, void *mqd, · 1 unmodified line | |||
| 313 | 295 | ||
| 314 | m = get_mqd(mqd); | 296 | m = get_mqd(mqd); |
| 315 | 297 | ||
| 316 | acquire_queue(kgd, pipe_id, queue_id); | 298 | acquire_queue(adev, pipe_id, queue_id); |
| 317 | 299 | ||
| 318 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; | 300 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; |
| 319 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); | 301 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); |
| @@ -349,16 +331,15 @@ static int kgd_hiq_mqd_load(struct kgd_dev *kgd, void *mqd, · 29 unmodified lines | |||
| 349 | 331 | ||
| 350 | out_unlock: | 332 | out_unlock: |
| 351 | spin_unlock(&adev->gfx.kiq.ring_lock); | 333 | spin_unlock(&adev->gfx.kiq.ring_lock); |
| 352 | release_queue(kgd); | 334 | release_queue(adev); |
| 353 | 335 | ||
| 354 | return r; | 336 | return r; |
| 355 | } | 337 | } |
| 356 | 338 | ||
| 357 | static int kgd_hqd_dump(struct kgd_dev *kgd, | 339 | static int kgd_hqd_dump(struct amdgpu_device *adev, |
| 358 | uint32_t pipe_id, uint32_t queue_id, | 340 | uint32_t pipe_id, uint32_t queue_id, |
| 359 | uint32_t (**dump)[2], uint32_t *n_regs) | 341 | uint32_t (**dump)[2], uint32_t *n_regs) |
| 360 | { | 342 | { |
| 361 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 362 | uint32_t i = 0, reg; | 343 | uint32_t i = 0, reg; |
| 363 | #define HQD_N_REGS 56 | 344 | #define HQD_N_REGS 56 |
| 364 | #define DUMP_REG(addr) do { \ | 345 | #define DUMP_REG(addr) do { \ |
| @@ -372,13 +353,13 @@ static int kgd_hqd_dump(struct kgd_dev *kgd, · 7 unmodified lines | |||
| 372 | if (*dump == NULL) | 353 | if (*dump == NULL) |
| 373 | return -ENOMEM; | 354 | return -ENOMEM; |
| 374 | 355 | ||
| 375 | acquire_queue(kgd, pipe_id, queue_id); | 356 | acquire_queue(adev, pipe_id, queue_id); |
| 376 | 357 | ||
| 377 | for (reg = SOC15_REG_OFFSET(GC, 0, mmCP_MQD_BASE_ADDR); | 358 | for (reg = SOC15_REG_OFFSET(GC, 0, mmCP_MQD_BASE_ADDR); |
| 378 | reg <= SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_HI); reg++) | 359 | reg <= SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_HI); reg++) |
| 379 | DUMP_REG(reg); | 360 | DUMP_REG(reg); |
| 380 | 361 | ||
| 381 | release_queue(kgd); | 362 | release_queue(adev); |
| 382 | 363 | ||
| 383 | WARN_ON_ONCE(i != HQD_N_REGS); | 364 | WARN_ON_ONCE(i != HQD_N_REGS); |
| 384 | *n_regs = i; | 365 | *n_regs = i; |
| @@ -386,10 +367,9 @@ static int kgd_hqd_dump(struct kgd_dev *kgd, · 1 unmodified line | |||
| 386 | return 0; | 367 | return 0; |
| 387 | } | 368 | } |
| 388 | 369 | ||
| 389 | static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, | 370 | static int kgd_hqd_sdma_load(struct amdgpu_device *adev, void *mqd, |
| 390 | uint32_t __user *wptr, struct mm_struct *mm) | 371 | uint32_t __user *wptr, struct mm_struct *mm) |
| 391 | { | 372 | { |
| 392 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 393 | struct v10_sdma_mqd *m; | 373 | struct v10_sdma_mqd *m; |
| 394 | uint32_t sdma_rlc_reg_offset; | 374 | uint32_t sdma_rlc_reg_offset; |
| 395 | unsigned long end_jiffies; | 375 | unsigned long end_jiffies; |
| @@ -456,11 +436,10 @@ static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, · 60 unmodified lines | |||
| 456 | return 0; | 436 | return 0; |
| 457 | } | 437 | } |
| 458 | 438 | ||
| 459 | static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, | 439 | static int kgd_hqd_sdma_dump(struct amdgpu_device *adev, |
| 460 | uint32_t engine_id, uint32_t queue_id, | 440 | uint32_t engine_id, uint32_t queue_id, |
| 461 | uint32_t (**dump)[2], uint32_t *n_regs) | 441 | uint32_t (**dump)[2], uint32_t *n_regs) |
| 462 | { | 442 | { |
| 463 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 464 | uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, | 443 | uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, |
| 465 | engine_id, queue_id); | 444 | engine_id, queue_id); |
| 466 | uint32_t i = 0, reg; | 445 | uint32_t i = 0, reg; |
| @@ -488,15 +467,15 @@ static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, · 21 unmodified lines | |||
| 488 | return 0; | 467 | return 0; |
| 489 | } | 468 | } |
| 490 | 469 | ||
| 491 | static bool kgd_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address, | 470 | static bool kgd_hqd_is_occupied(struct amdgpu_device *adev, |
| 492 | uint32_t pipe_id, uint32_t queue_id) | 471 | uint64_t queue_address, uint32_t pipe_id, |
| 472 | uint32_t queue_id) | ||
| 493 | { | 473 | { |
| 494 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 495 | uint32_t act; | 474 | uint32_t act; |
| 496 | bool retval = false; | 475 | bool retval = false; |
| 497 | uint32_t low, high; | 476 | uint32_t low, high; |
| 498 | 477 | ||
| 499 | acquire_queue(kgd, pipe_id, queue_id); | 478 | acquire_queue(adev, pipe_id, queue_id); |
| 500 | act = RREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE); | 479 | act = RREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE); |
| 501 | if (act) { | 480 | if (act) { |
| 502 | low = lower_32_bits(queue_address >> 8); | 481 | low = lower_32_bits(queue_address >> 8); |
| @@ -506,13 +485,12 @@ static bool kgd_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address, · 3 unmodified lines | |||
| 506 | high == RREG32_SOC15(GC, 0, mmCP_HQD_PQ_BASE_HI)) | 485 | high == RREG32_SOC15(GC, 0, mmCP_HQD_PQ_BASE_HI)) |
| 507 | retval = true; | 486 | retval = true; |
| 508 | } | 487 | } |
| 509 | release_queue(kgd); | 488 | release_queue(adev); |
| 510 | return retval; | 489 | return retval; |
| 511 | } | 490 | } |
| 512 | 491 | ||
| 513 | static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd) | 492 | static bool kgd_hqd_sdma_is_occupied(struct amdgpu_device *adev, void *mqd) |
| 514 | { | 493 | { |
| 515 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 516 | struct v10_sdma_mqd *m; | 494 | struct v10_sdma_mqd *m; |
| 517 | uint32_t sdma_rlc_reg_offset; | 495 | uint32_t sdma_rlc_reg_offset; |
| 518 | uint32_t sdma_rlc_rb_cntl; | 496 | uint32_t sdma_rlc_rb_cntl; |
| @@ -529,12 +507,11 @@ static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd) · 10 unmodified lines | |||
| 529 | return false; | 507 | return false; |
| 530 | } | 508 | } |
| 531 | 509 | ||
| 532 | static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd, | 510 | static int kgd_hqd_destroy(struct amdgpu_device *adev, void *mqd, |
| 533 | enum kfd_preempt_type reset_type, | 511 | enum kfd_preempt_type reset_type, |
| 534 | unsigned int utimeout, uint32_t pipe_id, | 512 | unsigned int utimeout, uint32_t pipe_id, |
| 535 | uint32_t queue_id) | 513 | uint32_t queue_id) |
| 536 | { | 514 | { |
| 537 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 538 | enum hqd_dequeue_request_type type; | 515 | enum hqd_dequeue_request_type type; |
| 539 | unsigned long end_jiffies; | 516 | unsigned long end_jiffies; |
| 540 | uint32_t temp; | 517 | uint32_t temp; |
| @@ -548,7 +525,7 @@ static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd, · 7 unmodified lines | |||
| 548 | int retry; | 525 | int retry; |
| 549 | #endif | 526 | #endif |
| 550 | 527 | ||
| 551 | acquire_queue(kgd, pipe_id, queue_id); | 528 | acquire_queue(adev, pipe_id, queue_id); |
| 552 | 529 | ||
| 553 | if (m->cp_hqd_vmid == 0) | 530 | if (m->cp_hqd_vmid == 0) |
| 554 | WREG32_FIELD15(GC, 0, RLC_CP_SCHEDULERS, scheduler1, 0); | 531 | WREG32_FIELD15(GC, 0, RLC_CP_SCHEDULERS, scheduler1, 0); |
| @@ -633,20 +610,19 @@ loop: · 78 unmodified lines | |||
| 633 | break; | 610 | break; |
| 634 | if (time_after(jiffies, end_jiffies)) { | 611 | if (time_after(jiffies, end_jiffies)) { |
| 635 | pr_err("cp queue preemption time out.\n"); | 612 | pr_err("cp queue preemption time out.\n"); |
| 636 | release_queue(kgd); | 613 | release_queue(adev); |
| 637 | return -ETIME; | 614 | return -ETIME; |
| 638 | } | 615 | } |
| 639 | usleep_range(500, 1000); | 616 | usleep_range(500, 1000); |
| 640 | } | 617 | } |
| 641 | 618 | ||
| 642 | release_queue(kgd); | 619 | release_queue(adev); |
| 643 | return 0; | 620 | return 0; |
| 644 | } | 621 | } |
| 645 | 622 | ||
| 646 | static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, | 623 | static int kgd_hqd_sdma_destroy(struct amdgpu_device *adev, void *mqd, |
| 647 | unsigned int utimeout) | 624 | unsigned int utimeout) |
| 648 | { | 625 | { |
| 649 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 650 | struct v10_sdma_mqd *m; | 626 | struct v10_sdma_mqd *m; |
| 651 | uint32_t sdma_rlc_reg_offset; | 627 | uint32_t sdma_rlc_reg_offset; |
| 652 | uint32_t temp; | 628 | uint32_t temp; |
| @@ -683,11 +659,10 @@ static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, · 30 unmodified lines | |||
| 683 | return 0; | 659 | return 0; |
| 684 | } | 660 | } |
| 685 | 661 | ||
| 686 | static bool get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd, | 662 | static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, |
| 687 | uint8_t vmid, uint16_t *p_pasid) | 663 | uint8_t vmid, uint16_t *p_pasid) |
| 688 | { | 664 | { |
| 689 | uint32_t value; | 665 | uint32_t value; |
| 690 | struct amdgpu_device *adev = (struct amdgpu_device *) kgd; | ||
| 691 | 666 | ||
| 692 | value = RREG32(SOC15_REG_OFFSET(ATHUB, 0, mmATC_VMID0_PASID_MAPPING) | 667 | value = RREG32(SOC15_REG_OFFSET(ATHUB, 0, mmATC_VMID0_PASID_MAPPING) |
| 693 | + vmid); | 668 | + vmid); |
| @@ -696,12 +671,12 @@ static bool get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd, · 2 unmodified lines | |||
| 696 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); | 671 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); |
| 697 | } | 672 | } |
| 698 | 673 | ||
| 699 | static int kgd_address_watch_disable(struct kgd_dev *kgd) | 674 | static int kgd_address_watch_disable(struct amdgpu_device *adev) |
| 700 | { | 675 | { |
| 701 | return 0; | 676 | return 0; |
| 702 | } | 677 | } |
| 703 | 678 | ||
| 704 | static int kgd_address_watch_execute(struct kgd_dev *kgd, | 679 | static int kgd_address_watch_execute(struct amdgpu_device *adev, |
| 705 | unsigned int watch_point_id, | 680 | unsigned int watch_point_id, |
| 706 | uint32_t cntl_val, | 681 | uint32_t cntl_val, |
| 707 | uint32_t addr_hi, | 682 | uint32_t addr_hi, |
| @@ -710,11 +685,10 @@ static int kgd_address_watch_execute(struct kgd_dev *kgd, · 2 unmodified lines | |||
| 710 | return 0; | 685 | return 0; |
| 711 | } | 686 | } |
| 712 | 687 | ||
| 713 | static int kgd_wave_control_execute(struct kgd_dev *kgd, | 688 | static int kgd_wave_control_execute(struct amdgpu_device *adev, |
| 714 | uint32_t gfx_index_val, | 689 | uint32_t gfx_index_val, |
| 715 | uint32_t sq_cmd) | 690 | uint32_t sq_cmd) |
| 716 | { | 691 | { |
| 717 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 718 | uint32_t data = 0; | 692 | uint32_t data = 0; |
| 719 | 693 | ||
| 720 | mutex_lock(&adev->grbm_idx_mutex); | 694 | mutex_lock(&adev->grbm_idx_mutex); |
| @@ -735,18 +709,16 @@ static int kgd_wave_control_execute(struct kgd_dev *kgd, · 14 unmodified lines | |||
| 735 | return 0; | 709 | return 0; |
| 736 | } | 710 | } |
| 737 | 711 | ||
| 738 | static uint32_t kgd_address_watch_get_offset(struct kgd_dev *kgd, | 712 | static uint32_t kgd_address_watch_get_offset(struct amdgpu_device *adev, |
| 739 | unsigned int watch_point_id, | 713 | unsigned int watch_point_id, |
| 740 | unsigned int reg_offset) | 714 | unsigned int reg_offset) |
| 741 | { | 715 | { |
| 742 | return 0; | 716 | return 0; |
| 743 | } | 717 | } |
| 744 | 718 | ||
| 745 | static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, | 719 | static void set_vm_context_page_table_base(struct amdgpu_device *adev, |
| 746 | uint64_t page_table_base) | 720 | uint32_t vmid, uint64_t page_table_base) |
| 747 | { | 721 | { |
| 748 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 749 | |||
| 750 | if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) { | 722 | if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) { |
| 751 | pr_err("trying to set page table base for wrong VMID %u\n", | 723 | pr_err("trying to set page table base for wrong VMID %u\n", |
| 752 | vmid); | 724 | vmid); |
| @@ -757,12 +729,10 @@ static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, · 4 unmodified lines | |||
| 757 | adev->gfxhub.funcs->setup_vm_pt_regs(adev, vmid, page_table_base); | 729 | adev->gfxhub.funcs->setup_vm_pt_regs(adev, vmid, page_table_base); |
| 758 | } | 730 | } |
| 759 | 731 | ||
| 760 | static void program_trap_handler_settings(struct kgd_dev *kgd, | 732 | static void program_trap_handler_settings(struct amdgpu_device *adev, |
| 761 | uint32_t vmid, uint64_t tba_addr, uint64_t tma_addr) | 733 | uint32_t vmid, uint64_t tba_addr, uint64_t tma_addr) |
| 762 | { | 734 | { |
| 763 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | 735 | lock_srbm(adev, 0, 0, 0, vmid); |
| 764 | |||
| 765 | lock_srbm(kgd, 0, 0, 0, vmid); | ||
| 766 | 736 | ||
| 767 | /* | 737 | /* |
| 768 | * Program TBA registers | 738 | * Program TBA registers |
| @@ -781,7 +751,7 @@ static void program_trap_handler_settings(struct kgd_dev *kgd, · 12 unmodified lines | |||
| 781 | WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_SHADER_TMA_HI), | 751 | WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_SHADER_TMA_HI), |
| 782 | upper_32_bits(tma_addr >> 8)); | 752 | upper_32_bits(tma_addr >> 8)); |
| 783 | 753 | ||
| 784 | unlock_srbm(kgd); | 754 | unlock_srbm(adev); |
| 785 | } | 755 | } |
| 786 | 756 | ||
| 787 | const struct kfd2kgd_calls gfx_v10_kfd2kgd = { | 757 | const struct kfd2kgd_calls gfx_v10_kfd2kgd = { |
| @@ -38,37 +38,26 @@ enum hqd_dequeue_request_type { · 37 unmodified lines | |||
| 38 | SAVE_WAVES | 38 | SAVE_WAVES |
| 39 | }; | 39 | }; |
| 40 | 40 | ||
| 41 | static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd) | 41 | static void lock_srbm(struct amdgpu_device *adev, uint32_t mec, uint32_t pipe, |
| 42 | { | ||
| 43 | return (struct amdgpu_device *)kgd; | ||
| 44 | } | ||
| 45 | |||
| 46 | static void lock_srbm(struct kgd_dev *kgd, uint32_t mec, uint32_t pipe, | ||
| 47 | uint32_t queue, uint32_t vmid) | 42 | uint32_t queue, uint32_t vmid) |
| 48 | { | 43 | { |
| 49 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 50 | |||
| 51 | mutex_lock(&adev->srbm_mutex); | 44 | mutex_lock(&adev->srbm_mutex); |
| 52 | nv_grbm_select(adev, mec, pipe, queue, vmid); | 45 | nv_grbm_select(adev, mec, pipe, queue, vmid); |
| 53 | } | 46 | } |
| 54 | 47 | ||
| 55 | static void unlock_srbm(struct kgd_dev *kgd) | 48 | static void unlock_srbm(struct amdgpu_device *adev) |
| 56 | { | 49 | { |
| 57 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 58 | |||
| 59 | nv_grbm_select(adev, 0, 0, 0, 0); | 50 | nv_grbm_select(adev, 0, 0, 0, 0); |
| 60 | mutex_unlock(&adev->srbm_mutex); | 51 | mutex_unlock(&adev->srbm_mutex); |
| 61 | } | 52 | } |
| 62 | 53 | ||
| 63 | static void acquire_queue(struct kgd_dev *kgd, uint32_t pipe_id, | 54 | static void acquire_queue(struct amdgpu_device *adev, uint32_t pipe_id, |
| 64 | uint32_t queue_id) | 55 | uint32_t queue_id) |
| 65 | { | 56 | { |
| 66 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 67 | |||
| 68 | uint32_t mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; | 57 | uint32_t mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; |
| 69 | uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); | 58 | uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); |
| 70 | 59 | ||
| 71 | lock_srbm(kgd, mec, pipe, queue_id, 0); | 60 | lock_srbm(adev, mec, pipe, queue_id, 0); |
| 72 | } | 61 | } |
| 73 | 62 | ||
| 74 | static uint64_t get_queue_mask(struct amdgpu_device *adev, | 63 | static uint64_t get_queue_mask(struct amdgpu_device *adev, |
| @@ -80,34 +69,30 @@ static uint64_t get_queue_mask(struct amdgpu_device *adev, · 5 unmodified lines | |||
| 80 | return 1ull << bit; | 69 | return 1ull << bit; |
| 81 | } | 70 | } |
| 82 | 71 | ||
| 83 | static void release_queue(struct kgd_dev *kgd) | 72 | static void release_queue(struct amdgpu_device *adev) |
| 84 | { | 73 | { |
| 85 | unlock_srbm(kgd); | 74 | unlock_srbm(adev); |
| 86 | } | 75 | } |
| 87 | 76 | ||
| 88 | static void program_sh_mem_settings_v10_3(struct kgd_dev *kgd, uint32_t vmid, | 77 | static void program_sh_mem_settings_v10_3(struct amdgpu_device *adev, uint32_t vmid, |
| 89 | uint32_t sh_mem_config, | 78 | uint32_t sh_mem_config, |
| 90 | uint32_t sh_mem_ape1_base, | 79 | uint32_t sh_mem_ape1_base, |
| 91 | uint32_t sh_mem_ape1_limit, | 80 | uint32_t sh_mem_ape1_limit, |
| 92 | uint32_t sh_mem_bases) | 81 | uint32_t sh_mem_bases) |
| 93 | { | 82 | { |
| 94 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | 83 | lock_srbm(adev, 0, 0, 0, vmid); |
| 95 | |||
| 96 | lock_srbm(kgd, 0, 0, 0, vmid); | ||
| 97 | 84 | ||
| 98 | WREG32_SOC15(GC, 0, mmSH_MEM_CONFIG, sh_mem_config); | 85 | WREG32_SOC15(GC, 0, mmSH_MEM_CONFIG, sh_mem_config); |
| 99 | WREG32_SOC15(GC, 0, mmSH_MEM_BASES, sh_mem_bases); | 86 | WREG32_SOC15(GC, 0, mmSH_MEM_BASES, sh_mem_bases); |
| 100 | /* APE1 no longer exists on GFX9 */ | 87 | /* APE1 no longer exists on GFX9 */ |
| 101 | 88 | ||
| 102 | unlock_srbm(kgd); | 89 | unlock_srbm(adev); |
| 103 | } | 90 | } |
| 104 | 91 | ||
| 105 | /* ATC is defeatured on Sienna_Cichlid */ | 92 | /* ATC is defeatured on Sienna_Cichlid */ |
| 106 | static int set_pasid_vmid_mapping_v10_3(struct kgd_dev *kgd, unsigned int pasid, | 93 | static int set_pasid_vmid_mapping_v10_3(struct amdgpu_device *adev, unsigned int pasid, |
| 107 | unsigned int vmid) | 94 | unsigned int vmid) |
| 108 | { | 95 | { |
| 109 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 110 | |||
| 111 | uint32_t value = pasid << IH_VMID_0_LUT__PASID__SHIFT; | 96 | uint32_t value = pasid << IH_VMID_0_LUT__PASID__SHIFT; |
| 112 | 97 | ||
| 113 | /* Mapping vmid to pasid also for IH block */ | 98 | /* Mapping vmid to pasid also for IH block */ |
| @@ -118,22 +103,21 @@ static int set_pasid_vmid_mapping_v10_3(struct kgd_dev *kgd, unsigned int pasid, · 4 unmodified lines | |||
| 118 | return 0; | 103 | return 0; |
| 119 | } | 104 | } |
| 120 | 105 | ||
| 121 | static int init_interrupts_v10_3(struct kgd_dev *kgd, uint32_t pipe_id) | 106 | static int init_interrupts_v10_3(struct amdgpu_device *adev, uint32_t pipe_id) |
| 122 | { | 107 | { |
| 123 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 124 | uint32_t mec; | 108 | uint32_t mec; |
| 125 | uint32_t pipe; | 109 | uint32_t pipe; |
| 126 | 110 | ||
| 127 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; | 111 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; |
| 128 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); | 112 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); |
| 129 | 113 | ||
| 130 | lock_srbm(kgd, mec, pipe, 0, 0); | 114 | lock_srbm(adev, mec, pipe, 0, 0); |
| 131 | 115 | ||
| 132 | WREG32_SOC15(GC, 0, mmCPC_INT_CNTL, | 116 | WREG32_SOC15(GC, 0, mmCPC_INT_CNTL, |
| 133 | CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK | | 117 | CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK | |
| 134 | CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK); | 118 | CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK); |
| 135 | 119 | ||
| 136 | unlock_srbm(kgd); | 120 | unlock_srbm(adev); |
| 137 | 121 | ||
| 138 | return 0; | 122 | return 0; |
| 139 | } | 123 | } |
| @@ -188,12 +172,11 @@ static inline struct v10_sdma_mqd *get_sdma_mqd(void *mqd) · 48 unmodified lines | |||
| 188 | return (struct v10_sdma_mqd *)mqd; | 172 | return (struct v10_sdma_mqd *)mqd; |
| 189 | } | 173 | } |
| 190 | 174 | ||
| 191 | static int hqd_load_v10_3(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, | 175 | static int hqd_load_v10_3(struct amdgpu_device *adev, void *mqd, |
| 192 | uint32_t queue_id, uint32_t __user *wptr, | 176 | uint32_t pipe_id, uint32_t queue_id, |
| 193 | uint32_t wptr_shift, uint32_t wptr_mask, | 177 | uint32_t __user *wptr, uint32_t wptr_shift, |
| 194 | struct mm_struct *mm) | 178 | uint32_t wptr_mask, struct mm_struct *mm) |
| 195 | { | 179 | { |
| 196 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 197 | struct v10_compute_mqd *m; | 180 | struct v10_compute_mqd *m; |
| 198 | uint32_t *mqd_hqd; | 181 | uint32_t *mqd_hqd; |
| 199 | uint32_t reg, hqd_base, data; | 182 | uint32_t reg, hqd_base, data; |
| @@ -201,7 +184,7 @@ static int hqd_load_v10_3(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, · 1 unmodified line | |||
| 201 | m = get_mqd(mqd); | 184 | m = get_mqd(mqd); |
| 202 | 185 | ||
| 203 | pr_debug("Load hqd of pipe %d queue %d\n", pipe_id, queue_id); | 186 | pr_debug("Load hqd of pipe %d queue %d\n", pipe_id, queue_id); |
| 204 | acquire_queue(kgd, pipe_id, queue_id); | 187 | acquire_queue(adev, pipe_id, queue_id); |
| 205 | 188 | ||
| 206 | /* HIQ is set during driver init period with vmid set to 0*/ | 189 | /* HIQ is set during driver init period with vmid set to 0*/ |
| 207 | if (m->cp_hqd_vmid == 0) { | 190 | if (m->cp_hqd_vmid == 0) { |
| @@ -281,16 +264,15 @@ static int hqd_load_v10_3(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, · 73 unmodified lines | |||
| 281 | data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1); | 264 | data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1); |
| 282 | WREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE, data); | 265 | WREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE, data); |
| 283 | 266 | ||
| 284 | release_queue(kgd); | 267 | release_queue(adev); |
| 285 | 268 | ||
| 286 | return 0; | 269 | return 0; |
| 287 | } | 270 | } |
| 288 | 271 | ||
| 289 | static int hiq_mqd_load_v10_3(struct kgd_dev *kgd, void *mqd, | 272 | static int hiq_mqd_load_v10_3(struct amdgpu_device *adev, void *mqd, |
| 290 | uint32_t pipe_id, uint32_t queue_id, | 273 | uint32_t pipe_id, uint32_t queue_id, |
| 291 | uint32_t doorbell_off) | 274 | uint32_t doorbell_off) |
| 292 | { | 275 | { |
| 293 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 294 | struct amdgpu_ring *kiq_ring = &adev->gfx.kiq.ring; | 276 | struct amdgpu_ring *kiq_ring = &adev->gfx.kiq.ring; |
| 295 | struct v10_compute_mqd *m; | 277 | struct v10_compute_mqd *m; |
| 296 | uint32_t mec, pipe; | 278 | uint32_t mec, pipe; |
| @@ -298,7 +280,7 @@ static int hiq_mqd_load_v10_3(struct kgd_dev *kgd, void *mqd, · 1 unmodified line | |||
| 298 | 280 | ||
| 299 | m = get_mqd(mqd); | 281 | m = get_mqd(mqd); |
| 300 | 282 | ||
| 301 | acquire_queue(kgd, pipe_id, queue_id); | 283 | acquire_queue(adev, pipe_id, queue_id); |
| 302 | 284 | ||
| 303 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; | 285 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; |
| 304 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); | 286 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); |
| @@ -334,16 +316,15 @@ static int hiq_mqd_load_v10_3(struct kgd_dev *kgd, void *mqd, · 29 unmodified lines | |||
| 334 | 316 | ||
| 335 | out_unlock: | 317 | out_unlock: |
| 336 | spin_unlock(&adev->gfx.kiq.ring_lock); | 318 | spin_unlock(&adev->gfx.kiq.ring_lock); |
| 337 | release_queue(kgd); | 319 | release_queue(adev); |
| 338 | 320 | ||
| 339 | return r; | 321 | return r; |
| 340 | } | 322 | } |
| 341 | 323 | ||
| 342 | static int hqd_dump_v10_3(struct kgd_dev *kgd, | 324 | static int hqd_dump_v10_3(struct amdgpu_device *adev, |
| 343 | uint32_t pipe_id, uint32_t queue_id, | 325 | uint32_t pipe_id, uint32_t queue_id, |
| 344 | uint32_t (**dump)[2], uint32_t *n_regs) | 326 | uint32_t (**dump)[2], uint32_t *n_regs) |
| 345 | { | 327 | { |
| 346 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 347 | uint32_t i = 0, reg; | 328 | uint32_t i = 0, reg; |
| 348 | #define HQD_N_REGS 56 | 329 | #define HQD_N_REGS 56 |
| 349 | #define DUMP_REG(addr) do { \ | 330 | #define DUMP_REG(addr) do { \ |
| @@ -357,13 +338,13 @@ static int hqd_dump_v10_3(struct kgd_dev *kgd, · 7 unmodified lines | |||
| 357 | if (*dump == NULL) | 338 | if (*dump == NULL) |
| 358 | return -ENOMEM; | 339 | return -ENOMEM; |
| 359 | 340 | ||
| 360 | acquire_queue(kgd, pipe_id, queue_id); | 341 | acquire_queue(adev, pipe_id, queue_id); |
| 361 | 342 | ||
| 362 | for (reg = SOC15_REG_OFFSET(GC, 0, mmCP_MQD_BASE_ADDR); | 343 | for (reg = SOC15_REG_OFFSET(GC, 0, mmCP_MQD_BASE_ADDR); |
| 363 | reg <= SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_HI); reg++) | 344 | reg <= SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_HI); reg++) |
| 364 | DUMP_REG(reg); | 345 | DUMP_REG(reg); |
| 365 | 346 | ||
| 366 | release_queue(kgd); | 347 | release_queue(adev); |
| 367 | 348 | ||
| 368 | WARN_ON_ONCE(i != HQD_N_REGS); | 349 | WARN_ON_ONCE(i != HQD_N_REGS); |
| 369 | *n_regs = i; | 350 | *n_regs = i; |
| @@ -371,10 +352,9 @@ static int hqd_dump_v10_3(struct kgd_dev *kgd, · 1 unmodified line | |||
| 371 | return 0; | 352 | return 0; |
| 372 | } | 353 | } |
| 373 | 354 | ||
| 374 | static int hqd_sdma_load_v10_3(struct kgd_dev *kgd, void *mqd, | 355 | static int hqd_sdma_load_v10_3(struct amdgpu_device *adev, void *mqd, |
| 375 | uint32_t __user *wptr, struct mm_struct *mm) | 356 | uint32_t __user *wptr, struct mm_struct *mm) |
| 376 | { | 357 | { |
| 377 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 378 | struct v10_sdma_mqd *m; | 358 | struct v10_sdma_mqd *m; |
| 379 | uint32_t sdma_rlc_reg_offset; | 359 | uint32_t sdma_rlc_reg_offset; |
| 380 | unsigned long end_jiffies; | 360 | unsigned long end_jiffies; |
| @@ -441,11 +421,10 @@ static int hqd_sdma_load_v10_3(struct kgd_dev *kgd, void *mqd, · 60 unmodified lines | |||
| 441 | return 0; | 421 | return 0; |
| 442 | } | 422 | } |
| 443 | 423 | ||
| 444 | static int hqd_sdma_dump_v10_3(struct kgd_dev *kgd, | 424 | static int hqd_sdma_dump_v10_3(struct amdgpu_device *adev, |
| 445 | uint32_t engine_id, uint32_t queue_id, | 425 | uint32_t engine_id, uint32_t queue_id, |
| 446 | uint32_t (**dump)[2], uint32_t *n_regs) | 426 | uint32_t (**dump)[2], uint32_t *n_regs) |
| 447 | { | 427 | { |
| 448 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 449 | uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, | 428 | uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, |
| 450 | engine_id, queue_id); | 429 | engine_id, queue_id); |
| 451 | uint32_t i = 0, reg; | 430 | uint32_t i = 0, reg; |
| @@ -473,15 +452,15 @@ static int hqd_sdma_dump_v10_3(struct kgd_dev *kgd, · 21 unmodified lines | |||
| 473 | return 0; | 452 | return 0; |
| 474 | } | 453 | } |
| 475 | 454 | ||
| 476 | static bool hqd_is_occupied_v10_3(struct kgd_dev *kgd, uint64_t queue_address, | 455 | static bool hqd_is_occupied_v10_3(struct amdgpu_device *adev, |
| 477 | uint32_t pipe_id, uint32_t queue_id) | 456 | uint64_t queue_address, uint32_t pipe_id, |
| 457 | uint32_t queue_id) | ||
| 478 | { | 458 | { |
| 479 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 480 | uint32_t act; | 459 | uint32_t act; |
| 481 | bool retval = false; | 460 | bool retval = false; |
| 482 | uint32_t low, high; | 461 | uint32_t low, high; |
| 483 | 462 | ||
| 484 | acquire_queue(kgd, pipe_id, queue_id); | 463 | acquire_queue(adev, pipe_id, queue_id); |
| 485 | act = RREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE); | 464 | act = RREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE); |
| 486 | if (act) { | 465 | if (act) { |
| 487 | low = lower_32_bits(queue_address >> 8); | 466 | low = lower_32_bits(queue_address >> 8); |
| @@ -491,13 +470,13 @@ static bool hqd_is_occupied_v10_3(struct kgd_dev *kgd, uint64_t queue_address, · 3 unmodified lines | |||
| 491 | high == RREG32_SOC15(GC, 0, mmCP_HQD_PQ_BASE_HI)) | 470 | high == RREG32_SOC15(GC, 0, mmCP_HQD_PQ_BASE_HI)) |
| 492 | retval = true; | 471 | retval = true; |
| 493 | } | 472 | } |
| 494 | release_queue(kgd); | 473 | release_queue(adev); |
| 495 | return retval; | 474 | return retval; |
| 496 | } | 475 | } |
| 497 | 476 | ||
| 498 | static bool hqd_sdma_is_occupied_v10_3(struct kgd_dev *kgd, void *mqd) | 477 | static bool hqd_sdma_is_occupied_v10_3(struct amdgpu_device *adev, |
| 478 | void *mqd) | ||
| 499 | { | 479 | { |
| 500 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 501 | struct v10_sdma_mqd *m; | 480 | struct v10_sdma_mqd *m; |
| 502 | uint32_t sdma_rlc_reg_offset; | 481 | uint32_t sdma_rlc_reg_offset; |
| 503 | uint32_t sdma_rlc_rb_cntl; | 482 | uint32_t sdma_rlc_rb_cntl; |
| @@ -514,18 +493,17 @@ static bool hqd_sdma_is_occupied_v10_3(struct kgd_dev *kgd, void *mqd) · 10 unmodified lines | |||
| 514 | return false; | 493 | return false; |
| 515 | } | 494 | } |
| 516 | 495 | ||
| 517 | static int hqd_destroy_v10_3(struct kgd_dev *kgd, void *mqd, | 496 | static int hqd_destroy_v10_3(struct amdgpu_device *adev, void *mqd, |
| 518 | enum kfd_preempt_type reset_type, | 497 | enum kfd_preempt_type reset_type, |
| 519 | unsigned int utimeout, uint32_t pipe_id, | 498 | unsigned int utimeout, uint32_t pipe_id, |
| 520 | uint32_t queue_id) | 499 | uint32_t queue_id) |
| 521 | { | 500 | { |
| 522 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 523 | enum hqd_dequeue_request_type type; | 501 | enum hqd_dequeue_request_type type; |
| 524 | unsigned long end_jiffies; | 502 | unsigned long end_jiffies; |
| 525 | uint32_t temp; | 503 | uint32_t temp; |
| 526 | struct v10_compute_mqd *m = get_mqd(mqd); | 504 | struct v10_compute_mqd *m = get_mqd(mqd); |
| 527 | 505 | ||
| 528 | acquire_queue(kgd, pipe_id, queue_id); | 506 | acquire_queue(adev, pipe_id, queue_id); |
| 529 | 507 | ||
| 530 | if (m->cp_hqd_vmid == 0) | 508 | if (m->cp_hqd_vmid == 0) |
| 531 | WREG32_FIELD15(GC, 0, RLC_CP_SCHEDULERS, scheduler1, 0); | 509 | WREG32_FIELD15(GC, 0, RLC_CP_SCHEDULERS, scheduler1, 0); |
| @@ -555,20 +533,19 @@ static int hqd_destroy_v10_3(struct kgd_dev *kgd, void *mqd, · 23 unmodified lines | |||
| 555 | if (time_after(jiffies, end_jiffies)) { | 533 | if (time_after(jiffies, end_jiffies)) { |
| 556 | pr_err("cp queue pipe %d queue %d preemption failed\n", | 534 | pr_err("cp queue pipe %d queue %d preemption failed\n", |
| 557 | pipe_id, queue_id); | 535 | pipe_id, queue_id); |
| 558 | release_queue(kgd); | 536 | release_queue(adev); |
| 559 | return -ETIME; | 537 | return -ETIME; |
| 560 | } | 538 | } |
| 561 | usleep_range(500, 1000); | 539 | usleep_range(500, 1000); |
| 562 | } | 540 | } |
| 563 | 541 | ||
| 564 | release_queue(kgd); | 542 | release_queue(adev); |
| 565 | return 0; | 543 | return 0; |
| 566 | } | 544 | } |
| 567 | 545 | ||
| 568 | static int hqd_sdma_destroy_v10_3(struct kgd_dev *kgd, void *mqd, | 546 | static int hqd_sdma_destroy_v10_3(struct amdgpu_device *adev, void *mqd, |
| 569 | unsigned int utimeout) | 547 | unsigned int utimeout) |
| 570 | { | 548 | { |
| 571 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 572 | struct v10_sdma_mqd *m; | 549 | struct v10_sdma_mqd *m; |
| 573 | uint32_t sdma_rlc_reg_offset; | 550 | uint32_t sdma_rlc_reg_offset; |
| 574 | uint32_t temp; | 551 | uint32_t temp; |
| @@ -606,12 +583,12 @@ static int hqd_sdma_destroy_v10_3(struct kgd_dev *kgd, void *mqd, · 31 unmodified lines | |||
| 606 | } | 583 | } |
| 607 | 584 | ||
| 608 | 585 | ||
| 609 | static int address_watch_disable_v10_3(struct kgd_dev *kgd) | 586 | static int address_watch_disable_v10_3(struct amdgpu_device *adev) |
| 610 | { | 587 | { |
| 611 | return 0; | 588 | return 0; |
| 612 | } | 589 | } |
| 613 | 590 | ||
| 614 | static int address_watch_execute_v10_3(struct kgd_dev *kgd, | 591 | static int address_watch_execute_v10_3(struct amdgpu_device *adev, |
| 615 | unsigned int watch_point_id, | 592 | unsigned int watch_point_id, |
| 616 | uint32_t cntl_val, | 593 | uint32_t cntl_val, |
| 617 | uint32_t addr_hi, | 594 | uint32_t addr_hi, |
| @@ -620,11 +597,10 @@ static int address_watch_execute_v10_3(struct kgd_dev *kgd, · 2 unmodified lines | |||
| 620 | return 0; | 597 | return 0; |
| 621 | } | 598 | } |
| 622 | 599 | ||
| 623 | static int wave_control_execute_v10_3(struct kgd_dev *kgd, | 600 | static int wave_control_execute_v10_3(struct amdgpu_device *adev, |
| 624 | uint32_t gfx_index_val, | 601 | uint32_t gfx_index_val, |
| 625 | uint32_t sq_cmd) | 602 | uint32_t sq_cmd) |
| 626 | { | 603 | { |
| 627 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 628 | uint32_t data = 0; | 604 | uint32_t data = 0; |
| 629 | 605 | ||
| 630 | mutex_lock(&adev->grbm_idx_mutex); | 606 | mutex_lock(&adev->grbm_idx_mutex); |
| @@ -645,28 +621,24 @@ static int wave_control_execute_v10_3(struct kgd_dev *kgd, · 14 unmodified lines | |||
| 645 | return 0; | 621 | return 0; |
| 646 | } | 622 | } |
| 647 | 623 | ||
| 648 | static uint32_t address_watch_get_offset_v10_3(struct kgd_dev *kgd, | 624 | static uint32_t address_watch_get_offset_v10_3(struct amdgpu_device *adev, |
| 649 | unsigned int watch_point_id, | 625 | unsigned int watch_point_id, |
| 650 | unsigned int reg_offset) | 626 | unsigned int reg_offset) |
| 651 | { | 627 | { |
| 652 | return 0; | 628 | return 0; |
| 653 | } | 629 | } |
| 654 | 630 | ||
| 655 | static void set_vm_context_page_table_base_v10_3(struct kgd_dev *kgd, uint32_t vmid, | 631 | static void set_vm_context_page_table_base_v10_3(struct amdgpu_device *adev, |
| 656 | uint64_t page_table_base) | 632 | uint32_t vmid, uint64_t page_table_base) |
| 657 | { | 633 | { |
| 658 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 659 | |||
| 660 | /* SDMA is on gfxhub as well for Navi1* series */ | 634 | /* SDMA is on gfxhub as well for Navi1* series */ |
| 661 | adev->gfxhub.funcs->setup_vm_pt_regs(adev, vmid, page_table_base); | 635 | adev->gfxhub.funcs->setup_vm_pt_regs(adev, vmid, page_table_base); |
| 662 | } | 636 | } |
| 663 | 637 | ||
| 664 | static void program_trap_handler_settings_v10_3(struct kgd_dev *kgd, | 638 | static void program_trap_handler_settings_v10_3(struct amdgpu_device *adev, |
| 665 | uint32_t vmid, uint64_t tba_addr, uint64_t tma_addr) | 639 | uint32_t vmid, uint64_t tba_addr, uint64_t tma_addr) |
| 666 | { | 640 | { |
| 667 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | 641 | lock_srbm(adev, 0, 0, 0, vmid); |
| 668 | |||
| 669 | lock_srbm(kgd, 0, 0, 0, vmid); | ||
| 670 | 642 | ||
| 671 | /* | 643 | /* |
| 672 | * Program TBA registers | 644 | * Program TBA registers |
| @@ -685,15 +657,14 @@ static void program_trap_handler_settings_v10_3(struct kgd_dev *kgd, · 12 unmodified lines | |||
| 685 | WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_SHADER_TMA_HI), | 657 | WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_SHADER_TMA_HI), |
| 686 | upper_32_bits(tma_addr >> 8)); | 658 | upper_32_bits(tma_addr >> 8)); |
| 687 | 659 | ||
| 688 | unlock_srbm(kgd); | 660 | unlock_srbm(adev); |
| 689 | } | 661 | } |
| 690 | 662 | ||
| 691 | #if 0 | 663 | #if 0 |
| 692 | uint32_t enable_debug_trap_v10_3(struct kgd_dev *kgd, | 664 | uint32_t enable_debug_trap_v10_3(struct amdgpu_device *adev, |
| 693 | uint32_t trap_debug_wave_launch_mode, | 665 | uint32_t trap_debug_wave_launch_mode, |
| 694 | uint32_t vmid) | 666 | uint32_t vmid) |
| 695 | { | 667 | { |
| 696 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 697 | uint32_t data = 0; | 668 | uint32_t data = 0; |
| 698 | uint32_t orig_wave_cntl_value; | 669 | uint32_t orig_wave_cntl_value; |
| 699 | uint32_t orig_stall_vmid; | 670 | uint32_t orig_stall_vmid; |
| @@ -720,10 +691,8 @@ uint32_t enable_debug_trap_v10_3(struct kgd_dev *kgd, · 20 unmodified lines | |||
| 720 | return 0; | 691 | return 0; |
| 721 | } | 692 | } |
| 722 | 693 | ||
| 723 | uint32_t disable_debug_trap_v10_3(struct kgd_dev *kgd) | 694 | uint32_t disable_debug_trap_v10_3(struct amdgpu_device *adev) |
| 724 | { | 695 | { |
| 725 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 726 | |||
| 727 | mutex_lock(&adev->grbm_idx_mutex); | 696 | mutex_lock(&adev->grbm_idx_mutex); |
| 728 | 697 | ||
| 729 | WREG32(SOC15_REG_OFFSET(GC, 0, mmSPI_GDBG_TRAP_MASK), 0); | 698 | WREG32(SOC15_REG_OFFSET(GC, 0, mmSPI_GDBG_TRAP_MASK), 0); |
| @@ -733,11 +702,10 @@ uint32_t disable_debug_trap_v10_3(struct kgd_dev *kgd) · 3 unmodified lines | |||
| 733 | return 0; | 702 | return 0; |
| 734 | } | 703 | } |
| 735 | 704 | ||
| 736 | uint32_t set_wave_launch_trap_override_v10_3(struct kgd_dev *kgd, | 705 | uint32_t set_wave_launch_trap_override_v10_3(struct amdgpu_device *adev, |
| 737 | uint32_t trap_override, | 706 | uint32_t trap_override, |
| 738 | uint32_t trap_mask) | 707 | uint32_t trap_mask) |
| 739 | { | 708 | { |
| 740 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 741 | uint32_t data = 0; | 709 | uint32_t data = 0; |
| 742 | 710 | ||
| 743 | mutex_lock(&adev->grbm_idx_mutex); | 711 | mutex_lock(&adev->grbm_idx_mutex); |
| @@ -762,11 +730,10 @@ uint32_t set_wave_launch_trap_override_v10_3(struct kgd_dev *kgd, · 18 unmodified lines | |||
| 762 | return 0; | 730 | return 0; |
| 763 | } | 731 | } |
| 764 | 732 | ||
| 765 | uint32_t set_wave_launch_mode_v10_3(struct kgd_dev *kgd, | 733 | uint32_t set_wave_launch_mode_v10_3(struct amdgpu_device *adev, |
| 766 | uint8_t wave_launch_mode, | 734 | uint8_t wave_launch_mode, |
| 767 | uint32_t vmid) | 735 | uint32_t vmid) |
| 768 | { | 736 | { |
| 769 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 770 | uint32_t data = 0; | 737 | uint32_t data = 0; |
| 771 | bool is_stall_mode; | 738 | bool is_stall_mode; |
| 772 | bool is_mode_set; | 739 | bool is_mode_set; |
| @@ -805,16 +772,14 @@ uint32_t set_wave_launch_mode_v10_3(struct kgd_dev *kgd, · 32 unmodified lines | |||
| 805 | * sem_rearm_wait_time -- Wait Count for Semaphore re-arm. | 772 | * sem_rearm_wait_time -- Wait Count for Semaphore re-arm. |
| 806 | * deq_retry_wait_time -- Wait Count for Global Wave Syncs. | 773 | * deq_retry_wait_time -- Wait Count for Global Wave Syncs. |
| 807 | */ | 774 | */ |
| 808 | void get_iq_wait_times_v10_3(struct kgd_dev *kgd, | 775 | void get_iq_wait_times_v10_3(struct amdgpu_device *adev, |
| 809 | uint32_t *wait_times) | 776 | uint32_t *wait_times) |
| 810 | 777 | ||
| 811 | { | 778 | { |
| 812 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 813 | |||
| 814 | *wait_times = RREG32(SOC15_REG_OFFSET(GC, 0, mmCP_IQ_WAIT_TIME2)); | 779 | *wait_times = RREG32(SOC15_REG_OFFSET(GC, 0, mmCP_IQ_WAIT_TIME2)); |
| 815 | } | 780 | } |
| 816 | 781 | ||
| 817 | void build_grace_period_packet_info_v10_3(struct kgd_dev *kgd, | 782 | void build_grace_period_packet_info_v10_3(struct amdgpu_device *adev, |
| 818 | uint32_t wait_times, | 783 | uint32_t wait_times, |
| 819 | uint32_t grace_period, | 784 | uint32_t grace_period, |
| 820 | uint32_t *reg_offset, | 785 | uint32_t *reg_offset, |
| @@ -82,68 +82,54 @@ union TCP_WATCH_CNTL_BITS { · 81 unmodified lines | |||
| 82 | float f32All; | 82 | float f32All; |
| 83 | }; | 83 | }; |
| 84 | 84 | ||
| 85 | static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd) | 85 | static void lock_srbm(struct amdgpu_device *adev, uint32_t mec, uint32_t pipe, |
| 86 | { | ||
| 87 | return (struct amdgpu_device *)kgd; | ||
| 88 | } | ||
| 89 | |||
| 90 | static void lock_srbm(struct kgd_dev *kgd, uint32_t mec, uint32_t pipe, | ||
| 91 | uint32_t queue, uint32_t vmid) | 86 | uint32_t queue, uint32_t vmid) |
| 92 | { | 87 | { |
| 93 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 94 | uint32_t value = PIPEID(pipe) | MEID(mec) | VMID(vmid) | QUEUEID(queue); | 88 | uint32_t value = PIPEID(pipe) | MEID(mec) | VMID(vmid) | QUEUEID(queue); |
| 95 | 89 | ||
| 96 | mutex_lock(&adev->srbm_mutex); | 90 | mutex_lock(&adev->srbm_mutex); |
| 97 | WREG32(mmSRBM_GFX_CNTL, value); | 91 | WREG32(mmSRBM_GFX_CNTL, value); |
| 98 | } | 92 | } |
| 99 | 93 | ||
| 100 | static void unlock_srbm(struct kgd_dev *kgd) | 94 | static void unlock_srbm(struct amdgpu_device *adev) |
| 101 | { | 95 | { |
| 102 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 103 | |||
| 104 | WREG32(mmSRBM_GFX_CNTL, 0); | 96 | WREG32(mmSRBM_GFX_CNTL, 0); |
| 105 | mutex_unlock(&adev->srbm_mutex); | 97 | mutex_unlock(&adev->srbm_mutex); |
| 106 | } | 98 | } |
| 107 | 99 | ||
| 108 | static void acquire_queue(struct kgd_dev *kgd, uint32_t pipe_id, | 100 | static void acquire_queue(struct amdgpu_device *adev, uint32_t pipe_id, |
| 109 | uint32_t queue_id) | 101 | uint32_t queue_id) |
| 110 | { | 102 | { |
| 111 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 112 | |||
| 113 | uint32_t mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; | 103 | uint32_t mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; |
| 114 | uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); | 104 | uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); |
| 115 | 105 | ||
| 116 | lock_srbm(kgd, mec, pipe, queue_id, 0); | 106 | lock_srbm(adev, mec, pipe, queue_id, 0); |
| 117 | } | 107 | } |
| 118 | 108 | ||
| 119 | static void release_queue(struct kgd_dev *kgd) | 109 | static void release_queue(struct amdgpu_device *adev) |
| 120 | { | 110 | { |
| 121 | unlock_srbm(kgd); | 111 | unlock_srbm(adev); |
| 122 | } | 112 | } |
| 123 | 113 | ||
| 124 | static void kgd_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid, | 114 | static void kgd_program_sh_mem_settings(struct amdgpu_device *adev, uint32_t vmid, |
| 125 | uint32_t sh_mem_config, | 115 | uint32_t sh_mem_config, |
| 126 | uint32_t sh_mem_ape1_base, | 116 | uint32_t sh_mem_ape1_base, |
| 127 | uint32_t sh_mem_ape1_limit, | 117 | uint32_t sh_mem_ape1_limit, |
| 128 | uint32_t sh_mem_bases) | 118 | uint32_t sh_mem_bases) |
| 129 | { | 119 | { |
| 130 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | 120 | lock_srbm(adev, 0, 0, 0, vmid); |
| 131 | |||
| 132 | lock_srbm(kgd, 0, 0, 0, vmid); | ||
| 133 | 121 | ||
| 134 | WREG32(mmSH_MEM_CONFIG, sh_mem_config); | 122 | WREG32(mmSH_MEM_CONFIG, sh_mem_config); |
| 135 | WREG32(mmSH_MEM_APE1_BASE, sh_mem_ape1_base); | 123 | WREG32(mmSH_MEM_APE1_BASE, sh_mem_ape1_base); |
| 136 | WREG32(mmSH_MEM_APE1_LIMIT, sh_mem_ape1_limit); | 124 | WREG32(mmSH_MEM_APE1_LIMIT, sh_mem_ape1_limit); |
| 137 | WREG32(mmSH_MEM_BASES, sh_mem_bases); | 125 | WREG32(mmSH_MEM_BASES, sh_mem_bases); |
| 138 | 126 | ||
| 139 | unlock_srbm(kgd); | 127 | unlock_srbm(adev); |
| 140 | } | 128 | } |
| 141 | 129 | ||
| 142 | static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid, | 130 | static int kgd_set_pasid_vmid_mapping(struct amdgpu_device *adev, u32 pasid, |
| 143 | unsigned int vmid) | 131 | unsigned int vmid) |
| 144 | { | 132 | { |
| 145 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 146 | |||
| 147 | /* | 133 | /* |
| 148 | * We have to assume that there is no outstanding mapping. | 134 | * We have to assume that there is no outstanding mapping. |
| 149 | * The ATC_VMID_PASID_MAPPING_UPDATE_STATUS bit could be 0 because | 135 | * The ATC_VMID_PASID_MAPPING_UPDATE_STATUS bit could be 0 because |
| @@ -165,21 +151,20 @@ static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid, · 15 unmodified lines | |||
| 165 | return 0; | 151 | return 0; |
| 166 | } | 152 | } |
| 167 | 153 | ||
| 168 | static int kgd_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id) | 154 | static int kgd_init_interrupts(struct amdgpu_device *adev, uint32_t pipe_id) |
| 169 | { | 155 | { |
| 170 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 171 | uint32_t mec; | 156 | uint32_t mec; |
| 172 | uint32_t pipe; | 157 | uint32_t pipe; |
| 173 | 158 | ||
| 174 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; | 159 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; |
| 175 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); | 160 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); |
| 176 | 161 | ||
| 177 | lock_srbm(kgd, mec, pipe, 0, 0); | 162 | lock_srbm(adev, mec, pipe, 0, 0); |
| 178 | 163 | ||
| 179 | WREG32(mmCPC_INT_CNTL, CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK | | 164 | WREG32(mmCPC_INT_CNTL, CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK | |
| 180 | CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK); | 165 | CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK); |
| 181 | 166 | ||
| 182 | unlock_srbm(kgd); | 167 | unlock_srbm(adev); |
| 183 | 168 | ||
| 184 | return 0; | 169 | return 0; |
| 185 | } | 170 | } |
| @@ -207,12 +192,11 @@ static inline struct cik_sdma_rlc_registers *get_sdma_mqd(void *mqd) · 21 unmodified lines | |||
| 207 | return (struct cik_sdma_rlc_registers *)mqd; | 192 | return (struct cik_sdma_rlc_registers *)mqd; |
| 208 | } | 193 | } |
| 209 | 194 | ||
| 210 | static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, | 195 | static int kgd_hqd_load(struct amdgpu_device *adev, void *mqd, |
| 211 | uint32_t queue_id, uint32_t __user *wptr, | 196 | uint32_t pipe_id, uint32_t queue_id, |
| 212 | uint32_t wptr_shift, uint32_t wptr_mask, | 197 | uint32_t __user *wptr, uint32_t wptr_shift, |
| 213 | struct mm_struct *mm) | 198 | uint32_t wptr_mask, struct mm_struct *mm) |
| 214 | { | 199 | { |
| 215 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 216 | struct cik_mqd *m; | 200 | struct cik_mqd *m; |
| 217 | uint32_t *mqd_hqd; | 201 | uint32_t *mqd_hqd; |
| 218 | uint32_t reg, wptr_val, data; | 202 | uint32_t reg, wptr_val, data; |
| @@ -220,7 +204,7 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, · 1 unmodified line | |||
| 220 | 204 | ||
| 221 | m = get_mqd(mqd); | 205 | m = get_mqd(mqd); |
| 222 | 206 | ||
| 223 | acquire_queue(kgd, pipe_id, queue_id); | 207 | acquire_queue(adev, pipe_id, queue_id); |
| 224 | 208 | ||
| 225 | /* HQD registers extend from CP_MQD_BASE_ADDR to CP_MQD_CONTROL. */ | 209 | /* HQD registers extend from CP_MQD_BASE_ADDR to CP_MQD_CONTROL. */ |
| 226 | mqd_hqd = &m->cp_mqd_base_addr_lo; | 210 | mqd_hqd = &m->cp_mqd_base_addr_lo; |
| @@ -239,25 +223,24 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, · 12 unmodified lines | |||
| 239 | * release srbm_mutex to avoid circular dependency between | 223 | * release srbm_mutex to avoid circular dependency between |
| 240 | * srbm_mutex->mm_sem->reservation_ww_class_mutex->srbm_mutex. | 224 | * srbm_mutex->mm_sem->reservation_ww_class_mutex->srbm_mutex. |
| 241 | */ | 225 | */ |
| 242 | release_queue(kgd); | 226 | release_queue(adev); |
| 243 | valid_wptr = read_user_wptr(mm, wptr, wptr_val); | 227 | valid_wptr = read_user_wptr(mm, wptr, wptr_val); |
| 244 | acquire_queue(kgd, pipe_id, queue_id); | 228 | acquire_queue(adev, pipe_id, queue_id); |
| 245 | if (valid_wptr) | 229 | if (valid_wptr) |
| 246 | WREG32(mmCP_HQD_PQ_WPTR, (wptr_val << wptr_shift) & wptr_mask); | 230 | WREG32(mmCP_HQD_PQ_WPTR, (wptr_val << wptr_shift) & wptr_mask); |
| 247 | 231 | ||
| 248 | data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1); | 232 | data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1); |
| 249 | WREG32(mmCP_HQD_ACTIVE, data); | 233 | WREG32(mmCP_HQD_ACTIVE, data); |
| 250 | 234 | ||
| 251 | release_queue(kgd); | 235 | release_queue(adev); |
| 252 | 236 | ||
| 253 | return 0; | 237 | return 0; |
| 254 | } | 238 | } |
| 255 | 239 | ||
| 256 | static int kgd_hqd_dump(struct kgd_dev *kgd, | 240 | static int kgd_hqd_dump(struct amdgpu_device *adev, |
| 257 | uint32_t pipe_id, uint32_t queue_id, | 241 | uint32_t pipe_id, uint32_t queue_id, |
| 258 | uint32_t (**dump)[2], uint32_t *n_regs) | 242 | uint32_t (**dump)[2], uint32_t *n_regs) |
| 259 | { | 243 | { |
| 260 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 261 | uint32_t i = 0, reg; | 244 | uint32_t i = 0, reg; |
| 262 | #define HQD_N_REGS (35+4) | 245 | #define HQD_N_REGS (35+4) |
| 263 | #define DUMP_REG(addr) do { \ | 246 | #define DUMP_REG(addr) do { \ |
| @@ -271,7 +254,7 @@ static int kgd_hqd_dump(struct kgd_dev *kgd, · 7 unmodified lines | |||
| 271 | if (*dump == NULL) | 254 | if (*dump == NULL) |
| 272 | return -ENOMEM; | 255 | return -ENOMEM; |
| 273 | 256 | ||
| 274 | acquire_queue(kgd, pipe_id, queue_id); | 257 | acquire_queue(adev, pipe_id, queue_id); |
| 275 | 258 | ||
| 276 | DUMP_REG(mmCOMPUTE_STATIC_THREAD_MGMT_SE0); | 259 | DUMP_REG(mmCOMPUTE_STATIC_THREAD_MGMT_SE0); |
| 277 | DUMP_REG(mmCOMPUTE_STATIC_THREAD_MGMT_SE1); | 260 | DUMP_REG(mmCOMPUTE_STATIC_THREAD_MGMT_SE1); |
| @@ -281,7 +264,7 @@ static int kgd_hqd_dump(struct kgd_dev *kgd, · 3 unmodified lines | |||
| 281 | for (reg = mmCP_MQD_BASE_ADDR; reg <= mmCP_MQD_CONTROL; reg++) | 264 | for (reg = mmCP_MQD_BASE_ADDR; reg <= mmCP_MQD_CONTROL; reg++) |
| 282 | DUMP_REG(reg); | 265 | DUMP_REG(reg); |
| 283 | 266 | ||
| 284 | release_queue(kgd); | 267 | release_queue(adev); |
| 285 | 268 | ||
| 286 | WARN_ON_ONCE(i != HQD_N_REGS); | 269 | WARN_ON_ONCE(i != HQD_N_REGS); |
| 287 | *n_regs = i; | 270 | *n_regs = i; |
| @@ -289,10 +272,9 @@ static int kgd_hqd_dump(struct kgd_dev *kgd, · 1 unmodified line | |||
| 289 | return 0; | 272 | return 0; |
| 290 | } | 273 | } |
| 291 | 274 | ||
| 292 | static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, | 275 | static int kgd_hqd_sdma_load(struct amdgpu_device *adev, void *mqd, |
| 293 | uint32_t __user *wptr, struct mm_struct *mm) | 276 | uint32_t __user *wptr, struct mm_struct *mm) |
| 294 | { | 277 | { |
| 295 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 296 | struct cik_sdma_rlc_registers *m; | 278 | struct cik_sdma_rlc_registers *m; |
| 297 | unsigned long end_jiffies; | 279 | unsigned long end_jiffies; |
| 298 | uint32_t sdma_rlc_reg_offset; | 280 | uint32_t sdma_rlc_reg_offset; |
| @@ -345,11 +327,10 @@ static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, · 46 unmodified lines | |||
| 345 | return 0; | 327 | return 0; |
| 346 | } | 328 | } |
| 347 | 329 | ||
| 348 | static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, | 330 | static int kgd_hqd_sdma_dump(struct amdgpu_device *adev, |
| 349 | uint32_t engine_id, uint32_t queue_id, | 331 | uint32_t engine_id, uint32_t queue_id, |
| 350 | uint32_t (**dump)[2], uint32_t *n_regs) | 332 | uint32_t (**dump)[2], uint32_t *n_regs) |
| 351 | { | 333 | { |
| 352 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 353 | uint32_t sdma_offset = engine_id * SDMA1_REGISTER_OFFSET + | 334 | uint32_t sdma_offset = engine_id * SDMA1_REGISTER_OFFSET + |
| 354 | queue_id * KFD_CIK_SDMA_QUEUE_OFFSET; | 335 | queue_id * KFD_CIK_SDMA_QUEUE_OFFSET; |
| 355 | uint32_t i = 0, reg; | 336 | uint32_t i = 0, reg; |
| @@ -372,15 +353,15 @@ static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, · 16 unmodified lines | |||
| 372 | return 0; | 353 | return 0; |
| 373 | } | 354 | } |
| 374 | 355 | ||
| 375 | static bool kgd_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address, | 356 | static bool kgd_hqd_is_occupied(struct amdgpu_device *adev, |
| 376 | uint32_t pipe_id, uint32_t queue_id) | 357 | uint64_t queue_address, uint32_t pipe_id, |
| 358 | uint32_t queue_id) | ||
| 377 | { | 359 | { |
| 378 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 379 | uint32_t act; | 360 | uint32_t act; |
| 380 | bool retval = false; | 361 | bool retval = false; |
| 381 | uint32_t low, high; | 362 | uint32_t low, high; |
| 382 | 363 | ||
| 383 | acquire_queue(kgd, pipe_id, queue_id); | 364 | acquire_queue(adev, pipe_id, queue_id); |
| 384 | act = RREG32(mmCP_HQD_ACTIVE); | 365 | act = RREG32(mmCP_HQD_ACTIVE); |
| 385 | if (act) { | 366 | if (act) { |
| 386 | low = lower_32_bits(queue_address >> 8); | 367 | low = lower_32_bits(queue_address >> 8); |
| @@ -390,13 +371,12 @@ static bool kgd_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address, · 3 unmodified lines | |||
| 390 | high == RREG32(mmCP_HQD_PQ_BASE_HI)) | 371 | high == RREG32(mmCP_HQD_PQ_BASE_HI)) |
| 391 | retval = true; | 372 | retval = true; |
| 392 | } | 373 | } |
| 393 | release_queue(kgd); | 374 | release_queue(adev); |
| 394 | return retval; | 375 | return retval; |
| 395 | } | 376 | } |
| 396 | 377 | ||
| 397 | static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd) | 378 | static bool kgd_hqd_sdma_is_occupied(struct amdgpu_device *adev, void *mqd) |
| 398 | { | 379 | { |
| 399 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 400 | struct cik_sdma_rlc_registers *m; | 380 | struct cik_sdma_rlc_registers *m; |
| 401 | uint32_t sdma_rlc_reg_offset; | 381 | uint32_t sdma_rlc_reg_offset; |
| 402 | uint32_t sdma_rlc_rb_cntl; | 382 | uint32_t sdma_rlc_rb_cntl; |
| @@ -412,12 +392,11 @@ static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd) · 9 unmodified lines | |||
| 412 | return false; | 392 | return false; |
| 413 | } | 393 | } |
| 414 | 394 | ||
| 415 | static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd, | 395 | static int kgd_hqd_destroy(struct amdgpu_device *adev, void *mqd, |
| 416 | enum kfd_preempt_type reset_type, | 396 | enum kfd_preempt_type reset_type, |
| 417 | unsigned int utimeout, uint32_t pipe_id, | 397 | unsigned int utimeout, uint32_t pipe_id, |
| 418 | uint32_t queue_id) | 398 | uint32_t queue_id) |
| 419 | { | 399 | { |
| 420 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 421 | uint32_t temp; | 400 | uint32_t temp; |
| 422 | enum hqd_dequeue_request_type type; | 401 | enum hqd_dequeue_request_type type; |
| 423 | unsigned long flags, end_jiffies; | 402 | unsigned long flags, end_jiffies; |
| @@ -426,7 +405,7 @@ static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd, · 2 unmodified lines | |||
| 426 | if (amdgpu_in_reset(adev)) | 405 | if (amdgpu_in_reset(adev)) |
| 427 | return -EIO; | 406 | return -EIO; |
| 428 | 407 | ||
| 429 | acquire_queue(kgd, pipe_id, queue_id); | 408 | acquire_queue(adev, pipe_id, queue_id); |
| 430 | WREG32(mmCP_HQD_PQ_DOORBELL_CONTROL, 0); | 409 | WREG32(mmCP_HQD_PQ_DOORBELL_CONTROL, 0); |
| 431 | 410 | ||
| 432 | switch (reset_type) { | 411 | switch (reset_type) { |
| @@ -504,20 +483,19 @@ loop: · 71 unmodified lines | |||
| 504 | break; | 483 | break; |
| 505 | if (time_after(jiffies, end_jiffies)) { | 484 | if (time_after(jiffies, end_jiffies)) { |
| 506 | pr_err("cp queue preemption time out\n"); | 485 | pr_err("cp queue preemption time out\n"); |
| 507 | release_queue(kgd); | 486 | release_queue(adev); |
| 508 | return -ETIME; | 487 | return -ETIME; |
| 509 | } | 488 | } |
| 510 | usleep_range(500, 1000); | 489 | usleep_range(500, 1000); |
| 511 | } | 490 | } |
| 512 | 491 | ||
| 513 | release_queue(kgd); | 492 | release_queue(adev); |
| 514 | return 0; | 493 | return 0; |
| 515 | } | 494 | } |
| 516 | 495 | ||
| 517 | static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, | 496 | static int kgd_hqd_sdma_destroy(struct amdgpu_device *adev, void *mqd, |
| 518 | unsigned int utimeout) | 497 | unsigned int utimeout) |
| 519 | { | 498 | { |
| 520 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 521 | struct cik_sdma_rlc_registers *m; | 499 | struct cik_sdma_rlc_registers *m; |
| 522 | uint32_t sdma_rlc_reg_offset; | 500 | uint32_t sdma_rlc_reg_offset; |
| 523 | uint32_t temp; | 501 | uint32_t temp; |
| @@ -551,9 +529,8 @@ static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, · 27 unmodified lines | |||
| 551 | return 0; | 529 | return 0; |
| 552 | } | 530 | } |
| 553 | 531 | ||
| 554 | static int kgd_address_watch_disable(struct kgd_dev *kgd) | 532 | static int kgd_address_watch_disable(struct amdgpu_device *adev) |
| 555 | { | 533 | { |
| 556 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 557 | union TCP_WATCH_CNTL_BITS cntl; | 534 | union TCP_WATCH_CNTL_BITS cntl; |
| 558 | unsigned int i; | 535 | unsigned int i; |
| 559 | 536 | ||
| @@ -571,13 +548,12 @@ static int kgd_address_watch_disable(struct kgd_dev *kgd) · 11 unmodified lines | |||
| 571 | return 0; | 548 | return 0; |
| 572 | } | 549 | } |
| 573 | 550 | ||
| 574 | static int kgd_address_watch_execute(struct kgd_dev *kgd, | 551 | static int kgd_address_watch_execute(struct amdgpu_device *adev, |
| 575 | unsigned int watch_point_id, | 552 | unsigned int watch_point_id, |
| 576 | uint32_t cntl_val, | 553 | uint32_t cntl_val, |
| 577 | uint32_t addr_hi, | 554 | uint32_t addr_hi, |
| 578 | uint32_t addr_lo) | 555 | uint32_t addr_lo) |
| 579 | { | 556 | { |
| 580 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 581 | union TCP_WATCH_CNTL_BITS cntl; | 557 | union TCP_WATCH_CNTL_BITS cntl; |
| 582 | 558 | ||
| 583 | cntl.u32All = cntl_val; | 559 | cntl.u32All = cntl_val; |
| @@ -602,11 +578,10 @@ static int kgd_address_watch_execute(struct kgd_dev *kgd, · 18 unmodified lines | |||
| 602 | return 0; | 578 | return 0; |
| 603 | } | 579 | } |
| 604 | 580 | ||
| 605 | static int kgd_wave_control_execute(struct kgd_dev *kgd, | 581 | static int kgd_wave_control_execute(struct amdgpu_device *adev, |
| 606 | uint32_t gfx_index_val, | 582 | uint32_t gfx_index_val, |
| 607 | uint32_t sq_cmd) | 583 | uint32_t sq_cmd) |
| 608 | { | 584 | { |
| 609 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 610 | uint32_t data; | 585 | uint32_t data; |
| 611 | 586 | ||
| 612 | mutex_lock(&adev->grbm_idx_mutex); | 587 | mutex_lock(&adev->grbm_idx_mutex); |
| @@ -627,18 +602,17 @@ static int kgd_wave_control_execute(struct kgd_dev *kgd, · 14 unmodified lines | |||
| 627 | return 0; | 602 | return 0; |
| 628 | } | 603 | } |
| 629 | 604 | ||
| 630 | static uint32_t kgd_address_watch_get_offset(struct kgd_dev *kgd, | 605 | static uint32_t kgd_address_watch_get_offset(struct amdgpu_device *adev, |
| 631 | unsigned int watch_point_id, | 606 | unsigned int watch_point_id, |
| 632 | unsigned int reg_offset) | 607 | unsigned int reg_offset) |
| 633 | { | 608 | { |
| 634 | return watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX + reg_offset]; | 609 | return watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX + reg_offset]; |
| 635 | } | 610 | } |
| 636 | 611 | ||
| 637 | static bool get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd, | 612 | static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, |
| 638 | uint8_t vmid, uint16_t *p_pasid) | 613 | uint8_t vmid, uint16_t *p_pasid) |
| 639 | { | 614 | { |
| 640 | uint32_t value; | 615 | uint32_t value; |
| 641 | struct amdgpu_device *adev = (struct amdgpu_device *) kgd; | ||
| 642 | 616 | ||
| 643 | value = RREG32(mmATC_VMID0_PASID_MAPPING + vmid); | 617 | value = RREG32(mmATC_VMID0_PASID_MAPPING + vmid); |
| 644 | *p_pasid = value & ATC_VMID0_PASID_MAPPING__PASID_MASK; | 618 | *p_pasid = value & ATC_VMID0_PASID_MAPPING__PASID_MASK; |
| @@ -646,21 +620,17 @@ static bool get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd, · 1 unmodified line | |||
| 646 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); | 620 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); |
| 647 | } | 621 | } |
| 648 | 622 | ||
| 649 | static void set_scratch_backing_va(struct kgd_dev *kgd, | 623 | static void set_scratch_backing_va(struct amdgpu_device *adev, |
| 650 | uint64_t va, uint32_t vmid) | 624 | uint64_t va, uint32_t vmid) |
| 651 | { | 625 | { |
| 652 | struct amdgpu_device *adev = (struct amdgpu_device *) kgd; | 626 | lock_srbm(adev, 0, 0, 0, vmid); |
| 653 | |||
| 654 | lock_srbm(kgd, 0, 0, 0, vmid); | ||
| 655 | WREG32(mmSH_HIDDEN_PRIVATE_BASE_VMID, va); | 627 | WREG32(mmSH_HIDDEN_PRIVATE_BASE_VMID, va); |
| 656 | unlock_srbm(kgd); | 628 | unlock_srbm(adev); |
| 657 | } | 629 | } |
| 658 | 630 | ||
| 659 | static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, | 631 | static void set_vm_context_page_table_base(struct amdgpu_device *adev, |
| 660 | uint64_t page_table_base) | 632 | uint32_t vmid, uint64_t page_table_base) |
| 661 | { | 633 | { |
| 662 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 663 | |||
| 664 | if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) { | 634 | if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) { |
| 665 | pr_err("trying to set page table base for wrong VMID\n"); | 635 | pr_err("trying to set page table base for wrong VMID\n"); |
| 666 | return; | 636 | return; |
| @@ -676,10 +646,8 @@ static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, · 9 unmodified lines | |||
| 676 | * @vmid: vmid pointer | 646 | * @vmid: vmid pointer |
| 677 | * read vmid from register (CIK). | 647 | * read vmid from register (CIK). |
| 678 | */ | 648 | */ |
| 679 | static uint32_t read_vmid_from_vmfault_reg(struct kgd_dev *kgd) | 649 | static uint32_t read_vmid_from_vmfault_reg(struct amdgpu_device *adev) |
| 680 | { | 650 | { |
| 681 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 682 | |||
| 683 | uint32_t status = RREG32(mmVM_CONTEXT1_PROTECTION_FAULT_STATUS); | 651 | uint32_t status = RREG32(mmVM_CONTEXT1_PROTECTION_FAULT_STATUS); |
| 684 | 652 | ||
| 685 | return REG_GET_FIELD(status, VM_CONTEXT1_PROTECTION_FAULT_STATUS, VMID); | 653 | return REG_GET_FIELD(status, VM_CONTEXT1_PROTECTION_FAULT_STATUS, VMID); |
| @@ -39,68 +39,54 @@ enum hqd_dequeue_request_type { · 38 unmodified lines | |||
| 39 | RESET_WAVES | 39 | RESET_WAVES |
| 40 | }; | 40 | }; |
| 41 | 41 | ||
| 42 | static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd) | 42 | static void lock_srbm(struct amdgpu_device *adev, uint32_t mec, uint32_t pipe, |
| 43 | { | ||
| 44 | return (struct amdgpu_device *)kgd; | ||
| 45 | } | ||
| 46 | |||
| 47 | static void lock_srbm(struct kgd_dev *kgd, uint32_t mec, uint32_t pipe, | ||
| 48 | uint32_t queue, uint32_t vmid) | 43 | uint32_t queue, uint32_t vmid) |
| 49 | { | 44 | { |
| 50 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 51 | uint32_t value = PIPEID(pipe) | MEID(mec) | VMID(vmid) | QUEUEID(queue); | 45 | uint32_t value = PIPEID(pipe) | MEID(mec) | VMID(vmid) | QUEUEID(queue); |
| 52 | 46 | ||
| 53 | mutex_lock(&adev->srbm_mutex); | 47 | mutex_lock(&adev->srbm_mutex); |
| 54 | WREG32(mmSRBM_GFX_CNTL, value); | 48 | WREG32(mmSRBM_GFX_CNTL, value); |
| 55 | } | 49 | } |
| 56 | 50 | ||
| 57 | static void unlock_srbm(struct kgd_dev *kgd) | 51 | static void unlock_srbm(struct amdgpu_device *adev) |
| 58 | { | 52 | { |
| 59 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 60 | |||
| 61 | WREG32(mmSRBM_GFX_CNTL, 0); | 53 | WREG32(mmSRBM_GFX_CNTL, 0); |
| 62 | mutex_unlock(&adev->srbm_mutex); | 54 | mutex_unlock(&adev->srbm_mutex); |
| 63 | } | 55 | } |
| 64 | 56 | ||
| 65 | static void acquire_queue(struct kgd_dev *kgd, uint32_t pipe_id, | 57 | static void acquire_queue(struct amdgpu_device *adev, uint32_t pipe_id, |
| 66 | uint32_t queue_id) | 58 | uint32_t queue_id) |
| 67 | { | 59 | { |
| 68 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 69 | |||
| 70 | uint32_t mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; | 60 | uint32_t mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; |
| 71 | uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); | 61 | uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); |
| 72 | 62 | ||
| 73 | lock_srbm(kgd, mec, pipe, queue_id, 0); | 63 | lock_srbm(adev, mec, pipe, queue_id, 0); |
| 74 | } | 64 | } |
| 75 | 65 | ||
| 76 | static void release_queue(struct kgd_dev *kgd) | 66 | static void release_queue(struct amdgpu_device *adev) |
| 77 | { | 67 | { |
| 78 | unlock_srbm(kgd); | 68 | unlock_srbm(adev); |
| 79 | } | 69 | } |
| 80 | 70 | ||
| 81 | static void kgd_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid, | 71 | static void kgd_program_sh_mem_settings(struct amdgpu_device *adev, uint32_t vmid, |
| 82 | uint32_t sh_mem_config, | 72 | uint32_t sh_mem_config, |
| 83 | uint32_t sh_mem_ape1_base, | 73 | uint32_t sh_mem_ape1_base, |
| 84 | uint32_t sh_mem_ape1_limit, | 74 | uint32_t sh_mem_ape1_limit, |
| 85 | uint32_t sh_mem_bases) | 75 | uint32_t sh_mem_bases) |
| 86 | { | 76 | { |
| 87 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | 77 | lock_srbm(adev, 0, 0, 0, vmid); |
| 88 | |||
| 89 | lock_srbm(kgd, 0, 0, 0, vmid); | ||
| 90 | 78 | ||
| 91 | WREG32(mmSH_MEM_CONFIG, sh_mem_config); | 79 | WREG32(mmSH_MEM_CONFIG, sh_mem_config); |
| 92 | WREG32(mmSH_MEM_APE1_BASE, sh_mem_ape1_base); | 80 | WREG32(mmSH_MEM_APE1_BASE, sh_mem_ape1_base); |
| 93 | WREG32(mmSH_MEM_APE1_LIMIT, sh_mem_ape1_limit); | 81 | WREG32(mmSH_MEM_APE1_LIMIT, sh_mem_ape1_limit); |
| 94 | WREG32(mmSH_MEM_BASES, sh_mem_bases); | 82 | WREG32(mmSH_MEM_BASES, sh_mem_bases); |
| 95 | 83 | ||
| 96 | unlock_srbm(kgd); | 84 | unlock_srbm(adev); |
| 97 | } | 85 | } |
| 98 | 86 | ||
| 99 | static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid, | 87 | static int kgd_set_pasid_vmid_mapping(struct amdgpu_device *adev, u32 pasid, |
| 100 | unsigned int vmid) | 88 | unsigned int vmid) |
| 101 | { | 89 | { |
| 102 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 103 | |||
| 104 | /* | 90 | /* |
| 105 | * We have to assume that there is no outstanding mapping. | 91 | * We have to assume that there is no outstanding mapping. |
| 106 | * The ATC_VMID_PASID_MAPPING_UPDATE_STATUS bit could be 0 because | 92 | * The ATC_VMID_PASID_MAPPING_UPDATE_STATUS bit could be 0 because |
| @@ -123,21 +109,20 @@ static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid, · 16 unmodified lines | |||
| 123 | return 0; | 109 | return 0; |
| 124 | } | 110 | } |
| 125 | 111 | ||
| 126 | static int kgd_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id) | 112 | static int kgd_init_interrupts(struct amdgpu_device *adev, uint32_t pipe_id) |
| 127 | { | 113 | { |
| 128 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 129 | uint32_t mec; | 114 | uint32_t mec; |
| 130 | uint32_t pipe; | 115 | uint32_t pipe; |
| 131 | 116 | ||
| 132 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; | 117 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; |
| 133 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); | 118 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); |
| 134 | 119 | ||
| 135 | lock_srbm(kgd, mec, pipe, 0, 0); | 120 | lock_srbm(adev, mec, pipe, 0, 0); |
| 136 | 121 | ||
| 137 | WREG32(mmCPC_INT_CNTL, CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK | | 122 | WREG32(mmCPC_INT_CNTL, CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK | |
| 138 | CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK); | 123 | CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK); |
| 139 | 124 | ||
| 140 | unlock_srbm(kgd); | 125 | unlock_srbm(adev); |
| 141 | 126 | ||
| 142 | return 0; | 127 | return 0; |
| 143 | } | 128 | } |
| @@ -165,12 +150,11 @@ static inline struct vi_sdma_mqd *get_sdma_mqd(void *mqd) · 21 unmodified lines | |||
| 165 | return (struct vi_sdma_mqd *)mqd; | 150 | return (struct vi_sdma_mqd *)mqd; |
| 166 | } | 151 | } |
| 167 | 152 | ||
| 168 | static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, | 153 | static int kgd_hqd_load(struct amdgpu_device *adev, void *mqd, |
| 169 | uint32_t queue_id, uint32_t __user *wptr, | 154 | uint32_t pipe_id, uint32_t queue_id, |
| 170 | uint32_t wptr_shift, uint32_t wptr_mask, | 155 | uint32_t __user *wptr, uint32_t wptr_shift, |
| 171 | struct mm_struct *mm) | 156 | uint32_t wptr_mask, struct mm_struct *mm) |
| 172 | { | 157 | { |
| 173 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 174 | struct vi_mqd *m; | 158 | struct vi_mqd *m; |
| 175 | uint32_t *mqd_hqd; | 159 | uint32_t *mqd_hqd; |
| 176 | uint32_t reg, wptr_val, data; | 160 | uint32_t reg, wptr_val, data; |
| @@ -178,7 +162,7 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, · 1 unmodified line | |||
| 178 | 162 | ||
| 179 | m = get_mqd(mqd); | 163 | m = get_mqd(mqd); |
| 180 | 164 | ||
| 181 | acquire_queue(kgd, pipe_id, queue_id); | 165 | acquire_queue(adev, pipe_id, queue_id); |
| 182 | 166 | ||
| 183 | /* HIQ is set during driver init period with vmid set to 0*/ | 167 | /* HIQ is set during driver init period with vmid set to 0*/ |
| 184 | if (m->cp_hqd_vmid == 0) { | 168 | if (m->cp_hqd_vmid == 0) { |
| @@ -206,7 +190,7 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, · 21 unmodified lines | |||
| 206 | * on ASICs that do not support context-save. | 190 | * on ASICs that do not support context-save. |
| 207 | * EOP writes/reads can start anywhere in the ring. | 191 | * EOP writes/reads can start anywhere in the ring. |
| 208 | */ | 192 | */ |
| 209 | if (get_amdgpu_device(kgd)->asic_type != CHIP_TONGA) { | 193 | if (adev->asic_type != CHIP_TONGA) { |
| 210 | WREG32(mmCP_HQD_EOP_RPTR, m->cp_hqd_eop_rptr); | 194 | WREG32(mmCP_HQD_EOP_RPTR, m->cp_hqd_eop_rptr); |
| 211 | WREG32(mmCP_HQD_EOP_WPTR, m->cp_hqd_eop_wptr); | 195 | WREG32(mmCP_HQD_EOP_WPTR, m->cp_hqd_eop_wptr); |
| 212 | WREG32(mmCP_HQD_EOP_WPTR_MEM, m->cp_hqd_eop_wptr_mem); | 196 | WREG32(mmCP_HQD_EOP_WPTR_MEM, m->cp_hqd_eop_wptr_mem); |
| @@ -226,25 +210,24 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, · 13 unmodified lines | |||
| 226 | * release srbm_mutex to avoid circular dependency between | 210 | * release srbm_mutex to avoid circular dependency between |
| 227 | * srbm_mutex->mm_sem->reservation_ww_class_mutex->srbm_mutex. | 211 | * srbm_mutex->mm_sem->reservation_ww_class_mutex->srbm_mutex. |
| 228 | */ | 212 | */ |
| 229 | release_queue(kgd); | 213 | release_queue(adev); |
| 230 | valid_wptr = read_user_wptr(mm, wptr, wptr_val); | 214 | valid_wptr = read_user_wptr(mm, wptr, wptr_val); |
| 231 | acquire_queue(kgd, pipe_id, queue_id); | 215 | acquire_queue(adev, pipe_id, queue_id); |
| 232 | if (valid_wptr) | 216 | if (valid_wptr) |
| 233 | WREG32(mmCP_HQD_PQ_WPTR, (wptr_val << wptr_shift) & wptr_mask); | 217 | WREG32(mmCP_HQD_PQ_WPTR, (wptr_val << wptr_shift) & wptr_mask); |
| 234 | 218 | ||
| 235 | data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1); | 219 | data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1); |
| 236 | WREG32(mmCP_HQD_ACTIVE, data); | 220 | WREG32(mmCP_HQD_ACTIVE, data); |
| 237 | 221 | ||
| 238 | release_queue(kgd); | 222 | release_queue(adev); |
| 239 | 223 | ||
| 240 | return 0; | 224 | return 0; |
| 241 | } | 225 | } |
| 242 | 226 | ||
| 243 | static int kgd_hqd_dump(struct kgd_dev *kgd, | 227 | static int kgd_hqd_dump(struct amdgpu_device *adev, |
| 244 | uint32_t pipe_id, uint32_t queue_id, | 228 | uint32_t pipe_id, uint32_t queue_id, |
| 245 | uint32_t (**dump)[2], uint32_t *n_regs) | 229 | uint32_t (**dump)[2], uint32_t *n_regs) |
| 246 | { | 230 | { |
| 247 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 248 | uint32_t i = 0, reg; | 231 | uint32_t i = 0, reg; |
| 249 | #define HQD_N_REGS (54+4) | 232 | #define HQD_N_REGS (54+4) |
| 250 | #define DUMP_REG(addr) do { \ | 233 | #define DUMP_REG(addr) do { \ |
| @@ -258,7 +241,7 @@ static int kgd_hqd_dump(struct kgd_dev *kgd, · 7 unmodified lines | |||
| 258 | if (*dump == NULL) | 241 | if (*dump == NULL) |
| 259 | return -ENOMEM; | 242 | return -ENOMEM; |
| 260 | 243 | ||
| 261 | acquire_queue(kgd, pipe_id, queue_id); | 244 | acquire_queue(adev, pipe_id, queue_id); |
| 262 | 245 | ||
| 263 | DUMP_REG(mmCOMPUTE_STATIC_THREAD_MGMT_SE0); | 246 | DUMP_REG(mmCOMPUTE_STATIC_THREAD_MGMT_SE0); |
| 264 | DUMP_REG(mmCOMPUTE_STATIC_THREAD_MGMT_SE1); | 247 | DUMP_REG(mmCOMPUTE_STATIC_THREAD_MGMT_SE1); |
| @@ -268,7 +251,7 @@ static int kgd_hqd_dump(struct kgd_dev *kgd, · 3 unmodified lines | |||
| 268 | for (reg = mmCP_MQD_BASE_ADDR; reg <= mmCP_HQD_EOP_DONES; reg++) | 251 | for (reg = mmCP_MQD_BASE_ADDR; reg <= mmCP_HQD_EOP_DONES; reg++) |
| 269 | DUMP_REG(reg); | 252 | DUMP_REG(reg); |
| 270 | 253 | ||
| 271 | release_queue(kgd); | 254 | release_queue(adev); |
| 272 | 255 | ||
| 273 | WARN_ON_ONCE(i != HQD_N_REGS); | 256 | WARN_ON_ONCE(i != HQD_N_REGS); |
| 274 | *n_regs = i; | 257 | *n_regs = i; |
| @@ -276,10 +259,9 @@ static int kgd_hqd_dump(struct kgd_dev *kgd, · 1 unmodified line | |||
| 276 | return 0; | 259 | return 0; |
| 277 | } | 260 | } |
| 278 | 261 | ||
| 279 | static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, | 262 | static int kgd_hqd_sdma_load(struct amdgpu_device *adev, void *mqd, |
| 280 | uint32_t __user *wptr, struct mm_struct *mm) | 263 | uint32_t __user *wptr, struct mm_struct *mm) |
| 281 | { | 264 | { |
| 282 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 283 | struct vi_sdma_mqd *m; | 265 | struct vi_sdma_mqd *m; |
| 284 | unsigned long end_jiffies; | 266 | unsigned long end_jiffies; |
| 285 | uint32_t sdma_rlc_reg_offset; | 267 | uint32_t sdma_rlc_reg_offset; |
| @@ -331,11 +313,10 @@ static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, · 45 unmodified lines | |||
| 331 | return 0; | 313 | return 0; |
| 332 | } | 314 | } |
| 333 | 315 | ||
| 334 | static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, | 316 | static int kgd_hqd_sdma_dump(struct amdgpu_device *adev, |
| 335 | uint32_t engine_id, uint32_t queue_id, | 317 | uint32_t engine_id, uint32_t queue_id, |
| 336 | uint32_t (**dump)[2], uint32_t *n_regs) | 318 | uint32_t (**dump)[2], uint32_t *n_regs) |
| 337 | { | 319 | { |
| 338 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 339 | uint32_t sdma_offset = engine_id * SDMA1_REGISTER_OFFSET + | 320 | uint32_t sdma_offset = engine_id * SDMA1_REGISTER_OFFSET + |
| 340 | queue_id * KFD_VI_SDMA_QUEUE_OFFSET; | 321 | queue_id * KFD_VI_SDMA_QUEUE_OFFSET; |
| 341 | uint32_t i = 0, reg; | 322 | uint32_t i = 0, reg; |
| @@ -367,15 +348,15 @@ static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, · 25 unmodified lines | |||
| 367 | return 0; | 348 | return 0; |
| 368 | } | 349 | } |
| 369 | 350 | ||
| 370 | static bool kgd_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address, | 351 | static bool kgd_hqd_is_occupied(struct amdgpu_device *adev, |
| 371 | uint32_t pipe_id, uint32_t queue_id) | 352 | uint64_t queue_address, uint32_t pipe_id, |
| 353 | uint32_t queue_id) | ||
| 372 | { | 354 | { |
| 373 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 374 | uint32_t act; | 355 | uint32_t act; |
| 375 | bool retval = false; | 356 | bool retval = false; |
| 376 | uint32_t low, high; | 357 | uint32_t low, high; |
| 377 | 358 | ||
| 378 | acquire_queue(kgd, pipe_id, queue_id); | 359 | acquire_queue(adev, pipe_id, queue_id); |
| 379 | act = RREG32(mmCP_HQD_ACTIVE); | 360 | act = RREG32(mmCP_HQD_ACTIVE); |
| 380 | if (act) { | 361 | if (act) { |
| 381 | low = lower_32_bits(queue_address >> 8); | 362 | low = lower_32_bits(queue_address >> 8); |
| @@ -385,13 +366,12 @@ static bool kgd_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address, · 3 unmodified lines | |||
| 385 | high == RREG32(mmCP_HQD_PQ_BASE_HI)) | 366 | high == RREG32(mmCP_HQD_PQ_BASE_HI)) |
| 386 | retval = true; | 367 | retval = true; |
| 387 | } | 368 | } |
| 388 | release_queue(kgd); | 369 | release_queue(adev); |
| 389 | return retval; | 370 | return retval; |
| 390 | } | 371 | } |
| 391 | 372 | ||
| 392 | static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd) | 373 | static bool kgd_hqd_sdma_is_occupied(struct amdgpu_device *adev, void *mqd) |
| 393 | { | 374 | { |
| 394 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 395 | struct vi_sdma_mqd *m; | 375 | struct vi_sdma_mqd *m; |
| 396 | uint32_t sdma_rlc_reg_offset; | 376 | uint32_t sdma_rlc_reg_offset; |
| 397 | uint32_t sdma_rlc_rb_cntl; | 377 | uint32_t sdma_rlc_rb_cntl; |
| @@ -407,12 +387,11 @@ static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd) · 9 unmodified lines | |||
| 407 | return false; | 387 | return false; |
| 408 | } | 388 | } |
| 409 | 389 | ||
| 410 | static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd, | 390 | static int kgd_hqd_destroy(struct amdgpu_device *adev, void *mqd, |
| 411 | enum kfd_preempt_type reset_type, | 391 | enum kfd_preempt_type reset_type, |
| 412 | unsigned int utimeout, uint32_t pipe_id, | 392 | unsigned int utimeout, uint32_t pipe_id, |
| 413 | uint32_t queue_id) | 393 | uint32_t queue_id) |
| 414 | { | 394 | { |
| 415 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 416 | uint32_t temp; | 395 | uint32_t temp; |
| 417 | enum hqd_dequeue_request_type type; | 396 | enum hqd_dequeue_request_type type; |
| 418 | unsigned long flags, end_jiffies; | 397 | unsigned long flags, end_jiffies; |
| @@ -422,7 +401,7 @@ static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd, · 3 unmodified lines | |||
| 422 | if (amdgpu_in_reset(adev)) | 401 | if (amdgpu_in_reset(adev)) |
| 423 | return -EIO; | 402 | return -EIO; |
| 424 | 403 | ||
| 425 | acquire_queue(kgd, pipe_id, queue_id); | 404 | acquire_queue(adev, pipe_id, queue_id); |
| 426 | 405 | ||
| 427 | if (m->cp_hqd_vmid == 0) | 406 | if (m->cp_hqd_vmid == 0) |
| 428 | WREG32_FIELD(RLC_CP_SCHEDULERS, scheduler1, 0); | 407 | WREG32_FIELD(RLC_CP_SCHEDULERS, scheduler1, 0); |
| @@ -502,20 +481,19 @@ loop: · 73 unmodified lines | |||
| 502 | break; | 481 | break; |
| 503 | if (time_after(jiffies, end_jiffies)) { | 482 | if (time_after(jiffies, end_jiffies)) { |
| 504 | pr_err("cp queue preemption time out.\n"); | 483 | pr_err("cp queue preemption time out.\n"); |
| 505 | release_queue(kgd); | 484 | release_queue(adev); |
| 506 | return -ETIME; | 485 | return -ETIME; |
| 507 | } | 486 | } |
| 508 | usleep_range(500, 1000); | 487 | usleep_range(500, 1000); |
| 509 | } | 488 | } |
| 510 | 489 | ||
| 511 | release_queue(kgd); | 490 | release_queue(adev); |
| 512 | return 0; | 491 | return 0; |
| 513 | } | 492 | } |
| 514 | 493 | ||
| 515 | static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, | 494 | static int kgd_hqd_sdma_destroy(struct amdgpu_device *adev, void *mqd, |
| 516 | unsigned int utimeout) | 495 | unsigned int utimeout) |
| 517 | { | 496 | { |
| 518 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 519 | struct vi_sdma_mqd *m; | 497 | struct vi_sdma_mqd *m; |
| 520 | uint32_t sdma_rlc_reg_offset; | 498 | uint32_t sdma_rlc_reg_offset; |
| 521 | uint32_t temp; | 499 | uint32_t temp; |
| @@ -549,11 +527,10 @@ static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, · 27 unmodified lines | |||
| 549 | return 0; | 527 | return 0; |
| 550 | } | 528 | } |
| 551 | 529 | ||
| 552 | static bool get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd, | 530 | static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, |
| 553 | uint8_t vmid, uint16_t *p_pasid) | 531 | uint8_t vmid, uint16_t *p_pasid) |
| 554 | { | 532 | { |
| 555 | uint32_t value; | 533 | uint32_t value; |
| 556 | struct amdgpu_device *adev = (struct amdgpu_device *) kgd; | ||
| 557 | 534 | ||
| 558 | value = RREG32(mmATC_VMID0_PASID_MAPPING + vmid); | 535 | value = RREG32(mmATC_VMID0_PASID_MAPPING + vmid); |
| 559 | *p_pasid = value & ATC_VMID0_PASID_MAPPING__PASID_MASK; | 536 | *p_pasid = value & ATC_VMID0_PASID_MAPPING__PASID_MASK; |
| @@ -561,12 +538,12 @@ static bool get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd, · 1 unmodified line | |||
| 561 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); | 538 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); |
| 562 | } | 539 | } |
| 563 | 540 | ||
| 564 | static int kgd_address_watch_disable(struct kgd_dev *kgd) | 541 | static int kgd_address_watch_disable(struct amdgpu_device *adev) |
| 565 | { | 542 | { |
| 566 | return 0; | 543 | return 0; |
| 567 | } | 544 | } |
| 568 | 545 | ||
| 569 | static int kgd_address_watch_execute(struct kgd_dev *kgd, | 546 | static int kgd_address_watch_execute(struct amdgpu_device *adev, |
| 570 | unsigned int watch_point_id, | 547 | unsigned int watch_point_id, |
| 571 | uint32_t cntl_val, | 548 | uint32_t cntl_val, |
| 572 | uint32_t addr_hi, | 549 | uint32_t addr_hi, |
| @@ -575,11 +552,10 @@ static int kgd_address_watch_execute(struct kgd_dev *kgd, · 2 unmodified lines | |||
| 575 | return 0; | 552 | return 0; |
| 576 | } | 553 | } |
| 577 | 554 | ||
| 578 | static int kgd_wave_control_execute(struct kgd_dev *kgd, | 555 | static int kgd_wave_control_execute(struct amdgpu_device *adev, |
| 579 | uint32_t gfx_index_val, | 556 | uint32_t gfx_index_val, |
| 580 | uint32_t sq_cmd) | 557 | uint32_t sq_cmd) |
| 581 | { | 558 | { |
| 582 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 583 | uint32_t data = 0; | 559 | uint32_t data = 0; |
| 584 | 560 | ||
| 585 | mutex_lock(&adev->grbm_idx_mutex); | 561 | mutex_lock(&adev->grbm_idx_mutex); |
| @@ -600,28 +576,24 @@ static int kgd_wave_control_execute(struct kgd_dev *kgd, · 14 unmodified lines | |||
| 600 | return 0; | 576 | return 0; |
| 601 | } | 577 | } |
| 602 | 578 | ||
| 603 | static uint32_t kgd_address_watch_get_offset(struct kgd_dev *kgd, | 579 | static uint32_t kgd_address_watch_get_offset(struct amdgpu_device *adev, |
| 604 | unsigned int watch_point_id, | 580 | unsigned int watch_point_id, |
| 605 | unsigned int reg_offset) | 581 | unsigned int reg_offset) |
| 606 | { | 582 | { |
| 607 | return 0; | 583 | return 0; |
| 608 | } | 584 | } |
| 609 | 585 | ||
| 610 | static void set_scratch_backing_va(struct kgd_dev *kgd, | 586 | static void set_scratch_backing_va(struct amdgpu_device *adev, |
| 611 | uint64_t va, uint32_t vmid) | 587 | uint64_t va, uint32_t vmid) |
| 612 | { | 588 | { |
| 613 | struct amdgpu_device *adev = (struct amdgpu_device *) kgd; | 589 | lock_srbm(adev, 0, 0, 0, vmid); |
| 614 | |||
| 615 | lock_srbm(kgd, 0, 0, 0, vmid); | ||
| 616 | WREG32(mmSH_HIDDEN_PRIVATE_BASE_VMID, va); | 590 | WREG32(mmSH_HIDDEN_PRIVATE_BASE_VMID, va); |
| 617 | unlock_srbm(kgd); | 591 | unlock_srbm(adev); |
| 618 | } | 592 | } |
| 619 | 593 | ||
| 620 | static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, | 594 | static void set_vm_context_page_table_base(struct amdgpu_device *adev, |
| 621 | uint64_t page_table_base) | 595 | uint32_t vmid, uint64_t page_table_base) |
| 622 | { | 596 | { |
| 623 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 624 | |||
| 625 | if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) { | 597 | if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) { |
| 626 | pr_err("trying to set page table base for wrong VMID\n"); | 598 | pr_err("trying to set page table base for wrong VMID\n"); |
| 627 | return; | 599 | return; |
| @@ -46,37 +46,26 @@ enum hqd_dequeue_request_type { · 45 unmodified lines | |||
| 46 | SAVE_WAVES | 46 | SAVE_WAVES |
| 47 | }; | 47 | }; |
| 48 | 48 | ||
| 49 | static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd) | 49 | static void lock_srbm(struct amdgpu_device *adev, uint32_t mec, uint32_t pipe, |
| 50 | { | ||
| 51 | return (struct amdgpu_device *)kgd; | ||
| 52 | } | ||
| 53 | |||
| 54 | static void lock_srbm(struct kgd_dev *kgd, uint32_t mec, uint32_t pipe, | ||
| 55 | uint32_t queue, uint32_t vmid) | 50 | uint32_t queue, uint32_t vmid) |
| 56 | { | 51 | { |
| 57 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 58 | |||
| 59 | mutex_lock(&adev->srbm_mutex); | 52 | mutex_lock(&adev->srbm_mutex); |
| 60 | soc15_grbm_select(adev, mec, pipe, queue, vmid); | 53 | soc15_grbm_select(adev, mec, pipe, queue, vmid); |
| 61 | } | 54 | } |
| 62 | 55 | ||
| 63 | static void unlock_srbm(struct kgd_dev *kgd) | 56 | static void unlock_srbm(struct amdgpu_device *adev) |
| 64 | { | 57 | { |
| 65 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 66 | |||
| 67 | soc15_grbm_select(adev, 0, 0, 0, 0); | 58 | soc15_grbm_select(adev, 0, 0, 0, 0); |
| 68 | mutex_unlock(&adev->srbm_mutex); | 59 | mutex_unlock(&adev->srbm_mutex); |
| 69 | } | 60 | } |
| 70 | 61 | ||
| 71 | static void acquire_queue(struct kgd_dev *kgd, uint32_t pipe_id, | 62 | static void acquire_queue(struct amdgpu_device *adev, uint32_t pipe_id, |
| 72 | uint32_t queue_id) | 63 | uint32_t queue_id) |
| 73 | { | 64 | { |
| 74 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 75 | |||
| 76 | uint32_t mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; | 65 | uint32_t mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; |
| 77 | uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); | 66 | uint32_t pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); |
| 78 | 67 | ||
| 79 | lock_srbm(kgd, mec, pipe, queue_id, 0); | 68 | lock_srbm(adev, mec, pipe, queue_id, 0); |
| 80 | } | 69 | } |
| 81 | 70 | ||
| 82 | static uint64_t get_queue_mask(struct amdgpu_device *adev, | 71 | static uint64_t get_queue_mask(struct amdgpu_device *adev, |
| @@ -88,33 +77,29 @@ static uint64_t get_queue_mask(struct amdgpu_device *adev, · 5 unmodified lines | |||
| 88 | return 1ull << bit; | 77 | return 1ull << bit; |
| 89 | } | 78 | } |
| 90 | 79 | ||
| 91 | static void release_queue(struct kgd_dev *kgd) | 80 | static void release_queue(struct amdgpu_device *adev) |
| 92 | { | 81 | { |
| 93 | unlock_srbm(kgd); | 82 | unlock_srbm(adev); |
| 94 | } | 83 | } |
| 95 | 84 | ||
| 96 | void kgd_gfx_v9_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid, | 85 | void kgd_gfx_v9_program_sh_mem_settings(struct amdgpu_device *adev, uint32_t vmid, |
| 97 | uint32_t sh_mem_config, | 86 | uint32_t sh_mem_config, |
| 98 | uint32_t sh_mem_ape1_base, | 87 | uint32_t sh_mem_ape1_base, |
| 99 | uint32_t sh_mem_ape1_limit, | 88 | uint32_t sh_mem_ape1_limit, |
| 100 | uint32_t sh_mem_bases) | 89 | uint32_t sh_mem_bases) |
| 101 | { | 90 | { |
| 102 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | 91 | lock_srbm(adev, 0, 0, 0, vmid); |
| 103 | |||
| 104 | lock_srbm(kgd, 0, 0, 0, vmid); | ||
| 105 | 92 | ||
| 106 | WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmSH_MEM_CONFIG), sh_mem_config); | 93 | WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmSH_MEM_CONFIG), sh_mem_config); |
| 107 | WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmSH_MEM_BASES), sh_mem_bases); | 94 | WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmSH_MEM_BASES), sh_mem_bases); |
| 108 | /* APE1 no longer exists on GFX9 */ | 95 | /* APE1 no longer exists on GFX9 */ |
| 109 | 96 | ||
| 110 | unlock_srbm(kgd); | 97 | unlock_srbm(adev); |
| 111 | } | 98 | } |
| 112 | 99 | ||
| 113 | int kgd_gfx_v9_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid, | 100 | int kgd_gfx_v9_set_pasid_vmid_mapping(struct amdgpu_device *adev, u32 pasid, |
| 114 | unsigned int vmid) | 101 | unsigned int vmid) |
| 115 | { | 102 | { |
| 116 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 117 | |||
| 118 | /* | 103 | /* |
| 119 | * We have to assume that there is no outstanding mapping. | 104 | * We have to assume that there is no outstanding mapping. |
| 120 | * The ATC_VMID_PASID_MAPPING_UPDATE_STATUS bit could be 0 because | 105 | * The ATC_VMID_PASID_MAPPING_UPDATE_STATUS bit could be 0 because |
| @@ -171,22 +156,21 @@ int kgd_gfx_v9_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid, · 50 unmodified lines | |||
| 171 | * but still works | 156 | * but still works |
| 172 | */ | 157 | */ |
| 173 | 158 | ||
| 174 | int kgd_gfx_v9_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id) | 159 | int kgd_gfx_v9_init_interrupts(struct amdgpu_device *adev, uint32_t pipe_id) |
| 175 | { | 160 | { |
| 176 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 177 | uint32_t mec; | 161 | uint32_t mec; |
| 178 | uint32_t pipe; | 162 | uint32_t pipe; |
| 179 | 163 | ||
| 180 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; | 164 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; |
| 181 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); | 165 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); |
| 182 | 166 | ||
| 183 | lock_srbm(kgd, mec, pipe, 0, 0); | 167 | lock_srbm(adev, mec, pipe, 0, 0); |
| 184 | 168 | ||
| 185 | WREG32(SOC15_REG_OFFSET(GC, 0, mmCPC_INT_CNTL), | 169 | WREG32_SOC15(GC, 0, mmCPC_INT_CNTL, |
| 186 | CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK | | 170 | CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK | |
| 187 | CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK); | 171 | CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK); |
| 188 | 172 | ||
| 189 | unlock_srbm(kgd); | 173 | unlock_srbm(adev); |
| 190 | 174 | ||
| 191 | return 0; | 175 | return 0; |
| 192 | } | 176 | } |
| @@ -233,19 +217,18 @@ static inline struct v9_sdma_mqd *get_sdma_mqd(void *mqd) · 40 unmodified lines | |||
| 233 | return (struct v9_sdma_mqd *)mqd; | 217 | return (struct v9_sdma_mqd *)mqd; |
| 234 | } | 218 | } |
| 235 | 219 | ||
| 236 | int kgd_gfx_v9_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, | 220 | int kgd_gfx_v9_hqd_load(struct amdgpu_device *adev, void *mqd, |
| 237 | uint32_t queue_id, uint32_t __user *wptr, | 221 | uint32_t pipe_id, uint32_t queue_id, |
| 238 | uint32_t wptr_shift, uint32_t wptr_mask, | 222 | uint32_t __user *wptr, uint32_t wptr_shift, |
| 239 | struct mm_struct *mm) | 223 | uint32_t wptr_mask, struct mm_struct *mm) |
| 240 | { | 224 | { |
| 241 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 242 | struct v9_mqd *m; | 225 | struct v9_mqd *m; |
| 243 | uint32_t *mqd_hqd; | 226 | uint32_t *mqd_hqd; |
| 244 | uint32_t reg, hqd_base, data; | 227 | uint32_t reg, hqd_base, data; |
| 245 | 228 | ||
| 246 | m = get_mqd(mqd); | 229 | m = get_mqd(mqd); |
| 247 | 230 | ||
| 248 | acquire_queue(kgd, pipe_id, queue_id); | 231 | acquire_queue(adev, pipe_id, queue_id); |
| 249 | 232 | ||
| 250 | /* HQD registers extend from CP_MQD_BASE_ADDR to CP_HQD_EOP_WPTR_MEM. */ | 233 | /* HQD registers extend from CP_MQD_BASE_ADDR to CP_HQD_EOP_WPTR_MEM. */ |
| 251 | mqd_hqd = &m->cp_mqd_base_addr_lo; | 234 | mqd_hqd = &m->cp_mqd_base_addr_lo; |
| @@ -296,7 +279,7 @@ int kgd_gfx_v9_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, · 44 unmodified lines | |||
| 296 | lower_32_bits((uintptr_t)wptr)); | 279 | lower_32_bits((uintptr_t)wptr)); |
| 297 | WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_POLL_ADDR_HI), | 280 | WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_POLL_ADDR_HI), |
| 298 | upper_32_bits((uintptr_t)wptr)); | 281 | upper_32_bits((uintptr_t)wptr)); |
| 299 | WREG32(SOC15_REG_OFFSET(GC, 0, mmCP_PQ_WPTR_POLL_CNTL1), | 282 | WREG32_SOC15(GC, 0, mmCP_PQ_WPTR_POLL_CNTL1, |
| 300 | (uint32_t)get_queue_mask(adev, pipe_id, queue_id)); | 283 | (uint32_t)get_queue_mask(adev, pipe_id, queue_id)); |
| 301 | } | 284 | } |
| 302 | 285 | ||
| @@ -308,16 +291,15 @@ int kgd_gfx_v9_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, · 5 unmodified lines | |||
| 308 | data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1); | 291 | data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1); |
| 309 | WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_ACTIVE), data); | 292 | WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_ACTIVE), data); |
| 310 | 293 | ||
| 311 | release_queue(kgd); | 294 | release_queue(adev); |
| 312 | 295 | ||
| 313 | return 0; | 296 | return 0; |
| 314 | } | 297 | } |
| 315 | 298 | ||
| 316 | int kgd_gfx_v9_hiq_mqd_load(struct kgd_dev *kgd, void *mqd, | 299 | int kgd_gfx_v9_hiq_mqd_load(struct amdgpu_device *adev, void *mqd, |
| 317 | uint32_t pipe_id, uint32_t queue_id, | 300 | uint32_t pipe_id, uint32_t queue_id, |
| 318 | uint32_t doorbell_off) | 301 | uint32_t doorbell_off) |
| 319 | { | 302 | { |
| 320 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 321 | struct amdgpu_ring *kiq_ring = &adev->gfx.kiq.ring; | 303 | struct amdgpu_ring *kiq_ring = &adev->gfx.kiq.ring; |
| 322 | struct v9_mqd *m; | 304 | struct v9_mqd *m; |
| 323 | uint32_t mec, pipe; | 305 | uint32_t mec, pipe; |
| @@ -325,7 +307,7 @@ int kgd_gfx_v9_hiq_mqd_load(struct kgd_dev *kgd, void *mqd, · 1 unmodified line | |||
| 325 | 307 | ||
| 326 | m = get_mqd(mqd); | 308 | m = get_mqd(mqd); |
| 327 | 309 | ||
| 328 | acquire_queue(kgd, pipe_id, queue_id); | 310 | acquire_queue(adev, pipe_id, queue_id); |
| 329 | 311 | ||
| 330 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; | 312 | mec = (pipe_id / adev->gfx.mec.num_pipe_per_mec) + 1; |
| 331 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); | 313 | pipe = (pipe_id % adev->gfx.mec.num_pipe_per_mec); |
| @@ -361,16 +343,15 @@ int kgd_gfx_v9_hiq_mqd_load(struct kgd_dev *kgd, void *mqd, · 29 unmodified lines | |||
| 361 | 343 | ||
| 362 | out_unlock: | 344 | out_unlock: |
| 363 | spin_unlock(&adev->gfx.kiq.ring_lock); | 345 | spin_unlock(&adev->gfx.kiq.ring_lock); |
| 364 | release_queue(kgd); | 346 | release_queue(adev); |
| 365 | 347 | ||
| 366 | return r; | 348 | return r; |
| 367 | } | 349 | } |
| 368 | 350 | ||
| 369 | int kgd_gfx_v9_hqd_dump(struct kgd_dev *kgd, | 351 | int kgd_gfx_v9_hqd_dump(struct amdgpu_device *adev, |
| 370 | uint32_t pipe_id, uint32_t queue_id, | 352 | uint32_t pipe_id, uint32_t queue_id, |
| 371 | uint32_t (**dump)[2], uint32_t *n_regs) | 353 | uint32_t (**dump)[2], uint32_t *n_regs) |
| 372 | { | 354 | { |
| 373 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 374 | uint32_t i = 0, reg; | 355 | uint32_t i = 0, reg; |
| 375 | #define HQD_N_REGS 56 | 356 | #define HQD_N_REGS 56 |
| 376 | #define DUMP_REG(addr) do { \ | 357 | #define DUMP_REG(addr) do { \ |
| @@ -384,13 +365,13 @@ int kgd_gfx_v9_hqd_dump(struct kgd_dev *kgd, · 7 unmodified lines | |||
| 384 | if (*dump == NULL) | 365 | if (*dump == NULL) |
| 385 | return -ENOMEM; | 366 | return -ENOMEM; |
| 386 | 367 | ||
| 387 | acquire_queue(kgd, pipe_id, queue_id); | 368 | acquire_queue(adev, pipe_id, queue_id); |
| 388 | 369 | ||
| 389 | for (reg = SOC15_REG_OFFSET(GC, 0, mmCP_MQD_BASE_ADDR); | 370 | for (reg = SOC15_REG_OFFSET(GC, 0, mmCP_MQD_BASE_ADDR); |
| 390 | reg <= SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_HI); reg++) | 371 | reg <= SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_HI); reg++) |
| 391 | DUMP_REG(reg); | 372 | DUMP_REG(reg); |
| 392 | 373 | ||
| 393 | release_queue(kgd); | 374 | release_queue(adev); |
| 394 | 375 | ||
| 395 | WARN_ON_ONCE(i != HQD_N_REGS); | 376 | WARN_ON_ONCE(i != HQD_N_REGS); |
| 396 | *n_regs = i; | 377 | *n_regs = i; |
| @@ -398,10 +379,9 @@ int kgd_gfx_v9_hqd_dump(struct kgd_dev *kgd, · 1 unmodified line | |||
| 398 | return 0; | 379 | return 0; |
| 399 | } | 380 | } |
| 400 | 381 | ||
| 401 | static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, | 382 | static int kgd_hqd_sdma_load(struct amdgpu_device *adev, void *mqd, |
| 402 | uint32_t __user *wptr, struct mm_struct *mm) | 383 | uint32_t __user *wptr, struct mm_struct *mm) |
| 403 | { | 384 | { |
| 404 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 405 | struct v9_sdma_mqd *m; | 385 | struct v9_sdma_mqd *m; |
| 406 | uint32_t sdma_rlc_reg_offset; | 386 | uint32_t sdma_rlc_reg_offset; |
| 407 | unsigned long end_jiffies; | 387 | unsigned long end_jiffies; |
| @@ -468,11 +448,10 @@ static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, · 60 unmodified lines | |||
| 468 | return 0; | 448 | return 0; |
| 469 | } | 449 | } |
| 470 | 450 | ||
| 471 | static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, | 451 | static int kgd_hqd_sdma_dump(struct amdgpu_device *adev, |
| 472 | uint32_t engine_id, uint32_t queue_id, | 452 | uint32_t engine_id, uint32_t queue_id, |
| 473 | uint32_t (**dump)[2], uint32_t *n_regs) | 453 | uint32_t (**dump)[2], uint32_t *n_regs) |
| 474 | { | 454 | { |
| 475 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 476 | uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, | 455 | uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, |
| 477 | engine_id, queue_id); | 456 | engine_id, queue_id); |
| 478 | uint32_t i = 0, reg; | 457 | uint32_t i = 0, reg; |
| @@ -500,31 +479,30 @@ static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, · 21 unmodified lines | |||
| 500 | return 0; | 479 | return 0; |
| 501 | } | 480 | } |
| 502 | 481 | ||
| 503 | bool kgd_gfx_v9_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address, | 482 | bool kgd_gfx_v9_hqd_is_occupied(struct amdgpu_device *adev, |
| 504 | uint32_t pipe_id, uint32_t queue_id) | 483 | uint64_t queue_address, uint32_t pipe_id, |
| 484 | uint32_t queue_id) | ||
| 505 | { | 485 | { |
| 506 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 507 | uint32_t act; | 486 | uint32_t act; |
| 508 | bool retval = false; | 487 | bool retval = false; |
| 509 | uint32_t low, high; | 488 | uint32_t low, high; |
| 510 | 489 | ||
| 511 | acquire_queue(kgd, pipe_id, queue_id); | 490 | acquire_queue(adev, pipe_id, queue_id); |
| 512 | act = RREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_ACTIVE)); | 491 | act = RREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE); |
| 513 | if (act) { | 492 | if (act) { |
| 514 | low = lower_32_bits(queue_address >> 8); | 493 | low = lower_32_bits(queue_address >> 8); |
| 515 | high = upper_32_bits(queue_address >> 8); | 494 | high = upper_32_bits(queue_address >> 8); |
| 516 | 495 | ||
| 517 | if (low == RREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_BASE)) && | 496 | if (low == RREG32_SOC15(GC, 0, mmCP_HQD_PQ_BASE) && |
| 518 | high == RREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_BASE_HI))) | 497 | high == RREG32_SOC15(GC, 0, mmCP_HQD_PQ_BASE_HI)) |
| 519 | retval = true; | 498 | retval = true; |
| 520 | } | 499 | } |
| 521 | release_queue(kgd); | 500 | release_queue(adev); |
| 522 | return retval; | 501 | return retval; |
| 523 | } | 502 | } |
| 524 | 503 | ||
| 525 | static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd) | 504 | static bool kgd_hqd_sdma_is_occupied(struct amdgpu_device *adev, void *mqd) |
| 526 | { | 505 | { |
| 527 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 528 | struct v9_sdma_mqd *m; | 506 | struct v9_sdma_mqd *m; |
| 529 | uint32_t sdma_rlc_reg_offset; | 507 | uint32_t sdma_rlc_reg_offset; |
| 530 | uint32_t sdma_rlc_rb_cntl; | 508 | uint32_t sdma_rlc_rb_cntl; |
| @@ -541,12 +519,11 @@ static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd) · 10 unmodified lines | |||
| 541 | return false; | 519 | return false; |
| 542 | } | 520 | } |
| 543 | 521 | ||
| 544 | int kgd_gfx_v9_hqd_destroy(struct kgd_dev *kgd, void *mqd, | 522 | int kgd_gfx_v9_hqd_destroy(struct amdgpu_device *adev, void *mqd, |
| 545 | enum kfd_preempt_type reset_type, | 523 | enum kfd_preempt_type reset_type, |
| 546 | unsigned int utimeout, uint32_t pipe_id, | 524 | unsigned int utimeout, uint32_t pipe_id, |
| 547 | uint32_t queue_id) | 525 | uint32_t queue_id) |
| 548 | { | 526 | { |
| 549 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 550 | enum hqd_dequeue_request_type type; | 527 | enum hqd_dequeue_request_type type; |
| 551 | unsigned long end_jiffies; | 528 | unsigned long end_jiffies; |
| 552 | uint32_t temp; | 529 | uint32_t temp; |
| @@ -555,7 +532,7 @@ int kgd_gfx_v9_hqd_destroy(struct kgd_dev *kgd, void *mqd, · 2 unmodified lines | |||
| 555 | if (amdgpu_in_reset(adev)) | 532 | if (amdgpu_in_reset(adev)) |
| 556 | return -EIO; | 533 | return -EIO; |
| 557 | 534 | ||
| 558 | acquire_queue(kgd, pipe_id, queue_id); | 535 | acquire_queue(adev, pipe_id, queue_id); |
| 559 | 536 | ||
| 560 | if (m->cp_hqd_vmid == 0) | 537 | if (m->cp_hqd_vmid == 0) |
| 561 | WREG32_FIELD15_RLC(GC, 0, RLC_CP_SCHEDULERS, scheduler1, 0); | 538 | WREG32_FIELD15_RLC(GC, 0, RLC_CP_SCHEDULERS, scheduler1, 0); |
| @@ -579,25 +556,24 @@ int kgd_gfx_v9_hqd_destroy(struct kgd_dev *kgd, void *mqd, · 17 unmodified lines | |||
| 579 | 556 | ||
| 580 | end_jiffies = (utimeout * HZ / 1000) + jiffies; | 557 | end_jiffies = (utimeout * HZ / 1000) + jiffies; |
| 581 | while (true) { | 558 | while (true) { |
| 582 | temp = RREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_ACTIVE)); | 559 | temp = RREG32_SOC15(GC, 0, mmCP_HQD_ACTIVE); |
| 583 | if (!(temp & CP_HQD_ACTIVE__ACTIVE_MASK)) | 560 | if (!(temp & CP_HQD_ACTIVE__ACTIVE_MASK)) |
| 584 | break; | 561 | break; |
| 585 | if (time_after(jiffies, end_jiffies)) { | 562 | if (time_after(jiffies, end_jiffies)) { |
| 586 | pr_err("cp queue preemption time out.\n"); | 563 | pr_err("cp queue preemption time out.\n"); |
| 587 | release_queue(kgd); | 564 | release_queue(adev); |
| 588 | return -ETIME; | 565 | return -ETIME; |
| 589 | } | 566 | } |
| 590 | usleep_range(500, 1000); | 567 | usleep_range(500, 1000); |
| 591 | } | 568 | } |
| 592 | 569 | ||
| 593 | release_queue(kgd); | 570 | release_queue(adev); |
| 594 | return 0; | 571 | return 0; |
| 595 | } | 572 | } |
| 596 | 573 | ||
| 597 | static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, | 574 | static int kgd_hqd_sdma_destroy(struct amdgpu_device *adev, void *mqd, |
| 598 | unsigned int utimeout) | 575 | unsigned int utimeout) |
| 599 | { | 576 | { |
| 600 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 601 | struct v9_sdma_mqd *m; | 577 | struct v9_sdma_mqd *m; |
| 602 | uint32_t sdma_rlc_reg_offset; | 578 | uint32_t sdma_rlc_reg_offset; |
| 603 | uint32_t temp; | 579 | uint32_t temp; |
| @@ -634,11 +610,10 @@ static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, · 30 unmodified lines | |||
| 634 | return 0; | 610 | return 0; |
| 635 | } | 611 | } |
| 636 | 612 | ||
| 637 | bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd, | 613 | bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, |
| 638 | uint8_t vmid, uint16_t *p_pasid) | 614 | uint8_t vmid, uint16_t *p_pasid) |
| 639 | { | 615 | { |
| 640 | uint32_t value; | 616 | uint32_t value; |
| 641 | struct amdgpu_device *adev = (struct amdgpu_device *) kgd; | ||
| 642 | 617 | ||
| 643 | value = RREG32(SOC15_REG_OFFSET(ATHUB, 0, mmATC_VMID0_PASID_MAPPING) | 618 | value = RREG32(SOC15_REG_OFFSET(ATHUB, 0, mmATC_VMID0_PASID_MAPPING) |
| 644 | + vmid); | 619 | + vmid); |
| @@ -647,12 +622,12 @@ bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd, · 2 unmodified lines | |||
| 647 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); | 622 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); |
| 648 | } | 623 | } |
| 649 | 624 | ||
| 650 | int kgd_gfx_v9_address_watch_disable(struct kgd_dev *kgd) | 625 | int kgd_gfx_v9_address_watch_disable(struct amdgpu_device *adev) |
| 651 | { | 626 | { |
| 652 | return 0; | 627 | return 0; |
| 653 | } | 628 | } |
| 654 | 629 | ||
| 655 | int kgd_gfx_v9_address_watch_execute(struct kgd_dev *kgd, | 630 | int kgd_gfx_v9_address_watch_execute(struct amdgpu_device *adev, |
| 656 | unsigned int watch_point_id, | 631 | unsigned int watch_point_id, |
| 657 | uint32_t cntl_val, | 632 | uint32_t cntl_val, |
| 658 | uint32_t addr_hi, | 633 | uint32_t addr_hi, |
| @@ -661,17 +636,16 @@ int kgd_gfx_v9_address_watch_execute(struct kgd_dev *kgd, · 2 unmodified lines | |||
| 661 | return 0; | 636 | return 0; |
| 662 | } | 637 | } |
| 663 | 638 | ||
| 664 | int kgd_gfx_v9_wave_control_execute(struct kgd_dev *kgd, | 639 | int kgd_gfx_v9_wave_control_execute(struct amdgpu_device *adev, |
| 665 | uint32_t gfx_index_val, | 640 | uint32_t gfx_index_val, |
| 666 | uint32_t sq_cmd) | 641 | uint32_t sq_cmd) |
| 667 | { | 642 | { |
| 668 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 669 | uint32_t data = 0; | 643 | uint32_t data = 0; |
| 670 | 644 | ||
| 671 | mutex_lock(&adev->grbm_idx_mutex); | 645 | mutex_lock(&adev->grbm_idx_mutex); |
| 672 | 646 | ||
| 673 | WREG32_SOC15_RLC_SHADOW(GC, 0, mmGRBM_GFX_INDEX, gfx_index_val); | 647 | WREG32_SOC15_RLC_SHADOW(GC, 0, mmGRBM_GFX_INDEX, gfx_index_val); |
| 674 | WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_CMD), sq_cmd); | 648 | WREG32_SOC15(GC, 0, mmSQ_CMD, sq_cmd); |
| 675 | 649 | ||
| 676 | data = REG_SET_FIELD(data, GRBM_GFX_INDEX, | 650 | data = REG_SET_FIELD(data, GRBM_GFX_INDEX, |
| 677 | INSTANCE_BROADCAST_WRITES, 1); | 651 | INSTANCE_BROADCAST_WRITES, 1); |
| @@ -686,18 +660,16 @@ int kgd_gfx_v9_wave_control_execute(struct kgd_dev *kgd, · 8 unmodified lines | |||
| 686 | return 0; | 660 | return 0; |
| 687 | } | 661 | } |
| 688 | 662 | ||
| 689 | uint32_t kgd_gfx_v9_address_watch_get_offset(struct kgd_dev *kgd, | 663 | uint32_t kgd_gfx_v9_address_watch_get_offset(struct amdgpu_device *adev, |
| 690 | unsigned int watch_point_id, | 664 | unsigned int watch_point_id, |
| 691 | unsigned int reg_offset) | 665 | unsigned int reg_offset) |
| 692 | { | 666 | { |
| 693 | return 0; | 667 | return 0; |
| 694 | } | 668 | } |
| 695 | 669 | ||
| 696 | void kgd_gfx_v9_set_vm_context_page_table_base(struct kgd_dev *kgd, | 670 | void kgd_gfx_v9_set_vm_context_page_table_base(struct amdgpu_device *adev, |
| 697 | uint32_t vmid, uint64_t page_table_base) | 671 | uint32_t vmid, uint64_t page_table_base) |
| 698 | { | 672 | { |
| 699 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 700 | |||
| 701 | if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) { | 673 | if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) { |
| 702 | pr_err("trying to set page table base for wrong VMID %u\n", | 674 | pr_err("trying to set page table base for wrong VMID %u\n", |
| 703 | vmid); | 675 | vmid); |
| @@ -750,7 +722,7 @@ static void get_wave_count(struct amdgpu_device *adev, int queue_idx, · 46 unmodified lines | |||
| 750 | pipe_idx = queue_idx / adev->gfx.mec.num_queue_per_pipe; | 722 | pipe_idx = queue_idx / adev->gfx.mec.num_queue_per_pipe; |
| 751 | queue_slot = queue_idx % adev->gfx.mec.num_queue_per_pipe; | 723 | queue_slot = queue_idx % adev->gfx.mec.num_queue_per_pipe; |
| 752 | soc15_grbm_select(adev, 1, pipe_idx, queue_slot, 0); | 724 | soc15_grbm_select(adev, 1, pipe_idx, queue_slot, 0); |
| 753 | reg_val = RREG32(SOC15_REG_OFFSET(GC, 0, mmSPI_CSQ_WF_ACTIVE_COUNT_0) + | 725 | reg_val = RREG32_SOC15_IP(GC, SOC15_REG_OFFSET(GC, 0, mmSPI_CSQ_WF_ACTIVE_COUNT_0) + |
| 754 | queue_slot); | 726 | queue_slot); |
| 755 | *wave_cnt = reg_val & SPI_CSQ_WF_ACTIVE_COUNT_0__COUNT_MASK; | 727 | *wave_cnt = reg_val & SPI_CSQ_WF_ACTIVE_COUNT_0__COUNT_MASK; |
| 756 | if (*wave_cnt != 0) | 728 | if (*wave_cnt != 0) |
| @@ -804,7 +776,7 @@ static void get_wave_count(struct amdgpu_device *adev, int queue_idx, · 47 unmodified lines | |||
| 804 | * | 776 | * |
| 805 | * Reading registers referenced above involves programming GRBM appropriately | 777 | * Reading registers referenced above involves programming GRBM appropriately |
| 806 | */ | 778 | */ |
| 807 | void kgd_gfx_v9_get_cu_occupancy(struct kgd_dev *kgd, int pasid, | 779 | void kgd_gfx_v9_get_cu_occupancy(struct amdgpu_device *adev, int pasid, |
| 808 | int *pasid_wave_cnt, int *max_waves_per_cu) | 780 | int *pasid_wave_cnt, int *max_waves_per_cu) |
| 809 | { | 781 | { |
| 810 | int qidx; | 782 | int qidx; |
| @@ -818,10 +790,8 @@ void kgd_gfx_v9_get_cu_occupancy(struct kgd_dev *kgd, int pasid, · 7 unmodified lines | |||
| 818 | int pasid_tmp; | 790 | int pasid_tmp; |
| 819 | int max_queue_cnt; | 791 | int max_queue_cnt; |
| 820 | int vmid_wave_cnt = 0; | 792 | int vmid_wave_cnt = 0; |
| 821 | struct amdgpu_device *adev; | ||
| 822 | DECLARE_BITMAP(cp_queue_bitmap, KGD_MAX_QUEUES); | 793 | DECLARE_BITMAP(cp_queue_bitmap, KGD_MAX_QUEUES); |
| 823 | 794 | ||
| 824 | adev = get_amdgpu_device(kgd); | ||
| 825 | lock_spi_csq_mutexes(adev); | 795 | lock_spi_csq_mutexes(adev); |
| 826 | soc15_grbm_select(adev, 1, 0, 0, 0); | 796 | soc15_grbm_select(adev, 1, 0, 0, 0); |
| 827 | 797 | ||
| @@ -839,8 +809,7 @@ void kgd_gfx_v9_get_cu_occupancy(struct kgd_dev *kgd, int pasid, · 11 unmodified lines | |||
| 839 | for (sh_idx = 0; sh_idx < sh_cnt; sh_idx++) { | 809 | for (sh_idx = 0; sh_idx < sh_cnt; sh_idx++) { |
| 840 | 810 | ||
| 841 | gfx_v9_0_select_se_sh(adev, se_idx, sh_idx, 0xffffffff); | 811 | gfx_v9_0_select_se_sh(adev, se_idx, sh_idx, 0xffffffff); |
| 842 | queue_map = RREG32(SOC15_REG_OFFSET(GC, 0, | 812 | queue_map = RREG32_SOC15(GC, 0, mmSPI_CSQ_WF_ACTIVE_STATUS); |
| 843 | mmSPI_CSQ_WF_ACTIVE_STATUS)); | ||
| 844 | 813 | ||
| 845 | /* | 814 | /* |
| 846 | * Assumption: queue map encodes following schema: four | 815 | * Assumption: queue map encodes following schema: four |
| @@ -882,30 +851,28 @@ void kgd_gfx_v9_get_cu_occupancy(struct kgd_dev *kgd, int pasid, · 35 unmodified lines | |||
| 882 | adev->gfx.cu_info.max_waves_per_simd; | 851 | adev->gfx.cu_info.max_waves_per_simd; |
| 883 | } | 852 | } |
| 884 | 853 | ||
| 885 | void kgd_gfx_v9_program_trap_handler_settings(struct kgd_dev *kgd, | 854 | void kgd_gfx_v9_program_trap_handler_settings(struct amdgpu_device *adev, |
| 886 | uint32_t vmid, uint64_t tba_addr, uint64_t tma_addr) | 855 | uint32_t vmid, uint64_t tba_addr, uint64_t tma_addr) |
| 887 | { | 856 | { |
| 888 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | 857 | lock_srbm(adev, 0, 0, 0, vmid); |
| 889 | |||
| 890 | lock_srbm(kgd, 0, 0, 0, vmid); | ||
| 891 | 858 | ||
| 892 | /* | 859 | /* |
| 893 | * Program TBA registers | 860 | * Program TBA registers |
| 894 | */ | 861 | */ |
| 895 | WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_SHADER_TBA_LO), | 862 | WREG32_SOC15(GC, 0, mmSQ_SHADER_TBA_LO, |
| 896 | lower_32_bits(tba_addr >> 8)); | 863 | lower_32_bits(tba_addr >> 8)); |
| 897 | WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_SHADER_TBA_HI), | 864 | WREG32_SOC15(GC, 0, mmSQ_SHADER_TBA_HI, |
| 898 | upper_32_bits(tba_addr >> 8)); | 865 | upper_32_bits(tba_addr >> 8)); |
| 899 | 866 | ||
| 900 | /* | 867 | /* |
| 901 | * Program TMA registers | 868 | * Program TMA registers |
| 902 | */ | 869 | */ |
| 903 | WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_SHADER_TMA_LO), | 870 | WREG32_SOC15(GC, 0, mmSQ_SHADER_TMA_LO, |
| 904 | lower_32_bits(tma_addr >> 8)); | 871 | lower_32_bits(tma_addr >> 8)); |
| 905 | WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_SHADER_TMA_HI), | 872 | WREG32_SOC15(GC, 0, mmSQ_SHADER_TMA_HI, |
| 906 | upper_32_bits(tma_addr >> 8)); | 873 | upper_32_bits(tma_addr >> 8)); |
| 907 | 874 | ||
| 908 | unlock_srbm(kgd); | 875 | unlock_srbm(adev); |
| 909 | } | 876 | } |
| 910 | 877 | ||
| 911 | const struct kfd2kgd_calls gfx_v9_kfd2kgd = { | 878 | const struct kfd2kgd_calls gfx_v9_kfd2kgd = { |
| @@ -22,48 +22,49 @@ · 21 unmodified lines | |||
| 22 | 22 | ||
| 23 | 23 | ||
| 24 | 24 | ||
| 25 | void kgd_gfx_v9_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid, | 25 | void kgd_gfx_v9_program_sh_mem_settings(struct amdgpu_device *adev, uint32_t vmid, |
| 26 | uint32_t sh_mem_config, | 26 | uint32_t sh_mem_config, |
| 27 | uint32_t sh_mem_ape1_base, uint32_t sh_mem_ape1_limit, | 27 | uint32_t sh_mem_ape1_base, uint32_t sh_mem_ape1_limit, |
| 28 | uint32_t sh_mem_bases); | 28 | uint32_t sh_mem_bases); |
| 29 | int kgd_gfx_v9_set_pasid_vmid_mapping(struct kgd_dev *kgd, u32 pasid, | 29 | int kgd_gfx_v9_set_pasid_vmid_mapping(struct amdgpu_device *adev, u32 pasid, |
| 30 | unsigned int vmid); | 30 | unsigned int vmid); |
| 31 | int kgd_gfx_v9_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id); | 31 | int kgd_gfx_v9_init_interrupts(struct amdgpu_device *adev, uint32_t pipe_id); |
| 32 | int kgd_gfx_v9_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, | 32 | int kgd_gfx_v9_hqd_load(struct amdgpu_device *adev, void *mqd, uint32_t pipe_id, |
| 33 | uint32_t queue_id, uint32_t __user *wptr, | 33 | uint32_t queue_id, uint32_t __user *wptr, |
| 34 | uint32_t wptr_shift, uint32_t wptr_mask, | 34 | uint32_t wptr_shift, uint32_t wptr_mask, |
| 35 | struct mm_struct *mm); | 35 | struct mm_struct *mm); |
| 36 | int kgd_gfx_v9_hiq_mqd_load(struct kgd_dev *kgd, void *mqd, | 36 | int kgd_gfx_v9_hiq_mqd_load(struct amdgpu_device *adev, void *mqd, |
| 37 | uint32_t pipe_id, uint32_t queue_id, | 37 | uint32_t pipe_id, uint32_t queue_id, |
| 38 | uint32_t doorbell_off); | 38 | uint32_t doorbell_off); |
| 39 | int kgd_gfx_v9_hqd_dump(struct kgd_dev *kgd, | 39 | int kgd_gfx_v9_hqd_dump(struct amdgpu_device *adev, |
| 40 | uint32_t pipe_id, uint32_t queue_id, | 40 | uint32_t pipe_id, uint32_t queue_id, |
| 41 | uint32_t (**dump)[2], uint32_t *n_regs); | 41 | uint32_t (**dump)[2], uint32_t *n_regs); |
| 42 | bool kgd_gfx_v9_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address, | 42 | bool kgd_gfx_v9_hqd_is_occupied(struct amdgpu_device *adev, |
| 43 | uint32_t pipe_id, uint32_t queue_id); | 43 | uint64_t queue_address, uint32_t pipe_id, |
| 44 | int kgd_gfx_v9_hqd_destroy(struct kgd_dev *kgd, void *mqd, | 44 | uint32_t queue_id); |
| 45 | int kgd_gfx_v9_hqd_destroy(struct amdgpu_device *adev, void *mqd, | ||
| 45 | enum kfd_preempt_type reset_type, | 46 | enum kfd_preempt_type reset_type, |
| 46 | unsigned int utimeout, uint32_t pipe_id, | 47 | unsigned int utimeout, uint32_t pipe_id, |
| 47 | uint32_t queue_id); | 48 | uint32_t queue_id); |
| 48 | int kgd_gfx_v9_address_watch_disable(struct kgd_dev *kgd); | 49 | int kgd_gfx_v9_address_watch_disable(struct amdgpu_device *adev); |
| 49 | int kgd_gfx_v9_address_watch_execute(struct kgd_dev *kgd, | 50 | int kgd_gfx_v9_address_watch_execute(struct amdgpu_device *adev, |
| 50 | unsigned int watch_point_id, | 51 | unsigned int watch_point_id, |
| 51 | uint32_t cntl_val, | 52 | uint32_t cntl_val, |
| 52 | uint32_t addr_hi, | 53 | uint32_t addr_hi, |
| 53 | uint32_t addr_lo); | 54 | uint32_t addr_lo); |
| 54 | int kgd_gfx_v9_wave_control_execute(struct kgd_dev *kgd, | 55 | int kgd_gfx_v9_wave_control_execute(struct amdgpu_device *adev, |
| 55 | uint32_t gfx_index_val, | 56 | uint32_t gfx_index_val, |
| 56 | uint32_t sq_cmd); | 57 | uint32_t sq_cmd); |
| 57 | uint32_t kgd_gfx_v9_address_watch_get_offset(struct kgd_dev *kgd, | 58 | uint32_t kgd_gfx_v9_address_watch_get_offset(struct amdgpu_device *adev, |
| 58 | unsigned int watch_point_id, | 59 | unsigned int watch_point_id, |
| 59 | unsigned int reg_offset); | 60 | unsigned int reg_offset); |
| 60 | 61 | ||
| 61 | bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd, | 62 | bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, |
| 62 | uint8_t vmid, uint16_t *p_pasid); | 63 | uint8_t vmid, uint16_t *p_pasid); |
| 63 | 64 | ||
| 64 | void kgd_gfx_v9_set_vm_context_page_table_base(struct kgd_dev *kgd, | 65 | void kgd_gfx_v9_set_vm_context_page_table_base(struct amdgpu_device *adev, |
| 65 | uint32_t vmid, uint64_t page_table_base); | 66 | uint32_t vmid, uint64_t page_table_base); |
| 66 | void kgd_gfx_v9_get_cu_occupancy(struct kgd_dev *kgd, int pasid, | 67 | void kgd_gfx_v9_get_cu_occupancy(struct amdgpu_device *adev, int pasid, |
| 67 | int *pasid_wave_cnt, int *max_waves_per_cu); | 68 | int *pasid_wave_cnt, int *max_waves_per_cu); |
| 68 | void kgd_gfx_v9_program_trap_handler_settings(struct kgd_dev *kgd, | 69 | void kgd_gfx_v9_program_trap_handler_settings(struct amdgpu_device *adev, |
| 69 | uint32_t vmid, uint64_t tba_addr, uint64_t tma_addr); | 70 | uint32_t vmid, uint64_t tba_addr, uint64_t tma_addr); |
| @@ -60,12 +60,6 @@ static const char * const domain_bit_to_string[] = { · 59 unmodified lines | |||
| 60 | 60 | ||
| 61 | static void amdgpu_amdkfd_restore_userptr_worker(struct work_struct *work); | 61 | static void amdgpu_amdkfd_restore_userptr_worker(struct work_struct *work); |
| 62 | 62 | ||
| 63 | |||
| 64 | static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd) | ||
| 65 | { | ||
| 66 | return (struct amdgpu_device *)kgd; | ||
| 67 | } | ||
| 68 | |||
| 69 | static bool kfd_mem_is_attached(struct amdgpu_vm *avm, | 63 | static bool kfd_mem_is_attached(struct amdgpu_vm *avm, |
| 70 | struct kgd_mem *mem) | 64 | struct kgd_mem *mem) |
| 71 | { | 65 | { |
| @@ -126,8 +120,19 @@ static size_t amdgpu_amdkfd_acc_size(uint64_t size) · 54 unmodified lines | |||
| 126 | PAGE_ALIGN(size); | 120 | PAGE_ALIGN(size); |
| 127 | } | 121 | } |
| 128 | 122 | ||
| 123 | /** | ||
| 124 | * @amdgpu_amdkfd_reserve_mem_limit() - Decrease available memory by size | ||
| 125 | * of buffer including any reserved for control structures | ||
| 126 | * | ||
| 127 | * @adev: Device to which allocated BO belongs to | ||
| 128 | * @size: Size of buffer, in bytes, encapsulated by B0. This should be | ||
| 129 | * equivalent to amdgpu_bo_size(BO) | ||
| 130 | * @alloc_flag: Flag used in allocating a BO as noted above | ||
| 131 | * | ||
| 132 | * Return: returns -ENOMEM in case of error, ZERO otherwise | ||
| 133 | */ | ||
| 129 | static int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev, | 134 | static int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev, |
| 130 | uint64_t size, u32 domain, bool sg) | 135 | uint64_t size, u32 alloc_flag) |
| 131 | { | 136 | { |
| 132 | uint64_t reserved_for_pt = | 137 | uint64_t reserved_for_pt = |
| 133 | ESTIMATE_PT_SIZE(amdgpu_amdkfd_total_mem_size); | 138 | ESTIMATE_PT_SIZE(amdgpu_amdkfd_total_mem_size); |
| @@ -137,20 +142,24 @@ static int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev, · 3 unmodified lines | |||
| 137 | acc_size = amdgpu_amdkfd_acc_size(size); | 142 | acc_size = amdgpu_amdkfd_acc_size(size); |
| 138 | 143 | ||
| 139 | vram_needed = 0; | 144 | vram_needed = 0; |
| 140 | if (domain == AMDGPU_GEM_DOMAIN_GTT) { | 145 | if (alloc_flag & KFD_IOC_ALLOC_MEM_FLAGS_GTT) { |
| 141 | /* TTM GTT memory */ | ||
| 142 | system_mem_needed = acc_size + size; | 146 | system_mem_needed = acc_size + size; |
| 143 | ttm_mem_needed = acc_size + size; | 147 | ttm_mem_needed = acc_size + size; |
| 144 | } else if (domain == AMDGPU_GEM_DOMAIN_CPU && !sg) { | 148 | } else if (alloc_flag & KFD_IOC_ALLOC_MEM_FLAGS_VRAM) { |
| 145 | /* Userptr */ | 149 | system_mem_needed = acc_size; |
| 150 | ttm_mem_needed = acc_size; | ||
| 151 | vram_needed = size; | ||
| 152 | } else if (alloc_flag & KFD_IOC_ALLOC_MEM_FLAGS_USERPTR) { | ||
| 146 | system_mem_needed = acc_size + size; | 153 | system_mem_needed = acc_size + size; |
| 147 | ttm_mem_needed = acc_size; | 154 | ttm_mem_needed = acc_size; |
| 148 | } else { | 155 | } else if (alloc_flag & |
| 149 | /* VRAM and SG */ | 156 | (KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL | |
| 157 | KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP)) { | ||
| 150 | system_mem_needed = acc_size; | 158 | system_mem_needed = acc_size; |
| 151 | ttm_mem_needed = acc_size; | 159 | ttm_mem_needed = acc_size; |
| 152 | if (domain == AMDGPU_GEM_DOMAIN_VRAM) | 160 | } else { |
| 153 | vram_needed = size; | 161 | pr_err("%s: Invalid BO type %#x\n", __func__, alloc_flag); |
| 162 | return -ENOMEM; | ||
| 154 | } | 163 | } |
| 155 | 164 | ||
| 156 | spin_lock(&kfd_mem_limit.mem_limit_lock); | 165 | spin_lock(&kfd_mem_limit.mem_limit_lock); |
| @@ -166,64 +175,72 @@ static int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev, · 9 unmodified lines | |||
| 166 | (adev->kfd.vram_used + vram_needed > | 175 | (adev->kfd.vram_used + vram_needed > |
| 167 | adev->gmc.real_vram_size - reserved_for_pt)) { | 176 | adev->gmc.real_vram_size - reserved_for_pt)) { |
| 168 | ret = -ENOMEM; | 177 | ret = -ENOMEM; |
| 169 | } else { | 178 | goto release; |
| 170 | kfd_mem_limit.system_mem_used += system_mem_needed; | ||
| 171 | kfd_mem_limit.ttm_mem_used += ttm_mem_needed; | ||
| 172 | adev->kfd.vram_used += vram_needed; | ||
| 173 | } | 179 | } |
| 174 | 180 | ||
| 181 | /* Update memory accounting by decreasing available system | ||
| 182 | * memory, TTM memory and GPU memory as computed above | ||
| 183 | */ | ||
| 184 | adev->kfd.vram_used += vram_needed; | ||
| 185 | kfd_mem_limit.system_mem_used += system_mem_needed; | ||
| 186 | kfd_mem_limit.ttm_mem_used += ttm_mem_needed; | ||
| 187 | |||
| 188 | release: | ||
| 175 | spin_unlock(&kfd_mem_limit.mem_limit_lock); | 189 | spin_unlock(&kfd_mem_limit.mem_limit_lock); |
| 176 | return ret; | 190 | return ret; |
| 177 | } | 191 | } |
| 178 | 192 | ||
| 179 | static void unreserve_mem_limit(struct amdgpu_device *adev, | 193 | static void unreserve_mem_limit(struct amdgpu_device *adev, |
| 180 | uint64_t size, u32 domain, bool sg) | 194 | uint64_t size, u32 alloc_flag) |
| 181 | { | 195 | { |
| 182 | size_t acc_size; | 196 | size_t acc_size; |
| 183 | 197 | ||
| 184 | acc_size = amdgpu_amdkfd_acc_size(size); | 198 | acc_size = amdgpu_amdkfd_acc_size(size); |
| 185 | 199 | ||
| 186 | spin_lock(&kfd_mem_limit.mem_limit_lock); | 200 | spin_lock(&kfd_mem_limit.mem_limit_lock); |
| 187 | if (domain == AMDGPU_GEM_DOMAIN_GTT) { | 201 | |
| 202 | if (alloc_flag & KFD_IOC_ALLOC_MEM_FLAGS_GTT) { | ||
| 188 | kfd_mem_limit.system_mem_used -= (acc_size + size); | 203 | kfd_mem_limit.system_mem_used -= (acc_size + size); |
| 189 | kfd_mem_limit.ttm_mem_used -= (acc_size + size); | 204 | kfd_mem_limit.ttm_mem_used -= (acc_size + size); |
| 190 | } else if (domain == AMDGPU_GEM_DOMAIN_CPU && !sg) { | 205 | } else if (alloc_flag & KFD_IOC_ALLOC_MEM_FLAGS_VRAM) { |
| 206 | kfd_mem_limit.system_mem_used -= acc_size; | ||
| 207 | kfd_mem_limit.ttm_mem_used -= acc_size; | ||
| 208 | adev->kfd.vram_used -= size; | ||
| 209 | } else if (alloc_flag & KFD_IOC_ALLOC_MEM_FLAGS_USERPTR) { | ||
| 191 | kfd_mem_limit.system_mem_used -= (acc_size + size); | 210 | kfd_mem_limit.system_mem_used -= (acc_size + size); |
| 192 | kfd_mem_limit.ttm_mem_used -= acc_size; | 211 | kfd_mem_limit.ttm_mem_used -= acc_size; |
| 193 | } else { | 212 | } else if (alloc_flag & |
| 213 | (KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL | | ||
| 214 | KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP)) { | ||
| 194 | kfd_mem_limit.system_mem_used -= acc_size; | 215 | kfd_mem_limit.system_mem_used -= acc_size; |
| 195 | kfd_mem_limit.ttm_mem_used -= acc_size; | 216 | kfd_mem_limit.ttm_mem_used -= acc_size; |
| 196 | if (domain == AMDGPU_GEM_DOMAIN_VRAM) { | 217 | } else { |
| 197 | adev->kfd.vram_used -= size; | 218 | pr_err("%s: Invalid BO type %#x\n", __func__, alloc_flag); |
| 198 | WARN_ONCE(adev->kfd.vram_used < 0, | 219 | goto release; |
| 199 | "kfd VRAM memory accounting unbalanced"); | ||
| 200 | } | ||
| 201 | } | 220 | } |
| 202 | WARN_ONCE(kfd_mem_limit.system_mem_used < 0, | 221 | |
| 203 | "kfd system memory accounting unbalanced"); | 222 | WARN_ONCE(adev->kfd.vram_used < 0, |
| 223 | "KFD VRAM memory accounting unbalanced"); | ||
| 204 | WARN_ONCE(kfd_mem_limit.ttm_mem_used < 0, | 224 | WARN_ONCE(kfd_mem_limit.ttm_mem_used < 0, |
| 205 | "kfd TTM memory accounting unbalanced"); | 225 | "KFD TTM memory accounting unbalanced"); |
| 226 | WARN_ONCE(kfd_mem_limit.system_mem_used < 0, | ||
| 227 | "KFD system memory accounting unbalanced"); | ||
| 206 | 228 | ||
| 229 | release: | ||
| 207 | spin_unlock(&kfd_mem_limit.mem_limit_lock); | 230 | spin_unlock(&kfd_mem_limit.mem_limit_lock); |
| 208 | } | 231 | } |
| 209 | 232 | ||
| 210 | void amdgpu_amdkfd_release_notify(struct amdgpu_bo *bo) | 233 | void amdgpu_amdkfd_release_notify(struct amdgpu_bo *bo) |
| 211 | { | 234 | { |
| 212 | struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); | 235 | struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); |
| 213 | u32 domain = bo->preferred_domains; | 236 | u32 alloc_flags = bo->kfd_bo->alloc_flags; |
| 214 | bool sg = (bo->preferred_domains == AMDGPU_GEM_DOMAIN_CPU); | 237 | u64 size = amdgpu_bo_size(bo); |
| 215 | 238 | ||
| 216 | if (bo->flags & AMDGPU_AMDKFD_CREATE_USERPTR_BO) { | 239 | unreserve_mem_limit(adev, size, alloc_flags); |
| 217 | domain = AMDGPU_GEM_DOMAIN_CPU; | ||
| 218 | sg = false; | ||
| 219 | } | ||
| 220 | |||
| 221 | unreserve_mem_limit(adev, amdgpu_bo_size(bo), domain, sg); | ||
| 222 | 240 | ||
| 223 | kfree(bo->kfd_bo); | 241 | kfree(bo->kfd_bo); |
| 224 | } | 242 | } |
| 225 | 243 | ||
| 226 | |||
| 227 | /* amdgpu_amdkfd_remove_eviction_fence - Removes eviction fence from BO's | 244 | /* amdgpu_amdkfd_remove_eviction_fence - Removes eviction fence from BO's |
| 228 | * reservation object. | 245 | * reservation object. |
| 229 | * | 246 | * |
| @@ -691,10 +708,12 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, · 461 unmodified lines | |||
| 691 | pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, | 708 | pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, |
| 692 | va + bo_size, vm); | 709 | va + bo_size, vm); |
| 693 | 710 | ||
| 694 | if (adev == bo_adev || (mem->domain == AMDGPU_GEM_DOMAIN_VRAM && | 711 | if (adev == bo_adev || |
| 695 | amdgpu_xgmi_same_hive(adev, bo_adev))) { | 712 | (amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm) && adev->ram_is_direct_mapped) || |
| 696 | /* Mappings on the local GPU and VRAM mappings in the | 713 | (mem->domain == AMDGPU_GEM_DOMAIN_VRAM && amdgpu_xgmi_same_hive(adev, bo_adev))) { |
| 697 | * local hive share the original BO | 714 | /* Mappings on the local GPU, or VRAM mappings in the |
| 715 | * local hive, or userptr mapping IOMMU direct map mode | ||
| 716 | * share the original BO | ||
| 698 | */ | 717 | */ |
| 699 | attachment[i]->type = KFD_MEM_ATT_SHARED; | 718 | attachment[i]->type = KFD_MEM_ATT_SHARED; |
| 700 | bo[i] = mem->bo; | 719 | bo[i] = mem->bo; |
| @@ -1272,12 +1291,60 @@ create_evict_fence_fail: · 571 unmodified lines | |||
| 1272 | return ret; | 1291 | return ret; |
| 1273 | } | 1292 | } |
| 1274 | 1293 | ||
| 1275 | int amdgpu_amdkfd_gpuvm_acquire_process_vm(struct kgd_dev *kgd, | 1294 | /** |
| 1295 | * amdgpu_amdkfd_gpuvm_pin_bo() - Pins a BO using following criteria | ||
| 1296 | * @bo: Handle of buffer object being pinned | ||
| 1297 | * @domain: Domain into which BO should be pinned | ||
| 1298 | * | ||
| 1299 | * - USERPTR BOs are UNPINNABLE and will return error | ||
| 1300 | * - All other BO types (GTT, VRAM, MMIO and DOORBELL) will have their | ||
| 1301 | * PIN count incremented. It is valid to PIN a BO multiple times | ||
| 1302 | * | ||
| 1303 | * Return: ZERO if successful in pinning, Non-Zero in case of error. | ||
| 1304 | */ | ||
| 1305 | static int amdgpu_amdkfd_gpuvm_pin_bo(struct amdgpu_bo *bo, u32 domain) | ||
| 1306 | { | ||
| 1307 | int ret = 0; | ||
| 1308 | |||
| 1309 | ret = amdgpu_bo_reserve(bo, false); | ||
| 1310 | if (unlikely(ret)) | ||
| 1311 | return ret; | ||
| 1312 | |||
| 1313 | ret = amdgpu_bo_pin_restricted(bo, domain, 0, 0); | ||
| 1314 | if (ret) | ||
| 1315 | pr_err("Error in Pinning BO to domain: %d\n", domain); | ||
| 1316 | |||
| 1317 | amdgpu_bo_sync_wait(bo, AMDGPU_FENCE_OWNER_KFD, false); | ||
| 1318 | amdgpu_bo_unreserve(bo); | ||
| 1319 | |||
| 1320 | return ret; | ||
| 1321 | } | ||
| 1322 | |||
| 1323 | /** | ||
| 1324 | * amdgpu_amdkfd_gpuvm_unpin_bo() - Unpins BO using following criteria | ||
| 1325 | * @bo: Handle of buffer object being unpinned | ||
| 1326 | * | ||
| 1327 | * - Is a illegal request for USERPTR BOs and is ignored | ||
| 1328 | * - All other BO types (GTT, VRAM, MMIO and DOORBELL) will have their | ||
| 1329 | * PIN count decremented. Calls to UNPIN must balance calls to PIN | ||
| 1330 | */ | ||
| 1331 | static void amdgpu_amdkfd_gpuvm_unpin_bo(struct amdgpu_bo *bo) | ||
| 1332 | { | ||
| 1333 | int ret = 0; | ||
| 1334 | |||
| 1335 | ret = amdgpu_bo_reserve(bo, false); | ||
| 1336 | if (unlikely(ret)) | ||
| 1337 | return; | ||
| 1338 | |||
| 1339 | amdgpu_bo_unpin(bo); | ||
| 1340 | amdgpu_bo_unreserve(bo); | ||
| 1341 | } | ||
| 1342 | |||
| 1343 | int amdgpu_amdkfd_gpuvm_acquire_process_vm(struct amdgpu_device *adev, | ||
| 1276 | struct file *filp, u32 pasid, | 1344 | struct file *filp, u32 pasid, |
| 1277 | void **process_info, | 1345 | void **process_info, |
| 1278 | struct dma_fence **ef) | 1346 | struct dma_fence **ef) |
| 1279 | { | 1347 | { |
| 1280 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 1281 | struct amdgpu_fpriv *drv_priv; | 1348 | struct amdgpu_fpriv *drv_priv; |
| 1282 | struct amdgpu_vm *avm; | 1349 | struct amdgpu_vm *avm; |
| 1283 | int ret; | 1350 | int ret; |
| @@ -1353,12 +1420,12 @@ void amdgpu_amdkfd_gpuvm_destroy_cb(struct amdgpu_device *adev, · 69 unmodified lines | |||
| 1353 | } | 1420 | } |
| 1354 | } | 1421 | } |
| 1355 | 1422 | ||
| 1356 | void amdgpu_amdkfd_gpuvm_release_process_vm(struct kgd_dev *kgd, void *drm_priv) | 1423 | void amdgpu_amdkfd_gpuvm_release_process_vm(struct amdgpu_device *adev, |
| 1424 | void *drm_priv) | ||
| 1357 | { | 1425 | { |
| 1358 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 1359 | struct amdgpu_vm *avm; | 1426 | struct amdgpu_vm *avm; |
| 1360 | 1427 | ||
| 1361 | if (WARN_ON(!kgd || !drm_priv)) | 1428 | if (WARN_ON(!adev || !drm_priv)) |
| 1362 | return; | 1429 | return; |
| 1363 | 1430 | ||
| 1364 | avm = drm_priv_to_vm(drm_priv); | 1431 | avm = drm_priv_to_vm(drm_priv); |
| @@ -1386,11 +1453,10 @@ uint64_t amdgpu_amdkfd_gpuvm_get_process_page_dir(void *drm_priv) · 21 unmodified lines | |||
| 1386 | } | 1453 | } |
| 1387 | 1454 | ||
| 1388 | int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( | 1455 | int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( |
| 1389 | struct kgd_dev *kgd, uint64_t va, uint64_t size, | 1456 | struct amdgpu_device *adev, uint64_t va, uint64_t size, |
| 1390 | void *drm_priv, struct kgd_mem **mem, | 1457 | void *drm_priv, struct kgd_mem **mem, |
| 1391 | uint64_t *offset, uint32_t flags) | 1458 | uint64_t *offset, uint32_t flags) |
| 1392 | { | 1459 | { |
| 1393 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 1394 | struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); | 1460 | struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); |
| 1395 | enum ttm_bo_type bo_type = ttm_bo_type_device; | 1461 | enum ttm_bo_type bo_type = ttm_bo_type_device; |
| 1396 | struct sg_table *sg = NULL; | 1462 | struct sg_table *sg = NULL; |
| @@ -1454,7 +1520,7 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( · 57 unmodified lines | |||
| 1454 | 1520 | ||
| 1455 | amdgpu_sync_create(&(*mem)->sync); | 1521 | amdgpu_sync_create(&(*mem)->sync); |
| 1456 | 1522 | ||
| 1457 | ret = amdgpu_amdkfd_reserve_mem_limit(adev, size, alloc_domain, !!sg); | 1523 | ret = amdgpu_amdkfd_reserve_mem_limit(adev, size, flags); |
| 1458 | if (ret) { | 1524 | if (ret) { |
| 1459 | pr_debug("Insufficient memory\n"); | 1525 | pr_debug("Insufficient memory\n"); |
| 1460 | goto err_reserve_limit; | 1526 | goto err_reserve_limit; |
| @@ -1495,6 +1561,15 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( · 34 unmodified lines | |||
| 1495 | ret = init_user_pages(*mem, user_addr); | 1561 | ret = init_user_pages(*mem, user_addr); |
| 1496 | if (ret) | 1562 | if (ret) |
| 1497 | goto allocate_init_user_pages_failed; | 1563 | goto allocate_init_user_pages_failed; |
| 1564 | } else if (flags & (KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL | | ||
| 1565 | KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP)) { | ||
| 1566 | ret = amdgpu_amdkfd_gpuvm_pin_bo(bo, AMDGPU_GEM_DOMAIN_GTT); | ||
| 1567 | if (ret) { | ||
| 1568 | pr_err("Pinning MMIO/DOORBELL BO during ALLOC FAILED\n"); | ||
| 1569 | goto err_pin_bo; | ||
| 1570 | } | ||
| 1571 | bo->allowed_domains = AMDGPU_GEM_DOMAIN_GTT; | ||
| 1572 | bo->preferred_domains = AMDGPU_GEM_DOMAIN_GTT; | ||
| 1498 | } | 1573 | } |
| 1499 | 1574 | ||
| 1500 | if (offset) | 1575 | if (offset) |
| @@ -1503,13 +1578,14 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( · 2 unmodified lines | |||
| 1503 | return 0; | 1578 | return 0; |
| 1504 | 1579 | ||
| 1505 | allocate_init_user_pages_failed: | 1580 | allocate_init_user_pages_failed: |
| 1581 | err_pin_bo: | ||
| 1506 | remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info); | 1582 | remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info); |
| 1507 | drm_vma_node_revoke(&gobj->vma_node, drm_priv); | 1583 | drm_vma_node_revoke(&gobj->vma_node, drm_priv); |
| 1508 | err_node_allow: | 1584 | err_node_allow: |
| 1509 | /* Don't unreserve system mem limit twice */ | 1585 | /* Don't unreserve system mem limit twice */ |
| 1510 | goto err_reserve_limit; | 1586 | goto err_reserve_limit; |
| 1511 | err_bo_create: | 1587 | err_bo_create: |
| 1512 | unreserve_mem_limit(adev, size, alloc_domain, !!sg); | 1588 | unreserve_mem_limit(adev, size, flags); |
| 1513 | err_reserve_limit: | 1589 | err_reserve_limit: |
| 1514 | mutex_destroy(&(*mem)->lock); | 1590 | mutex_destroy(&(*mem)->lock); |
| 1515 | if (gobj) | 1591 | if (gobj) |
| @@ -1525,7 +1601,7 @@ err: · 9 unmodified lines | |||
| 1525 | } | 1601 | } |
| 1526 | 1602 | ||
| 1527 | int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( | 1603 | int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( |
| 1528 | struct kgd_dev *kgd, struct kgd_mem *mem, void *drm_priv, | 1604 | struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv, |
| 1529 | uint64_t *size) | 1605 | uint64_t *size) |
| 1530 | { | 1606 | { |
| 1531 | struct amdkfd_process_info *process_info = mem->process_info; | 1607 | struct amdkfd_process_info *process_info = mem->process_info; |
| @@ -1538,6 +1614,14 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( · 6 unmodified lines | |||
| 1538 | bool is_imported = false; | 1614 | bool is_imported = false; |
| 1539 | 1615 | ||
| 1540 | mutex_lock(&mem->lock); | 1616 | mutex_lock(&mem->lock); |
| 1617 | |||
| 1618 | /* Unpin MMIO/DOORBELL BO's that were pinnned during allocation */ | ||
| 1619 | if (mem->alloc_flags & | ||
| 1620 | (KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL | | ||
| 1621 | KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP)) { | ||
| 1622 | amdgpu_amdkfd_gpuvm_unpin_bo(mem->bo); | ||
| 1623 | } | ||
| 1624 | |||
| 1541 | mapped_to_gpu_memory = mem->mapped_to_gpu_memory; | 1625 | mapped_to_gpu_memory = mem->mapped_to_gpu_memory; |
| 1542 | is_imported = mem->is_imported; | 1626 | is_imported = mem->is_imported; |
| 1543 | mutex_unlock(&mem->lock); | 1627 | mutex_unlock(&mem->lock); |
| @@ -1617,10 +1701,9 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( · 73 unmodified lines | |||
| 1617 | } | 1701 | } |
| 1618 | 1702 | ||
| 1619 | int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( | 1703 | int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( |
| 1620 | struct kgd_dev *kgd, struct kgd_mem *mem, | 1704 | struct amdgpu_device *adev, struct kgd_mem *mem, |
| 1621 | void *drm_priv, bool *table_freed) | 1705 | void *drm_priv, bool *table_freed) |
| 1622 | { | 1706 | { |
| 1623 | struct amdgpu_device *adev = get_amdgpu_device(kgd); | ||
| 1624 | struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); | 1707 | struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); |
| 1625 | int ret; | 1708 | int ret; |
| 1626 | struct amdgpu_bo *bo; | 1709 | struct amdgpu_bo *bo; |
| @@ -1747,7 +1830,7 @@ out: · 120 unmodified lines | |||
| 1747 | } | 1830 | } |
| 1748 | 1831 | ||
| 1749 | int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( | 1832 | int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( |
| 1750 | struct kgd_dev *kgd, struct kgd_mem *mem, void *drm_priv) | 1833 | struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv) |
| 1751 | { | 1834 | { |
| 1752 | struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); | 1835 | struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); |
| 1753 | struct amdkfd_process_info *process_info = avm->process_info; | 1836 | struct amdkfd_process_info *process_info = avm->process_info; |
| @@ -1808,7 +1891,7 @@ out: · 54 unmodified lines | |||
| 1808 | } | 1891 | } |
| 1809 | 1892 | ||
| 1810 | int amdgpu_amdkfd_gpuvm_sync_memory( | 1893 | int amdgpu_amdkfd_gpuvm_sync_memory( |
| 1811 | struct kgd_dev *kgd, struct kgd_mem *mem, bool intr) | 1894 | struct amdgpu_device *adev, struct kgd_mem *mem, bool intr) |
| 1812 | { | 1895 | { |
| 1813 | struct amdgpu_sync sync; | 1896 | struct amdgpu_sync sync; |
| 1814 | int ret; | 1897 | int ret; |
| @@ -1824,7 +1907,7 @@ int amdgpu_amdkfd_gpuvm_sync_memory( · 9 unmodified lines | |||
| 1824 | return ret; | 1907 | return ret; |
| 1825 | } | 1908 | } |
| 1826 | 1909 | ||
| 1827 | int amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(struct kgd_dev *kgd, | 1910 | int amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(struct amdgpu_device *adev, |
| 1828 | struct kgd_mem *mem, void **kptr, uint64_t *size) | 1911 | struct kgd_mem *mem, void **kptr, uint64_t *size) |
| 1829 | { | 1912 | { |
| 1830 | int ret; | 1913 | int ret; |
| @@ -1880,7 +1963,8 @@ bo_reserve_failed: · 49 unmodified lines | |||
| 1880 | return ret; | 1963 | return ret; |
| 1881 | } | 1964 | } |
| 1882 | 1965 | ||
| 1883 | void amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(struct kgd_dev *kgd, struct kgd_mem *mem) | 1966 | void amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(struct amdgpu_device *adev, |
| 1967 | struct kgd_mem *mem) | ||
| 1884 | { | 1968 | { |
| 1885 | struct amdgpu_bo *bo = mem->bo; | 1969 | struct amdgpu_bo *bo = mem->bo; |
| 1886 | 1970 | ||
| @@ -1890,12 +1974,9 @@ void amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(struct kgd_dev *kgd, struct kg · 3 unmodified lines | |||
| 1890 | amdgpu_bo_unreserve(bo); | 1974 | amdgpu_bo_unreserve(bo); |
| 1891 | } | 1975 | } |
| 1892 | 1976 | ||
| 1893 | int amdgpu_amdkfd_gpuvm_get_vm_fault_info(struct kgd_dev *kgd, | 1977 | int amdgpu_amdkfd_gpuvm_get_vm_fault_info(struct amdgpu_device *adev, |
| 1894 | struct kfd_vm_fault_info *mem) | 1978 | struct kfd_vm_fault_info *mem) |
| 1895 | { | 1979 | { |
| 1896 | struct amdgpu_device *adev; | ||
| 1897 | |||
| 1898 | adev = (struct amdgpu_device *)kgd; | ||
| 1899 | if (atomic_read(&adev->gmc.vm_fault_info_updated) == 1) { | 1980 | if (atomic_read(&adev->gmc.vm_fault_info_updated) == 1) { |
| 1900 | *mem = *adev->gmc.vm_fault_info; | 1981 | *mem = *adev->gmc.vm_fault_info; |
| 1901 | mb(); | 1982 | mb(); |
| @@ -1904,13 +1985,12 @@ int amdgpu_amdkfd_gpuvm_get_vm_fault_info(struct kgd_dev *kgd, · 2 unmodified lines | |||
| 1904 | return 0; | 1985 | return 0; |
| 1905 | } | 1986 | } |
| 1906 | 1987 | ||
| 1907 | int amdgpu_amdkfd_gpuvm_import_dmabuf(struct kgd_dev *kgd, | 1988 | int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev, |
| 1908 | struct dma_buf *dma_buf, | 1989 | struct dma_buf *dma_buf, |
| 1909 | uint64_t va, void *drm_priv, | 1990 | uint64_t va, void *drm_priv, |
| 1910 | struct kgd_mem **mem, uint64_t *size, | 1991 | struct kgd_mem **mem, uint64_t *size, |
| 1911 | uint64_t *mmap_offset) | 1992 | uint64_t *mmap_offset) |
| 1912 | { | 1993 | { |
| 1913 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 1914 | struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); | 1994 | struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); |
| 1915 | struct drm_gem_object *obj; | 1995 | struct drm_gem_object *obj; |
| 1916 | struct amdgpu_bo *bo; | 1996 | struct amdgpu_bo *bo; |
| @@ -2537,11 +2617,9 @@ int amdgpu_amdkfd_remove_gws_from_process(void *info, void *mem) · 620 unmodified lines | |||
| 2537 | } | 2617 | } |
| 2538 | 2618 | ||
| 2539 | /* Returns GPU-specific tiling mode information */ | 2619 | /* Returns GPU-specific tiling mode information */ |
| 2540 | int amdgpu_amdkfd_get_tile_config(struct kgd_dev *kgd, | 2620 | int amdgpu_amdkfd_get_tile_config(struct amdgpu_device *adev, |
| 2541 | struct tile_config *config) | 2621 | struct tile_config *config) |
| 2542 | { | 2622 | { |
| 2543 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | ||
| 2544 | |||
| 2545 | config->gb_addr_config = adev->gfx.config.gb_addr_config; | 2623 | config->gb_addr_config = adev->gfx.config.gb_addr_config; |
| 2546 | config->tile_config_ptr = adev->gfx.config.tile_mode_array; | 2624 | config->tile_config_ptr = adev->gfx.config.tile_mode_array; |
| 2547 | config->num_tile_configs = | 2625 | config->num_tile_configs = |
| @@ -470,8 +470,8 @@ bool amdgpu_atomfirmware_dynamic_boot_config_supported(struct amdgpu_device *ade · 469 unmodified lines | |||
| 470 | 470 | ||
| 471 | /** | 471 | /** |
| 472 | * amdgpu_atomfirmware_ras_rom_addr -- Get the RAS EEPROM addr from VBIOS | 472 | * amdgpu_atomfirmware_ras_rom_addr -- Get the RAS EEPROM addr from VBIOS |
| 473 | * adev: amdgpu_device pointer | 473 | * @adev: amdgpu_device pointer |
| 474 | * i2c_address: pointer to u8; if not NULL, will contain | 474 | * @i2c_address: pointer to u8; if not NULL, will contain |
| 475 | * the RAS EEPROM address if the function returns true | 475 | * the RAS EEPROM address if the function returns true |
| 476 | * | 476 | * |
| 477 | * Return true if VBIOS supports RAS EEPROM address reporting, | 477 | * Return true if VBIOS supports RAS EEPROM address reporting, |
| @@ -11,6 +11,7 @@ · 10 unmodified lines | |||
| 11 | #include <linux/pci.h> | 11 | #include <linux/pci.h> |
| 12 | #include <linux/delay.h> | 12 | #include <linux/delay.h> |
| 13 | 13 | ||
| 14 | #include "amdgpu.h" | ||
| 14 | #include "amd_acpi.h" | 15 | #include "amd_acpi.h" |
| 15 | 16 | ||
| 16 | #define AMDGPU_PX_QUIRK_FORCE_ATPX (1 << 0) | 17 | #define AMDGPU_PX_QUIRK_FORCE_ATPX (1 << 0) |
| @@ -165,7 +166,7 @@ static void amdgpu_atpx_parse_functions(struct amdgpu_atpx_functions *f, u32 mas · 148 unmodified lines | |||
| 165 | } | 166 | } |
| 166 | 167 | ||
| 167 | /** | 168 | /** |
| 168 | * amdgpu_atpx_validate_functions - validate ATPX functions | 169 | * amdgpu_atpx_validate - validate ATPX functions |
| 169 | * | 170 | * |
| 170 | * @atpx: amdgpu atpx struct | 171 | * @atpx: amdgpu atpx struct |
| 171 | * | 172 | * |
| @@ -108,7 +108,7 @@ int amdgpu_connector_get_monitor_bpc(struct drm_connector *connector) · 107 unmodified lines | |||
| 108 | case DRM_MODE_CONNECTOR_DVII: | 108 | case DRM_MODE_CONNECTOR_DVII: |
| 109 | case DRM_MODE_CONNECTOR_HDMIB: | 109 | case DRM_MODE_CONNECTOR_HDMIB: |
| 110 | if (amdgpu_connector->use_digital) { | 110 | if (amdgpu_connector->use_digital) { |
| 111 | if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) { | 111 | if (connector->display_info.is_hdmi) { |
| 112 | if (connector->display_info.bpc) | 112 | if (connector->display_info.bpc) |
| 113 | bpc = connector->display_info.bpc; | 113 | bpc = connector->display_info.bpc; |
| 114 | } | 114 | } |
| @@ -116,7 +116,7 @@ int amdgpu_connector_get_monitor_bpc(struct drm_connector *connector) · 1 unmodified line | |||
| 116 | break; | 116 | break; |
| 117 | case DRM_MODE_CONNECTOR_DVID: | 117 | case DRM_MODE_CONNECTOR_DVID: |
| 118 | case DRM_MODE_CONNECTOR_HDMIA: | 118 | case DRM_MODE_CONNECTOR_HDMIA: |
| 119 | if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) { | 119 | if (connector->display_info.is_hdmi) { |
| 120 | if (connector->display_info.bpc) | 120 | if (connector->display_info.bpc) |
| 121 | bpc = connector->display_info.bpc; | 121 | bpc = connector->display_info.bpc; |
| 122 | } | 122 | } |
| @@ -125,7 +125,7 @@ int amdgpu_connector_get_monitor_bpc(struct drm_connector *connector) · 2 unmodified lines | |||
| 125 | dig_connector = amdgpu_connector->con_priv; | 125 | dig_connector = amdgpu_connector->con_priv; |
| 126 | if ((dig_connector->dp_sink_type == CONNECTOR_OBJECT_ID_DISPLAYPORT) || | 126 | if ((dig_connector->dp_sink_type == CONNECTOR_OBJECT_ID_DISPLAYPORT) || |
| 127 | (dig_connector->dp_sink_type == CONNECTOR_OBJECT_ID_eDP) || | 127 | (dig_connector->dp_sink_type == CONNECTOR_OBJECT_ID_eDP) || |
| 128 | drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) { | 128 | connector->display_info.is_hdmi) { |
| 129 | if (connector->display_info.bpc) | 129 | if (connector->display_info.bpc) |
| 130 | bpc = connector->display_info.bpc; | 130 | bpc = connector->display_info.bpc; |
| 131 | } | 131 | } |
| @@ -149,7 +149,7 @@ int amdgpu_connector_get_monitor_bpc(struct drm_connector *connector) · 17 unmodified lines | |||
| 149 | break; | 149 | break; |
| 150 | } | 150 | } |
| 151 | 151 | ||
| 152 | if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) { | 152 | if (connector->display_info.is_hdmi) { |
| 153 | /* | 153 | /* |
| 154 | * Pre DCE-8 hw can't handle > 12 bpc, and more than 12 bpc doesn't make | 154 | * Pre DCE-8 hw can't handle > 12 bpc, and more than 12 bpc doesn't make |
| 155 | * much sense without support for > 12 bpc framebuffers. RGB 4:4:4 at | 155 | * much sense without support for > 12 bpc framebuffers. RGB 4:4:4 at |
| @@ -315,8 +315,10 @@ static void amdgpu_connector_get_edid(struct drm_connector *connector) · 159 unmodified lines | |||
| 315 | if (!amdgpu_connector->edid) { | 315 | if (!amdgpu_connector->edid) { |
| 316 | /* some laptops provide a hardcoded edid in rom for LCDs */ | 316 | /* some laptops provide a hardcoded edid in rom for LCDs */ |
| 317 | if (((connector->connector_type == DRM_MODE_CONNECTOR_LVDS) || | 317 | if (((connector->connector_type == DRM_MODE_CONNECTOR_LVDS) || |
| 318 | (connector->connector_type == DRM_MODE_CONNECTOR_eDP))) | 318 | (connector->connector_type == DRM_MODE_CONNECTOR_eDP))) { |
| 319 | amdgpu_connector->edid = amdgpu_connector_get_hardcoded_edid(adev); | 319 | amdgpu_connector->edid = amdgpu_connector_get_hardcoded_edid(adev); |
| 320 | drm_connector_update_edid_property(connector, amdgpu_connector->edid); | ||
| 321 | } | ||
| 320 | } | 322 | } |
| 321 | } | 323 | } |
| 322 | 324 | ||
| @@ -326,6 +328,7 @@ static void amdgpu_connector_free_edid(struct drm_connector *connector) · 3 unmodified lines | |||
| 326 | 328 | ||
| 327 | kfree(amdgpu_connector->edid); | 329 | kfree(amdgpu_connector->edid); |
| 328 | amdgpu_connector->edid = NULL; | 330 | amdgpu_connector->edid = NULL; |
| 331 | drm_connector_update_edid_property(connector, NULL); | ||
| 329 | } | 332 | } |
| 330 | 333 | ||
| 331 | static int amdgpu_connector_ddc_get_modes(struct drm_connector *connector) | 334 | static int amdgpu_connector_ddc_get_modes(struct drm_connector *connector) |
| @@ -387,6 +390,9 @@ amdgpu_connector_lcd_native_mode(struct drm_encoder *encoder) · 55 unmodified lines | |||
| 387 | native_mode->vdisplay != 0 && | 390 | native_mode->vdisplay != 0 && |
| 388 | native_mode->clock != 0) { | 391 | native_mode->clock != 0) { |
| 389 | mode = drm_mode_duplicate(dev, native_mode); | 392 | mode = drm_mode_duplicate(dev, native_mode); |
| 393 | if (!mode) | ||
| 394 | return NULL; | ||
| 395 | |||
| 390 | mode->type = DRM_MODE_TYPE_PREFERRED | DRM_MODE_TYPE_DRIVER; | 396 | mode->type = DRM_MODE_TYPE_PREFERRED | DRM_MODE_TYPE_DRIVER; |
| 391 | drm_mode_set_name(mode); | 397 | drm_mode_set_name(mode); |
| 392 | 398 | ||
| @@ -401,6 +407,9 @@ amdgpu_connector_lcd_native_mode(struct drm_encoder *encoder) · 8 unmodified lines | |||
| 401 | * simpler. | 407 | * simpler. |
| 402 | */ | 408 | */ |
| 403 | mode = drm_cvt_mode(dev, native_mode->hdisplay, native_mode->vdisplay, 60, true, false, false); | 409 | mode = drm_cvt_mode(dev, native_mode->hdisplay, native_mode->vdisplay, 60, true, false, false); |
| 410 | if (!mode) | ||
| 411 | return NULL; | ||
| 412 | |||
| 404 | mode->type = DRM_MODE_TYPE_PREFERRED | DRM_MODE_TYPE_DRIVER; | 413 | mode->type = DRM_MODE_TYPE_PREFERRED | DRM_MODE_TYPE_DRIVER; |
| 405 | DRM_DEBUG_KMS("Adding cvt approximation of native panel mode %s\n", mode->name); | 414 | DRM_DEBUG_KMS("Adding cvt approximation of native panel mode %s\n", mode->name); |
| 406 | } | 415 | } |
| @@ -1171,7 +1180,7 @@ static enum drm_mode_status amdgpu_connector_dvi_mode_valid(struct drm_connector · 764 unmodified lines | |||
| 1171 | (amdgpu_connector->connector_object_id == CONNECTOR_OBJECT_ID_DUAL_LINK_DVI_D) || | 1180 | (amdgpu_connector->connector_object_id == CONNECTOR_OBJECT_ID_DUAL_LINK_DVI_D) || |
| 1172 | (amdgpu_connector->connector_object_id == CONNECTOR_OBJECT_ID_HDMI_TYPE_B)) { | 1181 | (amdgpu_connector->connector_object_id == CONNECTOR_OBJECT_ID_HDMI_TYPE_B)) { |
| 1173 | return MODE_OK; | 1182 | return MODE_OK; |
| 1174 | } else if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) { | 1183 | } else if (connector->display_info.is_hdmi) { |
| 1175 | /* HDMI 1.3+ supports max clock of 340 Mhz */ | 1184 | /* HDMI 1.3+ supports max clock of 340 Mhz */ |
| 1176 | if (mode->clock > 340000) | 1185 | if (mode->clock > 340000) |
| 1177 | return MODE_CLOCK_HIGH; | 1186 | return MODE_CLOCK_HIGH; |
| @@ -1463,7 +1472,7 @@ static enum drm_mode_status amdgpu_connector_dp_mode_valid(struct drm_connector · 285 unmodified lines | |||
| 1463 | (amdgpu_dig_connector->dp_sink_type == CONNECTOR_OBJECT_ID_eDP)) { | 1472 | (amdgpu_dig_connector->dp_sink_type == CONNECTOR_OBJECT_ID_eDP)) { |
| 1464 | return amdgpu_atombios_dp_mode_valid_helper(connector, mode); | 1473 | return amdgpu_atombios_dp_mode_valid_helper(connector, mode); |
| 1465 | } else { | 1474 | } else { |
| 1466 | if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) { | 1475 | if (connector->display_info.is_hdmi) { |
| 1467 | /* HDMI 1.3+ supports max clock of 340 Mhz */ | 1476 | /* HDMI 1.3+ supports max clock of 340 Mhz */ |
| 1468 | if (mode->clock > 340000) | 1477 | if (mode->clock > 340000) |
| 1469 | return MODE_CLOCK_HIGH; | 1478 | return MODE_CLOCK_HIGH; |
| @@ -1618,6 +1618,9 @@ int amdgpu_debugfs_init(struct amdgpu_device *adev) · 1617 unmodified lines | |||
| 1618 | if (!debugfs_initialized()) | 1618 | if (!debugfs_initialized()) |
| 1619 | return 0; | 1619 | return 0; |
| 1620 | 1620 | ||
| 1621 | debugfs_create_x32("amdgpu_smu_debug", 0600, root, | ||
| 1622 | &adev->pm.smu_debug_mask); | ||
| 1623 | |||
| 1621 | ent = debugfs_create_file("amdgpu_preempt_ib", 0600, root, adev, | 1624 | ent = debugfs_create_file("amdgpu_preempt_ib", 0600, root, adev, |
| 1622 | &fops_ib_preempt); | 1625 | &fops_ib_preempt); |
| 1623 | if (IS_ERR(ent)) { | 1626 | if (IS_ERR(ent)) { |
| @@ -30,6 +30,7 @@ · 29 unmodified lines | |||
| 30 | #include <linux/module.h> | 30 | #include <linux/module.h> |
| 31 | #include <linux/console.h> | 31 | #include <linux/console.h> |
| 32 | #include <linux/slab.h> | 32 | #include <linux/slab.h> |
| 33 | #include <linux/iommu.h> | ||
| 33 | 34 | ||
| 34 | #include <drm/drm_atomic_helper.h> | 35 | #include <drm/drm_atomic_helper.h> |
| 35 | #include <drm/drm_probe_helper.h> | 36 | #include <drm/drm_probe_helper.h> |
| @@ -331,7 +332,7 @@ void amdgpu_device_mm_access(struct amdgpu_device *adev, loff_t pos, · 295 unmodified lines | |||
| 331 | } | 332 | } |
| 332 | 333 | ||
| 333 | /** | 334 | /** |
| 334 | * amdgpu_device_vram_access - access vram by vram aperature | 335 | * amdgpu_device_aper_access - access vram by vram aperature |
| 335 | * | 336 | * |
| 336 | * @adev: amdgpu_device pointer | 337 | * @adev: amdgpu_device pointer |
| 337 | * @pos: offset of the buffer in vram | 338 | * @pos: offset of the buffer in vram |
| @@ -550,11 +551,11 @@ void amdgpu_device_wreg(struct amdgpu_device *adev, · 212 unmodified lines | |||
| 550 | trace_amdgpu_device_wreg(adev->pdev->device, reg, v); | 551 | trace_amdgpu_device_wreg(adev->pdev->device, reg, v); |
| 551 | } | 552 | } |
| 552 | 553 | ||
| 553 | /* | 554 | /** |
| 554 | * amdgpu_mm_wreg_mmio_rlc - write register either with mmio or with RLC path if in range | 555 | * amdgpu_mm_wreg_mmio_rlc - write register either with mmio or with RLC path if in range |
| 555 | * | 556 | * |
| 556 | * this function is invoked only the debugfs register access | 557 | * this function is invoked only the debugfs register access |
| 557 | * */ | 558 | */ |
| 558 | void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev, | 559 | void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev, |
| 559 | uint32_t reg, uint32_t v) | 560 | uint32_t reg, uint32_t v) |
| 560 | { | 561 | { |
| @@ -1100,7 +1101,7 @@ static void amdgpu_device_wb_fini(struct amdgpu_device *adev) · 539 unmodified lines | |||
| 1100 | } | 1101 | } |
| 1101 | 1102 | ||
| 1102 | /** | 1103 | /** |
| 1103 | * amdgpu_device_wb_init- Init Writeback driver info and allocate memory | 1104 | * amdgpu_device_wb_init - Init Writeback driver info and allocate memory |
| 1104 | * | 1105 | * |
| 1105 | * @adev: amdgpu_device pointer | 1106 | * @adev: amdgpu_device pointer |
| 1106 | * | 1107 | * |
| @@ -2316,6 +2317,10 @@ static int amdgpu_device_ip_init(struct amdgpu_device *adev) · 1209 unmodified lines | |||
| 2316 | 2317 | ||
| 2317 | /* need to do gmc hw init early so we can allocate gpu mem */ | 2318 | /* need to do gmc hw init early so we can allocate gpu mem */ |
| 2318 | if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GMC) { | 2319 | if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GMC) { |
| 2320 | /* Try to reserve bad pages early */ | ||
| 2321 | if (amdgpu_sriov_vf(adev)) | ||
| 2322 | amdgpu_virt_exchange_data(adev); | ||
| 2323 | |||
| 2319 | r = amdgpu_device_vram_scratch_init(adev); | 2324 | r = amdgpu_device_vram_scratch_init(adev); |
| 2320 | if (r) { | 2325 | if (r) { |
| 2321 | DRM_ERROR("amdgpu_vram_scratch_init failed %d\n", r); | 2326 | DRM_ERROR("amdgpu_vram_scratch_init failed %d\n", r); |
| @@ -2347,7 +2352,7 @@ static int amdgpu_device_ip_init(struct amdgpu_device *adev) · 25 unmodified lines | |||
| 2347 | } | 2352 | } |
| 2348 | 2353 | ||
| 2349 | if (amdgpu_sriov_vf(adev)) | 2354 | if (amdgpu_sriov_vf(adev)) |
| 2350 | amdgpu_virt_init_data_exchange(adev); | 2355 | amdgpu_virt_exchange_data(adev); |
| 2351 | 2356 | ||
| 2352 | r = amdgpu_ib_pool_init(adev); | 2357 | r = amdgpu_ib_pool_init(adev); |
| 2353 | if (r) { | 2358 | if (r) { |
| @@ -2614,11 +2619,10 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) · 260 unmodified lines | |||
| 2614 | if (r) | 2619 | if (r) |
| 2615 | DRM_ERROR("enable mgpu fan boost failed (%d).\n", r); | 2620 | DRM_ERROR("enable mgpu fan boost failed (%d).\n", r); |
| 2616 | 2621 | ||
| 2617 | /* For XGMI + passthrough configuration on arcturus, enable light SBR */ | 2622 | /* For passthrough configuration on arcturus and aldebaran, enable special handling SBR */ |
| 2618 | if (adev->asic_type == CHIP_ARCTURUS && | 2623 | if (amdgpu_passthrough(adev) && ((adev->asic_type == CHIP_ARCTURUS && adev->gmc.xgmi.num_physical_nodes > 1)|| |
| 2619 | amdgpu_passthrough(adev) && | 2624 | adev->asic_type == CHIP_ALDEBARAN )) |
| 2620 | adev->gmc.xgmi.num_physical_nodes > 1) | 2625 | smu_handle_passthrough_sbr(&adev->smu, true); |
| 2621 | smu_set_light_sbr(&adev->smu, true); | ||
| 2622 | 2626 | ||
| 2623 | if (adev->gmc.xgmi.num_physical_nodes > 1) { | 2627 | if (adev->gmc.xgmi.num_physical_nodes > 1) { |
| 2624 | mutex_lock(&mgpu_info.mutex); | 2628 | mutex_lock(&mgpu_info.mutex); |
| @@ -2657,6 +2661,36 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) · 32 unmodified lines | |||
| 2657 | return 0; | 2661 | return 0; |
| 2658 | } | 2662 | } |
| 2659 | 2663 | ||
| 2664 | /** | ||
| 2665 | * amdgpu_device_smu_fini_early - smu hw_fini wrapper | ||
| 2666 | * | ||
| 2667 | * @adev: amdgpu_device pointer | ||
| 2668 | * | ||
| 2669 | * For ASICs need to disable SMC first | ||
| 2670 | */ | ||
| 2671 | static void amdgpu_device_smu_fini_early(struct amdgpu_device *adev) | ||
| 2672 | { | ||
| 2673 | int i, r; | ||
| 2674 | |||
| 2675 | if (adev->ip_versions[GC_HWIP][0] > IP_VERSION(9, 0, 0)) | ||
| 2676 | return; | ||
| 2677 | |||
| 2678 | for (i = 0; i < adev->num_ip_blocks; i++) { | ||
| 2679 | if (!adev->ip_blocks[i].status.hw) | ||
| 2680 | continue; | ||
| 2681 | if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_SMC) { | ||
| 2682 | r = adev->ip_blocks[i].version->funcs->hw_fini((void *)adev); | ||
| 2683 | /* XXX handle errors */ | ||
| 2684 | if (r) { | ||
| 2685 | DRM_DEBUG("hw_fini of IP block <%s> failed %d\n", | ||
| 2686 | adev->ip_blocks[i].version->funcs->name, r); | ||
| 2687 | } | ||
| 2688 | adev->ip_blocks[i].status.hw = false; | ||
| 2689 | break; | ||
| 2690 | } | ||
| 2691 | } | ||
| 2692 | } | ||
| 2693 | |||
| 2660 | static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) | 2694 | static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) |
| 2661 | { | 2695 | { |
| 2662 | int i, r; | 2696 | int i, r; |
| @@ -2677,21 +2711,8 @@ static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) · 14 unmodified lines | |||
| 2677 | amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE); | 2711 | amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE); |
| 2678 | amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE); | 2712 | amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE); |
| 2679 | 2713 | ||
| 2680 | /* need to disable SMC first */ | 2714 | /* Workaroud for ASICs need to disable SMC first */ |
| 2681 | for (i = 0; i < adev->num_ip_blocks; i++) { | 2715 | amdgpu_device_smu_fini_early(adev); |
| 2682 | if (!adev->ip_blocks[i].status.hw) | ||
| 2683 | continue; | ||
| 2684 | if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_SMC) { | ||
| 2685 | r = adev->ip_blocks[i].version->funcs->hw_fini((void *)adev); | ||
| 2686 | /* XXX handle errors */ | ||
| 2687 | if (r) { | ||
| 2688 | DRM_DEBUG("hw_fini of IP block <%s> failed %d\n", | ||
| 2689 | adev->ip_blocks[i].version->funcs->name, r); | ||
| 2690 | } | ||
| 2691 | adev->ip_blocks[i].status.hw = false; | ||
| 2692 | break; | ||
| 2693 | } | ||
| 2694 | } | ||
| 2695 | 2716 | ||
| 2696 | for (i = adev->num_ip_blocks - 1; i >= 0; i--) { | 2717 | for (i = adev->num_ip_blocks - 1; i >= 0; i--) { |
| 2697 | if (!adev->ip_blocks[i].status.hw) | 2718 | if (!adev->ip_blocks[i].status.hw) |
| @@ -2733,8 +2754,6 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev) · 35 unmodified lines | |||
| 2733 | if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done) | 2754 | if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done) |
| 2734 | amdgpu_virt_release_ras_err_handler_data(adev); | 2755 | amdgpu_virt_release_ras_err_handler_data(adev); |
| 2735 | 2756 | ||
| 2736 | amdgpu_ras_pre_fini(adev); | ||
| 2737 | |||
| 2738 | if (adev->gmc.xgmi.num_physical_nodes > 1) | 2757 | if (adev->gmc.xgmi.num_physical_nodes > 1) |
| 2739 | amdgpu_xgmi_remove_device(adev); | 2758 | amdgpu_xgmi_remove_device(adev); |
| 2740 | 2759 | ||
| @@ -3373,6 +3392,22 @@ static int amdgpu_device_get_job_timeout_settings(struct amdgpu_device *adev) · 632 unmodified lines | |||
| 3373 | return ret; | 3392 | return ret; |
| 3374 | } | 3393 | } |
| 3375 | 3394 | ||
| 3395 | /** | ||
| 3396 | * amdgpu_device_check_iommu_direct_map - check if RAM direct mapped to GPU | ||
| 3397 | * | ||
| 3398 | * @adev: amdgpu_device pointer | ||
| 3399 | * | ||
| 3400 | * RAM direct mapped to GPU if IOMMU is not enabled or is pass through mode | ||
| 3401 | */ | ||
| 3402 | static void amdgpu_device_check_iommu_direct_map(struct amdgpu_device *adev) | ||
| 3403 | { | ||
| 3404 | struct iommu_domain *domain; | ||
| 3405 | |||
| 3406 | domain = iommu_get_domain_for_dev(adev->dev); | ||
| 3407 | if (!domain || domain->type == IOMMU_DOMAIN_IDENTITY) | ||
| 3408 | adev->ram_is_direct_mapped = true; | ||
| 3409 | } | ||
| 3410 | |||
| 3376 | static const struct attribute *amdgpu_dev_attributes[] = { | 3411 | static const struct attribute *amdgpu_dev_attributes[] = { |
| 3377 | &dev_attr_product_name.attr, | 3412 | &dev_attr_product_name.attr, |
| 3378 | &dev_attr_product_number.attr, | 3413 | &dev_attr_product_number.attr, |
| @@ -3547,6 +3582,13 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 168 unmodified lines | |||
| 3547 | if (r) | 3582 | if (r) |
| 3548 | return r; | 3583 | return r; |
| 3549 | 3584 | ||
| 3585 | /* Need to get xgmi info early to decide the reset behavior*/ | ||
| 3586 | if (adev->gmc.xgmi.supported) { | ||
| 3587 | r = adev->gfxhub.funcs->get_xgmi_info(adev); | ||
| 3588 | if (r) | ||
| 3589 | return r; | ||
| 3590 | } | ||
| 3591 | |||
| 3550 | /* enable PCIE atomic ops */ | 3592 | /* enable PCIE atomic ops */ |
| 3551 | if (amdgpu_sriov_vf(adev)) | 3593 | if (amdgpu_sriov_vf(adev)) |
| 3552 | adev->have_atomics_support = ((struct amd_sriov_msg_pf2vf_info *) | 3594 | adev->have_atomics_support = ((struct amd_sriov_msg_pf2vf_info *) |
| @@ -3693,8 +3735,6 @@ fence_driver_init: · 140 unmodified lines | |||
| 3693 | /* Get a log2 for easy divisions. */ | 3735 | /* Get a log2 for easy divisions. */ |
| 3694 | adev->mm_stats.log2_max_MBps = ilog2(max(1u, max_MBps)); | 3736 | adev->mm_stats.log2_max_MBps = ilog2(max(1u, max_MBps)); |
| 3695 | 3737 | ||
| 3696 | amdgpu_fbdev_init(adev); | ||
| 3697 | |||
| 3698 | r = amdgpu_pm_sysfs_init(adev); | 3738 | r = amdgpu_pm_sysfs_init(adev); |
| 3699 | if (r) { | 3739 | if (r) { |
| 3700 | adev->pm_sysfs_en = false; | 3740 | adev->pm_sysfs_en = false; |
| @@ -3778,6 +3818,8 @@ fence_driver_init: · 77 unmodified lines | |||
| 3778 | queue_delayed_work(system_wq, &mgpu_info.delayed_reset_work, | 3818 | queue_delayed_work(system_wq, &mgpu_info.delayed_reset_work, |
| 3779 | msecs_to_jiffies(AMDGPU_RESUME_MS)); | 3819 | msecs_to_jiffies(AMDGPU_RESUME_MS)); |
| 3780 | 3820 | ||
| 3821 | amdgpu_device_check_iommu_direct_map(adev); | ||
| 3822 | |||
| 3781 | return 0; | 3823 | return 0; |
| 3782 | 3824 | ||
| 3783 | release_ras_con: | 3825 | release_ras_con: |
| @@ -3811,7 +3853,7 @@ static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) · 27 unmodified lines | |||
| 3811 | } | 3853 | } |
| 3812 | 3854 | ||
| 3813 | /** | 3855 | /** |
| 3814 | * amdgpu_device_fini - tear down the driver | 3856 | * amdgpu_device_fini_hw - tear down the driver |
| 3815 | * | 3857 | * |
| 3816 | * @adev: amdgpu_device pointer | 3858 | * @adev: amdgpu_device pointer |
| 3817 | * | 3859 | * |
| @@ -3852,17 +3894,21 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) · 34 unmodified lines | |||
| 3852 | amdgpu_ucode_sysfs_fini(adev); | 3894 | amdgpu_ucode_sysfs_fini(adev); |
| 3853 | sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); | 3895 | sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); |
| 3854 | 3896 | ||
| 3855 | amdgpu_fbdev_fini(adev); | 3897 | /* disable ras feature must before hw fini */ |
| 3898 | amdgpu_ras_pre_fini(adev); | ||
| 3856 | 3899 | ||
| 3857 | amdgpu_device_ip_fini_early(adev); | 3900 | amdgpu_device_ip_fini_early(adev); |
| 3858 | 3901 | ||
| 3859 | amdgpu_irq_fini_hw(adev); | 3902 | amdgpu_irq_fini_hw(adev); |
| 3860 | 3903 | ||
| 3861 | ttm_device_clear_dma_mappings(&adev->mman.bdev); | 3904 | if (adev->mman.initialized) |
| 3905 | ttm_device_clear_dma_mappings(&adev->mman.bdev); | ||
| 3862 | 3906 | ||
| 3863 | amdgpu_gart_dummy_page_fini(adev); | 3907 | amdgpu_gart_dummy_page_fini(adev); |
| 3864 | 3908 | ||
| 3865 | amdgpu_device_unmap_mmio(adev); | 3909 | if (drm_dev_is_unplugged(adev_to_drm(adev))) |
| 3910 | amdgpu_device_unmap_mmio(adev); | ||
| 3911 | |||
| 3866 | } | 3912 | } |
| 3867 | 3913 | ||
| 3868 | void amdgpu_device_fini_sw(struct amdgpu_device *adev) | 3914 | void amdgpu_device_fini_sw(struct amdgpu_device *adev) |
| @@ -3948,7 +3994,7 @@ int amdgpu_device_suspend(struct drm_device *dev, bool fbcon) · 79 unmodified lines | |||
| 3948 | drm_kms_helper_poll_disable(dev); | 3994 | drm_kms_helper_poll_disable(dev); |
| 3949 | 3995 | ||
| 3950 | if (fbcon) | 3996 | if (fbcon) |
| 3951 | amdgpu_fbdev_set_suspend(adev, 1); | 3997 | drm_fb_helper_set_suspend_unlocked(adev_to_drm(adev)->fb_helper, true); |
| 3952 | 3998 | ||
| 3953 | cancel_delayed_work_sync(&adev->delayed_init_work); | 3999 | cancel_delayed_work_sync(&adev->delayed_init_work); |
| 3954 | 4000 | ||
| @@ -4025,7 +4071,7 @@ int amdgpu_device_resume(struct drm_device *dev, bool fbcon) · 70 unmodified lines | |||
| 4025 | flush_delayed_work(&adev->delayed_init_work); | 4071 | flush_delayed_work(&adev->delayed_init_work); |
| 4026 | 4072 | ||
| 4027 | if (fbcon) | 4073 | if (fbcon) |
| 4028 | amdgpu_fbdev_set_suspend(adev, 0); | 4074 | drm_fb_helper_set_suspend_unlocked(adev_to_drm(adev)->fb_helper, false); |
| 4029 | 4075 | ||
| 4030 | drm_kms_helper_poll_enable(dev); | 4076 | drm_kms_helper_poll_enable(dev); |
| 4031 | 4077 | ||
| @@ -4294,6 +4340,9 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev, · 262 unmodified lines | |||
| 4294 | bool from_hypervisor) | 4340 | bool from_hypervisor) |
| 4295 | { | 4341 | { |
| 4296 | int r; | 4342 | int r; |
| 4343 | struct amdgpu_hive_info *hive = NULL; | ||
| 4344 | |||
| 4345 | amdgpu_amdkfd_pre_reset(adev); | ||
| 4297 | 4346 | ||
| 4298 | amdgpu_amdkfd_pre_reset(adev); | 4347 | amdgpu_amdkfd_pre_reset(adev); |
| 4299 | 4348 | ||
| @@ -4322,9 +4371,19 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev, · 22 unmodified lines | |||
| 4322 | if (r) | 4371 | if (r) |
| 4323 | goto error; | 4372 | goto error; |
| 4324 | 4373 | ||
| 4325 | amdgpu_irq_gpu_reset_resume_helper(adev); | 4374 | hive = amdgpu_get_xgmi_hive(adev); |
| 4326 | r = amdgpu_ib_ring_tests(adev); | 4375 | /* Update PSP FW topology after reset */ |
| 4327 | amdgpu_amdkfd_post_reset(adev); | 4376 | if (hive && adev->gmc.xgmi.num_physical_nodes > 1) |
| 4377 | r = amdgpu_xgmi_update_topology(hive, adev); | ||
| 4378 | |||
| 4379 | if (hive) | ||
| 4380 | amdgpu_put_xgmi_hive(hive); | ||
| 4381 | |||
| 4382 | if (!r) { | ||
| 4383 | amdgpu_irq_gpu_reset_resume_helper(adev); | ||
| 4384 | r = amdgpu_ib_ring_tests(adev); | ||
| 4385 | amdgpu_amdkfd_post_reset(adev); | ||
| 4386 | } | ||
| 4328 | 4387 | ||
| 4329 | error: | 4388 | error: |
| 4330 | if (!r && adev->virt.gim_feature & AMDGIM_FEATURE_GIM_FLR_VRAMLOST) { | 4389 | if (!r && adev->virt.gim_feature & AMDGIM_FEATURE_GIM_FLR_VRAMLOST) { |
| @@ -4650,7 +4709,7 @@ int amdgpu_do_asic_reset(struct list_head *device_list_handle, · 319 unmodified lines | |||
| 4650 | if (r) | 4709 | if (r) |
| 4651 | goto out; | 4710 | goto out; |
| 4652 | 4711 | ||
| 4653 | amdgpu_fbdev_set_suspend(tmp_adev, 0); | 4712 | drm_fb_helper_set_suspend_unlocked(adev_to_drm(tmp_adev)->fb_helper, false); |
| 4654 | 4713 | ||
| 4655 | /* | 4714 | /* |
| 4656 | * The GPU enters bad state once faulty pages | 4715 | * The GPU enters bad state once faulty pages |
| @@ -4749,7 +4808,7 @@ static int amdgpu_device_lock_hive_adev(struct amdgpu_device *adev, struct amdgp · 92 unmodified lines | |||
| 4749 | { | 4808 | { |
| 4750 | struct amdgpu_device *tmp_adev = NULL; | 4809 | struct amdgpu_device *tmp_adev = NULL; |
| 4751 | 4810 | ||
| 4752 | if (adev->gmc.xgmi.num_physical_nodes > 1) { | 4811 | if (!amdgpu_sriov_vf(adev) && (adev->gmc.xgmi.num_physical_nodes > 1)) { |
| 4753 | if (!hive) { | 4812 | if (!hive) { |
| 4754 | dev_err(adev->dev, "Hive is NULL while device has multiple xgmi nodes"); | 4813 | dev_err(adev->dev, "Hive is NULL while device has multiple xgmi nodes"); |
| 4755 | return -ENODEV; | 4814 | return -ENODEV; |
| @@ -4961,7 +5020,8 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 205 unmodified lines | |||
| 4961 | * We always reset all schedulers for device and all devices for XGMI | 5020 | * We always reset all schedulers for device and all devices for XGMI |
| 4962 | * hive so that should take care of them too. | 5021 | * hive so that should take care of them too. |
| 4963 | */ | 5022 | */ |
| 4964 | hive = amdgpu_get_xgmi_hive(adev); | 5023 | if (!amdgpu_sriov_vf(adev)) |
| 5024 | hive = amdgpu_get_xgmi_hive(adev); | ||
| 4965 | if (hive) { | 5025 | if (hive) { |
| 4966 | if (atomic_cmpxchg(&hive->in_reset, 0, 1) != 0) { | 5026 | if (atomic_cmpxchg(&hive->in_reset, 0, 1) != 0) { |
| 4967 | DRM_INFO("Bailing on TDR for s_job:%llx, hive: %llx as another already in progress", | 5027 | DRM_INFO("Bailing on TDR for s_job:%llx, hive: %llx as another already in progress", |
| @@ -5002,7 +5062,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 34 unmodified lines | |||
| 5002 | * to put adev in the 1st position. | 5062 | * to put adev in the 1st position. |
| 5003 | */ | 5063 | */ |
| 5004 | INIT_LIST_HEAD(&device_list); | 5064 | INIT_LIST_HEAD(&device_list); |
| 5005 | if (adev->gmc.xgmi.num_physical_nodes > 1) { | 5065 | if (!amdgpu_sriov_vf(adev) && (adev->gmc.xgmi.num_physical_nodes > 1)) { |
| 5006 | list_for_each_entry(tmp_adev, &hive->device_list, gmc.xgmi.head) | 5066 | list_for_each_entry(tmp_adev, &hive->device_list, gmc.xgmi.head) |
| 5007 | list_add_tail(&tmp_adev->reset_list, &device_list); | 5067 | list_add_tail(&tmp_adev->reset_list, &device_list); |
| 5008 | if (!list_is_first(&adev->reset_list, &device_list)) | 5068 | if (!list_is_first(&adev->reset_list, &device_list)) |
| @@ -5041,7 +5101,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 32 unmodified lines | |||
| 5041 | */ | 5101 | */ |
| 5042 | amdgpu_unregister_gpu_instance(tmp_adev); | 5102 | amdgpu_unregister_gpu_instance(tmp_adev); |
| 5043 | 5103 | ||
| 5044 | amdgpu_fbdev_set_suspend(tmp_adev, 1); | 5104 | drm_fb_helper_set_suspend_unlocked(adev_to_drm(adev)->fb_helper, true); |
| 5045 | 5105 | ||
| 5046 | /* disable ras on ALL IPs */ | 5106 | /* disable ras on ALL IPs */ |
| 5047 | if (!need_emergency_restart && | 5107 | if (!need_emergency_restart && |
| @@ -5636,3 +5696,42 @@ void amdgpu_device_invalidate_hdp(struct amdgpu_device *adev, · 588 unmodified lines | |||
| 5636 | 5696 | ||
| 5637 | amdgpu_asic_invalidate_hdp(adev, ring); | 5697 | amdgpu_asic_invalidate_hdp(adev, ring); |
| 5638 | } | 5698 | } |
| 5699 | |||
| 5700 | /** | ||
| 5701 | * amdgpu_device_halt() - bring hardware to some kind of halt state | ||
| 5702 | * | ||
| 5703 | * @adev: amdgpu_device pointer | ||
| 5704 | * | ||
| 5705 | * Bring hardware to some kind of halt state so that no one can touch it | ||
| 5706 | * any more. It will help to maintain error context when error occurred. | ||
| 5707 | * Compare to a simple hang, the system will keep stable at least for SSH | ||
| 5708 | * access. Then it should be trivial to inspect the hardware state and | ||
| 5709 | * see what's going on. Implemented as following: | ||
| 5710 | * | ||
| 5711 | * 1. drm_dev_unplug() makes device inaccessible to user space(IOCTLs, etc), | ||
| 5712 | * clears all CPU mappings to device, disallows remappings through page faults | ||
| 5713 | * 2. amdgpu_irq_disable_all() disables all interrupts | ||
| 5714 | * 3. amdgpu_fence_driver_hw_fini() signals all HW fences | ||
| 5715 | * 4. set adev->no_hw_access to avoid potential crashes after setp 5 | ||
| 5716 | * 5. amdgpu_device_unmap_mmio() clears all MMIO mappings | ||
| 5717 | * 6. pci_disable_device() and pci_wait_for_pending_transaction() | ||
| 5718 | * flush any in flight DMA operations | ||
| 5719 | */ | ||
| 5720 | void amdgpu_device_halt(struct amdgpu_device *adev) | ||
| 5721 | { | ||
| 5722 | struct pci_dev *pdev = adev->pdev; | ||
| 5723 | struct drm_device *ddev = adev_to_drm(adev); | ||
| 5724 | |||
| 5725 | drm_dev_unplug(ddev); | ||
| 5726 | |||
| 5727 | amdgpu_irq_disable_all(adev); | ||
| 5728 | |||
| 5729 | amdgpu_fence_driver_hw_fini(adev); | ||
| 5730 | |||
| 5731 | adev->no_hw_access = true; | ||
| 5732 | |||
| 5733 | amdgpu_device_unmap_mmio(adev); | ||
| 5734 | |||
| 5735 | pci_disable_device(pdev); | ||
| 5736 | pci_wait_for_pending_transaction(pdev); | ||
| 5737 | } | ||
| @@ -67,7 +67,8 @@ · 66 unmodified lines | |||
| 67 | #include "smuio_v11_0_6.h" | 67 | #include "smuio_v11_0_6.h" |
| 68 | #include "smuio_v13_0.h" | 68 | #include "smuio_v13_0.h" |
| 69 | 69 | ||
| 70 | MODULE_FIRMWARE("amdgpu/ip_discovery.bin"); | 70 | #define FIRMWARE_IP_DISCOVERY "amdgpu/ip_discovery.bin" |
| 71 | MODULE_FIRMWARE(FIRMWARE_IP_DISCOVERY); | ||
| 71 | 72 | ||
| 72 | #define mmRCC_CONFIG_MEMSIZE 0xde3 | 73 | #define mmRCC_CONFIG_MEMSIZE 0xde3 |
| 73 | #define mmMM_INDEX 0x0 | 74 | #define mmMM_INDEX 0x0 |
| @@ -179,7 +180,7 @@ static int hw_id_map[MAX_HWIP] = { · 105 unmodified lines | |||
| 179 | [DCI_HWIP] = DCI_HWID, | 180 | [DCI_HWIP] = DCI_HWID, |
| 180 | }; | 181 | }; |
| 181 | 182 | ||
| 182 | static int amdgpu_discovery_read_binary(struct amdgpu_device *adev, uint8_t *binary) | 183 | static int amdgpu_discovery_read_binary_from_vram(struct amdgpu_device *adev, uint8_t *binary) |
| 183 | { | 184 | { |
| 184 | uint64_t vram_size = (uint64_t)RREG32(mmRCC_CONFIG_MEMSIZE) << 20; | 185 | uint64_t vram_size = (uint64_t)RREG32(mmRCC_CONFIG_MEMSIZE) << 20; |
| 185 | uint64_t pos = vram_size - DISCOVERY_TMR_OFFSET; | 186 | uint64_t pos = vram_size - DISCOVERY_TMR_OFFSET; |
| @@ -189,6 +190,34 @@ static int amdgpu_discovery_read_binary(struct amdgpu_device *adev, uint8_t *bin · 3 unmodified lines | |||
| 189 | return 0; | 190 | return 0; |
| 190 | } | 191 | } |
| 191 | 192 | ||
| 193 | static int amdgpu_discovery_read_binary_from_file(struct amdgpu_device *adev, uint8_t *binary) | ||
| 194 | { | ||
| 195 | const struct firmware *fw; | ||
| 196 | const char *fw_name; | ||
| 197 | int r; | ||
| 198 | |||
| 199 | switch (amdgpu_discovery) { | ||
| 200 | case 2: | ||
| 201 | fw_name = FIRMWARE_IP_DISCOVERY; | ||
| 202 | break; | ||
| 203 | default: | ||
| 204 | dev_warn(adev->dev, "amdgpu_discovery is not set properly\n"); | ||
| 205 | return -EINVAL; | ||
| 206 | } | ||
| 207 | |||
| 208 | r = request_firmware(&fw, fw_name, adev->dev); | ||
| 209 | if (r) { | ||
| 210 | dev_err(adev->dev, "can't load firmware \"%s\"\n", | ||
| 211 | fw_name); | ||
| 212 | return r; | ||
| 213 | } | ||
| 214 | |||
| 215 | memcpy((u8 *)binary, (u8 *)fw->data, adev->mman.discovery_tmr_size); | ||
| 216 | release_firmware(fw); | ||
| 217 | |||
| 218 | return 0; | ||
| 219 | } | ||
| 220 | |||
| 192 | static uint16_t amdgpu_discovery_calculate_checksum(uint8_t *data, uint32_t size) | 221 | static uint16_t amdgpu_discovery_calculate_checksum(uint8_t *data, uint32_t size) |
| 193 | { | 222 | { |
| 194 | uint16_t checksum = 0; | 223 | uint16_t checksum = 0; |
| @@ -206,13 +235,20 @@ static inline bool amdgpu_discovery_verify_checksum(uint8_t *data, uint32_t size · 11 unmodified lines | |||
| 206 | return !!(amdgpu_discovery_calculate_checksum(data, size) == expected); | 235 | return !!(amdgpu_discovery_calculate_checksum(data, size) == expected); |
| 207 | } | 236 | } |
| 208 | 237 | ||
| 238 | static inline bool amdgpu_discovery_verify_binary_signature(uint8_t *binary) | ||
| 239 | { | ||
| 240 | struct binary_header *bhdr; | ||
| 241 | bhdr = (struct binary_header *)binary; | ||
| 242 | |||
| 243 | return (le32_to_cpu(bhdr->binary_signature) == BINARY_SIGNATURE); | ||
| 244 | } | ||
| 245 | |||
| 209 | static int amdgpu_discovery_init(struct amdgpu_device *adev) | 246 | static int amdgpu_discovery_init(struct amdgpu_device *adev) |
| 210 | { | 247 | { |
| 211 | struct table_info *info; | 248 | struct table_info *info; |
| 212 | struct binary_header *bhdr; | 249 | struct binary_header *bhdr; |
| 213 | struct ip_discovery_header *ihdr; | 250 | struct ip_discovery_header *ihdr; |
| 214 | struct gpu_info_header *ghdr; | 251 | struct gpu_info_header *ghdr; |
| 215 | const struct firmware *fw; | ||
| 216 | uint16_t offset; | 252 | uint16_t offset; |
| 217 | uint16_t size; | 253 | uint16_t size; |
| 218 | uint16_t checksum; | 254 | uint16_t checksum; |
| @@ -223,31 +259,32 @@ static int amdgpu_discovery_init(struct amdgpu_device *adev) · 4 unmodified lines | |||
| 223 | if (!adev->mman.discovery_bin) | 259 | if (!adev->mman.discovery_bin) |
| 224 | return -ENOMEM; | 260 | return -ENOMEM; |
| 225 | 261 | ||
| 226 | if (amdgpu_discovery == 2) { | 262 | r = amdgpu_discovery_read_binary_from_vram(adev, adev->mman.discovery_bin); |
| 227 | r = request_firmware(&fw, "amdgpu/ip_discovery.bin", adev->dev); | 263 | if (r) { |
| 228 | if (r) | 264 | dev_err(adev->dev, "failed to read ip discovery binary from vram\n"); |
| 229 | goto get_from_vram; | 265 | r = -EINVAL; |
| 230 | dev_info(adev->dev, "Using IP discovery from file\n"); | 266 | goto out; |
| 231 | memcpy((u8 *)adev->mman.discovery_bin, (u8 *)fw->data, | 267 | } |
| 232 | adev->mman.discovery_tmr_size); | 268 | |
| 233 | release_firmware(fw); | 269 | if(!amdgpu_discovery_verify_binary_signature(adev->mman.discovery_bin)) { |
| 234 | } else { | 270 | dev_warn(adev->dev, "get invalid ip discovery binary signature from vram\n"); |
| 235 | get_from_vram: | 271 | /* retry read ip discovery binary from file */ |
| 236 | r = amdgpu_discovery_read_binary(adev, adev->mman.discovery_bin); | 272 | r = amdgpu_discovery_read_binary_from_file(adev, adev->mman.discovery_bin); |
| 237 | if (r) { | 273 | if (r) { |
| 238 | DRM_ERROR("failed to read ip discovery binary\n"); | 274 | dev_err(adev->dev, "failed to read ip discovery binary from file\n"); |
| 275 | r = -EINVAL; | ||
| 276 | goto out; | ||
| 277 | } | ||
| 278 | /* check the ip discovery binary signature */ | ||
| 279 | if(!amdgpu_discovery_verify_binary_signature(adev->mman.discovery_bin)) { | ||
| 280 | dev_warn(adev->dev, "get invalid ip discovery binary signature from file\n"); | ||
| 281 | r = -EINVAL; | ||
| 239 | goto out; | 282 | goto out; |
| 240 | } | 283 | } |
| 241 | } | 284 | } |
| 242 | 285 | ||
| 243 | bhdr = (struct binary_header *)adev->mman.discovery_bin; | 286 | bhdr = (struct binary_header *)adev->mman.discovery_bin; |
| 244 | 287 | ||
| 245 | if (le32_to_cpu(bhdr->binary_signature) != BINARY_SIGNATURE) { | ||
| 246 | DRM_ERROR("invalid ip discovery binary signature\n"); | ||
| 247 | r = -EINVAL; | ||
| 248 | goto out; | ||
| 249 | } | ||
| 250 | |||
| 251 | offset = offsetof(struct binary_header, binary_checksum) + | 288 | offset = offsetof(struct binary_header, binary_checksum) + |
| 252 | sizeof(bhdr->binary_checksum); | 289 | sizeof(bhdr->binary_checksum); |
| 253 | size = le16_to_cpu(bhdr->binary_size) - offset; | 290 | size = le16_to_cpu(bhdr->binary_size) - offset; |
| @@ -255,7 +292,7 @@ get_from_vram: · 1 unmodified line | |||
| 255 | 292 | ||
| 256 | if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset, | 293 | if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset, |
| 257 | size, checksum)) { | 294 | size, checksum)) { |
| 258 | DRM_ERROR("invalid ip discovery binary checksum\n"); | 295 | dev_err(adev->dev, "invalid ip discovery binary checksum\n"); |
| 259 | r = -EINVAL; | 296 | r = -EINVAL; |
| 260 | goto out; | 297 | goto out; |
| 261 | } | 298 | } |
| @@ -266,14 +303,14 @@ get_from_vram: · 4 unmodified lines | |||
| 266 | ihdr = (struct ip_discovery_header *)(adev->mman.discovery_bin + offset); | 303 | ihdr = (struct ip_discovery_header *)(adev->mman.discovery_bin + offset); |
| 267 | 304 | ||
| 268 | if (le32_to_cpu(ihdr->signature) != DISCOVERY_TABLE_SIGNATURE) { | 305 | if (le32_to_cpu(ihdr->signature) != DISCOVERY_TABLE_SIGNATURE) { |
| 269 | DRM_ERROR("invalid ip discovery data table signature\n"); | 306 | dev_err(adev->dev, "invalid ip discovery data table signature\n"); |
| 270 | r = -EINVAL; | 307 | r = -EINVAL; |
| 271 | goto out; | 308 | goto out; |
| 272 | } | 309 | } |
| 273 | 310 | ||
| 274 | if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset, | 311 | if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset, |
| 275 | le16_to_cpu(ihdr->size), checksum)) { | 312 | le16_to_cpu(ihdr->size), checksum)) { |
| 276 | DRM_ERROR("invalid ip discovery data table checksum\n"); | 313 | dev_err(adev->dev, "invalid ip discovery data table checksum\n"); |
| 277 | r = -EINVAL; | 314 | r = -EINVAL; |
| 278 | goto out; | 315 | goto out; |
| 279 | } | 316 | } |
| @@ -285,7 +322,7 @@ get_from_vram: · 5 unmodified lines | |||
| 285 | 322 | ||
| 286 | if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset, | 323 | if (!amdgpu_discovery_verify_checksum(adev->mman.discovery_bin + offset, |
| 287 | le32_to_cpu(ghdr->size), checksum)) { | 324 | le32_to_cpu(ghdr->size), checksum)) { |
| 288 | DRM_ERROR("invalid gc data table checksum\n"); | 325 | dev_err(adev->dev, "invalid gc data table checksum\n"); |
| 289 | r = -EINVAL; | 326 | r = -EINVAL; |
| 290 | goto out; | 327 | goto out; |
| 291 | } | 328 | } |
| @@ -379,8 +416,18 @@ int amdgpu_discovery_reg_base_init(struct amdgpu_device *adev) · 87 unmodified lines | |||
| 379 | ip->major, ip->minor, | 416 | ip->major, ip->minor, |
| 380 | ip->revision); | 417 | ip->revision); |
| 381 | 418 | ||
| 382 | if (le16_to_cpu(ip->hw_id) == VCN_HWID) | 419 | if (le16_to_cpu(ip->hw_id) == VCN_HWID) { |
| 420 | /* Bit [5:0]: original revision value | ||
| 421 | * Bit [7:6]: en/decode capability: | ||
| 422 | * 0b00 : VCN function normally | ||
| 423 | * 0b10 : encode is disabled | ||
| 424 | * 0b01 : decode is disabled | ||
| 425 | */ | ||
| 426 | adev->vcn.vcn_config[adev->vcn.num_vcn_inst] = | ||
| 427 | ip->revision & 0xc0; | ||
| 428 | ip->revision &= ~0xc0; | ||
| 383 | adev->vcn.num_vcn_inst++; | 429 | adev->vcn.num_vcn_inst++; |
| 430 | } | ||
| 384 | if (le16_to_cpu(ip->hw_id) == SDMA0_HWID || | 431 | if (le16_to_cpu(ip->hw_id) == SDMA0_HWID || |
| 385 | le16_to_cpu(ip->hw_id) == SDMA1_HWID || | 432 | le16_to_cpu(ip->hw_id) == SDMA1_HWID || |
| 386 | le16_to_cpu(ip->hw_id) == SDMA2_HWID || | 433 | le16_to_cpu(ip->hw_id) == SDMA2_HWID || |
| @@ -472,14 +519,6 @@ int amdgpu_discovery_get_ip_version(struct amdgpu_device *adev, int hw_id, int n · 85 unmodified lines | |||
| 472 | return -EINVAL; | 519 | return -EINVAL; |
| 473 | } | 520 | } |
| 474 | 521 | ||
| 475 | |||
| 476 | int amdgpu_discovery_get_vcn_version(struct amdgpu_device *adev, int vcn_instance, | ||
| 477 | int *major, int *minor, int *revision) | ||
| 478 | { | ||
| 479 | return amdgpu_discovery_get_ip_version(adev, VCN_HWID, | ||
| 480 | vcn_instance, major, minor, revision); | ||
| 481 | } | ||
| 482 | |||
| 483 | void amdgpu_discovery_harvest_ip(struct amdgpu_device *adev) | 522 | void amdgpu_discovery_harvest_ip(struct amdgpu_device *adev) |
| 484 | { | 523 | { |
| 485 | struct binary_header *bhdr; | 524 | struct binary_header *bhdr; |
| @@ -949,7 +988,6 @@ static int amdgpu_discovery_set_mm_ip_blocks(struct amdgpu_device *adev) · 463 unmodified lines | |||
| 949 | break; | 988 | break; |
| 950 | case IP_VERSION(3, 0, 0): | 989 | case IP_VERSION(3, 0, 0): |
| 951 | case IP_VERSION(3, 0, 16): | 990 | case IP_VERSION(3, 0, 16): |
| 952 | case IP_VERSION(3, 0, 64): | ||
| 953 | case IP_VERSION(3, 1, 1): | 991 | case IP_VERSION(3, 1, 1): |
| 954 | case IP_VERSION(3, 0, 2): | 992 | case IP_VERSION(3, 0, 2): |
| 955 | case IP_VERSION(3, 0, 192): | 993 | case IP_VERSION(3, 0, 192): |
| @@ -986,7 +1024,7 @@ static int amdgpu_discovery_set_mes_ip_blocks(struct amdgpu_device *adev) · 30 unmodified lines | |||
| 986 | amdgpu_device_ip_block_add(adev, &mes_v10_1_ip_block); | 1024 | amdgpu_device_ip_block_add(adev, &mes_v10_1_ip_block); |
| 987 | break; | 1025 | break; |
| 988 | default: | 1026 | default: |
| 989 | break;; | 1027 | break; |
| 990 | } | 1028 | } |
| 991 | return 0; | 1029 | return 0; |
| 992 | } | 1030 | } |
| @@ -33,8 +33,6 @@ void amdgpu_discovery_harvest_ip(struct amdgpu_device *adev); · 32 unmodified lines | |||
| 33 | int amdgpu_discovery_get_ip_version(struct amdgpu_device *adev, int hw_id, int number_instance, | 33 | int amdgpu_discovery_get_ip_version(struct amdgpu_device *adev, int hw_id, int number_instance, |
| 34 | int *major, int *minor, int *revision); | 34 | int *major, int *minor, int *revision); |
| 35 | 35 | ||
| 36 | int amdgpu_discovery_get_vcn_version(struct amdgpu_device *adev, int vcn_instance, | ||
| 37 | int *major, int *minor, int *revision); | ||
| 38 | int amdgpu_discovery_get_gfx_info(struct amdgpu_device *adev); | 36 | int amdgpu_discovery_get_gfx_info(struct amdgpu_device *adev); |
| 39 | int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev); | 37 | int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev); |
| 40 | 38 | ||
| @@ -83,9 +83,6 @@ static void amdgpu_display_flip_work_func(struct work_struct *__work) · 82 unmodified lines | |||
| 83 | unsigned i; | 83 | unsigned i; |
| 84 | int vpos, hpos; | 84 | int vpos, hpos; |
| 85 | 85 | ||
| 86 | if (amdgpu_display_flip_handle_fence(work, &work->excl)) | ||
| 87 | return; | ||
| 88 | |||
| 89 | for (i = 0; i < work->shared_count; ++i) | 86 | for (i = 0; i < work->shared_count; ++i) |
| 90 | if (amdgpu_display_flip_handle_fence(work, &work->shared[i])) | 87 | if (amdgpu_display_flip_handle_fence(work, &work->shared[i])) |
| 91 | return; | 88 | return; |
| @@ -203,7 +200,7 @@ int amdgpu_display_crtc_page_flip_target(struct drm_crtc *crtc, · 111 unmodified lines | |||
| 203 | goto unpin; | 200 | goto unpin; |
| 204 | } | 201 | } |
| 205 | 202 | ||
| 206 | r = dma_resv_get_fences(new_abo->tbo.base.resv, &work->excl, | 203 | r = dma_resv_get_fences(new_abo->tbo.base.resv, NULL, |
| 207 | &work->shared_count, &work->shared); | 204 | &work->shared_count, &work->shared); |
| 208 | if (unlikely(r != 0)) { | 205 | if (unlikely(r != 0)) { |
| 209 | DRM_ERROR("failed to get fences for buffer\n"); | 206 | DRM_ERROR("failed to get fences for buffer\n"); |
| @@ -253,7 +250,6 @@ unreserve: · 43 unmodified lines | |||
| 253 | 250 | ||
| 254 | cleanup: | 251 | cleanup: |
| 255 | amdgpu_bo_unref(&work->old_abo); | 252 | amdgpu_bo_unref(&work->old_abo); |
| 256 | dma_fence_put(work->excl); | ||
| 257 | for (i = 0; i < work->shared_count; ++i) | 253 | for (i = 0; i < work->shared_count; ++i) |
| 258 | dma_fence_put(work->shared[i]); | 254 | dma_fence_put(work->shared[i]); |
| 259 | kfree(work->shared); | 255 | kfree(work->shared); |
| @@ -1364,7 +1360,7 @@ bool amdgpu_display_crtc_scaling_mode_fixup(struct drm_crtc *crtc, · 1104 unmodified lines | |||
| 1364 | if ((!(mode->flags & DRM_MODE_FLAG_INTERLACE)) && | 1360 | if ((!(mode->flags & DRM_MODE_FLAG_INTERLACE)) && |
| 1365 | ((amdgpu_encoder->underscan_type == UNDERSCAN_ON) || | 1361 | ((amdgpu_encoder->underscan_type == UNDERSCAN_ON) || |
| 1366 | ((amdgpu_encoder->underscan_type == UNDERSCAN_AUTO) && | 1362 | ((amdgpu_encoder->underscan_type == UNDERSCAN_AUTO) && |
| 1367 | drm_detect_hdmi_monitor(amdgpu_connector_edid(connector)) && | 1363 | connector->display_info.is_hdmi && |
| 1368 | amdgpu_display_is_hdtv_mode(mode)))) { | 1364 | amdgpu_display_is_hdtv_mode(mode)))) { |
| 1369 | if (amdgpu_encoder->underscan_hborder != 0) | 1365 | if (amdgpu_encoder->underscan_hborder != 0) |
| 1370 | amdgpu_crtc->h_border = amdgpu_encoder->underscan_hborder; | 1366 | amdgpu_crtc->h_border = amdgpu_encoder->underscan_hborder; |
| @@ -1603,13 +1599,10 @@ int amdgpu_display_suspend_helper(struct amdgpu_device *adev) · 232 unmodified lines | |||
| 1603 | continue; | 1599 | continue; |
| 1604 | } | 1600 | } |
| 1605 | robj = gem_to_amdgpu_bo(fb->obj[0]); | 1601 | robj = gem_to_amdgpu_bo(fb->obj[0]); |
| 1606 | /* don't unpin kernel fb objects */ | 1602 | r = amdgpu_bo_reserve(robj, true); |
| 1607 | if (!amdgpu_fbdev_robj_is_fb(adev, robj)) { | 1603 | if (r == 0) { |
| 1608 | r = amdgpu_bo_reserve(robj, true); | 1604 | amdgpu_bo_unpin(robj); |
| 1609 | if (r == 0) { | 1605 | amdgpu_bo_unreserve(robj); |
| 1610 | amdgpu_bo_unpin(robj); | ||
| 1611 | amdgpu_bo_unreserve(robj); | ||
| 1612 | } | ||
| 1613 | } | 1606 | } |
| 1614 | } | 1607 | } |
| 1615 | return 0; | 1608 | return 0; |
| @@ -61,9 +61,6 @@ static int amdgpu_dma_buf_attach(struct dma_buf *dmabuf, · 60 unmodified lines | |||
| 61 | if (pci_p2pdma_distance_many(adev->pdev, &attach->dev, 1, true) < 0) | 61 | if (pci_p2pdma_distance_many(adev->pdev, &attach->dev, 1, true) < 0) |
| 62 | attach->peer2peer = false; | 62 | attach->peer2peer = false; |
| 63 | 63 | ||
| 64 | if (attach->dev->driver == adev->dev->driver) | ||
| 65 | return 0; | ||
| 66 | |||
| 67 | r = pm_runtime_get_sync(adev_to_drm(adev)->dev); | 64 | r = pm_runtime_get_sync(adev_to_drm(adev)->dev); |
| 68 | if (r < 0) | 65 | if (r < 0) |
| 69 | goto out; | 66 | goto out; |
| @@ -31,7 +31,6 @@ · 30 unmodified lines | |||
| 31 | #include "amdgpu_drv.h" | 31 | #include "amdgpu_drv.h" |
| 32 | 32 | ||
| 33 | #include <drm/drm_pciids.h> | 33 | #include <drm/drm_pciids.h> |
| 34 | #include <linux/console.h> | ||
| 35 | #include <linux/module.h> | 34 | #include <linux/module.h> |
| 36 | #include <linux/pm_runtime.h> | 35 | #include <linux/pm_runtime.h> |
| 37 | #include <linux/vga_switcheroo.h> | 36 | #include <linux/vga_switcheroo.h> |
| @@ -315,9 +314,12 @@ module_param_named(dpm, amdgpu_dpm, int, 0444); · 277 unmodified lines | |||
| 315 | 314 | ||
| 316 | /** | 315 | /** |
| 317 | * DOC: fw_load_type (int) | 316 | * DOC: fw_load_type (int) |
| 318 | * Set different firmware loading type for debugging (0 = direct, 1 = SMU, 2 = PSP). The default is -1 (auto). | 317 | * Set different firmware loading type for debugging, if supported. |
| 318 | * Set to 0 to force direct loading if supported by the ASIC. Set | ||
| 319 | * to -1 to select the default loading mode for the ASIC, as defined | ||
| 320 | * by the driver. The default is -1 (auto). | ||
| 319 | */ | 321 | */ |
| 320 | MODULE_PARM_DESC(fw_load_type, "firmware loading type (0 = direct, 1 = SMU, 2 = PSP, -1 = auto)"); | 322 | MODULE_PARM_DESC(fw_load_type, "firmware loading type (0 = force direct if supported, -1 = auto)"); |
| 321 | module_param_named(fw_load_type, amdgpu_fw_load_type, int, 0444); | 323 | module_param_named(fw_load_type, amdgpu_fw_load_type, int, 0444); |
| 322 | 324 | ||
| 323 | /** | 325 | /** |
| @@ -2031,6 +2033,19 @@ retry_init: · 1707 unmodified lines | |||
| 2031 | goto err_pci; | 2033 | goto err_pci; |
| 2032 | } | 2034 | } |
| 2033 | 2035 | ||
| 2036 | /* | ||
| 2037 | * 1. don't init fbdev on hw without DCE | ||
| 2038 | * 2. don't init fbdev if there are no connectors | ||
| 2039 | */ | ||
| 2040 | if (adev->mode_info.mode_config_initialized && | ||
| 2041 | !list_empty(&adev_to_drm(adev)->mode_config.connector_list)) { | ||
| 2042 | /* select 8 bpp console on low vram cards */ | ||
| 2043 | if (adev->gmc.real_vram_size <= (32*1024*1024)) | ||
| 2044 | drm_fbdev_generic_setup(adev_to_drm(adev), 8); | ||
| 2045 | else | ||
| 2046 | drm_fbdev_generic_setup(adev_to_drm(adev), 32); | ||
| 2047 | } | ||
| 2048 | |||
| 2034 | ret = amdgpu_debugfs_init(adev); | 2049 | ret = amdgpu_debugfs_init(adev); |
| 2035 | if (ret) | 2050 | if (ret) |
| 2036 | DRM_ERROR("Creating debugfs files failed (%d).\n", ret); | 2051 | DRM_ERROR("Creating debugfs files failed (%d).\n", ret); |
| @@ -2563,10 +2578,8 @@ static int __init amdgpu_init(void) · 526 unmodified lines | |||
| 2563 | { | 2578 | { |
| 2564 | int r; | 2579 | int r; |
| 2565 | 2580 | ||
| 2566 | if (vgacon_text_force()) { | 2581 | if (drm_firmware_drivers_only()) |
| 2567 | DRM_ERROR("VGACON disables amdgpu kernel modesetting.\n"); | ||
| 2568 | return -EINVAL; | 2582 | return -EINVAL; |
| 2569 | } | ||
| 2570 | 2583 | ||
| 2571 | r = amdgpu_sync_init(); | 2584 | r = amdgpu_sync_init(); |
| 2572 | if (r) | 2585 | if (r) |
| @@ -45,4 +45,7 @@ · 44 unmodified lines | |||
| 45 | long amdgpu_drm_ioctl(struct file *filp, | 45 | long amdgpu_drm_ioctl(struct file *filp, |
| 46 | unsigned int cmd, unsigned long arg); | 46 | unsigned int cmd, unsigned long arg); |
| 47 | 47 | ||
| 48 | long amdgpu_kms_compat_ioctl(struct file *filp, | ||
| 49 | unsigned int cmd, unsigned long arg); | ||
| 50 | |||
| 48 | #endif | 51 | #endif |
| @@ -222,7 +222,7 @@ bool amdgpu_dig_monitor_is_duallink(struct drm_encoder *encoder, · 221 unmodified lines | |||
| 222 | case DRM_MODE_CONNECTOR_HDMIB: | 222 | case DRM_MODE_CONNECTOR_HDMIB: |
| 223 | if (amdgpu_connector->use_digital) { | 223 | if (amdgpu_connector->use_digital) { |
| 224 | /* HDMI 1.3 supports up to 340 Mhz over single link */ | 224 | /* HDMI 1.3 supports up to 340 Mhz over single link */ |
| 225 | if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) { | 225 | if (connector->display_info.is_hdmi) { |
| 226 | if (pixel_clock > 340000) | 226 | if (pixel_clock > 340000) |
| 227 | return true; | 227 | return true; |
| 228 | else | 228 | else |
| @@ -244,7 +244,7 @@ bool amdgpu_dig_monitor_is_duallink(struct drm_encoder *encoder, · 15 unmodified lines | |||
| 244 | return false; | 244 | return false; |
| 245 | else { | 245 | else { |
| 246 | /* HDMI 1.3 supports up to 340 Mhz over single link */ | 246 | /* HDMI 1.3 supports up to 340 Mhz over single link */ |
| 247 | if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector))) { | 247 | if (connector->display_info.is_hdmi) { |
| 248 | if (pixel_clock > 340000) | 248 | if (pixel_clock > 340000) |
| 249 | return true; | 249 | return true; |
| 250 | else | 250 | else |
| @@ -1,388 +0,0 @@ | |||
| 1 | /* | ||
| 2 | * Copyright © 2007 David Airlie | ||
| 3 | * | ||
| 4 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 5 | * copy of this software and associated documentation files (the "Software"), | ||
| 6 | * to deal in the Software without restriction, including without limitation | ||
| 7 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 8 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 9 | * Software is furnished to do so, subject to the following conditions: | ||
| 10 | * | ||
| 11 | * The above copyright notice and this permission notice (including the next | ||
| 12 | * paragraph) shall be included in all copies or substantial portions of the | ||
| 13 | * Software. | ||
| 14 | * | ||
| 15 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 16 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 17 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 18 | * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| 19 | * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING | ||
| 20 | * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER | ||
| 21 | * DEALINGS IN THE SOFTWARE. | ||
| 22 | * | ||
| 23 | * Authors: | ||
| 24 | * David Airlie | ||
| 25 | */ | ||
| 26 | |||
| 27 | #include <linux/module.h> | ||
| 28 | #include <linux/pm_runtime.h> | ||
| 29 | #include <linux/slab.h> | ||
| 30 | #include <linux/vga_switcheroo.h> | ||
| 31 | |||
| 32 | #include <drm/amdgpu_drm.h> | ||
| 33 | #include <drm/drm_crtc.h> | ||
| 34 | #include <drm/drm_crtc_helper.h> | ||
| 35 | #include <drm/drm_fb_helper.h> | ||
| 36 | #include <drm/drm_fourcc.h> | ||
| 37 | |||
| 38 | #include "amdgpu.h" | ||
| 39 | #include "cikd.h" | ||
| 40 | #include "amdgpu_gem.h" | ||
| 41 | |||
| 42 | #include "amdgpu_display.h" | ||
| 43 | |||
| 44 | /* object hierarchy - | ||
| 45 | this contains a helper + a amdgpu fb | ||
| 46 | the helper contains a pointer to amdgpu framebuffer baseclass. | ||
| 47 | */ | ||
| 48 | |||
| 49 | static int | ||
| 50 | amdgpufb_open(struct fb_info *info, int user) | ||
| 51 | { | ||
| 52 | struct drm_fb_helper *fb_helper = info->par; | ||
| 53 | int ret = pm_runtime_get_sync(fb_helper->dev->dev); | ||
| 54 | if (ret < 0 && ret != -EACCES) { | ||
| 55 | pm_runtime_mark_last_busy(fb_helper->dev->dev); | ||
| 56 | pm_runtime_put_autosuspend(fb_helper->dev->dev); | ||
| 57 | return ret; | ||
| 58 | } | ||
| 59 | return 0; | ||
| 60 | } | ||
| 61 | |||
| 62 | static int | ||
| 63 | amdgpufb_release(struct fb_info *info, int user) | ||
| 64 | { | ||
| 65 | struct drm_fb_helper *fb_helper = info->par; | ||
| 66 | |||
| 67 | pm_runtime_mark_last_busy(fb_helper->dev->dev); | ||
| 68 | pm_runtime_put_autosuspend(fb_helper->dev->dev); | ||
| 69 | return 0; | ||
| 70 | } | ||
| 71 | |||
| 72 | static const struct fb_ops amdgpufb_ops = { | ||
| 73 | .owner = THIS_MODULE, | ||
| 74 | DRM_FB_HELPER_DEFAULT_OPS, | ||
| 75 | .fb_open = amdgpufb_open, | ||
| 76 | .fb_release = amdgpufb_release, | ||
| 77 | .fb_fillrect = drm_fb_helper_cfb_fillrect, | ||
| 78 | .fb_copyarea = drm_fb_helper_cfb_copyarea, | ||
| 79 | .fb_imageblit = drm_fb_helper_cfb_imageblit, | ||
| 80 | }; | ||
| 81 | |||
| 82 | |||
| 83 | int amdgpu_align_pitch(struct amdgpu_device *adev, int width, int cpp, bool tiled) | ||
| 84 | { | ||
| 85 | int aligned = width; | ||
| 86 | int pitch_mask = 0; | ||
| 87 | |||
| 88 | switch (cpp) { | ||
| 89 | case 1: | ||
| 90 | pitch_mask = 255; | ||
| 91 | break; | ||
| 92 | case 2: | ||
| 93 | pitch_mask = 127; | ||
| 94 | break; | ||
| 95 | case 3: | ||
| 96 | case 4: | ||
| 97 | pitch_mask = 63; | ||
| 98 | break; | ||
| 99 | } | ||
| 100 | |||
| 101 | aligned += pitch_mask; | ||
| 102 | aligned &= ~pitch_mask; | ||
| 103 | return aligned * cpp; | ||
| 104 | } | ||
| 105 | |||
| 106 | static void amdgpufb_destroy_pinned_object(struct drm_gem_object *gobj) | ||
| 107 | { | ||
| 108 | struct amdgpu_bo *abo = gem_to_amdgpu_bo(gobj); | ||
| 109 | int ret; | ||
| 110 | |||
| 111 | ret = amdgpu_bo_reserve(abo, true); | ||
| 112 | if (likely(ret == 0)) { | ||
| 113 | amdgpu_bo_kunmap(abo); | ||
| 114 | amdgpu_bo_unpin(abo); | ||
| 115 | amdgpu_bo_unreserve(abo); | ||
| 116 | } | ||
| 117 | drm_gem_object_put(gobj); | ||
| 118 | } | ||
| 119 | |||
| 120 | static int amdgpufb_create_pinned_object(struct amdgpu_fbdev *rfbdev, | ||
| 121 | struct drm_mode_fb_cmd2 *mode_cmd, | ||
| 122 | struct drm_gem_object **gobj_p) | ||
| 123 | { | ||
| 124 | const struct drm_format_info *info; | ||
| 125 | struct amdgpu_device *adev = rfbdev->adev; | ||
| 126 | struct drm_gem_object *gobj = NULL; | ||
| 127 | struct amdgpu_bo *abo = NULL; | ||
| 128 | bool fb_tiled = false; /* useful for testing */ | ||
| 129 | u32 tiling_flags = 0, domain; | ||
| 130 | int ret; | ||
| 131 | int aligned_size, size; | ||
| 132 | int height = mode_cmd->height; | ||
| 133 | u32 cpp; | ||
| 134 | u64 flags = AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED | | ||
| 135 | AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS | | ||
| 136 | AMDGPU_GEM_CREATE_VRAM_CLEARED; | ||
| 137 | |||
| 138 | info = drm_get_format_info(adev_to_drm(adev), mode_cmd); | ||
| 139 | cpp = info->cpp[0]; | ||
| 140 | |||
| 141 | /* need to align pitch with crtc limits */ | ||
| 142 | mode_cmd->pitches[0] = amdgpu_align_pitch(adev, mode_cmd->width, cpp, | ||
| 143 | fb_tiled); | ||
| 144 | domain = amdgpu_display_supported_domains(adev, flags); | ||
| 145 | height = ALIGN(mode_cmd->height, 8); | ||
| 146 | size = mode_cmd->pitches[0] * height; | ||
| 147 | aligned_size = ALIGN(size, PAGE_SIZE); | ||
| 148 | ret = amdgpu_gem_object_create(adev, aligned_size, 0, domain, flags, | ||
| 149 | ttm_bo_type_device, NULL, &gobj); | ||
| 150 | if (ret) { | ||
| 151 | pr_err("failed to allocate framebuffer (%d)\n", aligned_size); | ||
| 152 | return -ENOMEM; | ||
| 153 | } | ||
| 154 | abo = gem_to_amdgpu_bo(gobj); | ||
| 155 | |||
| 156 | if (fb_tiled) | ||
| 157 | tiling_flags = AMDGPU_TILING_SET(ARRAY_MODE, GRPH_ARRAY_2D_TILED_THIN1); | ||
| 158 | |||
| 159 | ret = amdgpu_bo_reserve(abo, false); | ||
| 160 | if (unlikely(ret != 0)) | ||
| 161 | goto out_unref; | ||
| 162 | |||
| 163 | if (tiling_flags) { | ||
| 164 | ret = amdgpu_bo_set_tiling_flags(abo, | ||
| 165 | tiling_flags); | ||
| 166 | if (ret) | ||
| 167 | dev_err(adev->dev, "FB failed to set tiling flags\n"); | ||
| 168 | } | ||
| 169 | |||
| 170 | ret = amdgpu_bo_pin(abo, domain); | ||
| 171 | if (ret) { | ||
| 172 | amdgpu_bo_unreserve(abo); | ||
| 173 | goto out_unref; | ||
| 174 | } | ||
| 175 | |||
| 176 | ret = amdgpu_ttm_alloc_gart(&abo->tbo); | ||
| 177 | if (ret) { | ||
| 178 | amdgpu_bo_unreserve(abo); | ||
| 179 | dev_err(adev->dev, "%p bind failed\n", abo); | ||
| 180 | goto out_unref; | ||
| 181 | } | ||
| 182 | |||
| 183 | ret = amdgpu_bo_kmap(abo, NULL); | ||
| 184 | amdgpu_bo_unreserve(abo); | ||
| 185 | if (ret) { | ||
| 186 | goto out_unref; | ||
| 187 | } | ||
| 188 | |||
| 189 | *gobj_p = gobj; | ||
| 190 | return 0; | ||
| 191 | out_unref: | ||
| 192 | amdgpufb_destroy_pinned_object(gobj); | ||
| 193 | *gobj_p = NULL; | ||
| 194 | return ret; | ||
| 195 | } | ||
| 196 | |||
| 197 | static int amdgpufb_create(struct drm_fb_helper *helper, | ||
| 198 | struct drm_fb_helper_surface_size *sizes) | ||
| 199 | { | ||
| 200 | struct amdgpu_fbdev *rfbdev = (struct amdgpu_fbdev *)helper; | ||
| 201 | struct amdgpu_device *adev = rfbdev->adev; | ||
| 202 | struct fb_info *info; | ||
| 203 | struct drm_framebuffer *fb = NULL; | ||
| 204 | struct drm_mode_fb_cmd2 mode_cmd; | ||
| 205 | struct drm_gem_object *gobj = NULL; | ||
| 206 | struct amdgpu_bo *abo = NULL; | ||
| 207 | int ret; | ||
| 208 | |||
| 209 | memset(&mode_cmd, 0, sizeof(mode_cmd)); | ||
| 210 | mode_cmd.width = sizes->surface_width; | ||
| 211 | mode_cmd.height = sizes->surface_height; | ||
| 212 | |||
| 213 | if (sizes->surface_bpp == 24) | ||
| 214 | sizes->surface_bpp = 32; | ||
| 215 | |||
| 216 | mode_cmd.pixel_format = drm_mode_legacy_fb_format(sizes->surface_bpp, | ||
| 217 | sizes->surface_depth); | ||
| 218 | |||
| 219 | ret = amdgpufb_create_pinned_object(rfbdev, &mode_cmd, &gobj); | ||
| 220 | if (ret) { | ||
| 221 | DRM_ERROR("failed to create fbcon object %d\n", ret); | ||
| 222 | return ret; | ||
| 223 | } | ||
| 224 | |||
| 225 | abo = gem_to_amdgpu_bo(gobj); | ||
| 226 | |||
| 227 | /* okay we have an object now allocate the framebuffer */ | ||
| 228 | info = drm_fb_helper_alloc_fbi(helper); | ||
| 229 | if (IS_ERR(info)) { | ||
| 230 | ret = PTR_ERR(info); | ||
| 231 | goto out; | ||
| 232 | } | ||
| 233 | |||
| 234 | ret = amdgpu_display_gem_fb_init(adev_to_drm(adev), &rfbdev->rfb, | ||
| 235 | &mode_cmd, gobj); | ||
| 236 | if (ret) { | ||
| 237 | DRM_ERROR("failed to initialize framebuffer %d\n", ret); | ||
| 238 | goto out; | ||
| 239 | } | ||
| 240 | |||
| 241 | fb = &rfbdev->rfb.base; | ||
| 242 | |||
| 243 | /* setup helper */ | ||
| 244 | rfbdev->helper.fb = fb; | ||
| 245 | |||
| 246 | info->fbops = &amdgpufb_ops; | ||
| 247 | |||
| 248 | info->fix.smem_start = amdgpu_gmc_vram_cpu_pa(adev, abo); | ||
| 249 | info->fix.smem_len = amdgpu_bo_size(abo); | ||
| 250 | info->screen_base = amdgpu_bo_kptr(abo); | ||
| 251 | info->screen_size = amdgpu_bo_size(abo); | ||
| 252 | |||
| 253 | drm_fb_helper_fill_info(info, &rfbdev->helper, sizes); | ||
| 254 | |||
| 255 | /* setup aperture base/size for vesafb takeover */ | ||
| 256 | info->apertures->ranges[0].base = adev_to_drm(adev)->mode_config.fb_base; | ||
| 257 | info->apertures->ranges[0].size = adev->gmc.aper_size; | ||
| 258 | |||
| 259 | /* Use default scratch pixmap (info->pixmap.flags = FB_PIXMAP_SYSTEM) */ | ||
| 260 | |||
| 261 | if (info->screen_base == NULL) { | ||
| 262 | ret = -ENOSPC; | ||
| 263 | goto out; | ||
| 264 | } | ||
| 265 | |||
| 266 | DRM_INFO("fb mappable at 0x%lX\n", info->fix.smem_start); | ||
| 267 | DRM_INFO("vram apper at 0x%lX\n", (unsigned long)adev->gmc.aper_base); | ||
| 268 | DRM_INFO("size %lu\n", (unsigned long)amdgpu_bo_size(abo)); | ||
| 269 | DRM_INFO("fb depth is %d\n", fb->format->depth); | ||
| 270 | DRM_INFO(" pitch is %d\n", fb->pitches[0]); | ||
| 271 | |||
| 272 | vga_switcheroo_client_fb_set(adev->pdev, info); | ||
| 273 | return 0; | ||
| 274 | |||
| 275 | out: | ||
| 276 | if (fb && ret) { | ||
| 277 | drm_gem_object_put(gobj); | ||
| 278 | drm_framebuffer_unregister_private(fb); | ||
| 279 | drm_framebuffer_cleanup(fb); | ||
| 280 | kfree(fb); | ||
| 281 | } | ||
| 282 | return ret; | ||
| 283 | } | ||
| 284 | |||
| 285 | static int amdgpu_fbdev_destroy(struct drm_device *dev, struct amdgpu_fbdev *rfbdev) | ||
| 286 | { | ||
| 287 | struct amdgpu_framebuffer *rfb = &rfbdev->rfb; | ||
| 288 | int i; | ||
| 289 | |||
| 290 | drm_fb_helper_unregister_fbi(&rfbdev->helper); | ||
| 291 | |||
| 292 | if (rfb->base.obj[0]) { | ||
| 293 | for (i = 0; i < rfb->base.format->num_planes; i++) | ||
| 294 | drm_gem_object_put(rfb->base.obj[0]); | ||
| 295 | amdgpufb_destroy_pinned_object(rfb->base.obj[0]); | ||
| 296 | rfb->base.obj[0] = NULL; | ||
| 297 | drm_framebuffer_unregister_private(&rfb->base); | ||
| 298 | drm_framebuffer_cleanup(&rfb->base); | ||
| 299 | } | ||
| 300 | drm_fb_helper_fini(&rfbdev->helper); | ||
| 301 | |||
| 302 | return 0; | ||
| 303 | } | ||
| 304 | |||
| 305 | static const struct drm_fb_helper_funcs amdgpu_fb_helper_funcs = { | ||
| 306 | .fb_probe = amdgpufb_create, | ||
| 307 | }; | ||
| 308 | |||
| 309 | int amdgpu_fbdev_init(struct amdgpu_device *adev) | ||
| 310 | { | ||
| 311 | struct amdgpu_fbdev *rfbdev; | ||
| 312 | int bpp_sel = 32; | ||
| 313 | int ret; | ||
| 314 | |||
| 315 | /* don't init fbdev on hw without DCE */ | ||
| 316 | if (!adev->mode_info.mode_config_initialized) | ||
| 317 | return 0; | ||
| 318 | |||
| 319 | /* don't init fbdev if there are no connectors */ | ||
| 320 | if (list_empty(&adev_to_drm(adev)->mode_config.connector_list)) | ||
| 321 | return 0; | ||
| 322 | |||
| 323 | /* select 8 bpp console on low vram cards */ | ||
| 324 | if (adev->gmc.real_vram_size <= (32*1024*1024)) | ||
| 325 | bpp_sel = 8; | ||
| 326 | |||
| 327 | rfbdev = kzalloc(sizeof(struct amdgpu_fbdev), GFP_KERNEL); | ||
| 328 | if (!rfbdev) | ||
| 329 | return -ENOMEM; | ||
| 330 | |||
| 331 | rfbdev->adev = adev; | ||
| 332 | adev->mode_info.rfbdev = rfbdev; | ||
| 333 | |||
| 334 | drm_fb_helper_prepare(adev_to_drm(adev), &rfbdev->helper, | ||
| 335 | &amdgpu_fb_helper_funcs); | ||
| 336 | |||
| 337 | ret = drm_fb_helper_init(adev_to_drm(adev), &rfbdev->helper); | ||
| 338 | if (ret) { | ||
| 339 | kfree(rfbdev); | ||
| 340 | return ret; | ||
| 341 | } | ||
| 342 | |||
| 343 | /* disable all the possible outputs/crtcs before entering KMS mode */ | ||
| 344 | if (!amdgpu_device_has_dc_support(adev) && !amdgpu_virtual_display) | ||
| 345 | drm_helper_disable_unused_functions(adev_to_drm(adev)); | ||
| 346 | |||
| 347 | drm_fb_helper_initial_config(&rfbdev->helper, bpp_sel); | ||
| 348 | return 0; | ||
| 349 | } | ||
| 350 | |||
| 351 | void amdgpu_fbdev_fini(struct amdgpu_device *adev) | ||
| 352 | { | ||
| 353 | if (!adev->mode_info.rfbdev) | ||
| 354 | return; | ||
| 355 | |||
| 356 | amdgpu_fbdev_destroy(adev_to_drm(adev), adev->mode_info.rfbdev); | ||
| 357 | kfree(adev->mode_info.rfbdev); | ||
| 358 | adev->mode_info.rfbdev = NULL; | ||
| 359 | } | ||
| 360 | |||
| 361 | void amdgpu_fbdev_set_suspend(struct amdgpu_device *adev, int state) | ||
| 362 | { | ||
| 363 | if (adev->mode_info.rfbdev) | ||
| 364 | drm_fb_helper_set_suspend_unlocked(&adev->mode_info.rfbdev->helper, | ||
| 365 | state); | ||
| 366 | } | ||
| 367 | |||
| 368 | int amdgpu_fbdev_total_size(struct amdgpu_device *adev) | ||
| 369 | { | ||
| 370 | struct amdgpu_bo *robj; | ||
| 371 | int size = 0; | ||
| 372 | |||
| 373 | if (!adev->mode_info.rfbdev) | ||
| 374 | return 0; | ||
| 375 | |||
| 376 | robj = gem_to_amdgpu_bo(adev->mode_info.rfbdev->rfb.base.obj[0]); | ||
| 377 | size += amdgpu_bo_size(robj); | ||
| 378 | return size; | ||
| 379 | } | ||
| 380 | |||
| 381 | bool amdgpu_fbdev_robj_is_fb(struct amdgpu_device *adev, struct amdgpu_bo *robj) | ||
| 382 | { | ||
| 383 | if (!adev->mode_info.rfbdev) | ||
| 384 | return false; | ||
| 385 | if (robj == gem_to_amdgpu_bo(adev->mode_info.rfbdev->rfb.base.obj[0])) | ||
| 386 | return true; | ||
| 387 | return false; | ||
| 388 | } | ||
| @@ -56,6 +56,9 @@ static bool is_fru_eeprom_supported(struct amdgpu_device *adev) · 55 unmodified lines | |||
| 56 | return true; | 56 | return true; |
| 57 | else | 57 | else |
| 58 | return false; | 58 | return false; |
| 59 | case CHIP_ALDEBARAN: | ||
| 60 | /* All Aldebaran SKUs have the FRU */ | ||
| 61 | return true; | ||
| 59 | default: | 62 | default: |
| 60 | return false; | 63 | return false; |
| 61 | } | 64 | } |
| @@ -88,13 +91,17 @@ static int amdgpu_fru_read_eeprom(struct amdgpu_device *adev, uint32_t addrptr, · 26 unmodified lines | |||
| 88 | 91 | ||
| 89 | int amdgpu_fru_get_product_info(struct amdgpu_device *adev) | 92 | int amdgpu_fru_get_product_info(struct amdgpu_device *adev) |
| 90 | { | 93 | { |
| 91 | unsigned char buff[34]; | 94 | unsigned char buff[AMDGPU_PRODUCT_NAME_LEN+2]; |
| 92 | u32 addrptr; | 95 | u32 addrptr; |
| 93 | int size, len; | 96 | int size, len; |
| 97 | int offset = 2; | ||
| 94 | 98 | ||
| 95 | if (!is_fru_eeprom_supported(adev)) | 99 | if (!is_fru_eeprom_supported(adev)) |
| 96 | return 0; | 100 | return 0; |
| 97 | 101 | ||
| 102 | if (adev->asic_type == CHIP_ALDEBARAN) | ||
| 103 | offset = 0; | ||
| 104 | |||
| 98 | /* If algo exists, it means that the i2c_adapter's initialized */ | 105 | /* If algo exists, it means that the i2c_adapter's initialized */ |
| 99 | if (!adev->pm.smu_i2c.algo) { | 106 | if (!adev->pm.smu_i2c.algo) { |
| 100 | DRM_WARN("Cannot access FRU, EEPROM accessor not initialized"); | 107 | DRM_WARN("Cannot access FRU, EEPROM accessor not initialized"); |
| @@ -131,15 +138,13 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) · 30 unmodified lines | |||
| 131 | } | 138 | } |
| 132 | 139 | ||
| 133 | len = size; | 140 | len = size; |
| 134 | /* Product name should only be 32 characters. Any more, | 141 | if (len >= AMDGPU_PRODUCT_NAME_LEN) { |
| 135 | * and something could be wrong. Cap it at 32 to be safe | 142 | DRM_WARN("FRU Product Name is larger than %d characters. This is likely a mistake", |
| 136 | */ | 143 | AMDGPU_PRODUCT_NAME_LEN); |
| 137 | if (len >= sizeof(adev->product_name)) { | 144 | len = AMDGPU_PRODUCT_NAME_LEN - 1; |
| 138 | DRM_WARN("FRU Product Number is larger than 32 characters. This is likely a mistake"); | ||
| 139 | len = sizeof(adev->product_name) - 1; | ||
| 140 | } | 145 | } |
| 141 | /* Start at 2 due to buff using fields 0 and 1 for the address */ | 146 | /* Start at 2 due to buff using fields 0 and 1 for the address */ |
| 142 | memcpy(adev->product_name, &buff[2], len); | 147 | memcpy(adev->product_name, &buff[offset], len); |
| 143 | adev->product_name[len] = '\0'; | 148 | adev->product_name[len] = '\0'; |
| 144 | 149 | ||
| 145 | addrptr += size + 1; | 150 | addrptr += size + 1; |
| @@ -157,7 +162,7 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) · 11 unmodified lines | |||
| 157 | DRM_WARN("FRU Product Number is larger than 16 characters. This is likely a mistake"); | 162 | DRM_WARN("FRU Product Number is larger than 16 characters. This is likely a mistake"); |
| 158 | len = sizeof(adev->product_number) - 1; | 163 | len = sizeof(adev->product_number) - 1; |
| 159 | } | 164 | } |
| 160 | memcpy(adev->product_number, &buff[2], len); | 165 | memcpy(adev->product_number, &buff[offset], len); |
| 161 | adev->product_number[len] = '\0'; | 166 | adev->product_number[len] = '\0'; |
| 162 | 167 | ||
| 163 | addrptr += size + 1; | 168 | addrptr += size + 1; |
| @@ -184,7 +189,7 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) · 20 unmodified lines | |||
| 184 | DRM_WARN("FRU Serial Number is larger than 16 characters. This is likely a mistake"); | 189 | DRM_WARN("FRU Serial Number is larger than 16 characters. This is likely a mistake"); |
| 185 | len = sizeof(adev->serial) - 1; | 190 | len = sizeof(adev->serial) - 1; |
| 186 | } | 191 | } |
| 187 | memcpy(adev->serial, &buff[2], len); | 192 | memcpy(adev->serial, &buff[offset], len); |
| 188 | adev->serial[len] = '\0'; | 193 | adev->serial[len] = '\0'; |
| 189 | 194 | ||
| 190 | return 0; | 195 | return 0; |
| @@ -264,6 +264,9 @@ static int amdgpu_gem_object_mmap(struct drm_gem_object *obj, struct vm_area_str · 263 unmodified lines | |||
| 264 | !(vma->vm_flags & (VM_READ | VM_WRITE | VM_EXEC))) | 264 | !(vma->vm_flags & (VM_READ | VM_WRITE | VM_EXEC))) |
| 265 | vma->vm_flags &= ~VM_MAYWRITE; | 265 | vma->vm_flags &= ~VM_MAYWRITE; |
| 266 | 266 | ||
| 267 | if (bo->kfd_bo) | ||
| 268 | vma->vm_flags |= VM_DONTCOPY; | ||
| 269 | |||
| 267 | return drm_gem_ttm_mmap(obj, vma); | 270 | return drm_gem_ttm_mmap(obj, vma); |
| 268 | } | 271 | } |
| 269 | 272 | ||
| @@ -877,6 +880,32 @@ out: · 607 unmodified lines | |||
| 877 | return r; | 880 | return r; |
| 878 | } | 881 | } |
| 879 | 882 | ||
| 883 | static int amdgpu_gem_align_pitch(struct amdgpu_device *adev, | ||
| 884 | int width, | ||
| 885 | int cpp, | ||
| 886 | bool tiled) | ||
| 887 | { | ||
| 888 | int aligned = width; | ||
| 889 | int pitch_mask = 0; | ||
| 890 | |||
| 891 | switch (cpp) { | ||
| 892 | case 1: | ||
| 893 | pitch_mask = 255; | ||
| 894 | break; | ||
| 895 | case 2: | ||
| 896 | pitch_mask = 127; | ||
| 897 | break; | ||
| 898 | case 3: | ||
| 899 | case 4: | ||
| 900 | pitch_mask = 63; | ||
| 901 | break; | ||
| 902 | } | ||
| 903 | |||
| 904 | aligned += pitch_mask; | ||
| 905 | aligned &= ~pitch_mask; | ||
| 906 | return aligned * cpp; | ||
| 907 | } | ||
| 908 | |||
| 880 | int amdgpu_mode_dumb_create(struct drm_file *file_priv, | 909 | int amdgpu_mode_dumb_create(struct drm_file *file_priv, |
| 881 | struct drm_device *dev, | 910 | struct drm_device *dev, |
| 882 | struct drm_mode_create_dumb *args) | 911 | struct drm_mode_create_dumb *args) |
| @@ -885,7 +914,8 @@ int amdgpu_mode_dumb_create(struct drm_file *file_priv, · 2 unmodified lines | |||
| 885 | struct drm_gem_object *gobj; | 914 | struct drm_gem_object *gobj; |
| 886 | uint32_t handle; | 915 | uint32_t handle; |
| 887 | u64 flags = AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED | | 916 | u64 flags = AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED | |
| 888 | AMDGPU_GEM_CREATE_CPU_GTT_USWC; | 917 | AMDGPU_GEM_CREATE_CPU_GTT_USWC | |
| 918 | AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS; | ||
| 889 | u32 domain; | 919 | u32 domain; |
| 890 | int r; | 920 | int r; |
| 891 | 921 | ||
| @@ -897,8 +927,8 @@ int amdgpu_mode_dumb_create(struct drm_file *file_priv, · 5 unmodified lines | |||
| 897 | if (adev->mman.buffer_funcs_enabled) | 927 | if (adev->mman.buffer_funcs_enabled) |
| 898 | flags |= AMDGPU_GEM_CREATE_VRAM_CLEARED; | 928 | flags |= AMDGPU_GEM_CREATE_VRAM_CLEARED; |
| 899 | 929 | ||
| 900 | args->pitch = amdgpu_align_pitch(adev, args->width, | 930 | args->pitch = amdgpu_gem_align_pitch(adev, args->width, |
| 901 | DIV_ROUND_UP(args->bpp, 8), 0); | 931 | DIV_ROUND_UP(args->bpp, 8), 0); |
| 902 | args->size = (u64)args->pitch * args->height; | 932 | args->size = (u64)args->pitch * args->height; |
| 903 | args->size = ALIGN(args->size, PAGE_SIZE); | 933 | args->size = ALIGN(args->size, PAGE_SIZE); |
| 904 | domain = amdgpu_bo_get_preferred_domain(adev, | 934 | domain = amdgpu_bo_get_preferred_domain(adev, |
| @@ -350,6 +350,7 @@ static inline uint64_t amdgpu_gmc_fault_key(uint64_t addr, uint16_t pasid) · 349 unmodified lines | |||
| 350 | * amdgpu_gmc_filter_faults - filter VM faults | 350 | * amdgpu_gmc_filter_faults - filter VM faults |
| 351 | * | 351 | * |
| 352 | * @adev: amdgpu device structure | 352 | * @adev: amdgpu device structure |
| 353 | * @ih: interrupt ring that the fault received from | ||
| 353 | * @addr: address of the VM fault | 354 | * @addr: address of the VM fault |
| 354 | * @pasid: PASID of the process causing the fault | 355 | * @pasid: PASID of the process causing the fault |
| 355 | * @timestamp: timestamp of the fault | 356 | * @timestamp: timestamp of the fault |
| @@ -358,7 +359,8 @@ static inline uint64_t amdgpu_gmc_fault_key(uint64_t addr, uint16_t pasid) · 2 unmodified lines | |||
| 358 | * True if the fault was filtered and should not be processed further. | 359 | * True if the fault was filtered and should not be processed further. |
| 359 | * False if the fault is a new one and needs to be handled. | 360 | * False if the fault is a new one and needs to be handled. |
| 360 | */ | 361 | */ |
| 361 | bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev, uint64_t addr, | 362 | bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev, |
| 363 | struct amdgpu_ih_ring *ih, uint64_t addr, | ||
| 362 | uint16_t pasid, uint64_t timestamp) | 364 | uint16_t pasid, uint64_t timestamp) |
| 363 | { | 365 | { |
| 364 | struct amdgpu_gmc *gmc = &adev->gmc; | 366 | struct amdgpu_gmc *gmc = &adev->gmc; |
| @@ -366,6 +368,10 @@ bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev, uint64_t addr, · 1 unmodified line | |||
| 366 | struct amdgpu_gmc_fault *fault; | 368 | struct amdgpu_gmc_fault *fault; |
| 367 | uint32_t hash; | 369 | uint32_t hash; |
| 368 | 370 | ||
| 371 | /* Stale retry fault if timestamp goes backward */ | ||
| 372 | if (amdgpu_ih_ts_after(timestamp, ih->processed_timestamp)) | ||
| 373 | return true; | ||
| 374 | |||
| 369 | /* If we don't have space left in the ring buffer return immediately */ | 375 | /* If we don't have space left in the ring buffer return immediately */ |
| 370 | stamp = max(timestamp, AMDGPU_GMC_FAULT_TIMEOUT + 1) - | 376 | stamp = max(timestamp, AMDGPU_GMC_FAULT_TIMEOUT + 1) - |
| 371 | AMDGPU_GMC_FAULT_TIMEOUT; | 377 | AMDGPU_GMC_FAULT_TIMEOUT; |
| @@ -316,7 +316,8 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, · 315 unmodified lines | |||
| 316 | struct amdgpu_gmc *mc); | 316 | struct amdgpu_gmc *mc); |
| 317 | void amdgpu_gmc_agp_location(struct amdgpu_device *adev, | 317 | void amdgpu_gmc_agp_location(struct amdgpu_device *adev, |
| 318 | struct amdgpu_gmc *mc); | 318 | struct amdgpu_gmc *mc); |
| 319 | bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev, uint64_t addr, | 319 | bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev, |
| 320 | struct amdgpu_ih_ring *ih, uint64_t addr, | ||
| 320 | uint16_t pasid, uint64_t timestamp); | 321 | uint16_t pasid, uint64_t timestamp); |
| 321 | void amdgpu_gmc_filter_faults_remove(struct amdgpu_device *adev, uint64_t addr, | 322 | void amdgpu_gmc_filter_faults_remove(struct amdgpu_device *adev, uint64_t addr, |
| 322 | uint16_t pasid); | 323 | uint16_t pasid); |
| @@ -164,52 +164,32 @@ void amdgpu_ih_ring_write(struct amdgpu_ih_ring *ih, const uint32_t *iv, · 163 unmodified lines | |||
| 164 | } | 164 | } |
| 165 | } | 165 | } |
| 166 | 166 | ||
| 167 | /* Waiter helper that checks current rptr matches or passes checkpoint wptr */ | ||
| 168 | static bool amdgpu_ih_has_checkpoint_processed(struct amdgpu_device *adev, | ||
| 169 | struct amdgpu_ih_ring *ih, | ||
| 170 | uint32_t checkpoint_wptr, | ||
| 171 | uint32_t *prev_rptr) | ||
| 172 | { | ||
| 173 | uint32_t cur_rptr = ih->rptr | (*prev_rptr & ~ih->ptr_mask); | ||
| 174 | |||
| 175 | /* rptr has wrapped. */ | ||
| 176 | if (cur_rptr < *prev_rptr) | ||
| 177 | cur_rptr += ih->ptr_mask + 1; | ||
| 178 | *prev_rptr = cur_rptr; | ||
| 179 | |||
| 180 | /* check ring is empty to workaround missing wptr overflow flag */ | ||
| 181 | return cur_rptr >= checkpoint_wptr || | ||
| 182 | (cur_rptr & ih->ptr_mask) == amdgpu_ih_get_wptr(adev, ih); | ||
| 183 | } | ||
| 184 | |||
| 185 | /** | 167 | /** |
| 186 | * amdgpu_ih_wait_on_checkpoint_process - wait to process IVs up to checkpoint | 168 | * amdgpu_ih_wait_on_checkpoint_process_ts - wait to process IVs up to checkpoint |
| 187 | * | 169 | * |
| 188 | * @adev: amdgpu_device pointer | 170 | * @adev: amdgpu_device pointer |
| 189 | * @ih: ih ring to process | 171 | * @ih: ih ring to process |
| 190 | * | 172 | * |
| 191 | * Used to ensure ring has processed IVs up to the checkpoint write pointer. | 173 | * Used to ensure ring has processed IVs up to the checkpoint write pointer. |
| 192 | */ | 174 | */ |
| 193 | int amdgpu_ih_wait_on_checkpoint_process(struct amdgpu_device *adev, | 175 | int amdgpu_ih_wait_on_checkpoint_process_ts(struct amdgpu_device *adev, |
| 194 | struct amdgpu_ih_ring *ih) | 176 | struct amdgpu_ih_ring *ih) |
| 195 | { | 177 | { |
| 196 | uint32_t checkpoint_wptr, rptr; | 178 | uint32_t checkpoint_wptr; |
| 179 | uint64_t checkpoint_ts; | ||
| 180 | long timeout = HZ; | ||
| 197 | 181 | ||
| 198 | if (!ih->enabled || adev->shutdown) | 182 | if (!ih->enabled || adev->shutdown) |
| 199 | return -ENODEV; | 183 | return -ENODEV; |
| 200 | 184 | ||
| 201 | checkpoint_wptr = amdgpu_ih_get_wptr(adev, ih); | 185 | checkpoint_wptr = amdgpu_ih_get_wptr(adev, ih); |
| 202 | /* Order wptr with rptr. */ | 186 | /* Order wptr with ring data. */ |
| 203 | rmb(); | 187 | rmb(); |
| 204 | rptr = READ_ONCE(ih->rptr); | 188 | checkpoint_ts = amdgpu_ih_decode_iv_ts(adev, ih, checkpoint_wptr, -1); |
| 205 | |||
| 206 | /* wptr has wrapped. */ | ||
| 207 | if (rptr > checkpoint_wptr) | ||
| 208 | checkpoint_wptr += ih->ptr_mask + 1; | ||
| 209 | 189 | ||
| 210 | return wait_event_interruptible(ih->wait_process, | 190 | return wait_event_interruptible_timeout(ih->wait_process, |
| 211 | amdgpu_ih_has_checkpoint_processed(adev, ih, | 191 | amdgpu_ih_ts_after(checkpoint_ts, ih->processed_timestamp) || |
| 212 | checkpoint_wptr, &rptr)); | 192 | ih->rptr == amdgpu_ih_get_wptr(adev, ih), timeout); |
| 213 | } | 193 | } |
| 214 | 194 | ||
| 215 | /** | 195 | /** |
| @@ -299,3 +279,18 @@ void amdgpu_ih_decode_iv_helper(struct amdgpu_device *adev, · 83 unmodified lines | |||
| 299 | /* wptr/rptr are in bytes! */ | 279 | /* wptr/rptr are in bytes! */ |
| 300 | ih->rptr += 32; | 280 | ih->rptr += 32; |
| 301 | } | 281 | } |
| 282 | |||
| 283 | uint64_t amdgpu_ih_decode_iv_ts_helper(struct amdgpu_ih_ring *ih, u32 rptr, | ||
| 284 | signed int offset) | ||
| 285 | { | ||
| 286 | uint32_t iv_size = 32; | ||
| 287 | uint32_t ring_index; | ||
| 288 | uint32_t dw1, dw2; | ||
| 289 | |||
| 290 | rptr += iv_size * offset; | ||
| 291 | ring_index = (rptr & ih->ptr_mask) >> 2; | ||
| 292 | |||
| 293 | dw1 = le32_to_cpu(ih->ring[ring_index + 1]); | ||
| 294 | dw2 = le32_to_cpu(ih->ring[ring_index + 2]); | ||
| 295 | return dw1 | ((u64)(dw2 & 0xffff) << 32); | ||
| 296 | } | ||
| @@ -68,20 +68,30 @@ struct amdgpu_ih_ring { · 67 unmodified lines | |||
| 68 | 68 | ||
| 69 | /* For waiting on IH processing at checkpoint. */ | 69 | /* For waiting on IH processing at checkpoint. */ |
| 70 | wait_queue_head_t wait_process; | 70 | wait_queue_head_t wait_process; |
| 71 | uint64_t processed_timestamp; | ||
| 71 | }; | 72 | }; |
| 72 | 73 | ||
| 74 | /* return true if time stamp t2 is after t1 with 48bit wrap around */ | ||
| 75 | #define amdgpu_ih_ts_after(t1, t2) \ | ||
| 76 | (((int64_t)((t2) << 16) - (int64_t)((t1) << 16)) > 0LL) | ||
| 77 | |||
| 73 | /* provided by the ih block */ | 78 | /* provided by the ih block */ |
| 74 | struct amdgpu_ih_funcs { | 79 | struct amdgpu_ih_funcs { |
| 75 | /* ring read/write ptr handling, called from interrupt context */ | 80 | /* ring read/write ptr handling, called from interrupt context */ |
| 76 | u32 (*get_wptr)(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih); | 81 | u32 (*get_wptr)(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih); |
| 77 | void (*decode_iv)(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, | 82 | void (*decode_iv)(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, |
| 78 | struct amdgpu_iv_entry *entry); | 83 | struct amdgpu_iv_entry *entry); |
| 84 | uint64_t (*decode_iv_ts)(struct amdgpu_ih_ring *ih, u32 rptr, | ||
| 85 | signed int offset); | ||
| 79 | void (*set_rptr)(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih); | 86 | void (*set_rptr)(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih); |
| 80 | }; | 87 | }; |
| 81 | 88 | ||
| 82 | #define amdgpu_ih_get_wptr(adev, ih) (adev)->irq.ih_funcs->get_wptr((adev), (ih)) | 89 | #define amdgpu_ih_get_wptr(adev, ih) (adev)->irq.ih_funcs->get_wptr((adev), (ih)) |
| 83 | #define amdgpu_ih_decode_iv(adev, iv) \ | 90 | #define amdgpu_ih_decode_iv(adev, iv) \ |
| 84 | (adev)->irq.ih_funcs->decode_iv((adev), (ih), (iv)) | 91 | (adev)->irq.ih_funcs->decode_iv((adev), (ih), (iv)) |
| 92 | #define amdgpu_ih_decode_iv_ts(adev, ih, rptr, offset) \ | ||
| 93 | (WARN_ON_ONCE(!(adev)->irq.ih_funcs->decode_iv_ts) ? 0 : \ | ||
| 94 | (adev)->irq.ih_funcs->decode_iv_ts((ih), (rptr), (offset))) | ||
| 85 | #define amdgpu_ih_set_rptr(adev, ih) (adev)->irq.ih_funcs->set_rptr((adev), (ih)) | 95 | #define amdgpu_ih_set_rptr(adev, ih) (adev)->irq.ih_funcs->set_rptr((adev), (ih)) |
| 86 | 96 | ||
| 87 | int amdgpu_ih_ring_init(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, | 97 | int amdgpu_ih_ring_init(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, |
| @@ -89,10 +99,12 @@ int amdgpu_ih_ring_init(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, · 1 unmodified line | |||
| 89 | void amdgpu_ih_ring_fini(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih); | 99 | void amdgpu_ih_ring_fini(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih); |
| 90 | void amdgpu_ih_ring_write(struct amdgpu_ih_ring *ih, const uint32_t *iv, | 100 | void amdgpu_ih_ring_write(struct amdgpu_ih_ring *ih, const uint32_t *iv, |
| 91 | unsigned int num_dw); | 101 | unsigned int num_dw); |
| 92 | int amdgpu_ih_wait_on_checkpoint_process(struct amdgpu_device *adev, | 102 | int amdgpu_ih_wait_on_checkpoint_process_ts(struct amdgpu_device *adev, |
| 93 | struct amdgpu_ih_ring *ih); | 103 | struct amdgpu_ih_ring *ih); |
| 94 | int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih); | 104 | int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih); |
| 95 | void amdgpu_ih_decode_iv_helper(struct amdgpu_device *adev, | 105 | void amdgpu_ih_decode_iv_helper(struct amdgpu_device *adev, |
| 96 | struct amdgpu_ih_ring *ih, | 106 | struct amdgpu_ih_ring *ih, |
| 97 | struct amdgpu_iv_entry *entry); | 107 | struct amdgpu_iv_entry *entry); |
| 108 | uint64_t amdgpu_ih_decode_iv_ts_helper(struct amdgpu_ih_ring *ih, u32 rptr, | ||
| 109 | signed int offset); | ||
| 98 | #endif | 110 | #endif |
| @@ -1,4 +1,4 @@ | |||
| 1 | /** | 1 | /* |
| 2 | * \file amdgpu_ioc32.c | 2 | * \file amdgpu_ioc32.c |
| 3 | * | 3 | * |
| 4 | * 32-bit ioctl compatibility routines for the AMDGPU DRM. | 4 | * 32-bit ioctl compatibility routines for the AMDGPU DRM. |
| @@ -37,12 +37,9 @@ · 32 unmodified lines | |||
| 37 | long amdgpu_kms_compat_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) | 37 | long amdgpu_kms_compat_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) |
| 38 | { | 38 | { |
| 39 | unsigned int nr = DRM_IOCTL_NR(cmd); | 39 | unsigned int nr = DRM_IOCTL_NR(cmd); |
| 40 | int ret; | ||
| 41 | 40 | ||
| 42 | if (nr < DRM_COMMAND_BASE) | 41 | if (nr < DRM_COMMAND_BASE) |
| 43 | return drm_compat_ioctl(filp, cmd, arg); | 42 | return drm_compat_ioctl(filp, cmd, arg); |
| 44 | 43 | ||
| 45 | ret = amdgpu_drm_ioctl(filp, cmd, arg); | 44 | return amdgpu_drm_ioctl(filp, cmd, arg); |
| 46 | |||
| 47 | return ret; | ||
| 48 | } | 45 | } |
| @@ -333,7 +333,6 @@ int amdgpu_irq_init(struct amdgpu_device *adev) · 332 unmodified lines | |||
| 333 | if (!amdgpu_device_has_dc_support(adev)) { | 333 | if (!amdgpu_device_has_dc_support(adev)) { |
| 334 | if (!adev->enable_virtual_display) | 334 | if (!adev->enable_virtual_display) |
| 335 | /* Disable vblank IRQs aggressively for power-saving */ | 335 | /* Disable vblank IRQs aggressively for power-saving */ |
| 336 | /* XXX: can this be enabled for DC? */ | ||
| 337 | adev_to_drm(adev)->vblank_disable_immediate = true; | 336 | adev_to_drm(adev)->vblank_disable_immediate = true; |
| 338 | 337 | ||
| 339 | r = drm_vblank_init(adev_to_drm(adev), adev->mode_info.num_crtc); | 338 | r = drm_vblank_init(adev_to_drm(adev), adev->mode_info.num_crtc); |
| @@ -391,7 +390,7 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev) · 51 unmodified lines | |||
| 391 | } | 390 | } |
| 392 | 391 | ||
| 393 | /** | 392 | /** |
| 394 | * amdgpu_irq_fini - shut down interrupt handling | 393 | * amdgpu_irq_fini_sw - shut down interrupt handling |
| 395 | * | 394 | * |
| 396 | * @adev: amdgpu device pointer | 395 | * @adev: amdgpu device pointer |
| 397 | * | 396 | * |
| @@ -529,6 +528,9 @@ void amdgpu_irq_dispatch(struct amdgpu_device *adev, · 131 unmodified lines | |||
| 529 | /* Send it to amdkfd as well if it isn't already handled */ | 528 | /* Send it to amdkfd as well if it isn't already handled */ |
| 530 | if (!handled) | 529 | if (!handled) |
| 531 | amdgpu_amdkfd_interrupt(adev, entry.iv_entry); | 530 | amdgpu_amdkfd_interrupt(adev, entry.iv_entry); |
| 531 | |||
| 532 | if (amdgpu_ih_ts_after(ih->processed_timestamp, entry.timestamp)) | ||
| 533 | ih->processed_timestamp = entry.timestamp; | ||
| 532 | } | 534 | } |
| 533 | 535 | ||
| 534 | /** | 536 | /** |
| @@ -232,8 +232,6 @@ struct amdgpu_i2c_chan { · 231 unmodified lines | |||
| 232 | struct mutex mutex; | 232 | struct mutex mutex; |
| 233 | }; | 233 | }; |
| 234 | 234 | ||
| 235 | struct amdgpu_fbdev; | ||
| 236 | |||
| 237 | struct amdgpu_afmt { | 235 | struct amdgpu_afmt { |
| 238 | bool enabled; | 236 | bool enabled; |
| 239 | int offset; | 237 | int offset; |
| @@ -309,13 +307,6 @@ struct amdgpu_framebuffer { · 69 unmodified lines | |||
| 309 | uint64_t address; | 307 | uint64_t address; |
| 310 | }; | 308 | }; |
| 311 | 309 | ||
| 312 | struct amdgpu_fbdev { | ||
| 313 | struct drm_fb_helper helper; | ||
| 314 | struct amdgpu_framebuffer rfb; | ||
| 315 | struct list_head fbdev_list; | ||
| 316 | struct amdgpu_device *adev; | ||
| 317 | }; | ||
| 318 | |||
| 319 | struct amdgpu_mode_info { | 310 | struct amdgpu_mode_info { |
| 320 | struct atom_context *atom_context; | 311 | struct atom_context *atom_context; |
| 321 | struct card_info *atom_card_info; | 312 | struct card_info *atom_card_info; |
| @@ -341,8 +332,6 @@ struct amdgpu_mode_info { · 19 unmodified lines | |||
| 341 | struct edid *bios_hardcoded_edid; | 332 | struct edid *bios_hardcoded_edid; |
| 342 | int bios_hardcoded_edid_size; | 333 | int bios_hardcoded_edid_size; |
| 343 | 334 | ||
| 344 | /* pointer to fbdev info structure */ | ||
| 345 | struct amdgpu_fbdev *rfbdev; | ||
| 346 | /* firmware flags */ | 335 | /* firmware flags */ |
| 347 | u32 firmware_flags; | 336 | u32 firmware_flags; |
| 348 | /* pointer to backlight encoder */ | 337 | /* pointer to backlight encoder */ |
| @@ -631,15 +620,6 @@ bool amdgpu_crtc_get_scanout_position(struct drm_crtc *crtc, · 282 unmodified lines | |||
| 631 | int *hpos, ktime_t *stime, ktime_t *etime, | 620 | int *hpos, ktime_t *stime, ktime_t *etime, |
| 632 | const struct drm_display_mode *mode); | 621 | const struct drm_display_mode *mode); |
| 633 | 622 | ||
| 634 | /* fbdev layer */ | ||
| 635 | int amdgpu_fbdev_init(struct amdgpu_device *adev); | ||
| 636 | void amdgpu_fbdev_fini(struct amdgpu_device *adev); | ||
| 637 | void amdgpu_fbdev_set_suspend(struct amdgpu_device *adev, int state); | ||
| 638 | int amdgpu_fbdev_total_size(struct amdgpu_device *adev); | ||
| 639 | bool amdgpu_fbdev_robj_is_fb(struct amdgpu_device *adev, struct amdgpu_bo *robj); | ||
| 640 | |||
| 641 | int amdgpu_align_pitch(struct amdgpu_device *adev, int width, int bpp, bool tiled); | ||
| 642 | |||
| 643 | /* amdgpu_display.c */ | 623 | /* amdgpu_display.c */ |
| 644 | void amdgpu_display_print_display_setup(struct drm_device *dev); | 624 | void amdgpu_display_print_display_setup(struct drm_device *dev); |
| 645 | int amdgpu_display_modeset_create_props(struct amdgpu_device *adev); | 625 | int amdgpu_display_modeset_create_props(struct amdgpu_device *adev); |
| @@ -1032,9 +1032,14 @@ int amdgpu_bo_init(struct amdgpu_device *adev) · 1031 unmodified lines | |||
| 1032 | /* On A+A platform, VRAM can be mapped as WB */ | 1032 | /* On A+A platform, VRAM can be mapped as WB */ |
| 1033 | if (!adev->gmc.xgmi.connected_to_cpu) { | 1033 | if (!adev->gmc.xgmi.connected_to_cpu) { |
| 1034 | /* reserve PAT memory space to WC for VRAM */ | 1034 | /* reserve PAT memory space to WC for VRAM */ |
| 1035 | arch_io_reserve_memtype_wc(adev->gmc.aper_base, | 1035 | int r = arch_io_reserve_memtype_wc(adev->gmc.aper_base, |
| 1036 | adev->gmc.aper_size); | 1036 | adev->gmc.aper_size); |
| 1037 | 1037 | ||
| 1038 | if (r) { | ||
| 1039 | DRM_ERROR("Unable to set WC memtype for the aperture base\n"); | ||
| 1040 | return r; | ||
| 1041 | } | ||
| 1042 | |||
| 1038 | /* Add an MTRR for the VRAM */ | 1043 | /* Add an MTRR for the VRAM */ |
| 1039 | adev->gmc.vram_mtrr = arch_phys_wc_add(adev->gmc.aper_base, | 1044 | adev->gmc.vram_mtrr = arch_phys_wc_add(adev->gmc.aper_base, |
| 1040 | adev->gmc.aper_size); | 1045 | adev->gmc.aper_size); |
| @@ -69,6 +69,7 @@ static void amdgpu_pll_reduce_ratio(unsigned *nom, unsigned *den, · 68 unmodified lines | |||
| 69 | /** | 69 | /** |
| 70 | * amdgpu_pll_get_fb_ref_div - feedback and ref divider calculation | 70 | * amdgpu_pll_get_fb_ref_div - feedback and ref divider calculation |
| 71 | * | 71 | * |
| 72 | * @adev: amdgpu_device pointer | ||
| 72 | * @nom: nominator | 73 | * @nom: nominator |
| 73 | * @den: denominator | 74 | * @den: denominator |
| 74 | * @post_div: post divider | 75 | * @post_div: post divider |
| @@ -106,6 +107,7 @@ static void amdgpu_pll_get_fb_ref_div(struct amdgpu_device *adev, unsigned int n · 31 unmodified lines | |||
| 106 | /** | 107 | /** |
| 107 | * amdgpu_pll_compute - compute PLL paramaters | 108 | * amdgpu_pll_compute - compute PLL paramaters |
| 108 | * | 109 | * |
| 110 | * @adev: amdgpu_device pointer | ||
| 109 | * @pll: information about the PLL | 111 | * @pll: information about the PLL |
| 110 | * @freq: requested frequency | 112 | * @freq: requested frequency |
| 111 | * @dot_clock_p: resulting pixel clock | 113 | * @dot_clock_p: resulting pixel clock |
| @@ -233,6 +233,10 @@ static void amdgpu_perf_start(struct perf_event *event, int flags) · 232 unmodified lines | |||
| 233 | if (WARN_ON_ONCE(!(hwc->state & PERF_HES_STOPPED))) | 233 | if (WARN_ON_ONCE(!(hwc->state & PERF_HES_STOPPED))) |
| 234 | return; | 234 | return; |
| 235 | 235 | ||
| 236 | if ((!pe->adev->df.funcs) || | ||
| 237 | (!pe->adev->df.funcs->pmc_start)) | ||
| 238 | return; | ||
| 239 | |||
| 236 | WARN_ON_ONCE(!(hwc->state & PERF_HES_UPTODATE)); | 240 | WARN_ON_ONCE(!(hwc->state & PERF_HES_UPTODATE)); |
| 237 | hwc->state = 0; | 241 | hwc->state = 0; |
| 238 | 242 | ||
| @@ -268,6 +272,10 @@ static void amdgpu_perf_read(struct perf_event *event) · 29 unmodified lines | |||
| 268 | pmu); | 272 | pmu); |
| 269 | u64 count, prev; | 273 | u64 count, prev; |
| 270 | 274 | ||
| 275 | if ((!pe->adev->df.funcs) || | ||
| 276 | (!pe->adev->df.funcs->pmc_get_count)) | ||
| 277 | return; | ||
| 278 | |||
| 271 | do { | 279 | do { |
| 272 | prev = local64_read(&hwc->prev_count); | 280 | prev = local64_read(&hwc->prev_count); |
| 273 | 281 | ||
| @@ -297,6 +305,10 @@ static void amdgpu_perf_stop(struct perf_event *event, int flags) · 23 unmodified lines | |||
| 297 | if (hwc->state & PERF_HES_UPTODATE) | 305 | if (hwc->state & PERF_HES_UPTODATE) |
| 298 | return; | 306 | return; |
| 299 | 307 | ||
| 308 | if ((!pe->adev->df.funcs) || | ||
| 309 | (!pe->adev->df.funcs->pmc_stop)) | ||
| 310 | return; | ||
| 311 | |||
| 300 | switch (hwc->config_base) { | 312 | switch (hwc->config_base) { |
| 301 | case AMDGPU_PMU_EVENT_CONFIG_TYPE_DF: | 313 | case AMDGPU_PMU_EVENT_CONFIG_TYPE_DF: |
| 302 | case AMDGPU_PMU_EVENT_CONFIG_TYPE_XGMI: | 314 | case AMDGPU_PMU_EVENT_CONFIG_TYPE_XGMI: |
| @@ -326,6 +338,10 @@ static int amdgpu_perf_add(struct perf_event *event, int flags) · 23 unmodified lines | |||
| 326 | struct amdgpu_pmu_entry, | 338 | struct amdgpu_pmu_entry, |
| 327 | pmu); | 339 | pmu); |
| 328 | 340 | ||
| 341 | if ((!pe->adev->df.funcs) || | ||
| 342 | (!pe->adev->df.funcs->pmc_start)) | ||
| 343 | return -EINVAL; | ||
| 344 | |||
| 329 | switch (pe->pmu_perf_type) { | 345 | switch (pe->pmu_perf_type) { |
| 330 | case AMDGPU_PMU_PERF_TYPE_DF: | 346 | case AMDGPU_PMU_PERF_TYPE_DF: |
| 331 | hwc->config_base = AMDGPU_PMU_EVENT_CONFIG_TYPE_DF; | 347 | hwc->config_base = AMDGPU_PMU_EVENT_CONFIG_TYPE_DF; |
| @@ -371,6 +387,9 @@ static void amdgpu_perf_del(struct perf_event *event, int flags) · 39 unmodified lines | |||
| 371 | struct amdgpu_pmu_entry *pe = container_of(event->pmu, | 387 | struct amdgpu_pmu_entry *pe = container_of(event->pmu, |
| 372 | struct amdgpu_pmu_entry, | 388 | struct amdgpu_pmu_entry, |
| 373 | pmu); | 389 | pmu); |
| 390 | if ((!pe->adev->df.funcs) || | ||
| 391 | (!pe->adev->df.funcs->pmc_stop)) | ||
| 392 | return; | ||
| 374 | 393 | ||
| 375 | amdgpu_perf_stop(event, PERF_EF_UPDATE); | 394 | amdgpu_perf_stop(event, PERF_EF_UPDATE); |
| 376 | 395 | ||
| @@ -59,7 +59,7 @@ static DEVICE_ATTR_RO(mem_info_preempt_used); · 58 unmodified lines | |||
| 59 | * @man: TTM memory type manager | 59 | * @man: TTM memory type manager |
| 60 | * @tbo: TTM BO we need this range for | 60 | * @tbo: TTM BO we need this range for |
| 61 | * @place: placement flags and restrictions | 61 | * @place: placement flags and restrictions |
| 62 | * @mem: the resulting mem object | 62 | * @res: TTM memory object |
| 63 | * | 63 | * |
| 64 | * Dummy, just count the space used without allocating resources or any limit. | 64 | * Dummy, just count the space used without allocating resources or any limit. |
| 65 | */ | 65 | */ |
| @@ -85,7 +85,7 @@ static int amdgpu_preempt_mgr_new(struct ttm_resource_manager *man, · 19 unmodified lines | |||
| 85 | * amdgpu_preempt_mgr_del - free ranges | 85 | * amdgpu_preempt_mgr_del - free ranges |
| 86 | * | 86 | * |
| 87 | * @man: TTM memory type manager | 87 | * @man: TTM memory type manager |
| 88 | * @mem: TTM memory object | 88 | * @res: TTM memory object |
| 89 | * | 89 | * |
| 90 | * Free the allocated GTT again. | 90 | * Free the allocated GTT again. |
| 91 | */ | 91 | */ |
| @@ -518,7 +518,7 @@ static struct psp_gfx_cmd_resp *acquire_psp_cmd_buf(struct psp_context *psp) · 517 unmodified lines | |||
| 518 | return cmd; | 518 | return cmd; |
| 519 | } | 519 | } |
| 520 | 520 | ||
| 521 | void release_psp_cmd_buf(struct psp_context *psp) | 521 | static void release_psp_cmd_buf(struct psp_context *psp) |
| 522 | { | 522 | { |
| 523 | mutex_unlock(&psp->mutex); | 523 | mutex_unlock(&psp->mutex); |
| 524 | } | 524 | } |
| @@ -2017,12 +2017,16 @@ static int psp_hw_start(struct psp_context *psp) · 1492 unmodified lines | |||
| 2017 | return ret; | 2017 | return ret; |
| 2018 | } | 2018 | } |
| 2019 | 2019 | ||
| 2020 | if (amdgpu_sriov_vf(adev) && amdgpu_in_reset(adev)) | ||
| 2021 | goto skip_pin_bo; | ||
| 2022 | |||
| 2020 | ret = psp_tmr_init(psp); | 2023 | ret = psp_tmr_init(psp); |
| 2021 | if (ret) { | 2024 | if (ret) { |
| 2022 | DRM_ERROR("PSP tmr init failed!\n"); | 2025 | DRM_ERROR("PSP tmr init failed!\n"); |
| 2023 | return ret; | 2026 | return ret; |
| 2024 | } | 2027 | } |
| 2025 | 2028 | ||
| 2029 | skip_pin_bo: | ||
| 2026 | /* | 2030 | /* |
| 2027 | * For ASICs with DF Cstate management centralized | 2031 | * For ASICs with DF Cstate management centralized |
| 2028 | * to PMFW, TMR setup should be performed after PMFW | 2032 | * to PMFW, TMR setup should be performed after PMFW |
| @@ -2452,6 +2456,18 @@ skip_memalloc: · 423 unmodified lines | |||
| 2452 | return ret; | 2456 | return ret; |
| 2453 | } | 2457 | } |
| 2454 | 2458 | ||
| 2459 | if (amdgpu_sriov_vf(adev) && amdgpu_in_reset(adev)) { | ||
| 2460 | if (adev->gmc.xgmi.num_physical_nodes > 1) { | ||
| 2461 | ret = psp_xgmi_initialize(psp, false, true); | ||
| 2462 | /* Warning the XGMI seesion initialize failure | ||
| 2463 | * Instead of stop driver initialization | ||
| 2464 | */ | ||
| 2465 | if (ret) | ||
| 2466 | dev_err(psp->adev->dev, | ||
| 2467 | "XGMI: Failed to initialize XGMI session\n"); | ||
| 2468 | } | ||
| 2469 | } | ||
| 2470 | |||
| 2455 | if (psp->ta_fw) { | 2471 | if (psp->ta_fw) { |
| 2456 | ret = psp_ras_initialize(psp); | 2472 | ret = psp_ras_initialize(psp); |
| 2457 | if (ret) | 2473 | if (ret) |
| @@ -867,9 +867,9 @@ static int amdgpu_ras_enable_all_features(struct amdgpu_device *adev, · 866 unmodified lines | |||
| 867 | /* feature ctl end */ | 867 | /* feature ctl end */ |
| 868 | 868 | ||
| 869 | 869 | ||
| 870 | void amdgpu_ras_mca_query_error_status(struct amdgpu_device *adev, | 870 | static void amdgpu_ras_mca_query_error_status(struct amdgpu_device *adev, |
| 871 | struct ras_common_if *ras_block, | 871 | struct ras_common_if *ras_block, |
| 872 | struct ras_err_data *err_data) | 872 | struct ras_err_data *err_data) |
| 873 | { | 873 | { |
| 874 | switch (ras_block->sub_block_index) { | 874 | switch (ras_block->sub_block_index) { |
| 875 | case AMDGPU_RAS_MCA_BLOCK__MP0: | 875 | case AMDGPU_RAS_MCA_BLOCK__MP0: |
| @@ -892,6 +892,38 @@ void amdgpu_ras_mca_query_error_status(struct amdgpu_device *adev, · 16 unmodified lines | |||
| 892 | } | 892 | } |
| 893 | } | 893 | } |
| 894 | 894 | ||
| 895 | static void amdgpu_ras_get_ecc_info(struct amdgpu_device *adev, struct ras_err_data *err_data) | ||
| 896 | { | ||
| 897 | struct amdgpu_ras *ras = amdgpu_ras_get_context(adev); | ||
| 898 | int ret = 0; | ||
| 899 | |||
| 900 | /* | ||
| 901 | * choosing right query method according to | ||
| 902 | * whether smu support query error information | ||
| 903 | */ | ||
| 904 | ret = smu_get_ecc_info(&adev->smu, (void *)&(ras->umc_ecc)); | ||
| 905 | if (ret == -EOPNOTSUPP) { | ||
| 906 | if (adev->umc.ras_funcs && | ||
| 907 | adev->umc.ras_funcs->query_ras_error_count) | ||
| 908 | adev->umc.ras_funcs->query_ras_error_count(adev, err_data); | ||
| 909 | |||
| 910 | /* umc query_ras_error_address is also responsible for clearing | ||
| 911 | * error status | ||
| 912 | */ | ||
| 913 | if (adev->umc.ras_funcs && | ||
| 914 | adev->umc.ras_funcs->query_ras_error_address) | ||
| 915 | adev->umc.ras_funcs->query_ras_error_address(adev, err_data); | ||
| 916 | } else if (!ret) { | ||
| 917 | if (adev->umc.ras_funcs && | ||
| 918 | adev->umc.ras_funcs->ecc_info_query_ras_error_count) | ||
| 919 | adev->umc.ras_funcs->ecc_info_query_ras_error_count(adev, err_data); | ||
| 920 | |||
| 921 | if (adev->umc.ras_funcs && | ||
| 922 | adev->umc.ras_funcs->ecc_info_query_ras_error_address) | ||
| 923 | adev->umc.ras_funcs->ecc_info_query_ras_error_address(adev, err_data); | ||
| 924 | } | ||
| 925 | } | ||
| 926 | |||
| 895 | /* query/inject/cure begin */ | 927 | /* query/inject/cure begin */ |
| 896 | int amdgpu_ras_query_error_status(struct amdgpu_device *adev, | 928 | int amdgpu_ras_query_error_status(struct amdgpu_device *adev, |
| 897 | struct ras_query_if *info) | 929 | struct ras_query_if *info) |
| @@ -905,15 +937,7 @@ int amdgpu_ras_query_error_status(struct amdgpu_device *adev, · 7 unmodified lines | |||
| 905 | 937 | ||
| 906 | switch (info->head.block) { | 938 | switch (info->head.block) { |
| 907 | case AMDGPU_RAS_BLOCK__UMC: | 939 | case AMDGPU_RAS_BLOCK__UMC: |
| 908 | if (adev->umc.ras_funcs && | 940 | amdgpu_ras_get_ecc_info(adev, &err_data); |
| 909 | adev->umc.ras_funcs->query_ras_error_count) | ||
| 910 | adev->umc.ras_funcs->query_ras_error_count(adev, &err_data); | ||
| 911 | /* umc query_ras_error_address is also responsible for clearing | ||
| 912 | * error status | ||
| 913 | */ | ||
| 914 | if (adev->umc.ras_funcs && | ||
| 915 | adev->umc.ras_funcs->query_ras_error_address) | ||
| 916 | adev->umc.ras_funcs->query_ras_error_address(adev, &err_data); | ||
| 917 | break; | 941 | break; |
| 918 | case AMDGPU_RAS_BLOCK__SDMA: | 942 | case AMDGPU_RAS_BLOCK__SDMA: |
| 919 | if (adev->sdma.funcs->query_ras_error_count) { | 943 | if (adev->sdma.funcs->query_ras_error_count) { |
| @@ -1137,9 +1161,9 @@ int amdgpu_ras_error_inject(struct amdgpu_device *adev, · 217 unmodified lines | |||
| 1137 | 1161 | ||
| 1138 | /** | 1162 | /** |
| 1139 | * amdgpu_ras_query_error_count -- Get error counts of all IPs | 1163 | * amdgpu_ras_query_error_count -- Get error counts of all IPs |
| 1140 | * adev: pointer to AMD GPU device | 1164 | * @adev: pointer to AMD GPU device |
| 1141 | * ce_count: pointer to an integer to be set to the count of correctible errors. | 1165 | * @ce_count: pointer to an integer to be set to the count of correctible errors. |
| 1142 | * ue_count: pointer to an integer to be set to the count of uncorrectible | 1166 | * @ue_count: pointer to an integer to be set to the count of uncorrectible |
| 1143 | * errors. | 1167 | * errors. |
| 1144 | * | 1168 | * |
| 1145 | * If set, @ce_count or @ue_count, count and return the corresponding | 1169 | * If set, @ce_count or @ue_count, count and return the corresponding |
| @@ -1723,6 +1747,16 @@ static void amdgpu_ras_log_on_err_counter(struct amdgpu_device *adev) · 577 unmodified lines | |||
| 1723 | if (info.head.block == AMDGPU_RAS_BLOCK__PCIE_BIF) | 1747 | if (info.head.block == AMDGPU_RAS_BLOCK__PCIE_BIF) |
| 1724 | continue; | 1748 | continue; |
| 1725 | 1749 | ||
| 1750 | /* | ||
| 1751 | * this is a workaround for aldebaran, skip send msg to | ||
| 1752 | * smu to get ecc_info table due to smu handle get ecc | ||
| 1753 | * info table failed temporarily. | ||
| 1754 | * should be removed until smu fix handle ecc_info table. | ||
| 1755 | */ | ||
| 1756 | if ((info.head.block == AMDGPU_RAS_BLOCK__UMC) && | ||
| 1757 | (adev->ip_versions[MP1_HWIP][0] == IP_VERSION(13, 0, 2))) | ||
| 1758 | continue; | ||
| 1759 | |||
| 1726 | amdgpu_ras_query_error_status(adev, &info); | 1760 | amdgpu_ras_query_error_status(adev, &info); |
| 1727 | } | 1761 | } |
| 1728 | } | 1762 | } |
| @@ -1935,9 +1969,11 @@ int amdgpu_ras_save_bad_pages(struct amdgpu_device *adev) · 206 unmodified lines | |||
| 1935 | if (!con || !con->eh_data) | 1969 | if (!con || !con->eh_data) |
| 1936 | return 0; | 1970 | return 0; |
| 1937 | 1971 | ||
| 1972 | mutex_lock(&con->recovery_lock); | ||
| 1938 | control = &con->eeprom_control; | 1973 | control = &con->eeprom_control; |
| 1939 | data = con->eh_data; | 1974 | data = con->eh_data; |
| 1940 | save_count = data->count - control->ras_num_recs; | 1975 | save_count = data->count - control->ras_num_recs; |
| 1976 | mutex_unlock(&con->recovery_lock); | ||
| 1941 | /* only new entries are saved */ | 1977 | /* only new entries are saved */ |
| 1942 | if (save_count > 0) { | 1978 | if (save_count > 0) { |
| 1943 | if (amdgpu_ras_eeprom_append(control, | 1979 | if (amdgpu_ras_eeprom_append(control, |
| @@ -2336,7 +2372,11 @@ int amdgpu_ras_init(struct amdgpu_device *adev) · 392 unmodified lines | |||
| 2336 | } | 2372 | } |
| 2337 | 2373 | ||
| 2338 | /* Init poison supported flag, the default value is false */ | 2374 | /* Init poison supported flag, the default value is false */ |
| 2339 | if (adev->df.funcs && | 2375 | if (adev->gmc.xgmi.connected_to_cpu) { |
| 2376 | /* enabled by default when GPU is connected to CPU */ | ||
| 2377 | con->poison_supported = true; | ||
| 2378 | } | ||
| 2379 | else if (adev->df.funcs && | ||
| 2340 | adev->df.funcs->query_ras_poison_mode && | 2380 | adev->df.funcs->query_ras_poison_mode && |
| 2341 | adev->umc.ras_funcs && | 2381 | adev->umc.ras_funcs && |
| 2342 | adev->umc.ras_funcs->query_ras_poison_mode) { | 2382 | adev->umc.ras_funcs->query_ras_poison_mode) { |
| @@ -2477,7 +2517,6 @@ void amdgpu_ras_late_fini(struct amdgpu_device *adev, · 134 unmodified lines | |||
| 2477 | amdgpu_ras_sysfs_remove(adev, ras_block); | 2517 | amdgpu_ras_sysfs_remove(adev, ras_block); |
| 2478 | if (ih_info->cb) | 2518 | if (ih_info->cb) |
| 2479 | amdgpu_ras_interrupt_remove_handler(adev, ih_info); | 2519 | amdgpu_ras_interrupt_remove_handler(adev, ih_info); |
| 2480 | amdgpu_ras_feature_enable(adev, ras_block, 0); | ||
| 2481 | } | 2520 | } |
| 2482 | 2521 | ||
| 2483 | /* do some init work after IP late init as dependence. | 2522 | /* do some init work after IP late init as dependence. |
| @@ -319,6 +319,19 @@ struct ras_common_if { · 318 unmodified lines | |||
| 319 | char name[32]; | 319 | char name[32]; |
| 320 | }; | 320 | }; |
| 321 | 321 | ||
| 322 | #define MAX_UMC_CHANNEL_NUM 32 | ||
| 323 | |||
| 324 | struct ecc_info_per_ch { | ||
| 325 | uint16_t ce_count_lo_chip; | ||
| 326 | uint16_t ce_count_hi_chip; | ||
| 327 | uint64_t mca_umc_status; | ||
| 328 | uint64_t mca_umc_addr; | ||
| 329 | }; | ||
| 330 | |||
| 331 | struct umc_ecc_info { | ||
| 332 | struct ecc_info_per_ch ecc[MAX_UMC_CHANNEL_NUM]; | ||
| 333 | }; | ||
| 334 | |||
| 322 | struct amdgpu_ras { | 335 | struct amdgpu_ras { |
| 323 | /* ras infrastructure */ | 336 | /* ras infrastructure */ |
| 324 | /* for ras itself. */ | 337 | /* for ras itself. */ |
| @@ -358,6 +371,9 @@ struct amdgpu_ras { · 33 unmodified lines | |||
| 358 | struct delayed_work ras_counte_delay_work; | 371 | struct delayed_work ras_counte_delay_work; |
| 359 | atomic_t ras_ue_count; | 372 | atomic_t ras_ue_count; |
| 360 | atomic_t ras_ce_count; | 373 | atomic_t ras_ce_count; |
| 374 | |||
| 375 | /* record umc error info queried from smu */ | ||
| 376 | struct umc_ecc_info umc_ecc; | ||
| 361 | }; | 377 | }; |
| 362 | 378 | ||
| 363 | struct ras_fs_data { | 379 | struct ras_fs_data { |
| @@ -252,41 +252,25 @@ int amdgpu_sync_resv(struct amdgpu_device *adev, struct amdgpu_sync *sync, · 251 unmodified lines | |||
| 252 | struct dma_resv *resv, enum amdgpu_sync_mode mode, | 252 | struct dma_resv *resv, enum amdgpu_sync_mode mode, |
| 253 | void *owner) | 253 | void *owner) |
| 254 | { | 254 | { |
| 255 | struct dma_resv_list *flist; | 255 | struct dma_resv_iter cursor; |
| 256 | struct dma_fence *f; | 256 | struct dma_fence *f; |
| 257 | unsigned i; | 257 | int r; |
| 258 | int r = 0; | ||
| 259 | 258 | ||
| 260 | if (resv == NULL) | 259 | if (resv == NULL) |
| 261 | return -EINVAL; | 260 | return -EINVAL; |
| 262 | 261 | ||
| 263 | /* always sync to the exclusive fence */ | 262 | dma_resv_for_each_fence(&cursor, resv, true, f) { |
| 264 | f = dma_resv_excl_fence(resv); | 263 | dma_fence_chain_for_each(f, f) { |
| 265 | dma_fence_chain_for_each(f, f) { | 264 | struct dma_fence_chain *chain = to_dma_fence_chain(f); |
| 266 | struct dma_fence_chain *chain = to_dma_fence_chain(f); | 265 | |
| 267 | 266 | if (amdgpu_sync_test_fence(adev, mode, owner, chain ? | |
| 268 | if (amdgpu_sync_test_fence(adev, mode, owner, chain ? | 267 | chain->fence : f)) { |
| 269 | chain->fence : f)) { | 268 | r = amdgpu_sync_fence(sync, f); |
| 270 | r = amdgpu_sync_fence(sync, f); | 269 | dma_fence_put(f); |
| 271 | dma_fence_put(f); | 270 | if (r) |
| 272 | if (r) | 271 | return r; |
| 273 | return r; | 272 | break; |
| 274 | break; | 273 | } |
| 275 | } | ||
| 276 | } | ||
| 277 | |||
| 278 | flist = dma_resv_shared_list(resv); | ||
| 279 | if (!flist) | ||
| 280 | return 0; | ||
| 281 | |||
| 282 | for (i = 0; i < flist->shared_count; ++i) { | ||
| 283 | f = rcu_dereference_protected(flist->shared[i], | ||
| 284 | dma_resv_held(resv)); | ||
| 285 | |||
| 286 | if (amdgpu_sync_test_fence(adev, mode, owner, f)) { | ||
| 287 | r = amdgpu_sync_fence(sync, f); | ||
| 288 | if (r) | ||
| 289 | return r; | ||
| 290 | } | 274 | } |
| 291 | } | 275 | } |
| 292 | return 0; | 276 | return 0; |
| @@ -116,17 +116,8 @@ static void amdgpu_evict_flags(struct ttm_buffer_object *bo, · 115 unmodified lines | |||
| 116 | 116 | ||
| 117 | abo = ttm_to_amdgpu_bo(bo); | 117 | abo = ttm_to_amdgpu_bo(bo); |
| 118 | if (abo->flags & AMDGPU_AMDKFD_CREATE_SVM_BO) { | 118 | if (abo->flags & AMDGPU_AMDKFD_CREATE_SVM_BO) { |
| 119 | struct dma_fence *fence; | ||
| 120 | struct dma_resv *resv = &bo->base._resv; | ||
| 121 | |||
| 122 | rcu_read_lock(); | ||
| 123 | fence = rcu_dereference(resv->fence_excl); | ||
| 124 | if (fence && !fence->ops->signaled) | ||
| 125 | dma_fence_enable_sw_signaling(fence); | ||
| 126 | |||
| 127 | placement->num_placement = 0; | 119 | placement->num_placement = 0; |
| 128 | placement->num_busy_placement = 0; | 120 | placement->num_busy_placement = 0; |
| 129 | rcu_read_unlock(); | ||
| 130 | return; | 121 | return; |
| 131 | } | 122 | } |
| 132 | 123 | ||
| @@ -922,11 +913,6 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev, · 789 unmodified lines | |||
| 922 | ttm->num_pages, bo_mem, ttm); | 913 | ttm->num_pages, bo_mem, ttm); |
| 923 | } | 914 | } |
| 924 | 915 | ||
| 925 | if (bo_mem->mem_type == AMDGPU_PL_GDS || | ||
| 926 | bo_mem->mem_type == AMDGPU_PL_GWS || | ||
| 927 | bo_mem->mem_type == AMDGPU_PL_OA) | ||
| 928 | return -EINVAL; | ||
| 929 | |||
| 930 | if (bo_mem->mem_type != TTM_PL_TT || | 916 | if (bo_mem->mem_type != TTM_PL_TT || |
| 931 | !amdgpu_gtt_mgr_has_gart_addr(bo_mem)) { | 917 | !amdgpu_gtt_mgr_has_gart_addr(bo_mem)) { |
| 932 | gtt->offset = AMDGPU_BO_INVALID_OFFSET; | 918 | gtt->offset = AMDGPU_BO_INVALID_OFFSET; |
| @@ -1353,10 +1339,9 @@ static bool amdgpu_ttm_bo_eviction_valuable(struct ttm_buffer_object *bo, · 420 unmodified lines | |||
| 1353 | const struct ttm_place *place) | 1339 | const struct ttm_place *place) |
| 1354 | { | 1340 | { |
| 1355 | unsigned long num_pages = bo->resource->num_pages; | 1341 | unsigned long num_pages = bo->resource->num_pages; |
| 1342 | struct dma_resv_iter resv_cursor; | ||
| 1356 | struct amdgpu_res_cursor cursor; | 1343 | struct amdgpu_res_cursor cursor; |
| 1357 | struct dma_resv_list *flist; | ||
| 1358 | struct dma_fence *f; | 1344 | struct dma_fence *f; |
| 1359 | int i; | ||
| 1360 | 1345 | ||
| 1361 | /* Swapout? */ | 1346 | /* Swapout? */ |
| 1362 | if (bo->resource->mem_type == TTM_PL_SYSTEM) | 1347 | if (bo->resource->mem_type == TTM_PL_SYSTEM) |
| @@ -1370,14 +1355,9 @@ static bool amdgpu_ttm_bo_eviction_valuable(struct ttm_buffer_object *bo, · 7 unmodified lines | |||
| 1370 | * If true, then return false as any KFD process needs all its BOs to | 1355 | * If true, then return false as any KFD process needs all its BOs to |
| 1371 | * be resident to run successfully | 1356 | * be resident to run successfully |
| 1372 | */ | 1357 | */ |
| 1373 | flist = dma_resv_shared_list(bo->base.resv); | 1358 | dma_resv_for_each_fence(&resv_cursor, bo->base.resv, true, f) { |
| 1374 | if (flist) { | 1359 | if (amdkfd_fence_check_mm(f, current->mm)) |
| 1375 | for (i = 0; i < flist->shared_count; ++i) { | 1360 | return false; |
| 1376 | f = rcu_dereference_protected(flist->shared[i], | ||
| 1377 | dma_resv_held(bo->base.resv)); | ||
| 1378 | if (amdkfd_fence_check_mm(f, current->mm)) | ||
| 1379 | return false; | ||
| 1380 | } | ||
| 1381 | } | 1361 | } |
| 1382 | 1362 | ||
| 1383 | switch (bo->resource->mem_type) { | 1363 | switch (bo->resource->mem_type) { |
| @@ -23,6 +23,120 @@ · 22 unmodified lines | |||
| 23 | 23 | ||
| 24 | #include "amdgpu_ras.h" | 24 | #include "amdgpu_ras.h" |
| 25 | 25 | ||
| 26 | static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev, | ||
| 27 | void *ras_error_status, | ||
| 28 | struct amdgpu_iv_entry *entry, | ||
| 29 | bool reset) | ||
| 30 | { | ||
| 31 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; | ||
| 32 | struct amdgpu_ras *con = amdgpu_ras_get_context(adev); | ||
| 33 | int ret = 0; | ||
| 34 | |||
| 35 | kgd2kfd_set_sram_ecc_flag(adev->kfd.dev); | ||
| 36 | ret = smu_get_ecc_info(&adev->smu, (void *)&(con->umc_ecc)); | ||
| 37 | if (ret == -EOPNOTSUPP) { | ||
| 38 | if (adev->umc.ras_funcs && | ||
| 39 | adev->umc.ras_funcs->query_ras_error_count) | ||
| 40 | adev->umc.ras_funcs->query_ras_error_count(adev, ras_error_status); | ||
| 41 | |||
| 42 | if (adev->umc.ras_funcs && | ||
| 43 | adev->umc.ras_funcs->query_ras_error_address && | ||
| 44 | adev->umc.max_ras_err_cnt_per_query) { | ||
| 45 | err_data->err_addr = | ||
| 46 | kcalloc(adev->umc.max_ras_err_cnt_per_query, | ||
| 47 | sizeof(struct eeprom_table_record), GFP_KERNEL); | ||
| 48 | |||
| 49 | /* still call query_ras_error_address to clear error status | ||
| 50 | * even NOMEM error is encountered | ||
| 51 | */ | ||
| 52 | if(!err_data->err_addr) | ||
| 53 | dev_warn(adev->dev, "Failed to alloc memory for " | ||
| 54 | "umc error address record!\n"); | ||
| 55 | |||
| 56 | /* umc query_ras_error_address is also responsible for clearing | ||
| 57 | * error status | ||
| 58 | */ | ||
| 59 | adev->umc.ras_funcs->query_ras_error_address(adev, ras_error_status); | ||
| 60 | } | ||
| 61 | } else if (!ret) { | ||
| 62 | if (adev->umc.ras_funcs && | ||
| 63 | adev->umc.ras_funcs->ecc_info_query_ras_error_count) | ||
| 64 | adev->umc.ras_funcs->ecc_info_query_ras_error_count(adev, ras_error_status); | ||
| 65 | |||
| 66 | if (adev->umc.ras_funcs && | ||
| 67 | adev->umc.ras_funcs->ecc_info_query_ras_error_address && | ||
| 68 | adev->umc.max_ras_err_cnt_per_query) { | ||
| 69 | err_data->err_addr = | ||
| 70 | kcalloc(adev->umc.max_ras_err_cnt_per_query, | ||
| 71 | sizeof(struct eeprom_table_record), GFP_KERNEL); | ||
| 72 | |||
| 73 | /* still call query_ras_error_address to clear error status | ||
| 74 | * even NOMEM error is encountered | ||
| 75 | */ | ||
| 76 | if(!err_data->err_addr) | ||
| 77 | dev_warn(adev->dev, "Failed to alloc memory for " | ||
| 78 | "umc error address record!\n"); | ||
| 79 | |||
| 80 | /* umc query_ras_error_address is also responsible for clearing | ||
| 81 | * error status | ||
| 82 | */ | ||
| 83 | adev->umc.ras_funcs->ecc_info_query_ras_error_address(adev, ras_error_status); | ||
| 84 | } | ||
| 85 | } | ||
| 86 | |||
| 87 | /* only uncorrectable error needs gpu reset */ | ||
| 88 | if (err_data->ue_count) { | ||
| 89 | dev_info(adev->dev, "%ld uncorrectable hardware errors " | ||
| 90 | "detected in UMC block\n", | ||
| 91 | err_data->ue_count); | ||
| 92 | |||
| 93 | if ((amdgpu_bad_page_threshold != 0) && | ||
| 94 | err_data->err_addr_cnt) { | ||
| 95 | amdgpu_ras_add_bad_pages(adev, err_data->err_addr, | ||
| 96 | err_data->err_addr_cnt); | ||
| 97 | amdgpu_ras_save_bad_pages(adev); | ||
| 98 | |||
| 99 | if (adev->smu.ppt_funcs && adev->smu.ppt_funcs->send_hbm_bad_pages_num) | ||
| 100 | adev->smu.ppt_funcs->send_hbm_bad_pages_num(&adev->smu, con->eeprom_control.ras_num_recs); | ||
| 101 | } | ||
| 102 | |||
| 103 | if (reset) | ||
| 104 | amdgpu_ras_reset_gpu(adev); | ||
| 105 | } | ||
| 106 | |||
| 107 | kfree(err_data->err_addr); | ||
| 108 | return AMDGPU_RAS_SUCCESS; | ||
| 109 | } | ||
| 110 | |||
| 111 | int amdgpu_umc_poison_handler(struct amdgpu_device *adev, | ||
| 112 | void *ras_error_status, | ||
| 113 | bool reset) | ||
| 114 | { | ||
| 115 | int ret; | ||
| 116 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; | ||
| 117 | struct ras_common_if head = { | ||
| 118 | .block = AMDGPU_RAS_BLOCK__UMC, | ||
| 119 | }; | ||
| 120 | struct ras_manager *obj = amdgpu_ras_find_obj(adev, &head); | ||
| 121 | |||
| 122 | ret = | ||
| 123 | amdgpu_umc_do_page_retirement(adev, ras_error_status, NULL, reset); | ||
| 124 | |||
| 125 | if (ret == AMDGPU_RAS_SUCCESS && obj) { | ||
| 126 | obj->err_data.ue_count += err_data->ue_count; | ||
| 127 | obj->err_data.ce_count += err_data->ce_count; | ||
| 128 | } | ||
| 129 | |||
| 130 | return ret; | ||
| 131 | } | ||
| 132 | |||
| 133 | static int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev, | ||
| 134 | void *ras_error_status, | ||
| 135 | struct amdgpu_iv_entry *entry) | ||
| 136 | { | ||
| 137 | return amdgpu_umc_do_page_retirement(adev, ras_error_status, entry, true); | ||
| 138 | } | ||
| 139 | |||
| 26 | int amdgpu_umc_ras_late_init(struct amdgpu_device *adev) | 140 | int amdgpu_umc_ras_late_init(struct amdgpu_device *adev) |
| 27 | { | 141 | { |
| 28 | int r; | 142 | int r; |
| @@ -88,61 +202,6 @@ void amdgpu_umc_ras_fini(struct amdgpu_device *adev) · 59 unmodified lines | |||
| 88 | } | 202 | } |
| 89 | } | 203 | } |
| 90 | 204 | ||
| 91 | int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev, | ||
| 92 | void *ras_error_status, | ||
| 93 | struct amdgpu_iv_entry *entry) | ||
| 94 | { | ||
| 95 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; | ||
| 96 | struct amdgpu_ras *con = amdgpu_ras_get_context(adev); | ||
| 97 | |||
| 98 | kgd2kfd_set_sram_ecc_flag(adev->kfd.dev); | ||
| 99 | if (adev->umc.ras_funcs && | ||
| 100 | adev->umc.ras_funcs->query_ras_error_count) | ||
| 101 | adev->umc.ras_funcs->query_ras_error_count(adev, ras_error_status); | ||
| 102 | |||
| 103 | if (adev->umc.ras_funcs && | ||
| 104 | adev->umc.ras_funcs->query_ras_error_address && | ||
| 105 | adev->umc.max_ras_err_cnt_per_query) { | ||
| 106 | err_data->err_addr = | ||
| 107 | kcalloc(adev->umc.max_ras_err_cnt_per_query, | ||
| 108 | sizeof(struct eeprom_table_record), GFP_KERNEL); | ||
| 109 | |||
| 110 | /* still call query_ras_error_address to clear error status | ||
| 111 | * even NOMEM error is encountered | ||
| 112 | */ | ||
| 113 | if(!err_data->err_addr) | ||
| 114 | dev_warn(adev->dev, "Failed to alloc memory for " | ||
| 115 | "umc error address record!\n"); | ||
| 116 | |||
| 117 | /* umc query_ras_error_address is also responsible for clearing | ||
| 118 | * error status | ||
| 119 | */ | ||
| 120 | adev->umc.ras_funcs->query_ras_error_address(adev, ras_error_status); | ||
| 121 | } | ||
| 122 | |||
| 123 | /* only uncorrectable error needs gpu reset */ | ||
| 124 | if (err_data->ue_count) { | ||
| 125 | dev_info(adev->dev, "%ld uncorrectable hardware errors " | ||
| 126 | "detected in UMC block\n", | ||
| 127 | err_data->ue_count); | ||
| 128 | |||
| 129 | if ((amdgpu_bad_page_threshold != 0) && | ||
| 130 | err_data->err_addr_cnt) { | ||
| 131 | amdgpu_ras_add_bad_pages(adev, err_data->err_addr, | ||
| 132 | err_data->err_addr_cnt); | ||
| 133 | amdgpu_ras_save_bad_pages(adev); | ||
| 134 | |||
| 135 | if (adev->smu.ppt_funcs && adev->smu.ppt_funcs->send_hbm_bad_pages_num) | ||
| 136 | adev->smu.ppt_funcs->send_hbm_bad_pages_num(&adev->smu, con->eeprom_control.ras_num_recs); | ||
| 137 | } | ||
| 138 | |||
| 139 | amdgpu_ras_reset_gpu(adev); | ||
| 140 | } | ||
| 141 | |||
| 142 | kfree(err_data->err_addr); | ||
| 143 | return AMDGPU_RAS_SUCCESS; | ||
| 144 | } | ||
| 145 | |||
| 146 | int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev, | 205 | int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev, |
| 147 | struct amdgpu_irq_src *source, | 206 | struct amdgpu_irq_src *source, |
| 148 | struct amdgpu_iv_entry *entry) | 207 | struct amdgpu_iv_entry *entry) |
| @@ -49,6 +49,10 @@ struct amdgpu_umc_ras_funcs { · 48 unmodified lines | |||
| 49 | void (*query_ras_error_address)(struct amdgpu_device *adev, | 49 | void (*query_ras_error_address)(struct amdgpu_device *adev, |
| 50 | void *ras_error_status); | 50 | void *ras_error_status); |
| 51 | bool (*query_ras_poison_mode)(struct amdgpu_device *adev); | 51 | bool (*query_ras_poison_mode)(struct amdgpu_device *adev); |
| 52 | void (*ecc_info_query_ras_error_count)(struct amdgpu_device *adev, | ||
| 53 | void *ras_error_status); | ||
| 54 | void (*ecc_info_query_ras_error_address)(struct amdgpu_device *adev, | ||
| 55 | void *ras_error_status); | ||
| 52 | }; | 56 | }; |
| 53 | 57 | ||
| 54 | struct amdgpu_umc_funcs { | 58 | struct amdgpu_umc_funcs { |
| @@ -74,9 +78,9 @@ struct amdgpu_umc { · 19 unmodified lines | |||
| 74 | 78 | ||
| 75 | int amdgpu_umc_ras_late_init(struct amdgpu_device *adev); | 79 | int amdgpu_umc_ras_late_init(struct amdgpu_device *adev); |
| 76 | void amdgpu_umc_ras_fini(struct amdgpu_device *adev); | 80 | void amdgpu_umc_ras_fini(struct amdgpu_device *adev); |
| 77 | int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev, | 81 | int amdgpu_umc_poison_handler(struct amdgpu_device *adev, |
| 78 | void *ras_error_status, | 82 | void *ras_error_status, |
| 79 | struct amdgpu_iv_entry *entry); | 83 | bool reset); |
| 80 | int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev, | 84 | int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev, |
| 81 | struct amdgpu_irq_src *source, | 85 | struct amdgpu_irq_src *source, |
| 82 | struct amdgpu_iv_entry *entry); | 86 | struct amdgpu_iv_entry *entry); |
| @@ -434,7 +434,6 @@ void amdgpu_vce_free_handles(struct amdgpu_device *adev, struct drm_file *filp) · 433 unmodified lines | |||
| 434 | * | 434 | * |
| 435 | * @ring: ring we should submit the msg to | 435 | * @ring: ring we should submit the msg to |
| 436 | * @handle: VCE session handle to use | 436 | * @handle: VCE session handle to use |
| 437 | * @bo: amdgpu object for which we query the offset | ||
| 438 | * @fence: optional fence to return | 437 | * @fence: optional fence to return |
| 439 | * | 438 | * |
| 440 | * Open up a stream for HW test | 439 | * Open up a stream for HW test |
| @@ -286,20 +286,13 @@ int amdgpu_vcn_sw_fini(struct amdgpu_device *adev) · 285 unmodified lines | |||
| 286 | bool amdgpu_vcn_is_disabled_vcn(struct amdgpu_device *adev, enum vcn_ring_type type, uint32_t vcn_instance) | 286 | bool amdgpu_vcn_is_disabled_vcn(struct amdgpu_device *adev, enum vcn_ring_type type, uint32_t vcn_instance) |
| 287 | { | 287 | { |
| 288 | bool ret = false; | 288 | bool ret = false; |
| 289 | int vcn_config = adev->vcn.vcn_config[vcn_instance]; | ||
| 289 | 290 | ||
| 290 | int major; | 291 | if ((type == VCN_ENCODE_RING) && (vcn_config & VCN_BLOCK_ENCODE_DISABLE_MASK)) { |
| 291 | int minor; | ||
| 292 | int revision; | ||
| 293 | |||
| 294 | /* if cannot find IP data, then this VCN does not exist */ | ||
| 295 | if (amdgpu_discovery_get_vcn_version(adev, vcn_instance, &major, &minor, &revision) != 0) | ||
| 296 | return true; | ||
| 297 | |||
| 298 | if ((type == VCN_ENCODE_RING) && (revision & VCN_BLOCK_ENCODE_DISABLE_MASK)) { | ||
| 299 | ret = true; | 292 | ret = true; |
| 300 | } else if ((type == VCN_DECODE_RING) && (revision & VCN_BLOCK_DECODE_DISABLE_MASK)) { | 293 | } else if ((type == VCN_DECODE_RING) && (vcn_config & VCN_BLOCK_DECODE_DISABLE_MASK)) { |
| 301 | ret = true; | 294 | ret = true; |
| 302 | } else if ((type == VCN_UNIFIED_RING) && (revision & VCN_BLOCK_QUEUE_DISABLE_MASK)) { | 295 | } else if ((type == VCN_UNIFIED_RING) && (vcn_config & VCN_BLOCK_QUEUE_DISABLE_MASK)) { |
| 303 | ret = true; | 296 | ret = true; |
| 304 | } | 297 | } |
| 305 | 298 | ||
| @@ -235,6 +235,7 @@ struct amdgpu_vcn { · 234 unmodified lines | |||
| 235 | 235 | ||
| 236 | uint8_t num_vcn_inst; | 236 | uint8_t num_vcn_inst; |
| 237 | struct amdgpu_vcn_inst inst[AMDGPU_MAX_VCN_INSTANCES]; | 237 | struct amdgpu_vcn_inst inst[AMDGPU_MAX_VCN_INSTANCES]; |
| 238 | uint8_t vcn_config[AMDGPU_MAX_VCN_INSTANCES]; | ||
| 238 | struct amdgpu_vcn_reg internal; | 239 | struct amdgpu_vcn_reg internal; |
| 239 | struct mutex vcn_pg_lock; | 240 | struct mutex vcn_pg_lock; |
| 240 | struct mutex vcn1_jpeg1_workaround; | 241 | struct mutex vcn1_jpeg1_workaround; |
| @@ -283,17 +283,15 @@ static int amdgpu_virt_init_ras_err_handler_data(struct amdgpu_device *adev) · 282 unmodified lines | |||
| 283 | 283 | ||
| 284 | *data = kmalloc(sizeof(struct amdgpu_virt_ras_err_handler_data), GFP_KERNEL); | 284 | *data = kmalloc(sizeof(struct amdgpu_virt_ras_err_handler_data), GFP_KERNEL); |
| 285 | if (!*data) | 285 | if (!*data) |
| 286 | return -ENOMEM; | 286 | goto data_failure; |
| 287 | 287 | ||
| 288 | bps = kmalloc_array(align_space, sizeof((*data)->bps), GFP_KERNEL); | 288 | bps = kmalloc_array(align_space, sizeof((*data)->bps), GFP_KERNEL); |
| 289 | bps_bo = kmalloc_array(align_space, sizeof((*data)->bps_bo), GFP_KERNEL); | 289 | if (!bps) |
| 290 | goto bps_failure; | ||
| 290 | 291 | ||
| 291 | if (!bps || !bps_bo) { | 292 | bps_bo = kmalloc_array(align_space, sizeof((*data)->bps_bo), GFP_KERNEL); |
| 292 | kfree(bps); | 293 | if (!bps_bo) |
| 293 | kfree(bps_bo); | 294 | goto bps_bo_failure; |
| 294 | kfree(*data); | ||
| 295 | return -ENOMEM; | ||
| 296 | } | ||
| 297 | 295 | ||
| 298 | (*data)->bps = bps; | 296 | (*data)->bps = bps; |
| 299 | (*data)->bps_bo = bps_bo; | 297 | (*data)->bps_bo = bps_bo; |
| @@ -303,6 +301,13 @@ static int amdgpu_virt_init_ras_err_handler_data(struct amdgpu_device *adev) · 3 unmodified lines | |||
| 303 | virt->ras_init_done = true; | 301 | virt->ras_init_done = true; |
| 304 | 302 | ||
| 305 | return 0; | 303 | return 0; |
| 304 | |||
| 305 | bps_bo_failure: | ||
| 306 | kfree(bps); | ||
| 307 | bps_failure: | ||
| 308 | kfree(*data); | ||
| 309 | data_failure: | ||
| 310 | return -ENOMEM; | ||
| 306 | } | 311 | } |
| 307 | 312 | ||
| 308 | static void amdgpu_virt_ras_release_bp(struct amdgpu_device *adev) | 313 | static void amdgpu_virt_ras_release_bp(struct amdgpu_device *adev) |
| @@ -617,17 +622,35 @@ void amdgpu_virt_fini_data_exchange(struct amdgpu_device *adev) · 308 unmodified lines | |||
| 617 | 622 | ||
| 618 | void amdgpu_virt_init_data_exchange(struct amdgpu_device *adev) | 623 | void amdgpu_virt_init_data_exchange(struct amdgpu_device *adev) |
| 619 | { | 624 | { |
| 620 | uint64_t bp_block_offset = 0; | ||
| 621 | uint32_t bp_block_size = 0; | ||
| 622 | struct amd_sriov_msg_pf2vf_info *pf2vf_v2 = NULL; | ||
| 623 | |||
| 624 | adev->virt.fw_reserve.p_pf2vf = NULL; | 625 | adev->virt.fw_reserve.p_pf2vf = NULL; |
| 625 | adev->virt.fw_reserve.p_vf2pf = NULL; | 626 | adev->virt.fw_reserve.p_vf2pf = NULL; |
| 626 | adev->virt.vf2pf_update_interval_ms = 0; | 627 | adev->virt.vf2pf_update_interval_ms = 0; |
| 627 | 628 | ||
| 628 | if (adev->mman.fw_vram_usage_va != NULL) { | 629 | if (adev->bios != NULL) { |
| 629 | adev->virt.vf2pf_update_interval_ms = 2000; | 630 | adev->virt.vf2pf_update_interval_ms = 2000; |
| 630 | 631 | ||
| 632 | adev->virt.fw_reserve.p_pf2vf = | ||
| 633 | (struct amd_sriov_msg_pf2vf_info_header *) | ||
| 634 | (adev->bios + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB << 10)); | ||
| 635 | |||
| 636 | amdgpu_virt_read_pf2vf_data(adev); | ||
| 637 | } | ||
| 638 | |||
| 639 | if (adev->virt.vf2pf_update_interval_ms != 0) { | ||
| 640 | INIT_DELAYED_WORK(&adev->virt.vf2pf_work, amdgpu_virt_update_vf2pf_work_item); | ||
| 641 | schedule_delayed_work(&(adev->virt.vf2pf_work), msecs_to_jiffies(adev->virt.vf2pf_update_interval_ms)); | ||
| 642 | } | ||
| 643 | } | ||
| 644 | |||
| 645 | |||
| 646 | void amdgpu_virt_exchange_data(struct amdgpu_device *adev) | ||
| 647 | { | ||
| 648 | uint64_t bp_block_offset = 0; | ||
| 649 | uint32_t bp_block_size = 0; | ||
| 650 | struct amd_sriov_msg_pf2vf_info *pf2vf_v2 = NULL; | ||
| 651 | |||
| 652 | if (adev->mman.fw_vram_usage_va != NULL) { | ||
| 653 | |||
| 631 | adev->virt.fw_reserve.p_pf2vf = | 654 | adev->virt.fw_reserve.p_pf2vf = |
| 632 | (struct amd_sriov_msg_pf2vf_info_header *) | 655 | (struct amd_sriov_msg_pf2vf_info_header *) |
| 633 | (adev->mman.fw_vram_usage_va + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB << 10)); | 656 | (adev->mman.fw_vram_usage_va + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB << 10)); |
| @@ -658,16 +681,10 @@ void amdgpu_virt_init_data_exchange(struct amdgpu_device *adev) · 24 unmodified lines | |||
| 658 | (adev->bios + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB << 10)); | 681 | (adev->bios + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB << 10)); |
| 659 | 682 | ||
| 660 | amdgpu_virt_read_pf2vf_data(adev); | 683 | amdgpu_virt_read_pf2vf_data(adev); |
| 661 | |||
| 662 | return; | ||
| 663 | } | ||
| 664 | |||
| 665 | if (adev->virt.vf2pf_update_interval_ms != 0) { | ||
| 666 | INIT_DELAYED_WORK(&adev->virt.vf2pf_work, amdgpu_virt_update_vf2pf_work_item); | ||
| 667 | schedule_delayed_work(&(adev->virt.vf2pf_work), adev->virt.vf2pf_update_interval_ms); | ||
| 668 | } | 684 | } |
| 669 | } | 685 | } |
| 670 | 686 | ||
| 687 | |||
| 671 | void amdgpu_detect_virtualization(struct amdgpu_device *adev) | 688 | void amdgpu_detect_virtualization(struct amdgpu_device *adev) |
| 672 | { | 689 | { |
| 673 | uint32_t reg; | 690 | uint32_t reg; |
| @@ -308,6 +308,7 @@ int amdgpu_virt_alloc_mm_table(struct amdgpu_device *adev); · 307 unmodified lines | |||
| 308 | void amdgpu_virt_free_mm_table(struct amdgpu_device *adev); | 308 | void amdgpu_virt_free_mm_table(struct amdgpu_device *adev); |
| 309 | void amdgpu_virt_release_ras_err_handler_data(struct amdgpu_device *adev); | 309 | void amdgpu_virt_release_ras_err_handler_data(struct amdgpu_device *adev); |
| 310 | void amdgpu_virt_init_data_exchange(struct amdgpu_device *adev); | 310 | void amdgpu_virt_init_data_exchange(struct amdgpu_device *adev); |
| 311 | void amdgpu_virt_exchange_data(struct amdgpu_device *adev); | ||
| 311 | void amdgpu_virt_fini_data_exchange(struct amdgpu_device *adev); | 312 | void amdgpu_virt_fini_data_exchange(struct amdgpu_device *adev); |
| 312 | void amdgpu_detect_virtualization(struct amdgpu_device *adev); | 313 | void amdgpu_detect_virtualization(struct amdgpu_device *adev); |
| 313 | 314 | ||
| @@ -16,6 +16,8 @@ · 15 unmodified lines | |||
| 16 | #include "ivsrcid/ivsrcid_vislands30.h" | 16 | #include "ivsrcid/ivsrcid_vislands30.h" |
| 17 | #include "amdgpu_vkms.h" | 17 | #include "amdgpu_vkms.h" |
| 18 | #include "amdgpu_display.h" | 18 | #include "amdgpu_display.h" |
| 19 | #include "atom.h" | ||
| 20 | #include "amdgpu_irq.h" | ||
| 19 | 21 | ||
| 20 | /** | 22 | /** |
| 21 | * DOC: amdgpu_vkms | 23 | * DOC: amdgpu_vkms |
| @@ -41,16 +43,16 @@ static const u32 amdgpu_vkms_formats[] = { · 19 unmodified lines | |||
| 41 | 43 | ||
| 42 | static enum hrtimer_restart amdgpu_vkms_vblank_simulate(struct hrtimer *timer) | 44 | static enum hrtimer_restart amdgpu_vkms_vblank_simulate(struct hrtimer *timer) |
| 43 | { | 45 | { |
| 44 | struct amdgpu_vkms_output *output = container_of(timer, | 46 | struct amdgpu_crtc *amdgpu_crtc = container_of(timer, struct amdgpu_crtc, vblank_timer); |
| 45 | struct amdgpu_vkms_output, | 47 | struct drm_crtc *crtc = &amdgpu_crtc->base; |
| 46 | vblank_hrtimer); | 48 | struct amdgpu_vkms_output *output = drm_crtc_to_amdgpu_vkms_output(crtc); |
| 47 | struct drm_crtc *crtc = &output->crtc; | ||
| 48 | u64 ret_overrun; | 49 | u64 ret_overrun; |
| 49 | bool ret; | 50 | bool ret; |
| 50 | 51 | ||
| 51 | ret_overrun = hrtimer_forward_now(&output->vblank_hrtimer, | 52 | ret_overrun = hrtimer_forward_now(&amdgpu_crtc->vblank_timer, |
| 52 | output->period_ns); | 53 | output->period_ns); |
| 53 | WARN_ON(ret_overrun != 1); | 54 | if (ret_overrun != 1) |
| 55 | DRM_WARN("%s: vblank timer overrun\n", __func__); | ||
| 54 | 56 | ||
| 55 | ret = drm_crtc_handle_vblank(crtc); | 57 | ret = drm_crtc_handle_vblank(crtc); |
| 56 | if (!ret) | 58 | if (!ret) |
| @@ -65,22 +67,21 @@ static int amdgpu_vkms_enable_vblank(struct drm_crtc *crtc) · 8 unmodified lines | |||
| 65 | unsigned int pipe = drm_crtc_index(crtc); | 67 | unsigned int pipe = drm_crtc_index(crtc); |
| 66 | struct drm_vblank_crtc *vblank = &dev->vblank[pipe]; | 68 | struct drm_vblank_crtc *vblank = &dev->vblank[pipe]; |
| 67 | struct amdgpu_vkms_output *out = drm_crtc_to_amdgpu_vkms_output(crtc); | 69 | struct amdgpu_vkms_output *out = drm_crtc_to_amdgpu_vkms_output(crtc); |
| 70 | struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc); | ||
| 68 | 71 | ||
| 69 | drm_calc_timestamping_constants(crtc, &crtc->mode); | 72 | drm_calc_timestamping_constants(crtc, &crtc->mode); |
| 70 | 73 | ||
| 71 | hrtimer_init(&out->vblank_hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL); | ||
| 72 | out->vblank_hrtimer.function = &amdgpu_vkms_vblank_simulate; | ||
| 73 | out->period_ns = ktime_set(0, vblank->framedur_ns); | 74 | out->period_ns = ktime_set(0, vblank->framedur_ns); |
| 74 | hrtimer_start(&out->vblank_hrtimer, out->period_ns, HRTIMER_MODE_REL); | 75 | hrtimer_start(&amdgpu_crtc->vblank_timer, out->period_ns, HRTIMER_MODE_REL); |
| 75 | 76 | ||
| 76 | return 0; | 77 | return 0; |
| 77 | } | 78 | } |
| 78 | 79 | ||
| 79 | static void amdgpu_vkms_disable_vblank(struct drm_crtc *crtc) | 80 | static void amdgpu_vkms_disable_vblank(struct drm_crtc *crtc) |
| 80 | { | 81 | { |
| 81 | struct amdgpu_vkms_output *out = drm_crtc_to_amdgpu_vkms_output(crtc); | 82 | struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc); |
| 82 | 83 | ||
| 83 | hrtimer_cancel(&out->vblank_hrtimer); | 84 | hrtimer_cancel(&amdgpu_crtc->vblank_timer); |
| 84 | } | 85 | } |
| 85 | 86 | ||
| 86 | static bool amdgpu_vkms_get_vblank_timestamp(struct drm_crtc *crtc, | 87 | static bool amdgpu_vkms_get_vblank_timestamp(struct drm_crtc *crtc, |
| @@ -92,13 +93,14 @@ static bool amdgpu_vkms_get_vblank_timestamp(struct drm_crtc *crtc, · 5 unmodified lines | |||
| 92 | unsigned int pipe = crtc->index; | 93 | unsigned int pipe = crtc->index; |
| 93 | struct amdgpu_vkms_output *output = drm_crtc_to_amdgpu_vkms_output(crtc); | 94 | struct amdgpu_vkms_output *output = drm_crtc_to_amdgpu_vkms_output(crtc); |
| 94 | struct drm_vblank_crtc *vblank = &dev->vblank[pipe]; | 95 | struct drm_vblank_crtc *vblank = &dev->vblank[pipe]; |
| 96 | struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc); | ||
| 95 | 97 | ||
| 96 | if (!READ_ONCE(vblank->enabled)) { | 98 | if (!READ_ONCE(vblank->enabled)) { |
| 97 | *vblank_time = ktime_get(); | 99 | *vblank_time = ktime_get(); |
| 98 | return true; | 100 | return true; |
| 99 | } | 101 | } |
| 100 | 102 | ||
| 101 | *vblank_time = READ_ONCE(output->vblank_hrtimer.node.expires); | 103 | *vblank_time = READ_ONCE(amdgpu_crtc->vblank_timer.node.expires); |
| 102 | 104 | ||
| 103 | if (WARN_ON(*vblank_time == vblank->time)) | 105 | if (WARN_ON(*vblank_time == vblank->time)) |
| 104 | return true; | 106 | return true; |
| @@ -165,6 +167,8 @@ static const struct drm_crtc_helper_funcs amdgpu_vkms_crtc_helper_funcs = { · 60 unmodified lines | |||
| 165 | static int amdgpu_vkms_crtc_init(struct drm_device *dev, struct drm_crtc *crtc, | 167 | static int amdgpu_vkms_crtc_init(struct drm_device *dev, struct drm_crtc *crtc, |
| 166 | struct drm_plane *primary, struct drm_plane *cursor) | 168 | struct drm_plane *primary, struct drm_plane *cursor) |
| 167 | { | 169 | { |
| 170 | struct amdgpu_device *adev = drm_to_adev(dev); | ||
| 171 | struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc); | ||
| 168 | int ret; | 172 | int ret; |
| 169 | 173 | ||
| 170 | ret = drm_crtc_init_with_planes(dev, crtc, primary, cursor, | 174 | ret = drm_crtc_init_with_planes(dev, crtc, primary, cursor, |
| @@ -176,6 +180,17 @@ static int amdgpu_vkms_crtc_init(struct drm_device *dev, struct drm_crtc *crtc, · 5 unmodified lines | |||
| 176 | 180 | ||
| 177 | drm_crtc_helper_add(crtc, &amdgpu_vkms_crtc_helper_funcs); | 181 | drm_crtc_helper_add(crtc, &amdgpu_vkms_crtc_helper_funcs); |
| 178 | 182 | ||
| 183 | amdgpu_crtc->crtc_id = drm_crtc_index(crtc); | ||
| 184 | adev->mode_info.crtcs[drm_crtc_index(crtc)] = amdgpu_crtc; | ||
| 185 | |||
| 186 | amdgpu_crtc->pll_id = ATOM_PPLL_INVALID; | ||
| 187 | amdgpu_crtc->encoder = NULL; | ||
| 188 | amdgpu_crtc->connector = NULL; | ||
| 189 | amdgpu_crtc->vsync_timer_enabled = AMDGPU_IRQ_STATE_DISABLE; | ||
| 190 | |||
| 191 | hrtimer_init(&amdgpu_crtc->vblank_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL); | ||
| 192 | amdgpu_crtc->vblank_timer.function = &amdgpu_vkms_vblank_simulate; | ||
| 193 | |||
| 179 | return ret; | 194 | return ret; |
| 180 | } | 195 | } |
| 181 | 196 | ||
| @@ -375,6 +390,7 @@ static struct drm_plane *amdgpu_vkms_plane_init(struct drm_device *dev, · 193 unmodified lines | |||
| 375 | int index) | 390 | int index) |
| 376 | { | 391 | { |
| 377 | struct drm_plane *plane; | 392 | struct drm_plane *plane; |
| 393 | uint64_t modifiers[] = {DRM_FORMAT_MOD_LINEAR, DRM_FORMAT_MOD_INVALID}; | ||
| 378 | int ret; | 394 | int ret; |
| 379 | 395 | ||
| 380 | plane = kzalloc(sizeof(*plane), GFP_KERNEL); | 396 | plane = kzalloc(sizeof(*plane), GFP_KERNEL); |
| @@ -385,7 +401,7 @@ static struct drm_plane *amdgpu_vkms_plane_init(struct drm_device *dev, · 4 unmodified lines | |||
| 385 | &amdgpu_vkms_plane_funcs, | 401 | &amdgpu_vkms_plane_funcs, |
| 386 | amdgpu_vkms_formats, | 402 | amdgpu_vkms_formats, |
| 387 | ARRAY_SIZE(amdgpu_vkms_formats), | 403 | ARRAY_SIZE(amdgpu_vkms_formats), |
| 388 | NULL, type, NULL); | 404 | modifiers, type, NULL); |
| 389 | if (ret) { | 405 | if (ret) { |
| 390 | kfree(plane); | 406 | kfree(plane); |
| 391 | return ERR_PTR(ret); | 407 | return ERR_PTR(ret); |
| @@ -396,12 +412,12 @@ static struct drm_plane *amdgpu_vkms_plane_init(struct drm_device *dev, · 4 unmodified lines | |||
| 396 | return plane; | 412 | return plane; |
| 397 | } | 413 | } |
| 398 | 414 | ||
| 399 | int amdgpu_vkms_output_init(struct drm_device *dev, | 415 | static int amdgpu_vkms_output_init(struct drm_device *dev, struct |
| 400 | struct amdgpu_vkms_output *output, int index) | 416 | amdgpu_vkms_output *output, int index) |
| 401 | { | 417 | { |
| 402 | struct drm_connector *connector = &output->connector; | 418 | struct drm_connector *connector = &output->connector; |
| 403 | struct drm_encoder *encoder = &output->encoder; | 419 | struct drm_encoder *encoder = &output->encoder; |
| 404 | struct drm_crtc *crtc = &output->crtc; | 420 | struct drm_crtc *crtc = &output->crtc.base; |
| 405 | struct drm_plane *primary, *cursor = NULL; | 421 | struct drm_plane *primary, *cursor = NULL; |
| 406 | int ret; | 422 | int ret; |
| 407 | 423 | ||
| @@ -465,6 +481,11 @@ static int amdgpu_vkms_sw_init(void *handle) · 57 unmodified lines | |||
| 465 | int r, i; | 481 | int r, i; |
| 466 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; | 482 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 467 | 483 | ||
| 484 | adev->amdgpu_vkms_output = kcalloc(adev->mode_info.num_crtc, | ||
| 485 | sizeof(struct amdgpu_vkms_output), GFP_KERNEL); | ||
| 486 | if (!adev->amdgpu_vkms_output) | ||
| 487 | return -ENOMEM; | ||
| 488 | |||
| 468 | adev_to_drm(adev)->max_vblank_count = 0; | 489 | adev_to_drm(adev)->max_vblank_count = 0; |
| 469 | 490 | ||
| 470 | adev_to_drm(adev)->mode_config.funcs = &amdgpu_vkms_mode_funcs; | 491 | adev_to_drm(adev)->mode_config.funcs = &amdgpu_vkms_mode_funcs; |
| @@ -481,10 +502,6 @@ static int amdgpu_vkms_sw_init(void *handle) · 10 unmodified lines | |||
| 481 | if (r) | 502 | if (r) |
| 482 | return r; | 503 | return r; |
| 483 | 504 | ||
| 484 | adev->amdgpu_vkms_output = kcalloc(adev->mode_info.num_crtc, sizeof(struct amdgpu_vkms_output), GFP_KERNEL); | ||
| 485 | if (!adev->amdgpu_vkms_output) | ||
| 486 | return -ENOMEM; | ||
| 487 | |||
| 488 | /* allocate crtcs, encoders, connectors */ | 505 | /* allocate crtcs, encoders, connectors */ |
| 489 | for (i = 0; i < adev->mode_info.num_crtc; i++) { | 506 | for (i = 0; i < adev->mode_info.num_crtc; i++) { |
| 490 | r = amdgpu_vkms_output_init(adev_to_drm(adev), &adev->amdgpu_vkms_output[i], i); | 507 | r = amdgpu_vkms_output_init(adev_to_drm(adev), &adev->amdgpu_vkms_output[i], i); |
| @@ -504,15 +521,16 @@ static int amdgpu_vkms_sw_fini(void *handle) · 13 unmodified lines | |||
| 504 | int i = 0; | 521 | int i = 0; |
| 505 | 522 | ||
| 506 | for (i = 0; i < adev->mode_info.num_crtc; i++) | 523 | for (i = 0; i < adev->mode_info.num_crtc; i++) |
| 507 | if (adev->amdgpu_vkms_output[i].vblank_hrtimer.function) | 524 | if (adev->mode_info.crtcs[i]) |
| 508 | hrtimer_cancel(&adev->amdgpu_vkms_output[i].vblank_hrtimer); | 525 | hrtimer_cancel(&adev->mode_info.crtcs[i]->vblank_timer); |
| 509 | |||
| 510 | kfree(adev->mode_info.bios_hardcoded_edid); | ||
| 511 | kfree(adev->amdgpu_vkms_output); | ||
| 512 | 526 | ||
| 513 | drm_kms_helper_poll_fini(adev_to_drm(adev)); | 527 | drm_kms_helper_poll_fini(adev_to_drm(adev)); |
| 528 | drm_mode_config_cleanup(adev_to_drm(adev)); | ||
| 514 | 529 | ||
| 515 | adev->mode_info.mode_config_initialized = false; | 530 | adev->mode_info.mode_config_initialized = false; |
| 531 | |||
| 532 | kfree(adev->mode_info.bios_hardcoded_edid); | ||
| 533 | kfree(adev->amdgpu_vkms_output); | ||
| 516 | return 0; | 534 | return 0; |
| 517 | } | 535 | } |
| 518 | 536 | ||
| @@ -10,15 +10,14 @@ · 9 unmodified lines | |||
| 10 | #define YRES_MAX 16384 | 10 | #define YRES_MAX 16384 |
| 11 | 11 | ||
| 12 | #define drm_crtc_to_amdgpu_vkms_output(target) \ | 12 | #define drm_crtc_to_amdgpu_vkms_output(target) \ |
| 13 | container_of(target, struct amdgpu_vkms_output, crtc) | 13 | container_of(target, struct amdgpu_vkms_output, crtc.base) |
| 14 | 14 | ||
| 15 | extern const struct amdgpu_ip_block_version amdgpu_vkms_ip_block; | 15 | extern const struct amdgpu_ip_block_version amdgpu_vkms_ip_block; |
| 16 | 16 | ||
| 17 | struct amdgpu_vkms_output { | 17 | struct amdgpu_vkms_output { |
| 18 | struct drm_crtc crtc; | 18 | struct amdgpu_crtc crtc; |
| 19 | struct drm_encoder encoder; | 19 | struct drm_encoder encoder; |
| 20 | struct drm_connector connector; | 20 | struct drm_connector connector; |
| 21 | struct hrtimer vblank_hrtimer; | ||
| 22 | ktime_t period_ns; | 21 | ktime_t period_ns; |
| 23 | struct drm_pending_vblank_event *event; | 22 | struct drm_pending_vblank_event *event; |
| 24 | }; | 23 | }; |
| @@ -53,7 +53,7 @@ · 52 unmodified lines | |||
| 53 | * can be mapped as snooped (cached system pages) or unsnooped | 53 | * can be mapped as snooped (cached system pages) or unsnooped |
| 54 | * (uncached system pages). | 54 | * (uncached system pages). |
| 55 | * Each VM has an ID associated with it and there is a page table | 55 | * Each VM has an ID associated with it and there is a page table |
| 56 | * associated with each VMID. When execting a command buffer, | 56 | * associated with each VMID. When executing a command buffer, |
| 57 | * the kernel tells the the ring what VMID to use for that command | 57 | * the kernel tells the the ring what VMID to use for that command |
| 58 | * buffer. VMIDs are allocated dynamically as commands are submitted. | 58 | * buffer. VMIDs are allocated dynamically as commands are submitted. |
| 59 | * The userspace drivers maintain their own address space and the kernel | 59 | * The userspace drivers maintain their own address space and the kernel |
| @@ -2102,30 +2102,14 @@ static void amdgpu_vm_free_mapping(struct amdgpu_device *adev, · 2042 unmodified lines | |||
| 2102 | static void amdgpu_vm_prt_fini(struct amdgpu_device *adev, struct amdgpu_vm *vm) | 2102 | static void amdgpu_vm_prt_fini(struct amdgpu_device *adev, struct amdgpu_vm *vm) |
| 2103 | { | 2103 | { |
| 2104 | struct dma_resv *resv = vm->root.bo->tbo.base.resv; | 2104 | struct dma_resv *resv = vm->root.bo->tbo.base.resv; |
| 2105 | struct dma_fence *excl, **shared; | 2105 | struct dma_resv_iter cursor; |
| 2106 | unsigned i, shared_count; | 2106 | struct dma_fence *fence; |
| 2107 | int r; | ||
| 2108 | 2107 | ||
| 2109 | r = dma_resv_get_fences(resv, &excl, &shared_count, &shared); | 2108 | dma_resv_for_each_fence(&cursor, resv, true, fence) { |
| 2110 | if (r) { | 2109 | /* Add a callback for each fence in the reservation object */ |
| 2111 | /* Not enough memory to grab the fence list, as last resort | ||
| 2112 | * block for all the fences to complete. | ||
| 2113 | */ | ||
| 2114 | dma_resv_wait_timeout(resv, true, false, | ||
| 2115 | MAX_SCHEDULE_TIMEOUT); | ||
| 2116 | return; | ||
| 2117 | } | ||
| 2118 | |||
| 2119 | /* Add a callback for each fence in the reservation object */ | ||
| 2120 | amdgpu_vm_prt_get(adev); | ||
| 2121 | amdgpu_vm_add_prt_cb(adev, excl); | ||
| 2122 | |||
| 2123 | for (i = 0; i < shared_count; ++i) { | ||
| 2124 | amdgpu_vm_prt_get(adev); | 2110 | amdgpu_vm_prt_get(adev); |
| 2125 | amdgpu_vm_add_prt_cb(adev, shared[i]); | 2111 | amdgpu_vm_add_prt_cb(adev, fence); |
| 2126 | } | 2112 | } |
| 2127 | |||
| 2128 | kfree(shared); | ||
| 2129 | } | 2113 | } |
| 2130 | 2114 | ||
| 2131 | /** | 2115 | /** |
| @@ -265,6 +265,11 @@ static ssize_t amdgpu_xgmi_show_error(struct device *dev, · 264 unmodified lines | |||
| 265 | ficaa_pie_ctl_in = AMDGPU_XGMI_SET_FICAA(0x200); | 265 | ficaa_pie_ctl_in = AMDGPU_XGMI_SET_FICAA(0x200); |
| 266 | ficaa_pie_status_in = AMDGPU_XGMI_SET_FICAA(0x208); | 266 | ficaa_pie_status_in = AMDGPU_XGMI_SET_FICAA(0x208); |
| 267 | 267 | ||
| 268 | if ((!adev->df.funcs) || | ||
| 269 | (!adev->df.funcs->get_fica) || | ||
| 270 | (!adev->df.funcs->set_fica)) | ||
| 271 | return -EINVAL; | ||
| 272 | |||
| 268 | fica_out = adev->df.funcs->get_fica(adev, ficaa_pie_ctl_in); | 273 | fica_out = adev->df.funcs->get_fica(adev, ficaa_pie_ctl_in); |
| 269 | if (fica_out != 0x1f) | 274 | if (fica_out != 0x1f) |
| 270 | pr_err("xGMI error counters not enabled!\n"); | 275 | pr_err("xGMI error counters not enabled!\n"); |
| @@ -469,7 +469,7 @@ int amdgpu_atombios_encoder_get_encoder_mode(struct drm_encoder *encoder) · 468 unmodified lines | |||
| 469 | if (amdgpu_connector->use_digital && | 469 | if (amdgpu_connector->use_digital && |
| 470 | (amdgpu_connector->audio == AMDGPU_AUDIO_ENABLE)) | 470 | (amdgpu_connector->audio == AMDGPU_AUDIO_ENABLE)) |
| 471 | return ATOM_ENCODER_MODE_HDMI; | 471 | return ATOM_ENCODER_MODE_HDMI; |
| 472 | else if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector)) && | 472 | else if (connector->display_info.is_hdmi && |
| 473 | (amdgpu_connector->audio == AMDGPU_AUDIO_AUTO)) | 473 | (amdgpu_connector->audio == AMDGPU_AUDIO_AUTO)) |
| 474 | return ATOM_ENCODER_MODE_HDMI; | 474 | return ATOM_ENCODER_MODE_HDMI; |
| 475 | else if (amdgpu_connector->use_digital) | 475 | else if (amdgpu_connector->use_digital) |
| @@ -488,7 +488,7 @@ int amdgpu_atombios_encoder_get_encoder_mode(struct drm_encoder *encoder) · 12 unmodified lines | |||
| 488 | if (amdgpu_audio != 0) { | 488 | if (amdgpu_audio != 0) { |
| 489 | if (amdgpu_connector->audio == AMDGPU_AUDIO_ENABLE) | 489 | if (amdgpu_connector->audio == AMDGPU_AUDIO_ENABLE) |
| 490 | return ATOM_ENCODER_MODE_HDMI; | 490 | return ATOM_ENCODER_MODE_HDMI; |
| 491 | else if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector)) && | 491 | else if (connector->display_info.is_hdmi && |
| 492 | (amdgpu_connector->audio == AMDGPU_AUDIO_AUTO)) | 492 | (amdgpu_connector->audio == AMDGPU_AUDIO_AUTO)) |
| 493 | return ATOM_ENCODER_MODE_HDMI; | 493 | return ATOM_ENCODER_MODE_HDMI; |
| 494 | else | 494 | else |
| @@ -506,7 +506,7 @@ int amdgpu_atombios_encoder_get_encoder_mode(struct drm_encoder *encoder) · 11 unmodified lines | |||
| 506 | } else if (amdgpu_audio != 0) { | 506 | } else if (amdgpu_audio != 0) { |
| 507 | if (amdgpu_connector->audio == AMDGPU_AUDIO_ENABLE) | 507 | if (amdgpu_connector->audio == AMDGPU_AUDIO_ENABLE) |
| 508 | return ATOM_ENCODER_MODE_HDMI; | 508 | return ATOM_ENCODER_MODE_HDMI; |
| 509 | else if (drm_detect_hdmi_monitor(amdgpu_connector_edid(connector)) && | 509 | else if (connector->display_info.is_hdmi && |
| 510 | (amdgpu_connector->audio == AMDGPU_AUDIO_AUTO)) | 510 | (amdgpu_connector->audio == AMDGPU_AUDIO_AUTO)) |
| 511 | return ATOM_ENCODER_MODE_HDMI; | 511 | return ATOM_ENCODER_MODE_HDMI; |
| 512 | else | 512 | else |
| @@ -2092,22 +2092,18 @@ static int dce_v8_0_pick_dig_encoder(struct drm_encoder *encoder) · 2091 unmodified lines | |||
| 2092 | return 1; | 2092 | return 1; |
| 2093 | else | 2093 | else |
| 2094 | return 0; | 2094 | return 0; |
| 2095 | break; | ||
| 2096 | case ENCODER_OBJECT_ID_INTERNAL_UNIPHY1: | 2095 | case ENCODER_OBJECT_ID_INTERNAL_UNIPHY1: |
| 2097 | if (dig->linkb) | 2096 | if (dig->linkb) |
| 2098 | return 3; | 2097 | return 3; |
| 2099 | else | 2098 | else |
| 2100 | return 2; | 2099 | return 2; |
| 2101 | break; | ||
| 2102 | case ENCODER_OBJECT_ID_INTERNAL_UNIPHY2: | 2100 | case ENCODER_OBJECT_ID_INTERNAL_UNIPHY2: |
| 2103 | if (dig->linkb) | 2101 | if (dig->linkb) |
| 2104 | return 5; | 2102 | return 5; |
| 2105 | else | 2103 | else |
| 2106 | return 4; | 2104 | return 4; |
| 2107 | break; | ||
| 2108 | case ENCODER_OBJECT_ID_INTERNAL_UNIPHY3: | 2105 | case ENCODER_OBJECT_ID_INTERNAL_UNIPHY3: |
| 2109 | return 6; | 2106 | return 6; |
| 2110 | break; | ||
| 2111 | default: | 2107 | default: |
| 2112 | DRM_ERROR("invalid encoder_id: 0x%x\n", amdgpu_encoder->encoder_id); | 2108 | DRM_ERROR("invalid encoder_id: 0x%x\n", amdgpu_encoder->encoder_id); |
| 2113 | return 0; | 2109 | return 0; |
| @@ -63,6 +63,13 @@ · 62 unmodified lines | |||
| 63 | #define mmGCEA_PROBE_MAP 0x070c | 63 | #define mmGCEA_PROBE_MAP 0x070c |
| 64 | #define mmGCEA_PROBE_MAP_BASE_IDX 0 | 64 | #define mmGCEA_PROBE_MAP_BASE_IDX 0 |
| 65 | 65 | ||
| 66 | #define GFX9_RLCG_GC_WRITE_OLD (0x8 << 28) | ||
| 67 | #define GFX9_RLCG_GC_WRITE (0x0 << 28) | ||
| 68 | #define GFX9_RLCG_GC_READ (0x1 << 28) | ||
| 69 | #define GFX9_RLCG_VFGATE_DISABLED 0x4000000 | ||
| 70 | #define GFX9_RLCG_WRONG_OPERATION_TYPE 0x2000000 | ||
| 71 | #define GFX9_RLCG_NOT_IN_RANGE 0x1000000 | ||
| 72 | |||
| 66 | MODULE_FIRMWARE("amdgpu/vega10_ce.bin"); | 73 | MODULE_FIRMWARE("amdgpu/vega10_ce.bin"); |
| 67 | MODULE_FIRMWARE("amdgpu/vega10_pfp.bin"); | 74 | MODULE_FIRMWARE("amdgpu/vega10_pfp.bin"); |
| 68 | MODULE_FIRMWARE("amdgpu/vega10_me.bin"); | 75 | MODULE_FIRMWARE("amdgpu/vega10_me.bin"); |
| @@ -739,7 +746,7 @@ static const u32 GFX_RLC_SRM_INDEX_CNTL_DATA_OFFSETS[] = · 670 unmodified lines | |||
| 739 | mmRLC_SRM_INDEX_CNTL_DATA_7 - mmRLC_SRM_INDEX_CNTL_DATA_0, | 746 | mmRLC_SRM_INDEX_CNTL_DATA_7 - mmRLC_SRM_INDEX_CNTL_DATA_0, |
| 740 | }; | 747 | }; |
| 741 | 748 | ||
| 742 | static void gfx_v9_0_rlcg_w(struct amdgpu_device *adev, u32 offset, u32 v, u32 flag) | 749 | static u32 gfx_v9_0_rlcg_rw(struct amdgpu_device *adev, u32 offset, u32 v, uint32_t flag) |
| 743 | { | 750 | { |
| 744 | static void *scratch_reg0; | 751 | static void *scratch_reg0; |
| 745 | static void *scratch_reg1; | 752 | static void *scratch_reg1; |
| @@ -748,21 +755,20 @@ static void gfx_v9_0_rlcg_w(struct amdgpu_device *adev, u32 offset, u32 v, u32 f · 2 unmodified lines | |||
| 748 | static void *spare_int; | 755 | static void *spare_int; |
| 749 | static uint32_t grbm_cntl; | 756 | static uint32_t grbm_cntl; |
| 750 | static uint32_t grbm_idx; | 757 | static uint32_t grbm_idx; |
| 758 | uint32_t i = 0; | ||
| 759 | uint32_t retries = 50000; | ||
| 760 | u32 ret = 0; | ||
| 761 | u32 tmp; | ||
| 751 | 762 | ||
| 752 | scratch_reg0 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0)*4; | 763 | scratch_reg0 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0)*4; |
| 753 | scratch_reg1 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1)*4; | 764 | scratch_reg1 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1)*4; |
| 754 | scratch_reg2 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG2)*4; | 765 | scratch_reg2 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG2_BASE_IDX] + mmSCRATCH_REG2)*4; |
| 755 | scratch_reg3 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG3)*4; | 766 | scratch_reg3 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG3_BASE_IDX] + mmSCRATCH_REG3)*4; |
| 756 | spare_int = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT)*4; | 767 | spare_int = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT)*4; |
| 757 | 768 | ||
| 758 | grbm_cntl = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_CNTL_BASE_IDX] + mmGRBM_GFX_CNTL; | 769 | grbm_cntl = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_CNTL_BASE_IDX] + mmGRBM_GFX_CNTL; |
| 759 | grbm_idx = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_INDEX_BASE_IDX] + mmGRBM_GFX_INDEX; | 770 | grbm_idx = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_INDEX_BASE_IDX] + mmGRBM_GFX_INDEX; |
| 760 | 771 | ||
| 761 | if (amdgpu_sriov_runtime(adev)) { | ||
| 762 | pr_err("shouldn't call rlcg write register during runtime\n"); | ||
| 763 | return; | ||
| 764 | } | ||
| 765 | |||
| 766 | if (offset == grbm_cntl || offset == grbm_idx) { | 772 | if (offset == grbm_cntl || offset == grbm_idx) { |
| 767 | if (offset == grbm_cntl) | 773 | if (offset == grbm_cntl) |
| 768 | writel(v, scratch_reg2); | 774 | writel(v, scratch_reg2); |
| @@ -771,41 +777,95 @@ static void gfx_v9_0_rlcg_w(struct amdgpu_device *adev, u32 offset, u32 v, u32 f · 2 unmodified lines | |||
| 771 | 777 | ||
| 772 | writel(v, ((void __iomem *)adev->rmmio) + (offset * 4)); | 778 | writel(v, ((void __iomem *)adev->rmmio) + (offset * 4)); |
| 773 | } else { | 779 | } else { |
| 774 | uint32_t i = 0; | 780 | /* |
| 775 | uint32_t retries = 50000; | 781 | * SCRATCH_REG0 = read/write value |
| 776 | 782 | * SCRATCH_REG1[30:28] = command | |
| 783 | * SCRATCH_REG1[19:0] = address in dword | ||
| 784 | * SCRATCH_REG1[26:24] = Error reporting | ||
| 785 | */ | ||
| 777 | writel(v, scratch_reg0); | 786 | writel(v, scratch_reg0); |
| 778 | writel(offset | 0x80000000, scratch_reg1); | 787 | writel(offset | flag, scratch_reg1); |
| 779 | writel(1, spare_int); | 788 | writel(1, spare_int); |
| 780 | for (i = 0; i < retries; i++) { | ||
| 781 | u32 tmp; | ||
| 782 | 789 | ||
| 790 | for (i = 0; i < retries; i++) { | ||
| 783 | tmp = readl(scratch_reg1); | 791 | tmp = readl(scratch_reg1); |
| 784 | if (!(tmp & 0x80000000)) | 792 | if (!(tmp & flag)) |
| 785 | break; | 793 | break; |
| 786 | 794 | ||
| 787 | udelay(10); | 795 | udelay(10); |
| 788 | } | 796 | } |
| 789 | if (i >= retries) | 797 | |
| 790 | pr_err("timeout: rlcg program reg:0x%05x failed !\n", offset); | 798 | if (i >= retries) { |
| 799 | if (amdgpu_sriov_reg_indirect_gc(adev)) { | ||
| 800 | if (tmp & GFX9_RLCG_VFGATE_DISABLED) | ||
| 801 | pr_err("The vfgate is disabled, program reg:0x%05x failed!\n", offset); | ||
| 802 | else if (tmp & GFX9_RLCG_WRONG_OPERATION_TYPE) | ||
| 803 | pr_err("Wrong operation type, program reg:0x%05x failed!\n", offset); | ||
| 804 | else if (tmp & GFX9_RLCG_NOT_IN_RANGE) | ||
| 805 | pr_err("The register is not in range, program reg:0x%05x failed!\n", offset); | ||
| 806 | else | ||
| 807 | pr_err("Unknown error type, program reg:0x%05x failed!\n", offset); | ||
| 808 | } else | ||
| 809 | pr_err("timeout: rlcg program reg:0x%05x failed!\n", offset); | ||
| 810 | } | ||
| 791 | } | 811 | } |
| 792 | 812 | ||
| 813 | ret = readl(scratch_reg0); | ||
| 814 | |||
| 815 | return ret; | ||
| 816 | } | ||
| 817 | |||
| 818 | static bool gfx_v9_0_get_rlcg_flag(struct amdgpu_device *adev, u32 acc_flags, u32 hwip, | ||
| 819 | int write, u32 *rlcg_flag) | ||
| 820 | { | ||
| 821 | |||
| 822 | switch (hwip) { | ||
| 823 | case GC_HWIP: | ||
| 824 | if (amdgpu_sriov_reg_indirect_gc(adev)) { | ||
| 825 | *rlcg_flag = write ? GFX9_RLCG_GC_WRITE : GFX9_RLCG_GC_READ; | ||
| 826 | |||
| 827 | return true; | ||
| 828 | /* only in new version, AMDGPU_REGS_NO_KIQ and AMDGPU_REGS_RLC enabled simultaneously */ | ||
| 829 | } else if ((acc_flags & AMDGPU_REGS_RLC) && !(acc_flags & AMDGPU_REGS_NO_KIQ) && write) { | ||
| 830 | *rlcg_flag = GFX9_RLCG_GC_WRITE_OLD; | ||
| 831 | return true; | ||
| 832 | } | ||
| 833 | |||
| 834 | break; | ||
| 835 | default: | ||
| 836 | return false; | ||
| 837 | } | ||
| 838 | |||
| 839 | return false; | ||
| 840 | } | ||
| 841 | |||
| 842 | static u32 gfx_v9_0_sriov_rreg(struct amdgpu_device *adev, u32 offset, u32 acc_flags, u32 hwip) | ||
| 843 | { | ||
| 844 | u32 rlcg_flag; | ||
| 845 | |||
| 846 | if (!amdgpu_sriov_runtime(adev) && gfx_v9_0_get_rlcg_flag(adev, acc_flags, hwip, 0, &rlcg_flag)) | ||
| 847 | return gfx_v9_0_rlcg_rw(adev, offset, 0, rlcg_flag); | ||
| 848 | |||
| 849 | if (acc_flags & AMDGPU_REGS_NO_KIQ) | ||
| 850 | return RREG32_NO_KIQ(offset); | ||
| 851 | else | ||
| 852 | return RREG32(offset); | ||
| 793 | } | 853 | } |
| 794 | 854 | ||
| 795 | static void gfx_v9_0_sriov_wreg(struct amdgpu_device *adev, u32 offset, | 855 | static void gfx_v9_0_sriov_wreg(struct amdgpu_device *adev, u32 offset, |
| 796 | u32 v, u32 acc_flags, u32 hwip) | 856 | u32 value, u32 acc_flags, u32 hwip) |
| 797 | { | 857 | { |
| 798 | if ((acc_flags & AMDGPU_REGS_RLC) && | 858 | u32 rlcg_flag; |
| 799 | amdgpu_sriov_fullaccess(adev)) { | ||
| 800 | gfx_v9_0_rlcg_w(adev, offset, v, acc_flags); | ||
| 801 | 859 | ||
| 860 | if (!amdgpu_sriov_runtime(adev) && gfx_v9_0_get_rlcg_flag(adev, acc_flags, hwip, 1, &rlcg_flag)) { | ||
| 861 | gfx_v9_0_rlcg_rw(adev, offset, value, rlcg_flag); | ||
| 802 | return; | 862 | return; |
| 803 | } | 863 | } |
| 804 | 864 | ||
| 805 | if (acc_flags & AMDGPU_REGS_NO_KIQ) | 865 | if (acc_flags & AMDGPU_REGS_NO_KIQ) |
| 806 | WREG32_NO_KIQ(offset, v); | 866 | WREG32_NO_KIQ(offset, value); |
| 807 | else | 867 | else |
| 808 | WREG32(offset, v); | 868 | WREG32(offset, value); |
| 809 | } | 869 | } |
| 810 | 870 | ||
| 811 | #define VEGA10_GB_ADDR_CONFIG_GOLDEN 0x2a114042 | 871 | #define VEGA10_GB_ADDR_CONFIG_GOLDEN 0x2a114042 |
| @@ -5135,7 +5195,7 @@ static void gfx_v9_0_update_spm_vmid(struct amdgpu_device *adev, unsigned vmid) · 4323 unmodified lines | |||
| 5135 | if (amdgpu_sriov_is_pp_one_vf(adev)) | 5195 | if (amdgpu_sriov_is_pp_one_vf(adev)) |
| 5136 | data = RREG32_NO_KIQ(reg); | 5196 | data = RREG32_NO_KIQ(reg); |
| 5137 | else | 5197 | else |
| 5138 | data = RREG32(reg); | 5198 | data = RREG32_SOC15(GC, 0, mmRLC_SPM_MC_CNTL); |
| 5139 | 5199 | ||
| 5140 | data &= ~RLC_SPM_MC_CNTL__RLC_SPM_VMID_MASK; | 5200 | data &= ~RLC_SPM_MC_CNTL__RLC_SPM_VMID_MASK; |
| 5141 | data |= (vmid & RLC_SPM_MC_CNTL__RLC_SPM_VMID_MASK) << RLC_SPM_MC_CNTL__RLC_SPM_VMID__SHIFT; | 5201 | data |= (vmid & RLC_SPM_MC_CNTL__RLC_SPM_VMID_MASK) << RLC_SPM_MC_CNTL__RLC_SPM_VMID__SHIFT; |
| @@ -5191,6 +5251,7 @@ static const struct amdgpu_rlc_funcs gfx_v9_0_rlc_funcs = { · 49 unmodified lines | |||
| 5191 | .start = gfx_v9_0_rlc_start, | 5251 | .start = gfx_v9_0_rlc_start, |
| 5192 | .update_spm_vmid = gfx_v9_0_update_spm_vmid, | 5252 | .update_spm_vmid = gfx_v9_0_update_spm_vmid, |
| 5193 | .sriov_wreg = gfx_v9_0_sriov_wreg, | 5253 | .sriov_wreg = gfx_v9_0_sriov_wreg, |
| 5254 | .sriov_rreg = gfx_v9_0_sriov_rreg, | ||
| 5194 | .is_rlcg_access_range = gfx_v9_0_is_rlcg_access_range, | 5255 | .is_rlcg_access_range = gfx_v9_0_is_rlcg_access_range, |
| 5195 | }; | 5256 | }; |
| 5196 | 5257 | ||
| @@ -5796,16 +5857,16 @@ static void gfx_v9_0_set_compute_eop_interrupt_state(struct amdgpu_device *adev, · 599 unmodified lines | |||
| 5796 | 5857 | ||
| 5797 | switch (state) { | 5858 | switch (state) { |
| 5798 | case AMDGPU_IRQ_STATE_DISABLE: | 5859 | case AMDGPU_IRQ_STATE_DISABLE: |
| 5799 | mec_int_cntl = RREG32(mec_int_cntl_reg); | 5860 | mec_int_cntl = RREG32_SOC15_IP(GC,mec_int_cntl_reg); |
| 5800 | mec_int_cntl = REG_SET_FIELD(mec_int_cntl, CP_ME1_PIPE0_INT_CNTL, | 5861 | mec_int_cntl = REG_SET_FIELD(mec_int_cntl, CP_ME1_PIPE0_INT_CNTL, |
| 5801 | TIME_STAMP_INT_ENABLE, 0); | 5862 | TIME_STAMP_INT_ENABLE, 0); |
| 5802 | WREG32(mec_int_cntl_reg, mec_int_cntl); | 5863 | WREG32_SOC15_IP(GC, mec_int_cntl_reg, mec_int_cntl); |
| 5803 | break; | 5864 | break; |
| 5804 | case AMDGPU_IRQ_STATE_ENABLE: | 5865 | case AMDGPU_IRQ_STATE_ENABLE: |
| 5805 | mec_int_cntl = RREG32(mec_int_cntl_reg); | 5866 | mec_int_cntl = RREG32_SOC15_IP(GC, mec_int_cntl_reg); |
| 5806 | mec_int_cntl = REG_SET_FIELD(mec_int_cntl, CP_ME1_PIPE0_INT_CNTL, | 5867 | mec_int_cntl = REG_SET_FIELD(mec_int_cntl, CP_ME1_PIPE0_INT_CNTL, |
| 5807 | TIME_STAMP_INT_ENABLE, 1); | 5868 | TIME_STAMP_INT_ENABLE, 1); |
| 5808 | WREG32(mec_int_cntl_reg, mec_int_cntl); | 5869 | WREG32_SOC15_IP(GC, mec_int_cntl_reg, mec_int_cntl); |
| 5809 | break; | 5870 | break; |
| 5810 | default: | 5871 | default: |
| 5811 | break; | 5872 | break; |
| @@ -107,7 +107,7 @@ static int gmc_v10_0_process_interrupt(struct amdgpu_device *adev, · 106 unmodified lines | |||
| 107 | 107 | ||
| 108 | /* Process it onyl if it's the first fault for this address */ | 108 | /* Process it onyl if it's the first fault for this address */ |
| 109 | if (entry->ih != &adev->irq.ih_soft && | 109 | if (entry->ih != &adev->irq.ih_soft && |
| 110 | amdgpu_gmc_filter_faults(adev, addr, entry->pasid, | 110 | amdgpu_gmc_filter_faults(adev, entry->ih, addr, entry->pasid, |
| 111 | entry->timestamp)) | 111 | entry->timestamp)) |
| 112 | return 1; | 112 | return 1; |
| 113 | 113 | ||
| @@ -914,12 +914,6 @@ static int gmc_v10_0_sw_init(void *handle) · 800 unmodified lines | |||
| 914 | return r; | 914 | return r; |
| 915 | } | 915 | } |
| 916 | 916 | ||
| 917 | if (adev->gmc.xgmi.supported) { | ||
| 918 | r = adev->gfxhub.funcs->get_xgmi_info(adev); | ||
| 919 | if (r) | ||
| 920 | return r; | ||
| 921 | } | ||
| 922 | |||
| 923 | r = gmc_v10_0_mc_init(adev); | 917 | r = gmc_v10_0_mc_init(adev); |
| 924 | if (r) | 918 | if (r) |
| 925 | return r; | 919 | return r; |
| @@ -992,10 +986,14 @@ static int gmc_v10_0_gart_enable(struct amdgpu_device *adev) · 66 unmodified lines | |||
| 992 | return -EINVAL; | 986 | return -EINVAL; |
| 993 | } | 987 | } |
| 994 | 988 | ||
| 989 | if (amdgpu_sriov_vf(adev) && amdgpu_in_reset(adev)) | ||
| 990 | goto skip_pin_bo; | ||
| 991 | |||
| 995 | r = amdgpu_gart_table_vram_pin(adev); | 992 | r = amdgpu_gart_table_vram_pin(adev); |
| 996 | if (r) | 993 | if (r) |
| 997 | return r; | 994 | return r; |
| 998 | 995 | ||
| 996 | skip_pin_bo: | ||
| 999 | r = adev->gfxhub.funcs->gart_enable(adev); | 997 | r = adev->gfxhub.funcs->gart_enable(adev); |
| 1000 | if (r) | 998 | if (r) |
| 1001 | return r; | 999 | return r; |
| @@ -515,10 +515,10 @@ static void gmc_v8_0_mc_program(struct amdgpu_device *adev) · 514 unmodified lines | |||
| 515 | static int gmc_v8_0_mc_init(struct amdgpu_device *adev) | 515 | static int gmc_v8_0_mc_init(struct amdgpu_device *adev) |
| 516 | { | 516 | { |
| 517 | int r; | 517 | int r; |
| 518 | u32 tmp; | ||
| 518 | 519 | ||
| 519 | adev->gmc.vram_width = amdgpu_atombios_get_vram_width(adev); | 520 | adev->gmc.vram_width = amdgpu_atombios_get_vram_width(adev); |
| 520 | if (!adev->gmc.vram_width) { | 521 | if (!adev->gmc.vram_width) { |
| 521 | u32 tmp; | ||
| 522 | int chansize, numchan; | 522 | int chansize, numchan; |
| 523 | 523 | ||
| 524 | /* Get VRAM informations */ | 524 | /* Get VRAM informations */ |
| @@ -562,8 +562,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) · 37 unmodified lines | |||
| 562 | adev->gmc.vram_width = numchan * chansize; | 562 | adev->gmc.vram_width = numchan * chansize; |
| 563 | } | 563 | } |
| 564 | /* size in MB on si */ | 564 | /* size in MB on si */ |
| 565 | adev->gmc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; | 565 | tmp = RREG32(mmCONFIG_MEMSIZE); |
| 566 | adev->gmc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; | 566 | /* some boards may have garbage in the upper 16 bits */ |
| 567 | if (tmp & 0xffff0000) { | ||
| 568 | DRM_INFO("Probable bad vram size: 0x%08x\n", tmp); | ||
| 569 | if (tmp & 0xffff) | ||
| 570 | tmp &= 0xffff; | ||
| 571 | } | ||
| 572 | adev->gmc.mc_vram_size = tmp * 1024ULL * 1024ULL; | ||
| 573 | adev->gmc.real_vram_size = adev->gmc.mc_vram_size; | ||
| 567 | 574 | ||
| 568 | if (!(adev->flags & AMD_IS_APU)) { | 575 | if (!(adev->flags & AMD_IS_APU)) { |
| 569 | r = amdgpu_device_resize_fb_bar(adev); | 576 | r = amdgpu_device_resize_fb_bar(adev); |
| @@ -478,9 +478,18 @@ static int gmc_v9_0_vm_fault_interrupt_state(struct amdgpu_device *adev, · 477 unmodified lines | |||
| 478 | hub = &adev->vmhub[j]; | 478 | hub = &adev->vmhub[j]; |
| 479 | for (i = 0; i < 16; i++) { | 479 | for (i = 0; i < 16; i++) { |
| 480 | reg = hub->vm_context0_cntl + i; | 480 | reg = hub->vm_context0_cntl + i; |
| 481 | tmp = RREG32(reg); | 481 | |
| 482 | if (j == AMDGPU_GFXHUB_0) | ||
| 483 | tmp = RREG32_SOC15_IP(GC, reg); | ||
| 484 | else | ||
| 485 | tmp = RREG32_SOC15_IP(MMHUB, reg); | ||
| 486 | |||
| 482 | tmp &= ~bits; | 487 | tmp &= ~bits; |
| 483 | WREG32(reg, tmp); | 488 | |
| 489 | if (j == AMDGPU_GFXHUB_0) | ||
| 490 | WREG32_SOC15_IP(GC, reg, tmp); | ||
| 491 | else | ||
| 492 | WREG32_SOC15_IP(MMHUB, reg, tmp); | ||
| 484 | } | 493 | } |
| 485 | } | 494 | } |
| 486 | break; | 495 | break; |
| @@ -489,9 +498,18 @@ static int gmc_v9_0_vm_fault_interrupt_state(struct amdgpu_device *adev, · 2 unmodified lines | |||
| 489 | hub = &adev->vmhub[j]; | 498 | hub = &adev->vmhub[j]; |
| 490 | for (i = 0; i < 16; i++) { | 499 | for (i = 0; i < 16; i++) { |
| 491 | reg = hub->vm_context0_cntl + i; | 500 | reg = hub->vm_context0_cntl + i; |
| 492 | tmp = RREG32(reg); | 501 | |
| 502 | if (j == AMDGPU_GFXHUB_0) | ||
| 503 | tmp = RREG32_SOC15_IP(GC, reg); | ||
| 504 | else | ||
| 505 | tmp = RREG32_SOC15_IP(MMHUB, reg); | ||
| 506 | |||
| 493 | tmp |= bits; | 507 | tmp |= bits; |
| 494 | WREG32(reg, tmp); | 508 | |
| 509 | if (j == AMDGPU_GFXHUB_0) | ||
| 510 | WREG32_SOC15_IP(GC, reg, tmp); | ||
| 511 | else | ||
| 512 | WREG32_SOC15_IP(MMHUB, reg, tmp); | ||
| 495 | } | 513 | } |
| 496 | } | 514 | } |
| 497 | break; | 515 | break; |
| @@ -523,7 +541,7 @@ static int gmc_v9_0_process_interrupt(struct amdgpu_device *adev, · 25 unmodified lines | |||
| 523 | 541 | ||
| 524 | /* Process it onyl if it's the first fault for this address */ | 542 | /* Process it onyl if it's the first fault for this address */ |
| 525 | if (entry->ih != &adev->irq.ih_soft && | 543 | if (entry->ih != &adev->irq.ih_soft && |
| 526 | amdgpu_gmc_filter_faults(adev, addr, entry->pasid, | 544 | amdgpu_gmc_filter_faults(adev, entry->ih, addr, entry->pasid, |
| 527 | entry->timestamp)) | 545 | entry->timestamp)) |
| 528 | return 1; | 546 | return 1; |
| 529 | 547 | ||
| @@ -788,9 +806,12 @@ static void gmc_v9_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid, · 258 unmodified lines | |||
| 788 | /* TODO: It needs to continue working on debugging with semaphore for GFXHUB as well. */ | 806 | /* TODO: It needs to continue working on debugging with semaphore for GFXHUB as well. */ |
| 789 | if (use_semaphore) { | 807 | if (use_semaphore) { |
| 790 | for (j = 0; j < adev->usec_timeout; j++) { | 808 | for (j = 0; j < adev->usec_timeout; j++) { |
| 791 | /* a read return value of 1 means semaphore acuqire */ | 809 | /* a read return value of 1 means semaphore acquire */ |
| 792 | tmp = RREG32_NO_KIQ(hub->vm_inv_eng0_sem + | 810 | if (vmhub == AMDGPU_GFXHUB_0) |
| 793 | hub->eng_distance * eng); | 811 | tmp = RREG32_SOC15_IP_NO_KIQ(GC, hub->vm_inv_eng0_sem + hub->eng_distance * eng); |
| 812 | else | ||
| 813 | tmp = RREG32_SOC15_IP_NO_KIQ(MMHUB, hub->vm_inv_eng0_sem + hub->eng_distance * eng); | ||
| 814 | |||
| 794 | if (tmp & 0x1) | 815 | if (tmp & 0x1) |
| 795 | break; | 816 | break; |
| 796 | udelay(1); | 817 | udelay(1); |
| @@ -801,8 +822,10 @@ static void gmc_v9_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid, · 4 unmodified lines | |||
| 801 | } | 822 | } |
| 802 | 823 | ||
| 803 | do { | 824 | do { |
| 804 | WREG32_NO_KIQ(hub->vm_inv_eng0_req + | 825 | if (vmhub == AMDGPU_GFXHUB_0) |
| 805 | hub->eng_distance * eng, inv_req); | 826 | WREG32_SOC15_IP_NO_KIQ(GC, hub->vm_inv_eng0_req + hub->eng_distance * eng, inv_req); |
| 827 | else | ||
| 828 | WREG32_SOC15_IP_NO_KIQ(MMHUB, hub->vm_inv_eng0_req + hub->eng_distance * eng, inv_req); | ||
| 806 | 829 | ||
| 807 | /* | 830 | /* |
| 808 | * Issue a dummy read to wait for the ACK register to | 831 | * Issue a dummy read to wait for the ACK register to |
| @@ -815,8 +838,11 @@ static void gmc_v9_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid, · 6 unmodified lines | |||
| 815 | hub->eng_distance * eng); | 838 | hub->eng_distance * eng); |
| 816 | 839 | ||
| 817 | for (j = 0; j < adev->usec_timeout; j++) { | 840 | for (j = 0; j < adev->usec_timeout; j++) { |
| 818 | tmp = RREG32_NO_KIQ(hub->vm_inv_eng0_ack + | 841 | if (vmhub == AMDGPU_GFXHUB_0) |
| 819 | hub->eng_distance * eng); | 842 | tmp = RREG32_SOC15_IP_NO_KIQ(GC, hub->vm_inv_eng0_ack + hub->eng_distance * eng); |
| 843 | else | ||
| 844 | tmp = RREG32_SOC15_IP_NO_KIQ(MMHUB, hub->vm_inv_eng0_ack + hub->eng_distance * eng); | ||
| 845 | |||
| 820 | if (tmp & (1 << vmid)) | 846 | if (tmp & (1 << vmid)) |
| 821 | break; | 847 | break; |
| 822 | udelay(1); | 848 | udelay(1); |
| @@ -827,13 +853,16 @@ static void gmc_v9_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid, · 4 unmodified lines | |||
| 827 | } while (inv_req); | 853 | } while (inv_req); |
| 828 | 854 | ||
| 829 | /* TODO: It needs to continue working on debugging with semaphore for GFXHUB as well. */ | 855 | /* TODO: It needs to continue working on debugging with semaphore for GFXHUB as well. */ |
| 830 | if (use_semaphore) | 856 | if (use_semaphore) { |
| 831 | /* | 857 | /* |
| 832 | * add semaphore release after invalidation, | 858 | * add semaphore release after invalidation, |
| 833 | * write with 0 means semaphore release | 859 | * write with 0 means semaphore release |
| 834 | */ | 860 | */ |
| 835 | WREG32_NO_KIQ(hub->vm_inv_eng0_sem + | 861 | if (vmhub == AMDGPU_GFXHUB_0) |
| 836 | hub->eng_distance * eng, 0); | 862 | WREG32_SOC15_IP_NO_KIQ(GC, hub->vm_inv_eng0_sem + hub->eng_distance * eng, 0); |
| 863 | else | ||
| 864 | WREG32_SOC15_IP_NO_KIQ(MMHUB, hub->vm_inv_eng0_sem + hub->eng_distance * eng, 0); | ||
| 865 | } | ||
| 837 | 866 | ||
| 838 | spin_unlock(&adev->gmc.invalidate_lock); | 867 | spin_unlock(&adev->gmc.invalidate_lock); |
| 839 | 868 | ||
| @@ -1294,7 +1323,8 @@ static int gmc_v9_0_late_init(void *handle) · 454 unmodified lines | |||
| 1294 | if (!amdgpu_sriov_vf(adev) && | 1323 | if (!amdgpu_sriov_vf(adev) && |
| 1295 | (adev->ip_versions[UMC_HWIP][0] == IP_VERSION(6, 0, 0))) { | 1324 | (adev->ip_versions[UMC_HWIP][0] == IP_VERSION(6, 0, 0))) { |
| 1296 | if (!(adev->ras_enabled & (1 << AMDGPU_RAS_BLOCK__UMC))) { | 1325 | if (!(adev->ras_enabled & (1 << AMDGPU_RAS_BLOCK__UMC))) { |
| 1297 | if (adev->df.funcs->enable_ecc_force_par_wr_rmw) | 1326 | if (adev->df.funcs && |
| 1327 | adev->df.funcs->enable_ecc_force_par_wr_rmw) | ||
| 1298 | adev->df.funcs->enable_ecc_force_par_wr_rmw(adev, false); | 1328 | adev->df.funcs->enable_ecc_force_par_wr_rmw(adev, false); |
| 1299 | } | 1329 | } |
| 1300 | } | 1330 | } |
| @@ -1505,9 +1535,11 @@ static int gmc_v9_0_sw_init(void *handle) · 204 unmodified lines | |||
| 1505 | chansize = 64; | 1535 | chansize = 64; |
| 1506 | else | 1536 | else |
| 1507 | chansize = 128; | 1537 | chansize = 128; |
| 1508 | 1538 | if (adev->df.funcs && | |
| 1509 | numchan = adev->df.funcs->get_hbm_channel_number(adev); | 1539 | adev->df.funcs->get_hbm_channel_number) { |
| 1510 | adev->gmc.vram_width = numchan * chansize; | 1540 | numchan = adev->df.funcs->get_hbm_channel_number(adev); |
| 1541 | adev->gmc.vram_width = numchan * chansize; | ||
| 1542 | } | ||
| 1511 | } | 1543 | } |
| 1512 | 1544 | ||
| 1513 | adev->gmc.vram_type = vram_type; | 1545 | adev->gmc.vram_type = vram_type; |
| @@ -1596,12 +1628,6 @@ static int gmc_v9_0_sw_init(void *handle) · 82 unmodified lines | |||
| 1596 | } | 1628 | } |
| 1597 | adev->need_swiotlb = drm_need_swiotlb(44); | 1629 | adev->need_swiotlb = drm_need_swiotlb(44); |
| 1598 | 1630 | ||
| 1599 | if (adev->gmc.xgmi.supported) { | ||
| 1600 | r = adev->gfxhub.funcs->get_xgmi_info(adev); | ||
| 1601 | if (r) | ||
| 1602 | return r; | ||
| 1603 | } | ||
| 1604 | |||
| 1605 | r = gmc_v9_0_mc_init(adev); | 1631 | r = gmc_v9_0_mc_init(adev); |
| 1606 | if (r) | 1632 | if (r) |
| 1607 | return r; | 1633 | return r; |
| @@ -1714,10 +1740,14 @@ static int gmc_v9_0_gart_enable(struct amdgpu_device *adev) · 106 unmodified lines | |||
| 1714 | return -EINVAL; | 1740 | return -EINVAL; |
| 1715 | } | 1741 | } |
| 1716 | 1742 | ||
| 1743 | if (amdgpu_sriov_vf(adev) && amdgpu_in_reset(adev)) | ||
| 1744 | goto skip_pin_bo; | ||
| 1745 | |||
| 1717 | r = amdgpu_gart_table_vram_pin(adev); | 1746 | r = amdgpu_gart_table_vram_pin(adev); |
| 1718 | if (r) | 1747 | if (r) |
| 1719 | return r; | 1748 | return r; |
| 1720 | 1749 | ||
| 1750 | skip_pin_bo: | ||
| 1721 | r = adev->gfxhub.funcs->gart_enable(adev); | 1751 | r = adev->gfxhub.funcs->gart_enable(adev); |
| 1722 | if (r) | 1752 | if (r) |
| 1723 | return r; | 1753 | return r; |
| @@ -1742,7 +1772,7 @@ static int gmc_v9_0_hw_init(void *handle) · 18 unmodified lines | |||
| 1742 | { | 1772 | { |
| 1743 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; | 1773 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 1744 | bool value; | 1774 | bool value; |
| 1745 | int r, i; | 1775 | int i; |
| 1746 | 1776 | ||
| 1747 | /* The sequence of these two function calls matters.*/ | 1777 | /* The sequence of these two function calls matters.*/ |
| 1748 | gmc_v9_0_init_golden_registers(adev); | 1778 | gmc_v9_0_init_golden_registers(adev); |
| @@ -1777,9 +1807,7 @@ static int gmc_v9_0_hw_init(void *handle) · 28 unmodified lines | |||
| 1777 | if (adev->umc.funcs && adev->umc.funcs->init_registers) | 1807 | if (adev->umc.funcs && adev->umc.funcs->init_registers) |
| 1778 | adev->umc.funcs->init_registers(adev); | 1808 | adev->umc.funcs->init_registers(adev); |
| 1779 | 1809 | ||
| 1780 | r = gmc_v9_0_gart_enable(adev); | 1810 | return gmc_v9_0_gart_enable(adev); |
| 1781 | |||
| 1782 | return r; | ||
| 1783 | } | 1811 | } |
| 1784 | 1812 | ||
| 1785 | /** | 1813 | /** |
| @@ -252,11 +252,12 @@ static void xgpu_ai_mailbox_flr_work(struct work_struct *work) · 251 unmodified lines | |||
| 252 | * otherwise the mailbox msg will be ruined/reseted by | 252 | * otherwise the mailbox msg will be ruined/reseted by |
| 253 | * the VF FLR. | 253 | * the VF FLR. |
| 254 | */ | 254 | */ |
| 255 | if (!down_write_trylock(&adev->reset_sem)) | 255 | if (atomic_cmpxchg(&adev->in_gpu_reset, 0, 1) != 0) |
| 256 | return; | 256 | return; |
| 257 | 257 | ||
| 258 | down_write(&adev->reset_sem); | ||
| 259 | |||
| 258 | amdgpu_virt_fini_data_exchange(adev); | 260 | amdgpu_virt_fini_data_exchange(adev); |
| 259 | atomic_set(&adev->in_gpu_reset, 1); | ||
| 260 | 261 | ||
| 261 | xgpu_ai_mailbox_trans_msg(adev, IDH_READY_TO_RESET, 0, 0, 0); | 262 | xgpu_ai_mailbox_trans_msg(adev, IDH_READY_TO_RESET, 0, 0, 0); |
| 262 | 263 | ||
| @@ -26,7 +26,7 @@ · 25 unmodified lines | |||
| 26 | 26 | ||
| 27 | #define AI_MAILBOX_POLL_ACK_TIMEDOUT 500 | 27 | #define AI_MAILBOX_POLL_ACK_TIMEDOUT 500 |
| 28 | #define AI_MAILBOX_POLL_MSG_TIMEDOUT 6000 | 28 | #define AI_MAILBOX_POLL_MSG_TIMEDOUT 6000 |
| 29 | #define AI_MAILBOX_POLL_FLR_TIMEDOUT 5000 | 29 | #define AI_MAILBOX_POLL_FLR_TIMEDOUT 10000 |
| 30 | #define AI_MAILBOX_POLL_MSG_REP_MAX 11 | 30 | #define AI_MAILBOX_POLL_MSG_REP_MAX 11 |
| 31 | 31 | ||
| 32 | enum idh_request { | 32 | enum idh_request { |
| @@ -281,11 +281,12 @@ static void xgpu_nv_mailbox_flr_work(struct work_struct *work) · 280 unmodified lines | |||
| 281 | * otherwise the mailbox msg will be ruined/reseted by | 281 | * otherwise the mailbox msg will be ruined/reseted by |
| 282 | * the VF FLR. | 282 | * the VF FLR. |
| 283 | */ | 283 | */ |
| 284 | if (!down_write_trylock(&adev->reset_sem)) | 284 | if (atomic_cmpxchg(&adev->in_gpu_reset, 0, 1) != 0) |
| 285 | return; | 285 | return; |
| 286 | 286 | ||
| 287 | down_write(&adev->reset_sem); | ||
| 288 | |||
| 287 | amdgpu_virt_fini_data_exchange(adev); | 289 | amdgpu_virt_fini_data_exchange(adev); |
| 288 | atomic_set(&adev->in_gpu_reset, 1); | ||
| 289 | 290 | ||
| 290 | xgpu_nv_mailbox_trans_msg(adev, IDH_READY_TO_RESET, 0, 0, 0); | 291 | xgpu_nv_mailbox_trans_msg(adev, IDH_READY_TO_RESET, 0, 0, 0); |
| 291 | 292 | ||
| @@ -716,6 +716,7 @@ static const struct amd_ip_funcs navi10_ih_ip_funcs = { · 715 unmodified lines | |||
| 716 | static const struct amdgpu_ih_funcs navi10_ih_funcs = { | 716 | static const struct amdgpu_ih_funcs navi10_ih_funcs = { |
| 717 | .get_wptr = navi10_ih_get_wptr, | 717 | .get_wptr = navi10_ih_get_wptr, |
| 718 | .decode_iv = amdgpu_ih_decode_iv_helper, | 718 | .decode_iv = amdgpu_ih_decode_iv_helper, |
| 719 | .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, | ||
| 719 | .set_rptr = navi10_ih_set_rptr | 720 | .set_rptr = navi10_ih_set_rptr |
| 720 | }; | 721 | }; |
| 721 | 722 | ||
| @@ -277,13 +277,15 @@ static bool psp_v11_0_is_sos_alive(struct psp_context *psp) · 276 unmodified lines | |||
| 277 | return sol_reg != 0x0; | 277 | return sol_reg != 0x0; |
| 278 | } | 278 | } |
| 279 | 279 | ||
| 280 | static int psp_v11_0_bootloader_load_kdb(struct psp_context *psp) | 280 | static int psp_v11_0_bootloader_load_component(struct psp_context *psp, |
| 281 | struct psp_bin_desc *bin_desc, | ||
| 282 | enum psp_bootloader_cmd bl_cmd) | ||
| 281 | { | 283 | { |
| 282 | int ret; | 284 | int ret; |
| 283 | uint32_t psp_gfxdrv_command_reg = 0; | 285 | uint32_t psp_gfxdrv_command_reg = 0; |
| 284 | struct amdgpu_device *adev = psp->adev; | 286 | struct amdgpu_device *adev = psp->adev; |
| 285 | 287 | ||
| 286 | /* Check tOS sign of life register to confirm sys driver and sOS | 288 | /* Check sOS sign of life register to confirm sys driver and sOS |
| 287 | * are already been loaded. | 289 | * are already been loaded. |
| 288 | */ | 290 | */ |
| 289 | if (psp_v11_0_is_sos_alive(psp)) | 291 | if (psp_v11_0_is_sos_alive(psp)) |
| @@ -293,13 +295,13 @@ static int psp_v11_0_bootloader_load_kdb(struct psp_context *psp) · 3 unmodified lines | |||
| 293 | if (ret) | 295 | if (ret) |
| 294 | return ret; | 296 | return ret; |
| 295 | 297 | ||
| 296 | /* Copy PSP KDB binary to memory */ | 298 | /* Copy PSP System Driver binary to memory */ |
| 297 | psp_copy_fw(psp, psp->kdb.start_addr, psp->kdb.size_bytes); | 299 | psp_copy_fw(psp, bin_desc->start_addr, bin_desc->size_bytes); |
| 298 | 300 | ||
| 299 | /* Provide the PSP KDB to bootloader */ | 301 | /* Provide the sys driver to bootloader */ |
| 300 | WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, | 302 | WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, |
| 301 | (uint32_t)(psp->fw_pri_mc_addr >> 20)); | 303 | (uint32_t)(psp->fw_pri_mc_addr >> 20)); |
| 302 | psp_gfxdrv_command_reg = PSP_BL__LOAD_KEY_DATABASE; | 304 | psp_gfxdrv_command_reg = bl_cmd; |
| 303 | WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_35, | 305 | WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_35, |
| 304 | psp_gfxdrv_command_reg); | 306 | psp_gfxdrv_command_reg); |
| 305 | 307 | ||
| @@ -308,69 +310,19 @@ static int psp_v11_0_bootloader_load_kdb(struct psp_context *psp) · 2 unmodified lines | |||
| 308 | return ret; | 310 | return ret; |
| 309 | } | 311 | } |
| 310 | 312 | ||
| 311 | static int psp_v11_0_bootloader_load_spl(struct psp_context *psp) | 313 | static int psp_v11_0_bootloader_load_kdb(struct psp_context *psp) |
| 312 | { | 314 | { |
| 313 | int ret; | 315 | return psp_v11_0_bootloader_load_component(psp, &psp->kdb, PSP_BL__LOAD_KEY_DATABASE); |
| 314 | uint32_t psp_gfxdrv_command_reg = 0; | 316 | } |
| 315 | struct amdgpu_device *adev = psp->adev; | ||
| 316 | |||
| 317 | /* Check tOS sign of life register to confirm sys driver and sOS | ||
| 318 | * are already been loaded. | ||
| 319 | */ | ||
| 320 | if (psp_v11_0_is_sos_alive(psp)) | ||
| 321 | return 0; | ||
| 322 | |||
| 323 | ret = psp_v11_0_wait_for_bootloader(psp); | ||
| 324 | if (ret) | ||
| 325 | return ret; | ||
| 326 | |||
| 327 | /* Copy PSP SPL binary to memory */ | ||
| 328 | psp_copy_fw(psp, psp->spl.start_addr, psp->spl.size_bytes); | ||
| 329 | |||
| 330 | /* Provide the PSP SPL to bootloader */ | ||
| 331 | WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, | ||
| 332 | (uint32_t)(psp->fw_pri_mc_addr >> 20)); | ||
| 333 | psp_gfxdrv_command_reg = PSP_BL__LOAD_TOS_SPL_TABLE; | ||
| 334 | WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_35, | ||
| 335 | psp_gfxdrv_command_reg); | ||
| 336 | |||
| 337 | ret = psp_v11_0_wait_for_bootloader(psp); | ||
| 338 | 317 | ||
| 339 | return ret; | 318 | static int psp_v11_0_bootloader_load_spl(struct psp_context *psp) |
| 319 | { | ||
| 320 | return psp_v11_0_bootloader_load_component(psp, &psp->spl, PSP_BL__LOAD_TOS_SPL_TABLE); | ||
| 340 | } | 321 | } |
| 341 | 322 | ||
| 342 | static int psp_v11_0_bootloader_load_sysdrv(struct psp_context *psp) | 323 | static int psp_v11_0_bootloader_load_sysdrv(struct psp_context *psp) |
| 343 | { | 324 | { |
| 344 | int ret; | 325 | return psp_v11_0_bootloader_load_component(psp, &psp->sys, PSP_BL__LOAD_SYSDRV); |
| 345 | uint32_t psp_gfxdrv_command_reg = 0; | ||
| 346 | struct amdgpu_device *adev = psp->adev; | ||
| 347 | |||
| 348 | /* Check sOS sign of life register to confirm sys driver and sOS | ||
| 349 | * are already been loaded. | ||
| 350 | */ | ||
| 351 | if (psp_v11_0_is_sos_alive(psp)) | ||
| 352 | return 0; | ||
| 353 | |||
| 354 | ret = psp_v11_0_wait_for_bootloader(psp); | ||
| 355 | if (ret) | ||
| 356 | return ret; | ||
| 357 | |||
| 358 | /* Copy PSP System Driver binary to memory */ | ||
| 359 | psp_copy_fw(psp, psp->sys.start_addr, psp->sys.size_bytes); | ||
| 360 | |||
| 361 | /* Provide the sys driver to bootloader */ | ||
| 362 | WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, | ||
| 363 | (uint32_t)(psp->fw_pri_mc_addr >> 20)); | ||
| 364 | psp_gfxdrv_command_reg = PSP_BL__LOAD_SYSDRV; | ||
| 365 | WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_35, | ||
| 366 | psp_gfxdrv_command_reg); | ||
| 367 | |||
| 368 | /* there might be handshake issue with hardware which needs delay */ | ||
| 369 | mdelay(20); | ||
| 370 | |||
| 371 | ret = psp_v11_0_wait_for_bootloader(psp); | ||
| 372 | |||
| 373 | return ret; | ||
| 374 | } | 326 | } |
| 375 | 327 | ||
| 376 | static int psp_v11_0_bootloader_load_sos(struct psp_context *psp) | 328 | static int psp_v11_0_bootloader_load_sos(struct psp_context *psp) |
| @@ -481,8 +481,6 @@ static void sdma_v5_0_ring_emit_ib(struct amdgpu_ring *ring, · 480 unmodified lines | |||
| 481 | * sdma_v5_0_ring_emit_mem_sync - flush the IB by graphics cache rinse | 481 | * sdma_v5_0_ring_emit_mem_sync - flush the IB by graphics cache rinse |
| 482 | * | 482 | * |
| 483 | * @ring: amdgpu ring pointer | 483 | * @ring: amdgpu ring pointer |
| 484 | * @job: job to retrieve vmid from | ||
| 485 | * @ib: IB object to schedule | ||
| 486 | * | 484 | * |
| 487 | * flush the IB by graphics cache rinse. | 485 | * flush the IB by graphics cache rinse. |
| 488 | */ | 486 | */ |
| @@ -368,8 +368,6 @@ static void sdma_v5_2_ring_emit_ib(struct amdgpu_ring *ring, · 367 unmodified lines | |||
| 368 | * sdma_v5_2_ring_emit_mem_sync - flush the IB by graphics cache rinse | 368 | * sdma_v5_2_ring_emit_mem_sync - flush the IB by graphics cache rinse |
| 369 | * | 369 | * |
| 370 | * @ring: amdgpu ring pointer | 370 | * @ring: amdgpu ring pointer |
| 371 | * @job: job to retrieve vmid from | ||
| 372 | * @ib: IB object to schedule | ||
| 373 | * | 371 | * |
| 374 | * flush the IB by graphics cache rinse. | 372 | * flush the IB by graphics cache rinse. |
| 375 | */ | 373 | */ |
| @@ -544,9 +542,6 @@ static void sdma_v5_2_ctx_switch_enable(struct amdgpu_device *adev, bool enable) · 168 unmodified lines | |||
| 544 | } | 542 | } |
| 545 | 543 | ||
| 546 | for (i = 0; i < adev->sdma.num_instances; i++) { | 544 | for (i = 0; i < adev->sdma.num_instances; i++) { |
| 547 | f32_cntl = RREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL)); | ||
| 548 | f32_cntl = REG_SET_FIELD(f32_cntl, SDMA0_CNTL, | ||
| 549 | AUTO_CTXSW_ENABLE, enable ? 1 : 0); | ||
| 550 | if (enable && amdgpu_sdma_phase_quantum) { | 545 | if (enable && amdgpu_sdma_phase_quantum) { |
| 551 | WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_PHASE0_QUANTUM), | 546 | WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_PHASE0_QUANTUM), |
| 552 | phase_quantum); | 547 | phase_quantum); |
| @@ -555,7 +550,13 @@ static void sdma_v5_2_ctx_switch_enable(struct amdgpu_device *adev, bool enable) · 2 unmodified lines | |||
| 555 | WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_PHASE2_QUANTUM), | 550 | WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_PHASE2_QUANTUM), |
| 556 | phase_quantum); | 551 | phase_quantum); |
| 557 | } | 552 | } |
| 558 | WREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL), f32_cntl); | 553 | |
| 554 | if (!amdgpu_sriov_vf(adev)) { | ||
| 555 | f32_cntl = RREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL)); | ||
| 556 | f32_cntl = REG_SET_FIELD(f32_cntl, SDMA0_CNTL, | ||
| 557 | AUTO_CTXSW_ENABLE, enable ? 1 : 0); | ||
| 558 | WREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL), f32_cntl); | ||
| 559 | } | ||
| 559 | } | 560 | } |
| 560 | 561 | ||
| 561 | } | 562 | } |
| @@ -578,10 +579,12 @@ static void sdma_v5_2_enable(struct amdgpu_device *adev, bool enable) · 16 unmodified lines | |||
| 578 | sdma_v5_2_rlc_stop(adev); | 579 | sdma_v5_2_rlc_stop(adev); |
| 579 | } | 580 | } |
| 580 | 581 | ||
| 581 | for (i = 0; i < adev->sdma.num_instances; i++) { | 582 | if (!amdgpu_sriov_vf(adev)) { |
| 582 | f32_cntl = RREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_F32_CNTL)); | 583 | for (i = 0; i < adev->sdma.num_instances; i++) { |
| 583 | f32_cntl = REG_SET_FIELD(f32_cntl, SDMA0_F32_CNTL, HALT, enable ? 0 : 1); | 584 | f32_cntl = RREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_F32_CNTL)); |
| 584 | WREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_F32_CNTL), f32_cntl); | 585 | f32_cntl = REG_SET_FIELD(f32_cntl, SDMA0_F32_CNTL, HALT, enable ? 0 : 1); |
| 586 | WREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_F32_CNTL), f32_cntl); | ||
| 587 | } | ||
| 585 | } | 588 | } |
| 586 | } | 589 | } |
| 587 | 590 | ||
| @@ -610,7 +613,8 @@ static int sdma_v5_2_gfx_resume(struct amdgpu_device *adev) · 22 unmodified lines | |||
| 610 | ring = &adev->sdma.instance[i].ring; | 613 | ring = &adev->sdma.instance[i].ring; |
| 611 | wb_offset = (ring->rptr_offs * 4); | 614 | wb_offset = (ring->rptr_offs * 4); |
| 612 | 615 | ||
| 613 | WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_SEM_WAIT_FAIL_TIMER_CNTL), 0); | 616 | if (!amdgpu_sriov_vf(adev)) |
| 617 | WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_SEM_WAIT_FAIL_TIMER_CNTL), 0); | ||
| 614 | 618 | ||
| 615 | /* Set ring buffer size in dwords */ | 619 | /* Set ring buffer size in dwords */ |
| 616 | rb_bufsz = order_base_2(ring->ring_size / 4); | 620 | rb_bufsz = order_base_2(ring->ring_size / 4); |
| @@ -685,32 +689,34 @@ static int sdma_v5_2_gfx_resume(struct amdgpu_device *adev) · 68 unmodified lines | |||
| 685 | sdma_v5_2_ring_set_wptr(ring); | 689 | sdma_v5_2_ring_set_wptr(ring); |
| 686 | 690 | ||
| 687 | /* set minor_ptr_update to 0 after wptr programed */ | 691 | /* set minor_ptr_update to 0 after wptr programed */ |
| 688 | WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_GFX_MINOR_PTR_UPDATE), 0); | ||
| 689 | 692 | ||
| 690 | /* set utc l1 enable flag always to 1 */ | 693 | WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_GFX_MINOR_PTR_UPDATE), 0); |
| 691 | temp = RREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL)); | ||
| 692 | temp = REG_SET_FIELD(temp, SDMA0_CNTL, UTC_L1_ENABLE, 1); | ||
| 693 | |||
| 694 | /* enable MCBP */ | ||
| 695 | temp = REG_SET_FIELD(temp, SDMA0_CNTL, MIDCMD_PREEMPT_ENABLE, 1); | ||
| 696 | WREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL), temp); | ||
| 697 | |||
| 698 | /* Set up RESP_MODE to non-copy addresses */ | ||
| 699 | temp = RREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_CNTL)); | ||
| 700 | temp = REG_SET_FIELD(temp, SDMA0_UTCL1_CNTL, RESP_MODE, 3); | ||
| 701 | temp = REG_SET_FIELD(temp, SDMA0_UTCL1_CNTL, REDO_DELAY, 9); | ||
| 702 | WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_CNTL), temp); | ||
| 703 | |||
| 704 | /* program default cache read and write policy */ | ||
| 705 | temp = RREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_PAGE)); | ||
| 706 | /* clean read policy and write policy bits */ | ||
| 707 | temp &= 0xFF0FFF; | ||
| 708 | temp |= ((CACHE_READ_POLICY_L2__DEFAULT << 12) | | ||
| 709 | (CACHE_WRITE_POLICY_L2__DEFAULT << 14) | | ||
| 710 | SDMA0_UTCL1_PAGE__LLC_NOALLOC_MASK); | ||
| 711 | WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_PAGE), temp); | ||
| 712 | 694 | ||
| 695 | /* SRIOV VF has no control of any of registers below */ | ||
| 713 | if (!amdgpu_sriov_vf(adev)) { | 696 | if (!amdgpu_sriov_vf(adev)) { |
| 697 | /* set utc l1 enable flag always to 1 */ | ||
| 698 | temp = RREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL)); | ||
| 699 | temp = REG_SET_FIELD(temp, SDMA0_CNTL, UTC_L1_ENABLE, 1); | ||
| 700 | |||
| 701 | /* enable MCBP */ | ||
| 702 | temp = REG_SET_FIELD(temp, SDMA0_CNTL, MIDCMD_PREEMPT_ENABLE, 1); | ||
| 703 | WREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_CNTL), temp); | ||
| 704 | |||
| 705 | /* Set up RESP_MODE to non-copy addresses */ | ||
| 706 | temp = RREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_CNTL)); | ||
| 707 | temp = REG_SET_FIELD(temp, SDMA0_UTCL1_CNTL, RESP_MODE, 3); | ||
| 708 | temp = REG_SET_FIELD(temp, SDMA0_UTCL1_CNTL, REDO_DELAY, 9); | ||
| 709 | WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_CNTL), temp); | ||
| 710 | |||
| 711 | /* program default cache read and write policy */ | ||
| 712 | temp = RREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_PAGE)); | ||
| 713 | /* clean read policy and write policy bits */ | ||
| 714 | temp &= 0xFF0FFF; | ||
| 715 | temp |= ((CACHE_READ_POLICY_L2__DEFAULT << 12) | | ||
| 716 | (CACHE_WRITE_POLICY_L2__DEFAULT << 14) | | ||
| 717 | SDMA0_UTCL1_PAGE__LLC_NOALLOC_MASK); | ||
| 718 | WREG32_SOC15_IP(GC, sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_UTCL1_PAGE), temp); | ||
| 719 | |||
| 714 | /* unhalt engine */ | 720 | /* unhalt engine */ |
| 715 | temp = RREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_F32_CNTL)); | 721 | temp = RREG32(sdma_v5_2_get_reg_offset(adev, i, mmSDMA0_F32_CNTL)); |
| 716 | temp = REG_SET_FIELD(temp, SDMA0_F32_CNTL, HALT, 0); | 722 | temp = REG_SET_FIELD(temp, SDMA0_F32_CNTL, HALT, 0); |
| @@ -1438,13 +1444,14 @@ static int sdma_v5_2_set_trap_irq_state(struct amdgpu_device *adev, · 721 unmodified lines | |||
| 1438 | enum amdgpu_interrupt_state state) | 1444 | enum amdgpu_interrupt_state state) |
| 1439 | { | 1445 | { |
| 1440 | u32 sdma_cntl; | 1446 | u32 sdma_cntl; |
| 1441 | |||
| 1442 | u32 reg_offset = sdma_v5_2_get_reg_offset(adev, type, mmSDMA0_CNTL); | 1447 | u32 reg_offset = sdma_v5_2_get_reg_offset(adev, type, mmSDMA0_CNTL); |
| 1443 | 1448 | ||
| 1444 | sdma_cntl = RREG32(reg_offset); | 1449 | if (!amdgpu_sriov_vf(adev)) { |
| 1445 | sdma_cntl = REG_SET_FIELD(sdma_cntl, SDMA0_CNTL, TRAP_ENABLE, | 1450 | sdma_cntl = RREG32(reg_offset); |
| 1446 | state == AMDGPU_IRQ_STATE_ENABLE ? 1 : 0); | 1451 | sdma_cntl = REG_SET_FIELD(sdma_cntl, SDMA0_CNTL, TRAP_ENABLE, |
| 1447 | WREG32(reg_offset, sdma_cntl); | 1452 | state == AMDGPU_IRQ_STATE_ENABLE ? 1 : 0); |
| 1453 | WREG32(reg_offset, sdma_cntl); | ||
| 1454 | } | ||
| 1448 | 1455 | ||
| 1449 | return 0; | 1456 | return 0; |
| 1450 | } | 1457 | } |
| @@ -744,7 +744,7 @@ static void soc15_reg_base_init(struct amdgpu_device *adev) · 743 unmodified lines | |||
| 744 | vega10_reg_base_init(adev); | 744 | vega10_reg_base_init(adev); |
| 745 | break; | 745 | break; |
| 746 | case CHIP_RENOIR: | 746 | case CHIP_RENOIR: |
| 747 | /* It's safe to do ip discovery here for Renior, | 747 | /* It's safe to do ip discovery here for Renoir, |
| 748 | * it doesn't support SRIOV. */ | 748 | * it doesn't support SRIOV. */ |
| 749 | if (amdgpu_discovery) { | 749 | if (amdgpu_discovery) { |
| 750 | r = amdgpu_discovery_reg_base_init(adev); | 750 | r = amdgpu_discovery_reg_base_init(adev); |
| @@ -1238,7 +1238,9 @@ static int soc15_common_sw_init(void *handle) · 487 unmodified lines | |||
| 1238 | if (amdgpu_sriov_vf(adev)) | 1238 | if (amdgpu_sriov_vf(adev)) |
| 1239 | xgpu_ai_mailbox_add_irq_id(adev); | 1239 | xgpu_ai_mailbox_add_irq_id(adev); |
| 1240 | 1240 | ||
| 1241 | adev->df.funcs->sw_init(adev); | 1241 | if (adev->df.funcs && |
| 1242 | adev->df.funcs->sw_init) | ||
| 1243 | adev->df.funcs->sw_init(adev); | ||
| 1242 | 1244 | ||
| 1243 | return 0; | 1245 | return 0; |
| 1244 | } | 1246 | } |
| @@ -1250,7 +1252,10 @@ static int soc15_common_sw_fini(void *handle) · 5 unmodified lines | |||
| 1250 | if (adev->nbio.ras_funcs && | 1252 | if (adev->nbio.ras_funcs && |
| 1251 | adev->nbio.ras_funcs->ras_fini) | 1253 | adev->nbio.ras_funcs->ras_fini) |
| 1252 | adev->nbio.ras_funcs->ras_fini(adev); | 1254 | adev->nbio.ras_funcs->ras_fini(adev); |
| 1253 | adev->df.funcs->sw_fini(adev); | 1255 | |
| 1256 | if (adev->df.funcs && | ||
| 1257 | adev->df.funcs->sw_fini) | ||
| 1258 | adev->df.funcs->sw_fini(adev); | ||
| 1254 | return 0; | 1259 | return 0; |
| 1255 | } | 1260 | } |
| 1256 | 1261 | ||
| @@ -51,6 +51,8 @@ · 50 unmodified lines | |||
| 51 | 51 | ||
| 52 | #define RREG32_SOC15_IP(ip, reg) __RREG32_SOC15_RLC__(reg, 0, ip##_HWIP) | 52 | #define RREG32_SOC15_IP(ip, reg) __RREG32_SOC15_RLC__(reg, 0, ip##_HWIP) |
| 53 | 53 | ||
| 54 | #define RREG32_SOC15_IP_NO_KIQ(ip, reg) __RREG32_SOC15_RLC__(reg, AMDGPU_REGS_NO_KIQ, ip##_HWIP) | ||
| 55 | |||
| 54 | #define RREG32_SOC15_NO_KIQ(ip, inst, reg) \ | 56 | #define RREG32_SOC15_NO_KIQ(ip, inst, reg) \ |
| 55 | __RREG32_SOC15_RLC__(adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg, \ | 57 | __RREG32_SOC15_RLC__(adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg, \ |
| 56 | AMDGPU_REGS_NO_KIQ, ip##_HWIP) | 58 | AMDGPU_REGS_NO_KIQ, ip##_HWIP) |
| @@ -65,6 +67,9 @@ · 8 unmodified lines | |||
| 65 | #define WREG32_SOC15_IP(ip, reg, value) \ | 67 | #define WREG32_SOC15_IP(ip, reg, value) \ |
| 66 | __WREG32_SOC15_RLC__(reg, value, 0, ip##_HWIP) | 68 | __WREG32_SOC15_RLC__(reg, value, 0, ip##_HWIP) |
| 67 | 69 | ||
| 70 | #define WREG32_SOC15_IP_NO_KIQ(ip, reg, value) \ | ||
| 71 | __WREG32_SOC15_RLC__(reg, value, AMDGPU_REGS_NO_KIQ, ip##_HWIP) | ||
| 72 | |||
| 68 | #define WREG32_SOC15_NO_KIQ(ip, inst, reg, value) \ | 73 | #define WREG32_SOC15_NO_KIQ(ip, inst, reg, value) \ |
| 69 | __WREG32_SOC15_RLC__(adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg, \ | 74 | __WREG32_SOC15_RLC__(adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg, \ |
| 70 | value, AMDGPU_REGS_NO_KIQ, ip##_HWIP) | 75 | value, AMDGPU_REGS_NO_KIQ, ip##_HWIP) |
| @@ -50,6 +50,165 @@ static inline uint32_t get_umc_v6_7_reg_offset(struct amdgpu_device *adev, · 49 unmodified lines | |||
| 50 | return adev->umc.channel_offs * ch_inst + UMC_V6_7_INST_DIST * umc_inst; | 50 | return adev->umc.channel_offs * ch_inst + UMC_V6_7_INST_DIST * umc_inst; |
| 51 | } | 51 | } |
| 52 | 52 | ||
| 53 | static inline uint32_t get_umc_v6_7_channel_index(struct amdgpu_device *adev, | ||
| 54 | uint32_t umc_inst, | ||
| 55 | uint32_t ch_inst) | ||
| 56 | { | ||
| 57 | return adev->umc.channel_idx_tbl[umc_inst * adev->umc.channel_inst_num + ch_inst]; | ||
| 58 | } | ||
| 59 | |||
| 60 | static void umc_v6_7_ecc_info_query_correctable_error_count(struct amdgpu_device *adev, | ||
| 61 | uint32_t channel_index, | ||
| 62 | unsigned long *error_count) | ||
| 63 | { | ||
| 64 | uint32_t ecc_err_cnt; | ||
| 65 | uint64_t mc_umc_status; | ||
| 66 | struct amdgpu_ras *ras = amdgpu_ras_get_context(adev); | ||
| 67 | |||
| 68 | /* | ||
| 69 | * select the lower chip and check the error count | ||
| 70 | * skip add error count, calc error counter only from mca_umc_status | ||
| 71 | */ | ||
| 72 | ecc_err_cnt = ras->umc_ecc.ecc[channel_index].ce_count_lo_chip; | ||
| 73 | |||
| 74 | /* | ||
| 75 | * select the higher chip and check the err counter | ||
| 76 | * skip add error count, calc error counter only from mca_umc_status | ||
| 77 | */ | ||
| 78 | ecc_err_cnt = ras->umc_ecc.ecc[channel_index].ce_count_hi_chip; | ||
| 79 | |||
| 80 | /* check for SRAM correctable error | ||
| 81 | MCUMC_STATUS is a 64 bit register */ | ||
| 82 | mc_umc_status = ras->umc_ecc.ecc[channel_index].mca_umc_status; | ||
| 83 | if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 && | ||
| 84 | REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, CECC) == 1) | ||
| 85 | *error_count += 1; | ||
| 86 | } | ||
| 87 | |||
| 88 | static void umc_v6_7_ecc_info_querry_uncorrectable_error_count(struct amdgpu_device *adev, | ||
| 89 | uint32_t channel_index, | ||
| 90 | unsigned long *error_count) | ||
| 91 | { | ||
| 92 | uint64_t mc_umc_status; | ||
| 93 | struct amdgpu_ras *ras = amdgpu_ras_get_context(adev); | ||
| 94 | |||
| 95 | /* check the MCUMC_STATUS */ | ||
| 96 | mc_umc_status = ras->umc_ecc.ecc[channel_index].mca_umc_status; | ||
| 97 | if ((REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1) && | ||
| 98 | (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Deferred) == 1 || | ||
| 99 | REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 || | ||
| 100 | REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, PCC) == 1 || | ||
| 101 | REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UC) == 1 || | ||
| 102 | REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, TCC) == 1)) | ||
| 103 | *error_count += 1; | ||
| 104 | } | ||
| 105 | |||
| 106 | static void umc_v6_7_ecc_info_query_ras_error_count(struct amdgpu_device *adev, | ||
| 107 | void *ras_error_status) | ||
| 108 | { | ||
| 109 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; | ||
| 110 | |||
| 111 | uint32_t umc_inst = 0; | ||
| 112 | uint32_t ch_inst = 0; | ||
| 113 | uint32_t umc_reg_offset = 0; | ||
| 114 | uint32_t channel_index = 0; | ||
| 115 | |||
| 116 | /*TODO: driver needs to toggle DF Cstate to ensure | ||
| 117 | * safe access of UMC registers. Will add the protection */ | ||
| 118 | LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) { | ||
| 119 | umc_reg_offset = get_umc_v6_7_reg_offset(adev, | ||
| 120 | umc_inst, | ||
| 121 | ch_inst); | ||
| 122 | channel_index = get_umc_v6_7_channel_index(adev, | ||
| 123 | umc_inst, | ||
| 124 | ch_inst); | ||
| 125 | umc_v6_7_ecc_info_query_correctable_error_count(adev, | ||
| 126 | channel_index, | ||
| 127 | &(err_data->ce_count)); | ||
| 128 | umc_v6_7_ecc_info_querry_uncorrectable_error_count(adev, | ||
| 129 | channel_index, | ||
| 130 | &(err_data->ue_count)); | ||
| 131 | } | ||
| 132 | } | ||
| 133 | |||
| 134 | static void umc_v6_7_ecc_info_query_error_address(struct amdgpu_device *adev, | ||
| 135 | struct ras_err_data *err_data, | ||
| 136 | uint32_t umc_reg_offset, | ||
| 137 | uint32_t ch_inst, | ||
| 138 | uint32_t umc_inst) | ||
| 139 | { | ||
| 140 | uint64_t mc_umc_status, err_addr, retired_page; | ||
| 141 | struct eeprom_table_record *err_rec; | ||
| 142 | uint32_t channel_index; | ||
| 143 | struct amdgpu_ras *ras = amdgpu_ras_get_context(adev); | ||
| 144 | |||
| 145 | channel_index = | ||
| 146 | adev->umc.channel_idx_tbl[umc_inst * adev->umc.channel_inst_num + ch_inst]; | ||
| 147 | |||
| 148 | mc_umc_status = ras->umc_ecc.ecc[channel_index].mca_umc_status; | ||
| 149 | |||
| 150 | if (mc_umc_status == 0) | ||
| 151 | return; | ||
| 152 | |||
| 153 | if (!err_data->err_addr) | ||
| 154 | return; | ||
| 155 | |||
| 156 | err_rec = &err_data->err_addr[err_data->err_addr_cnt]; | ||
| 157 | |||
| 158 | /* calculate error address if ue/ce error is detected */ | ||
| 159 | if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 && | ||
| 160 | (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 || | ||
| 161 | REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, CECC) == 1)) { | ||
| 162 | |||
| 163 | err_addr = ras->umc_ecc.ecc[channel_index].mca_umc_addr; | ||
| 164 | err_addr = REG_GET_FIELD(err_addr, MCA_UMC_UMC0_MCUMC_ADDRT0, ErrorAddr); | ||
| 165 | |||
| 166 | /* translate umc channel address to soc pa, 3 parts are included */ | ||
| 167 | retired_page = ADDR_OF_8KB_BLOCK(err_addr) | | ||
| 168 | ADDR_OF_256B_BLOCK(channel_index) | | ||
| 169 | OFFSET_IN_256B_BLOCK(err_addr); | ||
| 170 | |||
| 171 | /* we only save ue error information currently, ce is skipped */ | ||
| 172 | if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) | ||
| 173 | == 1) { | ||
| 174 | err_rec->address = err_addr; | ||
| 175 | /* page frame address is saved */ | ||
| 176 | err_rec->retired_page = retired_page >> AMDGPU_GPU_PAGE_SHIFT; | ||
| 177 | err_rec->ts = (uint64_t)ktime_get_real_seconds(); | ||
| 178 | err_rec->err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE; | ||
| 179 | err_rec->cu = 0; | ||
| 180 | err_rec->mem_channel = channel_index; | ||
| 181 | err_rec->mcumc_id = umc_inst; | ||
| 182 | |||
| 183 | err_data->err_addr_cnt++; | ||
| 184 | } | ||
| 185 | } | ||
| 186 | } | ||
| 187 | |||
| 188 | static void umc_v6_7_ecc_info_query_ras_error_address(struct amdgpu_device *adev, | ||
| 189 | void *ras_error_status) | ||
| 190 | { | ||
| 191 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; | ||
| 192 | |||
| 193 | uint32_t umc_inst = 0; | ||
| 194 | uint32_t ch_inst = 0; | ||
| 195 | uint32_t umc_reg_offset = 0; | ||
| 196 | |||
| 197 | /*TODO: driver needs to toggle DF Cstate to ensure | ||
| 198 | * safe access of UMC resgisters. Will add the protection | ||
| 199 | * when firmware interface is ready */ | ||
| 200 | LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) { | ||
| 201 | umc_reg_offset = get_umc_v6_7_reg_offset(adev, | ||
| 202 | umc_inst, | ||
| 203 | ch_inst); | ||
| 204 | umc_v6_7_ecc_info_query_error_address(adev, | ||
| 205 | err_data, | ||
| 206 | umc_reg_offset, | ||
| 207 | ch_inst, | ||
| 208 | umc_inst); | ||
| 209 | } | ||
| 210 | } | ||
| 211 | |||
| 53 | static void umc_v6_7_query_correctable_error_count(struct amdgpu_device *adev, | 212 | static void umc_v6_7_query_correctable_error_count(struct amdgpu_device *adev, |
| 54 | uint32_t umc_reg_offset, | 213 | uint32_t umc_reg_offset, |
| 55 | unsigned long *error_count) | 214 | unsigned long *error_count) |
| @@ -327,4 +486,6 @@ const struct amdgpu_umc_ras_funcs umc_v6_7_ras_funcs = { · 271 unmodified lines | |||
| 327 | .query_ras_error_count = umc_v6_7_query_ras_error_count, | 486 | .query_ras_error_count = umc_v6_7_query_ras_error_count, |
| 328 | .query_ras_error_address = umc_v6_7_query_ras_error_address, | 487 | .query_ras_error_address = umc_v6_7_query_ras_error_address, |
| 329 | .query_ras_poison_mode = umc_v6_7_query_ras_poison_mode, | 488 | .query_ras_poison_mode = umc_v6_7_query_ras_poison_mode, |
| 489 | .ecc_info_query_ras_error_count = umc_v6_7_ecc_info_query_ras_error_count, | ||
| 490 | .ecc_info_query_ras_error_address = umc_v6_7_ecc_info_query_ras_error_address, | ||
| 330 | }; | 491 | }; |
| @@ -640,6 +640,7 @@ const struct amd_ip_funcs vega10_ih_ip_funcs = { · 639 unmodified lines | |||
| 640 | static const struct amdgpu_ih_funcs vega10_ih_funcs = { | 640 | static const struct amdgpu_ih_funcs vega10_ih_funcs = { |
| 641 | .get_wptr = vega10_ih_get_wptr, | 641 | .get_wptr = vega10_ih_get_wptr, |
| 642 | .decode_iv = amdgpu_ih_decode_iv_helper, | 642 | .decode_iv = amdgpu_ih_decode_iv_helper, |
| 643 | .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, | ||
| 643 | .set_rptr = vega10_ih_set_rptr | 644 | .set_rptr = vega10_ih_set_rptr |
| 644 | }; | 645 | }; |
| 645 | 646 | ||
| @@ -688,6 +688,7 @@ const struct amd_ip_funcs vega20_ih_ip_funcs = { · 687 unmodified lines | |||
| 688 | static const struct amdgpu_ih_funcs vega20_ih_funcs = { | 688 | static const struct amdgpu_ih_funcs vega20_ih_funcs = { |
| 689 | .get_wptr = vega20_ih_get_wptr, | 689 | .get_wptr = vega20_ih_get_wptr, |
| 690 | .decode_iv = amdgpu_ih_decode_iv_helper, | 690 | .decode_iv = amdgpu_ih_decode_iv_helper, |
| 691 | .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, | ||
| 691 | .set_rptr = vega20_ih_set_rptr | 692 | .set_rptr = vega20_ih_set_rptr |
| 692 | }; | 693 | }; |
| 693 | 694 | ||
| @@ -43,15 +43,15 @@ static bool cik_event_interrupt_isr(struct kfd_dev *dev, · 42 unmodified lines | |||
| 43 | */ | 43 | */ |
| 44 | if ((ihre->source_id == CIK_INTSRC_GFX_PAGE_INV_FAULT || | 44 | if ((ihre->source_id == CIK_INTSRC_GFX_PAGE_INV_FAULT || |
| 45 | ihre->source_id == CIK_INTSRC_GFX_MEM_PROT_FAULT) && | 45 | ihre->source_id == CIK_INTSRC_GFX_MEM_PROT_FAULT) && |
| 46 | dev->device_info->asic_family == CHIP_HAWAII) { | 46 | dev->adev->asic_type == CHIP_HAWAII) { |
| 47 | struct cik_ih_ring_entry *tmp_ihre = | 47 | struct cik_ih_ring_entry *tmp_ihre = |
| 48 | (struct cik_ih_ring_entry *)patched_ihre; | 48 | (struct cik_ih_ring_entry *)patched_ihre; |
| 49 | 49 | ||
| 50 | *patched_flag = true; | 50 | *patched_flag = true; |
| 51 | *tmp_ihre = *ihre; | 51 | *tmp_ihre = *ihre; |
| 52 | 52 | ||
| 53 | vmid = f2g->read_vmid_from_vmfault_reg(dev->kgd); | 53 | vmid = f2g->read_vmid_from_vmfault_reg(dev->adev); |
| 54 | ret = f2g->get_atc_vmid_pasid_mapping_info(dev->kgd, vmid, &pasid); | 54 | ret = f2g->get_atc_vmid_pasid_mapping_info(dev->adev, vmid, &pasid); |
| 55 | 55 | ||
| 56 | tmp_ihre->ring_id &= 0x000000ff; | 56 | tmp_ihre->ring_id &= 0x000000ff; |
| 57 | tmp_ihre->ring_id |= vmid << 8; | 57 | tmp_ihre->ring_id |= vmid << 8; |
| @@ -113,7 +113,7 @@ static void cik_event_interrupt_wq(struct kfd_dev *dev, · 55 unmodified lines | |||
| 113 | kfd_process_vm_fault(dev->dqm, pasid); | 113 | kfd_process_vm_fault(dev->dqm, pasid); |
| 114 | 114 | ||
| 115 | memset(&info, 0, sizeof(info)); | 115 | memset(&info, 0, sizeof(info)); |
| 116 | amdgpu_amdkfd_gpuvm_get_vm_fault_info(dev->kgd, &info); | 116 | amdgpu_amdkfd_gpuvm_get_vm_fault_info(dev->adev, &info); |
| 117 | if (!info.page_addr && !info.status) | 117 | if (!info.page_addr && !info.status) |
| 118 | return; | 118 | return; |
| 119 | 119 | ||
| @@ -321,7 +321,7 @@ static int kfd_ioctl_create_queue(struct file *filep, struct kfd_process *p, · 320 unmodified lines | |||
| 321 | /* Return gpu_id as doorbell offset for mmap usage */ | 321 | /* Return gpu_id as doorbell offset for mmap usage */ |
| 322 | args->doorbell_offset = KFD_MMAP_TYPE_DOORBELL; | 322 | args->doorbell_offset = KFD_MMAP_TYPE_DOORBELL; |
| 323 | args->doorbell_offset |= KFD_MMAP_GPU_ID(args->gpu_id); | 323 | args->doorbell_offset |= KFD_MMAP_GPU_ID(args->gpu_id); |
| 324 | if (KFD_IS_SOC15(dev->device_info->asic_family)) | 324 | if (KFD_IS_SOC15(dev)) |
| 325 | /* On SOC15 ASICs, include the doorbell offset within the | 325 | /* On SOC15 ASICs, include the doorbell offset within the |
| 326 | * process doorbell frame, which is 2 pages. | 326 | * process doorbell frame, which is 2 pages. |
| 327 | */ | 327 | */ |
| @@ -580,7 +580,7 @@ static int kfd_ioctl_dbg_register(struct file *filep, · 252 unmodified lines | |||
| 580 | if (!dev) | 580 | if (!dev) |
| 581 | return -EINVAL; | 581 | return -EINVAL; |
| 582 | 582 | ||
| 583 | if (dev->device_info->asic_family == CHIP_CARRIZO) { | 583 | if (dev->adev->asic_type == CHIP_CARRIZO) { |
| 584 | pr_debug("kfd_ioctl_dbg_register not supported on CZ\n"); | 584 | pr_debug("kfd_ioctl_dbg_register not supported on CZ\n"); |
| 585 | return -EINVAL; | 585 | return -EINVAL; |
| 586 | } | 586 | } |
| @@ -631,7 +631,7 @@ static int kfd_ioctl_dbg_unregister(struct file *filep, · 44 unmodified lines | |||
| 631 | if (!dev || !dev->dbgmgr) | 631 | if (!dev || !dev->dbgmgr) |
| 632 | return -EINVAL; | 632 | return -EINVAL; |
| 633 | 633 | ||
| 634 | if (dev->device_info->asic_family == CHIP_CARRIZO) { | 634 | if (dev->adev->asic_type == CHIP_CARRIZO) { |
| 635 | pr_debug("kfd_ioctl_dbg_unregister not supported on CZ\n"); | 635 | pr_debug("kfd_ioctl_dbg_unregister not supported on CZ\n"); |
| 636 | return -EINVAL; | 636 | return -EINVAL; |
| 637 | } | 637 | } |
| @@ -676,7 +676,7 @@ static int kfd_ioctl_dbg_address_watch(struct file *filep, · 38 unmodified lines | |||
| 676 | if (!dev) | 676 | if (!dev) |
| 677 | return -EINVAL; | 677 | return -EINVAL; |
| 678 | 678 | ||
| 679 | if (dev->device_info->asic_family == CHIP_CARRIZO) { | 679 | if (dev->adev->asic_type == CHIP_CARRIZO) { |
| 680 | pr_debug("kfd_ioctl_dbg_wave_control not supported on CZ\n"); | 680 | pr_debug("kfd_ioctl_dbg_wave_control not supported on CZ\n"); |
| 681 | return -EINVAL; | 681 | return -EINVAL; |
| 682 | } | 682 | } |
| @@ -784,7 +784,7 @@ static int kfd_ioctl_dbg_wave_control(struct file *filep, · 101 unmodified lines | |||
| 784 | if (!dev) | 784 | if (!dev) |
| 785 | return -EINVAL; | 785 | return -EINVAL; |
| 786 | 786 | ||
| 787 | if (dev->device_info->asic_family == CHIP_CARRIZO) { | 787 | if (dev->adev->asic_type == CHIP_CARRIZO) { |
| 788 | pr_debug("kfd_ioctl_dbg_wave_control not supported on CZ\n"); | 788 | pr_debug("kfd_ioctl_dbg_wave_control not supported on CZ\n"); |
| 789 | return -EINVAL; | 789 | return -EINVAL; |
| 790 | } | 790 | } |
| @@ -851,7 +851,7 @@ static int kfd_ioctl_get_clock_counters(struct file *filep, · 60 unmodified lines | |||
| 851 | dev = kfd_device_by_id(args->gpu_id); | 851 | dev = kfd_device_by_id(args->gpu_id); |
| 852 | if (dev) | 852 | if (dev) |
| 853 | /* Reading GPU clock counter from KGD */ | 853 | /* Reading GPU clock counter from KGD */ |
| 854 | args->gpu_clock_counter = amdgpu_amdkfd_get_gpu_clock_counter(dev->kgd); | 854 | args->gpu_clock_counter = amdgpu_amdkfd_get_gpu_clock_counter(dev->adev); |
| 855 | else | 855 | else |
| 856 | /* Node without GPU resource */ | 856 | /* Node without GPU resource */ |
| 857 | args->gpu_clock_counter = 0; | 857 | args->gpu_clock_counter = 0; |
| @@ -1041,7 +1041,7 @@ static int kfd_ioctl_create_event(struct file *filp, struct kfd_process *p, · 183 unmodified lines | |||
| 1041 | goto out_unlock; | 1041 | goto out_unlock; |
| 1042 | } | 1042 | } |
| 1043 | 1043 | ||
| 1044 | err = amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(kfd->kgd, | 1044 | err = amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(kfd->adev, |
| 1045 | mem, &kern_addr, &size); | 1045 | mem, &kern_addr, &size); |
| 1046 | if (err) { | 1046 | if (err) { |
| 1047 | pr_err("Failed to map event page to kernel\n"); | 1047 | pr_err("Failed to map event page to kernel\n"); |
| @@ -1051,7 +1051,7 @@ static int kfd_ioctl_create_event(struct file *filp, struct kfd_process *p, · 3 unmodified lines | |||
| 1051 | err = kfd_event_page_set(p, kern_addr, size); | 1051 | err = kfd_event_page_set(p, kern_addr, size); |
| 1052 | if (err) { | 1052 | if (err) { |
| 1053 | pr_err("Failed to set event page\n"); | 1053 | pr_err("Failed to set event page\n"); |
| 1054 | amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(kfd->kgd, mem); | 1054 | amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(kfd->adev, mem); |
| 1055 | goto out_unlock; | 1055 | goto out_unlock; |
| 1056 | } | 1056 | } |
| 1057 | 1057 | ||
| @@ -1137,7 +1137,7 @@ static int kfd_ioctl_set_scratch_backing_va(struct file *filep, · 79 unmodified lines | |||
| 1137 | if (dev->dqm->sched_policy == KFD_SCHED_POLICY_NO_HWS && | 1137 | if (dev->dqm->sched_policy == KFD_SCHED_POLICY_NO_HWS && |
| 1138 | pdd->qpd.vmid != 0 && dev->kfd2kgd->set_scratch_backing_va) | 1138 | pdd->qpd.vmid != 0 && dev->kfd2kgd->set_scratch_backing_va) |
| 1139 | dev->kfd2kgd->set_scratch_backing_va( | 1139 | dev->kfd2kgd->set_scratch_backing_va( |
| 1140 | dev->kgd, args->va_addr, pdd->qpd.vmid); | 1140 | dev->adev, args->va_addr, pdd->qpd.vmid); |
| 1141 | 1141 | ||
| 1142 | return 0; | 1142 | return 0; |
| 1143 | 1143 | ||
| @@ -1158,7 +1158,7 @@ static int kfd_ioctl_get_tile_config(struct file *filep, · 14 unmodified lines | |||
| 1158 | if (!dev) | 1158 | if (!dev) |
| 1159 | return -EINVAL; | 1159 | return -EINVAL; |
| 1160 | 1160 | ||
| 1161 | amdgpu_amdkfd_get_tile_config(dev->kgd, &config); | 1161 | amdgpu_amdkfd_get_tile_config(dev->adev, &config); |
| 1162 | 1162 | ||
| 1163 | args->gb_addr_config = config.gb_addr_config; | 1163 | args->gb_addr_config = config.gb_addr_config; |
| 1164 | args->num_banks = config.num_banks; | 1164 | args->num_banks = config.num_banks; |
| @@ -1244,7 +1244,7 @@ bool kfd_dev_is_large_bar(struct kfd_dev *dev) · 79 unmodified lines | |||
| 1244 | if (dev->use_iommu_v2) | 1244 | if (dev->use_iommu_v2) |
| 1245 | return false; | 1245 | return false; |
| 1246 | 1246 | ||
| 1247 | amdgpu_amdkfd_get_local_mem_info(dev->kgd, &mem_info); | 1247 | amdgpu_amdkfd_get_local_mem_info(dev->adev, &mem_info); |
| 1248 | if (mem_info.local_mem_size_private == 0 && | 1248 | if (mem_info.local_mem_size_private == 0 && |
| 1249 | mem_info.local_mem_size_public > 0) | 1249 | mem_info.local_mem_size_public > 0) |
| 1250 | return true; | 1250 | return true; |
| @@ -1313,7 +1313,7 @@ static int kfd_ioctl_alloc_memory_of_gpu(struct file *filep, · 62 unmodified lines | |||
| 1313 | err = -EINVAL; | 1313 | err = -EINVAL; |
| 1314 | goto err_unlock; | 1314 | goto err_unlock; |
| 1315 | } | 1315 | } |
| 1316 | offset = amdgpu_amdkfd_get_mmio_remap_phys_addr(dev->kgd); | 1316 | offset = dev->adev->rmmio_remap.bus_addr; |
| 1317 | if (!offset) { | 1317 | if (!offset) { |
| 1318 | err = -ENOMEM; | 1318 | err = -ENOMEM; |
| 1319 | goto err_unlock; | 1319 | goto err_unlock; |
| @@ -1321,7 +1321,7 @@ static int kfd_ioctl_alloc_memory_of_gpu(struct file *filep, · 1 unmodified line | |||
| 1321 | } | 1321 | } |
| 1322 | 1322 | ||
| 1323 | err = amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( | 1323 | err = amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( |
| 1324 | dev->kgd, args->va_addr, args->size, | 1324 | dev->adev, args->va_addr, args->size, |
| 1325 | pdd->drm_priv, (struct kgd_mem **) &mem, &offset, | 1325 | pdd->drm_priv, (struct kgd_mem **) &mem, &offset, |
| 1326 | flags); | 1326 | flags); |
| 1327 | 1327 | ||
| @@ -1353,7 +1353,7 @@ static int kfd_ioctl_alloc_memory_of_gpu(struct file *filep, · 25 unmodified lines | |||
| 1353 | return 0; | 1353 | return 0; |
| 1354 | 1354 | ||
| 1355 | err_free: | 1355 | err_free: |
| 1356 | amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->kgd, (struct kgd_mem *)mem, | 1356 | amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->adev, (struct kgd_mem *)mem, |
| 1357 | pdd->drm_priv, NULL); | 1357 | pdd->drm_priv, NULL); |
| 1358 | err_unlock: | 1358 | err_unlock: |
| 1359 | mutex_unlock(&p->mutex); | 1359 | mutex_unlock(&p->mutex); |
| @@ -1399,7 +1399,7 @@ static int kfd_ioctl_free_memory_of_gpu(struct file *filep, · 39 unmodified lines | |||
| 1399 | goto err_unlock; | 1399 | goto err_unlock; |
| 1400 | } | 1400 | } |
| 1401 | 1401 | ||
| 1402 | ret = amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->kgd, | 1402 | ret = amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->adev, |
| 1403 | (struct kgd_mem *)mem, pdd->drm_priv, &size); | 1403 | (struct kgd_mem *)mem, pdd->drm_priv, &size); |
| 1404 | 1404 | ||
| 1405 | /* If freeing the buffer failed, leave the handle in place for | 1405 | /* If freeing the buffer failed, leave the handle in place for |
| @@ -1484,7 +1484,7 @@ static int kfd_ioctl_map_memory_to_gpu(struct file *filep, · 78 unmodified lines | |||
| 1484 | goto get_mem_obj_from_handle_failed; | 1484 | goto get_mem_obj_from_handle_failed; |
| 1485 | } | 1485 | } |
| 1486 | err = amdgpu_amdkfd_gpuvm_map_memory_to_gpu( | 1486 | err = amdgpu_amdkfd_gpuvm_map_memory_to_gpu( |
| 1487 | peer->kgd, (struct kgd_mem *)mem, | 1487 | peer->adev, (struct kgd_mem *)mem, |
| 1488 | peer_pdd->drm_priv, &table_freed); | 1488 | peer_pdd->drm_priv, &table_freed); |
| 1489 | if (err) { | 1489 | if (err) { |
| 1490 | pr_err("Failed to map to gpu %d/%d\n", | 1490 | pr_err("Failed to map to gpu %d/%d\n", |
| @@ -1496,7 +1496,7 @@ static int kfd_ioctl_map_memory_to_gpu(struct file *filep, · 5 unmodified lines | |||
| 1496 | 1496 | ||
| 1497 | mutex_unlock(&p->mutex); | 1497 | mutex_unlock(&p->mutex); |
| 1498 | 1498 | ||
| 1499 | err = amdgpu_amdkfd_gpuvm_sync_memory(dev->kgd, (struct kgd_mem *) mem, true); | 1499 | err = amdgpu_amdkfd_gpuvm_sync_memory(dev->adev, (struct kgd_mem *) mem, true); |
| 1500 | if (err) { | 1500 | if (err) { |
| 1501 | pr_debug("Sync memory failed, wait interrupted by user signal\n"); | 1501 | pr_debug("Sync memory failed, wait interrupted by user signal\n"); |
| 1502 | goto sync_memory_failed; | 1502 | goto sync_memory_failed; |
| @@ -1593,7 +1593,7 @@ static int kfd_ioctl_unmap_memory_from_gpu(struct file *filep, · 90 unmodified lines | |||
| 1593 | goto get_mem_obj_from_handle_failed; | 1593 | goto get_mem_obj_from_handle_failed; |
| 1594 | } | 1594 | } |
| 1595 | err = amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( | 1595 | err = amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( |
| 1596 | peer->kgd, (struct kgd_mem *)mem, peer_pdd->drm_priv); | 1596 | peer->adev, (struct kgd_mem *)mem, peer_pdd->drm_priv); |
| 1597 | if (err) { | 1597 | if (err) { |
| 1598 | pr_err("Failed to unmap from gpu %d/%d\n", | 1598 | pr_err("Failed to unmap from gpu %d/%d\n", |
| 1599 | i, args->n_devices); | 1599 | i, args->n_devices); |
| @@ -1603,8 +1603,8 @@ static int kfd_ioctl_unmap_memory_from_gpu(struct file *filep, · 3 unmodified lines | |||
| 1603 | } | 1603 | } |
| 1604 | mutex_unlock(&p->mutex); | 1604 | mutex_unlock(&p->mutex); |
| 1605 | 1605 | ||
| 1606 | if (dev->device_info->asic_family == CHIP_ALDEBARAN) { | 1606 | if (KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 2)) { |
| 1607 | err = amdgpu_amdkfd_gpuvm_sync_memory(dev->kgd, | 1607 | err = amdgpu_amdkfd_gpuvm_sync_memory(dev->adev, |
| 1608 | (struct kgd_mem *) mem, true); | 1608 | (struct kgd_mem *) mem, true); |
| 1609 | if (err) { | 1609 | if (err) { |
| 1610 | pr_debug("Sync memory failed, wait interrupted by user signal\n"); | 1610 | pr_debug("Sync memory failed, wait interrupted by user signal\n"); |
| @@ -1680,7 +1680,7 @@ static int kfd_ioctl_get_dmabuf_info(struct file *filep, · 69 unmodified lines | |||
| 1680 | { | 1680 | { |
| 1681 | struct kfd_ioctl_get_dmabuf_info_args *args = data; | 1681 | struct kfd_ioctl_get_dmabuf_info_args *args = data; |
| 1682 | struct kfd_dev *dev = NULL; | 1682 | struct kfd_dev *dev = NULL; |
| 1683 | struct kgd_dev *dma_buf_kgd; | 1683 | struct amdgpu_device *dmabuf_adev; |
| 1684 | void *metadata_buffer = NULL; | 1684 | void *metadata_buffer = NULL; |
| 1685 | uint32_t flags; | 1685 | uint32_t flags; |
| 1686 | unsigned int i; | 1686 | unsigned int i; |
| @@ -1700,15 +1700,15 @@ static int kfd_ioctl_get_dmabuf_info(struct file *filep, · 13 unmodified lines | |||
| 1700 | } | 1700 | } |
| 1701 | 1701 | ||
| 1702 | /* Get dmabuf info from KGD */ | 1702 | /* Get dmabuf info from KGD */ |
| 1703 | r = amdgpu_amdkfd_get_dmabuf_info(dev->kgd, args->dmabuf_fd, | 1703 | r = amdgpu_amdkfd_get_dmabuf_info(dev->adev, args->dmabuf_fd, |
| 1704 | &dma_buf_kgd, &args->size, | 1704 | &dmabuf_adev, &args->size, |
| 1705 | metadata_buffer, args->metadata_size, | 1705 | metadata_buffer, args->metadata_size, |
| 1706 | &args->metadata_size, &flags); | 1706 | &args->metadata_size, &flags); |
| 1707 | if (r) | 1707 | if (r) |
| 1708 | goto exit; | 1708 | goto exit; |
| 1709 | 1709 | ||
| 1710 | /* Reverse-lookup gpu_id from kgd pointer */ | 1710 | /* Reverse-lookup gpu_id from kgd pointer */ |
| 1711 | dev = kfd_device_by_kgd(dma_buf_kgd); | 1711 | dev = kfd_device_by_adev(dmabuf_adev); |
| 1712 | if (!dev) { | 1712 | if (!dev) { |
| 1713 | r = -EINVAL; | 1713 | r = -EINVAL; |
| 1714 | goto exit; | 1714 | goto exit; |
| @@ -1758,7 +1758,7 @@ static int kfd_ioctl_import_dmabuf(struct file *filep, · 43 unmodified lines | |||
| 1758 | goto err_unlock; | 1758 | goto err_unlock; |
| 1759 | } | 1759 | } |
| 1760 | 1760 | ||
| 1761 | r = amdgpu_amdkfd_gpuvm_import_dmabuf(dev->kgd, dmabuf, | 1761 | r = amdgpu_amdkfd_gpuvm_import_dmabuf(dev->adev, dmabuf, |
| 1762 | args->va_addr, pdd->drm_priv, | 1762 | args->va_addr, pdd->drm_priv, |
| 1763 | (struct kgd_mem **)&mem, &size, | 1763 | (struct kgd_mem **)&mem, &size, |
| 1764 | NULL); | 1764 | NULL); |
| @@ -1779,7 +1779,7 @@ static int kfd_ioctl_import_dmabuf(struct file *filep, · 14 unmodified lines | |||
| 1779 | return 0; | 1779 | return 0; |
| 1780 | 1780 | ||
| 1781 | err_free: | 1781 | err_free: |
| 1782 | amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->kgd, (struct kgd_mem *)mem, | 1782 | amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->adev, (struct kgd_mem *)mem, |
| 1783 | pdd->drm_priv, NULL); | 1783 | pdd->drm_priv, NULL); |
| 1784 | err_unlock: | 1784 | err_unlock: |
| 1785 | mutex_unlock(&p->mutex); | 1785 | mutex_unlock(&p->mutex); |
| @@ -2066,7 +2066,7 @@ static int kfd_mmio_mmap(struct kfd_dev *dev, struct kfd_process *process, · 280 unmodified lines | |||
| 2066 | if (vma->vm_end - vma->vm_start != PAGE_SIZE) | 2066 | if (vma->vm_end - vma->vm_start != PAGE_SIZE) |
| 2067 | return -EINVAL; | 2067 | return -EINVAL; |
| 2068 | 2068 | ||
| 2069 | address = amdgpu_amdkfd_get_mmio_remap_phys_addr(dev->kgd); | 2069 | address = dev->adev->rmmio_remap.bus_addr; |
| 2070 | 2070 | ||
| 2071 | vma->vm_flags |= VM_IO | VM_DONTCOPY | VM_DONTEXPAND | VM_NORESERVE | | 2071 | vma->vm_flags |= VM_IO | VM_DONTCOPY | VM_DONTEXPAND | VM_NORESERVE | |
| 2072 | VM_DONTDUMP | VM_PFNMAP; | 2072 | VM_DONTDUMP | VM_PFNMAP; |
| @@ -1340,7 +1340,7 @@ static int kfd_fill_gpu_cache_info(struct kfd_dev *kdev, · 1339 unmodified lines | |||
| 1340 | int ret; | 1340 | int ret; |
| 1341 | unsigned int num_cu_shared; | 1341 | unsigned int num_cu_shared; |
| 1342 | 1342 | ||
| 1343 | switch (kdev->device_info->asic_family) { | 1343 | switch (kdev->adev->asic_type) { |
| 1344 | case CHIP_KAVERI: | 1344 | case CHIP_KAVERI: |
| 1345 | pcache_info = kaveri_cache_info; | 1345 | pcache_info = kaveri_cache_info; |
| 1346 | num_of_cache_types = ARRAY_SIZE(kaveri_cache_info); | 1346 | num_of_cache_types = ARRAY_SIZE(kaveri_cache_info); |
| @@ -1377,67 +1377,71 @@ static int kfd_fill_gpu_cache_info(struct kfd_dev *kdev, · 30 unmodified lines | |||
| 1377 | pcache_info = vegam_cache_info; | 1377 | pcache_info = vegam_cache_info; |
| 1378 | num_of_cache_types = ARRAY_SIZE(vegam_cache_info); | 1378 | num_of_cache_types = ARRAY_SIZE(vegam_cache_info); |
| 1379 | break; | 1379 | break; |
| 1380 | case CHIP_VEGA10: | ||
| 1381 | pcache_info = vega10_cache_info; | ||
| 1382 | num_of_cache_types = ARRAY_SIZE(vega10_cache_info); | ||
| 1383 | break; | ||
| 1384 | case CHIP_VEGA12: | ||
| 1385 | pcache_info = vega12_cache_info; | ||
| 1386 | num_of_cache_types = ARRAY_SIZE(vega12_cache_info); | ||
| 1387 | break; | ||
| 1388 | case CHIP_VEGA20: | ||
| 1389 | case CHIP_ARCTURUS: | ||
| 1390 | pcache_info = vega20_cache_info; | ||
| 1391 | num_of_cache_types = ARRAY_SIZE(vega20_cache_info); | ||
| 1392 | break; | ||
| 1393 | case CHIP_ALDEBARAN: | ||
| 1394 | pcache_info = aldebaran_cache_info; | ||
| 1395 | num_of_cache_types = ARRAY_SIZE(aldebaran_cache_info); | ||
| 1396 | break; | ||
| 1397 | case CHIP_RAVEN: | ||
| 1398 | pcache_info = raven_cache_info; | ||
| 1399 | num_of_cache_types = ARRAY_SIZE(raven_cache_info); | ||
| 1400 | break; | ||
| 1401 | case CHIP_RENOIR: | ||
| 1402 | pcache_info = renoir_cache_info; | ||
| 1403 | num_of_cache_types = ARRAY_SIZE(renoir_cache_info); | ||
| 1404 | break; | ||
| 1405 | case CHIP_NAVI10: | ||
| 1406 | case CHIP_NAVI12: | ||
| 1407 | case CHIP_CYAN_SKILLFISH: | ||
| 1408 | pcache_info = navi10_cache_info; | ||
| 1409 | num_of_cache_types = ARRAY_SIZE(navi10_cache_info); | ||
| 1410 | break; | ||
| 1411 | case CHIP_NAVI14: | ||
| 1412 | pcache_info = navi14_cache_info; | ||
| 1413 | num_of_cache_types = ARRAY_SIZE(navi14_cache_info); | ||
| 1414 | break; | ||
| 1415 | case CHIP_SIENNA_CICHLID: | ||
| 1416 | pcache_info = sienna_cichlid_cache_info; | ||
| 1417 | num_of_cache_types = ARRAY_SIZE(sienna_cichlid_cache_info); | ||
| 1418 | break; | ||
| 1419 | case CHIP_NAVY_FLOUNDER: | ||
| 1420 | pcache_info = navy_flounder_cache_info; | ||
| 1421 | num_of_cache_types = ARRAY_SIZE(navy_flounder_cache_info); | ||
| 1422 | break; | ||
| 1423 | case CHIP_DIMGREY_CAVEFISH: | ||
| 1424 | pcache_info = dimgrey_cavefish_cache_info; | ||
| 1425 | num_of_cache_types = ARRAY_SIZE(dimgrey_cavefish_cache_info); | ||
| 1426 | break; | ||
| 1427 | case CHIP_VANGOGH: | ||
| 1428 | pcache_info = vangogh_cache_info; | ||
| 1429 | num_of_cache_types = ARRAY_SIZE(vangogh_cache_info); | ||
| 1430 | break; | ||
| 1431 | case CHIP_BEIGE_GOBY: | ||
| 1432 | pcache_info = beige_goby_cache_info; | ||
| 1433 | num_of_cache_types = ARRAY_SIZE(beige_goby_cache_info); | ||
| 1434 | break; | ||
| 1435 | case CHIP_YELLOW_CARP: | ||
| 1436 | pcache_info = yellow_carp_cache_info; | ||
| 1437 | num_of_cache_types = ARRAY_SIZE(yellow_carp_cache_info); | ||
| 1438 | break; | ||
| 1439 | default: | 1380 | default: |
| 1440 | return -EINVAL; | 1381 | switch(KFD_GC_VERSION(kdev)) { |
| 1382 | case IP_VERSION(9, 0, 1): | ||
| 1383 | pcache_info = vega10_cache_info; | ||
| 1384 | num_of_cache_types = ARRAY_SIZE(vega10_cache_info); | ||
| 1385 | break; | ||
| 1386 | case IP_VERSION(9, 2, 1): | ||
| 1387 | pcache_info = vega12_cache_info; | ||
| 1388 | num_of_cache_types = ARRAY_SIZE(vega12_cache_info); | ||
| 1389 | break; | ||
| 1390 | case IP_VERSION(9, 4, 0): | ||
| 1391 | case IP_VERSION(9, 4, 1): | ||
| 1392 | pcache_info = vega20_cache_info; | ||
| 1393 | num_of_cache_types = ARRAY_SIZE(vega20_cache_info); | ||
| 1394 | break; | ||
| 1395 | case IP_VERSION(9, 4, 2): | ||
| 1396 | pcache_info = aldebaran_cache_info; | ||
| 1397 | num_of_cache_types = ARRAY_SIZE(aldebaran_cache_info); | ||
| 1398 | break; | ||
| 1399 | case IP_VERSION(9, 1, 0): | ||
| 1400 | case IP_VERSION(9, 2, 2): | ||
| 1401 | pcache_info = raven_cache_info; | ||
| 1402 | num_of_cache_types = ARRAY_SIZE(raven_cache_info); | ||
| 1403 | break; | ||
| 1404 | case IP_VERSION(9, 3, 0): | ||
| 1405 | pcache_info = renoir_cache_info; | ||
| 1406 | num_of_cache_types = ARRAY_SIZE(renoir_cache_info); | ||
| 1407 | break; | ||
| 1408 | case IP_VERSION(10, 1, 10): | ||
| 1409 | case IP_VERSION(10, 1, 2): | ||
| 1410 | case IP_VERSION(10, 1, 3): | ||
| 1411 | pcache_info = navi10_cache_info; | ||
| 1412 | num_of_cache_types = ARRAY_SIZE(navi10_cache_info); | ||
| 1413 | break; | ||
| 1414 | case IP_VERSION(10, 1, 1): | ||
| 1415 | pcache_info = navi14_cache_info; | ||
| 1416 | num_of_cache_types = ARRAY_SIZE(navi14_cache_info); | ||
| 1417 | break; | ||
| 1418 | case IP_VERSION(10, 3, 0): | ||
| 1419 | pcache_info = sienna_cichlid_cache_info; | ||
| 1420 | num_of_cache_types = ARRAY_SIZE(sienna_cichlid_cache_info); | ||
| 1421 | break; | ||
| 1422 | case IP_VERSION(10, 3, 2): | ||
| 1423 | pcache_info = navy_flounder_cache_info; | ||
| 1424 | num_of_cache_types = ARRAY_SIZE(navy_flounder_cache_info); | ||
| 1425 | break; | ||
| 1426 | case IP_VERSION(10, 3, 4): | ||
| 1427 | pcache_info = dimgrey_cavefish_cache_info; | ||
| 1428 | num_of_cache_types = ARRAY_SIZE(dimgrey_cavefish_cache_info); | ||
| 1429 | break; | ||
| 1430 | case IP_VERSION(10, 3, 1): | ||
| 1431 | pcache_info = vangogh_cache_info; | ||
| 1432 | num_of_cache_types = ARRAY_SIZE(vangogh_cache_info); | ||
| 1433 | break; | ||
| 1434 | case IP_VERSION(10, 3, 5): | ||
| 1435 | pcache_info = beige_goby_cache_info; | ||
| 1436 | num_of_cache_types = ARRAY_SIZE(beige_goby_cache_info); | ||
| 1437 | break; | ||
| 1438 | case IP_VERSION(10, 3, 3): | ||
| 1439 | pcache_info = yellow_carp_cache_info; | ||
| 1440 | num_of_cache_types = ARRAY_SIZE(yellow_carp_cache_info); | ||
| 1441 | break; | ||
| 1442 | default: | ||
| 1443 | return -EINVAL; | ||
| 1444 | } | ||
| 1441 | } | 1445 | } |
| 1442 | 1446 | ||
| 1443 | *size_filled = 0; | 1447 | *size_filled = 0; |
| @@ -1963,8 +1967,6 @@ static int kfd_fill_gpu_direct_io_link_to_cpu(int *avail_size, · 519 unmodified lines | |||
| 1963 | struct crat_subtype_iolink *sub_type_hdr, | 1967 | struct crat_subtype_iolink *sub_type_hdr, |
| 1964 | uint32_t proximity_domain) | 1968 | uint32_t proximity_domain) |
| 1965 | { | 1969 | { |
| 1966 | struct amdgpu_device *adev = (struct amdgpu_device *)kdev->kgd; | ||
| 1967 | |||
| 1968 | *avail_size -= sizeof(struct crat_subtype_iolink); | 1970 | *avail_size -= sizeof(struct crat_subtype_iolink); |
| 1969 | if (*avail_size < 0) | 1971 | if (*avail_size < 0) |
| 1970 | return -ENOMEM; | 1972 | return -ENOMEM; |
| @@ -1981,7 +1983,7 @@ static int kfd_fill_gpu_direct_io_link_to_cpu(int *avail_size, · 10 unmodified lines | |||
| 1981 | /* Fill in IOLINK subtype. | 1983 | /* Fill in IOLINK subtype. |
| 1982 | * TODO: Fill-in other fields of iolink subtype | 1984 | * TODO: Fill-in other fields of iolink subtype |
| 1983 | */ | 1985 | */ |
| 1984 | if (adev->gmc.xgmi.connected_to_cpu) { | 1986 | if (kdev->adev->gmc.xgmi.connected_to_cpu) { |
| 1985 | /* | 1987 | /* |
| 1986 | * with host gpu xgmi link, host can access gpu memory whether | 1988 | * with host gpu xgmi link, host can access gpu memory whether |
| 1987 | * or not pcie bar type is large, so always create bidirectional | 1989 | * or not pcie bar type is large, so always create bidirectional |
| @@ -1990,19 +1992,19 @@ static int kfd_fill_gpu_direct_io_link_to_cpu(int *avail_size, · 2 unmodified lines | |||
| 1990 | sub_type_hdr->flags |= CRAT_IOLINK_FLAGS_BI_DIRECTIONAL; | 1992 | sub_type_hdr->flags |= CRAT_IOLINK_FLAGS_BI_DIRECTIONAL; |
| 1991 | sub_type_hdr->io_interface_type = CRAT_IOLINK_TYPE_XGMI; | 1993 | sub_type_hdr->io_interface_type = CRAT_IOLINK_TYPE_XGMI; |
| 1992 | sub_type_hdr->num_hops_xgmi = 1; | 1994 | sub_type_hdr->num_hops_xgmi = 1; |
| 1993 | if (adev->asic_type == CHIP_ALDEBARAN) { | 1995 | if (KFD_GC_VERSION(kdev) == IP_VERSION(9, 4, 2)) { |
| 1994 | sub_type_hdr->minimum_bandwidth_mbs = | 1996 | sub_type_hdr->minimum_bandwidth_mbs = |
| 1995 | amdgpu_amdkfd_get_xgmi_bandwidth_mbytes( | 1997 | amdgpu_amdkfd_get_xgmi_bandwidth_mbytes( |
| 1996 | kdev->kgd, NULL, true); | 1998 | kdev->adev, NULL, true); |
| 1997 | sub_type_hdr->maximum_bandwidth_mbs = | 1999 | sub_type_hdr->maximum_bandwidth_mbs = |
| 1998 | sub_type_hdr->minimum_bandwidth_mbs; | 2000 | sub_type_hdr->minimum_bandwidth_mbs; |
| 1999 | } | 2001 | } |
| 2000 | } else { | 2002 | } else { |
| 2001 | sub_type_hdr->io_interface_type = CRAT_IOLINK_TYPE_PCIEXPRESS; | 2003 | sub_type_hdr->io_interface_type = CRAT_IOLINK_TYPE_PCIEXPRESS; |
| 2002 | sub_type_hdr->minimum_bandwidth_mbs = | 2004 | sub_type_hdr->minimum_bandwidth_mbs = |
| 2003 | amdgpu_amdkfd_get_pcie_bandwidth_mbytes(kdev->kgd, true); | 2005 | amdgpu_amdkfd_get_pcie_bandwidth_mbytes(kdev->adev, true); |
| 2004 | sub_type_hdr->maximum_bandwidth_mbs = | 2006 | sub_type_hdr->maximum_bandwidth_mbs = |
| 2005 | amdgpu_amdkfd_get_pcie_bandwidth_mbytes(kdev->kgd, false); | 2007 | amdgpu_amdkfd_get_pcie_bandwidth_mbytes(kdev->adev, false); |
| 2006 | } | 2008 | } |
| 2007 | 2009 | ||
| 2008 | sub_type_hdr->proximity_domain_from = proximity_domain; | 2010 | sub_type_hdr->proximity_domain_from = proximity_domain; |
| @@ -2044,11 +2046,11 @@ static int kfd_fill_gpu_xgmi_link_to_gpu(int *avail_size, · 35 unmodified lines | |||
| 2044 | sub_type_hdr->proximity_domain_from = proximity_domain_from; | 2046 | sub_type_hdr->proximity_domain_from = proximity_domain_from; |
| 2045 | sub_type_hdr->proximity_domain_to = proximity_domain_to; | 2047 | sub_type_hdr->proximity_domain_to = proximity_domain_to; |
| 2046 | sub_type_hdr->num_hops_xgmi = | 2048 | sub_type_hdr->num_hops_xgmi = |
| 2047 | amdgpu_amdkfd_get_xgmi_hops_count(kdev->kgd, peer_kdev->kgd); | 2049 | amdgpu_amdkfd_get_xgmi_hops_count(kdev->adev, peer_kdev->adev); |
| 2048 | sub_type_hdr->maximum_bandwidth_mbs = | 2050 | sub_type_hdr->maximum_bandwidth_mbs = |
| 2049 | amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(kdev->kgd, peer_kdev->kgd, false); | 2051 | amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(kdev->adev, peer_kdev->adev, false); |
| 2050 | sub_type_hdr->minimum_bandwidth_mbs = sub_type_hdr->maximum_bandwidth_mbs ? | 2052 | sub_type_hdr->minimum_bandwidth_mbs = sub_type_hdr->maximum_bandwidth_mbs ? |
| 2051 | amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(kdev->kgd, NULL, true) : 0; | 2053 | amdgpu_amdkfd_get_xgmi_bandwidth_mbytes(kdev->adev, NULL, true) : 0; |
| 2052 | 2054 | ||
| 2053 | return 0; | 2055 | return 0; |
| 2054 | } | 2056 | } |
| @@ -2114,7 +2116,7 @@ static int kfd_create_vcrat_image_gpu(void *pcrat_image, · 59 unmodified lines | |||
| 2114 | cu->flags |= CRAT_CU_FLAGS_GPU_PRESENT; | 2116 | cu->flags |= CRAT_CU_FLAGS_GPU_PRESENT; |
| 2115 | cu->proximity_domain = proximity_domain; | 2117 | cu->proximity_domain = proximity_domain; |
| 2116 | 2118 | ||
| 2117 | amdgpu_amdkfd_get_cu_info(kdev->kgd, &cu_info); | 2119 | amdgpu_amdkfd_get_cu_info(kdev->adev, &cu_info); |
| 2118 | cu->num_simd_per_cu = cu_info.simd_per_cu; | 2120 | cu->num_simd_per_cu = cu_info.simd_per_cu; |
| 2119 | cu->num_simd_cores = cu_info.simd_per_cu * cu_info.cu_active_number; | 2121 | cu->num_simd_cores = cu_info.simd_per_cu * cu_info.cu_active_number; |
| 2120 | cu->max_waves_simd = cu_info.max_waves_per_simd; | 2122 | cu->max_waves_simd = cu_info.max_waves_per_simd; |
| @@ -2145,7 +2147,7 @@ static int kfd_create_vcrat_image_gpu(void *pcrat_image, · 24 unmodified lines | |||
| 2145 | * report the total FB size (public+private) as a single | 2147 | * report the total FB size (public+private) as a single |
| 2146 | * private heap. | 2148 | * private heap. |
| 2147 | */ | 2149 | */ |
| 2148 | amdgpu_amdkfd_get_local_mem_info(kdev->kgd, &local_mem_info); | 2150 | amdgpu_amdkfd_get_local_mem_info(kdev->adev, &local_mem_info); |
| 2149 | sub_type_hdr = (typeof(sub_type_hdr))((char *)sub_type_hdr + | 2151 | sub_type_hdr = (typeof(sub_type_hdr))((char *)sub_type_hdr + |
| 2150 | sub_type_hdr->length); | 2152 | sub_type_hdr->length); |
| 2151 | 2153 | ||
| @@ -41,7 +41,7 @@ · 40 unmodified lines | |||
| 41 | 41 | ||
| 42 | static void dbgdev_address_watch_disable_nodiq(struct kfd_dev *dev) | 42 | static void dbgdev_address_watch_disable_nodiq(struct kfd_dev *dev) |
| 43 | { | 43 | { |
| 44 | dev->kfd2kgd->address_watch_disable(dev->kgd); | 44 | dev->kfd2kgd->address_watch_disable(dev->adev); |
| 45 | } | 45 | } |
| 46 | 46 | ||
| 47 | static int dbgdev_diq_submit_ib(struct kfd_dbgdev *dbgdev, | 47 | static int dbgdev_diq_submit_ib(struct kfd_dbgdev *dbgdev, |
| @@ -322,7 +322,7 @@ static int dbgdev_address_watch_nodiq(struct kfd_dbgdev *dbgdev, · 274 unmodified lines | |||
| 322 | pr_debug("\t\t%30s\n", "* * * * * * * * * * * * * * * * * *"); | 322 | pr_debug("\t\t%30s\n", "* * * * * * * * * * * * * * * * * *"); |
| 323 | 323 | ||
| 324 | pdd->dev->kfd2kgd->address_watch_execute( | 324 | pdd->dev->kfd2kgd->address_watch_execute( |
| 325 | dbgdev->dev->kgd, | 325 | dbgdev->dev->adev, |
| 326 | i, | 326 | i, |
| 327 | cntl.u32All, | 327 | cntl.u32All, |
| 328 | addrHi.u32All, | 328 | addrHi.u32All, |
| @@ -420,7 +420,7 @@ static int dbgdev_address_watch_diq(struct kfd_dbgdev *dbgdev, · 91 unmodified lines | |||
| 420 | 420 | ||
| 421 | aw_reg_add_dword = | 421 | aw_reg_add_dword = |
| 422 | dbgdev->dev->kfd2kgd->address_watch_get_offset( | 422 | dbgdev->dev->kfd2kgd->address_watch_get_offset( |
| 423 | dbgdev->dev->kgd, | 423 | dbgdev->dev->adev, |
| 424 | i, | 424 | i, |
| 425 | ADDRESS_WATCH_REG_CNTL); | 425 | ADDRESS_WATCH_REG_CNTL); |
| 426 | 426 | ||
| @@ -431,7 +431,7 @@ static int dbgdev_address_watch_diq(struct kfd_dbgdev *dbgdev, · 4 unmodified lines | |||
| 431 | 431 | ||
| 432 | aw_reg_add_dword = | 432 | aw_reg_add_dword = |
| 433 | dbgdev->dev->kfd2kgd->address_watch_get_offset( | 433 | dbgdev->dev->kfd2kgd->address_watch_get_offset( |
| 434 | dbgdev->dev->kgd, | 434 | dbgdev->dev->adev, |
| 435 | i, | 435 | i, |
| 436 | ADDRESS_WATCH_REG_ADDR_HI); | 436 | ADDRESS_WATCH_REG_ADDR_HI); |
| 437 | 437 | ||
| @@ -441,7 +441,7 @@ static int dbgdev_address_watch_diq(struct kfd_dbgdev *dbgdev, · 3 unmodified lines | |||
| 441 | 441 | ||
| 442 | aw_reg_add_dword = | 442 | aw_reg_add_dword = |
| 443 | dbgdev->dev->kfd2kgd->address_watch_get_offset( | 443 | dbgdev->dev->kfd2kgd->address_watch_get_offset( |
| 444 | dbgdev->dev->kgd, | 444 | dbgdev->dev->adev, |
| 445 | i, | 445 | i, |
| 446 | ADDRESS_WATCH_REG_ADDR_LO); | 446 | ADDRESS_WATCH_REG_ADDR_LO); |
| 447 | 447 | ||
| @@ -457,7 +457,7 @@ static int dbgdev_address_watch_diq(struct kfd_dbgdev *dbgdev, · 9 unmodified lines | |||
| 457 | 457 | ||
| 458 | aw_reg_add_dword = | 458 | aw_reg_add_dword = |
| 459 | dbgdev->dev->kfd2kgd->address_watch_get_offset( | 459 | dbgdev->dev->kfd2kgd->address_watch_get_offset( |
| 460 | dbgdev->dev->kgd, | 460 | dbgdev->dev->adev, |
| 461 | i, | 461 | i, |
| 462 | ADDRESS_WATCH_REG_CNTL); | 462 | ADDRESS_WATCH_REG_CNTL); |
| 463 | 463 | ||
| @@ -752,7 +752,7 @@ static int dbgdev_wave_control_nodiq(struct kfd_dbgdev *dbgdev, · 288 unmodified lines | |||
| 752 | 752 | ||
| 753 | pr_debug("\t\t %30s\n", "* * * * * * * * * * * * * * * * * *"); | 753 | pr_debug("\t\t %30s\n", "* * * * * * * * * * * * * * * * * *"); |
| 754 | 754 | ||
| 755 | return dbgdev->dev->kfd2kgd->wave_control_execute(dbgdev->dev->kgd, | 755 | return dbgdev->dev->kfd2kgd->wave_control_execute(dbgdev->dev->adev, |
| 756 | reg_gfx_index.u32All, | 756 | reg_gfx_index.u32All, |
| 757 | reg_sq_cmd.u32All); | 757 | reg_sq_cmd.u32All); |
| 758 | } | 758 | } |
| @@ -784,7 +784,7 @@ int dbgdev_wave_reset_wavefronts(struct kfd_dev *dev, struct kfd_process *p) · 25 unmodified lines | |||
| 784 | 784 | ||
| 785 | for (vmid = first_vmid_to_scan; vmid <= last_vmid_to_scan; vmid++) { | 785 | for (vmid = first_vmid_to_scan; vmid <= last_vmid_to_scan; vmid++) { |
| 786 | status = dev->kfd2kgd->get_atc_vmid_pasid_mapping_info | 786 | status = dev->kfd2kgd->get_atc_vmid_pasid_mapping_info |
| 787 | (dev->kgd, vmid, &queried_pasid); | 787 | (dev->adev, vmid, &queried_pasid); |
| 788 | 788 | ||
| 789 | if (status && queried_pasid == p->pasid) { | 789 | if (status && queried_pasid == p->pasid) { |
| 790 | pr_debug("Killing wave fronts of vmid %d and pasid 0x%x\n", | 790 | pr_debug("Killing wave fronts of vmid %d and pasid 0x%x\n", |
| @@ -811,7 +811,7 @@ int dbgdev_wave_reset_wavefronts(struct kfd_dev *dev, struct kfd_process *p) · 20 unmodified lines | |||
| 811 | /* for non DIQ we need to patch the VMID: */ | 811 | /* for non DIQ we need to patch the VMID: */ |
| 812 | reg_sq_cmd.bits.vm_id = vmid; | 812 | reg_sq_cmd.bits.vm_id = vmid; |
| 813 | 813 | ||
| 814 | dev->kfd2kgd->wave_control_execute(dev->kgd, | 814 | dev->kfd2kgd->wave_control_execute(dev->adev, |
| 815 | reg_gfx_index.u32All, | 815 | reg_gfx_index.u32All, |
| 816 | reg_sq_cmd.u32All); | 816 | reg_sq_cmd.u32All); |
| 817 | 817 | ||
| @@ -53,770 +53,310 @@ extern const struct kfd2kgd_calls aldebaran_kfd2kgd; · 52 unmodified lines | |||
| 53 | extern const struct kfd2kgd_calls gfx_v10_kfd2kgd; | 53 | extern const struct kfd2kgd_calls gfx_v10_kfd2kgd; |
| 54 | extern const struct kfd2kgd_calls gfx_v10_3_kfd2kgd; | 54 | extern const struct kfd2kgd_calls gfx_v10_3_kfd2kgd; |
| 55 | 55 | ||
| 56 | #ifdef KFD_SUPPORT_IOMMU_V2 | ||
| 57 | static const struct kfd_device_info kaveri_device_info = { | ||
| 58 | .asic_family = CHIP_KAVERI, | ||
| 59 | .asic_name = "kaveri", | ||
| 60 | .gfx_target_version = 70000, | ||
| 61 | .max_pasid_bits = 16, | ||
| 62 | /* max num of queues for KV.TODO should be a dynamic value */ | ||
| 63 | .max_no_of_hqd = 24, | ||
| 64 | .doorbell_size = 4, | ||
| 65 | .ih_ring_entry_size = 4 * sizeof(uint32_t), | ||
| 66 | .event_interrupt_class = &event_interrupt_class_cik, | ||
| 67 | .num_of_watch_points = 4, | ||
| 68 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 69 | .supports_cwsr = false, | ||
| 70 | .needs_iommu_device = true, | ||
| 71 | .needs_pci_atomics = false, | ||
| 72 | .num_sdma_engines = 2, | ||
| 73 | .num_xgmi_sdma_engines = 0, | ||
| 74 | .num_sdma_queues_per_engine = 2, | ||
| 75 | }; | ||
| 76 | |||
| 77 | static const struct kfd_device_info carrizo_device_info = { | ||
| 78 | .asic_family = CHIP_CARRIZO, | ||
| 79 | .asic_name = "carrizo", | ||
| 80 | .gfx_target_version = 80001, | ||
| 81 | .max_pasid_bits = 16, | ||
| 82 | /* max num of queues for CZ.TODO should be a dynamic value */ | ||
| 83 | .max_no_of_hqd = 24, | ||
| 84 | .doorbell_size = 4, | ||
| 85 | .ih_ring_entry_size = 4 * sizeof(uint32_t), | ||
| 86 | .event_interrupt_class = &event_interrupt_class_cik, | ||
| 87 | .num_of_watch_points = 4, | ||
| 88 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 89 | .supports_cwsr = true, | ||
| 90 | .needs_iommu_device = true, | ||
| 91 | .needs_pci_atomics = false, | ||
| 92 | .num_sdma_engines = 2, | ||
| 93 | .num_xgmi_sdma_engines = 0, | ||
| 94 | .num_sdma_queues_per_engine = 2, | ||
| 95 | }; | ||
| 96 | |||
| 97 | static const struct kfd_device_info raven_device_info = { | ||
| 98 | .asic_family = CHIP_RAVEN, | ||
| 99 | .asic_name = "raven", | ||
| 100 | .gfx_target_version = 90002, | ||
| 101 | .max_pasid_bits = 16, | ||
| 102 | .max_no_of_hqd = 24, | ||
| 103 | .doorbell_size = 8, | ||
| 104 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 105 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 106 | .num_of_watch_points = 4, | ||
| 107 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 108 | .supports_cwsr = true, | ||
| 109 | .needs_iommu_device = true, | ||
| 110 | .needs_pci_atomics = true, | ||
| 111 | .num_sdma_engines = 1, | ||
| 112 | .num_xgmi_sdma_engines = 0, | ||
| 113 | .num_sdma_queues_per_engine = 2, | ||
| 114 | }; | ||
| 115 | #endif | ||
| 116 | |||
| 117 | #ifdef CONFIG_DRM_AMDGPU_CIK | ||
| 118 | static const struct kfd_device_info hawaii_device_info = { | ||
| 119 | .asic_family = CHIP_HAWAII, | ||
| 120 | .asic_name = "hawaii", | ||
| 121 | .gfx_target_version = 70001, | ||
| 122 | .max_pasid_bits = 16, | ||
| 123 | /* max num of queues for KV.TODO should be a dynamic value */ | ||
| 124 | .max_no_of_hqd = 24, | ||
| 125 | .doorbell_size = 4, | ||
| 126 | .ih_ring_entry_size = 4 * sizeof(uint32_t), | ||
| 127 | .event_interrupt_class = &event_interrupt_class_cik, | ||
| 128 | .num_of_watch_points = 4, | ||
| 129 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 130 | .supports_cwsr = false, | ||
| 131 | .needs_iommu_device = false, | ||
| 132 | .needs_pci_atomics = false, | ||
| 133 | .num_sdma_engines = 2, | ||
| 134 | .num_xgmi_sdma_engines = 0, | ||
| 135 | .num_sdma_queues_per_engine = 2, | ||
| 136 | }; | ||
| 137 | #endif | ||
| 138 | |||
| 139 | static const struct kfd_device_info tonga_device_info = { | ||
| 140 | .asic_family = CHIP_TONGA, | ||
| 141 | .asic_name = "tonga", | ||
| 142 | .gfx_target_version = 80002, | ||
| 143 | .max_pasid_bits = 16, | ||
| 144 | .max_no_of_hqd = 24, | ||
| 145 | .doorbell_size = 4, | ||
| 146 | .ih_ring_entry_size = 4 * sizeof(uint32_t), | ||
| 147 | .event_interrupt_class = &event_interrupt_class_cik, | ||
| 148 | .num_of_watch_points = 4, | ||
| 149 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 150 | .supports_cwsr = false, | ||
| 151 | .needs_iommu_device = false, | ||
| 152 | .needs_pci_atomics = true, | ||
| 153 | .num_sdma_engines = 2, | ||
| 154 | .num_xgmi_sdma_engines = 0, | ||
| 155 | .num_sdma_queues_per_engine = 2, | ||
| 156 | }; | ||
| 157 | |||
| 158 | static const struct kfd_device_info fiji_device_info = { | ||
| 159 | .asic_family = CHIP_FIJI, | ||
| 160 | .asic_name = "fiji", | ||
| 161 | .gfx_target_version = 80003, | ||
| 162 | .max_pasid_bits = 16, | ||
| 163 | .max_no_of_hqd = 24, | ||
| 164 | .doorbell_size = 4, | ||
| 165 | .ih_ring_entry_size = 4 * sizeof(uint32_t), | ||
| 166 | .event_interrupt_class = &event_interrupt_class_cik, | ||
| 167 | .num_of_watch_points = 4, | ||
| 168 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 169 | .supports_cwsr = true, | ||
| 170 | .needs_iommu_device = false, | ||
| 171 | .needs_pci_atomics = true, | ||
| 172 | .num_sdma_engines = 2, | ||
| 173 | .num_xgmi_sdma_engines = 0, | ||
| 174 | .num_sdma_queues_per_engine = 2, | ||
| 175 | }; | ||
| 176 | |||
| 177 | static const struct kfd_device_info fiji_vf_device_info = { | ||
| 178 | .asic_family = CHIP_FIJI, | ||
| 179 | .asic_name = "fiji", | ||
| 180 | .gfx_target_version = 80003, | ||
| 181 | .max_pasid_bits = 16, | ||
| 182 | .max_no_of_hqd = 24, | ||
| 183 | .doorbell_size = 4, | ||
| 184 | .ih_ring_entry_size = 4 * sizeof(uint32_t), | ||
| 185 | .event_interrupt_class = &event_interrupt_class_cik, | ||
| 186 | .num_of_watch_points = 4, | ||
| 187 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 188 | .supports_cwsr = true, | ||
| 189 | .needs_iommu_device = false, | ||
| 190 | .needs_pci_atomics = false, | ||
| 191 | .num_sdma_engines = 2, | ||
| 192 | .num_xgmi_sdma_engines = 0, | ||
| 193 | .num_sdma_queues_per_engine = 2, | ||
| 194 | }; | ||
| 195 | |||
| 196 | |||
| 197 | static const struct kfd_device_info polaris10_device_info = { | ||
| 198 | .asic_family = CHIP_POLARIS10, | ||
| 199 | .asic_name = "polaris10", | ||
| 200 | .gfx_target_version = 80003, | ||
| 201 | .max_pasid_bits = 16, | ||
| 202 | .max_no_of_hqd = 24, | ||
| 203 | .doorbell_size = 4, | ||
| 204 | .ih_ring_entry_size = 4 * sizeof(uint32_t), | ||
| 205 | .event_interrupt_class = &event_interrupt_class_cik, | ||
| 206 | .num_of_watch_points = 4, | ||
| 207 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 208 | .supports_cwsr = true, | ||
| 209 | .needs_iommu_device = false, | ||
| 210 | .needs_pci_atomics = true, | ||
| 211 | .num_sdma_engines = 2, | ||
| 212 | .num_xgmi_sdma_engines = 0, | ||
| 213 | .num_sdma_queues_per_engine = 2, | ||
| 214 | }; | ||
| 215 | |||
| 216 | static const struct kfd_device_info polaris10_vf_device_info = { | ||
| 217 | .asic_family = CHIP_POLARIS10, | ||
| 218 | .asic_name = "polaris10", | ||
| 219 | .gfx_target_version = 80003, | ||
| 220 | .max_pasid_bits = 16, | ||
| 221 | .max_no_of_hqd = 24, | ||
| 222 | .doorbell_size = 4, | ||
| 223 | .ih_ring_entry_size = 4 * sizeof(uint32_t), | ||
| 224 | .event_interrupt_class = &event_interrupt_class_cik, | ||
| 225 | .num_of_watch_points = 4, | ||
| 226 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 227 | .supports_cwsr = true, | ||
| 228 | .needs_iommu_device = false, | ||
| 229 | .needs_pci_atomics = false, | ||
| 230 | .num_sdma_engines = 2, | ||
| 231 | .num_xgmi_sdma_engines = 0, | ||
| 232 | .num_sdma_queues_per_engine = 2, | ||
| 233 | }; | ||
| 234 | |||
| 235 | static const struct kfd_device_info polaris11_device_info = { | ||
| 236 | .asic_family = CHIP_POLARIS11, | ||
| 237 | .asic_name = "polaris11", | ||
| 238 | .gfx_target_version = 80003, | ||
| 239 | .max_pasid_bits = 16, | ||
| 240 | .max_no_of_hqd = 24, | ||
| 241 | .doorbell_size = 4, | ||
| 242 | .ih_ring_entry_size = 4 * sizeof(uint32_t), | ||
| 243 | .event_interrupt_class = &event_interrupt_class_cik, | ||
| 244 | .num_of_watch_points = 4, | ||
| 245 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 246 | .supports_cwsr = true, | ||
| 247 | .needs_iommu_device = false, | ||
| 248 | .needs_pci_atomics = true, | ||
| 249 | .num_sdma_engines = 2, | ||
| 250 | .num_xgmi_sdma_engines = 0, | ||
| 251 | .num_sdma_queues_per_engine = 2, | ||
| 252 | }; | ||
| 253 | |||
| 254 | static const struct kfd_device_info polaris12_device_info = { | ||
| 255 | .asic_family = CHIP_POLARIS12, | ||
| 256 | .asic_name = "polaris12", | ||
| 257 | .gfx_target_version = 80003, | ||
| 258 | .max_pasid_bits = 16, | ||
| 259 | .max_no_of_hqd = 24, | ||
| 260 | .doorbell_size = 4, | ||
| 261 | .ih_ring_entry_size = 4 * sizeof(uint32_t), | ||
| 262 | .event_interrupt_class = &event_interrupt_class_cik, | ||
| 263 | .num_of_watch_points = 4, | ||
| 264 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 265 | .supports_cwsr = true, | ||
| 266 | .needs_iommu_device = false, | ||
| 267 | .needs_pci_atomics = true, | ||
| 268 | .num_sdma_engines = 2, | ||
| 269 | .num_xgmi_sdma_engines = 0, | ||
| 270 | .num_sdma_queues_per_engine = 2, | ||
| 271 | }; | ||
| 272 | |||
| 273 | static const struct kfd_device_info vegam_device_info = { | ||
| 274 | .asic_family = CHIP_VEGAM, | ||
| 275 | .asic_name = "vegam", | ||
| 276 | .gfx_target_version = 80003, | ||
| 277 | .max_pasid_bits = 16, | ||
| 278 | .max_no_of_hqd = 24, | ||
| 279 | .doorbell_size = 4, | ||
| 280 | .ih_ring_entry_size = 4 * sizeof(uint32_t), | ||
| 281 | .event_interrupt_class = &event_interrupt_class_cik, | ||
| 282 | .num_of_watch_points = 4, | ||
| 283 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 284 | .supports_cwsr = true, | ||
| 285 | .needs_iommu_device = false, | ||
| 286 | .needs_pci_atomics = true, | ||
| 287 | .num_sdma_engines = 2, | ||
| 288 | .num_xgmi_sdma_engines = 0, | ||
| 289 | .num_sdma_queues_per_engine = 2, | ||
| 290 | }; | ||
| 291 | |||
| 292 | static const struct kfd_device_info vega10_device_info = { | ||
| 293 | .asic_family = CHIP_VEGA10, | ||
| 294 | .asic_name = "vega10", | ||
| 295 | .gfx_target_version = 90000, | ||
| 296 | .max_pasid_bits = 16, | ||
| 297 | .max_no_of_hqd = 24, | ||
| 298 | .doorbell_size = 8, | ||
| 299 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 300 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 301 | .num_of_watch_points = 4, | ||
| 302 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 303 | .supports_cwsr = true, | ||
| 304 | .needs_iommu_device = false, | ||
| 305 | .needs_pci_atomics = false, | ||
| 306 | .num_sdma_engines = 2, | ||
| 307 | .num_xgmi_sdma_engines = 0, | ||
| 308 | .num_sdma_queues_per_engine = 2, | ||
| 309 | }; | ||
| 310 | |||
| 311 | static const struct kfd_device_info vega10_vf_device_info = { | ||
| 312 | .asic_family = CHIP_VEGA10, | ||
| 313 | .asic_name = "vega10", | ||
| 314 | .gfx_target_version = 90000, | ||
| 315 | .max_pasid_bits = 16, | ||
| 316 | .max_no_of_hqd = 24, | ||
| 317 | .doorbell_size = 8, | ||
| 318 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 319 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 320 | .num_of_watch_points = 4, | ||
| 321 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 322 | .supports_cwsr = true, | ||
| 323 | .needs_iommu_device = false, | ||
| 324 | .needs_pci_atomics = false, | ||
| 325 | .num_sdma_engines = 2, | ||
| 326 | .num_xgmi_sdma_engines = 0, | ||
| 327 | .num_sdma_queues_per_engine = 2, | ||
| 328 | }; | ||
| 329 | |||
| 330 | static const struct kfd_device_info vega12_device_info = { | ||
| 331 | .asic_family = CHIP_VEGA12, | ||
| 332 | .asic_name = "vega12", | ||
| 333 | .gfx_target_version = 90004, | ||
| 334 | .max_pasid_bits = 16, | ||
| 335 | .max_no_of_hqd = 24, | ||
| 336 | .doorbell_size = 8, | ||
| 337 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 338 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 339 | .num_of_watch_points = 4, | ||
| 340 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 341 | .supports_cwsr = true, | ||
| 342 | .needs_iommu_device = false, | ||
| 343 | .needs_pci_atomics = false, | ||
| 344 | .num_sdma_engines = 2, | ||
| 345 | .num_xgmi_sdma_engines = 0, | ||
| 346 | .num_sdma_queues_per_engine = 2, | ||
| 347 | }; | ||
| 348 | |||
| 349 | static const struct kfd_device_info vega20_device_info = { | ||
| 350 | .asic_family = CHIP_VEGA20, | ||
| 351 | .asic_name = "vega20", | ||
| 352 | .gfx_target_version = 90006, | ||
| 353 | .max_pasid_bits = 16, | ||
| 354 | .max_no_of_hqd = 24, | ||
| 355 | .doorbell_size = 8, | ||
| 356 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 357 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 358 | .num_of_watch_points = 4, | ||
| 359 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 360 | .supports_cwsr = true, | ||
| 361 | .needs_iommu_device = false, | ||
| 362 | .needs_pci_atomics = false, | ||
| 363 | .num_sdma_engines = 2, | ||
| 364 | .num_xgmi_sdma_engines = 0, | ||
| 365 | .num_sdma_queues_per_engine = 8, | ||
| 366 | }; | ||
| 367 | |||
| 368 | static const struct kfd_device_info arcturus_device_info = { | ||
| 369 | .asic_family = CHIP_ARCTURUS, | ||
| 370 | .asic_name = "arcturus", | ||
| 371 | .gfx_target_version = 90008, | ||
| 372 | .max_pasid_bits = 16, | ||
| 373 | .max_no_of_hqd = 24, | ||
| 374 | .doorbell_size = 8, | ||
| 375 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 376 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 377 | .num_of_watch_points = 4, | ||
| 378 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 379 | .supports_cwsr = true, | ||
| 380 | .needs_iommu_device = false, | ||
| 381 | .needs_pci_atomics = false, | ||
| 382 | .num_sdma_engines = 2, | ||
| 383 | .num_xgmi_sdma_engines = 6, | ||
| 384 | .num_sdma_queues_per_engine = 8, | ||
| 385 | }; | ||
| 386 | |||
| 387 | static const struct kfd_device_info aldebaran_device_info = { | ||
| 388 | .asic_family = CHIP_ALDEBARAN, | ||
| 389 | .asic_name = "aldebaran", | ||
| 390 | .gfx_target_version = 90010, | ||
| 391 | .max_pasid_bits = 16, | ||
| 392 | .max_no_of_hqd = 24, | ||
| 393 | .doorbell_size = 8, | ||
| 394 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 395 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 396 | .num_of_watch_points = 4, | ||
| 397 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 398 | .supports_cwsr = true, | ||
| 399 | .needs_iommu_device = false, | ||
| 400 | .needs_pci_atomics = false, | ||
| 401 | .num_sdma_engines = 2, | ||
| 402 | .num_xgmi_sdma_engines = 3, | ||
| 403 | .num_sdma_queues_per_engine = 8, | ||
| 404 | }; | ||
| 405 | |||
| 406 | static const struct kfd_device_info renoir_device_info = { | ||
| 407 | .asic_family = CHIP_RENOIR, | ||
| 408 | .asic_name = "renoir", | ||
| 409 | .gfx_target_version = 90012, | ||
| 410 | .max_pasid_bits = 16, | ||
| 411 | .max_no_of_hqd = 24, | ||
| 412 | .doorbell_size = 8, | ||
| 413 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 414 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 415 | .num_of_watch_points = 4, | ||
| 416 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 417 | .supports_cwsr = true, | ||
| 418 | .needs_iommu_device = false, | ||
| 419 | .needs_pci_atomics = false, | ||
| 420 | .num_sdma_engines = 1, | ||
| 421 | .num_xgmi_sdma_engines = 0, | ||
| 422 | .num_sdma_queues_per_engine = 2, | ||
| 423 | }; | ||
| 424 | |||
| 425 | static const struct kfd_device_info navi10_device_info = { | ||
| 426 | .asic_family = CHIP_NAVI10, | ||
| 427 | .asic_name = "navi10", | ||
| 428 | .gfx_target_version = 100100, | ||
| 429 | .max_pasid_bits = 16, | ||
| 430 | .max_no_of_hqd = 24, | ||
| 431 | .doorbell_size = 8, | ||
| 432 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 433 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 434 | .num_of_watch_points = 4, | ||
| 435 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 436 | .needs_iommu_device = false, | ||
| 437 | .supports_cwsr = true, | ||
| 438 | .needs_pci_atomics = true, | ||
| 439 | .no_atomic_fw_version = 145, | ||
| 440 | .num_sdma_engines = 2, | ||
| 441 | .num_xgmi_sdma_engines = 0, | ||
| 442 | .num_sdma_queues_per_engine = 8, | ||
| 443 | }; | ||
| 444 | |||
| 445 | static const struct kfd_device_info navi12_device_info = { | ||
| 446 | .asic_family = CHIP_NAVI12, | ||
| 447 | .asic_name = "navi12", | ||
| 448 | .gfx_target_version = 100101, | ||
| 449 | .max_pasid_bits = 16, | ||
| 450 | .max_no_of_hqd = 24, | ||
| 451 | .doorbell_size = 8, | ||
| 452 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 453 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 454 | .num_of_watch_points = 4, | ||
| 455 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 456 | .needs_iommu_device = false, | ||
| 457 | .supports_cwsr = true, | ||
| 458 | .needs_pci_atomics = true, | ||
| 459 | .no_atomic_fw_version = 145, | ||
| 460 | .num_sdma_engines = 2, | ||
| 461 | .num_xgmi_sdma_engines = 0, | ||
| 462 | .num_sdma_queues_per_engine = 8, | ||
| 463 | }; | ||
| 464 | |||
| 465 | static const struct kfd_device_info navi14_device_info = { | ||
| 466 | .asic_family = CHIP_NAVI14, | ||
| 467 | .asic_name = "navi14", | ||
| 468 | .gfx_target_version = 100102, | ||
| 469 | .max_pasid_bits = 16, | ||
| 470 | .max_no_of_hqd = 24, | ||
| 471 | .doorbell_size = 8, | ||
| 472 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 473 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 474 | .num_of_watch_points = 4, | ||
| 475 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 476 | .needs_iommu_device = false, | ||
| 477 | .supports_cwsr = true, | ||
| 478 | .needs_pci_atomics = true, | ||
| 479 | .no_atomic_fw_version = 145, | ||
| 480 | .num_sdma_engines = 2, | ||
| 481 | .num_xgmi_sdma_engines = 0, | ||
| 482 | .num_sdma_queues_per_engine = 8, | ||
| 483 | }; | ||
| 484 | |||
| 485 | static const struct kfd_device_info sienna_cichlid_device_info = { | ||
| 486 | .asic_family = CHIP_SIENNA_CICHLID, | ||
| 487 | .asic_name = "sienna_cichlid", | ||
| 488 | .gfx_target_version = 100300, | ||
| 489 | .max_pasid_bits = 16, | ||
| 490 | .max_no_of_hqd = 24, | ||
| 491 | .doorbell_size = 8, | ||
| 492 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 493 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 494 | .num_of_watch_points = 4, | ||
| 495 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 496 | .needs_iommu_device = false, | ||
| 497 | .supports_cwsr = true, | ||
| 498 | .needs_pci_atomics = true, | ||
| 499 | .no_atomic_fw_version = 92, | ||
| 500 | .num_sdma_engines = 4, | ||
| 501 | .num_xgmi_sdma_engines = 0, | ||
| 502 | .num_sdma_queues_per_engine = 8, | ||
| 503 | }; | ||
| 504 | |||
| 505 | static const struct kfd_device_info navy_flounder_device_info = { | ||
| 506 | .asic_family = CHIP_NAVY_FLOUNDER, | ||
| 507 | .asic_name = "navy_flounder", | ||
| 508 | .gfx_target_version = 100301, | ||
| 509 | .max_pasid_bits = 16, | ||
| 510 | .max_no_of_hqd = 24, | ||
| 511 | .doorbell_size = 8, | ||
| 512 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 513 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 514 | .num_of_watch_points = 4, | ||
| 515 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 516 | .needs_iommu_device = false, | ||
| 517 | .supports_cwsr = true, | ||
| 518 | .needs_pci_atomics = true, | ||
| 519 | .no_atomic_fw_version = 92, | ||
| 520 | .num_sdma_engines = 2, | ||
| 521 | .num_xgmi_sdma_engines = 0, | ||
| 522 | .num_sdma_queues_per_engine = 8, | ||
| 523 | }; | ||
| 524 | |||
| 525 | static const struct kfd_device_info vangogh_device_info = { | ||
| 526 | .asic_family = CHIP_VANGOGH, | ||
| 527 | .asic_name = "vangogh", | ||
| 528 | .gfx_target_version = 100303, | ||
| 529 | .max_pasid_bits = 16, | ||
| 530 | .max_no_of_hqd = 24, | ||
| 531 | .doorbell_size = 8, | ||
| 532 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 533 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 534 | .num_of_watch_points = 4, | ||
| 535 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 536 | .needs_iommu_device = false, | ||
| 537 | .supports_cwsr = true, | ||
| 538 | .needs_pci_atomics = true, | ||
| 539 | .no_atomic_fw_version = 92, | ||
| 540 | .num_sdma_engines = 1, | ||
| 541 | .num_xgmi_sdma_engines = 0, | ||
| 542 | .num_sdma_queues_per_engine = 2, | ||
| 543 | }; | ||
| 544 | |||
| 545 | static const struct kfd_device_info dimgrey_cavefish_device_info = { | ||
| 546 | .asic_family = CHIP_DIMGREY_CAVEFISH, | ||
| 547 | .asic_name = "dimgrey_cavefish", | ||
| 548 | .gfx_target_version = 100302, | ||
| 549 | .max_pasid_bits = 16, | ||
| 550 | .max_no_of_hqd = 24, | ||
| 551 | .doorbell_size = 8, | ||
| 552 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 553 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 554 | .num_of_watch_points = 4, | ||
| 555 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 556 | .needs_iommu_device = false, | ||
| 557 | .supports_cwsr = true, | ||
| 558 | .needs_pci_atomics = true, | ||
| 559 | .no_atomic_fw_version = 92, | ||
| 560 | .num_sdma_engines = 2, | ||
| 561 | .num_xgmi_sdma_engines = 0, | ||
| 562 | .num_sdma_queues_per_engine = 8, | ||
| 563 | }; | ||
| 564 | |||
| 565 | static const struct kfd_device_info beige_goby_device_info = { | ||
| 566 | .asic_family = CHIP_BEIGE_GOBY, | ||
| 567 | .asic_name = "beige_goby", | ||
| 568 | .gfx_target_version = 100304, | ||
| 569 | .max_pasid_bits = 16, | ||
| 570 | .max_no_of_hqd = 24, | ||
| 571 | .doorbell_size = 8, | ||
| 572 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 573 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 574 | .num_of_watch_points = 4, | ||
| 575 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 576 | .needs_iommu_device = false, | ||
| 577 | .supports_cwsr = true, | ||
| 578 | .needs_pci_atomics = true, | ||
| 579 | .no_atomic_fw_version = 92, | ||
| 580 | .num_sdma_engines = 1, | ||
| 581 | .num_xgmi_sdma_engines = 0, | ||
| 582 | .num_sdma_queues_per_engine = 8, | ||
| 583 | }; | ||
| 584 | |||
| 585 | static const struct kfd_device_info yellow_carp_device_info = { | ||
| 586 | .asic_family = CHIP_YELLOW_CARP, | ||
| 587 | .asic_name = "yellow_carp", | ||
| 588 | .gfx_target_version = 100305, | ||
| 589 | .max_pasid_bits = 16, | ||
| 590 | .max_no_of_hqd = 24, | ||
| 591 | .doorbell_size = 8, | ||
| 592 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 593 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 594 | .num_of_watch_points = 4, | ||
| 595 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 596 | .needs_iommu_device = false, | ||
| 597 | .supports_cwsr = true, | ||
| 598 | .needs_pci_atomics = true, | ||
| 599 | .no_atomic_fw_version = 92, | ||
| 600 | .num_sdma_engines = 1, | ||
| 601 | .num_xgmi_sdma_engines = 0, | ||
| 602 | .num_sdma_queues_per_engine = 2, | ||
| 603 | }; | ||
| 604 | |||
| 605 | static const struct kfd_device_info cyan_skillfish_device_info = { | ||
| 606 | .asic_family = CHIP_CYAN_SKILLFISH, | ||
| 607 | .asic_name = "cyan_skillfish", | ||
| 608 | .gfx_target_version = 100103, | ||
| 609 | .max_pasid_bits = 16, | ||
| 610 | .max_no_of_hqd = 24, | ||
| 611 | .doorbell_size = 8, | ||
| 612 | .ih_ring_entry_size = 8 * sizeof(uint32_t), | ||
| 613 | .event_interrupt_class = &event_interrupt_class_v9, | ||
| 614 | .num_of_watch_points = 4, | ||
| 615 | .mqd_size_aligned = MQD_SIZE_ALIGNED, | ||
| 616 | .needs_iommu_device = false, | ||
| 617 | .supports_cwsr = true, | ||
| 618 | .needs_pci_atomics = true, | ||
| 619 | .num_sdma_engines = 2, | ||
| 620 | .num_xgmi_sdma_engines = 0, | ||
| 621 | .num_sdma_queues_per_engine = 8, | ||
| 622 | }; | ||
| 623 | |||
| 624 | static int kfd_gtt_sa_init(struct kfd_dev *kfd, unsigned int buf_size, | 56 | static int kfd_gtt_sa_init(struct kfd_dev *kfd, unsigned int buf_size, |
| 625 | unsigned int chunk_size); | 57 | unsigned int chunk_size); |
| 626 | static void kfd_gtt_sa_fini(struct kfd_dev *kfd); | 58 | static void kfd_gtt_sa_fini(struct kfd_dev *kfd); |
| 627 | 59 | ||
| 628 | static int kfd_resume(struct kfd_dev *kfd); | 60 | static int kfd_resume(struct kfd_dev *kfd); |
| 629 | 61 | ||
| 630 | struct kfd_dev *kgd2kfd_probe(struct kgd_dev *kgd, bool vf) | 62 | static void kfd_device_info_set_sdma_queue_num(struct kfd_dev *kfd) |
| 631 | { | 63 | { |
| 632 | struct kfd_dev *kfd; | 64 | uint32_t sdma_version = kfd->adev->ip_versions[SDMA0_HWIP][0]; |
| 633 | const struct kfd_device_info *device_info; | 65 | |
| 634 | const struct kfd2kgd_calls *f2g; | 66 | switch (sdma_version) { |
| 635 | struct amdgpu_device *adev = (struct amdgpu_device *)kgd; | 67 | case IP_VERSION(4, 0, 0):/* VEGA10 */ |
| 68 | case IP_VERSION(4, 0, 1):/* VEGA12 */ | ||
| 69 | case IP_VERSION(4, 1, 0):/* RAVEN */ | ||
| 70 | case IP_VERSION(4, 1, 1):/* RAVEN */ | ||
| 71 | case IP_VERSION(4, 1, 2):/* RENIOR */ | ||
| 72 | case IP_VERSION(5, 2, 1):/* VANGOGH */ | ||
| 73 | case IP_VERSION(5, 2, 3):/* YELLOW_CARP */ | ||
| 74 | kfd->device_info.num_sdma_queues_per_engine = 2; | ||
| 75 | break; | ||
| 76 | case IP_VERSION(4, 2, 0):/* VEGA20 */ | ||
| 77 | case IP_VERSION(4, 2, 2):/* ARCTUTUS */ | ||
| 78 | case IP_VERSION(4, 4, 0):/* ALDEBARAN */ | ||
| 79 | case IP_VERSION(5, 0, 0):/* NAVI10 */ | ||
| 80 | case IP_VERSION(5, 0, 1):/* CYAN_SKILLFISH */ | ||
| 81 | case IP_VERSION(5, 0, 2):/* NAVI14 */ | ||
| 82 | case IP_VERSION(5, 0, 5):/* NAVI12 */ | ||
| 83 | case IP_VERSION(5, 2, 0):/* SIENNA_CICHLID */ | ||
| 84 | case IP_VERSION(5, 2, 2):/* NAVY_FLOUDER */ | ||
| 85 | case IP_VERSION(5, 2, 4):/* DIMGREY_CAVEFISH */ | ||
| 86 | case IP_VERSION(5, 2, 5):/* BEIGE_GOBY */ | ||
| 87 | kfd->device_info.num_sdma_queues_per_engine = 8; | ||
| 88 | break; | ||
| 89 | default: | ||
| 90 | dev_warn(kfd_device, | ||
| 91 | "Default sdma queue per engine(8) is set due to " | ||
| 92 | "mismatch of sdma ip block(SDMA_HWIP:0x%x).\n", | ||
| 93 | sdma_version); | ||
| 94 | kfd->device_info.num_sdma_queues_per_engine = 8; | ||
| 95 | } | ||
| 96 | } | ||
| 97 | |||
| 98 | static void kfd_device_info_set_event_interrupt_class(struct kfd_dev *kfd) | ||
| 99 | { | ||
| 100 | uint32_t gc_version = KFD_GC_VERSION(kfd); | ||
| 101 | |||
| 102 | switch (gc_version) { | ||
| 103 | case IP_VERSION(9, 0, 1): /* VEGA10 */ | ||
| 104 | case IP_VERSION(9, 1, 0): /* RAVEN */ | ||
| 105 | case IP_VERSION(9, 2, 1): /* VEGA12 */ | ||
| 106 | case IP_VERSION(9, 2, 2): /* RAVEN */ | ||
| 107 | case IP_VERSION(9, 3, 0): /* RENOIR */ | ||
| 108 | case IP_VERSION(9, 4, 0): /* VEGA20 */ | ||
| 109 | case IP_VERSION(9, 4, 1): /* ARCTURUS */ | ||
| 110 | case IP_VERSION(9, 4, 2): /* ALDEBARAN */ | ||
| 111 | case IP_VERSION(10, 3, 1): /* VANGOGH */ | ||
| 112 | case IP_VERSION(10, 3, 3): /* YELLOW_CARP */ | ||
| 113 | case IP_VERSION(10, 1, 3): /* CYAN_SKILLFISH */ | ||
| 114 | case IP_VERSION(10, 1, 10): /* NAVI10 */ | ||
| 115 | case IP_VERSION(10, 1, 2): /* NAVI12 */ | ||
| 116 | case IP_VERSION(10, 1, 1): /* NAVI14 */ | ||
| 117 | case IP_VERSION(10, 3, 0): /* SIENNA_CICHLID */ | ||
| 118 | case IP_VERSION(10, 3, 2): /* NAVY_FLOUNDER */ | ||
| 119 | case IP_VERSION(10, 3, 4): /* DIMGREY_CAVEFISH */ | ||
| 120 | case IP_VERSION(10, 3, 5): /* BEIGE_GOBY */ | ||
| 121 | kfd->device_info.event_interrupt_class = &event_interrupt_class_v9; | ||
| 122 | break; | ||
| 123 | default: | ||
| 124 | dev_warn(kfd_device, "v9 event interrupt handler is set due to " | ||
| 125 | "mismatch of gc ip block(GC_HWIP:0x%x).\n", gc_version); | ||
| 126 | kfd->device_info.event_interrupt_class = &event_interrupt_class_v9; | ||
| 127 | } | ||
| 128 | } | ||
| 129 | |||
| 130 | static void kfd_device_info_init(struct kfd_dev *kfd, | ||
| 131 | bool vf, uint32_t gfx_target_version) | ||
| 132 | { | ||
| 133 | uint32_t gc_version = KFD_GC_VERSION(kfd); | ||
| 134 | uint32_t asic_type = kfd->adev->asic_type; | ||
| 135 | |||
| 136 | kfd->device_info.max_pasid_bits = 16; | ||
| 137 | kfd->device_info.max_no_of_hqd = 24; | ||
| 138 | kfd->device_info.num_of_watch_points = 4; | ||
| 139 | kfd->device_info.mqd_size_aligned = MQD_SIZE_ALIGNED; | ||
| 140 | kfd->device_info.gfx_target_version = gfx_target_version; | ||
| 141 | |||
| 142 | if (KFD_IS_SOC15(kfd)) { | ||
| 143 | kfd->device_info.doorbell_size = 8; | ||
| 144 | kfd->device_info.ih_ring_entry_size = 8 * sizeof(uint32_t); | ||
| 145 | kfd->device_info.supports_cwsr = true; | ||
| 146 | |||
| 147 | kfd_device_info_set_sdma_queue_num(kfd); | ||
| 148 | |||
| 149 | kfd_device_info_set_event_interrupt_class(kfd); | ||
| 150 | |||
| 151 | /* Raven */ | ||
| 152 | if (gc_version == IP_VERSION(9, 1, 0) || | ||
| 153 | gc_version == IP_VERSION(9, 2, 2)) | ||
| 154 | kfd->device_info.needs_iommu_device = true; | ||
| 155 | |||
| 156 | if (gc_version < IP_VERSION(11, 0, 0)) { | ||
| 157 | /* Navi2x+, Navi1x+ */ | ||
| 158 | if (gc_version >= IP_VERSION(10, 3, 0)) | ||
| 159 | kfd->device_info.no_atomic_fw_version = 92; | ||
| 160 | else if (gc_version >= IP_VERSION(10, 1, 1)) | ||
| 161 | kfd->device_info.no_atomic_fw_version = 145; | ||
| 162 | |||
| 163 | /* Navi1x+ */ | ||
| 164 | if (gc_version >= IP_VERSION(10, 1, 1)) | ||
| 165 | kfd->device_info.needs_pci_atomics = true; | ||
| 166 | } | ||
| 167 | } else { | ||
| 168 | kfd->device_info.doorbell_size = 4; | ||
| 169 | kfd->device_info.ih_ring_entry_size = 4 * sizeof(uint32_t); | ||
| 170 | kfd->device_info.event_interrupt_class = &event_interrupt_class_cik; | ||
| 171 | kfd->device_info.num_sdma_queues_per_engine = 2; | ||
| 172 | |||
| 173 | if (asic_type != CHIP_KAVERI && | ||
| 174 | asic_type != CHIP_HAWAII && | ||
| 175 | asic_type != CHIP_TONGA) | ||
| 176 | kfd->device_info.supports_cwsr = true; | ||
| 177 | |||
| 178 | if (asic_type == CHIP_KAVERI || | ||
| 179 | asic_type == CHIP_CARRIZO) | ||
| 180 | kfd->device_info.needs_iommu_device = true; | ||
| 181 | |||
| 182 | if (asic_type != CHIP_HAWAII && !vf) | ||
| 183 | kfd->device_info.needs_pci_atomics = true; | ||
| 184 | } | ||
| 185 | } | ||
| 186 | |||
| 187 | struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf) | ||
| 188 | { | ||
| 189 | struct kfd_dev *kfd = NULL; | ||
| 190 | const struct kfd2kgd_calls *f2g = NULL; | ||
| 636 | struct pci_dev *pdev = adev->pdev; | 191 | struct pci_dev *pdev = adev->pdev; |
| 192 | uint32_t gfx_target_version = 0; | ||
| 637 | 193 | ||
| 638 | switch (adev->asic_type) { | 194 | switch (adev->asic_type) { |
| 639 | #ifdef KFD_SUPPORT_IOMMU_V2 | 195 | #ifdef KFD_SUPPORT_IOMMU_V2 |
| 640 | #ifdef CONFIG_DRM_AMDGPU_CIK | 196 | #ifdef CONFIG_DRM_AMDGPU_CIK |
| 641 | case CHIP_KAVERI: | 197 | case CHIP_KAVERI: |
| 642 | if (vf) | 198 | gfx_target_version = 70000; |
| 643 | device_info = NULL; | 199 | if (!vf) |
| 644 | else | 200 | f2g = &gfx_v7_kfd2kgd; |
| 645 | device_info = &kaveri_device_info; | ||
| 646 | f2g = &gfx_v7_kfd2kgd; | ||
| 647 | break; | 201 | break; |
| 648 | #endif | 202 | #endif |
| 649 | case CHIP_CARRIZO: | 203 | case CHIP_CARRIZO: |
| 650 | if (vf) | 204 | gfx_target_version = 80001; |
| 651 | device_info = NULL; | 205 | if (!vf) |
| 652 | else | 206 | f2g = &gfx_v8_kfd2kgd; |
| 653 | device_info = &carrizo_device_info; | ||
| 654 | f2g = &gfx_v8_kfd2kgd; | ||
| 655 | break; | 207 | break; |
| 656 | #endif | 208 | #endif |
| 657 | #ifdef CONFIG_DRM_AMDGPU_CIK | 209 | #ifdef CONFIG_DRM_AMDGPU_CIK |
| 658 | case CHIP_HAWAII: | 210 | case CHIP_HAWAII: |
| 659 | if (vf) | 211 | gfx_target_version = 70001; |
| 660 | device_info = NULL; | 212 | if (!amdgpu_exp_hw_support) |
| 661 | else | 213 | pr_info( |
| 662 | device_info = &hawaii_device_info; | 214 | "KFD support on Hawaii is experimental. See modparam exp_hw_support\n" |
| 663 | f2g = &gfx_v7_kfd2kgd; | 215 | ); |
| 216 | else if (!vf) | ||
| 217 | f2g = &gfx_v7_kfd2kgd; | ||
| 664 | break; | 218 | break; |
| 665 | #endif | 219 | #endif |
| 666 | case CHIP_TONGA: | 220 | case CHIP_TONGA: |
| 667 | if (vf) | 221 | gfx_target_version = 80002; |
| 668 | device_info = NULL; | 222 | if (!vf) |
| 669 | else | 223 | f2g = &gfx_v8_kfd2kgd; |
| 670 | device_info = &tonga_device_info; | ||
| 671 | f2g = &gfx_v8_kfd2kgd; | ||
| 672 | break; | 224 | break; |
| 673 | case CHIP_FIJI: | 225 | case CHIP_FIJI: |
| 674 | if (vf) | 226 | gfx_target_version = 80003; |
| 675 | device_info = &fiji_vf_device_info; | ||
| 676 | else | ||
| 677 | device_info = &fiji_device_info; | ||
| 678 | f2g = &gfx_v8_kfd2kgd; | 227 | f2g = &gfx_v8_kfd2kgd; |
| 679 | break; | 228 | break; |
| 680 | case CHIP_POLARIS10: | 229 | case CHIP_POLARIS10: |
| 681 | if (vf) | 230 | gfx_target_version = 80003; |
| 682 | device_info = &polaris10_vf_device_info; | ||
| 683 | else | ||
| 684 | device_info = &polaris10_device_info; | ||
| 685 | f2g = &gfx_v8_kfd2kgd; | 231 | f2g = &gfx_v8_kfd2kgd; |
| 686 | break; | 232 | break; |
| 687 | case CHIP_POLARIS11: | 233 | case CHIP_POLARIS11: |
| 688 | if (vf) | 234 | gfx_target_version = 80003; |
| 689 | device_info = NULL; | 235 | if (!vf) |
| 690 | else | 236 | f2g = &gfx_v8_kfd2kgd; |
| 691 | device_info = &polaris11_device_info; | ||
| 692 | f2g = &gfx_v8_kfd2kgd; | ||
| 693 | break; | 237 | break; |
| 694 | case CHIP_POLARIS12: | 238 | case CHIP_POLARIS12: |
| 695 | if (vf) | 239 | gfx_target_version = 80003; |
| 696 | device_info = NULL; | 240 | if (!vf) |
| 697 | else | 241 | f2g = &gfx_v8_kfd2kgd; |
| 698 | device_info = &polaris12_device_info; | ||
| 699 | f2g = &gfx_v8_kfd2kgd; | ||
| 700 | break; | 242 | break; |
| 701 | case CHIP_VEGAM: | 243 | case CHIP_VEGAM: |
| 702 | if (vf) | 244 | gfx_target_version = 80003; |
| 703 | device_info = NULL; | 245 | if (!vf) |
| 704 | else | 246 | f2g = &gfx_v8_kfd2kgd; |
| 705 | device_info = &vegam_device_info; | ||
| 706 | f2g = &gfx_v8_kfd2kgd; | ||
| 707 | break; | 247 | break; |
| 708 | default: | 248 | default: |
| 709 | switch (adev->ip_versions[GC_HWIP][0]) { | 249 | switch (adev->ip_versions[GC_HWIP][0]) { |
| 250 | /* Vega 10 */ | ||
| 710 | case IP_VERSION(9, 0, 1): | 251 | case IP_VERSION(9, 0, 1): |
| 711 | if (vf) | 252 | gfx_target_version = 90000; |
| 712 | device_info = &vega10_vf_device_info; | ||
| 713 | else | ||
| 714 | device_info = &vega10_device_info; | ||
| 715 | f2g = &gfx_v9_kfd2kgd; | 253 | f2g = &gfx_v9_kfd2kgd; |
| 716 | break; | 254 | break; |
| 717 | #ifdef KFD_SUPPORT_IOMMU_V2 | 255 | #ifdef KFD_SUPPORT_IOMMU_V2 |
| 256 | /* Raven */ | ||
| 718 | case IP_VERSION(9, 1, 0): | 257 | case IP_VERSION(9, 1, 0): |
| 719 | case IP_VERSION(9, 2, 2): | 258 | case IP_VERSION(9, 2, 2): |
| 720 | if (vf) | 259 | gfx_target_version = 90002; |
| 721 | device_info = NULL; | 260 | if (!vf) |
| 722 | else | 261 | f2g = &gfx_v9_kfd2kgd; |
| 723 | device_info = &raven_device_info; | ||
| 724 | f2g = &gfx_v9_kfd2kgd; | ||
| 725 | break; | 262 | break; |
| 726 | #endif | 263 | #endif |
| 264 | /* Vega12 */ | ||
| 727 | case IP_VERSION(9, 2, 1): | 265 | case IP_VERSION(9, 2, 1): |
| 728 | if (vf) | 266 | gfx_target_version = 90004; |
| 729 | device_info = NULL; | 267 | if (!vf) |
| 730 | else | 268 | f2g = &gfx_v9_kfd2kgd; |
| 731 | device_info = &vega12_device_info; | ||
| 732 | f2g = &gfx_v9_kfd2kgd; | ||
| 733 | break; | 269 | break; |
| 270 | /* Renoir */ | ||
| 734 | case IP_VERSION(9, 3, 0): | 271 | case IP_VERSION(9, 3, 0): |
| 735 | if (vf) | 272 | gfx_target_version = 90012; |
| 736 | device_info = NULL; | 273 | if (!vf) |
| 737 | else | 274 | f2g = &gfx_v9_kfd2kgd; |
| 738 | device_info = &renoir_device_info; | ||
| 739 | f2g = &gfx_v9_kfd2kgd; | ||
| 740 | break; | 275 | break; |
| 276 | /* Vega20 */ | ||
| 741 | case IP_VERSION(9, 4, 0): | 277 | case IP_VERSION(9, 4, 0): |
| 742 | if (vf) | 278 | gfx_target_version = 90006; |
| 743 | device_info = NULL; | 279 | if (!vf) |
| 744 | else | 280 | f2g = &gfx_v9_kfd2kgd; |
| 745 | device_info = &vega20_device_info; | ||
| 746 | f2g = &gfx_v9_kfd2kgd; | ||
| 747 | break; | 281 | break; |
| 282 | /* Arcturus */ | ||
| 748 | case IP_VERSION(9, 4, 1): | 283 | case IP_VERSION(9, 4, 1): |
| 749 | device_info = &arcturus_device_info; | 284 | gfx_target_version = 90008; |
| 750 | f2g = &arcturus_kfd2kgd; | 285 | f2g = &arcturus_kfd2kgd; |
| 751 | break; | 286 | break; |
| 287 | /* Aldebaran */ | ||
| 752 | case IP_VERSION(9, 4, 2): | 288 | case IP_VERSION(9, 4, 2): |
| 753 | device_info = &aldebaran_device_info; | 289 | gfx_target_version = 90010; |
| 754 | f2g = &aldebaran_kfd2kgd; | 290 | f2g = &aldebaran_kfd2kgd; |
| 755 | break; | 291 | break; |
| 292 | /* Navi10 */ | ||
| 756 | case IP_VERSION(10, 1, 10): | 293 | case IP_VERSION(10, 1, 10): |
| 757 | if (vf) | 294 | gfx_target_version = 100100; |
| 758 | device_info = NULL; | 295 | if (!vf) |
| 759 | else | 296 | f2g = &gfx_v10_kfd2kgd; |
| 760 | device_info = &navi10_device_info; | ||
| 761 | f2g = &gfx_v10_kfd2kgd; | ||
| 762 | break; | 297 | break; |
| 298 | /* Navi12 */ | ||
| 763 | case IP_VERSION(10, 1, 2): | 299 | case IP_VERSION(10, 1, 2): |
| 764 | device_info = &navi12_device_info; | 300 | gfx_target_version = 100101; |
| 765 | f2g = &gfx_v10_kfd2kgd; | 301 | f2g = &gfx_v10_kfd2kgd; |
| 766 | break; | 302 | break; |
| 303 | /* Navi14 */ | ||
| 767 | case IP_VERSION(10, 1, 1): | 304 | case IP_VERSION(10, 1, 1): |
| 768 | if (vf) | 305 | gfx_target_version = 100102; |
| 769 | device_info = NULL; | 306 | if (!vf) |
| 770 | else | 307 | f2g = &gfx_v10_kfd2kgd; |
| 771 | device_info = &navi14_device_info; | ||
| 772 | f2g = &gfx_v10_kfd2kgd; | ||
| 773 | break; | 308 | break; |
| 309 | /* Cyan Skillfish */ | ||
| 774 | case IP_VERSION(10, 1, 3): | 310 | case IP_VERSION(10, 1, 3): |
| 775 | if (vf) | 311 | gfx_target_version = 100103; |
| 776 | device_info = NULL; | 312 | if (!vf) |
| 777 | else | 313 | f2g = &gfx_v10_kfd2kgd; |
| 778 | device_info = &cyan_skillfish_device_info; | ||
| 779 | f2g = &gfx_v10_kfd2kgd; | ||
| 780 | break; | 314 | break; |
| 315 | /* Sienna Cichlid */ | ||
| 781 | case IP_VERSION(10, 3, 0): | 316 | case IP_VERSION(10, 3, 0): |
| 782 | device_info = &sienna_cichlid_device_info; | 317 | gfx_target_version = 100300; |
| 783 | f2g = &gfx_v10_3_kfd2kgd; | 318 | f2g = &gfx_v10_3_kfd2kgd; |
| 784 | break; | 319 | break; |
| 320 | /* Navy Flounder */ | ||
| 785 | case IP_VERSION(10, 3, 2): | 321 | case IP_VERSION(10, 3, 2): |
| 786 | device_info = &navy_flounder_device_info; | 322 | gfx_target_version = 100301; |
| 787 | f2g = &gfx_v10_3_kfd2kgd; | 323 | f2g = &gfx_v10_3_kfd2kgd; |
| 788 | break; | 324 | break; |
| 325 | /* Van Gogh */ | ||
| 789 | case IP_VERSION(10, 3, 1): | 326 | case IP_VERSION(10, 3, 1): |
| 790 | if (vf) | 327 | gfx_target_version = 100303; |
| 791 | device_info = NULL; | 328 | if (!vf) |
| 792 | else | 329 | f2g = &gfx_v10_3_kfd2kgd; |
| 793 | device_info = &vangogh_device_info; | ||
| 794 | f2g = &gfx_v10_3_kfd2kgd; | ||
| 795 | break; | 330 | break; |
| 331 | /* Dimgrey Cavefish */ | ||
| 796 | case IP_VERSION(10, 3, 4): | 332 | case IP_VERSION(10, 3, 4): |
| 797 | device_info = &dimgrey_cavefish_device_info; | 333 | gfx_target_version = 100302; |
| 798 | f2g = &gfx_v10_3_kfd2kgd; | 334 | f2g = &gfx_v10_3_kfd2kgd; |
| 799 | break; | 335 | break; |
| 336 | /* Beige Goby */ | ||
| 800 | case IP_VERSION(10, 3, 5): | 337 | case IP_VERSION(10, 3, 5): |
| 801 | device_info = &beige_goby_device_info; | 338 | gfx_target_version = 100304; |
| 802 | f2g = &gfx_v10_3_kfd2kgd; | 339 | f2g = &gfx_v10_3_kfd2kgd; |
| 803 | break; | 340 | break; |
| 341 | /* Yellow Carp */ | ||
| 804 | case IP_VERSION(10, 3, 3): | 342 | case IP_VERSION(10, 3, 3): |
| 805 | if (vf) | 343 | gfx_target_version = 100305; |
| 806 | device_info = NULL; | 344 | if (!vf) |
| 807 | else | 345 | f2g = &gfx_v10_3_kfd2kgd; |
| 808 | device_info = &yellow_carp_device_info; | ||
| 809 | f2g = &gfx_v10_3_kfd2kgd; | ||
| 810 | break; | 346 | break; |
| 811 | default: | 347 | default: |
| 812 | return NULL; | 348 | break; |
| 813 | } | 349 | } |
| 814 | break; | 350 | break; |
| 815 | } | 351 | } |
| 816 | 352 | ||
| 817 | if (!device_info || !f2g) { | 353 | if (!f2g) { |
| 818 | dev_err(kfd_device, "%s %s not supported in kfd\n", | 354 | if (adev->ip_versions[GC_HWIP][0]) |
| 819 | amdgpu_asic_name[adev->asic_type], vf ? "VF" : ""); | 355 | dev_err(kfd_device, "GC IP %06x %s not supported in kfd\n", |
| 356 | adev->ip_versions[GC_HWIP][0], vf ? "VF" : ""); | ||
| 357 | else | ||
| 358 | dev_err(kfd_device, "%s %s not supported in kfd\n", | ||
| 359 | amdgpu_asic_name[adev->asic_type], vf ? "VF" : ""); | ||
| 820 | return NULL; | 360 | return NULL; |
| 821 | } | 361 | } |
| 822 | 362 | ||
| @@ -824,8 +364,8 @@ struct kfd_dev *kgd2kfd_probe(struct kgd_dev *kgd, bool vf) · 1 unmodified line | |||
| 824 | if (!kfd) | 364 | if (!kfd) |
| 825 | return NULL; | 365 | return NULL; |
| 826 | 366 | ||
| 827 | kfd->kgd = kgd; | 367 | kfd->adev = adev; |
| 828 | kfd->device_info = device_info; | 368 | kfd_device_info_init(kfd, vf, gfx_target_version); |
| 829 | kfd->pdev = pdev; | 369 | kfd->pdev = pdev; |
| 830 | kfd->init_complete = false; | 370 | kfd->init_complete = false; |
| 831 | kfd->kfd2kgd = f2g; | 371 | kfd->kfd2kgd = f2g; |
| @@ -844,24 +384,24 @@ struct kfd_dev *kgd2kfd_probe(struct kgd_dev *kgd, bool vf) · 12 unmodified lines | |||
| 844 | 384 | ||
| 845 | static void kfd_cwsr_init(struct kfd_dev *kfd) | 385 | static void kfd_cwsr_init(struct kfd_dev *kfd) |
| 846 | { | 386 | { |
| 847 | if (cwsr_enable && kfd->device_info->supports_cwsr) { | 387 | if (cwsr_enable && kfd->device_info.supports_cwsr) { |
| 848 | if (kfd->device_info->asic_family < CHIP_VEGA10) { | 388 | if (KFD_GC_VERSION(kfd) < IP_VERSION(9, 0, 1)) { |
| 849 | BUILD_BUG_ON(sizeof(cwsr_trap_gfx8_hex) > PAGE_SIZE); | 389 | BUILD_BUG_ON(sizeof(cwsr_trap_gfx8_hex) > PAGE_SIZE); |
| 850 | kfd->cwsr_isa = cwsr_trap_gfx8_hex; | 390 | kfd->cwsr_isa = cwsr_trap_gfx8_hex; |
| 851 | kfd->cwsr_isa_size = sizeof(cwsr_trap_gfx8_hex); | 391 | kfd->cwsr_isa_size = sizeof(cwsr_trap_gfx8_hex); |
| 852 | } else if (kfd->device_info->asic_family == CHIP_ARCTURUS) { | 392 | } else if (KFD_GC_VERSION(kfd) == IP_VERSION(9, 4, 1)) { |
| 853 | BUILD_BUG_ON(sizeof(cwsr_trap_arcturus_hex) > PAGE_SIZE); | 393 | BUILD_BUG_ON(sizeof(cwsr_trap_arcturus_hex) > PAGE_SIZE); |
| 854 | kfd->cwsr_isa = cwsr_trap_arcturus_hex; | 394 | kfd->cwsr_isa = cwsr_trap_arcturus_hex; |
| 855 | kfd->cwsr_isa_size = sizeof(cwsr_trap_arcturus_hex); | 395 | kfd->cwsr_isa_size = sizeof(cwsr_trap_arcturus_hex); |
| 856 | } else if (kfd->device_info->asic_family == CHIP_ALDEBARAN) { | 396 | } else if (KFD_GC_VERSION(kfd) == IP_VERSION(9, 4, 2)) { |
| 857 | BUILD_BUG_ON(sizeof(cwsr_trap_aldebaran_hex) > PAGE_SIZE); | 397 | BUILD_BUG_ON(sizeof(cwsr_trap_aldebaran_hex) > PAGE_SIZE); |
| 858 | kfd->cwsr_isa = cwsr_trap_aldebaran_hex; | 398 | kfd->cwsr_isa = cwsr_trap_aldebaran_hex; |
| 859 | kfd->cwsr_isa_size = sizeof(cwsr_trap_aldebaran_hex); | 399 | kfd->cwsr_isa_size = sizeof(cwsr_trap_aldebaran_hex); |
| 860 | } else if (kfd->device_info->asic_family < CHIP_NAVI10) { | 400 | } else if (KFD_GC_VERSION(kfd) < IP_VERSION(10, 1, 1)) { |
| 861 | BUILD_BUG_ON(sizeof(cwsr_trap_gfx9_hex) > PAGE_SIZE); | 401 | BUILD_BUG_ON(sizeof(cwsr_trap_gfx9_hex) > PAGE_SIZE); |
| 862 | kfd->cwsr_isa = cwsr_trap_gfx9_hex; | 402 | kfd->cwsr_isa = cwsr_trap_gfx9_hex; |
| 863 | kfd->cwsr_isa_size = sizeof(cwsr_trap_gfx9_hex); | 403 | kfd->cwsr_isa_size = sizeof(cwsr_trap_gfx9_hex); |
| 864 | } else if (kfd->device_info->asic_family < CHIP_SIENNA_CICHLID) { | 404 | } else if (KFD_GC_VERSION(kfd) < IP_VERSION(10, 3, 0)) { |
| 865 | BUILD_BUG_ON(sizeof(cwsr_trap_nv1x_hex) > PAGE_SIZE); | 405 | BUILD_BUG_ON(sizeof(cwsr_trap_nv1x_hex) > PAGE_SIZE); |
| 866 | kfd->cwsr_isa = cwsr_trap_nv1x_hex; | 406 | kfd->cwsr_isa = cwsr_trap_nv1x_hex; |
| 867 | kfd->cwsr_isa_size = sizeof(cwsr_trap_nv1x_hex); | 407 | kfd->cwsr_isa_size = sizeof(cwsr_trap_nv1x_hex); |
| @@ -882,18 +422,17 @@ static int kfd_gws_init(struct kfd_dev *kfd) · 14 unmodified lines | |||
| 882 | if (kfd->dqm->sched_policy == KFD_SCHED_POLICY_NO_HWS) | 422 | if (kfd->dqm->sched_policy == KFD_SCHED_POLICY_NO_HWS) |
| 883 | return 0; | 423 | return 0; |
| 884 | 424 | ||
| 885 | if (hws_gws_support | 425 | if (hws_gws_support || (KFD_IS_SOC15(kfd) && |
| 886 | || (kfd->device_info->asic_family == CHIP_VEGA10 | 426 | ((KFD_GC_VERSION(kfd) == IP_VERSION(9, 0, 1) |
| 887 | && kfd->mec2_fw_version >= 0x81b3) | 427 | && kfd->mec2_fw_version >= 0x81b3) || |
| 888 | || (kfd->device_info->asic_family >= CHIP_VEGA12 | 428 | (KFD_GC_VERSION(kfd) <= IP_VERSION(9, 4, 0) |
| 889 | && kfd->device_info->asic_family <= CHIP_RAVEN | 429 | && kfd->mec2_fw_version >= 0x1b3) || |
| 890 | && kfd->mec2_fw_version >= 0x1b3) | 430 | (KFD_GC_VERSION(kfd) == IP_VERSION(9, 4, 1) |
| 891 | || (kfd->device_info->asic_family == CHIP_ARCTURUS | 431 | && kfd->mec2_fw_version >= 0x30) || |
| 892 | && kfd->mec2_fw_version >= 0x30) | 432 | (KFD_GC_VERSION(kfd) == IP_VERSION(9, 4, 2) |
| 893 | || (kfd->device_info->asic_family == CHIP_ALDEBARAN | 433 | && kfd->mec2_fw_version >= 0x28)))) |
| 894 | && kfd->mec2_fw_version >= 0x28)) | 434 | ret = amdgpu_amdkfd_alloc_gws(kfd->adev, |
| 895 | ret = amdgpu_amdkfd_alloc_gws(kfd->kgd, | 435 | kfd->adev->gds.gws_size, &kfd->gws); |
| 896 | amdgpu_amdkfd_get_num_gws(kfd->kgd), &kfd->gws); | ||
| 897 | 436 | ||
| 898 | return ret; | 437 | return ret; |
| 899 | } | 438 | } |
| @@ -910,11 +449,11 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, · 10 unmodified lines | |||
| 910 | unsigned int size, map_process_packet_size; | 449 | unsigned int size, map_process_packet_size; |
| 911 | 450 | ||
| 912 | kfd->ddev = ddev; | 451 | kfd->ddev = ddev; |
| 913 | kfd->mec_fw_version = amdgpu_amdkfd_get_fw_version(kfd->kgd, | 452 | kfd->mec_fw_version = amdgpu_amdkfd_get_fw_version(kfd->adev, |
| 914 | KGD_ENGINE_MEC1); | 453 | KGD_ENGINE_MEC1); |
| 915 | kfd->mec2_fw_version = amdgpu_amdkfd_get_fw_version(kfd->kgd, | 454 | kfd->mec2_fw_version = amdgpu_amdkfd_get_fw_version(kfd->adev, |
| 916 | KGD_ENGINE_MEC2); | 455 | KGD_ENGINE_MEC2); |
| 917 | kfd->sdma_fw_version = amdgpu_amdkfd_get_fw_version(kfd->kgd, | 456 | kfd->sdma_fw_version = amdgpu_amdkfd_get_fw_version(kfd->adev, |
| 918 | KGD_ENGINE_SDMA1); | 457 | KGD_ENGINE_SDMA1); |
| 919 | kfd->shared_resources = *gpu_resources; | 458 | kfd->shared_resources = *gpu_resources; |
| 920 | 459 | ||
| @@ -927,16 +466,16 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, · 6 unmodified lines | |||
| 927 | * 32 and 64-bit requests are possible and must be | 466 | * 32 and 64-bit requests are possible and must be |
| 928 | * supported. | 467 | * supported. |
| 929 | */ | 468 | */ |
| 930 | kfd->pci_atomic_requested = amdgpu_amdkfd_have_atomics_support(kfd->kgd); | 469 | kfd->pci_atomic_requested = amdgpu_amdkfd_have_atomics_support(kfd->adev); |
| 931 | if (!kfd->pci_atomic_requested && | 470 | if (!kfd->pci_atomic_requested && |
| 932 | kfd->device_info->needs_pci_atomics && | 471 | kfd->device_info.needs_pci_atomics && |
| 933 | (!kfd->device_info->no_atomic_fw_version || | 472 | (!kfd->device_info.no_atomic_fw_version || |
| 934 | kfd->mec_fw_version < kfd->device_info->no_atomic_fw_version)) { | 473 | kfd->mec_fw_version < kfd->device_info.no_atomic_fw_version)) { |
| 935 | dev_info(kfd_device, | 474 | dev_info(kfd_device, |
| 936 | "skipped device %x:%x, PCI rejects atomics %d<%d\n", | 475 | "skipped device %x:%x, PCI rejects atomics %d<%d\n", |
| 937 | kfd->pdev->vendor, kfd->pdev->device, | 476 | kfd->pdev->vendor, kfd->pdev->device, |
| 938 | kfd->mec_fw_version, | 477 | kfd->mec_fw_version, |
| 939 | kfd->device_info->no_atomic_fw_version); | 478 | kfd->device_info.no_atomic_fw_version); |
| 940 | return false; | 479 | return false; |
| 941 | } | 480 | } |
| 942 | 481 | ||
| @@ -953,16 +492,15 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, · 10 unmodified lines | |||
| 953 | 492 | ||
| 954 | /* calculate max size of mqds needed for queues */ | 493 | /* calculate max size of mqds needed for queues */ |
| 955 | size = max_num_of_queues_per_device * | 494 | size = max_num_of_queues_per_device * |
| 956 | kfd->device_info->mqd_size_aligned; | 495 | kfd->device_info.mqd_size_aligned; |
| 957 | 496 | ||
| 958 | /* | 497 | /* |
| 959 | * calculate max size of runlist packet. | 498 | * calculate max size of runlist packet. |
| 960 | * There can be only 2 packets at once | 499 | * There can be only 2 packets at once |
| 961 | */ | 500 | */ |
| 962 | map_process_packet_size = | 501 | map_process_packet_size = KFD_GC_VERSION(kfd) == IP_VERSION(9, 4, 2) ? |
| 963 | kfd->device_info->asic_family == CHIP_ALDEBARAN ? | ||
| 964 | sizeof(struct pm4_mes_map_process_aldebaran) : | 502 | sizeof(struct pm4_mes_map_process_aldebaran) : |
| 965 | sizeof(struct pm4_mes_map_process); | 503 | sizeof(struct pm4_mes_map_process); |
| 966 | size += (KFD_MAX_NUM_OF_PROCESSES * map_process_packet_size + | 504 | size += (KFD_MAX_NUM_OF_PROCESSES * map_process_packet_size + |
| 967 | max_num_of_queues_per_device * sizeof(struct pm4_mes_map_queues) | 505 | max_num_of_queues_per_device * sizeof(struct pm4_mes_map_queues) |
| 968 | + sizeof(struct pm4_mes_runlist)) * 2; | 506 | + sizeof(struct pm4_mes_runlist)) * 2; |
| @@ -974,7 +512,7 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, · 5 unmodified lines | |||
| 974 | size += 512 * 1024; | 512 | size += 512 * 1024; |
| 975 | 513 | ||
| 976 | if (amdgpu_amdkfd_alloc_gtt_mem( | 514 | if (amdgpu_amdkfd_alloc_gtt_mem( |
| 977 | kfd->kgd, size, &kfd->gtt_mem, | 515 | kfd->adev, size, &kfd->gtt_mem, |
| 978 | &kfd->gtt_start_gpu_addr, &kfd->gtt_start_cpu_ptr, | 516 | &kfd->gtt_start_gpu_addr, &kfd->gtt_start_cpu_ptr, |
| 979 | false)) { | 517 | false)) { |
| 980 | dev_err(kfd_device, "Could not allocate %d bytes\n", size); | 518 | dev_err(kfd_device, "Could not allocate %d bytes\n", size); |
| @@ -995,9 +533,9 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, · 14 unmodified lines | |||
| 995 | goto kfd_doorbell_error; | 533 | goto kfd_doorbell_error; |
| 996 | } | 534 | } |
| 997 | 535 | ||
| 998 | kfd->hive_id = amdgpu_amdkfd_get_hive_id(kfd->kgd); | 536 | kfd->hive_id = kfd->adev->gmc.xgmi.hive_id; |
| 999 | 537 | ||
| 1000 | kfd->noretry = amdgpu_amdkfd_get_noretry(kfd->kgd); | 538 | kfd->noretry = kfd->adev->gmc.noretry; |
| 1001 | 539 | ||
| 1002 | if (kfd_interrupt_init(kfd)) { | 540 | if (kfd_interrupt_init(kfd)) { |
| 1003 | dev_err(kfd_device, "Error initializing interrupts\n"); | 541 | dev_err(kfd_device, "Error initializing interrupts\n"); |
| @@ -1015,7 +553,7 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, · 11 unmodified lines | |||
| 1015 | */ | 553 | */ |
| 1016 | if (kfd_gws_init(kfd)) { | 554 | if (kfd_gws_init(kfd)) { |
| 1017 | dev_err(kfd_device, "Could not allocate %d gws\n", | 555 | dev_err(kfd_device, "Could not allocate %d gws\n", |
| 1018 | amdgpu_amdkfd_get_num_gws(kfd->kgd)); | 556 | kfd->adev->gds.gws_size); |
| 1019 | goto gws_error; | 557 | goto gws_error; |
| 1020 | } | 558 | } |
| 1021 | 559 | ||
| @@ -1030,7 +568,7 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, · 8 unmodified lines | |||
| 1030 | 568 | ||
| 1031 | kfd_cwsr_init(kfd); | 569 | kfd_cwsr_init(kfd); |
| 1032 | 570 | ||
| 1033 | svm_migrate_init((struct amdgpu_device *)kfd->kgd); | 571 | svm_migrate_init(kfd->adev); |
| 1034 | 572 | ||
| 1035 | if(kgd2kfd_resume_iommu(kfd)) | 573 | if(kgd2kfd_resume_iommu(kfd)) |
| 1036 | goto device_iommu_error; | 574 | goto device_iommu_error; |
| @@ -1068,10 +606,10 @@ kfd_interrupt_error: · 31 unmodified lines | |||
| 1068 | kfd_doorbell_error: | 606 | kfd_doorbell_error: |
| 1069 | kfd_gtt_sa_fini(kfd); | 607 | kfd_gtt_sa_fini(kfd); |
| 1070 | kfd_gtt_sa_init_error: | 608 | kfd_gtt_sa_init_error: |
| 1071 | amdgpu_amdkfd_free_gtt_mem(kfd->kgd, kfd->gtt_mem); | 609 | amdgpu_amdkfd_free_gtt_mem(kfd->adev, kfd->gtt_mem); |
| 1072 | alloc_gtt_mem_failure: | 610 | alloc_gtt_mem_failure: |
| 1073 | if (kfd->gws) | 611 | if (kfd->gws) |
| 1074 | amdgpu_amdkfd_free_gws(kfd->kgd, kfd->gws); | 612 | amdgpu_amdkfd_free_gws(kfd->adev, kfd->gws); |
| 1075 | dev_err(kfd_device, | 613 | dev_err(kfd_device, |
| 1076 | "device %x:%x NOT added due to errors\n", | 614 | "device %x:%x NOT added due to errors\n", |
| 1077 | kfd->pdev->vendor, kfd->pdev->device); | 615 | kfd->pdev->vendor, kfd->pdev->device); |
| @@ -1088,9 +626,9 @@ void kgd2kfd_device_exit(struct kfd_dev *kfd) · 10 unmodified lines | |||
| 1088 | kfd_doorbell_fini(kfd); | 626 | kfd_doorbell_fini(kfd); |
| 1089 | ida_destroy(&kfd->doorbell_ida); | 627 | ida_destroy(&kfd->doorbell_ida); |
| 1090 | kfd_gtt_sa_fini(kfd); | 628 | kfd_gtt_sa_fini(kfd); |
| 1091 | amdgpu_amdkfd_free_gtt_mem(kfd->kgd, kfd->gtt_mem); | 629 | amdgpu_amdkfd_free_gtt_mem(kfd->adev, kfd->gtt_mem); |
| 1092 | if (kfd->gws) | 630 | if (kfd->gws) |
| 1093 | amdgpu_amdkfd_free_gws(kfd->kgd, kfd->gws); | 631 | amdgpu_amdkfd_free_gws(kfd->adev, kfd->gws); |
| 1094 | } | 632 | } |
| 1095 | 633 | ||
| 1096 | kfree(kfd); | 634 | kfree(kfd); |
| @@ -1229,7 +767,7 @@ void kgd2kfd_interrupt(struct kfd_dev *kfd, const void *ih_ring_entry) · 132 unmodified lines | |||
| 1229 | if (!kfd->init_complete) | 767 | if (!kfd->init_complete) |
| 1230 | return; | 768 | return; |
| 1231 | 769 | ||
| 1232 | if (kfd->device_info->ih_ring_entry_size > sizeof(patched_ihre)) { | 770 | if (kfd->device_info.ih_ring_entry_size > sizeof(patched_ihre)) { |
| 1233 | dev_err_once(kfd_device, "Ring entry too small\n"); | 771 | dev_err_once(kfd_device, "Ring entry too small\n"); |
| 1234 | return; | 772 | return; |
| 1235 | } | 773 | } |
| @@ -1526,7 +1064,7 @@ void kgd2kfd_set_sram_ecc_flag(struct kfd_dev *kfd) · 290 unmodified lines | |||
| 1526 | void kfd_inc_compute_active(struct kfd_dev *kfd) | 1064 | void kfd_inc_compute_active(struct kfd_dev *kfd) |
| 1527 | { | 1065 | { |
| 1528 | if (atomic_inc_return(&kfd->compute_profile) == 1) | 1066 | if (atomic_inc_return(&kfd->compute_profile) == 1) |
| 1529 | amdgpu_amdkfd_set_compute_idle(kfd->kgd, false); | 1067 | amdgpu_amdkfd_set_compute_idle(kfd->adev, false); |
| 1530 | } | 1068 | } |
| 1531 | 1069 | ||
| 1532 | void kfd_dec_compute_active(struct kfd_dev *kfd) | 1070 | void kfd_dec_compute_active(struct kfd_dev *kfd) |
| @@ -1534,7 +1072,7 @@ void kfd_dec_compute_active(struct kfd_dev *kfd) · 1 unmodified line | |||
| 1534 | int count = atomic_dec_return(&kfd->compute_profile); | 1072 | int count = atomic_dec_return(&kfd->compute_profile); |
| 1535 | 1073 | ||
| 1536 | if (count == 0) | 1074 | if (count == 0) |
| 1537 | amdgpu_amdkfd_set_compute_idle(kfd->kgd, true); | 1075 | amdgpu_amdkfd_set_compute_idle(kfd->adev, true); |
| 1538 | WARN_ONCE(count < 0, "Compute profile ref. count error"); | 1076 | WARN_ONCE(count < 0, "Compute profile ref. count error"); |
| 1539 | } | 1077 | } |
| 1540 | 1078 | ||
| @@ -1544,6 +1082,26 @@ void kgd2kfd_smi_event_throttle(struct kfd_dev *kfd, uint64_t throttle_bitmask) · 3 unmodified lines | |||
| 1544 | kfd_smi_event_update_thermal_throttling(kfd, throttle_bitmask); | 1082 | kfd_smi_event_update_thermal_throttling(kfd, throttle_bitmask); |
| 1545 | } | 1083 | } |
| 1546 | 1084 | ||
| 1085 | /* kfd_get_num_sdma_engines returns the number of PCIe optimized SDMA and | ||
| 1086 | * kfd_get_num_xgmi_sdma_engines returns the number of XGMI SDMA. | ||
| 1087 | * When the device has more than two engines, we reserve two for PCIe to enable | ||
| 1088 | * full-duplex and the rest are used as XGMI. | ||
| 1089 | */ | ||
| 1090 | unsigned int kfd_get_num_sdma_engines(struct kfd_dev *kdev) | ||
| 1091 | { | ||
| 1092 | /* If XGMI is not supported, all SDMA engines are PCIe */ | ||
| 1093 | if (!kdev->adev->gmc.xgmi.supported) | ||
| 1094 | return kdev->adev->sdma.num_instances; | ||
| 1095 | |||
| 1096 | return min(kdev->adev->sdma.num_instances, 2); | ||
| 1097 | } | ||
| 1098 | |||
| 1099 | unsigned int kfd_get_num_xgmi_sdma_engines(struct kfd_dev *kdev) | ||
| 1100 | { | ||
| 1101 | /* After reserved for PCIe, the rest of engines are XGMI */ | ||
| 1102 | return kdev->adev->sdma.num_instances - kfd_get_num_sdma_engines(kdev); | ||
| 1103 | } | ||
| 1104 | |||
| 1547 | #if defined(CONFIG_DEBUG_FS) | 1105 | #if defined(CONFIG_DEBUG_FS) |
| 1548 | 1106 | ||
| 1549 | /* This function will send a package to HIQ to hang the HWS | 1107 | /* This function will send a package to HIQ to hang the HWS |
| @@ -47,7 +47,7 @@ static int execute_queues_cpsch(struct device_queue_manager *dqm, · 46 unmodified lines | |||
| 47 | uint32_t filter_param); | 47 | uint32_t filter_param); |
| 48 | static int unmap_queues_cpsch(struct device_queue_manager *dqm, | 48 | static int unmap_queues_cpsch(struct device_queue_manager *dqm, |
| 49 | enum kfd_unmap_queues_filter filter, | 49 | enum kfd_unmap_queues_filter filter, |
| 50 | uint32_t filter_param); | 50 | uint32_t filter_param, bool reset); |
| 51 | 51 | ||
| 52 | static int map_queues_cpsch(struct device_queue_manager *dqm); | 52 | static int map_queues_cpsch(struct device_queue_manager *dqm); |
| 53 | 53 | ||
| @@ -99,38 +99,29 @@ unsigned int get_pipes_per_mec(struct device_queue_manager *dqm) · 45 unmodified lines | |||
| 99 | return dqm->dev->shared_resources.num_pipe_per_mec; | 99 | return dqm->dev->shared_resources.num_pipe_per_mec; |
| 100 | } | 100 | } |
| 101 | 101 | ||
| 102 | static unsigned int get_num_sdma_engines(struct device_queue_manager *dqm) | ||
| 103 | { | ||
| 104 | return dqm->dev->device_info->num_sdma_engines; | ||
| 105 | } | ||
| 106 | |||
| 107 | static unsigned int get_num_xgmi_sdma_engines(struct device_queue_manager *dqm) | ||
| 108 | { | ||
| 109 | return dqm->dev->device_info->num_xgmi_sdma_engines; | ||
| 110 | } | ||
| 111 | |||
| 112 | static unsigned int get_num_all_sdma_engines(struct device_queue_manager *dqm) | 102 | static unsigned int get_num_all_sdma_engines(struct device_queue_manager *dqm) |
| 113 | { | 103 | { |
| 114 | return get_num_sdma_engines(dqm) + get_num_xgmi_sdma_engines(dqm); | 104 | return kfd_get_num_sdma_engines(dqm->dev) + |
| 105 | kfd_get_num_xgmi_sdma_engines(dqm->dev); | ||
| 115 | } | 106 | } |
| 116 | 107 | ||
| 117 | unsigned int get_num_sdma_queues(struct device_queue_manager *dqm) | 108 | unsigned int get_num_sdma_queues(struct device_queue_manager *dqm) |
| 118 | { | 109 | { |
| 119 | return dqm->dev->device_info->num_sdma_engines | 110 | return kfd_get_num_sdma_engines(dqm->dev) * |
| 120 | * dqm->dev->device_info->num_sdma_queues_per_engine; | 111 | dqm->dev->device_info.num_sdma_queues_per_engine; |
| 121 | } | 112 | } |
| 122 | 113 | ||
| 123 | unsigned int get_num_xgmi_sdma_queues(struct device_queue_manager *dqm) | 114 | unsigned int get_num_xgmi_sdma_queues(struct device_queue_manager *dqm) |
| 124 | { | 115 | { |
| 125 | return dqm->dev->device_info->num_xgmi_sdma_engines | 116 | return kfd_get_num_xgmi_sdma_engines(dqm->dev) * |
| 126 | * dqm->dev->device_info->num_sdma_queues_per_engine; | 117 | dqm->dev->device_info.num_sdma_queues_per_engine; |
| 127 | } | 118 | } |
| 128 | 119 | ||
| 129 | void program_sh_mem_settings(struct device_queue_manager *dqm, | 120 | void program_sh_mem_settings(struct device_queue_manager *dqm, |
| 130 | struct qcm_process_device *qpd) | 121 | struct qcm_process_device *qpd) |
| 131 | { | 122 | { |
| 132 | return dqm->dev->kfd2kgd->program_sh_mem_settings( | 123 | return dqm->dev->kfd2kgd->program_sh_mem_settings( |
| 133 | dqm->dev->kgd, qpd->vmid, | 124 | dqm->dev->adev, qpd->vmid, |
| 134 | qpd->sh_mem_config, | 125 | qpd->sh_mem_config, |
| 135 | qpd->sh_mem_ape1_base, | 126 | qpd->sh_mem_ape1_base, |
| 136 | qpd->sh_mem_ape1_limit, | 127 | qpd->sh_mem_ape1_limit, |
| @@ -157,7 +148,7 @@ static int allocate_doorbell(struct qcm_process_device *qpd, struct queue *q) · 20 unmodified lines | |||
| 157 | { | 148 | { |
| 158 | struct kfd_dev *dev = qpd->dqm->dev; | 149 | struct kfd_dev *dev = qpd->dqm->dev; |
| 159 | 150 | ||
| 160 | if (!KFD_IS_SOC15(dev->device_info->asic_family)) { | 151 | if (!KFD_IS_SOC15(dev)) { |
| 161 | /* On pre-SOC15 chips we need to use the queue ID to | 152 | /* On pre-SOC15 chips we need to use the queue ID to |
| 162 | * preserve the user mode ABI. | 153 | * preserve the user mode ABI. |
| 163 | */ | 154 | */ |
| @@ -202,7 +193,7 @@ static void deallocate_doorbell(struct qcm_process_device *qpd, · 38 unmodified lines | |||
| 202 | unsigned int old; | 193 | unsigned int old; |
| 203 | struct kfd_dev *dev = qpd->dqm->dev; | 194 | struct kfd_dev *dev = qpd->dqm->dev; |
| 204 | 195 | ||
| 205 | if (!KFD_IS_SOC15(dev->device_info->asic_family) || | 196 | if (!KFD_IS_SOC15(dev) || |
| 206 | q->properties.type == KFD_QUEUE_TYPE_SDMA || | 197 | q->properties.type == KFD_QUEUE_TYPE_SDMA || |
| 207 | q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI) | 198 | q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI) |
| 208 | return; | 199 | return; |
| @@ -216,7 +207,7 @@ static void program_trap_handler_settings(struct device_queue_manager *dqm, · 7 unmodified lines | |||
| 216 | { | 207 | { |
| 217 | if (dqm->dev->kfd2kgd->program_trap_handler_settings) | 208 | if (dqm->dev->kfd2kgd->program_trap_handler_settings) |
| 218 | dqm->dev->kfd2kgd->program_trap_handler_settings( | 209 | dqm->dev->kfd2kgd->program_trap_handler_settings( |
| 219 | dqm->dev->kgd, qpd->vmid, | 210 | dqm->dev->adev, qpd->vmid, |
| 220 | qpd->tba_addr, qpd->tma_addr); | 211 | qpd->tba_addr, qpd->tma_addr); |
| 221 | } | 212 | } |
| 222 | 213 | ||
| @@ -250,21 +241,20 @@ static int allocate_vmid(struct device_queue_manager *dqm, · 27 unmodified lines | |||
| 250 | 241 | ||
| 251 | program_sh_mem_settings(dqm, qpd); | 242 | program_sh_mem_settings(dqm, qpd); |
| 252 | 243 | ||
| 253 | if (dqm->dev->device_info->asic_family >= CHIP_VEGA10 && | 244 | if (KFD_IS_SOC15(dqm->dev) && dqm->dev->cwsr_enabled) |
| 254 | dqm->dev->cwsr_enabled) | ||
| 255 | program_trap_handler_settings(dqm, qpd); | 245 | program_trap_handler_settings(dqm, qpd); |
| 256 | 246 | ||
| 257 | /* qpd->page_table_base is set earlier when register_process() | 247 | /* qpd->page_table_base is set earlier when register_process() |
| 258 | * is called, i.e. when the first queue is created. | 248 | * is called, i.e. when the first queue is created. |
| 259 | */ | 249 | */ |
| 260 | dqm->dev->kfd2kgd->set_vm_context_page_table_base(dqm->dev->kgd, | 250 | dqm->dev->kfd2kgd->set_vm_context_page_table_base(dqm->dev->adev, |
| 261 | qpd->vmid, | 251 | qpd->vmid, |
| 262 | qpd->page_table_base); | 252 | qpd->page_table_base); |
| 263 | /* invalidate the VM context after pasid and vmid mapping is set up */ | 253 | /* invalidate the VM context after pasid and vmid mapping is set up */ |
| 264 | kfd_flush_tlb(qpd_to_pdd(qpd), TLB_FLUSH_LEGACY); | 254 | kfd_flush_tlb(qpd_to_pdd(qpd), TLB_FLUSH_LEGACY); |
| 265 | 255 | ||
| 266 | if (dqm->dev->kfd2kgd->set_scratch_backing_va) | 256 | if (dqm->dev->kfd2kgd->set_scratch_backing_va) |
| 267 | dqm->dev->kfd2kgd->set_scratch_backing_va(dqm->dev->kgd, | 257 | dqm->dev->kfd2kgd->set_scratch_backing_va(dqm->dev->adev, |
| 268 | qpd->sh_hidden_private_base, qpd->vmid); | 258 | qpd->sh_hidden_private_base, qpd->vmid); |
| 269 | 259 | ||
| 270 | return 0; | 260 | return 0; |
| @@ -283,7 +273,7 @@ static int flush_texture_cache_nocpsch(struct kfd_dev *kdev, · 12 unmodified lines | |||
| 283 | if (ret) | 273 | if (ret) |
| 284 | return ret; | 274 | return ret; |
| 285 | 275 | ||
| 286 | return amdgpu_amdkfd_submit_ib(kdev->kgd, KGD_ENGINE_MEC1, qpd->vmid, | 276 | return amdgpu_amdkfd_submit_ib(kdev->adev, KGD_ENGINE_MEC1, qpd->vmid, |
| 287 | qpd->ib_base, (uint32_t *)qpd->ib_kaddr, | 277 | qpd->ib_base, (uint32_t *)qpd->ib_kaddr, |
| 288 | pmf->release_mem_size / sizeof(uint32_t)); | 278 | pmf->release_mem_size / sizeof(uint32_t)); |
| 289 | } | 279 | } |
| @@ -293,7 +283,7 @@ static void deallocate_vmid(struct device_queue_manager *dqm, · 3 unmodified lines | |||
| 293 | struct queue *q) | 283 | struct queue *q) |
| 294 | { | 284 | { |
| 295 | /* On GFX v7, CP doesn't flush TC at dequeue */ | 285 | /* On GFX v7, CP doesn't flush TC at dequeue */ |
| 296 | if (q->device->device_info->asic_family == CHIP_HAWAII) | 286 | if (q->device->adev->asic_type == CHIP_HAWAII) |
| 297 | if (flush_texture_cache_nocpsch(q->device, qpd)) | 287 | if (flush_texture_cache_nocpsch(q->device, qpd)) |
| 298 | pr_err("Failed to flush TC\n"); | 288 | pr_err("Failed to flush TC\n"); |
| 299 | 289 | ||
| @@ -580,7 +570,7 @@ static int update_queue(struct device_queue_manager *dqm, struct queue *q, · 280 unmodified lines | |||
| 580 | /* Make sure the queue is unmapped before updating the MQD */ | 570 | /* Make sure the queue is unmapped before updating the MQD */ |
| 581 | if (dqm->sched_policy != KFD_SCHED_POLICY_NO_HWS) { | 571 | if (dqm->sched_policy != KFD_SCHED_POLICY_NO_HWS) { |
| 582 | retval = unmap_queues_cpsch(dqm, | 572 | retval = unmap_queues_cpsch(dqm, |
| 583 | KFD_UNMAP_QUEUES_FILTER_DYNAMIC_QUEUES, 0); | 573 | KFD_UNMAP_QUEUES_FILTER_DYNAMIC_QUEUES, 0, false); |
| 584 | if (retval) { | 574 | if (retval) { |
| 585 | pr_err("unmap queue failed\n"); | 575 | pr_err("unmap queue failed\n"); |
| 586 | goto out_unlock; | 576 | goto out_unlock; |
| @@ -776,7 +766,7 @@ static int restore_process_queues_nocpsch(struct device_queue_manager *dqm, · 189 unmodified lines | |||
| 776 | 766 | ||
| 777 | if (!list_empty(&qpd->queues_list)) { | 767 | if (!list_empty(&qpd->queues_list)) { |
| 778 | dqm->dev->kfd2kgd->set_vm_context_page_table_base( | 768 | dqm->dev->kfd2kgd->set_vm_context_page_table_base( |
| 779 | dqm->dev->kgd, | 769 | dqm->dev->adev, |
| 780 | qpd->vmid, | 770 | qpd->vmid, |
| 781 | qpd->page_table_base); | 771 | qpd->page_table_base); |
| 782 | kfd_flush_tlb(pdd, TLB_FLUSH_LEGACY); | 772 | kfd_flush_tlb(pdd, TLB_FLUSH_LEGACY); |
| @@ -954,7 +944,7 @@ set_pasid_vmid_mapping(struct device_queue_manager *dqm, u32 pasid, · 171 unmodified lines | |||
| 954 | unsigned int vmid) | 944 | unsigned int vmid) |
| 955 | { | 945 | { |
| 956 | return dqm->dev->kfd2kgd->set_pasid_vmid_mapping( | 946 | return dqm->dev->kfd2kgd->set_pasid_vmid_mapping( |
| 957 | dqm->dev->kgd, pasid, vmid); | 947 | dqm->dev->adev, pasid, vmid); |
| 958 | } | 948 | } |
| 959 | 949 | ||
| 960 | static void init_interrupts(struct device_queue_manager *dqm) | 950 | static void init_interrupts(struct device_queue_manager *dqm) |
| @@ -963,7 +953,7 @@ static void init_interrupts(struct device_queue_manager *dqm) · 2 unmodified lines | |||
| 963 | 953 | ||
| 964 | for (i = 0 ; i < get_pipes_per_mec(dqm) ; i++) | 954 | for (i = 0 ; i < get_pipes_per_mec(dqm) ; i++) |
| 965 | if (is_pipe_enabled(dqm, 0, i)) | 955 | if (is_pipe_enabled(dqm, 0, i)) |
| 966 | dqm->dev->kfd2kgd->init_interrupts(dqm->dev->kgd, i); | 956 | dqm->dev->kfd2kgd->init_interrupts(dqm->dev->adev, i); |
| 967 | } | 957 | } |
| 968 | 958 | ||
| 969 | static int initialize_nocpsch(struct device_queue_manager *dqm) | 959 | static int initialize_nocpsch(struct device_queue_manager *dqm) |
| @@ -1017,7 +1007,7 @@ static int start_nocpsch(struct device_queue_manager *dqm) · 47 unmodified lines | |||
| 1017 | pr_info("SW scheduler is used"); | 1007 | pr_info("SW scheduler is used"); |
| 1018 | init_interrupts(dqm); | 1008 | init_interrupts(dqm); |
| 1019 | 1009 | ||
| 1020 | if (dqm->dev->device_info->asic_family == CHIP_HAWAII) | 1010 | if (dqm->dev->adev->asic_type == CHIP_HAWAII) |
| 1021 | return pm_init(&dqm->packet_mgr, dqm); | 1011 | return pm_init(&dqm->packet_mgr, dqm); |
| 1022 | dqm->sched_running = true; | 1012 | dqm->sched_running = true; |
| 1023 | 1013 | ||
| @@ -1026,7 +1016,7 @@ static int start_nocpsch(struct device_queue_manager *dqm) · 2 unmodified lines | |||
| 1026 | 1016 | ||
| 1027 | static int stop_nocpsch(struct device_queue_manager *dqm) | 1017 | static int stop_nocpsch(struct device_queue_manager *dqm) |
| 1028 | { | 1018 | { |
| 1029 | if (dqm->dev->device_info->asic_family == CHIP_HAWAII) | 1019 | if (dqm->dev->adev->asic_type == CHIP_HAWAII) |
| 1030 | pm_uninit(&dqm->packet_mgr, false); | 1020 | pm_uninit(&dqm->packet_mgr, false); |
| 1031 | dqm->sched_running = false; | 1021 | dqm->sched_running = false; |
| 1032 | 1022 | ||
| @@ -1055,9 +1045,9 @@ static int allocate_sdma_queue(struct device_queue_manager *dqm, · 22 unmodified lines | |||
| 1055 | dqm->sdma_bitmap &= ~(1ULL << bit); | 1045 | dqm->sdma_bitmap &= ~(1ULL << bit); |
| 1056 | q->sdma_id = bit; | 1046 | q->sdma_id = bit; |
| 1057 | q->properties.sdma_engine_id = q->sdma_id % | 1047 | q->properties.sdma_engine_id = q->sdma_id % |
| 1058 | get_num_sdma_engines(dqm); | 1048 | kfd_get_num_sdma_engines(dqm->dev); |
| 1059 | q->properties.sdma_queue_id = q->sdma_id / | 1049 | q->properties.sdma_queue_id = q->sdma_id / |
| 1060 | get_num_sdma_engines(dqm); | 1050 | kfd_get_num_sdma_engines(dqm->dev); |
| 1061 | } else if (q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI) { | 1051 | } else if (q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI) { |
| 1062 | if (dqm->xgmi_sdma_bitmap == 0) { | 1052 | if (dqm->xgmi_sdma_bitmap == 0) { |
| 1063 | pr_err("No more XGMI SDMA queue to allocate\n"); | 1053 | pr_err("No more XGMI SDMA queue to allocate\n"); |
| @@ -1072,10 +1062,11 @@ static int allocate_sdma_queue(struct device_queue_manager *dqm, · 8 unmodified lines | |||
| 1072 | * assumes the first N engines are always | 1062 | * assumes the first N engines are always |
| 1073 | * PCIe-optimized ones | 1063 | * PCIe-optimized ones |
| 1074 | */ | 1064 | */ |
| 1075 | q->properties.sdma_engine_id = get_num_sdma_engines(dqm) + | 1065 | q->properties.sdma_engine_id = |
| 1076 | q->sdma_id % get_num_xgmi_sdma_engines(dqm); | 1066 | kfd_get_num_sdma_engines(dqm->dev) + |
| 1067 | q->sdma_id % kfd_get_num_xgmi_sdma_engines(dqm->dev); | ||
| 1077 | q->properties.sdma_queue_id = q->sdma_id / | 1068 | q->properties.sdma_queue_id = q->sdma_id / |
| 1078 | get_num_xgmi_sdma_engines(dqm); | 1069 | kfd_get_num_xgmi_sdma_engines(dqm->dev); |
| 1079 | } | 1070 | } |
| 1080 | 1071 | ||
| 1081 | pr_debug("SDMA engine id: %d\n", q->properties.sdma_engine_id); | 1072 | pr_debug("SDMA engine id: %d\n", q->properties.sdma_engine_id); |
| @@ -1132,7 +1123,7 @@ static int set_sched_resources(struct device_queue_manager *dqm) · 50 unmodified lines | |||
| 1132 | 1123 | ||
| 1133 | res.queue_mask |= 1ull | 1124 | res.queue_mask |= 1ull |
| 1134 | << amdgpu_queue_mask_bit_to_set_resource_bit( | 1125 | << amdgpu_queue_mask_bit_to_set_resource_bit( |
| 1135 | (struct amdgpu_device *)dqm->dev->kgd, i); | 1126 | dqm->dev->adev, i); |
| 1136 | } | 1127 | } |
| 1137 | res.gws_mask = ~0ull; | 1128 | res.gws_mask = ~0ull; |
| 1138 | res.oac_mask = res.gds_heap_base = res.gds_heap_size = 0; | 1129 | res.oac_mask = res.gds_heap_base = res.gds_heap_size = 0; |
| @@ -1232,7 +1223,7 @@ static int stop_cpsch(struct device_queue_manager *dqm) · 93 unmodified lines | |||
| 1232 | } | 1223 | } |
| 1233 | 1224 | ||
| 1234 | if (!dqm->is_hws_hang) | 1225 | if (!dqm->is_hws_hang) |
| 1235 | unmap_queues_cpsch(dqm, KFD_UNMAP_QUEUES_FILTER_ALL_QUEUES, 0); | 1226 | unmap_queues_cpsch(dqm, KFD_UNMAP_QUEUES_FILTER_ALL_QUEUES, 0, false); |
| 1236 | hanging = dqm->is_hws_hang || dqm->is_resetting; | 1227 | hanging = dqm->is_hws_hang || dqm->is_resetting; |
| 1237 | dqm->sched_running = false; | 1228 | dqm->sched_running = false; |
| 1238 | 1229 | ||
| @@ -1428,7 +1419,7 @@ static int map_queues_cpsch(struct device_queue_manager *dqm) · 189 unmodified lines | |||
| 1428 | /* dqm->lock mutex has to be locked before calling this function */ | 1419 | /* dqm->lock mutex has to be locked before calling this function */ |
| 1429 | static int unmap_queues_cpsch(struct device_queue_manager *dqm, | 1420 | static int unmap_queues_cpsch(struct device_queue_manager *dqm, |
| 1430 | enum kfd_unmap_queues_filter filter, | 1421 | enum kfd_unmap_queues_filter filter, |
| 1431 | uint32_t filter_param) | 1422 | uint32_t filter_param, bool reset) |
| 1432 | { | 1423 | { |
| 1433 | int retval = 0; | 1424 | int retval = 0; |
| 1434 | struct mqd_manager *mqd_mgr; | 1425 | struct mqd_manager *mqd_mgr; |
| @@ -1441,7 +1432,7 @@ static int unmap_queues_cpsch(struct device_queue_manager *dqm, · 6 unmodified lines | |||
| 1441 | return retval; | 1432 | return retval; |
| 1442 | 1433 | ||
| 1443 | retval = pm_send_unmap_queue(&dqm->packet_mgr, KFD_QUEUE_TYPE_COMPUTE, | 1434 | retval = pm_send_unmap_queue(&dqm->packet_mgr, KFD_QUEUE_TYPE_COMPUTE, |
| 1444 | filter, filter_param, false, 0); | 1435 | filter, filter_param, reset, 0); |
| 1445 | if (retval) | 1436 | if (retval) |
| 1446 | return retval; | 1437 | return retval; |
| 1447 | 1438 | ||
| @@ -1485,6 +1476,21 @@ static int unmap_queues_cpsch(struct device_queue_manager *dqm, · 37 unmodified lines | |||
| 1485 | return retval; | 1476 | return retval; |
| 1486 | } | 1477 | } |
| 1487 | 1478 | ||
| 1479 | /* only for compute queue */ | ||
| 1480 | static int reset_queues_cpsch(struct device_queue_manager *dqm, | ||
| 1481 | uint16_t pasid) | ||
| 1482 | { | ||
| 1483 | int retval; | ||
| 1484 | |||
| 1485 | dqm_lock(dqm); | ||
| 1486 | |||
| 1487 | retval = unmap_queues_cpsch(dqm, KFD_UNMAP_QUEUES_FILTER_BY_PASID, | ||
| 1488 | pasid, true); | ||
| 1489 | |||
| 1490 | dqm_unlock(dqm); | ||
| 1491 | return retval; | ||
| 1492 | } | ||
| 1493 | |||
| 1488 | /* dqm->lock mutex has to be locked before calling this function */ | 1494 | /* dqm->lock mutex has to be locked before calling this function */ |
| 1489 | static int execute_queues_cpsch(struct device_queue_manager *dqm, | 1495 | static int execute_queues_cpsch(struct device_queue_manager *dqm, |
| 1490 | enum kfd_unmap_queues_filter filter, | 1496 | enum kfd_unmap_queues_filter filter, |
| @@ -1494,7 +1500,7 @@ static int execute_queues_cpsch(struct device_queue_manager *dqm, · 3 unmodified lines | |||
| 1494 | 1500 | ||
| 1495 | if (dqm->is_hws_hang) | 1501 | if (dqm->is_hws_hang) |
| 1496 | return -EIO; | 1502 | return -EIO; |
| 1497 | retval = unmap_queues_cpsch(dqm, filter, filter_param); | 1503 | retval = unmap_queues_cpsch(dqm, filter, filter_param, false); |
| 1498 | if (retval) | 1504 | if (retval) |
| 1499 | return retval; | 1505 | return retval; |
| 1500 | 1506 | ||
| @@ -1847,10 +1853,10 @@ static int allocate_hiq_sdma_mqd(struct device_queue_manager *dqm) · 346 unmodified lines | |||
| 1847 | struct kfd_mem_obj *mem_obj = &dqm->hiq_sdma_mqd; | 1853 | struct kfd_mem_obj *mem_obj = &dqm->hiq_sdma_mqd; |
| 1848 | uint32_t size = dqm->mqd_mgrs[KFD_MQD_TYPE_SDMA]->mqd_size * | 1854 | uint32_t size = dqm->mqd_mgrs[KFD_MQD_TYPE_SDMA]->mqd_size * |
| 1849 | get_num_all_sdma_engines(dqm) * | 1855 | get_num_all_sdma_engines(dqm) * |
| 1850 | dev->device_info->num_sdma_queues_per_engine + | 1856 | dev->device_info.num_sdma_queues_per_engine + |
| 1851 | dqm->mqd_mgrs[KFD_MQD_TYPE_HIQ]->mqd_size; | 1857 | dqm->mqd_mgrs[KFD_MQD_TYPE_HIQ]->mqd_size; |
| 1852 | 1858 | ||
| 1853 | retval = amdgpu_amdkfd_alloc_gtt_mem(dev->kgd, size, | 1859 | retval = amdgpu_amdkfd_alloc_gtt_mem(dev->adev, size, |
| 1854 | &(mem_obj->gtt_mem), &(mem_obj->gpu_addr), | 1860 | &(mem_obj->gtt_mem), &(mem_obj->gpu_addr), |
| 1855 | (void *)&(mem_obj->cpu_ptr), false); | 1861 | (void *)&(mem_obj->cpu_ptr), false); |
| 1856 | 1862 | ||
| @@ -1867,7 +1873,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_dev *dev) · 10 unmodified lines | |||
| 1867 | if (!dqm) | 1873 | if (!dqm) |
| 1868 | return NULL; | 1874 | return NULL; |
| 1869 | 1875 | ||
| 1870 | switch (dev->device_info->asic_family) { | 1876 | switch (dev->adev->asic_type) { |
| 1871 | /* HWS is not available on Hawaii. */ | 1877 | /* HWS is not available on Hawaii. */ |
| 1872 | case CHIP_HAWAII: | 1878 | case CHIP_HAWAII: |
| 1873 | /* HWS depends on CWSR for timely dequeue. CWSR is not | 1879 | /* HWS depends on CWSR for timely dequeue. CWSR is not |
| @@ -1905,6 +1911,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_dev *dev) · 31 unmodified lines | |||
| 1905 | dqm->ops.evict_process_queues = evict_process_queues_cpsch; | 1911 | dqm->ops.evict_process_queues = evict_process_queues_cpsch; |
| 1906 | dqm->ops.restore_process_queues = restore_process_queues_cpsch; | 1912 | dqm->ops.restore_process_queues = restore_process_queues_cpsch; |
| 1907 | dqm->ops.get_wave_state = get_wave_state; | 1913 | dqm->ops.get_wave_state = get_wave_state; |
| 1914 | dqm->ops.reset_queues = reset_queues_cpsch; | ||
| 1908 | break; | 1915 | break; |
| 1909 | case KFD_SCHED_POLICY_NO_HWS: | 1916 | case KFD_SCHED_POLICY_NO_HWS: |
| 1910 | /* initialize dqm for no cp scheduling */ | 1917 | /* initialize dqm for no cp scheduling */ |
| @@ -1930,7 +1937,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_dev *dev) · 19 unmodified lines | |||
| 1930 | goto out_free; | 1937 | goto out_free; |
| 1931 | } | 1938 | } |
| 1932 | 1939 | ||
| 1933 | switch (dev->device_info->asic_family) { | 1940 | switch (dev->adev->asic_type) { |
| 1934 | case CHIP_CARRIZO: | 1941 | case CHIP_CARRIZO: |
| 1935 | device_queue_manager_init_vi(&dqm->asic_ops); | 1942 | device_queue_manager_init_vi(&dqm->asic_ops); |
| 1936 | break; | 1943 | break; |
| @@ -1952,31 +1959,16 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_dev *dev) · 15 unmodified lines | |||
| 1952 | device_queue_manager_init_vi_tonga(&dqm->asic_ops); | 1959 | device_queue_manager_init_vi_tonga(&dqm->asic_ops); |
| 1953 | break; | 1960 | break; |
| 1954 | 1961 | ||
| 1955 | case CHIP_VEGA10: | ||
| 1956 | case CHIP_VEGA12: | ||
| 1957 | case CHIP_VEGA20: | ||
| 1958 | case CHIP_RAVEN: | ||
| 1959 | case CHIP_RENOIR: | ||
| 1960 | case CHIP_ARCTURUS: | ||
| 1961 | case CHIP_ALDEBARAN: | ||
| 1962 | device_queue_manager_init_v9(&dqm->asic_ops); | ||
| 1963 | break; | ||
| 1964 | case CHIP_NAVI10: | ||
| 1965 | case CHIP_NAVI12: | ||
| 1966 | case CHIP_NAVI14: | ||
| 1967 | case CHIP_SIENNA_CICHLID: | ||
| 1968 | case CHIP_NAVY_FLOUNDER: | ||
| 1969 | case CHIP_VANGOGH: | ||
| 1970 | case CHIP_DIMGREY_CAVEFISH: | ||
| 1971 | case CHIP_BEIGE_GOBY: | ||
| 1972 | case CHIP_YELLOW_CARP: | ||
| 1973 | case CHIP_CYAN_SKILLFISH: | ||
| 1974 | device_queue_manager_init_v10_navi10(&dqm->asic_ops); | ||
| 1975 | break; | ||
| 1976 | default: | 1962 | default: |
| 1977 | WARN(1, "Unexpected ASIC family %u", | 1963 | if (KFD_GC_VERSION(dev) >= IP_VERSION(10, 1, 1)) |
| 1978 | dev->device_info->asic_family); | 1964 | device_queue_manager_init_v10_navi10(&dqm->asic_ops); |
| 1979 | goto out_free; | 1965 | else if (KFD_GC_VERSION(dev) >= IP_VERSION(9, 0, 1)) |
| 1966 | device_queue_manager_init_v9(&dqm->asic_ops); | ||
| 1967 | else { | ||
| 1968 | WARN(1, "Unexpected ASIC family %u", | ||
| 1969 | dev->adev->asic_type); | ||
| 1970 | goto out_free; | ||
| 1971 | } | ||
| 1980 | } | 1972 | } |
| 1981 | 1973 | ||
| 1982 | if (init_mqd_managers(dqm)) | 1974 | if (init_mqd_managers(dqm)) |
| @@ -2000,7 +1992,7 @@ static void deallocate_hiq_sdma_mqd(struct kfd_dev *dev, · 17 unmodified lines | |||
| 2000 | { | 1992 | { |
| 2001 | WARN(!mqd, "No hiq sdma mqd trunk to free"); | 1993 | WARN(!mqd, "No hiq sdma mqd trunk to free"); |
| 2002 | 1994 | ||
| 2003 | amdgpu_amdkfd_free_gtt_mem(dev->kgd, mqd->gtt_mem); | 1995 | amdgpu_amdkfd_free_gtt_mem(dev->adev, mqd->gtt_mem); |
| 2004 | } | 1996 | } |
| 2005 | 1997 | ||
| 2006 | void device_queue_manager_uninit(struct device_queue_manager *dqm) | 1998 | void device_queue_manager_uninit(struct device_queue_manager *dqm) |
| @@ -2031,7 +2023,7 @@ static void kfd_process_hw_exception(struct work_struct *work) · 24 unmodified lines | |||
| 2031 | { | 2023 | { |
| 2032 | struct device_queue_manager *dqm = container_of(work, | 2024 | struct device_queue_manager *dqm = container_of(work, |
| 2033 | struct device_queue_manager, hw_exception_work); | 2025 | struct device_queue_manager, hw_exception_work); |
| 2034 | amdgpu_amdkfd_gpu_reset(dqm->dev->kgd); | 2026 | amdgpu_amdkfd_gpu_reset(dqm->dev->adev); |
| 2035 | } | 2027 | } |
| 2036 | 2028 | ||
| 2037 | #if defined(CONFIG_DEBUG_FS) | 2029 | #if defined(CONFIG_DEBUG_FS) |
| @@ -2070,7 +2062,7 @@ int dqm_debugfs_hqds(struct seq_file *m, void *data) · 32 unmodified lines | |||
| 2070 | return 0; | 2062 | return 0; |
| 2071 | } | 2063 | } |
| 2072 | 2064 | ||
| 2073 | r = dqm->dev->kfd2kgd->hqd_dump(dqm->dev->kgd, | 2065 | r = dqm->dev->kfd2kgd->hqd_dump(dqm->dev->adev, |
| 2074 | KFD_CIK_HIQ_PIPE, KFD_CIK_HIQ_QUEUE, | 2066 | KFD_CIK_HIQ_PIPE, KFD_CIK_HIQ_QUEUE, |
| 2075 | &dump, &n_regs); | 2067 | &dump, &n_regs); |
| 2076 | if (!r) { | 2068 | if (!r) { |
| @@ -2092,7 +2084,7 @@ int dqm_debugfs_hqds(struct seq_file *m, void *data) · 15 unmodified lines | |||
| 2092 | continue; | 2084 | continue; |
| 2093 | 2085 | ||
| 2094 | r = dqm->dev->kfd2kgd->hqd_dump( | 2086 | r = dqm->dev->kfd2kgd->hqd_dump( |
| 2095 | dqm->dev->kgd, pipe, queue, &dump, &n_regs); | 2087 | dqm->dev->adev, pipe, queue, &dump, &n_regs); |
| 2096 | if (r) | 2088 | if (r) |
| 2097 | break; | 2089 | break; |
| 2098 | 2090 | ||
| @@ -2106,10 +2098,10 @@ int dqm_debugfs_hqds(struct seq_file *m, void *data) · 7 unmodified lines | |||
| 2106 | 2098 | ||
| 2107 | for (pipe = 0; pipe < get_num_all_sdma_engines(dqm); pipe++) { | 2099 | for (pipe = 0; pipe < get_num_all_sdma_engines(dqm); pipe++) { |
| 2108 | for (queue = 0; | 2100 | for (queue = 0; |
| 2109 | queue < dqm->dev->device_info->num_sdma_queues_per_engine; | 2101 | queue < dqm->dev->device_info.num_sdma_queues_per_engine; |
| 2110 | queue++) { | 2102 | queue++) { |
| 2111 | r = dqm->dev->kfd2kgd->hqd_sdma_dump( | 2103 | r = dqm->dev->kfd2kgd->hqd_sdma_dump( |
| 2112 | dqm->dev->kgd, pipe, queue, &dump, &n_regs); | 2104 | dqm->dev->adev, pipe, queue, &dump, &n_regs); |
| 2113 | if (r) | 2105 | if (r) |
| 2114 | break; | 2106 | break; |
| 2115 | 2107 | ||
| @@ -81,6 +81,8 @@ struct device_process_node { · 80 unmodified lines | |||
| 81 | * | 81 | * |
| 82 | * @get_wave_state: Retrieves context save state and optionally copies the | 82 | * @get_wave_state: Retrieves context save state and optionally copies the |
| 83 | * control stack, if kept in the MQD, to the given userspace address. | 83 | * control stack, if kept in the MQD, to the given userspace address. |
| 84 | * | ||
| 85 | * @reset_queues: reset queues which consume RAS poison | ||
| 84 | */ | 86 | */ |
| 85 | 87 | ||
| 86 | struct device_queue_manager_ops { | 88 | struct device_queue_manager_ops { |
| @@ -134,6 +136,9 @@ struct device_queue_manager_ops { · 47 unmodified lines | |||
| 134 | void __user *ctl_stack, | 136 | void __user *ctl_stack, |
| 135 | u32 *ctl_stack_used_size, | 137 | u32 *ctl_stack_used_size, |
| 136 | u32 *save_area_used_size); | 138 | u32 *save_area_used_size); |
| 139 | |||
| 140 | int (*reset_queues)(struct device_queue_manager *dqm, | ||
| 141 | uint16_t pasid); | ||
| 137 | }; | 142 | }; |
| 138 | 143 | ||
| 139 | struct device_queue_manager_asic_ops { | 144 | struct device_queue_manager_asic_ops { |
| @@ -62,7 +62,7 @@ static int update_qpd_v9(struct device_queue_manager *dqm, · 61 unmodified lines | |||
| 62 | SH_MEM_ALIGNMENT_MODE_UNALIGNED << | 62 | SH_MEM_ALIGNMENT_MODE_UNALIGNED << |
| 63 | SH_MEM_CONFIG__ALIGNMENT_MODE__SHIFT; | 63 | SH_MEM_CONFIG__ALIGNMENT_MODE__SHIFT; |
| 64 | 64 | ||
| 65 | if (dqm->dev->device_info->asic_family == CHIP_ALDEBARAN) { | 65 | if (KFD_GC_VERSION(dqm->dev) == IP_VERSION(9, 4, 2)) { |
| 66 | /* Aldebaran can safely support different XNACK modes | 66 | /* Aldebaran can safely support different XNACK modes |
| 67 | * per process | 67 | * per process |
| 68 | */ | 68 | */ |
| @@ -48,7 +48,7 @@ · 47 unmodified lines | |||
| 48 | /* # of doorbell bytes allocated for each process. */ | 48 | /* # of doorbell bytes allocated for each process. */ |
| 49 | size_t kfd_doorbell_process_slice(struct kfd_dev *kfd) | 49 | size_t kfd_doorbell_process_slice(struct kfd_dev *kfd) |
| 50 | { | 50 | { |
| 51 | return roundup(kfd->device_info->doorbell_size * | 51 | return roundup(kfd->device_info.doorbell_size * |
| 52 | KFD_MAX_NUM_OF_QUEUES_PER_PROCESS, | 52 | KFD_MAX_NUM_OF_QUEUES_PER_PROCESS, |
| 53 | PAGE_SIZE); | 53 | PAGE_SIZE); |
| 54 | } | 54 | } |
| @@ -180,7 +180,7 @@ void __iomem *kfd_get_kernel_doorbell(struct kfd_dev *kfd, · 125 unmodified lines | |||
| 180 | if (inx >= KFD_MAX_NUM_OF_QUEUES_PER_PROCESS) | 180 | if (inx >= KFD_MAX_NUM_OF_QUEUES_PER_PROCESS) |
| 181 | return NULL; | 181 | return NULL; |
| 182 | 182 | ||
| 183 | inx *= kfd->device_info->doorbell_size / sizeof(u32); | 183 | inx *= kfd->device_info.doorbell_size / sizeof(u32); |
| 184 | 184 | ||
| 185 | /* | 185 | /* |
| 186 | * Calculating the kernel doorbell offset using the first | 186 | * Calculating the kernel doorbell offset using the first |
| @@ -201,7 +201,7 @@ void kfd_release_kernel_doorbell(struct kfd_dev *kfd, u32 __iomem *db_addr) · 14 unmodified lines | |||
| 201 | unsigned int inx; | 201 | unsigned int inx; |
| 202 | 202 | ||
| 203 | inx = (unsigned int)(db_addr - kfd->doorbell_kernel_ptr) | 203 | inx = (unsigned int)(db_addr - kfd->doorbell_kernel_ptr) |
| 204 | * sizeof(u32) / kfd->device_info->doorbell_size; | 204 | * sizeof(u32) / kfd->device_info.doorbell_size; |
| 205 | 205 | ||
| 206 | mutex_lock(&kfd->doorbell_mutex); | 206 | mutex_lock(&kfd->doorbell_mutex); |
| 207 | __clear_bit(inx, kfd->doorbell_available_index); | 207 | __clear_bit(inx, kfd->doorbell_available_index); |
| @@ -239,7 +239,7 @@ unsigned int kfd_get_doorbell_dw_offset_in_bar(struct kfd_dev *kfd, · 31 unmodified lines | |||
| 239 | return kfd->doorbell_base_dw_offset + | 239 | return kfd->doorbell_base_dw_offset + |
| 240 | pdd->doorbell_index | 240 | pdd->doorbell_index |
| 241 | * kfd_doorbell_process_slice(kfd) / sizeof(u32) + | 241 | * kfd_doorbell_process_slice(kfd) / sizeof(u32) + |
| 242 | doorbell_id * kfd->device_info->doorbell_size / sizeof(u32); | 242 | doorbell_id * kfd->device_info.doorbell_size / sizeof(u32); |
| 243 | } | 243 | } |
| 244 | 244 | ||
| 245 | uint64_t kfd_get_number_elems(struct kfd_dev *kfd) | 245 | uint64_t kfd_get_number_elems(struct kfd_dev *kfd) |
| @@ -935,8 +935,10 @@ void kfd_signal_iommu_event(struct kfd_dev *dev, u32 pasid, · 934 unmodified lines | |||
| 935 | /* Workaround on Raven to not kill the process when memory is freed | 935 | /* Workaround on Raven to not kill the process when memory is freed |
| 936 | * before IOMMU is able to finish processing all the excessive PPRs | 936 | * before IOMMU is able to finish processing all the excessive PPRs |
| 937 | */ | 937 | */ |
| 938 | if (dev->device_info->asic_family != CHIP_RAVEN && | 938 | |
| 939 | dev->device_info->asic_family != CHIP_RENOIR) { | 939 | if (KFD_GC_VERSION(dev) != IP_VERSION(9, 1, 0) && |
| 940 | KFD_GC_VERSION(dev) != IP_VERSION(9, 2, 2) && | ||
| 941 | KFD_GC_VERSION(dev) != IP_VERSION(9, 3, 0)) { | ||
| 940 | mutex_lock(&p->event_mutex); | 942 | mutex_lock(&p->event_mutex); |
| 941 | 943 | ||
| 942 | /* Lookup events by type and signal them */ | 944 | /* Lookup events by type and signal them */ |
| @@ -394,7 +394,7 @@ int kfd_init_apertures(struct kfd_process *process) · 393 unmodified lines | |||
| 394 | pdd->gpuvm_base = pdd->gpuvm_limit = 0; | 394 | pdd->gpuvm_base = pdd->gpuvm_limit = 0; |
| 395 | pdd->scratch_base = pdd->scratch_limit = 0; | 395 | pdd->scratch_base = pdd->scratch_limit = 0; |
| 396 | } else { | 396 | } else { |
| 397 | switch (dev->device_info->asic_family) { | 397 | switch (dev->adev->asic_type) { |
| 398 | case CHIP_KAVERI: | 398 | case CHIP_KAVERI: |
| 399 | case CHIP_HAWAII: | 399 | case CHIP_HAWAII: |
| 400 | case CHIP_CARRIZO: | 400 | case CHIP_CARRIZO: |
| @@ -406,29 +406,14 @@ int kfd_init_apertures(struct kfd_process *process) · 5 unmodified lines | |||
| 406 | case CHIP_VEGAM: | 406 | case CHIP_VEGAM: |
| 407 | kfd_init_apertures_vi(pdd, id); | 407 | kfd_init_apertures_vi(pdd, id); |
| 408 | break; | 408 | break; |
| 409 | case CHIP_VEGA10: | ||
| 410 | case CHIP_VEGA12: | ||
| 411 | case CHIP_VEGA20: | ||
| 412 | case CHIP_RAVEN: | ||
| 413 | case CHIP_RENOIR: | ||
| 414 | case CHIP_ARCTURUS: | ||
| 415 | case CHIP_ALDEBARAN: | ||
| 416 | case CHIP_NAVI10: | ||
| 417 | case CHIP_NAVI12: | ||
| 418 | case CHIP_NAVI14: | ||
| 419 | case CHIP_SIENNA_CICHLID: | ||
| 420 | case CHIP_NAVY_FLOUNDER: | ||
| 421 | case CHIP_VANGOGH: | ||
| 422 | case CHIP_DIMGREY_CAVEFISH: | ||
| 423 | case CHIP_BEIGE_GOBY: | ||
| 424 | case CHIP_YELLOW_CARP: | ||
| 425 | case CHIP_CYAN_SKILLFISH: | ||
| 426 | kfd_init_apertures_v9(pdd, id); | ||
| 427 | break; | ||
| 428 | default: | 409 | default: |
| 429 | WARN(1, "Unexpected ASIC family %u", | 410 | if (KFD_GC_VERSION(dev) >= IP_VERSION(9, 0, 1)) |
| 430 | dev->device_info->asic_family); | 411 | kfd_init_apertures_v9(pdd, id); |
| 431 | return -EINVAL; | 412 | else { |
| 413 | WARN(1, "Unexpected ASIC family %u", | ||
| 414 | dev->adev->asic_type); | ||
| 415 | return -EINVAL; | ||
| 416 | } | ||
| 432 | } | 417 | } |
| 433 | 418 | ||
| 434 | if (!dev->use_iommu_v2) { | 419 | if (!dev->use_iommu_v2) { |
| @@ -89,6 +89,44 @@ enum SQ_INTERRUPT_ERROR_TYPE { · 88 unmodified lines | |||
| 89 | #define KFD_SQ_INT_DATA__ERR_TYPE_MASK 0xF00000 | 89 | #define KFD_SQ_INT_DATA__ERR_TYPE_MASK 0xF00000 |
| 90 | #define KFD_SQ_INT_DATA__ERR_TYPE__SHIFT 20 | 90 | #define KFD_SQ_INT_DATA__ERR_TYPE__SHIFT 20 |
| 91 | 91 | ||
| 92 | static void event_interrupt_poison_consumption(struct kfd_dev *dev, | ||
| 93 | uint16_t pasid, uint16_t source_id) | ||
| 94 | { | ||
| 95 | int ret = -EINVAL; | ||
| 96 | struct kfd_process *p = kfd_lookup_process_by_pasid(pasid); | ||
| 97 | |||
| 98 | if (!p) | ||
| 99 | return; | ||
| 100 | |||
| 101 | /* all queues of a process will be unmapped in one time */ | ||
| 102 | if (atomic_read(&p->poison)) { | ||
| 103 | kfd_unref_process(p); | ||
| 104 | return; | ||
| 105 | } | ||
| 106 | |||
| 107 | atomic_set(&p->poison, 1); | ||
| 108 | kfd_unref_process(p); | ||
| 109 | |||
| 110 | switch (source_id) { | ||
| 111 | case SOC15_INTSRC_SQ_INTERRUPT_MSG: | ||
| 112 | if (dev->dqm->ops.reset_queues) | ||
| 113 | ret = dev->dqm->ops.reset_queues(dev->dqm, pasid); | ||
| 114 | break; | ||
| 115 | case SOC15_INTSRC_SDMA_ECC: | ||
| 116 | default: | ||
| 117 | break; | ||
| 118 | } | ||
| 119 | |||
| 120 | kfd_signal_poison_consumed_event(dev, pasid); | ||
| 121 | |||
| 122 | /* resetting queue passes, do page retirement without gpu reset | ||
| 123 | resetting queue fails, fallback to gpu reset solution */ | ||
| 124 | if (!ret) | ||
| 125 | amdgpu_amdkfd_ras_poison_consumption_handler(dev->adev, false); | ||
| 126 | else | ||
| 127 | amdgpu_amdkfd_ras_poison_consumption_handler(dev->adev, true); | ||
| 128 | } | ||
| 129 | |||
| 92 | static bool event_interrupt_isr_v9(struct kfd_dev *dev, | 130 | static bool event_interrupt_isr_v9(struct kfd_dev *dev, |
| 93 | const uint32_t *ih_ring_entry, | 131 | const uint32_t *ih_ring_entry, |
| 94 | uint32_t *patched_ihre, | 132 | uint32_t *patched_ihre, |
| @@ -135,7 +173,7 @@ static bool event_interrupt_isr_v9(struct kfd_dev *dev, · 40 unmodified lines | |||
| 135 | 173 | ||
| 136 | *patched_flag = true; | 174 | *patched_flag = true; |
| 137 | memcpy(patched_ihre, ih_ring_entry, | 175 | memcpy(patched_ihre, ih_ring_entry, |
| 138 | dev->device_info->ih_ring_entry_size); | 176 | dev->device_info.ih_ring_entry_size); |
| 139 | 177 | ||
| 140 | pasid = dev->dqm->vmid_pasid[vmid]; | 178 | pasid = dev->dqm->vmid_pasid[vmid]; |
| 141 | 179 | ||
| @@ -230,8 +268,7 @@ static void event_interrupt_wq_v9(struct kfd_dev *dev, · 88 unmodified lines | |||
| 230 | sq_intr_err); | 268 | sq_intr_err); |
| 231 | if (sq_intr_err != SQ_INTERRUPT_ERROR_TYPE_ILLEGAL_INST && | 269 | if (sq_intr_err != SQ_INTERRUPT_ERROR_TYPE_ILLEGAL_INST && |
| 232 | sq_intr_err != SQ_INTERRUPT_ERROR_TYPE_MEMVIOL) { | 270 | sq_intr_err != SQ_INTERRUPT_ERROR_TYPE_MEMVIOL) { |
| 233 | kfd_signal_poison_consumed_event(dev, pasid); | 271 | event_interrupt_poison_consumption(dev, pasid, source_id); |
| 234 | amdgpu_amdkfd_ras_poison_consumption_handler(dev->kgd); | ||
| 235 | return; | 272 | return; |
| 236 | } | 273 | } |
| 237 | break; | 274 | break; |
| @@ -252,8 +289,7 @@ static void event_interrupt_wq_v9(struct kfd_dev *dev, · 14 unmodified lines | |||
| 252 | if (source_id == SOC15_INTSRC_SDMA_TRAP) { | 289 | if (source_id == SOC15_INTSRC_SDMA_TRAP) { |
| 253 | kfd_signal_event_interrupt(pasid, context_id0 & 0xfffffff, 28); | 290 | kfd_signal_event_interrupt(pasid, context_id0 & 0xfffffff, 28); |
| 254 | } else if (source_id == SOC15_INTSRC_SDMA_ECC) { | 291 | } else if (source_id == SOC15_INTSRC_SDMA_ECC) { |
| 255 | kfd_signal_poison_consumed_event(dev, pasid); | 292 | event_interrupt_poison_consumption(dev, pasid, source_id); |
| 256 | amdgpu_amdkfd_ras_poison_consumption_handler(dev->kgd); | ||
| 257 | return; | 293 | return; |
| 258 | } | 294 | } |
| 259 | } else if (client_id == SOC15_IH_CLIENTID_VMC || | 295 | } else if (client_id == SOC15_IH_CLIENTID_VMC || |
| @@ -54,7 +54,7 @@ int kfd_interrupt_init(struct kfd_dev *kfd) · 53 unmodified lines | |||
| 54 | int r; | 54 | int r; |
| 55 | 55 | ||
| 56 | r = kfifo_alloc(&kfd->ih_fifo, | 56 | r = kfifo_alloc(&kfd->ih_fifo, |
| 57 | KFD_IH_NUM_ENTRIES * kfd->device_info->ih_ring_entry_size, | 57 | KFD_IH_NUM_ENTRIES * kfd->device_info.ih_ring_entry_size, |
| 58 | GFP_KERNEL); | 58 | GFP_KERNEL); |
| 59 | if (r) { | 59 | if (r) { |
| 60 | dev_err(kfd_chardev(), "Failed to allocate IH fifo\n"); | 60 | dev_err(kfd_chardev(), "Failed to allocate IH fifo\n"); |
| @@ -114,8 +114,8 @@ bool enqueue_ih_ring_entry(struct kfd_dev *kfd, const void *ih_ring_entry) · 53 unmodified lines | |||
| 114 | int count; | 114 | int count; |
| 115 | 115 | ||
| 116 | count = kfifo_in(&kfd->ih_fifo, ih_ring_entry, | 116 | count = kfifo_in(&kfd->ih_fifo, ih_ring_entry, |
| 117 | kfd->device_info->ih_ring_entry_size); | 117 | kfd->device_info.ih_ring_entry_size); |
| 118 | if (count != kfd->device_info->ih_ring_entry_size) { | 118 | if (count != kfd->device_info.ih_ring_entry_size) { |
| 119 | dev_err_ratelimited(kfd_chardev(), | 119 | dev_err_ratelimited(kfd_chardev(), |
| 120 | "Interrupt ring overflow, dropping interrupt %d\n", | 120 | "Interrupt ring overflow, dropping interrupt %d\n", |
| 121 | count); | 121 | count); |
| @@ -133,11 +133,11 @@ static bool dequeue_ih_ring_entry(struct kfd_dev *kfd, void *ih_ring_entry) · 11 unmodified lines | |||
| 133 | int count; | 133 | int count; |
| 134 | 134 | ||
| 135 | count = kfifo_out(&kfd->ih_fifo, ih_ring_entry, | 135 | count = kfifo_out(&kfd->ih_fifo, ih_ring_entry, |
| 136 | kfd->device_info->ih_ring_entry_size); | 136 | kfd->device_info.ih_ring_entry_size); |
| 137 | 137 | ||
| 138 | WARN_ON(count && count != kfd->device_info->ih_ring_entry_size); | 138 | WARN_ON(count && count != kfd->device_info.ih_ring_entry_size); |
| 139 | 139 | ||
| 140 | return count == kfd->device_info->ih_ring_entry_size; | 140 | return count == kfd->device_info.ih_ring_entry_size; |
| 141 | } | 141 | } |
| 142 | 142 | ||
| 143 | static void interrupt_wq(struct work_struct *work) | 143 | static void interrupt_wq(struct work_struct *work) |
| @@ -146,13 +146,13 @@ static void interrupt_wq(struct work_struct *work) · 2 unmodified lines | |||
| 146 | interrupt_work); | 146 | interrupt_work); |
| 147 | uint32_t ih_ring_entry[KFD_MAX_RING_ENTRY_SIZE]; | 147 | uint32_t ih_ring_entry[KFD_MAX_RING_ENTRY_SIZE]; |
| 148 | 148 | ||
| 149 | if (dev->device_info->ih_ring_entry_size > sizeof(ih_ring_entry)) { | 149 | if (dev->device_info.ih_ring_entry_size > sizeof(ih_ring_entry)) { |
| 150 | dev_err_once(kfd_chardev(), "Ring entry too small\n"); | 150 | dev_err_once(kfd_chardev(), "Ring entry too small\n"); |
| 151 | return; | 151 | return; |
| 152 | } | 152 | } |
| 153 | 153 | ||
| 154 | while (dequeue_ih_ring_entry(dev, ih_ring_entry)) | 154 | while (dequeue_ih_ring_entry(dev, ih_ring_entry)) |
| 155 | dev->device_info->event_interrupt_class->interrupt_wq(dev, | 155 | dev->device_info.event_interrupt_class->interrupt_wq(dev, |
| 156 | ih_ring_entry); | 156 | ih_ring_entry); |
| 157 | } | 157 | } |
| 158 | 158 | ||
| @@ -163,7 +163,7 @@ bool interrupt_is_wanted(struct kfd_dev *dev, · 4 unmodified lines | |||
| 163 | /* integer and bitwise OR so there is no boolean short-circuiting */ | 163 | /* integer and bitwise OR so there is no boolean short-circuiting */ |
| 164 | unsigned int wanted = 0; | 164 | unsigned int wanted = 0; |
| 165 | 165 | ||
| 166 | wanted |= dev->device_info->event_interrupt_class->interrupt_isr(dev, | 166 | wanted |= dev->device_info.event_interrupt_class->interrupt_isr(dev, |
| 167 | ih_ring_entry, patched_ihre, flag); | 167 | ih_ring_entry, patched_ihre, flag); |
| 168 | 168 | ||
| 169 | return wanted != 0; | 169 | return wanted != 0; |
| @@ -89,7 +89,7 @@ int kfd_iommu_device_init(struct kfd_dev *kfd) · 88 unmodified lines | |||
| 89 | } | 89 | } |
| 90 | 90 | ||
| 91 | pasid_limit = min_t(unsigned int, | 91 | pasid_limit = min_t(unsigned int, |
| 92 | (unsigned int)(1 << kfd->device_info->max_pasid_bits), | 92 | (unsigned int)(1 << kfd->device_info.max_pasid_bits), |
| 93 | iommu_info.max_pasids); | 93 | iommu_info.max_pasids); |
| 94 | 94 | ||
| 95 | if (!kfd_set_pasid_limit(pasid_limit)) { | 95 | if (!kfd_set_pasid_limit(pasid_limit)) { |
| @@ -91,7 +91,7 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_dev *dev, · 90 unmodified lines | |||
| 91 | kq->pq_gpu_addr = kq->pq->gpu_addr; | 91 | kq->pq_gpu_addr = kq->pq->gpu_addr; |
| 92 | 92 | ||
| 93 | /* For CIK family asics, kq->eop_mem is not needed */ | 93 | /* For CIK family asics, kq->eop_mem is not needed */ |
| 94 | if (dev->device_info->asic_family > CHIP_MULLINS) { | 94 | if (dev->adev->asic_type > CHIP_MULLINS) { |
| 95 | retval = kfd_gtt_sa_allocate(dev, PAGE_SIZE, &kq->eop_mem); | 95 | retval = kfd_gtt_sa_allocate(dev, PAGE_SIZE, &kq->eop_mem); |
| 96 | if (retval != 0) | 96 | if (retval != 0) |
| 97 | goto err_eop_allocate_vidmem; | 97 | goto err_eop_allocate_vidmem; |
| @@ -111,7 +111,7 @@ static bool kq_initialize(struct kernel_queue *kq, struct kfd_dev *dev, · 13 unmodified lines | |||
| 111 | kq->rptr_kernel = kq->rptr_mem->cpu_ptr; | 111 | kq->rptr_kernel = kq->rptr_mem->cpu_ptr; |
| 112 | kq->rptr_gpu_addr = kq->rptr_mem->gpu_addr; | 112 | kq->rptr_gpu_addr = kq->rptr_mem->gpu_addr; |
| 113 | 113 | ||
| 114 | retval = kfd_gtt_sa_allocate(dev, dev->device_info->doorbell_size, | 114 | retval = kfd_gtt_sa_allocate(dev, dev->device_info.doorbell_size, |
| 115 | &kq->wptr_mem); | 115 | &kq->wptr_mem); |
| 116 | 116 | ||
| 117 | if (retval != 0) | 117 | if (retval != 0) |
| @@ -297,7 +297,7 @@ void kq_submit_packet(struct kernel_queue *kq) · 179 unmodified lines | |||
| 297 | } | 297 | } |
| 298 | pr_debug("\n"); | 298 | pr_debug("\n"); |
| 299 | #endif | 299 | #endif |
| 300 | if (kq->dev->device_info->doorbell_size == 8) { | 300 | if (kq->dev->device_info.doorbell_size == 8) { |
| 301 | *kq->wptr64_kernel = kq->pending_wptr64; | 301 | *kq->wptr64_kernel = kq->pending_wptr64; |
| 302 | write_kernel_doorbell64(kq->queue->properties.doorbell_ptr, | 302 | write_kernel_doorbell64(kq->queue->properties.doorbell_ptr, |
| 303 | kq->pending_wptr64); | 303 | kq->pending_wptr64); |
| @@ -310,7 +310,7 @@ void kq_submit_packet(struct kernel_queue *kq) · 6 unmodified lines | |||
| 310 | 310 | ||
| 311 | void kq_rollback_packet(struct kernel_queue *kq) | 311 | void kq_rollback_packet(struct kernel_queue *kq) |
| 312 | { | 312 | { |
| 313 | if (kq->dev->device_info->doorbell_size == 8) { | 313 | if (kq->dev->device_info.doorbell_size == 8) { |
| 314 | kq->pending_wptr64 = *kq->wptr64_kernel; | 314 | kq->pending_wptr64 = *kq->wptr64_kernel; |
| 315 | kq->pending_wptr = *kq->wptr_kernel % | 315 | kq->pending_wptr = *kq->wptr_kernel % |
| 316 | (kq->queue->properties.queue_size / 4); | 316 | (kq->queue->properties.queue_size / 4); |
| @@ -108,8 +108,8 @@ error_free: · 107 unmodified lines | |||
| 108 | * svm_migrate_copy_memory_gart - sdma copy data between ram and vram | 108 | * svm_migrate_copy_memory_gart - sdma copy data between ram and vram |
| 109 | * | 109 | * |
| 110 | * @adev: amdgpu device the sdma ring running | 110 | * @adev: amdgpu device the sdma ring running |
| 111 | * @src: source page address array | 111 | * @sys: system DMA pointer to be copied |
| 112 | * @dst: destination page address array | 112 | * @vram: vram destination DMA pointer |
| 113 | * @npages: number of pages to copy | 113 | * @npages: number of pages to copy |
| 114 | * @direction: enum MIGRATION_COPY_DIR | 114 | * @direction: enum MIGRATION_COPY_DIR |
| 115 | * @mfence: output, sdma fence to signal after sdma is done | 115 | * @mfence: output, sdma fence to signal after sdma is done |
| @@ -549,7 +549,7 @@ static void svm_migrate_page_free(struct page *page) · 433 unmodified lines | |||
| 549 | 549 | ||
| 550 | if (svm_bo) { | 550 | if (svm_bo) { |
| 551 | pr_debug_ratelimited("ref: %d\n", kref_read(&svm_bo->kref)); | 551 | pr_debug_ratelimited("ref: %d\n", kref_read(&svm_bo->kref)); |
| 552 | svm_range_bo_unref(svm_bo); | 552 | svm_range_bo_unref_async(svm_bo); |
| 553 | } | 553 | } |
| 554 | } | 554 | } |
| 555 | 555 | ||
| @@ -938,7 +938,7 @@ int svm_migrate_init(struct amdgpu_device *adev) · 382 unmodified lines | |||
| 938 | void *r; | 938 | void *r; |
| 939 | 939 | ||
| 940 | /* Page migration works on Vega10 or newer */ | 940 | /* Page migration works on Vega10 or newer */ |
| 941 | if (kfddev->device_info->asic_family < CHIP_VEGA10) | 941 | if (!KFD_IS_SOC15(kfddev)) |
| 942 | return -EINVAL; | 942 | return -EINVAL; |
| 943 | 943 | ||
| 944 | pgmap = &kfddev->pgmap; | 944 | pgmap = &kfddev->pgmap; |
| @@ -71,7 +71,7 @@ struct kfd_mem_obj *allocate_sdma_mqd(struct kfd_dev *dev, · 70 unmodified lines | |||
| 71 | return NULL; | 71 | return NULL; |
| 72 | 72 | ||
| 73 | offset = (q->sdma_engine_id * | 73 | offset = (q->sdma_engine_id * |
| 74 | dev->device_info->num_sdma_queues_per_engine + | 74 | dev->device_info.num_sdma_queues_per_engine + |
| 75 | q->sdma_queue_id) * | 75 | q->sdma_queue_id) * |
| 76 | dev->dqm->mqd_mgrs[KFD_MQD_TYPE_SDMA]->mqd_size; | 76 | dev->dqm->mqd_mgrs[KFD_MQD_TYPE_SDMA]->mqd_size; |
| 77 | 77 | ||
| @@ -100,7 +100,7 @@ void mqd_symmetrically_map_cu_mask(struct mqd_manager *mm, · 22 unmodified lines | |||
| 100 | struct kfd_cu_info cu_info; | 100 | struct kfd_cu_info cu_info; |
| 101 | uint32_t cu_per_sh[KFD_MAX_NUM_SE][KFD_MAX_NUM_SH_PER_SE] = {0}; | 101 | uint32_t cu_per_sh[KFD_MAX_NUM_SE][KFD_MAX_NUM_SH_PER_SE] = {0}; |
| 102 | int i, se, sh, cu; | 102 | int i, se, sh, cu; |
| 103 | amdgpu_amdkfd_get_cu_info(mm->dev->kgd, &cu_info); | 103 | amdgpu_amdkfd_get_cu_info(mm->dev->adev, &cu_info); |
| 104 | 104 | ||
| 105 | if (cu_mask_count > cu_info.cu_active_number) | 105 | if (cu_mask_count > cu_info.cu_active_number) |
| 106 | cu_mask_count = cu_info.cu_active_number; | 106 | cu_mask_count = cu_info.cu_active_number; |
| @@ -171,7 +171,7 @@ static int load_mqd(struct mqd_manager *mm, void *mqd, uint32_t pipe_id, · 170 unmodified lines | |||
| 171 | uint32_t wptr_shift = (p->format == KFD_QUEUE_FORMAT_AQL ? 4 : 0); | 171 | uint32_t wptr_shift = (p->format == KFD_QUEUE_FORMAT_AQL ? 4 : 0); |
| 172 | uint32_t wptr_mask = (uint32_t)((p->queue_size / 4) - 1); | 172 | uint32_t wptr_mask = (uint32_t)((p->queue_size / 4) - 1); |
| 173 | 173 | ||
| 174 | return mm->dev->kfd2kgd->hqd_load(mm->dev->kgd, mqd, pipe_id, queue_id, | 174 | return mm->dev->kfd2kgd->hqd_load(mm->dev->adev, mqd, pipe_id, queue_id, |
| 175 | (uint32_t __user *)p->write_ptr, | 175 | (uint32_t __user *)p->write_ptr, |
| 176 | wptr_shift, wptr_mask, mms); | 176 | wptr_shift, wptr_mask, mms); |
| 177 | } | 177 | } |
| @@ -180,7 +180,7 @@ static int load_mqd_sdma(struct mqd_manager *mm, void *mqd, · 2 unmodified lines | |||
| 180 | uint32_t pipe_id, uint32_t queue_id, | 180 | uint32_t pipe_id, uint32_t queue_id, |
| 181 | struct queue_properties *p, struct mm_struct *mms) | 181 | struct queue_properties *p, struct mm_struct *mms) |
| 182 | { | 182 | { |
| 183 | return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->kgd, mqd, | 183 | return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->adev, mqd, |
| 184 | (uint32_t __user *)p->write_ptr, | 184 | (uint32_t __user *)p->write_ptr, |
| 185 | mms); | 185 | mms); |
| 186 | } | 186 | } |
| @@ -276,7 +276,7 @@ static int destroy_mqd(struct mqd_manager *mm, void *mqd, · 89 unmodified lines | |||
| 276 | unsigned int timeout, uint32_t pipe_id, | 276 | unsigned int timeout, uint32_t pipe_id, |
| 277 | uint32_t queue_id) | 277 | uint32_t queue_id) |
| 278 | { | 278 | { |
| 279 | return mm->dev->kfd2kgd->hqd_destroy(mm->dev->kgd, mqd, type, timeout, | 279 | return mm->dev->kfd2kgd->hqd_destroy(mm->dev->adev, mqd, type, timeout, |
| 280 | pipe_id, queue_id); | 280 | pipe_id, queue_id); |
| 281 | } | 281 | } |
| 282 | 282 | ||
| @@ -289,7 +289,7 @@ static int destroy_mqd_sdma(struct mqd_manager *mm, void *mqd, · 6 unmodified lines | |||
| 289 | unsigned int timeout, uint32_t pipe_id, | 289 | unsigned int timeout, uint32_t pipe_id, |
| 290 | uint32_t queue_id) | 290 | uint32_t queue_id) |
| 291 | { | 291 | { |
| 292 | return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->kgd, mqd, timeout); | 292 | return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->adev, mqd, timeout); |
| 293 | } | 293 | } |
| 294 | 294 | ||
| 295 | static bool is_occupied(struct mqd_manager *mm, void *mqd, | 295 | static bool is_occupied(struct mqd_manager *mm, void *mqd, |
| @@ -297,7 +297,7 @@ static bool is_occupied(struct mqd_manager *mm, void *mqd, · 1 unmodified line | |||
| 297 | uint32_t queue_id) | 297 | uint32_t queue_id) |
| 298 | { | 298 | { |
| 299 | 299 | ||
| 300 | return mm->dev->kfd2kgd->hqd_is_occupied(mm->dev->kgd, queue_address, | 300 | return mm->dev->kfd2kgd->hqd_is_occupied(mm->dev->adev, queue_address, |
| 301 | pipe_id, queue_id); | 301 | pipe_id, queue_id); |
| 302 | 302 | ||
| 303 | } | 303 | } |
| @@ -306,7 +306,7 @@ static bool is_occupied_sdma(struct mqd_manager *mm, void *mqd, · 2 unmodified lines | |||
| 306 | uint64_t queue_address, uint32_t pipe_id, | 306 | uint64_t queue_address, uint32_t pipe_id, |
| 307 | uint32_t queue_id) | 307 | uint32_t queue_id) |
| 308 | { | 308 | { |
| 309 | return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->kgd, mqd); | 309 | return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->adev, mqd); |
| 310 | } | 310 | } |
| 311 | 311 | ||
| 312 | /* | 312 | /* |
| @@ -148,7 +148,7 @@ static int load_mqd(struct mqd_manager *mm, void *mqd, · 147 unmodified lines | |||
| 148 | /* AQL write pointer counts in 64B packets, PM4/CP counts in dwords. */ | 148 | /* AQL write pointer counts in 64B packets, PM4/CP counts in dwords. */ |
| 149 | uint32_t wptr_shift = (p->format == KFD_QUEUE_FORMAT_AQL ? 4 : 0); | 149 | uint32_t wptr_shift = (p->format == KFD_QUEUE_FORMAT_AQL ? 4 : 0); |
| 150 | 150 | ||
| 151 | r = mm->dev->kfd2kgd->hqd_load(mm->dev->kgd, mqd, pipe_id, queue_id, | 151 | r = mm->dev->kfd2kgd->hqd_load(mm->dev->adev, mqd, pipe_id, queue_id, |
| 152 | (uint32_t __user *)p->write_ptr, | 152 | (uint32_t __user *)p->write_ptr, |
| 153 | wptr_shift, 0, mms); | 153 | wptr_shift, 0, mms); |
| 154 | return r; | 154 | return r; |
| @@ -158,7 +158,7 @@ static int hiq_load_mqd_kiq(struct mqd_manager *mm, void *mqd, · 3 unmodified lines | |||
| 158 | uint32_t pipe_id, uint32_t queue_id, | 158 | uint32_t pipe_id, uint32_t queue_id, |
| 159 | struct queue_properties *p, struct mm_struct *mms) | 159 | struct queue_properties *p, struct mm_struct *mms) |
| 160 | { | 160 | { |
| 161 | return mm->dev->kfd2kgd->hiq_mqd_load(mm->dev->kgd, mqd, pipe_id, | 161 | return mm->dev->kfd2kgd->hiq_mqd_load(mm->dev->adev, mqd, pipe_id, |
| 162 | queue_id, p->doorbell_off); | 162 | queue_id, p->doorbell_off); |
| 163 | } | 163 | } |
| 164 | 164 | ||
| @@ -239,7 +239,7 @@ static int destroy_mqd(struct mqd_manager *mm, void *mqd, · 74 unmodified lines | |||
| 239 | uint32_t queue_id) | 239 | uint32_t queue_id) |
| 240 | { | 240 | { |
| 241 | return mm->dev->kfd2kgd->hqd_destroy | 241 | return mm->dev->kfd2kgd->hqd_destroy |
| 242 | (mm->dev->kgd, mqd, type, timeout, | 242 | (mm->dev->adev, mqd, type, timeout, |
| 243 | pipe_id, queue_id); | 243 | pipe_id, queue_id); |
| 244 | } | 244 | } |
| 245 | 245 | ||
| @@ -254,7 +254,7 @@ static bool is_occupied(struct mqd_manager *mm, void *mqd, · 8 unmodified lines | |||
| 254 | uint32_t queue_id) | 254 | uint32_t queue_id) |
| 255 | { | 255 | { |
| 256 | return mm->dev->kfd2kgd->hqd_is_occupied( | 256 | return mm->dev->kfd2kgd->hqd_is_occupied( |
| 257 | mm->dev->kgd, queue_address, | 257 | mm->dev->adev, queue_address, |
| 258 | pipe_id, queue_id); | 258 | pipe_id, queue_id); |
| 259 | } | 259 | } |
| 260 | 260 | ||
| @@ -320,7 +320,7 @@ static int load_mqd_sdma(struct mqd_manager *mm, void *mqd, · 59 unmodified lines | |||
| 320 | uint32_t pipe_id, uint32_t queue_id, | 320 | uint32_t pipe_id, uint32_t queue_id, |
| 321 | struct queue_properties *p, struct mm_struct *mms) | 321 | struct queue_properties *p, struct mm_struct *mms) |
| 322 | { | 322 | { |
| 323 | return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->kgd, mqd, | 323 | return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->adev, mqd, |
| 324 | (uint32_t __user *)p->write_ptr, | 324 | (uint32_t __user *)p->write_ptr, |
| 325 | mms); | 325 | mms); |
| 326 | } | 326 | } |
| @@ -363,14 +363,14 @@ static int destroy_mqd_sdma(struct mqd_manager *mm, void *mqd, · 36 unmodified lines | |||
| 363 | unsigned int timeout, uint32_t pipe_id, | 363 | unsigned int timeout, uint32_t pipe_id, |
| 364 | uint32_t queue_id) | 364 | uint32_t queue_id) |
| 365 | { | 365 | { |
| 366 | return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->kgd, mqd, timeout); | 366 | return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->adev, mqd, timeout); |
| 367 | } | 367 | } |
| 368 | 368 | ||
| 369 | static bool is_occupied_sdma(struct mqd_manager *mm, void *mqd, | 369 | static bool is_occupied_sdma(struct mqd_manager *mm, void *mqd, |
| 370 | uint64_t queue_address, uint32_t pipe_id, | 370 | uint64_t queue_address, uint32_t pipe_id, |
| 371 | uint32_t queue_id) | 371 | uint32_t queue_id) |
| 372 | { | 372 | { |
| 373 | return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->kgd, mqd); | 373 | return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->adev, mqd); |
| 374 | } | 374 | } |
| 375 | 375 | ||
| 376 | #if defined(CONFIG_DEBUG_FS) | 376 | #if defined(CONFIG_DEBUG_FS) |
| @@ -108,7 +108,7 @@ static struct kfd_mem_obj *allocate_mqd(struct kfd_dev *kfd, · 107 unmodified lines | |||
| 108 | mqd_mem_obj = kzalloc(sizeof(struct kfd_mem_obj), GFP_KERNEL); | 108 | mqd_mem_obj = kzalloc(sizeof(struct kfd_mem_obj), GFP_KERNEL); |
| 109 | if (!mqd_mem_obj) | 109 | if (!mqd_mem_obj) |
| 110 | return NULL; | 110 | return NULL; |
| 111 | retval = amdgpu_amdkfd_alloc_gtt_mem(kfd->kgd, | 111 | retval = amdgpu_amdkfd_alloc_gtt_mem(kfd->adev, |
| 112 | ALIGN(q->ctl_stack_size, PAGE_SIZE) + | 112 | ALIGN(q->ctl_stack_size, PAGE_SIZE) + |
| 113 | ALIGN(sizeof(struct v9_mqd), PAGE_SIZE), | 113 | ALIGN(sizeof(struct v9_mqd), PAGE_SIZE), |
| 114 | &(mqd_mem_obj->gtt_mem), | 114 | &(mqd_mem_obj->gtt_mem), |
| @@ -199,7 +199,7 @@ static int load_mqd(struct mqd_manager *mm, void *mqd, · 84 unmodified lines | |||
| 199 | /* AQL write pointer counts in 64B packets, PM4/CP counts in dwords. */ | 199 | /* AQL write pointer counts in 64B packets, PM4/CP counts in dwords. */ |
| 200 | uint32_t wptr_shift = (p->format == KFD_QUEUE_FORMAT_AQL ? 4 : 0); | 200 | uint32_t wptr_shift = (p->format == KFD_QUEUE_FORMAT_AQL ? 4 : 0); |
| 201 | 201 | ||
| 202 | return mm->dev->kfd2kgd->hqd_load(mm->dev->kgd, mqd, pipe_id, queue_id, | 202 | return mm->dev->kfd2kgd->hqd_load(mm->dev->adev, mqd, pipe_id, queue_id, |
| 203 | (uint32_t __user *)p->write_ptr, | 203 | (uint32_t __user *)p->write_ptr, |
| 204 | wptr_shift, 0, mms); | 204 | wptr_shift, 0, mms); |
| 205 | } | 205 | } |
| @@ -208,7 +208,7 @@ static int hiq_load_mqd_kiq(struct mqd_manager *mm, void *mqd, · 2 unmodified lines | |||
| 208 | uint32_t pipe_id, uint32_t queue_id, | 208 | uint32_t pipe_id, uint32_t queue_id, |
| 209 | struct queue_properties *p, struct mm_struct *mms) | 209 | struct queue_properties *p, struct mm_struct *mms) |
| 210 | { | 210 | { |
| 211 | return mm->dev->kfd2kgd->hiq_mqd_load(mm->dev->kgd, mqd, pipe_id, | 211 | return mm->dev->kfd2kgd->hiq_mqd_load(mm->dev->adev, mqd, pipe_id, |
| 212 | queue_id, p->doorbell_off); | 212 | queue_id, p->doorbell_off); |
| 213 | } | 213 | } |
| 214 | 214 | ||
| @@ -291,7 +291,7 @@ static int destroy_mqd(struct mqd_manager *mm, void *mqd, · 76 unmodified lines | |||
| 291 | uint32_t queue_id) | 291 | uint32_t queue_id) |
| 292 | { | 292 | { |
| 293 | return mm->dev->kfd2kgd->hqd_destroy | 293 | return mm->dev->kfd2kgd->hqd_destroy |
| 294 | (mm->dev->kgd, mqd, type, timeout, | 294 | (mm->dev->adev, mqd, type, timeout, |
| 295 | pipe_id, queue_id); | 295 | pipe_id, queue_id); |
| 296 | } | 296 | } |
| 297 | 297 | ||
| @@ -301,7 +301,7 @@ static void free_mqd(struct mqd_manager *mm, void *mqd, · 3 unmodified lines | |||
| 301 | struct kfd_dev *kfd = mm->dev; | 301 | struct kfd_dev *kfd = mm->dev; |
| 302 | 302 | ||
| 303 | if (mqd_mem_obj->gtt_mem) { | 303 | if (mqd_mem_obj->gtt_mem) { |
| 304 | amdgpu_amdkfd_free_gtt_mem(kfd->kgd, mqd_mem_obj->gtt_mem); | 304 | amdgpu_amdkfd_free_gtt_mem(kfd->adev, mqd_mem_obj->gtt_mem); |
| 305 | kfree(mqd_mem_obj); | 305 | kfree(mqd_mem_obj); |
| 306 | } else { | 306 | } else { |
| 307 | kfd_gtt_sa_free(mm->dev, mqd_mem_obj); | 307 | kfd_gtt_sa_free(mm->dev, mqd_mem_obj); |
| @@ -313,7 +313,7 @@ static bool is_occupied(struct mqd_manager *mm, void *mqd, · 5 unmodified lines | |||
| 313 | uint32_t queue_id) | 313 | uint32_t queue_id) |
| 314 | { | 314 | { |
| 315 | return mm->dev->kfd2kgd->hqd_is_occupied( | 315 | return mm->dev->kfd2kgd->hqd_is_occupied( |
| 316 | mm->dev->kgd, queue_address, | 316 | mm->dev->adev, queue_address, |
| 317 | pipe_id, queue_id); | 317 | pipe_id, queue_id); |
| 318 | } | 318 | } |
| 319 | 319 | ||
| @@ -375,7 +375,7 @@ static int load_mqd_sdma(struct mqd_manager *mm, void *mqd, · 55 unmodified lines | |||
| 375 | uint32_t pipe_id, uint32_t queue_id, | 375 | uint32_t pipe_id, uint32_t queue_id, |
| 376 | struct queue_properties *p, struct mm_struct *mms) | 376 | struct queue_properties *p, struct mm_struct *mms) |
| 377 | { | 377 | { |
| 378 | return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->kgd, mqd, | 378 | return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->adev, mqd, |
| 379 | (uint32_t __user *)p->write_ptr, | 379 | (uint32_t __user *)p->write_ptr, |
| 380 | mms); | 380 | mms); |
| 381 | } | 381 | } |
| @@ -418,14 +418,14 @@ static int destroy_mqd_sdma(struct mqd_manager *mm, void *mqd, · 36 unmodified lines | |||
| 418 | unsigned int timeout, uint32_t pipe_id, | 418 | unsigned int timeout, uint32_t pipe_id, |
| 419 | uint32_t queue_id) | 419 | uint32_t queue_id) |
| 420 | { | 420 | { |
| 421 | return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->kgd, mqd, timeout); | 421 | return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->adev, mqd, timeout); |
| 422 | } | 422 | } |
| 423 | 423 | ||
| 424 | static bool is_occupied_sdma(struct mqd_manager *mm, void *mqd, | 424 | static bool is_occupied_sdma(struct mqd_manager *mm, void *mqd, |
| 425 | uint64_t queue_address, uint32_t pipe_id, | 425 | uint64_t queue_address, uint32_t pipe_id, |
| 426 | uint32_t queue_id) | 426 | uint32_t queue_id) |
| 427 | { | 427 | { |
| 428 | return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->kgd, mqd); | 428 | return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->adev, mqd); |
| 429 | } | 429 | } |
| 430 | 430 | ||
| 431 | #if defined(CONFIG_DEBUG_FS) | 431 | #if defined(CONFIG_DEBUG_FS) |
| @@ -162,7 +162,7 @@ static int load_mqd(struct mqd_manager *mm, void *mqd, · 161 unmodified lines | |||
| 162 | uint32_t wptr_shift = (p->format == KFD_QUEUE_FORMAT_AQL ? 4 : 0); | 162 | uint32_t wptr_shift = (p->format == KFD_QUEUE_FORMAT_AQL ? 4 : 0); |
| 163 | uint32_t wptr_mask = (uint32_t)((p->queue_size / 4) - 1); | 163 | uint32_t wptr_mask = (uint32_t)((p->queue_size / 4) - 1); |
| 164 | 164 | ||
| 165 | return mm->dev->kfd2kgd->hqd_load(mm->dev->kgd, mqd, pipe_id, queue_id, | 165 | return mm->dev->kfd2kgd->hqd_load(mm->dev->adev, mqd, pipe_id, queue_id, |
| 166 | (uint32_t __user *)p->write_ptr, | 166 | (uint32_t __user *)p->write_ptr, |
| 167 | wptr_shift, wptr_mask, mms); | 167 | wptr_shift, wptr_mask, mms); |
| 168 | } | 168 | } |
| @@ -265,7 +265,7 @@ static int destroy_mqd(struct mqd_manager *mm, void *mqd, · 96 unmodified lines | |||
| 265 | uint32_t queue_id) | 265 | uint32_t queue_id) |
| 266 | { | 266 | { |
| 267 | return mm->dev->kfd2kgd->hqd_destroy | 267 | return mm->dev->kfd2kgd->hqd_destroy |
| 268 | (mm->dev->kgd, mqd, type, timeout, | 268 | (mm->dev->adev, mqd, type, timeout, |
| 269 | pipe_id, queue_id); | 269 | pipe_id, queue_id); |
| 270 | } | 270 | } |
| 271 | 271 | ||
| @@ -280,7 +280,7 @@ static bool is_occupied(struct mqd_manager *mm, void *mqd, · 8 unmodified lines | |||
| 280 | uint32_t queue_id) | 280 | uint32_t queue_id) |
| 281 | { | 281 | { |
| 282 | return mm->dev->kfd2kgd->hqd_is_occupied( | 282 | return mm->dev->kfd2kgd->hqd_is_occupied( |
| 283 | mm->dev->kgd, queue_address, | 283 | mm->dev->adev, queue_address, |
| 284 | pipe_id, queue_id); | 284 | pipe_id, queue_id); |
| 285 | } | 285 | } |
| 286 | 286 | ||
| @@ -347,7 +347,7 @@ static int load_mqd_sdma(struct mqd_manager *mm, void *mqd, · 60 unmodified lines | |||
| 347 | uint32_t pipe_id, uint32_t queue_id, | 347 | uint32_t pipe_id, uint32_t queue_id, |
| 348 | struct queue_properties *p, struct mm_struct *mms) | 348 | struct queue_properties *p, struct mm_struct *mms) |
| 349 | { | 349 | { |
| 350 | return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->kgd, mqd, | 350 | return mm->dev->kfd2kgd->hqd_sdma_load(mm->dev->adev, mqd, |
| 351 | (uint32_t __user *)p->write_ptr, | 351 | (uint32_t __user *)p->write_ptr, |
| 352 | mms); | 352 | mms); |
| 353 | } | 353 | } |
| @@ -389,14 +389,14 @@ static int destroy_mqd_sdma(struct mqd_manager *mm, void *mqd, · 35 unmodified lines | |||
| 389 | unsigned int timeout, uint32_t pipe_id, | 389 | unsigned int timeout, uint32_t pipe_id, |
| 390 | uint32_t queue_id) | 390 | uint32_t queue_id) |
| 391 | { | 391 | { |
| 392 | return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->kgd, mqd, timeout); | 392 | return mm->dev->kfd2kgd->hqd_sdma_destroy(mm->dev->adev, mqd, timeout); |
| 393 | } | 393 | } |
| 394 | 394 | ||
| 395 | static bool is_occupied_sdma(struct mqd_manager *mm, void *mqd, | 395 | static bool is_occupied_sdma(struct mqd_manager *mm, void *mqd, |
| 396 | uint64_t queue_address, uint32_t pipe_id, | 396 | uint64_t queue_address, uint32_t pipe_id, |
| 397 | uint32_t queue_id) | 397 | uint32_t queue_id) |
| 398 | { | 398 | { |
| 399 | return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->kgd, mqd); | 399 | return mm->dev->kfd2kgd->hqd_sdma_is_occupied(mm->dev->adev, mqd); |
| 400 | } | 400 | } |
| 401 | 401 | ||
| 402 | #if defined(CONFIG_DEBUG_FS) | 402 | #if defined(CONFIG_DEBUG_FS) |
| @@ -223,7 +223,7 @@ static int pm_create_runlist_ib(struct packet_manager *pm, · 222 unmodified lines | |||
| 223 | 223 | ||
| 224 | int pm_init(struct packet_manager *pm, struct device_queue_manager *dqm) | 224 | int pm_init(struct packet_manager *pm, struct device_queue_manager *dqm) |
| 225 | { | 225 | { |
| 226 | switch (dqm->dev->device_info->asic_family) { | 226 | switch (dqm->dev->adev->asic_type) { |
| 227 | case CHIP_KAVERI: | 227 | case CHIP_KAVERI: |
| 228 | case CHIP_HAWAII: | 228 | case CHIP_HAWAII: |
| 229 | /* PM4 packet structures on CIK are the same as on VI */ | 229 | /* PM4 packet structures on CIK are the same as on VI */ |
| @@ -236,31 +236,16 @@ int pm_init(struct packet_manager *pm, struct device_queue_manager *dqm) · 6 unmodified lines | |||
| 236 | case CHIP_VEGAM: | 236 | case CHIP_VEGAM: |
| 237 | pm->pmf = &kfd_vi_pm_funcs; | 237 | pm->pmf = &kfd_vi_pm_funcs; |
| 238 | break; | 238 | break; |
| 239 | case CHIP_VEGA10: | ||
| 240 | case CHIP_VEGA12: | ||
| 241 | case CHIP_VEGA20: | ||
| 242 | case CHIP_RAVEN: | ||
| 243 | case CHIP_RENOIR: | ||
| 244 | case CHIP_ARCTURUS: | ||
| 245 | case CHIP_NAVI10: | ||
| 246 | case CHIP_NAVI12: | ||
| 247 | case CHIP_NAVI14: | ||
| 248 | case CHIP_SIENNA_CICHLID: | ||
| 249 | case CHIP_NAVY_FLOUNDER: | ||
| 250 | case CHIP_VANGOGH: | ||
| 251 | case CHIP_DIMGREY_CAVEFISH: | ||
| 252 | case CHIP_BEIGE_GOBY: | ||
| 253 | case CHIP_YELLOW_CARP: | ||
| 254 | case CHIP_CYAN_SKILLFISH: | ||
| 255 | pm->pmf = &kfd_v9_pm_funcs; | ||
| 256 | break; | ||
| 257 | case CHIP_ALDEBARAN: | ||
| 258 | pm->pmf = &kfd_aldebaran_pm_funcs; | ||
| 259 | break; | ||
| 260 | default: | 239 | default: |
| 261 | WARN(1, "Unexpected ASIC family %u", | 240 | if (KFD_GC_VERSION(dqm->dev) == IP_VERSION(9, 4, 2)) |
| 262 | dqm->dev->device_info->asic_family); | 241 | pm->pmf = &kfd_aldebaran_pm_funcs; |
| 263 | return -EINVAL; | 242 | else if (KFD_GC_VERSION(dqm->dev) >= IP_VERSION(9, 0, 1)) |
| 243 | pm->pmf = &kfd_v9_pm_funcs; | ||
| 244 | else { | ||
| 245 | WARN(1, "Unexpected ASIC family %u", | ||
| 246 | dqm->dev->adev->asic_type); | ||
| 247 | return -EINVAL; | ||
| 248 | } | ||
| 264 | } | 249 | } |
| 265 | 250 | ||
| 266 | pm->dqm = dqm; | 251 | pm->dqm = dqm; |
| @@ -110,8 +110,8 @@ static int pm_runlist_vi(struct packet_manager *pm, uint32_t *buffer, · 109 unmodified lines | |||
| 110 | return 0; | 110 | return 0; |
| 111 | } | 111 | } |
| 112 | 112 | ||
| 113 | int pm_set_resources_vi(struct packet_manager *pm, uint32_t *buffer, | 113 | static int pm_set_resources_vi(struct packet_manager *pm, uint32_t *buffer, |
| 114 | struct scheduling_resources *res) | 114 | struct scheduling_resources *res) |
| 115 | { | 115 | { |
| 116 | struct pm4_mes_set_resources *packet; | 116 | struct pm4_mes_set_resources *packet; |
| 117 | 117 | ||
| @@ -183,7 +183,8 @@ enum cache_policy { · 182 unmodified lines | |||
| 183 | cache_policy_noncoherent | 183 | cache_policy_noncoherent |
| 184 | }; | 184 | }; |
| 185 | 185 | ||
| 186 | #define KFD_IS_SOC15(chip) ((chip) >= CHIP_VEGA10) | 186 | #define KFD_GC_VERSION(dev) ((dev)->adev->ip_versions[GC_HWIP][0]) |
| 187 | #define KFD_IS_SOC15(dev) ((KFD_GC_VERSION(dev)) >= (IP_VERSION(9, 0, 1))) | ||
| 187 | 188 | ||
| 188 | struct kfd_event_interrupt_class { | 189 | struct kfd_event_interrupt_class { |
| 189 | bool (*interrupt_isr)(struct kfd_dev *dev, | 190 | bool (*interrupt_isr)(struct kfd_dev *dev, |
| @@ -194,8 +195,6 @@ struct kfd_event_interrupt_class { · 4 unmodified lines | |||
| 194 | }; | 195 | }; |
| 195 | 196 | ||
| 196 | struct kfd_device_info { | 197 | struct kfd_device_info { |
| 197 | enum amd_asic_type asic_family; | ||
| 198 | const char *asic_name; | ||
| 199 | uint32_t gfx_target_version; | 198 | uint32_t gfx_target_version; |
| 200 | const struct kfd_event_interrupt_class *event_interrupt_class; | 199 | const struct kfd_event_interrupt_class *event_interrupt_class; |
| 201 | unsigned int max_pasid_bits; | 200 | unsigned int max_pasid_bits; |
| @@ -208,11 +207,12 @@ struct kfd_device_info { · 6 unmodified lines | |||
| 208 | bool needs_iommu_device; | 207 | bool needs_iommu_device; |
| 209 | bool needs_pci_atomics; | 208 | bool needs_pci_atomics; |
| 210 | uint32_t no_atomic_fw_version; | 209 | uint32_t no_atomic_fw_version; |
| 211 | unsigned int num_sdma_engines; | ||
| 212 | unsigned int num_xgmi_sdma_engines; | ||
| 213 | unsigned int num_sdma_queues_per_engine; | 210 | unsigned int num_sdma_queues_per_engine; |
| 214 | }; | 211 | }; |
| 215 | 212 | ||
| 213 | unsigned int kfd_get_num_sdma_engines(struct kfd_dev *kdev); | ||
| 214 | unsigned int kfd_get_num_xgmi_sdma_engines(struct kfd_dev *kdev); | ||
| 215 | |||
| 216 | struct kfd_mem_obj { | 216 | struct kfd_mem_obj { |
| 217 | uint32_t range_start; | 217 | uint32_t range_start; |
| 218 | uint32_t range_end; | 218 | uint32_t range_end; |
| @@ -228,9 +228,9 @@ struct kfd_vmid_info { · 9 unmodified lines | |||
| 228 | }; | 228 | }; |
| 229 | 229 | ||
| 230 | struct kfd_dev { | 230 | struct kfd_dev { |
| 231 | struct kgd_dev *kgd; | 231 | struct amdgpu_device *adev; |
| 232 | 232 | ||
| 233 | const struct kfd_device_info *device_info; | 233 | struct kfd_device_info device_info; |
| 234 | struct pci_dev *pdev; | 234 | struct pci_dev *pdev; |
| 235 | struct drm_device *ddev; | 235 | struct drm_device *ddev; |
| 236 | 236 | ||
| @@ -856,6 +856,8 @@ struct kfd_process { · 619 unmodified lines | |||
| 856 | struct svm_range_list svms; | 856 | struct svm_range_list svms; |
| 857 | 857 | ||
| 858 | bool xnack_enabled; | 858 | bool xnack_enabled; |
| 859 | |||
| 860 | atomic_t poison; | ||
| 859 | }; | 861 | }; |
| 860 | 862 | ||
| 861 | #define KFD_PROCESS_TABLE_SIZE 5 /* bits: 32 entries */ | 863 | #define KFD_PROCESS_TABLE_SIZE 5 /* bits: 32 entries */ |
| @@ -891,7 +893,7 @@ struct kfd_process *kfd_lookup_process_by_pasid(u32 pasid); · 29 unmodified lines | |||
| 891 | struct kfd_process *kfd_lookup_process_by_mm(const struct mm_struct *mm); | 893 | struct kfd_process *kfd_lookup_process_by_mm(const struct mm_struct *mm); |
| 892 | 894 | ||
| 893 | int kfd_process_gpuidx_from_gpuid(struct kfd_process *p, uint32_t gpu_id); | 895 | int kfd_process_gpuidx_from_gpuid(struct kfd_process *p, uint32_t gpu_id); |
| 894 | int kfd_process_gpuid_from_kgd(struct kfd_process *p, | 896 | int kfd_process_gpuid_from_adev(struct kfd_process *p, |
| 895 | struct amdgpu_device *adev, uint32_t *gpuid, | 897 | struct amdgpu_device *adev, uint32_t *gpuid, |
| 896 | uint32_t *gpuidx); | 898 | uint32_t *gpuidx); |
| 897 | static inline int kfd_process_gpuid_from_gpuidx(struct kfd_process *p, | 899 | static inline int kfd_process_gpuid_from_gpuidx(struct kfd_process *p, |
| @@ -984,7 +986,7 @@ struct kfd_topology_device *kfd_topology_device_by_proximity_domain( · 86 unmodified lines | |||
| 984 | struct kfd_topology_device *kfd_topology_device_by_id(uint32_t gpu_id); | 986 | struct kfd_topology_device *kfd_topology_device_by_id(uint32_t gpu_id); |
| 985 | struct kfd_dev *kfd_device_by_id(uint32_t gpu_id); | 987 | struct kfd_dev *kfd_device_by_id(uint32_t gpu_id); |
| 986 | struct kfd_dev *kfd_device_by_pci_dev(const struct pci_dev *pdev); | 988 | struct kfd_dev *kfd_device_by_pci_dev(const struct pci_dev *pdev); |
| 987 | struct kfd_dev *kfd_device_by_kgd(const struct kgd_dev *kgd); | 989 | struct kfd_dev *kfd_device_by_adev(const struct amdgpu_device *adev); |
| 988 | int kfd_topology_enum_kfd_devices(uint8_t idx, struct kfd_dev **kdev); | 990 | int kfd_topology_enum_kfd_devices(uint8_t idx, struct kfd_dev **kdev); |
| 989 | int kfd_numa_node_to_apic_id(int numa_node_id); | 991 | int kfd_numa_node_to_apic_id(int numa_node_id); |
| 990 | void kfd_double_confirm_iommu_support(struct kfd_dev *gpu); | 992 | void kfd_double_confirm_iommu_support(struct kfd_dev *gpu); |
| @@ -251,14 +251,13 @@ cleanup: · 250 unmodified lines | |||
| 251 | } | 251 | } |
| 252 | 252 | ||
| 253 | /** | 253 | /** |
| 254 | * @kfd_get_cu_occupancy - Collect number of waves in-flight on this device | 254 | * kfd_get_cu_occupancy - Collect number of waves in-flight on this device |
| 255 | * by current process. Translates acquired wave count into number of compute units | 255 | * by current process. Translates acquired wave count into number of compute units |
| 256 | * that are occupied. | 256 | * that are occupied. |
| 257 | * | 257 | * |
| 258 | * @atr: Handle of attribute that allows reporting of wave count. The attribute | 258 | * @attr: Handle of attribute that allows reporting of wave count. The attribute |
| 259 | * handle encapsulates GPU device it is associated with, thereby allowing collection | 259 | * handle encapsulates GPU device it is associated with, thereby allowing collection |
| 260 | * of waves in flight, etc | 260 | * of waves in flight, etc |
| 261 | * | ||
| 262 | * @buffer: Handle of user provided buffer updated with wave count | 261 | * @buffer: Handle of user provided buffer updated with wave count |
| 263 | * | 262 | * |
| 264 | * Return: Number of bytes written to user buffer or an error value | 263 | * Return: Number of bytes written to user buffer or an error value |
| @@ -288,7 +287,7 @@ static int kfd_get_cu_occupancy(struct attribute *attr, char *buffer) · 23 unmodified lines | |||
| 288 | /* Collect wave count from device if it supports */ | 287 | /* Collect wave count from device if it supports */ |
| 289 | wave_cnt = 0; | 288 | wave_cnt = 0; |
| 290 | max_waves_per_cu = 0; | 289 | max_waves_per_cu = 0; |
| 291 | dev->kfd2kgd->get_cu_occupancy(dev->kgd, proc->pasid, &wave_cnt, | 290 | dev->kfd2kgd->get_cu_occupancy(dev->adev, proc->pasid, &wave_cnt, |
| 292 | &max_waves_per_cu); | 291 | &max_waves_per_cu); |
| 293 | 292 | ||
| 294 | /* Translate wave count to number of compute units */ | 293 | /* Translate wave count to number of compute units */ |
| @@ -692,12 +691,12 @@ static void kfd_process_free_gpuvm(struct kgd_mem *mem, · 397 unmodified lines | |||
| 692 | struct kfd_dev *dev = pdd->dev; | 691 | struct kfd_dev *dev = pdd->dev; |
| 693 | 692 | ||
| 694 | if (kptr) { | 693 | if (kptr) { |
| 695 | amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(dev->kgd, mem); | 694 | amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(dev->adev, mem); |
| 696 | kptr = NULL; | 695 | kptr = NULL; |
| 697 | } | 696 | } |
| 698 | 697 | ||
| 699 | amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(dev->kgd, mem, pdd->drm_priv); | 698 | amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(dev->adev, mem, pdd->drm_priv); |
| 700 | amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->kgd, mem, pdd->drm_priv, | 699 | amdgpu_amdkfd_gpuvm_free_memory_of_gpu(dev->adev, mem, pdd->drm_priv, |
| 701 | NULL); | 700 | NULL); |
| 702 | } | 701 | } |
| 703 | 702 | ||
| @@ -714,24 +713,24 @@ static int kfd_process_alloc_gpuvm(struct kfd_process_device *pdd, · 10 unmodified lines | |||
| 714 | struct kfd_dev *kdev = pdd->dev; | 713 | struct kfd_dev *kdev = pdd->dev; |
| 715 | int err; | 714 | int err; |
| 716 | 715 | ||
| 717 | err = amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(kdev->kgd, gpu_va, size, | 716 | err = amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(kdev->adev, gpu_va, size, |
| 718 | pdd->drm_priv, mem, NULL, flags); | 717 | pdd->drm_priv, mem, NULL, flags); |
| 719 | if (err) | 718 | if (err) |
| 720 | goto err_alloc_mem; | 719 | goto err_alloc_mem; |
| 721 | 720 | ||
| 722 | err = amdgpu_amdkfd_gpuvm_map_memory_to_gpu(kdev->kgd, *mem, | 721 | err = amdgpu_amdkfd_gpuvm_map_memory_to_gpu(kdev->adev, *mem, |
| 723 | pdd->drm_priv, NULL); | 722 | pdd->drm_priv, NULL); |
| 724 | if (err) | 723 | if (err) |
| 725 | goto err_map_mem; | 724 | goto err_map_mem; |
| 726 | 725 | ||
| 727 | err = amdgpu_amdkfd_gpuvm_sync_memory(kdev->kgd, *mem, true); | 726 | err = amdgpu_amdkfd_gpuvm_sync_memory(kdev->adev, *mem, true); |
| 728 | if (err) { | 727 | if (err) { |
| 729 | pr_debug("Sync memory failed, wait interrupted by user signal\n"); | 728 | pr_debug("Sync memory failed, wait interrupted by user signal\n"); |
| 730 | goto sync_memory_failed; | 729 | goto sync_memory_failed; |
| 731 | } | 730 | } |
| 732 | 731 | ||
| 733 | if (kptr) { | 732 | if (kptr) { |
| 734 | err = amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(kdev->kgd, | 733 | err = amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(kdev->adev, |
| 735 | (struct kgd_mem *)*mem, kptr, NULL); | 734 | (struct kgd_mem *)*mem, kptr, NULL); |
| 736 | if (err) { | 735 | if (err) { |
| 737 | pr_debug("Map GTT BO to kernel failed\n"); | 736 | pr_debug("Map GTT BO to kernel failed\n"); |
| @@ -742,10 +741,10 @@ static int kfd_process_alloc_gpuvm(struct kfd_process_device *pdd, · 4 unmodified lines | |||
| 742 | return err; | 741 | return err; |
| 743 | 742 | ||
| 744 | sync_memory_failed: | 743 | sync_memory_failed: |
| 745 | amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(kdev->kgd, *mem, pdd->drm_priv); | 744 | amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(kdev->adev, *mem, pdd->drm_priv); |
| 746 | 745 | ||
| 747 | err_map_mem: | 746 | err_map_mem: |
| 748 | amdgpu_amdkfd_gpuvm_free_memory_of_gpu(kdev->kgd, *mem, pdd->drm_priv, | 747 | amdgpu_amdkfd_gpuvm_free_memory_of_gpu(kdev->adev, *mem, pdd->drm_priv, |
| 749 | NULL); | 748 | NULL); |
| 750 | err_alloc_mem: | 749 | err_alloc_mem: |
| 751 | *mem = NULL; | 750 | *mem = NULL; |
| @@ -940,10 +939,10 @@ static void kfd_process_device_free_bos(struct kfd_process_device *pdd) · 188 unmodified lines | |||
| 940 | if (!peer_pdd->drm_priv) | 939 | if (!peer_pdd->drm_priv) |
| 941 | continue; | 940 | continue; |
| 942 | amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( | 941 | amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( |
| 943 | peer_pdd->dev->kgd, mem, peer_pdd->drm_priv); | 942 | peer_pdd->dev->adev, mem, peer_pdd->drm_priv); |
| 944 | } | 943 | } |
| 945 | 944 | ||
| 946 | amdgpu_amdkfd_gpuvm_free_memory_of_gpu(pdd->dev->kgd, mem, | 945 | amdgpu_amdkfd_gpuvm_free_memory_of_gpu(pdd->dev->adev, mem, |
| 947 | pdd->drm_priv, NULL); | 946 | pdd->drm_priv, NULL); |
| 948 | kfd_process_device_remove_obj_handle(pdd, id); | 947 | kfd_process_device_remove_obj_handle(pdd, id); |
| 949 | } | 948 | } |
| @@ -974,7 +973,7 @@ static void kfd_process_kunmap_signal_bo(struct kfd_process *p) · 24 unmodified lines | |||
| 974 | if (!mem) | 973 | if (!mem) |
| 975 | goto out; | 974 | goto out; |
| 976 | 975 | ||
| 977 | amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(kdev->kgd, mem); | 976 | amdgpu_amdkfd_gpuvm_unmap_gtt_bo_from_kernel(kdev->adev, mem); |
| 978 | 977 | ||
| 979 | out: | 978 | out: |
| 980 | mutex_unlock(&p->mutex); | 979 | mutex_unlock(&p->mutex); |
| @@ -1003,7 +1002,7 @@ static void kfd_process_destroy_pdds(struct kfd_process *p) · 22 unmodified lines | |||
| 1003 | 1002 | ||
| 1004 | if (pdd->drm_file) { | 1003 | if (pdd->drm_file) { |
| 1005 | amdgpu_amdkfd_gpuvm_release_process_vm( | 1004 | amdgpu_amdkfd_gpuvm_release_process_vm( |
| 1006 | pdd->dev->kgd, pdd->drm_priv); | 1005 | pdd->dev->adev, pdd->drm_priv); |
| 1007 | fput(pdd->drm_file); | 1006 | fput(pdd->drm_file); |
| 1008 | } | 1007 | } |
| 1009 | 1008 | ||
| @@ -1011,7 +1010,7 @@ static void kfd_process_destroy_pdds(struct kfd_process *p) · 1 unmodified line | |||
| 1011 | free_pages((unsigned long)pdd->qpd.cwsr_kaddr, | 1010 | free_pages((unsigned long)pdd->qpd.cwsr_kaddr, |
| 1012 | get_order(KFD_CWSR_TBA_TMA_SIZE)); | 1011 | get_order(KFD_CWSR_TBA_TMA_SIZE)); |
| 1013 | 1012 | ||
| 1014 | kfree(pdd->qpd.doorbell_bitmap); | 1013 | bitmap_free(pdd->qpd.doorbell_bitmap); |
| 1015 | idr_destroy(&pdd->alloc_idr); | 1014 | idr_destroy(&pdd->alloc_idr); |
| 1016 | 1015 | ||
| 1017 | kfd_free_process_doorbells(pdd->dev, pdd->doorbell_index); | 1016 | kfd_free_process_doorbells(pdd->dev, pdd->doorbell_index); |
| @@ -1317,14 +1316,13 @@ bool kfd_process_xnack_mode(struct kfd_process *p, bool supported) · 299 unmodified lines | |||
| 1317 | * support the SVM APIs and don't need to be considered | 1316 | * support the SVM APIs and don't need to be considered |
| 1318 | * for the XNACK mode selection. | 1317 | * for the XNACK mode selection. |
| 1319 | */ | 1318 | */ |
| 1320 | if (dev->device_info->asic_family < CHIP_VEGA10) | 1319 | if (!KFD_IS_SOC15(dev)) |
| 1321 | continue; | 1320 | continue; |
| 1322 | /* Aldebaran can always support XNACK because it can support | 1321 | /* Aldebaran can always support XNACK because it can support |
| 1323 | * per-process XNACK mode selection. But let the dev->noretry | 1322 | * per-process XNACK mode selection. But let the dev->noretry |
| 1324 | * setting still influence the default XNACK mode. | 1323 | * setting still influence the default XNACK mode. |
| 1325 | */ | 1324 | */ |
| 1326 | if (supported && | 1325 | if (supported && KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 2)) |
| 1327 | dev->device_info->asic_family == CHIP_ALDEBARAN) | ||
| 1328 | continue; | 1326 | continue; |
| 1329 | 1327 | ||
| 1330 | /* GFXv10 and later GPUs do not support shader preemption | 1328 | /* GFXv10 and later GPUs do not support shader preemption |
| @@ -1332,7 +1330,7 @@ bool kfd_process_xnack_mode(struct kfd_process *p, bool supported) · 1 unmodified line | |||
| 1332 | * management and memory-manager-related preemptions or | 1330 | * management and memory-manager-related preemptions or |
| 1333 | * even deadlocks. | 1331 | * even deadlocks. |
| 1334 | */ | 1332 | */ |
| 1335 | if (dev->device_info->asic_family >= CHIP_NAVI10) | 1333 | if (KFD_GC_VERSION(dev) >= IP_VERSION(10, 1, 1)) |
| 1336 | return false; | 1334 | return false; |
| 1337 | 1335 | ||
| 1338 | if (dev->noretry) | 1336 | if (dev->noretry) |
| @@ -1431,12 +1429,11 @@ static int init_doorbell_bitmap(struct qcm_process_device *qpd, · 92 unmodified lines | |||
| 1431 | int range_start = dev->shared_resources.non_cp_doorbells_start; | 1429 | int range_start = dev->shared_resources.non_cp_doorbells_start; |
| 1432 | int range_end = dev->shared_resources.non_cp_doorbells_end; | 1430 | int range_end = dev->shared_resources.non_cp_doorbells_end; |
| 1433 | 1431 | ||
| 1434 | if (!KFD_IS_SOC15(dev->device_info->asic_family)) | 1432 | if (!KFD_IS_SOC15(dev)) |
| 1435 | return 0; | 1433 | return 0; |
| 1436 | 1434 | ||
| 1437 | qpd->doorbell_bitmap = | 1435 | qpd->doorbell_bitmap = bitmap_zalloc(KFD_MAX_NUM_OF_QUEUES_PER_PROCESS, |
| 1438 | kzalloc(DIV_ROUND_UP(KFD_MAX_NUM_OF_QUEUES_PER_PROCESS, | 1436 | GFP_KERNEL); |
| 1439 | BITS_PER_BYTE), GFP_KERNEL); | ||
| 1440 | if (!qpd->doorbell_bitmap) | 1437 | if (!qpd->doorbell_bitmap) |
| 1441 | return -ENOMEM; | 1438 | return -ENOMEM; |
| 1442 | 1439 | ||
| @@ -1448,9 +1445,9 @@ static int init_doorbell_bitmap(struct qcm_process_device *qpd, · 5 unmodified lines | |||
| 1448 | 1445 | ||
| 1449 | for (i = 0; i < KFD_MAX_NUM_OF_QUEUES_PER_PROCESS / 2; i++) { | 1446 | for (i = 0; i < KFD_MAX_NUM_OF_QUEUES_PER_PROCESS / 2; i++) { |
| 1450 | if (i >= range_start && i <= range_end) { | 1447 | if (i >= range_start && i <= range_end) { |
| 1451 | set_bit(i, qpd->doorbell_bitmap); | 1448 | __set_bit(i, qpd->doorbell_bitmap); |
| 1452 | set_bit(i + KFD_QUEUE_DOORBELL_MIRROR_OFFSET, | 1449 | __set_bit(i + KFD_QUEUE_DOORBELL_MIRROR_OFFSET, |
| 1453 | qpd->doorbell_bitmap); | 1450 | qpd->doorbell_bitmap); |
| 1454 | } | 1451 | } |
| 1455 | } | 1452 | } |
| 1456 | 1453 | ||
| @@ -1547,7 +1544,7 @@ int kfd_process_device_init_vm(struct kfd_process_device *pdd, · 90 unmodified lines | |||
| 1547 | dev = pdd->dev; | 1544 | dev = pdd->dev; |
| 1548 | 1545 | ||
| 1549 | ret = amdgpu_amdkfd_gpuvm_acquire_process_vm( | 1546 | ret = amdgpu_amdkfd_gpuvm_acquire_process_vm( |
| 1550 | dev->kgd, drm_file, p->pasid, | 1547 | dev->adev, drm_file, p->pasid, |
| 1551 | &p->kgd_process_info, &p->ef); | 1548 | &p->kgd_process_info, &p->ef); |
| 1552 | if (ret) { | 1549 | if (ret) { |
| 1553 | pr_err("Failed to create process VM object\n"); | 1550 | pr_err("Failed to create process VM object\n"); |
| @@ -1779,14 +1776,13 @@ int kfd_process_gpuidx_from_gpuid(struct kfd_process *p, uint32_t gpu_id) · 225 unmodified lines | |||
| 1779 | } | 1776 | } |
| 1780 | 1777 | ||
| 1781 | int | 1778 | int |
| 1782 | kfd_process_gpuid_from_kgd(struct kfd_process *p, struct amdgpu_device *adev, | 1779 | kfd_process_gpuid_from_adev(struct kfd_process *p, struct amdgpu_device *adev, |
| 1783 | uint32_t *gpuid, uint32_t *gpuidx) | 1780 | uint32_t *gpuid, uint32_t *gpuidx) |
| 1784 | { | 1781 | { |
| 1785 | struct kgd_dev *kgd = (struct kgd_dev *)adev; | ||
| 1786 | int i; | 1782 | int i; |
| 1787 | 1783 | ||
| 1788 | for (i = 0; i < p->n_pdds; i++) | 1784 | for (i = 0; i < p->n_pdds; i++) |
| 1789 | if (p->pdds[i] && p->pdds[i]->dev->kgd == kgd) { | 1785 | if (p->pdds[i] && p->pdds[i]->dev->adev == adev) { |
| 1790 | *gpuid = p->pdds[i]->dev->id; | 1786 | *gpuid = p->pdds[i]->dev->id; |
| 1791 | *gpuidx = i; | 1787 | *gpuidx = i; |
| 1792 | return 0; | 1788 | return 0; |
| @@ -1951,10 +1947,10 @@ void kfd_flush_tlb(struct kfd_process_device *pdd, enum TLB_FLUSH_TYPE type) · 158 unmodified lines | |||
| 1951 | * only happens when the first queue is created. | 1947 | * only happens when the first queue is created. |
| 1952 | */ | 1948 | */ |
| 1953 | if (pdd->qpd.vmid) | 1949 | if (pdd->qpd.vmid) |
| 1954 | amdgpu_amdkfd_flush_gpu_tlb_vmid(dev->kgd, | 1950 | amdgpu_amdkfd_flush_gpu_tlb_vmid(dev->adev, |
| 1955 | pdd->qpd.vmid); | 1951 | pdd->qpd.vmid); |
| 1956 | } else { | 1952 | } else { |
| 1957 | amdgpu_amdkfd_flush_gpu_tlb_pasid(dev->kgd, | 1953 | amdgpu_amdkfd_flush_gpu_tlb_pasid(dev->adev, |
| 1958 | pdd->process->pasid, type); | 1954 | pdd->process->pasid, type); |
| 1959 | } | 1955 | } |
| 1960 | } | 1956 | } |
| @@ -118,7 +118,7 @@ int pqm_set_gws(struct process_queue_manager *pqm, unsigned int qid, · 117 unmodified lines | |||
| 118 | return ret; | 118 | return ret; |
| 119 | 119 | ||
| 120 | pqn->q->gws = mem; | 120 | pqn->q->gws = mem; |
| 121 | pdd->qpd.num_gws = gws ? amdgpu_amdkfd_get_num_gws(dev->kgd) : 0; | 121 | pdd->qpd.num_gws = gws ? dev->adev->gds.gws_size : 0; |
| 122 | 122 | ||
| 123 | return pqn->q->device->dqm->ops.update_queue(pqn->q->device->dqm, | 123 | return pqn->q->device->dqm->ops.update_queue(pqn->q->device->dqm, |
| 124 | pqn->q, NULL); | 124 | pqn->q, NULL); |
| @@ -135,9 +135,8 @@ void kfd_process_dequeue_from_all_devices(struct kfd_process *p) · 10 unmodified lines | |||
| 135 | int pqm_init(struct process_queue_manager *pqm, struct kfd_process *p) | 135 | int pqm_init(struct process_queue_manager *pqm, struct kfd_process *p) |
| 136 | { | 136 | { |
| 137 | INIT_LIST_HEAD(&pqm->queues); | 137 | INIT_LIST_HEAD(&pqm->queues); |
| 138 | pqm->queue_slot_bitmap = | 138 | pqm->queue_slot_bitmap = bitmap_zalloc(KFD_MAX_NUM_OF_QUEUES_PER_PROCESS, |
| 139 | kzalloc(DIV_ROUND_UP(KFD_MAX_NUM_OF_QUEUES_PER_PROCESS, | 139 | GFP_KERNEL); |
| 140 | BITS_PER_BYTE), GFP_KERNEL); | ||
| 141 | if (!pqm->queue_slot_bitmap) | 140 | if (!pqm->queue_slot_bitmap) |
| 142 | return -ENOMEM; | 141 | return -ENOMEM; |
| 143 | pqm->process = p; | 142 | pqm->process = p; |
| @@ -159,7 +158,7 @@ void pqm_uninit(struct process_queue_manager *pqm) · 15 unmodified lines | |||
| 159 | kfree(pqn); | 158 | kfree(pqn); |
| 160 | } | 159 | } |
| 161 | 160 | ||
| 162 | kfree(pqm->queue_slot_bitmap); | 161 | bitmap_free(pqm->queue_slot_bitmap); |
| 163 | pqm->queue_slot_bitmap = NULL; | 162 | pqm->queue_slot_bitmap = NULL; |
| 164 | } | 163 | } |
| 165 | 164 | ||
| @@ -220,7 +219,7 @@ int pqm_create_queue(struct process_queue_manager *pqm, · 54 unmodified lines | |||
| 220 | * Hence we also check the type as well | 219 | * Hence we also check the type as well |
| 221 | */ | 220 | */ |
| 222 | if ((pdd->qpd.is_debug) || (type == KFD_QUEUE_TYPE_DIQ)) | 221 | if ((pdd->qpd.is_debug) || (type == KFD_QUEUE_TYPE_DIQ)) |
| 223 | max_queues = dev->device_info->max_no_of_hqd/2; | 222 | max_queues = dev->device_info.max_no_of_hqd/2; |
| 224 | 223 | ||
| 225 | if (pdd->qpd.queue_count >= max_queues) | 224 | if (pdd->qpd.queue_count >= max_queues) |
| 226 | return -ENOSPC; | 225 | return -ENOSPC; |
| @@ -207,7 +207,6 @@ void kfd_smi_event_update_gpu_reset(struct kfd_dev *dev, bool post_reset) · 206 unmodified lines | |||
| 207 | void kfd_smi_event_update_thermal_throttling(struct kfd_dev *dev, | 207 | void kfd_smi_event_update_thermal_throttling(struct kfd_dev *dev, |
| 208 | uint64_t throttle_bitmask) | 208 | uint64_t throttle_bitmask) |
| 209 | { | 209 | { |
| 210 | struct amdgpu_device *adev = (struct amdgpu_device *)dev->kgd; | ||
| 211 | /* | 210 | /* |
| 212 | * ThermalThrottle msg = throttle_bitmask(8): | 211 | * ThermalThrottle msg = throttle_bitmask(8): |
| 213 | * thermal_interrupt_count(16): | 212 | * thermal_interrupt_count(16): |
| @@ -223,14 +222,13 @@ void kfd_smi_event_update_thermal_throttling(struct kfd_dev *dev, · 9 unmodified lines | |||
| 223 | 222 | ||
| 224 | len = snprintf(fifo_in, sizeof(fifo_in), "%x %llx:%llx\n", | 223 | len = snprintf(fifo_in, sizeof(fifo_in), "%x %llx:%llx\n", |
| 225 | KFD_SMI_EVENT_THERMAL_THROTTLE, throttle_bitmask, | 224 | KFD_SMI_EVENT_THERMAL_THROTTLE, throttle_bitmask, |
| 226 | atomic64_read(&adev->smu.throttle_int_counter)); | 225 | atomic64_read(&dev->adev->smu.throttle_int_counter)); |
| 227 | 226 | ||
| 228 | add_event_to_kfifo(dev, KFD_SMI_EVENT_THERMAL_THROTTLE, fifo_in, len); | 227 | add_event_to_kfifo(dev, KFD_SMI_EVENT_THERMAL_THROTTLE, fifo_in, len); |
| 229 | } | 228 | } |
| 230 | 229 | ||
| 231 | void kfd_smi_event_update_vmfault(struct kfd_dev *dev, uint16_t pasid) | 230 | void kfd_smi_event_update_vmfault(struct kfd_dev *dev, uint16_t pasid) |
| 232 | { | 231 | { |
| 233 | struct amdgpu_device *adev = (struct amdgpu_device *)dev->kgd; | ||
| 234 | struct amdgpu_task_info task_info; | 232 | struct amdgpu_task_info task_info; |
| 235 | /* VmFault msg = (hex)uint32_pid(8) + :(1) + task name(16) = 25 */ | 233 | /* VmFault msg = (hex)uint32_pid(8) + :(1) + task name(16) = 25 */ |
| 236 | /* 1 byte event + 1 byte space + 25 bytes msg + 1 byte \n + | 234 | /* 1 byte event + 1 byte space + 25 bytes msg + 1 byte \n + |
| @@ -243,7 +241,7 @@ void kfd_smi_event_update_vmfault(struct kfd_dev *dev, uint16_t pasid) · 6 unmodified lines | |||
| 243 | return; | 241 | return; |
| 244 | 242 | ||
| 245 | memset(&task_info, 0, sizeof(struct amdgpu_task_info)); | 243 | memset(&task_info, 0, sizeof(struct amdgpu_task_info)); |
| 246 | amdgpu_vm_get_task_info(adev, pasid, &task_info); | 244 | amdgpu_vm_get_task_info(dev->adev, pasid, &task_info); |
| 247 | /* Report VM faults from user applications, not retry from kernel */ | 245 | /* Report VM faults from user applications, not retry from kernel */ |
| 248 | if (!task_info.pid) | 246 | if (!task_info.pid) |
| 249 | return; | 247 | return; |
| @@ -193,7 +193,6 @@ svm_range_dma_map(struct svm_range *prange, unsigned long *bitmap, · 192 unmodified lines | |||
| 193 | 193 | ||
| 194 | for_each_set_bit(gpuidx, bitmap, MAX_GPU_INSTANCE) { | 194 | for_each_set_bit(gpuidx, bitmap, MAX_GPU_INSTANCE) { |
| 195 | struct kfd_process_device *pdd; | 195 | struct kfd_process_device *pdd; |
| 196 | struct amdgpu_device *adev; | ||
| 197 | 196 | ||
| 198 | pr_debug("mapping to gpu idx 0x%x\n", gpuidx); | 197 | pr_debug("mapping to gpu idx 0x%x\n", gpuidx); |
| 199 | pdd = kfd_process_device_from_gpuidx(p, gpuidx); | 198 | pdd = kfd_process_device_from_gpuidx(p, gpuidx); |
| @@ -201,9 +200,8 @@ svm_range_dma_map(struct svm_range *prange, unsigned long *bitmap, · 1 unmodified line | |||
| 201 | pr_debug("failed to find device idx %d\n", gpuidx); | 200 | pr_debug("failed to find device idx %d\n", gpuidx); |
| 202 | return -EINVAL; | 201 | return -EINVAL; |
| 203 | } | 202 | } |
| 204 | adev = (struct amdgpu_device *)pdd->dev->kgd; | ||
| 205 | 203 | ||
| 206 | r = svm_range_dma_map_dev(adev, prange, offset, npages, | 204 | r = svm_range_dma_map_dev(pdd->dev->adev, prange, offset, npages, |
| 207 | hmm_pfns, gpuidx); | 205 | hmm_pfns, gpuidx); |
| 208 | if (r) | 206 | if (r) |
| 209 | break; | 207 | break; |
| @@ -334,6 +332,8 @@ static void svm_range_bo_release(struct kref *kref) · 124 unmodified lines | |||
| 334 | struct svm_range_bo *svm_bo; | 332 | struct svm_range_bo *svm_bo; |
| 335 | 333 | ||
| 336 | svm_bo = container_of(kref, struct svm_range_bo, kref); | 334 | svm_bo = container_of(kref, struct svm_range_bo, kref); |
| 335 | pr_debug("svm_bo 0x%p\n", svm_bo); | ||
| 336 | |||
| 337 | spin_lock(&svm_bo->list_lock); | 337 | spin_lock(&svm_bo->list_lock); |
| 338 | while (!list_empty(&svm_bo->range_list)) { | 338 | while (!list_empty(&svm_bo->range_list)) { |
| 339 | struct svm_range *prange = | 339 | struct svm_range *prange = |
| @@ -367,12 +367,33 @@ static void svm_range_bo_release(struct kref *kref) · 27 unmodified lines | |||
| 367 | kfree(svm_bo); | 367 | kfree(svm_bo); |
| 368 | } | 368 | } |
| 369 | 369 | ||
| 370 | void svm_range_bo_unref(struct svm_range_bo *svm_bo) | 370 | static void svm_range_bo_wq_release(struct work_struct *work) |
| 371 | { | 371 | { |
| 372 | if (!svm_bo) | 372 | struct svm_range_bo *svm_bo; |
| 373 | return; | 373 | |
| 374 | svm_bo = container_of(work, struct svm_range_bo, release_work); | ||
| 375 | svm_range_bo_release(&svm_bo->kref); | ||
| 376 | } | ||
| 377 | |||
| 378 | static void svm_range_bo_release_async(struct kref *kref) | ||
| 379 | { | ||
| 380 | struct svm_range_bo *svm_bo; | ||
| 374 | 381 | ||
| 375 | kref_put(&svm_bo->kref, svm_range_bo_release); | 382 | svm_bo = container_of(kref, struct svm_range_bo, kref); |
| 383 | pr_debug("svm_bo 0x%p\n", svm_bo); | ||
| 384 | INIT_WORK(&svm_bo->release_work, svm_range_bo_wq_release); | ||
| 385 | schedule_work(&svm_bo->release_work); | ||
| 386 | } | ||
| 387 | |||
| 388 | void svm_range_bo_unref_async(struct svm_range_bo *svm_bo) | ||
| 389 | { | ||
| 390 | kref_put(&svm_bo->kref, svm_range_bo_release_async); | ||
| 391 | } | ||
| 392 | |||
| 393 | static void svm_range_bo_unref(struct svm_range_bo *svm_bo) | ||
| 394 | { | ||
| 395 | if (svm_bo) | ||
| 396 | kref_put(&svm_bo->kref, svm_range_bo_release); | ||
| 376 | } | 397 | } |
| 377 | 398 | ||
| 378 | static bool | 399 | static bool |
| @@ -581,7 +602,7 @@ svm_range_get_adev_by_id(struct svm_range *prange, uint32_t gpu_id) · 202 unmodified lines | |||
| 581 | return NULL; | 602 | return NULL; |
| 582 | } | 603 | } |
| 583 | 604 | ||
| 584 | return (struct amdgpu_device *)pdd->dev->kgd; | 605 | return pdd->dev->adev; |
| 585 | } | 606 | } |
| 586 | 607 | ||
| 587 | struct kfd_process_device * | 608 | struct kfd_process_device * |
| @@ -593,7 +614,7 @@ svm_range_get_pdd_by_adev(struct svm_range *prange, struct amdgpu_device *adev) · 5 unmodified lines | |||
| 593 | 614 | ||
| 594 | p = container_of(prange->svms, struct kfd_process, svms); | 615 | p = container_of(prange->svms, struct kfd_process, svms); |
| 595 | 616 | ||
| 596 | r = kfd_process_gpuid_from_kgd(p, adev, &gpuid, &gpu_idx); | 617 | r = kfd_process_gpuid_from_adev(p, adev, &gpuid, &gpu_idx); |
| 597 | if (r) { | 618 | if (r) { |
| 598 | pr_debug("failed to get device id by adev %p\n", adev); | 619 | pr_debug("failed to get device id by adev %p\n", adev); |
| 599 | return NULL; | 620 | return NULL; |
| @@ -706,6 +727,61 @@ svm_range_apply_attrs(struct kfd_process *p, struct svm_range *prange, · 106 unmodified lines | |||
| 706 | } | 727 | } |
| 707 | } | 728 | } |
| 708 | 729 | ||
| 730 | static bool | ||
| 731 | svm_range_is_same_attrs(struct kfd_process *p, struct svm_range *prange, | ||
| 732 | uint32_t nattr, struct kfd_ioctl_svm_attribute *attrs) | ||
| 733 | { | ||
| 734 | uint32_t i; | ||
| 735 | int gpuidx; | ||
| 736 | |||
| 737 | for (i = 0; i < nattr; i++) { | ||
| 738 | switch (attrs[i].type) { | ||
| 739 | case KFD_IOCTL_SVM_ATTR_PREFERRED_LOC: | ||
| 740 | if (prange->preferred_loc != attrs[i].value) | ||
| 741 | return false; | ||
| 742 | break; | ||
| 743 | case KFD_IOCTL_SVM_ATTR_PREFETCH_LOC: | ||
| 744 | /* Prefetch should always trigger a migration even | ||
| 745 | * if the value of the attribute didn't change. | ||
| 746 | */ | ||
| 747 | return false; | ||
| 748 | case KFD_IOCTL_SVM_ATTR_ACCESS: | ||
| 749 | case KFD_IOCTL_SVM_ATTR_ACCESS_IN_PLACE: | ||
| 750 | case KFD_IOCTL_SVM_ATTR_NO_ACCESS: | ||
| 751 | gpuidx = kfd_process_gpuidx_from_gpuid(p, | ||
| 752 | attrs[i].value); | ||
| 753 | if (attrs[i].type == KFD_IOCTL_SVM_ATTR_NO_ACCESS) { | ||
| 754 | if (test_bit(gpuidx, prange->bitmap_access) || | ||
| 755 | test_bit(gpuidx, prange->bitmap_aip)) | ||
| 756 | return false; | ||
| 757 | } else if (attrs[i].type == KFD_IOCTL_SVM_ATTR_ACCESS) { | ||
| 758 | if (!test_bit(gpuidx, prange->bitmap_access)) | ||
| 759 | return false; | ||
| 760 | } else { | ||
| 761 | if (!test_bit(gpuidx, prange->bitmap_aip)) | ||
| 762 | return false; | ||
| 763 | } | ||
| 764 | break; | ||
| 765 | case KFD_IOCTL_SVM_ATTR_SET_FLAGS: | ||
| 766 | if ((prange->flags & attrs[i].value) != attrs[i].value) | ||
| 767 | return false; | ||
| 768 | break; | ||
| 769 | case KFD_IOCTL_SVM_ATTR_CLR_FLAGS: | ||
| 770 | if ((prange->flags & attrs[i].value) != 0) | ||
| 771 | return false; | ||
| 772 | break; | ||
| 773 | case KFD_IOCTL_SVM_ATTR_GRANULARITY: | ||
| 774 | if (prange->granularity != attrs[i].value) | ||
| 775 | return false; | ||
| 776 | break; | ||
| 777 | default: | ||
| 778 | WARN_ONCE(1, "svm_range_check_attrs wasn't called?"); | ||
| 779 | } | ||
| 780 | } | ||
| 781 | |||
| 782 | return true; | ||
| 783 | } | ||
| 784 | |||
| 709 | /** | 785 | /** |
| 710 | * svm_range_debug_dump - print all range information from svms | 786 | * svm_range_debug_dump - print all range information from svms |
| 711 | * @svms: svm range list header | 787 | * @svms: svm range list header |
| @@ -743,14 +819,6 @@ static void svm_range_debug_dump(struct svm_range_list *svms) · 31 unmodified lines | |||
| 743 | } | 819 | } |
| 744 | } | 820 | } |
| 745 | 821 | ||
| 746 | static bool | ||
| 747 | svm_range_is_same_attrs(struct svm_range *old, struct svm_range *new) | ||
| 748 | { | ||
| 749 | return (old->prefetch_loc == new->prefetch_loc && | ||
| 750 | old->flags == new->flags && | ||
| 751 | old->granularity == new->granularity); | ||
| 752 | } | ||
| 753 | |||
| 754 | static int | 822 | static int |
| 755 | svm_range_split_array(void *ppnew, void *ppold, size_t size, | 823 | svm_range_split_array(void *ppnew, void *ppold, size_t size, |
| 756 | uint64_t old_start, uint64_t old_n, | 824 | uint64_t old_start, uint64_t old_n, |
| @@ -943,7 +1011,7 @@ svm_range_split(struct svm_range *prange, uint64_t start, uint64_t last, · 186 unmodified lines | |||
| 943 | } | 1011 | } |
| 944 | 1012 | ||
| 945 | static int | 1013 | static int |
| 946 | svm_range_split_tail(struct svm_range *prange, struct svm_range *new, | 1014 | svm_range_split_tail(struct svm_range *prange, |
| 947 | uint64_t new_last, struct list_head *insert_list) | 1015 | uint64_t new_last, struct list_head *insert_list) |
| 948 | { | 1016 | { |
| 949 | struct svm_range *tail; | 1017 | struct svm_range *tail; |
| @@ -955,7 +1023,7 @@ svm_range_split_tail(struct svm_range *prange, struct svm_range *new, · 5 unmodified lines | |||
| 955 | } | 1023 | } |
| 956 | 1024 | ||
| 957 | static int | 1025 | static int |
| 958 | svm_range_split_head(struct svm_range *prange, struct svm_range *new, | 1026 | svm_range_split_head(struct svm_range *prange, |
| 959 | uint64_t new_start, struct list_head *insert_list) | 1027 | uint64_t new_start, struct list_head *insert_list) |
| 960 | { | 1028 | { |
| 961 | struct svm_range *head; | 1029 | struct svm_range *head; |
| @@ -1053,8 +1121,8 @@ svm_range_get_pte_flags(struct amdgpu_device *adev, struct svm_range *prange, · 91 unmodified lines | |||
| 1053 | if (domain == SVM_RANGE_VRAM_DOMAIN) | 1121 | if (domain == SVM_RANGE_VRAM_DOMAIN) |
| 1054 | bo_adev = amdgpu_ttm_adev(prange->svm_bo->bo->tbo.bdev); | 1122 | bo_adev = amdgpu_ttm_adev(prange->svm_bo->bo->tbo.bdev); |
| 1055 | 1123 | ||
| 1056 | switch (adev->asic_type) { | 1124 | switch (KFD_GC_VERSION(adev->kfd.dev)) { |
| 1057 | case CHIP_ARCTURUS: | 1125 | case IP_VERSION(9, 4, 1): |
| 1058 | if (domain == SVM_RANGE_VRAM_DOMAIN) { | 1126 | if (domain == SVM_RANGE_VRAM_DOMAIN) { |
| 1059 | if (bo_adev == adev) { | 1127 | if (bo_adev == adev) { |
| 1060 | mapping_flags |= coherent ? | 1128 | mapping_flags |= coherent ? |
| @@ -1070,7 +1138,7 @@ svm_range_get_pte_flags(struct amdgpu_device *adev, struct svm_range *prange, · 9 unmodified lines | |||
| 1070 | AMDGPU_VM_MTYPE_UC : AMDGPU_VM_MTYPE_NC; | 1138 | AMDGPU_VM_MTYPE_UC : AMDGPU_VM_MTYPE_NC; |
| 1071 | } | 1139 | } |
| 1072 | break; | 1140 | break; |
| 1073 | case CHIP_ALDEBARAN: | 1141 | case IP_VERSION(9, 4, 2): |
| 1074 | if (domain == SVM_RANGE_VRAM_DOMAIN) { | 1142 | if (domain == SVM_RANGE_VRAM_DOMAIN) { |
| 1075 | if (bo_adev == adev) { | 1143 | if (bo_adev == adev) { |
| 1076 | mapping_flags |= coherent ? | 1144 | mapping_flags |= coherent ? |
| @@ -1129,7 +1197,6 @@ svm_range_unmap_from_gpus(struct svm_range *prange, unsigned long start, · 52 unmodified lines | |||
| 1129 | DECLARE_BITMAP(bitmap, MAX_GPU_INSTANCE); | 1197 | DECLARE_BITMAP(bitmap, MAX_GPU_INSTANCE); |
| 1130 | struct kfd_process_device *pdd; | 1198 | struct kfd_process_device *pdd; |
| 1131 | struct dma_fence *fence = NULL; | 1199 | struct dma_fence *fence = NULL; |
| 1132 | struct amdgpu_device *adev; | ||
| 1133 | struct kfd_process *p; | 1200 | struct kfd_process *p; |
| 1134 | uint32_t gpuidx; | 1201 | uint32_t gpuidx; |
| 1135 | int r = 0; | 1202 | int r = 0; |
| @@ -1145,9 +1212,9 @@ svm_range_unmap_from_gpus(struct svm_range *prange, unsigned long start, · 9 unmodified lines | |||
| 1145 | pr_debug("failed to find device idx %d\n", gpuidx); | 1212 | pr_debug("failed to find device idx %d\n", gpuidx); |
| 1146 | return -EINVAL; | 1213 | return -EINVAL; |
| 1147 | } | 1214 | } |
| 1148 | adev = (struct amdgpu_device *)pdd->dev->kgd; | ||
| 1149 | 1215 | ||
| 1150 | r = svm_range_unmap_from_gpu(adev, drm_priv_to_vm(pdd->drm_priv), | 1216 | r = svm_range_unmap_from_gpu(pdd->dev->adev, |
| 1217 | drm_priv_to_vm(pdd->drm_priv), | ||
| 1151 | start, last, &fence); | 1218 | start, last, &fence); |
| 1152 | if (r) | 1219 | if (r) |
| 1153 | break; | 1220 | break; |
| @@ -1159,7 +1226,7 @@ svm_range_unmap_from_gpus(struct svm_range *prange, unsigned long start, · 5 unmodified lines | |||
| 1159 | if (r) | 1226 | if (r) |
| 1160 | break; | 1227 | break; |
| 1161 | } | 1228 | } |
| 1162 | amdgpu_amdkfd_flush_gpu_tlb_pasid((struct kgd_dev *)adev, | 1229 | amdgpu_amdkfd_flush_gpu_tlb_pasid(pdd->dev->adev, |
| 1163 | p->pasid, TLB_FLUSH_HEAVYWEIGHT); | 1230 | p->pasid, TLB_FLUSH_HEAVYWEIGHT); |
| 1164 | } | 1231 | } |
| 1165 | 1232 | ||
| @@ -1172,7 +1239,6 @@ svm_range_map_to_gpu(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 6 unmodified lines | |||
| 1172 | unsigned long npages, bool readonly, dma_addr_t *dma_addr, | 1239 | unsigned long npages, bool readonly, dma_addr_t *dma_addr, |
| 1173 | struct amdgpu_device *bo_adev, struct dma_fence **fence) | 1240 | struct amdgpu_device *bo_adev, struct dma_fence **fence) |
| 1174 | { | 1241 | { |
| 1175 | struct amdgpu_bo_va bo_va; | ||
| 1176 | bool table_freed = false; | 1242 | bool table_freed = false; |
| 1177 | uint64_t pte_flags; | 1243 | uint64_t pte_flags; |
| 1178 | unsigned long last_start; | 1244 | unsigned long last_start; |
| @@ -1185,9 +1251,6 @@ svm_range_map_to_gpu(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 6 unmodified lines | |||
| 1185 | pr_debug("svms 0x%p [0x%lx 0x%lx] readonly %d\n", prange->svms, | 1251 | pr_debug("svms 0x%p [0x%lx 0x%lx] readonly %d\n", prange->svms, |
| 1186 | last_start, last_start + npages - 1, readonly); | 1252 | last_start, last_start + npages - 1, readonly); |
| 1187 | 1253 | ||
| 1188 | if (prange->svm_bo && prange->ttm_res) | ||
| 1189 | bo_va.is_xgmi = amdgpu_xgmi_same_hive(adev, bo_adev); | ||
| 1190 | |||
| 1191 | for (i = offset; i < offset + npages; i++) { | 1254 | for (i = offset; i < offset + npages; i++) { |
| 1192 | last_domain = dma_addr[i] & SVM_RANGE_VRAM_DOMAIN; | 1255 | last_domain = dma_addr[i] & SVM_RANGE_VRAM_DOMAIN; |
| 1193 | dma_addr[i] &= ~SVM_RANGE_VRAM_DOMAIN; | 1256 | dma_addr[i] &= ~SVM_RANGE_VRAM_DOMAIN; |
| @@ -1243,8 +1306,7 @@ svm_range_map_to_gpu(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 49 unmodified lines | |||
| 1243 | struct kfd_process *p; | 1306 | struct kfd_process *p; |
| 1244 | 1307 | ||
| 1245 | p = container_of(prange->svms, struct kfd_process, svms); | 1308 | p = container_of(prange->svms, struct kfd_process, svms); |
| 1246 | amdgpu_amdkfd_flush_gpu_tlb_pasid((struct kgd_dev *)adev, | 1309 | amdgpu_amdkfd_flush_gpu_tlb_pasid(adev, p->pasid, TLB_FLUSH_LEGACY); |
| 1247 | p->pasid, TLB_FLUSH_LEGACY); | ||
| 1248 | } | 1310 | } |
| 1249 | out: | 1311 | out: |
| 1250 | return r; | 1312 | return r; |
| @@ -1257,7 +1319,6 @@ svm_range_map_to_gpus(struct svm_range *prange, unsigned long offset, · 6 unmodified lines | |||
| 1257 | { | 1319 | { |
| 1258 | struct kfd_process_device *pdd; | 1320 | struct kfd_process_device *pdd; |
| 1259 | struct amdgpu_device *bo_adev; | 1321 | struct amdgpu_device *bo_adev; |
| 1260 | struct amdgpu_device *adev; | ||
| 1261 | struct kfd_process *p; | 1322 | struct kfd_process *p; |
| 1262 | struct dma_fence *fence = NULL; | 1323 | struct dma_fence *fence = NULL; |
| 1263 | uint32_t gpuidx; | 1324 | uint32_t gpuidx; |
| @@ -1276,19 +1337,18 @@ svm_range_map_to_gpus(struct svm_range *prange, unsigned long offset, · 12 unmodified lines | |||
| 1276 | pr_debug("failed to find device idx %d\n", gpuidx); | 1337 | pr_debug("failed to find device idx %d\n", gpuidx); |
| 1277 | return -EINVAL; | 1338 | return -EINVAL; |
| 1278 | } | 1339 | } |
| 1279 | adev = (struct amdgpu_device *)pdd->dev->kgd; | ||
| 1280 | 1340 | ||
| 1281 | pdd = kfd_bind_process_to_device(pdd->dev, p); | 1341 | pdd = kfd_bind_process_to_device(pdd->dev, p); |
| 1282 | if (IS_ERR(pdd)) | 1342 | if (IS_ERR(pdd)) |
| 1283 | return -EINVAL; | 1343 | return -EINVAL; |
| 1284 | 1344 | ||
| 1285 | if (bo_adev && adev != bo_adev && | 1345 | if (bo_adev && pdd->dev->adev != bo_adev && |
| 1286 | !amdgpu_xgmi_same_hive(adev, bo_adev)) { | 1346 | !amdgpu_xgmi_same_hive(pdd->dev->adev, bo_adev)) { |
| 1287 | pr_debug("cannot map to device idx %d\n", gpuidx); | 1347 | pr_debug("cannot map to device idx %d\n", gpuidx); |
| 1288 | continue; | 1348 | continue; |
| 1289 | } | 1349 | } |
| 1290 | 1350 | ||
| 1291 | r = svm_range_map_to_gpu(adev, drm_priv_to_vm(pdd->drm_priv), | 1351 | r = svm_range_map_to_gpu(pdd->dev->adev, drm_priv_to_vm(pdd->drm_priv), |
| 1292 | prange, offset, npages, readonly, | 1352 | prange, offset, npages, readonly, |
| 1293 | prange->dma_addr[gpuidx], | 1353 | prange->dma_addr[gpuidx], |
| 1294 | bo_adev, wait ? &fence : NULL); | 1354 | bo_adev, wait ? &fence : NULL); |
| @@ -1322,7 +1382,6 @@ struct svm_validate_context { · 27 unmodified lines | |||
| 1322 | static int svm_range_reserve_bos(struct svm_validate_context *ctx) | 1382 | static int svm_range_reserve_bos(struct svm_validate_context *ctx) |
| 1323 | { | 1383 | { |
| 1324 | struct kfd_process_device *pdd; | 1384 | struct kfd_process_device *pdd; |
| 1325 | struct amdgpu_device *adev; | ||
| 1326 | struct amdgpu_vm *vm; | 1385 | struct amdgpu_vm *vm; |
| 1327 | uint32_t gpuidx; | 1386 | uint32_t gpuidx; |
| 1328 | int r; | 1387 | int r; |
| @@ -1334,7 +1393,6 @@ static int svm_range_reserve_bos(struct svm_validate_context *ctx) · 5 unmodified lines | |||
| 1334 | pr_debug("failed to find device idx %d\n", gpuidx); | 1393 | pr_debug("failed to find device idx %d\n", gpuidx); |
| 1335 | return -EINVAL; | 1394 | return -EINVAL; |
| 1336 | } | 1395 | } |
| 1337 | adev = (struct amdgpu_device *)pdd->dev->kgd; | ||
| 1338 | vm = drm_priv_to_vm(pdd->drm_priv); | 1396 | vm = drm_priv_to_vm(pdd->drm_priv); |
| 1339 | 1397 | ||
| 1340 | ctx->tv[gpuidx].bo = &vm->root.bo->tbo; | 1398 | ctx->tv[gpuidx].bo = &vm->root.bo->tbo; |
| @@ -1356,9 +1414,9 @@ static int svm_range_reserve_bos(struct svm_validate_context *ctx) · 15 unmodified lines | |||
| 1356 | r = -EINVAL; | 1414 | r = -EINVAL; |
| 1357 | goto unreserve_out; | 1415 | goto unreserve_out; |
| 1358 | } | 1416 | } |
| 1359 | adev = (struct amdgpu_device *)pdd->dev->kgd; | ||
| 1360 | 1417 | ||
| 1361 | r = amdgpu_vm_validate_pt_bos(adev, drm_priv_to_vm(pdd->drm_priv), | 1418 | r = amdgpu_vm_validate_pt_bos(pdd->dev->adev, |
| 1419 | drm_priv_to_vm(pdd->drm_priv), | ||
| 1362 | svm_range_bo_validate, NULL); | 1420 | svm_range_bo_validate, NULL); |
| 1363 | if (r) { | 1421 | if (r) { |
| 1364 | pr_debug("failed %d validate pt bos\n", r); | 1422 | pr_debug("failed %d validate pt bos\n", r); |
| @@ -1381,12 +1439,10 @@ static void svm_range_unreserve_bos(struct svm_validate_context *ctx) · 16 unmodified lines | |||
| 1381 | static void *kfd_svm_page_owner(struct kfd_process *p, int32_t gpuidx) | 1439 | static void *kfd_svm_page_owner(struct kfd_process *p, int32_t gpuidx) |
| 1382 | { | 1440 | { |
| 1383 | struct kfd_process_device *pdd; | 1441 | struct kfd_process_device *pdd; |
| 1384 | struct amdgpu_device *adev; | ||
| 1385 | 1442 | ||
| 1386 | pdd = kfd_process_device_from_gpuidx(p, gpuidx); | 1443 | pdd = kfd_process_device_from_gpuidx(p, gpuidx); |
| 1387 | adev = (struct amdgpu_device *)pdd->dev->kgd; | ||
| 1388 | 1444 | ||
| 1389 | return SVM_ADEV_PGMAP_OWNER(adev); | 1445 | return SVM_ADEV_PGMAP_OWNER(pdd->dev->adev); |
| 1390 | } | 1446 | } |
| 1391 | 1447 | ||
| 1392 | /* | 1448 | /* |
| @@ -1660,6 +1716,10 @@ out_reschedule: · 267 unmodified lines | |||
| 1660 | 1716 | ||
| 1661 | /** | 1717 | /** |
| 1662 | * svm_range_evict - evict svm range | 1718 | * svm_range_evict - evict svm range |
| 1719 | * @prange: svm range structure | ||
| 1720 | * @mm: current process mm_struct | ||
| 1721 | * @start: starting process queue number | ||
| 1722 | * @last: last process queue number | ||
| 1663 | * | 1723 | * |
| 1664 | * Stop all queues of the process to ensure GPU doesn't access the memory, then | 1724 | * Stop all queues of the process to ensure GPU doesn't access the memory, then |
| 1665 | * return to let CPU evict the buffer and proceed CPU pagetable update. | 1725 | * return to let CPU evict the buffer and proceed CPU pagetable update. |
| @@ -1764,46 +1824,49 @@ static struct svm_range *svm_range_clone(struct svm_range *old) · 98 unmodified lines | |||
| 1764 | } | 1824 | } |
| 1765 | 1825 | ||
| 1766 | /** | 1826 | /** |
| 1767 | * svm_range_handle_overlap - split overlap ranges | 1827 | * svm_range_add - add svm range and handle overlap |
| 1768 | * @svms: svm range list header | 1828 | * @p: the range add to this process svms |
| 1769 | * @new: range added with this attributes | 1829 | * @start: page size aligned |
| 1770 | * @start: range added start address, in pages | 1830 | * @size: page size aligned |
| 1771 | * @last: range last address, in pages | 1831 | * @nattr: number of attributes |
| 1772 | * @update_list: output, the ranges attributes are updated. For set_attr, this | 1832 | * @attrs: array of attributes |
| 1773 | * will do validation and map to GPUs. For unmap, this will be | 1833 | * @update_list: output, the ranges need validate and update GPU mapping |
| 1774 | * removed and unmap from GPUs | 1834 | * @insert_list: output, the ranges need insert to svms |
| 1775 | * @insert_list: output, the ranges will be inserted into svms, attributes are | 1835 | * @remove_list: output, the ranges are replaced and need remove from svms |
| 1776 | * not changes. For set_attr, this will add into svms. | ||
| 1777 | * @remove_list:output, the ranges will be removed from svms | ||
| 1778 | * @left: the remaining range after overlap, For set_attr, this will be added | ||
| 1779 | * as new range. | ||
| 1780 | * | 1836 | * |
| 1781 | * Total have 5 overlap cases. | 1837 | * Check if the virtual address range has overlap with any existing ranges, |
| 1838 | * split partly overlapping ranges and add new ranges in the gaps. All changes | ||
| 1839 | * should be applied to the range_list and interval tree transactionally. If | ||
| 1840 | * any range split or allocation fails, the entire update fails. Therefore any | ||
| 1841 | * existing overlapping svm_ranges are cloned and the original svm_ranges left | ||
| 1842 | * unchanged. | ||
| 1782 | * | 1843 | * |
| 1783 | * This function handles overlap of an address interval with existing | 1844 | * If the transaction succeeds, the caller can update and insert clones and |
| 1784 | * struct svm_ranges for applying new attributes. This may require | 1845 | * new ranges, then free the originals. |
| 1785 | * splitting existing struct svm_ranges. All changes should be applied to | ||
| 1786 | * the range_list and interval tree transactionally. If any split operation | ||
| 1787 | * fails, the entire update fails. Therefore the existing overlapping | ||
| 1788 | * svm_ranges are cloned and the original svm_ranges left unchanged. If the | ||
| 1789 | * transaction succeeds, the modified clones are added and the originals | ||
| 1790 | * freed. Otherwise the clones are removed and the old svm_ranges remain. | ||
| 1791 | * | 1846 | * |
| 1792 | * Context: The caller must hold svms->lock | 1847 | * Otherwise the caller can free the clones and new ranges, while the old |
| 1848 | * svm_ranges remain unchanged. | ||
| 1849 | * | ||
| 1850 | * Context: Process context, caller must hold svms->lock | ||
| 1851 | * | ||
| 1852 | * Return: | ||
| 1853 | * 0 - OK, otherwise error code | ||
| 1793 | */ | 1854 | */ |
| 1794 | static int | 1855 | static int |
| 1795 | svm_range_handle_overlap(struct svm_range_list *svms, struct svm_range *new, | 1856 | svm_range_add(struct kfd_process *p, uint64_t start, uint64_t size, |
| 1796 | unsigned long start, unsigned long last, | 1857 | uint32_t nattr, struct kfd_ioctl_svm_attribute *attrs, |
| 1797 | struct list_head *update_list, | 1858 | struct list_head *update_list, struct list_head *insert_list, |
| 1798 | struct list_head *insert_list, | 1859 | struct list_head *remove_list) |
| 1799 | struct list_head *remove_list, | ||
| 1800 | unsigned long *left) | ||
| 1801 | { | 1860 | { |
| 1861 | unsigned long last = start + size - 1UL; | ||
| 1862 | struct svm_range_list *svms = &p->svms; | ||
| 1802 | struct interval_tree_node *node; | 1863 | struct interval_tree_node *node; |
| 1803 | struct svm_range *prange; | 1864 | struct svm_range *prange; |
| 1804 | struct svm_range *tmp; | 1865 | struct svm_range *tmp; |
| 1805 | int r = 0; | 1866 | int r = 0; |
| 1806 | 1867 | ||
| 1868 | pr_debug("svms 0x%p [0x%llx 0x%lx]\n", &p->svms, start, last); | ||
| 1869 | |||
| 1807 | INIT_LIST_HEAD(update_list); | 1870 | INIT_LIST_HEAD(update_list); |
| 1808 | INIT_LIST_HEAD(insert_list); | 1871 | INIT_LIST_HEAD(insert_list); |
| 1809 | INIT_LIST_HEAD(remove_list); | 1872 | INIT_LIST_HEAD(remove_list); |
| @@ -1811,18 +1874,24 @@ svm_range_handle_overlap(struct svm_range_list *svms, struct svm_range *new, · 1 unmodified line | |||
| 1811 | node = interval_tree_iter_first(&svms->objects, start, last); | 1874 | node = interval_tree_iter_first(&svms->objects, start, last); |
| 1812 | while (node) { | 1875 | while (node) { |
| 1813 | struct interval_tree_node *next; | 1876 | struct interval_tree_node *next; |
| 1814 | struct svm_range *old; | ||
| 1815 | unsigned long next_start; | 1877 | unsigned long next_start; |
| 1816 | 1878 | ||
| 1817 | pr_debug("found overlap node [0x%lx 0x%lx]\n", node->start, | 1879 | pr_debug("found overlap node [0x%lx 0x%lx]\n", node->start, |
| 1818 | node->last); | 1880 | node->last); |
| 1819 | 1881 | ||
| 1820 | old = container_of(node, struct svm_range, it_node); | 1882 | prange = container_of(node, struct svm_range, it_node); |
| 1821 | next = interval_tree_iter_next(node, start, last); | 1883 | next = interval_tree_iter_next(node, start, last); |
| 1822 | next_start = min(node->last, last) + 1; | 1884 | next_start = min(node->last, last) + 1; |
| 1823 | 1885 | ||
| 1824 | if (node->start < start || node->last > last) { | 1886 | if (svm_range_is_same_attrs(p, prange, nattr, attrs)) { |
| 1825 | /* node intersects the updated range, clone+split it */ | 1887 | /* nothing to do */ |
| 1888 | } else if (node->start < start || node->last > last) { | ||
| 1889 | /* node intersects the update range and its attributes | ||
| 1890 | * will change. Clone and split it, apply updates only | ||
| 1891 | * to the overlapping part | ||
| 1892 | */ | ||
| 1893 | struct svm_range *old = prange; | ||
| 1894 | |||
| 1826 | prange = svm_range_clone(old); | 1895 | prange = svm_range_clone(old); |
| 1827 | if (!prange) { | 1896 | if (!prange) { |
| 1828 | r = -ENOMEM; | 1897 | r = -ENOMEM; |
| @@ -1831,17 +1900,18 @@ svm_range_handle_overlap(struct svm_range_list *svms, struct svm_range *new, · 2 unmodified lines | |||
| 1831 | 1900 | ||
| 1832 | list_add(&old->remove_list, remove_list); | 1901 | list_add(&old->remove_list, remove_list); |
| 1833 | list_add(&prange->insert_list, insert_list); | 1902 | list_add(&prange->insert_list, insert_list); |
| 1903 | list_add(&prange->update_list, update_list); | ||
| 1834 | 1904 | ||
| 1835 | if (node->start < start) { | 1905 | if (node->start < start) { |
| 1836 | pr_debug("change old range start\n"); | 1906 | pr_debug("change old range start\n"); |
| 1837 | r = svm_range_split_head(prange, new, start, | 1907 | r = svm_range_split_head(prange, start, |
| 1838 | insert_list); | 1908 | insert_list); |
| 1839 | if (r) | 1909 | if (r) |
| 1840 | goto out; | 1910 | goto out; |
| 1841 | } | 1911 | } |
| 1842 | if (node->last > last) { | 1912 | if (node->last > last) { |
| 1843 | pr_debug("change old range last\n"); | 1913 | pr_debug("change old range last\n"); |
| 1844 | r = svm_range_split_tail(prange, new, last, | 1914 | r = svm_range_split_tail(prange, last, |
| 1845 | insert_list); | 1915 | insert_list); |
| 1846 | if (r) | 1916 | if (r) |
| 1847 | goto out; | 1917 | goto out; |
| @@ -1850,16 +1920,12 @@ svm_range_handle_overlap(struct svm_range_list *svms, struct svm_range *new, · 2 unmodified lines | |||
| 1850 | /* The node is contained within start..last, | 1920 | /* The node is contained within start..last, |
| 1851 | * just update it | 1921 | * just update it |
| 1852 | */ | 1922 | */ |
| 1853 | prange = old; | ||
| 1854 | } | ||
| 1855 | |||
| 1856 | if (!svm_range_is_same_attrs(prange, new)) | ||
| 1857 | list_add(&prange->update_list, update_list); | 1923 | list_add(&prange->update_list, update_list); |
| 1924 | } | ||
| 1858 | 1925 | ||
| 1859 | /* insert a new node if needed */ | 1926 | /* insert a new node if needed */ |
| 1860 | if (node->start > start) { | 1927 | if (node->start > start) { |
| 1861 | prange = svm_range_new(prange->svms, start, | 1928 | prange = svm_range_new(svms, start, node->start - 1); |
| 1862 | node->start - 1); | ||
| 1863 | if (!prange) { | 1929 | if (!prange) { |
| 1864 | r = -ENOMEM; | 1930 | r = -ENOMEM; |
| 1865 | goto out; | 1931 | goto out; |
| @@ -1873,8 +1939,16 @@ svm_range_handle_overlap(struct svm_range_list *svms, struct svm_range *new, · 7 unmodified lines | |||
| 1873 | start = next_start; | 1939 | start = next_start; |
| 1874 | } | 1940 | } |
| 1875 | 1941 | ||
| 1876 | if (left && start <= last) | 1942 | /* add a final range at the end if needed */ |
| 1877 | *left = last - start + 1; | 1943 | if (start <= last) { |
| 1944 | prange = svm_range_new(svms, start, last); | ||
| 1945 | if (!prange) { | ||
| 1946 | r = -ENOMEM; | ||
| 1947 | goto out; | ||
| 1948 | } | ||
| 1949 | list_add(&prange->insert_list, insert_list); | ||
| 1950 | list_add(&prange->update_list, update_list); | ||
| 1951 | } | ||
| 1878 | 1952 | ||
| 1879 | out: | 1953 | out: |
| 1880 | if (r) | 1954 | if (r) |
| @@ -1962,7 +2036,6 @@ svm_range_handle_list_op(struct svm_range_list *svms, struct svm_range *prange) · 81 unmodified lines | |||
| 1962 | static void svm_range_drain_retry_fault(struct svm_range_list *svms) | 2036 | static void svm_range_drain_retry_fault(struct svm_range_list *svms) |
| 1963 | { | 2037 | { |
| 1964 | struct kfd_process_device *pdd; | 2038 | struct kfd_process_device *pdd; |
| 1965 | struct amdgpu_device *adev; | ||
| 1966 | struct kfd_process *p; | 2039 | struct kfd_process *p; |
| 1967 | int drain; | 2040 | int drain; |
| 1968 | uint32_t i; | 2041 | uint32_t i; |
| @@ -1980,9 +2053,9 @@ restart: · 11 unmodified lines | |||
| 1980 | continue; | 2053 | continue; |
| 1981 | 2054 | ||
| 1982 | pr_debug("drain retry fault gpu %d svms %p\n", i, svms); | 2055 | pr_debug("drain retry fault gpu %d svms %p\n", i, svms); |
| 1983 | adev = (struct amdgpu_device *)pdd->dev->kgd; | ||
| 1984 | 2056 | ||
| 1985 | amdgpu_ih_wait_on_checkpoint_process(adev, &adev->irq.ih1); | 2057 | amdgpu_ih_wait_on_checkpoint_process_ts(pdd->dev->adev, |
| 2058 | &pdd->dev->adev->irq.ih1); | ||
| 1986 | pr_debug("drain retry fault gpu %d svms 0x%p done\n", i, svms); | 2059 | pr_debug("drain retry fault gpu %d svms 0x%p done\n", i, svms); |
| 1987 | } | 2060 | } |
| 1988 | if (atomic_cmpxchg(&svms->drain_pagefaults, drain, 0) != drain) | 2061 | if (atomic_cmpxchg(&svms->drain_pagefaults, drain, 0) != drain) |
| @@ -2172,6 +2245,9 @@ svm_range_unmap_from_cpu(struct mm_struct *mm, struct svm_range *prange, · 183 unmodified lines | |||
| 2172 | 2245 | ||
| 2173 | /** | 2246 | /** |
| 2174 | * svm_range_cpu_invalidate_pagetables - interval notifier callback | 2247 | * svm_range_cpu_invalidate_pagetables - interval notifier callback |
| 2248 | * @mni: mmu_interval_notifier struct | ||
| 2249 | * @range: mmu_notifier_range struct | ||
| 2250 | * @cur_seq: value to pass to mmu_interval_set_seq() | ||
| 2175 | * | 2251 | * |
| 2176 | * If event is MMU_NOTIFY_UNMAP, this is from CPU unmap range, otherwise, it | 2252 | * If event is MMU_NOTIFY_UNMAP, this is from CPU unmap range, otherwise, it |
| 2177 | * is from migration, or CPU page invalidation callback. | 2253 | * is from migration, or CPU page invalidation callback. |
| @@ -2201,8 +2277,8 @@ svm_range_cpu_invalidate_pagetables(struct mmu_interval_notifier *mni, · 23 unmodified lines | |||
| 2201 | 2277 | ||
| 2202 | start = mni->interval_tree.start; | 2278 | start = mni->interval_tree.start; |
| 2203 | last = mni->interval_tree.last; | 2279 | last = mni->interval_tree.last; |
| 2204 | start = (start > range->start ? start : range->start) >> PAGE_SHIFT; | 2280 | start = max(start, range->start) >> PAGE_SHIFT; |
| 2205 | last = (last < (range->end - 1) ? last : range->end - 1) >> PAGE_SHIFT; | 2281 | last = min(last, range->end - 1) >> PAGE_SHIFT; |
| 2206 | pr_debug("[0x%lx 0x%lx] range[0x%lx 0x%lx] notifier[0x%lx 0x%lx] %d\n", | 2282 | pr_debug("[0x%lx 0x%lx] range[0x%lx 0x%lx] notifier[0x%lx 0x%lx] %d\n", |
| 2207 | start, last, range->start >> PAGE_SHIFT, | 2283 | start, last, range->start >> PAGE_SHIFT, |
| 2208 | (range->end - 1) >> PAGE_SHIFT, | 2284 | (range->end - 1) >> PAGE_SHIFT, |
| @@ -2304,7 +2380,7 @@ svm_range_best_restore_location(struct svm_range *prange, · 95 unmodified lines | |||
| 2304 | 2380 | ||
| 2305 | p = container_of(prange->svms, struct kfd_process, svms); | 2381 | p = container_of(prange->svms, struct kfd_process, svms); |
| 2306 | 2382 | ||
| 2307 | r = kfd_process_gpuid_from_kgd(p, adev, &gpuid, gpuidx); | 2383 | r = kfd_process_gpuid_from_adev(p, adev, &gpuid, gpuidx); |
| 2308 | if (r < 0) { | 2384 | if (r < 0) { |
| 2309 | pr_debug("failed to get gpuid from kgd\n"); | 2385 | pr_debug("failed to get gpuid from kgd\n"); |
| 2310 | return -1; | 2386 | return -1; |
| @@ -2481,7 +2557,7 @@ svm_range *svm_range_create_unregistered_range(struct amdgpu_device *adev, · 170 unmodified lines | |||
| 2481 | pr_debug("Failed to create prange in address [0x%llx]\n", addr); | 2557 | pr_debug("Failed to create prange in address [0x%llx]\n", addr); |
| 2482 | return NULL; | 2558 | return NULL; |
| 2483 | } | 2559 | } |
| 2484 | if (kfd_process_gpuid_from_kgd(p, adev, &gpuid, &gpuidx)) { | 2560 | if (kfd_process_gpuid_from_adev(p, adev, &gpuid, &gpuidx)) { |
| 2485 | pr_debug("failed to get gpuid from kgd\n"); | 2561 | pr_debug("failed to get gpuid from kgd\n"); |
| 2486 | svm_range_free(prange); | 2562 | svm_range_free(prange); |
| 2487 | return NULL; | 2563 | return NULL; |
| @@ -2548,7 +2624,7 @@ svm_range_count_fault(struct amdgpu_device *adev, struct kfd_process *p, · 60 unmodified lines | |||
| 2548 | uint32_t gpuid; | 2624 | uint32_t gpuid; |
| 2549 | int r; | 2625 | int r; |
| 2550 | 2626 | ||
| 2551 | r = kfd_process_gpuid_from_kgd(p, adev, &gpuid, &gpuidx); | 2627 | r = kfd_process_gpuid_from_adev(p, adev, &gpuid, &gpuidx); |
| 2552 | if (r < 0) | 2628 | if (r < 0) |
| 2553 | return; | 2629 | return; |
| 2554 | } | 2630 | } |
| @@ -2894,59 +2970,6 @@ svm_range_is_valid(struct kfd_process *p, uint64_t start, uint64_t size) · 339 unmodified lines | |||
| 2894 | NULL); | 2970 | NULL); |
| 2895 | } | 2971 | } |
| 2896 | 2972 | ||
| 2897 | /** | ||
| 2898 | * svm_range_add - add svm range and handle overlap | ||
| 2899 | * @p: the range add to this process svms | ||
| 2900 | * @start: page size aligned | ||
| 2901 | * @size: page size aligned | ||
| 2902 | * @nattr: number of attributes | ||
| 2903 | * @attrs: array of attributes | ||
| 2904 | * @update_list: output, the ranges need validate and update GPU mapping | ||
| 2905 | * @insert_list: output, the ranges need insert to svms | ||
| 2906 | * @remove_list: output, the ranges are replaced and need remove from svms | ||
| 2907 | * | ||
| 2908 | * Check if the virtual address range has overlap with the registered ranges, | ||
| 2909 | * split the overlapped range, copy and adjust pages address and vram nodes in | ||
| 2910 | * old and new ranges. | ||
| 2911 | * | ||
| 2912 | * Context: Process context, caller must hold svms->lock | ||
| 2913 | * | ||
| 2914 | * Return: | ||
| 2915 | * 0 - OK, otherwise error code | ||
| 2916 | */ | ||
| 2917 | static int | ||
| 2918 | svm_range_add(struct kfd_process *p, uint64_t start, uint64_t size, | ||
| 2919 | uint32_t nattr, struct kfd_ioctl_svm_attribute *attrs, | ||
| 2920 | struct list_head *update_list, struct list_head *insert_list, | ||
| 2921 | struct list_head *remove_list) | ||
| 2922 | { | ||
| 2923 | uint64_t last = start + size - 1UL; | ||
| 2924 | struct svm_range_list *svms; | ||
| 2925 | struct svm_range new = {0}; | ||
| 2926 | struct svm_range *prange; | ||
| 2927 | unsigned long left = 0; | ||
| 2928 | int r = 0; | ||
| 2929 | |||
| 2930 | pr_debug("svms 0x%p [0x%llx 0x%llx]\n", &p->svms, start, last); | ||
| 2931 | |||
| 2932 | svm_range_apply_attrs(p, &new, nattr, attrs); | ||
| 2933 | |||
| 2934 | svms = &p->svms; | ||
| 2935 | |||
| 2936 | r = svm_range_handle_overlap(svms, &new, start, last, update_list, | ||
| 2937 | insert_list, remove_list, &left); | ||
| 2938 | if (r) | ||
| 2939 | return r; | ||
| 2940 | |||
| 2941 | if (left) { | ||
| 2942 | prange = svm_range_new(svms, last - left + 1, last); | ||
| 2943 | list_add(&prange->insert_list, insert_list); | ||
| 2944 | list_add(&prange->update_list, update_list); | ||
| 2945 | } | ||
| 2946 | |||
| 2947 | return 0; | ||
| 2948 | } | ||
| 2949 | |||
| 2950 | /** | 2973 | /** |
| 2951 | * svm_range_best_prefetch_location - decide the best prefetch location | 2974 | * svm_range_best_prefetch_location - decide the best prefetch location |
| 2952 | * @prange: svm range structure | 2975 | * @prange: svm range structure |
| @@ -2980,7 +3003,6 @@ svm_range_best_prefetch_location(struct svm_range *prange) · 27 unmodified lines | |||
| 2980 | uint32_t best_loc = prange->prefetch_loc; | 3003 | uint32_t best_loc = prange->prefetch_loc; |
| 2981 | struct kfd_process_device *pdd; | 3004 | struct kfd_process_device *pdd; |
| 2982 | struct amdgpu_device *bo_adev; | 3005 | struct amdgpu_device *bo_adev; |
| 2983 | struct amdgpu_device *adev; | ||
| 2984 | struct kfd_process *p; | 3006 | struct kfd_process *p; |
| 2985 | uint32_t gpuidx; | 3007 | uint32_t gpuidx; |
| 2986 | 3008 | ||
| @@ -3008,12 +3030,11 @@ svm_range_best_prefetch_location(struct svm_range *prange) · 21 unmodified lines | |||
| 3008 | pr_debug("failed to get device by idx 0x%x\n", gpuidx); | 3030 | pr_debug("failed to get device by idx 0x%x\n", gpuidx); |
| 3009 | continue; | 3031 | continue; |
| 3010 | } | 3032 | } |
| 3011 | adev = (struct amdgpu_device *)pdd->dev->kgd; | ||
| 3012 | 3033 | ||
| 3013 | if (adev == bo_adev) | 3034 | if (pdd->dev->adev == bo_adev) |
| 3014 | continue; | 3035 | continue; |
| 3015 | 3036 | ||
| 3016 | if (!amdgpu_xgmi_same_hive(adev, bo_adev)) { | 3037 | if (!amdgpu_xgmi_same_hive(pdd->dev->adev, bo_adev)) { |
| 3017 | best_loc = 0; | 3038 | best_loc = 0; |
| 3018 | break; | 3039 | break; |
| 3019 | } | 3040 | } |
| @@ -48,6 +48,7 @@ struct svm_range_bo { · 47 unmodified lines | |||
| 48 | struct work_struct eviction_work; | 48 | struct work_struct eviction_work; |
| 49 | struct svm_range_list *svms; | 49 | struct svm_range_list *svms; |
| 50 | uint32_t evicting; | 50 | uint32_t evicting; |
| 51 | struct work_struct release_work; | ||
| 51 | }; | 52 | }; |
| 52 | 53 | ||
| 53 | enum svm_work_list_ops { | 54 | enum svm_work_list_ops { |
| @@ -195,7 +196,7 @@ void svm_range_list_lock_and_flush_work(struct svm_range_list *svms, struct mm_s · 141 unmodified lines | |||
| 195 | */ | 196 | */ |
| 196 | #define KFD_IS_SVM_API_SUPPORTED(dev) ((dev)->pgmap.type != 0) | 197 | #define KFD_IS_SVM_API_SUPPORTED(dev) ((dev)->pgmap.type != 0) |
| 197 | 198 | ||
| 198 | void svm_range_bo_unref(struct svm_range_bo *svm_bo); | 199 | void svm_range_bo_unref_async(struct svm_range_bo *svm_bo); |
| 199 | #else | 200 | #else |
| 200 | 201 | ||
| 201 | struct kfd_process; | 202 | struct kfd_process; |
| @@ -113,7 +113,7 @@ struct kfd_dev *kfd_device_by_pci_dev(const struct pci_dev *pdev) · 112 unmodified lines | |||
| 113 | return device; | 113 | return device; |
| 114 | } | 114 | } |
| 115 | 115 | ||
| 116 | struct kfd_dev *kfd_device_by_kgd(const struct kgd_dev *kgd) | 116 | struct kfd_dev *kfd_device_by_adev(const struct amdgpu_device *adev) |
| 117 | { | 117 | { |
| 118 | struct kfd_topology_device *top_dev; | 118 | struct kfd_topology_device *top_dev; |
| 119 | struct kfd_dev *device = NULL; | 119 | struct kfd_dev *device = NULL; |
| @@ -121,7 +121,7 @@ struct kfd_dev *kfd_device_by_kgd(const struct kgd_dev *kgd) · 1 unmodified line | |||
| 121 | down_read(&topology_lock); | 121 | down_read(&topology_lock); |
| 122 | 122 | ||
| 123 | list_for_each_entry(top_dev, &topology_device_list, list) | 123 | list_for_each_entry(top_dev, &topology_device_list, list) |
| 124 | if (top_dev->gpu && top_dev->gpu->kgd == kgd) { | 124 | if (top_dev->gpu && top_dev->gpu->adev == adev) { |
| 125 | device = top_dev->gpu; | 125 | device = top_dev->gpu; |
| 126 | break; | 126 | break; |
| 127 | } | 127 | } |
| @@ -503,7 +503,7 @@ static ssize_t node_show(struct kobject *kobj, struct attribute *attr, · 375 unmodified lines | |||
| 503 | 503 | ||
| 504 | if (dev->gpu) { | 504 | if (dev->gpu) { |
| 505 | log_max_watch_addr = | 505 | log_max_watch_addr = |
| 506 | __ilog2_u32(dev->gpu->device_info->num_of_watch_points); | 506 | __ilog2_u32(dev->gpu->device_info.num_of_watch_points); |
| 507 | 507 | ||
| 508 | if (log_max_watch_addr) { | 508 | if (log_max_watch_addr) { |
| 509 | dev->node_props.capability |= | 509 | dev->node_props.capability |= |
| @@ -515,7 +515,7 @@ static ssize_t node_show(struct kobject *kobj, struct attribute *attr, · 5 unmodified lines | |||
| 515 | HSA_CAP_WATCH_POINTS_TOTALBITS_MASK); | 515 | HSA_CAP_WATCH_POINTS_TOTALBITS_MASK); |
| 516 | } | 516 | } |
| 517 | 517 | ||
| 518 | if (dev->gpu->device_info->asic_family == CHIP_TONGA) | 518 | if (dev->gpu->adev->asic_type == CHIP_TONGA) |
| 519 | dev->node_props.capability |= | 519 | dev->node_props.capability |= |
| 520 | HSA_CAP_AQL_QUEUE_DOUBLE_MAP; | 520 | HSA_CAP_AQL_QUEUE_DOUBLE_MAP; |
| 521 | 521 | ||
| @@ -531,7 +531,7 @@ static ssize_t node_show(struct kobject *kobj, struct attribute *attr, · 9 unmodified lines | |||
| 531 | sysfs_show_32bit_prop(buffer, offs, "sdma_fw_version", | 531 | sysfs_show_32bit_prop(buffer, offs, "sdma_fw_version", |
| 532 | dev->gpu->sdma_fw_version); | 532 | dev->gpu->sdma_fw_version); |
| 533 | sysfs_show_64bit_prop(buffer, offs, "unique_id", | 533 | sysfs_show_64bit_prop(buffer, offs, "unique_id", |
| 534 | amdgpu_amdkfd_get_unique_id(dev->gpu->kgd)); | 534 | dev->gpu->adev->unique_id); |
| 535 | 535 | ||
| 536 | } | 536 | } |
| 537 | 537 | ||
| @@ -1106,7 +1106,7 @@ static uint32_t kfd_generate_gpu_id(struct kfd_dev *gpu) · 568 unmodified lines | |||
| 1106 | if (!gpu) | 1106 | if (!gpu) |
| 1107 | return 0; | 1107 | return 0; |
| 1108 | 1108 | ||
| 1109 | amdgpu_amdkfd_get_local_mem_info(gpu->kgd, &local_mem_info); | 1109 | amdgpu_amdkfd_get_local_mem_info(gpu->adev, &local_mem_info); |
| 1110 | 1110 | ||
| 1111 | local_mem_size = local_mem_info.local_mem_size_private + | 1111 | local_mem_size = local_mem_info.local_mem_size_private + |
| 1112 | local_mem_info.local_mem_size_public; | 1112 | local_mem_info.local_mem_size_public; |
| @@ -1189,7 +1189,7 @@ static void kfd_fill_mem_clk_max_info(struct kfd_topology_device *dev) · 76 unmodified lines | |||
| 1189 | * for APUs - If CRAT from ACPI reports more than one bank, then | 1189 | * for APUs - If CRAT from ACPI reports more than one bank, then |
| 1190 | * all the banks will report the same mem_clk_max information | 1190 | * all the banks will report the same mem_clk_max information |
| 1191 | */ | 1191 | */ |
| 1192 | amdgpu_amdkfd_get_local_mem_info(dev->gpu->kgd, &local_mem_info); | 1192 | amdgpu_amdkfd_get_local_mem_info(dev->gpu->adev, &local_mem_info); |
| 1193 | 1193 | ||
| 1194 | list_for_each_entry(mem, &dev->mem_props, list) | 1194 | list_for_each_entry(mem, &dev->mem_props, list) |
| 1195 | mem->mem_clk_max = local_mem_info.mem_clk_max; | 1195 | mem->mem_clk_max = local_mem_info.mem_clk_max; |
| @@ -1217,8 +1217,7 @@ static void kfd_set_iolink_no_atomics(struct kfd_topology_device *dev, · 21 unmodified lines | |||
| 1217 | /* set gpu (dev) flags. */ | 1217 | /* set gpu (dev) flags. */ |
| 1218 | } else { | 1218 | } else { |
| 1219 | if (!dev->gpu->pci_atomic_requested || | 1219 | if (!dev->gpu->pci_atomic_requested || |
| 1220 | dev->gpu->device_info->asic_family == | 1220 | dev->gpu->adev->asic_type == CHIP_HAWAII) |
| 1221 | CHIP_HAWAII) | ||
| 1222 | link->flags |= CRAT_IOLINK_FLAGS_NO_ATOMICS_32_BIT | | 1221 | link->flags |= CRAT_IOLINK_FLAGS_NO_ATOMICS_32_BIT | |
| 1223 | CRAT_IOLINK_FLAGS_NO_ATOMICS_64_BIT; | 1222 | CRAT_IOLINK_FLAGS_NO_ATOMICS_64_BIT; |
| 1224 | } | 1223 | } |
| @@ -1239,7 +1238,7 @@ static void kfd_set_iolink_non_coherent(struct kfd_topology_device *to_dev, · 14 unmodified lines | |||
| 1239 | */ | 1238 | */ |
| 1240 | if (inbound_link->iolink_type == CRAT_IOLINK_TYPE_PCIEXPRESS || | 1239 | if (inbound_link->iolink_type == CRAT_IOLINK_TYPE_PCIEXPRESS || |
| 1241 | (inbound_link->iolink_type == CRAT_IOLINK_TYPE_XGMI && | 1240 | (inbound_link->iolink_type == CRAT_IOLINK_TYPE_XGMI && |
| 1242 | to_dev->gpu->device_info->asic_family == CHIP_VEGA20)) { | 1241 | KFD_GC_VERSION(to_dev->gpu) == IP_VERSION(9, 4, 0))) { |
| 1243 | outbound_link->flags |= CRAT_IOLINK_FLAGS_NON_COHERENT; | 1242 | outbound_link->flags |= CRAT_IOLINK_FLAGS_NON_COHERENT; |
| 1244 | inbound_link->flags |= CRAT_IOLINK_FLAGS_NON_COHERENT; | 1243 | inbound_link->flags |= CRAT_IOLINK_FLAGS_NON_COHERENT; |
| 1245 | } | 1244 | } |
| @@ -1286,7 +1285,8 @@ int kfd_topology_add_device(struct kfd_dev *gpu) · 40 unmodified lines | |||
| 1286 | void *crat_image = NULL; | 1285 | void *crat_image = NULL; |
| 1287 | size_t image_size = 0; | 1286 | size_t image_size = 0; |
| 1288 | int proximity_domain; | 1287 | int proximity_domain; |
| 1289 | struct amdgpu_device *adev; | 1288 | int i; |
| 1289 | const char *asic_name = amdgpu_asic_name[gpu->adev->asic_type]; | ||
| 1290 | 1290 | ||
| 1291 | INIT_LIST_HEAD(&temp_topology_device_list); | 1291 | INIT_LIST_HEAD(&temp_topology_device_list); |
| 1292 | 1292 | ||
| @@ -1296,10 +1296,8 @@ int kfd_topology_add_device(struct kfd_dev *gpu) · 3 unmodified lines | |||
| 1296 | 1296 | ||
| 1297 | proximity_domain = atomic_inc_return(&topology_crat_proximity_domain); | 1297 | proximity_domain = atomic_inc_return(&topology_crat_proximity_domain); |
| 1298 | 1298 | ||
| 1299 | adev = (struct amdgpu_device *)(gpu->kgd); | ||
| 1300 | |||
| 1301 | /* Include the CPU in xGMI hive if xGMI connected by assigning it the hive ID. */ | 1299 | /* Include the CPU in xGMI hive if xGMI connected by assigning it the hive ID. */ |
| 1302 | if (gpu->hive_id && adev->gmc.xgmi.connected_to_cpu) { | 1300 | if (gpu->hive_id && gpu->adev->gmc.xgmi.connected_to_cpu) { |
| 1303 | struct kfd_topology_device *top_dev; | 1301 | struct kfd_topology_device *top_dev; |
| 1304 | 1302 | ||
| 1305 | down_read(&topology_lock); | 1303 | down_read(&topology_lock); |
| @@ -1372,45 +1370,48 @@ int kfd_topology_add_device(struct kfd_dev *gpu) · 66 unmodified lines | |||
| 1372 | * needed for the topology | 1370 | * needed for the topology |
| 1373 | */ | 1371 | */ |
| 1374 | 1372 | ||
| 1375 | amdgpu_amdkfd_get_cu_info(dev->gpu->kgd, &cu_info); | 1373 | amdgpu_amdkfd_get_cu_info(dev->gpu->adev, &cu_info); |
| 1376 | 1374 | ||
| 1377 | strncpy(dev->node_props.name, gpu->device_info->asic_name, | 1375 | for (i = 0; i < KFD_TOPOLOGY_PUBLIC_NAME_SIZE-1; i++) { |
| 1378 | KFD_TOPOLOGY_PUBLIC_NAME_SIZE); | 1376 | dev->node_props.name[i] = __tolower(asic_name[i]); |
| 1377 | if (asic_name[i] == '\0') | ||
| 1378 | break; | ||
| 1379 | } | ||
| 1380 | dev->node_props.name[i] = '\0'; | ||
| 1379 | 1381 | ||
| 1380 | dev->node_props.simd_arrays_per_engine = | 1382 | dev->node_props.simd_arrays_per_engine = |
| 1381 | cu_info.num_shader_arrays_per_engine; | 1383 | cu_info.num_shader_arrays_per_engine; |
| 1382 | 1384 | ||
| 1383 | dev->node_props.gfx_target_version = gpu->device_info->gfx_target_version; | 1385 | dev->node_props.gfx_target_version = gpu->device_info.gfx_target_version; |
| 1384 | dev->node_props.vendor_id = gpu->pdev->vendor; | 1386 | dev->node_props.vendor_id = gpu->pdev->vendor; |
| 1385 | dev->node_props.device_id = gpu->pdev->device; | 1387 | dev->node_props.device_id = gpu->pdev->device; |
| 1386 | dev->node_props.capability |= | 1388 | dev->node_props.capability |= |
| 1387 | ((amdgpu_amdkfd_get_asic_rev_id(dev->gpu->kgd) << | 1389 | ((dev->gpu->adev->rev_id << HSA_CAP_ASIC_REVISION_SHIFT) & |
| 1388 | HSA_CAP_ASIC_REVISION_SHIFT) & | ||
| 1389 | HSA_CAP_ASIC_REVISION_MASK); | 1390 | HSA_CAP_ASIC_REVISION_MASK); |
| 1390 | dev->node_props.location_id = pci_dev_id(gpu->pdev); | 1391 | dev->node_props.location_id = pci_dev_id(gpu->pdev); |
| 1391 | dev->node_props.domain = pci_domain_nr(gpu->pdev->bus); | 1392 | dev->node_props.domain = pci_domain_nr(gpu->pdev->bus); |
| 1392 | dev->node_props.max_engine_clk_fcompute = | 1393 | dev->node_props.max_engine_clk_fcompute = |
| 1393 | amdgpu_amdkfd_get_max_engine_clock_in_mhz(dev->gpu->kgd); | 1394 | amdgpu_amdkfd_get_max_engine_clock_in_mhz(dev->gpu->adev); |
| 1394 | dev->node_props.max_engine_clk_ccompute = | 1395 | dev->node_props.max_engine_clk_ccompute = |
| 1395 | cpufreq_quick_get_max(0) / 1000; | 1396 | cpufreq_quick_get_max(0) / 1000; |
| 1396 | dev->node_props.drm_render_minor = | 1397 | dev->node_props.drm_render_minor = |
| 1397 | gpu->shared_resources.drm_render_minor; | 1398 | gpu->shared_resources.drm_render_minor; |
| 1398 | 1399 | ||
| 1399 | dev->node_props.hive_id = gpu->hive_id; | 1400 | dev->node_props.hive_id = gpu->hive_id; |
| 1400 | dev->node_props.num_sdma_engines = gpu->device_info->num_sdma_engines; | 1401 | dev->node_props.num_sdma_engines = kfd_get_num_sdma_engines(gpu); |
| 1401 | dev->node_props.num_sdma_xgmi_engines = | 1402 | dev->node_props.num_sdma_xgmi_engines = |
| 1402 | gpu->device_info->num_xgmi_sdma_engines; | 1403 | kfd_get_num_xgmi_sdma_engines(gpu); |
| 1403 | dev->node_props.num_sdma_queues_per_engine = | 1404 | dev->node_props.num_sdma_queues_per_engine = |
| 1404 | gpu->device_info->num_sdma_queues_per_engine; | 1405 | gpu->device_info.num_sdma_queues_per_engine; |
| 1405 | dev->node_props.num_gws = (dev->gpu->gws && | 1406 | dev->node_props.num_gws = (dev->gpu->gws && |
| 1406 | dev->gpu->dqm->sched_policy != KFD_SCHED_POLICY_NO_HWS) ? | 1407 | dev->gpu->dqm->sched_policy != KFD_SCHED_POLICY_NO_HWS) ? |
| 1407 | amdgpu_amdkfd_get_num_gws(dev->gpu->kgd) : 0; | 1408 | dev->gpu->adev->gds.gws_size : 0; |
| 1408 | dev->node_props.num_cp_queues = get_cp_queues_num(dev->gpu->dqm); | 1409 | dev->node_props.num_cp_queues = get_cp_queues_num(dev->gpu->dqm); |
| 1409 | 1410 | ||
| 1410 | kfd_fill_mem_clk_max_info(dev); | 1411 | kfd_fill_mem_clk_max_info(dev); |
| 1411 | kfd_fill_iolink_non_crat_info(dev); | 1412 | kfd_fill_iolink_non_crat_info(dev); |
| 1412 | 1413 | ||
| 1413 | switch (dev->gpu->device_info->asic_family) { | 1414 | switch (dev->gpu->adev->asic_type) { |
| 1414 | case CHIP_KAVERI: | 1415 | case CHIP_KAVERI: |
| 1415 | case CHIP_HAWAII: | 1416 | case CHIP_HAWAII: |
| 1416 | case CHIP_TONGA: | 1417 | case CHIP_TONGA: |
| @@ -1429,30 +1430,14 @@ int kfd_topology_add_device(struct kfd_dev *gpu) · 12 unmodified lines | |||
| 1429 | HSA_CAP_DOORBELL_TYPE_TOTALBITS_SHIFT) & | 1430 | HSA_CAP_DOORBELL_TYPE_TOTALBITS_SHIFT) & |
| 1430 | HSA_CAP_DOORBELL_TYPE_TOTALBITS_MASK); | 1431 | HSA_CAP_DOORBELL_TYPE_TOTALBITS_MASK); |
| 1431 | break; | 1432 | break; |
| 1432 | case CHIP_VEGA10: | ||
| 1433 | case CHIP_VEGA12: | ||
| 1434 | case CHIP_VEGA20: | ||
| 1435 | case CHIP_RAVEN: | ||
| 1436 | case CHIP_RENOIR: | ||
| 1437 | case CHIP_ARCTURUS: | ||
| 1438 | case CHIP_ALDEBARAN: | ||
| 1439 | case CHIP_NAVI10: | ||
| 1440 | case CHIP_NAVI12: | ||
| 1441 | case CHIP_NAVI14: | ||
| 1442 | case CHIP_SIENNA_CICHLID: | ||
| 1443 | case CHIP_NAVY_FLOUNDER: | ||
| 1444 | case CHIP_VANGOGH: | ||
| 1445 | case CHIP_DIMGREY_CAVEFISH: | ||
| 1446 | case CHIP_BEIGE_GOBY: | ||
| 1447 | case CHIP_YELLOW_CARP: | ||
| 1448 | case CHIP_CYAN_SKILLFISH: | ||
| 1449 | dev->node_props.capability |= ((HSA_CAP_DOORBELL_TYPE_2_0 << | ||
| 1450 | HSA_CAP_DOORBELL_TYPE_TOTALBITS_SHIFT) & | ||
| 1451 | HSA_CAP_DOORBELL_TYPE_TOTALBITS_MASK); | ||
| 1452 | break; | ||
| 1453 | default: | 1433 | default: |
| 1454 | WARN(1, "Unexpected ASIC family %u", | 1434 | if (KFD_GC_VERSION(dev->gpu) >= IP_VERSION(9, 0, 1)) |
| 1455 | dev->gpu->device_info->asic_family); | 1435 | dev->node_props.capability |= ((HSA_CAP_DOORBELL_TYPE_2_0 << |
| 1436 | HSA_CAP_DOORBELL_TYPE_TOTALBITS_SHIFT) & | ||
| 1437 | HSA_CAP_DOORBELL_TYPE_TOTALBITS_MASK); | ||
| 1438 | else | ||
| 1439 | WARN(1, "Unexpected ASIC family %u", | ||
| 1440 | dev->gpu->adev->asic_type); | ||
| 1456 | } | 1441 | } |
| 1457 | 1442 | ||
| 1458 | /* | 1443 | /* |
| @@ -1469,7 +1454,7 @@ int kfd_topology_add_device(struct kfd_dev *gpu) · 10 unmodified lines | |||
| 1469 | * because it doesn't consider masked out CUs | 1454 | * because it doesn't consider masked out CUs |
| 1470 | * max_waves_per_simd: Carrizo reports wrong max_waves_per_simd | 1455 | * max_waves_per_simd: Carrizo reports wrong max_waves_per_simd |
| 1471 | */ | 1456 | */ |
| 1472 | if (dev->gpu->device_info->asic_family == CHIP_CARRIZO) { | 1457 | if (dev->gpu->adev->asic_type == CHIP_CARRIZO) { |
| 1473 | dev->node_props.simd_count = | 1458 | dev->node_props.simd_count = |
| 1474 | cu_info.simd_per_cu * cu_info.cu_active_number; | 1459 | cu_info.simd_per_cu * cu_info.cu_active_number; |
| 1475 | dev->node_props.max_waves_per_simd = 10; | 1460 | dev->node_props.max_waves_per_simd = 10; |
| @@ -1477,16 +1462,17 @@ int kfd_topology_add_device(struct kfd_dev *gpu) · 1 unmodified line | |||
| 1477 | 1462 | ||
| 1478 | /* kfd only concerns sram ecc on GFX and HBM ecc on UMC */ | 1463 | /* kfd only concerns sram ecc on GFX and HBM ecc on UMC */ |
| 1479 | dev->node_props.capability |= | 1464 | dev->node_props.capability |= |
| 1480 | ((adev->ras_enabled & BIT(AMDGPU_RAS_BLOCK__GFX)) != 0) ? | 1465 | ((dev->gpu->adev->ras_enabled & BIT(AMDGPU_RAS_BLOCK__GFX)) != 0) ? |
| 1481 | HSA_CAP_SRAM_EDCSUPPORTED : 0; | 1466 | HSA_CAP_SRAM_EDCSUPPORTED : 0; |
| 1482 | dev->node_props.capability |= ((adev->ras_enabled & BIT(AMDGPU_RAS_BLOCK__UMC)) != 0) ? | 1467 | dev->node_props.capability |= |
| 1468 | ((dev->gpu->adev->ras_enabled & BIT(AMDGPU_RAS_BLOCK__UMC)) != 0) ? | ||
| 1483 | HSA_CAP_MEM_EDCSUPPORTED : 0; | 1469 | HSA_CAP_MEM_EDCSUPPORTED : 0; |
| 1484 | 1470 | ||
| 1485 | if (adev->asic_type != CHIP_VEGA10) | 1471 | if (KFD_GC_VERSION(dev->gpu) != IP_VERSION(9, 0, 1)) |
| 1486 | dev->node_props.capability |= (adev->ras_enabled != 0) ? | 1472 | dev->node_props.capability |= (dev->gpu->adev->ras_enabled != 0) ? |
| 1487 | HSA_CAP_RASEVENTNOTIFY : 0; | 1473 | HSA_CAP_RASEVENTNOTIFY : 0; |
| 1488 | 1474 | ||
| 1489 | if (KFD_IS_SVM_API_SUPPORTED(adev->kfd.dev)) | 1475 | if (KFD_IS_SVM_API_SUPPORTED(dev->gpu->adev->kfd.dev)) |
| 1490 | dev->node_props.capability |= HSA_CAP_SVMAPI_SUPPORTED; | 1476 | dev->node_props.capability |= HSA_CAP_SVMAPI_SUPPORTED; |
| 1491 | 1477 | ||
| 1492 | kfd_debug_print_topology(); | 1478 | kfd_debug_print_topology(); |
| @@ -1592,7 +1578,7 @@ void kfd_double_confirm_iommu_support(struct kfd_dev *gpu) · 99 unmodified lines | |||
| 1592 | 1578 | ||
| 1593 | gpu->use_iommu_v2 = false; | 1579 | gpu->use_iommu_v2 = false; |
| 1594 | 1580 | ||
| 1595 | if (!gpu->device_info->needs_iommu_device) | 1581 | if (!gpu->device_info.needs_iommu_device) |
| 1596 | return; | 1582 | return; |
| 1597 | 1583 | ||
| 1598 | down_read(&topology_lock); | 1584 | down_read(&topology_lock); |
| @@ -25,38 +25,11 @@ · 24 unmodified lines | |||
| 25 | 25 | ||
| 26 | #include <linux/types.h> | 26 | #include <linux/types.h> |
| 27 | #include <linux/list.h> | 27 | #include <linux/list.h> |
| 28 | #include <linux/kfd_sysfs.h> | ||
| 28 | #include "kfd_crat.h" | 29 | #include "kfd_crat.h" |
| 29 | 30 | ||
| 30 | #define KFD_TOPOLOGY_PUBLIC_NAME_SIZE 32 | 31 | #define KFD_TOPOLOGY_PUBLIC_NAME_SIZE 32 |
| 31 | 32 | ||
| 32 | #define HSA_CAP_HOT_PLUGGABLE 0x00000001 | ||
| 33 | #define HSA_CAP_ATS_PRESENT 0x00000002 | ||
| 34 | #define HSA_CAP_SHARED_WITH_GRAPHICS 0x00000004 | ||
| 35 | #define HSA_CAP_QUEUE_SIZE_POW2 0x00000008 | ||
| 36 | #define HSA_CAP_QUEUE_SIZE_32BIT 0x00000010 | ||
| 37 | #define HSA_CAP_QUEUE_IDLE_EVENT 0x00000020 | ||
| 38 | #define HSA_CAP_VA_LIMIT 0x00000040 | ||
| 39 | #define HSA_CAP_WATCH_POINTS_SUPPORTED 0x00000080 | ||
| 40 | #define HSA_CAP_WATCH_POINTS_TOTALBITS_MASK 0x00000f00 | ||
| 41 | #define HSA_CAP_WATCH_POINTS_TOTALBITS_SHIFT 8 | ||
| 42 | #define HSA_CAP_DOORBELL_TYPE_TOTALBITS_MASK 0x00003000 | ||
| 43 | #define HSA_CAP_DOORBELL_TYPE_TOTALBITS_SHIFT 12 | ||
| 44 | |||
| 45 | #define HSA_CAP_DOORBELL_TYPE_PRE_1_0 0x0 | ||
| 46 | #define HSA_CAP_DOORBELL_TYPE_1_0 0x1 | ||
| 47 | #define HSA_CAP_DOORBELL_TYPE_2_0 0x2 | ||
| 48 | #define HSA_CAP_AQL_QUEUE_DOUBLE_MAP 0x00004000 | ||
| 49 | |||
| 50 | #define HSA_CAP_RESERVED_WAS_SRAM_EDCSUPPORTED 0x00080000 /* Old buggy user mode depends on this being 0 */ | ||
| 51 | #define HSA_CAP_MEM_EDCSUPPORTED 0x00100000 | ||
| 52 | #define HSA_CAP_RASEVENTNOTIFY 0x00200000 | ||
| 53 | #define HSA_CAP_ASIC_REVISION_MASK 0x03c00000 | ||
| 54 | #define HSA_CAP_ASIC_REVISION_SHIFT 22 | ||
| 55 | #define HSA_CAP_SRAM_EDCSUPPORTED 0x04000000 | ||
| 56 | #define HSA_CAP_SVMAPI_SUPPORTED 0x08000000 | ||
| 57 | #define HSA_CAP_FLAGS_COHERENTHOSTACCESS 0x10000000 | ||
| 58 | #define HSA_CAP_RESERVED 0xe00f8000 | ||
| 59 | |||
| 60 | struct kfd_node_properties { | 33 | struct kfd_node_properties { |
| 61 | uint64_t hive_id; | 34 | uint64_t hive_id; |
| 62 | uint32_t cpu_cores_count; | 35 | uint32_t cpu_cores_count; |
| @@ -93,17 +66,6 @@ struct kfd_node_properties { · 30 unmodified lines | |||
| 93 | char name[KFD_TOPOLOGY_PUBLIC_NAME_SIZE]; | 66 | char name[KFD_TOPOLOGY_PUBLIC_NAME_SIZE]; |
| 94 | }; | 67 | }; |
| 95 | 68 | ||
| 96 | #define HSA_MEM_HEAP_TYPE_SYSTEM 0 | ||
| 97 | #define HSA_MEM_HEAP_TYPE_FB_PUBLIC 1 | ||
| 98 | #define HSA_MEM_HEAP_TYPE_FB_PRIVATE 2 | ||
| 99 | #define HSA_MEM_HEAP_TYPE_GPU_GDS 3 | ||
| 100 | #define HSA_MEM_HEAP_TYPE_GPU_LDS 4 | ||
| 101 | #define HSA_MEM_HEAP_TYPE_GPU_SCRATCH 5 | ||
| 102 | |||
| 103 | #define HSA_MEM_FLAGS_HOT_PLUGGABLE 0x00000001 | ||
| 104 | #define HSA_MEM_FLAGS_NON_VOLATILE 0x00000002 | ||
| 105 | #define HSA_MEM_FLAGS_RESERVED 0xfffffffc | ||
| 106 | |||
| 107 | struct kfd_mem_properties { | 69 | struct kfd_mem_properties { |
| 108 | struct list_head list; | 70 | struct list_head list; |
| 109 | uint32_t heap_type; | 71 | uint32_t heap_type; |
| @@ -116,12 +78,6 @@ struct kfd_mem_properties { · 6 unmodified lines | |||
| 116 | struct attribute attr; | 78 | struct attribute attr; |
| 117 | }; | 79 | }; |
| 118 | 80 | ||
| 119 | #define HSA_CACHE_TYPE_DATA 0x00000001 | ||
| 120 | #define HSA_CACHE_TYPE_INSTRUCTION 0x00000002 | ||
| 121 | #define HSA_CACHE_TYPE_CPU 0x00000004 | ||
| 122 | #define HSA_CACHE_TYPE_HSACU 0x00000008 | ||
| 123 | #define HSA_CACHE_TYPE_RESERVED 0xfffffff0 | ||
| 124 | |||
| 125 | struct kfd_cache_properties { | 81 | struct kfd_cache_properties { |
| 126 | struct list_head list; | 82 | struct list_head list; |
| 127 | uint32_t processor_id_low; | 83 | uint32_t processor_id_low; |
| @@ -624,7 +624,7 @@ static void dm_dcn_vertical_interrupt0_high_irq(void *interrupt_params) · 623 unmodified lines | |||
| 624 | #endif /* CONFIG_DRM_AMD_SECURE_DISPLAY */ | 624 | #endif /* CONFIG_DRM_AMD_SECURE_DISPLAY */ |
| 625 | 625 | ||
| 626 | /** | 626 | /** |
| 627 | * dmub_aux_setconfig_reply_callback - Callback for AUX or SET_CONFIG command. | 627 | * dmub_aux_setconfig_callback - Callback for AUX or SET_CONFIG command. |
| 628 | * @adev: amdgpu_device pointer | 628 | * @adev: amdgpu_device pointer |
| 629 | * @notify: dmub notification structure | 629 | * @notify: dmub notification structure |
| 630 | * | 630 | * |
| @@ -632,7 +632,8 @@ static void dm_dcn_vertical_interrupt0_high_irq(void *interrupt_params) · 1 unmodified line | |||
| 632 | * Copies dmub notification to DM which is to be read by AUX command. | 632 | * Copies dmub notification to DM which is to be read by AUX command. |
| 633 | * issuing thread and also signals the event to wake up the thread. | 633 | * issuing thread and also signals the event to wake up the thread. |
| 634 | */ | 634 | */ |
| 635 | void dmub_aux_setconfig_callback(struct amdgpu_device *adev, struct dmub_notification *notify) | 635 | static void dmub_aux_setconfig_callback(struct amdgpu_device *adev, |
| 636 | struct dmub_notification *notify) | ||
| 636 | { | 637 | { |
| 637 | if (adev->dm.dmub_notify) | 638 | if (adev->dm.dmub_notify) |
| 638 | memcpy(adev->dm.dmub_notify, notify, sizeof(struct dmub_notification)); | 639 | memcpy(adev->dm.dmub_notify, notify, sizeof(struct dmub_notification)); |
| @@ -648,7 +649,8 @@ void dmub_aux_setconfig_callback(struct amdgpu_device *adev, struct dmub_notific · 9 unmodified lines | |||
| 648 | * Dmub Hpd interrupt processing callback. Gets displayindex through the | 649 | * Dmub Hpd interrupt processing callback. Gets displayindex through the |
| 649 | * ink index and calls helper to do the processing. | 650 | * ink index and calls helper to do the processing. |
| 650 | */ | 651 | */ |
| 651 | void dmub_hpd_callback(struct amdgpu_device *adev, struct dmub_notification *notify) | 652 | static void dmub_hpd_callback(struct amdgpu_device *adev, |
| 653 | struct dmub_notification *notify) | ||
| 652 | { | 654 | { |
| 653 | struct amdgpu_dm_connector *aconnector; | 655 | struct amdgpu_dm_connector *aconnector; |
| 654 | struct amdgpu_dm_connector *hpd_aconnector = NULL; | 656 | struct amdgpu_dm_connector *hpd_aconnector = NULL; |
| @@ -705,8 +707,10 @@ void dmub_hpd_callback(struct amdgpu_device *adev, struct dmub_notification *not · 50 unmodified lines | |||
| 705 | * to dmub interrupt handling thread | 707 | * to dmub interrupt handling thread |
| 706 | * Return: true if successfully registered, false if there is existing registration | 708 | * Return: true if successfully registered, false if there is existing registration |
| 707 | */ | 709 | */ |
| 708 | bool register_dmub_notify_callback(struct amdgpu_device *adev, enum dmub_notification_type type, | 710 | static bool register_dmub_notify_callback(struct amdgpu_device *adev, |
| 709 | dmub_notify_interrupt_callback_t callback, bool dmub_int_thread_offload) | 711 | enum dmub_notification_type type, |
| 712 | dmub_notify_interrupt_callback_t callback, | ||
| 713 | bool dmub_int_thread_offload) | ||
| 710 | { | 714 | { |
| 711 | if (callback != NULL && type < ARRAY_SIZE(adev->dm.dmub_thread_offload)) { | 715 | if (callback != NULL && type < ARRAY_SIZE(adev->dm.dmub_thread_offload)) { |
| 712 | adev->dm.dmub_callback[type] = callback; | 716 | adev->dm.dmub_callback[type] = callback; |
| @@ -790,8 +794,7 @@ static void dm_dmub_outbox1_low_irq(void *interrupt_params) · 77 unmodified lines | |||
| 790 | plink = adev->dm.dc->links[notify.link_index]; | 794 | plink = adev->dm.dc->links[notify.link_index]; |
| 791 | if (plink) { | 795 | if (plink) { |
| 792 | plink->hpd_status = | 796 | plink->hpd_status = |
| 793 | notify.hpd_status == | 797 | notify.hpd_status == DP_HPD_PLUG; |
| 794 | DP_HPD_PLUG ? true : false; | ||
| 795 | } | 798 | } |
| 796 | } | 799 | } |
| 797 | queue_work(adev->dm.delayed_hpd_wq, &dmub_hpd_wrk->handle_hpd_work); | 800 | queue_work(adev->dm.delayed_hpd_wq, &dmub_hpd_wrk->handle_hpd_work); |
| @@ -1459,8 +1462,21 @@ static int amdgpu_dm_init(struct amdgpu_device *adev) · 661 unmodified lines | |||
| 1459 | if (amdgpu_dc_feature_mask & DC_EDP_NO_POWER_SEQUENCING) | 1462 | if (amdgpu_dc_feature_mask & DC_EDP_NO_POWER_SEQUENCING) |
| 1460 | init_data.flags.edp_no_power_sequencing = true; | 1463 | init_data.flags.edp_no_power_sequencing = true; |
| 1461 | 1464 | ||
| 1465 | #ifdef CONFIG_DRM_AMD_DC_DCN | ||
| 1466 | if (amdgpu_dc_feature_mask & DC_DISABLE_LTTPR_DP1_4A) | ||
| 1467 | init_data.flags.allow_lttpr_non_transparent_mode.bits.DP1_4A = true; | ||
| 1468 | if (amdgpu_dc_feature_mask & DC_DISABLE_LTTPR_DP2_0) | ||
| 1469 | init_data.flags.allow_lttpr_non_transparent_mode.bits.DP2_0 = true; | ||
| 1470 | #endif | ||
| 1471 | |||
| 1462 | init_data.flags.power_down_display_on_boot = true; | 1472 | init_data.flags.power_down_display_on_boot = true; |
| 1463 | 1473 | ||
| 1474 | if (check_seamless_boot_capability(adev)) { | ||
| 1475 | init_data.flags.power_down_display_on_boot = false; | ||
| 1476 | init_data.flags.allow_seamless_boot_optimization = true; | ||
| 1477 | DRM_INFO("Seamless boot condition check passed\n"); | ||
| 1478 | } | ||
| 1479 | |||
| 1464 | INIT_LIST_HEAD(&adev->dm.da_list); | 1480 | INIT_LIST_HEAD(&adev->dm.da_list); |
| 1465 | /* Display Core create. */ | 1481 | /* Display Core create. */ |
| 1466 | adev->dm.dc = dc_create(&init_data); | 1482 | adev->dm.dc = dc_create(&init_data); |
| @@ -1485,8 +1501,10 @@ static int amdgpu_dm_init(struct amdgpu_device *adev) · 18 unmodified lines | |||
| 1485 | if (amdgpu_dc_debug_mask & DC_DISABLE_STUTTER) | 1501 | if (amdgpu_dc_debug_mask & DC_DISABLE_STUTTER) |
| 1486 | adev->dm.dc->debug.disable_stutter = true; | 1502 | adev->dm.dc->debug.disable_stutter = true; |
| 1487 | 1503 | ||
| 1488 | if (amdgpu_dc_debug_mask & DC_DISABLE_DSC) | 1504 | if (amdgpu_dc_debug_mask & DC_DISABLE_DSC) { |
| 1489 | adev->dm.dc->debug.disable_dsc = true; | 1505 | adev->dm.dc->debug.disable_dsc = true; |
| 1506 | adev->dm.dc->debug.disable_dsc_edp = true; | ||
| 1507 | } | ||
| 1490 | 1508 | ||
| 1491 | if (amdgpu_dc_debug_mask & DC_DISABLE_CLOCK_GATING) | 1509 | if (amdgpu_dc_debug_mask & DC_DISABLE_CLOCK_GATING) |
| 1492 | adev->dm.dc->debug.disable_clock_gate = true; | 1510 | adev->dm.dc->debug.disable_clock_gate = true; |
| @@ -2309,14 +2327,6 @@ static enum dc_status amdgpu_dm_commit_zero_streams(struct dc *dc) · 816 unmodified lines | |||
| 2309 | goto fail; | 2327 | goto fail; |
| 2310 | } | 2328 | } |
| 2311 | 2329 | ||
| 2312 | |||
| 2313 | res = dc_validate_global_state(dc, context, false); | ||
| 2314 | |||
| 2315 | if (res != DC_OK) { | ||
| 2316 | DRM_ERROR("%s:resource validation failed, dc_status:%d\n", __func__, res); | ||
| 2317 | goto fail; | ||
| 2318 | } | ||
| 2319 | |||
| 2320 | res = dc_commit_state(dc, context); | 2330 | res = dc_commit_state(dc, context); |
| 2321 | 2331 | ||
| 2322 | fail: | 2332 | fail: |
| @@ -2600,15 +2610,6 @@ static int dm_resume(void *handle) · 277 unmodified lines | |||
| 2600 | = 0xffffffff; | 2610 | = 0xffffffff; |
| 2601 | } | 2611 | } |
| 2602 | } | 2612 | } |
| 2603 | #if defined(CONFIG_DRM_AMD_DC_DCN) | ||
| 2604 | /* | ||
| 2605 | * Resource allocation happens for link encoders for newer ASIC in | ||
| 2606 | * dc_validate_global_state, so we need to revalidate it. | ||
| 2607 | * | ||
| 2608 | * This shouldn't fail (it passed once before), so warn if it does. | ||
| 2609 | */ | ||
| 2610 | WARN_ON(dc_validate_global_state(dm->dc, dc_state, false) != DC_OK); | ||
| 2611 | #endif | ||
| 2612 | 2613 | ||
| 2613 | WARN_ON(!dc_commit_state(dm->dc, dc_state)); | 2614 | WARN_ON(!dc_commit_state(dm->dc, dc_state)); |
| 2614 | 2615 | ||
| @@ -2965,13 +2966,12 @@ void amdgpu_dm_update_connector_after_detect( · 350 unmodified lines | |||
| 2965 | aconnector->edid = | 2966 | aconnector->edid = |
| 2966 | (struct edid *)sink->dc_edid.raw_edid; | 2967 | (struct edid *)sink->dc_edid.raw_edid; |
| 2967 | 2968 | ||
| 2968 | drm_connector_update_edid_property(connector, | ||
| 2969 | aconnector->edid); | ||
| 2970 | if (aconnector->dc_link->aux_mode) | 2969 | if (aconnector->dc_link->aux_mode) |
| 2971 | drm_dp_cec_set_edid(&aconnector->dm_dp_aux.aux, | 2970 | drm_dp_cec_set_edid(&aconnector->dm_dp_aux.aux, |
| 2972 | aconnector->edid); | 2971 | aconnector->edid); |
| 2973 | } | 2972 | } |
| 2974 | 2973 | ||
| 2974 | drm_connector_update_edid_property(connector, aconnector->edid); | ||
| 2975 | amdgpu_dm_update_freesync_caps(connector, aconnector->edid); | 2975 | amdgpu_dm_update_freesync_caps(connector, aconnector->edid); |
| 2976 | update_connector_ext_caps(aconnector); | 2976 | update_connector_ext_caps(aconnector); |
| 2977 | } else { | 2977 | } else { |
| @@ -3039,7 +3039,7 @@ static void handle_hpd_irq_helper(struct amdgpu_dm_connector *aconnector) · 61 unmodified lines | |||
| 3039 | drm_modeset_unlock_all(dev); | 3039 | drm_modeset_unlock_all(dev); |
| 3040 | 3040 | ||
| 3041 | if (aconnector->base.force == DRM_FORCE_UNSPECIFIED) | 3041 | if (aconnector->base.force == DRM_FORCE_UNSPECIFIED) |
| 3042 | drm_kms_helper_hotplug_event(dev); | 3042 | drm_kms_helper_connector_hotplug_event(connector); |
| 3043 | 3043 | ||
| 3044 | } else if (dc_link_detect(aconnector->dc_link, DETECT_REASON_HPD)) { | 3044 | } else if (dc_link_detect(aconnector->dc_link, DETECT_REASON_HPD)) { |
| 3045 | if (new_connection_type == dc_connection_none && | 3045 | if (new_connection_type == dc_connection_none && |
| @@ -3054,7 +3054,7 @@ static void handle_hpd_irq_helper(struct amdgpu_dm_connector *aconnector) · 8 unmodified lines | |||
| 3054 | drm_modeset_unlock_all(dev); | 3054 | drm_modeset_unlock_all(dev); |
| 3055 | 3055 | ||
| 3056 | if (aconnector->base.force == DRM_FORCE_UNSPECIFIED) | 3056 | if (aconnector->base.force == DRM_FORCE_UNSPECIFIED) |
| 3057 | drm_kms_helper_hotplug_event(dev); | 3057 | drm_kms_helper_connector_hotplug_event(connector); |
| 3058 | } | 3058 | } |
| 3059 | mutex_unlock(&aconnector->hpd_lock); | 3059 | mutex_unlock(&aconnector->hpd_lock); |
| 3060 | 3060 | ||
| @@ -3248,7 +3248,7 @@ out: · 187 unmodified lines | |||
| 3248 | dm_restore_drm_connector_state(dev, connector); | 3248 | dm_restore_drm_connector_state(dev, connector); |
| 3249 | drm_modeset_unlock_all(dev); | 3249 | drm_modeset_unlock_all(dev); |
| 3250 | 3250 | ||
| 3251 | drm_kms_helper_hotplug_event(dev); | 3251 | drm_kms_helper_connector_hotplug_event(connector); |
| 3252 | } else if (dc_link_detect(dc_link, DETECT_REASON_HPDRX)) { | 3252 | } else if (dc_link_detect(dc_link, DETECT_REASON_HPDRX)) { |
| 3253 | 3253 | ||
| 3254 | if (aconnector->fake_enable) | 3254 | if (aconnector->fake_enable) |
| @@ -3261,7 +3261,7 @@ out: · 6 unmodified lines | |||
| 3261 | dm_restore_drm_connector_state(dev, connector); | 3261 | dm_restore_drm_connector_state(dev, connector); |
| 3262 | drm_modeset_unlock_all(dev); | 3262 | drm_modeset_unlock_all(dev); |
| 3263 | 3263 | ||
| 3264 | drm_kms_helper_hotplug_event(dev); | 3264 | drm_kms_helper_connector_hotplug_event(connector); |
| 3265 | } | 3265 | } |
| 3266 | } | 3266 | } |
| 3267 | #ifdef CONFIG_DRM_AMD_DC_HDCP | 3267 | #ifdef CONFIG_DRM_AMD_DC_HDCP |
| @@ -4281,6 +4281,14 @@ static int amdgpu_dm_initialize_drm_device(struct amdgpu_device *adev) · 1013 unmodified lines | |||
| 4281 | 4281 | ||
| 4282 | } | 4282 | } |
| 4283 | 4283 | ||
| 4284 | /* | ||
| 4285 | * Disable vblank IRQs aggressively for power-saving. | ||
| 4286 | * | ||
| 4287 | * TODO: Fix vblank control helpers to delay PSR entry to allow this when PSR | ||
| 4288 | * is also supported. | ||
| 4289 | */ | ||
| 4290 | adev_to_drm(adev)->vblank_disable_immediate = !psr_feature_enabled; | ||
| 4291 | |||
| 4284 | /* Software is initialized. Now we can register interrupt handlers. */ | 4292 | /* Software is initialized. Now we can register interrupt handlers. */ |
| 4285 | switch (adev->asic_type) { | 4293 | switch (adev->asic_type) { |
| 4286 | #if defined(CONFIG_DRM_AMD_DC_SI) | 4294 | #if defined(CONFIG_DRM_AMD_DC_SI) |
| @@ -6066,11 +6074,72 @@ static void update_dsc_caps(struct amdgpu_dm_connector *aconnector, · 1779 unmodified lines | |||
| 6066 | { | 6074 | { |
| 6067 | stream->timing.flags.DSC = 0; | 6075 | stream->timing.flags.DSC = 0; |
| 6068 | 6076 | ||
| 6069 | if (aconnector->dc_link && sink->sink_signal == SIGNAL_TYPE_DISPLAY_PORT) { | 6077 | if (aconnector->dc_link && (sink->sink_signal == SIGNAL_TYPE_DISPLAY_PORT || |
| 6070 | dc_dsc_parse_dsc_dpcd(aconnector->dc_link->ctx->dc, | 6078 | sink->sink_signal == SIGNAL_TYPE_EDP)) { |
| 6071 | aconnector->dc_link->dpcd_caps.dsc_caps.dsc_basic_caps.raw, | 6079 | if (sink->link->dpcd_caps.dongle_type == DISPLAY_DONGLE_NONE || |
| 6072 | aconnector->dc_link->dpcd_caps.dsc_caps.dsc_branch_decoder_caps.raw, | 6080 | sink->link->dpcd_caps.dongle_type == DISPLAY_DONGLE_DP_HDMI_CONVERTER) |
| 6073 | dsc_caps); | 6081 | dc_dsc_parse_dsc_dpcd(aconnector->dc_link->ctx->dc, |
| 6082 | aconnector->dc_link->dpcd_caps.dsc_caps.dsc_basic_caps.raw, | ||
| 6083 | aconnector->dc_link->dpcd_caps.dsc_caps.dsc_branch_decoder_caps.raw, | ||
| 6084 | dsc_caps); | ||
| 6085 | } | ||
| 6086 | } | ||
| 6087 | |||
| 6088 | static void apply_dsc_policy_for_edp(struct amdgpu_dm_connector *aconnector, | ||
| 6089 | struct dc_sink *sink, struct dc_stream_state *stream, | ||
| 6090 | struct dsc_dec_dpcd_caps *dsc_caps, | ||
| 6091 | uint32_t max_dsc_target_bpp_limit_override) | ||
| 6092 | { | ||
| 6093 | const struct dc_link_settings *verified_link_cap = NULL; | ||
| 6094 | uint32_t link_bw_in_kbps; | ||
| 6095 | uint32_t edp_min_bpp_x16, edp_max_bpp_x16; | ||
| 6096 | struct dc *dc = sink->ctx->dc; | ||
| 6097 | struct dc_dsc_bw_range bw_range = {0}; | ||
| 6098 | struct dc_dsc_config dsc_cfg = {0}; | ||
| 6099 | |||
| 6100 | verified_link_cap = dc_link_get_link_cap(stream->link); | ||
| 6101 | link_bw_in_kbps = dc_link_bandwidth_kbps(stream->link, verified_link_cap); | ||
| 6102 | edp_min_bpp_x16 = 8 * 16; | ||
| 6103 | edp_max_bpp_x16 = 8 * 16; | ||
| 6104 | |||
| 6105 | if (edp_max_bpp_x16 > dsc_caps->edp_max_bits_per_pixel) | ||
| 6106 | edp_max_bpp_x16 = dsc_caps->edp_max_bits_per_pixel; | ||
| 6107 | |||
| 6108 | if (edp_max_bpp_x16 < edp_min_bpp_x16) | ||
| 6109 | edp_min_bpp_x16 = edp_max_bpp_x16; | ||
| 6110 | |||
| 6111 | if (dc_dsc_compute_bandwidth_range(dc->res_pool->dscs[0], | ||
| 6112 | dc->debug.dsc_min_slice_height_override, | ||
| 6113 | edp_min_bpp_x16, edp_max_bpp_x16, | ||
| 6114 | dsc_caps, | ||
| 6115 | &stream->timing, | ||
| 6116 | &bw_range)) { | ||
| 6117 | |||
| 6118 | if (bw_range.max_kbps < link_bw_in_kbps) { | ||
| 6119 | if (dc_dsc_compute_config(dc->res_pool->dscs[0], | ||
| 6120 | dsc_caps, | ||
| 6121 | dc->debug.dsc_min_slice_height_override, | ||
| 6122 | max_dsc_target_bpp_limit_override, | ||
| 6123 | 0, | ||
| 6124 | &stream->timing, | ||
| 6125 | &dsc_cfg)) { | ||
| 6126 | stream->timing.dsc_cfg = dsc_cfg; | ||
| 6127 | stream->timing.flags.DSC = 1; | ||
| 6128 | stream->timing.dsc_cfg.bits_per_pixel = edp_max_bpp_x16; | ||
| 6129 | } | ||
| 6130 | return; | ||
| 6131 | } | ||
| 6132 | } | ||
| 6133 | |||
| 6134 | if (dc_dsc_compute_config(dc->res_pool->dscs[0], | ||
| 6135 | dsc_caps, | ||
| 6136 | dc->debug.dsc_min_slice_height_override, | ||
| 6137 | max_dsc_target_bpp_limit_override, | ||
| 6138 | link_bw_in_kbps, | ||
| 6139 | &stream->timing, | ||
| 6140 | &dsc_cfg)) { | ||
| 6141 | stream->timing.dsc_cfg = dsc_cfg; | ||
| 6142 | stream->timing.flags.DSC = 1; | ||
| 6074 | } | 6143 | } |
| 6075 | } | 6144 | } |
| 6076 | 6145 | ||
| @@ -6081,6 +6150,9 @@ static void apply_dsc_policy_for_stream(struct amdgpu_dm_connector *aconnector, · 4 unmodified lines | |||
| 6081 | struct drm_connector *drm_connector = &aconnector->base; | 6150 | struct drm_connector *drm_connector = &aconnector->base; |
| 6082 | uint32_t link_bandwidth_kbps; | 6151 | uint32_t link_bandwidth_kbps; |
| 6083 | uint32_t max_dsc_target_bpp_limit_override = 0; | 6152 | uint32_t max_dsc_target_bpp_limit_override = 0; |
| 6153 | struct dc *dc = sink->ctx->dc; | ||
| 6154 | uint32_t max_supported_bw_in_kbps, timing_bw_in_kbps; | ||
| 6155 | uint32_t dsc_max_supported_bw_in_kbps; | ||
| 6084 | 6156 | ||
| 6085 | link_bandwidth_kbps = dc_link_bandwidth_kbps(aconnector->dc_link, | 6157 | link_bandwidth_kbps = dc_link_bandwidth_kbps(aconnector->dc_link, |
| 6086 | dc_link_get_link_cap(aconnector->dc_link)); | 6158 | dc_link_get_link_cap(aconnector->dc_link)); |
| @@ -6093,17 +6165,43 @@ static void apply_dsc_policy_for_stream(struct amdgpu_dm_connector *aconnector, · 6 unmodified lines | |||
| 6093 | dc_dsc_policy_set_enable_dsc_when_not_needed( | 6165 | dc_dsc_policy_set_enable_dsc_when_not_needed( |
| 6094 | aconnector->dsc_settings.dsc_force_enable == DSC_CLK_FORCE_ENABLE); | 6166 | aconnector->dsc_settings.dsc_force_enable == DSC_CLK_FORCE_ENABLE); |
| 6095 | 6167 | ||
| 6096 | if (aconnector->dc_link && sink->sink_signal == SIGNAL_TYPE_DISPLAY_PORT) { | 6168 | if (aconnector->dc_link && sink->sink_signal == SIGNAL_TYPE_EDP && !dc->debug.disable_dsc_edp && |
| 6169 | dc->caps.edp_dsc_support && aconnector->dsc_settings.dsc_force_enable != DSC_CLK_FORCE_DISABLE) { | ||
| 6170 | |||
| 6171 | apply_dsc_policy_for_edp(aconnector, sink, stream, dsc_caps, max_dsc_target_bpp_limit_override); | ||
| 6097 | 6172 | ||
| 6098 | if (dc_dsc_compute_config(aconnector->dc_link->ctx->dc->res_pool->dscs[0], | 6173 | } else if (aconnector->dc_link && sink->sink_signal == SIGNAL_TYPE_DISPLAY_PORT) { |
| 6174 | if (sink->link->dpcd_caps.dongle_type == DISPLAY_DONGLE_NONE) { | ||
| 6175 | if (dc_dsc_compute_config(aconnector->dc_link->ctx->dc->res_pool->dscs[0], | ||
| 6099 | dsc_caps, | 6176 | dsc_caps, |
| 6100 | aconnector->dc_link->ctx->dc->debug.dsc_min_slice_height_override, | 6177 | aconnector->dc_link->ctx->dc->debug.dsc_min_slice_height_override, |
| 6101 | max_dsc_target_bpp_limit_override, | 6178 | max_dsc_target_bpp_limit_override, |
| 6102 | link_bandwidth_kbps, | 6179 | link_bandwidth_kbps, |
| 6103 | &stream->timing, | 6180 | &stream->timing, |
| 6104 | &stream->timing.dsc_cfg)) { | 6181 | &stream->timing.dsc_cfg)) { |
| 6105 | stream->timing.flags.DSC = 1; | 6182 | stream->timing.flags.DSC = 1; |
| 6106 | DRM_DEBUG_DRIVER("%s: [%s] DSC is selected from SST RX\n", __func__, drm_connector->name); | 6183 | DRM_DEBUG_DRIVER("%s: [%s] DSC is selected from SST RX\n", |
| 6184 | __func__, drm_connector->name); | ||
| 6185 | } | ||
| 6186 | } else if (sink->link->dpcd_caps.dongle_type == DISPLAY_DONGLE_DP_HDMI_CONVERTER) { | ||
| 6187 | timing_bw_in_kbps = dc_bandwidth_in_kbps_from_timing(&stream->timing); | ||
| 6188 | max_supported_bw_in_kbps = link_bandwidth_kbps; | ||
| 6189 | dsc_max_supported_bw_in_kbps = link_bandwidth_kbps; | ||
| 6190 | |||
| 6191 | if (timing_bw_in_kbps > max_supported_bw_in_kbps && | ||
| 6192 | max_supported_bw_in_kbps > 0 && | ||
| 6193 | dsc_max_supported_bw_in_kbps > 0) | ||
| 6194 | if (dc_dsc_compute_config(aconnector->dc_link->ctx->dc->res_pool->dscs[0], | ||
| 6195 | dsc_caps, | ||
| 6196 | aconnector->dc_link->ctx->dc->debug.dsc_min_slice_height_override, | ||
| 6197 | max_dsc_target_bpp_limit_override, | ||
| 6198 | dsc_max_supported_bw_in_kbps, | ||
| 6199 | &stream->timing, | ||
| 6200 | &stream->timing.dsc_cfg)) { | ||
| 6201 | stream->timing.flags.DSC = 1; | ||
| 6202 | DRM_DEBUG_DRIVER("%s: [%s] DSC is selected from DP-HDMI PCON\n", | ||
| 6203 | __func__, drm_connector->name); | ||
| 6204 | } | ||
| 6107 | } | 6205 | } |
| 6108 | } | 6206 | } |
| 6109 | 6207 | ||
| @@ -8247,15 +8345,8 @@ void amdgpu_dm_connector_init_helper(struct amdgpu_display_manager *dm, · 2137 unmodified lines | |||
| 8247 | break; | 8345 | break; |
| 8248 | case DRM_MODE_CONNECTOR_DisplayPort: | 8346 | case DRM_MODE_CONNECTOR_DisplayPort: |
| 8249 | aconnector->base.polled = DRM_CONNECTOR_POLL_HPD; | 8347 | aconnector->base.polled = DRM_CONNECTOR_POLL_HPD; |
| 8250 | if (link->is_dig_mapping_flexible && | 8348 | link->link_enc = dp_get_link_enc(link); |
| 8251 | link->dc->res_pool->funcs->link_encs_assign) { | 8349 | ASSERT(link->link_enc); |
| 8252 | link->link_enc = | ||
| 8253 | link_enc_cfg_get_link_enc_used_by_link(link->ctx->dc, link); | ||
| 8254 | if (!link->link_enc) | ||
| 8255 | link->link_enc = | ||
| 8256 | link_enc_cfg_get_next_avail_link_enc(link->ctx->dc); | ||
| 8257 | } | ||
| 8258 | |||
| 8259 | if (link->link_enc) | 8350 | if (link->link_enc) |
| 8260 | aconnector->base.ycbcr_420_allowed = | 8351 | aconnector->base.ycbcr_420_allowed = |
| 8261 | link->link_enc->features.dp_ycbcr420_supported ? true : false; | 8352 | link->link_enc->features.dp_ycbcr420_supported ? true : false; |
| @@ -10658,6 +10749,24 @@ static int dm_update_plane_state(struct dc *dc, · 2396 unmodified lines | |||
| 10658 | return ret; | 10749 | return ret; |
| 10659 | } | 10750 | } |
| 10660 | 10751 | ||
| 10752 | static void dm_get_oriented_plane_size(struct drm_plane_state *plane_state, | ||
| 10753 | int *src_w, int *src_h) | ||
| 10754 | { | ||
| 10755 | switch (plane_state->rotation & DRM_MODE_ROTATE_MASK) { | ||
| 10756 | case DRM_MODE_ROTATE_90: | ||
| 10757 | case DRM_MODE_ROTATE_270: | ||
| 10758 | *src_w = plane_state->src_h >> 16; | ||
| 10759 | *src_h = plane_state->src_w >> 16; | ||
| 10760 | break; | ||
| 10761 | case DRM_MODE_ROTATE_0: | ||
| 10762 | case DRM_MODE_ROTATE_180: | ||
| 10763 | default: | ||
| 10764 | *src_w = plane_state->src_w >> 16; | ||
| 10765 | *src_h = plane_state->src_h >> 16; | ||
| 10766 | break; | ||
| 10767 | } | ||
| 10768 | } | ||
| 10769 | |||
| 10661 | static int dm_check_crtc_cursor(struct drm_atomic_state *state, | 10770 | static int dm_check_crtc_cursor(struct drm_atomic_state *state, |
| 10662 | struct drm_crtc *crtc, | 10771 | struct drm_crtc *crtc, |
| 10663 | struct drm_crtc_state *new_crtc_state) | 10772 | struct drm_crtc_state *new_crtc_state) |
| @@ -10666,6 +10775,8 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state, · 2 unmodified lines | |||
| 10666 | struct drm_plane_state *new_cursor_state, *new_underlying_state; | 10775 | struct drm_plane_state *new_cursor_state, *new_underlying_state; |
| 10667 | int i; | 10776 | int i; |
| 10668 | int cursor_scale_w, cursor_scale_h, underlying_scale_w, underlying_scale_h; | 10777 | int cursor_scale_w, cursor_scale_h, underlying_scale_w, underlying_scale_h; |
| 10778 | int cursor_src_w, cursor_src_h; | ||
| 10779 | int underlying_src_w, underlying_src_h; | ||
| 10669 | 10780 | ||
| 10670 | /* On DCE and DCN there is no dedicated hardware cursor plane. We get a | 10781 | /* On DCE and DCN there is no dedicated hardware cursor plane. We get a |
| 10671 | * cursor per pipe but it's going to inherit the scaling and | 10782 | * cursor per pipe but it's going to inherit the scaling and |
| @@ -10677,10 +10788,9 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state, · 5 unmodified lines | |||
| 10677 | return 0; | 10788 | return 0; |
| 10678 | } | 10789 | } |
| 10679 | 10790 | ||
| 10680 | cursor_scale_w = new_cursor_state->crtc_w * 1000 / | 10791 | dm_get_oriented_plane_size(new_cursor_state, &cursor_src_w, &cursor_src_h); |
| 10681 | (new_cursor_state->src_w >> 16); | 10792 | cursor_scale_w = new_cursor_state->crtc_w * 1000 / cursor_src_w; |
| 10682 | cursor_scale_h = new_cursor_state->crtc_h * 1000 / | 10793 | cursor_scale_h = new_cursor_state->crtc_h * 1000 / cursor_src_h; |
| 10683 | (new_cursor_state->src_h >> 16); | ||
| 10684 | 10794 | ||
| 10685 | for_each_new_plane_in_state_reverse(state, underlying, new_underlying_state, i) { | 10795 | for_each_new_plane_in_state_reverse(state, underlying, new_underlying_state, i) { |
| 10686 | /* Narrow down to non-cursor planes on the same CRTC as the cursor */ | 10796 | /* Narrow down to non-cursor planes on the same CRTC as the cursor */ |
| @@ -10691,10 +10801,10 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state, · 4 unmodified lines | |||
| 10691 | if (!new_underlying_state->fb) | 10801 | if (!new_underlying_state->fb) |
| 10692 | continue; | 10802 | continue; |
| 10693 | 10803 | ||
| 10694 | underlying_scale_w = new_underlying_state->crtc_w * 1000 / | 10804 | dm_get_oriented_plane_size(new_underlying_state, |
| 10695 | (new_underlying_state->src_w >> 16); | 10805 | &underlying_src_w, &underlying_src_h); |
| 10696 | underlying_scale_h = new_underlying_state->crtc_h * 1000 / | 10806 | underlying_scale_w = new_underlying_state->crtc_w * 1000 / underlying_src_w; |
| 10697 | (new_underlying_state->src_h >> 16); | 10807 | underlying_scale_h = new_underlying_state->crtc_h * 1000 / underlying_src_h; |
| 10698 | 10808 | ||
| 10699 | if (cursor_scale_w != underlying_scale_w || | 10809 | if (cursor_scale_w != underlying_scale_w || |
| 10700 | cursor_scale_h != underlying_scale_h) { | 10810 | cursor_scale_h != underlying_scale_h) { |
| @@ -10789,8 +10899,10 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev, · 88 unmodified lines | |||
| 10789 | trace_amdgpu_dm_atomic_check_begin(state); | 10899 | trace_amdgpu_dm_atomic_check_begin(state); |
| 10790 | 10900 | ||
| 10791 | ret = drm_atomic_helper_check_modeset(dev, state); | 10901 | ret = drm_atomic_helper_check_modeset(dev, state); |
| 10792 | if (ret) | 10902 | if (ret) { |
| 10903 | DRM_DEBUG_DRIVER("drm_atomic_helper_check_modeset() failed\n"); | ||
| 10793 | goto fail; | 10904 | goto fail; |
| 10905 | } | ||
| 10794 | 10906 | ||
| 10795 | /* Check connector changes */ | 10907 | /* Check connector changes */ |
| 10796 | for_each_oldnew_connector_in_state(state, connector, old_con_state, new_con_state, i) { | 10908 | for_each_oldnew_connector_in_state(state, connector, old_con_state, new_con_state, i) { |
| @@ -10806,6 +10918,7 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev, · 9 unmodified lines | |||
| 10806 | 10918 | ||
| 10807 | new_crtc_state = drm_atomic_get_crtc_state(state, new_con_state->crtc); | 10919 | new_crtc_state = drm_atomic_get_crtc_state(state, new_con_state->crtc); |
| 10808 | if (IS_ERR(new_crtc_state)) { | 10920 | if (IS_ERR(new_crtc_state)) { |
| 10921 | DRM_DEBUG_DRIVER("drm_atomic_get_crtc_state() failed\n"); | ||
| 10809 | ret = PTR_ERR(new_crtc_state); | 10922 | ret = PTR_ERR(new_crtc_state); |
| 10810 | goto fail; | 10923 | goto fail; |
| 10811 | } | 10924 | } |
| @@ -10820,8 +10933,10 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev, · 8 unmodified lines | |||
| 10820 | for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) { | 10933 | for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) { |
| 10821 | if (drm_atomic_crtc_needs_modeset(new_crtc_state)) { | 10934 | if (drm_atomic_crtc_needs_modeset(new_crtc_state)) { |
| 10822 | ret = add_affected_mst_dsc_crtcs(state, crtc); | 10935 | ret = add_affected_mst_dsc_crtcs(state, crtc); |
| 10823 | if (ret) | 10936 | if (ret) { |
| 10937 | DRM_DEBUG_DRIVER("add_affected_mst_dsc_crtcs() failed\n"); | ||
| 10824 | goto fail; | 10938 | goto fail; |
| 10939 | } | ||
| 10825 | } | 10940 | } |
| 10826 | } | 10941 | } |
| 10827 | } | 10942 | } |
| @@ -10836,19 +10951,25 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev, · 8 unmodified lines | |||
| 10836 | continue; | 10951 | continue; |
| 10837 | 10952 | ||
| 10838 | ret = amdgpu_dm_verify_lut_sizes(new_crtc_state); | 10953 | ret = amdgpu_dm_verify_lut_sizes(new_crtc_state); |
| 10839 | if (ret) | 10954 | if (ret) { |
| 10955 | DRM_DEBUG_DRIVER("amdgpu_dm_verify_lut_sizes() failed\n"); | ||
| 10840 | goto fail; | 10956 | goto fail; |
| 10957 | } | ||
| 10841 | 10958 | ||
| 10842 | if (!new_crtc_state->enable) | 10959 | if (!new_crtc_state->enable) |
| 10843 | continue; | 10960 | continue; |
| 10844 | 10961 | ||
| 10845 | ret = drm_atomic_add_affected_connectors(state, crtc); | 10962 | ret = drm_atomic_add_affected_connectors(state, crtc); |
| 10846 | if (ret) | 10963 | if (ret) { |
| 10964 | DRM_DEBUG_DRIVER("drm_atomic_add_affected_connectors() failed\n"); | ||
| 10847 | goto fail; | 10965 | goto fail; |
| 10966 | } | ||
| 10848 | 10967 | ||
| 10849 | ret = drm_atomic_add_affected_planes(state, crtc); | 10968 | ret = drm_atomic_add_affected_planes(state, crtc); |
| 10850 | if (ret) | 10969 | if (ret) { |
| 10970 | DRM_DEBUG_DRIVER("drm_atomic_add_affected_planes() failed\n"); | ||
| 10851 | goto fail; | 10971 | goto fail; |
| 10972 | } | ||
| 10852 | 10973 | ||
| 10853 | if (dm_old_crtc_state->dsc_force_changed) | 10974 | if (dm_old_crtc_state->dsc_force_changed) |
| 10854 | new_crtc_state->mode_changed = true; | 10975 | new_crtc_state->mode_changed = true; |
| @@ -10885,6 +11006,7 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev, · 30 unmodified lines | |||
| 10885 | 11006 | ||
| 10886 | if (IS_ERR(new_plane_state)) { | 11007 | if (IS_ERR(new_plane_state)) { |
| 10887 | ret = PTR_ERR(new_plane_state); | 11008 | ret = PTR_ERR(new_plane_state); |
| 11009 | DRM_DEBUG_DRIVER("new_plane_state is BAD\n"); | ||
| 10888 | goto fail; | 11010 | goto fail; |
| 10889 | } | 11011 | } |
| 10890 | } | 11012 | } |
| @@ -10897,8 +11019,10 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev, · 6 unmodified lines | |||
| 10897 | new_plane_state, | 11019 | new_plane_state, |
| 10898 | false, | 11020 | false, |
| 10899 | &lock_and_validation_needed); | 11021 | &lock_and_validation_needed); |
| 10900 | if (ret) | 11022 | if (ret) { |
| 11023 | DRM_DEBUG_DRIVER("dm_update_plane_state() failed\n"); | ||
| 10901 | goto fail; | 11024 | goto fail; |
| 11025 | } | ||
| 10902 | } | 11026 | } |
| 10903 | 11027 | ||
| 10904 | /* Disable all crtcs which require disable */ | 11028 | /* Disable all crtcs which require disable */ |
| @@ -10908,8 +11032,10 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev, · 3 unmodified lines | |||
| 10908 | new_crtc_state, | 11032 | new_crtc_state, |
| 10909 | false, | 11033 | false, |
| 10910 | &lock_and_validation_needed); | 11034 | &lock_and_validation_needed); |
| 10911 | if (ret) | 11035 | if (ret) { |
| 11036 | DRM_DEBUG_DRIVER("DISABLE: dm_update_crtc_state() failed\n"); | ||
| 10912 | goto fail; | 11037 | goto fail; |
| 11038 | } | ||
| 10913 | } | 11039 | } |
| 10914 | 11040 | ||
| 10915 | /* Enable all crtcs which require enable */ | 11041 | /* Enable all crtcs which require enable */ |
| @@ -10919,8 +11045,10 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev, · 3 unmodified lines | |||
| 10919 | new_crtc_state, | 11045 | new_crtc_state, |
| 10920 | true, | 11046 | true, |
| 10921 | &lock_and_validation_needed); | 11047 | &lock_and_validation_needed); |
| 10922 | if (ret) | 11048 | if (ret) { |
| 11049 | DRM_DEBUG_DRIVER("ENABLE: dm_update_crtc_state() failed\n"); | ||
| 10923 | goto fail; | 11050 | goto fail; |
| 11051 | } | ||
| 10924 | } | 11052 | } |
| 10925 | 11053 | ||
| 10926 | /* Add new/modified planes */ | 11054 | /* Add new/modified planes */ |
| @@ -10930,20 +11058,26 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev, · 3 unmodified lines | |||
| 10930 | new_plane_state, | 11058 | new_plane_state, |
| 10931 | true, | 11059 | true, |
| 10932 | &lock_and_validation_needed); | 11060 | &lock_and_validation_needed); |
| 10933 | if (ret) | 11061 | if (ret) { |
| 11062 | DRM_DEBUG_DRIVER("dm_update_plane_state() failed\n"); | ||
| 10934 | goto fail; | 11063 | goto fail; |
| 11064 | } | ||
| 10935 | } | 11065 | } |
| 10936 | 11066 | ||
| 10937 | /* Run this here since we want to validate the streams we created */ | 11067 | /* Run this here since we want to validate the streams we created */ |
| 10938 | ret = drm_atomic_helper_check_planes(dev, state); | 11068 | ret = drm_atomic_helper_check_planes(dev, state); |
| 10939 | if (ret) | 11069 | if (ret) { |
| 11070 | DRM_DEBUG_DRIVER("drm_atomic_helper_check_planes() failed\n"); | ||
| 10940 | goto fail; | 11071 | goto fail; |
| 11072 | } | ||
| 10941 | 11073 | ||
| 10942 | /* Check cursor planes scaling */ | 11074 | /* Check cursor planes scaling */ |
| 10943 | for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) { | 11075 | for_each_new_crtc_in_state(state, crtc, new_crtc_state, i) { |
| 10944 | ret = dm_check_crtc_cursor(state, crtc, new_crtc_state); | 11076 | ret = dm_check_crtc_cursor(state, crtc, new_crtc_state); |
| 10945 | if (ret) | 11077 | if (ret) { |
| 11078 | DRM_DEBUG_DRIVER("dm_check_crtc_cursor() failed\n"); | ||
| 10946 | goto fail; | 11079 | goto fail; |
| 11080 | } | ||
| 10947 | } | 11081 | } |
| 10948 | 11082 | ||
| 10949 | if (state->legacy_cursor_update) { | 11083 | if (state->legacy_cursor_update) { |
| @@ -11030,20 +11164,28 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev, · 80 unmodified lines | |||
| 11030 | */ | 11164 | */ |
| 11031 | if (lock_and_validation_needed) { | 11165 | if (lock_and_validation_needed) { |
| 11032 | ret = dm_atomic_get_state(state, &dm_state); | 11166 | ret = dm_atomic_get_state(state, &dm_state); |
| 11033 | if (ret) | 11167 | if (ret) { |
| 11168 | DRM_DEBUG_DRIVER("dm_atomic_get_state() failed\n"); | ||
| 11034 | goto fail; | 11169 | goto fail; |
| 11170 | } | ||
| 11035 | 11171 | ||
| 11036 | ret = do_aquire_global_lock(dev, state); | 11172 | ret = do_aquire_global_lock(dev, state); |
| 11037 | if (ret) | 11173 | if (ret) { |
| 11174 | DRM_DEBUG_DRIVER("do_aquire_global_lock() failed\n"); | ||
| 11038 | goto fail; | 11175 | goto fail; |
| 11176 | } | ||
| 11039 | 11177 | ||
| 11040 | #if defined(CONFIG_DRM_AMD_DC_DCN) | 11178 | #if defined(CONFIG_DRM_AMD_DC_DCN) |
| 11041 | if (!compute_mst_dsc_configs_for_state(state, dm_state->context, vars)) | 11179 | if (!compute_mst_dsc_configs_for_state(state, dm_state->context, vars)) { |
| 11180 | DRM_DEBUG_DRIVER("compute_mst_dsc_configs_for_state() failed\n"); | ||
| 11042 | goto fail; | 11181 | goto fail; |
| 11182 | } | ||
| 11043 | 11183 | ||
| 11044 | ret = dm_update_mst_vcpi_slots_for_dsc(state, dm_state->context, vars); | 11184 | ret = dm_update_mst_vcpi_slots_for_dsc(state, dm_state->context, vars); |
| 11045 | if (ret) | 11185 | if (ret) { |
| 11186 | DRM_DEBUG_DRIVER("dm_update_mst_vcpi_slots_for_dsc() failed\n"); | ||
| 11046 | goto fail; | 11187 | goto fail; |
| 11188 | } | ||
| 11047 | #endif | 11189 | #endif |
| 11048 | 11190 | ||
| 11049 | /* | 11191 | /* |
| @@ -11053,12 +11195,13 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev, · 3 unmodified lines | |||
| 11053 | * to get stuck in an infinite loop and hang eventually. | 11195 | * to get stuck in an infinite loop and hang eventually. |
| 11054 | */ | 11196 | */ |
| 11055 | ret = drm_dp_mst_atomic_check(state); | 11197 | ret = drm_dp_mst_atomic_check(state); |
| 11056 | if (ret) | 11198 | if (ret) { |
| 11199 | DRM_DEBUG_DRIVER("drm_dp_mst_atomic_check() failed\n"); | ||
| 11057 | goto fail; | 11200 | goto fail; |
| 11058 | status = dc_validate_global_state(dc, dm_state->context, false); | 11201 | } |
| 11202 | status = dc_validate_global_state(dc, dm_state->context, true); | ||
| 11059 | if (status != DC_OK) { | 11203 | if (status != DC_OK) { |
| 11060 | drm_dbg_atomic(dev, | 11204 | DRM_DEBUG_DRIVER("DC global validation failure: %s (%d)", |
| 11061 | "DC global validation failure: %s (%d)", | ||
| 11062 | dc_status_to_str(status), status); | 11205 | dc_status_to_str(status), status); |
| 11063 | ret = -EINVAL; | 11206 | ret = -EINVAL; |
| 11064 | goto fail; | 11207 | goto fail; |
| @@ -11180,7 +11323,7 @@ static bool dm_edid_parser_send_cea(struct amdgpu_display_manager *dm, · 115 unmodified lines | |||
| 11180 | sizeof(cmd.edid_cea) - sizeof(cmd.edid_cea.header); | 11323 | sizeof(cmd.edid_cea) - sizeof(cmd.edid_cea.header); |
| 11181 | input->offset = offset; | 11324 | input->offset = offset; |
| 11182 | input->length = length; | 11325 | input->length = length; |
| 11183 | input->total_length = total_length; | 11326 | input->cea_total_length = total_length; |
| 11184 | memcpy(input->payload, data, length); | 11327 | memcpy(input->payload, data, length); |
| 11185 | 11328 | ||
| 11186 | res = dc_dmub_srv_cmd_with_reply_data(dm->dc->ctx->dmub_srv, &cmd); | 11329 | res = dc_dmub_srv_cmd_with_reply_data(dm->dc->ctx->dmub_srv, &cmd); |
| @@ -11487,8 +11630,10 @@ uint32_t dm_read_reg_func(const struct dc_context *ctx, uint32_t address, · 300 unmodified lines | |||
| 11487 | return value; | 11630 | return value; |
| 11488 | } | 11631 | } |
| 11489 | 11632 | ||
| 11490 | int amdgpu_dm_set_dmub_async_sync_status(bool is_cmd_aux, struct dc_context *ctx, | 11633 | static int amdgpu_dm_set_dmub_async_sync_status(bool is_cmd_aux, |
| 11491 | uint8_t status_type, uint32_t *operation_result) | 11634 | struct dc_context *ctx, |
| 11635 | uint8_t status_type, | ||
| 11636 | uint32_t *operation_result) | ||
| 11492 | { | 11637 | { |
| 11493 | struct amdgpu_device *adev = ctx->driver_context; | 11638 | struct amdgpu_device *adev = ctx->driver_context; |
| 11494 | int return_status = -1; | 11639 | int return_status = -1; |
| @@ -11559,3 +11704,24 @@ int amdgpu_dm_process_dmub_aux_transfer_sync(bool is_cmd_aux, struct dc_context · 64 unmodified lines | |||
| 11559 | ctx, DMUB_ASYNC_TO_SYNC_ACCESS_SUCCESS, | 11704 | ctx, DMUB_ASYNC_TO_SYNC_ACCESS_SUCCESS, |
| 11560 | (uint32_t *)operation_result); | 11705 | (uint32_t *)operation_result); |
| 11561 | } | 11706 | } |
| 11707 | |||
| 11708 | /* | ||
| 11709 | * Check whether seamless boot is supported. | ||
| 11710 | * | ||
| 11711 | * So far we only support seamless boot on CHIP_VANGOGH. | ||
| 11712 | * If everything goes well, we may consider expanding | ||
| 11713 | * seamless boot to other ASICs. | ||
| 11714 | */ | ||
| 11715 | bool check_seamless_boot_capability(struct amdgpu_device *adev) | ||
| 11716 | { | ||
| 11717 | switch (adev->asic_type) { | ||
| 11718 | case CHIP_VANGOGH: | ||
| 11719 | if (!adev->mman.keep_stolen_vga_memory) | ||
| 11720 | return true; | ||
| 11721 | break; | ||
| 11722 | default: | ||
| 11723 | break; | ||
| 11724 | } | ||
| 11725 | |||
| 11726 | return false; | ||
| 11727 | } | ||
| @@ -50,9 +50,9 @@ · 49 unmodified lines | |||
| 50 | 50 | ||
| 51 | #define AMDGPU_DMUB_NOTIFICATION_MAX 5 | 51 | #define AMDGPU_DMUB_NOTIFICATION_MAX 5 |
| 52 | 52 | ||
| 53 | /** | 53 | /* |
| 54 | * DMUB Async to Sync Mechanism Status | 54 | * DMUB Async to Sync Mechanism Status |
| 55 | **/ | 55 | */ |
| 56 | #define DMUB_ASYNC_TO_SYNC_ACCESS_FAIL 1 | 56 | #define DMUB_ASYNC_TO_SYNC_ACCESS_FAIL 1 |
| 57 | #define DMUB_ASYNC_TO_SYNC_ACCESS_TIMEOUT 2 | 57 | #define DMUB_ASYNC_TO_SYNC_ACCESS_TIMEOUT 2 |
| 58 | #define DMUB_ASYNC_TO_SYNC_ACCESS_SUCCESS 3 | 58 | #define DMUB_ASYNC_TO_SYNC_ACCESS_SUCCESS 3 |
| @@ -731,4 +731,7 @@ extern const struct drm_encoder_helper_funcs amdgpu_dm_encoder_helper_funcs; · 672 unmodified lines | |||
| 731 | int amdgpu_dm_process_dmub_aux_transfer_sync(bool is_cmd_aux, | 731 | int amdgpu_dm_process_dmub_aux_transfer_sync(bool is_cmd_aux, |
| 732 | struct dc_context *ctx, unsigned int link_index, | 732 | struct dc_context *ctx, unsigned int link_index, |
| 733 | void *payload, void *operation_result); | 733 | void *payload, void *operation_result); |
| 734 | |||
| 735 | bool check_seamless_boot_capability(struct amdgpu_device *adev); | ||
| 736 | |||
| 734 | #endif /* __AMDGPU_DM_H__ */ | 737 | #endif /* __AMDGPU_DM_H__ */ |
| @@ -285,8 +285,12 @@ static int __set_input_tf(struct dc_transfer_func *func, · 284 unmodified lines | |||
| 285 | } | 285 | } |
| 286 | 286 | ||
| 287 | /** | 287 | /** |
| 288 | * amdgpu_dm_verify_lut_sizes | ||
| 289 | * @crtc_state: the DRM CRTC state | ||
| 290 | * | ||
| 288 | * Verifies that the Degamma and Gamma LUTs attached to the |crtc_state| are of | 291 | * Verifies that the Degamma and Gamma LUTs attached to the |crtc_state| are of |
| 289 | * the expected size. | 292 | * the expected size. |
| 293 | * | ||
| 290 | * Returns 0 on success. | 294 | * Returns 0 on success. |
| 291 | */ | 295 | */ |
| 292 | int amdgpu_dm_verify_lut_sizes(const struct drm_crtc_state *crtc_state) | 296 | int amdgpu_dm_verify_lut_sizes(const struct drm_crtc_state *crtc_state) |
| @@ -824,6 +824,48 @@ static int dmub_fw_state_show(struct seq_file *m, void *data) · 823 unmodified lines | |||
| 824 | return seq_write(m, state_base, state_size); | 824 | return seq_write(m, state_base, state_size); |
| 825 | } | 825 | } |
| 826 | 826 | ||
| 827 | /* psr_capability_show() - show eDP panel PSR capability | ||
| 828 | * | ||
| 829 | * The read function: sink_psr_capability_show | ||
| 830 | * Shows if sink has PSR capability or not. | ||
| 831 | * If yes - the PSR version is appended | ||
| 832 | * | ||
| 833 | * cat /sys/kernel/debug/dri/0/eDP-X/psr_capability | ||
| 834 | * | ||
| 835 | * Expected output: | ||
| 836 | * "Sink support: no\n" - if panel doesn't support PSR | ||
| 837 | * "Sink support: yes [0x01]\n" - if panel supports PSR1 | ||
| 838 | * "Driver support: no\n" - if driver doesn't support PSR | ||
| 839 | * "Driver support: yes [0x01]\n" - if driver supports PSR1 | ||
| 840 | */ | ||
| 841 | static int psr_capability_show(struct seq_file *m, void *data) | ||
| 842 | { | ||
| 843 | struct drm_connector *connector = m->private; | ||
| 844 | struct amdgpu_dm_connector *aconnector = to_amdgpu_dm_connector(connector); | ||
| 845 | struct dc_link *link = aconnector->dc_link; | ||
| 846 | |||
| 847 | if (!link) | ||
| 848 | return -ENODEV; | ||
| 849 | |||
| 850 | if (link->type == dc_connection_none) | ||
| 851 | return -ENODEV; | ||
| 852 | |||
| 853 | if (!(link->connector_signal & SIGNAL_TYPE_EDP)) | ||
| 854 | return -ENODEV; | ||
| 855 | |||
| 856 | seq_printf(m, "Sink support: %s", yesno(link->dpcd_caps.psr_caps.psr_version != 0)); | ||
| 857 | if (link->dpcd_caps.psr_caps.psr_version) | ||
| 858 | seq_printf(m, " [0x%02x]", link->dpcd_caps.psr_caps.psr_version); | ||
| 859 | seq_puts(m, "\n"); | ||
| 860 | |||
| 861 | seq_printf(m, "Driver support: %s", yesno(link->psr_settings.psr_feature_enabled)); | ||
| 862 | if (link->psr_settings.psr_version) | ||
| 863 | seq_printf(m, " [0x%02x]", link->psr_settings.psr_version); | ||
| 864 | seq_puts(m, "\n"); | ||
| 865 | |||
| 866 | return 0; | ||
| 867 | } | ||
| 868 | |||
| 827 | /* | 869 | /* |
| 828 | * Returns the current and maximum output bpc for the connector. | 870 | * Returns the current and maximum output bpc for the connector. |
| 829 | * Example usage: cat /sys/kernel/debug/dri/0/DP-1/output_bpc | 871 | * Example usage: cat /sys/kernel/debug/dri/0/DP-1/output_bpc |
| @@ -1243,7 +1285,7 @@ static ssize_t trigger_hotplug(struct file *f, const char __user *buf, · 413 unmodified lines | |||
| 1243 | dm_restore_drm_connector_state(dev, connector); | 1285 | dm_restore_drm_connector_state(dev, connector); |
| 1244 | drm_modeset_unlock_all(dev); | 1286 | drm_modeset_unlock_all(dev); |
| 1245 | 1287 | ||
| 1246 | drm_kms_helper_hotplug_event(dev); | 1288 | drm_kms_helper_connector_hotplug_event(connector); |
| 1247 | } else if (param[0] == 0) { | 1289 | } else if (param[0] == 0) { |
| 1248 | if (!aconnector->dc_link) | 1290 | if (!aconnector->dc_link) |
| 1249 | goto unlock; | 1291 | goto unlock; |
| @@ -1265,7 +1307,7 @@ static ssize_t trigger_hotplug(struct file *f, const char __user *buf, · 15 unmodified lines | |||
| 1265 | dm_restore_drm_connector_state(dev, connector); | 1307 | dm_restore_drm_connector_state(dev, connector); |
| 1266 | drm_modeset_unlock_all(dev); | 1308 | drm_modeset_unlock_all(dev); |
| 1267 | 1309 | ||
| 1268 | drm_kms_helper_hotplug_event(dev); | 1310 | drm_kms_helper_connector_hotplug_event(connector); |
| 1269 | } | 1311 | } |
| 1270 | 1312 | ||
| 1271 | unlock: | 1313 | unlock: |
| @@ -2467,6 +2509,7 @@ DEFINE_SHOW_ATTRIBUTE(dp_lttpr_status); · 1195 unmodified lines | |||
| 2467 | DEFINE_SHOW_ATTRIBUTE(hdcp_sink_capability); | 2509 | DEFINE_SHOW_ATTRIBUTE(hdcp_sink_capability); |
| 2468 | #endif | 2510 | #endif |
| 2469 | DEFINE_SHOW_ATTRIBUTE(internal_display); | 2511 | DEFINE_SHOW_ATTRIBUTE(internal_display); |
| 2512 | DEFINE_SHOW_ATTRIBUTE(psr_capability); | ||
| 2470 | 2513 | ||
| 2471 | static const struct file_operations dp_dsc_clock_en_debugfs_fops = { | 2514 | static const struct file_operations dp_dsc_clock_en_debugfs_fops = { |
| 2472 | .owner = THIS_MODULE, | 2515 | .owner = THIS_MODULE, |
| @@ -2712,6 +2755,138 @@ static const struct { · 239 unmodified lines | |||
| 2712 | {"internal_display", &internal_display_fops} | 2755 | {"internal_display", &internal_display_fops} |
| 2713 | }; | 2756 | }; |
| 2714 | 2757 | ||
| 2758 | /* | ||
| 2759 | * Returns supported customized link rates by this eDP panel. | ||
| 2760 | * Example usage: cat /sys/kernel/debug/dri/0/eDP-x/ilr_setting | ||
| 2761 | */ | ||
| 2762 | static int edp_ilr_show(struct seq_file *m, void *unused) | ||
| 2763 | { | ||
| 2764 | struct amdgpu_dm_connector *aconnector = to_amdgpu_dm_connector(m->private); | ||
| 2765 | struct dc_link *link = aconnector->dc_link; | ||
| 2766 | uint8_t supported_link_rates[16]; | ||
| 2767 | uint32_t link_rate_in_khz; | ||
| 2768 | uint32_t entry = 0; | ||
| 2769 | uint8_t dpcd_rev; | ||
| 2770 | |||
| 2771 | memset(supported_link_rates, 0, sizeof(supported_link_rates)); | ||
| 2772 | dm_helpers_dp_read_dpcd(link->ctx, link, DP_SUPPORTED_LINK_RATES, | ||
| 2773 | supported_link_rates, sizeof(supported_link_rates)); | ||
| 2774 | |||
| 2775 | dpcd_rev = link->dpcd_caps.dpcd_rev.raw; | ||
| 2776 | |||
| 2777 | if (dpcd_rev >= DP_DPCD_REV_13 && | ||
| 2778 | (supported_link_rates[entry+1] != 0 || supported_link_rates[entry] != 0)) { | ||
| 2779 | |||
| 2780 | for (entry = 0; entry < 16; entry += 2) { | ||
| 2781 | link_rate_in_khz = (supported_link_rates[entry+1] * 0x100 + | ||
| 2782 | supported_link_rates[entry]) * 200; | ||
| 2783 | seq_printf(m, "[%d] %d kHz\n", entry/2, link_rate_in_khz); | ||
| 2784 | } | ||
| 2785 | } else { | ||
| 2786 | seq_printf(m, "ILR is not supported by this eDP panel.\n"); | ||
| 2787 | } | ||
| 2788 | |||
| 2789 | return 0; | ||
| 2790 | } | ||
| 2791 | |||
| 2792 | /* | ||
| 2793 | * Set supported customized link rate to eDP panel. | ||
| 2794 | * | ||
| 2795 | * echo <lane_count> <link_rate option> > ilr_setting | ||
| 2796 | * | ||
| 2797 | * for example, supported ILR : [0] 1620000 kHz [1] 2160000 kHz [2] 2430000 kHz ... | ||
| 2798 | * echo 4 1 > /sys/kernel/debug/dri/0/eDP-x/ilr_setting | ||
| 2799 | * to set 4 lanes and 2.16 GHz | ||
| 2800 | */ | ||
| 2801 | static ssize_t edp_ilr_write(struct file *f, const char __user *buf, | ||
| 2802 | size_t size, loff_t *pos) | ||
| 2803 | { | ||
| 2804 | struct amdgpu_dm_connector *connector = file_inode(f)->i_private; | ||
| 2805 | struct dc_link *link = connector->dc_link; | ||
| 2806 | struct amdgpu_device *adev = drm_to_adev(connector->base.dev); | ||
| 2807 | struct dc *dc = (struct dc *)link->dc; | ||
| 2808 | struct dc_link_settings prefer_link_settings; | ||
| 2809 | char *wr_buf = NULL; | ||
| 2810 | const uint32_t wr_buf_size = 40; | ||
| 2811 | /* 0: lane_count; 1: link_rate */ | ||
| 2812 | int max_param_num = 2; | ||
| 2813 | uint8_t param_nums = 0; | ||
| 2814 | long param[2]; | ||
| 2815 | bool valid_input = true; | ||
| 2816 | |||
| 2817 | if (size == 0) | ||
| 2818 | return -EINVAL; | ||
| 2819 | |||
| 2820 | wr_buf = kcalloc(wr_buf_size, sizeof(char), GFP_KERNEL); | ||
| 2821 | if (!wr_buf) | ||
| 2822 | return -ENOMEM; | ||
| 2823 | |||
| 2824 | if (parse_write_buffer_into_params(wr_buf, wr_buf_size, | ||
| 2825 | (long *)param, buf, | ||
| 2826 | max_param_num, | ||
| 2827 | ¶m_nums)) { | ||
| 2828 | kfree(wr_buf); | ||
| 2829 | return -EINVAL; | ||
| 2830 | } | ||
| 2831 | |||
| 2832 | if (param_nums <= 0) { | ||
| 2833 | kfree(wr_buf); | ||
| 2834 | return -EINVAL; | ||
| 2835 | } | ||
| 2836 | |||
| 2837 | switch (param[0]) { | ||
| 2838 | case LANE_COUNT_ONE: | ||
| 2839 | case LANE_COUNT_TWO: | ||
| 2840 | case LANE_COUNT_FOUR: | ||
| 2841 | break; | ||
| 2842 | default: | ||
| 2843 | valid_input = false; | ||
| 2844 | break; | ||
| 2845 | } | ||
| 2846 | |||
| 2847 | if (param[1] >= link->dpcd_caps.edp_supported_link_rates_count) | ||
| 2848 | valid_input = false; | ||
| 2849 | |||
| 2850 | if (!valid_input) { | ||
| 2851 | kfree(wr_buf); | ||
| 2852 | DRM_DEBUG_DRIVER("Invalid Input value. No HW will be programmed\n"); | ||
| 2853 | prefer_link_settings.use_link_rate_set = false; | ||
| 2854 | dc_link_set_preferred_training_settings(dc, NULL, NULL, link, true); | ||
| 2855 | return size; | ||
| 2856 | } | ||
| 2857 | |||
| 2858 | /* save user force lane_count, link_rate to preferred settings | ||
| 2859 | * spread spectrum will not be changed | ||
| 2860 | */ | ||
| 2861 | prefer_link_settings.link_spread = link->cur_link_settings.link_spread; | ||
| 2862 | prefer_link_settings.lane_count = param[0]; | ||
| 2863 | prefer_link_settings.use_link_rate_set = true; | ||
| 2864 | prefer_link_settings.link_rate_set = param[1]; | ||
| 2865 | prefer_link_settings.link_rate = link->dpcd_caps.edp_supported_link_rates[param[1]]; | ||
| 2866 | |||
| 2867 | mutex_lock(&adev->dm.dc_lock); | ||
| 2868 | dc_link_set_preferred_training_settings(dc, &prefer_link_settings, | ||
| 2869 | NULL, link, false); | ||
| 2870 | mutex_unlock(&adev->dm.dc_lock); | ||
| 2871 | |||
| 2872 | kfree(wr_buf); | ||
| 2873 | return size; | ||
| 2874 | } | ||
| 2875 | |||
| 2876 | static int edp_ilr_open(struct inode *inode, struct file *file) | ||
| 2877 | { | ||
| 2878 | return single_open(file, edp_ilr_show, inode->i_private); | ||
| 2879 | } | ||
| 2880 | |||
| 2881 | static const struct file_operations edp_ilr_debugfs_fops = { | ||
| 2882 | .owner = THIS_MODULE, | ||
| 2883 | .open = edp_ilr_open, | ||
| 2884 | .read = seq_read, | ||
| 2885 | .llseek = seq_lseek, | ||
| 2886 | .release = single_release, | ||
| 2887 | .write = edp_ilr_write | ||
| 2888 | }; | ||
| 2889 | |||
| 2715 | void connector_debugfs_init(struct amdgpu_dm_connector *connector) | 2890 | void connector_debugfs_init(struct amdgpu_dm_connector *connector) |
| 2716 | { | 2891 | { |
| 2717 | int i; | 2892 | int i; |
| @@ -2726,11 +2901,14 @@ void connector_debugfs_init(struct amdgpu_dm_connector *connector) · 8 unmodified lines | |||
| 2726 | } | 2901 | } |
| 2727 | } | 2902 | } |
| 2728 | if (connector->base.connector_type == DRM_MODE_CONNECTOR_eDP) { | 2903 | if (connector->base.connector_type == DRM_MODE_CONNECTOR_eDP) { |
| 2904 | debugfs_create_file_unsafe("psr_capability", 0444, dir, connector, &psr_capability_fops); | ||
| 2729 | debugfs_create_file_unsafe("psr_state", 0444, dir, connector, &psr_fops); | 2905 | debugfs_create_file_unsafe("psr_state", 0444, dir, connector, &psr_fops); |
| 2730 | debugfs_create_file("amdgpu_current_backlight_pwm", 0444, dir, connector, | 2906 | debugfs_create_file("amdgpu_current_backlight_pwm", 0444, dir, connector, |
| 2731 | ¤t_backlight_fops); | 2907 | ¤t_backlight_fops); |
| 2732 | debugfs_create_file("amdgpu_target_backlight_pwm", 0444, dir, connector, | 2908 | debugfs_create_file("amdgpu_target_backlight_pwm", 0444, dir, connector, |
| 2733 | &target_backlight_fops); | 2909 | &target_backlight_fops); |
| 2910 | debugfs_create_file("ilr_setting", 0644, dir, connector, | ||
| 2911 | &edp_ilr_debugfs_fops); | ||
| 2734 | } | 2912 | } |
| 2735 | 2913 | ||
| 2736 | for (i = 0; i < ARRAY_SIZE(connector_debugfs_entries); i++) { | 2914 | for (i = 0; i < ARRAY_SIZE(connector_debugfs_entries); i++) { |
| @@ -2909,10 +3087,13 @@ static int crc_win_update_set(void *data, u64 val) · 172 unmodified lines | |||
| 2909 | struct amdgpu_device *adev = drm_to_adev(new_crtc->dev); | 3087 | struct amdgpu_device *adev = drm_to_adev(new_crtc->dev); |
| 2910 | struct crc_rd_work *crc_rd_wrk = adev->dm.crc_rd_wrk; | 3088 | struct crc_rd_work *crc_rd_wrk = adev->dm.crc_rd_wrk; |
| 2911 | 3089 | ||
| 3090 | if (!crc_rd_wrk) | ||
| 3091 | return 0; | ||
| 3092 | |||
| 2912 | if (val) { | 3093 | if (val) { |
| 2913 | spin_lock_irq(&adev_to_drm(adev)->event_lock); | 3094 | spin_lock_irq(&adev_to_drm(adev)->event_lock); |
| 2914 | spin_lock_irq(&crc_rd_wrk->crc_rd_work_lock); | 3095 | spin_lock_irq(&crc_rd_wrk->crc_rd_work_lock); |
| 2915 | if (crc_rd_wrk && crc_rd_wrk->crtc) { | 3096 | if (crc_rd_wrk->crtc) { |
| 2916 | old_crtc = crc_rd_wrk->crtc; | 3097 | old_crtc = crc_rd_wrk->crtc; |
| 2917 | old_acrtc = to_amdgpu_crtc(old_crtc); | 3098 | old_acrtc = to_amdgpu_crtc(old_crtc); |
| 2918 | } | 3099 | } |
| @@ -3190,6 +3371,32 @@ static int disable_hpd_get(void *data, u64 *val) · 271 unmodified lines | |||
| 3190 | DEFINE_DEBUGFS_ATTRIBUTE(disable_hpd_ops, disable_hpd_get, | 3371 | DEFINE_DEBUGFS_ATTRIBUTE(disable_hpd_ops, disable_hpd_get, |
| 3191 | disable_hpd_set, "%llu\n"); | 3372 | disable_hpd_set, "%llu\n"); |
| 3192 | 3373 | ||
| 3374 | #if defined(CONFIG_DRM_AMD_DC_DCN) | ||
| 3375 | /* | ||
| 3376 | * Temporary w/a to force sst sequence in M42D DP2 mst receiver | ||
| 3377 | * Example usage: echo 1 > /sys/kernel/debug/dri/0/amdgpu_dm_dp_set_mst_en_for_sst | ||
| 3378 | */ | ||
| 3379 | static int dp_force_sst_set(void *data, u64 val) | ||
| 3380 | { | ||
| 3381 | struct amdgpu_device *adev = data; | ||
| 3382 | |||
| 3383 | adev->dm.dc->debug.set_mst_en_for_sst = val; | ||
| 3384 | |||
| 3385 | return 0; | ||
| 3386 | } | ||
| 3387 | |||
| 3388 | static int dp_force_sst_get(void *data, u64 *val) | ||
| 3389 | { | ||
| 3390 | struct amdgpu_device *adev = data; | ||
| 3391 | |||
| 3392 | *val = adev->dm.dc->debug.set_mst_en_for_sst; | ||
| 3393 | |||
| 3394 | return 0; | ||
| 3395 | } | ||
| 3396 | DEFINE_DEBUGFS_ATTRIBUTE(dp_set_mst_en_for_sst_ops, dp_force_sst_get, | ||
| 3397 | dp_force_sst_set, "%llu\n"); | ||
| 3398 | #endif | ||
| 3399 | |||
| 3193 | /* | 3400 | /* |
| 3194 | * Sets the DC visual confirm debug option from the given string. | 3401 | * Sets the DC visual confirm debug option from the given string. |
| 3195 | * Example usage: echo 1 > /sys/kernel/debug/dri/0/amdgpu_visual_confirm | 3402 | * Example usage: echo 1 > /sys/kernel/debug/dri/0/amdgpu_visual_confirm |
| @@ -3299,6 +3506,10 @@ void dtn_debugfs_init(struct amdgpu_device *adev) · 103 unmodified lines | |||
| 3299 | adev, &mst_topo_fops); | 3506 | adev, &mst_topo_fops); |
| 3300 | debugfs_create_file("amdgpu_dm_dtn_log", 0644, root, adev, | 3507 | debugfs_create_file("amdgpu_dm_dtn_log", 0644, root, adev, |
| 3301 | &dtn_log_fops); | 3508 | &dtn_log_fops); |
| 3509 | #if defined(CONFIG_DRM_AMD_DC_DCN) | ||
| 3510 | debugfs_create_file("amdgpu_dm_dp_set_mst_en_for_sst", 0644, root, adev, | ||
| 3511 | &dp_set_mst_en_for_sst_ops); | ||
| 3512 | #endif | ||
| 3302 | 3513 | ||
| 3303 | debugfs_create_file_unsafe("amdgpu_dm_visual_confirm", 0644, root, adev, | 3514 | debugfs_create_file_unsafe("amdgpu_dm_visual_confirm", 0644, root, adev, |
| 3304 | &visual_confirm_fops); | 3515 | &visual_confirm_fops); |
| @@ -83,16 +83,17 @@ static int amdgpu_dm_patch_edid_caps(struct dc_edid_caps *edid_caps) · 82 unmodified lines | |||
| 83 | * void | 83 | * void |
| 84 | * */ | 84 | * */ |
| 85 | enum dc_edid_status dm_helpers_parse_edid_caps( | 85 | enum dc_edid_status dm_helpers_parse_edid_caps( |
| 86 | struct dc_context *ctx, | 86 | struct dc_link *link, |
| 87 | const struct dc_edid *edid, | 87 | const struct dc_edid *edid, |
| 88 | struct dc_edid_caps *edid_caps) | 88 | struct dc_edid_caps *edid_caps) |
| 89 | { | 89 | { |
| 90 | struct amdgpu_dm_connector *aconnector = link->priv; | ||
| 91 | struct drm_connector *connector = &aconnector->base; | ||
| 90 | struct edid *edid_buf = (struct edid *) edid->raw_edid; | 92 | struct edid *edid_buf = (struct edid *) edid->raw_edid; |
| 91 | struct cea_sad *sads; | 93 | struct cea_sad *sads; |
| 92 | int sad_count = -1; | 94 | int sad_count = -1; |
| 93 | int sadb_count = -1; | 95 | int sadb_count = -1; |
| 94 | int i = 0; | 96 | int i = 0; |
| 95 | int j = 0; | ||
| 96 | uint8_t *sadb = NULL; | 97 | uint8_t *sadb = NULL; |
| 97 | 98 | ||
| 98 | enum dc_edid_status result = EDID_OK; | 99 | enum dc_edid_status result = EDID_OK; |
| @@ -111,23 +112,11 @@ enum dc_edid_status dm_helpers_parse_edid_caps( · 12 unmodified lines | |||
| 111 | edid_caps->manufacture_week = edid_buf->mfg_week; | 112 | edid_caps->manufacture_week = edid_buf->mfg_week; |
| 112 | edid_caps->manufacture_year = edid_buf->mfg_year; | 113 | edid_caps->manufacture_year = edid_buf->mfg_year; |
| 113 | 114 | ||
| 114 | /* One of the four detailed_timings stores the monitor name. It's | 115 | drm_edid_get_monitor_name(edid_buf, |
| 115 | * stored in an array of length 13. */ | 116 | edid_caps->display_name, |
| 116 | for (i = 0; i < 4; i++) { | 117 | AUDIO_INFO_DISPLAY_NAME_SIZE_IN_CHARS); |
| 117 | if (edid_buf->detailed_timings[i].data.other_data.type == 0xfc) { | ||
| 118 | while (j < 13 && edid_buf->detailed_timings[i].data.other_data.data.str.str[j]) { | ||
| 119 | if (edid_buf->detailed_timings[i].data.other_data.data.str.str[j] == '\n') | ||
| 120 | break; | ||
| 121 | |||
| 122 | edid_caps->display_name[j] = | ||
| 123 | edid_buf->detailed_timings[i].data.other_data.data.str.str[j]; | ||
| 124 | j++; | ||
| 125 | } | ||
| 126 | } | ||
| 127 | } | ||
| 128 | 118 | ||
| 129 | edid_caps->edid_hdmi = drm_detect_hdmi_monitor( | 119 | edid_caps->edid_hdmi = connector->display_info.is_hdmi; |
| 130 | (struct edid *) edid->raw_edid); | ||
| 131 | 120 | ||
| 132 | sad_count = drm_edid_to_sad((struct edid *) edid->raw_edid, &sads); | 121 | sad_count = drm_edid_to_sad((struct edid *) edid->raw_edid, &sads); |
| 133 | if (sad_count <= 0) | 122 | if (sad_count <= 0) |
| @@ -584,9 +573,18 @@ bool dm_helpers_dp_write_dsc_enable( · 450 unmodified lines | |||
| 584 | ret = drm_dp_dpcd_write(aconnector->dsc_aux, DP_DSC_ENABLE, &enable_dsc, 1); | 573 | ret = drm_dp_dpcd_write(aconnector->dsc_aux, DP_DSC_ENABLE, &enable_dsc, 1); |
| 585 | } | 574 | } |
| 586 | 575 | ||
| 587 | if (stream->signal == SIGNAL_TYPE_DISPLAY_PORT) { | 576 | if (stream->signal == SIGNAL_TYPE_DISPLAY_PORT || stream->signal == SIGNAL_TYPE_EDP) { |
| 588 | ret = dm_helpers_dp_write_dpcd(ctx, stream->link, DP_DSC_ENABLE, &enable_dsc, 1); | 577 | #if defined(CONFIG_DRM_AMD_DC_DCN) |
| 589 | DC_LOG_DC("Send DSC %s to sst display\n", enable_dsc ? "enable" : "disable"); | 578 | if (stream->sink->link->dpcd_caps.dongle_type == DISPLAY_DONGLE_NONE) { |
| 579 | #endif | ||
| 580 | ret = dm_helpers_dp_write_dpcd(ctx, stream->link, DP_DSC_ENABLE, &enable_dsc, 1); | ||
| 581 | DC_LOG_DC("Send DSC %s to SST RX\n", enable_dsc ? "enable" : "disable"); | ||
| 582 | #if defined(CONFIG_DRM_AMD_DC_DCN) | ||
| 583 | } else if (stream->sink->link->dpcd_caps.dongle_type == DISPLAY_DONGLE_DP_HDMI_CONVERTER) { | ||
| 584 | ret = dm_helpers_dp_write_dpcd(ctx, stream->link, DP_DSC_ENABLE, &enable_dsc, 1); | ||
| 585 | DC_LOG_DC("Send DSC %s to DP-HDMI PCON\n", enable_dsc ? "enable" : "disable"); | ||
| 586 | } | ||
| 587 | #endif | ||
| 590 | } | 588 | } |
| 591 | 589 | ||
| 592 | return (ret > 0); | 590 | return (ret > 0); |
| @@ -650,14 +648,8 @@ enum dc_edid_status dm_helpers_read_local_edid( · 57 unmodified lines | |||
| 650 | /* We don't need the original edid anymore */ | 648 | /* We don't need the original edid anymore */ |
| 651 | kfree(edid); | 649 | kfree(edid); |
| 652 | 650 | ||
| 653 | /* connector->display_info will be parsed from EDID and saved | 651 | /* connector->display_info is parsed from EDID and saved |
| 654 | * into drm_connector->display_info from edid by call stack | 652 | * into drm_connector->display_info |
| 655 | * below: | ||
| 656 | * drm_parse_ycbcr420_deep_color_info | ||
| 657 | * drm_parse_hdmi_forum_vsdb | ||
| 658 | * drm_parse_cea_ext | ||
| 659 | * drm_add_display_info | ||
| 660 | * drm_connector_update_edid_property | ||
| 661 | * | 653 | * |
| 662 | * drm_connector->display_info will be used by amdgpu_dm funcs, | 654 | * drm_connector->display_info will be used by amdgpu_dm funcs, |
| 663 | * like fill_stream_properties_from_drm_display_mode | 655 | * like fill_stream_properties_from_drm_display_mode |
| @@ -665,7 +657,7 @@ enum dc_edid_status dm_helpers_read_local_edid( · 1 unmodified line | |||
| 665 | amdgpu_dm_update_connector_after_detect(aconnector); | 657 | amdgpu_dm_update_connector_after_detect(aconnector); |
| 666 | 658 | ||
| 667 | edid_status = dm_helpers_parse_edid_caps( | 659 | edid_status = dm_helpers_parse_edid_caps( |
| 668 | ctx, | 660 | link, |
| 669 | &sink->dc_edid, | 661 | &sink->dc_edid, |
| 670 | &sink->edid_caps); | 662 | &sink->edid_caps); |
| 671 | 663 | ||
| @@ -26,6 +26,73 @@ · 25 unmodified lines | |||
| 26 | #include "amdgpu_dm_psr.h" | 26 | #include "amdgpu_dm_psr.h" |
| 27 | #include "dc.h" | 27 | #include "dc.h" |
| 28 | #include "dm_helpers.h" | 28 | #include "dm_helpers.h" |
| 29 | #include "amdgpu_dm.h" | ||
| 30 | |||
| 31 | static bool link_get_psr_caps(struct dc_link *link) | ||
| 32 | { | ||
| 33 | uint8_t psr_dpcd_data[EDP_PSR_RECEIVER_CAP_SIZE]; | ||
| 34 | uint8_t edp_rev_dpcd_data; | ||
| 35 | |||
| 36 | |||
| 37 | |||
| 38 | if (!dm_helpers_dp_read_dpcd(NULL, link, DP_PSR_SUPPORT, | ||
| 39 | psr_dpcd_data, sizeof(psr_dpcd_data))) | ||
| 40 | return false; | ||
| 41 | |||
| 42 | if (!dm_helpers_dp_read_dpcd(NULL, link, DP_EDP_DPCD_REV, | ||
| 43 | &edp_rev_dpcd_data, sizeof(edp_rev_dpcd_data))) | ||
| 44 | return false; | ||
| 45 | |||
| 46 | link->dpcd_caps.psr_caps.psr_version = psr_dpcd_data[0]; | ||
| 47 | link->dpcd_caps.psr_caps.edp_revision = edp_rev_dpcd_data; | ||
| 48 | |||
| 49 | #ifdef CONFIG_DRM_AMD_DC_DCN | ||
| 50 | if (link->dpcd_caps.psr_caps.psr_version > 0x1) { | ||
| 51 | uint8_t alpm_dpcd_data; | ||
| 52 | uint8_t su_granularity_dpcd_data; | ||
| 53 | |||
| 54 | if (!dm_helpers_dp_read_dpcd(NULL, link, DP_RECEIVER_ALPM_CAP, | ||
| 55 | &alpm_dpcd_data, sizeof(alpm_dpcd_data))) | ||
| 56 | return false; | ||
| 57 | |||
| 58 | if (!dm_helpers_dp_read_dpcd(NULL, link, DP_PSR2_SU_Y_GRANULARITY, | ||
| 59 | &su_granularity_dpcd_data, sizeof(su_granularity_dpcd_data))) | ||
| 60 | return false; | ||
| 61 | |||
| 62 | link->dpcd_caps.psr_caps.y_coordinate_required = psr_dpcd_data[1] & DP_PSR2_SU_Y_COORDINATE_REQUIRED; | ||
| 63 | link->dpcd_caps.psr_caps.su_granularity_required = psr_dpcd_data[1] & DP_PSR2_SU_GRANULARITY_REQUIRED; | ||
| 64 | |||
| 65 | link->dpcd_caps.psr_caps.alpm_cap = alpm_dpcd_data & DP_ALPM_CAP; | ||
| 66 | link->dpcd_caps.psr_caps.standby_support = alpm_dpcd_data & (1 << 1); | ||
| 67 | |||
| 68 | link->dpcd_caps.psr_caps.su_y_granularity = su_granularity_dpcd_data; | ||
| 69 | } | ||
| 70 | #endif | ||
| 71 | return true; | ||
| 72 | } | ||
| 73 | |||
| 74 | #ifdef CONFIG_DRM_AMD_DC_DCN | ||
| 75 | static bool link_supports_psrsu(struct dc_link *link) | ||
| 76 | { | ||
| 77 | struct dc *dc = link->ctx->dc; | ||
| 78 | |||
| 79 | if (!dc->caps.dmcub_support) | ||
| 80 | return false; | ||
| 81 | |||
| 82 | if (dc->ctx->dce_version < DCN_VERSION_3_1) | ||
| 83 | return false; | ||
| 84 | |||
| 85 | if (!link->dpcd_caps.psr_caps.alpm_cap || | ||
| 86 | !link->dpcd_caps.psr_caps.y_coordinate_required) | ||
| 87 | return false; | ||
| 88 | |||
| 89 | if (link->dpcd_caps.psr_caps.su_granularity_required && | ||
| 90 | !link->dpcd_caps.psr_caps.su_y_granularity) | ||
| 91 | return false; | ||
| 92 | |||
| 93 | return true; | ||
| 94 | } | ||
| 95 | #endif | ||
| 29 | 96 | ||
| 30 | /* | 97 | /* |
| 31 | * amdgpu_dm_set_psr_caps() - set link psr capabilities | 98 | * amdgpu_dm_set_psr_caps() - set link psr capabilities |
| @@ -34,26 +101,34 @@ · 2 unmodified lines | |||
| 34 | */ | 101 | */ |
| 35 | void amdgpu_dm_set_psr_caps(struct dc_link *link) | 102 | void amdgpu_dm_set_psr_caps(struct dc_link *link) |
| 36 | { | 103 | { |
| 37 | uint8_t dpcd_data[EDP_PSR_RECEIVER_CAP_SIZE]; | ||
| 38 | |||
| 39 | if (!(link->connector_signal & SIGNAL_TYPE_EDP)) | 104 | if (!(link->connector_signal & SIGNAL_TYPE_EDP)) |
| 40 | return; | 105 | return; |
| 106 | |||
| 41 | if (link->type == dc_connection_none) | 107 | if (link->type == dc_connection_none) |
| 42 | return; | 108 | return; |
| 43 | if (dm_helpers_dp_read_dpcd(NULL, link, DP_PSR_SUPPORT, | 109 | |
| 44 | dpcd_data, sizeof(dpcd_data))) { | 110 | if (!link_get_psr_caps(link)) { |
| 45 | link->dpcd_caps.psr_caps.psr_version = dpcd_data[0]; | 111 | DRM_ERROR("amdgpu: Failed to read PSR Caps!\n"); |
| 46 | 112 | return; | |
| 47 | if (dpcd_data[0] == 0) { | 113 | } |
| 48 | link->psr_settings.psr_version = DC_PSR_VERSION_UNSUPPORTED; | 114 | |
| 49 | link->psr_settings.psr_feature_enabled = false; | 115 | if (link->dpcd_caps.psr_caps.psr_version == 0) { |
| 50 | } else { | 116 | link->psr_settings.psr_version = DC_PSR_VERSION_UNSUPPORTED; |
| 117 | link->psr_settings.psr_feature_enabled = false; | ||
| 118 | |||
| 119 | } else { | ||
| 120 | #ifdef CONFIG_DRM_AMD_DC_DCN | ||
| 121 | if (link_supports_psrsu(link)) | ||
| 122 | link->psr_settings.psr_version = DC_PSR_VERSION_SU_1; | ||
| 123 | else | ||
| 124 | #endif | ||
| 51 | link->psr_settings.psr_version = DC_PSR_VERSION_1; | 125 | link->psr_settings.psr_version = DC_PSR_VERSION_1; |
| 52 | link->psr_settings.psr_feature_enabled = true; | ||
| 53 | } | ||
| 54 | 126 | ||
| 55 | DRM_INFO("PSR support:%d\n", link->psr_settings.psr_feature_enabled); | 127 | link->psr_settings.psr_feature_enabled = true; |
| 56 | } | 128 | } |
| 129 | |||
| 130 | DRM_INFO("PSR support:%d\n", link->psr_settings.psr_feature_enabled); | ||
| 131 | |||
| 57 | } | 132 | } |
| 58 | 133 | ||
| 59 | /* | 134 | /* |
| @@ -2995,7 +2995,7 @@ static bool bios_parser2_construct( · 2994 unmodified lines | |||
| 2995 | &bp->object_info_tbl.revision); | 2995 | &bp->object_info_tbl.revision); |
| 2996 | 2996 | ||
| 2997 | if (bp->object_info_tbl.revision.major == 1 | 2997 | if (bp->object_info_tbl.revision.major == 1 |
| 2998 | && bp->object_info_tbl.revision.minor >= 4) { | 2998 | && bp->object_info_tbl.revision.minor == 4) { |
| 2999 | struct display_object_info_table_v1_4 *tbl_v1_4; | 2999 | struct display_object_info_table_v1_4 *tbl_v1_4; |
| 3000 | 3000 | ||
| 3001 | tbl_v1_4 = GET_IMAGE(struct display_object_info_table_v1_4, | 3001 | tbl_v1_4 = GET_IMAGE(struct display_object_info_table_v1_4, |
| @@ -3004,8 +3004,10 @@ static bool bios_parser2_construct( · 2 unmodified lines | |||
| 3004 | return false; | 3004 | return false; |
| 3005 | 3005 | ||
| 3006 | bp->object_info_tbl.v1_4 = tbl_v1_4; | 3006 | bp->object_info_tbl.v1_4 = tbl_v1_4; |
| 3007 | } else | 3007 | } else { |
| 3008 | ASSERT(0); | ||
| 3008 | return false; | 3009 | return false; |
| 3010 | } | ||
| 3009 | 3011 | ||
| 3010 | dal_firmware_parser_init_cmd_tbl(bp); | 3012 | dal_firmware_parser_init_cmd_tbl(bp); |
| 3011 | dal_bios_parser_init_cmd_tbl_helper2(&bp->cmd_helper, dce_version); | 3013 | dal_bios_parser_init_cmd_tbl_helper2(&bp->cmd_helper, dce_version); |
| @@ -739,7 +739,9 @@ static void hack_bounding_box(struct dcn_bw_internal_vars *v, · 738 unmodified lines | |||
| 739 | hack_force_pipe_split(v, context->streams[0]->timing.pix_clk_100hz); | 739 | hack_force_pipe_split(v, context->streams[0]->timing.pix_clk_100hz); |
| 740 | } | 740 | } |
| 741 | 741 | ||
| 742 | unsigned int get_highest_allowed_voltage_level(uint32_t chip_family, uint32_t hw_internal_rev, uint32_t pci_revision_id) | 742 | static unsigned int get_highest_allowed_voltage_level(uint32_t chip_family, |
| 743 | uint32_t hw_internal_rev, | ||
| 744 | uint32_t pci_revision_id) | ||
| 743 | { | 745 | { |
| 744 | /* for low power RV2 variants, the highest voltage level we want is 0 */ | 746 | /* for low power RV2 variants, the highest voltage level we want is 0 */ |
| 745 | if ((chip_family == FAMILY_RV) && | 747 | if ((chip_family == FAMILY_RV) && |
| @@ -763,7 +765,7 @@ unsigned int get_highest_allowed_voltage_level(uint32_t chip_family, uint32_t hw · 17 unmodified lines | |||
| 763 | return 4; | 765 | return 4; |
| 764 | } | 766 | } |
| 765 | 767 | ||
| 766 | bool dcn_validate_bandwidth( | 768 | bool dcn10_validate_bandwidth( |
| 767 | struct dc *dc, | 769 | struct dc *dc, |
| 768 | struct dc_state *context, | 770 | struct dc_state *context, |
| 769 | bool fast_validate) | 771 | bool fast_validate) |
| @@ -308,8 +308,7 @@ void dc_destroy_clk_mgr(struct clk_mgr *clk_mgr_base) · 307 unmodified lines | |||
| 308 | case FAMILY_NV: | 308 | case FAMILY_NV: |
| 309 | if (ASICREV_IS_SIENNA_CICHLID_P(clk_mgr_base->ctx->asic_id.hw_internal_rev)) { | 309 | if (ASICREV_IS_SIENNA_CICHLID_P(clk_mgr_base->ctx->asic_id.hw_internal_rev)) { |
| 310 | dcn3_clk_mgr_destroy(clk_mgr); | 310 | dcn3_clk_mgr_destroy(clk_mgr); |
| 311 | } | 311 | } else if (ASICREV_IS_DIMGREY_CAVEFISH_P(clk_mgr_base->ctx->asic_id.hw_internal_rev)) { |
| 312 | if (ASICREV_IS_DIMGREY_CAVEFISH_P(clk_mgr_base->ctx->asic_id.hw_internal_rev)) { | ||
| 313 | dcn3_clk_mgr_destroy(clk_mgr); | 312 | dcn3_clk_mgr_destroy(clk_mgr); |
| 314 | } | 313 | } |
| 315 | if (ASICREV_IS_BEIGE_GOBY_P(clk_mgr_base->ctx->asic_id.hw_internal_rev)) { | 314 | if (ASICREV_IS_BEIGE_GOBY_P(clk_mgr_base->ctx->asic_id.hw_internal_rev)) { |
| @@ -34,7 +34,7 @@ · 33 unmodified lines | |||
| 34 | #include "rv1_clk_mgr_vbios_smu.h" | 34 | #include "rv1_clk_mgr_vbios_smu.h" |
| 35 | #include "rv1_clk_mgr_clk.h" | 35 | #include "rv1_clk_mgr_clk.h" |
| 36 | 36 | ||
| 37 | void rv1_init_clocks(struct clk_mgr *clk_mgr) | 37 | static void rv1_init_clocks(struct clk_mgr *clk_mgr) |
| 38 | { | 38 | { |
| 39 | memset(&(clk_mgr->clks), 0, sizeof(struct dc_clocks)); | 39 | memset(&(clk_mgr->clks), 0, sizeof(struct dc_clocks)); |
| 40 | } | 40 | } |
| @@ -28,6 +28,8 @@ · 27 unmodified lines | |||
| 28 | #include "reg_helper.h" | 28 | #include "reg_helper.h" |
| 29 | #include <linux/delay.h> | 29 | #include <linux/delay.h> |
| 30 | 30 | ||
| 31 | #include "rv1_clk_mgr_vbios_smu.h" | ||
| 32 | |||
| 31 | #define MAX_INSTANCE 5 | 33 | #define MAX_INSTANCE 5 |
| 32 | #define MAX_SEGMENT 5 | 34 | #define MAX_SEGMENT 5 |
| 33 | 35 | ||
| @@ -409,7 +409,7 @@ void dcn2_init_clocks(struct clk_mgr *clk_mgr) · 408 unmodified lines | |||
| 409 | clk_mgr->clks.prev_p_state_change_support = true; | 409 | clk_mgr->clks.prev_p_state_change_support = true; |
| 410 | } | 410 | } |
| 411 | 411 | ||
| 412 | void dcn2_enable_pme_wa(struct clk_mgr *clk_mgr_base) | 412 | static void dcn2_enable_pme_wa(struct clk_mgr *clk_mgr_base) |
| 413 | { | 413 | { |
| 414 | struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base); | 414 | struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base); |
| 415 | struct pp_smu_funcs_nv *pp_smu = NULL; | 415 | struct pp_smu_funcs_nv *pp_smu = NULL; |
| @@ -74,42 +74,6 @@ static const struct clk_mgr_mask clk_mgr_mask = { · 73 unmodified lines | |||
| 74 | CLK_COMMON_MASK_SH_LIST_DCN201_BASE(_MASK) | 74 | CLK_COMMON_MASK_SH_LIST_DCN201_BASE(_MASK) |
| 75 | }; | 75 | }; |
| 76 | 76 | ||
| 77 | void dcn201_update_clocks_vbios(struct clk_mgr *clk_mgr, | ||
| 78 | struct dc_state *context, | ||
| 79 | bool safe_to_lower) | ||
| 80 | { | ||
| 81 | struct dc_clocks *new_clocks = &context->bw_ctx.bw.dcn.clk; | ||
| 82 | |||
| 83 | bool update_dppclk = false; | ||
| 84 | bool update_dispclk = false; | ||
| 85 | |||
| 86 | if (should_set_clock(safe_to_lower, new_clocks->dppclk_khz, clk_mgr->clks.dppclk_khz)) { | ||
| 87 | clk_mgr->clks.dppclk_khz = new_clocks->dppclk_khz; | ||
| 88 | update_dppclk = true; | ||
| 89 | } | ||
| 90 | |||
| 91 | if (should_set_clock(safe_to_lower, new_clocks->dispclk_khz, clk_mgr->clks.dispclk_khz)) { | ||
| 92 | clk_mgr->clks.dispclk_khz = new_clocks->dispclk_khz; | ||
| 93 | update_dispclk = true; | ||
| 94 | } | ||
| 95 | |||
| 96 | if (update_dppclk || update_dispclk) { | ||
| 97 | struct bp_set_dce_clock_parameters dce_clk_params; | ||
| 98 | struct dc_bios *bp = clk_mgr->ctx->dc_bios; | ||
| 99 | |||
| 100 | if (update_dispclk) { | ||
| 101 | memset(&dce_clk_params, 0, sizeof(dce_clk_params)); | ||
| 102 | dce_clk_params.target_clock_frequency = new_clocks->dispclk_khz; | ||
| 103 | dce_clk_params.pll_id = CLOCK_SOURCE_ID_DFS; | ||
| 104 | dce_clk_params.clock_type = DCECLOCK_TYPE_DISPLAY_CLOCK; | ||
| 105 | bp->funcs->set_dce_clock(bp, &dce_clk_params); | ||
| 106 | } | ||
| 107 | /* currently there is no DCECLOCK_TYPE_DPPCLK type defined in VBIOS interface. | ||
| 108 | * vbios program DPPCLK to the same DispCLK limitation | ||
| 109 | */ | ||
| 110 | } | ||
| 111 | } | ||
| 112 | |||
| 113 | static void dcn201_init_clocks(struct clk_mgr *clk_mgr) | 77 | static void dcn201_init_clocks(struct clk_mgr *clk_mgr) |
| 114 | { | 78 | { |
| 115 | memset(&(clk_mgr->clks), 0, sizeof(struct dc_clocks)); | 79 | memset(&(clk_mgr->clks), 0, sizeof(struct dc_clocks)); |
| @@ -126,10 +90,8 @@ static void dcn201_update_clocks(struct clk_mgr *clk_mgr_base, · 10 unmodified lines | |||
| 126 | struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base); | 90 | struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base); |
| 127 | struct dc_clocks *new_clocks = &context->bw_ctx.bw.dcn.clk; | 91 | struct dc_clocks *new_clocks = &context->bw_ctx.bw.dcn.clk; |
| 128 | struct dc *dc = clk_mgr_base->ctx->dc; | 92 | struct dc *dc = clk_mgr_base->ctx->dc; |
| 129 | int display_count; | ||
| 130 | bool update_dppclk = false; | 93 | bool update_dppclk = false; |
| 131 | bool update_dispclk = false; | 94 | bool update_dispclk = false; |
| 132 | bool enter_display_off = false; | ||
| 133 | bool dpp_clock_lowered = false; | 95 | bool dpp_clock_lowered = false; |
| 134 | bool force_reset = false; | 96 | bool force_reset = false; |
| 135 | bool p_state_change_support; | 97 | bool p_state_change_support; |
| @@ -145,10 +107,7 @@ static void dcn201_update_clocks(struct clk_mgr *clk_mgr_base, · 9 unmodified lines | |||
| 145 | dcn2_read_clocks_from_hw_dentist(clk_mgr_base); | 107 | dcn2_read_clocks_from_hw_dentist(clk_mgr_base); |
| 146 | } | 108 | } |
| 147 | 109 | ||
| 148 | display_count = clk_mgr_helper_get_active_display_cnt(dc, context); | 110 | clk_mgr_helper_get_active_display_cnt(dc, context); |
| 149 | |||
| 150 | if (display_count == 0) | ||
| 151 | enter_display_off = true; | ||
| 152 | 111 | ||
| 153 | if (should_set_clock(safe_to_lower, new_clocks->phyclk_khz, clk_mgr_base->clks.phyclk_khz)) | 112 | if (should_set_clock(safe_to_lower, new_clocks->phyclk_khz, clk_mgr_base->clks.phyclk_khz)) |
| 154 | clk_mgr_base->clks.phyclk_khz = new_clocks->phyclk_khz; | 113 | clk_mgr_base->clks.phyclk_khz = new_clocks->phyclk_khz; |
| @@ -56,9 +56,7 @@ · 55 unmodified lines | |||
| 56 | 56 | ||
| 57 | 57 | ||
| 58 | /* TODO: evaluate how to lower or disable all dcn clocks in screen off case */ | 58 | /* TODO: evaluate how to lower or disable all dcn clocks in screen off case */ |
| 59 | int rn_get_active_display_cnt_wa( | 59 | static int rn_get_active_display_cnt_wa(struct dc *dc, struct dc_state *context) |
| 60 | struct dc *dc, | ||
| 61 | struct dc_state *context) | ||
| 62 | { | 60 | { |
| 63 | int i, display_count; | 61 | int i, display_count; |
| 64 | bool tmds_present = false; | 62 | bool tmds_present = false; |
| @@ -77,7 +75,8 @@ int rn_get_active_display_cnt_wa( · 12 unmodified lines | |||
| 77 | const struct dc_link *link = dc->links[i]; | 75 | const struct dc_link *link = dc->links[i]; |
| 78 | 76 | ||
| 79 | /* abusing the fact that the dig and phy are coupled to see if the phy is enabled */ | 77 | /* abusing the fact that the dig and phy are coupled to see if the phy is enabled */ |
| 80 | if (link->link_enc->funcs->is_dig_enabled(link->link_enc)) | 78 | if (link->link_enc->funcs->is_dig_enabled && |
| 79 | link->link_enc->funcs->is_dig_enabled(link->link_enc)) | ||
| 81 | display_count++; | 80 | display_count++; |
| 82 | } | 81 | } |
| 83 | 82 | ||
| @@ -88,7 +87,7 @@ int rn_get_active_display_cnt_wa( · 4 unmodified lines | |||
| 88 | return display_count; | 87 | return display_count; |
| 89 | } | 88 | } |
| 90 | 89 | ||
| 91 | void rn_set_low_power_state(struct clk_mgr *clk_mgr_base) | 90 | static void rn_set_low_power_state(struct clk_mgr *clk_mgr_base) |
| 92 | { | 91 | { |
| 93 | struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base); | 92 | struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base); |
| 94 | 93 | ||
| @@ -122,7 +121,7 @@ static void rn_update_clocks_update_dpp_dto(struct clk_mgr_internal *clk_mgr, · 27 unmodified lines | |||
| 122 | } | 121 | } |
| 123 | 122 | ||
| 124 | 123 | ||
| 125 | void rn_update_clocks(struct clk_mgr *clk_mgr_base, | 124 | static void rn_update_clocks(struct clk_mgr *clk_mgr_base, |
| 126 | struct dc_state *context, | 125 | struct dc_state *context, |
| 127 | bool safe_to_lower) | 126 | bool safe_to_lower) |
| 128 | { | 127 | { |
| @@ -437,25 +436,14 @@ static void rn_dump_clk_registers(struct clk_state_registers_and_bypass *regs_an · 308 unmodified lines | |||
| 437 | } | 436 | } |
| 438 | } | 437 | } |
| 439 | 438 | ||
| 440 | /* This function produce translated logical clk state values*/ | 439 | static void rn_enable_pme_wa(struct clk_mgr *clk_mgr_base) |
| 441 | void rn_get_clk_states(struct clk_mgr *clk_mgr_base, struct clk_states *s) | ||
| 442 | { | ||
| 443 | struct clk_state_registers_and_bypass sb = { 0 }; | ||
| 444 | struct clk_log_info log_info = { 0 }; | ||
| 445 | |||
| 446 | rn_dump_clk_registers(&sb, clk_mgr_base, &log_info); | ||
| 447 | |||
| 448 | s->dprefclk_khz = sb.dprefclk * 1000; | ||
| 449 | } | ||
| 450 | |||
| 451 | void rn_enable_pme_wa(struct clk_mgr *clk_mgr_base) | ||
| 452 | { | 440 | { |
| 453 | struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base); | 441 | struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base); |
| 454 | 442 | ||
| 455 | rn_vbios_smu_enable_pme_wa(clk_mgr); | 443 | rn_vbios_smu_enable_pme_wa(clk_mgr); |
| 456 | } | 444 | } |
| 457 | 445 | ||
| 458 | void rn_init_clocks(struct clk_mgr *clk_mgr) | 446 | static void rn_init_clocks(struct clk_mgr *clk_mgr) |
| 459 | { | 447 | { |
| 460 | memset(&(clk_mgr->clks), 0, sizeof(struct dc_clocks)); | 448 | memset(&(clk_mgr->clks), 0, sizeof(struct dc_clocks)); |
| 461 | // Assumption is that boot state always supports pstate | 449 | // Assumption is that boot state always supports pstate |
| @@ -33,6 +33,8 @@ · 32 unmodified lines | |||
| 33 | #include "mp/mp_12_0_0_offset.h" | 33 | #include "mp/mp_12_0_0_offset.h" |
| 34 | #include "mp/mp_12_0_0_sh_mask.h" | 34 | #include "mp/mp_12_0_0_sh_mask.h" |
| 35 | 35 | ||
| 36 | #include "rn_clk_mgr_vbios_smu.h" | ||
| 37 | |||
| 36 | #define REG(reg_name) \ | 38 | #define REG(reg_name) \ |
| 37 | (MP0_BASE.instance[0].segment[mm ## reg_name ## _BASE_IDX] + mm ## reg_name) | 39 | (MP0_BASE.instance[0].segment[mm ## reg_name ## _BASE_IDX] + mm ## reg_name) |
| 38 | 40 | ||
| @@ -86,7 +88,9 @@ static uint32_t rn_smu_wait_for_response(struct clk_mgr_internal *clk_mgr, unsig · 47 unmodified lines | |||
| 86 | } | 88 | } |
| 87 | 89 | ||
| 88 | 90 | ||
| 89 | int rn_vbios_smu_send_msg_with_param(struct clk_mgr_internal *clk_mgr, unsigned int msg_id, unsigned int param) | 91 | static int rn_vbios_smu_send_msg_with_param(struct clk_mgr_internal *clk_mgr, |
| 92 | unsigned int msg_id, | ||
| 93 | unsigned int param) | ||
| 90 | { | 94 | { |
| 91 | uint32_t result; | 95 | uint32_t result; |
| 92 | 96 | ||
| @@ -252,6 +252,7 @@ static void dcn3_update_clocks(struct clk_mgr *clk_mgr_base, · 251 unmodified lines | |||
| 252 | bool update_dispclk = false; | 252 | bool update_dispclk = false; |
| 253 | bool enter_display_off = false; | 253 | bool enter_display_off = false; |
| 254 | bool dpp_clock_lowered = false; | 254 | bool dpp_clock_lowered = false; |
| 255 | bool update_pstate_unsupported_clk = false; | ||
| 255 | struct dmcu *dmcu = clk_mgr_base->ctx->dc->res_pool->dmcu; | 256 | struct dmcu *dmcu = clk_mgr_base->ctx->dc->res_pool->dmcu; |
| 256 | bool force_reset = false; | 257 | bool force_reset = false; |
| 257 | bool update_uclk = false; | 258 | bool update_uclk = false; |
| @@ -299,13 +300,28 @@ static void dcn3_update_clocks(struct clk_mgr *clk_mgr_base, · 41 unmodified lines | |||
| 299 | clk_mgr_base->clks.prev_p_state_change_support = clk_mgr_base->clks.p_state_change_support; | 300 | clk_mgr_base->clks.prev_p_state_change_support = clk_mgr_base->clks.p_state_change_support; |
| 300 | total_plane_count = clk_mgr_helper_get_active_plane_cnt(dc, context); | 301 | total_plane_count = clk_mgr_helper_get_active_plane_cnt(dc, context); |
| 301 | p_state_change_support = new_clocks->p_state_change_support || (total_plane_count == 0); | 302 | p_state_change_support = new_clocks->p_state_change_support || (total_plane_count == 0); |
| 302 | if (should_update_pstate_support(safe_to_lower, p_state_change_support, clk_mgr_base->clks.p_state_change_support)) { | 303 | |
| 304 | // invalidate the current P-State forced min in certain dc_mode_softmax situations | ||
| 305 | if (dc->clk_mgr->dc_mode_softmax_enabled && safe_to_lower && !p_state_change_support) { | ||
| 306 | if ((new_clocks->dramclk_khz <= dc->clk_mgr->bw_params->dc_mode_softmax_memclk * 1000) != | ||
| 307 | (clk_mgr_base->clks.dramclk_khz <= dc->clk_mgr->bw_params->dc_mode_softmax_memclk * 1000)) | ||
| 308 | update_pstate_unsupported_clk = true; | ||
| 309 | } | ||
| 310 | |||
| 311 | if (should_update_pstate_support(safe_to_lower, p_state_change_support, clk_mgr_base->clks.p_state_change_support) || | ||
| 312 | update_pstate_unsupported_clk) { | ||
| 303 | clk_mgr_base->clks.p_state_change_support = p_state_change_support; | 313 | clk_mgr_base->clks.p_state_change_support = p_state_change_support; |
| 304 | 314 | ||
| 305 | /* to disable P-State switching, set UCLK min = max */ | 315 | /* to disable P-State switching, set UCLK min = max */ |
| 306 | if (!clk_mgr_base->clks.p_state_change_support) | 316 | if (!clk_mgr_base->clks.p_state_change_support) { |
| 307 | dcn30_smu_set_hard_min_by_freq(clk_mgr, PPCLK_UCLK, | 317 | if (dc->clk_mgr->dc_mode_softmax_enabled && |
| 318 | new_clocks->dramclk_khz <= dc->clk_mgr->bw_params->dc_mode_softmax_memclk * 1000) | ||
| 319 | dcn30_smu_set_hard_min_by_freq(clk_mgr, PPCLK_UCLK, | ||
| 320 | dc->clk_mgr->bw_params->dc_mode_softmax_memclk); | ||
| 321 | else | ||
| 322 | dcn30_smu_set_hard_min_by_freq(clk_mgr, PPCLK_UCLK, | ||
| 308 | clk_mgr_base->bw_params->clk_table.entries[clk_mgr_base->bw_params->clk_table.num_entries - 1].memclk_mhz); | 323 | clk_mgr_base->bw_params->clk_table.entries[clk_mgr_base->bw_params->clk_table.num_entries - 1].memclk_mhz); |
| 324 | } | ||
| 309 | } | 325 | } |
| 310 | 326 | ||
| 311 | /* Always update saved value, even if new value not set due to P-State switching unsupported */ | 327 | /* Always update saved value, even if new value not set due to P-State switching unsupported */ |
| @@ -421,6 +437,24 @@ static void dcn3_set_hard_max_memclk(struct clk_mgr *clk_mgr_base) · 109 unmodified lines | |||
| 421 | clk_mgr_base->bw_params->clk_table.entries[clk_mgr_base->bw_params->clk_table.num_entries - 1].memclk_mhz); | 437 | clk_mgr_base->bw_params->clk_table.entries[clk_mgr_base->bw_params->clk_table.num_entries - 1].memclk_mhz); |
| 422 | } | 438 | } |
| 423 | 439 | ||
| 440 | static void dcn3_set_max_memclk(struct clk_mgr *clk_mgr_base, unsigned int memclk_mhz) | ||
| 441 | { | ||
| 442 | struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base); | ||
| 443 | |||
| 444 | if (!clk_mgr->smu_present) | ||
| 445 | return; | ||
| 446 | |||
| 447 | dcn30_smu_set_hard_max_by_freq(clk_mgr, PPCLK_UCLK, memclk_mhz); | ||
| 448 | } | ||
| 449 | static void dcn3_set_min_memclk(struct clk_mgr *clk_mgr_base, unsigned int memclk_mhz) | ||
| 450 | { | ||
| 451 | struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base); | ||
| 452 | |||
| 453 | if (!clk_mgr->smu_present) | ||
| 454 | return; | ||
| 455 | dcn30_smu_set_hard_min_by_freq(clk_mgr, PPCLK_UCLK, memclk_mhz); | ||
| 456 | } | ||
| 457 | |||
| 424 | /* Get current memclk states, update bounding box */ | 458 | /* Get current memclk states, update bounding box */ |
| 425 | static void dcn3_get_memclk_states_from_smu(struct clk_mgr *clk_mgr_base) | 459 | static void dcn3_get_memclk_states_from_smu(struct clk_mgr *clk_mgr_base) |
| 426 | { | 460 | { |
| @@ -436,6 +470,8 @@ static void dcn3_get_memclk_states_from_smu(struct clk_mgr *clk_mgr_base) · 9 unmodified lines | |||
| 436 | &num_levels); | 470 | &num_levels); |
| 437 | clk_mgr_base->bw_params->clk_table.num_entries = num_levels ? num_levels : 1; | 471 | clk_mgr_base->bw_params->clk_table.num_entries = num_levels ? num_levels : 1; |
| 438 | 472 | ||
| 473 | clk_mgr_base->bw_params->dc_mode_softmax_memclk = dcn30_smu_get_dc_mode_max_dpm_freq(clk_mgr, PPCLK_UCLK); | ||
| 474 | |||
| 439 | /* Refresh bounding box */ | 475 | /* Refresh bounding box */ |
| 440 | clk_mgr_base->ctx->dc->res_pool->funcs->update_bw_bounding_box( | 476 | clk_mgr_base->ctx->dc->res_pool->funcs->update_bw_bounding_box( |
| 441 | clk_mgr->base.ctx->dc, clk_mgr_base->bw_params); | 477 | clk_mgr->base.ctx->dc, clk_mgr_base->bw_params); |
| @@ -505,6 +541,8 @@ static struct clk_mgr_funcs dcn3_funcs = { · 63 unmodified lines | |||
| 505 | .notify_wm_ranges = dcn3_notify_wm_ranges, | 541 | .notify_wm_ranges = dcn3_notify_wm_ranges, |
| 506 | .set_hard_min_memclk = dcn3_set_hard_min_memclk, | 542 | .set_hard_min_memclk = dcn3_set_hard_min_memclk, |
| 507 | .set_hard_max_memclk = dcn3_set_hard_max_memclk, | 543 | .set_hard_max_memclk = dcn3_set_hard_max_memclk, |
| 544 | .set_max_memclk = dcn3_set_max_memclk, | ||
| 545 | .set_min_memclk = dcn3_set_min_memclk, | ||
| 508 | .get_memclk_states_from_smu = dcn3_get_memclk_states_from_smu, | 546 | .get_memclk_states_from_smu = dcn3_get_memclk_states_from_smu, |
| 509 | .are_clock_states_equal = dcn3_are_clock_states_equal, | 547 | .are_clock_states_equal = dcn3_are_clock_states_equal, |
| 510 | .enable_pme_wa = dcn3_enable_pme_wa, | 548 | .enable_pme_wa = dcn3_enable_pme_wa, |
| @@ -88,9 +88,9 @@ static uint32_t dcn301_smu_wait_for_response(struct clk_mgr_internal *clk_mgr, u · 87 unmodified lines | |||
| 88 | return res_val; | 88 | return res_val; |
| 89 | } | 89 | } |
| 90 | 90 | ||
| 91 | int dcn301_smu_send_msg_with_param( | 91 | static int dcn301_smu_send_msg_with_param(struct clk_mgr_internal *clk_mgr, |
| 92 | struct clk_mgr_internal *clk_mgr, | 92 | unsigned int msg_id, |
| 93 | unsigned int msg_id, unsigned int param) | 93 | unsigned int param) |
| 94 | { | 94 | { |
| 95 | uint32_t result; | 95 | uint32_t result; |
| 96 | 96 | ||
| @@ -89,9 +89,9 @@ static int vg_get_active_display_cnt_wa( · 88 unmodified lines | |||
| 89 | return display_count; | 89 | return display_count; |
| 90 | } | 90 | } |
| 91 | 91 | ||
| 92 | void vg_update_clocks(struct clk_mgr *clk_mgr_base, | 92 | static void vg_update_clocks(struct clk_mgr *clk_mgr_base, |
| 93 | struct dc_state *context, | 93 | struct dc_state *context, |
| 94 | bool safe_to_lower) | 94 | bool safe_to_lower) |
| 95 | { | 95 | { |
| 96 | struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base); | 96 | struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base); |
| 97 | struct dc_clocks *new_clocks = &context->bw_ctx.bw.dcn.clk; | 97 | struct dc_clocks *new_clocks = &context->bw_ctx.bw.dcn.clk; |
| @@ -367,18 +367,6 @@ static void vg_dump_clk_registers(struct clk_state_registers_and_bypass *regs_an · 269 unmodified lines | |||
| 367 | } | 367 | } |
| 368 | } | 368 | } |
| 369 | 369 | ||
| 370 | /* This function produce translated logical clk state values*/ | ||
| 371 | void vg_get_clk_states(struct clk_mgr *clk_mgr_base, struct clk_states *s) | ||
| 372 | { | ||
| 373 | |||
| 374 | struct clk_state_registers_and_bypass sb = { 0 }; | ||
| 375 | struct clk_log_info log_info = { 0 }; | ||
| 376 | |||
| 377 | vg_dump_clk_registers(&sb, clk_mgr_base, &log_info); | ||
| 378 | |||
| 379 | s->dprefclk_khz = sb.dprefclk * 1000; | ||
| 380 | } | ||
| 381 | |||
| 382 | static void vg_enable_pme_wa(struct clk_mgr *clk_mgr_base) | 370 | static void vg_enable_pme_wa(struct clk_mgr *clk_mgr_base) |
| 383 | { | 371 | { |
| 384 | struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base); | 372 | struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base); |
| @@ -386,7 +374,7 @@ static void vg_enable_pme_wa(struct clk_mgr *clk_mgr_base) · 1 unmodified line | |||
| 386 | dcn301_smu_enable_pme_wa(clk_mgr); | 374 | dcn301_smu_enable_pme_wa(clk_mgr); |
| 387 | } | 375 | } |
| 388 | 376 | ||
| 389 | void vg_init_clocks(struct clk_mgr *clk_mgr) | 377 | static void vg_init_clocks(struct clk_mgr *clk_mgr) |
| 390 | { | 378 | { |
| 391 | memset(&(clk_mgr->clks), 0, sizeof(struct dc_clocks)); | 379 | memset(&(clk_mgr->clks), 0, sizeof(struct dc_clocks)); |
| 392 | // Assumption is that boot state always supports pstate | 380 | // Assumption is that boot state always supports pstate |
| @@ -753,7 +741,7 @@ void vg_clk_mgr_construct( · 360 unmodified lines | |||
| 753 | sizeof(struct watermarks), | 741 | sizeof(struct watermarks), |
| 754 | &clk_mgr->smu_wm_set.mc_address.quad_part); | 742 | &clk_mgr->smu_wm_set.mc_address.quad_part); |
| 755 | 743 | ||
| 756 | if (clk_mgr->smu_wm_set.wm_set == 0) { | 744 | if (!clk_mgr->smu_wm_set.wm_set) { |
| 757 | clk_mgr->smu_wm_set.wm_set = &dummy_wms; | 745 | clk_mgr->smu_wm_set.wm_set = &dummy_wms; |
| 758 | clk_mgr->smu_wm_set.mc_address.quad_part = 0; | 746 | clk_mgr->smu_wm_set.mc_address.quad_part = 0; |
| 759 | } | 747 | } |
| @@ -66,7 +66,7 @@ · 65 unmodified lines | |||
| 66 | #define TO_CLK_MGR_DCN31(clk_mgr)\ | 66 | #define TO_CLK_MGR_DCN31(clk_mgr)\ |
| 67 | container_of(clk_mgr, struct clk_mgr_dcn31, base) | 67 | container_of(clk_mgr, struct clk_mgr_dcn31, base) |
| 68 | 68 | ||
| 69 | int dcn31_get_active_display_cnt_wa( | 69 | static int dcn31_get_active_display_cnt_wa( |
| 70 | struct dc *dc, | 70 | struct dc *dc, |
| 71 | struct dc_state *context) | 71 | struct dc_state *context) |
| 72 | { | 72 | { |
| @@ -118,7 +118,7 @@ static void dcn31_disable_otg_wa(struct clk_mgr *clk_mgr_base, bool disable) · 45 unmodified lines | |||
| 118 | } | 118 | } |
| 119 | } | 119 | } |
| 120 | 120 | ||
| 121 | static void dcn31_update_clocks(struct clk_mgr *clk_mgr_base, | 121 | void dcn31_update_clocks(struct clk_mgr *clk_mgr_base, |
| 122 | struct dc_state *context, | 122 | struct dc_state *context, |
| 123 | bool safe_to_lower) | 123 | bool safe_to_lower) |
| 124 | { | 124 | { |
| @@ -285,7 +285,7 @@ static void dcn31_enable_pme_wa(struct clk_mgr *clk_mgr_base) · 160 unmodified lines | |||
| 285 | dcn31_smu_enable_pme_wa(clk_mgr); | 285 | dcn31_smu_enable_pme_wa(clk_mgr); |
| 286 | } | 286 | } |
| 287 | 287 | ||
| 288 | static void dcn31_init_clocks(struct clk_mgr *clk_mgr) | 288 | void dcn31_init_clocks(struct clk_mgr *clk_mgr) |
| 289 | { | 289 | { |
| 290 | memset(&(clk_mgr->clks), 0, sizeof(struct dc_clocks)); | 290 | memset(&(clk_mgr->clks), 0, sizeof(struct dc_clocks)); |
| 291 | // Assumption is that boot state always supports pstate | 291 | // Assumption is that boot state always supports pstate |
| @@ -295,7 +295,7 @@ static void dcn31_init_clocks(struct clk_mgr *clk_mgr) · 3 unmodified lines | |||
| 295 | clk_mgr->clks.zstate_support = DCN_ZSTATE_SUPPORT_UNKNOWN; | 295 | clk_mgr->clks.zstate_support = DCN_ZSTATE_SUPPORT_UNKNOWN; |
| 296 | } | 296 | } |
| 297 | 297 | ||
| 298 | static bool dcn31_are_clock_states_equal(struct dc_clocks *a, | 298 | bool dcn31_are_clock_states_equal(struct dc_clocks *a, |
| 299 | struct dc_clocks *b) | 299 | struct dc_clocks *b) |
| 300 | { | 300 | { |
| 301 | if (a->dispclk_khz != b->dispclk_khz) | 301 | if (a->dispclk_khz != b->dispclk_khz) |
| @@ -541,10 +541,9 @@ static unsigned int find_clk_for_voltage( · 239 unmodified lines | |||
| 541 | return clock; | 541 | return clock; |
| 542 | } | 542 | } |
| 543 | 543 | ||
| 544 | void dcn31_clk_mgr_helper_populate_bw_params( | 544 | static void dcn31_clk_mgr_helper_populate_bw_params(struct clk_mgr_internal *clk_mgr, |
| 545 | struct clk_mgr_internal *clk_mgr, | 545 | struct integrated_info *bios_info, |
| 546 | struct integrated_info *bios_info, | 546 | const DpmClocks_t *clock_table) |
| 547 | const DpmClocks_t *clock_table) | ||
| 548 | { | 547 | { |
| 549 | int i, j; | 548 | int i, j; |
| 550 | struct clk_bw_params *bw_params = clk_mgr->base.bw_params; | 549 | struct clk_bw_params *bw_params = clk_mgr->base.bw_params; |
| @@ -39,6 +39,13 @@ struct clk_mgr_dcn31 { · 38 unmodified lines | |||
| 39 | struct dcn31_smu_watermark_set smu_wm_set; | 39 | struct dcn31_smu_watermark_set smu_wm_set; |
| 40 | }; | 40 | }; |
| 41 | 41 | ||
| 42 | bool dcn31_are_clock_states_equal(struct dc_clocks *a, | ||
| 43 | struct dc_clocks *b); | ||
| 44 | void dcn31_init_clocks(struct clk_mgr *clk_mgr); | ||
| 45 | void dcn31_update_clocks(struct clk_mgr *clk_mgr_base, | ||
| 46 | struct dc_state *context, | ||
| 47 | bool safe_to_lower); | ||
| 48 | |||
| 42 | void dcn31_clk_mgr_construct(struct dc_context *ctx, | 49 | void dcn31_clk_mgr_construct(struct dc_context *ctx, |
| 43 | struct clk_mgr_dcn31 *clk_mgr, | 50 | struct clk_mgr_dcn31 *clk_mgr, |
| 44 | struct pp_smu_funcs *pp_smu, | 51 | struct pp_smu_funcs *pp_smu, |
| @@ -95,9 +95,9 @@ static uint32_t dcn31_smu_wait_for_response(struct clk_mgr_internal *clk_mgr, un · 94 unmodified lines | |||
| 95 | return res_val; | 95 | return res_val; |
| 96 | } | 96 | } |
| 97 | 97 | ||
| 98 | int dcn31_smu_send_msg_with_param( | 98 | static int dcn31_smu_send_msg_with_param(struct clk_mgr_internal *clk_mgr, |
| 99 | struct clk_mgr_internal *clk_mgr, | 99 | unsigned int msg_id, |
| 100 | unsigned int msg_id, unsigned int param) | 100 | unsigned int param) |
| 101 | { | 101 | { |
| 102 | uint32_t result; | 102 | uint32_t result; |
| 103 | 103 | ||
| @@ -221,9 +221,9 @@ static bool create_links( · 220 unmodified lines | |||
| 221 | link = link_create(&link_init_params); | 221 | link = link_create(&link_init_params); |
| 222 | 222 | ||
| 223 | if (link) { | 223 | if (link) { |
| 224 | dc->links[dc->link_count] = link; | 224 | dc->links[dc->link_count] = link; |
| 225 | link->dc = dc; | 225 | link->dc = dc; |
| 226 | ++dc->link_count; | 226 | ++dc->link_count; |
| 227 | } | 227 | } |
| 228 | } | 228 | } |
| 229 | 229 | ||
| @@ -274,24 +274,6 @@ static bool create_links( · 44 unmodified lines | |||
| 274 | goto failed_alloc; | 274 | goto failed_alloc; |
| 275 | } | 275 | } |
| 276 | 276 | ||
| 277 | #if defined(CONFIG_DRM_AMD_DC_DCN) | ||
| 278 | if (IS_FPGA_MAXIMUS_DC(dc->ctx->dce_environment) && | ||
| 279 | dc->caps.dp_hpo && | ||
| 280 | link->dc->res_pool->res_cap->num_hpo_dp_link_encoder > 0) { | ||
| 281 | /* FPGA case - Allocate HPO DP link encoder */ | ||
| 282 | if (i < link->dc->res_pool->res_cap->num_hpo_dp_link_encoder) { | ||
| 283 | link->hpo_dp_link_enc = link->dc->res_pool->hpo_dp_link_enc[i]; | ||
| 284 | |||
| 285 | if (link->hpo_dp_link_enc == NULL) { | ||
| 286 | BREAK_TO_DEBUGGER(); | ||
| 287 | goto failed_alloc; | ||
| 288 | } | ||
| 289 | link->hpo_dp_link_enc->hpd_source = link->link_enc->hpd_source; | ||
| 290 | link->hpo_dp_link_enc->transmitter = link->link_enc->transmitter; | ||
| 291 | } | ||
| 292 | } | ||
| 293 | #endif | ||
| 294 | |||
| 295 | link->link_status.dpcd_caps = &link->dpcd_caps; | 277 | link->link_status.dpcd_caps = &link->dpcd_caps; |
| 296 | 278 | ||
| 297 | enc_init.ctx = dc->ctx; | 279 | enc_init.ctx = dc->ctx; |
| @@ -808,6 +790,10 @@ void dc_stream_set_static_screen_params(struct dc *dc, · 510 unmodified lines | |||
| 808 | 790 | ||
| 809 | static void dc_destruct(struct dc *dc) | 791 | static void dc_destruct(struct dc *dc) |
| 810 | { | 792 | { |
| 793 | // reset link encoder assignment table on destruct | ||
| 794 | if (dc->res_pool && dc->res_pool->funcs->link_encs_assign) | ||
| 795 | link_enc_cfg_init(dc, dc->current_state); | ||
| 796 | |||
| 811 | if (dc->current_state) { | 797 | if (dc->current_state) { |
| 812 | dc_release_state(dc->current_state); | 798 | dc_release_state(dc->current_state); |
| 813 | dc->current_state = NULL; | 799 | dc->current_state = NULL; |
| @@ -1016,8 +1002,6 @@ static bool dc_construct(struct dc *dc, · 202 unmodified lines | |||
| 1016 | goto fail; | 1002 | goto fail; |
| 1017 | } | 1003 | } |
| 1018 | 1004 | ||
| 1019 | dc_resource_state_construct(dc, dc->current_state); | ||
| 1020 | |||
| 1021 | if (!create_links(dc, init_params->num_virtual_links)) | 1005 | if (!create_links(dc, init_params->num_virtual_links)) |
| 1022 | goto fail; | 1006 | goto fail; |
| 1023 | 1007 | ||
| @@ -1027,8 +1011,7 @@ static bool dc_construct(struct dc *dc, · 3 unmodified lines | |||
| 1027 | if (!create_link_encoders(dc)) | 1011 | if (!create_link_encoders(dc)) |
| 1028 | goto fail; | 1012 | goto fail; |
| 1029 | 1013 | ||
| 1030 | /* Initialise DIG link encoder resource tracking variables. */ | 1014 | dc_resource_state_construct(dc, dc->current_state); |
| 1031 | link_enc_cfg_init(dc, dc->current_state); | ||
| 1032 | 1015 | ||
| 1033 | return true; | 1016 | return true; |
| 1034 | 1017 | ||
| @@ -1421,22 +1404,29 @@ static void program_timing_sync( · 386 unmodified lines | |||
| 1421 | status->timing_sync_info.master = false; | 1404 | status->timing_sync_info.master = false; |
| 1422 | 1405 | ||
| 1423 | } | 1406 | } |
| 1424 | /* remove any other unblanked pipes as they have already been synced */ | ||
| 1425 | for (j = j + 1; j < group_size; j++) { | ||
| 1426 | bool is_blanked; | ||
| 1427 | 1407 | ||
| 1428 | if (pipe_set[j]->stream_res.opp->funcs->dpg_is_blanked) | 1408 | /* remove any other pipes that are already been synced */ |
| 1429 | is_blanked = | 1409 | if (dc->config.use_pipe_ctx_sync_logic) { |
| 1430 | pipe_set[j]->stream_res.opp->funcs->dpg_is_blanked(pipe_set[j]->stream_res.opp); | 1410 | /* check pipe's syncd to decide which pipe to be removed */ |
| 1431 | else | 1411 | for (j = 1; j < group_size; j++) { |
| 1432 | is_blanked = | 1412 | if (pipe_set[j]->pipe_idx_syncd == pipe_set[0]->pipe_idx_syncd) { |
| 1433 | pipe_set[j]->stream_res.tg->funcs->is_blanked(pipe_set[j]->stream_res.tg); | 1413 | group_size--; |
| 1434 | if (!is_blanked) { | 1414 | pipe_set[j] = pipe_set[group_size]; |
| 1435 | group_size--; | 1415 | j--; |
| 1436 | pipe_set[j] = pipe_set[group_size]; | 1416 | } else |
| 1437 | j--; | 1417 | /* link slave pipe's syncd with master pipe */ |
| 1418 | pipe_set[j]->pipe_idx_syncd = pipe_set[0]->pipe_idx_syncd; | ||
| 1438 | } | 1419 | } |
| 1439 | } | 1420 | } else { |
| 1421 | /* remove any other pipes by checking valid plane */ | ||
| 1422 | for (j = j + 1; j < group_size; j++) { | ||
| 1423 | if (pipe_set[j]->plane_state) { | ||
| 1424 | group_size--; | ||
| 1425 | pipe_set[j] = pipe_set[group_size]; | ||
| 1426 | j--; | ||
| 1427 | } | ||
| 1428 | } | ||
| 1429 | } | ||
| 1440 | 1430 | ||
| 1441 | if (group_size > 1) { | 1431 | if (group_size > 1) { |
| 1442 | if (sync_type == TIMING_SYNCHRONIZABLE) { | 1432 | if (sync_type == TIMING_SYNCHRONIZABLE) { |
| @@ -1830,6 +1820,19 @@ bool dc_commit_state(struct dc *dc, struct dc_state *context) · 387 unmodified lines | |||
| 1830 | dc_stream_log(dc, stream); | 1820 | dc_stream_log(dc, stream); |
| 1831 | } | 1821 | } |
| 1832 | 1822 | ||
| 1823 | /* | ||
| 1824 | * Previous validation was perfomred with fast_validation = true and | ||
| 1825 | * the full DML state required for hardware programming was skipped. | ||
| 1826 | * | ||
| 1827 | * Re-validate here to calculate these parameters / watermarks. | ||
| 1828 | */ | ||
| 1829 | result = dc_validate_global_state(dc, context, false); | ||
| 1830 | if (result != DC_OK) { | ||
| 1831 | DC_LOG_ERROR("DC commit global validation failure: %s (%d)", | ||
| 1832 | dc_status_to_str(result), result); | ||
| 1833 | return result; | ||
| 1834 | } | ||
| 1835 | |||
| 1833 | result = dc_commit_state_no_check(dc, context); | 1836 | result = dc_commit_state_no_check(dc, context); |
| 1834 | 1837 | ||
| 1835 | return (result == DC_OK); | 1838 | return (result == DC_OK); |
| @@ -2870,7 +2873,8 @@ static void commit_planes_for_stream(struct dc *dc, · 1034 unmodified lines | |||
| 2870 | #endif | 2873 | #endif |
| 2871 | 2874 | ||
| 2872 | if ((update_type != UPDATE_TYPE_FAST) && stream->update_flags.bits.dsc_changed) | 2875 | if ((update_type != UPDATE_TYPE_FAST) && stream->update_flags.bits.dsc_changed) |
| 2873 | if (top_pipe_to_program->stream_res.tg->funcs->lock_doublebuffer_enable) { | 2876 | if (top_pipe_to_program && |
| 2877 | top_pipe_to_program->stream_res.tg->funcs->lock_doublebuffer_enable) { | ||
| 2874 | if (should_use_dmub_lock(stream->link)) { | 2878 | if (should_use_dmub_lock(stream->link)) { |
| 2875 | union dmub_hw_lock_flags hw_locks = { 0 }; | 2879 | union dmub_hw_lock_flags hw_locks = { 0 }; |
| 2876 | struct dmub_hw_lock_inst_flags inst_flags = { 0 }; | 2880 | struct dmub_hw_lock_inst_flags inst_flags = { 0 }; |
| @@ -2979,12 +2983,12 @@ static void commit_planes_for_stream(struct dc *dc, · 102 unmodified lines | |||
| 2979 | #ifdef CONFIG_DRM_AMD_DC_DCN | 2983 | #ifdef CONFIG_DRM_AMD_DC_DCN |
| 2980 | if (dc->debug.validate_dml_output) { | 2984 | if (dc->debug.validate_dml_output) { |
| 2981 | for (i = 0; i < dc->res_pool->pipe_count; i++) { | 2985 | for (i = 0; i < dc->res_pool->pipe_count; i++) { |
| 2982 | struct pipe_ctx cur_pipe = context->res_ctx.pipe_ctx[i]; | 2986 | struct pipe_ctx *cur_pipe = &context->res_ctx.pipe_ctx[i]; |
| 2983 | if (cur_pipe.stream == NULL) | 2987 | if (cur_pipe->stream == NULL) |
| 2984 | continue; | 2988 | continue; |
| 2985 | 2989 | ||
| 2986 | cur_pipe.plane_res.hubp->funcs->validate_dml_output( | 2990 | cur_pipe->plane_res.hubp->funcs->validate_dml_output( |
| 2987 | cur_pipe.plane_res.hubp, dc->ctx, | 2991 | cur_pipe->plane_res.hubp, dc->ctx, |
| 2988 | &context->res_ctx.pipe_ctx[i].rq_regs, | 2992 | &context->res_ctx.pipe_ctx[i].rq_regs, |
| 2989 | &context->res_ctx.pipe_ctx[i].dlg_regs, | 2993 | &context->res_ctx.pipe_ctx[i].dlg_regs, |
| 2990 | &context->res_ctx.pipe_ctx[i].ttu_regs); | 2994 | &context->res_ctx.pipe_ctx[i].ttu_regs); |
| @@ -3426,7 +3430,7 @@ struct dc_sink *dc_link_add_remote_sink( · 435 unmodified lines | |||
| 3426 | goto fail_add_sink; | 3430 | goto fail_add_sink; |
| 3427 | 3431 | ||
| 3428 | edid_status = dm_helpers_parse_edid_caps( | 3432 | edid_status = dm_helpers_parse_edid_caps( |
| 3429 | link->ctx, | 3433 | link, |
| 3430 | &dc_sink->dc_edid, | 3434 | &dc_sink->dc_edid, |
| 3431 | &dc_sink->edid_caps); | 3435 | &dc_sink->edid_caps); |
| 3432 | 3436 | ||
| @@ -3583,6 +3587,98 @@ void dc_lock_memory_clock_frequency(struct dc *dc) · 150 unmodified lines | |||
| 3583 | core_link_enable_stream(dc->current_state, &dc->current_state->res_ctx.pipe_ctx[i]); | 3587 | core_link_enable_stream(dc->current_state, &dc->current_state->res_ctx.pipe_ctx[i]); |
| 3584 | } | 3588 | } |
| 3585 | 3589 | ||
| 3590 | static void blank_and_force_memclk(struct dc *dc, bool apply, unsigned int memclk_mhz) | ||
| 3591 | { | ||
| 3592 | struct dc_state *context = dc->current_state; | ||
| 3593 | struct hubp *hubp; | ||
| 3594 | struct pipe_ctx *pipe; | ||
| 3595 | int i; | ||
| 3596 | |||
| 3597 | for (i = 0; i < dc->res_pool->pipe_count; i++) { | ||
| 3598 | pipe = &context->res_ctx.pipe_ctx[i]; | ||
| 3599 | |||
| 3600 | if (pipe->stream != NULL) { | ||
| 3601 | dc->hwss.disable_pixel_data(dc, pipe, true); | ||
| 3602 | |||
| 3603 | // wait for double buffer | ||
| 3604 | pipe->stream_res.tg->funcs->wait_for_state(pipe->stream_res.tg, CRTC_STATE_VACTIVE); | ||
| 3605 | pipe->stream_res.tg->funcs->wait_for_state(pipe->stream_res.tg, CRTC_STATE_VBLANK); | ||
| 3606 | pipe->stream_res.tg->funcs->wait_for_state(pipe->stream_res.tg, CRTC_STATE_VACTIVE); | ||
| 3607 | |||
| 3608 | hubp = pipe->plane_res.hubp; | ||
| 3609 | hubp->funcs->set_blank_regs(hubp, true); | ||
| 3610 | } | ||
| 3611 | } | ||
| 3612 | |||
| 3613 | dc->clk_mgr->funcs->set_max_memclk(dc->clk_mgr, memclk_mhz); | ||
| 3614 | dc->clk_mgr->funcs->set_min_memclk(dc->clk_mgr, memclk_mhz); | ||
| 3615 | |||
| 3616 | for (i = 0; i < dc->res_pool->pipe_count; i++) { | ||
| 3617 | pipe = &context->res_ctx.pipe_ctx[i]; | ||
| 3618 | |||
| 3619 | if (pipe->stream != NULL) { | ||
| 3620 | dc->hwss.disable_pixel_data(dc, pipe, false); | ||
| 3621 | |||
| 3622 | hubp = pipe->plane_res.hubp; | ||
| 3623 | hubp->funcs->set_blank_regs(hubp, false); | ||
| 3624 | } | ||
| 3625 | } | ||
| 3626 | } | ||
| 3627 | |||
| 3628 | |||
| 3629 | /** | ||
| 3630 | * dc_enable_dcmode_clk_limit() - lower clocks in dc (battery) mode | ||
| 3631 | * @dc: pointer to dc of the dm calling this | ||
| 3632 | * @enable: True = transition to DC mode, false = transition back to AC mode | ||
| 3633 | * | ||
| 3634 | * Some SoCs define additional clock limits when in DC mode, DM should | ||
| 3635 | * invoke this function when the platform undergoes a power source transition | ||
| 3636 | * so DC can apply/unapply the limit. This interface may be disruptive to | ||
| 3637 | * the onscreen content. | ||
| 3638 | * | ||
| 3639 | * Context: Triggered by OS through DM interface, or manually by escape calls. | ||
| 3640 | * Need to hold a dclock when doing so. | ||
| 3641 | * | ||
| 3642 | * Return: none (void function) | ||
| 3643 | * | ||
| 3644 | */ | ||
| 3645 | void dc_enable_dcmode_clk_limit(struct dc *dc, bool enable) | ||
| 3646 | { | ||
| 3647 | uint32_t hw_internal_rev = dc->ctx->asic_id.hw_internal_rev; | ||
| 3648 | unsigned int softMax, maxDPM, funcMin; | ||
| 3649 | bool p_state_change_support; | ||
| 3650 | |||
| 3651 | if (!ASICREV_IS_BEIGE_GOBY_P(hw_internal_rev)) | ||
| 3652 | return; | ||
| 3653 | |||
| 3654 | softMax = dc->clk_mgr->bw_params->dc_mode_softmax_memclk; | ||
| 3655 | maxDPM = dc->clk_mgr->bw_params->clk_table.entries[dc->clk_mgr->bw_params->clk_table.num_entries - 1].memclk_mhz; | ||
| 3656 | funcMin = (dc->clk_mgr->clks.dramclk_khz + 999) / 1000; | ||
| 3657 | p_state_change_support = dc->clk_mgr->clks.p_state_change_support; | ||
| 3658 | |||
| 3659 | if (enable && !dc->clk_mgr->dc_mode_softmax_enabled) { | ||
| 3660 | if (p_state_change_support) { | ||
| 3661 | if (funcMin <= softMax) | ||
| 3662 | dc->clk_mgr->funcs->set_max_memclk(dc->clk_mgr, softMax); | ||
| 3663 | // else: No-Op | ||
| 3664 | } else { | ||
| 3665 | if (funcMin <= softMax) | ||
| 3666 | blank_and_force_memclk(dc, true, softMax); | ||
| 3667 | // else: No-Op | ||
| 3668 | } | ||
| 3669 | } else if (!enable && dc->clk_mgr->dc_mode_softmax_enabled) { | ||
| 3670 | if (p_state_change_support) { | ||
| 3671 | if (funcMin <= softMax) | ||
| 3672 | dc->clk_mgr->funcs->set_max_memclk(dc->clk_mgr, maxDPM); | ||
| 3673 | // else: No-Op | ||
| 3674 | } else { | ||
| 3675 | if (funcMin <= softMax) | ||
| 3676 | blank_and_force_memclk(dc, true, maxDPM); | ||
| 3677 | // else: No-Op | ||
| 3678 | } | ||
| 3679 | } | ||
| 3680 | dc->clk_mgr->dc_mode_softmax_enabled = enable; | ||
| 3681 | } | ||
| 3586 | bool dc_is_plane_eligible_for_idle_optimizations(struct dc *dc, struct dc_plane_state *plane, | 3682 | bool dc_is_plane_eligible_for_idle_optimizations(struct dc *dc, struct dc_plane_state *plane, |
| 3587 | struct dc_cursor_attributes *cursor_attr) | 3683 | struct dc_cursor_attributes *cursor_attr) |
| 3588 | { | 3684 | { |
| @@ -422,6 +422,8 @@ char *dc_status_to_str(enum dc_status status) · 421 unmodified lines | |||
| 422 | return "The operation is not supported."; | 422 | return "The operation is not supported."; |
| 423 | case DC_UNSUPPORTED_VALUE: | 423 | case DC_UNSUPPORTED_VALUE: |
| 424 | return "The value specified is not supported."; | 424 | return "The value specified is not supported."; |
| 425 | case DC_NO_LINK_ENC_RESOURCE: | ||
| 426 | return "No link encoder resource"; | ||
| 425 | case DC_ERROR_UNEXPECTED: | 427 | case DC_ERROR_UNEXPECTED: |
| 426 | return "Unexpected error"; | 428 | return "Unexpected error"; |
| 427 | } | 429 | } |
| @@ -66,31 +66,6 @@ · 65 unmodified lines | |||
| 66 | /******************************************************************************* | 66 | /******************************************************************************* |
| 67 | * Private functions | 67 | * Private functions |
| 68 | ******************************************************************************/ | 68 | ******************************************************************************/ |
| 69 | #if defined(CONFIG_DRM_AMD_DC_DCN) | ||
| 70 | static bool add_dp_hpo_link_encoder_to_link(struct dc_link *link) | ||
| 71 | { | ||
| 72 | struct hpo_dp_link_encoder *enc = resource_get_unused_hpo_dp_link_encoder( | ||
| 73 | link->dc->res_pool); | ||
| 74 | |||
| 75 | if (!link->hpo_dp_link_enc && enc) { | ||
| 76 | link->hpo_dp_link_enc = enc; | ||
| 77 | link->hpo_dp_link_enc->transmitter = link->link_enc->transmitter; | ||
| 78 | link->hpo_dp_link_enc->hpd_source = link->link_enc->hpd_source; | ||
| 79 | } | ||
| 80 | |||
| 81 | return (link->hpo_dp_link_enc != NULL); | ||
| 82 | } | ||
| 83 | |||
| 84 | static void remove_dp_hpo_link_encoder_from_link(struct dc_link *link) | ||
| 85 | { | ||
| 86 | if (link->hpo_dp_link_enc) { | ||
| 87 | link->hpo_dp_link_enc->hpd_source = HPD_SOURCEID_UNKNOWN; | ||
| 88 | link->hpo_dp_link_enc->transmitter = TRANSMITTER_UNKNOWN; | ||
| 89 | link->hpo_dp_link_enc = NULL; | ||
| 90 | } | ||
| 91 | } | ||
| 92 | #endif | ||
| 93 | |||
| 94 | static void dc_link_destruct(struct dc_link *link) | 69 | static void dc_link_destruct(struct dc_link *link) |
| 95 | { | 70 | { |
| 96 | int i; | 71 | int i; |
| @@ -118,12 +93,6 @@ static void dc_link_destruct(struct dc_link *link) · 21 unmodified lines | |||
| 118 | link->link_enc->funcs->destroy(&link->link_enc); | 93 | link->link_enc->funcs->destroy(&link->link_enc); |
| 119 | } | 94 | } |
| 120 | 95 | ||
| 121 | #if defined(CONFIG_DRM_AMD_DC_DCN) | ||
| 122 | if (link->hpo_dp_link_enc) { | ||
| 123 | remove_dp_hpo_link_encoder_from_link(link); | ||
| 124 | } | ||
| 125 | #endif | ||
| 126 | |||
| 127 | if (link->local_sink) | 96 | if (link->local_sink) |
| 128 | dc_sink_release(link->local_sink); | 97 | dc_sink_release(link->local_sink); |
| 129 | 98 | ||
| @@ -270,10 +239,10 @@ bool dc_link_detect_sink(struct dc_link *link, enum dc_connection_type *type) · 140 unmodified lines | |||
| 270 | 239 | ||
| 271 | /* Link may not have physical HPD pin. */ | 240 | /* Link may not have physical HPD pin. */ |
| 272 | if (link->ep_type != DISPLAY_ENDPOINT_PHY) { | 241 | if (link->ep_type != DISPLAY_ENDPOINT_PHY) { |
| 273 | if (link->hpd_status) | 242 | if (link->is_hpd_pending || !link->hpd_status) |
| 274 | *type = dc_connection_single; | ||
| 275 | else | ||
| 276 | *type = dc_connection_none; | 243 | *type = dc_connection_none; |
| 244 | else | ||
| 245 | *type = dc_connection_single; | ||
| 277 | 246 | ||
| 278 | return true; | 247 | return true; |
| 279 | } | 248 | } |
| @@ -881,6 +850,7 @@ static bool dc_link_detect_helper(struct dc_link *link, · 601 unmodified lines | |||
| 881 | enum dc_connection_type pre_connection_type = dc_connection_none; | 850 | enum dc_connection_type pre_connection_type = dc_connection_none; |
| 882 | bool perform_dp_seamless_boot = false; | 851 | bool perform_dp_seamless_boot = false; |
| 883 | const uint32_t post_oui_delay = 30; // 30ms | 852 | const uint32_t post_oui_delay = 30; // 30ms |
| 853 | struct link_resource link_res = { 0 }; | ||
| 884 | 854 | ||
| 885 | DC_LOGGER_INIT(link->ctx->logger); | 855 | DC_LOGGER_INIT(link->ctx->logger); |
| 886 | 856 | ||
| @@ -975,7 +945,10 @@ static bool dc_link_detect_helper(struct dc_link *link, · 88 unmodified lines | |||
| 975 | 945 | ||
| 976 | #if defined(CONFIG_DRM_AMD_DC_DCN) | 946 | #if defined(CONFIG_DRM_AMD_DC_DCN) |
| 977 | if (dp_get_link_encoding_format(&link->reported_link_cap) == DP_128b_132b_ENCODING) | 947 | if (dp_get_link_encoding_format(&link->reported_link_cap) == DP_128b_132b_ENCODING) |
| 978 | add_dp_hpo_link_encoder_to_link(link); | 948 | link_res.hpo_dp_link_enc = resource_get_hpo_dp_link_enc_for_det_lt( |
| 949 | &link->dc->current_state->res_ctx, | ||
| 950 | link->dc->res_pool, | ||
| 951 | link); | ||
| 979 | #endif | 952 | #endif |
| 980 | 953 | ||
| 981 | if (link->type == dc_connection_mst_branch) { | 954 | if (link->type == dc_connection_mst_branch) { |
| @@ -986,7 +959,7 @@ static bool dc_link_detect_helper(struct dc_link *link, · 4 unmodified lines | |||
| 986 | * empty which leads to allocate_mst_payload() has "0" | 959 | * empty which leads to allocate_mst_payload() has "0" |
| 987 | * pbn_per_slot value leading to exception on dc_fixpt_div() | 960 | * pbn_per_slot value leading to exception on dc_fixpt_div() |
| 988 | */ | 961 | */ |
| 989 | dp_verify_mst_link_cap(link); | 962 | dp_verify_mst_link_cap(link, &link_res); |
| 990 | 963 | ||
| 991 | /* | 964 | /* |
| 992 | * This call will initiate MST topology discovery. Which | 965 | * This call will initiate MST topology discovery. Which |
| @@ -1150,6 +1123,7 @@ static bool dc_link_detect_helper(struct dc_link *link, · 157 unmodified lines | |||
| 1150 | // verify link cap for SST non-seamless boot | 1123 | // verify link cap for SST non-seamless boot |
| 1151 | if (!perform_dp_seamless_boot) | 1124 | if (!perform_dp_seamless_boot) |
| 1152 | dp_verify_link_cap_with_retries(link, | 1125 | dp_verify_link_cap_with_retries(link, |
| 1126 | &link_res, | ||
| 1153 | &link->reported_link_cap, | 1127 | &link->reported_link_cap, |
| 1154 | LINK_TRAINING_MAX_VERIFY_RETRY); | 1128 | LINK_TRAINING_MAX_VERIFY_RETRY); |
| 1155 | } else { | 1129 | } else { |
| @@ -1844,6 +1818,8 @@ static void enable_stream_features(struct pipe_ctx *pipe_ctx) · 688 unmodified lines | |||
| 1844 | union down_spread_ctrl old_downspread; | 1818 | union down_spread_ctrl old_downspread; |
| 1845 | union down_spread_ctrl new_downspread; | 1819 | union down_spread_ctrl new_downspread; |
| 1846 | 1820 | ||
| 1821 | memset(&old_downspread, 0, sizeof(old_downspread)); | ||
| 1822 | |||
| 1847 | core_link_read_dpcd(link, DP_DOWNSPREAD_CTRL, | 1823 | core_link_read_dpcd(link, DP_DOWNSPREAD_CTRL, |
| 1848 | &old_downspread.raw, sizeof(old_downspread)); | 1824 | &old_downspread.raw, sizeof(old_downspread)); |
| 1849 | 1825 | ||
| @@ -2015,6 +1991,57 @@ static enum dc_status enable_link_dp_mst( · 165 unmodified lines | |||
| 2015 | return enable_link_dp(state, pipe_ctx); | 1991 | return enable_link_dp(state, pipe_ctx); |
| 2016 | } | 1992 | } |
| 2017 | 1993 | ||
| 1994 | void dc_link_blank_all_dp_displays(struct dc *dc) | ||
| 1995 | { | ||
| 1996 | unsigned int i; | ||
| 1997 | uint8_t dpcd_power_state = '\0'; | ||
| 1998 | enum dc_status status = DC_ERROR_UNEXPECTED; | ||
| 1999 | |||
| 2000 | for (i = 0; i < dc->link_count; i++) { | ||
| 2001 | if ((dc->links[i]->connector_signal != SIGNAL_TYPE_DISPLAY_PORT) || | ||
| 2002 | (dc->links[i]->priv == NULL) || (dc->links[i]->local_sink == NULL)) | ||
| 2003 | continue; | ||
| 2004 | |||
| 2005 | /* DP 2.0 spec requires that we read LTTPR caps first */ | ||
| 2006 | dp_retrieve_lttpr_cap(dc->links[i]); | ||
| 2007 | /* if any of the displays are lit up turn them off */ | ||
| 2008 | status = core_link_read_dpcd(dc->links[i], DP_SET_POWER, | ||
| 2009 | &dpcd_power_state, sizeof(dpcd_power_state)); | ||
| 2010 | |||
| 2011 | if (status == DC_OK && dpcd_power_state == DP_POWER_STATE_D0) | ||
| 2012 | dc_link_blank_dp_stream(dc->links[i], true); | ||
| 2013 | } | ||
| 2014 | |||
| 2015 | } | ||
| 2016 | |||
| 2017 | void dc_link_blank_dp_stream(struct dc_link *link, bool hw_init) | ||
| 2018 | { | ||
| 2019 | unsigned int j; | ||
| 2020 | struct dc *dc = link->ctx->dc; | ||
| 2021 | enum signal_type signal = link->connector_signal; | ||
| 2022 | |||
| 2023 | if ((signal == SIGNAL_TYPE_EDP) || | ||
| 2024 | (signal == SIGNAL_TYPE_DISPLAY_PORT)) { | ||
| 2025 | if (link->ep_type == DISPLAY_ENDPOINT_PHY && | ||
| 2026 | link->link_enc->funcs->get_dig_frontend && | ||
| 2027 | link->link_enc->funcs->is_dig_enabled(link->link_enc)) { | ||
| 2028 | unsigned int fe = link->link_enc->funcs->get_dig_frontend(link->link_enc); | ||
| 2029 | |||
| 2030 | if (fe != ENGINE_ID_UNKNOWN) | ||
| 2031 | for (j = 0; j < dc->res_pool->stream_enc_count; j++) { | ||
| 2032 | if (fe == dc->res_pool->stream_enc[j]->id) { | ||
| 2033 | dc->res_pool->stream_enc[j]->funcs->dp_blank(link, | ||
| 2034 | dc->res_pool->stream_enc[j]); | ||
| 2035 | break; | ||
| 2036 | } | ||
| 2037 | } | ||
| 2038 | } | ||
| 2039 | |||
| 2040 | if ((!link->wa_flags.dp_keep_receiver_powered) || hw_init) | ||
| 2041 | dp_receiver_power_ctrl(link, false); | ||
| 2042 | } | ||
| 2043 | } | ||
| 2044 | |||
| 2018 | static bool get_ext_hdmi_settings(struct pipe_ctx *pipe_ctx, | 2045 | static bool get_ext_hdmi_settings(struct pipe_ctx *pipe_ctx, |
| 2019 | enum engine_id eng_id, | 2046 | enum engine_id eng_id, |
| 2020 | struct ext_hdmi_settings *settings) | 2047 | struct ext_hdmi_settings *settings) |
| @@ -2452,7 +2479,8 @@ static void write_i2c_redriver_setting( · 431 unmodified lines | |||
| 2452 | DC_LOG_DEBUG("Set redriver failed"); | 2479 | DC_LOG_DEBUG("Set redriver failed"); |
| 2453 | } | 2480 | } |
| 2454 | 2481 | ||
| 2455 | static void disable_link(struct dc_link *link, enum signal_type signal) | 2482 | static void disable_link(struct dc_link *link, const struct link_resource *link_res, |
| 2483 | enum signal_type signal) | ||
| 2456 | { | 2484 | { |
| 2457 | /* | 2485 | /* |
| 2458 | * TODO: implement call for dp_set_hw_test_pattern | 2486 | * TODO: implement call for dp_set_hw_test_pattern |
| @@ -2471,20 +2499,20 @@ static void disable_link(struct dc_link *link, enum signal_type signal) · 12 unmodified lines | |||
| 2471 | struct dc_link_settings link_settings = link->cur_link_settings; | 2499 | struct dc_link_settings link_settings = link->cur_link_settings; |
| 2472 | #endif | 2500 | #endif |
| 2473 | if (dc_is_dp_sst_signal(signal)) | 2501 | if (dc_is_dp_sst_signal(signal)) |
| 2474 | dp_disable_link_phy(link, signal); | 2502 | dp_disable_link_phy(link, link_res, signal); |
| 2475 | else | 2503 | else |
| 2476 | dp_disable_link_phy_mst(link, signal); | 2504 | dp_disable_link_phy_mst(link, link_res, signal); |
| 2477 | 2505 | ||
| 2478 | if (dc_is_dp_sst_signal(signal) || | 2506 | if (dc_is_dp_sst_signal(signal) || |
| 2479 | link->mst_stream_alloc_table.stream_count == 0) { | 2507 | link->mst_stream_alloc_table.stream_count == 0) { |
| 2480 | #if defined(CONFIG_DRM_AMD_DC_DCN) | 2508 | #if defined(CONFIG_DRM_AMD_DC_DCN) |
| 2481 | if (dp_get_link_encoding_format(&link_settings) == DP_8b_10b_ENCODING) { | 2509 | if (dp_get_link_encoding_format(&link_settings) == DP_8b_10b_ENCODING) { |
| 2482 | dp_set_fec_enable(link, false); | 2510 | dp_set_fec_enable(link, false); |
| 2483 | dp_set_fec_ready(link, false); | 2511 | dp_set_fec_ready(link, link_res, false); |
| 2484 | } | 2512 | } |
| 2485 | #else | 2513 | #else |
| 2486 | dp_set_fec_enable(link, false); | 2514 | dp_set_fec_enable(link, false); |
| 2487 | dp_set_fec_ready(link, false); | 2515 | dp_set_fec_ready(link, link_res, false); |
| 2488 | #endif | 2516 | #endif |
| 2489 | } | 2517 | } |
| 2490 | } else { | 2518 | } else { |
| @@ -2595,7 +2623,7 @@ static enum dc_status enable_link( · 104 unmodified lines | |||
| 2595 | * new link settings. | 2623 | * new link settings. |
| 2596 | */ | 2624 | */ |
| 2597 | if (link->link_status.link_active) { | 2625 | if (link->link_status.link_active) { |
| 2598 | disable_link(link, pipe_ctx->stream->signal); | 2626 | disable_link(link, &pipe_ctx->link_res, pipe_ctx->stream->signal); |
| 2599 | } | 2627 | } |
| 2600 | 2628 | ||
| 2601 | switch (pipe_ctx->stream->signal) { | 2629 | switch (pipe_ctx->stream->signal) { |
| @@ -2715,8 +2743,23 @@ static bool dp_active_dongle_validate_timing( · 113 unmodified lines | |||
| 2715 | return false; | 2743 | return false; |
| 2716 | } | 2744 | } |
| 2717 | 2745 | ||
| 2746 | #if defined(CONFIG_DRM_AMD_DC_DCN) | ||
| 2747 | if (dongle_caps->dp_hdmi_frl_max_link_bw_in_kbps > 0) { // DP to HDMI FRL converter | ||
| 2748 | struct dc_crtc_timing outputTiming = *timing; | ||
| 2749 | |||
| 2750 | if (timing->flags.DSC && !timing->dsc_cfg.is_frl) | ||
| 2751 | /* DP input has DSC, HDMI FRL output doesn't have DSC, remove DSC from output timing */ | ||
| 2752 | outputTiming.flags.DSC = 0; | ||
| 2753 | if (dc_bandwidth_in_kbps_from_timing(&outputTiming) > dongle_caps->dp_hdmi_frl_max_link_bw_in_kbps) | ||
| 2754 | return false; | ||
| 2755 | } else { // DP to HDMI TMDS converter | ||
| 2756 | if (get_timing_pixel_clock_100hz(timing) > (dongle_caps->dp_hdmi_max_pixel_clk_in_khz * 10)) | ||
| 2757 | return false; | ||
| 2758 | } | ||
| 2759 | #else | ||
| 2718 | if (get_timing_pixel_clock_100hz(timing) > (dongle_caps->dp_hdmi_max_pixel_clk_in_khz * 10)) | 2760 | if (get_timing_pixel_clock_100hz(timing) > (dongle_caps->dp_hdmi_max_pixel_clk_in_khz * 10)) |
| 2719 | return false; | 2761 | return false; |
| 2762 | #endif | ||
| 2720 | 2763 | ||
| 2721 | #if defined(CONFIG_DRM_AMD_DC_DCN) | 2764 | #if defined(CONFIG_DRM_AMD_DC_DCN) |
| 2722 | } | 2765 | } |
| @@ -2962,7 +3005,7 @@ bool dc_link_set_psr_allow_active(struct dc_link *link, const bool *allow_active · 239 unmodified lines | |||
| 2962 | link->psr_settings.psr_power_opt = *power_opts; | 3005 | link->psr_settings.psr_power_opt = *power_opts; |
| 2963 | 3006 | ||
| 2964 | if (psr != NULL && link->psr_settings.psr_feature_enabled && psr->funcs->psr_set_power_opt) | 3007 | if (psr != NULL && link->psr_settings.psr_feature_enabled && psr->funcs->psr_set_power_opt) |
| 2965 | psr->funcs->psr_set_power_opt(psr, link->psr_settings.psr_power_opt); | 3008 | psr->funcs->psr_set_power_opt(psr, link->psr_settings.psr_power_opt, panel_inst); |
| 2966 | } | 3009 | } |
| 2967 | 3010 | ||
| 2968 | /* Enable or Disable PSR */ | 3011 | /* Enable or Disable PSR */ |
| @@ -3350,11 +3393,12 @@ static void dc_log_vcp_x_y(const struct dc_link *link, struct fixed31_32 avg_tim · 381 unmodified lines | |||
| 3350 | /* | 3393 | /* |
| 3351 | * Payload allocation/deallocation for SST introduced in DP2.0 | 3394 | * Payload allocation/deallocation for SST introduced in DP2.0 |
| 3352 | */ | 3395 | */ |
| 3353 | enum dc_status dc_link_update_sst_payload(struct pipe_ctx *pipe_ctx, bool allocate) | 3396 | static enum dc_status dc_link_update_sst_payload(struct pipe_ctx *pipe_ctx, |
| 3397 | bool allocate) | ||
| 3354 | { | 3398 | { |
| 3355 | struct dc_stream_state *stream = pipe_ctx->stream; | 3399 | struct dc_stream_state *stream = pipe_ctx->stream; |
| 3356 | struct dc_link *link = stream->link; | 3400 | struct dc_link *link = stream->link; |
| 3357 | struct hpo_dp_link_encoder *hpo_dp_link_encoder = link->hpo_dp_link_enc; | 3401 | struct hpo_dp_link_encoder *hpo_dp_link_encoder = pipe_ctx->link_res.hpo_dp_link_enc; |
| 3358 | struct hpo_dp_stream_encoder *hpo_dp_stream_encoder = pipe_ctx->stream_res.hpo_dp_stream_enc; | 3402 | struct hpo_dp_stream_encoder *hpo_dp_stream_encoder = pipe_ctx->stream_res.hpo_dp_stream_enc; |
| 3359 | struct link_mst_stream_allocation_table proposed_table = {0}; | 3403 | struct link_mst_stream_allocation_table proposed_table = {0}; |
| 3360 | struct fixed31_32 avg_time_slots_per_mtp; | 3404 | struct fixed31_32 avg_time_slots_per_mtp; |
| @@ -3436,7 +3480,7 @@ enum dc_status dc_link_allocate_mst_payload(struct pipe_ctx *pipe_ctx) · 75 unmodified lines | |||
| 3436 | struct link_encoder *link_encoder = NULL; | 3480 | struct link_encoder *link_encoder = NULL; |
| 3437 | struct stream_encoder *stream_encoder = pipe_ctx->stream_res.stream_enc; | 3481 | struct stream_encoder *stream_encoder = pipe_ctx->stream_res.stream_enc; |
| 3438 | #if defined(CONFIG_DRM_AMD_DC_DCN) | 3482 | #if defined(CONFIG_DRM_AMD_DC_DCN) |
| 3439 | struct hpo_dp_link_encoder *hpo_dp_link_encoder = link->hpo_dp_link_enc; | 3483 | struct hpo_dp_link_encoder *hpo_dp_link_encoder = pipe_ctx->link_res.hpo_dp_link_enc; |
| 3440 | struct hpo_dp_stream_encoder *hpo_dp_stream_encoder = pipe_ctx->stream_res.hpo_dp_stream_enc; | 3484 | struct hpo_dp_stream_encoder *hpo_dp_stream_encoder = pipe_ctx->stream_res.hpo_dp_stream_enc; |
| 3441 | #endif | 3485 | #endif |
| 3442 | struct dp_mst_stream_allocation_table proposed_table = {0}; | 3486 | struct dp_mst_stream_allocation_table proposed_table = {0}; |
| @@ -3766,7 +3810,7 @@ static enum dc_status deallocate_mst_payload(struct pipe_ctx *pipe_ctx) · 323 unmodified lines | |||
| 3766 | struct link_encoder *link_encoder = NULL; | 3810 | struct link_encoder *link_encoder = NULL; |
| 3767 | struct stream_encoder *stream_encoder = pipe_ctx->stream_res.stream_enc; | 3811 | struct stream_encoder *stream_encoder = pipe_ctx->stream_res.stream_enc; |
| 3768 | #if defined(CONFIG_DRM_AMD_DC_DCN) | 3812 | #if defined(CONFIG_DRM_AMD_DC_DCN) |
| 3769 | struct hpo_dp_link_encoder *hpo_dp_link_encoder = link->hpo_dp_link_enc; | 3813 | struct hpo_dp_link_encoder *hpo_dp_link_encoder = pipe_ctx->link_res.hpo_dp_link_enc; |
| 3770 | struct hpo_dp_stream_encoder *hpo_dp_stream_encoder = pipe_ctx->stream_res.hpo_dp_stream_enc; | 3814 | struct hpo_dp_stream_encoder *hpo_dp_stream_encoder = pipe_ctx->stream_res.hpo_dp_stream_enc; |
| 3771 | #endif | 3815 | #endif |
| 3772 | struct dp_mst_stream_allocation_table proposed_table = {0}; | 3816 | struct dp_mst_stream_allocation_table proposed_table = {0}; |
| @@ -3929,9 +3973,6 @@ static void update_psp_stream_config(struct pipe_ctx *pipe_ctx, bool dpms_off) · 156 unmodified lines | |||
| 3929 | struct cp_psp *cp_psp = &pipe_ctx->stream->ctx->cp_psp; | 3973 | struct cp_psp *cp_psp = &pipe_ctx->stream->ctx->cp_psp; |
| 3930 | #if defined(CONFIG_DRM_AMD_DC_DCN) | 3974 | #if defined(CONFIG_DRM_AMD_DC_DCN) |
| 3931 | struct link_encoder *link_enc = NULL; | 3975 | struct link_encoder *link_enc = NULL; |
| 3932 | struct dc_state *state = pipe_ctx->stream->ctx->dc->current_state; | ||
| 3933 | struct link_enc_assignment link_enc_assign; | ||
| 3934 | int i; | ||
| 3935 | #endif | 3976 | #endif |
| 3936 | 3977 | ||
| 3937 | if (cp_psp && cp_psp->funcs.update_stream_config) { | 3978 | if (cp_psp && cp_psp->funcs.update_stream_config) { |
| @@ -3956,18 +3997,15 @@ static void update_psp_stream_config(struct pipe_ctx *pipe_ctx, bool dpms_off) · 18 unmodified lines | |||
| 3956 | pipe_ctx->stream->ctx->dc, | 3997 | pipe_ctx->stream->ctx->dc, |
| 3957 | pipe_ctx->stream); | 3998 | pipe_ctx->stream); |
| 3958 | } | 3999 | } |
| 4000 | ASSERT(link_enc); | ||
| 4001 | |||
| 3959 | // Initialize PHY ID with ABCDE - 01234 mapping except when it is B0 | 4002 | // Initialize PHY ID with ABCDE - 01234 mapping except when it is B0 |
| 3960 | config.phy_idx = link_enc->transmitter - TRANSMITTER_UNIPHY_A; | 4003 | config.phy_idx = link_enc->transmitter - TRANSMITTER_UNIPHY_A; |
| 3961 | 4004 | ||
| 3962 | //look up the link_enc_assignment for the current pipe_ctx | ||
| 3963 | for (i = 0; i < state->stream_count; i++) { | ||
| 3964 | if (pipe_ctx->stream == state->streams[i]) { | ||
| 3965 | link_enc_assign = state->res_ctx.link_enc_cfg_ctx.link_enc_assignments[i]; | ||
| 3966 | } | ||
| 3967 | } | ||
| 3968 | // Add flag to guard new A0 DIG mapping | 4005 | // Add flag to guard new A0 DIG mapping |
| 3969 | if (pipe_ctx->stream->ctx->dc->enable_c20_dtm_b0 == true) { | 4006 | if (pipe_ctx->stream->ctx->dc->enable_c20_dtm_b0 == true && |
| 3970 | config.dig_be = link_enc_assign.eng_id; | 4007 | pipe_ctx->stream->link->dc->ctx->dce_version == DCN_VERSION_3_1) { |
| 4008 | config.dig_be = link_enc->preferred_engine; | ||
| 3971 | config.dio_output_type = pipe_ctx->stream->link->ep_type; | 4009 | config.dio_output_type = pipe_ctx->stream->link->ep_type; |
| 3972 | config.dio_output_idx = link_enc->transmitter - TRANSMITTER_UNIPHY_A; | 4010 | config.dio_output_idx = link_enc->transmitter - TRANSMITTER_UNIPHY_A; |
| 3973 | } else { | 4011 | } else { |
| @@ -3979,10 +4017,8 @@ static void update_psp_stream_config(struct pipe_ctx *pipe_ctx, bool dpms_off) · 5 unmodified lines | |||
| 3979 | if (pipe_ctx->stream->ctx->dc->enable_c20_dtm_b0 == true && | 4017 | if (pipe_ctx->stream->ctx->dc->enable_c20_dtm_b0 == true && |
| 3980 | link_enc->ctx->asic_id.hw_internal_rev == YELLOW_CARP_B0) { | 4018 | link_enc->ctx->asic_id.hw_internal_rev == YELLOW_CARP_B0) { |
| 3981 | if (pipe_ctx->stream->link->ep_type == DISPLAY_ENDPOINT_USB4_DPIA) { | 4019 | if (pipe_ctx->stream->link->ep_type == DISPLAY_ENDPOINT_USB4_DPIA) { |
| 3982 | link_enc = link_enc_assign.stream->link_enc; | ||
| 3983 | |||
| 3984 | // enum ID 1-4 maps to DPIA PHY ID 0-3 | 4020 | // enum ID 1-4 maps to DPIA PHY ID 0-3 |
| 3985 | config.phy_idx = link_enc_assign.ep_id.link_id.enum_id - ENUM_ID_1; | 4021 | config.phy_idx = pipe_ctx->stream->link->link_id.enum_id - ENUM_ID_1; |
| 3986 | } else { // for non DPIA mode over B0, ABCDE maps to 01564 | 4022 | } else { // for non DPIA mode over B0, ABCDE maps to 01564 |
| 3987 | 4023 | ||
| 3988 | switch (link_enc->transmitter) { | 4024 | switch (link_enc->transmitter) { |
| @@ -4019,7 +4055,8 @@ static void update_psp_stream_config(struct pipe_ctx *pipe_ctx, bool dpms_off) · 30 unmodified lines | |||
| 4019 | config.link_enc_idx = link_enc->transmitter - TRANSMITTER_UNIPHY_A; | 4055 | config.link_enc_idx = link_enc->transmitter - TRANSMITTER_UNIPHY_A; |
| 4020 | if (is_dp_128b_132b_signal(pipe_ctx)) { | 4056 | if (is_dp_128b_132b_signal(pipe_ctx)) { |
| 4021 | config.stream_enc_idx = pipe_ctx->stream_res.hpo_dp_stream_enc->id - ENGINE_ID_HPO_DP_0; | 4057 | config.stream_enc_idx = pipe_ctx->stream_res.hpo_dp_stream_enc->id - ENGINE_ID_HPO_DP_0; |
| 4022 | config.link_enc_idx = pipe_ctx->stream->link->hpo_dp_link_enc->inst; | 4058 | |
| 4059 | config.link_enc_idx = pipe_ctx->link_res.hpo_dp_link_enc->inst; | ||
| 4023 | config.dp2_enabled = 1; | 4060 | config.dp2_enabled = 1; |
| 4024 | } | 4061 | } |
| 4025 | #endif | 4062 | #endif |
| @@ -4050,7 +4087,7 @@ static void fpga_dp_hpo_enable_link_and_stream(struct dc_state *state, struct pi · 24 unmodified lines | |||
| 4050 | stream->link->cur_link_settings = link_settings; | 4087 | stream->link->cur_link_settings = link_settings; |
| 4051 | 4088 | ||
| 4052 | /* Enable clock, Configure lane count, and Enable Link Encoder*/ | 4089 | /* Enable clock, Configure lane count, and Enable Link Encoder*/ |
| 4053 | enable_dp_hpo_output(stream->link, &stream->link->cur_link_settings); | 4090 | enable_dp_hpo_output(stream->link, &pipe_ctx->link_res, &stream->link->cur_link_settings); |
| 4054 | 4091 | ||
| 4055 | #ifdef DIAGS_BUILD | 4092 | #ifdef DIAGS_BUILD |
| 4056 | /* Workaround for FPGA HPO capture DP link data: | 4093 | /* Workaround for FPGA HPO capture DP link data: |
| @@ -4100,12 +4137,12 @@ static void fpga_dp_hpo_enable_link_and_stream(struct dc_state *state, struct pi · 43 unmodified lines | |||
| 4100 | proposed_table.stream_allocations[0].hpo_dp_stream_enc = pipe_ctx->stream_res.hpo_dp_stream_enc; | 4137 | proposed_table.stream_allocations[0].hpo_dp_stream_enc = pipe_ctx->stream_res.hpo_dp_stream_enc; |
| 4101 | } | 4138 | } |
| 4102 | 4139 | ||
| 4103 | stream->link->hpo_dp_link_enc->funcs->update_stream_allocation_table( | 4140 | pipe_ctx->link_res.hpo_dp_link_enc->funcs->update_stream_allocation_table( |
| 4104 | stream->link->hpo_dp_link_enc, | 4141 | pipe_ctx->link_res.hpo_dp_link_enc, |
| 4105 | &proposed_table); | 4142 | &proposed_table); |
| 4106 | 4143 | ||
| 4107 | stream->link->hpo_dp_link_enc->funcs->set_throttled_vcp_size( | 4144 | pipe_ctx->link_res.hpo_dp_link_enc->funcs->set_throttled_vcp_size( |
| 4108 | stream->link->hpo_dp_link_enc, | 4145 | pipe_ctx->link_res.hpo_dp_link_enc, |
| 4109 | pipe_ctx->stream_res.hpo_dp_stream_enc->inst, | 4146 | pipe_ctx->stream_res.hpo_dp_stream_enc->inst, |
| 4110 | avg_time_slots_per_mtp); | 4147 | avg_time_slots_per_mtp); |
| 4111 | 4148 | ||
| @@ -4255,7 +4292,8 @@ void core_link_enable_stream( · 143 unmodified lines | |||
| 4255 | /* eDP lit up by bios already, no need to enable again. */ | 4292 | /* eDP lit up by bios already, no need to enable again. */ |
| 4256 | if (pipe_ctx->stream->signal == SIGNAL_TYPE_EDP && | 4293 | if (pipe_ctx->stream->signal == SIGNAL_TYPE_EDP && |
| 4257 | apply_edp_fast_boot_optimization && | 4294 | apply_edp_fast_boot_optimization && |
| 4258 | !pipe_ctx->stream->timing.flags.DSC) { | 4295 | !pipe_ctx->stream->timing.flags.DSC && |
| 4296 | !pipe_ctx->next_odm_pipe) { | ||
| 4259 | pipe_ctx->stream->dpms_off = false; | 4297 | pipe_ctx->stream->dpms_off = false; |
| 4260 | #if defined(CONFIG_DRM_AMD_DC_HDCP) | 4298 | #if defined(CONFIG_DRM_AMD_DC_HDCP) |
| 4261 | update_psp_stream_config(pipe_ctx, false); | 4299 | update_psp_stream_config(pipe_ctx, false); |
| @@ -4293,7 +4331,8 @@ void core_link_enable_stream( · 31 unmodified lines | |||
| 4293 | if (status != DC_FAIL_DP_LINK_TRAINING || | 4331 | if (status != DC_FAIL_DP_LINK_TRAINING || |
| 4294 | pipe_ctx->stream->signal == SIGNAL_TYPE_DISPLAY_PORT_MST) { | 4332 | pipe_ctx->stream->signal == SIGNAL_TYPE_DISPLAY_PORT_MST) { |
| 4295 | if (false == stream->link->link_status.link_active) | 4333 | if (false == stream->link->link_status.link_active) |
| 4296 | disable_link(stream->link, pipe_ctx->stream->signal); | 4334 | disable_link(stream->link, &pipe_ctx->link_res, |
| 4335 | pipe_ctx->stream->signal); | ||
| 4297 | BREAK_TO_DEBUGGER(); | 4336 | BREAK_TO_DEBUGGER(); |
| 4298 | return; | 4337 | return; |
| 4299 | } | 4338 | } |
| @@ -4442,14 +4481,14 @@ void core_link_disable_stream(struct pipe_ctx *pipe_ctx) · 142 unmodified lines | |||
| 4442 | * state machine. | 4481 | * state machine. |
| 4443 | * In DP2 or MST mode, our encoder will stay video active | 4482 | * In DP2 or MST mode, our encoder will stay video active |
| 4444 | */ | 4483 | */ |
| 4445 | disable_link(pipe_ctx->stream->link, pipe_ctx->stream->signal); | 4484 | disable_link(pipe_ctx->stream->link, &pipe_ctx->link_res, pipe_ctx->stream->signal); |
| 4446 | dc->hwss.disable_stream(pipe_ctx); | 4485 | dc->hwss.disable_stream(pipe_ctx); |
| 4447 | } else { | 4486 | } else { |
| 4448 | dc->hwss.disable_stream(pipe_ctx); | 4487 | dc->hwss.disable_stream(pipe_ctx); |
| 4449 | disable_link(pipe_ctx->stream->link, pipe_ctx->stream->signal); | 4488 | disable_link(pipe_ctx->stream->link, &pipe_ctx->link_res, pipe_ctx->stream->signal); |
| 4450 | } | 4489 | } |
| 4451 | #else | 4490 | #else |
| 4452 | disable_link(pipe_ctx->stream->link, pipe_ctx->stream->signal); | 4491 | disable_link(pipe_ctx->stream->link, &pipe_ctx->link_res, pipe_ctx->stream->signal); |
| 4453 | 4492 | ||
| 4454 | dc->hwss.disable_stream(pipe_ctx); | 4493 | dc->hwss.disable_stream(pipe_ctx); |
| 4455 | #endif | 4494 | #endif |
| @@ -4532,16 +4571,22 @@ void dc_link_set_drive_settings(struct dc *dc, · 76 unmodified lines | |||
| 4532 | { | 4571 | { |
| 4533 | 4572 | ||
| 4534 | int i; | 4573 | int i; |
| 4574 | struct pipe_ctx *pipe = NULL; | ||
| 4575 | const struct link_resource *link_res; | ||
| 4535 | 4576 | ||
| 4536 | for (i = 0; i < dc->link_count; i++) { | 4577 | link_res = dc_link_get_cur_link_res(link); |
| 4537 | if (dc->links[i] == link) | ||
| 4538 | break; | ||
| 4539 | } | ||
| 4540 | 4578 | ||
| 4541 | if (i >= dc->link_count) | 4579 | for (i = 0; i < MAX_PIPES; i++) { |
| 4580 | pipe = &dc->current_state->res_ctx.pipe_ctx[i]; | ||
| 4581 | if (pipe->stream && pipe->stream->link) { | ||
| 4582 | if (pipe->stream->link == link) | ||
| 4583 | break; | ||
| 4584 | } | ||
| 4585 | } | ||
| 4586 | if (pipe && link_res) | ||
| 4587 | dc_link_dp_set_drive_settings(pipe->stream->link, link_res, lt_settings); | ||
| 4588 | else | ||
| 4542 | ASSERT_CRITICAL(false); | 4589 | ASSERT_CRITICAL(false); |
| 4543 | |||
| 4544 | dc_link_dp_set_drive_settings(dc->links[i], lt_settings); | ||
| 4545 | } | 4590 | } |
| 4546 | 4591 | ||
| 4547 | void dc_link_set_preferred_link_settings(struct dc *dc, | 4592 | void dc_link_set_preferred_link_settings(struct dc *dc, |
| @@ -4602,11 +4647,9 @@ void dc_link_set_preferred_training_settings(struct dc *dc, · 54 unmodified lines | |||
| 4602 | if (link_setting != NULL) { | 4647 | if (link_setting != NULL) { |
| 4603 | link->preferred_link_setting = *link_setting; | 4648 | link->preferred_link_setting = *link_setting; |
| 4604 | #if defined(CONFIG_DRM_AMD_DC_DCN) | 4649 | #if defined(CONFIG_DRM_AMD_DC_DCN) |
| 4605 | if (dp_get_link_encoding_format(link_setting) == | 4650 | if (dp_get_link_encoding_format(link_setting) == DP_128b_132b_ENCODING) |
| 4606 | DP_128b_132b_ENCODING && !link->hpo_dp_link_enc) { | 4651 | /* TODO: add dc update for acquiring link res */ |
| 4607 | if (!add_dp_hpo_link_encoder_to_link(link)) | 4652 | skip_immediate_retrain = true; |
| 4608 | memset(&link->preferred_link_setting, 0, sizeof(link->preferred_link_setting)); | ||
| 4609 | } | ||
| 4610 | #endif | 4653 | #endif |
| 4611 | } else { | 4654 | } else { |
| 4612 | link->preferred_link_setting.lane_count = LANE_COUNT_UNKNOWN; | 4655 | link->preferred_link_setting.lane_count = LANE_COUNT_UNKNOWN; |
| @@ -4733,6 +4776,9 @@ void dc_link_overwrite_extended_receiver_cap( · 120 unmodified lines | |||
| 4733 | 4776 | ||
| 4734 | bool dc_link_is_fec_supported(const struct dc_link *link) | 4777 | bool dc_link_is_fec_supported(const struct dc_link *link) |
| 4735 | { | 4778 | { |
| 4779 | /* TODO - use asic cap instead of link_enc->features | ||
| 4780 | * we no longer know which link enc to use for this link before commit | ||
| 4781 | */ | ||
| 4736 | struct link_encoder *link_enc = NULL; | 4782 | struct link_encoder *link_enc = NULL; |
| 4737 | 4783 | ||
| 4738 | /* Links supporting dynamically assigned link encoder will be assigned next | 4784 | /* Links supporting dynamically assigned link encoder will be assigned next |
| @@ -4762,6 +4808,8 @@ bool dc_link_should_enable_fec(const struct dc_link *link) · 23 unmodified lines | |||
| 4762 | link->local_sink && | 4808 | link->local_sink && |
| 4763 | link->local_sink->edid_caps.panel_patch.disable_fec) || | 4809 | link->local_sink->edid_caps.panel_patch.disable_fec) || |
| 4764 | (link->connector_signal == SIGNAL_TYPE_EDP | 4810 | (link->connector_signal == SIGNAL_TYPE_EDP |
| 4811 | // enable FEC for EDP if DSC is supported | ||
| 4812 | && link->dpcd_caps.dsc_caps.dsc_basic_caps.fields.dsc_support.DSC_SUPPORT == false | ||
| 4765 | )) | 4813 | )) |
| 4766 | is_fec_disable = true; | 4814 | is_fec_disable = true; |
| 4767 | 4815 | ||
| @@ -4825,3 +4873,125 @@ uint32_t dc_bandwidth_in_kbps_from_timing( · 57 unmodified lines | |||
| 4825 | return kbps; | 4873 | return kbps; |
| 4826 | 4874 | ||
| 4827 | } | 4875 | } |
| 4876 | |||
| 4877 | const struct link_resource *dc_link_get_cur_link_res(const struct dc_link *link) | ||
| 4878 | { | ||
| 4879 | int i; | ||
| 4880 | struct pipe_ctx *pipe = NULL; | ||
| 4881 | const struct link_resource *link_res = NULL; | ||
| 4882 | |||
| 4883 | for (i = 0; i < MAX_PIPES; i++) { | ||
| 4884 | pipe = &link->dc->current_state->res_ctx.pipe_ctx[i]; | ||
| 4885 | if (pipe->stream && pipe->stream->link && pipe->top_pipe == NULL) { | ||
| 4886 | if (pipe->stream->link == link) { | ||
| 4887 | link_res = &pipe->link_res; | ||
| 4888 | break; | ||
| 4889 | } | ||
| 4890 | } | ||
| 4891 | } | ||
| 4892 | |||
| 4893 | return link_res; | ||
| 4894 | } | ||
| 4895 | |||
| 4896 | /** | ||
| 4897 | * dc_get_cur_link_res_map() - take a snapshot of current link resource allocation state | ||
| 4898 | * @dc: pointer to dc of the dm calling this | ||
| 4899 | * @map: a dc link resource snapshot defined internally to dc. | ||
| 4900 | * | ||
| 4901 | * DM needs to capture a snapshot of current link resource allocation mapping | ||
| 4902 | * and store it in its persistent storage. | ||
| 4903 | * | ||
| 4904 | * Some of the link resource is using first come first serve policy. | ||
| 4905 | * The allocation mapping depends on original hotplug order. This information | ||
| 4906 | * is lost after driver is loaded next time. The snapshot is used in order to | ||
| 4907 | * restore link resource to its previous state so user will get consistent | ||
| 4908 | * link capability allocation across reboot. | ||
| 4909 | * | ||
| 4910 | * Return: none (void function) | ||
| 4911 | * | ||
| 4912 | */ | ||
| 4913 | void dc_get_cur_link_res_map(const struct dc *dc, uint32_t *map) | ||
| 4914 | { | ||
| 4915 | #if defined(CONFIG_DRM_AMD_DC_DCN) | ||
| 4916 | struct dc_link *link; | ||
| 4917 | uint8_t i; | ||
| 4918 | uint32_t hpo_dp_recycle_map = 0; | ||
| 4919 | |||
| 4920 | *map = 0; | ||
| 4921 | |||
| 4922 | if (dc->caps.dp_hpo) { | ||
| 4923 | for (i = 0; i < dc->caps.max_links; i++) { | ||
| 4924 | link = dc->links[i]; | ||
| 4925 | if (link->link_status.link_active && | ||
| 4926 | dp_get_link_encoding_format(&link->reported_link_cap) == DP_128b_132b_ENCODING && | ||
| 4927 | dp_get_link_encoding_format(&link->cur_link_settings) != DP_128b_132b_ENCODING) | ||
| 4928 | /* hpo dp link encoder is considered as recycled, when RX reports 128b/132b encoding capability | ||
| 4929 | * but current link doesn't use it. | ||
| 4930 | */ | ||
| 4931 | hpo_dp_recycle_map |= (1 << i); | ||
| 4932 | } | ||
| 4933 | *map |= (hpo_dp_recycle_map << LINK_RES_HPO_DP_REC_MAP__SHIFT); | ||
| 4934 | } | ||
| 4935 | #endif | ||
| 4936 | } | ||
| 4937 | |||
| 4938 | /** | ||
| 4939 | * dc_restore_link_res_map() - restore link resource allocation state from a snapshot | ||
| 4940 | * @dc: pointer to dc of the dm calling this | ||
| 4941 | * @map: a dc link resource snapshot defined internally to dc. | ||
| 4942 | * | ||
| 4943 | * DM needs to call this function after initial link detection on boot and | ||
| 4944 | * before first commit streams to restore link resource allocation state | ||
| 4945 | * from previous boot session. | ||
| 4946 | * | ||
| 4947 | * Some of the link resource is using first come first serve policy. | ||
| 4948 | * The allocation mapping depends on original hotplug order. This information | ||
| 4949 | * is lost after driver is loaded next time. The snapshot is used in order to | ||
| 4950 | * restore link resource to its previous state so user will get consistent | ||
| 4951 | * link capability allocation across reboot. | ||
| 4952 | * | ||
| 4953 | * Return: none (void function) | ||
| 4954 | * | ||
| 4955 | */ | ||
| 4956 | void dc_restore_link_res_map(const struct dc *dc, uint32_t *map) | ||
| 4957 | { | ||
| 4958 | #if defined(CONFIG_DRM_AMD_DC_DCN) | ||
| 4959 | struct dc_link *link; | ||
| 4960 | uint8_t i; | ||
| 4961 | unsigned int available_hpo_dp_count; | ||
| 4962 | uint32_t hpo_dp_recycle_map = (*map & LINK_RES_HPO_DP_REC_MAP__MASK) | ||
| 4963 | >> LINK_RES_HPO_DP_REC_MAP__SHIFT; | ||
| 4964 | |||
| 4965 | if (dc->caps.dp_hpo) { | ||
| 4966 | available_hpo_dp_count = dc->res_pool->hpo_dp_link_enc_count; | ||
| 4967 | /* remove excess 128b/132b encoding support for not recycled links */ | ||
| 4968 | for (i = 0; i < dc->caps.max_links; i++) { | ||
| 4969 | if ((hpo_dp_recycle_map & (1 << i)) == 0) { | ||
| 4970 | link = dc->links[i]; | ||
| 4971 | if (link->type != dc_connection_none && | ||
| 4972 | dp_get_link_encoding_format(&link->verified_link_cap) == DP_128b_132b_ENCODING) { | ||
| 4973 | if (available_hpo_dp_count > 0) | ||
| 4974 | available_hpo_dp_count--; | ||
| 4975 | else | ||
| 4976 | /* remove 128b/132b encoding capability by limiting verified link rate to HBR3 */ | ||
| 4977 | link->verified_link_cap.link_rate = LINK_RATE_HIGH3; | ||
| 4978 | } | ||
| 4979 | } | ||
| 4980 | } | ||
| 4981 | /* remove excess 128b/132b encoding support for recycled links */ | ||
| 4982 | for (i = 0; i < dc->caps.max_links; i++) { | ||
| 4983 | if ((hpo_dp_recycle_map & (1 << i)) != 0) { | ||
| 4984 | link = dc->links[i]; | ||
| 4985 | if (link->type != dc_connection_none && | ||
| 4986 | dp_get_link_encoding_format(&link->verified_link_cap) == DP_128b_132b_ENCODING) { | ||
| 4987 | if (available_hpo_dp_count > 0) | ||
| 4988 | available_hpo_dp_count--; | ||
| 4989 | else | ||
| 4990 | /* remove 128b/132b encoding capability by limiting verified link rate to HBR3 */ | ||
| 4991 | link->verified_link_cap.link_rate = LINK_RATE_HIGH3; | ||
| 4992 | } | ||
| 4993 | } | ||
| 4994 | } | ||
| 4995 | } | ||
| 4996 | #endif | ||
| 4997 | } | ||
| @@ -626,7 +626,7 @@ bool dal_ddc_submit_aux_command(struct ddc_service *ddc, · 625 unmodified lines | |||
| 626 | do { | 626 | do { |
| 627 | struct aux_payload current_payload; | 627 | struct aux_payload current_payload; |
| 628 | bool is_end_of_payload = (retrieved + DEFAULT_AUX_MAX_DATA_SIZE) >= | 628 | bool is_end_of_payload = (retrieved + DEFAULT_AUX_MAX_DATA_SIZE) >= |
| 629 | payload->length ? true : false; | 629 | payload->length; |
| 630 | uint32_t payload_length = is_end_of_payload ? | 630 | uint32_t payload_length = is_end_of_payload ? |
| 631 | payload->length - retrieved : DEFAULT_AUX_MAX_DATA_SIZE; | 631 | payload->length - retrieved : DEFAULT_AUX_MAX_DATA_SIZE; |
| 632 | 632 | ||
| @@ -100,6 +100,7 @@ static const struct dp_lt_fallback_entry dp_lt_fallbacks[] = { · 99 unmodified lines | |||
| 100 | #endif | 100 | #endif |
| 101 | 101 | ||
| 102 | static bool decide_fallback_link_setting( | 102 | static bool decide_fallback_link_setting( |
| 103 | struct dc_link *link, | ||
| 103 | struct dc_link_settings initial_link_settings, | 104 | struct dc_link_settings initial_link_settings, |
| 104 | struct dc_link_settings *current_link_setting, | 105 | struct dc_link_settings *current_link_setting, |
| 105 | enum link_training_result training_result); | 106 | enum link_training_result training_result); |
| @@ -398,6 +399,223 @@ static uint8_t get_dpcd_link_rate(const struct dc_link_settings *link_settings) · 292 unmodified lines | |||
| 398 | } | 399 | } |
| 399 | #endif | 400 | #endif |
| 400 | 401 | ||
| 402 | static void vendor_specific_lttpr_wa_one_start(struct dc_link *link) | ||
| 403 | { | ||
| 404 | const uint8_t vendor_lttpr_write_data[4] = {0x1, 0x50, 0x63, 0xff}; | ||
| 405 | const uint8_t offset = dp_convert_to_count( | ||
| 406 | link->dpcd_caps.lttpr_caps.phy_repeater_cnt); | ||
| 407 | uint32_t vendor_lttpr_write_address = 0xF004F; | ||
| 408 | |||
| 409 | if (offset != 0xFF) | ||
| 410 | vendor_lttpr_write_address += | ||
| 411 | ((DP_REPEATER_CONFIGURATION_AND_STATUS_SIZE) * (offset - 1)); | ||
| 412 | |||
| 413 | /* W/A for certain LTTPR to reset their lane settings, part one of two */ | ||
| 414 | core_link_write_dpcd( | ||
| 415 | link, | ||
| 416 | vendor_lttpr_write_address, | ||
| 417 | &vendor_lttpr_write_data[0], | ||
| 418 | sizeof(vendor_lttpr_write_data)); | ||
| 419 | } | ||
| 420 | |||
| 421 | static void vendor_specific_lttpr_wa_one_end( | ||
| 422 | struct dc_link *link, | ||
| 423 | uint8_t retry_count) | ||
| 424 | { | ||
| 425 | const uint8_t vendor_lttpr_write_data[4] = {0x1, 0x50, 0x63, 0x0}; | ||
| 426 | const uint8_t offset = dp_convert_to_count( | ||
| 427 | link->dpcd_caps.lttpr_caps.phy_repeater_cnt); | ||
| 428 | uint32_t vendor_lttpr_write_address = 0xF004F; | ||
| 429 | |||
| 430 | if (!retry_count) { | ||
| 431 | if (offset != 0xFF) | ||
| 432 | vendor_lttpr_write_address += | ||
| 433 | ((DP_REPEATER_CONFIGURATION_AND_STATUS_SIZE) * (offset - 1)); | ||
| 434 | |||
| 435 | /* W/A for certain LTTPR to reset their lane settings, part two of two */ | ||
| 436 | core_link_write_dpcd( | ||
| 437 | link, | ||
| 438 | vendor_lttpr_write_address, | ||
| 439 | &vendor_lttpr_write_data[0], | ||
| 440 | sizeof(vendor_lttpr_write_data)); | ||
| 441 | } | ||
| 442 | } | ||
| 443 | |||
| 444 | static void vendor_specific_lttpr_wa_one_two( | ||
| 445 | struct dc_link *link, | ||
| 446 | const uint8_t rate) | ||
| 447 | { | ||
| 448 | if (link->apply_vendor_specific_lttpr_link_rate_wa) { | ||
| 449 | uint8_t toggle_rate = 0x0; | ||
| 450 | |||
| 451 | if (rate == 0x6) | ||
| 452 | toggle_rate = 0xA; | ||
| 453 | else | ||
| 454 | toggle_rate = 0x6; | ||
| 455 | |||
| 456 | if (link->vendor_specific_lttpr_link_rate_wa == rate) { | ||
| 457 | /* W/A for certain LTTPR to reset internal state for link training */ | ||
| 458 | core_link_write_dpcd( | ||
| 459 | link, | ||
| 460 | DP_LINK_BW_SET, | ||
| 461 | &toggle_rate, | ||
| 462 | 1); | ||
| 463 | } | ||
| 464 | |||
| 465 | /* Store the last attempted link rate for this link */ | ||
| 466 | link->vendor_specific_lttpr_link_rate_wa = rate; | ||
| 467 | } | ||
| 468 | } | ||
| 469 | |||
| 470 | static void vendor_specific_lttpr_wa_three( | ||
| 471 | struct dc_link *link, | ||
| 472 | union lane_adjust dpcd_lane_adjust[LANE_COUNT_DP_MAX]) | ||
| 473 | { | ||
| 474 | const uint8_t vendor_lttpr_write_data_vs[3] = {0x0, 0x53, 0x63}; | ||
| 475 | const uint8_t vendor_lttpr_write_data_pe[3] = {0x0, 0x54, 0x63}; | ||
| 476 | const uint8_t offset = dp_convert_to_count( | ||
| 477 | link->dpcd_caps.lttpr_caps.phy_repeater_cnt); | ||
| 478 | uint32_t vendor_lttpr_write_address = 0xF004F; | ||
| 479 | uint32_t vendor_lttpr_read_address = 0xF0053; | ||
| 480 | uint8_t dprx_vs = 0; | ||
| 481 | uint8_t dprx_pe = 0; | ||
| 482 | uint8_t lane; | ||
| 483 | |||
| 484 | if (offset != 0xFF) { | ||
| 485 | vendor_lttpr_write_address += | ||
| 486 | ((DP_REPEATER_CONFIGURATION_AND_STATUS_SIZE) * (offset - 1)); | ||
| 487 | vendor_lttpr_read_address += | ||
| 488 | ((DP_REPEATER_CONFIGURATION_AND_STATUS_SIZE) * (offset - 1)); | ||
| 489 | } | ||
| 490 | |||
| 491 | /* W/A to read lane settings requested by DPRX */ | ||
| 492 | core_link_write_dpcd( | ||
| 493 | link, | ||
| 494 | vendor_lttpr_write_address, | ||
| 495 | &vendor_lttpr_write_data_vs[0], | ||
| 496 | sizeof(vendor_lttpr_write_data_vs)); | ||
| 497 | core_link_read_dpcd( | ||
| 498 | link, | ||
| 499 | vendor_lttpr_read_address, | ||
| 500 | &dprx_vs, | ||
| 501 | 1); | ||
| 502 | core_link_write_dpcd( | ||
| 503 | link, | ||
| 504 | vendor_lttpr_write_address, | ||
| 505 | &vendor_lttpr_write_data_pe[0], | ||
| 506 | sizeof(vendor_lttpr_write_data_pe)); | ||
| 507 | core_link_read_dpcd( | ||
| 508 | link, | ||
| 509 | vendor_lttpr_read_address, | ||
| 510 | &dprx_pe, | ||
| 511 | 1); | ||
| 512 | |||
| 513 | for (lane = 0; lane < LANE_COUNT_DP_MAX; lane++) { | ||
| 514 | dpcd_lane_adjust[lane].bits.VOLTAGE_SWING_LANE = (dprx_vs >> (2 * lane)) & 0x3; | ||
| 515 | dpcd_lane_adjust[lane].bits.PRE_EMPHASIS_LANE = (dprx_pe >> (2 * lane)) & 0x3; | ||
| 516 | } | ||
| 517 | } | ||
| 518 | |||
| 519 | static void vendor_specific_lttpr_wa_three_dpcd( | ||
| 520 | struct dc_link *link, | ||
| 521 | union dpcd_training_lane dpcd_lane_adjust[LANE_COUNT_DP_MAX]) | ||
| 522 | { | ||
| 523 | union lane_adjust lane_adjust[LANE_COUNT_DP_MAX]; | ||
| 524 | uint8_t lane = 0; | ||
| 525 | |||
| 526 | vendor_specific_lttpr_wa_three(link, lane_adjust); | ||
| 527 | |||
| 528 | for (lane = 0; lane < LANE_COUNT_DP_MAX; lane++) { | ||
| 529 | dpcd_lane_adjust[lane].bits.VOLTAGE_SWING_SET = lane_adjust[lane].bits.VOLTAGE_SWING_LANE; | ||
| 530 | dpcd_lane_adjust[lane].bits.PRE_EMPHASIS_SET = lane_adjust[lane].bits.PRE_EMPHASIS_LANE; | ||
| 531 | } | ||
| 532 | } | ||
| 533 | |||
| 534 | static void vendor_specific_lttpr_wa_four( | ||
| 535 | struct dc_link *link, | ||
| 536 | bool apply_wa) | ||
| 537 | { | ||
| 538 | const uint8_t vendor_lttpr_write_data_one[4] = {0x1, 0x55, 0x63, 0x8}; | ||
| 539 | const uint8_t vendor_lttpr_write_data_two[4] = {0x1, 0x55, 0x63, 0x0}; | ||
| 540 | const uint8_t offset = dp_convert_to_count( | ||
| 541 | link->dpcd_caps.lttpr_caps.phy_repeater_cnt); | ||
| 542 | uint32_t vendor_lttpr_write_address = 0xF004F; | ||
| 543 | #if defined(CONFIG_DRM_AMD_DC_DP2_0) | ||
| 544 | uint8_t sink_status = 0; | ||
| 545 | uint8_t i; | ||
| 546 | #endif | ||
| 547 | |||
| 548 | if (offset != 0xFF) | ||
| 549 | vendor_lttpr_write_address += | ||
| 550 | ((DP_REPEATER_CONFIGURATION_AND_STATUS_SIZE) * (offset - 1)); | ||
| 551 | |||
| 552 | /* W/A to pass through DPCD write of TPS=0 to DPRX */ | ||
| 553 | if (apply_wa) { | ||
| 554 | core_link_write_dpcd( | ||
| 555 | link, | ||
| 556 | vendor_lttpr_write_address, | ||
| 557 | &vendor_lttpr_write_data_one[0], | ||
| 558 | sizeof(vendor_lttpr_write_data_one)); | ||
| 559 | } | ||
| 560 | |||
| 561 | /* clear training pattern set */ | ||
| 562 | dpcd_set_training_pattern(link, DP_TRAINING_PATTERN_VIDEOIDLE); | ||
| 563 | |||
| 564 | if (apply_wa) { | ||
| 565 | core_link_write_dpcd( | ||
| 566 | link, | ||
| 567 | vendor_lttpr_write_address, | ||
| 568 | &vendor_lttpr_write_data_two[0], | ||
| 569 | sizeof(vendor_lttpr_write_data_two)); | ||
| 570 | } | ||
| 571 | |||
| 572 | #if defined(CONFIG_DRM_AMD_DC_DP2_0) | ||
| 573 | /* poll for intra-hop disable */ | ||
| 574 | for (i = 0; i < 10; i++) { | ||
| 575 | if ((core_link_read_dpcd(link, DP_SINK_STATUS, &sink_status, 1) == DC_OK) && | ||
| 576 | (sink_status & DP_INTRA_HOP_AUX_REPLY_INDICATION) == 0) | ||
| 577 | break; | ||
| 578 | udelay(1000); | ||
| 579 | } | ||
| 580 | #endif | ||
| 581 | } | ||
| 582 | |||
| 583 | static void vendor_specific_lttpr_wa_five( | ||
| 584 | struct dc_link *link, | ||
| 585 | const union dpcd_training_lane dpcd_lane_adjust[LANE_COUNT_DP_MAX], | ||
| 586 | uint8_t lane_count) | ||
| 587 | { | ||
| 588 | const uint32_t vendor_lttpr_write_address = 0xF004F; | ||
| 589 | const uint8_t vendor_lttpr_write_data_reset[4] = {0x1, 0x50, 0x63, 0xFF}; | ||
| 590 | uint8_t vendor_lttpr_write_data_vs[4] = {0x1, 0x51, 0x63, 0x0}; | ||
| 591 | uint8_t vendor_lttpr_write_data_pe[4] = {0x1, 0x52, 0x63, 0x0}; | ||
| 592 | uint8_t lane = 0; | ||
| 593 | |||
| 594 | for (lane = 0; lane < lane_count; lane++) { | ||
| 595 | vendor_lttpr_write_data_vs[3] |= | ||
| 596 | dpcd_lane_adjust[lane].bits.VOLTAGE_SWING_SET << (2 * lane); | ||
| 597 | vendor_lttpr_write_data_pe[3] |= | ||
| 598 | dpcd_lane_adjust[lane].bits.PRE_EMPHASIS_SET << (2 * lane); | ||
| 599 | } | ||
| 600 | |||
| 601 | /* Force LTTPR to output desired VS and PE */ | ||
| 602 | core_link_write_dpcd( | ||
| 603 | link, | ||
| 604 | vendor_lttpr_write_address, | ||
| 605 | &vendor_lttpr_write_data_reset[0], | ||
| 606 | sizeof(vendor_lttpr_write_data_reset)); | ||
| 607 | core_link_write_dpcd( | ||
| 608 | link, | ||
| 609 | vendor_lttpr_write_address, | ||
| 610 | &vendor_lttpr_write_data_vs[0], | ||
| 611 | sizeof(vendor_lttpr_write_data_vs)); | ||
| 612 | core_link_write_dpcd( | ||
| 613 | link, | ||
| 614 | vendor_lttpr_write_address, | ||
| 615 | &vendor_lttpr_write_data_pe[0], | ||
| 616 | sizeof(vendor_lttpr_write_data_pe)); | ||
| 617 | } | ||
| 618 | |||
| 401 | enum dc_status dpcd_set_link_settings( | 619 | enum dc_status dpcd_set_link_settings( |
| 402 | struct dc_link *link, | 620 | struct dc_link *link, |
| 403 | const struct link_training_settings *lt_settings) | 621 | const struct link_training_settings *lt_settings) |
| @@ -430,7 +648,7 @@ enum dc_status dpcd_set_link_settings( · 26 unmodified lines | |||
| 430 | status = core_link_write_dpcd(link, DP_LANE_COUNT_SET, | 648 | status = core_link_write_dpcd(link, DP_LANE_COUNT_SET, |
| 431 | &lane_count_set.raw, 1); | 649 | &lane_count_set.raw, 1); |
| 432 | 650 | ||
| 433 | if (link->dpcd_caps.dpcd_rev.raw >= DPCD_REV_14 && | 651 | if (link->dpcd_caps.dpcd_rev.raw >= DPCD_REV_13 && |
| 434 | lt_settings->link_settings.use_link_rate_set == true) { | 652 | lt_settings->link_settings.use_link_rate_set == true) { |
| 435 | rate = 0; | 653 | rate = 0; |
| 436 | /* WA for some MUX chips that will power down with eDP and lose supported | 654 | /* WA for some MUX chips that will power down with eDP and lose supported |
| @@ -452,6 +670,15 @@ enum dc_status dpcd_set_link_settings( · 15 unmodified lines | |||
| 452 | #else | 670 | #else |
| 453 | rate = (uint8_t) (lt_settings->link_settings.link_rate); | 671 | rate = (uint8_t) (lt_settings->link_settings.link_rate); |
| 454 | #endif | 672 | #endif |
| 673 | if (link->dc->debug.apply_vendor_specific_lttpr_wa && | ||
| 674 | (link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN) && | ||
| 675 | link->lttpr_mode == LTTPR_MODE_TRANSPARENT) | ||
| 676 | vendor_specific_lttpr_wa_one_start(link); | ||
| 677 | |||
| 678 | if (link->dc->debug.apply_vendor_specific_lttpr_wa && | ||
| 679 | (link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN)) | ||
| 680 | vendor_specific_lttpr_wa_one_two(link, rate); | ||
| 681 | |||
| 455 | status = core_link_write_dpcd(link, DP_LINK_BW_SET, &rate, 1); | 682 | status = core_link_write_dpcd(link, DP_LINK_BW_SET, &rate, 1); |
| 456 | } | 683 | } |
| 457 | 684 | ||
| @@ -1024,6 +1251,7 @@ bool dp_is_max_vs_reached( · 566 unmodified lines | |||
| 1024 | 1251 | ||
| 1025 | static bool perform_post_lt_adj_req_sequence( | 1252 | static bool perform_post_lt_adj_req_sequence( |
| 1026 | struct dc_link *link, | 1253 | struct dc_link *link, |
| 1254 | const struct link_resource *link_res, | ||
| 1027 | struct link_training_settings *lt_settings) | 1255 | struct link_training_settings *lt_settings) |
| 1028 | { | 1256 | { |
| 1029 | enum dc_lane_count lane_count = | 1257 | enum dc_lane_count lane_count = |
| @@ -1087,6 +1315,7 @@ static bool perform_post_lt_adj_req_sequence( · 57 unmodified lines | |||
| 1087 | lt_settings->hw_lane_settings, lt_settings->dpcd_lane_settings); | 1315 | lt_settings->hw_lane_settings, lt_settings->dpcd_lane_settings); |
| 1088 | 1316 | ||
| 1089 | dc_link_dp_set_drive_settings(link, | 1317 | dc_link_dp_set_drive_settings(link, |
| 1318 | link_res, | ||
| 1090 | lt_settings); | 1319 | lt_settings); |
| 1091 | break; | 1320 | break; |
| 1092 | } | 1321 | } |
| @@ -1161,6 +1390,7 @@ enum link_training_result dp_get_cr_failure(enum dc_lane_count ln_count, · 68 unmodified lines | |||
| 1161 | 1390 | ||
| 1162 | static enum link_training_result perform_channel_equalization_sequence( | 1391 | static enum link_training_result perform_channel_equalization_sequence( |
| 1163 | struct dc_link *link, | 1392 | struct dc_link *link, |
| 1393 | const struct link_resource *link_res, | ||
| 1164 | struct link_training_settings *lt_settings, | 1394 | struct link_training_settings *lt_settings, |
| 1165 | uint32_t offset) | 1395 | uint32_t offset) |
| 1166 | { | 1396 | { |
| @@ -1183,12 +1413,12 @@ static enum link_training_result perform_channel_equalization_sequence( · 16 unmodified lines | |||
| 1183 | tr_pattern = DP_TRAINING_PATTERN_SEQUENCE_4; | 1413 | tr_pattern = DP_TRAINING_PATTERN_SEQUENCE_4; |
| 1184 | #endif | 1414 | #endif |
| 1185 | 1415 | ||
| 1186 | dp_set_hw_training_pattern(link, tr_pattern, offset); | 1416 | dp_set_hw_training_pattern(link, link_res, tr_pattern, offset); |
| 1187 | 1417 | ||
| 1188 | for (retries_ch_eq = 0; retries_ch_eq <= LINK_TRAINING_MAX_RETRY_COUNT; | 1418 | for (retries_ch_eq = 0; retries_ch_eq <= LINK_TRAINING_MAX_RETRY_COUNT; |
| 1189 | retries_ch_eq++) { | 1419 | retries_ch_eq++) { |
| 1190 | 1420 | ||
| 1191 | dp_set_hw_lane_settings(link, lt_settings, offset); | 1421 | dp_set_hw_lane_settings(link, link_res, lt_settings, offset); |
| 1192 | 1422 | ||
| 1193 | /* 2. update DPCD*/ | 1423 | /* 2. update DPCD*/ |
| 1194 | if (!retries_ch_eq) | 1424 | if (!retries_ch_eq) |
| @@ -1211,6 +1441,12 @@ static enum link_training_result perform_channel_equalization_sequence( · 16 unmodified lines | |||
| 1211 | dp_translate_training_aux_read_interval( | 1441 | dp_translate_training_aux_read_interval( |
| 1212 | link->dpcd_caps.lttpr_caps.aux_rd_interval[offset - 1]); | 1442 | link->dpcd_caps.lttpr_caps.aux_rd_interval[offset - 1]); |
| 1213 | 1443 | ||
| 1444 | if (link->dc->debug.apply_vendor_specific_lttpr_wa && | ||
| 1445 | (link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN) && | ||
| 1446 | link->lttpr_mode == LTTPR_MODE_TRANSPARENT) { | ||
| 1447 | wait_time_microsec = 16000; | ||
| 1448 | } | ||
| 1449 | |||
| 1214 | dp_wait_for_training_aux_rd_interval( | 1450 | dp_wait_for_training_aux_rd_interval( |
| 1215 | link, | 1451 | link, |
| 1216 | wait_time_microsec); | 1452 | wait_time_microsec); |
| @@ -1246,18 +1482,20 @@ static enum link_training_result perform_channel_equalization_sequence( · 29 unmodified lines | |||
| 1246 | } | 1482 | } |
| 1247 | 1483 | ||
| 1248 | static void start_clock_recovery_pattern_early(struct dc_link *link, | 1484 | static void start_clock_recovery_pattern_early(struct dc_link *link, |
| 1485 | const struct link_resource *link_res, | ||
| 1249 | struct link_training_settings *lt_settings, | 1486 | struct link_training_settings *lt_settings, |
| 1250 | uint32_t offset) | 1487 | uint32_t offset) |
| 1251 | { | 1488 | { |
| 1252 | DC_LOG_HW_LINK_TRAINING("%s\n GPU sends TPS1. Wait 400us.\n", | 1489 | DC_LOG_HW_LINK_TRAINING("%s\n GPU sends TPS1. Wait 400us.\n", |
| 1253 | __func__); | 1490 | __func__); |
| 1254 | dp_set_hw_training_pattern(link, lt_settings->pattern_for_cr, offset); | 1491 | dp_set_hw_training_pattern(link, link_res, lt_settings->pattern_for_cr, offset); |
| 1255 | dp_set_hw_lane_settings(link, lt_settings, offset); | 1492 | dp_set_hw_lane_settings(link, link_res, lt_settings, offset); |
| 1256 | udelay(400); | 1493 | udelay(400); |
| 1257 | } | 1494 | } |
| 1258 | 1495 | ||
| 1259 | static enum link_training_result perform_clock_recovery_sequence( | 1496 | static enum link_training_result perform_clock_recovery_sequence( |
| 1260 | struct dc_link *link, | 1497 | struct dc_link *link, |
| 1498 | const struct link_resource *link_res, | ||
| 1261 | struct link_training_settings *lt_settings, | 1499 | struct link_training_settings *lt_settings, |
| 1262 | uint32_t offset) | 1500 | uint32_t offset) |
| 1263 | { | 1501 | { |
| @@ -1273,7 +1511,7 @@ static enum link_training_result perform_clock_recovery_sequence( · 9 unmodified lines | |||
| 1273 | retry_count = 0; | 1511 | retry_count = 0; |
| 1274 | 1512 | ||
| 1275 | if (!link->ctx->dc->work_arounds.lt_early_cr_pattern) | 1513 | if (!link->ctx->dc->work_arounds.lt_early_cr_pattern) |
| 1276 | dp_set_hw_training_pattern(link, lt_settings->pattern_for_cr, offset); | 1514 | dp_set_hw_training_pattern(link, link_res, lt_settings->pattern_for_cr, offset); |
| 1277 | 1515 | ||
| 1278 | /* najeeb - The synaptics MST hub can put the LT in | 1516 | /* najeeb - The synaptics MST hub can put the LT in |
| 1279 | * infinite loop by switching the VS | 1517 | * infinite loop by switching the VS |
| @@ -1290,6 +1528,7 @@ static enum link_training_result perform_clock_recovery_sequence( · 10 unmodified lines | |||
| 1290 | /* 1. call HWSS to set lane settings*/ | 1528 | /* 1. call HWSS to set lane settings*/ |
| 1291 | dp_set_hw_lane_settings( | 1529 | dp_set_hw_lane_settings( |
| 1292 | link, | 1530 | link, |
| 1531 | link_res, | ||
| 1293 | lt_settings, | 1532 | lt_settings, |
| 1294 | offset); | 1533 | offset); |
| 1295 | 1534 | ||
| @@ -1311,8 +1550,10 @@ static enum link_training_result perform_clock_recovery_sequence( · 15 unmodified lines | |||
| 1311 | /* 3. wait receiver to lock-on*/ | 1550 | /* 3. wait receiver to lock-on*/ |
| 1312 | wait_time_microsec = lt_settings->cr_pattern_time; | 1551 | wait_time_microsec = lt_settings->cr_pattern_time; |
| 1313 | 1552 | ||
| 1314 | if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT) | 1553 | if (link->dc->debug.apply_vendor_specific_lttpr_wa && |
| 1315 | wait_time_microsec = TRAINING_AUX_RD_INTERVAL; | 1554 | (link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN)) { |
| 1555 | wait_time_microsec = 16000; | ||
| 1556 | } | ||
| 1316 | 1557 | ||
| 1317 | dp_wait_for_training_aux_rd_interval( | 1558 | dp_wait_for_training_aux_rd_interval( |
| 1318 | link, | 1559 | link, |
| @@ -1329,6 +1570,13 @@ static enum link_training_result perform_clock_recovery_sequence( · 10 unmodified lines | |||
| 1329 | dpcd_lane_adjust, | 1570 | dpcd_lane_adjust, |
| 1330 | offset); | 1571 | offset); |
| 1331 | 1572 | ||
| 1573 | if (link->dc->debug.apply_vendor_specific_lttpr_wa && | ||
| 1574 | (link->chip_caps & EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN) && | ||
| 1575 | link->lttpr_mode == LTTPR_MODE_TRANSPARENT) { | ||
| 1576 | vendor_specific_lttpr_wa_one_end(link, retry_count); | ||
| 1577 | vendor_specific_lttpr_wa_three(link, dpcd_lane_adjust); | ||
| 1578 | } | ||
| 1579 | |||
| 1332 | /* 5. check CR done*/ | 1580 | /* 5. check CR done*/ |
| 1333 | if (dp_is_cr_done(lane_count, dpcd_lane_status)) | 1581 | if (dp_is_cr_done(lane_count, dpcd_lane_status)) |
| 1334 | return LINK_TRAINING_SUCCESS; | 1582 | return LINK_TRAINING_SUCCESS; |
| @@ -1379,13 +1627,14 @@ static enum link_training_result perform_clock_recovery_sequence( · 44 unmodified lines | |||
| 1379 | 1627 | ||
| 1380 | static inline enum link_training_result dp_transition_to_video_idle( | 1628 | static inline enum link_training_result dp_transition_to_video_idle( |
| 1381 | struct dc_link *link, | 1629 | struct dc_link *link, |
| 1630 | const struct link_resource *link_res, | ||
| 1382 | struct link_training_settings *lt_settings, | 1631 | struct link_training_settings *lt_settings, |
| 1383 | enum link_training_result status) | 1632 | enum link_training_result status) |
| 1384 | { | 1633 | { |
| 1385 | union lane_count_set lane_count_set = {0}; | 1634 | union lane_count_set lane_count_set = {0}; |
| 1386 | 1635 | ||
| 1387 | /* 4. mainlink output idle pattern*/ | 1636 | /* 4. mainlink output idle pattern*/ |
| 1388 | dp_set_hw_test_pattern(link, DP_TEST_PATTERN_VIDEO_MODE, NULL, 0); | 1637 | dp_set_hw_test_pattern(link, link_res, DP_TEST_PATTERN_VIDEO_MODE, NULL, 0); |
| 1389 | 1638 | ||
| 1390 | /* | 1639 | /* |
| 1391 | * 5. post training adjust if required | 1640 | * 5. post training adjust if required |
| @@ -1409,7 +1658,7 @@ static inline enum link_training_result dp_transition_to_video_idle( · 17 unmodified lines | |||
| 1409 | } | 1658 | } |
| 1410 | 1659 | ||
| 1411 | if (status == LINK_TRAINING_SUCCESS && | 1660 | if (status == LINK_TRAINING_SUCCESS && |
| 1412 | perform_post_lt_adj_req_sequence(link, lt_settings) == false) | 1661 | perform_post_lt_adj_req_sequence(link, link_res, lt_settings) == false) |
| 1413 | status = LINK_TRAINING_LQA_FAIL; | 1662 | status = LINK_TRAINING_LQA_FAIL; |
| 1414 | 1663 | ||
| 1415 | lane_count_set.bits.LANE_COUNT_SET = lt_settings->link_settings.lane_count; | 1664 | lane_count_set.bits.LANE_COUNT_SET = lt_settings->link_settings.lane_count; |
| @@ -1852,10 +2101,11 @@ static void print_status_message( · 436 unmodified lines | |||
| 1852 | 2101 | ||
| 1853 | void dc_link_dp_set_drive_settings( | 2102 | void dc_link_dp_set_drive_settings( |
| 1854 | struct dc_link *link, | 2103 | struct dc_link *link, |
| 2104 | const struct link_resource *link_res, | ||
| 1855 | struct link_training_settings *lt_settings) | 2105 | struct link_training_settings *lt_settings) |
| 1856 | { | 2106 | { |
| 1857 | /* program ASIC PHY settings*/ | 2107 | /* program ASIC PHY settings*/ |
| 1858 | dp_set_hw_lane_settings(link, lt_settings, DPRX); | 2108 | dp_set_hw_lane_settings(link, link_res, lt_settings, DPRX); |
| 1859 | 2109 | ||
| 1860 | dp_hw_to_dpcd_lane_settings(lt_settings, | 2110 | dp_hw_to_dpcd_lane_settings(lt_settings, |
| 1861 | lt_settings->hw_lane_settings, lt_settings->dpcd_lane_settings); | 2111 | lt_settings->hw_lane_settings, lt_settings->dpcd_lane_settings); |
| @@ -1866,6 +2116,7 @@ void dc_link_dp_set_drive_settings( · 4 unmodified lines | |||
| 1866 | 2116 | ||
| 1867 | bool dc_link_dp_perform_link_training_skip_aux( | 2117 | bool dc_link_dp_perform_link_training_skip_aux( |
| 1868 | struct dc_link *link, | 2118 | struct dc_link *link, |
| 2119 | const struct link_resource *link_res, | ||
| 1869 | const struct dc_link_settings *link_setting) | 2120 | const struct dc_link_settings *link_setting) |
| 1870 | { | 2121 | { |
| 1871 | struct link_training_settings lt_settings = {0}; | 2122 | struct link_training_settings lt_settings = {0}; |
| @@ -1882,10 +2133,10 @@ bool dc_link_dp_perform_link_training_skip_aux( · 10 unmodified lines | |||
| 1882 | /* 1. Perform_clock_recovery_sequence. */ | 2133 | /* 1. Perform_clock_recovery_sequence. */ |
| 1883 | 2134 | ||
| 1884 | /* transmit training pattern for clock recovery */ | 2135 | /* transmit training pattern for clock recovery */ |
| 1885 | dp_set_hw_training_pattern(link, lt_settings.pattern_for_cr, DPRX); | 2136 | dp_set_hw_training_pattern(link, link_res, lt_settings.pattern_for_cr, DPRX); |
| 1886 | 2137 | ||
| 1887 | /* call HWSS to set lane settings*/ | 2138 | /* call HWSS to set lane settings*/ |
| 1888 | dp_set_hw_lane_settings(link, <_settings, DPRX); | 2139 | dp_set_hw_lane_settings(link, link_res, <_settings, DPRX); |
| 1889 | 2140 | ||
| 1890 | /* wait receiver to lock-on*/ | 2141 | /* wait receiver to lock-on*/ |
| 1891 | dp_wait_for_training_aux_rd_interval(link, lt_settings.cr_pattern_time); | 2142 | dp_wait_for_training_aux_rd_interval(link, lt_settings.cr_pattern_time); |
| @@ -1893,10 +2144,10 @@ bool dc_link_dp_perform_link_training_skip_aux( · 1 unmodified line | |||
| 1893 | /* 2. Perform_channel_equalization_sequence. */ | 2144 | /* 2. Perform_channel_equalization_sequence. */ |
| 1894 | 2145 | ||
| 1895 | /* transmit training pattern for channel equalization. */ | 2146 | /* transmit training pattern for channel equalization. */ |
| 1896 | dp_set_hw_training_pattern(link, lt_settings.pattern_for_eq, DPRX); | 2147 | dp_set_hw_training_pattern(link, link_res, lt_settings.pattern_for_eq, DPRX); |
| 1897 | 2148 | ||
| 1898 | /* call HWSS to set lane settings*/ | 2149 | /* call HWSS to set lane settings*/ |
| 1899 | dp_set_hw_lane_settings(link, <_settings, DPRX); | 2150 | dp_set_hw_lane_settings(link, link_res, <_settings, DPRX); |
| 1900 | 2151 | ||
| 1901 | /* wait receiver to lock-on. */ | 2152 | /* wait receiver to lock-on. */ |
| 1902 | dp_wait_for_training_aux_rd_interval(link, lt_settings.eq_pattern_time); | 2153 | dp_wait_for_training_aux_rd_interval(link, lt_settings.eq_pattern_time); |
| @@ -1904,7 +2155,7 @@ bool dc_link_dp_perform_link_training_skip_aux( · 1 unmodified line | |||
| 1904 | /* 3. Perform_link_training_int. */ | 2155 | /* 3. Perform_link_training_int. */ |
| 1905 | 2156 | ||
| 1906 | /* Mainlink output idle pattern. */ | 2157 | /* Mainlink output idle pattern. */ |
| 1907 | dp_set_hw_test_pattern(link, DP_TEST_PATTERN_VIDEO_MODE, NULL, 0); | 2158 | dp_set_hw_test_pattern(link, link_res, DP_TEST_PATTERN_VIDEO_MODE, NULL, 0); |
| 1908 | 2159 | ||
| 1909 | print_status_message(link, <_settings, LINK_TRAINING_SUCCESS); | 2160 | print_status_message(link, <_settings, LINK_TRAINING_SUCCESS); |
| 1910 | 2161 | ||
| @@ -1985,6 +2236,7 @@ static void dpcd_128b_132b_get_aux_rd_interval(struct dc_link *link, · 74 unmodified lines | |||
| 1985 | 2236 | ||
| 1986 | static enum link_training_result dp_perform_128b_132b_channel_eq_done_sequence( | 2237 | static enum link_training_result dp_perform_128b_132b_channel_eq_done_sequence( |
| 1987 | struct dc_link *link, | 2238 | struct dc_link *link, |
| 2239 | const struct link_resource *link_res, | ||
| 1988 | struct link_training_settings *lt_settings) | 2240 | struct link_training_settings *lt_settings) |
| 1989 | { | 2241 | { |
| 1990 | uint8_t loop_count; | 2242 | uint8_t loop_count; |
| @@ -1996,7 +2248,7 @@ static enum link_training_result dp_perform_128b_132b_channel_eq_done_sequence( · 5 unmodified lines | |||
| 1996 | union lane_adjust dpcd_lane_adjust[LANE_COUNT_DP_MAX] = {0}; | 2248 | union lane_adjust dpcd_lane_adjust[LANE_COUNT_DP_MAX] = {0}; |
| 1997 | 2249 | ||
| 1998 | /* Transmit 128b/132b_TPS1 over Main-Link */ | 2250 | /* Transmit 128b/132b_TPS1 over Main-Link */ |
| 1999 | dp_set_hw_training_pattern(link, lt_settings->pattern_for_cr, DPRX); | 2251 | dp_set_hw_training_pattern(link, link_res, lt_settings->pattern_for_cr, DPRX); |
| 2000 | /* Set TRAINING_PATTERN_SET to 01h */ | 2252 | /* Set TRAINING_PATTERN_SET to 01h */ |
| 2001 | dpcd_set_training_pattern(link, lt_settings->pattern_for_cr); | 2253 | dpcd_set_training_pattern(link, lt_settings->pattern_for_cr); |
| 2002 | 2254 | ||
| @@ -2006,8 +2258,8 @@ static enum link_training_result dp_perform_128b_132b_channel_eq_done_sequence( · 3 unmodified lines | |||
| 2006 | &dpcd_lane_status_updated, dpcd_lane_adjust, DPRX); | 2258 | &dpcd_lane_status_updated, dpcd_lane_adjust, DPRX); |
| 2007 | dp_decide_lane_settings(lt_settings, dpcd_lane_adjust, | 2259 | dp_decide_lane_settings(lt_settings, dpcd_lane_adjust, |
| 2008 | lt_settings->hw_lane_settings, lt_settings->dpcd_lane_settings); | 2260 | lt_settings->hw_lane_settings, lt_settings->dpcd_lane_settings); |
| 2009 | dp_set_hw_lane_settings(link, lt_settings, DPRX); | 2261 | dp_set_hw_lane_settings(link, link_res, lt_settings, DPRX); |
| 2010 | dp_set_hw_training_pattern(link, lt_settings->pattern_for_eq, DPRX); | 2262 | dp_set_hw_training_pattern(link, link_res, lt_settings->pattern_for_eq, DPRX); |
| 2011 | 2263 | ||
| 2012 | /* Set loop counter to start from 1 */ | 2264 | /* Set loop counter to start from 1 */ |
| 2013 | loop_count = 1; | 2265 | loop_count = 1; |
| @@ -2034,7 +2286,7 @@ static enum link_training_result dp_perform_128b_132b_channel_eq_done_sequence( · 20 unmodified lines | |||
| 2034 | } else if (dpcd_lane_status_updated.bits.LT_FAILED_128b_132b) { | 2286 | } else if (dpcd_lane_status_updated.bits.LT_FAILED_128b_132b) { |
| 2035 | status = DP_128b_132b_LT_FAILED; | 2287 | status = DP_128b_132b_LT_FAILED; |
| 2036 | } else { | 2288 | } else { |
| 2037 | dp_set_hw_lane_settings(link, lt_settings, DPRX); | 2289 | dp_set_hw_lane_settings(link, link_res, lt_settings, DPRX); |
| 2038 | dpcd_set_lane_settings(link, lt_settings, DPRX); | 2290 | dpcd_set_lane_settings(link, lt_settings, DPRX); |
| 2039 | } | 2291 | } |
| 2040 | loop_count++; | 2292 | loop_count++; |
| @@ -2063,6 +2315,7 @@ static enum link_training_result dp_perform_128b_132b_channel_eq_done_sequence( · 22 unmodified lines | |||
| 2063 | 2315 | ||
| 2064 | static enum link_training_result dp_perform_128b_132b_cds_done_sequence( | 2316 | static enum link_training_result dp_perform_128b_132b_cds_done_sequence( |
| 2065 | struct dc_link *link, | 2317 | struct dc_link *link, |
| 2318 | const struct link_resource *link_res, | ||
| 2066 | struct link_training_settings *lt_settings) | 2319 | struct link_training_settings *lt_settings) |
| 2067 | { | 2320 | { |
| 2068 | /* Assumption: assume hardware has transmitted eq pattern */ | 2321 | /* Assumption: assume hardware has transmitted eq pattern */ |
| @@ -2099,6 +2352,7 @@ static enum link_training_result dp_perform_128b_132b_cds_done_sequence( · 30 unmodified lines | |||
| 2099 | 2352 | ||
| 2100 | static enum link_training_result dp_perform_8b_10b_link_training( | 2353 | static enum link_training_result dp_perform_8b_10b_link_training( |
| 2101 | struct dc_link *link, | 2354 | struct dc_link *link, |
| 2355 | const struct link_resource *link_res, | ||
| 2102 | struct link_training_settings *lt_settings) | 2356 | struct link_training_settings *lt_settings) |
| 2103 | { | 2357 | { |
| 2104 | enum link_training_result status = LINK_TRAINING_SUCCESS; | 2358 | enum link_training_result status = LINK_TRAINING_SUCCESS; |
| @@ -2108,7 +2362,7 @@ static enum link_training_result dp_perform_8b_10b_link_training( · 3 unmodified lines | |||
| 2108 | uint8_t lane = 0; | 2362 | uint8_t lane = 0; |
| 2109 | 2363 | ||
| 2110 | if (link->ctx->dc->work_arounds.lt_early_cr_pattern) | 2364 | if (link->ctx->dc->work_arounds.lt_early_cr_pattern) |
| 2111 | start_clock_recovery_pattern_early(link, lt_settings, DPRX); | 2365 | start_clock_recovery_pattern_early(link, link_res, lt_settings, DPRX); |
| 2112 | 2366 | ||
| 2113 | /* 1. set link rate, lane count and spread. */ | 2367 | /* 1. set link rate, lane count and spread. */ |
| 2114 | dpcd_set_link_settings(link, lt_settings); | 2368 | dpcd_set_link_settings(link, lt_settings); |
| @@ -2122,12 +2376,13 @@ static enum link_training_result dp_perform_8b_10b_link_training( · 7 unmodified lines | |||
| 2122 | 2376 | ||
| 2123 | for (repeater_id = repeater_cnt; (repeater_id > 0 && status == LINK_TRAINING_SUCCESS); | 2377 | for (repeater_id = repeater_cnt; (repeater_id > 0 && status == LINK_TRAINING_SUCCESS); |
| 2124 | repeater_id--) { | 2378 | repeater_id--) { |
| 2125 | status = perform_clock_recovery_sequence(link, lt_settings, repeater_id); | 2379 | status = perform_clock_recovery_sequence(link, link_res, lt_settings, repeater_id); |
| 2126 | 2380 | ||
| 2127 | if (status != LINK_TRAINING_SUCCESS) | 2381 | if (status != LINK_TRAINING_SUCCESS) |
| 2128 | break; | 2382 | break; |
| 2129 | 2383 | ||
| 2130 | status = perform_channel_equalization_sequence(link, | 2384 | status = perform_channel_equalization_sequence(link, |
| 2385 | link_res, | ||
| 2131 | lt_settings, | 2386 | lt_settings, |
| 2132 | repeater_id); | 2387 | repeater_id); |
| 2133 | 2388 | ||
| @@ -2142,9 +2397,10 @@ static enum link_training_result dp_perform_8b_10b_link_training( · 8 unmodified lines | |||
| 2142 | } | 2397 | } |
| 2143 | 2398 | ||
| 2144 | if (status == LINK_TRAINING_SUCCESS) { | 2399 | if (status == LINK_TRAINING_SUCCESS) { |
| 2145 | status = perform_clock_recovery_sequence(link, lt_settings, DPRX); | 2400 | status = perform_clock_recovery_sequence(link, link_res, lt_settings, DPRX); |
| 2146 | if (status == LINK_TRAINING_SUCCESS) { | 2401 | if (status == LINK_TRAINING_SUCCESS) { |
| 2147 | status = perform_channel_equalization_sequence(link, | 2402 | status = perform_channel_equalization_sequence(link, |
| 2403 | link_res, | ||
| 2148 | lt_settings, | 2404 | lt_settings, |
| 2149 | DPRX); | 2405 | DPRX); |
| 2150 | } | 2406 | } |
| @@ -2156,6 +2412,7 @@ static enum link_training_result dp_perform_8b_10b_link_training( · 5 unmodified lines | |||
| 2156 | #if defined(CONFIG_DRM_AMD_DC_DCN) | 2412 | #if defined(CONFIG_DRM_AMD_DC_DCN) |
| 2157 | static enum link_training_result dp_perform_128b_132b_link_training( | 2413 | static enum link_training_result dp_perform_128b_132b_link_training( |
| 2158 | struct dc_link *link, | 2414 | struct dc_link *link, |
| 2415 | const struct link_resource *link_res, | ||
| 2159 | struct link_training_settings *lt_settings) | 2416 | struct link_training_settings *lt_settings) |
| 2160 | { | 2417 | { |
| 2161 | enum link_training_result result = LINK_TRAINING_SUCCESS; | 2418 | enum link_training_result result = LINK_TRAINING_SUCCESS; |
1147 more lines aren’t shown here.
This diff is large: its 214 lines aren’t shown here.
This diff is large: its 117 lines aren’t shown here.
This diff is large: its 203 lines aren’t shown here.
This diff is large: its 246 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 116 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 149 lines aren’t shown here.
This diff is large: its 224 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 101 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 120 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 103 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 184 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 133 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 603 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 1890 lines aren’t shown here.
This diff is large: its 285 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 144 lines aren’t shown here.
This diff is large: its 124 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 120 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 85 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 205 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 164 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 208 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 90 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 153 lines aren’t shown here.
This diff is large: its 198 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 697 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 119 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 505 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 178 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 204 lines aren’t shown here.
This diff is large: its 666 lines aren’t shown here.
This diff is large: its 433 lines aren’t shown here.
This diff is large: its 541 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 247 lines aren’t shown here.
This diff is large: its 375 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 631 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 211 lines aren’t shown here.
This diff is large: its 364 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 468 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 101 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 111 lines aren’t shown here.
This diff is large: its 313 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 156 lines aren’t shown here.
This diff is large: its 387 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 123 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 253 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 199 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 381 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 352 lines aren’t shown here.
This diff is large: its 341 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 366 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 248 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 172 lines aren’t shown here.
This diff is large: its 590 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 271 lines aren’t shown here.
This diff is large: its 2927 lines aren’t shown here.
This diff is large: its 120 lines aren’t shown here.
This diff is large: its 161 lines aren’t shown here.
This diff is large: its 203 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 588 lines aren’t shown here.
This diff is large: its 259 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 345 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 83 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 1027 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 2505 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 180 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 246 lines aren’t shown here.
This diff is large: its 126 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 502 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 649 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 249 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 595 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 254 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 831 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 110 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 316 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 186 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 331 lines aren’t shown here.
This diff is large: its 225 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 1079 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 628 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 150 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 297 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 207 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 238 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 582 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 133 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 120 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 204 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 688 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 85 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 214 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 543 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 264 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 196 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 164 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 238 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 360 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 682 lines aren’t shown here.
This diff is large: its 131 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 468 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 323 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 365 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 911 lines aren’t shown here.
This diff is large: its 271 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 132 lines aren’t shown here.
This diff is large: its 83 lines aren’t shown here.
This diff is large: its 155 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 126 lines aren’t shown here.
This diff is large: its 529 lines aren’t shown here.
This diff is large: its 726 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 135 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 122 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 203 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 304 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 146 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 90 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 518 lines aren’t shown here.
This diff is large: its 251 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 280 lines aren’t shown here.
This diff is large: its 326 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 206 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 480 lines aren’t shown here.
This diff is large: its 285 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 448 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 294 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 181 lines aren’t shown here.
This diff is large: its 82 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 156 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 187 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 240 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 945 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 135 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 148 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 101 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 145 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 199 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 389 lines aren’t shown here.
This diff is large: its 266 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 133 lines aren’t shown here.
This diff is large: its 1374 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 128 lines aren’t shown here.
This diff is large: its 222 lines aren’t shown here.
This diff is large: its 218 lines aren’t shown here.
This diff is large: its 277 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 103 lines aren’t shown here.
This diff is large: its 144 lines aren’t shown here.
This diff is large: its 184 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 101 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 157 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 245 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 213 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 507 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 446 lines aren’t shown here.
This diff is large: its 234 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 332 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 324 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 703 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 128 lines aren’t shown here.
This diff is large: its 553 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 820 lines aren’t shown here.
This diff is large: its 173 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 173 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 164 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 306 lines aren’t shown here.
This diff is large: its 881 lines aren’t shown here.
This diff is large: its 110 lines aren’t shown here.
This diff is large: its 206 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 1074 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 346 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 232 lines aren’t shown here.
This diff is large: its 253 lines aren’t shown here.
This diff is large: its 481 lines aren’t shown here.
This diff is large: its 318 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 467 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 158 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 226 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 110 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 220 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 1357 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 111 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 242 lines aren’t shown here.
This diff is large: its 300 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 684 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 432 lines aren’t shown here.
This diff is large: its 131 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 822 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 258 lines aren’t shown here.
This diff is large: its 360 lines aren’t shown here.
This diff is large: its 169 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 119 lines aren’t shown here.
This diff is large: its 295 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 200 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 199 lines aren’t shown here.
This diff is large: its 82 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 464 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 185 lines aren’t shown here.
This diff is large: its 289 lines aren’t shown here.
This diff is large: its 116 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 111 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 133 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 300 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 239 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 231 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 218 lines aren’t shown here.
This diff is large: its 195 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 85 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 139 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
