Backmerge drm-misc-next to pick up some dependencies for drm/msm patches, in particular: https://patchwork.freedesktop.org/patch/570219/?series=127251&rev=1 https://patchwork.freedesktop.org/series/123411/
Unified · Split
| @@ -36,8 +36,9 @@ AIC100 DID (0xa100). · 35 unmodified lines | |||
| 36 | 36 | ||
| 37 | AIC100 does not implement FLR (function level reset). | 37 | AIC100 does not implement FLR (function level reset). |
| 38 | 38 | ||
| 39 | AIC100 implements MSI but does not implement MSI-X. AIC100 requires 17 MSIs to | 39 | AIC100 implements MSI but does not implement MSI-X. AIC100 prefers 17 MSIs to |
| 40 | operate (1 for MHI, 16 for the DMA Bridge). | 40 | operate (1 for MHI, 16 for the DMA Bridge). Falling back to 1 MSI is possible in |
| 41 | scenarios where reserving 32 MSIs isn't feasible. | ||
| 41 | 42 | ||
| 42 | As a PCIe device, AIC100 utilizes BARs to provide host interfaces to the device | 43 | As a PCIe device, AIC100 utilizes BARs to provide host interfaces to the device |
| 43 | hardware. AIC100 provides 3, 64-bit BARs. | 44 | hardware. AIC100 provides 3, 64-bit BARs. |
| @@ -220,10 +221,14 @@ of the defined channels, and their uses. · 176 unmodified lines | |||
| 220 | +----------------+---------+----------+----------------------------------------+ | 221 | +----------------+---------+----------+----------------------------------------+ |
| 221 | | QAIC_DEBUG | 18 & 19 | AMSS | Not used. | | 222 | | QAIC_DEBUG | 18 & 19 | AMSS | Not used. | |
| 222 | +----------------+---------+----------+----------------------------------------+ | 223 | +----------------+---------+----------+----------------------------------------+ |
| 223 | | QAIC_TIMESYNC | 20 & 21 | SBL/AMSS | Used to synchronize timestamps in the | | 224 | | QAIC_TIMESYNC | 20 & 21 | SBL | Used to synchronize timestamps in the | |
| 224 | | | | | device side logs with the host time | | 225 | | | | | device side logs with the host time | |
| 225 | | | | | source. | | 226 | | | | | source. | |
| 226 | +----------------+---------+----------+----------------------------------------+ | 227 | +----------------+---------+----------+----------------------------------------+ |
| 228 | | QAIC_TIMESYNC | 22 & 23 | AMSS | Used to periodically synchronize | | ||
| 229 | | _PERIODIC | | | timestamps in the device side logs with| | ||
| 230 | | | | | the host time source. | | ||
| 231 | +----------------+---------+----------+----------------------------------------+ | ||
| 227 | 232 | ||
| 228 | DMA Bridge | 233 | DMA Bridge |
| 229 | ========== | 234 | ========== |
| @@ -10,6 +10,9 @@ accelerator products. · 9 unmodified lines | |||
| 10 | Interrupts | 10 | Interrupts |
| 11 | ========== | 11 | ========== |
| 12 | 12 | ||
| 13 | IRQ Storm Mitigation | ||
| 14 | -------------------- | ||
| 15 | |||
| 13 | While the AIC100 DMA Bridge hardware implements an IRQ storm mitigation | 16 | While the AIC100 DMA Bridge hardware implements an IRQ storm mitigation |
| 14 | mechanism, it is still possible for an IRQ storm to occur. A storm can happen | 17 | mechanism, it is still possible for an IRQ storm to occur. A storm can happen |
| 15 | if the workload is particularly quick, and the host is responsive. If the host | 18 | if the workload is particularly quick, and the host is responsive. If the host |
| @@ -35,6 +38,26 @@ generates 100k IRQs per second (per /proc/interrupts) is reduced to roughly 64 · 19 unmodified lines | |||
| 35 | IRQs over 5 minutes while keeping the host system stable, and having the same | 38 | IRQs over 5 minutes while keeping the host system stable, and having the same |
| 36 | workload throughput performance (within run to run noise variation). | 39 | workload throughput performance (within run to run noise variation). |
| 37 | 40 | ||
| 41 | Single MSI Mode | ||
| 42 | --------------- | ||
| 43 | |||
| 44 | MultiMSI is not well supported on all systems; virtualized ones even less so | ||
| 45 | (circa 2023). Between hypervisors masking the PCIe MSI capability structure to | ||
| 46 | large memory requirements for vIOMMUs (required for supporting MultiMSI), it is | ||
| 47 | useful to be able to fall back to a single MSI when needed. | ||
| 48 | |||
| 49 | To support this fallback, we allow the case where only one MSI is able to be | ||
| 50 | allocated, and share that one MSI between MHI and the DBCs. The device detects | ||
| 51 | when only one MSI has been configured and directs the interrupts for the DBCs | ||
| 52 | to the interrupt normally used for MHI. Unfortunately this means that the | ||
| 53 | interrupt handlers for every DBC and MHI wake up for every interrupt that | ||
| 54 | arrives; however, the DBC threaded irq handlers only are started when work to be | ||
| 55 | done is detected (MHI will always start its threaded handler). | ||
| 56 | |||
| 57 | If the DBC is configured to force MSI interrupts, this can circumvent the | ||
| 58 | software IRQ storm mitigation mentioned above. Since the MSI is shared it is | ||
| 59 | never disabled, allowing each new entry to the FIFO to trigger a new interrupt. | ||
| 60 | |||
| 38 | 61 | ||
| 39 | Neural Network Control (NNC) Protocol | 62 | Neural Network Control (NNC) Protocol |
| 40 | ===================================== | 63 | ===================================== |
| @@ -70,8 +93,15 @@ commands (does not impact QAIC). · 29 unmodified lines | |||
| 70 | uAPI | 93 | uAPI |
| 71 | ==== | 94 | ==== |
| 72 | 95 | ||
| 96 | QAIC creates an accel device per phsyical PCIe device. This accel device exists | ||
| 97 | for as long as the PCIe device is known to Linux. | ||
| 98 | |||
| 99 | The PCIe device may not be in the state to accept requests from userspace at | ||
| 100 | all times. QAIC will trigger KOBJ_ONLINE/OFFLINE uevents to advertise when the | ||
| 101 | device can accept requests (ONLINE) and when the device is no longer accepting | ||
| 102 | requests (OFFLINE) because of a reset or other state transition. | ||
| 103 | |||
| 73 | QAIC defines a number of driver specific IOCTLs as part of the userspace API. | 104 | QAIC defines a number of driver specific IOCTLs as part of the userspace API. |
| 74 | This section describes those APIs. | ||
| 75 | 105 | ||
| 76 | DRM_IOCTL_QAIC_MANAGE | 106 | DRM_IOCTL_QAIC_MANAGE |
| 77 | This IOCTL allows userspace to send a NNC request to the QSM. The call will | 107 | This IOCTL allows userspace to send a NNC request to the QSM. The call will |
| @@ -178,3 +208,8 @@ overrides this for that call. Default is 5000 (5 seconds). · 100 unmodified lines | |||
| 178 | 208 | ||
| 179 | Sets the polling interval in microseconds (us) when datapath polling is active. | 209 | Sets the polling interval in microseconds (us) when datapath polling is active. |
| 180 | Takes effect at the next polling interval. Default is 100 (100 us). | 210 | Takes effect at the next polling interval. Default is 100 (100 us). |
| 211 | |||
| 212 | **timesync_delay_ms (unsigned int)** | ||
| 213 | |||
| 214 | Sets the time interval in milliseconds (ms) between two consecutive timesync | ||
| 215 | operations. Default is 1000 (1000 ms). | ||
| @@ -375,9 +375,9 @@ Developer web site of Loongson and LoongArch (Software and Documentation): · 374 unmodified lines | |||
| 375 | 375 | ||
| 376 | Documentation of LoongArch ISA: | 376 | Documentation of LoongArch ISA: |
| 377 | 377 | ||
| 378 | https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.02-CN.pdf (in Chinese) | 378 | https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.10-CN.pdf (in Chinese) |
| 379 | 379 | ||
| 380 | https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.02-EN.pdf (in English) | 380 | https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.10-EN.pdf (in English) |
| 381 | 381 | ||
| 382 | Documentation of LoongArch ELF psABI: | 382 | Documentation of LoongArch ELF psABI: |
| 383 | 383 | ||
| @@ -77,7 +77,7 @@ Protocol 2.14 BURNT BY INCORRECT COMMIT · 76 unmodified lines | |||
| 77 | Protocol 2.15 (Kernel 5.5) Added the kernel_info and kernel_info.setup_type_max. | 77 | Protocol 2.15 (Kernel 5.5) Added the kernel_info and kernel_info.setup_type_max. |
| 78 | ============= ============================================================ | 78 | ============= ============================================================ |
| 79 | 79 | ||
| 80 | .. note:: | 80 | .. note:: |
| 81 | The protocol version number should be changed only if the setup header | 81 | The protocol version number should be changed only if the setup header |
| 82 | is changed. There is no need to update the version number if boot_params | 82 | is changed. There is no need to update the version number if boot_params |
| 83 | or kernel_info are changed. Additionally, it is recommended to use | 83 | or kernel_info are changed. Additionally, it is recommended to use |
| @@ -153,6 +153,8 @@ NOTE: Some pages, such as DAX pages, cannot be pinned with longterm pins. That's · 152 unmodified lines | |||
| 153 | because DAX pages do not have a separate page cache, and so "pinning" implies | 153 | because DAX pages do not have a separate page cache, and so "pinning" implies |
| 154 | locking down file system blocks, which is not (yet) supported in that way. | 154 | locking down file system blocks, which is not (yet) supported in that way. |
| 155 | 155 | ||
| 156 | .. _mmu-notifier-registration-case: | ||
| 157 | |||
| 156 | CASE 3: MMU notifier registration, with or without page faulting hardware | 158 | CASE 3: MMU notifier registration, with or without page faulting hardware |
| 157 | ------------------------------------------------------------------------- | 159 | ------------------------------------------------------------------------- |
| 158 | Device drivers can pin pages via get_user_pages*(), and register for mmu | 160 | Device drivers can pin pages via get_user_pages*(), and register for mmu |
| @@ -55,6 +55,27 @@ properties: · 54 unmodified lines | |||
| 55 | - port@0 | 55 | - port@0 |
| 56 | - port@1 | 56 | - port@1 |
| 57 | 57 | ||
| 58 | vcchdmipll-supply: | ||
| 59 | description: A 1.8V supply that powers the HDMI PLL. | ||
| 60 | |||
| 61 | vcchdmitx-supply: | ||
| 62 | description: A 1.8V supply that powers the HDMI TX part. | ||
| 63 | |||
| 64 | vcclvdspll-supply: | ||
| 65 | description: A 1.8V supply that powers the LVDS PLL. | ||
| 66 | |||
| 67 | vcclvdstx-supply: | ||
| 68 | description: A 1.8V supply that powers the LVDS TX part. | ||
| 69 | |||
| 70 | vccmipirx-supply: | ||
| 71 | description: A 1.8V supply that powers the MIPI RX part. | ||
| 72 | |||
| 73 | vccsysclk-supply: | ||
| 74 | description: A 1.8V supply that powers the SYSCLK. | ||
| 75 | |||
| 76 | vdd-supply: | ||
| 77 | description: A 1.8V supply that powers the digital part. | ||
| 78 | |||
| 58 | required: | 79 | required: |
| 59 | - compatible | 80 | - compatible |
| 60 | - reg | 81 | - reg |
| @@ -23,6 +23,7 @@ properties: · 22 unmodified lines | |||
| 23 | items: | 23 | items: |
| 24 | - enum: | 24 | - enum: |
| 25 | - hannstar,hsd060bhw4 | 25 | - hannstar,hsd060bhw4 |
| 26 | - powkiddy,x55-panel | ||
| 26 | - const: himax,hx8394 | 27 | - const: himax,hx8394 |
| 27 | 28 | ||
| 28 | reg: true | 29 | reg: true |
| @@ -31,6 +32,8 @@ properties: · 2 unmodified lines | |||
| 31 | 32 | ||
| 32 | backlight: true | 33 | backlight: true |
| 33 | 34 | ||
| 35 | rotation: true | ||
| 36 | |||
| 34 | port: true | 37 | port: true |
| 35 | 38 | ||
| 36 | vcc-supply: | 39 | vcc-supply: |
| @@ -16,6 +16,7 @@ properties: · 15 unmodified lines | |||
| 16 | compatible: | 16 | compatible: |
| 17 | items: | 17 | items: |
| 18 | - enum: | 18 | - enum: |
| 19 | - ampire,am8001280g | ||
| 19 | - bananapi,lhr050h41 | 20 | - bananapi,lhr050h41 |
| 20 | - feixin,k101-im2byl02 | 21 | - feixin,k101-im2byl02 |
| 21 | - tdo,tl050hdv35 | 22 | - tdo,tl050hdv35 |
| @@ -21,7 +21,7 @@ properties: · 20 unmodified lines | |||
| 21 | - enum: | 21 | - enum: |
| 22 | - anbernic,rg351v-panel | 22 | - anbernic,rg351v-panel |
| 23 | - anbernic,rg353p-panel | 23 | - anbernic,rg353p-panel |
| 24 | - anbernic,rg353v-panel | 24 | - powkiddy,rk2023-panel |
| 25 | - const: newvision,nv3051d | 25 | - const: newvision,nv3051d |
| 26 | 26 | ||
| 27 | reg: true | 27 | reg: true |
| @@ -73,6 +73,8 @@ properties: · 72 unmodified lines | |||
| 73 | - auo,t215hvn01 | 73 | - auo,t215hvn01 |
| 74 | # Shanghai AVIC Optoelectronics 7" 1024x600 color TFT-LCD panel | 74 | # Shanghai AVIC Optoelectronics 7" 1024x600 color TFT-LCD panel |
| 75 | - avic,tm070ddh03 | 75 | - avic,tm070ddh03 |
| 76 | # BOE BP101WX1-100 10.1" WXGA (1280x800) LVDS panel | ||
| 77 | - boe,bp101wx1-100 | ||
| 76 | # BOE EV121WXM-N10-1850 12.1" WXGA (1280x800) TFT LCD panel | 78 | # BOE EV121WXM-N10-1850 12.1" WXGA (1280x800) TFT LCD panel |
| 77 | - boe,ev121wxm-n10-1850 | 79 | - boe,ev121wxm-n10-1850 |
| 78 | # BOE HV070WSA-100 7.01" WSVGA TFT LCD panel | 80 | # BOE HV070WSA-100 7.01" WSVGA TFT LCD panel |
| @@ -144,6 +146,8 @@ properties: · 65 unmodified lines | |||
| 144 | - edt,etmv570g2dhu | 146 | - edt,etmv570g2dhu |
| 145 | # E Ink VB3300-KCA | 147 | # E Ink VB3300-KCA |
| 146 | - eink,vb3300-kca | 148 | - eink,vb3300-kca |
| 149 | # Evervision Electronics Co. Ltd. VGG644804 5.7" VGA TFT LCD Panel | ||
| 150 | - evervision,vgg644804 | ||
| 147 | # Evervision Electronics Co. Ltd. VGG804821 5.0" WVGA TFT LCD Panel | 151 | # Evervision Electronics Co. Ltd. VGG804821 5.0" WVGA TFT LCD Panel |
| 148 | - evervision,vgg804821 | 152 | - evervision,vgg804821 |
| 149 | # Foxlink Group 5" WVGA TFT LCD panel | 153 | # Foxlink Group 5" WVGA TFT LCD panel |
| @@ -23,6 +23,7 @@ properties: · 22 unmodified lines | |||
| 23 | compatible: | 23 | compatible: |
| 24 | enum: | 24 | enum: |
| 25 | - ti,am625-dss | 25 | - ti,am625-dss |
| 26 | - ti,am62a7,dss | ||
| 26 | - ti,am65x-dss | 27 | - ti,am65x-dss |
| 27 | 28 | ||
| 28 | reg: | 29 | reg: |
| @@ -87,6 +88,7 @@ properties: · 58 unmodified lines | |||
| 87 | For AM65x DSS, the OLDI output port node from video port 1. | 88 | For AM65x DSS, the OLDI output port node from video port 1. |
| 88 | For AM625 DSS, the internal DPI output port node from video | 89 | For AM625 DSS, the internal DPI output port node from video |
| 89 | port 1. | 90 | port 1. |
| 91 | For AM62A7 DSS, the port is tied off inside the SoC. | ||
| 90 | 92 | ||
| 91 | port@1: | 93 | port@1: |
| 92 | $ref: /schemas/graph.yaml#/properties/port | 94 | $ref: /schemas/graph.yaml#/properties/port |
| @@ -108,6 +110,18 @@ properties: · 15 unmodified lines | |||
| 108 | Input memory (from main memory to dispc) bandwidth limit in | 110 | Input memory (from main memory to dispc) bandwidth limit in |
| 109 | bytes per second | 111 | bytes per second |
| 110 | 112 | ||
| 113 | allOf: | ||
| 114 | - if: | ||
| 115 | properties: | ||
| 116 | compatible: | ||
| 117 | contains: | ||
| 118 | const: ti,am62a7-dss | ||
| 119 | then: | ||
| 120 | properties: | ||
| 121 | ports: | ||
| 122 | properties: | ||
| 123 | port@0: false | ||
| 124 | |||
| 111 | required: | 125 | required: |
| 112 | - compatible | 126 | - compatible |
| 113 | - reg | 127 | - reg |
| @@ -29,6 +29,7 @@ properties: · 28 unmodified lines | |||
| 29 | - allwinner,sun50i-a64-mali | 29 | - allwinner,sun50i-a64-mali |
| 30 | - rockchip,rk3036-mali | 30 | - rockchip,rk3036-mali |
| 31 | - rockchip,rk3066-mali | 31 | - rockchip,rk3066-mali |
| 32 | - rockchip,rk3128-mali | ||
| 32 | - rockchip,rk3188-mali | 33 | - rockchip,rk3188-mali |
| 33 | - rockchip,rk3228-mali | 34 | - rockchip,rk3228-mali |
| 34 | - samsung,exynos4210-mali | 35 | - samsung,exynos4210-mali |
| @@ -17,6 +17,7 @@ properties: · 16 unmodified lines | |||
| 17 | compatible: | 17 | compatible: |
| 18 | enum: | 18 | enum: |
| 19 | - brcm,2711-v3d | 19 | - brcm,2711-v3d |
| 20 | - brcm,2712-v3d | ||
| 20 | - brcm,7268-v3d | 21 | - brcm,7268-v3d |
| 21 | - brcm,7278-v3d | 22 | - brcm,7278-v3d |
| 22 | 23 | ||
| @@ -0,0 +1,73 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | # Copyright (c) 2023 Imagination Technologies Ltd. | ||
| 3 | %YAML 1.2 | ||
| 4 | --- | ||
| 5 | $id: http://devicetree.org/schemas/gpu/img,powervr.yaml# | ||
| 6 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 7 | |||
| 8 | title: Imagination Technologies PowerVR and IMG GPU | ||
| 9 | |||
| 10 | maintainers: | ||
| 11 | - Frank Binns <frank.binns@imgtec.com> | ||
| 12 | |||
| 13 | properties: | ||
| 14 | compatible: | ||
| 15 | items: | ||
| 16 | - enum: | ||
| 17 | - ti,am62-gpu | ||
| 18 | - const: img,img-axe # IMG AXE GPU model/revision is fully discoverable | ||
| 19 | |||
| 20 | reg: | ||
| 21 | maxItems: 1 | ||
| 22 | |||
| 23 | clocks: | ||
| 24 | minItems: 1 | ||
| 25 | maxItems: 3 | ||
| 26 | |||
| 27 | clock-names: | ||
| 28 | items: | ||
| 29 | - const: core | ||
| 30 | - const: mem | ||
| 31 | - const: sys | ||
| 32 | minItems: 1 | ||
| 33 | |||
| 34 | interrupts: | ||
| 35 | maxItems: 1 | ||
| 36 | |||
| 37 | power-domains: | ||
| 38 | maxItems: 1 | ||
| 39 | |||
| 40 | required: | ||
| 41 | - compatible | ||
| 42 | - reg | ||
| 43 | - clocks | ||
| 44 | - clock-names | ||
| 45 | - interrupts | ||
| 46 | |||
| 47 | additionalProperties: false | ||
| 48 | |||
| 49 | allOf: | ||
| 50 | - if: | ||
| 51 | properties: | ||
| 52 | compatible: | ||
| 53 | contains: | ||
| 54 | const: ti,am62-gpu | ||
| 55 | then: | ||
| 56 | properties: | ||
| 57 | clocks: | ||
| 58 | maxItems: 1 | ||
| 59 | |||
| 60 | examples: | ||
| 61 | - | | ||
| 62 | #include <dt-bindings/interrupt-controller/irq.h> | ||
| 63 | #include <dt-bindings/interrupt-controller/arm-gic.h> | ||
| 64 | #include <dt-bindings/soc/ti,sci_pm_domain.h> | ||
| 65 | |||
| 66 | gpu@fd00000 { | ||
| 67 | compatible = "ti,am62-gpu", "img,img-axe"; | ||
| 68 | reg = <0x0fd00000 0x20000>; | ||
| 69 | clocks = <&k3_clks 187 0>; | ||
| 70 | clock-names = "core"; | ||
| 71 | interrupts = <GIC_SPI 86 IRQ_TYPE_LEVEL_HIGH>; | ||
| 72 | power-domains = <&k3_pds 187 TI_SCI_PD_EXCLUSIVE>; | ||
| 73 | }; | ||
| @@ -275,12 +275,12 @@ allOf: · 274 unmodified lines | |||
| 275 | properties: | 275 | properties: |
| 276 | rx-internal-delay-ps: | 276 | rx-internal-delay-ps: |
| 277 | description: | 277 | description: |
| 278 | RGMII Receive Clock Delay defined in pico seconds.This is used for | 278 | RGMII Receive Clock Delay defined in pico seconds. This is used for |
| 279 | controllers that have configurable RX internal delays. If this | 279 | controllers that have configurable RX internal delays. If this |
| 280 | property is present then the MAC applies the RX delay. | 280 | property is present then the MAC applies the RX delay. |
| 281 | tx-internal-delay-ps: | 281 | tx-internal-delay-ps: |
| 282 | description: | 282 | description: |
| 283 | RGMII Transmit Clock Delay defined in pico seconds.This is used for | 283 | RGMII Transmit Clock Delay defined in pico seconds. This is used for |
| 284 | controllers that have configurable TX internal delays. If this | 284 | controllers that have configurable TX internal delays. If this |
| 285 | property is present then the MAC applies the TX delay. | 285 | property is present then the MAC applies the TX delay. |
| 286 | 286 | ||
| @@ -36,6 +36,7 @@ properties: · 35 unmodified lines | |||
| 36 | - qcom,sm8350-ufshc | 36 | - qcom,sm8350-ufshc |
| 37 | - qcom,sm8450-ufshc | 37 | - qcom,sm8450-ufshc |
| 38 | - qcom,sm8550-ufshc | 38 | - qcom,sm8550-ufshc |
| 39 | - qcom,sm8650-ufshc | ||
| 39 | - const: qcom,ufshc | 40 | - const: qcom,ufshc |
| 40 | - const: jedec,ufs-2.0 | 41 | - const: jedec,ufs-2.0 |
| 41 | 42 | ||
| @@ -122,6 +123,7 @@ allOf: · 80 unmodified lines | |||
| 122 | - qcom,sm8350-ufshc | 123 | - qcom,sm8350-ufshc |
| 123 | - qcom,sm8450-ufshc | 124 | - qcom,sm8450-ufshc |
| 124 | - qcom,sm8550-ufshc | 125 | - qcom,sm8550-ufshc |
| 126 | - qcom,sm8650-ufshc | ||
| 125 | then: | 127 | then: |
| 126 | properties: | 128 | properties: |
| 127 | clocks: | 129 | clocks: |
| @@ -36,7 +36,11 @@ properties: · 35 unmodified lines | |||
| 36 | 36 | ||
| 37 | vdd-supply: | 37 | vdd-supply: |
| 38 | description: | 38 | description: |
| 39 | VDD power supply to the hub | 39 | 3V3 power supply to the hub |
| 40 | |||
| 41 | vdd2-supply: | ||
| 42 | description: | ||
| 43 | 1V2 power supply to the hub | ||
| 40 | 44 | ||
| 41 | peer-hub: | 45 | peer-hub: |
| 42 | $ref: /schemas/types.yaml#/definitions/phandle | 46 | $ref: /schemas/types.yaml#/definitions/phandle |
| @@ -62,6 +66,7 @@ allOf: · 19 unmodified lines | |||
| 62 | properties: | 66 | properties: |
| 63 | reset-gpios: false | 67 | reset-gpios: false |
| 64 | vdd-supply: false | 68 | vdd-supply: false |
| 69 | vdd2-supply: false | ||
| 65 | peer-hub: false | 70 | peer-hub: false |
| 66 | i2c-bus: false | 71 | i2c-bus: false |
| 67 | else: | 72 | else: |
| @@ -521,8 +521,8 @@ examples: · 520 unmodified lines | |||
| 521 | 521 | ||
| 522 | interrupts = <GIC_SPI 131 IRQ_TYPE_LEVEL_HIGH>, | 522 | interrupts = <GIC_SPI 131 IRQ_TYPE_LEVEL_HIGH>, |
| 523 | <GIC_SPI 486 IRQ_TYPE_LEVEL_HIGH>, | 523 | <GIC_SPI 486 IRQ_TYPE_LEVEL_HIGH>, |
| 524 | <GIC_SPI 488 IRQ_TYPE_LEVEL_HIGH>, | 524 | <GIC_SPI 488 IRQ_TYPE_EDGE_BOTH>, |
| 525 | <GIC_SPI 489 IRQ_TYPE_LEVEL_HIGH>; | 525 | <GIC_SPI 489 IRQ_TYPE_EDGE_BOTH>; |
| 526 | interrupt-names = "hs_phy_irq", "ss_phy_irq", | 526 | interrupt-names = "hs_phy_irq", "ss_phy_irq", |
| 527 | "dm_hs_phy_irq", "dp_hs_phy_irq"; | 527 | "dm_hs_phy_irq", "dp_hs_phy_irq"; |
| 528 | 528 | ||
| @@ -41,7 +41,7 @@ examples: · 40 unmodified lines | |||
| 41 | - | | 41 | - | |
| 42 | usb { | 42 | usb { |
| 43 | phys = <&usb2_phy1>, <&usb3_phy1>; | 43 | phys = <&usb2_phy1>, <&usb3_phy1>; |
| 44 | phy-names = "usb"; | 44 | phy-names = "usb2", "usb3"; |
| 45 | #address-cells = <1>; | 45 | #address-cells = <1>; |
| 46 | #size-cells = <0>; | 46 | #size-cells = <0>; |
| 47 | 47 | ||
| @@ -91,6 +91,10 @@ compatibility checking tool (fsck.erofs), and a debugging tool (dump.erofs): · 90 unmodified lines | |||
| 91 | 91 | ||
| 92 | - git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git | 92 | - git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git |
| 93 | 93 | ||
| 94 | For more information, please also refer to the documentation site: | ||
| 95 | |||
| 96 | - https://erofs.docs.kernel.org | ||
| 97 | |||
| 94 | Bugs and patches are welcome, please kindly help us and send to the following | 98 | Bugs and patches are welcome, please kindly help us and send to the following |
| 95 | linux-erofs mailing list: | 99 | linux-erofs mailing list: |
| 96 | 100 | ||
| @@ -3,9 +3,11 @@ GPU Driver Documentation · 2 unmodified lines | |||
| 3 | ======================== | 3 | ======================== |
| 4 | 4 | ||
| 5 | .. toctree:: | 5 | .. toctree:: |
| 6 | :maxdepth: 3 | ||
| 6 | 7 | ||
| 7 | amdgpu/index | 8 | amdgpu/index |
| 8 | i915 | 9 | i915 |
| 10 | imagination/index | ||
| 9 | mcde | 11 | mcde |
| 10 | meson | 12 | meson |
| 11 | pl111 | 13 | pl111 |
| @@ -363,6 +363,12 @@ EDID Helper Functions Reference · 362 unmodified lines | |||
| 363 | .. kernel-doc:: drivers/gpu/drm/drm_edid.c | 363 | .. kernel-doc:: drivers/gpu/drm/drm_edid.c |
| 364 | :export: | 364 | :export: |
| 365 | 365 | ||
| 366 | .. kernel-doc:: include/drm/drm_eld.h | ||
| 367 | :internal: | ||
| 368 | |||
| 369 | .. kernel-doc:: drivers/gpu/drm/drm_eld.c | ||
| 370 | :export: | ||
| 371 | |||
| 366 | SCDC Helper Functions Reference | 372 | SCDC Helper Functions Reference |
| 367 | =============================== | 373 | =============================== |
| 368 | 374 | ||
| @@ -548,6 +548,8 @@ Plane Composition Properties · 547 unmodified lines | |||
| 548 | .. kernel-doc:: drivers/gpu/drm/drm_blend.c | 548 | .. kernel-doc:: drivers/gpu/drm/drm_blend.c |
| 549 | :doc: overview | 549 | :doc: overview |
| 550 | 550 | ||
| 551 | .. _damage_tracking_properties: | ||
| 552 | |||
| 551 | Damage Tracking Properties | 553 | Damage Tracking Properties |
| 552 | -------------------------- | 554 | -------------------------- |
| 553 | 555 | ||
| @@ -579,6 +581,12 @@ Variable Refresh Properties · 25 unmodified lines | |||
| 579 | .. kernel-doc:: drivers/gpu/drm/drm_connector.c | 581 | .. kernel-doc:: drivers/gpu/drm/drm_connector.c |
| 580 | :doc: Variable refresh properties | 582 | :doc: Variable refresh properties |
| 581 | 583 | ||
| 584 | Cursor Hotspot Properties | ||
| 585 | --------------------------- | ||
| 586 | |||
| 587 | .. kernel-doc:: drivers/gpu/drm/drm_plane.c | ||
| 588 | :doc: hotspot properties | ||
| 589 | |||
| 582 | Existing KMS Properties | 590 | Existing KMS Properties |
| 583 | ----------------------- | 591 | ----------------------- |
| 584 | 592 | ||
| @@ -466,6 +466,8 @@ DRM MM Range Allocator Function References · 465 unmodified lines | |||
| 466 | .. kernel-doc:: drivers/gpu/drm/drm_mm.c | 466 | .. kernel-doc:: drivers/gpu/drm/drm_mm.c |
| 467 | :export: | 467 | :export: |
| 468 | 468 | ||
| 469 | .. _drm_gpuvm: | ||
| 470 | |||
| 469 | DRM GPUVM | 471 | DRM GPUVM |
| 470 | ========= | 472 | ========= |
| 471 | 473 | ||
| @@ -481,6 +483,8 @@ Split and Merge · 9 unmodified lines | |||
| 481 | .. kernel-doc:: drivers/gpu/drm/drm_gpuvm.c | 483 | .. kernel-doc:: drivers/gpu/drm/drm_gpuvm.c |
| 482 | :doc: Split and Merge | 484 | :doc: Split and Merge |
| 483 | 485 | ||
| 486 | .. _drm_gpuvm_locking: | ||
| 487 | |||
| 484 | Locking | 488 | Locking |
| 485 | ------- | 489 | ------- |
| 486 | 490 | ||
| @@ -552,6 +556,12 @@ Overview · 65 unmodified lines | |||
| 552 | .. kernel-doc:: drivers/gpu/drm/scheduler/sched_main.c | 556 | .. kernel-doc:: drivers/gpu/drm/scheduler/sched_main.c |
| 553 | :doc: Overview | 557 | :doc: Overview |
| 554 | 558 | ||
| 559 | Flow Control | ||
| 560 | ------------ | ||
| 561 | |||
| 562 | .. kernel-doc:: drivers/gpu/drm/scheduler/sched_main.c | ||
| 563 | :doc: Flow Control | ||
| 564 | |||
| 555 | Scheduler Function References | 565 | Scheduler Function References |
| 556 | ----------------------------- | 566 | ----------------------------- |
| 557 | 567 | ||
| @@ -0,0 +1,582 @@ | |||
| 1 | .. SPDX-License-Identifier: (GPL-2.0+ OR MIT) | ||
| 2 | |||
| 3 | =============== | ||
| 4 | VM_BIND locking | ||
| 5 | =============== | ||
| 6 | |||
| 7 | This document attempts to describe what's needed to get VM_BIND locking right, | ||
| 8 | including the userptr mmu_notifier locking. It also discusses some | ||
| 9 | optimizations to get rid of the looping through of all userptr mappings and | ||
| 10 | external / shared object mappings that is needed in the simplest | ||
| 11 | implementation. In addition, there is a section describing the VM_BIND locking | ||
| 12 | required for implementing recoverable pagefaults. | ||
| 13 | |||
| 14 | The DRM GPUVM set of helpers | ||
| 15 | ============================ | ||
| 16 | |||
| 17 | There is a set of helpers for drivers implementing VM_BIND, and this | ||
| 18 | set of helpers implements much, but not all of the locking described | ||
| 19 | in this document. In particular, it is currently lacking a userptr | ||
| 20 | implementation. This document does not intend to describe the DRM GPUVM | ||
| 21 | implementation in detail, but it is covered in :ref:`its own | ||
| 22 | documentation <drm_gpuvm>`. It is highly recommended for any driver | ||
| 23 | implementing VM_BIND to use the DRM GPUVM helpers and to extend it if | ||
| 24 | common functionality is missing. | ||
| 25 | |||
| 26 | Nomenclature | ||
| 27 | ============ | ||
| 28 | |||
| 29 | * ``gpu_vm``: Abstraction of a virtual GPU address space with | ||
| 30 | meta-data. Typically one per client (DRM file-private), or one per | ||
| 31 | execution context. | ||
| 32 | * ``gpu_vma``: Abstraction of a GPU address range within a gpu_vm with | ||
| 33 | associated meta-data. The backing storage of a gpu_vma can either be | ||
| 34 | a GEM object or anonymous or page-cache pages mapped also into the CPU | ||
| 35 | address space for the process. | ||
| 36 | * ``gpu_vm_bo``: Abstracts the association of a GEM object and | ||
| 37 | a VM. The GEM object maintains a list of gpu_vm_bos, where each gpu_vm_bo | ||
| 38 | maintains a list of gpu_vmas. | ||
| 39 | * ``userptr gpu_vma or just userptr``: A gpu_vma, whose backing store | ||
| 40 | is anonymous or page-cache pages as described above. | ||
| 41 | * ``revalidating``: Revalidating a gpu_vma means making the latest version | ||
| 42 | of the backing store resident and making sure the gpu_vma's | ||
| 43 | page-table entries point to that backing store. | ||
| 44 | * ``dma_fence``: A struct dma_fence that is similar to a struct completion | ||
| 45 | and which tracks GPU activity. When the GPU activity is finished, | ||
| 46 | the dma_fence signals. Please refer to the ``DMA Fences`` section of | ||
| 47 | the :doc:`dma-buf doc </driver-api/dma-buf>`. | ||
| 48 | * ``dma_resv``: A struct dma_resv (a.k.a reservation object) that is used | ||
| 49 | to track GPU activity in the form of multiple dma_fences on a | ||
| 50 | gpu_vm or a GEM object. The dma_resv contains an array / list | ||
| 51 | of dma_fences and a lock that needs to be held when adding | ||
| 52 | additional dma_fences to the dma_resv. The lock is of a type that | ||
| 53 | allows deadlock-safe locking of multiple dma_resvs in arbitrary | ||
| 54 | order. Please refer to the ``Reservation Objects`` section of the | ||
| 55 | :doc:`dma-buf doc </driver-api/dma-buf>`. | ||
| 56 | * ``exec function``: An exec function is a function that revalidates all | ||
| 57 | affected gpu_vmas, submits a GPU command batch and registers the | ||
| 58 | dma_fence representing the GPU command's activity with all affected | ||
| 59 | dma_resvs. For completeness, although not covered by this document, | ||
| 60 | it's worth mentioning that an exec function may also be the | ||
| 61 | revalidation worker that is used by some drivers in compute / | ||
| 62 | long-running mode. | ||
| 63 | * ``local object``: A GEM object which is only mapped within a | ||
| 64 | single VM. Local GEM objects share the gpu_vm's dma_resv. | ||
| 65 | * ``external object``: a.k.a shared object: A GEM object which may be shared | ||
| 66 | by multiple gpu_vms and whose backing storage may be shared with | ||
| 67 | other drivers. | ||
| 68 | |||
| 69 | Locks and locking order | ||
| 70 | ======================= | ||
| 71 | |||
| 72 | One of the benefits of VM_BIND is that local GEM objects share the gpu_vm's | ||
| 73 | dma_resv object and hence the dma_resv lock. So, even with a huge | ||
| 74 | number of local GEM objects, only one lock is needed to make the exec | ||
| 75 | sequence atomic. | ||
| 76 | |||
| 77 | The following locks and locking orders are used: | ||
| 78 | |||
| 79 | * The ``gpu_vm->lock`` (optionally an rwsem). Protects the gpu_vm's | ||
| 80 | data structure keeping track of gpu_vmas. It can also protect the | ||
| 81 | gpu_vm's list of userptr gpu_vmas. With a CPU mm analogy this would | ||
| 82 | correspond to the mmap_lock. An rwsem allows several readers to walk | ||
| 83 | the VM tree concurrently, but the benefit of that concurrency most | ||
| 84 | likely varies from driver to driver. | ||
| 85 | * The ``userptr_seqlock``. This lock is taken in read mode for each | ||
| 86 | userptr gpu_vma on the gpu_vm's userptr list, and in write mode during mmu | ||
| 87 | notifier invalidation. This is not a real seqlock but described in | ||
| 88 | ``mm/mmu_notifier.c`` as a "Collision-retry read-side/write-side | ||
| 89 | 'lock' a lot like a seqcount. However this allows multiple | ||
| 90 | write-sides to hold it at once...". The read side critical section | ||
| 91 | is enclosed by ``mmu_interval_read_begin() / | ||
| 92 | mmu_interval_read_retry()`` with ``mmu_interval_read_begin()`` | ||
| 93 | sleeping if the write side is held. | ||
| 94 | The write side is held by the core mm while calling mmu interval | ||
| 95 | invalidation notifiers. | ||
| 96 | * The ``gpu_vm->resv`` lock. Protects the gpu_vm's list of gpu_vmas needing | ||
| 97 | rebinding, as well as the residency state of all the gpu_vm's local | ||
| 98 | GEM objects. | ||
| 99 | Furthermore, it typically protects the gpu_vm's list of evicted and | ||
| 100 | external GEM objects. | ||
| 101 | * The ``gpu_vm->userptr_notifier_lock``. This is an rwsem that is | ||
| 102 | taken in read mode during exec and write mode during a mmu notifier | ||
| 103 | invalidation. The userptr notifier lock is per gpu_vm. | ||
| 104 | * The ``gem_object->gpuva_lock`` This lock protects the GEM object's | ||
| 105 | list of gpu_vm_bos. This is usually the same lock as the GEM | ||
| 106 | object's dma_resv, but some drivers protects this list differently, | ||
| 107 | see below. | ||
| 108 | * The ``gpu_vm list spinlocks``. With some implementations they are needed | ||
| 109 | to be able to update the gpu_vm evicted- and external object | ||
| 110 | list. For those implementations, the spinlocks are grabbed when the | ||
| 111 | lists are manipulated. However, to avoid locking order violations | ||
| 112 | with the dma_resv locks, a special scheme is needed when iterating | ||
| 113 | over the lists. | ||
| 114 | |||
| 115 | .. _gpu_vma lifetime: | ||
| 116 | |||
| 117 | Protection and lifetime of gpu_vm_bos and gpu_vmas | ||
| 118 | ================================================== | ||
| 119 | |||
| 120 | The GEM object's list of gpu_vm_bos, and the gpu_vm_bo's list of gpu_vmas | ||
| 121 | is protected by the ``gem_object->gpuva_lock``, which is typically the | ||
| 122 | same as the GEM object's dma_resv, but if the driver | ||
| 123 | needs to access these lists from within a dma_fence signalling | ||
| 124 | critical section, it can instead choose to protect it with a | ||
| 125 | separate lock, which can be locked from within the dma_fence signalling | ||
| 126 | critical section. Such drivers then need to pay additional attention | ||
| 127 | to what locks need to be taken from within the loop when iterating | ||
| 128 | over the gpu_vm_bo and gpu_vma lists to avoid locking-order violations. | ||
| 129 | |||
| 130 | The DRM GPUVM set of helpers provide lockdep asserts that this lock is | ||
| 131 | held in relevant situations and also provides a means of making itself | ||
| 132 | aware of which lock is actually used: :c:func:`drm_gem_gpuva_set_lock`. | ||
| 133 | |||
| 134 | Each gpu_vm_bo holds a reference counted pointer to the underlying GEM | ||
| 135 | object, and each gpu_vma holds a reference counted pointer to the | ||
| 136 | gpu_vm_bo. When iterating over the GEM object's list of gpu_vm_bos and | ||
| 137 | over the gpu_vm_bo's list of gpu_vmas, the ``gem_object->gpuva_lock`` must | ||
| 138 | not be dropped, otherwise, gpu_vmas attached to a gpu_vm_bo may | ||
| 139 | disappear without notice since those are not reference-counted. A | ||
| 140 | driver may implement its own scheme to allow this at the expense of | ||
| 141 | additional complexity, but this is outside the scope of this document. | ||
| 142 | |||
| 143 | In the DRM GPUVM implementation, each gpu_vm_bo and each gpu_vma | ||
| 144 | holds a reference count on the gpu_vm itself. Due to this, and to avoid circular | ||
| 145 | reference counting, cleanup of the gpu_vm's gpu_vmas must not be done from the | ||
| 146 | gpu_vm's destructor. Drivers typically implements a gpu_vm close | ||
| 147 | function for this cleanup. The gpu_vm close function will abort gpu | ||
| 148 | execution using this VM, unmap all gpu_vmas and release page-table memory. | ||
| 149 | |||
| 150 | Revalidation and eviction of local objects | ||
| 151 | ========================================== | ||
| 152 | |||
| 153 | Note that in all the code examples given below we use simplified | ||
| 154 | pseudo-code. In particular, the dma_resv deadlock avoidance algorithm | ||
| 155 | as well as reserving memory for dma_resv fences is left out. | ||
| 156 | |||
| 157 | Revalidation | ||
| 158 | ____________ | ||
| 159 | With VM_BIND, all local objects need to be resident when the gpu is | ||
| 160 | executing using the gpu_vm, and the objects need to have valid | ||
| 161 | gpu_vmas set up pointing to them. Typically, each gpu command buffer | ||
| 162 | submission is therefore preceded with a re-validation section: | ||
| 163 | |||
| 164 | .. code-block:: C | ||
| 165 | |||
| 166 | dma_resv_lock(gpu_vm->resv); | ||
| 167 | |||
| 168 | // Validation section starts here. | ||
| 169 | for_each_gpu_vm_bo_on_evict_list(&gpu_vm->evict_list, &gpu_vm_bo) { | ||
| 170 | validate_gem_bo(&gpu_vm_bo->gem_bo); | ||
| 171 | |||
| 172 | // The following list iteration needs the Gem object's | ||
| 173 | // dma_resv to be held (it protects the gpu_vm_bo's list of | ||
| 174 | // gpu_vmas, but since local gem objects share the gpu_vm's | ||
| 175 | // dma_resv, it is already held at this point. | ||
| 176 | for_each_gpu_vma_of_gpu_vm_bo(&gpu_vm_bo, &gpu_vma) | ||
| 177 | move_gpu_vma_to_rebind_list(&gpu_vma, &gpu_vm->rebind_list); | ||
| 178 | } | ||
| 179 | |||
| 180 | for_each_gpu_vma_on_rebind_list(&gpu vm->rebind_list, &gpu_vma) { | ||
| 181 | rebind_gpu_vma(&gpu_vma); | ||
| 182 | remove_gpu_vma_from_rebind_list(&gpu_vma); | ||
| 183 | } | ||
| 184 | // Validation section ends here, and job submission starts. | ||
| 185 | |||
| 186 | add_dependencies(&gpu_job, &gpu_vm->resv); | ||
| 187 | job_dma_fence = gpu_submit(&gpu_job)); | ||
| 188 | |||
| 189 | add_dma_fence(job_dma_fence, &gpu_vm->resv); | ||
| 190 | dma_resv_unlock(gpu_vm->resv); | ||
| 191 | |||
| 192 | The reason for having a separate gpu_vm rebind list is that there | ||
| 193 | might be userptr gpu_vmas that are not mapping a buffer object that | ||
| 194 | also need rebinding. | ||
| 195 | |||
| 196 | Eviction | ||
| 197 | ________ | ||
| 198 | |||
| 199 | Eviction of one of these local objects will then look similar to the | ||
| 200 | following: | ||
| 201 | |||
| 202 | .. code-block:: C | ||
| 203 | |||
| 204 | obj = get_object_from_lru(); | ||
| 205 | |||
| 206 | dma_resv_lock(obj->resv); | ||
| 207 | for_each_gpu_vm_bo_of_obj(obj, &gpu_vm_bo); | ||
| 208 | add_gpu_vm_bo_to_evict_list(&gpu_vm_bo, &gpu_vm->evict_list); | ||
| 209 | |||
| 210 | add_dependencies(&eviction_job, &obj->resv); | ||
| 211 | job_dma_fence = gpu_submit(&eviction_job); | ||
| 212 | add_dma_fence(&obj->resv, job_dma_fence); | ||
| 213 | |||
| 214 | dma_resv_unlock(&obj->resv); | ||
| 215 | put_object(obj); | ||
| 216 | |||
| 217 | Note that since the object is local to the gpu_vm, it will share the gpu_vm's | ||
| 218 | dma_resv lock such that ``obj->resv == gpu_vm->resv``. | ||
| 219 | The gpu_vm_bos marked for eviction are put on the gpu_vm's evict list, | ||
| 220 | which is protected by ``gpu_vm->resv``. During eviction all local | ||
| 221 | objects have their dma_resv locked and, due to the above equality, also | ||
| 222 | the gpu_vm's dma_resv protecting the gpu_vm's evict list is locked. | ||
| 223 | |||
| 224 | With VM_BIND, gpu_vmas don't need to be unbound before eviction, | ||
| 225 | since the driver must ensure that the eviction blit or copy will wait | ||
| 226 | for GPU idle or depend on all previous GPU activity. Furthermore, any | ||
| 227 | subsequent attempt by the GPU to access freed memory through the | ||
| 228 | gpu_vma will be preceded by a new exec function, with a revalidation | ||
| 229 | section which will make sure all gpu_vmas are rebound. The eviction | ||
| 230 | code holding the object's dma_resv while revalidating will ensure a | ||
| 231 | new exec function may not race with the eviction. | ||
| 232 | |||
| 233 | A driver can be implemented in such a way that, on each exec function, | ||
| 234 | only a subset of vmas are selected for rebind. In this case, all vmas that are | ||
| 235 | *not* selected for rebind must be unbound before the exec | ||
| 236 | function workload is submitted. | ||
| 237 | |||
| 238 | Locking with external buffer objects | ||
| 239 | ==================================== | ||
| 240 | |||
| 241 | Since external buffer objects may be shared by multiple gpu_vm's they | ||
| 242 | can't share their reservation object with a single gpu_vm. Instead | ||
| 243 | they need to have a reservation object of their own. The external | ||
| 244 | objects bound to a gpu_vm using one or many gpu_vmas are therefore put on a | ||
| 245 | per-gpu_vm list which is protected by the gpu_vm's dma_resv lock or | ||
| 246 | one of the :ref:`gpu_vm list spinlocks <Spinlock iteration>`. Once | ||
| 247 | the gpu_vm's reservation object is locked, it is safe to traverse the | ||
| 248 | external object list and lock the dma_resvs of all external | ||
| 249 | objects. However, if instead a list spinlock is used, a more elaborate | ||
| 250 | iteration scheme needs to be used. | ||
| 251 | |||
| 252 | At eviction time, the gpu_vm_bos of *all* the gpu_vms an external | ||
| 253 | object is bound to need to be put on their gpu_vm's evict list. | ||
| 254 | However, when evicting an external object, the dma_resvs of the | ||
| 255 | gpu_vms the object is bound to are typically not held. Only | ||
| 256 | the object's private dma_resv can be guaranteed to be held. If there | ||
| 257 | is a ww_acquire context at hand at eviction time we could grab those | ||
| 258 | dma_resvs but that could cause expensive ww_mutex rollbacks. A simple | ||
| 259 | option is to just mark the gpu_vm_bos of the evicted gem object with | ||
| 260 | an ``evicted`` bool that is inspected before the next time the | ||
| 261 | corresponding gpu_vm evicted list needs to be traversed. For example, when | ||
| 262 | traversing the list of external objects and locking them. At that time, | ||
| 263 | both the gpu_vm's dma_resv and the object's dma_resv is held, and the | ||
| 264 | gpu_vm_bo marked evicted, can then be added to the gpu_vm's list of | ||
| 265 | evicted gpu_vm_bos. The ``evicted`` bool is formally protected by the | ||
| 266 | object's dma_resv. | ||
| 267 | |||
| 268 | The exec function becomes | ||
| 269 | |||
| 270 | .. code-block:: C | ||
| 271 | |||
| 272 | dma_resv_lock(gpu_vm->resv); | ||
| 273 | |||
| 274 | // External object list is protected by the gpu_vm->resv lock. | ||
| 275 | for_each_gpu_vm_bo_on_extobj_list(gpu_vm, &gpu_vm_bo) { | ||
| 276 | dma_resv_lock(gpu_vm_bo.gem_obj->resv); | ||
| 277 | if (gpu_vm_bo_marked_evicted(&gpu_vm_bo)) | ||
| 278 | add_gpu_vm_bo_to_evict_list(&gpu_vm_bo, &gpu_vm->evict_list); | ||
| 279 | } | ||
| 280 | |||
| 281 | for_each_gpu_vm_bo_on_evict_list(&gpu_vm->evict_list, &gpu_vm_bo) { | ||
| 282 | validate_gem_bo(&gpu_vm_bo->gem_bo); | ||
| 283 | |||
| 284 | for_each_gpu_vma_of_gpu_vm_bo(&gpu_vm_bo, &gpu_vma) | ||
| 285 | move_gpu_vma_to_rebind_list(&gpu_vma, &gpu_vm->rebind_list); | ||
| 286 | } | ||
| 287 | |||
| 288 | for_each_gpu_vma_on_rebind_list(&gpu vm->rebind_list, &gpu_vma) { | ||
| 289 | rebind_gpu_vma(&gpu_vma); | ||
| 290 | remove_gpu_vma_from_rebind_list(&gpu_vma); | ||
| 291 | } | ||
| 292 | |||
| 293 | add_dependencies(&gpu_job, &gpu_vm->resv); | ||
| 294 | job_dma_fence = gpu_submit(&gpu_job)); | ||
| 295 | |||
| 296 | add_dma_fence(job_dma_fence, &gpu_vm->resv); | ||
| 297 | for_each_external_obj(gpu_vm, &obj) | ||
| 298 | add_dma_fence(job_dma_fence, &obj->resv); | ||
| 299 | dma_resv_unlock_all_resv_locks(); | ||
| 300 | |||
| 301 | And the corresponding shared-object aware eviction would look like: | ||
| 302 | |||
| 303 | .. code-block:: C | ||
| 304 | |||
| 305 | obj = get_object_from_lru(); | ||
| 306 | |||
| 307 | dma_resv_lock(obj->resv); | ||
| 308 | for_each_gpu_vm_bo_of_obj(obj, &gpu_vm_bo) | ||
| 309 | if (object_is_vm_local(obj)) | ||
| 310 | add_gpu_vm_bo_to_evict_list(&gpu_vm_bo, &gpu_vm->evict_list); | ||
| 311 | else | ||
| 312 | mark_gpu_vm_bo_evicted(&gpu_vm_bo); | ||
| 313 | |||
| 314 | add_dependencies(&eviction_job, &obj->resv); | ||
| 315 | job_dma_fence = gpu_submit(&eviction_job); | ||
| 316 | add_dma_fence(&obj->resv, job_dma_fence); | ||
| 317 | |||
| 318 | dma_resv_unlock(&obj->resv); | ||
| 319 | put_object(obj); | ||
| 320 | |||
| 321 | .. _Spinlock iteration: | ||
| 322 | |||
| 323 | Accessing the gpu_vm's lists without the dma_resv lock held | ||
| 324 | =========================================================== | ||
| 325 | |||
| 326 | Some drivers will hold the gpu_vm's dma_resv lock when accessing the | ||
| 327 | gpu_vm's evict list and external objects lists. However, there are | ||
| 328 | drivers that need to access these lists without the dma_resv lock | ||
| 329 | held, for example due to asynchronous state updates from within the | ||
| 330 | dma_fence signalling critical path. In such cases, a spinlock can be | ||
| 331 | used to protect manipulation of the lists. However, since higher level | ||
| 332 | sleeping locks need to be taken for each list item while iterating | ||
| 333 | over the lists, the items already iterated over need to be | ||
| 334 | temporarily moved to a private list and the spinlock released | ||
| 335 | while processing each item: | ||
| 336 | |||
| 337 | .. code block:: C | ||
| 338 | |||
| 339 | struct list_head still_in_list; | ||
| 340 | |||
| 341 | INIT_LIST_HEAD(&still_in_list); | ||
| 342 | |||
| 343 | spin_lock(&gpu_vm->list_lock); | ||
| 344 | do { | ||
| 345 | struct list_head *entry = list_first_entry_or_null(&gpu_vm->list, head); | ||
| 346 | |||
| 347 | if (!entry) | ||
| 348 | break; | ||
| 349 | |||
| 350 | list_move_tail(&entry->head, &still_in_list); | ||
| 351 | list_entry_get_unless_zero(entry); | ||
| 352 | spin_unlock(&gpu_vm->list_lock); | ||
| 353 | |||
| 354 | process(entry); | ||
| 355 | |||
| 356 | spin_lock(&gpu_vm->list_lock); | ||
| 357 | list_entry_put(entry); | ||
| 358 | } while (true); | ||
| 359 | |||
| 360 | list_splice_tail(&still_in_list, &gpu_vm->list); | ||
| 361 | spin_unlock(&gpu_vm->list_lock); | ||
| 362 | |||
| 363 | Due to the additional locking and atomic operations, drivers that *can* | ||
| 364 | avoid accessing the gpu_vm's list outside of the dma_resv lock | ||
| 365 | might want to avoid also this iteration scheme. Particularly, if the | ||
| 366 | driver anticipates a large number of list items. For lists where the | ||
| 367 | anticipated number of list items is small, where list iteration doesn't | ||
| 368 | happen very often or if there is a significant additional cost | ||
| 369 | associated with each iteration, the atomic operation overhead | ||
| 370 | associated with this type of iteration is, most likely, negligible. Note that | ||
| 371 | if this scheme is used, it is necessary to make sure this list | ||
| 372 | iteration is protected by an outer level lock or semaphore, since list | ||
| 373 | items are temporarily pulled off the list while iterating, and it is | ||
| 374 | also worth mentioning that the local list ``still_in_list`` should | ||
| 375 | also be considered protected by the ``gpu_vm->list_lock``, and it is | ||
| 376 | thus possible that items can be removed also from the local list | ||
| 377 | concurrently with list iteration. | ||
| 378 | |||
| 379 | Please refer to the :ref:`DRM GPUVM locking section | ||
| 380 | <drm_gpuvm_locking>` and its internal | ||
| 381 | :c:func:`get_next_vm_bo_from_list` function. | ||
| 382 | |||
| 383 | |||
| 384 | userptr gpu_vmas | ||
| 385 | ================ | ||
| 386 | |||
| 387 | A userptr gpu_vma is a gpu_vma that, instead of mapping a buffer object to a | ||
| 388 | GPU virtual address range, directly maps a CPU mm range of anonymous- | ||
| 389 | or file page-cache pages. | ||
| 390 | A very simple approach would be to just pin the pages using | ||
| 391 | pin_user_pages() at bind time and unpin them at unbind time, but this | ||
| 392 | creates a Denial-Of-Service vector since a single user-space process | ||
| 393 | would be able to pin down all of system memory, which is not | ||
| 394 | desirable. (For special use-cases and assuming proper accounting pinning might | ||
| 395 | still be a desirable feature, though). What we need to do in the | ||
| 396 | general case is to obtain a reference to the desired pages, make sure | ||
| 397 | we are notified using a MMU notifier just before the CPU mm unmaps the | ||
| 398 | pages, dirty them if they are not mapped read-only to the GPU, and | ||
| 399 | then drop the reference. | ||
| 400 | When we are notified by the MMU notifier that CPU mm is about to drop the | ||
| 401 | pages, we need to stop GPU access to the pages by waiting for VM idle | ||
| 402 | in the MMU notifier and make sure that before the next time the GPU | ||
| 403 | tries to access whatever is now present in the CPU mm range, we unmap | ||
| 404 | the old pages from the GPU page tables and repeat the process of | ||
| 405 | obtaining new page references. (See the :ref:`notifier example | ||
| 406 | <Invalidation example>` below). Note that when the core mm decides to | ||
| 407 | laundry pages, we get such an unmap MMU notification and can mark the | ||
| 408 | pages dirty again before the next GPU access. We also get similar MMU | ||
| 409 | notifications for NUMA accounting which the GPU driver doesn't really | ||
| 410 | need to care about, but so far it has proven difficult to exclude | ||
| 411 | certain notifications. | ||
| 412 | |||
| 413 | Using a MMU notifier for device DMA (and other methods) is described in | ||
| 414 | :ref:`the pin_user_pages() documentation <mmu-notifier-registration-case>`. | ||
| 415 | |||
| 416 | Now, the method of obtaining struct page references using | ||
| 417 | get_user_pages() unfortunately can't be used under a dma_resv lock | ||
| 418 | since that would violate the locking order of the dma_resv lock vs the | ||
| 419 | mmap_lock that is grabbed when resolving a CPU pagefault. This means | ||
| 420 | the gpu_vm's list of userptr gpu_vmas needs to be protected by an | ||
| 421 | outer lock, which in our example below is the ``gpu_vm->lock``. | ||
| 422 | |||
| 423 | The MMU interval seqlock for a userptr gpu_vma is used in the following | ||
| 424 | way: | ||
| 425 | |||
| 426 | .. code-block:: C | ||
| 427 | |||
| 428 | // Exclusive locking mode here is strictly needed only if there are | ||
| 429 | // invalidated userptr gpu_vmas present, to avoid concurrent userptr | ||
| 430 | // revalidations of the same userptr gpu_vma. | ||
| 431 | down_write(&gpu_vm->lock); | ||
| 432 | retry: | ||
| 433 | |||
| 434 | // Note: mmu_interval_read_begin() blocks until there is no | ||
| 435 | // invalidation notifier running anymore. | ||
| 436 | seq = mmu_interval_read_begin(&gpu_vma->userptr_interval); | ||
| 437 | if (seq != gpu_vma->saved_seq) { | ||
| 438 | obtain_new_page_pointers(&gpu_vma); | ||
| 439 | dma_resv_lock(&gpu_vm->resv); | ||
| 440 | add_gpu_vma_to_revalidate_list(&gpu_vma, &gpu_vm); | ||
| 441 | dma_resv_unlock(&gpu_vm->resv); | ||
| 442 | gpu_vma->saved_seq = seq; | ||
| 443 | } | ||
| 444 | |||
| 445 | // The usual revalidation goes here. | ||
| 446 | |||
| 447 | // Final userptr sequence validation may not happen before the | ||
| 448 | // submission dma_fence is added to the gpu_vm's resv, from the POW | ||
| 449 | // of the MMU invalidation notifier. Hence the | ||
| 450 | // userptr_notifier_lock that will make them appear atomic. | ||
| 451 | |||
| 452 | add_dependencies(&gpu_job, &gpu_vm->resv); | ||
| 453 | down_read(&gpu_vm->userptr_notifier_lock); | ||
| 454 | if (mmu_interval_read_retry(&gpu_vma->userptr_interval, gpu_vma->saved_seq)) { | ||
| 455 | up_read(&gpu_vm->userptr_notifier_lock); | ||
| 456 | goto retry; | ||
| 457 | } | ||
| 458 | |||
| 459 | job_dma_fence = gpu_submit(&gpu_job)); | ||
| 460 | |||
| 461 | add_dma_fence(job_dma_fence, &gpu_vm->resv); | ||
| 462 | |||
| 463 | for_each_external_obj(gpu_vm, &obj) | ||
| 464 | add_dma_fence(job_dma_fence, &obj->resv); | ||
| 465 | |||
| 466 | dma_resv_unlock_all_resv_locks(); | ||
| 467 | up_read(&gpu_vm->userptr_notifier_lock); | ||
| 468 | up_write(&gpu_vm->lock); | ||
| 469 | |||
| 470 | The code between ``mmu_interval_read_begin()`` and the | ||
| 471 | ``mmu_interval_read_retry()`` marks the read side critical section of | ||
| 472 | what we call the ``userptr_seqlock``. In reality, the gpu_vm's userptr | ||
| 473 | gpu_vma list is looped through, and the check is done for *all* of its | ||
| 474 | userptr gpu_vmas, although we only show a single one here. | ||
| 475 | |||
| 476 | The userptr gpu_vma MMU invalidation notifier might be called from | ||
| 477 | reclaim context and, again, to avoid locking order violations, we can't | ||
| 478 | take any dma_resv lock nor the gpu_vm->lock from within it. | ||
| 479 | |||
| 480 | .. _Invalidation example: | ||
| 481 | .. code-block:: C | ||
| 482 | |||
| 483 | bool gpu_vma_userptr_invalidate(userptr_interval, cur_seq) | ||
| 484 | { | ||
| 485 | // Make sure the exec function either sees the new sequence | ||
| 486 | // and backs off or we wait for the dma-fence: | ||
| 487 | |||
| 488 | down_write(&gpu_vm->userptr_notifier_lock); | ||
| 489 | mmu_interval_set_seq(userptr_interval, cur_seq); | ||
| 490 | up_write(&gpu_vm->userptr_notifier_lock); | ||
| 491 | |||
| 492 | // At this point, the exec function can't succeed in | ||
| 493 | // submitting a new job, because cur_seq is an invalid | ||
| 494 | // sequence number and will always cause a retry. When all | ||
| 495 | // invalidation callbacks, the mmu notifier core will flip | ||
| 496 | // the sequence number to a valid one. However we need to | ||
| 497 | // stop gpu access to the old pages here. | ||
| 498 | |||
| 499 | dma_resv_wait_timeout(&gpu_vm->resv, DMA_RESV_USAGE_BOOKKEEP, | ||
| 500 | false, MAX_SCHEDULE_TIMEOUT); | ||
| 501 | return true; | ||
| 502 | } | ||
| 503 | |||
| 504 | When this invalidation notifier returns, the GPU can no longer be | ||
| 505 | accessing the old pages of the userptr gpu_vma and needs to redo the | ||
| 506 | page-binding before a new GPU submission can succeed. | ||
| 507 | |||
| 508 | Efficient userptr gpu_vma exec_function iteration | ||
| 509 | _________________________________________________ | ||
| 510 | |||
| 511 | If the gpu_vm's list of userptr gpu_vmas becomes large, it's | ||
| 512 | inefficient to iterate through the complete lists of userptrs on each | ||
| 513 | exec function to check whether each userptr gpu_vma's saved | ||
| 514 | sequence number is stale. A solution to this is to put all | ||
| 515 | *invalidated* userptr gpu_vmas on a separate gpu_vm list and | ||
| 516 | only check the gpu_vmas present on this list on each exec | ||
| 517 | function. This list will then lend itself very-well to the spinlock | ||
| 518 | locking scheme that is | ||
| 519 | :ref:`described in the spinlock iteration section <Spinlock iteration>`, since | ||
| 520 | in the mmu notifier, where we add the invalidated gpu_vmas to the | ||
| 521 | list, it's not possible to take any outer locks like the | ||
| 522 | ``gpu_vm->lock`` or the ``gpu_vm->resv`` lock. Note that the | ||
| 523 | ``gpu_vm->lock`` still needs to be taken while iterating to ensure the list is | ||
| 524 | complete, as also mentioned in that section. | ||
| 525 | |||
| 526 | If using an invalidated userptr list like this, the retry check in the | ||
| 527 | exec function trivially becomes a check for invalidated list empty. | ||
| 528 | |||
| 529 | Locking at bind and unbind time | ||
| 530 | =============================== | ||
| 531 | |||
| 532 | At bind time, assuming a GEM object backed gpu_vma, each | ||
| 533 | gpu_vma needs to be associated with a gpu_vm_bo and that | ||
| 534 | gpu_vm_bo in turn needs to be added to the GEM object's | ||
| 535 | gpu_vm_bo list, and possibly to the gpu_vm's external object | ||
| 536 | list. This is referred to as *linking* the gpu_vma, and typically | ||
| 537 | requires that the ``gpu_vm->lock`` and the ``gem_object->gpuva_lock`` | ||
| 538 | are held. When unlinking a gpu_vma the same locks should be held, | ||
| 539 | and that ensures that when iterating over ``gpu_vmas`, either under | ||
| 540 | the ``gpu_vm->resv`` or the GEM object's dma_resv, that the gpu_vmas | ||
| 541 | stay alive as long as the lock under which we iterate is not released. For | ||
| 542 | userptr gpu_vmas it's similarly required that during vma destroy, the | ||
| 543 | outer ``gpu_vm->lock`` is held, since otherwise when iterating over | ||
| 544 | the invalidated userptr list as described in the previous section, | ||
| 545 | there is nothing keeping those userptr gpu_vmas alive. | ||
| 546 | |||
| 547 | Locking for recoverable page-fault page-table updates | ||
| 548 | ===================================================== | ||
| 549 | |||
| 550 | There are two important things we need to ensure with locking for | ||
| 551 | recoverable page-faults: | ||
| 552 | |||
| 553 | * At the time we return pages back to the system / allocator for | ||
| 554 | reuse, there should be no remaining GPU mappings and any GPU TLB | ||
| 555 | must have been flushed. | ||
| 556 | * The unmapping and mapping of a gpu_vma must not race. | ||
| 557 | |||
| 558 | Since the unmapping (or zapping) of GPU ptes is typically taking place | ||
| 559 | where it is hard or even impossible to take any outer level locks we | ||
| 560 | must either introduce a new lock that is held at both mapping and | ||
| 561 | unmapping time, or look at the locks we do hold at unmapping time and | ||
| 562 | make sure that they are held also at mapping time. For userptr | ||
| 563 | gpu_vmas, the ``userptr_seqlock`` is held in write mode in the mmu | ||
| 564 | invalidation notifier where zapping happens. Hence, if the | ||
| 565 | ``userptr_seqlock`` as well as the ``gpu_vm->userptr_notifier_lock`` | ||
| 566 | is held in read mode during mapping, it will not race with the | ||
| 567 | zapping. For GEM object backed gpu_vmas, zapping will take place under | ||
| 568 | the GEM object's dma_resv and ensuring that the dma_resv is held also | ||
| 569 | when populating the page-tables for any gpu_vma pointing to the GEM | ||
| 570 | object, will similarly ensure we are race-free. | ||
| 571 | |||
| 572 | If any part of the mapping is performed asynchronously | ||
| 573 | under a dma-fence with these locks released, the zapping will need to | ||
| 574 | wait for that dma-fence to signal under the relevant lock before | ||
| 575 | starting to modify the page-table. | ||
| 576 | |||
| 577 | Since modifying the | ||
| 578 | page-table structure in a way that frees up page-table memory | ||
| 579 | might also require outer level locks, the zapping of GPU ptes | ||
| 580 | typically focuses only on zeroing page-table or page-directory entries | ||
| 581 | and flushing TLB, whereas freeing of page-table memory is deferred to | ||
| 582 | unbind or rebind time. | ||
| @@ -0,0 +1,13 @@ | |||
| 1 | ======================================= | ||
| 2 | drm/imagination PowerVR Graphics Driver | ||
| 3 | ======================================= | ||
| 4 | |||
| 5 | .. kernel-doc:: drivers/gpu/drm/imagination/pvr_drv.c | ||
| 6 | :doc: PowerVR (Series 6 and later) and IMG Graphics Driver | ||
| 7 | |||
| 8 | Contents | ||
| 9 | ======== | ||
| 10 | .. toctree:: | ||
| 11 | :maxdepth: 2 | ||
| 12 | |||
| 13 | uapi | ||
| @@ -0,0 +1,171 @@ | |||
| 1 | ==== | ||
| 2 | UAPI | ||
| 3 | ==== | ||
| 4 | The sources associated with this section can be found in ``pvr_drm.h``. | ||
| 5 | |||
| 6 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 7 | :doc: PowerVR UAPI | ||
| 8 | |||
| 9 | OBJECT ARRAYS | ||
| 10 | ============= | ||
| 11 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 12 | :identifiers: drm_pvr_obj_array | ||
| 13 | |||
| 14 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 15 | :identifiers: DRM_PVR_OBJ_ARRAY | ||
| 16 | |||
| 17 | IOCTLS | ||
| 18 | ====== | ||
| 19 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 20 | :doc: PowerVR IOCTL interface | ||
| 21 | |||
| 22 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 23 | :identifiers: PVR_IOCTL | ||
| 24 | |||
| 25 | DEV_QUERY | ||
| 26 | --------- | ||
| 27 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 28 | :doc: PowerVR IOCTL DEV_QUERY interface | ||
| 29 | |||
| 30 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 31 | :identifiers: drm_pvr_dev_query | ||
| 32 | |||
| 33 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 34 | :identifiers: drm_pvr_ioctl_dev_query_args | ||
| 35 | |||
| 36 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 37 | :identifiers: drm_pvr_dev_query_gpu_info | ||
| 38 | drm_pvr_dev_query_runtime_info | ||
| 39 | drm_pvr_dev_query_hwrt_info | ||
| 40 | drm_pvr_dev_query_quirks | ||
| 41 | drm_pvr_dev_query_enhancements | ||
| 42 | |||
| 43 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 44 | :identifiers: drm_pvr_heap_id | ||
| 45 | drm_pvr_heap | ||
| 46 | drm_pvr_dev_query_heap_info | ||
| 47 | |||
| 48 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 49 | :identifiers: drm_pvr_static_data_area_usage | ||
| 50 | drm_pvr_static_data_area | ||
| 51 | drm_pvr_dev_query_static_data_areas | ||
| 52 | |||
| 53 | CREATE_BO | ||
| 54 | --------- | ||
| 55 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 56 | :doc: PowerVR IOCTL CREATE_BO interface | ||
| 57 | |||
| 58 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 59 | :identifiers: drm_pvr_ioctl_create_bo_args | ||
| 60 | |||
| 61 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 62 | :doc: Flags for CREATE_BO | ||
| 63 | |||
| 64 | GET_BO_MMAP_OFFSET | ||
| 65 | ------------------ | ||
| 66 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 67 | :doc: PowerVR IOCTL GET_BO_MMAP_OFFSET interface | ||
| 68 | |||
| 69 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 70 | :identifiers: drm_pvr_ioctl_get_bo_mmap_offset_args | ||
| 71 | |||
| 72 | CREATE_VM_CONTEXT and DESTROY_VM_CONTEXT | ||
| 73 | ---------------------------------------- | ||
| 74 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 75 | :doc: PowerVR IOCTL CREATE_VM_CONTEXT and DESTROY_VM_CONTEXT interfaces | ||
| 76 | |||
| 77 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 78 | :identifiers: drm_pvr_ioctl_create_vm_context_args | ||
| 79 | drm_pvr_ioctl_destroy_vm_context_args | ||
| 80 | |||
| 81 | VM_MAP and VM_UNMAP | ||
| 82 | ------------------- | ||
| 83 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 84 | :doc: PowerVR IOCTL VM_MAP and VM_UNMAP interfaces | ||
| 85 | |||
| 86 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 87 | :identifiers: drm_pvr_ioctl_vm_map_args | ||
| 88 | drm_pvr_ioctl_vm_unmap_args | ||
| 89 | |||
| 90 | CREATE_CONTEXT and DESTROY_CONTEXT | ||
| 91 | ---------------------------------- | ||
| 92 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 93 | :doc: PowerVR IOCTL CREATE_CONTEXT and DESTROY_CONTEXT interfaces | ||
| 94 | |||
| 95 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 96 | :identifiers: drm_pvr_ioctl_create_context_args | ||
| 97 | |||
| 98 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 99 | :identifiers: drm_pvr_ctx_priority | ||
| 100 | drm_pvr_ctx_type | ||
| 101 | drm_pvr_static_render_context_state | ||
| 102 | drm_pvr_static_render_context_state_format | ||
| 103 | drm_pvr_reset_framework | ||
| 104 | drm_pvr_reset_framework_format | ||
| 105 | |||
| 106 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 107 | :identifiers: drm_pvr_ioctl_destroy_context_args | ||
| 108 | |||
| 109 | CREATE_FREE_LIST and DESTROY_FREE_LIST | ||
| 110 | -------------------------------------- | ||
| 111 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 112 | :doc: PowerVR IOCTL CREATE_FREE_LIST and DESTROY_FREE_LIST interfaces | ||
| 113 | |||
| 114 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 115 | :identifiers: drm_pvr_ioctl_create_free_list_args | ||
| 116 | |||
| 117 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 118 | :identifiers: drm_pvr_ioctl_destroy_free_list_args | ||
| 119 | |||
| 120 | CREATE_HWRT_DATASET and DESTROY_HWRT_DATASET | ||
| 121 | -------------------------------------------- | ||
| 122 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 123 | :doc: PowerVR IOCTL CREATE_HWRT_DATASET and DESTROY_HWRT_DATASET interfaces | ||
| 124 | |||
| 125 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 126 | :identifiers: drm_pvr_ioctl_create_hwrt_dataset_args | ||
| 127 | |||
| 128 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 129 | :identifiers: drm_pvr_create_hwrt_geom_data_args | ||
| 130 | drm_pvr_create_hwrt_rt_data_args | ||
| 131 | |||
| 132 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 133 | :identifiers: drm_pvr_ioctl_destroy_hwrt_dataset_args | ||
| 134 | |||
| 135 | SUBMIT_JOBS | ||
| 136 | ----------- | ||
| 137 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 138 | :doc: PowerVR IOCTL SUBMIT_JOBS interface | ||
| 139 | |||
| 140 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 141 | :doc: Flags for the drm_pvr_sync_op object. | ||
| 142 | |||
| 143 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 144 | :identifiers: drm_pvr_ioctl_submit_jobs_args | ||
| 145 | |||
| 146 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 147 | :doc: Flags for SUBMIT_JOB ioctl geometry command. | ||
| 148 | |||
| 149 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 150 | :doc: Flags for SUBMIT_JOB ioctl fragment command. | ||
| 151 | |||
| 152 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 153 | :doc: Flags for SUBMIT_JOB ioctl compute command. | ||
| 154 | |||
| 155 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 156 | :doc: Flags for SUBMIT_JOB ioctl transfer command. | ||
| 157 | |||
| 158 | .. kernel-doc:: include/uapi/drm/pvr_drm.h | ||
| 159 | :identifiers: drm_pvr_sync_op | ||
| 160 | drm_pvr_job_type | ||
| 161 | drm_pvr_hwrt_data_ref | ||
| 162 | drm_pvr_job | ||
| 163 | |||
| 164 | Internal notes | ||
| 165 | ============== | ||
| 166 | .. kernel-doc:: drivers/gpu/drm/imagination/pvr_device.h | ||
| 167 | :doc: IOCTL validation helpers | ||
| 168 | |||
| 169 | .. kernel-doc:: drivers/gpu/drm/imagination/pvr_device.h | ||
| 170 | :identifiers: PVR_STATIC_ASSERT_64BIT_ALIGNED PVR_IOCTL_UNION_PADDING_CHECK | ||
| 171 | pvr_ioctl_union_padding_check | ||
| @@ -7,3 +7,4 @@ Misc DRM driver uAPI- and feature implementation guidelines · 6 unmodified lines | |||
| 7 | .. toctree:: | 7 | .. toctree:: |
| 8 | 8 | ||
| 9 | drm-vm-bind-async | 9 | drm-vm-bind-async |
| 10 | drm-vm-bind-locking | ||
| @@ -70,35 +70,42 @@ When the time comes for Xe, the protection will be lifted on Xe and kept in i915 · 69 unmodified lines | |||
| 70 | Xe – Pre-Merge Goals - Work-in-Progress | 70 | Xe – Pre-Merge Goals - Work-in-Progress |
| 71 | ======================================= | 71 | ======================================= |
| 72 | 72 | ||
| 73 | Drm_scheduler | 73 | Display integration with i915 |
| 74 | ------------- | 74 | ----------------------------- |
| 75 | Xe primarily uses Firmware based scheduling (GuC FW). However, it will use | 75 | In order to share the display code with the i915 driver so that there is maximum |
| 76 | drm_scheduler as the scheduler ‘frontend’ for userspace submission in order to | 76 | reuse, the i915/display/ code is built twice, once for i915.ko and then for |
| 77 | resolve syncobj and dma-buf implicit sync dependencies. However, drm_scheduler is | 77 | xe.ko. Currently, the i915/display code in Xe tree is polluted with many 'ifdefs' |
| 78 | not yet prepared to handle the 1-to-1 relationship between drm_gpu_scheduler and | 78 | depending on the build target. The goal is to refactor both Xe and i915/display |
| 79 | drm_sched_entity. | 79 | code simultaneously in order to get a clean result before they land upstream, so |
| 80 | that display can already be part of the initial pull request towards drm-next. | ||
| 80 | 81 | ||
| 81 | Deeper changes to drm_scheduler should *not* be required to get Xe accepted, but | 82 | However, display code should not gate the acceptance of Xe in upstream. Xe |
| 82 | some consensus needs to be reached between Xe and other community drivers that | 83 | patches will be refactored in a way that display code can be removed, if needed, |
| 83 | could also benefit from this work, for coupling FW based/assisted submission such | 84 | from the first pull request of Xe towards drm-next. The expectation is that when |
| 84 | as the ARM’s new Mali GPU driver, and others. | 85 | both drivers are part of the drm-tip, the introduction of cleaner patches will be |
| 86 | easier and speed up. | ||
| 85 | 87 | ||
| 86 | As a key measurable result, the patch series introducing Xe itself shall not | 88 | Xe – uAPI high level overview |
| 87 | depend on any other patch touching drm_scheduler itself that was not yet merged | 89 | ============================= |
| 88 | through drm-misc. This, by itself, already includes the reach of an agreement for | ||
| 89 | uniform 1 to 1 relationship implementation / usage across drivers. | ||
| 90 | 90 | ||
| 91 | ASYNC VM_BIND | 91 | ...Warning: To be done in follow up patches after/when/where the main consensus in various items are individually reached. |
| 92 | ------------- | ||
| 93 | Although having a common DRM level IOCTL for VM_BIND is not a requirement to get | ||
| 94 | Xe merged, it is mandatory to have a consensus with other drivers and Mesa. | ||
| 95 | It needs to be clear how to handle async VM_BIND and interactions with userspace | ||
| 96 | memory fences. Ideally with helper support so people don't get it wrong in all | ||
| 97 | possible ways. | ||
| 98 | 92 | ||
| 99 | As a key measurable result, the benefits of ASYNC VM_BIND and a discussion of | 93 | Xe – Pre-Merge Goals - Completed |
| 100 | various flavors, error handling and sample API suggestions are documented in | 94 | ================================ |
| 101 | :doc:`The ASYNC VM_BIND document </gpu/drm-vm-bind-async>`. | 95 | |
| 96 | Drm_exec | ||
| 97 | -------- | ||
| 98 | Helper to make dma_resv locking for a big number of buffers is getting removed in | ||
| 99 | the drm_exec series proposed in https://patchwork.freedesktop.org/patch/524376/ | ||
| 100 | If that happens, Xe needs to change and incorporate the changes in the driver. | ||
| 101 | The goal is to engage with the Community to understand if the best approach is to | ||
| 102 | move that to the drivers that are using it or if we should keep the helpers in | ||
| 103 | place waiting for Xe to get merged. | ||
| 104 | |||
| 105 | This item ties into the GPUVA, VM_BIND, and even long-running compute support. | ||
| 106 | |||
| 107 | As a key measurable result, we need to have a community consensus documented in | ||
| 108 | this document and the Xe driver prepared for the changes, if necessary. | ||
| 102 | 109 | ||
| 103 | Userptr integration and vm_bind | 110 | Userptr integration and vm_bind |
| 104 | ------------------------------- | 111 | ------------------------------- |
| @@ -123,10 +130,45 @@ Documentation should include: · 18 unmodified lines | |||
| 123 | 130 | ||
| 124 | * O(1) complexity under VM_BIND. | 131 | * O(1) complexity under VM_BIND. |
| 125 | 132 | ||
| 133 | The document is now included in the drm documentation :doc:`here </gpu/drm-vm-bind-async>`. | ||
| 134 | |||
| 126 | Some parts of userptr like mmu_notifiers should become GPUVA or DRM helpers when | 135 | Some parts of userptr like mmu_notifiers should become GPUVA or DRM helpers when |
| 127 | the second driver supporting VM_BIND+userptr appears. Details to be defined when | 136 | the second driver supporting VM_BIND+userptr appears. Details to be defined when |
| 128 | the time comes. | 137 | the time comes. |
| 129 | 138 | ||
| 139 | The DRM GPUVM helpers do not yet include the userptr parts, but discussions | ||
| 140 | about implementing them are ongoing. | ||
| 141 | |||
| 142 | ASYNC VM_BIND | ||
| 143 | ------------- | ||
| 144 | Although having a common DRM level IOCTL for VM_BIND is not a requirement to get | ||
| 145 | Xe merged, it is mandatory to have a consensus with other drivers and Mesa. | ||
| 146 | It needs to be clear how to handle async VM_BIND and interactions with userspace | ||
| 147 | memory fences. Ideally with helper support so people don't get it wrong in all | ||
| 148 | possible ways. | ||
| 149 | |||
| 150 | As a key measurable result, the benefits of ASYNC VM_BIND and a discussion of | ||
| 151 | various flavors, error handling and sample API suggestions are documented in | ||
| 152 | :doc:`The ASYNC VM_BIND document </gpu/drm-vm-bind-async>`. | ||
| 153 | |||
| 154 | Drm_scheduler | ||
| 155 | ------------- | ||
| 156 | Xe primarily uses Firmware based scheduling (GuC FW). However, it will use | ||
| 157 | drm_scheduler as the scheduler ‘frontend’ for userspace submission in order to | ||
| 158 | resolve syncobj and dma-buf implicit sync dependencies. However, drm_scheduler is | ||
| 159 | not yet prepared to handle the 1-to-1 relationship between drm_gpu_scheduler and | ||
| 160 | drm_sched_entity. | ||
| 161 | |||
| 162 | Deeper changes to drm_scheduler should *not* be required to get Xe accepted, but | ||
| 163 | some consensus needs to be reached between Xe and other community drivers that | ||
| 164 | could also benefit from this work, for coupling FW based/assisted submission such | ||
| 165 | as the ARM’s new Mali GPU driver, and others. | ||
| 166 | |||
| 167 | As a key measurable result, the patch series introducing Xe itself shall not | ||
| 168 | depend on any other patch touching drm_scheduler itself that was not yet merged | ||
| 169 | through drm-misc. This, by itself, already includes the reach of an agreement for | ||
| 170 | uniform 1 to 1 relationship implementation / usage across drivers. | ||
| 171 | |||
| 130 | Long running compute: minimal data structure/scaffolding | 172 | Long running compute: minimal data structure/scaffolding |
| 131 | -------------------------------------------------------- | 173 | -------------------------------------------------------- |
| 132 | The generic scheduler code needs to include the handling of endless compute | 174 | The generic scheduler code needs to include the handling of endless compute |
| @@ -139,46 +181,6 @@ this minimal drm/scheduler work, if needed, merged to drm-misc in a way that any · 6 unmodified lines | |||
| 139 | drm driver, including Xe, could re-use and add their own individual needs on top | 181 | drm driver, including Xe, could re-use and add their own individual needs on top |
| 140 | in a next stage. However, this should not block the initial merge. | 182 | in a next stage. However, this should not block the initial merge. |
| 141 | 183 | ||
| 142 | This is a non-blocker item since the driver without the support for the long | ||
| 143 | running compute enabled is not a showstopper. | ||
| 144 | |||
| 145 | Display integration with i915 | ||
| 146 | ----------------------------- | ||
| 147 | In order to share the display code with the i915 driver so that there is maximum | ||
| 148 | reuse, the i915/display/ code is built twice, once for i915.ko and then for | ||
| 149 | xe.ko. Currently, the i915/display code in Xe tree is polluted with many 'ifdefs' | ||
| 150 | depending on the build target. The goal is to refactor both Xe and i915/display | ||
| 151 | code simultaneously in order to get a clean result before they land upstream, so | ||
| 152 | that display can already be part of the initial pull request towards drm-next. | ||
| 153 | |||
| 154 | However, display code should not gate the acceptance of Xe in upstream. Xe | ||
| 155 | patches will be refactored in a way that display code can be removed, if needed, | ||
| 156 | from the first pull request of Xe towards drm-next. The expectation is that when | ||
| 157 | both drivers are part of the drm-tip, the introduction of cleaner patches will be | ||
| 158 | easier and speed up. | ||
| 159 | |||
| 160 | Drm_exec | ||
| 161 | -------- | ||
| 162 | Helper to make dma_resv locking for a big number of buffers is getting removed in | ||
| 163 | the drm_exec series proposed in https://patchwork.freedesktop.org/patch/524376/ | ||
| 164 | If that happens, Xe needs to change and incorporate the changes in the driver. | ||
| 165 | The goal is to engage with the Community to understand if the best approach is to | ||
| 166 | move that to the drivers that are using it or if we should keep the helpers in | ||
| 167 | place waiting for Xe to get merged. | ||
| 168 | |||
| 169 | This item ties into the GPUVA, VM_BIND, and even long-running compute support. | ||
| 170 | |||
| 171 | As a key measurable result, we need to have a community consensus documented in | ||
| 172 | this document and the Xe driver prepared for the changes, if necessary. | ||
| 173 | |||
| 174 | Xe – uAPI high level overview | ||
| 175 | ============================= | ||
| 176 | |||
| 177 | ...Warning: To be done in follow up patches after/when/where the main consensus in various items are individually reached. | ||
| 178 | |||
| 179 | Xe – Pre-Merge Goals - Completed | ||
| 180 | ================================ | ||
| 181 | |||
| 182 | Dev_coredump | 184 | Dev_coredump |
| 183 | ------------ | 185 | ------------ |
| 184 | 186 | ||
| @@ -337,8 +337,8 @@ connector register/unregister fixes · 336 unmodified lines | |||
| 337 | 337 | ||
| 338 | Level: Intermediate | 338 | Level: Intermediate |
| 339 | 339 | ||
| 340 | Remove load/unload callbacks from all non-DRIVER_LEGACY drivers | 340 | Remove load/unload callbacks |
| 341 | --------------------------------------------------------------- | 341 | ---------------------------- |
| 342 | 342 | ||
| 343 | The load/unload callbacks in struct &drm_driver are very much midlayers, plus | 343 | The load/unload callbacks in struct &drm_driver are very much midlayers, plus |
| 344 | for historical reasons they get the ordering wrong (and we can't fix that) | 344 | for historical reasons they get the ordering wrong (and we can't fix that) |
| @@ -347,8 +347,7 @@ between setting up the &drm_driver structure and calling drm_dev_register(). · 2 unmodified lines | |||
| 347 | - Rework drivers to no longer use the load/unload callbacks, directly coding the | 347 | - Rework drivers to no longer use the load/unload callbacks, directly coding the |
| 348 | load/unload sequence into the driver's probe function. | 348 | load/unload sequence into the driver's probe function. |
| 349 | 349 | ||
| 350 | - Once all non-DRIVER_LEGACY drivers are converted, disallow the load/unload | 350 | - Once all drivers are converted, remove the load/unload callbacks. |
| 351 | callbacks for all modern drivers. | ||
| 352 | 351 | ||
| 353 | Contact: Daniel Vetter | 352 | Contact: Daniel Vetter |
| 354 | 353 | ||
| @@ -621,6 +620,23 @@ Contact: Javier Martinez Canillas <javierm@redhat.com> · 266 unmodified lines | |||
| 621 | 620 | ||
| 622 | Level: Intermediate | 621 | Level: Intermediate |
| 623 | 622 | ||
| 623 | Clean up and document former selftests suites | ||
| 624 | --------------------------------------------- | ||
| 625 | |||
| 626 | Some KUnit test suites (drm_buddy, drm_cmdline_parser, drm_damage_helper, | ||
| 627 | drm_format, drm_framebuffer, drm_dp_mst_helper, drm_mm, drm_plane_helper and | ||
| 628 | drm_rect) are former selftests suites that have been converted over when KUnit | ||
| 629 | was first introduced. | ||
| 630 | |||
| 631 | These suites were fairly undocumented, and with different goals than what unit | ||
| 632 | tests can be. Trying to identify what each test in these suites actually test | ||
| 633 | for, whether that makes sense for a unit test, and either remove it if it | ||
| 634 | doesn't or document it if it does would be of great help. | ||
| 635 | |||
| 636 | Contact: Maxime Ripard <mripard@kernel.org> | ||
| 637 | |||
| 638 | Level: Intermediate | ||
| 639 | |||
| 624 | Enable trinity for DRM | 640 | Enable trinity for DRM |
| 625 | ---------------------- | 641 | ---------------------- |
| 626 | 642 | ||
| @@ -765,6 +781,29 @@ Contact: Hans de Goede · 138 unmodified lines | |||
| 765 | 781 | ||
| 766 | Level: Advanced | 782 | Level: Advanced |
| 767 | 783 | ||
| 784 | Buffer age or other damage accumulation algorithm for buffer damage | ||
| 785 | =================================================================== | ||
| 786 | |||
| 787 | Drivers that do per-buffer uploads, need a buffer damage handling (rather than | ||
| 788 | frame damage like drivers that do per-plane or per-CRTC uploads), but there is | ||
| 789 | no support to get the buffer age or any other damage accumulation algorithm. | ||
| 790 | |||
| 791 | For this reason, the damage helpers just fallback to a full plane update if the | ||
| 792 | framebuffer attached to a plane has changed since the last page-flip. Drivers | ||
| 793 | set &drm_plane_state.ignore_damage_clips to true as indication to | ||
| 794 | drm_atomic_helper_damage_iter_init() and drm_atomic_helper_damage_iter_next() | ||
| 795 | helpers that the damage clips should be ignored. | ||
| 796 | |||
| 797 | This should be improved to get damage tracking properly working on drivers that | ||
| 798 | do per-buffer uploads. | ||
| 799 | |||
| 800 | More information about damage tracking and references to learning materials can | ||
| 801 | be found in :ref:`damage_tracking_properties`. | ||
| 802 | |||
| 803 | Contact: Javier Martinez Canillas <javierm@redhat.com> | ||
| 804 | |||
| 805 | Level: Advanced | ||
| 806 | |||
| 768 | Outside DRM | 807 | Outside DRM |
| 769 | =========== | 808 | =========== |
| 770 | 809 | ||
| @@ -193,9 +193,23 @@ Review timelines · 192 unmodified lines | |||
| 193 | Generally speaking, the patches get triaged quickly (in less than | 193 | Generally speaking, the patches get triaged quickly (in less than |
| 194 | 48h). But be patient, if your patch is active in patchwork (i.e. it's | 194 | 48h). But be patient, if your patch is active in patchwork (i.e. it's |
| 195 | listed on the project's patch list) the chances it was missed are close to zero. | 195 | listed on the project's patch list) the chances it was missed are close to zero. |
| 196 | Asking the maintainer for status updates on your | 196 | |
| 197 | patch is a good way to ensure your patch is ignored or pushed to the | 197 | The high volume of development on netdev makes reviewers move on |
| 198 | bottom of the priority list. | 198 | from discussions relatively quickly. New comments and replies |
| 199 | are very unlikely to arrive after a week of silence. If a patch | ||
| 200 | is no longer active in patchwork and the thread went idle for more | ||
| 201 | than a week - clarify the next steps and/or post the next version. | ||
| 202 | |||
| 203 | For RFC postings specifically, if nobody responded in a week - reviewers | ||
| 204 | either missed the posting or have no strong opinions. If the code is ready, | ||
| 205 | repost as a PATCH. | ||
| 206 | |||
| 207 | Emails saying just "ping" or "bump" are considered rude. If you can't figure | ||
| 208 | out the status of the patch from patchwork or where the discussion has | ||
| 209 | landed - describe your best guess and ask if it's correct. For example:: | ||
| 210 | |||
| 211 | I don't understand what the next steps are. Person X seems to be unhappy | ||
| 212 | with A, should I do B and repost the patches? | ||
| 199 | 213 | ||
| 200 | .. _Changes requested: | 214 | .. _Changes requested: |
| 201 | 215 | ||
| @@ -338,9 +338,9 @@ Loongson与LoongArch的开发者网站（软件与文档资源）： · 337 unmodified lines | |||
| 338 | 338 | ||
| 339 | LoongArch指令集架构的文档： | 339 | LoongArch指令集架构的文档： |
| 340 | 340 | ||
| 341 | https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.02-CN.pdf （中文版） | 341 | https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.10-CN.pdf （中文版） |
| 342 | 342 | ||
| 343 | https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.02-EN.pdf （英文版） | 343 | https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.10-EN.pdf （英文版） |
| 344 | 344 | ||
| 345 | LoongArch的ELF psABI文档： | 345 | LoongArch的ELF psABI文档： |
| 346 | 346 | ||
| @@ -6503,8 +6503,7 @@ T: git git://anongit.freedesktop.org/drm/drm-misc · 6502 unmodified lines | |||
| 6503 | F: drivers/gpu/drm/sun4i/sun8i* | 6503 | F: drivers/gpu/drm/sun4i/sun8i* |
| 6504 | 6504 | ||
| 6505 | DRM DRIVER FOR ARM PL111 CLCD | 6505 | DRM DRIVER FOR ARM PL111 CLCD |
| 6506 | M: Emma Anholt <emma@anholt.net> | 6506 | S: Orphan |
| 6507 | S: Supported | ||
| 6508 | T: git git://anongit.freedesktop.org/drm/drm-misc | 6507 | T: git git://anongit.freedesktop.org/drm/drm-misc |
| 6509 | F: drivers/gpu/drm/pl111/ | 6508 | F: drivers/gpu/drm/pl111/ |
| 6510 | 6509 | ||
| @@ -6619,8 +6618,7 @@ F: Documentation/devicetree/bindings/display/panel/himax,hx8394.yaml · 108 unmodified lines | |||
| 6619 | F: drivers/gpu/drm/panel/panel-himax-hx8394.c | 6618 | F: drivers/gpu/drm/panel/panel-himax-hx8394.c |
| 6620 | 6619 | ||
| 6621 | DRM DRIVER FOR HX8357D PANELS | 6620 | DRM DRIVER FOR HX8357D PANELS |
| 6622 | M: Emma Anholt <emma@anholt.net> | 6621 | S: Orphan |
| 6623 | S: Maintained | ||
| 6624 | T: git git://anongit.freedesktop.org/drm/drm-misc | 6622 | T: git git://anongit.freedesktop.org/drm/drm-misc |
| 6625 | F: Documentation/devicetree/bindings/display/himax,hx8357d.txt | 6623 | F: Documentation/devicetree/bindings/display/himax,hx8357d.txt |
| 6626 | F: drivers/gpu/drm/tiny/hx8357d.c | 6624 | F: drivers/gpu/drm/tiny/hx8357d.c |
| @@ -7213,8 +7211,8 @@ F: Documentation/devicetree/bindings/display/ti/ · 586 unmodified lines | |||
| 7213 | F: drivers/gpu/drm/omapdrm/ | 7211 | F: drivers/gpu/drm/omapdrm/ |
| 7214 | 7212 | ||
| 7215 | DRM DRIVERS FOR V3D | 7213 | DRM DRIVERS FOR V3D |
| 7216 | M: Emma Anholt <emma@anholt.net> | ||
| 7217 | M: Melissa Wen <mwen@igalia.com> | 7214 | M: Melissa Wen <mwen@igalia.com> |
| 7215 | M: Maíra Canal <mcanal@igalia.com> | ||
| 7218 | S: Supported | 7216 | S: Supported |
| 7219 | T: git git://anongit.freedesktop.org/drm/drm-misc | 7217 | T: git git://anongit.freedesktop.org/drm/drm-misc |
| 7220 | F: Documentation/devicetree/bindings/gpu/brcm,bcm-v3d.yaml | 7218 | F: Documentation/devicetree/bindings/gpu/brcm,bcm-v3d.yaml |
| @@ -7222,7 +7220,6 @@ F: drivers/gpu/drm/v3d/ · 1 unmodified line | |||
| 7222 | F: include/uapi/drm/v3d_drm.h | 7220 | F: include/uapi/drm/v3d_drm.h |
| 7223 | 7221 | ||
| 7224 | DRM DRIVERS FOR VC4 | 7222 | DRM DRIVERS FOR VC4 |
| 7225 | M: Emma Anholt <emma@anholt.net> | ||
| 7226 | M: Maxime Ripard <mripard@kernel.org> | 7223 | M: Maxime Ripard <mripard@kernel.org> |
| 7227 | S: Supported | 7224 | S: Supported |
| 7228 | T: git git://github.com/anholt/linux | 7225 | T: git git://github.com/anholt/linux |
| @@ -7855,6 +7852,7 @@ R: Yue Hu <huyue2@coolpad.com> · 626 unmodified lines | |||
| 7855 | R: Jeffle Xu <jefflexu@linux.alibaba.com> | 7852 | R: Jeffle Xu <jefflexu@linux.alibaba.com> |
| 7856 | L: linux-erofs@lists.ozlabs.org | 7853 | L: linux-erofs@lists.ozlabs.org |
| 7857 | S: Maintained | 7854 | S: Maintained |
| 7855 | W: https://erofs.docs.kernel.org | ||
| 7858 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs.git | 7856 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs.git |
| 7859 | F: Documentation/ABI/testing/sysfs-fs-erofs | 7857 | F: Documentation/ABI/testing/sysfs-fs-erofs |
| 7860 | F: Documentation/filesystems/erofs.rst | 7858 | F: Documentation/filesystems/erofs.rst |
| @@ -8950,7 +8948,6 @@ S: Maintained · 1089 unmodified lines | |||
| 8950 | F: scripts/get_maintainer.pl | 8948 | F: scripts/get_maintainer.pl |
| 8951 | 8949 | ||
| 8952 | GFS2 FILE SYSTEM | 8950 | GFS2 FILE SYSTEM |
| 8953 | M: Bob Peterson <rpeterso@redhat.com> | ||
| 8954 | M: Andreas Gruenbacher <agruenba@redhat.com> | 8951 | M: Andreas Gruenbacher <agruenba@redhat.com> |
| 8955 | L: gfs2@lists.linux.dev | 8952 | L: gfs2@lists.linux.dev |
| 8956 | S: Supported | 8953 | S: Supported |
| @@ -10395,6 +10392,17 @@ IMGTEC IR DECODER DRIVER · 1438 unmodified lines | |||
| 10395 | S: Orphan | 10392 | S: Orphan |
| 10396 | F: drivers/media/rc/img-ir/ | 10393 | F: drivers/media/rc/img-ir/ |
| 10397 | 10394 | ||
| 10395 | IMGTEC POWERVR DRM DRIVER | ||
| 10396 | M: Frank Binns <frank.binns@imgtec.com> | ||
| 10397 | M: Donald Robson <donald.robson@imgtec.com> | ||
| 10398 | M: Matt Coster <matt.coster@imgtec.com> | ||
| 10399 | S: Supported | ||
| 10400 | T: git git://anongit.freedesktop.org/drm/drm-misc | ||
| 10401 | F: Documentation/devicetree/bindings/gpu/img,powervr.yaml | ||
| 10402 | F: Documentation/gpu/imagination/ | ||
| 10403 | F: drivers/gpu/drm/imagination/ | ||
| 10404 | F: include/uapi/drm/pvr_drm.h | ||
| 10405 | |||
| 10398 | IMON SOUNDGRAPH USB IR RECEIVER | 10406 | IMON SOUNDGRAPH USB IR RECEIVER |
| 10399 | M: Sean Young <sean@mess.org> | 10407 | M: Sean Young <sean@mess.org> |
| 10400 | L: linux-media@vger.kernel.org | 10408 | L: linux-media@vger.kernel.org |
| @@ -10639,9 +10647,9 @@ M: Rodrigo Vivi <rodrigo.vivi@intel.com> · 238 unmodified lines | |||
| 10639 | M: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> | 10647 | M: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> |
| 10640 | L: intel-gfx@lists.freedesktop.org | 10648 | L: intel-gfx@lists.freedesktop.org |
| 10641 | S: Supported | 10649 | S: Supported |
| 10642 | W: https://01.org/linuxgraphics/ | 10650 | W: https://drm.pages.freedesktop.org/intel-docs/ |
| 10643 | Q: http://patchwork.freedesktop.org/project/intel-gfx/ | 10651 | Q: http://patchwork.freedesktop.org/project/intel-gfx/ |
| 10644 | B: https://gitlab.freedesktop.org/drm/intel/-/wikis/How-to-file-i915-bugs | 10652 | B: https://drm.pages.freedesktop.org/intel-docs/how-to-file-i915-bugs.html |
| 10645 | C: irc://irc.oftc.net/intel-gfx | 10653 | C: irc://irc.oftc.net/intel-gfx |
| 10646 | T: git git://anongit.freedesktop.org/drm-intel | 10654 | T: git git://anongit.freedesktop.org/drm-intel |
| 10647 | F: Documentation/ABI/testing/sysfs-driver-intel-i915-hwmon | 10655 | F: Documentation/ABI/testing/sysfs-driver-intel-i915-hwmon |
| @@ -11025,7 +11033,6 @@ F: drivers/net/wireless/intel/iwlwifi/ · 377 unmodified lines | |||
| 11025 | 11033 | ||
| 11026 | INTEL WMI SLIM BOOTLOADER (SBL) FIRMWARE UPDATE DRIVER | 11034 | INTEL WMI SLIM BOOTLOADER (SBL) FIRMWARE UPDATE DRIVER |
| 11027 | M: Jithu Joseph <jithu.joseph@intel.com> | 11035 | M: Jithu Joseph <jithu.joseph@intel.com> |
| 11028 | R: Maurice Ma <maurice.ma@intel.com> | ||
| 11029 | S: Maintained | 11036 | S: Maintained |
| 11030 | W: https://slimbootloader.github.io/security/firmware-update.html | 11037 | W: https://slimbootloader.github.io/security/firmware-update.html |
| 11031 | F: drivers/platform/x86/intel/wmi/sbl-fw-update.c | 11038 | F: drivers/platform/x86/intel/wmi/sbl-fw-update.c |
| @@ -13779,7 +13786,6 @@ F: drivers/net/ethernet/mellanox/mlxfw/ · 2747 unmodified lines | |||
| 13779 | MELLANOX HARDWARE PLATFORM SUPPORT | 13786 | MELLANOX HARDWARE PLATFORM SUPPORT |
| 13780 | M: Hans de Goede <hdegoede@redhat.com> | 13787 | M: Hans de Goede <hdegoede@redhat.com> |
| 13781 | M: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> | 13788 | M: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> |
| 13782 | M: Mark Gross <markgross@kernel.org> | ||
| 13783 | M: Vadim Pasternak <vadimp@nvidia.com> | 13789 | M: Vadim Pasternak <vadimp@nvidia.com> |
| 13784 | L: platform-driver-x86@vger.kernel.org | 13790 | L: platform-driver-x86@vger.kernel.org |
| 13785 | S: Supported | 13791 | S: Supported |
| @@ -14388,7 +14394,6 @@ F: drivers/platform/surface/surface_gpe.c · 602 unmodified lines | |||
| 14388 | MICROSOFT SURFACE HARDWARE PLATFORM SUPPORT | 14394 | MICROSOFT SURFACE HARDWARE PLATFORM SUPPORT |
| 14389 | M: Hans de Goede <hdegoede@redhat.com> | 14395 | M: Hans de Goede <hdegoede@redhat.com> |
| 14390 | M: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> | 14396 | M: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> |
| 14391 | M: Mark Gross <markgross@kernel.org> | ||
| 14392 | M: Maximilian Luz <luzmaximilian@gmail.com> | 14397 | M: Maximilian Luz <luzmaximilian@gmail.com> |
| 14393 | L: platform-driver-x86@vger.kernel.org | 14398 | L: platform-driver-x86@vger.kernel.org |
| 14394 | S: Maintained | 14399 | S: Maintained |
| @@ -14995,6 +15000,7 @@ M: Jakub Kicinski <kuba@kernel.org> · 600 unmodified lines | |||
| 14995 | M: Paolo Abeni <pabeni@redhat.com> | 15000 | M: Paolo Abeni <pabeni@redhat.com> |
| 14996 | L: netdev@vger.kernel.org | 15001 | L: netdev@vger.kernel.org |
| 14997 | S: Maintained | 15002 | S: Maintained |
| 15003 | P: Documentation/process/maintainer-netdev.rst | ||
| 14998 | Q: https://patchwork.kernel.org/project/netdevbpf/list/ | 15004 | Q: https://patchwork.kernel.org/project/netdevbpf/list/ |
| 14999 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git | 15005 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git |
| 15000 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git | 15006 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git |
| @@ -15046,6 +15052,7 @@ M: Jakub Kicinski <kuba@kernel.org> · 45 unmodified lines | |||
| 15046 | M: Paolo Abeni <pabeni@redhat.com> | 15052 | M: Paolo Abeni <pabeni@redhat.com> |
| 15047 | L: netdev@vger.kernel.org | 15053 | L: netdev@vger.kernel.org |
| 15048 | S: Maintained | 15054 | S: Maintained |
| 15055 | P: Documentation/process/maintainer-netdev.rst | ||
| 15049 | Q: https://patchwork.kernel.org/project/netdevbpf/list/ | 15056 | Q: https://patchwork.kernel.org/project/netdevbpf/list/ |
| 15050 | B: mailto:netdev@vger.kernel.org | 15057 | B: mailto:netdev@vger.kernel.org |
| 15051 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git | 15058 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git |
| @@ -15056,6 +15063,7 @@ F: Documentation/networking/ · 4 unmodified lines | |||
| 15056 | F: Documentation/process/maintainer-netdev.rst | 15063 | F: Documentation/process/maintainer-netdev.rst |
| 15057 | F: Documentation/userspace-api/netlink/ | 15064 | F: Documentation/userspace-api/netlink/ |
| 15058 | F: include/linux/in.h | 15065 | F: include/linux/in.h |
| 15066 | F: include/linux/indirect_call_wrapper.h | ||
| 15059 | F: include/linux/net.h | 15067 | F: include/linux/net.h |
| 15060 | F: include/linux/netdevice.h | 15068 | F: include/linux/netdevice.h |
| 15061 | F: include/net/ | 15069 | F: include/net/ |
| @@ -21769,7 +21777,9 @@ F: Documentation/devicetree/bindings/counter/ti-eqep.yaml · 6707 unmodified lines | |||
| 21769 | F: drivers/counter/ti-eqep.c | 21777 | F: drivers/counter/ti-eqep.c |
| 21770 | 21778 | ||
| 21771 | TI ETHERNET SWITCH DRIVER (CPSW) | 21779 | TI ETHERNET SWITCH DRIVER (CPSW) |
| 21772 | R: Grygorii Strashko <grygorii.strashko@ti.com> | 21780 | R: Siddharth Vadapalli <s-vadapalli@ti.com> |
| 21781 | R: Ravi Gunasekaran <r-gunasekaran@ti.com> | ||
| 21782 | R: Roger Quadros <rogerq@kernel.org> | ||
| 21773 | L: linux-omap@vger.kernel.org | 21783 | L: linux-omap@vger.kernel.org |
| 21774 | L: netdev@vger.kernel.org | 21784 | L: netdev@vger.kernel.org |
| 21775 | S: Maintained | 21785 | S: Maintained |
| @@ -21793,6 +21803,15 @@ F: Documentation/devicetree/bindings/media/i2c/ti,ds90* · 17 unmodified lines | |||
| 21793 | F: drivers/media/i2c/ds90* | 21803 | F: drivers/media/i2c/ds90* |
| 21794 | F: include/media/i2c/ds90* | 21804 | F: include/media/i2c/ds90* |
| 21795 | 21805 | ||
| 21806 | TI ICSSG ETHERNET DRIVER (ICSSG) | ||
| 21807 | R: MD Danish Anwar <danishanwar@ti.com> | ||
| 21808 | R: Roger Quadros <rogerq@kernel.org> | ||
| 21809 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) | ||
| 21810 | L: netdev@vger.kernel.org | ||
| 21811 | S: Maintained | ||
| 21812 | F: Documentation/devicetree/bindings/net/ti,icss*.yaml | ||
| 21813 | F: drivers/net/ethernet/ti/icssg/* | ||
| 21814 | |||
| 21796 | TI J721E CSI2RX DRIVER | 21815 | TI J721E CSI2RX DRIVER |
| 21797 | M: Jai Luthra <j-luthra@ti.com> | 21816 | M: Jai Luthra <j-luthra@ti.com> |
| 21798 | L: linux-media@vger.kernel.org | 21817 | L: linux-media@vger.kernel.org |
| @@ -22068,6 +22087,7 @@ F: drivers/watchdog/tqmx86_wdt.c · 269 unmodified lines | |||
| 22068 | TRACING | 22087 | TRACING |
| 22069 | M: Steven Rostedt <rostedt@goodmis.org> | 22088 | M: Steven Rostedt <rostedt@goodmis.org> |
| 22070 | M: Masami Hiramatsu <mhiramat@kernel.org> | 22089 | M: Masami Hiramatsu <mhiramat@kernel.org> |
| 22090 | R: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> | ||
| 22071 | L: linux-kernel@vger.kernel.org | 22091 | L: linux-kernel@vger.kernel.org |
| 22072 | L: linux-trace-kernel@vger.kernel.org | 22092 | L: linux-trace-kernel@vger.kernel.org |
| 22073 | S: Maintained | 22093 | S: Maintained |
| @@ -23654,7 +23674,6 @@ F: drivers/platform/x86/x86-android-tablets/ · 1580 unmodified lines | |||
| 23654 | X86 PLATFORM DRIVERS | 23674 | X86 PLATFORM DRIVERS |
| 23655 | M: Hans de Goede <hdegoede@redhat.com> | 23675 | M: Hans de Goede <hdegoede@redhat.com> |
| 23656 | M: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> | 23676 | M: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> |
| 23657 | M: Mark Gross <markgross@kernel.org> | ||
| 23658 | L: platform-driver-x86@vger.kernel.org | 23677 | L: platform-driver-x86@vger.kernel.org |
| 23659 | S: Maintained | 23678 | S: Maintained |
| 23660 | Q: https://patchwork.kernel.org/project/platform-driver-x86/list/ | 23679 | Q: https://patchwork.kernel.org/project/platform-driver-x86/list/ |
| @@ -23692,6 +23711,20 @@ F: arch/x86/kernel/dumpstack.c · 31 unmodified lines | |||
| 23692 | F: arch/x86/kernel/stacktrace.c | 23711 | F: arch/x86/kernel/stacktrace.c |
| 23693 | F: arch/x86/kernel/unwind_*.c | 23712 | F: arch/x86/kernel/unwind_*.c |
| 23694 | 23713 | ||
| 23714 | X86 TRUST DOMAIN EXTENSIONS (TDX) | ||
| 23715 | M: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> | ||
| 23716 | R: Dave Hansen <dave.hansen@linux.intel.com> | ||
| 23717 | L: x86@kernel.org | ||
| 23718 | L: linux-coco@lists.linux.dev | ||
| 23719 | S: Supported | ||
| 23720 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86/tdx | ||
| 23721 | F: arch/x86/boot/compressed/tdx* | ||
| 23722 | F: arch/x86/coco/tdx/ | ||
| 23723 | F: arch/x86/include/asm/shared/tdx.h | ||
| 23724 | F: arch/x86/include/asm/tdx.h | ||
| 23725 | F: arch/x86/virt/vmx/tdx/ | ||
| 23726 | F: drivers/virt/coco/tdx-guest | ||
| 23727 | |||
| 23695 | X86 VDSO | 23728 | X86 VDSO |
| 23696 | M: Andy Lutomirski <luto@kernel.org> | 23729 | M: Andy Lutomirski <luto@kernel.org> |
| 23697 | L: linux-kernel@vger.kernel.org | 23730 | L: linux-kernel@vger.kernel.org |
| @@ -23872,8 +23905,7 @@ T: git git://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git · 174 unmodified lines | |||
| 23872 | P: Documentation/filesystems/xfs-maintainer-entry-profile.rst | 23905 | P: Documentation/filesystems/xfs-maintainer-entry-profile.rst |
| 23873 | F: Documentation/ABI/testing/sysfs-fs-xfs | 23906 | F: Documentation/ABI/testing/sysfs-fs-xfs |
| 23874 | F: Documentation/admin-guide/xfs.rst | 23907 | F: Documentation/admin-guide/xfs.rst |
| 23875 | F: Documentation/filesystems/xfs-delayed-logging-design.rst | 23908 | F: Documentation/filesystems/xfs-* |
| 23876 | F: Documentation/filesystems/xfs-self-describing-metadata.rst | ||
| 23877 | F: fs/xfs/ | 23909 | F: fs/xfs/ |
| 23878 | F: include/uapi/linux/dqblk_xfs.h | 23910 | F: include/uapi/linux/dqblk_xfs.h |
| 23879 | F: include/uapi/linux/fsmap.h | 23911 | F: include/uapi/linux/fsmap.h |
| @@ -2,7 +2,7 @@ · 1 unmodified line | |||
| 2 | VERSION = 6 | 2 | VERSION = 6 |
| 3 | PATCHLEVEL = 7 | 3 | PATCHLEVEL = 7 |
| 4 | SUBLEVEL = 0 | 4 | SUBLEVEL = 0 |
| 5 | EXTRAVERSION = -rc1 | 5 | EXTRAVERSION = -rc3 |
| 6 | NAME = Hurr durr I'ma ninja sloth | 6 | NAME = Hurr durr I'ma ninja sloth |
| 7 | 7 | ||
| 8 | # *DOCUMENTATION* | 8 | # *DOCUMENTATION* |
| @@ -484,7 +484,8 @@ static int __init xen_guest_init(void) · 483 unmodified lines | |||
| 484 | * for secondary CPUs as they are brought up. | 484 | * for secondary CPUs as they are brought up. |
| 485 | * For uniformity we use VCPUOP_register_vcpu_info even on cpu0. | 485 | * For uniformity we use VCPUOP_register_vcpu_info even on cpu0. |
| 486 | */ | 486 | */ |
| 487 | xen_vcpu_info = alloc_percpu(struct vcpu_info); | 487 | xen_vcpu_info = __alloc_percpu(sizeof(struct vcpu_info), |
| 488 | 1 << fls(sizeof(struct vcpu_info) - 1)); | ||
| 488 | if (xen_vcpu_info == NULL) | 489 | if (xen_vcpu_info == NULL) |
| 489 | return -ENOMEM; | 490 | return -ENOMEM; |
| 490 | 491 | ||
| @@ -158,7 +158,7 @@ endif · 157 unmodified lines | |||
| 158 | 158 | ||
| 159 | all: $(notdir $(KBUILD_IMAGE)) | 159 | all: $(notdir $(KBUILD_IMAGE)) |
| 160 | 160 | ||
| 161 | 161 | vmlinuz.efi: Image | |
| 162 | Image vmlinuz.efi: vmlinux | 162 | Image vmlinuz.efi: vmlinux |
| 163 | $(Q)$(MAKE) $(build)=$(boot) $(boot)/$@ | 163 | $(Q)$(MAKE) $(build)=$(boot) $(boot)/$@ |
| 164 | 164 | ||
| @@ -21,9 +21,22 @@ static inline bool arch_parse_debug_rodata(char *arg) · 20 unmodified lines | |||
| 21 | extern bool rodata_enabled; | 21 | extern bool rodata_enabled; |
| 22 | extern bool rodata_full; | 22 | extern bool rodata_full; |
| 23 | 23 | ||
| 24 | if (arg && !strcmp(arg, "full")) { | 24 | if (!arg) |
| 25 | return false; | ||
| 26 | |||
| 27 | if (!strcmp(arg, "full")) { | ||
| 28 | rodata_enabled = rodata_full = true; | ||
| 29 | return true; | ||
| 30 | } | ||
| 31 | |||
| 32 | if (!strcmp(arg, "off")) { | ||
| 33 | rodata_enabled = rodata_full = false; | ||
| 34 | return true; | ||
| 35 | } | ||
| 36 | |||
| 37 | if (!strcmp(arg, "on")) { | ||
| 25 | rodata_enabled = true; | 38 | rodata_enabled = true; |
| 26 | rodata_full = true; | 39 | rodata_full = false; |
| 27 | return true; | 40 | return true; |
| 28 | } | 41 | } |
| 29 | 42 | ||
| @@ -29,8 +29,8 @@ bool can_set_direct_map(void) · 28 unmodified lines | |||
| 29 | * | 29 | * |
| 30 | * KFENCE pool requires page-granular mapping if initialized late. | 30 | * KFENCE pool requires page-granular mapping if initialized late. |
| 31 | */ | 31 | */ |
| 32 | return (rodata_enabled && rodata_full) || debug_pagealloc_enabled() || | 32 | return rodata_full || debug_pagealloc_enabled() || |
| 33 | arm64_kfence_can_set_direct_map(); | 33 | arm64_kfence_can_set_direct_map(); |
| 34 | } | 34 | } |
| 35 | 35 | ||
| 36 | static int change_page_range(pte_t *ptep, unsigned long addr, void *data) | 36 | static int change_page_range(pte_t *ptep, unsigned long addr, void *data) |
| @@ -105,8 +105,7 @@ static int change_memory_common(unsigned long addr, int numpages, · 68 unmodified lines | |||
| 105 | * If we are manipulating read-only permissions, apply the same | 105 | * If we are manipulating read-only permissions, apply the same |
| 106 | * change to the linear mapping of the pages that back this VM area. | 106 | * change to the linear mapping of the pages that back this VM area. |
| 107 | */ | 107 | */ |
| 108 | if (rodata_enabled && | 108 | if (rodata_full && (pgprot_val(set_mask) == PTE_RDONLY || |
| 109 | rodata_full && (pgprot_val(set_mask) == PTE_RDONLY || | ||
| 110 | pgprot_val(clear_mask) == PTE_RDONLY)) { | 109 | pgprot_val(clear_mask) == PTE_RDONLY)) { |
| 111 | for (i = 0; i < area->nr_pages; i++) { | 110 | for (i = 0; i < area->nr_pages; i++) { |
| 112 | __change_memory_common((u64)page_address(area->pages[i]), | 111 | __change_memory_common((u64)page_address(area->pages[i]), |
| @@ -68,6 +68,7 @@ LDFLAGS_vmlinux += -static -n -nostdlib · 67 unmodified lines | |||
| 68 | ifdef CONFIG_AS_HAS_EXPLICIT_RELOCS | 68 | ifdef CONFIG_AS_HAS_EXPLICIT_RELOCS |
| 69 | cflags-y += $(call cc-option,-mexplicit-relocs) | 69 | cflags-y += $(call cc-option,-mexplicit-relocs) |
| 70 | KBUILD_CFLAGS_KERNEL += $(call cc-option,-mdirect-extern-access) | 70 | KBUILD_CFLAGS_KERNEL += $(call cc-option,-mdirect-extern-access) |
| 71 | KBUILD_CFLAGS_KERNEL += $(call cc-option,-fdirect-access-external-data) | ||
| 71 | KBUILD_AFLAGS_MODULE += $(call cc-option,-fno-direct-access-external-data) | 72 | KBUILD_AFLAGS_MODULE += $(call cc-option,-fno-direct-access-external-data) |
| 72 | KBUILD_CFLAGS_MODULE += $(call cc-option,-fno-direct-access-external-data) | 73 | KBUILD_CFLAGS_MODULE += $(call cc-option,-fno-direct-access-external-data) |
| 73 | KBUILD_AFLAGS_MODULE += $(call cc-option,-mno-relax) $(call cc-option,-Wa$(comma)-mno-relax) | 74 | KBUILD_AFLAGS_MODULE += $(call cc-option,-mno-relax) $(call cc-option,-Wa$(comma)-mno-relax) |
| @@ -142,6 +143,8 @@ vdso-install-y += arch/loongarch/vdso/vdso.so.dbg · 68 unmodified lines | |||
| 142 | 143 | ||
| 143 | all: $(notdir $(KBUILD_IMAGE)) | 144 | all: $(notdir $(KBUILD_IMAGE)) |
| 144 | 145 | ||
| 146 | vmlinuz.efi: vmlinux.efi | ||
| 147 | |||
| 145 | vmlinux.elf vmlinux.efi vmlinuz.efi: vmlinux | 148 | vmlinux.elf vmlinux.efi vmlinuz.efi: vmlinux |
| 146 | $(Q)$(MAKE) $(build)=$(boot) $(bootvars-y) $(boot)/$@ | 149 | $(Q)$(MAKE) $(build)=$(boot) $(bootvars-y) $(boot)/$@ |
| 147 | 150 | ||
| @@ -609,8 +609,7 @@ · 608 unmodified lines | |||
| 609 | lu32i.d \reg, 0 | 609 | lu32i.d \reg, 0 |
| 610 | lu52i.d \reg, \reg, 0 | 610 | lu52i.d \reg, \reg, 0 |
| 611 | .pushsection ".la_abs", "aw", %progbits | 611 | .pushsection ".la_abs", "aw", %progbits |
| 612 | 768: | 612 | .dword 766b |
| 613 | .dword 768b-766b | ||
| 614 | .dword \sym | 613 | .dword \sym |
| 615 | .popsection | 614 | .popsection |
| 616 | #endif | 615 | #endif |
| @@ -40,13 +40,13 @@ static __always_inline unsigned long __percpu_##op(void *ptr, \ · 39 unmodified lines | |||
| 40 | switch (size) { \ | 40 | switch (size) { \ |
| 41 | case 4: \ | 41 | case 4: \ |
| 42 | __asm__ __volatile__( \ | 42 | __asm__ __volatile__( \ |
| 43 | "am"#asm_op".w" " %[ret], %[val], %[ptr] \n" \ | 43 | "am"#asm_op".w" " %[ret], %[val], %[ptr] \n" \ |
| 44 | : [ret] "=&r" (ret), [ptr] "+ZB"(*(u32 *)ptr) \ | 44 | : [ret] "=&r" (ret), [ptr] "+ZB"(*(u32 *)ptr) \ |
| 45 | : [val] "r" (val)); \ | 45 | : [val] "r" (val)); \ |
| 46 | break; \ | 46 | break; \ |
| 47 | case 8: \ | 47 | case 8: \ |
| 48 | __asm__ __volatile__( \ | 48 | __asm__ __volatile__( \ |
| 49 | "am"#asm_op".d" " %[ret], %[val], %[ptr] \n" \ | 49 | "am"#asm_op".d" " %[ret], %[val], %[ptr] \n" \ |
| 50 | : [ret] "=&r" (ret), [ptr] "+ZB"(*(u64 *)ptr) \ | 50 | : [ret] "=&r" (ret), [ptr] "+ZB"(*(u64 *)ptr) \ |
| 51 | : [val] "r" (val)); \ | 51 | : [val] "r" (val)); \ |
| 52 | break; \ | 52 | break; \ |
| @@ -63,7 +63,7 @@ PERCPU_OP(and, and, &) · 10 unmodified lines | |||
| 63 | PERCPU_OP(or, or, |) | 63 | PERCPU_OP(or, or, |) |
| 64 | #undef PERCPU_OP | 64 | #undef PERCPU_OP |
| 65 | 65 | ||
| 66 | static __always_inline unsigned long __percpu_read(void *ptr, int size) | 66 | static __always_inline unsigned long __percpu_read(void __percpu *ptr, int size) |
| 67 | { | 67 | { |
| 68 | unsigned long ret; | 68 | unsigned long ret; |
| 69 | 69 | ||
| @@ -100,7 +100,7 @@ static __always_inline unsigned long __percpu_read(void *ptr, int size) · 30 unmodified lines | |||
| 100 | return ret; | 100 | return ret; |
| 101 | } | 101 | } |
| 102 | 102 | ||
| 103 | static __always_inline void __percpu_write(void *ptr, unsigned long val, int size) | 103 | static __always_inline void __percpu_write(void __percpu *ptr, unsigned long val, int size) |
| 104 | { | 104 | { |
| 105 | switch (size) { | 105 | switch (size) { |
| 106 | case 1: | 106 | case 1: |
| @@ -132,8 +132,7 @@ static __always_inline void __percpu_write(void *ptr, unsigned long val, int siz · 25 unmodified lines | |||
| 132 | } | 132 | } |
| 133 | } | 133 | } |
| 134 | 134 | ||
| 135 | static __always_inline unsigned long __percpu_xchg(void *ptr, unsigned long val, | 135 | static __always_inline unsigned long __percpu_xchg(void *ptr, unsigned long val, int size) |
| 136 | int size) | ||
| 137 | { | 136 | { |
| 138 | switch (size) { | 137 | switch (size) { |
| 139 | case 1: | 138 | case 1: |
| @@ -25,7 +25,7 @@ extern void set_merr_handler(unsigned long offset, void *addr, unsigned long len · 24 unmodified lines | |||
| 25 | #ifdef CONFIG_RELOCATABLE | 25 | #ifdef CONFIG_RELOCATABLE |
| 26 | 26 | ||
| 27 | struct rela_la_abs { | 27 | struct rela_la_abs { |
| 28 | long offset; | 28 | long pc; |
| 29 | long symvalue; | 29 | long symvalue; |
| 30 | }; | 30 | }; |
| 31 | 31 | ||
| @@ -52,7 +52,7 @@ static inline void __init relocate_absolute(long random_offset) · 51 unmodified lines | |||
| 52 | for (p = begin; (void *)p < end; p++) { | 52 | for (p = begin; (void *)p < end; p++) { |
| 53 | long v = p->symvalue; | 53 | long v = p->symvalue; |
| 54 | uint32_t lu12iw, ori, lu32id, lu52id; | 54 | uint32_t lu12iw, ori, lu32id, lu52id; |
| 55 | union loongarch_instruction *insn = (void *)p - p->offset; | 55 | union loongarch_instruction *insn = (void *)p->pc; |
| 56 | 56 | ||
| 57 | lu12iw = (v >> 12) & 0xfffff; | 57 | lu12iw = (v >> 12) & 0xfffff; |
| 58 | ori = v & 0xfff; | 58 | ori = v & 0xfff; |
| @@ -102,6 +102,14 @@ static inline __init unsigned long get_random_boot(void) · 43 unmodified lines | |||
| 102 | return hash; | 102 | return hash; |
| 103 | } | 103 | } |
| 104 | 104 | ||
| 105 | static int __init nokaslr(char *p) | ||
| 106 | { | ||
| 107 | pr_info("KASLR is disabled.\n"); | ||
| 108 | |||
| 109 | return 0; /* Print a notice and silence the boot warning */ | ||
| 110 | } | ||
| 111 | early_param("nokaslr", nokaslr); | ||
| 112 | |||
| 105 | static inline __init bool kaslr_disabled(void) | 113 | static inline __init bool kaslr_disabled(void) |
| 106 | { | 114 | { |
| 107 | char *str; | 115 | char *str; |
| @@ -58,14 +58,16 @@ static int constant_set_state_oneshot(struct clock_event_device *evt) · 57 unmodified lines | |||
| 58 | return 0; | 58 | return 0; |
| 59 | } | 59 | } |
| 60 | 60 | ||
| 61 | static int constant_set_state_oneshot_stopped(struct clock_event_device *evt) | 61 | static int constant_set_state_periodic(struct clock_event_device *evt) |
| 62 | { | 62 | { |
| 63 | unsigned long period; | ||
| 63 | unsigned long timer_config; | 64 | unsigned long timer_config; |
| 64 | 65 | ||
| 65 | raw_spin_lock(&state_lock); | 66 | raw_spin_lock(&state_lock); |
| 66 | 67 | ||
| 67 | timer_config = csr_read64(LOONGARCH_CSR_TCFG); | 68 | period = const_clock_freq / HZ; |
| 68 | timer_config &= ~CSR_TCFG_EN; | 69 | timer_config = period & CSR_TCFG_VAL; |
| 70 | timer_config |= (CSR_TCFG_PERIOD | CSR_TCFG_EN); | ||
| 69 | csr_write64(timer_config, LOONGARCH_CSR_TCFG); | 71 | csr_write64(timer_config, LOONGARCH_CSR_TCFG); |
| 70 | 72 | ||
| 71 | raw_spin_unlock(&state_lock); | 73 | raw_spin_unlock(&state_lock); |
| @@ -73,16 +75,14 @@ static int constant_set_state_oneshot_stopped(struct clock_event_device *evt) · 1 unmodified line | |||
| 73 | return 0; | 75 | return 0; |
| 74 | } | 76 | } |
| 75 | 77 | ||
| 76 | static int constant_set_state_periodic(struct clock_event_device *evt) | 78 | static int constant_set_state_shutdown(struct clock_event_device *evt) |
| 77 | { | 79 | { |
| 78 | unsigned long period; | ||
| 79 | unsigned long timer_config; | 80 | unsigned long timer_config; |
| 80 | 81 | ||
| 81 | raw_spin_lock(&state_lock); | 82 | raw_spin_lock(&state_lock); |
| 82 | 83 | ||
| 83 | period = const_clock_freq / HZ; | 84 | timer_config = csr_read64(LOONGARCH_CSR_TCFG); |
| 84 | timer_config = period & CSR_TCFG_VAL; | 85 | timer_config &= ~CSR_TCFG_EN; |
| 85 | timer_config |= (CSR_TCFG_PERIOD | CSR_TCFG_EN); | ||
| 86 | csr_write64(timer_config, LOONGARCH_CSR_TCFG); | 86 | csr_write64(timer_config, LOONGARCH_CSR_TCFG); |
| 87 | 87 | ||
| 88 | raw_spin_unlock(&state_lock); | 88 | raw_spin_unlock(&state_lock); |
| @@ -90,11 +90,6 @@ static int constant_set_state_periodic(struct clock_event_device *evt) · 1 unmodified line | |||
| 90 | return 0; | 90 | return 0; |
| 91 | } | 91 | } |
| 92 | 92 | ||
| 93 | static int constant_set_state_shutdown(struct clock_event_device *evt) | ||
| 94 | { | ||
| 95 | return 0; | ||
| 96 | } | ||
| 97 | |||
| 98 | static int constant_timer_next_event(unsigned long delta, struct clock_event_device *evt) | 93 | static int constant_timer_next_event(unsigned long delta, struct clock_event_device *evt) |
| 99 | { | 94 | { |
| 100 | unsigned long timer_config; | 95 | unsigned long timer_config; |
| @@ -161,7 +156,7 @@ int constant_clockevent_init(void) · 60 unmodified lines | |||
| 161 | cd->rating = 320; | 156 | cd->rating = 320; |
| 162 | cd->cpumask = cpumask_of(cpu); | 157 | cd->cpumask = cpumask_of(cpu); |
| 163 | cd->set_state_oneshot = constant_set_state_oneshot; | 158 | cd->set_state_oneshot = constant_set_state_oneshot; |
| 164 | cd->set_state_oneshot_stopped = constant_set_state_oneshot_stopped; | 159 | cd->set_state_oneshot_stopped = constant_set_state_shutdown; |
| 165 | cd->set_state_periodic = constant_set_state_periodic; | 160 | cd->set_state_periodic = constant_set_state_periodic; |
| 166 | cd->set_state_shutdown = constant_set_state_shutdown; | 161 | cd->set_state_shutdown = constant_set_state_shutdown; |
| 167 | cd->set_next_event = constant_timer_next_event; | 162 | cd->set_next_event = constant_timer_next_event; |
| @@ -13,13 +13,13 @@ struct page *dmw_virt_to_page(unsigned long kaddr) · 12 unmodified lines | |||
| 13 | { | 13 | { |
| 14 | return pfn_to_page(virt_to_pfn(kaddr)); | 14 | return pfn_to_page(virt_to_pfn(kaddr)); |
| 15 | } | 15 | } |
| 16 | EXPORT_SYMBOL_GPL(dmw_virt_to_page); | 16 | EXPORT_SYMBOL(dmw_virt_to_page); |
| 17 | 17 | ||
| 18 | struct page *tlb_virt_to_page(unsigned long kaddr) | 18 | struct page *tlb_virt_to_page(unsigned long kaddr) |
| 19 | { | 19 | { |
| 20 | return pfn_to_page(pte_pfn(*virt_to_kpte(kaddr))); | 20 | return pfn_to_page(pte_pfn(*virt_to_kpte(kaddr))); |
| 21 | } | 21 | } |
| 22 | EXPORT_SYMBOL_GPL(tlb_virt_to_page); | 22 | EXPORT_SYMBOL(tlb_virt_to_page); |
| 23 | 23 | ||
| 24 | pgd_t *pgd_alloc(struct mm_struct *mm) | 24 | pgd_t *pgd_alloc(struct mm_struct *mm) |
| 25 | { | 25 | { |
| @@ -115,9 +115,12 @@ config ARCH_HAS_ILOG2_U64 · 114 unmodified lines | |||
| 115 | default n | 115 | default n |
| 116 | 116 | ||
| 117 | config GENERIC_BUG | 117 | config GENERIC_BUG |
| 118 | bool | 118 | def_bool y |
| 119 | default y | ||
| 120 | depends on BUG | 119 | depends on BUG |
| 120 | select GENERIC_BUG_RELATIVE_POINTERS if 64BIT | ||
| 121 | |||
| 122 | config GENERIC_BUG_RELATIVE_POINTERS | ||
| 123 | bool | ||
| 121 | 124 | ||
| 122 | config GENERIC_HWEIGHT | 125 | config GENERIC_HWEIGHT |
| 123 | bool | 126 | bool |
| @@ -140,11 +143,11 @@ config ARCH_MMAP_RND_COMPAT_BITS_MIN · 16 unmodified lines | |||
| 140 | default 8 | 143 | default 8 |
| 141 | 144 | ||
| 142 | config ARCH_MMAP_RND_BITS_MAX | 145 | config ARCH_MMAP_RND_BITS_MAX |
| 143 | default 24 if 64BIT | 146 | default 18 if 64BIT |
| 144 | default 17 | 147 | default 13 |
| 145 | 148 | ||
| 146 | config ARCH_MMAP_RND_COMPAT_BITS_MAX | 149 | config ARCH_MMAP_RND_COMPAT_BITS_MAX |
| 147 | default 17 | 150 | default 13 |
| 148 | 151 | ||
| 149 | # unless you want to implement ACPI on PA-RISC ... ;-) | 152 | # unless you want to implement ACPI on PA-RISC ... ;-) |
| 150 | config PM | 153 | config PM |
| @@ -34,7 +34,8 @@ void apply_alternatives(struct alt_instr *start, struct alt_instr *end, · 33 unmodified lines | |||
| 34 | 34 | ||
| 35 | /* Alternative SMP implementation. */ | 35 | /* Alternative SMP implementation. */ |
| 36 | #define ALTERNATIVE(cond, replacement) "!0:" \ | 36 | #define ALTERNATIVE(cond, replacement) "!0:" \ |
| 37 | ".section .altinstructions, \"aw\" !" \ | 37 | ".section .altinstructions, \"a\" !" \ |
| 38 | ".align 4 !" \ | ||
| 38 | ".word (0b-4-.) !" \ | 39 | ".word (0b-4-.) !" \ |
| 39 | ".hword 1, " __stringify(cond) " !" \ | 40 | ".hword 1, " __stringify(cond) " !" \ |
| 40 | ".word " __stringify(replacement) " !" \ | 41 | ".word " __stringify(replacement) " !" \ |
| @@ -44,7 +45,8 @@ void apply_alternatives(struct alt_instr *start, struct alt_instr *end, · 3 unmodified lines | |||
| 44 | 45 | ||
| 45 | /* to replace one single instructions by a new instruction */ | 46 | /* to replace one single instructions by a new instruction */ |
| 46 | #define ALTERNATIVE(from, to, cond, replacement)\ | 47 | #define ALTERNATIVE(from, to, cond, replacement)\ |
| 47 | .section .altinstructions, "aw" ! \ | 48 | .section .altinstructions, "a" ! \ |
| 49 | .align 4 ! \ | ||
| 48 | .word (from - .) ! \ | 50 | .word (from - .) ! \ |
| 49 | .hword (to - from)/4, cond ! \ | 51 | .hword (to - from)/4, cond ! \ |
| 50 | .word replacement ! \ | 52 | .word replacement ! \ |
| @@ -52,7 +54,8 @@ void apply_alternatives(struct alt_instr *start, struct alt_instr *end, · 1 unmodified line | |||
| 52 | 54 | ||
| 53 | /* to replace multiple instructions by new code */ | 55 | /* to replace multiple instructions by new code */ |
| 54 | #define ALTERNATIVE_CODE(from, num_instructions, cond, new_instr_ptr)\ | 56 | #define ALTERNATIVE_CODE(from, num_instructions, cond, new_instr_ptr)\ |
| 55 | .section .altinstructions, "aw" ! \ | 57 | .section .altinstructions, "a" ! \ |
| 58 | .align 4 ! \ | ||
| 56 | .word (from - .) ! \ | 59 | .word (from - .) ! \ |
| 57 | .hword -num_instructions, cond ! \ | 60 | .hword -num_instructions, cond ! \ |
| 58 | .word (new_instr_ptr - .) ! \ | 61 | .word (new_instr_ptr - .) ! \ |
| @@ -574,6 +574,7 @@ · 573 unmodified lines | |||
| 574 | */ | 574 | */ |
| 575 | #define ASM_EXCEPTIONTABLE_ENTRY(fault_addr, except_addr) \ | 575 | #define ASM_EXCEPTIONTABLE_ENTRY(fault_addr, except_addr) \ |
| 576 | .section __ex_table,"aw" ! \ | 576 | .section __ex_table,"aw" ! \ |
| 577 | .align 4 ! \ | ||
| 577 | .word (fault_addr - .), (except_addr - .) ! \ | 578 | .word (fault_addr - .), (except_addr - .) ! \ |
| 578 | .previous | 579 | .previous |
| 579 | 580 | ||
| @@ -17,24 +17,27 @@ · 16 unmodified lines | |||
| 17 | #define PARISC_BUG_BREAK_ASM "break 0x1f, 0x1fff" | 17 | #define PARISC_BUG_BREAK_ASM "break 0x1f, 0x1fff" |
| 18 | #define PARISC_BUG_BREAK_INSN 0x03ffe01f /* PARISC_BUG_BREAK_ASM */ | 18 | #define PARISC_BUG_BREAK_INSN 0x03ffe01f /* PARISC_BUG_BREAK_ASM */ |
| 19 | 19 | ||
| 20 | #if defined(CONFIG_64BIT) | 20 | #ifdef CONFIG_GENERIC_BUG_RELATIVE_POINTERS |
| 21 | #define ASM_WORD_INSN ".dword\t" | 21 | # define __BUG_REL(val) ".word " __stringify(val) " - ." |
| 22 | #else | 22 | #else |
| 23 | #define ASM_WORD_INSN ".word\t" | 23 | # define __BUG_REL(val) ".word " __stringify(val) |
| 24 | #endif | 24 | #endif |
| 25 | 25 | ||
| 26 | |||
| 26 | #ifdef CONFIG_DEBUG_BUGVERBOSE | 27 | #ifdef CONFIG_DEBUG_BUGVERBOSE |
| 27 | #define BUG() \ | 28 | #define BUG() \ |
| 28 | do { \ | 29 | do { \ |
| 29 | asm volatile("\n" \ | 30 | asm volatile("\n" \ |
| 30 | "1:\t" PARISC_BUG_BREAK_ASM "\n" \ | 31 | "1:\t" PARISC_BUG_BREAK_ASM "\n" \ |
| 31 | "\t.pushsection __bug_table,\"aw\"\n" \ | 32 | "\t.pushsection __bug_table,\"a\"\n" \ |
| 32 | "2:\t" ASM_WORD_INSN "1b, %c0\n" \ | 33 | "\t.align 4\n" \ |
| 33 | "\t.short %c1, %c2\n" \ | 34 | "2:\t" __BUG_REL(1b) "\n" \ |
| 34 | "\t.org 2b+%c3\n" \ | 35 | "\t" __BUG_REL(%c0) "\n" \ |
| 36 | "\t.short %1, %2\n" \ | ||
| 37 | "\t.blockz %3-2*4-2*2\n" \ | ||
| 35 | "\t.popsection" \ | 38 | "\t.popsection" \ |
| 36 | : : "i" (__FILE__), "i" (__LINE__), \ | 39 | : : "i" (__FILE__), "i" (__LINE__), \ |
| 37 | "i" (0), "i" (sizeof(struct bug_entry)) ); \ | 40 | "i" (0), "i" (sizeof(struct bug_entry)) ); \ |
| 38 | unreachable(); \ | 41 | unreachable(); \ |
| 39 | } while(0) | 42 | } while(0) |
| 40 | 43 | ||
| @@ -51,10 +54,12 @@ · 10 unmodified lines | |||
| 51 | do { \ | 54 | do { \ |
| 52 | asm volatile("\n" \ | 55 | asm volatile("\n" \ |
| 53 | "1:\t" PARISC_BUG_BREAK_ASM "\n" \ | 56 | "1:\t" PARISC_BUG_BREAK_ASM "\n" \ |
| 54 | "\t.pushsection __bug_table,\"aw\"\n" \ | 57 | "\t.pushsection __bug_table,\"a\"\n" \ |
| 55 | "2:\t" ASM_WORD_INSN "1b, %c0\n" \ | 58 | "\t.align 4\n" \ |
| 56 | "\t.short %c1, %c2\n" \ | 59 | "2:\t" __BUG_REL(1b) "\n" \ |
| 57 | "\t.org 2b+%c3\n" \ | 60 | "\t" __BUG_REL(%c0) "\n" \ |
| 61 | "\t.short %1, %2\n" \ | ||
| 62 | "\t.blockz %3-2*4-2*2\n" \ | ||
| 58 | "\t.popsection" \ | 63 | "\t.popsection" \ |
| 59 | : : "i" (__FILE__), "i" (__LINE__), \ | 64 | : : "i" (__FILE__), "i" (__LINE__), \ |
| 60 | "i" (BUGFLAG_WARNING|(flags)), \ | 65 | "i" (BUGFLAG_WARNING|(flags)), \ |
| @@ -65,10 +70,11 @@ · 4 unmodified lines | |||
| 65 | do { \ | 70 | do { \ |
| 66 | asm volatile("\n" \ | 71 | asm volatile("\n" \ |
| 67 | "1:\t" PARISC_BUG_BREAK_ASM "\n" \ | 72 | "1:\t" PARISC_BUG_BREAK_ASM "\n" \ |
| 68 | "\t.pushsection __bug_table,\"aw\"\n" \ | 73 | "\t.pushsection __bug_table,\"a\"\n" \ |
| 69 | "2:\t" ASM_WORD_INSN "1b\n" \ | 74 | "\t.align %2\n" \ |
| 70 | "\t.short %c0\n" \ | 75 | "2:\t" __BUG_REL(1b) "\n" \ |
| 71 | "\t.org 2b+%c1\n" \ | 76 | "\t.short %0\n" \ |
| 77 | "\t.blockz %1-4-2\n" \ | ||
| 72 | "\t.popsection" \ | 78 | "\t.popsection" \ |
| 73 | : : "i" (BUGFLAG_WARNING|(flags)), \ | 79 | : : "i" (BUGFLAG_WARNING|(flags)), \ |
| 74 | "i" (sizeof(struct bug_entry)) ); \ | 80 | "i" (sizeof(struct bug_entry)) ); \ |
| @@ -349,15 +349,7 @@ struct pt_regs; /* forward declaration... */ · 348 unmodified lines | |||
| 349 | 349 | ||
| 350 | #define ELF_HWCAP 0 | 350 | #define ELF_HWCAP 0 |
| 351 | 351 | ||
| 352 | /* Masks for stack and mmap randomization */ | 352 | #define STACK_RND_MASK 0x7ff /* 8MB of VA */ |
| 353 | #define BRK_RND_MASK (is_32bit_task() ? 0x07ffUL : 0x3ffffUL) | ||
| 354 | #define MMAP_RND_MASK (is_32bit_task() ? 0x1fffUL : 0x3ffffUL) | ||
| 355 | #define STACK_RND_MASK MMAP_RND_MASK | ||
| 356 | |||
| 357 | struct mm_struct; | ||
| 358 | extern unsigned long arch_randomize_brk(struct mm_struct *); | ||
| 359 | #define arch_randomize_brk arch_randomize_brk | ||
| 360 | |||
| 361 | 353 | ||
| 362 | #define ARCH_HAS_SETUP_ADDITIONAL_PAGES 1 | 354 | #define ARCH_HAS_SETUP_ADDITIONAL_PAGES 1 |
| 363 | struct linux_binprm; | 355 | struct linux_binprm; |
| @@ -15,10 +15,12 @@ static __always_inline bool arch_static_branch(struct static_key *key, bool bran · 14 unmodified lines | |||
| 15 | asm_volatile_goto("1:\n\t" | 15 | asm_volatile_goto("1:\n\t" |
| 16 | "nop\n\t" | 16 | "nop\n\t" |
| 17 | ".pushsection __jump_table, \"aw\"\n\t" | 17 | ".pushsection __jump_table, \"aw\"\n\t" |
| 18 | ".align %1\n\t" | ||
| 18 | ".word 1b - ., %l[l_yes] - .\n\t" | 19 | ".word 1b - ., %l[l_yes] - .\n\t" |
| 19 | __stringify(ASM_ULONG_INSN) " %c0 - .\n\t" | 20 | __stringify(ASM_ULONG_INSN) " %c0 - .\n\t" |
| 20 | ".popsection\n\t" | 21 | ".popsection\n\t" |
| 21 | : : "i" (&((char *)key)[branch]) : : l_yes); | 22 | : : "i" (&((char *)key)[branch]), "i" (sizeof(long)) |
| 23 | : : l_yes); | ||
| 22 | 24 | ||
| 23 | return false; | 25 | return false; |
| 24 | l_yes: | 26 | l_yes: |
| @@ -30,10 +32,12 @@ static __always_inline bool arch_static_branch_jump(struct static_key *key, bool · 5 unmodified lines | |||
| 30 | asm_volatile_goto("1:\n\t" | 32 | asm_volatile_goto("1:\n\t" |
| 31 | "b,n %l[l_yes]\n\t" | 33 | "b,n %l[l_yes]\n\t" |
| 32 | ".pushsection __jump_table, \"aw\"\n\t" | 34 | ".pushsection __jump_table, \"aw\"\n\t" |
| 35 | ".align %1\n\t" | ||
| 33 | ".word 1b - ., %l[l_yes] - .\n\t" | 36 | ".word 1b - ., %l[l_yes] - .\n\t" |
| 34 | __stringify(ASM_ULONG_INSN) " %c0 - .\n\t" | 37 | __stringify(ASM_ULONG_INSN) " %c0 - .\n\t" |
| 35 | ".popsection\n\t" | 38 | ".popsection\n\t" |
| 36 | : : "i" (&((char *)key)[branch]) : : l_yes); | 39 | : : "i" (&((char *)key)[branch]), "i" (sizeof(long)) |
| 40 | : : l_yes); | ||
| 37 | 41 | ||
| 38 | return false; | 42 | return false; |
| 39 | l_yes: | 43 | l_yes: |
| @@ -55,7 +55,7 @@ · 54 unmodified lines | |||
| 55 | }) | 55 | }) |
| 56 | 56 | ||
| 57 | #ifdef CONFIG_SMP | 57 | #ifdef CONFIG_SMP |
| 58 | # define __lock_aligned __section(".data..lock_aligned") | 58 | # define __lock_aligned __section(".data..lock_aligned") __aligned(16) |
| 59 | #endif | 59 | #endif |
| 60 | 60 | ||
| 61 | #endif /* __PARISC_LDCW_H */ | 61 | #endif /* __PARISC_LDCW_H */ |
| @@ -47,6 +47,8 @@ · 46 unmodified lines | |||
| 47 | 47 | ||
| 48 | #ifndef __ASSEMBLY__ | 48 | #ifndef __ASSEMBLY__ |
| 49 | 49 | ||
| 50 | struct rlimit; | ||
| 51 | unsigned long mmap_upper_limit(struct rlimit *rlim_stack); | ||
| 50 | unsigned long calc_max_stack_size(unsigned long stack_max); | 52 | unsigned long calc_max_stack_size(unsigned long stack_max); |
| 51 | 53 | ||
| 52 | /* | 54 | /* |
| @@ -41,6 +41,7 @@ struct exception_table_entry { · 40 unmodified lines | |||
| 41 | 41 | ||
| 42 | #define ASM_EXCEPTIONTABLE_ENTRY( fault_addr, except_addr )\ | 42 | #define ASM_EXCEPTIONTABLE_ENTRY( fault_addr, except_addr )\ |
| 43 | ".section __ex_table,\"aw\"\n" \ | 43 | ".section __ex_table,\"aw\"\n" \ |
| 44 | ".align 4\n" \ | ||
| 44 | ".word (" #fault_addr " - .), (" #except_addr " - .)\n\t" \ | 45 | ".word (" #fault_addr " - .), (" #except_addr " - .)\n\t" \ |
| 45 | ".previous\n" | 46 | ".previous\n" |
| 46 | 47 | ||
| @@ -75,7 +75,6 @@ · 74 unmodified lines | |||
| 75 | 75 | ||
| 76 | /* We now return you to your regularly scheduled HPUX. */ | 76 | /* We now return you to your regularly scheduled HPUX. */ |
| 77 | 77 | ||
| 78 | #define ENOSYM 215 /* symbol does not exist in executable */ | ||
| 79 | #define ENOTSOCK 216 /* Socket operation on non-socket */ | 78 | #define ENOTSOCK 216 /* Socket operation on non-socket */ |
| 80 | #define EDESTADDRREQ 217 /* Destination address required */ | 79 | #define EDESTADDRREQ 217 /* Destination address required */ |
| 81 | #define EMSGSIZE 218 /* Message too long */ | 80 | #define EMSGSIZE 218 /* Message too long */ |
| @@ -101,7 +100,6 @@ · 19 unmodified lines | |||
| 101 | #define ETIMEDOUT 238 /* Connection timed out */ | 100 | #define ETIMEDOUT 238 /* Connection timed out */ |
| 102 | #define ECONNREFUSED 239 /* Connection refused */ | 101 | #define ECONNREFUSED 239 /* Connection refused */ |
| 103 | #define EREFUSED ECONNREFUSED /* for HP's NFS apparently */ | 102 | #define EREFUSED ECONNREFUSED /* for HP's NFS apparently */ |
| 104 | #define EREMOTERELEASE 240 /* Remote peer released connection */ | ||
| 105 | #define EHOSTDOWN 241 /* Host is down */ | 103 | #define EHOSTDOWN 241 /* Host is down */ |
| 106 | #define EHOSTUNREACH 242 /* No route to host */ | 104 | #define EHOSTUNREACH 242 /* No route to host */ |
| 107 | 105 | ||
| @@ -383,7 +383,7 @@ show_cpuinfo (struct seq_file *m, void *v) · 382 unmodified lines | |||
| 383 | char cpu_name[60], *p; | 383 | char cpu_name[60], *p; |
| 384 | 384 | ||
| 385 | /* strip PA path from CPU name to not confuse lscpu */ | 385 | /* strip PA path from CPU name to not confuse lscpu */ |
| 386 | strlcpy(cpu_name, per_cpu(cpu_data, 0).dev->name, sizeof(cpu_name)); | 386 | strscpy(cpu_name, per_cpu(cpu_data, 0).dev->name, sizeof(cpu_name)); |
| 387 | p = strrchr(cpu_name, '['); | 387 | p = strrchr(cpu_name, '['); |
| 388 | if (p) | 388 | if (p) |
| 389 | *(--p) = 0; | 389 | *(--p) = 0; |
| @@ -77,7 +77,7 @@ unsigned long calc_max_stack_size(unsigned long stack_max) · 76 unmodified lines | |||
| 77 | * indicating that "current" should be used instead of a passed-in | 77 | * indicating that "current" should be used instead of a passed-in |
| 78 | * value from the exec bprm as done with arch_pick_mmap_layout(). | 78 | * value from the exec bprm as done with arch_pick_mmap_layout(). |
| 79 | */ | 79 | */ |
| 80 | static unsigned long mmap_upper_limit(struct rlimit *rlim_stack) | 80 | unsigned long mmap_upper_limit(struct rlimit *rlim_stack) |
| 81 | { | 81 | { |
| 82 | unsigned long stack_base; | 82 | unsigned long stack_base; |
| 83 | 83 | ||
| @@ -130,6 +130,7 @@ SECTIONS · 129 unmodified lines | |||
| 130 | RO_DATA(8) | 130 | RO_DATA(8) |
| 131 | 131 | ||
| 132 | /* unwind info */ | 132 | /* unwind info */ |
| 133 | . = ALIGN(4); | ||
| 133 | .PARISC.unwind : { | 134 | .PARISC.unwind : { |
| 134 | __start___unwind = .; | 135 | __start___unwind = .; |
| 135 | *(.PARISC.unwind) | 136 | *(.PARISC.unwind) |
| @@ -228,7 +228,6 @@ typedef struct thread_struct thread_struct; · 227 unmodified lines | |||
| 228 | execve_tail(); \ | 228 | execve_tail(); \ |
| 229 | } while (0) | 229 | } while (0) |
| 230 | 230 | ||
| 231 | /* Forward declaration, a strange C thing */ | ||
| 232 | struct task_struct; | 231 | struct task_struct; |
| 233 | struct mm_struct; | 232 | struct mm_struct; |
| 234 | struct seq_file; | 233 | struct seq_file; |
| @@ -666,6 +666,7 @@ static int __init ipl_init(void) · 665 unmodified lines | |||
| 666 | &ipl_ccw_attr_group_lpar); | 666 | &ipl_ccw_attr_group_lpar); |
| 667 | break; | 667 | break; |
| 668 | case IPL_TYPE_ECKD: | 668 | case IPL_TYPE_ECKD: |
| 669 | case IPL_TYPE_ECKD_DUMP: | ||
| 669 | rc = sysfs_create_group(&ipl_kset->kobj, &ipl_eckd_attr_group); | 670 | rc = sysfs_create_group(&ipl_kset->kobj, &ipl_eckd_attr_group); |
| 670 | break; | 671 | break; |
| 671 | case IPL_TYPE_FCP: | 672 | case IPL_TYPE_FCP: |
| @@ -279,12 +279,6 @@ static int paicrypt_event_init(struct perf_event *event) · 278 unmodified lines | |||
| 279 | if (IS_ERR(cpump)) | 279 | if (IS_ERR(cpump)) |
| 280 | return PTR_ERR(cpump); | 280 | return PTR_ERR(cpump); |
| 281 | 281 | ||
| 282 | /* Event initialization sets last_tag to 0. When later on the events | ||
| 283 | * are deleted and re-added, do not reset the event count value to zero. | ||
| 284 | * Events are added, deleted and re-added when 2 or more events | ||
| 285 | * are active at the same time. | ||
| 286 | */ | ||
| 287 | event->hw.last_tag = 0; | ||
| 288 | event->destroy = paicrypt_event_destroy; | 282 | event->destroy = paicrypt_event_destroy; |
| 289 | 283 | ||
| 290 | if (a->sample_period) { | 284 | if (a->sample_period) { |
| @@ -318,6 +312,11 @@ static void paicrypt_start(struct perf_event *event, int flags) · 27 unmodified lines | |||
| 318 | { | 312 | { |
| 319 | u64 sum; | 313 | u64 sum; |
| 320 | 314 | ||
| 315 | /* Event initialization sets last_tag to 0. When later on the events | ||
| 316 | * are deleted and re-added, do not reset the event count value to zero. | ||
| 317 | * Events are added, deleted and re-added when 2 or more events | ||
| 318 | * are active at the same time. | ||
| 319 | */ | ||
| 321 | if (!event->hw.last_tag) { | 320 | if (!event->hw.last_tag) { |
| 322 | event->hw.last_tag = 1; | 321 | event->hw.last_tag = 1; |
| 323 | sum = paicrypt_getall(event); /* Get current value */ | 322 | sum = paicrypt_getall(event); /* Get current value */ |
| @@ -260,7 +260,6 @@ static int paiext_event_init(struct perf_event *event) · 259 unmodified lines | |||
| 260 | rc = paiext_alloc(a, event); | 260 | rc = paiext_alloc(a, event); |
| 261 | if (rc) | 261 | if (rc) |
| 262 | return rc; | 262 | return rc; |
| 263 | event->hw.last_tag = 0; | ||
| 264 | event->destroy = paiext_event_destroy; | 263 | event->destroy = paiext_event_destroy; |
| 265 | 264 | ||
| 266 | if (a->sample_period) { | 265 | if (a->sample_period) { |
| @@ -4660,7 +4660,7 @@ static void intel_pmu_check_hybrid_pmus(struct x86_hybrid_pmu *pmu) · 4659 unmodified lines | |||
| 4660 | if (pmu->intel_cap.pebs_output_pt_available) | 4660 | if (pmu->intel_cap.pebs_output_pt_available) |
| 4661 | pmu->pmu.capabilities |= PERF_PMU_CAP_AUX_OUTPUT; | 4661 | pmu->pmu.capabilities |= PERF_PMU_CAP_AUX_OUTPUT; |
| 4662 | else | 4662 | else |
| 4663 | pmu->pmu.capabilities |= ~PERF_PMU_CAP_AUX_OUTPUT; | 4663 | pmu->pmu.capabilities &= ~PERF_PMU_CAP_AUX_OUTPUT; |
| 4664 | 4664 | ||
| 4665 | intel_pmu_check_event_constraints(pmu->event_constraints, | 4665 | intel_pmu_check_event_constraints(pmu->event_constraints, |
| 4666 | pmu->num_counters, | 4666 | pmu->num_counters, |
| @@ -15,6 +15,7 @@ · 14 unmodified lines | |||
| 15 | #include <linux/io.h> | 15 | #include <linux/io.h> |
| 16 | #include <asm/apic.h> | 16 | #include <asm/apic.h> |
| 17 | #include <asm/desc.h> | 17 | #include <asm/desc.h> |
| 18 | #include <asm/e820/api.h> | ||
| 18 | #include <asm/sev.h> | 19 | #include <asm/sev.h> |
| 19 | #include <asm/ibt.h> | 20 | #include <asm/ibt.h> |
| 20 | #include <asm/hypervisor.h> | 21 | #include <asm/hypervisor.h> |
| @@ -286,15 +287,31 @@ static int hv_cpu_die(unsigned int cpu) · 265 unmodified lines | |||
| 286 | 287 | ||
| 287 | static int __init hv_pci_init(void) | 288 | static int __init hv_pci_init(void) |
| 288 | { | 289 | { |
| 289 | int gen2vm = efi_enabled(EFI_BOOT); | 290 | bool gen2vm = efi_enabled(EFI_BOOT); |
| 290 | 291 | ||
| 291 | /* | 292 | /* |
| 292 | * For Generation-2 VM, we exit from pci_arch_init() by returning 0. | 293 | * A Generation-2 VM doesn't support legacy PCI/PCIe, so both |
| 293 | * The purpose is to suppress the harmless warning: | 294 | * raw_pci_ops and raw_pci_ext_ops are NULL, and pci_subsys_init() -> |
| 295 | * pcibios_init() doesn't call pcibios_resource_survey() -> | ||
| 296 | * e820__reserve_resources_late(); as a result, any emulated persistent | ||
| 297 | * memory of E820_TYPE_PRAM (12) via the kernel parameter | ||
| 298 | * memmap=nn[KMG]!ss is not added into iomem_resource and hence can't be | ||
| 299 | * detected by register_e820_pmem(). Fix this by directly calling | ||
| 300 | * e820__reserve_resources_late() here: e820__reserve_resources_late() | ||
| 301 | * depends on e820__reserve_resources(), which has been called earlier | ||
| 302 | * from setup_arch(). Note: e820__reserve_resources_late() also adds | ||
| 303 | * any memory of E820_TYPE_PMEM (7) into iomem_resource, and | ||
| 304 | * acpi_nfit_register_region() -> acpi_nfit_insert_resource() -> | ||
| 305 | * region_intersects() returns REGION_INTERSECTS, so the memory of | ||
| 306 | * E820_TYPE_PMEM won't get added twice. | ||
| 307 | * | ||
| 308 | * We return 0 here so that pci_arch_init() won't print the warning: | ||
| 294 | * "PCI: Fatal: No config space access function found" | 309 | * "PCI: Fatal: No config space access function found" |
| 295 | */ | 310 | */ |
| 296 | if (gen2vm) | 311 | if (gen2vm) { |
| 312 | e820__reserve_resources_late(); | ||
| 297 | return 0; | 313 | return 0; |
| 314 | } | ||
| 298 | 315 | ||
| 299 | /* For Generation-1 VM, we'll proceed in pci_arch_init(). */ | 316 | /* For Generation-1 VM, we'll proceed in pci_arch_init(). */ |
| 300 | return 1; | 317 | return 1; |
| @@ -16,6 +16,9 @@ · 15 unmodified lines | |||
| 16 | #include <asm/x86_init.h> | 16 | #include <asm/x86_init.h> |
| 17 | #include <asm/cpufeature.h> | 17 | #include <asm/cpufeature.h> |
| 18 | #include <asm/irq_vectors.h> | 18 | #include <asm/irq_vectors.h> |
| 19 | #include <asm/xen/hypervisor.h> | ||
| 20 | |||
| 21 | #include <xen/xen.h> | ||
| 19 | 22 | ||
| 20 | #ifdef CONFIG_ACPI_APEI | 23 | #ifdef CONFIG_ACPI_APEI |
| 21 | # include <asm/pgtable_types.h> | 24 | # include <asm/pgtable_types.h> |
| @@ -127,6 +130,17 @@ static inline void arch_acpi_set_proc_cap_bits(u32 *cap) · 105 unmodified lines | |||
| 127 | if (!cpu_has(c, X86_FEATURE_MWAIT) || | 130 | if (!cpu_has(c, X86_FEATURE_MWAIT) || |
| 128 | boot_option_idle_override == IDLE_NOMWAIT) | 131 | boot_option_idle_override == IDLE_NOMWAIT) |
| 129 | *cap &= ~(ACPI_PROC_CAP_C_C1_FFH | ACPI_PROC_CAP_C_C2C3_FFH); | 132 | *cap &= ~(ACPI_PROC_CAP_C_C1_FFH | ACPI_PROC_CAP_C_C2C3_FFH); |
| 133 | |||
| 134 | if (xen_initial_domain()) { | ||
| 135 | /* | ||
| 136 | * When Linux is running as Xen dom0, the hypervisor is the | ||
| 137 | * entity in charge of the processor power management, and so | ||
| 138 | * Xen needs to check the OS capabilities reported in the | ||
| 139 | * processor capabilities buffer matches what the hypervisor | ||
| 140 | * driver supports. | ||
| 141 | */ | ||
| 142 | xen_sanitize_proc_cap_bits(cap); | ||
| 143 | } | ||
| 130 | } | 144 | } |
| 131 | 145 | ||
| 132 | static inline bool acpi_has_cpu_in_madt(void) | 146 | static inline bool acpi_has_cpu_in_madt(void) |
| @@ -100,4 +100,13 @@ static inline void leave_lazy(enum xen_lazy_mode mode) · 99 unmodified lines | |||
| 100 | 100 | ||
| 101 | enum xen_lazy_mode xen_get_lazy_mode(void); | 101 | enum xen_lazy_mode xen_get_lazy_mode(void); |
| 102 | 102 | ||
| 103 | #if defined(CONFIG_XEN_DOM0) && defined(CONFIG_ACPI) | ||
| 104 | void xen_sanitize_proc_cap_bits(uint32_t *buf); | ||
| 105 | #else | ||
| 106 | static inline void xen_sanitize_proc_cap_bits(uint32_t *buf) | ||
| 107 | { | ||
| 108 | BUG(); | ||
| 109 | } | ||
| 110 | #endif | ||
| 111 | |||
| 103 | #endif /* _ASM_X86_XEN_HYPERVISOR_H */ | 112 | #endif /* _ASM_X86_XEN_HYPERVISOR_H */ |
| @@ -63,6 +63,7 @@ int acpi_fix_pin2_polarity __initdata; · 62 unmodified lines | |||
| 63 | 63 | ||
| 64 | #ifdef CONFIG_X86_LOCAL_APIC | 64 | #ifdef CONFIG_X86_LOCAL_APIC |
| 65 | static u64 acpi_lapic_addr __initdata = APIC_DEFAULT_PHYS_BASE; | 65 | static u64 acpi_lapic_addr __initdata = APIC_DEFAULT_PHYS_BASE; |
| 66 | static bool has_lapic_cpus __initdata; | ||
| 66 | static bool acpi_support_online_capable; | 67 | static bool acpi_support_online_capable; |
| 67 | #endif | 68 | #endif |
| 68 | 69 | ||
| @@ -232,6 +233,14 @@ acpi_parse_x2apic(union acpi_subtable_headers *header, const unsigned long end) · 163 unmodified lines | |||
| 232 | if (!acpi_is_processor_usable(processor->lapic_flags)) | 233 | if (!acpi_is_processor_usable(processor->lapic_flags)) |
| 233 | return 0; | 234 | return 0; |
| 234 | 235 | ||
| 236 | /* | ||
| 237 | * According to https://uefi.org/specs/ACPI/6.5/05_ACPI_Software_Programming_Model.html#processor-local-x2apic-structure | ||
| 238 | * when MADT provides both valid LAPIC and x2APIC entries, the APIC ID | ||
| 239 | * in x2APIC must be equal or greater than 0xff. | ||
| 240 | */ | ||
| 241 | if (has_lapic_cpus && apic_id < 0xff) | ||
| 242 | return 0; | ||
| 243 | |||
| 235 | /* | 244 | /* |
| 236 | * We need to register disabled CPU as well to permit | 245 | * We need to register disabled CPU as well to permit |
| 237 | * counting disabled CPUs. This allows us to size | 246 | * counting disabled CPUs. This allows us to size |
| @@ -1114,10 +1123,7 @@ static int __init early_acpi_parse_madt_lapic_addr_ovr(void) · 876 unmodified lines | |||
| 1114 | 1123 | ||
| 1115 | static int __init acpi_parse_madt_lapic_entries(void) | 1124 | static int __init acpi_parse_madt_lapic_entries(void) |
| 1116 | { | 1125 | { |
| 1117 | int count; | 1126 | int count, x2count = 0; |
| 1118 | int x2count = 0; | ||
| 1119 | int ret; | ||
| 1120 | struct acpi_subtable_proc madt_proc[2]; | ||
| 1121 | 1127 | ||
| 1122 | if (!boot_cpu_has(X86_FEATURE_APIC)) | 1128 | if (!boot_cpu_has(X86_FEATURE_APIC)) |
| 1123 | return -ENODEV; | 1129 | return -ENODEV; |
| @@ -1126,21 +1132,11 @@ static int __init acpi_parse_madt_lapic_entries(void) · 2 unmodified lines | |||
| 1126 | acpi_parse_sapic, MAX_LOCAL_APIC); | 1132 | acpi_parse_sapic, MAX_LOCAL_APIC); |
| 1127 | 1133 | ||
| 1128 | if (!count) { | 1134 | if (!count) { |
| 1129 | memset(madt_proc, 0, sizeof(madt_proc)); | 1135 | count = acpi_table_parse_madt(ACPI_MADT_TYPE_LOCAL_APIC, |
| 1130 | madt_proc[0].id = ACPI_MADT_TYPE_LOCAL_APIC; | 1136 | acpi_parse_lapic, MAX_LOCAL_APIC); |
| 1131 | madt_proc[0].handler = acpi_parse_lapic; | 1137 | has_lapic_cpus = count > 0; |
| 1132 | madt_proc[1].id = ACPI_MADT_TYPE_LOCAL_X2APIC; | 1138 | x2count = acpi_table_parse_madt(ACPI_MADT_TYPE_LOCAL_X2APIC, |
| 1133 | madt_proc[1].handler = acpi_parse_x2apic; | 1139 | acpi_parse_x2apic, MAX_LOCAL_APIC); |
| 1134 | ret = acpi_table_parse_entries_array(ACPI_SIG_MADT, | ||
| 1135 | sizeof(struct acpi_table_madt), | ||
| 1136 | madt_proc, ARRAY_SIZE(madt_proc), MAX_LOCAL_APIC); | ||
| 1137 | if (ret < 0) { | ||
| 1138 | pr_err("Error parsing LAPIC/X2APIC entries\n"); | ||
| 1139 | return ret; | ||
| 1140 | } | ||
| 1141 | |||
| 1142 | count = madt_proc[0].count; | ||
| 1143 | x2count = madt_proc[1].count; | ||
| 1144 | } | 1140 | } |
| 1145 | if (!count && !x2count) { | 1141 | if (!count && !x2count) { |
| 1146 | pr_err("No LAPIC entries present\n"); | 1142 | pr_err("No LAPIC entries present\n"); |
| @@ -104,8 +104,6 @@ struct cont_desc { · 103 unmodified lines | |||
| 104 | size_t size; | 104 | size_t size; |
| 105 | }; | 105 | }; |
| 106 | 106 | ||
| 107 | static u32 ucode_new_rev; | ||
| 108 | |||
| 109 | /* | 107 | /* |
| 110 | * Microcode patch container file is prepended to the initrd in cpio | 108 | * Microcode patch container file is prepended to the initrd in cpio |
| 111 | * format. See Documentation/arch/x86/microcode.rst | 109 | * format. See Documentation/arch/x86/microcode.rst |
| @@ -442,12 +440,11 @@ static int __apply_microcode_amd(struct microcode_amd *mc) · 330 unmodified lines | |||
| 442 | * | 440 | * |
| 443 | * Returns true if container found (sets @desc), false otherwise. | 441 | * Returns true if container found (sets @desc), false otherwise. |
| 444 | */ | 442 | */ |
| 445 | static bool early_apply_microcode(u32 cpuid_1_eax, void *ucode, size_t size) | 443 | static bool early_apply_microcode(u32 cpuid_1_eax, u32 old_rev, void *ucode, size_t size) |
| 446 | { | 444 | { |
| 447 | struct cont_desc desc = { 0 }; | 445 | struct cont_desc desc = { 0 }; |
| 448 | struct microcode_amd *mc; | 446 | struct microcode_amd *mc; |
| 449 | bool ret = false; | 447 | bool ret = false; |
| 450 | u32 rev, dummy; | ||
| 451 | 448 | ||
| 452 | desc.cpuid_1_eax = cpuid_1_eax; | 449 | desc.cpuid_1_eax = cpuid_1_eax; |
| 453 | 450 | ||
| @@ -457,22 +454,15 @@ static bool early_apply_microcode(u32 cpuid_1_eax, void *ucode, size_t size) · 3 unmodified lines | |||
| 457 | if (!mc) | 454 | if (!mc) |
| 458 | return ret; | 455 | return ret; |
| 459 | 456 | ||
| 460 | native_rdmsr(MSR_AMD64_PATCH_LEVEL, rev, dummy); | ||
| 461 | |||
| 462 | /* | 457 | /* |
| 463 | * Allow application of the same revision to pick up SMT-specific | 458 | * Allow application of the same revision to pick up SMT-specific |
| 464 | * changes even if the revision of the other SMT thread is already | 459 | * changes even if the revision of the other SMT thread is already |
| 465 | * up-to-date. | 460 | * up-to-date. |
| 466 | */ | 461 | */ |
| 467 | if (rev > mc->hdr.patch_id) | 462 | if (old_rev > mc->hdr.patch_id) |
| 468 | return ret; | 463 | return ret; |
| 469 | 464 | ||
| 470 | if (!__apply_microcode_amd(mc)) { | 465 | return !__apply_microcode_amd(mc); |
| 471 | ucode_new_rev = mc->hdr.patch_id; | ||
| 472 | ret = true; | ||
| 473 | } | ||
| 474 | |||
| 475 | return ret; | ||
| 476 | } | 466 | } |
| 477 | 467 | ||
| 478 | static bool get_builtin_microcode(struct cpio_data *cp, unsigned int family) | 468 | static bool get_builtin_microcode(struct cpio_data *cp, unsigned int family) |
| @@ -506,9 +496,12 @@ static void __init find_blobs_in_containers(unsigned int cpuid_1_eax, struct cpi · 27 unmodified lines | |||
| 506 | *ret = cp; | 496 | *ret = cp; |
| 507 | } | 497 | } |
| 508 | 498 | ||
| 509 | void __init load_ucode_amd_bsp(unsigned int cpuid_1_eax) | 499 | void __init load_ucode_amd_bsp(struct early_load_data *ed, unsigned int cpuid_1_eax) |
| 510 | { | 500 | { |
| 511 | struct cpio_data cp = { }; | 501 | struct cpio_data cp = { }; |
| 502 | u32 dummy; | ||
| 503 | |||
| 504 | native_rdmsr(MSR_AMD64_PATCH_LEVEL, ed->old_rev, dummy); | ||
| 512 | 505 | ||
| 513 | /* Needed in load_microcode_amd() */ | 506 | /* Needed in load_microcode_amd() */ |
| 514 | ucode_cpu_info[0].cpu_sig.sig = cpuid_1_eax; | 507 | ucode_cpu_info[0].cpu_sig.sig = cpuid_1_eax; |
| @@ -517,7 +510,8 @@ void __init load_ucode_amd_bsp(unsigned int cpuid_1_eax) · 2 unmodified lines | |||
| 517 | if (!(cp.data && cp.size)) | 510 | if (!(cp.data && cp.size)) |
| 518 | return; | 511 | return; |
| 519 | 512 | ||
| 520 | early_apply_microcode(cpuid_1_eax, cp.data, cp.size); | 513 | if (early_apply_microcode(cpuid_1_eax, ed->old_rev, cp.data, cp.size)) |
| 514 | native_rdmsr(MSR_AMD64_PATCH_LEVEL, ed->new_rev, dummy); | ||
| 521 | } | 515 | } |
| 522 | 516 | ||
| 523 | static enum ucode_state load_microcode_amd(u8 family, const u8 *data, size_t size); | 517 | static enum ucode_state load_microcode_amd(u8 family, const u8 *data, size_t size); |
| @@ -625,10 +619,8 @@ void reload_ucode_amd(unsigned int cpu) · 101 unmodified lines | |||
| 625 | rdmsr(MSR_AMD64_PATCH_LEVEL, rev, dummy); | 619 | rdmsr(MSR_AMD64_PATCH_LEVEL, rev, dummy); |
| 626 | 620 | ||
| 627 | if (rev < mc->hdr.patch_id) { | 621 | if (rev < mc->hdr.patch_id) { |
| 628 | if (!__apply_microcode_amd(mc)) { | 622 | if (!__apply_microcode_amd(mc)) |
| 629 | ucode_new_rev = mc->hdr.patch_id; | 623 | pr_info_once("reload revision: 0x%08x\n", mc->hdr.patch_id); |
| 630 | pr_info("reload patch_level=0x%08x\n", ucode_new_rev); | ||
| 631 | } | ||
| 632 | } | 624 | } |
| 633 | } | 625 | } |
| 634 | 626 | ||
| @@ -649,8 +641,6 @@ static int collect_cpu_info_amd(int cpu, struct cpu_signature *csig) · 14 unmodified lines | |||
| 649 | if (p && (p->patch_id == csig->rev)) | 641 | if (p && (p->patch_id == csig->rev)) |
| 650 | uci->mc = p->data; | 642 | uci->mc = p->data; |
| 651 | 643 | ||
| 652 | pr_info("CPU%d: patch_level=0x%08x\n", cpu, csig->rev); | ||
| 653 | |||
| 654 | return 0; | 644 | return 0; |
| 655 | } | 645 | } |
| 656 | 646 | ||
| @@ -691,8 +681,6 @@ static enum ucode_state apply_microcode_amd(int cpu) · 34 unmodified lines | |||
| 691 | rev = mc_amd->hdr.patch_id; | 681 | rev = mc_amd->hdr.patch_id; |
| 692 | ret = UCODE_UPDATED; | 682 | ret = UCODE_UPDATED; |
| 693 | 683 | ||
| 694 | pr_info("CPU%d: new patch_level=0x%08x\n", cpu, rev); | ||
| 695 | |||
| 696 | out: | 684 | out: |
| 697 | uci->cpu_sig.rev = rev; | 685 | uci->cpu_sig.rev = rev; |
| 698 | c->microcode = rev; | 686 | c->microcode = rev; |
| @@ -935,11 +923,6 @@ struct microcode_ops * __init init_amd_microcode(void) · 236 unmodified lines | |||
| 935 | pr_warn("AMD CPU family 0x%x not supported\n", c->x86); | 923 | pr_warn("AMD CPU family 0x%x not supported\n", c->x86); |
| 936 | return NULL; | 924 | return NULL; |
| 937 | } | 925 | } |
| 938 | |||
| 939 | if (ucode_new_rev) | ||
| 940 | pr_info_once("microcode updated early to new patch_level=0x%08x\n", | ||
| 941 | ucode_new_rev); | ||
| 942 | |||
| 943 | return µcode_amd_ops; | 926 | return µcode_amd_ops; |
| 944 | } | 927 | } |
| 945 | 928 | ||
| @@ -41,8 +41,6 @@ · 40 unmodified lines | |||
| 41 | 41 | ||
| 42 | #include "internal.h" | 42 | #include "internal.h" |
| 43 | 43 | ||
| 44 | #define DRIVER_VERSION "2.2" | ||
| 45 | |||
| 46 | static struct microcode_ops *microcode_ops; | 44 | static struct microcode_ops *microcode_ops; |
| 47 | bool dis_ucode_ldr = true; | 45 | bool dis_ucode_ldr = true; |
| 48 | 46 | ||
| @@ -77,6 +75,8 @@ static u32 final_levels[] = { · 28 unmodified lines | |||
| 77 | 0, /* T-101 terminator */ | 75 | 0, /* T-101 terminator */ |
| 78 | }; | 76 | }; |
| 79 | 77 | ||
| 78 | struct early_load_data early_data; | ||
| 79 | |||
| 80 | /* | 80 | /* |
| 81 | * Check the current patch level on this CPU. | 81 | * Check the current patch level on this CPU. |
| 82 | * | 82 | * |
| @@ -155,9 +155,9 @@ void __init load_ucode_bsp(void) · 72 unmodified lines | |||
| 155 | return; | 155 | return; |
| 156 | 156 | ||
| 157 | if (intel) | 157 | if (intel) |
| 158 | load_ucode_intel_bsp(); | 158 | load_ucode_intel_bsp(&early_data); |
| 159 | else | 159 | else |
| 160 | load_ucode_amd_bsp(cpuid_1_eax); | 160 | load_ucode_amd_bsp(&early_data, cpuid_1_eax); |
| 161 | } | 161 | } |
| 162 | 162 | ||
| 163 | void load_ucode_ap(void) | 163 | void load_ucode_ap(void) |
| @@ -828,6 +828,11 @@ static int __init microcode_init(void) · 664 unmodified lines | |||
| 828 | if (!microcode_ops) | 828 | if (!microcode_ops) |
| 829 | return -ENODEV; | 829 | return -ENODEV; |
| 830 | 830 | ||
| 831 | pr_info_once("Current revision: 0x%08x\n", (early_data.new_rev ?: early_data.old_rev)); | ||
| 832 | |||
| 833 | if (early_data.new_rev) | ||
| 834 | pr_info_once("Updated early from: 0x%08x\n", early_data.old_rev); | ||
| 835 | |||
| 831 | microcode_pdev = platform_device_register_simple("microcode", -1, NULL, 0); | 836 | microcode_pdev = platform_device_register_simple("microcode", -1, NULL, 0); |
| 832 | if (IS_ERR(microcode_pdev)) | 837 | if (IS_ERR(microcode_pdev)) |
| 833 | return PTR_ERR(microcode_pdev); | 838 | return PTR_ERR(microcode_pdev); |
| @@ -846,8 +851,6 @@ static int __init microcode_init(void) · 12 unmodified lines | |||
| 846 | cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/microcode:online", | 851 | cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/microcode:online", |
| 847 | mc_cpu_online, mc_cpu_down_prep); | 852 | mc_cpu_online, mc_cpu_down_prep); |
| 848 | 853 | ||
| 849 | pr_info("Microcode Update Driver: v%s.", DRIVER_VERSION); | ||
| 850 | |||
| 851 | return 0; | 854 | return 0; |
| 852 | 855 | ||
| 853 | out_pdev: | 856 | out_pdev: |
| @@ -339,16 +339,9 @@ static enum ucode_state __apply_microcode(struct ucode_cpu_info *uci, · 338 unmodified lines | |||
| 339 | static enum ucode_state apply_microcode_early(struct ucode_cpu_info *uci) | 339 | static enum ucode_state apply_microcode_early(struct ucode_cpu_info *uci) |
| 340 | { | 340 | { |
| 341 | struct microcode_intel *mc = uci->mc; | 341 | struct microcode_intel *mc = uci->mc; |
| 342 | enum ucode_state ret; | 342 | u32 cur_rev; |
| 343 | u32 cur_rev, date; | ||
| 344 | 343 | ||
| 345 | ret = __apply_microcode(uci, mc, &cur_rev); | 344 | return __apply_microcode(uci, mc, &cur_rev); |
| 346 | if (ret == UCODE_UPDATED) { | ||
| 347 | date = mc->hdr.date; | ||
| 348 | pr_info_once("updated early: 0x%x -> 0x%x, date = %04x-%02x-%02x\n", | ||
| 349 | cur_rev, mc->hdr.rev, date & 0xffff, date >> 24, (date >> 16) & 0xff); | ||
| 350 | } | ||
| 351 | return ret; | ||
| 352 | } | 345 | } |
| 353 | 346 | ||
| 354 | static __init bool load_builtin_intel_microcode(struct cpio_data *cp) | 347 | static __init bool load_builtin_intel_microcode(struct cpio_data *cp) |
| @@ -413,13 +406,17 @@ static int __init save_builtin_microcode(void) · 58 unmodified lines | |||
| 413 | early_initcall(save_builtin_microcode); | 406 | early_initcall(save_builtin_microcode); |
| 414 | 407 | ||
| 415 | /* Load microcode on BSP from initrd or builtin blobs */ | 408 | /* Load microcode on BSP from initrd or builtin blobs */ |
| 416 | void __init load_ucode_intel_bsp(void) | 409 | void __init load_ucode_intel_bsp(struct early_load_data *ed) |
| 417 | { | 410 | { |
| 418 | struct ucode_cpu_info uci; | 411 | struct ucode_cpu_info uci; |
| 419 | 412 | ||
| 413 | ed->old_rev = intel_get_microcode_revision(); | ||
| 414 | |||
| 420 | uci.mc = get_microcode_blob(&uci, false); | 415 | uci.mc = get_microcode_blob(&uci, false); |
| 421 | if (uci.mc && apply_microcode_early(&uci) == UCODE_UPDATED) | 416 | if (uci.mc && apply_microcode_early(&uci) == UCODE_UPDATED) |
| 422 | ucode_patch_va = UCODE_BSP_LOADED; | 417 | ucode_patch_va = UCODE_BSP_LOADED; |
| 418 | |||
| 419 | ed->new_rev = uci.cpu_sig.rev; | ||
| 423 | } | 420 | } |
| 424 | 421 | ||
| 425 | void load_ucode_intel_ap(void) | 422 | void load_ucode_intel_ap(void) |
| @@ -37,6 +37,12 @@ struct microcode_ops { · 36 unmodified lines | |||
| 37 | use_nmi : 1; | 37 | use_nmi : 1; |
| 38 | }; | 38 | }; |
| 39 | 39 | ||
| 40 | struct early_load_data { | ||
| 41 | u32 old_rev; | ||
| 42 | u32 new_rev; | ||
| 43 | }; | ||
| 44 | |||
| 45 | extern struct early_load_data early_data; | ||
| 40 | extern struct ucode_cpu_info ucode_cpu_info[]; | 46 | extern struct ucode_cpu_info ucode_cpu_info[]; |
| 41 | struct cpio_data find_microcode_in_initrd(const char *path); | 47 | struct cpio_data find_microcode_in_initrd(const char *path); |
| 42 | 48 | ||
| @@ -92,14 +98,14 @@ extern bool dis_ucode_ldr; · 49 unmodified lines | |||
| 92 | extern bool force_minrev; | 98 | extern bool force_minrev; |
| 93 | 99 | ||
| 94 | #ifdef CONFIG_CPU_SUP_AMD | 100 | #ifdef CONFIG_CPU_SUP_AMD |
| 95 | void load_ucode_amd_bsp(unsigned int family); | 101 | void load_ucode_amd_bsp(struct early_load_data *ed, unsigned int family); |
| 96 | void load_ucode_amd_ap(unsigned int family); | 102 | void load_ucode_amd_ap(unsigned int family); |
| 97 | int save_microcode_in_initrd_amd(unsigned int family); | 103 | int save_microcode_in_initrd_amd(unsigned int family); |
| 98 | void reload_ucode_amd(unsigned int cpu); | 104 | void reload_ucode_amd(unsigned int cpu); |
| 99 | struct microcode_ops *init_amd_microcode(void); | 105 | struct microcode_ops *init_amd_microcode(void); |
| 100 | void exit_amd_microcode(void); | 106 | void exit_amd_microcode(void); |
| 101 | #else /* CONFIG_CPU_SUP_AMD */ | 107 | #else /* CONFIG_CPU_SUP_AMD */ |
| 102 | static inline void load_ucode_amd_bsp(unsigned int family) { } | 108 | static inline void load_ucode_amd_bsp(struct early_load_data *ed, unsigned int family) { } |
| 103 | static inline void load_ucode_amd_ap(unsigned int family) { } | 109 | static inline void load_ucode_amd_ap(unsigned int family) { } |
| 104 | static inline int save_microcode_in_initrd_amd(unsigned int family) { return -EINVAL; } | 110 | static inline int save_microcode_in_initrd_amd(unsigned int family) { return -EINVAL; } |
| 105 | static inline void reload_ucode_amd(unsigned int cpu) { } | 111 | static inline void reload_ucode_amd(unsigned int cpu) { } |
| @@ -108,12 +114,12 @@ static inline void exit_amd_microcode(void) { } · 2 unmodified lines | |||
| 108 | #endif /* !CONFIG_CPU_SUP_AMD */ | 114 | #endif /* !CONFIG_CPU_SUP_AMD */ |
| 109 | 115 | ||
| 110 | #ifdef CONFIG_CPU_SUP_INTEL | 116 | #ifdef CONFIG_CPU_SUP_INTEL |
| 111 | void load_ucode_intel_bsp(void); | 117 | void load_ucode_intel_bsp(struct early_load_data *ed); |
| 112 | void load_ucode_intel_ap(void); | 118 | void load_ucode_intel_ap(void); |
| 113 | void reload_ucode_intel(void); | 119 | void reload_ucode_intel(void); |
| 114 | struct microcode_ops *init_intel_microcode(void); | 120 | struct microcode_ops *init_intel_microcode(void); |
| 115 | #else /* CONFIG_CPU_SUP_INTEL */ | 121 | #else /* CONFIG_CPU_SUP_INTEL */ |
| 116 | static inline void load_ucode_intel_bsp(void) { } | 122 | static inline void load_ucode_intel_bsp(struct early_load_data *ed) { } |
| 117 | static inline void load_ucode_intel_ap(void) { } | 123 | static inline void load_ucode_intel_ap(void) { } |
| 118 | static inline void reload_ucode_intel(void) { } | 124 | static inline void reload_ucode_intel(void) { } |
| 119 | static inline struct microcode_ops *init_intel_microcode(void) { return NULL; } | 125 | static inline struct microcode_ops *init_intel_microcode(void) { return NULL; } |
| @@ -262,11 +262,14 @@ static uint32_t __init ms_hyperv_platform(void) · 261 unmodified lines | |||
| 262 | static int hv_nmi_unknown(unsigned int val, struct pt_regs *regs) | 262 | static int hv_nmi_unknown(unsigned int val, struct pt_regs *regs) |
| 263 | { | 263 | { |
| 264 | static atomic_t nmi_cpu = ATOMIC_INIT(-1); | 264 | static atomic_t nmi_cpu = ATOMIC_INIT(-1); |
| 265 | unsigned int old_cpu, this_cpu; | ||
| 265 | 266 | ||
| 266 | if (!unknown_nmi_panic) | 267 | if (!unknown_nmi_panic) |
| 267 | return NMI_DONE; | 268 | return NMI_DONE; |
| 268 | 269 | ||
| 269 | if (atomic_cmpxchg(&nmi_cpu, -1, raw_smp_processor_id()) != -1) | 270 | old_cpu = -1; |
| 271 | this_cpu = raw_smp_processor_id(); | ||
| 272 | if (!atomic_try_cmpxchg(&nmi_cpu, &old_cpu, this_cpu)) | ||
| 270 | return NMI_HANDLED; | 273 | return NMI_HANDLED; |
| 271 | 274 | ||
| 272 | return NMI_DONE; | 275 | return NMI_DONE; |
| @@ -175,9 +175,6 @@ int x64_setup_rt_frame(struct ksignal *ksig, struct pt_regs *regs) · 174 unmodified lines | |||
| 175 | frame = get_sigframe(ksig, regs, sizeof(struct rt_sigframe), &fp); | 175 | frame = get_sigframe(ksig, regs, sizeof(struct rt_sigframe), &fp); |
| 176 | uc_flags = frame_uc_flags(regs); | 176 | uc_flags = frame_uc_flags(regs); |
| 177 | 177 | ||
| 178 | if (setup_signal_shadow_stack(ksig)) | ||
| 179 | return -EFAULT; | ||
| 180 | |||
| 181 | if (!user_access_begin(frame, sizeof(*frame))) | 178 | if (!user_access_begin(frame, sizeof(*frame))) |
| 182 | return -EFAULT; | 179 | return -EFAULT; |
| 183 | 180 | ||
| @@ -198,6 +195,9 @@ int x64_setup_rt_frame(struct ksignal *ksig, struct pt_regs *regs) · 14 unmodified lines | |||
| 198 | return -EFAULT; | 195 | return -EFAULT; |
| 199 | } | 196 | } |
| 200 | 197 | ||
| 198 | if (setup_signal_shadow_stack(ksig)) | ||
| 199 | return -EFAULT; | ||
| 200 | |||
| 201 | /* Set up registers for signal handler */ | 201 | /* Set up registers for signal handler */ |
| 202 | regs->di = ksig->sig; | 202 | regs->di = ksig->sig; |
| 203 | /* In case the signal handler was declared without prototypes */ | 203 | /* In case the signal handler was declared without prototypes */ |
| @@ -425,6 +425,8 @@ void bdev_set_nr_sectors(struct block_device *bdev, sector_t sectors) · 424 unmodified lines | |||
| 425 | 425 | ||
| 426 | void bdev_add(struct block_device *bdev, dev_t dev) | 426 | void bdev_add(struct block_device *bdev, dev_t dev) |
| 427 | { | 427 | { |
| 428 | if (bdev_stable_writes(bdev)) | ||
| 429 | mapping_set_stable_writes(bdev->bd_inode->i_mapping); | ||
| 428 | bdev->bd_dev = dev; | 430 | bdev->bd_dev = dev; |
| 429 | bdev->bd_inode->i_rdev = dev; | 431 | bdev->bd_inode->i_rdev = dev; |
| 430 | bdev->bd_inode->i_ino = dev; | 432 | bdev->bd_inode->i_ino = dev; |
| @@ -577,6 +577,7 @@ static void blkg_destroy_all(struct gendisk *disk) · 576 unmodified lines | |||
| 577 | struct request_queue *q = disk->queue; | 577 | struct request_queue *q = disk->queue; |
| 578 | struct blkcg_gq *blkg, *n; | 578 | struct blkcg_gq *blkg, *n; |
| 579 | int count = BLKG_DESTROY_BATCH_SIZE; | 579 | int count = BLKG_DESTROY_BATCH_SIZE; |
| 580 | int i; | ||
| 580 | 581 | ||
| 581 | restart: | 582 | restart: |
| 582 | spin_lock_irq(&q->queue_lock); | 583 | spin_lock_irq(&q->queue_lock); |
| @@ -602,6 +603,18 @@ restart: · 19 unmodified lines | |||
| 602 | } | 603 | } |
| 603 | } | 604 | } |
| 604 | 605 | ||
| 606 | /* | ||
| 607 | * Mark policy deactivated since policy offline has been done, and | ||
| 608 | * the free is scheduled, so future blkcg_deactivate_policy() can | ||
| 609 | * be bypassed | ||
| 610 | */ | ||
| 611 | for (i = 0; i < BLKCG_MAX_POLS; i++) { | ||
| 612 | struct blkcg_policy *pol = blkcg_policy[i]; | ||
| 613 | |||
| 614 | if (pol) | ||
| 615 | __clear_bit(pol->plid, q->blkcg_pols); | ||
| 616 | } | ||
| 617 | |||
| 605 | q->root_blkg = NULL; | 618 | q->root_blkg = NULL; |
| 606 | spin_unlock_irq(&q->queue_lock); | 619 | spin_unlock_irq(&q->queue_lock); |
| 607 | } | 620 | } |
| @@ -249,8 +249,6 @@ static inline struct blkcg_gq *blkg_lookup(struct blkcg *blkcg, · 248 unmodified lines | |||
| 249 | { | 249 | { |
| 250 | struct blkcg_gq *blkg; | 250 | struct blkcg_gq *blkg; |
| 251 | 251 | ||
| 252 | WARN_ON_ONCE(!rcu_read_lock_held()); | ||
| 253 | |||
| 254 | if (blkcg == &blkcg_root) | 252 | if (blkcg == &blkcg_root) |
| 255 | return q->root_blkg; | 253 | return q->root_blkg; |
| 256 | 254 | ||
| @@ -2858,11 +2858,8 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q, · 2857 unmodified lines | |||
| 2858 | }; | 2858 | }; |
| 2859 | struct request *rq; | 2859 | struct request *rq; |
| 2860 | 2860 | ||
| 2861 | if (unlikely(bio_queue_enter(bio))) | ||
| 2862 | return NULL; | ||
| 2863 | |||
| 2864 | if (blk_mq_attempt_bio_merge(q, bio, nsegs)) | 2861 | if (blk_mq_attempt_bio_merge(q, bio, nsegs)) |
| 2865 | goto queue_exit; | 2862 | return NULL; |
| 2866 | 2863 | ||
| 2867 | rq_qos_throttle(q, bio); | 2864 | rq_qos_throttle(q, bio); |
| 2868 | 2865 | ||
| @@ -2878,35 +2875,23 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q, · 9 unmodified lines | |||
| 2878 | rq_qos_cleanup(q, bio); | 2875 | rq_qos_cleanup(q, bio); |
| 2879 | if (bio->bi_opf & REQ_NOWAIT) | 2876 | if (bio->bi_opf & REQ_NOWAIT) |
| 2880 | bio_wouldblock_error(bio); | 2877 | bio_wouldblock_error(bio); |
| 2881 | queue_exit: | ||
| 2882 | blk_queue_exit(q); | ||
| 2883 | return NULL; | 2878 | return NULL; |
| 2884 | } | 2879 | } |
| 2885 | 2880 | ||
| 2886 | static inline struct request *blk_mq_get_cached_request(struct request_queue *q, | 2881 | /* return true if this @rq can be used for @bio */ |
| 2887 | struct blk_plug *plug, struct bio **bio, unsigned int nsegs) | 2882 | static bool blk_mq_can_use_cached_rq(struct request *rq, struct blk_plug *plug, |
| 2883 | struct bio *bio) | ||
| 2888 | { | 2884 | { |
| 2889 | struct request *rq; | 2885 | enum hctx_type type = blk_mq_get_hctx_type(bio->bi_opf); |
| 2890 | enum hctx_type type, hctx_type; | 2886 | enum hctx_type hctx_type = rq->mq_hctx->type; |
| 2891 | 2887 | ||
| 2892 | if (!plug) | 2888 | WARN_ON_ONCE(rq_list_peek(&plug->cached_rq) != rq); |
| 2893 | return NULL; | ||
| 2894 | rq = rq_list_peek(&plug->cached_rq); | ||
| 2895 | if (!rq || rq->q != q) | ||
| 2896 | return NULL; | ||
| 2897 | 2889 | ||
| 2898 | if (blk_mq_attempt_bio_merge(q, *bio, nsegs)) { | ||
| 2899 | *bio = NULL; | ||
| 2900 | return NULL; | ||
| 2901 | } | ||
| 2902 | |||
| 2903 | type = blk_mq_get_hctx_type((*bio)->bi_opf); | ||
| 2904 | hctx_type = rq->mq_hctx->type; | ||
| 2905 | if (type != hctx_type && | 2890 | if (type != hctx_type && |
| 2906 | !(type == HCTX_TYPE_READ && hctx_type == HCTX_TYPE_DEFAULT)) | 2891 | !(type == HCTX_TYPE_READ && hctx_type == HCTX_TYPE_DEFAULT)) |
| 2907 | return NULL; | 2892 | return false; |
| 2908 | if (op_is_flush(rq->cmd_flags) != op_is_flush((*bio)->bi_opf)) | 2893 | if (op_is_flush(rq->cmd_flags) != op_is_flush(bio->bi_opf)) |
| 2909 | return NULL; | 2894 | return false; |
| 2910 | 2895 | ||
| 2911 | /* | 2896 | /* |
| 2912 | * If any qos ->throttle() end up blocking, we will have flushed the | 2897 | * If any qos ->throttle() end up blocking, we will have flushed the |
| @@ -2914,12 +2899,12 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, · 1 unmodified line | |||
| 2914 | * before we throttle. | 2899 | * before we throttle. |
| 2915 | */ | 2900 | */ |
| 2916 | plug->cached_rq = rq_list_next(rq); | 2901 | plug->cached_rq = rq_list_next(rq); |
| 2917 | rq_qos_throttle(q, *bio); | 2902 | rq_qos_throttle(rq->q, bio); |
| 2918 | 2903 | ||
| 2919 | blk_mq_rq_time_init(rq, 0); | 2904 | blk_mq_rq_time_init(rq, 0); |
| 2920 | rq->cmd_flags = (*bio)->bi_opf; | 2905 | rq->cmd_flags = bio->bi_opf; |
| 2921 | INIT_LIST_HEAD(&rq->queuelist); | 2906 | INIT_LIST_HEAD(&rq->queuelist); |
| 2922 | return rq; | 2907 | return true; |
| 2923 | } | 2908 | } |
| 2924 | 2909 | ||
| 2925 | static void bio_set_ioprio(struct bio *bio) | 2910 | static void bio_set_ioprio(struct bio *bio) |
| @@ -2949,7 +2934,7 @@ void blk_mq_submit_bio(struct bio *bio) · 23 unmodified lines | |||
| 2949 | struct blk_plug *plug = blk_mq_plug(bio); | 2934 | struct blk_plug *plug = blk_mq_plug(bio); |
| 2950 | const int is_sync = op_is_sync(bio->bi_opf); | 2935 | const int is_sync = op_is_sync(bio->bi_opf); |
| 2951 | struct blk_mq_hw_ctx *hctx; | 2936 | struct blk_mq_hw_ctx *hctx; |
| 2952 | struct request *rq; | 2937 | struct request *rq = NULL; |
| 2953 | unsigned int nr_segs = 1; | 2938 | unsigned int nr_segs = 1; |
| 2954 | blk_status_t ret; | 2939 | blk_status_t ret; |
| 2955 | 2940 | ||
| @@ -2960,20 +2945,36 @@ void blk_mq_submit_bio(struct bio *bio) · 4 unmodified lines | |||
| 2960 | return; | 2945 | return; |
| 2961 | } | 2946 | } |
| 2962 | 2947 | ||
| 2963 | if (!bio_integrity_prep(bio)) | ||
| 2964 | return; | ||
| 2965 | |||
| 2966 | bio_set_ioprio(bio); | 2948 | bio_set_ioprio(bio); |
| 2967 | 2949 | ||
| 2968 | rq = blk_mq_get_cached_request(q, plug, &bio, nr_segs); | 2950 | if (plug) { |
| 2969 | if (!rq) { | 2951 | rq = rq_list_peek(&plug->cached_rq); |
| 2970 | if (!bio) | 2952 | if (rq && rq->q != q) |
| 2953 | rq = NULL; | ||
| 2954 | } | ||
| 2955 | if (rq) { | ||
| 2956 | if (!bio_integrity_prep(bio)) | ||
| 2971 | return; | 2957 | return; |
| 2972 | rq = blk_mq_get_new_requests(q, plug, bio, nr_segs); | 2958 | if (blk_mq_attempt_bio_merge(q, bio, nr_segs)) |
| 2973 | if (unlikely(!rq)) | ||
| 2974 | return; | 2959 | return; |
| 2960 | if (blk_mq_can_use_cached_rq(rq, plug, bio)) | ||
| 2961 | goto done; | ||
| 2962 | percpu_ref_get(&q->q_usage_counter); | ||
| 2963 | } else { | ||
| 2964 | if (unlikely(bio_queue_enter(bio))) | ||
| 2965 | return; | ||
| 2966 | if (!bio_integrity_prep(bio)) | ||
| 2967 | goto fail; | ||
| 2968 | } | ||
| 2969 | |||
| 2970 | rq = blk_mq_get_new_requests(q, plug, bio, nr_segs); | ||
| 2971 | if (unlikely(!rq)) { | ||
| 2972 | fail: | ||
| 2973 | blk_queue_exit(q); | ||
| 2974 | return; | ||
| 2975 | } | 2975 | } |
| 2976 | 2976 | ||
| 2977 | done: | ||
| 2977 | trace_block_getrq(bio); | 2978 | trace_block_getrq(bio); |
| 2978 | 2979 | ||
| 2979 | rq_qos_track(q, rq, bio); | 2980 | rq_qos_track(q, rq, bio); |
| @@ -163,38 +163,15 @@ EXPORT_SYMBOL(blk_pre_runtime_resume); · 162 unmodified lines | |||
| 163 | * @q: the queue of the device | 163 | * @q: the queue of the device |
| 164 | * | 164 | * |
| 165 | * Description: | 165 | * Description: |
| 166 | * For historical reasons, this routine merely calls blk_set_runtime_active() | 166 | * Restart the queue of a runtime suspended device. It does this regardless |
| 167 | * to do the real work of restarting the queue. It does this regardless of | 167 | * of whether the device's runtime-resume succeeded; even if it failed the |
| 168 | * whether the device's runtime-resume succeeded; even if it failed the | ||
| 169 | * driver or error handler will need to communicate with the device. | 168 | * driver or error handler will need to communicate with the device. |
| 170 | * | 169 | * |
| 171 | * This function should be called near the end of the device's | 170 | * This function should be called near the end of the device's |
| 172 | * runtime_resume callback. | 171 | * runtime_resume callback to correct queue runtime PM status and re-enable |
| 172 | * peeking requests from the queue. | ||
| 173 | */ | 173 | */ |
| 174 | void blk_post_runtime_resume(struct request_queue *q) | 174 | void blk_post_runtime_resume(struct request_queue *q) |
| 175 | { | ||
| 176 | blk_set_runtime_active(q); | ||
| 177 | } | ||
| 178 | EXPORT_SYMBOL(blk_post_runtime_resume); | ||
| 179 | |||
| 180 | /** | ||
| 181 | * blk_set_runtime_active - Force runtime status of the queue to be active | ||
| 182 | * @q: the queue of the device | ||
| 183 | * | ||
| 184 | * If the device is left runtime suspended during system suspend the resume | ||
| 185 | * hook typically resumes the device and corrects runtime status | ||
| 186 | * accordingly. However, that does not affect the queue runtime PM status | ||
| 187 | * which is still "suspended". This prevents processing requests from the | ||
| 188 | * queue. | ||
| 189 | * | ||
| 190 | * This function can be used in driver's resume hook to correct queue | ||
| 191 | * runtime PM status and re-enable peeking requests from the queue. It | ||
| 192 | * should be called before first request is added to the queue. | ||
| 193 | * | ||
| 194 | * This function is also called by blk_post_runtime_resume() for | ||
| 195 | * runtime resumes. It does everything necessary to restart the queue. | ||
| 196 | */ | ||
| 197 | void blk_set_runtime_active(struct request_queue *q) | ||
| 198 | { | 175 | { |
| 199 | int old_status; | 176 | int old_status; |
| 200 | 177 | ||
| @@ -211,4 +188,4 @@ void blk_set_runtime_active(struct request_queue *q) · 10 unmodified lines | |||
| 211 | if (old_status != RPM_ACTIVE) | 188 | if (old_status != RPM_ACTIVE) |
| 212 | blk_clear_pm_only(q); | 189 | blk_clear_pm_only(q); |
| 213 | } | 190 | } |
| 214 | EXPORT_SYMBOL(blk_set_runtime_active); | 191 | EXPORT_SYMBOL(blk_post_runtime_resume); |
| @@ -1320,6 +1320,7 @@ static void tg_conf_updated(struct throtl_grp *tg, bool global) · 1319 unmodified lines | |||
| 1320 | tg_bps_limit(tg, READ), tg_bps_limit(tg, WRITE), | 1320 | tg_bps_limit(tg, READ), tg_bps_limit(tg, WRITE), |
| 1321 | tg_iops_limit(tg, READ), tg_iops_limit(tg, WRITE)); | 1321 | tg_iops_limit(tg, READ), tg_iops_limit(tg, WRITE)); |
| 1322 | 1322 | ||
| 1323 | rcu_read_lock(); | ||
| 1323 | /* | 1324 | /* |
| 1324 | * Update has_rules[] flags for the updated tg's subtree. A tg is | 1325 | * Update has_rules[] flags for the updated tg's subtree. A tg is |
| 1325 | * considered to have rules if either the tg itself or any of its | 1326 | * considered to have rules if either the tg itself or any of its |
| @@ -1347,6 +1348,7 @@ static void tg_conf_updated(struct throtl_grp *tg, bool global) · 21 unmodified lines | |||
| 1347 | this_tg->latency_target = max(this_tg->latency_target, | 1348 | this_tg->latency_target = max(this_tg->latency_target, |
| 1348 | parent_tg->latency_target); | 1349 | parent_tg->latency_target); |
| 1349 | } | 1350 | } |
| 1351 | rcu_read_unlock(); | ||
| 1350 | 1352 | ||
| 1351 | /* | 1353 | /* |
| 1352 | * We're already holding queue_lock and know @tg is valid. Let's | 1354 | * We're already holding queue_lock and know @tg is valid. Let's |
| @@ -11,6 +11,7 @@ · 10 unmodified lines | |||
| 11 | #include <linux/idr.h> | 11 | #include <linux/idr.h> |
| 12 | 12 | ||
| 13 | #include <drm/drm_accel.h> | 13 | #include <drm/drm_accel.h> |
| 14 | #include <drm/drm_auth.h> | ||
| 14 | #include <drm/drm_debugfs.h> | 15 | #include <drm/drm_debugfs.h> |
| 15 | #include <drm/drm_drv.h> | 16 | #include <drm/drm_drv.h> |
| 16 | #include <drm/drm_file.h> | 17 | #include <drm/drm_file.h> |
| @@ -1,16 +1,17 @@ | |||
| 1 | # SPDX-License-Identifier: GPL-2.0-only | 1 | # SPDX-License-Identifier: GPL-2.0-only |
| 2 | 2 | ||
| 3 | config DRM_ACCEL_IVPU | 3 | config DRM_ACCEL_IVPU |
| 4 | tristate "Intel VPU for Meteor Lake and newer" | 4 | tristate "Intel NPU (Neural Processing Unit)" |
| 5 | depends on DRM_ACCEL | 5 | depends on DRM_ACCEL |
| 6 | depends on X86_64 && !UML | 6 | depends on X86_64 && !UML |
| 7 | depends on PCI && PCI_MSI | 7 | depends on PCI && PCI_MSI |
| 8 | select FW_LOADER | 8 | select FW_LOADER |
| 9 | select SHMEM | 9 | select DRM_GEM_SHMEM_HELPER |
| 10 | select GENERIC_ALLOCATOR | 10 | select GENERIC_ALLOCATOR |
| 11 | help | 11 | help |
| 12 | Choose this option if you have a system that has an 14th generation Intel CPU | 12 | Choose this option if you have a system with an 14th generation |
| 13 | or newer. VPU stands for Versatile Processing Unit and it's a CPU-integrated | 13 | Intel CPU (Meteor Lake) or newer. Intel NPU (formerly called Intel VPU) |
| 14 | inference accelerator for Computer Vision and Deep Learning applications. | 14 | is a CPU-integrated inference accelerator for Computer Vision |
| 15 | and Deep Learning applications. | ||
| 15 | 16 | ||
| 16 | If "M" is selected, the module will be called intel_vpu. | 17 | If "M" is selected, the module will be called intel_vpu. |
| @@ -14,6 +14,7 @@ · 13 unmodified lines | |||
| 14 | #include "ivpu_fw.h" | 14 | #include "ivpu_fw.h" |
| 15 | #include "ivpu_fw_log.h" | 15 | #include "ivpu_fw_log.h" |
| 16 | #include "ivpu_gem.h" | 16 | #include "ivpu_gem.h" |
| 17 | #include "ivpu_hw.h" | ||
| 17 | #include "ivpu_jsm_msg.h" | 18 | #include "ivpu_jsm_msg.h" |
| 18 | #include "ivpu_pm.h" | 19 | #include "ivpu_pm.h" |
| 19 | 20 | ||
| @@ -115,6 +116,31 @@ static const struct drm_debugfs_info vdev_debugfs_list[] = { · 95 unmodified lines | |||
| 115 | {"reset_pending", reset_pending_show, 0}, | 116 | {"reset_pending", reset_pending_show, 0}, |
| 116 | }; | 117 | }; |
| 117 | 118 | ||
| 119 | static ssize_t | ||
| 120 | dvfs_mode_fops_write(struct file *file, const char __user *user_buf, size_t size, loff_t *pos) | ||
| 121 | { | ||
| 122 | struct ivpu_device *vdev = file->private_data; | ||
| 123 | struct ivpu_fw_info *fw = vdev->fw; | ||
| 124 | u32 dvfs_mode; | ||
| 125 | int ret; | ||
| 126 | |||
| 127 | ret = kstrtou32_from_user(user_buf, size, 0, &dvfs_mode); | ||
| 128 | if (ret < 0) | ||
| 129 | return ret; | ||
| 130 | |||
| 131 | fw->dvfs_mode = dvfs_mode; | ||
| 132 | |||
| 133 | ivpu_pm_schedule_recovery(vdev); | ||
| 134 | |||
| 135 | return size; | ||
| 136 | } | ||
| 137 | |||
| 138 | static const struct file_operations dvfs_mode_fops = { | ||
| 139 | .owner = THIS_MODULE, | ||
| 140 | .open = simple_open, | ||
| 141 | .write = dvfs_mode_fops_write, | ||
| 142 | }; | ||
| 143 | |||
| 118 | static int fw_log_show(struct seq_file *s, void *v) | 144 | static int fw_log_show(struct seq_file *s, void *v) |
| 119 | { | 145 | { |
| 120 | struct ivpu_device *vdev = s->private; | 146 | struct ivpu_device *vdev = s->private; |
| @@ -151,6 +177,30 @@ static const struct file_operations fw_log_fops = { · 30 unmodified lines | |||
| 151 | .release = single_release, | 177 | .release = single_release, |
| 152 | }; | 178 | }; |
| 153 | 179 | ||
| 180 | static ssize_t | ||
| 181 | fw_profiling_freq_fops_write(struct file *file, const char __user *user_buf, | ||
| 182 | size_t size, loff_t *pos) | ||
| 183 | { | ||
| 184 | struct ivpu_device *vdev = file->private_data; | ||
| 185 | bool enable; | ||
| 186 | int ret; | ||
| 187 | |||
| 188 | ret = kstrtobool_from_user(user_buf, size, &enable); | ||
| 189 | if (ret < 0) | ||
| 190 | return ret; | ||
| 191 | |||
| 192 | ivpu_hw_profiling_freq_drive(vdev, enable); | ||
| 193 | ivpu_pm_schedule_recovery(vdev); | ||
| 194 | |||
| 195 | return size; | ||
| 196 | } | ||
| 197 | |||
| 198 | static const struct file_operations fw_profiling_freq_fops = { | ||
| 199 | .owner = THIS_MODULE, | ||
| 200 | .open = simple_open, | ||
| 201 | .write = fw_profiling_freq_fops_write, | ||
| 202 | }; | ||
| 203 | |||
| 154 | static ssize_t | 204 | static ssize_t |
| 155 | fw_trace_destination_mask_fops_write(struct file *file, const char __user *user_buf, | 205 | fw_trace_destination_mask_fops_write(struct file *file, const char __user *user_buf, |
| 156 | size_t size, loff_t *pos) | 206 | size_t size, loff_t *pos) |
| @@ -280,6 +330,9 @@ void ivpu_debugfs_init(struct ivpu_device *vdev) · 123 unmodified lines | |||
| 280 | debugfs_create_file("force_recovery", 0200, debugfs_root, vdev, | 330 | debugfs_create_file("force_recovery", 0200, debugfs_root, vdev, |
| 281 | &ivpu_force_recovery_fops); | 331 | &ivpu_force_recovery_fops); |
| 282 | 332 | ||
| 333 | debugfs_create_file("dvfs_mode", 0200, debugfs_root, vdev, | ||
| 334 | &dvfs_mode_fops); | ||
| 335 | |||
| 283 | debugfs_create_file("fw_log", 0644, debugfs_root, vdev, | 336 | debugfs_create_file("fw_log", 0644, debugfs_root, vdev, |
| 284 | &fw_log_fops); | 337 | &fw_log_fops); |
| 285 | debugfs_create_file("fw_trace_destination_mask", 0200, debugfs_root, vdev, | 338 | debugfs_create_file("fw_trace_destination_mask", 0200, debugfs_root, vdev, |
| @@ -291,4 +344,8 @@ void ivpu_debugfs_init(struct ivpu_device *vdev) · 5 unmodified lines | |||
| 291 | 344 | ||
| 292 | debugfs_create_file("reset_engine", 0200, debugfs_root, vdev, | 345 | debugfs_create_file("reset_engine", 0200, debugfs_root, vdev, |
| 293 | &ivpu_reset_engine_fops); | 346 | &ivpu_reset_engine_fops); |
| 347 | |||
| 348 | if (ivpu_hw_gen(vdev) >= IVPU_HW_40XX) | ||
| 349 | debugfs_create_file("fw_profiling_freq_drive", 0200, | ||
| 350 | debugfs_root, vdev, &fw_profiling_freq_fops); | ||
| 294 | } | 351 | } |
| @@ -31,8 +31,6 @@ · 30 unmodified lines | |||
| 31 | __stringify(DRM_IVPU_DRIVER_MINOR) "." | 31 | __stringify(DRM_IVPU_DRIVER_MINOR) "." |
| 32 | #endif | 32 | #endif |
| 33 | 33 | ||
| 34 | static const struct drm_driver driver; | ||
| 35 | |||
| 36 | static struct lock_class_key submitted_jobs_xa_lock_class_key; | 34 | static struct lock_class_key submitted_jobs_xa_lock_class_key; |
| 37 | 35 | ||
| 38 | int ivpu_dbg_mask; | 36 | int ivpu_dbg_mask; |
| @@ -41,7 +39,7 @@ MODULE_PARM_DESC(dbg_mask, "Driver debug mask. See IVPU_DBG_* macros."); · 2 unmodified lines | |||
| 41 | 39 | ||
| 42 | int ivpu_test_mode; | 40 | int ivpu_test_mode; |
| 43 | module_param_named_unsafe(test_mode, ivpu_test_mode, int, 0644); | 41 | module_param_named_unsafe(test_mode, ivpu_test_mode, int, 0644); |
| 44 | MODULE_PARM_DESC(test_mode, "Test mode: 0 - normal operation, 1 - fw unit test, 2 - null hw"); | 42 | MODULE_PARM_DESC(test_mode, "Test mode mask. See IVPU_TEST_MODE_* macros."); |
| 45 | 43 | ||
| 46 | u8 ivpu_pll_min_ratio; | 44 | u8 ivpu_pll_min_ratio; |
| 47 | module_param_named(pll_min_ratio, ivpu_pll_min_ratio, byte, 0644); | 45 | module_param_named(pll_min_ratio, ivpu_pll_min_ratio, byte, 0644); |
| @@ -93,8 +91,8 @@ static void file_priv_release(struct kref *ref) · 45 unmodified lines | |||
| 93 | ivpu_dbg(vdev, FILE, "file_priv release: ctx %u\n", file_priv->ctx.id); | 91 | ivpu_dbg(vdev, FILE, "file_priv release: ctx %u\n", file_priv->ctx.id); |
| 94 | 92 | ||
| 95 | ivpu_cmdq_release_all(file_priv); | 93 | ivpu_cmdq_release_all(file_priv); |
| 96 | ivpu_bo_remove_all_bos_from_context(&file_priv->ctx); | ||
| 97 | ivpu_jsm_context_release(vdev, file_priv->ctx.id); | 94 | ivpu_jsm_context_release(vdev, file_priv->ctx.id); |
| 95 | ivpu_bo_remove_all_bos_from_context(vdev, &file_priv->ctx); | ||
| 98 | ivpu_mmu_user_context_fini(vdev, &file_priv->ctx); | 96 | ivpu_mmu_user_context_fini(vdev, &file_priv->ctx); |
| 99 | drm_WARN_ON(&vdev->drm, xa_erase_irq(&vdev->context_xa, file_priv->ctx.id) != file_priv); | 97 | drm_WARN_ON(&vdev->drm, xa_erase_irq(&vdev->context_xa, file_priv->ctx.id) != file_priv); |
| 100 | mutex_destroy(&file_priv->lock); | 98 | mutex_destroy(&file_priv->lock); |
| @@ -317,16 +315,14 @@ static int ivpu_wait_for_ready(struct ivpu_device *vdev) · 216 unmodified lines | |||
| 317 | unsigned long timeout; | 315 | unsigned long timeout; |
| 318 | int ret; | 316 | int ret; |
| 319 | 317 | ||
| 320 | if (ivpu_test_mode == IVPU_TEST_MODE_FW_TEST) | 318 | if (ivpu_test_mode & IVPU_TEST_MODE_FW_TEST) |
| 321 | return 0; | 319 | return 0; |
| 322 | 320 | ||
| 323 | ivpu_ipc_consumer_add(vdev, &cons, IVPU_IPC_CHAN_BOOT_MSG); | 321 | ivpu_ipc_consumer_add(vdev, &cons, IVPU_IPC_CHAN_BOOT_MSG, NULL); |
| 324 | 322 | ||
| 325 | timeout = jiffies + msecs_to_jiffies(vdev->timeout.boot); | 323 | timeout = jiffies + msecs_to_jiffies(vdev->timeout.boot); |
| 326 | while (1) { | 324 | while (1) { |
| 327 | ret = ivpu_ipc_irq_handler(vdev); | 325 | ivpu_ipc_irq_handler(vdev, NULL); |
| 328 | if (ret) | ||
| 329 | break; | ||
| 330 | ret = ivpu_ipc_receive(vdev, &cons, &ipc_hdr, NULL, 0); | 326 | ret = ivpu_ipc_receive(vdev, &cons, &ipc_hdr, NULL, 0); |
| 331 | if (ret != -ETIMEDOUT || time_after_eq(jiffies, timeout)) | 327 | if (ret != -ETIMEDOUT || time_after_eq(jiffies, timeout)) |
| 332 | break; | 328 | break; |
| @@ -362,7 +358,7 @@ int ivpu_boot(struct ivpu_device *vdev) · 29 unmodified lines | |||
| 362 | int ret; | 358 | int ret; |
| 363 | 359 | ||
| 364 | /* Update boot params located at first 4KB of FW memory */ | 360 | /* Update boot params located at first 4KB of FW memory */ |
| 365 | ivpu_fw_boot_params_setup(vdev, vdev->fw->mem->kvaddr); | 361 | ivpu_fw_boot_params_setup(vdev, ivpu_bo_vaddr(vdev->fw->mem)); |
| 366 | 362 | ||
| 367 | ret = ivpu_hw_boot_fw(vdev); | 363 | ret = ivpu_hw_boot_fw(vdev); |
| 368 | if (ret) { | 364 | if (ret) { |
| @@ -414,7 +410,9 @@ static const struct drm_driver driver = { · 45 unmodified lines | |||
| 414 | 410 | ||
| 415 | .open = ivpu_open, | 411 | .open = ivpu_open, |
| 416 | .postclose = ivpu_postclose, | 412 | .postclose = ivpu_postclose, |
| 417 | .gem_prime_import = ivpu_gem_prime_import, | 413 | |
| 414 | .gem_create_object = ivpu_gem_create_object, | ||
| 415 | .gem_prime_import_sg_table = drm_gem_shmem_prime_import_sg_table, | ||
| 418 | 416 | ||
| 419 | .ioctls = ivpu_drm_ioctls, | 417 | .ioctls = ivpu_drm_ioctls, |
| 420 | .num_ioctls = ARRAY_SIZE(ivpu_drm_ioctls), | 418 | .num_ioctls = ARRAY_SIZE(ivpu_drm_ioctls), |
| @@ -427,6 +425,13 @@ static const struct drm_driver driver = { · 6 unmodified lines | |||
| 427 | .minor = DRM_IVPU_DRIVER_MINOR, | 425 | .minor = DRM_IVPU_DRIVER_MINOR, |
| 428 | }; | 426 | }; |
| 429 | 427 | ||
| 428 | static irqreturn_t ivpu_irq_thread_handler(int irq, void *arg) | ||
| 429 | { | ||
| 430 | struct ivpu_device *vdev = arg; | ||
| 431 | |||
| 432 | return ivpu_ipc_irq_thread_handler(vdev); | ||
| 433 | } | ||
| 434 | |||
| 430 | static int ivpu_irq_init(struct ivpu_device *vdev) | 435 | static int ivpu_irq_init(struct ivpu_device *vdev) |
| 431 | { | 436 | { |
| 432 | struct pci_dev *pdev = to_pci_dev(vdev->drm.dev); | 437 | struct pci_dev *pdev = to_pci_dev(vdev->drm.dev); |
| @@ -440,8 +445,8 @@ static int ivpu_irq_init(struct ivpu_device *vdev) · 7 unmodified lines | |||
| 440 | 445 | ||
| 441 | vdev->irq = pci_irq_vector(pdev, 0); | 446 | vdev->irq = pci_irq_vector(pdev, 0); |
| 442 | 447 | ||
| 443 | ret = devm_request_irq(vdev->drm.dev, vdev->irq, vdev->hw->ops->irq_handler, | 448 | ret = devm_request_threaded_irq(vdev->drm.dev, vdev->irq, vdev->hw->ops->irq_handler, |
| 444 | IRQF_NO_AUTOEN, DRIVER_NAME, vdev); | 449 | ivpu_irq_thread_handler, IRQF_NO_AUTOEN, DRIVER_NAME, vdev); |
| 445 | if (ret) | 450 | if (ret) |
| 446 | ivpu_err(vdev, "Failed to request an IRQ %d\n", ret); | 451 | ivpu_err(vdev, "Failed to request an IRQ %d\n", ret); |
| 447 | 452 | ||
| @@ -533,6 +538,11 @@ static int ivpu_dev_init(struct ivpu_device *vdev) · 85 unmodified lines | |||
| 533 | xa_init_flags(&vdev->context_xa, XA_FLAGS_ALLOC); | 538 | xa_init_flags(&vdev->context_xa, XA_FLAGS_ALLOC); |
| 534 | xa_init_flags(&vdev->submitted_jobs_xa, XA_FLAGS_ALLOC1); | 539 | xa_init_flags(&vdev->submitted_jobs_xa, XA_FLAGS_ALLOC1); |
| 535 | lockdep_set_class(&vdev->submitted_jobs_xa.xa_lock, &submitted_jobs_xa_lock_class_key); | 540 | lockdep_set_class(&vdev->submitted_jobs_xa.xa_lock, &submitted_jobs_xa_lock_class_key); |
| 541 | INIT_LIST_HEAD(&vdev->bo_list); | ||
| 542 | |||
| 543 | ret = drmm_mutex_init(&vdev->drm, &vdev->bo_list_lock); | ||
| 544 | if (ret) | ||
| 545 | goto err_xa_destroy; | ||
| 536 | 546 | ||
| 537 | ret = ivpu_pci_init(vdev); | 547 | ret = ivpu_pci_init(vdev); |
| 538 | if (ret) | 548 | if (ret) |
| @@ -550,7 +560,7 @@ static int ivpu_dev_init(struct ivpu_device *vdev) · 11 unmodified lines | |||
| 550 | /* Power up early so the rest of init code can access VPU registers */ | 560 | /* Power up early so the rest of init code can access VPU registers */ |
| 551 | ret = ivpu_hw_power_up(vdev); | 561 | ret = ivpu_hw_power_up(vdev); |
| 552 | if (ret) | 562 | if (ret) |
| 553 | goto err_xa_destroy; | 563 | goto err_power_down; |
| 554 | 564 | ||
| 555 | ret = ivpu_mmu_global_context_init(vdev); | 565 | ret = ivpu_mmu_global_context_init(vdev); |
| 556 | if (ret) | 566 | if (ret) |
| @@ -574,20 +584,15 @@ static int ivpu_dev_init(struct ivpu_device *vdev) · 17 unmodified lines | |||
| 574 | 584 | ||
| 575 | ivpu_pm_init(vdev); | 585 | ivpu_pm_init(vdev); |
| 576 | 586 | ||
| 577 | ret = ivpu_job_done_thread_init(vdev); | ||
| 578 | if (ret) | ||
| 579 | goto err_ipc_fini; | ||
| 580 | |||
| 581 | ret = ivpu_boot(vdev); | 587 | ret = ivpu_boot(vdev); |
| 582 | if (ret) | 588 | if (ret) |
| 583 | goto err_job_done_thread_fini; | 589 | goto err_ipc_fini; |
| 584 | 590 | ||
| 591 | ivpu_job_done_consumer_init(vdev); | ||
| 585 | ivpu_pm_enable(vdev); | 592 | ivpu_pm_enable(vdev); |
| 586 | 593 | ||
| 587 | return 0; | 594 | return 0; |
| 588 | 595 | ||
| 589 | err_job_done_thread_fini: | ||
| 590 | ivpu_job_done_thread_fini(vdev); | ||
| 591 | err_ipc_fini: | 596 | err_ipc_fini: |
| 592 | ivpu_ipc_fini(vdev); | 597 | ivpu_ipc_fini(vdev); |
| 593 | err_fw_fini: | 598 | err_fw_fini: |
| @@ -612,7 +617,7 @@ static void ivpu_dev_fini(struct ivpu_device *vdev) · 18 unmodified lines | |||
| 612 | ivpu_shutdown(vdev); | 617 | ivpu_shutdown(vdev); |
| 613 | if (IVPU_WA(d3hot_after_power_off)) | 618 | if (IVPU_WA(d3hot_after_power_off)) |
| 614 | pci_set_power_state(to_pci_dev(vdev->drm.dev), PCI_D3hot); | 619 | pci_set_power_state(to_pci_dev(vdev->drm.dev), PCI_D3hot); |
| 615 | ivpu_job_done_thread_fini(vdev); | 620 | ivpu_job_done_consumer_fini(vdev); |
| 616 | ivpu_pm_cancel_recovery(vdev); | 621 | ivpu_pm_cancel_recovery(vdev); |
| 617 | 622 | ||
| 618 | ivpu_ipc_fini(vdev); | 623 | ivpu_ipc_fini(vdev); |
| @@ -17,9 +17,10 @@ · 16 unmodified lines | |||
| 17 | #include <uapi/drm/ivpu_accel.h> | 17 | #include <uapi/drm/ivpu_accel.h> |
| 18 | 18 | ||
| 19 | #include "ivpu_mmu_context.h" | 19 | #include "ivpu_mmu_context.h" |
| 20 | #include "ivpu_ipc.h" | ||
| 20 | 21 | ||
| 21 | #define DRIVER_NAME "intel_vpu" | 22 | #define DRIVER_NAME "intel_vpu" |
| 22 | #define DRIVER_DESC "Driver for Intel Versatile Processing Unit (VPU)" | 23 | #define DRIVER_DESC "Driver for Intel NPU (Neural Processing Unit)" |
| 23 | #define DRIVER_DATE "20230117" | 24 | #define DRIVER_DATE "20230117" |
| 24 | 25 | ||
| 25 | #define PCI_DEVICE_ID_MTL 0x7d1d | 26 | #define PCI_DEVICE_ID_MTL 0x7d1d |
| @@ -88,6 +89,7 @@ struct ivpu_wa_table { · 62 unmodified lines | |||
| 88 | bool d3hot_after_power_off; | 89 | bool d3hot_after_power_off; |
| 89 | bool interrupt_clear_with_0; | 90 | bool interrupt_clear_with_0; |
| 90 | bool disable_clock_relinquish; | 91 | bool disable_clock_relinquish; |
| 92 | bool disable_d0i3_msg; | ||
| 91 | }; | 93 | }; |
| 92 | 94 | ||
| 93 | struct ivpu_hw_info; | 95 | struct ivpu_hw_info; |
| @@ -115,8 +117,11 @@ struct ivpu_device { · 21 unmodified lines | |||
| 115 | struct xarray context_xa; | 117 | struct xarray context_xa; |
| 116 | struct xa_limit context_xa_limit; | 118 | struct xa_limit context_xa_limit; |
| 117 | 119 | ||
| 120 | struct mutex bo_list_lock; /* Protects bo_list */ | ||
| 121 | struct list_head bo_list; | ||
| 122 | |||
| 118 | struct xarray submitted_jobs_xa; | 123 | struct xarray submitted_jobs_xa; |
| 119 | struct task_struct *job_done_thread; | 124 | struct ivpu_ipc_consumer job_done_consumer; |
| 120 | 125 | ||
| 121 | atomic64_t unique_id_counter; | 126 | atomic64_t unique_id_counter; |
| 122 | 127 | ||
| @@ -126,6 +131,7 @@ struct ivpu_device { · 3 unmodified lines | |||
| 126 | int tdr; | 131 | int tdr; |
| 127 | int reschedule_suspend; | 132 | int reschedule_suspend; |
| 128 | int autosuspend; | 133 | int autosuspend; |
| 134 | int d0i3_entry_msg; | ||
| 129 | } timeout; | 135 | } timeout; |
| 130 | }; | 136 | }; |
| 131 | 137 | ||
| @@ -148,9 +154,11 @@ extern u8 ivpu_pll_min_ratio; · 16 unmodified lines | |||
| 148 | extern u8 ivpu_pll_max_ratio; | 154 | extern u8 ivpu_pll_max_ratio; |
| 149 | extern bool ivpu_disable_mmu_cont_pages; | 155 | extern bool ivpu_disable_mmu_cont_pages; |
| 150 | 156 | ||
| 151 | #define IVPU_TEST_MODE_DISABLED 0 | 157 | #define IVPU_TEST_MODE_FW_TEST BIT(0) |
| 152 | #define IVPU_TEST_MODE_FW_TEST 1 | 158 | #define IVPU_TEST_MODE_NULL_HW BIT(1) |
| 153 | #define IVPU_TEST_MODE_NULL_HW 2 | 159 | #define IVPU_TEST_MODE_NULL_SUBMISSION BIT(2) |
| 160 | #define IVPU_TEST_MODE_D0I3_MSG_DISABLE BIT(4) | ||
| 161 | #define IVPU_TEST_MODE_D0I3_MSG_ENABLE BIT(5) | ||
| 154 | extern int ivpu_test_mode; | 162 | extern int ivpu_test_mode; |
| 155 | 163 | ||
| 156 | struct ivpu_file_priv *ivpu_file_priv_get(struct ivpu_file_priv *file_priv); | 164 | struct ivpu_file_priv *ivpu_file_priv_get(struct ivpu_file_priv *file_priv); |
| @@ -33,12 +33,17 @@ · 32 unmodified lines | |||
| 33 | 33 | ||
| 34 | #define ADDR_TO_L2_CACHE_CFG(addr) ((addr) >> 31) | 34 | #define ADDR_TO_L2_CACHE_CFG(addr) ((addr) >> 31) |
| 35 | 35 | ||
| 36 | #define IVPU_FW_CHECK_API(vdev, fw_hdr, name, min_major) \ | 36 | /* Check if FW API is compatible with the driver */ |
| 37 | #define IVPU_FW_CHECK_API_COMPAT(vdev, fw_hdr, name, min_major) \ | ||
| 37 | ivpu_fw_check_api(vdev, fw_hdr, #name, \ | 38 | ivpu_fw_check_api(vdev, fw_hdr, #name, \ |
| 38 | VPU_##name##_API_VER_INDEX, \ | 39 | VPU_##name##_API_VER_INDEX, \ |
| 39 | VPU_##name##_API_VER_MAJOR, \ | 40 | VPU_##name##_API_VER_MAJOR, \ |
| 40 | VPU_##name##_API_VER_MINOR, min_major) | 41 | VPU_##name##_API_VER_MINOR, min_major) |
| 41 | 42 | ||
| 43 | /* Check if API version is lower that the given version */ | ||
| 44 | #define IVPU_FW_CHECK_API_VER_LT(vdev, fw_hdr, name, major, minor) \ | ||
| 45 | ivpu_fw_check_api_ver_lt(vdev, fw_hdr, #name, VPU_##name##_API_VER_INDEX, major, minor) | ||
| 46 | |||
| 42 | static char *ivpu_firmware; | 47 | static char *ivpu_firmware; |
| 43 | module_param_named_unsafe(firmware, ivpu_firmware, charp, 0644); | 48 | module_param_named_unsafe(firmware, ivpu_firmware, charp, 0644); |
| 44 | MODULE_PARM_DESC(firmware, "VPU firmware binary in /lib/firmware/.."); | 49 | MODULE_PARM_DESC(firmware, "VPU firmware binary in /lib/firmware/.."); |
| @@ -105,6 +110,19 @@ ivpu_fw_check_api(struct ivpu_device *vdev, const struct vpu_firmware_header *fw · 60 unmodified lines | |||
| 105 | return 0; | 110 | return 0; |
| 106 | } | 111 | } |
| 107 | 112 | ||
| 113 | static bool | ||
| 114 | ivpu_fw_check_api_ver_lt(struct ivpu_device *vdev, const struct vpu_firmware_header *fw_hdr, | ||
| 115 | const char *str, int index, u16 major, u16 minor) | ||
| 116 | { | ||
| 117 | u16 fw_major = (u16)(fw_hdr->api_version[index] >> 16); | ||
| 118 | u16 fw_minor = (u16)(fw_hdr->api_version[index]); | ||
| 119 | |||
| 120 | if (fw_major < major || (fw_major == major && fw_minor < minor)) | ||
| 121 | return true; | ||
| 122 | |||
| 123 | return false; | ||
| 124 | } | ||
| 125 | |||
| 108 | static int ivpu_fw_parse(struct ivpu_device *vdev) | 126 | static int ivpu_fw_parse(struct ivpu_device *vdev) |
| 109 | { | 127 | { |
| 110 | struct ivpu_fw_info *fw = vdev->fw; | 128 | struct ivpu_fw_info *fw = vdev->fw; |
| @@ -164,9 +182,9 @@ static int ivpu_fw_parse(struct ivpu_device *vdev) · 53 unmodified lines | |||
| 164 | ivpu_info(vdev, "Firmware: %s, version: %s", fw->name, | 182 | ivpu_info(vdev, "Firmware: %s, version: %s", fw->name, |
| 165 | (const char *)fw_hdr + VPU_FW_HEADER_SIZE); | 183 | (const char *)fw_hdr + VPU_FW_HEADER_SIZE); |
| 166 | 184 | ||
| 167 | if (IVPU_FW_CHECK_API(vdev, fw_hdr, BOOT, 3)) | 185 | if (IVPU_FW_CHECK_API_COMPAT(vdev, fw_hdr, BOOT, 3)) |
| 168 | return -EINVAL; | 186 | return -EINVAL; |
| 169 | if (IVPU_FW_CHECK_API(vdev, fw_hdr, JSM, 3)) | 187 | if (IVPU_FW_CHECK_API_COMPAT(vdev, fw_hdr, JSM, 3)) |
| 170 | return -EINVAL; | 188 | return -EINVAL; |
| 171 | 189 | ||
| 172 | fw->runtime_addr = runtime_addr; | 190 | fw->runtime_addr = runtime_addr; |
| @@ -182,6 +200,8 @@ static int ivpu_fw_parse(struct ivpu_device *vdev) · 9 unmodified lines | |||
| 182 | fw->trace_destination_mask = VPU_TRACE_DESTINATION_VERBOSE_TRACING; | 200 | fw->trace_destination_mask = VPU_TRACE_DESTINATION_VERBOSE_TRACING; |
| 183 | fw->trace_hw_component_mask = -1; | 201 | fw->trace_hw_component_mask = -1; |
| 184 | 202 | ||
| 203 | fw->dvfs_mode = 0; | ||
| 204 | |||
| 185 | ivpu_dbg(vdev, FW_BOOT, "Size: file %lu image %u runtime %u shavenn %u\n", | 205 | ivpu_dbg(vdev, FW_BOOT, "Size: file %lu image %u runtime %u shavenn %u\n", |
| 186 | fw->file->size, fw->image_size, fw->runtime_size, fw->shave_nn_size); | 206 | fw->file->size, fw->image_size, fw->runtime_size, fw->shave_nn_size); |
| 187 | ivpu_dbg(vdev, FW_BOOT, "Address: runtime 0x%llx, load 0x%llx, entry point 0x%llx\n", | 207 | ivpu_dbg(vdev, FW_BOOT, "Address: runtime 0x%llx, load 0x%llx, entry point 0x%llx\n", |
| @@ -195,6 +215,24 @@ static void ivpu_fw_release(struct ivpu_device *vdev) · 7 unmodified lines | |||
| 195 | release_firmware(vdev->fw->file); | 215 | release_firmware(vdev->fw->file); |
| 196 | } | 216 | } |
| 197 | 217 | ||
| 218 | /* Initialize workarounds that depend on FW version */ | ||
| 219 | static void | ||
| 220 | ivpu_fw_init_wa(struct ivpu_device *vdev) | ||
| 221 | { | ||
| 222 | const struct vpu_firmware_header *fw_hdr = (const void *)vdev->fw->file->data; | ||
| 223 | |||
| 224 | if (IVPU_FW_CHECK_API_VER_LT(vdev, fw_hdr, BOOT, 3, 17) || | ||
| 225 | (ivpu_hw_gen(vdev) > IVPU_HW_37XX) || | ||
| 226 | (ivpu_test_mode & IVPU_TEST_MODE_D0I3_MSG_DISABLE)) | ||
| 227 | vdev->wa.disable_d0i3_msg = true; | ||
| 228 | |||
| 229 | /* Force enable the feature for testing purposes */ | ||
| 230 | if (ivpu_test_mode & IVPU_TEST_MODE_D0I3_MSG_ENABLE) | ||
| 231 | vdev->wa.disable_d0i3_msg = false; | ||
| 232 | |||
| 233 | IVPU_PRINT_WA(disable_d0i3_msg); | ||
| 234 | } | ||
| 235 | |||
| 198 | static int ivpu_fw_update_global_range(struct ivpu_device *vdev) | 236 | static int ivpu_fw_update_global_range(struct ivpu_device *vdev) |
| 199 | { | 237 | { |
| 200 | struct ivpu_fw_info *fw = vdev->fw; | 238 | struct ivpu_fw_info *fw = vdev->fw; |
| @@ -248,7 +286,7 @@ static int ivpu_fw_mem_init(struct ivpu_device *vdev) · 47 unmodified lines | |||
| 248 | 286 | ||
| 249 | if (fw->shave_nn_size) { | 287 | if (fw->shave_nn_size) { |
| 250 | fw->mem_shave_nn = ivpu_bo_alloc_internal(vdev, vdev->hw->ranges.shave.start, | 288 | fw->mem_shave_nn = ivpu_bo_alloc_internal(vdev, vdev->hw->ranges.shave.start, |
| 251 | fw->shave_nn_size, DRM_IVPU_BO_UNCACHED); | 289 | fw->shave_nn_size, DRM_IVPU_BO_WC); |
| 252 | if (!fw->mem_shave_nn) { | 290 | if (!fw->mem_shave_nn) { |
| 253 | ivpu_err(vdev, "Failed to allocate shavenn buffer\n"); | 291 | ivpu_err(vdev, "Failed to allocate shavenn buffer\n"); |
| 254 | ret = -ENOMEM; | 292 | ret = -ENOMEM; |
| @@ -297,6 +335,8 @@ int ivpu_fw_init(struct ivpu_device *vdev) · 42 unmodified lines | |||
| 297 | if (ret) | 335 | if (ret) |
| 298 | goto err_fw_release; | 336 | goto err_fw_release; |
| 299 | 337 | ||
| 338 | ivpu_fw_init_wa(vdev); | ||
| 339 | |||
| 300 | ret = ivpu_fw_mem_init(vdev); | 340 | ret = ivpu_fw_mem_init(vdev); |
| 301 | if (ret) | 341 | if (ret) |
| 302 | goto err_fw_release; | 342 | goto err_fw_release; |
| @@ -422,14 +462,31 @@ static void ivpu_fw_boot_params_print(struct ivpu_device *vdev, struct vpu_boot_ · 119 unmodified lines | |||
| 422 | boot_params->punit_telemetry_sram_size); | 462 | boot_params->punit_telemetry_sram_size); |
| 423 | ivpu_dbg(vdev, FW_BOOT, "boot_params.vpu_telemetry_enable = 0x%x\n", | 463 | ivpu_dbg(vdev, FW_BOOT, "boot_params.vpu_telemetry_enable = 0x%x\n", |
| 424 | boot_params->vpu_telemetry_enable); | 464 | boot_params->vpu_telemetry_enable); |
| 465 | ivpu_dbg(vdev, FW_BOOT, "boot_params.dvfs_mode = %u\n", | ||
| 466 | boot_params->dvfs_mode); | ||
| 467 | ivpu_dbg(vdev, FW_BOOT, "boot_params.d0i3_delayed_entry = %d\n", | ||
| 468 | boot_params->d0i3_delayed_entry); | ||
| 469 | ivpu_dbg(vdev, FW_BOOT, "boot_params.d0i3_residency_time_us = %lld\n", | ||
| 470 | boot_params->d0i3_residency_time_us); | ||
| 471 | ivpu_dbg(vdev, FW_BOOT, "boot_params.d0i3_entry_vpu_ts = %llu\n", | ||
| 472 | boot_params->d0i3_entry_vpu_ts); | ||
| 425 | } | 473 | } |
| 426 | 474 | ||
| 427 | void ivpu_fw_boot_params_setup(struct ivpu_device *vdev, struct vpu_boot_params *boot_params) | 475 | void ivpu_fw_boot_params_setup(struct ivpu_device *vdev, struct vpu_boot_params *boot_params) |
| 428 | { | 476 | { |
| 429 | struct ivpu_bo *ipc_mem_rx = vdev->ipc->mem_rx; | 477 | struct ivpu_bo *ipc_mem_rx = vdev->ipc->mem_rx; |
| 430 | 478 | ||
| 431 | /* In case of warm boot we only have to reset the entrypoint addr */ | 479 | /* In case of warm boot only update variable params */ |
| 432 | if (!ivpu_fw_is_cold_boot(vdev)) { | 480 | if (!ivpu_fw_is_cold_boot(vdev)) { |
| 481 | boot_params->d0i3_residency_time_us = | ||
| 482 | ktime_us_delta(ktime_get_boottime(), vdev->hw->d0i3_entry_host_ts); | ||
| 483 | boot_params->d0i3_entry_vpu_ts = vdev->hw->d0i3_entry_vpu_ts; | ||
| 484 | |||
| 485 | ivpu_dbg(vdev, FW_BOOT, "boot_params.d0i3_residency_time_us = %lld\n", | ||
| 486 | boot_params->d0i3_residency_time_us); | ||
| 487 | ivpu_dbg(vdev, FW_BOOT, "boot_params.d0i3_entry_vpu_ts = %llu\n", | ||
| 488 | boot_params->d0i3_entry_vpu_ts); | ||
| 489 | |||
| 433 | boot_params->save_restore_ret_address = 0; | 490 | boot_params->save_restore_ret_address = 0; |
| 434 | vdev->pm->is_warmboot = true; | 491 | vdev->pm->is_warmboot = true; |
| 435 | wmb(); /* Flush WC buffers after writing save_restore_ret_address */ | 492 | wmb(); /* Flush WC buffers after writing save_restore_ret_address */ |
| @@ -442,6 +499,13 @@ void ivpu_fw_boot_params_setup(struct ivpu_device *vdev, struct vpu_boot_params · 6 unmodified lines | |||
| 442 | boot_params->vpu_id = to_pci_dev(vdev->drm.dev)->bus->number; | 499 | boot_params->vpu_id = to_pci_dev(vdev->drm.dev)->bus->number; |
| 443 | boot_params->frequency = ivpu_hw_reg_pll_freq_get(vdev); | 500 | boot_params->frequency = ivpu_hw_reg_pll_freq_get(vdev); |
| 444 | 501 | ||
| 502 | /* | ||
| 503 | * This param is a debug firmware feature. It switches default clock | ||
| 504 | * to higher resolution one for fine-grained and more accurate firmware | ||
| 505 | * task profiling. | ||
| 506 | */ | ||
| 507 | boot_params->perf_clk_frequency = ivpu_hw_profiling_freq_get(vdev); | ||
| 508 | |||
| 445 | /* | 509 | /* |
| 446 | * Uncached region of VPU address space, covers IPC buffers, job queues | 510 | * Uncached region of VPU address space, covers IPC buffers, job queues |
| 447 | * and log buffers, programmable to L2$ Uncached by VPU MTRR | 511 | * and log buffers, programmable to L2$ Uncached by VPU MTRR |
| @@ -493,6 +557,11 @@ void ivpu_fw_boot_params_setup(struct ivpu_device *vdev, struct vpu_boot_params · 45 unmodified lines | |||
| 493 | boot_params->punit_telemetry_sram_base = ivpu_hw_reg_telemetry_offset_get(vdev); | 557 | boot_params->punit_telemetry_sram_base = ivpu_hw_reg_telemetry_offset_get(vdev); |
| 494 | boot_params->punit_telemetry_sram_size = ivpu_hw_reg_telemetry_size_get(vdev); | 558 | boot_params->punit_telemetry_sram_size = ivpu_hw_reg_telemetry_size_get(vdev); |
| 495 | boot_params->vpu_telemetry_enable = ivpu_hw_reg_telemetry_enable_get(vdev); | 559 | boot_params->vpu_telemetry_enable = ivpu_hw_reg_telemetry_enable_get(vdev); |
| 560 | boot_params->dvfs_mode = vdev->fw->dvfs_mode; | ||
| 561 | if (!IVPU_WA(disable_d0i3_msg)) | ||
| 562 | boot_params->d0i3_delayed_entry = 1; | ||
| 563 | boot_params->d0i3_residency_time_us = 0; | ||
| 564 | boot_params->d0i3_entry_vpu_ts = 0; | ||
| 496 | 565 | ||
| 497 | wmb(); /* Flush WC buffers after writing bootparams */ | 566 | wmb(); /* Flush WC buffers after writing bootparams */ |
| 498 | 567 | ||
| @@ -27,6 +27,7 @@ struct ivpu_fw_info { · 26 unmodified lines | |||
| 27 | u32 trace_level; | 27 | u32 trace_level; |
| 28 | u32 trace_destination_mask; | 28 | u32 trace_destination_mask; |
| 29 | u64 trace_hw_component_mask; | 29 | u64 trace_hw_component_mask; |
| 30 | u32 dvfs_mode; | ||
| 30 | }; | 31 | }; |
| 31 | 32 | ||
| 32 | int ivpu_fw_init(struct ivpu_device *vdev); | 33 | int ivpu_fw_init(struct ivpu_device *vdev); |
| @@ -20,215 +20,18 @@ · 19 unmodified lines | |||
| 20 | #include "ivpu_mmu.h" | 20 | #include "ivpu_mmu.h" |
| 21 | #include "ivpu_mmu_context.h" | 21 | #include "ivpu_mmu_context.h" |
| 22 | 22 | ||
| 23 | MODULE_IMPORT_NS(DMA_BUF); | ||
| 24 | |||
| 25 | static const struct drm_gem_object_funcs ivpu_gem_funcs; | 23 | static const struct drm_gem_object_funcs ivpu_gem_funcs; |
| 26 | 24 | ||
| 27 | static struct lock_class_key prime_bo_lock_class_key; | 25 | static inline void ivpu_dbg_bo(struct ivpu_device *vdev, struct ivpu_bo *bo, const char *action) |
| 28 | |||
| 29 | static int __must_check prime_alloc_pages_locked(struct ivpu_bo *bo) | ||
| 30 | { | ||
| 31 | /* Pages are managed by the underlying dma-buf */ | ||
| 32 | return 0; | ||
| 33 | } | ||
| 34 | |||
| 35 | static void prime_free_pages_locked(struct ivpu_bo *bo) | ||
| 36 | { | ||
| 37 | /* Pages are managed by the underlying dma-buf */ | ||
| 38 | } | ||
| 39 | |||
| 40 | static int prime_map_pages_locked(struct ivpu_bo *bo) | ||
| 41 | { | ||
| 42 | struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); | ||
| 43 | struct sg_table *sgt; | ||
| 44 | |||
| 45 | sgt = dma_buf_map_attachment_unlocked(bo->base.import_attach, DMA_BIDIRECTIONAL); | ||
| 46 | if (IS_ERR(sgt)) { | ||
| 47 | ivpu_err(vdev, "Failed to map attachment: %ld\n", PTR_ERR(sgt)); | ||
| 48 | return PTR_ERR(sgt); | ||
| 49 | } | ||
| 50 | |||
| 51 | bo->sgt = sgt; | ||
| 52 | return 0; | ||
| 53 | } | ||
| 54 | |||
| 55 | static void prime_unmap_pages_locked(struct ivpu_bo *bo) | ||
| 56 | { | ||
| 57 | dma_buf_unmap_attachment_unlocked(bo->base.import_attach, bo->sgt, DMA_BIDIRECTIONAL); | ||
| 58 | bo->sgt = NULL; | ||
| 59 | } | ||
| 60 | |||
| 61 | static const struct ivpu_bo_ops prime_ops = { | ||
| 62 | .type = IVPU_BO_TYPE_PRIME, | ||
| 63 | .name = "prime", | ||
| 64 | .alloc_pages = prime_alloc_pages_locked, | ||
| 65 | .free_pages = prime_free_pages_locked, | ||
| 66 | .map_pages = prime_map_pages_locked, | ||
| 67 | .unmap_pages = prime_unmap_pages_locked, | ||
| 68 | }; | ||
| 69 | |||
| 70 | static int __must_check shmem_alloc_pages_locked(struct ivpu_bo *bo) | ||
| 71 | { | ||
| 72 | int npages = ivpu_bo_size(bo) >> PAGE_SHIFT; | ||
| 73 | struct page **pages; | ||
| 74 | |||
| 75 | pages = drm_gem_get_pages(&bo->base); | ||
| 76 | if (IS_ERR(pages)) | ||
| 77 | return PTR_ERR(pages); | ||
| 78 | |||
| 79 | if (bo->flags & DRM_IVPU_BO_WC) | ||
| 80 | set_pages_array_wc(pages, npages); | ||
| 81 | else if (bo->flags & DRM_IVPU_BO_UNCACHED) | ||
| 82 | set_pages_array_uc(pages, npages); | ||
| 83 | |||
| 84 | bo->pages = pages; | ||
| 85 | return 0; | ||
| 86 | } | ||
| 87 | |||
| 88 | static void shmem_free_pages_locked(struct ivpu_bo *bo) | ||
| 89 | { | ||
| 90 | if (ivpu_bo_cache_mode(bo) != DRM_IVPU_BO_CACHED) | ||
| 91 | set_pages_array_wb(bo->pages, ivpu_bo_size(bo) >> PAGE_SHIFT); | ||
| 92 | |||
| 93 | drm_gem_put_pages(&bo->base, bo->pages, true, false); | ||
| 94 | bo->pages = NULL; | ||
| 95 | } | ||
| 96 | |||
| 97 | static int ivpu_bo_map_pages_locked(struct ivpu_bo *bo) | ||
| 98 | { | ||
| 99 | int npages = ivpu_bo_size(bo) >> PAGE_SHIFT; | ||
| 100 | struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); | ||
| 101 | struct sg_table *sgt; | ||
| 102 | int ret; | ||
| 103 | |||
| 104 | sgt = drm_prime_pages_to_sg(&vdev->drm, bo->pages, npages); | ||
| 105 | if (IS_ERR(sgt)) { | ||
| 106 | ivpu_err(vdev, "Failed to allocate sgtable\n"); | ||
| 107 | return PTR_ERR(sgt); | ||
| 108 | } | ||
| 109 | |||
| 110 | ret = dma_map_sgtable(vdev->drm.dev, sgt, DMA_BIDIRECTIONAL, 0); | ||
| 111 | if (ret) { | ||
| 112 | ivpu_err(vdev, "Failed to map BO in IOMMU: %d\n", ret); | ||
| 113 | goto err_free_sgt; | ||
| 114 | } | ||
| 115 | |||
| 116 | bo->sgt = sgt; | ||
| 117 | return 0; | ||
| 118 | |||
| 119 | err_free_sgt: | ||
| 120 | kfree(sgt); | ||
| 121 | return ret; | ||
| 122 | } | ||
| 123 | |||
| 124 | static void ivpu_bo_unmap_pages_locked(struct ivpu_bo *bo) | ||
| 125 | { | ||
| 126 | struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); | ||
| 127 | |||
| 128 | dma_unmap_sgtable(vdev->drm.dev, bo->sgt, DMA_BIDIRECTIONAL, 0); | ||
| 129 | sg_free_table(bo->sgt); | ||
| 130 | kfree(bo->sgt); | ||
| 131 | bo->sgt = NULL; | ||
| 132 | } | ||
| 133 | |||
| 134 | static const struct ivpu_bo_ops shmem_ops = { | ||
| 135 | .type = IVPU_BO_TYPE_SHMEM, | ||
| 136 | .name = "shmem", | ||
| 137 | .alloc_pages = shmem_alloc_pages_locked, | ||
| 138 | .free_pages = shmem_free_pages_locked, | ||
| 139 | .map_pages = ivpu_bo_map_pages_locked, | ||
| 140 | .unmap_pages = ivpu_bo_unmap_pages_locked, | ||
| 141 | }; | ||
| 142 | |||
| 143 | static int __must_check internal_alloc_pages_locked(struct ivpu_bo *bo) | ||
| 144 | { | ||
| 145 | unsigned int i, npages = ivpu_bo_size(bo) >> PAGE_SHIFT; | ||
| 146 | struct page **pages; | ||
| 147 | int ret; | ||
| 148 | |||
| 149 | pages = kvmalloc_array(npages, sizeof(*bo->pages), GFP_KERNEL); | ||
| 150 | if (!pages) | ||
| 151 | return -ENOMEM; | ||
| 152 | |||
| 153 | for (i = 0; i < npages; i++) { | ||
| 154 | pages[i] = alloc_page(GFP_KERNEL | __GFP_HIGHMEM | __GFP_ZERO); | ||
| 155 | if (!pages[i]) { | ||
| 156 | ret = -ENOMEM; | ||
| 157 | goto err_free_pages; | ||
| 158 | } | ||
| 159 | cond_resched(); | ||
| 160 | } | ||
| 161 | |||
| 162 | bo->pages = pages; | ||
| 163 | return 0; | ||
| 164 | |||
| 165 | err_free_pages: | ||
| 166 | while (i--) | ||
| 167 | put_page(pages[i]); | ||
| 168 | kvfree(pages); | ||
| 169 | return ret; | ||
| 170 | } | ||
| 171 | |||
| 172 | static void internal_free_pages_locked(struct ivpu_bo *bo) | ||
| 173 | { | ||
| 174 | unsigned int i, npages = ivpu_bo_size(bo) >> PAGE_SHIFT; | ||
| 175 | |||
| 176 | if (ivpu_bo_cache_mode(bo) != DRM_IVPU_BO_CACHED) | ||
| 177 | set_pages_array_wb(bo->pages, ivpu_bo_size(bo) >> PAGE_SHIFT); | ||
| 178 | |||
| 179 | for (i = 0; i < npages; i++) | ||
| 180 | put_page(bo->pages[i]); | ||
| 181 | |||
| 182 | kvfree(bo->pages); | ||
| 183 | bo->pages = NULL; | ||
| 184 | } | ||
| 185 | |||
| 186 | static const struct ivpu_bo_ops internal_ops = { | ||
| 187 | .type = IVPU_BO_TYPE_INTERNAL, | ||
| 188 | .name = "internal", | ||
| 189 | .alloc_pages = internal_alloc_pages_locked, | ||
| 190 | .free_pages = internal_free_pages_locked, | ||
| 191 | .map_pages = ivpu_bo_map_pages_locked, | ||
| 192 | .unmap_pages = ivpu_bo_unmap_pages_locked, | ||
| 193 | }; | ||
| 194 | |||
| 195 | static int __must_check ivpu_bo_alloc_and_map_pages_locked(struct ivpu_bo *bo) | ||
| 196 | { | ||
| 197 | struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); | ||
| 198 | int ret; | ||
| 199 | |||
| 200 | lockdep_assert_held(&bo->lock); | ||
| 201 | drm_WARN_ON(&vdev->drm, bo->sgt); | ||
| 202 | |||
| 203 | ret = bo->ops->alloc_pages(bo); | ||
| 204 | if (ret) { | ||
| 205 | ivpu_err(vdev, "Failed to allocate pages for BO: %d", ret); | ||
| 206 | return ret; | ||
| 207 | } | ||
| 208 | |||
| 209 | ret = bo->ops->map_pages(bo); | ||
| 210 | if (ret) { | ||
| 211 | ivpu_err(vdev, "Failed to map pages for BO: %d", ret); | ||
| 212 | goto err_free_pages; | ||
| 213 | } | ||
| 214 | return ret; | ||
| 215 | |||
| 216 | err_free_pages: | ||
| 217 | bo->ops->free_pages(bo); | ||
| 218 | return ret; | ||
| 219 | } | ||
| 220 | |||
| 221 | static void ivpu_bo_unmap_and_free_pages(struct ivpu_bo *bo) | ||
| 222 | { | 26 | { |
| 223 | mutex_lock(&bo->lock); | 27 | if (bo->ctx) |
| 224 | 28 | ivpu_dbg(vdev, BO, "%6s: size %zu has_pages %d dma_mapped %d handle %u ctx %d vpu_addr 0x%llx mmu_mapped %d\n", | |
| 225 | WARN_ON(!bo->sgt); | 29 | action, ivpu_bo_size(bo), (bool)bo->base.pages, (bool)bo->base.sgt, |
| 226 | bo->ops->unmap_pages(bo); | 30 | bo->handle, bo->ctx->id, bo->vpu_addr, bo->mmu_mapped); |
| 227 | WARN_ON(bo->sgt); | 31 | else |
| 228 | bo->ops->free_pages(bo); | 32 | ivpu_dbg(vdev, BO, "%6s: size %zu has_pages %d dma_mapped %d handle %u (not added to context)\n", |
| 229 | WARN_ON(bo->pages); | 33 | action, ivpu_bo_size(bo), (bool)bo->base.pages, (bool)bo->base.sgt, |
| 230 | 34 | bo->handle); | |
| 231 | mutex_unlock(&bo->lock); | ||
| 232 | } | 35 | } |
| 233 | 36 | ||
| 234 | /* | 37 | /* |
| @@ -245,21 +48,24 @@ int __must_check ivpu_bo_pin(struct ivpu_bo *bo) · 10 unmodified lines | |||
| 245 | 48 | ||
| 246 | mutex_lock(&bo->lock); | 49 | mutex_lock(&bo->lock); |
| 247 | 50 | ||
| 248 | if (!bo->vpu_addr) { | 51 | ivpu_dbg_bo(vdev, bo, "pin"); |
| 249 | ivpu_err(vdev, "vpu_addr not set for BO ctx_id: %d handle: %d\n", | 52 | |
| 250 | bo->ctx->id, bo->handle); | 53 | if (!bo->ctx) { |
| 54 | ivpu_err(vdev, "vpu_addr not allocated for BO %d\n", bo->handle); | ||
| 251 | ret = -EINVAL; | 55 | ret = -EINVAL; |
| 252 | goto unlock; | 56 | goto unlock; |
| 253 | } | 57 | } |
| 254 | 58 | ||
| 255 | if (!bo->sgt) { | 59 | if (!bo->mmu_mapped) { |
| 256 | ret = ivpu_bo_alloc_and_map_pages_locked(bo); | 60 | struct sg_table *sgt = drm_gem_shmem_get_pages_sgt(&bo->base); |
| 257 | if (ret) | 61 | |
| 62 | if (IS_ERR(sgt)) { | ||
| 63 | ret = PTR_ERR(sgt); | ||
| 64 | ivpu_err(vdev, "Failed to map BO in IOMMU: %d\n", ret); | ||
| 258 | goto unlock; | 65 | goto unlock; |
| 259 | } | 66 | } |
| 260 | 67 | ||
| 261 | if (!bo->mmu_mapped) { | 68 | ret = ivpu_mmu_context_map_sgt(vdev, bo->ctx, bo->vpu_addr, sgt, |
| 262 | ret = ivpu_mmu_context_map_sgt(vdev, bo->ctx, bo->vpu_addr, bo->sgt, | ||
| 263 | ivpu_bo_is_snooped(bo)); | 69 | ivpu_bo_is_snooped(bo)); |
| 264 | if (ret) { | 70 | if (ret) { |
| 265 | ivpu_err(vdev, "Failed to map BO in MMU: %d\n", ret); | 71 | ivpu_err(vdev, "Failed to map BO in MMU: %d\n", ret); |
| @@ -281,248 +87,213 @@ ivpu_bo_alloc_vpu_addr(struct ivpu_bo *bo, struct ivpu_mmu_context *ctx, · 15 unmodified lines | |||
| 281 | struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); | 87 | struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); |
| 282 | int ret; | 88 | int ret; |
| 283 | 89 | ||
| 284 | if (!range) { | 90 | mutex_lock(&bo->lock); |
| 285 | if (bo->flags & DRM_IVPU_BO_SHAVE_MEM) | ||
| 286 | range = &vdev->hw->ranges.shave; | ||
| 287 | else if (bo->flags & DRM_IVPU_BO_DMA_MEM) | ||
| 288 | range = &vdev->hw->ranges.dma; | ||
| 289 | else | ||
| 290 | range = &vdev->hw->ranges.user; | ||
| 291 | } | ||
| 292 | 91 | ||
| 293 | mutex_lock(&ctx->lock); | 92 | ret = ivpu_mmu_context_insert_node(ctx, range, ivpu_bo_size(bo), &bo->mm_node); |
| 294 | ret = ivpu_mmu_context_insert_node_locked(ctx, range, ivpu_bo_size(bo), &bo->mm_node); | ||
| 295 | if (!ret) { | 93 | if (!ret) { |
| 296 | bo->ctx = ctx; | 94 | bo->ctx = ctx; |
| 297 | bo->vpu_addr = bo->mm_node.start; | 95 | bo->vpu_addr = bo->mm_node.start; |
| 298 | list_add_tail(&bo->ctx_node, &ctx->bo_list); | 96 | } else { |
| 97 | ivpu_err(vdev, "Failed to add BO to context %u: %d\n", ctx->id, ret); | ||
| 299 | } | 98 | } |
| 300 | mutex_unlock(&ctx->lock); | 99 | |
| 100 | ivpu_dbg_bo(vdev, bo, "alloc"); | ||
| 101 | |||
| 102 | mutex_unlock(&bo->lock); | ||
| 301 | 103 | ||
| 302 | return ret; | 104 | return ret; |
| 303 | } | 105 | } |
| 304 | 106 | ||
| 305 | static void ivpu_bo_free_vpu_addr(struct ivpu_bo *bo) | 107 | static void ivpu_bo_unbind_locked(struct ivpu_bo *bo) |
| 306 | { | 108 | { |
| 307 | struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); | 109 | struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); |
| 308 | struct ivpu_mmu_context *ctx = bo->ctx; | ||
| 309 | 110 | ||
| 310 | ivpu_dbg(vdev, BO, "remove from ctx: ctx %d vpu_addr 0x%llx allocated %d mmu_mapped %d\n", | 111 | lockdep_assert_held(&bo->lock); |
| 311 | ctx->id, bo->vpu_addr, (bool)bo->sgt, bo->mmu_mapped); | ||
| 312 | 112 | ||
| 313 | mutex_lock(&bo->lock); | 113 | ivpu_dbg_bo(vdev, bo, "unbind"); |
| 114 | |||
| 115 | /* TODO: dma_unmap */ | ||
| 314 | 116 | ||
| 315 | if (bo->mmu_mapped) { | 117 | if (bo->mmu_mapped) { |
| 316 | drm_WARN_ON(&vdev->drm, !bo->sgt); | 118 | drm_WARN_ON(&vdev->drm, !bo->ctx); |
| 317 | ivpu_mmu_context_unmap_sgt(vdev, ctx, bo->vpu_addr, bo->sgt); | 119 | drm_WARN_ON(&vdev->drm, !bo->vpu_addr); |
| 120 | drm_WARN_ON(&vdev->drm, !bo->base.sgt); | ||
| 121 | ivpu_mmu_context_unmap_sgt(vdev, bo->ctx, bo->vpu_addr, bo->base.sgt); | ||
| 318 | bo->mmu_mapped = false; | 122 | bo->mmu_mapped = false; |
| 319 | } | 123 | } |
| 320 | 124 | ||
| 321 | mutex_lock(&ctx->lock); | 125 | if (bo->ctx) { |
| 322 | list_del(&bo->ctx_node); | 126 | ivpu_mmu_context_remove_node(bo->ctx, &bo->mm_node); |
| 323 | bo->vpu_addr = 0; | 127 | bo->vpu_addr = 0; |
| 324 | bo->ctx = NULL; | 128 | bo->ctx = NULL; |
| 325 | ivpu_mmu_context_remove_node_locked(ctx, &bo->mm_node); | 129 | } |
| 326 | mutex_unlock(&ctx->lock); | 130 | } |
| 327 | 131 | ||
| 132 | static void ivpu_bo_unbind(struct ivpu_bo *bo) | ||
| 133 | { | ||
| 134 | mutex_lock(&bo->lock); | ||
| 135 | ivpu_bo_unbind_locked(bo); | ||
| 328 | mutex_unlock(&bo->lock); | 136 | mutex_unlock(&bo->lock); |
| 329 | } | 137 | } |
| 330 | 138 | ||
| 331 | void ivpu_bo_remove_all_bos_from_context(struct ivpu_mmu_context *ctx) | 139 | void ivpu_bo_remove_all_bos_from_context(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx) |
| 332 | { | 140 | { |
| 333 | struct ivpu_bo *bo, *tmp; | 141 | struct ivpu_bo *bo; |
| 142 | |||
| 143 | if (drm_WARN_ON(&vdev->drm, !ctx)) | ||
| 144 | return; | ||
| 334 | 145 | ||
| 335 | list_for_each_entry_safe(bo, tmp, &ctx->bo_list, ctx_node) | 146 | mutex_lock(&vdev->bo_list_lock); |
| 336 | ivpu_bo_free_vpu_addr(bo); | 147 | list_for_each_entry(bo, &vdev->bo_list, bo_list_node) { |
| 148 | mutex_lock(&bo->lock); | ||
| 149 | if (bo->ctx == ctx) | ||
| 150 | ivpu_bo_unbind_locked(bo); | ||
| 151 | mutex_unlock(&bo->lock); | ||
| 152 | } | ||
| 153 | mutex_unlock(&vdev->bo_list_lock); | ||
| 337 | } | 154 | } |
| 338 | 155 | ||
| 339 | static struct ivpu_bo * | 156 | struct drm_gem_object *ivpu_gem_create_object(struct drm_device *dev, size_t size) |
| 340 | ivpu_bo_alloc(struct ivpu_device *vdev, struct ivpu_mmu_context *mmu_context, | ||
| 341 | u64 size, u32 flags, const struct ivpu_bo_ops *ops, | ||
| 342 | const struct ivpu_addr_range *range, u64 user_ptr) | ||
| 343 | { | 157 | { |
| 344 | struct ivpu_bo *bo; | 158 | struct ivpu_bo *bo; |
| 345 | int ret = 0; | ||
| 346 | |||
| 347 | if (drm_WARN_ON(&vdev->drm, size == 0 || !PAGE_ALIGNED(size))) | ||
| 348 | return ERR_PTR(-EINVAL); | ||
| 349 | 159 | ||
| 350 | switch (flags & DRM_IVPU_BO_CACHE_MASK) { | 160 | if (size == 0 || !PAGE_ALIGNED(size)) |
| 351 | case DRM_IVPU_BO_CACHED: | ||
| 352 | case DRM_IVPU_BO_UNCACHED: | ||
| 353 | case DRM_IVPU_BO_WC: | ||
| 354 | break; | ||
| 355 | default: | ||
| 356 | return ERR_PTR(-EINVAL); | 161 | return ERR_PTR(-EINVAL); |
| 357 | } | ||
| 358 | 162 | ||
| 359 | bo = kzalloc(sizeof(*bo), GFP_KERNEL); | 163 | bo = kzalloc(sizeof(*bo), GFP_KERNEL); |
| 360 | if (!bo) | 164 | if (!bo) |
| 361 | return ERR_PTR(-ENOMEM); | 165 | return ERR_PTR(-ENOMEM); |
| 362 | 166 | ||
| 363 | mutex_init(&bo->lock); | 167 | bo->base.base.funcs = &ivpu_gem_funcs; |
| 364 | bo->base.funcs = &ivpu_gem_funcs; | 168 | bo->base.pages_mark_dirty_on_put = true; /* VPU can dirty a BO anytime */ |
| 365 | bo->flags = flags; | ||
| 366 | bo->ops = ops; | ||
| 367 | bo->user_ptr = user_ptr; | ||
| 368 | |||
| 369 | if (ops->type == IVPU_BO_TYPE_SHMEM) | ||
| 370 | ret = drm_gem_object_init(&vdev->drm, &bo->base, size); | ||
| 371 | else | ||
| 372 | drm_gem_private_object_init(&vdev->drm, &bo->base, size); | ||
| 373 | |||
| 374 | if (ret) { | ||
| 375 | ivpu_err(vdev, "Failed to initialize drm object\n"); | ||
| 376 | goto err_free; | ||
| 377 | } | ||
| 378 | |||
| 379 | if (flags & DRM_IVPU_BO_MAPPABLE) { | ||
| 380 | ret = drm_gem_create_mmap_offset(&bo->base); | ||
| 381 | if (ret) { | ||
| 382 | ivpu_err(vdev, "Failed to allocate mmap offset\n"); | ||
| 383 | goto err_release; | ||
| 384 | } | ||
| 385 | } | ||
| 386 | |||
| 387 | if (mmu_context) { | ||
| 388 | ret = ivpu_bo_alloc_vpu_addr(bo, mmu_context, range); | ||
| 389 | if (ret) { | ||
| 390 | ivpu_err(vdev, "Failed to add BO to context: %d\n", ret); | ||
| 391 | goto err_release; | ||
| 392 | } | ||
| 393 | } | ||
| 394 | 169 | ||
| 395 | return bo; | 170 | INIT_LIST_HEAD(&bo->bo_list_node); |
| 171 | mutex_init(&bo->lock); | ||
| 396 | 172 | ||
| 397 | err_release: | 173 | return &bo->base.base; |
| 398 | drm_gem_object_release(&bo->base); | ||
| 399 | err_free: | ||
| 400 | kfree(bo); | ||
| 401 | return ERR_PTR(ret); | ||
| 402 | } | 174 | } |
| 403 | 175 | ||
| 404 | static void ivpu_bo_free(struct drm_gem_object *obj) | 176 | static struct ivpu_bo * |
| 177 | ivpu_bo_create(struct ivpu_device *vdev, u64 size, u32 flags) | ||
| 405 | { | 178 | { |
| 406 | struct ivpu_bo *bo = to_ivpu_bo(obj); | 179 | struct drm_gem_shmem_object *shmem; |
| 407 | struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); | 180 | struct ivpu_bo *bo; |
| 408 | |||
| 409 | if (bo->ctx) | ||
| 410 | ivpu_dbg(vdev, BO, "free: ctx %d vpu_addr 0x%llx allocated %d mmu_mapped %d\n", | ||
| 411 | bo->ctx->id, bo->vpu_addr, (bool)bo->sgt, bo->mmu_mapped); | ||
| 412 | else | ||
| 413 | ivpu_dbg(vdev, BO, "free: ctx (released) allocated %d mmu_mapped %d\n", | ||
| 414 | (bool)bo->sgt, bo->mmu_mapped); | ||
| 415 | |||
| 416 | drm_WARN_ON(&vdev->drm, !dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_READ)); | ||
| 417 | 181 | ||
| 418 | vunmap(bo->kvaddr); | 182 | switch (flags & DRM_IVPU_BO_CACHE_MASK) { |
| 183 | case DRM_IVPU_BO_CACHED: | ||
| 184 | case DRM_IVPU_BO_WC: | ||
| 185 | break; | ||
| 186 | default: | ||
| 187 | return ERR_PTR(-EINVAL); | ||
| 188 | } | ||
| 419 | 189 | ||
| 420 | if (bo->ctx) | 190 | shmem = drm_gem_shmem_create(&vdev->drm, size); |
| 421 | ivpu_bo_free_vpu_addr(bo); | 191 | if (IS_ERR(shmem)) |
| 192 | return ERR_CAST(shmem); | ||
| 422 | 193 | ||
| 423 | if (bo->sgt) | 194 | bo = to_ivpu_bo(&shmem->base); |
| 424 | ivpu_bo_unmap_and_free_pages(bo); | 195 | bo->base.map_wc = flags & DRM_IVPU_BO_WC; |
| 196 | bo->flags = flags; | ||
| 425 | 197 | ||
| 426 | if (bo->base.import_attach) | 198 | mutex_lock(&vdev->bo_list_lock); |
| 427 | drm_prime_gem_destroy(&bo->base, bo->sgt); | 199 | list_add_tail(&bo->bo_list_node, &vdev->bo_list); |
| 200 | mutex_unlock(&vdev->bo_list_lock); | ||
| 428 | 201 | ||
| 429 | drm_gem_object_release(&bo->base); | 202 | ivpu_dbg(vdev, BO, "create: vpu_addr 0x%llx size %zu flags 0x%x\n", |
| 203 | bo->vpu_addr, bo->base.base.size, flags); | ||
| 430 | 204 | ||
| 431 | mutex_destroy(&bo->lock); | 205 | return bo; |
| 432 | kfree(bo); | ||
| 433 | } | 206 | } |
| 434 | 207 | ||
| 435 | static int ivpu_bo_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) | 208 | static int ivpu_bo_open(struct drm_gem_object *obj, struct drm_file *file) |
| 436 | { | 209 | { |
| 210 | struct ivpu_file_priv *file_priv = file->driver_priv; | ||
| 211 | struct ivpu_device *vdev = file_priv->vdev; | ||
| 437 | struct ivpu_bo *bo = to_ivpu_bo(obj); | 212 | struct ivpu_bo *bo = to_ivpu_bo(obj); |
| 438 | struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); | 213 | struct ivpu_addr_range *range; |
| 439 | |||
| 440 | ivpu_dbg(vdev, BO, "mmap: ctx %u handle %u vpu_addr 0x%llx size %zu type %s", | ||
| 441 | bo->ctx->id, bo->handle, bo->vpu_addr, ivpu_bo_size(bo), bo->ops->name); | ||
| 442 | 214 | ||
| 443 | if (obj->import_attach) { | 215 | if (bo->flags & DRM_IVPU_BO_SHAVE_MEM) |
| 444 | /* Drop the reference drm_gem_mmap_obj() acquired.*/ | 216 | range = &vdev->hw->ranges.shave; |
| 445 | drm_gem_object_put(obj); | 217 | else if (bo->flags & DRM_IVPU_BO_DMA_MEM) |
| 446 | vma->vm_private_data = NULL; | 218 | range = &vdev->hw->ranges.dma; |
| 447 | return dma_buf_mmap(obj->dma_buf, vma, 0); | 219 | else |
| 448 | } | 220 | range = &vdev->hw->ranges.user; |
| 449 | |||
| 450 | vm_flags_set(vma, VM_PFNMAP | VM_DONTEXPAND); | ||
| 451 | vma->vm_page_prot = ivpu_bo_pgprot(bo, vm_get_page_prot(vma->vm_flags)); | ||
| 452 | 221 | ||
| 453 | return 0; | 222 | return ivpu_bo_alloc_vpu_addr(bo, &file_priv->ctx, range); |
| 454 | } | 223 | } |
| 455 | 224 | ||
| 456 | static struct sg_table *ivpu_bo_get_sg_table(struct drm_gem_object *obj) | 225 | static void ivpu_bo_free(struct drm_gem_object *obj) |
| 457 | { | 226 | { |
| 227 | struct ivpu_device *vdev = to_ivpu_device(obj->dev); | ||
| 458 | struct ivpu_bo *bo = to_ivpu_bo(obj); | 228 | struct ivpu_bo *bo = to_ivpu_bo(obj); |
| 459 | loff_t npages = obj->size >> PAGE_SHIFT; | ||
| 460 | int ret = 0; | ||
| 461 | 229 | ||
| 462 | mutex_lock(&bo->lock); | 230 | mutex_lock(&vdev->bo_list_lock); |
| 231 | list_del(&bo->bo_list_node); | ||
| 232 | mutex_unlock(&vdev->bo_list_lock); | ||
| 463 | 233 | ||
| 464 | if (!bo->sgt) | 234 | drm_WARN_ON(&vdev->drm, !dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_READ)); |
| 465 | ret = ivpu_bo_alloc_and_map_pages_locked(bo); | ||
| 466 | 235 | ||
| 467 | mutex_unlock(&bo->lock); | 236 | ivpu_dbg_bo(vdev, bo, "free"); |
| 468 | 237 | ||
| 469 | if (ret) | 238 | ivpu_bo_unbind(bo); |
| 470 | return ERR_PTR(ret); | 239 | mutex_destroy(&bo->lock); |
| 471 | 240 | ||
| 472 | return drm_prime_pages_to_sg(obj->dev, bo->pages, npages); | 241 | drm_WARN_ON(obj->dev, bo->base.pages_use_count > 1); |
| 242 | drm_gem_shmem_free(&bo->base); | ||
| 473 | } | 243 | } |
| 474 | 244 | ||
| 475 | static vm_fault_t ivpu_vm_fault(struct vm_fault *vmf) | 245 | static const struct dma_buf_ops ivpu_bo_dmabuf_ops = { |
| 476 | { | 246 | .cache_sgt_mapping = true, |
| 477 | struct vm_area_struct *vma = vmf->vma; | 247 | .attach = drm_gem_map_attach, |
| 478 | struct drm_gem_object *obj = vma->vm_private_data; | 248 | .detach = drm_gem_map_detach, |
| 479 | struct ivpu_bo *bo = to_ivpu_bo(obj); | 249 | .map_dma_buf = drm_gem_map_dma_buf, |
| 480 | loff_t npages = obj->size >> PAGE_SHIFT; | 250 | .unmap_dma_buf = drm_gem_unmap_dma_buf, |
| 481 | pgoff_t page_offset; | 251 | .release = drm_gem_dmabuf_release, |
| 482 | struct page *page; | 252 | .mmap = drm_gem_dmabuf_mmap, |
| 483 | vm_fault_t ret; | 253 | .vmap = drm_gem_dmabuf_vmap, |
| 484 | int err; | 254 | .vunmap = drm_gem_dmabuf_vunmap, |
| 485 | 255 | }; | |
| 486 | mutex_lock(&bo->lock); | ||
| 487 | |||
| 488 | if (!bo->sgt) { | ||
| 489 | err = ivpu_bo_alloc_and_map_pages_locked(bo); | ||
| 490 | if (err) { | ||
| 491 | ret = vmf_error(err); | ||
| 492 | goto unlock; | ||
| 493 | } | ||
| 494 | } | ||
| 495 | |||
| 496 | /* We don't use vmf->pgoff since that has the fake offset */ | ||
| 497 | page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT; | ||
| 498 | if (page_offset >= npages) { | ||
| 499 | ret = VM_FAULT_SIGBUS; | ||
| 500 | } else { | ||
| 501 | page = bo->pages[page_offset]; | ||
| 502 | ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page)); | ||
| 503 | } | ||
| 504 | |||
| 505 | unlock: | ||
| 506 | mutex_unlock(&bo->lock); | ||
| 507 | 256 | ||
| 508 | return ret; | 257 | static struct dma_buf *ivpu_bo_export(struct drm_gem_object *obj, int flags) |
| 258 | { | ||
| 259 | struct drm_device *dev = obj->dev; | ||
| 260 | struct dma_buf_export_info exp_info = { | ||
| 261 | .exp_name = KBUILD_MODNAME, | ||
| 262 | .owner = dev->driver->fops->owner, | ||
| 263 | .ops = &ivpu_bo_dmabuf_ops, | ||
| 264 | .size = obj->size, | ||
| 265 | .flags = flags, | ||
| 266 | .priv = obj, | ||
| 267 | .resv = obj->resv, | ||
| 268 | }; | ||
| 269 | void *sgt; | ||
| 270 | |||
| 271 | /* | ||
| 272 | * Make sure that pages are allocated and dma-mapped before exporting the bo. | ||
| 273 | * DMA-mapping is required if the bo will be imported to the same device. | ||
| 274 | */ | ||
| 275 | sgt = drm_gem_shmem_get_pages_sgt(to_drm_gem_shmem_obj(obj)); | ||
| 276 | if (IS_ERR(sgt)) | ||
| 277 | return sgt; | ||
| 278 | |||
| 279 | return drm_gem_dmabuf_export(dev, &exp_info); | ||
| 509 | } | 280 | } |
| 510 | 281 | ||
| 511 | static const struct vm_operations_struct ivpu_vm_ops = { | ||
| 512 | .fault = ivpu_vm_fault, | ||
| 513 | .open = drm_gem_vm_open, | ||
| 514 | .close = drm_gem_vm_close, | ||
| 515 | }; | ||
| 516 | |||
| 517 | static const struct drm_gem_object_funcs ivpu_gem_funcs = { | 282 | static const struct drm_gem_object_funcs ivpu_gem_funcs = { |
| 518 | .free = ivpu_bo_free, | 283 | .free = ivpu_bo_free, |
| 519 | .mmap = ivpu_bo_mmap, | 284 | .open = ivpu_bo_open, |
| 520 | .vm_ops = &ivpu_vm_ops, | 285 | .export = ivpu_bo_export, |
| 521 | .get_sg_table = ivpu_bo_get_sg_table, | 286 | .print_info = drm_gem_shmem_object_print_info, |
| 287 | .pin = drm_gem_shmem_object_pin, | ||
| 288 | .unpin = drm_gem_shmem_object_unpin, | ||
| 289 | .get_sg_table = drm_gem_shmem_object_get_sg_table, | ||
| 290 | .vmap = drm_gem_shmem_object_vmap, | ||
| 291 | .vunmap = drm_gem_shmem_object_vunmap, | ||
| 292 | .mmap = drm_gem_shmem_object_mmap, | ||
| 293 | .vm_ops = &drm_gem_shmem_vm_ops, | ||
| 522 | }; | 294 | }; |
| 523 | 295 | ||
| 524 | int | 296 | int ivpu_bo_create_ioctl(struct drm_device *dev, void *data, struct drm_file *file) |
| 525 | ivpu_bo_create_ioctl(struct drm_device *dev, void *data, struct drm_file *file) | ||
| 526 | { | 297 | { |
| 527 | struct ivpu_file_priv *file_priv = file->driver_priv; | 298 | struct ivpu_file_priv *file_priv = file->driver_priv; |
| 528 | struct ivpu_device *vdev = file_priv->vdev; | 299 | struct ivpu_device *vdev = file_priv->vdev; |
| @@ -537,23 +308,20 @@ ivpu_bo_create_ioctl(struct drm_device *dev, void *data, struct drm_file *file) · 8 unmodified lines | |||
| 537 | if (size == 0) | 308 | if (size == 0) |
| 538 | return -EINVAL; | 309 | return -EINVAL; |
| 539 | 310 | ||
| 540 | bo = ivpu_bo_alloc(vdev, &file_priv->ctx, size, args->flags, &shmem_ops, NULL, 0); | 311 | bo = ivpu_bo_create(vdev, size, args->flags); |
| 541 | if (IS_ERR(bo)) { | 312 | if (IS_ERR(bo)) { |
| 542 | ivpu_err(vdev, "Failed to create BO: %pe (ctx %u size %llu flags 0x%x)", | 313 | ivpu_err(vdev, "Failed to create BO: %pe (ctx %u size %llu flags 0x%x)", |
| 543 | bo, file_priv->ctx.id, args->size, args->flags); | 314 | bo, file_priv->ctx.id, args->size, args->flags); |
| 544 | return PTR_ERR(bo); | 315 | return PTR_ERR(bo); |
| 545 | } | 316 | } |
| 546 | 317 | ||
| 547 | ret = drm_gem_handle_create(file, &bo->base, &bo->handle); | 318 | ret = drm_gem_handle_create(file, &bo->base.base, &bo->handle); |
| 548 | if (!ret) { | 319 | if (!ret) { |
| 549 | args->vpu_addr = bo->vpu_addr; | 320 | args->vpu_addr = bo->vpu_addr; |
| 550 | args->handle = bo->handle; | 321 | args->handle = bo->handle; |
| 551 | } | 322 | } |
| 552 | 323 | ||
| 553 | drm_gem_object_put(&bo->base); | 324 | drm_gem_object_put(&bo->base.base); |
| 554 | |||
| 555 | ivpu_dbg(vdev, BO, "alloc shmem: ctx %u vpu_addr 0x%llx size %zu flags 0x%x\n", | ||
| 556 | file_priv->ctx.id, bo->vpu_addr, ivpu_bo_size(bo), bo->flags); | ||
| 557 | 325 | ||
| 558 | return ret; | 326 | return ret; |
| 559 | } | 327 | } |
| @@ -563,8 +331,8 @@ ivpu_bo_alloc_internal(struct ivpu_device *vdev, u64 vpu_addr, u64 size, u32 fla · 3 unmodified lines | |||
| 563 | { | 331 | { |
| 564 | const struct ivpu_addr_range *range; | 332 | const struct ivpu_addr_range *range; |
| 565 | struct ivpu_addr_range fixed_range; | 333 | struct ivpu_addr_range fixed_range; |
| 334 | struct iosys_map map; | ||
| 566 | struct ivpu_bo *bo; | 335 | struct ivpu_bo *bo; |
| 567 | pgprot_t prot; | ||
| 568 | int ret; | 336 | int ret; |
| 569 | 337 | ||
| 570 | drm_WARN_ON(&vdev->drm, !PAGE_ALIGNED(vpu_addr)); | 338 | drm_WARN_ON(&vdev->drm, !PAGE_ALIGNED(vpu_addr)); |
| @@ -578,81 +346,42 @@ ivpu_bo_alloc_internal(struct ivpu_device *vdev, u64 vpu_addr, u64 size, u32 fla · 7 unmodified lines | |||
| 578 | range = &vdev->hw->ranges.global; | 346 | range = &vdev->hw->ranges.global; |
| 579 | } | 347 | } |
| 580 | 348 | ||
| 581 | bo = ivpu_bo_alloc(vdev, &vdev->gctx, size, flags, &internal_ops, range, 0); | 349 | bo = ivpu_bo_create(vdev, size, flags); |
| 582 | if (IS_ERR(bo)) { | 350 | if (IS_ERR(bo)) { |
| 583 | ivpu_err(vdev, "Failed to create BO: %pe (vpu_addr 0x%llx size %llu flags 0x%x)", | 351 | ivpu_err(vdev, "Failed to create BO: %pe (vpu_addr 0x%llx size %llu flags 0x%x)", |
| 584 | bo, vpu_addr, size, flags); | 352 | bo, vpu_addr, size, flags); |
| 585 | return NULL; | 353 | return NULL; |
| 586 | } | 354 | } |
| 587 | 355 | ||
| 588 | ret = ivpu_bo_pin(bo); | 356 | ret = ivpu_bo_alloc_vpu_addr(bo, &vdev->gctx, range); |
| 589 | if (ret) | 357 | if (ret) |
| 590 | goto err_put; | 358 | goto err_put; |
| 591 | 359 | ||
| 592 | if (ivpu_bo_cache_mode(bo) != DRM_IVPU_BO_CACHED) | 360 | ret = ivpu_bo_pin(bo); |
| 593 | drm_clflush_pages(bo->pages, ivpu_bo_size(bo) >> PAGE_SHIFT); | 361 | if (ret) |
| 594 | |||
| 595 | if (bo->flags & DRM_IVPU_BO_WC) | ||
| 596 | set_pages_array_wc(bo->pages, ivpu_bo_size(bo) >> PAGE_SHIFT); | ||
| 597 | else if (bo->flags & DRM_IVPU_BO_UNCACHED) | ||
| 598 | set_pages_array_uc(bo->pages, ivpu_bo_size(bo) >> PAGE_SHIFT); | ||
| 599 | |||
| 600 | prot = ivpu_bo_pgprot(bo, PAGE_KERNEL); | ||
| 601 | bo->kvaddr = vmap(bo->pages, ivpu_bo_size(bo) >> PAGE_SHIFT, VM_MAP, prot); | ||
| 602 | if (!bo->kvaddr) { | ||
| 603 | ivpu_err(vdev, "Failed to map BO into kernel virtual memory\n"); | ||
| 604 | goto err_put; | 362 | goto err_put; |
| 605 | } | ||
| 606 | 363 | ||
| 607 | ivpu_dbg(vdev, BO, "alloc internal: ctx 0 vpu_addr 0x%llx size %zu flags 0x%x\n", | 364 | ret = drm_gem_shmem_vmap(&bo->base, &map); |
| 608 | bo->vpu_addr, ivpu_bo_size(bo), flags); | 365 | if (ret) |
| 366 | goto err_put; | ||
| 609 | 367 | ||
| 610 | return bo; | 368 | return bo; |
| 611 | 369 | ||
| 612 | err_put: | 370 | err_put: |
| 613 | drm_gem_object_put(&bo->base); | 371 | drm_gem_object_put(&bo->base.base); |
| 614 | return NULL; | 372 | return NULL; |
| 615 | } | 373 | } |
| 616 | 374 | ||
| 617 | void ivpu_bo_free_internal(struct ivpu_bo *bo) | 375 | void ivpu_bo_free_internal(struct ivpu_bo *bo) |
| 618 | { | 376 | { |
| 619 | drm_gem_object_put(&bo->base); | 377 | struct iosys_map map = IOSYS_MAP_INIT_VADDR(bo->base.vaddr); |
| 620 | } | ||
| 621 | |||
| 622 | struct drm_gem_object *ivpu_gem_prime_import(struct drm_device *dev, struct dma_buf *buf) | ||
| 623 | { | ||
| 624 | struct ivpu_device *vdev = to_ivpu_device(dev); | ||
| 625 | struct dma_buf_attachment *attach; | ||
| 626 | struct ivpu_bo *bo; | ||
| 627 | |||
| 628 | attach = dma_buf_attach(buf, dev->dev); | ||
| 629 | if (IS_ERR(attach)) | ||
| 630 | return ERR_CAST(attach); | ||
| 631 | |||
| 632 | get_dma_buf(buf); | ||
| 633 | 378 | ||
| 634 | bo = ivpu_bo_alloc(vdev, NULL, buf->size, DRM_IVPU_BO_MAPPABLE, &prime_ops, NULL, 0); | 379 | drm_gem_shmem_vunmap(&bo->base, &map); |
| 635 | if (IS_ERR(bo)) { | 380 | drm_gem_object_put(&bo->base.base); |
| 636 | ivpu_err(vdev, "Failed to import BO: %pe (size %lu)", bo, buf->size); | ||
| 637 | goto err_detach; | ||
| 638 | } | ||
| 639 | |||
| 640 | lockdep_set_class(&bo->lock, &prime_bo_lock_class_key); | ||
| 641 | |||
| 642 | bo->base.import_attach = attach; | ||
| 643 | |||
| 644 | return &bo->base; | ||
| 645 | |||
| 646 | err_detach: | ||
| 647 | dma_buf_detach(buf, attach); | ||
| 648 | dma_buf_put(buf); | ||
| 649 | return ERR_CAST(bo); | ||
| 650 | } | 381 | } |
| 651 | 382 | ||
| 652 | int ivpu_bo_info_ioctl(struct drm_device *dev, void *data, struct drm_file *file) | 383 | int ivpu_bo_info_ioctl(struct drm_device *dev, void *data, struct drm_file *file) |
| 653 | { | 384 | { |
| 654 | struct ivpu_file_priv *file_priv = file->driver_priv; | ||
| 655 | struct ivpu_device *vdev = to_ivpu_device(dev); | ||
| 656 | struct drm_ivpu_bo_info *args = data; | 385 | struct drm_ivpu_bo_info *args = data; |
| 657 | struct drm_gem_object *obj; | 386 | struct drm_gem_object *obj; |
| 658 | struct ivpu_bo *bo; | 387 | struct ivpu_bo *bo; |
| @@ -665,21 +394,12 @@ int ivpu_bo_info_ioctl(struct drm_device *dev, void *data, struct drm_file *file · 6 unmodified lines | |||
| 665 | bo = to_ivpu_bo(obj); | 394 | bo = to_ivpu_bo(obj); |
| 666 | 395 | ||
| 667 | mutex_lock(&bo->lock); | 396 | mutex_lock(&bo->lock); |
| 668 | |||
| 669 | if (!bo->ctx) { | ||
| 670 | ret = ivpu_bo_alloc_vpu_addr(bo, &file_priv->ctx, NULL); | ||
| 671 | if (ret) { | ||
| 672 | ivpu_err(vdev, "Failed to allocate vpu_addr: %d\n", ret); | ||
| 673 | goto unlock; | ||
| 674 | } | ||
| 675 | } | ||
| 676 | |||
| 677 | args->flags = bo->flags; | 397 | args->flags = bo->flags; |
| 678 | args->mmap_offset = drm_vma_node_offset_addr(&obj->vma_node); | 398 | args->mmap_offset = drm_vma_node_offset_addr(&obj->vma_node); |
| 679 | args->vpu_addr = bo->vpu_addr; | 399 | args->vpu_addr = bo->vpu_addr; |
| 680 | args->size = obj->size; | 400 | args->size = obj->size; |
| 681 | unlock: | ||
| 682 | mutex_unlock(&bo->lock); | 401 | mutex_unlock(&bo->lock); |
| 402 | |||
| 683 | drm_gem_object_put(obj); | 403 | drm_gem_object_put(obj); |
| 684 | return ret; | 404 | return ret; |
| 685 | } | 405 | } |
| @@ -714,41 +434,41 @@ static void ivpu_bo_print_info(struct ivpu_bo *bo, struct drm_printer *p) · 28 unmodified lines | |||
| 714 | { | 434 | { |
| 715 | unsigned long dma_refcount = 0; | 435 | unsigned long dma_refcount = 0; |
| 716 | 436 | ||
| 717 | if (bo->base.dma_buf && bo->base.dma_buf->file) | 437 | mutex_lock(&bo->lock); |
| 718 | dma_refcount = atomic_long_read(&bo->base.dma_buf->file->f_count); | 438 | |
| 439 | if (bo->base.base.dma_buf && bo->base.base.dma_buf->file) | ||
| 440 | dma_refcount = atomic_long_read(&bo->base.base.dma_buf->file->f_count); | ||
| 441 | |||
| 442 | drm_printf(p, "%-3u %-6d 0x%-12llx %-10lu 0x%-8x %-4u %-8lu", | ||
| 443 | bo->ctx->id, bo->handle, bo->vpu_addr, bo->base.base.size, | ||
| 444 | bo->flags, kref_read(&bo->base.base.refcount), dma_refcount); | ||
| 445 | |||
| 446 | if (bo->base.base.import_attach) | ||
| 447 | drm_printf(p, " imported"); | ||
| 448 | |||
| 449 | if (bo->base.pages) | ||
| 450 | drm_printf(p, " has_pages"); | ||
| 719 | 451 | ||
| 720 | drm_printf(p, "%5u %6d %16llx %10lu %10u %12lu %14s\n", | 452 | if (bo->mmu_mapped) |
| 721 | bo->ctx->id, bo->handle, bo->vpu_addr, ivpu_bo_size(bo), | 453 | drm_printf(p, " mmu_mapped"); |
| 722 | kref_read(&bo->base.refcount), dma_refcount, bo->ops->name); | 454 | |
| 455 | drm_printf(p, "\n"); | ||
| 456 | |||
| 457 | mutex_unlock(&bo->lock); | ||
| 723 | } | 458 | } |
| 724 | 459 | ||
| 725 | void ivpu_bo_list(struct drm_device *dev, struct drm_printer *p) | 460 | void ivpu_bo_list(struct drm_device *dev, struct drm_printer *p) |
| 726 | { | 461 | { |
| 727 | struct ivpu_device *vdev = to_ivpu_device(dev); | 462 | struct ivpu_device *vdev = to_ivpu_device(dev); |
| 728 | struct ivpu_file_priv *file_priv; | ||
| 729 | unsigned long ctx_id; | ||
| 730 | struct ivpu_bo *bo; | 463 | struct ivpu_bo *bo; |
| 731 | 464 | ||
| 732 | drm_printf(p, "%5s %6s %16s %10s %10s %12s %14s\n", | 465 | drm_printf(p, "%-3s %-6s %-14s %-10s %-10s %-4s %-8s %s\n", |
| 733 | "ctx", "handle", "vpu_addr", "size", "refcount", "dma_refcount", "type"); | 466 | "ctx", "handle", "vpu_addr", "size", "flags", "refs", "dma_refs", "attribs"); |
| 734 | 467 | ||
| 735 | mutex_lock(&vdev->gctx.lock); | 468 | mutex_lock(&vdev->bo_list_lock); |
| 736 | list_for_each_entry(bo, &vdev->gctx.bo_list, ctx_node) | 469 | list_for_each_entry(bo, &vdev->bo_list, bo_list_node) |
| 737 | ivpu_bo_print_info(bo, p); | 470 | ivpu_bo_print_info(bo, p); |
| 738 | mutex_unlock(&vdev->gctx.lock); | 471 | mutex_unlock(&vdev->bo_list_lock); |
| 739 | |||
| 740 | xa_for_each(&vdev->context_xa, ctx_id, file_priv) { | ||
| 741 | file_priv = ivpu_file_priv_get_by_ctx_id(vdev, ctx_id); | ||
| 742 | if (!file_priv) | ||
| 743 | continue; | ||
| 744 | |||
| 745 | mutex_lock(&file_priv->ctx.lock); | ||
| 746 | list_for_each_entry(bo, &file_priv->ctx.bo_list, ctx_node) | ||
| 747 | ivpu_bo_print_info(bo, p); | ||
| 748 | mutex_unlock(&file_priv->ctx.lock); | ||
| 749 | |||
| 750 | ivpu_file_priv_put(&file_priv); | ||
| 751 | } | ||
| 752 | } | 472 | } |
| 753 | 473 | ||
| 754 | void ivpu_bo_list_print(struct drm_device *dev) | 474 | void ivpu_bo_list_print(struct drm_device *dev) |
| @@ -6,84 +6,52 @@ · 5 unmodified lines | |||
| 6 | #define __IVPU_GEM_H__ | 6 | #define __IVPU_GEM_H__ |
| 7 | 7 | ||
| 8 | #include <drm/drm_gem.h> | 8 | #include <drm/drm_gem.h> |
| 9 | #include <drm/drm_gem_shmem_helper.h> | ||
| 9 | #include <drm/drm_mm.h> | 10 | #include <drm/drm_mm.h> |
| 10 | 11 | ||
| 11 | struct dma_buf; | ||
| 12 | struct ivpu_bo_ops; | ||
| 13 | struct ivpu_file_priv; | 12 | struct ivpu_file_priv; |
| 14 | 13 | ||
| 15 | struct ivpu_bo { | 14 | struct ivpu_bo { |
| 16 | struct drm_gem_object base; | 15 | struct drm_gem_shmem_object base; |
| 17 | const struct ivpu_bo_ops *ops; | ||
| 18 | |||
| 19 | struct ivpu_mmu_context *ctx; | 16 | struct ivpu_mmu_context *ctx; |
| 20 | struct list_head ctx_node; | 17 | struct list_head bo_list_node; |
| 21 | struct drm_mm_node mm_node; | 18 | struct drm_mm_node mm_node; |
| 22 | 19 | ||
| 23 | struct mutex lock; /* Protects: pages, sgt, mmu_mapped */ | 20 | struct mutex lock; /* Protects: ctx, mmu_mapped, vpu_addr */ |
| 24 | struct sg_table *sgt; | ||
| 25 | struct page **pages; | ||
| 26 | bool mmu_mapped; | ||
| 27 | |||
| 28 | void *kvaddr; | ||
| 29 | u64 vpu_addr; | 21 | u64 vpu_addr; |
| 30 | u32 handle; | 22 | u32 handle; |
| 31 | u32 flags; | 23 | u32 flags; |
| 32 | uintptr_t user_ptr; | 24 | u32 job_status; /* Valid only for command buffer */ |
| 33 | u32 job_status; | 25 | bool mmu_mapped; |
| 34 | }; | ||
| 35 | |||
| 36 | enum ivpu_bo_type { | ||
| 37 | IVPU_BO_TYPE_SHMEM = 1, | ||
| 38 | IVPU_BO_TYPE_INTERNAL, | ||
| 39 | IVPU_BO_TYPE_PRIME, | ||
| 40 | }; | ||
| 41 | |||
| 42 | struct ivpu_bo_ops { | ||
| 43 | enum ivpu_bo_type type; | ||
| 44 | const char *name; | ||
| 45 | int (*alloc_pages)(struct ivpu_bo *bo); | ||
| 46 | void (*free_pages)(struct ivpu_bo *bo); | ||
| 47 | int (*map_pages)(struct ivpu_bo *bo); | ||
| 48 | void (*unmap_pages)(struct ivpu_bo *bo); | ||
| 49 | }; | 26 | }; |
| 50 | 27 | ||
| 51 | int ivpu_bo_pin(struct ivpu_bo *bo); | 28 | int ivpu_bo_pin(struct ivpu_bo *bo); |
| 52 | void ivpu_bo_remove_all_bos_from_context(struct ivpu_mmu_context *ctx); | 29 | void ivpu_bo_remove_all_bos_from_context(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx); |
| 53 | void ivpu_bo_list(struct drm_device *dev, struct drm_printer *p); | ||
| 54 | void ivpu_bo_list_print(struct drm_device *dev); | ||
| 55 | 30 | ||
| 56 | struct ivpu_bo * | 31 | struct drm_gem_object *ivpu_gem_create_object(struct drm_device *dev, size_t size); |
| 57 | ivpu_bo_alloc_internal(struct ivpu_device *vdev, u64 vpu_addr, u64 size, u32 flags); | 32 | struct ivpu_bo *ivpu_bo_alloc_internal(struct ivpu_device *vdev, u64 vpu_addr, u64 size, u32 flags); |
| 58 | void ivpu_bo_free_internal(struct ivpu_bo *bo); | 33 | void ivpu_bo_free_internal(struct ivpu_bo *bo); |
| 59 | struct drm_gem_object *ivpu_gem_prime_import(struct drm_device *dev, struct dma_buf *dma_buf); | ||
| 60 | void ivpu_bo_unmap_sgt_and_remove_from_context(struct ivpu_bo *bo); | ||
| 61 | 34 | ||
| 62 | int ivpu_bo_create_ioctl(struct drm_device *dev, void *data, struct drm_file *file); | 35 | int ivpu_bo_create_ioctl(struct drm_device *dev, void *data, struct drm_file *file); |
| 63 | int ivpu_bo_info_ioctl(struct drm_device *dev, void *data, struct drm_file *file); | 36 | int ivpu_bo_info_ioctl(struct drm_device *dev, void *data, struct drm_file *file); |
| 64 | int ivpu_bo_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file); | 37 | int ivpu_bo_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file); |
| 65 | 38 | ||
| 39 | void ivpu_bo_list(struct drm_device *dev, struct drm_printer *p); | ||
| 40 | void ivpu_bo_list_print(struct drm_device *dev); | ||
| 41 | |||
| 66 | static inline struct ivpu_bo *to_ivpu_bo(struct drm_gem_object *obj) | 42 | static inline struct ivpu_bo *to_ivpu_bo(struct drm_gem_object *obj) |
| 67 | { | 43 | { |
| 68 | return container_of(obj, struct ivpu_bo, base); | 44 | return container_of(obj, struct ivpu_bo, base.base); |
| 69 | } | 45 | } |
| 70 | 46 | ||
| 71 | static inline void *ivpu_bo_vaddr(struct ivpu_bo *bo) | 47 | static inline void *ivpu_bo_vaddr(struct ivpu_bo *bo) |
| 72 | { | 48 | { |
| 73 | return bo->kvaddr; | 49 | return bo->base.vaddr; |
| 74 | } | 50 | } |
| 75 | 51 | ||
| 76 | static inline size_t ivpu_bo_size(struct ivpu_bo *bo) | 52 | static inline size_t ivpu_bo_size(struct ivpu_bo *bo) |
| 77 | { | 53 | { |
| 78 | return bo->base.size; | 54 | return bo->base.base.size; |
| 79 | } | ||
| 80 | |||
| 81 | static inline struct page *ivpu_bo_get_page(struct ivpu_bo *bo, u64 offset) | ||
| 82 | { | ||
| 83 | if (offset > ivpu_bo_size(bo) || !bo->pages) | ||
| 84 | return NULL; | ||
| 85 | |||
| 86 | return bo->pages[offset / PAGE_SIZE]; | ||
| 87 | } | 55 | } |
| 88 | 56 | ||
| 89 | static inline u32 ivpu_bo_cache_mode(struct ivpu_bo *bo) | 57 | static inline u32 ivpu_bo_cache_mode(struct ivpu_bo *bo) |
| @@ -96,20 +64,9 @@ static inline bool ivpu_bo_is_snooped(struct ivpu_bo *bo) · 6 unmodified lines | |||
| 96 | return ivpu_bo_cache_mode(bo) == DRM_IVPU_BO_CACHED; | 64 | return ivpu_bo_cache_mode(bo) == DRM_IVPU_BO_CACHED; |
| 97 | } | 65 | } |
| 98 | 66 | ||
| 99 | static inline pgprot_t ivpu_bo_pgprot(struct ivpu_bo *bo, pgprot_t prot) | ||
| 100 | { | ||
| 101 | if (bo->flags & DRM_IVPU_BO_WC) | ||
| 102 | return pgprot_writecombine(prot); | ||
| 103 | |||
| 104 | if (bo->flags & DRM_IVPU_BO_UNCACHED) | ||
| 105 | return pgprot_noncached(prot); | ||
| 106 | |||
| 107 | return prot; | ||
| 108 | } | ||
| 109 | |||
| 110 | static inline struct ivpu_device *ivpu_bo_to_vdev(struct ivpu_bo *bo) | 67 | static inline struct ivpu_device *ivpu_bo_to_vdev(struct ivpu_bo *bo) |
| 111 | { | 68 | { |
| 112 | return to_ivpu_device(bo->base.dev); | 69 | return to_ivpu_device(bo->base.base.dev); |
| 113 | } | 70 | } |
| 114 | 71 | ||
| 115 | static inline void *ivpu_to_cpu_addr(struct ivpu_bo *bo, u32 vpu_addr) | 72 | static inline void *ivpu_to_cpu_addr(struct ivpu_bo *bo, u32 vpu_addr) |
| @@ -15,8 +15,11 @@ struct ivpu_hw_ops { · 14 unmodified lines | |||
| 15 | int (*power_down)(struct ivpu_device *vdev); | 15 | int (*power_down)(struct ivpu_device *vdev); |
| 16 | int (*reset)(struct ivpu_device *vdev); | 16 | int (*reset)(struct ivpu_device *vdev); |
| 17 | bool (*is_idle)(struct ivpu_device *vdev); | 17 | bool (*is_idle)(struct ivpu_device *vdev); |
| 18 | int (*wait_for_idle)(struct ivpu_device *vdev); | ||
| 18 | void (*wdt_disable)(struct ivpu_device *vdev); | 19 | void (*wdt_disable)(struct ivpu_device *vdev); |
| 19 | void (*diagnose_failure)(struct ivpu_device *vdev); | 20 | void (*diagnose_failure)(struct ivpu_device *vdev); |
| 21 | u32 (*profiling_freq_get)(struct ivpu_device *vdev); | ||
| 22 | void (*profiling_freq_drive)(struct ivpu_device *vdev, bool enable); | ||
| 20 | u32 (*reg_pll_freq_get)(struct ivpu_device *vdev); | 23 | u32 (*reg_pll_freq_get)(struct ivpu_device *vdev); |
| 21 | u32 (*reg_telemetry_offset_get)(struct ivpu_device *vdev); | 24 | u32 (*reg_telemetry_offset_get)(struct ivpu_device *vdev); |
| 22 | u32 (*reg_telemetry_size_get)(struct ivpu_device *vdev); | 25 | u32 (*reg_telemetry_size_get)(struct ivpu_device *vdev); |
| @@ -58,6 +61,8 @@ struct ivpu_hw_info { · 35 unmodified lines | |||
| 58 | u32 sku; | 61 | u32 sku; |
| 59 | u16 config; | 62 | u16 config; |
| 60 | int dma_bits; | 63 | int dma_bits; |
| 64 | ktime_t d0i3_entry_host_ts; | ||
| 65 | u64 d0i3_entry_vpu_ts; | ||
| 61 | }; | 66 | }; |
| 62 | 67 | ||
| 63 | extern const struct ivpu_hw_ops ivpu_hw_37xx_ops; | 68 | extern const struct ivpu_hw_ops ivpu_hw_37xx_ops; |
| @@ -85,6 +90,11 @@ static inline bool ivpu_hw_is_idle(struct ivpu_device *vdev) · 21 unmodified lines | |||
| 85 | return vdev->hw->ops->is_idle(vdev); | 90 | return vdev->hw->ops->is_idle(vdev); |
| 86 | }; | 91 | }; |
| 87 | 92 | ||
| 93 | static inline int ivpu_hw_wait_for_idle(struct ivpu_device *vdev) | ||
| 94 | { | ||
| 95 | return vdev->hw->ops->wait_for_idle(vdev); | ||
| 96 | }; | ||
| 97 | |||
| 88 | static inline int ivpu_hw_power_down(struct ivpu_device *vdev) | 98 | static inline int ivpu_hw_power_down(struct ivpu_device *vdev) |
| 89 | { | 99 | { |
| 90 | ivpu_dbg(vdev, PM, "HW power down\n"); | 100 | ivpu_dbg(vdev, PM, "HW power down\n"); |
| @@ -104,6 +114,16 @@ static inline void ivpu_hw_wdt_disable(struct ivpu_device *vdev) · 13 unmodified lines | |||
| 104 | vdev->hw->ops->wdt_disable(vdev); | 114 | vdev->hw->ops->wdt_disable(vdev); |
| 105 | }; | 115 | }; |
| 106 | 116 | ||
| 117 | static inline u32 ivpu_hw_profiling_freq_get(struct ivpu_device *vdev) | ||
| 118 | { | ||
| 119 | return vdev->hw->ops->profiling_freq_get(vdev); | ||
| 120 | }; | ||
| 121 | |||
| 122 | static inline void ivpu_hw_profiling_freq_drive(struct ivpu_device *vdev, bool enable) | ||
| 123 | { | ||
| 124 | return vdev->hw->ops->profiling_freq_drive(vdev, enable); | ||
| 125 | }; | ||
| 126 | |||
| 107 | /* Register indirect accesses */ | 127 | /* Register indirect accesses */ |
| 108 | static inline u32 ivpu_hw_reg_pll_freq_get(struct ivpu_device *vdev) | 128 | static inline u32 ivpu_hw_reg_pll_freq_get(struct ivpu_device *vdev) |
| 109 | { | 129 | { |
| @@ -29,6 +29,7 @@ · 28 unmodified lines | |||
| 29 | 29 | ||
| 30 | #define PLL_REF_CLK_FREQ (50 * 1000000) | 30 | #define PLL_REF_CLK_FREQ (50 * 1000000) |
| 31 | #define PLL_SIMULATION_FREQ (10 * 1000000) | 31 | #define PLL_SIMULATION_FREQ (10 * 1000000) |
| 32 | #define PLL_PROF_CLK_FREQ (38400 * 1000) | ||
| 32 | #define PLL_DEFAULT_EPP_VALUE 0x80 | 33 | #define PLL_DEFAULT_EPP_VALUE 0x80 |
| 33 | 34 | ||
| 34 | #define TIM_SAFE_ENABLE 0xf1d0dead | 35 | #define TIM_SAFE_ENABLE 0xf1d0dead |
| @@ -37,7 +38,7 @@ · 2 unmodified lines | |||
| 37 | #define TIMEOUT_US (150 * USEC_PER_MSEC) | 38 | #define TIMEOUT_US (150 * USEC_PER_MSEC) |
| 38 | #define PWR_ISLAND_STATUS_TIMEOUT_US (5 * USEC_PER_MSEC) | 39 | #define PWR_ISLAND_STATUS_TIMEOUT_US (5 * USEC_PER_MSEC) |
| 39 | #define PLL_TIMEOUT_US (1500 * USEC_PER_MSEC) | 40 | #define PLL_TIMEOUT_US (1500 * USEC_PER_MSEC) |
| 40 | #define IDLE_TIMEOUT_US (500 * USEC_PER_MSEC) | 41 | #define IDLE_TIMEOUT_US (5 * USEC_PER_MSEC) |
| 41 | 42 | ||
| 42 | #define ICB_0_IRQ_MASK ((REG_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, HOST_IPC_FIFO_INT)) | \ | 43 | #define ICB_0_IRQ_MASK ((REG_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, HOST_IPC_FIFO_INT)) | \ |
| 43 | (REG_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, MMU_IRQ_0_INT)) | \ | 44 | (REG_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, MMU_IRQ_0_INT)) | \ |
| @@ -90,6 +91,7 @@ static void ivpu_hw_timeouts_init(struct ivpu_device *vdev) · 46 unmodified lines | |||
| 90 | vdev->timeout.tdr = 2000; | 91 | vdev->timeout.tdr = 2000; |
| 91 | vdev->timeout.reschedule_suspend = 10; | 92 | vdev->timeout.reschedule_suspend = 10; |
| 92 | vdev->timeout.autosuspend = 10; | 93 | vdev->timeout.autosuspend = 10; |
| 94 | vdev->timeout.d0i3_entry_msg = 5; | ||
| 93 | } | 95 | } |
| 94 | 96 | ||
| 95 | static int ivpu_pll_wait_for_cmd_send(struct ivpu_device *vdev) | 97 | static int ivpu_pll_wait_for_cmd_send(struct ivpu_device *vdev) |
| @@ -502,6 +504,16 @@ static int ivpu_boot_pwr_domain_enable(struct ivpu_device *vdev) · 406 unmodified lines | |||
| 502 | return ret; | 504 | return ret; |
| 503 | } | 505 | } |
| 504 | 506 | ||
| 507 | static int ivpu_boot_pwr_domain_disable(struct ivpu_device *vdev) | ||
| 508 | { | ||
| 509 | ivpu_boot_dpu_active_drive(vdev, false); | ||
| 510 | ivpu_boot_pwr_island_isolation_drive(vdev, true); | ||
| 511 | ivpu_boot_pwr_island_trickle_drive(vdev, false); | ||
| 512 | ivpu_boot_pwr_island_drive(vdev, false); | ||
| 513 | |||
| 514 | return ivpu_boot_wait_for_pwr_island_status(vdev, 0x0); | ||
| 515 | } | ||
| 516 | |||
| 505 | static void ivpu_boot_no_snoop_enable(struct ivpu_device *vdev) | 517 | static void ivpu_boot_no_snoop_enable(struct ivpu_device *vdev) |
| 506 | { | 518 | { |
| 507 | u32 val = REGV_RD32(VPU_37XX_HOST_IF_TCU_PTW_OVERRIDES); | 519 | u32 val = REGV_RD32(VPU_37XX_HOST_IF_TCU_PTW_OVERRIDES); |
| @@ -600,25 +612,17 @@ static int ivpu_hw_37xx_info_init(struct ivpu_device *vdev) · 92 unmodified lines | |||
| 600 | 612 | ||
| 601 | static int ivpu_hw_37xx_reset(struct ivpu_device *vdev) | 613 | static int ivpu_hw_37xx_reset(struct ivpu_device *vdev) |
| 602 | { | 614 | { |
| 603 | int ret; | 615 | int ret = 0; |
| 604 | u32 val; | ||
| 605 | |||
| 606 | if (IVPU_WA(punit_disabled)) | ||
| 607 | return 0; | ||
| 608 | 616 | ||
| 609 | ret = REGB_POLL_FLD(VPU_37XX_BUTTRESS_VPU_IP_RESET, TRIGGER, 0, TIMEOUT_US); | 617 | if (ivpu_boot_pwr_domain_disable(vdev)) { |
| 610 | if (ret) { | 618 | ivpu_err(vdev, "Failed to disable power domain\n"); |
| 611 | ivpu_err(vdev, "Timed out waiting for TRIGGER bit\n"); | 619 | ret = -EIO; |
| 612 | return ret; | ||
| 613 | } | 620 | } |
| 614 | 621 | ||
| 615 | val = REGB_RD32(VPU_37XX_BUTTRESS_VPU_IP_RESET); | 622 | if (ivpu_pll_disable(vdev)) { |
| 616 | val = REG_SET_FLD(VPU_37XX_BUTTRESS_VPU_IP_RESET, TRIGGER, val); | 623 | ivpu_err(vdev, "Failed to disable PLL\n"); |
| 617 | REGB_WR32(VPU_37XX_BUTTRESS_VPU_IP_RESET, val); | 624 | ret = -EIO; |
| 618 | 625 | } | |
| 619 | ret = REGB_POLL_FLD(VPU_37XX_BUTTRESS_VPU_IP_RESET, TRIGGER, 0, TIMEOUT_US); | ||
| 620 | if (ret) | ||
| 621 | ivpu_err(vdev, "Timed out waiting for RESET completion\n"); | ||
| 622 | 626 | ||
| 623 | return ret; | 627 | return ret; |
| 624 | } | 628 | } |
| @@ -651,10 +655,6 @@ static int ivpu_hw_37xx_power_up(struct ivpu_device *vdev) · 26 unmodified lines | |||
| 651 | { | 655 | { |
| 652 | int ret; | 656 | int ret; |
| 653 | 657 | ||
| 654 | ret = ivpu_hw_37xx_reset(vdev); | ||
| 655 | if (ret) | ||
| 656 | ivpu_warn(vdev, "Failed to reset HW: %d\n", ret); | ||
| 657 | |||
| 658 | ret = ivpu_hw_37xx_d0i3_disable(vdev); | 658 | ret = ivpu_hw_37xx_d0i3_disable(vdev); |
| 659 | if (ret) | 659 | if (ret) |
| 660 | ivpu_warn(vdev, "Failed to disable D0I3: %d\n", ret); | 660 | ivpu_warn(vdev, "Failed to disable D0I3: %d\n", ret); |
| @@ -718,15 +718,28 @@ static bool ivpu_hw_37xx_is_idle(struct ivpu_device *vdev) · 57 unmodified lines | |||
| 718 | REG_TEST_FLD(VPU_37XX_BUTTRESS_VPU_STATUS, IDLE, val); | 718 | REG_TEST_FLD(VPU_37XX_BUTTRESS_VPU_STATUS, IDLE, val); |
| 719 | } | 719 | } |
| 720 | 720 | ||
| 721 | static int ivpu_hw_37xx_wait_for_idle(struct ivpu_device *vdev) | ||
| 722 | { | ||
| 723 | return REGB_POLL_FLD(VPU_37XX_BUTTRESS_VPU_STATUS, IDLE, 0x1, IDLE_TIMEOUT_US); | ||
| 724 | } | ||
| 725 | |||
| 726 | static void ivpu_hw_37xx_save_d0i3_entry_timestamp(struct ivpu_device *vdev) | ||
| 727 | { | ||
| 728 | vdev->hw->d0i3_entry_host_ts = ktime_get_boottime(); | ||
| 729 | vdev->hw->d0i3_entry_vpu_ts = REGV_RD64(VPU_37XX_CPU_SS_TIM_PERF_FREE_CNT); | ||
| 730 | } | ||
| 731 | |||
| 721 | static int ivpu_hw_37xx_power_down(struct ivpu_device *vdev) | 732 | static int ivpu_hw_37xx_power_down(struct ivpu_device *vdev) |
| 722 | { | 733 | { |
| 723 | int ret = 0; | 734 | int ret = 0; |
| 724 | 735 | ||
| 725 | if (!ivpu_hw_37xx_is_idle(vdev) && ivpu_hw_37xx_reset(vdev)) | 736 | ivpu_hw_37xx_save_d0i3_entry_timestamp(vdev); |
| 726 | ivpu_err(vdev, "Failed to reset the VPU\n"); | ||
| 727 | 737 | ||
| 728 | if (ivpu_pll_disable(vdev)) { | 738 | if (!ivpu_hw_37xx_is_idle(vdev)) |
| 729 | ivpu_err(vdev, "Failed to disable PLL\n"); | 739 | ivpu_warn(vdev, "VPU not idle during power down\n"); |
| 740 | |||
| 741 | if (ivpu_hw_37xx_reset(vdev)) { | ||
| 742 | ivpu_err(vdev, "Failed to reset VPU\n"); | ||
| 730 | ret = -EIO; | 743 | ret = -EIO; |
| 731 | } | 744 | } |
| 732 | 745 | ||
| @@ -756,6 +769,16 @@ static void ivpu_hw_37xx_wdt_disable(struct ivpu_device *vdev) · 23 unmodified lines | |||
| 756 | REGV_WR32(VPU_37XX_CPU_SS_TIM_GEN_CONFIG, val); | 769 | REGV_WR32(VPU_37XX_CPU_SS_TIM_GEN_CONFIG, val); |
| 757 | } | 770 | } |
| 758 | 771 | ||
| 772 | static u32 ivpu_hw_37xx_profiling_freq_get(struct ivpu_device *vdev) | ||
| 773 | { | ||
| 774 | return PLL_PROF_CLK_FREQ; | ||
| 775 | } | ||
| 776 | |||
| 777 | static void ivpu_hw_37xx_profiling_freq_drive(struct ivpu_device *vdev, bool enable) | ||
| 778 | { | ||
| 779 | /* Profiling freq - is a debug feature. Unavailable on VPU 37XX. */ | ||
| 780 | } | ||
| 781 | |||
| 759 | static u32 ivpu_hw_37xx_pll_to_freq(u32 ratio, u32 config) | 782 | static u32 ivpu_hw_37xx_pll_to_freq(u32 ratio, u32 config) |
| 760 | { | 783 | { |
| 761 | u32 pll_clock = PLL_REF_CLK_FREQ * ratio; | 784 | u32 pll_clock = PLL_REF_CLK_FREQ * ratio; |
| @@ -867,17 +890,20 @@ static void ivpu_hw_37xx_irq_noc_firewall_handler(struct ivpu_device *vdev) · 105 unmodified lines | |||
| 867 | } | 890 | } |
| 868 | 891 | ||
| 869 | /* Handler for IRQs from VPU core (irqV) */ | 892 | /* Handler for IRQs from VPU core (irqV) */ |
| 870 | static u32 ivpu_hw_37xx_irqv_handler(struct ivpu_device *vdev, int irq) | 893 | static bool ivpu_hw_37xx_irqv_handler(struct ivpu_device *vdev, int irq, bool *wake_thread) |
| 871 | { | 894 | { |
| 872 | u32 status = REGV_RD32(VPU_37XX_HOST_SS_ICB_STATUS_0) & ICB_0_IRQ_MASK; | 895 | u32 status = REGV_RD32(VPU_37XX_HOST_SS_ICB_STATUS_0) & ICB_0_IRQ_MASK; |
| 873 | 896 | ||
| 897 | if (!status) | ||
| 898 | return false; | ||
| 899 | |||
| 874 | REGV_WR32(VPU_37XX_HOST_SS_ICB_CLEAR_0, status); | 900 | REGV_WR32(VPU_37XX_HOST_SS_ICB_CLEAR_0, status); |
| 875 | 901 | ||
| 876 | if (REG_TEST_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, MMU_IRQ_0_INT, status)) | 902 | if (REG_TEST_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, MMU_IRQ_0_INT, status)) |
| 877 | ivpu_mmu_irq_evtq_handler(vdev); | 903 | ivpu_mmu_irq_evtq_handler(vdev); |
| 878 | 904 | ||
| 879 | if (REG_TEST_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, HOST_IPC_FIFO_INT, status)) | 905 | if (REG_TEST_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, HOST_IPC_FIFO_INT, status)) |
| 880 | ivpu_ipc_irq_handler(vdev); | 906 | ivpu_ipc_irq_handler(vdev, wake_thread); |
| 881 | 907 | ||
| 882 | if (REG_TEST_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, MMU_IRQ_1_INT, status)) | 908 | if (REG_TEST_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, MMU_IRQ_1_INT, status)) |
| 883 | ivpu_dbg(vdev, IRQ, "MMU sync complete\n"); | 909 | ivpu_dbg(vdev, IRQ, "MMU sync complete\n"); |
| @@ -894,17 +920,17 @@ static u32 ivpu_hw_37xx_irqv_handler(struct ivpu_device *vdev, int irq) · 10 unmodified lines | |||
| 894 | if (REG_TEST_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, NOC_FIREWALL_INT, status)) | 920 | if (REG_TEST_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, NOC_FIREWALL_INT, status)) |
| 895 | ivpu_hw_37xx_irq_noc_firewall_handler(vdev); | 921 | ivpu_hw_37xx_irq_noc_firewall_handler(vdev); |
| 896 | 922 | ||
| 897 | return status; | 923 | return true; |
| 898 | } | 924 | } |
| 899 | 925 | ||
| 900 | /* Handler for IRQs from Buttress core (irqB) */ | 926 | /* Handler for IRQs from Buttress core (irqB) */ |
| 901 | static u32 ivpu_hw_37xx_irqb_handler(struct ivpu_device *vdev, int irq) | 927 | static bool ivpu_hw_37xx_irqb_handler(struct ivpu_device *vdev, int irq) |
| 902 | { | 928 | { |
| 903 | u32 status = REGB_RD32(VPU_37XX_BUTTRESS_INTERRUPT_STAT) & BUTTRESS_IRQ_MASK; | 929 | u32 status = REGB_RD32(VPU_37XX_BUTTRESS_INTERRUPT_STAT) & BUTTRESS_IRQ_MASK; |
| 904 | bool schedule_recovery = false; | 930 | bool schedule_recovery = false; |
| 905 | 931 | ||
| 906 | if (status == 0) | 932 | if (!status) |
| 907 | return 0; | 933 | return false; |
| 908 | 934 | ||
| 909 | if (REG_TEST_FLD(VPU_37XX_BUTTRESS_INTERRUPT_STAT, FREQ_CHANGE, status)) | 935 | if (REG_TEST_FLD(VPU_37XX_BUTTRESS_INTERRUPT_STAT, FREQ_CHANGE, status)) |
| 910 | ivpu_dbg(vdev, IRQ, "FREQ_CHANGE irq: %08x", | 936 | ivpu_dbg(vdev, IRQ, "FREQ_CHANGE irq: %08x", |
| @@ -940,23 +966,27 @@ static u32 ivpu_hw_37xx_irqb_handler(struct ivpu_device *vdev, int irq) · 29 unmodified lines | |||
| 940 | if (schedule_recovery) | 966 | if (schedule_recovery) |
| 941 | ivpu_pm_schedule_recovery(vdev); | 967 | ivpu_pm_schedule_recovery(vdev); |
| 942 | 968 | ||
| 943 | return status; | 969 | return true; |
| 944 | } | 970 | } |
| 945 | 971 | ||
| 946 | static irqreturn_t ivpu_hw_37xx_irq_handler(int irq, void *ptr) | 972 | static irqreturn_t ivpu_hw_37xx_irq_handler(int irq, void *ptr) |
| 947 | { | 973 | { |
| 948 | struct ivpu_device *vdev = ptr; | 974 | struct ivpu_device *vdev = ptr; |
| 949 | u32 ret_irqv, ret_irqb; | 975 | bool irqv_handled, irqb_handled, wake_thread = false; |
| 950 | 976 | ||
| 951 | REGB_WR32(VPU_37XX_BUTTRESS_GLOBAL_INT_MASK, 0x1); | 977 | REGB_WR32(VPU_37XX_BUTTRESS_GLOBAL_INT_MASK, 0x1); |
| 952 | 978 | ||
| 953 | ret_irqv = ivpu_hw_37xx_irqv_handler(vdev, irq); | 979 | irqv_handled = ivpu_hw_37xx_irqv_handler(vdev, irq, &wake_thread); |
| 954 | ret_irqb = ivpu_hw_37xx_irqb_handler(vdev, irq); | 980 | irqb_handled = ivpu_hw_37xx_irqb_handler(vdev, irq); |
| 955 | 981 | ||
| 956 | /* Re-enable global interrupts to re-trigger MSI for pending interrupts */ | 982 | /* Re-enable global interrupts to re-trigger MSI for pending interrupts */ |
| 957 | REGB_WR32(VPU_37XX_BUTTRESS_GLOBAL_INT_MASK, 0x0); | 983 | REGB_WR32(VPU_37XX_BUTTRESS_GLOBAL_INT_MASK, 0x0); |
| 958 | 984 | ||
| 959 | return IRQ_RETVAL(ret_irqb | ret_irqv); | 985 | if (wake_thread) |
| 986 | return IRQ_WAKE_THREAD; | ||
| 987 | if (irqv_handled || irqb_handled) | ||
| 988 | return IRQ_HANDLED; | ||
| 989 | return IRQ_NONE; | ||
| 960 | } | 990 | } |
| 961 | 991 | ||
| 962 | static void ivpu_hw_37xx_diagnose_failure(struct ivpu_device *vdev) | 992 | static void ivpu_hw_37xx_diagnose_failure(struct ivpu_device *vdev) |
| @@ -993,11 +1023,14 @@ const struct ivpu_hw_ops ivpu_hw_37xx_ops = { · 30 unmodified lines | |||
| 993 | .info_init = ivpu_hw_37xx_info_init, | 1023 | .info_init = ivpu_hw_37xx_info_init, |
| 994 | .power_up = ivpu_hw_37xx_power_up, | 1024 | .power_up = ivpu_hw_37xx_power_up, |
| 995 | .is_idle = ivpu_hw_37xx_is_idle, | 1025 | .is_idle = ivpu_hw_37xx_is_idle, |
| 1026 | .wait_for_idle = ivpu_hw_37xx_wait_for_idle, | ||
| 996 | .power_down = ivpu_hw_37xx_power_down, | 1027 | .power_down = ivpu_hw_37xx_power_down, |
| 997 | .reset = ivpu_hw_37xx_reset, | 1028 | .reset = ivpu_hw_37xx_reset, |
| 998 | .boot_fw = ivpu_hw_37xx_boot_fw, | 1029 | .boot_fw = ivpu_hw_37xx_boot_fw, |
| 999 | .wdt_disable = ivpu_hw_37xx_wdt_disable, | 1030 | .wdt_disable = ivpu_hw_37xx_wdt_disable, |
| 1000 | .diagnose_failure = ivpu_hw_37xx_diagnose_failure, | 1031 | .diagnose_failure = ivpu_hw_37xx_diagnose_failure, |
| 1032 | .profiling_freq_get = ivpu_hw_37xx_profiling_freq_get, | ||
| 1033 | .profiling_freq_drive = ivpu_hw_37xx_profiling_freq_drive, | ||
| 1001 | .reg_pll_freq_get = ivpu_hw_37xx_reg_pll_freq_get, | 1034 | .reg_pll_freq_get = ivpu_hw_37xx_reg_pll_freq_get, |
| 1002 | .reg_telemetry_offset_get = ivpu_hw_37xx_reg_telemetry_offset_get, | 1035 | .reg_telemetry_offset_get = ivpu_hw_37xx_reg_telemetry_offset_get, |
| 1003 | .reg_telemetry_size_get = ivpu_hw_37xx_reg_telemetry_size_get, | 1036 | .reg_telemetry_size_get = ivpu_hw_37xx_reg_telemetry_size_get, |
| @@ -240,6 +240,8 @@ · 239 unmodified lines | |||
| 240 | #define VPU_37XX_CPU_SS_TIM_GEN_CONFIG 0x06021008u | 240 | #define VPU_37XX_CPU_SS_TIM_GEN_CONFIG 0x06021008u |
| 241 | #define VPU_37XX_CPU_SS_TIM_GEN_CONFIG_WDOG_TO_INT_CLR_MASK BIT_MASK(9) | 241 | #define VPU_37XX_CPU_SS_TIM_GEN_CONFIG_WDOG_TO_INT_CLR_MASK BIT_MASK(9) |
| 242 | 242 | ||
| 243 | #define VPU_37XX_CPU_SS_TIM_PERF_FREE_CNT 0x06029000u | ||
| 244 | |||
| 243 | #define VPU_37XX_CPU_SS_DOORBELL_0 0x06300000u | 245 | #define VPU_37XX_CPU_SS_DOORBELL_0 0x06300000u |
| 244 | #define VPU_37XX_CPU_SS_DOORBELL_0_SET_MASK BIT_MASK(0) | 246 | #define VPU_37XX_CPU_SS_DOORBELL_0_SET_MASK BIT_MASK(0) |
| 245 | 247 | ||
| @@ -39,6 +39,7 @@ · 38 unmodified lines | |||
| 39 | #define TIMEOUT_US (150 * USEC_PER_MSEC) | 39 | #define TIMEOUT_US (150 * USEC_PER_MSEC) |
| 40 | #define PWR_ISLAND_STATUS_TIMEOUT_US (5 * USEC_PER_MSEC) | 40 | #define PWR_ISLAND_STATUS_TIMEOUT_US (5 * USEC_PER_MSEC) |
| 41 | #define PLL_TIMEOUT_US (1500 * USEC_PER_MSEC) | 41 | #define PLL_TIMEOUT_US (1500 * USEC_PER_MSEC) |
| 42 | #define IDLE_TIMEOUT_US (5 * USEC_PER_MSEC) | ||
| 42 | 43 | ||
| 43 | #define WEIGHTS_DEFAULT 0xf711f711u | 44 | #define WEIGHTS_DEFAULT 0xf711f711u |
| 44 | #define WEIGHTS_ATS_DEFAULT 0x0000f711u | 45 | #define WEIGHTS_ATS_DEFAULT 0x0000f711u |
| @@ -139,18 +140,21 @@ static void ivpu_hw_timeouts_init(struct ivpu_device *vdev) · 94 unmodified lines | |||
| 139 | vdev->timeout.tdr = 2000000; | 140 | vdev->timeout.tdr = 2000000; |
| 140 | vdev->timeout.reschedule_suspend = 1000; | 141 | vdev->timeout.reschedule_suspend = 1000; |
| 141 | vdev->timeout.autosuspend = -1; | 142 | vdev->timeout.autosuspend = -1; |
| 143 | vdev->timeout.d0i3_entry_msg = 500; | ||
| 142 | } else if (ivpu_is_simics(vdev)) { | 144 | } else if (ivpu_is_simics(vdev)) { |
| 143 | vdev->timeout.boot = 50; | 145 | vdev->timeout.boot = 50; |
| 144 | vdev->timeout.jsm = 500; | 146 | vdev->timeout.jsm = 500; |
| 145 | vdev->timeout.tdr = 10000; | 147 | vdev->timeout.tdr = 10000; |
| 146 | vdev->timeout.reschedule_suspend = 10; | 148 | vdev->timeout.reschedule_suspend = 10; |
| 147 | vdev->timeout.autosuspend = -1; | 149 | vdev->timeout.autosuspend = -1; |
| 150 | vdev->timeout.d0i3_entry_msg = 100; | ||
| 148 | } else { | 151 | } else { |
| 149 | vdev->timeout.boot = 1000; | 152 | vdev->timeout.boot = 1000; |
| 150 | vdev->timeout.jsm = 500; | 153 | vdev->timeout.jsm = 500; |
| 151 | vdev->timeout.tdr = 2000; | 154 | vdev->timeout.tdr = 2000; |
| 152 | vdev->timeout.reschedule_suspend = 10; | 155 | vdev->timeout.reschedule_suspend = 10; |
| 153 | vdev->timeout.autosuspend = 10; | 156 | vdev->timeout.autosuspend = 10; |
| 157 | vdev->timeout.d0i3_entry_msg = 5; | ||
| 154 | } | 158 | } |
| 155 | } | 159 | } |
| 156 | 160 | ||
| @@ -824,12 +828,6 @@ static int ivpu_hw_40xx_power_up(struct ivpu_device *vdev) · 667 unmodified lines | |||
| 824 | { | 828 | { |
| 825 | int ret; | 829 | int ret; |
| 826 | 830 | ||
| 827 | ret = ivpu_hw_40xx_reset(vdev); | ||
| 828 | if (ret) { | ||
| 829 | ivpu_err(vdev, "Failed to reset HW: %d\n", ret); | ||
| 830 | return ret; | ||
| 831 | } | ||
| 832 | |||
| 833 | ret = ivpu_hw_40xx_d0i3_disable(vdev); | 831 | ret = ivpu_hw_40xx_d0i3_disable(vdev); |
| 834 | if (ret) | 832 | if (ret) |
| 835 | ivpu_warn(vdev, "Failed to disable D0I3: %d\n", ret); | 833 | ivpu_warn(vdev, "Failed to disable D0I3: %d\n", ret); |
| @@ -898,10 +896,23 @@ static bool ivpu_hw_40xx_is_idle(struct ivpu_device *vdev) · 62 unmodified lines | |||
| 898 | REG_TEST_FLD(VPU_40XX_BUTTRESS_VPU_STATUS, IDLE, val); | 896 | REG_TEST_FLD(VPU_40XX_BUTTRESS_VPU_STATUS, IDLE, val); |
| 899 | } | 897 | } |
| 900 | 898 | ||
| 899 | static int ivpu_hw_40xx_wait_for_idle(struct ivpu_device *vdev) | ||
| 900 | { | ||
| 901 | return REGB_POLL_FLD(VPU_40XX_BUTTRESS_VPU_STATUS, IDLE, 0x1, IDLE_TIMEOUT_US); | ||
| 902 | } | ||
| 903 | |||
| 904 | static void ivpu_hw_40xx_save_d0i3_entry_timestamp(struct ivpu_device *vdev) | ||
| 905 | { | ||
| 906 | vdev->hw->d0i3_entry_host_ts = ktime_get_boottime(); | ||
| 907 | vdev->hw->d0i3_entry_vpu_ts = REGV_RD64(VPU_40XX_CPU_SS_TIM_PERF_EXT_FREE_CNT); | ||
| 908 | } | ||
| 909 | |||
| 901 | static int ivpu_hw_40xx_power_down(struct ivpu_device *vdev) | 910 | static int ivpu_hw_40xx_power_down(struct ivpu_device *vdev) |
| 902 | { | 911 | { |
| 903 | int ret = 0; | 912 | int ret = 0; |
| 904 | 913 | ||
| 914 | ivpu_hw_40xx_save_d0i3_entry_timestamp(vdev); | ||
| 915 | |||
| 905 | if (!ivpu_hw_40xx_is_idle(vdev) && ivpu_hw_40xx_reset(vdev)) | 916 | if (!ivpu_hw_40xx_is_idle(vdev) && ivpu_hw_40xx_reset(vdev)) |
| 906 | ivpu_warn(vdev, "Failed to reset the VPU\n"); | 917 | ivpu_warn(vdev, "Failed to reset the VPU\n"); |
| 907 | 918 | ||
| @@ -933,6 +944,19 @@ static void ivpu_hw_40xx_wdt_disable(struct ivpu_device *vdev) · 25 unmodified lines | |||
| 933 | REGV_WR32(VPU_40XX_CPU_SS_TIM_GEN_CONFIG, val); | 944 | REGV_WR32(VPU_40XX_CPU_SS_TIM_GEN_CONFIG, val); |
| 934 | } | 945 | } |
| 935 | 946 | ||
| 947 | static u32 ivpu_hw_40xx_profiling_freq_get(struct ivpu_device *vdev) | ||
| 948 | { | ||
| 949 | return vdev->hw->pll.profiling_freq; | ||
| 950 | } | ||
| 951 | |||
| 952 | static void ivpu_hw_40xx_profiling_freq_drive(struct ivpu_device *vdev, bool enable) | ||
| 953 | { | ||
| 954 | if (enable) | ||
| 955 | vdev->hw->pll.profiling_freq = PLL_PROFILING_FREQ_HIGH; | ||
| 956 | else | ||
| 957 | vdev->hw->pll.profiling_freq = PLL_PROFILING_FREQ_DEFAULT; | ||
| 958 | } | ||
| 959 | |||
| 936 | /* Register indirect accesses */ | 960 | /* Register indirect accesses */ |
| 937 | static u32 ivpu_hw_40xx_reg_pll_freq_get(struct ivpu_device *vdev) | 961 | static u32 ivpu_hw_40xx_reg_pll_freq_get(struct ivpu_device *vdev) |
| 938 | { | 962 | { |
| @@ -1023,13 +1047,12 @@ static void ivpu_hw_40xx_irq_noc_firewall_handler(struct ivpu_device *vdev) · 84 unmodified lines | |||
| 1023 | } | 1047 | } |
| 1024 | 1048 | ||
| 1025 | /* Handler for IRQs from VPU core (irqV) */ | 1049 | /* Handler for IRQs from VPU core (irqV) */ |
| 1026 | static irqreturn_t ivpu_hw_40xx_irqv_handler(struct ivpu_device *vdev, int irq) | 1050 | static bool ivpu_hw_40xx_irqv_handler(struct ivpu_device *vdev, int irq, bool *wake_thread) |
| 1027 | { | 1051 | { |
| 1028 | u32 status = REGV_RD32(VPU_40XX_HOST_SS_ICB_STATUS_0) & ICB_0_IRQ_MASK; | 1052 | u32 status = REGV_RD32(VPU_40XX_HOST_SS_ICB_STATUS_0) & ICB_0_IRQ_MASK; |
| 1029 | irqreturn_t ret = IRQ_NONE; | ||
| 1030 | 1053 | ||
| 1031 | if (!status) | 1054 | if (!status) |
| 1032 | return IRQ_NONE; | 1055 | return false; |
| 1033 | 1056 | ||
| 1034 | REGV_WR32(VPU_40XX_HOST_SS_ICB_CLEAR_0, status); | 1057 | REGV_WR32(VPU_40XX_HOST_SS_ICB_CLEAR_0, status); |
| 1035 | 1058 | ||
| @@ -1037,7 +1060,7 @@ static irqreturn_t ivpu_hw_40xx_irqv_handler(struct ivpu_device *vdev, int irq) · 1 unmodified line | |||
| 1037 | ivpu_mmu_irq_evtq_handler(vdev); | 1060 | ivpu_mmu_irq_evtq_handler(vdev); |
| 1038 | 1061 | ||
| 1039 | if (REG_TEST_FLD(VPU_40XX_HOST_SS_ICB_STATUS_0, HOST_IPC_FIFO_INT, status)) | 1062 | if (REG_TEST_FLD(VPU_40XX_HOST_SS_ICB_STATUS_0, HOST_IPC_FIFO_INT, status)) |
| 1040 | ret |= ivpu_ipc_irq_handler(vdev); | 1063 | ivpu_ipc_irq_handler(vdev, wake_thread); |
| 1041 | 1064 | ||
| 1042 | if (REG_TEST_FLD(VPU_40XX_HOST_SS_ICB_STATUS_0, MMU_IRQ_1_INT, status)) | 1065 | if (REG_TEST_FLD(VPU_40XX_HOST_SS_ICB_STATUS_0, MMU_IRQ_1_INT, status)) |
| 1043 | ivpu_dbg(vdev, IRQ, "MMU sync complete\n"); | 1066 | ivpu_dbg(vdev, IRQ, "MMU sync complete\n"); |
| @@ -1054,17 +1077,17 @@ static irqreturn_t ivpu_hw_40xx_irqv_handler(struct ivpu_device *vdev, int irq) · 10 unmodified lines | |||
| 1054 | if (REG_TEST_FLD(VPU_40XX_HOST_SS_ICB_STATUS_0, NOC_FIREWALL_INT, status)) | 1077 | if (REG_TEST_FLD(VPU_40XX_HOST_SS_ICB_STATUS_0, NOC_FIREWALL_INT, status)) |
| 1055 | ivpu_hw_40xx_irq_noc_firewall_handler(vdev); | 1078 | ivpu_hw_40xx_irq_noc_firewall_handler(vdev); |
| 1056 | 1079 | ||
| 1057 | return ret; | 1080 | return true; |
| 1058 | } | 1081 | } |
| 1059 | 1082 | ||
| 1060 | /* Handler for IRQs from Buttress core (irqB) */ | 1083 | /* Handler for IRQs from Buttress core (irqB) */ |
| 1061 | static irqreturn_t ivpu_hw_40xx_irqb_handler(struct ivpu_device *vdev, int irq) | 1084 | static bool ivpu_hw_40xx_irqb_handler(struct ivpu_device *vdev, int irq) |
| 1062 | { | 1085 | { |
| 1063 | bool schedule_recovery = false; | 1086 | bool schedule_recovery = false; |
| 1064 | u32 status = REGB_RD32(VPU_40XX_BUTTRESS_INTERRUPT_STAT) & BUTTRESS_IRQ_MASK; | 1087 | u32 status = REGB_RD32(VPU_40XX_BUTTRESS_INTERRUPT_STAT) & BUTTRESS_IRQ_MASK; |
| 1065 | 1088 | ||
| 1066 | if (status == 0) | 1089 | if (!status) |
| 1067 | return IRQ_NONE; | 1090 | return false; |
| 1068 | 1091 | ||
| 1069 | if (REG_TEST_FLD(VPU_40XX_BUTTRESS_INTERRUPT_STAT, FREQ_CHANGE, status)) | 1092 | if (REG_TEST_FLD(VPU_40XX_BUTTRESS_INTERRUPT_STAT, FREQ_CHANGE, status)) |
| 1070 | ivpu_dbg(vdev, IRQ, "FREQ_CHANGE"); | 1093 | ivpu_dbg(vdev, IRQ, "FREQ_CHANGE"); |
| @@ -1116,26 +1139,27 @@ static irqreturn_t ivpu_hw_40xx_irqb_handler(struct ivpu_device *vdev, int irq) · 45 unmodified lines | |||
| 1116 | if (schedule_recovery) | 1139 | if (schedule_recovery) |
| 1117 | ivpu_pm_schedule_recovery(vdev); | 1140 | ivpu_pm_schedule_recovery(vdev); |
| 1118 | 1141 | ||
| 1119 | return IRQ_HANDLED; | 1142 | return true; |
| 1120 | } | 1143 | } |
| 1121 | 1144 | ||
| 1122 | static irqreturn_t ivpu_hw_40xx_irq_handler(int irq, void *ptr) | 1145 | static irqreturn_t ivpu_hw_40xx_irq_handler(int irq, void *ptr) |
| 1123 | { | 1146 | { |
| 1147 | bool irqv_handled, irqb_handled, wake_thread = false; | ||
| 1124 | struct ivpu_device *vdev = ptr; | 1148 | struct ivpu_device *vdev = ptr; |
| 1125 | irqreturn_t ret = IRQ_NONE; | ||
| 1126 | 1149 | ||
| 1127 | REGB_WR32(VPU_40XX_BUTTRESS_GLOBAL_INT_MASK, 0x1); | 1150 | REGB_WR32(VPU_40XX_BUTTRESS_GLOBAL_INT_MASK, 0x1); |
| 1128 | 1151 | ||
| 1129 | ret |= ivpu_hw_40xx_irqv_handler(vdev, irq); | 1152 | irqv_handled = ivpu_hw_40xx_irqv_handler(vdev, irq, &wake_thread); |
| 1130 | ret |= ivpu_hw_40xx_irqb_handler(vdev, irq); | 1153 | irqb_handled = ivpu_hw_40xx_irqb_handler(vdev, irq); |
| 1131 | 1154 | ||
| 1132 | /* Re-enable global interrupts to re-trigger MSI for pending interrupts */ | 1155 | /* Re-enable global interrupts to re-trigger MSI for pending interrupts */ |
| 1133 | REGB_WR32(VPU_40XX_BUTTRESS_GLOBAL_INT_MASK, 0x0); | 1156 | REGB_WR32(VPU_40XX_BUTTRESS_GLOBAL_INT_MASK, 0x0); |
| 1134 | 1157 | ||
| 1135 | if (ret & IRQ_WAKE_THREAD) | 1158 | if (wake_thread) |
| 1136 | return IRQ_WAKE_THREAD; | 1159 | return IRQ_WAKE_THREAD; |
| 1137 | 1160 | if (irqv_handled || irqb_handled) | |
| 1138 | return ret; | 1161 | return IRQ_HANDLED; |
| 1162 | return IRQ_NONE; | ||
| 1139 | } | 1163 | } |
| 1140 | 1164 | ||
| 1141 | static void ivpu_hw_40xx_diagnose_failure(struct ivpu_device *vdev) | 1165 | static void ivpu_hw_40xx_diagnose_failure(struct ivpu_device *vdev) |
| @@ -1185,11 +1209,14 @@ const struct ivpu_hw_ops ivpu_hw_40xx_ops = { · 43 unmodified lines | |||
| 1185 | .info_init = ivpu_hw_40xx_info_init, | 1209 | .info_init = ivpu_hw_40xx_info_init, |
| 1186 | .power_up = ivpu_hw_40xx_power_up, | 1210 | .power_up = ivpu_hw_40xx_power_up, |
| 1187 | .is_idle = ivpu_hw_40xx_is_idle, | 1211 | .is_idle = ivpu_hw_40xx_is_idle, |
| 1212 | .wait_for_idle = ivpu_hw_40xx_wait_for_idle, | ||
| 1188 | .power_down = ivpu_hw_40xx_power_down, | 1213 | .power_down = ivpu_hw_40xx_power_down, |
| 1189 | .reset = ivpu_hw_40xx_reset, | 1214 | .reset = ivpu_hw_40xx_reset, |
| 1190 | .boot_fw = ivpu_hw_40xx_boot_fw, | 1215 | .boot_fw = ivpu_hw_40xx_boot_fw, |
| 1191 | .wdt_disable = ivpu_hw_40xx_wdt_disable, | 1216 | .wdt_disable = ivpu_hw_40xx_wdt_disable, |
| 1192 | .diagnose_failure = ivpu_hw_40xx_diagnose_failure, | 1217 | .diagnose_failure = ivpu_hw_40xx_diagnose_failure, |
| 1218 | .profiling_freq_get = ivpu_hw_40xx_profiling_freq_get, | ||
| 1219 | .profiling_freq_drive = ivpu_hw_40xx_profiling_freq_drive, | ||
| 1193 | .reg_pll_freq_get = ivpu_hw_40xx_reg_pll_freq_get, | 1220 | .reg_pll_freq_get = ivpu_hw_40xx_reg_pll_freq_get, |
| 1194 | .reg_telemetry_offset_get = ivpu_hw_40xx_reg_telemetry_offset_get, | 1221 | .reg_telemetry_offset_get = ivpu_hw_40xx_reg_telemetry_offset_get, |
| 1195 | .reg_telemetry_size_get = ivpu_hw_40xx_reg_telemetry_size_get, | 1222 | .reg_telemetry_size_get = ivpu_hw_40xx_reg_telemetry_size_get, |
| @@ -5,7 +5,7 @@ · 4 unmodified lines | |||
| 5 | 5 | ||
| 6 | #include <linux/genalloc.h> | 6 | #include <linux/genalloc.h> |
| 7 | #include <linux/highmem.h> | 7 | #include <linux/highmem.h> |
| 8 | #include <linux/kthread.h> | 8 | #include <linux/pm_runtime.h> |
| 9 | #include <linux/wait.h> | 9 | #include <linux/wait.h> |
| 10 | 10 | ||
| 11 | #include "ivpu_drv.h" | 11 | #include "ivpu_drv.h" |
| @@ -17,19 +17,12 @@ · 5 unmodified lines | |||
| 17 | #include "ivpu_pm.h" | 17 | #include "ivpu_pm.h" |
| 18 | 18 | ||
| 19 | #define IPC_MAX_RX_MSG 128 | 19 | #define IPC_MAX_RX_MSG 128 |
| 20 | #define IS_KTHREAD() (get_current()->flags & PF_KTHREAD) | ||
| 21 | 20 | ||
| 22 | struct ivpu_ipc_tx_buf { | 21 | struct ivpu_ipc_tx_buf { |
| 23 | struct ivpu_ipc_hdr ipc; | 22 | struct ivpu_ipc_hdr ipc; |
| 24 | struct vpu_jsm_msg jsm; | 23 | struct vpu_jsm_msg jsm; |
| 25 | }; | 24 | }; |
| 26 | 25 | ||
| 27 | struct ivpu_ipc_rx_msg { | ||
| 28 | struct list_head link; | ||
| 29 | struct ivpu_ipc_hdr *ipc_hdr; | ||
| 30 | struct vpu_jsm_msg *jsm_msg; | ||
| 31 | }; | ||
| 32 | |||
| 33 | static void ivpu_ipc_msg_dump(struct ivpu_device *vdev, char *c, | 26 | static void ivpu_ipc_msg_dump(struct ivpu_device *vdev, char *c, |
| 34 | struct ivpu_ipc_hdr *ipc_hdr, u32 vpu_addr) | 27 | struct ivpu_ipc_hdr *ipc_hdr, u32 vpu_addr) |
| 35 | { | 28 | { |
| @@ -139,8 +132,49 @@ static void ivpu_ipc_tx(struct ivpu_device *vdev, u32 vpu_addr) · 103 unmodified lines | |||
| 139 | ivpu_hw_reg_ipc_tx_set(vdev, vpu_addr); | 132 | ivpu_hw_reg_ipc_tx_set(vdev, vpu_addr); |
| 140 | } | 133 | } |
| 141 | 134 | ||
| 142 | void | 135 | static void |
| 143 | ivpu_ipc_consumer_add(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, u32 channel) | 136 | ivpu_ipc_rx_msg_add(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, |
| 137 | struct ivpu_ipc_hdr *ipc_hdr, struct vpu_jsm_msg *jsm_msg) | ||
| 138 | { | ||
| 139 | struct ivpu_ipc_info *ipc = vdev->ipc; | ||
| 140 | struct ivpu_ipc_rx_msg *rx_msg; | ||
| 141 | |||
| 142 | lockdep_assert_held(&ipc->cons_lock); | ||
| 143 | lockdep_assert_irqs_disabled(); | ||
| 144 | |||
| 145 | rx_msg = kzalloc(sizeof(*rx_msg), GFP_ATOMIC); | ||
| 146 | if (!rx_msg) { | ||
| 147 | ivpu_ipc_rx_mark_free(vdev, ipc_hdr, jsm_msg); | ||
| 148 | return; | ||
| 149 | } | ||
| 150 | |||
| 151 | atomic_inc(&ipc->rx_msg_count); | ||
| 152 | |||
| 153 | rx_msg->ipc_hdr = ipc_hdr; | ||
| 154 | rx_msg->jsm_msg = jsm_msg; | ||
| 155 | rx_msg->callback = cons->rx_callback; | ||
| 156 | |||
| 157 | if (rx_msg->callback) { | ||
| 158 | list_add_tail(&rx_msg->link, &ipc->cb_msg_list); | ||
| 159 | } else { | ||
| 160 | spin_lock(&cons->rx_lock); | ||
| 161 | list_add_tail(&rx_msg->link, &cons->rx_msg_list); | ||
| 162 | spin_unlock(&cons->rx_lock); | ||
| 163 | wake_up(&cons->rx_msg_wq); | ||
| 164 | } | ||
| 165 | } | ||
| 166 | |||
| 167 | static void | ||
| 168 | ivpu_ipc_rx_msg_del(struct ivpu_device *vdev, struct ivpu_ipc_rx_msg *rx_msg) | ||
| 169 | { | ||
| 170 | list_del(&rx_msg->link); | ||
| 171 | ivpu_ipc_rx_mark_free(vdev, rx_msg->ipc_hdr, rx_msg->jsm_msg); | ||
| 172 | atomic_dec(&vdev->ipc->rx_msg_count); | ||
| 173 | kfree(rx_msg); | ||
| 174 | } | ||
| 175 | |||
| 176 | void ivpu_ipc_consumer_add(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, | ||
| 177 | u32 channel, ivpu_ipc_rx_callback_t rx_callback) | ||
| 144 | { | 178 | { |
| 145 | struct ivpu_ipc_info *ipc = vdev->ipc; | 179 | struct ivpu_ipc_info *ipc = vdev->ipc; |
| 146 | 180 | ||
| @@ -148,13 +182,15 @@ ivpu_ipc_consumer_add(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, · 1 unmodified line | |||
| 148 | cons->channel = channel; | 182 | cons->channel = channel; |
| 149 | cons->tx_vpu_addr = 0; | 183 | cons->tx_vpu_addr = 0; |
| 150 | cons->request_id = 0; | 184 | cons->request_id = 0; |
| 151 | spin_lock_init(&cons->rx_msg_lock); | 185 | cons->aborted = false; |
| 186 | cons->rx_callback = rx_callback; | ||
| 187 | spin_lock_init(&cons->rx_lock); | ||
| 152 | INIT_LIST_HEAD(&cons->rx_msg_list); | 188 | INIT_LIST_HEAD(&cons->rx_msg_list); |
| 153 | init_waitqueue_head(&cons->rx_msg_wq); | 189 | init_waitqueue_head(&cons->rx_msg_wq); |
| 154 | 190 | ||
| 155 | spin_lock_irq(&ipc->cons_list_lock); | 191 | spin_lock_irq(&ipc->cons_lock); |
| 156 | list_add_tail(&cons->link, &ipc->cons_list); | 192 | list_add_tail(&cons->link, &ipc->cons_list); |
| 157 | spin_unlock_irq(&ipc->cons_list_lock); | 193 | spin_unlock_irq(&ipc->cons_lock); |
| 158 | } | 194 | } |
| 159 | 195 | ||
| 160 | void ivpu_ipc_consumer_del(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons) | 196 | void ivpu_ipc_consumer_del(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons) |
| @@ -162,18 +198,14 @@ void ivpu_ipc_consumer_del(struct ivpu_device *vdev, struct ivpu_ipc_consumer *c · 1 unmodified line | |||
| 162 | struct ivpu_ipc_info *ipc = vdev->ipc; | 198 | struct ivpu_ipc_info *ipc = vdev->ipc; |
| 163 | struct ivpu_ipc_rx_msg *rx_msg, *r; | 199 | struct ivpu_ipc_rx_msg *rx_msg, *r; |
| 164 | 200 | ||
| 165 | spin_lock_irq(&ipc->cons_list_lock); | 201 | spin_lock_irq(&ipc->cons_lock); |
| 166 | list_del(&cons->link); | 202 | list_del(&cons->link); |
| 167 | spin_unlock_irq(&ipc->cons_list_lock); | 203 | spin_unlock_irq(&ipc->cons_lock); |
| 168 | 204 | ||
| 169 | spin_lock_irq(&cons->rx_msg_lock); | 205 | spin_lock_irq(&cons->rx_lock); |
| 170 | list_for_each_entry_safe(rx_msg, r, &cons->rx_msg_list, link) { | 206 | list_for_each_entry_safe(rx_msg, r, &cons->rx_msg_list, link) |
| 171 | list_del(&rx_msg->link); | 207 | ivpu_ipc_rx_msg_del(vdev, rx_msg); |
| 172 | ivpu_ipc_rx_mark_free(vdev, rx_msg->ipc_hdr, rx_msg->jsm_msg); | 208 | spin_unlock_irq(&cons->rx_lock); |
| 173 | atomic_dec(&ipc->rx_msg_count); | ||
| 174 | kfree(rx_msg); | ||
| 175 | } | ||
| 176 | spin_unlock_irq(&cons->rx_msg_lock); | ||
| 177 | 209 | ||
| 178 | ivpu_ipc_tx_release(vdev, cons->tx_vpu_addr); | 210 | ivpu_ipc_tx_release(vdev, cons->tx_vpu_addr); |
| 179 | } | 211 | } |
| @@ -202,52 +234,61 @@ unlock: · 22 unmodified lines | |||
| 202 | return ret; | 234 | return ret; |
| 203 | } | 235 | } |
| 204 | 236 | ||
| 237 | static bool ivpu_ipc_rx_need_wakeup(struct ivpu_ipc_consumer *cons) | ||
| 238 | { | ||
| 239 | bool ret; | ||
| 240 | |||
| 241 | spin_lock_irq(&cons->rx_lock); | ||
| 242 | ret = !list_empty(&cons->rx_msg_list) || cons->aborted; | ||
| 243 | spin_unlock_irq(&cons->rx_lock); | ||
| 244 | |||
| 245 | return ret; | ||
| 246 | } | ||
| 247 | |||
| 205 | int ivpu_ipc_receive(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, | 248 | int ivpu_ipc_receive(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, |
| 206 | struct ivpu_ipc_hdr *ipc_buf, | 249 | struct ivpu_ipc_hdr *ipc_buf, |
| 207 | struct vpu_jsm_msg *ipc_payload, unsigned long timeout_ms) | 250 | struct vpu_jsm_msg *jsm_msg, unsigned long timeout_ms) |
| 208 | { | 251 | { |
| 209 | struct ivpu_ipc_info *ipc = vdev->ipc; | ||
| 210 | struct ivpu_ipc_rx_msg *rx_msg; | 252 | struct ivpu_ipc_rx_msg *rx_msg; |
| 211 | int wait_ret, ret = 0; | 253 | int wait_ret, ret = 0; |
| 212 | 254 | ||
| 255 | if (drm_WARN_ONCE(&vdev->drm, cons->rx_callback, "Consumer works only in async mode\n")) | ||
| 256 | return -EINVAL; | ||
| 257 | |||
| 213 | wait_ret = wait_event_timeout(cons->rx_msg_wq, | 258 | wait_ret = wait_event_timeout(cons->rx_msg_wq, |
| 214 | (IS_KTHREAD() && kthread_should_stop()) || | 259 | ivpu_ipc_rx_need_wakeup(cons), |
| 215 | !list_empty(&cons->rx_msg_list), | ||
| 216 | msecs_to_jiffies(timeout_ms)); | 260 | msecs_to_jiffies(timeout_ms)); |
| 217 | 261 | ||
| 218 | if (IS_KTHREAD() && kthread_should_stop()) | ||
| 219 | return -EINTR; | ||
| 220 | |||
| 221 | if (wait_ret == 0) | 262 | if (wait_ret == 0) |
| 222 | return -ETIMEDOUT; | 263 | return -ETIMEDOUT; |
| 223 | 264 | ||
| 224 | spin_lock_irq(&cons->rx_msg_lock); | 265 | spin_lock_irq(&cons->rx_lock); |
| 266 | if (cons->aborted) { | ||
| 267 | spin_unlock_irq(&cons->rx_lock); | ||
| 268 | return -ECANCELED; | ||
| 269 | } | ||
| 225 | rx_msg = list_first_entry_or_null(&cons->rx_msg_list, struct ivpu_ipc_rx_msg, link); | 270 | rx_msg = list_first_entry_or_null(&cons->rx_msg_list, struct ivpu_ipc_rx_msg, link); |
| 226 | if (!rx_msg) { | 271 | if (!rx_msg) { |
| 227 | spin_unlock_irq(&cons->rx_msg_lock); | 272 | spin_unlock_irq(&cons->rx_lock); |
| 228 | return -EAGAIN; | 273 | return -EAGAIN; |
| 229 | } | 274 | } |
| 230 | list_del(&rx_msg->link); | ||
| 231 | spin_unlock_irq(&cons->rx_msg_lock); | ||
| 232 | 275 | ||
| 233 | if (ipc_buf) | 276 | if (ipc_buf) |
| 234 | memcpy(ipc_buf, rx_msg->ipc_hdr, sizeof(*ipc_buf)); | 277 | memcpy(ipc_buf, rx_msg->ipc_hdr, sizeof(*ipc_buf)); |
| 235 | if (rx_msg->jsm_msg) { | 278 | if (rx_msg->jsm_msg) { |
| 236 | u32 size = min_t(int, rx_msg->ipc_hdr->data_size, sizeof(*ipc_payload)); | 279 | u32 size = min_t(int, rx_msg->ipc_hdr->data_size, sizeof(*jsm_msg)); |
| 237 | 280 | ||
| 238 | if (rx_msg->jsm_msg->result != VPU_JSM_STATUS_SUCCESS) { | 281 | if (rx_msg->jsm_msg->result != VPU_JSM_STATUS_SUCCESS) { |
| 239 | ivpu_dbg(vdev, IPC, "IPC resp result error: %d\n", rx_msg->jsm_msg->result); | 282 | ivpu_dbg(vdev, IPC, "IPC resp result error: %d\n", rx_msg->jsm_msg->result); |
| 240 | ret = -EBADMSG; | 283 | ret = -EBADMSG; |
| 241 | } | 284 | } |
| 242 | 285 | ||
| 243 | if (ipc_payload) | 286 | if (jsm_msg) |
| 244 | memcpy(ipc_payload, rx_msg->jsm_msg, size); | 287 | memcpy(jsm_msg, rx_msg->jsm_msg, size); |
| 245 | } | 288 | } |
| 246 | 289 | ||
| 247 | ivpu_ipc_rx_mark_free(vdev, rx_msg->ipc_hdr, rx_msg->jsm_msg); | 290 | ivpu_ipc_rx_msg_del(vdev, rx_msg); |
| 248 | atomic_dec(&ipc->rx_msg_count); | 291 | spin_unlock_irq(&cons->rx_lock); |
| 249 | kfree(rx_msg); | ||
| 250 | |||
| 251 | return ret; | 292 | return ret; |
| 252 | } | 293 | } |
| 253 | 294 | ||
| @@ -260,7 +301,7 @@ ivpu_ipc_send_receive_internal(struct ivpu_device *vdev, struct vpu_jsm_msg *req · 6 unmodified lines | |||
| 260 | struct ivpu_ipc_consumer cons; | 301 | struct ivpu_ipc_consumer cons; |
| 261 | int ret; | 302 | int ret; |
| 262 | 303 | ||
| 263 | ivpu_ipc_consumer_add(vdev, &cons, channel); | 304 | ivpu_ipc_consumer_add(vdev, &cons, channel, NULL); |
| 264 | 305 | ||
| 265 | ret = ivpu_ipc_send(vdev, &cons, req); | 306 | ret = ivpu_ipc_send(vdev, &cons, req); |
| 266 | if (ret) { | 307 | if (ret) { |
| @@ -285,23 +326,19 @@ consumer_del: · 18 unmodified lines | |||
| 285 | return ret; | 326 | return ret; |
| 286 | } | 327 | } |
| 287 | 328 | ||
| 288 | int ivpu_ipc_send_receive(struct ivpu_device *vdev, struct vpu_jsm_msg *req, | 329 | int ivpu_ipc_send_receive_active(struct ivpu_device *vdev, struct vpu_jsm_msg *req, |
| 289 | enum vpu_ipc_msg_type expected_resp_type, | 330 | enum vpu_ipc_msg_type expected_resp, struct vpu_jsm_msg *resp, |
| 290 | struct vpu_jsm_msg *resp, u32 channel, | 331 | u32 channel, unsigned long timeout_ms) |
| 291 | unsigned long timeout_ms) | ||
| 292 | { | 332 | { |
| 293 | struct vpu_jsm_msg hb_req = { .type = VPU_JSM_MSG_QUERY_ENGINE_HB }; | 333 | struct vpu_jsm_msg hb_req = { .type = VPU_JSM_MSG_QUERY_ENGINE_HB }; |
| 294 | struct vpu_jsm_msg hb_resp; | 334 | struct vpu_jsm_msg hb_resp; |
| 295 | int ret, hb_ret; | 335 | int ret, hb_ret; |
| 296 | 336 | ||
| 297 | ret = ivpu_rpm_get(vdev); | 337 | drm_WARN_ON(&vdev->drm, pm_runtime_status_suspended(vdev->drm.dev)); |
| 298 | if (ret < 0) | ||
| 299 | return ret; | ||
| 300 | 338 | ||
| 301 | ret = ivpu_ipc_send_receive_internal(vdev, req, expected_resp_type, resp, | 339 | ret = ivpu_ipc_send_receive_internal(vdev, req, expected_resp, resp, channel, timeout_ms); |
| 302 | channel, timeout_ms); | ||
| 303 | if (ret != -ETIMEDOUT) | 340 | if (ret != -ETIMEDOUT) |
| 304 | goto rpm_put; | 341 | return ret; |
| 305 | 342 | ||
| 306 | hb_ret = ivpu_ipc_send_receive_internal(vdev, &hb_req, VPU_JSM_MSG_QUERY_ENGINE_HB_DONE, | 343 | hb_ret = ivpu_ipc_send_receive_internal(vdev, &hb_req, VPU_JSM_MSG_QUERY_ENGINE_HB_DONE, |
| 307 | &hb_resp, VPU_IPC_CHAN_ASYNC_CMD, | 344 | &hb_resp, VPU_IPC_CHAN_ASYNC_CMD, |
| @@ -311,7 +348,21 @@ int ivpu_ipc_send_receive(struct ivpu_device *vdev, struct vpu_jsm_msg *req, · 3 unmodified lines | |||
| 311 | ivpu_pm_schedule_recovery(vdev); | 348 | ivpu_pm_schedule_recovery(vdev); |
| 312 | } | 349 | } |
| 313 | 350 | ||
| 314 | rpm_put: | 351 | return ret; |
| 352 | } | ||
| 353 | |||
| 354 | int ivpu_ipc_send_receive(struct ivpu_device *vdev, struct vpu_jsm_msg *req, | ||
| 355 | enum vpu_ipc_msg_type expected_resp, struct vpu_jsm_msg *resp, | ||
| 356 | u32 channel, unsigned long timeout_ms) | ||
| 357 | { | ||
| 358 | int ret; | ||
| 359 | |||
| 360 | ret = ivpu_rpm_get(vdev); | ||
| 361 | if (ret < 0) | ||
| 362 | return ret; | ||
| 363 | |||
| 364 | ret = ivpu_ipc_send_receive_active(vdev, req, expected_resp, resp, channel, timeout_ms); | ||
| 365 | |||
| 315 | ivpu_rpm_put(vdev); | 366 | ivpu_rpm_put(vdev); |
| 316 | return ret; | 367 | return ret; |
| 317 | } | 368 | } |
| @@ -329,35 +380,7 @@ ivpu_ipc_match_consumer(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons · 11 unmodified lines | |||
| 329 | return false; | 380 | return false; |
| 330 | } | 381 | } |
| 331 | 382 | ||
| 332 | static void | 383 | void ivpu_ipc_irq_handler(struct ivpu_device *vdev, bool *wake_thread) |
| 333 | ivpu_ipc_dispatch(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, | ||
| 334 | struct ivpu_ipc_hdr *ipc_hdr, struct vpu_jsm_msg *jsm_msg) | ||
| 335 | { | ||
| 336 | struct ivpu_ipc_info *ipc = vdev->ipc; | ||
| 337 | struct ivpu_ipc_rx_msg *rx_msg; | ||
| 338 | unsigned long flags; | ||
| 339 | |||
| 340 | lockdep_assert_held(&ipc->cons_list_lock); | ||
| 341 | |||
| 342 | rx_msg = kzalloc(sizeof(*rx_msg), GFP_ATOMIC); | ||
| 343 | if (!rx_msg) { | ||
| 344 | ivpu_ipc_rx_mark_free(vdev, ipc_hdr, jsm_msg); | ||
| 345 | return; | ||
| 346 | } | ||
| 347 | |||
| 348 | atomic_inc(&ipc->rx_msg_count); | ||
| 349 | |||
| 350 | rx_msg->ipc_hdr = ipc_hdr; | ||
| 351 | rx_msg->jsm_msg = jsm_msg; | ||
| 352 | |||
| 353 | spin_lock_irqsave(&cons->rx_msg_lock, flags); | ||
| 354 | list_add_tail(&rx_msg->link, &cons->rx_msg_list); | ||
| 355 | spin_unlock_irqrestore(&cons->rx_msg_lock, flags); | ||
| 356 | |||
| 357 | wake_up(&cons->rx_msg_wq); | ||
| 358 | } | ||
| 359 | |||
| 360 | int ivpu_ipc_irq_handler(struct ivpu_device *vdev) | ||
| 361 | { | 384 | { |
| 362 | struct ivpu_ipc_info *ipc = vdev->ipc; | 385 | struct ivpu_ipc_info *ipc = vdev->ipc; |
| 363 | struct ivpu_ipc_consumer *cons; | 386 | struct ivpu_ipc_consumer *cons; |
| @@ -375,7 +398,7 @@ int ivpu_ipc_irq_handler(struct ivpu_device *vdev) · 11 unmodified lines | |||
| 375 | vpu_addr = ivpu_hw_reg_ipc_rx_addr_get(vdev); | 398 | vpu_addr = ivpu_hw_reg_ipc_rx_addr_get(vdev); |
| 376 | if (vpu_addr == REG_IO_ERROR) { | 399 | if (vpu_addr == REG_IO_ERROR) { |
| 377 | ivpu_err_ratelimited(vdev, "Failed to read IPC rx addr register\n"); | 400 | ivpu_err_ratelimited(vdev, "Failed to read IPC rx addr register\n"); |
| 378 | return -EIO; | 401 | return; |
| 379 | } | 402 | } |
| 380 | 403 | ||
| 381 | ipc_hdr = ivpu_to_cpu_addr(ipc->mem_rx, vpu_addr); | 404 | ipc_hdr = ivpu_to_cpu_addr(ipc->mem_rx, vpu_addr); |
| @@ -405,15 +428,15 @@ int ivpu_ipc_irq_handler(struct ivpu_device *vdev) · 23 unmodified lines | |||
| 405 | } | 428 | } |
| 406 | 429 | ||
| 407 | dispatched = false; | 430 | dispatched = false; |
| 408 | spin_lock_irqsave(&ipc->cons_list_lock, flags); | 431 | spin_lock_irqsave(&ipc->cons_lock, flags); |
| 409 | list_for_each_entry(cons, &ipc->cons_list, link) { | 432 | list_for_each_entry(cons, &ipc->cons_list, link) { |
| 410 | if (ivpu_ipc_match_consumer(vdev, cons, ipc_hdr, jsm_msg)) { | 433 | if (ivpu_ipc_match_consumer(vdev, cons, ipc_hdr, jsm_msg)) { |
| 411 | ivpu_ipc_dispatch(vdev, cons, ipc_hdr, jsm_msg); | 434 | ivpu_ipc_rx_msg_add(vdev, cons, ipc_hdr, jsm_msg); |
| 412 | dispatched = true; | 435 | dispatched = true; |
| 413 | break; | 436 | break; |
| 414 | } | 437 | } |
| 415 | } | 438 | } |
| 416 | spin_unlock_irqrestore(&ipc->cons_list_lock, flags); | 439 | spin_unlock_irqrestore(&ipc->cons_lock, flags); |
| 417 | 440 | ||
| 418 | if (!dispatched) { | 441 | if (!dispatched) { |
| 419 | ivpu_dbg(vdev, IPC, "IPC RX msg 0x%x dropped (no consumer)\n", vpu_addr); | 442 | ivpu_dbg(vdev, IPC, "IPC RX msg 0x%x dropped (no consumer)\n", vpu_addr); |
| @@ -421,7 +444,28 @@ int ivpu_ipc_irq_handler(struct ivpu_device *vdev) · 1 unmodified line | |||
| 421 | } | 444 | } |
| 422 | } | 445 | } |
| 423 | 446 | ||
| 424 | return 0; | 447 | if (wake_thread) |
| 448 | *wake_thread = !list_empty(&ipc->cb_msg_list); | ||
| 449 | } | ||
| 450 | |||
| 451 | irqreturn_t ivpu_ipc_irq_thread_handler(struct ivpu_device *vdev) | ||
| 452 | { | ||
| 453 | struct ivpu_ipc_info *ipc = vdev->ipc; | ||
| 454 | struct ivpu_ipc_rx_msg *rx_msg, *r; | ||
| 455 | struct list_head cb_msg_list; | ||
| 456 | |||
| 457 | INIT_LIST_HEAD(&cb_msg_list); | ||
| 458 | |||
| 459 | spin_lock_irq(&ipc->cons_lock); | ||
| 460 | list_splice_tail_init(&ipc->cb_msg_list, &cb_msg_list); | ||
| 461 | spin_unlock_irq(&ipc->cons_lock); | ||
| 462 | |||
| 463 | list_for_each_entry_safe(rx_msg, r, &cb_msg_list, link) { | ||
| 464 | rx_msg->callback(vdev, rx_msg->ipc_hdr, rx_msg->jsm_msg); | ||
| 465 | ivpu_ipc_rx_msg_del(vdev, rx_msg); | ||
| 466 | } | ||
| 467 | |||
| 468 | return IRQ_HANDLED; | ||
| 425 | } | 469 | } |
| 426 | 470 | ||
| 427 | int ivpu_ipc_init(struct ivpu_device *vdev) | 471 | int ivpu_ipc_init(struct ivpu_device *vdev) |
| @@ -456,10 +500,10 @@ int ivpu_ipc_init(struct ivpu_device *vdev) · 28 unmodified lines | |||
| 456 | goto err_free_rx; | 500 | goto err_free_rx; |
| 457 | } | 501 | } |
| 458 | 502 | ||
| 503 | spin_lock_init(&ipc->cons_lock); | ||
| 459 | INIT_LIST_HEAD(&ipc->cons_list); | 504 | INIT_LIST_HEAD(&ipc->cons_list); |
| 460 | spin_lock_init(&ipc->cons_list_lock); | 505 | INIT_LIST_HEAD(&ipc->cb_msg_list); |
| 461 | drmm_mutex_init(&vdev->drm, &ipc->lock); | 506 | drmm_mutex_init(&vdev->drm, &ipc->lock); |
| 462 | |||
| 463 | ivpu_ipc_reset(vdev); | 507 | ivpu_ipc_reset(vdev); |
| 464 | return 0; | 508 | return 0; |
| 465 | 509 | ||
| @@ -472,6 +516,13 @@ err_free_tx: · 6 unmodified lines | |||
| 472 | 516 | ||
| 473 | void ivpu_ipc_fini(struct ivpu_device *vdev) | 517 | void ivpu_ipc_fini(struct ivpu_device *vdev) |
| 474 | { | 518 | { |
| 519 | struct ivpu_ipc_info *ipc = vdev->ipc; | ||
| 520 | |||
| 521 | drm_WARN_ON(&vdev->drm, ipc->on); | ||
| 522 | drm_WARN_ON(&vdev->drm, !list_empty(&ipc->cons_list)); | ||
| 523 | drm_WARN_ON(&vdev->drm, !list_empty(&ipc->cb_msg_list)); | ||
| 524 | drm_WARN_ON(&vdev->drm, atomic_read(&ipc->rx_msg_count) > 0); | ||
| 525 | |||
| 475 | ivpu_ipc_mem_fini(vdev); | 526 | ivpu_ipc_mem_fini(vdev); |
| 476 | } | 527 | } |
| 477 | 528 | ||
| @@ -488,16 +539,27 @@ void ivpu_ipc_disable(struct ivpu_device *vdev) · 10 unmodified lines | |||
| 488 | { | 539 | { |
| 489 | struct ivpu_ipc_info *ipc = vdev->ipc; | 540 | struct ivpu_ipc_info *ipc = vdev->ipc; |
| 490 | struct ivpu_ipc_consumer *cons, *c; | 541 | struct ivpu_ipc_consumer *cons, *c; |
| 491 | unsigned long flags; | 542 | struct ivpu_ipc_rx_msg *rx_msg, *r; |
| 543 | |||
| 544 | drm_WARN_ON(&vdev->drm, !list_empty(&ipc->cb_msg_list)); | ||
| 492 | 545 | ||
| 493 | mutex_lock(&ipc->lock); | 546 | mutex_lock(&ipc->lock); |
| 494 | ipc->on = false; | 547 | ipc->on = false; |
| 495 | mutex_unlock(&ipc->lock); | 548 | mutex_unlock(&ipc->lock); |
| 496 | 549 | ||
| 497 | spin_lock_irqsave(&ipc->cons_list_lock, flags); | 550 | spin_lock_irq(&ipc->cons_lock); |
| 498 | list_for_each_entry_safe(cons, c, &ipc->cons_list, link) | 551 | list_for_each_entry_safe(cons, c, &ipc->cons_list, link) { |
| 552 | spin_lock(&cons->rx_lock); | ||
| 553 | if (!cons->rx_callback) | ||
| 554 | cons->aborted = true; | ||
| 555 | list_for_each_entry_safe(rx_msg, r, &cons->rx_msg_list, link) | ||
| 556 | ivpu_ipc_rx_msg_del(vdev, rx_msg); | ||
| 557 | spin_unlock(&cons->rx_lock); | ||
| 499 | wake_up(&cons->rx_msg_wq); | 558 | wake_up(&cons->rx_msg_wq); |
| 500 | spin_unlock_irqrestore(&ipc->cons_list_lock, flags); | 559 | } |
| 560 | spin_unlock_irq(&ipc->cons_lock); | ||
| 561 | |||
| 562 | drm_WARN_ON(&vdev->drm, atomic_read(&ipc->rx_msg_count) > 0); | ||
| 501 | } | 563 | } |
| 502 | 564 | ||
| 503 | void ivpu_ipc_reset(struct ivpu_device *vdev) | 565 | void ivpu_ipc_reset(struct ivpu_device *vdev) |
| @@ -505,6 +567,7 @@ void ivpu_ipc_reset(struct ivpu_device *vdev) · 1 unmodified line | |||
| 505 | struct ivpu_ipc_info *ipc = vdev->ipc; | 567 | struct ivpu_ipc_info *ipc = vdev->ipc; |
| 506 | 568 | ||
| 507 | mutex_lock(&ipc->lock); | 569 | mutex_lock(&ipc->lock); |
| 570 | drm_WARN_ON(&vdev->drm, ipc->on); | ||
| 508 | 571 | ||
| 509 | memset(ivpu_bo_vaddr(ipc->mem_tx), 0, ivpu_bo_size(ipc->mem_tx)); | 572 | memset(ivpu_bo_vaddr(ipc->mem_tx), 0, ivpu_bo_size(ipc->mem_tx)); |
| 510 | memset(ivpu_bo_vaddr(ipc->mem_rx), 0, ivpu_bo_size(ipc->mem_rx)); | 573 | memset(ivpu_bo_vaddr(ipc->mem_rx), 0, ivpu_bo_size(ipc->mem_rx)); |
| @@ -42,13 +42,26 @@ struct ivpu_ipc_hdr { · 41 unmodified lines | |||
| 42 | u8 status; | 42 | u8 status; |
| 43 | } __packed __aligned(IVPU_IPC_ALIGNMENT); | 43 | } __packed __aligned(IVPU_IPC_ALIGNMENT); |
| 44 | 44 | ||
| 45 | typedef void (*ivpu_ipc_rx_callback_t)(struct ivpu_device *vdev, | ||
| 46 | struct ivpu_ipc_hdr *ipc_hdr, | ||
| 47 | struct vpu_jsm_msg *jsm_msg); | ||
| 48 | |||
| 49 | struct ivpu_ipc_rx_msg { | ||
| 50 | struct list_head link; | ||
| 51 | struct ivpu_ipc_hdr *ipc_hdr; | ||
| 52 | struct vpu_jsm_msg *jsm_msg; | ||
| 53 | ivpu_ipc_rx_callback_t callback; | ||
| 54 | }; | ||
| 55 | |||
| 45 | struct ivpu_ipc_consumer { | 56 | struct ivpu_ipc_consumer { |
| 46 | struct list_head link; | 57 | struct list_head link; |
| 47 | u32 channel; | 58 | u32 channel; |
| 48 | u32 tx_vpu_addr; | 59 | u32 tx_vpu_addr; |
| 49 | u32 request_id; | 60 | u32 request_id; |
| 61 | bool aborted; | ||
| 62 | ivpu_ipc_rx_callback_t rx_callback; | ||
| 50 | 63 | ||
| 51 | spinlock_t rx_msg_lock; /* Protects rx_msg_list */ | 64 | spinlock_t rx_lock; /* Protects rx_msg_list and aborted */ |
| 52 | struct list_head rx_msg_list; | 65 | struct list_head rx_msg_list; |
| 53 | wait_queue_head_t rx_msg_wq; | 66 | wait_queue_head_t rx_msg_wq; |
| 54 | }; | 67 | }; |
| @@ -60,8 +73,9 @@ struct ivpu_ipc_info { · 5 unmodified lines | |||
| 60 | 73 | ||
| 61 | atomic_t rx_msg_count; | 74 | atomic_t rx_msg_count; |
| 62 | 75 | ||
| 63 | spinlock_t cons_list_lock; /* Protects cons_list */ | 76 | spinlock_t cons_lock; /* Protects cons_list and cb_msg_list */ |
| 64 | struct list_head cons_list; | 77 | struct list_head cons_list; |
| 78 | struct list_head cb_msg_list; | ||
| 65 | 79 | ||
| 66 | atomic_t request_id; | 80 | atomic_t request_id; |
| 67 | struct mutex lock; /* Lock on status */ | 81 | struct mutex lock; /* Lock on status */ |
| @@ -75,19 +89,22 @@ void ivpu_ipc_enable(struct ivpu_device *vdev); · 7 unmodified lines | |||
| 75 | void ivpu_ipc_disable(struct ivpu_device *vdev); | 89 | void ivpu_ipc_disable(struct ivpu_device *vdev); |
| 76 | void ivpu_ipc_reset(struct ivpu_device *vdev); | 90 | void ivpu_ipc_reset(struct ivpu_device *vdev); |
| 77 | 91 | ||
| 78 | int ivpu_ipc_irq_handler(struct ivpu_device *vdev); | 92 | void ivpu_ipc_irq_handler(struct ivpu_device *vdev, bool *wake_thread); |
| 93 | irqreturn_t ivpu_ipc_irq_thread_handler(struct ivpu_device *vdev); | ||
| 79 | 94 | ||
| 80 | void ivpu_ipc_consumer_add(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, | 95 | void ivpu_ipc_consumer_add(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, |
| 81 | u32 channel); | 96 | u32 channel, ivpu_ipc_rx_callback_t callback); |
| 82 | void ivpu_ipc_consumer_del(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons); | 97 | void ivpu_ipc_consumer_del(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons); |
| 83 | 98 | ||
| 84 | int ivpu_ipc_receive(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, | 99 | int ivpu_ipc_receive(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, |
| 85 | struct ivpu_ipc_hdr *ipc_buf, struct vpu_jsm_msg *ipc_payload, | 100 | struct ivpu_ipc_hdr *ipc_buf, struct vpu_jsm_msg *jsm_msg, |
| 86 | unsigned long timeout_ms); | 101 | unsigned long timeout_ms); |
| 87 | 102 | ||
| 103 | int ivpu_ipc_send_receive_active(struct ivpu_device *vdev, struct vpu_jsm_msg *req, | ||
| 104 | enum vpu_ipc_msg_type expected_resp, struct vpu_jsm_msg *resp, | ||
| 105 | u32 channel, unsigned long timeout_ms); | ||
| 88 | int ivpu_ipc_send_receive(struct ivpu_device *vdev, struct vpu_jsm_msg *req, | 106 | int ivpu_ipc_send_receive(struct ivpu_device *vdev, struct vpu_jsm_msg *req, |
| 89 | enum vpu_ipc_msg_type expected_resp_type, | 107 | enum vpu_ipc_msg_type expected_resp, struct vpu_jsm_msg *resp, |
| 90 | struct vpu_jsm_msg *resp, u32 channel, | 108 | u32 channel, unsigned long timeout_ms); |
| 91 | unsigned long timeout_ms); | ||
| 92 | 109 | ||
| 93 | #endif /* __IVPU_IPC_H__ */ | 110 | #endif /* __IVPU_IPC_H__ */ |
| @@ -7,7 +7,6 @@ · 6 unmodified lines | |||
| 7 | 7 | ||
| 8 | #include <linux/bitfield.h> | 8 | #include <linux/bitfield.h> |
| 9 | #include <linux/highmem.h> | 9 | #include <linux/highmem.h> |
| 10 | #include <linux/kthread.h> | ||
| 11 | #include <linux/pci.h> | 10 | #include <linux/pci.h> |
| 12 | #include <linux/module.h> | 11 | #include <linux/module.h> |
| 13 | #include <uapi/drm/ivpu_accel.h> | 12 | #include <uapi/drm/ivpu_accel.h> |
| @@ -24,10 +23,6 @@ · 10 unmodified lines | |||
| 24 | #define JOB_ID_CONTEXT_MASK GENMASK(31, 8) | 23 | #define JOB_ID_CONTEXT_MASK GENMASK(31, 8) |
| 25 | #define JOB_MAX_BUFFER_COUNT 65535 | 24 | #define JOB_MAX_BUFFER_COUNT 65535 |
| 26 | 25 | ||
| 27 | static unsigned int ivpu_tdr_timeout_ms; | ||
| 28 | module_param_named(tdr_timeout_ms, ivpu_tdr_timeout_ms, uint, 0644); | ||
| 29 | MODULE_PARM_DESC(tdr_timeout_ms, "Timeout for device hang detection, in milliseconds, 0 - default"); | ||
| 30 | |||
| 31 | static void ivpu_cmdq_ring_db(struct ivpu_device *vdev, struct ivpu_cmdq *cmdq) | 26 | static void ivpu_cmdq_ring_db(struct ivpu_device *vdev, struct ivpu_cmdq *cmdq) |
| 32 | { | 27 | { |
| 33 | ivpu_hw_reg_db_set(vdev, cmdq->db_id); | 28 | ivpu_hw_reg_db_set(vdev, cmdq->db_id); |
| @@ -196,6 +191,8 @@ static int ivpu_cmdq_push_job(struct ivpu_cmdq *cmdq, struct ivpu_job *job) · 162 unmodified lines | |||
| 196 | entry->batch_buf_addr = job->cmd_buf_vpu_addr; | 191 | entry->batch_buf_addr = job->cmd_buf_vpu_addr; |
| 197 | entry->job_id = job->job_id; | 192 | entry->job_id = job->job_id; |
| 198 | entry->flags = 0; | 193 | entry->flags = 0; |
| 194 | if (unlikely(ivpu_test_mode & IVPU_TEST_MODE_NULL_SUBMISSION)) | ||
| 195 | entry->flags = VPU_JOB_FLAGS_NULL_SUBMISSION_MASK; | ||
| 199 | wmb(); /* Ensure that tail is updated after filling entry */ | 196 | wmb(); /* Ensure that tail is updated after filling entry */ |
| 200 | header->tail = next_entry; | 197 | header->tail = next_entry; |
| 201 | wmb(); /* Flush WC buffer for jobq header */ | 198 | wmb(); /* Flush WC buffer for jobq header */ |
| @@ -264,7 +261,7 @@ static void job_release(struct kref *ref) · 62 unmodified lines | |||
| 264 | 261 | ||
| 265 | for (i = 0; i < job->bo_count; i++) | 262 | for (i = 0; i < job->bo_count; i++) |
| 266 | if (job->bos[i]) | 263 | if (job->bos[i]) |
| 267 | drm_gem_object_put(&job->bos[i]->base); | 264 | drm_gem_object_put(&job->bos[i]->base.base); |
| 268 | 265 | ||
| 269 | dma_fence_put(job->done_fence); | 266 | dma_fence_put(job->done_fence); |
| 270 | ivpu_file_priv_put(&job->file_priv); | 267 | ivpu_file_priv_put(&job->file_priv); |
| @@ -340,23 +337,12 @@ static int ivpu_job_done(struct ivpu_device *vdev, u32 job_id, u32 job_status) · 69 unmodified lines | |||
| 340 | ivpu_dbg(vdev, JOB, "Job complete: id %3u ctx %2d engine %d status 0x%x\n", | 337 | ivpu_dbg(vdev, JOB, "Job complete: id %3u ctx %2d engine %d status 0x%x\n", |
| 341 | job->job_id, job->file_priv->ctx.id, job->engine_idx, job_status); | 338 | job->job_id, job->file_priv->ctx.id, job->engine_idx, job_status); |
| 342 | 339 | ||
| 340 | ivpu_stop_job_timeout_detection(vdev); | ||
| 341 | |||
| 343 | job_put(job); | 342 | job_put(job); |
| 344 | return 0; | 343 | return 0; |
| 345 | } | 344 | } |
| 346 | 345 | ||
| 347 | static void ivpu_job_done_message(struct ivpu_device *vdev, void *msg) | ||
| 348 | { | ||
| 349 | struct vpu_ipc_msg_payload_job_done *payload; | ||
| 350 | struct vpu_jsm_msg *job_ret_msg = msg; | ||
| 351 | int ret; | ||
| 352 | |||
| 353 | payload = (struct vpu_ipc_msg_payload_job_done *)&job_ret_msg->payload; | ||
| 354 | |||
| 355 | ret = ivpu_job_done(vdev, payload->job_id, payload->job_status); | ||
| 356 | if (ret) | ||
| 357 | ivpu_err(vdev, "Failed to finish job %d: %d\n", payload->job_id, ret); | ||
| 358 | } | ||
| 359 | |||
| 360 | void ivpu_jobs_abort_all(struct ivpu_device *vdev) | 346 | void ivpu_jobs_abort_all(struct ivpu_device *vdev) |
| 361 | { | 347 | { |
| 362 | struct ivpu_job *job; | 348 | struct ivpu_job *job; |
| @@ -398,11 +384,13 @@ static int ivpu_direct_job_submission(struct ivpu_job *job) · 35 unmodified lines | |||
| 398 | if (ret) | 384 | if (ret) |
| 399 | goto err_xa_erase; | 385 | goto err_xa_erase; |
| 400 | 386 | ||
| 387 | ivpu_start_job_timeout_detection(vdev); | ||
| 388 | |||
| 401 | ivpu_dbg(vdev, JOB, "Job submitted: id %3u addr 0x%llx ctx %2d engine %d next %d\n", | 389 | ivpu_dbg(vdev, JOB, "Job submitted: id %3u addr 0x%llx ctx %2d engine %d next %d\n", |
| 402 | job->job_id, job->cmd_buf_vpu_addr, file_priv->ctx.id, | 390 | job->job_id, job->cmd_buf_vpu_addr, file_priv->ctx.id, |
| 403 | job->engine_idx, cmdq->jobq->header.tail); | 391 | job->engine_idx, cmdq->jobq->header.tail); |
| 404 | 392 | ||
| 405 | if (ivpu_test_mode == IVPU_TEST_MODE_NULL_HW) { | 393 | if (ivpu_test_mode & IVPU_TEST_MODE_NULL_HW) { |
| 406 | ivpu_job_done(vdev, job->job_id, VPU_JSM_STATUS_SUCCESS); | 394 | ivpu_job_done(vdev, job->job_id, VPU_JSM_STATUS_SUCCESS); |
| 407 | cmdq->jobq->header.head = cmdq->jobq->header.tail; | 395 | cmdq->jobq->header.head = cmdq->jobq->header.tail; |
| 408 | wmb(); /* Flush WC buffer for jobq header */ | 396 | wmb(); /* Flush WC buffer for jobq header */ |
| @@ -448,7 +436,7 @@ ivpu_job_prepare_bos_for_submit(struct drm_file *file, struct ivpu_job *job, u32 · 39 unmodified lines | |||
| 448 | } | 436 | } |
| 449 | 437 | ||
| 450 | bo = job->bos[CMD_BUF_IDX]; | 438 | bo = job->bos[CMD_BUF_IDX]; |
| 451 | if (!dma_resv_test_signaled(bo->base.resv, DMA_RESV_USAGE_READ)) { | 439 | if (!dma_resv_test_signaled(bo->base.base.resv, DMA_RESV_USAGE_READ)) { |
| 452 | ivpu_warn(vdev, "Buffer is already in use\n"); | 440 | ivpu_warn(vdev, "Buffer is already in use\n"); |
| 453 | return -EBUSY; | 441 | return -EBUSY; |
| 454 | } | 442 | } |
| @@ -468,7 +456,7 @@ ivpu_job_prepare_bos_for_submit(struct drm_file *file, struct ivpu_job *job, u32 · 13 unmodified lines | |||
| 468 | } | 456 | } |
| 469 | 457 | ||
| 470 | for (i = 0; i < buf_count; i++) { | 458 | for (i = 0; i < buf_count; i++) { |
| 471 | ret = dma_resv_reserve_fences(job->bos[i]->base.resv, 1); | 459 | ret = dma_resv_reserve_fences(job->bos[i]->base.base.resv, 1); |
| 472 | if (ret) { | 460 | if (ret) { |
| 473 | ivpu_warn(vdev, "Failed to reserve fences: %d\n", ret); | 461 | ivpu_warn(vdev, "Failed to reserve fences: %d\n", ret); |
| 474 | goto unlock_reservations; | 462 | goto unlock_reservations; |
| @@ -477,7 +465,7 @@ ivpu_job_prepare_bos_for_submit(struct drm_file *file, struct ivpu_job *job, u32 · 2 unmodified lines | |||
| 477 | 465 | ||
| 478 | for (i = 0; i < buf_count; i++) { | 466 | for (i = 0; i < buf_count; i++) { |
| 479 | usage = (i == CMD_BUF_IDX) ? DMA_RESV_USAGE_WRITE : DMA_RESV_USAGE_BOOKKEEP; | 467 | usage = (i == CMD_BUF_IDX) ? DMA_RESV_USAGE_WRITE : DMA_RESV_USAGE_BOOKKEEP; |
| 480 | dma_resv_add_fence(job->bos[i]->base.resv, job->done_fence, usage); | 468 | dma_resv_add_fence(job->bos[i]->base.base.resv, job->done_fence, usage); |
| 481 | } | 469 | } |
| 482 | 470 | ||
| 483 | unlock_reservations: | 471 | unlock_reservations: |
| @@ -562,61 +550,36 @@ free_handles: · 78 unmodified lines | |||
| 562 | return ret; | 550 | return ret; |
| 563 | } | 551 | } |
| 564 | 552 | ||
| 565 | static int ivpu_job_done_thread(void *arg) | 553 | static void |
| 554 | ivpu_job_done_callback(struct ivpu_device *vdev, struct ivpu_ipc_hdr *ipc_hdr, | ||
| 555 | struct vpu_jsm_msg *jsm_msg) | ||
| 566 | { | 556 | { |
| 567 | struct ivpu_device *vdev = (struct ivpu_device *)arg; | 557 | struct vpu_ipc_msg_payload_job_done *payload; |
| 568 | struct ivpu_ipc_consumer cons; | ||
| 569 | struct vpu_jsm_msg jsm_msg; | ||
| 570 | bool jobs_submitted; | ||
| 571 | unsigned int timeout; | ||
| 572 | int ret; | 558 | int ret; |
| 573 | 559 | ||
| 574 | ivpu_dbg(vdev, JOB, "Started %s\n", __func__); | 560 | if (!jsm_msg) { |
| 575 | 561 | ivpu_err(vdev, "IPC message has no JSM payload\n"); | |
| 576 | ivpu_ipc_consumer_add(vdev, &cons, VPU_IPC_CHAN_JOB_RET); | 562 | return; |
| 577 | |||
| 578 | while (!kthread_should_stop()) { | ||
| 579 | timeout = ivpu_tdr_timeout_ms ? ivpu_tdr_timeout_ms : vdev->timeout.tdr; | ||
| 580 | jobs_submitted = !xa_empty(&vdev->submitted_jobs_xa); | ||
| 581 | ret = ivpu_ipc_receive(vdev, &cons, NULL, &jsm_msg, timeout); | ||
| 582 | if (!ret) { | ||
| 583 | ivpu_job_done_message(vdev, &jsm_msg); | ||
| 584 | } else if (ret == -ETIMEDOUT) { | ||
| 585 | if (jobs_submitted && !xa_empty(&vdev->submitted_jobs_xa)) { | ||
| 586 | ivpu_err(vdev, "TDR detected, timeout %d ms", timeout); | ||
| 587 | ivpu_hw_diagnose_failure(vdev); | ||
| 588 | ivpu_pm_schedule_recovery(vdev); | ||
| 589 | } | ||
| 590 | } | ||
| 591 | } | 563 | } |
| 592 | 564 | ||
| 593 | ivpu_ipc_consumer_del(vdev, &cons); | 565 | if (jsm_msg->result != VPU_JSM_STATUS_SUCCESS) { |
| 594 | 566 | ivpu_err(vdev, "Invalid JSM message result: %d\n", jsm_msg->result); | |
| 595 | ivpu_jobs_abort_all(vdev); | 567 | return; |
| 568 | } | ||
| 596 | 569 | ||
| 597 | ivpu_dbg(vdev, JOB, "Stopped %s\n", __func__); | 570 | payload = (struct vpu_ipc_msg_payload_job_done *)&jsm_msg->payload; |
| 598 | return 0; | 571 | ret = ivpu_job_done(vdev, payload->job_id, payload->job_status); |
| 572 | if (!ret && !xa_empty(&vdev->submitted_jobs_xa)) | ||
| 573 | ivpu_start_job_timeout_detection(vdev); | ||
| 599 | } | 574 | } |
| 600 | 575 | ||
| 601 | int ivpu_job_done_thread_init(struct ivpu_device *vdev) | 576 | void ivpu_job_done_consumer_init(struct ivpu_device *vdev) |
| 602 | { | 577 | { |
| 603 | struct task_struct *thread; | 578 | ivpu_ipc_consumer_add(vdev, &vdev->job_done_consumer, |
| 604 | 579 | VPU_IPC_CHAN_JOB_RET, ivpu_job_done_callback); | |
| 605 | thread = kthread_run(&ivpu_job_done_thread, (void *)vdev, "ivpu_job_done_thread"); | ||
| 606 | if (IS_ERR(thread)) { | ||
| 607 | ivpu_err(vdev, "Failed to start job completion thread\n"); | ||
| 608 | return -EIO; | ||
| 609 | } | ||
| 610 | |||
| 611 | get_task_struct(thread); | ||
| 612 | wake_up_process(thread); | ||
| 613 | |||
| 614 | vdev->job_done_thread = thread; | ||
| 615 | |||
| 616 | return 0; | ||
| 617 | } | 580 | } |
| 618 | 581 | ||
| 619 | void ivpu_job_done_thread_fini(struct ivpu_device *vdev) | 582 | void ivpu_job_done_consumer_fini(struct ivpu_device *vdev) |
| 620 | { | 583 | { |
| 621 | kthread_stop_put(vdev->job_done_thread); | 584 | ivpu_ipc_consumer_del(vdev, &vdev->job_done_consumer); |
| 622 | } | 585 | } |
| @@ -59,8 +59,8 @@ int ivpu_submit_ioctl(struct drm_device *dev, void *data, struct drm_file *file) · 58 unmodified lines | |||
| 59 | void ivpu_cmdq_release_all(struct ivpu_file_priv *file_priv); | 59 | void ivpu_cmdq_release_all(struct ivpu_file_priv *file_priv); |
| 60 | void ivpu_cmdq_reset_all_contexts(struct ivpu_device *vdev); | 60 | void ivpu_cmdq_reset_all_contexts(struct ivpu_device *vdev); |
| 61 | 61 | ||
| 62 | int ivpu_job_done_thread_init(struct ivpu_device *vdev); | 62 | void ivpu_job_done_consumer_init(struct ivpu_device *vdev); |
| 63 | void ivpu_job_done_thread_fini(struct ivpu_device *vdev); | 63 | void ivpu_job_done_consumer_fini(struct ivpu_device *vdev); |
| 64 | 64 | ||
| 65 | void ivpu_jobs_abort_all(struct ivpu_device *vdev); | 65 | void ivpu_jobs_abort_all(struct ivpu_device *vdev); |
| 66 | 66 | ||
| @@ -4,6 +4,7 @@ · 3 unmodified lines | |||
| 4 | */ | 4 | */ |
| 5 | 5 | ||
| 6 | #include "ivpu_drv.h" | 6 | #include "ivpu_drv.h" |
| 7 | #include "ivpu_hw.h" | ||
| 7 | #include "ivpu_ipc.h" | 8 | #include "ivpu_ipc.h" |
| 8 | #include "ivpu_jsm_msg.h" | 9 | #include "ivpu_jsm_msg.h" |
| 9 | 10 | ||
| @@ -36,6 +37,17 @@ const char *ivpu_jsm_msg_type_to_str(enum vpu_ipc_msg_type type) · 26 unmodified lines | |||
| 36 | IVPU_CASE_TO_STR(VPU_JSM_MSG_DESTROY_CMD_QUEUE); | 37 | IVPU_CASE_TO_STR(VPU_JSM_MSG_DESTROY_CMD_QUEUE); |
| 37 | IVPU_CASE_TO_STR(VPU_JSM_MSG_SET_CONTEXT_SCHED_PROPERTIES); | 38 | IVPU_CASE_TO_STR(VPU_JSM_MSG_SET_CONTEXT_SCHED_PROPERTIES); |
| 38 | IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_REGISTER_DB); | 39 | IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_REGISTER_DB); |
| 40 | IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_RESUME_CMDQ); | ||
| 41 | IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_SUSPEND_CMDQ); | ||
| 42 | IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_RESUME_CMDQ_RSP); | ||
| 43 | IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_SUSPEND_CMDQ_DONE); | ||
| 44 | IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG); | ||
| 45 | IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG_RSP); | ||
| 46 | IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION); | ||
| 47 | IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_ENGINE_RESUME); | ||
| 48 | IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_RESUME_ENGINE_DONE); | ||
| 49 | IVPU_CASE_TO_STR(VPU_JSM_MSG_STATE_DUMP); | ||
| 50 | IVPU_CASE_TO_STR(VPU_JSM_MSG_STATE_DUMP_RSP); | ||
| 39 | IVPU_CASE_TO_STR(VPU_JSM_MSG_BLOB_DEINIT); | 51 | IVPU_CASE_TO_STR(VPU_JSM_MSG_BLOB_DEINIT); |
| 40 | IVPU_CASE_TO_STR(VPU_JSM_MSG_DYNDBG_CONTROL); | 52 | IVPU_CASE_TO_STR(VPU_JSM_MSG_DYNDBG_CONTROL); |
| 41 | IVPU_CASE_TO_STR(VPU_JSM_MSG_JOB_DONE); | 53 | IVPU_CASE_TO_STR(VPU_JSM_MSG_JOB_DONE); |
| @@ -65,6 +77,12 @@ const char *ivpu_jsm_msg_type_to_str(enum vpu_ipc_msg_type type) · 23 unmodified lines | |||
| 65 | IVPU_CASE_TO_STR(VPU_JSM_MSG_SET_CONTEXT_SCHED_PROPERTIES_RSP); | 77 | IVPU_CASE_TO_STR(VPU_JSM_MSG_SET_CONTEXT_SCHED_PROPERTIES_RSP); |
| 66 | IVPU_CASE_TO_STR(VPU_JSM_MSG_BLOB_DEINIT_DONE); | 78 | IVPU_CASE_TO_STR(VPU_JSM_MSG_BLOB_DEINIT_DONE); |
| 67 | IVPU_CASE_TO_STR(VPU_JSM_MSG_DYNDBG_CONTROL_RSP); | 79 | IVPU_CASE_TO_STR(VPU_JSM_MSG_DYNDBG_CONTROL_RSP); |
| 80 | IVPU_CASE_TO_STR(VPU_JSM_MSG_PWR_D0I3_ENTER); | ||
| 81 | IVPU_CASE_TO_STR(VPU_JSM_MSG_PWR_D0I3_ENTER_DONE); | ||
| 82 | IVPU_CASE_TO_STR(VPU_JSM_MSG_DCT_ENABLE); | ||
| 83 | IVPU_CASE_TO_STR(VPU_JSM_MSG_DCT_ENABLE_DONE); | ||
| 84 | IVPU_CASE_TO_STR(VPU_JSM_MSG_DCT_DISABLE); | ||
| 85 | IVPU_CASE_TO_STR(VPU_JSM_MSG_DCT_DISABLE_DONE); | ||
| 68 | } | 86 | } |
| 69 | #undef IVPU_CASE_TO_STR | 87 | #undef IVPU_CASE_TO_STR |
| 70 | 88 | ||
| @@ -243,3 +261,23 @@ int ivpu_jsm_context_release(struct ivpu_device *vdev, u32 host_ssid) · 172 unmodified lines | |||
| 243 | return ivpu_ipc_send_receive(vdev, &req, VPU_JSM_MSG_SSID_RELEASE_DONE, &resp, | 261 | return ivpu_ipc_send_receive(vdev, &req, VPU_JSM_MSG_SSID_RELEASE_DONE, &resp, |
| 244 | VPU_IPC_CHAN_ASYNC_CMD, vdev->timeout.jsm); | 262 | VPU_IPC_CHAN_ASYNC_CMD, vdev->timeout.jsm); |
| 245 | } | 263 | } |
| 264 | |||
| 265 | int ivpu_jsm_pwr_d0i3_enter(struct ivpu_device *vdev) | ||
| 266 | { | ||
| 267 | struct vpu_jsm_msg req = { .type = VPU_JSM_MSG_PWR_D0I3_ENTER }; | ||
| 268 | struct vpu_jsm_msg resp; | ||
| 269 | int ret; | ||
| 270 | |||
| 271 | if (IVPU_WA(disable_d0i3_msg)) | ||
| 272 | return 0; | ||
| 273 | |||
| 274 | req.payload.pwr_d0i3_enter.send_response = 1; | ||
| 275 | |||
| 276 | ret = ivpu_ipc_send_receive_active(vdev, &req, VPU_JSM_MSG_PWR_D0I3_ENTER_DONE, | ||
| 277 | &resp, VPU_IPC_CHAN_GEN_CMD, | ||
| 278 | vdev->timeout.d0i3_entry_msg); | ||
| 279 | if (ret) | ||
| 280 | return ret; | ||
| 281 | |||
| 282 | return ivpu_hw_wait_for_idle(vdev); | ||
| 283 | } | ||
| @@ -22,4 +22,5 @@ int ivpu_jsm_trace_get_capability(struct ivpu_device *vdev, u32 *trace_destinati · 21 unmodified lines | |||
| 22 | int ivpu_jsm_trace_set_config(struct ivpu_device *vdev, u32 trace_level, u32 trace_destination_mask, | 22 | int ivpu_jsm_trace_set_config(struct ivpu_device *vdev, u32 trace_level, u32 trace_destination_mask, |
| 23 | u64 trace_hw_component_mask); | 23 | u64 trace_hw_component_mask); |
| 24 | int ivpu_jsm_context_release(struct ivpu_device *vdev, u32 host_ssid); | 24 | int ivpu_jsm_context_release(struct ivpu_device *vdev, u32 host_ssid); |
| 25 | int ivpu_jsm_pwr_d0i3_enter(struct ivpu_device *vdev); | ||
| 25 | #endif | 26 | #endif |
| @@ -230,7 +230,12 @@ · 229 unmodified lines | |||
| 230 | (REG_FLD(IVPU_MMU_REG_GERROR, MSI_PRIQ_ABT)) | \ | 230 | (REG_FLD(IVPU_MMU_REG_GERROR, MSI_PRIQ_ABT)) | \ |
| 231 | (REG_FLD(IVPU_MMU_REG_GERROR, MSI_ABT))) | 231 | (REG_FLD(IVPU_MMU_REG_GERROR, MSI_ABT))) |
| 232 | 232 | ||
| 233 | static char *ivpu_mmu_event_to_str(u32 cmd) | 233 | #define IVPU_MMU_CERROR_NONE 0x0 |
| 234 | #define IVPU_MMU_CERROR_ILL 0x1 | ||
| 235 | #define IVPU_MMU_CERROR_ABT 0x2 | ||
| 236 | #define IVPU_MMU_CERROR_ATC_INV_SYNC 0x3 | ||
| 237 | |||
| 238 | static const char *ivpu_mmu_event_to_str(u32 cmd) | ||
| 234 | { | 239 | { |
| 235 | switch (cmd) { | 240 | switch (cmd) { |
| 236 | case IVPU_MMU_EVT_F_UUT: | 241 | case IVPU_MMU_EVT_F_UUT: |
| @@ -276,6 +281,22 @@ static char *ivpu_mmu_event_to_str(u32 cmd) · 39 unmodified lines | |||
| 276 | } | 281 | } |
| 277 | } | 282 | } |
| 278 | 283 | ||
| 284 | static const char *ivpu_mmu_cmdq_err_to_str(u32 err) | ||
| 285 | { | ||
| 286 | switch (err) { | ||
| 287 | case IVPU_MMU_CERROR_NONE: | ||
| 288 | return "No CMDQ Error"; | ||
| 289 | case IVPU_MMU_CERROR_ILL: | ||
| 290 | return "Illegal command"; | ||
| 291 | case IVPU_MMU_CERROR_ABT: | ||
| 292 | return "External abort on CMDQ read"; | ||
| 293 | case IVPU_MMU_CERROR_ATC_INV_SYNC: | ||
| 294 | return "Sync failed to complete ATS invalidation"; | ||
| 295 | default: | ||
| 296 | return "Unknown CMDQ Error"; | ||
| 297 | } | ||
| 298 | } | ||
| 299 | |||
| 279 | static void ivpu_mmu_config_check(struct ivpu_device *vdev) | 300 | static void ivpu_mmu_config_check(struct ivpu_device *vdev) |
| 280 | { | 301 | { |
| 281 | u32 val_ref; | 302 | u32 val_ref; |
| @@ -479,10 +500,7 @@ static int ivpu_mmu_cmdq_sync(struct ivpu_device *vdev) · 197 unmodified lines | |||
| 479 | u64 val; | 500 | u64 val; |
| 480 | int ret; | 501 | int ret; |
| 481 | 502 | ||
| 482 | val = FIELD_PREP(IVPU_MMU_CMD_OPCODE, CMD_SYNC) | | 503 | val = FIELD_PREP(IVPU_MMU_CMD_OPCODE, CMD_SYNC); |
| 483 | FIELD_PREP(IVPU_MMU_CMD_SYNC_0_CS, 0x2) | | ||
| 484 | FIELD_PREP(IVPU_MMU_CMD_SYNC_0_MSH, 0x3) | | ||
| 485 | FIELD_PREP(IVPU_MMU_CMD_SYNC_0_MSI_ATTR, 0xf); | ||
| 486 | 504 | ||
| 487 | ret = ivpu_mmu_cmdq_cmd_write(vdev, "SYNC", val, 0); | 505 | ret = ivpu_mmu_cmdq_cmd_write(vdev, "SYNC", val, 0); |
| 488 | if (ret) | 506 | if (ret) |
| @@ -492,8 +510,15 @@ static int ivpu_mmu_cmdq_sync(struct ivpu_device *vdev) · 3 unmodified lines | |||
| 492 | REGV_WR32(IVPU_MMU_REG_CMDQ_PROD, q->prod); | 510 | REGV_WR32(IVPU_MMU_REG_CMDQ_PROD, q->prod); |
| 493 | 511 | ||
| 494 | ret = ivpu_mmu_cmdq_wait_for_cons(vdev); | 512 | ret = ivpu_mmu_cmdq_wait_for_cons(vdev); |
| 495 | if (ret) | 513 | if (ret) { |
| 496 | ivpu_err(vdev, "Timed out waiting for consumer: %d\n", ret); | 514 | u32 err; |
| 515 | |||
| 516 | val = REGV_RD32(IVPU_MMU_REG_CMDQ_CONS); | ||
| 517 | err = REG_GET_FLD(IVPU_MMU_REG_CMDQ_CONS, ERR, val); | ||
| 518 | |||
| 519 | ivpu_err(vdev, "Timed out waiting for MMU consumer: %d, error: %s\n", ret, | ||
| 520 | ivpu_mmu_cmdq_err_to_str(err)); | ||
| 521 | } | ||
| 497 | 522 | ||
| 498 | return ret; | 523 | return ret; |
| 499 | } | 524 | } |
| @@ -750,9 +775,12 @@ int ivpu_mmu_init(struct ivpu_device *vdev) · 250 unmodified lines | |||
| 750 | 775 | ||
| 751 | ivpu_dbg(vdev, MMU, "Init..\n"); | 776 | ivpu_dbg(vdev, MMU, "Init..\n"); |
| 752 | 777 | ||
| 753 | drmm_mutex_init(&vdev->drm, &mmu->lock); | ||
| 754 | ivpu_mmu_config_check(vdev); | 778 | ivpu_mmu_config_check(vdev); |
| 755 | 779 | ||
| 780 | ret = drmm_mutex_init(&vdev->drm, &mmu->lock); | ||
| 781 | if (ret) | ||
| 782 | return ret; | ||
| 783 | |||
| 756 | ret = ivpu_mmu_structs_alloc(vdev); | 784 | ret = ivpu_mmu_structs_alloc(vdev); |
| 757 | if (ret) | 785 | if (ret) |
| 758 | return ret; | 786 | return ret; |
| @@ -5,6 +5,9 @@ · 4 unmodified lines | |||
| 5 | 5 | ||
| 6 | #include <linux/bitfield.h> | 6 | #include <linux/bitfield.h> |
| 7 | #include <linux/highmem.h> | 7 | #include <linux/highmem.h> |
| 8 | #include <linux/set_memory.h> | ||
| 9 | |||
| 10 | #include <drm/drm_cache.h> | ||
| 8 | 11 | ||
| 9 | #include "ivpu_drv.h" | 12 | #include "ivpu_drv.h" |
| 10 | #include "ivpu_hw.h" | 13 | #include "ivpu_hw.h" |
| @@ -39,12 +42,57 @@ · 28 unmodified lines | |||
| 39 | #define IVPU_MMU_ENTRY_MAPPED (IVPU_MMU_ENTRY_FLAG_AF | IVPU_MMU_ENTRY_FLAG_USER | \ | 42 | #define IVPU_MMU_ENTRY_MAPPED (IVPU_MMU_ENTRY_FLAG_AF | IVPU_MMU_ENTRY_FLAG_USER | \ |
| 40 | IVPU_MMU_ENTRY_FLAG_NG | IVPU_MMU_ENTRY_VALID) | 43 | IVPU_MMU_ENTRY_FLAG_NG | IVPU_MMU_ENTRY_VALID) |
| 41 | 44 | ||
| 45 | static void *ivpu_pgtable_alloc_page(struct ivpu_device *vdev, dma_addr_t *dma) | ||
| 46 | { | ||
| 47 | dma_addr_t dma_addr; | ||
| 48 | struct page *page; | ||
| 49 | void *cpu; | ||
| 50 | |||
| 51 | page = alloc_page(GFP_KERNEL | __GFP_HIGHMEM | __GFP_ZERO); | ||
| 52 | if (!page) | ||
| 53 | return NULL; | ||
| 54 | |||
| 55 | set_pages_array_wc(&page, 1); | ||
| 56 | |||
| 57 | dma_addr = dma_map_page(vdev->drm.dev, page, 0, PAGE_SIZE, DMA_BIDIRECTIONAL); | ||
| 58 | if (dma_mapping_error(vdev->drm.dev, dma_addr)) | ||
| 59 | goto err_free_page; | ||
| 60 | |||
| 61 | cpu = vmap(&page, 1, VM_MAP, pgprot_writecombine(PAGE_KERNEL)); | ||
| 62 | if (!cpu) | ||
| 63 | goto err_dma_unmap_page; | ||
| 64 | |||
| 65 | |||
| 66 | *dma = dma_addr; | ||
| 67 | return cpu; | ||
| 68 | |||
| 69 | err_dma_unmap_page: | ||
| 70 | dma_unmap_page(vdev->drm.dev, dma_addr, PAGE_SIZE, DMA_BIDIRECTIONAL); | ||
| 71 | |||
| 72 | err_free_page: | ||
| 73 | put_page(page); | ||
| 74 | return NULL; | ||
| 75 | } | ||
| 76 | |||
| 77 | static void ivpu_pgtable_free_page(struct ivpu_device *vdev, u64 *cpu_addr, dma_addr_t dma_addr) | ||
| 78 | { | ||
| 79 | struct page *page; | ||
| 80 | |||
| 81 | if (cpu_addr) { | ||
| 82 | page = vmalloc_to_page(cpu_addr); | ||
| 83 | vunmap(cpu_addr); | ||
| 84 | dma_unmap_page(vdev->drm.dev, dma_addr & ~IVPU_MMU_ENTRY_FLAGS_MASK, PAGE_SIZE, | ||
| 85 | DMA_BIDIRECTIONAL); | ||
| 86 | set_pages_array_wb(&page, 1); | ||
| 87 | put_page(page); | ||
| 88 | } | ||
| 89 | } | ||
| 90 | |||
| 42 | static int ivpu_mmu_pgtable_init(struct ivpu_device *vdev, struct ivpu_mmu_pgtable *pgtable) | 91 | static int ivpu_mmu_pgtable_init(struct ivpu_device *vdev, struct ivpu_mmu_pgtable *pgtable) |
| 43 | { | 92 | { |
| 44 | dma_addr_t pgd_dma; | 93 | dma_addr_t pgd_dma; |
| 45 | 94 | ||
| 46 | pgtable->pgd_dma_ptr = dma_alloc_coherent(vdev->drm.dev, IVPU_MMU_PGTABLE_SIZE, &pgd_dma, | 95 | pgtable->pgd_dma_ptr = ivpu_pgtable_alloc_page(vdev, &pgd_dma); |
| 47 | GFP_KERNEL); | ||
| 48 | if (!pgtable->pgd_dma_ptr) | 96 | if (!pgtable->pgd_dma_ptr) |
| 49 | return -ENOMEM; | 97 | return -ENOMEM; |
| 50 | 98 | ||
| @@ -53,13 +101,6 @@ static int ivpu_mmu_pgtable_init(struct ivpu_device *vdev, struct ivpu_mmu_pgtab · 2 unmodified lines | |||
| 53 | return 0; | 101 | return 0; |
| 54 | } | 102 | } |
| 55 | 103 | ||
| 56 | static void ivpu_mmu_pgtable_free(struct ivpu_device *vdev, u64 *cpu_addr, dma_addr_t dma_addr) | ||
| 57 | { | ||
| 58 | if (cpu_addr) | ||
| 59 | dma_free_coherent(vdev->drm.dev, IVPU_MMU_PGTABLE_SIZE, cpu_addr, | ||
| 60 | dma_addr & ~IVPU_MMU_ENTRY_FLAGS_MASK); | ||
| 61 | } | ||
| 62 | |||
| 63 | static void ivpu_mmu_pgtables_free(struct ivpu_device *vdev, struct ivpu_mmu_pgtable *pgtable) | 104 | static void ivpu_mmu_pgtables_free(struct ivpu_device *vdev, struct ivpu_mmu_pgtable *pgtable) |
| 64 | { | 105 | { |
| 65 | int pgd_idx, pud_idx, pmd_idx; | 106 | int pgd_idx, pud_idx, pmd_idx; |
| @@ -84,19 +125,19 @@ static void ivpu_mmu_pgtables_free(struct ivpu_device *vdev, struct ivpu_mmu_pgt · 18 unmodified lines | |||
| 84 | pte_dma_ptr = pgtable->pte_ptrs[pgd_idx][pud_idx][pmd_idx]; | 125 | pte_dma_ptr = pgtable->pte_ptrs[pgd_idx][pud_idx][pmd_idx]; |
| 85 | pte_dma = pgtable->pmd_ptrs[pgd_idx][pud_idx][pmd_idx]; | 126 | pte_dma = pgtable->pmd_ptrs[pgd_idx][pud_idx][pmd_idx]; |
| 86 | 127 | ||
| 87 | ivpu_mmu_pgtable_free(vdev, pte_dma_ptr, pte_dma); | 128 | ivpu_pgtable_free_page(vdev, pte_dma_ptr, pte_dma); |
| 88 | } | 129 | } |
| 89 | 130 | ||
| 90 | kfree(pgtable->pte_ptrs[pgd_idx][pud_idx]); | 131 | kfree(pgtable->pte_ptrs[pgd_idx][pud_idx]); |
| 91 | ivpu_mmu_pgtable_free(vdev, pmd_dma_ptr, pmd_dma); | 132 | ivpu_pgtable_free_page(vdev, pmd_dma_ptr, pmd_dma); |
| 92 | } | 133 | } |
| 93 | 134 | ||
| 94 | kfree(pgtable->pmd_ptrs[pgd_idx]); | 135 | kfree(pgtable->pmd_ptrs[pgd_idx]); |
| 95 | kfree(pgtable->pte_ptrs[pgd_idx]); | 136 | kfree(pgtable->pte_ptrs[pgd_idx]); |
| 96 | ivpu_mmu_pgtable_free(vdev, pud_dma_ptr, pud_dma); | 137 | ivpu_pgtable_free_page(vdev, pud_dma_ptr, pud_dma); |
| 97 | } | 138 | } |
| 98 | 139 | ||
| 99 | ivpu_mmu_pgtable_free(vdev, pgtable->pgd_dma_ptr, pgtable->pgd_dma); | 140 | ivpu_pgtable_free_page(vdev, pgtable->pgd_dma_ptr, pgtable->pgd_dma); |
| 100 | } | 141 | } |
| 101 | 142 | ||
| 102 | static u64* | 143 | static u64* |
| @@ -108,7 +149,7 @@ ivpu_mmu_ensure_pud(struct ivpu_device *vdev, struct ivpu_mmu_pgtable *pgtable, · 5 unmodified lines | |||
| 108 | if (pud_dma_ptr) | 149 | if (pud_dma_ptr) |
| 109 | return pud_dma_ptr; | 150 | return pud_dma_ptr; |
| 110 | 151 | ||
| 111 | pud_dma_ptr = dma_alloc_wc(vdev->drm.dev, IVPU_MMU_PGTABLE_SIZE, &pud_dma, GFP_KERNEL); | 152 | pud_dma_ptr = ivpu_pgtable_alloc_page(vdev, &pud_dma); |
| 112 | if (!pud_dma_ptr) | 153 | if (!pud_dma_ptr) |
| 113 | return NULL; | 154 | return NULL; |
| 114 | 155 | ||
| @@ -131,7 +172,7 @@ err_free_pmd_ptrs: · 16 unmodified lines | |||
| 131 | kfree(pgtable->pmd_ptrs[pgd_idx]); | 172 | kfree(pgtable->pmd_ptrs[pgd_idx]); |
| 132 | 173 | ||
| 133 | err_free_pud_dma_ptr: | 174 | err_free_pud_dma_ptr: |
| 134 | ivpu_mmu_pgtable_free(vdev, pud_dma_ptr, pud_dma); | 175 | ivpu_pgtable_free_page(vdev, pud_dma_ptr, pud_dma); |
| 135 | return NULL; | 176 | return NULL; |
| 136 | } | 177 | } |
| 137 | 178 | ||
| @@ -145,7 +186,7 @@ ivpu_mmu_ensure_pmd(struct ivpu_device *vdev, struct ivpu_mmu_pgtable *pgtable, · 7 unmodified lines | |||
| 145 | if (pmd_dma_ptr) | 186 | if (pmd_dma_ptr) |
| 146 | return pmd_dma_ptr; | 187 | return pmd_dma_ptr; |
| 147 | 188 | ||
| 148 | pmd_dma_ptr = dma_alloc_wc(vdev->drm.dev, IVPU_MMU_PGTABLE_SIZE, &pmd_dma, GFP_KERNEL); | 189 | pmd_dma_ptr = ivpu_pgtable_alloc_page(vdev, &pmd_dma); |
| 149 | if (!pmd_dma_ptr) | 190 | if (!pmd_dma_ptr) |
| 150 | return NULL; | 191 | return NULL; |
| 151 | 192 | ||
| @@ -160,7 +201,7 @@ ivpu_mmu_ensure_pmd(struct ivpu_device *vdev, struct ivpu_mmu_pgtable *pgtable, · 8 unmodified lines | |||
| 160 | return pmd_dma_ptr; | 201 | return pmd_dma_ptr; |
| 161 | 202 | ||
| 162 | err_free_pmd_dma_ptr: | 203 | err_free_pmd_dma_ptr: |
| 163 | ivpu_mmu_pgtable_free(vdev, pmd_dma_ptr, pmd_dma); | 204 | ivpu_pgtable_free_page(vdev, pmd_dma_ptr, pmd_dma); |
| 164 | return NULL; | 205 | return NULL; |
| 165 | } | 206 | } |
| 166 | 207 | ||
| @@ -174,7 +215,7 @@ ivpu_mmu_ensure_pte(struct ivpu_device *vdev, struct ivpu_mmu_pgtable *pgtable, · 7 unmodified lines | |||
| 174 | if (pte_dma_ptr) | 215 | if (pte_dma_ptr) |
| 175 | return pte_dma_ptr; | 216 | return pte_dma_ptr; |
| 176 | 217 | ||
| 177 | pte_dma_ptr = dma_alloc_wc(vdev->drm.dev, IVPU_MMU_PGTABLE_SIZE, &pte_dma, GFP_KERNEL); | 218 | pte_dma_ptr = ivpu_pgtable_alloc_page(vdev, &pte_dma); |
| 178 | if (!pte_dma_ptr) | 219 | if (!pte_dma_ptr) |
| 179 | return NULL; | 220 | return NULL; |
| 180 | 221 | ||
| @@ -249,38 +290,6 @@ static void ivpu_mmu_context_unmap_page(struct ivpu_mmu_context *ctx, u64 vpu_ad · 68 unmodified lines | |||
| 249 | ctx->pgtable.pte_ptrs[pgd_idx][pud_idx][pmd_idx][pte_idx] = IVPU_MMU_ENTRY_INVALID; | 290 | ctx->pgtable.pte_ptrs[pgd_idx][pud_idx][pmd_idx][pte_idx] = IVPU_MMU_ENTRY_INVALID; |
| 250 | } | 291 | } |
| 251 | 292 | ||
| 252 | static void | ||
| 253 | ivpu_mmu_context_flush_page_tables(struct ivpu_mmu_context *ctx, u64 vpu_addr, size_t size) | ||
| 254 | { | ||
| 255 | struct ivpu_mmu_pgtable *pgtable = &ctx->pgtable; | ||
| 256 | u64 end_addr = vpu_addr + size; | ||
| 257 | |||
| 258 | /* Align to PMD entry (2 MB) */ | ||
| 259 | vpu_addr &= ~(IVPU_MMU_PTE_MAP_SIZE - 1); | ||
| 260 | |||
| 261 | while (vpu_addr < end_addr) { | ||
| 262 | int pgd_idx = FIELD_GET(IVPU_MMU_PGD_INDEX_MASK, vpu_addr); | ||
| 263 | u64 pud_end = (pgd_idx + 1) * (u64)IVPU_MMU_PUD_MAP_SIZE; | ||
| 264 | |||
| 265 | while (vpu_addr < end_addr && vpu_addr < pud_end) { | ||
| 266 | int pud_idx = FIELD_GET(IVPU_MMU_PUD_INDEX_MASK, vpu_addr); | ||
| 267 | u64 pmd_end = (pud_idx + 1) * (u64)IVPU_MMU_PMD_MAP_SIZE; | ||
| 268 | |||
| 269 | while (vpu_addr < end_addr && vpu_addr < pmd_end) { | ||
| 270 | int pmd_idx = FIELD_GET(IVPU_MMU_PMD_INDEX_MASK, vpu_addr); | ||
| 271 | |||
| 272 | clflush_cache_range(pgtable->pte_ptrs[pgd_idx][pud_idx][pmd_idx], | ||
| 273 | IVPU_MMU_PGTABLE_SIZE); | ||
| 274 | vpu_addr += IVPU_MMU_PTE_MAP_SIZE; | ||
| 275 | } | ||
| 276 | clflush_cache_range(pgtable->pmd_ptrs[pgd_idx][pud_idx], | ||
| 277 | IVPU_MMU_PGTABLE_SIZE); | ||
| 278 | } | ||
| 279 | clflush_cache_range(pgtable->pud_ptrs[pgd_idx], IVPU_MMU_PGTABLE_SIZE); | ||
| 280 | } | ||
| 281 | clflush_cache_range(pgtable->pgd_dma_ptr, IVPU_MMU_PGTABLE_SIZE); | ||
| 282 | } | ||
| 283 | |||
| 284 | static int | 293 | static int |
| 285 | ivpu_mmu_context_map_pages(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, | 294 | ivpu_mmu_context_map_pages(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, |
| 286 | u64 vpu_addr, dma_addr_t dma_addr, size_t size, u64 prot) | 295 | u64 vpu_addr, dma_addr_t dma_addr, size_t size, u64 prot) |
| @@ -327,6 +336,9 @@ ivpu_mmu_context_map_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, · 40 unmodified lines | |||
| 327 | u64 prot; | 336 | u64 prot; |
| 328 | u64 i; | 337 | u64 i; |
| 329 | 338 | ||
| 339 | if (drm_WARN_ON(&vdev->drm, !ctx)) | ||
| 340 | return -EINVAL; | ||
| 341 | |||
| 330 | if (!IS_ALIGNED(vpu_addr, IVPU_MMU_PAGE_SIZE)) | 342 | if (!IS_ALIGNED(vpu_addr, IVPU_MMU_PAGE_SIZE)) |
| 331 | return -EINVAL; | 343 | return -EINVAL; |
| 332 | 344 | ||
| @@ -349,10 +361,11 @@ ivpu_mmu_context_map_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, · 16 unmodified lines | |||
| 349 | mutex_unlock(&ctx->lock); | 361 | mutex_unlock(&ctx->lock); |
| 350 | return ret; | 362 | return ret; |
| 351 | } | 363 | } |
| 352 | ivpu_mmu_context_flush_page_tables(ctx, vpu_addr, size); | ||
| 353 | vpu_addr += size; | 364 | vpu_addr += size; |
| 354 | } | 365 | } |
| 355 | 366 | ||
| 367 | /* Ensure page table modifications are flushed from wc buffers to memory */ | ||
| 368 | wmb(); | ||
| 356 | mutex_unlock(&ctx->lock); | 369 | mutex_unlock(&ctx->lock); |
| 357 | 370 | ||
| 358 | ret = ivpu_mmu_invalidate_tlb(vdev, ctx->id); | 371 | ret = ivpu_mmu_invalidate_tlb(vdev, ctx->id); |
| @@ -369,8 +382,8 @@ ivpu_mmu_context_unmap_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ct · 10 unmodified lines | |||
| 369 | int ret; | 382 | int ret; |
| 370 | u64 i; | 383 | u64 i; |
| 371 | 384 | ||
| 372 | if (!IS_ALIGNED(vpu_addr, IVPU_MMU_PAGE_SIZE)) | 385 | if (drm_WARN_ON(&vdev->drm, !ctx)) |
| 373 | ivpu_warn(vdev, "Unaligned vpu_addr: 0x%llx\n", vpu_addr); | 386 | return; |
| 374 | 387 | ||
| 375 | mutex_lock(&ctx->lock); | 388 | mutex_lock(&ctx->lock); |
| 376 | 389 | ||
| @@ -378,10 +391,11 @@ ivpu_mmu_context_unmap_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ct · 1 unmodified line | |||
| 378 | size_t size = sg_dma_len(sg) + sg->offset; | 391 | size_t size = sg_dma_len(sg) + sg->offset; |
| 379 | 392 | ||
| 380 | ivpu_mmu_context_unmap_pages(ctx, vpu_addr, size); | 393 | ivpu_mmu_context_unmap_pages(ctx, vpu_addr, size); |
| 381 | ivpu_mmu_context_flush_page_tables(ctx, vpu_addr, size); | ||
| 382 | vpu_addr += size; | 394 | vpu_addr += size; |
| 383 | } | 395 | } |
| 384 | 396 | ||
| 397 | /* Ensure page table modifications are flushed from wc buffers to memory */ | ||
| 398 | wmb(); | ||
| 385 | mutex_unlock(&ctx->lock); | 399 | mutex_unlock(&ctx->lock); |
| 386 | 400 | ||
| 387 | ret = ivpu_mmu_invalidate_tlb(vdev, ctx->id); | 401 | ret = ivpu_mmu_invalidate_tlb(vdev, ctx->id); |
| @@ -390,28 +404,34 @@ ivpu_mmu_context_unmap_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ct · 2 unmodified lines | |||
| 390 | } | 404 | } |
| 391 | 405 | ||
| 392 | int | 406 | int |
| 393 | ivpu_mmu_context_insert_node_locked(struct ivpu_mmu_context *ctx, | 407 | ivpu_mmu_context_insert_node(struct ivpu_mmu_context *ctx, const struct ivpu_addr_range *range, |
| 394 | const struct ivpu_addr_range *range, | 408 | u64 size, struct drm_mm_node *node) |
| 395 | u64 size, struct drm_mm_node *node) | ||
| 396 | { | 409 | { |
| 397 | lockdep_assert_held(&ctx->lock); | 410 | int ret; |
| 411 | |||
| 412 | WARN_ON(!range); | ||
| 398 | 413 | ||
| 414 | mutex_lock(&ctx->lock); | ||
| 399 | if (!ivpu_disable_mmu_cont_pages && size >= IVPU_MMU_CONT_PAGES_SIZE) { | 415 | if (!ivpu_disable_mmu_cont_pages && size >= IVPU_MMU_CONT_PAGES_SIZE) { |
| 400 | if (!drm_mm_insert_node_in_range(&ctx->mm, node, size, IVPU_MMU_CONT_PAGES_SIZE, 0, | 416 | ret = drm_mm_insert_node_in_range(&ctx->mm, node, size, IVPU_MMU_CONT_PAGES_SIZE, 0, |
| 401 | range->start, range->end, DRM_MM_INSERT_BEST)) | 417 | range->start, range->end, DRM_MM_INSERT_BEST); |
| 402 | return 0; | 418 | if (!ret) |
| 419 | goto unlock; | ||
| 403 | } | 420 | } |
| 404 | 421 | ||
| 405 | return drm_mm_insert_node_in_range(&ctx->mm, node, size, IVPU_MMU_PAGE_SIZE, 0, | 422 | ret = drm_mm_insert_node_in_range(&ctx->mm, node, size, IVPU_MMU_PAGE_SIZE, 0, |
| 406 | range->start, range->end, DRM_MM_INSERT_BEST); | 423 | range->start, range->end, DRM_MM_INSERT_BEST); |
| 424 | unlock: | ||
| 425 | mutex_unlock(&ctx->lock); | ||
| 426 | return ret; | ||
| 407 | } | 427 | } |
| 408 | 428 | ||
| 409 | void | 429 | void |
| 410 | ivpu_mmu_context_remove_node_locked(struct ivpu_mmu_context *ctx, struct drm_mm_node *node) | 430 | ivpu_mmu_context_remove_node(struct ivpu_mmu_context *ctx, struct drm_mm_node *node) |
| 411 | { | 431 | { |
| 412 | lockdep_assert_held(&ctx->lock); | 432 | mutex_lock(&ctx->lock); |
| 413 | |||
| 414 | drm_mm_remove_node(node); | 433 | drm_mm_remove_node(node); |
| 434 | mutex_unlock(&ctx->lock); | ||
| 415 | } | 435 | } |
| 416 | 436 | ||
| 417 | static int | 437 | static int |
| @@ -421,7 +441,6 @@ ivpu_mmu_context_init(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, u3 · 3 unmodified lines | |||
| 421 | int ret; | 441 | int ret; |
| 422 | 442 | ||
| 423 | mutex_init(&ctx->lock); | 443 | mutex_init(&ctx->lock); |
| 424 | INIT_LIST_HEAD(&ctx->bo_list); | ||
| 425 | 444 | ||
| 426 | ret = ivpu_mmu_pgtable_init(vdev, &ctx->pgtable); | 445 | ret = ivpu_mmu_pgtable_init(vdev, &ctx->pgtable); |
| 427 | if (ret) { | 446 | if (ret) { |
| @@ -23,10 +23,9 @@ struct ivpu_mmu_pgtable { · 22 unmodified lines | |||
| 23 | }; | 23 | }; |
| 24 | 24 | ||
| 25 | struct ivpu_mmu_context { | 25 | struct ivpu_mmu_context { |
| 26 | struct mutex lock; /* protects: mm, pgtable, bo_list */ | 26 | struct mutex lock; /* Protects: mm, pgtable */ |
| 27 | struct drm_mm mm; | 27 | struct drm_mm mm; |
| 28 | struct ivpu_mmu_pgtable pgtable; | 28 | struct ivpu_mmu_pgtable pgtable; |
| 29 | struct list_head bo_list; | ||
| 30 | u32 id; | 29 | u32 id; |
| 31 | }; | 30 | }; |
| 32 | 31 | ||
| @@ -39,11 +38,9 @@ int ivpu_mmu_user_context_init(struct ivpu_device *vdev, struct ivpu_mmu_context · 6 unmodified lines | |||
| 39 | void ivpu_mmu_user_context_fini(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx); | 38 | void ivpu_mmu_user_context_fini(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx); |
| 40 | void ivpu_mmu_user_context_mark_invalid(struct ivpu_device *vdev, u32 ssid); | 39 | void ivpu_mmu_user_context_mark_invalid(struct ivpu_device *vdev, u32 ssid); |
| 41 | 40 | ||
| 42 | int ivpu_mmu_context_insert_node_locked(struct ivpu_mmu_context *ctx, | 41 | int ivpu_mmu_context_insert_node(struct ivpu_mmu_context *ctx, const struct ivpu_addr_range *range, |
| 43 | const struct ivpu_addr_range *range, | 42 | u64 size, struct drm_mm_node *node); |
| 44 | u64 size, struct drm_mm_node *node); | 43 | void ivpu_mmu_context_remove_node(struct ivpu_mmu_context *ctx, struct drm_mm_node *node); |
| 45 | void ivpu_mmu_context_remove_node_locked(struct ivpu_mmu_context *ctx, | ||
| 46 | struct drm_mm_node *node); | ||
| 47 | 44 | ||
| 48 | int ivpu_mmu_context_map_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, | 45 | int ivpu_mmu_context_map_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, |
| 49 | u64 vpu_addr, struct sg_table *sgt, bool llc_coherent); | 46 | u64 vpu_addr, struct sg_table *sgt, bool llc_coherent); |
| @@ -15,6 +15,7 @@ · 14 unmodified lines | |||
| 15 | #include "ivpu_fw.h" | 15 | #include "ivpu_fw.h" |
| 16 | #include "ivpu_ipc.h" | 16 | #include "ivpu_ipc.h" |
| 17 | #include "ivpu_job.h" | 17 | #include "ivpu_job.h" |
| 18 | #include "ivpu_jsm_msg.h" | ||
| 18 | #include "ivpu_mmu.h" | 19 | #include "ivpu_mmu.h" |
| 19 | #include "ivpu_pm.h" | 20 | #include "ivpu_pm.h" |
| 20 | 21 | ||
| @@ -22,6 +23,10 @@ static bool ivpu_disable_recovery; · 1 unmodified line | |||
| 22 | module_param_named_unsafe(disable_recovery, ivpu_disable_recovery, bool, 0644); | 23 | module_param_named_unsafe(disable_recovery, ivpu_disable_recovery, bool, 0644); |
| 23 | MODULE_PARM_DESC(disable_recovery, "Disables recovery when VPU hang is detected"); | 24 | MODULE_PARM_DESC(disable_recovery, "Disables recovery when VPU hang is detected"); |
| 24 | 25 | ||
| 26 | static unsigned long ivpu_tdr_timeout_ms; | ||
| 27 | module_param_named(tdr_timeout_ms, ivpu_tdr_timeout_ms, ulong, 0644); | ||
| 28 | MODULE_PARM_DESC(tdr_timeout_ms, "Timeout for device hang detection, in milliseconds, 0 - default"); | ||
| 29 | |||
| 25 | #define PM_RESCHEDULE_LIMIT 5 | 30 | #define PM_RESCHEDULE_LIMIT 5 |
| 26 | 31 | ||
| 27 | static void ivpu_pm_prepare_cold_boot(struct ivpu_device *vdev) | 32 | static void ivpu_pm_prepare_cold_boot(struct ivpu_device *vdev) |
| @@ -69,27 +74,31 @@ retry: · 41 unmodified lines | |||
| 69 | ret = ivpu_hw_power_up(vdev); | 74 | ret = ivpu_hw_power_up(vdev); |
| 70 | if (ret) { | 75 | if (ret) { |
| 71 | ivpu_err(vdev, "Failed to power up HW: %d\n", ret); | 76 | ivpu_err(vdev, "Failed to power up HW: %d\n", ret); |
| 72 | return ret; | 77 | goto err_power_down; |
| 73 | } | 78 | } |
| 74 | 79 | ||
| 75 | ret = ivpu_mmu_enable(vdev); | 80 | ret = ivpu_mmu_enable(vdev); |
| 76 | if (ret) { | 81 | if (ret) { |
| 77 | ivpu_err(vdev, "Failed to resume MMU: %d\n", ret); | 82 | ivpu_err(vdev, "Failed to resume MMU: %d\n", ret); |
| 78 | ivpu_hw_power_down(vdev); | 83 | goto err_power_down; |
| 79 | return ret; | ||
| 80 | } | 84 | } |
| 81 | 85 | ||
| 82 | ret = ivpu_boot(vdev); | 86 | ret = ivpu_boot(vdev); |
| 83 | if (ret) { | 87 | if (ret) |
| 84 | ivpu_mmu_disable(vdev); | 88 | goto err_mmu_disable; |
| 85 | ivpu_hw_power_down(vdev); | 89 | |
| 86 | if (!ivpu_fw_is_cold_boot(vdev)) { | 90 | return 0; |
| 87 | ivpu_warn(vdev, "Failed to resume the FW: %d. Retrying cold boot..\n", ret); | 91 | |
| 88 | ivpu_pm_prepare_cold_boot(vdev); | 92 | err_mmu_disable: |
| 89 | goto retry; | 93 | ivpu_mmu_disable(vdev); |
| 90 | } else { | 94 | err_power_down: |
| 91 | ivpu_err(vdev, "Failed to resume the FW: %d\n", ret); | 95 | ivpu_hw_power_down(vdev); |
| 92 | } | 96 | |
| 97 | if (!ivpu_fw_is_cold_boot(vdev)) { | ||
| 98 | ivpu_pm_prepare_cold_boot(vdev); | ||
| 99 | goto retry; | ||
| 100 | } else { | ||
| 101 | ivpu_err(vdev, "Failed to resume the FW: %d\n", ret); | ||
| 93 | } | 102 | } |
| 94 | 103 | ||
| 95 | return ret; | 104 | return ret; |
| @@ -136,6 +145,31 @@ void ivpu_pm_schedule_recovery(struct ivpu_device *vdev) · 40 unmodified lines | |||
| 136 | } | 145 | } |
| 137 | } | 146 | } |
| 138 | 147 | ||
| 148 | static void ivpu_job_timeout_work(struct work_struct *work) | ||
| 149 | { | ||
| 150 | struct ivpu_pm_info *pm = container_of(work, struct ivpu_pm_info, job_timeout_work.work); | ||
| 151 | struct ivpu_device *vdev = pm->vdev; | ||
| 152 | unsigned long timeout_ms = ivpu_tdr_timeout_ms ? ivpu_tdr_timeout_ms : vdev->timeout.tdr; | ||
| 153 | |||
| 154 | ivpu_err(vdev, "TDR detected, timeout %lu ms", timeout_ms); | ||
| 155 | ivpu_hw_diagnose_failure(vdev); | ||
| 156 | |||
| 157 | ivpu_pm_schedule_recovery(vdev); | ||
| 158 | } | ||
| 159 | |||
| 160 | void ivpu_start_job_timeout_detection(struct ivpu_device *vdev) | ||
| 161 | { | ||
| 162 | unsigned long timeout_ms = ivpu_tdr_timeout_ms ? ivpu_tdr_timeout_ms : vdev->timeout.tdr; | ||
| 163 | |||
| 164 | /* No-op if already queued */ | ||
| 165 | queue_delayed_work(system_wq, &vdev->pm->job_timeout_work, msecs_to_jiffies(timeout_ms)); | ||
| 166 | } | ||
| 167 | |||
| 168 | void ivpu_stop_job_timeout_detection(struct ivpu_device *vdev) | ||
| 169 | { | ||
| 170 | cancel_delayed_work_sync(&vdev->pm->job_timeout_work); | ||
| 171 | } | ||
| 172 | |||
| 139 | int ivpu_pm_suspend_cb(struct device *dev) | 173 | int ivpu_pm_suspend_cb(struct device *dev) |
| 140 | { | 174 | { |
| 141 | struct drm_device *drm = dev_get_drvdata(dev); | 175 | struct drm_device *drm = dev_get_drvdata(dev); |
| @@ -153,6 +187,8 @@ int ivpu_pm_suspend_cb(struct device *dev) · 11 unmodified lines | |||
| 153 | } | 187 | } |
| 154 | } | 188 | } |
| 155 | 189 | ||
| 190 | ivpu_jsm_pwr_d0i3_enter(vdev); | ||
| 191 | |||
| 156 | ivpu_suspend(vdev); | 192 | ivpu_suspend(vdev); |
| 157 | ivpu_pm_prepare_warm_boot(vdev); | 193 | ivpu_pm_prepare_warm_boot(vdev); |
| 158 | 194 | ||
| @@ -188,6 +224,7 @@ int ivpu_pm_runtime_suspend_cb(struct device *dev) · 29 unmodified lines | |||
| 188 | { | 224 | { |
| 189 | struct drm_device *drm = dev_get_drvdata(dev); | 225 | struct drm_device *drm = dev_get_drvdata(dev); |
| 190 | struct ivpu_device *vdev = to_ivpu_device(drm); | 226 | struct ivpu_device *vdev = to_ivpu_device(drm); |
| 227 | bool hw_is_idle = true; | ||
| 191 | int ret; | 228 | int ret; |
| 192 | 229 | ||
| 193 | ivpu_dbg(vdev, PM, "Runtime suspend..\n"); | 230 | ivpu_dbg(vdev, PM, "Runtime suspend..\n"); |
| @@ -200,11 +237,16 @@ int ivpu_pm_runtime_suspend_cb(struct device *dev) · 6 unmodified lines | |||
| 200 | return -EAGAIN; | 237 | return -EAGAIN; |
| 201 | } | 238 | } |
| 202 | 239 | ||
| 240 | if (!vdev->pm->suspend_reschedule_counter) | ||
| 241 | hw_is_idle = false; | ||
| 242 | else if (ivpu_jsm_pwr_d0i3_enter(vdev)) | ||
| 243 | hw_is_idle = false; | ||
| 244 | |||
| 203 | ret = ivpu_suspend(vdev); | 245 | ret = ivpu_suspend(vdev); |
| 204 | if (ret) | 246 | if (ret) |
| 205 | ivpu_err(vdev, "Failed to set suspend VPU: %d\n", ret); | 247 | ivpu_err(vdev, "Failed to set suspend VPU: %d\n", ret); |
| 206 | 248 | ||
| 207 | if (!vdev->pm->suspend_reschedule_counter) { | 249 | if (!hw_is_idle) { |
| 208 | ivpu_warn(vdev, "VPU failed to enter idle, force suspended.\n"); | 250 | ivpu_warn(vdev, "VPU failed to enter idle, force suspended.\n"); |
| 209 | ivpu_pm_prepare_cold_boot(vdev); | 251 | ivpu_pm_prepare_cold_boot(vdev); |
| 210 | } else { | 252 | } else { |
| @@ -250,9 +292,6 @@ int ivpu_rpm_get_if_active(struct ivpu_device *vdev) · 39 unmodified lines | |||
| 250 | { | 292 | { |
| 251 | int ret; | 293 | int ret; |
| 252 | 294 | ||
| 253 | ivpu_dbg(vdev, RPM, "rpm_get_if_active count %d\n", | ||
| 254 | atomic_read(&vdev->drm.dev->power.usage_count)); | ||
| 255 | |||
| 256 | ret = pm_runtime_get_if_active(vdev->drm.dev, false); | 295 | ret = pm_runtime_get_if_active(vdev->drm.dev, false); |
| 257 | drm_WARN_ON(&vdev->drm, ret < 0); | 296 | drm_WARN_ON(&vdev->drm, ret < 0); |
| 258 | 297 | ||
| @@ -307,6 +346,7 @@ void ivpu_pm_init(struct ivpu_device *vdev) · 48 unmodified lines | |||
| 307 | 346 | ||
| 308 | atomic_set(&pm->in_reset, 0); | 347 | atomic_set(&pm->in_reset, 0); |
| 309 | INIT_WORK(&pm->recovery_work, ivpu_pm_recovery_work); | 348 | INIT_WORK(&pm->recovery_work, ivpu_pm_recovery_work); |
| 349 | INIT_DELAYED_WORK(&pm->job_timeout_work, ivpu_job_timeout_work); | ||
| 310 | 350 | ||
| 311 | if (ivpu_disable_recovery) | 351 | if (ivpu_disable_recovery) |
| 312 | delay = -1; | 352 | delay = -1; |
| @@ -321,6 +361,7 @@ void ivpu_pm_init(struct ivpu_device *vdev) · 8 unmodified lines | |||
| 321 | 361 | ||
| 322 | void ivpu_pm_cancel_recovery(struct ivpu_device *vdev) | 362 | void ivpu_pm_cancel_recovery(struct ivpu_device *vdev) |
| 323 | { | 363 | { |
| 364 | drm_WARN_ON(&vdev->drm, delayed_work_pending(&vdev->pm->job_timeout_work)); | ||
| 324 | cancel_work_sync(&vdev->pm->recovery_work); | 365 | cancel_work_sync(&vdev->pm->recovery_work); |
| 325 | } | 366 | } |
| 326 | 367 | ||
| @@ -12,6 +12,7 @@ struct ivpu_device; · 11 unmodified lines | |||
| 12 | 12 | ||
| 13 | struct ivpu_pm_info { | 13 | struct ivpu_pm_info { |
| 14 | struct ivpu_device *vdev; | 14 | struct ivpu_device *vdev; |
| 15 | struct delayed_work job_timeout_work; | ||
| 15 | struct work_struct recovery_work; | 16 | struct work_struct recovery_work; |
| 16 | atomic_t in_reset; | 17 | atomic_t in_reset; |
| 17 | atomic_t reset_counter; | 18 | atomic_t reset_counter; |
| @@ -37,5 +38,7 @@ int __must_check ivpu_rpm_get_if_active(struct ivpu_device *vdev); · 19 unmodified lines | |||
| 37 | void ivpu_rpm_put(struct ivpu_device *vdev); | 38 | void ivpu_rpm_put(struct ivpu_device *vdev); |
| 38 | 39 | ||
| 39 | void ivpu_pm_schedule_recovery(struct ivpu_device *vdev); | 40 | void ivpu_pm_schedule_recovery(struct ivpu_device *vdev); |
| 41 | void ivpu_start_job_timeout_detection(struct ivpu_device *vdev); | ||
| 42 | void ivpu_stop_job_timeout_detection(struct ivpu_device *vdev); | ||
| 40 | 43 | ||
| 41 | #endif /* __IVPU_PM_H__ */ | 44 | #endif /* __IVPU_PM_H__ */ |
| @@ -11,7 +11,10 @@ · 10 unmodified lines | |||
| 11 | * The bellow values will be used to construct the version info this way: | 11 | * The bellow values will be used to construct the version info this way: |
| 12 | * fw_bin_header->api_version[VPU_BOOT_API_VER_ID] = (VPU_BOOT_API_VER_MAJOR << 16) | | 12 | * fw_bin_header->api_version[VPU_BOOT_API_VER_ID] = (VPU_BOOT_API_VER_MAJOR << 16) | |
| 13 | * VPU_BOOT_API_VER_MINOR; | 13 | * VPU_BOOT_API_VER_MINOR; |
| 14 | * VPU_BOOT_API_VER_PATCH will be ignored. KMD and compatibility is not affected if this changes. | 14 | * VPU_BOOT_API_VER_PATCH will be ignored. KMD and compatibility is not affected if this changes |
| 15 | * This information is collected by using vpuip_2/application/vpuFirmware/make_std_fw_image.py | ||
| 16 | * If a header is missing this info we ignore the header, if a header is missing or contains | ||
| 17 | * partial info a build error will be generated. | ||
| 15 | */ | 18 | */ |
| 16 | 19 | ||
| 17 | /* | 20 | /* |
| @@ -24,12 +27,12 @@ · 6 unmodified lines | |||
| 24 | * Minor version changes when API backward compatibility is preserved. | 27 | * Minor version changes when API backward compatibility is preserved. |
| 25 | * Resets to 0 if Major version is incremented. | 28 | * Resets to 0 if Major version is incremented. |
| 26 | */ | 29 | */ |
| 27 | #define VPU_BOOT_API_VER_MINOR 12 | 30 | #define VPU_BOOT_API_VER_MINOR 20 |
| 28 | 31 | ||
| 29 | /* | 32 | /* |
| 30 | * API header changed (field names, documentation, formatting) but API itself has not been changed | 33 | * API header changed (field names, documentation, formatting) but API itself has not been changed |
| 31 | */ | 34 | */ |
| 32 | #define VPU_BOOT_API_VER_PATCH 2 | 35 | #define VPU_BOOT_API_VER_PATCH 4 |
| 33 | 36 | ||
| 34 | /* | 37 | /* |
| 35 | * Index in the API version table | 38 | * Index in the API version table |
| @@ -63,6 +66,12 @@ struct vpu_firmware_header { · 27 unmodified lines | |||
| 63 | /* Size of memory require for firmware execution */ | 66 | /* Size of memory require for firmware execution */ |
| 64 | u32 runtime_size; | 67 | u32 runtime_size; |
| 65 | u32 shave_nn_fw_size; | 68 | u32 shave_nn_fw_size; |
| 69 | /* Size of primary preemption buffer. */ | ||
| 70 | u32 preemption_buffer_1_size; | ||
| 71 | /* Size of secondary preemption buffer. */ | ||
| 72 | u32 preemption_buffer_2_size; | ||
| 73 | /* Space reserved for future preemption-related fields. */ | ||
| 74 | u32 preemption_reserved[6]; | ||
| 66 | }; | 75 | }; |
| 67 | 76 | ||
| 68 | /* | 77 | /* |
| @@ -89,6 +98,14 @@ enum VPU_BOOT_L2_CACHE_CFG_TYPE { · 20 unmodified lines | |||
| 89 | VPU_BOOT_L2_CACHE_CFG_NUM = 2 | 98 | VPU_BOOT_L2_CACHE_CFG_NUM = 2 |
| 90 | }; | 99 | }; |
| 91 | 100 | ||
| 101 | /** VPU MCA ECC signalling mode. By default, no signalling is used */ | ||
| 102 | enum VPU_BOOT_MCA_ECC_SIGNAL_TYPE { | ||
| 103 | VPU_BOOT_MCA_ECC_NONE = 0, | ||
| 104 | VPU_BOOT_MCA_ECC_CORR = 1, | ||
| 105 | VPU_BOOT_MCA_ECC_FATAL = 2, | ||
| 106 | VPU_BOOT_MCA_ECC_BOTH = 3 | ||
| 107 | }; | ||
| 108 | |||
| 92 | /** | 109 | /** |
| 93 | * Logging destinations. | 110 | * Logging destinations. |
| 94 | * | 111 | * |
| @@ -131,9 +148,11 @@ enum vpu_trace_destination { · 36 unmodified lines | |||
| 131 | #define VPU_TRACE_PROC_BIT_ACT_SHV_3 22 | 148 | #define VPU_TRACE_PROC_BIT_ACT_SHV_3 22 |
| 132 | #define VPU_TRACE_PROC_NO_OF_HW_DEVS 23 | 149 | #define VPU_TRACE_PROC_NO_OF_HW_DEVS 23 |
| 133 | 150 | ||
| 134 | /* KMB HW component IDs are sequential, so define first and last IDs. */ | 151 | /* VPU 30xx HW component IDs are sequential, so define first and last IDs. */ |
| 135 | #define VPU_TRACE_PROC_BIT_KMB_FIRST VPU_TRACE_PROC_BIT_LRT | 152 | #define VPU_TRACE_PROC_BIT_30XX_FIRST VPU_TRACE_PROC_BIT_LRT |
| 136 | #define VPU_TRACE_PROC_BIT_KMB_LAST VPU_TRACE_PROC_BIT_SHV_15 | 153 | #define VPU_TRACE_PROC_BIT_30XX_LAST VPU_TRACE_PROC_BIT_SHV_15 |
| 154 | #define VPU_TRACE_PROC_BIT_KMB_FIRST VPU_TRACE_PROC_BIT_30XX_FIRST | ||
| 155 | #define VPU_TRACE_PROC_BIT_KMB_LAST VPU_TRACE_PROC_BIT_30XX_LAST | ||
| 137 | 156 | ||
| 138 | struct vpu_boot_l2_cache_config { | 157 | struct vpu_boot_l2_cache_config { |
| 139 | u8 use; | 158 | u8 use; |
| @@ -148,6 +167,25 @@ struct vpu_warm_boot_section { · 8 unmodified lines | |||
| 148 | u32 is_clear_op; | 167 | u32 is_clear_op; |
| 149 | }; | 168 | }; |
| 150 | 169 | ||
| 170 | /* | ||
| 171 | * When HW scheduling mode is enabled, a present period is defined. | ||
| 172 | * It will be used by VPU to swap between normal and focus priorities | ||
| 173 | * to prevent starving of normal priority band (when implemented). | ||
| 174 | * Host must provide a valid value at boot time in | ||
| 175 | * `vpu_focus_present_timer_ms`. If the value provided by the host is not within the | ||
| 176 | * defined range a default value will be used. Here we define the min. and max. | ||
| 177 | * allowed values and the and default value of the present period. Units are milliseconds. | ||
| 178 | */ | ||
| 179 | #define VPU_PRESENT_CALL_PERIOD_MS_DEFAULT 50 | ||
| 180 | #define VPU_PRESENT_CALL_PERIOD_MS_MIN 16 | ||
| 181 | #define VPU_PRESENT_CALL_PERIOD_MS_MAX 10000 | ||
| 182 | |||
| 183 | /** | ||
| 184 | * Macros to enable various operation modes within the VPU. | ||
| 185 | * To be defined as part of 32 bit mask. | ||
| 186 | */ | ||
| 187 | #define VPU_OP_MODE_SURVIVABILITY 0x1 | ||
| 188 | |||
| 151 | struct vpu_boot_params { | 189 | struct vpu_boot_params { |
| 152 | u32 magic; | 190 | u32 magic; |
| 153 | u32 vpu_id; | 191 | u32 vpu_id; |
| @@ -218,6 +256,7 @@ struct vpu_boot_params { · 64 unmodified lines | |||
| 218 | * the threshold will not be logged); applies to every enabled logging | 256 | * the threshold will not be logged); applies to every enabled logging |
| 219 | * destination and loggable HW component. See 'mvLog_t' enum for acceptable | 257 | * destination and loggable HW component. See 'mvLog_t' enum for acceptable |
| 220 | * values. | 258 | * values. |
| 259 | * TODO: EISW-33556: Move log level definition (mvLog_t) to this file. | ||
| 221 | */ | 260 | */ |
| 222 | u32 default_trace_level; | 261 | u32 default_trace_level; |
| 223 | u32 boot_type; | 262 | u32 boot_type; |
| @@ -249,7 +288,36 @@ struct vpu_boot_params { · 25 unmodified lines | |||
| 249 | u32 temp_sensor_period_ms; | 288 | u32 temp_sensor_period_ms; |
| 250 | /** PLL ratio for efficient clock frequency */ | 289 | /** PLL ratio for efficient clock frequency */ |
| 251 | u32 pn_freq_pll_ratio; | 290 | u32 pn_freq_pll_ratio; |
| 252 | u32 pad4[28]; | 291 | /** DVFS Mode: Default: 0, Max Performance: 1, On Demand: 2, Power Save: 3 */ |
| 292 | u32 dvfs_mode; | ||
| 293 | /** | ||
| 294 | * Depending on DVFS Mode: | ||
| 295 | * On-demand: Default if 0. | ||
| 296 | * Bit 0-7 - uint8_t: Highest residency percent | ||
| 297 | * Bit 8-15 - uint8_t: High residency percent | ||
| 298 | * Bit 16-23 - uint8_t: Low residency percent | ||
| 299 | * Bit 24-31 - uint8_t: Lowest residency percent | ||
| 300 | * Bit 32-35 - unsigned 4b: PLL Ratio increase amount on highest residency | ||
| 301 | * Bit 36-39 - unsigned 4b: PLL Ratio increase amount on high residency | ||
| 302 | * Bit 40-43 - unsigned 4b: PLL Ratio decrease amount on low residency | ||
| 303 | * Bit 44-47 - unsigned 4b: PLL Ratio decrease amount on lowest frequency | ||
| 304 | * Bit 48-55 - uint8_t: Period (ms) for residency decisions | ||
| 305 | * Bit 56-63 - uint8_t: Averaging windows (as multiples of period. Max: 30 decimal) | ||
| 306 | * Power Save/Max Performance: Unused | ||
| 307 | */ | ||
| 308 | u64 dvfs_param; | ||
| 309 | /** | ||
| 310 | * D0i3 delayed entry | ||
| 311 | * Bit0: Disable CPU state save on D0i2 entry flow. | ||
| 312 | * 0: Every D0i2 entry saves state. Save state IPC message ignored. | ||
| 313 | * 1: IPC message required to save state on D0i3 entry flow. | ||
| 314 | */ | ||
| 315 | u32 d0i3_delayed_entry; | ||
| 316 | /* Time spent by VPU in D0i3 state */ | ||
| 317 | u64 d0i3_residency_time_us; | ||
| 318 | /* Value of VPU perf counter at the time of entering D0i3 state . */ | ||
| 319 | u64 d0i3_entry_vpu_ts; | ||
| 320 | u32 pad4[20]; | ||
| 253 | /* Warm boot information: 0x400 - 0x43F */ | 321 | /* Warm boot information: 0x400 - 0x43F */ |
| 254 | u32 warm_boot_sections_count; | 322 | u32 warm_boot_sections_count; |
| 255 | u32 warm_boot_start_address_reference; | 323 | u32 warm_boot_start_address_reference; |
| @@ -274,8 +342,12 @@ struct vpu_boot_params { · 18 unmodified lines | |||
| 274 | u32 vpu_scheduling_mode; | 342 | u32 vpu_scheduling_mode; |
| 275 | /* Present call period in milliseconds. */ | 343 | /* Present call period in milliseconds. */ |
| 276 | u32 vpu_focus_present_timer_ms; | 344 | u32 vpu_focus_present_timer_ms; |
| 277 | /* Unused/reserved: 0x478 - 0xFFF */ | 345 | /* VPU ECC Signaling */ |
| 278 | u32 pad6[738]; | 346 | u32 vpu_uses_ecc_mca_signal; |
| 347 | /* Values defined by VPU_OP_MODE* macros */ | ||
| 348 | u32 vpu_operation_mode; | ||
| 349 | /* Unused/reserved: 0x480 - 0xFFF */ | ||
| 350 | u32 pad6[736]; | ||
| 279 | }; | 351 | }; |
| 280 | 352 | ||
| 281 | /* | 353 | /* |
| @@ -22,12 +22,12 @@ · 21 unmodified lines | |||
| 22 | /* | 22 | /* |
| 23 | * Minor version changes when API backward compatibility is preserved. | 23 | * Minor version changes when API backward compatibility is preserved. |
| 24 | */ | 24 | */ |
| 25 | #define VPU_JSM_API_VER_MINOR 0 | 25 | #define VPU_JSM_API_VER_MINOR 15 |
| 26 | 26 | ||
| 27 | /* | 27 | /* |
| 28 | * API header changed (field names, documentation, formatting) but API itself has not been changed | 28 | * API header changed (field names, documentation, formatting) but API itself has not been changed |
| 29 | */ | 29 | */ |
| 30 | #define VPU_JSM_API_VER_PATCH 1 | 30 | #define VPU_JSM_API_VER_PATCH 0 |
| 31 | 31 | ||
| 32 | /* | 32 | /* |
| 33 | * Index in the API version table | 33 | * Index in the API version table |
| @@ -84,11 +84,13 @@ · 50 unmodified lines | |||
| 84 | * Job flags bit masks. | 84 | * Job flags bit masks. |
| 85 | */ | 85 | */ |
| 86 | #define VPU_JOB_FLAGS_NULL_SUBMISSION_MASK 0x00000001 | 86 | #define VPU_JOB_FLAGS_NULL_SUBMISSION_MASK 0x00000001 |
| 87 | #define VPU_JOB_FLAGS_PRIVATE_DATA_MASK 0xFF000000 | ||
| 87 | 88 | ||
| 88 | /* | 89 | /* |
| 89 | * Sizes of the reserved areas in jobs, in bytes. | 90 | * Sizes of the reserved areas in jobs, in bytes. |
| 90 | */ | 91 | */ |
| 91 | #define VPU_JOB_RESERVED_BYTES 16 | 92 | #define VPU_JOB_RESERVED_BYTES 8 |
| 93 | |||
| 92 | /* | 94 | /* |
| 93 | * Sizes of the reserved areas in job queues, in bytes. | 95 | * Sizes of the reserved areas in job queues, in bytes. |
| 94 | */ | 96 | */ |
| @@ -108,6 +110,20 @@ · 13 unmodified lines | |||
| 108 | */ | 110 | */ |
| 109 | #define VPU_DYNDBG_CMD_MAX_LEN 96 | 111 | #define VPU_DYNDBG_CMD_MAX_LEN 96 |
| 110 | 112 | ||
| 113 | /* | ||
| 114 | * For HWS command queue scheduling, we can prioritise command queues inside the | ||
| 115 | * same process with a relative in-process priority. Valid values for relative | ||
| 116 | * priority are given below - max and min. | ||
| 117 | */ | ||
| 118 | #define VPU_HWS_COMMAND_QUEUE_MAX_IN_PROCESS_PRIORITY 7 | ||
| 119 | #define VPU_HWS_COMMAND_QUEUE_MIN_IN_PROCESS_PRIORITY -7 | ||
| 120 | |||
| 121 | /* | ||
| 122 | * For HWS priority scheduling, we can have multiple realtime priority bands. | ||
| 123 | * They are numbered 0 to a MAX. | ||
| 124 | */ | ||
| 125 | #define VPU_HWS_MAX_REALTIME_PRIORITY_LEVEL 31U | ||
| 126 | |||
| 111 | /* | 127 | /* |
| 112 | * Job format. | 128 | * Job format. |
| 113 | */ | 129 | */ |
| @@ -117,8 +133,14 @@ struct vpu_job_queue_entry { · 3 unmodified lines | |||
| 117 | u32 flags; /**< Flags bit field, see VPU_JOB_FLAGS_* above */ | 133 | u32 flags; /**< Flags bit field, see VPU_JOB_FLAGS_* above */ |
| 118 | u64 root_page_table_addr; /**< Address of root page table to use for this job */ | 134 | u64 root_page_table_addr; /**< Address of root page table to use for this job */ |
| 119 | u64 root_page_table_update_counter; /**< Page tables update events counter */ | 135 | u64 root_page_table_update_counter; /**< Page tables update events counter */ |
| 120 | u64 preemption_buffer_address; /**< Address of the preemption buffer to use for this job */ | 136 | u64 primary_preempt_buf_addr; |
| 121 | u64 preemption_buffer_size; /**< Size of the preemption buffer to use for this job */ | 137 | /**< Address of the primary preemption buffer to use for this job */ |
| 138 | u32 primary_preempt_buf_size; | ||
| 139 | /**< Size of the primary preemption buffer to use for this job */ | ||
| 140 | u32 secondary_preempt_buf_size; | ||
| 141 | /**< Size of secondary preemption buffer to use for this job */ | ||
| 142 | u64 secondary_preempt_buf_addr; | ||
| 143 | /**< Address of secondary preemption buffer to use for this job */ | ||
| 122 | u8 reserved_0[VPU_JOB_RESERVED_BYTES]; | 144 | u8 reserved_0[VPU_JOB_RESERVED_BYTES]; |
| 123 | }; | 145 | }; |
| 124 | 146 | ||
| @@ -152,6 +174,46 @@ enum vpu_trace_entity_type { · 27 unmodified lines | |||
| 152 | VPU_TRACE_ENTITY_TYPE_HW_COMPONENT = 2, | 174 | VPU_TRACE_ENTITY_TYPE_HW_COMPONENT = 2, |
| 153 | }; | 175 | }; |
| 154 | 176 | ||
| 177 | /* | ||
| 178 | * HWS specific log buffer header details. | ||
| 179 | * Total size is 32 bytes. | ||
| 180 | */ | ||
| 181 | struct vpu_hws_log_buffer_header { | ||
| 182 | /* Written by VPU after adding a log entry. Initialised by host to 0. */ | ||
| 183 | u32 first_free_entry_index; | ||
| 184 | /* Incremented by VPU every time the VPU overwrites the 0th entry; | ||
| 185 | * initialised by host to 0. | ||
| 186 | */ | ||
| 187 | u32 wraparound_count; | ||
| 188 | /* | ||
| 189 | * This is the number of buffers that can be stored in the log buffer provided by the host. | ||
| 190 | * It is written by host before passing buffer to VPU. VPU should consider it read-only. | ||
| 191 | */ | ||
| 192 | u64 num_of_entries; | ||
| 193 | u64 reserved[2]; | ||
| 194 | }; | ||
| 195 | |||
| 196 | /* | ||
| 197 | * HWS specific log buffer entry details. | ||
| 198 | * Total size is 32 bytes. | ||
| 199 | */ | ||
| 200 | struct vpu_hws_log_buffer_entry { | ||
| 201 | /* VPU timestamp must be an invariant timer tick (not impacted by DVFS) */ | ||
| 202 | u64 vpu_timestamp; | ||
| 203 | /* | ||
| 204 | * Operation type: | ||
| 205 | * 0 - context state change | ||
| 206 | * 1 - queue new work | ||
| 207 | * 2 - queue unwait sync object | ||
| 208 | * 3 - queue no more work | ||
| 209 | * 4 - queue wait sync object | ||
| 210 | */ | ||
| 211 | u32 operation_type; | ||
| 212 | u32 reserved; | ||
| 213 | /* Operation data depends on operation type */ | ||
| 214 | u64 operation_data[2]; | ||
| 215 | }; | ||
| 216 | |||
| 155 | /* | 217 | /* |
| 156 | * Host <-> VPU IPC messages types. | 218 | * Host <-> VPU IPC messages types. |
| 157 | */ | 219 | */ |
| @@ -228,6 +290,23 @@ enum vpu_ipc_msg_type { · 70 unmodified lines | |||
| 228 | * deallocated or reassigned to another context. | 290 | * deallocated or reassigned to another context. |
| 229 | */ | 291 | */ |
| 230 | VPU_JSM_MSG_HWS_REGISTER_DB = 0x1117, | 292 | VPU_JSM_MSG_HWS_REGISTER_DB = 0x1117, |
| 293 | /** Control command: Log buffer setting */ | ||
| 294 | VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG = 0x1118, | ||
| 295 | /* Control command: Suspend command queue. */ | ||
| 296 | VPU_JSM_MSG_HWS_SUSPEND_CMDQ = 0x1119, | ||
| 297 | /* Control command: Resume command queue */ | ||
| 298 | VPU_JSM_MSG_HWS_RESUME_CMDQ = 0x111a, | ||
| 299 | /* Control command: Resume engine after reset */ | ||
| 300 | VPU_JSM_MSG_HWS_ENGINE_RESUME = 0x111b, | ||
| 301 | /* Control command: Enable survivability/DCT mode */ | ||
| 302 | VPU_JSM_MSG_DCT_ENABLE = 0x111c, | ||
| 303 | /* Control command: Disable survivability/DCT mode */ | ||
| 304 | VPU_JSM_MSG_DCT_DISABLE = 0x111d, | ||
| 305 | /** | ||
| 306 | * Dump VPU state. To be used for debug purposes only. | ||
| 307 | * NOTE: Please introduce new ASYNC commands before this one. * | ||
| 308 | */ | ||
| 309 | VPU_JSM_MSG_STATE_DUMP = 0x11FF, | ||
| 231 | /* IPC Host -> Device, General commands */ | 310 | /* IPC Host -> Device, General commands */ |
| 232 | VPU_JSM_MSG_GENERAL_CMD = 0x1200, | 311 | VPU_JSM_MSG_GENERAL_CMD = 0x1200, |
| 233 | VPU_JSM_MSG_BLOB_DEINIT = VPU_JSM_MSG_GENERAL_CMD, | 312 | VPU_JSM_MSG_BLOB_DEINIT = VPU_JSM_MSG_GENERAL_CMD, |
| @@ -236,6 +315,10 @@ enum vpu_ipc_msg_type { · 2 unmodified lines | |||
| 236 | * Linux command: `echo '<dyndbg_cmd>' > <debugfs>/dynamic_debug/control`. | 315 | * Linux command: `echo '<dyndbg_cmd>' > <debugfs>/dynamic_debug/control`. |
| 237 | */ | 316 | */ |
| 238 | VPU_JSM_MSG_DYNDBG_CONTROL = 0x1201, | 317 | VPU_JSM_MSG_DYNDBG_CONTROL = 0x1201, |
| 318 | /** | ||
| 319 | * Perform the save procedure for the D0i3 entry | ||
| 320 | */ | ||
| 321 | VPU_JSM_MSG_PWR_D0I3_ENTER = 0x1202, | ||
| 239 | /* IPC Device -> Host, Job completion */ | 322 | /* IPC Device -> Host, Job completion */ |
| 240 | VPU_JSM_MSG_JOB_DONE = 0x2100, | 323 | VPU_JSM_MSG_JOB_DONE = 0x2100, |
| 241 | /* IPC Device -> Host, Async command completion */ | 324 | /* IPC Device -> Host, Async command completion */ |
| @@ -304,11 +387,35 @@ enum vpu_ipc_msg_type { · 62 unmodified lines | |||
| 304 | VPU_JSM_MSG_DESTROY_CMD_QUEUE_RSP = 0x2216, | 387 | VPU_JSM_MSG_DESTROY_CMD_QUEUE_RSP = 0x2216, |
| 305 | /** Response to control command: Set context scheduling properties */ | 388 | /** Response to control command: Set context scheduling properties */ |
| 306 | VPU_JSM_MSG_SET_CONTEXT_SCHED_PROPERTIES_RSP = 0x2217, | 389 | VPU_JSM_MSG_SET_CONTEXT_SCHED_PROPERTIES_RSP = 0x2217, |
| 390 | /** Response to control command: Log buffer setting */ | ||
| 391 | VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG_RSP = 0x2218, | ||
| 392 | /* IPC Device -> Host, HWS notify index entry of log buffer written */ | ||
| 393 | VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION = 0x2219, | ||
| 394 | /* IPC Device -> Host, HWS completion of a context suspend request */ | ||
| 395 | VPU_JSM_MSG_HWS_SUSPEND_CMDQ_DONE = 0x221a, | ||
| 396 | /* Response to control command: Resume command queue */ | ||
| 397 | VPU_JSM_MSG_HWS_RESUME_CMDQ_RSP = 0x221b, | ||
| 398 | /* Response to control command: Resume engine command response */ | ||
| 399 | VPU_JSM_MSG_HWS_RESUME_ENGINE_DONE = 0x221c, | ||
| 400 | /* Response to control command: Enable survivability/DCT mode */ | ||
| 401 | VPU_JSM_MSG_DCT_ENABLE_DONE = 0x221d, | ||
| 402 | /* Response to control command: Disable survivability/DCT mode */ | ||
| 403 | VPU_JSM_MSG_DCT_DISABLE_DONE = 0x221e, | ||
| 404 | /** | ||
| 405 | * Response to state dump control command. | ||
| 406 | * NOTE: Please introduce new ASYNC responses before this one. * | ||
| 407 | */ | ||
| 408 | VPU_JSM_MSG_STATE_DUMP_RSP = 0x22FF, | ||
| 307 | /* IPC Device -> Host, General command completion */ | 409 | /* IPC Device -> Host, General command completion */ |
| 308 | VPU_JSM_MSG_GENERAL_CMD_DONE = 0x2300, | 410 | VPU_JSM_MSG_GENERAL_CMD_DONE = 0x2300, |
| 309 | VPU_JSM_MSG_BLOB_DEINIT_DONE = VPU_JSM_MSG_GENERAL_CMD_DONE, | 411 | VPU_JSM_MSG_BLOB_DEINIT_DONE = VPU_JSM_MSG_GENERAL_CMD_DONE, |
| 310 | /** Response to VPU_JSM_MSG_DYNDBG_CONTROL. */ | 412 | /** Response to VPU_JSM_MSG_DYNDBG_CONTROL. */ |
| 311 | VPU_JSM_MSG_DYNDBG_CONTROL_RSP = 0x2301, | 413 | VPU_JSM_MSG_DYNDBG_CONTROL_RSP = 0x2301, |
| 414 | /** | ||
| 415 | * Acknowledgment of completion of the save procedure initiated by | ||
| 416 | * VPU_JSM_MSG_PWR_D0I3_ENTER | ||
| 417 | */ | ||
| 418 | VPU_JSM_MSG_PWR_D0I3_ENTER_DONE = 0x2302, | ||
| 312 | }; | 419 | }; |
| 313 | 420 | ||
| 314 | enum vpu_ipc_msg_status { VPU_JSM_MSG_FREE, VPU_JSM_MSG_ALLOCATED }; | 421 | enum vpu_ipc_msg_status { VPU_JSM_MSG_FREE, VPU_JSM_MSG_ALLOCATED }; |
| @@ -593,12 +700,12 @@ struct vpu_ipc_msg_payload_hws_priority_band_setup { · 278 unmodified lines | |||
| 593 | * Default quantum in 100ns units for scheduling across processes | 700 | * Default quantum in 100ns units for scheduling across processes |
| 594 | * within a priority band | 701 | * within a priority band |
| 595 | */ | 702 | */ |
| 596 | u64 process_quantum[VPU_HWS_NUM_PRIORITY_BANDS]; | 703 | u32 process_quantum[VPU_HWS_NUM_PRIORITY_BANDS]; |
| 597 | /* | 704 | /* |
| 598 | * Default grace period in 100ns units for processes that preempt each | 705 | * Default grace period in 100ns units for processes that preempt each |
| 599 | * other within a priority band | 706 | * other within a priority band |
| 600 | */ | 707 | */ |
| 601 | u64 process_grace_period[VPU_HWS_NUM_PRIORITY_BANDS]; | 708 | u32 process_grace_period[VPU_HWS_NUM_PRIORITY_BANDS]; |
| 602 | /* | 709 | /* |
| 603 | * For normal priority band, specifies the target VPU percentage | 710 | * For normal priority band, specifies the target VPU percentage |
| 604 | * in situations when it's starved by the focus band. | 711 | * in situations when it's starved by the focus band. |
| @@ -608,32 +715,51 @@ struct vpu_ipc_msg_payload_hws_priority_band_setup { · 3 unmodified lines | |||
| 608 | u32 reserved_0; | 715 | u32 reserved_0; |
| 609 | }; | 716 | }; |
| 610 | 717 | ||
| 611 | /* HWS create command queue request */ | 718 | /* |
| 719 | * @brief HWS create command queue request. | ||
| 720 | * Host will create a command queue via this command. | ||
| 721 | * Note: Cmdq group is a handle of an object which | ||
| 722 | * may contain one or more command queues. | ||
| 723 | * @see VPU_JSM_MSG_CREATE_CMD_QUEUE | ||
| 724 | * @see VPU_JSM_MSG_CREATE_CMD_QUEUE_RSP | ||
| 725 | */ | ||
| 612 | struct vpu_ipc_msg_payload_hws_create_cmdq { | 726 | struct vpu_ipc_msg_payload_hws_create_cmdq { |
| 613 | /* Process id */ | 727 | /* Process id */ |
| 614 | u64 process_id; | 728 | u64 process_id; |
| 615 | /* Host SSID */ | 729 | /* Host SSID */ |
| 616 | u32 host_ssid; | 730 | u32 host_ssid; |
| 617 | /* Zero Padding */ | 731 | /* Engine for which queue is being created */ |
| 618 | u32 reserved; | 732 | u32 engine_idx; |
| 733 | /* | ||
| 734 | * Cmdq group may be set to 0 or equal to | ||
| 735 | * cmdq_id while each priority band contains | ||
| 736 | * only single engine instances. | ||
| 737 | */ | ||
| 738 | u64 cmdq_group; | ||
| 619 | /* Command queue id */ | 739 | /* Command queue id */ |
| 620 | u64 cmdq_id; | 740 | u64 cmdq_id; |
| 621 | /* Command queue base */ | 741 | /* Command queue base */ |
| 622 | u64 cmdq_base; | 742 | u64 cmdq_base; |
| 623 | /* Command queue size */ | 743 | /* Command queue size */ |
| 624 | u32 cmdq_size; | 744 | u32 cmdq_size; |
| 625 | /* Reserved */ | 745 | /* Zero padding */ |
| 626 | u32 reserved_0; | 746 | u32 reserved_0; |
| 627 | }; | 747 | }; |
| 628 | 748 | ||
| 629 | /* HWS create command queue response */ | 749 | /* |
| 750 | * @brief HWS create command queue response. | ||
| 751 | * @see VPU_JSM_MSG_CREATE_CMD_QUEUE | ||
| 752 | * @see VPU_JSM_MSG_CREATE_CMD_QUEUE_RSP | ||
| 753 | */ | ||
| 630 | struct vpu_ipc_msg_payload_hws_create_cmdq_rsp { | 754 | struct vpu_ipc_msg_payload_hws_create_cmdq_rsp { |
| 631 | /* Process id */ | 755 | /* Process id */ |
| 632 | u64 process_id; | 756 | u64 process_id; |
| 633 | /* Host SSID */ | 757 | /* Host SSID */ |
| 634 | u32 host_ssid; | 758 | u32 host_ssid; |
| 635 | /* Zero Padding */ | 759 | /* Engine for which queue is being created */ |
| 636 | u32 reserved; | 760 | u32 engine_idx; |
| 761 | /* Command queue group */ | ||
| 762 | u64 cmdq_group; | ||
| 637 | /* Command queue id */ | 763 | /* Command queue id */ |
| 638 | u64 cmdq_id; | 764 | u64 cmdq_id; |
| 639 | }; | 765 | }; |
| @@ -661,7 +787,7 @@ struct vpu_ipc_msg_payload_hws_set_context_sched_properties { · 21 unmodified lines | |||
| 661 | /* Inside realtime band assigns a further priority */ | 787 | /* Inside realtime band assigns a further priority */ |
| 662 | u32 realtime_priority_level; | 788 | u32 realtime_priority_level; |
| 663 | /* Priority relative to other contexts in the same process */ | 789 | /* Priority relative to other contexts in the same process */ |
| 664 | u32 in_process_priority; | 790 | s32 in_process_priority; |
| 665 | /* Zero padding / Reserved */ | 791 | /* Zero padding / Reserved */ |
| 666 | u32 reserved_1; | 792 | u32 reserved_1; |
| 667 | /* Context quantum relative to other contexts of same priority in the same process */ | 793 | /* Context quantum relative to other contexts of same priority in the same process */ |
| @@ -694,6 +820,123 @@ struct vpu_jsm_hws_register_db { · 26 unmodified lines | |||
| 694 | u64 cmdq_size; | 820 | u64 cmdq_size; |
| 695 | }; | 821 | }; |
| 696 | 822 | ||
| 823 | /* | ||
| 824 | * @brief Structure to set another buffer to be used for scheduling-related logging. | ||
| 825 | * The size of the logging buffer and the number of entries is defined as part of the | ||
| 826 | * buffer itself as described next. | ||
| 827 | * The log buffer received from the host is made up of; | ||
| 828 | * - header: 32 bytes in size, as shown in 'struct vpu_hws_log_buffer_header'. | ||
| 829 | * The header contains the number of log entries in the buffer. | ||
| 830 | * - log entry: 0 to n-1, each log entry is 32 bytes in size, as shown in | ||
| 831 | * 'struct vpu_hws_log_buffer_entry'. | ||
| 832 | * The entry contains the VPU timestamp, operation type and data. | ||
| 833 | * The host should provide the notify index value of log buffer to VPU. This is a | ||
| 834 | * value defined within the log buffer and when written to will generate the | ||
| 835 | * scheduling log notification. | ||
| 836 | * The host should set engine_idx and vpu_log_buffer_va to 0 to disable logging | ||
| 837 | * for a particular engine. | ||
| 838 | * VPU will handle one log buffer for each of supported engines. | ||
| 839 | * VPU should allow the logging to consume one host_ssid. | ||
| 840 | * @see VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG | ||
| 841 | * @see VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG_RSP | ||
| 842 | * @see VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION | ||
| 843 | */ | ||
| 844 | struct vpu_ipc_msg_payload_hws_set_scheduling_log { | ||
| 845 | /* Engine ordinal */ | ||
| 846 | u32 engine_idx; | ||
| 847 | /* Host SSID */ | ||
| 848 | u32 host_ssid; | ||
| 849 | /* | ||
| 850 | * VPU log buffer virtual address. | ||
| 851 | * Set to 0 to disable logging for this engine. | ||
| 852 | */ | ||
| 853 | u64 vpu_log_buffer_va; | ||
| 854 | /* | ||
| 855 | * Notify index of log buffer. VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION | ||
| 856 | * is generated when an event log is written to this index. | ||
| 857 | */ | ||
| 858 | u64 notify_index; | ||
| 859 | }; | ||
| 860 | |||
| 861 | /* | ||
| 862 | * @brief The scheduling log notification is generated by VPU when it writes | ||
| 863 | * an event into the log buffer at the notify_index. VPU notifies host with | ||
| 864 | * VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION. This is an asynchronous | ||
| 865 | * message from VPU to host. | ||
| 866 | * @see VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION | ||
| 867 | * @see VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG | ||
| 868 | */ | ||
| 869 | struct vpu_ipc_msg_payload_hws_scheduling_log_notification { | ||
| 870 | /* Engine ordinal */ | ||
| 871 | u32 engine_idx; | ||
| 872 | /* Zero Padding */ | ||
| 873 | u32 reserved_0; | ||
| 874 | }; | ||
| 875 | |||
| 876 | /* | ||
| 877 | * @brief HWS suspend command queue request and done structure. | ||
| 878 | * Host will request the suspend of contexts and VPU will; | ||
| 879 | * - Suspend all work on this context | ||
| 880 | * - Preempt any running work | ||
| 881 | * - Asynchronously perform the above and return success immediately once | ||
| 882 | * all items above are started successfully | ||
| 883 | * - Notify the host of completion of these operations via | ||
| 884 | * VPU_JSM_MSG_HWS_SUSPEND_CMDQ_DONE | ||
| 885 | * - Reject any other context operations on a context with an in-flight | ||
| 886 | * suspend request running | ||
| 887 | * Same structure used when VPU notifies host of completion of a context suspend | ||
| 888 | * request. The ids and suspend fence value reported in this command will match | ||
| 889 | * the one in the request from the host to suspend the context. Once suspend is | ||
| 890 | * complete, VPU will not access any data relating to this command queue until | ||
| 891 | * it is resumed. | ||
| 892 | * @see VPU_JSM_MSG_HWS_SUSPEND_CMDQ | ||
| 893 | * @see VPU_JSM_MSG_HWS_SUSPEND_CMDQ_DONE | ||
| 894 | */ | ||
| 895 | struct vpu_ipc_msg_payload_hws_suspend_cmdq { | ||
| 896 | /* Host SSID */ | ||
| 897 | u32 host_ssid; | ||
| 898 | /* Zero Padding */ | ||
| 899 | u32 reserved_0; | ||
| 900 | /* Command queue id */ | ||
| 901 | u64 cmdq_id; | ||
| 902 | /* | ||
| 903 | * Suspend fence value - reported by the VPU suspend context | ||
| 904 | * completed once suspend is complete. | ||
| 905 | */ | ||
| 906 | u64 suspend_fence_value; | ||
| 907 | }; | ||
| 908 | |||
| 909 | /* | ||
| 910 | * @brief HWS Resume command queue request / response structure. | ||
| 911 | * Host will request the resume of a context; | ||
| 912 | * - VPU will resume all work on this context | ||
| 913 | * - Scheduler will allow this context to be scheduled | ||
| 914 | * @see VPU_JSM_MSG_HWS_RESUME_CMDQ | ||
| 915 | * @see VPU_JSM_MSG_HWS_RESUME_CMDQ_RSP | ||
| 916 | */ | ||
| 917 | struct vpu_ipc_msg_payload_hws_resume_cmdq { | ||
| 918 | /* Host SSID */ | ||
| 919 | u32 host_ssid; | ||
| 920 | /* Zero Padding */ | ||
| 921 | u32 reserved_0; | ||
| 922 | /* Command queue id */ | ||
| 923 | u64 cmdq_id; | ||
| 924 | }; | ||
| 925 | |||
| 926 | /* | ||
| 927 | * @brief HWS Resume engine request / response structure. | ||
| 928 | * After a HWS engine reset, all scheduling is stopped on VPU until a engine resume. | ||
| 929 | * Host shall send this command to resume scheduling of any valid queue. | ||
| 930 | * @see VPU_JSM_MSG_HWS_RESUME_ENGINE | ||
| 931 | * @see VPU_JSM_MSG_HWS_RESUME_ENGINE_DONE | ||
| 932 | */ | ||
| 933 | struct vpu_ipc_msg_payload_hws_resume_engine { | ||
| 934 | /* Engine to be resumed */ | ||
| 935 | u32 engine_idx; | ||
| 936 | /* Reserved */ | ||
| 937 | u32 reserved_0; | ||
| 938 | }; | ||
| 939 | |||
| 697 | /** | 940 | /** |
| 698 | * Payload for VPU_JSM_MSG_TRACE_SET_CONFIG[_RSP] and | 941 | * Payload for VPU_JSM_MSG_TRACE_SET_CONFIG[_RSP] and |
| 699 | * VPU_JSM_MSG_TRACE_GET_CONFIG_RSP messages. | 942 | * VPU_JSM_MSG_TRACE_GET_CONFIG_RSP messages. |
| @@ -938,6 +1181,35 @@ struct vpu_ipc_msg_payload_dyndbg_control { · 238 unmodified lines | |||
| 938 | char dyndbg_cmd[VPU_DYNDBG_CMD_MAX_LEN]; | 1181 | char dyndbg_cmd[VPU_DYNDBG_CMD_MAX_LEN]; |
| 939 | }; | 1182 | }; |
| 940 | 1183 | ||
| 1184 | /** | ||
| 1185 | * Payload for VPU_JSM_MSG_PWR_D0I3_ENTER | ||
| 1186 | * | ||
| 1187 | * This is a bi-directional payload. | ||
| 1188 | */ | ||
| 1189 | struct vpu_ipc_msg_payload_pwr_d0i3_enter { | ||
| 1190 | /** | ||
| 1191 | * 0: VPU_JSM_MSG_PWR_D0I3_ENTER_DONE is not sent to the host driver | ||
| 1192 | * The driver will poll for D0i2 Idle state transitions. | ||
| 1193 | * 1: VPU_JSM_MSG_PWR_D0I3_ENTER_DONE is sent after VPU state save is complete | ||
| 1194 | */ | ||
| 1195 | u32 send_response; | ||
| 1196 | u32 reserved_0; | ||
| 1197 | }; | ||
| 1198 | |||
| 1199 | /** | ||
| 1200 | * Payload for VPU_JSM_MSG_DCT_ENABLE message. | ||
| 1201 | * | ||
| 1202 | * Default values for DCT active/inactive times are 5.3ms and 30ms respectively, | ||
| 1203 | * corresponding to a 85% duty cycle. This payload allows the host to tune these | ||
| 1204 | * values according to application requirements. | ||
| 1205 | */ | ||
| 1206 | struct vpu_ipc_msg_payload_pwr_dct_control { | ||
| 1207 | /** Duty cycle active time in microseconds */ | ||
| 1208 | u32 dct_active_us; | ||
| 1209 | /** Duty cycle inactive time in microseconds */ | ||
| 1210 | u32 dct_inactive_us; | ||
| 1211 | }; | ||
| 1212 | |||
| 941 | /* | 1213 | /* |
| 942 | * Payloads union, used to define complete message format. | 1214 | * Payloads union, used to define complete message format. |
| 943 | */ | 1215 | */ |
| @@ -974,6 +1246,13 @@ union vpu_ipc_msg_payload { · 30 unmodified lines | |||
| 974 | struct vpu_ipc_msg_payload_hws_destroy_cmdq hws_destroy_cmdq; | 1246 | struct vpu_ipc_msg_payload_hws_destroy_cmdq hws_destroy_cmdq; |
| 975 | struct vpu_ipc_msg_payload_hws_set_context_sched_properties | 1247 | struct vpu_ipc_msg_payload_hws_set_context_sched_properties |
| 976 | hws_set_context_sched_properties; | 1248 | hws_set_context_sched_properties; |
| 1249 | struct vpu_ipc_msg_payload_hws_set_scheduling_log hws_set_scheduling_log; | ||
| 1250 | struct vpu_ipc_msg_payload_hws_scheduling_log_notification hws_scheduling_log_notification; | ||
| 1251 | struct vpu_ipc_msg_payload_hws_suspend_cmdq hws_suspend_cmdq; | ||
| 1252 | struct vpu_ipc_msg_payload_hws_resume_cmdq hws_resume_cmdq; | ||
| 1253 | struct vpu_ipc_msg_payload_hws_resume_engine hws_resume_engine; | ||
| 1254 | struct vpu_ipc_msg_payload_pwr_d0i3_enter pwr_d0i3_enter; | ||
| 1255 | struct vpu_ipc_msg_payload_pwr_dct_control pwr_dct_control; | ||
| 977 | }; | 1256 | }; |
| 978 | 1257 | ||
| 979 | /* | 1258 | /* |
| @@ -9,4 +9,5 @@ qaic-y := \ · 8 unmodified lines | |||
| 9 | mhi_controller.o \ | 9 | mhi_controller.o \ |
| 10 | qaic_control.o \ | 10 | qaic_control.o \ |
| 11 | qaic_data.o \ | 11 | qaic_data.o \ |
| 12 | qaic_drv.o | 12 | qaic_drv.o \ |
| 13 | qaic_timesync.o | ||
| @@ -348,7 +348,7 @@ static struct mhi_channel_config aic100_channels[] = { · 347 unmodified lines | |||
| 348 | .local_elements = 0, | 348 | .local_elements = 0, |
| 349 | .event_ring = 0, | 349 | .event_ring = 0, |
| 350 | .dir = DMA_TO_DEVICE, | 350 | .dir = DMA_TO_DEVICE, |
| 351 | .ee_mask = MHI_CH_EE_SBL | MHI_CH_EE_AMSS, | 351 | .ee_mask = MHI_CH_EE_SBL, |
| 352 | .pollcfg = 0, | 352 | .pollcfg = 0, |
| 353 | .doorbell = MHI_DB_BRST_DISABLE, | 353 | .doorbell = MHI_DB_BRST_DISABLE, |
| 354 | .lpm_notify = false, | 354 | .lpm_notify = false, |
| @@ -364,7 +364,39 @@ static struct mhi_channel_config aic100_channels[] = { · 9 unmodified lines | |||
| 364 | .local_elements = 0, | 364 | .local_elements = 0, |
| 365 | .event_ring = 0, | 365 | .event_ring = 0, |
| 366 | .dir = DMA_FROM_DEVICE, | 366 | .dir = DMA_FROM_DEVICE, |
| 367 | .ee_mask = MHI_CH_EE_SBL | MHI_CH_EE_AMSS, | 367 | .ee_mask = MHI_CH_EE_SBL, |
| 368 | .pollcfg = 0, | ||
| 369 | .doorbell = MHI_DB_BRST_DISABLE, | ||
| 370 | .lpm_notify = false, | ||
| 371 | .offload_channel = false, | ||
| 372 | .doorbell_mode_switch = false, | ||
| 373 | .auto_queue = false, | ||
| 374 | .wake_capable = false, | ||
| 375 | }, | ||
| 376 | { | ||
| 377 | .name = "QAIC_TIMESYNC_PERIODIC", | ||
| 378 | .num = 22, | ||
| 379 | .num_elements = 32, | ||
| 380 | .local_elements = 0, | ||
| 381 | .event_ring = 0, | ||
| 382 | .dir = DMA_TO_DEVICE, | ||
| 383 | .ee_mask = MHI_CH_EE_AMSS, | ||
| 384 | .pollcfg = 0, | ||
| 385 | .doorbell = MHI_DB_BRST_DISABLE, | ||
| 386 | .lpm_notify = false, | ||
| 387 | .offload_channel = false, | ||
| 388 | .doorbell_mode_switch = false, | ||
| 389 | .auto_queue = false, | ||
| 390 | .wake_capable = false, | ||
| 391 | }, | ||
| 392 | { | ||
| 393 | .num = 23, | ||
| 394 | .name = "QAIC_TIMESYNC_PERIODIC", | ||
| 395 | .num_elements = 32, | ||
| 396 | .local_elements = 0, | ||
| 397 | .event_ring = 0, | ||
| 398 | .dir = DMA_FROM_DEVICE, | ||
| 399 | .ee_mask = MHI_CH_EE_AMSS, | ||
| 368 | .pollcfg = 0, | 400 | .pollcfg = 0, |
| 369 | .doorbell = MHI_DB_BRST_DISABLE, | 401 | .doorbell = MHI_DB_BRST_DISABLE, |
| 370 | .lpm_notify = false, | 402 | .lpm_notify = false, |
| @@ -437,7 +469,7 @@ static void mhi_status_cb(struct mhi_controller *mhi_cntrl, enum mhi_callback re · 66 unmodified lines | |||
| 437 | pci_err(qdev->pdev, "Fatal error received from device. Attempting to recover\n"); | 469 | pci_err(qdev->pdev, "Fatal error received from device. Attempting to recover\n"); |
| 438 | /* this event occurs in non-atomic context */ | 470 | /* this event occurs in non-atomic context */ |
| 439 | if (reason == MHI_CB_SYS_ERROR) | 471 | if (reason == MHI_CB_SYS_ERROR) |
| 440 | qaic_dev_reset_clean_local_state(qdev, true); | 472 | qaic_dev_reset_clean_local_state(qdev); |
| 441 | } | 473 | } |
| 442 | 474 | ||
| 443 | static int mhi_reset_and_async_power_up(struct mhi_controller *mhi_cntrl) | 475 | static int mhi_reset_and_async_power_up(struct mhi_controller *mhi_cntrl) |
| @@ -468,7 +500,7 @@ static int mhi_reset_and_async_power_up(struct mhi_controller *mhi_cntrl) · 24 unmodified lines | |||
| 468 | } | 500 | } |
| 469 | 501 | ||
| 470 | struct mhi_controller *qaic_mhi_register_controller(struct pci_dev *pci_dev, void __iomem *mhi_bar, | 502 | struct mhi_controller *qaic_mhi_register_controller(struct pci_dev *pci_dev, void __iomem *mhi_bar, |
| 471 | int mhi_irq) | 503 | int mhi_irq, bool shared_msi) |
| 472 | { | 504 | { |
| 473 | struct mhi_controller *mhi_cntrl; | 505 | struct mhi_controller *mhi_cntrl; |
| 474 | int ret; | 506 | int ret; |
| @@ -500,6 +532,10 @@ struct mhi_controller *qaic_mhi_register_controller(struct pci_dev *pci_dev, voi · 25 unmodified lines | |||
| 500 | return ERR_PTR(-ENOMEM); | 532 | return ERR_PTR(-ENOMEM); |
| 501 | 533 | ||
| 502 | mhi_cntrl->irq[0] = mhi_irq; | 534 | mhi_cntrl->irq[0] = mhi_irq; |
| 535 | |||
| 536 | if (shared_msi) /* MSI shared with data path, no IRQF_NO_SUSPEND */ | ||
| 537 | mhi_cntrl->irq_flags = IRQF_SHARED; | ||
| 538 | |||
| 503 | mhi_cntrl->fw_image = "qcom/aic100/sbl.bin"; | 539 | mhi_cntrl->fw_image = "qcom/aic100/sbl.bin"; |
| 504 | 540 | ||
| 505 | /* use latest configured timeout */ | 541 | /* use latest configured timeout */ |
| @@ -8,7 +8,7 @@ · 7 unmodified lines | |||
| 8 | #define MHICONTROLLERQAIC_H_ | 8 | #define MHICONTROLLERQAIC_H_ |
| 9 | 9 | ||
| 10 | struct mhi_controller *qaic_mhi_register_controller(struct pci_dev *pci_dev, void __iomem *mhi_bar, | 10 | struct mhi_controller *qaic_mhi_register_controller(struct pci_dev *pci_dev, void __iomem *mhi_bar, |
| 11 | int mhi_irq); | 11 | int mhi_irq, bool shared_msi); |
| 12 | void qaic_mhi_free_controller(struct mhi_controller *mhi_cntrl, bool link_up); | 12 | void qaic_mhi_free_controller(struct mhi_controller *mhi_cntrl, bool link_up); |
| 13 | void qaic_mhi_start_reset(struct mhi_controller *mhi_cntrl); | 13 | void qaic_mhi_start_reset(struct mhi_controller *mhi_cntrl); |
| 14 | void qaic_mhi_reset_done(struct mhi_controller *mhi_cntrl); | 14 | void qaic_mhi_reset_done(struct mhi_controller *mhi_cntrl); |
| @@ -31,6 +31,15 @@ · 30 unmodified lines | |||
| 31 | #define to_drm(qddev) (&(qddev)->drm) | 31 | #define to_drm(qddev) (&(qddev)->drm) |
| 32 | #define to_accel_kdev(qddev) (to_drm(qddev)->accel->kdev) /* Return Linux device of accel node */ | 32 | #define to_accel_kdev(qddev) (to_drm(qddev)->accel->kdev) /* Return Linux device of accel node */ |
| 33 | 33 | ||
| 34 | enum __packed dev_states { | ||
| 35 | /* Device is offline or will be very soon */ | ||
| 36 | QAIC_OFFLINE, | ||
| 37 | /* Device is booting, not clear if it's in a usable state */ | ||
| 38 | QAIC_BOOT, | ||
| 39 | /* Device is fully operational */ | ||
| 40 | QAIC_ONLINE, | ||
| 41 | }; | ||
| 42 | |||
| 34 | extern bool datapath_polling; | 43 | extern bool datapath_polling; |
| 35 | 44 | ||
| 36 | struct qaic_user { | 45 | struct qaic_user { |
| @@ -121,8 +130,10 @@ struct qaic_device { · 84 unmodified lines | |||
| 121 | struct workqueue_struct *cntl_wq; | 130 | struct workqueue_struct *cntl_wq; |
| 122 | /* Synchronizes all the users of device during cleanup */ | 131 | /* Synchronizes all the users of device during cleanup */ |
| 123 | struct srcu_struct dev_lock; | 132 | struct srcu_struct dev_lock; |
| 124 | /* true: Device under reset; false: Device not under reset */ | 133 | /* Track the state of the device during resets */ |
| 125 | bool in_reset; | 134 | enum dev_states dev_state; |
| 135 | /* true: single MSI is used to operate device */ | ||
| 136 | bool single_msi; | ||
| 126 | /* | 137 | /* |
| 127 | * true: A tx MHI transaction has failed and a rx buffer is still queued | 138 | * true: A tx MHI transaction has failed and a rx buffer is still queued |
| 128 | * in control device. Such a buffer is considered lost rx buffer | 139 | * in control device. Such a buffer is considered lost rx buffer |
| @@ -137,6 +148,10 @@ struct qaic_device { · 8 unmodified lines | |||
| 137 | u32 (*gen_crc)(void *msg); | 148 | u32 (*gen_crc)(void *msg); |
| 138 | /* Validate the CRC of a control message */ | 149 | /* Validate the CRC of a control message */ |
| 139 | bool (*valid_crc)(void *msg); | 150 | bool (*valid_crc)(void *msg); |
| 151 | /* MHI "QAIC_TIMESYNC" channel device */ | ||
| 152 | struct mhi_device *qts_ch; | ||
| 153 | /* Work queue for tasks related to MHI "QAIC_TIMESYNC" channel */ | ||
| 154 | struct workqueue_struct *qts_wq; | ||
| 140 | }; | 155 | }; |
| 141 | 156 | ||
| 142 | struct qaic_drm_device { | 157 | struct qaic_drm_device { |
| @@ -268,7 +283,7 @@ void wakeup_dbc(struct qaic_device *qdev, u32 dbc_id); · 125 unmodified lines | |||
| 268 | void release_dbc(struct qaic_device *qdev, u32 dbc_id); | 283 | void release_dbc(struct qaic_device *qdev, u32 dbc_id); |
| 269 | 284 | ||
| 270 | void wake_all_cntl(struct qaic_device *qdev); | 285 | void wake_all_cntl(struct qaic_device *qdev); |
| 271 | void qaic_dev_reset_clean_local_state(struct qaic_device *qdev, bool exit_reset); | 286 | void qaic_dev_reset_clean_local_state(struct qaic_device *qdev); |
| 272 | 287 | ||
| 273 | struct drm_gem_object *qaic_gem_prime_import(struct drm_device *dev, struct dma_buf *dma_buf); | 288 | struct drm_gem_object *qaic_gem_prime_import(struct drm_device *dev, struct dma_buf *dma_buf); |
| 274 | 289 | ||
| @@ -1022,7 +1022,8 @@ static void *msg_xfer(struct qaic_device *qdev, struct wrapper_list *wrappers, u · 1021 unmodified lines | |||
| 1022 | int xfer_count = 0; | 1022 | int xfer_count = 0; |
| 1023 | int retry_count; | 1023 | int retry_count; |
| 1024 | 1024 | ||
| 1025 | if (qdev->in_reset) { | 1025 | /* Allow QAIC_BOOT state since we need to check control protocol version */ |
| 1026 | if (qdev->dev_state == QAIC_OFFLINE) { | ||
| 1026 | mutex_unlock(&qdev->cntl_mutex); | 1027 | mutex_unlock(&qdev->cntl_mutex); |
| 1027 | return ERR_PTR(-ENODEV); | 1028 | return ERR_PTR(-ENODEV); |
| 1028 | } | 1029 | } |
| @@ -1138,7 +1139,7 @@ static int abort_dma_cont(struct qaic_device *qdev, struct wrapper_list *wrapper · 109 unmodified lines | |||
| 1138 | if (!list_is_first(&wrapper->list, &wrappers->list)) | 1139 | if (!list_is_first(&wrapper->list, &wrappers->list)) |
| 1139 | kref_put(&wrapper->ref_count, free_wrapper); | 1140 | kref_put(&wrapper->ref_count, free_wrapper); |
| 1140 | 1141 | ||
| 1141 | wrapper = add_wrapper(wrappers, offsetof(struct wrapper_msg, trans) + sizeof(*out_trans)); | 1142 | wrapper = add_wrapper(wrappers, sizeof(*wrapper)); |
| 1142 | 1143 | ||
| 1143 | if (!wrapper) | 1144 | if (!wrapper) |
| 1144 | return -ENOMEM; | 1145 | return -ENOMEM; |
| @@ -1306,7 +1307,7 @@ int qaic_manage_ioctl(struct drm_device *dev, void *data, struct drm_file *file_ · 161 unmodified lines | |||
| 1306 | qdev = usr->qddev->qdev; | 1307 | qdev = usr->qddev->qdev; |
| 1307 | 1308 | ||
| 1308 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); | 1309 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 1309 | if (qdev->in_reset) { | 1310 | if (qdev->dev_state != QAIC_ONLINE) { |
| 1310 | srcu_read_unlock(&qdev->dev_lock, qdev_rcu_id); | 1311 | srcu_read_unlock(&qdev->dev_lock, qdev_rcu_id); |
| 1311 | srcu_read_unlock(&usr->qddev_lock, usr_rcu_id); | 1312 | srcu_read_unlock(&usr->qddev_lock, usr_rcu_id); |
| 1312 | return -ENODEV; | 1313 | return -ENODEV; |
| @@ -51,6 +51,7 @@ · 50 unmodified lines | |||
| 51 | }) | 51 | }) |
| 52 | #define NUM_EVENTS 128 | 52 | #define NUM_EVENTS 128 |
| 53 | #define NUM_DELAYS 10 | 53 | #define NUM_DELAYS 10 |
| 54 | #define fifo_at(base, offset) ((base) + (offset) * get_dbc_req_elem_size()) | ||
| 54 | 55 | ||
| 55 | static unsigned int wait_exec_default_timeout_ms = 5000; /* 5 sec default */ | 56 | static unsigned int wait_exec_default_timeout_ms = 5000; /* 5 sec default */ |
| 56 | module_param(wait_exec_default_timeout_ms, uint, 0600); | 57 | module_param(wait_exec_default_timeout_ms, uint, 0600); |
| @@ -451,7 +452,7 @@ static int create_sgt(struct qaic_device *qdev, struct sg_table **sgt_out, u64 s · 394 unmodified lines | |||
| 451 | * later | 452 | * later |
| 452 | */ | 453 | */ |
| 453 | buf_extra = (PAGE_SIZE - size % PAGE_SIZE) % PAGE_SIZE; | 454 | buf_extra = (PAGE_SIZE - size % PAGE_SIZE) % PAGE_SIZE; |
| 454 | max_order = min(MAX_ORDER - 1, get_order(size)); | 455 | max_order = min(MAX_ORDER, get_order(size)); |
| 455 | } else { | 456 | } else { |
| 456 | /* allocate a single page for book keeping */ | 457 | /* allocate a single page for book keeping */ |
| 457 | nr_pages = 1; | 458 | nr_pages = 1; |
| @@ -689,7 +690,7 @@ int qaic_create_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *fi · 231 unmodified lines | |||
| 689 | 690 | ||
| 690 | qdev = usr->qddev->qdev; | 691 | qdev = usr->qddev->qdev; |
| 691 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); | 692 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 692 | if (qdev->in_reset) { | 693 | if (qdev->dev_state != QAIC_ONLINE) { |
| 693 | ret = -ENODEV; | 694 | ret = -ENODEV; |
| 694 | goto unlock_dev_srcu; | 695 | goto unlock_dev_srcu; |
| 695 | } | 696 | } |
| @@ -748,7 +749,7 @@ int qaic_mmap_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *file · 52 unmodified lines | |||
| 748 | 749 | ||
| 749 | qdev = usr->qddev->qdev; | 750 | qdev = usr->qddev->qdev; |
| 750 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); | 751 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 751 | if (qdev->in_reset) { | 752 | if (qdev->dev_state != QAIC_ONLINE) { |
| 752 | ret = -ENODEV; | 753 | ret = -ENODEV; |
| 753 | goto unlock_dev_srcu; | 754 | goto unlock_dev_srcu; |
| 754 | } | 755 | } |
| @@ -969,7 +970,7 @@ int qaic_attach_slice_bo_ioctl(struct drm_device *dev, void *data, struct drm_fi · 214 unmodified lines | |||
| 969 | 970 | ||
| 970 | qdev = usr->qddev->qdev; | 971 | qdev = usr->qddev->qdev; |
| 971 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); | 972 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 972 | if (qdev->in_reset) { | 973 | if (qdev->dev_state != QAIC_ONLINE) { |
| 973 | ret = -ENODEV; | 974 | ret = -ENODEV; |
| 974 | goto unlock_dev_srcu; | 975 | goto unlock_dev_srcu; |
| 975 | } | 976 | } |
| @@ -1058,6 +1059,16 @@ unlock_usr_srcu: · 82 unmodified lines | |||
| 1058 | return ret; | 1059 | return ret; |
| 1059 | } | 1060 | } |
| 1060 | 1061 | ||
| 1062 | static inline u32 fifo_space_avail(u32 head, u32 tail, u32 q_size) | ||
| 1063 | { | ||
| 1064 | u32 avail = head - tail - 1; | ||
| 1065 | |||
| 1066 | if (head <= tail) | ||
| 1067 | avail += q_size; | ||
| 1068 | |||
| 1069 | return avail; | ||
| 1070 | } | ||
| 1071 | |||
| 1061 | static inline int copy_exec_reqs(struct qaic_device *qdev, struct bo_slice *slice, u32 dbc_id, | 1072 | static inline int copy_exec_reqs(struct qaic_device *qdev, struct bo_slice *slice, u32 dbc_id, |
| 1062 | u32 head, u32 *ptail) | 1073 | u32 head, u32 *ptail) |
| 1063 | { | 1074 | { |
| @@ -1066,27 +1077,20 @@ static inline int copy_exec_reqs(struct qaic_device *qdev, struct bo_slice *slic · 2 unmodified lines | |||
| 1066 | u32 tail = *ptail; | 1077 | u32 tail = *ptail; |
| 1067 | u32 avail; | 1078 | u32 avail; |
| 1068 | 1079 | ||
| 1069 | avail = head - tail; | 1080 | avail = fifo_space_avail(head, tail, dbc->nelem); |
| 1070 | if (head <= tail) | ||
| 1071 | avail += dbc->nelem; | ||
| 1072 | |||
| 1073 | --avail; | ||
| 1074 | |||
| 1075 | if (avail < slice->nents) | 1081 | if (avail < slice->nents) |
| 1076 | return -EAGAIN; | 1082 | return -EAGAIN; |
| 1077 | 1083 | ||
| 1078 | if (tail + slice->nents > dbc->nelem) { | 1084 | if (tail + slice->nents > dbc->nelem) { |
| 1079 | avail = dbc->nelem - tail; | 1085 | avail = dbc->nelem - tail; |
| 1080 | avail = min_t(u32, avail, slice->nents); | 1086 | avail = min_t(u32, avail, slice->nents); |
| 1081 | memcpy(dbc->req_q_base + tail * get_dbc_req_elem_size(), reqs, | 1087 | memcpy(fifo_at(dbc->req_q_base, tail), reqs, sizeof(*reqs) * avail); |
| 1082 | sizeof(*reqs) * avail); | ||
| 1083 | reqs += avail; | 1088 | reqs += avail; |
| 1084 | avail = slice->nents - avail; | 1089 | avail = slice->nents - avail; |
| 1085 | if (avail) | 1090 | if (avail) |
| 1086 | memcpy(dbc->req_q_base, reqs, sizeof(*reqs) * avail); | 1091 | memcpy(dbc->req_q_base, reqs, sizeof(*reqs) * avail); |
| 1087 | } else { | 1092 | } else { |
| 1088 | memcpy(dbc->req_q_base + tail * get_dbc_req_elem_size(), reqs, | 1093 | memcpy(fifo_at(dbc->req_q_base, tail), reqs, sizeof(*reqs) * slice->nents); |
| 1089 | sizeof(*reqs) * slice->nents); | ||
| 1090 | } | 1094 | } |
| 1091 | 1095 | ||
| 1092 | *ptail = (tail + slice->nents) % dbc->nelem; | 1096 | *ptail = (tail + slice->nents) % dbc->nelem; |
| @@ -1094,46 +1098,31 @@ static inline int copy_exec_reqs(struct qaic_device *qdev, struct bo_slice *slic · 1 unmodified line | |||
| 1094 | return 0; | 1098 | return 0; |
| 1095 | } | 1099 | } |
| 1096 | 1100 | ||
| 1097 | /* | ||
| 1098 | * Based on the value of resize we may only need to transmit first_n | ||
| 1099 | * entries and the last entry, with last_bytes to send from the last entry. | ||
| 1100 | * Note that first_n could be 0. | ||
| 1101 | */ | ||
| 1102 | static inline int copy_partial_exec_reqs(struct qaic_device *qdev, struct bo_slice *slice, | 1101 | static inline int copy_partial_exec_reqs(struct qaic_device *qdev, struct bo_slice *slice, |
| 1103 | u64 resize, u32 dbc_id, u32 head, u32 *ptail) | 1102 | u64 resize, struct dma_bridge_chan *dbc, u32 head, |
| 1103 | u32 *ptail) | ||
| 1104 | { | 1104 | { |
| 1105 | struct dma_bridge_chan *dbc = &qdev->dbc[dbc_id]; | ||
| 1106 | struct dbc_req *reqs = slice->reqs; | 1105 | struct dbc_req *reqs = slice->reqs; |
| 1107 | struct dbc_req *last_req; | 1106 | struct dbc_req *last_req; |
| 1108 | u32 tail = *ptail; | 1107 | u32 tail = *ptail; |
| 1109 | u64 total_bytes; | ||
| 1110 | u64 last_bytes; | 1108 | u64 last_bytes; |
| 1111 | u32 first_n; | 1109 | u32 first_n; |
| 1112 | u32 avail; | 1110 | u32 avail; |
| 1113 | int ret; | ||
| 1114 | int i; | ||
| 1115 | |||
| 1116 | avail = head - tail; | ||
| 1117 | if (head <= tail) | ||
| 1118 | avail += dbc->nelem; | ||
| 1119 | 1111 | ||
| 1120 | --avail; | 1112 | avail = fifo_space_avail(head, tail, dbc->nelem); |
| 1121 | 1113 | ||
| 1122 | total_bytes = 0; | 1114 | /* |
| 1123 | for (i = 0; i < slice->nents; i++) { | 1115 | * After this for loop is complete, first_n represents the index |
| 1124 | total_bytes += le32_to_cpu(reqs[i].len); | 1116 | * of the last DMA request of this slice that needs to be |
| 1125 | if (total_bytes >= resize) | 1117 | * transferred after resizing and last_bytes represents DMA size |
| 1118 | * of that request. | ||
| 1119 | */ | ||
| 1120 | last_bytes = resize; | ||
| 1121 | for (first_n = 0; first_n < slice->nents; first_n++) | ||
| 1122 | if (last_bytes > le32_to_cpu(reqs[first_n].len)) | ||
| 1123 | last_bytes -= le32_to_cpu(reqs[first_n].len); | ||
| 1124 | else | ||
| 1126 | break; | 1125 | break; |
| 1127 | } | ||
| 1128 | |||
| 1129 | if (total_bytes < resize) { | ||
| 1130 | /* User space should have used the full buffer path. */ | ||
| 1131 | ret = -EINVAL; | ||
| 1132 | return ret; | ||
| 1133 | } | ||
| 1134 | |||
| 1135 | first_n = i; | ||
| 1136 | last_bytes = i ? resize + le32_to_cpu(reqs[i].len) - total_bytes : resize; | ||
| 1137 | 1126 | ||
| 1138 | if (avail < (first_n + 1)) | 1127 | if (avail < (first_n + 1)) |
| 1139 | return -EAGAIN; | 1128 | return -EAGAIN; |
| @@ -1142,22 +1131,21 @@ static inline int copy_partial_exec_reqs(struct qaic_device *qdev, struct bo_sli · 2 unmodified lines | |||
| 1142 | if (tail + first_n > dbc->nelem) { | 1131 | if (tail + first_n > dbc->nelem) { |
| 1143 | avail = dbc->nelem - tail; | 1132 | avail = dbc->nelem - tail; |
| 1144 | avail = min_t(u32, avail, first_n); | 1133 | avail = min_t(u32, avail, first_n); |
| 1145 | memcpy(dbc->req_q_base + tail * get_dbc_req_elem_size(), reqs, | 1134 | memcpy(fifo_at(dbc->req_q_base, tail), reqs, sizeof(*reqs) * avail); |
| 1146 | sizeof(*reqs) * avail); | ||
| 1147 | last_req = reqs + avail; | 1135 | last_req = reqs + avail; |
| 1148 | avail = first_n - avail; | 1136 | avail = first_n - avail; |
| 1149 | if (avail) | 1137 | if (avail) |
| 1150 | memcpy(dbc->req_q_base, last_req, sizeof(*reqs) * avail); | 1138 | memcpy(dbc->req_q_base, last_req, sizeof(*reqs) * avail); |
| 1151 | } else { | 1139 | } else { |
| 1152 | memcpy(dbc->req_q_base + tail * get_dbc_req_elem_size(), reqs, | 1140 | memcpy(fifo_at(dbc->req_q_base, tail), reqs, sizeof(*reqs) * first_n); |
| 1153 | sizeof(*reqs) * first_n); | ||
| 1154 | } | 1141 | } |
| 1155 | } | 1142 | } |
| 1156 | 1143 | ||
| 1157 | /* Copy over the last entry. Here we need to adjust len to the left over | 1144 | /* |
| 1145 | * Copy over the last entry. Here we need to adjust len to the left over | ||
| 1158 | * size, and set src and dst to the entry it is copied to. | 1146 | * size, and set src and dst to the entry it is copied to. |
| 1159 | */ | 1147 | */ |
| 1160 | last_req = dbc->req_q_base + (tail + first_n) % dbc->nelem * get_dbc_req_elem_size(); | 1148 | last_req = fifo_at(dbc->req_q_base, (tail + first_n) % dbc->nelem); |
| 1161 | memcpy(last_req, reqs + slice->nents - 1, sizeof(*reqs)); | 1149 | memcpy(last_req, reqs + slice->nents - 1, sizeof(*reqs)); |
| 1162 | 1150 | ||
| 1163 | /* | 1151 | /* |
| @@ -1168,6 +1156,9 @@ static inline int copy_partial_exec_reqs(struct qaic_device *qdev, struct bo_sli · 4 unmodified lines | |||
| 1168 | last_req->len = cpu_to_le32((u32)last_bytes); | 1156 | last_req->len = cpu_to_le32((u32)last_bytes); |
| 1169 | last_req->src_addr = reqs[first_n].src_addr; | 1157 | last_req->src_addr = reqs[first_n].src_addr; |
| 1170 | last_req->dest_addr = reqs[first_n].dest_addr; | 1158 | last_req->dest_addr = reqs[first_n].dest_addr; |
| 1159 | if (!last_bytes) | ||
| 1160 | /* Disable DMA transfer */ | ||
| 1161 | last_req->cmd = GENMASK(7, 2) & reqs[first_n].cmd; | ||
| 1171 | 1162 | ||
| 1172 | *ptail = (tail + first_n + 1) % dbc->nelem; | 1163 | *ptail = (tail + first_n + 1) % dbc->nelem; |
| 1173 | 1164 | ||
| @@ -1227,26 +1218,17 @@ static int send_bo_list_to_device(struct qaic_device *qdev, struct drm_file *fil · 53 unmodified lines | |||
| 1227 | bo->req_id = dbc->next_req_id++; | 1218 | bo->req_id = dbc->next_req_id++; |
| 1228 | 1219 | ||
| 1229 | list_for_each_entry(slice, &bo->slices, slice) { | 1220 | list_for_each_entry(slice, &bo->slices, slice) { |
| 1230 | /* | ||
| 1231 | * If this slice does not fall under the given | ||
| 1232 | * resize then skip this slice and continue the loop | ||
| 1233 | */ | ||
| 1234 | if (is_partial && pexec[i].resize && pexec[i].resize <= slice->offset) | ||
| 1235 | continue; | ||
| 1236 | |||
| 1237 | for (j = 0; j < slice->nents; j++) | 1221 | for (j = 0; j < slice->nents; j++) |
| 1238 | slice->reqs[j].req_id = cpu_to_le16(bo->req_id); | 1222 | slice->reqs[j].req_id = cpu_to_le16(bo->req_id); |
| 1239 | 1223 | ||
| 1240 | /* | 1224 | if (is_partial && (!pexec[i].resize || pexec[i].resize <= slice->offset)) |
| 1241 | * If it is a partial execute ioctl call then check if | 1225 | /* Configure the slice for no DMA transfer */ |
| 1242 | * resize has cut this slice short then do a partial copy | 1226 | ret = copy_partial_exec_reqs(qdev, slice, 0, dbc, head, tail); |
| 1243 | * else do complete copy | 1227 | else if (is_partial && pexec[i].resize < slice->offset + slice->size) |
| 1244 | */ | 1228 | /* Configure the slice to be partially DMA transferred */ |
| 1245 | if (is_partial && pexec[i].resize && | ||
| 1246 | pexec[i].resize < slice->offset + slice->size) | ||
| 1247 | ret = copy_partial_exec_reqs(qdev, slice, | 1229 | ret = copy_partial_exec_reqs(qdev, slice, |
| 1248 | pexec[i].resize - slice->offset, | 1230 | pexec[i].resize - slice->offset, dbc, |
| 1249 | dbc->id, head, tail); | 1231 | head, tail); |
| 1250 | else | 1232 | else |
| 1251 | ret = copy_exec_reqs(qdev, slice, dbc->id, head, tail); | 1233 | ret = copy_exec_reqs(qdev, slice, dbc->id, head, tail); |
| 1252 | if (ret) { | 1234 | if (ret) { |
| @@ -1359,7 +1341,7 @@ static int __qaic_execute_bo_ioctl(struct drm_device *dev, void *data, struct dr · 106 unmodified lines | |||
| 1359 | 1341 | ||
| 1360 | qdev = usr->qddev->qdev; | 1342 | qdev = usr->qddev->qdev; |
| 1361 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); | 1343 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 1362 | if (qdev->in_reset) { | 1344 | if (qdev->dev_state != QAIC_ONLINE) { |
| 1363 | ret = -ENODEV; | 1345 | ret = -ENODEV; |
| 1364 | goto unlock_dev_srcu; | 1346 | goto unlock_dev_srcu; |
| 1365 | } | 1347 | } |
| @@ -1466,6 +1448,16 @@ irqreturn_t dbc_irq_handler(int irq, void *data) · 100 unmodified lines | |||
| 1466 | 1448 | ||
| 1467 | rcu_id = srcu_read_lock(&dbc->ch_lock); | 1449 | rcu_id = srcu_read_lock(&dbc->ch_lock); |
| 1468 | 1450 | ||
| 1451 | if (datapath_polling) { | ||
| 1452 | srcu_read_unlock(&dbc->ch_lock, rcu_id); | ||
| 1453 | /* | ||
| 1454 | * Normally datapath_polling will not have irqs enabled, but | ||
| 1455 | * when running with only one MSI the interrupt is shared with | ||
| 1456 | * MHI so it cannot be disabled. Return ASAP instead. | ||
| 1457 | */ | ||
| 1458 | return IRQ_HANDLED; | ||
| 1459 | } | ||
| 1460 | |||
| 1469 | if (!dbc->usr) { | 1461 | if (!dbc->usr) { |
| 1470 | srcu_read_unlock(&dbc->ch_lock, rcu_id); | 1462 | srcu_read_unlock(&dbc->ch_lock, rcu_id); |
| 1471 | return IRQ_HANDLED; | 1463 | return IRQ_HANDLED; |
| @@ -1488,7 +1480,8 @@ irqreturn_t dbc_irq_handler(int irq, void *data) · 16 unmodified lines | |||
| 1488 | return IRQ_NONE; | 1480 | return IRQ_NONE; |
| 1489 | } | 1481 | } |
| 1490 | 1482 | ||
| 1491 | disable_irq_nosync(irq); | 1483 | if (!dbc->qdev->single_msi) |
| 1484 | disable_irq_nosync(irq); | ||
| 1492 | srcu_read_unlock(&dbc->ch_lock, rcu_id); | 1485 | srcu_read_unlock(&dbc->ch_lock, rcu_id); |
| 1493 | return IRQ_WAKE_THREAD; | 1486 | return IRQ_WAKE_THREAD; |
| 1494 | } | 1487 | } |
| @@ -1504,7 +1497,7 @@ void irq_polling_work(struct work_struct *work) · 9 unmodified lines | |||
| 1504 | rcu_id = srcu_read_lock(&dbc->ch_lock); | 1497 | rcu_id = srcu_read_lock(&dbc->ch_lock); |
| 1505 | 1498 | ||
| 1506 | while (1) { | 1499 | while (1) { |
| 1507 | if (dbc->qdev->in_reset) { | 1500 | if (dbc->qdev->dev_state != QAIC_ONLINE) { |
| 1508 | srcu_read_unlock(&dbc->ch_lock, rcu_id); | 1501 | srcu_read_unlock(&dbc->ch_lock, rcu_id); |
| 1509 | return; | 1502 | return; |
| 1510 | } | 1503 | } |
| @@ -1559,12 +1552,12 @@ irqreturn_t dbc_irq_threaded_fn(int irq, void *data) · 48 unmodified lines | |||
| 1559 | u32 tail; | 1552 | u32 tail; |
| 1560 | 1553 | ||
| 1561 | rcu_id = srcu_read_lock(&dbc->ch_lock); | 1554 | rcu_id = srcu_read_lock(&dbc->ch_lock); |
| 1555 | qdev = dbc->qdev; | ||
| 1562 | 1556 | ||
| 1563 | head = readl(dbc->dbc_base + RSPHP_OFF); | 1557 | head = readl(dbc->dbc_base + RSPHP_OFF); |
| 1564 | if (head == U32_MAX) /* PCI link error */ | 1558 | if (head == U32_MAX) /* PCI link error */ |
| 1565 | goto error_out; | 1559 | goto error_out; |
| 1566 | 1560 | ||
| 1567 | qdev = dbc->qdev; | ||
| 1568 | read_fifo: | 1561 | read_fifo: |
| 1569 | 1562 | ||
| 1570 | if (!event_count) { | 1563 | if (!event_count) { |
| @@ -1645,14 +1638,14 @@ read_fifo: · 74 unmodified lines | |||
| 1645 | goto read_fifo; | 1638 | goto read_fifo; |
| 1646 | 1639 | ||
| 1647 | normal_out: | 1640 | normal_out: |
| 1648 | if (likely(!datapath_polling)) | 1641 | if (!qdev->single_msi && likely(!datapath_polling)) |
| 1649 | enable_irq(irq); | 1642 | enable_irq(irq); |
| 1650 | else | 1643 | else if (unlikely(datapath_polling)) |
| 1651 | schedule_work(&dbc->poll_work); | 1644 | schedule_work(&dbc->poll_work); |
| 1652 | /* checking the fifo and enabling irqs is a race, missed event check */ | 1645 | /* checking the fifo and enabling irqs is a race, missed event check */ |
| 1653 | tail = readl(dbc->dbc_base + RSPTP_OFF); | 1646 | tail = readl(dbc->dbc_base + RSPTP_OFF); |
| 1654 | if (tail != U32_MAX && head != tail) { | 1647 | if (tail != U32_MAX && head != tail) { |
| 1655 | if (likely(!datapath_polling)) | 1648 | if (!qdev->single_msi && likely(!datapath_polling)) |
| 1656 | disable_irq_nosync(irq); | 1649 | disable_irq_nosync(irq); |
| 1657 | goto read_fifo; | 1650 | goto read_fifo; |
| 1658 | } | 1651 | } |
| @@ -1661,9 +1654,9 @@ normal_out: · 2 unmodified lines | |||
| 1661 | 1654 | ||
| 1662 | error_out: | 1655 | error_out: |
| 1663 | srcu_read_unlock(&dbc->ch_lock, rcu_id); | 1656 | srcu_read_unlock(&dbc->ch_lock, rcu_id); |
| 1664 | if (likely(!datapath_polling)) | 1657 | if (!qdev->single_msi && likely(!datapath_polling)) |
| 1665 | enable_irq(irq); | 1658 | enable_irq(irq); |
| 1666 | else | 1659 | else if (unlikely(datapath_polling)) |
| 1667 | schedule_work(&dbc->poll_work); | 1660 | schedule_work(&dbc->poll_work); |
| 1668 | 1661 | ||
| 1669 | return IRQ_HANDLED; | 1662 | return IRQ_HANDLED; |
| @@ -1694,7 +1687,7 @@ int qaic_wait_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *file · 24 unmodified lines | |||
| 1694 | 1687 | ||
| 1695 | qdev = usr->qddev->qdev; | 1688 | qdev = usr->qddev->qdev; |
| 1696 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); | 1689 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 1697 | if (qdev->in_reset) { | 1690 | if (qdev->dev_state != QAIC_ONLINE) { |
| 1698 | ret = -ENODEV; | 1691 | ret = -ENODEV; |
| 1699 | goto unlock_dev_srcu; | 1692 | goto unlock_dev_srcu; |
| 1700 | } | 1693 | } |
| @@ -1763,7 +1756,7 @@ int qaic_perf_stats_bo_ioctl(struct drm_device *dev, void *data, struct drm_file · 62 unmodified lines | |||
| 1763 | 1756 | ||
| 1764 | qdev = usr->qddev->qdev; | 1757 | qdev = usr->qddev->qdev; |
| 1765 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); | 1758 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 1766 | if (qdev->in_reset) { | 1759 | if (qdev->dev_state != QAIC_ONLINE) { |
| 1767 | ret = -ENODEV; | 1760 | ret = -ENODEV; |
| 1768 | goto unlock_dev_srcu; | 1761 | goto unlock_dev_srcu; |
| 1769 | } | 1762 | } |
| @@ -1854,7 +1847,7 @@ int qaic_detach_slice_bo_ioctl(struct drm_device *dev, void *data, struct drm_fi · 84 unmodified lines | |||
| 1854 | 1847 | ||
| 1855 | qdev = usr->qddev->qdev; | 1848 | qdev = usr->qddev->qdev; |
| 1856 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); | 1849 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 1857 | if (qdev->in_reset) { | 1850 | if (qdev->dev_state != QAIC_ONLINE) { |
| 1858 | ret = -ENODEV; | 1851 | ret = -ENODEV; |
| 1859 | goto unlock_dev_srcu; | 1852 | goto unlock_dev_srcu; |
| 1860 | } | 1853 | } |
| @@ -8,6 +8,7 @@ · 7 unmodified lines | |||
| 8 | #include <linux/idr.h> | 8 | #include <linux/idr.h> |
| 9 | #include <linux/interrupt.h> | 9 | #include <linux/interrupt.h> |
| 10 | #include <linux/list.h> | 10 | #include <linux/list.h> |
| 11 | #include <linux/kobject.h> | ||
| 11 | #include <linux/kref.h> | 12 | #include <linux/kref.h> |
| 12 | #include <linux/mhi.h> | 13 | #include <linux/mhi.h> |
| 13 | #include <linux/module.h> | 14 | #include <linux/module.h> |
| @@ -27,6 +28,7 @@ · 13 unmodified lines | |||
| 27 | 28 | ||
| 28 | #include "mhi_controller.h" | 29 | #include "mhi_controller.h" |
| 29 | #include "qaic.h" | 30 | #include "qaic.h" |
| 31 | #include "qaic_timesync.h" | ||
| 30 | 32 | ||
| 31 | MODULE_IMPORT_NS(DMA_BUF); | 33 | MODULE_IMPORT_NS(DMA_BUF); |
| 32 | 34 | ||
| @@ -42,9 +44,6 @@ MODULE_PARM_DESC(datapath_polling, "Operate the datapath in polling mode"); · 9 unmodified lines | |||
| 42 | static bool link_up; | 44 | static bool link_up; |
| 43 | static DEFINE_IDA(qaic_usrs); | 45 | static DEFINE_IDA(qaic_usrs); |
| 44 | 46 | ||
| 45 | static int qaic_create_drm_device(struct qaic_device *qdev, s32 partition_id); | ||
| 46 | static void qaic_destroy_drm_device(struct qaic_device *qdev, s32 partition_id); | ||
| 47 | |||
| 48 | static void free_usr(struct kref *kref) | 47 | static void free_usr(struct kref *kref) |
| 49 | { | 48 | { |
| 50 | struct qaic_user *usr = container_of(kref, struct qaic_user, ref_count); | 49 | struct qaic_user *usr = container_of(kref, struct qaic_user, ref_count); |
| @@ -63,7 +62,7 @@ static int qaic_open(struct drm_device *dev, struct drm_file *file) · 12 unmodified lines | |||
| 63 | int ret; | 62 | int ret; |
| 64 | 63 | ||
| 65 | rcu_id = srcu_read_lock(&qdev->dev_lock); | 64 | rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 66 | if (qdev->in_reset) { | 65 | if (qdev->dev_state != QAIC_ONLINE) { |
| 67 | ret = -ENODEV; | 66 | ret = -ENODEV; |
| 68 | goto dev_unlock; | 67 | goto dev_unlock; |
| 69 | } | 68 | } |
| @@ -120,7 +119,7 @@ static void qaic_postclose(struct drm_device *dev, struct drm_file *file) · 50 unmodified lines | |||
| 120 | if (qddev) { | 119 | if (qddev) { |
| 121 | qdev = qddev->qdev; | 120 | qdev = qddev->qdev; |
| 122 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); | 121 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 123 | if (!qdev->in_reset) { | 122 | if (qdev->dev_state == QAIC_ONLINE) { |
| 124 | qaic_release_usr(qdev, usr); | 123 | qaic_release_usr(qdev, usr); |
| 125 | for (i = 0; i < qdev->num_dbc; ++i) | 124 | for (i = 0; i < qdev->num_dbc; ++i) |
| 126 | if (qdev->dbc[i].usr && qdev->dbc[i].usr->handle == usr->handle) | 125 | if (qdev->dbc[i].usr && qdev->dbc[i].usr->handle == usr->handle) |
| @@ -182,13 +181,6 @@ static int qaic_create_drm_device(struct qaic_device *qdev, s32 partition_id) · 55 unmodified lines | |||
| 182 | 181 | ||
| 183 | qddev->partition_id = partition_id; | 182 | qddev->partition_id = partition_id; |
| 184 | 183 | ||
| 185 | /* | ||
| 186 | * drm_dev_unregister() sets the driver data to NULL and | ||
| 187 | * drm_dev_register() does not update the driver data. During a SOC | ||
| 188 | * reset drm dev is unregistered and registered again leaving the | ||
| 189 | * driver data to NULL. | ||
| 190 | */ | ||
| 191 | dev_set_drvdata(to_accel_kdev(qddev), drm->accel); | ||
| 192 | ret = drm_dev_register(drm, 0); | 184 | ret = drm_dev_register(drm, 0); |
| 193 | if (ret) | 185 | if (ret) |
| 194 | pci_dbg(qdev->pdev, "drm_dev_register failed %d\n", ret); | 186 | pci_dbg(qdev->pdev, "drm_dev_register failed %d\n", ret); |
| @@ -202,7 +194,6 @@ static void qaic_destroy_drm_device(struct qaic_device *qdev, s32 partition_id) · 7 unmodified lines | |||
| 202 | struct drm_device *drm = to_drm(qddev); | 194 | struct drm_device *drm = to_drm(qddev); |
| 203 | struct qaic_user *usr; | 195 | struct qaic_user *usr; |
| 204 | 196 | ||
| 205 | drm_dev_get(drm); | ||
| 206 | drm_dev_unregister(drm); | 197 | drm_dev_unregister(drm); |
| 207 | qddev->partition_id = 0; | 198 | qddev->partition_id = 0; |
| 208 | /* | 199 | /* |
| @@ -231,7 +222,6 @@ static void qaic_destroy_drm_device(struct qaic_device *qdev, s32 partition_id) · 22 unmodified lines | |||
| 231 | mutex_lock(&qddev->users_mutex); | 222 | mutex_lock(&qddev->users_mutex); |
| 232 | } | 223 | } |
| 233 | mutex_unlock(&qddev->users_mutex); | 224 | mutex_unlock(&qddev->users_mutex); |
| 234 | drm_dev_put(drm); | ||
| 235 | } | 225 | } |
| 236 | 226 | ||
| 237 | static int qaic_mhi_probe(struct mhi_device *mhi_dev, const struct mhi_device_id *id) | 227 | static int qaic_mhi_probe(struct mhi_device *mhi_dev, const struct mhi_device_id *id) |
| @@ -253,8 +243,6 @@ static int qaic_mhi_probe(struct mhi_device *mhi_dev, const struct mhi_device_id · 15 unmodified lines | |||
| 253 | 243 | ||
| 254 | qdev = pci_get_drvdata(to_pci_dev(mhi_dev->mhi_cntrl->cntrl_dev)); | 244 | qdev = pci_get_drvdata(to_pci_dev(mhi_dev->mhi_cntrl->cntrl_dev)); |
| 255 | 245 | ||
| 256 | qdev->in_reset = false; | ||
| 257 | |||
| 258 | dev_set_drvdata(&mhi_dev->dev, qdev); | 246 | dev_set_drvdata(&mhi_dev->dev, qdev); |
| 259 | qdev->cntl_ch = mhi_dev; | 247 | qdev->cntl_ch = mhi_dev; |
| 260 | 248 | ||
| @@ -264,6 +252,7 @@ static int qaic_mhi_probe(struct mhi_device *mhi_dev, const struct mhi_device_id · 3 unmodified lines | |||
| 264 | return ret; | 252 | return ret; |
| 265 | } | 253 | } |
| 266 | 254 | ||
| 255 | qdev->dev_state = QAIC_BOOT; | ||
| 267 | ret = get_cntl_version(qdev, NULL, &major, &minor); | 256 | ret = get_cntl_version(qdev, NULL, &major, &minor); |
| 268 | if (ret || major != CNTL_MAJOR || minor > CNTL_MINOR) { | 257 | if (ret || major != CNTL_MAJOR || minor > CNTL_MINOR) { |
| 269 | pci_err(qdev->pdev, "%s: Control protocol version (%d.%d) not supported. Supported version is (%d.%d). Ret: %d\n", | 258 | pci_err(qdev->pdev, "%s: Control protocol version (%d.%d) not supported. Supported version is (%d.%d). Ret: %d\n", |
| @@ -271,8 +260,8 @@ static int qaic_mhi_probe(struct mhi_device *mhi_dev, const struct mhi_device_id · 1 unmodified line | |||
| 271 | ret = -EINVAL; | 260 | ret = -EINVAL; |
| 272 | goto close_control; | 261 | goto close_control; |
| 273 | } | 262 | } |
| 274 | 263 | qdev->dev_state = QAIC_ONLINE; | |
| 275 | ret = qaic_create_drm_device(qdev, QAIC_NO_PARTITION); | 264 | kobject_uevent(&(to_accel_kdev(qdev->qddev))->kobj, KOBJ_ONLINE); |
| 276 | 265 | ||
| 277 | return ret; | 266 | return ret; |
| 278 | 267 | ||
| @@ -290,7 +279,8 @@ static void qaic_notify_reset(struct qaic_device *qdev) · 11 unmodified lines | |||
| 290 | { | 279 | { |
| 291 | int i; | 280 | int i; |
| 292 | 281 | ||
| 293 | qdev->in_reset = true; | 282 | kobject_uevent(&(to_accel_kdev(qdev->qddev))->kobj, KOBJ_OFFLINE); |
| 283 | qdev->dev_state = QAIC_OFFLINE; | ||
| 294 | /* wake up any waiters to avoid waiting for timeouts at sync */ | 284 | /* wake up any waiters to avoid waiting for timeouts at sync */ |
| 295 | wake_all_cntl(qdev); | 285 | wake_all_cntl(qdev); |
| 296 | for (i = 0; i < qdev->num_dbc; ++i) | 286 | for (i = 0; i < qdev->num_dbc; ++i) |
| @@ -298,21 +288,15 @@ static void qaic_notify_reset(struct qaic_device *qdev) · 1 unmodified line | |||
| 298 | synchronize_srcu(&qdev->dev_lock); | 288 | synchronize_srcu(&qdev->dev_lock); |
| 299 | } | 289 | } |
| 300 | 290 | ||
| 301 | void qaic_dev_reset_clean_local_state(struct qaic_device *qdev, bool exit_reset) | 291 | void qaic_dev_reset_clean_local_state(struct qaic_device *qdev) |
| 302 | { | 292 | { |
| 303 | int i; | 293 | int i; |
| 304 | 294 | ||
| 305 | qaic_notify_reset(qdev); | 295 | qaic_notify_reset(qdev); |
| 306 | 296 | ||
| 307 | /* remove drmdevs to prevent new users from coming in */ | ||
| 308 | qaic_destroy_drm_device(qdev, QAIC_NO_PARTITION); | ||
| 309 | |||
| 310 | /* start tearing things down */ | 297 | /* start tearing things down */ |
| 311 | for (i = 0; i < qdev->num_dbc; ++i) | 298 | for (i = 0; i < qdev->num_dbc; ++i) |
| 312 | release_dbc(qdev, i); | 299 | release_dbc(qdev, i); |
| 313 | |||
| 314 | if (exit_reset) | ||
| 315 | qdev->in_reset = false; | ||
| 316 | } | 300 | } |
| 317 | 301 | ||
| 318 | static void cleanup_qdev(struct qaic_device *qdev) | 302 | static void cleanup_qdev(struct qaic_device *qdev) |
| @@ -324,6 +308,7 @@ static void cleanup_qdev(struct qaic_device *qdev) · 5 unmodified lines | |||
| 324 | cleanup_srcu_struct(&qdev->dev_lock); | 308 | cleanup_srcu_struct(&qdev->dev_lock); |
| 325 | pci_set_drvdata(qdev->pdev, NULL); | 309 | pci_set_drvdata(qdev->pdev, NULL); |
| 326 | destroy_workqueue(qdev->cntl_wq); | 310 | destroy_workqueue(qdev->cntl_wq); |
| 311 | destroy_workqueue(qdev->qts_wq); | ||
| 327 | } | 312 | } |
| 328 | 313 | ||
| 329 | static struct qaic_device *create_qdev(struct pci_dev *pdev, const struct pci_device_id *id) | 314 | static struct qaic_device *create_qdev(struct pci_dev *pdev, const struct pci_device_id *id) |
| @@ -336,6 +321,7 @@ static struct qaic_device *create_qdev(struct pci_dev *pdev, const struct pci_de · 6 unmodified lines | |||
| 336 | if (!qdev) | 321 | if (!qdev) |
| 337 | return NULL; | 322 | return NULL; |
| 338 | 323 | ||
| 324 | qdev->dev_state = QAIC_OFFLINE; | ||
| 339 | if (id->device == PCI_DEV_AIC100) { | 325 | if (id->device == PCI_DEV_AIC100) { |
| 340 | qdev->num_dbc = 16; | 326 | qdev->num_dbc = 16; |
| 341 | qdev->dbc = devm_kcalloc(&pdev->dev, qdev->num_dbc, sizeof(*qdev->dbc), GFP_KERNEL); | 327 | qdev->dbc = devm_kcalloc(&pdev->dev, qdev->num_dbc, sizeof(*qdev->dbc), GFP_KERNEL); |
| @@ -347,6 +333,12 @@ static struct qaic_device *create_qdev(struct pci_dev *pdev, const struct pci_de · 5 unmodified lines | |||
| 347 | if (!qdev->cntl_wq) | 333 | if (!qdev->cntl_wq) |
| 348 | return NULL; | 334 | return NULL; |
| 349 | 335 | ||
| 336 | qdev->qts_wq = alloc_workqueue("qaic_ts", WQ_UNBOUND, 0); | ||
| 337 | if (!qdev->qts_wq) { | ||
| 338 | destroy_workqueue(qdev->cntl_wq); | ||
| 339 | return NULL; | ||
| 340 | } | ||
| 341 | |||
| 350 | pci_set_drvdata(pdev, qdev); | 342 | pci_set_drvdata(pdev, qdev); |
| 351 | qdev->pdev = pdev; | 343 | qdev->pdev = pdev; |
| 352 | 344 | ||
| @@ -424,14 +416,24 @@ static int init_msi(struct qaic_device *qdev, struct pci_dev *pdev) · 71 unmodified lines | |||
| 424 | int i; | 416 | int i; |
| 425 | 417 | ||
| 426 | /* Managed release since we use pcim_enable_device */ | 418 | /* Managed release since we use pcim_enable_device */ |
| 427 | ret = pci_alloc_irq_vectors(pdev, 1, 32, PCI_IRQ_MSI); | 419 | ret = pci_alloc_irq_vectors(pdev, 32, 32, PCI_IRQ_MSI); |
| 428 | if (ret < 0) | 420 | if (ret == -ENOSPC) { |
| 429 | return ret; | 421 | ret = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_MSI); |
| 422 | if (ret < 0) | ||
| 423 | return ret; | ||
| 430 | 424 | ||
| 431 | if (ret < 32) { | 425 | /* |
| 432 | pci_err(pdev, "%s: Requested 32 MSIs. Obtained %d MSIs which is less than the 32 required.\n", | 426 | * Operate in one MSI mode. All interrupts will be directed to |
| 433 | __func__, ret); | 427 | * MSI0; every interrupt will wake up all the interrupt handlers |
| 434 | return -ENODEV; | 428 | * (MHI and DBC[0-15]). Since the interrupt is now shared, it is |
| 429 | * not disabled during DBC threaded handler, but only one thread | ||
| 430 | * will be allowed to run per DBC, so while it can be | ||
| 431 | * interrupted, it shouldn't race with itself. | ||
| 432 | */ | ||
| 433 | qdev->single_msi = true; | ||
| 434 | pci_info(pdev, "Allocating 32 MSIs failed, operating in 1 MSI mode. Performance may be impacted.\n"); | ||
| 435 | } else if (ret < 0) { | ||
| 436 | return ret; | ||
| 435 | } | 437 | } |
| 436 | 438 | ||
| 437 | mhi_irq = pci_irq_vector(pdev, 0); | 439 | mhi_irq = pci_irq_vector(pdev, 0); |
| @@ -439,15 +441,17 @@ static int init_msi(struct qaic_device *qdev, struct pci_dev *pdev) · 1 unmodified line | |||
| 439 | return mhi_irq; | 441 | return mhi_irq; |
| 440 | 442 | ||
| 441 | for (i = 0; i < qdev->num_dbc; ++i) { | 443 | for (i = 0; i < qdev->num_dbc; ++i) { |
| 442 | ret = devm_request_threaded_irq(&pdev->dev, pci_irq_vector(pdev, i + 1), | 444 | ret = devm_request_threaded_irq(&pdev->dev, |
| 445 | pci_irq_vector(pdev, qdev->single_msi ? 0 : i + 1), | ||
| 443 | dbc_irq_handler, dbc_irq_threaded_fn, IRQF_SHARED, | 446 | dbc_irq_handler, dbc_irq_threaded_fn, IRQF_SHARED, |
| 444 | "qaic_dbc", &qdev->dbc[i]); | 447 | "qaic_dbc", &qdev->dbc[i]); |
| 445 | if (ret) | 448 | if (ret) |
| 446 | return ret; | 449 | return ret; |
| 447 | 450 | ||
| 448 | if (datapath_polling) { | 451 | if (datapath_polling) { |
| 449 | qdev->dbc[i].irq = pci_irq_vector(pdev, i + 1); | 452 | qdev->dbc[i].irq = pci_irq_vector(pdev, qdev->single_msi ? 0 : i + 1); |
| 450 | disable_irq_nosync(qdev->dbc[i].irq); | 453 | if (!qdev->single_msi) |
| 454 | disable_irq_nosync(qdev->dbc[i].irq); | ||
| 451 | INIT_WORK(&qdev->dbc[i].poll_work, irq_polling_work); | 455 | INIT_WORK(&qdev->dbc[i].poll_work, irq_polling_work); |
| 452 | } | 456 | } |
| 453 | } | 457 | } |
| @@ -479,14 +483,21 @@ static int qaic_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) · 25 unmodified lines | |||
| 479 | goto cleanup_qdev; | 483 | goto cleanup_qdev; |
| 480 | } | 484 | } |
| 481 | 485 | ||
| 482 | qdev->mhi_cntrl = qaic_mhi_register_controller(pdev, qdev->bar_0, mhi_irq); | 486 | ret = qaic_create_drm_device(qdev, QAIC_NO_PARTITION); |
| 487 | if (ret) | ||
| 488 | goto cleanup_qdev; | ||
| 489 | |||
| 490 | qdev->mhi_cntrl = qaic_mhi_register_controller(pdev, qdev->bar_0, mhi_irq, | ||
| 491 | qdev->single_msi); | ||
| 483 | if (IS_ERR(qdev->mhi_cntrl)) { | 492 | if (IS_ERR(qdev->mhi_cntrl)) { |
| 484 | ret = PTR_ERR(qdev->mhi_cntrl); | 493 | ret = PTR_ERR(qdev->mhi_cntrl); |
| 485 | goto cleanup_qdev; | 494 | goto cleanup_drm_dev; |
| 486 | } | 495 | } |
| 487 | 496 | ||
| 488 | return 0; | 497 | return 0; |
| 489 | 498 | ||
| 499 | cleanup_drm_dev: | ||
| 500 | qaic_destroy_drm_device(qdev, QAIC_NO_PARTITION); | ||
| 490 | cleanup_qdev: | 501 | cleanup_qdev: |
| 491 | cleanup_qdev(qdev); | 502 | cleanup_qdev(qdev); |
| 492 | return ret; | 503 | return ret; |
| @@ -499,7 +510,8 @@ static void qaic_pci_remove(struct pci_dev *pdev) · 6 unmodified lines | |||
| 499 | if (!qdev) | 510 | if (!qdev) |
| 500 | return; | 511 | return; |
| 501 | 512 | ||
| 502 | qaic_dev_reset_clean_local_state(qdev, false); | 513 | qaic_dev_reset_clean_local_state(qdev); |
| 514 | qaic_destroy_drm_device(qdev, QAIC_NO_PARTITION); | ||
| 503 | qaic_mhi_free_controller(qdev->mhi_cntrl, link_up); | 515 | qaic_mhi_free_controller(qdev->mhi_cntrl, link_up); |
| 504 | cleanup_qdev(qdev); | 516 | cleanup_qdev(qdev); |
| 505 | } | 517 | } |
| @@ -522,14 +534,13 @@ static void qaic_pci_reset_prepare(struct pci_dev *pdev) · 16 unmodified lines | |||
| 522 | 534 | ||
| 523 | qaic_notify_reset(qdev); | 535 | qaic_notify_reset(qdev); |
| 524 | qaic_mhi_start_reset(qdev->mhi_cntrl); | 536 | qaic_mhi_start_reset(qdev->mhi_cntrl); |
| 525 | qaic_dev_reset_clean_local_state(qdev, false); | 537 | qaic_dev_reset_clean_local_state(qdev); |
| 526 | } | 538 | } |
| 527 | 539 | ||
| 528 | static void qaic_pci_reset_done(struct pci_dev *pdev) | 540 | static void qaic_pci_reset_done(struct pci_dev *pdev) |
| 529 | { | 541 | { |
| 530 | struct qaic_device *qdev = pci_get_drvdata(pdev); | 542 | struct qaic_device *qdev = pci_get_drvdata(pdev); |
| 531 | 543 | ||
| 532 | qdev->in_reset = false; | ||
| 533 | qaic_mhi_reset_done(qdev->mhi_cntrl); | 544 | qaic_mhi_reset_done(qdev->mhi_cntrl); |
| 534 | } | 545 | } |
| 535 | 546 | ||
| @@ -586,6 +597,10 @@ static int __init qaic_init(void) · 50 unmodified lines | |||
| 586 | goto free_pci; | 597 | goto free_pci; |
| 587 | } | 598 | } |
| 588 | 599 | ||
| 600 | ret = qaic_timesync_init(); | ||
| 601 | if (ret) | ||
| 602 | pr_debug("qaic: qaic_timesync_init failed %d\n", ret); | ||
| 603 | |||
| 589 | return 0; | 604 | return 0; |
| 590 | 605 | ||
| 591 | free_pci: | 606 | free_pci: |
| @@ -611,6 +626,7 @@ static void __exit qaic_exit(void) · 19 unmodified lines | |||
| 611 | * reinitializing the link_up state after the cleanup is done. | 626 | * reinitializing the link_up state after the cleanup is done. |
| 612 | */ | 627 | */ |
| 613 | link_up = true; | 628 | link_up = true; |
| 629 | qaic_timesync_deinit(); | ||
| 614 | mhi_driver_unregister(&qaic_mhi_driver); | 630 | mhi_driver_unregister(&qaic_mhi_driver); |
| 615 | pci_unregister_driver(&qaic_pci_driver); | 631 | pci_unregister_driver(&qaic_pci_driver); |
| 616 | } | 632 | } |
| @@ -0,0 +1,395 @@ | |||
| 1 | // SPDX-License-Identifier: GPL-2.0-only | ||
| 2 | /* Copyright (c) 2023 Qualcomm Innovation Center, Inc. All rights reserved. */ | ||
| 3 | |||
| 4 | #include <linux/io.h> | ||
| 5 | #include <linux/kernel.h> | ||
| 6 | #include <linux/math64.h> | ||
| 7 | #include <linux/mhi.h> | ||
| 8 | #include <linux/mod_devicetable.h> | ||
| 9 | #include <linux/module.h> | ||
| 10 | #include <linux/time64.h> | ||
| 11 | #include <linux/timer.h> | ||
| 12 | |||
| 13 | #include "qaic.h" | ||
| 14 | #include "qaic_timesync.h" | ||
| 15 | |||
| 16 | #define QTIMER_REG_OFFSET 0xa28 | ||
| 17 | #define QAIC_TIMESYNC_SIGNATURE 0x55aa | ||
| 18 | #define QAIC_CONV_QTIMER_TO_US(qtimer) (mul_u64_u32_div(qtimer, 10, 192)) | ||
| 19 | |||
| 20 | static unsigned int timesync_delay_ms = 1000; /* 1 sec default */ | ||
| 21 | module_param(timesync_delay_ms, uint, 0600); | ||
| 22 | MODULE_PARM_DESC(timesync_delay_ms, "Delay in ms between two consecutive timesync operations"); | ||
| 23 | |||
| 24 | enum qts_msg_type { | ||
| 25 | QAIC_TS_CMD_TO_HOST, | ||
| 26 | QAIC_TS_SYNC_REQ, | ||
| 27 | QAIC_TS_ACK_TO_HOST, | ||
| 28 | QAIC_TS_MSG_TYPE_MAX | ||
| 29 | }; | ||
| 30 | |||
| 31 | /** | ||
| 32 | * struct qts_hdr - Timesync message header structure. | ||
| 33 | * @signature: Unique signature to identify the timesync message. | ||
| 34 | * @reserved_1: Reserved for future use. | ||
| 35 | * @reserved_2: Reserved for future use. | ||
| 36 | * @msg_type: sub-type of the timesync message. | ||
| 37 | * @reserved_3: Reserved for future use. | ||
| 38 | */ | ||
| 39 | struct qts_hdr { | ||
| 40 | __le16 signature; | ||
| 41 | __le16 reserved_1; | ||
| 42 | u8 reserved_2; | ||
| 43 | u8 msg_type; | ||
| 44 | __le16 reserved_3; | ||
| 45 | } __packed; | ||
| 46 | |||
| 47 | /** | ||
| 48 | * struct qts_timeval - Structure to carry time information. | ||
| 49 | * @tv_sec: Seconds part of the time. | ||
| 50 | * @tv_usec: uS (microseconds) part of the time. | ||
| 51 | */ | ||
| 52 | struct qts_timeval { | ||
| 53 | __le64 tv_sec; | ||
| 54 | __le64 tv_usec; | ||
| 55 | } __packed; | ||
| 56 | |||
| 57 | /** | ||
| 58 | * struct qts_host_time_sync_msg_data - Structure to denote the timesync message. | ||
| 59 | * @header: Header of the timesync message. | ||
| 60 | * @data: Time information. | ||
| 61 | */ | ||
| 62 | struct qts_host_time_sync_msg_data { | ||
| 63 | struct qts_hdr header; | ||
| 64 | struct qts_timeval data; | ||
| 65 | } __packed; | ||
| 66 | |||
| 67 | /** | ||
| 68 | * struct mqts_dev - MHI QAIC Timesync Control device. | ||
| 69 | * @qdev: Pointer to the root device struct driven by QAIC driver. | ||
| 70 | * @mhi_dev: Pointer to associated MHI device. | ||
| 71 | * @timer: Timer handle used for timesync. | ||
| 72 | * @qtimer_addr: Device QTimer register pointer. | ||
| 73 | * @buff_in_use: atomic variable to track if the sync_msg buffer is in use. | ||
| 74 | * @dev: Device pointer to qdev->pdev->dev stored for easy access. | ||
| 75 | * @sync_msg: Buffer used to send timesync message over MHI. | ||
| 76 | */ | ||
| 77 | struct mqts_dev { | ||
| 78 | struct qaic_device *qdev; | ||
| 79 | struct mhi_device *mhi_dev; | ||
| 80 | struct timer_list timer; | ||
| 81 | void __iomem *qtimer_addr; | ||
| 82 | atomic_t buff_in_use; | ||
| 83 | struct device *dev; | ||
| 84 | struct qts_host_time_sync_msg_data *sync_msg; | ||
| 85 | }; | ||
| 86 | |||
| 87 | struct qts_resp_msg { | ||
| 88 | struct qts_hdr hdr; | ||
| 89 | } __packed; | ||
| 90 | |||
| 91 | struct qts_resp { | ||
| 92 | struct qts_resp_msg data; | ||
| 93 | struct work_struct work; | ||
| 94 | struct qaic_device *qdev; | ||
| 95 | }; | ||
| 96 | |||
| 97 | #ifdef readq | ||
| 98 | static u64 read_qtimer(const volatile void __iomem *addr) | ||
| 99 | { | ||
| 100 | return readq(addr); | ||
| 101 | } | ||
| 102 | #else | ||
| 103 | static u64 read_qtimer(const volatile void __iomem *addr) | ||
| 104 | { | ||
| 105 | u64 low, high; | ||
| 106 | |||
| 107 | low = readl(addr); | ||
| 108 | high = readl(addr + sizeof(u32)); | ||
| 109 | return low | (high << 32); | ||
| 110 | } | ||
| 111 | #endif | ||
| 112 | |||
| 113 | static void qaic_timesync_ul_xfer_cb(struct mhi_device *mhi_dev, struct mhi_result *mhi_result) | ||
| 114 | { | ||
| 115 | struct mqts_dev *mqtsdev = dev_get_drvdata(&mhi_dev->dev); | ||
| 116 | |||
| 117 | dev_dbg(mqtsdev->dev, "%s status: %d xfer_len: %zu\n", __func__, | ||
| 118 | mhi_result->transaction_status, mhi_result->bytes_xferd); | ||
| 119 | |||
| 120 | atomic_set(&mqtsdev->buff_in_use, 0); | ||
| 121 | } | ||
| 122 | |||
| 123 | static void qaic_timesync_dl_xfer_cb(struct mhi_device *mhi_dev, struct mhi_result *mhi_result) | ||
| 124 | { | ||
| 125 | struct mqts_dev *mqtsdev = dev_get_drvdata(&mhi_dev->dev); | ||
| 126 | |||
| 127 | dev_err(mqtsdev->dev, "%s no data expected on dl channel\n", __func__); | ||
| 128 | } | ||
| 129 | |||
| 130 | static void qaic_timesync_timer(struct timer_list *t) | ||
| 131 | { | ||
| 132 | struct mqts_dev *mqtsdev = from_timer(mqtsdev, t, timer); | ||
| 133 | struct qts_host_time_sync_msg_data *sync_msg; | ||
| 134 | u64 device_qtimer_us; | ||
| 135 | u64 device_qtimer; | ||
| 136 | u64 host_time_us; | ||
| 137 | u64 offset_us; | ||
| 138 | u64 host_sec; | ||
| 139 | int ret; | ||
| 140 | |||
| 141 | if (atomic_read(&mqtsdev->buff_in_use)) { | ||
| 142 | dev_dbg(mqtsdev->dev, "%s buffer not free, schedule next cycle\n", __func__); | ||
| 143 | goto mod_timer; | ||
| 144 | } | ||
| 145 | atomic_set(&mqtsdev->buff_in_use, 1); | ||
| 146 | |||
| 147 | sync_msg = mqtsdev->sync_msg; | ||
| 148 | sync_msg->header.signature = cpu_to_le16(QAIC_TIMESYNC_SIGNATURE); | ||
| 149 | sync_msg->header.msg_type = QAIC_TS_SYNC_REQ; | ||
| 150 | /* Read host UTC time and convert to uS*/ | ||
| 151 | host_time_us = div_u64(ktime_get_real_ns(), NSEC_PER_USEC); | ||
| 152 | device_qtimer = read_qtimer(mqtsdev->qtimer_addr); | ||
| 153 | device_qtimer_us = QAIC_CONV_QTIMER_TO_US(device_qtimer); | ||
| 154 | /* Offset between host UTC and device time */ | ||
| 155 | offset_us = host_time_us - device_qtimer_us; | ||
| 156 | |||
| 157 | host_sec = div_u64(offset_us, USEC_PER_SEC); | ||
| 158 | sync_msg->data.tv_usec = cpu_to_le64(offset_us - host_sec * USEC_PER_SEC); | ||
| 159 | sync_msg->data.tv_sec = cpu_to_le64(host_sec); | ||
| 160 | ret = mhi_queue_buf(mqtsdev->mhi_dev, DMA_TO_DEVICE, sync_msg, sizeof(*sync_msg), MHI_EOT); | ||
| 161 | if (ret && (ret != -EAGAIN)) { | ||
| 162 | dev_err(mqtsdev->dev, "%s unable to queue to mhi:%d\n", __func__, ret); | ||
| 163 | return; | ||
| 164 | } else if (ret == -EAGAIN) { | ||
| 165 | atomic_set(&mqtsdev->buff_in_use, 0); | ||
| 166 | } | ||
| 167 | |||
| 168 | mod_timer: | ||
| 169 | ret = mod_timer(t, jiffies + msecs_to_jiffies(timesync_delay_ms)); | ||
| 170 | if (ret) | ||
| 171 | dev_err(mqtsdev->dev, "%s mod_timer error:%d\n", __func__, ret); | ||
| 172 | } | ||
| 173 | |||
| 174 | static int qaic_timesync_probe(struct mhi_device *mhi_dev, const struct mhi_device_id *id) | ||
| 175 | { | ||
| 176 | struct qaic_device *qdev = pci_get_drvdata(to_pci_dev(mhi_dev->mhi_cntrl->cntrl_dev)); | ||
| 177 | struct mqts_dev *mqtsdev; | ||
| 178 | struct timer_list *timer; | ||
| 179 | int ret; | ||
| 180 | |||
| 181 | mqtsdev = kzalloc(sizeof(*mqtsdev), GFP_KERNEL); | ||
| 182 | if (!mqtsdev) { | ||
| 183 | ret = -ENOMEM; | ||
| 184 | goto out; | ||
| 185 | } | ||
| 186 | |||
| 187 | timer = &mqtsdev->timer; | ||
| 188 | mqtsdev->mhi_dev = mhi_dev; | ||
| 189 | mqtsdev->qdev = qdev; | ||
| 190 | mqtsdev->dev = &qdev->pdev->dev; | ||
| 191 | |||
| 192 | mqtsdev->sync_msg = kzalloc(sizeof(*mqtsdev->sync_msg), GFP_KERNEL); | ||
| 193 | if (!mqtsdev->sync_msg) { | ||
| 194 | ret = -ENOMEM; | ||
| 195 | goto free_mqts_dev; | ||
| 196 | } | ||
| 197 | atomic_set(&mqtsdev->buff_in_use, 0); | ||
| 198 | |||
| 199 | ret = mhi_prepare_for_transfer(mhi_dev); | ||
| 200 | if (ret) | ||
| 201 | goto free_sync_msg; | ||
| 202 | |||
| 203 | /* Qtimer register pointer */ | ||
| 204 | mqtsdev->qtimer_addr = qdev->bar_0 + QTIMER_REG_OFFSET; | ||
| 205 | timer_setup(timer, qaic_timesync_timer, 0); | ||
| 206 | timer->expires = jiffies + msecs_to_jiffies(timesync_delay_ms); | ||
| 207 | add_timer(timer); | ||
| 208 | dev_set_drvdata(&mhi_dev->dev, mqtsdev); | ||
| 209 | |||
| 210 | return 0; | ||
| 211 | |||
| 212 | free_sync_msg: | ||
| 213 | kfree(mqtsdev->sync_msg); | ||
| 214 | free_mqts_dev: | ||
| 215 | kfree(mqtsdev); | ||
| 216 | out: | ||
| 217 | return ret; | ||
| 218 | }; | ||
| 219 | |||
| 220 | static void qaic_timesync_remove(struct mhi_device *mhi_dev) | ||
| 221 | { | ||
| 222 | struct mqts_dev *mqtsdev = dev_get_drvdata(&mhi_dev->dev); | ||
| 223 | |||
| 224 | del_timer_sync(&mqtsdev->timer); | ||
| 225 | mhi_unprepare_from_transfer(mqtsdev->mhi_dev); | ||
| 226 | kfree(mqtsdev->sync_msg); | ||
| 227 | kfree(mqtsdev); | ||
| 228 | } | ||
| 229 | |||
| 230 | static const struct mhi_device_id qaic_timesync_match_table[] = { | ||
| 231 | { .chan = "QAIC_TIMESYNC_PERIODIC"}, | ||
| 232 | {}, | ||
| 233 | }; | ||
| 234 | |||
| 235 | MODULE_DEVICE_TABLE(mhi, qaic_timesync_match_table); | ||
| 236 | |||
| 237 | static struct mhi_driver qaic_timesync_driver = { | ||
| 238 | .id_table = qaic_timesync_match_table, | ||
| 239 | .remove = qaic_timesync_remove, | ||
| 240 | .probe = qaic_timesync_probe, | ||
| 241 | .ul_xfer_cb = qaic_timesync_ul_xfer_cb, | ||
| 242 | .dl_xfer_cb = qaic_timesync_dl_xfer_cb, | ||
| 243 | .driver = { | ||
| 244 | .name = "qaic_timesync_periodic", | ||
| 245 | }, | ||
| 246 | }; | ||
| 247 | |||
| 248 | static void qaic_boot_timesync_worker(struct work_struct *work) | ||
| 249 | { | ||
| 250 | struct qts_resp *resp = container_of(work, struct qts_resp, work); | ||
| 251 | struct qts_host_time_sync_msg_data *req; | ||
| 252 | struct qts_resp_msg data = resp->data; | ||
| 253 | struct qaic_device *qdev = resp->qdev; | ||
| 254 | struct mhi_device *mhi_dev; | ||
| 255 | struct timespec64 ts; | ||
| 256 | int ret; | ||
| 257 | |||
| 258 | mhi_dev = qdev->qts_ch; | ||
| 259 | /* Queue the response message beforehand to avoid race conditions */ | ||
| 260 | ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE, &resp->data, sizeof(resp->data), MHI_EOT); | ||
| 261 | if (ret) { | ||
| 262 | kfree(resp); | ||
| 263 | dev_warn(&mhi_dev->dev, "Failed to re-queue response buffer %d\n", ret); | ||
| 264 | return; | ||
| 265 | } | ||
| 266 | |||
| 267 | switch (data.hdr.msg_type) { | ||
| 268 | case QAIC_TS_CMD_TO_HOST: | ||
| 269 | req = kzalloc(sizeof(*req), GFP_KERNEL); | ||
| 270 | if (!req) | ||
| 271 | break; | ||
| 272 | |||
| 273 | req->header = data.hdr; | ||
| 274 | req->header.msg_type = QAIC_TS_SYNC_REQ; | ||
| 275 | ktime_get_real_ts64(&ts); | ||
| 276 | req->data.tv_sec = cpu_to_le64(ts.tv_sec); | ||
| 277 | req->data.tv_usec = cpu_to_le64(div_u64(ts.tv_nsec, NSEC_PER_USEC)); | ||
| 278 | |||
| 279 | ret = mhi_queue_buf(mhi_dev, DMA_TO_DEVICE, req, sizeof(*req), MHI_EOT); | ||
| 280 | if (ret) { | ||
| 281 | kfree(req); | ||
| 282 | dev_dbg(&mhi_dev->dev, "Failed to send request message. Error %d\n", ret); | ||
| 283 | } | ||
| 284 | break; | ||
| 285 | case QAIC_TS_ACK_TO_HOST: | ||
| 286 | dev_dbg(&mhi_dev->dev, "ACK received from device\n"); | ||
| 287 | break; | ||
| 288 | default: | ||
| 289 | dev_err(&mhi_dev->dev, "Invalid message type %u.\n", data.hdr.msg_type); | ||
| 290 | } | ||
| 291 | } | ||
| 292 | |||
| 293 | static int qaic_boot_timesync_queue_resp(struct mhi_device *mhi_dev, struct qaic_device *qdev) | ||
| 294 | { | ||
| 295 | struct qts_resp *resp; | ||
| 296 | int ret; | ||
| 297 | |||
| 298 | resp = kzalloc(sizeof(*resp), GFP_KERNEL); | ||
| 299 | if (!resp) | ||
| 300 | return -ENOMEM; | ||
| 301 | |||
| 302 | resp->qdev = qdev; | ||
| 303 | INIT_WORK(&resp->work, qaic_boot_timesync_worker); | ||
| 304 | |||
| 305 | ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE, &resp->data, sizeof(resp->data), MHI_EOT); | ||
| 306 | if (ret) { | ||
| 307 | kfree(resp); | ||
| 308 | dev_warn(&mhi_dev->dev, "Failed to queue response buffer %d\n", ret); | ||
| 309 | return ret; | ||
| 310 | } | ||
| 311 | |||
| 312 | return 0; | ||
| 313 | } | ||
| 314 | |||
| 315 | static void qaic_boot_timesync_remove(struct mhi_device *mhi_dev) | ||
| 316 | { | ||
| 317 | struct qaic_device *qdev; | ||
| 318 | |||
| 319 | qdev = dev_get_drvdata(&mhi_dev->dev); | ||
| 320 | mhi_unprepare_from_transfer(qdev->qts_ch); | ||
| 321 | qdev->qts_ch = NULL; | ||
| 322 | } | ||
| 323 | |||
| 324 | static int qaic_boot_timesync_probe(struct mhi_device *mhi_dev, const struct mhi_device_id *id) | ||
| 325 | { | ||
| 326 | struct qaic_device *qdev = pci_get_drvdata(to_pci_dev(mhi_dev->mhi_cntrl->cntrl_dev)); | ||
| 327 | int ret; | ||
| 328 | |||
| 329 | ret = mhi_prepare_for_transfer(mhi_dev); | ||
| 330 | if (ret) | ||
| 331 | return ret; | ||
| 332 | |||
| 333 | qdev->qts_ch = mhi_dev; | ||
| 334 | dev_set_drvdata(&mhi_dev->dev, qdev); | ||
| 335 | |||
| 336 | ret = qaic_boot_timesync_queue_resp(mhi_dev, qdev); | ||
| 337 | if (ret) { | ||
| 338 | dev_set_drvdata(&mhi_dev->dev, NULL); | ||
| 339 | qdev->qts_ch = NULL; | ||
| 340 | mhi_unprepare_from_transfer(mhi_dev); | ||
| 341 | } | ||
| 342 | |||
| 343 | return ret; | ||
| 344 | } | ||
| 345 | |||
| 346 | static void qaic_boot_timesync_ul_xfer_cb(struct mhi_device *mhi_dev, struct mhi_result *mhi_result) | ||
| 347 | { | ||
| 348 | kfree(mhi_result->buf_addr); | ||
| 349 | } | ||
| 350 | |||
| 351 | static void qaic_boot_timesync_dl_xfer_cb(struct mhi_device *mhi_dev, struct mhi_result *mhi_result) | ||
| 352 | { | ||
| 353 | struct qts_resp *resp = container_of(mhi_result->buf_addr, struct qts_resp, data); | ||
| 354 | |||
| 355 | if (mhi_result->transaction_status || mhi_result->bytes_xferd != sizeof(resp->data)) { | ||
| 356 | kfree(resp); | ||
| 357 | return; | ||
| 358 | } | ||
| 359 | |||
| 360 | queue_work(resp->qdev->qts_wq, &resp->work); | ||
| 361 | } | ||
| 362 | |||
| 363 | static const struct mhi_device_id qaic_boot_timesync_match_table[] = { | ||
| 364 | { .chan = "QAIC_TIMESYNC"}, | ||
| 365 | {}, | ||
| 366 | }; | ||
| 367 | |||
| 368 | static struct mhi_driver qaic_boot_timesync_driver = { | ||
| 369 | .id_table = qaic_boot_timesync_match_table, | ||
| 370 | .remove = qaic_boot_timesync_remove, | ||
| 371 | .probe = qaic_boot_timesync_probe, | ||
| 372 | .ul_xfer_cb = qaic_boot_timesync_ul_xfer_cb, | ||
| 373 | .dl_xfer_cb = qaic_boot_timesync_dl_xfer_cb, | ||
| 374 | .driver = { | ||
| 375 | .name = "qaic_timesync", | ||
| 376 | }, | ||
| 377 | }; | ||
| 378 | |||
| 379 | int qaic_timesync_init(void) | ||
| 380 | { | ||
| 381 | int ret; | ||
| 382 | |||
| 383 | ret = mhi_driver_register(&qaic_timesync_driver); | ||
| 384 | if (ret) | ||
| 385 | return ret; | ||
| 386 | ret = mhi_driver_register(&qaic_boot_timesync_driver); | ||
| 387 | |||
| 388 | return ret; | ||
| 389 | } | ||
| 390 | |||
| 391 | void qaic_timesync_deinit(void) | ||
| 392 | { | ||
| 393 | mhi_driver_unregister(&qaic_boot_timesync_driver); | ||
| 394 | mhi_driver_unregister(&qaic_timesync_driver); | ||
| 395 | } | ||
| @@ -0,0 +1,11 @@ | |||
| 1 | /* SPDX-License-Identifier: GPL-2.0-only | ||
| 2 | * | ||
| 3 | * Copyright (c) 2023 Qualcomm Innovation Center, Inc. All rights reserved. | ||
| 4 | */ | ||
| 5 | |||
| 6 | #ifndef __QAIC_TIMESYNC_H__ | ||
| 7 | #define __QAIC_TIMESYNC_H__ | ||
| 8 | |||
| 9 | int qaic_timesync_init(void); | ||
| 10 | void qaic_timesync_deinit(void); | ||
| 11 | #endif /* __QAIC_TIMESYNC_H__ */ | ||
| @@ -2031,7 +2031,7 @@ static int acpi_video_bus_add(struct acpi_device *device) · 2030 unmodified lines | |||
| 2031 | * HP ZBook Fury 16 G10 requires ACPI video's child devices have _PS0 | 2031 | * HP ZBook Fury 16 G10 requires ACPI video's child devices have _PS0 |
| 2032 | * evaluated to have functional panel brightness control. | 2032 | * evaluated to have functional panel brightness control. |
| 2033 | */ | 2033 | */ |
| 2034 | acpi_device_fix_up_power_extended(device); | 2034 | acpi_device_fix_up_power_children(device); |
| 2035 | 2035 | ||
| 2036 | pr_info("%s [%s] (multi-head: %s rom: %s post: %s)\n", | 2036 | pr_info("%s [%s] (multi-head: %s rom: %s post: %s)\n", |
| 2037 | ACPI_VIDEO_DEVICE_NAME, acpi_device_bid(device), | 2037 | ACPI_VIDEO_DEVICE_NAME, acpi_device_bid(device), |
| @@ -397,6 +397,19 @@ void acpi_device_fix_up_power_extended(struct acpi_device *adev) · 396 unmodified lines | |||
| 397 | } | 397 | } |
| 398 | EXPORT_SYMBOL_GPL(acpi_device_fix_up_power_extended); | 398 | EXPORT_SYMBOL_GPL(acpi_device_fix_up_power_extended); |
| 399 | 399 | ||
| 400 | /** | ||
| 401 | * acpi_device_fix_up_power_children - Force a device's children into D0. | ||
| 402 | * @adev: Parent device object whose children's power state is to be fixed up. | ||
| 403 | * | ||
| 404 | * Call acpi_device_fix_up_power() for @adev's children so long as they | ||
| 405 | * are reported as present and enabled. | ||
| 406 | */ | ||
| 407 | void acpi_device_fix_up_power_children(struct acpi_device *adev) | ||
| 408 | { | ||
| 409 | acpi_dev_for_each_child(adev, fix_up_power_if_applicable, NULL); | ||
| 410 | } | ||
| 411 | EXPORT_SYMBOL_GPL(acpi_device_fix_up_power_children); | ||
| 412 | |||
| 400 | int acpi_device_update_power(struct acpi_device *device, int *state_p) | 413 | int acpi_device_update_power(struct acpi_device *device, int *state_p) |
| 401 | { | 414 | { |
| 402 | int state; | 415 | int state; |
| @@ -592,7 +592,7 @@ static int acpi_idle_play_dead(struct cpuidle_device *dev, int index) · 591 unmodified lines | |||
| 592 | while (1) { | 592 | while (1) { |
| 593 | 593 | ||
| 594 | if (cx->entry_method == ACPI_CSTATE_HALT) | 594 | if (cx->entry_method == ACPI_CSTATE_HALT) |
| 595 | safe_halt(); | 595 | raw_safe_halt(); |
| 596 | else if (cx->entry_method == ACPI_CSTATE_SYSTEMIO) { | 596 | else if (cx->entry_method == ACPI_CSTATE_SYSTEMIO) { |
| 597 | io_idle(cx->address); | 597 | io_idle(cx->address); |
| 598 | } else | 598 | } else |
| @@ -447,6 +447,13 @@ static const struct dmi_system_id irq1_level_low_skip_override[] = { · 446 unmodified lines | |||
| 447 | DMI_MATCH(DMI_BOARD_NAME, "B1402CBA"), | 447 | DMI_MATCH(DMI_BOARD_NAME, "B1402CBA"), |
| 448 | }, | 448 | }, |
| 449 | }, | 449 | }, |
| 450 | { | ||
| 451 | /* Asus ExpertBook B1402CVA */ | ||
| 452 | .matches = { | ||
| 453 | DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."), | ||
| 454 | DMI_MATCH(DMI_BOARD_NAME, "B1402CVA"), | ||
| 455 | }, | ||
| 456 | }, | ||
| 450 | { | 457 | { |
| 451 | /* Asus ExpertBook B1502CBA */ | 458 | /* Asus ExpertBook B1502CBA */ |
| 452 | .matches = { | 459 | .matches = { |
| @@ -82,6 +82,9 @@ static int isapnp_init_one(struct pnp_dev *idev, const struct pnp_device_id *dev · 81 unmodified lines | |||
| 82 | if (pnp_port_valid(idev, 1)) { | 82 | if (pnp_port_valid(idev, 1)) { |
| 83 | ctl_addr = devm_ioport_map(&idev->dev, | 83 | ctl_addr = devm_ioport_map(&idev->dev, |
| 84 | pnp_port_start(idev, 1), 1); | 84 | pnp_port_start(idev, 1), 1); |
| 85 | if (!ctl_addr) | ||
| 86 | return -ENOMEM; | ||
| 87 | |||
| 85 | ap->ioaddr.altstatus_addr = ctl_addr; | 88 | ap->ioaddr.altstatus_addr = ctl_addr; |
| 86 | ap->ioaddr.ctl_addr = ctl_addr; | 89 | ap->ioaddr.ctl_addr = ctl_addr; |
| 87 | ap->ops = &isapnp_port_ops; | 90 | ap->ops = &isapnp_port_ops; |
| @@ -112,10 +112,7 @@ config CFAG12864B · 111 unmodified lines | |||
| 112 | depends on X86 | 112 | depends on X86 |
| 113 | depends on FB | 113 | depends on FB |
| 114 | depends on KS0108 | 114 | depends on KS0108 |
| 115 | select FB_SYS_FILLRECT | 115 | select FB_SYSMEM_HELPERS |
| 116 | select FB_SYS_COPYAREA | ||
| 117 | select FB_SYS_IMAGEBLIT | ||
| 118 | select FB_SYS_FOPS | ||
| 119 | default n | 116 | default n |
| 120 | help | 117 | help |
| 121 | If you have a Crystalfontz 128x64 2-color LCD, cfag12864b Series, | 118 | If you have a Crystalfontz 128x64 2-color LCD, cfag12864b Series, |
| @@ -170,10 +167,7 @@ config IMG_ASCII_LCD · 48 unmodified lines | |||
| 170 | config HT16K33 | 167 | config HT16K33 |
| 171 | tristate "Holtek Ht16K33 LED controller with keyscan" | 168 | tristate "Holtek Ht16K33 LED controller with keyscan" |
| 172 | depends on FB && I2C && INPUT | 169 | depends on FB && I2C && INPUT |
| 173 | select FB_SYS_FOPS | 170 | select FB_SYSMEM_HELPERS |
| 174 | select FB_SYS_FILLRECT | ||
| 175 | select FB_SYS_COPYAREA | ||
| 176 | select FB_SYS_IMAGEBLIT | ||
| 177 | select INPUT_MATRIXKMAP | 171 | select INPUT_MATRIXKMAP |
| 178 | select FB_BACKLIGHT | 172 | select FB_BACKLIGHT |
| 179 | select NEW_LEDS | 173 | select NEW_LEDS |
| @@ -51,16 +51,15 @@ static int cfag12864bfb_mmap(struct fb_info *info, struct vm_area_struct *vma) · 50 unmodified lines | |||
| 51 | { | 51 | { |
| 52 | struct page *pages = virt_to_page(cfag12864b_buffer); | 52 | struct page *pages = virt_to_page(cfag12864b_buffer); |
| 53 | 53 | ||
| 54 | vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot); | ||
| 55 | |||
| 54 | return vm_map_pages_zero(vma, &pages, 1); | 56 | return vm_map_pages_zero(vma, &pages, 1); |
| 55 | } | 57 | } |
| 56 | 58 | ||
| 57 | static const struct fb_ops cfag12864bfb_ops = { | 59 | static const struct fb_ops cfag12864bfb_ops = { |
| 58 | .owner = THIS_MODULE, | 60 | .owner = THIS_MODULE, |
| 59 | .fb_read = fb_sys_read, | 61 | __FB_DEFAULT_SYSMEM_OPS_RDWR, |
| 60 | .fb_write = fb_sys_write, | 62 | __FB_DEFAULT_SYSMEM_OPS_DRAW, |
| 61 | .fb_fillrect = sys_fillrect, | ||
| 62 | .fb_copyarea = sys_copyarea, | ||
| 63 | .fb_imageblit = sys_imageblit, | ||
| 64 | .fb_mmap = cfag12864bfb_mmap, | 63 | .fb_mmap = cfag12864bfb_mmap, |
| 65 | }; | 64 | }; |
| 66 | 65 | ||
| @@ -72,6 +71,7 @@ static int cfag12864bfb_probe(struct platform_device *device) · 5 unmodified lines | |||
| 72 | if (!info) | 71 | if (!info) |
| 73 | goto none; | 72 | goto none; |
| 74 | 73 | ||
| 74 | info->flags = FBINFO_VIRTFB; | ||
| 75 | info->screen_buffer = cfag12864b_buffer; | 75 | info->screen_buffer = cfag12864b_buffer; |
| 76 | info->screen_size = CFAG12864B_SIZE; | 76 | info->screen_size = CFAG12864B_SIZE; |
| 77 | info->fbops = &cfag12864bfb_ops; | 77 | info->fbops = &cfag12864bfb_ops; |
| @@ -351,17 +351,16 @@ static int ht16k33_mmap(struct fb_info *info, struct vm_area_struct *vma) · 350 unmodified lines | |||
| 351 | struct ht16k33_priv *priv = info->par; | 351 | struct ht16k33_priv *priv = info->par; |
| 352 | struct page *pages = virt_to_page(priv->fbdev.buffer); | 352 | struct page *pages = virt_to_page(priv->fbdev.buffer); |
| 353 | 353 | ||
| 354 | vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot); | ||
| 355 | |||
| 354 | return vm_map_pages_zero(vma, &pages, 1); | 356 | return vm_map_pages_zero(vma, &pages, 1); |
| 355 | } | 357 | } |
| 356 | 358 | ||
| 357 | static const struct fb_ops ht16k33_fb_ops = { | 359 | static const struct fb_ops ht16k33_fb_ops = { |
| 358 | .owner = THIS_MODULE, | 360 | .owner = THIS_MODULE, |
| 359 | .fb_read = fb_sys_read, | 361 | __FB_DEFAULT_SYSMEM_OPS_RDWR, |
| 360 | .fb_write = fb_sys_write, | ||
| 361 | .fb_blank = ht16k33_blank, | 362 | .fb_blank = ht16k33_blank, |
| 362 | .fb_fillrect = sys_fillrect, | 363 | __FB_DEFAULT_SYSMEM_OPS_DRAW, |
| 363 | .fb_copyarea = sys_copyarea, | ||
| 364 | .fb_imageblit = sys_imageblit, | ||
| 365 | .fb_mmap = ht16k33_mmap, | 364 | .fb_mmap = ht16k33_mmap, |
| 366 | }; | 365 | }; |
| 367 | 366 | ||
| @@ -640,6 +639,7 @@ static int ht16k33_fbdev_probe(struct device *dev, struct ht16k33_priv *priv, · 272 unmodified lines | |||
| 640 | 639 | ||
| 641 | INIT_DELAYED_WORK(&priv->work, ht16k33_fb_update); | 640 | INIT_DELAYED_WORK(&priv->work, ht16k33_fb_update); |
| 642 | fbdev->info->fbops = &ht16k33_fb_ops; | 641 | fbdev->info->fbops = &ht16k33_fb_ops; |
| 642 | fbdev->info->flags |= FBINFO_VIRTFB; | ||
| 643 | fbdev->info->screen_buffer = fbdev->buffer; | 643 | fbdev->info->screen_buffer = fbdev->buffer; |
| 644 | fbdev->info->screen_size = HT16K33_FB_SIZE; | 644 | fbdev->info->screen_size = HT16K33_FB_SIZE; |
| 645 | fbdev->info->fix = ht16k33_fb_fix; | 645 | fbdev->info->fix = ht16k33_fb_fix; |
| @@ -67,6 +67,7 @@ struct nbd_sock { · 66 unmodified lines | |||
| 67 | struct recv_thread_args { | 67 | struct recv_thread_args { |
| 68 | struct work_struct work; | 68 | struct work_struct work; |
| 69 | struct nbd_device *nbd; | 69 | struct nbd_device *nbd; |
| 70 | struct nbd_sock *nsock; | ||
| 70 | int index; | 71 | int index; |
| 71 | }; | 72 | }; |
| 72 | 73 | ||
| @@ -395,6 +396,22 @@ static u32 req_to_nbd_cmd_type(struct request *req) · 322 unmodified lines | |||
| 395 | } | 396 | } |
| 396 | } | 397 | } |
| 397 | 398 | ||
| 399 | static struct nbd_config *nbd_get_config_unlocked(struct nbd_device *nbd) | ||
| 400 | { | ||
| 401 | if (refcount_inc_not_zero(&nbd->config_refs)) { | ||
| 402 | /* | ||
| 403 | * Add smp_mb__after_atomic to ensure that reading nbd->config_refs | ||
| 404 | * and reading nbd->config is ordered. The pair is the barrier in | ||
| 405 | * nbd_alloc_and_init_config(), avoid nbd->config_refs is set | ||
| 406 | * before nbd->config. | ||
| 407 | */ | ||
| 408 | smp_mb__after_atomic(); | ||
| 409 | return nbd->config; | ||
| 410 | } | ||
| 411 | |||
| 412 | return NULL; | ||
| 413 | } | ||
| 414 | |||
| 398 | static enum blk_eh_timer_return nbd_xmit_timeout(struct request *req) | 415 | static enum blk_eh_timer_return nbd_xmit_timeout(struct request *req) |
| 399 | { | 416 | { |
| 400 | struct nbd_cmd *cmd = blk_mq_rq_to_pdu(req); | 417 | struct nbd_cmd *cmd = blk_mq_rq_to_pdu(req); |
| @@ -409,13 +426,13 @@ static enum blk_eh_timer_return nbd_xmit_timeout(struct request *req) · 8 unmodified lines | |||
| 409 | return BLK_EH_DONE; | 426 | return BLK_EH_DONE; |
| 410 | } | 427 | } |
| 411 | 428 | ||
| 412 | if (!refcount_inc_not_zero(&nbd->config_refs)) { | 429 | config = nbd_get_config_unlocked(nbd); |
| 430 | if (!config) { | ||
| 413 | cmd->status = BLK_STS_TIMEOUT; | 431 | cmd->status = BLK_STS_TIMEOUT; |
| 414 | __clear_bit(NBD_CMD_INFLIGHT, &cmd->flags); | 432 | __clear_bit(NBD_CMD_INFLIGHT, &cmd->flags); |
| 415 | mutex_unlock(&cmd->lock); | 433 | mutex_unlock(&cmd->lock); |
| 416 | goto done; | 434 | goto done; |
| 417 | } | 435 | } |
| 418 | config = nbd->config; | ||
| 419 | 436 | ||
| 420 | if (config->num_connections > 1 || | 437 | if (config->num_connections > 1 || |
| 421 | (config->num_connections == 1 && nbd->tag_set.timeout)) { | 438 | (config->num_connections == 1 && nbd->tag_set.timeout)) { |
| @@ -489,15 +506,9 @@ done: · 67 unmodified lines | |||
| 489 | return BLK_EH_DONE; | 506 | return BLK_EH_DONE; |
| 490 | } | 507 | } |
| 491 | 508 | ||
| 492 | /* | 509 | static int __sock_xmit(struct nbd_device *nbd, struct socket *sock, int send, |
| 493 | * Send or receive packet. Return a positive value on success and | 510 | struct iov_iter *iter, int msg_flags, int *sent) |
| 494 | * negtive value on failue, and never return 0. | ||
| 495 | */ | ||
| 496 | static int sock_xmit(struct nbd_device *nbd, int index, int send, | ||
| 497 | struct iov_iter *iter, int msg_flags, int *sent) | ||
| 498 | { | 511 | { |
| 499 | struct nbd_config *config = nbd->config; | ||
| 500 | struct socket *sock = config->socks[index]->sock; | ||
| 501 | int result; | 512 | int result; |
| 502 | struct msghdr msg; | 513 | struct msghdr msg; |
| 503 | unsigned int noreclaim_flag; | 514 | unsigned int noreclaim_flag; |
| @@ -540,6 +551,19 @@ static int sock_xmit(struct nbd_device *nbd, int index, int send, · 36 unmodified lines | |||
| 540 | return result; | 551 | return result; |
| 541 | } | 552 | } |
| 542 | 553 | ||
| 554 | /* | ||
| 555 | * Send or receive packet. Return a positive value on success and | ||
| 556 | * negtive value on failure, and never return 0. | ||
| 557 | */ | ||
| 558 | static int sock_xmit(struct nbd_device *nbd, int index, int send, | ||
| 559 | struct iov_iter *iter, int msg_flags, int *sent) | ||
| 560 | { | ||
| 561 | struct nbd_config *config = nbd->config; | ||
| 562 | struct socket *sock = config->socks[index]->sock; | ||
| 563 | |||
| 564 | return __sock_xmit(nbd, sock, send, iter, msg_flags, sent); | ||
| 565 | } | ||
| 566 | |||
| 543 | /* | 567 | /* |
| 544 | * Different settings for sk->sk_sndtimeo can result in different return values | 568 | * Different settings for sk->sk_sndtimeo can result in different return values |
| 545 | * if there is a signal pending when we enter sendmsg, because reasons? | 569 | * if there is a signal pending when we enter sendmsg, because reasons? |
| @@ -696,7 +720,7 @@ out: · 150 unmodified lines | |||
| 696 | return 0; | 720 | return 0; |
| 697 | } | 721 | } |
| 698 | 722 | ||
| 699 | static int nbd_read_reply(struct nbd_device *nbd, int index, | 723 | static int nbd_read_reply(struct nbd_device *nbd, struct socket *sock, |
| 700 | struct nbd_reply *reply) | 724 | struct nbd_reply *reply) |
| 701 | { | 725 | { |
| 702 | struct kvec iov = {.iov_base = reply, .iov_len = sizeof(*reply)}; | 726 | struct kvec iov = {.iov_base = reply, .iov_len = sizeof(*reply)}; |
| @@ -705,7 +729,7 @@ static int nbd_read_reply(struct nbd_device *nbd, int index, · 2 unmodified lines | |||
| 705 | 729 | ||
| 706 | reply->magic = 0; | 730 | reply->magic = 0; |
| 707 | iov_iter_kvec(&to, ITER_DEST, &iov, 1, sizeof(*reply)); | 731 | iov_iter_kvec(&to, ITER_DEST, &iov, 1, sizeof(*reply)); |
| 708 | result = sock_xmit(nbd, index, 0, &to, MSG_WAITALL, NULL); | 732 | result = __sock_xmit(nbd, sock, 0, &to, MSG_WAITALL, NULL); |
| 709 | if (result < 0) { | 733 | if (result < 0) { |
| 710 | if (!nbd_disconnected(nbd->config)) | 734 | if (!nbd_disconnected(nbd->config)) |
| 711 | dev_err(disk_to_dev(nbd->disk), | 735 | dev_err(disk_to_dev(nbd->disk), |
| @@ -829,14 +853,14 @@ static void recv_work(struct work_struct *work) · 117 unmodified lines | |||
| 829 | struct nbd_device *nbd = args->nbd; | 853 | struct nbd_device *nbd = args->nbd; |
| 830 | struct nbd_config *config = nbd->config; | 854 | struct nbd_config *config = nbd->config; |
| 831 | struct request_queue *q = nbd->disk->queue; | 855 | struct request_queue *q = nbd->disk->queue; |
| 832 | struct nbd_sock *nsock; | 856 | struct nbd_sock *nsock = args->nsock; |
| 833 | struct nbd_cmd *cmd; | 857 | struct nbd_cmd *cmd; |
| 834 | struct request *rq; | 858 | struct request *rq; |
| 835 | 859 | ||
| 836 | while (1) { | 860 | while (1) { |
| 837 | struct nbd_reply reply; | 861 | struct nbd_reply reply; |
| 838 | 862 | ||
| 839 | if (nbd_read_reply(nbd, args->index, &reply)) | 863 | if (nbd_read_reply(nbd, nsock->sock, &reply)) |
| 840 | break; | 864 | break; |
| 841 | 865 | ||
| 842 | /* | 866 | /* |
| @@ -871,7 +895,6 @@ static void recv_work(struct work_struct *work) · 28 unmodified lines | |||
| 871 | percpu_ref_put(&q->q_usage_counter); | 895 | percpu_ref_put(&q->q_usage_counter); |
| 872 | } | 896 | } |
| 873 | 897 | ||
| 874 | nsock = config->socks[args->index]; | ||
| 875 | mutex_lock(&nsock->tx_lock); | 898 | mutex_lock(&nsock->tx_lock); |
| 876 | nbd_mark_nsock_dead(nbd, nsock, 1); | 899 | nbd_mark_nsock_dead(nbd, nsock, 1); |
| 877 | mutex_unlock(&nsock->tx_lock); | 900 | mutex_unlock(&nsock->tx_lock); |
| @@ -977,12 +1000,12 @@ static int nbd_handle_cmd(struct nbd_cmd *cmd, int index) · 99 unmodified lines | |||
| 977 | struct nbd_sock *nsock; | 1000 | struct nbd_sock *nsock; |
| 978 | int ret; | 1001 | int ret; |
| 979 | 1002 | ||
| 980 | if (!refcount_inc_not_zero(&nbd->config_refs)) { | 1003 | config = nbd_get_config_unlocked(nbd); |
| 1004 | if (!config) { | ||
| 981 | dev_err_ratelimited(disk_to_dev(nbd->disk), | 1005 | dev_err_ratelimited(disk_to_dev(nbd->disk), |
| 982 | "Socks array is empty\n"); | 1006 | "Socks array is empty\n"); |
| 983 | return -EINVAL; | 1007 | return -EINVAL; |
| 984 | } | 1008 | } |
| 985 | config = nbd->config; | ||
| 986 | 1009 | ||
| 987 | if (index >= config->num_connections) { | 1010 | if (index >= config->num_connections) { |
| 988 | dev_err_ratelimited(disk_to_dev(nbd->disk), | 1011 | dev_err_ratelimited(disk_to_dev(nbd->disk), |
| @@ -1215,6 +1238,7 @@ static int nbd_reconnect_socket(struct nbd_device *nbd, unsigned long arg) · 226 unmodified lines | |||
| 1215 | INIT_WORK(&args->work, recv_work); | 1238 | INIT_WORK(&args->work, recv_work); |
| 1216 | args->index = i; | 1239 | args->index = i; |
| 1217 | args->nbd = nbd; | 1240 | args->nbd = nbd; |
| 1241 | args->nsock = nsock; | ||
| 1218 | nsock->cookie++; | 1242 | nsock->cookie++; |
| 1219 | mutex_unlock(&nsock->tx_lock); | 1243 | mutex_unlock(&nsock->tx_lock); |
| 1220 | sockfd_put(old); | 1244 | sockfd_put(old); |
| @@ -1397,6 +1421,7 @@ static int nbd_start_device(struct nbd_device *nbd) · 176 unmodified lines | |||
| 1397 | refcount_inc(&nbd->config_refs); | 1421 | refcount_inc(&nbd->config_refs); |
| 1398 | INIT_WORK(&args->work, recv_work); | 1422 | INIT_WORK(&args->work, recv_work); |
| 1399 | args->nbd = nbd; | 1423 | args->nbd = nbd; |
| 1424 | args->nsock = config->socks[i]; | ||
| 1400 | args->index = i; | 1425 | args->index = i; |
| 1401 | queue_work(nbd->recv_workq, &args->work); | 1426 | queue_work(nbd->recv_workq, &args->work); |
| 1402 | } | 1427 | } |
| @@ -1530,17 +1555,20 @@ static int nbd_ioctl(struct block_device *bdev, blk_mode_t mode, · 127 unmodified lines | |||
| 1530 | return error; | 1555 | return error; |
| 1531 | } | 1556 | } |
| 1532 | 1557 | ||
| 1533 | static struct nbd_config *nbd_alloc_config(void) | 1558 | static int nbd_alloc_and_init_config(struct nbd_device *nbd) |
| 1534 | { | 1559 | { |
| 1535 | struct nbd_config *config; | 1560 | struct nbd_config *config; |
| 1536 | 1561 | ||
| 1562 | if (WARN_ON(nbd->config)) | ||
| 1563 | return -EINVAL; | ||
| 1564 | |||
| 1537 | if (!try_module_get(THIS_MODULE)) | 1565 | if (!try_module_get(THIS_MODULE)) |
| 1538 | return ERR_PTR(-ENODEV); | 1566 | return -ENODEV; |
| 1539 | 1567 | ||
| 1540 | config = kzalloc(sizeof(struct nbd_config), GFP_NOFS); | 1568 | config = kzalloc(sizeof(struct nbd_config), GFP_NOFS); |
| 1541 | if (!config) { | 1569 | if (!config) { |
| 1542 | module_put(THIS_MODULE); | 1570 | module_put(THIS_MODULE); |
| 1543 | return ERR_PTR(-ENOMEM); | 1571 | return -ENOMEM; |
| 1544 | } | 1572 | } |
| 1545 | 1573 | ||
| 1546 | atomic_set(&config->recv_threads, 0); | 1574 | atomic_set(&config->recv_threads, 0); |
| @@ -1548,12 +1576,24 @@ static struct nbd_config *nbd_alloc_config(void) · 1 unmodified line | |||
| 1548 | init_waitqueue_head(&config->conn_wait); | 1576 | init_waitqueue_head(&config->conn_wait); |
| 1549 | config->blksize_bits = NBD_DEF_BLKSIZE_BITS; | 1577 | config->blksize_bits = NBD_DEF_BLKSIZE_BITS; |
| 1550 | atomic_set(&config->live_connections, 0); | 1578 | atomic_set(&config->live_connections, 0); |
| 1551 | return config; | 1579 | |
| 1580 | nbd->config = config; | ||
| 1581 | /* | ||
| 1582 | * Order refcount_set(&nbd->config_refs, 1) and nbd->config assignment, | ||
| 1583 | * its pair is the barrier in nbd_get_config_unlocked(). | ||
| 1584 | * So nbd_get_config_unlocked() won't see nbd->config as null after | ||
| 1585 | * refcount_inc_not_zero() succeed. | ||
| 1586 | */ | ||
| 1587 | smp_mb__before_atomic(); | ||
| 1588 | refcount_set(&nbd->config_refs, 1); | ||
| 1589 | |||
| 1590 | return 0; | ||
| 1552 | } | 1591 | } |
| 1553 | 1592 | ||
| 1554 | static int nbd_open(struct gendisk *disk, blk_mode_t mode) | 1593 | static int nbd_open(struct gendisk *disk, blk_mode_t mode) |
| 1555 | { | 1594 | { |
| 1556 | struct nbd_device *nbd; | 1595 | struct nbd_device *nbd; |
| 1596 | struct nbd_config *config; | ||
| 1557 | int ret = 0; | 1597 | int ret = 0; |
| 1558 | 1598 | ||
| 1559 | mutex_lock(&nbd_index_mutex); | 1599 | mutex_lock(&nbd_index_mutex); |
| @@ -1566,27 +1606,25 @@ static int nbd_open(struct gendisk *disk, blk_mode_t mode) · 6 unmodified lines | |||
| 1566 | ret = -ENXIO; | 1606 | ret = -ENXIO; |
| 1567 | goto out; | 1607 | goto out; |
| 1568 | } | 1608 | } |
| 1569 | if (!refcount_inc_not_zero(&nbd->config_refs)) { | ||
| 1570 | struct nbd_config *config; | ||
| 1571 | 1609 | ||
| 1610 | config = nbd_get_config_unlocked(nbd); | ||
| 1611 | if (!config) { | ||
| 1572 | mutex_lock(&nbd->config_lock); | 1612 | mutex_lock(&nbd->config_lock); |
| 1573 | if (refcount_inc_not_zero(&nbd->config_refs)) { | 1613 | if (refcount_inc_not_zero(&nbd->config_refs)) { |
| 1574 | mutex_unlock(&nbd->config_lock); | 1614 | mutex_unlock(&nbd->config_lock); |
| 1575 | goto out; | 1615 | goto out; |
| 1576 | } | 1616 | } |
| 1577 | config = nbd_alloc_config(); | 1617 | ret = nbd_alloc_and_init_config(nbd); |
| 1578 | if (IS_ERR(config)) { | 1618 | if (ret) { |
| 1579 | ret = PTR_ERR(config); | ||
| 1580 | mutex_unlock(&nbd->config_lock); | 1619 | mutex_unlock(&nbd->config_lock); |
| 1581 | goto out; | 1620 | goto out; |
| 1582 | } | 1621 | } |
| 1583 | nbd->config = config; | 1622 | |
| 1584 | refcount_set(&nbd->config_refs, 1); | ||
| 1585 | refcount_inc(&nbd->refs); | 1623 | refcount_inc(&nbd->refs); |
| 1586 | mutex_unlock(&nbd->config_lock); | 1624 | mutex_unlock(&nbd->config_lock); |
| 1587 | if (max_part) | 1625 | if (max_part) |
| 1588 | set_bit(GD_NEED_PART_SCAN, &disk->state); | 1626 | set_bit(GD_NEED_PART_SCAN, &disk->state); |
| 1589 | } else if (nbd_disconnected(nbd->config)) { | 1627 | } else if (nbd_disconnected(config)) { |
| 1590 | if (max_part) | 1628 | if (max_part) |
| 1591 | set_bit(GD_NEED_PART_SCAN, &disk->state); | 1629 | set_bit(GD_NEED_PART_SCAN, &disk->state); |
| 1592 | } | 1630 | } |
| @@ -1990,22 +2028,17 @@ again: · 397 unmodified lines | |||
| 1990 | pr_err("nbd%d already in use\n", index); | 2028 | pr_err("nbd%d already in use\n", index); |
| 1991 | return -EBUSY; | 2029 | return -EBUSY; |
| 1992 | } | 2030 | } |
| 1993 | if (WARN_ON(nbd->config)) { | 2031 | |
| 1994 | mutex_unlock(&nbd->config_lock); | 2032 | ret = nbd_alloc_and_init_config(nbd); |
| 1995 | nbd_put(nbd); | 2033 | if (ret) { |
| 1996 | return -EINVAL; | ||
| 1997 | } | ||
| 1998 | config = nbd_alloc_config(); | ||
| 1999 | if (IS_ERR(config)) { | ||
| 2000 | mutex_unlock(&nbd->config_lock); | 2034 | mutex_unlock(&nbd->config_lock); |
| 2001 | nbd_put(nbd); | 2035 | nbd_put(nbd); |
| 2002 | pr_err("couldn't allocate config\n"); | 2036 | pr_err("couldn't allocate config\n"); |
| 2003 | return PTR_ERR(config); | 2037 | return ret; |
| 2004 | } | 2038 | } |
| 2005 | nbd->config = config; | ||
| 2006 | refcount_set(&nbd->config_refs, 1); | ||
| 2007 | set_bit(NBD_RT_BOUND, &config->runtime_flags); | ||
| 2008 | 2039 | ||
| 2040 | config = nbd->config; | ||
| 2041 | set_bit(NBD_RT_BOUND, &config->runtime_flags); | ||
| 2009 | ret = nbd_genl_size_set(info, nbd); | 2042 | ret = nbd_genl_size_set(info, nbd); |
| 2010 | if (ret) | 2043 | if (ret) |
| 2011 | goto out; | 2044 | goto out; |
| @@ -2208,7 +2241,8 @@ static int nbd_genl_reconfigure(struct sk_buff *skb, struct genl_info *info) · 196 unmodified lines | |||
| 2208 | } | 2241 | } |
| 2209 | mutex_unlock(&nbd_index_mutex); | 2242 | mutex_unlock(&nbd_index_mutex); |
| 2210 | 2243 | ||
| 2211 | if (!refcount_inc_not_zero(&nbd->config_refs)) { | 2244 | config = nbd_get_config_unlocked(nbd); |
| 2245 | if (!config) { | ||
| 2212 | dev_err(nbd_to_dev(nbd), | 2246 | dev_err(nbd_to_dev(nbd), |
| 2213 | "not configured, cannot reconfigure\n"); | 2247 | "not configured, cannot reconfigure\n"); |
| 2214 | nbd_put(nbd); | 2248 | nbd_put(nbd); |
| @@ -2216,7 +2250,6 @@ static int nbd_genl_reconfigure(struct sk_buff *skb, struct genl_info *info) · 1 unmodified line | |||
| 2216 | } | 2250 | } |
| 2217 | 2251 | ||
| 2218 | mutex_lock(&nbd->config_lock); | 2252 | mutex_lock(&nbd->config_lock); |
| 2219 | config = nbd->config; | ||
| 2220 | if (!test_bit(NBD_RT_BOUND, &config->runtime_flags) || | 2253 | if (!test_bit(NBD_RT_BOUND, &config->runtime_flags) || |
| 2221 | !nbd->pid) { | 2254 | !nbd->pid) { |
| 2222 | dev_err(nbd_to_dev(nbd), | 2255 | dev_err(nbd_to_dev(nbd), |
| @@ -1464,19 +1464,13 @@ blk_status_t null_process_cmd(struct nullb_cmd *cmd, enum req_op op, · 1463 unmodified lines | |||
| 1464 | return BLK_STS_OK; | 1464 | return BLK_STS_OK; |
| 1465 | } | 1465 | } |
| 1466 | 1466 | ||
| 1467 | static blk_status_t null_handle_cmd(struct nullb_cmd *cmd, sector_t sector, | 1467 | static void null_handle_cmd(struct nullb_cmd *cmd, sector_t sector, |
| 1468 | sector_t nr_sectors, enum req_op op) | 1468 | sector_t nr_sectors, enum req_op op) |
| 1469 | { | 1469 | { |
| 1470 | struct nullb_device *dev = cmd->nq->dev; | 1470 | struct nullb_device *dev = cmd->nq->dev; |
| 1471 | struct nullb *nullb = dev->nullb; | 1471 | struct nullb *nullb = dev->nullb; |
| 1472 | blk_status_t sts; | 1472 | blk_status_t sts; |
| 1473 | 1473 | ||
| 1474 | if (test_bit(NULLB_DEV_FL_THROTTLED, &dev->flags)) { | ||
| 1475 | sts = null_handle_throttled(cmd); | ||
| 1476 | if (sts != BLK_STS_OK) | ||
| 1477 | return sts; | ||
| 1478 | } | ||
| 1479 | |||
| 1480 | if (op == REQ_OP_FLUSH) { | 1474 | if (op == REQ_OP_FLUSH) { |
| 1481 | cmd->error = errno_to_blk_status(null_handle_flush(nullb)); | 1475 | cmd->error = errno_to_blk_status(null_handle_flush(nullb)); |
| 1482 | goto out; | 1476 | goto out; |
| @@ -1493,7 +1487,6 @@ static blk_status_t null_handle_cmd(struct nullb_cmd *cmd, sector_t sector, · 10 unmodified lines | |||
| 1493 | 1487 | ||
| 1494 | out: | 1488 | out: |
| 1495 | nullb_complete_cmd(cmd); | 1489 | nullb_complete_cmd(cmd); |
| 1496 | return BLK_STS_OK; | ||
| 1497 | } | 1490 | } |
| 1498 | 1491 | ||
| 1499 | static enum hrtimer_restart nullb_bwtimer_fn(struct hrtimer *timer) | 1492 | static enum hrtimer_restart nullb_bwtimer_fn(struct hrtimer *timer) |
| @@ -1724,8 +1717,6 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx, · 224 unmodified lines | |||
| 1724 | cmd->fake_timeout = should_timeout_request(rq) || | 1717 | cmd->fake_timeout = should_timeout_request(rq) || |
| 1725 | blk_should_fake_timeout(rq->q); | 1718 | blk_should_fake_timeout(rq->q); |
| 1726 | 1719 | ||
| 1727 | blk_mq_start_request(rq); | ||
| 1728 | |||
| 1729 | if (should_requeue_request(rq)) { | 1720 | if (should_requeue_request(rq)) { |
| 1730 | /* | 1721 | /* |
| 1731 | * Alternate between hitting the core BUSY path, and the | 1722 | * Alternate between hitting the core BUSY path, and the |
| @@ -1738,6 +1729,15 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx, · 6 unmodified lines | |||
| 1738 | return BLK_STS_OK; | 1729 | return BLK_STS_OK; |
| 1739 | } | 1730 | } |
| 1740 | 1731 | ||
| 1732 | if (test_bit(NULLB_DEV_FL_THROTTLED, &nq->dev->flags)) { | ||
| 1733 | blk_status_t sts = null_handle_throttled(cmd); | ||
| 1734 | |||
| 1735 | if (sts != BLK_STS_OK) | ||
| 1736 | return sts; | ||
| 1737 | } | ||
| 1738 | |||
| 1739 | blk_mq_start_request(rq); | ||
| 1740 | |||
| 1741 | if (is_poll) { | 1741 | if (is_poll) { |
| 1742 | spin_lock(&nq->poll_lock); | 1742 | spin_lock(&nq->poll_lock); |
| 1743 | list_add_tail(&rq->queuelist, &nq->poll_list); | 1743 | list_add_tail(&rq->queuelist, &nq->poll_list); |
| @@ -1747,7 +1747,8 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx, · 3 unmodified lines | |||
| 1747 | if (cmd->fake_timeout) | 1747 | if (cmd->fake_timeout) |
| 1748 | return BLK_STS_OK; | 1748 | return BLK_STS_OK; |
| 1749 | 1749 | ||
| 1750 | return null_handle_cmd(cmd, sector, nr_sectors, req_op(rq)); | 1750 | null_handle_cmd(cmd, sector, nr_sectors, req_op(rq)); |
| 1751 | return BLK_STS_OK; | ||
| 1751 | } | 1752 | } |
| 1752 | 1753 | ||
| 1753 | static void null_queue_rqs(struct request **rqlist) | 1754 | static void null_queue_rqs(struct request **rqlist) |
| @@ -1,12 +1,6 @@ | |||
| 1 | # SPDX-License-Identifier: GPL-2.0 | 1 | # SPDX-License-Identifier: GPL-2.0 |
| 2 | agpgart-y := backend.o generic.o isoch.o | 2 | agpgart-y := backend.o generic.o isoch.o |
| 3 | 3 | ||
| 4 | ifeq ($(CONFIG_DRM_LEGACY),y) | ||
| 5 | agpgart-$(CONFIG_COMPAT) += compat_ioctl.o | ||
| 6 | agpgart-y += frontend.o | ||
| 7 | endif | ||
| 8 | |||
| 9 | |||
| 10 | obj-$(CONFIG_AGP) += agpgart.o | 4 | obj-$(CONFIG_AGP) += agpgart.o |
| 11 | obj-$(CONFIG_AGP_ALI) += ali-agp.o | 5 | obj-$(CONFIG_AGP_ALI) += ali-agp.o |
| 12 | obj-$(CONFIG_AGP_ATI) += ati-agp.o | 6 | obj-$(CONFIG_AGP_ATI) += ati-agp.o |
| @@ -185,15 +185,6 @@ void agp_put_bridge(struct agp_bridge_data *bridge); · 184 unmodified lines | |||
| 185 | int agp_add_bridge(struct agp_bridge_data *bridge); | 185 | int agp_add_bridge(struct agp_bridge_data *bridge); |
| 186 | void agp_remove_bridge(struct agp_bridge_data *bridge); | 186 | void agp_remove_bridge(struct agp_bridge_data *bridge); |
| 187 | 187 | ||
| 188 | /* Frontend routines. */ | ||
| 189 | #if IS_ENABLED(CONFIG_DRM_LEGACY) | ||
| 190 | int agp_frontend_initialize(void); | ||
| 191 | void agp_frontend_cleanup(void); | ||
| 192 | #else | ||
| 193 | static inline int agp_frontend_initialize(void) { return 0; } | ||
| 194 | static inline void agp_frontend_cleanup(void) {} | ||
| 195 | #endif | ||
| 196 | |||
| 197 | /* Generic routines. */ | 188 | /* Generic routines. */ |
| 198 | void agp_generic_enable(struct agp_bridge_data *bridge, u32 mode); | 189 | void agp_generic_enable(struct agp_bridge_data *bridge, u32 mode); |
| 199 | int agp_generic_create_gatt_table(struct agp_bridge_data *bridge); | 190 | int agp_generic_create_gatt_table(struct agp_bridge_data *bridge); |
| @@ -293,13 +293,6 @@ int agp_add_bridge(struct agp_bridge_data *bridge) · 292 unmodified lines | |||
| 293 | } | 293 | } |
| 294 | 294 | ||
| 295 | if (list_empty(&agp_bridges)) { | 295 | if (list_empty(&agp_bridges)) { |
| 296 | error = agp_frontend_initialize(); | ||
| 297 | if (error) { | ||
| 298 | dev_info(&bridge->dev->dev, | ||
| 299 | "agp_frontend_initialize() failed\n"); | ||
| 300 | goto frontend_err; | ||
| 301 | } | ||
| 302 | |||
| 303 | dev_info(&bridge->dev->dev, "AGP aperture is %dM @ 0x%lx\n", | 296 | dev_info(&bridge->dev->dev, "AGP aperture is %dM @ 0x%lx\n", |
| 304 | bridge->driver->fetch_size(), bridge->gart_bus_addr); | 297 | bridge->driver->fetch_size(), bridge->gart_bus_addr); |
| 305 | 298 | ||
| @@ -308,8 +301,6 @@ int agp_add_bridge(struct agp_bridge_data *bridge) · 2 unmodified lines | |||
| 308 | list_add(&bridge->list, &agp_bridges); | 301 | list_add(&bridge->list, &agp_bridges); |
| 309 | return 0; | 302 | return 0; |
| 310 | 303 | ||
| 311 | frontend_err: | ||
| 312 | agp_backend_cleanup(bridge); | ||
| 313 | err_out: | 304 | err_out: |
| 314 | module_put(bridge->driver->owner); | 305 | module_put(bridge->driver->owner); |
| 315 | err_put_bridge: | 306 | err_put_bridge: |
| @@ -323,8 +314,6 @@ void agp_remove_bridge(struct agp_bridge_data *bridge) · 7 unmodified lines | |||
| 323 | { | 314 | { |
| 324 | agp_backend_cleanup(bridge); | 315 | agp_backend_cleanup(bridge); |
| 325 | list_del(&bridge->list); | 316 | list_del(&bridge->list); |
| 326 | if (list_empty(&agp_bridges)) | ||
| 327 | agp_frontend_cleanup(); | ||
| 328 | module_put(bridge->driver->owner); | 317 | module_put(bridge->driver->owner); |
| 329 | } | 318 | } |
| 330 | EXPORT_SYMBOL_GPL(agp_remove_bridge); | 319 | EXPORT_SYMBOL_GPL(agp_remove_bridge); |
| @@ -1,291 +0,0 @@ | |||
| 1 | /* | ||
| 2 | * AGPGART driver frontend compatibility ioctls | ||
| 3 | * Copyright (C) 2004 Silicon Graphics, Inc. | ||
| 4 | * Copyright (C) 2002-2003 Dave Jones | ||
| 5 | * Copyright (C) 1999 Jeff Hartmann | ||
| 6 | * Copyright (C) 1999 Precision Insight, Inc. | ||
| 7 | * Copyright (C) 1999 Xi Graphics, Inc. | ||
| 8 | * | ||
| 9 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 10 | * copy of this software and associated documentation files (the "Software"), | ||
| 11 | * to deal in the Software without restriction, including without limitation | ||
| 12 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 13 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 14 | * Software is furnished to do so, subject to the following conditions: | ||
| 15 | * | ||
| 16 | * The above copyright notice and this permission notice shall be included | ||
| 17 | * in all copies or substantial portions of the Software. | ||
| 18 | * | ||
| 19 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS | ||
| 20 | * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 21 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 22 | * JEFF HARTMANN, OR ANY OTHER CONTRIBUTORS BE LIABLE FOR ANY CLAIM, | ||
| 23 | * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR | ||
| 24 | * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE | ||
| 25 | * OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. | ||
| 26 | * | ||
| 27 | */ | ||
| 28 | |||
| 29 | #include <linux/kernel.h> | ||
| 30 | #include <linux/pci.h> | ||
| 31 | #include <linux/fs.h> | ||
| 32 | #include <linux/agpgart.h> | ||
| 33 | #include <linux/slab.h> | ||
| 34 | #include <linux/uaccess.h> | ||
| 35 | #include "agp.h" | ||
| 36 | #include "compat_ioctl.h" | ||
| 37 | |||
| 38 | static int compat_agpioc_info_wrap(struct agp_file_private *priv, void __user *arg) | ||
| 39 | { | ||
| 40 | struct agp_info32 userinfo; | ||
| 41 | struct agp_kern_info kerninfo; | ||
| 42 | |||
| 43 | agp_copy_info(agp_bridge, &kerninfo); | ||
| 44 | |||
| 45 | userinfo.version.major = kerninfo.version.major; | ||
| 46 | userinfo.version.minor = kerninfo.version.minor; | ||
| 47 | userinfo.bridge_id = kerninfo.device->vendor | | ||
| 48 | (kerninfo.device->device << 16); | ||
| 49 | userinfo.agp_mode = kerninfo.mode; | ||
| 50 | userinfo.aper_base = (compat_long_t)kerninfo.aper_base; | ||
| 51 | userinfo.aper_size = kerninfo.aper_size; | ||
| 52 | userinfo.pg_total = userinfo.pg_system = kerninfo.max_memory; | ||
| 53 | userinfo.pg_used = kerninfo.current_memory; | ||
| 54 | |||
| 55 | if (copy_to_user(arg, &userinfo, sizeof(userinfo))) | ||
| 56 | return -EFAULT; | ||
| 57 | |||
| 58 | return 0; | ||
| 59 | } | ||
| 60 | |||
| 61 | static int compat_agpioc_reserve_wrap(struct agp_file_private *priv, void __user *arg) | ||
| 62 | { | ||
| 63 | struct agp_region32 ureserve; | ||
| 64 | struct agp_region kreserve; | ||
| 65 | struct agp_client *client; | ||
| 66 | struct agp_file_private *client_priv; | ||
| 67 | |||
| 68 | DBG(""); | ||
| 69 | if (copy_from_user(&ureserve, arg, sizeof(ureserve))) | ||
| 70 | return -EFAULT; | ||
| 71 | |||
| 72 | if ((unsigned) ureserve.seg_count >= ~0U/sizeof(struct agp_segment32)) | ||
| 73 | return -EFAULT; | ||
| 74 | |||
| 75 | kreserve.pid = ureserve.pid; | ||
| 76 | kreserve.seg_count = ureserve.seg_count; | ||
| 77 | |||
| 78 | client = agp_find_client_by_pid(kreserve.pid); | ||
| 79 | |||
| 80 | if (kreserve.seg_count == 0) { | ||
| 81 | /* remove a client */ | ||
| 82 | client_priv = agp_find_private(kreserve.pid); | ||
| 83 | |||
| 84 | if (client_priv != NULL) { | ||
| 85 | set_bit(AGP_FF_IS_CLIENT, &client_priv->access_flags); | ||
| 86 | set_bit(AGP_FF_IS_VALID, &client_priv->access_flags); | ||
| 87 | } | ||
| 88 | if (client == NULL) { | ||
| 89 | /* client is already removed */ | ||
| 90 | return 0; | ||
| 91 | } | ||
| 92 | return agp_remove_client(kreserve.pid); | ||
| 93 | } else { | ||
| 94 | struct agp_segment32 *usegment; | ||
| 95 | struct agp_segment *ksegment; | ||
| 96 | int seg; | ||
| 97 | |||
| 98 | if (ureserve.seg_count >= 16384) | ||
| 99 | return -EINVAL; | ||
| 100 | |||
| 101 | usegment = kmalloc_array(ureserve.seg_count, | ||
| 102 | sizeof(*usegment), | ||
| 103 | GFP_KERNEL); | ||
| 104 | if (!usegment) | ||
| 105 | return -ENOMEM; | ||
| 106 | |||
| 107 | ksegment = kmalloc_array(kreserve.seg_count, | ||
| 108 | sizeof(*ksegment), | ||
| 109 | GFP_KERNEL); | ||
| 110 | if (!ksegment) { | ||
| 111 | kfree(usegment); | ||
| 112 | return -ENOMEM; | ||
| 113 | } | ||
| 114 | |||
| 115 | if (copy_from_user(usegment, (void __user *) ureserve.seg_list, | ||
| 116 | sizeof(*usegment) * ureserve.seg_count)) { | ||
| 117 | kfree(usegment); | ||
| 118 | kfree(ksegment); | ||
| 119 | return -EFAULT; | ||
| 120 | } | ||
| 121 | |||
| 122 | for (seg = 0; seg < ureserve.seg_count; seg++) { | ||
| 123 | ksegment[seg].pg_start = usegment[seg].pg_start; | ||
| 124 | ksegment[seg].pg_count = usegment[seg].pg_count; | ||
| 125 | ksegment[seg].prot = usegment[seg].prot; | ||
| 126 | } | ||
| 127 | |||
| 128 | kfree(usegment); | ||
| 129 | kreserve.seg_list = ksegment; | ||
| 130 | |||
| 131 | if (client == NULL) { | ||
| 132 | /* Create the client and add the segment */ | ||
| 133 | client = agp_create_client(kreserve.pid); | ||
| 134 | |||
| 135 | if (client == NULL) { | ||
| 136 | kfree(ksegment); | ||
| 137 | return -ENOMEM; | ||
| 138 | } | ||
| 139 | client_priv = agp_find_private(kreserve.pid); | ||
| 140 | |||
| 141 | if (client_priv != NULL) { | ||
| 142 | set_bit(AGP_FF_IS_CLIENT, &client_priv->access_flags); | ||
| 143 | set_bit(AGP_FF_IS_VALID, &client_priv->access_flags); | ||
| 144 | } | ||
| 145 | } | ||
| 146 | return agp_create_segment(client, &kreserve); | ||
| 147 | } | ||
| 148 | /* Will never really happen */ | ||
| 149 | return -EINVAL; | ||
| 150 | } | ||
| 151 | |||
| 152 | static int compat_agpioc_allocate_wrap(struct agp_file_private *priv, void __user *arg) | ||
| 153 | { | ||
| 154 | struct agp_memory *memory; | ||
| 155 | struct agp_allocate32 alloc; | ||
| 156 | |||
| 157 | DBG(""); | ||
| 158 | if (copy_from_user(&alloc, arg, sizeof(alloc))) | ||
| 159 | return -EFAULT; | ||
| 160 | |||
| 161 | memory = agp_allocate_memory_wrap(alloc.pg_count, alloc.type); | ||
| 162 | |||
| 163 | if (memory == NULL) | ||
| 164 | return -ENOMEM; | ||
| 165 | |||
| 166 | alloc.key = memory->key; | ||
| 167 | alloc.physical = memory->physical; | ||
| 168 | |||
| 169 | if (copy_to_user(arg, &alloc, sizeof(alloc))) { | ||
| 170 | agp_free_memory_wrap(memory); | ||
| 171 | return -EFAULT; | ||
| 172 | } | ||
| 173 | return 0; | ||
| 174 | } | ||
| 175 | |||
| 176 | static int compat_agpioc_bind_wrap(struct agp_file_private *priv, void __user *arg) | ||
| 177 | { | ||
| 178 | struct agp_bind32 bind_info; | ||
| 179 | struct agp_memory *memory; | ||
| 180 | |||
| 181 | DBG(""); | ||
| 182 | if (copy_from_user(&bind_info, arg, sizeof(bind_info))) | ||
| 183 | return -EFAULT; | ||
| 184 | |||
| 185 | memory = agp_find_mem_by_key(bind_info.key); | ||
| 186 | |||
| 187 | if (memory == NULL) | ||
| 188 | return -EINVAL; | ||
| 189 | |||
| 190 | return agp_bind_memory(memory, bind_info.pg_start); | ||
| 191 | } | ||
| 192 | |||
| 193 | static int compat_agpioc_unbind_wrap(struct agp_file_private *priv, void __user *arg) | ||
| 194 | { | ||
| 195 | struct agp_memory *memory; | ||
| 196 | struct agp_unbind32 unbind; | ||
| 197 | |||
| 198 | DBG(""); | ||
| 199 | if (copy_from_user(&unbind, arg, sizeof(unbind))) | ||
| 200 | return -EFAULT; | ||
| 201 | |||
| 202 | memory = agp_find_mem_by_key(unbind.key); | ||
| 203 | |||
| 204 | if (memory == NULL) | ||
| 205 | return -EINVAL; | ||
| 206 | |||
| 207 | return agp_unbind_memory(memory); | ||
| 208 | } | ||
| 209 | |||
| 210 | long compat_agp_ioctl(struct file *file, unsigned int cmd, unsigned long arg) | ||
| 211 | { | ||
| 212 | struct agp_file_private *curr_priv = file->private_data; | ||
| 213 | int ret_val = -ENOTTY; | ||
| 214 | |||
| 215 | mutex_lock(&(agp_fe.agp_mutex)); | ||
| 216 | |||
| 217 | if ((agp_fe.current_controller == NULL) && | ||
| 218 | (cmd != AGPIOC_ACQUIRE32)) { | ||
| 219 | ret_val = -EINVAL; | ||
| 220 | goto ioctl_out; | ||
| 221 | } | ||
| 222 | if ((agp_fe.backend_acquired != true) && | ||
| 223 | (cmd != AGPIOC_ACQUIRE32)) { | ||
| 224 | ret_val = -EBUSY; | ||
| 225 | goto ioctl_out; | ||
| 226 | } | ||
| 227 | if (cmd != AGPIOC_ACQUIRE32) { | ||
| 228 | if (!(test_bit(AGP_FF_IS_CONTROLLER, &curr_priv->access_flags))) { | ||
| 229 | ret_val = -EPERM; | ||
| 230 | goto ioctl_out; | ||
| 231 | } | ||
| 232 | /* Use the original pid of the controller, | ||
| 233 | * in case it's threaded */ | ||
| 234 | |||
| 235 | if (agp_fe.current_controller->pid != curr_priv->my_pid) { | ||
| 236 | ret_val = -EBUSY; | ||
| 237 | goto ioctl_out; | ||
| 238 | } | ||
| 239 | } | ||
| 240 | |||
| 241 | switch (cmd) { | ||
| 242 | case AGPIOC_INFO32: | ||
| 243 | ret_val = compat_agpioc_info_wrap(curr_priv, (void __user *) arg); | ||
| 244 | break; | ||
| 245 | |||
| 246 | case AGPIOC_ACQUIRE32: | ||
| 247 | ret_val = agpioc_acquire_wrap(curr_priv); | ||
| 248 | break; | ||
| 249 | |||
| 250 | case AGPIOC_RELEASE32: | ||
| 251 | ret_val = agpioc_release_wrap(curr_priv); | ||
| 252 | break; | ||
| 253 | |||
| 254 | case AGPIOC_SETUP32: | ||
| 255 | ret_val = agpioc_setup_wrap(curr_priv, (void __user *) arg); | ||
| 256 | break; | ||
| 257 | |||
| 258 | case AGPIOC_RESERVE32: | ||
| 259 | ret_val = compat_agpioc_reserve_wrap(curr_priv, (void __user *) arg); | ||
| 260 | break; | ||
| 261 | |||
| 262 | case AGPIOC_PROTECT32: | ||
| 263 | ret_val = agpioc_protect_wrap(curr_priv); | ||
| 264 | break; | ||
| 265 | |||
| 266 | case AGPIOC_ALLOCATE32: | ||
| 267 | ret_val = compat_agpioc_allocate_wrap(curr_priv, (void __user *) arg); | ||
| 268 | break; | ||
| 269 | |||
| 270 | case AGPIOC_DEALLOCATE32: | ||
| 271 | ret_val = agpioc_deallocate_wrap(curr_priv, (int) arg); | ||
| 272 | break; | ||
| 273 | |||
| 274 | case AGPIOC_BIND32: | ||
| 275 | ret_val = compat_agpioc_bind_wrap(curr_priv, (void __user *) arg); | ||
| 276 | break; | ||
| 277 | |||
| 278 | case AGPIOC_UNBIND32: | ||
| 279 | ret_val = compat_agpioc_unbind_wrap(curr_priv, (void __user *) arg); | ||
| 280 | break; | ||
| 281 | |||
| 282 | case AGPIOC_CHIPSET_FLUSH32: | ||
| 283 | break; | ||
| 284 | } | ||
| 285 | |||
| 286 | ioctl_out: | ||
| 287 | DBG("ioctl returns %d\n", ret_val); | ||
| 288 | mutex_unlock(&(agp_fe.agp_mutex)); | ||
| 289 | return ret_val; | ||
| 290 | } | ||
| 291 | |||
| @@ -1,106 +0,0 @@ | |||
| 1 | /* | ||
| 2 | * Copyright (C) 1999 Jeff Hartmann | ||
| 3 | * Copyright (C) 1999 Precision Insight, Inc. | ||
| 4 | * Copyright (C) 1999 Xi Graphics, Inc. | ||
| 5 | * | ||
| 6 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 7 | * copy of this software and associated documentation files (the "Software"), | ||
| 8 | * to deal in the Software without restriction, including without limitation | ||
| 9 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 10 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 11 | * Software is furnished to do so, subject to the following conditions: | ||
| 12 | * | ||
| 13 | * The above copyright notice and this permission notice shall be included | ||
| 14 | * in all copies or substantial portions of the Software. | ||
| 15 | * | ||
| 16 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS | ||
| 17 | * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 18 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 19 | * JEFF HARTMANN, OR ANY OTHER CONTRIBUTORS BE LIABLE FOR ANY CLAIM, | ||
| 20 | * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR | ||
| 21 | * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE | ||
| 22 | * OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. | ||
| 23 | * | ||
| 24 | */ | ||
| 25 | |||
| 26 | #ifndef _AGP_COMPAT_IOCTL_H | ||
| 27 | #define _AGP_COMPAT_IOCTL_H | ||
| 28 | |||
| 29 | #include <linux/compat.h> | ||
| 30 | #include <linux/agpgart.h> | ||
| 31 | |||
| 32 | #define AGPIOC_INFO32 _IOR (AGPIOC_BASE, 0, compat_uptr_t) | ||
| 33 | #define AGPIOC_ACQUIRE32 _IO (AGPIOC_BASE, 1) | ||
| 34 | #define AGPIOC_RELEASE32 _IO (AGPIOC_BASE, 2) | ||
| 35 | #define AGPIOC_SETUP32 _IOW (AGPIOC_BASE, 3, compat_uptr_t) | ||
| 36 | #define AGPIOC_RESERVE32 _IOW (AGPIOC_BASE, 4, compat_uptr_t) | ||
| 37 | #define AGPIOC_PROTECT32 _IOW (AGPIOC_BASE, 5, compat_uptr_t) | ||
| 38 | #define AGPIOC_ALLOCATE32 _IOWR(AGPIOC_BASE, 6, compat_uptr_t) | ||
| 39 | #define AGPIOC_DEALLOCATE32 _IOW (AGPIOC_BASE, 7, compat_int_t) | ||
| 40 | #define AGPIOC_BIND32 _IOW (AGPIOC_BASE, 8, compat_uptr_t) | ||
| 41 | #define AGPIOC_UNBIND32 _IOW (AGPIOC_BASE, 9, compat_uptr_t) | ||
| 42 | #define AGPIOC_CHIPSET_FLUSH32 _IO (AGPIOC_BASE, 10) | ||
| 43 | |||
| 44 | struct agp_info32 { | ||
| 45 | struct agp_version version; /* version of the driver */ | ||
| 46 | u32 bridge_id; /* bridge vendor/device */ | ||
| 47 | u32 agp_mode; /* mode info of bridge */ | ||
| 48 | compat_long_t aper_base; /* base of aperture */ | ||
| 49 | compat_size_t aper_size; /* size of aperture */ | ||
| 50 | compat_size_t pg_total; /* max pages (swap + system) */ | ||
| 51 | compat_size_t pg_system; /* max pages (system) */ | ||
| 52 | compat_size_t pg_used; /* current pages used */ | ||
| 53 | }; | ||
| 54 | |||
| 55 | /* | ||
| 56 | * The "prot" down below needs still a "sleep" flag somehow ... | ||
| 57 | */ | ||
| 58 | struct agp_segment32 { | ||
| 59 | compat_off_t pg_start; /* starting page to populate */ | ||
| 60 | compat_size_t pg_count; /* number of pages */ | ||
| 61 | compat_int_t prot; /* prot flags for mmap */ | ||
| 62 | }; | ||
| 63 | |||
| 64 | struct agp_region32 { | ||
| 65 | compat_pid_t pid; /* pid of process */ | ||
| 66 | compat_size_t seg_count; /* number of segments */ | ||
| 67 | struct agp_segment32 *seg_list; | ||
| 68 | }; | ||
| 69 | |||
| 70 | struct agp_allocate32 { | ||
| 71 | compat_int_t key; /* tag of allocation */ | ||
| 72 | compat_size_t pg_count; /* number of pages */ | ||
| 73 | u32 type; /* 0 == normal, other devspec */ | ||
| 74 | u32 physical; /* device specific (some devices | ||
| 75 | * need a phys address of the | ||
| 76 | * actual page behind the gatt | ||
| 77 | * table) */ | ||
| 78 | }; | ||
| 79 | |||
| 80 | struct agp_bind32 { | ||
| 81 | compat_int_t key; /* tag of allocation */ | ||
| 82 | compat_off_t pg_start; /* starting page to populate */ | ||
| 83 | }; | ||
| 84 | |||
| 85 | struct agp_unbind32 { | ||
| 86 | compat_int_t key; /* tag of allocation */ | ||
| 87 | u32 priority; /* priority for paging out */ | ||
| 88 | }; | ||
| 89 | |||
| 90 | extern struct agp_front_data agp_fe; | ||
| 91 | |||
| 92 | int agpioc_acquire_wrap(struct agp_file_private *priv); | ||
| 93 | int agpioc_release_wrap(struct agp_file_private *priv); | ||
| 94 | int agpioc_protect_wrap(struct agp_file_private *priv); | ||
| 95 | int agpioc_setup_wrap(struct agp_file_private *priv, void __user *arg); | ||
| 96 | int agpioc_deallocate_wrap(struct agp_file_private *priv, int arg); | ||
| 97 | struct agp_file_private *agp_find_private(pid_t pid); | ||
| 98 | struct agp_client *agp_create_client(pid_t id); | ||
| 99 | int agp_remove_client(pid_t id); | ||
| 100 | int agp_create_segment(struct agp_client *client, struct agp_region *region); | ||
| 101 | void agp_free_memory_wrap(struct agp_memory *memory); | ||
| 102 | struct agp_memory *agp_allocate_memory_wrap(size_t pg_count, u32 type); | ||
| 103 | struct agp_memory *agp_find_mem_by_key(int key); | ||
| 104 | struct agp_client *agp_find_client_by_pid(pid_t id); | ||
| 105 | |||
| 106 | #endif /* _AGP_COMPAT_H */ | ||
| @@ -1,1068 +0,0 @@ | |||
| 1 | /* | ||
| 2 | * AGPGART driver frontend | ||
| 3 | * Copyright (C) 2004 Silicon Graphics, Inc. | ||
| 4 | * Copyright (C) 2002-2003 Dave Jones | ||
| 5 | * Copyright (C) 1999 Jeff Hartmann | ||
| 6 | * Copyright (C) 1999 Precision Insight, Inc. | ||
| 7 | * Copyright (C) 1999 Xi Graphics, Inc. | ||
| 8 | * | ||
| 9 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 10 | * copy of this software and associated documentation files (the "Software"), | ||
| 11 | * to deal in the Software without restriction, including without limitation | ||
| 12 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 13 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 14 | * Software is furnished to do so, subject to the following conditions: | ||
| 15 | * | ||
| 16 | * The above copyright notice and this permission notice shall be included | ||
| 17 | * in all copies or substantial portions of the Software. | ||
| 18 | * | ||
| 19 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS | ||
| 20 | * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 21 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 22 | * JEFF HARTMANN, OR ANY OTHER CONTRIBUTORS BE LIABLE FOR ANY CLAIM, | ||
| 23 | * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR | ||
| 24 | * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE | ||
| 25 | * OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. | ||
| 26 | * | ||
| 27 | */ | ||
| 28 | |||
| 29 | #include <linux/types.h> | ||
| 30 | #include <linux/kernel.h> | ||
| 31 | #include <linux/module.h> | ||
| 32 | #include <linux/mman.h> | ||
| 33 | #include <linux/pci.h> | ||
| 34 | #include <linux/miscdevice.h> | ||
| 35 | #include <linux/agp_backend.h> | ||
| 36 | #include <linux/agpgart.h> | ||
| 37 | #include <linux/slab.h> | ||
| 38 | #include <linux/mm.h> | ||
| 39 | #include <linux/fs.h> | ||
| 40 | #include <linux/sched.h> | ||
| 41 | #include <linux/uaccess.h> | ||
| 42 | |||
| 43 | #include "agp.h" | ||
| 44 | #include "compat_ioctl.h" | ||
| 45 | |||
| 46 | struct agp_front_data agp_fe; | ||
| 47 | |||
| 48 | struct agp_memory *agp_find_mem_by_key(int key) | ||
| 49 | { | ||
| 50 | struct agp_memory *curr; | ||
| 51 | |||
| 52 | if (agp_fe.current_controller == NULL) | ||
| 53 | return NULL; | ||
| 54 | |||
| 55 | curr = agp_fe.current_controller->pool; | ||
| 56 | |||
| 57 | while (curr != NULL) { | ||
| 58 | if (curr->key == key) | ||
| 59 | break; | ||
| 60 | curr = curr->next; | ||
| 61 | } | ||
| 62 | |||
| 63 | DBG("key=%d -> mem=%p", key, curr); | ||
| 64 | return curr; | ||
| 65 | } | ||
| 66 | |||
| 67 | static void agp_remove_from_pool(struct agp_memory *temp) | ||
| 68 | { | ||
| 69 | struct agp_memory *prev; | ||
| 70 | struct agp_memory *next; | ||
| 71 | |||
| 72 | /* Check to see if this is even in the memory pool */ | ||
| 73 | |||
| 74 | DBG("mem=%p", temp); | ||
| 75 | if (agp_find_mem_by_key(temp->key) != NULL) { | ||
| 76 | next = temp->next; | ||
| 77 | prev = temp->prev; | ||
| 78 | |||
| 79 | if (prev != NULL) { | ||
| 80 | prev->next = next; | ||
| 81 | if (next != NULL) | ||
| 82 | next->prev = prev; | ||
| 83 | |||
| 84 | } else { | ||
| 85 | /* This is the first item on the list */ | ||
| 86 | if (next != NULL) | ||
| 87 | next->prev = NULL; | ||
| 88 | |||
| 89 | agp_fe.current_controller->pool = next; | ||
| 90 | } | ||
| 91 | } | ||
| 92 | } | ||
| 93 | |||
| 94 | /* | ||
| 95 | * Routines for managing each client's segment list - | ||
| 96 | * These routines handle adding and removing segments | ||
| 97 | * to each auth'ed client. | ||
| 98 | */ | ||
| 99 | |||
| 100 | static struct | ||
| 101 | agp_segment_priv *agp_find_seg_in_client(const struct agp_client *client, | ||
| 102 | unsigned long offset, | ||
| 103 | int size, pgprot_t page_prot) | ||
| 104 | { | ||
| 105 | struct agp_segment_priv *seg; | ||
| 106 | int i; | ||
| 107 | off_t pg_start; | ||
| 108 | size_t pg_count; | ||
| 109 | |||
| 110 | pg_start = offset / 4096; | ||
| 111 | pg_count = size / 4096; | ||
| 112 | seg = *(client->segments); | ||
| 113 | |||
| 114 | for (i = 0; i < client->num_segments; i++) { | ||
| 115 | if ((seg[i].pg_start == pg_start) && | ||
| 116 | (seg[i].pg_count == pg_count) && | ||
| 117 | (pgprot_val(seg[i].prot) == pgprot_val(page_prot))) { | ||
| 118 | return seg + i; | ||
| 119 | } | ||
| 120 | } | ||
| 121 | |||
| 122 | return NULL; | ||
| 123 | } | ||
| 124 | |||
| 125 | static void agp_remove_seg_from_client(struct agp_client *client) | ||
| 126 | { | ||
| 127 | DBG("client=%p", client); | ||
| 128 | |||
| 129 | if (client->segments != NULL) { | ||
| 130 | if (*(client->segments) != NULL) { | ||
| 131 | DBG("Freeing %p from client %p", *(client->segments), client); | ||
| 132 | kfree(*(client->segments)); | ||
| 133 | } | ||
| 134 | DBG("Freeing %p from client %p", client->segments, client); | ||
| 135 | kfree(client->segments); | ||
| 136 | client->segments = NULL; | ||
| 137 | } | ||
| 138 | } | ||
| 139 | |||
| 140 | static void agp_add_seg_to_client(struct agp_client *client, | ||
| 141 | struct agp_segment_priv ** seg, int num_segments) | ||
| 142 | { | ||
| 143 | struct agp_segment_priv **prev_seg; | ||
| 144 | |||
| 145 | prev_seg = client->segments; | ||
| 146 | |||
| 147 | if (prev_seg != NULL) | ||
| 148 | agp_remove_seg_from_client(client); | ||
| 149 | |||
| 150 | DBG("Adding seg %p (%d segments) to client %p", seg, num_segments, client); | ||
| 151 | client->num_segments = num_segments; | ||
| 152 | client->segments = seg; | ||
| 153 | } | ||
| 154 | |||
| 155 | static pgprot_t agp_convert_mmap_flags(int prot) | ||
| 156 | { | ||
| 157 | unsigned long prot_bits; | ||
| 158 | |||
| 159 | prot_bits = calc_vm_prot_bits(prot, 0) | VM_SHARED; | ||
| 160 | return vm_get_page_prot(prot_bits); | ||
| 161 | } | ||
| 162 | |||
| 163 | int agp_create_segment(struct agp_client *client, struct agp_region *region) | ||
| 164 | { | ||
| 165 | struct agp_segment_priv **ret_seg; | ||
| 166 | struct agp_segment_priv *seg; | ||
| 167 | struct agp_segment *user_seg; | ||
| 168 | size_t i; | ||
| 169 | |||
| 170 | seg = kzalloc((sizeof(struct agp_segment_priv) * region->seg_count), GFP_KERNEL); | ||
| 171 | if (seg == NULL) { | ||
| 172 | kfree(region->seg_list); | ||
| 173 | region->seg_list = NULL; | ||
| 174 | return -ENOMEM; | ||
| 175 | } | ||
| 176 | user_seg = region->seg_list; | ||
| 177 | |||
| 178 | for (i = 0; i < region->seg_count; i++) { | ||
| 179 | seg[i].pg_start = user_seg[i].pg_start; | ||
| 180 | seg[i].pg_count = user_seg[i].pg_count; | ||
| 181 | seg[i].prot = agp_convert_mmap_flags(user_seg[i].prot); | ||
| 182 | } | ||
| 183 | kfree(region->seg_list); | ||
| 184 | region->seg_list = NULL; | ||
| 185 | |||
| 186 | ret_seg = kmalloc(sizeof(void *), GFP_KERNEL); | ||
| 187 | if (ret_seg == NULL) { | ||
| 188 | kfree(seg); | ||
| 189 | return -ENOMEM; | ||
| 190 | } | ||
| 191 | *ret_seg = seg; | ||
| 192 | agp_add_seg_to_client(client, ret_seg, region->seg_count); | ||
| 193 | return 0; | ||
| 194 | } | ||
| 195 | |||
| 196 | /* End - Routines for managing each client's segment list */ | ||
| 197 | |||
| 198 | /* This function must only be called when current_controller != NULL */ | ||
| 199 | static void agp_insert_into_pool(struct agp_memory * temp) | ||
| 200 | { | ||
| 201 | struct agp_memory *prev; | ||
| 202 | |||
| 203 | prev = agp_fe.current_controller->pool; | ||
| 204 | |||
| 205 | if (prev != NULL) { | ||
| 206 | prev->prev = temp; | ||
| 207 | temp->next = prev; | ||
| 208 | } | ||
| 209 | agp_fe.current_controller->pool = temp; | ||
| 210 | } | ||
| 211 | |||
| 212 | |||
| 213 | /* File private list routines */ | ||
| 214 | |||
| 215 | struct agp_file_private *agp_find_private(pid_t pid) | ||
| 216 | { | ||
| 217 | struct agp_file_private *curr; | ||
| 218 | |||
| 219 | curr = agp_fe.file_priv_list; | ||
| 220 | |||
| 221 | while (curr != NULL) { | ||
| 222 | if (curr->my_pid == pid) | ||
| 223 | return curr; | ||
| 224 | curr = curr->next; | ||
| 225 | } | ||
| 226 | |||
| 227 | return NULL; | ||
| 228 | } | ||
| 229 | |||
| 230 | static void agp_insert_file_private(struct agp_file_private * priv) | ||
| 231 | { | ||
| 232 | struct agp_file_private *prev; | ||
| 233 | |||
| 234 | prev = agp_fe.file_priv_list; | ||
| 235 | |||
| 236 | if (prev != NULL) | ||
| 237 | prev->prev = priv; | ||
| 238 | priv->next = prev; | ||
| 239 | agp_fe.file_priv_list = priv; | ||
| 240 | } | ||
| 241 | |||
| 242 | static void agp_remove_file_private(struct agp_file_private * priv) | ||
| 243 | { | ||
| 244 | struct agp_file_private *next; | ||
| 245 | struct agp_file_private *prev; | ||
| 246 | |||
| 247 | next = priv->next; | ||
| 248 | prev = priv->prev; | ||
| 249 | |||
| 250 | if (prev != NULL) { | ||
| 251 | prev->next = next; | ||
| 252 | |||
| 253 | if (next != NULL) | ||
| 254 | next->prev = prev; | ||
| 255 | |||
| 256 | } else { | ||
| 257 | if (next != NULL) | ||
| 258 | next->prev = NULL; | ||
| 259 | |||
| 260 | agp_fe.file_priv_list = next; | ||
| 261 | } | ||
| 262 | } | ||
| 263 | |||
| 264 | /* End - File flag list routines */ | ||
| 265 | |||
| 266 | /* | ||
| 267 | * Wrappers for agp_free_memory & agp_allocate_memory | ||
| 268 | * These make sure that internal lists are kept updated. | ||
| 269 | */ | ||
| 270 | void agp_free_memory_wrap(struct agp_memory *memory) | ||
| 271 | { | ||
| 272 | agp_remove_from_pool(memory); | ||
| 273 | agp_free_memory(memory); | ||
| 274 | } | ||
| 275 | |||
| 276 | struct agp_memory *agp_allocate_memory_wrap(size_t pg_count, u32 type) | ||
| 277 | { | ||
| 278 | struct agp_memory *memory; | ||
| 279 | |||
| 280 | memory = agp_allocate_memory(agp_bridge, pg_count, type); | ||
| 281 | if (memory == NULL) | ||
| 282 | return NULL; | ||
| 283 | |||
| 284 | agp_insert_into_pool(memory); | ||
| 285 | return memory; | ||
| 286 | } | ||
| 287 | |||
| 288 | /* Routines for managing the list of controllers - | ||
| 289 | * These routines manage the current controller, and the list of | ||
| 290 | * controllers | ||
| 291 | */ | ||
| 292 | |||
| 293 | static struct agp_controller *agp_find_controller_by_pid(pid_t id) | ||
| 294 | { | ||
| 295 | struct agp_controller *controller; | ||
| 296 | |||
| 297 | controller = agp_fe.controllers; | ||
| 298 | |||
| 299 | while (controller != NULL) { | ||
| 300 | if (controller->pid == id) | ||
| 301 | return controller; | ||
| 302 | controller = controller->next; | ||
| 303 | } | ||
| 304 | |||
| 305 | return NULL; | ||
| 306 | } | ||
| 307 | |||
| 308 | static struct agp_controller *agp_create_controller(pid_t id) | ||
| 309 | { | ||
| 310 | struct agp_controller *controller; | ||
| 311 | |||
| 312 | controller = kzalloc(sizeof(struct agp_controller), GFP_KERNEL); | ||
| 313 | if (controller == NULL) | ||
| 314 | return NULL; | ||
| 315 | |||
| 316 | controller->pid = id; | ||
| 317 | return controller; | ||
| 318 | } | ||
| 319 | |||
| 320 | static int agp_insert_controller(struct agp_controller *controller) | ||
| 321 | { | ||
| 322 | struct agp_controller *prev_controller; | ||
| 323 | |||
| 324 | prev_controller = agp_fe.controllers; | ||
| 325 | controller->next = prev_controller; | ||
| 326 | |||
| 327 | if (prev_controller != NULL) | ||
| 328 | prev_controller->prev = controller; | ||
| 329 | |||
| 330 | agp_fe.controllers = controller; | ||
| 331 | |||
| 332 | return 0; | ||
| 333 | } | ||
| 334 | |||
| 335 | static void agp_remove_all_clients(struct agp_controller *controller) | ||
| 336 | { | ||
| 337 | struct agp_client *client; | ||
| 338 | struct agp_client *temp; | ||
| 339 | |||
| 340 | client = controller->clients; | ||
| 341 | |||
| 342 | while (client) { | ||
| 343 | struct agp_file_private *priv; | ||
| 344 | |||
| 345 | temp = client; | ||
| 346 | agp_remove_seg_from_client(temp); | ||
| 347 | priv = agp_find_private(temp->pid); | ||
| 348 | |||
| 349 | if (priv != NULL) { | ||
| 350 | clear_bit(AGP_FF_IS_VALID, &priv->access_flags); | ||
| 351 | clear_bit(AGP_FF_IS_CLIENT, &priv->access_flags); | ||
| 352 | } | ||
| 353 | client = client->next; | ||
| 354 | kfree(temp); | ||
| 355 | } | ||
| 356 | } | ||
| 357 | |||
| 358 | static void agp_remove_all_memory(struct agp_controller *controller) | ||
| 359 | { | ||
| 360 | struct agp_memory *memory; | ||
| 361 | struct agp_memory *temp; | ||
| 362 | |||
| 363 | memory = controller->pool; | ||
| 364 | |||
| 365 | while (memory) { | ||
| 366 | temp = memory; | ||
| 367 | memory = memory->next; | ||
| 368 | agp_free_memory_wrap(temp); | ||
| 369 | } | ||
| 370 | } | ||
| 371 | |||
| 372 | static int agp_remove_controller(struct agp_controller *controller) | ||
| 373 | { | ||
| 374 | struct agp_controller *prev_controller; | ||
| 375 | struct agp_controller *next_controller; | ||
| 376 | |||
| 377 | prev_controller = controller->prev; | ||
| 378 | next_controller = controller->next; | ||
| 379 | |||
| 380 | if (prev_controller != NULL) { | ||
| 381 | prev_controller->next = next_controller; | ||
| 382 | if (next_controller != NULL) | ||
| 383 | next_controller->prev = prev_controller; | ||
| 384 | |||
| 385 | } else { | ||
| 386 | if (next_controller != NULL) | ||
| 387 | next_controller->prev = NULL; | ||
| 388 | |||
| 389 | agp_fe.controllers = next_controller; | ||
| 390 | } | ||
| 391 | |||
| 392 | agp_remove_all_memory(controller); | ||
| 393 | agp_remove_all_clients(controller); | ||
| 394 | |||
| 395 | if (agp_fe.current_controller == controller) { | ||
| 396 | agp_fe.current_controller = NULL; | ||
| 397 | agp_fe.backend_acquired = false; | ||
| 398 | agp_backend_release(agp_bridge); | ||
| 399 | } | ||
| 400 | kfree(controller); | ||
| 401 | return 0; | ||
| 402 | } | ||
| 403 | |||
| 404 | static void agp_controller_make_current(struct agp_controller *controller) | ||
| 405 | { | ||
| 406 | struct agp_client *clients; | ||
| 407 | |||
| 408 | clients = controller->clients; | ||
| 409 | |||
| 410 | while (clients != NULL) { | ||
| 411 | struct agp_file_private *priv; | ||
| 412 | |||
| 413 | priv = agp_find_private(clients->pid); | ||
| 414 | |||
| 415 | if (priv != NULL) { | ||
| 416 | set_bit(AGP_FF_IS_VALID, &priv->access_flags); | ||
| 417 | set_bit(AGP_FF_IS_CLIENT, &priv->access_flags); | ||
| 418 | } | ||
| 419 | clients = clients->next; | ||
| 420 | } | ||
| 421 | |||
| 422 | agp_fe.current_controller = controller; | ||
| 423 | } | ||
| 424 | |||
| 425 | static void agp_controller_release_current(struct agp_controller *controller, | ||
| 426 | struct agp_file_private *controller_priv) | ||
| 427 | { | ||
| 428 | struct agp_client *clients; | ||
| 429 | |||
| 430 | clear_bit(AGP_FF_IS_VALID, &controller_priv->access_flags); | ||
| 431 | clients = controller->clients; | ||
| 432 | |||
| 433 | while (clients != NULL) { | ||
| 434 | struct agp_file_private *priv; | ||
| 435 | |||
| 436 | priv = agp_find_private(clients->pid); | ||
| 437 | |||
| 438 | if (priv != NULL) | ||
| 439 | clear_bit(AGP_FF_IS_VALID, &priv->access_flags); | ||
| 440 | |||
| 441 | clients = clients->next; | ||
| 442 | } | ||
| 443 | |||
| 444 | agp_fe.current_controller = NULL; | ||
| 445 | agp_fe.used_by_controller = false; | ||
| 446 | agp_backend_release(agp_bridge); | ||
| 447 | } | ||
| 448 | |||
| 449 | /* | ||
| 450 | * Routines for managing client lists - | ||
| 451 | * These routines are for managing the list of auth'ed clients. | ||
| 452 | */ | ||
| 453 | |||
| 454 | static struct agp_client | ||
| 455 | *agp_find_client_in_controller(struct agp_controller *controller, pid_t id) | ||
| 456 | { | ||
| 457 | struct agp_client *client; | ||
| 458 | |||
| 459 | if (controller == NULL) | ||
| 460 | return NULL; | ||
| 461 | |||
| 462 | client = controller->clients; | ||
| 463 | |||
| 464 | while (client != NULL) { | ||
| 465 | if (client->pid == id) | ||
| 466 | return client; | ||
| 467 | client = client->next; | ||
| 468 | } | ||
| 469 | |||
| 470 | return NULL; | ||
| 471 | } | ||
| 472 | |||
| 473 | static struct agp_controller *agp_find_controller_for_client(pid_t id) | ||
| 474 | { | ||
| 475 | struct agp_controller *controller; | ||
| 476 | |||
| 477 | controller = agp_fe.controllers; | ||
| 478 | |||
| 479 | while (controller != NULL) { | ||
| 480 | if ((agp_find_client_in_controller(controller, id)) != NULL) | ||
| 481 | return controller; | ||
| 482 | controller = controller->next; | ||
| 483 | } | ||
| 484 | |||
| 485 | return NULL; | ||
| 486 | } | ||
| 487 | |||
| 488 | struct agp_client *agp_find_client_by_pid(pid_t id) | ||
| 489 | { | ||
| 490 | struct agp_client *temp; | ||
| 491 | |||
| 492 | if (agp_fe.current_controller == NULL) | ||
| 493 | return NULL; | ||
| 494 | |||
| 495 | temp = agp_find_client_in_controller(agp_fe.current_controller, id); | ||
| 496 | return temp; | ||
| 497 | } | ||
| 498 | |||
| 499 | static void agp_insert_client(struct agp_client *client) | ||
| 500 | { | ||
| 501 | struct agp_client *prev_client; | ||
| 502 | |||
| 503 | prev_client = agp_fe.current_controller->clients; | ||
| 504 | client->next = prev_client; | ||
| 505 | |||
| 506 | if (prev_client != NULL) | ||
| 507 | prev_client->prev = client; | ||
| 508 | |||
| 509 | agp_fe.current_controller->clients = client; | ||
| 510 | agp_fe.current_controller->num_clients++; | ||
| 511 | } | ||
| 512 | |||
| 513 | struct agp_client *agp_create_client(pid_t id) | ||
| 514 | { | ||
| 515 | struct agp_client *new_client; | ||
| 516 | |||
| 517 | new_client = kzalloc(sizeof(struct agp_client), GFP_KERNEL); | ||
| 518 | if (new_client == NULL) | ||
| 519 | return NULL; | ||
| 520 | |||
| 521 | new_client->pid = id; | ||
| 522 | agp_insert_client(new_client); | ||
| 523 | return new_client; | ||
| 524 | } | ||
| 525 | |||
| 526 | int agp_remove_client(pid_t id) | ||
| 527 | { | ||
| 528 | struct agp_client *client; | ||
| 529 | struct agp_client *prev_client; | ||
| 530 | struct agp_client *next_client; | ||
| 531 | struct agp_controller *controller; | ||
| 532 | |||
| 533 | controller = agp_find_controller_for_client(id); | ||
| 534 | if (controller == NULL) | ||
| 535 | return -EINVAL; | ||
| 536 | |||
| 537 | client = agp_find_client_in_controller(controller, id); | ||
| 538 | if (client == NULL) | ||
| 539 | return -EINVAL; | ||
| 540 | |||
| 541 | prev_client = client->prev; | ||
| 542 | next_client = client->next; | ||
| 543 | |||
| 544 | if (prev_client != NULL) { | ||
| 545 | prev_client->next = next_client; | ||
| 546 | if (next_client != NULL) | ||
| 547 | next_client->prev = prev_client; | ||
| 548 | |||
| 549 | } else { | ||
| 550 | if (next_client != NULL) | ||
| 551 | next_client->prev = NULL; | ||
| 552 | controller->clients = next_client; | ||
| 553 | } | ||
| 554 | |||
| 555 | controller->num_clients--; | ||
| 556 | agp_remove_seg_from_client(client); | ||
| 557 | kfree(client); | ||
| 558 | return 0; | ||
| 559 | } | ||
| 560 | |||
| 561 | /* End - Routines for managing client lists */ | ||
| 562 | |||
| 563 | /* File Operations */ | ||
| 564 | |||
| 565 | static int agp_mmap(struct file *file, struct vm_area_struct *vma) | ||
| 566 | { | ||
| 567 | unsigned int size, current_size; | ||
| 568 | unsigned long offset; | ||
| 569 | struct agp_client *client; | ||
| 570 | struct agp_file_private *priv = file->private_data; | ||
| 571 | struct agp_kern_info kerninfo; | ||
| 572 | |||
| 573 | mutex_lock(&(agp_fe.agp_mutex)); | ||
| 574 | |||
| 575 | if (agp_fe.backend_acquired != true) | ||
| 576 | goto out_eperm; | ||
| 577 | |||
| 578 | if (!(test_bit(AGP_FF_IS_VALID, &priv->access_flags))) | ||
| 579 | goto out_eperm; | ||
| 580 | |||
| 581 | agp_copy_info(agp_bridge, &kerninfo); | ||
| 582 | size = vma->vm_end - vma->vm_start; | ||
| 583 | current_size = kerninfo.aper_size; | ||
| 584 | current_size = current_size * 0x100000; | ||
| 585 | offset = vma->vm_pgoff << PAGE_SHIFT; | ||
| 586 | DBG("%lx:%lx", offset, offset+size); | ||
| 587 | |||
| 588 | if (test_bit(AGP_FF_IS_CLIENT, &priv->access_flags)) { | ||
| 589 | if ((size + offset) > current_size) | ||
| 590 | goto out_inval; | ||
| 591 | |||
| 592 | client = agp_find_client_by_pid(current->pid); | ||
| 593 | |||
| 594 | if (client == NULL) | ||
| 595 | goto out_eperm; | ||
| 596 | |||
| 597 | if (!agp_find_seg_in_client(client, offset, size, vma->vm_page_prot)) | ||
| 598 | goto out_inval; | ||
| 599 | |||
| 600 | DBG("client vm_ops=%p", kerninfo.vm_ops); | ||
| 601 | if (kerninfo.vm_ops) { | ||
| 602 | vma->vm_ops = kerninfo.vm_ops; | ||
| 603 | } else if (io_remap_pfn_range(vma, vma->vm_start, | ||
| 604 | (kerninfo.aper_base + offset) >> PAGE_SHIFT, | ||
| 605 | size, | ||
| 606 | pgprot_writecombine(vma->vm_page_prot))) { | ||
| 607 | goto out_again; | ||
| 608 | } | ||
| 609 | mutex_unlock(&(agp_fe.agp_mutex)); | ||
| 610 | return 0; | ||
| 611 | } | ||
| 612 | |||
| 613 | if (test_bit(AGP_FF_IS_CONTROLLER, &priv->access_flags)) { | ||
| 614 | if (size != current_size) | ||
| 615 | goto out_inval; | ||
| 616 | |||
| 617 | DBG("controller vm_ops=%p", kerninfo.vm_ops); | ||
| 618 | if (kerninfo.vm_ops) { | ||
| 619 | vma->vm_ops = kerninfo.vm_ops; | ||
| 620 | } else if (io_remap_pfn_range(vma, vma->vm_start, | ||
| 621 | kerninfo.aper_base >> PAGE_SHIFT, | ||
| 622 | size, | ||
| 623 | pgprot_writecombine(vma->vm_page_prot))) { | ||
| 624 | goto out_again; | ||
| 625 | } | ||
| 626 | mutex_unlock(&(agp_fe.agp_mutex)); | ||
| 627 | return 0; | ||
| 628 | } | ||
| 629 | |||
| 630 | out_eperm: | ||
| 631 | mutex_unlock(&(agp_fe.agp_mutex)); | ||
| 632 | return -EPERM; | ||
| 633 | |||
| 634 | out_inval: | ||
| 635 | mutex_unlock(&(agp_fe.agp_mutex)); | ||
| 636 | return -EINVAL; | ||
| 637 | |||
| 638 | out_again: | ||
| 639 | mutex_unlock(&(agp_fe.agp_mutex)); | ||
| 640 | return -EAGAIN; | ||
| 641 | } | ||
| 642 | |||
| 643 | static int agp_release(struct inode *inode, struct file *file) | ||
| 644 | { | ||
| 645 | struct agp_file_private *priv = file->private_data; | ||
| 646 | |||
| 647 | mutex_lock(&(agp_fe.agp_mutex)); | ||
| 648 | |||
| 649 | DBG("priv=%p", priv); | ||
| 650 | |||
| 651 | if (test_bit(AGP_FF_IS_CONTROLLER, &priv->access_flags)) { | ||
| 652 | struct agp_controller *controller; | ||
| 653 | |||
| 654 | controller = agp_find_controller_by_pid(priv->my_pid); | ||
| 655 | |||
| 656 | if (controller != NULL) { | ||
| 657 | if (controller == agp_fe.current_controller) | ||
| 658 | agp_controller_release_current(controller, priv); | ||
| 659 | agp_remove_controller(controller); | ||
| 660 | controller = NULL; | ||
| 661 | } | ||
| 662 | } | ||
| 663 | |||
| 664 | if (test_bit(AGP_FF_IS_CLIENT, &priv->access_flags)) | ||
| 665 | agp_remove_client(priv->my_pid); | ||
| 666 | |||
| 667 | agp_remove_file_private(priv); | ||
| 668 | kfree(priv); | ||
| 669 | file->private_data = NULL; | ||
| 670 | mutex_unlock(&(agp_fe.agp_mutex)); | ||
| 671 | return 0; | ||
| 672 | } | ||
| 673 | |||
| 674 | static int agp_open(struct inode *inode, struct file *file) | ||
| 675 | { | ||
| 676 | int minor = iminor(inode); | ||
| 677 | struct agp_file_private *priv; | ||
| 678 | struct agp_client *client; | ||
| 679 | |||
| 680 | if (minor != AGPGART_MINOR) | ||
| 681 | return -ENXIO; | ||
| 682 | |||
| 683 | mutex_lock(&(agp_fe.agp_mutex)); | ||
| 684 | |||
| 685 | priv = kzalloc(sizeof(struct agp_file_private), GFP_KERNEL); | ||
| 686 | if (priv == NULL) { | ||
| 687 | mutex_unlock(&(agp_fe.agp_mutex)); | ||
| 688 | return -ENOMEM; | ||
| 689 | } | ||
| 690 | |||
| 691 | set_bit(AGP_FF_ALLOW_CLIENT, &priv->access_flags); | ||
| 692 | priv->my_pid = current->pid; | ||
| 693 | |||
| 694 | if (capable(CAP_SYS_RAWIO)) | ||
| 695 | /* Root priv, can be controller */ | ||
| 696 | set_bit(AGP_FF_ALLOW_CONTROLLER, &priv->access_flags); | ||
| 697 | |||
| 698 | client = agp_find_client_by_pid(current->pid); | ||
| 699 | |||
| 700 | if (client != NULL) { | ||
| 701 | set_bit(AGP_FF_IS_CLIENT, &priv->access_flags); | ||
| 702 | set_bit(AGP_FF_IS_VALID, &priv->access_flags); | ||
| 703 | } | ||
| 704 | file->private_data = (void *) priv; | ||
| 705 | agp_insert_file_private(priv); | ||
| 706 | DBG("private=%p, client=%p", priv, client); | ||
| 707 | |||
| 708 | mutex_unlock(&(agp_fe.agp_mutex)); | ||
| 709 | |||
| 710 | return 0; | ||
| 711 | } | ||
| 712 | |||
| 713 | static int agpioc_info_wrap(struct agp_file_private *priv, void __user *arg) | ||
| 714 | { | ||
| 715 | struct agp_info userinfo; | ||
| 716 | struct agp_kern_info kerninfo; | ||
| 717 | |||
| 718 | agp_copy_info(agp_bridge, &kerninfo); | ||
| 719 | |||
| 720 | memset(&userinfo, 0, sizeof(userinfo)); | ||
| 721 | userinfo.version.major = kerninfo.version.major; | ||
| 722 | userinfo.version.minor = kerninfo.version.minor; | ||
| 723 | userinfo.bridge_id = kerninfo.device->vendor | | ||
| 724 | (kerninfo.device->device << 16); | ||
| 725 | userinfo.agp_mode = kerninfo.mode; | ||
| 726 | userinfo.aper_base = kerninfo.aper_base; | ||
| 727 | userinfo.aper_size = kerninfo.aper_size; | ||
| 728 | userinfo.pg_total = userinfo.pg_system = kerninfo.max_memory; | ||
| 729 | userinfo.pg_used = kerninfo.current_memory; | ||
| 730 | |||
| 731 | if (copy_to_user(arg, &userinfo, sizeof(struct agp_info))) | ||
| 732 | return -EFAULT; | ||
| 733 | |||
| 734 | return 0; | ||
| 735 | } | ||
| 736 | |||
| 737 | int agpioc_acquire_wrap(struct agp_file_private *priv) | ||
| 738 | { | ||
| 739 | struct agp_controller *controller; | ||
| 740 | |||
| 741 | DBG(""); | ||
| 742 | |||
| 743 | if (!(test_bit(AGP_FF_ALLOW_CONTROLLER, &priv->access_flags))) | ||
| 744 | return -EPERM; | ||
| 745 | |||
| 746 | if (agp_fe.current_controller != NULL) | ||
| 747 | return -EBUSY; | ||
| 748 | |||
| 749 | if (!agp_bridge) | ||
| 750 | return -ENODEV; | ||
| 751 | |||
| 752 | if (atomic_read(&agp_bridge->agp_in_use)) | ||
| 753 | return -EBUSY; | ||
| 754 | |||
| 755 | atomic_inc(&agp_bridge->agp_in_use); | ||
| 756 | |||
| 757 | agp_fe.backend_acquired = true; | ||
| 758 | |||
| 759 | controller = agp_find_controller_by_pid(priv->my_pid); | ||
| 760 | |||
| 761 | if (controller != NULL) { | ||
| 762 | agp_controller_make_current(controller); | ||
| 763 | } else { | ||
| 764 | controller = agp_create_controller(priv->my_pid); | ||
| 765 | |||
| 766 | if (controller == NULL) { | ||
| 767 | agp_fe.backend_acquired = false; | ||
| 768 | agp_backend_release(agp_bridge); | ||
| 769 | return -ENOMEM; | ||
| 770 | } | ||
| 771 | agp_insert_controller(controller); | ||
| 772 | agp_controller_make_current(controller); | ||
| 773 | } | ||
| 774 | |||
| 775 | set_bit(AGP_FF_IS_CONTROLLER, &priv->access_flags); | ||
| 776 | set_bit(AGP_FF_IS_VALID, &priv->access_flags); | ||
| 777 | return 0; | ||
| 778 | } | ||
| 779 | |||
| 780 | int agpioc_release_wrap(struct agp_file_private *priv) | ||
| 781 | { | ||
| 782 | DBG(""); | ||
| 783 | agp_controller_release_current(agp_fe.current_controller, priv); | ||
| 784 | return 0; | ||
| 785 | } | ||
| 786 | |||
| 787 | int agpioc_setup_wrap(struct agp_file_private *priv, void __user *arg) | ||
| 788 | { | ||
| 789 | struct agp_setup mode; | ||
| 790 | |||
| 791 | DBG(""); | ||
| 792 | if (copy_from_user(&mode, arg, sizeof(struct agp_setup))) | ||
| 793 | return -EFAULT; | ||
| 794 | |||
| 795 | agp_enable(agp_bridge, mode.agp_mode); | ||
| 796 | return 0; | ||
| 797 | } | ||
| 798 | |||
| 799 | static int agpioc_reserve_wrap(struct agp_file_private *priv, void __user *arg) | ||
| 800 | { | ||
| 801 | struct agp_region reserve; | ||
| 802 | struct agp_client *client; | ||
| 803 | struct agp_file_private *client_priv; | ||
| 804 | |||
| 805 | DBG(""); | ||
| 806 | if (copy_from_user(&reserve, arg, sizeof(struct agp_region))) | ||
| 807 | return -EFAULT; | ||
| 808 | |||
| 809 | if ((unsigned) reserve.seg_count >= ~0U/sizeof(struct agp_segment)) | ||
| 810 | return -EFAULT; | ||
| 811 | |||
| 812 | client = agp_find_client_by_pid(reserve.pid); | ||
| 813 | |||
| 814 | if (reserve.seg_count == 0) { | ||
| 815 | /* remove a client */ | ||
| 816 | client_priv = agp_find_private(reserve.pid); | ||
| 817 | |||
| 818 | if (client_priv != NULL) { | ||
| 819 | set_bit(AGP_FF_IS_CLIENT, &client_priv->access_flags); | ||
| 820 | set_bit(AGP_FF_IS_VALID, &client_priv->access_flags); | ||
| 821 | } | ||
| 822 | if (client == NULL) { | ||
| 823 | /* client is already removed */ | ||
| 824 | return 0; | ||
| 825 | } | ||
| 826 | return agp_remove_client(reserve.pid); | ||
| 827 | } else { | ||
| 828 | struct agp_segment *segment; | ||
| 829 | |||
| 830 | if (reserve.seg_count >= 16384) | ||
| 831 | return -EINVAL; | ||
| 832 | |||
| 833 | segment = kmalloc((sizeof(struct agp_segment) * reserve.seg_count), | ||
| 834 | GFP_KERNEL); | ||
| 835 | |||
| 836 | if (segment == NULL) | ||
| 837 | return -ENOMEM; | ||
| 838 | |||
| 839 | if (copy_from_user(segment, (void __user *) reserve.seg_list, | ||
| 840 | sizeof(struct agp_segment) * reserve.seg_count)) { | ||
| 841 | kfree(segment); | ||
| 842 | return -EFAULT; | ||
| 843 | } | ||
| 844 | reserve.seg_list = segment; | ||
| 845 | |||
| 846 | if (client == NULL) { | ||
| 847 | /* Create the client and add the segment */ | ||
| 848 | client = agp_create_client(reserve.pid); | ||
| 849 | |||
| 850 | if (client == NULL) { | ||
| 851 | kfree(segment); | ||
| 852 | return -ENOMEM; | ||
| 853 | } | ||
| 854 | client_priv = agp_find_private(reserve.pid); | ||
| 855 | |||
| 856 | if (client_priv != NULL) { | ||
| 857 | set_bit(AGP_FF_IS_CLIENT, &client_priv->access_flags); | ||
| 858 | set_bit(AGP_FF_IS_VALID, &client_priv->access_flags); | ||
| 859 | } | ||
| 860 | } | ||
| 861 | return agp_create_segment(client, &reserve); | ||
| 862 | } | ||
| 863 | /* Will never really happen */ | ||
| 864 | return -EINVAL; | ||
| 865 | } | ||
| 866 | |||
| 867 | int agpioc_protect_wrap(struct agp_file_private *priv) | ||
| 868 | { | ||
| 869 | DBG(""); | ||
| 870 | /* This function is not currently implemented */ | ||
| 871 | return -EINVAL; | ||
| 872 | } | ||
| 873 | |||
| 874 | static int agpioc_allocate_wrap(struct agp_file_private *priv, void __user *arg) | ||
| 875 | { | ||
| 876 | struct agp_memory *memory; | ||
| 877 | struct agp_allocate alloc; | ||
| 878 | |||
| 879 | DBG(""); | ||
| 880 | if (copy_from_user(&alloc, arg, sizeof(struct agp_allocate))) | ||
| 881 | return -EFAULT; | ||
| 882 | |||
| 883 | if (alloc.type >= AGP_USER_TYPES) | ||
| 884 | return -EINVAL; | ||
| 885 | |||
| 886 | memory = agp_allocate_memory_wrap(alloc.pg_count, alloc.type); | ||
| 887 | |||
| 888 | if (memory == NULL) | ||
| 889 | return -ENOMEM; | ||
| 890 | |||
| 891 | alloc.key = memory->key; | ||
| 892 | alloc.physical = memory->physical; | ||
| 893 | |||
| 894 | if (copy_to_user(arg, &alloc, sizeof(struct agp_allocate))) { | ||
| 895 | agp_free_memory_wrap(memory); | ||
| 896 | return -EFAULT; | ||
| 897 | } | ||
| 898 | return 0; | ||
| 899 | } | ||
| 900 | |||
| 901 | int agpioc_deallocate_wrap(struct agp_file_private *priv, int arg) | ||
| 902 | { | ||
| 903 | struct agp_memory *memory; | ||
| 904 | |||
| 905 | DBG(""); | ||
| 906 | memory = agp_find_mem_by_key(arg); | ||
| 907 | |||
| 908 | if (memory == NULL) | ||
| 909 | return -EINVAL; | ||
| 910 | |||
| 911 | agp_free_memory_wrap(memory); | ||
| 912 | return 0; | ||
| 913 | } | ||
| 914 | |||
| 915 | static int agpioc_bind_wrap(struct agp_file_private *priv, void __user *arg) | ||
| 916 | { | ||
| 917 | struct agp_bind bind_info; | ||
| 918 | struct agp_memory *memory; | ||
| 919 | |||
| 920 | DBG(""); | ||
| 921 | if (copy_from_user(&bind_info, arg, sizeof(struct agp_bind))) | ||
| 922 | return -EFAULT; | ||
| 923 | |||
| 924 | memory = agp_find_mem_by_key(bind_info.key); | ||
| 925 | |||
| 926 | if (memory == NULL) | ||
| 927 | return -EINVAL; | ||
| 928 | |||
| 929 | return agp_bind_memory(memory, bind_info.pg_start); | ||
| 930 | } | ||
| 931 | |||
| 932 | static int agpioc_unbind_wrap(struct agp_file_private *priv, void __user *arg) | ||
| 933 | { | ||
| 934 | struct agp_memory *memory; | ||
| 935 | struct agp_unbind unbind; | ||
| 936 | |||
| 937 | DBG(""); | ||
| 938 | if (copy_from_user(&unbind, arg, sizeof(struct agp_unbind))) | ||
| 939 | return -EFAULT; | ||
| 940 | |||
| 941 | memory = agp_find_mem_by_key(unbind.key); | ||
| 942 | |||
| 943 | if (memory == NULL) | ||
| 944 | return -EINVAL; | ||
| 945 | |||
| 946 | return agp_unbind_memory(memory); | ||
| 947 | } | ||
| 948 | |||
| 949 | static long agp_ioctl(struct file *file, | ||
| 950 | unsigned int cmd, unsigned long arg) | ||
| 951 | { | ||
| 952 | struct agp_file_private *curr_priv = file->private_data; | ||
| 953 | int ret_val = -ENOTTY; | ||
| 954 | |||
| 955 | DBG("priv=%p, cmd=%x", curr_priv, cmd); | ||
| 956 | mutex_lock(&(agp_fe.agp_mutex)); | ||
| 957 | |||
| 958 | if ((agp_fe.current_controller == NULL) && | ||
| 959 | (cmd != AGPIOC_ACQUIRE)) { | ||
| 960 | ret_val = -EINVAL; | ||
| 961 | goto ioctl_out; | ||
| 962 | } | ||
| 963 | if ((agp_fe.backend_acquired != true) && | ||
| 964 | (cmd != AGPIOC_ACQUIRE)) { | ||
| 965 | ret_val = -EBUSY; | ||
| 966 | goto ioctl_out; | ||
| 967 | } | ||
| 968 | if (cmd != AGPIOC_ACQUIRE) { | ||
| 969 | if (!(test_bit(AGP_FF_IS_CONTROLLER, &curr_priv->access_flags))) { | ||
| 970 | ret_val = -EPERM; | ||
| 971 | goto ioctl_out; | ||
| 972 | } | ||
| 973 | /* Use the original pid of the controller, | ||
| 974 | * in case it's threaded */ | ||
| 975 | |||
| 976 | if (agp_fe.current_controller->pid != curr_priv->my_pid) { | ||
| 977 | ret_val = -EBUSY; | ||
| 978 | goto ioctl_out; | ||
| 979 | } | ||
| 980 | } | ||
| 981 | |||
| 982 | switch (cmd) { | ||
| 983 | case AGPIOC_INFO: | ||
| 984 | ret_val = agpioc_info_wrap(curr_priv, (void __user *) arg); | ||
| 985 | break; | ||
| 986 | |||
| 987 | case AGPIOC_ACQUIRE: | ||
| 988 | ret_val = agpioc_acquire_wrap(curr_priv); | ||
| 989 | break; | ||
| 990 | |||
| 991 | case AGPIOC_RELEASE: | ||
| 992 | ret_val = agpioc_release_wrap(curr_priv); | ||
| 993 | break; | ||
| 994 | |||
| 995 | case AGPIOC_SETUP: | ||
| 996 | ret_val = agpioc_setup_wrap(curr_priv, (void __user *) arg); | ||
| 997 | break; | ||
| 998 | |||
| 999 | case AGPIOC_RESERVE: | ||
| 1000 | ret_val = agpioc_reserve_wrap(curr_priv, (void __user *) arg); | ||
| 1001 | break; | ||
| 1002 | |||
| 1003 | case AGPIOC_PROTECT: | ||
| 1004 | ret_val = agpioc_protect_wrap(curr_priv); | ||
| 1005 | break; | ||
| 1006 | |||
| 1007 | case AGPIOC_ALLOCATE: | ||
| 1008 | ret_val = agpioc_allocate_wrap(curr_priv, (void __user *) arg); | ||
| 1009 | break; | ||
| 1010 | |||
| 1011 | case AGPIOC_DEALLOCATE: | ||
| 1012 | ret_val = agpioc_deallocate_wrap(curr_priv, (int) arg); | ||
| 1013 | break; | ||
| 1014 | |||
| 1015 | case AGPIOC_BIND: | ||
| 1016 | ret_val = agpioc_bind_wrap(curr_priv, (void __user *) arg); | ||
| 1017 | break; | ||
| 1018 | |||
| 1019 | case AGPIOC_UNBIND: | ||
| 1020 | ret_val = agpioc_unbind_wrap(curr_priv, (void __user *) arg); | ||
| 1021 | break; | ||
| 1022 | |||
| 1023 | case AGPIOC_CHIPSET_FLUSH: | ||
| 1024 | break; | ||
| 1025 | } | ||
| 1026 | |||
| 1027 | ioctl_out: | ||
| 1028 | DBG("ioctl returns %d\n", ret_val); | ||
| 1029 | mutex_unlock(&(agp_fe.agp_mutex)); | ||
| 1030 | return ret_val; | ||
| 1031 | } | ||
| 1032 | |||
| 1033 | static const struct file_operations agp_fops = | ||
| 1034 | { | ||
| 1035 | .owner = THIS_MODULE, | ||
| 1036 | .llseek = no_llseek, | ||
| 1037 | .unlocked_ioctl = agp_ioctl, | ||
| 1038 | #ifdef CONFIG_COMPAT | ||
| 1039 | .compat_ioctl = compat_agp_ioctl, | ||
| 1040 | #endif | ||
| 1041 | .mmap = agp_mmap, | ||
| 1042 | .open = agp_open, | ||
| 1043 | .release = agp_release, | ||
| 1044 | }; | ||
| 1045 | |||
| 1046 | static struct miscdevice agp_miscdev = | ||
| 1047 | { | ||
| 1048 | .minor = AGPGART_MINOR, | ||
| 1049 | .name = "agpgart", | ||
| 1050 | .fops = &agp_fops | ||
| 1051 | }; | ||
| 1052 | |||
| 1053 | int agp_frontend_initialize(void) | ||
| 1054 | { | ||
| 1055 | memset(&agp_fe, 0, sizeof(struct agp_front_data)); | ||
| 1056 | mutex_init(&(agp_fe.agp_mutex)); | ||
| 1057 | |||
| 1058 | if (misc_register(&agp_miscdev)) { | ||
| 1059 | printk(KERN_ERR PFX "unable to get minor: %d\n", AGPGART_MINOR); | ||
| 1060 | return -EIO; | ||
| 1061 | } | ||
| 1062 | return 0; | ||
| 1063 | } | ||
| 1064 | |||
| 1065 | void agp_frontend_cleanup(void) | ||
| 1066 | { | ||
| 1067 | misc_deregister(&agp_miscdev); | ||
| 1068 | } | ||
| @@ -46,12 +46,12 @@ static char *dmabuffs_dname(struct dentry *dentry, char *buffer, int buflen) · 45 unmodified lines | |||
| 46 | { | 46 | { |
| 47 | struct dma_buf *dmabuf; | 47 | struct dma_buf *dmabuf; |
| 48 | char name[DMA_BUF_NAME_LEN]; | 48 | char name[DMA_BUF_NAME_LEN]; |
| 49 | size_t ret = 0; | 49 | ssize_t ret = 0; |
| 50 | 50 | ||
| 51 | dmabuf = dentry->d_fsdata; | 51 | dmabuf = dentry->d_fsdata; |
| 52 | spin_lock(&dmabuf->name_lock); | 52 | spin_lock(&dmabuf->name_lock); |
| 53 | if (dmabuf->name) | 53 | if (dmabuf->name) |
| 54 | ret = strlcpy(name, dmabuf->name, DMA_BUF_NAME_LEN); | 54 | ret = strscpy(name, dmabuf->name, sizeof(name)); |
| 55 | spin_unlock(&dmabuf->name_lock); | 55 | spin_unlock(&dmabuf->name_lock); |
| 56 | 56 | ||
| 57 | return dynamic_dname(buffer, buflen, "/%s:%s", | 57 | return dynamic_dname(buffer, buflen, "/%s:%s", |
| @@ -934,7 +934,8 @@ EXPORT_SYMBOL(dma_fence_wait_any_timeout); · 933 unmodified lines | |||
| 934 | * the GPU's devfreq to reduce frequency, when in fact the opposite is what is | 934 | * the GPU's devfreq to reduce frequency, when in fact the opposite is what is |
| 935 | * needed. | 935 | * needed. |
| 936 | * | 936 | * |
| 937 | * To this end, deadline hint(s) can be set on a &dma_fence via &dma_fence_set_deadline. | 937 | * To this end, deadline hint(s) can be set on a &dma_fence via &dma_fence_set_deadline |
| 938 | * (or indirectly via userspace facing ioctls like &sync_set_deadline). | ||
| 938 | * The deadline hint provides a way for the waiting driver, or userspace, to | 939 | * The deadline hint provides a way for the waiting driver, or userspace, to |
| 939 | * convey an appropriate sense of urgency to the signaling driver. | 940 | * convey an appropriate sense of urgency to the signaling driver. |
| 940 | * | 941 | * |
| @@ -52,12 +52,33 @@ struct sw_sync_create_fence_data { · 51 unmodified lines | |||
| 52 | __s32 fence; /* fd of new fence */ | 52 | __s32 fence; /* fd of new fence */ |
| 53 | }; | 53 | }; |
| 54 | 54 | ||
| 55 | /** | ||
| 56 | * struct sw_sync_get_deadline - get the deadline hint of a sw_sync fence | ||
| 57 | * @deadline_ns: absolute time of the deadline | ||
| 58 | * @pad: must be zero | ||
| 59 | * @fence_fd: the sw_sync fence fd (in) | ||
| 60 | * | ||
| 61 | * Return the earliest deadline set on the fence. The timebase for the | ||
| 62 | * deadline is CLOCK_MONOTONIC (same as vblank). If there is no deadline | ||
| 63 | * set on the fence, this ioctl will return -ENOENT. | ||
| 64 | */ | ||
| 65 | struct sw_sync_get_deadline { | ||
| 66 | __u64 deadline_ns; | ||
| 67 | __u32 pad; | ||
| 68 | __s32 fence_fd; | ||
| 69 | }; | ||
| 70 | |||
| 55 | #define SW_SYNC_IOC_MAGIC 'W' | 71 | #define SW_SYNC_IOC_MAGIC 'W' |
| 56 | 72 | ||
| 57 | #define SW_SYNC_IOC_CREATE_FENCE _IOWR(SW_SYNC_IOC_MAGIC, 0,\ | 73 | #define SW_SYNC_IOC_CREATE_FENCE _IOWR(SW_SYNC_IOC_MAGIC, 0,\ |
| 58 | struct sw_sync_create_fence_data) | 74 | struct sw_sync_create_fence_data) |
| 59 | 75 | ||
| 60 | #define SW_SYNC_IOC_INC _IOW(SW_SYNC_IOC_MAGIC, 1, __u32) | 76 | #define SW_SYNC_IOC_INC _IOW(SW_SYNC_IOC_MAGIC, 1, __u32) |
| 77 | #define SW_SYNC_GET_DEADLINE _IOWR(SW_SYNC_IOC_MAGIC, 2, \ | ||
| 78 | struct sw_sync_get_deadline) | ||
| 79 | |||
| 80 | |||
| 81 | #define SW_SYNC_HAS_DEADLINE_BIT DMA_FENCE_FLAG_USER_BITS | ||
| 61 | 82 | ||
| 62 | static const struct dma_fence_ops timeline_fence_ops; | 83 | static const struct dma_fence_ops timeline_fence_ops; |
| 63 | 84 | ||
| @@ -171,6 +192,22 @@ static void timeline_fence_timeline_value_str(struct dma_fence *fence, · 107 unmodified lines | |||
| 171 | snprintf(str, size, "%d", parent->value); | 192 | snprintf(str, size, "%d", parent->value); |
| 172 | } | 193 | } |
| 173 | 194 | ||
| 195 | static void timeline_fence_set_deadline(struct dma_fence *fence, ktime_t deadline) | ||
| 196 | { | ||
| 197 | struct sync_pt *pt = dma_fence_to_sync_pt(fence); | ||
| 198 | unsigned long flags; | ||
| 199 | |||
| 200 | spin_lock_irqsave(fence->lock, flags); | ||
| 201 | if (test_bit(SW_SYNC_HAS_DEADLINE_BIT, &fence->flags)) { | ||
| 202 | if (ktime_before(deadline, pt->deadline)) | ||
| 203 | pt->deadline = deadline; | ||
| 204 | } else { | ||
| 205 | pt->deadline = deadline; | ||
| 206 | __set_bit(SW_SYNC_HAS_DEADLINE_BIT, &fence->flags); | ||
| 207 | } | ||
| 208 | spin_unlock_irqrestore(fence->lock, flags); | ||
| 209 | } | ||
| 210 | |||
| 174 | static const struct dma_fence_ops timeline_fence_ops = { | 211 | static const struct dma_fence_ops timeline_fence_ops = { |
| 175 | .get_driver_name = timeline_fence_get_driver_name, | 212 | .get_driver_name = timeline_fence_get_driver_name, |
| 176 | .get_timeline_name = timeline_fence_get_timeline_name, | 213 | .get_timeline_name = timeline_fence_get_timeline_name, |
| @@ -179,6 +216,7 @@ static const struct dma_fence_ops timeline_fence_ops = { · 2 unmodified lines | |||
| 179 | .release = timeline_fence_release, | 216 | .release = timeline_fence_release, |
| 180 | .fence_value_str = timeline_fence_value_str, | 217 | .fence_value_str = timeline_fence_value_str, |
| 181 | .timeline_value_str = timeline_fence_timeline_value_str, | 218 | .timeline_value_str = timeline_fence_timeline_value_str, |
| 219 | .set_deadline = timeline_fence_set_deadline, | ||
| 182 | }; | 220 | }; |
| 183 | 221 | ||
| 184 | /** | 222 | /** |
| @@ -387,6 +425,47 @@ static long sw_sync_ioctl_inc(struct sync_timeline *obj, unsigned long arg) · 202 unmodified lines | |||
| 387 | return 0; | 425 | return 0; |
| 388 | } | 426 | } |
| 389 | 427 | ||
| 428 | static int sw_sync_ioctl_get_deadline(struct sync_timeline *obj, unsigned long arg) | ||
| 429 | { | ||
| 430 | struct sw_sync_get_deadline data; | ||
| 431 | struct dma_fence *fence; | ||
| 432 | unsigned long flags; | ||
| 433 | struct sync_pt *pt; | ||
| 434 | int ret = 0; | ||
| 435 | |||
| 436 | if (copy_from_user(&data, (void __user *)arg, sizeof(data))) | ||
| 437 | return -EFAULT; | ||
| 438 | |||
| 439 | if (data.deadline_ns || data.pad) | ||
| 440 | return -EINVAL; | ||
| 441 | |||
| 442 | fence = sync_file_get_fence(data.fence_fd); | ||
| 443 | if (!fence) | ||
| 444 | return -EINVAL; | ||
| 445 | |||
| 446 | pt = dma_fence_to_sync_pt(fence); | ||
| 447 | if (!pt) | ||
| 448 | return -EINVAL; | ||
| 449 | |||
| 450 | spin_lock_irqsave(fence->lock, flags); | ||
| 451 | if (test_bit(SW_SYNC_HAS_DEADLINE_BIT, &fence->flags)) { | ||
| 452 | data.deadline_ns = ktime_to_ns(pt->deadline); | ||
| 453 | } else { | ||
| 454 | ret = -ENOENT; | ||
| 455 | } | ||
| 456 | spin_unlock_irqrestore(fence->lock, flags); | ||
| 457 | |||
| 458 | dma_fence_put(fence); | ||
| 459 | |||
| 460 | if (ret) | ||
| 461 | return ret; | ||
| 462 | |||
| 463 | if (copy_to_user((void __user *)arg, &data, sizeof(data))) | ||
| 464 | return -EFAULT; | ||
| 465 | |||
| 466 | return 0; | ||
| 467 | } | ||
| 468 | |||
| 390 | static long sw_sync_ioctl(struct file *file, unsigned int cmd, | 469 | static long sw_sync_ioctl(struct file *file, unsigned int cmd, |
| 391 | unsigned long arg) | 470 | unsigned long arg) |
| 392 | { | 471 | { |
| @@ -399,6 +478,9 @@ static long sw_sync_ioctl(struct file *file, unsigned int cmd, · 6 unmodified lines | |||
| 399 | case SW_SYNC_IOC_INC: | 478 | case SW_SYNC_IOC_INC: |
| 400 | return sw_sync_ioctl_inc(obj, arg); | 479 | return sw_sync_ioctl_inc(obj, arg); |
| 401 | 480 | ||
| 481 | case SW_SYNC_GET_DEADLINE: | ||
| 482 | return sw_sync_ioctl_get_deadline(obj, arg); | ||
| 483 | |||
| 402 | default: | 484 | default: |
| 403 | return -ENOTTY; | 485 | return -ENOTTY; |
| 404 | } | 486 | } |
| @@ -55,11 +55,13 @@ static inline struct sync_timeline *dma_fence_parent(struct dma_fence *fence) · 54 unmodified lines | |||
| 55 | * @base: base fence object | 55 | * @base: base fence object |
| 56 | * @link: link on the sync timeline's list | 56 | * @link: link on the sync timeline's list |
| 57 | * @node: node in the sync timeline's tree | 57 | * @node: node in the sync timeline's tree |
| 58 | * @deadline: the earliest fence deadline hint | ||
| 58 | */ | 59 | */ |
| 59 | struct sync_pt { | 60 | struct sync_pt { |
| 60 | struct dma_fence base; | 61 | struct dma_fence base; |
| 61 | struct list_head link; | 62 | struct list_head link; |
| 62 | struct rb_node node; | 63 | struct rb_node node; |
| 64 | ktime_t deadline; | ||
| 63 | }; | 65 | }; |
| 64 | 66 | ||
| 65 | extern const struct file_operations sw_sync_debugfs_fops; | 67 | extern const struct file_operations sw_sync_debugfs_fops; |
| @@ -347,6 +347,22 @@ out: · 346 unmodified lines | |||
| 347 | return ret; | 347 | return ret; |
| 348 | } | 348 | } |
| 349 | 349 | ||
| 350 | static int sync_file_ioctl_set_deadline(struct sync_file *sync_file, | ||
| 351 | unsigned long arg) | ||
| 352 | { | ||
| 353 | struct sync_set_deadline ts; | ||
| 354 | |||
| 355 | if (copy_from_user(&ts, (void __user *)arg, sizeof(ts))) | ||
| 356 | return -EFAULT; | ||
| 357 | |||
| 358 | if (ts.pad) | ||
| 359 | return -EINVAL; | ||
| 360 | |||
| 361 | dma_fence_set_deadline(sync_file->fence, ns_to_ktime(ts.deadline_ns)); | ||
| 362 | |||
| 363 | return 0; | ||
| 364 | } | ||
| 365 | |||
| 350 | static long sync_file_ioctl(struct file *file, unsigned int cmd, | 366 | static long sync_file_ioctl(struct file *file, unsigned int cmd, |
| 351 | unsigned long arg) | 367 | unsigned long arg) |
| 352 | { | 368 | { |
| @@ -359,6 +375,9 @@ static long sync_file_ioctl(struct file *file, unsigned int cmd, · 6 unmodified lines | |||
| 359 | case SYNC_IOC_FILE_INFO: | 375 | case SYNC_IOC_FILE_INFO: |
| 360 | return sync_file_ioctl_fence_info(sync_file, arg); | 376 | return sync_file_ioctl_fence_info(sync_file, arg); |
| 361 | 377 | ||
| 378 | case SYNC_IOC_SET_DEADLINE: | ||
| 379 | return sync_file_ioctl_set_deadline(sync_file, arg); | ||
| 380 | |||
| 362 | default: | 381 | default: |
| 363 | return -ENOTTY; | 382 | return -ENOTTY; |
| 364 | } | 383 | } |
| @@ -1093,9 +1093,10 @@ int dpll_nl_pin_id_get_doit(struct sk_buff *skb, struct genl_info *info) · 1092 unmodified lines | |||
| 1093 | return -ENOMEM; | 1093 | return -ENOMEM; |
| 1094 | hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0, | 1094 | hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0, |
| 1095 | DPLL_CMD_PIN_ID_GET); | 1095 | DPLL_CMD_PIN_ID_GET); |
| 1096 | if (!hdr) | 1096 | if (!hdr) { |
| 1097 | nlmsg_free(msg); | ||
| 1097 | return -EMSGSIZE; | 1098 | return -EMSGSIZE; |
| 1098 | 1099 | } | |
| 1099 | pin = dpll_pin_find_from_nlattr(info); | 1100 | pin = dpll_pin_find_from_nlattr(info); |
| 1100 | if (!IS_ERR(pin)) { | 1101 | if (!IS_ERR(pin)) { |
| 1101 | ret = dpll_msg_add_pin_handle(msg, pin); | 1102 | ret = dpll_msg_add_pin_handle(msg, pin); |
| @@ -1123,8 +1124,10 @@ int dpll_nl_pin_get_doit(struct sk_buff *skb, struct genl_info *info) · 21 unmodified lines | |||
| 1123 | return -ENOMEM; | 1124 | return -ENOMEM; |
| 1124 | hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0, | 1125 | hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0, |
| 1125 | DPLL_CMD_PIN_GET); | 1126 | DPLL_CMD_PIN_GET); |
| 1126 | if (!hdr) | 1127 | if (!hdr) { |
| 1128 | nlmsg_free(msg); | ||
| 1127 | return -EMSGSIZE; | 1129 | return -EMSGSIZE; |
| 1130 | } | ||
| 1128 | ret = dpll_cmd_pin_get_one(msg, pin, info->extack); | 1131 | ret = dpll_cmd_pin_get_one(msg, pin, info->extack); |
| 1129 | if (ret) { | 1132 | if (ret) { |
| 1130 | nlmsg_free(msg); | 1133 | nlmsg_free(msg); |
| @@ -1256,8 +1259,10 @@ int dpll_nl_device_id_get_doit(struct sk_buff *skb, struct genl_info *info) · 125 unmodified lines | |||
| 1256 | return -ENOMEM; | 1259 | return -ENOMEM; |
| 1257 | hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0, | 1260 | hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0, |
| 1258 | DPLL_CMD_DEVICE_ID_GET); | 1261 | DPLL_CMD_DEVICE_ID_GET); |
| 1259 | if (!hdr) | 1262 | if (!hdr) { |
| 1263 | nlmsg_free(msg); | ||
| 1260 | return -EMSGSIZE; | 1264 | return -EMSGSIZE; |
| 1265 | } | ||
| 1261 | 1266 | ||
| 1262 | dpll = dpll_device_find_from_nlattr(info); | 1267 | dpll = dpll_device_find_from_nlattr(info); |
| 1263 | if (!IS_ERR(dpll)) { | 1268 | if (!IS_ERR(dpll)) { |
| @@ -1284,8 +1289,10 @@ int dpll_nl_device_get_doit(struct sk_buff *skb, struct genl_info *info) · 20 unmodified lines | |||
| 1284 | return -ENOMEM; | 1289 | return -ENOMEM; |
| 1285 | hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0, | 1290 | hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0, |
| 1286 | DPLL_CMD_DEVICE_GET); | 1291 | DPLL_CMD_DEVICE_GET); |
| 1287 | if (!hdr) | 1292 | if (!hdr) { |
| 1293 | nlmsg_free(msg); | ||
| 1288 | return -EMSGSIZE; | 1294 | return -EMSGSIZE; |
| 1295 | } | ||
| 1289 | 1296 | ||
| 1290 | ret = dpll_device_get_one(dpll, msg, info->extack); | 1297 | ret = dpll_device_get_one(dpll, msg, info->extack); |
| 1291 | if (ret) { | 1298 | if (ret) { |
| @@ -131,7 +131,7 @@ config RASPBERRYPI_FIRMWARE · 130 unmodified lines | |||
| 131 | 131 | ||
| 132 | config FW_CFG_SYSFS | 132 | config FW_CFG_SYSFS |
| 133 | tristate "QEMU fw_cfg device support in sysfs" | 133 | tristate "QEMU fw_cfg device support in sysfs" |
| 134 | depends on SYSFS && (ARM || ARM64 || PARISC || PPC_PMAC || SPARC || X86) | 134 | depends on SYSFS && (ARM || ARM64 || PARISC || PPC_PMAC || RISCV || SPARC || X86) |
| 135 | depends on HAS_IOPORT_MAP | 135 | depends on HAS_IOPORT_MAP |
| 136 | default n | 136 | default n |
| 137 | help | 137 | help |
| @@ -211,7 +211,7 @@ static void fw_cfg_io_cleanup(void) · 210 unmodified lines | |||
| 211 | 211 | ||
| 212 | /* arch-specific ctrl & data register offsets are not available in ACPI, DT */ | 212 | /* arch-specific ctrl & data register offsets are not available in ACPI, DT */ |
| 213 | #if !(defined(FW_CFG_CTRL_OFF) && defined(FW_CFG_DATA_OFF)) | 213 | #if !(defined(FW_CFG_CTRL_OFF) && defined(FW_CFG_DATA_OFF)) |
| 214 | # if (defined(CONFIG_ARM) || defined(CONFIG_ARM64)) | 214 | # if (defined(CONFIG_ARM) || defined(CONFIG_ARM64) || defined(CONFIG_RISCV)) |
| 215 | # define FW_CFG_CTRL_OFF 0x08 | 215 | # define FW_CFG_CTRL_OFF 0x08 |
| 216 | # define FW_CFG_DATA_OFF 0x00 | 216 | # define FW_CFG_DATA_OFF 0x00 |
| 217 | # define FW_CFG_DMA_OFF 0x10 | 217 | # define FW_CFG_DMA_OFF 0x10 |
| @@ -74,16 +74,17 @@ config DRM_KUNIT_TEST_HELPERS · 73 unmodified lines | |||
| 74 | 74 | ||
| 75 | config DRM_KUNIT_TEST | 75 | config DRM_KUNIT_TEST |
| 76 | tristate "KUnit tests for DRM" if !KUNIT_ALL_TESTS | 76 | tristate "KUnit tests for DRM" if !KUNIT_ALL_TESTS |
| 77 | depends on DRM && KUNIT | 77 | depends on DRM && KUNIT && MMU |
| 78 | select PRIME_NUMBERS | 78 | select DRM_BUDDY |
| 79 | select DRM_DISPLAY_DP_HELPER | 79 | select DRM_DISPLAY_DP_HELPER |
| 80 | select DRM_DISPLAY_HELPER | 80 | select DRM_DISPLAY_HELPER |
| 81 | select DRM_LIB_RANDOM | 81 | select DRM_EXEC |
| 82 | select DRM_KMS_HELPER | ||
| 83 | select DRM_BUDDY | ||
| 84 | select DRM_EXPORT_FOR_TESTS if m | 82 | select DRM_EXPORT_FOR_TESTS if m |
| 83 | select DRM_GEM_SHMEM_HELPER | ||
| 84 | select DRM_KMS_HELPER | ||
| 85 | select DRM_KUNIT_TEST_HELPERS | 85 | select DRM_KUNIT_TEST_HELPERS |
| 86 | select DRM_EXEC | 86 | select DRM_LIB_RANDOM |
| 87 | select PRIME_NUMBERS | ||
| 87 | default KUNIT_ALL_TESTS | 88 | default KUNIT_ALL_TESTS |
| 88 | help | 89 | help |
| 89 | This builds unit tests for DRM. This option is not useful for | 90 | This builds unit tests for DRM. This option is not useful for |
| @@ -394,6 +395,8 @@ source "drivers/gpu/drm/solomon/Kconfig" · 304 unmodified lines | |||
| 394 | 395 | ||
| 395 | source "drivers/gpu/drm/sprd/Kconfig" | 396 | source "drivers/gpu/drm/sprd/Kconfig" |
| 396 | 397 | ||
| 398 | source "drivers/gpu/drm/imagination/Kconfig" | ||
| 399 | |||
| 397 | config DRM_HYPERV | 400 | config DRM_HYPERV |
| 398 | tristate "DRM Support for Hyper-V synthetic video device" | 401 | tristate "DRM Support for Hyper-V synthetic video device" |
| 399 | depends on DRM && PCI && MMU && HYPERV | 402 | depends on DRM && PCI && MMU && HYPERV |
| @@ -407,27 +410,6 @@ config DRM_HYPERV · 7 unmodified lines | |||
| 407 | 410 | ||
| 408 | If M is selected the module will be called hyperv_drm. | 411 | If M is selected the module will be called hyperv_drm. |
| 409 | 412 | ||
| 410 | # Keep legacy drivers last | ||
| 411 | |||
| 412 | menuconfig DRM_LEGACY | ||
| 413 | bool "Enable legacy drivers (DANGEROUS)" | ||
| 414 | depends on DRM && MMU | ||
| 415 | help | ||
| 416 | Enable legacy DRI1 drivers. Those drivers expose unsafe and dangerous | ||
| 417 | APIs to user-space, which can be used to circumvent access | ||
| 418 | restrictions and other security measures. For backwards compatibility | ||
| 419 | those drivers are still available, but their use is highly | ||
| 420 | inadvisable and might harm your system. | ||
| 421 | |||
| 422 | You are recommended to use the safe modeset-only drivers instead, and | ||
| 423 | perform 3D emulation in user-space. | ||
| 424 | |||
| 425 | Unless you have strong reasons to go rogue, say "N". | ||
| 426 | |||
| 427 | if DRM_LEGACY | ||
| 428 | # leave here to list legacy drivers | ||
| 429 | endif # DRM_LEGACY | ||
| 430 | |||
| 431 | config DRM_EXPORT_FOR_TESTS | 413 | config DRM_EXPORT_FOR_TESTS |
| 432 | bool | 414 | bool |
| 433 | 415 | ||
| @@ -22,6 +22,7 @@ drm-y := \ · 21 unmodified lines | |||
| 22 | drm_drv.o \ | 22 | drm_drv.o \ |
| 23 | drm_dumb_buffers.o \ | 23 | drm_dumb_buffers.o \ |
| 24 | drm_edid.o \ | 24 | drm_edid.o \ |
| 25 | drm_eld.o \ | ||
| 25 | drm_encoder.o \ | 26 | drm_encoder.o \ |
| 26 | drm_file.o \ | 27 | drm_file.o \ |
| 27 | drm_fourcc.o \ | 28 | drm_fourcc.o \ |
| @@ -46,18 +47,6 @@ drm-y := \ · 18 unmodified lines | |||
| 46 | drm_vblank_work.o \ | 47 | drm_vblank_work.o \ |
| 47 | drm_vma_manager.o \ | 48 | drm_vma_manager.o \ |
| 48 | drm_writeback.o | 49 | drm_writeback.o |
| 49 | drm-$(CONFIG_DRM_LEGACY) += \ | ||
| 50 | drm_agpsupport.o \ | ||
| 51 | drm_bufs.o \ | ||
| 52 | drm_context.o \ | ||
| 53 | drm_dma.o \ | ||
| 54 | drm_hashtab.o \ | ||
| 55 | drm_irq.o \ | ||
| 56 | drm_legacy_misc.o \ | ||
| 57 | drm_lock.o \ | ||
| 58 | drm_memory.o \ | ||
| 59 | drm_scatter.o \ | ||
| 60 | drm_vm.o | ||
| 61 | drm-$(CONFIG_DRM_LIB_RANDOM) += lib/drm_random.o | 50 | drm-$(CONFIG_DRM_LIB_RANDOM) += lib/drm_random.o |
| 62 | drm-$(CONFIG_COMPAT) += drm_ioc32.o | 51 | drm-$(CONFIG_COMPAT) += drm_ioc32.o |
| 63 | drm-$(CONFIG_DRM_PANEL) += drm_panel.o | 52 | drm-$(CONFIG_DRM_PANEL) += drm_panel.o |
| @@ -198,3 +187,4 @@ obj-$(CONFIG_DRM_HYPERV) += hyperv/ · 134 unmodified lines | |||
| 198 | obj-y += solomon/ | 187 | obj-y += solomon/ |
| 199 | obj-$(CONFIG_DRM_SPRD) += sprd/ | 188 | obj-$(CONFIG_DRM_SPRD) += sprd/ |
| 200 | obj-$(CONFIG_DRM_LOONGSON) += loongson/ | 189 | obj-$(CONFIG_DRM_LOONGSON) += loongson/ |
| 190 | obj-$(CONFIG_DRM_POWERVR) += imagination/ | ||
| @@ -248,6 +248,7 @@ extern int amdgpu_umsch_mm; · 247 unmodified lines | |||
| 248 | extern int amdgpu_seamless; | 248 | extern int amdgpu_seamless; |
| 249 | 249 | ||
| 250 | extern int amdgpu_user_partt_mode; | 250 | extern int amdgpu_user_partt_mode; |
| 251 | extern int amdgpu_agp; | ||
| 251 | 252 | ||
| 252 | #define AMDGPU_VM_MAX_NUM_CTX 4096 | 253 | #define AMDGPU_VM_MAX_NUM_CTX 4096 |
| 253 | #define AMDGPU_SG_THRESHOLD (256*1024*1024) | 254 | #define AMDGPU_SG_THRESHOLD (256*1024*1024) |
| @@ -290,7 +290,7 @@ static int suspend_resume_compute_scheduler(struct amdgpu_device *adev, bool sus · 289 unmodified lines | |||
| 290 | for (i = 0; i < adev->gfx.num_compute_rings; i++) { | 290 | for (i = 0; i < adev->gfx.num_compute_rings; i++) { |
| 291 | struct amdgpu_ring *ring = &adev->gfx.compute_ring[i]; | 291 | struct amdgpu_ring *ring = &adev->gfx.compute_ring[i]; |
| 292 | 292 | ||
| 293 | if (!(ring && ring->sched.thread)) | 293 | if (!(ring && drm_sched_wqueue_ready(&ring->sched))) |
| 294 | continue; | 294 | continue; |
| 295 | 295 | ||
| 296 | /* stop secheduler and drain ring. */ | 296 | /* stop secheduler and drain ring. */ |
| @@ -207,7 +207,7 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p, · 206 unmodified lines | |||
| 207 | } | 207 | } |
| 208 | 208 | ||
| 209 | for (i = 0; i < p->nchunks; i++) { | 209 | for (i = 0; i < p->nchunks; i++) { |
| 210 | struct drm_amdgpu_cs_chunk __user **chunk_ptr = NULL; | 210 | struct drm_amdgpu_cs_chunk __user *chunk_ptr = NULL; |
| 211 | struct drm_amdgpu_cs_chunk user_chunk; | 211 | struct drm_amdgpu_cs_chunk user_chunk; |
| 212 | uint32_t __user *cdata; | 212 | uint32_t __user *cdata; |
| 213 | 213 | ||
| @@ -73,10 +73,10 @@ amdgpu_ctx_to_drm_sched_prio(int32_t ctx_prio) · 72 unmodified lines | |||
| 73 | return DRM_SCHED_PRIORITY_NORMAL; | 73 | return DRM_SCHED_PRIORITY_NORMAL; |
| 74 | 74 | ||
| 75 | case AMDGPU_CTX_PRIORITY_VERY_LOW: | 75 | case AMDGPU_CTX_PRIORITY_VERY_LOW: |
| 76 | return DRM_SCHED_PRIORITY_MIN; | 76 | return DRM_SCHED_PRIORITY_LOW; |
| 77 | 77 | ||
| 78 | case AMDGPU_CTX_PRIORITY_LOW: | 78 | case AMDGPU_CTX_PRIORITY_LOW: |
| 79 | return DRM_SCHED_PRIORITY_MIN; | 79 | return DRM_SCHED_PRIORITY_LOW; |
| 80 | 80 | ||
| 81 | case AMDGPU_CTX_PRIORITY_NORMAL: | 81 | case AMDGPU_CTX_PRIORITY_NORMAL: |
| 82 | return DRM_SCHED_PRIORITY_NORMAL; | 82 | return DRM_SCHED_PRIORITY_NORMAL; |
| @@ -1665,9 +1665,9 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused) · 1664 unmodified lines | |||
| 1665 | for (i = 0; i < AMDGPU_MAX_RINGS; i++) { | 1665 | for (i = 0; i < AMDGPU_MAX_RINGS; i++) { |
| 1666 | struct amdgpu_ring *ring = adev->rings[i]; | 1666 | struct amdgpu_ring *ring = adev->rings[i]; |
| 1667 | 1667 | ||
| 1668 | if (!ring || !ring->sched.thread) | 1668 | if (!ring || !drm_sched_wqueue_ready(&ring->sched)) |
| 1669 | continue; | 1669 | continue; |
| 1670 | kthread_park(ring->sched.thread); | 1670 | drm_sched_wqueue_stop(&ring->sched); |
| 1671 | } | 1671 | } |
| 1672 | 1672 | ||
| 1673 | seq_puts(m, "run ib test:\n"); | 1673 | seq_puts(m, "run ib test:\n"); |
| @@ -1681,9 +1681,9 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused) · 7 unmodified lines | |||
| 1681 | for (i = 0; i < AMDGPU_MAX_RINGS; i++) { | 1681 | for (i = 0; i < AMDGPU_MAX_RINGS; i++) { |
| 1682 | struct amdgpu_ring *ring = adev->rings[i]; | 1682 | struct amdgpu_ring *ring = adev->rings[i]; |
| 1683 | 1683 | ||
| 1684 | if (!ring || !ring->sched.thread) | 1684 | if (!ring || !drm_sched_wqueue_ready(&ring->sched)) |
| 1685 | continue; | 1685 | continue; |
| 1686 | kthread_unpark(ring->sched.thread); | 1686 | drm_sched_wqueue_start(&ring->sched); |
| 1687 | } | 1687 | } |
| 1688 | 1688 | ||
| 1689 | up_write(&adev->reset_domain->sem); | 1689 | up_write(&adev->reset_domain->sem); |
| @@ -1903,7 +1903,8 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) · 213 unmodified lines | |||
| 1903 | 1903 | ||
| 1904 | ring = adev->rings[val]; | 1904 | ring = adev->rings[val]; |
| 1905 | 1905 | ||
| 1906 | if (!ring || !ring->funcs->preempt_ib || !ring->sched.thread) | 1906 | if (!ring || !ring->funcs->preempt_ib || |
| 1907 | !drm_sched_wqueue_ready(&ring->sched)) | ||
| 1907 | return -EINVAL; | 1908 | return -EINVAL; |
| 1908 | 1909 | ||
| 1909 | /* the last preemption failed */ | 1910 | /* the last preemption failed */ |
| @@ -1921,7 +1922,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) · 11 unmodified lines | |||
| 1921 | goto pro_end; | 1922 | goto pro_end; |
| 1922 | 1923 | ||
| 1923 | /* stop the scheduler */ | 1924 | /* stop the scheduler */ |
| 1924 | kthread_park(ring->sched.thread); | 1925 | drm_sched_wqueue_stop(&ring->sched); |
| 1925 | 1926 | ||
| 1926 | /* preempt the IB */ | 1927 | /* preempt the IB */ |
| 1927 | r = amdgpu_ring_preempt_ib(ring); | 1928 | r = amdgpu_ring_preempt_ib(ring); |
| @@ -1955,7 +1956,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) · 27 unmodified lines | |||
| 1955 | 1956 | ||
| 1956 | failure: | 1957 | failure: |
| 1957 | /* restart the scheduler */ | 1958 | /* restart the scheduler */ |
| 1958 | kthread_unpark(ring->sched.thread); | 1959 | drm_sched_wqueue_start(&ring->sched); |
| 1959 | 1960 | ||
| 1960 | up_read(&adev->reset_domain->sem); | 1961 | up_read(&adev->reset_domain->sem); |
| 1961 | 1962 | ||
| @@ -2573,7 +2573,7 @@ static int amdgpu_device_init_schedulers(struct amdgpu_device *adev) · 2572 unmodified lines | |||
| 2573 | break; | 2573 | break; |
| 2574 | } | 2574 | } |
| 2575 | 2575 | ||
| 2576 | r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, | 2576 | r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, NULL, |
| 2577 | DRM_SCHED_PRIORITY_COUNT, | 2577 | DRM_SCHED_PRIORITY_COUNT, |
| 2578 | ring->num_hw_submission, 0, | 2578 | ring->num_hw_submission, 0, |
| 2579 | timeout, adev->reset_domain->wq, | 2579 | timeout, adev->reset_domain->wq, |
| @@ -4964,7 +4964,7 @@ bool amdgpu_device_has_job_running(struct amdgpu_device *adev) · 2384 unmodified lines | |||
| 4964 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { | 4964 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { |
| 4965 | struct amdgpu_ring *ring = adev->rings[i]; | 4965 | struct amdgpu_ring *ring = adev->rings[i]; |
| 4966 | 4966 | ||
| 4967 | if (!ring || !ring->sched.thread) | 4967 | if (!ring || !drm_sched_wqueue_ready(&ring->sched)) |
| 4968 | continue; | 4968 | continue; |
| 4969 | 4969 | ||
| 4970 | spin_lock(&ring->sched.job_list_lock); | 4970 | spin_lock(&ring->sched.job_list_lock); |
| @@ -5103,7 +5103,7 @@ int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev, · 132 unmodified lines | |||
| 5103 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { | 5103 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { |
| 5104 | struct amdgpu_ring *ring = adev->rings[i]; | 5104 | struct amdgpu_ring *ring = adev->rings[i]; |
| 5105 | 5105 | ||
| 5106 | if (!ring || !ring->sched.thread) | 5106 | if (!ring || !drm_sched_wqueue_ready(&ring->sched)) |
| 5107 | continue; | 5107 | continue; |
| 5108 | 5108 | ||
| 5109 | /* Clear job fence from fence drv to avoid force_completion | 5109 | /* Clear job fence from fence drv to avoid force_completion |
| @@ -5592,7 +5592,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 482 unmodified lines | |||
| 5592 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { | 5592 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { |
| 5593 | struct amdgpu_ring *ring = tmp_adev->rings[i]; | 5593 | struct amdgpu_ring *ring = tmp_adev->rings[i]; |
| 5594 | 5594 | ||
| 5595 | if (!ring || !ring->sched.thread) | 5595 | if (!ring || !drm_sched_wqueue_ready(&ring->sched)) |
| 5596 | continue; | 5596 | continue; |
| 5597 | 5597 | ||
| 5598 | drm_sched_stop(&ring->sched, job ? &job->base : NULL); | 5598 | drm_sched_stop(&ring->sched, job ? &job->base : NULL); |
| @@ -5668,7 +5668,7 @@ skip_hw_reset: · 69 unmodified lines | |||
| 5668 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { | 5668 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { |
| 5669 | struct amdgpu_ring *ring = tmp_adev->rings[i]; | 5669 | struct amdgpu_ring *ring = tmp_adev->rings[i]; |
| 5670 | 5670 | ||
| 5671 | if (!ring || !ring->sched.thread) | 5671 | if (!ring || !drm_sched_wqueue_ready(&ring->sched)) |
| 5672 | continue; | 5672 | continue; |
| 5673 | 5673 | ||
| 5674 | drm_sched_start(&ring->sched, true); | 5674 | drm_sched_start(&ring->sched, true); |
| @@ -5991,7 +5991,7 @@ pci_ers_result_t amdgpu_pci_error_detected(struct pci_dev *pdev, pci_channel_sta · 316 unmodified lines | |||
| 5991 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { | 5991 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { |
| 5992 | struct amdgpu_ring *ring = adev->rings[i]; | 5992 | struct amdgpu_ring *ring = adev->rings[i]; |
| 5993 | 5993 | ||
| 5994 | if (!ring || !ring->sched.thread) | 5994 | if (!ring || !drm_sched_wqueue_ready(&ring->sched)) |
| 5995 | continue; | 5995 | continue; |
| 5996 | 5996 | ||
| 5997 | drm_sched_stop(&ring->sched, NULL); | 5997 | drm_sched_stop(&ring->sched, NULL); |
| @@ -6119,7 +6119,7 @@ void amdgpu_pci_resume(struct pci_dev *pdev) · 121 unmodified lines | |||
| 6119 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { | 6119 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { |
| 6120 | struct amdgpu_ring *ring = adev->rings[i]; | 6120 | struct amdgpu_ring *ring = adev->rings[i]; |
| 6121 | 6121 | ||
| 6122 | if (!ring || !ring->sched.thread) | 6122 | if (!ring || !drm_sched_wqueue_ready(&ring->sched)) |
| 6123 | continue; | 6123 | continue; |
| 6124 | 6124 | ||
| 6125 | drm_sched_start(&ring->sched, true); | 6125 | drm_sched_start(&ring->sched, true); |
| @@ -207,6 +207,7 @@ int amdgpu_user_partt_mode = AMDGPU_AUTO_COMPUTE_PARTITION_MODE; · 206 unmodified lines | |||
| 207 | int amdgpu_umsch_mm; | 207 | int amdgpu_umsch_mm; |
| 208 | int amdgpu_seamless = -1; /* auto */ | 208 | int amdgpu_seamless = -1; /* auto */ |
| 209 | uint amdgpu_debug_mask; | 209 | uint amdgpu_debug_mask; |
| 210 | int amdgpu_agp = -1; /* auto */ | ||
| 210 | 211 | ||
| 211 | static void amdgpu_drv_delayed_reset_work_handler(struct work_struct *work); | 212 | static void amdgpu_drv_delayed_reset_work_handler(struct work_struct *work); |
| 212 | 213 | ||
| @@ -961,6 +962,15 @@ module_param_named(seamless, amdgpu_seamless, int, 0444); · 748 unmodified lines | |||
| 961 | MODULE_PARM_DESC(debug_mask, "debug options for amdgpu, disabled by default"); | 962 | MODULE_PARM_DESC(debug_mask, "debug options for amdgpu, disabled by default"); |
| 962 | module_param_named(debug_mask, amdgpu_debug_mask, uint, 0444); | 963 | module_param_named(debug_mask, amdgpu_debug_mask, uint, 0444); |
| 963 | 964 | ||
| 965 | /** | ||
| 966 | * DOC: agp (int) | ||
| 967 | * Enable the AGP aperture. This provides an aperture in the GPU's internal | ||
| 968 | * address space for direct access to system memory. Note that these accesses | ||
| 969 | * are non-snooped, so they are only used for access to uncached memory. | ||
| 970 | */ | ||
| 971 | MODULE_PARM_DESC(agp, "AGP (-1 = auto (default), 0 = disable, 1 = enable)"); | ||
| 972 | module_param_named(agp, amdgpu_agp, int, 0444); | ||
| 973 | |||
| 964 | /* These devices are not supported by amdgpu. | 974 | /* These devices are not supported by amdgpu. |
| 965 | * They are supported by the mach64, r128, radeon drivers | 975 | * They are supported by the mach64, r128, radeon drivers |
| 966 | */ | 976 | */ |
| @@ -115,7 +115,7 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 114 unmodified lines | |||
| 115 | if (!entity) | 115 | if (!entity) |
| 116 | return 0; | 116 | return 0; |
| 117 | 117 | ||
| 118 | return drm_sched_job_init(&(*job)->base, entity, owner); | 118 | return drm_sched_job_init(&(*job)->base, entity, 1, owner); |
| 119 | } | 119 | } |
| 120 | 120 | ||
| 121 | int amdgpu_job_alloc_with_ib(struct amdgpu_device *adev, | 121 | int amdgpu_job_alloc_with_ib(struct amdgpu_device *adev, |
| @@ -325,7 +325,7 @@ void amdgpu_job_stop_all_jobs_on_sched(struct drm_gpu_scheduler *sched) · 203 unmodified lines | |||
| 325 | int i; | 325 | int i; |
| 326 | 326 | ||
| 327 | /* Signal all jobs not yet scheduled */ | 327 | /* Signal all jobs not yet scheduled */ |
| 328 | for (i = sched->num_rqs - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) { | 328 | for (i = DRM_SCHED_PRIORITY_KERNEL; i < sched->num_rqs; i++) { |
| 329 | struct drm_sched_rq *rq = sched->sched_rq[i]; | 329 | struct drm_sched_rq *rq = sched->sched_rq[i]; |
| 330 | spin_lock(&rq->lock); | 330 | spin_lock(&rq->lock); |
| 331 | list_for_each_entry(s_entity, &rq->entities, list) { | 331 | list_for_each_entry(s_entity, &rq->entities, list) { |
| @@ -1473,6 +1473,11 @@ int psp_xgmi_get_topology_info(struct psp_context *psp, · 1472 unmodified lines | |||
| 1473 | topology->nodes[i].num_links = (requires_reflection && topology->nodes[i].num_links) ? | 1473 | topology->nodes[i].num_links = (requires_reflection && topology->nodes[i].num_links) ? |
| 1474 | topology->nodes[i].num_links : node_num_links; | 1474 | topology->nodes[i].num_links : node_num_links; |
| 1475 | } | 1475 | } |
| 1476 | /* popluate the connected port num info if supported and available */ | ||
| 1477 | if (ta_port_num_support && topology->nodes[i].num_links) { | ||
| 1478 | memcpy(topology->nodes[i].port_num, link_extend_info_output->nodes[i].port_num, | ||
| 1479 | sizeof(struct xgmi_connected_port_num) * TA_XGMI__MAX_PORT_NUM); | ||
| 1480 | } | ||
| 1476 | 1481 | ||
| 1477 | /* reflect the topology information for bi-directionality */ | 1482 | /* reflect the topology information for bi-directionality */ |
| 1478 | if (requires_reflection && topology->nodes[i].num_hops) | 1483 | if (requires_reflection && topology->nodes[i].num_hops) |
| @@ -150,6 +150,7 @@ struct psp_xgmi_node_info { · 149 unmodified lines | |||
| 150 | uint8_t is_sharing_enabled; | 150 | uint8_t is_sharing_enabled; |
| 151 | enum ta_xgmi_assigned_sdma_engine sdma_engine; | 151 | enum ta_xgmi_assigned_sdma_engine sdma_engine; |
| 152 | uint8_t num_links; | 152 | uint8_t num_links; |
| 153 | struct xgmi_connected_port_num port_num[TA_XGMI__MAX_PORT_NUM]; | ||
| 153 | }; | 154 | }; |
| 154 | 155 | ||
| 155 | struct psp_xgmi_topology_info { | 156 | struct psp_xgmi_topology_info { |
| @@ -1188,7 +1188,7 @@ static int amdgpu_ras_query_error_status_helper(struct amdgpu_device *adev, · 1187 unmodified lines | |||
| 1188 | } | 1188 | } |
| 1189 | 1189 | ||
| 1190 | if (block_obj->hw_ops->query_ras_error_count) | 1190 | if (block_obj->hw_ops->query_ras_error_count) |
| 1191 | block_obj->hw_ops->query_ras_error_count(adev, &err_data); | 1191 | block_obj->hw_ops->query_ras_error_count(adev, err_data); |
| 1192 | 1192 | ||
| 1193 | if ((info->head.block == AMDGPU_RAS_BLOCK__SDMA) || | 1193 | if ((info->head.block == AMDGPU_RAS_BLOCK__SDMA) || |
| 1194 | (info->head.block == AMDGPU_RAS_BLOCK__GFX) || | 1194 | (info->head.block == AMDGPU_RAS_BLOCK__GFX) || |
| @@ -398,6 +398,7 @@ int amdgpu_uvd_sw_fini(struct amdgpu_device *adev) · 397 unmodified lines | |||
| 398 | * amdgpu_uvd_entity_init - init entity | 398 | * amdgpu_uvd_entity_init - init entity |
| 399 | * | 399 | * |
| 400 | * @adev: amdgpu_device pointer | 400 | * @adev: amdgpu_device pointer |
| 401 | * @ring: amdgpu_ring pointer to check | ||
| 401 | * | 402 | * |
| 402 | * Initialize the entity used for handle management in the kernel driver. | 403 | * Initialize the entity used for handle management in the kernel driver. |
| 403 | */ | 404 | */ |
| @@ -230,6 +230,7 @@ int amdgpu_vce_sw_fini(struct amdgpu_device *adev) · 229 unmodified lines | |||
| 230 | * amdgpu_vce_entity_init - init entity | 230 | * amdgpu_vce_entity_init - init entity |
| 231 | * | 231 | * |
| 232 | * @adev: amdgpu_device pointer | 232 | * @adev: amdgpu_device pointer |
| 233 | * @ring: amdgpu_ring pointer to check | ||
| 233 | * | 234 | * |
| 234 | * Initialize the entity used for handle management in the kernel driver. | 235 | * Initialize the entity used for handle management in the kernel driver. |
| 235 | */ | 236 | */ |
| @@ -675,7 +675,7 @@ static void gmc_v10_0_vram_gtt_location(struct amdgpu_device *adev, · 674 unmodified lines | |||
| 675 | amdgpu_gmc_set_agp_default(adev, mc); | 675 | amdgpu_gmc_set_agp_default(adev, mc); |
| 676 | amdgpu_gmc_vram_location(adev, &adev->gmc, base); | 676 | amdgpu_gmc_vram_location(adev, &adev->gmc, base); |
| 677 | amdgpu_gmc_gart_location(adev, mc, AMDGPU_GART_PLACEMENT_BEST_FIT); | 677 | amdgpu_gmc_gart_location(adev, mc, AMDGPU_GART_PLACEMENT_BEST_FIT); |
| 678 | if (!amdgpu_sriov_vf(adev)) | 678 | if (!amdgpu_sriov_vf(adev) && (amdgpu_agp == 1)) |
| 679 | amdgpu_gmc_agp_location(adev, mc); | 679 | amdgpu_gmc_agp_location(adev, mc); |
| 680 | 680 | ||
| 681 | /* base offset of vram pages */ | 681 | /* base offset of vram pages */ |
| @@ -640,8 +640,9 @@ static void gmc_v11_0_vram_gtt_location(struct amdgpu_device *adev, · 639 unmodified lines | |||
| 640 | amdgpu_gmc_set_agp_default(adev, mc); | 640 | amdgpu_gmc_set_agp_default(adev, mc); |
| 641 | amdgpu_gmc_vram_location(adev, &adev->gmc, base); | 641 | amdgpu_gmc_vram_location(adev, &adev->gmc, base); |
| 642 | amdgpu_gmc_gart_location(adev, mc, AMDGPU_GART_PLACEMENT_HIGH); | 642 | amdgpu_gmc_gart_location(adev, mc, AMDGPU_GART_PLACEMENT_HIGH); |
| 643 | if (!amdgpu_sriov_vf(adev) || | 643 | if (!amdgpu_sriov_vf(adev) && |
| 644 | (amdgpu_ip_version(adev, GC_HWIP, 0) < IP_VERSION(11, 5, 0))) | 644 | (amdgpu_ip_version(adev, GC_HWIP, 0) < IP_VERSION(11, 5, 0)) && |
| 645 | (amdgpu_agp == 1)) | ||
| 645 | amdgpu_gmc_agp_location(adev, mc); | 646 | amdgpu_gmc_agp_location(adev, mc); |
| 646 | 647 | ||
| 647 | /* base offset of vram pages */ | 648 | /* base offset of vram pages */ |
| @@ -1630,7 +1630,7 @@ static void gmc_v9_0_vram_gtt_location(struct amdgpu_device *adev, · 1629 unmodified lines | |||
| 1630 | } else { | 1630 | } else { |
| 1631 | amdgpu_gmc_vram_location(adev, mc, base); | 1631 | amdgpu_gmc_vram_location(adev, mc, base); |
| 1632 | amdgpu_gmc_gart_location(adev, mc, AMDGPU_GART_PLACEMENT_BEST_FIT); | 1632 | amdgpu_gmc_gart_location(adev, mc, AMDGPU_GART_PLACEMENT_BEST_FIT); |
| 1633 | if (!amdgpu_sriov_vf(adev)) | 1633 | if (!amdgpu_sriov_vf(adev) && (amdgpu_agp == 1)) |
| 1634 | amdgpu_gmc_agp_location(adev, mc); | 1634 | amdgpu_gmc_agp_location(adev, mc); |
| 1635 | } | 1635 | } |
| 1636 | /* base offset of vram pages */ | 1636 | /* base offset of vram pages */ |
| @@ -2170,8 +2170,6 @@ static int gmc_v9_0_sw_fini(void *handle) · 533 unmodified lines | |||
| 2170 | 2170 | ||
| 2171 | if (amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 4, 3)) | 2171 | if (amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 4, 3)) |
| 2172 | amdgpu_gmc_sysfs_fini(adev); | 2172 | amdgpu_gmc_sysfs_fini(adev); |
| 2173 | adev->gmc.num_mem_partitions = 0; | ||
| 2174 | kfree(adev->gmc.mem_partitions); | ||
| 2175 | 2173 | ||
| 2176 | amdgpu_gmc_ras_fini(adev); | 2174 | amdgpu_gmc_ras_fini(adev); |
| 2177 | amdgpu_gem_force_release(adev); | 2175 | amdgpu_gem_force_release(adev); |
| @@ -2185,6 +2183,9 @@ static int gmc_v9_0_sw_fini(void *handle) · 7 unmodified lines | |||
| 2185 | amdgpu_bo_free_kernel(&adev->gmc.pdb0_bo, NULL, &adev->gmc.ptr_pdb0); | 2183 | amdgpu_bo_free_kernel(&adev->gmc.pdb0_bo, NULL, &adev->gmc.ptr_pdb0); |
| 2186 | amdgpu_bo_fini(adev); | 2184 | amdgpu_bo_fini(adev); |
| 2187 | 2185 | ||
| 2186 | adev->gmc.num_mem_partitions = 0; | ||
| 2187 | kfree(adev->gmc.mem_partitions); | ||
| 2188 | |||
| 2188 | return 0; | 2189 | return 0; |
| 2189 | } | 2190 | } |
| 2190 | 2191 | ||
| @@ -130,6 +130,9 @@ static void mmhub_v1_8_init_system_aperture_regs(struct amdgpu_device *adev) · 129 unmodified lines | |||
| 130 | uint64_t value; | 130 | uint64_t value; |
| 131 | int i; | 131 | int i; |
| 132 | 132 | ||
| 133 | if (amdgpu_sriov_vf(adev)) | ||
| 134 | return; | ||
| 135 | |||
| 133 | inst_mask = adev->aid_mask; | 136 | inst_mask = adev->aid_mask; |
| 134 | for_each_inst(i, inst_mask) { | 137 | for_each_inst(i, inst_mask) { |
| 135 | /* Program the AGP BAR */ | 138 | /* Program the AGP BAR */ |
| @@ -139,9 +142,6 @@ static void mmhub_v1_8_init_system_aperture_regs(struct amdgpu_device *adev) · 3 unmodified lines | |||
| 139 | WREG32_SOC15(MMHUB, i, regMC_VM_AGP_TOP, | 142 | WREG32_SOC15(MMHUB, i, regMC_VM_AGP_TOP, |
| 140 | adev->gmc.agp_end >> 24); | 143 | adev->gmc.agp_end >> 24); |
| 141 | 144 | ||
| 142 | if (amdgpu_sriov_vf(adev)) | ||
| 143 | return; | ||
| 144 | |||
| 145 | /* Program the system aperture low logical page number. */ | 145 | /* Program the system aperture low logical page number. */ |
| 146 | WREG32_SOC15(MMHUB, i, regMC_VM_SYSTEM_APERTURE_LOW_ADDR, | 146 | WREG32_SOC15(MMHUB, i, regMC_VM_SYSTEM_APERTURE_LOW_ADDR, |
| 147 | min(adev->gmc.fb_start, adev->gmc.agp_start) >> 18); | 147 | min(adev->gmc.fb_start, adev->gmc.agp_start) >> 18); |
| @@ -85,12 +85,13 @@ · 84 unmodified lines | |||
| 85 | #include <drm/drm_atomic_uapi.h> | 85 | #include <drm/drm_atomic_uapi.h> |
| 86 | #include <drm/drm_atomic_helper.h> | 86 | #include <drm/drm_atomic_helper.h> |
| 87 | #include <drm/drm_blend.h> | 87 | #include <drm/drm_blend.h> |
| 88 | #include <drm/drm_fixed.h> | ||
| 88 | #include <drm/drm_fourcc.h> | 89 | #include <drm/drm_fourcc.h> |
| 89 | #include <drm/drm_edid.h> | 90 | #include <drm/drm_edid.h> |
| 91 | #include <drm/drm_eld.h> | ||
| 90 | #include <drm/drm_vblank.h> | 92 | #include <drm/drm_vblank.h> |
| 91 | #include <drm/drm_audio_component.h> | 93 | #include <drm/drm_audio_component.h> |
| 92 | #include <drm/drm_gem_atomic_helper.h> | 94 | #include <drm/drm_gem_atomic_helper.h> |
| 93 | #include <drm/drm_plane_helper.h> | ||
| 94 | 95 | ||
| 95 | #include <acpi/video.h> | 96 | #include <acpi/video.h> |
| 96 | 97 | ||
| @@ -2079,7 +2080,7 @@ static int dm_dmub_sw_init(struct amdgpu_device *adev) · 1982 unmodified lines | |||
| 2079 | struct dmub_srv_create_params create_params; | 2080 | struct dmub_srv_create_params create_params; |
| 2080 | struct dmub_srv_region_params region_params; | 2081 | struct dmub_srv_region_params region_params; |
| 2081 | struct dmub_srv_region_info region_info; | 2082 | struct dmub_srv_region_info region_info; |
| 2082 | struct dmub_srv_fb_params fb_params; | 2083 | struct dmub_srv_memory_params memory_params; |
| 2083 | struct dmub_srv_fb_info *fb_info; | 2084 | struct dmub_srv_fb_info *fb_info; |
| 2084 | struct dmub_srv *dmub_srv; | 2085 | struct dmub_srv *dmub_srv; |
| 2085 | const struct dmcub_firmware_header_v1_0 *hdr; | 2086 | const struct dmcub_firmware_header_v1_0 *hdr; |
| @@ -2182,6 +2183,7 @@ static int dm_dmub_sw_init(struct amdgpu_device *adev) · 96 unmodified lines | |||
| 2182 | adev->dm.dmub_fw->data + | 2183 | adev->dm.dmub_fw->data + |
| 2183 | le32_to_cpu(hdr->header.ucode_array_offset_bytes) + | 2184 | le32_to_cpu(hdr->header.ucode_array_offset_bytes) + |
| 2184 | PSP_HEADER_BYTES; | 2185 | PSP_HEADER_BYTES; |
| 2186 | region_params.is_mailbox_in_inbox = false; | ||
| 2185 | 2187 | ||
| 2186 | status = dmub_srv_calc_region_info(dmub_srv, ®ion_params, | 2188 | status = dmub_srv_calc_region_info(dmub_srv, ®ion_params, |
| 2187 | ®ion_info); | 2189 | ®ion_info); |
| @@ -2205,10 +2207,10 @@ static int dm_dmub_sw_init(struct amdgpu_device *adev) · 17 unmodified lines | |||
| 2205 | return r; | 2207 | return r; |
| 2206 | 2208 | ||
| 2207 | /* Rebase the regions on the framebuffer address. */ | 2209 | /* Rebase the regions on the framebuffer address. */ |
| 2208 | memset(&fb_params, 0, sizeof(fb_params)); | 2210 | memset(&memory_params, 0, sizeof(memory_params)); |
| 2209 | fb_params.cpu_addr = adev->dm.dmub_bo_cpu_addr; | 2211 | memory_params.cpu_fb_addr = adev->dm.dmub_bo_cpu_addr; |
| 2210 | fb_params.gpu_addr = adev->dm.dmub_bo_gpu_addr; | 2212 | memory_params.gpu_fb_addr = adev->dm.dmub_bo_gpu_addr; |
| 2211 | fb_params.region_info = ®ion_info; | 2213 | memory_params.region_info = ®ion_info; |
| 2212 | 2214 | ||
| 2213 | adev->dm.dmub_fb_info = | 2215 | adev->dm.dmub_fb_info = |
| 2214 | kzalloc(sizeof(*adev->dm.dmub_fb_info), GFP_KERNEL); | 2216 | kzalloc(sizeof(*adev->dm.dmub_fb_info), GFP_KERNEL); |
| @@ -2220,7 +2222,7 @@ static int dm_dmub_sw_init(struct amdgpu_device *adev) · 5 unmodified lines | |||
| 2220 | return -ENOMEM; | 2222 | return -ENOMEM; |
| 2221 | } | 2223 | } |
| 2222 | 2224 | ||
| 2223 | status = dmub_srv_calc_fb_info(dmub_srv, &fb_params, fb_info); | 2225 | status = dmub_srv_calc_mem_info(dmub_srv, &memory_params, fb_info); |
| 2224 | if (status != DMUB_STATUS_OK) { | 2226 | if (status != DMUB_STATUS_OK) { |
| 2225 | DRM_ERROR("Error calculating DMUB FB info: %d\n", status); | 2227 | DRM_ERROR("Error calculating DMUB FB info: %d\n", status); |
| 2226 | return -EINVAL; | 2228 | return -EINVAL; |
| @@ -6908,8 +6910,8 @@ static int dm_encoder_helper_atomic_check(struct drm_encoder *encoder, · 4681 unmodified lines | |||
| 6908 | if (IS_ERR(mst_state)) | 6910 | if (IS_ERR(mst_state)) |
| 6909 | return PTR_ERR(mst_state); | 6911 | return PTR_ERR(mst_state); |
| 6910 | 6912 | ||
| 6911 | if (!mst_state->pbn_div) | 6913 | if (!mst_state->pbn_div.full) |
| 6912 | mst_state->pbn_div = dm_mst_get_pbn_divider(aconnector->mst_root->dc_link); | 6914 | mst_state->pbn_div.full = dfixed_const(dm_mst_get_pbn_divider(aconnector->mst_root->dc_link)); |
| 6913 | 6915 | ||
| 6914 | if (!state->duplicated) { | 6916 | if (!state->duplicated) { |
| 6915 | int max_bpc = conn_state->max_requested_bpc; | 6917 | int max_bpc = conn_state->max_requested_bpc; |
| @@ -6921,7 +6923,7 @@ static int dm_encoder_helper_atomic_check(struct drm_encoder *encoder, · 5 unmodified lines | |||
| 6921 | max_bpc); | 6923 | max_bpc); |
| 6922 | bpp = convert_dc_color_depth_into_bpc(color_depth) * 3; | 6924 | bpp = convert_dc_color_depth_into_bpc(color_depth) * 3; |
| 6923 | clock = adjusted_mode->clock; | 6925 | clock = adjusted_mode->clock; |
| 6924 | dm_new_connector_state->pbn = drm_dp_calc_pbn_mode(clock, bpp, false); | 6926 | dm_new_connector_state->pbn = drm_dp_calc_pbn_mode(clock, bpp << 4); |
| 6925 | } | 6927 | } |
| 6926 | 6928 | ||
| 6927 | dm_new_connector_state->vcpi_slots = | 6929 | dm_new_connector_state->vcpi_slots = |
| @@ -7481,6 +7483,9 @@ static int amdgpu_dm_i2c_xfer(struct i2c_adapter *i2c_adap, · 553 unmodified lines | |||
| 7481 | int i; | 7483 | int i; |
| 7482 | int result = -EIO; | 7484 | int result = -EIO; |
| 7483 | 7485 | ||
| 7486 | if (!ddc_service->ddc_pin || !ddc_service->ddc_pin->hw_info.hw_supported) | ||
| 7487 | return result; | ||
| 7488 | |||
| 7484 | cmd.payloads = kcalloc(num, sizeof(struct i2c_payload), GFP_KERNEL); | 7489 | cmd.payloads = kcalloc(num, sizeof(struct i2c_payload), GFP_KERNEL); |
| 7485 | 7490 | ||
| 7486 | if (!cmd.payloads) | 7491 | if (!cmd.payloads) |
| @@ -9603,14 +9608,14 @@ static bool should_reset_plane(struct drm_atomic_state *state, · 2116 unmodified lines | |||
| 9603 | struct drm_plane *other; | 9608 | struct drm_plane *other; |
| 9604 | struct drm_plane_state *old_other_state, *new_other_state; | 9609 | struct drm_plane_state *old_other_state, *new_other_state; |
| 9605 | struct drm_crtc_state *new_crtc_state; | 9610 | struct drm_crtc_state *new_crtc_state; |
| 9611 | struct amdgpu_device *adev = drm_to_adev(plane->dev); | ||
| 9606 | int i; | 9612 | int i; |
| 9607 | 9613 | ||
| 9608 | /* | 9614 | /* |
| 9609 | * TODO: Remove this hack once the checks below are sufficient | 9615 | * TODO: Remove this hack for all asics once it proves that the |
| 9610 | * enough to determine when we need to reset all the planes on | 9616 | * fast updates works fine on DCN3.2+. |
| 9611 | * the stream. | ||
| 9612 | */ | 9617 | */ |
| 9613 | if (state->allow_modeset) | 9618 | if (adev->ip_versions[DCE_HWIP][0] < IP_VERSION(3, 2, 0) && state->allow_modeset) |
| 9614 | return true; | 9619 | return true; |
| 9615 | 9620 | ||
| 9616 | /* Exit early if we know that we're adding or removing the plane. */ | 9621 | /* Exit early if we know that we're adding or removing the plane. */ |
| @@ -31,6 +31,7 @@ · 30 unmodified lines | |||
| 31 | #include <drm/drm_probe_helper.h> | 31 | #include <drm/drm_probe_helper.h> |
| 32 | #include <drm/amdgpu_drm.h> | 32 | #include <drm/amdgpu_drm.h> |
| 33 | #include <drm/drm_edid.h> | 33 | #include <drm/drm_edid.h> |
| 34 | #include <drm/drm_fixed.h> | ||
| 34 | 35 | ||
| 35 | #include "dm_services.h" | 36 | #include "dm_services.h" |
| 36 | #include "amdgpu.h" | 37 | #include "amdgpu.h" |
| @@ -210,7 +211,7 @@ static void dm_helpers_construct_old_payload( · 173 unmodified lines | |||
| 210 | struct drm_dp_mst_atomic_payload *old_payload) | 211 | struct drm_dp_mst_atomic_payload *old_payload) |
| 211 | { | 212 | { |
| 212 | struct drm_dp_mst_atomic_payload *pos; | 213 | struct drm_dp_mst_atomic_payload *pos; |
| 213 | int pbn_per_slot = mst_state->pbn_div; | 214 | int pbn_per_slot = dfixed_trunc(mst_state->pbn_div); |
| 214 | u8 next_payload_vc_start = mgr->next_start_slot; | 215 | u8 next_payload_vc_start = mgr->next_start_slot; |
| 215 | u8 payload_vc_start = new_payload->vc_start_slot; | 216 | u8 payload_vc_start = new_payload->vc_start_slot; |
| 216 | u8 allocated_time_slots; | 217 | u8 allocated_time_slots; |
| @@ -536,11 +537,8 @@ bool dm_helpers_dp_read_dpcd( · 319 unmodified lines | |||
| 536 | 537 | ||
| 537 | struct amdgpu_dm_connector *aconnector = link->priv; | 538 | struct amdgpu_dm_connector *aconnector = link->priv; |
| 538 | 539 | ||
| 539 | if (!aconnector) { | 540 | if (!aconnector) |
| 540 | drm_dbg_dp(aconnector->base.dev, | ||
| 541 | "Failed to find connector for link!\n"); | ||
| 542 | return false; | 541 | return false; |
| 543 | } | ||
| 544 | 542 | ||
| 545 | return drm_dp_dpcd_read(&aconnector->dm_dp_aux.aux, address, data, | 543 | return drm_dp_dpcd_read(&aconnector->dm_dp_aux.aux, address, data, |
| 546 | size) == size; | 544 | size) == size; |
| @@ -27,6 +27,7 @@ · 26 unmodified lines | |||
| 27 | #include <drm/display/drm_dp_mst_helper.h> | 27 | #include <drm/display/drm_dp_mst_helper.h> |
| 28 | #include <drm/drm_atomic.h> | 28 | #include <drm/drm_atomic.h> |
| 29 | #include <drm/drm_atomic_helper.h> | 29 | #include <drm/drm_atomic_helper.h> |
| 30 | #include <drm/drm_fixed.h> | ||
| 30 | #include "dm_services.h" | 31 | #include "dm_services.h" |
| 31 | #include "amdgpu.h" | 32 | #include "amdgpu.h" |
| 32 | #include "amdgpu_dm.h" | 33 | #include "amdgpu_dm.h" |
| @@ -941,10 +942,10 @@ static int increase_dsc_bpp(struct drm_atomic_state *state, · 908 unmodified lines | |||
| 941 | link_timeslots_used = 0; | 942 | link_timeslots_used = 0; |
| 942 | 943 | ||
| 943 | for (i = 0; i < count; i++) | 944 | for (i = 0; i < count; i++) |
| 944 | link_timeslots_used += DIV_ROUND_UP(vars[i + k].pbn, mst_state->pbn_div); | 945 | link_timeslots_used += DIV_ROUND_UP(vars[i + k].pbn, dfixed_trunc(mst_state->pbn_div)); |
| 945 | 946 | ||
| 946 | fair_pbn_alloc = | 947 | fair_pbn_alloc = |
| 947 | (63 - link_timeslots_used) / remaining_to_increase * mst_state->pbn_div; | 948 | (63 - link_timeslots_used) / remaining_to_increase * dfixed_trunc(mst_state->pbn_div); |
| 948 | 949 | ||
| 949 | if (initial_slack[next_index] > fair_pbn_alloc) { | 950 | if (initial_slack[next_index] > fair_pbn_alloc) { |
| 950 | vars[next_index].pbn += fair_pbn_alloc; | 951 | vars[next_index].pbn += fair_pbn_alloc; |
| @@ -1604,31 +1605,31 @@ enum dc_status dm_dp_mst_is_port_support_mode( · 653 unmodified lines | |||
| 1604 | unsigned int upper_link_bw_in_kbps = 0, down_link_bw_in_kbps = 0; | 1605 | unsigned int upper_link_bw_in_kbps = 0, down_link_bw_in_kbps = 0; |
| 1605 | unsigned int max_compressed_bw_in_kbps = 0; | 1606 | unsigned int max_compressed_bw_in_kbps = 0; |
| 1606 | struct dc_dsc_bw_range bw_range = {0}; | 1607 | struct dc_dsc_bw_range bw_range = {0}; |
| 1607 | struct drm_dp_mst_topology_mgr *mst_mgr; | 1608 | uint16_t full_pbn = aconnector->mst_output_port->full_pbn; |
| 1608 | 1609 | ||
| 1609 | /* | 1610 | /* |
| 1610 | * check if the mode could be supported if DSC pass-through is supported | 1611 | * Consider the case with the depth of the mst topology tree is equal or less than 2 |
| 1611 | * AND check if there enough bandwidth available to support the mode | 1612 | * A. When dsc bitstream can be transmitted along the entire path |
| 1612 | * with DSC enabled. | 1613 | * 1. dsc is possible between source and branch/leaf device (common dsc params is possible), AND |
| 1614 | * 2. dsc passthrough supported at MST branch, or | ||
| 1615 | * 3. dsc decoding supported at leaf MST device | ||
| 1616 | * Use maximum dsc compression as bw constraint | ||
| 1617 | * B. When dsc bitstream cannot be transmitted along the entire path | ||
| 1618 | * Use native bw as bw constraint | ||
| 1613 | */ | 1619 | */ |
| 1614 | if (is_dsc_common_config_possible(stream, &bw_range) && | 1620 | if (is_dsc_common_config_possible(stream, &bw_range) && |
| 1615 | aconnector->mst_output_port->passthrough_aux) { | 1621 | (aconnector->mst_output_port->passthrough_aux || |
| 1616 | mst_mgr = aconnector->mst_output_port->mgr; | 1622 | aconnector->dsc_aux == &aconnector->mst_output_port->aux)) { |
| 1617 | mutex_lock(&mst_mgr->lock); | ||
| 1618 | |||
| 1619 | cur_link_settings = stream->link->verified_link_cap; | 1623 | cur_link_settings = stream->link->verified_link_cap; |
| 1620 | 1624 | ||
| 1621 | upper_link_bw_in_kbps = dc_link_bandwidth_kbps(aconnector->dc_link, | 1625 | upper_link_bw_in_kbps = dc_link_bandwidth_kbps(aconnector->dc_link, |
| 1622 | &cur_link_settings | 1626 | &cur_link_settings); |
| 1623 | ); | 1627 | down_link_bw_in_kbps = kbps_from_pbn(full_pbn); |
| 1624 | down_link_bw_in_kbps = kbps_from_pbn(aconnector->mst_output_port->full_pbn); | ||
| 1625 | 1628 | ||
| 1626 | /* pick the bottleneck */ | 1629 | /* pick the bottleneck */ |
| 1627 | end_to_end_bw_in_kbps = min(upper_link_bw_in_kbps, | 1630 | end_to_end_bw_in_kbps = min(upper_link_bw_in_kbps, |
| 1628 | down_link_bw_in_kbps); | 1631 | down_link_bw_in_kbps); |
| 1629 | 1632 | ||
| 1630 | mutex_unlock(&mst_mgr->lock); | ||
| 1631 | |||
| 1632 | /* | 1633 | /* |
| 1633 | * use the maximum dsc compression bandwidth as the required | 1634 | * use the maximum dsc compression bandwidth as the required |
| 1634 | * bandwidth for the mode | 1635 | * bandwidth for the mode |
| @@ -1642,9 +1643,8 @@ enum dc_status dm_dp_mst_is_port_support_mode( · 7 unmodified lines | |||
| 1642 | } else { | 1643 | } else { |
| 1643 | /* check if mode could be supported within full_pbn */ | 1644 | /* check if mode could be supported within full_pbn */ |
| 1644 | bpp = convert_dc_color_depth_into_bpc(stream->timing.display_color_depth) * 3; | 1645 | bpp = convert_dc_color_depth_into_bpc(stream->timing.display_color_depth) * 3; |
| 1645 | pbn = drm_dp_calc_pbn_mode(stream->timing.pix_clk_100hz / 10, bpp, false); | 1646 | pbn = drm_dp_calc_pbn_mode(stream->timing.pix_clk_100hz / 10, bpp << 4); |
| 1646 | 1647 | if (pbn > full_pbn) | |
| 1647 | if (pbn > aconnector->mst_output_port->full_pbn) | ||
| 1648 | return DC_FAIL_BANDWIDTH_VALIDATE; | 1648 | return DC_FAIL_BANDWIDTH_VALIDATE; |
| 1649 | } | 1649 | } |
| 1650 | 1650 | ||
| @@ -820,22 +820,22 @@ static void dcn35_set_idle_state(struct clk_mgr *clk_mgr_base, bool allow_idle) · 819 unmodified lines | |||
| 820 | 820 | ||
| 821 | if (dc->config.disable_ips == DMUB_IPS_ENABLE || | 821 | if (dc->config.disable_ips == DMUB_IPS_ENABLE || |
| 822 | dc->config.disable_ips == DMUB_IPS_DISABLE_DYNAMIC) { | 822 | dc->config.disable_ips == DMUB_IPS_DISABLE_DYNAMIC) { |
| 823 | val |= DMUB_IPS1_ALLOW_MASK; | ||
| 824 | val |= DMUB_IPS2_ALLOW_MASK; | ||
| 825 | } else if (dc->config.disable_ips == DMUB_IPS_DISABLE_IPS1) { | ||
| 826 | val = val & ~DMUB_IPS1_ALLOW_MASK; | 823 | val = val & ~DMUB_IPS1_ALLOW_MASK; |
| 827 | val = val & ~DMUB_IPS2_ALLOW_MASK; | 824 | val = val & ~DMUB_IPS2_ALLOW_MASK; |
| 828 | } else if (dc->config.disable_ips == DMUB_IPS_DISABLE_IPS2) { | 825 | } else if (dc->config.disable_ips == DMUB_IPS_DISABLE_IPS1) { |
| 829 | val |= DMUB_IPS1_ALLOW_MASK; | ||
| 830 | val = val & ~DMUB_IPS2_ALLOW_MASK; | ||
| 831 | } else if (dc->config.disable_ips == DMUB_IPS_DISABLE_IPS2_Z10) { | ||
| 832 | val |= DMUB_IPS1_ALLOW_MASK; | 826 | val |= DMUB_IPS1_ALLOW_MASK; |
| 833 | val |= DMUB_IPS2_ALLOW_MASK; | 827 | val |= DMUB_IPS2_ALLOW_MASK; |
| 828 | } else if (dc->config.disable_ips == DMUB_IPS_DISABLE_IPS2) { | ||
| 829 | val = val & ~DMUB_IPS1_ALLOW_MASK; | ||
| 830 | val |= DMUB_IPS2_ALLOW_MASK; | ||
| 831 | } else if (dc->config.disable_ips == DMUB_IPS_DISABLE_IPS2_Z10) { | ||
| 832 | val = val & ~DMUB_IPS1_ALLOW_MASK; | ||
| 833 | val = val & ~DMUB_IPS2_ALLOW_MASK; | ||
| 834 | } | 834 | } |
| 835 | 835 | ||
| 836 | if (!allow_idle) { | 836 | if (!allow_idle) { |
| 837 | val = val & ~DMUB_IPS1_ALLOW_MASK; | 837 | val |= DMUB_IPS1_ALLOW_MASK; |
| 838 | val = val & ~DMUB_IPS2_ALLOW_MASK; | 838 | val |= DMUB_IPS2_ALLOW_MASK; |
| 839 | } | 839 | } |
| 840 | 840 | ||
| 841 | dcn35_smu_write_ips_scratch(clk_mgr, val); | 841 | dcn35_smu_write_ips_scratch(clk_mgr, val); |
| @@ -3178,7 +3178,7 @@ static bool update_planes_and_stream_state(struct dc *dc, · 3177 unmodified lines | |||
| 3178 | struct pipe_ctx *otg_master = resource_get_otg_master_for_stream(&context->res_ctx, | 3178 | struct pipe_ctx *otg_master = resource_get_otg_master_for_stream(&context->res_ctx, |
| 3179 | context->streams[i]); | 3179 | context->streams[i]); |
| 3180 | 3180 | ||
| 3181 | if (otg_master->stream->test_pattern.type != DP_TEST_PATTERN_VIDEO_MODE) | 3181 | if (otg_master && otg_master->stream->test_pattern.type != DP_TEST_PATTERN_VIDEO_MODE) |
| 3182 | resource_build_test_pattern_params(&context->res_ctx, otg_master); | 3182 | resource_build_test_pattern_params(&context->res_ctx, otg_master); |
| 3183 | } | 3183 | } |
| 3184 | } | 3184 | } |
| @@ -4934,8 +4934,8 @@ bool dc_dmub_is_ips_idle_state(struct dc *dc) · 1749 unmodified lines | |||
| 4934 | if (dc->hwss.get_idle_state) | 4934 | if (dc->hwss.get_idle_state) |
| 4935 | idle_state = dc->hwss.get_idle_state(dc); | 4935 | idle_state = dc->hwss.get_idle_state(dc); |
| 4936 | 4936 | ||
| 4937 | if ((idle_state & DMUB_IPS1_ALLOW_MASK) || | 4937 | if (!(idle_state & DMUB_IPS1_ALLOW_MASK) || |
| 4938 | (idle_state & DMUB_IPS2_ALLOW_MASK)) | 4938 | !(idle_state & DMUB_IPS2_ALLOW_MASK)) |
| 4939 | return true; | 4939 | return true; |
| 4940 | 4940 | ||
| 4941 | return false; | 4941 | return false; |
| @@ -5190,6 +5190,9 @@ bool dc_resource_acquire_secondary_pipe_for_mpc_odm_legacy( · 5189 unmodified lines | |||
| 5190 | sec_next = sec_pipe->next_odm_pipe; | 5190 | sec_next = sec_pipe->next_odm_pipe; |
| 5191 | sec_prev = sec_pipe->prev_odm_pipe; | 5191 | sec_prev = sec_pipe->prev_odm_pipe; |
| 5192 | 5192 | ||
| 5193 | if (pri_pipe == NULL) | ||
| 5194 | return false; | ||
| 5195 | |||
| 5193 | *sec_pipe = *pri_pipe; | 5196 | *sec_pipe = *pri_pipe; |
| 5194 | 5197 | ||
| 5195 | sec_pipe->top_pipe = sec_top; | 5198 | sec_pipe->top_pipe = sec_top; |
| @@ -1202,11 +1202,11 @@ void dc_dmub_srv_exit_low_power_state(const struct dc *dc) · 1201 unmodified lines | |||
| 1202 | allow_state = dc->hwss.get_idle_state(dc); | 1202 | allow_state = dc->hwss.get_idle_state(dc); |
| 1203 | dc->hwss.set_idle_state(dc, false); | 1203 | dc->hwss.set_idle_state(dc, false); |
| 1204 | 1204 | ||
| 1205 | if (allow_state & DMUB_IPS2_ALLOW_MASK) { | 1205 | if (!(allow_state & DMUB_IPS2_ALLOW_MASK)) { |
| 1206 | // Wait for evaluation time | 1206 | // Wait for evaluation time |
| 1207 | udelay(dc->debug.ips2_eval_delay_us); | 1207 | udelay(dc->debug.ips2_eval_delay_us); |
| 1208 | commit_state = dc->hwss.get_idle_state(dc); | 1208 | commit_state = dc->hwss.get_idle_state(dc); |
| 1209 | if (commit_state & DMUB_IPS2_COMMIT_MASK) { | 1209 | if (!(commit_state & DMUB_IPS2_COMMIT_MASK)) { |
| 1210 | // Tell PMFW to exit low power state | 1210 | // Tell PMFW to exit low power state |
| 1211 | dc->clk_mgr->funcs->exit_low_power_state(dc->clk_mgr); | 1211 | dc->clk_mgr->funcs->exit_low_power_state(dc->clk_mgr); |
| 1212 | 1212 | ||
| @@ -1216,7 +1216,7 @@ void dc_dmub_srv_exit_low_power_state(const struct dc *dc) · 3 unmodified lines | |||
| 1216 | 1216 | ||
| 1217 | for (i = 0; i < max_num_polls; ++i) { | 1217 | for (i = 0; i < max_num_polls; ++i) { |
| 1218 | commit_state = dc->hwss.get_idle_state(dc); | 1218 | commit_state = dc->hwss.get_idle_state(dc); |
| 1219 | if (!(commit_state & DMUB_IPS2_COMMIT_MASK)) | 1219 | if (commit_state & DMUB_IPS2_COMMIT_MASK) |
| 1220 | break; | 1220 | break; |
| 1221 | 1221 | ||
| 1222 | udelay(1); | 1222 | udelay(1); |
| @@ -1235,10 +1235,10 @@ void dc_dmub_srv_exit_low_power_state(const struct dc *dc) · 12 unmodified lines | |||
| 1235 | } | 1235 | } |
| 1236 | 1236 | ||
| 1237 | dc_dmub_srv_notify_idle(dc, false); | 1237 | dc_dmub_srv_notify_idle(dc, false); |
| 1238 | if (allow_state & DMUB_IPS1_ALLOW_MASK) { | 1238 | if (!(allow_state & DMUB_IPS1_ALLOW_MASK)) { |
| 1239 | for (i = 0; i < max_num_polls; ++i) { | 1239 | for (i = 0; i < max_num_polls; ++i) { |
| 1240 | commit_state = dc->hwss.get_idle_state(dc); | 1240 | commit_state = dc->hwss.get_idle_state(dc); |
| 1241 | if (!(commit_state & DMUB_IPS1_COMMIT_MASK)) | 1241 | if (commit_state & DMUB_IPS1_COMMIT_MASK) |
| 1242 | break; | 1242 | break; |
| 1243 | 1243 | ||
| 1244 | udelay(1); | 1244 | udelay(1); |
| @@ -177,6 +177,7 @@ struct dc_panel_patch { · 176 unmodified lines | |||
| 177 | unsigned int disable_fams; | 177 | unsigned int disable_fams; |
| 178 | unsigned int skip_avmute; | 178 | unsigned int skip_avmute; |
| 179 | unsigned int mst_start_top_delay; | 179 | unsigned int mst_start_top_delay; |
| 180 | unsigned int remove_sink_ext_caps; | ||
| 180 | }; | 181 | }; |
| 181 | 182 | ||
| 182 | struct dc_edid_caps { | 183 | struct dc_edid_caps { |
| @@ -261,12 +261,6 @@ static void enc35_stream_encoder_enable( · 260 unmodified lines | |||
| 261 | /* invalid mode ! */ | 261 | /* invalid mode ! */ |
| 262 | ASSERT_CRITICAL(false); | 262 | ASSERT_CRITICAL(false); |
| 263 | } | 263 | } |
| 264 | |||
| 265 | REG_UPDATE(DIG_FE_CLK_CNTL, DIG_FE_CLK_EN, 1); | ||
| 266 | REG_UPDATE(DIG_FE_EN_CNTL, DIG_FE_ENABLE, 1); | ||
| 267 | } else { | ||
| 268 | REG_UPDATE(DIG_FE_EN_CNTL, DIG_FE_ENABLE, 0); | ||
| 269 | REG_UPDATE(DIG_FE_CLK_CNTL, DIG_FE_CLK_EN, 0); | ||
| 270 | } | 264 | } |
| 271 | } | 265 | } |
| 272 | 266 | ||
| @@ -436,6 +430,8 @@ static void enc35_disable_fifo(struct stream_encoder *enc) · 163 unmodified lines | |||
| 436 | struct dcn10_stream_encoder *enc1 = DCN10STRENC_FROM_STRENC(enc); | 430 | struct dcn10_stream_encoder *enc1 = DCN10STRENC_FROM_STRENC(enc); |
| 437 | 431 | ||
| 438 | REG_UPDATE(DIG_FIFO_CTRL0, DIG_FIFO_ENABLE, 0); | 432 | REG_UPDATE(DIG_FIFO_CTRL0, DIG_FIFO_ENABLE, 0); |
| 433 | REG_UPDATE(DIG_FE_EN_CNTL, DIG_FE_ENABLE, 0); | ||
| 434 | REG_UPDATE(DIG_FE_CLK_CNTL, DIG_FE_CLK_EN, 0); | ||
| 439 | } | 435 | } |
| 440 | 436 | ||
| 441 | static void enc35_enable_fifo(struct stream_encoder *enc) | 437 | static void enc35_enable_fifo(struct stream_encoder *enc) |
| @@ -443,6 +439,8 @@ static void enc35_enable_fifo(struct stream_encoder *enc) · 1 unmodified line | |||
| 443 | struct dcn10_stream_encoder *enc1 = DCN10STRENC_FROM_STRENC(enc); | 439 | struct dcn10_stream_encoder *enc1 = DCN10STRENC_FROM_STRENC(enc); |
| 444 | 440 | ||
| 445 | REG_UPDATE(DIG_FIFO_CTRL0, DIG_FIFO_READ_START_LEVEL, 0x7); | 441 | REG_UPDATE(DIG_FIFO_CTRL0, DIG_FIFO_READ_START_LEVEL, 0x7); |
| 442 | REG_UPDATE(DIG_FE_CLK_CNTL, DIG_FE_CLK_EN, 1); | ||
| 443 | REG_UPDATE(DIG_FE_EN_CNTL, DIG_FE_ENABLE, 1); | ||
| 446 | 444 | ||
| 447 | enc35_reset_fifo(enc, true); | 445 | enc35_reset_fifo(enc, true); |
| 448 | enc35_reset_fifo(enc, false); | 446 | enc35_reset_fifo(enc, false); |
| @@ -1088,6 +1088,9 @@ static bool detect_link_and_local_sink(struct dc_link *link, · 1087 unmodified lines | |||
| 1088 | if (sink->edid_caps.panel_patch.skip_scdc_overwrite) | 1088 | if (sink->edid_caps.panel_patch.skip_scdc_overwrite) |
| 1089 | link->ctx->dc->debug.hdmi20_disable = true; | 1089 | link->ctx->dc->debug.hdmi20_disable = true; |
| 1090 | 1090 | ||
| 1091 | if (sink->edid_caps.panel_patch.remove_sink_ext_caps) | ||
| 1092 | link->dpcd_sink_ext_caps.raw = 0; | ||
| 1093 | |||
| 1091 | if (dc_is_hdmi_signal(link->connector_signal)) | 1094 | if (dc_is_hdmi_signal(link->connector_signal)) |
| 1092 | read_scdc_caps(link->ddc, link->local_sink); | 1095 | read_scdc_caps(link->ddc, link->local_sink); |
| 1093 | 1096 | ||
| @@ -195,6 +195,7 @@ struct dmub_srv_region_params { · 194 unmodified lines | |||
| 195 | uint32_t vbios_size; | 195 | uint32_t vbios_size; |
| 196 | const uint8_t *fw_inst_const; | 196 | const uint8_t *fw_inst_const; |
| 197 | const uint8_t *fw_bss_data; | 197 | const uint8_t *fw_bss_data; |
| 198 | bool is_mailbox_in_inbox; | ||
| 198 | }; | 199 | }; |
| 199 | 200 | ||
| 200 | /** | 201 | /** |
| @@ -214,20 +215,25 @@ struct dmub_srv_region_params { · 13 unmodified lines | |||
| 214 | */ | 215 | */ |
| 215 | struct dmub_srv_region_info { | 216 | struct dmub_srv_region_info { |
| 216 | uint32_t fb_size; | 217 | uint32_t fb_size; |
| 218 | uint32_t inbox_size; | ||
| 217 | uint8_t num_regions; | 219 | uint8_t num_regions; |
| 218 | struct dmub_region regions[DMUB_WINDOW_TOTAL]; | 220 | struct dmub_region regions[DMUB_WINDOW_TOTAL]; |
| 219 | }; | 221 | }; |
| 220 | 222 | ||
| 221 | /** | 223 | /** |
| 222 | * struct dmub_srv_fb_params - parameters used for driver fb setup | 224 | * struct dmub_srv_memory_params - parameters used for driver fb setup |
| 223 | * @region_info: region info calculated by dmub service | 225 | * @region_info: region info calculated by dmub service |
| 224 | * @cpu_addr: base cpu address for the framebuffer | 226 | * @cpu_fb_addr: base cpu address for the framebuffer |
| 225 | * @gpu_addr: base gpu virtual address for the framebuffer | 227 | * @cpu_inbox_addr: base cpu address for the gart |
| 228 | * @gpu_fb_addr: base gpu virtual address for the framebuffer | ||
| 229 | * @gpu_inbox_addr: base gpu virtual address for the gart | ||
| 226 | */ | 230 | */ |
| 227 | struct dmub_srv_fb_params { | 231 | struct dmub_srv_memory_params { |
| 228 | const struct dmub_srv_region_info *region_info; | 232 | const struct dmub_srv_region_info *region_info; |
| 229 | void *cpu_addr; | 233 | void *cpu_fb_addr; |
| 230 | uint64_t gpu_addr; | 234 | void *cpu_inbox_addr; |
| 235 | uint64_t gpu_fb_addr; | ||
| 236 | uint64_t gpu_inbox_addr; | ||
| 231 | }; | 237 | }; |
| 232 | 238 | ||
| 233 | /** | 239 | /** |
| @@ -563,8 +569,8 @@ dmub_srv_calc_region_info(struct dmub_srv *dmub, · 329 unmodified lines | |||
| 563 | * DMUB_STATUS_OK - success | 569 | * DMUB_STATUS_OK - success |
| 564 | * DMUB_STATUS_INVALID - unspecified error | 570 | * DMUB_STATUS_INVALID - unspecified error |
| 565 | */ | 571 | */ |
| 566 | enum dmub_status dmub_srv_calc_fb_info(struct dmub_srv *dmub, | 572 | enum dmub_status dmub_srv_calc_mem_info(struct dmub_srv *dmub, |
| 567 | const struct dmub_srv_fb_params *params, | 573 | const struct dmub_srv_memory_params *params, |
| 568 | struct dmub_srv_fb_info *out); | 574 | struct dmub_srv_fb_info *out); |
| 569 | 575 | ||
| 570 | /** | 576 | /** |
| @@ -434,7 +434,7 @@ dmub_srv_calc_region_info(struct dmub_srv *dmub, · 433 unmodified lines | |||
| 434 | uint32_t fw_state_size = DMUB_FW_STATE_SIZE; | 434 | uint32_t fw_state_size = DMUB_FW_STATE_SIZE; |
| 435 | uint32_t trace_buffer_size = DMUB_TRACE_BUFFER_SIZE; | 435 | uint32_t trace_buffer_size = DMUB_TRACE_BUFFER_SIZE; |
| 436 | uint32_t scratch_mem_size = DMUB_SCRATCH_MEM_SIZE; | 436 | uint32_t scratch_mem_size = DMUB_SCRATCH_MEM_SIZE; |
| 437 | 437 | uint32_t previous_top = 0; | |
| 438 | if (!dmub->sw_init) | 438 | if (!dmub->sw_init) |
| 439 | return DMUB_STATUS_INVALID; | 439 | return DMUB_STATUS_INVALID; |
| 440 | 440 | ||
| @@ -459,8 +459,15 @@ dmub_srv_calc_region_info(struct dmub_srv *dmub, · 18 unmodified lines | |||
| 459 | bios->base = dmub_align(stack->top, 256); | 459 | bios->base = dmub_align(stack->top, 256); |
| 460 | bios->top = bios->base + params->vbios_size; | 460 | bios->top = bios->base + params->vbios_size; |
| 461 | 461 | ||
| 462 | mail->base = dmub_align(bios->top, 256); | 462 | if (params->is_mailbox_in_inbox) { |
| 463 | mail->top = mail->base + DMUB_MAILBOX_SIZE; | 463 | mail->base = 0; |
| 464 | mail->top = mail->base + DMUB_MAILBOX_SIZE; | ||
| 465 | previous_top = bios->top; | ||
| 466 | } else { | ||
| 467 | mail->base = dmub_align(bios->top, 256); | ||
| 468 | mail->top = mail->base + DMUB_MAILBOX_SIZE; | ||
| 469 | previous_top = mail->top; | ||
| 470 | } | ||
| 464 | 471 | ||
| 465 | fw_info = dmub_get_fw_meta_info(params); | 472 | fw_info = dmub_get_fw_meta_info(params); |
| 466 | 473 | ||
| @@ -479,7 +486,7 @@ dmub_srv_calc_region_info(struct dmub_srv *dmub, · 12 unmodified lines | |||
| 479 | dmub->fw_version = fw_info->fw_version; | 486 | dmub->fw_version = fw_info->fw_version; |
| 480 | } | 487 | } |
| 481 | 488 | ||
| 482 | trace_buff->base = dmub_align(mail->top, 256); | 489 | trace_buff->base = dmub_align(previous_top, 256); |
| 483 | trace_buff->top = trace_buff->base + dmub_align(trace_buffer_size, 64); | 490 | trace_buff->top = trace_buff->base + dmub_align(trace_buffer_size, 64); |
| 484 | 491 | ||
| 485 | fw_state->base = dmub_align(trace_buff->top, 256); | 492 | fw_state->base = dmub_align(trace_buff->top, 256); |
| @@ -490,11 +497,14 @@ dmub_srv_calc_region_info(struct dmub_srv *dmub, · 4 unmodified lines | |||
| 490 | 497 | ||
| 491 | out->fb_size = dmub_align(scratch_mem->top, 4096); | 498 | out->fb_size = dmub_align(scratch_mem->top, 4096); |
| 492 | 499 | ||
| 500 | if (params->is_mailbox_in_inbox) | ||
| 501 | out->inbox_size = dmub_align(mail->top, 4096); | ||
| 502 | |||
| 493 | return DMUB_STATUS_OK; | 503 | return DMUB_STATUS_OK; |
| 494 | } | 504 | } |
| 495 | 505 | ||
| 496 | enum dmub_status dmub_srv_calc_fb_info(struct dmub_srv *dmub, | 506 | enum dmub_status dmub_srv_calc_mem_info(struct dmub_srv *dmub, |
| 497 | const struct dmub_srv_fb_params *params, | 507 | const struct dmub_srv_memory_params *params, |
| 498 | struct dmub_srv_fb_info *out) | 508 | struct dmub_srv_fb_info *out) |
| 499 | { | 509 | { |
| 500 | uint8_t *cpu_base; | 510 | uint8_t *cpu_base; |
| @@ -509,8 +519,8 @@ enum dmub_status dmub_srv_calc_fb_info(struct dmub_srv *dmub, · 8 unmodified lines | |||
| 509 | if (params->region_info->num_regions != DMUB_NUM_WINDOWS) | 519 | if (params->region_info->num_regions != DMUB_NUM_WINDOWS) |
| 510 | return DMUB_STATUS_INVALID; | 520 | return DMUB_STATUS_INVALID; |
| 511 | 521 | ||
| 512 | cpu_base = (uint8_t *)params->cpu_addr; | 522 | cpu_base = (uint8_t *)params->cpu_fb_addr; |
| 513 | gpu_base = params->gpu_addr; | 523 | gpu_base = params->gpu_fb_addr; |
| 514 | 524 | ||
| 515 | for (i = 0; i < DMUB_NUM_WINDOWS; ++i) { | 525 | for (i = 0; i < DMUB_NUM_WINDOWS; ++i) { |
| 516 | const struct dmub_region *reg = | 526 | const struct dmub_region *reg = |
| @@ -518,6 +528,12 @@ enum dmub_status dmub_srv_calc_fb_info(struct dmub_srv *dmub, · 1 unmodified line | |||
| 518 | 528 | ||
| 519 | out->fb[i].cpu_addr = cpu_base + reg->base; | 529 | out->fb[i].cpu_addr = cpu_base + reg->base; |
| 520 | out->fb[i].gpu_addr = gpu_base + reg->base; | 530 | out->fb[i].gpu_addr = gpu_base + reg->base; |
| 531 | |||
| 532 | if (i == DMUB_WINDOW_4_MAILBOX && params->cpu_inbox_addr != 0) { | ||
| 533 | out->fb[i].cpu_addr = (uint8_t *)params->cpu_inbox_addr + reg->base; | ||
| 534 | out->fb[i].gpu_addr = params->gpu_inbox_addr + reg->base; | ||
| 535 | } | ||
| 536 | |||
| 521 | out->fb[i].size = reg->top - reg->base; | 537 | out->fb[i].size = reg->top - reg->base; |
| 522 | } | 538 | } |
| 523 | 539 | ||
| @@ -707,9 +723,16 @@ enum dmub_status dmub_srv_sync_inbox1(struct dmub_srv *dmub) · 183 unmodified lines | |||
| 707 | return DMUB_STATUS_INVALID; | 723 | return DMUB_STATUS_INVALID; |
| 708 | 724 | ||
| 709 | if (dmub->hw_funcs.get_inbox1_rptr && dmub->hw_funcs.get_inbox1_wptr) { | 725 | if (dmub->hw_funcs.get_inbox1_rptr && dmub->hw_funcs.get_inbox1_wptr) { |
| 710 | dmub->inbox1_rb.rptr = dmub->hw_funcs.get_inbox1_rptr(dmub); | 726 | uint32_t rptr = dmub->hw_funcs.get_inbox1_rptr(dmub); |
| 711 | dmub->inbox1_rb.wrpt = dmub->hw_funcs.get_inbox1_wptr(dmub); | 727 | uint32_t wptr = dmub->hw_funcs.get_inbox1_wptr(dmub); |
| 712 | dmub->inbox1_last_wptr = dmub->inbox1_rb.wrpt; | 728 | |
| 729 | if (rptr > dmub->inbox1_rb.capacity || wptr > dmub->inbox1_rb.capacity) { | ||
| 730 | return DMUB_STATUS_HW_FAILURE; | ||
| 731 | } else { | ||
| 732 | dmub->inbox1_rb.rptr = rptr; | ||
| 733 | dmub->inbox1_rb.wrpt = wptr; | ||
| 734 | dmub->inbox1_last_wptr = dmub->inbox1_rb.wrpt; | ||
| 735 | } | ||
| 713 | } | 736 | } |
| 714 | 737 | ||
| 715 | return DMUB_STATUS_OK; | 738 | return DMUB_STATUS_OK; |
| @@ -743,6 +766,11 @@ enum dmub_status dmub_srv_cmd_queue(struct dmub_srv *dmub, · 27 unmodified lines | |||
| 743 | if (!dmub->hw_init) | 766 | if (!dmub->hw_init) |
| 744 | return DMUB_STATUS_INVALID; | 767 | return DMUB_STATUS_INVALID; |
| 745 | 768 | ||
| 769 | if (dmub->inbox1_rb.rptr > dmub->inbox1_rb.capacity || | ||
| 770 | dmub->inbox1_rb.wrpt > dmub->inbox1_rb.capacity) { | ||
| 771 | return DMUB_STATUS_HW_FAILURE; | ||
| 772 | } | ||
| 773 | |||
| 746 | if (dmub_rb_push_front(&dmub->inbox1_rb, cmd)) | 774 | if (dmub_rb_push_front(&dmub->inbox1_rb, cmd)) |
| 747 | return DMUB_STATUS_OK; | 775 | return DMUB_STATUS_OK; |
| 748 | 776 | ||
| @@ -123,7 +123,7 @@ typedef enum { · 122 unmodified lines | |||
| 123 | VOLTAGE_GUARDBAND_COUNT | 123 | VOLTAGE_GUARDBAND_COUNT |
| 124 | } GFX_GUARDBAND_e; | 124 | } GFX_GUARDBAND_e; |
| 125 | 125 | ||
| 126 | #define SMU_METRICS_TABLE_VERSION 0x8 | 126 | #define SMU_METRICS_TABLE_VERSION 0x9 |
| 127 | 127 | ||
| 128 | typedef struct __attribute__((packed, aligned(4))) { | 128 | typedef struct __attribute__((packed, aligned(4))) { |
| 129 | uint32_t AccumulationCounter; | 129 | uint32_t AccumulationCounter; |
| @@ -211,6 +211,14 @@ typedef struct __attribute__((packed, aligned(4))) { · 81 unmodified lines | |||
| 211 | //XGMI Data tranfser size | 211 | //XGMI Data tranfser size |
| 212 | uint64_t XgmiReadDataSizeAcc[8];//in KByte | 212 | uint64_t XgmiReadDataSizeAcc[8];//in KByte |
| 213 | uint64_t XgmiWriteDataSizeAcc[8];//in KByte | 213 | uint64_t XgmiWriteDataSizeAcc[8];//in KByte |
| 214 | |||
| 215 | //PCIE BW Data and error count | ||
| 216 | uint32_t PcieBandwidth[4]; | ||
| 217 | uint32_t PCIeL0ToRecoveryCountAcc; // The Pcie counter itself is accumulated | ||
| 218 | uint32_t PCIenReplayAAcc; // The Pcie counter itself is accumulated | ||
| 219 | uint32_t PCIenReplayARolloverCountAcc; // The Pcie counter itself is accumulated | ||
| 220 | uint32_t PCIeNAKSentCountAcc; // The Pcie counter itself is accumulated | ||
| 221 | uint32_t PCIeNAKReceivedCountAcc; // The Pcie counter itself is accumulated | ||
| 214 | } MetricsTable_t; | 222 | } MetricsTable_t; |
| 215 | 223 | ||
| 216 | #define SMU_VF_METRICS_TABLE_VERSION 0x3 | 224 | #define SMU_VF_METRICS_TABLE_VERSION 0x3 |
| @@ -1454,7 +1454,7 @@ static int smu_v13_0_6_register_irq_handler(struct smu_context *smu) · 1453 unmodified lines | |||
| 1454 | 1454 | ||
| 1455 | static int smu_v13_0_6_notify_unload(struct smu_context *smu) | 1455 | static int smu_v13_0_6_notify_unload(struct smu_context *smu) |
| 1456 | { | 1456 | { |
| 1457 | if (smu->smc_fw_version <= 0x553500) | 1457 | if (amdgpu_in_reset(smu->adev)) |
| 1458 | return 0; | 1458 | return 0; |
| 1459 | 1459 | ||
| 1460 | dev_dbg(smu->adev->dev, "Notify PMFW about driver unload"); | 1460 | dev_dbg(smu->adev->dev, "Notify PMFW about driver unload"); |
| @@ -2095,6 +2095,14 @@ static ssize_t smu_v13_0_6_get_gpu_metrics(struct smu_context *smu, void **table · 634 unmodified lines | |||
| 2095 | smu_v13_0_6_get_current_pcie_link_speed(smu); | 2095 | smu_v13_0_6_get_current_pcie_link_speed(smu); |
| 2096 | gpu_metrics->pcie_bandwidth_acc = | 2096 | gpu_metrics->pcie_bandwidth_acc = |
| 2097 | SMUQ10_ROUND(metrics->PcieBandwidthAcc[0]); | 2097 | SMUQ10_ROUND(metrics->PcieBandwidthAcc[0]); |
| 2098 | gpu_metrics->pcie_bandwidth_inst = | ||
| 2099 | SMUQ10_ROUND(metrics->PcieBandwidth[0]); | ||
| 2100 | gpu_metrics->pcie_l0_to_recov_count_acc = | ||
| 2101 | metrics->PCIeL0ToRecoveryCountAcc; | ||
| 2102 | gpu_metrics->pcie_replay_count_acc = | ||
| 2103 | metrics->PCIenReplayAAcc; | ||
| 2104 | gpu_metrics->pcie_replay_rover_count_acc = | ||
| 2105 | metrics->PCIenReplayARolloverCountAcc; | ||
| 2098 | } | 2106 | } |
| 2099 | 2107 | ||
| 2100 | gpu_metrics->system_clock_counter = ktime_get_boottime_ns(); | 2108 | gpu_metrics->system_clock_counter = ktime_get_boottime_ns(); |
| @@ -7,8 +7,9 @@ · 6 unmodified lines | |||
| 7 | #include <linux/clk.h> | 7 | #include <linux/clk.h> |
| 8 | #include <linux/component.h> | 8 | #include <linux/component.h> |
| 9 | #include <linux/module.h> | 9 | #include <linux/module.h> |
| 10 | #include <linux/of_device.h> | 10 | #include <linux/of.h> |
| 11 | #include <linux/platform_device.h> | 11 | #include <linux/platform_device.h> |
| 12 | #include <linux/property.h> | ||
| 12 | 13 | ||
| 13 | #include <drm/drm_atomic.h> | 14 | #include <drm/drm_atomic.h> |
| 14 | #include <drm/drm_atomic_helper.h> | 15 | #include <drm/drm_atomic_helper.h> |
| @@ -1012,26 +1013,17 @@ armada_lcd_bind(struct device *dev, struct device *master, void *data) · 997 unmodified lines | |||
| 1012 | int irq = platform_get_irq(pdev, 0); | 1013 | int irq = platform_get_irq(pdev, 0); |
| 1013 | const struct armada_variant *variant; | 1014 | const struct armada_variant *variant; |
| 1014 | struct device_node *port = NULL; | 1015 | struct device_node *port = NULL; |
| 1016 | struct device_node *np, *parent = dev->of_node; | ||
| 1015 | 1017 | ||
| 1016 | if (irq < 0) | 1018 | if (irq < 0) |
| 1017 | return irq; | 1019 | return irq; |
| 1018 | 1020 | ||
| 1019 | if (!dev->of_node) { | ||
| 1020 | const struct platform_device_id *id; | ||
| 1021 | 1021 | ||
| 1022 | id = platform_get_device_id(pdev); | 1022 | variant = device_get_match_data(dev); |
| 1023 | if (!id) | 1023 | if (!variant) |
| 1024 | return -ENXIO; | 1024 | return -ENXIO; |
| 1025 | |||
| 1026 | variant = (const struct armada_variant *)id->driver_data; | ||
| 1027 | } else { | ||
| 1028 | const struct of_device_id *match; | ||
| 1029 | struct device_node *np, *parent = dev->of_node; | ||
| 1030 | |||
| 1031 | match = of_match_device(dev->driver->of_match_table, dev); | ||
| 1032 | if (!match) | ||
| 1033 | return -ENXIO; | ||
| 1034 | 1025 | ||
| 1026 | if (parent) { | ||
| 1035 | np = of_get_child_by_name(parent, "ports"); | 1027 | np = of_get_child_by_name(parent, "ports"); |
| 1036 | if (np) | 1028 | if (np) |
| 1037 | parent = np; | 1029 | parent = np; |
| @@ -1041,8 +1033,6 @@ armada_lcd_bind(struct device *dev, struct device *master, void *data) · 3 unmodified lines | |||
| 1041 | dev_err(dev, "no port node found in %pOF\n", parent); | 1033 | dev_err(dev, "no port node found in %pOF\n", parent); |
| 1042 | return -ENXIO; | 1034 | return -ENXIO; |
| 1043 | } | 1035 | } |
| 1044 | |||
| 1045 | variant = match->data; | ||
| 1046 | } | 1036 | } |
| 1047 | 1037 | ||
| 1048 | return armada_drm_crtc_create(drm, dev, res, irq, variant, port); | 1038 | return armada_drm_crtc_create(drm, dev, res, irq, variant, port); |
| @@ -1066,10 +1056,9 @@ static int armada_lcd_probe(struct platform_device *pdev) · 17 unmodified lines | |||
| 1066 | return component_add(&pdev->dev, &armada_lcd_ops); | 1056 | return component_add(&pdev->dev, &armada_lcd_ops); |
| 1067 | } | 1057 | } |
| 1068 | 1058 | ||
| 1069 | static int armada_lcd_remove(struct platform_device *pdev) | 1059 | static void armada_lcd_remove(struct platform_device *pdev) |
| 1070 | { | 1060 | { |
| 1071 | component_del(&pdev->dev, &armada_lcd_ops); | 1061 | component_del(&pdev->dev, &armada_lcd_ops); |
| 1072 | return 0; | ||
| 1073 | } | 1062 | } |
| 1074 | 1063 | ||
| 1075 | static const struct of_device_id armada_lcd_of_match[] = { | 1064 | static const struct of_device_id armada_lcd_of_match[] = { |
| @@ -1095,7 +1084,7 @@ MODULE_DEVICE_TABLE(platform, armada_lcd_platform_ids); · 19 unmodified lines | |||
| 1095 | 1084 | ||
| 1096 | struct platform_driver armada_lcd_platform_driver = { | 1085 | struct platform_driver armada_lcd_platform_driver = { |
| 1097 | .probe = armada_lcd_probe, | 1086 | .probe = armada_lcd_probe, |
| 1098 | .remove = armada_lcd_remove, | 1087 | .remove_new = armada_lcd_remove, |
| 1099 | .driver = { | 1088 | .driver = { |
| 1100 | .name = "armada-lcd", | 1089 | .name = "armada-lcd", |
| 1101 | .owner = THIS_MODULE, | 1090 | .owner = THIS_MODULE, |
| @@ -226,10 +226,9 @@ static int armada_drm_probe(struct platform_device *pdev) · 225 unmodified lines | |||
| 226 | match); | 226 | match); |
| 227 | } | 227 | } |
| 228 | 228 | ||
| 229 | static int armada_drm_remove(struct platform_device *pdev) | 229 | static void armada_drm_remove(struct platform_device *pdev) |
| 230 | { | 230 | { |
| 231 | component_master_del(&pdev->dev, &armada_master_ops); | 231 | component_master_del(&pdev->dev, &armada_master_ops); |
| 232 | return 0; | ||
| 233 | } | 232 | } |
| 234 | 233 | ||
| 235 | static void armada_drm_shutdown(struct platform_device *pdev) | 234 | static void armada_drm_shutdown(struct platform_device *pdev) |
| @@ -249,7 +248,7 @@ MODULE_DEVICE_TABLE(platform, armada_drm_platform_ids); · 13 unmodified lines | |||
| 249 | 248 | ||
| 250 | static struct platform_driver armada_drm_platform_driver = { | 249 | static struct platform_driver armada_drm_platform_driver = { |
| 251 | .probe = armada_drm_probe, | 250 | .probe = armada_drm_probe, |
| 252 | .remove = armada_drm_remove, | 251 | .remove_new = armada_drm_remove, |
| 253 | .shutdown = armada_drm_shutdown, | 252 | .shutdown = armada_drm_shutdown, |
| 254 | .driver = { | 253 | .driver = { |
| 255 | .name = "armada-drm", | 254 | .name = "armada-drm", |
| @@ -6,10 +6,10 @@ · 5 unmodified lines | |||
| 6 | #include <linux/irq.h> | 6 | #include <linux/irq.h> |
| 7 | #include <linux/mfd/syscon.h> | 7 | #include <linux/mfd/syscon.h> |
| 8 | #include <linux/module.h> | 8 | #include <linux/module.h> |
| 9 | #include <linux/of.h> | 9 | #include <linux/mod_devicetable.h> |
| 10 | #include <linux/of_device.h> | ||
| 11 | #include <linux/of_reserved_mem.h> | 10 | #include <linux/of_reserved_mem.h> |
| 12 | #include <linux/platform_device.h> | 11 | #include <linux/platform_device.h> |
| 12 | #include <linux/property.h> | ||
| 13 | #include <linux/regmap.h> | 13 | #include <linux/regmap.h> |
| 14 | #include <linux/reset.h> | 14 | #include <linux/reset.h> |
| 15 | 15 | ||
| @@ -143,7 +143,6 @@ static int aspeed_gfx_load(struct drm_device *drm) · 127 unmodified lines | |||
| 143 | struct aspeed_gfx *priv = to_aspeed_gfx(drm); | 143 | struct aspeed_gfx *priv = to_aspeed_gfx(drm); |
| 144 | struct device_node *np = pdev->dev.of_node; | 144 | struct device_node *np = pdev->dev.of_node; |
| 145 | const struct aspeed_gfx_config *config; | 145 | const struct aspeed_gfx_config *config; |
| 146 | const struct of_device_id *match; | ||
| 147 | struct resource *res; | 146 | struct resource *res; |
| 148 | int ret; | 147 | int ret; |
| 149 | 148 | ||
| @@ -152,10 +151,9 @@ static int aspeed_gfx_load(struct drm_device *drm) · 2 unmodified lines | |||
| 152 | if (IS_ERR(priv->base)) | 151 | if (IS_ERR(priv->base)) |
| 153 | return PTR_ERR(priv->base); | 152 | return PTR_ERR(priv->base); |
| 154 | 153 | ||
| 155 | match = of_match_device(aspeed_gfx_match, &pdev->dev); | 154 | config = device_get_match_data(&pdev->dev); |
| 156 | if (!match) | 155 | if (!config) |
| 157 | return -EINVAL; | 156 | return -EINVAL; |
| 158 | config = match->data; | ||
| 159 | 157 | ||
| 160 | priv->dac_reg = config->dac_reg; | 158 | priv->dac_reg = config->dac_reg; |
| 161 | priv->int_clr_reg = config->int_clear_reg; | 159 | priv->int_clr_reg = config->int_clear_reg; |
| @@ -89,11 +89,194 @@ static const struct pci_device_id ast_pciidlist[] = { · 88 unmodified lines | |||
| 89 | 89 | ||
| 90 | MODULE_DEVICE_TABLE(pci, ast_pciidlist); | 90 | MODULE_DEVICE_TABLE(pci, ast_pciidlist); |
| 91 | 91 | ||
| 92 | static bool ast_is_vga_enabled(void __iomem *ioregs) | ||
| 93 | { | ||
| 94 | u8 vgaer = __ast_read8(ioregs, AST_IO_VGAER); | ||
| 95 | |||
| 96 | return vgaer & AST_IO_VGAER_VGA_ENABLE; | ||
| 97 | } | ||
| 98 | |||
| 99 | static void ast_enable_vga(void __iomem *ioregs) | ||
| 100 | { | ||
| 101 | __ast_write8(ioregs, AST_IO_VGAER, AST_IO_VGAER_VGA_ENABLE); | ||
| 102 | __ast_write8(ioregs, AST_IO_VGAMR_W, AST_IO_VGAMR_IOSEL); | ||
| 103 | } | ||
| 104 | |||
| 105 | /* | ||
| 106 | * Run this function as part of the HW device cleanup; not | ||
| 107 | * when the DRM device gets released. | ||
| 108 | */ | ||
| 109 | static void ast_enable_mmio_release(void *data) | ||
| 110 | { | ||
| 111 | void __iomem *ioregs = (void __force __iomem *)data; | ||
| 112 | |||
| 113 | /* enable standard VGA decode */ | ||
| 114 | __ast_write8_i(ioregs, AST_IO_VGACRI, 0xa1, AST_IO_VGACRA1_MMIO_ENABLED); | ||
| 115 | } | ||
| 116 | |||
| 117 | static int ast_enable_mmio(struct device *dev, void __iomem *ioregs) | ||
| 118 | { | ||
| 119 | void *data = (void __force *)ioregs; | ||
| 120 | |||
| 121 | __ast_write8_i(ioregs, AST_IO_VGACRI, 0xa1, | ||
| 122 | AST_IO_VGACRA1_MMIO_ENABLED | | ||
| 123 | AST_IO_VGACRA1_VGAIO_DISABLED); | ||
| 124 | |||
| 125 | return devm_add_action_or_reset(dev, ast_enable_mmio_release, data); | ||
| 126 | } | ||
| 127 | |||
| 128 | static void ast_open_key(void __iomem *ioregs) | ||
| 129 | { | ||
| 130 | __ast_write8_i(ioregs, AST_IO_VGACRI, 0x80, AST_IO_VGACR80_PASSWORD); | ||
| 131 | } | ||
| 132 | |||
| 133 | static int ast_detect_chip(struct pci_dev *pdev, | ||
| 134 | void __iomem *regs, void __iomem *ioregs, | ||
| 135 | enum ast_chip *chip_out, | ||
| 136 | enum ast_config_mode *config_mode_out) | ||
| 137 | { | ||
| 138 | struct device *dev = &pdev->dev; | ||
| 139 | struct device_node *np = dev->of_node; | ||
| 140 | enum ast_config_mode config_mode = ast_use_defaults; | ||
| 141 | uint32_t scu_rev = 0xffffffff; | ||
| 142 | enum ast_chip chip; | ||
| 143 | u32 data; | ||
| 144 | u8 vgacrd0, vgacrd1; | ||
| 145 | |||
| 146 | /* | ||
| 147 | * Find configuration mode and read SCU revision | ||
| 148 | */ | ||
| 149 | |||
| 150 | /* Check if we have device-tree properties */ | ||
| 151 | if (np && !of_property_read_u32(np, "aspeed,scu-revision-id", &data)) { | ||
| 152 | /* We do, disable P2A access */ | ||
| 153 | config_mode = ast_use_dt; | ||
| 154 | scu_rev = data; | ||
| 155 | } else if (pdev->device == PCI_CHIP_AST2000) { // Not all families have a P2A bridge | ||
| 156 | /* | ||
| 157 | * The BMC will set SCU 0x40 D[12] to 1 if the P2 bridge | ||
| 158 | * is disabled. We force using P2A if VGA only mode bit | ||
| 159 | * is set D[7] | ||
| 160 | */ | ||
| 161 | vgacrd0 = __ast_read8_i(ioregs, AST_IO_VGACRI, 0xd0); | ||
| 162 | vgacrd1 = __ast_read8_i(ioregs, AST_IO_VGACRI, 0xd1); | ||
| 163 | if (!(vgacrd0 & 0x80) || !(vgacrd1 & 0x10)) { | ||
| 164 | |||
| 165 | /* | ||
| 166 | * We have a P2A bridge and it is enabled. | ||
| 167 | */ | ||
| 168 | |||
| 169 | /* Patch AST2500/AST2510 */ | ||
| 170 | if ((pdev->revision & 0xf0) == 0x40) { | ||
| 171 | if (!(vgacrd0 & AST_VRAM_INIT_STATUS_MASK)) | ||
| 172 | ast_patch_ahb_2500(regs); | ||
| 173 | } | ||
| 174 | |||
| 175 | /* Double check that it's actually working */ | ||
| 176 | data = __ast_read32(regs, 0xf004); | ||
| 177 | if ((data != 0xffffffff) && (data != 0x00)) { | ||
| 178 | config_mode = ast_use_p2a; | ||
| 179 | |||
| 180 | /* Read SCU7c (silicon revision register) */ | ||
| 181 | __ast_write32(regs, 0xf004, 0x1e6e0000); | ||
| 182 | __ast_write32(regs, 0xf000, 0x1); | ||
| 183 | scu_rev = __ast_read32(regs, 0x1207c); | ||
| 184 | } | ||
| 185 | } | ||
| 186 | } | ||
| 187 | |||
| 188 | switch (config_mode) { | ||
| 189 | case ast_use_defaults: | ||
| 190 | dev_info(dev, "Using default configuration\n"); | ||
| 191 | break; | ||
| 192 | case ast_use_dt: | ||
| 193 | dev_info(dev, "Using device-tree for configuration\n"); | ||
| 194 | break; | ||
| 195 | case ast_use_p2a: | ||
| 196 | dev_info(dev, "Using P2A bridge for configuration\n"); | ||
| 197 | break; | ||
| 198 | } | ||
| 199 | |||
| 200 | /* | ||
| 201 | * Identify chipset | ||
| 202 | */ | ||
| 203 | |||
| 204 | if (pdev->revision >= 0x50) { | ||
| 205 | chip = AST2600; | ||
| 206 | dev_info(dev, "AST 2600 detected\n"); | ||
| 207 | } else if (pdev->revision >= 0x40) { | ||
| 208 | switch (scu_rev & 0x300) { | ||
| 209 | case 0x0100: | ||
| 210 | chip = AST2510; | ||
| 211 | dev_info(dev, "AST 2510 detected\n"); | ||
| 212 | break; | ||
| 213 | default: | ||
| 214 | chip = AST2500; | ||
| 215 | dev_info(dev, "AST 2500 detected\n"); | ||
| 216 | break; | ||
| 217 | } | ||
| 218 | } else if (pdev->revision >= 0x30) { | ||
| 219 | switch (scu_rev & 0x300) { | ||
| 220 | case 0x0100: | ||
| 221 | chip = AST1400; | ||
| 222 | dev_info(dev, "AST 1400 detected\n"); | ||
| 223 | break; | ||
| 224 | default: | ||
| 225 | chip = AST2400; | ||
| 226 | dev_info(dev, "AST 2400 detected\n"); | ||
| 227 | break; | ||
| 228 | } | ||
| 229 | } else if (pdev->revision >= 0x20) { | ||
| 230 | switch (scu_rev & 0x300) { | ||
| 231 | case 0x0000: | ||
| 232 | chip = AST1300; | ||
| 233 | dev_info(dev, "AST 1300 detected\n"); | ||
| 234 | break; | ||
| 235 | default: | ||
| 236 | chip = AST2300; | ||
| 237 | dev_info(dev, "AST 2300 detected\n"); | ||
| 238 | break; | ||
| 239 | } | ||
| 240 | } else if (pdev->revision >= 0x10) { | ||
| 241 | switch (scu_rev & 0x0300) { | ||
| 242 | case 0x0200: | ||
| 243 | chip = AST1100; | ||
| 244 | dev_info(dev, "AST 1100 detected\n"); | ||
| 245 | break; | ||
| 246 | case 0x0100: | ||
| 247 | chip = AST2200; | ||
| 248 | dev_info(dev, "AST 2200 detected\n"); | ||
| 249 | break; | ||
| 250 | case 0x0000: | ||
| 251 | chip = AST2150; | ||
| 252 | dev_info(dev, "AST 2150 detected\n"); | ||
| 253 | break; | ||
| 254 | default: | ||
| 255 | chip = AST2100; | ||
| 256 | dev_info(dev, "AST 2100 detected\n"); | ||
| 257 | break; | ||
| 258 | } | ||
| 259 | } else { | ||
| 260 | chip = AST2000; | ||
| 261 | dev_info(dev, "AST 2000 detected\n"); | ||
| 262 | } | ||
| 263 | |||
| 264 | *chip_out = chip; | ||
| 265 | *config_mode_out = config_mode; | ||
| 266 | |||
| 267 | return 0; | ||
| 268 | } | ||
| 269 | |||
| 92 | static int ast_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) | 270 | static int ast_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) |
| 93 | { | 271 | { |
| 94 | struct ast_device *ast; | 272 | struct device *dev = &pdev->dev; |
| 95 | struct drm_device *dev; | ||
| 96 | int ret; | 273 | int ret; |
| 274 | void __iomem *regs; | ||
| 275 | void __iomem *ioregs; | ||
| 276 | enum ast_config_mode config_mode; | ||
| 277 | enum ast_chip chip; | ||
| 278 | struct drm_device *drm; | ||
| 279 | bool need_post = false; | ||
| 97 | 280 | ||
| 98 | ret = drm_aperture_remove_conflicting_pci_framebuffers(pdev, &ast_driver); | 281 | ret = drm_aperture_remove_conflicting_pci_framebuffers(pdev, &ast_driver); |
| 99 | if (ret) | 282 | if (ret) |
| @@ -103,16 +286,80 @@ static int ast_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 3 unmodified lines | |||
| 103 | if (ret) | 286 | if (ret) |
| 104 | return ret; | 287 | return ret; |
| 105 | 288 | ||
| 106 | ast = ast_device_create(&ast_driver, pdev, ent->driver_data); | 289 | regs = pcim_iomap(pdev, 1, 0); |
| 107 | if (IS_ERR(ast)) | 290 | if (!regs) |
| 108 | return PTR_ERR(ast); | 291 | return -EIO; |
| 109 | dev = &ast->base; | 292 | |
| 293 | if (pdev->revision >= 0x40) { | ||
| 294 | /* | ||
| 295 | * On AST2500 and later models, MMIO is enabled by | ||
| 296 | * default. Adopt it to be compatible with ARM. | ||
| 297 | */ | ||
| 298 | resource_size_t len = pci_resource_len(pdev, 1); | ||
| 299 | |||
| 300 | if (len < AST_IO_MM_OFFSET) | ||
| 301 | return -EIO; | ||
| 302 | if ((len - AST_IO_MM_OFFSET) < AST_IO_MM_LENGTH) | ||
| 303 | return -EIO; | ||
| 304 | ioregs = regs + AST_IO_MM_OFFSET; | ||
| 305 | } else if (pci_resource_flags(pdev, 2) & IORESOURCE_IO) { | ||
| 306 | /* | ||
| 307 | * Map I/O registers if we have a PCI BAR for I/O. | ||
| 308 | */ | ||
| 309 | resource_size_t len = pci_resource_len(pdev, 2); | ||
| 310 | |||
| 311 | if (len < AST_IO_MM_LENGTH) | ||
| 312 | return -EIO; | ||
| 313 | ioregs = pcim_iomap(pdev, 2, 0); | ||
| 314 | if (!ioregs) | ||
| 315 | return -EIO; | ||
| 316 | } else { | ||
| 317 | /* | ||
| 318 | * Anything else is best effort. | ||
| 319 | */ | ||
| 320 | resource_size_t len = pci_resource_len(pdev, 1); | ||
| 321 | |||
| 322 | if (len < AST_IO_MM_OFFSET) | ||
| 323 | return -EIO; | ||
| 324 | if ((len - AST_IO_MM_OFFSET) < AST_IO_MM_LENGTH) | ||
| 325 | return -EIO; | ||
| 326 | ioregs = regs + AST_IO_MM_OFFSET; | ||
| 327 | |||
| 328 | dev_info(dev, "Platform has no I/O space, using MMIO\n"); | ||
| 329 | } | ||
| 330 | |||
| 331 | if (!ast_is_vga_enabled(ioregs)) { | ||
| 332 | dev_info(dev, "VGA not enabled on entry, requesting chip POST\n"); | ||
| 333 | need_post = true; | ||
| 334 | } | ||
| 335 | |||
| 336 | /* | ||
| 337 | * If VGA isn't enabled, we need to enable now or subsequent | ||
| 338 | * access to the scratch registers will fail. | ||
| 339 | */ | ||
| 340 | if (need_post) | ||
| 341 | ast_enable_vga(ioregs); | ||
| 342 | /* Enable extended register access */ | ||
| 343 | ast_open_key(ioregs); | ||
| 344 | |||
| 345 | ret = ast_enable_mmio(dev, ioregs); | ||
| 346 | if (ret) | ||
| 347 | return ret; | ||
| 348 | |||
| 349 | ret = ast_detect_chip(pdev, regs, ioregs, &chip, &config_mode); | ||
| 350 | if (ret) | ||
| 351 | return ret; | ||
| 352 | |||
| 353 | drm = ast_device_create(pdev, &ast_driver, chip, config_mode, regs, ioregs, need_post); | ||
| 354 | if (IS_ERR(drm)) | ||
| 355 | return PTR_ERR(drm); | ||
| 356 | pci_set_drvdata(pdev, drm); | ||
| 110 | 357 | ||
| 111 | ret = drm_dev_register(dev, ent->driver_data); | 358 | ret = drm_dev_register(drm, ent->driver_data); |
| 112 | if (ret) | 359 | if (ret) |
| 113 | return ret; | 360 | return ret; |
| 114 | 361 | ||
| 115 | drm_fbdev_generic_setup(dev, 32); | 362 | drm_fbdev_generic_setup(drm, 32); |
| 116 | 363 | ||
| 117 | return 0; | 364 | return 0; |
| 118 | } | 365 | } |
| @@ -98,6 +98,12 @@ enum ast_tx_chip { · 97 unmodified lines | |||
| 98 | #define AST_TX_DP501_BIT BIT(AST_TX_DP501) | 98 | #define AST_TX_DP501_BIT BIT(AST_TX_DP501) |
| 99 | #define AST_TX_ASTDP_BIT BIT(AST_TX_ASTDP) | 99 | #define AST_TX_ASTDP_BIT BIT(AST_TX_ASTDP) |
| 100 | 100 | ||
| 101 | enum ast_config_mode { | ||
| 102 | ast_use_p2a, | ||
| 103 | ast_use_dt, | ||
| 104 | ast_use_defaults | ||
| 105 | }; | ||
| 106 | |||
| 101 | #define AST_DRAM_512Mx16 0 | 107 | #define AST_DRAM_512Mx16 0 |
| 102 | #define AST_DRAM_1Gx16 1 | 108 | #define AST_DRAM_1Gx16 1 |
| 103 | #define AST_DRAM_512Mx32 2 | 109 | #define AST_DRAM_512Mx32 2 |
| @@ -174,6 +180,17 @@ to_ast_sil164_connector(struct drm_connector *connector) · 70 unmodified lines | |||
| 174 | return container_of(connector, struct ast_sil164_connector, base); | 180 | return container_of(connector, struct ast_sil164_connector, base); |
| 175 | } | 181 | } |
| 176 | 182 | ||
| 183 | struct ast_bmc_connector { | ||
| 184 | struct drm_connector base; | ||
| 185 | struct drm_connector *physical_connector; | ||
| 186 | }; | ||
| 187 | |||
| 188 | static inline struct ast_bmc_connector * | ||
| 189 | to_ast_bmc_connector(struct drm_connector *connector) | ||
| 190 | { | ||
| 191 | return container_of(connector, struct ast_bmc_connector, base); | ||
| 192 | } | ||
| 193 | |||
| 177 | /* | 194 | /* |
| 178 | * Device | 195 | * Device |
| 179 | */ | 196 | */ |
| @@ -181,12 +198,13 @@ to_ast_sil164_connector(struct drm_connector *connector) · 1 unmodified line | |||
| 181 | struct ast_device { | 198 | struct ast_device { |
| 182 | struct drm_device base; | 199 | struct drm_device base; |
| 183 | 200 | ||
| 184 | struct mutex ioregs_lock; /* Protects access to I/O registers in ioregs */ | ||
| 185 | void __iomem *regs; | 201 | void __iomem *regs; |
| 186 | void __iomem *ioregs; | 202 | void __iomem *ioregs; |
| 187 | void __iomem *dp501_fw_buf; | 203 | void __iomem *dp501_fw_buf; |
| 188 | 204 | ||
| 205 | enum ast_config_mode config_mode; | ||
| 189 | enum ast_chip chip; | 206 | enum ast_chip chip; |
| 207 | |||
| 190 | uint32_t dram_bus_width; | 208 | uint32_t dram_bus_width; |
| 191 | uint32_t dram_type; | 209 | uint32_t dram_type; |
| 192 | uint32_t mclk; | 210 | uint32_t mclk; |
| @@ -196,6 +214,8 @@ struct ast_device { · 3 unmodified lines | |||
| 196 | unsigned long vram_size; | 214 | unsigned long vram_size; |
| 197 | unsigned long vram_fb_available; | 215 | unsigned long vram_fb_available; |
| 198 | 216 | ||
| 217 | struct mutex modeset_lock; /* Protects access to modeset I/O registers in ioregs */ | ||
| 218 | |||
| 199 | struct ast_plane primary_plane; | 219 | struct ast_plane primary_plane; |
| 200 | struct ast_plane cursor_plane; | 220 | struct ast_plane cursor_plane; |
| 201 | struct drm_crtc crtc; | 221 | struct drm_crtc crtc; |
| @@ -218,16 +238,11 @@ struct ast_device { · 16 unmodified lines | |||
| 218 | } astdp; | 238 | } astdp; |
| 219 | struct { | 239 | struct { |
| 220 | struct drm_encoder encoder; | 240 | struct drm_encoder encoder; |
| 221 | struct drm_connector connector; | 241 | struct ast_bmc_connector bmc_connector; |
| 222 | } bmc; | 242 | } bmc; |
| 223 | } output; | 243 | } output; |
| 224 | 244 | ||
| 225 | bool support_wide_screen; | 245 | bool support_wide_screen; |
| 226 | enum { | ||
| 227 | ast_use_p2a, | ||
| 228 | ast_use_dt, | ||
| 229 | ast_use_defaults | ||
| 230 | } config_mode; | ||
| 231 | 246 | ||
| 232 | unsigned long tx_chip_types; /* bitfield of enum ast_chip_type */ | 247 | unsigned long tx_chip_types; /* bitfield of enum ast_chip_type */ |
| 233 | u8 *dp501_fw_addr; | 248 | u8 *dp501_fw_addr; |
| @@ -239,9 +254,13 @@ static inline struct ast_device *to_ast_device(struct drm_device *dev) · 5 unmodified lines | |||
| 239 | return container_of(dev, struct ast_device, base); | 254 | return container_of(dev, struct ast_device, base); |
| 240 | } | 255 | } |
| 241 | 256 | ||
| 242 | struct ast_device *ast_device_create(const struct drm_driver *drv, | 257 | struct drm_device *ast_device_create(struct pci_dev *pdev, |
| 243 | struct pci_dev *pdev, | 258 | const struct drm_driver *drv, |
| 244 | unsigned long flags); | 259 | enum ast_chip chip, |
| 260 | enum ast_config_mode config_mode, | ||
| 261 | void __iomem *regs, | ||
| 262 | void __iomem *ioregs, | ||
| 263 | bool need_post); | ||
| 245 | 264 | ||
| 246 | static inline unsigned long __ast_gen(struct ast_device *ast) | 265 | static inline unsigned long __ast_gen(struct ast_device *ast) |
| 247 | { | 266 | { |
| @@ -261,55 +280,94 @@ static inline bool __ast_gen_is_eq(struct ast_device *ast, unsigned long gen) · 13 unmodified lines | |||
| 261 | #define IS_AST_GEN6(__ast) __ast_gen_is_eq(__ast, 6) | 280 | #define IS_AST_GEN6(__ast) __ast_gen_is_eq(__ast, 6) |
| 262 | #define IS_AST_GEN7(__ast) __ast_gen_is_eq(__ast, 7) | 281 | #define IS_AST_GEN7(__ast) __ast_gen_is_eq(__ast, 7) |
| 263 | 282 | ||
| 283 | static inline u8 __ast_read8(const void __iomem *addr, u32 reg) | ||
| 284 | { | ||
| 285 | return ioread8(addr + reg); | ||
| 286 | } | ||
| 287 | |||
| 288 | static inline u32 __ast_read32(const void __iomem *addr, u32 reg) | ||
| 289 | { | ||
| 290 | return ioread32(addr + reg); | ||
| 291 | } | ||
| 292 | |||
| 293 | static inline void __ast_write8(void __iomem *addr, u32 reg, u8 val) | ||
| 294 | { | ||
| 295 | iowrite8(val, addr + reg); | ||
| 296 | } | ||
| 297 | |||
| 298 | static inline void __ast_write32(void __iomem *addr, u32 reg, u32 val) | ||
| 299 | { | ||
| 300 | iowrite32(val, addr + reg); | ||
| 301 | } | ||
| 302 | |||
| 303 | static inline u8 __ast_read8_i(void __iomem *addr, u32 reg, u8 index) | ||
| 304 | { | ||
| 305 | __ast_write8(addr, reg, index); | ||
| 306 | return __ast_read8(addr, reg + 1); | ||
| 307 | } | ||
| 308 | |||
| 309 | static inline u8 __ast_read8_i_masked(void __iomem *addr, u32 reg, u8 index, u8 read_mask) | ||
| 310 | { | ||
| 311 | u8 val = __ast_read8_i(addr, reg, index); | ||
| 312 | |||
| 313 | return val & read_mask; | ||
| 314 | } | ||
| 315 | |||
| 316 | static inline void __ast_write8_i(void __iomem *addr, u32 reg, u8 index, u8 val) | ||
| 317 | { | ||
| 318 | __ast_write8(addr, reg, index); | ||
| 319 | __ast_write8(addr, reg + 1, val); | ||
| 320 | } | ||
| 321 | |||
| 322 | static inline void __ast_write8_i_masked(void __iomem *addr, u32 reg, u8 index, u8 read_mask, | ||
| 323 | u8 val) | ||
| 324 | { | ||
| 325 | u8 tmp = __ast_read8_i_masked(addr, reg, index, read_mask); | ||
| 326 | |||
| 327 | tmp |= val; | ||
| 328 | __ast_write8_i(addr, reg, index, tmp); | ||
| 329 | } | ||
| 330 | |||
| 264 | static inline u32 ast_read32(struct ast_device *ast, u32 reg) | 331 | static inline u32 ast_read32(struct ast_device *ast, u32 reg) |
| 265 | { | 332 | { |
| 266 | return ioread32(ast->regs + reg); | 333 | return __ast_read32(ast->regs, reg); |
| 267 | } | 334 | } |
| 268 | 335 | ||
| 269 | static inline void ast_write32(struct ast_device *ast, u32 reg, u32 val) | 336 | static inline void ast_write32(struct ast_device *ast, u32 reg, u32 val) |
| 270 | { | 337 | { |
| 271 | iowrite32(val, ast->regs + reg); | 338 | __ast_write32(ast->regs, reg, val); |
| 272 | } | 339 | } |
| 273 | 340 | ||
| 274 | static inline u8 ast_io_read8(struct ast_device *ast, u32 reg) | 341 | static inline u8 ast_io_read8(struct ast_device *ast, u32 reg) |
| 275 | { | 342 | { |
| 276 | return ioread8(ast->ioregs + reg); | 343 | return __ast_read8(ast->ioregs, reg); |
| 277 | } | 344 | } |
| 278 | 345 | ||
| 279 | static inline void ast_io_write8(struct ast_device *ast, u32 reg, u8 val) | 346 | static inline void ast_io_write8(struct ast_device *ast, u32 reg, u8 val) |
| 280 | { | 347 | { |
| 281 | iowrite8(val, ast->ioregs + reg); | 348 | __ast_write8(ast->ioregs, reg, val); |
| 282 | } | 349 | } |
| 283 | 350 | ||
| 284 | static inline u8 ast_get_index_reg(struct ast_device *ast, u32 base, u8 index) | 351 | static inline u8 ast_get_index_reg(struct ast_device *ast, u32 base, u8 index) |
| 285 | { | 352 | { |
| 286 | ast_io_write8(ast, base, index); | 353 | return __ast_read8_i(ast->ioregs, base, index); |
| 287 | ++base; | ||
| 288 | return ast_io_read8(ast, base); | ||
| 289 | } | 354 | } |
| 290 | 355 | ||
| 291 | static inline u8 ast_get_index_reg_mask(struct ast_device *ast, u32 base, u8 index, | 356 | static inline u8 ast_get_index_reg_mask(struct ast_device *ast, u32 base, u8 index, |
| 292 | u8 preserve_mask) | 357 | u8 preserve_mask) |
| 293 | { | 358 | { |
| 294 | u8 val = ast_get_index_reg(ast, base, index); | 359 | return __ast_read8_i_masked(ast->ioregs, base, index, preserve_mask); |
| 295 | |||
| 296 | return val & preserve_mask; | ||
| 297 | } | 360 | } |
| 298 | 361 | ||
| 299 | static inline void ast_set_index_reg(struct ast_device *ast, u32 base, u8 index, u8 val) | 362 | static inline void ast_set_index_reg(struct ast_device *ast, u32 base, u8 index, u8 val) |
| 300 | { | 363 | { |
| 301 | ast_io_write8(ast, base, index); | 364 | __ast_write8_i(ast->ioregs, base, index, val); |
| 302 | ++base; | ||
| 303 | ast_io_write8(ast, base, val); | ||
| 304 | } | 365 | } |
| 305 | 366 | ||
| 306 | static inline void ast_set_index_reg_mask(struct ast_device *ast, u32 base, u8 index, | 367 | static inline void ast_set_index_reg_mask(struct ast_device *ast, u32 base, u8 index, |
| 307 | u8 preserve_mask, u8 val) | 368 | u8 preserve_mask, u8 val) |
| 308 | { | 369 | { |
| 309 | u8 tmp = ast_get_index_reg_mask(ast, base, index, preserve_mask); | 370 | __ast_write8_i_masked(ast->ioregs, base, index, preserve_mask, val); |
| 310 | |||
| 311 | tmp |= val; | ||
| 312 | ast_set_index_reg(ast, base, index, tmp); | ||
| 313 | } | 371 | } |
| 314 | 372 | ||
| 315 | #define AST_VIDMEM_SIZE_8M 0x00800000 | 373 | #define AST_VIDMEM_SIZE_8M 0x00800000 |
| @@ -431,7 +489,7 @@ int ast_mm_init(struct ast_device *ast); · 115 unmodified lines | |||
| 431 | void ast_post_gpu(struct drm_device *dev); | 489 | void ast_post_gpu(struct drm_device *dev); |
| 432 | u32 ast_mindwm(struct ast_device *ast, u32 r); | 490 | u32 ast_mindwm(struct ast_device *ast, u32 r); |
| 433 | void ast_moutdwm(struct ast_device *ast, u32 r, u32 v); | 491 | void ast_moutdwm(struct ast_device *ast, u32 r, u32 v); |
| 434 | void ast_patch_ahb_2500(struct ast_device *ast); | 492 | void ast_patch_ahb_2500(void __iomem *regs); |
| 435 | /* ast dp501 */ | 493 | /* ast dp501 */ |
| 436 | void ast_set_dp501_video_output(struct drm_device *dev, u8 mode); | 494 | void ast_set_dp501_video_output(struct drm_device *dev, u8 mode); |
| 437 | bool ast_backup_fw(struct drm_device *dev, u8 *addr, u32 size); | 495 | bool ast_backup_fw(struct drm_device *dev, u8 *addr, u32 size); |
| @@ -35,180 +35,6 @@ · 34 unmodified lines | |||
| 35 | 35 | ||
| 36 | #include "ast_drv.h" | 36 | #include "ast_drv.h" |
| 37 | 37 | ||
| 38 | static bool ast_is_vga_enabled(struct drm_device *dev) | ||
| 39 | { | ||
| 40 | struct ast_device *ast = to_ast_device(dev); | ||
| 41 | u8 ch; | ||
| 42 | |||
| 43 | ch = ast_io_read8(ast, AST_IO_VGAER); | ||
| 44 | |||
| 45 | return !!(ch & 0x01); | ||
| 46 | } | ||
| 47 | |||
| 48 | static void ast_enable_vga(struct drm_device *dev) | ||
| 49 | { | ||
| 50 | struct ast_device *ast = to_ast_device(dev); | ||
| 51 | |||
| 52 | ast_io_write8(ast, AST_IO_VGAER, 0x01); | ||
| 53 | ast_io_write8(ast, AST_IO_VGAMR_W, 0x01); | ||
| 54 | } | ||
| 55 | |||
| 56 | /* | ||
| 57 | * Run this function as part of the HW device cleanup; not | ||
| 58 | * when the DRM device gets released. | ||
| 59 | */ | ||
| 60 | static void ast_enable_mmio_release(void *data) | ||
| 61 | { | ||
| 62 | struct ast_device *ast = data; | ||
| 63 | |||
| 64 | /* enable standard VGA decode */ | ||
| 65 | ast_set_index_reg(ast, AST_IO_VGACRI, 0xa1, 0x04); | ||
| 66 | } | ||
| 67 | |||
| 68 | static int ast_enable_mmio(struct ast_device *ast) | ||
| 69 | { | ||
| 70 | struct drm_device *dev = &ast->base; | ||
| 71 | |||
| 72 | ast_set_index_reg(ast, AST_IO_VGACRI, 0xa1, 0x06); | ||
| 73 | |||
| 74 | return devm_add_action_or_reset(dev->dev, ast_enable_mmio_release, ast); | ||
| 75 | } | ||
| 76 | |||
| 77 | static void ast_open_key(struct ast_device *ast) | ||
| 78 | { | ||
| 79 | ast_set_index_reg(ast, AST_IO_VGACRI, 0x80, 0xA8); | ||
| 80 | } | ||
| 81 | |||
| 82 | static int ast_device_config_init(struct ast_device *ast) | ||
| 83 | { | ||
| 84 | struct drm_device *dev = &ast->base; | ||
| 85 | struct pci_dev *pdev = to_pci_dev(dev->dev); | ||
| 86 | struct device_node *np = dev->dev->of_node; | ||
| 87 | uint32_t scu_rev = 0xffffffff; | ||
| 88 | u32 data; | ||
| 89 | u8 jregd0, jregd1; | ||
| 90 | |||
| 91 | /* | ||
| 92 | * Find configuration mode and read SCU revision | ||
| 93 | */ | ||
| 94 | |||
| 95 | ast->config_mode = ast_use_defaults; | ||
| 96 | |||
| 97 | /* Check if we have device-tree properties */ | ||
| 98 | if (np && !of_property_read_u32(np, "aspeed,scu-revision-id", &data)) { | ||
| 99 | /* We do, disable P2A access */ | ||
| 100 | ast->config_mode = ast_use_dt; | ||
| 101 | scu_rev = data; | ||
| 102 | } else if (pdev->device == PCI_CHIP_AST2000) { // Not all families have a P2A bridge | ||
| 103 | /* | ||
| 104 | * The BMC will set SCU 0x40 D[12] to 1 if the P2 bridge | ||
| 105 | * is disabled. We force using P2A if VGA only mode bit | ||
| 106 | * is set D[7] | ||
| 107 | */ | ||
| 108 | jregd0 = ast_get_index_reg_mask(ast, AST_IO_VGACRI, 0xd0, 0xff); | ||
| 109 | jregd1 = ast_get_index_reg_mask(ast, AST_IO_VGACRI, 0xd1, 0xff); | ||
| 110 | if (!(jregd0 & 0x80) || !(jregd1 & 0x10)) { | ||
| 111 | |||
| 112 | /* | ||
| 113 | * We have a P2A bridge and it is enabled. | ||
| 114 | */ | ||
| 115 | |||
| 116 | /* Patch AST2500/AST2510 */ | ||
| 117 | if ((pdev->revision & 0xf0) == 0x40) { | ||
| 118 | if (!(jregd0 & AST_VRAM_INIT_STATUS_MASK)) | ||
| 119 | ast_patch_ahb_2500(ast); | ||
| 120 | } | ||
| 121 | |||
| 122 | /* Double check that it's actually working */ | ||
| 123 | data = ast_read32(ast, 0xf004); | ||
| 124 | if ((data != 0xffffffff) && (data != 0x00)) { | ||
| 125 | ast->config_mode = ast_use_p2a; | ||
| 126 | |||
| 127 | /* Read SCU7c (silicon revision register) */ | ||
| 128 | ast_write32(ast, 0xf004, 0x1e6e0000); | ||
| 129 | ast_write32(ast, 0xf000, 0x1); | ||
| 130 | scu_rev = ast_read32(ast, 0x1207c); | ||
| 131 | } | ||
| 132 | } | ||
| 133 | } | ||
| 134 | |||
| 135 | switch (ast->config_mode) { | ||
| 136 | case ast_use_defaults: | ||
| 137 | drm_info(dev, "Using default configuration\n"); | ||
| 138 | break; | ||
| 139 | case ast_use_dt: | ||
| 140 | drm_info(dev, "Using device-tree for configuration\n"); | ||
| 141 | break; | ||
| 142 | case ast_use_p2a: | ||
| 143 | drm_info(dev, "Using P2A bridge for configuration\n"); | ||
| 144 | break; | ||
| 145 | } | ||
| 146 | |||
| 147 | /* | ||
| 148 | * Identify chipset | ||
| 149 | */ | ||
| 150 | |||
| 151 | if (pdev->revision >= 0x50) { | ||
| 152 | ast->chip = AST2600; | ||
| 153 | drm_info(dev, "AST 2600 detected\n"); | ||
| 154 | } else if (pdev->revision >= 0x40) { | ||
| 155 | switch (scu_rev & 0x300) { | ||
| 156 | case 0x0100: | ||
| 157 | ast->chip = AST2510; | ||
| 158 | drm_info(dev, "AST 2510 detected\n"); | ||
| 159 | break; | ||
| 160 | default: | ||
| 161 | ast->chip = AST2500; | ||
| 162 | drm_info(dev, "AST 2500 detected\n"); | ||
| 163 | } | ||
| 164 | } else if (pdev->revision >= 0x30) { | ||
| 165 | switch (scu_rev & 0x300) { | ||
| 166 | case 0x0100: | ||
| 167 | ast->chip = AST1400; | ||
| 168 | drm_info(dev, "AST 1400 detected\n"); | ||
| 169 | break; | ||
| 170 | default: | ||
| 171 | ast->chip = AST2400; | ||
| 172 | drm_info(dev, "AST 2400 detected\n"); | ||
| 173 | } | ||
| 174 | } else if (pdev->revision >= 0x20) { | ||
| 175 | switch (scu_rev & 0x300) { | ||
| 176 | case 0x0000: | ||
| 177 | ast->chip = AST1300; | ||
| 178 | drm_info(dev, "AST 1300 detected\n"); | ||
| 179 | break; | ||
| 180 | default: | ||
| 181 | ast->chip = AST2300; | ||
| 182 | drm_info(dev, "AST 2300 detected\n"); | ||
| 183 | break; | ||
| 184 | } | ||
| 185 | } else if (pdev->revision >= 0x10) { | ||
| 186 | switch (scu_rev & 0x0300) { | ||
| 187 | case 0x0200: | ||
| 188 | ast->chip = AST1100; | ||
| 189 | drm_info(dev, "AST 1100 detected\n"); | ||
| 190 | break; | ||
| 191 | case 0x0100: | ||
| 192 | ast->chip = AST2200; | ||
| 193 | drm_info(dev, "AST 2200 detected\n"); | ||
| 194 | break; | ||
| 195 | case 0x0000: | ||
| 196 | ast->chip = AST2150; | ||
| 197 | drm_info(dev, "AST 2150 detected\n"); | ||
| 198 | break; | ||
| 199 | default: | ||
| 200 | ast->chip = AST2100; | ||
| 201 | drm_info(dev, "AST 2100 detected\n"); | ||
| 202 | break; | ||
| 203 | } | ||
| 204 | } else { | ||
| 205 | ast->chip = AST2000; | ||
| 206 | drm_info(dev, "AST 2000 detected\n"); | ||
| 207 | } | ||
| 208 | |||
| 209 | return 0; | ||
| 210 | } | ||
| 211 | |||
| 212 | static void ast_detect_widescreen(struct ast_device *ast) | 38 | static void ast_detect_widescreen(struct ast_device *ast) |
| 213 | { | 39 | { |
| 214 | u8 jreg; | 40 | u8 jreg; |
| @@ -424,69 +250,27 @@ static int ast_get_dram_info(struct drm_device *dev) · 209 unmodified lines | |||
| 424 | return 0; | 250 | return 0; |
| 425 | } | 251 | } |
| 426 | 252 | ||
| 427 | struct ast_device *ast_device_create(const struct drm_driver *drv, | 253 | struct drm_device *ast_device_create(struct pci_dev *pdev, |
| 428 | struct pci_dev *pdev, | 254 | const struct drm_driver *drv, |
| 429 | unsigned long flags) | 255 | enum ast_chip chip, |
| 256 | enum ast_config_mode config_mode, | ||
| 257 | void __iomem *regs, | ||
| 258 | void __iomem *ioregs, | ||
| 259 | bool need_post) | ||
| 430 | { | 260 | { |
| 431 | struct drm_device *dev; | 261 | struct drm_device *dev; |
| 432 | struct ast_device *ast; | 262 | struct ast_device *ast; |
| 433 | bool need_post = false; | 263 | int ret; |
| 434 | int ret = 0; | ||
| 435 | 264 | ||
| 436 | ast = devm_drm_dev_alloc(&pdev->dev, drv, struct ast_device, base); | 265 | ast = devm_drm_dev_alloc(&pdev->dev, drv, struct ast_device, base); |
| 437 | if (IS_ERR(ast)) | 266 | if (IS_ERR(ast)) |
| 438 | return ast; | 267 | return ERR_CAST(ast); |
| 439 | dev = &ast->base; | 268 | dev = &ast->base; |
| 440 | 269 | ||
| 441 | pci_set_drvdata(pdev, dev); | 270 | ast->chip = chip; |
| 442 | 271 | ast->config_mode = config_mode; | |
| 443 | ret = drmm_mutex_init(dev, &ast->ioregs_lock); | 272 | ast->regs = regs; |
| 444 | if (ret) | 273 | ast->ioregs = ioregs; |
| 445 | return ERR_PTR(ret); | ||
| 446 | |||
| 447 | ast->regs = pcim_iomap(pdev, 1, 0); | ||
| 448 | if (!ast->regs) | ||
| 449 | return ERR_PTR(-EIO); | ||
| 450 | |||
| 451 | /* | ||
| 452 | * After AST2500, MMIO is enabled by default, and it should be adopted | ||
| 453 | * to be compatible with Arm. | ||
| 454 | */ | ||
| 455 | if (pdev->revision >= 0x40) { | ||
| 456 | ast->ioregs = ast->regs + AST_IO_MM_OFFSET; | ||
| 457 | } else if (!(pci_resource_flags(pdev, 2) & IORESOURCE_IO)) { | ||
| 458 | drm_info(dev, "platform has no IO space, trying MMIO\n"); | ||
| 459 | ast->ioregs = ast->regs + AST_IO_MM_OFFSET; | ||
| 460 | } | ||
| 461 | |||
| 462 | /* "map" IO regs if the above hasn't done so already */ | ||
| 463 | if (!ast->ioregs) { | ||
| 464 | ast->ioregs = pcim_iomap(pdev, 2, 0); | ||
| 465 | if (!ast->ioregs) | ||
| 466 | return ERR_PTR(-EIO); | ||
| 467 | } | ||
| 468 | |||
| 469 | if (!ast_is_vga_enabled(dev)) { | ||
| 470 | drm_info(dev, "VGA not enabled on entry, requesting chip POST\n"); | ||
| 471 | need_post = true; | ||
| 472 | } | ||
| 473 | |||
| 474 | /* | ||
| 475 | * If VGA isn't enabled, we need to enable now or subsequent | ||
| 476 | * access to the scratch registers will fail. | ||
| 477 | */ | ||
| 478 | if (need_post) | ||
| 479 | ast_enable_vga(dev); | ||
| 480 | |||
| 481 | /* Enable extended register access */ | ||
| 482 | ast_open_key(ast); | ||
| 483 | ret = ast_enable_mmio(ast); | ||
| 484 | if (ret) | ||
| 485 | return ERR_PTR(ret); | ||
| 486 | |||
| 487 | ret = ast_device_config_init(ast); | ||
| 488 | if (ret) | ||
| 489 | return ERR_PTR(ret); | ||
| 490 | 274 | ||
| 491 | ast_detect_widescreen(ast); | 275 | ast_detect_widescreen(ast); |
| 492 | ast_detect_tx_chip(ast, need_post); | 276 | ast_detect_tx_chip(ast, need_post); |
| @@ -517,5 +301,5 @@ struct ast_device *ast_device_create(const struct drm_driver *drv, · 24 unmodified lines | |||
| 517 | if (ret) | 301 | if (ret) |
| 518 | return ERR_PTR(ret); | 302 | return ERR_PTR(ret); |
| 519 | 303 | ||
| 520 | return ast; | 304 | return dev; |
| 521 | } | 305 | } |
| @@ -1358,13 +1358,13 @@ static int ast_vga_connector_helper_get_modes(struct drm_connector *connector) · 1357 unmodified lines | |||
| 1358 | * Protect access to I/O registers from concurrent modesetting | 1358 | * Protect access to I/O registers from concurrent modesetting |
| 1359 | * by acquiring the I/O-register lock. | 1359 | * by acquiring the I/O-register lock. |
| 1360 | */ | 1360 | */ |
| 1361 | mutex_lock(&ast->ioregs_lock); | 1361 | mutex_lock(&ast->modeset_lock); |
| 1362 | 1362 | ||
| 1363 | edid = drm_get_edid(connector, &ast_vga_connector->i2c->adapter); | 1363 | edid = drm_get_edid(connector, &ast_vga_connector->i2c->adapter); |
| 1364 | if (!edid) | 1364 | if (!edid) |
| 1365 | goto err_mutex_unlock; | 1365 | goto err_mutex_unlock; |
| 1366 | 1366 | ||
| 1367 | mutex_unlock(&ast->ioregs_lock); | 1367 | mutex_unlock(&ast->modeset_lock); |
| 1368 | 1368 | ||
| 1369 | count = drm_add_edid_modes(connector, edid); | 1369 | count = drm_add_edid_modes(connector, edid); |
| 1370 | kfree(edid); | 1370 | kfree(edid); |
| @@ -1372,7 +1372,7 @@ static int ast_vga_connector_helper_get_modes(struct drm_connector *connector) · 1 unmodified line | |||
| 1372 | return count; | 1372 | return count; |
| 1373 | 1373 | ||
| 1374 | err_mutex_unlock: | 1374 | err_mutex_unlock: |
| 1375 | mutex_unlock(&ast->ioregs_lock); | 1375 | mutex_unlock(&ast->modeset_lock); |
| 1376 | err_drm_connector_update_edid_property: | 1376 | err_drm_connector_update_edid_property: |
| 1377 | drm_connector_update_edid_property(connector, NULL); | 1377 | drm_connector_update_edid_property(connector, NULL); |
| 1378 | return 0; | 1378 | return 0; |
| @@ -1464,13 +1464,13 @@ static int ast_sil164_connector_helper_get_modes(struct drm_connector *connector · 85 unmodified lines | |||
| 1464 | * Protect access to I/O registers from concurrent modesetting | 1464 | * Protect access to I/O registers from concurrent modesetting |
| 1465 | * by acquiring the I/O-register lock. | 1465 | * by acquiring the I/O-register lock. |
| 1466 | */ | 1466 | */ |
| 1467 | mutex_lock(&ast->ioregs_lock); | 1467 | mutex_lock(&ast->modeset_lock); |
| 1468 | 1468 | ||
| 1469 | edid = drm_get_edid(connector, &ast_sil164_connector->i2c->adapter); | 1469 | edid = drm_get_edid(connector, &ast_sil164_connector->i2c->adapter); |
| 1470 | if (!edid) | 1470 | if (!edid) |
| 1471 | goto err_mutex_unlock; | 1471 | goto err_mutex_unlock; |
| 1472 | 1472 | ||
| 1473 | mutex_unlock(&ast->ioregs_lock); | 1473 | mutex_unlock(&ast->modeset_lock); |
| 1474 | 1474 | ||
| 1475 | count = drm_add_edid_modes(connector, edid); | 1475 | count = drm_add_edid_modes(connector, edid); |
| 1476 | kfree(edid); | 1476 | kfree(edid); |
| @@ -1478,7 +1478,7 @@ static int ast_sil164_connector_helper_get_modes(struct drm_connector *connector · 1 unmodified line | |||
| 1478 | return count; | 1478 | return count; |
| 1479 | 1479 | ||
| 1480 | err_mutex_unlock: | 1480 | err_mutex_unlock: |
| 1481 | mutex_unlock(&ast->ioregs_lock); | 1481 | mutex_unlock(&ast->modeset_lock); |
| 1482 | err_drm_connector_update_edid_property: | 1482 | err_drm_connector_update_edid_property: |
| 1483 | drm_connector_update_edid_property(connector, NULL); | 1483 | drm_connector_update_edid_property(connector, NULL); |
| 1484 | return 0; | 1484 | return 0; |
| @@ -1670,13 +1670,13 @@ static int ast_astdp_connector_helper_get_modes(struct drm_connector *connector) · 185 unmodified lines | |||
| 1670 | * Protect access to I/O registers from concurrent modesetting | 1670 | * Protect access to I/O registers from concurrent modesetting |
| 1671 | * by acquiring the I/O-register lock. | 1671 | * by acquiring the I/O-register lock. |
| 1672 | */ | 1672 | */ |
| 1673 | mutex_lock(&ast->ioregs_lock); | 1673 | mutex_lock(&ast->modeset_lock); |
| 1674 | 1674 | ||
| 1675 | succ = ast_astdp_read_edid(connector->dev, edid); | 1675 | succ = ast_astdp_read_edid(connector->dev, edid); |
| 1676 | if (succ < 0) | 1676 | if (succ < 0) |
| 1677 | goto err_mutex_unlock; | 1677 | goto err_mutex_unlock; |
| 1678 | 1678 | ||
| 1679 | mutex_unlock(&ast->ioregs_lock); | 1679 | mutex_unlock(&ast->modeset_lock); |
| 1680 | 1680 | ||
| 1681 | drm_connector_update_edid_property(connector, edid); | 1681 | drm_connector_update_edid_property(connector, edid); |
| 1682 | count = drm_add_edid_modes(connector, edid); | 1682 | count = drm_add_edid_modes(connector, edid); |
| @@ -1685,7 +1685,7 @@ static int ast_astdp_connector_helper_get_modes(struct drm_connector *connector) · 2 unmodified lines | |||
| 1685 | return count; | 1685 | return count; |
| 1686 | 1686 | ||
| 1687 | err_mutex_unlock: | 1687 | err_mutex_unlock: |
| 1688 | mutex_unlock(&ast->ioregs_lock); | 1688 | mutex_unlock(&ast->modeset_lock); |
| 1689 | kfree(edid); | 1689 | kfree(edid); |
| 1690 | err_drm_connector_update_edid_property: | 1690 | err_drm_connector_update_edid_property: |
| 1691 | drm_connector_update_edid_property(connector, NULL); | 1691 | drm_connector_update_edid_property(connector, NULL); |
| @@ -1767,6 +1767,30 @@ static const struct drm_encoder_funcs ast_bmc_encoder_funcs = { · 75 unmodified lines | |||
| 1767 | .destroy = drm_encoder_cleanup, | 1767 | .destroy = drm_encoder_cleanup, |
| 1768 | }; | 1768 | }; |
| 1769 | 1769 | ||
| 1770 | static int ast_bmc_connector_helper_detect_ctx(struct drm_connector *connector, | ||
| 1771 | struct drm_modeset_acquire_ctx *ctx, | ||
| 1772 | bool force) | ||
| 1773 | { | ||
| 1774 | struct ast_bmc_connector *bmc_connector = to_ast_bmc_connector(connector); | ||
| 1775 | struct drm_connector *physical_connector = bmc_connector->physical_connector; | ||
| 1776 | |||
| 1777 | /* | ||
| 1778 | * Most user-space compositors cannot handle more than one connected | ||
| 1779 | * connector per CRTC. Hence, we only mark the BMC as connected if the | ||
| 1780 | * physical connector is disconnected. If the physical connector's status | ||
| 1781 | * is connected or unknown, the BMC remains disconnected. This has no | ||
| 1782 | * effect on the output of the BMC. | ||
| 1783 | * | ||
| 1784 | * FIXME: Remove this logic once user-space compositors can handle more | ||
| 1785 | * than one connector per CRTC. The BMC should always be connected. | ||
| 1786 | */ | ||
| 1787 | |||
| 1788 | if (physical_connector && physical_connector->status == connector_status_disconnected) | ||
| 1789 | return connector_status_connected; | ||
| 1790 | |||
| 1791 | return connector_status_disconnected; | ||
| 1792 | } | ||
| 1793 | |||
| 1770 | static int ast_bmc_connector_helper_get_modes(struct drm_connector *connector) | 1794 | static int ast_bmc_connector_helper_get_modes(struct drm_connector *connector) |
| 1771 | { | 1795 | { |
| 1772 | return drm_add_modes_noedid(connector, 4096, 4096); | 1796 | return drm_add_modes_noedid(connector, 4096, 4096); |
| @@ -1774,6 +1798,7 @@ static int ast_bmc_connector_helper_get_modes(struct drm_connector *connector) · 1 unmodified line | |||
| 1774 | 1798 | ||
| 1775 | static const struct drm_connector_helper_funcs ast_bmc_connector_helper_funcs = { | 1799 | static const struct drm_connector_helper_funcs ast_bmc_connector_helper_funcs = { |
| 1776 | .get_modes = ast_bmc_connector_helper_get_modes, | 1800 | .get_modes = ast_bmc_connector_helper_get_modes, |
| 1801 | .detect_ctx = ast_bmc_connector_helper_detect_ctx, | ||
| 1777 | }; | 1802 | }; |
| 1778 | 1803 | ||
| 1779 | static const struct drm_connector_funcs ast_bmc_connector_funcs = { | 1804 | static const struct drm_connector_funcs ast_bmc_connector_funcs = { |
| @@ -1784,12 +1809,33 @@ static const struct drm_connector_funcs ast_bmc_connector_funcs = { · 4 unmodified lines | |||
| 1784 | .atomic_destroy_state = drm_atomic_helper_connector_destroy_state, | 1809 | .atomic_destroy_state = drm_atomic_helper_connector_destroy_state, |
| 1785 | }; | 1810 | }; |
| 1786 | 1811 | ||
| 1787 | static int ast_bmc_output_init(struct ast_device *ast) | 1812 | static int ast_bmc_connector_init(struct drm_device *dev, |
| 1813 | struct ast_bmc_connector *bmc_connector, | ||
| 1814 | struct drm_connector *physical_connector) | ||
| 1815 | { | ||
| 1816 | struct drm_connector *connector = &bmc_connector->base; | ||
| 1817 | int ret; | ||
| 1818 | |||
| 1819 | ret = drm_connector_init(dev, connector, &ast_bmc_connector_funcs, | ||
| 1820 | DRM_MODE_CONNECTOR_VIRTUAL); | ||
| 1821 | if (ret) | ||
| 1822 | return ret; | ||
| 1823 | |||
| 1824 | drm_connector_helper_add(connector, &ast_bmc_connector_helper_funcs); | ||
| 1825 | |||
| 1826 | bmc_connector->physical_connector = physical_connector; | ||
| 1827 | |||
| 1828 | return 0; | ||
| 1829 | } | ||
| 1830 | |||
| 1831 | static int ast_bmc_output_init(struct ast_device *ast, | ||
| 1832 | struct drm_connector *physical_connector) | ||
| 1788 | { | 1833 | { |
| 1789 | struct drm_device *dev = &ast->base; | 1834 | struct drm_device *dev = &ast->base; |
| 1790 | struct drm_crtc *crtc = &ast->crtc; | 1835 | struct drm_crtc *crtc = &ast->crtc; |
| 1791 | struct drm_encoder *encoder = &ast->output.bmc.encoder; | 1836 | struct drm_encoder *encoder = &ast->output.bmc.encoder; |
| 1792 | struct drm_connector *connector = &ast->output.bmc.connector; | 1837 | struct ast_bmc_connector *bmc_connector = &ast->output.bmc.bmc_connector; |
| 1838 | struct drm_connector *connector = &bmc_connector->base; | ||
| 1793 | int ret; | 1839 | int ret; |
| 1794 | 1840 | ||
| 1795 | ret = drm_encoder_init(dev, encoder, | 1841 | ret = drm_encoder_init(dev, encoder, |
| @@ -1799,13 +1845,10 @@ static int ast_bmc_output_init(struct ast_device *ast) · 3 unmodified lines | |||
| 1799 | return ret; | 1845 | return ret; |
| 1800 | encoder->possible_crtcs = drm_crtc_mask(crtc); | 1846 | encoder->possible_crtcs = drm_crtc_mask(crtc); |
| 1801 | 1847 | ||
| 1802 | ret = drm_connector_init(dev, connector, &ast_bmc_connector_funcs, | 1848 | ret = ast_bmc_connector_init(dev, bmc_connector, physical_connector); |
| 1803 | DRM_MODE_CONNECTOR_VIRTUAL); | ||
| 1804 | if (ret) | 1849 | if (ret) |
| 1805 | return ret; | 1850 | return ret; |
| 1806 | 1851 | ||
| 1807 | drm_connector_helper_add(connector, &ast_bmc_connector_helper_funcs); | ||
| 1808 | |||
| 1809 | ret = drm_connector_attach_encoder(connector, encoder); | 1852 | ret = drm_connector_attach_encoder(connector, encoder); |
| 1810 | if (ret) | 1853 | if (ret) |
| 1811 | return ret; | 1854 | return ret; |
| @@ -1827,9 +1870,9 @@ static void ast_mode_config_helper_atomic_commit_tail(struct drm_atomic_state *s · 15 unmodified lines | |||
| 1827 | * display modes. Protect access to I/O registers by acquiring | 1870 | * display modes. Protect access to I/O registers by acquiring |
| 1828 | * the I/O-register lock. Released in atomic_flush(). | 1871 | * the I/O-register lock. Released in atomic_flush(). |
| 1829 | */ | 1872 | */ |
| 1830 | mutex_lock(&ast->ioregs_lock); | 1873 | mutex_lock(&ast->modeset_lock); |
| 1831 | drm_atomic_helper_commit_tail_rpm(state); | 1874 | drm_atomic_helper_commit_tail_rpm(state); |
| 1832 | mutex_unlock(&ast->ioregs_lock); | 1875 | mutex_unlock(&ast->modeset_lock); |
| 1833 | } | 1876 | } |
| 1834 | 1877 | ||
| 1835 | static const struct drm_mode_config_helper_funcs ast_mode_config_helper_funcs = { | 1878 | static const struct drm_mode_config_helper_funcs ast_mode_config_helper_funcs = { |
| @@ -1864,8 +1907,13 @@ static const struct drm_mode_config_funcs ast_mode_config_funcs = { · 28 unmodified lines | |||
| 1864 | int ast_mode_config_init(struct ast_device *ast) | 1907 | int ast_mode_config_init(struct ast_device *ast) |
| 1865 | { | 1908 | { |
| 1866 | struct drm_device *dev = &ast->base; | 1909 | struct drm_device *dev = &ast->base; |
| 1910 | struct drm_connector *physical_connector = NULL; | ||
| 1867 | int ret; | 1911 | int ret; |
| 1868 | 1912 | ||
| 1913 | ret = drmm_mutex_init(dev, &ast->modeset_lock); | ||
| 1914 | if (ret) | ||
| 1915 | return ret; | ||
| 1916 | |||
| 1869 | ret = drmm_mode_config_init(dev); | 1917 | ret = drmm_mode_config_init(dev); |
| 1870 | if (ret) | 1918 | if (ret) |
| 1871 | return ret; | 1919 | return ret; |
| @@ -1904,23 +1952,27 @@ int ast_mode_config_init(struct ast_device *ast) · 32 unmodified lines | |||
| 1904 | ret = ast_vga_output_init(ast); | 1952 | ret = ast_vga_output_init(ast); |
| 1905 | if (ret) | 1953 | if (ret) |
| 1906 | return ret; | 1954 | return ret; |
| 1955 | physical_connector = &ast->output.vga.vga_connector.base; | ||
| 1907 | } | 1956 | } |
| 1908 | if (ast->tx_chip_types & AST_TX_SIL164_BIT) { | 1957 | if (ast->tx_chip_types & AST_TX_SIL164_BIT) { |
| 1909 | ret = ast_sil164_output_init(ast); | 1958 | ret = ast_sil164_output_init(ast); |
| 1910 | if (ret) | 1959 | if (ret) |
| 1911 | return ret; | 1960 | return ret; |
| 1961 | physical_connector = &ast->output.sil164.sil164_connector.base; | ||
| 1912 | } | 1962 | } |
| 1913 | if (ast->tx_chip_types & AST_TX_DP501_BIT) { | 1963 | if (ast->tx_chip_types & AST_TX_DP501_BIT) { |
| 1914 | ret = ast_dp501_output_init(ast); | 1964 | ret = ast_dp501_output_init(ast); |
| 1915 | if (ret) | 1965 | if (ret) |
| 1916 | return ret; | 1966 | return ret; |
| 1967 | physical_connector = &ast->output.dp501.connector; | ||
| 1917 | } | 1968 | } |
| 1918 | if (ast->tx_chip_types & AST_TX_ASTDP_BIT) { | 1969 | if (ast->tx_chip_types & AST_TX_ASTDP_BIT) { |
| 1919 | ret = ast_astdp_output_init(ast); | 1970 | ret = ast_astdp_output_init(ast); |
| 1920 | if (ret) | 1971 | if (ret) |
| 1921 | return ret; | 1972 | return ret; |
| 1973 | physical_connector = &ast->output.astdp.connector; | ||
| 1922 | } | 1974 | } |
| 1923 | ret = ast_bmc_output_init(ast); | 1975 | ret = ast_bmc_output_init(ast, physical_connector); |
| 1924 | if (ret) | 1976 | if (ret) |
| 1925 | return ret; | 1977 | return ret; |
| 1926 | 1978 | ||
| @@ -77,28 +77,42 @@ ast_set_def_ext_reg(struct drm_device *dev) · 76 unmodified lines | |||
| 77 | ast_set_index_reg_mask(ast, AST_IO_VGACRI, 0xb6, 0xff, reg); | 77 | ast_set_index_reg_mask(ast, AST_IO_VGACRI, 0xb6, 0xff, reg); |
| 78 | } | 78 | } |
| 79 | 79 | ||
| 80 | u32 ast_mindwm(struct ast_device *ast, u32 r) | 80 | static u32 __ast_mindwm(void __iomem *regs, u32 r) |
| 81 | { | 81 | { |
| 82 | uint32_t data; | 82 | u32 data; |
| 83 | 83 | ||
| 84 | ast_write32(ast, 0xf004, r & 0xffff0000); | 84 | __ast_write32(regs, 0xf004, r & 0xffff0000); |
| 85 | ast_write32(ast, 0xf000, 0x1); | 85 | __ast_write32(regs, 0xf000, 0x1); |
| 86 | 86 | ||
| 87 | do { | 87 | do { |
| 88 | data = ast_read32(ast, 0xf004) & 0xffff0000; | 88 | data = __ast_read32(regs, 0xf004) & 0xffff0000; |
| 89 | } while (data != (r & 0xffff0000)); | 89 | } while (data != (r & 0xffff0000)); |
| 90 | return ast_read32(ast, 0x10000 + (r & 0x0000ffff)); | 90 | |
| 91 | return __ast_read32(regs, 0x10000 + (r & 0x0000ffff)); | ||
| 91 | } | 92 | } |
| 92 | 93 | ||
| 93 | void ast_moutdwm(struct ast_device *ast, u32 r, u32 v) | 94 | static void __ast_moutdwm(void __iomem *regs, u32 r, u32 v) |
| 94 | { | 95 | { |
| 95 | uint32_t data; | 96 | u32 data; |
| 96 | ast_write32(ast, 0xf004, r & 0xffff0000); | 97 | |
| 97 | ast_write32(ast, 0xf000, 0x1); | 98 | __ast_write32(regs, 0xf004, r & 0xffff0000); |
| 99 | __ast_write32(regs, 0xf000, 0x1); | ||
| 100 | |||
| 98 | do { | 101 | do { |
| 99 | data = ast_read32(ast, 0xf004) & 0xffff0000; | 102 | data = __ast_read32(regs, 0xf004) & 0xffff0000; |
| 100 | } while (data != (r & 0xffff0000)); | 103 | } while (data != (r & 0xffff0000)); |
| 101 | ast_write32(ast, 0x10000 + (r & 0x0000ffff), v); | 104 | |
| 105 | __ast_write32(regs, 0x10000 + (r & 0x0000ffff), v); | ||
| 106 | } | ||
| 107 | |||
| 108 | u32 ast_mindwm(struct ast_device *ast, u32 r) | ||
| 109 | { | ||
| 110 | return __ast_mindwm(ast->regs, r); | ||
| 111 | } | ||
| 112 | |||
| 113 | void ast_moutdwm(struct ast_device *ast, u32 r, u32 v) | ||
| 114 | { | ||
| 115 | __ast_moutdwm(ast->regs, r, v); | ||
| 102 | } | 116 | } |
| 103 | 117 | ||
| 104 | /* | 118 | /* |
| @@ -1987,17 +2001,18 @@ static bool ast_dram_init_2500(struct ast_device *ast) · 1882 unmodified lines | |||
| 1987 | return true; | 2001 | return true; |
| 1988 | } | 2002 | } |
| 1989 | 2003 | ||
| 1990 | void ast_patch_ahb_2500(struct ast_device *ast) | 2004 | void ast_patch_ahb_2500(void __iomem *regs) |
| 1991 | { | 2005 | { |
| 1992 | u32 data; | 2006 | u32 data; |
| 1993 | 2007 | ||
| 1994 | /* Clear bus lock condition */ | 2008 | /* Clear bus lock condition */ |
| 1995 | ast_moutdwm(ast, 0x1e600000, 0xAEED1A03); | 2009 | __ast_moutdwm(regs, 0x1e600000, 0xAEED1A03); |
| 1996 | ast_moutdwm(ast, 0x1e600084, 0x00010000); | 2010 | __ast_moutdwm(regs, 0x1e600084, 0x00010000); |
| 1997 | ast_moutdwm(ast, 0x1e600088, 0x00000000); | 2011 | __ast_moutdwm(regs, 0x1e600088, 0x00000000); |
| 1998 | ast_moutdwm(ast, 0x1e6e2000, 0x1688A8A8); | 2012 | __ast_moutdwm(regs, 0x1e6e2000, 0x1688A8A8); |
| 1999 | data = ast_mindwm(ast, 0x1e6e2070); | 2013 | |
| 2000 | if (data & 0x08000000) { /* check fast reset */ | 2014 | data = __ast_mindwm(regs, 0x1e6e2070); |
| 2015 | if (data & 0x08000000) { /* check fast reset */ | ||
| 2001 | /* | 2016 | /* |
| 2002 | * If "Fast restet" is enabled for ARM-ICE debugger, | 2017 | * If "Fast restet" is enabled for ARM-ICE debugger, |
| 2003 | * then WDT needs to enable, that | 2018 | * then WDT needs to enable, that |
| @@ -2009,16 +2024,18 @@ void ast_patch_ahb_2500(struct ast_device *ast) · 5 unmodified lines | |||
| 2009 | * [1]:= 1:WDT will be cleeared and disabled after timeout occurs | 2024 | * [1]:= 1:WDT will be cleeared and disabled after timeout occurs |
| 2010 | * [0]:= 1:WDT enable | 2025 | * [0]:= 1:WDT enable |
| 2011 | */ | 2026 | */ |
| 2012 | ast_moutdwm(ast, 0x1E785004, 0x00000010); | 2027 | __ast_moutdwm(regs, 0x1E785004, 0x00000010); |
| 2013 | ast_moutdwm(ast, 0x1E785008, 0x00004755); | 2028 | __ast_moutdwm(regs, 0x1E785008, 0x00004755); |
| 2014 | ast_moutdwm(ast, 0x1E78500c, 0x00000033); | 2029 | __ast_moutdwm(regs, 0x1E78500c, 0x00000033); |
| 2015 | udelay(1000); | 2030 | udelay(1000); |
| 2016 | } | 2031 | } |
| 2032 | |||
| 2017 | do { | 2033 | do { |
| 2018 | ast_moutdwm(ast, 0x1e6e2000, 0x1688A8A8); | 2034 | __ast_moutdwm(regs, 0x1e6e2000, 0x1688A8A8); |
| 2019 | data = ast_mindwm(ast, 0x1e6e2000); | 2035 | data = __ast_mindwm(regs, 0x1e6e2000); |
| 2020 | } while (data != 1); | 2036 | } while (data != 1); |
| 2021 | ast_moutdwm(ast, 0x1e6e207c, 0x08000000); /* clear fast reset */ | 2037 | |
| 2038 | __ast_moutdwm(regs, 0x1e6e207c, 0x08000000); /* clear fast reset */ | ||
| 2022 | } | 2039 | } |
| 2023 | 2040 | ||
| 2024 | void ast_post_chip_2500(struct drm_device *dev) | 2041 | void ast_post_chip_2500(struct drm_device *dev) |
| @@ -2030,7 +2047,7 @@ void ast_post_chip_2500(struct drm_device *dev) · 5 unmodified lines | |||
| 2030 | reg = ast_get_index_reg_mask(ast, AST_IO_VGACRI, 0xd0, 0xff); | 2047 | reg = ast_get_index_reg_mask(ast, AST_IO_VGACRI, 0xd0, 0xff); |
| 2031 | if ((reg & AST_VRAM_INIT_STATUS_MASK) == 0) {/* vga only */ | 2048 | if ((reg & AST_VRAM_INIT_STATUS_MASK) == 0) {/* vga only */ |
| 2032 | /* Clear bus lock condition */ | 2049 | /* Clear bus lock condition */ |
| 2033 | ast_patch_ahb_2500(ast); | 2050 | ast_patch_ahb_2500(ast->regs); |
| 2034 | 2051 | ||
| 2035 | /* Disable watchdog */ | 2052 | /* Disable watchdog */ |
| 2036 | ast_moutdwm(ast, 0x1E78502C, 0x00000000); | 2053 | ast_moutdwm(ast, 0x1E78502C, 0x00000000); |
| @@ -10,10 +10,17 @@ · 9 unmodified lines | |||
| 10 | */ | 10 | */ |
| 11 | 11 | ||
| 12 | #define AST_IO_MM_OFFSET (0x380) | 12 | #define AST_IO_MM_OFFSET (0x380) |
| 13 | #define AST_IO_MM_LENGTH (128) | ||
| 13 | 14 | ||
| 14 | #define AST_IO_VGAARI_W (0x40) | 15 | #define AST_IO_VGAARI_W (0x40) |
| 16 | |||
| 15 | #define AST_IO_VGAMR_W (0x42) | 17 | #define AST_IO_VGAMR_W (0x42) |
| 18 | #define AST_IO_VGAMR_R (0x4c) | ||
| 19 | #define AST_IO_VGAMR_IOSEL BIT(0) | ||
| 20 | |||
| 16 | #define AST_IO_VGAER (0x43) | 21 | #define AST_IO_VGAER (0x43) |
| 22 | #define AST_IO_VGAER_VGA_ENABLE BIT(0) | ||
| 23 | |||
| 17 | #define AST_IO_VGASRI (0x44) | 24 | #define AST_IO_VGASRI (0x44) |
| 18 | #define AST_IO_VGADRR (0x47) | 25 | #define AST_IO_VGADRR (0x47) |
| 19 | #define AST_IO_VGADWR (0x48) | 26 | #define AST_IO_VGADWR (0x48) |
| @@ -21,14 +28,15 @@ · 1 unmodified line | |||
| 21 | #define AST_IO_VGAGRI (0x4E) | 28 | #define AST_IO_VGAGRI (0x4E) |
| 22 | 29 | ||
| 23 | #define AST_IO_VGACRI (0x54) | 30 | #define AST_IO_VGACRI (0x54) |
| 31 | #define AST_IO_VGACR80_PASSWORD (0xa8) | ||
| 32 | #define AST_IO_VGACRA1_VGAIO_DISABLED BIT(1) | ||
| 33 | #define AST_IO_VGACRA1_MMIO_ENABLED BIT(2) | ||
| 24 | #define AST_IO_VGACRCB_HWC_16BPP BIT(0) /* set: ARGB4444, cleared: 2bpp palette */ | 34 | #define AST_IO_VGACRCB_HWC_16BPP BIT(0) /* set: ARGB4444, cleared: 2bpp palette */ |
| 25 | #define AST_IO_VGACRCB_HWC_ENABLED BIT(1) | 35 | #define AST_IO_VGACRCB_HWC_ENABLED BIT(1) |
| 26 | 36 | ||
| 27 | #define AST_IO_VGAIR1_R (0x5A) | 37 | #define AST_IO_VGAIR1_R (0x5A) |
| 28 | #define AST_IO_VGAIR1_VREFRESH BIT(3) | 38 | #define AST_IO_VGAIR1_VREFRESH BIT(3) |
| 29 | 39 | ||
| 30 | #define AST_IO_VGAMR_R (0x4C) | ||
| 31 | |||
| 32 | /* | 40 | /* |
| 33 | * Display Transmitter Type | 41 | * Display Transmitter Type |
| 34 | */ | 42 | */ |
| @@ -12,6 +12,23 @@ config DRM_PANEL_BRIDGE · 11 unmodified lines | |||
| 12 | help | 12 | help |
| 13 | DRM bridge wrapper of DRM panels | 13 | DRM bridge wrapper of DRM panels |
| 14 | 14 | ||
| 15 | config DRM_AUX_BRIDGE | ||
| 16 | tristate | ||
| 17 | depends on DRM_BRIDGE && OF | ||
| 18 | select AUXILIARY_BUS | ||
| 19 | select DRM_PANEL_BRIDGE | ||
| 20 | help | ||
| 21 | Simple transparent bridge that is used by several non-DRM drivers to | ||
| 22 | build bridges chain. | ||
| 23 | |||
| 24 | config DRM_AUX_HPD_BRIDGE | ||
| 25 | tristate | ||
| 26 | depends on DRM_BRIDGE && OF | ||
| 27 | select AUXILIARY_BUS | ||
| 28 | help | ||
| 29 | Simple bridge that terminates the bridge chain and provides HPD | ||
| 30 | support. | ||
| 31 | |||
| 15 | menu "Display Interface Bridges" | 32 | menu "Display Interface Bridges" |
| 16 | depends on DRM && DRM_BRIDGE | 33 | depends on DRM && DRM_BRIDGE |
| 17 | 34 | ||
| @@ -1,4 +1,6 @@ | |||
| 1 | # SPDX-License-Identifier: GPL-2.0 | 1 | # SPDX-License-Identifier: GPL-2.0 |
| 2 | obj-$(CONFIG_DRM_AUX_BRIDGE) += aux-bridge.o | ||
| 3 | obj-$(CONFIG_DRM_AUX_HPD_BRIDGE) += aux-hpd-bridge.o | ||
| 2 | obj-$(CONFIG_DRM_CHIPONE_ICN6211) += chipone-icn6211.o | 4 | obj-$(CONFIG_DRM_CHIPONE_ICN6211) += chipone-icn6211.o |
| 3 | obj-$(CONFIG_DRM_CHRONTEL_CH7033) += chrontel-ch7033.o | 5 | obj-$(CONFIG_DRM_CHRONTEL_CH7033) += chrontel-ch7033.o |
| 4 | obj-$(CONFIG_DRM_CROS_EC_ANX7688) += cros-ec-anx7688.o | 6 | obj-$(CONFIG_DRM_CROS_EC_ANX7688) += cros-ec-anx7688.o |
| @@ -1298,10 +1298,32 @@ static void anx7625_config(struct anx7625_data *ctx) · 1297 unmodified lines | |||
| 1298 | XTAL_FRQ_SEL, XTAL_FRQ_27M); | 1298 | XTAL_FRQ_SEL, XTAL_FRQ_27M); |
| 1299 | } | 1299 | } |
| 1300 | 1300 | ||
| 1301 | static int anx7625_hpd_timer_config(struct anx7625_data *ctx) | ||
| 1302 | { | ||
| 1303 | int ret; | ||
| 1304 | |||
| 1305 | /* Set irq detect window to 2ms */ | ||
| 1306 | ret = anx7625_reg_write(ctx, ctx->i2c.tx_p2_client, | ||
| 1307 | HPD_DET_TIMER_BIT0_7, HPD_TIME & 0xFF); | ||
| 1308 | ret |= anx7625_reg_write(ctx, ctx->i2c.tx_p2_client, | ||
| 1309 | HPD_DET_TIMER_BIT8_15, | ||
| 1310 | (HPD_TIME >> 8) & 0xFF); | ||
| 1311 | ret |= anx7625_reg_write(ctx, ctx->i2c.tx_p2_client, | ||
| 1312 | HPD_DET_TIMER_BIT16_23, | ||
| 1313 | (HPD_TIME >> 16) & 0xFF); | ||
| 1314 | |||
| 1315 | return ret; | ||
| 1316 | } | ||
| 1317 | |||
| 1318 | static int anx7625_read_hpd_gpio_config_status(struct anx7625_data *ctx) | ||
| 1319 | { | ||
| 1320 | return anx7625_reg_read(ctx, ctx->i2c.rx_p0_client, GPIO_CTRL_2); | ||
| 1321 | } | ||
| 1322 | |||
| 1301 | static void anx7625_disable_pd_protocol(struct anx7625_data *ctx) | 1323 | static void anx7625_disable_pd_protocol(struct anx7625_data *ctx) |
| 1302 | { | 1324 | { |
| 1303 | struct device *dev = ctx->dev; | 1325 | struct device *dev = ctx->dev; |
| 1304 | int ret; | 1326 | int ret, val; |
| 1305 | 1327 | ||
| 1306 | /* Reset main ocm */ | 1328 | /* Reset main ocm */ |
| 1307 | ret = anx7625_reg_write(ctx, ctx->i2c.rx_p0_client, 0x88, 0x40); | 1329 | ret = anx7625_reg_write(ctx, ctx->i2c.rx_p0_client, 0x88, 0x40); |
| @@ -1315,6 +1337,19 @@ static void anx7625_disable_pd_protocol(struct anx7625_data *ctx) · 7 unmodified lines | |||
| 1315 | DRM_DEV_DEBUG_DRIVER(dev, "disable PD feature fail.\n"); | 1337 | DRM_DEV_DEBUG_DRIVER(dev, "disable PD feature fail.\n"); |
| 1316 | else | 1338 | else |
| 1317 | DRM_DEV_DEBUG_DRIVER(dev, "disable PD feature succeeded.\n"); | 1339 | DRM_DEV_DEBUG_DRIVER(dev, "disable PD feature succeeded.\n"); |
| 1340 | |||
| 1341 | /* | ||
| 1342 | * Make sure the HPD GPIO already be configured after OCM release before | ||
| 1343 | * setting HPD detect window register. Here we poll the status register | ||
| 1344 | * at maximum 40ms, then config HPD irq detect window register | ||
| 1345 | */ | ||
| 1346 | readx_poll_timeout(anx7625_read_hpd_gpio_config_status, | ||
| 1347 | ctx, val, | ||
| 1348 | ((val & HPD_SOURCE) || (val < 0)), | ||
| 1349 | 2000, 2000 * 20); | ||
| 1350 | |||
| 1351 | /* Set HPD irq detect window to 2ms */ | ||
| 1352 | anx7625_hpd_timer_config(ctx); | ||
| 1318 | } | 1353 | } |
| 1319 | 1354 | ||
| 1320 | static int anx7625_ocm_loading_check(struct anx7625_data *ctx) | 1355 | static int anx7625_ocm_loading_check(struct anx7625_data *ctx) |
| @@ -1437,20 +1472,6 @@ static void anx7625_start_dp_work(struct anx7625_data *ctx) · 116 unmodified lines | |||
| 1437 | 1472 | ||
| 1438 | static int anx7625_read_hpd_status_p0(struct anx7625_data *ctx) | 1473 | static int anx7625_read_hpd_status_p0(struct anx7625_data *ctx) |
| 1439 | { | 1474 | { |
| 1440 | int ret; | ||
| 1441 | |||
| 1442 | /* Set irq detect window to 2ms */ | ||
| 1443 | ret = anx7625_reg_write(ctx, ctx->i2c.tx_p2_client, | ||
| 1444 | HPD_DET_TIMER_BIT0_7, HPD_TIME & 0xFF); | ||
| 1445 | ret |= anx7625_reg_write(ctx, ctx->i2c.tx_p2_client, | ||
| 1446 | HPD_DET_TIMER_BIT8_15, | ||
| 1447 | (HPD_TIME >> 8) & 0xFF); | ||
| 1448 | ret |= anx7625_reg_write(ctx, ctx->i2c.tx_p2_client, | ||
| 1449 | HPD_DET_TIMER_BIT16_23, | ||
| 1450 | (HPD_TIME >> 16) & 0xFF); | ||
| 1451 | if (ret < 0) | ||
| 1452 | return ret; | ||
| 1453 | |||
| 1454 | return anx7625_reg_read(ctx, ctx->i2c.rx_p0_client, SYSTEM_STSTUS); | 1475 | return anx7625_reg_read(ctx, ctx->i2c.rx_p0_client, SYSTEM_STSTUS); |
| 1455 | } | 1476 | } |
| 1456 | 1477 | ||
| @@ -1464,9 +1485,6 @@ static int _anx7625_hpd_polling(struct anx7625_data *ctx, · 7 unmodified lines | |||
| 1464 | if (ctx->pdata.intp_irq) | 1485 | if (ctx->pdata.intp_irq) |
| 1465 | return 0; | 1486 | return 0; |
| 1466 | 1487 | ||
| 1467 | /* Delay 200ms for FW HPD de-bounce */ | ||
| 1468 | msleep(200); | ||
| 1469 | |||
| 1470 | ret = readx_poll_timeout(anx7625_read_hpd_status_p0, | 1488 | ret = readx_poll_timeout(anx7625_read_hpd_status_p0, |
| 1471 | ctx, val, | 1489 | ctx, val, |
| 1472 | ((val & HPD_STATUS) || (val < 0)), | 1490 | ((val & HPD_STATUS) || (val < 0)), |
| @@ -259,6 +259,10 @@ · 258 unmodified lines | |||
| 259 | #define AP_MIPI_RX_EN BIT(5) /* 1: MIPI RX input in 0: no RX in */ | 259 | #define AP_MIPI_RX_EN BIT(5) /* 1: MIPI RX input in 0: no RX in */ |
| 260 | #define AP_DISABLE_PD BIT(6) | 260 | #define AP_DISABLE_PD BIT(6) |
| 261 | #define AP_DISABLE_DISPLAY BIT(7) | 261 | #define AP_DISABLE_DISPLAY BIT(7) |
| 262 | |||
| 263 | #define GPIO_CTRL_2 0x49 | ||
| 264 | #define HPD_SOURCE BIT(6) | ||
| 265 | |||
| 262 | /***************************************************************/ | 266 | /***************************************************************/ |
| 263 | /* Register definition of device address 0x84 */ | 267 | /* Register definition of device address 0x84 */ |
| 264 | #define MIPI_PHY_CONTROL_3 0x03 | 268 | #define MIPI_PHY_CONTROL_3 0x03 |
| @@ -0,0 +1,140 @@ | |||
| 1 | // SPDX-License-Identifier: GPL-2.0+ | ||
| 2 | /* | ||
| 3 | * Copyright (C) 2023 Linaro Ltd. | ||
| 4 | * | ||
| 5 | * Author: Dmitry Baryshkov <dmitry.baryshkov@linaro.org> | ||
| 6 | */ | ||
| 7 | #include <linux/auxiliary_bus.h> | ||
| 8 | #include <linux/module.h> | ||
| 9 | |||
| 10 | #include <drm/drm_bridge.h> | ||
| 11 | #include <drm/bridge/aux-bridge.h> | ||
| 12 | |||
| 13 | static DEFINE_IDA(drm_aux_bridge_ida); | ||
| 14 | |||
| 15 | static void drm_aux_bridge_release(struct device *dev) | ||
| 16 | { | ||
| 17 | struct auxiliary_device *adev = to_auxiliary_dev(dev); | ||
| 18 | |||
| 19 | ida_free(&drm_aux_bridge_ida, adev->id); | ||
| 20 | |||
| 21 | kfree(adev); | ||
| 22 | } | ||
| 23 | |||
| 24 | static void drm_aux_bridge_unregister_adev(void *_adev) | ||
| 25 | { | ||
| 26 | struct auxiliary_device *adev = _adev; | ||
| 27 | |||
| 28 | auxiliary_device_delete(adev); | ||
| 29 | auxiliary_device_uninit(adev); | ||
| 30 | } | ||
| 31 | |||
| 32 | /** | ||
| 33 | * drm_aux_bridge_register - Create a simple bridge device to link the chain | ||
| 34 | * @parent: device instance providing this bridge | ||
| 35 | * | ||
| 36 | * Creates a simple DRM bridge that doesn't implement any drm_bridge | ||
| 37 | * operations. Such bridges merely fill a place in the bridge chain linking | ||
| 38 | * surrounding DRM bridges. | ||
| 39 | * | ||
| 40 | * Return: zero on success, negative error code on failure | ||
| 41 | */ | ||
| 42 | int drm_aux_bridge_register(struct device *parent) | ||
| 43 | { | ||
| 44 | struct auxiliary_device *adev; | ||
| 45 | int ret; | ||
| 46 | |||
| 47 | adev = kzalloc(sizeof(*adev), GFP_KERNEL); | ||
| 48 | if (!adev) | ||
| 49 | return -ENOMEM; | ||
| 50 | |||
| 51 | ret = ida_alloc(&drm_aux_bridge_ida, GFP_KERNEL); | ||
| 52 | if (ret < 0) { | ||
| 53 | kfree(adev); | ||
| 54 | return ret; | ||
| 55 | } | ||
| 56 | |||
| 57 | adev->id = ret; | ||
| 58 | adev->name = "aux_bridge"; | ||
| 59 | adev->dev.parent = parent; | ||
| 60 | adev->dev.of_node = parent->of_node; | ||
| 61 | adev->dev.release = drm_aux_bridge_release; | ||
| 62 | |||
| 63 | ret = auxiliary_device_init(adev); | ||
| 64 | if (ret) { | ||
| 65 | ida_free(&drm_aux_bridge_ida, adev->id); | ||
| 66 | kfree(adev); | ||
| 67 | return ret; | ||
| 68 | } | ||
| 69 | |||
| 70 | ret = auxiliary_device_add(adev); | ||
| 71 | if (ret) { | ||
| 72 | auxiliary_device_uninit(adev); | ||
| 73 | return ret; | ||
| 74 | } | ||
| 75 | |||
| 76 | return devm_add_action_or_reset(parent, drm_aux_bridge_unregister_adev, adev); | ||
| 77 | } | ||
| 78 | EXPORT_SYMBOL_GPL(drm_aux_bridge_register); | ||
| 79 | |||
| 80 | struct drm_aux_bridge_data { | ||
| 81 | struct drm_bridge bridge; | ||
| 82 | struct drm_bridge *next_bridge; | ||
| 83 | struct device *dev; | ||
| 84 | }; | ||
| 85 | |||
| 86 | static int drm_aux_bridge_attach(struct drm_bridge *bridge, | ||
| 87 | enum drm_bridge_attach_flags flags) | ||
| 88 | { | ||
| 89 | struct drm_aux_bridge_data *data; | ||
| 90 | |||
| 91 | if (!(flags & DRM_BRIDGE_ATTACH_NO_CONNECTOR)) | ||
| 92 | return -EINVAL; | ||
| 93 | |||
| 94 | data = container_of(bridge, struct drm_aux_bridge_data, bridge); | ||
| 95 | |||
| 96 | return drm_bridge_attach(bridge->encoder, data->next_bridge, bridge, | ||
| 97 | DRM_BRIDGE_ATTACH_NO_CONNECTOR); | ||
| 98 | } | ||
| 99 | |||
| 100 | static const struct drm_bridge_funcs drm_aux_bridge_funcs = { | ||
| 101 | .attach = drm_aux_bridge_attach, | ||
| 102 | }; | ||
| 103 | |||
| 104 | static int drm_aux_bridge_probe(struct auxiliary_device *auxdev, | ||
| 105 | const struct auxiliary_device_id *id) | ||
| 106 | { | ||
| 107 | struct drm_aux_bridge_data *data; | ||
| 108 | |||
| 109 | data = devm_kzalloc(&auxdev->dev, sizeof(*data), GFP_KERNEL); | ||
| 110 | if (!data) | ||
| 111 | return -ENOMEM; | ||
| 112 | |||
| 113 | data->dev = &auxdev->dev; | ||
| 114 | data->next_bridge = devm_drm_of_get_bridge(&auxdev->dev, auxdev->dev.of_node, 0, 0); | ||
| 115 | if (IS_ERR(data->next_bridge)) | ||
| 116 | return dev_err_probe(&auxdev->dev, PTR_ERR(data->next_bridge), | ||
| 117 | "failed to acquire drm_bridge\n"); | ||
| 118 | |||
| 119 | data->bridge.funcs = &drm_aux_bridge_funcs; | ||
| 120 | data->bridge.of_node = data->dev->of_node; | ||
| 121 | |||
| 122 | return devm_drm_bridge_add(data->dev, &data->bridge); | ||
| 123 | } | ||
| 124 | |||
| 125 | static const struct auxiliary_device_id drm_aux_bridge_table[] = { | ||
| 126 | { .name = KBUILD_MODNAME ".aux_bridge" }, | ||
| 127 | {}, | ||
| 128 | }; | ||
| 129 | MODULE_DEVICE_TABLE(auxiliary, drm_aux_bridge_table); | ||
| 130 | |||
| 131 | static struct auxiliary_driver drm_aux_bridge_drv = { | ||
| 132 | .name = "aux_bridge", | ||
| 133 | .id_table = drm_aux_bridge_table, | ||
| 134 | .probe = drm_aux_bridge_probe, | ||
| 135 | }; | ||
| 136 | module_auxiliary_driver(drm_aux_bridge_drv); | ||
| 137 | |||
| 138 | MODULE_AUTHOR("Dmitry Baryshkov <dmitry.baryshkov@linaro.org>"); | ||
| 139 | MODULE_DESCRIPTION("DRM transparent bridge"); | ||
| 140 | MODULE_LICENSE("GPL"); | ||
| @@ -0,0 +1,163 @@ | |||
| 1 | // SPDX-License-Identifier: GPL-2.0+ | ||
| 2 | /* | ||
| 3 | * Copyright (C) 2023 Linaro Ltd. | ||
| 4 | * | ||
| 5 | * Author: Dmitry Baryshkov <dmitry.baryshkov@linaro.org> | ||
| 6 | */ | ||
| 7 | #include <linux/auxiliary_bus.h> | ||
| 8 | #include <linux/module.h> | ||
| 9 | #include <linux/of.h> | ||
| 10 | |||
| 11 | #include <drm/drm_bridge.h> | ||
| 12 | #include <drm/bridge/aux-bridge.h> | ||
| 13 | |||
| 14 | static DEFINE_IDA(drm_aux_hpd_bridge_ida); | ||
| 15 | |||
| 16 | struct drm_aux_hpd_bridge_data { | ||
| 17 | struct drm_bridge bridge; | ||
| 18 | struct device *dev; | ||
| 19 | }; | ||
| 20 | |||
| 21 | static void drm_aux_hpd_bridge_release(struct device *dev) | ||
| 22 | { | ||
| 23 | struct auxiliary_device *adev = to_auxiliary_dev(dev); | ||
| 24 | |||
| 25 | ida_free(&drm_aux_hpd_bridge_ida, adev->id); | ||
| 26 | |||
| 27 | of_node_put(adev->dev.platform_data); | ||
| 28 | |||
| 29 | kfree(adev); | ||
| 30 | } | ||
| 31 | |||
| 32 | static void drm_aux_hpd_bridge_unregister_adev(void *_adev) | ||
| 33 | { | ||
| 34 | struct auxiliary_device *adev = _adev; | ||
| 35 | |||
| 36 | auxiliary_device_delete(adev); | ||
| 37 | auxiliary_device_uninit(adev); | ||
| 38 | } | ||
| 39 | |||
| 40 | /** | ||
| 41 | * drm_dp_hpd_bridge_register - Create a simple HPD DisplayPort bridge | ||
| 42 | * @parent: device instance providing this bridge | ||
| 43 | * @np: device node pointer corresponding to this bridge instance | ||
| 44 | * | ||
| 45 | * Creates a simple DRM bridge with the type set to | ||
| 46 | * DRM_MODE_CONNECTOR_DisplayPort, which terminates the bridge chain and is | ||
| 47 | * able to send the HPD events. | ||
| 48 | * | ||
| 49 | * Return: device instance that will handle created bridge or an error code | ||
| 50 | * encoded into the pointer. | ||
| 51 | */ | ||
| 52 | struct device *drm_dp_hpd_bridge_register(struct device *parent, | ||
| 53 | struct device_node *np) | ||
| 54 | { | ||
| 55 | struct auxiliary_device *adev; | ||
| 56 | int ret; | ||
| 57 | |||
| 58 | adev = kzalloc(sizeof(*adev), GFP_KERNEL); | ||
| 59 | if (!adev) | ||
| 60 | return ERR_PTR(-ENOMEM); | ||
| 61 | |||
| 62 | ret = ida_alloc(&drm_aux_hpd_bridge_ida, GFP_KERNEL); | ||
| 63 | if (ret < 0) { | ||
| 64 | kfree(adev); | ||
| 65 | return ERR_PTR(ret); | ||
| 66 | } | ||
| 67 | |||
| 68 | adev->id = ret; | ||
| 69 | adev->name = "dp_hpd_bridge"; | ||
| 70 | adev->dev.parent = parent; | ||
| 71 | adev->dev.of_node = parent->of_node; | ||
| 72 | adev->dev.release = drm_aux_hpd_bridge_release; | ||
| 73 | adev->dev.platform_data = np; | ||
| 74 | |||
| 75 | ret = auxiliary_device_init(adev); | ||
| 76 | if (ret) { | ||
| 77 | ida_free(&drm_aux_hpd_bridge_ida, adev->id); | ||
| 78 | kfree(adev); | ||
| 79 | return ERR_PTR(ret); | ||
| 80 | } | ||
| 81 | |||
| 82 | ret = auxiliary_device_add(adev); | ||
| 83 | if (ret) { | ||
| 84 | auxiliary_device_uninit(adev); | ||
| 85 | return ERR_PTR(ret); | ||
| 86 | } | ||
| 87 | |||
| 88 | ret = devm_add_action_or_reset(parent, drm_aux_hpd_bridge_unregister_adev, adev); | ||
| 89 | if (ret) | ||
| 90 | return ERR_PTR(ret); | ||
| 91 | |||
| 92 | return &adev->dev; | ||
| 93 | } | ||
| 94 | EXPORT_SYMBOL_GPL(drm_dp_hpd_bridge_register); | ||
| 95 | |||
| 96 | /** | ||
| 97 | * drm_aux_hpd_bridge_notify - notify hot plug detection events | ||
| 98 | * @dev: device created for the HPD bridge | ||
| 99 | * @status: output connection status | ||
| 100 | * | ||
| 101 | * A wrapper around drm_bridge_hpd_notify() that is used to report hot plug | ||
| 102 | * detection events for bridges created via drm_dp_hpd_bridge_register(). | ||
| 103 | * | ||
| 104 | * This function shall be called in a context that can sleep. | ||
| 105 | */ | ||
| 106 | void drm_aux_hpd_bridge_notify(struct device *dev, enum drm_connector_status status) | ||
| 107 | { | ||
| 108 | struct auxiliary_device *adev = to_auxiliary_dev(dev); | ||
| 109 | struct drm_aux_hpd_bridge_data *data = auxiliary_get_drvdata(adev); | ||
| 110 | |||
| 111 | if (!data) | ||
| 112 | return; | ||
| 113 | |||
| 114 | drm_bridge_hpd_notify(&data->bridge, status); | ||
| 115 | } | ||
| 116 | EXPORT_SYMBOL_GPL(drm_aux_hpd_bridge_notify); | ||
| 117 | |||
| 118 | static int drm_aux_hpd_bridge_attach(struct drm_bridge *bridge, | ||
| 119 | enum drm_bridge_attach_flags flags) | ||
| 120 | { | ||
| 121 | return flags & DRM_BRIDGE_ATTACH_NO_CONNECTOR ? 0 : -EINVAL; | ||
| 122 | } | ||
| 123 | |||
| 124 | static const struct drm_bridge_funcs drm_aux_hpd_bridge_funcs = { | ||
| 125 | .attach = drm_aux_hpd_bridge_attach, | ||
| 126 | }; | ||
| 127 | |||
| 128 | static int drm_aux_hpd_bridge_probe(struct auxiliary_device *auxdev, | ||
| 129 | const struct auxiliary_device_id *id) | ||
| 130 | { | ||
| 131 | struct drm_aux_hpd_bridge_data *data; | ||
| 132 | |||
| 133 | data = devm_kzalloc(&auxdev->dev, sizeof(*data), GFP_KERNEL); | ||
| 134 | if (!data) | ||
| 135 | return -ENOMEM; | ||
| 136 | |||
| 137 | data->dev = &auxdev->dev; | ||
| 138 | data->bridge.funcs = &drm_aux_hpd_bridge_funcs; | ||
| 139 | data->bridge.of_node = dev_get_platdata(data->dev); | ||
| 140 | data->bridge.ops = DRM_BRIDGE_OP_HPD; | ||
| 141 | data->bridge.type = id->driver_data; | ||
| 142 | |||
| 143 | auxiliary_set_drvdata(auxdev, data); | ||
| 144 | |||
| 145 | return devm_drm_bridge_add(data->dev, &data->bridge); | ||
| 146 | } | ||
| 147 | |||
| 148 | static const struct auxiliary_device_id drm_aux_hpd_bridge_table[] = { | ||
| 149 | { .name = KBUILD_MODNAME ".dp_hpd_bridge", .driver_data = DRM_MODE_CONNECTOR_DisplayPort, }, | ||
| 150 | {}, | ||
| 151 | }; | ||
| 152 | MODULE_DEVICE_TABLE(auxiliary, drm_aux_hpd_bridge_table); | ||
| 153 | |||
| 154 | static struct auxiliary_driver drm_aux_hpd_bridge_drv = { | ||
| 155 | .name = "aux_hpd_bridge", | ||
| 156 | .id_table = drm_aux_hpd_bridge_table, | ||
| 157 | .probe = drm_aux_hpd_bridge_probe, | ||
| 158 | }; | ||
| 159 | module_auxiliary_driver(drm_aux_hpd_bridge_drv); | ||
| 160 | |||
| 161 | MODULE_AUTHOR("Dmitry Baryshkov <dmitry.baryshkov@linaro.org>"); | ||
| 162 | MODULE_DESCRIPTION("DRM HPD bridge"); | ||
| 163 | MODULE_LICENSE("GPL"); | ||
| @@ -2596,11 +2596,10 @@ clk_disable: · 2595 unmodified lines | |||
| 2596 | return ret; | 2596 | return ret; |
| 2597 | } | 2597 | } |
| 2598 | 2598 | ||
| 2599 | static int cdns_mhdp_remove(struct platform_device *pdev) | 2599 | static void cdns_mhdp_remove(struct platform_device *pdev) |
| 2600 | { | 2600 | { |
| 2601 | struct cdns_mhdp_device *mhdp = platform_get_drvdata(pdev); | 2601 | struct cdns_mhdp_device *mhdp = platform_get_drvdata(pdev); |
| 2602 | unsigned long timeout = msecs_to_jiffies(100); | 2602 | unsigned long timeout = msecs_to_jiffies(100); |
| 2603 | bool stop_fw = false; | ||
| 2604 | int ret; | 2603 | int ret; |
| 2605 | 2604 | ||
| 2606 | drm_bridge_remove(&mhdp->bridge); | 2605 | drm_bridge_remove(&mhdp->bridge); |
| @@ -2608,18 +2607,19 @@ static int cdns_mhdp_remove(struct platform_device *pdev) · 1 unmodified line | |||
| 2608 | ret = wait_event_timeout(mhdp->fw_load_wq, | 2607 | ret = wait_event_timeout(mhdp->fw_load_wq, |
| 2609 | mhdp->hw_state == MHDP_HW_READY, | 2608 | mhdp->hw_state == MHDP_HW_READY, |
| 2610 | timeout); | 2609 | timeout); |
| 2611 | if (ret == 0) | ||
| 2612 | dev_err(mhdp->dev, "%s: Timeout waiting for fw loading\n", | ||
| 2613 | __func__); | ||
| 2614 | else | ||
| 2615 | stop_fw = true; | ||
| 2616 | |||
| 2617 | spin_lock(&mhdp->start_lock); | 2610 | spin_lock(&mhdp->start_lock); |
| 2618 | mhdp->hw_state = MHDP_HW_STOPPED; | 2611 | mhdp->hw_state = MHDP_HW_STOPPED; |
| 2619 | spin_unlock(&mhdp->start_lock); | 2612 | spin_unlock(&mhdp->start_lock); |
| 2620 | 2613 | ||
| 2621 | if (stop_fw) | 2614 | if (ret == 0) { |
| 2615 | dev_err(mhdp->dev, "%s: Timeout waiting for fw loading\n", | ||
| 2616 | __func__); | ||
| 2617 | } else { | ||
| 2622 | ret = cdns_mhdp_set_firmware_active(mhdp, false); | 2618 | ret = cdns_mhdp_set_firmware_active(mhdp, false); |
| 2619 | if (ret) | ||
| 2620 | dev_err(mhdp->dev, "Failed to stop firmware (%pe)\n", | ||
| 2621 | ERR_PTR(ret)); | ||
| 2622 | } | ||
| 2623 | 2623 | ||
| 2624 | phy_exit(mhdp->phy); | 2624 | phy_exit(mhdp->phy); |
| 2625 | 2625 | ||
| @@ -2634,8 +2634,6 @@ static int cdns_mhdp_remove(struct platform_device *pdev) · 8 unmodified lines | |||
| 2634 | /* Ignoring mhdp->hdcp.check_work and mhdp->hdcp.prop_work here. */ | 2634 | /* Ignoring mhdp->hdcp.check_work and mhdp->hdcp.prop_work here. */ |
| 2635 | 2635 | ||
| 2636 | clk_disable_unprepare(mhdp->clk); | 2636 | clk_disable_unprepare(mhdp->clk); |
| 2637 | |||
| 2638 | return ret; | ||
| 2639 | } | 2637 | } |
| 2640 | 2638 | ||
| 2641 | static const struct of_device_id mhdp_ids[] = { | 2639 | static const struct of_device_id mhdp_ids[] = { |
| @@ -2658,7 +2656,7 @@ static struct platform_driver mhdp_driver = { · 16 unmodified lines | |||
| 2658 | .of_match_table = mhdp_ids, | 2656 | .of_match_table = mhdp_ids, |
| 2659 | }, | 2657 | }, |
| 2660 | .probe = cdns_mhdp_probe, | 2658 | .probe = cdns_mhdp_probe, |
| 2661 | .remove = cdns_mhdp_remove, | 2659 | .remove_new = cdns_mhdp_remove, |
| 2662 | }; | 2660 | }; |
| 2663 | module_platform_driver(mhdp_driver); | 2661 | module_platform_driver(mhdp_driver); |
| 2664 | 2662 | ||
| @@ -403,7 +403,8 @@ static int _cdns_mhdp_hdcp_disable(struct cdns_mhdp_device *mhdp) · 402 unmodified lines | |||
| 403 | 403 | ||
| 404 | static int _cdns_mhdp_hdcp_enable(struct cdns_mhdp_device *mhdp, u8 content_type) | 404 | static int _cdns_mhdp_hdcp_enable(struct cdns_mhdp_device *mhdp, u8 content_type) |
| 405 | { | 405 | { |
| 406 | int ret, tries = 3; | 406 | int ret = -EINVAL; |
| 407 | int tries = 3; | ||
| 407 | u32 i; | 408 | u32 i; |
| 408 | 409 | ||
| 409 | for (i = 0; i < tries; i++) { | 410 | for (i = 0; i < tries; i++) { |
| @@ -226,8 +226,8 @@ dphy_pll_get_configure_from_opts(struct imx93_dsi *dsi, · 225 unmodified lines | |||
| 226 | unsigned long fout; | 226 | unsigned long fout; |
| 227 | unsigned long best_fout = 0; | 227 | unsigned long best_fout = 0; |
| 228 | unsigned int fvco_div; | 228 | unsigned int fvco_div; |
| 229 | unsigned int min_n, max_n, n, best_n; | 229 | unsigned int min_n, max_n, n, best_n = UINT_MAX; |
| 230 | unsigned long m, best_m; | 230 | unsigned long m, best_m = 0; |
| 231 | unsigned long min_delta = ULONG_MAX; | 231 | unsigned long min_delta = ULONG_MAX; |
| 232 | unsigned long delta; | 232 | unsigned long delta; |
| 233 | u64 tmp; | 233 | u64 tmp; |
| @@ -43,6 +43,8 @@ struct lt8912 { · 42 unmodified lines | |||
| 43 | 43 | ||
| 44 | struct videomode mode; | 44 | struct videomode mode; |
| 45 | 45 | ||
| 46 | struct regulator_bulk_data supplies[7]; | ||
| 47 | |||
| 46 | u8 data_lanes; | 48 | u8 data_lanes; |
| 47 | bool is_power_on; | 49 | bool is_power_on; |
| 48 | }; | 50 | }; |
| @@ -257,6 +259,12 @@ static int lt8912_free_i2c(struct lt8912 *lt) · 208 unmodified lines | |||
| 257 | 259 | ||
| 258 | static int lt8912_hard_power_on(struct lt8912 *lt) | 260 | static int lt8912_hard_power_on(struct lt8912 *lt) |
| 259 | { | 261 | { |
| 262 | int ret; | ||
| 263 | |||
| 264 | ret = regulator_bulk_enable(ARRAY_SIZE(lt->supplies), lt->supplies); | ||
| 265 | if (ret) | ||
| 266 | return ret; | ||
| 267 | |||
| 260 | gpiod_set_value_cansleep(lt->gp_reset, 0); | 268 | gpiod_set_value_cansleep(lt->gp_reset, 0); |
| 261 | msleep(20); | 269 | msleep(20); |
| 262 | 270 | ||
| @@ -267,6 +275,9 @@ static void lt8912_hard_power_off(struct lt8912 *lt) · 4 unmodified lines | |||
| 267 | { | 275 | { |
| 268 | gpiod_set_value_cansleep(lt->gp_reset, 1); | 276 | gpiod_set_value_cansleep(lt->gp_reset, 1); |
| 269 | msleep(20); | 277 | msleep(20); |
| 278 | |||
| 279 | regulator_bulk_disable(ARRAY_SIZE(lt->supplies), lt->supplies); | ||
| 280 | |||
| 270 | lt->is_power_on = false; | 281 | lt->is_power_on = false; |
| 271 | } | 282 | } |
| 272 | 283 | ||
| @@ -634,6 +645,48 @@ static const struct drm_bridge_funcs lt8912_bridge_funcs = { · 361 unmodified lines | |||
| 634 | .get_edid = lt8912_bridge_get_edid, | 645 | .get_edid = lt8912_bridge_get_edid, |
| 635 | }; | 646 | }; |
| 636 | 647 | ||
| 648 | static int lt8912_bridge_resume(struct device *dev) | ||
| 649 | { | ||
| 650 | struct lt8912 *lt = dev_get_drvdata(dev); | ||
| 651 | int ret; | ||
| 652 | |||
| 653 | ret = lt8912_hard_power_on(lt); | ||
| 654 | if (ret) | ||
| 655 | return ret; | ||
| 656 | |||
| 657 | ret = lt8912_soft_power_on(lt); | ||
| 658 | if (ret) | ||
| 659 | return ret; | ||
| 660 | |||
| 661 | return lt8912_video_on(lt); | ||
| 662 | } | ||
| 663 | |||
| 664 | static int lt8912_bridge_suspend(struct device *dev) | ||
| 665 | { | ||
| 666 | struct lt8912 *lt = dev_get_drvdata(dev); | ||
| 667 | |||
| 668 | lt8912_hard_power_off(lt); | ||
| 669 | |||
| 670 | return 0; | ||
| 671 | } | ||
| 672 | |||
| 673 | static DEFINE_SIMPLE_DEV_PM_OPS(lt8912_bridge_pm_ops, lt8912_bridge_suspend, lt8912_bridge_resume); | ||
| 674 | |||
| 675 | static int lt8912_get_regulators(struct lt8912 *lt) | ||
| 676 | { | ||
| 677 | unsigned int i; | ||
| 678 | const char * const supply_names[] = { | ||
| 679 | "vdd", "vccmipirx", "vccsysclk", "vcclvdstx", | ||
| 680 | "vcchdmitx", "vcclvdspll", "vcchdmipll" | ||
| 681 | }; | ||
| 682 | |||
| 683 | for (i = 0; i < ARRAY_SIZE(lt->supplies); i++) | ||
| 684 | lt->supplies[i].supply = supply_names[i]; | ||
| 685 | |||
| 686 | return devm_regulator_bulk_get(lt->dev, ARRAY_SIZE(lt->supplies), | ||
| 687 | lt->supplies); | ||
| 688 | } | ||
| 689 | |||
| 637 | static int lt8912_parse_dt(struct lt8912 *lt) | 690 | static int lt8912_parse_dt(struct lt8912 *lt) |
| 638 | { | 691 | { |
| 639 | struct gpio_desc *gp_reset; | 692 | struct gpio_desc *gp_reset; |
| @@ -685,6 +738,10 @@ static int lt8912_parse_dt(struct lt8912 *lt) · 45 unmodified lines | |||
| 685 | goto err_free_host_node; | 738 | goto err_free_host_node; |
| 686 | } | 739 | } |
| 687 | 740 | ||
| 741 | ret = lt8912_get_regulators(lt); | ||
| 742 | if (ret) | ||
| 743 | goto err_free_host_node; | ||
| 744 | |||
| 688 | of_node_put(port_node); | 745 | of_node_put(port_node); |
| 689 | return 0; | 746 | return 0; |
| 690 | 747 | ||
| @@ -770,6 +827,7 @@ static struct i2c_driver lt8912_i2c_driver = { · 79 unmodified lines | |||
| 770 | .driver = { | 827 | .driver = { |
| 771 | .name = "lt8912", | 828 | .name = "lt8912", |
| 772 | .of_match_table = lt8912_dt_match, | 829 | .of_match_table = lt8912_dt_match, |
| 830 | .pm = pm_sleep_ptr(<8912_bridge_pm_ops), | ||
| 773 | }, | 831 | }, |
| 774 | .probe = lt8912_probe, | 832 | .probe = lt8912_probe, |
| 775 | .remove = lt8912_remove, | 833 | .remove = lt8912_remove, |
| @@ -54,13 +54,13 @@ static int ptn3460_read_bytes(struct ptn3460_bridge *ptn_bridge, char addr, · 53 unmodified lines | |||
| 54 | int ret; | 54 | int ret; |
| 55 | 55 | ||
| 56 | ret = i2c_master_send(ptn_bridge->client, &addr, 1); | 56 | ret = i2c_master_send(ptn_bridge->client, &addr, 1); |
| 57 | if (ret <= 0) { | 57 | if (ret < 0) { |
| 58 | DRM_ERROR("Failed to send i2c command, ret=%d\n", ret); | 58 | DRM_ERROR("Failed to send i2c command, ret=%d\n", ret); |
| 59 | return ret; | 59 | return ret; |
| 60 | } | 60 | } |
| 61 | 61 | ||
| 62 | ret = i2c_master_recv(ptn_bridge->client, buf, len); | 62 | ret = i2c_master_recv(ptn_bridge->client, buf, len); |
| 63 | if (ret <= 0) { | 63 | if (ret < 0) { |
| 64 | DRM_ERROR("Failed to recv i2c data, ret=%d\n", ret); | 64 | DRM_ERROR("Failed to recv i2c data, ret=%d\n", ret); |
| 65 | return ret; | 65 | return ret; |
| 66 | } | 66 | } |
| @@ -78,7 +78,7 @@ static int ptn3460_write_byte(struct ptn3460_bridge *ptn_bridge, char addr, · 11 unmodified lines | |||
| 78 | buf[1] = val; | 78 | buf[1] = val; |
| 79 | 79 | ||
| 80 | ret = i2c_master_send(ptn_bridge->client, buf, ARRAY_SIZE(buf)); | 80 | ret = i2c_master_send(ptn_bridge->client, buf, ARRAY_SIZE(buf)); |
| 81 | if (ret <= 0) { | 81 | if (ret < 0) { |
| 82 | DRM_ERROR("Failed to send i2c command, ret=%d\n", ret); | 82 | DRM_ERROR("Failed to send i2c command, ret=%d\n", ret); |
| 83 | return ret; | 83 | return ret; |
| 84 | } | 84 | } |
| @@ -2273,7 +2273,7 @@ static int tc_probe(struct i2c_client *client) · 2272 unmodified lines | |||
| 2273 | } else { | 2273 | } else { |
| 2274 | if (tc->hpd_pin < 0 || tc->hpd_pin > 1) { | 2274 | if (tc->hpd_pin < 0 || tc->hpd_pin > 1) { |
| 2275 | dev_err(dev, "failed to parse HPD number\n"); | 2275 | dev_err(dev, "failed to parse HPD number\n"); |
| 2276 | return ret; | 2276 | return -EINVAL; |
| 2277 | } | 2277 | } |
| 2278 | } | 2278 | } |
| 2279 | 2279 | ||
| @@ -1413,11 +1413,9 @@ static int ti_sn_pwm_apply(struct pwm_chip *chip, struct pwm_device *pwm, · 1412 unmodified lines | |||
| 1413 | int ret; | 1413 | int ret; |
| 1414 | 1414 | ||
| 1415 | if (!pdata->pwm_enabled) { | 1415 | if (!pdata->pwm_enabled) { |
| 1416 | ret = pm_runtime_get_sync(pdata->dev); | 1416 | ret = pm_runtime_resume_and_get(pdata->dev); |
| 1417 | if (ret < 0) { | 1417 | if (ret < 0) |
| 1418 | pm_runtime_put_sync(pdata->dev); | ||
| 1419 | return ret; | 1418 | return ret; |
| 1420 | } | ||
| 1421 | } | 1419 | } |
| 1422 | 1420 | ||
| 1423 | if (state->enabled) { | 1421 | if (state->enabled) { |
| @@ -179,13 +179,11 @@ static int tpd12s015_probe(struct platform_device *pdev) · 178 unmodified lines | |||
| 179 | return 0; | 179 | return 0; |
| 180 | } | 180 | } |
| 181 | 181 | ||
| 182 | static int __exit tpd12s015_remove(struct platform_device *pdev) | 182 | static void tpd12s015_remove(struct platform_device *pdev) |
| 183 | { | 183 | { |
| 184 | struct tpd12s015_device *tpd = platform_get_drvdata(pdev); | 184 | struct tpd12s015_device *tpd = platform_get_drvdata(pdev); |
| 185 | 185 | ||
| 186 | drm_bridge_remove(&tpd->bridge); | 186 | drm_bridge_remove(&tpd->bridge); |
| 187 | |||
| 188 | return 0; | ||
| 189 | } | 187 | } |
| 190 | 188 | ||
| 191 | static const struct of_device_id tpd12s015_of_match[] = { | 189 | static const struct of_device_id tpd12s015_of_match[] = { |
| @@ -197,7 +195,7 @@ MODULE_DEVICE_TABLE(of, tpd12s015_of_match); · 5 unmodified lines | |||
| 197 | 195 | ||
| 198 | static struct platform_driver tpd12s015_driver = { | 196 | static struct platform_driver tpd12s015_driver = { |
| 199 | .probe = tpd12s015_probe, | 197 | .probe = tpd12s015_probe, |
| 200 | .remove = __exit_p(tpd12s015_remove), | 198 | .remove_new = tpd12s015_remove, |
| 201 | .driver = { | 199 | .driver = { |
| 202 | .name = "tpd12s015", | 200 | .name = "tpd12s015", |
| 203 | .of_match_table = tpd12s015_of_match, | 201 | .of_match_table = tpd12s015_of_match, |
| @@ -5,7 +5,7 @@ termcolor==2.3.0 · 4 unmodified lines | |||
| 5 | certifi==2023.7.22 | 5 | certifi==2023.7.22 |
| 6 | charset-normalizer==3.2.0 | 6 | charset-normalizer==3.2.0 |
| 7 | idna==3.4 | 7 | idna==3.4 |
| 8 | pip==23.2.1 | 8 | pip==23.3 |
| 9 | python-gitlab==3.15.0 | 9 | python-gitlab==3.15.0 |
| 10 | requests==2.31.0 | 10 | requests==2.31.0 |
| 11 | requests-toolbelt==1.0.0 | 11 | requests-toolbelt==1.0.0 |
| @@ -13,5 +13,5 @@ ruamel.yaml==0.17.32 · 1 unmodified line | |||
| 13 | ruamel.yaml.clib==0.2.7 | 13 | ruamel.yaml.clib==0.2.7 |
| 14 | setuptools==68.0.0 | 14 | setuptools==68.0.0 |
| 15 | tenacity==8.2.3 | 15 | tenacity==8.2.3 |
| 16 | urllib3==2.0.4 | 16 | urllib3==2.0.7 |
| 17 | wheel==0.41.1 | 17 | wheel==0.41.1 |
| @@ -2245,6 +2245,8 @@ static const struct dpcd_quirk dpcd_quirk_list[] = { · 2244 unmodified lines | |||
| 2245 | { OUI(0x00, 0x00, 0x00), DEVICE_ID('C', 'H', '7', '5', '1', '1'), false, BIT(DP_DPCD_QUIRK_NO_SINK_COUNT) }, | 2245 | { OUI(0x00, 0x00, 0x00), DEVICE_ID('C', 'H', '7', '5', '1', '1'), false, BIT(DP_DPCD_QUIRK_NO_SINK_COUNT) }, |
| 2246 | /* Synaptics DP1.4 MST hubs can support DSC without virtual DPCD */ | 2246 | /* Synaptics DP1.4 MST hubs can support DSC without virtual DPCD */ |
| 2247 | { OUI(0x90, 0xCC, 0x24), DEVICE_ID_ANY, true, BIT(DP_DPCD_QUIRK_DSC_WITHOUT_VIRTUAL_DPCD) }, | 2247 | { OUI(0x90, 0xCC, 0x24), DEVICE_ID_ANY, true, BIT(DP_DPCD_QUIRK_DSC_WITHOUT_VIRTUAL_DPCD) }, |
| 2248 | /* Synaptics DP1.4 MST hubs require DSC for some modes on which it applies HBLANK expansion. */ | ||
| 2249 | { OUI(0x90, 0xCC, 0x24), DEVICE_ID_ANY, true, BIT(DP_DPCD_QUIRK_HBLANK_EXPANSION_REQUIRES_DSC) }, | ||
| 2248 | /* Apple MacBookPro 2017 15 inch eDP Retina panel reports too low DP_MAX_LINK_RATE */ | 2250 | /* Apple MacBookPro 2017 15 inch eDP Retina panel reports too low DP_MAX_LINK_RATE */ |
| 2249 | { OUI(0x00, 0x10, 0xfa), DEVICE_ID(101, 68, 21, 101, 98, 97), false, BIT(DP_DPCD_QUIRK_CAN_DO_MAX_LINK_RATE_3_24_GBPS) }, | 2251 | { OUI(0x00, 0x10, 0xfa), DEVICE_ID(101, 68, 21, 101, 98, 97), false, BIT(DP_DPCD_QUIRK_CAN_DO_MAX_LINK_RATE_3_24_GBPS) }, |
| 2250 | }; | 2252 | }; |
| @@ -2326,6 +2328,33 @@ int drm_dp_read_desc(struct drm_dp_aux *aux, struct drm_dp_desc *desc, · 75 unmodified lines | |||
| 2326 | } | 2328 | } |
| 2327 | EXPORT_SYMBOL(drm_dp_read_desc); | 2329 | EXPORT_SYMBOL(drm_dp_read_desc); |
| 2328 | 2330 | ||
| 2331 | /** | ||
| 2332 | * drm_dp_dsc_sink_bpp_incr() - Get bits per pixel increment | ||
| 2333 | * @dsc_dpcd: DSC capabilities from DPCD | ||
| 2334 | * | ||
| 2335 | * Returns the bpp precision supported by the DP sink. | ||
| 2336 | */ | ||
| 2337 | u8 drm_dp_dsc_sink_bpp_incr(const u8 dsc_dpcd[DP_DSC_RECEIVER_CAP_SIZE]) | ||
| 2338 | { | ||
| 2339 | u8 bpp_increment_dpcd = dsc_dpcd[DP_DSC_BITS_PER_PIXEL_INC - DP_DSC_SUPPORT]; | ||
| 2340 | |||
| 2341 | switch (bpp_increment_dpcd) { | ||
| 2342 | case DP_DSC_BITS_PER_PIXEL_1_16: | ||
| 2343 | return 16; | ||
| 2344 | case DP_DSC_BITS_PER_PIXEL_1_8: | ||
| 2345 | return 8; | ||
| 2346 | case DP_DSC_BITS_PER_PIXEL_1_4: | ||
| 2347 | return 4; | ||
| 2348 | case DP_DSC_BITS_PER_PIXEL_1_2: | ||
| 2349 | return 2; | ||
| 2350 | case DP_DSC_BITS_PER_PIXEL_1_1: | ||
| 2351 | return 1; | ||
| 2352 | } | ||
| 2353 | |||
| 2354 | return 0; | ||
| 2355 | } | ||
| 2356 | EXPORT_SYMBOL(drm_dp_dsc_sink_bpp_incr); | ||
| 2357 | |||
| 2329 | /** | 2358 | /** |
| 2330 | * drm_dp_dsc_sink_max_slice_count() - Get the max slice count | 2359 | * drm_dp_dsc_sink_max_slice_count() - Get the max slice count |
| 2331 | * supported by the DSC sink. | 2360 | * supported by the DSC sink. |
| @@ -3898,3 +3927,135 @@ int drm_panel_dp_aux_backlight(struct drm_panel *panel, struct drm_dp_aux *aux) · 1566 unmodified lines | |||
| 3898 | EXPORT_SYMBOL(drm_panel_dp_aux_backlight); | 3927 | EXPORT_SYMBOL(drm_panel_dp_aux_backlight); |
| 3899 | 3928 | ||
| 3900 | #endif | 3929 | #endif |
| 3930 | |||
| 3931 | /* See DP Standard v2.1 2.6.4.4.1.1, 2.8.4.4, 2.8.7 */ | ||
| 3932 | static int drm_dp_link_symbol_cycles(int lane_count, int pixels, int bpp_x16, | ||
| 3933 | int symbol_size, bool is_mst) | ||
| 3934 | { | ||
| 3935 | int cycles = DIV_ROUND_UP(pixels * bpp_x16, 16 * symbol_size * lane_count); | ||
| 3936 | int align = is_mst ? 4 / lane_count : 1; | ||
| 3937 | |||
| 3938 | return ALIGN(cycles, align); | ||
| 3939 | } | ||
| 3940 | |||
| 3941 | static int drm_dp_link_dsc_symbol_cycles(int lane_count, int pixels, int slice_count, | ||
| 3942 | int bpp_x16, int symbol_size, bool is_mst) | ||
| 3943 | { | ||
| 3944 | int slice_pixels = DIV_ROUND_UP(pixels, slice_count); | ||
| 3945 | int slice_data_cycles = drm_dp_link_symbol_cycles(lane_count, slice_pixels, | ||
| 3946 | bpp_x16, symbol_size, is_mst); | ||
| 3947 | int slice_eoc_cycles = is_mst ? 4 / lane_count : 1; | ||
| 3948 | |||
| 3949 | return slice_count * (slice_data_cycles + slice_eoc_cycles); | ||
| 3950 | } | ||
| 3951 | |||
| 3952 | /** | ||
| 3953 | * drm_dp_bw_overhead - Calculate the BW overhead of a DP link stream | ||
| 3954 | * @lane_count: DP link lane count | ||
| 3955 | * @hactive: pixel count of the active period in one scanline of the stream | ||
| 3956 | * @dsc_slice_count: DSC slice count if @flags/DRM_DP_LINK_BW_OVERHEAD_DSC is set | ||
| 3957 | * @bpp_x16: bits per pixel in .4 binary fixed point | ||
| 3958 | * @flags: DRM_DP_OVERHEAD_x flags | ||
| 3959 | * | ||
| 3960 | * Calculate the BW allocation overhead of a DP link stream, depending | ||
| 3961 | * on the link's | ||
| 3962 | * - @lane_count | ||
| 3963 | * - SST/MST mode (@flags / %DRM_DP_OVERHEAD_MST) | ||
| 3964 | * - symbol size (@flags / %DRM_DP_OVERHEAD_UHBR) | ||
| 3965 | * - FEC mode (@flags / %DRM_DP_OVERHEAD_FEC) | ||
| 3966 | * - SSC/REF_CLK mode (@flags / %DRM_DP_OVERHEAD_SSC_REF_CLK) | ||
| 3967 | * as well as the stream's | ||
| 3968 | * - @hactive timing | ||
| 3969 | * - @bpp_x16 color depth | ||
| 3970 | * - compression mode (@flags / %DRM_DP_OVERHEAD_DSC). | ||
| 3971 | * Note that this overhead doesn't account for the 8b/10b, 128b/132b | ||
| 3972 | * channel coding efficiency, for that see | ||
| 3973 | * @drm_dp_link_bw_channel_coding_efficiency(). | ||
| 3974 | * | ||
| 3975 | * Returns the overhead as 100% + overhead% in 1ppm units. | ||
| 3976 | */ | ||
| 3977 | int drm_dp_bw_overhead(int lane_count, int hactive, | ||
| 3978 | int dsc_slice_count, | ||
| 3979 | int bpp_x16, unsigned long flags) | ||
| 3980 | { | ||
| 3981 | int symbol_size = flags & DRM_DP_BW_OVERHEAD_UHBR ? 32 : 8; | ||
| 3982 | bool is_mst = flags & DRM_DP_BW_OVERHEAD_MST; | ||
| 3983 | u32 overhead = 1000000; | ||
| 3984 | int symbol_cycles; | ||
| 3985 | |||
| 3986 | /* | ||
| 3987 | * DP Standard v2.1 2.6.4.1 | ||
| 3988 | * SSC downspread and ref clock variation margin: | ||
| 3989 | * 5300ppm + 300ppm ~ 0.6% | ||
| 3990 | */ | ||
| 3991 | if (flags & DRM_DP_BW_OVERHEAD_SSC_REF_CLK) | ||
| 3992 | overhead += 6000; | ||
| 3993 | |||
| 3994 | /* | ||
| 3995 | * DP Standard v2.1 2.6.4.1.1, 3.5.1.5.4: | ||
| 3996 | * FEC symbol insertions for 8b/10b channel coding: | ||
| 3997 | * After each 250 data symbols on 2-4 lanes: | ||
| 3998 | * 250 LL + 5 FEC_PARITY_PH + 1 CD_ADJ (256 byte FEC block) | ||
| 3999 | * After each 2 x 250 data symbols on 1 lane: | ||
| 4000 | * 2 * 250 LL + 11 FEC_PARITY_PH + 1 CD_ADJ (512 byte FEC block) | ||
| 4001 | * After 256 (2-4 lanes) or 128 (1 lane) FEC blocks: | ||
| 4002 | * 256 * 256 bytes + 1 FEC_PM | ||
| 4003 | * or | ||
| 4004 | * 128 * 512 bytes + 1 FEC_PM | ||
| 4005 | * (256 * 6 + 1) / (256 * 250) = 2.4015625 % | ||
| 4006 | */ | ||
| 4007 | if (flags & DRM_DP_BW_OVERHEAD_FEC) | ||
| 4008 | overhead += 24016; | ||
| 4009 | |||
| 4010 | /* | ||
| 4011 | * DP Standard v2.1 2.7.9, 5.9.7 | ||
| 4012 | * The FEC overhead for UHBR is accounted for in its 96.71% channel | ||
| 4013 | * coding efficiency. | ||
| 4014 | */ | ||
| 4015 | WARN_ON((flags & DRM_DP_BW_OVERHEAD_UHBR) && | ||
| 4016 | (flags & DRM_DP_BW_OVERHEAD_FEC)); | ||
| 4017 | |||
| 4018 | if (flags & DRM_DP_BW_OVERHEAD_DSC) | ||
| 4019 | symbol_cycles = drm_dp_link_dsc_symbol_cycles(lane_count, hactive, | ||
| 4020 | dsc_slice_count, | ||
| 4021 | bpp_x16, symbol_size, | ||
| 4022 | is_mst); | ||
| 4023 | else | ||
| 4024 | symbol_cycles = drm_dp_link_symbol_cycles(lane_count, hactive, | ||
| 4025 | bpp_x16, symbol_size, | ||
| 4026 | is_mst); | ||
| 4027 | |||
| 4028 | return DIV_ROUND_UP_ULL(mul_u32_u32(symbol_cycles * symbol_size * lane_count, | ||
| 4029 | overhead * 16), | ||
| 4030 | hactive * bpp_x16); | ||
| 4031 | } | ||
| 4032 | EXPORT_SYMBOL(drm_dp_bw_overhead); | ||
| 4033 | |||
| 4034 | /** | ||
| 4035 | * drm_dp_bw_channel_coding_efficiency - Get a DP link's channel coding efficiency | ||
| 4036 | * @is_uhbr: Whether the link has a 128b/132b channel coding | ||
| 4037 | * | ||
| 4038 | * Return the channel coding efficiency of the given DP link type, which is | ||
| 4039 | * either 8b/10b or 128b/132b (aka UHBR). The corresponding overhead includes | ||
| 4040 | * the 8b -> 10b, 128b -> 132b pixel data to link symbol conversion overhead | ||
| 4041 | * and for 128b/132b any link or PHY level control symbol insertion overhead | ||
| 4042 | * (LLCP, FEC, PHY sync, see DP Standard v2.1 3.5.2.18). For 8b/10b the | ||
| 4043 | * corresponding FEC overhead is BW allocation specific, included in the value | ||
| 4044 | * returned by drm_dp_bw_overhead(). | ||
| 4045 | * | ||
| 4046 | * Returns the efficiency in the 100%/coding-overhead% ratio in | ||
| 4047 | * 1ppm units. | ||
| 4048 | */ | ||
| 4049 | int drm_dp_bw_channel_coding_efficiency(bool is_uhbr) | ||
| 4050 | { | ||
| 4051 | if (is_uhbr) | ||
| 4052 | return 967100; | ||
| 4053 | else | ||
| 4054 | /* | ||
| 4055 | * Note that on 8b/10b MST the efficiency is only | ||
| 4056 | * 78.75% due to the 1 out of 64 MTPH packet overhead, | ||
| 4057 | * not accounted for here. | ||
| 4058 | */ | ||
| 4059 | return 800000; | ||
| 4060 | } | ||
| 4061 | EXPORT_SYMBOL(drm_dp_bw_channel_coding_efficiency); | ||
| @@ -43,6 +43,7 @@ · 42 unmodified lines | |||
| 43 | #include <drm/drm_atomic_helper.h> | 43 | #include <drm/drm_atomic_helper.h> |
| 44 | #include <drm/drm_drv.h> | 44 | #include <drm/drm_drv.h> |
| 45 | #include <drm/drm_edid.h> | 45 | #include <drm/drm_edid.h> |
| 46 | #include <drm/drm_fixed.h> | ||
| 46 | #include <drm/drm_print.h> | 47 | #include <drm/drm_print.h> |
| 47 | #include <drm/drm_probe_helper.h> | 48 | #include <drm/drm_probe_helper.h> |
| 48 | 49 | ||
| @@ -3578,16 +3579,26 @@ static int drm_dp_send_up_ack_reply(struct drm_dp_mst_topology_mgr *mgr, · 3529 unmodified lines | |||
| 3578 | * value is in units of PBNs/(timeslots/1 MTP). This value can be used to | 3579 | * value is in units of PBNs/(timeslots/1 MTP). This value can be used to |
| 3579 | * convert the number of PBNs required for a given stream to the number of | 3580 | * convert the number of PBNs required for a given stream to the number of |
| 3580 | * timeslots this stream requires in each MTP. | 3581 | * timeslots this stream requires in each MTP. |
| 3582 | * | ||
| 3583 | * Returns the BW / timeslot value in 20.12 fixed point format. | ||
| 3581 | */ | 3584 | */ |
| 3582 | int drm_dp_get_vc_payload_bw(const struct drm_dp_mst_topology_mgr *mgr, | 3585 | fixed20_12 drm_dp_get_vc_payload_bw(const struct drm_dp_mst_topology_mgr *mgr, |
| 3583 | int link_rate, int link_lane_count) | 3586 | int link_rate, int link_lane_count) |
| 3584 | { | 3587 | { |
| 3588 | int ch_coding_efficiency = | ||
| 3589 | drm_dp_bw_channel_coding_efficiency(drm_dp_is_uhbr_rate(link_rate)); | ||
| 3590 | fixed20_12 ret; | ||
| 3591 | |||
| 3585 | if (link_rate == 0 || link_lane_count == 0) | 3592 | if (link_rate == 0 || link_lane_count == 0) |
| 3586 | drm_dbg_kms(mgr->dev, "invalid link rate/lane count: (%d / %d)\n", | 3593 | drm_dbg_kms(mgr->dev, "invalid link rate/lane count: (%d / %d)\n", |
| 3587 | link_rate, link_lane_count); | 3594 | link_rate, link_lane_count); |
| 3588 | 3595 | ||
| 3589 | /* See DP v2.0 2.6.4.2, VCPayload_Bandwidth_for_OneTimeSlotPer_MTP_Allocation */ | 3596 | /* See DP v2.0 2.6.4.2, 2.7.6.3 VCPayload_Bandwidth_for_OneTimeSlotPer_MTP_Allocation */ |
| 3590 | return link_rate * link_lane_count / 54000; | 3597 | ret.full = DIV_ROUND_DOWN_ULL(mul_u32_u32(link_rate * link_lane_count, |
| 3598 | ch_coding_efficiency), | ||
| 3599 | (1000000ULL * 8 * 5400) >> 12); | ||
| 3600 | |||
| 3601 | return ret; | ||
| 3591 | } | 3602 | } |
| 3592 | EXPORT_SYMBOL(drm_dp_get_vc_payload_bw); | 3603 | EXPORT_SYMBOL(drm_dp_get_vc_payload_bw); |
| 3593 | 3604 | ||
| @@ -4335,7 +4346,7 @@ int drm_dp_atomic_find_time_slots(struct drm_atomic_state *state, · 741 unmodified lines | |||
| 4335 | } | 4346 | } |
| 4336 | } | 4347 | } |
| 4337 | 4348 | ||
| 4338 | req_slots = DIV_ROUND_UP(pbn, topology_state->pbn_div); | 4349 | req_slots = DIV_ROUND_UP(dfixed_const(pbn), topology_state->pbn_div.full); |
| 4339 | 4350 | ||
| 4340 | drm_dbg_atomic(mgr->dev, "[CONNECTOR:%d:%s] [MST PORT:%p] TU %d -> %d\n", | 4351 | drm_dbg_atomic(mgr->dev, "[CONNECTOR:%d:%s] [MST PORT:%p] TU %d -> %d\n", |
| 4341 | port->connector->base.id, port->connector->name, | 4352 | port->connector->base.id, port->connector->name, |
| @@ -4718,35 +4729,36 @@ EXPORT_SYMBOL(drm_dp_check_act_status); · 376 unmodified lines | |||
| 4718 | 4729 | ||
| 4719 | /** | 4730 | /** |
| 4720 | * drm_dp_calc_pbn_mode() - Calculate the PBN for a mode. | 4731 | * drm_dp_calc_pbn_mode() - Calculate the PBN for a mode. |
| 4721 | * @clock: dot clock for the mode | 4732 | * @clock: dot clock |
| 4722 | * @bpp: bpp for the mode. | 4733 | * @bpp: bpp as .4 binary fixed point |
| 4723 | * @dsc: DSC mode. If true, bpp has units of 1/16 of a bit per pixel | ||
| 4724 | * | 4734 | * |
| 4725 | * This uses the formula in the spec to calculate the PBN value for a mode. | 4735 | * This uses the formula in the spec to calculate the PBN value for a mode. |
| 4726 | */ | 4736 | */ |
| 4727 | int drm_dp_calc_pbn_mode(int clock, int bpp, bool dsc) | 4737 | int drm_dp_calc_pbn_mode(int clock, int bpp) |
| 4728 | { | 4738 | { |
| 4729 | /* | 4739 | /* |
| 4730 | * margin 5300ppm + 300ppm ~ 0.6% as per spec, factor is 1.006 | ||
| 4731 | * The unit of 54/64Mbytes/sec is an arbitrary unit chosen based on | 4740 | * The unit of 54/64Mbytes/sec is an arbitrary unit chosen based on |
| 4732 | * common multiplier to render an integer PBN for all link rate/lane | 4741 | * common multiplier to render an integer PBN for all link rate/lane |
| 4733 | * counts combinations | 4742 | * counts combinations |
| 4734 | * calculate | 4743 | * calculate |
| 4735 | * peak_kbps *= (1006/1000) | 4744 | * peak_kbps = clock * bpp / 16 |
| 4736 | * peak_kbps *= (64/54) | 4745 | * peak_kbps *= SSC overhead / 1000000 |
| 4737 | * peak_kbps *= 8 convert to bytes | 4746 | * peak_kbps /= 8 convert to Kbytes |
| 4738 | * | 4747 | * peak_kBps *= (64/54) / 1000 convert to PBN |
| 4739 | * If the bpp is in units of 1/16, further divide by 16. Put this | ||
| 4740 | * factor in the numerator rather than the denominator to avoid | ||
| 4741 | * integer overflow | ||
| 4742 | */ | 4748 | */ |
| 4749 | /* | ||
| 4750 | * TODO: Use the actual link and mode parameters to calculate | ||
| 4751 | * the overhead. For now it's assumed that these are | ||
| 4752 | * 4 link lanes, 4096 hactive pixels, which don't add any | ||
| 4753 | * significant data padding overhead and that there is no DSC | ||
| 4754 | * or FEC overhead. | ||
| 4755 | */ | ||
| 4756 | int overhead = drm_dp_bw_overhead(4, 4096, 0, bpp, | ||
| 4757 | DRM_DP_BW_OVERHEAD_MST | | ||
| 4758 | DRM_DP_BW_OVERHEAD_SSC_REF_CLK); | ||
| 4743 | 4759 | ||
| 4744 | if (dsc) | 4760 | return DIV64_U64_ROUND_UP(mul_u32_u32(clock * bpp, 64 * overhead >> 4), |
| 4745 | return DIV_ROUND_UP_ULL(mul_u32_u32(clock * (bpp / 16), 64 * 1006), | 4761 | 1000000ULL * 8 * 54 * 1000); |
| 4746 | 8 * 54 * 1000 * 1000); | ||
| 4747 | |||
| 4748 | return DIV_ROUND_UP_ULL(mul_u32_u32(clock * bpp, 64 * 1006), | ||
| 4749 | 8 * 54 * 1000 * 1000); | ||
| 4750 | } | 4762 | } |
| 4751 | EXPORT_SYMBOL(drm_dp_calc_pbn_mode); | 4763 | EXPORT_SYMBOL(drm_dp_calc_pbn_mode); |
| 4752 | 4764 | ||
| @@ -4871,7 +4883,8 @@ void drm_dp_mst_dump_topology(struct seq_file *m, · 118 unmodified lines | |||
| 4871 | state = to_drm_dp_mst_topology_state(mgr->base.state); | 4883 | state = to_drm_dp_mst_topology_state(mgr->base.state); |
| 4872 | seq_printf(m, "\n*** Atomic state info ***\n"); | 4884 | seq_printf(m, "\n*** Atomic state info ***\n"); |
| 4873 | seq_printf(m, "payload_mask: %x, max_payloads: %d, start_slot: %u, pbn_div: %d\n", | 4885 | seq_printf(m, "payload_mask: %x, max_payloads: %d, start_slot: %u, pbn_div: %d\n", |
| 4874 | state->payload_mask, mgr->max_payloads, state->start_slot, state->pbn_div); | 4886 | state->payload_mask, mgr->max_payloads, state->start_slot, |
| 4887 | dfixed_trunc(state->pbn_div)); | ||
| 4875 | 4888 | ||
| 4876 | seq_printf(m, "\n| idx | port | vcpi | slots | pbn | dsc | status | sink name |\n"); | 4889 | seq_printf(m, "\n| idx | port | vcpi | slots | pbn | dsc | status | sink name |\n"); |
| 4877 | for (i = 0; i < mgr->max_payloads; i++) { | 4890 | for (i = 0; i < mgr->max_payloads; i++) { |
| @@ -5136,13 +5149,67 @@ static bool drm_dp_mst_port_downstream_of_branch(struct drm_dp_mst_port *port, · 258 unmodified lines | |||
| 5136 | return false; | 5149 | return false; |
| 5137 | } | 5150 | } |
| 5138 | 5151 | ||
| 5152 | static bool | ||
| 5153 | drm_dp_mst_port_downstream_of_parent_locked(struct drm_dp_mst_topology_mgr *mgr, | ||
| 5154 | struct drm_dp_mst_port *port, | ||
| 5155 | struct drm_dp_mst_port *parent) | ||
| 5156 | { | ||
| 5157 | if (!mgr->mst_primary) | ||
| 5158 | return false; | ||
| 5159 | |||
| 5160 | port = drm_dp_mst_topology_get_port_validated_locked(mgr->mst_primary, | ||
| 5161 | port); | ||
| 5162 | if (!port) | ||
| 5163 | return false; | ||
| 5164 | |||
| 5165 | if (!parent) | ||
| 5166 | return true; | ||
| 5167 | |||
| 5168 | parent = drm_dp_mst_topology_get_port_validated_locked(mgr->mst_primary, | ||
| 5169 | parent); | ||
| 5170 | if (!parent) | ||
| 5171 | return false; | ||
| 5172 | |||
| 5173 | if (!parent->mstb) | ||
| 5174 | return false; | ||
| 5175 | |||
| 5176 | return drm_dp_mst_port_downstream_of_branch(port, parent->mstb); | ||
| 5177 | } | ||
| 5178 | |||
| 5179 | /** | ||
| 5180 | * drm_dp_mst_port_downstream_of_parent - check if a port is downstream of a parent port | ||
| 5181 | * @mgr: MST topology manager | ||
| 5182 | * @port: the port being looked up | ||
| 5183 | * @parent: the parent port | ||
| 5184 | * | ||
| 5185 | * The function returns %true if @port is downstream of @parent. If @parent is | ||
| 5186 | * %NULL - denoting the root port - the function returns %true if @port is in | ||
| 5187 | * @mgr's topology. | ||
| 5188 | */ | ||
| 5189 | bool | ||
| 5190 | drm_dp_mst_port_downstream_of_parent(struct drm_dp_mst_topology_mgr *mgr, | ||
| 5191 | struct drm_dp_mst_port *port, | ||
| 5192 | struct drm_dp_mst_port *parent) | ||
| 5193 | { | ||
| 5194 | bool ret; | ||
| 5195 | |||
| 5196 | mutex_lock(&mgr->lock); | ||
| 5197 | ret = drm_dp_mst_port_downstream_of_parent_locked(mgr, port, parent); | ||
| 5198 | mutex_unlock(&mgr->lock); | ||
| 5199 | |||
| 5200 | return ret; | ||
| 5201 | } | ||
| 5202 | EXPORT_SYMBOL(drm_dp_mst_port_downstream_of_parent); | ||
| 5203 | |||
| 5139 | static int | 5204 | static int |
| 5140 | drm_dp_mst_atomic_check_port_bw_limit(struct drm_dp_mst_port *port, | 5205 | drm_dp_mst_atomic_check_port_bw_limit(struct drm_dp_mst_port *port, |
| 5141 | struct drm_dp_mst_topology_state *state); | 5206 | struct drm_dp_mst_topology_state *state, |
| 5207 | struct drm_dp_mst_port **failing_port); | ||
| 5142 | 5208 | ||
| 5143 | static int | 5209 | static int |
| 5144 | drm_dp_mst_atomic_check_mstb_bw_limit(struct drm_dp_mst_branch *mstb, | 5210 | drm_dp_mst_atomic_check_mstb_bw_limit(struct drm_dp_mst_branch *mstb, |
| 5145 | struct drm_dp_mst_topology_state *state) | 5211 | struct drm_dp_mst_topology_state *state, |
| 5212 | struct drm_dp_mst_port **failing_port) | ||
| 5146 | { | 5213 | { |
| 5147 | struct drm_dp_mst_atomic_payload *payload; | 5214 | struct drm_dp_mst_atomic_payload *payload; |
| 5148 | struct drm_dp_mst_port *port; | 5215 | struct drm_dp_mst_port *port; |
| @@ -5171,7 +5238,7 @@ drm_dp_mst_atomic_check_mstb_bw_limit(struct drm_dp_mst_branch *mstb, · 22 unmodified lines | |||
| 5171 | drm_dbg_atomic(mstb->mgr->dev, "[MSTB:%p] Checking bandwidth limits\n", mstb); | 5238 | drm_dbg_atomic(mstb->mgr->dev, "[MSTB:%p] Checking bandwidth limits\n", mstb); |
| 5172 | 5239 | ||
| 5173 | list_for_each_entry(port, &mstb->ports, next) { | 5240 | list_for_each_entry(port, &mstb->ports, next) { |
| 5174 | ret = drm_dp_mst_atomic_check_port_bw_limit(port, state); | 5241 | ret = drm_dp_mst_atomic_check_port_bw_limit(port, state, failing_port); |
| 5175 | if (ret < 0) | 5242 | if (ret < 0) |
| 5176 | return ret; | 5243 | return ret; |
| 5177 | 5244 | ||
| @@ -5183,7 +5250,8 @@ drm_dp_mst_atomic_check_mstb_bw_limit(struct drm_dp_mst_branch *mstb, · 5 unmodified lines | |||
| 5183 | 5250 | ||
| 5184 | static int | 5251 | static int |
| 5185 | drm_dp_mst_atomic_check_port_bw_limit(struct drm_dp_mst_port *port, | 5252 | drm_dp_mst_atomic_check_port_bw_limit(struct drm_dp_mst_port *port, |
| 5186 | struct drm_dp_mst_topology_state *state) | 5253 | struct drm_dp_mst_topology_state *state, |
| 5254 | struct drm_dp_mst_port **failing_port) | ||
| 5187 | { | 5255 | { |
| 5188 | struct drm_dp_mst_atomic_payload *payload; | 5256 | struct drm_dp_mst_atomic_payload *payload; |
| 5189 | int pbn_used = 0; | 5257 | int pbn_used = 0; |
| @@ -5204,13 +5272,15 @@ drm_dp_mst_atomic_check_port_bw_limit(struct drm_dp_mst_port *port, · 14 unmodified lines | |||
| 5204 | drm_dbg_atomic(port->mgr->dev, | 5272 | drm_dbg_atomic(port->mgr->dev, |
| 5205 | "[MSTB:%p] [MST PORT:%p] no BW available for the port\n", | 5273 | "[MSTB:%p] [MST PORT:%p] no BW available for the port\n", |
| 5206 | port->parent, port); | 5274 | port->parent, port); |
| 5275 | *failing_port = port; | ||
| 5207 | return -EINVAL; | 5276 | return -EINVAL; |
| 5208 | } | 5277 | } |
| 5209 | 5278 | ||
| 5210 | pbn_used = payload->pbn; | 5279 | pbn_used = payload->pbn; |
| 5211 | } else { | 5280 | } else { |
| 5212 | pbn_used = drm_dp_mst_atomic_check_mstb_bw_limit(port->mstb, | 5281 | pbn_used = drm_dp_mst_atomic_check_mstb_bw_limit(port->mstb, |
| 5213 | state); | 5282 | state, |
| 5283 | failing_port); | ||
| 5214 | if (pbn_used <= 0) | 5284 | if (pbn_used <= 0) |
| 5215 | return pbn_used; | 5285 | return pbn_used; |
| 5216 | } | 5286 | } |
| @@ -5219,6 +5289,7 @@ drm_dp_mst_atomic_check_port_bw_limit(struct drm_dp_mst_port *port, · 2 unmodified lines | |||
| 5219 | drm_dbg_atomic(port->mgr->dev, | 5289 | drm_dbg_atomic(port->mgr->dev, |
| 5220 | "[MSTB:%p] [MST PORT:%p] required PBN of %d exceeds port limit of %d\n", | 5290 | "[MSTB:%p] [MST PORT:%p] required PBN of %d exceeds port limit of %d\n", |
| 5221 | port->parent, port, pbn_used, port->full_pbn); | 5291 | port->parent, port, pbn_used, port->full_pbn); |
| 5292 | *failing_port = port; | ||
| 5222 | return -ENOSPC; | 5293 | return -ENOSPC; |
| 5223 | } | 5294 | } |
| 5224 | 5295 | ||
| @@ -5271,10 +5342,10 @@ drm_dp_mst_atomic_check_payload_alloc_limits(struct drm_dp_mst_topology_mgr *mgr · 46 unmodified lines | |||
| 5271 | } | 5342 | } |
| 5272 | 5343 | ||
| 5273 | if (!payload_count) | 5344 | if (!payload_count) |
| 5274 | mst_state->pbn_div = 0; | 5345 | mst_state->pbn_div.full = dfixed_const(0); |
| 5275 | 5346 | ||
| 5276 | drm_dbg_atomic(mgr->dev, "[MST MGR:%p] mst state %p TU pbn_div=%d avail=%d used=%d\n", | 5347 | drm_dbg_atomic(mgr->dev, "[MST MGR:%p] mst state %p TU pbn_div=%d avail=%d used=%d\n", |
| 5277 | mgr, mst_state, mst_state->pbn_div, avail_slots, | 5348 | mgr, mst_state, dfixed_trunc(mst_state->pbn_div), avail_slots, |
| 5278 | mst_state->total_avail_slots - avail_slots); | 5349 | mst_state->total_avail_slots - avail_slots); |
| 5279 | 5350 | ||
| 5280 | return 0; | 5351 | return 0; |
| @@ -5396,20 +5467,82 @@ int drm_dp_mst_atomic_enable_dsc(struct drm_atomic_state *state, · 115 unmodified lines | |||
| 5396 | } | 5467 | } |
| 5397 | EXPORT_SYMBOL(drm_dp_mst_atomic_enable_dsc); | 5468 | EXPORT_SYMBOL(drm_dp_mst_atomic_enable_dsc); |
| 5398 | 5469 | ||
| 5470 | /** | ||
| 5471 | * drm_dp_mst_atomic_check_mgr - Check the atomic state of an MST topology manager | ||
| 5472 | * @state: The global atomic state | ||
| 5473 | * @mgr: Manager to check | ||
| 5474 | * @mst_state: The MST atomic state for @mgr | ||
| 5475 | * @failing_port: Returns the port with a BW limitation | ||
| 5476 | * | ||
| 5477 | * Checks the given MST manager's topology state for an atomic update to ensure | ||
| 5478 | * that it's valid. This includes checking whether there's enough bandwidth to | ||
| 5479 | * support the new timeslot allocations in the atomic update. | ||
| 5480 | * | ||
| 5481 | * Any atomic drivers supporting DP MST must make sure to call this or | ||
| 5482 | * the drm_dp_mst_atomic_check() function after checking the rest of their state | ||
| 5483 | * in their &drm_mode_config_funcs.atomic_check() callback. | ||
| 5484 | * | ||
| 5485 | * See also: | ||
| 5486 | * drm_dp_mst_atomic_check() | ||
| 5487 | * drm_dp_atomic_find_time_slots() | ||
| 5488 | * drm_dp_atomic_release_time_slots() | ||
| 5489 | * | ||
| 5490 | * Returns: | ||
| 5491 | * - 0 if the new state is valid | ||
| 5492 | * - %-ENOSPC, if the new state is invalid, because of BW limitation | ||
| 5493 | * @failing_port is set to: | ||
| 5494 | * - The non-root port where a BW limit check failed | ||
| 5495 | * with all the ports downstream of @failing_port passing | ||
| 5496 | * the BW limit check. | ||
| 5497 | * The returned port pointer is valid until at least | ||
| 5498 | * one payload downstream of it exists. | ||
| 5499 | * - %NULL if the BW limit check failed at the root port | ||
| 5500 | * with all the ports downstream of the root port passing | ||
| 5501 | * the BW limit check. | ||
| 5502 | * - %-EINVAL, if the new state is invalid, because the root port has | ||
| 5503 | * too many payloads. | ||
| 5504 | */ | ||
| 5505 | int drm_dp_mst_atomic_check_mgr(struct drm_atomic_state *state, | ||
| 5506 | struct drm_dp_mst_topology_mgr *mgr, | ||
| 5507 | struct drm_dp_mst_topology_state *mst_state, | ||
| 5508 | struct drm_dp_mst_port **failing_port) | ||
| 5509 | { | ||
| 5510 | int ret; | ||
| 5511 | |||
| 5512 | *failing_port = NULL; | ||
| 5513 | |||
| 5514 | if (!mgr->mst_state) | ||
| 5515 | return 0; | ||
| 5516 | |||
| 5517 | mutex_lock(&mgr->lock); | ||
| 5518 | ret = drm_dp_mst_atomic_check_mstb_bw_limit(mgr->mst_primary, | ||
| 5519 | mst_state, | ||
| 5520 | failing_port); | ||
| 5521 | mutex_unlock(&mgr->lock); | ||
| 5522 | |||
| 5523 | if (ret < 0) | ||
| 5524 | return ret; | ||
| 5525 | |||
| 5526 | return drm_dp_mst_atomic_check_payload_alloc_limits(mgr, mst_state); | ||
| 5527 | } | ||
| 5528 | EXPORT_SYMBOL(drm_dp_mst_atomic_check_mgr); | ||
| 5529 | |||
| 5399 | /** | 5530 | /** |
| 5400 | * drm_dp_mst_atomic_check - Check that the new state of an MST topology in an | 5531 | * drm_dp_mst_atomic_check - Check that the new state of an MST topology in an |
| 5401 | * atomic update is valid | 5532 | * atomic update is valid |
| 5402 | * @state: Pointer to the new &struct drm_dp_mst_topology_state | 5533 | * @state: Pointer to the new &struct drm_dp_mst_topology_state |
| 5403 | * | 5534 | * |
| 5404 | * Checks the given topology state for an atomic update to ensure that it's | 5535 | * Checks the given topology state for an atomic update to ensure that it's |
| 5405 | * valid. This includes checking whether there's enough bandwidth to support | 5536 | * valid, calling drm_dp_mst_atomic_check_mgr() for all MST manager in the |
| 5406 | * the new timeslot allocations in the atomic update. | 5537 | * atomic state. This includes checking whether there's enough bandwidth to |
| 5538 | * support the new timeslot allocations in the atomic update. | ||
| 5407 | * | 5539 | * |
| 5408 | * Any atomic drivers supporting DP MST must make sure to call this after | 5540 | * Any atomic drivers supporting DP MST must make sure to call this after |
| 5409 | * checking the rest of their state in their | 5541 | * checking the rest of their state in their |
| 5410 | * &drm_mode_config_funcs.atomic_check() callback. | 5542 | * &drm_mode_config_funcs.atomic_check() callback. |
| 5411 | * | 5543 | * |
| 5412 | * See also: | 5544 | * See also: |
| 5545 | * drm_dp_mst_atomic_check_mgr() | ||
| 5413 | * drm_dp_atomic_find_time_slots() | 5546 | * drm_dp_atomic_find_time_slots() |
| 5414 | * drm_dp_atomic_release_time_slots() | 5547 | * drm_dp_atomic_release_time_slots() |
| 5415 | * | 5548 | * |
| @@ -5424,21 +5557,11 @@ int drm_dp_mst_atomic_check(struct drm_atomic_state *state) · 8 unmodified lines | |||
| 5424 | int i, ret = 0; | 5557 | int i, ret = 0; |
| 5425 | 5558 | ||
| 5426 | for_each_new_mst_mgr_in_state(state, mgr, mst_state, i) { | 5559 | for_each_new_mst_mgr_in_state(state, mgr, mst_state, i) { |
| 5427 | if (!mgr->mst_state) | 5560 | struct drm_dp_mst_port *tmp_port; |
| 5428 | continue; | ||
| 5429 | 5561 | ||
| 5430 | ret = drm_dp_mst_atomic_check_payload_alloc_limits(mgr, mst_state); | 5562 | ret = drm_dp_mst_atomic_check_mgr(state, mgr, mst_state, &tmp_port); |
| 5431 | if (ret) | 5563 | if (ret) |
| 5432 | break; | 5564 | break; |
| 5433 | |||
| 5434 | mutex_lock(&mgr->lock); | ||
| 5435 | ret = drm_dp_mst_atomic_check_mstb_bw_limit(mgr->mst_primary, | ||
| 5436 | mst_state); | ||
| 5437 | mutex_unlock(&mgr->lock); | ||
| 5438 | if (ret < 0) | ||
| 5439 | break; | ||
| 5440 | else | ||
| 5441 | ret = 0; | ||
| 5442 | } | 5565 | } |
| 5443 | 5566 | ||
| 5444 | return ret; | 5567 | return ret; |
| @@ -5894,6 +6017,7 @@ static bool drm_dp_mst_is_virtual_dpcd(struct drm_dp_mst_port *port) · 449 unmodified lines | |||
| 5894 | struct drm_dp_aux *drm_dp_mst_dsc_aux_for_port(struct drm_dp_mst_port *port) | 6017 | struct drm_dp_aux *drm_dp_mst_dsc_aux_for_port(struct drm_dp_mst_port *port) |
| 5895 | { | 6018 | { |
| 5896 | struct drm_dp_mst_port *immediate_upstream_port; | 6019 | struct drm_dp_mst_port *immediate_upstream_port; |
| 6020 | struct drm_dp_aux *immediate_upstream_aux; | ||
| 5897 | struct drm_dp_mst_port *fec_port; | 6021 | struct drm_dp_mst_port *fec_port; |
| 5898 | struct drm_dp_desc desc = {}; | 6022 | struct drm_dp_desc desc = {}; |
| 5899 | u8 endpoint_fec; | 6023 | u8 endpoint_fec; |
| @@ -5958,21 +6082,25 @@ struct drm_dp_aux *drm_dp_mst_dsc_aux_for_port(struct drm_dp_mst_port *port) · 58 unmodified lines | |||
| 5958 | * - Port is on primary branch device | 6082 | * - Port is on primary branch device |
| 5959 | * - Not a VGA adapter (DP_DWN_STRM_PORT_TYPE_ANALOG) | 6083 | * - Not a VGA adapter (DP_DWN_STRM_PORT_TYPE_ANALOG) |
| 5960 | */ | 6084 | */ |
| 5961 | if (drm_dp_read_desc(port->mgr->aux, &desc, true)) | 6085 | if (immediate_upstream_port) |
| 6086 | immediate_upstream_aux = &immediate_upstream_port->aux; | ||
| 6087 | else | ||
| 6088 | immediate_upstream_aux = port->mgr->aux; | ||
| 6089 | |||
| 6090 | if (drm_dp_read_desc(immediate_upstream_aux, &desc, true)) | ||
| 5962 | return NULL; | 6091 | return NULL; |
| 5963 | 6092 | ||
| 5964 | if (drm_dp_has_quirk(&desc, DP_DPCD_QUIRK_DSC_WITHOUT_VIRTUAL_DPCD) && | 6093 | if (drm_dp_has_quirk(&desc, DP_DPCD_QUIRK_DSC_WITHOUT_VIRTUAL_DPCD)) { |
| 5965 | port->mgr->dpcd[DP_DPCD_REV] >= DP_DPCD_REV_14 && | ||
| 5966 | port->parent == port->mgr->mst_primary) { | ||
| 5967 | u8 dpcd_ext[DP_RECEIVER_CAP_SIZE]; | 6094 | u8 dpcd_ext[DP_RECEIVER_CAP_SIZE]; |
| 5968 | 6095 | ||
| 5969 | if (drm_dp_read_dpcd_caps(port->mgr->aux, dpcd_ext) < 0) | 6096 | if (drm_dp_read_dpcd_caps(immediate_upstream_aux, dpcd_ext) < 0) |
| 5970 | return NULL; | 6097 | return NULL; |
| 5971 | 6098 | ||
| 5972 | if ((dpcd_ext[DP_DOWNSTREAMPORT_PRESENT] & DP_DWN_STRM_PORT_PRESENT) && | 6099 | if (dpcd_ext[DP_DPCD_REV] >= DP_DPCD_REV_14 && |
| 6100 | ((dpcd_ext[DP_DOWNSTREAMPORT_PRESENT] & DP_DWN_STRM_PORT_PRESENT) && | ||
| 5973 | ((dpcd_ext[DP_DOWNSTREAMPORT_PRESENT] & DP_DWN_STRM_PORT_TYPE_MASK) | 6101 | ((dpcd_ext[DP_DOWNSTREAMPORT_PRESENT] & DP_DWN_STRM_PORT_TYPE_MASK) |
| 5974 | != DP_DWN_STRM_PORT_TYPE_ANALOG)) | 6102 | != DP_DWN_STRM_PORT_TYPE_ANALOG))) |
| 5975 | return port->mgr->aux; | 6103 | return immediate_upstream_aux; |
| 5976 | } | 6104 | } |
| 5977 | 6105 | ||
| 5978 | /* | 6106 | /* |
| @@ -1,451 +0,0 @@ | |||
| 1 | /* | ||
| 2 | * \file drm_agpsupport.c | ||
| 3 | * DRM support for AGP/GART backend | ||
| 4 | * | ||
| 5 | * \author Rickard E. (Rik) Faith <faith@valinux.com> | ||
| 6 | * \author Gareth Hughes <gareth@valinux.com> | ||
| 7 | */ | ||
| 8 | |||
| 9 | /* | ||
| 10 | * Copyright 1999 Precision Insight, Inc., Cedar Park, Texas. | ||
| 11 | * Copyright 2000 VA Linux Systems, Inc., Sunnyvale, California. | ||
| 12 | * All Rights Reserved. | ||
| 13 | * | ||
| 14 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 15 | * copy of this software and associated documentation files (the "Software"), | ||
| 16 | * to deal in the Software without restriction, including without limitation | ||
| 17 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 18 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 19 | * Software is furnished to do so, subject to the following conditions: | ||
| 20 | * | ||
| 21 | * The above copyright notice and this permission notice (including the next | ||
| 22 | * paragraph) shall be included in all copies or substantial portions of the | ||
| 23 | * Software. | ||
| 24 | * | ||
| 25 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 26 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 27 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 28 | * VA LINUX SYSTEMS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR | ||
| 29 | * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | ||
| 30 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | ||
| 31 | * OTHER DEALINGS IN THE SOFTWARE. | ||
| 32 | */ | ||
| 33 | |||
| 34 | #include <linux/module.h> | ||
| 35 | #include <linux/pci.h> | ||
| 36 | #include <linux/slab.h> | ||
| 37 | |||
| 38 | #if IS_ENABLED(CONFIG_AGP) | ||
| 39 | #include <asm/agp.h> | ||
| 40 | #endif | ||
| 41 | |||
| 42 | #include <drm/drm_device.h> | ||
| 43 | #include <drm/drm_drv.h> | ||
| 44 | #include <drm/drm_file.h> | ||
| 45 | #include <drm/drm_print.h> | ||
| 46 | |||
| 47 | #include "drm_legacy.h" | ||
| 48 | |||
| 49 | #if IS_ENABLED(CONFIG_AGP) | ||
| 50 | |||
| 51 | /* | ||
| 52 | * Get AGP information. | ||
| 53 | * | ||
| 54 | * \return zero on success or a negative number on failure. | ||
| 55 | * | ||
| 56 | * Verifies the AGP device has been initialized and acquired and fills in the | ||
| 57 | * drm_agp_info structure with the information in drm_agp_head::agp_info. | ||
| 58 | */ | ||
| 59 | int drm_legacy_agp_info(struct drm_device *dev, struct drm_agp_info *info) | ||
| 60 | { | ||
| 61 | struct agp_kern_info *kern; | ||
| 62 | |||
| 63 | if (!dev->agp || !dev->agp->acquired) | ||
| 64 | return -EINVAL; | ||
| 65 | |||
| 66 | kern = &dev->agp->agp_info; | ||
| 67 | info->agp_version_major = kern->version.major; | ||
| 68 | info->agp_version_minor = kern->version.minor; | ||
| 69 | info->mode = kern->mode; | ||
| 70 | info->aperture_base = kern->aper_base; | ||
| 71 | info->aperture_size = kern->aper_size * 1024 * 1024; | ||
| 72 | info->memory_allowed = kern->max_memory << PAGE_SHIFT; | ||
| 73 | info->memory_used = kern->current_memory << PAGE_SHIFT; | ||
| 74 | info->id_vendor = kern->device->vendor; | ||
| 75 | info->id_device = kern->device->device; | ||
| 76 | |||
| 77 | return 0; | ||
| 78 | } | ||
| 79 | EXPORT_SYMBOL(drm_legacy_agp_info); | ||
| 80 | |||
| 81 | int drm_legacy_agp_info_ioctl(struct drm_device *dev, void *data, | ||
| 82 | struct drm_file *file_priv) | ||
| 83 | { | ||
| 84 | struct drm_agp_info *info = data; | ||
| 85 | int err; | ||
| 86 | |||
| 87 | err = drm_legacy_agp_info(dev, info); | ||
| 88 | if (err) | ||
| 89 | return err; | ||
| 90 | |||
| 91 | return 0; | ||
| 92 | } | ||
| 93 | |||
| 94 | /* | ||
| 95 | * Acquire the AGP device. | ||
| 96 | * | ||
| 97 | * \param dev DRM device that is to acquire AGP. | ||
| 98 | * \return zero on success or a negative number on failure. | ||
| 99 | * | ||
| 100 | * Verifies the AGP device hasn't been acquired before and calls | ||
| 101 | * \c agp_backend_acquire. | ||
| 102 | */ | ||
| 103 | int drm_legacy_agp_acquire(struct drm_device *dev) | ||
| 104 | { | ||
| 105 | struct pci_dev *pdev = to_pci_dev(dev->dev); | ||
| 106 | |||
| 107 | if (!dev->agp) | ||
| 108 | return -ENODEV; | ||
| 109 | if (dev->agp->acquired) | ||
| 110 | return -EBUSY; | ||
| 111 | dev->agp->bridge = agp_backend_acquire(pdev); | ||
| 112 | if (!dev->agp->bridge) | ||
| 113 | return -ENODEV; | ||
| 114 | dev->agp->acquired = 1; | ||
| 115 | return 0; | ||
| 116 | } | ||
| 117 | EXPORT_SYMBOL(drm_legacy_agp_acquire); | ||
| 118 | |||
| 119 | /* | ||
| 120 | * Acquire the AGP device (ioctl). | ||
| 121 | * | ||
| 122 | * \return zero on success or a negative number on failure. | ||
| 123 | * | ||
| 124 | * Verifies the AGP device hasn't been acquired before and calls | ||
| 125 | * \c agp_backend_acquire. | ||
| 126 | */ | ||
| 127 | int drm_legacy_agp_acquire_ioctl(struct drm_device *dev, void *data, | ||
| 128 | struct drm_file *file_priv) | ||
| 129 | { | ||
| 130 | return drm_legacy_agp_acquire((struct drm_device *)file_priv->minor->dev); | ||
| 131 | } | ||
| 132 | |||
| 133 | /* | ||
| 134 | * Release the AGP device. | ||
| 135 | * | ||
| 136 | * \param dev DRM device that is to release AGP. | ||
| 137 | * \return zero on success or a negative number on failure. | ||
| 138 | * | ||
| 139 | * Verifies the AGP device has been acquired and calls \c agp_backend_release. | ||
| 140 | */ | ||
| 141 | int drm_legacy_agp_release(struct drm_device *dev) | ||
| 142 | { | ||
| 143 | if (!dev->agp || !dev->agp->acquired) | ||
| 144 | return -EINVAL; | ||
| 145 | agp_backend_release(dev->agp->bridge); | ||
| 146 | dev->agp->acquired = 0; | ||
| 147 | return 0; | ||
| 148 | } | ||
| 149 | EXPORT_SYMBOL(drm_legacy_agp_release); | ||
| 150 | |||
| 151 | int drm_legacy_agp_release_ioctl(struct drm_device *dev, void *data, | ||
| 152 | struct drm_file *file_priv) | ||
| 153 | { | ||
| 154 | return drm_legacy_agp_release(dev); | ||
| 155 | } | ||
| 156 | |||
| 157 | /* | ||
| 158 | * Enable the AGP bus. | ||
| 159 | * | ||
| 160 | * \param dev DRM device that has previously acquired AGP. | ||
| 161 | * \param mode Requested AGP mode. | ||
| 162 | * \return zero on success or a negative number on failure. | ||
| 163 | * | ||
| 164 | * Verifies the AGP device has been acquired but not enabled, and calls | ||
| 165 | * \c agp_enable. | ||
| 166 | */ | ||
| 167 | int drm_legacy_agp_enable(struct drm_device *dev, struct drm_agp_mode mode) | ||
| 168 | { | ||
| 169 | if (!dev->agp || !dev->agp->acquired) | ||
| 170 | return -EINVAL; | ||
| 171 | |||
| 172 | dev->agp->mode = mode.mode; | ||
| 173 | agp_enable(dev->agp->bridge, mode.mode); | ||
| 174 | dev->agp->enabled = 1; | ||
| 175 | return 0; | ||
| 176 | } | ||
| 177 | EXPORT_SYMBOL(drm_legacy_agp_enable); | ||
| 178 | |||
| 179 | int drm_legacy_agp_enable_ioctl(struct drm_device *dev, void *data, | ||
| 180 | struct drm_file *file_priv) | ||
| 181 | { | ||
| 182 | struct drm_agp_mode *mode = data; | ||
| 183 | |||
| 184 | return drm_legacy_agp_enable(dev, *mode); | ||
| 185 | } | ||
| 186 | |||
| 187 | /* | ||
| 188 | * Allocate AGP memory. | ||
| 189 | * | ||
| 190 | * \return zero on success or a negative number on failure. | ||
| 191 | * | ||
| 192 | * Verifies the AGP device is present and has been acquired, allocates the | ||
| 193 | * memory via agp_allocate_memory() and creates a drm_agp_mem entry for it. | ||
| 194 | */ | ||
| 195 | int drm_legacy_agp_alloc(struct drm_device *dev, struct drm_agp_buffer *request) | ||
| 196 | { | ||
| 197 | struct drm_agp_mem *entry; | ||
| 198 | struct agp_memory *memory; | ||
| 199 | unsigned long pages; | ||
| 200 | u32 type; | ||
| 201 | |||
| 202 | if (!dev->agp || !dev->agp->acquired) | ||
| 203 | return -EINVAL; | ||
| 204 | entry = kzalloc(sizeof(*entry), GFP_KERNEL); | ||
| 205 | if (!entry) | ||
| 206 | return -ENOMEM; | ||
| 207 | |||
| 208 | pages = DIV_ROUND_UP(request->size, PAGE_SIZE); | ||
| 209 | type = (u32) request->type; | ||
| 210 | memory = agp_allocate_memory(dev->agp->bridge, pages, type); | ||
| 211 | if (!memory) { | ||
| 212 | kfree(entry); | ||
| 213 | return -ENOMEM; | ||
| 214 | } | ||
| 215 | |||
| 216 | entry->handle = (unsigned long)memory->key + 1; | ||
| 217 | entry->memory = memory; | ||
| 218 | entry->bound = 0; | ||
| 219 | entry->pages = pages; | ||
| 220 | list_add(&entry->head, &dev->agp->memory); | ||
| 221 | |||
| 222 | request->handle = entry->handle; | ||
| 223 | request->physical = memory->physical; | ||
| 224 | |||
| 225 | return 0; | ||
| 226 | } | ||
| 227 | EXPORT_SYMBOL(drm_legacy_agp_alloc); | ||
| 228 | |||
| 229 | |||
| 230 | int drm_legacy_agp_alloc_ioctl(struct drm_device *dev, void *data, | ||
| 231 | struct drm_file *file_priv) | ||
| 232 | { | ||
| 233 | struct drm_agp_buffer *request = data; | ||
| 234 | |||
| 235 | return drm_legacy_agp_alloc(dev, request); | ||
| 236 | } | ||
| 237 | |||
| 238 | /* | ||
| 239 | * Search for the AGP memory entry associated with a handle. | ||
| 240 | * | ||
| 241 | * \param dev DRM device structure. | ||
| 242 | * \param handle AGP memory handle. | ||
| 243 | * \return pointer to the drm_agp_mem structure associated with \p handle. | ||
| 244 | * | ||
| 245 | * Walks through drm_agp_head::memory until finding a matching handle. | ||
| 246 | */ | ||
| 247 | static struct drm_agp_mem *drm_legacy_agp_lookup_entry(struct drm_device *dev, | ||
| 248 | unsigned long handle) | ||
| 249 | { | ||
| 250 | struct drm_agp_mem *entry; | ||
| 251 | |||
| 252 | list_for_each_entry(entry, &dev->agp->memory, head) { | ||
| 253 | if (entry->handle == handle) | ||
| 254 | return entry; | ||
| 255 | } | ||
| 256 | return NULL; | ||
| 257 | } | ||
| 258 | |||
| 259 | /* | ||
| 260 | * Unbind AGP memory from the GATT (ioctl). | ||
| 261 | * | ||
| 262 | * \return zero on success or a negative number on failure. | ||
| 263 | * | ||
| 264 | * Verifies the AGP device is present and acquired, looks-up the AGP memory | ||
| 265 | * entry and passes it to the unbind_agp() function. | ||
| 266 | */ | ||
| 267 | int drm_legacy_agp_unbind(struct drm_device *dev, struct drm_agp_binding *request) | ||
| 268 | { | ||
| 269 | struct drm_agp_mem *entry; | ||
| 270 | int ret; | ||
| 271 | |||
| 272 | if (!dev->agp || !dev->agp->acquired) | ||
| 273 | return -EINVAL; | ||
| 274 | entry = drm_legacy_agp_lookup_entry(dev, request->handle); | ||
| 275 | if (!entry || !entry->bound) | ||
| 276 | return -EINVAL; | ||
| 277 | ret = agp_unbind_memory(entry->memory); | ||
| 278 | if (ret == 0) | ||
| 279 | entry->bound = 0; | ||
| 280 | return ret; | ||
| 281 | } | ||
| 282 | EXPORT_SYMBOL(drm_legacy_agp_unbind); | ||
| 283 | |||
| 284 | |||
| 285 | int drm_legacy_agp_unbind_ioctl(struct drm_device *dev, void *data, | ||
| 286 | struct drm_file *file_priv) | ||
| 287 | { | ||
| 288 | struct drm_agp_binding *request = data; | ||
| 289 | |||
| 290 | return drm_legacy_agp_unbind(dev, request); | ||
| 291 | } | ||
| 292 | |||
| 293 | /* | ||
| 294 | * Bind AGP memory into the GATT (ioctl) | ||
| 295 | * | ||
| 296 | * \return zero on success or a negative number on failure. | ||
| 297 | * | ||
| 298 | * Verifies the AGP device is present and has been acquired and that no memory | ||
| 299 | * is currently bound into the GATT. Looks-up the AGP memory entry and passes | ||
| 300 | * it to bind_agp() function. | ||
| 301 | */ | ||
| 302 | int drm_legacy_agp_bind(struct drm_device *dev, struct drm_agp_binding *request) | ||
| 303 | { | ||
| 304 | struct drm_agp_mem *entry; | ||
| 305 | int retcode; | ||
| 306 | int page; | ||
| 307 | |||
| 308 | if (!dev->agp || !dev->agp->acquired) | ||
| 309 | return -EINVAL; | ||
| 310 | entry = drm_legacy_agp_lookup_entry(dev, request->handle); | ||
| 311 | if (!entry || entry->bound) | ||
| 312 | return -EINVAL; | ||
| 313 | page = DIV_ROUND_UP(request->offset, PAGE_SIZE); | ||
| 314 | retcode = agp_bind_memory(entry->memory, page); | ||
| 315 | if (retcode) | ||
| 316 | return retcode; | ||
| 317 | entry->bound = dev->agp->base + (page << PAGE_SHIFT); | ||
| 318 | DRM_DEBUG("base = 0x%lx entry->bound = 0x%lx\n", | ||
| 319 | dev->agp->base, entry->bound); | ||
| 320 | return 0; | ||
| 321 | } | ||
| 322 | EXPORT_SYMBOL(drm_legacy_agp_bind); | ||
| 323 | |||
| 324 | |||
| 325 | int drm_legacy_agp_bind_ioctl(struct drm_device *dev, void *data, | ||
| 326 | struct drm_file *file_priv) | ||
| 327 | { | ||
| 328 | struct drm_agp_binding *request = data; | ||
| 329 | |||
| 330 | return drm_legacy_agp_bind(dev, request); | ||
| 331 | } | ||
| 332 | |||
| 333 | /* | ||
| 334 | * Free AGP memory (ioctl). | ||
| 335 | * | ||
| 336 | * \return zero on success or a negative number on failure. | ||
| 337 | * | ||
| 338 | * Verifies the AGP device is present and has been acquired and looks up the | ||
| 339 | * AGP memory entry. If the memory is currently bound, unbind it via | ||
| 340 | * unbind_agp(). Frees it via free_agp() as well as the entry itself | ||
| 341 | * and unlinks from the doubly linked list it's inserted in. | ||
| 342 | */ | ||
| 343 | int drm_legacy_agp_free(struct drm_device *dev, struct drm_agp_buffer *request) | ||
| 344 | { | ||
| 345 | struct drm_agp_mem *entry; | ||
| 346 | |||
| 347 | if (!dev->agp || !dev->agp->acquired) | ||
| 348 | return -EINVAL; | ||
| 349 | entry = drm_legacy_agp_lookup_entry(dev, request->handle); | ||
| 350 | if (!entry) | ||
| 351 | return -EINVAL; | ||
| 352 | if (entry->bound) | ||
| 353 | agp_unbind_memory(entry->memory); | ||
| 354 | |||
| 355 | list_del(&entry->head); | ||
| 356 | |||
| 357 | agp_free_memory(entry->memory); | ||
| 358 | kfree(entry); | ||
| 359 | return 0; | ||
| 360 | } | ||
| 361 | EXPORT_SYMBOL(drm_legacy_agp_free); | ||
| 362 | |||
| 363 | |||
| 364 | int drm_legacy_agp_free_ioctl(struct drm_device *dev, void *data, | ||
| 365 | struct drm_file *file_priv) | ||
| 366 | { | ||
| 367 | struct drm_agp_buffer *request = data; | ||
| 368 | |||
| 369 | return drm_legacy_agp_free(dev, request); | ||
| 370 | } | ||
| 371 | |||
| 372 | /* | ||
| 373 | * Initialize the AGP resources. | ||
| 374 | * | ||
| 375 | * \return pointer to a drm_agp_head structure. | ||
| 376 | * | ||
| 377 | * Gets the drm_agp_t structure which is made available by the agpgart module | ||
| 378 | * via the inter_module_* functions. Creates and initializes a drm_agp_head | ||
| 379 | * structure. | ||
| 380 | * | ||
| 381 | * Note that final cleanup of the kmalloced structure is directly done in | ||
| 382 | * drm_pci_agp_destroy. | ||
| 383 | */ | ||
| 384 | struct drm_agp_head *drm_legacy_agp_init(struct drm_device *dev) | ||
| 385 | { | ||
| 386 | struct pci_dev *pdev = to_pci_dev(dev->dev); | ||
| 387 | struct drm_agp_head *head = NULL; | ||
| 388 | |||
| 389 | head = kzalloc(sizeof(*head), GFP_KERNEL); | ||
| 390 | if (!head) | ||
| 391 | return NULL; | ||
| 392 | head->bridge = agp_find_bridge(pdev); | ||
| 393 | if (!head->bridge) { | ||
| 394 | head->bridge = agp_backend_acquire(pdev); | ||
| 395 | if (!head->bridge) { | ||
| 396 | kfree(head); | ||
| 397 | return NULL; | ||
| 398 | } | ||
| 399 | agp_copy_info(head->bridge, &head->agp_info); | ||
| 400 | agp_backend_release(head->bridge); | ||
| 401 | } else { | ||
| 402 | agp_copy_info(head->bridge, &head->agp_info); | ||
| 403 | } | ||
| 404 | if (head->agp_info.chipset == NOT_SUPPORTED) { | ||
| 405 | kfree(head); | ||
| 406 | return NULL; | ||
| 407 | } | ||
| 408 | INIT_LIST_HEAD(&head->memory); | ||
| 409 | head->cant_use_aperture = head->agp_info.cant_use_aperture; | ||
| 410 | head->page_mask = head->agp_info.page_mask; | ||
| 411 | head->base = head->agp_info.aper_base; | ||
| 412 | return head; | ||
| 413 | } | ||
| 414 | /* Only exported for i810.ko */ | ||
| 415 | EXPORT_SYMBOL(drm_legacy_agp_init); | ||
| 416 | |||
| 417 | /** | ||
| 418 | * drm_legacy_agp_clear - Clear AGP resource list | ||
| 419 | * @dev: DRM device | ||
| 420 | * | ||
| 421 | * Iterate over all AGP resources and remove them. But keep the AGP head | ||
| 422 | * intact so it can still be used. It is safe to call this if AGP is disabled or | ||
| 423 | * was already removed. | ||
| 424 | * | ||
| 425 | * Cleanup is only done for drivers who have DRIVER_LEGACY set. | ||
| 426 | */ | ||
| 427 | void drm_legacy_agp_clear(struct drm_device *dev) | ||
| 428 | { | ||
| 429 | struct drm_agp_mem *entry, *tempe; | ||
| 430 | |||
| 431 | if (!dev->agp) | ||
| 432 | return; | ||
| 433 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 434 | return; | ||
| 435 | |||
| 436 | list_for_each_entry_safe(entry, tempe, &dev->agp->memory, head) { | ||
| 437 | if (entry->bound) | ||
| 438 | agp_unbind_memory(entry->memory); | ||
| 439 | agp_free_memory(entry->memory); | ||
| 440 | kfree(entry); | ||
| 441 | } | ||
| 442 | INIT_LIST_HEAD(&dev->agp->memory); | ||
| 443 | |||
| 444 | if (dev->agp->acquired) | ||
| 445 | drm_legacy_agp_release(dev); | ||
| 446 | |||
| 447 | dev->agp->acquired = 0; | ||
| 448 | dev->agp->enabled = 0; | ||
| 449 | } | ||
| 450 | |||
| 451 | #endif | ||
| @@ -1773,6 +1773,7 @@ static void __drm_state_dump(struct drm_device *dev, struct drm_printer *p, · 1772 unmodified lines | |||
| 1773 | struct drm_crtc *crtc; | 1773 | struct drm_crtc *crtc; |
| 1774 | struct drm_connector *connector; | 1774 | struct drm_connector *connector; |
| 1775 | struct drm_connector_list_iter conn_iter; | 1775 | struct drm_connector_list_iter conn_iter; |
| 1776 | struct drm_private_obj *obj; | ||
| 1776 | 1777 | ||
| 1777 | if (!drm_drv_uses_atomic_modeset(dev)) | 1778 | if (!drm_drv_uses_atomic_modeset(dev)) |
| 1778 | return; | 1779 | return; |
| @@ -1801,6 +1802,14 @@ static void __drm_state_dump(struct drm_device *dev, struct drm_printer *p, · 22 unmodified lines | |||
| 1801 | if (take_locks) | 1802 | if (take_locks) |
| 1802 | drm_modeset_unlock(&dev->mode_config.connection_mutex); | 1803 | drm_modeset_unlock(&dev->mode_config.connection_mutex); |
| 1803 | drm_connector_list_iter_end(&conn_iter); | 1804 | drm_connector_list_iter_end(&conn_iter); |
| 1805 | |||
| 1806 | list_for_each_entry(obj, &config->privobj_list, head) { | ||
| 1807 | if (take_locks) | ||
| 1808 | drm_modeset_lock(&obj->lock, NULL); | ||
| 1809 | drm_atomic_private_obj_print_state(p, obj->state); | ||
| 1810 | if (take_locks) | ||
| 1811 | drm_modeset_unlock(&obj->lock); | ||
| 1812 | } | ||
| 1804 | } | 1813 | } |
| 1805 | 1814 | ||
| 1806 | /** | 1815 | /** |
| @@ -795,9 +795,9 @@ drm_atomic_helper_check_modeset(struct drm_device *dev, · 794 unmodified lines | |||
| 795 | EXPORT_SYMBOL(drm_atomic_helper_check_modeset); | 795 | EXPORT_SYMBOL(drm_atomic_helper_check_modeset); |
| 796 | 796 | ||
| 797 | /** | 797 | /** |
| 798 | * drm_atomic_helper_check_wb_encoder_state() - Check writeback encoder state | 798 | * drm_atomic_helper_check_wb_connector_state() - Check writeback connector state |
| 799 | * @encoder: encoder state to check | 799 | * @connector: corresponding connector |
| 800 | * @conn_state: connector state to check | 800 | * @state: the driver state object |
| 801 | * | 801 | * |
| 802 | * Checks if the writeback connector state is valid, and returns an error if it | 802 | * Checks if the writeback connector state is valid, and returns an error if it |
| 803 | * isn't. | 803 | * isn't. |
| @@ -806,9 +806,11 @@ EXPORT_SYMBOL(drm_atomic_helper_check_modeset); · 2 unmodified lines | |||
| 806 | * Zero for success or -errno | 806 | * Zero for success or -errno |
| 807 | */ | 807 | */ |
| 808 | int | 808 | int |
| 809 | drm_atomic_helper_check_wb_encoder_state(struct drm_encoder *encoder, | 809 | drm_atomic_helper_check_wb_connector_state(struct drm_connector *connector, |
| 810 | struct drm_connector_state *conn_state) | 810 | struct drm_atomic_state *state) |
| 811 | { | 811 | { |
| 812 | struct drm_connector_state *conn_state = | ||
| 813 | drm_atomic_get_new_connector_state(state, connector); | ||
| 812 | struct drm_writeback_job *wb_job = conn_state->writeback_job; | 814 | struct drm_writeback_job *wb_job = conn_state->writeback_job; |
| 813 | struct drm_property_blob *pixel_format_blob; | 815 | struct drm_property_blob *pixel_format_blob; |
| 814 | struct drm_framebuffer *fb; | 816 | struct drm_framebuffer *fb; |
| @@ -827,11 +829,11 @@ drm_atomic_helper_check_wb_encoder_state(struct drm_encoder *encoder, · 12 unmodified lines | |||
| 827 | if (fb->format->format == formats[i]) | 829 | if (fb->format->format == formats[i]) |
| 828 | return 0; | 830 | return 0; |
| 829 | 831 | ||
| 830 | drm_dbg_kms(encoder->dev, "Invalid pixel format %p4cc\n", &fb->format->format); | 832 | drm_dbg_kms(connector->dev, "Invalid pixel format %p4cc\n", &fb->format->format); |
| 831 | 833 | ||
| 832 | return -EINVAL; | 834 | return -EINVAL; |
| 833 | } | 835 | } |
| 834 | EXPORT_SYMBOL(drm_atomic_helper_check_wb_encoder_state); | 836 | EXPORT_SYMBOL(drm_atomic_helper_check_wb_connector_state); |
| 835 | 837 | ||
| 836 | /** | 838 | /** |
| 837 | * drm_atomic_helper_check_plane_state() - Check plane state for validity | 839 | * drm_atomic_helper_check_plane_state() - Check plane state for validity |
| @@ -2382,10 +2384,10 @@ int drm_atomic_helper_setup_commit(struct drm_atomic_state *state, · 1544 unmodified lines | |||
| 2382 | EXPORT_SYMBOL(drm_atomic_helper_setup_commit); | 2384 | EXPORT_SYMBOL(drm_atomic_helper_setup_commit); |
| 2383 | 2385 | ||
| 2384 | /** | 2386 | /** |
| 2385 | * drm_atomic_helper_wait_for_dependencies - wait for required preceeding commits | 2387 | * drm_atomic_helper_wait_for_dependencies - wait for required preceding commits |
| 2386 | * @old_state: atomic state object with old state structures | 2388 | * @old_state: atomic state object with old state structures |
| 2387 | * | 2389 | * |
| 2388 | * This function waits for all preceeding commits that touch the same CRTC as | 2390 | * This function waits for all preceding commits that touch the same CRTC as |
| 2389 | * @old_state to both be committed to the hardware (as signalled by | 2391 | * @old_state to both be committed to the hardware (as signalled by |
| 2390 | * drm_atomic_helper_commit_hw_done()) and executed by the hardware (as signalled | 2392 | * drm_atomic_helper_commit_hw_done()) and executed by the hardware (as signalled |
| 2391 | * by calling drm_crtc_send_vblank_event() on the &drm_crtc_state.event). | 2393 | * by calling drm_crtc_send_vblank_event() on the &drm_crtc_state.event). |
| @@ -275,6 +275,20 @@ void __drm_atomic_helper_plane_state_reset(struct drm_plane_state *plane_state, · 274 unmodified lines | |||
| 275 | plane_state->normalized_zpos = val; | 275 | plane_state->normalized_zpos = val; |
| 276 | } | 276 | } |
| 277 | } | 277 | } |
| 278 | |||
| 279 | if (plane->hotspot_x_property) { | ||
| 280 | if (!drm_object_property_get_default_value(&plane->base, | ||
| 281 | plane->hotspot_x_property, | ||
| 282 | &val)) | ||
| 283 | plane_state->hotspot_x = val; | ||
| 284 | } | ||
| 285 | |||
| 286 | if (plane->hotspot_y_property) { | ||
| 287 | if (!drm_object_property_get_default_value(&plane->base, | ||
| 288 | plane->hotspot_y_property, | ||
| 289 | &val)) | ||
| 290 | plane_state->hotspot_y = val; | ||
| 291 | } | ||
| 278 | } | 292 | } |
| 279 | EXPORT_SYMBOL(__drm_atomic_helper_plane_state_reset); | 293 | EXPORT_SYMBOL(__drm_atomic_helper_plane_state_reset); |
| 280 | 294 | ||
| @@ -593,6 +593,22 @@ static int drm_atomic_plane_set_property(struct drm_plane *plane, · 592 unmodified lines | |||
| 593 | } else if (plane->funcs->atomic_set_property) { | 593 | } else if (plane->funcs->atomic_set_property) { |
| 594 | return plane->funcs->atomic_set_property(plane, state, | 594 | return plane->funcs->atomic_set_property(plane, state, |
| 595 | property, val); | 595 | property, val); |
| 596 | } else if (property == plane->hotspot_x_property) { | ||
| 597 | if (plane->type != DRM_PLANE_TYPE_CURSOR) { | ||
| 598 | drm_dbg_atomic(plane->dev, | ||
| 599 | "[PLANE:%d:%s] is not a cursor plane: 0x%llx\n", | ||
| 600 | plane->base.id, plane->name, val); | ||
| 601 | return -EINVAL; | ||
| 602 | } | ||
| 603 | state->hotspot_x = val; | ||
| 604 | } else if (property == plane->hotspot_y_property) { | ||
| 605 | if (plane->type != DRM_PLANE_TYPE_CURSOR) { | ||
| 606 | drm_dbg_atomic(plane->dev, | ||
| 607 | "[PLANE:%d:%s] is not a cursor plane: 0x%llx\n", | ||
| 608 | plane->base.id, plane->name, val); | ||
| 609 | return -EINVAL; | ||
| 610 | } | ||
| 611 | state->hotspot_y = val; | ||
| 596 | } else { | 612 | } else { |
| 597 | drm_dbg_atomic(plane->dev, | 613 | drm_dbg_atomic(plane->dev, |
| 598 | "[PLANE:%d:%s] unknown property [PROP:%d:%s]\n", | 614 | "[PLANE:%d:%s] unknown property [PROP:%d:%s]\n", |
| @@ -653,6 +669,10 @@ drm_atomic_plane_get_property(struct drm_plane *plane, · 54 unmodified lines | |||
| 653 | *val = state->scaling_filter; | 669 | *val = state->scaling_filter; |
| 654 | } else if (plane->funcs->atomic_get_property) { | 670 | } else if (plane->funcs->atomic_get_property) { |
| 655 | return plane->funcs->atomic_get_property(plane, state, property, val); | 671 | return plane->funcs->atomic_get_property(plane, state, property, val); |
| 672 | } else if (property == plane->hotspot_x_property) { | ||
| 673 | *val = state->hotspot_x; | ||
| 674 | } else if (property == plane->hotspot_y_property) { | ||
| 675 | *val = state->hotspot_y; | ||
| 656 | } else { | 676 | } else { |
| 657 | drm_dbg_atomic(dev, | 677 | drm_dbg_atomic(dev, |
| 658 | "[PLANE:%d:%s] unknown property [PROP:%d:%s]\n", | 678 | "[PLANE:%d:%s] unknown property [PROP:%d:%s]\n", |
| @@ -1006,13 +1026,28 @@ out: · 347 unmodified lines | |||
| 1006 | return ret; | 1026 | return ret; |
| 1007 | } | 1027 | } |
| 1008 | 1028 | ||
| 1029 | static int drm_atomic_check_prop_changes(int ret, uint64_t old_val, uint64_t prop_value, | ||
| 1030 | struct drm_property *prop) | ||
| 1031 | { | ||
| 1032 | if (ret != 0 || old_val != prop_value) { | ||
| 1033 | drm_dbg_atomic(prop->dev, | ||
| 1034 | "[PROP:%d:%s] No prop can be changed during async flip\n", | ||
| 1035 | prop->base.id, prop->name); | ||
| 1036 | return -EINVAL; | ||
| 1037 | } | ||
| 1038 | |||
| 1039 | return 0; | ||
| 1040 | } | ||
| 1041 | |||
| 1009 | int drm_atomic_set_property(struct drm_atomic_state *state, | 1042 | int drm_atomic_set_property(struct drm_atomic_state *state, |
| 1010 | struct drm_file *file_priv, | 1043 | struct drm_file *file_priv, |
| 1011 | struct drm_mode_object *obj, | 1044 | struct drm_mode_object *obj, |
| 1012 | struct drm_property *prop, | 1045 | struct drm_property *prop, |
| 1013 | uint64_t prop_value) | 1046 | u64 prop_value, |
| 1047 | bool async_flip) | ||
| 1014 | { | 1048 | { |
| 1015 | struct drm_mode_object *ref; | 1049 | struct drm_mode_object *ref; |
| 1050 | u64 old_val; | ||
| 1016 | int ret; | 1051 | int ret; |
| 1017 | 1052 | ||
| 1018 | if (!drm_property_change_valid_get(prop, prop_value, &ref)) | 1053 | if (!drm_property_change_valid_get(prop, prop_value, &ref)) |
| @@ -1029,6 +1064,13 @@ int drm_atomic_set_property(struct drm_atomic_state *state, · 10 unmodified lines | |||
| 1029 | break; | 1064 | break; |
| 1030 | } | 1065 | } |
| 1031 | 1066 | ||
| 1067 | if (async_flip) { | ||
| 1068 | ret = drm_atomic_connector_get_property(connector, connector_state, | ||
| 1069 | prop, &old_val); | ||
| 1070 | ret = drm_atomic_check_prop_changes(ret, old_val, prop_value, prop); | ||
| 1071 | break; | ||
| 1072 | } | ||
| 1073 | |||
| 1032 | ret = drm_atomic_connector_set_property(connector, | 1074 | ret = drm_atomic_connector_set_property(connector, |
| 1033 | connector_state, file_priv, | 1075 | connector_state, file_priv, |
| 1034 | prop, prop_value); | 1076 | prop, prop_value); |
| @@ -1044,6 +1086,13 @@ int drm_atomic_set_property(struct drm_atomic_state *state, · 9 unmodified lines | |||
| 1044 | break; | 1086 | break; |
| 1045 | } | 1087 | } |
| 1046 | 1088 | ||
| 1089 | if (async_flip) { | ||
| 1090 | ret = drm_atomic_crtc_get_property(crtc, crtc_state, | ||
| 1091 | prop, &old_val); | ||
| 1092 | ret = drm_atomic_check_prop_changes(ret, old_val, prop_value, prop); | ||
| 1093 | break; | ||
| 1094 | } | ||
| 1095 | |||
| 1047 | ret = drm_atomic_crtc_set_property(crtc, | 1096 | ret = drm_atomic_crtc_set_property(crtc, |
| 1048 | crtc_state, prop, prop_value); | 1097 | crtc_state, prop, prop_value); |
| 1049 | break; | 1098 | break; |
| @@ -1051,6 +1100,7 @@ int drm_atomic_set_property(struct drm_atomic_state *state, · 1 unmodified line | |||
| 1051 | case DRM_MODE_OBJECT_PLANE: { | 1100 | case DRM_MODE_OBJECT_PLANE: { |
| 1052 | struct drm_plane *plane = obj_to_plane(obj); | 1101 | struct drm_plane *plane = obj_to_plane(obj); |
| 1053 | struct drm_plane_state *plane_state; | 1102 | struct drm_plane_state *plane_state; |
| 1103 | struct drm_mode_config *config = &plane->dev->mode_config; | ||
| 1054 | 1104 | ||
| 1055 | plane_state = drm_atomic_get_plane_state(state, plane); | 1105 | plane_state = drm_atomic_get_plane_state(state, plane); |
| 1056 | if (IS_ERR(plane_state)) { | 1106 | if (IS_ERR(plane_state)) { |
| @@ -1058,6 +1108,21 @@ int drm_atomic_set_property(struct drm_atomic_state *state, · 1 unmodified line | |||
| 1058 | break; | 1108 | break; |
| 1059 | } | 1109 | } |
| 1060 | 1110 | ||
| 1111 | if (async_flip && prop != config->prop_fb_id) { | ||
| 1112 | ret = drm_atomic_plane_get_property(plane, plane_state, | ||
| 1113 | prop, &old_val); | ||
| 1114 | ret = drm_atomic_check_prop_changes(ret, old_val, prop_value, prop); | ||
| 1115 | break; | ||
| 1116 | } | ||
| 1117 | |||
| 1118 | if (async_flip && plane_state->plane->type != DRM_PLANE_TYPE_PRIMARY) { | ||
| 1119 | drm_dbg_atomic(prop->dev, | ||
| 1120 | "[OBJECT:%d] Only primary planes can be changed during async flip\n", | ||
| 1121 | obj->id); | ||
| 1122 | ret = -EINVAL; | ||
| 1123 | break; | ||
| 1124 | } | ||
| 1125 | |||
| 1061 | ret = drm_atomic_plane_set_property(plane, | 1126 | ret = drm_atomic_plane_set_property(plane, |
| 1062 | plane_state, file_priv, | 1127 | plane_state, file_priv, |
| 1063 | prop, prop_value); | 1128 | prop, prop_value); |
| @@ -1323,6 +1388,18 @@ static void complete_signaling(struct drm_device *dev, · 259 unmodified lines | |||
| 1323 | kfree(fence_state); | 1388 | kfree(fence_state); |
| 1324 | } | 1389 | } |
| 1325 | 1390 | ||
| 1391 | static void | ||
| 1392 | set_async_flip(struct drm_atomic_state *state) | ||
| 1393 | { | ||
| 1394 | struct drm_crtc *crtc; | ||
| 1395 | struct drm_crtc_state *crtc_state; | ||
| 1396 | int i; | ||
| 1397 | |||
| 1398 | for_each_new_crtc_in_state(state, crtc, crtc_state, i) { | ||
| 1399 | crtc_state->async_flip = true; | ||
| 1400 | } | ||
| 1401 | } | ||
| 1402 | |||
| 1326 | int drm_mode_atomic_ioctl(struct drm_device *dev, | 1403 | int drm_mode_atomic_ioctl(struct drm_device *dev, |
| 1327 | void *data, struct drm_file *file_priv) | 1404 | void *data, struct drm_file *file_priv) |
| 1328 | { | 1405 | { |
| @@ -1337,6 +1414,7 @@ int drm_mode_atomic_ioctl(struct drm_device *dev, · 8 unmodified lines | |||
| 1337 | struct drm_out_fence_state *fence_state; | 1414 | struct drm_out_fence_state *fence_state; |
| 1338 | int ret = 0; | 1415 | int ret = 0; |
| 1339 | unsigned int i, j, num_fences; | 1416 | unsigned int i, j, num_fences; |
| 1417 | bool async_flip = false; | ||
| 1340 | 1418 | ||
| 1341 | /* disallow for drivers not supporting atomic: */ | 1419 | /* disallow for drivers not supporting atomic: */ |
| 1342 | if (!drm_core_check_feature(dev, DRIVER_ATOMIC)) | 1420 | if (!drm_core_check_feature(dev, DRIVER_ATOMIC)) |
| @@ -1363,9 +1441,13 @@ int drm_mode_atomic_ioctl(struct drm_device *dev, · 20 unmodified lines | |||
| 1363 | } | 1441 | } |
| 1364 | 1442 | ||
| 1365 | if (arg->flags & DRM_MODE_PAGE_FLIP_ASYNC) { | 1443 | if (arg->flags & DRM_MODE_PAGE_FLIP_ASYNC) { |
| 1366 | drm_dbg_atomic(dev, | 1444 | if (!dev->mode_config.async_page_flip) { |
| 1367 | "commit failed: invalid flag DRM_MODE_PAGE_FLIP_ASYNC\n"); | 1445 | drm_dbg_atomic(dev, |
| 1368 | return -EINVAL; | 1446 | "commit failed: DRM_MODE_PAGE_FLIP_ASYNC not supported\n"); |
| 1447 | return -EINVAL; | ||
| 1448 | } | ||
| 1449 | |||
| 1450 | async_flip = true; | ||
| 1369 | } | 1451 | } |
| 1370 | 1452 | ||
| 1371 | /* can't test and expect an event at the same time. */ | 1453 | /* can't test and expect an event at the same time. */ |
| @@ -1450,8 +1532,8 @@ retry: · 78 unmodified lines | |||
| 1450 | goto out; | 1532 | goto out; |
| 1451 | } | 1533 | } |
| 1452 | 1534 | ||
| 1453 | ret = drm_atomic_set_property(state, file_priv, | 1535 | ret = drm_atomic_set_property(state, file_priv, obj, |
| 1454 | obj, prop, prop_value); | 1536 | prop, prop_value, async_flip); |
| 1455 | if (ret) { | 1537 | if (ret) { |
| 1456 | drm_mode_object_put(obj); | 1538 | drm_mode_object_put(obj); |
| 1457 | goto out; | 1539 | goto out; |
| @@ -1468,6 +1550,9 @@ retry: · 10 unmodified lines | |||
| 1468 | if (ret) | 1550 | if (ret) |
| 1469 | goto out; | 1551 | goto out; |
| 1470 | 1552 | ||
| 1553 | if (arg->flags & DRM_MODE_PAGE_FLIP_ASYNC) | ||
| 1554 | set_async_flip(state); | ||
| 1555 | |||
| 1471 | if (arg->flags & DRM_MODE_ATOMIC_TEST_ONLY) { | 1556 | if (arg->flags & DRM_MODE_ATOMIC_TEST_ONLY) { |
| 1472 | ret = drm_atomic_check_only(state); | 1557 | ret = drm_atomic_check_only(state); |
| 1473 | } else if (arg->flags & DRM_MODE_ATOMIC_NONBLOCK) { | 1558 | } else if (arg->flags & DRM_MODE_ATOMIC_NONBLOCK) { |
| @@ -37,13 +37,12 @@ · 36 unmodified lines | |||
| 37 | #include <drm/drm_print.h> | 37 | #include <drm/drm_print.h> |
| 38 | 38 | ||
| 39 | #include "drm_internal.h" | 39 | #include "drm_internal.h" |
| 40 | #include "drm_legacy.h" | ||
| 41 | 40 | ||
| 42 | /** | 41 | /** |
| 43 | * DOC: master and authentication | 42 | * DOC: master and authentication |
| 44 | * | 43 | * |
| 45 | * &struct drm_master is used to track groups of clients with open | 44 | * &struct drm_master is used to track groups of clients with open |
| 46 | * primary/legacy device nodes. For every &struct drm_file which has had at | 45 | * primary device nodes. For every &struct drm_file which has had at |
| 47 | * least once successfully became the device master (either through the | 46 | * least once successfully became the device master (either through the |
| 48 | * SET_MASTER IOCTL, or implicitly through opening the primary device node when | 47 | * SET_MASTER IOCTL, or implicitly through opening the primary device node when |
| 49 | * no one else is the current master that time) there exists one &drm_master. | 48 | * no one else is the current master that time) there exists one &drm_master. |
| @@ -139,7 +138,6 @@ struct drm_master *drm_master_create(struct drm_device *dev) · 89 unmodified lines | |||
| 139 | return NULL; | 138 | return NULL; |
| 140 | 139 | ||
| 141 | kref_init(&master->refcount); | 140 | kref_init(&master->refcount); |
| 142 | drm_master_legacy_init(master); | ||
| 143 | idr_init_base(&master->magic_map, 1); | 141 | idr_init_base(&master->magic_map, 1); |
| 144 | master->dev = dev; | 142 | master->dev = dev; |
| 145 | 143 | ||
| @@ -365,8 +363,6 @@ void drm_master_release(struct drm_file *file_priv) · 219 unmodified lines | |||
| 365 | if (!drm_is_current_master_locked(file_priv)) | 363 | if (!drm_is_current_master_locked(file_priv)) |
| 366 | goto out; | 364 | goto out; |
| 367 | 365 | ||
| 368 | drm_legacy_lock_master_cleanup(dev, master); | ||
| 369 | |||
| 370 | if (dev->master == file_priv->master) | 366 | if (dev->master == file_priv->master) |
| 371 | drm_drop_master(dev, file_priv); | 367 | drm_drop_master(dev, file_priv); |
| 372 | out: | 368 | out: |
| @@ -429,8 +425,6 @@ static void drm_master_destroy(struct kref *kref) · 56 unmodified lines | |||
| 429 | if (drm_core_check_feature(dev, DRIVER_MODESET)) | 425 | if (drm_core_check_feature(dev, DRIVER_MODESET)) |
| 430 | drm_lease_destroy(master); | 426 | drm_lease_destroy(master); |
| 431 | 427 | ||
| 432 | drm_legacy_master_rmmaps(dev, master); | ||
| 433 | |||
| 434 | idr_destroy(&master->magic_map); | 428 | idr_destroy(&master->magic_map); |
| 435 | idr_destroy(&master->leases); | 429 | idr_destroy(&master->leases); |
| 436 | idr_destroy(&master->lessee_idr); | 430 | idr_destroy(&master->lessee_idr); |
| @@ -1347,50 +1347,6 @@ struct drm_bridge *of_drm_find_bridge(struct device_node *np) · 1346 unmodified lines | |||
| 1347 | EXPORT_SYMBOL(of_drm_find_bridge); | 1347 | EXPORT_SYMBOL(of_drm_find_bridge); |
| 1348 | #endif | 1348 | #endif |
| 1349 | 1349 | ||
| 1350 | #ifdef CONFIG_DEBUG_FS | ||
| 1351 | static int drm_bridge_chains_info(struct seq_file *m, void *data) | ||
| 1352 | { | ||
| 1353 | struct drm_debugfs_entry *entry = m->private; | ||
| 1354 | struct drm_device *dev = entry->dev; | ||
| 1355 | struct drm_printer p = drm_seq_file_printer(m); | ||
| 1356 | struct drm_mode_config *config = &dev->mode_config; | ||
| 1357 | struct drm_encoder *encoder; | ||
| 1358 | unsigned int bridge_idx = 0; | ||
| 1359 | |||
| 1360 | list_for_each_entry(encoder, &config->encoder_list, head) { | ||
| 1361 | struct drm_bridge *bridge; | ||
| 1362 | |||
| 1363 | drm_printf(&p, "encoder[%u]\n", encoder->base.id); | ||
| 1364 | |||
| 1365 | drm_for_each_bridge_in_chain(encoder, bridge) { | ||
| 1366 | drm_printf(&p, "\tbridge[%u] type: %u, ops: %#x", | ||
| 1367 | bridge_idx, bridge->type, bridge->ops); | ||
| 1368 | |||
| 1369 | #ifdef CONFIG_OF | ||
| 1370 | if (bridge->of_node) | ||
| 1371 | drm_printf(&p, ", OF: %pOFfc", bridge->of_node); | ||
| 1372 | #endif | ||
| 1373 | |||
| 1374 | drm_printf(&p, "\n"); | ||
| 1375 | |||
| 1376 | bridge_idx++; | ||
| 1377 | } | ||
| 1378 | } | ||
| 1379 | |||
| 1380 | return 0; | ||
| 1381 | } | ||
| 1382 | |||
| 1383 | static const struct drm_debugfs_info drm_bridge_debugfs_list[] = { | ||
| 1384 | { "bridge_chains", drm_bridge_chains_info, 0 }, | ||
| 1385 | }; | ||
| 1386 | |||
| 1387 | void drm_bridge_debugfs_init(struct drm_device *dev) | ||
| 1388 | { | ||
| 1389 | drm_debugfs_add_files(dev, drm_bridge_debugfs_list, | ||
| 1390 | ARRAY_SIZE(drm_bridge_debugfs_list)); | ||
| 1391 | } | ||
| 1392 | #endif | ||
| 1393 | |||
| 1394 | MODULE_AUTHOR("Ajay Kumar <ajaykumar.rs@samsung.com>"); | 1350 | MODULE_AUTHOR("Ajay Kumar <ajaykumar.rs@samsung.com>"); |
| 1395 | MODULE_DESCRIPTION("DRM bridge infrastructure"); | 1351 | MODULE_DESCRIPTION("DRM bridge infrastructure"); |
| 1396 | MODULE_LICENSE("GPL and additional rights"); | 1352 | MODULE_LICENSE("GPL and additional rights"); |
| @@ -198,12 +198,6 @@ static void drm_bridge_connector_destroy(struct drm_connector *connector) · 197 unmodified lines | |||
| 198 | struct drm_bridge_connector *bridge_connector = | 198 | struct drm_bridge_connector *bridge_connector = |
| 199 | to_drm_bridge_connector(connector); | 199 | to_drm_bridge_connector(connector); |
| 200 | 200 | ||
| 201 | if (bridge_connector->bridge_hpd) { | ||
| 202 | struct drm_bridge *hpd = bridge_connector->bridge_hpd; | ||
| 203 | |||
| 204 | drm_bridge_hpd_disable(hpd); | ||
| 205 | } | ||
| 206 | |||
| 207 | drm_connector_unregister(connector); | 201 | drm_connector_unregister(connector); |
| 208 | drm_connector_cleanup(connector); | 202 | drm_connector_cleanup(connector); |
| 209 | 203 | ||
| @@ -1,1627 +0,0 @@ | |||
| 1 | /* | ||
| 2 | * Legacy: Generic DRM Buffer Management | ||
| 3 | * | ||
| 4 | * Copyright 1999, 2000 Precision Insight, Inc., Cedar Park, Texas. | ||
| 5 | * Copyright 2000 VA Linux Systems, Inc., Sunnyvale, California. | ||
| 6 | * All Rights Reserved. | ||
| 7 | * | ||
| 8 | * Author: Rickard E. (Rik) Faith <faith@valinux.com> | ||
| 9 | * Author: Gareth Hughes <gareth@valinux.com> | ||
| 10 | * | ||
| 11 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 12 | * copy of this software and associated documentation files (the "Software"), | ||
| 13 | * to deal in the Software without restriction, including without limitation | ||
| 14 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 15 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 16 | * Software is furnished to do so, subject to the following conditions: | ||
| 17 | * | ||
| 18 | * The above copyright notice and this permission notice (including the next | ||
| 19 | * paragraph) shall be included in all copies or substantial portions of the | ||
| 20 | * Software. | ||
| 21 | * | ||
| 22 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 23 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 24 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 25 | * VA LINUX SYSTEMS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR | ||
| 26 | * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | ||
| 27 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | ||
| 28 | * OTHER DEALINGS IN THE SOFTWARE. | ||
| 29 | */ | ||
| 30 | |||
| 31 | #include <linux/export.h> | ||
| 32 | #include <linux/log2.h> | ||
| 33 | #include <linux/mm.h> | ||
| 34 | #include <linux/mman.h> | ||
| 35 | #include <linux/nospec.h> | ||
| 36 | #include <linux/pci.h> | ||
| 37 | #include <linux/slab.h> | ||
| 38 | #include <linux/uaccess.h> | ||
| 39 | #include <linux/vmalloc.h> | ||
| 40 | |||
| 41 | #include <asm/shmparam.h> | ||
| 42 | |||
| 43 | #include <drm/drm_device.h> | ||
| 44 | #include <drm/drm_drv.h> | ||
| 45 | #include <drm/drm_file.h> | ||
| 46 | #include <drm/drm_print.h> | ||
| 47 | |||
| 48 | #include "drm_legacy.h" | ||
| 49 | |||
| 50 | |||
| 51 | static struct drm_map_list *drm_find_matching_map(struct drm_device *dev, | ||
| 52 | struct drm_local_map *map) | ||
| 53 | { | ||
| 54 | struct drm_map_list *entry; | ||
| 55 | |||
| 56 | list_for_each_entry(entry, &dev->maplist, head) { | ||
| 57 | /* | ||
| 58 | * Because the kernel-userspace ABI is fixed at a 32-bit offset | ||
| 59 | * while PCI resources may live above that, we only compare the | ||
| 60 | * lower 32 bits of the map offset for maps of type | ||
| 61 | * _DRM_FRAMEBUFFER or _DRM_REGISTERS. | ||
| 62 | * It is assumed that if a driver have more than one resource | ||
| 63 | * of each type, the lower 32 bits are different. | ||
| 64 | */ | ||
| 65 | if (!entry->map || | ||
| 66 | map->type != entry->map->type || | ||
| 67 | entry->master != dev->master) | ||
| 68 | continue; | ||
| 69 | switch (map->type) { | ||
| 70 | case _DRM_SHM: | ||
| 71 | if (map->flags != _DRM_CONTAINS_LOCK) | ||
| 72 | break; | ||
| 73 | return entry; | ||
| 74 | case _DRM_REGISTERS: | ||
| 75 | case _DRM_FRAME_BUFFER: | ||
| 76 | if ((entry->map->offset & 0xffffffff) == | ||
| 77 | (map->offset & 0xffffffff)) | ||
| 78 | return entry; | ||
| 79 | break; | ||
| 80 | default: /* Make gcc happy */ | ||
| 81 | break; | ||
| 82 | } | ||
| 83 | if (entry->map->offset == map->offset) | ||
| 84 | return entry; | ||
| 85 | } | ||
| 86 | |||
| 87 | return NULL; | ||
| 88 | } | ||
| 89 | |||
| 90 | static int drm_map_handle(struct drm_device *dev, struct drm_hash_item *hash, | ||
| 91 | unsigned long user_token, int hashed_handle, int shm) | ||
| 92 | { | ||
| 93 | int use_hashed_handle, shift; | ||
| 94 | unsigned long add; | ||
| 95 | |||
| 96 | #if (BITS_PER_LONG == 64) | ||
| 97 | use_hashed_handle = ((user_token & 0xFFFFFFFF00000000UL) || hashed_handle); | ||
| 98 | #elif (BITS_PER_LONG == 32) | ||
| 99 | use_hashed_handle = hashed_handle; | ||
| 100 | #else | ||
| 101 | #error Unsupported long size. Neither 64 nor 32 bits. | ||
| 102 | #endif | ||
| 103 | |||
| 104 | if (!use_hashed_handle) { | ||
| 105 | int ret; | ||
| 106 | |||
| 107 | hash->key = user_token >> PAGE_SHIFT; | ||
| 108 | ret = drm_ht_insert_item(&dev->map_hash, hash); | ||
| 109 | if (ret != -EINVAL) | ||
| 110 | return ret; | ||
| 111 | } | ||
| 112 | |||
| 113 | shift = 0; | ||
| 114 | add = DRM_MAP_HASH_OFFSET >> PAGE_SHIFT; | ||
| 115 | if (shm && (SHMLBA > PAGE_SIZE)) { | ||
| 116 | int bits = ilog2(SHMLBA >> PAGE_SHIFT) + 1; | ||
| 117 | |||
| 118 | /* For shared memory, we have to preserve the SHMLBA | ||
| 119 | * bits of the eventual vma->vm_pgoff value during | ||
| 120 | * mmap(). Otherwise we run into cache aliasing problems | ||
| 121 | * on some platforms. On these platforms, the pgoff of | ||
| 122 | * a mmap() request is used to pick a suitable virtual | ||
| 123 | * address for the mmap() region such that it will not | ||
| 124 | * cause cache aliasing problems. | ||
| 125 | * | ||
| 126 | * Therefore, make sure the SHMLBA relevant bits of the | ||
| 127 | * hash value we use are equal to those in the original | ||
| 128 | * kernel virtual address. | ||
| 129 | */ | ||
| 130 | shift = bits; | ||
| 131 | add |= ((user_token >> PAGE_SHIFT) & ((1UL << bits) - 1UL)); | ||
| 132 | } | ||
| 133 | |||
| 134 | return drm_ht_just_insert_please(&dev->map_hash, hash, | ||
| 135 | user_token, 32 - PAGE_SHIFT - 3, | ||
| 136 | shift, add); | ||
| 137 | } | ||
| 138 | |||
| 139 | /* | ||
| 140 | * Core function to create a range of memory available for mapping by a | ||
| 141 | * non-root process. | ||
| 142 | * | ||
| 143 | * Adjusts the memory offset to its absolute value according to the mapping | ||
| 144 | * type. Adds the map to the map list drm_device::maplist. Adds MTRR's where | ||
| 145 | * applicable and if supported by the kernel. | ||
| 146 | */ | ||
| 147 | static int drm_addmap_core(struct drm_device *dev, resource_size_t offset, | ||
| 148 | unsigned int size, enum drm_map_type type, | ||
| 149 | enum drm_map_flags flags, | ||
| 150 | struct drm_map_list **maplist) | ||
| 151 | { | ||
| 152 | struct drm_local_map *map; | ||
| 153 | struct drm_map_list *list; | ||
| 154 | unsigned long user_token; | ||
| 155 | int ret; | ||
| 156 | |||
| 157 | map = kmalloc(sizeof(*map), GFP_KERNEL); | ||
| 158 | if (!map) | ||
| 159 | return -ENOMEM; | ||
| 160 | |||
| 161 | map->offset = offset; | ||
| 162 | map->size = size; | ||
| 163 | map->flags = flags; | ||
| 164 | map->type = type; | ||
| 165 | |||
| 166 | /* Only allow shared memory to be removable since we only keep enough | ||
| 167 | * book keeping information about shared memory to allow for removal | ||
| 168 | * when processes fork. | ||
| 169 | */ | ||
| 170 | if ((map->flags & _DRM_REMOVABLE) && map->type != _DRM_SHM) { | ||
| 171 | kfree(map); | ||
| 172 | return -EINVAL; | ||
| 173 | } | ||
| 174 | DRM_DEBUG("offset = 0x%08llx, size = 0x%08lx, type = %d\n", | ||
| 175 | (unsigned long long)map->offset, map->size, map->type); | ||
| 176 | |||
| 177 | /* page-align _DRM_SHM maps. They are allocated here so there is no security | ||
| 178 | * hole created by that and it works around various broken drivers that use | ||
| 179 | * a non-aligned quantity to map the SAREA. --BenH | ||
| 180 | */ | ||
| 181 | if (map->type == _DRM_SHM) | ||
| 182 | map->size = PAGE_ALIGN(map->size); | ||
| 183 | |||
| 184 | if ((map->offset & (~(resource_size_t)PAGE_MASK)) || (map->size & (~PAGE_MASK))) { | ||
| 185 | kfree(map); | ||
| 186 | return -EINVAL; | ||
| 187 | } | ||
| 188 | map->mtrr = -1; | ||
| 189 | map->handle = NULL; | ||
| 190 | |||
| 191 | switch (map->type) { | ||
| 192 | case _DRM_REGISTERS: | ||
| 193 | case _DRM_FRAME_BUFFER: | ||
| 194 | #if !defined(__sparc__) && !defined(__alpha__) && !defined(__ia64__) && !defined(__powerpc64__) && !defined(__x86_64__) && !defined(__arm__) | ||
| 195 | if (map->offset + (map->size-1) < map->offset || | ||
| 196 | map->offset < virt_to_phys(high_memory)) { | ||
| 197 | kfree(map); | ||
| 198 | return -EINVAL; | ||
| 199 | } | ||
| 200 | #endif | ||
| 201 | /* Some drivers preinitialize some maps, without the X Server | ||
| 202 | * needing to be aware of it. Therefore, we just return success | ||
| 203 | * when the server tries to create a duplicate map. | ||
| 204 | */ | ||
| 205 | list = drm_find_matching_map(dev, map); | ||
| 206 | if (list != NULL) { | ||
| 207 | if (list->map->size != map->size) { | ||
| 208 | DRM_DEBUG("Matching maps of type %d with " | ||
| 209 | "mismatched sizes, (%ld vs %ld)\n", | ||
| 210 | map->type, map->size, | ||
| 211 | list->map->size); | ||
| 212 | list->map->size = map->size; | ||
| 213 | } | ||
| 214 | |||
| 215 | kfree(map); | ||
| 216 | *maplist = list; | ||
| 217 | return 0; | ||
| 218 | } | ||
| 219 | |||
| 220 | if (map->type == _DRM_FRAME_BUFFER || | ||
| 221 | (map->flags & _DRM_WRITE_COMBINING)) { | ||
| 222 | map->mtrr = | ||
| 223 | arch_phys_wc_add(map->offset, map->size); | ||
| 224 | } | ||
| 225 | if (map->type == _DRM_REGISTERS) { | ||
| 226 | if (map->flags & _DRM_WRITE_COMBINING) | ||
| 227 | map->handle = ioremap_wc(map->offset, | ||
| 228 | map->size); | ||
| 229 | else | ||
| 230 | map->handle = ioremap(map->offset, map->size); | ||
| 231 | if (!map->handle) { | ||
| 232 | kfree(map); | ||
| 233 | return -ENOMEM; | ||
| 234 | } | ||
| 235 | } | ||
| 236 | |||
| 237 | break; | ||
| 238 | case _DRM_SHM: | ||
| 239 | list = drm_find_matching_map(dev, map); | ||
| 240 | if (list != NULL) { | ||
| 241 | if (list->map->size != map->size) { | ||
| 242 | DRM_DEBUG("Matching maps of type %d with " | ||
| 243 | "mismatched sizes, (%ld vs %ld)\n", | ||
| 244 | map->type, map->size, list->map->size); | ||
| 245 | list->map->size = map->size; | ||
| 246 | } | ||
| 247 | |||
| 248 | kfree(map); | ||
| 249 | *maplist = list; | ||
| 250 | return 0; | ||
| 251 | } | ||
| 252 | map->handle = vmalloc_user(map->size); | ||
| 253 | DRM_DEBUG("%lu %d %p\n", | ||
| 254 | map->size, order_base_2(map->size), map->handle); | ||
| 255 | if (!map->handle) { | ||
| 256 | kfree(map); | ||
| 257 | return -ENOMEM; | ||
| 258 | } | ||
| 259 | map->offset = (unsigned long)map->handle; | ||
| 260 | if (map->flags & _DRM_CONTAINS_LOCK) { | ||
| 261 | /* Prevent a 2nd X Server from creating a 2nd lock */ | ||
| 262 | if (dev->master->lock.hw_lock != NULL) { | ||
| 263 | vfree(map->handle); | ||
| 264 | kfree(map); | ||
| 265 | return -EBUSY; | ||
| 266 | } | ||
| 267 | dev->sigdata.lock = dev->master->lock.hw_lock = map->handle; /* Pointer to lock */ | ||
| 268 | } | ||
| 269 | break; | ||
| 270 | case _DRM_AGP: { | ||
| 271 | struct drm_agp_mem *entry; | ||
| 272 | int valid = 0; | ||
| 273 | |||
| 274 | if (!dev->agp) { | ||
| 275 | kfree(map); | ||
| 276 | return -EINVAL; | ||
| 277 | } | ||
| 278 | #ifdef __alpha__ | ||
| 279 | map->offset += dev->hose->mem_space->start; | ||
| 280 | #endif | ||
| 281 | /* In some cases (i810 driver), user space may have already | ||
| 282 | * added the AGP base itself, because dev->agp->base previously | ||
| 283 | * only got set during AGP enable. So, only add the base | ||
| 284 | * address if the map's offset isn't already within the | ||
| 285 | * aperture. | ||
| 286 | */ | ||
| 287 | if (map->offset < dev->agp->base || | ||
| 288 | map->offset > dev->agp->base + | ||
| 289 | dev->agp->agp_info.aper_size * 1024 * 1024 - 1) { | ||
| 290 | map->offset += dev->agp->base; | ||
| 291 | } | ||
| 292 | map->mtrr = dev->agp->agp_mtrr; /* for getmap */ | ||
| 293 | |||
| 294 | /* This assumes the DRM is in total control of AGP space. | ||
| 295 | * It's not always the case as AGP can be in the control | ||
| 296 | * of user space (i.e. i810 driver). So this loop will get | ||
| 297 | * skipped and we double check that dev->agp->memory is | ||
| 298 | * actually set as well as being invalid before EPERM'ing | ||
| 299 | */ | ||
| 300 | list_for_each_entry(entry, &dev->agp->memory, head) { | ||
| 301 | if ((map->offset >= entry->bound) && | ||
| 302 | (map->offset + map->size <= entry->bound + entry->pages * PAGE_SIZE)) { | ||
| 303 | valid = 1; | ||
| 304 | break; | ||
| 305 | } | ||
| 306 | } | ||
| 307 | if (!list_empty(&dev->agp->memory) && !valid) { | ||
| 308 | kfree(map); | ||
| 309 | return -EPERM; | ||
| 310 | } | ||
| 311 | DRM_DEBUG("AGP offset = 0x%08llx, size = 0x%08lx\n", | ||
| 312 | (unsigned long long)map->offset, map->size); | ||
| 313 | |||
| 314 | break; | ||
| 315 | } | ||
| 316 | case _DRM_SCATTER_GATHER: | ||
| 317 | if (!dev->sg) { | ||
| 318 | kfree(map); | ||
| 319 | return -EINVAL; | ||
| 320 | } | ||
| 321 | map->offset += (unsigned long)dev->sg->virtual; | ||
| 322 | break; | ||
| 323 | case _DRM_CONSISTENT: | ||
| 324 | /* dma_addr_t is 64bit on i386 with CONFIG_HIGHMEM64G, | ||
| 325 | * As we're limiting the address to 2^32-1 (or less), | ||
| 326 | * casting it down to 32 bits is no problem, but we | ||
| 327 | * need to point to a 64bit variable first. | ||
| 328 | */ | ||
| 329 | map->handle = dma_alloc_coherent(dev->dev, | ||
| 330 | map->size, | ||
| 331 | &map->offset, | ||
| 332 | GFP_KERNEL); | ||
| 333 | if (!map->handle) { | ||
| 334 | kfree(map); | ||
| 335 | return -ENOMEM; | ||
| 336 | } | ||
| 337 | break; | ||
| 338 | default: | ||
| 339 | kfree(map); | ||
| 340 | return -EINVAL; | ||
| 341 | } | ||
| 342 | |||
| 343 | list = kzalloc(sizeof(*list), GFP_KERNEL); | ||
| 344 | if (!list) { | ||
| 345 | if (map->type == _DRM_REGISTERS) | ||
| 346 | iounmap(map->handle); | ||
| 347 | kfree(map); | ||
| 348 | return -EINVAL; | ||
| 349 | } | ||
| 350 | list->map = map; | ||
| 351 | |||
| 352 | mutex_lock(&dev->struct_mutex); | ||
| 353 | list_add(&list->head, &dev->maplist); | ||
| 354 | |||
| 355 | /* Assign a 32-bit handle */ | ||
| 356 | /* We do it here so that dev->struct_mutex protects the increment */ | ||
| 357 | user_token = (map->type == _DRM_SHM) ? (unsigned long)map->handle : | ||
| 358 | map->offset; | ||
| 359 | ret = drm_map_handle(dev, &list->hash, user_token, 0, | ||
| 360 | (map->type == _DRM_SHM)); | ||
| 361 | if (ret) { | ||
| 362 | if (map->type == _DRM_REGISTERS) | ||
| 363 | iounmap(map->handle); | ||
| 364 | kfree(map); | ||
| 365 | kfree(list); | ||
| 366 | mutex_unlock(&dev->struct_mutex); | ||
| 367 | return ret; | ||
| 368 | } | ||
| 369 | |||
| 370 | list->user_token = list->hash.key << PAGE_SHIFT; | ||
| 371 | mutex_unlock(&dev->struct_mutex); | ||
| 372 | |||
| 373 | if (!(map->flags & _DRM_DRIVER)) | ||
| 374 | list->master = dev->master; | ||
| 375 | *maplist = list; | ||
| 376 | return 0; | ||
| 377 | } | ||
| 378 | |||
| 379 | int drm_legacy_addmap(struct drm_device *dev, resource_size_t offset, | ||
| 380 | unsigned int size, enum drm_map_type type, | ||
| 381 | enum drm_map_flags flags, struct drm_local_map **map_ptr) | ||
| 382 | { | ||
| 383 | struct drm_map_list *list; | ||
| 384 | int rc; | ||
| 385 | |||
| 386 | rc = drm_addmap_core(dev, offset, size, type, flags, &list); | ||
| 387 | if (!rc) | ||
| 388 | *map_ptr = list->map; | ||
| 389 | return rc; | ||
| 390 | } | ||
| 391 | EXPORT_SYMBOL(drm_legacy_addmap); | ||
| 392 | |||
| 393 | struct drm_local_map *drm_legacy_findmap(struct drm_device *dev, | ||
| 394 | unsigned int token) | ||
| 395 | { | ||
| 396 | struct drm_map_list *_entry; | ||
| 397 | |||
| 398 | list_for_each_entry(_entry, &dev->maplist, head) | ||
| 399 | if (_entry->user_token == token) | ||
| 400 | return _entry->map; | ||
| 401 | return NULL; | ||
| 402 | } | ||
| 403 | EXPORT_SYMBOL(drm_legacy_findmap); | ||
| 404 | |||
| 405 | /* | ||
| 406 | * Ioctl to specify a range of memory that is available for mapping by a | ||
| 407 | * non-root process. | ||
| 408 | * | ||
| 409 | * \param inode device inode. | ||
| 410 | * \param file_priv DRM file private. | ||
| 411 | * \param cmd command. | ||
| 412 | * \param arg pointer to a drm_map structure. | ||
| 413 | * \return zero on success or a negative value on error. | ||
| 414 | * | ||
| 415 | */ | ||
| 416 | int drm_legacy_addmap_ioctl(struct drm_device *dev, void *data, | ||
| 417 | struct drm_file *file_priv) | ||
| 418 | { | ||
| 419 | struct drm_map *map = data; | ||
| 420 | struct drm_map_list *maplist; | ||
| 421 | int err; | ||
| 422 | |||
| 423 | if (!(capable(CAP_SYS_ADMIN) || map->type == _DRM_AGP || map->type == _DRM_SHM)) | ||
| 424 | return -EPERM; | ||
| 425 | |||
| 426 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 427 | return -EOPNOTSUPP; | ||
| 428 | |||
| 429 | err = drm_addmap_core(dev, map->offset, map->size, map->type, | ||
| 430 | map->flags, &maplist); | ||
| 431 | |||
| 432 | if (err) | ||
| 433 | return err; | ||
| 434 | |||
| 435 | /* avoid a warning on 64-bit, this casting isn't very nice, but the API is set so too late */ | ||
| 436 | map->handle = (void *)(unsigned long)maplist->user_token; | ||
| 437 | |||
| 438 | /* | ||
| 439 | * It appears that there are no users of this value whatsoever -- | ||
| 440 | * drmAddMap just discards it. Let's not encourage its use. | ||
| 441 | * (Keeping drm_addmap_core's returned mtrr value would be wrong -- | ||
| 442 | * it's not a real mtrr index anymore.) | ||
| 443 | */ | ||
| 444 | map->mtrr = -1; | ||
| 445 | |||
| 446 | return 0; | ||
| 447 | } | ||
| 448 | |||
| 449 | /* | ||
| 450 | * Get a mapping information. | ||
| 451 | * | ||
| 452 | * \param inode device inode. | ||
| 453 | * \param file_priv DRM file private. | ||
| 454 | * \param cmd command. | ||
| 455 | * \param arg user argument, pointing to a drm_map structure. | ||
| 456 | * | ||
| 457 | * \return zero on success or a negative number on failure. | ||
| 458 | * | ||
| 459 | * Searches for the mapping with the specified offset and copies its information | ||
| 460 | * into userspace | ||
| 461 | */ | ||
| 462 | int drm_legacy_getmap_ioctl(struct drm_device *dev, void *data, | ||
| 463 | struct drm_file *file_priv) | ||
| 464 | { | ||
| 465 | struct drm_map *map = data; | ||
| 466 | struct drm_map_list *r_list = NULL; | ||
| 467 | struct list_head *list; | ||
| 468 | int idx; | ||
| 469 | int i; | ||
| 470 | |||
| 471 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 472 | return -EOPNOTSUPP; | ||
| 473 | |||
| 474 | idx = map->offset; | ||
| 475 | if (idx < 0) | ||
| 476 | return -EINVAL; | ||
| 477 | |||
| 478 | i = 0; | ||
| 479 | mutex_lock(&dev->struct_mutex); | ||
| 480 | list_for_each(list, &dev->maplist) { | ||
| 481 | if (i == idx) { | ||
| 482 | r_list = list_entry(list, struct drm_map_list, head); | ||
| 483 | break; | ||
| 484 | } | ||
| 485 | i++; | ||
| 486 | } | ||
| 487 | if (!r_list || !r_list->map) { | ||
| 488 | mutex_unlock(&dev->struct_mutex); | ||
| 489 | return -EINVAL; | ||
| 490 | } | ||
| 491 | |||
| 492 | map->offset = r_list->map->offset; | ||
| 493 | map->size = r_list->map->size; | ||
| 494 | map->type = r_list->map->type; | ||
| 495 | map->flags = r_list->map->flags; | ||
| 496 | map->handle = (void *)(unsigned long) r_list->user_token; | ||
| 497 | map->mtrr = arch_phys_wc_index(r_list->map->mtrr); | ||
| 498 | |||
| 499 | mutex_unlock(&dev->struct_mutex); | ||
| 500 | |||
| 501 | return 0; | ||
| 502 | } | ||
| 503 | |||
| 504 | /* | ||
| 505 | * Remove a map private from list and deallocate resources if the mapping | ||
| 506 | * isn't in use. | ||
| 507 | * | ||
| 508 | * Searches the map on drm_device::maplist, removes it from the list, see if | ||
| 509 | * it's being used, and free any associated resource (such as MTRR's) if it's not | ||
| 510 | * being on use. | ||
| 511 | * | ||
| 512 | * \sa drm_legacy_addmap | ||
| 513 | */ | ||
| 514 | int drm_legacy_rmmap_locked(struct drm_device *dev, struct drm_local_map *map) | ||
| 515 | { | ||
| 516 | struct drm_map_list *r_list = NULL, *list_t; | ||
| 517 | int found = 0; | ||
| 518 | struct drm_master *master; | ||
| 519 | |||
| 520 | /* Find the list entry for the map and remove it */ | ||
| 521 | list_for_each_entry_safe(r_list, list_t, &dev->maplist, head) { | ||
| 522 | if (r_list->map == map) { | ||
| 523 | master = r_list->master; | ||
| 524 | list_del(&r_list->head); | ||
| 525 | drm_ht_remove_key(&dev->map_hash, | ||
| 526 | r_list->user_token >> PAGE_SHIFT); | ||
| 527 | kfree(r_list); | ||
| 528 | found = 1; | ||
| 529 | break; | ||
| 530 | } | ||
| 531 | } | ||
| 532 | |||
| 533 | if (!found) | ||
| 534 | return -EINVAL; | ||
| 535 | |||
| 536 | switch (map->type) { | ||
| 537 | case _DRM_REGISTERS: | ||
| 538 | iounmap(map->handle); | ||
| 539 | fallthrough; | ||
| 540 | case _DRM_FRAME_BUFFER: | ||
| 541 | arch_phys_wc_del(map->mtrr); | ||
| 542 | break; | ||
| 543 | case _DRM_SHM: | ||
| 544 | vfree(map->handle); | ||
| 545 | if (master) { | ||
| 546 | if (dev->sigdata.lock == master->lock.hw_lock) | ||
| 547 | dev->sigdata.lock = NULL; | ||
| 548 | master->lock.hw_lock = NULL; /* SHM removed */ | ||
| 549 | master->lock.file_priv = NULL; | ||
| 550 | wake_up_interruptible_all(&master->lock.lock_queue); | ||
| 551 | } | ||
| 552 | break; | ||
| 553 | case _DRM_AGP: | ||
| 554 | case _DRM_SCATTER_GATHER: | ||
| 555 | break; | ||
| 556 | case _DRM_CONSISTENT: | ||
| 557 | dma_free_coherent(dev->dev, | ||
| 558 | map->size, | ||
| 559 | map->handle, | ||
| 560 | map->offset); | ||
| 561 | break; | ||
| 562 | } | ||
| 563 | kfree(map); | ||
| 564 | |||
| 565 | return 0; | ||
| 566 | } | ||
| 567 | EXPORT_SYMBOL(drm_legacy_rmmap_locked); | ||
| 568 | |||
| 569 | void drm_legacy_rmmap(struct drm_device *dev, struct drm_local_map *map) | ||
| 570 | { | ||
| 571 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 572 | return; | ||
| 573 | |||
| 574 | mutex_lock(&dev->struct_mutex); | ||
| 575 | drm_legacy_rmmap_locked(dev, map); | ||
| 576 | mutex_unlock(&dev->struct_mutex); | ||
| 577 | } | ||
| 578 | EXPORT_SYMBOL(drm_legacy_rmmap); | ||
| 579 | |||
| 580 | void drm_legacy_master_rmmaps(struct drm_device *dev, struct drm_master *master) | ||
| 581 | { | ||
| 582 | struct drm_map_list *r_list, *list_temp; | ||
| 583 | |||
| 584 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 585 | return; | ||
| 586 | |||
| 587 | mutex_lock(&dev->struct_mutex); | ||
| 588 | list_for_each_entry_safe(r_list, list_temp, &dev->maplist, head) { | ||
| 589 | if (r_list->master == master) { | ||
| 590 | drm_legacy_rmmap_locked(dev, r_list->map); | ||
| 591 | r_list = NULL; | ||
| 592 | } | ||
| 593 | } | ||
| 594 | mutex_unlock(&dev->struct_mutex); | ||
| 595 | } | ||
| 596 | |||
| 597 | void drm_legacy_rmmaps(struct drm_device *dev) | ||
| 598 | { | ||
| 599 | struct drm_map_list *r_list, *list_temp; | ||
| 600 | |||
| 601 | list_for_each_entry_safe(r_list, list_temp, &dev->maplist, head) | ||
| 602 | drm_legacy_rmmap(dev, r_list->map); | ||
| 603 | } | ||
| 604 | |||
| 605 | /* The rmmap ioctl appears to be unnecessary. All mappings are torn down on | ||
| 606 | * the last close of the device, and this is necessary for cleanup when things | ||
| 607 | * exit uncleanly. Therefore, having userland manually remove mappings seems | ||
| 608 | * like a pointless exercise since they're going away anyway. | ||
| 609 | * | ||
| 610 | * One use case might be after addmap is allowed for normal users for SHM and | ||
| 611 | * gets used by drivers that the server doesn't need to care about. This seems | ||
| 612 | * unlikely. | ||
| 613 | * | ||
| 614 | * \param inode device inode. | ||
| 615 | * \param file_priv DRM file private. | ||
| 616 | * \param cmd command. | ||
| 617 | * \param arg pointer to a struct drm_map structure. | ||
| 618 | * \return zero on success or a negative value on error. | ||
| 619 | */ | ||
| 620 | int drm_legacy_rmmap_ioctl(struct drm_device *dev, void *data, | ||
| 621 | struct drm_file *file_priv) | ||
| 622 | { | ||
| 623 | struct drm_map *request = data; | ||
| 624 | struct drm_local_map *map = NULL; | ||
| 625 | struct drm_map_list *r_list; | ||
| 626 | int ret; | ||
| 627 | |||
| 628 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 629 | return -EOPNOTSUPP; | ||
| 630 | |||
| 631 | mutex_lock(&dev->struct_mutex); | ||
| 632 | list_for_each_entry(r_list, &dev->maplist, head) { | ||
| 633 | if (r_list->map && | ||
| 634 | r_list->user_token == (unsigned long)request->handle && | ||
| 635 | r_list->map->flags & _DRM_REMOVABLE) { | ||
| 636 | map = r_list->map; | ||
| 637 | break; | ||
| 638 | } | ||
| 639 | } | ||
| 640 | |||
| 641 | /* List has wrapped around to the head pointer, or it's empty we didn't | ||
| 642 | * find anything. | ||
| 643 | */ | ||
| 644 | if (list_empty(&dev->maplist) || !map) { | ||
| 645 | mutex_unlock(&dev->struct_mutex); | ||
| 646 | return -EINVAL; | ||
| 647 | } | ||
| 648 | |||
| 649 | /* Register and framebuffer maps are permanent */ | ||
| 650 | if ((map->type == _DRM_REGISTERS) || (map->type == _DRM_FRAME_BUFFER)) { | ||
| 651 | mutex_unlock(&dev->struct_mutex); | ||
| 652 | return 0; | ||
| 653 | } | ||
| 654 | |||
| 655 | ret = drm_legacy_rmmap_locked(dev, map); | ||
| 656 | |||
| 657 | mutex_unlock(&dev->struct_mutex); | ||
| 658 | |||
| 659 | return ret; | ||
| 660 | } | ||
| 661 | |||
| 662 | /* | ||
| 663 | * Cleanup after an error on one of the addbufs() functions. | ||
| 664 | * | ||
| 665 | * \param dev DRM device. | ||
| 666 | * \param entry buffer entry where the error occurred. | ||
| 667 | * | ||
| 668 | * Frees any pages and buffers associated with the given entry. | ||
| 669 | */ | ||
| 670 | static void drm_cleanup_buf_error(struct drm_device *dev, | ||
| 671 | struct drm_buf_entry *entry) | ||
| 672 | { | ||
| 673 | drm_dma_handle_t *dmah; | ||
| 674 | int i; | ||
| 675 | |||
| 676 | if (entry->seg_count) { | ||
| 677 | for (i = 0; i < entry->seg_count; i++) { | ||
| 678 | if (entry->seglist[i]) { | ||
| 679 | dmah = entry->seglist[i]; | ||
| 680 | dma_free_coherent(dev->dev, | ||
| 681 | dmah->size, | ||
| 682 | dmah->vaddr, | ||
| 683 | dmah->busaddr); | ||
| 684 | kfree(dmah); | ||
| 685 | } | ||
| 686 | } | ||
| 687 | kfree(entry->seglist); | ||
| 688 | |||
| 689 | entry->seg_count = 0; | ||
| 690 | } | ||
| 691 | |||
| 692 | if (entry->buf_count) { | ||
| 693 | for (i = 0; i < entry->buf_count; i++) { | ||
| 694 | kfree(entry->buflist[i].dev_private); | ||
| 695 | } | ||
| 696 | kfree(entry->buflist); | ||
| 697 | |||
| 698 | entry->buf_count = 0; | ||
| 699 | } | ||
| 700 | } | ||
| 701 | |||
| 702 | #if IS_ENABLED(CONFIG_AGP) | ||
| 703 | /* | ||
| 704 | * Add AGP buffers for DMA transfers. | ||
| 705 | * | ||
| 706 | * \param dev struct drm_device to which the buffers are to be added. | ||
| 707 | * \param request pointer to a struct drm_buf_desc describing the request. | ||
| 708 | * \return zero on success or a negative number on failure. | ||
| 709 | * | ||
| 710 | * After some sanity checks creates a drm_buf structure for each buffer and | ||
| 711 | * reallocates the buffer list of the same size order to accommodate the new | ||
| 712 | * buffers. | ||
| 713 | */ | ||
| 714 | int drm_legacy_addbufs_agp(struct drm_device *dev, | ||
| 715 | struct drm_buf_desc *request) | ||
| 716 | { | ||
| 717 | struct drm_device_dma *dma = dev->dma; | ||
| 718 | struct drm_buf_entry *entry; | ||
| 719 | struct drm_agp_mem *agp_entry; | ||
| 720 | struct drm_buf *buf; | ||
| 721 | unsigned long offset; | ||
| 722 | unsigned long agp_offset; | ||
| 723 | int count; | ||
| 724 | int order; | ||
| 725 | int size; | ||
| 726 | int alignment; | ||
| 727 | int page_order; | ||
| 728 | int total; | ||
| 729 | int byte_count; | ||
| 730 | int i, valid; | ||
| 731 | struct drm_buf **temp_buflist; | ||
| 732 | |||
| 733 | if (!dma) | ||
| 734 | return -EINVAL; | ||
| 735 | |||
| 736 | count = request->count; | ||
| 737 | order = order_base_2(request->size); | ||
| 738 | size = 1 << order; | ||
| 739 | |||
| 740 | alignment = (request->flags & _DRM_PAGE_ALIGN) | ||
| 741 | ? PAGE_ALIGN(size) : size; | ||
| 742 | page_order = order - PAGE_SHIFT > 0 ? order - PAGE_SHIFT : 0; | ||
| 743 | total = PAGE_SIZE << page_order; | ||
| 744 | |||
| 745 | byte_count = 0; | ||
| 746 | agp_offset = dev->agp->base + request->agp_start; | ||
| 747 | |||
| 748 | DRM_DEBUG("count: %d\n", count); | ||
| 749 | DRM_DEBUG("order: %d\n", order); | ||
| 750 | DRM_DEBUG("size: %d\n", size); | ||
| 751 | DRM_DEBUG("agp_offset: %lx\n", agp_offset); | ||
| 752 | DRM_DEBUG("alignment: %d\n", alignment); | ||
| 753 | DRM_DEBUG("page_order: %d\n", page_order); | ||
| 754 | DRM_DEBUG("total: %d\n", total); | ||
| 755 | |||
| 756 | if (order < DRM_MIN_ORDER || order > DRM_MAX_ORDER) | ||
| 757 | return -EINVAL; | ||
| 758 | |||
| 759 | /* Make sure buffers are located in AGP memory that we own */ | ||
| 760 | valid = 0; | ||
| 761 | list_for_each_entry(agp_entry, &dev->agp->memory, head) { | ||
| 762 | if ((agp_offset >= agp_entry->bound) && | ||
| 763 | (agp_offset + total * count <= agp_entry->bound + agp_entry->pages * PAGE_SIZE)) { | ||
| 764 | valid = 1; | ||
| 765 | break; | ||
| 766 | } | ||
| 767 | } | ||
| 768 | if (!list_empty(&dev->agp->memory) && !valid) { | ||
| 769 | DRM_DEBUG("zone invalid\n"); | ||
| 770 | return -EINVAL; | ||
| 771 | } | ||
| 772 | spin_lock(&dev->buf_lock); | ||
| 773 | if (dev->buf_use) { | ||
| 774 | spin_unlock(&dev->buf_lock); | ||
| 775 | return -EBUSY; | ||
| 776 | } | ||
| 777 | atomic_inc(&dev->buf_alloc); | ||
| 778 | spin_unlock(&dev->buf_lock); | ||
| 779 | |||
| 780 | mutex_lock(&dev->struct_mutex); | ||
| 781 | entry = &dma->bufs[order]; | ||
| 782 | if (entry->buf_count) { | ||
| 783 | mutex_unlock(&dev->struct_mutex); | ||
| 784 | atomic_dec(&dev->buf_alloc); | ||
| 785 | return -ENOMEM; /* May only call once for each order */ | ||
| 786 | } | ||
| 787 | |||
| 788 | if (count < 0 || count > 4096) { | ||
| 789 | mutex_unlock(&dev->struct_mutex); | ||
| 790 | atomic_dec(&dev->buf_alloc); | ||
| 791 | return -EINVAL; | ||
| 792 | } | ||
| 793 | |||
| 794 | entry->buflist = kcalloc(count, sizeof(*entry->buflist), GFP_KERNEL); | ||
| 795 | if (!entry->buflist) { | ||
| 796 | mutex_unlock(&dev->struct_mutex); | ||
| 797 | atomic_dec(&dev->buf_alloc); | ||
| 798 | return -ENOMEM; | ||
| 799 | } | ||
| 800 | |||
| 801 | entry->buf_size = size; | ||
| 802 | entry->page_order = page_order; | ||
| 803 | |||
| 804 | offset = 0; | ||
| 805 | |||
| 806 | while (entry->buf_count < count) { | ||
| 807 | buf = &entry->buflist[entry->buf_count]; | ||
| 808 | buf->idx = dma->buf_count + entry->buf_count; | ||
| 809 | buf->total = alignment; | ||
| 810 | buf->order = order; | ||
| 811 | buf->used = 0; | ||
| 812 | |||
| 813 | buf->offset = (dma->byte_count + offset); | ||
| 814 | buf->bus_address = agp_offset + offset; | ||
| 815 | buf->address = (void *)(agp_offset + offset); | ||
| 816 | buf->next = NULL; | ||
| 817 | buf->waiting = 0; | ||
| 818 | buf->pending = 0; | ||
| 819 | buf->file_priv = NULL; | ||
| 820 | |||
| 821 | buf->dev_priv_size = dev->driver->dev_priv_size; | ||
| 822 | buf->dev_private = kzalloc(buf->dev_priv_size, GFP_KERNEL); | ||
| 823 | if (!buf->dev_private) { | ||
| 824 | /* Set count correctly so we free the proper amount. */ | ||
| 825 | entry->buf_count = count; | ||
| 826 | drm_cleanup_buf_error(dev, entry); | ||
| 827 | mutex_unlock(&dev->struct_mutex); | ||
| 828 | atomic_dec(&dev->buf_alloc); | ||
| 829 | return -ENOMEM; | ||
| 830 | } | ||
| 831 | |||
| 832 | DRM_DEBUG("buffer %d @ %p\n", entry->buf_count, buf->address); | ||
| 833 | |||
| 834 | offset += alignment; | ||
| 835 | entry->buf_count++; | ||
| 836 | byte_count += PAGE_SIZE << page_order; | ||
| 837 | } | ||
| 838 | |||
| 839 | DRM_DEBUG("byte_count: %d\n", byte_count); | ||
| 840 | |||
| 841 | temp_buflist = krealloc(dma->buflist, | ||
| 842 | (dma->buf_count + entry->buf_count) * | ||
| 843 | sizeof(*dma->buflist), GFP_KERNEL); | ||
| 844 | if (!temp_buflist) { | ||
| 845 | /* Free the entry because it isn't valid */ | ||
| 846 | drm_cleanup_buf_error(dev, entry); | ||
| 847 | mutex_unlock(&dev->struct_mutex); | ||
| 848 | atomic_dec(&dev->buf_alloc); | ||
| 849 | return -ENOMEM; | ||
| 850 | } | ||
| 851 | dma->buflist = temp_buflist; | ||
| 852 | |||
| 853 | for (i = 0; i < entry->buf_count; i++) { | ||
| 854 | dma->buflist[i + dma->buf_count] = &entry->buflist[i]; | ||
| 855 | } | ||
| 856 | |||
| 857 | dma->buf_count += entry->buf_count; | ||
| 858 | dma->seg_count += entry->seg_count; | ||
| 859 | dma->page_count += byte_count >> PAGE_SHIFT; | ||
| 860 | dma->byte_count += byte_count; | ||
| 861 | |||
| 862 | DRM_DEBUG("dma->buf_count : %d\n", dma->buf_count); | ||
| 863 | DRM_DEBUG("entry->buf_count : %d\n", entry->buf_count); | ||
| 864 | |||
| 865 | mutex_unlock(&dev->struct_mutex); | ||
| 866 | |||
| 867 | request->count = entry->buf_count; | ||
| 868 | request->size = size; | ||
| 869 | |||
| 870 | dma->flags = _DRM_DMA_USE_AGP; | ||
| 871 | |||
| 872 | atomic_dec(&dev->buf_alloc); | ||
| 873 | return 0; | ||
| 874 | } | ||
| 875 | EXPORT_SYMBOL(drm_legacy_addbufs_agp); | ||
| 876 | #endif /* CONFIG_AGP */ | ||
| 877 | |||
| 878 | int drm_legacy_addbufs_pci(struct drm_device *dev, | ||
| 879 | struct drm_buf_desc *request) | ||
| 880 | { | ||
| 881 | struct drm_device_dma *dma = dev->dma; | ||
| 882 | int count; | ||
| 883 | int order; | ||
| 884 | int size; | ||
| 885 | int total; | ||
| 886 | int page_order; | ||
| 887 | struct drm_buf_entry *entry; | ||
| 888 | drm_dma_handle_t *dmah; | ||
| 889 | struct drm_buf *buf; | ||
| 890 | int alignment; | ||
| 891 | unsigned long offset; | ||
| 892 | int i; | ||
| 893 | int byte_count; | ||
| 894 | int page_count; | ||
| 895 | unsigned long *temp_pagelist; | ||
| 896 | struct drm_buf **temp_buflist; | ||
| 897 | |||
| 898 | if (!drm_core_check_feature(dev, DRIVER_PCI_DMA)) | ||
| 899 | return -EOPNOTSUPP; | ||
| 900 | |||
| 901 | if (!dma) | ||
| 902 | return -EINVAL; | ||
| 903 | |||
| 904 | if (!capable(CAP_SYS_ADMIN)) | ||
| 905 | return -EPERM; | ||
| 906 | |||
| 907 | count = request->count; | ||
| 908 | order = order_base_2(request->size); | ||
| 909 | size = 1 << order; | ||
| 910 | |||
| 911 | DRM_DEBUG("count=%d, size=%d (%d), order=%d\n", | ||
| 912 | request->count, request->size, size, order); | ||
| 913 | |||
| 914 | if (order < DRM_MIN_ORDER || order > DRM_MAX_ORDER) | ||
| 915 | return -EINVAL; | ||
| 916 | |||
| 917 | alignment = (request->flags & _DRM_PAGE_ALIGN) | ||
| 918 | ? PAGE_ALIGN(size) : size; | ||
| 919 | page_order = order - PAGE_SHIFT > 0 ? order - PAGE_SHIFT : 0; | ||
| 920 | total = PAGE_SIZE << page_order; | ||
| 921 | |||
| 922 | spin_lock(&dev->buf_lock); | ||
| 923 | if (dev->buf_use) { | ||
| 924 | spin_unlock(&dev->buf_lock); | ||
| 925 | return -EBUSY; | ||
| 926 | } | ||
| 927 | atomic_inc(&dev->buf_alloc); | ||
| 928 | spin_unlock(&dev->buf_lock); | ||
| 929 | |||
| 930 | mutex_lock(&dev->struct_mutex); | ||
| 931 | entry = &dma->bufs[order]; | ||
| 932 | if (entry->buf_count) { | ||
| 933 | mutex_unlock(&dev->struct_mutex); | ||
| 934 | atomic_dec(&dev->buf_alloc); | ||
| 935 | return -ENOMEM; /* May only call once for each order */ | ||
| 936 | } | ||
| 937 | |||
| 938 | if (count < 0 || count > 4096) { | ||
| 939 | mutex_unlock(&dev->struct_mutex); | ||
| 940 | atomic_dec(&dev->buf_alloc); | ||
| 941 | return -EINVAL; | ||
| 942 | } | ||
| 943 | |||
| 944 | entry->buflist = kcalloc(count, sizeof(*entry->buflist), GFP_KERNEL); | ||
| 945 | if (!entry->buflist) { | ||
| 946 | mutex_unlock(&dev->struct_mutex); | ||
| 947 | atomic_dec(&dev->buf_alloc); | ||
| 948 | return -ENOMEM; | ||
| 949 | } | ||
| 950 | |||
| 951 | entry->seglist = kcalloc(count, sizeof(*entry->seglist), GFP_KERNEL); | ||
| 952 | if (!entry->seglist) { | ||
| 953 | kfree(entry->buflist); | ||
| 954 | mutex_unlock(&dev->struct_mutex); | ||
| 955 | atomic_dec(&dev->buf_alloc); | ||
| 956 | return -ENOMEM; | ||
| 957 | } | ||
| 958 | |||
| 959 | /* Keep the original pagelist until we know all the allocations | ||
| 960 | * have succeeded | ||
| 961 | */ | ||
| 962 | temp_pagelist = kmalloc_array(dma->page_count + (count << page_order), | ||
| 963 | sizeof(*dma->pagelist), | ||
| 964 | GFP_KERNEL); | ||
| 965 | if (!temp_pagelist) { | ||
| 966 | kfree(entry->buflist); | ||
| 967 | kfree(entry->seglist); | ||
| 968 | mutex_unlock(&dev->struct_mutex); | ||
| 969 | atomic_dec(&dev->buf_alloc); | ||
| 970 | return -ENOMEM; | ||
| 971 | } | ||
| 972 | memcpy(temp_pagelist, | ||
| 973 | dma->pagelist, dma->page_count * sizeof(*dma->pagelist)); | ||
| 974 | DRM_DEBUG("pagelist: %d entries\n", | ||
| 975 | dma->page_count + (count << page_order)); | ||
| 976 | |||
| 977 | entry->buf_size = size; | ||
| 978 | entry->page_order = page_order; | ||
| 979 | byte_count = 0; | ||
| 980 | page_count = 0; | ||
| 981 | |||
| 982 | while (entry->buf_count < count) { | ||
| 983 | dmah = kmalloc(sizeof(drm_dma_handle_t), GFP_KERNEL); | ||
| 984 | if (!dmah) { | ||
| 985 | /* Set count correctly so we free the proper amount. */ | ||
| 986 | entry->buf_count = count; | ||
| 987 | entry->seg_count = count; | ||
| 988 | drm_cleanup_buf_error(dev, entry); | ||
| 989 | kfree(temp_pagelist); | ||
| 990 | mutex_unlock(&dev->struct_mutex); | ||
| 991 | atomic_dec(&dev->buf_alloc); | ||
| 992 | return -ENOMEM; | ||
| 993 | } | ||
| 994 | |||
| 995 | dmah->size = total; | ||
| 996 | dmah->vaddr = dma_alloc_coherent(dev->dev, | ||
| 997 | dmah->size, | ||
| 998 | &dmah->busaddr, | ||
| 999 | GFP_KERNEL); | ||
| 1000 | if (!dmah->vaddr) { | ||
| 1001 | kfree(dmah); | ||
| 1002 | |||
| 1003 | /* Set count correctly so we free the proper amount. */ | ||
| 1004 | entry->buf_count = count; | ||
| 1005 | entry->seg_count = count; | ||
| 1006 | drm_cleanup_buf_error(dev, entry); | ||
| 1007 | kfree(temp_pagelist); | ||
| 1008 | mutex_unlock(&dev->struct_mutex); | ||
| 1009 | atomic_dec(&dev->buf_alloc); | ||
| 1010 | return -ENOMEM; | ||
| 1011 | } | ||
| 1012 | entry->seglist[entry->seg_count++] = dmah; | ||
| 1013 | for (i = 0; i < (1 << page_order); i++) { | ||
| 1014 | DRM_DEBUG("page %d @ 0x%08lx\n", | ||
| 1015 | dma->page_count + page_count, | ||
| 1016 | (unsigned long)dmah->vaddr + PAGE_SIZE * i); | ||
| 1017 | temp_pagelist[dma->page_count + page_count++] | ||
| 1018 | = (unsigned long)dmah->vaddr + PAGE_SIZE * i; | ||
| 1019 | } | ||
| 1020 | for (offset = 0; | ||
| 1021 | offset + size <= total && entry->buf_count < count; | ||
| 1022 | offset += alignment, ++entry->buf_count) { | ||
| 1023 | buf = &entry->buflist[entry->buf_count]; | ||
| 1024 | buf->idx = dma->buf_count + entry->buf_count; | ||
| 1025 | buf->total = alignment; | ||
| 1026 | buf->order = order; | ||
| 1027 | buf->used = 0; | ||
| 1028 | buf->offset = (dma->byte_count + byte_count + offset); | ||
| 1029 | buf->address = (void *)(dmah->vaddr + offset); | ||
| 1030 | buf->bus_address = dmah->busaddr + offset; | ||
| 1031 | buf->next = NULL; | ||
| 1032 | buf->waiting = 0; | ||
| 1033 | buf->pending = 0; | ||
| 1034 | buf->file_priv = NULL; | ||
| 1035 | |||
| 1036 | buf->dev_priv_size = dev->driver->dev_priv_size; | ||
| 1037 | buf->dev_private = kzalloc(buf->dev_priv_size, | ||
| 1038 | GFP_KERNEL); | ||
| 1039 | if (!buf->dev_private) { | ||
| 1040 | /* Set count correctly so we free the proper amount. */ | ||
| 1041 | entry->buf_count = count; | ||
| 1042 | entry->seg_count = count; | ||
| 1043 | drm_cleanup_buf_error(dev, entry); | ||
| 1044 | kfree(temp_pagelist); | ||
| 1045 | mutex_unlock(&dev->struct_mutex); | ||
| 1046 | atomic_dec(&dev->buf_alloc); | ||
| 1047 | return -ENOMEM; | ||
| 1048 | } | ||
| 1049 | |||
| 1050 | DRM_DEBUG("buffer %d @ %p\n", | ||
| 1051 | entry->buf_count, buf->address); | ||
| 1052 | } | ||
| 1053 | byte_count += PAGE_SIZE << page_order; | ||
| 1054 | } | ||
| 1055 | |||
| 1056 | temp_buflist = krealloc(dma->buflist, | ||
| 1057 | (dma->buf_count + entry->buf_count) * | ||
| 1058 | sizeof(*dma->buflist), GFP_KERNEL); | ||
| 1059 | if (!temp_buflist) { | ||
| 1060 | /* Free the entry because it isn't valid */ | ||
| 1061 | drm_cleanup_buf_error(dev, entry); | ||
| 1062 | kfree(temp_pagelist); | ||
| 1063 | mutex_unlock(&dev->struct_mutex); | ||
| 1064 | atomic_dec(&dev->buf_alloc); | ||
| 1065 | return -ENOMEM; | ||
| 1066 | } | ||
| 1067 | dma->buflist = temp_buflist; | ||
| 1068 | |||
| 1069 | for (i = 0; i < entry->buf_count; i++) { | ||
| 1070 | dma->buflist[i + dma->buf_count] = &entry->buflist[i]; | ||
| 1071 | } | ||
| 1072 | |||
| 1073 | /* No allocations failed, so now we can replace the original pagelist | ||
| 1074 | * with the new one. | ||
| 1075 | */ | ||
| 1076 | if (dma->page_count) { | ||
| 1077 | kfree(dma->pagelist); | ||
| 1078 | } | ||
| 1079 | dma->pagelist = temp_pagelist; | ||
| 1080 | |||
| 1081 | dma->buf_count += entry->buf_count; | ||
| 1082 | dma->seg_count += entry->seg_count; | ||
| 1083 | dma->page_count += entry->seg_count << page_order; | ||
| 1084 | dma->byte_count += PAGE_SIZE * (entry->seg_count << page_order); | ||
| 1085 | |||
| 1086 | mutex_unlock(&dev->struct_mutex); | ||
| 1087 | |||
| 1088 | request->count = entry->buf_count; | ||
| 1089 | request->size = size; | ||
| 1090 | |||
| 1091 | if (request->flags & _DRM_PCI_BUFFER_RO) | ||
| 1092 | dma->flags = _DRM_DMA_USE_PCI_RO; | ||
| 1093 | |||
| 1094 | atomic_dec(&dev->buf_alloc); | ||
| 1095 | return 0; | ||
| 1096 | |||
| 1097 | } | ||
| 1098 | EXPORT_SYMBOL(drm_legacy_addbufs_pci); | ||
| 1099 | |||
| 1100 | static int drm_legacy_addbufs_sg(struct drm_device *dev, | ||
| 1101 | struct drm_buf_desc *request) | ||
| 1102 | { | ||
| 1103 | struct drm_device_dma *dma = dev->dma; | ||
| 1104 | struct drm_buf_entry *entry; | ||
| 1105 | struct drm_buf *buf; | ||
| 1106 | unsigned long offset; | ||
| 1107 | unsigned long agp_offset; | ||
| 1108 | int count; | ||
| 1109 | int order; | ||
| 1110 | int size; | ||
| 1111 | int alignment; | ||
| 1112 | int page_order; | ||
| 1113 | int total; | ||
| 1114 | int byte_count; | ||
| 1115 | int i; | ||
| 1116 | struct drm_buf **temp_buflist; | ||
| 1117 | |||
| 1118 | if (!drm_core_check_feature(dev, DRIVER_SG)) | ||
| 1119 | return -EOPNOTSUPP; | ||
| 1120 | |||
| 1121 | if (!dma) | ||
| 1122 | return -EINVAL; | ||
| 1123 | |||
| 1124 | if (!capable(CAP_SYS_ADMIN)) | ||
| 1125 | return -EPERM; | ||
| 1126 | |||
| 1127 | count = request->count; | ||
| 1128 | order = order_base_2(request->size); | ||
| 1129 | size = 1 << order; | ||
| 1130 | |||
| 1131 | alignment = (request->flags & _DRM_PAGE_ALIGN) | ||
| 1132 | ? PAGE_ALIGN(size) : size; | ||
| 1133 | page_order = order - PAGE_SHIFT > 0 ? order - PAGE_SHIFT : 0; | ||
| 1134 | total = PAGE_SIZE << page_order; | ||
| 1135 | |||
| 1136 | byte_count = 0; | ||
| 1137 | agp_offset = request->agp_start; | ||
| 1138 | |||
| 1139 | DRM_DEBUG("count: %d\n", count); | ||
| 1140 | DRM_DEBUG("order: %d\n", order); | ||
| 1141 | DRM_DEBUG("size: %d\n", size); | ||
| 1142 | DRM_DEBUG("agp_offset: %lu\n", agp_offset); | ||
| 1143 | DRM_DEBUG("alignment: %d\n", alignment); | ||
| 1144 | DRM_DEBUG("page_order: %d\n", page_order); | ||
| 1145 | DRM_DEBUG("total: %d\n", total); | ||
| 1146 | |||
| 1147 | if (order < DRM_MIN_ORDER || order > DRM_MAX_ORDER) | ||
| 1148 | return -EINVAL; | ||
| 1149 | |||
| 1150 | spin_lock(&dev->buf_lock); | ||
| 1151 | if (dev->buf_use) { | ||
| 1152 | spin_unlock(&dev->buf_lock); | ||
| 1153 | return -EBUSY; | ||
| 1154 | } | ||
| 1155 | atomic_inc(&dev->buf_alloc); | ||
| 1156 | spin_unlock(&dev->buf_lock); | ||
| 1157 | |||
| 1158 | mutex_lock(&dev->struct_mutex); | ||
| 1159 | entry = &dma->bufs[order]; | ||
| 1160 | if (entry->buf_count) { | ||
| 1161 | mutex_unlock(&dev->struct_mutex); | ||
| 1162 | atomic_dec(&dev->buf_alloc); | ||
| 1163 | return -ENOMEM; /* May only call once for each order */ | ||
| 1164 | } | ||
| 1165 | |||
| 1166 | if (count < 0 || count > 4096) { | ||
| 1167 | mutex_unlock(&dev->struct_mutex); | ||
| 1168 | atomic_dec(&dev->buf_alloc); | ||
| 1169 | return -EINVAL; | ||
| 1170 | } | ||
| 1171 | |||
| 1172 | entry->buflist = kcalloc(count, sizeof(*entry->buflist), GFP_KERNEL); | ||
| 1173 | if (!entry->buflist) { | ||
| 1174 | mutex_unlock(&dev->struct_mutex); | ||
| 1175 | atomic_dec(&dev->buf_alloc); | ||
| 1176 | return -ENOMEM; | ||
| 1177 | } | ||
| 1178 | |||
| 1179 | entry->buf_size = size; | ||
| 1180 | entry->page_order = page_order; | ||
| 1181 | |||
| 1182 | offset = 0; | ||
| 1183 | |||
| 1184 | while (entry->buf_count < count) { | ||
| 1185 | buf = &entry->buflist[entry->buf_count]; | ||
| 1186 | buf->idx = dma->buf_count + entry->buf_count; | ||
| 1187 | buf->total = alignment; | ||
| 1188 | buf->order = order; | ||
| 1189 | buf->used = 0; | ||
| 1190 | |||
| 1191 | buf->offset = (dma->byte_count + offset); | ||
| 1192 | buf->bus_address = agp_offset + offset; | ||
| 1193 | buf->address = (void *)(agp_offset + offset | ||
| 1194 | + (unsigned long)dev->sg->virtual); | ||
| 1195 | buf->next = NULL; | ||
| 1196 | buf->waiting = 0; | ||
| 1197 | buf->pending = 0; | ||
| 1198 | buf->file_priv = NULL; | ||
| 1199 | |||
| 1200 | buf->dev_priv_size = dev->driver->dev_priv_size; | ||
| 1201 | buf->dev_private = kzalloc(buf->dev_priv_size, GFP_KERNEL); | ||
| 1202 | if (!buf->dev_private) { | ||
| 1203 | /* Set count correctly so we free the proper amount. */ | ||
| 1204 | entry->buf_count = count; | ||
| 1205 | drm_cleanup_buf_error(dev, entry); | ||
| 1206 | mutex_unlock(&dev->struct_mutex); | ||
| 1207 | atomic_dec(&dev->buf_alloc); | ||
| 1208 | return -ENOMEM; | ||
| 1209 | } | ||
| 1210 | |||
| 1211 | DRM_DEBUG("buffer %d @ %p\n", entry->buf_count, buf->address); | ||
| 1212 | |||
| 1213 | offset += alignment; | ||
| 1214 | entry->buf_count++; | ||
| 1215 | byte_count += PAGE_SIZE << page_order; | ||
| 1216 | } | ||
| 1217 | |||
| 1218 | DRM_DEBUG("byte_count: %d\n", byte_count); | ||
| 1219 | |||
| 1220 | temp_buflist = krealloc(dma->buflist, | ||
| 1221 | (dma->buf_count + entry->buf_count) * | ||
| 1222 | sizeof(*dma->buflist), GFP_KERNEL); | ||
| 1223 | if (!temp_buflist) { | ||
| 1224 | /* Free the entry because it isn't valid */ | ||
| 1225 | drm_cleanup_buf_error(dev, entry); | ||
| 1226 | mutex_unlock(&dev->struct_mutex); | ||
| 1227 | atomic_dec(&dev->buf_alloc); | ||
| 1228 | return -ENOMEM; | ||
| 1229 | } | ||
| 1230 | dma->buflist = temp_buflist; | ||
| 1231 | |||
| 1232 | for (i = 0; i < entry->buf_count; i++) { | ||
| 1233 | dma->buflist[i + dma->buf_count] = &entry->buflist[i]; | ||
| 1234 | } | ||
| 1235 | |||
| 1236 | dma->buf_count += entry->buf_count; | ||
| 1237 | dma->seg_count += entry->seg_count; | ||
| 1238 | dma->page_count += byte_count >> PAGE_SHIFT; | ||
| 1239 | dma->byte_count += byte_count; | ||
| 1240 | |||
| 1241 | DRM_DEBUG("dma->buf_count : %d\n", dma->buf_count); | ||
| 1242 | DRM_DEBUG("entry->buf_count : %d\n", entry->buf_count); | ||
| 1243 | |||
| 1244 | mutex_unlock(&dev->struct_mutex); | ||
| 1245 | |||
| 1246 | request->count = entry->buf_count; | ||
| 1247 | request->size = size; | ||
| 1248 | |||
| 1249 | dma->flags = _DRM_DMA_USE_SG; | ||
| 1250 | |||
| 1251 | atomic_dec(&dev->buf_alloc); | ||
| 1252 | return 0; | ||
| 1253 | } | ||
| 1254 | |||
| 1255 | /* | ||
| 1256 | * Add buffers for DMA transfers (ioctl). | ||
| 1257 | * | ||
| 1258 | * \param inode device inode. | ||
| 1259 | * \param file_priv DRM file private. | ||
| 1260 | * \param cmd command. | ||
| 1261 | * \param arg pointer to a struct drm_buf_desc request. | ||
| 1262 | * \return zero on success or a negative number on failure. | ||
| 1263 | * | ||
| 1264 | * According with the memory type specified in drm_buf_desc::flags and the | ||
| 1265 | * build options, it dispatches the call either to addbufs_agp(), | ||
| 1266 | * addbufs_sg() or addbufs_pci() for AGP, scatter-gather or consistent | ||
| 1267 | * PCI memory respectively. | ||
| 1268 | */ | ||
| 1269 | int drm_legacy_addbufs(struct drm_device *dev, void *data, | ||
| 1270 | struct drm_file *file_priv) | ||
| 1271 | { | ||
| 1272 | struct drm_buf_desc *request = data; | ||
| 1273 | int ret; | ||
| 1274 | |||
| 1275 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 1276 | return -EOPNOTSUPP; | ||
| 1277 | |||
| 1278 | if (!drm_core_check_feature(dev, DRIVER_HAVE_DMA)) | ||
| 1279 | return -EOPNOTSUPP; | ||
| 1280 | |||
| 1281 | #if IS_ENABLED(CONFIG_AGP) | ||
| 1282 | if (request->flags & _DRM_AGP_BUFFER) | ||
| 1283 | ret = drm_legacy_addbufs_agp(dev, request); | ||
| 1284 | else | ||
| 1285 | #endif | ||
| 1286 | if (request->flags & _DRM_SG_BUFFER) | ||
| 1287 | ret = drm_legacy_addbufs_sg(dev, request); | ||
| 1288 | else if (request->flags & _DRM_FB_BUFFER) | ||
| 1289 | ret = -EINVAL; | ||
| 1290 | else | ||
| 1291 | ret = drm_legacy_addbufs_pci(dev, request); | ||
| 1292 | |||
| 1293 | return ret; | ||
| 1294 | } | ||
| 1295 | |||
| 1296 | /* | ||
| 1297 | * Get information about the buffer mappings. | ||
| 1298 | * | ||
| 1299 | * This was originally mean for debugging purposes, or by a sophisticated | ||
| 1300 | * client library to determine how best to use the available buffers (e.g., | ||
| 1301 | * large buffers can be used for image transfer). | ||
| 1302 | * | ||
| 1303 | * \param inode device inode. | ||
| 1304 | * \param file_priv DRM file private. | ||
| 1305 | * \param cmd command. | ||
| 1306 | * \param arg pointer to a drm_buf_info structure. | ||
| 1307 | * \return zero on success or a negative number on failure. | ||
| 1308 | * | ||
| 1309 | * Increments drm_device::buf_use while holding the drm_device::buf_lock | ||
| 1310 | * lock, preventing of allocating more buffers after this call. Information | ||
| 1311 | * about each requested buffer is then copied into user space. | ||
| 1312 | */ | ||
| 1313 | int __drm_legacy_infobufs(struct drm_device *dev, | ||
| 1314 | void *data, int *p, | ||
| 1315 | int (*f)(void *, int, struct drm_buf_entry *)) | ||
| 1316 | { | ||
| 1317 | struct drm_device_dma *dma = dev->dma; | ||
| 1318 | int i; | ||
| 1319 | int count; | ||
| 1320 | |||
| 1321 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 1322 | return -EOPNOTSUPP; | ||
| 1323 | |||
| 1324 | if (!drm_core_check_feature(dev, DRIVER_HAVE_DMA)) | ||
| 1325 | return -EOPNOTSUPP; | ||
| 1326 | |||
| 1327 | if (!dma) | ||
| 1328 | return -EINVAL; | ||
| 1329 | |||
| 1330 | spin_lock(&dev->buf_lock); | ||
| 1331 | if (atomic_read(&dev->buf_alloc)) { | ||
| 1332 | spin_unlock(&dev->buf_lock); | ||
| 1333 | return -EBUSY; | ||
| 1334 | } | ||
| 1335 | ++dev->buf_use; /* Can't allocate more after this call */ | ||
| 1336 | spin_unlock(&dev->buf_lock); | ||
| 1337 | |||
| 1338 | for (i = 0, count = 0; i < DRM_MAX_ORDER + 1; i++) { | ||
| 1339 | if (dma->bufs[i].buf_count) | ||
| 1340 | ++count; | ||
| 1341 | } | ||
| 1342 | |||
| 1343 | DRM_DEBUG("count = %d\n", count); | ||
| 1344 | |||
| 1345 | if (*p >= count) { | ||
| 1346 | for (i = 0, count = 0; i < DRM_MAX_ORDER + 1; i++) { | ||
| 1347 | struct drm_buf_entry *from = &dma->bufs[i]; | ||
| 1348 | |||
| 1349 | if (from->buf_count) { | ||
| 1350 | if (f(data, count, from) < 0) | ||
| 1351 | return -EFAULT; | ||
| 1352 | DRM_DEBUG("%d %d %d %d %d\n", | ||
| 1353 | i, | ||
| 1354 | dma->bufs[i].buf_count, | ||
| 1355 | dma->bufs[i].buf_size, | ||
| 1356 | dma->bufs[i].low_mark, | ||
| 1357 | dma->bufs[i].high_mark); | ||
| 1358 | ++count; | ||
| 1359 | } | ||
| 1360 | } | ||
| 1361 | } | ||
| 1362 | *p = count; | ||
| 1363 | |||
| 1364 | return 0; | ||
| 1365 | } | ||
| 1366 | |||
| 1367 | static int copy_one_buf(void *data, int count, struct drm_buf_entry *from) | ||
| 1368 | { | ||
| 1369 | struct drm_buf_info *request = data; | ||
| 1370 | struct drm_buf_desc __user *to = &request->list[count]; | ||
| 1371 | struct drm_buf_desc v = {.count = from->buf_count, | ||
| 1372 | .size = from->buf_size, | ||
| 1373 | .low_mark = from->low_mark, | ||
| 1374 | .high_mark = from->high_mark}; | ||
| 1375 | |||
| 1376 | if (copy_to_user(to, &v, offsetof(struct drm_buf_desc, flags))) | ||
| 1377 | return -EFAULT; | ||
| 1378 | return 0; | ||
| 1379 | } | ||
| 1380 | |||
| 1381 | int drm_legacy_infobufs(struct drm_device *dev, void *data, | ||
| 1382 | struct drm_file *file_priv) | ||
| 1383 | { | ||
| 1384 | struct drm_buf_info *request = data; | ||
| 1385 | |||
| 1386 | return __drm_legacy_infobufs(dev, data, &request->count, copy_one_buf); | ||
| 1387 | } | ||
| 1388 | |||
| 1389 | /* | ||
| 1390 | * Specifies a low and high water mark for buffer allocation | ||
| 1391 | * | ||
| 1392 | * \param inode device inode. | ||
| 1393 | * \param file_priv DRM file private. | ||
| 1394 | * \param cmd command. | ||
| 1395 | * \param arg a pointer to a drm_buf_desc structure. | ||
| 1396 | * \return zero on success or a negative number on failure. | ||
| 1397 | * | ||
| 1398 | * Verifies that the size order is bounded between the admissible orders and | ||
| 1399 | * updates the respective drm_device_dma::bufs entry low and high water mark. | ||
| 1400 | * | ||
| 1401 | * \note This ioctl is deprecated and mostly never used. | ||
| 1402 | */ | ||
| 1403 | int drm_legacy_markbufs(struct drm_device *dev, void *data, | ||
| 1404 | struct drm_file *file_priv) | ||
| 1405 | { | ||
| 1406 | struct drm_device_dma *dma = dev->dma; | ||
| 1407 | struct drm_buf_desc *request = data; | ||
| 1408 | int order; | ||
| 1409 | struct drm_buf_entry *entry; | ||
| 1410 | |||
| 1411 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 1412 | return -EOPNOTSUPP; | ||
| 1413 | |||
| 1414 | if (!drm_core_check_feature(dev, DRIVER_HAVE_DMA)) | ||
| 1415 | return -EOPNOTSUPP; | ||
| 1416 | |||
| 1417 | if (!dma) | ||
| 1418 | return -EINVAL; | ||
| 1419 | |||
| 1420 | DRM_DEBUG("%d, %d, %d\n", | ||
| 1421 | request->size, request->low_mark, request->high_mark); | ||
| 1422 | order = order_base_2(request->size); | ||
| 1423 | if (order < DRM_MIN_ORDER || order > DRM_MAX_ORDER) | ||
| 1424 | return -EINVAL; | ||
| 1425 | entry = &dma->bufs[order]; | ||
| 1426 | |||
| 1427 | if (request->low_mark < 0 || request->low_mark > entry->buf_count) | ||
| 1428 | return -EINVAL; | ||
| 1429 | if (request->high_mark < 0 || request->high_mark > entry->buf_count) | ||
| 1430 | return -EINVAL; | ||
| 1431 | |||
| 1432 | entry->low_mark = request->low_mark; | ||
| 1433 | entry->high_mark = request->high_mark; | ||
| 1434 | |||
| 1435 | return 0; | ||
| 1436 | } | ||
| 1437 | |||
| 1438 | /* | ||
| 1439 | * Unreserve the buffers in list, previously reserved using drmDMA. | ||
| 1440 | * | ||
| 1441 | * \param inode device inode. | ||
| 1442 | * \param file_priv DRM file private. | ||
| 1443 | * \param cmd command. | ||
| 1444 | * \param arg pointer to a drm_buf_free structure. | ||
| 1445 | * \return zero on success or a negative number on failure. | ||
| 1446 | * | ||
| 1447 | * Calls free_buffer() for each used buffer. | ||
| 1448 | * This function is primarily used for debugging. | ||
| 1449 | */ | ||
| 1450 | int drm_legacy_freebufs(struct drm_device *dev, void *data, | ||
| 1451 | struct drm_file *file_priv) | ||
| 1452 | { | ||
| 1453 | struct drm_device_dma *dma = dev->dma; | ||
| 1454 | struct drm_buf_free *request = data; | ||
| 1455 | int i; | ||
| 1456 | int idx; | ||
| 1457 | struct drm_buf *buf; | ||
| 1458 | |||
| 1459 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 1460 | return -EOPNOTSUPP; | ||
| 1461 | |||
| 1462 | if (!drm_core_check_feature(dev, DRIVER_HAVE_DMA)) | ||
| 1463 | return -EOPNOTSUPP; | ||
| 1464 | |||
| 1465 | if (!dma) | ||
| 1466 | return -EINVAL; | ||
| 1467 | |||
| 1468 | DRM_DEBUG("%d\n", request->count); | ||
| 1469 | for (i = 0; i < request->count; i++) { | ||
| 1470 | if (copy_from_user(&idx, &request->list[i], sizeof(idx))) | ||
| 1471 | return -EFAULT; | ||
| 1472 | if (idx < 0 || idx >= dma->buf_count) { | ||
| 1473 | DRM_ERROR("Index %d (of %d max)\n", | ||
| 1474 | idx, dma->buf_count - 1); | ||
| 1475 | return -EINVAL; | ||
| 1476 | } | ||
| 1477 | idx = array_index_nospec(idx, dma->buf_count); | ||
| 1478 | buf = dma->buflist[idx]; | ||
| 1479 | if (buf->file_priv != file_priv) { | ||
| 1480 | DRM_ERROR("Process %d freeing buffer not owned\n", | ||
| 1481 | task_pid_nr(current)); | ||
| 1482 | return -EINVAL; | ||
| 1483 | } | ||
| 1484 | drm_legacy_free_buffer(dev, buf); | ||
| 1485 | } | ||
| 1486 | |||
| 1487 | return 0; | ||
| 1488 | } | ||
| 1489 | |||
| 1490 | /* | ||
| 1491 | * Maps all of the DMA buffers into client-virtual space (ioctl). | ||
| 1492 | * | ||
| 1493 | * \param inode device inode. | ||
| 1494 | * \param file_priv DRM file private. | ||
| 1495 | * \param cmd command. | ||
| 1496 | * \param arg pointer to a drm_buf_map structure. | ||
| 1497 | * \return zero on success or a negative number on failure. | ||
| 1498 | * | ||
| 1499 | * Maps the AGP, SG or PCI buffer region with vm_mmap(), and copies information | ||
| 1500 | * about each buffer into user space. For PCI buffers, it calls vm_mmap() with | ||
| 1501 | * offset equal to 0, which drm_mmap() interprets as PCI buffers and calls | ||
| 1502 | * drm_mmap_dma(). | ||
| 1503 | */ | ||
| 1504 | int __drm_legacy_mapbufs(struct drm_device *dev, void *data, int *p, | ||
| 1505 | void __user **v, | ||
| 1506 | int (*f)(void *, int, unsigned long, | ||
| 1507 | struct drm_buf *), | ||
| 1508 | struct drm_file *file_priv) | ||
| 1509 | { | ||
| 1510 | struct drm_device_dma *dma = dev->dma; | ||
| 1511 | int retcode = 0; | ||
| 1512 | unsigned long virtual; | ||
| 1513 | int i; | ||
| 1514 | |||
| 1515 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 1516 | return -EOPNOTSUPP; | ||
| 1517 | |||
| 1518 | if (!drm_core_check_feature(dev, DRIVER_HAVE_DMA)) | ||
| 1519 | return -EOPNOTSUPP; | ||
| 1520 | |||
| 1521 | if (!dma) | ||
| 1522 | return -EINVAL; | ||
| 1523 | |||
| 1524 | spin_lock(&dev->buf_lock); | ||
| 1525 | if (atomic_read(&dev->buf_alloc)) { | ||
| 1526 | spin_unlock(&dev->buf_lock); | ||
| 1527 | return -EBUSY; | ||
| 1528 | } | ||
| 1529 | dev->buf_use++; /* Can't allocate more after this call */ | ||
| 1530 | spin_unlock(&dev->buf_lock); | ||
| 1531 | |||
| 1532 | if (*p >= dma->buf_count) { | ||
| 1533 | if ((dev->agp && (dma->flags & _DRM_DMA_USE_AGP)) | ||
| 1534 | || (drm_core_check_feature(dev, DRIVER_SG) | ||
| 1535 | && (dma->flags & _DRM_DMA_USE_SG))) { | ||
| 1536 | struct drm_local_map *map = dev->agp_buffer_map; | ||
| 1537 | unsigned long token = dev->agp_buffer_token; | ||
| 1538 | |||
| 1539 | if (!map) { | ||
| 1540 | retcode = -EINVAL; | ||
| 1541 | goto done; | ||
| 1542 | } | ||
| 1543 | virtual = vm_mmap(file_priv->filp, 0, map->size, | ||
| 1544 | PROT_READ | PROT_WRITE, | ||
| 1545 | MAP_SHARED, | ||
| 1546 | token); | ||
| 1547 | } else { | ||
| 1548 | virtual = vm_mmap(file_priv->filp, 0, dma->byte_count, | ||
| 1549 | PROT_READ | PROT_WRITE, | ||
| 1550 | MAP_SHARED, 0); | ||
| 1551 | } | ||
| 1552 | if (virtual > -1024UL) { | ||
| 1553 | /* Real error */ | ||
| 1554 | retcode = (signed long)virtual; | ||
| 1555 | goto done; | ||
| 1556 | } | ||
| 1557 | *v = (void __user *)virtual; | ||
| 1558 | |||
| 1559 | for (i = 0; i < dma->buf_count; i++) { | ||
| 1560 | if (f(data, i, virtual, dma->buflist[i]) < 0) { | ||
| 1561 | retcode = -EFAULT; | ||
| 1562 | goto done; | ||
| 1563 | } | ||
| 1564 | } | ||
| 1565 | } | ||
| 1566 | done: | ||
| 1567 | *p = dma->buf_count; | ||
| 1568 | DRM_DEBUG("%d buffers, retcode = %d\n", *p, retcode); | ||
| 1569 | |||
| 1570 | return retcode; | ||
| 1571 | } | ||
| 1572 | |||
| 1573 | static int map_one_buf(void *data, int idx, unsigned long virtual, | ||
| 1574 | struct drm_buf *buf) | ||
| 1575 | { | ||
| 1576 | struct drm_buf_map *request = data; | ||
| 1577 | unsigned long address = virtual + buf->offset; /* *** */ | ||
| 1578 | |||
| 1579 | if (copy_to_user(&request->list[idx].idx, &buf->idx, | ||
| 1580 | sizeof(request->list[0].idx))) | ||
| 1581 | return -EFAULT; | ||
| 1582 | if (copy_to_user(&request->list[idx].total, &buf->total, | ||
| 1583 | sizeof(request->list[0].total))) | ||
| 1584 | return -EFAULT; | ||
| 1585 | if (clear_user(&request->list[idx].used, sizeof(int))) | ||
| 1586 | return -EFAULT; | ||
| 1587 | if (copy_to_user(&request->list[idx].address, &address, | ||
| 1588 | sizeof(address))) | ||
| 1589 | return -EFAULT; | ||
| 1590 | return 0; | ||
| 1591 | } | ||
| 1592 | |||
| 1593 | int drm_legacy_mapbufs(struct drm_device *dev, void *data, | ||
| 1594 | struct drm_file *file_priv) | ||
| 1595 | { | ||
| 1596 | struct drm_buf_map *request = data; | ||
| 1597 | |||
| 1598 | return __drm_legacy_mapbufs(dev, data, &request->count, | ||
| 1599 | &request->virtual, map_one_buf, | ||
| 1600 | file_priv); | ||
| 1601 | } | ||
| 1602 | |||
| 1603 | int drm_legacy_dma_ioctl(struct drm_device *dev, void *data, | ||
| 1604 | struct drm_file *file_priv) | ||
| 1605 | { | ||
| 1606 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 1607 | return -EOPNOTSUPP; | ||
| 1608 | |||
| 1609 | if (dev->driver->dma_ioctl) | ||
| 1610 | return dev->driver->dma_ioctl(dev, data, file_priv); | ||
| 1611 | else | ||
| 1612 | return -EINVAL; | ||
| 1613 | } | ||
| 1614 | |||
| 1615 | struct drm_local_map *drm_legacy_getsarea(struct drm_device *dev) | ||
| 1616 | { | ||
| 1617 | struct drm_map_list *entry; | ||
| 1618 | |||
| 1619 | list_for_each_entry(entry, &dev->maplist, head) { | ||
| 1620 | if (entry->map && entry->map->type == _DRM_SHM && | ||
| 1621 | (entry->map->flags & _DRM_CONTAINS_LOCK)) { | ||
| 1622 | return entry->map; | ||
| 1623 | } | ||
| 1624 | } | ||
| 1625 | return NULL; | ||
| 1626 | } | ||
| 1627 | EXPORT_SYMBOL(drm_legacy_getsarea); | ||
| @@ -5,7 +5,6 @@ · 4 unmodified lines | |||
| 5 | 5 | ||
| 6 | #include <linux/iosys-map.h> | 6 | #include <linux/iosys-map.h> |
| 7 | #include <linux/list.h> | 7 | #include <linux/list.h> |
| 8 | #include <linux/module.h> | ||
| 9 | #include <linux/mutex.h> | 8 | #include <linux/mutex.h> |
| 10 | #include <linux/seq_file.h> | 9 | #include <linux/seq_file.h> |
| 11 | #include <linux/slab.h> | 10 | #include <linux/slab.h> |
| @@ -84,16 +83,13 @@ int drm_client_init(struct drm_device *dev, struct drm_client_dev *client, · 72 unmodified lines | |||
| 84 | if (!drm_core_check_feature(dev, DRIVER_MODESET) || !dev->driver->dumb_create) | 83 | if (!drm_core_check_feature(dev, DRIVER_MODESET) || !dev->driver->dumb_create) |
| 85 | return -EOPNOTSUPP; | 84 | return -EOPNOTSUPP; |
| 86 | 85 | ||
| 87 | if (funcs && !try_module_get(funcs->owner)) | ||
| 88 | return -ENODEV; | ||
| 89 | |||
| 90 | client->dev = dev; | 86 | client->dev = dev; |
| 91 | client->name = name; | 87 | client->name = name; |
| 92 | client->funcs = funcs; | 88 | client->funcs = funcs; |
| 93 | 89 | ||
| 94 | ret = drm_client_modeset_create(client); | 90 | ret = drm_client_modeset_create(client); |
| 95 | if (ret) | 91 | if (ret) |
| 96 | goto err_put_module; | 92 | return ret; |
| 97 | 93 | ||
| 98 | ret = drm_client_open(client); | 94 | ret = drm_client_open(client); |
| 99 | if (ret) | 95 | if (ret) |
| @@ -105,10 +101,6 @@ int drm_client_init(struct drm_device *dev, struct drm_client_dev *client, · 5 unmodified lines | |||
| 105 | 101 | ||
| 106 | err_free: | 102 | err_free: |
| 107 | drm_client_modeset_free(client); | 103 | drm_client_modeset_free(client); |
| 108 | err_put_module: | ||
| 109 | if (funcs) | ||
| 110 | module_put(funcs->owner); | ||
| 111 | |||
| 112 | return ret; | 104 | return ret; |
| 113 | } | 105 | } |
| 114 | EXPORT_SYMBOL(drm_client_init); | 106 | EXPORT_SYMBOL(drm_client_init); |
| @@ -177,8 +169,6 @@ void drm_client_release(struct drm_client_dev *client) · 62 unmodified lines | |||
| 177 | drm_client_modeset_free(client); | 169 | drm_client_modeset_free(client); |
| 178 | drm_client_close(client); | 170 | drm_client_close(client); |
| 179 | drm_dev_put(dev); | 171 | drm_dev_put(dev); |
| 180 | if (client->funcs) | ||
| 181 | module_put(client->funcs->owner); | ||
| 182 | } | 172 | } |
| 183 | EXPORT_SYMBOL(drm_client_release); | 173 | EXPORT_SYMBOL(drm_client_release); |
| 184 | 174 | ||
| @@ -1198,6 +1198,12 @@ static const u32 dp_colorspaces = · 1197 unmodified lines | |||
| 1198 | * drm_connector_set_path_property(), in the case of DP MST with the | 1198 | * drm_connector_set_path_property(), in the case of DP MST with the |
| 1199 | * path property the MST manager created. Userspace cannot change this | 1199 | * path property the MST manager created. Userspace cannot change this |
| 1200 | * property. | 1200 | * property. |
| 1201 | * | ||
| 1202 | * In the case of DP MST, the property has the format | ||
| 1203 | * ``mst:<parent>-<ports>`` where ``<parent>`` is the KMS object ID of the | ||
| 1204 | * parent connector and ``<ports>`` is a hyphen-separated list of DP MST | ||
| 1205 | * port numbers. Note, KMS object IDs are not guaranteed to be stable | ||
| 1206 | * across reboots. | ||
| 1201 | * TILE: | 1207 | * TILE: |
| 1202 | * Connector tile group property to indicate how a set of DRM connector | 1208 | * Connector tile group property to indicate how a set of DRM connector |
| 1203 | * compose together into one logical screen. This is used by both high-res | 1209 | * compose together into one logical screen. This is used by both high-res |
| @@ -1,513 +0,0 @@ | |||
| 1 | /* | ||
| 2 | * Legacy: Generic DRM Contexts | ||
| 3 | * | ||
| 4 | * Copyright 1999, 2000 Precision Insight, Inc., Cedar Park, Texas. | ||
| 5 | * Copyright 2000 VA Linux Systems, Inc., Sunnyvale, California. | ||
| 6 | * All Rights Reserved. | ||
| 7 | * | ||
| 8 | * Author: Rickard E. (Rik) Faith <faith@valinux.com> | ||
| 9 | * Author: Gareth Hughes <gareth@valinux.com> | ||
| 10 | * | ||
| 11 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 12 | * copy of this software and associated documentation files (the "Software"), | ||
| 13 | * to deal in the Software without restriction, including without limitation | ||
| 14 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 15 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 16 | * Software is furnished to do so, subject to the following conditions: | ||
| 17 | * | ||
| 18 | * The above copyright notice and this permission notice (including the next | ||
| 19 | * paragraph) shall be included in all copies or substantial portions of the | ||
| 20 | * Software. | ||
| 21 | * | ||
| 22 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 23 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 24 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 25 | * VA LINUX SYSTEMS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR | ||
| 26 | * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | ||
| 27 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | ||
| 28 | * OTHER DEALINGS IN THE SOFTWARE. | ||
| 29 | */ | ||
| 30 | |||
| 31 | #include <linux/slab.h> | ||
| 32 | #include <linux/uaccess.h> | ||
| 33 | |||
| 34 | #include <drm/drm_drv.h> | ||
| 35 | #include <drm/drm_file.h> | ||
| 36 | #include <drm/drm_print.h> | ||
| 37 | |||
| 38 | #include "drm_legacy.h" | ||
| 39 | |||
| 40 | struct drm_ctx_list { | ||
| 41 | struct list_head head; | ||
| 42 | drm_context_t handle; | ||
| 43 | struct drm_file *tag; | ||
| 44 | }; | ||
| 45 | |||
| 46 | /******************************************************************/ | ||
| 47 | /** \name Context bitmap support */ | ||
| 48 | /*@{*/ | ||
| 49 | |||
| 50 | /* | ||
| 51 | * Free a handle from the context bitmap. | ||
| 52 | * | ||
| 53 | * \param dev DRM device. | ||
| 54 | * \param ctx_handle context handle. | ||
| 55 | * | ||
| 56 | * Clears the bit specified by \p ctx_handle in drm_device::ctx_bitmap and the entry | ||
| 57 | * in drm_device::ctx_idr, while holding the drm_device::struct_mutex | ||
| 58 | * lock. | ||
| 59 | */ | ||
| 60 | void drm_legacy_ctxbitmap_free(struct drm_device * dev, int ctx_handle) | ||
| 61 | { | ||
| 62 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 63 | return; | ||
| 64 | |||
| 65 | mutex_lock(&dev->struct_mutex); | ||
| 66 | idr_remove(&dev->ctx_idr, ctx_handle); | ||
| 67 | mutex_unlock(&dev->struct_mutex); | ||
| 68 | } | ||
| 69 | |||
| 70 | /* | ||
| 71 | * Context bitmap allocation. | ||
| 72 | * | ||
| 73 | * \param dev DRM device. | ||
| 74 | * \return (non-negative) context handle on success or a negative number on failure. | ||
| 75 | * | ||
| 76 | * Allocate a new idr from drm_device::ctx_idr while holding the | ||
| 77 | * drm_device::struct_mutex lock. | ||
| 78 | */ | ||
| 79 | static int drm_legacy_ctxbitmap_next(struct drm_device * dev) | ||
| 80 | { | ||
| 81 | int ret; | ||
| 82 | |||
| 83 | mutex_lock(&dev->struct_mutex); | ||
| 84 | ret = idr_alloc(&dev->ctx_idr, NULL, DRM_RESERVED_CONTEXTS, 0, | ||
| 85 | GFP_KERNEL); | ||
| 86 | mutex_unlock(&dev->struct_mutex); | ||
| 87 | return ret; | ||
| 88 | } | ||
| 89 | |||
| 90 | /* | ||
| 91 | * Context bitmap initialization. | ||
| 92 | * | ||
| 93 | * \param dev DRM device. | ||
| 94 | * | ||
| 95 | * Initialise the drm_device::ctx_idr | ||
| 96 | */ | ||
| 97 | void drm_legacy_ctxbitmap_init(struct drm_device * dev) | ||
| 98 | { | ||
| 99 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 100 | return; | ||
| 101 | |||
| 102 | idr_init(&dev->ctx_idr); | ||
| 103 | } | ||
| 104 | |||
| 105 | /* | ||
| 106 | * Context bitmap cleanup. | ||
| 107 | * | ||
| 108 | * \param dev DRM device. | ||
| 109 | * | ||
| 110 | * Free all idr members using drm_ctx_sarea_free helper function | ||
| 111 | * while holding the drm_device::struct_mutex lock. | ||
| 112 | */ | ||
| 113 | void drm_legacy_ctxbitmap_cleanup(struct drm_device * dev) | ||
| 114 | { | ||
| 115 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 116 | return; | ||
| 117 | |||
| 118 | mutex_lock(&dev->struct_mutex); | ||
| 119 | idr_destroy(&dev->ctx_idr); | ||
| 120 | mutex_unlock(&dev->struct_mutex); | ||
| 121 | } | ||
| 122 | |||
| 123 | /** | ||
| 124 | * drm_legacy_ctxbitmap_flush() - Flush all contexts owned by a file | ||
| 125 | * @dev: DRM device to operate on | ||
| 126 | * @file: Open file to flush contexts for | ||
| 127 | * | ||
| 128 | * This iterates over all contexts on @dev and drops them if they're owned by | ||
| 129 | * @file. Note that after this call returns, new contexts might be added if | ||
| 130 | * the file is still alive. | ||
| 131 | */ | ||
| 132 | void drm_legacy_ctxbitmap_flush(struct drm_device *dev, struct drm_file *file) | ||
| 133 | { | ||
| 134 | struct drm_ctx_list *pos, *tmp; | ||
| 135 | |||
| 136 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 137 | return; | ||
| 138 | |||
| 139 | mutex_lock(&dev->ctxlist_mutex); | ||
| 140 | |||
| 141 | list_for_each_entry_safe(pos, tmp, &dev->ctxlist, head) { | ||
| 142 | if (pos->tag == file && | ||
| 143 | pos->handle != DRM_KERNEL_CONTEXT) { | ||
| 144 | if (dev->driver->context_dtor) | ||
| 145 | dev->driver->context_dtor(dev, pos->handle); | ||
| 146 | |||
| 147 | drm_legacy_ctxbitmap_free(dev, pos->handle); | ||
| 148 | list_del(&pos->head); | ||
| 149 | kfree(pos); | ||
| 150 | } | ||
| 151 | } | ||
| 152 | |||
| 153 | mutex_unlock(&dev->ctxlist_mutex); | ||
| 154 | } | ||
| 155 | |||
| 156 | /*@}*/ | ||
| 157 | |||
| 158 | /******************************************************************/ | ||
| 159 | /** \name Per Context SAREA Support */ | ||
| 160 | /*@{*/ | ||
| 161 | |||
| 162 | /* | ||
| 163 | * Get per-context SAREA. | ||
| 164 | * | ||
| 165 | * \param inode device inode. | ||
| 166 | * \param file_priv DRM file private. | ||
| 167 | * \param cmd command. | ||
| 168 | * \param arg user argument pointing to a drm_ctx_priv_map structure. | ||
| 169 | * \return zero on success or a negative number on failure. | ||
| 170 | * | ||
| 171 | * Gets the map from drm_device::ctx_idr with the handle specified and | ||
| 172 | * returns its handle. | ||
| 173 | */ | ||
| 174 | int drm_legacy_getsareactx(struct drm_device *dev, void *data, | ||
| 175 | struct drm_file *file_priv) | ||
| 176 | { | ||
| 177 | struct drm_ctx_priv_map *request = data; | ||
| 178 | struct drm_local_map *map; | ||
| 179 | struct drm_map_list *_entry; | ||
| 180 | |||
| 181 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 182 | return -EOPNOTSUPP; | ||
| 183 | |||
| 184 | mutex_lock(&dev->struct_mutex); | ||
| 185 | |||
| 186 | map = idr_find(&dev->ctx_idr, request->ctx_id); | ||
| 187 | if (!map) { | ||
| 188 | mutex_unlock(&dev->struct_mutex); | ||
| 189 | return -EINVAL; | ||
| 190 | } | ||
| 191 | |||
| 192 | request->handle = NULL; | ||
| 193 | list_for_each_entry(_entry, &dev->maplist, head) { | ||
| 194 | if (_entry->map == map) { | ||
| 195 | request->handle = | ||
| 196 | (void *)(unsigned long)_entry->user_token; | ||
| 197 | break; | ||
| 198 | } | ||
| 199 | } | ||
| 200 | |||
| 201 | mutex_unlock(&dev->struct_mutex); | ||
| 202 | |||
| 203 | if (request->handle == NULL) | ||
| 204 | return -EINVAL; | ||
| 205 | |||
| 206 | return 0; | ||
| 207 | } | ||
| 208 | |||
| 209 | /* | ||
| 210 | * Set per-context SAREA. | ||
| 211 | * | ||
| 212 | * \param inode device inode. | ||
| 213 | * \param file_priv DRM file private. | ||
| 214 | * \param cmd command. | ||
| 215 | * \param arg user argument pointing to a drm_ctx_priv_map structure. | ||
| 216 | * \return zero on success or a negative number on failure. | ||
| 217 | * | ||
| 218 | * Searches the mapping specified in \p arg and update the entry in | ||
| 219 | * drm_device::ctx_idr with it. | ||
| 220 | */ | ||
| 221 | int drm_legacy_setsareactx(struct drm_device *dev, void *data, | ||
| 222 | struct drm_file *file_priv) | ||
| 223 | { | ||
| 224 | struct drm_ctx_priv_map *request = data; | ||
| 225 | struct drm_local_map *map = NULL; | ||
| 226 | struct drm_map_list *r_list = NULL; | ||
| 227 | |||
| 228 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 229 | return -EOPNOTSUPP; | ||
| 230 | |||
| 231 | mutex_lock(&dev->struct_mutex); | ||
| 232 | list_for_each_entry(r_list, &dev->maplist, head) { | ||
| 233 | if (r_list->map | ||
| 234 | && r_list->user_token == (unsigned long) request->handle) | ||
| 235 | goto found; | ||
| 236 | } | ||
| 237 | bad: | ||
| 238 | mutex_unlock(&dev->struct_mutex); | ||
| 239 | return -EINVAL; | ||
| 240 | |||
| 241 | found: | ||
| 242 | map = r_list->map; | ||
| 243 | if (!map) | ||
| 244 | goto bad; | ||
| 245 | |||
| 246 | if (IS_ERR(idr_replace(&dev->ctx_idr, map, request->ctx_id))) | ||
| 247 | goto bad; | ||
| 248 | |||
| 249 | mutex_unlock(&dev->struct_mutex); | ||
| 250 | |||
| 251 | return 0; | ||
| 252 | } | ||
| 253 | |||
| 254 | /*@}*/ | ||
| 255 | |||
| 256 | /******************************************************************/ | ||
| 257 | /** \name The actual DRM context handling routines */ | ||
| 258 | /*@{*/ | ||
| 259 | |||
| 260 | /* | ||
| 261 | * Switch context. | ||
| 262 | * | ||
| 263 | * \param dev DRM device. | ||
| 264 | * \param old old context handle. | ||
| 265 | * \param new new context handle. | ||
| 266 | * \return zero on success or a negative number on failure. | ||
| 267 | * | ||
| 268 | * Attempt to set drm_device::context_flag. | ||
| 269 | */ | ||
| 270 | static int drm_context_switch(struct drm_device * dev, int old, int new) | ||
| 271 | { | ||
| 272 | if (test_and_set_bit(0, &dev->context_flag)) { | ||
| 273 | DRM_ERROR("Reentering -- FIXME\n"); | ||
| 274 | return -EBUSY; | ||
| 275 | } | ||
| 276 | |||
| 277 | DRM_DEBUG("Context switch from %d to %d\n", old, new); | ||
| 278 | |||
| 279 | if (new == dev->last_context) { | ||
| 280 | clear_bit(0, &dev->context_flag); | ||
| 281 | return 0; | ||
| 282 | } | ||
| 283 | |||
| 284 | return 0; | ||
| 285 | } | ||
| 286 | |||
| 287 | /* | ||
| 288 | * Complete context switch. | ||
| 289 | * | ||
| 290 | * \param dev DRM device. | ||
| 291 | * \param new new context handle. | ||
| 292 | * \return zero on success or a negative number on failure. | ||
| 293 | * | ||
| 294 | * Updates drm_device::last_context and drm_device::last_switch. Verifies the | ||
| 295 | * hardware lock is held, clears the drm_device::context_flag and wakes up | ||
| 296 | * drm_device::context_wait. | ||
| 297 | */ | ||
| 298 | static int drm_context_switch_complete(struct drm_device *dev, | ||
| 299 | struct drm_file *file_priv, int new) | ||
| 300 | { | ||
| 301 | dev->last_context = new; /* PRE/POST: This is the _only_ writer. */ | ||
| 302 | |||
| 303 | if (!_DRM_LOCK_IS_HELD(file_priv->master->lock.hw_lock->lock)) { | ||
| 304 | DRM_ERROR("Lock isn't held after context switch\n"); | ||
| 305 | } | ||
| 306 | |||
| 307 | /* If a context switch is ever initiated | ||
| 308 | when the kernel holds the lock, release | ||
| 309 | that lock here. | ||
| 310 | */ | ||
| 311 | clear_bit(0, &dev->context_flag); | ||
| 312 | |||
| 313 | return 0; | ||
| 314 | } | ||
| 315 | |||
| 316 | /* | ||
| 317 | * Reserve contexts. | ||
| 318 | * | ||
| 319 | * \param inode device inode. | ||
| 320 | * \param file_priv DRM file private. | ||
| 321 | * \param cmd command. | ||
| 322 | * \param arg user argument pointing to a drm_ctx_res structure. | ||
| 323 | * \return zero on success or a negative number on failure. | ||
| 324 | */ | ||
| 325 | int drm_legacy_resctx(struct drm_device *dev, void *data, | ||
| 326 | struct drm_file *file_priv) | ||
| 327 | { | ||
| 328 | struct drm_ctx_res *res = data; | ||
| 329 | struct drm_ctx ctx; | ||
| 330 | int i; | ||
| 331 | |||
| 332 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 333 | return -EOPNOTSUPP; | ||
| 334 | |||
| 335 | if (res->count >= DRM_RESERVED_CONTEXTS) { | ||
| 336 | memset(&ctx, 0, sizeof(ctx)); | ||
| 337 | for (i = 0; i < DRM_RESERVED_CONTEXTS; i++) { | ||
| 338 | ctx.handle = i; | ||
| 339 | if (copy_to_user(&res->contexts[i], &ctx, sizeof(ctx))) | ||
| 340 | return -EFAULT; | ||
| 341 | } | ||
| 342 | } | ||
| 343 | res->count = DRM_RESERVED_CONTEXTS; | ||
| 344 | |||
| 345 | return 0; | ||
| 346 | } | ||
| 347 | |||
| 348 | /* | ||
| 349 | * Add context. | ||
| 350 | * | ||
| 351 | * \param inode device inode. | ||
| 352 | * \param file_priv DRM file private. | ||
| 353 | * \param cmd command. | ||
| 354 | * \param arg user argument pointing to a drm_ctx structure. | ||
| 355 | * \return zero on success or a negative number on failure. | ||
| 356 | * | ||
| 357 | * Get a new handle for the context and copy to userspace. | ||
| 358 | */ | ||
| 359 | int drm_legacy_addctx(struct drm_device *dev, void *data, | ||
| 360 | struct drm_file *file_priv) | ||
| 361 | { | ||
| 362 | struct drm_ctx_list *ctx_entry; | ||
| 363 | struct drm_ctx *ctx = data; | ||
| 364 | int tmp_handle; | ||
| 365 | |||
| 366 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 367 | return -EOPNOTSUPP; | ||
| 368 | |||
| 369 | tmp_handle = drm_legacy_ctxbitmap_next(dev); | ||
| 370 | if (tmp_handle == DRM_KERNEL_CONTEXT) { | ||
| 371 | /* Skip kernel's context and get a new one. */ | ||
| 372 | tmp_handle = drm_legacy_ctxbitmap_next(dev); | ||
| 373 | } | ||
| 374 | DRM_DEBUG("%d\n", tmp_handle); | ||
| 375 | if (tmp_handle < 0) { | ||
| 376 | DRM_DEBUG("Not enough free contexts.\n"); | ||
| 377 | /* Should this return -EBUSY instead? */ | ||
| 378 | return tmp_handle; | ||
| 379 | } | ||
| 380 | |||
| 381 | ctx->handle = tmp_handle; | ||
| 382 | |||
| 383 | ctx_entry = kmalloc(sizeof(*ctx_entry), GFP_KERNEL); | ||
| 384 | if (!ctx_entry) { | ||
| 385 | DRM_DEBUG("out of memory\n"); | ||
| 386 | return -ENOMEM; | ||
| 387 | } | ||
| 388 | |||
| 389 | INIT_LIST_HEAD(&ctx_entry->head); | ||
| 390 | ctx_entry->handle = ctx->handle; | ||
| 391 | ctx_entry->tag = file_priv; | ||
| 392 | |||
| 393 | mutex_lock(&dev->ctxlist_mutex); | ||
| 394 | list_add(&ctx_entry->head, &dev->ctxlist); | ||
| 395 | mutex_unlock(&dev->ctxlist_mutex); | ||
| 396 | |||
| 397 | return 0; | ||
| 398 | } | ||
| 399 | |||
| 400 | /* | ||
| 401 | * Get context. | ||
| 402 | * | ||
| 403 | * \param inode device inode. | ||
| 404 | * \param file_priv DRM file private. | ||
| 405 | * \param cmd command. | ||
| 406 | * \param arg user argument pointing to a drm_ctx structure. | ||
| 407 | * \return zero on success or a negative number on failure. | ||
| 408 | */ | ||
| 409 | int drm_legacy_getctx(struct drm_device *dev, void *data, | ||
| 410 | struct drm_file *file_priv) | ||
| 411 | { | ||
| 412 | struct drm_ctx *ctx = data; | ||
| 413 | |||
| 414 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 415 | return -EOPNOTSUPP; | ||
| 416 | |||
| 417 | /* This is 0, because we don't handle any context flags */ | ||
| 418 | ctx->flags = 0; | ||
| 419 | |||
| 420 | return 0; | ||
| 421 | } | ||
| 422 | |||
| 423 | /* | ||
| 424 | * Switch context. | ||
| 425 | * | ||
| 426 | * \param inode device inode. | ||
| 427 | * \param file_priv DRM file private. | ||
| 428 | * \param cmd command. | ||
| 429 | * \param arg user argument pointing to a drm_ctx structure. | ||
| 430 | * \return zero on success or a negative number on failure. | ||
| 431 | * | ||
| 432 | * Calls context_switch(). | ||
| 433 | */ | ||
| 434 | int drm_legacy_switchctx(struct drm_device *dev, void *data, | ||
| 435 | struct drm_file *file_priv) | ||
| 436 | { | ||
| 437 | struct drm_ctx *ctx = data; | ||
| 438 | |||
| 439 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 440 | return -EOPNOTSUPP; | ||
| 441 | |||
| 442 | DRM_DEBUG("%d\n", ctx->handle); | ||
| 443 | return drm_context_switch(dev, dev->last_context, ctx->handle); | ||
| 444 | } | ||
| 445 | |||
| 446 | /* | ||
| 447 | * New context. | ||
| 448 | * | ||
| 449 | * \param inode device inode. | ||
| 450 | * \param file_priv DRM file private. | ||
| 451 | * \param cmd command. | ||
| 452 | * \param arg user argument pointing to a drm_ctx structure. | ||
| 453 | * \return zero on success or a negative number on failure. | ||
| 454 | * | ||
| 455 | * Calls context_switch_complete(). | ||
| 456 | */ | ||
| 457 | int drm_legacy_newctx(struct drm_device *dev, void *data, | ||
| 458 | struct drm_file *file_priv) | ||
| 459 | { | ||
| 460 | struct drm_ctx *ctx = data; | ||
| 461 | |||
| 462 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 463 | return -EOPNOTSUPP; | ||
| 464 | |||
| 465 | DRM_DEBUG("%d\n", ctx->handle); | ||
| 466 | drm_context_switch_complete(dev, file_priv, ctx->handle); | ||
| 467 | |||
| 468 | return 0; | ||
| 469 | } | ||
| 470 | |||
| 471 | /* | ||
| 472 | * Remove context. | ||
| 473 | * | ||
| 474 | * \param inode device inode. | ||
| 475 | * \param file_priv DRM file private. | ||
| 476 | * \param cmd command. | ||
| 477 | * \param arg user argument pointing to a drm_ctx structure. | ||
| 478 | * \return zero on success or a negative number on failure. | ||
| 479 | * | ||
| 480 | * If not the special kernel context, calls ctxbitmap_free() to free the specified context. | ||
| 481 | */ | ||
| 482 | int drm_legacy_rmctx(struct drm_device *dev, void *data, | ||
| 483 | struct drm_file *file_priv) | ||
| 484 | { | ||
| 485 | struct drm_ctx *ctx = data; | ||
| 486 | |||
| 487 | if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 488 | return -EOPNOTSUPP; | ||
| 489 | |||
| 490 | DRM_DEBUG("%d\n", ctx->handle); | ||
| 491 | if (ctx->handle != DRM_KERNEL_CONTEXT) { | ||
| 492 | if (dev->driver->context_dtor) | ||
| 493 | dev->driver->context_dtor(dev, ctx->handle); | ||
| 494 | drm_legacy_ctxbitmap_free(dev, ctx->handle); | ||
| 495 | } | ||
| 496 | |||
| 497 | mutex_lock(&dev->ctxlist_mutex); | ||
| 498 | if (!list_empty(&dev->ctxlist)) { | ||
| 499 | struct drm_ctx_list *pos, *n; | ||
| 500 | |||
| 501 | list_for_each_entry_safe(pos, n, &dev->ctxlist, head) { | ||
| 502 | if (pos->handle == ctx->handle) { | ||
| 503 | list_del(&pos->head); | ||
| 504 | kfree(pos); | ||
| 505 | } | ||
| 506 | } | ||
| 507 | } | ||
| 508 | mutex_unlock(&dev->ctxlist_mutex); | ||
| 509 | |||
| 510 | return 0; | ||
| 511 | } | ||
| 512 | |||
| 513 | /*@}*/ | ||
| @@ -439,11 +439,8 @@ EXPORT_SYMBOL(drm_crtc_helper_set_mode); · 438 unmodified lines | |||
| 439 | * @state: atomic state object | 439 | * @state: atomic state object |
| 440 | * | 440 | * |
| 441 | * Provides a default CRTC-state check handler for CRTCs that only have | 441 | * Provides a default CRTC-state check handler for CRTCs that only have |
| 442 | * one primary plane attached to it. | 442 | * one primary plane attached to it. This is often the case for the CRTC |
| 443 | * | 443 | * of simple framebuffers. |
| 444 | * This is often the case for the CRTC of simple framebuffers. See also | ||
| 445 | * drm_plane_helper_atomic_check() for the respective plane-state check | ||
| 446 | * helper function. | ||
| 447 | * | 444 | * |
| 448 | * RETURNS: | 445 | * RETURNS: |
| 449 | * Zero on success, or an errno code otherwise. | 446 | * Zero on success, or an errno code otherwise. |
| @@ -222,6 +222,8 @@ int drm_mode_addfb2_ioctl(struct drm_device *dev, · 221 unmodified lines | |||
| 222 | void *data, struct drm_file *file_priv); | 222 | void *data, struct drm_file *file_priv); |
| 223 | int drm_mode_rmfb_ioctl(struct drm_device *dev, | 223 | int drm_mode_rmfb_ioctl(struct drm_device *dev, |
| 224 | void *data, struct drm_file *file_priv); | 224 | void *data, struct drm_file *file_priv); |
| 225 | int drm_mode_closefb_ioctl(struct drm_device *dev, | ||
| 226 | void *data, struct drm_file *file_priv); | ||
| 225 | int drm_mode_getfb(struct drm_device *dev, | 227 | int drm_mode_getfb(struct drm_device *dev, |
| 226 | void *data, struct drm_file *file_priv); | 228 | void *data, struct drm_file *file_priv); |
| 227 | int drm_mode_getfb2_ioctl(struct drm_device *dev, | 229 | int drm_mode_getfb2_ioctl(struct drm_device *dev, |
| @@ -251,7 +253,7 @@ int drm_atomic_set_property(struct drm_atomic_state *state, · 23 unmodified lines | |||
| 251 | struct drm_file *file_priv, | 253 | struct drm_file *file_priv, |
| 252 | struct drm_mode_object *obj, | 254 | struct drm_mode_object *obj, |
| 253 | struct drm_property *prop, | 255 | struct drm_property *prop, |
| 254 | uint64_t prop_value); | 256 | u64 prop_value, bool async_flip); |
| 255 | int drm_atomic_get_property(struct drm_mode_object *obj, | 257 | int drm_atomic_get_property(struct drm_mode_object *obj, |
| 256 | struct drm_property *property, uint64_t *val); | 258 | struct drm_property *property, uint64_t *val); |
| 257 | 259 | ||
| @@ -241,7 +241,8 @@ drm_atomic_helper_damage_iter_init(struct drm_atomic_helper_damage_iter *iter, · 240 unmodified lines | |||
| 241 | iter->plane_src.x2 = (src.x2 >> 16) + !!(src.x2 & 0xFFFF); | 241 | iter->plane_src.x2 = (src.x2 >> 16) + !!(src.x2 & 0xFFFF); |
| 242 | iter->plane_src.y2 = (src.y2 >> 16) + !!(src.y2 & 0xFFFF); | 242 | iter->plane_src.y2 = (src.y2 >> 16) + !!(src.y2 & 0xFFFF); |
| 243 | 243 | ||
| 244 | if (!iter->clips || !drm_rect_equals(&state->src, &old_state->src)) { | 244 | if (!iter->clips || state->ignore_damage_clips || |
| 245 | !drm_rect_equals(&state->src, &old_state->src)) { | ||
| 245 | iter->clips = NULL; | 246 | iter->clips = NULL; |
| 246 | iter->num_clips = 0; | 247 | iter->num_clips = 0; |
| 247 | iter->full_update = true; | 248 | iter->full_update = true; |
| @@ -314,10 +314,8 @@ void drm_debugfs_dev_register(struct drm_device *dev) · 313 unmodified lines | |||
| 314 | drm_framebuffer_debugfs_init(dev); | 314 | drm_framebuffer_debugfs_init(dev); |
| 315 | drm_client_debugfs_init(dev); | 315 | drm_client_debugfs_init(dev); |
| 316 | } | 316 | } |
| 317 | if (drm_drv_uses_atomic_modeset(dev)) { | 317 | if (drm_drv_uses_atomic_modeset(dev)) |
| 318 | drm_atomic_debugfs_init(dev); | 318 | drm_atomic_debugfs_init(dev); |
| 319 | drm_bridge_debugfs_init(dev); | ||
| 320 | } | ||
| 321 | } | 319 | } |
| 322 | 320 | ||
| 323 | int drm_debugfs_register(struct drm_minor *minor, int minor_id, | 321 | int drm_debugfs_register(struct drm_minor *minor, int minor_id, |
| @@ -589,4 +587,65 @@ void drm_debugfs_crtc_remove(struct drm_crtc *crtc) · 265 unmodified lines | |||
| 589 | crtc->debugfs_entry = NULL; | 587 | crtc->debugfs_entry = NULL; |
| 590 | } | 588 | } |
| 591 | 589 | ||
| 590 | static int bridges_show(struct seq_file *m, void *data) | ||
| 591 | { | ||
| 592 | struct drm_encoder *encoder = m->private; | ||
| 593 | struct drm_printer p = drm_seq_file_printer(m); | ||
| 594 | struct drm_bridge *bridge; | ||
| 595 | unsigned int idx = 0; | ||
| 596 | |||
| 597 | drm_for_each_bridge_in_chain(encoder, bridge) { | ||
| 598 | drm_printf(&p, "bridge[%d]: %ps\n", idx++, bridge->funcs); | ||
| 599 | drm_printf(&p, "\ttype: [%d] %s\n", | ||
| 600 | bridge->type, | ||
| 601 | drm_get_connector_type_name(bridge->type)); | ||
| 602 | #ifdef CONFIG_OF | ||
| 603 | if (bridge->of_node) | ||
| 604 | drm_printf(&p, "\tOF: %pOFfc\n", bridge->of_node); | ||
| 605 | #endif | ||
| 606 | drm_printf(&p, "\tops: [0x%x]", bridge->ops); | ||
| 607 | if (bridge->ops & DRM_BRIDGE_OP_DETECT) | ||
| 608 | drm_puts(&p, " detect"); | ||
| 609 | if (bridge->ops & DRM_BRIDGE_OP_EDID) | ||
| 610 | drm_puts(&p, " edid"); | ||
| 611 | if (bridge->ops & DRM_BRIDGE_OP_HPD) | ||
| 612 | drm_puts(&p, " hpd"); | ||
| 613 | if (bridge->ops & DRM_BRIDGE_OP_MODES) | ||
| 614 | drm_puts(&p, " modes"); | ||
| 615 | drm_puts(&p, "\n"); | ||
| 616 | } | ||
| 617 | |||
| 618 | return 0; | ||
| 619 | } | ||
| 620 | DEFINE_SHOW_ATTRIBUTE(bridges); | ||
| 621 | |||
| 622 | void drm_debugfs_encoder_add(struct drm_encoder *encoder) | ||
| 623 | { | ||
| 624 | struct drm_minor *minor = encoder->dev->primary; | ||
| 625 | struct dentry *root; | ||
| 626 | char *name; | ||
| 627 | |||
| 628 | name = kasprintf(GFP_KERNEL, "encoder-%d", encoder->index); | ||
| 629 | if (!name) | ||
| 630 | return; | ||
| 631 | |||
| 632 | root = debugfs_create_dir(name, minor->debugfs_root); | ||
| 633 | kfree(name); | ||
| 634 | |||
| 635 | encoder->debugfs_entry = root; | ||
| 636 | |||
| 637 | /* bridges list */ | ||
| 638 | debugfs_create_file("bridges", 0444, root, encoder, | ||
| 639 | &bridges_fops); | ||
| 640 | |||
| 641 | if (encoder->funcs && encoder->funcs->debugfs_init) | ||
| 642 | encoder->funcs->debugfs_init(encoder, root); | ||
| 643 | } | ||
| 644 | |||
| 645 | void drm_debugfs_encoder_remove(struct drm_encoder *encoder) | ||
| 646 | { | ||
| 647 | debugfs_remove_recursive(encoder->debugfs_entry); | ||
| 648 | encoder->debugfs_entry = NULL; | ||
| 649 | } | ||
| 650 | |||
| 592 | #endif /* CONFIG_DEBUG_FS */ | 651 | #endif /* CONFIG_DEBUG_FS */ |
| @@ -1,178 +0,0 @@ | |||
| 1 | /* | ||
| 2 | * \file drm_dma.c | ||
| 3 | * DMA IOCTL and function support | ||
| 4 | * | ||
| 5 | * \author Rickard E. (Rik) Faith <faith@valinux.com> | ||
| 6 | * \author Gareth Hughes <gareth@valinux.com> | ||
| 7 | */ | ||
| 8 | |||
| 9 | /* | ||
| 10 | * Created: Fri Mar 19 14:30:16 1999 by faith@valinux.com | ||
| 11 | * | ||
| 12 | * Copyright 1999, 2000 Precision Insight, Inc., Cedar Park, Texas. | ||
| 13 | * Copyright 2000 VA Linux Systems, Inc., Sunnyvale, California. | ||
| 14 | * All Rights Reserved. | ||
| 15 | * | ||
| 16 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 17 | * copy of this software and associated documentation files (the "Software"), | ||
| 18 | * to deal in the Software without restriction, including without limitation | ||
| 19 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 20 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 21 | * Software is furnished to do so, subject to the following conditions: | ||
| 22 | * | ||
| 23 | * The above copyright notice and this permission notice (including the next | ||
| 24 | * paragraph) shall be included in all copies or substantial portions of the | ||
| 25 | * Software. | ||
| 26 | * | ||
| 27 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 28 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 29 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 30 | * VA LINUX SYSTEMS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR | ||
| 31 | * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | ||
| 32 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | ||
| 33 | * OTHER DEALINGS IN THE SOFTWARE. | ||
| 34 | */ | ||
| 35 | |||
| 36 | #include <linux/export.h> | ||
| 37 | #include <linux/pci.h> | ||
| 38 | |||
| 39 | #include <drm/drm_drv.h> | ||
| 40 | #include <drm/drm_print.h> | ||
| 41 | |||
| 42 | #include "drm_legacy.h" | ||
| 43 | |||
| 44 | /** | ||
| 45 | * drm_legacy_dma_setup() - Initialize the DMA data. | ||
| 46 | * | ||
| 47 | * @dev: DRM device. | ||
| 48 | * Return: zero on success or a negative value on failure. | ||
| 49 | * | ||
| 50 | * Allocate and initialize a drm_device_dma structure. | ||
| 51 | */ | ||
| 52 | int drm_legacy_dma_setup(struct drm_device *dev) | ||
| 53 | { | ||
| 54 | int i; | ||
| 55 | |||
| 56 | if (!drm_core_check_feature(dev, DRIVER_HAVE_DMA) || | ||
| 57 | !drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 58 | return 0; | ||
| 59 | |||
| 60 | dev->buf_use = 0; | ||
| 61 | atomic_set(&dev->buf_alloc, 0); | ||
| 62 | |||
| 63 | dev->dma = kzalloc(sizeof(*dev->dma), GFP_KERNEL); | ||
| 64 | if (!dev->dma) | ||
| 65 | return -ENOMEM; | ||
| 66 | |||
| 67 | for (i = 0; i <= DRM_MAX_ORDER; i++) | ||
| 68 | memset(&dev->dma->bufs[i], 0, sizeof(dev->dma->bufs[0])); | ||
| 69 | |||
| 70 | return 0; | ||
| 71 | } | ||
| 72 | |||
| 73 | /** | ||
| 74 | * drm_legacy_dma_takedown() - Cleanup the DMA resources. | ||
| 75 | * | ||
| 76 | * @dev: DRM device. | ||
| 77 | * | ||
| 78 | * Free all pages associated with DMA buffers, the buffers and pages lists, and | ||
| 79 | * finally the drm_device::dma structure itself. | ||
| 80 | */ | ||
| 81 | void drm_legacy_dma_takedown(struct drm_device *dev) | ||
| 82 | { | ||
| 83 | struct drm_device_dma *dma = dev->dma; | ||
| 84 | drm_dma_handle_t *dmah; | ||
| 85 | int i, j; | ||
| 86 | |||
| 87 | if (!drm_core_check_feature(dev, DRIVER_HAVE_DMA) || | ||
| 88 | !drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 89 | return; | ||
| 90 | |||
| 91 | if (!dma) | ||
| 92 | return; | ||
| 93 | |||
| 94 | /* Clear dma buffers */ | ||
| 95 | for (i = 0; i <= DRM_MAX_ORDER; i++) { | ||
| 96 | if (dma->bufs[i].seg_count) { | ||
| 97 | DRM_DEBUG("order %d: buf_count = %d," | ||
| 98 | " seg_count = %d\n", | ||
| 99 | i, | ||
| 100 | dma->bufs[i].buf_count, | ||
| 101 | dma->bufs[i].seg_count); | ||
| 102 | for (j = 0; j < dma->bufs[i].seg_count; j++) { | ||
| 103 | if (dma->bufs[i].seglist[j]) { | ||
| 104 | dmah = dma->bufs[i].seglist[j]; | ||
| 105 | dma_free_coherent(dev->dev, | ||
| 106 | dmah->size, | ||
| 107 | dmah->vaddr, | ||
| 108 | dmah->busaddr); | ||
| 109 | kfree(dmah); | ||
| 110 | } | ||
| 111 | } | ||
| 112 | kfree(dma->bufs[i].seglist); | ||
| 113 | } | ||
| 114 | if (dma->bufs[i].buf_count) { | ||
| 115 | for (j = 0; j < dma->bufs[i].buf_count; j++) { | ||
| 116 | kfree(dma->bufs[i].buflist[j].dev_private); | ||
| 117 | } | ||
| 118 | kfree(dma->bufs[i].buflist); | ||
| 119 | } | ||
| 120 | } | ||
| 121 | |||
| 122 | kfree(dma->buflist); | ||
| 123 | kfree(dma->pagelist); | ||
| 124 | kfree(dev->dma); | ||
| 125 | dev->dma = NULL; | ||
| 126 | } | ||
| 127 | |||
| 128 | /** | ||
| 129 | * drm_legacy_free_buffer() - Free a buffer. | ||
| 130 | * | ||
| 131 | * @dev: DRM device. | ||
| 132 | * @buf: buffer to free. | ||
| 133 | * | ||
| 134 | * Resets the fields of \p buf. | ||
| 135 | */ | ||
| 136 | void drm_legacy_free_buffer(struct drm_device *dev, struct drm_buf * buf) | ||
| 137 | { | ||
| 138 | if (!buf) | ||
| 139 | return; | ||
| 140 | |||
| 141 | buf->waiting = 0; | ||
| 142 | buf->pending = 0; | ||
| 143 | buf->file_priv = NULL; | ||
| 144 | buf->used = 0; | ||
| 145 | } | ||
| 146 | |||
| 147 | /** | ||
| 148 | * drm_legacy_reclaim_buffers() - Reclaim the buffers. | ||
| 149 | * | ||
| 150 | * @dev: DRM device. | ||
| 151 | * @file_priv: DRM file private. | ||
| 152 | * | ||
| 153 | * Frees each buffer associated with \p file_priv not already on the hardware. | ||
| 154 | */ | ||
| 155 | void drm_legacy_reclaim_buffers(struct drm_device *dev, | ||
| 156 | struct drm_file *file_priv) | ||
| 157 | { | ||
| 158 | struct drm_device_dma *dma = dev->dma; | ||
| 159 | int i; | ||
| 160 | |||
| 161 | if (!dma) | ||
| 162 | return; | ||
| 163 | for (i = 0; i < dma->buf_count; i++) { | ||
| 164 | if (dma->buflist[i]->file_priv == file_priv) { | ||
| 165 | switch (dma->buflist[i]->list) { | ||
| 166 | case DRM_LIST_NONE: | ||
| 167 | drm_legacy_free_buffer(dev, dma->buflist[i]); | ||
| 168 | break; | ||
| 169 | case DRM_LIST_WAIT: | ||
| 170 | dma->buflist[i]->list = DRM_LIST_RECLAIM; | ||
| 171 | break; | ||
| 172 | default: | ||
| 173 | /* Buffer already on hardware. */ | ||
| 174 | break; | ||
| 175 | } | ||
| 176 | } | ||
| 177 | } | ||
| 178 | } | ||
| @@ -48,7 +48,6 @@ · 47 unmodified lines | |||
| 48 | 48 | ||
| 49 | #include "drm_crtc_internal.h" | 49 | #include "drm_crtc_internal.h" |
| 50 | #include "drm_internal.h" | 50 | #include "drm_internal.h" |
| 51 | #include "drm_legacy.h" | ||
| 52 | 51 | ||
| 53 | MODULE_AUTHOR("Gareth Hughes, Leif Delgass, José Fonseca, Jon Smirl"); | 52 | MODULE_AUTHOR("Gareth Hughes, Leif Delgass, José Fonseca, Jon Smirl"); |
| 54 | MODULE_DESCRIPTION("DRM shared core routines"); | 53 | MODULE_DESCRIPTION("DRM shared core routines"); |
| @@ -585,8 +584,6 @@ static void drm_fs_inode_free(struct inode *inode) · 530 unmodified lines | |||
| 585 | 584 | ||
| 586 | static void drm_dev_init_release(struct drm_device *dev, void *res) | 585 | static void drm_dev_init_release(struct drm_device *dev, void *res) |
| 587 | { | 586 | { |
| 588 | drm_legacy_ctxbitmap_cleanup(dev); | ||
| 589 | drm_legacy_remove_map_hash(dev); | ||
| 590 | drm_fs_inode_free(dev->anon_inode); | 587 | drm_fs_inode_free(dev->anon_inode); |
| 591 | 588 | ||
| 592 | put_device(dev->dev); | 589 | put_device(dev->dev); |
| @@ -597,7 +594,6 @@ static void drm_dev_init_release(struct drm_device *dev, void *res) · 4 unmodified lines | |||
| 597 | mutex_destroy(&dev->clientlist_mutex); | 594 | mutex_destroy(&dev->clientlist_mutex); |
| 598 | mutex_destroy(&dev->filelist_mutex); | 595 | mutex_destroy(&dev->filelist_mutex); |
| 599 | mutex_destroy(&dev->struct_mutex); | 596 | mutex_destroy(&dev->struct_mutex); |
| 600 | drm_legacy_destroy_members(dev); | ||
| 601 | } | 597 | } |
| 602 | 598 | ||
| 603 | static int drm_dev_init(struct drm_device *dev, | 599 | static int drm_dev_init(struct drm_device *dev, |
| @@ -632,7 +628,6 @@ static int drm_dev_init(struct drm_device *dev, · 28 unmodified lines | |||
| 632 | return -EINVAL; | 628 | return -EINVAL; |
| 633 | } | 629 | } |
| 634 | 630 | ||
| 635 | drm_legacy_init_members(dev); | ||
| 636 | INIT_LIST_HEAD(&dev->filelist); | 631 | INIT_LIST_HEAD(&dev->filelist); |
| 637 | INIT_LIST_HEAD(&dev->filelist_internal); | 632 | INIT_LIST_HEAD(&dev->filelist_internal); |
| 638 | INIT_LIST_HEAD(&dev->clientlist); | 633 | INIT_LIST_HEAD(&dev->clientlist); |
| @@ -673,12 +668,6 @@ static int drm_dev_init(struct drm_device *dev, · 34 unmodified lines | |||
| 673 | goto err; | 668 | goto err; |
| 674 | } | 669 | } |
| 675 | 670 | ||
| 676 | ret = drm_legacy_create_map_hash(dev); | ||
| 677 | if (ret) | ||
| 678 | goto err; | ||
| 679 | |||
| 680 | drm_legacy_ctxbitmap_init(dev); | ||
| 681 | |||
| 682 | if (drm_core_check_feature(dev, DRIVER_GEM)) { | 671 | if (drm_core_check_feature(dev, DRIVER_GEM)) { |
| 683 | ret = drm_gem_init(dev); | 672 | ret = drm_gem_init(dev); |
| 684 | if (ret) { | 673 | if (ret) { |
| @@ -949,8 +938,11 @@ int drm_dev_register(struct drm_device *dev, unsigned long flags) · 264 unmodified lines | |||
| 949 | goto err_minors; | 938 | goto err_minors; |
| 950 | } | 939 | } |
| 951 | 940 | ||
| 952 | if (drm_core_check_feature(dev, DRIVER_MODESET)) | 941 | if (drm_core_check_feature(dev, DRIVER_MODESET)) { |
| 953 | drm_modeset_register_all(dev); | 942 | ret = drm_modeset_register_all(dev); |
| 943 | if (ret) | ||
| 944 | goto err_unload; | ||
| 945 | } | ||
| 954 | 946 | ||
| 955 | DRM_INFO("Initialized %s %d.%d.%d %s for %s on minor %d\n", | 947 | DRM_INFO("Initialized %s %d.%d.%d %s for %s on minor %d\n", |
| 956 | driver->name, driver->major, driver->minor, | 948 | driver->name, driver->major, driver->minor, |
| @@ -960,6 +952,9 @@ int drm_dev_register(struct drm_device *dev, unsigned long flags) · 3 unmodified lines | |||
| 960 | 952 | ||
| 961 | goto out_unlock; | 953 | goto out_unlock; |
| 962 | 954 | ||
| 955 | err_unload: | ||
| 956 | if (dev->driver->unload) | ||
| 957 | dev->driver->unload(dev); | ||
| 963 | err_minors: | 958 | err_minors: |
| 964 | remove_compat_control_link(dev); | 959 | remove_compat_control_link(dev); |
| 965 | drm_minor_unregister(dev, DRM_MINOR_ACCEL); | 960 | drm_minor_unregister(dev, DRM_MINOR_ACCEL); |
| @@ -990,9 +985,6 @@ EXPORT_SYMBOL(drm_dev_register); · 24 unmodified lines | |||
| 990 | */ | 985 | */ |
| 991 | void drm_dev_unregister(struct drm_device *dev) | 986 | void drm_dev_unregister(struct drm_device *dev) |
| 992 | { | 987 | { |
| 993 | if (drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 994 | drm_lastclose(dev); | ||
| 995 | |||
| 996 | dev->registered = false; | 988 | dev->registered = false; |
| 997 | 989 | ||
| 998 | drm_client_dev_unregister(dev); | 990 | drm_client_dev_unregister(dev); |
| @@ -1003,9 +995,6 @@ void drm_dev_unregister(struct drm_device *dev) · 4 unmodified lines | |||
| 1003 | if (dev->driver->unload) | 995 | if (dev->driver->unload) |
| 1004 | dev->driver->unload(dev); | 996 | dev->driver->unload(dev); |
| 1005 | 997 | ||
| 1006 | drm_legacy_pci_agp_destroy(dev); | ||
| 1007 | drm_legacy_rmmaps(dev); | ||
| 1008 | |||
| 1009 | remove_compat_control_link(dev); | 998 | remove_compat_control_link(dev); |
| 1010 | drm_minor_unregister(dev, DRM_MINOR_ACCEL); | 999 | drm_minor_unregister(dev, DRM_MINOR_ACCEL); |
| 1011 | drm_minor_unregister(dev, DRM_MINOR_PRIMARY); | 1000 | drm_minor_unregister(dev, DRM_MINOR_PRIMARY); |
| @@ -41,10 +41,12 @@ · 40 unmodified lines | |||
| 41 | #include <drm/drm_displayid.h> | 41 | #include <drm/drm_displayid.h> |
| 42 | #include <drm/drm_drv.h> | 42 | #include <drm/drm_drv.h> |
| 43 | #include <drm/drm_edid.h> | 43 | #include <drm/drm_edid.h> |
| 44 | #include <drm/drm_eld.h> | ||
| 44 | #include <drm/drm_encoder.h> | 45 | #include <drm/drm_encoder.h> |
| 45 | #include <drm/drm_print.h> | 46 | #include <drm/drm_print.h> |
| 46 | 47 | ||
| 47 | #include "drm_crtc_internal.h" | 48 | #include "drm_crtc_internal.h" |
| 49 | #include "drm_internal.h" | ||
| 48 | 50 | ||
| 49 | static int oui(u8 first, u8 second, u8 third) | 51 | static int oui(u8 first, u8 second, u8 third) |
| 50 | { | 52 | { |
| @@ -5509,6 +5511,27 @@ static void clear_eld(struct drm_connector *connector) · 5458 unmodified lines | |||
| 5509 | connector->audio_latency[1] = 0; | 5511 | connector->audio_latency[1] = 0; |
| 5510 | } | 5512 | } |
| 5511 | 5513 | ||
| 5514 | /* | ||
| 5515 | * Get 3-byte SAD buffer from struct cea_sad. | ||
| 5516 | */ | ||
| 5517 | void drm_edid_cta_sad_get(const struct cea_sad *cta_sad, u8 *sad) | ||
| 5518 | { | ||
| 5519 | sad[0] = cta_sad->format << 3 | cta_sad->channels; | ||
| 5520 | sad[1] = cta_sad->freq; | ||
| 5521 | sad[2] = cta_sad->byte2; | ||
| 5522 | } | ||
| 5523 | |||
| 5524 | /* | ||
| 5525 | * Set struct cea_sad from 3-byte SAD buffer. | ||
| 5526 | */ | ||
| 5527 | void drm_edid_cta_sad_set(struct cea_sad *cta_sad, const u8 *sad) | ||
| 5528 | { | ||
| 5529 | cta_sad->format = (sad[0] & 0x78) >> 3; | ||
| 5530 | cta_sad->channels = sad[0] & 0x07; | ||
| 5531 | cta_sad->freq = sad[1] & 0x7f; | ||
| 5532 | cta_sad->byte2 = sad[2]; | ||
| 5533 | } | ||
| 5534 | |||
| 5512 | /* | 5535 | /* |
| 5513 | * drm_edid_to_eld - build ELD from EDID | 5536 | * drm_edid_to_eld - build ELD from EDID |
| 5514 | * @connector: connector corresponding to the HDMI/DP sink | 5537 | * @connector: connector corresponding to the HDMI/DP sink |
| @@ -5593,7 +5616,7 @@ static void drm_edid_to_eld(struct drm_connector *connector, · 78 unmodified lines | |||
| 5593 | } | 5616 | } |
| 5594 | 5617 | ||
| 5595 | static int _drm_edid_to_sad(const struct drm_edid *drm_edid, | 5618 | static int _drm_edid_to_sad(const struct drm_edid *drm_edid, |
| 5596 | struct cea_sad **sads) | 5619 | struct cea_sad **psads) |
| 5597 | { | 5620 | { |
| 5598 | const struct cea_db *db; | 5621 | const struct cea_db *db; |
| 5599 | struct cea_db_iter iter; | 5622 | struct cea_db_iter iter; |
| @@ -5602,20 +5625,16 @@ static int _drm_edid_to_sad(const struct drm_edid *drm_edid, · 2 unmodified lines | |||
| 5602 | cea_db_iter_edid_begin(drm_edid, &iter); | 5625 | cea_db_iter_edid_begin(drm_edid, &iter); |
| 5603 | cea_db_iter_for_each(db, &iter) { | 5626 | cea_db_iter_for_each(db, &iter) { |
| 5604 | if (cea_db_tag(db) == CTA_DB_AUDIO) { | 5627 | if (cea_db_tag(db) == CTA_DB_AUDIO) { |
| 5605 | int j; | 5628 | struct cea_sad *sads; |
| 5629 | int i; | ||
| 5606 | 5630 | ||
| 5607 | count = cea_db_payload_len(db) / 3; /* SAD is 3B */ | 5631 | count = cea_db_payload_len(db) / 3; /* SAD is 3B */ |
| 5608 | *sads = kcalloc(count, sizeof(**sads), GFP_KERNEL); | 5632 | sads = kcalloc(count, sizeof(*sads), GFP_KERNEL); |
| 5609 | if (!*sads) | 5633 | *psads = sads; |
| 5634 | if (!sads) | ||
| 5610 | return -ENOMEM; | 5635 | return -ENOMEM; |
| 5611 | for (j = 0; j < count; j++) { | 5636 | for (i = 0; i < count; i++) |
| 5612 | const u8 *sad = &db->data[j * 3]; | 5637 | drm_edid_cta_sad_set(&sads[i], &db->data[i * 3]); |
| 5613 | |||
| 5614 | (*sads)[j].format = (sad[0] & 0x78) >> 3; | ||
| 5615 | (*sads)[j].channels = sad[0] & 0x7; | ||
| 5616 | (*sads)[j].freq = sad[1] & 0x7F; | ||
| 5617 | (*sads)[j].byte2 = sad[2]; | ||
| 5618 | } | ||
| 5619 | break; | 5638 | break; |
| 5620 | } | 5639 | } |
| 5621 | } | 5640 | } |
| @@ -23,22 +23,6 @@ module_param_string(edid_firmware, edid_firmware, sizeof(edid_firmware), 0644); · 22 unmodified lines | |||
| 23 | MODULE_PARM_DESC(edid_firmware, "Do not probe monitor, use specified EDID blob " | 23 | MODULE_PARM_DESC(edid_firmware, "Do not probe monitor, use specified EDID blob " |
| 24 | "from built-in data or /lib/firmware instead. "); | 24 | "from built-in data or /lib/firmware instead. "); |
| 25 | 25 | ||
| 26 | /* Use only for backward compatibility with drm_kms_helper.edid_firmware */ | ||
| 27 | int __drm_set_edid_firmware_path(const char *path) | ||
| 28 | { | ||
| 29 | scnprintf(edid_firmware, sizeof(edid_firmware), "%s", path); | ||
| 30 | |||
| 31 | return 0; | ||
| 32 | } | ||
| 33 | EXPORT_SYMBOL(__drm_set_edid_firmware_path); | ||
| 34 | |||
| 35 | /* Use only for backward compatibility with drm_kms_helper.edid_firmware */ | ||
| 36 | int __drm_get_edid_firmware_path(char *buf, size_t bufsize) | ||
| 37 | { | ||
| 38 | return scnprintf(buf, bufsize, "%s", edid_firmware); | ||
| 39 | } | ||
| 40 | EXPORT_SYMBOL(__drm_get_edid_firmware_path); | ||
| 41 | |||
| 42 | #define GENERIC_EDIDS 6 | 26 | #define GENERIC_EDIDS 6 |
| 43 | static const char * const generic_edid_name[GENERIC_EDIDS] = { | 27 | static const char * const generic_edid_name[GENERIC_EDIDS] = { |
| 44 | "edid/800x600.bin", | 28 | "edid/800x600.bin", |
| @@ -0,0 +1,55 @@ | |||
| 1 | // SPDX-License-Identifier: MIT | ||
| 2 | /* | ||
| 3 | * Copyright © 2023 Intel Corporation | ||
| 4 | */ | ||
| 5 | |||
| 6 | #include <drm/drm_edid.h> | ||
| 7 | #include <drm/drm_eld.h> | ||
| 8 | |||
| 9 | #include "drm_internal.h" | ||
| 10 | |||
| 11 | /** | ||
| 12 | * drm_eld_sad_get - get SAD from ELD to struct cea_sad | ||
| 13 | * @eld: ELD buffer | ||
| 14 | * @sad_index: SAD index | ||
| 15 | * @cta_sad: destination struct cea_sad | ||
| 16 | * | ||
| 17 | * @return: 0 on success, or negative on errors | ||
| 18 | */ | ||
| 19 | int drm_eld_sad_get(const u8 *eld, int sad_index, struct cea_sad *cta_sad) | ||
| 20 | { | ||
| 21 | const u8 *sad; | ||
| 22 | |||
| 23 | if (sad_index >= drm_eld_sad_count(eld)) | ||
| 24 | return -EINVAL; | ||
| 25 | |||
| 26 | sad = eld + DRM_ELD_CEA_SAD(drm_eld_mnl(eld), sad_index); | ||
| 27 | |||
| 28 | drm_edid_cta_sad_set(cta_sad, sad); | ||
| 29 | |||
| 30 | return 0; | ||
| 31 | } | ||
| 32 | EXPORT_SYMBOL(drm_eld_sad_get); | ||
| 33 | |||
| 34 | /** | ||
| 35 | * drm_eld_sad_set - set SAD to ELD from struct cea_sad | ||
| 36 | * @eld: ELD buffer | ||
| 37 | * @sad_index: SAD index | ||
| 38 | * @cta_sad: source struct cea_sad | ||
| 39 | * | ||
| 40 | * @return: 0 on success, or negative on errors | ||
| 41 | */ | ||
| 42 | int drm_eld_sad_set(u8 *eld, int sad_index, const struct cea_sad *cta_sad) | ||
| 43 | { | ||
| 44 | u8 *sad; | ||
| 45 | |||
| 46 | if (sad_index >= drm_eld_sad_count(eld)) | ||
| 47 | return -EINVAL; | ||
| 48 | |||
| 49 | sad = eld + DRM_ELD_CEA_SAD(drm_eld_mnl(eld), sad_index); | ||
| 50 | |||
| 51 | drm_edid_cta_sad_get(cta_sad, sad); | ||
| 52 | |||
| 53 | return 0; | ||
| 54 | } | ||
| 55 | EXPORT_SYMBOL(drm_eld_sad_set); | ||
| @@ -30,6 +30,7 @@ · 29 unmodified lines | |||
| 30 | #include <drm/drm_print.h> | 30 | #include <drm/drm_print.h> |
| 31 | 31 | ||
| 32 | #include "drm_crtc_internal.h" | 32 | #include "drm_crtc_internal.h" |
| 33 | #include "drm_internal.h" | ||
| 33 | 34 | ||
| 34 | /** | 35 | /** |
| 35 | * DOC: overview | 36 | * DOC: overview |
| @@ -74,6 +75,8 @@ int drm_encoder_register_all(struct drm_device *dev) · 38 unmodified lines | |||
| 74 | int ret = 0; | 75 | int ret = 0; |
| 75 | 76 | ||
| 76 | drm_for_each_encoder(encoder, dev) { | 77 | drm_for_each_encoder(encoder, dev) { |
| 78 | drm_debugfs_encoder_add(encoder); | ||
| 79 | |||
| 77 | if (encoder->funcs && encoder->funcs->late_register) | 80 | if (encoder->funcs && encoder->funcs->late_register) |
| 78 | ret = encoder->funcs->late_register(encoder); | 81 | ret = encoder->funcs->late_register(encoder); |
| 79 | if (ret) | 82 | if (ret) |
| @@ -90,6 +93,7 @@ void drm_encoder_unregister_all(struct drm_device *dev) · 10 unmodified lines | |||
| 90 | drm_for_each_encoder(encoder, dev) { | 93 | drm_for_each_encoder(encoder, dev) { |
| 91 | if (encoder->funcs && encoder->funcs->early_unregister) | 94 | if (encoder->funcs && encoder->funcs->early_unregister) |
| 92 | encoder->funcs->early_unregister(encoder); | 95 | encoder->funcs->early_unregister(encoder); |
| 96 | drm_debugfs_encoder_remove(encoder); | ||
| 93 | } | 97 | } |
| 94 | } | 98 | } |
| 95 | 99 | ||
| @@ -47,21 +47,12 @@ · 46 unmodified lines | |||
| 47 | 47 | ||
| 48 | #include "drm_crtc_internal.h" | 48 | #include "drm_crtc_internal.h" |
| 49 | #include "drm_internal.h" | 49 | #include "drm_internal.h" |
| 50 | #include "drm_legacy.h" | ||
| 51 | 50 | ||
| 52 | /* from BKL pushdown */ | 51 | /* from BKL pushdown */ |
| 53 | DEFINE_MUTEX(drm_global_mutex); | 52 | DEFINE_MUTEX(drm_global_mutex); |
| 54 | 53 | ||
| 55 | bool drm_dev_needs_global_mutex(struct drm_device *dev) | 54 | bool drm_dev_needs_global_mutex(struct drm_device *dev) |
| 56 | { | 55 | { |
| 57 | /* | ||
| 58 | * Legacy drivers rely on all kinds of BKL locking semantics, don't | ||
| 59 | * bother. They also still need BKL locking for their ioctls, so better | ||
| 60 | * safe than sorry. | ||
| 61 | */ | ||
| 62 | if (drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 63 | return true; | ||
| 64 | |||
| 65 | /* | 56 | /* |
| 66 | * The deprecated ->load callback must be called after the driver is | 57 | * The deprecated ->load callback must be called after the driver is |
| 67 | * already registered. This means such drivers rely on the BKL to make | 58 | * already registered. This means such drivers rely on the BKL to make |
| @@ -107,9 +98,7 @@ bool drm_dev_needs_global_mutex(struct drm_device *dev) · 39 unmodified lines | |||
| 107 | * drm_send_event() as the main starting points. | 98 | * drm_send_event() as the main starting points. |
| 108 | * | 99 | * |
| 109 | * The memory mapping implementation will vary depending on how the driver | 100 | * The memory mapping implementation will vary depending on how the driver |
| 110 | * manages memory. Legacy drivers will use the deprecated drm_legacy_mmap() | 101 | * manages memory. For GEM-based drivers this is drm_gem_mmap(). |
| 111 | * function, modern drivers should use one of the provided memory-manager | ||
| 112 | * specific implementations. For GEM-based drivers this is drm_gem_mmap(). | ||
| 113 | * | 102 | * |
| 114 | * No other file operations are supported by the DRM userspace API. Overall the | 103 | * No other file operations are supported by the DRM userspace API. Overall the |
| 115 | * following is an example &file_operations structure:: | 104 | * following is an example &file_operations structure:: |
| @@ -254,18 +243,6 @@ void drm_file_free(struct drm_file *file) · 138 unmodified lines | |||
| 254 | (long)old_encode_dev(file->minor->kdev->devt), | 243 | (long)old_encode_dev(file->minor->kdev->devt), |
| 255 | atomic_read(&dev->open_count)); | 244 | atomic_read(&dev->open_count)); |
| 256 | 245 | ||
| 257 | #ifdef CONFIG_DRM_LEGACY | ||
| 258 | if (drm_core_check_feature(dev, DRIVER_LEGACY) && | ||
| 259 | dev->driver->preclose) | ||
| 260 | dev->driver->preclose(dev, file); | ||
| 261 | #endif | ||
| 262 | |||
| 263 | if (drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 264 | drm_legacy_lock_release(dev, file->filp); | ||
| 265 | |||
| 266 | if (drm_core_check_feature(dev, DRIVER_HAVE_DMA)) | ||
| 267 | drm_legacy_reclaim_buffers(dev, file); | ||
| 268 | |||
| 269 | drm_events_release(file); | 246 | drm_events_release(file); |
| 270 | 247 | ||
| 271 | if (drm_core_check_feature(dev, DRIVER_MODESET)) { | 248 | if (drm_core_check_feature(dev, DRIVER_MODESET)) { |
| @@ -279,8 +256,6 @@ void drm_file_free(struct drm_file *file) · 7 unmodified lines | |||
| 279 | if (drm_core_check_feature(dev, DRIVER_GEM)) | 256 | if (drm_core_check_feature(dev, DRIVER_GEM)) |
| 280 | drm_gem_release(dev, file); | 257 | drm_gem_release(dev, file); |
| 281 | 258 | ||
| 282 | drm_legacy_ctxbitmap_flush(dev, file); | ||
| 283 | |||
| 284 | if (drm_is_primary_client(file)) | 259 | if (drm_is_primary_client(file)) |
| 285 | drm_master_release(file); | 260 | drm_master_release(file); |
| 286 | 261 | ||
| @@ -367,29 +342,6 @@ int drm_open_helper(struct file *filp, struct drm_minor *minor) · 80 unmodified lines | |||
| 367 | list_add(&priv->lhead, &dev->filelist); | 342 | list_add(&priv->lhead, &dev->filelist); |
| 368 | mutex_unlock(&dev->filelist_mutex); | 343 | mutex_unlock(&dev->filelist_mutex); |
| 369 | 344 | ||
| 370 | #ifdef CONFIG_DRM_LEGACY | ||
| 371 | #ifdef __alpha__ | ||
| 372 | /* | ||
| 373 | * Default the hose | ||
| 374 | */ | ||
| 375 | if (!dev->hose) { | ||
| 376 | struct pci_dev *pci_dev; | ||
| 377 | |||
| 378 | pci_dev = pci_get_class(PCI_CLASS_DISPLAY_VGA << 8, NULL); | ||
| 379 | if (pci_dev) { | ||
| 380 | dev->hose = pci_dev->sysdata; | ||
| 381 | pci_dev_put(pci_dev); | ||
| 382 | } | ||
| 383 | if (!dev->hose) { | ||
| 384 | struct pci_bus *b = list_entry(pci_root_buses.next, | ||
| 385 | struct pci_bus, node); | ||
| 386 | if (b) | ||
| 387 | dev->hose = b->sysdata; | ||
| 388 | } | ||
| 389 | } | ||
| 390 | #endif | ||
| 391 | #endif | ||
| 392 | |||
| 393 | return 0; | 345 | return 0; |
| 394 | } | 346 | } |
| 395 | 347 | ||
| @@ -411,7 +363,6 @@ int drm_open(struct inode *inode, struct file *filp) · 15 unmodified lines | |||
| 411 | struct drm_device *dev; | 363 | struct drm_device *dev; |
| 412 | struct drm_minor *minor; | 364 | struct drm_minor *minor; |
| 413 | int retcode; | 365 | int retcode; |
| 414 | int need_setup = 0; | ||
| 415 | 366 | ||
| 416 | minor = drm_minor_acquire(iminor(inode)); | 367 | minor = drm_minor_acquire(iminor(inode)); |
| 417 | if (IS_ERR(minor)) | 368 | if (IS_ERR(minor)) |
| @@ -421,8 +372,7 @@ int drm_open(struct inode *inode, struct file *filp) · 3 unmodified lines | |||
| 421 | if (drm_dev_needs_global_mutex(dev)) | 372 | if (drm_dev_needs_global_mutex(dev)) |
| 422 | mutex_lock(&drm_global_mutex); | 373 | mutex_lock(&drm_global_mutex); |
| 423 | 374 | ||
| 424 | if (!atomic_fetch_inc(&dev->open_count)) | 375 | atomic_fetch_inc(&dev->open_count); |
| 425 | need_setup = 1; | ||
| 426 | 376 | ||
| 427 | /* share address_space across all char-devs of a single device */ | 377 | /* share address_space across all char-devs of a single device */ |
| 428 | filp->f_mapping = dev->anon_inode->i_mapping; | 378 | filp->f_mapping = dev->anon_inode->i_mapping; |
| @@ -430,13 +380,6 @@ int drm_open(struct inode *inode, struct file *filp) · 1 unmodified line | |||
| 430 | retcode = drm_open_helper(filp, minor); | 380 | retcode = drm_open_helper(filp, minor); |
| 431 | if (retcode) | 381 | if (retcode) |
| 432 | goto err_undo; | 382 | goto err_undo; |
| 433 | if (need_setup) { | ||
| 434 | retcode = drm_legacy_setup(dev); | ||
| 435 | if (retcode) { | ||
| 436 | drm_close_helper(filp); | ||
| 437 | goto err_undo; | ||
| 438 | } | ||
| 439 | } | ||
| 440 | 383 | ||
| 441 | if (drm_dev_needs_global_mutex(dev)) | 384 | if (drm_dev_needs_global_mutex(dev)) |
| 442 | mutex_unlock(&drm_global_mutex); | 385 | mutex_unlock(&drm_global_mutex); |
| @@ -460,9 +403,6 @@ void drm_lastclose(struct drm_device * dev) · 17 unmodified lines | |||
| 460 | dev->driver->lastclose(dev); | 403 | dev->driver->lastclose(dev); |
| 461 | drm_dbg_core(dev, "driver lastclose completed\n"); | 404 | drm_dbg_core(dev, "driver lastclose completed\n"); |
| 462 | 405 | ||
| 463 | if (drm_core_check_feature(dev, DRIVER_LEGACY)) | ||
| 464 | drm_legacy_dev_reinit(dev); | ||
| 465 | |||
| 466 | drm_client_dev_restore(dev); | 406 | drm_client_dev_restore(dev); |
| 467 | } | 407 | } |
| 468 | 408 | ||
| @@ -913,7 +853,7 @@ static void print_size(struct drm_printer *p, const char *stat, · 444 unmodified lines | |||
| 913 | unsigned u; | 853 | unsigned u; |
| 914 | 854 | ||
| 915 | for (u = 0; u < ARRAY_SIZE(units) - 1; u++) { | 855 | for (u = 0; u < ARRAY_SIZE(units) - 1; u++) { |
| 916 | if (sz < SZ_1K) | 856 | if (sz == 0 || !IS_ALIGNED(sz, SZ_1K)) |
| 917 | break; | 857 | break; |
| 918 | sz = div_u64(sz, SZ_1K); | 858 | sz = div_u64(sz, SZ_1K); |
| 919 | } | 859 | } |
| @@ -958,7 +898,7 @@ void drm_show_memory_stats(struct drm_printer *p, struct drm_file *file) · 38 unmodified lines | |||
| 958 | { | 898 | { |
| 959 | struct drm_gem_object *obj; | 899 | struct drm_gem_object *obj; |
| 960 | struct drm_memory_stats status = {}; | 900 | struct drm_memory_stats status = {}; |
| 961 | enum drm_gem_object_status supported_status; | 901 | enum drm_gem_object_status supported_status = 0; |
| 962 | int id; | 902 | int id; |
| 963 | 903 | ||
| 964 | spin_lock(&file->table_lock); | 904 | spin_lock(&file->table_lock); |
| @@ -27,14 +27,12 @@ · 26 unmodified lines | |||
| 27 | #include <drm/drm_print.h> | 27 | #include <drm/drm_print.h> |
| 28 | #include <drm/drm_util.h> | 28 | #include <drm/drm_util.h> |
| 29 | 29 | ||
| 30 | /** | 30 | struct drm_flip_task { |
| 31 | * drm_flip_work_allocate_task - allocate a flip-work task | 31 | struct list_head node; |
| 32 | * @data: data associated to the task | 32 | void *data; |
| 33 | * @flags: allocator flags | 33 | }; |
| 34 | * | 34 | |
| 35 | * Allocate a drm_flip_task object and attach private data to it. | 35 | static struct drm_flip_task *drm_flip_work_allocate_task(void *data, gfp_t flags) |
| 36 | */ | ||
| 37 | struct drm_flip_task *drm_flip_work_allocate_task(void *data, gfp_t flags) | ||
| 38 | { | 36 | { |
| 39 | struct drm_flip_task *task; | 37 | struct drm_flip_task *task; |
| 40 | 38 | ||
| @@ -44,18 +42,8 @@ struct drm_flip_task *drm_flip_work_allocate_task(void *data, gfp_t flags) · 3 unmodified lines | |||
| 44 | 42 | ||
| 45 | return task; | 43 | return task; |
| 46 | } | 44 | } |
| 47 | EXPORT_SYMBOL(drm_flip_work_allocate_task); | ||
| 48 | 45 | ||
| 49 | /** | 46 | static void drm_flip_work_queue_task(struct drm_flip_work *work, struct drm_flip_task *task) |
| 50 | * drm_flip_work_queue_task - queue a specific task | ||
| 51 | * @work: the flip-work | ||
| 52 | * @task: the task to handle | ||
| 53 | * | ||
| 54 | * Queues task, that will later be run (passed back to drm_flip_func_t | ||
| 55 | * func) on a work queue after drm_flip_work_commit() is called. | ||
| 56 | */ | ||
| 57 | void drm_flip_work_queue_task(struct drm_flip_work *work, | ||
| 58 | struct drm_flip_task *task) | ||
| 59 | { | 47 | { |
| 60 | unsigned long flags; | 48 | unsigned long flags; |
| 61 | 49 | ||
| @@ -63,7 +51,6 @@ void drm_flip_work_queue_task(struct drm_flip_work *work, · 1 unmodified line | |||
| 63 | list_add_tail(&task->node, &work->queued); | 51 | list_add_tail(&task->node, &work->queued); |
| 64 | spin_unlock_irqrestore(&work->lock, flags); | 52 | spin_unlock_irqrestore(&work->lock, flags); |
| 65 | } | 53 | } |
| 66 | EXPORT_SYMBOL(drm_flip_work_queue_task); | ||
| 67 | 54 | ||
| 68 | /** | 55 | /** |
| 69 | * drm_flip_work_queue - queue work | 56 | * drm_flip_work_queue - queue work |
| @@ -20,6 +20,97 @@ · 19 unmodified lines | |||
| 20 | #include <drm/drm_print.h> | 20 | #include <drm/drm_print.h> |
| 21 | #include <drm/drm_rect.h> | 21 | #include <drm/drm_rect.h> |
| 22 | 22 | ||
| 23 | /** | ||
| 24 | * drm_format_conv_state_init - Initialize format-conversion state | ||
| 25 | * @state: The state to initialize | ||
| 26 | * | ||
| 27 | * Clears all fields in struct drm_format_conv_state. The state will | ||
| 28 | * be empty with no preallocated resources. | ||
| 29 | */ | ||
| 30 | void drm_format_conv_state_init(struct drm_format_conv_state *state) | ||
| 31 | { | ||
| 32 | state->tmp.mem = NULL; | ||
| 33 | state->tmp.size = 0; | ||
| 34 | state->tmp.preallocated = false; | ||
| 35 | } | ||
| 36 | EXPORT_SYMBOL(drm_format_conv_state_init); | ||
| 37 | |||
| 38 | /** | ||
| 39 | * drm_format_conv_state_copy - Copy format-conversion state | ||
| 40 | * @state: Destination state | ||
| 41 | * @old_state: Source state | ||
| 42 | * | ||
| 43 | * Copies format-conversion state from @old_state to @state; except for | ||
| 44 | * temporary storage. | ||
| 45 | */ | ||
| 46 | void drm_format_conv_state_copy(struct drm_format_conv_state *state, | ||
| 47 | const struct drm_format_conv_state *old_state) | ||
| 48 | { | ||
| 49 | /* | ||
| 50 | * So far, there's only temporary storage here, which we don't | ||
| 51 | * duplicate. Just clear the fields. | ||
| 52 | */ | ||
| 53 | state->tmp.mem = NULL; | ||
| 54 | state->tmp.size = 0; | ||
| 55 | state->tmp.preallocated = false; | ||
| 56 | } | ||
| 57 | EXPORT_SYMBOL(drm_format_conv_state_copy); | ||
| 58 | |||
| 59 | /** | ||
| 60 | * drm_format_conv_state_reserve - Allocates storage for format conversion | ||
| 61 | * @state: The format-conversion state | ||
| 62 | * @new_size: The minimum allocation size | ||
| 63 | * @flags: Flags for kmalloc() | ||
| 64 | * | ||
| 65 | * Allocates at least @new_size bytes and returns a pointer to the memory | ||
| 66 | * range. After calling this function, previously returned memory blocks | ||
| 67 | * are invalid. It's best to collect all memory requirements of a format | ||
| 68 | * conversion and call this function once to allocate the range. | ||
| 69 | * | ||
| 70 | * Returns: | ||
| 71 | * A pointer to the allocated memory range, or NULL otherwise. | ||
| 72 | */ | ||
| 73 | void *drm_format_conv_state_reserve(struct drm_format_conv_state *state, | ||
| 74 | size_t new_size, gfp_t flags) | ||
| 75 | { | ||
| 76 | void *mem; | ||
| 77 | |||
| 78 | if (new_size <= state->tmp.size) | ||
| 79 | goto out; | ||
| 80 | else if (state->tmp.preallocated) | ||
| 81 | return NULL; | ||
| 82 | |||
| 83 | mem = krealloc(state->tmp.mem, new_size, flags); | ||
| 84 | if (!mem) | ||
| 85 | return NULL; | ||
| 86 | |||
| 87 | state->tmp.mem = mem; | ||
| 88 | state->tmp.size = new_size; | ||
| 89 | |||
| 90 | out: | ||
| 91 | return state->tmp.mem; | ||
| 92 | } | ||
| 93 | EXPORT_SYMBOL(drm_format_conv_state_reserve); | ||
| 94 | |||
| 95 | /** | ||
| 96 | * drm_format_conv_state_release - Releases an format-conversion storage | ||
| 97 | * @state: The format-conversion state | ||
| 98 | * | ||
| 99 | * Releases the memory range references by the format-conversion state. | ||
| 100 | * After this call, all pointers to the memory are invalid. Prefer | ||
| 101 | * drm_format_conv_state_init() for cleaning up and unloading a driver. | ||
| 102 | */ | ||
| 103 | void drm_format_conv_state_release(struct drm_format_conv_state *state) | ||
| 104 | { | ||
| 105 | if (state->tmp.preallocated) | ||
| 106 | return; | ||
| 107 | |||
| 108 | kfree(state->tmp.mem); | ||
| 109 | state->tmp.mem = NULL; | ||
| 110 | state->tmp.size = 0; | ||
| 111 | } | ||
| 112 | EXPORT_SYMBOL(drm_format_conv_state_release); | ||
| 113 | |||
| 23 | static unsigned int clip_offset(const struct drm_rect *clip, unsigned int pitch, unsigned int cpp) | 114 | static unsigned int clip_offset(const struct drm_rect *clip, unsigned int pitch, unsigned int cpp) |
| 24 | { | 115 | { |
| 25 | return clip->y1 * pitch + clip->x1 * cpp; | 116 | return clip->y1 * pitch + clip->x1 * cpp; |
| @@ -45,6 +136,7 @@ EXPORT_SYMBOL(drm_fb_clip_offset); · 19 unmodified lines | |||
| 45 | static int __drm_fb_xfrm(void *dst, unsigned long dst_pitch, unsigned long dst_pixsize, | 136 | static int __drm_fb_xfrm(void *dst, unsigned long dst_pitch, unsigned long dst_pixsize, |
| 46 | const void *vaddr, const struct drm_framebuffer *fb, | 137 | const void *vaddr, const struct drm_framebuffer *fb, |
| 47 | const struct drm_rect *clip, bool vaddr_cached_hint, | 138 | const struct drm_rect *clip, bool vaddr_cached_hint, |
| 139 | struct drm_format_conv_state *state, | ||
| 48 | void (*xfrm_line)(void *dbuf, const void *sbuf, unsigned int npixels)) | 140 | void (*xfrm_line)(void *dbuf, const void *sbuf, unsigned int npixels)) |
| 49 | { | 141 | { |
| 50 | unsigned long linepixels = drm_rect_width(clip); | 142 | unsigned long linepixels = drm_rect_width(clip); |
| @@ -60,7 +152,7 @@ static int __drm_fb_xfrm(void *dst, unsigned long dst_pitch, unsigned long dst_p · 9 unmodified lines | |||
| 60 | * one line at a time. | 152 | * one line at a time. |
| 61 | */ | 153 | */ |
| 62 | if (!vaddr_cached_hint) { | 154 | if (!vaddr_cached_hint) { |
| 63 | stmp = kmalloc(sbuf_len, GFP_KERNEL); | 155 | stmp = drm_format_conv_state_reserve(state, sbuf_len, GFP_KERNEL); |
| 64 | if (!stmp) | 156 | if (!stmp) |
| 65 | return -ENOMEM; | 157 | return -ENOMEM; |
| 66 | } | 158 | } |
| @@ -79,8 +171,6 @@ static int __drm_fb_xfrm(void *dst, unsigned long dst_pitch, unsigned long dst_p · 12 unmodified lines | |||
| 79 | dst += dst_pitch; | 171 | dst += dst_pitch; |
| 80 | } | 172 | } |
| 81 | 173 | ||
| 82 | kfree(stmp); | ||
| 83 | |||
| 84 | return 0; | 174 | return 0; |
| 85 | } | 175 | } |
| 86 | 176 | ||
| @@ -88,6 +178,7 @@ static int __drm_fb_xfrm(void *dst, unsigned long dst_pitch, unsigned long dst_p · 1 unmodified line | |||
| 88 | static int __drm_fb_xfrm_toio(void __iomem *dst, unsigned long dst_pitch, unsigned long dst_pixsize, | 178 | static int __drm_fb_xfrm_toio(void __iomem *dst, unsigned long dst_pitch, unsigned long dst_pixsize, |
| 89 | const void *vaddr, const struct drm_framebuffer *fb, | 179 | const void *vaddr, const struct drm_framebuffer *fb, |
| 90 | const struct drm_rect *clip, bool vaddr_cached_hint, | 180 | const struct drm_rect *clip, bool vaddr_cached_hint, |
| 181 | struct drm_format_conv_state *state, | ||
| 91 | void (*xfrm_line)(void *dbuf, const void *sbuf, unsigned int npixels)) | 182 | void (*xfrm_line)(void *dbuf, const void *sbuf, unsigned int npixels)) |
| 92 | { | 183 | { |
| 93 | unsigned long linepixels = drm_rect_width(clip); | 184 | unsigned long linepixels = drm_rect_width(clip); |
| @@ -101,9 +192,9 @@ static int __drm_fb_xfrm_toio(void __iomem *dst, unsigned long dst_pitch, unsign · 7 unmodified lines | |||
| 101 | void *dbuf; | 192 | void *dbuf; |
| 102 | 193 | ||
| 103 | if (vaddr_cached_hint) { | 194 | if (vaddr_cached_hint) { |
| 104 | dbuf = kmalloc(dbuf_len, GFP_KERNEL); | 195 | dbuf = drm_format_conv_state_reserve(state, dbuf_len, GFP_KERNEL); |
| 105 | } else { | 196 | } else { |
| 106 | dbuf = kmalloc(stmp_off + sbuf_len, GFP_KERNEL); | 197 | dbuf = drm_format_conv_state_reserve(state, stmp_off + sbuf_len, GFP_KERNEL); |
| 107 | stmp = dbuf + stmp_off; | 198 | stmp = dbuf + stmp_off; |
| 108 | } | 199 | } |
| 109 | if (!dbuf) | 200 | if (!dbuf) |
| @@ -124,8 +215,6 @@ static int __drm_fb_xfrm_toio(void __iomem *dst, unsigned long dst_pitch, unsign · 14 unmodified lines | |||
| 124 | dst += dst_pitch; | 215 | dst += dst_pitch; |
| 125 | } | 216 | } |
| 126 | 217 | ||
| 127 | kfree(dbuf); | ||
| 128 | |||
| 129 | return 0; | 218 | return 0; |
| 130 | } | 219 | } |
| 131 | 220 | ||
| @@ -134,6 +223,7 @@ static int drm_fb_xfrm(struct iosys_map *dst, · 2 unmodified lines | |||
| 134 | const unsigned int *dst_pitch, const u8 *dst_pixsize, | 223 | const unsigned int *dst_pitch, const u8 *dst_pixsize, |
| 135 | const struct iosys_map *src, const struct drm_framebuffer *fb, | 224 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 136 | const struct drm_rect *clip, bool vaddr_cached_hint, | 225 | const struct drm_rect *clip, bool vaddr_cached_hint, |
| 226 | struct drm_format_conv_state *state, | ||
| 137 | void (*xfrm_line)(void *dbuf, const void *sbuf, unsigned int npixels)) | 227 | void (*xfrm_line)(void *dbuf, const void *sbuf, unsigned int npixels)) |
| 138 | { | 228 | { |
| 139 | static const unsigned int default_dst_pitch[DRM_FORMAT_MAX_PLANES] = { | 229 | static const unsigned int default_dst_pitch[DRM_FORMAT_MAX_PLANES] = { |
| @@ -146,10 +236,12 @@ static int drm_fb_xfrm(struct iosys_map *dst, · 6 unmodified lines | |||
| 146 | /* TODO: handle src in I/O memory here */ | 236 | /* TODO: handle src in I/O memory here */ |
| 147 | if (dst[0].is_iomem) | 237 | if (dst[0].is_iomem) |
| 148 | return __drm_fb_xfrm_toio(dst[0].vaddr_iomem, dst_pitch[0], dst_pixsize[0], | 238 | return __drm_fb_xfrm_toio(dst[0].vaddr_iomem, dst_pitch[0], dst_pixsize[0], |
| 149 | src[0].vaddr, fb, clip, vaddr_cached_hint, xfrm_line); | 239 | src[0].vaddr, fb, clip, vaddr_cached_hint, state, |
| 240 | xfrm_line); | ||
| 150 | else | 241 | else |
| 151 | return __drm_fb_xfrm(dst[0].vaddr, dst_pitch[0], dst_pixsize[0], | 242 | return __drm_fb_xfrm(dst[0].vaddr, dst_pitch[0], dst_pixsize[0], |
| 152 | src[0].vaddr, fb, clip, vaddr_cached_hint, xfrm_line); | 243 | src[0].vaddr, fb, clip, vaddr_cached_hint, state, |
| 244 | xfrm_line); | ||
| 153 | } | 245 | } |
| 154 | 246 | ||
| 155 | /** | 247 | /** |
| @@ -235,6 +327,7 @@ static void drm_fb_swab32_line(void *dbuf, const void *sbuf, unsigned int pixels · 79 unmodified lines | |||
| 235 | * @fb: DRM framebuffer | 327 | * @fb: DRM framebuffer |
| 236 | * @clip: Clip rectangle area to copy | 328 | * @clip: Clip rectangle area to copy |
| 237 | * @cached: Source buffer is mapped cached (eg. not write-combined) | 329 | * @cached: Source buffer is mapped cached (eg. not write-combined) |
| 330 | * @state: Transform and conversion state | ||
| 238 | * | 331 | * |
| 239 | * This function copies parts of a framebuffer to display memory and swaps per-pixel | 332 | * This function copies parts of a framebuffer to display memory and swaps per-pixel |
| 240 | * bytes during the process. Destination and framebuffer formats must match. The | 333 | * bytes during the process. Destination and framebuffer formats must match. The |
| @@ -249,7 +342,8 @@ static void drm_fb_swab32_line(void *dbuf, const void *sbuf, unsigned int pixels · 8 unmodified lines | |||
| 249 | */ | 342 | */ |
| 250 | void drm_fb_swab(struct iosys_map *dst, const unsigned int *dst_pitch, | 343 | void drm_fb_swab(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 251 | const struct iosys_map *src, const struct drm_framebuffer *fb, | 344 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 252 | const struct drm_rect *clip, bool cached) | 345 | const struct drm_rect *clip, bool cached, |
| 346 | struct drm_format_conv_state *state) | ||
| 253 | { | 347 | { |
| 254 | const struct drm_format_info *format = fb->format; | 348 | const struct drm_format_info *format = fb->format; |
| 255 | u8 cpp = DIV_ROUND_UP(drm_format_info_bpp(format, 0), 8); | 349 | u8 cpp = DIV_ROUND_UP(drm_format_info_bpp(format, 0), 8); |
| @@ -268,7 +362,7 @@ void drm_fb_swab(struct iosys_map *dst, const unsigned int *dst_pitch, · 12 unmodified lines | |||
| 268 | return; | 362 | return; |
| 269 | } | 363 | } |
| 270 | 364 | ||
| 271 | drm_fb_xfrm(dst, dst_pitch, &cpp, src, fb, clip, cached, swab_line); | 365 | drm_fb_xfrm(dst, dst_pitch, &cpp, src, fb, clip, cached, state, swab_line); |
| 272 | } | 366 | } |
| 273 | EXPORT_SYMBOL(drm_fb_swab); | 367 | EXPORT_SYMBOL(drm_fb_swab); |
| 274 | 368 | ||
| @@ -295,6 +389,7 @@ static void drm_fb_xrgb8888_to_rgb332_line(void *dbuf, const void *sbuf, unsigne · 20 unmodified lines | |||
| 295 | * @src: Array of XRGB8888 source buffers | 389 | * @src: Array of XRGB8888 source buffers |
| 296 | * @fb: DRM framebuffer | 390 | * @fb: DRM framebuffer |
| 297 | * @clip: Clip rectangle area to copy | 391 | * @clip: Clip rectangle area to copy |
| 392 | * @state: Transform and conversion state | ||
| 298 | * | 393 | * |
| 299 | * This function copies parts of a framebuffer to display memory and converts the | 394 | * This function copies parts of a framebuffer to display memory and converts the |
| 300 | * color format during the process. Destination and framebuffer formats must match. The | 395 | * color format during the process. Destination and framebuffer formats must match. The |
| @@ -309,13 +404,13 @@ static void drm_fb_xrgb8888_to_rgb332_line(void *dbuf, const void *sbuf, unsigne · 8 unmodified lines | |||
| 309 | */ | 404 | */ |
| 310 | void drm_fb_xrgb8888_to_rgb332(struct iosys_map *dst, const unsigned int *dst_pitch, | 405 | void drm_fb_xrgb8888_to_rgb332(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 311 | const struct iosys_map *src, const struct drm_framebuffer *fb, | 406 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 312 | const struct drm_rect *clip) | 407 | const struct drm_rect *clip, struct drm_format_conv_state *state) |
| 313 | { | 408 | { |
| 314 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { | 409 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 315 | 1, | 410 | 1, |
| 316 | }; | 411 | }; |
| 317 | 412 | ||
| 318 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | 413 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, |
| 319 | drm_fb_xrgb8888_to_rgb332_line); | 414 | drm_fb_xrgb8888_to_rgb332_line); |
| 320 | } | 415 | } |
| 321 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb332); | 416 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb332); |
| @@ -364,6 +459,7 @@ static void drm_fb_xrgb8888_to_rgb565_swab_line(void *dbuf, const void *sbuf, · 42 unmodified lines | |||
| 364 | * @src: Array of XRGB8888 source buffer | 459 | * @src: Array of XRGB8888 source buffer |
| 365 | * @fb: DRM framebuffer | 460 | * @fb: DRM framebuffer |
| 366 | * @clip: Clip rectangle area to copy | 461 | * @clip: Clip rectangle area to copy |
| 462 | * @state: Transform and conversion state | ||
| 367 | * @swab: Swap bytes | 463 | * @swab: Swap bytes |
| 368 | * | 464 | * |
| 369 | * This function copies parts of a framebuffer to display memory and converts the | 465 | * This function copies parts of a framebuffer to display memory and converts the |
| @@ -379,7 +475,8 @@ static void drm_fb_xrgb8888_to_rgb565_swab_line(void *dbuf, const void *sbuf, · 9 unmodified lines | |||
| 379 | */ | 475 | */ |
| 380 | void drm_fb_xrgb8888_to_rgb565(struct iosys_map *dst, const unsigned int *dst_pitch, | 476 | void drm_fb_xrgb8888_to_rgb565(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 381 | const struct iosys_map *src, const struct drm_framebuffer *fb, | 477 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 382 | const struct drm_rect *clip, bool swab) | 478 | const struct drm_rect *clip, struct drm_format_conv_state *state, |
| 479 | bool swab) | ||
| 383 | { | 480 | { |
| 384 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { | 481 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 385 | 2, | 482 | 2, |
| @@ -392,7 +489,7 @@ void drm_fb_xrgb8888_to_rgb565(struct iosys_map *dst, const unsigned int *dst_pi · 6 unmodified lines | |||
| 392 | else | 489 | else |
| 393 | xfrm_line = drm_fb_xrgb8888_to_rgb565_line; | 490 | xfrm_line = drm_fb_xrgb8888_to_rgb565_line; |
| 394 | 491 | ||
| 395 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, xfrm_line); | 492 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, xfrm_line); |
| 396 | } | 493 | } |
| 397 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb565); | 494 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb565); |
| 398 | 495 | ||
| @@ -421,6 +518,7 @@ static void drm_fb_xrgb8888_to_xrgb1555_line(void *dbuf, const void *sbuf, unsig · 22 unmodified lines | |||
| 421 | * @src: Array of XRGB8888 source buffer | 518 | * @src: Array of XRGB8888 source buffer |
| 422 | * @fb: DRM framebuffer | 519 | * @fb: DRM framebuffer |
| 423 | * @clip: Clip rectangle area to copy | 520 | * @clip: Clip rectangle area to copy |
| 521 | * @state: Transform and conversion state | ||
| 424 | * | 522 | * |
| 425 | * This function copies parts of a framebuffer to display memory and converts | 523 | * This function copies parts of a framebuffer to display memory and converts |
| 426 | * the color format during the process. The parameters @dst, @dst_pitch and | 524 | * the color format during the process. The parameters @dst, @dst_pitch and |
| @@ -436,13 +534,13 @@ static void drm_fb_xrgb8888_to_xrgb1555_line(void *dbuf, const void *sbuf, unsig · 9 unmodified lines | |||
| 436 | */ | 534 | */ |
| 437 | void drm_fb_xrgb8888_to_xrgb1555(struct iosys_map *dst, const unsigned int *dst_pitch, | 535 | void drm_fb_xrgb8888_to_xrgb1555(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 438 | const struct iosys_map *src, const struct drm_framebuffer *fb, | 536 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 439 | const struct drm_rect *clip) | 537 | const struct drm_rect *clip, struct drm_format_conv_state *state) |
| 440 | { | 538 | { |
| 441 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { | 539 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 442 | 2, | 540 | 2, |
| 443 | }; | 541 | }; |
| 444 | 542 | ||
| 445 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | 543 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, |
| 446 | drm_fb_xrgb8888_to_xrgb1555_line); | 544 | drm_fb_xrgb8888_to_xrgb1555_line); |
| 447 | } | 545 | } |
| 448 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_xrgb1555); | 546 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_xrgb1555); |
| @@ -473,6 +571,7 @@ static void drm_fb_xrgb8888_to_argb1555_line(void *dbuf, const void *sbuf, unsig · 24 unmodified lines | |||
| 473 | * @src: Array of XRGB8888 source buffer | 571 | * @src: Array of XRGB8888 source buffer |
| 474 | * @fb: DRM framebuffer | 572 | * @fb: DRM framebuffer |
| 475 | * @clip: Clip rectangle area to copy | 573 | * @clip: Clip rectangle area to copy |
| 574 | * @state: Transform and conversion state | ||
| 476 | * | 575 | * |
| 477 | * This function copies parts of a framebuffer to display memory and converts | 576 | * This function copies parts of a framebuffer to display memory and converts |
| 478 | * the color format during the process. The parameters @dst, @dst_pitch and | 577 | * the color format during the process. The parameters @dst, @dst_pitch and |
| @@ -488,13 +587,13 @@ static void drm_fb_xrgb8888_to_argb1555_line(void *dbuf, const void *sbuf, unsig · 9 unmodified lines | |||
| 488 | */ | 587 | */ |
| 489 | void drm_fb_xrgb8888_to_argb1555(struct iosys_map *dst, const unsigned int *dst_pitch, | 588 | void drm_fb_xrgb8888_to_argb1555(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 490 | const struct iosys_map *src, const struct drm_framebuffer *fb, | 589 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 491 | const struct drm_rect *clip) | 590 | const struct drm_rect *clip, struct drm_format_conv_state *state) |
| 492 | { | 591 | { |
| 493 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { | 592 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 494 | 2, | 593 | 2, |
| 495 | }; | 594 | }; |
| 496 | 595 | ||
| 497 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | 596 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, |
| 498 | drm_fb_xrgb8888_to_argb1555_line); | 597 | drm_fb_xrgb8888_to_argb1555_line); |
| 499 | } | 598 | } |
| 500 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_argb1555); | 599 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_argb1555); |
| @@ -525,6 +624,7 @@ static void drm_fb_xrgb8888_to_rgba5551_line(void *dbuf, const void *sbuf, unsig · 24 unmodified lines | |||
| 525 | * @src: Array of XRGB8888 source buffer | 624 | * @src: Array of XRGB8888 source buffer |
| 526 | * @fb: DRM framebuffer | 625 | * @fb: DRM framebuffer |
| 527 | * @clip: Clip rectangle area to copy | 626 | * @clip: Clip rectangle area to copy |
| 627 | * @state: Transform and conversion state | ||
| 528 | * | 628 | * |
| 529 | * This function copies parts of a framebuffer to display memory and converts | 629 | * This function copies parts of a framebuffer to display memory and converts |
| 530 | * the color format during the process. The parameters @dst, @dst_pitch and | 630 | * the color format during the process. The parameters @dst, @dst_pitch and |
| @@ -540,13 +640,13 @@ static void drm_fb_xrgb8888_to_rgba5551_line(void *dbuf, const void *sbuf, unsig · 9 unmodified lines | |||
| 540 | */ | 640 | */ |
| 541 | void drm_fb_xrgb8888_to_rgba5551(struct iosys_map *dst, const unsigned int *dst_pitch, | 641 | void drm_fb_xrgb8888_to_rgba5551(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 542 | const struct iosys_map *src, const struct drm_framebuffer *fb, | 642 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 543 | const struct drm_rect *clip) | 643 | const struct drm_rect *clip, struct drm_format_conv_state *state) |
| 544 | { | 644 | { |
| 545 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { | 645 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 546 | 2, | 646 | 2, |
| 547 | }; | 647 | }; |
| 548 | 648 | ||
| 549 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | 649 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, |
| 550 | drm_fb_xrgb8888_to_rgba5551_line); | 650 | drm_fb_xrgb8888_to_rgba5551_line); |
| 551 | } | 651 | } |
| 552 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgba5551); | 652 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgba5551); |
| @@ -575,6 +675,7 @@ static void drm_fb_xrgb8888_to_rgb888_line(void *dbuf, const void *sbuf, unsigne · 22 unmodified lines | |||
| 575 | * @src: Array of XRGB8888 source buffers | 675 | * @src: Array of XRGB8888 source buffers |
| 576 | * @fb: DRM framebuffer | 676 | * @fb: DRM framebuffer |
| 577 | * @clip: Clip rectangle area to copy | 677 | * @clip: Clip rectangle area to copy |
| 678 | * @state: Transform and conversion state | ||
| 578 | * | 679 | * |
| 579 | * This function copies parts of a framebuffer to display memory and converts the | 680 | * This function copies parts of a framebuffer to display memory and converts the |
| 580 | * color format during the process. Destination and framebuffer formats must match. The | 681 | * color format during the process. Destination and framebuffer formats must match. The |
| @@ -590,13 +691,13 @@ static void drm_fb_xrgb8888_to_rgb888_line(void *dbuf, const void *sbuf, unsigne · 9 unmodified lines | |||
| 590 | */ | 691 | */ |
| 591 | void drm_fb_xrgb8888_to_rgb888(struct iosys_map *dst, const unsigned int *dst_pitch, | 692 | void drm_fb_xrgb8888_to_rgb888(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 592 | const struct iosys_map *src, const struct drm_framebuffer *fb, | 693 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 593 | const struct drm_rect *clip) | 694 | const struct drm_rect *clip, struct drm_format_conv_state *state) |
| 594 | { | 695 | { |
| 595 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { | 696 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 596 | 3, | 697 | 3, |
| 597 | }; | 698 | }; |
| 598 | 699 | ||
| 599 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | 700 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, |
| 600 | drm_fb_xrgb8888_to_rgb888_line); | 701 | drm_fb_xrgb8888_to_rgb888_line); |
| 601 | } | 702 | } |
| 602 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb888); | 703 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb888); |
| @@ -623,6 +724,7 @@ static void drm_fb_xrgb8888_to_argb8888_line(void *dbuf, const void *sbuf, unsig · 20 unmodified lines | |||
| 623 | * @src: Array of XRGB8888 source buffer | 724 | * @src: Array of XRGB8888 source buffer |
| 624 | * @fb: DRM framebuffer | 725 | * @fb: DRM framebuffer |
| 625 | * @clip: Clip rectangle area to copy | 726 | * @clip: Clip rectangle area to copy |
| 727 | * @state: Transform and conversion state | ||
| 626 | * | 728 | * |
| 627 | * This function copies parts of a framebuffer to display memory and converts the | 729 | * This function copies parts of a framebuffer to display memory and converts the |
| 628 | * color format during the process. The parameters @dst, @dst_pitch and @src refer | 730 | * color format during the process. The parameters @dst, @dst_pitch and @src refer |
| @@ -638,13 +740,13 @@ static void drm_fb_xrgb8888_to_argb8888_line(void *dbuf, const void *sbuf, unsig · 9 unmodified lines | |||
| 638 | */ | 740 | */ |
| 639 | void drm_fb_xrgb8888_to_argb8888(struct iosys_map *dst, const unsigned int *dst_pitch, | 741 | void drm_fb_xrgb8888_to_argb8888(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 640 | const struct iosys_map *src, const struct drm_framebuffer *fb, | 742 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 641 | const struct drm_rect *clip) | 743 | const struct drm_rect *clip, struct drm_format_conv_state *state) |
| 642 | { | 744 | { |
| 643 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { | 745 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 644 | 4, | 746 | 4, |
| 645 | }; | 747 | }; |
| 646 | 748 | ||
| 647 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | 749 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, |
| 648 | drm_fb_xrgb8888_to_argb8888_line); | 750 | drm_fb_xrgb8888_to_argb8888_line); |
| 649 | } | 751 | } |
| 650 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_argb8888); | 752 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_argb8888); |
| @@ -669,13 +771,14 @@ static void drm_fb_xrgb8888_to_abgr8888_line(void *dbuf, const void *sbuf, unsig · 18 unmodified lines | |||
| 669 | static void drm_fb_xrgb8888_to_abgr8888(struct iosys_map *dst, const unsigned int *dst_pitch, | 771 | static void drm_fb_xrgb8888_to_abgr8888(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 670 | const struct iosys_map *src, | 772 | const struct iosys_map *src, |
| 671 | const struct drm_framebuffer *fb, | 773 | const struct drm_framebuffer *fb, |
| 672 | const struct drm_rect *clip) | 774 | const struct drm_rect *clip, |
| 775 | struct drm_format_conv_state *state) | ||
| 673 | { | 776 | { |
| 674 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { | 777 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 675 | 4, | 778 | 4, |
| 676 | }; | 779 | }; |
| 677 | 780 | ||
| 678 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | 781 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, |
| 679 | drm_fb_xrgb8888_to_abgr8888_line); | 782 | drm_fb_xrgb8888_to_abgr8888_line); |
| 680 | } | 783 | } |
| 681 | 784 | ||
| @@ -699,13 +802,14 @@ static void drm_fb_xrgb8888_to_xbgr8888_line(void *dbuf, const void *sbuf, unsig · 17 unmodified lines | |||
| 699 | static void drm_fb_xrgb8888_to_xbgr8888(struct iosys_map *dst, const unsigned int *dst_pitch, | 802 | static void drm_fb_xrgb8888_to_xbgr8888(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 700 | const struct iosys_map *src, | 803 | const struct iosys_map *src, |
| 701 | const struct drm_framebuffer *fb, | 804 | const struct drm_framebuffer *fb, |
| 702 | const struct drm_rect *clip) | 805 | const struct drm_rect *clip, |
| 806 | struct drm_format_conv_state *state) | ||
| 703 | { | 807 | { |
| 704 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { | 808 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 705 | 4, | 809 | 4, |
| 706 | }; | 810 | }; |
| 707 | 811 | ||
| 708 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | 812 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, |
| 709 | drm_fb_xrgb8888_to_xbgr8888_line); | 813 | drm_fb_xrgb8888_to_xbgr8888_line); |
| 710 | } | 814 | } |
| 711 | 815 | ||
| @@ -735,6 +839,7 @@ static void drm_fb_xrgb8888_to_xrgb2101010_line(void *dbuf, const void *sbuf, un · 23 unmodified lines | |||
| 735 | * @src: Array of XRGB8888 source buffers | 839 | * @src: Array of XRGB8888 source buffers |
| 736 | * @fb: DRM framebuffer | 840 | * @fb: DRM framebuffer |
| 737 | * @clip: Clip rectangle area to copy | 841 | * @clip: Clip rectangle area to copy |
| 842 | * @state: Transform and conversion state | ||
| 738 | * | 843 | * |
| 739 | * This function copies parts of a framebuffer to display memory and converts the | 844 | * This function copies parts of a framebuffer to display memory and converts the |
| 740 | * color format during the process. Destination and framebuffer formats must match. The | 845 | * color format during the process. Destination and framebuffer formats must match. The |
| @@ -750,13 +855,14 @@ static void drm_fb_xrgb8888_to_xrgb2101010_line(void *dbuf, const void *sbuf, un · 9 unmodified lines | |||
| 750 | */ | 855 | */ |
| 751 | void drm_fb_xrgb8888_to_xrgb2101010(struct iosys_map *dst, const unsigned int *dst_pitch, | 856 | void drm_fb_xrgb8888_to_xrgb2101010(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 752 | const struct iosys_map *src, const struct drm_framebuffer *fb, | 857 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 753 | const struct drm_rect *clip) | 858 | const struct drm_rect *clip, |
| 859 | struct drm_format_conv_state *state) | ||
| 754 | { | 860 | { |
| 755 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { | 861 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 756 | 4, | 862 | 4, |
| 757 | }; | 863 | }; |
| 758 | 864 | ||
| 759 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | 865 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, |
| 760 | drm_fb_xrgb8888_to_xrgb2101010_line); | 866 | drm_fb_xrgb8888_to_xrgb2101010_line); |
| 761 | } | 867 | } |
| 762 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_xrgb2101010); | 868 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_xrgb2101010); |
| @@ -788,6 +894,7 @@ static void drm_fb_xrgb8888_to_argb2101010_line(void *dbuf, const void *sbuf, un · 25 unmodified lines | |||
| 788 | * @src: Array of XRGB8888 source buffers | 894 | * @src: Array of XRGB8888 source buffers |
| 789 | * @fb: DRM framebuffer | 895 | * @fb: DRM framebuffer |
| 790 | * @clip: Clip rectangle area to copy | 896 | * @clip: Clip rectangle area to copy |
| 897 | * @state: Transform and conversion state | ||
| 791 | * | 898 | * |
| 792 | * This function copies parts of a framebuffer to display memory and converts | 899 | * This function copies parts of a framebuffer to display memory and converts |
| 793 | * the color format during the process. The parameters @dst, @dst_pitch and | 900 | * the color format during the process. The parameters @dst, @dst_pitch and |
| @@ -803,13 +910,14 @@ static void drm_fb_xrgb8888_to_argb2101010_line(void *dbuf, const void *sbuf, un · 9 unmodified lines | |||
| 803 | */ | 910 | */ |
| 804 | void drm_fb_xrgb8888_to_argb2101010(struct iosys_map *dst, const unsigned int *dst_pitch, | 911 | void drm_fb_xrgb8888_to_argb2101010(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 805 | const struct iosys_map *src, const struct drm_framebuffer *fb, | 912 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 806 | const struct drm_rect *clip) | 913 | const struct drm_rect *clip, |
| 914 | struct drm_format_conv_state *state) | ||
| 807 | { | 915 | { |
| 808 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { | 916 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 809 | 4, | 917 | 4, |
| 810 | }; | 918 | }; |
| 811 | 919 | ||
| 812 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | 920 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, |
| 813 | drm_fb_xrgb8888_to_argb2101010_line); | 921 | drm_fb_xrgb8888_to_argb2101010_line); |
| 814 | } | 922 | } |
| 815 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_argb2101010); | 923 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_argb2101010); |
| @@ -839,6 +947,7 @@ static void drm_fb_xrgb8888_to_gray8_line(void *dbuf, const void *sbuf, unsigned · 23 unmodified lines | |||
| 839 | * @src: Array of XRGB8888 source buffers | 947 | * @src: Array of XRGB8888 source buffers |
| 840 | * @fb: DRM framebuffer | 948 | * @fb: DRM framebuffer |
| 841 | * @clip: Clip rectangle area to copy | 949 | * @clip: Clip rectangle area to copy |
| 950 | * @state: Transform and conversion state | ||
| 842 | * | 951 | * |
| 843 | * This function copies parts of a framebuffer to display memory and converts the | 952 | * This function copies parts of a framebuffer to display memory and converts the |
| 844 | * color format during the process. Destination and framebuffer formats must match. The | 953 | * color format during the process. Destination and framebuffer formats must match. The |
| @@ -858,13 +967,13 @@ static void drm_fb_xrgb8888_to_gray8_line(void *dbuf, const void *sbuf, unsigned · 13 unmodified lines | |||
| 858 | */ | 967 | */ |
| 859 | void drm_fb_xrgb8888_to_gray8(struct iosys_map *dst, const unsigned int *dst_pitch, | 968 | void drm_fb_xrgb8888_to_gray8(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 860 | const struct iosys_map *src, const struct drm_framebuffer *fb, | 969 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 861 | const struct drm_rect *clip) | 970 | const struct drm_rect *clip, struct drm_format_conv_state *state) |
| 862 | { | 971 | { |
| 863 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { | 972 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 864 | 1, | 973 | 1, |
| 865 | }; | 974 | }; |
| 866 | 975 | ||
| 867 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | 976 | drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, |
| 868 | drm_fb_xrgb8888_to_gray8_line); | 977 | drm_fb_xrgb8888_to_gray8_line); |
| 869 | } | 978 | } |
| 870 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_gray8); | 979 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_gray8); |
| @@ -878,6 +987,7 @@ EXPORT_SYMBOL(drm_fb_xrgb8888_to_gray8); · 7 unmodified lines | |||
| 878 | * @src: The framebuffer memory to copy from | 987 | * @src: The framebuffer memory to copy from |
| 879 | * @fb: The framebuffer to copy from | 988 | * @fb: The framebuffer to copy from |
| 880 | * @clip: Clip rectangle area to copy | 989 | * @clip: Clip rectangle area to copy |
| 990 | * @state: Transform and conversion state | ||
| 881 | * | 991 | * |
| 882 | * This function copies parts of a framebuffer to display memory. If the | 992 | * This function copies parts of a framebuffer to display memory. If the |
| 883 | * formats of the display and the framebuffer mismatch, the blit function | 993 | * formats of the display and the framebuffer mismatch, the blit function |
| @@ -896,7 +1006,7 @@ EXPORT_SYMBOL(drm_fb_xrgb8888_to_gray8); · 12 unmodified lines | |||
| 896 | */ | 1006 | */ |
| 897 | int drm_fb_blit(struct iosys_map *dst, const unsigned int *dst_pitch, uint32_t dst_format, | 1007 | int drm_fb_blit(struct iosys_map *dst, const unsigned int *dst_pitch, uint32_t dst_format, |
| 898 | const struct iosys_map *src, const struct drm_framebuffer *fb, | 1008 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 899 | const struct drm_rect *clip) | 1009 | const struct drm_rect *clip, struct drm_format_conv_state *state) |
| 900 | { | 1010 | { |
| 901 | uint32_t fb_format = fb->format->format; | 1011 | uint32_t fb_format = fb->format->format; |
| 902 | 1012 | ||
| @@ -904,44 +1014,44 @@ int drm_fb_blit(struct iosys_map *dst, const unsigned int *dst_pitch, uint32_t d · 1 unmodified line | |||
| 904 | drm_fb_memcpy(dst, dst_pitch, src, fb, clip); | 1014 | drm_fb_memcpy(dst, dst_pitch, src, fb, clip); |
| 905 | return 0; | 1015 | return 0; |
| 906 | } else if (fb_format == (dst_format | DRM_FORMAT_BIG_ENDIAN)) { | 1016 | } else if (fb_format == (dst_format | DRM_FORMAT_BIG_ENDIAN)) { |
| 907 | drm_fb_swab(dst, dst_pitch, src, fb, clip, false); | 1017 | drm_fb_swab(dst, dst_pitch, src, fb, clip, false, state); |
| 908 | return 0; | 1018 | return 0; |
| 909 | } else if (fb_format == (dst_format & ~DRM_FORMAT_BIG_ENDIAN)) { | 1019 | } else if (fb_format == (dst_format & ~DRM_FORMAT_BIG_ENDIAN)) { |
| 910 | drm_fb_swab(dst, dst_pitch, src, fb, clip, false); | 1020 | drm_fb_swab(dst, dst_pitch, src, fb, clip, false, state); |
| 911 | return 0; | 1021 | return 0; |
| 912 | } else if (fb_format == DRM_FORMAT_XRGB8888) { | 1022 | } else if (fb_format == DRM_FORMAT_XRGB8888) { |
| 913 | if (dst_format == DRM_FORMAT_RGB565) { | 1023 | if (dst_format == DRM_FORMAT_RGB565) { |
| 914 | drm_fb_xrgb8888_to_rgb565(dst, dst_pitch, src, fb, clip, false); | 1024 | drm_fb_xrgb8888_to_rgb565(dst, dst_pitch, src, fb, clip, state, false); |
| 915 | return 0; | 1025 | return 0; |
| 916 | } else if (dst_format == DRM_FORMAT_XRGB1555) { | 1026 | } else if (dst_format == DRM_FORMAT_XRGB1555) { |
| 917 | drm_fb_xrgb8888_to_xrgb1555(dst, dst_pitch, src, fb, clip); | 1027 | drm_fb_xrgb8888_to_xrgb1555(dst, dst_pitch, src, fb, clip, state); |
| 918 | return 0; | 1028 | return 0; |
| 919 | } else if (dst_format == DRM_FORMAT_ARGB1555) { | 1029 | } else if (dst_format == DRM_FORMAT_ARGB1555) { |
| 920 | drm_fb_xrgb8888_to_argb1555(dst, dst_pitch, src, fb, clip); | 1030 | drm_fb_xrgb8888_to_argb1555(dst, dst_pitch, src, fb, clip, state); |
| 921 | return 0; | 1031 | return 0; |
| 922 | } else if (dst_format == DRM_FORMAT_RGBA5551) { | 1032 | } else if (dst_format == DRM_FORMAT_RGBA5551) { |
| 923 | drm_fb_xrgb8888_to_rgba5551(dst, dst_pitch, src, fb, clip); | 1033 | drm_fb_xrgb8888_to_rgba5551(dst, dst_pitch, src, fb, clip, state); |
| 924 | return 0; | 1034 | return 0; |
| 925 | } else if (dst_format == DRM_FORMAT_RGB888) { | 1035 | } else if (dst_format == DRM_FORMAT_RGB888) { |
| 926 | drm_fb_xrgb8888_to_rgb888(dst, dst_pitch, src, fb, clip); | 1036 | drm_fb_xrgb8888_to_rgb888(dst, dst_pitch, src, fb, clip, state); |
| 927 | return 0; | 1037 | return 0; |
| 928 | } else if (dst_format == DRM_FORMAT_ARGB8888) { | 1038 | } else if (dst_format == DRM_FORMAT_ARGB8888) { |
| 929 | drm_fb_xrgb8888_to_argb8888(dst, dst_pitch, src, fb, clip); | 1039 | drm_fb_xrgb8888_to_argb8888(dst, dst_pitch, src, fb, clip, state); |
| 930 | return 0; | 1040 | return 0; |
| 931 | } else if (dst_format == DRM_FORMAT_XBGR8888) { | 1041 | } else if (dst_format == DRM_FORMAT_XBGR8888) { |
| 932 | drm_fb_xrgb8888_to_xbgr8888(dst, dst_pitch, src, fb, clip); | 1042 | drm_fb_xrgb8888_to_xbgr8888(dst, dst_pitch, src, fb, clip, state); |
| 933 | return 0; | 1043 | return 0; |
| 934 | } else if (dst_format == DRM_FORMAT_ABGR8888) { | 1044 | } else if (dst_format == DRM_FORMAT_ABGR8888) { |
| 935 | drm_fb_xrgb8888_to_abgr8888(dst, dst_pitch, src, fb, clip); | 1045 | drm_fb_xrgb8888_to_abgr8888(dst, dst_pitch, src, fb, clip, state); |
| 936 | return 0; | 1046 | return 0; |
| 937 | } else if (dst_format == DRM_FORMAT_XRGB2101010) { | 1047 | } else if (dst_format == DRM_FORMAT_XRGB2101010) { |
| 938 | drm_fb_xrgb8888_to_xrgb2101010(dst, dst_pitch, src, fb, clip); | 1048 | drm_fb_xrgb8888_to_xrgb2101010(dst, dst_pitch, src, fb, clip, state); |
| 939 | return 0; | 1049 | return 0; |
| 940 | } else if (dst_format == DRM_FORMAT_ARGB2101010) { | 1050 | } else if (dst_format == DRM_FORMAT_ARGB2101010) { |
| 941 | drm_fb_xrgb8888_to_argb2101010(dst, dst_pitch, src, fb, clip); | 1051 | drm_fb_xrgb8888_to_argb2101010(dst, dst_pitch, src, fb, clip, state); |
| 942 | return 0; | 1052 | return 0; |
| 943 | } else if (dst_format == DRM_FORMAT_BGRX8888) { | 1053 | } else if (dst_format == DRM_FORMAT_BGRX8888) { |
| 944 | drm_fb_swab(dst, dst_pitch, src, fb, clip, false); | 1054 | drm_fb_swab(dst, dst_pitch, src, fb, clip, false, state); |
| 945 | return 0; | 1055 | return 0; |
| 946 | } | 1056 | } |
| 947 | } | 1057 | } |
| @@ -978,6 +1088,7 @@ static void drm_fb_gray8_to_mono_line(void *dbuf, const void *sbuf, unsigned int · 30 unmodified lines | |||
| 978 | * @src: Array of XRGB8888 source buffers | 1088 | * @src: Array of XRGB8888 source buffers |
| 979 | * @fb: DRM framebuffer | 1089 | * @fb: DRM framebuffer |
| 980 | * @clip: Clip rectangle area to copy | 1090 | * @clip: Clip rectangle area to copy |
| 1091 | * @state: Transform and conversion state | ||
| 981 | * | 1092 | * |
| 982 | * This function copies parts of a framebuffer to display memory and converts the | 1093 | * This function copies parts of a framebuffer to display memory and converts the |
| 983 | * color format during the process. Destination and framebuffer formats must match. The | 1094 | * color format during the process. Destination and framebuffer formats must match. The |
| @@ -1002,7 +1113,7 @@ static void drm_fb_gray8_to_mono_line(void *dbuf, const void *sbuf, unsigned int · 18 unmodified lines | |||
| 1002 | */ | 1113 | */ |
| 1003 | void drm_fb_xrgb8888_to_mono(struct iosys_map *dst, const unsigned int *dst_pitch, | 1114 | void drm_fb_xrgb8888_to_mono(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 1004 | const struct iosys_map *src, const struct drm_framebuffer *fb, | 1115 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 1005 | const struct drm_rect *clip) | 1116 | const struct drm_rect *clip, struct drm_format_conv_state *state) |
| 1006 | { | 1117 | { |
| 1007 | static const unsigned int default_dst_pitch[DRM_FORMAT_MAX_PLANES] = { | 1118 | static const unsigned int default_dst_pitch[DRM_FORMAT_MAX_PLANES] = { |
| 1008 | 0, 0, 0, 0 | 1119 | 0, 0, 0, 0 |
| @@ -1042,7 +1153,7 @@ void drm_fb_xrgb8888_to_mono(struct iosys_map *dst, const unsigned int *dst_pitc · 33 unmodified lines | |||
| 1042 | * Allocate a buffer to be used for both copying from the cma | 1153 | * Allocate a buffer to be used for both copying from the cma |
| 1043 | * memory and to store the intermediate grayscale line pixels. | 1154 | * memory and to store the intermediate grayscale line pixels. |
| 1044 | */ | 1155 | */ |
| 1045 | src32 = kmalloc(len_src32 + linepixels, GFP_KERNEL); | 1156 | src32 = drm_format_conv_state_reserve(state, len_src32 + linepixels, GFP_KERNEL); |
| 1046 | if (!src32) | 1157 | if (!src32) |
| 1047 | return; | 1158 | return; |
| 1048 | 1159 | ||
| @@ -1056,8 +1167,6 @@ void drm_fb_xrgb8888_to_mono(struct iosys_map *dst, const unsigned int *dst_pitc · 7 unmodified lines | |||
| 1056 | vaddr += fb->pitches[0]; | 1167 | vaddr += fb->pitches[0]; |
| 1057 | mono += dst_pitch_0; | 1168 | mono += dst_pitch_0; |
| 1058 | } | 1169 | } |
| 1059 | |||
| 1060 | kfree(src32); | ||
| 1061 | } | 1170 | } |
| 1062 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_mono); | 1171 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_mono); |
| 1063 | 1172 | ||
| @@ -394,6 +394,31 @@ static void drm_mode_rmfb_work_fn(struct work_struct *w) · 393 unmodified lines | |||
| 394 | } | 394 | } |
| 395 | } | 395 | } |
| 396 | 396 | ||
| 397 | static int drm_mode_closefb(struct drm_framebuffer *fb, | ||
| 398 | struct drm_file *file_priv) | ||
| 399 | { | ||
| 400 | struct drm_framebuffer *fbl; | ||
| 401 | bool found = false; | ||
| 402 | |||
| 403 | mutex_lock(&file_priv->fbs_lock); | ||
| 404 | list_for_each_entry(fbl, &file_priv->fbs, filp_head) | ||
| 405 | if (fb == fbl) | ||
| 406 | found = true; | ||
| 407 | |||
| 408 | if (!found) { | ||
| 409 | mutex_unlock(&file_priv->fbs_lock); | ||
| 410 | return -ENOENT; | ||
| 411 | } | ||
| 412 | |||
| 413 | list_del_init(&fb->filp_head); | ||
| 414 | mutex_unlock(&file_priv->fbs_lock); | ||
| 415 | |||
| 416 | /* Drop the reference that was stored in the fbs list */ | ||
| 417 | drm_framebuffer_put(fb); | ||
| 418 | |||
| 419 | return 0; | ||
| 420 | } | ||
| 421 | |||
| 397 | /** | 422 | /** |
| 398 | * drm_mode_rmfb - remove an FB from the configuration | 423 | * drm_mode_rmfb - remove an FB from the configuration |
| 399 | * @dev: drm device | 424 | * @dev: drm device |
| @@ -410,9 +435,8 @@ static void drm_mode_rmfb_work_fn(struct work_struct *w) · 10 unmodified lines | |||
| 410 | int drm_mode_rmfb(struct drm_device *dev, u32 fb_id, | 435 | int drm_mode_rmfb(struct drm_device *dev, u32 fb_id, |
| 411 | struct drm_file *file_priv) | 436 | struct drm_file *file_priv) |
| 412 | { | 437 | { |
| 413 | struct drm_framebuffer *fb = NULL; | 438 | struct drm_framebuffer *fb; |
| 414 | struct drm_framebuffer *fbl = NULL; | 439 | int ret; |
| 415 | int found = 0; | ||
| 416 | 440 | ||
| 417 | if (!drm_core_check_feature(dev, DRIVER_MODESET)) | 441 | if (!drm_core_check_feature(dev, DRIVER_MODESET)) |
| 418 | return -EOPNOTSUPP; | 442 | return -EOPNOTSUPP; |
| @@ -421,24 +445,13 @@ int drm_mode_rmfb(struct drm_device *dev, u32 fb_id, · 2 unmodified lines | |||
| 421 | if (!fb) | 445 | if (!fb) |
| 422 | return -ENOENT; | 446 | return -ENOENT; |
| 423 | 447 | ||
| 424 | mutex_lock(&file_priv->fbs_lock); | 448 | ret = drm_mode_closefb(fb, file_priv); |
| 425 | list_for_each_entry(fbl, &file_priv->fbs, filp_head) | 449 | if (ret != 0) { |
| 426 | if (fb == fbl) | 450 | drm_framebuffer_put(fb); |
| 427 | found = 1; | 451 | return ret; |
| 428 | if (!found) { | ||
| 429 | mutex_unlock(&file_priv->fbs_lock); | ||
| 430 | goto fail_unref; | ||
| 431 | } | 452 | } |
| 432 | 453 | ||
| 433 | list_del_init(&fb->filp_head); | ||
| 434 | mutex_unlock(&file_priv->fbs_lock); | ||
| 435 | |||
| 436 | /* drop the reference we picked up in framebuffer lookup */ | ||
| 437 | drm_framebuffer_put(fb); | ||
| 438 | |||
| 439 | /* | 454 | /* |
| 440 | * we now own the reference that was stored in the fbs list | ||
| 441 | * | ||
| 442 | * drm_framebuffer_remove may fail with -EINTR on pending signals, | 455 | * drm_framebuffer_remove may fail with -EINTR on pending signals, |
| 443 | * so run this in a separate stack as there's no way to correctly | 456 | * so run this in a separate stack as there's no way to correctly |
| 444 | * handle this after the fb is already removed from the lookup table. | 457 | * handle this after the fb is already removed from the lookup table. |
| @@ -457,10 +470,6 @@ int drm_mode_rmfb(struct drm_device *dev, u32 fb_id, · 12 unmodified lines | |||
| 457 | drm_framebuffer_put(fb); | 470 | drm_framebuffer_put(fb); |
| 458 | 471 | ||
| 459 | return 0; | 472 | return 0; |
| 460 | |||
| 461 | fail_unref: | ||
| 462 | drm_framebuffer_put(fb); | ||
| 463 | return -ENOENT; | ||
| 464 | } | 473 | } |
| 465 | 474 | ||
| 466 | int drm_mode_rmfb_ioctl(struct drm_device *dev, | 475 | int drm_mode_rmfb_ioctl(struct drm_device *dev, |
| @@ -471,6 +480,28 @@ int drm_mode_rmfb_ioctl(struct drm_device *dev, · 4 unmodified lines | |||
| 471 | return drm_mode_rmfb(dev, *fb_id, file_priv); | 480 | return drm_mode_rmfb(dev, *fb_id, file_priv); |
| 472 | } | 481 | } |
| 473 | 482 | ||
| 483 | int drm_mode_closefb_ioctl(struct drm_device *dev, | ||
| 484 | void *data, struct drm_file *file_priv) | ||
| 485 | { | ||
| 486 | struct drm_mode_closefb *r = data; | ||
| 487 | struct drm_framebuffer *fb; | ||
| 488 | int ret; | ||
| 489 | |||
| 490 | if (!drm_core_check_feature(dev, DRIVER_MODESET)) | ||
| 491 | return -EOPNOTSUPP; | ||
| 492 | |||
| 493 | if (r->pad) | ||
| 494 | return -EINVAL; | ||
| 495 | |||
| 496 | fb = drm_framebuffer_lookup(dev, file_priv, r->fb_id); | ||
| 497 | if (!fb) | ||
| 498 | return -ENOENT; | ||
| 499 | |||
| 500 | ret = drm_mode_closefb(fb, file_priv); | ||
| 501 | drm_framebuffer_put(fb); | ||
| 502 | return ret; | ||
| 503 | } | ||
| 504 | |||
| 474 | /** | 505 | /** |
| 475 | * drm_mode_getfb - get FB info | 506 | * drm_mode_getfb - get FB info |
| 476 | * @dev: drm device for the ioctl | 507 | * @dev: drm device for the ioctl |
| @@ -552,7 +583,7 @@ int drm_mode_getfb2_ioctl(struct drm_device *dev, · 75 unmodified lines | |||
| 552 | struct drm_mode_fb_cmd2 *r = data; | 583 | struct drm_mode_fb_cmd2 *r = data; |
| 553 | struct drm_framebuffer *fb; | 584 | struct drm_framebuffer *fb; |
| 554 | unsigned int i; | 585 | unsigned int i; |
| 555 | int ret; | 586 | int ret = 0; |
| 556 | 587 | ||
| 557 | if (!drm_core_check_feature(dev, DRIVER_MODESET)) | 588 | if (!drm_core_check_feature(dev, DRIVER_MODESET)) |
| 558 | return -EINVAL; | 589 | return -EINVAL; |
| @@ -218,7 +218,14 @@ void · 217 unmodified lines | |||
| 218 | __drm_gem_duplicate_shadow_plane_state(struct drm_plane *plane, | 218 | __drm_gem_duplicate_shadow_plane_state(struct drm_plane *plane, |
| 219 | struct drm_shadow_plane_state *new_shadow_plane_state) | 219 | struct drm_shadow_plane_state *new_shadow_plane_state) |
| 220 | { | 220 | { |
| 221 | struct drm_plane_state *plane_state = plane->state; | ||
| 222 | struct drm_shadow_plane_state *shadow_plane_state = | ||
| 223 | to_drm_shadow_plane_state(plane_state); | ||
| 224 | |||
| 221 | __drm_atomic_helper_plane_duplicate_state(plane, &new_shadow_plane_state->base); | 225 | __drm_atomic_helper_plane_duplicate_state(plane, &new_shadow_plane_state->base); |
| 226 | |||
| 227 | drm_format_conv_state_copy(&shadow_plane_state->fmtcnv_state, | ||
| 228 | &new_shadow_plane_state->fmtcnv_state); | ||
| 222 | } | 229 | } |
| 223 | EXPORT_SYMBOL(__drm_gem_duplicate_shadow_plane_state); | 230 | EXPORT_SYMBOL(__drm_gem_duplicate_shadow_plane_state); |
| 224 | 231 | ||
| @@ -266,6 +273,7 @@ EXPORT_SYMBOL(drm_gem_duplicate_shadow_plane_state); · 41 unmodified lines | |||
| 266 | */ | 273 | */ |
| 267 | void __drm_gem_destroy_shadow_plane_state(struct drm_shadow_plane_state *shadow_plane_state) | 274 | void __drm_gem_destroy_shadow_plane_state(struct drm_shadow_plane_state *shadow_plane_state) |
| 268 | { | 275 | { |
| 276 | drm_format_conv_state_release(&shadow_plane_state->fmtcnv_state); | ||
| 269 | __drm_atomic_helper_plane_destroy_state(&shadow_plane_state->base); | 277 | __drm_atomic_helper_plane_destroy_state(&shadow_plane_state->base); |
| 270 | } | 278 | } |
| 271 | EXPORT_SYMBOL(__drm_gem_destroy_shadow_plane_state); | 279 | EXPORT_SYMBOL(__drm_gem_destroy_shadow_plane_state); |
| @@ -302,6 +310,7 @@ void __drm_gem_reset_shadow_plane(struct drm_plane *plane, · 30 unmodified lines | |||
| 302 | struct drm_shadow_plane_state *shadow_plane_state) | 310 | struct drm_shadow_plane_state *shadow_plane_state) |
| 303 | { | 311 | { |
| 304 | __drm_atomic_helper_plane_reset(plane, &shadow_plane_state->base); | 312 | __drm_atomic_helper_plane_reset(plane, &shadow_plane_state->base); |
| 313 | drm_format_conv_state_init(&shadow_plane_state->fmtcnv_state); | ||
| 305 | } | 314 | } |
| 306 | EXPORT_SYMBOL(__drm_gem_reset_shadow_plane); | 315 | EXPORT_SYMBOL(__drm_gem_reset_shadow_plane); |
| 307 | 316 | ||
| @@ -61,6 +61,42 @@ · 60 unmodified lines | |||
| 61 | * contained within struct drm_gpuva already. Hence, for inserting &drm_gpuva | 61 | * contained within struct drm_gpuva already. Hence, for inserting &drm_gpuva |
| 62 | * entries from within dma-fence signalling critical sections it is enough to | 62 | * entries from within dma-fence signalling critical sections it is enough to |
| 63 | * pre-allocate the &drm_gpuva structures. | 63 | * pre-allocate the &drm_gpuva structures. |
| 64 | * | ||
| 65 | * &drm_gem_objects which are private to a single VM can share a common | ||
| 66 | * &dma_resv in order to improve locking efficiency (e.g. with &drm_exec). | ||
| 67 | * For this purpose drivers must pass a &drm_gem_object to drm_gpuvm_init(), in | ||
| 68 | * the following called 'resv object', which serves as the container of the | ||
| 69 | * GPUVM's shared &dma_resv. This resv object can be a driver specific | ||
| 70 | * &drm_gem_object, such as the &drm_gem_object containing the root page table, | ||
| 71 | * but it can also be a 'dummy' object, which can be allocated with | ||
| 72 | * drm_gpuvm_resv_object_alloc(). | ||
| 73 | * | ||
| 74 | * In order to connect a struct drm_gpuva its backing &drm_gem_object each | ||
| 75 | * &drm_gem_object maintains a list of &drm_gpuvm_bo structures, and each | ||
| 76 | * &drm_gpuvm_bo contains a list of &drm_gpuva structures. | ||
| 77 | * | ||
| 78 | * A &drm_gpuvm_bo is an abstraction that represents a combination of a | ||
| 79 | * &drm_gpuvm and a &drm_gem_object. Every such combination should be unique. | ||
| 80 | * This is ensured by the API through drm_gpuvm_bo_obtain() and | ||
| 81 | * drm_gpuvm_bo_obtain_prealloc() which first look into the corresponding | ||
| 82 | * &drm_gem_object list of &drm_gpuvm_bos for an existing instance of this | ||
| 83 | * particular combination. If not existent a new instance is created and linked | ||
| 84 | * to the &drm_gem_object. | ||
| 85 | * | ||
| 86 | * &drm_gpuvm_bo structures, since unique for a given &drm_gpuvm, are also used | ||
| 87 | * as entry for the &drm_gpuvm's lists of external and evicted objects. Those | ||
| 88 | * lists are maintained in order to accelerate locking of dma-resv locks and | ||
| 89 | * validation of evicted objects bound in a &drm_gpuvm. For instance, all | ||
| 90 | * &drm_gem_object's &dma_resv of a given &drm_gpuvm can be locked by calling | ||
| 91 | * drm_gpuvm_exec_lock(). Once locked drivers can call drm_gpuvm_validate() in | ||
| 92 | * order to validate all evicted &drm_gem_objects. It is also possible to lock | ||
| 93 | * additional &drm_gem_objects by providing the corresponding parameters to | ||
| 94 | * drm_gpuvm_exec_lock() as well as open code the &drm_exec loop while making | ||
| 95 | * use of helper functions such as drm_gpuvm_prepare_range() or | ||
| 96 | * drm_gpuvm_prepare_objects(). | ||
| 97 | * | ||
| 98 | * Every bound &drm_gem_object is treated as external object when its &dma_resv | ||
| 99 | * structure is different than the &drm_gpuvm's common &dma_resv structure. | ||
| 64 | */ | 100 | */ |
| 65 | 101 | ||
| 66 | /** | 102 | /** |
| @@ -386,21 +422,42 @@ · 319 unmodified lines | |||
| 386 | /** | 422 | /** |
| 387 | * DOC: Locking | 423 | * DOC: Locking |
| 388 | * | 424 | * |
| 389 | * Generally, the GPU VA manager does not take care of locking itself, it is | 425 | * In terms of managing &drm_gpuva entries DRM GPUVM does not take care of |
| 390 | * the drivers responsibility to take care about locking. Drivers might want to | 426 | * locking itself, it is the drivers responsibility to take care about locking. |
| 391 | * protect the following operations: inserting, removing and iterating | 427 | * Drivers might want to protect the following operations: inserting, removing |
| 392 | * &drm_gpuva objects as well as generating all kinds of operations, such as | 428 | * and iterating &drm_gpuva objects as well as generating all kinds of |
| 393 | * split / merge or prefetch. | 429 | * operations, such as split / merge or prefetch. |
| 394 | * | 430 | * |
| 395 | * The GPU VA manager also does not take care of the locking of the backing | 431 | * DRM GPUVM also does not take care of the locking of the backing |
| 396 | * &drm_gem_object buffers GPU VA lists by itself; drivers are responsible to | 432 | * &drm_gem_object buffers GPU VA lists and &drm_gpuvm_bo abstractions by |
| 397 | * enforce mutual exclusion using either the GEMs dma_resv lock or alternatively | 433 | * itself; drivers are responsible to enforce mutual exclusion using either the |
| 398 | * a driver specific external lock. For the latter see also | 434 | * GEMs dma_resv lock or alternatively a driver specific external lock. For the |
| 399 | * drm_gem_gpuva_set_lock(). | 435 | * latter see also drm_gem_gpuva_set_lock(). |
| 400 | * | 436 | * |
| 401 | * However, the GPU VA manager contains lockdep checks to ensure callers of its | 437 | * However, DRM GPUVM contains lockdep checks to ensure callers of its API hold |
| 402 | * API hold the corresponding lock whenever the &drm_gem_objects GPU VA list is | 438 | * the corresponding lock whenever the &drm_gem_objects GPU VA list is accessed |
| 403 | * accessed by functions such as drm_gpuva_link() or drm_gpuva_unlink(). | 439 | * by functions such as drm_gpuva_link() or drm_gpuva_unlink(), but also |
| 440 | * drm_gpuvm_bo_obtain() and drm_gpuvm_bo_put(). | ||
| 441 | * | ||
| 442 | * The latter is required since on creation and destruction of a &drm_gpuvm_bo | ||
| 443 | * the &drm_gpuvm_bo is attached / removed from the &drm_gem_objects gpuva list. | ||
| 444 | * Subsequent calls to drm_gpuvm_bo_obtain() for the same &drm_gpuvm and | ||
| 445 | * &drm_gem_object must be able to observe previous creations and destructions | ||
| 446 | * of &drm_gpuvm_bos in order to keep instances unique. | ||
| 447 | * | ||
| 448 | * The &drm_gpuvm's lists for keeping track of external and evicted objects are | ||
| 449 | * protected against concurrent insertion / removal and iteration internally. | ||
| 450 | * | ||
| 451 | * However, drivers still need ensure to protect concurrent calls to functions | ||
| 452 | * iterating those lists, namely drm_gpuvm_prepare_objects() and | ||
| 453 | * drm_gpuvm_validate(). | ||
| 454 | * | ||
| 455 | * Alternatively, drivers can set the &DRM_GPUVM_RESV_PROTECTED flag to indicate | ||
| 456 | * that the corresponding &dma_resv locks are held in order to protect the | ||
| 457 | * lists. If &DRM_GPUVM_RESV_PROTECTED is set, internal locking is disabled and | ||
| 458 | * the corresponding lockdep checks are enabled. This is an optimization for | ||
| 459 | * drivers which are capable of taking the corresponding &dma_resv locks and | ||
| 460 | * hence do not require internal locking. | ||
| 404 | */ | 461 | */ |
| 405 | 462 | ||
| 406 | /** | 463 | /** |
| @@ -430,6 +487,7 @@ · 23 unmodified lines | |||
| 430 | * { | 487 | * { |
| 431 | * struct drm_gpuva_ops *ops; | 488 | * struct drm_gpuva_ops *ops; |
| 432 | * struct drm_gpuva_op *op | 489 | * struct drm_gpuva_op *op |
| 490 | * struct drm_gpuvm_bo *vm_bo; | ||
| 433 | * | 491 | * |
| 434 | * driver_lock_va_space(); | 492 | * driver_lock_va_space(); |
| 435 | * ops = drm_gpuvm_sm_map_ops_create(gpuvm, addr, range, | 493 | * ops = drm_gpuvm_sm_map_ops_create(gpuvm, addr, range, |
| @@ -437,6 +495,10 @@ · 1 unmodified line | |||
| 437 | * if (IS_ERR(ops)) | 495 | * if (IS_ERR(ops)) |
| 438 | * return PTR_ERR(ops); | 496 | * return PTR_ERR(ops); |
| 439 | * | 497 | * |
| 498 | * vm_bo = drm_gpuvm_bo_obtain(gpuvm, obj); | ||
| 499 | * if (IS_ERR(vm_bo)) | ||
| 500 | * return PTR_ERR(vm_bo); | ||
| 501 | * | ||
| 440 | * drm_gpuva_for_each_op(op, ops) { | 502 | * drm_gpuva_for_each_op(op, ops) { |
| 441 | * struct drm_gpuva *va; | 503 | * struct drm_gpuva *va; |
| 442 | * | 504 | * |
| @@ -449,7 +511,7 @@ · 6 unmodified lines | |||
| 449 | * | 511 | * |
| 450 | * driver_vm_map(); | 512 | * driver_vm_map(); |
| 451 | * drm_gpuva_map(gpuvm, va, &op->map); | 513 | * drm_gpuva_map(gpuvm, va, &op->map); |
| 452 | * drm_gpuva_link(va); | 514 | * drm_gpuva_link(va, vm_bo); |
| 453 | * | 515 | * |
| 454 | * break; | 516 | * break; |
| 455 | * case DRM_GPUVA_OP_REMAP: { | 517 | * case DRM_GPUVA_OP_REMAP: { |
| @@ -476,11 +538,11 @@ · 20 unmodified lines | |||
| 476 | * driver_vm_remap(); | 538 | * driver_vm_remap(); |
| 477 | * drm_gpuva_remap(prev, next, &op->remap); | 539 | * drm_gpuva_remap(prev, next, &op->remap); |
| 478 | * | 540 | * |
| 479 | * drm_gpuva_unlink(va); | ||
| 480 | * if (prev) | 541 | * if (prev) |
| 481 | * drm_gpuva_link(prev); | 542 | * drm_gpuva_link(prev, va->vm_bo); |
| 482 | * if (next) | 543 | * if (next) |
| 483 | * drm_gpuva_link(next); | 544 | * drm_gpuva_link(next, va->vm_bo); |
| 545 | * drm_gpuva_unlink(va); | ||
| 484 | * | 546 | * |
| 485 | * break; | 547 | * break; |
| 486 | * } | 548 | * } |
| @@ -496,6 +558,7 @@ · 9 unmodified lines | |||
| 496 | * break; | 558 | * break; |
| 497 | * } | 559 | * } |
| 498 | * } | 560 | * } |
| 561 | * drm_gpuvm_bo_put(vm_bo); | ||
| 499 | * driver_unlock_va_space(); | 562 | * driver_unlock_va_space(); |
| 500 | * | 563 | * |
| 501 | * return 0; | 564 | * return 0; |
| @@ -505,6 +568,7 @@ · 3 unmodified lines | |||
| 505 | * | 568 | * |
| 506 | * struct driver_context { | 569 | * struct driver_context { |
| 507 | * struct drm_gpuvm *gpuvm; | 570 | * struct drm_gpuvm *gpuvm; |
| 571 | * struct drm_gpuvm_bo *vm_bo; | ||
| 508 | * struct drm_gpuva *new_va; | 572 | * struct drm_gpuva *new_va; |
| 509 | * struct drm_gpuva *prev_va; | 573 | * struct drm_gpuva *prev_va; |
| 510 | * struct drm_gpuva *next_va; | 574 | * struct drm_gpuva *next_va; |
| @@ -525,6 +589,7 @@ · 14 unmodified lines | |||
| 525 | * struct drm_gem_object *obj, u64 offset) | 589 | * struct drm_gem_object *obj, u64 offset) |
| 526 | * { | 590 | * { |
| 527 | * struct driver_context ctx; | 591 | * struct driver_context ctx; |
| 592 | * struct drm_gpuvm_bo *vm_bo; | ||
| 528 | * struct drm_gpuva_ops *ops; | 593 | * struct drm_gpuva_ops *ops; |
| 529 | * struct drm_gpuva_op *op; | 594 | * struct drm_gpuva_op *op; |
| 530 | * int ret = 0; | 595 | * int ret = 0; |
| @@ -534,16 +599,23 @@ · 3 unmodified lines | |||
| 534 | * ctx.new_va = kzalloc(sizeof(*ctx.new_va), GFP_KERNEL); | 599 | * ctx.new_va = kzalloc(sizeof(*ctx.new_va), GFP_KERNEL); |
| 535 | * ctx.prev_va = kzalloc(sizeof(*ctx.prev_va), GFP_KERNEL); | 600 | * ctx.prev_va = kzalloc(sizeof(*ctx.prev_va), GFP_KERNEL); |
| 536 | * ctx.next_va = kzalloc(sizeof(*ctx.next_va), GFP_KERNEL); | 601 | * ctx.next_va = kzalloc(sizeof(*ctx.next_va), GFP_KERNEL); |
| 537 | * if (!ctx.new_va || !ctx.prev_va || !ctx.next_va) { | 602 | * ctx.vm_bo = drm_gpuvm_bo_create(gpuvm, obj); |
| 603 | * if (!ctx.new_va || !ctx.prev_va || !ctx.next_va || !vm_bo) { | ||
| 538 | * ret = -ENOMEM; | 604 | * ret = -ENOMEM; |
| 539 | * goto out; | 605 | * goto out; |
| 540 | * } | 606 | * } |
| 541 | * | 607 | * |
| 608 | * // Typically protected with a driver specific GEM gpuva lock | ||
| 609 | * // used in the fence signaling path for drm_gpuva_link() and | ||
| 610 | * // drm_gpuva_unlink(), hence pre-allocate. | ||
| 611 | * ctx.vm_bo = drm_gpuvm_bo_obtain_prealloc(ctx.vm_bo); | ||
| 612 | * | ||
| 542 | * driver_lock_va_space(); | 613 | * driver_lock_va_space(); |
| 543 | * ret = drm_gpuvm_sm_map(gpuvm, &ctx, addr, range, obj, offset); | 614 | * ret = drm_gpuvm_sm_map(gpuvm, &ctx, addr, range, obj, offset); |
| 544 | * driver_unlock_va_space(); | 615 | * driver_unlock_va_space(); |
| 545 | * | 616 | * |
| 546 | * out: | 617 | * out: |
| 618 | * drm_gpuvm_bo_put(ctx.vm_bo); | ||
| 547 | * kfree(ctx.new_va); | 619 | * kfree(ctx.new_va); |
| 548 | * kfree(ctx.prev_va); | 620 | * kfree(ctx.prev_va); |
| 549 | * kfree(ctx.next_va); | 621 | * kfree(ctx.next_va); |
| @@ -556,7 +628,7 @@ · 6 unmodified lines | |||
| 556 | * | 628 | * |
| 557 | * drm_gpuva_map(ctx->vm, ctx->new_va, &op->map); | 629 | * drm_gpuva_map(ctx->vm, ctx->new_va, &op->map); |
| 558 | * | 630 | * |
| 559 | * drm_gpuva_link(ctx->new_va); | 631 | * drm_gpuva_link(ctx->new_va, ctx->vm_bo); |
| 560 | * | 632 | * |
| 561 | * // prevent the new GPUVA from being freed in | 633 | * // prevent the new GPUVA from being freed in |
| 562 | * // driver_mapping_create() | 634 | * // driver_mapping_create() |
| @@ -568,22 +640,23 @@ · 5 unmodified lines | |||
| 568 | * int driver_gpuva_remap(struct drm_gpuva_op *op, void *__ctx) | 640 | * int driver_gpuva_remap(struct drm_gpuva_op *op, void *__ctx) |
| 569 | * { | 641 | * { |
| 570 | * struct driver_context *ctx = __ctx; | 642 | * struct driver_context *ctx = __ctx; |
| 643 | * struct drm_gpuva *va = op->remap.unmap->va; | ||
| 571 | * | 644 | * |
| 572 | * drm_gpuva_remap(ctx->prev_va, ctx->next_va, &op->remap); | 645 | * drm_gpuva_remap(ctx->prev_va, ctx->next_va, &op->remap); |
| 573 | * | 646 | * |
| 574 | * drm_gpuva_unlink(op->remap.unmap->va); | ||
| 575 | * kfree(op->remap.unmap->va); | ||
| 576 | * | ||
| 577 | * if (op->remap.prev) { | 647 | * if (op->remap.prev) { |
| 578 | * drm_gpuva_link(ctx->prev_va); | 648 | * drm_gpuva_link(ctx->prev_va, va->vm_bo); |
| 579 | * ctx->prev_va = NULL; | 649 | * ctx->prev_va = NULL; |
| 580 | * } | 650 | * } |
| 581 | * | 651 | * |
| 582 | * if (op->remap.next) { | 652 | * if (op->remap.next) { |
| 583 | * drm_gpuva_link(ctx->next_va); | 653 | * drm_gpuva_link(ctx->next_va, va->vm_bo); |
| 584 | * ctx->next_va = NULL; | 654 | * ctx->next_va = NULL; |
| 585 | * } | 655 | * } |
| 586 | * | 656 | * |
| 657 | * drm_gpuva_unlink(va); | ||
| 658 | * kfree(va); | ||
| 659 | * | ||
| 587 | * return 0; | 660 | * return 0; |
| 588 | * } | 661 | * } |
| 589 | * | 662 | * |
| @@ -597,6 +670,201 @@ · 7 unmodified lines | |||
| 597 | * } | 670 | * } |
| 598 | */ | 671 | */ |
| 599 | 672 | ||
| 673 | /** | ||
| 674 | * get_next_vm_bo_from_list() - get the next vm_bo element | ||
| 675 | * @__gpuvm: the &drm_gpuvm | ||
| 676 | * @__list_name: the name of the list we're iterating on | ||
| 677 | * @__local_list: a pointer to the local list used to store already iterated items | ||
| 678 | * @__prev_vm_bo: the previous element we got from get_next_vm_bo_from_list() | ||
| 679 | * | ||
| 680 | * This helper is here to provide lockless list iteration. Lockless as in, the | ||
| 681 | * iterator releases the lock immediately after picking the first element from | ||
| 682 | * the list, so list insertion deletion can happen concurrently. | ||
| 683 | * | ||
| 684 | * Elements popped from the original list are kept in a local list, so removal | ||
| 685 | * and is_empty checks can still happen while we're iterating the list. | ||
| 686 | */ | ||
| 687 | #define get_next_vm_bo_from_list(__gpuvm, __list_name, __local_list, __prev_vm_bo) \ | ||
| 688 | ({ \ | ||
| 689 | struct drm_gpuvm_bo *__vm_bo = NULL; \ | ||
| 690 | \ | ||
| 691 | drm_gpuvm_bo_put(__prev_vm_bo); \ | ||
| 692 | \ | ||
| 693 | spin_lock(&(__gpuvm)->__list_name.lock); \ | ||
| 694 | if (!(__gpuvm)->__list_name.local_list) \ | ||
| 695 | (__gpuvm)->__list_name.local_list = __local_list; \ | ||
| 696 | else \ | ||
| 697 | drm_WARN_ON((__gpuvm)->drm, \ | ||
| 698 | (__gpuvm)->__list_name.local_list != __local_list); \ | ||
| 699 | \ | ||
| 700 | while (!list_empty(&(__gpuvm)->__list_name.list)) { \ | ||
| 701 | __vm_bo = list_first_entry(&(__gpuvm)->__list_name.list, \ | ||
| 702 | struct drm_gpuvm_bo, \ | ||
| 703 | list.entry.__list_name); \ | ||
| 704 | if (kref_get_unless_zero(&__vm_bo->kref)) { \ | ||
| 705 | list_move_tail(&(__vm_bo)->list.entry.__list_name, \ | ||
| 706 | __local_list); \ | ||
| 707 | break; \ | ||
| 708 | } else { \ | ||
| 709 | list_del_init(&(__vm_bo)->list.entry.__list_name); \ | ||
| 710 | __vm_bo = NULL; \ | ||
| 711 | } \ | ||
| 712 | } \ | ||
| 713 | spin_unlock(&(__gpuvm)->__list_name.lock); \ | ||
| 714 | \ | ||
| 715 | __vm_bo; \ | ||
| 716 | }) | ||
| 717 | |||
| 718 | /** | ||
| 719 | * for_each_vm_bo_in_list() - internal vm_bo list iterator | ||
| 720 | * @__gpuvm: the &drm_gpuvm | ||
| 721 | * @__list_name: the name of the list we're iterating on | ||
| 722 | * @__local_list: a pointer to the local list used to store already iterated items | ||
| 723 | * @__vm_bo: the struct drm_gpuvm_bo to assign in each iteration step | ||
| 724 | * | ||
| 725 | * This helper is here to provide lockless list iteration. Lockless as in, the | ||
| 726 | * iterator releases the lock immediately after picking the first element from the | ||
| 727 | * list, hence list insertion and deletion can happen concurrently. | ||
| 728 | * | ||
| 729 | * It is not allowed to re-assign the vm_bo pointer from inside this loop. | ||
| 730 | * | ||
| 731 | * Typical use: | ||
| 732 | * | ||
| 733 | * struct drm_gpuvm_bo *vm_bo; | ||
| 734 | * LIST_HEAD(my_local_list); | ||
| 735 | * | ||
| 736 | * ret = 0; | ||
| 737 | * for_each_vm_bo_in_list(gpuvm, <list_name>, &my_local_list, vm_bo) { | ||
| 738 | * ret = do_something_with_vm_bo(..., vm_bo); | ||
| 739 | * if (ret) | ||
| 740 | * break; | ||
| 741 | * } | ||
| 742 | * // Drop ref in case we break out of the loop. | ||
| 743 | * drm_gpuvm_bo_put(vm_bo); | ||
| 744 | * restore_vm_bo_list(gpuvm, <list_name>, &my_local_list); | ||
| 745 | * | ||
| 746 | * | ||
| 747 | * Only used for internal list iterations, not meant to be exposed to the outside | ||
| 748 | * world. | ||
| 749 | */ | ||
| 750 | #define for_each_vm_bo_in_list(__gpuvm, __list_name, __local_list, __vm_bo) \ | ||
| 751 | for (__vm_bo = get_next_vm_bo_from_list(__gpuvm, __list_name, \ | ||
| 752 | __local_list, NULL); \ | ||
| 753 | __vm_bo; \ | ||
| 754 | __vm_bo = get_next_vm_bo_from_list(__gpuvm, __list_name, \ | ||
| 755 | __local_list, __vm_bo)) | ||
| 756 | |||
| 757 | static void | ||
| 758 | __restore_vm_bo_list(struct drm_gpuvm *gpuvm, spinlock_t *lock, | ||
| 759 | struct list_head *list, struct list_head **local_list) | ||
| 760 | { | ||
| 761 | /* Merge back the two lists, moving local list elements to the | ||
| 762 | * head to preserve previous ordering, in case it matters. | ||
| 763 | */ | ||
| 764 | spin_lock(lock); | ||
| 765 | if (*local_list) { | ||
| 766 | list_splice(*local_list, list); | ||
| 767 | *local_list = NULL; | ||
| 768 | } | ||
| 769 | spin_unlock(lock); | ||
| 770 | } | ||
| 771 | |||
| 772 | /** | ||
| 773 | * restore_vm_bo_list() - move vm_bo elements back to their original list | ||
| 774 | * @__gpuvm: the &drm_gpuvm | ||
| 775 | * @__list_name: the name of the list we're iterating on | ||
| 776 | * | ||
| 777 | * When we're done iterating a vm_bo list, we should call restore_vm_bo_list() | ||
| 778 | * to restore the original state and let new iterations take place. | ||
| 779 | */ | ||
| 780 | #define restore_vm_bo_list(__gpuvm, __list_name) \ | ||
| 781 | __restore_vm_bo_list((__gpuvm), &(__gpuvm)->__list_name.lock, \ | ||
| 782 | &(__gpuvm)->__list_name.list, \ | ||
| 783 | &(__gpuvm)->__list_name.local_list) | ||
| 784 | |||
| 785 | static void | ||
| 786 | cond_spin_lock(spinlock_t *lock, bool cond) | ||
| 787 | { | ||
| 788 | if (cond) | ||
| 789 | spin_lock(lock); | ||
| 790 | } | ||
| 791 | |||
| 792 | static void | ||
| 793 | cond_spin_unlock(spinlock_t *lock, bool cond) | ||
| 794 | { | ||
| 795 | if (cond) | ||
| 796 | spin_unlock(lock); | ||
| 797 | } | ||
| 798 | |||
| 799 | static void | ||
| 800 | __drm_gpuvm_bo_list_add(struct drm_gpuvm *gpuvm, spinlock_t *lock, | ||
| 801 | struct list_head *entry, struct list_head *list) | ||
| 802 | { | ||
| 803 | cond_spin_lock(lock, !!lock); | ||
| 804 | if (list_empty(entry)) | ||
| 805 | list_add_tail(entry, list); | ||
| 806 | cond_spin_unlock(lock, !!lock); | ||
| 807 | } | ||
| 808 | |||
| 809 | /** | ||
| 810 | * drm_gpuvm_bo_list_add() - insert a vm_bo into the given list | ||
| 811 | * @__vm_bo: the &drm_gpuvm_bo | ||
| 812 | * @__list_name: the name of the list to insert into | ||
| 813 | * @__lock: whether to lock with the internal spinlock | ||
| 814 | * | ||
| 815 | * Inserts the given @__vm_bo into the list specified by @__list_name. | ||
| 816 | */ | ||
| 817 | #define drm_gpuvm_bo_list_add(__vm_bo, __list_name, __lock) \ | ||
| 818 | __drm_gpuvm_bo_list_add((__vm_bo)->vm, \ | ||
| 819 | __lock ? &(__vm_bo)->vm->__list_name.lock : \ | ||
| 820 | NULL, \ | ||
| 821 | &(__vm_bo)->list.entry.__list_name, \ | ||
| 822 | &(__vm_bo)->vm->__list_name.list) | ||
| 823 | |||
| 824 | static void | ||
| 825 | __drm_gpuvm_bo_list_del(struct drm_gpuvm *gpuvm, spinlock_t *lock, | ||
| 826 | struct list_head *entry, bool init) | ||
| 827 | { | ||
| 828 | cond_spin_lock(lock, !!lock); | ||
| 829 | if (init) { | ||
| 830 | if (!list_empty(entry)) | ||
| 831 | list_del_init(entry); | ||
| 832 | } else { | ||
| 833 | list_del(entry); | ||
| 834 | } | ||
| 835 | cond_spin_unlock(lock, !!lock); | ||
| 836 | } | ||
| 837 | |||
| 838 | /** | ||
| 839 | * drm_gpuvm_bo_list_del_init() - remove a vm_bo from the given list | ||
| 840 | * @__vm_bo: the &drm_gpuvm_bo | ||
| 841 | * @__list_name: the name of the list to insert into | ||
| 842 | * @__lock: whether to lock with the internal spinlock | ||
| 843 | * | ||
| 844 | * Removes the given @__vm_bo from the list specified by @__list_name. | ||
| 845 | */ | ||
| 846 | #define drm_gpuvm_bo_list_del_init(__vm_bo, __list_name, __lock) \ | ||
| 847 | __drm_gpuvm_bo_list_del((__vm_bo)->vm, \ | ||
| 848 | __lock ? &(__vm_bo)->vm->__list_name.lock : \ | ||
| 849 | NULL, \ | ||
| 850 | &(__vm_bo)->list.entry.__list_name, \ | ||
| 851 | true) | ||
| 852 | |||
| 853 | /** | ||
| 854 | * drm_gpuvm_bo_list_del() - remove a vm_bo from the given list | ||
| 855 | * @__vm_bo: the &drm_gpuvm_bo | ||
| 856 | * @__list_name: the name of the list to insert into | ||
| 857 | * @__lock: whether to lock with the internal spinlock | ||
| 858 | * | ||
| 859 | * Removes the given @__vm_bo from the list specified by @__list_name. | ||
| 860 | */ | ||
| 861 | #define drm_gpuvm_bo_list_del(__vm_bo, __list_name, __lock) \ | ||
| 862 | __drm_gpuvm_bo_list_del((__vm_bo)->vm, \ | ||
| 863 | __lock ? &(__vm_bo)->vm->__list_name.lock : \ | ||
| 864 | NULL, \ | ||
| 865 | &(__vm_bo)->list.entry.__list_name, \ | ||
| 866 | false) | ||
| 867 | |||
| 600 | #define to_drm_gpuva(__node) container_of((__node), struct drm_gpuva, rb.node) | 868 | #define to_drm_gpuva(__node) container_of((__node), struct drm_gpuva, rb.node) |
| 601 | 869 | ||
| 602 | #define GPUVA_START(node) ((node)->va.addr) | 870 | #define GPUVA_START(node) ((node)->va.addr) |
| @@ -618,8 +886,14 @@ drm_gpuvm_check_overflow(u64 addr, u64 range) · 15 unmodified lines | |||
| 618 | { | 886 | { |
| 619 | u64 end; | 887 | u64 end; |
| 620 | 888 | ||
| 621 | return WARN(check_add_overflow(addr, range, &end), | 889 | return check_add_overflow(addr, range, &end); |
| 622 | "GPUVA address limited to %zu bytes.\n", sizeof(end)); | 890 | } |
| 891 | |||
| 892 | static bool | ||
| 893 | drm_gpuvm_warn_check_overflow(struct drm_gpuvm *gpuvm, u64 addr, u64 range) | ||
| 894 | { | ||
| 895 | return drm_WARN(gpuvm->drm, drm_gpuvm_check_overflow(addr, range), | ||
| 896 | "GPUVA address limited to %zu bytes.\n", sizeof(addr)); | ||
| 623 | } | 897 | } |
| 624 | 898 | ||
| 625 | static bool | 899 | static bool |
| @@ -643,7 +917,18 @@ drm_gpuvm_in_kernel_node(struct drm_gpuvm *gpuvm, u64 addr, u64 range) · 17 unmodified lines | |||
| 643 | return krange && addr < kend && kstart < end; | 917 | return krange && addr < kend && kstart < end; |
| 644 | } | 918 | } |
| 645 | 919 | ||
| 646 | static bool | 920 | /** |
| 921 | * drm_gpuvm_range_valid() - checks whether the given range is valid for the | ||
| 922 | * given &drm_gpuvm | ||
| 923 | * @gpuvm: the GPUVM to check the range for | ||
| 924 | * @addr: the base address | ||
| 925 | * @range: the range starting from the base address | ||
| 926 | * | ||
| 927 | * Checks whether the range is within the GPUVM's managed boundaries. | ||
| 928 | * | ||
| 929 | * Returns: true for a valid range, false otherwise | ||
| 930 | */ | ||
| 931 | bool | ||
| 647 | drm_gpuvm_range_valid(struct drm_gpuvm *gpuvm, | 932 | drm_gpuvm_range_valid(struct drm_gpuvm *gpuvm, |
| 648 | u64 addr, u64 range) | 933 | u64 addr, u64 range) |
| 649 | { | 934 | { |
| @@ -651,11 +936,52 @@ drm_gpuvm_range_valid(struct drm_gpuvm *gpuvm, · 1 unmodified line | |||
| 651 | drm_gpuvm_in_mm_range(gpuvm, addr, range) && | 936 | drm_gpuvm_in_mm_range(gpuvm, addr, range) && |
| 652 | !drm_gpuvm_in_kernel_node(gpuvm, addr, range); | 937 | !drm_gpuvm_in_kernel_node(gpuvm, addr, range); |
| 653 | } | 938 | } |
| 939 | EXPORT_SYMBOL_GPL(drm_gpuvm_range_valid); | ||
| 940 | |||
| 941 | static void | ||
| 942 | drm_gpuvm_gem_object_free(struct drm_gem_object *obj) | ||
| 943 | { | ||
| 944 | drm_gem_object_release(obj); | ||
| 945 | kfree(obj); | ||
| 946 | } | ||
| 947 | |||
| 948 | static const struct drm_gem_object_funcs drm_gpuvm_object_funcs = { | ||
| 949 | .free = drm_gpuvm_gem_object_free, | ||
| 950 | }; | ||
| 951 | |||
| 952 | /** | ||
| 953 | * drm_gpuvm_resv_object_alloc() - allocate a dummy &drm_gem_object | ||
| 954 | * @drm: the drivers &drm_device | ||
| 955 | * | ||
| 956 | * Allocates a dummy &drm_gem_object which can be passed to drm_gpuvm_init() in | ||
| 957 | * order to serve as root GEM object providing the &drm_resv shared across | ||
| 958 | * &drm_gem_objects local to a single GPUVM. | ||
| 959 | * | ||
| 960 | * Returns: the &drm_gem_object on success, NULL on failure | ||
| 961 | */ | ||
| 962 | struct drm_gem_object * | ||
| 963 | drm_gpuvm_resv_object_alloc(struct drm_device *drm) | ||
| 964 | { | ||
| 965 | struct drm_gem_object *obj; | ||
| 966 | |||
| 967 | obj = kzalloc(sizeof(*obj), GFP_KERNEL); | ||
| 968 | if (!obj) | ||
| 969 | return NULL; | ||
| 970 | |||
| 971 | obj->funcs = &drm_gpuvm_object_funcs; | ||
| 972 | drm_gem_private_object_init(drm, obj, 0); | ||
| 973 | |||
| 974 | return obj; | ||
| 975 | } | ||
| 976 | EXPORT_SYMBOL_GPL(drm_gpuvm_resv_object_alloc); | ||
| 654 | 977 | ||
| 655 | /** | 978 | /** |
| 656 | * drm_gpuvm_init() - initialize a &drm_gpuvm | 979 | * drm_gpuvm_init() - initialize a &drm_gpuvm |
| 657 | * @gpuvm: pointer to the &drm_gpuvm to initialize | 980 | * @gpuvm: pointer to the &drm_gpuvm to initialize |
| 658 | * @name: the name of the GPU VA space | 981 | * @name: the name of the GPU VA space |
| 982 | * @flags: the &drm_gpuvm_flags for this GPUVM | ||
| 983 | * @drm: the &drm_device this VM resides in | ||
| 984 | * @r_obj: the resv &drm_gem_object providing the GPUVM's common &dma_resv | ||
| 659 | * @start_offset: the start offset of the GPU VA space | 985 | * @start_offset: the start offset of the GPU VA space |
| 660 | * @range: the size of the GPU VA space | 986 | * @range: the size of the GPU VA space |
| 661 | * @reserve_offset: the start of the kernel reserved GPU VA area | 987 | * @reserve_offset: the start of the kernel reserved GPU VA area |
| @@ -668,8 +994,10 @@ drm_gpuvm_range_valid(struct drm_gpuvm *gpuvm, · 6 unmodified lines | |||
| 668 | * &name is expected to be managed by the surrounding driver structures. | 994 | * &name is expected to be managed by the surrounding driver structures. |
| 669 | */ | 995 | */ |
| 670 | void | 996 | void |
| 671 | drm_gpuvm_init(struct drm_gpuvm *gpuvm, | 997 | drm_gpuvm_init(struct drm_gpuvm *gpuvm, const char *name, |
| 672 | const char *name, | 998 | enum drm_gpuvm_flags flags, |
| 999 | struct drm_device *drm, | ||
| 1000 | struct drm_gem_object *r_obj, | ||
| 673 | u64 start_offset, u64 range, | 1001 | u64 start_offset, u64 range, |
| 674 | u64 reserve_offset, u64 reserve_range, | 1002 | u64 reserve_offset, u64 reserve_range, |
| 675 | const struct drm_gpuvm_ops *ops) | 1003 | const struct drm_gpuvm_ops *ops) |
| @@ -677,45 +1005,713 @@ drm_gpuvm_init(struct drm_gpuvm *gpuvm, · 1 unmodified line | |||
| 677 | gpuvm->rb.tree = RB_ROOT_CACHED; | 1005 | gpuvm->rb.tree = RB_ROOT_CACHED; |
| 678 | INIT_LIST_HEAD(&gpuvm->rb.list); | 1006 | INIT_LIST_HEAD(&gpuvm->rb.list); |
| 679 | 1007 | ||
| 680 | drm_gpuvm_check_overflow(start_offset, range); | 1008 | INIT_LIST_HEAD(&gpuvm->extobj.list); |
| 681 | gpuvm->mm_start = start_offset; | 1009 | spin_lock_init(&gpuvm->extobj.lock); |
| 682 | gpuvm->mm_range = range; | 1010 | |
| 1011 | INIT_LIST_HEAD(&gpuvm->evict.list); | ||
| 1012 | spin_lock_init(&gpuvm->evict.lock); | ||
| 1013 | |||
| 1014 | kref_init(&gpuvm->kref); | ||
| 683 | 1015 | ||
| 684 | gpuvm->name = name ? name : "unknown"; | 1016 | gpuvm->name = name ? name : "unknown"; |
| 1017 | gpuvm->flags = flags; | ||
| 685 | gpuvm->ops = ops; | 1018 | gpuvm->ops = ops; |
| 1019 | gpuvm->drm = drm; | ||
| 1020 | gpuvm->r_obj = r_obj; | ||
| 686 | 1021 | ||
| 687 | memset(&gpuvm->kernel_alloc_node, 0, sizeof(struct drm_gpuva)); | 1022 | drm_gem_object_get(r_obj); |
| 1023 | |||
| 1024 | drm_gpuvm_warn_check_overflow(gpuvm, start_offset, range); | ||
| 1025 | gpuvm->mm_start = start_offset; | ||
| 1026 | gpuvm->mm_range = range; | ||
| 688 | 1027 | ||
| 1028 | memset(&gpuvm->kernel_alloc_node, 0, sizeof(struct drm_gpuva)); | ||
| 689 | if (reserve_range) { | 1029 | if (reserve_range) { |
| 690 | gpuvm->kernel_alloc_node.va.addr = reserve_offset; | 1030 | gpuvm->kernel_alloc_node.va.addr = reserve_offset; |
| 691 | gpuvm->kernel_alloc_node.va.range = reserve_range; | 1031 | gpuvm->kernel_alloc_node.va.range = reserve_range; |
| 692 | 1032 | ||
| 693 | if (likely(!drm_gpuvm_check_overflow(reserve_offset, | 1033 | if (likely(!drm_gpuvm_warn_check_overflow(gpuvm, reserve_offset, |
| 694 | reserve_range))) | 1034 | reserve_range))) |
| 695 | __drm_gpuva_insert(gpuvm, &gpuvm->kernel_alloc_node); | 1035 | __drm_gpuva_insert(gpuvm, &gpuvm->kernel_alloc_node); |
| 696 | } | 1036 | } |
| 697 | } | 1037 | } |
| 698 | EXPORT_SYMBOL_GPL(drm_gpuvm_init); | 1038 | EXPORT_SYMBOL_GPL(drm_gpuvm_init); |
| 699 | 1039 | ||
| 1040 | static void | ||
| 1041 | drm_gpuvm_fini(struct drm_gpuvm *gpuvm) | ||
| 1042 | { | ||
| 1043 | gpuvm->name = NULL; | ||
| 1044 | |||
| 1045 | if (gpuvm->kernel_alloc_node.va.range) | ||
| 1046 | __drm_gpuva_remove(&gpuvm->kernel_alloc_node); | ||
| 1047 | |||
| 1048 | drm_WARN(gpuvm->drm, !RB_EMPTY_ROOT(&gpuvm->rb.tree.rb_root), | ||
| 1049 | "GPUVA tree is not empty, potentially leaking memory.\n"); | ||
| 1050 | |||
| 1051 | drm_WARN(gpuvm->drm, !list_empty(&gpuvm->extobj.list), | ||
| 1052 | "Extobj list should be empty.\n"); | ||
| 1053 | drm_WARN(gpuvm->drm, !list_empty(&gpuvm->evict.list), | ||
| 1054 | "Evict list should be empty.\n"); | ||
| 1055 | |||
| 1056 | drm_gem_object_put(gpuvm->r_obj); | ||
| 1057 | } | ||
| 1058 | |||
| 1059 | static void | ||
| 1060 | drm_gpuvm_free(struct kref *kref) | ||
| 1061 | { | ||
| 1062 | struct drm_gpuvm *gpuvm = container_of(kref, struct drm_gpuvm, kref); | ||
| 1063 | |||
| 1064 | drm_gpuvm_fini(gpuvm); | ||
| 1065 | |||
| 1066 | if (drm_WARN_ON(gpuvm->drm, !gpuvm->ops->vm_free)) | ||
| 1067 | return; | ||
| 1068 | |||
| 1069 | gpuvm->ops->vm_free(gpuvm); | ||
| 1070 | } | ||
| 1071 | |||
| 700 | /** | 1072 | /** |
| 701 | * drm_gpuvm_destroy() - cleanup a &drm_gpuvm | 1073 | * drm_gpuvm_put() - drop a struct drm_gpuvm reference |
| 702 | * @gpuvm: pointer to the &drm_gpuvm to clean up | 1074 | * @gpuvm: the &drm_gpuvm to release the reference of |
| 1075 | * | ||
| 1076 | * This releases a reference to @gpuvm. | ||
| 703 | * | 1077 | * |
| 704 | * Note that it is a bug to call this function on a manager that still | 1078 | * This function may be called from atomic context. |
| 705 | * holds GPU VA mappings. | ||
| 706 | */ | 1079 | */ |
| 707 | void | 1080 | void |
| 708 | drm_gpuvm_destroy(struct drm_gpuvm *gpuvm) | 1081 | drm_gpuvm_put(struct drm_gpuvm *gpuvm) |
| 709 | { | 1082 | { |
| 710 | gpuvm->name = NULL; | 1083 | if (gpuvm) |
| 1084 | kref_put(&gpuvm->kref, drm_gpuvm_free); | ||
| 1085 | } | ||
| 1086 | EXPORT_SYMBOL_GPL(drm_gpuvm_put); | ||
| 711 | 1087 | ||
| 712 | if (gpuvm->kernel_alloc_node.va.range) | 1088 | static int |
| 713 | __drm_gpuva_remove(&gpuvm->kernel_alloc_node); | 1089 | exec_prepare_obj(struct drm_exec *exec, struct drm_gem_object *obj, |
| 1090 | unsigned int num_fences) | ||
| 1091 | { | ||
| 1092 | return num_fences ? drm_exec_prepare_obj(exec, obj, num_fences) : | ||
| 1093 | drm_exec_lock_obj(exec, obj); | ||
| 1094 | } | ||
| 714 | 1095 | ||
| 715 | WARN(!RB_EMPTY_ROOT(&gpuvm->rb.tree.rb_root), | 1096 | /** |
| 716 | "GPUVA tree is not empty, potentially leaking memory."); | 1097 | * drm_gpuvm_prepare_vm() - prepare the GPUVMs common dma-resv |
| 1098 | * @gpuvm: the &drm_gpuvm | ||
| 1099 | * @exec: the &drm_exec context | ||
| 1100 | * @num_fences: the amount of &dma_fences to reserve | ||
| 1101 | * | ||
| 1102 | * Calls drm_exec_prepare_obj() for the GPUVMs dummy &drm_gem_object; if | ||
| 1103 | * @num_fences is zero drm_exec_lock_obj() is called instead. | ||
| 1104 | * | ||
| 1105 | * Using this function directly, it is the drivers responsibility to call | ||
| 1106 | * drm_exec_init() and drm_exec_fini() accordingly. | ||
| 1107 | * | ||
| 1108 | * Returns: 0 on success, negative error code on failure. | ||
| 1109 | */ | ||
| 1110 | int | ||
| 1111 | drm_gpuvm_prepare_vm(struct drm_gpuvm *gpuvm, | ||
| 1112 | struct drm_exec *exec, | ||
| 1113 | unsigned int num_fences) | ||
| 1114 | { | ||
| 1115 | return exec_prepare_obj(exec, gpuvm->r_obj, num_fences); | ||
| 1116 | } | ||
| 1117 | EXPORT_SYMBOL_GPL(drm_gpuvm_prepare_vm); | ||
| 1118 | |||
| 1119 | static int | ||
| 1120 | __drm_gpuvm_prepare_objects(struct drm_gpuvm *gpuvm, | ||
| 1121 | struct drm_exec *exec, | ||
| 1122 | unsigned int num_fences) | ||
| 1123 | { | ||
| 1124 | struct drm_gpuvm_bo *vm_bo; | ||
| 1125 | LIST_HEAD(extobjs); | ||
| 1126 | int ret = 0; | ||
| 1127 | |||
| 1128 | for_each_vm_bo_in_list(gpuvm, extobj, &extobjs, vm_bo) { | ||
| 1129 | ret = exec_prepare_obj(exec, vm_bo->obj, num_fences); | ||
| 1130 | if (ret) | ||
| 1131 | break; | ||
| 1132 | } | ||
| 1133 | /* Drop ref in case we break out of the loop. */ | ||
| 1134 | drm_gpuvm_bo_put(vm_bo); | ||
| 1135 | restore_vm_bo_list(gpuvm, extobj); | ||
| 1136 | |||
| 1137 | return ret; | ||
| 717 | } | 1138 | } |
| 718 | EXPORT_SYMBOL_GPL(drm_gpuvm_destroy); | 1139 | |
| 1140 | static int | ||
| 1141 | drm_gpuvm_prepare_objects_locked(struct drm_gpuvm *gpuvm, | ||
| 1142 | struct drm_exec *exec, | ||
| 1143 | unsigned int num_fences) | ||
| 1144 | { | ||
| 1145 | struct drm_gpuvm_bo *vm_bo; | ||
| 1146 | int ret = 0; | ||
| 1147 | |||
| 1148 | drm_gpuvm_resv_assert_held(gpuvm); | ||
| 1149 | list_for_each_entry(vm_bo, &gpuvm->extobj.list, list.entry.extobj) { | ||
| 1150 | ret = exec_prepare_obj(exec, vm_bo->obj, num_fences); | ||
| 1151 | if (ret) | ||
| 1152 | break; | ||
| 1153 | |||
| 1154 | if (vm_bo->evicted) | ||
| 1155 | drm_gpuvm_bo_list_add(vm_bo, evict, false); | ||
| 1156 | } | ||
| 1157 | |||
| 1158 | return ret; | ||
| 1159 | } | ||
| 1160 | |||
| 1161 | /** | ||
| 1162 | * drm_gpuvm_prepare_objects() - prepare all assoiciated BOs | ||
| 1163 | * @gpuvm: the &drm_gpuvm | ||
| 1164 | * @exec: the &drm_exec locking context | ||
| 1165 | * @num_fences: the amount of &dma_fences to reserve | ||
| 1166 | * | ||
| 1167 | * Calls drm_exec_prepare_obj() for all &drm_gem_objects the given | ||
| 1168 | * &drm_gpuvm contains mappings of; if @num_fences is zero drm_exec_lock_obj() | ||
| 1169 | * is called instead. | ||
| 1170 | * | ||
| 1171 | * Using this function directly, it is the drivers responsibility to call | ||
| 1172 | * drm_exec_init() and drm_exec_fini() accordingly. | ||
| 1173 | * | ||
| 1174 | * Note: This function is safe against concurrent insertion and removal of | ||
| 1175 | * external objects, however it is not safe against concurrent usage itself. | ||
| 1176 | * | ||
| 1177 | * Drivers need to make sure to protect this case with either an outer VM lock | ||
| 1178 | * or by calling drm_gpuvm_prepare_vm() before this function within the | ||
| 1179 | * drm_exec_until_all_locked() loop, such that the GPUVM's dma-resv lock ensures | ||
| 1180 | * mutual exclusion. | ||
| 1181 | * | ||
| 1182 | * Returns: 0 on success, negative error code on failure. | ||
| 1183 | */ | ||
| 1184 | int | ||
| 1185 | drm_gpuvm_prepare_objects(struct drm_gpuvm *gpuvm, | ||
| 1186 | struct drm_exec *exec, | ||
| 1187 | unsigned int num_fences) | ||
| 1188 | { | ||
| 1189 | if (drm_gpuvm_resv_protected(gpuvm)) | ||
| 1190 | return drm_gpuvm_prepare_objects_locked(gpuvm, exec, | ||
| 1191 | num_fences); | ||
| 1192 | else | ||
| 1193 | return __drm_gpuvm_prepare_objects(gpuvm, exec, num_fences); | ||
| 1194 | } | ||
| 1195 | EXPORT_SYMBOL_GPL(drm_gpuvm_prepare_objects); | ||
| 1196 | |||
| 1197 | /** | ||
| 1198 | * drm_gpuvm_prepare_range() - prepare all BOs mapped within a given range | ||
| 1199 | * @gpuvm: the &drm_gpuvm | ||
| 1200 | * @exec: the &drm_exec locking context | ||
| 1201 | * @addr: the start address within the VA space | ||
| 1202 | * @range: the range to iterate within the VA space | ||
| 1203 | * @num_fences: the amount of &dma_fences to reserve | ||
| 1204 | * | ||
| 1205 | * Calls drm_exec_prepare_obj() for all &drm_gem_objects mapped between @addr | ||
| 1206 | * and @addr + @range; if @num_fences is zero drm_exec_lock_obj() is called | ||
| 1207 | * instead. | ||
| 1208 | * | ||
| 1209 | * Returns: 0 on success, negative error code on failure. | ||
| 1210 | */ | ||
| 1211 | int | ||
| 1212 | drm_gpuvm_prepare_range(struct drm_gpuvm *gpuvm, struct drm_exec *exec, | ||
| 1213 | u64 addr, u64 range, unsigned int num_fences) | ||
| 1214 | { | ||
| 1215 | struct drm_gpuva *va; | ||
| 1216 | u64 end = addr + range; | ||
| 1217 | int ret; | ||
| 1218 | |||
| 1219 | drm_gpuvm_for_each_va_range(va, gpuvm, addr, end) { | ||
| 1220 | struct drm_gem_object *obj = va->gem.obj; | ||
| 1221 | |||
| 1222 | ret = exec_prepare_obj(exec, obj, num_fences); | ||
| 1223 | if (ret) | ||
| 1224 | return ret; | ||
| 1225 | } | ||
| 1226 | |||
| 1227 | return 0; | ||
| 1228 | } | ||
| 1229 | EXPORT_SYMBOL_GPL(drm_gpuvm_prepare_range); | ||
| 1230 | |||
| 1231 | /** | ||
| 1232 | * drm_gpuvm_exec_lock() - lock all dma-resv of all assoiciated BOs | ||
| 1233 | * @vm_exec: the &drm_gpuvm_exec wrapper | ||
| 1234 | * | ||
| 1235 | * Acquires all dma-resv locks of all &drm_gem_objects the given | ||
| 1236 | * &drm_gpuvm contains mappings of. | ||
| 1237 | * | ||
| 1238 | * Addionally, when calling this function with struct drm_gpuvm_exec::extra | ||
| 1239 | * being set the driver receives the given @fn callback to lock additional | ||
| 1240 | * dma-resv in the context of the &drm_gpuvm_exec instance. Typically, drivers | ||
| 1241 | * would call drm_exec_prepare_obj() from within this callback. | ||
| 1242 | * | ||
| 1243 | * Returns: 0 on success, negative error code on failure. | ||
| 1244 | */ | ||
| 1245 | int | ||
| 1246 | drm_gpuvm_exec_lock(struct drm_gpuvm_exec *vm_exec) | ||
| 1247 | { | ||
| 1248 | struct drm_gpuvm *gpuvm = vm_exec->vm; | ||
| 1249 | struct drm_exec *exec = &vm_exec->exec; | ||
| 1250 | unsigned int num_fences = vm_exec->num_fences; | ||
| 1251 | int ret; | ||
| 1252 | |||
| 1253 | drm_exec_init(exec, vm_exec->flags); | ||
| 1254 | |||
| 1255 | drm_exec_until_all_locked(exec) { | ||
| 1256 | ret = drm_gpuvm_prepare_vm(gpuvm, exec, num_fences); | ||
| 1257 | drm_exec_retry_on_contention(exec); | ||
| 1258 | if (ret) | ||
| 1259 | goto err; | ||
| 1260 | |||
| 1261 | ret = drm_gpuvm_prepare_objects(gpuvm, exec, num_fences); | ||
| 1262 | drm_exec_retry_on_contention(exec); | ||
| 1263 | if (ret) | ||
| 1264 | goto err; | ||
| 1265 | |||
| 1266 | if (vm_exec->extra.fn) { | ||
| 1267 | ret = vm_exec->extra.fn(vm_exec); | ||
| 1268 | drm_exec_retry_on_contention(exec); | ||
| 1269 | if (ret) | ||
| 1270 | goto err; | ||
| 1271 | } | ||
| 1272 | } | ||
| 1273 | |||
| 1274 | return 0; | ||
| 1275 | |||
| 1276 | err: | ||
| 1277 | drm_exec_fini(exec); | ||
| 1278 | return ret; | ||
| 1279 | } | ||
| 1280 | EXPORT_SYMBOL_GPL(drm_gpuvm_exec_lock); | ||
| 1281 | |||
| 1282 | static int | ||
| 1283 | fn_lock_array(struct drm_gpuvm_exec *vm_exec) | ||
| 1284 | { | ||
| 1285 | struct { | ||
| 1286 | struct drm_gem_object **objs; | ||
| 1287 | unsigned int num_objs; | ||
| 1288 | } *args = vm_exec->extra.priv; | ||
| 1289 | |||
| 1290 | return drm_exec_prepare_array(&vm_exec->exec, args->objs, | ||
| 1291 | args->num_objs, vm_exec->num_fences); | ||
| 1292 | } | ||
| 1293 | |||
| 1294 | /** | ||
| 1295 | * drm_gpuvm_exec_lock_array() - lock all dma-resv of all assoiciated BOs | ||
| 1296 | * @vm_exec: the &drm_gpuvm_exec wrapper | ||
| 1297 | * @objs: additional &drm_gem_objects to lock | ||
| 1298 | * @num_objs: the number of additional &drm_gem_objects to lock | ||
| 1299 | * | ||
| 1300 | * Acquires all dma-resv locks of all &drm_gem_objects the given &drm_gpuvm | ||
| 1301 | * contains mappings of, plus the ones given through @objs. | ||
| 1302 | * | ||
| 1303 | * Returns: 0 on success, negative error code on failure. | ||
| 1304 | */ | ||
| 1305 | int | ||
| 1306 | drm_gpuvm_exec_lock_array(struct drm_gpuvm_exec *vm_exec, | ||
| 1307 | struct drm_gem_object **objs, | ||
| 1308 | unsigned int num_objs) | ||
| 1309 | { | ||
| 1310 | struct { | ||
| 1311 | struct drm_gem_object **objs; | ||
| 1312 | unsigned int num_objs; | ||
| 1313 | } args; | ||
| 1314 | |||
| 1315 | args.objs = objs; | ||
| 1316 | args.num_objs = num_objs; | ||
| 1317 | |||
| 1318 | vm_exec->extra.fn = fn_lock_array; | ||
| 1319 | vm_exec->extra.priv = &args; | ||
| 1320 | |||
| 1321 | return drm_gpuvm_exec_lock(vm_exec); | ||
| 1322 | } | ||
| 1323 | EXPORT_SYMBOL_GPL(drm_gpuvm_exec_lock_array); | ||
| 1324 | |||
| 1325 | /** | ||
| 1326 | * drm_gpuvm_exec_lock_range() - prepare all BOs mapped within a given range | ||
| 1327 | * @vm_exec: the &drm_gpuvm_exec wrapper | ||
| 1328 | * @addr: the start address within the VA space | ||
| 1329 | * @range: the range to iterate within the VA space | ||
| 1330 | * | ||
| 1331 | * Acquires all dma-resv locks of all &drm_gem_objects mapped between @addr and | ||
| 1332 | * @addr + @range. | ||
| 1333 | * | ||
| 1334 | * Returns: 0 on success, negative error code on failure. | ||
| 1335 | */ | ||
| 1336 | int | ||
| 1337 | drm_gpuvm_exec_lock_range(struct drm_gpuvm_exec *vm_exec, | ||
| 1338 | u64 addr, u64 range) | ||
| 1339 | { | ||
| 1340 | struct drm_gpuvm *gpuvm = vm_exec->vm; | ||
| 1341 | struct drm_exec *exec = &vm_exec->exec; | ||
| 1342 | int ret; | ||
| 1343 | |||
| 1344 | drm_exec_init(exec, vm_exec->flags); | ||
| 1345 | |||
| 1346 | drm_exec_until_all_locked(exec) { | ||
| 1347 | ret = drm_gpuvm_prepare_range(gpuvm, exec, addr, range, | ||
| 1348 | vm_exec->num_fences); | ||
| 1349 | drm_exec_retry_on_contention(exec); | ||
| 1350 | if (ret) | ||
| 1351 | goto err; | ||
| 1352 | } | ||
| 1353 | |||
| 1354 | return ret; | ||
| 1355 | |||
| 1356 | err: | ||
| 1357 | drm_exec_fini(exec); | ||
| 1358 | return ret; | ||
| 1359 | } | ||
| 1360 | EXPORT_SYMBOL_GPL(drm_gpuvm_exec_lock_range); | ||
| 1361 | |||
| 1362 | static int | ||
| 1363 | __drm_gpuvm_validate(struct drm_gpuvm *gpuvm, struct drm_exec *exec) | ||
| 1364 | { | ||
| 1365 | const struct drm_gpuvm_ops *ops = gpuvm->ops; | ||
| 1366 | struct drm_gpuvm_bo *vm_bo; | ||
| 1367 | LIST_HEAD(evict); | ||
| 1368 | int ret = 0; | ||
| 1369 | |||
| 1370 | for_each_vm_bo_in_list(gpuvm, evict, &evict, vm_bo) { | ||
| 1371 | ret = ops->vm_bo_validate(vm_bo, exec); | ||
| 1372 | if (ret) | ||
| 1373 | break; | ||
| 1374 | } | ||
| 1375 | /* Drop ref in case we break out of the loop. */ | ||
| 1376 | drm_gpuvm_bo_put(vm_bo); | ||
| 1377 | restore_vm_bo_list(gpuvm, evict); | ||
| 1378 | |||
| 1379 | return ret; | ||
| 1380 | } | ||
| 1381 | |||
| 1382 | static int | ||
| 1383 | drm_gpuvm_validate_locked(struct drm_gpuvm *gpuvm, struct drm_exec *exec) | ||
| 1384 | { | ||
| 1385 | const struct drm_gpuvm_ops *ops = gpuvm->ops; | ||
| 1386 | struct drm_gpuvm_bo *vm_bo, *next; | ||
| 1387 | int ret = 0; | ||
| 1388 | |||
| 1389 | drm_gpuvm_resv_assert_held(gpuvm); | ||
| 1390 | |||
| 1391 | list_for_each_entry_safe(vm_bo, next, &gpuvm->evict.list, | ||
| 1392 | list.entry.evict) { | ||
| 1393 | ret = ops->vm_bo_validate(vm_bo, exec); | ||
| 1394 | if (ret) | ||
| 1395 | break; | ||
| 1396 | |||
| 1397 | dma_resv_assert_held(vm_bo->obj->resv); | ||
| 1398 | if (!vm_bo->evicted) | ||
| 1399 | drm_gpuvm_bo_list_del_init(vm_bo, evict, false); | ||
| 1400 | } | ||
| 1401 | |||
| 1402 | return ret; | ||
| 1403 | } | ||
| 1404 | |||
| 1405 | /** | ||
| 1406 | * drm_gpuvm_validate() - validate all BOs marked as evicted | ||
| 1407 | * @gpuvm: the &drm_gpuvm to validate evicted BOs | ||
| 1408 | * @exec: the &drm_exec instance used for locking the GPUVM | ||
| 1409 | * | ||
| 1410 | * Calls the &drm_gpuvm_ops::vm_bo_validate callback for all evicted buffer | ||
| 1411 | * objects being mapped in the given &drm_gpuvm. | ||
| 1412 | * | ||
| 1413 | * Returns: 0 on success, negative error code on failure. | ||
| 1414 | */ | ||
| 1415 | int | ||
| 1416 | drm_gpuvm_validate(struct drm_gpuvm *gpuvm, struct drm_exec *exec) | ||
| 1417 | { | ||
| 1418 | const struct drm_gpuvm_ops *ops = gpuvm->ops; | ||
| 1419 | |||
| 1420 | if (unlikely(!ops || !ops->vm_bo_validate)) | ||
| 1421 | return -EOPNOTSUPP; | ||
| 1422 | |||
| 1423 | if (drm_gpuvm_resv_protected(gpuvm)) | ||
| 1424 | return drm_gpuvm_validate_locked(gpuvm, exec); | ||
| 1425 | else | ||
| 1426 | return __drm_gpuvm_validate(gpuvm, exec); | ||
| 1427 | } | ||
| 1428 | EXPORT_SYMBOL_GPL(drm_gpuvm_validate); | ||
| 1429 | |||
| 1430 | /** | ||
| 1431 | * drm_gpuvm_resv_add_fence - add fence to private and all extobj | ||
| 1432 | * dma-resv | ||
| 1433 | * @gpuvm: the &drm_gpuvm to add a fence to | ||
| 1434 | * @exec: the &drm_exec locking context | ||
| 1435 | * @fence: fence to add | ||
| 1436 | * @private_usage: private dma-resv usage | ||
| 1437 | * @extobj_usage: extobj dma-resv usage | ||
| 1438 | */ | ||
| 1439 | void | ||
| 1440 | drm_gpuvm_resv_add_fence(struct drm_gpuvm *gpuvm, | ||
| 1441 | struct drm_exec *exec, | ||
| 1442 | struct dma_fence *fence, | ||
| 1443 | enum dma_resv_usage private_usage, | ||
| 1444 | enum dma_resv_usage extobj_usage) | ||
| 1445 | { | ||
| 1446 | struct drm_gem_object *obj; | ||
| 1447 | unsigned long index; | ||
| 1448 | |||
| 1449 | drm_exec_for_each_locked_object(exec, index, obj) { | ||
| 1450 | dma_resv_assert_held(obj->resv); | ||
| 1451 | dma_resv_add_fence(obj->resv, fence, | ||
| 1452 | drm_gpuvm_is_extobj(gpuvm, obj) ? | ||
| 1453 | extobj_usage : private_usage); | ||
| 1454 | } | ||
| 1455 | } | ||
| 1456 | EXPORT_SYMBOL_GPL(drm_gpuvm_resv_add_fence); | ||
| 1457 | |||
| 1458 | /** | ||
| 1459 | * drm_gpuvm_bo_create() - create a new instance of struct drm_gpuvm_bo | ||
| 1460 | * @gpuvm: The &drm_gpuvm the @obj is mapped in. | ||
| 1461 | * @obj: The &drm_gem_object being mapped in the @gpuvm. | ||
| 1462 | * | ||
| 1463 | * If provided by the driver, this function uses the &drm_gpuvm_ops | ||
| 1464 | * vm_bo_alloc() callback to allocate. | ||
| 1465 | * | ||
| 1466 | * Returns: a pointer to the &drm_gpuvm_bo on success, NULL on failure | ||
| 1467 | */ | ||
| 1468 | struct drm_gpuvm_bo * | ||
| 1469 | drm_gpuvm_bo_create(struct drm_gpuvm *gpuvm, | ||
| 1470 | struct drm_gem_object *obj) | ||
| 1471 | { | ||
| 1472 | const struct drm_gpuvm_ops *ops = gpuvm->ops; | ||
| 1473 | struct drm_gpuvm_bo *vm_bo; | ||
| 1474 | |||
| 1475 | if (ops && ops->vm_bo_alloc) | ||
| 1476 | vm_bo = ops->vm_bo_alloc(); | ||
| 1477 | else | ||
| 1478 | vm_bo = kzalloc(sizeof(*vm_bo), GFP_KERNEL); | ||
| 1479 | |||
| 1480 | if (unlikely(!vm_bo)) | ||
| 1481 | return NULL; | ||
| 1482 | |||
| 1483 | vm_bo->vm = drm_gpuvm_get(gpuvm); | ||
| 1484 | vm_bo->obj = obj; | ||
| 1485 | drm_gem_object_get(obj); | ||
| 1486 | |||
| 1487 | kref_init(&vm_bo->kref); | ||
| 1488 | INIT_LIST_HEAD(&vm_bo->list.gpuva); | ||
| 1489 | INIT_LIST_HEAD(&vm_bo->list.entry.gem); | ||
| 1490 | |||
| 1491 | INIT_LIST_HEAD(&vm_bo->list.entry.extobj); | ||
| 1492 | INIT_LIST_HEAD(&vm_bo->list.entry.evict); | ||
| 1493 | |||
| 1494 | return vm_bo; | ||
| 1495 | } | ||
| 1496 | EXPORT_SYMBOL_GPL(drm_gpuvm_bo_create); | ||
| 1497 | |||
| 1498 | static void | ||
| 1499 | drm_gpuvm_bo_destroy(struct kref *kref) | ||
| 1500 | { | ||
| 1501 | struct drm_gpuvm_bo *vm_bo = container_of(kref, struct drm_gpuvm_bo, | ||
| 1502 | kref); | ||
| 1503 | struct drm_gpuvm *gpuvm = vm_bo->vm; | ||
| 1504 | const struct drm_gpuvm_ops *ops = gpuvm->ops; | ||
| 1505 | struct drm_gem_object *obj = vm_bo->obj; | ||
| 1506 | bool lock = !drm_gpuvm_resv_protected(gpuvm); | ||
| 1507 | |||
| 1508 | if (!lock) | ||
| 1509 | drm_gpuvm_resv_assert_held(gpuvm); | ||
| 1510 | |||
| 1511 | drm_gpuvm_bo_list_del(vm_bo, extobj, lock); | ||
| 1512 | drm_gpuvm_bo_list_del(vm_bo, evict, lock); | ||
| 1513 | |||
| 1514 | drm_gem_gpuva_assert_lock_held(obj); | ||
| 1515 | list_del(&vm_bo->list.entry.gem); | ||
| 1516 | |||
| 1517 | if (ops && ops->vm_bo_free) | ||
| 1518 | ops->vm_bo_free(vm_bo); | ||
| 1519 | else | ||
| 1520 | kfree(vm_bo); | ||
| 1521 | |||
| 1522 | drm_gpuvm_put(gpuvm); | ||
| 1523 | drm_gem_object_put(obj); | ||
| 1524 | } | ||
| 1525 | |||
| 1526 | /** | ||
| 1527 | * drm_gpuvm_bo_put() - drop a struct drm_gpuvm_bo reference | ||
| 1528 | * @vm_bo: the &drm_gpuvm_bo to release the reference of | ||
| 1529 | * | ||
| 1530 | * This releases a reference to @vm_bo. | ||
| 1531 | * | ||
| 1532 | * If the reference count drops to zero, the &gpuvm_bo is destroyed, which | ||
| 1533 | * includes removing it from the GEMs gpuva list. Hence, if a call to this | ||
| 1534 | * function can potentially let the reference count drop to zero the caller must | ||
| 1535 | * hold the dma-resv or driver specific GEM gpuva lock. | ||
| 1536 | * | ||
| 1537 | * This function may only be called from non-atomic context. | ||
| 1538 | * | ||
| 1539 | * Returns: true if vm_bo was destroyed, false otherwise. | ||
| 1540 | */ | ||
| 1541 | bool | ||
| 1542 | drm_gpuvm_bo_put(struct drm_gpuvm_bo *vm_bo) | ||
| 1543 | { | ||
| 1544 | might_sleep(); | ||
| 1545 | |||
| 1546 | if (vm_bo) | ||
| 1547 | return !!kref_put(&vm_bo->kref, drm_gpuvm_bo_destroy); | ||
| 1548 | |||
| 1549 | return false; | ||
| 1550 | } | ||
| 1551 | EXPORT_SYMBOL_GPL(drm_gpuvm_bo_put); | ||
| 1552 | |||
| 1553 | static struct drm_gpuvm_bo * | ||
| 1554 | __drm_gpuvm_bo_find(struct drm_gpuvm *gpuvm, | ||
| 1555 | struct drm_gem_object *obj) | ||
| 1556 | { | ||
| 1557 | struct drm_gpuvm_bo *vm_bo; | ||
| 1558 | |||
| 1559 | drm_gem_gpuva_assert_lock_held(obj); | ||
| 1560 | drm_gem_for_each_gpuvm_bo(vm_bo, obj) | ||
| 1561 | if (vm_bo->vm == gpuvm) | ||
| 1562 | return vm_bo; | ||
| 1563 | |||
| 1564 | return NULL; | ||
| 1565 | } | ||
| 1566 | |||
| 1567 | /** | ||
| 1568 | * drm_gpuvm_bo_find() - find the &drm_gpuvm_bo for the given | ||
| 1569 | * &drm_gpuvm and &drm_gem_object | ||
| 1570 | * @gpuvm: The &drm_gpuvm the @obj is mapped in. | ||
| 1571 | * @obj: The &drm_gem_object being mapped in the @gpuvm. | ||
| 1572 | * | ||
| 1573 | * Find the &drm_gpuvm_bo representing the combination of the given | ||
| 1574 | * &drm_gpuvm and &drm_gem_object. If found, increases the reference | ||
| 1575 | * count of the &drm_gpuvm_bo accordingly. | ||
| 1576 | * | ||
| 1577 | * Returns: a pointer to the &drm_gpuvm_bo on success, NULL on failure | ||
| 1578 | */ | ||
| 1579 | struct drm_gpuvm_bo * | ||
| 1580 | drm_gpuvm_bo_find(struct drm_gpuvm *gpuvm, | ||
| 1581 | struct drm_gem_object *obj) | ||
| 1582 | { | ||
| 1583 | struct drm_gpuvm_bo *vm_bo = __drm_gpuvm_bo_find(gpuvm, obj); | ||
| 1584 | |||
| 1585 | return vm_bo ? drm_gpuvm_bo_get(vm_bo) : NULL; | ||
| 1586 | } | ||
| 1587 | EXPORT_SYMBOL_GPL(drm_gpuvm_bo_find); | ||
| 1588 | |||
| 1589 | /** | ||
| 1590 | * drm_gpuvm_bo_obtain() - obtains and instance of the &drm_gpuvm_bo for the | ||
| 1591 | * given &drm_gpuvm and &drm_gem_object | ||
| 1592 | * @gpuvm: The &drm_gpuvm the @obj is mapped in. | ||
| 1593 | * @obj: The &drm_gem_object being mapped in the @gpuvm. | ||
| 1594 | * | ||
| 1595 | * Find the &drm_gpuvm_bo representing the combination of the given | ||
| 1596 | * &drm_gpuvm and &drm_gem_object. If found, increases the reference | ||
| 1597 | * count of the &drm_gpuvm_bo accordingly. If not found, allocates a new | ||
| 1598 | * &drm_gpuvm_bo. | ||
| 1599 | * | ||
| 1600 | * A new &drm_gpuvm_bo is added to the GEMs gpuva list. | ||
| 1601 | * | ||
| 1602 | * Returns: a pointer to the &drm_gpuvm_bo on success, an ERR_PTR on failure | ||
| 1603 | */ | ||
| 1604 | struct drm_gpuvm_bo * | ||
| 1605 | drm_gpuvm_bo_obtain(struct drm_gpuvm *gpuvm, | ||
| 1606 | struct drm_gem_object *obj) | ||
| 1607 | { | ||
| 1608 | struct drm_gpuvm_bo *vm_bo; | ||
| 1609 | |||
| 1610 | vm_bo = drm_gpuvm_bo_find(gpuvm, obj); | ||
| 1611 | if (vm_bo) | ||
| 1612 | return vm_bo; | ||
| 1613 | |||
| 1614 | vm_bo = drm_gpuvm_bo_create(gpuvm, obj); | ||
| 1615 | if (!vm_bo) | ||
| 1616 | return ERR_PTR(-ENOMEM); | ||
| 1617 | |||
| 1618 | drm_gem_gpuva_assert_lock_held(obj); | ||
| 1619 | list_add_tail(&vm_bo->list.entry.gem, &obj->gpuva.list); | ||
| 1620 | |||
| 1621 | return vm_bo; | ||
| 1622 | } | ||
| 1623 | EXPORT_SYMBOL_GPL(drm_gpuvm_bo_obtain); | ||
| 1624 | |||
| 1625 | /** | ||
| 1626 | * drm_gpuvm_bo_obtain_prealloc() - obtains and instance of the &drm_gpuvm_bo | ||
| 1627 | * for the given &drm_gpuvm and &drm_gem_object | ||
| 1628 | * @__vm_bo: A pre-allocated struct drm_gpuvm_bo. | ||
| 1629 | * | ||
| 1630 | * Find the &drm_gpuvm_bo representing the combination of the given | ||
| 1631 | * &drm_gpuvm and &drm_gem_object. If found, increases the reference | ||
| 1632 | * count of the found &drm_gpuvm_bo accordingly, while the @__vm_bo reference | ||
| 1633 | * count is decreased. If not found @__vm_bo is returned without further | ||
| 1634 | * increase of the reference count. | ||
| 1635 | * | ||
| 1636 | * A new &drm_gpuvm_bo is added to the GEMs gpuva list. | ||
| 1637 | * | ||
| 1638 | * Returns: a pointer to the found &drm_gpuvm_bo or @__vm_bo if no existing | ||
| 1639 | * &drm_gpuvm_bo was found | ||
| 1640 | */ | ||
| 1641 | struct drm_gpuvm_bo * | ||
| 1642 | drm_gpuvm_bo_obtain_prealloc(struct drm_gpuvm_bo *__vm_bo) | ||
| 1643 | { | ||
| 1644 | struct drm_gpuvm *gpuvm = __vm_bo->vm; | ||
| 1645 | struct drm_gem_object *obj = __vm_bo->obj; | ||
| 1646 | struct drm_gpuvm_bo *vm_bo; | ||
| 1647 | |||
| 1648 | vm_bo = drm_gpuvm_bo_find(gpuvm, obj); | ||
| 1649 | if (vm_bo) { | ||
| 1650 | drm_gpuvm_bo_put(__vm_bo); | ||
| 1651 | return vm_bo; | ||
| 1652 | } | ||
| 1653 | |||
| 1654 | drm_gem_gpuva_assert_lock_held(obj); | ||
| 1655 | list_add_tail(&__vm_bo->list.entry.gem, &obj->gpuva.list); | ||
| 1656 | |||
| 1657 | return __vm_bo; | ||
| 1658 | } | ||
| 1659 | EXPORT_SYMBOL_GPL(drm_gpuvm_bo_obtain_prealloc); | ||
| 1660 | |||
| 1661 | /** | ||
| 1662 | * drm_gpuvm_bo_extobj_add() - adds the &drm_gpuvm_bo to its &drm_gpuvm's | ||
| 1663 | * extobj list | ||
| 1664 | * @vm_bo: The &drm_gpuvm_bo to add to its &drm_gpuvm's the extobj list. | ||
| 1665 | * | ||
| 1666 | * Adds the given @vm_bo to its &drm_gpuvm's extobj list if not on the list | ||
| 1667 | * already and if the corresponding &drm_gem_object is an external object, | ||
| 1668 | * actually. | ||
| 1669 | */ | ||
| 1670 | void | ||
| 1671 | drm_gpuvm_bo_extobj_add(struct drm_gpuvm_bo *vm_bo) | ||
| 1672 | { | ||
| 1673 | struct drm_gpuvm *gpuvm = vm_bo->vm; | ||
| 1674 | bool lock = !drm_gpuvm_resv_protected(gpuvm); | ||
| 1675 | |||
| 1676 | if (!lock) | ||
| 1677 | drm_gpuvm_resv_assert_held(gpuvm); | ||
| 1678 | |||
| 1679 | if (drm_gpuvm_is_extobj(gpuvm, vm_bo->obj)) | ||
| 1680 | drm_gpuvm_bo_list_add(vm_bo, extobj, lock); | ||
| 1681 | } | ||
| 1682 | EXPORT_SYMBOL_GPL(drm_gpuvm_bo_extobj_add); | ||
| 1683 | |||
| 1684 | /** | ||
| 1685 | * drm_gpuvm_bo_evict() - add / remove a &drm_gpuvm_bo to / from the &drm_gpuvms | ||
| 1686 | * evicted list | ||
| 1687 | * @vm_bo: the &drm_gpuvm_bo to add or remove | ||
| 1688 | * @evict: indicates whether the object is evicted | ||
| 1689 | * | ||
| 1690 | * Adds a &drm_gpuvm_bo to or removes it from the &drm_gpuvms evicted list. | ||
| 1691 | */ | ||
| 1692 | void | ||
| 1693 | drm_gpuvm_bo_evict(struct drm_gpuvm_bo *vm_bo, bool evict) | ||
| 1694 | { | ||
| 1695 | struct drm_gpuvm *gpuvm = vm_bo->vm; | ||
| 1696 | struct drm_gem_object *obj = vm_bo->obj; | ||
| 1697 | bool lock = !drm_gpuvm_resv_protected(gpuvm); | ||
| 1698 | |||
| 1699 | dma_resv_assert_held(obj->resv); | ||
| 1700 | vm_bo->evicted = evict; | ||
| 1701 | |||
| 1702 | /* Can't add external objects to the evicted list directly if not using | ||
| 1703 | * internal spinlocks, since in this case the evicted list is protected | ||
| 1704 | * with the VM's common dma-resv lock. | ||
| 1705 | */ | ||
| 1706 | if (drm_gpuvm_is_extobj(gpuvm, obj) && !lock) | ||
| 1707 | return; | ||
| 1708 | |||
| 1709 | if (evict) | ||
| 1710 | drm_gpuvm_bo_list_add(vm_bo, evict, lock); | ||
| 1711 | else | ||
| 1712 | drm_gpuvm_bo_list_del_init(vm_bo, evict, lock); | ||
| 1713 | } | ||
| 1714 | EXPORT_SYMBOL_GPL(drm_gpuvm_bo_evict); | ||
| 719 | 1715 | ||
| 720 | static int | 1716 | static int |
| 721 | __drm_gpuva_insert(struct drm_gpuvm *gpuvm, | 1717 | __drm_gpuva_insert(struct drm_gpuvm *gpuvm, |
| @@ -764,11 +1760,21 @@ drm_gpuva_insert(struct drm_gpuvm *gpuvm, · 42 unmodified lines | |||
| 764 | { | 1760 | { |
| 765 | u64 addr = va->va.addr; | 1761 | u64 addr = va->va.addr; |
| 766 | u64 range = va->va.range; | 1762 | u64 range = va->va.range; |
| 1763 | int ret; | ||
| 767 | 1764 | ||
| 768 | if (unlikely(!drm_gpuvm_range_valid(gpuvm, addr, range))) | 1765 | if (unlikely(!drm_gpuvm_range_valid(gpuvm, addr, range))) |
| 769 | return -EINVAL; | 1766 | return -EINVAL; |
| 770 | 1767 | ||
| 771 | return __drm_gpuva_insert(gpuvm, va); | 1768 | ret = __drm_gpuva_insert(gpuvm, va); |
| 1769 | if (likely(!ret)) | ||
| 1770 | /* Take a reference of the GPUVM for the successfully inserted | ||
| 1771 | * drm_gpuva. We can't take the reference in | ||
| 1772 | * __drm_gpuva_insert() itself, since we don't want to increse | ||
| 1773 | * the reference count for the GPUVM's kernel_alloc_node. | ||
| 1774 | */ | ||
| 1775 | drm_gpuvm_get(gpuvm); | ||
| 1776 | |||
| 1777 | return ret; | ||
| 772 | } | 1778 | } |
| 773 | EXPORT_SYMBOL_GPL(drm_gpuva_insert); | 1779 | EXPORT_SYMBOL_GPL(drm_gpuva_insert); |
| 774 | 1780 | ||
| @@ -795,35 +1801,46 @@ drm_gpuva_remove(struct drm_gpuva *va) · 20 unmodified lines | |||
| 795 | struct drm_gpuvm *gpuvm = va->vm; | 1801 | struct drm_gpuvm *gpuvm = va->vm; |
| 796 | 1802 | ||
| 797 | if (unlikely(va == &gpuvm->kernel_alloc_node)) { | 1803 | if (unlikely(va == &gpuvm->kernel_alloc_node)) { |
| 798 | WARN(1, "Can't destroy kernel reserved node.\n"); | 1804 | drm_WARN(gpuvm->drm, 1, |
| 1805 | "Can't destroy kernel reserved node.\n"); | ||
| 799 | return; | 1806 | return; |
| 800 | } | 1807 | } |
| 801 | 1808 | ||
| 802 | __drm_gpuva_remove(va); | 1809 | __drm_gpuva_remove(va); |
| 1810 | drm_gpuvm_put(va->vm); | ||
| 803 | } | 1811 | } |
| 804 | EXPORT_SYMBOL_GPL(drm_gpuva_remove); | 1812 | EXPORT_SYMBOL_GPL(drm_gpuva_remove); |
| 805 | 1813 | ||
| 806 | /** | 1814 | /** |
| 807 | * drm_gpuva_link() - link a &drm_gpuva | 1815 | * drm_gpuva_link() - link a &drm_gpuva |
| 808 | * @va: the &drm_gpuva to link | 1816 | * @va: the &drm_gpuva to link |
| 1817 | * @vm_bo: the &drm_gpuvm_bo to add the &drm_gpuva to | ||
| 809 | * | 1818 | * |
| 810 | * This adds the given &va to the GPU VA list of the &drm_gem_object it is | 1819 | * This adds the given &va to the GPU VA list of the &drm_gpuvm_bo and the |
| 811 | * associated with. | 1820 | * &drm_gpuvm_bo to the &drm_gem_object it is associated with. |
| 1821 | * | ||
| 1822 | * For every &drm_gpuva entry added to the &drm_gpuvm_bo an additional | ||
| 1823 | * reference of the latter is taken. | ||
| 812 | * | 1824 | * |
| 813 | * This function expects the caller to protect the GEM's GPUVA list against | 1825 | * This function expects the caller to protect the GEM's GPUVA list against |
| 814 | * concurrent access using the GEMs dma_resv lock. | 1826 | * concurrent access using either the GEMs dma_resv lock or a driver specific |
| 1827 | * lock set through drm_gem_gpuva_set_lock(). | ||
| 815 | */ | 1828 | */ |
| 816 | void | 1829 | void |
| 817 | drm_gpuva_link(struct drm_gpuva *va) | 1830 | drm_gpuva_link(struct drm_gpuva *va, struct drm_gpuvm_bo *vm_bo) |
| 818 | { | 1831 | { |
| 819 | struct drm_gem_object *obj = va->gem.obj; | 1832 | struct drm_gem_object *obj = va->gem.obj; |
| 1833 | struct drm_gpuvm *gpuvm = va->vm; | ||
| 820 | 1834 | ||
| 821 | if (unlikely(!obj)) | 1835 | if (unlikely(!obj)) |
| 822 | return; | 1836 | return; |
| 823 | 1837 | ||
| 824 | drm_gem_gpuva_assert_lock_held(obj); | 1838 | drm_WARN_ON(gpuvm->drm, obj != vm_bo->obj); |
| 825 | 1839 | ||
| 826 | list_add_tail(&va->gem.entry, &obj->gpuva.list); | 1840 | va->vm_bo = drm_gpuvm_bo_get(vm_bo); |
| 1841 | |||
| 1842 | drm_gem_gpuva_assert_lock_held(obj); | ||
| 1843 | list_add_tail(&va->gem.entry, &vm_bo->list.gpuva); | ||
| 827 | } | 1844 | } |
| 828 | EXPORT_SYMBOL_GPL(drm_gpuva_link); | 1845 | EXPORT_SYMBOL_GPL(drm_gpuva_link); |
| 829 | 1846 | ||
| @@ -834,20 +1851,31 @@ EXPORT_SYMBOL_GPL(drm_gpuva_link); · 4 unmodified lines | |||
| 834 | * This removes the given &va from the GPU VA list of the &drm_gem_object it is | 1851 | * This removes the given &va from the GPU VA list of the &drm_gem_object it is |
| 835 | * associated with. | 1852 | * associated with. |
| 836 | * | 1853 | * |
| 1854 | * This removes the given &va from the GPU VA list of the &drm_gpuvm_bo and | ||
| 1855 | * the &drm_gpuvm_bo from the &drm_gem_object it is associated with in case | ||
| 1856 | * this call unlinks the last &drm_gpuva from the &drm_gpuvm_bo. | ||
| 1857 | * | ||
| 1858 | * For every &drm_gpuva entry removed from the &drm_gpuvm_bo a reference of | ||
| 1859 | * the latter is dropped. | ||
| 1860 | * | ||
| 837 | * This function expects the caller to protect the GEM's GPUVA list against | 1861 | * This function expects the caller to protect the GEM's GPUVA list against |
| 838 | * concurrent access using the GEMs dma_resv lock. | 1862 | * concurrent access using either the GEMs dma_resv lock or a driver specific |
| 1863 | * lock set through drm_gem_gpuva_set_lock(). | ||
| 839 | */ | 1864 | */ |
| 840 | void | 1865 | void |
| 841 | drm_gpuva_unlink(struct drm_gpuva *va) | 1866 | drm_gpuva_unlink(struct drm_gpuva *va) |
| 842 | { | 1867 | { |
| 843 | struct drm_gem_object *obj = va->gem.obj; | 1868 | struct drm_gem_object *obj = va->gem.obj; |
| 1869 | struct drm_gpuvm_bo *vm_bo = va->vm_bo; | ||
| 844 | 1870 | ||
| 845 | if (unlikely(!obj)) | 1871 | if (unlikely(!obj)) |
| 846 | return; | 1872 | return; |
| 847 | 1873 | ||
| 848 | drm_gem_gpuva_assert_lock_held(obj); | 1874 | drm_gem_gpuva_assert_lock_held(obj); |
| 849 | |||
| 850 | list_del_init(&va->gem.entry); | 1875 | list_del_init(&va->gem.entry); |
| 1876 | |||
| 1877 | va->vm_bo = NULL; | ||
| 1878 | drm_gpuvm_bo_put(vm_bo); | ||
| 851 | } | 1879 | } |
| 852 | EXPORT_SYMBOL_GPL(drm_gpuva_unlink); | 1880 | EXPORT_SYMBOL_GPL(drm_gpuva_unlink); |
| 853 | 1881 | ||
| @@ -992,10 +2020,10 @@ drm_gpuva_remap(struct drm_gpuva *prev, · 138 unmodified lines | |||
| 992 | struct drm_gpuva *next, | 2020 | struct drm_gpuva *next, |
| 993 | struct drm_gpuva_op_remap *op) | 2021 | struct drm_gpuva_op_remap *op) |
| 994 | { | 2022 | { |
| 995 | struct drm_gpuva *curr = op->unmap->va; | 2023 | struct drm_gpuva *va = op->unmap->va; |
| 996 | struct drm_gpuvm *gpuvm = curr->vm; | 2024 | struct drm_gpuvm *gpuvm = va->vm; |
| 997 | 2025 | ||
| 998 | drm_gpuva_remove(curr); | 2026 | drm_gpuva_remove(va); |
| 999 | 2027 | ||
| 1000 | if (op->prev) { | 2028 | if (op->prev) { |
| 1001 | drm_gpuva_init_from_op(prev, op->prev); | 2029 | drm_gpuva_init_from_op(prev, op->prev); |
| @@ -1637,9 +2665,8 @@ err_free_ops: · 635 unmodified lines | |||
| 1637 | EXPORT_SYMBOL_GPL(drm_gpuvm_prefetch_ops_create); | 2665 | EXPORT_SYMBOL_GPL(drm_gpuvm_prefetch_ops_create); |
| 1638 | 2666 | ||
| 1639 | /** | 2667 | /** |
| 1640 | * drm_gpuvm_gem_unmap_ops_create() - creates the &drm_gpuva_ops to unmap a GEM | 2668 | * drm_gpuvm_bo_unmap_ops_create() - creates the &drm_gpuva_ops to unmap a GEM |
| 1641 | * @gpuvm: the &drm_gpuvm representing the GPU VA space | 2669 | * @vm_bo: the &drm_gpuvm_bo abstraction |
| 1642 | * @obj: the &drm_gem_object to unmap | ||
| 1643 | * | 2670 | * |
| 1644 | * This function creates a list of operations to perform unmapping for every | 2671 | * This function creates a list of operations to perform unmapping for every |
| 1645 | * GPUVA attached to a GEM. | 2672 | * GPUVA attached to a GEM. |
| @@ -1656,15 +2683,14 @@ EXPORT_SYMBOL_GPL(drm_gpuvm_prefetch_ops_create); · 10 unmodified lines | |||
| 1656 | * Returns: a pointer to the &drm_gpuva_ops on success, an ERR_PTR on failure | 2683 | * Returns: a pointer to the &drm_gpuva_ops on success, an ERR_PTR on failure |
| 1657 | */ | 2684 | */ |
| 1658 | struct drm_gpuva_ops * | 2685 | struct drm_gpuva_ops * |
| 1659 | drm_gpuvm_gem_unmap_ops_create(struct drm_gpuvm *gpuvm, | 2686 | drm_gpuvm_bo_unmap_ops_create(struct drm_gpuvm_bo *vm_bo) |
| 1660 | struct drm_gem_object *obj) | ||
| 1661 | { | 2687 | { |
| 1662 | struct drm_gpuva_ops *ops; | 2688 | struct drm_gpuva_ops *ops; |
| 1663 | struct drm_gpuva_op *op; | 2689 | struct drm_gpuva_op *op; |
| 1664 | struct drm_gpuva *va; | 2690 | struct drm_gpuva *va; |
| 1665 | int ret; | 2691 | int ret; |
| 1666 | 2692 | ||
| 1667 | drm_gem_gpuva_assert_lock_held(obj); | 2693 | drm_gem_gpuva_assert_lock_held(vm_bo->obj); |
| 1668 | 2694 | ||
| 1669 | ops = kzalloc(sizeof(*ops), GFP_KERNEL); | 2695 | ops = kzalloc(sizeof(*ops), GFP_KERNEL); |
| 1670 | if (!ops) | 2696 | if (!ops) |
| @@ -1672,8 +2698,8 @@ drm_gpuvm_gem_unmap_ops_create(struct drm_gpuvm *gpuvm, · 1 unmodified line | |||
| 1672 | 2698 | ||
| 1673 | INIT_LIST_HEAD(&ops->list); | 2699 | INIT_LIST_HEAD(&ops->list); |
| 1674 | 2700 | ||
| 1675 | drm_gem_for_each_gpuva(va, obj) { | 2701 | drm_gpuvm_bo_for_each_va(va, vm_bo) { |
| 1676 | op = gpuva_op_alloc(gpuvm); | 2702 | op = gpuva_op_alloc(vm_bo->vm); |
| 1677 | if (!op) { | 2703 | if (!op) { |
| 1678 | ret = -ENOMEM; | 2704 | ret = -ENOMEM; |
| 1679 | goto err_free_ops; | 2705 | goto err_free_ops; |
| @@ -1687,10 +2713,10 @@ drm_gpuvm_gem_unmap_ops_create(struct drm_gpuvm *gpuvm, · 7 unmodified lines | |||
| 1687 | return ops; | 2713 | return ops; |
| 1688 | 2714 | ||
| 1689 | err_free_ops: | 2715 | err_free_ops: |
| 1690 | drm_gpuva_ops_free(gpuvm, ops); | 2716 | drm_gpuva_ops_free(vm_bo->vm, ops); |
| 1691 | return ERR_PTR(ret); | 2717 | return ERR_PTR(ret); |
| 1692 | } | 2718 | } |
| 1693 | EXPORT_SYMBOL_GPL(drm_gpuvm_gem_unmap_ops_create); | 2719 | EXPORT_SYMBOL_GPL(drm_gpuvm_bo_unmap_ops_create); |
| 1694 | 2720 | ||
| 1695 | /** | 2721 | /** |
| 1696 | * drm_gpuva_ops_free() - free the given &drm_gpuva_ops | 2722 | * drm_gpuva_ops_free() - free the given &drm_gpuva_ops |
| @@ -1,203 +0,0 @@ | |||
| 1 | /************************************************************************** | ||
| 2 | * | ||
| 3 | * Copyright 2006 Tungsten Graphics, Inc., Bismarck, ND. USA. | ||
| 4 | * All Rights Reserved. | ||
| 5 | * | ||
| 6 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 7 | * copy of this software and associated documentation files (the | ||
| 8 | * "Software"), to deal in the Software without restriction, including | ||
| 9 | * without limitation the rights to use, copy, modify, merge, publish, | ||
| 10 | * distribute, sub license, and/or sell copies of the Software, and to | ||
| 11 | * permit persons to whom the Software is furnished to do so, subject to | ||
| 12 | * the following conditions: | ||
| 13 | * | ||
| 14 | * The above copyright notice and this permission notice (including the | ||
| 15 | * next paragraph) shall be included in all copies or substantial portions | ||
| 16 | * of the Software. | ||
| 17 | * | ||
| 18 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 19 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 20 | * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL | ||
| 21 | * THE COPYRIGHT HOLDERS, AUTHORS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, | ||
| 22 | * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR | ||
| 23 | * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE | ||
| 24 | * USE OR OTHER DEALINGS IN THE SOFTWARE. | ||
| 25 | * | ||
| 26 | * | ||
| 27 | **************************************************************************/ | ||
| 28 | /* | ||
| 29 | * Simple open hash tab implementation. | ||
| 30 | * | ||
| 31 | * Authors: | ||
| 32 | * Thomas Hellström <thomas-at-tungstengraphics-dot-com> | ||
| 33 | */ | ||
| 34 | |||
| 35 | #include <linux/hash.h> | ||
| 36 | #include <linux/mm.h> | ||
| 37 | #include <linux/rculist.h> | ||
| 38 | #include <linux/slab.h> | ||
| 39 | #include <linux/vmalloc.h> | ||
| 40 | |||
| 41 | #include <drm/drm_print.h> | ||
| 42 | |||
| 43 | #include "drm_legacy.h" | ||
| 44 | |||
| 45 | int drm_ht_create(struct drm_open_hash *ht, unsigned int order) | ||
| 46 | { | ||
| 47 | unsigned int size = 1 << order; | ||
| 48 | |||
| 49 | ht->order = order; | ||
| 50 | ht->table = NULL; | ||
| 51 | if (size <= PAGE_SIZE / sizeof(*ht->table)) | ||
| 52 | ht->table = kcalloc(size, sizeof(*ht->table), GFP_KERNEL); | ||
| 53 | else | ||
| 54 | ht->table = vzalloc(array_size(size, sizeof(*ht->table))); | ||
| 55 | if (!ht->table) { | ||
| 56 | DRM_ERROR("Out of memory for hash table\n"); | ||
| 57 | return -ENOMEM; | ||
| 58 | } | ||
| 59 | return 0; | ||
| 60 | } | ||
| 61 | |||
| 62 | void drm_ht_verbose_list(struct drm_open_hash *ht, unsigned long key) | ||
| 63 | { | ||
| 64 | struct drm_hash_item *entry; | ||
| 65 | struct hlist_head *h_list; | ||
| 66 | unsigned int hashed_key; | ||
| 67 | int count = 0; | ||
| 68 | |||
| 69 | hashed_key = hash_long(key, ht->order); | ||
| 70 | DRM_DEBUG("Key is 0x%08lx, Hashed key is 0x%08x\n", key, hashed_key); | ||
| 71 | h_list = &ht->table[hashed_key]; | ||
| 72 | hlist_for_each_entry(entry, h_list, head) | ||
| 73 | DRM_DEBUG("count %d, key: 0x%08lx\n", count++, entry->key); | ||
| 74 | } | ||
| 75 | |||
| 76 | static struct hlist_node *drm_ht_find_key(struct drm_open_hash *ht, | ||
| 77 | unsigned long key) | ||
| 78 | { | ||
| 79 | struct drm_hash_item *entry; | ||
| 80 | struct hlist_head *h_list; | ||
| 81 | unsigned int hashed_key; | ||
| 82 | |||
| 83 | hashed_key = hash_long(key, ht->order); | ||
| 84 | h_list = &ht->table[hashed_key]; | ||
| 85 | hlist_for_each_entry(entry, h_list, head) { | ||
| 86 | if (entry->key == key) | ||
| 87 | return &entry->head; | ||
| 88 | if (entry->key > key) | ||
| 89 | break; | ||
| 90 | } | ||
| 91 | return NULL; | ||
| 92 | } | ||
| 93 | |||
| 94 | static struct hlist_node *drm_ht_find_key_rcu(struct drm_open_hash *ht, | ||
| 95 | unsigned long key) | ||
| 96 | { | ||
| 97 | struct drm_hash_item *entry; | ||
| 98 | struct hlist_head *h_list; | ||
| 99 | unsigned int hashed_key; | ||
| 100 | |||
| 101 | hashed_key = hash_long(key, ht->order); | ||
| 102 | h_list = &ht->table[hashed_key]; | ||
| 103 | hlist_for_each_entry_rcu(entry, h_list, head) { | ||
| 104 | if (entry->key == key) | ||
| 105 | return &entry->head; | ||
| 106 | if (entry->key > key) | ||
| 107 | break; | ||
| 108 | } | ||
| 109 | return NULL; | ||
| 110 | } | ||
| 111 | |||
| 112 | int drm_ht_insert_item(struct drm_open_hash *ht, struct drm_hash_item *item) | ||
| 113 | { | ||
| 114 | struct drm_hash_item *entry; | ||
| 115 | struct hlist_head *h_list; | ||
| 116 | struct hlist_node *parent; | ||
| 117 | unsigned int hashed_key; | ||
| 118 | unsigned long key = item->key; | ||
| 119 | |||
84 more lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 656 lines aren’t shown here.
This diff is large: its 165 lines aren’t shown here.
This diff is large: its 205 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 291 lines aren’t shown here.
This diff is large: its 106 lines aren’t shown here.
This diff is large: its 374 lines aren’t shown here.
This diff is large: its 139 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 220 lines aren’t shown here.
This diff is large: its 185 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 221 lines aren’t shown here.
This diff is large: its 152 lines aren’t shown here.
This diff is large: its 136 lines aren’t shown here.
This diff is large: its 666 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 299 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 147 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 126 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 229 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 361 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 960 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 147 lines aren’t shown here.
This diff is large: its 177 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 218 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 90 lines aren’t shown here.
This diff is large: its 859 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 276 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 957 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 448 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 475 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 313 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 255 lines aren’t shown here.
This diff is large: its 83 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 511 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 728 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 129 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 646 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 268 lines aren’t shown here.
This diff is large: its 111 lines aren’t shown here.
This diff is large: its 465 lines aren’t shown here.
This diff is large: its 206 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 659 lines aren’t shown here.
This diff is large: its 722 lines aren’t shown here.
This diff is large: its 256 lines aren’t shown here.
This diff is large: its 187 lines aren’t shown here.
This diff is large: its 1502 lines aren’t shown here.
This diff is large: its 130 lines aren’t shown here.
This diff is large: its 626 lines aren’t shown here.
This diff is large: its 196 lines aren’t shown here.
This diff is large: its 1490 lines aren’t shown here.
This diff is large: its 510 lines aren’t shown here.
This diff is large: its 136 lines aren’t shown here.
This diff is large: its 556 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 253 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 307 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 472 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 415 lines aren’t shown here.
This diff is large: its 171 lines aren’t shown here.
This diff is large: its 550 lines aren’t shown here.
This diff is large: its 167 lines aren’t shown here.
This diff is large: its 787 lines aren’t shown here.
This diff is large: its 162 lines aren’t shown here.
This diff is large: its 2641 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 148 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 434 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 1433 lines aren’t shown here.
This diff is large: its 170 lines aren’t shown here.
This diff is large: its 6194 lines aren’t shown here.
This diff is large: its 160 lines aren’t shown here.
This diff is large: its 180 lines aren’t shown here.
This diff is large: its 2189 lines aren’t shown here.
This diff is large: its 494 lines aren’t shown here.
This diff is large: its 374 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 1649 lines aren’t shown here.
This diff is large: its 259 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 357 lines aren’t shown here.
This diff is large: its 336 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 137 lines aren’t shown here.
This diff is large: its 286 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 352 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 290 lines aren’t shown here.
This diff is large: its 85 lines aren’t shown here.
This diff is large: its 1093 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 238 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 132 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 315 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 680 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 241 lines aren’t shown here.
This diff is large: its 107 lines aren’t shown here.
This diff is large: its 280 lines aren’t shown here.
This diff is large: its 258 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 188 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 801 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 508 lines aren’t shown here.
This diff is large: its 206 lines aren’t shown here.
This diff is large: its 265 lines aren’t shown here.
This diff is large: its 384 lines aren’t shown here.
This diff is large: its 2107 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 210 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 245 lines aren’t shown here.
This diff is large: its 115 lines aren’t shown here.
This diff is large: its 250 lines aren’t shown here.
This diff is large: its 816 lines aren’t shown here.
This diff is large: its 172 lines aren’t shown here.
This diff is large: its 235 lines aren’t shown here.
This diff is large: its 555 lines aren’t shown here.
This diff is large: its 1321 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 117 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 205 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 215 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 123 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 130 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 244 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 124 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 198 lines aren’t shown here.
This diff is large: its 132 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 135 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 103 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 115 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 4 lines aren’t shown here.
This diff is large: its 1326 lines aren’t shown here.
This diff is large: its 762 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 154 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 163 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 129 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 151 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 103 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 196 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 1254 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 153 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 130 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 251 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 102 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 167 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 133 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 214 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 202 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 357 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 180 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 174 lines aren’t shown here.
This diff is large: its 165 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 637 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 332 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 150 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 1296 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 294 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 859 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 214 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 4073 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 498 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 220 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 243 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 132 lines aren’t shown here.
This diff is large: its 148 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.