Backmerge drm-misc-next to pick up some dependencies for drm/msm patches, in particular: https://patchwork.freedesktop.org/patch/570219/?series=127251&rev=1 https://patchwork.freedesktop.org/series/123411/
Unified · Split
| @@ -36,8 +36,9 @@ AIC100 DID (0xa100). · 35 unmodified lines | ||
| 36 | 36 | |
| 37 | 37 | AIC100 does not implement FLR (function level reset). |
| 38 | 38 | |
| 39 | −AIC100 implements MSI but does not implement MSI-X. AIC100 requires 17 MSIs to | |
| 40 | −operate (1 for MHI, 16 for the DMA Bridge). | |
| 39 | +AIC100 implements MSI but does not implement MSI-X. AIC100 prefers 17 MSIs to | |
| 40 | +operate (1 for MHI, 16 for the DMA Bridge). Falling back to 1 MSI is possible in | |
| 41 | +scenarios where reserving 32 MSIs isn't feasible. | |
| 41 | 42 | |
| 42 | 43 | As a PCIe device, AIC100 utilizes BARs to provide host interfaces to the device |
| 43 | 44 | hardware. AIC100 provides 3, 64-bit BARs. |
| @@ -220,10 +221,14 @@ of the defined channels, and their uses. · 176 unmodified lines | ||
| 220 | 221 | +----------------+---------+----------+----------------------------------------+ |
| 221 | 222 | | QAIC_DEBUG | 18 & 19 | AMSS | Not used. | |
| 222 | 223 | +----------------+---------+----------+----------------------------------------+ |
| 223 | −| QAIC_TIMESYNC | 20 & 21 | SBL/AMSS | Used to synchronize timestamps in the | | |
| 224 | +| QAIC_TIMESYNC | 20 & 21 | SBL | Used to synchronize timestamps in the | | |
| 224 | 225 | | | | | device side logs with the host time | |
| 225 | 226 | | | | | source. | |
| 226 | 227 | +----------------+---------+----------+----------------------------------------+ |
| 228 | +| QAIC_TIMESYNC | 22 & 23 | AMSS | Used to periodically synchronize | | |
| 229 | +| _PERIODIC | | | timestamps in the device side logs with| | |
| 230 | +| | | | the host time source. | | |
| 231 | ++----------------+---------+----------+----------------------------------------+ | |
| 227 | 232 | |
| 228 | 233 | DMA Bridge |
| 229 | 234 | ========== |
| @@ -10,6 +10,9 @@ accelerator products. · 9 unmodified lines | ||
| 10 | 10 | Interrupts |
| 11 | 11 | ========== |
| 12 | 12 | |
| 13 | +IRQ Storm Mitigation | |
| 14 | +-------------------- | |
| 15 | + | |
| 13 | 16 | While the AIC100 DMA Bridge hardware implements an IRQ storm mitigation |
| 14 | 17 | mechanism, it is still possible for an IRQ storm to occur. A storm can happen |
| 15 | 18 | if the workload is particularly quick, and the host is responsive. If the host |
| @@ -35,6 +38,26 @@ generates 100k IRQs per second (per /proc/interrupts) is reduced to roughly 64 · 19 unmodified lines | ||
| 35 | 38 | IRQs over 5 minutes while keeping the host system stable, and having the same |
| 36 | 39 | workload throughput performance (within run to run noise variation). |
| 37 | 40 | |
| 41 | +Single MSI Mode | |
| 42 | +--------------- | |
| 43 | + | |
| 44 | +MultiMSI is not well supported on all systems; virtualized ones even less so | |
| 45 | +(circa 2023). Between hypervisors masking the PCIe MSI capability structure to | |
| 46 | +large memory requirements for vIOMMUs (required for supporting MultiMSI), it is | |
| 47 | +useful to be able to fall back to a single MSI when needed. | |
| 48 | + | |
| 49 | +To support this fallback, we allow the case where only one MSI is able to be | |
| 50 | +allocated, and share that one MSI between MHI and the DBCs. The device detects | |
| 51 | +when only one MSI has been configured and directs the interrupts for the DBCs | |
| 52 | +to the interrupt normally used for MHI. Unfortunately this means that the | |
| 53 | +interrupt handlers for every DBC and MHI wake up for every interrupt that | |
| 54 | +arrives; however, the DBC threaded irq handlers only are started when work to be | |
| 55 | +done is detected (MHI will always start its threaded handler). | |
| 56 | + | |
| 57 | +If the DBC is configured to force MSI interrupts, this can circumvent the | |
| 58 | +software IRQ storm mitigation mentioned above. Since the MSI is shared it is | |
| 59 | +never disabled, allowing each new entry to the FIFO to trigger a new interrupt. | |
| 60 | + | |
| 38 | 61 | |
| 39 | 62 | Neural Network Control (NNC) Protocol |
| 40 | 63 | ===================================== |
| @@ -70,8 +93,15 @@ commands (does not impact QAIC). · 29 unmodified lines | ||
| 70 | 93 | uAPI |
| 71 | 94 | ==== |
| 72 | 95 | |
| 96 | +QAIC creates an accel device per phsyical PCIe device. This accel device exists | |
| 97 | +for as long as the PCIe device is known to Linux. | |
| 98 | + | |
| 99 | +The PCIe device may not be in the state to accept requests from userspace at | |
| 100 | +all times. QAIC will trigger KOBJ_ONLINE/OFFLINE uevents to advertise when the | |
| 101 | +device can accept requests (ONLINE) and when the device is no longer accepting | |
| 102 | +requests (OFFLINE) because of a reset or other state transition. | |
| 103 | + | |
| 73 | 104 | QAIC defines a number of driver specific IOCTLs as part of the userspace API. |
| 74 | −This section describes those APIs. | |
| 75 | 105 | |
| 76 | 106 | DRM_IOCTL_QAIC_MANAGE |
| 77 | 107 | This IOCTL allows userspace to send a NNC request to the QSM. The call will |
| @@ -178,3 +208,8 @@ overrides this for that call. Default is 5000 (5 seconds). · 100 unmodified lines | ||
| 178 | 208 | |
| 179 | 209 | Sets the polling interval in microseconds (us) when datapath polling is active. |
| 180 | 210 | Takes effect at the next polling interval. Default is 100 (100 us). |
| 211 | + | |
| 212 | +**timesync_delay_ms (unsigned int)** | |
| 213 | + | |
| 214 | +Sets the time interval in milliseconds (ms) between two consecutive timesync | |
| 215 | +operations. Default is 1000 (1000 ms). | |
| @@ -375,9 +375,9 @@ Developer web site of Loongson and LoongArch (Software and Documentation): · 374 unmodified lines | ||
| 375 | 375 | |
| 376 | 376 | Documentation of LoongArch ISA: |
| 377 | 377 | |
| 378 | − https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.02-CN.pdf (in Chinese) | |
| 378 | + https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.10-CN.pdf (in Chinese) | |
| 379 | 379 | |
| 380 | − https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.02-EN.pdf (in English) | |
| 380 | + https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.10-EN.pdf (in English) | |
| 381 | 381 | |
| 382 | 382 | Documentation of LoongArch ELF psABI: |
| 383 | 383 | |
| @@ -77,7 +77,7 @@ Protocol 2.14 BURNT BY INCORRECT COMMIT · 76 unmodified lines | ||
| 77 | 77 | Protocol 2.15 (Kernel 5.5) Added the kernel_info and kernel_info.setup_type_max. |
| 78 | 78 | ============= ============================================================ |
| 79 | 79 | |
| 80 | −.. note:: | |
| 80 | + .. note:: | |
| 81 | 81 | The protocol version number should be changed only if the setup header |
| 82 | 82 | is changed. There is no need to update the version number if boot_params |
| 83 | 83 | or kernel_info are changed. Additionally, it is recommended to use |
| @@ -153,6 +153,8 @@ NOTE: Some pages, such as DAX pages, cannot be pinned with longterm pins. That's · 152 unmodified lines | ||
| 153 | 153 | because DAX pages do not have a separate page cache, and so "pinning" implies |
| 154 | 154 | locking down file system blocks, which is not (yet) supported in that way. |
| 155 | 155 | |
| 156 | +.. _mmu-notifier-registration-case: | |
| 157 | + | |
| 156 | 158 | CASE 3: MMU notifier registration, with or without page faulting hardware |
| 157 | 159 | ------------------------------------------------------------------------- |
| 158 | 160 | Device drivers can pin pages via get_user_pages*(), and register for mmu |
| @@ -55,6 +55,27 @@ properties: · 54 unmodified lines | ||
| 55 | 55 | - port@0 |
| 56 | 56 | - port@1 |
| 57 | 57 | |
| 58 | + vcchdmipll-supply: | |
| 59 | + description: A 1.8V supply that powers the HDMI PLL. | |
| 60 | + | |
| 61 | + vcchdmitx-supply: | |
| 62 | + description: A 1.8V supply that powers the HDMI TX part. | |
| 63 | + | |
| 64 | + vcclvdspll-supply: | |
| 65 | + description: A 1.8V supply that powers the LVDS PLL. | |
| 66 | + | |
| 67 | + vcclvdstx-supply: | |
| 68 | + description: A 1.8V supply that powers the LVDS TX part. | |
| 69 | + | |
| 70 | + vccmipirx-supply: | |
| 71 | + description: A 1.8V supply that powers the MIPI RX part. | |
| 72 | + | |
| 73 | + vccsysclk-supply: | |
| 74 | + description: A 1.8V supply that powers the SYSCLK. | |
| 75 | + | |
| 76 | + vdd-supply: | |
| 77 | + description: A 1.8V supply that powers the digital part. | |
| 78 | + | |
| 58 | 79 | required: |
| 59 | 80 | - compatible |
| 60 | 81 | - reg |
| @@ -23,6 +23,7 @@ properties: · 22 unmodified lines | ||
| 23 | 23 | items: |
| 24 | 24 | - enum: |
| 25 | 25 | - hannstar,hsd060bhw4 |
| 26 | + - powkiddy,x55-panel | |
| 26 | 27 | - const: himax,hx8394 |
| 27 | 28 | |
| 28 | 29 | reg: true |
| @@ -31,6 +32,8 @@ properties: · 2 unmodified lines | ||
| 31 | 32 | |
| 32 | 33 | backlight: true |
| 33 | 34 | |
| 35 | + rotation: true | |
| 36 | + | |
| 34 | 37 | port: true |
| 35 | 38 | |
| 36 | 39 | vcc-supply: |
| @@ -16,6 +16,7 @@ properties: · 15 unmodified lines | ||
| 16 | 16 | compatible: |
| 17 | 17 | items: |
| 18 | 18 | - enum: |
| 19 | + - ampire,am8001280g | |
| 19 | 20 | - bananapi,lhr050h41 |
| 20 | 21 | - feixin,k101-im2byl02 |
| 21 | 22 | - tdo,tl050hdv35 |
| @@ -21,7 +21,7 @@ properties: · 20 unmodified lines | ||
| 21 | 21 | - enum: |
| 22 | 22 | - anbernic,rg351v-panel |
| 23 | 23 | - anbernic,rg353p-panel |
| 24 | − - anbernic,rg353v-panel | |
| 24 | + - powkiddy,rk2023-panel | |
| 25 | 25 | - const: newvision,nv3051d |
| 26 | 26 | |
| 27 | 27 | reg: true |
| @@ -73,6 +73,8 @@ properties: · 72 unmodified lines | ||
| 73 | 73 | - auo,t215hvn01 |
| 74 | 74 | # Shanghai AVIC Optoelectronics 7" 1024x600 color TFT-LCD panel |
| 75 | 75 | - avic,tm070ddh03 |
| 76 | + # BOE BP101WX1-100 10.1" WXGA (1280x800) LVDS panel | |
| 77 | + - boe,bp101wx1-100 | |
| 76 | 78 | # BOE EV121WXM-N10-1850 12.1" WXGA (1280x800) TFT LCD panel |
| 77 | 79 | - boe,ev121wxm-n10-1850 |
| 78 | 80 | # BOE HV070WSA-100 7.01" WSVGA TFT LCD panel |
| @@ -144,6 +146,8 @@ properties: · 65 unmodified lines | ||
| 144 | 146 | - edt,etmv570g2dhu |
| 145 | 147 | # E Ink VB3300-KCA |
| 146 | 148 | - eink,vb3300-kca |
| 149 | + # Evervision Electronics Co. Ltd. VGG644804 5.7" VGA TFT LCD Panel | |
| 150 | + - evervision,vgg644804 | |
| 147 | 151 | # Evervision Electronics Co. Ltd. VGG804821 5.0" WVGA TFT LCD Panel |
| 148 | 152 | - evervision,vgg804821 |
| 149 | 153 | # Foxlink Group 5" WVGA TFT LCD panel |
| @@ -23,6 +23,7 @@ properties: · 22 unmodified lines | ||
| 23 | 23 | compatible: |
| 24 | 24 | enum: |
| 25 | 25 | - ti,am625-dss |
| 26 | + - ti,am62a7,dss | |
| 26 | 27 | - ti,am65x-dss |
| 27 | 28 | |
| 28 | 29 | reg: |
| @@ -87,6 +88,7 @@ properties: · 58 unmodified lines | ||
| 87 | 88 | For AM65x DSS, the OLDI output port node from video port 1. |
| 88 | 89 | For AM625 DSS, the internal DPI output port node from video |
| 89 | 90 | port 1. |
| 91 | + For AM62A7 DSS, the port is tied off inside the SoC. | |
| 90 | 92 | |
| 91 | 93 | port@1: |
| 92 | 94 | $ref: /schemas/graph.yaml#/properties/port |
| @@ -108,6 +110,18 @@ properties: · 15 unmodified lines | ||
| 108 | 110 | Input memory (from main memory to dispc) bandwidth limit in |
| 109 | 111 | bytes per second |
| 110 | 112 | |
| 113 | +allOf: | |
| 114 | + - if: | |
| 115 | + properties: | |
| 116 | + compatible: | |
| 117 | + contains: | |
| 118 | + const: ti,am62a7-dss | |
| 119 | + then: | |
| 120 | + properties: | |
| 121 | + ports: | |
| 122 | + properties: | |
| 123 | + port@0: false | |
| 124 | + | |
| 111 | 125 | required: |
| 112 | 126 | - compatible |
| 113 | 127 | - reg |
| @@ -29,6 +29,7 @@ properties: · 28 unmodified lines | ||
| 29 | 29 | - allwinner,sun50i-a64-mali |
| 30 | 30 | - rockchip,rk3036-mali |
| 31 | 31 | - rockchip,rk3066-mali |
| 32 | + - rockchip,rk3128-mali | |
| 32 | 33 | - rockchip,rk3188-mali |
| 33 | 34 | - rockchip,rk3228-mali |
| 34 | 35 | - samsung,exynos4210-mali |
| @@ -17,6 +17,7 @@ properties: · 16 unmodified lines | ||
| 17 | 17 | compatible: |
| 18 | 18 | enum: |
| 19 | 19 | - brcm,2711-v3d |
| 20 | + - brcm,2712-v3d | |
| 20 | 21 | - brcm,7268-v3d |
| 21 | 22 | - brcm,7278-v3d |
| 22 | 23 | |
| @@ -0,0 +1,73 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +# Copyright (c) 2023 Imagination Technologies Ltd. | |
| 3 | +%YAML 1.2 | |
| 4 | +--- | |
| 5 | +$id: http://devicetree.org/schemas/gpu/img,powervr.yaml# | |
| 6 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 7 | + | |
| 8 | +title: Imagination Technologies PowerVR and IMG GPU | |
| 9 | + | |
| 10 | +maintainers: | |
| 11 | + - Frank Binns <frank.binns@imgtec.com> | |
| 12 | + | |
| 13 | +properties: | |
| 14 | + compatible: | |
| 15 | + items: | |
| 16 | + - enum: | |
| 17 | + - ti,am62-gpu | |
| 18 | + - const: img,img-axe # IMG AXE GPU model/revision is fully discoverable | |
| 19 | + | |
| 20 | + reg: | |
| 21 | + maxItems: 1 | |
| 22 | + | |
| 23 | + clocks: | |
| 24 | + minItems: 1 | |
| 25 | + maxItems: 3 | |
| 26 | + | |
| 27 | + clock-names: | |
| 28 | + items: | |
| 29 | + - const: core | |
| 30 | + - const: mem | |
| 31 | + - const: sys | |
| 32 | + minItems: 1 | |
| 33 | + | |
| 34 | + interrupts: | |
| 35 | + maxItems: 1 | |
| 36 | + | |
| 37 | + power-domains: | |
| 38 | + maxItems: 1 | |
| 39 | + | |
| 40 | +required: | |
| 41 | + - compatible | |
| 42 | + - reg | |
| 43 | + - clocks | |
| 44 | + - clock-names | |
| 45 | + - interrupts | |
| 46 | + | |
| 47 | +additionalProperties: false | |
| 48 | + | |
| 49 | +allOf: | |
| 50 | + - if: | |
| 51 | + properties: | |
| 52 | + compatible: | |
| 53 | + contains: | |
| 54 | + const: ti,am62-gpu | |
| 55 | + then: | |
| 56 | + properties: | |
| 57 | + clocks: | |
| 58 | + maxItems: 1 | |
| 59 | + | |
| 60 | +examples: | |
| 61 | + - | | |
| 62 | + #include <dt-bindings/interrupt-controller/irq.h> | |
| 63 | + #include <dt-bindings/interrupt-controller/arm-gic.h> | |
| 64 | + #include <dt-bindings/soc/ti,sci_pm_domain.h> | |
| 65 | + | |
| 66 | + gpu@fd00000 { | |
| 67 | + compatible = "ti,am62-gpu", "img,img-axe"; | |
| 68 | + reg = <0x0fd00000 0x20000>; | |
| 69 | + clocks = <&k3_clks 187 0>; | |
| 70 | + clock-names = "core"; | |
| 71 | + interrupts = <GIC_SPI 86 IRQ_TYPE_LEVEL_HIGH>; | |
| 72 | + power-domains = <&k3_pds 187 TI_SCI_PD_EXCLUSIVE>; | |
| 73 | + }; | |
| @@ -275,12 +275,12 @@ allOf: · 274 unmodified lines | ||
| 275 | 275 | properties: |
| 276 | 276 | rx-internal-delay-ps: |
| 277 | 277 | description: |
| 278 | − RGMII Receive Clock Delay defined in pico seconds.This is used for | |
| 278 | + RGMII Receive Clock Delay defined in pico seconds. This is used for | |
| 279 | 279 | controllers that have configurable RX internal delays. If this |
| 280 | 280 | property is present then the MAC applies the RX delay. |
| 281 | 281 | tx-internal-delay-ps: |
| 282 | 282 | description: |
| 283 | − RGMII Transmit Clock Delay defined in pico seconds.This is used for | |
| 283 | + RGMII Transmit Clock Delay defined in pico seconds. This is used for | |
| 284 | 284 | controllers that have configurable TX internal delays. If this |
| 285 | 285 | property is present then the MAC applies the TX delay. |
| 286 | 286 | |
| @@ -36,6 +36,7 @@ properties: · 35 unmodified lines | ||
| 36 | 36 | - qcom,sm8350-ufshc |
| 37 | 37 | - qcom,sm8450-ufshc |
| 38 | 38 | - qcom,sm8550-ufshc |
| 39 | + - qcom,sm8650-ufshc | |
| 39 | 40 | - const: qcom,ufshc |
| 40 | 41 | - const: jedec,ufs-2.0 |
| 41 | 42 | |
| @@ -122,6 +123,7 @@ allOf: · 80 unmodified lines | ||
| 122 | 123 | - qcom,sm8350-ufshc |
| 123 | 124 | - qcom,sm8450-ufshc |
| 124 | 125 | - qcom,sm8550-ufshc |
| 126 | + - qcom,sm8650-ufshc | |
| 125 | 127 | then: |
| 126 | 128 | properties: |
| 127 | 129 | clocks: |
| @@ -36,7 +36,11 @@ properties: · 35 unmodified lines | ||
| 36 | 36 | |
| 37 | 37 | vdd-supply: |
| 38 | 38 | description: |
| 39 | − VDD power supply to the hub | |
| 39 | + 3V3 power supply to the hub | |
| 40 | + | |
| 41 | + vdd2-supply: | |
| 42 | + description: | |
| 43 | + 1V2 power supply to the hub | |
| 40 | 44 | |
| 41 | 45 | peer-hub: |
| 42 | 46 | $ref: /schemas/types.yaml#/definitions/phandle |
| @@ -62,6 +66,7 @@ allOf: · 19 unmodified lines | ||
| 62 | 66 | properties: |
| 63 | 67 | reset-gpios: false |
| 64 | 68 | vdd-supply: false |
| 69 | + vdd2-supply: false | |
| 65 | 70 | peer-hub: false |
| 66 | 71 | i2c-bus: false |
| 67 | 72 | else: |
| @@ -521,8 +521,8 @@ examples: · 520 unmodified lines | ||
| 521 | 521 | |
| 522 | 522 | interrupts = <GIC_SPI 131 IRQ_TYPE_LEVEL_HIGH>, |
| 523 | 523 | <GIC_SPI 486 IRQ_TYPE_LEVEL_HIGH>, |
| 524 | − <GIC_SPI 488 IRQ_TYPE_LEVEL_HIGH>, | |
| 525 | − <GIC_SPI 489 IRQ_TYPE_LEVEL_HIGH>; | |
| 524 | + <GIC_SPI 488 IRQ_TYPE_EDGE_BOTH>, | |
| 525 | + <GIC_SPI 489 IRQ_TYPE_EDGE_BOTH>; | |
| 526 | 526 | interrupt-names = "hs_phy_irq", "ss_phy_irq", |
| 527 | 527 | "dm_hs_phy_irq", "dp_hs_phy_irq"; |
| 528 | 528 | |
| @@ -41,7 +41,7 @@ examples: · 40 unmodified lines | ||
| 41 | 41 | - | |
| 42 | 42 | usb { |
| 43 | 43 | phys = <&usb2_phy1>, <&usb3_phy1>; |
| 44 | − phy-names = "usb"; | |
| 44 | + phy-names = "usb2", "usb3"; | |
| 45 | 45 | #address-cells = <1>; |
| 46 | 46 | #size-cells = <0>; |
| 47 | 47 | |
| @@ -91,6 +91,10 @@ compatibility checking tool (fsck.erofs), and a debugging tool (dump.erofs): · 90 unmodified lines | ||
| 91 | 91 | |
| 92 | 92 | - git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git |
| 93 | 93 | |
| 94 | +For more information, please also refer to the documentation site: | |
| 95 | + | |
| 96 | +- https://erofs.docs.kernel.org | |
| 97 | + | |
| 94 | 98 | Bugs and patches are welcome, please kindly help us and send to the following |
| 95 | 99 | linux-erofs mailing list: |
| 96 | 100 | |
| @@ -3,9 +3,11 @@ GPU Driver Documentation · 2 unmodified lines | ||
| 3 | 3 | ======================== |
| 4 | 4 | |
| 5 | 5 | .. toctree:: |
| 6 | + :maxdepth: 3 | |
| 6 | 7 | |
| 7 | 8 | amdgpu/index |
| 8 | 9 | i915 |
| 10 | + imagination/index | |
| 9 | 11 | mcde |
| 10 | 12 | meson |
| 11 | 13 | pl111 |
| @@ -363,6 +363,12 @@ EDID Helper Functions Reference · 362 unmodified lines | ||
| 363 | 363 | .. kernel-doc:: drivers/gpu/drm/drm_edid.c |
| 364 | 364 | :export: |
| 365 | 365 | |
| 366 | +.. kernel-doc:: include/drm/drm_eld.h | |
| 367 | + :internal: | |
| 368 | + | |
| 369 | +.. kernel-doc:: drivers/gpu/drm/drm_eld.c | |
| 370 | + :export: | |
| 371 | + | |
| 366 | 372 | SCDC Helper Functions Reference |
| 367 | 373 | =============================== |
| 368 | 374 | |
| @@ -548,6 +548,8 @@ Plane Composition Properties · 547 unmodified lines | ||
| 548 | 548 | .. kernel-doc:: drivers/gpu/drm/drm_blend.c |
| 549 | 549 | :doc: overview |
| 550 | 550 | |
| 551 | +.. _damage_tracking_properties: | |
| 552 | + | |
| 551 | 553 | Damage Tracking Properties |
| 552 | 554 | -------------------------- |
| 553 | 555 | |
| @@ -579,6 +581,12 @@ Variable Refresh Properties · 25 unmodified lines | ||
| 579 | 581 | .. kernel-doc:: drivers/gpu/drm/drm_connector.c |
| 580 | 582 | :doc: Variable refresh properties |
| 581 | 583 | |
| 584 | +Cursor Hotspot Properties | |
| 585 | +--------------------------- | |
| 586 | + | |
| 587 | +.. kernel-doc:: drivers/gpu/drm/drm_plane.c | |
| 588 | + :doc: hotspot properties | |
| 589 | + | |
| 582 | 590 | Existing KMS Properties |
| 583 | 591 | ----------------------- |
| 584 | 592 | |
| @@ -466,6 +466,8 @@ DRM MM Range Allocator Function References · 465 unmodified lines | ||
| 466 | 466 | .. kernel-doc:: drivers/gpu/drm/drm_mm.c |
| 467 | 467 | :export: |
| 468 | 468 | |
| 469 | +.. _drm_gpuvm: | |
| 470 | + | |
| 469 | 471 | DRM GPUVM |
| 470 | 472 | ========= |
| 471 | 473 | |
| @@ -481,6 +483,8 @@ Split and Merge · 9 unmodified lines | ||
| 481 | 483 | .. kernel-doc:: drivers/gpu/drm/drm_gpuvm.c |
| 482 | 484 | :doc: Split and Merge |
| 483 | 485 | |
| 486 | +.. _drm_gpuvm_locking: | |
| 487 | + | |
| 484 | 488 | Locking |
| 485 | 489 | ------- |
| 486 | 490 | |
| @@ -552,6 +556,12 @@ Overview · 65 unmodified lines | ||
| 552 | 556 | .. kernel-doc:: drivers/gpu/drm/scheduler/sched_main.c |
| 553 | 557 | :doc: Overview |
| 554 | 558 | |
| 559 | +Flow Control | |
| 560 | +------------ | |
| 561 | + | |
| 562 | +.. kernel-doc:: drivers/gpu/drm/scheduler/sched_main.c | |
| 563 | + :doc: Flow Control | |
| 564 | + | |
| 555 | 565 | Scheduler Function References |
| 556 | 566 | ----------------------------- |
| 557 | 567 | |
| @@ -0,0 +1,582 @@ | ||
| 1 | +.. SPDX-License-Identifier: (GPL-2.0+ OR MIT) | |
| 2 | + | |
| 3 | +=============== | |
| 4 | +VM_BIND locking | |
| 5 | +=============== | |
| 6 | + | |
| 7 | +This document attempts to describe what's needed to get VM_BIND locking right, | |
| 8 | +including the userptr mmu_notifier locking. It also discusses some | |
| 9 | +optimizations to get rid of the looping through of all userptr mappings and | |
| 10 | +external / shared object mappings that is needed in the simplest | |
| 11 | +implementation. In addition, there is a section describing the VM_BIND locking | |
| 12 | +required for implementing recoverable pagefaults. | |
| 13 | + | |
| 14 | +The DRM GPUVM set of helpers | |
| 15 | +============================ | |
| 16 | + | |
| 17 | +There is a set of helpers for drivers implementing VM_BIND, and this | |
| 18 | +set of helpers implements much, but not all of the locking described | |
| 19 | +in this document. In particular, it is currently lacking a userptr | |
| 20 | +implementation. This document does not intend to describe the DRM GPUVM | |
| 21 | +implementation in detail, but it is covered in :ref:`its own | |
| 22 | +documentation <drm_gpuvm>`. It is highly recommended for any driver | |
| 23 | +implementing VM_BIND to use the DRM GPUVM helpers and to extend it if | |
| 24 | +common functionality is missing. | |
| 25 | + | |
| 26 | +Nomenclature | |
| 27 | +============ | |
| 28 | + | |
| 29 | +* ``gpu_vm``: Abstraction of a virtual GPU address space with | |
| 30 | + meta-data. Typically one per client (DRM file-private), or one per | |
| 31 | + execution context. | |
| 32 | +* ``gpu_vma``: Abstraction of a GPU address range within a gpu_vm with | |
| 33 | + associated meta-data. The backing storage of a gpu_vma can either be | |
| 34 | + a GEM object or anonymous or page-cache pages mapped also into the CPU | |
| 35 | + address space for the process. | |
| 36 | +* ``gpu_vm_bo``: Abstracts the association of a GEM object and | |
| 37 | + a VM. The GEM object maintains a list of gpu_vm_bos, where each gpu_vm_bo | |
| 38 | + maintains a list of gpu_vmas. | |
| 39 | +* ``userptr gpu_vma or just userptr``: A gpu_vma, whose backing store | |
| 40 | + is anonymous or page-cache pages as described above. | |
| 41 | +* ``revalidating``: Revalidating a gpu_vma means making the latest version | |
| 42 | + of the backing store resident and making sure the gpu_vma's | |
| 43 | + page-table entries point to that backing store. | |
| 44 | +* ``dma_fence``: A struct dma_fence that is similar to a struct completion | |
| 45 | + and which tracks GPU activity. When the GPU activity is finished, | |
| 46 | + the dma_fence signals. Please refer to the ``DMA Fences`` section of | |
| 47 | + the :doc:`dma-buf doc </driver-api/dma-buf>`. | |
| 48 | +* ``dma_resv``: A struct dma_resv (a.k.a reservation object) that is used | |
| 49 | + to track GPU activity in the form of multiple dma_fences on a | |
| 50 | + gpu_vm or a GEM object. The dma_resv contains an array / list | |
| 51 | + of dma_fences and a lock that needs to be held when adding | |
| 52 | + additional dma_fences to the dma_resv. The lock is of a type that | |
| 53 | + allows deadlock-safe locking of multiple dma_resvs in arbitrary | |
| 54 | + order. Please refer to the ``Reservation Objects`` section of the | |
| 55 | + :doc:`dma-buf doc </driver-api/dma-buf>`. | |
| 56 | +* ``exec function``: An exec function is a function that revalidates all | |
| 57 | + affected gpu_vmas, submits a GPU command batch and registers the | |
| 58 | + dma_fence representing the GPU command's activity with all affected | |
| 59 | + dma_resvs. For completeness, although not covered by this document, | |
| 60 | + it's worth mentioning that an exec function may also be the | |
| 61 | + revalidation worker that is used by some drivers in compute / | |
| 62 | + long-running mode. | |
| 63 | +* ``local object``: A GEM object which is only mapped within a | |
| 64 | + single VM. Local GEM objects share the gpu_vm's dma_resv. | |
| 65 | +* ``external object``: a.k.a shared object: A GEM object which may be shared | |
| 66 | + by multiple gpu_vms and whose backing storage may be shared with | |
| 67 | + other drivers. | |
| 68 | + | |
| 69 | +Locks and locking order | |
| 70 | +======================= | |
| 71 | + | |
| 72 | +One of the benefits of VM_BIND is that local GEM objects share the gpu_vm's | |
| 73 | +dma_resv object and hence the dma_resv lock. So, even with a huge | |
| 74 | +number of local GEM objects, only one lock is needed to make the exec | |
| 75 | +sequence atomic. | |
| 76 | + | |
| 77 | +The following locks and locking orders are used: | |
| 78 | + | |
| 79 | +* The ``gpu_vm->lock`` (optionally an rwsem). Protects the gpu_vm's | |
| 80 | + data structure keeping track of gpu_vmas. It can also protect the | |
| 81 | + gpu_vm's list of userptr gpu_vmas. With a CPU mm analogy this would | |
| 82 | + correspond to the mmap_lock. An rwsem allows several readers to walk | |
| 83 | + the VM tree concurrently, but the benefit of that concurrency most | |
| 84 | + likely varies from driver to driver. | |
| 85 | +* The ``userptr_seqlock``. This lock is taken in read mode for each | |
| 86 | + userptr gpu_vma on the gpu_vm's userptr list, and in write mode during mmu | |
| 87 | + notifier invalidation. This is not a real seqlock but described in | |
| 88 | + ``mm/mmu_notifier.c`` as a "Collision-retry read-side/write-side | |
| 89 | + 'lock' a lot like a seqcount. However this allows multiple | |
| 90 | + write-sides to hold it at once...". The read side critical section | |
| 91 | + is enclosed by ``mmu_interval_read_begin() / | |
| 92 | + mmu_interval_read_retry()`` with ``mmu_interval_read_begin()`` | |
| 93 | + sleeping if the write side is held. | |
| 94 | + The write side is held by the core mm while calling mmu interval | |
| 95 | + invalidation notifiers. | |
| 96 | +* The ``gpu_vm->resv`` lock. Protects the gpu_vm's list of gpu_vmas needing | |
| 97 | + rebinding, as well as the residency state of all the gpu_vm's local | |
| 98 | + GEM objects. | |
| 99 | + Furthermore, it typically protects the gpu_vm's list of evicted and | |
| 100 | + external GEM objects. | |
| 101 | +* The ``gpu_vm->userptr_notifier_lock``. This is an rwsem that is | |
| 102 | + taken in read mode during exec and write mode during a mmu notifier | |
| 103 | + invalidation. The userptr notifier lock is per gpu_vm. | |
| 104 | +* The ``gem_object->gpuva_lock`` This lock protects the GEM object's | |
| 105 | + list of gpu_vm_bos. This is usually the same lock as the GEM | |
| 106 | + object's dma_resv, but some drivers protects this list differently, | |
| 107 | + see below. | |
| 108 | +* The ``gpu_vm list spinlocks``. With some implementations they are needed | |
| 109 | + to be able to update the gpu_vm evicted- and external object | |
| 110 | + list. For those implementations, the spinlocks are grabbed when the | |
| 111 | + lists are manipulated. However, to avoid locking order violations | |
| 112 | + with the dma_resv locks, a special scheme is needed when iterating | |
| 113 | + over the lists. | |
| 114 | + | |
| 115 | +.. _gpu_vma lifetime: | |
| 116 | + | |
| 117 | +Protection and lifetime of gpu_vm_bos and gpu_vmas | |
| 118 | +================================================== | |
| 119 | + | |
| 120 | +The GEM object's list of gpu_vm_bos, and the gpu_vm_bo's list of gpu_vmas | |
| 121 | +is protected by the ``gem_object->gpuva_lock``, which is typically the | |
| 122 | +same as the GEM object's dma_resv, but if the driver | |
| 123 | +needs to access these lists from within a dma_fence signalling | |
| 124 | +critical section, it can instead choose to protect it with a | |
| 125 | +separate lock, which can be locked from within the dma_fence signalling | |
| 126 | +critical section. Such drivers then need to pay additional attention | |
| 127 | +to what locks need to be taken from within the loop when iterating | |
| 128 | +over the gpu_vm_bo and gpu_vma lists to avoid locking-order violations. | |
| 129 | + | |
| 130 | +The DRM GPUVM set of helpers provide lockdep asserts that this lock is | |
| 131 | +held in relevant situations and also provides a means of making itself | |
| 132 | +aware of which lock is actually used: :c:func:`drm_gem_gpuva_set_lock`. | |
| 133 | + | |
| 134 | +Each gpu_vm_bo holds a reference counted pointer to the underlying GEM | |
| 135 | +object, and each gpu_vma holds a reference counted pointer to the | |
| 136 | +gpu_vm_bo. When iterating over the GEM object's list of gpu_vm_bos and | |
| 137 | +over the gpu_vm_bo's list of gpu_vmas, the ``gem_object->gpuva_lock`` must | |
| 138 | +not be dropped, otherwise, gpu_vmas attached to a gpu_vm_bo may | |
| 139 | +disappear without notice since those are not reference-counted. A | |
| 140 | +driver may implement its own scheme to allow this at the expense of | |
| 141 | +additional complexity, but this is outside the scope of this document. | |
| 142 | + | |
| 143 | +In the DRM GPUVM implementation, each gpu_vm_bo and each gpu_vma | |
| 144 | +holds a reference count on the gpu_vm itself. Due to this, and to avoid circular | |
| 145 | +reference counting, cleanup of the gpu_vm's gpu_vmas must not be done from the | |
| 146 | +gpu_vm's destructor. Drivers typically implements a gpu_vm close | |
| 147 | +function for this cleanup. The gpu_vm close function will abort gpu | |
| 148 | +execution using this VM, unmap all gpu_vmas and release page-table memory. | |
| 149 | + | |
| 150 | +Revalidation and eviction of local objects | |
| 151 | +========================================== | |
| 152 | + | |
| 153 | +Note that in all the code examples given below we use simplified | |
| 154 | +pseudo-code. In particular, the dma_resv deadlock avoidance algorithm | |
| 155 | +as well as reserving memory for dma_resv fences is left out. | |
| 156 | + | |
| 157 | +Revalidation | |
| 158 | +____________ | |
| 159 | +With VM_BIND, all local objects need to be resident when the gpu is | |
| 160 | +executing using the gpu_vm, and the objects need to have valid | |
| 161 | +gpu_vmas set up pointing to them. Typically, each gpu command buffer | |
| 162 | +submission is therefore preceded with a re-validation section: | |
| 163 | + | |
| 164 | +.. code-block:: C | |
| 165 | + | |
| 166 | + dma_resv_lock(gpu_vm->resv); | |
| 167 | + | |
| 168 | + // Validation section starts here. | |
| 169 | + for_each_gpu_vm_bo_on_evict_list(&gpu_vm->evict_list, &gpu_vm_bo) { | |
| 170 | + validate_gem_bo(&gpu_vm_bo->gem_bo); | |
| 171 | + | |
| 172 | + // The following list iteration needs the Gem object's | |
| 173 | + // dma_resv to be held (it protects the gpu_vm_bo's list of | |
| 174 | + // gpu_vmas, but since local gem objects share the gpu_vm's | |
| 175 | + // dma_resv, it is already held at this point. | |
| 176 | + for_each_gpu_vma_of_gpu_vm_bo(&gpu_vm_bo, &gpu_vma) | |
| 177 | + move_gpu_vma_to_rebind_list(&gpu_vma, &gpu_vm->rebind_list); | |
| 178 | + } | |
| 179 | + | |
| 180 | + for_each_gpu_vma_on_rebind_list(&gpu vm->rebind_list, &gpu_vma) { | |
| 181 | + rebind_gpu_vma(&gpu_vma); | |
| 182 | + remove_gpu_vma_from_rebind_list(&gpu_vma); | |
| 183 | + } | |
| 184 | + // Validation section ends here, and job submission starts. | |
| 185 | + | |
| 186 | + add_dependencies(&gpu_job, &gpu_vm->resv); | |
| 187 | + job_dma_fence = gpu_submit(&gpu_job)); | |
| 188 | + | |
| 189 | + add_dma_fence(job_dma_fence, &gpu_vm->resv); | |
| 190 | + dma_resv_unlock(gpu_vm->resv); | |
| 191 | + | |
| 192 | +The reason for having a separate gpu_vm rebind list is that there | |
| 193 | +might be userptr gpu_vmas that are not mapping a buffer object that | |
| 194 | +also need rebinding. | |
| 195 | + | |
| 196 | +Eviction | |
| 197 | +________ | |
| 198 | + | |
| 199 | +Eviction of one of these local objects will then look similar to the | |
| 200 | +following: | |
| 201 | + | |
| 202 | +.. code-block:: C | |
| 203 | + | |
| 204 | + obj = get_object_from_lru(); | |
| 205 | + | |
| 206 | + dma_resv_lock(obj->resv); | |
| 207 | + for_each_gpu_vm_bo_of_obj(obj, &gpu_vm_bo); | |
| 208 | + add_gpu_vm_bo_to_evict_list(&gpu_vm_bo, &gpu_vm->evict_list); | |
| 209 | + | |
| 210 | + add_dependencies(&eviction_job, &obj->resv); | |
| 211 | + job_dma_fence = gpu_submit(&eviction_job); | |
| 212 | + add_dma_fence(&obj->resv, job_dma_fence); | |
| 213 | + | |
| 214 | + dma_resv_unlock(&obj->resv); | |
| 215 | + put_object(obj); | |
| 216 | + | |
| 217 | +Note that since the object is local to the gpu_vm, it will share the gpu_vm's | |
| 218 | +dma_resv lock such that ``obj->resv == gpu_vm->resv``. | |
| 219 | +The gpu_vm_bos marked for eviction are put on the gpu_vm's evict list, | |
| 220 | +which is protected by ``gpu_vm->resv``. During eviction all local | |
| 221 | +objects have their dma_resv locked and, due to the above equality, also | |
| 222 | +the gpu_vm's dma_resv protecting the gpu_vm's evict list is locked. | |
| 223 | + | |
| 224 | +With VM_BIND, gpu_vmas don't need to be unbound before eviction, | |
| 225 | +since the driver must ensure that the eviction blit or copy will wait | |
| 226 | +for GPU idle or depend on all previous GPU activity. Furthermore, any | |
| 227 | +subsequent attempt by the GPU to access freed memory through the | |
| 228 | +gpu_vma will be preceded by a new exec function, with a revalidation | |
| 229 | +section which will make sure all gpu_vmas are rebound. The eviction | |
| 230 | +code holding the object's dma_resv while revalidating will ensure a | |
| 231 | +new exec function may not race with the eviction. | |
| 232 | + | |
| 233 | +A driver can be implemented in such a way that, on each exec function, | |
| 234 | +only a subset of vmas are selected for rebind. In this case, all vmas that are | |
| 235 | +*not* selected for rebind must be unbound before the exec | |
| 236 | +function workload is submitted. | |
| 237 | + | |
| 238 | +Locking with external buffer objects | |
| 239 | +==================================== | |
| 240 | + | |
| 241 | +Since external buffer objects may be shared by multiple gpu_vm's they | |
| 242 | +can't share their reservation object with a single gpu_vm. Instead | |
| 243 | +they need to have a reservation object of their own. The external | |
| 244 | +objects bound to a gpu_vm using one or many gpu_vmas are therefore put on a | |
| 245 | +per-gpu_vm list which is protected by the gpu_vm's dma_resv lock or | |
| 246 | +one of the :ref:`gpu_vm list spinlocks <Spinlock iteration>`. Once | |
| 247 | +the gpu_vm's reservation object is locked, it is safe to traverse the | |
| 248 | +external object list and lock the dma_resvs of all external | |
| 249 | +objects. However, if instead a list spinlock is used, a more elaborate | |
| 250 | +iteration scheme needs to be used. | |
| 251 | + | |
| 252 | +At eviction time, the gpu_vm_bos of *all* the gpu_vms an external | |
| 253 | +object is bound to need to be put on their gpu_vm's evict list. | |
| 254 | +However, when evicting an external object, the dma_resvs of the | |
| 255 | +gpu_vms the object is bound to are typically not held. Only | |
| 256 | +the object's private dma_resv can be guaranteed to be held. If there | |
| 257 | +is a ww_acquire context at hand at eviction time we could grab those | |
| 258 | +dma_resvs but that could cause expensive ww_mutex rollbacks. A simple | |
| 259 | +option is to just mark the gpu_vm_bos of the evicted gem object with | |
| 260 | +an ``evicted`` bool that is inspected before the next time the | |
| 261 | +corresponding gpu_vm evicted list needs to be traversed. For example, when | |
| 262 | +traversing the list of external objects and locking them. At that time, | |
| 263 | +both the gpu_vm's dma_resv and the object's dma_resv is held, and the | |
| 264 | +gpu_vm_bo marked evicted, can then be added to the gpu_vm's list of | |
| 265 | +evicted gpu_vm_bos. The ``evicted`` bool is formally protected by the | |
| 266 | +object's dma_resv. | |
| 267 | + | |
| 268 | +The exec function becomes | |
| 269 | + | |
| 270 | +.. code-block:: C | |
| 271 | + | |
| 272 | + dma_resv_lock(gpu_vm->resv); | |
| 273 | + | |
| 274 | + // External object list is protected by the gpu_vm->resv lock. | |
| 275 | + for_each_gpu_vm_bo_on_extobj_list(gpu_vm, &gpu_vm_bo) { | |
| 276 | + dma_resv_lock(gpu_vm_bo.gem_obj->resv); | |
| 277 | + if (gpu_vm_bo_marked_evicted(&gpu_vm_bo)) | |
| 278 | + add_gpu_vm_bo_to_evict_list(&gpu_vm_bo, &gpu_vm->evict_list); | |
| 279 | + } | |
| 280 | + | |
| 281 | + for_each_gpu_vm_bo_on_evict_list(&gpu_vm->evict_list, &gpu_vm_bo) { | |
| 282 | + validate_gem_bo(&gpu_vm_bo->gem_bo); | |
| 283 | + | |
| 284 | + for_each_gpu_vma_of_gpu_vm_bo(&gpu_vm_bo, &gpu_vma) | |
| 285 | + move_gpu_vma_to_rebind_list(&gpu_vma, &gpu_vm->rebind_list); | |
| 286 | + } | |
| 287 | + | |
| 288 | + for_each_gpu_vma_on_rebind_list(&gpu vm->rebind_list, &gpu_vma) { | |
| 289 | + rebind_gpu_vma(&gpu_vma); | |
| 290 | + remove_gpu_vma_from_rebind_list(&gpu_vma); | |
| 291 | + } | |
| 292 | + | |
| 293 | + add_dependencies(&gpu_job, &gpu_vm->resv); | |
| 294 | + job_dma_fence = gpu_submit(&gpu_job)); | |
| 295 | + | |
| 296 | + add_dma_fence(job_dma_fence, &gpu_vm->resv); | |
| 297 | + for_each_external_obj(gpu_vm, &obj) | |
| 298 | + add_dma_fence(job_dma_fence, &obj->resv); | |
| 299 | + dma_resv_unlock_all_resv_locks(); | |
| 300 | + | |
| 301 | +And the corresponding shared-object aware eviction would look like: | |
| 302 | + | |
| 303 | +.. code-block:: C | |
| 304 | + | |
| 305 | + obj = get_object_from_lru(); | |
| 306 | + | |
| 307 | + dma_resv_lock(obj->resv); | |
| 308 | + for_each_gpu_vm_bo_of_obj(obj, &gpu_vm_bo) | |
| 309 | + if (object_is_vm_local(obj)) | |
| 310 | + add_gpu_vm_bo_to_evict_list(&gpu_vm_bo, &gpu_vm->evict_list); | |
| 311 | + else | |
| 312 | + mark_gpu_vm_bo_evicted(&gpu_vm_bo); | |
| 313 | + | |
| 314 | + add_dependencies(&eviction_job, &obj->resv); | |
| 315 | + job_dma_fence = gpu_submit(&eviction_job); | |
| 316 | + add_dma_fence(&obj->resv, job_dma_fence); | |
| 317 | + | |
| 318 | + dma_resv_unlock(&obj->resv); | |
| 319 | + put_object(obj); | |
| 320 | + | |
| 321 | +.. _Spinlock iteration: | |
| 322 | + | |
| 323 | +Accessing the gpu_vm's lists without the dma_resv lock held | |
| 324 | +=========================================================== | |
| 325 | + | |
| 326 | +Some drivers will hold the gpu_vm's dma_resv lock when accessing the | |
| 327 | +gpu_vm's evict list and external objects lists. However, there are | |
| 328 | +drivers that need to access these lists without the dma_resv lock | |
| 329 | +held, for example due to asynchronous state updates from within the | |
| 330 | +dma_fence signalling critical path. In such cases, a spinlock can be | |
| 331 | +used to protect manipulation of the lists. However, since higher level | |
| 332 | +sleeping locks need to be taken for each list item while iterating | |
| 333 | +over the lists, the items already iterated over need to be | |
| 334 | +temporarily moved to a private list and the spinlock released | |
| 335 | +while processing each item: | |
| 336 | + | |
| 337 | +.. code block:: C | |
| 338 | + | |
| 339 | + struct list_head still_in_list; | |
| 340 | + | |
| 341 | + INIT_LIST_HEAD(&still_in_list); | |
| 342 | + | |
| 343 | + spin_lock(&gpu_vm->list_lock); | |
| 344 | + do { | |
| 345 | + struct list_head *entry = list_first_entry_or_null(&gpu_vm->list, head); | |
| 346 | + | |
| 347 | + if (!entry) | |
| 348 | + break; | |
| 349 | + | |
| 350 | + list_move_tail(&entry->head, &still_in_list); | |
| 351 | + list_entry_get_unless_zero(entry); | |
| 352 | + spin_unlock(&gpu_vm->list_lock); | |
| 353 | + | |
| 354 | + process(entry); | |
| 355 | + | |
| 356 | + spin_lock(&gpu_vm->list_lock); | |
| 357 | + list_entry_put(entry); | |
| 358 | + } while (true); | |
| 359 | + | |
| 360 | + list_splice_tail(&still_in_list, &gpu_vm->list); | |
| 361 | + spin_unlock(&gpu_vm->list_lock); | |
| 362 | + | |
| 363 | +Due to the additional locking and atomic operations, drivers that *can* | |
| 364 | +avoid accessing the gpu_vm's list outside of the dma_resv lock | |
| 365 | +might want to avoid also this iteration scheme. Particularly, if the | |
| 366 | +driver anticipates a large number of list items. For lists where the | |
| 367 | +anticipated number of list items is small, where list iteration doesn't | |
| 368 | +happen very often or if there is a significant additional cost | |
| 369 | +associated with each iteration, the atomic operation overhead | |
| 370 | +associated with this type of iteration is, most likely, negligible. Note that | |
| 371 | +if this scheme is used, it is necessary to make sure this list | |
| 372 | +iteration is protected by an outer level lock or semaphore, since list | |
| 373 | +items are temporarily pulled off the list while iterating, and it is | |
| 374 | +also worth mentioning that the local list ``still_in_list`` should | |
| 375 | +also be considered protected by the ``gpu_vm->list_lock``, and it is | |
| 376 | +thus possible that items can be removed also from the local list | |
| 377 | +concurrently with list iteration. | |
| 378 | + | |
| 379 | +Please refer to the :ref:`DRM GPUVM locking section | |
| 380 | +<drm_gpuvm_locking>` and its internal | |
| 381 | +:c:func:`get_next_vm_bo_from_list` function. | |
| 382 | + | |
| 383 | + | |
| 384 | +userptr gpu_vmas | |
| 385 | +================ | |
| 386 | + | |
| 387 | +A userptr gpu_vma is a gpu_vma that, instead of mapping a buffer object to a | |
| 388 | +GPU virtual address range, directly maps a CPU mm range of anonymous- | |
| 389 | +or file page-cache pages. | |
| 390 | +A very simple approach would be to just pin the pages using | |
| 391 | +pin_user_pages() at bind time and unpin them at unbind time, but this | |
| 392 | +creates a Denial-Of-Service vector since a single user-space process | |
| 393 | +would be able to pin down all of system memory, which is not | |
| 394 | +desirable. (For special use-cases and assuming proper accounting pinning might | |
| 395 | +still be a desirable feature, though). What we need to do in the | |
| 396 | +general case is to obtain a reference to the desired pages, make sure | |
| 397 | +we are notified using a MMU notifier just before the CPU mm unmaps the | |
| 398 | +pages, dirty them if they are not mapped read-only to the GPU, and | |
| 399 | +then drop the reference. | |
| 400 | +When we are notified by the MMU notifier that CPU mm is about to drop the | |
| 401 | +pages, we need to stop GPU access to the pages by waiting for VM idle | |
| 402 | +in the MMU notifier and make sure that before the next time the GPU | |
| 403 | +tries to access whatever is now present in the CPU mm range, we unmap | |
| 404 | +the old pages from the GPU page tables and repeat the process of | |
| 405 | +obtaining new page references. (See the :ref:`notifier example | |
| 406 | +<Invalidation example>` below). Note that when the core mm decides to | |
| 407 | +laundry pages, we get such an unmap MMU notification and can mark the | |
| 408 | +pages dirty again before the next GPU access. We also get similar MMU | |
| 409 | +notifications for NUMA accounting which the GPU driver doesn't really | |
| 410 | +need to care about, but so far it has proven difficult to exclude | |
| 411 | +certain notifications. | |
| 412 | + | |
| 413 | +Using a MMU notifier for device DMA (and other methods) is described in | |
| 414 | +:ref:`the pin_user_pages() documentation <mmu-notifier-registration-case>`. | |
| 415 | + | |
| 416 | +Now, the method of obtaining struct page references using | |
| 417 | +get_user_pages() unfortunately can't be used under a dma_resv lock | |
| 418 | +since that would violate the locking order of the dma_resv lock vs the | |
| 419 | +mmap_lock that is grabbed when resolving a CPU pagefault. This means | |
| 420 | +the gpu_vm's list of userptr gpu_vmas needs to be protected by an | |
| 421 | +outer lock, which in our example below is the ``gpu_vm->lock``. | |
| 422 | + | |
| 423 | +The MMU interval seqlock for a userptr gpu_vma is used in the following | |
| 424 | +way: | |
| 425 | + | |
| 426 | +.. code-block:: C | |
| 427 | + | |
| 428 | + // Exclusive locking mode here is strictly needed only if there are | |
| 429 | + // invalidated userptr gpu_vmas present, to avoid concurrent userptr | |
| 430 | + // revalidations of the same userptr gpu_vma. | |
| 431 | + down_write(&gpu_vm->lock); | |
| 432 | + retry: | |
| 433 | + | |
| 434 | + // Note: mmu_interval_read_begin() blocks until there is no | |
| 435 | + // invalidation notifier running anymore. | |
| 436 | + seq = mmu_interval_read_begin(&gpu_vma->userptr_interval); | |
| 437 | + if (seq != gpu_vma->saved_seq) { | |
| 438 | + obtain_new_page_pointers(&gpu_vma); | |
| 439 | + dma_resv_lock(&gpu_vm->resv); | |
| 440 | + add_gpu_vma_to_revalidate_list(&gpu_vma, &gpu_vm); | |
| 441 | + dma_resv_unlock(&gpu_vm->resv); | |
| 442 | + gpu_vma->saved_seq = seq; | |
| 443 | + } | |
| 444 | + | |
| 445 | + // The usual revalidation goes here. | |
| 446 | + | |
| 447 | + // Final userptr sequence validation may not happen before the | |
| 448 | + // submission dma_fence is added to the gpu_vm's resv, from the POW | |
| 449 | + // of the MMU invalidation notifier. Hence the | |
| 450 | + // userptr_notifier_lock that will make them appear atomic. | |
| 451 | + | |
| 452 | + add_dependencies(&gpu_job, &gpu_vm->resv); | |
| 453 | + down_read(&gpu_vm->userptr_notifier_lock); | |
| 454 | + if (mmu_interval_read_retry(&gpu_vma->userptr_interval, gpu_vma->saved_seq)) { | |
| 455 | + up_read(&gpu_vm->userptr_notifier_lock); | |
| 456 | + goto retry; | |
| 457 | + } | |
| 458 | + | |
| 459 | + job_dma_fence = gpu_submit(&gpu_job)); | |
| 460 | + | |
| 461 | + add_dma_fence(job_dma_fence, &gpu_vm->resv); | |
| 462 | + | |
| 463 | + for_each_external_obj(gpu_vm, &obj) | |
| 464 | + add_dma_fence(job_dma_fence, &obj->resv); | |
| 465 | + | |
| 466 | + dma_resv_unlock_all_resv_locks(); | |
| 467 | + up_read(&gpu_vm->userptr_notifier_lock); | |
| 468 | + up_write(&gpu_vm->lock); | |
| 469 | + | |
| 470 | +The code between ``mmu_interval_read_begin()`` and the | |
| 471 | +``mmu_interval_read_retry()`` marks the read side critical section of | |
| 472 | +what we call the ``userptr_seqlock``. In reality, the gpu_vm's userptr | |
| 473 | +gpu_vma list is looped through, and the check is done for *all* of its | |
| 474 | +userptr gpu_vmas, although we only show a single one here. | |
| 475 | + | |
| 476 | +The userptr gpu_vma MMU invalidation notifier might be called from | |
| 477 | +reclaim context and, again, to avoid locking order violations, we can't | |
| 478 | +take any dma_resv lock nor the gpu_vm->lock from within it. | |
| 479 | + | |
| 480 | +.. _Invalidation example: | |
| 481 | +.. code-block:: C | |
| 482 | + | |
| 483 | + bool gpu_vma_userptr_invalidate(userptr_interval, cur_seq) | |
| 484 | + { | |
| 485 | + // Make sure the exec function either sees the new sequence | |
| 486 | + // and backs off or we wait for the dma-fence: | |
| 487 | + | |
| 488 | + down_write(&gpu_vm->userptr_notifier_lock); | |
| 489 | + mmu_interval_set_seq(userptr_interval, cur_seq); | |
| 490 | + up_write(&gpu_vm->userptr_notifier_lock); | |
| 491 | + | |
| 492 | + // At this point, the exec function can't succeed in | |
| 493 | + // submitting a new job, because cur_seq is an invalid | |
| 494 | + // sequence number and will always cause a retry. When all | |
| 495 | + // invalidation callbacks, the mmu notifier core will flip | |
| 496 | + // the sequence number to a valid one. However we need to | |
| 497 | + // stop gpu access to the old pages here. | |
| 498 | + | |
| 499 | + dma_resv_wait_timeout(&gpu_vm->resv, DMA_RESV_USAGE_BOOKKEEP, | |
| 500 | + false, MAX_SCHEDULE_TIMEOUT); | |
| 501 | + return true; | |
| 502 | + } | |
| 503 | + | |
| 504 | +When this invalidation notifier returns, the GPU can no longer be | |
| 505 | +accessing the old pages of the userptr gpu_vma and needs to redo the | |
| 506 | +page-binding before a new GPU submission can succeed. | |
| 507 | + | |
| 508 | +Efficient userptr gpu_vma exec_function iteration | |
| 509 | +_________________________________________________ | |
| 510 | + | |
| 511 | +If the gpu_vm's list of userptr gpu_vmas becomes large, it's | |
| 512 | +inefficient to iterate through the complete lists of userptrs on each | |
| 513 | +exec function to check whether each userptr gpu_vma's saved | |
| 514 | +sequence number is stale. A solution to this is to put all | |
| 515 | +*invalidated* userptr gpu_vmas on a separate gpu_vm list and | |
| 516 | +only check the gpu_vmas present on this list on each exec | |
| 517 | +function. This list will then lend itself very-well to the spinlock | |
| 518 | +locking scheme that is | |
| 519 | +:ref:`described in the spinlock iteration section <Spinlock iteration>`, since | |
| 520 | +in the mmu notifier, where we add the invalidated gpu_vmas to the | |
| 521 | +list, it's not possible to take any outer locks like the | |
| 522 | +``gpu_vm->lock`` or the ``gpu_vm->resv`` lock. Note that the | |
| 523 | +``gpu_vm->lock`` still needs to be taken while iterating to ensure the list is | |
| 524 | +complete, as also mentioned in that section. | |
| 525 | + | |
| 526 | +If using an invalidated userptr list like this, the retry check in the | |
| 527 | +exec function trivially becomes a check for invalidated list empty. | |
| 528 | + | |
| 529 | +Locking at bind and unbind time | |
| 530 | +=============================== | |
| 531 | + | |
| 532 | +At bind time, assuming a GEM object backed gpu_vma, each | |
| 533 | +gpu_vma needs to be associated with a gpu_vm_bo and that | |
| 534 | +gpu_vm_bo in turn needs to be added to the GEM object's | |
| 535 | +gpu_vm_bo list, and possibly to the gpu_vm's external object | |
| 536 | +list. This is referred to as *linking* the gpu_vma, and typically | |
| 537 | +requires that the ``gpu_vm->lock`` and the ``gem_object->gpuva_lock`` | |
| 538 | +are held. When unlinking a gpu_vma the same locks should be held, | |
| 539 | +and that ensures that when iterating over ``gpu_vmas`, either under | |
| 540 | +the ``gpu_vm->resv`` or the GEM object's dma_resv, that the gpu_vmas | |
| 541 | +stay alive as long as the lock under which we iterate is not released. For | |
| 542 | +userptr gpu_vmas it's similarly required that during vma destroy, the | |
| 543 | +outer ``gpu_vm->lock`` is held, since otherwise when iterating over | |
| 544 | +the invalidated userptr list as described in the previous section, | |
| 545 | +there is nothing keeping those userptr gpu_vmas alive. | |
| 546 | + | |
| 547 | +Locking for recoverable page-fault page-table updates | |
| 548 | +===================================================== | |
| 549 | + | |
| 550 | +There are two important things we need to ensure with locking for | |
| 551 | +recoverable page-faults: | |
| 552 | + | |
| 553 | +* At the time we return pages back to the system / allocator for | |
| 554 | + reuse, there should be no remaining GPU mappings and any GPU TLB | |
| 555 | + must have been flushed. | |
| 556 | +* The unmapping and mapping of a gpu_vma must not race. | |
| 557 | + | |
| 558 | +Since the unmapping (or zapping) of GPU ptes is typically taking place | |
| 559 | +where it is hard or even impossible to take any outer level locks we | |
| 560 | +must either introduce a new lock that is held at both mapping and | |
| 561 | +unmapping time, or look at the locks we do hold at unmapping time and | |
| 562 | +make sure that they are held also at mapping time. For userptr | |
| 563 | +gpu_vmas, the ``userptr_seqlock`` is held in write mode in the mmu | |
| 564 | +invalidation notifier where zapping happens. Hence, if the | |
| 565 | +``userptr_seqlock`` as well as the ``gpu_vm->userptr_notifier_lock`` | |
| 566 | +is held in read mode during mapping, it will not race with the | |
| 567 | +zapping. For GEM object backed gpu_vmas, zapping will take place under | |
| 568 | +the GEM object's dma_resv and ensuring that the dma_resv is held also | |
| 569 | +when populating the page-tables for any gpu_vma pointing to the GEM | |
| 570 | +object, will similarly ensure we are race-free. | |
| 571 | + | |
| 572 | +If any part of the mapping is performed asynchronously | |
| 573 | +under a dma-fence with these locks released, the zapping will need to | |
| 574 | +wait for that dma-fence to signal under the relevant lock before | |
| 575 | +starting to modify the page-table. | |
| 576 | + | |
| 577 | +Since modifying the | |
| 578 | +page-table structure in a way that frees up page-table memory | |
| 579 | +might also require outer level locks, the zapping of GPU ptes | |
| 580 | +typically focuses only on zeroing page-table or page-directory entries | |
| 581 | +and flushing TLB, whereas freeing of page-table memory is deferred to | |
| 582 | +unbind or rebind time. | |
| @@ -0,0 +1,13 @@ | ||
| 1 | +======================================= | |
| 2 | +drm/imagination PowerVR Graphics Driver | |
| 3 | +======================================= | |
| 4 | + | |
| 5 | +.. kernel-doc:: drivers/gpu/drm/imagination/pvr_drv.c | |
| 6 | + :doc: PowerVR (Series 6 and later) and IMG Graphics Driver | |
| 7 | + | |
| 8 | +Contents | |
| 9 | +======== | |
| 10 | +.. toctree:: | |
| 11 | + :maxdepth: 2 | |
| 12 | + | |
| 13 | + uapi | |
| @@ -0,0 +1,171 @@ | ||
| 1 | +==== | |
| 2 | +UAPI | |
| 3 | +==== | |
| 4 | +The sources associated with this section can be found in ``pvr_drm.h``. | |
| 5 | + | |
| 6 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 7 | + :doc: PowerVR UAPI | |
| 8 | + | |
| 9 | +OBJECT ARRAYS | |
| 10 | +============= | |
| 11 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 12 | + :identifiers: drm_pvr_obj_array | |
| 13 | + | |
| 14 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 15 | + :identifiers: DRM_PVR_OBJ_ARRAY | |
| 16 | + | |
| 17 | +IOCTLS | |
| 18 | +====== | |
| 19 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 20 | + :doc: PowerVR IOCTL interface | |
| 21 | + | |
| 22 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 23 | + :identifiers: PVR_IOCTL | |
| 24 | + | |
| 25 | +DEV_QUERY | |
| 26 | +--------- | |
| 27 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 28 | + :doc: PowerVR IOCTL DEV_QUERY interface | |
| 29 | + | |
| 30 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 31 | + :identifiers: drm_pvr_dev_query | |
| 32 | + | |
| 33 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 34 | + :identifiers: drm_pvr_ioctl_dev_query_args | |
| 35 | + | |
| 36 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 37 | + :identifiers: drm_pvr_dev_query_gpu_info | |
| 38 | + drm_pvr_dev_query_runtime_info | |
| 39 | + drm_pvr_dev_query_hwrt_info | |
| 40 | + drm_pvr_dev_query_quirks | |
| 41 | + drm_pvr_dev_query_enhancements | |
| 42 | + | |
| 43 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 44 | + :identifiers: drm_pvr_heap_id | |
| 45 | + drm_pvr_heap | |
| 46 | + drm_pvr_dev_query_heap_info | |
| 47 | + | |
| 48 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 49 | + :identifiers: drm_pvr_static_data_area_usage | |
| 50 | + drm_pvr_static_data_area | |
| 51 | + drm_pvr_dev_query_static_data_areas | |
| 52 | + | |
| 53 | +CREATE_BO | |
| 54 | +--------- | |
| 55 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 56 | + :doc: PowerVR IOCTL CREATE_BO interface | |
| 57 | + | |
| 58 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 59 | + :identifiers: drm_pvr_ioctl_create_bo_args | |
| 60 | + | |
| 61 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 62 | + :doc: Flags for CREATE_BO | |
| 63 | + | |
| 64 | +GET_BO_MMAP_OFFSET | |
| 65 | +------------------ | |
| 66 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 67 | + :doc: PowerVR IOCTL GET_BO_MMAP_OFFSET interface | |
| 68 | + | |
| 69 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 70 | + :identifiers: drm_pvr_ioctl_get_bo_mmap_offset_args | |
| 71 | + | |
| 72 | +CREATE_VM_CONTEXT and DESTROY_VM_CONTEXT | |
| 73 | +---------------------------------------- | |
| 74 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 75 | + :doc: PowerVR IOCTL CREATE_VM_CONTEXT and DESTROY_VM_CONTEXT interfaces | |
| 76 | + | |
| 77 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 78 | + :identifiers: drm_pvr_ioctl_create_vm_context_args | |
| 79 | + drm_pvr_ioctl_destroy_vm_context_args | |
| 80 | + | |
| 81 | +VM_MAP and VM_UNMAP | |
| 82 | +------------------- | |
| 83 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 84 | + :doc: PowerVR IOCTL VM_MAP and VM_UNMAP interfaces | |
| 85 | + | |
| 86 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 87 | + :identifiers: drm_pvr_ioctl_vm_map_args | |
| 88 | + drm_pvr_ioctl_vm_unmap_args | |
| 89 | + | |
| 90 | +CREATE_CONTEXT and DESTROY_CONTEXT | |
| 91 | +---------------------------------- | |
| 92 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 93 | + :doc: PowerVR IOCTL CREATE_CONTEXT and DESTROY_CONTEXT interfaces | |
| 94 | + | |
| 95 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 96 | + :identifiers: drm_pvr_ioctl_create_context_args | |
| 97 | + | |
| 98 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 99 | + :identifiers: drm_pvr_ctx_priority | |
| 100 | + drm_pvr_ctx_type | |
| 101 | + drm_pvr_static_render_context_state | |
| 102 | + drm_pvr_static_render_context_state_format | |
| 103 | + drm_pvr_reset_framework | |
| 104 | + drm_pvr_reset_framework_format | |
| 105 | + | |
| 106 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 107 | + :identifiers: drm_pvr_ioctl_destroy_context_args | |
| 108 | + | |
| 109 | +CREATE_FREE_LIST and DESTROY_FREE_LIST | |
| 110 | +-------------------------------------- | |
| 111 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 112 | + :doc: PowerVR IOCTL CREATE_FREE_LIST and DESTROY_FREE_LIST interfaces | |
| 113 | + | |
| 114 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 115 | + :identifiers: drm_pvr_ioctl_create_free_list_args | |
| 116 | + | |
| 117 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 118 | + :identifiers: drm_pvr_ioctl_destroy_free_list_args | |
| 119 | + | |
| 120 | +CREATE_HWRT_DATASET and DESTROY_HWRT_DATASET | |
| 121 | +-------------------------------------------- | |
| 122 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 123 | + :doc: PowerVR IOCTL CREATE_HWRT_DATASET and DESTROY_HWRT_DATASET interfaces | |
| 124 | + | |
| 125 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 126 | + :identifiers: drm_pvr_ioctl_create_hwrt_dataset_args | |
| 127 | + | |
| 128 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 129 | + :identifiers: drm_pvr_create_hwrt_geom_data_args | |
| 130 | + drm_pvr_create_hwrt_rt_data_args | |
| 131 | + | |
| 132 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 133 | + :identifiers: drm_pvr_ioctl_destroy_hwrt_dataset_args | |
| 134 | + | |
| 135 | +SUBMIT_JOBS | |
| 136 | +----------- | |
| 137 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 138 | + :doc: PowerVR IOCTL SUBMIT_JOBS interface | |
| 139 | + | |
| 140 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 141 | + :doc: Flags for the drm_pvr_sync_op object. | |
| 142 | + | |
| 143 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 144 | + :identifiers: drm_pvr_ioctl_submit_jobs_args | |
| 145 | + | |
| 146 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 147 | + :doc: Flags for SUBMIT_JOB ioctl geometry command. | |
| 148 | + | |
| 149 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 150 | + :doc: Flags for SUBMIT_JOB ioctl fragment command. | |
| 151 | + | |
| 152 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 153 | + :doc: Flags for SUBMIT_JOB ioctl compute command. | |
| 154 | + | |
| 155 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 156 | + :doc: Flags for SUBMIT_JOB ioctl transfer command. | |
| 157 | + | |
| 158 | +.. kernel-doc:: include/uapi/drm/pvr_drm.h | |
| 159 | + :identifiers: drm_pvr_sync_op | |
| 160 | + drm_pvr_job_type | |
| 161 | + drm_pvr_hwrt_data_ref | |
| 162 | + drm_pvr_job | |
| 163 | + | |
| 164 | +Internal notes | |
| 165 | +============== | |
| 166 | +.. kernel-doc:: drivers/gpu/drm/imagination/pvr_device.h | |
| 167 | + :doc: IOCTL validation helpers | |
| 168 | + | |
| 169 | +.. kernel-doc:: drivers/gpu/drm/imagination/pvr_device.h | |
| 170 | + :identifiers: PVR_STATIC_ASSERT_64BIT_ALIGNED PVR_IOCTL_UNION_PADDING_CHECK | |
| 171 | + pvr_ioctl_union_padding_check | |
| @@ -7,3 +7,4 @@ Misc DRM driver uAPI- and feature implementation guidelines · 6 unmodified lines | ||
| 7 | 7 | .. toctree:: |
| 8 | 8 | |
| 9 | 9 | drm-vm-bind-async |
| 10 | + drm-vm-bind-locking | |
| @@ -70,35 +70,42 @@ When the time comes for Xe, the protection will be lifted on Xe and kept in i915 · 69 unmodified lines | ||
| 70 | 70 | Xe – Pre-Merge Goals - Work-in-Progress |
| 71 | 71 | ======================================= |
| 72 | 72 | |
| 73 | −Drm_scheduler | |
| 74 | −------------- | |
| 75 | −Xe primarily uses Firmware based scheduling (GuC FW). However, it will use | |
| 76 | −drm_scheduler as the scheduler ‘frontend’ for userspace submission in order to | |
| 77 | −resolve syncobj and dma-buf implicit sync dependencies. However, drm_scheduler is | |
| 78 | −not yet prepared to handle the 1-to-1 relationship between drm_gpu_scheduler and | |
| 79 | −drm_sched_entity. | |
| 73 | +Display integration with i915 | |
| 74 | +----------------------------- | |
| 75 | +In order to share the display code with the i915 driver so that there is maximum | |
| 76 | +reuse, the i915/display/ code is built twice, once for i915.ko and then for | |
| 77 | +xe.ko. Currently, the i915/display code in Xe tree is polluted with many 'ifdefs' | |
| 78 | +depending on the build target. The goal is to refactor both Xe and i915/display | |
| 79 | +code simultaneously in order to get a clean result before they land upstream, so | |
| 80 | +that display can already be part of the initial pull request towards drm-next. | |
| 80 | 81 | |
| 81 | −Deeper changes to drm_scheduler should *not* be required to get Xe accepted, but | |
| 82 | −some consensus needs to be reached between Xe and other community drivers that | |
| 83 | −could also benefit from this work, for coupling FW based/assisted submission such | |
| 84 | −as the ARM’s new Mali GPU driver, and others. | |
| 82 | +However, display code should not gate the acceptance of Xe in upstream. Xe | |
| 83 | +patches will be refactored in a way that display code can be removed, if needed, | |
| 84 | +from the first pull request of Xe towards drm-next. The expectation is that when | |
| 85 | +both drivers are part of the drm-tip, the introduction of cleaner patches will be | |
| 86 | +easier and speed up. | |
| 85 | 87 | |
| 86 | −As a key measurable result, the patch series introducing Xe itself shall not | |
| 87 | −depend on any other patch touching drm_scheduler itself that was not yet merged | |
| 88 | −through drm-misc. This, by itself, already includes the reach of an agreement for | |
| 89 | −uniform 1 to 1 relationship implementation / usage across drivers. | |
| 88 | +Xe – uAPI high level overview | |
| 89 | +============================= | |
| 90 | 90 | |
| 91 | −ASYNC VM_BIND | |
| 92 | −------------- | |
| 93 | −Although having a common DRM level IOCTL for VM_BIND is not a requirement to get | |
| 94 | −Xe merged, it is mandatory to have a consensus with other drivers and Mesa. | |
| 95 | −It needs to be clear how to handle async VM_BIND and interactions with userspace | |
| 96 | −memory fences. Ideally with helper support so people don't get it wrong in all | |
| 97 | −possible ways. | |
| 91 | +...Warning: To be done in follow up patches after/when/where the main consensus in various items are individually reached. | |
| 98 | 92 | |
| 99 | −As a key measurable result, the benefits of ASYNC VM_BIND and a discussion of | |
| 100 | −various flavors, error handling and sample API suggestions are documented in | |
| 101 | −:doc:`The ASYNC VM_BIND document </gpu/drm-vm-bind-async>`. | |
| 93 | +Xe – Pre-Merge Goals - Completed | |
| 94 | +================================ | |
| 95 | + | |
| 96 | +Drm_exec | |
| 97 | +-------- | |
| 98 | +Helper to make dma_resv locking for a big number of buffers is getting removed in | |
| 99 | +the drm_exec series proposed in https://patchwork.freedesktop.org/patch/524376/ | |
| 100 | +If that happens, Xe needs to change and incorporate the changes in the driver. | |
| 101 | +The goal is to engage with the Community to understand if the best approach is to | |
| 102 | +move that to the drivers that are using it or if we should keep the helpers in | |
| 103 | +place waiting for Xe to get merged. | |
| 104 | + | |
| 105 | +This item ties into the GPUVA, VM_BIND, and even long-running compute support. | |
| 106 | + | |
| 107 | +As a key measurable result, we need to have a community consensus documented in | |
| 108 | +this document and the Xe driver prepared for the changes, if necessary. | |
| 102 | 109 | |
| 103 | 110 | Userptr integration and vm_bind |
| 104 | 111 | ------------------------------- |
| @@ -123,10 +130,45 @@ Documentation should include: · 18 unmodified lines | ||
| 123 | 130 | |
| 124 | 131 | * O(1) complexity under VM_BIND. |
| 125 | 132 | |
| 133 | +The document is now included in the drm documentation :doc:`here </gpu/drm-vm-bind-async>`. | |
| 134 | + | |
| 126 | 135 | Some parts of userptr like mmu_notifiers should become GPUVA or DRM helpers when |
| 127 | 136 | the second driver supporting VM_BIND+userptr appears. Details to be defined when |
| 128 | 137 | the time comes. |
| 129 | 138 | |
| 139 | +The DRM GPUVM helpers do not yet include the userptr parts, but discussions | |
| 140 | +about implementing them are ongoing. | |
| 141 | + | |
| 142 | +ASYNC VM_BIND | |
| 143 | +------------- | |
| 144 | +Although having a common DRM level IOCTL for VM_BIND is not a requirement to get | |
| 145 | +Xe merged, it is mandatory to have a consensus with other drivers and Mesa. | |
| 146 | +It needs to be clear how to handle async VM_BIND and interactions with userspace | |
| 147 | +memory fences. Ideally with helper support so people don't get it wrong in all | |
| 148 | +possible ways. | |
| 149 | + | |
| 150 | +As a key measurable result, the benefits of ASYNC VM_BIND and a discussion of | |
| 151 | +various flavors, error handling and sample API suggestions are documented in | |
| 152 | +:doc:`The ASYNC VM_BIND document </gpu/drm-vm-bind-async>`. | |
| 153 | + | |
| 154 | +Drm_scheduler | |
| 155 | +------------- | |
| 156 | +Xe primarily uses Firmware based scheduling (GuC FW). However, it will use | |
| 157 | +drm_scheduler as the scheduler ‘frontend’ for userspace submission in order to | |
| 158 | +resolve syncobj and dma-buf implicit sync dependencies. However, drm_scheduler is | |
| 159 | +not yet prepared to handle the 1-to-1 relationship between drm_gpu_scheduler and | |
| 160 | +drm_sched_entity. | |
| 161 | + | |
| 162 | +Deeper changes to drm_scheduler should *not* be required to get Xe accepted, but | |
| 163 | +some consensus needs to be reached between Xe and other community drivers that | |
| 164 | +could also benefit from this work, for coupling FW based/assisted submission such | |
| 165 | +as the ARM’s new Mali GPU driver, and others. | |
| 166 | + | |
| 167 | +As a key measurable result, the patch series introducing Xe itself shall not | |
| 168 | +depend on any other patch touching drm_scheduler itself that was not yet merged | |
| 169 | +through drm-misc. This, by itself, already includes the reach of an agreement for | |
| 170 | +uniform 1 to 1 relationship implementation / usage across drivers. | |
| 171 | + | |
| 130 | 172 | Long running compute: minimal data structure/scaffolding |
| 131 | 173 | -------------------------------------------------------- |
| 132 | 174 | The generic scheduler code needs to include the handling of endless compute |
| @@ -139,46 +181,6 @@ this minimal drm/scheduler work, if needed, merged to drm-misc in a way that any · 6 unmodified lines | ||
| 139 | 181 | drm driver, including Xe, could re-use and add their own individual needs on top |
| 140 | 182 | in a next stage. However, this should not block the initial merge. |
| 141 | 183 | |
| 142 | −This is a non-blocker item since the driver without the support for the long | |
| 143 | −running compute enabled is not a showstopper. | |
| 144 | − | |
| 145 | −Display integration with i915 | |
| 146 | −----------------------------- | |
| 147 | −In order to share the display code with the i915 driver so that there is maximum | |
| 148 | −reuse, the i915/display/ code is built twice, once for i915.ko and then for | |
| 149 | −xe.ko. Currently, the i915/display code in Xe tree is polluted with many 'ifdefs' | |
| 150 | −depending on the build target. The goal is to refactor both Xe and i915/display | |
| 151 | −code simultaneously in order to get a clean result before they land upstream, so | |
| 152 | −that display can already be part of the initial pull request towards drm-next. | |
| 153 | − | |
| 154 | −However, display code should not gate the acceptance of Xe in upstream. Xe | |
| 155 | −patches will be refactored in a way that display code can be removed, if needed, | |
| 156 | −from the first pull request of Xe towards drm-next. The expectation is that when | |
| 157 | −both drivers are part of the drm-tip, the introduction of cleaner patches will be | |
| 158 | −easier and speed up. | |
| 159 | − | |
| 160 | −Drm_exec | |
| 161 | −-------- | |
| 162 | −Helper to make dma_resv locking for a big number of buffers is getting removed in | |
| 163 | −the drm_exec series proposed in https://patchwork.freedesktop.org/patch/524376/ | |
| 164 | −If that happens, Xe needs to change and incorporate the changes in the driver. | |
| 165 | −The goal is to engage with the Community to understand if the best approach is to | |
| 166 | −move that to the drivers that are using it or if we should keep the helpers in | |
| 167 | −place waiting for Xe to get merged. | |
| 168 | − | |
| 169 | −This item ties into the GPUVA, VM_BIND, and even long-running compute support. | |
| 170 | − | |
| 171 | −As a key measurable result, we need to have a community consensus documented in | |
| 172 | −this document and the Xe driver prepared for the changes, if necessary. | |
| 173 | − | |
| 174 | −Xe – uAPI high level overview | |
| 175 | −============================= | |
| 176 | − | |
| 177 | −...Warning: To be done in follow up patches after/when/where the main consensus in various items are individually reached. | |
| 178 | − | |
| 179 | −Xe – Pre-Merge Goals - Completed | |
| 180 | −================================ | |
| 181 | − | |
| 182 | 184 | Dev_coredump |
| 183 | 185 | ------------ |
| 184 | 186 | |
| @@ -337,8 +337,8 @@ connector register/unregister fixes · 336 unmodified lines | ||
| 337 | 337 | |
| 338 | 338 | Level: Intermediate |
| 339 | 339 | |
| 340 | −Remove load/unload callbacks from all non-DRIVER_LEGACY drivers | |
| 341 | −--------------------------------------------------------------- | |
| 340 | +Remove load/unload callbacks | |
| 341 | +---------------------------- | |
| 342 | 342 | |
| 343 | 343 | The load/unload callbacks in struct &drm_driver are very much midlayers, plus |
| 344 | 344 | for historical reasons they get the ordering wrong (and we can't fix that) |
| @@ -347,8 +347,7 @@ between setting up the &drm_driver structure and calling drm_dev_register(). · 2 unmodified lines | ||
| 347 | 347 | - Rework drivers to no longer use the load/unload callbacks, directly coding the |
| 348 | 348 | load/unload sequence into the driver's probe function. |
| 349 | 349 | |
| 350 | −- Once all non-DRIVER_LEGACY drivers are converted, disallow the load/unload | |
| 351 | − callbacks for all modern drivers. | |
| 350 | +- Once all drivers are converted, remove the load/unload callbacks. | |
| 352 | 351 | |
| 353 | 352 | Contact: Daniel Vetter |
| 354 | 353 | |
| @@ -621,6 +620,23 @@ Contact: Javier Martinez Canillas <javierm@redhat.com> · 266 unmodified lines | ||
| 621 | 620 | |
| 622 | 621 | Level: Intermediate |
| 623 | 622 | |
| 623 | +Clean up and document former selftests suites | |
| 624 | +--------------------------------------------- | |
| 625 | + | |
| 626 | +Some KUnit test suites (drm_buddy, drm_cmdline_parser, drm_damage_helper, | |
| 627 | +drm_format, drm_framebuffer, drm_dp_mst_helper, drm_mm, drm_plane_helper and | |
| 628 | +drm_rect) are former selftests suites that have been converted over when KUnit | |
| 629 | +was first introduced. | |
| 630 | + | |
| 631 | +These suites were fairly undocumented, and with different goals than what unit | |
| 632 | +tests can be. Trying to identify what each test in these suites actually test | |
| 633 | +for, whether that makes sense for a unit test, and either remove it if it | |
| 634 | +doesn't or document it if it does would be of great help. | |
| 635 | + | |
| 636 | +Contact: Maxime Ripard <mripard@kernel.org> | |
| 637 | + | |
| 638 | +Level: Intermediate | |
| 639 | + | |
| 624 | 640 | Enable trinity for DRM |
| 625 | 641 | ---------------------- |
| 626 | 642 | |
| @@ -765,6 +781,29 @@ Contact: Hans de Goede · 138 unmodified lines | ||
| 765 | 781 | |
| 766 | 782 | Level: Advanced |
| 767 | 783 | |
| 784 | +Buffer age or other damage accumulation algorithm for buffer damage | |
| 785 | +=================================================================== | |
| 786 | + | |
| 787 | +Drivers that do per-buffer uploads, need a buffer damage handling (rather than | |
| 788 | +frame damage like drivers that do per-plane or per-CRTC uploads), but there is | |
| 789 | +no support to get the buffer age or any other damage accumulation algorithm. | |
| 790 | + | |
| 791 | +For this reason, the damage helpers just fallback to a full plane update if the | |
| 792 | +framebuffer attached to a plane has changed since the last page-flip. Drivers | |
| 793 | +set &drm_plane_state.ignore_damage_clips to true as indication to | |
| 794 | +drm_atomic_helper_damage_iter_init() and drm_atomic_helper_damage_iter_next() | |
| 795 | +helpers that the damage clips should be ignored. | |
| 796 | + | |
| 797 | +This should be improved to get damage tracking properly working on drivers that | |
| 798 | +do per-buffer uploads. | |
| 799 | + | |
| 800 | +More information about damage tracking and references to learning materials can | |
| 801 | +be found in :ref:`damage_tracking_properties`. | |
| 802 | + | |
| 803 | +Contact: Javier Martinez Canillas <javierm@redhat.com> | |
| 804 | + | |
| 805 | +Level: Advanced | |
| 806 | + | |
| 768 | 807 | Outside DRM |
| 769 | 808 | =========== |
| 770 | 809 | |
| @@ -193,9 +193,23 @@ Review timelines · 192 unmodified lines | ||
| 193 | 193 | Generally speaking, the patches get triaged quickly (in less than |
| 194 | 194 | 48h). But be patient, if your patch is active in patchwork (i.e. it's |
| 195 | 195 | listed on the project's patch list) the chances it was missed are close to zero. |
| 196 | −Asking the maintainer for status updates on your | |
| 197 | −patch is a good way to ensure your patch is ignored or pushed to the | |
| 198 | −bottom of the priority list. | |
| 196 | + | |
| 197 | +The high volume of development on netdev makes reviewers move on | |
| 198 | +from discussions relatively quickly. New comments and replies | |
| 199 | +are very unlikely to arrive after a week of silence. If a patch | |
| 200 | +is no longer active in patchwork and the thread went idle for more | |
| 201 | +than a week - clarify the next steps and/or post the next version. | |
| 202 | + | |
| 203 | +For RFC postings specifically, if nobody responded in a week - reviewers | |
| 204 | +either missed the posting or have no strong opinions. If the code is ready, | |
| 205 | +repost as a PATCH. | |
| 206 | + | |
| 207 | +Emails saying just "ping" or "bump" are considered rude. If you can't figure | |
| 208 | +out the status of the patch from patchwork or where the discussion has | |
| 209 | +landed - describe your best guess and ask if it's correct. For example:: | |
| 210 | + | |
| 211 | + I don't understand what the next steps are. Person X seems to be unhappy | |
| 212 | + with A, should I do B and repost the patches? | |
| 199 | 213 | |
| 200 | 214 | .. _Changes requested: |
| 201 | 215 | |
| @@ -338,9 +338,9 @@ Loongson与LoongArch的开发者网站（软件与文档资源）： · 337 unmodified lines | ||
| 338 | 338 | |
| 339 | 339 | LoongArch指令集架构的文档： |
| 340 | 340 | |
| 341 | − https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.02-CN.pdf （中文版） | |
| 341 | + https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.10-CN.pdf （中文版） | |
| 342 | 342 | |
| 343 | − https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.02-EN.pdf （英文版） | |
| 343 | + https://github.com/loongson/LoongArch-Documentation/releases/latest/download/LoongArch-Vol1-v1.10-EN.pdf （英文版） | |
| 344 | 344 | |
| 345 | 345 | LoongArch的ELF psABI文档： |
| 346 | 346 | |
| @@ -6503,8 +6503,7 @@ T: git git://anongit.freedesktop.org/drm/drm-misc · 6502 unmodified lines | ||
| 6503 | 6503 | F: drivers/gpu/drm/sun4i/sun8i* |
| 6504 | 6504 | |
| 6505 | 6505 | DRM DRIVER FOR ARM PL111 CLCD |
| 6506 | −M: Emma Anholt <emma@anholt.net> | |
| 6507 | −S: Supported | |
| 6506 | +S: Orphan | |
| 6508 | 6507 | T: git git://anongit.freedesktop.org/drm/drm-misc |
| 6509 | 6508 | F: drivers/gpu/drm/pl111/ |
| 6510 | 6509 | |
| @@ -6619,8 +6618,7 @@ F: Documentation/devicetree/bindings/display/panel/himax,hx8394.yaml · 108 unmodified lines | ||
| 6619 | 6618 | F: drivers/gpu/drm/panel/panel-himax-hx8394.c |
| 6620 | 6619 | |
| 6621 | 6620 | DRM DRIVER FOR HX8357D PANELS |
| 6622 | −M: Emma Anholt <emma@anholt.net> | |
| 6623 | −S: Maintained | |
| 6621 | +S: Orphan | |
| 6624 | 6622 | T: git git://anongit.freedesktop.org/drm/drm-misc |
| 6625 | 6623 | F: Documentation/devicetree/bindings/display/himax,hx8357d.txt |
| 6626 | 6624 | F: drivers/gpu/drm/tiny/hx8357d.c |
| @@ -7213,8 +7211,8 @@ F: Documentation/devicetree/bindings/display/ti/ · 586 unmodified lines | ||
| 7213 | 7211 | F: drivers/gpu/drm/omapdrm/ |
| 7214 | 7212 | |
| 7215 | 7213 | DRM DRIVERS FOR V3D |
| 7216 | −M: Emma Anholt <emma@anholt.net> | |
| 7217 | 7214 | M: Melissa Wen <mwen@igalia.com> |
| 7215 | +M: Maíra Canal <mcanal@igalia.com> | |
| 7218 | 7216 | S: Supported |
| 7219 | 7217 | T: git git://anongit.freedesktop.org/drm/drm-misc |
| 7220 | 7218 | F: Documentation/devicetree/bindings/gpu/brcm,bcm-v3d.yaml |
| @@ -7222,7 +7220,6 @@ F: drivers/gpu/drm/v3d/ · 1 unmodified line | ||
| 7222 | 7220 | F: include/uapi/drm/v3d_drm.h |
| 7223 | 7221 | |
| 7224 | 7222 | DRM DRIVERS FOR VC4 |
| 7225 | −M: Emma Anholt <emma@anholt.net> | |
| 7226 | 7223 | M: Maxime Ripard <mripard@kernel.org> |
| 7227 | 7224 | S: Supported |
| 7228 | 7225 | T: git git://github.com/anholt/linux |
| @@ -7855,6 +7852,7 @@ R: Yue Hu <huyue2@coolpad.com> · 626 unmodified lines | ||
| 7855 | 7852 | R: Jeffle Xu <jefflexu@linux.alibaba.com> |
| 7856 | 7853 | L: linux-erofs@lists.ozlabs.org |
| 7857 | 7854 | S: Maintained |
| 7855 | +W: https://erofs.docs.kernel.org | |
| 7858 | 7856 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs.git |
| 7859 | 7857 | F: Documentation/ABI/testing/sysfs-fs-erofs |
| 7860 | 7858 | F: Documentation/filesystems/erofs.rst |
| @@ -8950,7 +8948,6 @@ S: Maintained · 1089 unmodified lines | ||
| 8950 | 8948 | F: scripts/get_maintainer.pl |
| 8951 | 8949 | |
| 8952 | 8950 | GFS2 FILE SYSTEM |
| 8953 | −M: Bob Peterson <rpeterso@redhat.com> | |
| 8954 | 8951 | M: Andreas Gruenbacher <agruenba@redhat.com> |
| 8955 | 8952 | L: gfs2@lists.linux.dev |
| 8956 | 8953 | S: Supported |
| @@ -10395,6 +10392,17 @@ IMGTEC IR DECODER DRIVER · 1438 unmodified lines | ||
| 10395 | 10392 | S: Orphan |
| 10396 | 10393 | F: drivers/media/rc/img-ir/ |
| 10397 | 10394 | |
| 10395 | +IMGTEC POWERVR DRM DRIVER | |
| 10396 | +M: Frank Binns <frank.binns@imgtec.com> | |
| 10397 | +M: Donald Robson <donald.robson@imgtec.com> | |
| 10398 | +M: Matt Coster <matt.coster@imgtec.com> | |
| 10399 | +S: Supported | |
| 10400 | +T: git git://anongit.freedesktop.org/drm/drm-misc | |
| 10401 | +F: Documentation/devicetree/bindings/gpu/img,powervr.yaml | |
| 10402 | +F: Documentation/gpu/imagination/ | |
| 10403 | +F: drivers/gpu/drm/imagination/ | |
| 10404 | +F: include/uapi/drm/pvr_drm.h | |
| 10405 | + | |
| 10398 | 10406 | IMON SOUNDGRAPH USB IR RECEIVER |
| 10399 | 10407 | M: Sean Young <sean@mess.org> |
| 10400 | 10408 | L: linux-media@vger.kernel.org |
| @@ -10639,9 +10647,9 @@ M: Rodrigo Vivi <rodrigo.vivi@intel.com> · 238 unmodified lines | ||
| 10639 | 10647 | M: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com> |
| 10640 | 10648 | L: intel-gfx@lists.freedesktop.org |
| 10641 | 10649 | S: Supported |
| 10642 | −W: https://01.org/linuxgraphics/ | |
| 10650 | +W: https://drm.pages.freedesktop.org/intel-docs/ | |
| 10643 | 10651 | Q: http://patchwork.freedesktop.org/project/intel-gfx/ |
| 10644 | −B: https://gitlab.freedesktop.org/drm/intel/-/wikis/How-to-file-i915-bugs | |
| 10652 | +B: https://drm.pages.freedesktop.org/intel-docs/how-to-file-i915-bugs.html | |
| 10645 | 10653 | C: irc://irc.oftc.net/intel-gfx |
| 10646 | 10654 | T: git git://anongit.freedesktop.org/drm-intel |
| 10647 | 10655 | F: Documentation/ABI/testing/sysfs-driver-intel-i915-hwmon |
| @@ -11025,7 +11033,6 @@ F: drivers/net/wireless/intel/iwlwifi/ · 377 unmodified lines | ||
| 11025 | 11033 | |
| 11026 | 11034 | INTEL WMI SLIM BOOTLOADER (SBL) FIRMWARE UPDATE DRIVER |
| 11027 | 11035 | M: Jithu Joseph <jithu.joseph@intel.com> |
| 11028 | −R: Maurice Ma <maurice.ma@intel.com> | |
| 11029 | 11036 | S: Maintained |
| 11030 | 11037 | W: https://slimbootloader.github.io/security/firmware-update.html |
| 11031 | 11038 | F: drivers/platform/x86/intel/wmi/sbl-fw-update.c |
| @@ -13779,7 +13786,6 @@ F: drivers/net/ethernet/mellanox/mlxfw/ · 2747 unmodified lines | ||
| 13779 | 13786 | MELLANOX HARDWARE PLATFORM SUPPORT |
| 13780 | 13787 | M: Hans de Goede <hdegoede@redhat.com> |
| 13781 | 13788 | M: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> |
| 13782 | −M: Mark Gross <markgross@kernel.org> | |
| 13783 | 13789 | M: Vadim Pasternak <vadimp@nvidia.com> |
| 13784 | 13790 | L: platform-driver-x86@vger.kernel.org |
| 13785 | 13791 | S: Supported |
| @@ -14388,7 +14394,6 @@ F: drivers/platform/surface/surface_gpe.c · 602 unmodified lines | ||
| 14388 | 14394 | MICROSOFT SURFACE HARDWARE PLATFORM SUPPORT |
| 14389 | 14395 | M: Hans de Goede <hdegoede@redhat.com> |
| 14390 | 14396 | M: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> |
| 14391 | −M: Mark Gross <markgross@kernel.org> | |
| 14392 | 14397 | M: Maximilian Luz <luzmaximilian@gmail.com> |
| 14393 | 14398 | L: platform-driver-x86@vger.kernel.org |
| 14394 | 14399 | S: Maintained |
| @@ -14995,6 +15000,7 @@ M: Jakub Kicinski <kuba@kernel.org> · 600 unmodified lines | ||
| 14995 | 15000 | M: Paolo Abeni <pabeni@redhat.com> |
| 14996 | 15001 | L: netdev@vger.kernel.org |
| 14997 | 15002 | S: Maintained |
| 15003 | +P: Documentation/process/maintainer-netdev.rst | |
| 14998 | 15004 | Q: https://patchwork.kernel.org/project/netdevbpf/list/ |
| 14999 | 15005 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git |
| 15000 | 15006 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git |
| @@ -15046,6 +15052,7 @@ M: Jakub Kicinski <kuba@kernel.org> · 45 unmodified lines | ||
| 15046 | 15052 | M: Paolo Abeni <pabeni@redhat.com> |
| 15047 | 15053 | L: netdev@vger.kernel.org |
| 15048 | 15054 | S: Maintained |
| 15055 | +P: Documentation/process/maintainer-netdev.rst | |
| 15049 | 15056 | Q: https://patchwork.kernel.org/project/netdevbpf/list/ |
| 15050 | 15057 | B: mailto:netdev@vger.kernel.org |
| 15051 | 15058 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git |
| @@ -15056,6 +15063,7 @@ F: Documentation/networking/ · 4 unmodified lines | ||
| 15056 | 15063 | F: Documentation/process/maintainer-netdev.rst |
| 15057 | 15064 | F: Documentation/userspace-api/netlink/ |
| 15058 | 15065 | F: include/linux/in.h |
| 15066 | +F: include/linux/indirect_call_wrapper.h | |
| 15059 | 15067 | F: include/linux/net.h |
| 15060 | 15068 | F: include/linux/netdevice.h |
| 15061 | 15069 | F: include/net/ |
| @@ -21769,7 +21777,9 @@ F: Documentation/devicetree/bindings/counter/ti-eqep.yaml · 6707 unmodified lines | ||
| 21769 | 21777 | F: drivers/counter/ti-eqep.c |
| 21770 | 21778 | |
| 21771 | 21779 | TI ETHERNET SWITCH DRIVER (CPSW) |
| 21772 | −R: Grygorii Strashko <grygorii.strashko@ti.com> | |
| 21780 | +R: Siddharth Vadapalli <s-vadapalli@ti.com> | |
| 21781 | +R: Ravi Gunasekaran <r-gunasekaran@ti.com> | |
| 21782 | +R: Roger Quadros <rogerq@kernel.org> | |
| 21773 | 21783 | L: linux-omap@vger.kernel.org |
| 21774 | 21784 | L: netdev@vger.kernel.org |
| 21775 | 21785 | S: Maintained |
| @@ -21793,6 +21803,15 @@ F: Documentation/devicetree/bindings/media/i2c/ti,ds90* · 17 unmodified lines | ||
| 21793 | 21803 | F: drivers/media/i2c/ds90* |
| 21794 | 21804 | F: include/media/i2c/ds90* |
| 21795 | 21805 | |
| 21806 | +TI ICSSG ETHERNET DRIVER (ICSSG) | |
| 21807 | +R: MD Danish Anwar <danishanwar@ti.com> | |
| 21808 | +R: Roger Quadros <rogerq@kernel.org> | |
| 21809 | +L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) | |
| 21810 | +L: netdev@vger.kernel.org | |
| 21811 | +S: Maintained | |
| 21812 | +F: Documentation/devicetree/bindings/net/ti,icss*.yaml | |
| 21813 | +F: drivers/net/ethernet/ti/icssg/* | |
| 21814 | + | |
| 21796 | 21815 | TI J721E CSI2RX DRIVER |
| 21797 | 21816 | M: Jai Luthra <j-luthra@ti.com> |
| 21798 | 21817 | L: linux-media@vger.kernel.org |
| @@ -22068,6 +22087,7 @@ F: drivers/watchdog/tqmx86_wdt.c · 269 unmodified lines | ||
| 22068 | 22087 | TRACING |
| 22069 | 22088 | M: Steven Rostedt <rostedt@goodmis.org> |
| 22070 | 22089 | M: Masami Hiramatsu <mhiramat@kernel.org> |
| 22090 | +R: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> | |
| 22071 | 22091 | L: linux-kernel@vger.kernel.org |
| 22072 | 22092 | L: linux-trace-kernel@vger.kernel.org |
| 22073 | 22093 | S: Maintained |
| @@ -23654,7 +23674,6 @@ F: drivers/platform/x86/x86-android-tablets/ · 1580 unmodified lines | ||
| 23654 | 23674 | X86 PLATFORM DRIVERS |
| 23655 | 23675 | M: Hans de Goede <hdegoede@redhat.com> |
| 23656 | 23676 | M: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> |
| 23657 | −M: Mark Gross <markgross@kernel.org> | |
| 23658 | 23677 | L: platform-driver-x86@vger.kernel.org |
| 23659 | 23678 | S: Maintained |
| 23660 | 23679 | Q: https://patchwork.kernel.org/project/platform-driver-x86/list/ |
| @@ -23692,6 +23711,20 @@ F: arch/x86/kernel/dumpstack.c · 31 unmodified lines | ||
| 23692 | 23711 | F: arch/x86/kernel/stacktrace.c |
| 23693 | 23712 | F: arch/x86/kernel/unwind_*.c |
| 23694 | 23713 | |
| 23714 | +X86 TRUST DOMAIN EXTENSIONS (TDX) | |
| 23715 | +M: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> | |
| 23716 | +R: Dave Hansen <dave.hansen@linux.intel.com> | |
| 23717 | +L: x86@kernel.org | |
| 23718 | +L: linux-coco@lists.linux.dev | |
| 23719 | +S: Supported | |
| 23720 | +T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86/tdx | |
| 23721 | +F: arch/x86/boot/compressed/tdx* | |
| 23722 | +F: arch/x86/coco/tdx/ | |
| 23723 | +F: arch/x86/include/asm/shared/tdx.h | |
| 23724 | +F: arch/x86/include/asm/tdx.h | |
| 23725 | +F: arch/x86/virt/vmx/tdx/ | |
| 23726 | +F: drivers/virt/coco/tdx-guest | |
| 23727 | + | |
| 23695 | 23728 | X86 VDSO |
| 23696 | 23729 | M: Andy Lutomirski <luto@kernel.org> |
| 23697 | 23730 | L: linux-kernel@vger.kernel.org |
| @@ -23872,8 +23905,7 @@ T: git git://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git · 174 unmodified lines | ||
| 23872 | 23905 | P: Documentation/filesystems/xfs-maintainer-entry-profile.rst |
| 23873 | 23906 | F: Documentation/ABI/testing/sysfs-fs-xfs |
| 23874 | 23907 | F: Documentation/admin-guide/xfs.rst |
| 23875 | −F: Documentation/filesystems/xfs-delayed-logging-design.rst | |
| 23876 | −F: Documentation/filesystems/xfs-self-describing-metadata.rst | |
| 23908 | +F: Documentation/filesystems/xfs-* | |
| 23877 | 23909 | F: fs/xfs/ |
| 23878 | 23910 | F: include/uapi/linux/dqblk_xfs.h |
| 23879 | 23911 | F: include/uapi/linux/fsmap.h |
| @@ -2,7 +2,7 @@ · 1 unmodified line | ||
| 2 | 2 | VERSION = 6 |
| 3 | 3 | PATCHLEVEL = 7 |
| 4 | 4 | SUBLEVEL = 0 |
| 5 | −EXTRAVERSION = -rc1 | |
| 5 | +EXTRAVERSION = -rc3 | |
| 6 | 6 | NAME = Hurr durr I'ma ninja sloth |
| 7 | 7 | |
| 8 | 8 | # *DOCUMENTATION* |
| @@ -484,7 +484,8 @@ static int __init xen_guest_init(void) · 483 unmodified lines | ||
| 484 | 484 | * for secondary CPUs as they are brought up. |
| 485 | 485 | * For uniformity we use VCPUOP_register_vcpu_info even on cpu0. |
| 486 | 486 | */ |
| 487 | − xen_vcpu_info = alloc_percpu(struct vcpu_info); | |
| 487 | + xen_vcpu_info = __alloc_percpu(sizeof(struct vcpu_info), | |
| 488 | + 1 << fls(sizeof(struct vcpu_info) - 1)); | |
| 488 | 489 | if (xen_vcpu_info == NULL) |
| 489 | 490 | return -ENOMEM; |
| 490 | 491 | |
| @@ -158,7 +158,7 @@ endif · 157 unmodified lines | ||
| 158 | 158 | |
| 159 | 159 | all: $(notdir $(KBUILD_IMAGE)) |
| 160 | 160 | |
| 161 | − | |
| 161 | +vmlinuz.efi: Image | |
| 162 | 162 | Image vmlinuz.efi: vmlinux |
| 163 | 163 | $(Q)$(MAKE) $(build)=$(boot) $(boot)/$@ |
| 164 | 164 | |
| @@ -21,9 +21,22 @@ static inline bool arch_parse_debug_rodata(char *arg) · 20 unmodified lines | ||
| 21 | 21 | extern bool rodata_enabled; |
| 22 | 22 | extern bool rodata_full; |
| 23 | 23 | |
| 24 | − if (arg && !strcmp(arg, "full")) { | |
| 24 | + if (!arg) | |
| 25 | + return false; | |
| 26 | + | |
| 27 | + if (!strcmp(arg, "full")) { | |
| 28 | + rodata_enabled = rodata_full = true; | |
| 29 | + return true; | |
| 30 | + } | |
| 31 | + | |
| 32 | + if (!strcmp(arg, "off")) { | |
| 33 | + rodata_enabled = rodata_full = false; | |
| 34 | + return true; | |
| 35 | + } | |
| 36 | + | |
| 37 | + if (!strcmp(arg, "on")) { | |
| 25 | 38 | rodata_enabled = true; |
| 26 | − rodata_full = true; | |
| 39 | + rodata_full = false; | |
| 27 | 40 | return true; |
| 28 | 41 | } |
| 29 | 42 | |
| @@ -29,8 +29,8 @@ bool can_set_direct_map(void) · 28 unmodified lines | ||
| 29 | 29 | * |
| 30 | 30 | * KFENCE pool requires page-granular mapping if initialized late. |
| 31 | 31 | */ |
| 32 | − return (rodata_enabled && rodata_full) || debug_pagealloc_enabled() || | |
| 33 | − arm64_kfence_can_set_direct_map(); | |
| 32 | + return rodata_full || debug_pagealloc_enabled() || | |
| 33 | + arm64_kfence_can_set_direct_map(); | |
| 34 | 34 | } |
| 35 | 35 | |
| 36 | 36 | static int change_page_range(pte_t *ptep, unsigned long addr, void *data) |
| @@ -105,8 +105,7 @@ static int change_memory_common(unsigned long addr, int numpages, · 68 unmodified lines | ||
| 105 | 105 | * If we are manipulating read-only permissions, apply the same |
| 106 | 106 | * change to the linear mapping of the pages that back this VM area. |
| 107 | 107 | */ |
| 108 | − if (rodata_enabled && | |
| 109 | − rodata_full && (pgprot_val(set_mask) == PTE_RDONLY || | |
| 108 | + if (rodata_full && (pgprot_val(set_mask) == PTE_RDONLY || | |
| 110 | 109 | pgprot_val(clear_mask) == PTE_RDONLY)) { |
| 111 | 110 | for (i = 0; i < area->nr_pages; i++) { |
| 112 | 111 | __change_memory_common((u64)page_address(area->pages[i]), |
| @@ -68,6 +68,7 @@ LDFLAGS_vmlinux += -static -n -nostdlib · 67 unmodified lines | ||
| 68 | 68 | ifdef CONFIG_AS_HAS_EXPLICIT_RELOCS |
| 69 | 69 | cflags-y += $(call cc-option,-mexplicit-relocs) |
| 70 | 70 | KBUILD_CFLAGS_KERNEL += $(call cc-option,-mdirect-extern-access) |
| 71 | +KBUILD_CFLAGS_KERNEL += $(call cc-option,-fdirect-access-external-data) | |
| 71 | 72 | KBUILD_AFLAGS_MODULE += $(call cc-option,-fno-direct-access-external-data) |
| 72 | 73 | KBUILD_CFLAGS_MODULE += $(call cc-option,-fno-direct-access-external-data) |
| 73 | 74 | KBUILD_AFLAGS_MODULE += $(call cc-option,-mno-relax) $(call cc-option,-Wa$(comma)-mno-relax) |
| @@ -142,6 +143,8 @@ vdso-install-y += arch/loongarch/vdso/vdso.so.dbg · 68 unmodified lines | ||
| 142 | 143 | |
| 143 | 144 | all: $(notdir $(KBUILD_IMAGE)) |
| 144 | 145 | |
| 146 | +vmlinuz.efi: vmlinux.efi | |
| 147 | + | |
| 145 | 148 | vmlinux.elf vmlinux.efi vmlinuz.efi: vmlinux |
| 146 | 149 | $(Q)$(MAKE) $(build)=$(boot) $(bootvars-y) $(boot)/$@ |
| 147 | 150 | |
| @@ -609,8 +609,7 @@ · 608 unmodified lines | ||
| 609 | 609 | lu32i.d \reg, 0 |
| 610 | 610 | lu52i.d \reg, \reg, 0 |
| 611 | 611 | .pushsection ".la_abs", "aw", %progbits |
| 612 | − 768: | |
| 613 | − .dword 768b-766b | |
| 612 | + .dword 766b | |
| 614 | 613 | .dword \sym |
| 615 | 614 | .popsection |
| 616 | 615 | #endif |
| @@ -40,13 +40,13 @@ static __always_inline unsigned long __percpu_##op(void *ptr, \ · 39 unmodified lines | ||
| 40 | 40 | switch (size) { \ |
| 41 | 41 | case 4: \ |
| 42 | 42 | __asm__ __volatile__( \ |
| 43 | − "am"#asm_op".w" " %[ret], %[val], %[ptr] \n" \ | |
| 43 | + "am"#asm_op".w" " %[ret], %[val], %[ptr] \n" \ | |
| 44 | 44 | : [ret] "=&r" (ret), [ptr] "+ZB"(*(u32 *)ptr) \ |
| 45 | 45 | : [val] "r" (val)); \ |
| 46 | 46 | break; \ |
| 47 | 47 | case 8: \ |
| 48 | 48 | __asm__ __volatile__( \ |
| 49 | − "am"#asm_op".d" " %[ret], %[val], %[ptr] \n" \ | |
| 49 | + "am"#asm_op".d" " %[ret], %[val], %[ptr] \n" \ | |
| 50 | 50 | : [ret] "=&r" (ret), [ptr] "+ZB"(*(u64 *)ptr) \ |
| 51 | 51 | : [val] "r" (val)); \ |
| 52 | 52 | break; \ |
| @@ -63,7 +63,7 @@ PERCPU_OP(and, and, &) · 10 unmodified lines | ||
| 63 | 63 | PERCPU_OP(or, or, |) |
| 64 | 64 | #undef PERCPU_OP |
| 65 | 65 | |
| 66 | −static __always_inline unsigned long __percpu_read(void *ptr, int size) | |
| 66 | +static __always_inline unsigned long __percpu_read(void __percpu *ptr, int size) | |
| 67 | 67 | { |
| 68 | 68 | unsigned long ret; |
| 69 | 69 | |
| @@ -100,7 +100,7 @@ static __always_inline unsigned long __percpu_read(void *ptr, int size) · 30 unmodified lines | ||
| 100 | 100 | return ret; |
| 101 | 101 | } |
| 102 | 102 | |
| 103 | −static __always_inline void __percpu_write(void *ptr, unsigned long val, int size) | |
| 103 | +static __always_inline void __percpu_write(void __percpu *ptr, unsigned long val, int size) | |
| 104 | 104 | { |
| 105 | 105 | switch (size) { |
| 106 | 106 | case 1: |
| @@ -132,8 +132,7 @@ static __always_inline void __percpu_write(void *ptr, unsigned long val, int siz · 25 unmodified lines | ||
| 132 | 132 | } |
| 133 | 133 | } |
| 134 | 134 | |
| 135 | −static __always_inline unsigned long __percpu_xchg(void *ptr, unsigned long val, | |
| 136 | − int size) | |
| 135 | +static __always_inline unsigned long __percpu_xchg(void *ptr, unsigned long val, int size) | |
| 137 | 136 | { |
| 138 | 137 | switch (size) { |
| 139 | 138 | case 1: |
| @@ -25,7 +25,7 @@ extern void set_merr_handler(unsigned long offset, void *addr, unsigned long len · 24 unmodified lines | ||
| 25 | 25 | #ifdef CONFIG_RELOCATABLE |
| 26 | 26 | |
| 27 | 27 | struct rela_la_abs { |
| 28 | − long offset; | |
| 28 | + long pc; | |
| 29 | 29 | long symvalue; |
| 30 | 30 | }; |
| 31 | 31 | |
| @@ -52,7 +52,7 @@ static inline void __init relocate_absolute(long random_offset) · 51 unmodified lines | ||
| 52 | 52 | for (p = begin; (void *)p < end; p++) { |
| 53 | 53 | long v = p->symvalue; |
| 54 | 54 | uint32_t lu12iw, ori, lu32id, lu52id; |
| 55 | − union loongarch_instruction *insn = (void *)p - p->offset; | |
| 55 | + union loongarch_instruction *insn = (void *)p->pc; | |
| 56 | 56 | |
| 57 | 57 | lu12iw = (v >> 12) & 0xfffff; |
| 58 | 58 | ori = v & 0xfff; |
| @@ -102,6 +102,14 @@ static inline __init unsigned long get_random_boot(void) · 43 unmodified lines | ||
| 102 | 102 | return hash; |
| 103 | 103 | } |
| 104 | 104 | |
| 105 | +static int __init nokaslr(char *p) | |
| 106 | +{ | |
| 107 | + pr_info("KASLR is disabled.\n"); | |
| 108 | + | |
| 109 | + return 0; /* Print a notice and silence the boot warning */ | |
| 110 | +} | |
| 111 | +early_param("nokaslr", nokaslr); | |
| 112 | + | |
| 105 | 113 | static inline __init bool kaslr_disabled(void) |
| 106 | 114 | { |
| 107 | 115 | char *str; |
| @@ -58,14 +58,16 @@ static int constant_set_state_oneshot(struct clock_event_device *evt) · 57 unmodified lines | ||
| 58 | 58 | return 0; |
| 59 | 59 | } |
| 60 | 60 | |
| 61 | −static int constant_set_state_oneshot_stopped(struct clock_event_device *evt) | |
| 61 | +static int constant_set_state_periodic(struct clock_event_device *evt) | |
| 62 | 62 | { |
| 63 | + unsigned long period; | |
| 63 | 64 | unsigned long timer_config; |
| 64 | 65 | |
| 65 | 66 | raw_spin_lock(&state_lock); |
| 66 | 67 | |
| 67 | − timer_config = csr_read64(LOONGARCH_CSR_TCFG); | |
| 68 | − timer_config &= ~CSR_TCFG_EN; | |
| 68 | + period = const_clock_freq / HZ; | |
| 69 | + timer_config = period & CSR_TCFG_VAL; | |
| 70 | + timer_config |= (CSR_TCFG_PERIOD | CSR_TCFG_EN); | |
| 69 | 71 | csr_write64(timer_config, LOONGARCH_CSR_TCFG); |
| 70 | 72 | |
| 71 | 73 | raw_spin_unlock(&state_lock); |
| @@ -73,16 +75,14 @@ static int constant_set_state_oneshot_stopped(struct clock_event_device *evt) · 1 unmodified line | ||
| 73 | 75 | return 0; |
| 74 | 76 | } |
| 75 | 77 | |
| 76 | −static int constant_set_state_periodic(struct clock_event_device *evt) | |
| 78 | +static int constant_set_state_shutdown(struct clock_event_device *evt) | |
| 77 | 79 | { |
| 78 | − unsigned long period; | |
| 79 | 80 | unsigned long timer_config; |
| 80 | 81 | |
| 81 | 82 | raw_spin_lock(&state_lock); |
| 82 | 83 | |
| 83 | − period = const_clock_freq / HZ; | |
| 84 | − timer_config = period & CSR_TCFG_VAL; | |
| 85 | − timer_config |= (CSR_TCFG_PERIOD | CSR_TCFG_EN); | |
| 84 | + timer_config = csr_read64(LOONGARCH_CSR_TCFG); | |
| 85 | + timer_config &= ~CSR_TCFG_EN; | |
| 86 | 86 | csr_write64(timer_config, LOONGARCH_CSR_TCFG); |
| 87 | 87 | |
| 88 | 88 | raw_spin_unlock(&state_lock); |
| @@ -90,11 +90,6 @@ static int constant_set_state_periodic(struct clock_event_device *evt) · 1 unmodified line | ||
| 90 | 90 | return 0; |
| 91 | 91 | } |
| 92 | 92 | |
| 93 | −static int constant_set_state_shutdown(struct clock_event_device *evt) | |
| 94 | −{ | |
| 95 | − return 0; | |
| 96 | −} | |
| 97 | − | |
| 98 | 93 | static int constant_timer_next_event(unsigned long delta, struct clock_event_device *evt) |
| 99 | 94 | { |
| 100 | 95 | unsigned long timer_config; |
| @@ -161,7 +156,7 @@ int constant_clockevent_init(void) · 60 unmodified lines | ||
| 161 | 156 | cd->rating = 320; |
| 162 | 157 | cd->cpumask = cpumask_of(cpu); |
| 163 | 158 | cd->set_state_oneshot = constant_set_state_oneshot; |
| 164 | − cd->set_state_oneshot_stopped = constant_set_state_oneshot_stopped; | |
| 159 | + cd->set_state_oneshot_stopped = constant_set_state_shutdown; | |
| 165 | 160 | cd->set_state_periodic = constant_set_state_periodic; |
| 166 | 161 | cd->set_state_shutdown = constant_set_state_shutdown; |
| 167 | 162 | cd->set_next_event = constant_timer_next_event; |
| @@ -13,13 +13,13 @@ struct page *dmw_virt_to_page(unsigned long kaddr) · 12 unmodified lines | ||
| 13 | 13 | { |
| 14 | 14 | return pfn_to_page(virt_to_pfn(kaddr)); |
| 15 | 15 | } |
| 16 | −EXPORT_SYMBOL_GPL(dmw_virt_to_page); | |
| 16 | +EXPORT_SYMBOL(dmw_virt_to_page); | |
| 17 | 17 | |
| 18 | 18 | struct page *tlb_virt_to_page(unsigned long kaddr) |
| 19 | 19 | { |
| 20 | 20 | return pfn_to_page(pte_pfn(*virt_to_kpte(kaddr))); |
| 21 | 21 | } |
| 22 | −EXPORT_SYMBOL_GPL(tlb_virt_to_page); | |
| 22 | +EXPORT_SYMBOL(tlb_virt_to_page); | |
| 23 | 23 | |
| 24 | 24 | pgd_t *pgd_alloc(struct mm_struct *mm) |
| 25 | 25 | { |
| @@ -115,9 +115,12 @@ config ARCH_HAS_ILOG2_U64 · 114 unmodified lines | ||
| 115 | 115 | default n |
| 116 | 116 | |
| 117 | 117 | config GENERIC_BUG |
| 118 | − bool | |
| 119 | − default y | |
| 118 | + def_bool y | |
| 120 | 119 | depends on BUG |
| 120 | + select GENERIC_BUG_RELATIVE_POINTERS if 64BIT | |
| 121 | + | |
| 122 | +config GENERIC_BUG_RELATIVE_POINTERS | |
| 123 | + bool | |
| 121 | 124 | |
| 122 | 125 | config GENERIC_HWEIGHT |
| 123 | 126 | bool |
| @@ -140,11 +143,11 @@ config ARCH_MMAP_RND_COMPAT_BITS_MIN · 16 unmodified lines | ||
| 140 | 143 | default 8 |
| 141 | 144 | |
| 142 | 145 | config ARCH_MMAP_RND_BITS_MAX |
| 143 | − default 24 if 64BIT | |
| 144 | − default 17 | |
| 146 | + default 18 if 64BIT | |
| 147 | + default 13 | |
| 145 | 148 | |
| 146 | 149 | config ARCH_MMAP_RND_COMPAT_BITS_MAX |
| 147 | − default 17 | |
| 150 | + default 13 | |
| 148 | 151 | |
| 149 | 152 | # unless you want to implement ACPI on PA-RISC ... ;-) |
| 150 | 153 | config PM |
| @@ -34,7 +34,8 @@ void apply_alternatives(struct alt_instr *start, struct alt_instr *end, · 33 unmodified lines | ||
| 34 | 34 | |
| 35 | 35 | /* Alternative SMP implementation. */ |
| 36 | 36 | #define ALTERNATIVE(cond, replacement) "!0:" \ |
| 37 | − ".section .altinstructions, \"aw\" !" \ | |
| 37 | + ".section .altinstructions, \"a\" !" \ | |
| 38 | + ".align 4 !" \ | |
| 38 | 39 | ".word (0b-4-.) !" \ |
| 39 | 40 | ".hword 1, " __stringify(cond) " !" \ |
| 40 | 41 | ".word " __stringify(replacement) " !" \ |
| @@ -44,7 +45,8 @@ void apply_alternatives(struct alt_instr *start, struct alt_instr *end, · 3 unmodified lines | ||
| 44 | 45 | |
| 45 | 46 | /* to replace one single instructions by a new instruction */ |
| 46 | 47 | #define ALTERNATIVE(from, to, cond, replacement)\ |
| 47 | − .section .altinstructions, "aw" ! \ | |
| 48 | + .section .altinstructions, "a" ! \ | |
| 49 | + .align 4 ! \ | |
| 48 | 50 | .word (from - .) ! \ |
| 49 | 51 | .hword (to - from)/4, cond ! \ |
| 50 | 52 | .word replacement ! \ |
| @@ -52,7 +54,8 @@ void apply_alternatives(struct alt_instr *start, struct alt_instr *end, · 1 unmodified line | ||
| 52 | 54 | |
| 53 | 55 | /* to replace multiple instructions by new code */ |
| 54 | 56 | #define ALTERNATIVE_CODE(from, num_instructions, cond, new_instr_ptr)\ |
| 55 | − .section .altinstructions, "aw" ! \ | |
| 57 | + .section .altinstructions, "a" ! \ | |
| 58 | + .align 4 ! \ | |
| 56 | 59 | .word (from - .) ! \ |
| 57 | 60 | .hword -num_instructions, cond ! \ |
| 58 | 61 | .word (new_instr_ptr - .) ! \ |
| @@ -574,6 +574,7 @@ · 573 unmodified lines | ||
| 574 | 574 | */ |
| 575 | 575 | #define ASM_EXCEPTIONTABLE_ENTRY(fault_addr, except_addr) \ |
| 576 | 576 | .section __ex_table,"aw" ! \ |
| 577 | + .align 4 ! \ | |
| 577 | 578 | .word (fault_addr - .), (except_addr - .) ! \ |
| 578 | 579 | .previous |
| 579 | 580 | |
| @@ -17,24 +17,27 @@ · 16 unmodified lines | ||
| 17 | 17 | #define PARISC_BUG_BREAK_ASM "break 0x1f, 0x1fff" |
| 18 | 18 | #define PARISC_BUG_BREAK_INSN 0x03ffe01f /* PARISC_BUG_BREAK_ASM */ |
| 19 | 19 | |
| 20 | −#if defined(CONFIG_64BIT) | |
| 21 | −#define ASM_WORD_INSN ".dword\t" | |
| 20 | +#ifdef CONFIG_GENERIC_BUG_RELATIVE_POINTERS | |
| 21 | +# define __BUG_REL(val) ".word " __stringify(val) " - ." | |
| 22 | 22 | #else |
| 23 | −#define ASM_WORD_INSN ".word\t" | |
| 23 | +# define __BUG_REL(val) ".word " __stringify(val) | |
| 24 | 24 | #endif |
| 25 | 25 | |
| 26 | + | |
| 26 | 27 | #ifdef CONFIG_DEBUG_BUGVERBOSE |
| 27 | 28 | #define BUG() \ |
| 28 | 29 | do { \ |
| 29 | 30 | asm volatile("\n" \ |
| 30 | 31 | "1:\t" PARISC_BUG_BREAK_ASM "\n" \ |
| 31 | − "\t.pushsection __bug_table,\"aw\"\n" \ | |
| 32 | − "2:\t" ASM_WORD_INSN "1b, %c0\n" \ | |
| 33 | − "\t.short %c1, %c2\n" \ | |
| 34 | − "\t.org 2b+%c3\n" \ | |
| 32 | + "\t.pushsection __bug_table,\"a\"\n" \ | |
| 33 | + "\t.align 4\n" \ | |
| 34 | + "2:\t" __BUG_REL(1b) "\n" \ | |
| 35 | + "\t" __BUG_REL(%c0) "\n" \ | |
| 36 | + "\t.short %1, %2\n" \ | |
| 37 | + "\t.blockz %3-2*4-2*2\n" \ | |
| 35 | 38 | "\t.popsection" \ |
| 36 | 39 | : : "i" (__FILE__), "i" (__LINE__), \ |
| 37 | − "i" (0), "i" (sizeof(struct bug_entry)) ); \ | |
| 40 | + "i" (0), "i" (sizeof(struct bug_entry)) ); \ | |
| 38 | 41 | unreachable(); \ |
| 39 | 42 | } while(0) |
| 40 | 43 | |
| @@ -51,10 +54,12 @@ · 10 unmodified lines | ||
| 51 | 54 | do { \ |
| 52 | 55 | asm volatile("\n" \ |
| 53 | 56 | "1:\t" PARISC_BUG_BREAK_ASM "\n" \ |
| 54 | − "\t.pushsection __bug_table,\"aw\"\n" \ | |
| 55 | − "2:\t" ASM_WORD_INSN "1b, %c0\n" \ | |
| 56 | − "\t.short %c1, %c2\n" \ | |
| 57 | − "\t.org 2b+%c3\n" \ | |
| 57 | + "\t.pushsection __bug_table,\"a\"\n" \ | |
| 58 | + "\t.align 4\n" \ | |
| 59 | + "2:\t" __BUG_REL(1b) "\n" \ | |
| 60 | + "\t" __BUG_REL(%c0) "\n" \ | |
| 61 | + "\t.short %1, %2\n" \ | |
| 62 | + "\t.blockz %3-2*4-2*2\n" \ | |
| 58 | 63 | "\t.popsection" \ |
| 59 | 64 | : : "i" (__FILE__), "i" (__LINE__), \ |
| 60 | 65 | "i" (BUGFLAG_WARNING|(flags)), \ |
| @@ -65,10 +70,11 @@ · 4 unmodified lines | ||
| 65 | 70 | do { \ |
| 66 | 71 | asm volatile("\n" \ |
| 67 | 72 | "1:\t" PARISC_BUG_BREAK_ASM "\n" \ |
| 68 | − "\t.pushsection __bug_table,\"aw\"\n" \ | |
| 69 | − "2:\t" ASM_WORD_INSN "1b\n" \ | |
| 70 | − "\t.short %c0\n" \ | |
| 71 | − "\t.org 2b+%c1\n" \ | |
| 73 | + "\t.pushsection __bug_table,\"a\"\n" \ | |
| 74 | + "\t.align %2\n" \ | |
| 75 | + "2:\t" __BUG_REL(1b) "\n" \ | |
| 76 | + "\t.short %0\n" \ | |
| 77 | + "\t.blockz %1-4-2\n" \ | |
| 72 | 78 | "\t.popsection" \ |
| 73 | 79 | : : "i" (BUGFLAG_WARNING|(flags)), \ |
| 74 | 80 | "i" (sizeof(struct bug_entry)) ); \ |
| @@ -349,15 +349,7 @@ struct pt_regs; /* forward declaration... */ · 348 unmodified lines | ||
| 349 | 349 | |
| 350 | 350 | #define ELF_HWCAP 0 |
| 351 | 351 | |
| 352 | −/* Masks for stack and mmap randomization */ | |
| 353 | −#define BRK_RND_MASK (is_32bit_task() ? 0x07ffUL : 0x3ffffUL) | |
| 354 | −#define MMAP_RND_MASK (is_32bit_task() ? 0x1fffUL : 0x3ffffUL) | |
| 355 | −#define STACK_RND_MASK MMAP_RND_MASK | |
| 356 | − | |
| 357 | −struct mm_struct; | |
| 358 | −extern unsigned long arch_randomize_brk(struct mm_struct *); | |
| 359 | −#define arch_randomize_brk arch_randomize_brk | |
| 360 | − | |
| 352 | +#define STACK_RND_MASK 0x7ff /* 8MB of VA */ | |
| 361 | 353 | |
| 362 | 354 | #define ARCH_HAS_SETUP_ADDITIONAL_PAGES 1 |
| 363 | 355 | struct linux_binprm; |
| @@ -15,10 +15,12 @@ static __always_inline bool arch_static_branch(struct static_key *key, bool bran · 14 unmodified lines | ||
| 15 | 15 | asm_volatile_goto("1:\n\t" |
| 16 | 16 | "nop\n\t" |
| 17 | 17 | ".pushsection __jump_table, \"aw\"\n\t" |
| 18 | + ".align %1\n\t" | |
| 18 | 19 | ".word 1b - ., %l[l_yes] - .\n\t" |
| 19 | 20 | __stringify(ASM_ULONG_INSN) " %c0 - .\n\t" |
| 20 | 21 | ".popsection\n\t" |
| 21 | − : : "i" (&((char *)key)[branch]) : : l_yes); | |
| 22 | + : : "i" (&((char *)key)[branch]), "i" (sizeof(long)) | |
| 23 | + : : l_yes); | |
| 22 | 24 | |
| 23 | 25 | return false; |
| 24 | 26 | l_yes: |
| @@ -30,10 +32,12 @@ static __always_inline bool arch_static_branch_jump(struct static_key *key, bool · 5 unmodified lines | ||
| 30 | 32 | asm_volatile_goto("1:\n\t" |
| 31 | 33 | "b,n %l[l_yes]\n\t" |
| 32 | 34 | ".pushsection __jump_table, \"aw\"\n\t" |
| 35 | + ".align %1\n\t" | |
| 33 | 36 | ".word 1b - ., %l[l_yes] - .\n\t" |
| 34 | 37 | __stringify(ASM_ULONG_INSN) " %c0 - .\n\t" |
| 35 | 38 | ".popsection\n\t" |
| 36 | − : : "i" (&((char *)key)[branch]) : : l_yes); | |
| 39 | + : : "i" (&((char *)key)[branch]), "i" (sizeof(long)) | |
| 40 | + : : l_yes); | |
| 37 | 41 | |
| 38 | 42 | return false; |
| 39 | 43 | l_yes: |
| @@ -55,7 +55,7 @@ · 54 unmodified lines | ||
| 55 | 55 | }) |
| 56 | 56 | |
| 57 | 57 | #ifdef CONFIG_SMP |
| 58 | −# define __lock_aligned __section(".data..lock_aligned") | |
| 58 | +# define __lock_aligned __section(".data..lock_aligned") __aligned(16) | |
| 59 | 59 | #endif |
| 60 | 60 | |
| 61 | 61 | #endif /* __PARISC_LDCW_H */ |
| @@ -47,6 +47,8 @@ · 46 unmodified lines | ||
| 47 | 47 | |
| 48 | 48 | #ifndef __ASSEMBLY__ |
| 49 | 49 | |
| 50 | +struct rlimit; | |
| 51 | +unsigned long mmap_upper_limit(struct rlimit *rlim_stack); | |
| 50 | 52 | unsigned long calc_max_stack_size(unsigned long stack_max); |
| 51 | 53 | |
| 52 | 54 | /* |
| @@ -41,6 +41,7 @@ struct exception_table_entry { · 40 unmodified lines | ||
| 41 | 41 | |
| 42 | 42 | #define ASM_EXCEPTIONTABLE_ENTRY( fault_addr, except_addr )\ |
| 43 | 43 | ".section __ex_table,\"aw\"\n" \ |
| 44 | + ".align 4\n" \ | |
| 44 | 45 | ".word (" #fault_addr " - .), (" #except_addr " - .)\n\t" \ |
| 45 | 46 | ".previous\n" |
| 46 | 47 | |
| @@ -75,7 +75,6 @@ · 74 unmodified lines | ||
| 75 | 75 | |
| 76 | 76 | /* We now return you to your regularly scheduled HPUX. */ |
| 77 | 77 | |
| 78 | −#define ENOSYM 215 /* symbol does not exist in executable */ | |
| 79 | 78 | #define ENOTSOCK 216 /* Socket operation on non-socket */ |
| 80 | 79 | #define EDESTADDRREQ 217 /* Destination address required */ |
| 81 | 80 | #define EMSGSIZE 218 /* Message too long */ |
| @@ -101,7 +100,6 @@ · 19 unmodified lines | ||
| 101 | 100 | #define ETIMEDOUT 238 /* Connection timed out */ |
| 102 | 101 | #define ECONNREFUSED 239 /* Connection refused */ |
| 103 | 102 | #define EREFUSED ECONNREFUSED /* for HP's NFS apparently */ |
| 104 | −#define EREMOTERELEASE 240 /* Remote peer released connection */ | |
| 105 | 103 | #define EHOSTDOWN 241 /* Host is down */ |
| 106 | 104 | #define EHOSTUNREACH 242 /* No route to host */ |
| 107 | 105 | |
| @@ -383,7 +383,7 @@ show_cpuinfo (struct seq_file *m, void *v) · 382 unmodified lines | ||
| 383 | 383 | char cpu_name[60], *p; |
| 384 | 384 | |
| 385 | 385 | /* strip PA path from CPU name to not confuse lscpu */ |
| 386 | − strlcpy(cpu_name, per_cpu(cpu_data, 0).dev->name, sizeof(cpu_name)); | |
| 386 | + strscpy(cpu_name, per_cpu(cpu_data, 0).dev->name, sizeof(cpu_name)); | |
| 387 | 387 | p = strrchr(cpu_name, '['); |
| 388 | 388 | if (p) |
| 389 | 389 | *(--p) = 0; |
| @@ -77,7 +77,7 @@ unsigned long calc_max_stack_size(unsigned long stack_max) · 76 unmodified lines | ||
| 77 | 77 | * indicating that "current" should be used instead of a passed-in |
| 78 | 78 | * value from the exec bprm as done with arch_pick_mmap_layout(). |
| 79 | 79 | */ |
| 80 | −static unsigned long mmap_upper_limit(struct rlimit *rlim_stack) | |
| 80 | +unsigned long mmap_upper_limit(struct rlimit *rlim_stack) | |
| 81 | 81 | { |
| 82 | 82 | unsigned long stack_base; |
| 83 | 83 | |
| @@ -130,6 +130,7 @@ SECTIONS · 129 unmodified lines | ||
| 130 | 130 | RO_DATA(8) |
| 131 | 131 | |
| 132 | 132 | /* unwind info */ |
| 133 | + . = ALIGN(4); | |
| 133 | 134 | .PARISC.unwind : { |
| 134 | 135 | __start___unwind = .; |
| 135 | 136 | *(.PARISC.unwind) |
| @@ -228,7 +228,6 @@ typedef struct thread_struct thread_struct; · 227 unmodified lines | ||
| 228 | 228 | execve_tail(); \ |
| 229 | 229 | } while (0) |
| 230 | 230 | |
| 231 | −/* Forward declaration, a strange C thing */ | |
| 232 | 231 | struct task_struct; |
| 233 | 232 | struct mm_struct; |
| 234 | 233 | struct seq_file; |
| @@ -666,6 +666,7 @@ static int __init ipl_init(void) · 665 unmodified lines | ||
| 666 | 666 | &ipl_ccw_attr_group_lpar); |
| 667 | 667 | break; |
| 668 | 668 | case IPL_TYPE_ECKD: |
| 669 | + case IPL_TYPE_ECKD_DUMP: | |
| 669 | 670 | rc = sysfs_create_group(&ipl_kset->kobj, &ipl_eckd_attr_group); |
| 670 | 671 | break; |
| 671 | 672 | case IPL_TYPE_FCP: |
| @@ -279,12 +279,6 @@ static int paicrypt_event_init(struct perf_event *event) · 278 unmodified lines | ||
| 279 | 279 | if (IS_ERR(cpump)) |
| 280 | 280 | return PTR_ERR(cpump); |
| 281 | 281 | |
| 282 | − /* Event initialization sets last_tag to 0. When later on the events | |
| 283 | − * are deleted and re-added, do not reset the event count value to zero. | |
| 284 | − * Events are added, deleted and re-added when 2 or more events | |
| 285 | − * are active at the same time. | |
| 286 | − */ | |
| 287 | − event->hw.last_tag = 0; | |
| 288 | 282 | event->destroy = paicrypt_event_destroy; |
| 289 | 283 | |
| 290 | 284 | if (a->sample_period) { |
| @@ -318,6 +312,11 @@ static void paicrypt_start(struct perf_event *event, int flags) · 27 unmodified lines | ||
| 318 | 312 | { |
| 319 | 313 | u64 sum; |
| 320 | 314 | |
| 315 | + /* Event initialization sets last_tag to 0. When later on the events | |
| 316 | + * are deleted and re-added, do not reset the event count value to zero. | |
| 317 | + * Events are added, deleted and re-added when 2 or more events | |
| 318 | + * are active at the same time. | |
| 319 | + */ | |
| 321 | 320 | if (!event->hw.last_tag) { |
| 322 | 321 | event->hw.last_tag = 1; |
| 323 | 322 | sum = paicrypt_getall(event); /* Get current value */ |
| @@ -260,7 +260,6 @@ static int paiext_event_init(struct perf_event *event) · 259 unmodified lines | ||
| 260 | 260 | rc = paiext_alloc(a, event); |
| 261 | 261 | if (rc) |
| 262 | 262 | return rc; |
| 263 | − event->hw.last_tag = 0; | |
| 264 | 263 | event->destroy = paiext_event_destroy; |
| 265 | 264 | |
| 266 | 265 | if (a->sample_period) { |
| @@ -4660,7 +4660,7 @@ static void intel_pmu_check_hybrid_pmus(struct x86_hybrid_pmu *pmu) · 4659 unmodified lines | ||
| 4660 | 4660 | if (pmu->intel_cap.pebs_output_pt_available) |
| 4661 | 4661 | pmu->pmu.capabilities |= PERF_PMU_CAP_AUX_OUTPUT; |
| 4662 | 4662 | else |
| 4663 | − pmu->pmu.capabilities |= ~PERF_PMU_CAP_AUX_OUTPUT; | |
| 4663 | + pmu->pmu.capabilities &= ~PERF_PMU_CAP_AUX_OUTPUT; | |
| 4664 | 4664 | |
| 4665 | 4665 | intel_pmu_check_event_constraints(pmu->event_constraints, |
| 4666 | 4666 | pmu->num_counters, |
| @@ -15,6 +15,7 @@ · 14 unmodified lines | ||
| 15 | 15 | #include <linux/io.h> |
| 16 | 16 | #include <asm/apic.h> |
| 17 | 17 | #include <asm/desc.h> |
| 18 | +#include <asm/e820/api.h> | |
| 18 | 19 | #include <asm/sev.h> |
| 19 | 20 | #include <asm/ibt.h> |
| 20 | 21 | #include <asm/hypervisor.h> |
| @@ -286,15 +287,31 @@ static int hv_cpu_die(unsigned int cpu) · 265 unmodified lines | ||
| 286 | 287 | |
| 287 | 288 | static int __init hv_pci_init(void) |
| 288 | 289 | { |
| 289 | − int gen2vm = efi_enabled(EFI_BOOT); | |
| 290 | + bool gen2vm = efi_enabled(EFI_BOOT); | |
| 290 | 291 | |
| 291 | 292 | /* |
| 292 | − * For Generation-2 VM, we exit from pci_arch_init() by returning 0. | |
| 293 | − * The purpose is to suppress the harmless warning: | |
| 293 | + * A Generation-2 VM doesn't support legacy PCI/PCIe, so both | |
| 294 | + * raw_pci_ops and raw_pci_ext_ops are NULL, and pci_subsys_init() -> | |
| 295 | + * pcibios_init() doesn't call pcibios_resource_survey() -> | |
| 296 | + * e820__reserve_resources_late(); as a result, any emulated persistent | |
| 297 | + * memory of E820_TYPE_PRAM (12) via the kernel parameter | |
| 298 | + * memmap=nn[KMG]!ss is not added into iomem_resource and hence can't be | |
| 299 | + * detected by register_e820_pmem(). Fix this by directly calling | |
| 300 | + * e820__reserve_resources_late() here: e820__reserve_resources_late() | |
| 301 | + * depends on e820__reserve_resources(), which has been called earlier | |
| 302 | + * from setup_arch(). Note: e820__reserve_resources_late() also adds | |
| 303 | + * any memory of E820_TYPE_PMEM (7) into iomem_resource, and | |
| 304 | + * acpi_nfit_register_region() -> acpi_nfit_insert_resource() -> | |
| 305 | + * region_intersects() returns REGION_INTERSECTS, so the memory of | |
| 306 | + * E820_TYPE_PMEM won't get added twice. | |
| 307 | + * | |
| 308 | + * We return 0 here so that pci_arch_init() won't print the warning: | |
| 294 | 309 | * "PCI: Fatal: No config space access function found" |
| 295 | 310 | */ |
| 296 | − if (gen2vm) | |
| 311 | + if (gen2vm) { | |
| 312 | + e820__reserve_resources_late(); | |
| 297 | 313 | return 0; |
| 314 | + } | |
| 298 | 315 | |
| 299 | 316 | /* For Generation-1 VM, we'll proceed in pci_arch_init(). */ |
| 300 | 317 | return 1; |
| @@ -16,6 +16,9 @@ · 15 unmodified lines | ||
| 16 | 16 | #include <asm/x86_init.h> |
| 17 | 17 | #include <asm/cpufeature.h> |
| 18 | 18 | #include <asm/irq_vectors.h> |
| 19 | +#include <asm/xen/hypervisor.h> | |
| 20 | + | |
| 21 | +#include <xen/xen.h> | |
| 19 | 22 | |
| 20 | 23 | #ifdef CONFIG_ACPI_APEI |
| 21 | 24 | # include <asm/pgtable_types.h> |
| @@ -127,6 +130,17 @@ static inline void arch_acpi_set_proc_cap_bits(u32 *cap) · 105 unmodified lines | ||
| 127 | 130 | if (!cpu_has(c, X86_FEATURE_MWAIT) || |
| 128 | 131 | boot_option_idle_override == IDLE_NOMWAIT) |
| 129 | 132 | *cap &= ~(ACPI_PROC_CAP_C_C1_FFH | ACPI_PROC_CAP_C_C2C3_FFH); |
| 133 | + | |
| 134 | + if (xen_initial_domain()) { | |
| 135 | + /* | |
| 136 | + * When Linux is running as Xen dom0, the hypervisor is the | |
| 137 | + * entity in charge of the processor power management, and so | |
| 138 | + * Xen needs to check the OS capabilities reported in the | |
| 139 | + * processor capabilities buffer matches what the hypervisor | |
| 140 | + * driver supports. | |
| 141 | + */ | |
| 142 | + xen_sanitize_proc_cap_bits(cap); | |
| 143 | + } | |
| 130 | 144 | } |
| 131 | 145 | |
| 132 | 146 | static inline bool acpi_has_cpu_in_madt(void) |
| @@ -100,4 +100,13 @@ static inline void leave_lazy(enum xen_lazy_mode mode) · 99 unmodified lines | ||
| 100 | 100 | |
| 101 | 101 | enum xen_lazy_mode xen_get_lazy_mode(void); |
| 102 | 102 | |
| 103 | +#if defined(CONFIG_XEN_DOM0) && defined(CONFIG_ACPI) | |
| 104 | +void xen_sanitize_proc_cap_bits(uint32_t *buf); | |
| 105 | +#else | |
| 106 | +static inline void xen_sanitize_proc_cap_bits(uint32_t *buf) | |
| 107 | +{ | |
| 108 | + BUG(); | |
| 109 | +} | |
| 110 | +#endif | |
| 111 | + | |
| 103 | 112 | #endif /* _ASM_X86_XEN_HYPERVISOR_H */ |
| @@ -63,6 +63,7 @@ int acpi_fix_pin2_polarity __initdata; · 62 unmodified lines | ||
| 63 | 63 | |
| 64 | 64 | #ifdef CONFIG_X86_LOCAL_APIC |
| 65 | 65 | static u64 acpi_lapic_addr __initdata = APIC_DEFAULT_PHYS_BASE; |
| 66 | +static bool has_lapic_cpus __initdata; | |
| 66 | 67 | static bool acpi_support_online_capable; |
| 67 | 68 | #endif |
| 68 | 69 | |
| @@ -232,6 +233,14 @@ acpi_parse_x2apic(union acpi_subtable_headers *header, const unsigned long end) · 163 unmodified lines | ||
| 232 | 233 | if (!acpi_is_processor_usable(processor->lapic_flags)) |
| 233 | 234 | return 0; |
| 234 | 235 | |
| 236 | + /* | |
| 237 | + * According to https://uefi.org/specs/ACPI/6.5/05_ACPI_Software_Programming_Model.html#processor-local-x2apic-structure | |
| 238 | + * when MADT provides both valid LAPIC and x2APIC entries, the APIC ID | |
| 239 | + * in x2APIC must be equal or greater than 0xff. | |
| 240 | + */ | |
| 241 | + if (has_lapic_cpus && apic_id < 0xff) | |
| 242 | + return 0; | |
| 243 | + | |
| 235 | 244 | /* |
| 236 | 245 | * We need to register disabled CPU as well to permit |
| 237 | 246 | * counting disabled CPUs. This allows us to size |
| @@ -1114,10 +1123,7 @@ static int __init early_acpi_parse_madt_lapic_addr_ovr(void) · 876 unmodified lines | ||
| 1114 | 1123 | |
| 1115 | 1124 | static int __init acpi_parse_madt_lapic_entries(void) |
| 1116 | 1125 | { |
| 1117 | − int count; | |
| 1118 | − int x2count = 0; | |
| 1119 | − int ret; | |
| 1120 | − struct acpi_subtable_proc madt_proc[2]; | |
| 1126 | + int count, x2count = 0; | |
| 1121 | 1127 | |
| 1122 | 1128 | if (!boot_cpu_has(X86_FEATURE_APIC)) |
| 1123 | 1129 | return -ENODEV; |
| @@ -1126,21 +1132,11 @@ static int __init acpi_parse_madt_lapic_entries(void) · 2 unmodified lines | ||
| 1126 | 1132 | acpi_parse_sapic, MAX_LOCAL_APIC); |
| 1127 | 1133 | |
| 1128 | 1134 | if (!count) { |
| 1129 | − memset(madt_proc, 0, sizeof(madt_proc)); | |
| 1130 | − madt_proc[0].id = ACPI_MADT_TYPE_LOCAL_APIC; | |
| 1131 | − madt_proc[0].handler = acpi_parse_lapic; | |
| 1132 | − madt_proc[1].id = ACPI_MADT_TYPE_LOCAL_X2APIC; | |
| 1133 | − madt_proc[1].handler = acpi_parse_x2apic; | |
| 1134 | − ret = acpi_table_parse_entries_array(ACPI_SIG_MADT, | |
| 1135 | − sizeof(struct acpi_table_madt), | |
| 1136 | − madt_proc, ARRAY_SIZE(madt_proc), MAX_LOCAL_APIC); | |
| 1137 | − if (ret < 0) { | |
| 1138 | − pr_err("Error parsing LAPIC/X2APIC entries\n"); | |
| 1139 | − return ret; | |
| 1140 | − } | |
| 1141 | − | |
| 1142 | − count = madt_proc[0].count; | |
| 1143 | − x2count = madt_proc[1].count; | |
| 1135 | + count = acpi_table_parse_madt(ACPI_MADT_TYPE_LOCAL_APIC, | |
| 1136 | + acpi_parse_lapic, MAX_LOCAL_APIC); | |
| 1137 | + has_lapic_cpus = count > 0; | |
| 1138 | + x2count = acpi_table_parse_madt(ACPI_MADT_TYPE_LOCAL_X2APIC, | |
| 1139 | + acpi_parse_x2apic, MAX_LOCAL_APIC); | |
| 1144 | 1140 | } |
| 1145 | 1141 | if (!count && !x2count) { |
| 1146 | 1142 | pr_err("No LAPIC entries present\n"); |
| @@ -104,8 +104,6 @@ struct cont_desc { · 103 unmodified lines | ||
| 104 | 104 | size_t size; |
| 105 | 105 | }; |
| 106 | 106 | |
| 107 | −static u32 ucode_new_rev; | |
| 108 | − | |
| 109 | 107 | /* |
| 110 | 108 | * Microcode patch container file is prepended to the initrd in cpio |
| 111 | 109 | * format. See Documentation/arch/x86/microcode.rst |
| @@ -442,12 +440,11 @@ static int __apply_microcode_amd(struct microcode_amd *mc) · 330 unmodified lines | ||
| 442 | 440 | * |
| 443 | 441 | * Returns true if container found (sets @desc), false otherwise. |
| 444 | 442 | */ |
| 445 | −static bool early_apply_microcode(u32 cpuid_1_eax, void *ucode, size_t size) | |
| 443 | +static bool early_apply_microcode(u32 cpuid_1_eax, u32 old_rev, void *ucode, size_t size) | |
| 446 | 444 | { |
| 447 | 445 | struct cont_desc desc = { 0 }; |
| 448 | 446 | struct microcode_amd *mc; |
| 449 | 447 | bool ret = false; |
| 450 | − u32 rev, dummy; | |
| 451 | 448 | |
| 452 | 449 | desc.cpuid_1_eax = cpuid_1_eax; |
| 453 | 450 | |
| @@ -457,22 +454,15 @@ static bool early_apply_microcode(u32 cpuid_1_eax, void *ucode, size_t size) · 3 unmodified lines | ||
| 457 | 454 | if (!mc) |
| 458 | 455 | return ret; |
| 459 | 456 | |
| 460 | − native_rdmsr(MSR_AMD64_PATCH_LEVEL, rev, dummy); | |
| 461 | − | |
| 462 | 457 | /* |
| 463 | 458 | * Allow application of the same revision to pick up SMT-specific |
| 464 | 459 | * changes even if the revision of the other SMT thread is already |
| 465 | 460 | * up-to-date. |
| 466 | 461 | */ |
| 467 | − if (rev > mc->hdr.patch_id) | |
| 462 | + if (old_rev > mc->hdr.patch_id) | |
| 468 | 463 | return ret; |
| 469 | 464 | |
| 470 | − if (!__apply_microcode_amd(mc)) { | |
| 471 | − ucode_new_rev = mc->hdr.patch_id; | |
| 472 | − ret = true; | |
| 473 | − } | |
| 474 | − | |
| 475 | − return ret; | |
| 465 | + return !__apply_microcode_amd(mc); | |
| 476 | 466 | } |
| 477 | 467 | |
| 478 | 468 | static bool get_builtin_microcode(struct cpio_data *cp, unsigned int family) |
| @@ -506,9 +496,12 @@ static void __init find_blobs_in_containers(unsigned int cpuid_1_eax, struct cpi · 27 unmodified lines | ||
| 506 | 496 | *ret = cp; |
| 507 | 497 | } |
| 508 | 498 | |
| 509 | −void __init load_ucode_amd_bsp(unsigned int cpuid_1_eax) | |
| 499 | +void __init load_ucode_amd_bsp(struct early_load_data *ed, unsigned int cpuid_1_eax) | |
| 510 | 500 | { |
| 511 | 501 | struct cpio_data cp = { }; |
| 502 | + u32 dummy; | |
| 503 | + | |
| 504 | + native_rdmsr(MSR_AMD64_PATCH_LEVEL, ed->old_rev, dummy); | |
| 512 | 505 | |
| 513 | 506 | /* Needed in load_microcode_amd() */ |
| 514 | 507 | ucode_cpu_info[0].cpu_sig.sig = cpuid_1_eax; |
| @@ -517,7 +510,8 @@ void __init load_ucode_amd_bsp(unsigned int cpuid_1_eax) · 2 unmodified lines | ||
| 517 | 510 | if (!(cp.data && cp.size)) |
| 518 | 511 | return; |
| 519 | 512 | |
| 520 | − early_apply_microcode(cpuid_1_eax, cp.data, cp.size); | |
| 513 | + if (early_apply_microcode(cpuid_1_eax, ed->old_rev, cp.data, cp.size)) | |
| 514 | + native_rdmsr(MSR_AMD64_PATCH_LEVEL, ed->new_rev, dummy); | |
| 521 | 515 | } |
| 522 | 516 | |
| 523 | 517 | static enum ucode_state load_microcode_amd(u8 family, const u8 *data, size_t size); |
| @@ -625,10 +619,8 @@ void reload_ucode_amd(unsigned int cpu) · 101 unmodified lines | ||
| 625 | 619 | rdmsr(MSR_AMD64_PATCH_LEVEL, rev, dummy); |
| 626 | 620 | |
| 627 | 621 | if (rev < mc->hdr.patch_id) { |
| 628 | − if (!__apply_microcode_amd(mc)) { | |
| 629 | − ucode_new_rev = mc->hdr.patch_id; | |
| 630 | − pr_info("reload patch_level=0x%08x\n", ucode_new_rev); | |
| 631 | − } | |
| 622 | + if (!__apply_microcode_amd(mc)) | |
| 623 | + pr_info_once("reload revision: 0x%08x\n", mc->hdr.patch_id); | |
| 632 | 624 | } |
| 633 | 625 | } |
| 634 | 626 | |
| @@ -649,8 +641,6 @@ static int collect_cpu_info_amd(int cpu, struct cpu_signature *csig) · 14 unmodified lines | ||
| 649 | 641 | if (p && (p->patch_id == csig->rev)) |
| 650 | 642 | uci->mc = p->data; |
| 651 | 643 | |
| 652 | − pr_info("CPU%d: patch_level=0x%08x\n", cpu, csig->rev); | |
| 653 | − | |
| 654 | 644 | return 0; |
| 655 | 645 | } |
| 656 | 646 | |
| @@ -691,8 +681,6 @@ static enum ucode_state apply_microcode_amd(int cpu) · 34 unmodified lines | ||
| 691 | 681 | rev = mc_amd->hdr.patch_id; |
| 692 | 682 | ret = UCODE_UPDATED; |
| 693 | 683 | |
| 694 | − pr_info("CPU%d: new patch_level=0x%08x\n", cpu, rev); | |
| 695 | − | |
| 696 | 684 | out: |
| 697 | 685 | uci->cpu_sig.rev = rev; |
| 698 | 686 | c->microcode = rev; |
| @@ -935,11 +923,6 @@ struct microcode_ops * __init init_amd_microcode(void) · 236 unmodified lines | ||
| 935 | 923 | pr_warn("AMD CPU family 0x%x not supported\n", c->x86); |
| 936 | 924 | return NULL; |
| 937 | 925 | } |
| 938 | − | |
| 939 | − if (ucode_new_rev) | |
| 940 | − pr_info_once("microcode updated early to new patch_level=0x%08x\n", | |
| 941 | − ucode_new_rev); | |
| 942 | − | |
| 943 | 926 | return µcode_amd_ops; |
| 944 | 927 | } |
| 945 | 928 | |
| @@ -41,8 +41,6 @@ · 40 unmodified lines | ||
| 41 | 41 | |
| 42 | 42 | #include "internal.h" |
| 43 | 43 | |
| 44 | −#define DRIVER_VERSION "2.2" | |
| 45 | − | |
| 46 | 44 | static struct microcode_ops *microcode_ops; |
| 47 | 45 | bool dis_ucode_ldr = true; |
| 48 | 46 | |
| @@ -77,6 +75,8 @@ static u32 final_levels[] = { · 28 unmodified lines | ||
| 77 | 75 | 0, /* T-101 terminator */ |
| 78 | 76 | }; |
| 79 | 77 | |
| 78 | +struct early_load_data early_data; | |
| 79 | + | |
| 80 | 80 | /* |
| 81 | 81 | * Check the current patch level on this CPU. |
| 82 | 82 | * |
| @@ -155,9 +155,9 @@ void __init load_ucode_bsp(void) · 72 unmodified lines | ||
| 155 | 155 | return; |
| 156 | 156 | |
| 157 | 157 | if (intel) |
| 158 | − load_ucode_intel_bsp(); | |
| 158 | + load_ucode_intel_bsp(&early_data); | |
| 159 | 159 | else |
| 160 | − load_ucode_amd_bsp(cpuid_1_eax); | |
| 160 | + load_ucode_amd_bsp(&early_data, cpuid_1_eax); | |
| 161 | 161 | } |
| 162 | 162 | |
| 163 | 163 | void load_ucode_ap(void) |
| @@ -828,6 +828,11 @@ static int __init microcode_init(void) · 664 unmodified lines | ||
| 828 | 828 | if (!microcode_ops) |
| 829 | 829 | return -ENODEV; |
| 830 | 830 | |
| 831 | + pr_info_once("Current revision: 0x%08x\n", (early_data.new_rev ?: early_data.old_rev)); | |
| 832 | + | |
| 833 | + if (early_data.new_rev) | |
| 834 | + pr_info_once("Updated early from: 0x%08x\n", early_data.old_rev); | |
| 835 | + | |
| 831 | 836 | microcode_pdev = platform_device_register_simple("microcode", -1, NULL, 0); |
| 832 | 837 | if (IS_ERR(microcode_pdev)) |
| 833 | 838 | return PTR_ERR(microcode_pdev); |
| @@ -846,8 +851,6 @@ static int __init microcode_init(void) · 12 unmodified lines | ||
| 846 | 851 | cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/microcode:online", |
| 847 | 852 | mc_cpu_online, mc_cpu_down_prep); |
| 848 | 853 | |
| 849 | − pr_info("Microcode Update Driver: v%s.", DRIVER_VERSION); | |
| 850 | − | |
| 851 | 854 | return 0; |
| 852 | 855 | |
| 853 | 856 | out_pdev: |
| @@ -339,16 +339,9 @@ static enum ucode_state __apply_microcode(struct ucode_cpu_info *uci, · 338 unmodified lines | ||
| 339 | 339 | static enum ucode_state apply_microcode_early(struct ucode_cpu_info *uci) |
| 340 | 340 | { |
| 341 | 341 | struct microcode_intel *mc = uci->mc; |
| 342 | − enum ucode_state ret; | |
| 343 | − u32 cur_rev, date; | |
| 342 | + u32 cur_rev; | |
| 344 | 343 | |
| 345 | − ret = __apply_microcode(uci, mc, &cur_rev); | |
| 346 | − if (ret == UCODE_UPDATED) { | |
| 347 | − date = mc->hdr.date; | |
| 348 | − pr_info_once("updated early: 0x%x -> 0x%x, date = %04x-%02x-%02x\n", | |
| 349 | − cur_rev, mc->hdr.rev, date & 0xffff, date >> 24, (date >> 16) & 0xff); | |
| 350 | − } | |
| 351 | − return ret; | |
| 344 | + return __apply_microcode(uci, mc, &cur_rev); | |
| 352 | 345 | } |
| 353 | 346 | |
| 354 | 347 | static __init bool load_builtin_intel_microcode(struct cpio_data *cp) |
| @@ -413,13 +406,17 @@ static int __init save_builtin_microcode(void) · 58 unmodified lines | ||
| 413 | 406 | early_initcall(save_builtin_microcode); |
| 414 | 407 | |
| 415 | 408 | /* Load microcode on BSP from initrd or builtin blobs */ |
| 416 | −void __init load_ucode_intel_bsp(void) | |
| 409 | +void __init load_ucode_intel_bsp(struct early_load_data *ed) | |
| 417 | 410 | { |
| 418 | 411 | struct ucode_cpu_info uci; |
| 419 | 412 | |
| 413 | + ed->old_rev = intel_get_microcode_revision(); | |
| 414 | + | |
| 420 | 415 | uci.mc = get_microcode_blob(&uci, false); |
| 421 | 416 | if (uci.mc && apply_microcode_early(&uci) == UCODE_UPDATED) |
| 422 | 417 | ucode_patch_va = UCODE_BSP_LOADED; |
| 418 | + | |
| 419 | + ed->new_rev = uci.cpu_sig.rev; | |
| 423 | 420 | } |
| 424 | 421 | |
| 425 | 422 | void load_ucode_intel_ap(void) |
| @@ -37,6 +37,12 @@ struct microcode_ops { · 36 unmodified lines | ||
| 37 | 37 | use_nmi : 1; |
| 38 | 38 | }; |
| 39 | 39 | |
| 40 | +struct early_load_data { | |
| 41 | + u32 old_rev; | |
| 42 | + u32 new_rev; | |
| 43 | +}; | |
| 44 | + | |
| 45 | +extern struct early_load_data early_data; | |
| 40 | 46 | extern struct ucode_cpu_info ucode_cpu_info[]; |
| 41 | 47 | struct cpio_data find_microcode_in_initrd(const char *path); |
| 42 | 48 | |
| @@ -92,14 +98,14 @@ extern bool dis_ucode_ldr; · 49 unmodified lines | ||
| 92 | 98 | extern bool force_minrev; |
| 93 | 99 | |
| 94 | 100 | #ifdef CONFIG_CPU_SUP_AMD |
| 95 | −void load_ucode_amd_bsp(unsigned int family); | |
| 101 | +void load_ucode_amd_bsp(struct early_load_data *ed, unsigned int family); | |
| 96 | 102 | void load_ucode_amd_ap(unsigned int family); |
| 97 | 103 | int save_microcode_in_initrd_amd(unsigned int family); |
| 98 | 104 | void reload_ucode_amd(unsigned int cpu); |
| 99 | 105 | struct microcode_ops *init_amd_microcode(void); |
| 100 | 106 | void exit_amd_microcode(void); |
| 101 | 107 | #else /* CONFIG_CPU_SUP_AMD */ |
| 102 | −static inline void load_ucode_amd_bsp(unsigned int family) { } | |
| 108 | +static inline void load_ucode_amd_bsp(struct early_load_data *ed, unsigned int family) { } | |
| 103 | 109 | static inline void load_ucode_amd_ap(unsigned int family) { } |
| 104 | 110 | static inline int save_microcode_in_initrd_amd(unsigned int family) { return -EINVAL; } |
| 105 | 111 | static inline void reload_ucode_amd(unsigned int cpu) { } |
| @@ -108,12 +114,12 @@ static inline void exit_amd_microcode(void) { } · 2 unmodified lines | ||
| 108 | 114 | #endif /* !CONFIG_CPU_SUP_AMD */ |
| 109 | 115 | |
| 110 | 116 | #ifdef CONFIG_CPU_SUP_INTEL |
| 111 | −void load_ucode_intel_bsp(void); | |
| 117 | +void load_ucode_intel_bsp(struct early_load_data *ed); | |
| 112 | 118 | void load_ucode_intel_ap(void); |
| 113 | 119 | void reload_ucode_intel(void); |
| 114 | 120 | struct microcode_ops *init_intel_microcode(void); |
| 115 | 121 | #else /* CONFIG_CPU_SUP_INTEL */ |
| 116 | −static inline void load_ucode_intel_bsp(void) { } | |
| 122 | +static inline void load_ucode_intel_bsp(struct early_load_data *ed) { } | |
| 117 | 123 | static inline void load_ucode_intel_ap(void) { } |
| 118 | 124 | static inline void reload_ucode_intel(void) { } |
| 119 | 125 | static inline struct microcode_ops *init_intel_microcode(void) { return NULL; } |
| @@ -262,11 +262,14 @@ static uint32_t __init ms_hyperv_platform(void) · 261 unmodified lines | ||
| 262 | 262 | static int hv_nmi_unknown(unsigned int val, struct pt_regs *regs) |
| 263 | 263 | { |
| 264 | 264 | static atomic_t nmi_cpu = ATOMIC_INIT(-1); |
| 265 | + unsigned int old_cpu, this_cpu; | |
| 265 | 266 | |
| 266 | 267 | if (!unknown_nmi_panic) |
| 267 | 268 | return NMI_DONE; |
| 268 | 269 | |
| 269 | − if (atomic_cmpxchg(&nmi_cpu, -1, raw_smp_processor_id()) != -1) | |
| 270 | + old_cpu = -1; | |
| 271 | + this_cpu = raw_smp_processor_id(); | |
| 272 | + if (!atomic_try_cmpxchg(&nmi_cpu, &old_cpu, this_cpu)) | |
| 270 | 273 | return NMI_HANDLED; |
| 271 | 274 | |
| 272 | 275 | return NMI_DONE; |
| @@ -175,9 +175,6 @@ int x64_setup_rt_frame(struct ksignal *ksig, struct pt_regs *regs) · 174 unmodified lines | ||
| 175 | 175 | frame = get_sigframe(ksig, regs, sizeof(struct rt_sigframe), &fp); |
| 176 | 176 | uc_flags = frame_uc_flags(regs); |
| 177 | 177 | |
| 178 | − if (setup_signal_shadow_stack(ksig)) | |
| 179 | − return -EFAULT; | |
| 180 | − | |
| 181 | 178 | if (!user_access_begin(frame, sizeof(*frame))) |
| 182 | 179 | return -EFAULT; |
| 183 | 180 | |
| @@ -198,6 +195,9 @@ int x64_setup_rt_frame(struct ksignal *ksig, struct pt_regs *regs) · 14 unmodified lines | ||
| 198 | 195 | return -EFAULT; |
| 199 | 196 | } |
| 200 | 197 | |
| 198 | + if (setup_signal_shadow_stack(ksig)) | |
| 199 | + return -EFAULT; | |
| 200 | + | |
| 201 | 201 | /* Set up registers for signal handler */ |
| 202 | 202 | regs->di = ksig->sig; |
| 203 | 203 | /* In case the signal handler was declared without prototypes */ |
| @@ -425,6 +425,8 @@ void bdev_set_nr_sectors(struct block_device *bdev, sector_t sectors) · 424 unmodified lines | ||
| 425 | 425 | |
| 426 | 426 | void bdev_add(struct block_device *bdev, dev_t dev) |
| 427 | 427 | { |
| 428 | + if (bdev_stable_writes(bdev)) | |
| 429 | + mapping_set_stable_writes(bdev->bd_inode->i_mapping); | |
| 428 | 430 | bdev->bd_dev = dev; |
| 429 | 431 | bdev->bd_inode->i_rdev = dev; |
| 430 | 432 | bdev->bd_inode->i_ino = dev; |
| @@ -577,6 +577,7 @@ static void blkg_destroy_all(struct gendisk *disk) · 576 unmodified lines | ||
| 577 | 577 | struct request_queue *q = disk->queue; |
| 578 | 578 | struct blkcg_gq *blkg, *n; |
| 579 | 579 | int count = BLKG_DESTROY_BATCH_SIZE; |
| 580 | + int i; | |
| 580 | 581 | |
| 581 | 582 | restart: |
| 582 | 583 | spin_lock_irq(&q->queue_lock); |
| @@ -602,6 +603,18 @@ restart: · 19 unmodified lines | ||
| 602 | 603 | } |
| 603 | 604 | } |
| 604 | 605 | |
| 606 | + /* | |
| 607 | + * Mark policy deactivated since policy offline has been done, and | |
| 608 | + * the free is scheduled, so future blkcg_deactivate_policy() can | |
| 609 | + * be bypassed | |
| 610 | + */ | |
| 611 | + for (i = 0; i < BLKCG_MAX_POLS; i++) { | |
| 612 | + struct blkcg_policy *pol = blkcg_policy[i]; | |
| 613 | + | |
| 614 | + if (pol) | |
| 615 | + __clear_bit(pol->plid, q->blkcg_pols); | |
| 616 | + } | |
| 617 | + | |
| 605 | 618 | q->root_blkg = NULL; |
| 606 | 619 | spin_unlock_irq(&q->queue_lock); |
| 607 | 620 | } |
| @@ -249,8 +249,6 @@ static inline struct blkcg_gq *blkg_lookup(struct blkcg *blkcg, · 248 unmodified lines | ||
| 249 | 249 | { |
| 250 | 250 | struct blkcg_gq *blkg; |
| 251 | 251 | |
| 252 | − WARN_ON_ONCE(!rcu_read_lock_held()); | |
| 253 | − | |
| 254 | 252 | if (blkcg == &blkcg_root) |
| 255 | 253 | return q->root_blkg; |
| 256 | 254 | |
| @@ -2858,11 +2858,8 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q, · 2857 unmodified lines | ||
| 2858 | 2858 | }; |
| 2859 | 2859 | struct request *rq; |
| 2860 | 2860 | |
| 2861 | − if (unlikely(bio_queue_enter(bio))) | |
| 2862 | − return NULL; | |
| 2863 | − | |
| 2864 | 2861 | if (blk_mq_attempt_bio_merge(q, bio, nsegs)) |
| 2865 | − goto queue_exit; | |
| 2862 | + return NULL; | |
| 2866 | 2863 | |
| 2867 | 2864 | rq_qos_throttle(q, bio); |
| 2868 | 2865 | |
| @@ -2878,35 +2875,23 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q, · 9 unmodified lines | ||
| 2878 | 2875 | rq_qos_cleanup(q, bio); |
| 2879 | 2876 | if (bio->bi_opf & REQ_NOWAIT) |
| 2880 | 2877 | bio_wouldblock_error(bio); |
| 2881 | −queue_exit: | |
| 2882 | − blk_queue_exit(q); | |
| 2883 | 2878 | return NULL; |
| 2884 | 2879 | } |
| 2885 | 2880 | |
| 2886 | −static inline struct request *blk_mq_get_cached_request(struct request_queue *q, | |
| 2887 | − struct blk_plug *plug, struct bio **bio, unsigned int nsegs) | |
| 2881 | +/* return true if this @rq can be used for @bio */ | |
| 2882 | +static bool blk_mq_can_use_cached_rq(struct request *rq, struct blk_plug *plug, | |
| 2883 | + struct bio *bio) | |
| 2888 | 2884 | { |
| 2889 | − struct request *rq; | |
| 2890 | − enum hctx_type type, hctx_type; | |
| 2885 | + enum hctx_type type = blk_mq_get_hctx_type(bio->bi_opf); | |
| 2886 | + enum hctx_type hctx_type = rq->mq_hctx->type; | |
| 2891 | 2887 | |
| 2892 | − if (!plug) | |
| 2893 | − return NULL; | |
| 2894 | − rq = rq_list_peek(&plug->cached_rq); | |
| 2895 | − if (!rq || rq->q != q) | |
| 2896 | − return NULL; | |
| 2888 | + WARN_ON_ONCE(rq_list_peek(&plug->cached_rq) != rq); | |
| 2897 | 2889 | |
| 2898 | − if (blk_mq_attempt_bio_merge(q, *bio, nsegs)) { | |
| 2899 | − *bio = NULL; | |
| 2900 | − return NULL; | |
| 2901 | − } | |
| 2902 | − | |
| 2903 | − type = blk_mq_get_hctx_type((*bio)->bi_opf); | |
| 2904 | − hctx_type = rq->mq_hctx->type; | |
| 2905 | 2890 | if (type != hctx_type && |
| 2906 | 2891 | !(type == HCTX_TYPE_READ && hctx_type == HCTX_TYPE_DEFAULT)) |
| 2907 | − return NULL; | |
| 2908 | − if (op_is_flush(rq->cmd_flags) != op_is_flush((*bio)->bi_opf)) | |
| 2909 | − return NULL; | |
| 2892 | + return false; | |
| 2893 | + if (op_is_flush(rq->cmd_flags) != op_is_flush(bio->bi_opf)) | |
| 2894 | + return false; | |
| 2910 | 2895 | |
| 2911 | 2896 | /* |
| 2912 | 2897 | * If any qos ->throttle() end up blocking, we will have flushed the |
| @@ -2914,12 +2899,12 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, · 1 unmodified line | ||
| 2914 | 2899 | * before we throttle. |
| 2915 | 2900 | */ |
| 2916 | 2901 | plug->cached_rq = rq_list_next(rq); |
| 2917 | − rq_qos_throttle(q, *bio); | |
| 2902 | + rq_qos_throttle(rq->q, bio); | |
| 2918 | 2903 | |
| 2919 | 2904 | blk_mq_rq_time_init(rq, 0); |
| 2920 | − rq->cmd_flags = (*bio)->bi_opf; | |
| 2905 | + rq->cmd_flags = bio->bi_opf; | |
| 2921 | 2906 | INIT_LIST_HEAD(&rq->queuelist); |
| 2922 | − return rq; | |
| 2907 | + return true; | |
| 2923 | 2908 | } |
| 2924 | 2909 | |
| 2925 | 2910 | static void bio_set_ioprio(struct bio *bio) |
| @@ -2949,7 +2934,7 @@ void blk_mq_submit_bio(struct bio *bio) · 23 unmodified lines | ||
| 2949 | 2934 | struct blk_plug *plug = blk_mq_plug(bio); |
| 2950 | 2935 | const int is_sync = op_is_sync(bio->bi_opf); |
| 2951 | 2936 | struct blk_mq_hw_ctx *hctx; |
| 2952 | − struct request *rq; | |
| 2937 | + struct request *rq = NULL; | |
| 2953 | 2938 | unsigned int nr_segs = 1; |
| 2954 | 2939 | blk_status_t ret; |
| 2955 | 2940 | |
| @@ -2960,20 +2945,36 @@ void blk_mq_submit_bio(struct bio *bio) · 4 unmodified lines | ||
| 2960 | 2945 | return; |
| 2961 | 2946 | } |
| 2962 | 2947 | |
| 2963 | − if (!bio_integrity_prep(bio)) | |
| 2964 | − return; | |
| 2965 | − | |
| 2966 | 2948 | bio_set_ioprio(bio); |
| 2967 | 2949 | |
| 2968 | − rq = blk_mq_get_cached_request(q, plug, &bio, nr_segs); | |
| 2969 | − if (!rq) { | |
| 2970 | − if (!bio) | |
| 2950 | + if (plug) { | |
| 2951 | + rq = rq_list_peek(&plug->cached_rq); | |
| 2952 | + if (rq && rq->q != q) | |
| 2953 | + rq = NULL; | |
| 2954 | + } | |
| 2955 | + if (rq) { | |
| 2956 | + if (!bio_integrity_prep(bio)) | |
| 2971 | 2957 | return; |
| 2972 | − rq = blk_mq_get_new_requests(q, plug, bio, nr_segs); | |
| 2973 | − if (unlikely(!rq)) | |
| 2958 | + if (blk_mq_attempt_bio_merge(q, bio, nr_segs)) | |
| 2974 | 2959 | return; |
| 2960 | + if (blk_mq_can_use_cached_rq(rq, plug, bio)) | |
| 2961 | + goto done; | |
| 2962 | + percpu_ref_get(&q->q_usage_counter); | |
| 2963 | + } else { | |
| 2964 | + if (unlikely(bio_queue_enter(bio))) | |
| 2965 | + return; | |
| 2966 | + if (!bio_integrity_prep(bio)) | |
| 2967 | + goto fail; | |
| 2968 | + } | |
| 2969 | + | |
| 2970 | + rq = blk_mq_get_new_requests(q, plug, bio, nr_segs); | |
| 2971 | + if (unlikely(!rq)) { | |
| 2972 | +fail: | |
| 2973 | + blk_queue_exit(q); | |
| 2974 | + return; | |
| 2975 | 2975 | } |
| 2976 | 2976 | |
| 2977 | +done: | |
| 2977 | 2978 | trace_block_getrq(bio); |
| 2978 | 2979 | |
| 2979 | 2980 | rq_qos_track(q, rq, bio); |
| @@ -163,38 +163,15 @@ EXPORT_SYMBOL(blk_pre_runtime_resume); · 162 unmodified lines | ||
| 163 | 163 | * @q: the queue of the device |
| 164 | 164 | * |
| 165 | 165 | * Description: |
| 166 | − * For historical reasons, this routine merely calls blk_set_runtime_active() | |
| 167 | − * to do the real work of restarting the queue. It does this regardless of | |
| 168 | − * whether the device's runtime-resume succeeded; even if it failed the | |
| 166 | + * Restart the queue of a runtime suspended device. It does this regardless | |
| 167 | + * of whether the device's runtime-resume succeeded; even if it failed the | |
| 169 | 168 | * driver or error handler will need to communicate with the device. |
| 170 | 169 | * |
| 171 | 170 | * This function should be called near the end of the device's |
| 172 | − * runtime_resume callback. | |
| 171 | + * runtime_resume callback to correct queue runtime PM status and re-enable | |
| 172 | + * peeking requests from the queue. | |
| 173 | 173 | */ |
| 174 | 174 | void blk_post_runtime_resume(struct request_queue *q) |
| 175 | −{ | |
| 176 | − blk_set_runtime_active(q); | |
| 177 | −} | |
| 178 | −EXPORT_SYMBOL(blk_post_runtime_resume); | |
| 179 | − | |
| 180 | −/** | |
| 181 | − * blk_set_runtime_active - Force runtime status of the queue to be active | |
| 182 | − * @q: the queue of the device | |
| 183 | − * | |
| 184 | − * If the device is left runtime suspended during system suspend the resume | |
| 185 | − * hook typically resumes the device and corrects runtime status | |
| 186 | − * accordingly. However, that does not affect the queue runtime PM status | |
| 187 | − * which is still "suspended". This prevents processing requests from the | |
| 188 | − * queue. | |
| 189 | − * | |
| 190 | − * This function can be used in driver's resume hook to correct queue | |
| 191 | − * runtime PM status and re-enable peeking requests from the queue. It | |
| 192 | − * should be called before first request is added to the queue. | |
| 193 | − * | |
| 194 | − * This function is also called by blk_post_runtime_resume() for | |
| 195 | − * runtime resumes. It does everything necessary to restart the queue. | |
| 196 | − */ | |
| 197 | −void blk_set_runtime_active(struct request_queue *q) | |
| 198 | 175 | { |
| 199 | 176 | int old_status; |
| 200 | 177 | |
| @@ -211,4 +188,4 @@ void blk_set_runtime_active(struct request_queue *q) · 10 unmodified lines | ||
| 211 | 188 | if (old_status != RPM_ACTIVE) |
| 212 | 189 | blk_clear_pm_only(q); |
| 213 | 190 | } |
| 214 | −EXPORT_SYMBOL(blk_set_runtime_active); | |
| 191 | +EXPORT_SYMBOL(blk_post_runtime_resume); | |
| @@ -1320,6 +1320,7 @@ static void tg_conf_updated(struct throtl_grp *tg, bool global) · 1319 unmodified lines | ||
| 1320 | 1320 | tg_bps_limit(tg, READ), tg_bps_limit(tg, WRITE), |
| 1321 | 1321 | tg_iops_limit(tg, READ), tg_iops_limit(tg, WRITE)); |
| 1322 | 1322 | |
| 1323 | + rcu_read_lock(); | |
| 1323 | 1324 | /* |
| 1324 | 1325 | * Update has_rules[] flags for the updated tg's subtree. A tg is |
| 1325 | 1326 | * considered to have rules if either the tg itself or any of its |
| @@ -1347,6 +1348,7 @@ static void tg_conf_updated(struct throtl_grp *tg, bool global) · 21 unmodified lines | ||
| 1347 | 1348 | this_tg->latency_target = max(this_tg->latency_target, |
| 1348 | 1349 | parent_tg->latency_target); |
| 1349 | 1350 | } |
| 1351 | + rcu_read_unlock(); | |
| 1350 | 1352 | |
| 1351 | 1353 | /* |
| 1352 | 1354 | * We're already holding queue_lock and know @tg is valid. Let's |
| @@ -11,6 +11,7 @@ · 10 unmodified lines | ||
| 11 | 11 | #include <linux/idr.h> |
| 12 | 12 | |
| 13 | 13 | #include <drm/drm_accel.h> |
| 14 | +#include <drm/drm_auth.h> | |
| 14 | 15 | #include <drm/drm_debugfs.h> |
| 15 | 16 | #include <drm/drm_drv.h> |
| 16 | 17 | #include <drm/drm_file.h> |
| @@ -1,16 +1,17 @@ | ||
| 1 | 1 | # SPDX-License-Identifier: GPL-2.0-only |
| 2 | 2 | |
| 3 | 3 | config DRM_ACCEL_IVPU |
| 4 | − tristate "Intel VPU for Meteor Lake and newer" | |
| 4 | + tristate "Intel NPU (Neural Processing Unit)" | |
| 5 | 5 | depends on DRM_ACCEL |
| 6 | 6 | depends on X86_64 && !UML |
| 7 | 7 | depends on PCI && PCI_MSI |
| 8 | 8 | select FW_LOADER |
| 9 | − select SHMEM | |
| 9 | + select DRM_GEM_SHMEM_HELPER | |
| 10 | 10 | select GENERIC_ALLOCATOR |
| 11 | 11 | help |
| 12 | − Choose this option if you have a system that has an 14th generation Intel CPU | |
| 13 | − or newer. VPU stands for Versatile Processing Unit and it's a CPU-integrated | |
| 14 | − inference accelerator for Computer Vision and Deep Learning applications. | |
| 12 | + Choose this option if you have a system with an 14th generation | |
| 13 | + Intel CPU (Meteor Lake) or newer. Intel NPU (formerly called Intel VPU) | |
| 14 | + is a CPU-integrated inference accelerator for Computer Vision | |
| 15 | + and Deep Learning applications. | |
| 15 | 16 | |
| 16 | 17 | If "M" is selected, the module will be called intel_vpu. |
| @@ -14,6 +14,7 @@ · 13 unmodified lines | ||
| 14 | 14 | #include "ivpu_fw.h" |
| 15 | 15 | #include "ivpu_fw_log.h" |
| 16 | 16 | #include "ivpu_gem.h" |
| 17 | +#include "ivpu_hw.h" | |
| 17 | 18 | #include "ivpu_jsm_msg.h" |
| 18 | 19 | #include "ivpu_pm.h" |
| 19 | 20 | |
| @@ -115,6 +116,31 @@ static const struct drm_debugfs_info vdev_debugfs_list[] = { · 95 unmodified lines | ||
| 115 | 116 | {"reset_pending", reset_pending_show, 0}, |
| 116 | 117 | }; |
| 117 | 118 | |
| 119 | +static ssize_t | |
| 120 | +dvfs_mode_fops_write(struct file *file, const char __user *user_buf, size_t size, loff_t *pos) | |
| 121 | +{ | |
| 122 | + struct ivpu_device *vdev = file->private_data; | |
| 123 | + struct ivpu_fw_info *fw = vdev->fw; | |
| 124 | + u32 dvfs_mode; | |
| 125 | + int ret; | |
| 126 | + | |
| 127 | + ret = kstrtou32_from_user(user_buf, size, 0, &dvfs_mode); | |
| 128 | + if (ret < 0) | |
| 129 | + return ret; | |
| 130 | + | |
| 131 | + fw->dvfs_mode = dvfs_mode; | |
| 132 | + | |
| 133 | + ivpu_pm_schedule_recovery(vdev); | |
| 134 | + | |
| 135 | + return size; | |
| 136 | +} | |
| 137 | + | |
| 138 | +static const struct file_operations dvfs_mode_fops = { | |
| 139 | + .owner = THIS_MODULE, | |
| 140 | + .open = simple_open, | |
| 141 | + .write = dvfs_mode_fops_write, | |
| 142 | +}; | |
| 143 | + | |
| 118 | 144 | static int fw_log_show(struct seq_file *s, void *v) |
| 119 | 145 | { |
| 120 | 146 | struct ivpu_device *vdev = s->private; |
| @@ -151,6 +177,30 @@ static const struct file_operations fw_log_fops = { · 30 unmodified lines | ||
| 151 | 177 | .release = single_release, |
| 152 | 178 | }; |
| 153 | 179 | |
| 180 | +static ssize_t | |
| 181 | +fw_profiling_freq_fops_write(struct file *file, const char __user *user_buf, | |
| 182 | + size_t size, loff_t *pos) | |
| 183 | +{ | |
| 184 | + struct ivpu_device *vdev = file->private_data; | |
| 185 | + bool enable; | |
| 186 | + int ret; | |
| 187 | + | |
| 188 | + ret = kstrtobool_from_user(user_buf, size, &enable); | |
| 189 | + if (ret < 0) | |
| 190 | + return ret; | |
| 191 | + | |
| 192 | + ivpu_hw_profiling_freq_drive(vdev, enable); | |
| 193 | + ivpu_pm_schedule_recovery(vdev); | |
| 194 | + | |
| 195 | + return size; | |
| 196 | +} | |
| 197 | + | |
| 198 | +static const struct file_operations fw_profiling_freq_fops = { | |
| 199 | + .owner = THIS_MODULE, | |
| 200 | + .open = simple_open, | |
| 201 | + .write = fw_profiling_freq_fops_write, | |
| 202 | +}; | |
| 203 | + | |
| 154 | 204 | static ssize_t |
| 155 | 205 | fw_trace_destination_mask_fops_write(struct file *file, const char __user *user_buf, |
| 156 | 206 | size_t size, loff_t *pos) |
| @@ -280,6 +330,9 @@ void ivpu_debugfs_init(struct ivpu_device *vdev) · 123 unmodified lines | ||
| 280 | 330 | debugfs_create_file("force_recovery", 0200, debugfs_root, vdev, |
| 281 | 331 | &ivpu_force_recovery_fops); |
| 282 | 332 | |
| 333 | + debugfs_create_file("dvfs_mode", 0200, debugfs_root, vdev, | |
| 334 | + &dvfs_mode_fops); | |
| 335 | + | |
| 283 | 336 | debugfs_create_file("fw_log", 0644, debugfs_root, vdev, |
| 284 | 337 | &fw_log_fops); |
| 285 | 338 | debugfs_create_file("fw_trace_destination_mask", 0200, debugfs_root, vdev, |
| @@ -291,4 +344,8 @@ void ivpu_debugfs_init(struct ivpu_device *vdev) · 5 unmodified lines | ||
| 291 | 344 | |
| 292 | 345 | debugfs_create_file("reset_engine", 0200, debugfs_root, vdev, |
| 293 | 346 | &ivpu_reset_engine_fops); |
| 347 | + | |
| 348 | + if (ivpu_hw_gen(vdev) >= IVPU_HW_40XX) | |
| 349 | + debugfs_create_file("fw_profiling_freq_drive", 0200, | |
| 350 | + debugfs_root, vdev, &fw_profiling_freq_fops); | |
| 294 | 351 | } |
| @@ -31,8 +31,6 @@ · 30 unmodified lines | ||
| 31 | 31 | __stringify(DRM_IVPU_DRIVER_MINOR) "." |
| 32 | 32 | #endif |
| 33 | 33 | |
| 34 | −static const struct drm_driver driver; | |
| 35 | − | |
| 36 | 34 | static struct lock_class_key submitted_jobs_xa_lock_class_key; |
| 37 | 35 | |
| 38 | 36 | int ivpu_dbg_mask; |
| @@ -41,7 +39,7 @@ MODULE_PARM_DESC(dbg_mask, "Driver debug mask. See IVPU_DBG_* macros."); · 2 unmodified lines | ||
| 41 | 39 | |
| 42 | 40 | int ivpu_test_mode; |
| 43 | 41 | module_param_named_unsafe(test_mode, ivpu_test_mode, int, 0644); |
| 44 | −MODULE_PARM_DESC(test_mode, "Test mode: 0 - normal operation, 1 - fw unit test, 2 - null hw"); | |
| 42 | +MODULE_PARM_DESC(test_mode, "Test mode mask. See IVPU_TEST_MODE_* macros."); | |
| 45 | 43 | |
| 46 | 44 | u8 ivpu_pll_min_ratio; |
| 47 | 45 | module_param_named(pll_min_ratio, ivpu_pll_min_ratio, byte, 0644); |
| @@ -93,8 +91,8 @@ static void file_priv_release(struct kref *ref) · 45 unmodified lines | ||
| 93 | 91 | ivpu_dbg(vdev, FILE, "file_priv release: ctx %u\n", file_priv->ctx.id); |
| 94 | 92 | |
| 95 | 93 | ivpu_cmdq_release_all(file_priv); |
| 96 | − ivpu_bo_remove_all_bos_from_context(&file_priv->ctx); | |
| 97 | 94 | ivpu_jsm_context_release(vdev, file_priv->ctx.id); |
| 95 | + ivpu_bo_remove_all_bos_from_context(vdev, &file_priv->ctx); | |
| 98 | 96 | ivpu_mmu_user_context_fini(vdev, &file_priv->ctx); |
| 99 | 97 | drm_WARN_ON(&vdev->drm, xa_erase_irq(&vdev->context_xa, file_priv->ctx.id) != file_priv); |
| 100 | 98 | mutex_destroy(&file_priv->lock); |
| @@ -317,16 +315,14 @@ static int ivpu_wait_for_ready(struct ivpu_device *vdev) · 216 unmodified lines | ||
| 317 | 315 | unsigned long timeout; |
| 318 | 316 | int ret; |
| 319 | 317 | |
| 320 | − if (ivpu_test_mode == IVPU_TEST_MODE_FW_TEST) | |
| 318 | + if (ivpu_test_mode & IVPU_TEST_MODE_FW_TEST) | |
| 321 | 319 | return 0; |
| 322 | 320 | |
| 323 | − ivpu_ipc_consumer_add(vdev, &cons, IVPU_IPC_CHAN_BOOT_MSG); | |
| 321 | + ivpu_ipc_consumer_add(vdev, &cons, IVPU_IPC_CHAN_BOOT_MSG, NULL); | |
| 324 | 322 | |
| 325 | 323 | timeout = jiffies + msecs_to_jiffies(vdev->timeout.boot); |
| 326 | 324 | while (1) { |
| 327 | − ret = ivpu_ipc_irq_handler(vdev); | |
| 328 | − if (ret) | |
| 329 | − break; | |
| 325 | + ivpu_ipc_irq_handler(vdev, NULL); | |
| 330 | 326 | ret = ivpu_ipc_receive(vdev, &cons, &ipc_hdr, NULL, 0); |
| 331 | 327 | if (ret != -ETIMEDOUT || time_after_eq(jiffies, timeout)) |
| 332 | 328 | break; |
| @@ -362,7 +358,7 @@ int ivpu_boot(struct ivpu_device *vdev) · 29 unmodified lines | ||
| 362 | 358 | int ret; |
| 363 | 359 | |
| 364 | 360 | /* Update boot params located at first 4KB of FW memory */ |
| 365 | − ivpu_fw_boot_params_setup(vdev, vdev->fw->mem->kvaddr); | |
| 361 | + ivpu_fw_boot_params_setup(vdev, ivpu_bo_vaddr(vdev->fw->mem)); | |
| 366 | 362 | |
| 367 | 363 | ret = ivpu_hw_boot_fw(vdev); |
| 368 | 364 | if (ret) { |
| @@ -414,7 +410,9 @@ static const struct drm_driver driver = { · 45 unmodified lines | ||
| 414 | 410 | |
| 415 | 411 | .open = ivpu_open, |
| 416 | 412 | .postclose = ivpu_postclose, |
| 417 | − .gem_prime_import = ivpu_gem_prime_import, | |
| 413 | + | |
| 414 | + .gem_create_object = ivpu_gem_create_object, | |
| 415 | + .gem_prime_import_sg_table = drm_gem_shmem_prime_import_sg_table, | |
| 418 | 416 | |
| 419 | 417 | .ioctls = ivpu_drm_ioctls, |
| 420 | 418 | .num_ioctls = ARRAY_SIZE(ivpu_drm_ioctls), |
| @@ -427,6 +425,13 @@ static const struct drm_driver driver = { · 6 unmodified lines | ||
| 427 | 425 | .minor = DRM_IVPU_DRIVER_MINOR, |
| 428 | 426 | }; |
| 429 | 427 | |
| 428 | +static irqreturn_t ivpu_irq_thread_handler(int irq, void *arg) | |
| 429 | +{ | |
| 430 | + struct ivpu_device *vdev = arg; | |
| 431 | + | |
| 432 | + return ivpu_ipc_irq_thread_handler(vdev); | |
| 433 | +} | |
| 434 | + | |
| 430 | 435 | static int ivpu_irq_init(struct ivpu_device *vdev) |
| 431 | 436 | { |
| 432 | 437 | struct pci_dev *pdev = to_pci_dev(vdev->drm.dev); |
| @@ -440,8 +445,8 @@ static int ivpu_irq_init(struct ivpu_device *vdev) · 7 unmodified lines | ||
| 440 | 445 | |
| 441 | 446 | vdev->irq = pci_irq_vector(pdev, 0); |
| 442 | 447 | |
| 443 | − ret = devm_request_irq(vdev->drm.dev, vdev->irq, vdev->hw->ops->irq_handler, | |
| 444 | − IRQF_NO_AUTOEN, DRIVER_NAME, vdev); | |
| 448 | + ret = devm_request_threaded_irq(vdev->drm.dev, vdev->irq, vdev->hw->ops->irq_handler, | |
| 449 | + ivpu_irq_thread_handler, IRQF_NO_AUTOEN, DRIVER_NAME, vdev); | |
| 445 | 450 | if (ret) |
| 446 | 451 | ivpu_err(vdev, "Failed to request an IRQ %d\n", ret); |
| 447 | 452 | |
| @@ -533,6 +538,11 @@ static int ivpu_dev_init(struct ivpu_device *vdev) · 85 unmodified lines | ||
| 533 | 538 | xa_init_flags(&vdev->context_xa, XA_FLAGS_ALLOC); |
| 534 | 539 | xa_init_flags(&vdev->submitted_jobs_xa, XA_FLAGS_ALLOC1); |
| 535 | 540 | lockdep_set_class(&vdev->submitted_jobs_xa.xa_lock, &submitted_jobs_xa_lock_class_key); |
| 541 | + INIT_LIST_HEAD(&vdev->bo_list); | |
| 542 | + | |
| 543 | + ret = drmm_mutex_init(&vdev->drm, &vdev->bo_list_lock); | |
| 544 | + if (ret) | |
| 545 | + goto err_xa_destroy; | |
| 536 | 546 | |
| 537 | 547 | ret = ivpu_pci_init(vdev); |
| 538 | 548 | if (ret) |
| @@ -550,7 +560,7 @@ static int ivpu_dev_init(struct ivpu_device *vdev) · 11 unmodified lines | ||
| 550 | 560 | /* Power up early so the rest of init code can access VPU registers */ |
| 551 | 561 | ret = ivpu_hw_power_up(vdev); |
| 552 | 562 | if (ret) |
| 553 | − goto err_xa_destroy; | |
| 563 | + goto err_power_down; | |
| 554 | 564 | |
| 555 | 565 | ret = ivpu_mmu_global_context_init(vdev); |
| 556 | 566 | if (ret) |
| @@ -574,20 +584,15 @@ static int ivpu_dev_init(struct ivpu_device *vdev) · 17 unmodified lines | ||
| 574 | 584 | |
| 575 | 585 | ivpu_pm_init(vdev); |
| 576 | 586 | |
| 577 | − ret = ivpu_job_done_thread_init(vdev); | |
| 578 | − if (ret) | |
| 579 | − goto err_ipc_fini; | |
| 580 | − | |
| 581 | 587 | ret = ivpu_boot(vdev); |
| 582 | 588 | if (ret) |
| 583 | − goto err_job_done_thread_fini; | |
| 589 | + goto err_ipc_fini; | |
| 584 | 590 | |
| 591 | + ivpu_job_done_consumer_init(vdev); | |
| 585 | 592 | ivpu_pm_enable(vdev); |
| 586 | 593 | |
| 587 | 594 | return 0; |
| 588 | 595 | |
| 589 | −err_job_done_thread_fini: | |
| 590 | − ivpu_job_done_thread_fini(vdev); | |
| 591 | 596 | err_ipc_fini: |
| 592 | 597 | ivpu_ipc_fini(vdev); |
| 593 | 598 | err_fw_fini: |
| @@ -612,7 +617,7 @@ static void ivpu_dev_fini(struct ivpu_device *vdev) · 18 unmodified lines | ||
| 612 | 617 | ivpu_shutdown(vdev); |
| 613 | 618 | if (IVPU_WA(d3hot_after_power_off)) |
| 614 | 619 | pci_set_power_state(to_pci_dev(vdev->drm.dev), PCI_D3hot); |
| 615 | − ivpu_job_done_thread_fini(vdev); | |
| 620 | + ivpu_job_done_consumer_fini(vdev); | |
| 616 | 621 | ivpu_pm_cancel_recovery(vdev); |
| 617 | 622 | |
| 618 | 623 | ivpu_ipc_fini(vdev); |
| @@ -17,9 +17,10 @@ · 16 unmodified lines | ||
| 17 | 17 | #include <uapi/drm/ivpu_accel.h> |
| 18 | 18 | |
| 19 | 19 | #include "ivpu_mmu_context.h" |
| 20 | +#include "ivpu_ipc.h" | |
| 20 | 21 | |
| 21 | 22 | #define DRIVER_NAME "intel_vpu" |
| 22 | −#define DRIVER_DESC "Driver for Intel Versatile Processing Unit (VPU)" | |
| 23 | +#define DRIVER_DESC "Driver for Intel NPU (Neural Processing Unit)" | |
| 23 | 24 | #define DRIVER_DATE "20230117" |
| 24 | 25 | |
| 25 | 26 | #define PCI_DEVICE_ID_MTL 0x7d1d |
| @@ -88,6 +89,7 @@ struct ivpu_wa_table { · 62 unmodified lines | ||
| 88 | 89 | bool d3hot_after_power_off; |
| 89 | 90 | bool interrupt_clear_with_0; |
| 90 | 91 | bool disable_clock_relinquish; |
| 92 | + bool disable_d0i3_msg; | |
| 91 | 93 | }; |
| 92 | 94 | |
| 93 | 95 | struct ivpu_hw_info; |
| @@ -115,8 +117,11 @@ struct ivpu_device { · 21 unmodified lines | ||
| 115 | 117 | struct xarray context_xa; |
| 116 | 118 | struct xa_limit context_xa_limit; |
| 117 | 119 | |
| 120 | + struct mutex bo_list_lock; /* Protects bo_list */ | |
| 121 | + struct list_head bo_list; | |
| 122 | + | |
| 118 | 123 | struct xarray submitted_jobs_xa; |
| 119 | − struct task_struct *job_done_thread; | |
| 124 | + struct ivpu_ipc_consumer job_done_consumer; | |
| 120 | 125 | |
| 121 | 126 | atomic64_t unique_id_counter; |
| 122 | 127 | |
| @@ -126,6 +131,7 @@ struct ivpu_device { · 3 unmodified lines | ||
| 126 | 131 | int tdr; |
| 127 | 132 | int reschedule_suspend; |
| 128 | 133 | int autosuspend; |
| 134 | + int d0i3_entry_msg; | |
| 129 | 135 | } timeout; |
| 130 | 136 | }; |
| 131 | 137 | |
| @@ -148,9 +154,11 @@ extern u8 ivpu_pll_min_ratio; · 16 unmodified lines | ||
| 148 | 154 | extern u8 ivpu_pll_max_ratio; |
| 149 | 155 | extern bool ivpu_disable_mmu_cont_pages; |
| 150 | 156 | |
| 151 | −#define IVPU_TEST_MODE_DISABLED 0 | |
| 152 | −#define IVPU_TEST_MODE_FW_TEST 1 | |
| 153 | −#define IVPU_TEST_MODE_NULL_HW 2 | |
| 157 | +#define IVPU_TEST_MODE_FW_TEST BIT(0) | |
| 158 | +#define IVPU_TEST_MODE_NULL_HW BIT(1) | |
| 159 | +#define IVPU_TEST_MODE_NULL_SUBMISSION BIT(2) | |
| 160 | +#define IVPU_TEST_MODE_D0I3_MSG_DISABLE BIT(4) | |
| 161 | +#define IVPU_TEST_MODE_D0I3_MSG_ENABLE BIT(5) | |
| 154 | 162 | extern int ivpu_test_mode; |
| 155 | 163 | |
| 156 | 164 | struct ivpu_file_priv *ivpu_file_priv_get(struct ivpu_file_priv *file_priv); |
| @@ -33,12 +33,17 @@ · 32 unmodified lines | ||
| 33 | 33 | |
| 34 | 34 | #define ADDR_TO_L2_CACHE_CFG(addr) ((addr) >> 31) |
| 35 | 35 | |
| 36 | −#define IVPU_FW_CHECK_API(vdev, fw_hdr, name, min_major) \ | |
| 36 | +/* Check if FW API is compatible with the driver */ | |
| 37 | +#define IVPU_FW_CHECK_API_COMPAT(vdev, fw_hdr, name, min_major) \ | |
| 37 | 38 | ivpu_fw_check_api(vdev, fw_hdr, #name, \ |
| 38 | 39 | VPU_##name##_API_VER_INDEX, \ |
| 39 | 40 | VPU_##name##_API_VER_MAJOR, \ |
| 40 | 41 | VPU_##name##_API_VER_MINOR, min_major) |
| 41 | 42 | |
| 43 | +/* Check if API version is lower that the given version */ | |
| 44 | +#define IVPU_FW_CHECK_API_VER_LT(vdev, fw_hdr, name, major, minor) \ | |
| 45 | + ivpu_fw_check_api_ver_lt(vdev, fw_hdr, #name, VPU_##name##_API_VER_INDEX, major, minor) | |
| 46 | + | |
| 42 | 47 | static char *ivpu_firmware; |
| 43 | 48 | module_param_named_unsafe(firmware, ivpu_firmware, charp, 0644); |
| 44 | 49 | MODULE_PARM_DESC(firmware, "VPU firmware binary in /lib/firmware/.."); |
| @@ -105,6 +110,19 @@ ivpu_fw_check_api(struct ivpu_device *vdev, const struct vpu_firmware_header *fw · 60 unmodified lines | ||
| 105 | 110 | return 0; |
| 106 | 111 | } |
| 107 | 112 | |
| 113 | +static bool | |
| 114 | +ivpu_fw_check_api_ver_lt(struct ivpu_device *vdev, const struct vpu_firmware_header *fw_hdr, | |
| 115 | + const char *str, int index, u16 major, u16 minor) | |
| 116 | +{ | |
| 117 | + u16 fw_major = (u16)(fw_hdr->api_version[index] >> 16); | |
| 118 | + u16 fw_minor = (u16)(fw_hdr->api_version[index]); | |
| 119 | + | |
| 120 | + if (fw_major < major || (fw_major == major && fw_minor < minor)) | |
| 121 | + return true; | |
| 122 | + | |
| 123 | + return false; | |
| 124 | +} | |
| 125 | + | |
| 108 | 126 | static int ivpu_fw_parse(struct ivpu_device *vdev) |
| 109 | 127 | { |
| 110 | 128 | struct ivpu_fw_info *fw = vdev->fw; |
| @@ -164,9 +182,9 @@ static int ivpu_fw_parse(struct ivpu_device *vdev) · 53 unmodified lines | ||
| 164 | 182 | ivpu_info(vdev, "Firmware: %s, version: %s", fw->name, |
| 165 | 183 | (const char *)fw_hdr + VPU_FW_HEADER_SIZE); |
| 166 | 184 | |
| 167 | − if (IVPU_FW_CHECK_API(vdev, fw_hdr, BOOT, 3)) | |
| 185 | + if (IVPU_FW_CHECK_API_COMPAT(vdev, fw_hdr, BOOT, 3)) | |
| 168 | 186 | return -EINVAL; |
| 169 | − if (IVPU_FW_CHECK_API(vdev, fw_hdr, JSM, 3)) | |
| 187 | + if (IVPU_FW_CHECK_API_COMPAT(vdev, fw_hdr, JSM, 3)) | |
| 170 | 188 | return -EINVAL; |
| 171 | 189 | |
| 172 | 190 | fw->runtime_addr = runtime_addr; |
| @@ -182,6 +200,8 @@ static int ivpu_fw_parse(struct ivpu_device *vdev) · 9 unmodified lines | ||
| 182 | 200 | fw->trace_destination_mask = VPU_TRACE_DESTINATION_VERBOSE_TRACING; |
| 183 | 201 | fw->trace_hw_component_mask = -1; |
| 184 | 202 | |
| 203 | + fw->dvfs_mode = 0; | |
| 204 | + | |
| 185 | 205 | ivpu_dbg(vdev, FW_BOOT, "Size: file %lu image %u runtime %u shavenn %u\n", |
| 186 | 206 | fw->file->size, fw->image_size, fw->runtime_size, fw->shave_nn_size); |
| 187 | 207 | ivpu_dbg(vdev, FW_BOOT, "Address: runtime 0x%llx, load 0x%llx, entry point 0x%llx\n", |
| @@ -195,6 +215,24 @@ static void ivpu_fw_release(struct ivpu_device *vdev) · 7 unmodified lines | ||
| 195 | 215 | release_firmware(vdev->fw->file); |
| 196 | 216 | } |
| 197 | 217 | |
| 218 | +/* Initialize workarounds that depend on FW version */ | |
| 219 | +static void | |
| 220 | +ivpu_fw_init_wa(struct ivpu_device *vdev) | |
| 221 | +{ | |
| 222 | + const struct vpu_firmware_header *fw_hdr = (const void *)vdev->fw->file->data; | |
| 223 | + | |
| 224 | + if (IVPU_FW_CHECK_API_VER_LT(vdev, fw_hdr, BOOT, 3, 17) || | |
| 225 | + (ivpu_hw_gen(vdev) > IVPU_HW_37XX) || | |
| 226 | + (ivpu_test_mode & IVPU_TEST_MODE_D0I3_MSG_DISABLE)) | |
| 227 | + vdev->wa.disable_d0i3_msg = true; | |
| 228 | + | |
| 229 | + /* Force enable the feature for testing purposes */ | |
| 230 | + if (ivpu_test_mode & IVPU_TEST_MODE_D0I3_MSG_ENABLE) | |
| 231 | + vdev->wa.disable_d0i3_msg = false; | |
| 232 | + | |
| 233 | + IVPU_PRINT_WA(disable_d0i3_msg); | |
| 234 | +} | |
| 235 | + | |
| 198 | 236 | static int ivpu_fw_update_global_range(struct ivpu_device *vdev) |
| 199 | 237 | { |
| 200 | 238 | struct ivpu_fw_info *fw = vdev->fw; |
| @@ -248,7 +286,7 @@ static int ivpu_fw_mem_init(struct ivpu_device *vdev) · 47 unmodified lines | ||
| 248 | 286 | |
| 249 | 287 | if (fw->shave_nn_size) { |
| 250 | 288 | fw->mem_shave_nn = ivpu_bo_alloc_internal(vdev, vdev->hw->ranges.shave.start, |
| 251 | − fw->shave_nn_size, DRM_IVPU_BO_UNCACHED); | |
| 289 | + fw->shave_nn_size, DRM_IVPU_BO_WC); | |
| 252 | 290 | if (!fw->mem_shave_nn) { |
| 253 | 291 | ivpu_err(vdev, "Failed to allocate shavenn buffer\n"); |
| 254 | 292 | ret = -ENOMEM; |
| @@ -297,6 +335,8 @@ int ivpu_fw_init(struct ivpu_device *vdev) · 42 unmodified lines | ||
| 297 | 335 | if (ret) |
| 298 | 336 | goto err_fw_release; |
| 299 | 337 | |
| 338 | + ivpu_fw_init_wa(vdev); | |
| 339 | + | |
| 300 | 340 | ret = ivpu_fw_mem_init(vdev); |
| 301 | 341 | if (ret) |
| 302 | 342 | goto err_fw_release; |
| @@ -422,14 +462,31 @@ static void ivpu_fw_boot_params_print(struct ivpu_device *vdev, struct vpu_boot_ · 119 unmodified lines | ||
| 422 | 462 | boot_params->punit_telemetry_sram_size); |
| 423 | 463 | ivpu_dbg(vdev, FW_BOOT, "boot_params.vpu_telemetry_enable = 0x%x\n", |
| 424 | 464 | boot_params->vpu_telemetry_enable); |
| 465 | + ivpu_dbg(vdev, FW_BOOT, "boot_params.dvfs_mode = %u\n", | |
| 466 | + boot_params->dvfs_mode); | |
| 467 | + ivpu_dbg(vdev, FW_BOOT, "boot_params.d0i3_delayed_entry = %d\n", | |
| 468 | + boot_params->d0i3_delayed_entry); | |
| 469 | + ivpu_dbg(vdev, FW_BOOT, "boot_params.d0i3_residency_time_us = %lld\n", | |
| 470 | + boot_params->d0i3_residency_time_us); | |
| 471 | + ivpu_dbg(vdev, FW_BOOT, "boot_params.d0i3_entry_vpu_ts = %llu\n", | |
| 472 | + boot_params->d0i3_entry_vpu_ts); | |
| 425 | 473 | } |
| 426 | 474 | |
| 427 | 475 | void ivpu_fw_boot_params_setup(struct ivpu_device *vdev, struct vpu_boot_params *boot_params) |
| 428 | 476 | { |
| 429 | 477 | struct ivpu_bo *ipc_mem_rx = vdev->ipc->mem_rx; |
| 430 | 478 | |
| 431 | − /* In case of warm boot we only have to reset the entrypoint addr */ | |
| 479 | + /* In case of warm boot only update variable params */ | |
| 432 | 480 | if (!ivpu_fw_is_cold_boot(vdev)) { |
| 481 | + boot_params->d0i3_residency_time_us = | |
| 482 | + ktime_us_delta(ktime_get_boottime(), vdev->hw->d0i3_entry_host_ts); | |
| 483 | + boot_params->d0i3_entry_vpu_ts = vdev->hw->d0i3_entry_vpu_ts; | |
| 484 | + | |
| 485 | + ivpu_dbg(vdev, FW_BOOT, "boot_params.d0i3_residency_time_us = %lld\n", | |
| 486 | + boot_params->d0i3_residency_time_us); | |
| 487 | + ivpu_dbg(vdev, FW_BOOT, "boot_params.d0i3_entry_vpu_ts = %llu\n", | |
| 488 | + boot_params->d0i3_entry_vpu_ts); | |
| 489 | + | |
| 433 | 490 | boot_params->save_restore_ret_address = 0; |
| 434 | 491 | vdev->pm->is_warmboot = true; |
| 435 | 492 | wmb(); /* Flush WC buffers after writing save_restore_ret_address */ |
| @@ -442,6 +499,13 @@ void ivpu_fw_boot_params_setup(struct ivpu_device *vdev, struct vpu_boot_params · 6 unmodified lines | ||
| 442 | 499 | boot_params->vpu_id = to_pci_dev(vdev->drm.dev)->bus->number; |
| 443 | 500 | boot_params->frequency = ivpu_hw_reg_pll_freq_get(vdev); |
| 444 | 501 | |
| 502 | + /* | |
| 503 | + * This param is a debug firmware feature. It switches default clock | |
| 504 | + * to higher resolution one for fine-grained and more accurate firmware | |
| 505 | + * task profiling. | |
| 506 | + */ | |
| 507 | + boot_params->perf_clk_frequency = ivpu_hw_profiling_freq_get(vdev); | |
| 508 | + | |
| 445 | 509 | /* |
| 446 | 510 | * Uncached region of VPU address space, covers IPC buffers, job queues |
| 447 | 511 | * and log buffers, programmable to L2$ Uncached by VPU MTRR |
| @@ -493,6 +557,11 @@ void ivpu_fw_boot_params_setup(struct ivpu_device *vdev, struct vpu_boot_params · 45 unmodified lines | ||
| 493 | 557 | boot_params->punit_telemetry_sram_base = ivpu_hw_reg_telemetry_offset_get(vdev); |
| 494 | 558 | boot_params->punit_telemetry_sram_size = ivpu_hw_reg_telemetry_size_get(vdev); |
| 495 | 559 | boot_params->vpu_telemetry_enable = ivpu_hw_reg_telemetry_enable_get(vdev); |
| 560 | + boot_params->dvfs_mode = vdev->fw->dvfs_mode; | |
| 561 | + if (!IVPU_WA(disable_d0i3_msg)) | |
| 562 | + boot_params->d0i3_delayed_entry = 1; | |
| 563 | + boot_params->d0i3_residency_time_us = 0; | |
| 564 | + boot_params->d0i3_entry_vpu_ts = 0; | |
| 496 | 565 | |
| 497 | 566 | wmb(); /* Flush WC buffers after writing bootparams */ |
| 498 | 567 | |
| @@ -27,6 +27,7 @@ struct ivpu_fw_info { · 26 unmodified lines | ||
| 27 | 27 | u32 trace_level; |
| 28 | 28 | u32 trace_destination_mask; |
| 29 | 29 | u64 trace_hw_component_mask; |
| 30 | + u32 dvfs_mode; | |
| 30 | 31 | }; |
| 31 | 32 | |
| 32 | 33 | int ivpu_fw_init(struct ivpu_device *vdev); |
| @@ -20,215 +20,18 @@ · 19 unmodified lines | ||
| 20 | 20 | #include "ivpu_mmu.h" |
| 21 | 21 | #include "ivpu_mmu_context.h" |
| 22 | 22 | |
| 23 | −MODULE_IMPORT_NS(DMA_BUF); | |
| 24 | − | |
| 25 | 23 | static const struct drm_gem_object_funcs ivpu_gem_funcs; |
| 26 | 24 | |
| 27 | −static struct lock_class_key prime_bo_lock_class_key; | |
| 28 | − | |
| 29 | −static int __must_check prime_alloc_pages_locked(struct ivpu_bo *bo) | |
| 30 | −{ | |
| 31 | − /* Pages are managed by the underlying dma-buf */ | |
| 32 | − return 0; | |
| 33 | −} | |
| 34 | − | |
| 35 | −static void prime_free_pages_locked(struct ivpu_bo *bo) | |
| 36 | −{ | |
| 37 | − /* Pages are managed by the underlying dma-buf */ | |
| 38 | −} | |
| 39 | − | |
| 40 | −static int prime_map_pages_locked(struct ivpu_bo *bo) | |
| 41 | −{ | |
| 42 | − struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); | |
| 43 | − struct sg_table *sgt; | |
| 44 | − | |
| 45 | − sgt = dma_buf_map_attachment_unlocked(bo->base.import_attach, DMA_BIDIRECTIONAL); | |
| 46 | − if (IS_ERR(sgt)) { | |
| 47 | − ivpu_err(vdev, "Failed to map attachment: %ld\n", PTR_ERR(sgt)); | |
| 48 | − return PTR_ERR(sgt); | |
| 49 | − } | |
| 50 | − | |
| 51 | − bo->sgt = sgt; | |
| 52 | − return 0; | |
| 53 | −} | |
| 54 | − | |
| 55 | −static void prime_unmap_pages_locked(struct ivpu_bo *bo) | |
| 56 | −{ | |
| 57 | − dma_buf_unmap_attachment_unlocked(bo->base.import_attach, bo->sgt, DMA_BIDIRECTIONAL); | |
| 58 | − bo->sgt = NULL; | |
| 59 | −} | |
| 60 | − | |
| 61 | −static const struct ivpu_bo_ops prime_ops = { | |
| 62 | − .type = IVPU_BO_TYPE_PRIME, | |
| 63 | − .name = "prime", | |
| 64 | − .alloc_pages = prime_alloc_pages_locked, | |
| 65 | − .free_pages = prime_free_pages_locked, | |
| 66 | − .map_pages = prime_map_pages_locked, | |
| 67 | − .unmap_pages = prime_unmap_pages_locked, | |
| 68 | −}; | |
| 69 | − | |
| 70 | −static int __must_check shmem_alloc_pages_locked(struct ivpu_bo *bo) | |
| 71 | −{ | |
| 72 | − int npages = ivpu_bo_size(bo) >> PAGE_SHIFT; | |
| 73 | − struct page **pages; | |
| 74 | − | |
| 75 | − pages = drm_gem_get_pages(&bo->base); | |
| 76 | − if (IS_ERR(pages)) | |
| 77 | − return PTR_ERR(pages); | |
| 78 | − | |
| 79 | − if (bo->flags & DRM_IVPU_BO_WC) | |
| 80 | − set_pages_array_wc(pages, npages); | |
| 81 | − else if (bo->flags & DRM_IVPU_BO_UNCACHED) | |
| 82 | − set_pages_array_uc(pages, npages); | |
| 83 | − | |
| 84 | − bo->pages = pages; | |
| 85 | − return 0; | |
| 86 | −} | |
| 87 | − | |
| 88 | −static void shmem_free_pages_locked(struct ivpu_bo *bo) | |
| 89 | −{ | |
| 90 | − if (ivpu_bo_cache_mode(bo) != DRM_IVPU_BO_CACHED) | |
| 91 | − set_pages_array_wb(bo->pages, ivpu_bo_size(bo) >> PAGE_SHIFT); | |
| 92 | − | |
| 93 | − drm_gem_put_pages(&bo->base, bo->pages, true, false); | |
| 94 | − bo->pages = NULL; | |
| 95 | −} | |
| 96 | − | |
| 97 | −static int ivpu_bo_map_pages_locked(struct ivpu_bo *bo) | |
| 98 | −{ | |
| 99 | − int npages = ivpu_bo_size(bo) >> PAGE_SHIFT; | |
| 100 | − struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); | |
| 101 | − struct sg_table *sgt; | |
| 102 | − int ret; | |
| 103 | − | |
| 104 | − sgt = drm_prime_pages_to_sg(&vdev->drm, bo->pages, npages); | |
| 105 | − if (IS_ERR(sgt)) { | |
| 106 | − ivpu_err(vdev, "Failed to allocate sgtable\n"); | |
| 107 | − return PTR_ERR(sgt); | |
| 108 | − } | |
| 109 | − | |
| 110 | − ret = dma_map_sgtable(vdev->drm.dev, sgt, DMA_BIDIRECTIONAL, 0); | |
| 111 | − if (ret) { | |
| 112 | − ivpu_err(vdev, "Failed to map BO in IOMMU: %d\n", ret); | |
| 113 | − goto err_free_sgt; | |
| 114 | − } | |
| 115 | − | |
| 116 | − bo->sgt = sgt; | |
| 117 | − return 0; | |
| 118 | − | |
| 119 | −err_free_sgt: | |
| 120 | − kfree(sgt); | |
| 121 | − return ret; | |
| 122 | −} | |
| 123 | − | |
| 124 | −static void ivpu_bo_unmap_pages_locked(struct ivpu_bo *bo) | |
| 125 | −{ | |
| 126 | − struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); | |
| 127 | − | |
| 128 | − dma_unmap_sgtable(vdev->drm.dev, bo->sgt, DMA_BIDIRECTIONAL, 0); | |
| 129 | − sg_free_table(bo->sgt); | |
| 130 | − kfree(bo->sgt); | |
| 131 | − bo->sgt = NULL; | |
| 132 | −} | |
| 133 | − | |
| 134 | −static const struct ivpu_bo_ops shmem_ops = { | |
| 135 | − .type = IVPU_BO_TYPE_SHMEM, | |
| 136 | − .name = "shmem", | |
| 137 | − .alloc_pages = shmem_alloc_pages_locked, | |
| 138 | − .free_pages = shmem_free_pages_locked, | |
| 139 | − .map_pages = ivpu_bo_map_pages_locked, | |
| 140 | − .unmap_pages = ivpu_bo_unmap_pages_locked, | |
| 141 | −}; | |
| 142 | − | |
| 143 | −static int __must_check internal_alloc_pages_locked(struct ivpu_bo *bo) | |
| 144 | −{ | |
| 145 | − unsigned int i, npages = ivpu_bo_size(bo) >> PAGE_SHIFT; | |
| 146 | − struct page **pages; | |
| 147 | − int ret; | |
| 148 | − | |
| 149 | − pages = kvmalloc_array(npages, sizeof(*bo->pages), GFP_KERNEL); | |
| 150 | − if (!pages) | |
| 151 | − return -ENOMEM; | |
| 152 | − | |
| 153 | − for (i = 0; i < npages; i++) { | |
| 154 | − pages[i] = alloc_page(GFP_KERNEL | __GFP_HIGHMEM | __GFP_ZERO); | |
| 155 | − if (!pages[i]) { | |
| 156 | − ret = -ENOMEM; | |
| 157 | − goto err_free_pages; | |
| 158 | − } | |
| 159 | − cond_resched(); | |
| 160 | − } | |
| 161 | − | |
| 162 | − bo->pages = pages; | |
| 163 | − return 0; | |
| 164 | − | |
| 165 | −err_free_pages: | |
| 166 | − while (i--) | |
| 167 | − put_page(pages[i]); | |
| 168 | − kvfree(pages); | |
| 169 | − return ret; | |
| 170 | −} | |
| 171 | − | |
| 172 | −static void internal_free_pages_locked(struct ivpu_bo *bo) | |
| 173 | −{ | |
| 174 | − unsigned int i, npages = ivpu_bo_size(bo) >> PAGE_SHIFT; | |
| 175 | − | |
| 176 | − if (ivpu_bo_cache_mode(bo) != DRM_IVPU_BO_CACHED) | |
| 177 | − set_pages_array_wb(bo->pages, ivpu_bo_size(bo) >> PAGE_SHIFT); | |
| 178 | − | |
| 179 | − for (i = 0; i < npages; i++) | |
| 180 | − put_page(bo->pages[i]); | |
| 181 | − | |
| 182 | − kvfree(bo->pages); | |
| 183 | − bo->pages = NULL; | |
| 184 | −} | |
| 185 | − | |
| 186 | −static const struct ivpu_bo_ops internal_ops = { | |
| 187 | − .type = IVPU_BO_TYPE_INTERNAL, | |
| 188 | − .name = "internal", | |
| 189 | − .alloc_pages = internal_alloc_pages_locked, | |
| 190 | − .free_pages = internal_free_pages_locked, | |
| 191 | − .map_pages = ivpu_bo_map_pages_locked, | |
| 192 | − .unmap_pages = ivpu_bo_unmap_pages_locked, | |
| 193 | −}; | |
| 194 | − | |
| 195 | −static int __must_check ivpu_bo_alloc_and_map_pages_locked(struct ivpu_bo *bo) | |
| 196 | −{ | |
| 197 | − struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); | |
| 198 | − int ret; | |
| 199 | − | |
| 200 | − lockdep_assert_held(&bo->lock); | |
| 201 | − drm_WARN_ON(&vdev->drm, bo->sgt); | |
| 202 | − | |
| 203 | − ret = bo->ops->alloc_pages(bo); | |
| 204 | − if (ret) { | |
| 205 | − ivpu_err(vdev, "Failed to allocate pages for BO: %d", ret); | |
| 206 | − return ret; | |
| 207 | − } | |
| 208 | − | |
| 209 | − ret = bo->ops->map_pages(bo); | |
| 210 | − if (ret) { | |
| 211 | − ivpu_err(vdev, "Failed to map pages for BO: %d", ret); | |
| 212 | − goto err_free_pages; | |
| 213 | − } | |
| 214 | − return ret; | |
| 215 | − | |
| 216 | −err_free_pages: | |
| 217 | − bo->ops->free_pages(bo); | |
| 218 | − return ret; | |
| 219 | −} | |
| 220 | − | |
| 221 | −static void ivpu_bo_unmap_and_free_pages(struct ivpu_bo *bo) | |
| 25 | +static inline void ivpu_dbg_bo(struct ivpu_device *vdev, struct ivpu_bo *bo, const char *action) | |
| 222 | 26 | { |
| 223 | − mutex_lock(&bo->lock); | |
| 224 | − | |
| 225 | − WARN_ON(!bo->sgt); | |
| 226 | − bo->ops->unmap_pages(bo); | |
| 227 | − WARN_ON(bo->sgt); | |
| 228 | − bo->ops->free_pages(bo); | |
| 229 | − WARN_ON(bo->pages); | |
| 230 | − | |
| 231 | − mutex_unlock(&bo->lock); | |
| 27 | + if (bo->ctx) | |
| 28 | + ivpu_dbg(vdev, BO, "%6s: size %zu has_pages %d dma_mapped %d handle %u ctx %d vpu_addr 0x%llx mmu_mapped %d\n", | |
| 29 | + action, ivpu_bo_size(bo), (bool)bo->base.pages, (bool)bo->base.sgt, | |
| 30 | + bo->handle, bo->ctx->id, bo->vpu_addr, bo->mmu_mapped); | |
| 31 | + else | |
| 32 | + ivpu_dbg(vdev, BO, "%6s: size %zu has_pages %d dma_mapped %d handle %u (not added to context)\n", | |
| 33 | + action, ivpu_bo_size(bo), (bool)bo->base.pages, (bool)bo->base.sgt, | |
| 34 | + bo->handle); | |
| 232 | 35 | } |
| 233 | 36 | |
| 234 | 37 | /* |
| @@ -245,21 +48,24 @@ int __must_check ivpu_bo_pin(struct ivpu_bo *bo) · 10 unmodified lines | ||
| 245 | 48 | |
| 246 | 49 | mutex_lock(&bo->lock); |
| 247 | 50 | |
| 248 | − if (!bo->vpu_addr) { | |
| 249 | − ivpu_err(vdev, "vpu_addr not set for BO ctx_id: %d handle: %d\n", | |
| 250 | − bo->ctx->id, bo->handle); | |
| 51 | + ivpu_dbg_bo(vdev, bo, "pin"); | |
| 52 | + | |
| 53 | + if (!bo->ctx) { | |
| 54 | + ivpu_err(vdev, "vpu_addr not allocated for BO %d\n", bo->handle); | |
| 251 | 55 | ret = -EINVAL; |
| 252 | 56 | goto unlock; |
| 253 | 57 | } |
| 254 | 58 | |
| 255 | − if (!bo->sgt) { | |
| 256 | − ret = ivpu_bo_alloc_and_map_pages_locked(bo); | |
| 257 | − if (ret) | |
| 59 | + if (!bo->mmu_mapped) { | |
| 60 | + struct sg_table *sgt = drm_gem_shmem_get_pages_sgt(&bo->base); | |
| 61 | + | |
| 62 | + if (IS_ERR(sgt)) { | |
| 63 | + ret = PTR_ERR(sgt); | |
| 64 | + ivpu_err(vdev, "Failed to map BO in IOMMU: %d\n", ret); | |
| 258 | 65 | goto unlock; |
| 259 | − } | |
| 66 | + } | |
| 260 | 67 | |
| 261 | − if (!bo->mmu_mapped) { | |
| 262 | − ret = ivpu_mmu_context_map_sgt(vdev, bo->ctx, bo->vpu_addr, bo->sgt, | |
| 68 | + ret = ivpu_mmu_context_map_sgt(vdev, bo->ctx, bo->vpu_addr, sgt, | |
| 263 | 69 | ivpu_bo_is_snooped(bo)); |
| 264 | 70 | if (ret) { |
| 265 | 71 | ivpu_err(vdev, "Failed to map BO in MMU: %d\n", ret); |
| @@ -281,248 +87,213 @@ ivpu_bo_alloc_vpu_addr(struct ivpu_bo *bo, struct ivpu_mmu_context *ctx, · 15 unmodified lines | ||
| 281 | 87 | struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); |
| 282 | 88 | int ret; |
| 283 | 89 | |
| 284 | − if (!range) { | |
| 285 | − if (bo->flags & DRM_IVPU_BO_SHAVE_MEM) | |
| 286 | − range = &vdev->hw->ranges.shave; | |
| 287 | − else if (bo->flags & DRM_IVPU_BO_DMA_MEM) | |
| 288 | − range = &vdev->hw->ranges.dma; | |
| 289 | − else | |
| 290 | − range = &vdev->hw->ranges.user; | |
| 291 | − } | |
| 90 | + mutex_lock(&bo->lock); | |
| 292 | 91 | |
| 293 | − mutex_lock(&ctx->lock); | |
| 294 | − ret = ivpu_mmu_context_insert_node_locked(ctx, range, ivpu_bo_size(bo), &bo->mm_node); | |
| 92 | + ret = ivpu_mmu_context_insert_node(ctx, range, ivpu_bo_size(bo), &bo->mm_node); | |
| 295 | 93 | if (!ret) { |
| 296 | 94 | bo->ctx = ctx; |
| 297 | 95 | bo->vpu_addr = bo->mm_node.start; |
| 298 | − list_add_tail(&bo->ctx_node, &ctx->bo_list); | |
| 96 | + } else { | |
| 97 | + ivpu_err(vdev, "Failed to add BO to context %u: %d\n", ctx->id, ret); | |
| 299 | 98 | } |
| 300 | − mutex_unlock(&ctx->lock); | |
| 99 | + | |
| 100 | + ivpu_dbg_bo(vdev, bo, "alloc"); | |
| 101 | + | |
| 102 | + mutex_unlock(&bo->lock); | |
| 301 | 103 | |
| 302 | 104 | return ret; |
| 303 | 105 | } |
| 304 | 106 | |
| 305 | −static void ivpu_bo_free_vpu_addr(struct ivpu_bo *bo) | |
| 107 | +static void ivpu_bo_unbind_locked(struct ivpu_bo *bo) | |
| 306 | 108 | { |
| 307 | 109 | struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); |
| 308 | − struct ivpu_mmu_context *ctx = bo->ctx; | |
| 309 | 110 | |
| 310 | − ivpu_dbg(vdev, BO, "remove from ctx: ctx %d vpu_addr 0x%llx allocated %d mmu_mapped %d\n", | |
| 311 | − ctx->id, bo->vpu_addr, (bool)bo->sgt, bo->mmu_mapped); | |
| 111 | + lockdep_assert_held(&bo->lock); | |
| 312 | 112 | |
| 313 | − mutex_lock(&bo->lock); | |
| 113 | + ivpu_dbg_bo(vdev, bo, "unbind"); | |
| 114 | + | |
| 115 | + /* TODO: dma_unmap */ | |
| 314 | 116 | |
| 315 | 117 | if (bo->mmu_mapped) { |
| 316 | − drm_WARN_ON(&vdev->drm, !bo->sgt); | |
| 317 | − ivpu_mmu_context_unmap_sgt(vdev, ctx, bo->vpu_addr, bo->sgt); | |
| 118 | + drm_WARN_ON(&vdev->drm, !bo->ctx); | |
| 119 | + drm_WARN_ON(&vdev->drm, !bo->vpu_addr); | |
| 120 | + drm_WARN_ON(&vdev->drm, !bo->base.sgt); | |
| 121 | + ivpu_mmu_context_unmap_sgt(vdev, bo->ctx, bo->vpu_addr, bo->base.sgt); | |
| 318 | 122 | bo->mmu_mapped = false; |
| 319 | 123 | } |
| 320 | 124 | |
| 321 | − mutex_lock(&ctx->lock); | |
| 322 | − list_del(&bo->ctx_node); | |
| 323 | − bo->vpu_addr = 0; | |
| 324 | − bo->ctx = NULL; | |
| 325 | − ivpu_mmu_context_remove_node_locked(ctx, &bo->mm_node); | |
| 326 | − mutex_unlock(&ctx->lock); | |
| 125 | + if (bo->ctx) { | |
| 126 | + ivpu_mmu_context_remove_node(bo->ctx, &bo->mm_node); | |
| 127 | + bo->vpu_addr = 0; | |
| 128 | + bo->ctx = NULL; | |
| 129 | + } | |
| 130 | +} | |
| 327 | 131 | |
| 132 | +static void ivpu_bo_unbind(struct ivpu_bo *bo) | |
| 133 | +{ | |
| 134 | + mutex_lock(&bo->lock); | |
| 135 | + ivpu_bo_unbind_locked(bo); | |
| 328 | 136 | mutex_unlock(&bo->lock); |
| 329 | 137 | } |
| 330 | 138 | |
| 331 | −void ivpu_bo_remove_all_bos_from_context(struct ivpu_mmu_context *ctx) | |
| 139 | +void ivpu_bo_remove_all_bos_from_context(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx) | |
| 332 | 140 | { |
| 333 | − struct ivpu_bo *bo, *tmp; | |
| 141 | + struct ivpu_bo *bo; | |
| 142 | + | |
| 143 | + if (drm_WARN_ON(&vdev->drm, !ctx)) | |
| 144 | + return; | |
| 334 | 145 | |
| 335 | − list_for_each_entry_safe(bo, tmp, &ctx->bo_list, ctx_node) | |
| 336 | − ivpu_bo_free_vpu_addr(bo); | |
| 146 | + mutex_lock(&vdev->bo_list_lock); | |
| 147 | + list_for_each_entry(bo, &vdev->bo_list, bo_list_node) { | |
| 148 | + mutex_lock(&bo->lock); | |
| 149 | + if (bo->ctx == ctx) | |
| 150 | + ivpu_bo_unbind_locked(bo); | |
| 151 | + mutex_unlock(&bo->lock); | |
| 152 | + } | |
| 153 | + mutex_unlock(&vdev->bo_list_lock); | |
| 337 | 154 | } |
| 338 | 155 | |
| 339 | −static struct ivpu_bo * | |
| 340 | −ivpu_bo_alloc(struct ivpu_device *vdev, struct ivpu_mmu_context *mmu_context, | |
| 341 | − u64 size, u32 flags, const struct ivpu_bo_ops *ops, | |
| 342 | − const struct ivpu_addr_range *range, u64 user_ptr) | |
| 156 | +struct drm_gem_object *ivpu_gem_create_object(struct drm_device *dev, size_t size) | |
| 343 | 157 | { |
| 344 | 158 | struct ivpu_bo *bo; |
| 345 | − int ret = 0; | |
| 346 | − | |
| 347 | − if (drm_WARN_ON(&vdev->drm, size == 0 || !PAGE_ALIGNED(size))) | |
| 348 | − return ERR_PTR(-EINVAL); | |
| 349 | 159 | |
| 350 | − switch (flags & DRM_IVPU_BO_CACHE_MASK) { | |
| 351 | − case DRM_IVPU_BO_CACHED: | |
| 352 | − case DRM_IVPU_BO_UNCACHED: | |
| 353 | − case DRM_IVPU_BO_WC: | |
| 354 | − break; | |
| 355 | − default: | |
| 160 | + if (size == 0 || !PAGE_ALIGNED(size)) | |
| 356 | 161 | return ERR_PTR(-EINVAL); |
| 357 | − } | |
| 358 | 162 | |
| 359 | 163 | bo = kzalloc(sizeof(*bo), GFP_KERNEL); |
| 360 | 164 | if (!bo) |
| 361 | 165 | return ERR_PTR(-ENOMEM); |
| 362 | 166 | |
| 363 | − mutex_init(&bo->lock); | |
| 364 | − bo->base.funcs = &ivpu_gem_funcs; | |
| 365 | − bo->flags = flags; | |
| 366 | − bo->ops = ops; | |
| 367 | − bo->user_ptr = user_ptr; | |
| 368 | − | |
| 369 | − if (ops->type == IVPU_BO_TYPE_SHMEM) | |
| 370 | − ret = drm_gem_object_init(&vdev->drm, &bo->base, size); | |
| 371 | − else | |
| 372 | − drm_gem_private_object_init(&vdev->drm, &bo->base, size); | |
| 373 | − | |
| 374 | − if (ret) { | |
| 375 | − ivpu_err(vdev, "Failed to initialize drm object\n"); | |
| 376 | − goto err_free; | |
| 377 | − } | |
| 378 | − | |
| 379 | − if (flags & DRM_IVPU_BO_MAPPABLE) { | |
| 380 | − ret = drm_gem_create_mmap_offset(&bo->base); | |
| 381 | − if (ret) { | |
| 382 | − ivpu_err(vdev, "Failed to allocate mmap offset\n"); | |
| 383 | − goto err_release; | |
| 384 | − } | |
| 385 | − } | |
| 386 | − | |
| 387 | − if (mmu_context) { | |
| 388 | − ret = ivpu_bo_alloc_vpu_addr(bo, mmu_context, range); | |
| 389 | − if (ret) { | |
| 390 | − ivpu_err(vdev, "Failed to add BO to context: %d\n", ret); | |
| 391 | − goto err_release; | |
| 392 | − } | |
| 393 | − } | |
| 167 | + bo->base.base.funcs = &ivpu_gem_funcs; | |
| 168 | + bo->base.pages_mark_dirty_on_put = true; /* VPU can dirty a BO anytime */ | |
| 394 | 169 | |
| 395 | − return bo; | |
| 170 | + INIT_LIST_HEAD(&bo->bo_list_node); | |
| 171 | + mutex_init(&bo->lock); | |
| 396 | 172 | |
| 397 | −err_release: | |
| 398 | − drm_gem_object_release(&bo->base); | |
| 399 | −err_free: | |
| 400 | − kfree(bo); | |
| 401 | − return ERR_PTR(ret); | |
| 173 | + return &bo->base.base; | |
| 402 | 174 | } |
| 403 | 175 | |
| 404 | −static void ivpu_bo_free(struct drm_gem_object *obj) | |
| 176 | +static struct ivpu_bo * | |
| 177 | +ivpu_bo_create(struct ivpu_device *vdev, u64 size, u32 flags) | |
| 405 | 178 | { |
| 406 | − struct ivpu_bo *bo = to_ivpu_bo(obj); | |
| 407 | − struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); | |
| 408 | − | |
| 409 | − if (bo->ctx) | |
| 410 | − ivpu_dbg(vdev, BO, "free: ctx %d vpu_addr 0x%llx allocated %d mmu_mapped %d\n", | |
| 411 | − bo->ctx->id, bo->vpu_addr, (bool)bo->sgt, bo->mmu_mapped); | |
| 412 | − else | |
| 413 | − ivpu_dbg(vdev, BO, "free: ctx (released) allocated %d mmu_mapped %d\n", | |
| 414 | − (bool)bo->sgt, bo->mmu_mapped); | |
| 415 | − | |
| 416 | − drm_WARN_ON(&vdev->drm, !dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_READ)); | |
| 179 | + struct drm_gem_shmem_object *shmem; | |
| 180 | + struct ivpu_bo *bo; | |
| 417 | 181 | |
| 418 | − vunmap(bo->kvaddr); | |
| 182 | + switch (flags & DRM_IVPU_BO_CACHE_MASK) { | |
| 183 | + case DRM_IVPU_BO_CACHED: | |
| 184 | + case DRM_IVPU_BO_WC: | |
| 185 | + break; | |
| 186 | + default: | |
| 187 | + return ERR_PTR(-EINVAL); | |
| 188 | + } | |
| 419 | 189 | |
| 420 | − if (bo->ctx) | |
| 421 | − ivpu_bo_free_vpu_addr(bo); | |
| 190 | + shmem = drm_gem_shmem_create(&vdev->drm, size); | |
| 191 | + if (IS_ERR(shmem)) | |
| 192 | + return ERR_CAST(shmem); | |
| 422 | 193 | |
| 423 | − if (bo->sgt) | |
| 424 | − ivpu_bo_unmap_and_free_pages(bo); | |
| 194 | + bo = to_ivpu_bo(&shmem->base); | |
| 195 | + bo->base.map_wc = flags & DRM_IVPU_BO_WC; | |
| 196 | + bo->flags = flags; | |
| 425 | 197 | |
| 426 | − if (bo->base.import_attach) | |
| 427 | − drm_prime_gem_destroy(&bo->base, bo->sgt); | |
| 198 | + mutex_lock(&vdev->bo_list_lock); | |
| 199 | + list_add_tail(&bo->bo_list_node, &vdev->bo_list); | |
| 200 | + mutex_unlock(&vdev->bo_list_lock); | |
| 428 | 201 | |
| 429 | − drm_gem_object_release(&bo->base); | |
| 202 | + ivpu_dbg(vdev, BO, "create: vpu_addr 0x%llx size %zu flags 0x%x\n", | |
| 203 | + bo->vpu_addr, bo->base.base.size, flags); | |
| 430 | 204 | |
| 431 | − mutex_destroy(&bo->lock); | |
| 432 | − kfree(bo); | |
| 205 | + return bo; | |
| 433 | 206 | } |
| 434 | 207 | |
| 435 | −static int ivpu_bo_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) | |
| 208 | +static int ivpu_bo_open(struct drm_gem_object *obj, struct drm_file *file) | |
| 436 | 209 | { |
| 210 | + struct ivpu_file_priv *file_priv = file->driver_priv; | |
| 211 | + struct ivpu_device *vdev = file_priv->vdev; | |
| 437 | 212 | struct ivpu_bo *bo = to_ivpu_bo(obj); |
| 438 | − struct ivpu_device *vdev = ivpu_bo_to_vdev(bo); | |
| 439 | − | |
| 440 | − ivpu_dbg(vdev, BO, "mmap: ctx %u handle %u vpu_addr 0x%llx size %zu type %s", | |
| 441 | − bo->ctx->id, bo->handle, bo->vpu_addr, ivpu_bo_size(bo), bo->ops->name); | |
| 213 | + struct ivpu_addr_range *range; | |
| 442 | 214 | |
| 443 | − if (obj->import_attach) { | |
| 444 | − /* Drop the reference drm_gem_mmap_obj() acquired.*/ | |
| 445 | − drm_gem_object_put(obj); | |
| 446 | − vma->vm_private_data = NULL; | |
| 447 | − return dma_buf_mmap(obj->dma_buf, vma, 0); | |
| 448 | − } | |
| 449 | − | |
| 450 | − vm_flags_set(vma, VM_PFNMAP | VM_DONTEXPAND); | |
| 451 | − vma->vm_page_prot = ivpu_bo_pgprot(bo, vm_get_page_prot(vma->vm_flags)); | |
| 215 | + if (bo->flags & DRM_IVPU_BO_SHAVE_MEM) | |
| 216 | + range = &vdev->hw->ranges.shave; | |
| 217 | + else if (bo->flags & DRM_IVPU_BO_DMA_MEM) | |
| 218 | + range = &vdev->hw->ranges.dma; | |
| 219 | + else | |
| 220 | + range = &vdev->hw->ranges.user; | |
| 452 | 221 | |
| 453 | − return 0; | |
| 222 | + return ivpu_bo_alloc_vpu_addr(bo, &file_priv->ctx, range); | |
| 454 | 223 | } |
| 455 | 224 | |
| 456 | −static struct sg_table *ivpu_bo_get_sg_table(struct drm_gem_object *obj) | |
| 225 | +static void ivpu_bo_free(struct drm_gem_object *obj) | |
| 457 | 226 | { |
| 227 | + struct ivpu_device *vdev = to_ivpu_device(obj->dev); | |
| 458 | 228 | struct ivpu_bo *bo = to_ivpu_bo(obj); |
| 459 | − loff_t npages = obj->size >> PAGE_SHIFT; | |
| 460 | − int ret = 0; | |
| 461 | 229 | |
| 462 | − mutex_lock(&bo->lock); | |
| 230 | + mutex_lock(&vdev->bo_list_lock); | |
| 231 | + list_del(&bo->bo_list_node); | |
| 232 | + mutex_unlock(&vdev->bo_list_lock); | |
| 463 | 233 | |
| 464 | − if (!bo->sgt) | |
| 465 | − ret = ivpu_bo_alloc_and_map_pages_locked(bo); | |
| 234 | + drm_WARN_ON(&vdev->drm, !dma_resv_test_signaled(obj->resv, DMA_RESV_USAGE_READ)); | |
| 466 | 235 | |
| 467 | − mutex_unlock(&bo->lock); | |
| 236 | + ivpu_dbg_bo(vdev, bo, "free"); | |
| 468 | 237 | |
| 469 | − if (ret) | |
| 470 | − return ERR_PTR(ret); | |
| 238 | + ivpu_bo_unbind(bo); | |
| 239 | + mutex_destroy(&bo->lock); | |
| 471 | 240 | |
| 472 | − return drm_prime_pages_to_sg(obj->dev, bo->pages, npages); | |
| 241 | + drm_WARN_ON(obj->dev, bo->base.pages_use_count > 1); | |
| 242 | + drm_gem_shmem_free(&bo->base); | |
| 473 | 243 | } |
| 474 | 244 | |
| 475 | −static vm_fault_t ivpu_vm_fault(struct vm_fault *vmf) | |
| 476 | −{ | |
| 477 | − struct vm_area_struct *vma = vmf->vma; | |
| 478 | − struct drm_gem_object *obj = vma->vm_private_data; | |
| 479 | − struct ivpu_bo *bo = to_ivpu_bo(obj); | |
| 480 | − loff_t npages = obj->size >> PAGE_SHIFT; | |
| 481 | − pgoff_t page_offset; | |
| 482 | − struct page *page; | |
| 483 | − vm_fault_t ret; | |
| 484 | − int err; | |
| 485 | − | |
| 486 | − mutex_lock(&bo->lock); | |
| 487 | − | |
| 488 | − if (!bo->sgt) { | |
| 489 | − err = ivpu_bo_alloc_and_map_pages_locked(bo); | |
| 490 | − if (err) { | |
| 491 | − ret = vmf_error(err); | |
| 492 | − goto unlock; | |
| 493 | − } | |
| 494 | − } | |
| 495 | − | |
| 496 | − /* We don't use vmf->pgoff since that has the fake offset */ | |
| 497 | − page_offset = (vmf->address - vma->vm_start) >> PAGE_SHIFT; | |
| 498 | − if (page_offset >= npages) { | |
| 499 | − ret = VM_FAULT_SIGBUS; | |
| 500 | − } else { | |
| 501 | − page = bo->pages[page_offset]; | |
| 502 | − ret = vmf_insert_pfn(vma, vmf->address, page_to_pfn(page)); | |
| 503 | − } | |
| 504 | − | |
| 505 | −unlock: | |
| 506 | − mutex_unlock(&bo->lock); | |
| 245 | +static const struct dma_buf_ops ivpu_bo_dmabuf_ops = { | |
| 246 | + .cache_sgt_mapping = true, | |
| 247 | + .attach = drm_gem_map_attach, | |
| 248 | + .detach = drm_gem_map_detach, | |
| 249 | + .map_dma_buf = drm_gem_map_dma_buf, | |
| 250 | + .unmap_dma_buf = drm_gem_unmap_dma_buf, | |
| 251 | + .release = drm_gem_dmabuf_release, | |
| 252 | + .mmap = drm_gem_dmabuf_mmap, | |
| 253 | + .vmap = drm_gem_dmabuf_vmap, | |
| 254 | + .vunmap = drm_gem_dmabuf_vunmap, | |
| 255 | +}; | |
| 507 | 256 | |
| 508 | − return ret; | |
| 257 | +static struct dma_buf *ivpu_bo_export(struct drm_gem_object *obj, int flags) | |
| 258 | +{ | |
| 259 | + struct drm_device *dev = obj->dev; | |
| 260 | + struct dma_buf_export_info exp_info = { | |
| 261 | + .exp_name = KBUILD_MODNAME, | |
| 262 | + .owner = dev->driver->fops->owner, | |
| 263 | + .ops = &ivpu_bo_dmabuf_ops, | |
| 264 | + .size = obj->size, | |
| 265 | + .flags = flags, | |
| 266 | + .priv = obj, | |
| 267 | + .resv = obj->resv, | |
| 268 | + }; | |
| 269 | + void *sgt; | |
| 270 | + | |
| 271 | + /* | |
| 272 | + * Make sure that pages are allocated and dma-mapped before exporting the bo. | |
| 273 | + * DMA-mapping is required if the bo will be imported to the same device. | |
| 274 | + */ | |
| 275 | + sgt = drm_gem_shmem_get_pages_sgt(to_drm_gem_shmem_obj(obj)); | |
| 276 | + if (IS_ERR(sgt)) | |
| 277 | + return sgt; | |
| 278 | + | |
| 279 | + return drm_gem_dmabuf_export(dev, &exp_info); | |
| 509 | 280 | } |
| 510 | 281 | |
| 511 | −static const struct vm_operations_struct ivpu_vm_ops = { | |
| 512 | − .fault = ivpu_vm_fault, | |
| 513 | − .open = drm_gem_vm_open, | |
| 514 | − .close = drm_gem_vm_close, | |
| 515 | −}; | |
| 516 | − | |
| 517 | 282 | static const struct drm_gem_object_funcs ivpu_gem_funcs = { |
| 518 | 283 | .free = ivpu_bo_free, |
| 519 | − .mmap = ivpu_bo_mmap, | |
| 520 | − .vm_ops = &ivpu_vm_ops, | |
| 521 | − .get_sg_table = ivpu_bo_get_sg_table, | |
| 284 | + .open = ivpu_bo_open, | |
| 285 | + .export = ivpu_bo_export, | |
| 286 | + .print_info = drm_gem_shmem_object_print_info, | |
| 287 | + .pin = drm_gem_shmem_object_pin, | |
| 288 | + .unpin = drm_gem_shmem_object_unpin, | |
| 289 | + .get_sg_table = drm_gem_shmem_object_get_sg_table, | |
| 290 | + .vmap = drm_gem_shmem_object_vmap, | |
| 291 | + .vunmap = drm_gem_shmem_object_vunmap, | |
| 292 | + .mmap = drm_gem_shmem_object_mmap, | |
| 293 | + .vm_ops = &drm_gem_shmem_vm_ops, | |
| 522 | 294 | }; |
| 523 | 295 | |
| 524 | −int | |
| 525 | −ivpu_bo_create_ioctl(struct drm_device *dev, void *data, struct drm_file *file) | |
| 296 | +int ivpu_bo_create_ioctl(struct drm_device *dev, void *data, struct drm_file *file) | |
| 526 | 297 | { |
| 527 | 298 | struct ivpu_file_priv *file_priv = file->driver_priv; |
| 528 | 299 | struct ivpu_device *vdev = file_priv->vdev; |
| @@ -537,23 +308,20 @@ ivpu_bo_create_ioctl(struct drm_device *dev, void *data, struct drm_file *file) · 8 unmodified lines | ||
| 537 | 308 | if (size == 0) |
| 538 | 309 | return -EINVAL; |
| 539 | 310 | |
| 540 | − bo = ivpu_bo_alloc(vdev, &file_priv->ctx, size, args->flags, &shmem_ops, NULL, 0); | |
| 311 | + bo = ivpu_bo_create(vdev, size, args->flags); | |
| 541 | 312 | if (IS_ERR(bo)) { |
| 542 | 313 | ivpu_err(vdev, "Failed to create BO: %pe (ctx %u size %llu flags 0x%x)", |
| 543 | 314 | bo, file_priv->ctx.id, args->size, args->flags); |
| 544 | 315 | return PTR_ERR(bo); |
| 545 | 316 | } |
| 546 | 317 | |
| 547 | − ret = drm_gem_handle_create(file, &bo->base, &bo->handle); | |
| 318 | + ret = drm_gem_handle_create(file, &bo->base.base, &bo->handle); | |
| 548 | 319 | if (!ret) { |
| 549 | 320 | args->vpu_addr = bo->vpu_addr; |
| 550 | 321 | args->handle = bo->handle; |
| 551 | 322 | } |
| 552 | 323 | |
| 553 | − drm_gem_object_put(&bo->base); | |
| 554 | − | |
| 555 | − ivpu_dbg(vdev, BO, "alloc shmem: ctx %u vpu_addr 0x%llx size %zu flags 0x%x\n", | |
| 556 | − file_priv->ctx.id, bo->vpu_addr, ivpu_bo_size(bo), bo->flags); | |
| 324 | + drm_gem_object_put(&bo->base.base); | |
| 557 | 325 | |
| 558 | 326 | return ret; |
| 559 | 327 | } |
| @@ -563,8 +331,8 @@ ivpu_bo_alloc_internal(struct ivpu_device *vdev, u64 vpu_addr, u64 size, u32 fla · 3 unmodified lines | ||
| 563 | 331 | { |
| 564 | 332 | const struct ivpu_addr_range *range; |
| 565 | 333 | struct ivpu_addr_range fixed_range; |
| 334 | + struct iosys_map map; | |
| 566 | 335 | struct ivpu_bo *bo; |
| 567 | − pgprot_t prot; | |
| 568 | 336 | int ret; |
| 569 | 337 | |
| 570 | 338 | drm_WARN_ON(&vdev->drm, !PAGE_ALIGNED(vpu_addr)); |
| @@ -578,81 +346,42 @@ ivpu_bo_alloc_internal(struct ivpu_device *vdev, u64 vpu_addr, u64 size, u32 fla · 7 unmodified lines | ||
| 578 | 346 | range = &vdev->hw->ranges.global; |
| 579 | 347 | } |
| 580 | 348 | |
| 581 | − bo = ivpu_bo_alloc(vdev, &vdev->gctx, size, flags, &internal_ops, range, 0); | |
| 349 | + bo = ivpu_bo_create(vdev, size, flags); | |
| 582 | 350 | if (IS_ERR(bo)) { |
| 583 | 351 | ivpu_err(vdev, "Failed to create BO: %pe (vpu_addr 0x%llx size %llu flags 0x%x)", |
| 584 | 352 | bo, vpu_addr, size, flags); |
| 585 | 353 | return NULL; |
| 586 | 354 | } |
| 587 | 355 | |
| 588 | − ret = ivpu_bo_pin(bo); | |
| 356 | + ret = ivpu_bo_alloc_vpu_addr(bo, &vdev->gctx, range); | |
| 589 | 357 | if (ret) |
| 590 | 358 | goto err_put; |
| 591 | 359 | |
| 592 | − if (ivpu_bo_cache_mode(bo) != DRM_IVPU_BO_CACHED) | |
| 593 | − drm_clflush_pages(bo->pages, ivpu_bo_size(bo) >> PAGE_SHIFT); | |
| 594 | − | |
| 595 | − if (bo->flags & DRM_IVPU_BO_WC) | |
| 596 | − set_pages_array_wc(bo->pages, ivpu_bo_size(bo) >> PAGE_SHIFT); | |
| 597 | − else if (bo->flags & DRM_IVPU_BO_UNCACHED) | |
| 598 | − set_pages_array_uc(bo->pages, ivpu_bo_size(bo) >> PAGE_SHIFT); | |
| 599 | − | |
| 600 | − prot = ivpu_bo_pgprot(bo, PAGE_KERNEL); | |
| 601 | − bo->kvaddr = vmap(bo->pages, ivpu_bo_size(bo) >> PAGE_SHIFT, VM_MAP, prot); | |
| 602 | − if (!bo->kvaddr) { | |
| 603 | − ivpu_err(vdev, "Failed to map BO into kernel virtual memory\n"); | |
| 360 | + ret = ivpu_bo_pin(bo); | |
| 361 | + if (ret) | |
| 604 | 362 | goto err_put; |
| 605 | − } | |
| 606 | 363 | |
| 607 | − ivpu_dbg(vdev, BO, "alloc internal: ctx 0 vpu_addr 0x%llx size %zu flags 0x%x\n", | |
| 608 | − bo->vpu_addr, ivpu_bo_size(bo), flags); | |
| 364 | + ret = drm_gem_shmem_vmap(&bo->base, &map); | |
| 365 | + if (ret) | |
| 366 | + goto err_put; | |
| 609 | 367 | |
| 610 | 368 | return bo; |
| 611 | 369 | |
| 612 | 370 | err_put: |
| 613 | − drm_gem_object_put(&bo->base); | |
| 371 | + drm_gem_object_put(&bo->base.base); | |
| 614 | 372 | return NULL; |
| 615 | 373 | } |
| 616 | 374 | |
| 617 | 375 | void ivpu_bo_free_internal(struct ivpu_bo *bo) |
| 618 | 376 | { |
| 619 | − drm_gem_object_put(&bo->base); | |
| 620 | −} | |
| 621 | − | |
| 622 | −struct drm_gem_object *ivpu_gem_prime_import(struct drm_device *dev, struct dma_buf *buf) | |
| 623 | −{ | |
| 624 | − struct ivpu_device *vdev = to_ivpu_device(dev); | |
| 625 | − struct dma_buf_attachment *attach; | |
| 626 | − struct ivpu_bo *bo; | |
| 627 | − | |
| 628 | − attach = dma_buf_attach(buf, dev->dev); | |
| 629 | − if (IS_ERR(attach)) | |
| 630 | − return ERR_CAST(attach); | |
| 631 | − | |
| 632 | − get_dma_buf(buf); | |
| 377 | + struct iosys_map map = IOSYS_MAP_INIT_VADDR(bo->base.vaddr); | |
| 633 | 378 | |
| 634 | − bo = ivpu_bo_alloc(vdev, NULL, buf->size, DRM_IVPU_BO_MAPPABLE, &prime_ops, NULL, 0); | |
| 635 | − if (IS_ERR(bo)) { | |
| 636 | − ivpu_err(vdev, "Failed to import BO: %pe (size %lu)", bo, buf->size); | |
| 637 | − goto err_detach; | |
| 638 | − } | |
| 639 | − | |
| 640 | − lockdep_set_class(&bo->lock, &prime_bo_lock_class_key); | |
| 641 | − | |
| 642 | − bo->base.import_attach = attach; | |
| 643 | − | |
| 644 | − return &bo->base; | |
| 645 | − | |
| 646 | −err_detach: | |
| 647 | − dma_buf_detach(buf, attach); | |
| 648 | − dma_buf_put(buf); | |
| 649 | − return ERR_CAST(bo); | |
| 379 | + drm_gem_shmem_vunmap(&bo->base, &map); | |
| 380 | + drm_gem_object_put(&bo->base.base); | |
| 650 | 381 | } |
| 651 | 382 | |
| 652 | 383 | int ivpu_bo_info_ioctl(struct drm_device *dev, void *data, struct drm_file *file) |
| 653 | 384 | { |
| 654 | − struct ivpu_file_priv *file_priv = file->driver_priv; | |
| 655 | − struct ivpu_device *vdev = to_ivpu_device(dev); | |
| 656 | 385 | struct drm_ivpu_bo_info *args = data; |
| 657 | 386 | struct drm_gem_object *obj; |
| 658 | 387 | struct ivpu_bo *bo; |
| @@ -665,21 +394,12 @@ int ivpu_bo_info_ioctl(struct drm_device *dev, void *data, struct drm_file *file · 6 unmodified lines | ||
| 665 | 394 | bo = to_ivpu_bo(obj); |
| 666 | 395 | |
| 667 | 396 | mutex_lock(&bo->lock); |
| 668 | − | |
| 669 | − if (!bo->ctx) { | |
| 670 | − ret = ivpu_bo_alloc_vpu_addr(bo, &file_priv->ctx, NULL); | |
| 671 | − if (ret) { | |
| 672 | − ivpu_err(vdev, "Failed to allocate vpu_addr: %d\n", ret); | |
| 673 | − goto unlock; | |
| 674 | − } | |
| 675 | − } | |
| 676 | − | |
| 677 | 397 | args->flags = bo->flags; |
| 678 | 398 | args->mmap_offset = drm_vma_node_offset_addr(&obj->vma_node); |
| 679 | 399 | args->vpu_addr = bo->vpu_addr; |
| 680 | 400 | args->size = obj->size; |
| 681 | −unlock: | |
| 682 | 401 | mutex_unlock(&bo->lock); |
| 402 | + | |
| 683 | 403 | drm_gem_object_put(obj); |
| 684 | 404 | return ret; |
| 685 | 405 | } |
| @@ -714,41 +434,41 @@ static void ivpu_bo_print_info(struct ivpu_bo *bo, struct drm_printer *p) · 28 unmodified lines | ||
| 714 | 434 | { |
| 715 | 435 | unsigned long dma_refcount = 0; |
| 716 | 436 | |
| 717 | − if (bo->base.dma_buf && bo->base.dma_buf->file) | |
| 718 | − dma_refcount = atomic_long_read(&bo->base.dma_buf->file->f_count); | |
| 437 | + mutex_lock(&bo->lock); | |
| 438 | + | |
| 439 | + if (bo->base.base.dma_buf && bo->base.base.dma_buf->file) | |
| 440 | + dma_refcount = atomic_long_read(&bo->base.base.dma_buf->file->f_count); | |
| 441 | + | |
| 442 | + drm_printf(p, "%-3u %-6d 0x%-12llx %-10lu 0x%-8x %-4u %-8lu", | |
| 443 | + bo->ctx->id, bo->handle, bo->vpu_addr, bo->base.base.size, | |
| 444 | + bo->flags, kref_read(&bo->base.base.refcount), dma_refcount); | |
| 445 | + | |
| 446 | + if (bo->base.base.import_attach) | |
| 447 | + drm_printf(p, " imported"); | |
| 448 | + | |
| 449 | + if (bo->base.pages) | |
| 450 | + drm_printf(p, " has_pages"); | |
| 719 | 451 | |
| 720 | − drm_printf(p, "%5u %6d %16llx %10lu %10u %12lu %14s\n", | |
| 721 | − bo->ctx->id, bo->handle, bo->vpu_addr, ivpu_bo_size(bo), | |
| 722 | − kref_read(&bo->base.refcount), dma_refcount, bo->ops->name); | |
| 452 | + if (bo->mmu_mapped) | |
| 453 | + drm_printf(p, " mmu_mapped"); | |
| 454 | + | |
| 455 | + drm_printf(p, "\n"); | |
| 456 | + | |
| 457 | + mutex_unlock(&bo->lock); | |
| 723 | 458 | } |
| 724 | 459 | |
| 725 | 460 | void ivpu_bo_list(struct drm_device *dev, struct drm_printer *p) |
| 726 | 461 | { |
| 727 | 462 | struct ivpu_device *vdev = to_ivpu_device(dev); |
| 728 | − struct ivpu_file_priv *file_priv; | |
| 729 | − unsigned long ctx_id; | |
| 730 | 463 | struct ivpu_bo *bo; |
| 731 | 464 | |
| 732 | − drm_printf(p, "%5s %6s %16s %10s %10s %12s %14s\n", | |
| 733 | − "ctx", "handle", "vpu_addr", "size", "refcount", "dma_refcount", "type"); | |
| 465 | + drm_printf(p, "%-3s %-6s %-14s %-10s %-10s %-4s %-8s %s\n", | |
| 466 | + "ctx", "handle", "vpu_addr", "size", "flags", "refs", "dma_refs", "attribs"); | |
| 734 | 467 | |
| 735 | − mutex_lock(&vdev->gctx.lock); | |
| 736 | − list_for_each_entry(bo, &vdev->gctx.bo_list, ctx_node) | |
| 468 | + mutex_lock(&vdev->bo_list_lock); | |
| 469 | + list_for_each_entry(bo, &vdev->bo_list, bo_list_node) | |
| 737 | 470 | ivpu_bo_print_info(bo, p); |
| 738 | − mutex_unlock(&vdev->gctx.lock); | |
| 739 | − | |
| 740 | − xa_for_each(&vdev->context_xa, ctx_id, file_priv) { | |
| 741 | − file_priv = ivpu_file_priv_get_by_ctx_id(vdev, ctx_id); | |
| 742 | − if (!file_priv) | |
| 743 | − continue; | |
| 744 | − | |
| 745 | − mutex_lock(&file_priv->ctx.lock); | |
| 746 | − list_for_each_entry(bo, &file_priv->ctx.bo_list, ctx_node) | |
| 747 | − ivpu_bo_print_info(bo, p); | |
| 748 | − mutex_unlock(&file_priv->ctx.lock); | |
| 749 | − | |
| 750 | − ivpu_file_priv_put(&file_priv); | |
| 751 | − } | |
| 471 | + mutex_unlock(&vdev->bo_list_lock); | |
| 752 | 472 | } |
| 753 | 473 | |
| 754 | 474 | void ivpu_bo_list_print(struct drm_device *dev) |
| @@ -6,84 +6,52 @@ · 5 unmodified lines | ||
| 6 | 6 | #define __IVPU_GEM_H__ |
| 7 | 7 | |
| 8 | 8 | #include <drm/drm_gem.h> |
| 9 | +#include <drm/drm_gem_shmem_helper.h> | |
| 9 | 10 | #include <drm/drm_mm.h> |
| 10 | 11 | |
| 11 | −struct dma_buf; | |
| 12 | −struct ivpu_bo_ops; | |
| 13 | 12 | struct ivpu_file_priv; |
| 14 | 13 | |
| 15 | 14 | struct ivpu_bo { |
| 16 | − struct drm_gem_object base; | |
| 17 | − const struct ivpu_bo_ops *ops; | |
| 18 | − | |
| 15 | + struct drm_gem_shmem_object base; | |
| 19 | 16 | struct ivpu_mmu_context *ctx; |
| 20 | − struct list_head ctx_node; | |
| 17 | + struct list_head bo_list_node; | |
| 21 | 18 | struct drm_mm_node mm_node; |
| 22 | 19 | |
| 23 | − struct mutex lock; /* Protects: pages, sgt, mmu_mapped */ | |
| 24 | − struct sg_table *sgt; | |
| 25 | − struct page **pages; | |
| 26 | − bool mmu_mapped; | |
| 27 | − | |
| 28 | − void *kvaddr; | |
| 20 | + struct mutex lock; /* Protects: ctx, mmu_mapped, vpu_addr */ | |
| 29 | 21 | u64 vpu_addr; |
| 30 | 22 | u32 handle; |
| 31 | 23 | u32 flags; |
| 32 | − uintptr_t user_ptr; | |
| 33 | − u32 job_status; | |
| 34 | −}; | |
| 35 | − | |
| 36 | −enum ivpu_bo_type { | |
| 37 | − IVPU_BO_TYPE_SHMEM = 1, | |
| 38 | − IVPU_BO_TYPE_INTERNAL, | |
| 39 | − IVPU_BO_TYPE_PRIME, | |
| 40 | −}; | |
| 41 | − | |
| 42 | −struct ivpu_bo_ops { | |
| 43 | − enum ivpu_bo_type type; | |
| 44 | − const char *name; | |
| 45 | − int (*alloc_pages)(struct ivpu_bo *bo); | |
| 46 | − void (*free_pages)(struct ivpu_bo *bo); | |
| 47 | − int (*map_pages)(struct ivpu_bo *bo); | |
| 48 | − void (*unmap_pages)(struct ivpu_bo *bo); | |
| 24 | + u32 job_status; /* Valid only for command buffer */ | |
| 25 | + bool mmu_mapped; | |
| 49 | 26 | }; |
| 50 | 27 | |
| 51 | 28 | int ivpu_bo_pin(struct ivpu_bo *bo); |
| 52 | −void ivpu_bo_remove_all_bos_from_context(struct ivpu_mmu_context *ctx); | |
| 53 | −void ivpu_bo_list(struct drm_device *dev, struct drm_printer *p); | |
| 54 | −void ivpu_bo_list_print(struct drm_device *dev); | |
| 29 | +void ivpu_bo_remove_all_bos_from_context(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx); | |
| 55 | 30 | |
| 56 | −struct ivpu_bo * | |
| 57 | −ivpu_bo_alloc_internal(struct ivpu_device *vdev, u64 vpu_addr, u64 size, u32 flags); | |
| 31 | +struct drm_gem_object *ivpu_gem_create_object(struct drm_device *dev, size_t size); | |
| 32 | +struct ivpu_bo *ivpu_bo_alloc_internal(struct ivpu_device *vdev, u64 vpu_addr, u64 size, u32 flags); | |
| 58 | 33 | void ivpu_bo_free_internal(struct ivpu_bo *bo); |
| 59 | −struct drm_gem_object *ivpu_gem_prime_import(struct drm_device *dev, struct dma_buf *dma_buf); | |
| 60 | −void ivpu_bo_unmap_sgt_and_remove_from_context(struct ivpu_bo *bo); | |
| 61 | 34 | |
| 62 | 35 | int ivpu_bo_create_ioctl(struct drm_device *dev, void *data, struct drm_file *file); |
| 63 | 36 | int ivpu_bo_info_ioctl(struct drm_device *dev, void *data, struct drm_file *file); |
| 64 | 37 | int ivpu_bo_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file); |
| 65 | 38 | |
| 39 | +void ivpu_bo_list(struct drm_device *dev, struct drm_printer *p); | |
| 40 | +void ivpu_bo_list_print(struct drm_device *dev); | |
| 41 | + | |
| 66 | 42 | static inline struct ivpu_bo *to_ivpu_bo(struct drm_gem_object *obj) |
| 67 | 43 | { |
| 68 | − return container_of(obj, struct ivpu_bo, base); | |
| 44 | + return container_of(obj, struct ivpu_bo, base.base); | |
| 69 | 45 | } |
| 70 | 46 | |
| 71 | 47 | static inline void *ivpu_bo_vaddr(struct ivpu_bo *bo) |
| 72 | 48 | { |
| 73 | − return bo->kvaddr; | |
| 49 | + return bo->base.vaddr; | |
| 74 | 50 | } |
| 75 | 51 | |
| 76 | 52 | static inline size_t ivpu_bo_size(struct ivpu_bo *bo) |
| 77 | 53 | { |
| 78 | − return bo->base.size; | |
| 79 | −} | |
| 80 | − | |
| 81 | −static inline struct page *ivpu_bo_get_page(struct ivpu_bo *bo, u64 offset) | |
| 82 | −{ | |
| 83 | − if (offset > ivpu_bo_size(bo) || !bo->pages) | |
| 84 | − return NULL; | |
| 85 | − | |
| 86 | − return bo->pages[offset / PAGE_SIZE]; | |
| 54 | + return bo->base.base.size; | |
| 87 | 55 | } |
| 88 | 56 | |
| 89 | 57 | static inline u32 ivpu_bo_cache_mode(struct ivpu_bo *bo) |
| @@ -96,20 +64,9 @@ static inline bool ivpu_bo_is_snooped(struct ivpu_bo *bo) · 6 unmodified lines | ||
| 96 | 64 | return ivpu_bo_cache_mode(bo) == DRM_IVPU_BO_CACHED; |
| 97 | 65 | } |
| 98 | 66 | |
| 99 | −static inline pgprot_t ivpu_bo_pgprot(struct ivpu_bo *bo, pgprot_t prot) | |
| 100 | −{ | |
| 101 | − if (bo->flags & DRM_IVPU_BO_WC) | |
| 102 | − return pgprot_writecombine(prot); | |
| 103 | − | |
| 104 | − if (bo->flags & DRM_IVPU_BO_UNCACHED) | |
| 105 | − return pgprot_noncached(prot); | |
| 106 | − | |
| 107 | − return prot; | |
| 108 | −} | |
| 109 | − | |
| 110 | 67 | static inline struct ivpu_device *ivpu_bo_to_vdev(struct ivpu_bo *bo) |
| 111 | 68 | { |
| 112 | − return to_ivpu_device(bo->base.dev); | |
| 69 | + return to_ivpu_device(bo->base.base.dev); | |
| 113 | 70 | } |
| 114 | 71 | |
| 115 | 72 | static inline void *ivpu_to_cpu_addr(struct ivpu_bo *bo, u32 vpu_addr) |
| @@ -15,8 +15,11 @@ struct ivpu_hw_ops { · 14 unmodified lines | ||
| 15 | 15 | int (*power_down)(struct ivpu_device *vdev); |
| 16 | 16 | int (*reset)(struct ivpu_device *vdev); |
| 17 | 17 | bool (*is_idle)(struct ivpu_device *vdev); |
| 18 | + int (*wait_for_idle)(struct ivpu_device *vdev); | |
| 18 | 19 | void (*wdt_disable)(struct ivpu_device *vdev); |
| 19 | 20 | void (*diagnose_failure)(struct ivpu_device *vdev); |
| 21 | + u32 (*profiling_freq_get)(struct ivpu_device *vdev); | |
| 22 | + void (*profiling_freq_drive)(struct ivpu_device *vdev, bool enable); | |
| 20 | 23 | u32 (*reg_pll_freq_get)(struct ivpu_device *vdev); |
| 21 | 24 | u32 (*reg_telemetry_offset_get)(struct ivpu_device *vdev); |
| 22 | 25 | u32 (*reg_telemetry_size_get)(struct ivpu_device *vdev); |
| @@ -58,6 +61,8 @@ struct ivpu_hw_info { · 35 unmodified lines | ||
| 58 | 61 | u32 sku; |
| 59 | 62 | u16 config; |
| 60 | 63 | int dma_bits; |
| 64 | + ktime_t d0i3_entry_host_ts; | |
| 65 | + u64 d0i3_entry_vpu_ts; | |
| 61 | 66 | }; |
| 62 | 67 | |
| 63 | 68 | extern const struct ivpu_hw_ops ivpu_hw_37xx_ops; |
| @@ -85,6 +90,11 @@ static inline bool ivpu_hw_is_idle(struct ivpu_device *vdev) · 21 unmodified lines | ||
| 85 | 90 | return vdev->hw->ops->is_idle(vdev); |
| 86 | 91 | }; |
| 87 | 92 | |
| 93 | +static inline int ivpu_hw_wait_for_idle(struct ivpu_device *vdev) | |
| 94 | +{ | |
| 95 | + return vdev->hw->ops->wait_for_idle(vdev); | |
| 96 | +}; | |
| 97 | + | |
| 88 | 98 | static inline int ivpu_hw_power_down(struct ivpu_device *vdev) |
| 89 | 99 | { |
| 90 | 100 | ivpu_dbg(vdev, PM, "HW power down\n"); |
| @@ -104,6 +114,16 @@ static inline void ivpu_hw_wdt_disable(struct ivpu_device *vdev) · 13 unmodified lines | ||
| 104 | 114 | vdev->hw->ops->wdt_disable(vdev); |
| 105 | 115 | }; |
| 106 | 116 | |
| 117 | +static inline u32 ivpu_hw_profiling_freq_get(struct ivpu_device *vdev) | |
| 118 | +{ | |
| 119 | + return vdev->hw->ops->profiling_freq_get(vdev); | |
| 120 | +}; | |
| 121 | + | |
| 122 | +static inline void ivpu_hw_profiling_freq_drive(struct ivpu_device *vdev, bool enable) | |
| 123 | +{ | |
| 124 | + return vdev->hw->ops->profiling_freq_drive(vdev, enable); | |
| 125 | +}; | |
| 126 | + | |
| 107 | 127 | /* Register indirect accesses */ |
| 108 | 128 | static inline u32 ivpu_hw_reg_pll_freq_get(struct ivpu_device *vdev) |
| 109 | 129 | { |
| @@ -29,6 +29,7 @@ · 28 unmodified lines | ||
| 29 | 29 | |
| 30 | 30 | #define PLL_REF_CLK_FREQ (50 * 1000000) |
| 31 | 31 | #define PLL_SIMULATION_FREQ (10 * 1000000) |
| 32 | +#define PLL_PROF_CLK_FREQ (38400 * 1000) | |
| 32 | 33 | #define PLL_DEFAULT_EPP_VALUE 0x80 |
| 33 | 34 | |
| 34 | 35 | #define TIM_SAFE_ENABLE 0xf1d0dead |
| @@ -37,7 +38,7 @@ · 2 unmodified lines | ||
| 37 | 38 | #define TIMEOUT_US (150 * USEC_PER_MSEC) |
| 38 | 39 | #define PWR_ISLAND_STATUS_TIMEOUT_US (5 * USEC_PER_MSEC) |
| 39 | 40 | #define PLL_TIMEOUT_US (1500 * USEC_PER_MSEC) |
| 40 | −#define IDLE_TIMEOUT_US (500 * USEC_PER_MSEC) | |
| 41 | +#define IDLE_TIMEOUT_US (5 * USEC_PER_MSEC) | |
| 41 | 42 | |
| 42 | 43 | #define ICB_0_IRQ_MASK ((REG_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, HOST_IPC_FIFO_INT)) | \ |
| 43 | 44 | (REG_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, MMU_IRQ_0_INT)) | \ |
| @@ -90,6 +91,7 @@ static void ivpu_hw_timeouts_init(struct ivpu_device *vdev) · 46 unmodified lines | ||
| 90 | 91 | vdev->timeout.tdr = 2000; |
| 91 | 92 | vdev->timeout.reschedule_suspend = 10; |
| 92 | 93 | vdev->timeout.autosuspend = 10; |
| 94 | + vdev->timeout.d0i3_entry_msg = 5; | |
| 93 | 95 | } |
| 94 | 96 | |
| 95 | 97 | static int ivpu_pll_wait_for_cmd_send(struct ivpu_device *vdev) |
| @@ -502,6 +504,16 @@ static int ivpu_boot_pwr_domain_enable(struct ivpu_device *vdev) · 406 unmodified lines | ||
| 502 | 504 | return ret; |
| 503 | 505 | } |
| 504 | 506 | |
| 507 | +static int ivpu_boot_pwr_domain_disable(struct ivpu_device *vdev) | |
| 508 | +{ | |
| 509 | + ivpu_boot_dpu_active_drive(vdev, false); | |
| 510 | + ivpu_boot_pwr_island_isolation_drive(vdev, true); | |
| 511 | + ivpu_boot_pwr_island_trickle_drive(vdev, false); | |
| 512 | + ivpu_boot_pwr_island_drive(vdev, false); | |
| 513 | + | |
| 514 | + return ivpu_boot_wait_for_pwr_island_status(vdev, 0x0); | |
| 515 | +} | |
| 516 | + | |
| 505 | 517 | static void ivpu_boot_no_snoop_enable(struct ivpu_device *vdev) |
| 506 | 518 | { |
| 507 | 519 | u32 val = REGV_RD32(VPU_37XX_HOST_IF_TCU_PTW_OVERRIDES); |
| @@ -600,25 +612,17 @@ static int ivpu_hw_37xx_info_init(struct ivpu_device *vdev) · 92 unmodified lines | ||
| 600 | 612 | |
| 601 | 613 | static int ivpu_hw_37xx_reset(struct ivpu_device *vdev) |
| 602 | 614 | { |
| 603 | − int ret; | |
| 604 | − u32 val; | |
| 605 | − | |
| 606 | − if (IVPU_WA(punit_disabled)) | |
| 607 | − return 0; | |
| 615 | + int ret = 0; | |
| 608 | 616 | |
| 609 | − ret = REGB_POLL_FLD(VPU_37XX_BUTTRESS_VPU_IP_RESET, TRIGGER, 0, TIMEOUT_US); | |
| 610 | − if (ret) { | |
| 611 | − ivpu_err(vdev, "Timed out waiting for TRIGGER bit\n"); | |
| 612 | − return ret; | |
| 617 | + if (ivpu_boot_pwr_domain_disable(vdev)) { | |
| 618 | + ivpu_err(vdev, "Failed to disable power domain\n"); | |
| 619 | + ret = -EIO; | |
| 613 | 620 | } |
| 614 | 621 | |
| 615 | − val = REGB_RD32(VPU_37XX_BUTTRESS_VPU_IP_RESET); | |
| 616 | − val = REG_SET_FLD(VPU_37XX_BUTTRESS_VPU_IP_RESET, TRIGGER, val); | |
| 617 | − REGB_WR32(VPU_37XX_BUTTRESS_VPU_IP_RESET, val); | |
| 618 | − | |
| 619 | − ret = REGB_POLL_FLD(VPU_37XX_BUTTRESS_VPU_IP_RESET, TRIGGER, 0, TIMEOUT_US); | |
| 620 | − if (ret) | |
| 621 | − ivpu_err(vdev, "Timed out waiting for RESET completion\n"); | |
| 622 | + if (ivpu_pll_disable(vdev)) { | |
| 623 | + ivpu_err(vdev, "Failed to disable PLL\n"); | |
| 624 | + ret = -EIO; | |
| 625 | + } | |
| 622 | 626 | |
| 623 | 627 | return ret; |
| 624 | 628 | } |
| @@ -651,10 +655,6 @@ static int ivpu_hw_37xx_power_up(struct ivpu_device *vdev) · 26 unmodified lines | ||
| 651 | 655 | { |
| 652 | 656 | int ret; |
| 653 | 657 | |
| 654 | − ret = ivpu_hw_37xx_reset(vdev); | |
| 655 | − if (ret) | |
| 656 | − ivpu_warn(vdev, "Failed to reset HW: %d\n", ret); | |
| 657 | − | |
| 658 | 658 | ret = ivpu_hw_37xx_d0i3_disable(vdev); |
| 659 | 659 | if (ret) |
| 660 | 660 | ivpu_warn(vdev, "Failed to disable D0I3: %d\n", ret); |
| @@ -718,15 +718,28 @@ static bool ivpu_hw_37xx_is_idle(struct ivpu_device *vdev) · 57 unmodified lines | ||
| 718 | 718 | REG_TEST_FLD(VPU_37XX_BUTTRESS_VPU_STATUS, IDLE, val); |
| 719 | 719 | } |
| 720 | 720 | |
| 721 | +static int ivpu_hw_37xx_wait_for_idle(struct ivpu_device *vdev) | |
| 722 | +{ | |
| 723 | + return REGB_POLL_FLD(VPU_37XX_BUTTRESS_VPU_STATUS, IDLE, 0x1, IDLE_TIMEOUT_US); | |
| 724 | +} | |
| 725 | + | |
| 726 | +static void ivpu_hw_37xx_save_d0i3_entry_timestamp(struct ivpu_device *vdev) | |
| 727 | +{ | |
| 728 | + vdev->hw->d0i3_entry_host_ts = ktime_get_boottime(); | |
| 729 | + vdev->hw->d0i3_entry_vpu_ts = REGV_RD64(VPU_37XX_CPU_SS_TIM_PERF_FREE_CNT); | |
| 730 | +} | |
| 731 | + | |
| 721 | 732 | static int ivpu_hw_37xx_power_down(struct ivpu_device *vdev) |
| 722 | 733 | { |
| 723 | 734 | int ret = 0; |
| 724 | 735 | |
| 725 | − if (!ivpu_hw_37xx_is_idle(vdev) && ivpu_hw_37xx_reset(vdev)) | |
| 726 | − ivpu_err(vdev, "Failed to reset the VPU\n"); | |
| 736 | + ivpu_hw_37xx_save_d0i3_entry_timestamp(vdev); | |
| 727 | 737 | |
| 728 | − if (ivpu_pll_disable(vdev)) { | |
| 729 | − ivpu_err(vdev, "Failed to disable PLL\n"); | |
| 738 | + if (!ivpu_hw_37xx_is_idle(vdev)) | |
| 739 | + ivpu_warn(vdev, "VPU not idle during power down\n"); | |
| 740 | + | |
| 741 | + if (ivpu_hw_37xx_reset(vdev)) { | |
| 742 | + ivpu_err(vdev, "Failed to reset VPU\n"); | |
| 730 | 743 | ret = -EIO; |
| 731 | 744 | } |
| 732 | 745 | |
| @@ -756,6 +769,16 @@ static void ivpu_hw_37xx_wdt_disable(struct ivpu_device *vdev) · 23 unmodified lines | ||
| 756 | 769 | REGV_WR32(VPU_37XX_CPU_SS_TIM_GEN_CONFIG, val); |
| 757 | 770 | } |
| 758 | 771 | |
| 772 | +static u32 ivpu_hw_37xx_profiling_freq_get(struct ivpu_device *vdev) | |
| 773 | +{ | |
| 774 | + return PLL_PROF_CLK_FREQ; | |
| 775 | +} | |
| 776 | + | |
| 777 | +static void ivpu_hw_37xx_profiling_freq_drive(struct ivpu_device *vdev, bool enable) | |
| 778 | +{ | |
| 779 | + /* Profiling freq - is a debug feature. Unavailable on VPU 37XX. */ | |
| 780 | +} | |
| 781 | + | |
| 759 | 782 | static u32 ivpu_hw_37xx_pll_to_freq(u32 ratio, u32 config) |
| 760 | 783 | { |
| 761 | 784 | u32 pll_clock = PLL_REF_CLK_FREQ * ratio; |
| @@ -867,17 +890,20 @@ static void ivpu_hw_37xx_irq_noc_firewall_handler(struct ivpu_device *vdev) · 105 unmodified lines | ||
| 867 | 890 | } |
| 868 | 891 | |
| 869 | 892 | /* Handler for IRQs from VPU core (irqV) */ |
| 870 | −static u32 ivpu_hw_37xx_irqv_handler(struct ivpu_device *vdev, int irq) | |
| 893 | +static bool ivpu_hw_37xx_irqv_handler(struct ivpu_device *vdev, int irq, bool *wake_thread) | |
| 871 | 894 | { |
| 872 | 895 | u32 status = REGV_RD32(VPU_37XX_HOST_SS_ICB_STATUS_0) & ICB_0_IRQ_MASK; |
| 873 | 896 | |
| 897 | + if (!status) | |
| 898 | + return false; | |
| 899 | + | |
| 874 | 900 | REGV_WR32(VPU_37XX_HOST_SS_ICB_CLEAR_0, status); |
| 875 | 901 | |
| 876 | 902 | if (REG_TEST_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, MMU_IRQ_0_INT, status)) |
| 877 | 903 | ivpu_mmu_irq_evtq_handler(vdev); |
| 878 | 904 | |
| 879 | 905 | if (REG_TEST_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, HOST_IPC_FIFO_INT, status)) |
| 880 | − ivpu_ipc_irq_handler(vdev); | |
| 906 | + ivpu_ipc_irq_handler(vdev, wake_thread); | |
| 881 | 907 | |
| 882 | 908 | if (REG_TEST_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, MMU_IRQ_1_INT, status)) |
| 883 | 909 | ivpu_dbg(vdev, IRQ, "MMU sync complete\n"); |
| @@ -894,17 +920,17 @@ static u32 ivpu_hw_37xx_irqv_handler(struct ivpu_device *vdev, int irq) · 10 unmodified lines | ||
| 894 | 920 | if (REG_TEST_FLD(VPU_37XX_HOST_SS_ICB_STATUS_0, NOC_FIREWALL_INT, status)) |
| 895 | 921 | ivpu_hw_37xx_irq_noc_firewall_handler(vdev); |
| 896 | 922 | |
| 897 | − return status; | |
| 923 | + return true; | |
| 898 | 924 | } |
| 899 | 925 | |
| 900 | 926 | /* Handler for IRQs from Buttress core (irqB) */ |
| 901 | −static u32 ivpu_hw_37xx_irqb_handler(struct ivpu_device *vdev, int irq) | |
| 927 | +static bool ivpu_hw_37xx_irqb_handler(struct ivpu_device *vdev, int irq) | |
| 902 | 928 | { |
| 903 | 929 | u32 status = REGB_RD32(VPU_37XX_BUTTRESS_INTERRUPT_STAT) & BUTTRESS_IRQ_MASK; |
| 904 | 930 | bool schedule_recovery = false; |
| 905 | 931 | |
| 906 | − if (status == 0) | |
| 907 | − return 0; | |
| 932 | + if (!status) | |
| 933 | + return false; | |
| 908 | 934 | |
| 909 | 935 | if (REG_TEST_FLD(VPU_37XX_BUTTRESS_INTERRUPT_STAT, FREQ_CHANGE, status)) |
| 910 | 936 | ivpu_dbg(vdev, IRQ, "FREQ_CHANGE irq: %08x", |
| @@ -940,23 +966,27 @@ static u32 ivpu_hw_37xx_irqb_handler(struct ivpu_device *vdev, int irq) · 29 unmodified lines | ||
| 940 | 966 | if (schedule_recovery) |
| 941 | 967 | ivpu_pm_schedule_recovery(vdev); |
| 942 | 968 | |
| 943 | − return status; | |
| 969 | + return true; | |
| 944 | 970 | } |
| 945 | 971 | |
| 946 | 972 | static irqreturn_t ivpu_hw_37xx_irq_handler(int irq, void *ptr) |
| 947 | 973 | { |
| 948 | 974 | struct ivpu_device *vdev = ptr; |
| 949 | − u32 ret_irqv, ret_irqb; | |
| 975 | + bool irqv_handled, irqb_handled, wake_thread = false; | |
| 950 | 976 | |
| 951 | 977 | REGB_WR32(VPU_37XX_BUTTRESS_GLOBAL_INT_MASK, 0x1); |
| 952 | 978 | |
| 953 | − ret_irqv = ivpu_hw_37xx_irqv_handler(vdev, irq); | |
| 954 | − ret_irqb = ivpu_hw_37xx_irqb_handler(vdev, irq); | |
| 979 | + irqv_handled = ivpu_hw_37xx_irqv_handler(vdev, irq, &wake_thread); | |
| 980 | + irqb_handled = ivpu_hw_37xx_irqb_handler(vdev, irq); | |
| 955 | 981 | |
| 956 | 982 | /* Re-enable global interrupts to re-trigger MSI for pending interrupts */ |
| 957 | 983 | REGB_WR32(VPU_37XX_BUTTRESS_GLOBAL_INT_MASK, 0x0); |
| 958 | 984 | |
| 959 | − return IRQ_RETVAL(ret_irqb | ret_irqv); | |
| 985 | + if (wake_thread) | |
| 986 | + return IRQ_WAKE_THREAD; | |
| 987 | + if (irqv_handled || irqb_handled) | |
| 988 | + return IRQ_HANDLED; | |
| 989 | + return IRQ_NONE; | |
| 960 | 990 | } |
| 961 | 991 | |
| 962 | 992 | static void ivpu_hw_37xx_diagnose_failure(struct ivpu_device *vdev) |
| @@ -993,11 +1023,14 @@ const struct ivpu_hw_ops ivpu_hw_37xx_ops = { · 30 unmodified lines | ||
| 993 | 1023 | .info_init = ivpu_hw_37xx_info_init, |
| 994 | 1024 | .power_up = ivpu_hw_37xx_power_up, |
| 995 | 1025 | .is_idle = ivpu_hw_37xx_is_idle, |
| 1026 | + .wait_for_idle = ivpu_hw_37xx_wait_for_idle, | |
| 996 | 1027 | .power_down = ivpu_hw_37xx_power_down, |
| 997 | 1028 | .reset = ivpu_hw_37xx_reset, |
| 998 | 1029 | .boot_fw = ivpu_hw_37xx_boot_fw, |
| 999 | 1030 | .wdt_disable = ivpu_hw_37xx_wdt_disable, |
| 1000 | 1031 | .diagnose_failure = ivpu_hw_37xx_diagnose_failure, |
| 1032 | + .profiling_freq_get = ivpu_hw_37xx_profiling_freq_get, | |
| 1033 | + .profiling_freq_drive = ivpu_hw_37xx_profiling_freq_drive, | |
| 1001 | 1034 | .reg_pll_freq_get = ivpu_hw_37xx_reg_pll_freq_get, |
| 1002 | 1035 | .reg_telemetry_offset_get = ivpu_hw_37xx_reg_telemetry_offset_get, |
| 1003 | 1036 | .reg_telemetry_size_get = ivpu_hw_37xx_reg_telemetry_size_get, |
| @@ -240,6 +240,8 @@ · 239 unmodified lines | ||
| 240 | 240 | #define VPU_37XX_CPU_SS_TIM_GEN_CONFIG 0x06021008u |
| 241 | 241 | #define VPU_37XX_CPU_SS_TIM_GEN_CONFIG_WDOG_TO_INT_CLR_MASK BIT_MASK(9) |
| 242 | 242 | |
| 243 | +#define VPU_37XX_CPU_SS_TIM_PERF_FREE_CNT 0x06029000u | |
| 244 | + | |
| 243 | 245 | #define VPU_37XX_CPU_SS_DOORBELL_0 0x06300000u |
| 244 | 246 | #define VPU_37XX_CPU_SS_DOORBELL_0_SET_MASK BIT_MASK(0) |
| 245 | 247 | |
| @@ -39,6 +39,7 @@ · 38 unmodified lines | ||
| 39 | 39 | #define TIMEOUT_US (150 * USEC_PER_MSEC) |
| 40 | 40 | #define PWR_ISLAND_STATUS_TIMEOUT_US (5 * USEC_PER_MSEC) |
| 41 | 41 | #define PLL_TIMEOUT_US (1500 * USEC_PER_MSEC) |
| 42 | +#define IDLE_TIMEOUT_US (5 * USEC_PER_MSEC) | |
| 42 | 43 | |
| 43 | 44 | #define WEIGHTS_DEFAULT 0xf711f711u |
| 44 | 45 | #define WEIGHTS_ATS_DEFAULT 0x0000f711u |
| @@ -139,18 +140,21 @@ static void ivpu_hw_timeouts_init(struct ivpu_device *vdev) · 94 unmodified lines | ||
| 139 | 140 | vdev->timeout.tdr = 2000000; |
| 140 | 141 | vdev->timeout.reschedule_suspend = 1000; |
| 141 | 142 | vdev->timeout.autosuspend = -1; |
| 143 | + vdev->timeout.d0i3_entry_msg = 500; | |
| 142 | 144 | } else if (ivpu_is_simics(vdev)) { |
| 143 | 145 | vdev->timeout.boot = 50; |
| 144 | 146 | vdev->timeout.jsm = 500; |
| 145 | 147 | vdev->timeout.tdr = 10000; |
| 146 | 148 | vdev->timeout.reschedule_suspend = 10; |
| 147 | 149 | vdev->timeout.autosuspend = -1; |
| 150 | + vdev->timeout.d0i3_entry_msg = 100; | |
| 148 | 151 | } else { |
| 149 | 152 | vdev->timeout.boot = 1000; |
| 150 | 153 | vdev->timeout.jsm = 500; |
| 151 | 154 | vdev->timeout.tdr = 2000; |
| 152 | 155 | vdev->timeout.reschedule_suspend = 10; |
| 153 | 156 | vdev->timeout.autosuspend = 10; |
| 157 | + vdev->timeout.d0i3_entry_msg = 5; | |
| 154 | 158 | } |
| 155 | 159 | } |
| 156 | 160 | |
| @@ -824,12 +828,6 @@ static int ivpu_hw_40xx_power_up(struct ivpu_device *vdev) · 667 unmodified lines | ||
| 824 | 828 | { |
| 825 | 829 | int ret; |
| 826 | 830 | |
| 827 | − ret = ivpu_hw_40xx_reset(vdev); | |
| 828 | − if (ret) { | |
| 829 | − ivpu_err(vdev, "Failed to reset HW: %d\n", ret); | |
| 830 | − return ret; | |
| 831 | − } | |
| 832 | − | |
| 833 | 831 | ret = ivpu_hw_40xx_d0i3_disable(vdev); |
| 834 | 832 | if (ret) |
| 835 | 833 | ivpu_warn(vdev, "Failed to disable D0I3: %d\n", ret); |
| @@ -898,10 +896,23 @@ static bool ivpu_hw_40xx_is_idle(struct ivpu_device *vdev) · 62 unmodified lines | ||
| 898 | 896 | REG_TEST_FLD(VPU_40XX_BUTTRESS_VPU_STATUS, IDLE, val); |
| 899 | 897 | } |
| 900 | 898 | |
| 899 | +static int ivpu_hw_40xx_wait_for_idle(struct ivpu_device *vdev) | |
| 900 | +{ | |
| 901 | + return REGB_POLL_FLD(VPU_40XX_BUTTRESS_VPU_STATUS, IDLE, 0x1, IDLE_TIMEOUT_US); | |
| 902 | +} | |
| 903 | + | |
| 904 | +static void ivpu_hw_40xx_save_d0i3_entry_timestamp(struct ivpu_device *vdev) | |
| 905 | +{ | |
| 906 | + vdev->hw->d0i3_entry_host_ts = ktime_get_boottime(); | |
| 907 | + vdev->hw->d0i3_entry_vpu_ts = REGV_RD64(VPU_40XX_CPU_SS_TIM_PERF_EXT_FREE_CNT); | |
| 908 | +} | |
| 909 | + | |
| 901 | 910 | static int ivpu_hw_40xx_power_down(struct ivpu_device *vdev) |
| 902 | 911 | { |
| 903 | 912 | int ret = 0; |
| 904 | 913 | |
| 914 | + ivpu_hw_40xx_save_d0i3_entry_timestamp(vdev); | |
| 915 | + | |
| 905 | 916 | if (!ivpu_hw_40xx_is_idle(vdev) && ivpu_hw_40xx_reset(vdev)) |
| 906 | 917 | ivpu_warn(vdev, "Failed to reset the VPU\n"); |
| 907 | 918 | |
| @@ -933,6 +944,19 @@ static void ivpu_hw_40xx_wdt_disable(struct ivpu_device *vdev) · 25 unmodified lines | ||
| 933 | 944 | REGV_WR32(VPU_40XX_CPU_SS_TIM_GEN_CONFIG, val); |
| 934 | 945 | } |
| 935 | 946 | |
| 947 | +static u32 ivpu_hw_40xx_profiling_freq_get(struct ivpu_device *vdev) | |
| 948 | +{ | |
| 949 | + return vdev->hw->pll.profiling_freq; | |
| 950 | +} | |
| 951 | + | |
| 952 | +static void ivpu_hw_40xx_profiling_freq_drive(struct ivpu_device *vdev, bool enable) | |
| 953 | +{ | |
| 954 | + if (enable) | |
| 955 | + vdev->hw->pll.profiling_freq = PLL_PROFILING_FREQ_HIGH; | |
| 956 | + else | |
| 957 | + vdev->hw->pll.profiling_freq = PLL_PROFILING_FREQ_DEFAULT; | |
| 958 | +} | |
| 959 | + | |
| 936 | 960 | /* Register indirect accesses */ |
| 937 | 961 | static u32 ivpu_hw_40xx_reg_pll_freq_get(struct ivpu_device *vdev) |
| 938 | 962 | { |
| @@ -1023,13 +1047,12 @@ static void ivpu_hw_40xx_irq_noc_firewall_handler(struct ivpu_device *vdev) · 84 unmodified lines | ||
| 1023 | 1047 | } |
| 1024 | 1048 | |
| 1025 | 1049 | /* Handler for IRQs from VPU core (irqV) */ |
| 1026 | −static irqreturn_t ivpu_hw_40xx_irqv_handler(struct ivpu_device *vdev, int irq) | |
| 1050 | +static bool ivpu_hw_40xx_irqv_handler(struct ivpu_device *vdev, int irq, bool *wake_thread) | |
| 1027 | 1051 | { |
| 1028 | 1052 | u32 status = REGV_RD32(VPU_40XX_HOST_SS_ICB_STATUS_0) & ICB_0_IRQ_MASK; |
| 1029 | − irqreturn_t ret = IRQ_NONE; | |
| 1030 | 1053 | |
| 1031 | 1054 | if (!status) |
| 1032 | − return IRQ_NONE; | |
| 1055 | + return false; | |
| 1033 | 1056 | |
| 1034 | 1057 | REGV_WR32(VPU_40XX_HOST_SS_ICB_CLEAR_0, status); |
| 1035 | 1058 | |
| @@ -1037,7 +1060,7 @@ static irqreturn_t ivpu_hw_40xx_irqv_handler(struct ivpu_device *vdev, int irq) · 1 unmodified line | ||
| 1037 | 1060 | ivpu_mmu_irq_evtq_handler(vdev); |
| 1038 | 1061 | |
| 1039 | 1062 | if (REG_TEST_FLD(VPU_40XX_HOST_SS_ICB_STATUS_0, HOST_IPC_FIFO_INT, status)) |
| 1040 | − ret |= ivpu_ipc_irq_handler(vdev); | |
| 1063 | + ivpu_ipc_irq_handler(vdev, wake_thread); | |
| 1041 | 1064 | |
| 1042 | 1065 | if (REG_TEST_FLD(VPU_40XX_HOST_SS_ICB_STATUS_0, MMU_IRQ_1_INT, status)) |
| 1043 | 1066 | ivpu_dbg(vdev, IRQ, "MMU sync complete\n"); |
| @@ -1054,17 +1077,17 @@ static irqreturn_t ivpu_hw_40xx_irqv_handler(struct ivpu_device *vdev, int irq) · 10 unmodified lines | ||
| 1054 | 1077 | if (REG_TEST_FLD(VPU_40XX_HOST_SS_ICB_STATUS_0, NOC_FIREWALL_INT, status)) |
| 1055 | 1078 | ivpu_hw_40xx_irq_noc_firewall_handler(vdev); |
| 1056 | 1079 | |
| 1057 | − return ret; | |
| 1080 | + return true; | |
| 1058 | 1081 | } |
| 1059 | 1082 | |
| 1060 | 1083 | /* Handler for IRQs from Buttress core (irqB) */ |
| 1061 | −static irqreturn_t ivpu_hw_40xx_irqb_handler(struct ivpu_device *vdev, int irq) | |
| 1084 | +static bool ivpu_hw_40xx_irqb_handler(struct ivpu_device *vdev, int irq) | |
| 1062 | 1085 | { |
| 1063 | 1086 | bool schedule_recovery = false; |
| 1064 | 1087 | u32 status = REGB_RD32(VPU_40XX_BUTTRESS_INTERRUPT_STAT) & BUTTRESS_IRQ_MASK; |
| 1065 | 1088 | |
| 1066 | − if (status == 0) | |
| 1067 | − return IRQ_NONE; | |
| 1089 | + if (!status) | |
| 1090 | + return false; | |
| 1068 | 1091 | |
| 1069 | 1092 | if (REG_TEST_FLD(VPU_40XX_BUTTRESS_INTERRUPT_STAT, FREQ_CHANGE, status)) |
| 1070 | 1093 | ivpu_dbg(vdev, IRQ, "FREQ_CHANGE"); |
| @@ -1116,26 +1139,27 @@ static irqreturn_t ivpu_hw_40xx_irqb_handler(struct ivpu_device *vdev, int irq) · 45 unmodified lines | ||
| 1116 | 1139 | if (schedule_recovery) |
| 1117 | 1140 | ivpu_pm_schedule_recovery(vdev); |
| 1118 | 1141 | |
| 1119 | − return IRQ_HANDLED; | |
| 1142 | + return true; | |
| 1120 | 1143 | } |
| 1121 | 1144 | |
| 1122 | 1145 | static irqreturn_t ivpu_hw_40xx_irq_handler(int irq, void *ptr) |
| 1123 | 1146 | { |
| 1147 | + bool irqv_handled, irqb_handled, wake_thread = false; | |
| 1124 | 1148 | struct ivpu_device *vdev = ptr; |
| 1125 | − irqreturn_t ret = IRQ_NONE; | |
| 1126 | 1149 | |
| 1127 | 1150 | REGB_WR32(VPU_40XX_BUTTRESS_GLOBAL_INT_MASK, 0x1); |
| 1128 | 1151 | |
| 1129 | − ret |= ivpu_hw_40xx_irqv_handler(vdev, irq); | |
| 1130 | − ret |= ivpu_hw_40xx_irqb_handler(vdev, irq); | |
| 1152 | + irqv_handled = ivpu_hw_40xx_irqv_handler(vdev, irq, &wake_thread); | |
| 1153 | + irqb_handled = ivpu_hw_40xx_irqb_handler(vdev, irq); | |
| 1131 | 1154 | |
| 1132 | 1155 | /* Re-enable global interrupts to re-trigger MSI for pending interrupts */ |
| 1133 | 1156 | REGB_WR32(VPU_40XX_BUTTRESS_GLOBAL_INT_MASK, 0x0); |
| 1134 | 1157 | |
| 1135 | − if (ret & IRQ_WAKE_THREAD) | |
| 1158 | + if (wake_thread) | |
| 1136 | 1159 | return IRQ_WAKE_THREAD; |
| 1137 | − | |
| 1138 | − return ret; | |
| 1160 | + if (irqv_handled || irqb_handled) | |
| 1161 | + return IRQ_HANDLED; | |
| 1162 | + return IRQ_NONE; | |
| 1139 | 1163 | } |
| 1140 | 1164 | |
| 1141 | 1165 | static void ivpu_hw_40xx_diagnose_failure(struct ivpu_device *vdev) |
| @@ -1185,11 +1209,14 @@ const struct ivpu_hw_ops ivpu_hw_40xx_ops = { · 43 unmodified lines | ||
| 1185 | 1209 | .info_init = ivpu_hw_40xx_info_init, |
| 1186 | 1210 | .power_up = ivpu_hw_40xx_power_up, |
| 1187 | 1211 | .is_idle = ivpu_hw_40xx_is_idle, |
| 1212 | + .wait_for_idle = ivpu_hw_40xx_wait_for_idle, | |
| 1188 | 1213 | .power_down = ivpu_hw_40xx_power_down, |
| 1189 | 1214 | .reset = ivpu_hw_40xx_reset, |
| 1190 | 1215 | .boot_fw = ivpu_hw_40xx_boot_fw, |
| 1191 | 1216 | .wdt_disable = ivpu_hw_40xx_wdt_disable, |
| 1192 | 1217 | .diagnose_failure = ivpu_hw_40xx_diagnose_failure, |
| 1218 | + .profiling_freq_get = ivpu_hw_40xx_profiling_freq_get, | |
| 1219 | + .profiling_freq_drive = ivpu_hw_40xx_profiling_freq_drive, | |
| 1193 | 1220 | .reg_pll_freq_get = ivpu_hw_40xx_reg_pll_freq_get, |
| 1194 | 1221 | .reg_telemetry_offset_get = ivpu_hw_40xx_reg_telemetry_offset_get, |
| 1195 | 1222 | .reg_telemetry_size_get = ivpu_hw_40xx_reg_telemetry_size_get, |
| @@ -5,7 +5,7 @@ · 4 unmodified lines | ||
| 5 | 5 | |
| 6 | 6 | #include <linux/genalloc.h> |
| 7 | 7 | #include <linux/highmem.h> |
| 8 | −#include <linux/kthread.h> | |
| 8 | +#include <linux/pm_runtime.h> | |
| 9 | 9 | #include <linux/wait.h> |
| 10 | 10 | |
| 11 | 11 | #include "ivpu_drv.h" |
| @@ -17,19 +17,12 @@ · 5 unmodified lines | ||
| 17 | 17 | #include "ivpu_pm.h" |
| 18 | 18 | |
| 19 | 19 | #define IPC_MAX_RX_MSG 128 |
| 20 | −#define IS_KTHREAD() (get_current()->flags & PF_KTHREAD) | |
| 21 | 20 | |
| 22 | 21 | struct ivpu_ipc_tx_buf { |
| 23 | 22 | struct ivpu_ipc_hdr ipc; |
| 24 | 23 | struct vpu_jsm_msg jsm; |
| 25 | 24 | }; |
| 26 | 25 | |
| 27 | −struct ivpu_ipc_rx_msg { | |
| 28 | − struct list_head link; | |
| 29 | − struct ivpu_ipc_hdr *ipc_hdr; | |
| 30 | − struct vpu_jsm_msg *jsm_msg; | |
| 31 | −}; | |
| 32 | − | |
| 33 | 26 | static void ivpu_ipc_msg_dump(struct ivpu_device *vdev, char *c, |
| 34 | 27 | struct ivpu_ipc_hdr *ipc_hdr, u32 vpu_addr) |
| 35 | 28 | { |
| @@ -139,8 +132,49 @@ static void ivpu_ipc_tx(struct ivpu_device *vdev, u32 vpu_addr) · 103 unmodified lines | ||
| 139 | 132 | ivpu_hw_reg_ipc_tx_set(vdev, vpu_addr); |
| 140 | 133 | } |
| 141 | 134 | |
| 142 | −void | |
| 143 | −ivpu_ipc_consumer_add(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, u32 channel) | |
| 135 | +static void | |
| 136 | +ivpu_ipc_rx_msg_add(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, | |
| 137 | + struct ivpu_ipc_hdr *ipc_hdr, struct vpu_jsm_msg *jsm_msg) | |
| 138 | +{ | |
| 139 | + struct ivpu_ipc_info *ipc = vdev->ipc; | |
| 140 | + struct ivpu_ipc_rx_msg *rx_msg; | |
| 141 | + | |
| 142 | + lockdep_assert_held(&ipc->cons_lock); | |
| 143 | + lockdep_assert_irqs_disabled(); | |
| 144 | + | |
| 145 | + rx_msg = kzalloc(sizeof(*rx_msg), GFP_ATOMIC); | |
| 146 | + if (!rx_msg) { | |
| 147 | + ivpu_ipc_rx_mark_free(vdev, ipc_hdr, jsm_msg); | |
| 148 | + return; | |
| 149 | + } | |
| 150 | + | |
| 151 | + atomic_inc(&ipc->rx_msg_count); | |
| 152 | + | |
| 153 | + rx_msg->ipc_hdr = ipc_hdr; | |
| 154 | + rx_msg->jsm_msg = jsm_msg; | |
| 155 | + rx_msg->callback = cons->rx_callback; | |
| 156 | + | |
| 157 | + if (rx_msg->callback) { | |
| 158 | + list_add_tail(&rx_msg->link, &ipc->cb_msg_list); | |
| 159 | + } else { | |
| 160 | + spin_lock(&cons->rx_lock); | |
| 161 | + list_add_tail(&rx_msg->link, &cons->rx_msg_list); | |
| 162 | + spin_unlock(&cons->rx_lock); | |
| 163 | + wake_up(&cons->rx_msg_wq); | |
| 164 | + } | |
| 165 | +} | |
| 166 | + | |
| 167 | +static void | |
| 168 | +ivpu_ipc_rx_msg_del(struct ivpu_device *vdev, struct ivpu_ipc_rx_msg *rx_msg) | |
| 169 | +{ | |
| 170 | + list_del(&rx_msg->link); | |
| 171 | + ivpu_ipc_rx_mark_free(vdev, rx_msg->ipc_hdr, rx_msg->jsm_msg); | |
| 172 | + atomic_dec(&vdev->ipc->rx_msg_count); | |
| 173 | + kfree(rx_msg); | |
| 174 | +} | |
| 175 | + | |
| 176 | +void ivpu_ipc_consumer_add(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, | |
| 177 | + u32 channel, ivpu_ipc_rx_callback_t rx_callback) | |
| 144 | 178 | { |
| 145 | 179 | struct ivpu_ipc_info *ipc = vdev->ipc; |
| 146 | 180 | |
| @@ -148,13 +182,15 @@ ivpu_ipc_consumer_add(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, · 1 unmodified line | ||
| 148 | 182 | cons->channel = channel; |
| 149 | 183 | cons->tx_vpu_addr = 0; |
| 150 | 184 | cons->request_id = 0; |
| 151 | − spin_lock_init(&cons->rx_msg_lock); | |
| 185 | + cons->aborted = false; | |
| 186 | + cons->rx_callback = rx_callback; | |
| 187 | + spin_lock_init(&cons->rx_lock); | |
| 152 | 188 | INIT_LIST_HEAD(&cons->rx_msg_list); |
| 153 | 189 | init_waitqueue_head(&cons->rx_msg_wq); |
| 154 | 190 | |
| 155 | − spin_lock_irq(&ipc->cons_list_lock); | |
| 191 | + spin_lock_irq(&ipc->cons_lock); | |
| 156 | 192 | list_add_tail(&cons->link, &ipc->cons_list); |
| 157 | − spin_unlock_irq(&ipc->cons_list_lock); | |
| 193 | + spin_unlock_irq(&ipc->cons_lock); | |
| 158 | 194 | } |
| 159 | 195 | |
| 160 | 196 | void ivpu_ipc_consumer_del(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons) |
| @@ -162,18 +198,14 @@ void ivpu_ipc_consumer_del(struct ivpu_device *vdev, struct ivpu_ipc_consumer *c · 1 unmodified line | ||
| 162 | 198 | struct ivpu_ipc_info *ipc = vdev->ipc; |
| 163 | 199 | struct ivpu_ipc_rx_msg *rx_msg, *r; |
| 164 | 200 | |
| 165 | − spin_lock_irq(&ipc->cons_list_lock); | |
| 201 | + spin_lock_irq(&ipc->cons_lock); | |
| 166 | 202 | list_del(&cons->link); |
| 167 | − spin_unlock_irq(&ipc->cons_list_lock); | |
| 168 | − | |
| 169 | − spin_lock_irq(&cons->rx_msg_lock); | |
| 170 | − list_for_each_entry_safe(rx_msg, r, &cons->rx_msg_list, link) { | |
| 171 | − list_del(&rx_msg->link); | |
| 172 | − ivpu_ipc_rx_mark_free(vdev, rx_msg->ipc_hdr, rx_msg->jsm_msg); | |
| 173 | − atomic_dec(&ipc->rx_msg_count); | |
| 174 | − kfree(rx_msg); | |
| 175 | − } | |
| 176 | − spin_unlock_irq(&cons->rx_msg_lock); | |
| 203 | + spin_unlock_irq(&ipc->cons_lock); | |
| 204 | + | |
| 205 | + spin_lock_irq(&cons->rx_lock); | |
| 206 | + list_for_each_entry_safe(rx_msg, r, &cons->rx_msg_list, link) | |
| 207 | + ivpu_ipc_rx_msg_del(vdev, rx_msg); | |
| 208 | + spin_unlock_irq(&cons->rx_lock); | |
| 177 | 209 | |
| 178 | 210 | ivpu_ipc_tx_release(vdev, cons->tx_vpu_addr); |
| 179 | 211 | } |
| @@ -202,52 +234,61 @@ unlock: · 22 unmodified lines | ||
| 202 | 234 | return ret; |
| 203 | 235 | } |
| 204 | 236 | |
| 237 | +static bool ivpu_ipc_rx_need_wakeup(struct ivpu_ipc_consumer *cons) | |
| 238 | +{ | |
| 239 | + bool ret; | |
| 240 | + | |
| 241 | + spin_lock_irq(&cons->rx_lock); | |
| 242 | + ret = !list_empty(&cons->rx_msg_list) || cons->aborted; | |
| 243 | + spin_unlock_irq(&cons->rx_lock); | |
| 244 | + | |
| 245 | + return ret; | |
| 246 | +} | |
| 247 | + | |
| 205 | 248 | int ivpu_ipc_receive(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, |
| 206 | 249 | struct ivpu_ipc_hdr *ipc_buf, |
| 207 | − struct vpu_jsm_msg *ipc_payload, unsigned long timeout_ms) | |
| 250 | + struct vpu_jsm_msg *jsm_msg, unsigned long timeout_ms) | |
| 208 | 251 | { |
| 209 | − struct ivpu_ipc_info *ipc = vdev->ipc; | |
| 210 | 252 | struct ivpu_ipc_rx_msg *rx_msg; |
| 211 | 253 | int wait_ret, ret = 0; |
| 212 | 254 | |
| 255 | + if (drm_WARN_ONCE(&vdev->drm, cons->rx_callback, "Consumer works only in async mode\n")) | |
| 256 | + return -EINVAL; | |
| 257 | + | |
| 213 | 258 | wait_ret = wait_event_timeout(cons->rx_msg_wq, |
| 214 | − (IS_KTHREAD() && kthread_should_stop()) || | |
| 215 | − !list_empty(&cons->rx_msg_list), | |
| 259 | + ivpu_ipc_rx_need_wakeup(cons), | |
| 216 | 260 | msecs_to_jiffies(timeout_ms)); |
| 217 | 261 | |
| 218 | − if (IS_KTHREAD() && kthread_should_stop()) | |
| 219 | − return -EINTR; | |
| 220 | − | |
| 221 | 262 | if (wait_ret == 0) |
| 222 | 263 | return -ETIMEDOUT; |
| 223 | 264 | |
| 224 | − spin_lock_irq(&cons->rx_msg_lock); | |
| 265 | + spin_lock_irq(&cons->rx_lock); | |
| 266 | + if (cons->aborted) { | |
| 267 | + spin_unlock_irq(&cons->rx_lock); | |
| 268 | + return -ECANCELED; | |
| 269 | + } | |
| 225 | 270 | rx_msg = list_first_entry_or_null(&cons->rx_msg_list, struct ivpu_ipc_rx_msg, link); |
| 226 | 271 | if (!rx_msg) { |
| 227 | − spin_unlock_irq(&cons->rx_msg_lock); | |
| 272 | + spin_unlock_irq(&cons->rx_lock); | |
| 228 | 273 | return -EAGAIN; |
| 229 | 274 | } |
| 230 | − list_del(&rx_msg->link); | |
| 231 | − spin_unlock_irq(&cons->rx_msg_lock); | |
| 232 | 275 | |
| 233 | 276 | if (ipc_buf) |
| 234 | 277 | memcpy(ipc_buf, rx_msg->ipc_hdr, sizeof(*ipc_buf)); |
| 235 | 278 | if (rx_msg->jsm_msg) { |
| 236 | − u32 size = min_t(int, rx_msg->ipc_hdr->data_size, sizeof(*ipc_payload)); | |
| 279 | + u32 size = min_t(int, rx_msg->ipc_hdr->data_size, sizeof(*jsm_msg)); | |
| 237 | 280 | |
| 238 | 281 | if (rx_msg->jsm_msg->result != VPU_JSM_STATUS_SUCCESS) { |
| 239 | 282 | ivpu_dbg(vdev, IPC, "IPC resp result error: %d\n", rx_msg->jsm_msg->result); |
| 240 | 283 | ret = -EBADMSG; |
| 241 | 284 | } |
| 242 | 285 | |
| 243 | − if (ipc_payload) | |
| 244 | − memcpy(ipc_payload, rx_msg->jsm_msg, size); | |
| 286 | + if (jsm_msg) | |
| 287 | + memcpy(jsm_msg, rx_msg->jsm_msg, size); | |
| 245 | 288 | } |
| 246 | 289 | |
| 247 | − ivpu_ipc_rx_mark_free(vdev, rx_msg->ipc_hdr, rx_msg->jsm_msg); | |
| 248 | − atomic_dec(&ipc->rx_msg_count); | |
| 249 | − kfree(rx_msg); | |
| 250 | − | |
| 290 | + ivpu_ipc_rx_msg_del(vdev, rx_msg); | |
| 291 | + spin_unlock_irq(&cons->rx_lock); | |
| 251 | 292 | return ret; |
| 252 | 293 | } |
| 253 | 294 | |
| @@ -260,7 +301,7 @@ ivpu_ipc_send_receive_internal(struct ivpu_device *vdev, struct vpu_jsm_msg *req · 6 unmodified lines | ||
| 260 | 301 | struct ivpu_ipc_consumer cons; |
| 261 | 302 | int ret; |
| 262 | 303 | |
| 263 | − ivpu_ipc_consumer_add(vdev, &cons, channel); | |
| 304 | + ivpu_ipc_consumer_add(vdev, &cons, channel, NULL); | |
| 264 | 305 | |
| 265 | 306 | ret = ivpu_ipc_send(vdev, &cons, req); |
| 266 | 307 | if (ret) { |
| @@ -285,23 +326,19 @@ consumer_del: · 18 unmodified lines | ||
| 285 | 326 | return ret; |
| 286 | 327 | } |
| 287 | 328 | |
| 288 | −int ivpu_ipc_send_receive(struct ivpu_device *vdev, struct vpu_jsm_msg *req, | |
| 289 | − enum vpu_ipc_msg_type expected_resp_type, | |
| 290 | − struct vpu_jsm_msg *resp, u32 channel, | |
| 291 | − unsigned long timeout_ms) | |
| 329 | +int ivpu_ipc_send_receive_active(struct ivpu_device *vdev, struct vpu_jsm_msg *req, | |
| 330 | + enum vpu_ipc_msg_type expected_resp, struct vpu_jsm_msg *resp, | |
| 331 | + u32 channel, unsigned long timeout_ms) | |
| 292 | 332 | { |
| 293 | 333 | struct vpu_jsm_msg hb_req = { .type = VPU_JSM_MSG_QUERY_ENGINE_HB }; |
| 294 | 334 | struct vpu_jsm_msg hb_resp; |
| 295 | 335 | int ret, hb_ret; |
| 296 | 336 | |
| 297 | − ret = ivpu_rpm_get(vdev); | |
| 298 | − if (ret < 0) | |
| 299 | − return ret; | |
| 337 | + drm_WARN_ON(&vdev->drm, pm_runtime_status_suspended(vdev->drm.dev)); | |
| 300 | 338 | |
| 301 | − ret = ivpu_ipc_send_receive_internal(vdev, req, expected_resp_type, resp, | |
| 302 | − channel, timeout_ms); | |
| 339 | + ret = ivpu_ipc_send_receive_internal(vdev, req, expected_resp, resp, channel, timeout_ms); | |
| 303 | 340 | if (ret != -ETIMEDOUT) |
| 304 | − goto rpm_put; | |
| 341 | + return ret; | |
| 305 | 342 | |
| 306 | 343 | hb_ret = ivpu_ipc_send_receive_internal(vdev, &hb_req, VPU_JSM_MSG_QUERY_ENGINE_HB_DONE, |
| 307 | 344 | &hb_resp, VPU_IPC_CHAN_ASYNC_CMD, |
| @@ -311,7 +348,21 @@ int ivpu_ipc_send_receive(struct ivpu_device *vdev, struct vpu_jsm_msg *req, · 3 unmodified lines | ||
| 311 | 348 | ivpu_pm_schedule_recovery(vdev); |
| 312 | 349 | } |
| 313 | 350 | |
| 314 | −rpm_put: | |
| 351 | + return ret; | |
| 352 | +} | |
| 353 | + | |
| 354 | +int ivpu_ipc_send_receive(struct ivpu_device *vdev, struct vpu_jsm_msg *req, | |
| 355 | + enum vpu_ipc_msg_type expected_resp, struct vpu_jsm_msg *resp, | |
| 356 | + u32 channel, unsigned long timeout_ms) | |
| 357 | +{ | |
| 358 | + int ret; | |
| 359 | + | |
| 360 | + ret = ivpu_rpm_get(vdev); | |
| 361 | + if (ret < 0) | |
| 362 | + return ret; | |
| 363 | + | |
| 364 | + ret = ivpu_ipc_send_receive_active(vdev, req, expected_resp, resp, channel, timeout_ms); | |
| 365 | + | |
| 315 | 366 | ivpu_rpm_put(vdev); |
| 316 | 367 | return ret; |
| 317 | 368 | } |
| @@ -329,35 +380,7 @@ ivpu_ipc_match_consumer(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons · 11 unmodified lines | ||
| 329 | 380 | return false; |
| 330 | 381 | } |
| 331 | 382 | |
| 332 | −static void | |
| 333 | −ivpu_ipc_dispatch(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, | |
| 334 | − struct ivpu_ipc_hdr *ipc_hdr, struct vpu_jsm_msg *jsm_msg) | |
| 335 | −{ | |
| 336 | − struct ivpu_ipc_info *ipc = vdev->ipc; | |
| 337 | − struct ivpu_ipc_rx_msg *rx_msg; | |
| 338 | − unsigned long flags; | |
| 339 | − | |
| 340 | − lockdep_assert_held(&ipc->cons_list_lock); | |
| 341 | − | |
| 342 | − rx_msg = kzalloc(sizeof(*rx_msg), GFP_ATOMIC); | |
| 343 | − if (!rx_msg) { | |
| 344 | − ivpu_ipc_rx_mark_free(vdev, ipc_hdr, jsm_msg); | |
| 345 | − return; | |
| 346 | − } | |
| 347 | − | |
| 348 | − atomic_inc(&ipc->rx_msg_count); | |
| 349 | − | |
| 350 | − rx_msg->ipc_hdr = ipc_hdr; | |
| 351 | − rx_msg->jsm_msg = jsm_msg; | |
| 352 | − | |
| 353 | − spin_lock_irqsave(&cons->rx_msg_lock, flags); | |
| 354 | − list_add_tail(&rx_msg->link, &cons->rx_msg_list); | |
| 355 | − spin_unlock_irqrestore(&cons->rx_msg_lock, flags); | |
| 356 | − | |
| 357 | − wake_up(&cons->rx_msg_wq); | |
| 358 | −} | |
| 359 | − | |
| 360 | −int ivpu_ipc_irq_handler(struct ivpu_device *vdev) | |
| 383 | +void ivpu_ipc_irq_handler(struct ivpu_device *vdev, bool *wake_thread) | |
| 361 | 384 | { |
| 362 | 385 | struct ivpu_ipc_info *ipc = vdev->ipc; |
| 363 | 386 | struct ivpu_ipc_consumer *cons; |
| @@ -375,7 +398,7 @@ int ivpu_ipc_irq_handler(struct ivpu_device *vdev) · 11 unmodified lines | ||
| 375 | 398 | vpu_addr = ivpu_hw_reg_ipc_rx_addr_get(vdev); |
| 376 | 399 | if (vpu_addr == REG_IO_ERROR) { |
| 377 | 400 | ivpu_err_ratelimited(vdev, "Failed to read IPC rx addr register\n"); |
| 378 | − return -EIO; | |
| 401 | + return; | |
| 379 | 402 | } |
| 380 | 403 | |
| 381 | 404 | ipc_hdr = ivpu_to_cpu_addr(ipc->mem_rx, vpu_addr); |
| @@ -405,15 +428,15 @@ int ivpu_ipc_irq_handler(struct ivpu_device *vdev) · 23 unmodified lines | ||
| 405 | 428 | } |
| 406 | 429 | |
| 407 | 430 | dispatched = false; |
| 408 | − spin_lock_irqsave(&ipc->cons_list_lock, flags); | |
| 431 | + spin_lock_irqsave(&ipc->cons_lock, flags); | |
| 409 | 432 | list_for_each_entry(cons, &ipc->cons_list, link) { |
| 410 | 433 | if (ivpu_ipc_match_consumer(vdev, cons, ipc_hdr, jsm_msg)) { |
| 411 | − ivpu_ipc_dispatch(vdev, cons, ipc_hdr, jsm_msg); | |
| 434 | + ivpu_ipc_rx_msg_add(vdev, cons, ipc_hdr, jsm_msg); | |
| 412 | 435 | dispatched = true; |
| 413 | 436 | break; |
| 414 | 437 | } |
| 415 | 438 | } |
| 416 | − spin_unlock_irqrestore(&ipc->cons_list_lock, flags); | |
| 439 | + spin_unlock_irqrestore(&ipc->cons_lock, flags); | |
| 417 | 440 | |
| 418 | 441 | if (!dispatched) { |
| 419 | 442 | ivpu_dbg(vdev, IPC, "IPC RX msg 0x%x dropped (no consumer)\n", vpu_addr); |
| @@ -421,7 +444,28 @@ int ivpu_ipc_irq_handler(struct ivpu_device *vdev) · 1 unmodified line | ||
| 421 | 444 | } |
| 422 | 445 | } |
| 423 | 446 | |
| 424 | − return 0; | |
| 447 | + if (wake_thread) | |
| 448 | + *wake_thread = !list_empty(&ipc->cb_msg_list); | |
| 449 | +} | |
| 450 | + | |
| 451 | +irqreturn_t ivpu_ipc_irq_thread_handler(struct ivpu_device *vdev) | |
| 452 | +{ | |
| 453 | + struct ivpu_ipc_info *ipc = vdev->ipc; | |
| 454 | + struct ivpu_ipc_rx_msg *rx_msg, *r; | |
| 455 | + struct list_head cb_msg_list; | |
| 456 | + | |
| 457 | + INIT_LIST_HEAD(&cb_msg_list); | |
| 458 | + | |
| 459 | + spin_lock_irq(&ipc->cons_lock); | |
| 460 | + list_splice_tail_init(&ipc->cb_msg_list, &cb_msg_list); | |
| 461 | + spin_unlock_irq(&ipc->cons_lock); | |
| 462 | + | |
| 463 | + list_for_each_entry_safe(rx_msg, r, &cb_msg_list, link) { | |
| 464 | + rx_msg->callback(vdev, rx_msg->ipc_hdr, rx_msg->jsm_msg); | |
| 465 | + ivpu_ipc_rx_msg_del(vdev, rx_msg); | |
| 466 | + } | |
| 467 | + | |
| 468 | + return IRQ_HANDLED; | |
| 425 | 469 | } |
| 426 | 470 | |
| 427 | 471 | int ivpu_ipc_init(struct ivpu_device *vdev) |
| @@ -456,10 +500,10 @@ int ivpu_ipc_init(struct ivpu_device *vdev) · 28 unmodified lines | ||
| 456 | 500 | goto err_free_rx; |
| 457 | 501 | } |
| 458 | 502 | |
| 503 | + spin_lock_init(&ipc->cons_lock); | |
| 459 | 504 | INIT_LIST_HEAD(&ipc->cons_list); |
| 460 | − spin_lock_init(&ipc->cons_list_lock); | |
| 505 | + INIT_LIST_HEAD(&ipc->cb_msg_list); | |
| 461 | 506 | drmm_mutex_init(&vdev->drm, &ipc->lock); |
| 462 | − | |
| 463 | 507 | ivpu_ipc_reset(vdev); |
| 464 | 508 | return 0; |
| 465 | 509 | |
| @@ -472,6 +516,13 @@ err_free_tx: · 6 unmodified lines | ||
| 472 | 516 | |
| 473 | 517 | void ivpu_ipc_fini(struct ivpu_device *vdev) |
| 474 | 518 | { |
| 519 | + struct ivpu_ipc_info *ipc = vdev->ipc; | |
| 520 | + | |
| 521 | + drm_WARN_ON(&vdev->drm, ipc->on); | |
| 522 | + drm_WARN_ON(&vdev->drm, !list_empty(&ipc->cons_list)); | |
| 523 | + drm_WARN_ON(&vdev->drm, !list_empty(&ipc->cb_msg_list)); | |
| 524 | + drm_WARN_ON(&vdev->drm, atomic_read(&ipc->rx_msg_count) > 0); | |
| 525 | + | |
| 475 | 526 | ivpu_ipc_mem_fini(vdev); |
| 476 | 527 | } |
| 477 | 528 | |
| @@ -488,16 +539,27 @@ void ivpu_ipc_disable(struct ivpu_device *vdev) · 10 unmodified lines | ||
| 488 | 539 | { |
| 489 | 540 | struct ivpu_ipc_info *ipc = vdev->ipc; |
| 490 | 541 | struct ivpu_ipc_consumer *cons, *c; |
| 491 | − unsigned long flags; | |
| 542 | + struct ivpu_ipc_rx_msg *rx_msg, *r; | |
| 543 | + | |
| 544 | + drm_WARN_ON(&vdev->drm, !list_empty(&ipc->cb_msg_list)); | |
| 492 | 545 | |
| 493 | 546 | mutex_lock(&ipc->lock); |
| 494 | 547 | ipc->on = false; |
| 495 | 548 | mutex_unlock(&ipc->lock); |
| 496 | 549 | |
| 497 | − spin_lock_irqsave(&ipc->cons_list_lock, flags); | |
| 498 | − list_for_each_entry_safe(cons, c, &ipc->cons_list, link) | |
| 550 | + spin_lock_irq(&ipc->cons_lock); | |
| 551 | + list_for_each_entry_safe(cons, c, &ipc->cons_list, link) { | |
| 552 | + spin_lock(&cons->rx_lock); | |
| 553 | + if (!cons->rx_callback) | |
| 554 | + cons->aborted = true; | |
| 555 | + list_for_each_entry_safe(rx_msg, r, &cons->rx_msg_list, link) | |
| 556 | + ivpu_ipc_rx_msg_del(vdev, rx_msg); | |
| 557 | + spin_unlock(&cons->rx_lock); | |
| 499 | 558 | wake_up(&cons->rx_msg_wq); |
| 500 | − spin_unlock_irqrestore(&ipc->cons_list_lock, flags); | |
| 559 | + } | |
| 560 | + spin_unlock_irq(&ipc->cons_lock); | |
| 561 | + | |
| 562 | + drm_WARN_ON(&vdev->drm, atomic_read(&ipc->rx_msg_count) > 0); | |
| 501 | 563 | } |
| 502 | 564 | |
| 503 | 565 | void ivpu_ipc_reset(struct ivpu_device *vdev) |
| @@ -505,6 +567,7 @@ void ivpu_ipc_reset(struct ivpu_device *vdev) · 1 unmodified line | ||
| 505 | 567 | struct ivpu_ipc_info *ipc = vdev->ipc; |
| 506 | 568 | |
| 507 | 569 | mutex_lock(&ipc->lock); |
| 570 | + drm_WARN_ON(&vdev->drm, ipc->on); | |
| 508 | 571 | |
| 509 | 572 | memset(ivpu_bo_vaddr(ipc->mem_tx), 0, ivpu_bo_size(ipc->mem_tx)); |
| 510 | 573 | memset(ivpu_bo_vaddr(ipc->mem_rx), 0, ivpu_bo_size(ipc->mem_rx)); |
| @@ -42,13 +42,26 @@ struct ivpu_ipc_hdr { · 41 unmodified lines | ||
| 42 | 42 | u8 status; |
| 43 | 43 | } __packed __aligned(IVPU_IPC_ALIGNMENT); |
| 44 | 44 | |
| 45 | +typedef void (*ivpu_ipc_rx_callback_t)(struct ivpu_device *vdev, | |
| 46 | + struct ivpu_ipc_hdr *ipc_hdr, | |
| 47 | + struct vpu_jsm_msg *jsm_msg); | |
| 48 | + | |
| 49 | +struct ivpu_ipc_rx_msg { | |
| 50 | + struct list_head link; | |
| 51 | + struct ivpu_ipc_hdr *ipc_hdr; | |
| 52 | + struct vpu_jsm_msg *jsm_msg; | |
| 53 | + ivpu_ipc_rx_callback_t callback; | |
| 54 | +}; | |
| 55 | + | |
| 45 | 56 | struct ivpu_ipc_consumer { |
| 46 | 57 | struct list_head link; |
| 47 | 58 | u32 channel; |
| 48 | 59 | u32 tx_vpu_addr; |
| 49 | 60 | u32 request_id; |
| 61 | + bool aborted; | |
| 62 | + ivpu_ipc_rx_callback_t rx_callback; | |
| 50 | 63 | |
| 51 | − spinlock_t rx_msg_lock; /* Protects rx_msg_list */ | |
| 64 | + spinlock_t rx_lock; /* Protects rx_msg_list and aborted */ | |
| 52 | 65 | struct list_head rx_msg_list; |
| 53 | 66 | wait_queue_head_t rx_msg_wq; |
| 54 | 67 | }; |
| @@ -60,8 +73,9 @@ struct ivpu_ipc_info { · 5 unmodified lines | ||
| 60 | 73 | |
| 61 | 74 | atomic_t rx_msg_count; |
| 62 | 75 | |
| 63 | − spinlock_t cons_list_lock; /* Protects cons_list */ | |
| 76 | + spinlock_t cons_lock; /* Protects cons_list and cb_msg_list */ | |
| 64 | 77 | struct list_head cons_list; |
| 78 | + struct list_head cb_msg_list; | |
| 65 | 79 | |
| 66 | 80 | atomic_t request_id; |
| 67 | 81 | struct mutex lock; /* Lock on status */ |
| @@ -75,19 +89,22 @@ void ivpu_ipc_enable(struct ivpu_device *vdev); · 7 unmodified lines | ||
| 75 | 89 | void ivpu_ipc_disable(struct ivpu_device *vdev); |
| 76 | 90 | void ivpu_ipc_reset(struct ivpu_device *vdev); |
| 77 | 91 | |
| 78 | −int ivpu_ipc_irq_handler(struct ivpu_device *vdev); | |
| 92 | +void ivpu_ipc_irq_handler(struct ivpu_device *vdev, bool *wake_thread); | |
| 93 | +irqreturn_t ivpu_ipc_irq_thread_handler(struct ivpu_device *vdev); | |
| 79 | 94 | |
| 80 | 95 | void ivpu_ipc_consumer_add(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, |
| 81 | − u32 channel); | |
| 96 | + u32 channel, ivpu_ipc_rx_callback_t callback); | |
| 82 | 97 | void ivpu_ipc_consumer_del(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons); |
| 83 | 98 | |
| 84 | 99 | int ivpu_ipc_receive(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons, |
| 85 | − struct ivpu_ipc_hdr *ipc_buf, struct vpu_jsm_msg *ipc_payload, | |
| 100 | + struct ivpu_ipc_hdr *ipc_buf, struct vpu_jsm_msg *jsm_msg, | |
| 86 | 101 | unsigned long timeout_ms); |
| 87 | 102 | |
| 103 | +int ivpu_ipc_send_receive_active(struct ivpu_device *vdev, struct vpu_jsm_msg *req, | |
| 104 | + enum vpu_ipc_msg_type expected_resp, struct vpu_jsm_msg *resp, | |
| 105 | + u32 channel, unsigned long timeout_ms); | |
| 88 | 106 | int ivpu_ipc_send_receive(struct ivpu_device *vdev, struct vpu_jsm_msg *req, |
| 89 | − enum vpu_ipc_msg_type expected_resp_type, | |
| 90 | − struct vpu_jsm_msg *resp, u32 channel, | |
| 91 | − unsigned long timeout_ms); | |
| 107 | + enum vpu_ipc_msg_type expected_resp, struct vpu_jsm_msg *resp, | |
| 108 | + u32 channel, unsigned long timeout_ms); | |
| 92 | 109 | |
| 93 | 110 | #endif /* __IVPU_IPC_H__ */ |
| @@ -7,7 +7,6 @@ · 6 unmodified lines | ||
| 7 | 7 | |
| 8 | 8 | #include <linux/bitfield.h> |
| 9 | 9 | #include <linux/highmem.h> |
| 10 | −#include <linux/kthread.h> | |
| 11 | 10 | #include <linux/pci.h> |
| 12 | 11 | #include <linux/module.h> |
| 13 | 12 | #include <uapi/drm/ivpu_accel.h> |
| @@ -24,10 +23,6 @@ · 10 unmodified lines | ||
| 24 | 23 | #define JOB_ID_CONTEXT_MASK GENMASK(31, 8) |
| 25 | 24 | #define JOB_MAX_BUFFER_COUNT 65535 |
| 26 | 25 | |
| 27 | −static unsigned int ivpu_tdr_timeout_ms; | |
| 28 | −module_param_named(tdr_timeout_ms, ivpu_tdr_timeout_ms, uint, 0644); | |
| 29 | −MODULE_PARM_DESC(tdr_timeout_ms, "Timeout for device hang detection, in milliseconds, 0 - default"); | |
| 30 | − | |
| 31 | 26 | static void ivpu_cmdq_ring_db(struct ivpu_device *vdev, struct ivpu_cmdq *cmdq) |
| 32 | 27 | { |
| 33 | 28 | ivpu_hw_reg_db_set(vdev, cmdq->db_id); |
| @@ -196,6 +191,8 @@ static int ivpu_cmdq_push_job(struct ivpu_cmdq *cmdq, struct ivpu_job *job) · 162 unmodified lines | ||
| 196 | 191 | entry->batch_buf_addr = job->cmd_buf_vpu_addr; |
| 197 | 192 | entry->job_id = job->job_id; |
| 198 | 193 | entry->flags = 0; |
| 194 | + if (unlikely(ivpu_test_mode & IVPU_TEST_MODE_NULL_SUBMISSION)) | |
| 195 | + entry->flags = VPU_JOB_FLAGS_NULL_SUBMISSION_MASK; | |
| 199 | 196 | wmb(); /* Ensure that tail is updated after filling entry */ |
| 200 | 197 | header->tail = next_entry; |
| 201 | 198 | wmb(); /* Flush WC buffer for jobq header */ |
| @@ -264,7 +261,7 @@ static void job_release(struct kref *ref) · 62 unmodified lines | ||
| 264 | 261 | |
| 265 | 262 | for (i = 0; i < job->bo_count; i++) |
| 266 | 263 | if (job->bos[i]) |
| 267 | − drm_gem_object_put(&job->bos[i]->base); | |
| 264 | + drm_gem_object_put(&job->bos[i]->base.base); | |
| 268 | 265 | |
| 269 | 266 | dma_fence_put(job->done_fence); |
| 270 | 267 | ivpu_file_priv_put(&job->file_priv); |
| @@ -340,23 +337,12 @@ static int ivpu_job_done(struct ivpu_device *vdev, u32 job_id, u32 job_status) · 69 unmodified lines | ||
| 340 | 337 | ivpu_dbg(vdev, JOB, "Job complete: id %3u ctx %2d engine %d status 0x%x\n", |
| 341 | 338 | job->job_id, job->file_priv->ctx.id, job->engine_idx, job_status); |
| 342 | 339 | |
| 340 | + ivpu_stop_job_timeout_detection(vdev); | |
| 341 | + | |
| 343 | 342 | job_put(job); |
| 344 | 343 | return 0; |
| 345 | 344 | } |
| 346 | 345 | |
| 347 | −static void ivpu_job_done_message(struct ivpu_device *vdev, void *msg) | |
| 348 | −{ | |
| 349 | − struct vpu_ipc_msg_payload_job_done *payload; | |
| 350 | − struct vpu_jsm_msg *job_ret_msg = msg; | |
| 351 | − int ret; | |
| 352 | − | |
| 353 | − payload = (struct vpu_ipc_msg_payload_job_done *)&job_ret_msg->payload; | |
| 354 | − | |
| 355 | − ret = ivpu_job_done(vdev, payload->job_id, payload->job_status); | |
| 356 | − if (ret) | |
| 357 | − ivpu_err(vdev, "Failed to finish job %d: %d\n", payload->job_id, ret); | |
| 358 | −} | |
| 359 | − | |
| 360 | 346 | void ivpu_jobs_abort_all(struct ivpu_device *vdev) |
| 361 | 347 | { |
| 362 | 348 | struct ivpu_job *job; |
| @@ -398,11 +384,13 @@ static int ivpu_direct_job_submission(struct ivpu_job *job) · 35 unmodified lines | ||
| 398 | 384 | if (ret) |
| 399 | 385 | goto err_xa_erase; |
| 400 | 386 | |
| 387 | + ivpu_start_job_timeout_detection(vdev); | |
| 388 | + | |
| 401 | 389 | ivpu_dbg(vdev, JOB, "Job submitted: id %3u addr 0x%llx ctx %2d engine %d next %d\n", |
| 402 | 390 | job->job_id, job->cmd_buf_vpu_addr, file_priv->ctx.id, |
| 403 | 391 | job->engine_idx, cmdq->jobq->header.tail); |
| 404 | 392 | |
| 405 | − if (ivpu_test_mode == IVPU_TEST_MODE_NULL_HW) { | |
| 393 | + if (ivpu_test_mode & IVPU_TEST_MODE_NULL_HW) { | |
| 406 | 394 | ivpu_job_done(vdev, job->job_id, VPU_JSM_STATUS_SUCCESS); |
| 407 | 395 | cmdq->jobq->header.head = cmdq->jobq->header.tail; |
| 408 | 396 | wmb(); /* Flush WC buffer for jobq header */ |
| @@ -448,7 +436,7 @@ ivpu_job_prepare_bos_for_submit(struct drm_file *file, struct ivpu_job *job, u32 · 39 unmodified lines | ||
| 448 | 436 | } |
| 449 | 437 | |
| 450 | 438 | bo = job->bos[CMD_BUF_IDX]; |
| 451 | − if (!dma_resv_test_signaled(bo->base.resv, DMA_RESV_USAGE_READ)) { | |
| 439 | + if (!dma_resv_test_signaled(bo->base.base.resv, DMA_RESV_USAGE_READ)) { | |
| 452 | 440 | ivpu_warn(vdev, "Buffer is already in use\n"); |
| 453 | 441 | return -EBUSY; |
| 454 | 442 | } |
| @@ -468,7 +456,7 @@ ivpu_job_prepare_bos_for_submit(struct drm_file *file, struct ivpu_job *job, u32 · 13 unmodified lines | ||
| 468 | 456 | } |
| 469 | 457 | |
| 470 | 458 | for (i = 0; i < buf_count; i++) { |
| 471 | − ret = dma_resv_reserve_fences(job->bos[i]->base.resv, 1); | |
| 459 | + ret = dma_resv_reserve_fences(job->bos[i]->base.base.resv, 1); | |
| 472 | 460 | if (ret) { |
| 473 | 461 | ivpu_warn(vdev, "Failed to reserve fences: %d\n", ret); |
| 474 | 462 | goto unlock_reservations; |
| @@ -477,7 +465,7 @@ ivpu_job_prepare_bos_for_submit(struct drm_file *file, struct ivpu_job *job, u32 · 2 unmodified lines | ||
| 477 | 465 | |
| 478 | 466 | for (i = 0; i < buf_count; i++) { |
| 479 | 467 | usage = (i == CMD_BUF_IDX) ? DMA_RESV_USAGE_WRITE : DMA_RESV_USAGE_BOOKKEEP; |
| 480 | − dma_resv_add_fence(job->bos[i]->base.resv, job->done_fence, usage); | |
| 468 | + dma_resv_add_fence(job->bos[i]->base.base.resv, job->done_fence, usage); | |
| 481 | 469 | } |
| 482 | 470 | |
| 483 | 471 | unlock_reservations: |
| @@ -562,61 +550,36 @@ free_handles: · 78 unmodified lines | ||
| 562 | 550 | return ret; |
| 563 | 551 | } |
| 564 | 552 | |
| 565 | −static int ivpu_job_done_thread(void *arg) | |
| 553 | +static void | |
| 554 | +ivpu_job_done_callback(struct ivpu_device *vdev, struct ivpu_ipc_hdr *ipc_hdr, | |
| 555 | + struct vpu_jsm_msg *jsm_msg) | |
| 566 | 556 | { |
| 567 | − struct ivpu_device *vdev = (struct ivpu_device *)arg; | |
| 568 | − struct ivpu_ipc_consumer cons; | |
| 569 | − struct vpu_jsm_msg jsm_msg; | |
| 570 | − bool jobs_submitted; | |
| 571 | − unsigned int timeout; | |
| 557 | + struct vpu_ipc_msg_payload_job_done *payload; | |
| 572 | 558 | int ret; |
| 573 | 559 | |
| 574 | − ivpu_dbg(vdev, JOB, "Started %s\n", __func__); | |
| 575 | − | |
| 576 | − ivpu_ipc_consumer_add(vdev, &cons, VPU_IPC_CHAN_JOB_RET); | |
| 577 | − | |
| 578 | − while (!kthread_should_stop()) { | |
| 579 | − timeout = ivpu_tdr_timeout_ms ? ivpu_tdr_timeout_ms : vdev->timeout.tdr; | |
| 580 | − jobs_submitted = !xa_empty(&vdev->submitted_jobs_xa); | |
| 581 | − ret = ivpu_ipc_receive(vdev, &cons, NULL, &jsm_msg, timeout); | |
| 582 | − if (!ret) { | |
| 583 | − ivpu_job_done_message(vdev, &jsm_msg); | |
| 584 | − } else if (ret == -ETIMEDOUT) { | |
| 585 | − if (jobs_submitted && !xa_empty(&vdev->submitted_jobs_xa)) { | |
| 586 | − ivpu_err(vdev, "TDR detected, timeout %d ms", timeout); | |
| 587 | − ivpu_hw_diagnose_failure(vdev); | |
| 588 | − ivpu_pm_schedule_recovery(vdev); | |
| 589 | − } | |
| 590 | − } | |
| 560 | + if (!jsm_msg) { | |
| 561 | + ivpu_err(vdev, "IPC message has no JSM payload\n"); | |
| 562 | + return; | |
| 591 | 563 | } |
| 592 | 564 | |
| 593 | − ivpu_ipc_consumer_del(vdev, &cons); | |
| 594 | − | |
| 595 | − ivpu_jobs_abort_all(vdev); | |
| 565 | + if (jsm_msg->result != VPU_JSM_STATUS_SUCCESS) { | |
| 566 | + ivpu_err(vdev, "Invalid JSM message result: %d\n", jsm_msg->result); | |
| 567 | + return; | |
| 568 | + } | |
| 596 | 569 | |
| 597 | − ivpu_dbg(vdev, JOB, "Stopped %s\n", __func__); | |
| 598 | − return 0; | |
| 570 | + payload = (struct vpu_ipc_msg_payload_job_done *)&jsm_msg->payload; | |
| 571 | + ret = ivpu_job_done(vdev, payload->job_id, payload->job_status); | |
| 572 | + if (!ret && !xa_empty(&vdev->submitted_jobs_xa)) | |
| 573 | + ivpu_start_job_timeout_detection(vdev); | |
| 599 | 574 | } |
| 600 | 575 | |
| 601 | −int ivpu_job_done_thread_init(struct ivpu_device *vdev) | |
| 576 | +void ivpu_job_done_consumer_init(struct ivpu_device *vdev) | |
| 602 | 577 | { |
| 603 | − struct task_struct *thread; | |
| 604 | − | |
| 605 | − thread = kthread_run(&ivpu_job_done_thread, (void *)vdev, "ivpu_job_done_thread"); | |
| 606 | − if (IS_ERR(thread)) { | |
| 607 | − ivpu_err(vdev, "Failed to start job completion thread\n"); | |
| 608 | − return -EIO; | |
| 609 | − } | |
| 610 | − | |
| 611 | − get_task_struct(thread); | |
| 612 | − wake_up_process(thread); | |
| 613 | − | |
| 614 | − vdev->job_done_thread = thread; | |
| 615 | − | |
| 616 | − return 0; | |
| 578 | + ivpu_ipc_consumer_add(vdev, &vdev->job_done_consumer, | |
| 579 | + VPU_IPC_CHAN_JOB_RET, ivpu_job_done_callback); | |
| 617 | 580 | } |
| 618 | 581 | |
| 619 | −void ivpu_job_done_thread_fini(struct ivpu_device *vdev) | |
| 582 | +void ivpu_job_done_consumer_fini(struct ivpu_device *vdev) | |
| 620 | 583 | { |
| 621 | − kthread_stop_put(vdev->job_done_thread); | |
| 584 | + ivpu_ipc_consumer_del(vdev, &vdev->job_done_consumer); | |
| 622 | 585 | } |
| @@ -59,8 +59,8 @@ int ivpu_submit_ioctl(struct drm_device *dev, void *data, struct drm_file *file) · 58 unmodified lines | ||
| 59 | 59 | void ivpu_cmdq_release_all(struct ivpu_file_priv *file_priv); |
| 60 | 60 | void ivpu_cmdq_reset_all_contexts(struct ivpu_device *vdev); |
| 61 | 61 | |
| 62 | −int ivpu_job_done_thread_init(struct ivpu_device *vdev); | |
| 63 | −void ivpu_job_done_thread_fini(struct ivpu_device *vdev); | |
| 62 | +void ivpu_job_done_consumer_init(struct ivpu_device *vdev); | |
| 63 | +void ivpu_job_done_consumer_fini(struct ivpu_device *vdev); | |
| 64 | 64 | |
| 65 | 65 | void ivpu_jobs_abort_all(struct ivpu_device *vdev); |
| 66 | 66 | |
| @@ -4,6 +4,7 @@ · 3 unmodified lines | ||
| 4 | 4 | */ |
| 5 | 5 | |
| 6 | 6 | #include "ivpu_drv.h" |
| 7 | +#include "ivpu_hw.h" | |
| 7 | 8 | #include "ivpu_ipc.h" |
| 8 | 9 | #include "ivpu_jsm_msg.h" |
| 9 | 10 | |
| @@ -36,6 +37,17 @@ const char *ivpu_jsm_msg_type_to_str(enum vpu_ipc_msg_type type) · 26 unmodified lines | ||
| 36 | 37 | IVPU_CASE_TO_STR(VPU_JSM_MSG_DESTROY_CMD_QUEUE); |
| 37 | 38 | IVPU_CASE_TO_STR(VPU_JSM_MSG_SET_CONTEXT_SCHED_PROPERTIES); |
| 38 | 39 | IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_REGISTER_DB); |
| 40 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_RESUME_CMDQ); | |
| 41 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_SUSPEND_CMDQ); | |
| 42 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_RESUME_CMDQ_RSP); | |
| 43 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_SUSPEND_CMDQ_DONE); | |
| 44 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG); | |
| 45 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG_RSP); | |
| 46 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION); | |
| 47 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_ENGINE_RESUME); | |
| 48 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_HWS_RESUME_ENGINE_DONE); | |
| 49 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_STATE_DUMP); | |
| 50 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_STATE_DUMP_RSP); | |
| 39 | 51 | IVPU_CASE_TO_STR(VPU_JSM_MSG_BLOB_DEINIT); |
| 40 | 52 | IVPU_CASE_TO_STR(VPU_JSM_MSG_DYNDBG_CONTROL); |
| 41 | 53 | IVPU_CASE_TO_STR(VPU_JSM_MSG_JOB_DONE); |
| @@ -65,6 +77,12 @@ const char *ivpu_jsm_msg_type_to_str(enum vpu_ipc_msg_type type) · 23 unmodified lines | ||
| 65 | 77 | IVPU_CASE_TO_STR(VPU_JSM_MSG_SET_CONTEXT_SCHED_PROPERTIES_RSP); |
| 66 | 78 | IVPU_CASE_TO_STR(VPU_JSM_MSG_BLOB_DEINIT_DONE); |
| 67 | 79 | IVPU_CASE_TO_STR(VPU_JSM_MSG_DYNDBG_CONTROL_RSP); |
| 80 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_PWR_D0I3_ENTER); | |
| 81 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_PWR_D0I3_ENTER_DONE); | |
| 82 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_DCT_ENABLE); | |
| 83 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_DCT_ENABLE_DONE); | |
| 84 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_DCT_DISABLE); | |
| 85 | + IVPU_CASE_TO_STR(VPU_JSM_MSG_DCT_DISABLE_DONE); | |
| 68 | 86 | } |
| 69 | 87 | #undef IVPU_CASE_TO_STR |
| 70 | 88 | |
| @@ -243,3 +261,23 @@ int ivpu_jsm_context_release(struct ivpu_device *vdev, u32 host_ssid) · 172 unmodified lines | ||
| 243 | 261 | return ivpu_ipc_send_receive(vdev, &req, VPU_JSM_MSG_SSID_RELEASE_DONE, &resp, |
| 244 | 262 | VPU_IPC_CHAN_ASYNC_CMD, vdev->timeout.jsm); |
| 245 | 263 | } |
| 264 | + | |
| 265 | +int ivpu_jsm_pwr_d0i3_enter(struct ivpu_device *vdev) | |
| 266 | +{ | |
| 267 | + struct vpu_jsm_msg req = { .type = VPU_JSM_MSG_PWR_D0I3_ENTER }; | |
| 268 | + struct vpu_jsm_msg resp; | |
| 269 | + int ret; | |
| 270 | + | |
| 271 | + if (IVPU_WA(disable_d0i3_msg)) | |
| 272 | + return 0; | |
| 273 | + | |
| 274 | + req.payload.pwr_d0i3_enter.send_response = 1; | |
| 275 | + | |
| 276 | + ret = ivpu_ipc_send_receive_active(vdev, &req, VPU_JSM_MSG_PWR_D0I3_ENTER_DONE, | |
| 277 | + &resp, VPU_IPC_CHAN_GEN_CMD, | |
| 278 | + vdev->timeout.d0i3_entry_msg); | |
| 279 | + if (ret) | |
| 280 | + return ret; | |
| 281 | + | |
| 282 | + return ivpu_hw_wait_for_idle(vdev); | |
| 283 | +} | |
| @@ -22,4 +22,5 @@ int ivpu_jsm_trace_get_capability(struct ivpu_device *vdev, u32 *trace_destinati · 21 unmodified lines | ||
| 22 | 22 | int ivpu_jsm_trace_set_config(struct ivpu_device *vdev, u32 trace_level, u32 trace_destination_mask, |
| 23 | 23 | u64 trace_hw_component_mask); |
| 24 | 24 | int ivpu_jsm_context_release(struct ivpu_device *vdev, u32 host_ssid); |
| 25 | +int ivpu_jsm_pwr_d0i3_enter(struct ivpu_device *vdev); | |
| 25 | 26 | #endif |
| @@ -230,7 +230,12 @@ · 229 unmodified lines | ||
| 230 | 230 | (REG_FLD(IVPU_MMU_REG_GERROR, MSI_PRIQ_ABT)) | \ |
| 231 | 231 | (REG_FLD(IVPU_MMU_REG_GERROR, MSI_ABT))) |
| 232 | 232 | |
| 233 | −static char *ivpu_mmu_event_to_str(u32 cmd) | |
| 233 | +#define IVPU_MMU_CERROR_NONE 0x0 | |
| 234 | +#define IVPU_MMU_CERROR_ILL 0x1 | |
| 235 | +#define IVPU_MMU_CERROR_ABT 0x2 | |
| 236 | +#define IVPU_MMU_CERROR_ATC_INV_SYNC 0x3 | |
| 237 | + | |
| 238 | +static const char *ivpu_mmu_event_to_str(u32 cmd) | |
| 234 | 239 | { |
| 235 | 240 | switch (cmd) { |
| 236 | 241 | case IVPU_MMU_EVT_F_UUT: |
| @@ -276,6 +281,22 @@ static char *ivpu_mmu_event_to_str(u32 cmd) · 39 unmodified lines | ||
| 276 | 281 | } |
| 277 | 282 | } |
| 278 | 283 | |
| 284 | +static const char *ivpu_mmu_cmdq_err_to_str(u32 err) | |
| 285 | +{ | |
| 286 | + switch (err) { | |
| 287 | + case IVPU_MMU_CERROR_NONE: | |
| 288 | + return "No CMDQ Error"; | |
| 289 | + case IVPU_MMU_CERROR_ILL: | |
| 290 | + return "Illegal command"; | |
| 291 | + case IVPU_MMU_CERROR_ABT: | |
| 292 | + return "External abort on CMDQ read"; | |
| 293 | + case IVPU_MMU_CERROR_ATC_INV_SYNC: | |
| 294 | + return "Sync failed to complete ATS invalidation"; | |
| 295 | + default: | |
| 296 | + return "Unknown CMDQ Error"; | |
| 297 | + } | |
| 298 | +} | |
| 299 | + | |
| 279 | 300 | static void ivpu_mmu_config_check(struct ivpu_device *vdev) |
| 280 | 301 | { |
| 281 | 302 | u32 val_ref; |
| @@ -479,10 +500,7 @@ static int ivpu_mmu_cmdq_sync(struct ivpu_device *vdev) · 197 unmodified lines | ||
| 479 | 500 | u64 val; |
| 480 | 501 | int ret; |
| 481 | 502 | |
| 482 | − val = FIELD_PREP(IVPU_MMU_CMD_OPCODE, CMD_SYNC) | | |
| 483 | − FIELD_PREP(IVPU_MMU_CMD_SYNC_0_CS, 0x2) | | |
| 484 | − FIELD_PREP(IVPU_MMU_CMD_SYNC_0_MSH, 0x3) | | |
| 485 | − FIELD_PREP(IVPU_MMU_CMD_SYNC_0_MSI_ATTR, 0xf); | |
| 503 | + val = FIELD_PREP(IVPU_MMU_CMD_OPCODE, CMD_SYNC); | |
| 486 | 504 | |
| 487 | 505 | ret = ivpu_mmu_cmdq_cmd_write(vdev, "SYNC", val, 0); |
| 488 | 506 | if (ret) |
| @@ -492,8 +510,15 @@ static int ivpu_mmu_cmdq_sync(struct ivpu_device *vdev) · 3 unmodified lines | ||
| 492 | 510 | REGV_WR32(IVPU_MMU_REG_CMDQ_PROD, q->prod); |
| 493 | 511 | |
| 494 | 512 | ret = ivpu_mmu_cmdq_wait_for_cons(vdev); |
| 495 | − if (ret) | |
| 496 | − ivpu_err(vdev, "Timed out waiting for consumer: %d\n", ret); | |
| 513 | + if (ret) { | |
| 514 | + u32 err; | |
| 515 | + | |
| 516 | + val = REGV_RD32(IVPU_MMU_REG_CMDQ_CONS); | |
| 517 | + err = REG_GET_FLD(IVPU_MMU_REG_CMDQ_CONS, ERR, val); | |
| 518 | + | |
| 519 | + ivpu_err(vdev, "Timed out waiting for MMU consumer: %d, error: %s\n", ret, | |
| 520 | + ivpu_mmu_cmdq_err_to_str(err)); | |
| 521 | + } | |
| 497 | 522 | |
| 498 | 523 | return ret; |
| 499 | 524 | } |
| @@ -750,9 +775,12 @@ int ivpu_mmu_init(struct ivpu_device *vdev) · 250 unmodified lines | ||
| 750 | 775 | |
| 751 | 776 | ivpu_dbg(vdev, MMU, "Init..\n"); |
| 752 | 777 | |
| 753 | − drmm_mutex_init(&vdev->drm, &mmu->lock); | |
| 754 | 778 | ivpu_mmu_config_check(vdev); |
| 755 | 779 | |
| 780 | + ret = drmm_mutex_init(&vdev->drm, &mmu->lock); | |
| 781 | + if (ret) | |
| 782 | + return ret; | |
| 783 | + | |
| 756 | 784 | ret = ivpu_mmu_structs_alloc(vdev); |
| 757 | 785 | if (ret) |
| 758 | 786 | return ret; |
| @@ -5,6 +5,9 @@ · 4 unmodified lines | ||
| 5 | 5 | |
| 6 | 6 | #include <linux/bitfield.h> |
| 7 | 7 | #include <linux/highmem.h> |
| 8 | +#include <linux/set_memory.h> | |
| 9 | + | |
| 10 | +#include <drm/drm_cache.h> | |
| 8 | 11 | |
| 9 | 12 | #include "ivpu_drv.h" |
| 10 | 13 | #include "ivpu_hw.h" |
| @@ -39,12 +42,57 @@ · 28 unmodified lines | ||
| 39 | 42 | #define IVPU_MMU_ENTRY_MAPPED (IVPU_MMU_ENTRY_FLAG_AF | IVPU_MMU_ENTRY_FLAG_USER | \ |
| 40 | 43 | IVPU_MMU_ENTRY_FLAG_NG | IVPU_MMU_ENTRY_VALID) |
| 41 | 44 | |
| 45 | +static void *ivpu_pgtable_alloc_page(struct ivpu_device *vdev, dma_addr_t *dma) | |
| 46 | +{ | |
| 47 | + dma_addr_t dma_addr; | |
| 48 | + struct page *page; | |
| 49 | + void *cpu; | |
| 50 | + | |
| 51 | + page = alloc_page(GFP_KERNEL | __GFP_HIGHMEM | __GFP_ZERO); | |
| 52 | + if (!page) | |
| 53 | + return NULL; | |
| 54 | + | |
| 55 | + set_pages_array_wc(&page, 1); | |
| 56 | + | |
| 57 | + dma_addr = dma_map_page(vdev->drm.dev, page, 0, PAGE_SIZE, DMA_BIDIRECTIONAL); | |
| 58 | + if (dma_mapping_error(vdev->drm.dev, dma_addr)) | |
| 59 | + goto err_free_page; | |
| 60 | + | |
| 61 | + cpu = vmap(&page, 1, VM_MAP, pgprot_writecombine(PAGE_KERNEL)); | |
| 62 | + if (!cpu) | |
| 63 | + goto err_dma_unmap_page; | |
| 64 | + | |
| 65 | + | |
| 66 | + *dma = dma_addr; | |
| 67 | + return cpu; | |
| 68 | + | |
| 69 | +err_dma_unmap_page: | |
| 70 | + dma_unmap_page(vdev->drm.dev, dma_addr, PAGE_SIZE, DMA_BIDIRECTIONAL); | |
| 71 | + | |
| 72 | +err_free_page: | |
| 73 | + put_page(page); | |
| 74 | + return NULL; | |
| 75 | +} | |
| 76 | + | |
| 77 | +static void ivpu_pgtable_free_page(struct ivpu_device *vdev, u64 *cpu_addr, dma_addr_t dma_addr) | |
| 78 | +{ | |
| 79 | + struct page *page; | |
| 80 | + | |
| 81 | + if (cpu_addr) { | |
| 82 | + page = vmalloc_to_page(cpu_addr); | |
| 83 | + vunmap(cpu_addr); | |
| 84 | + dma_unmap_page(vdev->drm.dev, dma_addr & ~IVPU_MMU_ENTRY_FLAGS_MASK, PAGE_SIZE, | |
| 85 | + DMA_BIDIRECTIONAL); | |
| 86 | + set_pages_array_wb(&page, 1); | |
| 87 | + put_page(page); | |
| 88 | + } | |
| 89 | +} | |
| 90 | + | |
| 42 | 91 | static int ivpu_mmu_pgtable_init(struct ivpu_device *vdev, struct ivpu_mmu_pgtable *pgtable) |
| 43 | 92 | { |
| 44 | 93 | dma_addr_t pgd_dma; |
| 45 | 94 | |
| 46 | − pgtable->pgd_dma_ptr = dma_alloc_coherent(vdev->drm.dev, IVPU_MMU_PGTABLE_SIZE, &pgd_dma, | |
| 47 | − GFP_KERNEL); | |
| 95 | + pgtable->pgd_dma_ptr = ivpu_pgtable_alloc_page(vdev, &pgd_dma); | |
| 48 | 96 | if (!pgtable->pgd_dma_ptr) |
| 49 | 97 | return -ENOMEM; |
| 50 | 98 | |
| @@ -53,13 +101,6 @@ static int ivpu_mmu_pgtable_init(struct ivpu_device *vdev, struct ivpu_mmu_pgtab · 2 unmodified lines | ||
| 53 | 101 | return 0; |
| 54 | 102 | } |
| 55 | 103 | |
| 56 | −static void ivpu_mmu_pgtable_free(struct ivpu_device *vdev, u64 *cpu_addr, dma_addr_t dma_addr) | |
| 57 | −{ | |
| 58 | − if (cpu_addr) | |
| 59 | − dma_free_coherent(vdev->drm.dev, IVPU_MMU_PGTABLE_SIZE, cpu_addr, | |
| 60 | − dma_addr & ~IVPU_MMU_ENTRY_FLAGS_MASK); | |
| 61 | −} | |
| 62 | − | |
| 63 | 104 | static void ivpu_mmu_pgtables_free(struct ivpu_device *vdev, struct ivpu_mmu_pgtable *pgtable) |
| 64 | 105 | { |
| 65 | 106 | int pgd_idx, pud_idx, pmd_idx; |
| @@ -84,19 +125,19 @@ static void ivpu_mmu_pgtables_free(struct ivpu_device *vdev, struct ivpu_mmu_pgt · 18 unmodified lines | ||
| 84 | 125 | pte_dma_ptr = pgtable->pte_ptrs[pgd_idx][pud_idx][pmd_idx]; |
| 85 | 126 | pte_dma = pgtable->pmd_ptrs[pgd_idx][pud_idx][pmd_idx]; |
| 86 | 127 | |
| 87 | − ivpu_mmu_pgtable_free(vdev, pte_dma_ptr, pte_dma); | |
| 128 | + ivpu_pgtable_free_page(vdev, pte_dma_ptr, pte_dma); | |
| 88 | 129 | } |
| 89 | 130 | |
| 90 | 131 | kfree(pgtable->pte_ptrs[pgd_idx][pud_idx]); |
| 91 | − ivpu_mmu_pgtable_free(vdev, pmd_dma_ptr, pmd_dma); | |
| 132 | + ivpu_pgtable_free_page(vdev, pmd_dma_ptr, pmd_dma); | |
| 92 | 133 | } |
| 93 | 134 | |
| 94 | 135 | kfree(pgtable->pmd_ptrs[pgd_idx]); |
| 95 | 136 | kfree(pgtable->pte_ptrs[pgd_idx]); |
| 96 | − ivpu_mmu_pgtable_free(vdev, pud_dma_ptr, pud_dma); | |
| 137 | + ivpu_pgtable_free_page(vdev, pud_dma_ptr, pud_dma); | |
| 97 | 138 | } |
| 98 | 139 | |
| 99 | − ivpu_mmu_pgtable_free(vdev, pgtable->pgd_dma_ptr, pgtable->pgd_dma); | |
| 140 | + ivpu_pgtable_free_page(vdev, pgtable->pgd_dma_ptr, pgtable->pgd_dma); | |
| 100 | 141 | } |
| 101 | 142 | |
| 102 | 143 | static u64* |
| @@ -108,7 +149,7 @@ ivpu_mmu_ensure_pud(struct ivpu_device *vdev, struct ivpu_mmu_pgtable *pgtable, · 5 unmodified lines | ||
| 108 | 149 | if (pud_dma_ptr) |
| 109 | 150 | return pud_dma_ptr; |
| 110 | 151 | |
| 111 | − pud_dma_ptr = dma_alloc_wc(vdev->drm.dev, IVPU_MMU_PGTABLE_SIZE, &pud_dma, GFP_KERNEL); | |
| 152 | + pud_dma_ptr = ivpu_pgtable_alloc_page(vdev, &pud_dma); | |
| 112 | 153 | if (!pud_dma_ptr) |
| 113 | 154 | return NULL; |
| 114 | 155 | |
| @@ -131,7 +172,7 @@ err_free_pmd_ptrs: · 16 unmodified lines | ||
| 131 | 172 | kfree(pgtable->pmd_ptrs[pgd_idx]); |
| 132 | 173 | |
| 133 | 174 | err_free_pud_dma_ptr: |
| 134 | − ivpu_mmu_pgtable_free(vdev, pud_dma_ptr, pud_dma); | |
| 175 | + ivpu_pgtable_free_page(vdev, pud_dma_ptr, pud_dma); | |
| 135 | 176 | return NULL; |
| 136 | 177 | } |
| 137 | 178 | |
| @@ -145,7 +186,7 @@ ivpu_mmu_ensure_pmd(struct ivpu_device *vdev, struct ivpu_mmu_pgtable *pgtable, · 7 unmodified lines | ||
| 145 | 186 | if (pmd_dma_ptr) |
| 146 | 187 | return pmd_dma_ptr; |
| 147 | 188 | |
| 148 | − pmd_dma_ptr = dma_alloc_wc(vdev->drm.dev, IVPU_MMU_PGTABLE_SIZE, &pmd_dma, GFP_KERNEL); | |
| 189 | + pmd_dma_ptr = ivpu_pgtable_alloc_page(vdev, &pmd_dma); | |
| 149 | 190 | if (!pmd_dma_ptr) |
| 150 | 191 | return NULL; |
| 151 | 192 | |
| @@ -160,7 +201,7 @@ ivpu_mmu_ensure_pmd(struct ivpu_device *vdev, struct ivpu_mmu_pgtable *pgtable, · 8 unmodified lines | ||
| 160 | 201 | return pmd_dma_ptr; |
| 161 | 202 | |
| 162 | 203 | err_free_pmd_dma_ptr: |
| 163 | − ivpu_mmu_pgtable_free(vdev, pmd_dma_ptr, pmd_dma); | |
| 204 | + ivpu_pgtable_free_page(vdev, pmd_dma_ptr, pmd_dma); | |
| 164 | 205 | return NULL; |
| 165 | 206 | } |
| 166 | 207 | |
| @@ -174,7 +215,7 @@ ivpu_mmu_ensure_pte(struct ivpu_device *vdev, struct ivpu_mmu_pgtable *pgtable, · 7 unmodified lines | ||
| 174 | 215 | if (pte_dma_ptr) |
| 175 | 216 | return pte_dma_ptr; |
| 176 | 217 | |
| 177 | − pte_dma_ptr = dma_alloc_wc(vdev->drm.dev, IVPU_MMU_PGTABLE_SIZE, &pte_dma, GFP_KERNEL); | |
| 218 | + pte_dma_ptr = ivpu_pgtable_alloc_page(vdev, &pte_dma); | |
| 178 | 219 | if (!pte_dma_ptr) |
| 179 | 220 | return NULL; |
| 180 | 221 | |
| @@ -249,38 +290,6 @@ static void ivpu_mmu_context_unmap_page(struct ivpu_mmu_context *ctx, u64 vpu_ad · 68 unmodified lines | ||
| 249 | 290 | ctx->pgtable.pte_ptrs[pgd_idx][pud_idx][pmd_idx][pte_idx] = IVPU_MMU_ENTRY_INVALID; |
| 250 | 291 | } |
| 251 | 292 | |
| 252 | −static void | |
| 253 | −ivpu_mmu_context_flush_page_tables(struct ivpu_mmu_context *ctx, u64 vpu_addr, size_t size) | |
| 254 | −{ | |
| 255 | − struct ivpu_mmu_pgtable *pgtable = &ctx->pgtable; | |
| 256 | − u64 end_addr = vpu_addr + size; | |
| 257 | − | |
| 258 | − /* Align to PMD entry (2 MB) */ | |
| 259 | − vpu_addr &= ~(IVPU_MMU_PTE_MAP_SIZE - 1); | |
| 260 | − | |
| 261 | − while (vpu_addr < end_addr) { | |
| 262 | − int pgd_idx = FIELD_GET(IVPU_MMU_PGD_INDEX_MASK, vpu_addr); | |
| 263 | − u64 pud_end = (pgd_idx + 1) * (u64)IVPU_MMU_PUD_MAP_SIZE; | |
| 264 | − | |
| 265 | − while (vpu_addr < end_addr && vpu_addr < pud_end) { | |
| 266 | − int pud_idx = FIELD_GET(IVPU_MMU_PUD_INDEX_MASK, vpu_addr); | |
| 267 | − u64 pmd_end = (pud_idx + 1) * (u64)IVPU_MMU_PMD_MAP_SIZE; | |
| 268 | − | |
| 269 | − while (vpu_addr < end_addr && vpu_addr < pmd_end) { | |
| 270 | − int pmd_idx = FIELD_GET(IVPU_MMU_PMD_INDEX_MASK, vpu_addr); | |
| 271 | − | |
| 272 | − clflush_cache_range(pgtable->pte_ptrs[pgd_idx][pud_idx][pmd_idx], | |
| 273 | − IVPU_MMU_PGTABLE_SIZE); | |
| 274 | − vpu_addr += IVPU_MMU_PTE_MAP_SIZE; | |
| 275 | − } | |
| 276 | − clflush_cache_range(pgtable->pmd_ptrs[pgd_idx][pud_idx], | |
| 277 | − IVPU_MMU_PGTABLE_SIZE); | |
| 278 | − } | |
| 279 | − clflush_cache_range(pgtable->pud_ptrs[pgd_idx], IVPU_MMU_PGTABLE_SIZE); | |
| 280 | − } | |
| 281 | − clflush_cache_range(pgtable->pgd_dma_ptr, IVPU_MMU_PGTABLE_SIZE); | |
| 282 | −} | |
| 283 | − | |
| 284 | 293 | static int |
| 285 | 294 | ivpu_mmu_context_map_pages(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, |
| 286 | 295 | u64 vpu_addr, dma_addr_t dma_addr, size_t size, u64 prot) |
| @@ -327,6 +336,9 @@ ivpu_mmu_context_map_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, · 40 unmodified lines | ||
| 327 | 336 | u64 prot; |
| 328 | 337 | u64 i; |
| 329 | 338 | |
| 339 | + if (drm_WARN_ON(&vdev->drm, !ctx)) | |
| 340 | + return -EINVAL; | |
| 341 | + | |
| 330 | 342 | if (!IS_ALIGNED(vpu_addr, IVPU_MMU_PAGE_SIZE)) |
| 331 | 343 | return -EINVAL; |
| 332 | 344 | |
| @@ -349,10 +361,11 @@ ivpu_mmu_context_map_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, · 16 unmodified lines | ||
| 349 | 361 | mutex_unlock(&ctx->lock); |
| 350 | 362 | return ret; |
| 351 | 363 | } |
| 352 | − ivpu_mmu_context_flush_page_tables(ctx, vpu_addr, size); | |
| 353 | 364 | vpu_addr += size; |
| 354 | 365 | } |
| 355 | 366 | |
| 367 | + /* Ensure page table modifications are flushed from wc buffers to memory */ | |
| 368 | + wmb(); | |
| 356 | 369 | mutex_unlock(&ctx->lock); |
| 357 | 370 | |
| 358 | 371 | ret = ivpu_mmu_invalidate_tlb(vdev, ctx->id); |
| @@ -369,8 +382,8 @@ ivpu_mmu_context_unmap_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ct · 10 unmodified lines | ||
| 369 | 382 | int ret; |
| 370 | 383 | u64 i; |
| 371 | 384 | |
| 372 | − if (!IS_ALIGNED(vpu_addr, IVPU_MMU_PAGE_SIZE)) | |
| 373 | − ivpu_warn(vdev, "Unaligned vpu_addr: 0x%llx\n", vpu_addr); | |
| 385 | + if (drm_WARN_ON(&vdev->drm, !ctx)) | |
| 386 | + return; | |
| 374 | 387 | |
| 375 | 388 | mutex_lock(&ctx->lock); |
| 376 | 389 | |
| @@ -378,10 +391,11 @@ ivpu_mmu_context_unmap_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ct · 1 unmodified line | ||
| 378 | 391 | size_t size = sg_dma_len(sg) + sg->offset; |
| 379 | 392 | |
| 380 | 393 | ivpu_mmu_context_unmap_pages(ctx, vpu_addr, size); |
| 381 | − ivpu_mmu_context_flush_page_tables(ctx, vpu_addr, size); | |
| 382 | 394 | vpu_addr += size; |
| 383 | 395 | } |
| 384 | 396 | |
| 397 | + /* Ensure page table modifications are flushed from wc buffers to memory */ | |
| 398 | + wmb(); | |
| 385 | 399 | mutex_unlock(&ctx->lock); |
| 386 | 400 | |
| 387 | 401 | ret = ivpu_mmu_invalidate_tlb(vdev, ctx->id); |
| @@ -390,28 +404,34 @@ ivpu_mmu_context_unmap_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ct · 2 unmodified lines | ||
| 390 | 404 | } |
| 391 | 405 | |
| 392 | 406 | int |
| 393 | −ivpu_mmu_context_insert_node_locked(struct ivpu_mmu_context *ctx, | |
| 394 | − const struct ivpu_addr_range *range, | |
| 395 | − u64 size, struct drm_mm_node *node) | |
| 407 | +ivpu_mmu_context_insert_node(struct ivpu_mmu_context *ctx, const struct ivpu_addr_range *range, | |
| 408 | + u64 size, struct drm_mm_node *node) | |
| 396 | 409 | { |
| 397 | − lockdep_assert_held(&ctx->lock); | |
| 410 | + int ret; | |
| 411 | + | |
| 412 | + WARN_ON(!range); | |
| 398 | 413 | |
| 414 | + mutex_lock(&ctx->lock); | |
| 399 | 415 | if (!ivpu_disable_mmu_cont_pages && size >= IVPU_MMU_CONT_PAGES_SIZE) { |
| 400 | − if (!drm_mm_insert_node_in_range(&ctx->mm, node, size, IVPU_MMU_CONT_PAGES_SIZE, 0, | |
| 401 | − range->start, range->end, DRM_MM_INSERT_BEST)) | |
| 402 | − return 0; | |
| 416 | + ret = drm_mm_insert_node_in_range(&ctx->mm, node, size, IVPU_MMU_CONT_PAGES_SIZE, 0, | |
| 417 | + range->start, range->end, DRM_MM_INSERT_BEST); | |
| 418 | + if (!ret) | |
| 419 | + goto unlock; | |
| 403 | 420 | } |
| 404 | 421 | |
| 405 | − return drm_mm_insert_node_in_range(&ctx->mm, node, size, IVPU_MMU_PAGE_SIZE, 0, | |
| 406 | − range->start, range->end, DRM_MM_INSERT_BEST); | |
| 422 | + ret = drm_mm_insert_node_in_range(&ctx->mm, node, size, IVPU_MMU_PAGE_SIZE, 0, | |
| 423 | + range->start, range->end, DRM_MM_INSERT_BEST); | |
| 424 | +unlock: | |
| 425 | + mutex_unlock(&ctx->lock); | |
| 426 | + return ret; | |
| 407 | 427 | } |
| 408 | 428 | |
| 409 | 429 | void |
| 410 | −ivpu_mmu_context_remove_node_locked(struct ivpu_mmu_context *ctx, struct drm_mm_node *node) | |
| 430 | +ivpu_mmu_context_remove_node(struct ivpu_mmu_context *ctx, struct drm_mm_node *node) | |
| 411 | 431 | { |
| 412 | − lockdep_assert_held(&ctx->lock); | |
| 413 | − | |
| 432 | + mutex_lock(&ctx->lock); | |
| 414 | 433 | drm_mm_remove_node(node); |
| 434 | + mutex_unlock(&ctx->lock); | |
| 415 | 435 | } |
| 416 | 436 | |
| 417 | 437 | static int |
| @@ -421,7 +441,6 @@ ivpu_mmu_context_init(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, u3 · 3 unmodified lines | ||
| 421 | 441 | int ret; |
| 422 | 442 | |
| 423 | 443 | mutex_init(&ctx->lock); |
| 424 | − INIT_LIST_HEAD(&ctx->bo_list); | |
| 425 | 444 | |
| 426 | 445 | ret = ivpu_mmu_pgtable_init(vdev, &ctx->pgtable); |
| 427 | 446 | if (ret) { |
| @@ -23,10 +23,9 @@ struct ivpu_mmu_pgtable { · 22 unmodified lines | ||
| 23 | 23 | }; |
| 24 | 24 | |
| 25 | 25 | struct ivpu_mmu_context { |
| 26 | − struct mutex lock; /* protects: mm, pgtable, bo_list */ | |
| 26 | + struct mutex lock; /* Protects: mm, pgtable */ | |
| 27 | 27 | struct drm_mm mm; |
| 28 | 28 | struct ivpu_mmu_pgtable pgtable; |
| 29 | − struct list_head bo_list; | |
| 30 | 29 | u32 id; |
| 31 | 30 | }; |
| 32 | 31 | |
| @@ -39,11 +38,9 @@ int ivpu_mmu_user_context_init(struct ivpu_device *vdev, struct ivpu_mmu_context · 6 unmodified lines | ||
| 39 | 38 | void ivpu_mmu_user_context_fini(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx); |
| 40 | 39 | void ivpu_mmu_user_context_mark_invalid(struct ivpu_device *vdev, u32 ssid); |
| 41 | 40 | |
| 42 | −int ivpu_mmu_context_insert_node_locked(struct ivpu_mmu_context *ctx, | |
| 43 | − const struct ivpu_addr_range *range, | |
| 44 | − u64 size, struct drm_mm_node *node); | |
| 45 | −void ivpu_mmu_context_remove_node_locked(struct ivpu_mmu_context *ctx, | |
| 46 | − struct drm_mm_node *node); | |
| 41 | +int ivpu_mmu_context_insert_node(struct ivpu_mmu_context *ctx, const struct ivpu_addr_range *range, | |
| 42 | + u64 size, struct drm_mm_node *node); | |
| 43 | +void ivpu_mmu_context_remove_node(struct ivpu_mmu_context *ctx, struct drm_mm_node *node); | |
| 47 | 44 | |
| 48 | 45 | int ivpu_mmu_context_map_sgt(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, |
| 49 | 46 | u64 vpu_addr, struct sg_table *sgt, bool llc_coherent); |
| @@ -15,6 +15,7 @@ · 14 unmodified lines | ||
| 15 | 15 | #include "ivpu_fw.h" |
| 16 | 16 | #include "ivpu_ipc.h" |
| 17 | 17 | #include "ivpu_job.h" |
| 18 | +#include "ivpu_jsm_msg.h" | |
| 18 | 19 | #include "ivpu_mmu.h" |
| 19 | 20 | #include "ivpu_pm.h" |
| 20 | 21 | |
| @@ -22,6 +23,10 @@ static bool ivpu_disable_recovery; · 1 unmodified line | ||
| 22 | 23 | module_param_named_unsafe(disable_recovery, ivpu_disable_recovery, bool, 0644); |
| 23 | 24 | MODULE_PARM_DESC(disable_recovery, "Disables recovery when VPU hang is detected"); |
| 24 | 25 | |
| 26 | +static unsigned long ivpu_tdr_timeout_ms; | |
| 27 | +module_param_named(tdr_timeout_ms, ivpu_tdr_timeout_ms, ulong, 0644); | |
| 28 | +MODULE_PARM_DESC(tdr_timeout_ms, "Timeout for device hang detection, in milliseconds, 0 - default"); | |
| 29 | + | |
| 25 | 30 | #define PM_RESCHEDULE_LIMIT 5 |
| 26 | 31 | |
| 27 | 32 | static void ivpu_pm_prepare_cold_boot(struct ivpu_device *vdev) |
| @@ -69,27 +74,31 @@ retry: · 41 unmodified lines | ||
| 69 | 74 | ret = ivpu_hw_power_up(vdev); |
| 70 | 75 | if (ret) { |
| 71 | 76 | ivpu_err(vdev, "Failed to power up HW: %d\n", ret); |
| 72 | − return ret; | |
| 77 | + goto err_power_down; | |
| 73 | 78 | } |
| 74 | 79 | |
| 75 | 80 | ret = ivpu_mmu_enable(vdev); |
| 76 | 81 | if (ret) { |
| 77 | 82 | ivpu_err(vdev, "Failed to resume MMU: %d\n", ret); |
| 78 | − ivpu_hw_power_down(vdev); | |
| 79 | − return ret; | |
| 83 | + goto err_power_down; | |
| 80 | 84 | } |
| 81 | 85 | |
| 82 | 86 | ret = ivpu_boot(vdev); |
| 83 | − if (ret) { | |
| 84 | − ivpu_mmu_disable(vdev); | |
| 85 | − ivpu_hw_power_down(vdev); | |
| 86 | − if (!ivpu_fw_is_cold_boot(vdev)) { | |
| 87 | − ivpu_warn(vdev, "Failed to resume the FW: %d. Retrying cold boot..\n", ret); | |
| 88 | − ivpu_pm_prepare_cold_boot(vdev); | |
| 89 | − goto retry; | |
| 90 | − } else { | |
| 91 | − ivpu_err(vdev, "Failed to resume the FW: %d\n", ret); | |
| 92 | − } | |
| 87 | + if (ret) | |
| 88 | + goto err_mmu_disable; | |
| 89 | + | |
| 90 | + return 0; | |
| 91 | + | |
| 92 | +err_mmu_disable: | |
| 93 | + ivpu_mmu_disable(vdev); | |
| 94 | +err_power_down: | |
| 95 | + ivpu_hw_power_down(vdev); | |
| 96 | + | |
| 97 | + if (!ivpu_fw_is_cold_boot(vdev)) { | |
| 98 | + ivpu_pm_prepare_cold_boot(vdev); | |
| 99 | + goto retry; | |
| 100 | + } else { | |
| 101 | + ivpu_err(vdev, "Failed to resume the FW: %d\n", ret); | |
| 93 | 102 | } |
| 94 | 103 | |
| 95 | 104 | return ret; |
| @@ -136,6 +145,31 @@ void ivpu_pm_schedule_recovery(struct ivpu_device *vdev) · 40 unmodified lines | ||
| 136 | 145 | } |
| 137 | 146 | } |
| 138 | 147 | |
| 148 | +static void ivpu_job_timeout_work(struct work_struct *work) | |
| 149 | +{ | |
| 150 | + struct ivpu_pm_info *pm = container_of(work, struct ivpu_pm_info, job_timeout_work.work); | |
| 151 | + struct ivpu_device *vdev = pm->vdev; | |
| 152 | + unsigned long timeout_ms = ivpu_tdr_timeout_ms ? ivpu_tdr_timeout_ms : vdev->timeout.tdr; | |
| 153 | + | |
| 154 | + ivpu_err(vdev, "TDR detected, timeout %lu ms", timeout_ms); | |
| 155 | + ivpu_hw_diagnose_failure(vdev); | |
| 156 | + | |
| 157 | + ivpu_pm_schedule_recovery(vdev); | |
| 158 | +} | |
| 159 | + | |
| 160 | +void ivpu_start_job_timeout_detection(struct ivpu_device *vdev) | |
| 161 | +{ | |
| 162 | + unsigned long timeout_ms = ivpu_tdr_timeout_ms ? ivpu_tdr_timeout_ms : vdev->timeout.tdr; | |
| 163 | + | |
| 164 | + /* No-op if already queued */ | |
| 165 | + queue_delayed_work(system_wq, &vdev->pm->job_timeout_work, msecs_to_jiffies(timeout_ms)); | |
| 166 | +} | |
| 167 | + | |
| 168 | +void ivpu_stop_job_timeout_detection(struct ivpu_device *vdev) | |
| 169 | +{ | |
| 170 | + cancel_delayed_work_sync(&vdev->pm->job_timeout_work); | |
| 171 | +} | |
| 172 | + | |
| 139 | 173 | int ivpu_pm_suspend_cb(struct device *dev) |
| 140 | 174 | { |
| 141 | 175 | struct drm_device *drm = dev_get_drvdata(dev); |
| @@ -153,6 +187,8 @@ int ivpu_pm_suspend_cb(struct device *dev) · 11 unmodified lines | ||
| 153 | 187 | } |
| 154 | 188 | } |
| 155 | 189 | |
| 190 | + ivpu_jsm_pwr_d0i3_enter(vdev); | |
| 191 | + | |
| 156 | 192 | ivpu_suspend(vdev); |
| 157 | 193 | ivpu_pm_prepare_warm_boot(vdev); |
| 158 | 194 | |
| @@ -188,6 +224,7 @@ int ivpu_pm_runtime_suspend_cb(struct device *dev) · 29 unmodified lines | ||
| 188 | 224 | { |
| 189 | 225 | struct drm_device *drm = dev_get_drvdata(dev); |
| 190 | 226 | struct ivpu_device *vdev = to_ivpu_device(drm); |
| 227 | + bool hw_is_idle = true; | |
| 191 | 228 | int ret; |
| 192 | 229 | |
| 193 | 230 | ivpu_dbg(vdev, PM, "Runtime suspend..\n"); |
| @@ -200,11 +237,16 @@ int ivpu_pm_runtime_suspend_cb(struct device *dev) · 6 unmodified lines | ||
| 200 | 237 | return -EAGAIN; |
| 201 | 238 | } |
| 202 | 239 | |
| 240 | + if (!vdev->pm->suspend_reschedule_counter) | |
| 241 | + hw_is_idle = false; | |
| 242 | + else if (ivpu_jsm_pwr_d0i3_enter(vdev)) | |
| 243 | + hw_is_idle = false; | |
| 244 | + | |
| 203 | 245 | ret = ivpu_suspend(vdev); |
| 204 | 246 | if (ret) |
| 205 | 247 | ivpu_err(vdev, "Failed to set suspend VPU: %d\n", ret); |
| 206 | 248 | |
| 207 | − if (!vdev->pm->suspend_reschedule_counter) { | |
| 249 | + if (!hw_is_idle) { | |
| 208 | 250 | ivpu_warn(vdev, "VPU failed to enter idle, force suspended.\n"); |
| 209 | 251 | ivpu_pm_prepare_cold_boot(vdev); |
| 210 | 252 | } else { |
| @@ -250,9 +292,6 @@ int ivpu_rpm_get_if_active(struct ivpu_device *vdev) · 39 unmodified lines | ||
| 250 | 292 | { |
| 251 | 293 | int ret; |
| 252 | 294 | |
| 253 | − ivpu_dbg(vdev, RPM, "rpm_get_if_active count %d\n", | |
| 254 | − atomic_read(&vdev->drm.dev->power.usage_count)); | |
| 255 | − | |
| 256 | 295 | ret = pm_runtime_get_if_active(vdev->drm.dev, false); |
| 257 | 296 | drm_WARN_ON(&vdev->drm, ret < 0); |
| 258 | 297 | |
| @@ -307,6 +346,7 @@ void ivpu_pm_init(struct ivpu_device *vdev) · 48 unmodified lines | ||
| 307 | 346 | |
| 308 | 347 | atomic_set(&pm->in_reset, 0); |
| 309 | 348 | INIT_WORK(&pm->recovery_work, ivpu_pm_recovery_work); |
| 349 | + INIT_DELAYED_WORK(&pm->job_timeout_work, ivpu_job_timeout_work); | |
| 310 | 350 | |
| 311 | 351 | if (ivpu_disable_recovery) |
| 312 | 352 | delay = -1; |
| @@ -321,6 +361,7 @@ void ivpu_pm_init(struct ivpu_device *vdev) · 8 unmodified lines | ||
| 321 | 361 | |
| 322 | 362 | void ivpu_pm_cancel_recovery(struct ivpu_device *vdev) |
| 323 | 363 | { |
| 364 | + drm_WARN_ON(&vdev->drm, delayed_work_pending(&vdev->pm->job_timeout_work)); | |
| 324 | 365 | cancel_work_sync(&vdev->pm->recovery_work); |
| 325 | 366 | } |
| 326 | 367 | |
| @@ -12,6 +12,7 @@ struct ivpu_device; · 11 unmodified lines | ||
| 12 | 12 | |
| 13 | 13 | struct ivpu_pm_info { |
| 14 | 14 | struct ivpu_device *vdev; |
| 15 | + struct delayed_work job_timeout_work; | |
| 15 | 16 | struct work_struct recovery_work; |
| 16 | 17 | atomic_t in_reset; |
| 17 | 18 | atomic_t reset_counter; |
| @@ -37,5 +38,7 @@ int __must_check ivpu_rpm_get_if_active(struct ivpu_device *vdev); · 19 unmodified lines | ||
| 37 | 38 | void ivpu_rpm_put(struct ivpu_device *vdev); |
| 38 | 39 | |
| 39 | 40 | void ivpu_pm_schedule_recovery(struct ivpu_device *vdev); |
| 41 | +void ivpu_start_job_timeout_detection(struct ivpu_device *vdev); | |
| 42 | +void ivpu_stop_job_timeout_detection(struct ivpu_device *vdev); | |
| 40 | 43 | |
| 41 | 44 | #endif /* __IVPU_PM_H__ */ |
| @@ -11,7 +11,10 @@ · 10 unmodified lines | ||
| 11 | 11 | * The bellow values will be used to construct the version info this way: |
| 12 | 12 | * fw_bin_header->api_version[VPU_BOOT_API_VER_ID] = (VPU_BOOT_API_VER_MAJOR << 16) | |
| 13 | 13 | * VPU_BOOT_API_VER_MINOR; |
| 14 | − * VPU_BOOT_API_VER_PATCH will be ignored. KMD and compatibility is not affected if this changes. | |
| 14 | + * VPU_BOOT_API_VER_PATCH will be ignored. KMD and compatibility is not affected if this changes | |
| 15 | + * This information is collected by using vpuip_2/application/vpuFirmware/make_std_fw_image.py | |
| 16 | + * If a header is missing this info we ignore the header, if a header is missing or contains | |
| 17 | + * partial info a build error will be generated. | |
| 15 | 18 | */ |
| 16 | 19 | |
| 17 | 20 | /* |
| @@ -24,12 +27,12 @@ · 6 unmodified lines | ||
| 24 | 27 | * Minor version changes when API backward compatibility is preserved. |
| 25 | 28 | * Resets to 0 if Major version is incremented. |
| 26 | 29 | */ |
| 27 | −#define VPU_BOOT_API_VER_MINOR 12 | |
| 30 | +#define VPU_BOOT_API_VER_MINOR 20 | |
| 28 | 31 | |
| 29 | 32 | /* |
| 30 | 33 | * API header changed (field names, documentation, formatting) but API itself has not been changed |
| 31 | 34 | */ |
| 32 | −#define VPU_BOOT_API_VER_PATCH 2 | |
| 35 | +#define VPU_BOOT_API_VER_PATCH 4 | |
| 33 | 36 | |
| 34 | 37 | /* |
| 35 | 38 | * Index in the API version table |
| @@ -63,6 +66,12 @@ struct vpu_firmware_header { · 27 unmodified lines | ||
| 63 | 66 | /* Size of memory require for firmware execution */ |
| 64 | 67 | u32 runtime_size; |
| 65 | 68 | u32 shave_nn_fw_size; |
| 69 | + /* Size of primary preemption buffer. */ | |
| 70 | + u32 preemption_buffer_1_size; | |
| 71 | + /* Size of secondary preemption buffer. */ | |
| 72 | + u32 preemption_buffer_2_size; | |
| 73 | + /* Space reserved for future preemption-related fields. */ | |
| 74 | + u32 preemption_reserved[6]; | |
| 66 | 75 | }; |
| 67 | 76 | |
| 68 | 77 | /* |
| @@ -89,6 +98,14 @@ enum VPU_BOOT_L2_CACHE_CFG_TYPE { · 20 unmodified lines | ||
| 89 | 98 | VPU_BOOT_L2_CACHE_CFG_NUM = 2 |
| 90 | 99 | }; |
| 91 | 100 | |
| 101 | +/** VPU MCA ECC signalling mode. By default, no signalling is used */ | |
| 102 | +enum VPU_BOOT_MCA_ECC_SIGNAL_TYPE { | |
| 103 | + VPU_BOOT_MCA_ECC_NONE = 0, | |
| 104 | + VPU_BOOT_MCA_ECC_CORR = 1, | |
| 105 | + VPU_BOOT_MCA_ECC_FATAL = 2, | |
| 106 | + VPU_BOOT_MCA_ECC_BOTH = 3 | |
| 107 | +}; | |
| 108 | + | |
| 92 | 109 | /** |
| 93 | 110 | * Logging destinations. |
| 94 | 111 | * |
| @@ -131,9 +148,11 @@ enum vpu_trace_destination { · 36 unmodified lines | ||
| 131 | 148 | #define VPU_TRACE_PROC_BIT_ACT_SHV_3 22 |
| 132 | 149 | #define VPU_TRACE_PROC_NO_OF_HW_DEVS 23 |
| 133 | 150 | |
| 134 | −/* KMB HW component IDs are sequential, so define first and last IDs. */ | |
| 135 | −#define VPU_TRACE_PROC_BIT_KMB_FIRST VPU_TRACE_PROC_BIT_LRT | |
| 136 | −#define VPU_TRACE_PROC_BIT_KMB_LAST VPU_TRACE_PROC_BIT_SHV_15 | |
| 151 | +/* VPU 30xx HW component IDs are sequential, so define first and last IDs. */ | |
| 152 | +#define VPU_TRACE_PROC_BIT_30XX_FIRST VPU_TRACE_PROC_BIT_LRT | |
| 153 | +#define VPU_TRACE_PROC_BIT_30XX_LAST VPU_TRACE_PROC_BIT_SHV_15 | |
| 154 | +#define VPU_TRACE_PROC_BIT_KMB_FIRST VPU_TRACE_PROC_BIT_30XX_FIRST | |
| 155 | +#define VPU_TRACE_PROC_BIT_KMB_LAST VPU_TRACE_PROC_BIT_30XX_LAST | |
| 137 | 156 | |
| 138 | 157 | struct vpu_boot_l2_cache_config { |
| 139 | 158 | u8 use; |
| @@ -148,6 +167,25 @@ struct vpu_warm_boot_section { · 8 unmodified lines | ||
| 148 | 167 | u32 is_clear_op; |
| 149 | 168 | }; |
| 150 | 169 | |
| 170 | +/* | |
| 171 | + * When HW scheduling mode is enabled, a present period is defined. | |
| 172 | + * It will be used by VPU to swap between normal and focus priorities | |
| 173 | + * to prevent starving of normal priority band (when implemented). | |
| 174 | + * Host must provide a valid value at boot time in | |
| 175 | + * `vpu_focus_present_timer_ms`. If the value provided by the host is not within the | |
| 176 | + * defined range a default value will be used. Here we define the min. and max. | |
| 177 | + * allowed values and the and default value of the present period. Units are milliseconds. | |
| 178 | + */ | |
| 179 | +#define VPU_PRESENT_CALL_PERIOD_MS_DEFAULT 50 | |
| 180 | +#define VPU_PRESENT_CALL_PERIOD_MS_MIN 16 | |
| 181 | +#define VPU_PRESENT_CALL_PERIOD_MS_MAX 10000 | |
| 182 | + | |
| 183 | +/** | |
| 184 | + * Macros to enable various operation modes within the VPU. | |
| 185 | + * To be defined as part of 32 bit mask. | |
| 186 | + */ | |
| 187 | +#define VPU_OP_MODE_SURVIVABILITY 0x1 | |
| 188 | + | |
| 151 | 189 | struct vpu_boot_params { |
| 152 | 190 | u32 magic; |
| 153 | 191 | u32 vpu_id; |
| @@ -218,6 +256,7 @@ struct vpu_boot_params { · 64 unmodified lines | ||
| 218 | 256 | * the threshold will not be logged); applies to every enabled logging |
| 219 | 257 | * destination and loggable HW component. See 'mvLog_t' enum for acceptable |
| 220 | 258 | * values. |
| 259 | + * TODO: EISW-33556: Move log level definition (mvLog_t) to this file. | |
| 221 | 260 | */ |
| 222 | 261 | u32 default_trace_level; |
| 223 | 262 | u32 boot_type; |
| @@ -249,7 +288,36 @@ struct vpu_boot_params { · 25 unmodified lines | ||
| 249 | 288 | u32 temp_sensor_period_ms; |
| 250 | 289 | /** PLL ratio for efficient clock frequency */ |
| 251 | 290 | u32 pn_freq_pll_ratio; |
| 252 | − u32 pad4[28]; | |
| 291 | + /** DVFS Mode: Default: 0, Max Performance: 1, On Demand: 2, Power Save: 3 */ | |
| 292 | + u32 dvfs_mode; | |
| 293 | + /** | |
| 294 | + * Depending on DVFS Mode: | |
| 295 | + * On-demand: Default if 0. | |
| 296 | + * Bit 0-7 - uint8_t: Highest residency percent | |
| 297 | + * Bit 8-15 - uint8_t: High residency percent | |
| 298 | + * Bit 16-23 - uint8_t: Low residency percent | |
| 299 | + * Bit 24-31 - uint8_t: Lowest residency percent | |
| 300 | + * Bit 32-35 - unsigned 4b: PLL Ratio increase amount on highest residency | |
| 301 | + * Bit 36-39 - unsigned 4b: PLL Ratio increase amount on high residency | |
| 302 | + * Bit 40-43 - unsigned 4b: PLL Ratio decrease amount on low residency | |
| 303 | + * Bit 44-47 - unsigned 4b: PLL Ratio decrease amount on lowest frequency | |
| 304 | + * Bit 48-55 - uint8_t: Period (ms) for residency decisions | |
| 305 | + * Bit 56-63 - uint8_t: Averaging windows (as multiples of period. Max: 30 decimal) | |
| 306 | + * Power Save/Max Performance: Unused | |
| 307 | + */ | |
| 308 | + u64 dvfs_param; | |
| 309 | + /** | |
| 310 | + * D0i3 delayed entry | |
| 311 | + * Bit0: Disable CPU state save on D0i2 entry flow. | |
| 312 | + * 0: Every D0i2 entry saves state. Save state IPC message ignored. | |
| 313 | + * 1: IPC message required to save state on D0i3 entry flow. | |
| 314 | + */ | |
| 315 | + u32 d0i3_delayed_entry; | |
| 316 | + /* Time spent by VPU in D0i3 state */ | |
| 317 | + u64 d0i3_residency_time_us; | |
| 318 | + /* Value of VPU perf counter at the time of entering D0i3 state . */ | |
| 319 | + u64 d0i3_entry_vpu_ts; | |
| 320 | + u32 pad4[20]; | |
| 253 | 321 | /* Warm boot information: 0x400 - 0x43F */ |
| 254 | 322 | u32 warm_boot_sections_count; |
| 255 | 323 | u32 warm_boot_start_address_reference; |
| @@ -274,8 +342,12 @@ struct vpu_boot_params { · 18 unmodified lines | ||
| 274 | 342 | u32 vpu_scheduling_mode; |
| 275 | 343 | /* Present call period in milliseconds. */ |
| 276 | 344 | u32 vpu_focus_present_timer_ms; |
| 277 | − /* Unused/reserved: 0x478 - 0xFFF */ | |
| 278 | − u32 pad6[738]; | |
| 345 | + /* VPU ECC Signaling */ | |
| 346 | + u32 vpu_uses_ecc_mca_signal; | |
| 347 | + /* Values defined by VPU_OP_MODE* macros */ | |
| 348 | + u32 vpu_operation_mode; | |
| 349 | + /* Unused/reserved: 0x480 - 0xFFF */ | |
| 350 | + u32 pad6[736]; | |
| 279 | 351 | }; |
| 280 | 352 | |
| 281 | 353 | /* |
| @@ -22,12 +22,12 @@ · 21 unmodified lines | ||
| 22 | 22 | /* |
| 23 | 23 | * Minor version changes when API backward compatibility is preserved. |
| 24 | 24 | */ |
| 25 | −#define VPU_JSM_API_VER_MINOR 0 | |
| 25 | +#define VPU_JSM_API_VER_MINOR 15 | |
| 26 | 26 | |
| 27 | 27 | /* |
| 28 | 28 | * API header changed (field names, documentation, formatting) but API itself has not been changed |
| 29 | 29 | */ |
| 30 | −#define VPU_JSM_API_VER_PATCH 1 | |
| 30 | +#define VPU_JSM_API_VER_PATCH 0 | |
| 31 | 31 | |
| 32 | 32 | /* |
| 33 | 33 | * Index in the API version table |
| @@ -84,11 +84,13 @@ · 50 unmodified lines | ||
| 84 | 84 | * Job flags bit masks. |
| 85 | 85 | */ |
| 86 | 86 | #define VPU_JOB_FLAGS_NULL_SUBMISSION_MASK 0x00000001 |
| 87 | +#define VPU_JOB_FLAGS_PRIVATE_DATA_MASK 0xFF000000 | |
| 87 | 88 | |
| 88 | 89 | /* |
| 89 | 90 | * Sizes of the reserved areas in jobs, in bytes. |
| 90 | 91 | */ |
| 91 | −#define VPU_JOB_RESERVED_BYTES 16 | |
| 92 | +#define VPU_JOB_RESERVED_BYTES 8 | |
| 93 | + | |
| 92 | 94 | /* |
| 93 | 95 | * Sizes of the reserved areas in job queues, in bytes. |
| 94 | 96 | */ |
| @@ -108,6 +110,20 @@ · 13 unmodified lines | ||
| 108 | 110 | */ |
| 109 | 111 | #define VPU_DYNDBG_CMD_MAX_LEN 96 |
| 110 | 112 | |
| 113 | +/* | |
| 114 | + * For HWS command queue scheduling, we can prioritise command queues inside the | |
| 115 | + * same process with a relative in-process priority. Valid values for relative | |
| 116 | + * priority are given below - max and min. | |
| 117 | + */ | |
| 118 | +#define VPU_HWS_COMMAND_QUEUE_MAX_IN_PROCESS_PRIORITY 7 | |
| 119 | +#define VPU_HWS_COMMAND_QUEUE_MIN_IN_PROCESS_PRIORITY -7 | |
| 120 | + | |
| 121 | +/* | |
| 122 | + * For HWS priority scheduling, we can have multiple realtime priority bands. | |
| 123 | + * They are numbered 0 to a MAX. | |
| 124 | + */ | |
| 125 | +#define VPU_HWS_MAX_REALTIME_PRIORITY_LEVEL 31U | |
| 126 | + | |
| 111 | 127 | /* |
| 112 | 128 | * Job format. |
| 113 | 129 | */ |
| @@ -117,8 +133,14 @@ struct vpu_job_queue_entry { · 3 unmodified lines | ||
| 117 | 133 | u32 flags; /**< Flags bit field, see VPU_JOB_FLAGS_* above */ |
| 118 | 134 | u64 root_page_table_addr; /**< Address of root page table to use for this job */ |
| 119 | 135 | u64 root_page_table_update_counter; /**< Page tables update events counter */ |
| 120 | − u64 preemption_buffer_address; /**< Address of the preemption buffer to use for this job */ | |
| 121 | − u64 preemption_buffer_size; /**< Size of the preemption buffer to use for this job */ | |
| 136 | + u64 primary_preempt_buf_addr; | |
| 137 | + /**< Address of the primary preemption buffer to use for this job */ | |
| 138 | + u32 primary_preempt_buf_size; | |
| 139 | + /**< Size of the primary preemption buffer to use for this job */ | |
| 140 | + u32 secondary_preempt_buf_size; | |
| 141 | + /**< Size of secondary preemption buffer to use for this job */ | |
| 142 | + u64 secondary_preempt_buf_addr; | |
| 143 | + /**< Address of secondary preemption buffer to use for this job */ | |
| 122 | 144 | u8 reserved_0[VPU_JOB_RESERVED_BYTES]; |
| 123 | 145 | }; |
| 124 | 146 | |
| @@ -152,6 +174,46 @@ enum vpu_trace_entity_type { · 27 unmodified lines | ||
| 152 | 174 | VPU_TRACE_ENTITY_TYPE_HW_COMPONENT = 2, |
| 153 | 175 | }; |
| 154 | 176 | |
| 177 | +/* | |
| 178 | + * HWS specific log buffer header details. | |
| 179 | + * Total size is 32 bytes. | |
| 180 | + */ | |
| 181 | +struct vpu_hws_log_buffer_header { | |
| 182 | + /* Written by VPU after adding a log entry. Initialised by host to 0. */ | |
| 183 | + u32 first_free_entry_index; | |
| 184 | + /* Incremented by VPU every time the VPU overwrites the 0th entry; | |
| 185 | + * initialised by host to 0. | |
| 186 | + */ | |
| 187 | + u32 wraparound_count; | |
| 188 | + /* | |
| 189 | + * This is the number of buffers that can be stored in the log buffer provided by the host. | |
| 190 | + * It is written by host before passing buffer to VPU. VPU should consider it read-only. | |
| 191 | + */ | |
| 192 | + u64 num_of_entries; | |
| 193 | + u64 reserved[2]; | |
| 194 | +}; | |
| 195 | + | |
| 196 | +/* | |
| 197 | + * HWS specific log buffer entry details. | |
| 198 | + * Total size is 32 bytes. | |
| 199 | + */ | |
| 200 | +struct vpu_hws_log_buffer_entry { | |
| 201 | + /* VPU timestamp must be an invariant timer tick (not impacted by DVFS) */ | |
| 202 | + u64 vpu_timestamp; | |
| 203 | + /* | |
| 204 | + * Operation type: | |
| 205 | + * 0 - context state change | |
| 206 | + * 1 - queue new work | |
| 207 | + * 2 - queue unwait sync object | |
| 208 | + * 3 - queue no more work | |
| 209 | + * 4 - queue wait sync object | |
| 210 | + */ | |
| 211 | + u32 operation_type; | |
| 212 | + u32 reserved; | |
| 213 | + /* Operation data depends on operation type */ | |
| 214 | + u64 operation_data[2]; | |
| 215 | +}; | |
| 216 | + | |
| 155 | 217 | /* |
| 156 | 218 | * Host <-> VPU IPC messages types. |
| 157 | 219 | */ |
| @@ -228,6 +290,23 @@ enum vpu_ipc_msg_type { · 70 unmodified lines | ||
| 228 | 290 | * deallocated or reassigned to another context. |
| 229 | 291 | */ |
| 230 | 292 | VPU_JSM_MSG_HWS_REGISTER_DB = 0x1117, |
| 293 | + /** Control command: Log buffer setting */ | |
| 294 | + VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG = 0x1118, | |
| 295 | + /* Control command: Suspend command queue. */ | |
| 296 | + VPU_JSM_MSG_HWS_SUSPEND_CMDQ = 0x1119, | |
| 297 | + /* Control command: Resume command queue */ | |
| 298 | + VPU_JSM_MSG_HWS_RESUME_CMDQ = 0x111a, | |
| 299 | + /* Control command: Resume engine after reset */ | |
| 300 | + VPU_JSM_MSG_HWS_ENGINE_RESUME = 0x111b, | |
| 301 | + /* Control command: Enable survivability/DCT mode */ | |
| 302 | + VPU_JSM_MSG_DCT_ENABLE = 0x111c, | |
| 303 | + /* Control command: Disable survivability/DCT mode */ | |
| 304 | + VPU_JSM_MSG_DCT_DISABLE = 0x111d, | |
| 305 | + /** | |
| 306 | + * Dump VPU state. To be used for debug purposes only. | |
| 307 | + * NOTE: Please introduce new ASYNC commands before this one. * | |
| 308 | + */ | |
| 309 | + VPU_JSM_MSG_STATE_DUMP = 0x11FF, | |
| 231 | 310 | /* IPC Host -> Device, General commands */ |
| 232 | 311 | VPU_JSM_MSG_GENERAL_CMD = 0x1200, |
| 233 | 312 | VPU_JSM_MSG_BLOB_DEINIT = VPU_JSM_MSG_GENERAL_CMD, |
| @@ -236,6 +315,10 @@ enum vpu_ipc_msg_type { · 2 unmodified lines | ||
| 236 | 315 | * Linux command: `echo '<dyndbg_cmd>' > <debugfs>/dynamic_debug/control`. |
| 237 | 316 | */ |
| 238 | 317 | VPU_JSM_MSG_DYNDBG_CONTROL = 0x1201, |
| 318 | + /** | |
| 319 | + * Perform the save procedure for the D0i3 entry | |
| 320 | + */ | |
| 321 | + VPU_JSM_MSG_PWR_D0I3_ENTER = 0x1202, | |
| 239 | 322 | /* IPC Device -> Host, Job completion */ |
| 240 | 323 | VPU_JSM_MSG_JOB_DONE = 0x2100, |
| 241 | 324 | /* IPC Device -> Host, Async command completion */ |
| @@ -304,11 +387,35 @@ enum vpu_ipc_msg_type { · 62 unmodified lines | ||
| 304 | 387 | VPU_JSM_MSG_DESTROY_CMD_QUEUE_RSP = 0x2216, |
| 305 | 388 | /** Response to control command: Set context scheduling properties */ |
| 306 | 389 | VPU_JSM_MSG_SET_CONTEXT_SCHED_PROPERTIES_RSP = 0x2217, |
| 390 | + /** Response to control command: Log buffer setting */ | |
| 391 | + VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG_RSP = 0x2218, | |
| 392 | + /* IPC Device -> Host, HWS notify index entry of log buffer written */ | |
| 393 | + VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION = 0x2219, | |
| 394 | + /* IPC Device -> Host, HWS completion of a context suspend request */ | |
| 395 | + VPU_JSM_MSG_HWS_SUSPEND_CMDQ_DONE = 0x221a, | |
| 396 | + /* Response to control command: Resume command queue */ | |
| 397 | + VPU_JSM_MSG_HWS_RESUME_CMDQ_RSP = 0x221b, | |
| 398 | + /* Response to control command: Resume engine command response */ | |
| 399 | + VPU_JSM_MSG_HWS_RESUME_ENGINE_DONE = 0x221c, | |
| 400 | + /* Response to control command: Enable survivability/DCT mode */ | |
| 401 | + VPU_JSM_MSG_DCT_ENABLE_DONE = 0x221d, | |
| 402 | + /* Response to control command: Disable survivability/DCT mode */ | |
| 403 | + VPU_JSM_MSG_DCT_DISABLE_DONE = 0x221e, | |
| 404 | + /** | |
| 405 | + * Response to state dump control command. | |
| 406 | + * NOTE: Please introduce new ASYNC responses before this one. * | |
| 407 | + */ | |
| 408 | + VPU_JSM_MSG_STATE_DUMP_RSP = 0x22FF, | |
| 307 | 409 | /* IPC Device -> Host, General command completion */ |
| 308 | 410 | VPU_JSM_MSG_GENERAL_CMD_DONE = 0x2300, |
| 309 | 411 | VPU_JSM_MSG_BLOB_DEINIT_DONE = VPU_JSM_MSG_GENERAL_CMD_DONE, |
| 310 | 412 | /** Response to VPU_JSM_MSG_DYNDBG_CONTROL. */ |
| 311 | 413 | VPU_JSM_MSG_DYNDBG_CONTROL_RSP = 0x2301, |
| 414 | + /** | |
| 415 | + * Acknowledgment of completion of the save procedure initiated by | |
| 416 | + * VPU_JSM_MSG_PWR_D0I3_ENTER | |
| 417 | + */ | |
| 418 | + VPU_JSM_MSG_PWR_D0I3_ENTER_DONE = 0x2302, | |
| 312 | 419 | }; |
| 313 | 420 | |
| 314 | 421 | enum vpu_ipc_msg_status { VPU_JSM_MSG_FREE, VPU_JSM_MSG_ALLOCATED }; |
| @@ -593,12 +700,12 @@ struct vpu_ipc_msg_payload_hws_priority_band_setup { · 278 unmodified lines | ||
| 593 | 700 | * Default quantum in 100ns units for scheduling across processes |
| 594 | 701 | * within a priority band |
| 595 | 702 | */ |
| 596 | − u64 process_quantum[VPU_HWS_NUM_PRIORITY_BANDS]; | |
| 703 | + u32 process_quantum[VPU_HWS_NUM_PRIORITY_BANDS]; | |
| 597 | 704 | /* |
| 598 | 705 | * Default grace period in 100ns units for processes that preempt each |
| 599 | 706 | * other within a priority band |
| 600 | 707 | */ |
| 601 | − u64 process_grace_period[VPU_HWS_NUM_PRIORITY_BANDS]; | |
| 708 | + u32 process_grace_period[VPU_HWS_NUM_PRIORITY_BANDS]; | |
| 602 | 709 | /* |
| 603 | 710 | * For normal priority band, specifies the target VPU percentage |
| 604 | 711 | * in situations when it's starved by the focus band. |
| @@ -608,32 +715,51 @@ struct vpu_ipc_msg_payload_hws_priority_band_setup { · 3 unmodified lines | ||
| 608 | 715 | u32 reserved_0; |
| 609 | 716 | }; |
| 610 | 717 | |
| 611 | −/* HWS create command queue request */ | |
| 718 | +/* | |
| 719 | + * @brief HWS create command queue request. | |
| 720 | + * Host will create a command queue via this command. | |
| 721 | + * Note: Cmdq group is a handle of an object which | |
| 722 | + * may contain one or more command queues. | |
| 723 | + * @see VPU_JSM_MSG_CREATE_CMD_QUEUE | |
| 724 | + * @see VPU_JSM_MSG_CREATE_CMD_QUEUE_RSP | |
| 725 | + */ | |
| 612 | 726 | struct vpu_ipc_msg_payload_hws_create_cmdq { |
| 613 | 727 | /* Process id */ |
| 614 | 728 | u64 process_id; |
| 615 | 729 | /* Host SSID */ |
| 616 | 730 | u32 host_ssid; |
| 617 | − /* Zero Padding */ | |
| 618 | − u32 reserved; | |
| 731 | + /* Engine for which queue is being created */ | |
| 732 | + u32 engine_idx; | |
| 733 | + /* | |
| 734 | + * Cmdq group may be set to 0 or equal to | |
| 735 | + * cmdq_id while each priority band contains | |
| 736 | + * only single engine instances. | |
| 737 | + */ | |
| 738 | + u64 cmdq_group; | |
| 619 | 739 | /* Command queue id */ |
| 620 | 740 | u64 cmdq_id; |
| 621 | 741 | /* Command queue base */ |
| 622 | 742 | u64 cmdq_base; |
| 623 | 743 | /* Command queue size */ |
| 624 | 744 | u32 cmdq_size; |
| 625 | − /* Reserved */ | |
| 745 | + /* Zero padding */ | |
| 626 | 746 | u32 reserved_0; |
| 627 | 747 | }; |
| 628 | 748 | |
| 629 | −/* HWS create command queue response */ | |
| 749 | +/* | |
| 750 | + * @brief HWS create command queue response. | |
| 751 | + * @see VPU_JSM_MSG_CREATE_CMD_QUEUE | |
| 752 | + * @see VPU_JSM_MSG_CREATE_CMD_QUEUE_RSP | |
| 753 | + */ | |
| 630 | 754 | struct vpu_ipc_msg_payload_hws_create_cmdq_rsp { |
| 631 | 755 | /* Process id */ |
| 632 | 756 | u64 process_id; |
| 633 | 757 | /* Host SSID */ |
| 634 | 758 | u32 host_ssid; |
| 635 | − /* Zero Padding */ | |
| 636 | − u32 reserved; | |
| 759 | + /* Engine for which queue is being created */ | |
| 760 | + u32 engine_idx; | |
| 761 | + /* Command queue group */ | |
| 762 | + u64 cmdq_group; | |
| 637 | 763 | /* Command queue id */ |
| 638 | 764 | u64 cmdq_id; |
| 639 | 765 | }; |
| @@ -661,7 +787,7 @@ struct vpu_ipc_msg_payload_hws_set_context_sched_properties { · 21 unmodified lines | ||
| 661 | 787 | /* Inside realtime band assigns a further priority */ |
| 662 | 788 | u32 realtime_priority_level; |
| 663 | 789 | /* Priority relative to other contexts in the same process */ |
| 664 | − u32 in_process_priority; | |
| 790 | + s32 in_process_priority; | |
| 665 | 791 | /* Zero padding / Reserved */ |
| 666 | 792 | u32 reserved_1; |
| 667 | 793 | /* Context quantum relative to other contexts of same priority in the same process */ |
| @@ -694,6 +820,123 @@ struct vpu_jsm_hws_register_db { · 26 unmodified lines | ||
| 694 | 820 | u64 cmdq_size; |
| 695 | 821 | }; |
| 696 | 822 | |
| 823 | +/* | |
| 824 | + * @brief Structure to set another buffer to be used for scheduling-related logging. | |
| 825 | + * The size of the logging buffer and the number of entries is defined as part of the | |
| 826 | + * buffer itself as described next. | |
| 827 | + * The log buffer received from the host is made up of; | |
| 828 | + * - header: 32 bytes in size, as shown in 'struct vpu_hws_log_buffer_header'. | |
| 829 | + * The header contains the number of log entries in the buffer. | |
| 830 | + * - log entry: 0 to n-1, each log entry is 32 bytes in size, as shown in | |
| 831 | + * 'struct vpu_hws_log_buffer_entry'. | |
| 832 | + * The entry contains the VPU timestamp, operation type and data. | |
| 833 | + * The host should provide the notify index value of log buffer to VPU. This is a | |
| 834 | + * value defined within the log buffer and when written to will generate the | |
| 835 | + * scheduling log notification. | |
| 836 | + * The host should set engine_idx and vpu_log_buffer_va to 0 to disable logging | |
| 837 | + * for a particular engine. | |
| 838 | + * VPU will handle one log buffer for each of supported engines. | |
| 839 | + * VPU should allow the logging to consume one host_ssid. | |
| 840 | + * @see VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG | |
| 841 | + * @see VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG_RSP | |
| 842 | + * @see VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION | |
| 843 | + */ | |
| 844 | +struct vpu_ipc_msg_payload_hws_set_scheduling_log { | |
| 845 | + /* Engine ordinal */ | |
| 846 | + u32 engine_idx; | |
| 847 | + /* Host SSID */ | |
| 848 | + u32 host_ssid; | |
| 849 | + /* | |
| 850 | + * VPU log buffer virtual address. | |
| 851 | + * Set to 0 to disable logging for this engine. | |
| 852 | + */ | |
| 853 | + u64 vpu_log_buffer_va; | |
| 854 | + /* | |
| 855 | + * Notify index of log buffer. VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION | |
| 856 | + * is generated when an event log is written to this index. | |
| 857 | + */ | |
| 858 | + u64 notify_index; | |
| 859 | +}; | |
| 860 | + | |
| 861 | +/* | |
| 862 | + * @brief The scheduling log notification is generated by VPU when it writes | |
| 863 | + * an event into the log buffer at the notify_index. VPU notifies host with | |
| 864 | + * VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION. This is an asynchronous | |
| 865 | + * message from VPU to host. | |
| 866 | + * @see VPU_JSM_MSG_HWS_SCHEDULING_LOG_NOTIFICATION | |
| 867 | + * @see VPU_JSM_MSG_HWS_SET_SCHEDULING_LOG | |
| 868 | + */ | |
| 869 | +struct vpu_ipc_msg_payload_hws_scheduling_log_notification { | |
| 870 | + /* Engine ordinal */ | |
| 871 | + u32 engine_idx; | |
| 872 | + /* Zero Padding */ | |
| 873 | + u32 reserved_0; | |
| 874 | +}; | |
| 875 | + | |
| 876 | +/* | |
| 877 | + * @brief HWS suspend command queue request and done structure. | |
| 878 | + * Host will request the suspend of contexts and VPU will; | |
| 879 | + * - Suspend all work on this context | |
| 880 | + * - Preempt any running work | |
| 881 | + * - Asynchronously perform the above and return success immediately once | |
| 882 | + * all items above are started successfully | |
| 883 | + * - Notify the host of completion of these operations via | |
| 884 | + * VPU_JSM_MSG_HWS_SUSPEND_CMDQ_DONE | |
| 885 | + * - Reject any other context operations on a context with an in-flight | |
| 886 | + * suspend request running | |
| 887 | + * Same structure used when VPU notifies host of completion of a context suspend | |
| 888 | + * request. The ids and suspend fence value reported in this command will match | |
| 889 | + * the one in the request from the host to suspend the context. Once suspend is | |
| 890 | + * complete, VPU will not access any data relating to this command queue until | |
| 891 | + * it is resumed. | |
| 892 | + * @see VPU_JSM_MSG_HWS_SUSPEND_CMDQ | |
| 893 | + * @see VPU_JSM_MSG_HWS_SUSPEND_CMDQ_DONE | |
| 894 | + */ | |
| 895 | +struct vpu_ipc_msg_payload_hws_suspend_cmdq { | |
| 896 | + /* Host SSID */ | |
| 897 | + u32 host_ssid; | |
| 898 | + /* Zero Padding */ | |
| 899 | + u32 reserved_0; | |
| 900 | + /* Command queue id */ | |
| 901 | + u64 cmdq_id; | |
| 902 | + /* | |
| 903 | + * Suspend fence value - reported by the VPU suspend context | |
| 904 | + * completed once suspend is complete. | |
| 905 | + */ | |
| 906 | + u64 suspend_fence_value; | |
| 907 | +}; | |
| 908 | + | |
| 909 | +/* | |
| 910 | + * @brief HWS Resume command queue request / response structure. | |
| 911 | + * Host will request the resume of a context; | |
| 912 | + * - VPU will resume all work on this context | |
| 913 | + * - Scheduler will allow this context to be scheduled | |
| 914 | + * @see VPU_JSM_MSG_HWS_RESUME_CMDQ | |
| 915 | + * @see VPU_JSM_MSG_HWS_RESUME_CMDQ_RSP | |
| 916 | + */ | |
| 917 | +struct vpu_ipc_msg_payload_hws_resume_cmdq { | |
| 918 | + /* Host SSID */ | |
| 919 | + u32 host_ssid; | |
| 920 | + /* Zero Padding */ | |
| 921 | + u32 reserved_0; | |
| 922 | + /* Command queue id */ | |
| 923 | + u64 cmdq_id; | |
| 924 | +}; | |
| 925 | + | |
| 926 | +/* | |
| 927 | + * @brief HWS Resume engine request / response structure. | |
| 928 | + * After a HWS engine reset, all scheduling is stopped on VPU until a engine resume. | |
| 929 | + * Host shall send this command to resume scheduling of any valid queue. | |
| 930 | + * @see VPU_JSM_MSG_HWS_RESUME_ENGINE | |
| 931 | + * @see VPU_JSM_MSG_HWS_RESUME_ENGINE_DONE | |
| 932 | + */ | |
| 933 | +struct vpu_ipc_msg_payload_hws_resume_engine { | |
| 934 | + /* Engine to be resumed */ | |
| 935 | + u32 engine_idx; | |
| 936 | + /* Reserved */ | |
| 937 | + u32 reserved_0; | |
| 938 | +}; | |
| 939 | + | |
| 697 | 940 | /** |
| 698 | 941 | * Payload for VPU_JSM_MSG_TRACE_SET_CONFIG[_RSP] and |
| 699 | 942 | * VPU_JSM_MSG_TRACE_GET_CONFIG_RSP messages. |
| @@ -938,6 +1181,35 @@ struct vpu_ipc_msg_payload_dyndbg_control { · 238 unmodified lines | ||
| 938 | 1181 | char dyndbg_cmd[VPU_DYNDBG_CMD_MAX_LEN]; |
| 939 | 1182 | }; |
| 940 | 1183 | |
| 1184 | +/** | |
| 1185 | + * Payload for VPU_JSM_MSG_PWR_D0I3_ENTER | |
| 1186 | + * | |
| 1187 | + * This is a bi-directional payload. | |
| 1188 | + */ | |
| 1189 | +struct vpu_ipc_msg_payload_pwr_d0i3_enter { | |
| 1190 | + /** | |
| 1191 | + * 0: VPU_JSM_MSG_PWR_D0I3_ENTER_DONE is not sent to the host driver | |
| 1192 | + * The driver will poll for D0i2 Idle state transitions. | |
| 1193 | + * 1: VPU_JSM_MSG_PWR_D0I3_ENTER_DONE is sent after VPU state save is complete | |
| 1194 | + */ | |
| 1195 | + u32 send_response; | |
| 1196 | + u32 reserved_0; | |
| 1197 | +}; | |
| 1198 | + | |
| 1199 | +/** | |
| 1200 | + * Payload for VPU_JSM_MSG_DCT_ENABLE message. | |
| 1201 | + * | |
| 1202 | + * Default values for DCT active/inactive times are 5.3ms and 30ms respectively, | |
| 1203 | + * corresponding to a 85% duty cycle. This payload allows the host to tune these | |
| 1204 | + * values according to application requirements. | |
| 1205 | + */ | |
| 1206 | +struct vpu_ipc_msg_payload_pwr_dct_control { | |
| 1207 | + /** Duty cycle active time in microseconds */ | |
| 1208 | + u32 dct_active_us; | |
| 1209 | + /** Duty cycle inactive time in microseconds */ | |
| 1210 | + u32 dct_inactive_us; | |
| 1211 | +}; | |
| 1212 | + | |
| 941 | 1213 | /* |
| 942 | 1214 | * Payloads union, used to define complete message format. |
| 943 | 1215 | */ |
| @@ -974,6 +1246,13 @@ union vpu_ipc_msg_payload { · 30 unmodified lines | ||
| 974 | 1246 | struct vpu_ipc_msg_payload_hws_destroy_cmdq hws_destroy_cmdq; |
| 975 | 1247 | struct vpu_ipc_msg_payload_hws_set_context_sched_properties |
| 976 | 1248 | hws_set_context_sched_properties; |
| 1249 | + struct vpu_ipc_msg_payload_hws_set_scheduling_log hws_set_scheduling_log; | |
| 1250 | + struct vpu_ipc_msg_payload_hws_scheduling_log_notification hws_scheduling_log_notification; | |
| 1251 | + struct vpu_ipc_msg_payload_hws_suspend_cmdq hws_suspend_cmdq; | |
| 1252 | + struct vpu_ipc_msg_payload_hws_resume_cmdq hws_resume_cmdq; | |
| 1253 | + struct vpu_ipc_msg_payload_hws_resume_engine hws_resume_engine; | |
| 1254 | + struct vpu_ipc_msg_payload_pwr_d0i3_enter pwr_d0i3_enter; | |
| 1255 | + struct vpu_ipc_msg_payload_pwr_dct_control pwr_dct_control; | |
| 977 | 1256 | }; |
| 978 | 1257 | |
| 979 | 1258 | /* |
| @@ -9,4 +9,5 @@ qaic-y := \ · 8 unmodified lines | ||
| 9 | 9 | mhi_controller.o \ |
| 10 | 10 | qaic_control.o \ |
| 11 | 11 | qaic_data.o \ |
| 12 | − qaic_drv.o | |
| 12 | + qaic_drv.o \ | |
| 13 | + qaic_timesync.o | |
| @@ -348,7 +348,7 @@ static struct mhi_channel_config aic100_channels[] = { · 347 unmodified lines | ||
| 348 | 348 | .local_elements = 0, |
| 349 | 349 | .event_ring = 0, |
| 350 | 350 | .dir = DMA_TO_DEVICE, |
| 351 | − .ee_mask = MHI_CH_EE_SBL | MHI_CH_EE_AMSS, | |
| 351 | + .ee_mask = MHI_CH_EE_SBL, | |
| 352 | 352 | .pollcfg = 0, |
| 353 | 353 | .doorbell = MHI_DB_BRST_DISABLE, |
| 354 | 354 | .lpm_notify = false, |
| @@ -364,7 +364,39 @@ static struct mhi_channel_config aic100_channels[] = { · 9 unmodified lines | ||
| 364 | 364 | .local_elements = 0, |
| 365 | 365 | .event_ring = 0, |
| 366 | 366 | .dir = DMA_FROM_DEVICE, |
| 367 | − .ee_mask = MHI_CH_EE_SBL | MHI_CH_EE_AMSS, | |
| 367 | + .ee_mask = MHI_CH_EE_SBL, | |
| 368 | + .pollcfg = 0, | |
| 369 | + .doorbell = MHI_DB_BRST_DISABLE, | |
| 370 | + .lpm_notify = false, | |
| 371 | + .offload_channel = false, | |
| 372 | + .doorbell_mode_switch = false, | |
| 373 | + .auto_queue = false, | |
| 374 | + .wake_capable = false, | |
| 375 | + }, | |
| 376 | + { | |
| 377 | + .name = "QAIC_TIMESYNC_PERIODIC", | |
| 378 | + .num = 22, | |
| 379 | + .num_elements = 32, | |
| 380 | + .local_elements = 0, | |
| 381 | + .event_ring = 0, | |
| 382 | + .dir = DMA_TO_DEVICE, | |
| 383 | + .ee_mask = MHI_CH_EE_AMSS, | |
| 384 | + .pollcfg = 0, | |
| 385 | + .doorbell = MHI_DB_BRST_DISABLE, | |
| 386 | + .lpm_notify = false, | |
| 387 | + .offload_channel = false, | |
| 388 | + .doorbell_mode_switch = false, | |
| 389 | + .auto_queue = false, | |
| 390 | + .wake_capable = false, | |
| 391 | + }, | |
| 392 | + { | |
| 393 | + .num = 23, | |
| 394 | + .name = "QAIC_TIMESYNC_PERIODIC", | |
| 395 | + .num_elements = 32, | |
| 396 | + .local_elements = 0, | |
| 397 | + .event_ring = 0, | |
| 398 | + .dir = DMA_FROM_DEVICE, | |
| 399 | + .ee_mask = MHI_CH_EE_AMSS, | |
| 368 | 400 | .pollcfg = 0, |
| 369 | 401 | .doorbell = MHI_DB_BRST_DISABLE, |
| 370 | 402 | .lpm_notify = false, |
| @@ -437,7 +469,7 @@ static void mhi_status_cb(struct mhi_controller *mhi_cntrl, enum mhi_callback re · 66 unmodified lines | ||
| 437 | 469 | pci_err(qdev->pdev, "Fatal error received from device. Attempting to recover\n"); |
| 438 | 470 | /* this event occurs in non-atomic context */ |
| 439 | 471 | if (reason == MHI_CB_SYS_ERROR) |
| 440 | − qaic_dev_reset_clean_local_state(qdev, true); | |
| 472 | + qaic_dev_reset_clean_local_state(qdev); | |
| 441 | 473 | } |
| 442 | 474 | |
| 443 | 475 | static int mhi_reset_and_async_power_up(struct mhi_controller *mhi_cntrl) |
| @@ -468,7 +500,7 @@ static int mhi_reset_and_async_power_up(struct mhi_controller *mhi_cntrl) · 24 unmodified lines | ||
| 468 | 500 | } |
| 469 | 501 | |
| 470 | 502 | struct mhi_controller *qaic_mhi_register_controller(struct pci_dev *pci_dev, void __iomem *mhi_bar, |
| 471 | − int mhi_irq) | |
| 503 | + int mhi_irq, bool shared_msi) | |
| 472 | 504 | { |
| 473 | 505 | struct mhi_controller *mhi_cntrl; |
| 474 | 506 | int ret; |
| @@ -500,6 +532,10 @@ struct mhi_controller *qaic_mhi_register_controller(struct pci_dev *pci_dev, voi · 25 unmodified lines | ||
| 500 | 532 | return ERR_PTR(-ENOMEM); |
| 501 | 533 | |
| 502 | 534 | mhi_cntrl->irq[0] = mhi_irq; |
| 535 | + | |
| 536 | + if (shared_msi) /* MSI shared with data path, no IRQF_NO_SUSPEND */ | |
| 537 | + mhi_cntrl->irq_flags = IRQF_SHARED; | |
| 538 | + | |
| 503 | 539 | mhi_cntrl->fw_image = "qcom/aic100/sbl.bin"; |
| 504 | 540 | |
| 505 | 541 | /* use latest configured timeout */ |
| @@ -8,7 +8,7 @@ · 7 unmodified lines | ||
| 8 | 8 | #define MHICONTROLLERQAIC_H_ |
| 9 | 9 | |
| 10 | 10 | struct mhi_controller *qaic_mhi_register_controller(struct pci_dev *pci_dev, void __iomem *mhi_bar, |
| 11 | − int mhi_irq); | |
| 11 | + int mhi_irq, bool shared_msi); | |
| 12 | 12 | void qaic_mhi_free_controller(struct mhi_controller *mhi_cntrl, bool link_up); |
| 13 | 13 | void qaic_mhi_start_reset(struct mhi_controller *mhi_cntrl); |
| 14 | 14 | void qaic_mhi_reset_done(struct mhi_controller *mhi_cntrl); |
| @@ -31,6 +31,15 @@ · 30 unmodified lines | ||
| 31 | 31 | #define to_drm(qddev) (&(qddev)->drm) |
| 32 | 32 | #define to_accel_kdev(qddev) (to_drm(qddev)->accel->kdev) /* Return Linux device of accel node */ |
| 33 | 33 | |
| 34 | +enum __packed dev_states { | |
| 35 | + /* Device is offline or will be very soon */ | |
| 36 | + QAIC_OFFLINE, | |
| 37 | + /* Device is booting, not clear if it's in a usable state */ | |
| 38 | + QAIC_BOOT, | |
| 39 | + /* Device is fully operational */ | |
| 40 | + QAIC_ONLINE, | |
| 41 | +}; | |
| 42 | + | |
| 34 | 43 | extern bool datapath_polling; |
| 35 | 44 | |
| 36 | 45 | struct qaic_user { |
| @@ -121,8 +130,10 @@ struct qaic_device { · 84 unmodified lines | ||
| 121 | 130 | struct workqueue_struct *cntl_wq; |
| 122 | 131 | /* Synchronizes all the users of device during cleanup */ |
| 123 | 132 | struct srcu_struct dev_lock; |
| 124 | − /* true: Device under reset; false: Device not under reset */ | |
| 125 | − bool in_reset; | |
| 133 | + /* Track the state of the device during resets */ | |
| 134 | + enum dev_states dev_state; | |
| 135 | + /* true: single MSI is used to operate device */ | |
| 136 | + bool single_msi; | |
| 126 | 137 | /* |
| 127 | 138 | * true: A tx MHI transaction has failed and a rx buffer is still queued |
| 128 | 139 | * in control device. Such a buffer is considered lost rx buffer |
| @@ -137,6 +148,10 @@ struct qaic_device { · 8 unmodified lines | ||
| 137 | 148 | u32 (*gen_crc)(void *msg); |
| 138 | 149 | /* Validate the CRC of a control message */ |
| 139 | 150 | bool (*valid_crc)(void *msg); |
| 151 | + /* MHI "QAIC_TIMESYNC" channel device */ | |
| 152 | + struct mhi_device *qts_ch; | |
| 153 | + /* Work queue for tasks related to MHI "QAIC_TIMESYNC" channel */ | |
| 154 | + struct workqueue_struct *qts_wq; | |
| 140 | 155 | }; |
| 141 | 156 | |
| 142 | 157 | struct qaic_drm_device { |
| @@ -268,7 +283,7 @@ void wakeup_dbc(struct qaic_device *qdev, u32 dbc_id); · 125 unmodified lines | ||
| 268 | 283 | void release_dbc(struct qaic_device *qdev, u32 dbc_id); |
| 269 | 284 | |
| 270 | 285 | void wake_all_cntl(struct qaic_device *qdev); |
| 271 | −void qaic_dev_reset_clean_local_state(struct qaic_device *qdev, bool exit_reset); | |
| 286 | +void qaic_dev_reset_clean_local_state(struct qaic_device *qdev); | |
| 272 | 287 | |
| 273 | 288 | struct drm_gem_object *qaic_gem_prime_import(struct drm_device *dev, struct dma_buf *dma_buf); |
| 274 | 289 | |
| @@ -1022,7 +1022,8 @@ static void *msg_xfer(struct qaic_device *qdev, struct wrapper_list *wrappers, u · 1021 unmodified lines | ||
| 1022 | 1022 | int xfer_count = 0; |
| 1023 | 1023 | int retry_count; |
| 1024 | 1024 | |
| 1025 | − if (qdev->in_reset) { | |
| 1025 | + /* Allow QAIC_BOOT state since we need to check control protocol version */ | |
| 1026 | + if (qdev->dev_state == QAIC_OFFLINE) { | |
| 1026 | 1027 | mutex_unlock(&qdev->cntl_mutex); |
| 1027 | 1028 | return ERR_PTR(-ENODEV); |
| 1028 | 1029 | } |
| @@ -1138,7 +1139,7 @@ static int abort_dma_cont(struct qaic_device *qdev, struct wrapper_list *wrapper · 109 unmodified lines | ||
| 1138 | 1139 | if (!list_is_first(&wrapper->list, &wrappers->list)) |
| 1139 | 1140 | kref_put(&wrapper->ref_count, free_wrapper); |
| 1140 | 1141 | |
| 1141 | − wrapper = add_wrapper(wrappers, offsetof(struct wrapper_msg, trans) + sizeof(*out_trans)); | |
| 1142 | + wrapper = add_wrapper(wrappers, sizeof(*wrapper)); | |
| 1142 | 1143 | |
| 1143 | 1144 | if (!wrapper) |
| 1144 | 1145 | return -ENOMEM; |
| @@ -1306,7 +1307,7 @@ int qaic_manage_ioctl(struct drm_device *dev, void *data, struct drm_file *file_ · 161 unmodified lines | ||
| 1306 | 1307 | qdev = usr->qddev->qdev; |
| 1307 | 1308 | |
| 1308 | 1309 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 1309 | − if (qdev->in_reset) { | |
| 1310 | + if (qdev->dev_state != QAIC_ONLINE) { | |
| 1310 | 1311 | srcu_read_unlock(&qdev->dev_lock, qdev_rcu_id); |
| 1311 | 1312 | srcu_read_unlock(&usr->qddev_lock, usr_rcu_id); |
| 1312 | 1313 | return -ENODEV; |
| @@ -51,6 +51,7 @@ · 50 unmodified lines | ||
| 51 | 51 | }) |
| 52 | 52 | #define NUM_EVENTS 128 |
| 53 | 53 | #define NUM_DELAYS 10 |
| 54 | +#define fifo_at(base, offset) ((base) + (offset) * get_dbc_req_elem_size()) | |
| 54 | 55 | |
| 55 | 56 | static unsigned int wait_exec_default_timeout_ms = 5000; /* 5 sec default */ |
| 56 | 57 | module_param(wait_exec_default_timeout_ms, uint, 0600); |
| @@ -451,7 +452,7 @@ static int create_sgt(struct qaic_device *qdev, struct sg_table **sgt_out, u64 s · 394 unmodified lines | ||
| 451 | 452 | * later |
| 452 | 453 | */ |
| 453 | 454 | buf_extra = (PAGE_SIZE - size % PAGE_SIZE) % PAGE_SIZE; |
| 454 | − max_order = min(MAX_ORDER - 1, get_order(size)); | |
| 455 | + max_order = min(MAX_ORDER, get_order(size)); | |
| 455 | 456 | } else { |
| 456 | 457 | /* allocate a single page for book keeping */ |
| 457 | 458 | nr_pages = 1; |
| @@ -689,7 +690,7 @@ int qaic_create_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *fi · 231 unmodified lines | ||
| 689 | 690 | |
| 690 | 691 | qdev = usr->qddev->qdev; |
| 691 | 692 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 692 | − if (qdev->in_reset) { | |
| 693 | + if (qdev->dev_state != QAIC_ONLINE) { | |
| 693 | 694 | ret = -ENODEV; |
| 694 | 695 | goto unlock_dev_srcu; |
| 695 | 696 | } |
| @@ -748,7 +749,7 @@ int qaic_mmap_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *file · 52 unmodified lines | ||
| 748 | 749 | |
| 749 | 750 | qdev = usr->qddev->qdev; |
| 750 | 751 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 751 | − if (qdev->in_reset) { | |
| 752 | + if (qdev->dev_state != QAIC_ONLINE) { | |
| 752 | 753 | ret = -ENODEV; |
| 753 | 754 | goto unlock_dev_srcu; |
| 754 | 755 | } |
| @@ -969,7 +970,7 @@ int qaic_attach_slice_bo_ioctl(struct drm_device *dev, void *data, struct drm_fi · 214 unmodified lines | ||
| 969 | 970 | |
| 970 | 971 | qdev = usr->qddev->qdev; |
| 971 | 972 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 972 | − if (qdev->in_reset) { | |
| 973 | + if (qdev->dev_state != QAIC_ONLINE) { | |
| 973 | 974 | ret = -ENODEV; |
| 974 | 975 | goto unlock_dev_srcu; |
| 975 | 976 | } |
| @@ -1058,6 +1059,16 @@ unlock_usr_srcu: · 82 unmodified lines | ||
| 1058 | 1059 | return ret; |
| 1059 | 1060 | } |
| 1060 | 1061 | |
| 1062 | +static inline u32 fifo_space_avail(u32 head, u32 tail, u32 q_size) | |
| 1063 | +{ | |
| 1064 | + u32 avail = head - tail - 1; | |
| 1065 | + | |
| 1066 | + if (head <= tail) | |
| 1067 | + avail += q_size; | |
| 1068 | + | |
| 1069 | + return avail; | |
| 1070 | +} | |
| 1071 | + | |
| 1061 | 1072 | static inline int copy_exec_reqs(struct qaic_device *qdev, struct bo_slice *slice, u32 dbc_id, |
| 1062 | 1073 | u32 head, u32 *ptail) |
| 1063 | 1074 | { |
| @@ -1066,27 +1077,20 @@ static inline int copy_exec_reqs(struct qaic_device *qdev, struct bo_slice *slic · 2 unmodified lines | ||
| 1066 | 1077 | u32 tail = *ptail; |
| 1067 | 1078 | u32 avail; |
| 1068 | 1079 | |
| 1069 | − avail = head - tail; | |
| 1070 | − if (head <= tail) | |
| 1071 | − avail += dbc->nelem; | |
| 1072 | − | |
| 1073 | − --avail; | |
| 1074 | − | |
| 1080 | + avail = fifo_space_avail(head, tail, dbc->nelem); | |
| 1075 | 1081 | if (avail < slice->nents) |
| 1076 | 1082 | return -EAGAIN; |
| 1077 | 1083 | |
| 1078 | 1084 | if (tail + slice->nents > dbc->nelem) { |
| 1079 | 1085 | avail = dbc->nelem - tail; |
| 1080 | 1086 | avail = min_t(u32, avail, slice->nents); |
| 1081 | − memcpy(dbc->req_q_base + tail * get_dbc_req_elem_size(), reqs, | |
| 1082 | − sizeof(*reqs) * avail); | |
| 1087 | + memcpy(fifo_at(dbc->req_q_base, tail), reqs, sizeof(*reqs) * avail); | |
| 1083 | 1088 | reqs += avail; |
| 1084 | 1089 | avail = slice->nents - avail; |
| 1085 | 1090 | if (avail) |
| 1086 | 1091 | memcpy(dbc->req_q_base, reqs, sizeof(*reqs) * avail); |
| 1087 | 1092 | } else { |
| 1088 | − memcpy(dbc->req_q_base + tail * get_dbc_req_elem_size(), reqs, | |
| 1089 | − sizeof(*reqs) * slice->nents); | |
| 1093 | + memcpy(fifo_at(dbc->req_q_base, tail), reqs, sizeof(*reqs) * slice->nents); | |
| 1090 | 1094 | } |
| 1091 | 1095 | |
| 1092 | 1096 | *ptail = (tail + slice->nents) % dbc->nelem; |
| @@ -1094,46 +1098,31 @@ static inline int copy_exec_reqs(struct qaic_device *qdev, struct bo_slice *slic · 1 unmodified line | ||
| 1094 | 1098 | return 0; |
| 1095 | 1099 | } |
| 1096 | 1100 | |
| 1097 | −/* | |
| 1098 | − * Based on the value of resize we may only need to transmit first_n | |
| 1099 | − * entries and the last entry, with last_bytes to send from the last entry. | |
| 1100 | − * Note that first_n could be 0. | |
| 1101 | − */ | |
| 1102 | 1101 | static inline int copy_partial_exec_reqs(struct qaic_device *qdev, struct bo_slice *slice, |
| 1103 | − u64 resize, u32 dbc_id, u32 head, u32 *ptail) | |
| 1102 | + u64 resize, struct dma_bridge_chan *dbc, u32 head, | |
| 1103 | + u32 *ptail) | |
| 1104 | 1104 | { |
| 1105 | − struct dma_bridge_chan *dbc = &qdev->dbc[dbc_id]; | |
| 1106 | 1105 | struct dbc_req *reqs = slice->reqs; |
| 1107 | 1106 | struct dbc_req *last_req; |
| 1108 | 1107 | u32 tail = *ptail; |
| 1109 | − u64 total_bytes; | |
| 1110 | 1108 | u64 last_bytes; |
| 1111 | 1109 | u32 first_n; |
| 1112 | 1110 | u32 avail; |
| 1113 | − int ret; | |
| 1114 | − int i; | |
| 1115 | − | |
| 1116 | − avail = head - tail; | |
| 1117 | − if (head <= tail) | |
| 1118 | − avail += dbc->nelem; | |
| 1119 | 1111 | |
| 1120 | − --avail; | |
| 1112 | + avail = fifo_space_avail(head, tail, dbc->nelem); | |
| 1121 | 1113 | |
| 1122 | − total_bytes = 0; | |
| 1123 | − for (i = 0; i < slice->nents; i++) { | |
| 1124 | − total_bytes += le32_to_cpu(reqs[i].len); | |
| 1125 | − if (total_bytes >= resize) | |
| 1114 | + /* | |
| 1115 | + * After this for loop is complete, first_n represents the index | |
| 1116 | + * of the last DMA request of this slice that needs to be | |
| 1117 | + * transferred after resizing and last_bytes represents DMA size | |
| 1118 | + * of that request. | |
| 1119 | + */ | |
| 1120 | + last_bytes = resize; | |
| 1121 | + for (first_n = 0; first_n < slice->nents; first_n++) | |
| 1122 | + if (last_bytes > le32_to_cpu(reqs[first_n].len)) | |
| 1123 | + last_bytes -= le32_to_cpu(reqs[first_n].len); | |
| 1124 | + else | |
| 1126 | 1125 | break; |
| 1127 | − } | |
| 1128 | − | |
| 1129 | − if (total_bytes < resize) { | |
| 1130 | − /* User space should have used the full buffer path. */ | |
| 1131 | − ret = -EINVAL; | |
| 1132 | − return ret; | |
| 1133 | − } | |
| 1134 | − | |
| 1135 | − first_n = i; | |
| 1136 | − last_bytes = i ? resize + le32_to_cpu(reqs[i].len) - total_bytes : resize; | |
| 1137 | 1126 | |
| 1138 | 1127 | if (avail < (first_n + 1)) |
| 1139 | 1128 | return -EAGAIN; |
| @@ -1142,22 +1131,21 @@ static inline int copy_partial_exec_reqs(struct qaic_device *qdev, struct bo_sli · 2 unmodified lines | ||
| 1142 | 1131 | if (tail + first_n > dbc->nelem) { |
| 1143 | 1132 | avail = dbc->nelem - tail; |
| 1144 | 1133 | avail = min_t(u32, avail, first_n); |
| 1145 | − memcpy(dbc->req_q_base + tail * get_dbc_req_elem_size(), reqs, | |
| 1146 | − sizeof(*reqs) * avail); | |
| 1134 | + memcpy(fifo_at(dbc->req_q_base, tail), reqs, sizeof(*reqs) * avail); | |
| 1147 | 1135 | last_req = reqs + avail; |
| 1148 | 1136 | avail = first_n - avail; |
| 1149 | 1137 | if (avail) |
| 1150 | 1138 | memcpy(dbc->req_q_base, last_req, sizeof(*reqs) * avail); |
| 1151 | 1139 | } else { |
| 1152 | − memcpy(dbc->req_q_base + tail * get_dbc_req_elem_size(), reqs, | |
| 1153 | − sizeof(*reqs) * first_n); | |
| 1140 | + memcpy(fifo_at(dbc->req_q_base, tail), reqs, sizeof(*reqs) * first_n); | |
| 1154 | 1141 | } |
| 1155 | 1142 | } |
| 1156 | 1143 | |
| 1157 | − /* Copy over the last entry. Here we need to adjust len to the left over | |
| 1144 | + /* | |
| 1145 | + * Copy over the last entry. Here we need to adjust len to the left over | |
| 1158 | 1146 | * size, and set src and dst to the entry it is copied to. |
| 1159 | 1147 | */ |
| 1160 | − last_req = dbc->req_q_base + (tail + first_n) % dbc->nelem * get_dbc_req_elem_size(); | |
| 1148 | + last_req = fifo_at(dbc->req_q_base, (tail + first_n) % dbc->nelem); | |
| 1161 | 1149 | memcpy(last_req, reqs + slice->nents - 1, sizeof(*reqs)); |
| 1162 | 1150 | |
| 1163 | 1151 | /* |
| @@ -1168,6 +1156,9 @@ static inline int copy_partial_exec_reqs(struct qaic_device *qdev, struct bo_sli · 4 unmodified lines | ||
| 1168 | 1156 | last_req->len = cpu_to_le32((u32)last_bytes); |
| 1169 | 1157 | last_req->src_addr = reqs[first_n].src_addr; |
| 1170 | 1158 | last_req->dest_addr = reqs[first_n].dest_addr; |
| 1159 | + if (!last_bytes) | |
| 1160 | + /* Disable DMA transfer */ | |
| 1161 | + last_req->cmd = GENMASK(7, 2) & reqs[first_n].cmd; | |
| 1171 | 1162 | |
| 1172 | 1163 | *ptail = (tail + first_n + 1) % dbc->nelem; |
| 1173 | 1164 | |
| @@ -1227,26 +1218,17 @@ static int send_bo_list_to_device(struct qaic_device *qdev, struct drm_file *fil · 53 unmodified lines | ||
| 1227 | 1218 | bo->req_id = dbc->next_req_id++; |
| 1228 | 1219 | |
| 1229 | 1220 | list_for_each_entry(slice, &bo->slices, slice) { |
| 1230 | − /* | |
| 1231 | − * If this slice does not fall under the given | |
| 1232 | − * resize then skip this slice and continue the loop | |
| 1233 | − */ | |
| 1234 | − if (is_partial && pexec[i].resize && pexec[i].resize <= slice->offset) | |
| 1235 | − continue; | |
| 1236 | − | |
| 1237 | 1221 | for (j = 0; j < slice->nents; j++) |
| 1238 | 1222 | slice->reqs[j].req_id = cpu_to_le16(bo->req_id); |
| 1239 | 1223 | |
| 1240 | − /* | |
| 1241 | − * If it is a partial execute ioctl call then check if | |
| 1242 | − * resize has cut this slice short then do a partial copy | |
| 1243 | − * else do complete copy | |
| 1244 | − */ | |
| 1245 | − if (is_partial && pexec[i].resize && | |
| 1246 | − pexec[i].resize < slice->offset + slice->size) | |
| 1224 | + if (is_partial && (!pexec[i].resize || pexec[i].resize <= slice->offset)) | |
| 1225 | + /* Configure the slice for no DMA transfer */ | |
| 1226 | + ret = copy_partial_exec_reqs(qdev, slice, 0, dbc, head, tail); | |
| 1227 | + else if (is_partial && pexec[i].resize < slice->offset + slice->size) | |
| 1228 | + /* Configure the slice to be partially DMA transferred */ | |
| 1247 | 1229 | ret = copy_partial_exec_reqs(qdev, slice, |
| 1248 | − pexec[i].resize - slice->offset, | |
| 1249 | − dbc->id, head, tail); | |
| 1230 | + pexec[i].resize - slice->offset, dbc, | |
| 1231 | + head, tail); | |
| 1250 | 1232 | else |
| 1251 | 1233 | ret = copy_exec_reqs(qdev, slice, dbc->id, head, tail); |
| 1252 | 1234 | if (ret) { |
| @@ -1359,7 +1341,7 @@ static int __qaic_execute_bo_ioctl(struct drm_device *dev, void *data, struct dr · 106 unmodified lines | ||
| 1359 | 1341 | |
| 1360 | 1342 | qdev = usr->qddev->qdev; |
| 1361 | 1343 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 1362 | − if (qdev->in_reset) { | |
| 1344 | + if (qdev->dev_state != QAIC_ONLINE) { | |
| 1363 | 1345 | ret = -ENODEV; |
| 1364 | 1346 | goto unlock_dev_srcu; |
| 1365 | 1347 | } |
| @@ -1466,6 +1448,16 @@ irqreturn_t dbc_irq_handler(int irq, void *data) · 100 unmodified lines | ||
| 1466 | 1448 | |
| 1467 | 1449 | rcu_id = srcu_read_lock(&dbc->ch_lock); |
| 1468 | 1450 | |
| 1451 | + if (datapath_polling) { | |
| 1452 | + srcu_read_unlock(&dbc->ch_lock, rcu_id); | |
| 1453 | + /* | |
| 1454 | + * Normally datapath_polling will not have irqs enabled, but | |
| 1455 | + * when running with only one MSI the interrupt is shared with | |
| 1456 | + * MHI so it cannot be disabled. Return ASAP instead. | |
| 1457 | + */ | |
| 1458 | + return IRQ_HANDLED; | |
| 1459 | + } | |
| 1460 | + | |
| 1469 | 1461 | if (!dbc->usr) { |
| 1470 | 1462 | srcu_read_unlock(&dbc->ch_lock, rcu_id); |
| 1471 | 1463 | return IRQ_HANDLED; |
| @@ -1488,7 +1480,8 @@ irqreturn_t dbc_irq_handler(int irq, void *data) · 16 unmodified lines | ||
| 1488 | 1480 | return IRQ_NONE; |
| 1489 | 1481 | } |
| 1490 | 1482 | |
| 1491 | − disable_irq_nosync(irq); | |
| 1483 | + if (!dbc->qdev->single_msi) | |
| 1484 | + disable_irq_nosync(irq); | |
| 1492 | 1485 | srcu_read_unlock(&dbc->ch_lock, rcu_id); |
| 1493 | 1486 | return IRQ_WAKE_THREAD; |
| 1494 | 1487 | } |
| @@ -1504,7 +1497,7 @@ void irq_polling_work(struct work_struct *work) · 9 unmodified lines | ||
| 1504 | 1497 | rcu_id = srcu_read_lock(&dbc->ch_lock); |
| 1505 | 1498 | |
| 1506 | 1499 | while (1) { |
| 1507 | − if (dbc->qdev->in_reset) { | |
| 1500 | + if (dbc->qdev->dev_state != QAIC_ONLINE) { | |
| 1508 | 1501 | srcu_read_unlock(&dbc->ch_lock, rcu_id); |
| 1509 | 1502 | return; |
| 1510 | 1503 | } |
| @@ -1559,12 +1552,12 @@ irqreturn_t dbc_irq_threaded_fn(int irq, void *data) · 48 unmodified lines | ||
| 1559 | 1552 | u32 tail; |
| 1560 | 1553 | |
| 1561 | 1554 | rcu_id = srcu_read_lock(&dbc->ch_lock); |
| 1555 | + qdev = dbc->qdev; | |
| 1562 | 1556 | |
| 1563 | 1557 | head = readl(dbc->dbc_base + RSPHP_OFF); |
| 1564 | 1558 | if (head == U32_MAX) /* PCI link error */ |
| 1565 | 1559 | goto error_out; |
| 1566 | 1560 | |
| 1567 | − qdev = dbc->qdev; | |
| 1568 | 1561 | read_fifo: |
| 1569 | 1562 | |
| 1570 | 1563 | if (!event_count) { |
| @@ -1645,14 +1638,14 @@ read_fifo: · 74 unmodified lines | ||
| 1645 | 1638 | goto read_fifo; |
| 1646 | 1639 | |
| 1647 | 1640 | normal_out: |
| 1648 | − if (likely(!datapath_polling)) | |
| 1641 | + if (!qdev->single_msi && likely(!datapath_polling)) | |
| 1649 | 1642 | enable_irq(irq); |
| 1650 | − else | |
| 1643 | + else if (unlikely(datapath_polling)) | |
| 1651 | 1644 | schedule_work(&dbc->poll_work); |
| 1652 | 1645 | /* checking the fifo and enabling irqs is a race, missed event check */ |
| 1653 | 1646 | tail = readl(dbc->dbc_base + RSPTP_OFF); |
| 1654 | 1647 | if (tail != U32_MAX && head != tail) { |
| 1655 | − if (likely(!datapath_polling)) | |
| 1648 | + if (!qdev->single_msi && likely(!datapath_polling)) | |
| 1656 | 1649 | disable_irq_nosync(irq); |
| 1657 | 1650 | goto read_fifo; |
| 1658 | 1651 | } |
| @@ -1661,9 +1654,9 @@ normal_out: · 2 unmodified lines | ||
| 1661 | 1654 | |
| 1662 | 1655 | error_out: |
| 1663 | 1656 | srcu_read_unlock(&dbc->ch_lock, rcu_id); |
| 1664 | − if (likely(!datapath_polling)) | |
| 1657 | + if (!qdev->single_msi && likely(!datapath_polling)) | |
| 1665 | 1658 | enable_irq(irq); |
| 1666 | − else | |
| 1659 | + else if (unlikely(datapath_polling)) | |
| 1667 | 1660 | schedule_work(&dbc->poll_work); |
| 1668 | 1661 | |
| 1669 | 1662 | return IRQ_HANDLED; |
| @@ -1694,7 +1687,7 @@ int qaic_wait_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *file · 24 unmodified lines | ||
| 1694 | 1687 | |
| 1695 | 1688 | qdev = usr->qddev->qdev; |
| 1696 | 1689 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 1697 | − if (qdev->in_reset) { | |
| 1690 | + if (qdev->dev_state != QAIC_ONLINE) { | |
| 1698 | 1691 | ret = -ENODEV; |
| 1699 | 1692 | goto unlock_dev_srcu; |
| 1700 | 1693 | } |
| @@ -1763,7 +1756,7 @@ int qaic_perf_stats_bo_ioctl(struct drm_device *dev, void *data, struct drm_file · 62 unmodified lines | ||
| 1763 | 1756 | |
| 1764 | 1757 | qdev = usr->qddev->qdev; |
| 1765 | 1758 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 1766 | − if (qdev->in_reset) { | |
| 1759 | + if (qdev->dev_state != QAIC_ONLINE) { | |
| 1767 | 1760 | ret = -ENODEV; |
| 1768 | 1761 | goto unlock_dev_srcu; |
| 1769 | 1762 | } |
| @@ -1854,7 +1847,7 @@ int qaic_detach_slice_bo_ioctl(struct drm_device *dev, void *data, struct drm_fi · 84 unmodified lines | ||
| 1854 | 1847 | |
| 1855 | 1848 | qdev = usr->qddev->qdev; |
| 1856 | 1849 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 1857 | − if (qdev->in_reset) { | |
| 1850 | + if (qdev->dev_state != QAIC_ONLINE) { | |
| 1858 | 1851 | ret = -ENODEV; |
| 1859 | 1852 | goto unlock_dev_srcu; |
| 1860 | 1853 | } |
| @@ -8,6 +8,7 @@ · 7 unmodified lines | ||
| 8 | 8 | #include <linux/idr.h> |
| 9 | 9 | #include <linux/interrupt.h> |
| 10 | 10 | #include <linux/list.h> |
| 11 | +#include <linux/kobject.h> | |
| 11 | 12 | #include <linux/kref.h> |
| 12 | 13 | #include <linux/mhi.h> |
| 13 | 14 | #include <linux/module.h> |
| @@ -27,6 +28,7 @@ · 13 unmodified lines | ||
| 27 | 28 | |
| 28 | 29 | #include "mhi_controller.h" |
| 29 | 30 | #include "qaic.h" |
| 31 | +#include "qaic_timesync.h" | |
| 30 | 32 | |
| 31 | 33 | MODULE_IMPORT_NS(DMA_BUF); |
| 32 | 34 | |
| @@ -42,9 +44,6 @@ MODULE_PARM_DESC(datapath_polling, "Operate the datapath in polling mode"); · 9 unmodified lines | ||
| 42 | 44 | static bool link_up; |
| 43 | 45 | static DEFINE_IDA(qaic_usrs); |
| 44 | 46 | |
| 45 | −static int qaic_create_drm_device(struct qaic_device *qdev, s32 partition_id); | |
| 46 | −static void qaic_destroy_drm_device(struct qaic_device *qdev, s32 partition_id); | |
| 47 | − | |
| 48 | 47 | static void free_usr(struct kref *kref) |
| 49 | 48 | { |
| 50 | 49 | struct qaic_user *usr = container_of(kref, struct qaic_user, ref_count); |
| @@ -63,7 +62,7 @@ static int qaic_open(struct drm_device *dev, struct drm_file *file) · 12 unmodified lines | ||
| 63 | 62 | int ret; |
| 64 | 63 | |
| 65 | 64 | rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 66 | − if (qdev->in_reset) { | |
| 65 | + if (qdev->dev_state != QAIC_ONLINE) { | |
| 67 | 66 | ret = -ENODEV; |
| 68 | 67 | goto dev_unlock; |
| 69 | 68 | } |
| @@ -120,7 +119,7 @@ static void qaic_postclose(struct drm_device *dev, struct drm_file *file) · 50 unmodified lines | ||
| 120 | 119 | if (qddev) { |
| 121 | 120 | qdev = qddev->qdev; |
| 122 | 121 | qdev_rcu_id = srcu_read_lock(&qdev->dev_lock); |
| 123 | − if (!qdev->in_reset) { | |
| 122 | + if (qdev->dev_state == QAIC_ONLINE) { | |
| 124 | 123 | qaic_release_usr(qdev, usr); |
| 125 | 124 | for (i = 0; i < qdev->num_dbc; ++i) |
| 126 | 125 | if (qdev->dbc[i].usr && qdev->dbc[i].usr->handle == usr->handle) |
| @@ -182,13 +181,6 @@ static int qaic_create_drm_device(struct qaic_device *qdev, s32 partition_id) · 55 unmodified lines | ||
| 182 | 181 | |
| 183 | 182 | qddev->partition_id = partition_id; |
| 184 | 183 | |
| 185 | − /* | |
| 186 | − * drm_dev_unregister() sets the driver data to NULL and | |
| 187 | − * drm_dev_register() does not update the driver data. During a SOC | |
| 188 | − * reset drm dev is unregistered and registered again leaving the | |
| 189 | − * driver data to NULL. | |
| 190 | − */ | |
| 191 | − dev_set_drvdata(to_accel_kdev(qddev), drm->accel); | |
| 192 | 184 | ret = drm_dev_register(drm, 0); |
| 193 | 185 | if (ret) |
| 194 | 186 | pci_dbg(qdev->pdev, "drm_dev_register failed %d\n", ret); |
| @@ -202,7 +194,6 @@ static void qaic_destroy_drm_device(struct qaic_device *qdev, s32 partition_id) · 7 unmodified lines | ||
| 202 | 194 | struct drm_device *drm = to_drm(qddev); |
| 203 | 195 | struct qaic_user *usr; |
| 204 | 196 | |
| 205 | − drm_dev_get(drm); | |
| 206 | 197 | drm_dev_unregister(drm); |
| 207 | 198 | qddev->partition_id = 0; |
| 208 | 199 | /* |
| @@ -231,7 +222,6 @@ static void qaic_destroy_drm_device(struct qaic_device *qdev, s32 partition_id) · 22 unmodified lines | ||
| 231 | 222 | mutex_lock(&qddev->users_mutex); |
| 232 | 223 | } |
| 233 | 224 | mutex_unlock(&qddev->users_mutex); |
| 234 | − drm_dev_put(drm); | |
| 235 | 225 | } |
| 236 | 226 | |
| 237 | 227 | static int qaic_mhi_probe(struct mhi_device *mhi_dev, const struct mhi_device_id *id) |
| @@ -253,8 +243,6 @@ static int qaic_mhi_probe(struct mhi_device *mhi_dev, const struct mhi_device_id · 15 unmodified lines | ||
| 253 | 243 | |
| 254 | 244 | qdev = pci_get_drvdata(to_pci_dev(mhi_dev->mhi_cntrl->cntrl_dev)); |
| 255 | 245 | |
| 256 | − qdev->in_reset = false; | |
| 257 | − | |
| 258 | 246 | dev_set_drvdata(&mhi_dev->dev, qdev); |
| 259 | 247 | qdev->cntl_ch = mhi_dev; |
| 260 | 248 | |
| @@ -264,6 +252,7 @@ static int qaic_mhi_probe(struct mhi_device *mhi_dev, const struct mhi_device_id · 3 unmodified lines | ||
| 264 | 252 | return ret; |
| 265 | 253 | } |
| 266 | 254 | |
| 255 | + qdev->dev_state = QAIC_BOOT; | |
| 267 | 256 | ret = get_cntl_version(qdev, NULL, &major, &minor); |
| 268 | 257 | if (ret || major != CNTL_MAJOR || minor > CNTL_MINOR) { |
| 269 | 258 | pci_err(qdev->pdev, "%s: Control protocol version (%d.%d) not supported. Supported version is (%d.%d). Ret: %d\n", |
| @@ -271,8 +260,8 @@ static int qaic_mhi_probe(struct mhi_device *mhi_dev, const struct mhi_device_id · 1 unmodified line | ||
| 271 | 260 | ret = -EINVAL; |
| 272 | 261 | goto close_control; |
| 273 | 262 | } |
| 274 | − | |
| 275 | − ret = qaic_create_drm_device(qdev, QAIC_NO_PARTITION); | |
| 263 | + qdev->dev_state = QAIC_ONLINE; | |
| 264 | + kobject_uevent(&(to_accel_kdev(qdev->qddev))->kobj, KOBJ_ONLINE); | |
| 276 | 265 | |
| 277 | 266 | return ret; |
| 278 | 267 | |
| @@ -290,7 +279,8 @@ static void qaic_notify_reset(struct qaic_device *qdev) · 11 unmodified lines | ||
| 290 | 279 | { |
| 291 | 280 | int i; |
| 292 | 281 | |
| 293 | − qdev->in_reset = true; | |
| 282 | + kobject_uevent(&(to_accel_kdev(qdev->qddev))->kobj, KOBJ_OFFLINE); | |
| 283 | + qdev->dev_state = QAIC_OFFLINE; | |
| 294 | 284 | /* wake up any waiters to avoid waiting for timeouts at sync */ |
| 295 | 285 | wake_all_cntl(qdev); |
| 296 | 286 | for (i = 0; i < qdev->num_dbc; ++i) |
| @@ -298,21 +288,15 @@ static void qaic_notify_reset(struct qaic_device *qdev) · 1 unmodified line | ||
| 298 | 288 | synchronize_srcu(&qdev->dev_lock); |
| 299 | 289 | } |
| 300 | 290 | |
| 301 | −void qaic_dev_reset_clean_local_state(struct qaic_device *qdev, bool exit_reset) | |
| 291 | +void qaic_dev_reset_clean_local_state(struct qaic_device *qdev) | |
| 302 | 292 | { |
| 303 | 293 | int i; |
| 304 | 294 | |
| 305 | 295 | qaic_notify_reset(qdev); |
| 306 | 296 | |
| 307 | − /* remove drmdevs to prevent new users from coming in */ | |
| 308 | − qaic_destroy_drm_device(qdev, QAIC_NO_PARTITION); | |
| 309 | − | |
| 310 | 297 | /* start tearing things down */ |
| 311 | 298 | for (i = 0; i < qdev->num_dbc; ++i) |
| 312 | 299 | release_dbc(qdev, i); |
| 313 | − | |
| 314 | − if (exit_reset) | |
| 315 | − qdev->in_reset = false; | |
| 316 | 300 | } |
| 317 | 301 | |
| 318 | 302 | static void cleanup_qdev(struct qaic_device *qdev) |
| @@ -324,6 +308,7 @@ static void cleanup_qdev(struct qaic_device *qdev) · 5 unmodified lines | ||
| 324 | 308 | cleanup_srcu_struct(&qdev->dev_lock); |
| 325 | 309 | pci_set_drvdata(qdev->pdev, NULL); |
| 326 | 310 | destroy_workqueue(qdev->cntl_wq); |
| 311 | + destroy_workqueue(qdev->qts_wq); | |
| 327 | 312 | } |
| 328 | 313 | |
| 329 | 314 | static struct qaic_device *create_qdev(struct pci_dev *pdev, const struct pci_device_id *id) |
| @@ -336,6 +321,7 @@ static struct qaic_device *create_qdev(struct pci_dev *pdev, const struct pci_de · 6 unmodified lines | ||
| 336 | 321 | if (!qdev) |
| 337 | 322 | return NULL; |
| 338 | 323 | |
| 324 | + qdev->dev_state = QAIC_OFFLINE; | |
| 339 | 325 | if (id->device == PCI_DEV_AIC100) { |
| 340 | 326 | qdev->num_dbc = 16; |
| 341 | 327 | qdev->dbc = devm_kcalloc(&pdev->dev, qdev->num_dbc, sizeof(*qdev->dbc), GFP_KERNEL); |
| @@ -347,6 +333,12 @@ static struct qaic_device *create_qdev(struct pci_dev *pdev, const struct pci_de · 5 unmodified lines | ||
| 347 | 333 | if (!qdev->cntl_wq) |
| 348 | 334 | return NULL; |
| 349 | 335 | |
| 336 | + qdev->qts_wq = alloc_workqueue("qaic_ts", WQ_UNBOUND, 0); | |
| 337 | + if (!qdev->qts_wq) { | |
| 338 | + destroy_workqueue(qdev->cntl_wq); | |
| 339 | + return NULL; | |
| 340 | + } | |
| 341 | + | |
| 350 | 342 | pci_set_drvdata(pdev, qdev); |
| 351 | 343 | qdev->pdev = pdev; |
| 352 | 344 | |
| @@ -424,14 +416,24 @@ static int init_msi(struct qaic_device *qdev, struct pci_dev *pdev) · 71 unmodified lines | ||
| 424 | 416 | int i; |
| 425 | 417 | |
| 426 | 418 | /* Managed release since we use pcim_enable_device */ |
| 427 | − ret = pci_alloc_irq_vectors(pdev, 1, 32, PCI_IRQ_MSI); | |
| 428 | − if (ret < 0) | |
| 429 | − return ret; | |
| 419 | + ret = pci_alloc_irq_vectors(pdev, 32, 32, PCI_IRQ_MSI); | |
| 420 | + if (ret == -ENOSPC) { | |
| 421 | + ret = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_MSI); | |
| 422 | + if (ret < 0) | |
| 423 | + return ret; | |
| 430 | 424 | |
| 431 | − if (ret < 32) { | |
| 432 | − pci_err(pdev, "%s: Requested 32 MSIs. Obtained %d MSIs which is less than the 32 required.\n", | |
| 433 | − __func__, ret); | |
| 434 | − return -ENODEV; | |
| 425 | + /* | |
| 426 | + * Operate in one MSI mode. All interrupts will be directed to | |
| 427 | + * MSI0; every interrupt will wake up all the interrupt handlers | |
| 428 | + * (MHI and DBC[0-15]). Since the interrupt is now shared, it is | |
| 429 | + * not disabled during DBC threaded handler, but only one thread | |
| 430 | + * will be allowed to run per DBC, so while it can be | |
| 431 | + * interrupted, it shouldn't race with itself. | |
| 432 | + */ | |
| 433 | + qdev->single_msi = true; | |
| 434 | + pci_info(pdev, "Allocating 32 MSIs failed, operating in 1 MSI mode. Performance may be impacted.\n"); | |
| 435 | + } else if (ret < 0) { | |
| 436 | + return ret; | |
| 435 | 437 | } |
| 436 | 438 | |
| 437 | 439 | mhi_irq = pci_irq_vector(pdev, 0); |
| @@ -439,15 +441,17 @@ static int init_msi(struct qaic_device *qdev, struct pci_dev *pdev) · 1 unmodified line | ||
| 439 | 441 | return mhi_irq; |
| 440 | 442 | |
| 441 | 443 | for (i = 0; i < qdev->num_dbc; ++i) { |
| 442 | − ret = devm_request_threaded_irq(&pdev->dev, pci_irq_vector(pdev, i + 1), | |
| 444 | + ret = devm_request_threaded_irq(&pdev->dev, | |
| 445 | + pci_irq_vector(pdev, qdev->single_msi ? 0 : i + 1), | |
| 443 | 446 | dbc_irq_handler, dbc_irq_threaded_fn, IRQF_SHARED, |
| 444 | 447 | "qaic_dbc", &qdev->dbc[i]); |
| 445 | 448 | if (ret) |
| 446 | 449 | return ret; |
| 447 | 450 | |
| 448 | 451 | if (datapath_polling) { |
| 449 | − qdev->dbc[i].irq = pci_irq_vector(pdev, i + 1); | |
| 450 | − disable_irq_nosync(qdev->dbc[i].irq); | |
| 452 | + qdev->dbc[i].irq = pci_irq_vector(pdev, qdev->single_msi ? 0 : i + 1); | |
| 453 | + if (!qdev->single_msi) | |
| 454 | + disable_irq_nosync(qdev->dbc[i].irq); | |
| 451 | 455 | INIT_WORK(&qdev->dbc[i].poll_work, irq_polling_work); |
| 452 | 456 | } |
| 453 | 457 | } |
| @@ -479,14 +483,21 @@ static int qaic_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) · 25 unmodified lines | ||
| 479 | 483 | goto cleanup_qdev; |
| 480 | 484 | } |
| 481 | 485 | |
| 482 | − qdev->mhi_cntrl = qaic_mhi_register_controller(pdev, qdev->bar_0, mhi_irq); | |
| 486 | + ret = qaic_create_drm_device(qdev, QAIC_NO_PARTITION); | |
| 487 | + if (ret) | |
| 488 | + goto cleanup_qdev; | |
| 489 | + | |
| 490 | + qdev->mhi_cntrl = qaic_mhi_register_controller(pdev, qdev->bar_0, mhi_irq, | |
| 491 | + qdev->single_msi); | |
| 483 | 492 | if (IS_ERR(qdev->mhi_cntrl)) { |
| 484 | 493 | ret = PTR_ERR(qdev->mhi_cntrl); |
| 485 | − goto cleanup_qdev; | |
| 494 | + goto cleanup_drm_dev; | |
| 486 | 495 | } |
| 487 | 496 | |
| 488 | 497 | return 0; |
| 489 | 498 | |
| 499 | +cleanup_drm_dev: | |
| 500 | + qaic_destroy_drm_device(qdev, QAIC_NO_PARTITION); | |
| 490 | 501 | cleanup_qdev: |
| 491 | 502 | cleanup_qdev(qdev); |
| 492 | 503 | return ret; |
| @@ -499,7 +510,8 @@ static void qaic_pci_remove(struct pci_dev *pdev) · 6 unmodified lines | ||
| 499 | 510 | if (!qdev) |
| 500 | 511 | return; |
| 501 | 512 | |
| 502 | − qaic_dev_reset_clean_local_state(qdev, false); | |
| 513 | + qaic_dev_reset_clean_local_state(qdev); | |
| 514 | + qaic_destroy_drm_device(qdev, QAIC_NO_PARTITION); | |
| 503 | 515 | qaic_mhi_free_controller(qdev->mhi_cntrl, link_up); |
| 504 | 516 | cleanup_qdev(qdev); |
| 505 | 517 | } |
| @@ -522,14 +534,13 @@ static void qaic_pci_reset_prepare(struct pci_dev *pdev) · 16 unmodified lines | ||
| 522 | 534 | |
| 523 | 535 | qaic_notify_reset(qdev); |
| 524 | 536 | qaic_mhi_start_reset(qdev->mhi_cntrl); |
| 525 | − qaic_dev_reset_clean_local_state(qdev, false); | |
| 537 | + qaic_dev_reset_clean_local_state(qdev); | |
| 526 | 538 | } |
| 527 | 539 | |
| 528 | 540 | static void qaic_pci_reset_done(struct pci_dev *pdev) |
| 529 | 541 | { |
| 530 | 542 | struct qaic_device *qdev = pci_get_drvdata(pdev); |
| 531 | 543 | |
| 532 | − qdev->in_reset = false; | |
| 533 | 544 | qaic_mhi_reset_done(qdev->mhi_cntrl); |
| 534 | 545 | } |
| 535 | 546 | |
| @@ -586,6 +597,10 @@ static int __init qaic_init(void) · 50 unmodified lines | ||
| 586 | 597 | goto free_pci; |
| 587 | 598 | } |
| 588 | 599 | |
| 600 | + ret = qaic_timesync_init(); | |
| 601 | + if (ret) | |
| 602 | + pr_debug("qaic: qaic_timesync_init failed %d\n", ret); | |
| 603 | + | |
| 589 | 604 | return 0; |
| 590 | 605 | |
| 591 | 606 | free_pci: |
| @@ -611,6 +626,7 @@ static void __exit qaic_exit(void) · 19 unmodified lines | ||
| 611 | 626 | * reinitializing the link_up state after the cleanup is done. |
| 612 | 627 | */ |
| 613 | 628 | link_up = true; |
| 629 | + qaic_timesync_deinit(); | |
| 614 | 630 | mhi_driver_unregister(&qaic_mhi_driver); |
| 615 | 631 | pci_unregister_driver(&qaic_pci_driver); |
| 616 | 632 | } |
| @@ -0,0 +1,395 @@ | ||
| 1 | +// SPDX-License-Identifier: GPL-2.0-only | |
| 2 | +/* Copyright (c) 2023 Qualcomm Innovation Center, Inc. All rights reserved. */ | |
| 3 | + | |
| 4 | +#include <linux/io.h> | |
| 5 | +#include <linux/kernel.h> | |
| 6 | +#include <linux/math64.h> | |
| 7 | +#include <linux/mhi.h> | |
| 8 | +#include <linux/mod_devicetable.h> | |
| 9 | +#include <linux/module.h> | |
| 10 | +#include <linux/time64.h> | |
| 11 | +#include <linux/timer.h> | |
| 12 | + | |
| 13 | +#include "qaic.h" | |
| 14 | +#include "qaic_timesync.h" | |
| 15 | + | |
| 16 | +#define QTIMER_REG_OFFSET 0xa28 | |
| 17 | +#define QAIC_TIMESYNC_SIGNATURE 0x55aa | |
| 18 | +#define QAIC_CONV_QTIMER_TO_US(qtimer) (mul_u64_u32_div(qtimer, 10, 192)) | |
| 19 | + | |
| 20 | +static unsigned int timesync_delay_ms = 1000; /* 1 sec default */ | |
| 21 | +module_param(timesync_delay_ms, uint, 0600); | |
| 22 | +MODULE_PARM_DESC(timesync_delay_ms, "Delay in ms between two consecutive timesync operations"); | |
| 23 | + | |
| 24 | +enum qts_msg_type { | |
| 25 | + QAIC_TS_CMD_TO_HOST, | |
| 26 | + QAIC_TS_SYNC_REQ, | |
| 27 | + QAIC_TS_ACK_TO_HOST, | |
| 28 | + QAIC_TS_MSG_TYPE_MAX | |
| 29 | +}; | |
| 30 | + | |
| 31 | +/** | |
| 32 | + * struct qts_hdr - Timesync message header structure. | |
| 33 | + * @signature: Unique signature to identify the timesync message. | |
| 34 | + * @reserved_1: Reserved for future use. | |
| 35 | + * @reserved_2: Reserved for future use. | |
| 36 | + * @msg_type: sub-type of the timesync message. | |
| 37 | + * @reserved_3: Reserved for future use. | |
| 38 | + */ | |
| 39 | +struct qts_hdr { | |
| 40 | + __le16 signature; | |
| 41 | + __le16 reserved_1; | |
| 42 | + u8 reserved_2; | |
| 43 | + u8 msg_type; | |
| 44 | + __le16 reserved_3; | |
| 45 | +} __packed; | |
| 46 | + | |
| 47 | +/** | |
| 48 | + * struct qts_timeval - Structure to carry time information. | |
| 49 | + * @tv_sec: Seconds part of the time. | |
| 50 | + * @tv_usec: uS (microseconds) part of the time. | |
| 51 | + */ | |
| 52 | +struct qts_timeval { | |
| 53 | + __le64 tv_sec; | |
| 54 | + __le64 tv_usec; | |
| 55 | +} __packed; | |
| 56 | + | |
| 57 | +/** | |
| 58 | + * struct qts_host_time_sync_msg_data - Structure to denote the timesync message. | |
| 59 | + * @header: Header of the timesync message. | |
| 60 | + * @data: Time information. | |
| 61 | + */ | |
| 62 | +struct qts_host_time_sync_msg_data { | |
| 63 | + struct qts_hdr header; | |
| 64 | + struct qts_timeval data; | |
| 65 | +} __packed; | |
| 66 | + | |
| 67 | +/** | |
| 68 | + * struct mqts_dev - MHI QAIC Timesync Control device. | |
| 69 | + * @qdev: Pointer to the root device struct driven by QAIC driver. | |
| 70 | + * @mhi_dev: Pointer to associated MHI device. | |
| 71 | + * @timer: Timer handle used for timesync. | |
| 72 | + * @qtimer_addr: Device QTimer register pointer. | |
| 73 | + * @buff_in_use: atomic variable to track if the sync_msg buffer is in use. | |
| 74 | + * @dev: Device pointer to qdev->pdev->dev stored for easy access. | |
| 75 | + * @sync_msg: Buffer used to send timesync message over MHI. | |
| 76 | + */ | |
| 77 | +struct mqts_dev { | |
| 78 | + struct qaic_device *qdev; | |
| 79 | + struct mhi_device *mhi_dev; | |
| 80 | + struct timer_list timer; | |
| 81 | + void __iomem *qtimer_addr; | |
| 82 | + atomic_t buff_in_use; | |
| 83 | + struct device *dev; | |
| 84 | + struct qts_host_time_sync_msg_data *sync_msg; | |
| 85 | +}; | |
| 86 | + | |
| 87 | +struct qts_resp_msg { | |
| 88 | + struct qts_hdr hdr; | |
| 89 | +} __packed; | |
| 90 | + | |
| 91 | +struct qts_resp { | |
| 92 | + struct qts_resp_msg data; | |
| 93 | + struct work_struct work; | |
| 94 | + struct qaic_device *qdev; | |
| 95 | +}; | |
| 96 | + | |
| 97 | +#ifdef readq | |
| 98 | +static u64 read_qtimer(const volatile void __iomem *addr) | |
| 99 | +{ | |
| 100 | + return readq(addr); | |
| 101 | +} | |
| 102 | +#else | |
| 103 | +static u64 read_qtimer(const volatile void __iomem *addr) | |
| 104 | +{ | |
| 105 | + u64 low, high; | |
| 106 | + | |
| 107 | + low = readl(addr); | |
| 108 | + high = readl(addr + sizeof(u32)); | |
| 109 | + return low | (high << 32); | |
| 110 | +} | |
| 111 | +#endif | |
| 112 | + | |
| 113 | +static void qaic_timesync_ul_xfer_cb(struct mhi_device *mhi_dev, struct mhi_result *mhi_result) | |
| 114 | +{ | |
| 115 | + struct mqts_dev *mqtsdev = dev_get_drvdata(&mhi_dev->dev); | |
| 116 | + | |
| 117 | + dev_dbg(mqtsdev->dev, "%s status: %d xfer_len: %zu\n", __func__, | |
| 118 | + mhi_result->transaction_status, mhi_result->bytes_xferd); | |
| 119 | + | |
| 120 | + atomic_set(&mqtsdev->buff_in_use, 0); | |
| 121 | +} | |
| 122 | + | |
| 123 | +static void qaic_timesync_dl_xfer_cb(struct mhi_device *mhi_dev, struct mhi_result *mhi_result) | |
| 124 | +{ | |
| 125 | + struct mqts_dev *mqtsdev = dev_get_drvdata(&mhi_dev->dev); | |
| 126 | + | |
| 127 | + dev_err(mqtsdev->dev, "%s no data expected on dl channel\n", __func__); | |
| 128 | +} | |
| 129 | + | |
| 130 | +static void qaic_timesync_timer(struct timer_list *t) | |
| 131 | +{ | |
| 132 | + struct mqts_dev *mqtsdev = from_timer(mqtsdev, t, timer); | |
| 133 | + struct qts_host_time_sync_msg_data *sync_msg; | |
| 134 | + u64 device_qtimer_us; | |
| 135 | + u64 device_qtimer; | |
| 136 | + u64 host_time_us; | |
| 137 | + u64 offset_us; | |
| 138 | + u64 host_sec; | |
| 139 | + int ret; | |
| 140 | + | |
| 141 | + if (atomic_read(&mqtsdev->buff_in_use)) { | |
| 142 | + dev_dbg(mqtsdev->dev, "%s buffer not free, schedule next cycle\n", __func__); | |
| 143 | + goto mod_timer; | |
| 144 | + } | |
| 145 | + atomic_set(&mqtsdev->buff_in_use, 1); | |
| 146 | + | |
| 147 | + sync_msg = mqtsdev->sync_msg; | |
| 148 | + sync_msg->header.signature = cpu_to_le16(QAIC_TIMESYNC_SIGNATURE); | |
| 149 | + sync_msg->header.msg_type = QAIC_TS_SYNC_REQ; | |
| 150 | + /* Read host UTC time and convert to uS*/ | |
| 151 | + host_time_us = div_u64(ktime_get_real_ns(), NSEC_PER_USEC); | |
| 152 | + device_qtimer = read_qtimer(mqtsdev->qtimer_addr); | |
| 153 | + device_qtimer_us = QAIC_CONV_QTIMER_TO_US(device_qtimer); | |
| 154 | + /* Offset between host UTC and device time */ | |
| 155 | + offset_us = host_time_us - device_qtimer_us; | |
| 156 | + | |
| 157 | + host_sec = div_u64(offset_us, USEC_PER_SEC); | |
| 158 | + sync_msg->data.tv_usec = cpu_to_le64(offset_us - host_sec * USEC_PER_SEC); | |
| 159 | + sync_msg->data.tv_sec = cpu_to_le64(host_sec); | |
| 160 | + ret = mhi_queue_buf(mqtsdev->mhi_dev, DMA_TO_DEVICE, sync_msg, sizeof(*sync_msg), MHI_EOT); | |
| 161 | + if (ret && (ret != -EAGAIN)) { | |
| 162 | + dev_err(mqtsdev->dev, "%s unable to queue to mhi:%d\n", __func__, ret); | |
| 163 | + return; | |
| 164 | + } else if (ret == -EAGAIN) { | |
| 165 | + atomic_set(&mqtsdev->buff_in_use, 0); | |
| 166 | + } | |
| 167 | + | |
| 168 | +mod_timer: | |
| 169 | + ret = mod_timer(t, jiffies + msecs_to_jiffies(timesync_delay_ms)); | |
| 170 | + if (ret) | |
| 171 | + dev_err(mqtsdev->dev, "%s mod_timer error:%d\n", __func__, ret); | |
| 172 | +} | |
| 173 | + | |
| 174 | +static int qaic_timesync_probe(struct mhi_device *mhi_dev, const struct mhi_device_id *id) | |
| 175 | +{ | |
| 176 | + struct qaic_device *qdev = pci_get_drvdata(to_pci_dev(mhi_dev->mhi_cntrl->cntrl_dev)); | |
| 177 | + struct mqts_dev *mqtsdev; | |
| 178 | + struct timer_list *timer; | |
| 179 | + int ret; | |
| 180 | + | |
| 181 | + mqtsdev = kzalloc(sizeof(*mqtsdev), GFP_KERNEL); | |
| 182 | + if (!mqtsdev) { | |
| 183 | + ret = -ENOMEM; | |
| 184 | + goto out; | |
| 185 | + } | |
| 186 | + | |
| 187 | + timer = &mqtsdev->timer; | |
| 188 | + mqtsdev->mhi_dev = mhi_dev; | |
| 189 | + mqtsdev->qdev = qdev; | |
| 190 | + mqtsdev->dev = &qdev->pdev->dev; | |
| 191 | + | |
| 192 | + mqtsdev->sync_msg = kzalloc(sizeof(*mqtsdev->sync_msg), GFP_KERNEL); | |
| 193 | + if (!mqtsdev->sync_msg) { | |
| 194 | + ret = -ENOMEM; | |
| 195 | + goto free_mqts_dev; | |
| 196 | + } | |
| 197 | + atomic_set(&mqtsdev->buff_in_use, 0); | |
| 198 | + | |
| 199 | + ret = mhi_prepare_for_transfer(mhi_dev); | |
| 200 | + if (ret) | |
| 201 | + goto free_sync_msg; | |
| 202 | + | |
| 203 | + /* Qtimer register pointer */ | |
| 204 | + mqtsdev->qtimer_addr = qdev->bar_0 + QTIMER_REG_OFFSET; | |
| 205 | + timer_setup(timer, qaic_timesync_timer, 0); | |
| 206 | + timer->expires = jiffies + msecs_to_jiffies(timesync_delay_ms); | |
| 207 | + add_timer(timer); | |
| 208 | + dev_set_drvdata(&mhi_dev->dev, mqtsdev); | |
| 209 | + | |
| 210 | + return 0; | |
| 211 | + | |
| 212 | +free_sync_msg: | |
| 213 | + kfree(mqtsdev->sync_msg); | |
| 214 | +free_mqts_dev: | |
| 215 | + kfree(mqtsdev); | |
| 216 | +out: | |
| 217 | + return ret; | |
| 218 | +}; | |
| 219 | + | |
| 220 | +static void qaic_timesync_remove(struct mhi_device *mhi_dev) | |
| 221 | +{ | |
| 222 | + struct mqts_dev *mqtsdev = dev_get_drvdata(&mhi_dev->dev); | |
| 223 | + | |
| 224 | + del_timer_sync(&mqtsdev->timer); | |
| 225 | + mhi_unprepare_from_transfer(mqtsdev->mhi_dev); | |
| 226 | + kfree(mqtsdev->sync_msg); | |
| 227 | + kfree(mqtsdev); | |
| 228 | +} | |
| 229 | + | |
| 230 | +static const struct mhi_device_id qaic_timesync_match_table[] = { | |
| 231 | + { .chan = "QAIC_TIMESYNC_PERIODIC"}, | |
| 232 | + {}, | |
| 233 | +}; | |
| 234 | + | |
| 235 | +MODULE_DEVICE_TABLE(mhi, qaic_timesync_match_table); | |
| 236 | + | |
| 237 | +static struct mhi_driver qaic_timesync_driver = { | |
| 238 | + .id_table = qaic_timesync_match_table, | |
| 239 | + .remove = qaic_timesync_remove, | |
| 240 | + .probe = qaic_timesync_probe, | |
| 241 | + .ul_xfer_cb = qaic_timesync_ul_xfer_cb, | |
| 242 | + .dl_xfer_cb = qaic_timesync_dl_xfer_cb, | |
| 243 | + .driver = { | |
| 244 | + .name = "qaic_timesync_periodic", | |
| 245 | + }, | |
| 246 | +}; | |
| 247 | + | |
| 248 | +static void qaic_boot_timesync_worker(struct work_struct *work) | |
| 249 | +{ | |
| 250 | + struct qts_resp *resp = container_of(work, struct qts_resp, work); | |
| 251 | + struct qts_host_time_sync_msg_data *req; | |
| 252 | + struct qts_resp_msg data = resp->data; | |
| 253 | + struct qaic_device *qdev = resp->qdev; | |
| 254 | + struct mhi_device *mhi_dev; | |
| 255 | + struct timespec64 ts; | |
| 256 | + int ret; | |
| 257 | + | |
| 258 | + mhi_dev = qdev->qts_ch; | |
| 259 | + /* Queue the response message beforehand to avoid race conditions */ | |
| 260 | + ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE, &resp->data, sizeof(resp->data), MHI_EOT); | |
| 261 | + if (ret) { | |
| 262 | + kfree(resp); | |
| 263 | + dev_warn(&mhi_dev->dev, "Failed to re-queue response buffer %d\n", ret); | |
| 264 | + return; | |
| 265 | + } | |
| 266 | + | |
| 267 | + switch (data.hdr.msg_type) { | |
| 268 | + case QAIC_TS_CMD_TO_HOST: | |
| 269 | + req = kzalloc(sizeof(*req), GFP_KERNEL); | |
| 270 | + if (!req) | |
| 271 | + break; | |
| 272 | + | |
| 273 | + req->header = data.hdr; | |
| 274 | + req->header.msg_type = QAIC_TS_SYNC_REQ; | |
| 275 | + ktime_get_real_ts64(&ts); | |
| 276 | + req->data.tv_sec = cpu_to_le64(ts.tv_sec); | |
| 277 | + req->data.tv_usec = cpu_to_le64(div_u64(ts.tv_nsec, NSEC_PER_USEC)); | |
| 278 | + | |
| 279 | + ret = mhi_queue_buf(mhi_dev, DMA_TO_DEVICE, req, sizeof(*req), MHI_EOT); | |
| 280 | + if (ret) { | |
| 281 | + kfree(req); | |
| 282 | + dev_dbg(&mhi_dev->dev, "Failed to send request message. Error %d\n", ret); | |
| 283 | + } | |
| 284 | + break; | |
| 285 | + case QAIC_TS_ACK_TO_HOST: | |
| 286 | + dev_dbg(&mhi_dev->dev, "ACK received from device\n"); | |
| 287 | + break; | |
| 288 | + default: | |
| 289 | + dev_err(&mhi_dev->dev, "Invalid message type %u.\n", data.hdr.msg_type); | |
| 290 | + } | |
| 291 | +} | |
| 292 | + | |
| 293 | +static int qaic_boot_timesync_queue_resp(struct mhi_device *mhi_dev, struct qaic_device *qdev) | |
| 294 | +{ | |
| 295 | + struct qts_resp *resp; | |
| 296 | + int ret; | |
| 297 | + | |
| 298 | + resp = kzalloc(sizeof(*resp), GFP_KERNEL); | |
| 299 | + if (!resp) | |
| 300 | + return -ENOMEM; | |
| 301 | + | |
| 302 | + resp->qdev = qdev; | |
| 303 | + INIT_WORK(&resp->work, qaic_boot_timesync_worker); | |
| 304 | + | |
| 305 | + ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE, &resp->data, sizeof(resp->data), MHI_EOT); | |
| 306 | + if (ret) { | |
| 307 | + kfree(resp); | |
| 308 | + dev_warn(&mhi_dev->dev, "Failed to queue response buffer %d\n", ret); | |
| 309 | + return ret; | |
| 310 | + } | |
| 311 | + | |
| 312 | + return 0; | |
| 313 | +} | |
| 314 | + | |
| 315 | +static void qaic_boot_timesync_remove(struct mhi_device *mhi_dev) | |
| 316 | +{ | |
| 317 | + struct qaic_device *qdev; | |
| 318 | + | |
| 319 | + qdev = dev_get_drvdata(&mhi_dev->dev); | |
| 320 | + mhi_unprepare_from_transfer(qdev->qts_ch); | |
| 321 | + qdev->qts_ch = NULL; | |
| 322 | +} | |
| 323 | + | |
| 324 | +static int qaic_boot_timesync_probe(struct mhi_device *mhi_dev, const struct mhi_device_id *id) | |
| 325 | +{ | |
| 326 | + struct qaic_device *qdev = pci_get_drvdata(to_pci_dev(mhi_dev->mhi_cntrl->cntrl_dev)); | |
| 327 | + int ret; | |
| 328 | + | |
| 329 | + ret = mhi_prepare_for_transfer(mhi_dev); | |
| 330 | + if (ret) | |
| 331 | + return ret; | |
| 332 | + | |
| 333 | + qdev->qts_ch = mhi_dev; | |
| 334 | + dev_set_drvdata(&mhi_dev->dev, qdev); | |
| 335 | + | |
| 336 | + ret = qaic_boot_timesync_queue_resp(mhi_dev, qdev); | |
| 337 | + if (ret) { | |
| 338 | + dev_set_drvdata(&mhi_dev->dev, NULL); | |
| 339 | + qdev->qts_ch = NULL; | |
| 340 | + mhi_unprepare_from_transfer(mhi_dev); | |
| 341 | + } | |
| 342 | + | |
| 343 | + return ret; | |
| 344 | +} | |
| 345 | + | |
| 346 | +static void qaic_boot_timesync_ul_xfer_cb(struct mhi_device *mhi_dev, struct mhi_result *mhi_result) | |
| 347 | +{ | |
| 348 | + kfree(mhi_result->buf_addr); | |
| 349 | +} | |
| 350 | + | |
| 351 | +static void qaic_boot_timesync_dl_xfer_cb(struct mhi_device *mhi_dev, struct mhi_result *mhi_result) | |
| 352 | +{ | |
| 353 | + struct qts_resp *resp = container_of(mhi_result->buf_addr, struct qts_resp, data); | |
| 354 | + | |
| 355 | + if (mhi_result->transaction_status || mhi_result->bytes_xferd != sizeof(resp->data)) { | |
| 356 | + kfree(resp); | |
| 357 | + return; | |
| 358 | + } | |
| 359 | + | |
| 360 | + queue_work(resp->qdev->qts_wq, &resp->work); | |
| 361 | +} | |
| 362 | + | |
| 363 | +static const struct mhi_device_id qaic_boot_timesync_match_table[] = { | |
| 364 | + { .chan = "QAIC_TIMESYNC"}, | |
| 365 | + {}, | |
| 366 | +}; | |
| 367 | + | |
| 368 | +static struct mhi_driver qaic_boot_timesync_driver = { | |
| 369 | + .id_table = qaic_boot_timesync_match_table, | |
| 370 | + .remove = qaic_boot_timesync_remove, | |
| 371 | + .probe = qaic_boot_timesync_probe, | |
| 372 | + .ul_xfer_cb = qaic_boot_timesync_ul_xfer_cb, | |
| 373 | + .dl_xfer_cb = qaic_boot_timesync_dl_xfer_cb, | |
| 374 | + .driver = { | |
| 375 | + .name = "qaic_timesync", | |
| 376 | + }, | |
| 377 | +}; | |
| 378 | + | |
| 379 | +int qaic_timesync_init(void) | |
| 380 | +{ | |
| 381 | + int ret; | |
| 382 | + | |
| 383 | + ret = mhi_driver_register(&qaic_timesync_driver); | |
| 384 | + if (ret) | |
| 385 | + return ret; | |
| 386 | + ret = mhi_driver_register(&qaic_boot_timesync_driver); | |
| 387 | + | |
| 388 | + return ret; | |
| 389 | +} | |
| 390 | + | |
| 391 | +void qaic_timesync_deinit(void) | |
| 392 | +{ | |
| 393 | + mhi_driver_unregister(&qaic_boot_timesync_driver); | |
| 394 | + mhi_driver_unregister(&qaic_timesync_driver); | |
| 395 | +} | |
| @@ -0,0 +1,11 @@ | ||
| 1 | +/* SPDX-License-Identifier: GPL-2.0-only | |
| 2 | + * | |
| 3 | + * Copyright (c) 2023 Qualcomm Innovation Center, Inc. All rights reserved. | |
| 4 | + */ | |
| 5 | + | |
| 6 | +#ifndef __QAIC_TIMESYNC_H__ | |
| 7 | +#define __QAIC_TIMESYNC_H__ | |
| 8 | + | |
| 9 | +int qaic_timesync_init(void); | |
| 10 | +void qaic_timesync_deinit(void); | |
| 11 | +#endif /* __QAIC_TIMESYNC_H__ */ | |
| @@ -2031,7 +2031,7 @@ static int acpi_video_bus_add(struct acpi_device *device) · 2030 unmodified lines | ||
| 2031 | 2031 | * HP ZBook Fury 16 G10 requires ACPI video's child devices have _PS0 |
| 2032 | 2032 | * evaluated to have functional panel brightness control. |
| 2033 | 2033 | */ |
| 2034 | − acpi_device_fix_up_power_extended(device); | |
| 2034 | + acpi_device_fix_up_power_children(device); | |
| 2035 | 2035 | |
| 2036 | 2036 | pr_info("%s [%s] (multi-head: %s rom: %s post: %s)\n", |
| 2037 | 2037 | ACPI_VIDEO_DEVICE_NAME, acpi_device_bid(device), |
| @@ -397,6 +397,19 @@ void acpi_device_fix_up_power_extended(struct acpi_device *adev) · 396 unmodified lines | ||
| 397 | 397 | } |
| 398 | 398 | EXPORT_SYMBOL_GPL(acpi_device_fix_up_power_extended); |
| 399 | 399 | |
| 400 | +/** | |
| 401 | + * acpi_device_fix_up_power_children - Force a device's children into D0. | |
| 402 | + * @adev: Parent device object whose children's power state is to be fixed up. | |
| 403 | + * | |
| 404 | + * Call acpi_device_fix_up_power() for @adev's children so long as they | |
| 405 | + * are reported as present and enabled. | |
| 406 | + */ | |
| 407 | +void acpi_device_fix_up_power_children(struct acpi_device *adev) | |
| 408 | +{ | |
| 409 | + acpi_dev_for_each_child(adev, fix_up_power_if_applicable, NULL); | |
| 410 | +} | |
| 411 | +EXPORT_SYMBOL_GPL(acpi_device_fix_up_power_children); | |
| 412 | + | |
| 400 | 413 | int acpi_device_update_power(struct acpi_device *device, int *state_p) |
| 401 | 414 | { |
| 402 | 415 | int state; |
| @@ -592,7 +592,7 @@ static int acpi_idle_play_dead(struct cpuidle_device *dev, int index) · 591 unmodified lines | ||
| 592 | 592 | while (1) { |
| 593 | 593 | |
| 594 | 594 | if (cx->entry_method == ACPI_CSTATE_HALT) |
| 595 | − safe_halt(); | |
| 595 | + raw_safe_halt(); | |
| 596 | 596 | else if (cx->entry_method == ACPI_CSTATE_SYSTEMIO) { |
| 597 | 597 | io_idle(cx->address); |
| 598 | 598 | } else |
| @@ -447,6 +447,13 @@ static const struct dmi_system_id irq1_level_low_skip_override[] = { · 446 unmodified lines | ||
| 447 | 447 | DMI_MATCH(DMI_BOARD_NAME, "B1402CBA"), |
| 448 | 448 | }, |
| 449 | 449 | }, |
| 450 | + { | |
| 451 | + /* Asus ExpertBook B1402CVA */ | |
| 452 | + .matches = { | |
| 453 | + DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."), | |
| 454 | + DMI_MATCH(DMI_BOARD_NAME, "B1402CVA"), | |
| 455 | + }, | |
| 456 | + }, | |
| 450 | 457 | { |
| 451 | 458 | /* Asus ExpertBook B1502CBA */ |
| 452 | 459 | .matches = { |
| @@ -82,6 +82,9 @@ static int isapnp_init_one(struct pnp_dev *idev, const struct pnp_device_id *dev · 81 unmodified lines | ||
| 82 | 82 | if (pnp_port_valid(idev, 1)) { |
| 83 | 83 | ctl_addr = devm_ioport_map(&idev->dev, |
| 84 | 84 | pnp_port_start(idev, 1), 1); |
| 85 | + if (!ctl_addr) | |
| 86 | + return -ENOMEM; | |
| 87 | + | |
| 85 | 88 | ap->ioaddr.altstatus_addr = ctl_addr; |
| 86 | 89 | ap->ioaddr.ctl_addr = ctl_addr; |
| 87 | 90 | ap->ops = &isapnp_port_ops; |
| @@ -112,10 +112,7 @@ config CFAG12864B · 111 unmodified lines | ||
| 112 | 112 | depends on X86 |
| 113 | 113 | depends on FB |
| 114 | 114 | depends on KS0108 |
| 115 | − select FB_SYS_FILLRECT | |
| 116 | − select FB_SYS_COPYAREA | |
| 117 | − select FB_SYS_IMAGEBLIT | |
| 118 | − select FB_SYS_FOPS | |
| 115 | + select FB_SYSMEM_HELPERS | |
| 119 | 116 | default n |
| 120 | 117 | help |
| 121 | 118 | If you have a Crystalfontz 128x64 2-color LCD, cfag12864b Series, |
| @@ -170,10 +167,7 @@ config IMG_ASCII_LCD · 48 unmodified lines | ||
| 170 | 167 | config HT16K33 |
| 171 | 168 | tristate "Holtek Ht16K33 LED controller with keyscan" |
| 172 | 169 | depends on FB && I2C && INPUT |
| 173 | − select FB_SYS_FOPS | |
| 174 | − select FB_SYS_FILLRECT | |
| 175 | − select FB_SYS_COPYAREA | |
| 176 | − select FB_SYS_IMAGEBLIT | |
| 170 | + select FB_SYSMEM_HELPERS | |
| 177 | 171 | select INPUT_MATRIXKMAP |
| 178 | 172 | select FB_BACKLIGHT |
| 179 | 173 | select NEW_LEDS |
| @@ -51,16 +51,15 @@ static int cfag12864bfb_mmap(struct fb_info *info, struct vm_area_struct *vma) · 50 unmodified lines | ||
| 51 | 51 | { |
| 52 | 52 | struct page *pages = virt_to_page(cfag12864b_buffer); |
| 53 | 53 | |
| 54 | + vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot); | |
| 55 | + | |
| 54 | 56 | return vm_map_pages_zero(vma, &pages, 1); |
| 55 | 57 | } |
| 56 | 58 | |
| 57 | 59 | static const struct fb_ops cfag12864bfb_ops = { |
| 58 | 60 | .owner = THIS_MODULE, |
| 59 | − .fb_read = fb_sys_read, | |
| 60 | − .fb_write = fb_sys_write, | |
| 61 | − .fb_fillrect = sys_fillrect, | |
| 62 | − .fb_copyarea = sys_copyarea, | |
| 63 | − .fb_imageblit = sys_imageblit, | |
| 61 | + __FB_DEFAULT_SYSMEM_OPS_RDWR, | |
| 62 | + __FB_DEFAULT_SYSMEM_OPS_DRAW, | |
| 64 | 63 | .fb_mmap = cfag12864bfb_mmap, |
| 65 | 64 | }; |
| 66 | 65 | |
| @@ -72,6 +71,7 @@ static int cfag12864bfb_probe(struct platform_device *device) · 5 unmodified lines | ||
| 72 | 71 | if (!info) |
| 73 | 72 | goto none; |
| 74 | 73 | |
| 74 | + info->flags = FBINFO_VIRTFB; | |
| 75 | 75 | info->screen_buffer = cfag12864b_buffer; |
| 76 | 76 | info->screen_size = CFAG12864B_SIZE; |
| 77 | 77 | info->fbops = &cfag12864bfb_ops; |
| @@ -351,17 +351,16 @@ static int ht16k33_mmap(struct fb_info *info, struct vm_area_struct *vma) · 350 unmodified lines | ||
| 351 | 351 | struct ht16k33_priv *priv = info->par; |
| 352 | 352 | struct page *pages = virt_to_page(priv->fbdev.buffer); |
| 353 | 353 | |
| 354 | + vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot); | |
| 355 | + | |
| 354 | 356 | return vm_map_pages_zero(vma, &pages, 1); |
| 355 | 357 | } |
| 356 | 358 | |
| 357 | 359 | static const struct fb_ops ht16k33_fb_ops = { |
| 358 | 360 | .owner = THIS_MODULE, |
| 359 | − .fb_read = fb_sys_read, | |
| 360 | − .fb_write = fb_sys_write, | |
| 361 | + __FB_DEFAULT_SYSMEM_OPS_RDWR, | |
| 361 | 362 | .fb_blank = ht16k33_blank, |
| 362 | − .fb_fillrect = sys_fillrect, | |
| 363 | − .fb_copyarea = sys_copyarea, | |
| 364 | − .fb_imageblit = sys_imageblit, | |
| 363 | + __FB_DEFAULT_SYSMEM_OPS_DRAW, | |
| 365 | 364 | .fb_mmap = ht16k33_mmap, |
| 366 | 365 | }; |
| 367 | 366 | |
| @@ -640,6 +639,7 @@ static int ht16k33_fbdev_probe(struct device *dev, struct ht16k33_priv *priv, · 272 unmodified lines | ||
| 640 | 639 | |
| 641 | 640 | INIT_DELAYED_WORK(&priv->work, ht16k33_fb_update); |
| 642 | 641 | fbdev->info->fbops = &ht16k33_fb_ops; |
| 642 | + fbdev->info->flags |= FBINFO_VIRTFB; | |
| 643 | 643 | fbdev->info->screen_buffer = fbdev->buffer; |
| 644 | 644 | fbdev->info->screen_size = HT16K33_FB_SIZE; |
| 645 | 645 | fbdev->info->fix = ht16k33_fb_fix; |
| @@ -67,6 +67,7 @@ struct nbd_sock { · 66 unmodified lines | ||
| 67 | 67 | struct recv_thread_args { |
| 68 | 68 | struct work_struct work; |
| 69 | 69 | struct nbd_device *nbd; |
| 70 | + struct nbd_sock *nsock; | |
| 70 | 71 | int index; |
| 71 | 72 | }; |
| 72 | 73 | |
| @@ -395,6 +396,22 @@ static u32 req_to_nbd_cmd_type(struct request *req) · 322 unmodified lines | ||
| 395 | 396 | } |
| 396 | 397 | } |
| 397 | 398 | |
| 399 | +static struct nbd_config *nbd_get_config_unlocked(struct nbd_device *nbd) | |
| 400 | +{ | |
| 401 | + if (refcount_inc_not_zero(&nbd->config_refs)) { | |
| 402 | + /* | |
| 403 | + * Add smp_mb__after_atomic to ensure that reading nbd->config_refs | |
| 404 | + * and reading nbd->config is ordered. The pair is the barrier in | |
| 405 | + * nbd_alloc_and_init_config(), avoid nbd->config_refs is set | |
| 406 | + * before nbd->config. | |
| 407 | + */ | |
| 408 | + smp_mb__after_atomic(); | |
| 409 | + return nbd->config; | |
| 410 | + } | |
| 411 | + | |
| 412 | + return NULL; | |
| 413 | +} | |
| 414 | + | |
| 398 | 415 | static enum blk_eh_timer_return nbd_xmit_timeout(struct request *req) |
| 399 | 416 | { |
| 400 | 417 | struct nbd_cmd *cmd = blk_mq_rq_to_pdu(req); |
| @@ -409,13 +426,13 @@ static enum blk_eh_timer_return nbd_xmit_timeout(struct request *req) · 8 unmodified lines | ||
| 409 | 426 | return BLK_EH_DONE; |
| 410 | 427 | } |
| 411 | 428 | |
| 412 | − if (!refcount_inc_not_zero(&nbd->config_refs)) { | |
| 429 | + config = nbd_get_config_unlocked(nbd); | |
| 430 | + if (!config) { | |
| 413 | 431 | cmd->status = BLK_STS_TIMEOUT; |
| 414 | 432 | __clear_bit(NBD_CMD_INFLIGHT, &cmd->flags); |
| 415 | 433 | mutex_unlock(&cmd->lock); |
| 416 | 434 | goto done; |
| 417 | 435 | } |
| 418 | − config = nbd->config; | |
| 419 | 436 | |
| 420 | 437 | if (config->num_connections > 1 || |
| 421 | 438 | (config->num_connections == 1 && nbd->tag_set.timeout)) { |
| @@ -489,15 +506,9 @@ done: · 67 unmodified lines | ||
| 489 | 506 | return BLK_EH_DONE; |
| 490 | 507 | } |
| 491 | 508 | |
| 492 | −/* | |
| 493 | − * Send or receive packet. Return a positive value on success and | |
| 494 | − * negtive value on failue, and never return 0. | |
| 495 | − */ | |
| 496 | −static int sock_xmit(struct nbd_device *nbd, int index, int send, | |
| 497 | − struct iov_iter *iter, int msg_flags, int *sent) | |
| 509 | +static int __sock_xmit(struct nbd_device *nbd, struct socket *sock, int send, | |
| 510 | + struct iov_iter *iter, int msg_flags, int *sent) | |
| 498 | 511 | { |
| 499 | − struct nbd_config *config = nbd->config; | |
| 500 | − struct socket *sock = config->socks[index]->sock; | |
| 501 | 512 | int result; |
| 502 | 513 | struct msghdr msg; |
| 503 | 514 | unsigned int noreclaim_flag; |
| @@ -540,6 +551,19 @@ static int sock_xmit(struct nbd_device *nbd, int index, int send, · 36 unmodified lines | ||
| 540 | 551 | return result; |
| 541 | 552 | } |
| 542 | 553 | |
| 554 | +/* | |
| 555 | + * Send or receive packet. Return a positive value on success and | |
| 556 | + * negtive value on failure, and never return 0. | |
| 557 | + */ | |
| 558 | +static int sock_xmit(struct nbd_device *nbd, int index, int send, | |
| 559 | + struct iov_iter *iter, int msg_flags, int *sent) | |
| 560 | +{ | |
| 561 | + struct nbd_config *config = nbd->config; | |
| 562 | + struct socket *sock = config->socks[index]->sock; | |
| 563 | + | |
| 564 | + return __sock_xmit(nbd, sock, send, iter, msg_flags, sent); | |
| 565 | +} | |
| 566 | + | |
| 543 | 567 | /* |
| 544 | 568 | * Different settings for sk->sk_sndtimeo can result in different return values |
| 545 | 569 | * if there is a signal pending when we enter sendmsg, because reasons? |
| @@ -696,7 +720,7 @@ out: · 150 unmodified lines | ||
| 696 | 720 | return 0; |
| 697 | 721 | } |
| 698 | 722 | |
| 699 | −static int nbd_read_reply(struct nbd_device *nbd, int index, | |
| 723 | +static int nbd_read_reply(struct nbd_device *nbd, struct socket *sock, | |
| 700 | 724 | struct nbd_reply *reply) |
| 701 | 725 | { |
| 702 | 726 | struct kvec iov = {.iov_base = reply, .iov_len = sizeof(*reply)}; |
| @@ -705,7 +729,7 @@ static int nbd_read_reply(struct nbd_device *nbd, int index, · 2 unmodified lines | ||
| 705 | 729 | |
| 706 | 730 | reply->magic = 0; |
| 707 | 731 | iov_iter_kvec(&to, ITER_DEST, &iov, 1, sizeof(*reply)); |
| 708 | − result = sock_xmit(nbd, index, 0, &to, MSG_WAITALL, NULL); | |
| 732 | + result = __sock_xmit(nbd, sock, 0, &to, MSG_WAITALL, NULL); | |
| 709 | 733 | if (result < 0) { |
| 710 | 734 | if (!nbd_disconnected(nbd->config)) |
| 711 | 735 | dev_err(disk_to_dev(nbd->disk), |
| @@ -829,14 +853,14 @@ static void recv_work(struct work_struct *work) · 117 unmodified lines | ||
| 829 | 853 | struct nbd_device *nbd = args->nbd; |
| 830 | 854 | struct nbd_config *config = nbd->config; |
| 831 | 855 | struct request_queue *q = nbd->disk->queue; |
| 832 | − struct nbd_sock *nsock; | |
| 856 | + struct nbd_sock *nsock = args->nsock; | |
| 833 | 857 | struct nbd_cmd *cmd; |
| 834 | 858 | struct request *rq; |
| 835 | 859 | |
| 836 | 860 | while (1) { |
| 837 | 861 | struct nbd_reply reply; |
| 838 | 862 | |
| 839 | − if (nbd_read_reply(nbd, args->index, &reply)) | |
| 863 | + if (nbd_read_reply(nbd, nsock->sock, &reply)) | |
| 840 | 864 | break; |
| 841 | 865 | |
| 842 | 866 | /* |
| @@ -871,7 +895,6 @@ static void recv_work(struct work_struct *work) · 28 unmodified lines | ||
| 871 | 895 | percpu_ref_put(&q->q_usage_counter); |
| 872 | 896 | } |
| 873 | 897 | |
| 874 | − nsock = config->socks[args->index]; | |
| 875 | 898 | mutex_lock(&nsock->tx_lock); |
| 876 | 899 | nbd_mark_nsock_dead(nbd, nsock, 1); |
| 877 | 900 | mutex_unlock(&nsock->tx_lock); |
| @@ -977,12 +1000,12 @@ static int nbd_handle_cmd(struct nbd_cmd *cmd, int index) · 99 unmodified lines | ||
| 977 | 1000 | struct nbd_sock *nsock; |
| 978 | 1001 | int ret; |
| 979 | 1002 | |
| 980 | − if (!refcount_inc_not_zero(&nbd->config_refs)) { | |
| 1003 | + config = nbd_get_config_unlocked(nbd); | |
| 1004 | + if (!config) { | |
| 981 | 1005 | dev_err_ratelimited(disk_to_dev(nbd->disk), |
| 982 | 1006 | "Socks array is empty\n"); |
| 983 | 1007 | return -EINVAL; |
| 984 | 1008 | } |
| 985 | − config = nbd->config; | |
| 986 | 1009 | |
| 987 | 1010 | if (index >= config->num_connections) { |
| 988 | 1011 | dev_err_ratelimited(disk_to_dev(nbd->disk), |
| @@ -1215,6 +1238,7 @@ static int nbd_reconnect_socket(struct nbd_device *nbd, unsigned long arg) · 226 unmodified lines | ||
| 1215 | 1238 | INIT_WORK(&args->work, recv_work); |
| 1216 | 1239 | args->index = i; |
| 1217 | 1240 | args->nbd = nbd; |
| 1241 | + args->nsock = nsock; | |
| 1218 | 1242 | nsock->cookie++; |
| 1219 | 1243 | mutex_unlock(&nsock->tx_lock); |
| 1220 | 1244 | sockfd_put(old); |
| @@ -1397,6 +1421,7 @@ static int nbd_start_device(struct nbd_device *nbd) · 176 unmodified lines | ||
| 1397 | 1421 | refcount_inc(&nbd->config_refs); |
| 1398 | 1422 | INIT_WORK(&args->work, recv_work); |
| 1399 | 1423 | args->nbd = nbd; |
| 1424 | + args->nsock = config->socks[i]; | |
| 1400 | 1425 | args->index = i; |
| 1401 | 1426 | queue_work(nbd->recv_workq, &args->work); |
| 1402 | 1427 | } |
| @@ -1530,17 +1555,20 @@ static int nbd_ioctl(struct block_device *bdev, blk_mode_t mode, · 127 unmodified lines | ||
| 1530 | 1555 | return error; |
| 1531 | 1556 | } |
| 1532 | 1557 | |
| 1533 | −static struct nbd_config *nbd_alloc_config(void) | |
| 1558 | +static int nbd_alloc_and_init_config(struct nbd_device *nbd) | |
| 1534 | 1559 | { |
| 1535 | 1560 | struct nbd_config *config; |
| 1536 | 1561 | |
| 1562 | + if (WARN_ON(nbd->config)) | |
| 1563 | + return -EINVAL; | |
| 1564 | + | |
| 1537 | 1565 | if (!try_module_get(THIS_MODULE)) |
| 1538 | − return ERR_PTR(-ENODEV); | |
| 1566 | + return -ENODEV; | |
| 1539 | 1567 | |
| 1540 | 1568 | config = kzalloc(sizeof(struct nbd_config), GFP_NOFS); |
| 1541 | 1569 | if (!config) { |
| 1542 | 1570 | module_put(THIS_MODULE); |
| 1543 | − return ERR_PTR(-ENOMEM); | |
| 1571 | + return -ENOMEM; | |
| 1544 | 1572 | } |
| 1545 | 1573 | |
| 1546 | 1574 | atomic_set(&config->recv_threads, 0); |
| @@ -1548,12 +1576,24 @@ static struct nbd_config *nbd_alloc_config(void) · 1 unmodified line | ||
| 1548 | 1576 | init_waitqueue_head(&config->conn_wait); |
| 1549 | 1577 | config->blksize_bits = NBD_DEF_BLKSIZE_BITS; |
| 1550 | 1578 | atomic_set(&config->live_connections, 0); |
| 1551 | − return config; | |
| 1579 | + | |
| 1580 | + nbd->config = config; | |
| 1581 | + /* | |
| 1582 | + * Order refcount_set(&nbd->config_refs, 1) and nbd->config assignment, | |
| 1583 | + * its pair is the barrier in nbd_get_config_unlocked(). | |
| 1584 | + * So nbd_get_config_unlocked() won't see nbd->config as null after | |
| 1585 | + * refcount_inc_not_zero() succeed. | |
| 1586 | + */ | |
| 1587 | + smp_mb__before_atomic(); | |
| 1588 | + refcount_set(&nbd->config_refs, 1); | |
| 1589 | + | |
| 1590 | + return 0; | |
| 1552 | 1591 | } |
| 1553 | 1592 | |
| 1554 | 1593 | static int nbd_open(struct gendisk *disk, blk_mode_t mode) |
| 1555 | 1594 | { |
| 1556 | 1595 | struct nbd_device *nbd; |
| 1596 | + struct nbd_config *config; | |
| 1557 | 1597 | int ret = 0; |
| 1558 | 1598 | |
| 1559 | 1599 | mutex_lock(&nbd_index_mutex); |
| @@ -1566,27 +1606,25 @@ static int nbd_open(struct gendisk *disk, blk_mode_t mode) · 6 unmodified lines | ||
| 1566 | 1606 | ret = -ENXIO; |
| 1567 | 1607 | goto out; |
| 1568 | 1608 | } |
| 1569 | − if (!refcount_inc_not_zero(&nbd->config_refs)) { | |
| 1570 | − struct nbd_config *config; | |
| 1571 | 1609 | |
| 1610 | + config = nbd_get_config_unlocked(nbd); | |
| 1611 | + if (!config) { | |
| 1572 | 1612 | mutex_lock(&nbd->config_lock); |
| 1573 | 1613 | if (refcount_inc_not_zero(&nbd->config_refs)) { |
| 1574 | 1614 | mutex_unlock(&nbd->config_lock); |
| 1575 | 1615 | goto out; |
| 1576 | 1616 | } |
| 1577 | − config = nbd_alloc_config(); | |
| 1578 | − if (IS_ERR(config)) { | |
| 1579 | − ret = PTR_ERR(config); | |
| 1617 | + ret = nbd_alloc_and_init_config(nbd); | |
| 1618 | + if (ret) { | |
| 1580 | 1619 | mutex_unlock(&nbd->config_lock); |
| 1581 | 1620 | goto out; |
| 1582 | 1621 | } |
| 1583 | − nbd->config = config; | |
| 1584 | − refcount_set(&nbd->config_refs, 1); | |
| 1622 | + | |
| 1585 | 1623 | refcount_inc(&nbd->refs); |
| 1586 | 1624 | mutex_unlock(&nbd->config_lock); |
| 1587 | 1625 | if (max_part) |
| 1588 | 1626 | set_bit(GD_NEED_PART_SCAN, &disk->state); |
| 1589 | − } else if (nbd_disconnected(nbd->config)) { | |
| 1627 | + } else if (nbd_disconnected(config)) { | |
| 1590 | 1628 | if (max_part) |
| 1591 | 1629 | set_bit(GD_NEED_PART_SCAN, &disk->state); |
| 1592 | 1630 | } |
| @@ -1990,22 +2028,17 @@ again: · 397 unmodified lines | ||
| 1990 | 2028 | pr_err("nbd%d already in use\n", index); |
| 1991 | 2029 | return -EBUSY; |
| 1992 | 2030 | } |
| 1993 | − if (WARN_ON(nbd->config)) { | |
| 1994 | − mutex_unlock(&nbd->config_lock); | |
| 1995 | − nbd_put(nbd); | |
| 1996 | − return -EINVAL; | |
| 1997 | − } | |
| 1998 | − config = nbd_alloc_config(); | |
| 1999 | − if (IS_ERR(config)) { | |
| 2031 | + | |
| 2032 | + ret = nbd_alloc_and_init_config(nbd); | |
| 2033 | + if (ret) { | |
| 2000 | 2034 | mutex_unlock(&nbd->config_lock); |
| 2001 | 2035 | nbd_put(nbd); |
| 2002 | 2036 | pr_err("couldn't allocate config\n"); |
| 2003 | − return PTR_ERR(config); | |
| 2037 | + return ret; | |
| 2004 | 2038 | } |
| 2005 | − nbd->config = config; | |
| 2006 | − refcount_set(&nbd->config_refs, 1); | |
| 2007 | − set_bit(NBD_RT_BOUND, &config->runtime_flags); | |
| 2008 | 2039 | |
| 2040 | + config = nbd->config; | |
| 2041 | + set_bit(NBD_RT_BOUND, &config->runtime_flags); | |
| 2009 | 2042 | ret = nbd_genl_size_set(info, nbd); |
| 2010 | 2043 | if (ret) |
| 2011 | 2044 | goto out; |
| @@ -2208,7 +2241,8 @@ static int nbd_genl_reconfigure(struct sk_buff *skb, struct genl_info *info) · 196 unmodified lines | ||
| 2208 | 2241 | } |
| 2209 | 2242 | mutex_unlock(&nbd_index_mutex); |
| 2210 | 2243 | |
| 2211 | − if (!refcount_inc_not_zero(&nbd->config_refs)) { | |
| 2244 | + config = nbd_get_config_unlocked(nbd); | |
| 2245 | + if (!config) { | |
| 2212 | 2246 | dev_err(nbd_to_dev(nbd), |
| 2213 | 2247 | "not configured, cannot reconfigure\n"); |
| 2214 | 2248 | nbd_put(nbd); |
| @@ -2216,7 +2250,6 @@ static int nbd_genl_reconfigure(struct sk_buff *skb, struct genl_info *info) · 1 unmodified line | ||
| 2216 | 2250 | } |
| 2217 | 2251 | |
| 2218 | 2252 | mutex_lock(&nbd->config_lock); |
| 2219 | − config = nbd->config; | |
| 2220 | 2253 | if (!test_bit(NBD_RT_BOUND, &config->runtime_flags) || |
| 2221 | 2254 | !nbd->pid) { |
| 2222 | 2255 | dev_err(nbd_to_dev(nbd), |
| @@ -1464,19 +1464,13 @@ blk_status_t null_process_cmd(struct nullb_cmd *cmd, enum req_op op, · 1463 unmodified lines | ||
| 1464 | 1464 | return BLK_STS_OK; |
| 1465 | 1465 | } |
| 1466 | 1466 | |
| 1467 | −static blk_status_t null_handle_cmd(struct nullb_cmd *cmd, sector_t sector, | |
| 1468 | − sector_t nr_sectors, enum req_op op) | |
| 1467 | +static void null_handle_cmd(struct nullb_cmd *cmd, sector_t sector, | |
| 1468 | + sector_t nr_sectors, enum req_op op) | |
| 1469 | 1469 | { |
| 1470 | 1470 | struct nullb_device *dev = cmd->nq->dev; |
| 1471 | 1471 | struct nullb *nullb = dev->nullb; |
| 1472 | 1472 | blk_status_t sts; |
| 1473 | 1473 | |
| 1474 | − if (test_bit(NULLB_DEV_FL_THROTTLED, &dev->flags)) { | |
| 1475 | − sts = null_handle_throttled(cmd); | |
| 1476 | − if (sts != BLK_STS_OK) | |
| 1477 | − return sts; | |
| 1478 | − } | |
| 1479 | − | |
| 1480 | 1474 | if (op == REQ_OP_FLUSH) { |
| 1481 | 1475 | cmd->error = errno_to_blk_status(null_handle_flush(nullb)); |
| 1482 | 1476 | goto out; |
| @@ -1493,7 +1487,6 @@ static blk_status_t null_handle_cmd(struct nullb_cmd *cmd, sector_t sector, · 10 unmodified lines | ||
| 1493 | 1487 | |
| 1494 | 1488 | out: |
| 1495 | 1489 | nullb_complete_cmd(cmd); |
| 1496 | − return BLK_STS_OK; | |
| 1497 | 1490 | } |
| 1498 | 1491 | |
| 1499 | 1492 | static enum hrtimer_restart nullb_bwtimer_fn(struct hrtimer *timer) |
| @@ -1724,8 +1717,6 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx, · 224 unmodified lines | ||
| 1724 | 1717 | cmd->fake_timeout = should_timeout_request(rq) || |
| 1725 | 1718 | blk_should_fake_timeout(rq->q); |
| 1726 | 1719 | |
| 1727 | − blk_mq_start_request(rq); | |
| 1728 | − | |
| 1729 | 1720 | if (should_requeue_request(rq)) { |
| 1730 | 1721 | /* |
| 1731 | 1722 | * Alternate between hitting the core BUSY path, and the |
| @@ -1738,6 +1729,15 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx, · 6 unmodified lines | ||
| 1738 | 1729 | return BLK_STS_OK; |
| 1739 | 1730 | } |
| 1740 | 1731 | |
| 1732 | + if (test_bit(NULLB_DEV_FL_THROTTLED, &nq->dev->flags)) { | |
| 1733 | + blk_status_t sts = null_handle_throttled(cmd); | |
| 1734 | + | |
| 1735 | + if (sts != BLK_STS_OK) | |
| 1736 | + return sts; | |
| 1737 | + } | |
| 1738 | + | |
| 1739 | + blk_mq_start_request(rq); | |
| 1740 | + | |
| 1741 | 1741 | if (is_poll) { |
| 1742 | 1742 | spin_lock(&nq->poll_lock); |
| 1743 | 1743 | list_add_tail(&rq->queuelist, &nq->poll_list); |
| @@ -1747,7 +1747,8 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx, · 3 unmodified lines | ||
| 1747 | 1747 | if (cmd->fake_timeout) |
| 1748 | 1748 | return BLK_STS_OK; |
| 1749 | 1749 | |
| 1750 | − return null_handle_cmd(cmd, sector, nr_sectors, req_op(rq)); | |
| 1750 | + null_handle_cmd(cmd, sector, nr_sectors, req_op(rq)); | |
| 1751 | + return BLK_STS_OK; | |
| 1751 | 1752 | } |
| 1752 | 1753 | |
| 1753 | 1754 | static void null_queue_rqs(struct request **rqlist) |
| @@ -1,12 +1,6 @@ | ||
| 1 | 1 | # SPDX-License-Identifier: GPL-2.0 |
| 2 | 2 | agpgart-y := backend.o generic.o isoch.o |
| 3 | 3 | |
| 4 | −ifeq ($(CONFIG_DRM_LEGACY),y) | |
| 5 | −agpgart-$(CONFIG_COMPAT) += compat_ioctl.o | |
| 6 | −agpgart-y += frontend.o | |
| 7 | −endif | |
| 8 | − | |
| 9 | − | |
| 10 | 4 | obj-$(CONFIG_AGP) += agpgart.o |
| 11 | 5 | obj-$(CONFIG_AGP_ALI) += ali-agp.o |
| 12 | 6 | obj-$(CONFIG_AGP_ATI) += ati-agp.o |
| @@ -185,15 +185,6 @@ void agp_put_bridge(struct agp_bridge_data *bridge); · 184 unmodified lines | ||
| 185 | 185 | int agp_add_bridge(struct agp_bridge_data *bridge); |
| 186 | 186 | void agp_remove_bridge(struct agp_bridge_data *bridge); |
| 187 | 187 | |
| 188 | −/* Frontend routines. */ | |
| 189 | −#if IS_ENABLED(CONFIG_DRM_LEGACY) | |
| 190 | −int agp_frontend_initialize(void); | |
| 191 | −void agp_frontend_cleanup(void); | |
| 192 | −#else | |
| 193 | −static inline int agp_frontend_initialize(void) { return 0; } | |
| 194 | −static inline void agp_frontend_cleanup(void) {} | |
| 195 | −#endif | |
| 196 | − | |
| 197 | 188 | /* Generic routines. */ |
| 198 | 189 | void agp_generic_enable(struct agp_bridge_data *bridge, u32 mode); |
| 199 | 190 | int agp_generic_create_gatt_table(struct agp_bridge_data *bridge); |
| @@ -293,13 +293,6 @@ int agp_add_bridge(struct agp_bridge_data *bridge) · 292 unmodified lines | ||
| 293 | 293 | } |
| 294 | 294 | |
| 295 | 295 | if (list_empty(&agp_bridges)) { |
| 296 | − error = agp_frontend_initialize(); | |
| 297 | − if (error) { | |
| 298 | − dev_info(&bridge->dev->dev, | |
| 299 | − "agp_frontend_initialize() failed\n"); | |
| 300 | − goto frontend_err; | |
| 301 | − } | |
| 302 | − | |
| 303 | 296 | dev_info(&bridge->dev->dev, "AGP aperture is %dM @ 0x%lx\n", |
| 304 | 297 | bridge->driver->fetch_size(), bridge->gart_bus_addr); |
| 305 | 298 | |
| @@ -308,8 +301,6 @@ int agp_add_bridge(struct agp_bridge_data *bridge) · 2 unmodified lines | ||
| 308 | 301 | list_add(&bridge->list, &agp_bridges); |
| 309 | 302 | return 0; |
| 310 | 303 | |
| 311 | −frontend_err: | |
| 312 | − agp_backend_cleanup(bridge); | |
| 313 | 304 | err_out: |
| 314 | 305 | module_put(bridge->driver->owner); |
| 315 | 306 | err_put_bridge: |
| @@ -323,8 +314,6 @@ void agp_remove_bridge(struct agp_bridge_data *bridge) · 7 unmodified lines | ||
| 323 | 314 | { |
| 324 | 315 | agp_backend_cleanup(bridge); |
| 325 | 316 | list_del(&bridge->list); |
| 326 | − if (list_empty(&agp_bridges)) | |
| 327 | − agp_frontend_cleanup(); | |
| 328 | 317 | module_put(bridge->driver->owner); |
| 329 | 318 | } |
| 330 | 319 | EXPORT_SYMBOL_GPL(agp_remove_bridge); |
| @@ -1,291 +0,0 @@ | ||
| 1 | −/* | |
| 2 | − * AGPGART driver frontend compatibility ioctls | |
| 3 | − * Copyright (C) 2004 Silicon Graphics, Inc. | |
| 4 | − * Copyright (C) 2002-2003 Dave Jones | |
| 5 | − * Copyright (C) 1999 Jeff Hartmann | |
| 6 | − * Copyright (C) 1999 Precision Insight, Inc. | |
| 7 | − * Copyright (C) 1999 Xi Graphics, Inc. | |
| 8 | − * | |
| 9 | − * Permission is hereby granted, free of charge, to any person obtaining a | |
| 10 | − * copy of this software and associated documentation files (the "Software"), | |
| 11 | − * to deal in the Software without restriction, including without limitation | |
| 12 | − * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 13 | − * and/or sell copies of the Software, and to permit persons to whom the | |
| 14 | − * Software is furnished to do so, subject to the following conditions: | |
| 15 | − * | |
| 16 | − * The above copyright notice and this permission notice shall be included | |
| 17 | − * in all copies or substantial portions of the Software. | |
| 18 | − * | |
| 19 | − * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS | |
| 20 | − * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 21 | − * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 22 | − * JEFF HARTMANN, OR ANY OTHER CONTRIBUTORS BE LIABLE FOR ANY CLAIM, | |
| 23 | − * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR | |
| 24 | − * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE | |
| 25 | − * OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. | |
| 26 | − * | |
| 27 | − */ | |
| 28 | − | |
| 29 | −#include <linux/kernel.h> | |
| 30 | −#include <linux/pci.h> | |
| 31 | −#include <linux/fs.h> | |
| 32 | −#include <linux/agpgart.h> | |
| 33 | −#include <linux/slab.h> | |
| 34 | −#include <linux/uaccess.h> | |
| 35 | −#include "agp.h" | |
| 36 | −#include "compat_ioctl.h" | |
| 37 | − | |
| 38 | −static int compat_agpioc_info_wrap(struct agp_file_private *priv, void __user *arg) | |
| 39 | −{ | |
| 40 | − struct agp_info32 userinfo; | |
| 41 | − struct agp_kern_info kerninfo; | |
| 42 | − | |
| 43 | − agp_copy_info(agp_bridge, &kerninfo); | |
| 44 | − | |
| 45 | − userinfo.version.major = kerninfo.version.major; | |
| 46 | − userinfo.version.minor = kerninfo.version.minor; | |
| 47 | − userinfo.bridge_id = kerninfo.device->vendor | | |
| 48 | − (kerninfo.device->device << 16); | |
| 49 | − userinfo.agp_mode = kerninfo.mode; | |
| 50 | − userinfo.aper_base = (compat_long_t)kerninfo.aper_base; | |
| 51 | − userinfo.aper_size = kerninfo.aper_size; | |
| 52 | − userinfo.pg_total = userinfo.pg_system = kerninfo.max_memory; | |
| 53 | − userinfo.pg_used = kerninfo.current_memory; | |
| 54 | − | |
| 55 | − if (copy_to_user(arg, &userinfo, sizeof(userinfo))) | |
| 56 | − return -EFAULT; | |
| 57 | − | |
| 58 | − return 0; | |
| 59 | −} | |
| 60 | − | |
| 61 | −static int compat_agpioc_reserve_wrap(struct agp_file_private *priv, void __user *arg) | |
| 62 | −{ | |
| 63 | − struct agp_region32 ureserve; | |
| 64 | − struct agp_region kreserve; | |
| 65 | − struct agp_client *client; | |
| 66 | − struct agp_file_private *client_priv; | |
| 67 | − | |
| 68 | − DBG(""); | |
| 69 | − if (copy_from_user(&ureserve, arg, sizeof(ureserve))) | |
| 70 | − return -EFAULT; | |
| 71 | − | |
| 72 | − if ((unsigned) ureserve.seg_count >= ~0U/sizeof(struct agp_segment32)) | |
| 73 | − return -EFAULT; | |
| 74 | − | |
| 75 | − kreserve.pid = ureserve.pid; | |
| 76 | − kreserve.seg_count = ureserve.seg_count; | |
| 77 | − | |
| 78 | − client = agp_find_client_by_pid(kreserve.pid); | |
| 79 | − | |
| 80 | − if (kreserve.seg_count == 0) { | |
| 81 | − /* remove a client */ | |
| 82 | − client_priv = agp_find_private(kreserve.pid); | |
| 83 | − | |
| 84 | − if (client_priv != NULL) { | |
| 85 | − set_bit(AGP_FF_IS_CLIENT, &client_priv->access_flags); | |
| 86 | − set_bit(AGP_FF_IS_VALID, &client_priv->access_flags); | |
| 87 | − } | |
| 88 | − if (client == NULL) { | |
| 89 | − /* client is already removed */ | |
| 90 | − return 0; | |
| 91 | − } | |
| 92 | − return agp_remove_client(kreserve.pid); | |
| 93 | − } else { | |
| 94 | − struct agp_segment32 *usegment; | |
| 95 | − struct agp_segment *ksegment; | |
| 96 | − int seg; | |
| 97 | − | |
| 98 | − if (ureserve.seg_count >= 16384) | |
| 99 | − return -EINVAL; | |
| 100 | − | |
| 101 | − usegment = kmalloc_array(ureserve.seg_count, | |
| 102 | − sizeof(*usegment), | |
| 103 | − GFP_KERNEL); | |
| 104 | − if (!usegment) | |
| 105 | − return -ENOMEM; | |
| 106 | − | |
| 107 | − ksegment = kmalloc_array(kreserve.seg_count, | |
| 108 | − sizeof(*ksegment), | |
| 109 | − GFP_KERNEL); | |
| 110 | − if (!ksegment) { | |
| 111 | − kfree(usegment); | |
| 112 | − return -ENOMEM; | |
| 113 | − } | |
| 114 | − | |
| 115 | − if (copy_from_user(usegment, (void __user *) ureserve.seg_list, | |
| 116 | − sizeof(*usegment) * ureserve.seg_count)) { | |
| 117 | − kfree(usegment); | |
| 118 | − kfree(ksegment); | |
| 119 | − return -EFAULT; | |
| 120 | − } | |
| 121 | − | |
| 122 | − for (seg = 0; seg < ureserve.seg_count; seg++) { | |
| 123 | − ksegment[seg].pg_start = usegment[seg].pg_start; | |
| 124 | − ksegment[seg].pg_count = usegment[seg].pg_count; | |
| 125 | − ksegment[seg].prot = usegment[seg].prot; | |
| 126 | − } | |
| 127 | − | |
| 128 | − kfree(usegment); | |
| 129 | − kreserve.seg_list = ksegment; | |
| 130 | − | |
| 131 | − if (client == NULL) { | |
| 132 | − /* Create the client and add the segment */ | |
| 133 | − client = agp_create_client(kreserve.pid); | |
| 134 | − | |
| 135 | − if (client == NULL) { | |
| 136 | − kfree(ksegment); | |
| 137 | − return -ENOMEM; | |
| 138 | − } | |
| 139 | − client_priv = agp_find_private(kreserve.pid); | |
| 140 | − | |
| 141 | − if (client_priv != NULL) { | |
| 142 | − set_bit(AGP_FF_IS_CLIENT, &client_priv->access_flags); | |
| 143 | − set_bit(AGP_FF_IS_VALID, &client_priv->access_flags); | |
| 144 | − } | |
| 145 | − } | |
| 146 | − return agp_create_segment(client, &kreserve); | |
| 147 | − } | |
| 148 | − /* Will never really happen */ | |
| 149 | − return -EINVAL; | |
| 150 | −} | |
| 151 | − | |
| 152 | −static int compat_agpioc_allocate_wrap(struct agp_file_private *priv, void __user *arg) | |
| 153 | −{ | |
| 154 | − struct agp_memory *memory; | |
| 155 | − struct agp_allocate32 alloc; | |
| 156 | − | |
| 157 | − DBG(""); | |
| 158 | − if (copy_from_user(&alloc, arg, sizeof(alloc))) | |
| 159 | − return -EFAULT; | |
| 160 | − | |
| 161 | − memory = agp_allocate_memory_wrap(alloc.pg_count, alloc.type); | |
| 162 | − | |
| 163 | − if (memory == NULL) | |
| 164 | − return -ENOMEM; | |
| 165 | − | |
| 166 | − alloc.key = memory->key; | |
| 167 | − alloc.physical = memory->physical; | |
| 168 | − | |
| 169 | − if (copy_to_user(arg, &alloc, sizeof(alloc))) { | |
| 170 | − agp_free_memory_wrap(memory); | |
| 171 | − return -EFAULT; | |
| 172 | − } | |
| 173 | − return 0; | |
| 174 | −} | |
| 175 | − | |
| 176 | −static int compat_agpioc_bind_wrap(struct agp_file_private *priv, void __user *arg) | |
| 177 | −{ | |
| 178 | − struct agp_bind32 bind_info; | |
| 179 | − struct agp_memory *memory; | |
| 180 | − | |
| 181 | − DBG(""); | |
| 182 | − if (copy_from_user(&bind_info, arg, sizeof(bind_info))) | |
| 183 | − return -EFAULT; | |
| 184 | − | |
| 185 | − memory = agp_find_mem_by_key(bind_info.key); | |
| 186 | − | |
| 187 | − if (memory == NULL) | |
| 188 | − return -EINVAL; | |
| 189 | − | |
| 190 | − return agp_bind_memory(memory, bind_info.pg_start); | |
| 191 | −} | |
| 192 | − | |
| 193 | −static int compat_agpioc_unbind_wrap(struct agp_file_private *priv, void __user *arg) | |
| 194 | −{ | |
| 195 | − struct agp_memory *memory; | |
| 196 | − struct agp_unbind32 unbind; | |
| 197 | − | |
| 198 | − DBG(""); | |
| 199 | − if (copy_from_user(&unbind, arg, sizeof(unbind))) | |
| 200 | − return -EFAULT; | |
| 201 | − | |
| 202 | − memory = agp_find_mem_by_key(unbind.key); | |
| 203 | − | |
| 204 | − if (memory == NULL) | |
| 205 | − return -EINVAL; | |
| 206 | − | |
| 207 | − return agp_unbind_memory(memory); | |
| 208 | −} | |
| 209 | − | |
| 210 | −long compat_agp_ioctl(struct file *file, unsigned int cmd, unsigned long arg) | |
| 211 | −{ | |
| 212 | − struct agp_file_private *curr_priv = file->private_data; | |
| 213 | − int ret_val = -ENOTTY; | |
| 214 | − | |
| 215 | − mutex_lock(&(agp_fe.agp_mutex)); | |
| 216 | − | |
| 217 | − if ((agp_fe.current_controller == NULL) && | |
| 218 | − (cmd != AGPIOC_ACQUIRE32)) { | |
| 219 | − ret_val = -EINVAL; | |
| 220 | − goto ioctl_out; | |
| 221 | − } | |
| 222 | − if ((agp_fe.backend_acquired != true) && | |
| 223 | − (cmd != AGPIOC_ACQUIRE32)) { | |
| 224 | − ret_val = -EBUSY; | |
| 225 | − goto ioctl_out; | |
| 226 | − } | |
| 227 | − if (cmd != AGPIOC_ACQUIRE32) { | |
| 228 | − if (!(test_bit(AGP_FF_IS_CONTROLLER, &curr_priv->access_flags))) { | |
| 229 | − ret_val = -EPERM; | |
| 230 | − goto ioctl_out; | |
| 231 | − } | |
| 232 | − /* Use the original pid of the controller, | |
| 233 | − * in case it's threaded */ | |
| 234 | − | |
| 235 | − if (agp_fe.current_controller->pid != curr_priv->my_pid) { | |
| 236 | − ret_val = -EBUSY; | |
| 237 | − goto ioctl_out; | |
| 238 | − } | |
| 239 | − } | |
| 240 | − | |
| 241 | − switch (cmd) { | |
| 242 | − case AGPIOC_INFO32: | |
| 243 | − ret_val = compat_agpioc_info_wrap(curr_priv, (void __user *) arg); | |
| 244 | − break; | |
| 245 | − | |
| 246 | − case AGPIOC_ACQUIRE32: | |
| 247 | − ret_val = agpioc_acquire_wrap(curr_priv); | |
| 248 | − break; | |
| 249 | − | |
| 250 | − case AGPIOC_RELEASE32: | |
| 251 | − ret_val = agpioc_release_wrap(curr_priv); | |
| 252 | − break; | |
| 253 | − | |
| 254 | − case AGPIOC_SETUP32: | |
| 255 | − ret_val = agpioc_setup_wrap(curr_priv, (void __user *) arg); | |
| 256 | − break; | |
| 257 | − | |
| 258 | − case AGPIOC_RESERVE32: | |
| 259 | − ret_val = compat_agpioc_reserve_wrap(curr_priv, (void __user *) arg); | |
| 260 | − break; | |
| 261 | − | |
| 262 | − case AGPIOC_PROTECT32: | |
| 263 | − ret_val = agpioc_protect_wrap(curr_priv); | |
| 264 | − break; | |
| 265 | − | |
| 266 | − case AGPIOC_ALLOCATE32: | |
| 267 | − ret_val = compat_agpioc_allocate_wrap(curr_priv, (void __user *) arg); | |
| 268 | − break; | |
| 269 | − | |
| 270 | − case AGPIOC_DEALLOCATE32: | |
| 271 | − ret_val = agpioc_deallocate_wrap(curr_priv, (int) arg); | |
| 272 | − break; | |
| 273 | − | |
| 274 | − case AGPIOC_BIND32: | |
| 275 | − ret_val = compat_agpioc_bind_wrap(curr_priv, (void __user *) arg); | |
| 276 | − break; | |
| 277 | − | |
| 278 | − case AGPIOC_UNBIND32: | |
| 279 | − ret_val = compat_agpioc_unbind_wrap(curr_priv, (void __user *) arg); | |
| 280 | − break; | |
| 281 | − | |
| 282 | − case AGPIOC_CHIPSET_FLUSH32: | |
| 283 | − break; | |
| 284 | − } | |
| 285 | − | |
| 286 | −ioctl_out: | |
| 287 | − DBG("ioctl returns %d\n", ret_val); | |
| 288 | − mutex_unlock(&(agp_fe.agp_mutex)); | |
| 289 | − return ret_val; | |
| 290 | −} | |
| 291 | − | |
| @@ -1,106 +0,0 @@ | ||
| 1 | −/* | |
| 2 | − * Copyright (C) 1999 Jeff Hartmann | |
| 3 | − * Copyright (C) 1999 Precision Insight, Inc. | |
| 4 | − * Copyright (C) 1999 Xi Graphics, Inc. | |
| 5 | − * | |
| 6 | − * Permission is hereby granted, free of charge, to any person obtaining a | |
| 7 | − * copy of this software and associated documentation files (the "Software"), | |
| 8 | − * to deal in the Software without restriction, including without limitation | |
| 9 | − * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 10 | − * and/or sell copies of the Software, and to permit persons to whom the | |
| 11 | − * Software is furnished to do so, subject to the following conditions: | |
| 12 | − * | |
| 13 | − * The above copyright notice and this permission notice shall be included | |
| 14 | − * in all copies or substantial portions of the Software. | |
| 15 | − * | |
| 16 | − * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS | |
| 17 | − * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 18 | − * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 19 | − * JEFF HARTMANN, OR ANY OTHER CONTRIBUTORS BE LIABLE FOR ANY CLAIM, | |
| 20 | − * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR | |
| 21 | − * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE | |
| 22 | − * OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. | |
| 23 | − * | |
| 24 | − */ | |
| 25 | − | |
| 26 | −#ifndef _AGP_COMPAT_IOCTL_H | |
| 27 | −#define _AGP_COMPAT_IOCTL_H | |
| 28 | − | |
| 29 | −#include <linux/compat.h> | |
| 30 | −#include <linux/agpgart.h> | |
| 31 | − | |
| 32 | −#define AGPIOC_INFO32 _IOR (AGPIOC_BASE, 0, compat_uptr_t) | |
| 33 | −#define AGPIOC_ACQUIRE32 _IO (AGPIOC_BASE, 1) | |
| 34 | −#define AGPIOC_RELEASE32 _IO (AGPIOC_BASE, 2) | |
| 35 | −#define AGPIOC_SETUP32 _IOW (AGPIOC_BASE, 3, compat_uptr_t) | |
| 36 | −#define AGPIOC_RESERVE32 _IOW (AGPIOC_BASE, 4, compat_uptr_t) | |
| 37 | −#define AGPIOC_PROTECT32 _IOW (AGPIOC_BASE, 5, compat_uptr_t) | |
| 38 | −#define AGPIOC_ALLOCATE32 _IOWR(AGPIOC_BASE, 6, compat_uptr_t) | |
| 39 | −#define AGPIOC_DEALLOCATE32 _IOW (AGPIOC_BASE, 7, compat_int_t) | |
| 40 | −#define AGPIOC_BIND32 _IOW (AGPIOC_BASE, 8, compat_uptr_t) | |
| 41 | −#define AGPIOC_UNBIND32 _IOW (AGPIOC_BASE, 9, compat_uptr_t) | |
| 42 | −#define AGPIOC_CHIPSET_FLUSH32 _IO (AGPIOC_BASE, 10) | |
| 43 | − | |
| 44 | −struct agp_info32 { | |
| 45 | − struct agp_version version; /* version of the driver */ | |
| 46 | − u32 bridge_id; /* bridge vendor/device */ | |
| 47 | − u32 agp_mode; /* mode info of bridge */ | |
| 48 | − compat_long_t aper_base; /* base of aperture */ | |
| 49 | − compat_size_t aper_size; /* size of aperture */ | |
| 50 | − compat_size_t pg_total; /* max pages (swap + system) */ | |
| 51 | − compat_size_t pg_system; /* max pages (system) */ | |
| 52 | − compat_size_t pg_used; /* current pages used */ | |
| 53 | −}; | |
| 54 | − | |
| 55 | −/* | |
| 56 | − * The "prot" down below needs still a "sleep" flag somehow ... | |
| 57 | − */ | |
| 58 | −struct agp_segment32 { | |
| 59 | − compat_off_t pg_start; /* starting page to populate */ | |
| 60 | − compat_size_t pg_count; /* number of pages */ | |
| 61 | − compat_int_t prot; /* prot flags for mmap */ | |
| 62 | −}; | |
| 63 | − | |
| 64 | −struct agp_region32 { | |
| 65 | − compat_pid_t pid; /* pid of process */ | |
| 66 | − compat_size_t seg_count; /* number of segments */ | |
| 67 | − struct agp_segment32 *seg_list; | |
| 68 | −}; | |
| 69 | − | |
| 70 | −struct agp_allocate32 { | |
| 71 | − compat_int_t key; /* tag of allocation */ | |
| 72 | − compat_size_t pg_count; /* number of pages */ | |
| 73 | − u32 type; /* 0 == normal, other devspec */ | |
| 74 | − u32 physical; /* device specific (some devices | |
| 75 | − * need a phys address of the | |
| 76 | − * actual page behind the gatt | |
| 77 | − * table) */ | |
| 78 | −}; | |
| 79 | − | |
| 80 | −struct agp_bind32 { | |
| 81 | − compat_int_t key; /* tag of allocation */ | |
| 82 | − compat_off_t pg_start; /* starting page to populate */ | |
| 83 | −}; | |
| 84 | − | |
| 85 | −struct agp_unbind32 { | |
| 86 | − compat_int_t key; /* tag of allocation */ | |
| 87 | − u32 priority; /* priority for paging out */ | |
| 88 | −}; | |
| 89 | − | |
| 90 | −extern struct agp_front_data agp_fe; | |
| 91 | − | |
| 92 | −int agpioc_acquire_wrap(struct agp_file_private *priv); | |
| 93 | −int agpioc_release_wrap(struct agp_file_private *priv); | |
| 94 | −int agpioc_protect_wrap(struct agp_file_private *priv); | |
| 95 | −int agpioc_setup_wrap(struct agp_file_private *priv, void __user *arg); | |
| 96 | −int agpioc_deallocate_wrap(struct agp_file_private *priv, int arg); | |
| 97 | −struct agp_file_private *agp_find_private(pid_t pid); | |
| 98 | −struct agp_client *agp_create_client(pid_t id); | |
| 99 | −int agp_remove_client(pid_t id); | |
| 100 | −int agp_create_segment(struct agp_client *client, struct agp_region *region); | |
| 101 | −void agp_free_memory_wrap(struct agp_memory *memory); | |
| 102 | −struct agp_memory *agp_allocate_memory_wrap(size_t pg_count, u32 type); | |
| 103 | −struct agp_memory *agp_find_mem_by_key(int key); | |
| 104 | −struct agp_client *agp_find_client_by_pid(pid_t id); | |
| 105 | − | |
| 106 | −#endif /* _AGP_COMPAT_H */ | |
| @@ -1,1068 +0,0 @@ | ||
| 1 | −/* | |
| 2 | − * AGPGART driver frontend | |
| 3 | − * Copyright (C) 2004 Silicon Graphics, Inc. | |
| 4 | − * Copyright (C) 2002-2003 Dave Jones | |
| 5 | − * Copyright (C) 1999 Jeff Hartmann | |
| 6 | − * Copyright (C) 1999 Precision Insight, Inc. | |
| 7 | − * Copyright (C) 1999 Xi Graphics, Inc. | |
| 8 | − * | |
| 9 | − * Permission is hereby granted, free of charge, to any person obtaining a | |
| 10 | − * copy of this software and associated documentation files (the "Software"), | |
| 11 | − * to deal in the Software without restriction, including without limitation | |
| 12 | − * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 13 | − * and/or sell copies of the Software, and to permit persons to whom the | |
| 14 | − * Software is furnished to do so, subject to the following conditions: | |
| 15 | − * | |
| 16 | − * The above copyright notice and this permission notice shall be included | |
| 17 | − * in all copies or substantial portions of the Software. | |
| 18 | − * | |
| 19 | − * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS | |
| 20 | − * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 21 | − * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 22 | − * JEFF HARTMANN, OR ANY OTHER CONTRIBUTORS BE LIABLE FOR ANY CLAIM, | |
| 23 | − * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR | |
| 24 | − * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE | |
| 25 | − * OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. | |
| 26 | − * | |
| 27 | − */ | |
| 28 | − | |
| 29 | −#include <linux/types.h> | |
| 30 | −#include <linux/kernel.h> | |
| 31 | −#include <linux/module.h> | |
| 32 | −#include <linux/mman.h> | |
| 33 | −#include <linux/pci.h> | |
| 34 | −#include <linux/miscdevice.h> | |
| 35 | −#include <linux/agp_backend.h> | |
| 36 | −#include <linux/agpgart.h> | |
| 37 | −#include <linux/slab.h> | |
| 38 | −#include <linux/mm.h> | |
| 39 | −#include <linux/fs.h> | |
| 40 | −#include <linux/sched.h> | |
| 41 | −#include <linux/uaccess.h> | |
| 42 | − | |
| 43 | −#include "agp.h" | |
| 44 | −#include "compat_ioctl.h" | |
| 45 | − | |
| 46 | −struct agp_front_data agp_fe; | |
| 47 | − | |
| 48 | −struct agp_memory *agp_find_mem_by_key(int key) | |
| 49 | −{ | |
| 50 | − struct agp_memory *curr; | |
| 51 | − | |
| 52 | − if (agp_fe.current_controller == NULL) | |
| 53 | − return NULL; | |
| 54 | − | |
| 55 | − curr = agp_fe.current_controller->pool; | |
| 56 | − | |
| 57 | − while (curr != NULL) { | |
| 58 | − if (curr->key == key) | |
| 59 | − break; | |
| 60 | − curr = curr->next; | |
| 61 | − } | |
| 62 | − | |
| 63 | − DBG("key=%d -> mem=%p", key, curr); | |
| 64 | − return curr; | |
| 65 | −} | |
| 66 | − | |
| 67 | −static void agp_remove_from_pool(struct agp_memory *temp) | |
| 68 | −{ | |
| 69 | − struct agp_memory *prev; | |
| 70 | − struct agp_memory *next; | |
| 71 | − | |
| 72 | − /* Check to see if this is even in the memory pool */ | |
| 73 | − | |
| 74 | − DBG("mem=%p", temp); | |
| 75 | − if (agp_find_mem_by_key(temp->key) != NULL) { | |
| 76 | − next = temp->next; | |
| 77 | − prev = temp->prev; | |
| 78 | − | |
| 79 | − if (prev != NULL) { | |
| 80 | − prev->next = next; | |
| 81 | − if (next != NULL) | |
| 82 | − next->prev = prev; | |
| 83 | − | |
| 84 | − } else { | |
| 85 | − /* This is the first item on the list */ | |
| 86 | − if (next != NULL) | |
| 87 | − next->prev = NULL; | |
| 88 | − | |
| 89 | − agp_fe.current_controller->pool = next; | |
| 90 | − } | |
| 91 | − } | |
| 92 | −} | |
| 93 | − | |
| 94 | −/* | |
| 95 | − * Routines for managing each client's segment list - | |
| 96 | − * These routines handle adding and removing segments | |
| 97 | − * to each auth'ed client. | |
| 98 | − */ | |
| 99 | − | |
| 100 | −static struct | |
| 101 | −agp_segment_priv *agp_find_seg_in_client(const struct agp_client *client, | |
| 102 | − unsigned long offset, | |
| 103 | − int size, pgprot_t page_prot) | |
| 104 | −{ | |
| 105 | − struct agp_segment_priv *seg; | |
| 106 | − int i; | |
| 107 | − off_t pg_start; | |
| 108 | − size_t pg_count; | |
| 109 | − | |
| 110 | − pg_start = offset / 4096; | |
| 111 | − pg_count = size / 4096; | |
| 112 | − seg = *(client->segments); | |
| 113 | − | |
| 114 | − for (i = 0; i < client->num_segments; i++) { | |
| 115 | − if ((seg[i].pg_start == pg_start) && | |
| 116 | − (seg[i].pg_count == pg_count) && | |
| 117 | − (pgprot_val(seg[i].prot) == pgprot_val(page_prot))) { | |
| 118 | − return seg + i; | |
| 119 | − } | |
| 120 | − } | |
| 121 | − | |
| 122 | − return NULL; | |
| 123 | −} | |
| 124 | − | |
| 125 | −static void agp_remove_seg_from_client(struct agp_client *client) | |
| 126 | −{ | |
| 127 | − DBG("client=%p", client); | |
| 128 | − | |
| 129 | − if (client->segments != NULL) { | |
| 130 | − if (*(client->segments) != NULL) { | |
| 131 | − DBG("Freeing %p from client %p", *(client->segments), client); | |
| 132 | − kfree(*(client->segments)); | |
| 133 | − } | |
| 134 | − DBG("Freeing %p from client %p", client->segments, client); | |
| 135 | − kfree(client->segments); | |
| 136 | − client->segments = NULL; | |
| 137 | − } | |
| 138 | −} | |
| 139 | − | |
| 140 | −static void agp_add_seg_to_client(struct agp_client *client, | |
| 141 | − struct agp_segment_priv ** seg, int num_segments) | |
| 142 | −{ | |
| 143 | − struct agp_segment_priv **prev_seg; | |
| 144 | − | |
| 145 | − prev_seg = client->segments; | |
| 146 | − | |
| 147 | − if (prev_seg != NULL) | |
| 148 | − agp_remove_seg_from_client(client); | |
| 149 | − | |
| 150 | − DBG("Adding seg %p (%d segments) to client %p", seg, num_segments, client); | |
| 151 | − client->num_segments = num_segments; | |
| 152 | − client->segments = seg; | |
| 153 | −} | |
| 154 | − | |
| 155 | −static pgprot_t agp_convert_mmap_flags(int prot) | |
| 156 | −{ | |
| 157 | − unsigned long prot_bits; | |
| 158 | − | |
| 159 | − prot_bits = calc_vm_prot_bits(prot, 0) | VM_SHARED; | |
| 160 | − return vm_get_page_prot(prot_bits); | |
| 161 | −} | |
| 162 | − | |
| 163 | −int agp_create_segment(struct agp_client *client, struct agp_region *region) | |
| 164 | −{ | |
| 165 | − struct agp_segment_priv **ret_seg; | |
| 166 | − struct agp_segment_priv *seg; | |
| 167 | − struct agp_segment *user_seg; | |
| 168 | − size_t i; | |
| 169 | − | |
| 170 | − seg = kzalloc((sizeof(struct agp_segment_priv) * region->seg_count), GFP_KERNEL); | |
| 171 | − if (seg == NULL) { | |
| 172 | − kfree(region->seg_list); | |
| 173 | − region->seg_list = NULL; | |
| 174 | − return -ENOMEM; | |
| 175 | − } | |
| 176 | − user_seg = region->seg_list; | |
| 177 | − | |
| 178 | − for (i = 0; i < region->seg_count; i++) { | |
| 179 | − seg[i].pg_start = user_seg[i].pg_start; | |
| 180 | − seg[i].pg_count = user_seg[i].pg_count; | |
| 181 | − seg[i].prot = agp_convert_mmap_flags(user_seg[i].prot); | |
| 182 | − } | |
| 183 | − kfree(region->seg_list); | |
| 184 | − region->seg_list = NULL; | |
| 185 | − | |
| 186 | − ret_seg = kmalloc(sizeof(void *), GFP_KERNEL); | |
| 187 | − if (ret_seg == NULL) { | |
| 188 | − kfree(seg); | |
| 189 | − return -ENOMEM; | |
| 190 | − } | |
| 191 | − *ret_seg = seg; | |
| 192 | − agp_add_seg_to_client(client, ret_seg, region->seg_count); | |
| 193 | − return 0; | |
| 194 | −} | |
| 195 | − | |
| 196 | −/* End - Routines for managing each client's segment list */ | |
| 197 | − | |
| 198 | −/* This function must only be called when current_controller != NULL */ | |
| 199 | −static void agp_insert_into_pool(struct agp_memory * temp) | |
| 200 | −{ | |
| 201 | − struct agp_memory *prev; | |
| 202 | − | |
| 203 | − prev = agp_fe.current_controller->pool; | |
| 204 | − | |
| 205 | − if (prev != NULL) { | |
| 206 | − prev->prev = temp; | |
| 207 | − temp->next = prev; | |
| 208 | − } | |
| 209 | − agp_fe.current_controller->pool = temp; | |
| 210 | −} | |
| 211 | − | |
| 212 | − | |
| 213 | −/* File private list routines */ | |
| 214 | − | |
| 215 | −struct agp_file_private *agp_find_private(pid_t pid) | |
| 216 | −{ | |
| 217 | − struct agp_file_private *curr; | |
| 218 | − | |
| 219 | − curr = agp_fe.file_priv_list; | |
| 220 | − | |
| 221 | − while (curr != NULL) { | |
| 222 | − if (curr->my_pid == pid) | |
| 223 | − return curr; | |
| 224 | − curr = curr->next; | |
| 225 | − } | |
| 226 | − | |
| 227 | − return NULL; | |
| 228 | −} | |
| 229 | − | |
| 230 | −static void agp_insert_file_private(struct agp_file_private * priv) | |
| 231 | −{ | |
| 232 | − struct agp_file_private *prev; | |
| 233 | − | |
| 234 | − prev = agp_fe.file_priv_list; | |
| 235 | − | |
| 236 | − if (prev != NULL) | |
| 237 | − prev->prev = priv; | |
| 238 | − priv->next = prev; | |
| 239 | − agp_fe.file_priv_list = priv; | |
| 240 | −} | |
| 241 | − | |
| 242 | −static void agp_remove_file_private(struct agp_file_private * priv) | |
| 243 | −{ | |
| 244 | − struct agp_file_private *next; | |
| 245 | − struct agp_file_private *prev; | |
| 246 | − | |
| 247 | − next = priv->next; | |
| 248 | − prev = priv->prev; | |
| 249 | − | |
| 250 | − if (prev != NULL) { | |
| 251 | − prev->next = next; | |
| 252 | − | |
| 253 | − if (next != NULL) | |
| 254 | − next->prev = prev; | |
| 255 | − | |
| 256 | − } else { | |
| 257 | − if (next != NULL) | |
| 258 | − next->prev = NULL; | |
| 259 | − | |
| 260 | − agp_fe.file_priv_list = next; | |
| 261 | − } | |
| 262 | −} | |
| 263 | − | |
| 264 | −/* End - File flag list routines */ | |
| 265 | − | |
| 266 | −/* | |
| 267 | − * Wrappers for agp_free_memory & agp_allocate_memory | |
| 268 | − * These make sure that internal lists are kept updated. | |
| 269 | − */ | |
| 270 | −void agp_free_memory_wrap(struct agp_memory *memory) | |
| 271 | −{ | |
| 272 | − agp_remove_from_pool(memory); | |
| 273 | − agp_free_memory(memory); | |
| 274 | −} | |
| 275 | − | |
| 276 | −struct agp_memory *agp_allocate_memory_wrap(size_t pg_count, u32 type) | |
| 277 | −{ | |
| 278 | − struct agp_memory *memory; | |
| 279 | − | |
| 280 | − memory = agp_allocate_memory(agp_bridge, pg_count, type); | |
| 281 | − if (memory == NULL) | |
| 282 | − return NULL; | |
| 283 | − | |
| 284 | − agp_insert_into_pool(memory); | |
| 285 | − return memory; | |
| 286 | −} | |
| 287 | − | |
| 288 | −/* Routines for managing the list of controllers - | |
| 289 | − * These routines manage the current controller, and the list of | |
| 290 | − * controllers | |
| 291 | − */ | |
| 292 | − | |
| 293 | −static struct agp_controller *agp_find_controller_by_pid(pid_t id) | |
| 294 | −{ | |
| 295 | − struct agp_controller *controller; | |
| 296 | − | |
| 297 | − controller = agp_fe.controllers; | |
| 298 | − | |
| 299 | − while (controller != NULL) { | |
| 300 | − if (controller->pid == id) | |
| 301 | − return controller; | |
| 302 | − controller = controller->next; | |
| 303 | − } | |
| 304 | − | |
| 305 | − return NULL; | |
| 306 | −} | |
| 307 | − | |
| 308 | −static struct agp_controller *agp_create_controller(pid_t id) | |
| 309 | −{ | |
| 310 | − struct agp_controller *controller; | |
| 311 | − | |
| 312 | − controller = kzalloc(sizeof(struct agp_controller), GFP_KERNEL); | |
| 313 | − if (controller == NULL) | |
| 314 | − return NULL; | |
| 315 | − | |
| 316 | − controller->pid = id; | |
| 317 | − return controller; | |
| 318 | −} | |
| 319 | − | |
| 320 | −static int agp_insert_controller(struct agp_controller *controller) | |
| 321 | −{ | |
| 322 | − struct agp_controller *prev_controller; | |
| 323 | − | |
| 324 | − prev_controller = agp_fe.controllers; | |
| 325 | − controller->next = prev_controller; | |
| 326 | − | |
| 327 | − if (prev_controller != NULL) | |
| 328 | − prev_controller->prev = controller; | |
| 329 | − | |
| 330 | − agp_fe.controllers = controller; | |
| 331 | − | |
| 332 | − return 0; | |
| 333 | −} | |
| 334 | − | |
| 335 | −static void agp_remove_all_clients(struct agp_controller *controller) | |
| 336 | −{ | |
| 337 | − struct agp_client *client; | |
| 338 | − struct agp_client *temp; | |
| 339 | − | |
| 340 | − client = controller->clients; | |
| 341 | − | |
| 342 | − while (client) { | |
| 343 | − struct agp_file_private *priv; | |
| 344 | − | |
| 345 | − temp = client; | |
| 346 | − agp_remove_seg_from_client(temp); | |
| 347 | − priv = agp_find_private(temp->pid); | |
| 348 | − | |
| 349 | − if (priv != NULL) { | |
| 350 | − clear_bit(AGP_FF_IS_VALID, &priv->access_flags); | |
| 351 | − clear_bit(AGP_FF_IS_CLIENT, &priv->access_flags); | |
| 352 | − } | |
| 353 | − client = client->next; | |
| 354 | − kfree(temp); | |
| 355 | − } | |
| 356 | −} | |
| 357 | − | |
| 358 | −static void agp_remove_all_memory(struct agp_controller *controller) | |
| 359 | −{ | |
| 360 | − struct agp_memory *memory; | |
| 361 | − struct agp_memory *temp; | |
| 362 | − | |
| 363 | − memory = controller->pool; | |
| 364 | − | |
| 365 | − while (memory) { | |
| 366 | − temp = memory; | |
| 367 | − memory = memory->next; | |
| 368 | − agp_free_memory_wrap(temp); | |
| 369 | − } | |
| 370 | −} | |
| 371 | − | |
| 372 | −static int agp_remove_controller(struct agp_controller *controller) | |
| 373 | −{ | |
| 374 | − struct agp_controller *prev_controller; | |
| 375 | − struct agp_controller *next_controller; | |
| 376 | − | |
| 377 | − prev_controller = controller->prev; | |
| 378 | − next_controller = controller->next; | |
| 379 | − | |
| 380 | − if (prev_controller != NULL) { | |
| 381 | − prev_controller->next = next_controller; | |
| 382 | − if (next_controller != NULL) | |
| 383 | − next_controller->prev = prev_controller; | |
| 384 | − | |
| 385 | − } else { | |
| 386 | − if (next_controller != NULL) | |
| 387 | − next_controller->prev = NULL; | |
| 388 | − | |
| 389 | − agp_fe.controllers = next_controller; | |
| 390 | − } | |
| 391 | − | |
| 392 | − agp_remove_all_memory(controller); | |
| 393 | − agp_remove_all_clients(controller); | |
| 394 | − | |
| 395 | − if (agp_fe.current_controller == controller) { | |
| 396 | − agp_fe.current_controller = NULL; | |
| 397 | − agp_fe.backend_acquired = false; | |
| 398 | − agp_backend_release(agp_bridge); | |
| 399 | − } | |
| 400 | − kfree(controller); | |
| 401 | − return 0; | |
| 402 | −} | |
| 403 | − | |
| 404 | −static void agp_controller_make_current(struct agp_controller *controller) | |
| 405 | −{ | |
| 406 | − struct agp_client *clients; | |
| 407 | − | |
| 408 | − clients = controller->clients; | |
| 409 | − | |
| 410 | − while (clients != NULL) { | |
| 411 | − struct agp_file_private *priv; | |
| 412 | − | |
| 413 | − priv = agp_find_private(clients->pid); | |
| 414 | − | |
| 415 | − if (priv != NULL) { | |
| 416 | − set_bit(AGP_FF_IS_VALID, &priv->access_flags); | |
| 417 | − set_bit(AGP_FF_IS_CLIENT, &priv->access_flags); | |
| 418 | − } | |
| 419 | − clients = clients->next; | |
| 420 | − } | |
| 421 | − | |
| 422 | − agp_fe.current_controller = controller; | |
| 423 | −} | |
| 424 | − | |
| 425 | −static void agp_controller_release_current(struct agp_controller *controller, | |
| 426 | − struct agp_file_private *controller_priv) | |
| 427 | −{ | |
| 428 | − struct agp_client *clients; | |
| 429 | − | |
| 430 | − clear_bit(AGP_FF_IS_VALID, &controller_priv->access_flags); | |
| 431 | − clients = controller->clients; | |
| 432 | − | |
| 433 | − while (clients != NULL) { | |
| 434 | − struct agp_file_private *priv; | |
| 435 | − | |
| 436 | − priv = agp_find_private(clients->pid); | |
| 437 | − | |
| 438 | − if (priv != NULL) | |
| 439 | − clear_bit(AGP_FF_IS_VALID, &priv->access_flags); | |
| 440 | − | |
| 441 | − clients = clients->next; | |
| 442 | − } | |
| 443 | − | |
| 444 | − agp_fe.current_controller = NULL; | |
| 445 | − agp_fe.used_by_controller = false; | |
| 446 | − agp_backend_release(agp_bridge); | |
| 447 | −} | |
| 448 | − | |
| 449 | −/* | |
| 450 | − * Routines for managing client lists - | |
| 451 | − * These routines are for managing the list of auth'ed clients. | |
| 452 | − */ | |
| 453 | − | |
| 454 | −static struct agp_client | |
| 455 | −*agp_find_client_in_controller(struct agp_controller *controller, pid_t id) | |
| 456 | −{ | |
| 457 | − struct agp_client *client; | |
| 458 | − | |
| 459 | − if (controller == NULL) | |
| 460 | − return NULL; | |
| 461 | − | |
| 462 | − client = controller->clients; | |
| 463 | − | |
| 464 | − while (client != NULL) { | |
| 465 | − if (client->pid == id) | |
| 466 | − return client; | |
| 467 | − client = client->next; | |
| 468 | − } | |
| 469 | − | |
| 470 | − return NULL; | |
| 471 | −} | |
| 472 | − | |
| 473 | −static struct agp_controller *agp_find_controller_for_client(pid_t id) | |
| 474 | −{ | |
| 475 | − struct agp_controller *controller; | |
| 476 | − | |
| 477 | − controller = agp_fe.controllers; | |
| 478 | − | |
| 479 | − while (controller != NULL) { | |
| 480 | − if ((agp_find_client_in_controller(controller, id)) != NULL) | |
| 481 | − return controller; | |
| 482 | − controller = controller->next; | |
| 483 | − } | |
| 484 | − | |
| 485 | − return NULL; | |
| 486 | −} | |
| 487 | − | |
| 488 | −struct agp_client *agp_find_client_by_pid(pid_t id) | |
| 489 | −{ | |
| 490 | − struct agp_client *temp; | |
| 491 | − | |
| 492 | − if (agp_fe.current_controller == NULL) | |
| 493 | − return NULL; | |
| 494 | − | |
| 495 | − temp = agp_find_client_in_controller(agp_fe.current_controller, id); | |
| 496 | − return temp; | |
| 497 | −} | |
| 498 | − | |
| 499 | −static void agp_insert_client(struct agp_client *client) | |
| 500 | −{ | |
| 501 | − struct agp_client *prev_client; | |
| 502 | − | |
| 503 | − prev_client = agp_fe.current_controller->clients; | |
| 504 | − client->next = prev_client; | |
| 505 | − | |
| 506 | − if (prev_client != NULL) | |
| 507 | − prev_client->prev = client; | |
| 508 | − | |
| 509 | − agp_fe.current_controller->clients = client; | |
| 510 | − agp_fe.current_controller->num_clients++; | |
| 511 | −} | |
| 512 | − | |
| 513 | −struct agp_client *agp_create_client(pid_t id) | |
| 514 | −{ | |
| 515 | − struct agp_client *new_client; | |
| 516 | − | |
| 517 | − new_client = kzalloc(sizeof(struct agp_client), GFP_KERNEL); | |
| 518 | − if (new_client == NULL) | |
| 519 | − return NULL; | |
| 520 | − | |
| 521 | − new_client->pid = id; | |
| 522 | − agp_insert_client(new_client); | |
| 523 | − return new_client; | |
| 524 | −} | |
| 525 | − | |
| 526 | −int agp_remove_client(pid_t id) | |
| 527 | −{ | |
| 528 | − struct agp_client *client; | |
| 529 | − struct agp_client *prev_client; | |
| 530 | − struct agp_client *next_client; | |
| 531 | − struct agp_controller *controller; | |
| 532 | − | |
| 533 | − controller = agp_find_controller_for_client(id); | |
| 534 | − if (controller == NULL) | |
| 535 | − return -EINVAL; | |
| 536 | − | |
| 537 | − client = agp_find_client_in_controller(controller, id); | |
| 538 | − if (client == NULL) | |
| 539 | − return -EINVAL; | |
| 540 | − | |
| 541 | − prev_client = client->prev; | |
| 542 | − next_client = client->next; | |
| 543 | − | |
| 544 | − if (prev_client != NULL) { | |
| 545 | − prev_client->next = next_client; | |
| 546 | − if (next_client != NULL) | |
| 547 | − next_client->prev = prev_client; | |
| 548 | − | |
| 549 | − } else { | |
| 550 | − if (next_client != NULL) | |
| 551 | − next_client->prev = NULL; | |
| 552 | − controller->clients = next_client; | |
| 553 | − } | |
| 554 | − | |
| 555 | − controller->num_clients--; | |
| 556 | − agp_remove_seg_from_client(client); | |
| 557 | − kfree(client); | |
| 558 | − return 0; | |
| 559 | −} | |
| 560 | − | |
| 561 | −/* End - Routines for managing client lists */ | |
| 562 | − | |
| 563 | −/* File Operations */ | |
| 564 | − | |
| 565 | −static int agp_mmap(struct file *file, struct vm_area_struct *vma) | |
| 566 | −{ | |
| 567 | − unsigned int size, current_size; | |
| 568 | − unsigned long offset; | |
| 569 | − struct agp_client *client; | |
| 570 | − struct agp_file_private *priv = file->private_data; | |
| 571 | − struct agp_kern_info kerninfo; | |
| 572 | − | |
| 573 | − mutex_lock(&(agp_fe.agp_mutex)); | |
| 574 | − | |
| 575 | − if (agp_fe.backend_acquired != true) | |
| 576 | − goto out_eperm; | |
| 577 | − | |
| 578 | − if (!(test_bit(AGP_FF_IS_VALID, &priv->access_flags))) | |
| 579 | − goto out_eperm; | |
| 580 | − | |
| 581 | − agp_copy_info(agp_bridge, &kerninfo); | |
| 582 | − size = vma->vm_end - vma->vm_start; | |
| 583 | − current_size = kerninfo.aper_size; | |
| 584 | − current_size = current_size * 0x100000; | |
| 585 | − offset = vma->vm_pgoff << PAGE_SHIFT; | |
| 586 | − DBG("%lx:%lx", offset, offset+size); | |
| 587 | − | |
| 588 | − if (test_bit(AGP_FF_IS_CLIENT, &priv->access_flags)) { | |
| 589 | − if ((size + offset) > current_size) | |
| 590 | − goto out_inval; | |
| 591 | − | |
| 592 | − client = agp_find_client_by_pid(current->pid); | |
| 593 | − | |
| 594 | − if (client == NULL) | |
| 595 | − goto out_eperm; | |
| 596 | − | |
| 597 | − if (!agp_find_seg_in_client(client, offset, size, vma->vm_page_prot)) | |
| 598 | − goto out_inval; | |
| 599 | − | |
| 600 | − DBG("client vm_ops=%p", kerninfo.vm_ops); | |
| 601 | − if (kerninfo.vm_ops) { | |
| 602 | − vma->vm_ops = kerninfo.vm_ops; | |
| 603 | − } else if (io_remap_pfn_range(vma, vma->vm_start, | |
| 604 | − (kerninfo.aper_base + offset) >> PAGE_SHIFT, | |
| 605 | − size, | |
| 606 | − pgprot_writecombine(vma->vm_page_prot))) { | |
| 607 | − goto out_again; | |
| 608 | − } | |
| 609 | − mutex_unlock(&(agp_fe.agp_mutex)); | |
| 610 | − return 0; | |
| 611 | − } | |
| 612 | − | |
| 613 | − if (test_bit(AGP_FF_IS_CONTROLLER, &priv->access_flags)) { | |
| 614 | − if (size != current_size) | |
| 615 | − goto out_inval; | |
| 616 | − | |
| 617 | − DBG("controller vm_ops=%p", kerninfo.vm_ops); | |
| 618 | − if (kerninfo.vm_ops) { | |
| 619 | − vma->vm_ops = kerninfo.vm_ops; | |
| 620 | − } else if (io_remap_pfn_range(vma, vma->vm_start, | |
| 621 | − kerninfo.aper_base >> PAGE_SHIFT, | |
| 622 | − size, | |
| 623 | − pgprot_writecombine(vma->vm_page_prot))) { | |
| 624 | − goto out_again; | |
| 625 | − } | |
| 626 | − mutex_unlock(&(agp_fe.agp_mutex)); | |
| 627 | − return 0; | |
| 628 | − } | |
| 629 | − | |
| 630 | −out_eperm: | |
| 631 | − mutex_unlock(&(agp_fe.agp_mutex)); | |
| 632 | − return -EPERM; | |
| 633 | − | |
| 634 | −out_inval: | |
| 635 | − mutex_unlock(&(agp_fe.agp_mutex)); | |
| 636 | − return -EINVAL; | |
| 637 | − | |
| 638 | −out_again: | |
| 639 | − mutex_unlock(&(agp_fe.agp_mutex)); | |
| 640 | − return -EAGAIN; | |
| 641 | −} | |
| 642 | − | |
| 643 | −static int agp_release(struct inode *inode, struct file *file) | |
| 644 | −{ | |
| 645 | − struct agp_file_private *priv = file->private_data; | |
| 646 | − | |
| 647 | − mutex_lock(&(agp_fe.agp_mutex)); | |
| 648 | − | |
| 649 | − DBG("priv=%p", priv); | |
| 650 | − | |
| 651 | − if (test_bit(AGP_FF_IS_CONTROLLER, &priv->access_flags)) { | |
| 652 | − struct agp_controller *controller; | |
| 653 | − | |
| 654 | − controller = agp_find_controller_by_pid(priv->my_pid); | |
| 655 | − | |
| 656 | − if (controller != NULL) { | |
| 657 | − if (controller == agp_fe.current_controller) | |
| 658 | − agp_controller_release_current(controller, priv); | |
| 659 | − agp_remove_controller(controller); | |
| 660 | − controller = NULL; | |
| 661 | − } | |
| 662 | − } | |
| 663 | − | |
| 664 | − if (test_bit(AGP_FF_IS_CLIENT, &priv->access_flags)) | |
| 665 | − agp_remove_client(priv->my_pid); | |
| 666 | − | |
| 667 | − agp_remove_file_private(priv); | |
| 668 | − kfree(priv); | |
| 669 | − file->private_data = NULL; | |
| 670 | − mutex_unlock(&(agp_fe.agp_mutex)); | |
| 671 | − return 0; | |
| 672 | −} | |
| 673 | − | |
| 674 | −static int agp_open(struct inode *inode, struct file *file) | |
| 675 | −{ | |
| 676 | − int minor = iminor(inode); | |
| 677 | − struct agp_file_private *priv; | |
| 678 | − struct agp_client *client; | |
| 679 | − | |
| 680 | − if (minor != AGPGART_MINOR) | |
| 681 | − return -ENXIO; | |
| 682 | − | |
| 683 | − mutex_lock(&(agp_fe.agp_mutex)); | |
| 684 | − | |
| 685 | − priv = kzalloc(sizeof(struct agp_file_private), GFP_KERNEL); | |
| 686 | − if (priv == NULL) { | |
| 687 | − mutex_unlock(&(agp_fe.agp_mutex)); | |
| 688 | − return -ENOMEM; | |
| 689 | − } | |
| 690 | − | |
| 691 | − set_bit(AGP_FF_ALLOW_CLIENT, &priv->access_flags); | |
| 692 | − priv->my_pid = current->pid; | |
| 693 | − | |
| 694 | − if (capable(CAP_SYS_RAWIO)) | |
| 695 | − /* Root priv, can be controller */ | |
| 696 | − set_bit(AGP_FF_ALLOW_CONTROLLER, &priv->access_flags); | |
| 697 | − | |
| 698 | − client = agp_find_client_by_pid(current->pid); | |
| 699 | − | |
| 700 | − if (client != NULL) { | |
| 701 | − set_bit(AGP_FF_IS_CLIENT, &priv->access_flags); | |
| 702 | − set_bit(AGP_FF_IS_VALID, &priv->access_flags); | |
| 703 | − } | |
| 704 | − file->private_data = (void *) priv; | |
| 705 | − agp_insert_file_private(priv); | |
| 706 | − DBG("private=%p, client=%p", priv, client); | |
| 707 | − | |
| 708 | − mutex_unlock(&(agp_fe.agp_mutex)); | |
| 709 | − | |
| 710 | − return 0; | |
| 711 | −} | |
| 712 | − | |
| 713 | −static int agpioc_info_wrap(struct agp_file_private *priv, void __user *arg) | |
| 714 | −{ | |
| 715 | − struct agp_info userinfo; | |
| 716 | − struct agp_kern_info kerninfo; | |
| 717 | − | |
| 718 | − agp_copy_info(agp_bridge, &kerninfo); | |
| 719 | − | |
| 720 | − memset(&userinfo, 0, sizeof(userinfo)); | |
| 721 | − userinfo.version.major = kerninfo.version.major; | |
| 722 | − userinfo.version.minor = kerninfo.version.minor; | |
| 723 | − userinfo.bridge_id = kerninfo.device->vendor | | |
| 724 | − (kerninfo.device->device << 16); | |
| 725 | − userinfo.agp_mode = kerninfo.mode; | |
| 726 | − userinfo.aper_base = kerninfo.aper_base; | |
| 727 | − userinfo.aper_size = kerninfo.aper_size; | |
| 728 | − userinfo.pg_total = userinfo.pg_system = kerninfo.max_memory; | |
| 729 | − userinfo.pg_used = kerninfo.current_memory; | |
| 730 | − | |
| 731 | − if (copy_to_user(arg, &userinfo, sizeof(struct agp_info))) | |
| 732 | − return -EFAULT; | |
| 733 | − | |
| 734 | − return 0; | |
| 735 | −} | |
| 736 | − | |
| 737 | −int agpioc_acquire_wrap(struct agp_file_private *priv) | |
| 738 | −{ | |
| 739 | − struct agp_controller *controller; | |
| 740 | − | |
| 741 | − DBG(""); | |
| 742 | − | |
| 743 | − if (!(test_bit(AGP_FF_ALLOW_CONTROLLER, &priv->access_flags))) | |
| 744 | − return -EPERM; | |
| 745 | − | |
| 746 | − if (agp_fe.current_controller != NULL) | |
| 747 | − return -EBUSY; | |
| 748 | − | |
| 749 | − if (!agp_bridge) | |
| 750 | − return -ENODEV; | |
| 751 | − | |
| 752 | − if (atomic_read(&agp_bridge->agp_in_use)) | |
| 753 | − return -EBUSY; | |
| 754 | − | |
| 755 | − atomic_inc(&agp_bridge->agp_in_use); | |
| 756 | − | |
| 757 | − agp_fe.backend_acquired = true; | |
| 758 | − | |
| 759 | − controller = agp_find_controller_by_pid(priv->my_pid); | |
| 760 | − | |
| 761 | − if (controller != NULL) { | |
| 762 | − agp_controller_make_current(controller); | |
| 763 | − } else { | |
| 764 | − controller = agp_create_controller(priv->my_pid); | |
| 765 | − | |
| 766 | − if (controller == NULL) { | |
| 767 | − agp_fe.backend_acquired = false; | |
| 768 | − agp_backend_release(agp_bridge); | |
| 769 | − return -ENOMEM; | |
| 770 | − } | |
| 771 | − agp_insert_controller(controller); | |
| 772 | − agp_controller_make_current(controller); | |
| 773 | − } | |
| 774 | − | |
| 775 | − set_bit(AGP_FF_IS_CONTROLLER, &priv->access_flags); | |
| 776 | − set_bit(AGP_FF_IS_VALID, &priv->access_flags); | |
| 777 | − return 0; | |
| 778 | −} | |
| 779 | − | |
| 780 | −int agpioc_release_wrap(struct agp_file_private *priv) | |
| 781 | −{ | |
| 782 | − DBG(""); | |
| 783 | − agp_controller_release_current(agp_fe.current_controller, priv); | |
| 784 | − return 0; | |
| 785 | −} | |
| 786 | − | |
| 787 | −int agpioc_setup_wrap(struct agp_file_private *priv, void __user *arg) | |
| 788 | −{ | |
| 789 | − struct agp_setup mode; | |
| 790 | − | |
| 791 | − DBG(""); | |
| 792 | − if (copy_from_user(&mode, arg, sizeof(struct agp_setup))) | |
| 793 | − return -EFAULT; | |
| 794 | − | |
| 795 | − agp_enable(agp_bridge, mode.agp_mode); | |
| 796 | − return 0; | |
| 797 | −} | |
| 798 | − | |
| 799 | −static int agpioc_reserve_wrap(struct agp_file_private *priv, void __user *arg) | |
| 800 | −{ | |
| 801 | − struct agp_region reserve; | |
| 802 | − struct agp_client *client; | |
| 803 | − struct agp_file_private *client_priv; | |
| 804 | − | |
| 805 | − DBG(""); | |
| 806 | − if (copy_from_user(&reserve, arg, sizeof(struct agp_region))) | |
| 807 | − return -EFAULT; | |
| 808 | − | |
| 809 | − if ((unsigned) reserve.seg_count >= ~0U/sizeof(struct agp_segment)) | |
| 810 | − return -EFAULT; | |
| 811 | − | |
| 812 | − client = agp_find_client_by_pid(reserve.pid); | |
| 813 | − | |
| 814 | − if (reserve.seg_count == 0) { | |
| 815 | − /* remove a client */ | |
| 816 | − client_priv = agp_find_private(reserve.pid); | |
| 817 | − | |
| 818 | − if (client_priv != NULL) { | |
| 819 | − set_bit(AGP_FF_IS_CLIENT, &client_priv->access_flags); | |
| 820 | − set_bit(AGP_FF_IS_VALID, &client_priv->access_flags); | |
| 821 | − } | |
| 822 | − if (client == NULL) { | |
| 823 | − /* client is already removed */ | |
| 824 | − return 0; | |
| 825 | − } | |
| 826 | − return agp_remove_client(reserve.pid); | |
| 827 | − } else { | |
| 828 | − struct agp_segment *segment; | |
| 829 | − | |
| 830 | − if (reserve.seg_count >= 16384) | |
| 831 | − return -EINVAL; | |
| 832 | − | |
| 833 | − segment = kmalloc((sizeof(struct agp_segment) * reserve.seg_count), | |
| 834 | − GFP_KERNEL); | |
| 835 | − | |
| 836 | − if (segment == NULL) | |
| 837 | − return -ENOMEM; | |
| 838 | − | |
| 839 | − if (copy_from_user(segment, (void __user *) reserve.seg_list, | |
| 840 | − sizeof(struct agp_segment) * reserve.seg_count)) { | |
| 841 | − kfree(segment); | |
| 842 | − return -EFAULT; | |
| 843 | − } | |
| 844 | − reserve.seg_list = segment; | |
| 845 | − | |
| 846 | − if (client == NULL) { | |
| 847 | − /* Create the client and add the segment */ | |
| 848 | − client = agp_create_client(reserve.pid); | |
| 849 | − | |
| 850 | − if (client == NULL) { | |
| 851 | − kfree(segment); | |
| 852 | − return -ENOMEM; | |
| 853 | − } | |
| 854 | − client_priv = agp_find_private(reserve.pid); | |
| 855 | − | |
| 856 | − if (client_priv != NULL) { | |
| 857 | − set_bit(AGP_FF_IS_CLIENT, &client_priv->access_flags); | |
| 858 | − set_bit(AGP_FF_IS_VALID, &client_priv->access_flags); | |
| 859 | − } | |
| 860 | − } | |
| 861 | − return agp_create_segment(client, &reserve); | |
| 862 | − } | |
| 863 | − /* Will never really happen */ | |
| 864 | − return -EINVAL; | |
| 865 | −} | |
| 866 | − | |
| 867 | −int agpioc_protect_wrap(struct agp_file_private *priv) | |
| 868 | −{ | |
| 869 | − DBG(""); | |
| 870 | − /* This function is not currently implemented */ | |
| 871 | − return -EINVAL; | |
| 872 | −} | |
| 873 | − | |
| 874 | −static int agpioc_allocate_wrap(struct agp_file_private *priv, void __user *arg) | |
| 875 | −{ | |
| 876 | − struct agp_memory *memory; | |
| 877 | − struct agp_allocate alloc; | |
| 878 | − | |
| 879 | − DBG(""); | |
| 880 | − if (copy_from_user(&alloc, arg, sizeof(struct agp_allocate))) | |
| 881 | − return -EFAULT; | |
| 882 | − | |
| 883 | − if (alloc.type >= AGP_USER_TYPES) | |
| 884 | − return -EINVAL; | |
| 885 | − | |
| 886 | − memory = agp_allocate_memory_wrap(alloc.pg_count, alloc.type); | |
| 887 | − | |
| 888 | − if (memory == NULL) | |
| 889 | − return -ENOMEM; | |
| 890 | − | |
| 891 | − alloc.key = memory->key; | |
| 892 | − alloc.physical = memory->physical; | |
| 893 | − | |
| 894 | − if (copy_to_user(arg, &alloc, sizeof(struct agp_allocate))) { | |
| 895 | − agp_free_memory_wrap(memory); | |
| 896 | − return -EFAULT; | |
| 897 | − } | |
| 898 | − return 0; | |
| 899 | −} | |
| 900 | − | |
| 901 | −int agpioc_deallocate_wrap(struct agp_file_private *priv, int arg) | |
| 902 | −{ | |
| 903 | − struct agp_memory *memory; | |
| 904 | − | |
| 905 | − DBG(""); | |
| 906 | − memory = agp_find_mem_by_key(arg); | |
| 907 | − | |
| 908 | − if (memory == NULL) | |
| 909 | − return -EINVAL; | |
| 910 | − | |
| 911 | − agp_free_memory_wrap(memory); | |
| 912 | − return 0; | |
| 913 | −} | |
| 914 | − | |
| 915 | −static int agpioc_bind_wrap(struct agp_file_private *priv, void __user *arg) | |
| 916 | −{ | |
| 917 | − struct agp_bind bind_info; | |
| 918 | − struct agp_memory *memory; | |
| 919 | − | |
| 920 | − DBG(""); | |
| 921 | − if (copy_from_user(&bind_info, arg, sizeof(struct agp_bind))) | |
| 922 | − return -EFAULT; | |
| 923 | − | |
| 924 | − memory = agp_find_mem_by_key(bind_info.key); | |
| 925 | − | |
| 926 | − if (memory == NULL) | |
| 927 | − return -EINVAL; | |
| 928 | − | |
| 929 | − return agp_bind_memory(memory, bind_info.pg_start); | |
| 930 | −} | |
| 931 | − | |
| 932 | −static int agpioc_unbind_wrap(struct agp_file_private *priv, void __user *arg) | |
| 933 | −{ | |
| 934 | − struct agp_memory *memory; | |
| 935 | − struct agp_unbind unbind; | |
| 936 | − | |
| 937 | − DBG(""); | |
| 938 | − if (copy_from_user(&unbind, arg, sizeof(struct agp_unbind))) | |
| 939 | − return -EFAULT; | |
| 940 | − | |
| 941 | − memory = agp_find_mem_by_key(unbind.key); | |
| 942 | − | |
| 943 | − if (memory == NULL) | |
| 944 | − return -EINVAL; | |
| 945 | − | |
| 946 | − return agp_unbind_memory(memory); | |
| 947 | −} | |
| 948 | − | |
| 949 | −static long agp_ioctl(struct file *file, | |
| 950 | − unsigned int cmd, unsigned long arg) | |
| 951 | −{ | |
| 952 | − struct agp_file_private *curr_priv = file->private_data; | |
| 953 | − int ret_val = -ENOTTY; | |
| 954 | − | |
| 955 | − DBG("priv=%p, cmd=%x", curr_priv, cmd); | |
| 956 | − mutex_lock(&(agp_fe.agp_mutex)); | |
| 957 | − | |
| 958 | − if ((agp_fe.current_controller == NULL) && | |
| 959 | − (cmd != AGPIOC_ACQUIRE)) { | |
| 960 | − ret_val = -EINVAL; | |
| 961 | − goto ioctl_out; | |
| 962 | − } | |
| 963 | − if ((agp_fe.backend_acquired != true) && | |
| 964 | − (cmd != AGPIOC_ACQUIRE)) { | |
| 965 | − ret_val = -EBUSY; | |
| 966 | − goto ioctl_out; | |
| 967 | − } | |
| 968 | − if (cmd != AGPIOC_ACQUIRE) { | |
| 969 | − if (!(test_bit(AGP_FF_IS_CONTROLLER, &curr_priv->access_flags))) { | |
| 970 | − ret_val = -EPERM; | |
| 971 | − goto ioctl_out; | |
| 972 | − } | |
| 973 | − /* Use the original pid of the controller, | |
| 974 | − * in case it's threaded */ | |
| 975 | − | |
| 976 | − if (agp_fe.current_controller->pid != curr_priv->my_pid) { | |
| 977 | − ret_val = -EBUSY; | |
| 978 | − goto ioctl_out; | |
| 979 | − } | |
| 980 | − } | |
| 981 | − | |
| 982 | − switch (cmd) { | |
| 983 | − case AGPIOC_INFO: | |
| 984 | − ret_val = agpioc_info_wrap(curr_priv, (void __user *) arg); | |
| 985 | − break; | |
| 986 | − | |
| 987 | − case AGPIOC_ACQUIRE: | |
| 988 | − ret_val = agpioc_acquire_wrap(curr_priv); | |
| 989 | − break; | |
| 990 | − | |
| 991 | − case AGPIOC_RELEASE: | |
| 992 | − ret_val = agpioc_release_wrap(curr_priv); | |
| 993 | − break; | |
| 994 | − | |
| 995 | − case AGPIOC_SETUP: | |
| 996 | − ret_val = agpioc_setup_wrap(curr_priv, (void __user *) arg); | |
| 997 | − break; | |
| 998 | − | |
| 999 | − case AGPIOC_RESERVE: | |
| 1000 | − ret_val = agpioc_reserve_wrap(curr_priv, (void __user *) arg); | |
| 1001 | − break; | |
| 1002 | − | |
| 1003 | − case AGPIOC_PROTECT: | |
| 1004 | − ret_val = agpioc_protect_wrap(curr_priv); | |
| 1005 | − break; | |
| 1006 | − | |
| 1007 | − case AGPIOC_ALLOCATE: | |
| 1008 | − ret_val = agpioc_allocate_wrap(curr_priv, (void __user *) arg); | |
| 1009 | − break; | |
| 1010 | − | |
| 1011 | − case AGPIOC_DEALLOCATE: | |
| 1012 | − ret_val = agpioc_deallocate_wrap(curr_priv, (int) arg); | |
| 1013 | − break; | |
| 1014 | − | |
| 1015 | − case AGPIOC_BIND: | |
| 1016 | − ret_val = agpioc_bind_wrap(curr_priv, (void __user *) arg); | |
| 1017 | − break; | |
| 1018 | − | |
| 1019 | − case AGPIOC_UNBIND: | |
| 1020 | − ret_val = agpioc_unbind_wrap(curr_priv, (void __user *) arg); | |
| 1021 | − break; | |
| 1022 | − | |
| 1023 | − case AGPIOC_CHIPSET_FLUSH: | |
| 1024 | − break; | |
| 1025 | − } | |
| 1026 | − | |
| 1027 | −ioctl_out: | |
| 1028 | − DBG("ioctl returns %d\n", ret_val); | |
| 1029 | − mutex_unlock(&(agp_fe.agp_mutex)); | |
| 1030 | − return ret_val; | |
| 1031 | −} | |
| 1032 | − | |
| 1033 | −static const struct file_operations agp_fops = | |
| 1034 | −{ | |
| 1035 | − .owner = THIS_MODULE, | |
| 1036 | − .llseek = no_llseek, | |
| 1037 | − .unlocked_ioctl = agp_ioctl, | |
| 1038 | −#ifdef CONFIG_COMPAT | |
| 1039 | − .compat_ioctl = compat_agp_ioctl, | |
| 1040 | −#endif | |
| 1041 | − .mmap = agp_mmap, | |
| 1042 | − .open = agp_open, | |
| 1043 | − .release = agp_release, | |
| 1044 | −}; | |
| 1045 | − | |
| 1046 | −static struct miscdevice agp_miscdev = | |
| 1047 | −{ | |
| 1048 | − .minor = AGPGART_MINOR, | |
| 1049 | − .name = "agpgart", | |
| 1050 | − .fops = &agp_fops | |
| 1051 | −}; | |
| 1052 | − | |
| 1053 | −int agp_frontend_initialize(void) | |
| 1054 | −{ | |
| 1055 | − memset(&agp_fe, 0, sizeof(struct agp_front_data)); | |
| 1056 | − mutex_init(&(agp_fe.agp_mutex)); | |
| 1057 | − | |
| 1058 | − if (misc_register(&agp_miscdev)) { | |
| 1059 | − printk(KERN_ERR PFX "unable to get minor: %d\n", AGPGART_MINOR); | |
| 1060 | − return -EIO; | |
| 1061 | − } | |
| 1062 | − return 0; | |
| 1063 | −} | |
| 1064 | − | |
| 1065 | −void agp_frontend_cleanup(void) | |
| 1066 | −{ | |
| 1067 | − misc_deregister(&agp_miscdev); | |
| 1068 | −} | |
| @@ -46,12 +46,12 @@ static char *dmabuffs_dname(struct dentry *dentry, char *buffer, int buflen) · 45 unmodified lines | ||
| 46 | 46 | { |
| 47 | 47 | struct dma_buf *dmabuf; |
| 48 | 48 | char name[DMA_BUF_NAME_LEN]; |
| 49 | − size_t ret = 0; | |
| 49 | + ssize_t ret = 0; | |
| 50 | 50 | |
| 51 | 51 | dmabuf = dentry->d_fsdata; |
| 52 | 52 | spin_lock(&dmabuf->name_lock); |
| 53 | 53 | if (dmabuf->name) |
| 54 | − ret = strlcpy(name, dmabuf->name, DMA_BUF_NAME_LEN); | |
| 54 | + ret = strscpy(name, dmabuf->name, sizeof(name)); | |
| 55 | 55 | spin_unlock(&dmabuf->name_lock); |
| 56 | 56 | |
| 57 | 57 | return dynamic_dname(buffer, buflen, "/%s:%s", |
| @@ -934,7 +934,8 @@ EXPORT_SYMBOL(dma_fence_wait_any_timeout); · 933 unmodified lines | ||
| 934 | 934 | * the GPU's devfreq to reduce frequency, when in fact the opposite is what is |
| 935 | 935 | * needed. |
| 936 | 936 | * |
| 937 | − * To this end, deadline hint(s) can be set on a &dma_fence via &dma_fence_set_deadline. | |
| 937 | + * To this end, deadline hint(s) can be set on a &dma_fence via &dma_fence_set_deadline | |
| 938 | + * (or indirectly via userspace facing ioctls like &sync_set_deadline). | |
| 938 | 939 | * The deadline hint provides a way for the waiting driver, or userspace, to |
| 939 | 940 | * convey an appropriate sense of urgency to the signaling driver. |
| 940 | 941 | * |
| @@ -52,12 +52,33 @@ struct sw_sync_create_fence_data { · 51 unmodified lines | ||
| 52 | 52 | __s32 fence; /* fd of new fence */ |
| 53 | 53 | }; |
| 54 | 54 | |
| 55 | +/** | |
| 56 | + * struct sw_sync_get_deadline - get the deadline hint of a sw_sync fence | |
| 57 | + * @deadline_ns: absolute time of the deadline | |
| 58 | + * @pad: must be zero | |
| 59 | + * @fence_fd: the sw_sync fence fd (in) | |
| 60 | + * | |
| 61 | + * Return the earliest deadline set on the fence. The timebase for the | |
| 62 | + * deadline is CLOCK_MONOTONIC (same as vblank). If there is no deadline | |
| 63 | + * set on the fence, this ioctl will return -ENOENT. | |
| 64 | + */ | |
| 65 | +struct sw_sync_get_deadline { | |
| 66 | + __u64 deadline_ns; | |
| 67 | + __u32 pad; | |
| 68 | + __s32 fence_fd; | |
| 69 | +}; | |
| 70 | + | |
| 55 | 71 | #define SW_SYNC_IOC_MAGIC 'W' |
| 56 | 72 | |
| 57 | 73 | #define SW_SYNC_IOC_CREATE_FENCE _IOWR(SW_SYNC_IOC_MAGIC, 0,\ |
| 58 | 74 | struct sw_sync_create_fence_data) |
| 59 | 75 | |
| 60 | 76 | #define SW_SYNC_IOC_INC _IOW(SW_SYNC_IOC_MAGIC, 1, __u32) |
| 77 | +#define SW_SYNC_GET_DEADLINE _IOWR(SW_SYNC_IOC_MAGIC, 2, \ | |
| 78 | + struct sw_sync_get_deadline) | |
| 79 | + | |
| 80 | + | |
| 81 | +#define SW_SYNC_HAS_DEADLINE_BIT DMA_FENCE_FLAG_USER_BITS | |
| 61 | 82 | |
| 62 | 83 | static const struct dma_fence_ops timeline_fence_ops; |
| 63 | 84 | |
| @@ -171,6 +192,22 @@ static void timeline_fence_timeline_value_str(struct dma_fence *fence, · 107 unmodified lines | ||
| 171 | 192 | snprintf(str, size, "%d", parent->value); |
| 172 | 193 | } |
| 173 | 194 | |
| 195 | +static void timeline_fence_set_deadline(struct dma_fence *fence, ktime_t deadline) | |
| 196 | +{ | |
| 197 | + struct sync_pt *pt = dma_fence_to_sync_pt(fence); | |
| 198 | + unsigned long flags; | |
| 199 | + | |
| 200 | + spin_lock_irqsave(fence->lock, flags); | |
| 201 | + if (test_bit(SW_SYNC_HAS_DEADLINE_BIT, &fence->flags)) { | |
| 202 | + if (ktime_before(deadline, pt->deadline)) | |
| 203 | + pt->deadline = deadline; | |
| 204 | + } else { | |
| 205 | + pt->deadline = deadline; | |
| 206 | + __set_bit(SW_SYNC_HAS_DEADLINE_BIT, &fence->flags); | |
| 207 | + } | |
| 208 | + spin_unlock_irqrestore(fence->lock, flags); | |
| 209 | +} | |
| 210 | + | |
| 174 | 211 | static const struct dma_fence_ops timeline_fence_ops = { |
| 175 | 212 | .get_driver_name = timeline_fence_get_driver_name, |
| 176 | 213 | .get_timeline_name = timeline_fence_get_timeline_name, |
| @@ -179,6 +216,7 @@ static const struct dma_fence_ops timeline_fence_ops = { · 2 unmodified lines | ||
| 179 | 216 | .release = timeline_fence_release, |
| 180 | 217 | .fence_value_str = timeline_fence_value_str, |
| 181 | 218 | .timeline_value_str = timeline_fence_timeline_value_str, |
| 219 | + .set_deadline = timeline_fence_set_deadline, | |
| 182 | 220 | }; |
| 183 | 221 | |
| 184 | 222 | /** |
| @@ -387,6 +425,47 @@ static long sw_sync_ioctl_inc(struct sync_timeline *obj, unsigned long arg) · 202 unmodified lines | ||
| 387 | 425 | return 0; |
| 388 | 426 | } |
| 389 | 427 | |
| 428 | +static int sw_sync_ioctl_get_deadline(struct sync_timeline *obj, unsigned long arg) | |
| 429 | +{ | |
| 430 | + struct sw_sync_get_deadline data; | |
| 431 | + struct dma_fence *fence; | |
| 432 | + unsigned long flags; | |
| 433 | + struct sync_pt *pt; | |
| 434 | + int ret = 0; | |
| 435 | + | |
| 436 | + if (copy_from_user(&data, (void __user *)arg, sizeof(data))) | |
| 437 | + return -EFAULT; | |
| 438 | + | |
| 439 | + if (data.deadline_ns || data.pad) | |
| 440 | + return -EINVAL; | |
| 441 | + | |
| 442 | + fence = sync_file_get_fence(data.fence_fd); | |
| 443 | + if (!fence) | |
| 444 | + return -EINVAL; | |
| 445 | + | |
| 446 | + pt = dma_fence_to_sync_pt(fence); | |
| 447 | + if (!pt) | |
| 448 | + return -EINVAL; | |
| 449 | + | |
| 450 | + spin_lock_irqsave(fence->lock, flags); | |
| 451 | + if (test_bit(SW_SYNC_HAS_DEADLINE_BIT, &fence->flags)) { | |
| 452 | + data.deadline_ns = ktime_to_ns(pt->deadline); | |
| 453 | + } else { | |
| 454 | + ret = -ENOENT; | |
| 455 | + } | |
| 456 | + spin_unlock_irqrestore(fence->lock, flags); | |
| 457 | + | |
| 458 | + dma_fence_put(fence); | |
| 459 | + | |
| 460 | + if (ret) | |
| 461 | + return ret; | |
| 462 | + | |
| 463 | + if (copy_to_user((void __user *)arg, &data, sizeof(data))) | |
| 464 | + return -EFAULT; | |
| 465 | + | |
| 466 | + return 0; | |
| 467 | +} | |
| 468 | + | |
| 390 | 469 | static long sw_sync_ioctl(struct file *file, unsigned int cmd, |
| 391 | 470 | unsigned long arg) |
| 392 | 471 | { |
| @@ -399,6 +478,9 @@ static long sw_sync_ioctl(struct file *file, unsigned int cmd, · 6 unmodified lines | ||
| 399 | 478 | case SW_SYNC_IOC_INC: |
| 400 | 479 | return sw_sync_ioctl_inc(obj, arg); |
| 401 | 480 | |
| 481 | + case SW_SYNC_GET_DEADLINE: | |
| 482 | + return sw_sync_ioctl_get_deadline(obj, arg); | |
| 483 | + | |
| 402 | 484 | default: |
| 403 | 485 | return -ENOTTY; |
| 404 | 486 | } |
| @@ -55,11 +55,13 @@ static inline struct sync_timeline *dma_fence_parent(struct dma_fence *fence) · 54 unmodified lines | ||
| 55 | 55 | * @base: base fence object |
| 56 | 56 | * @link: link on the sync timeline's list |
| 57 | 57 | * @node: node in the sync timeline's tree |
| 58 | + * @deadline: the earliest fence deadline hint | |
| 58 | 59 | */ |
| 59 | 60 | struct sync_pt { |
| 60 | 61 | struct dma_fence base; |
| 61 | 62 | struct list_head link; |
| 62 | 63 | struct rb_node node; |
| 64 | + ktime_t deadline; | |
| 63 | 65 | }; |
| 64 | 66 | |
| 65 | 67 | extern const struct file_operations sw_sync_debugfs_fops; |
| @@ -347,6 +347,22 @@ out: · 346 unmodified lines | ||
| 347 | 347 | return ret; |
| 348 | 348 | } |
| 349 | 349 | |
| 350 | +static int sync_file_ioctl_set_deadline(struct sync_file *sync_file, | |
| 351 | + unsigned long arg) | |
| 352 | +{ | |
| 353 | + struct sync_set_deadline ts; | |
| 354 | + | |
| 355 | + if (copy_from_user(&ts, (void __user *)arg, sizeof(ts))) | |
| 356 | + return -EFAULT; | |
| 357 | + | |
| 358 | + if (ts.pad) | |
| 359 | + return -EINVAL; | |
| 360 | + | |
| 361 | + dma_fence_set_deadline(sync_file->fence, ns_to_ktime(ts.deadline_ns)); | |
| 362 | + | |
| 363 | + return 0; | |
| 364 | +} | |
| 365 | + | |
| 350 | 366 | static long sync_file_ioctl(struct file *file, unsigned int cmd, |
| 351 | 367 | unsigned long arg) |
| 352 | 368 | { |
| @@ -359,6 +375,9 @@ static long sync_file_ioctl(struct file *file, unsigned int cmd, · 6 unmodified lines | ||
| 359 | 375 | case SYNC_IOC_FILE_INFO: |
| 360 | 376 | return sync_file_ioctl_fence_info(sync_file, arg); |
| 361 | 377 | |
| 378 | + case SYNC_IOC_SET_DEADLINE: | |
| 379 | + return sync_file_ioctl_set_deadline(sync_file, arg); | |
| 380 | + | |
| 362 | 381 | default: |
| 363 | 382 | return -ENOTTY; |
| 364 | 383 | } |
| @@ -1093,9 +1093,10 @@ int dpll_nl_pin_id_get_doit(struct sk_buff *skb, struct genl_info *info) · 1092 unmodified lines | ||
| 1093 | 1093 | return -ENOMEM; |
| 1094 | 1094 | hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0, |
| 1095 | 1095 | DPLL_CMD_PIN_ID_GET); |
| 1096 | − if (!hdr) | |
| 1096 | + if (!hdr) { | |
| 1097 | + nlmsg_free(msg); | |
| 1097 | 1098 | return -EMSGSIZE; |
| 1098 | − | |
| 1099 | + } | |
| 1099 | 1100 | pin = dpll_pin_find_from_nlattr(info); |
| 1100 | 1101 | if (!IS_ERR(pin)) { |
| 1101 | 1102 | ret = dpll_msg_add_pin_handle(msg, pin); |
| @@ -1123,8 +1124,10 @@ int dpll_nl_pin_get_doit(struct sk_buff *skb, struct genl_info *info) · 21 unmodified lines | ||
| 1123 | 1124 | return -ENOMEM; |
| 1124 | 1125 | hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0, |
| 1125 | 1126 | DPLL_CMD_PIN_GET); |
| 1126 | − if (!hdr) | |
| 1127 | + if (!hdr) { | |
| 1128 | + nlmsg_free(msg); | |
| 1127 | 1129 | return -EMSGSIZE; |
| 1130 | + } | |
| 1128 | 1131 | ret = dpll_cmd_pin_get_one(msg, pin, info->extack); |
| 1129 | 1132 | if (ret) { |
| 1130 | 1133 | nlmsg_free(msg); |
| @@ -1256,8 +1259,10 @@ int dpll_nl_device_id_get_doit(struct sk_buff *skb, struct genl_info *info) · 125 unmodified lines | ||
| 1256 | 1259 | return -ENOMEM; |
| 1257 | 1260 | hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0, |
| 1258 | 1261 | DPLL_CMD_DEVICE_ID_GET); |
| 1259 | − if (!hdr) | |
| 1262 | + if (!hdr) { | |
| 1263 | + nlmsg_free(msg); | |
| 1260 | 1264 | return -EMSGSIZE; |
| 1265 | + } | |
| 1261 | 1266 | |
| 1262 | 1267 | dpll = dpll_device_find_from_nlattr(info); |
| 1263 | 1268 | if (!IS_ERR(dpll)) { |
| @@ -1284,8 +1289,10 @@ int dpll_nl_device_get_doit(struct sk_buff *skb, struct genl_info *info) · 20 unmodified lines | ||
| 1284 | 1289 | return -ENOMEM; |
| 1285 | 1290 | hdr = genlmsg_put_reply(msg, info, &dpll_nl_family, 0, |
| 1286 | 1291 | DPLL_CMD_DEVICE_GET); |
| 1287 | − if (!hdr) | |
| 1292 | + if (!hdr) { | |
| 1293 | + nlmsg_free(msg); | |
| 1288 | 1294 | return -EMSGSIZE; |
| 1295 | + } | |
| 1289 | 1296 | |
| 1290 | 1297 | ret = dpll_device_get_one(dpll, msg, info->extack); |
| 1291 | 1298 | if (ret) { |
| @@ -131,7 +131,7 @@ config RASPBERRYPI_FIRMWARE · 130 unmodified lines | ||
| 131 | 131 | |
| 132 | 132 | config FW_CFG_SYSFS |
| 133 | 133 | tristate "QEMU fw_cfg device support in sysfs" |
| 134 | − depends on SYSFS && (ARM || ARM64 || PARISC || PPC_PMAC || SPARC || X86) | |
| 134 | + depends on SYSFS && (ARM || ARM64 || PARISC || PPC_PMAC || RISCV || SPARC || X86) | |
| 135 | 135 | depends on HAS_IOPORT_MAP |
| 136 | 136 | default n |
| 137 | 137 | help |
| @@ -211,7 +211,7 @@ static void fw_cfg_io_cleanup(void) · 210 unmodified lines | ||
| 211 | 211 | |
| 212 | 212 | /* arch-specific ctrl & data register offsets are not available in ACPI, DT */ |
| 213 | 213 | #if !(defined(FW_CFG_CTRL_OFF) && defined(FW_CFG_DATA_OFF)) |
| 214 | −# if (defined(CONFIG_ARM) || defined(CONFIG_ARM64)) | |
| 214 | +# if (defined(CONFIG_ARM) || defined(CONFIG_ARM64) || defined(CONFIG_RISCV)) | |
| 215 | 215 | # define FW_CFG_CTRL_OFF 0x08 |
| 216 | 216 | # define FW_CFG_DATA_OFF 0x00 |
| 217 | 217 | # define FW_CFG_DMA_OFF 0x10 |
| @@ -74,16 +74,17 @@ config DRM_KUNIT_TEST_HELPERS · 73 unmodified lines | ||
| 74 | 74 | |
| 75 | 75 | config DRM_KUNIT_TEST |
| 76 | 76 | tristate "KUnit tests for DRM" if !KUNIT_ALL_TESTS |
| 77 | − depends on DRM && KUNIT | |
| 78 | − select PRIME_NUMBERS | |
| 77 | + depends on DRM && KUNIT && MMU | |
| 78 | + select DRM_BUDDY | |
| 79 | 79 | select DRM_DISPLAY_DP_HELPER |
| 80 | 80 | select DRM_DISPLAY_HELPER |
| 81 | − select DRM_LIB_RANDOM | |
| 82 | − select DRM_KMS_HELPER | |
| 83 | − select DRM_BUDDY | |
| 81 | + select DRM_EXEC | |
| 84 | 82 | select DRM_EXPORT_FOR_TESTS if m |
| 83 | + select DRM_GEM_SHMEM_HELPER | |
| 84 | + select DRM_KMS_HELPER | |
| 85 | 85 | select DRM_KUNIT_TEST_HELPERS |
| 86 | − select DRM_EXEC | |
| 86 | + select DRM_LIB_RANDOM | |
| 87 | + select PRIME_NUMBERS | |
| 87 | 88 | default KUNIT_ALL_TESTS |
| 88 | 89 | help |
| 89 | 90 | This builds unit tests for DRM. This option is not useful for |
| @@ -394,6 +395,8 @@ source "drivers/gpu/drm/solomon/Kconfig" · 304 unmodified lines | ||
| 394 | 395 | |
| 395 | 396 | source "drivers/gpu/drm/sprd/Kconfig" |
| 396 | 397 | |
| 398 | +source "drivers/gpu/drm/imagination/Kconfig" | |
| 399 | + | |
| 397 | 400 | config DRM_HYPERV |
| 398 | 401 | tristate "DRM Support for Hyper-V synthetic video device" |
| 399 | 402 | depends on DRM && PCI && MMU && HYPERV |
| @@ -407,27 +410,6 @@ config DRM_HYPERV · 7 unmodified lines | ||
| 407 | 410 | |
| 408 | 411 | If M is selected the module will be called hyperv_drm. |
| 409 | 412 | |
| 410 | −# Keep legacy drivers last | |
| 411 | − | |
| 412 | −menuconfig DRM_LEGACY | |
| 413 | − bool "Enable legacy drivers (DANGEROUS)" | |
| 414 | − depends on DRM && MMU | |
| 415 | − help | |
| 416 | − Enable legacy DRI1 drivers. Those drivers expose unsafe and dangerous | |
| 417 | − APIs to user-space, which can be used to circumvent access | |
| 418 | − restrictions and other security measures. For backwards compatibility | |
| 419 | − those drivers are still available, but their use is highly | |
| 420 | − inadvisable and might harm your system. | |
| 421 | − | |
| 422 | − You are recommended to use the safe modeset-only drivers instead, and | |
| 423 | − perform 3D emulation in user-space. | |
| 424 | − | |
| 425 | − Unless you have strong reasons to go rogue, say "N". | |
| 426 | − | |
| 427 | −if DRM_LEGACY | |
| 428 | −# leave here to list legacy drivers | |
| 429 | −endif # DRM_LEGACY | |
| 430 | − | |
| 431 | 413 | config DRM_EXPORT_FOR_TESTS |
| 432 | 414 | bool |
| 433 | 415 | |
| @@ -22,6 +22,7 @@ drm-y := \ · 21 unmodified lines | ||
| 22 | 22 | drm_drv.o \ |
| 23 | 23 | drm_dumb_buffers.o \ |
| 24 | 24 | drm_edid.o \ |
| 25 | + drm_eld.o \ | |
| 25 | 26 | drm_encoder.o \ |
| 26 | 27 | drm_file.o \ |
| 27 | 28 | drm_fourcc.o \ |
| @@ -46,18 +47,6 @@ drm-y := \ · 18 unmodified lines | ||
| 46 | 47 | drm_vblank_work.o \ |
| 47 | 48 | drm_vma_manager.o \ |
| 48 | 49 | drm_writeback.o |
| 49 | −drm-$(CONFIG_DRM_LEGACY) += \ | |
| 50 | − drm_agpsupport.o \ | |
| 51 | − drm_bufs.o \ | |
| 52 | − drm_context.o \ | |
| 53 | − drm_dma.o \ | |
| 54 | − drm_hashtab.o \ | |
| 55 | − drm_irq.o \ | |
| 56 | − drm_legacy_misc.o \ | |
| 57 | − drm_lock.o \ | |
| 58 | − drm_memory.o \ | |
| 59 | − drm_scatter.o \ | |
| 60 | − drm_vm.o | |
| 61 | 50 | drm-$(CONFIG_DRM_LIB_RANDOM) += lib/drm_random.o |
| 62 | 51 | drm-$(CONFIG_COMPAT) += drm_ioc32.o |
| 63 | 52 | drm-$(CONFIG_DRM_PANEL) += drm_panel.o |
| @@ -198,3 +187,4 @@ obj-$(CONFIG_DRM_HYPERV) += hyperv/ · 134 unmodified lines | ||
| 198 | 187 | obj-y += solomon/ |
| 199 | 188 | obj-$(CONFIG_DRM_SPRD) += sprd/ |
| 200 | 189 | obj-$(CONFIG_DRM_LOONGSON) += loongson/ |
| 190 | +obj-$(CONFIG_DRM_POWERVR) += imagination/ | |
| @@ -248,6 +248,7 @@ extern int amdgpu_umsch_mm; · 247 unmodified lines | ||
| 248 | 248 | extern int amdgpu_seamless; |
| 249 | 249 | |
| 250 | 250 | extern int amdgpu_user_partt_mode; |
| 251 | +extern int amdgpu_agp; | |
| 251 | 252 | |
| 252 | 253 | #define AMDGPU_VM_MAX_NUM_CTX 4096 |
| 253 | 254 | #define AMDGPU_SG_THRESHOLD (256*1024*1024) |
| @@ -290,7 +290,7 @@ static int suspend_resume_compute_scheduler(struct amdgpu_device *adev, bool sus · 289 unmodified lines | ||
| 290 | 290 | for (i = 0; i < adev->gfx.num_compute_rings; i++) { |
| 291 | 291 | struct amdgpu_ring *ring = &adev->gfx.compute_ring[i]; |
| 292 | 292 | |
| 293 | − if (!(ring && ring->sched.thread)) | |
| 293 | + if (!(ring && drm_sched_wqueue_ready(&ring->sched))) | |
| 294 | 294 | continue; |
| 295 | 295 | |
| 296 | 296 | /* stop secheduler and drain ring. */ |
| @@ -207,7 +207,7 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p, · 206 unmodified lines | ||
| 207 | 207 | } |
| 208 | 208 | |
| 209 | 209 | for (i = 0; i < p->nchunks; i++) { |
| 210 | − struct drm_amdgpu_cs_chunk __user **chunk_ptr = NULL; | |
| 210 | + struct drm_amdgpu_cs_chunk __user *chunk_ptr = NULL; | |
| 211 | 211 | struct drm_amdgpu_cs_chunk user_chunk; |
| 212 | 212 | uint32_t __user *cdata; |
| 213 | 213 | |
| @@ -73,10 +73,10 @@ amdgpu_ctx_to_drm_sched_prio(int32_t ctx_prio) · 72 unmodified lines | ||
| 73 | 73 | return DRM_SCHED_PRIORITY_NORMAL; |
| 74 | 74 | |
| 75 | 75 | case AMDGPU_CTX_PRIORITY_VERY_LOW: |
| 76 | − return DRM_SCHED_PRIORITY_MIN; | |
| 76 | + return DRM_SCHED_PRIORITY_LOW; | |
| 77 | 77 | |
| 78 | 78 | case AMDGPU_CTX_PRIORITY_LOW: |
| 79 | − return DRM_SCHED_PRIORITY_MIN; | |
| 79 | + return DRM_SCHED_PRIORITY_LOW; | |
| 80 | 80 | |
| 81 | 81 | case AMDGPU_CTX_PRIORITY_NORMAL: |
| 82 | 82 | return DRM_SCHED_PRIORITY_NORMAL; |
| @@ -1665,9 +1665,9 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused) · 1664 unmodified lines | ||
| 1665 | 1665 | for (i = 0; i < AMDGPU_MAX_RINGS; i++) { |
| 1666 | 1666 | struct amdgpu_ring *ring = adev->rings[i]; |
| 1667 | 1667 | |
| 1668 | − if (!ring || !ring->sched.thread) | |
| 1668 | + if (!ring || !drm_sched_wqueue_ready(&ring->sched)) | |
| 1669 | 1669 | continue; |
| 1670 | − kthread_park(ring->sched.thread); | |
| 1670 | + drm_sched_wqueue_stop(&ring->sched); | |
| 1671 | 1671 | } |
| 1672 | 1672 | |
| 1673 | 1673 | seq_puts(m, "run ib test:\n"); |
| @@ -1681,9 +1681,9 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused) · 7 unmodified lines | ||
| 1681 | 1681 | for (i = 0; i < AMDGPU_MAX_RINGS; i++) { |
| 1682 | 1682 | struct amdgpu_ring *ring = adev->rings[i]; |
| 1683 | 1683 | |
| 1684 | − if (!ring || !ring->sched.thread) | |
| 1684 | + if (!ring || !drm_sched_wqueue_ready(&ring->sched)) | |
| 1685 | 1685 | continue; |
| 1686 | − kthread_unpark(ring->sched.thread); | |
| 1686 | + drm_sched_wqueue_start(&ring->sched); | |
| 1687 | 1687 | } |
| 1688 | 1688 | |
| 1689 | 1689 | up_write(&adev->reset_domain->sem); |
| @@ -1903,7 +1903,8 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) · 213 unmodified lines | ||
| 1903 | 1903 | |
| 1904 | 1904 | ring = adev->rings[val]; |
| 1905 | 1905 | |
| 1906 | − if (!ring || !ring->funcs->preempt_ib || !ring->sched.thread) | |
| 1906 | + if (!ring || !ring->funcs->preempt_ib || | |
| 1907 | + !drm_sched_wqueue_ready(&ring->sched)) | |
| 1907 | 1908 | return -EINVAL; |
| 1908 | 1909 | |
| 1909 | 1910 | /* the last preemption failed */ |
| @@ -1921,7 +1922,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) · 11 unmodified lines | ||
| 1921 | 1922 | goto pro_end; |
| 1922 | 1923 | |
| 1923 | 1924 | /* stop the scheduler */ |
| 1924 | − kthread_park(ring->sched.thread); | |
| 1925 | + drm_sched_wqueue_stop(&ring->sched); | |
| 1925 | 1926 | |
| 1926 | 1927 | /* preempt the IB */ |
| 1927 | 1928 | r = amdgpu_ring_preempt_ib(ring); |
| @@ -1955,7 +1956,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) · 27 unmodified lines | ||
| 1955 | 1956 | |
| 1956 | 1957 | failure: |
| 1957 | 1958 | /* restart the scheduler */ |
| 1958 | − kthread_unpark(ring->sched.thread); | |
| 1959 | + drm_sched_wqueue_start(&ring->sched); | |
| 1959 | 1960 | |
| 1960 | 1961 | up_read(&adev->reset_domain->sem); |
| 1961 | 1962 | |
| @@ -2573,7 +2573,7 @@ static int amdgpu_device_init_schedulers(struct amdgpu_device *adev) · 2572 unmodified lines | ||
| 2573 | 2573 | break; |
| 2574 | 2574 | } |
| 2575 | 2575 | |
| 2576 | − r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, | |
| 2576 | + r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, NULL, | |
| 2577 | 2577 | DRM_SCHED_PRIORITY_COUNT, |
| 2578 | 2578 | ring->num_hw_submission, 0, |
| 2579 | 2579 | timeout, adev->reset_domain->wq, |
| @@ -4964,7 +4964,7 @@ bool amdgpu_device_has_job_running(struct amdgpu_device *adev) · 2384 unmodified lines | ||
| 4964 | 4964 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { |
| 4965 | 4965 | struct amdgpu_ring *ring = adev->rings[i]; |
| 4966 | 4966 | |
| 4967 | − if (!ring || !ring->sched.thread) | |
| 4967 | + if (!ring || !drm_sched_wqueue_ready(&ring->sched)) | |
| 4968 | 4968 | continue; |
| 4969 | 4969 | |
| 4970 | 4970 | spin_lock(&ring->sched.job_list_lock); |
| @@ -5103,7 +5103,7 @@ int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev, · 132 unmodified lines | ||
| 5103 | 5103 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { |
| 5104 | 5104 | struct amdgpu_ring *ring = adev->rings[i]; |
| 5105 | 5105 | |
| 5106 | − if (!ring || !ring->sched.thread) | |
| 5106 | + if (!ring || !drm_sched_wqueue_ready(&ring->sched)) | |
| 5107 | 5107 | continue; |
| 5108 | 5108 | |
| 5109 | 5109 | /* Clear job fence from fence drv to avoid force_completion |
| @@ -5592,7 +5592,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 482 unmodified lines | ||
| 5592 | 5592 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { |
| 5593 | 5593 | struct amdgpu_ring *ring = tmp_adev->rings[i]; |
| 5594 | 5594 | |
| 5595 | − if (!ring || !ring->sched.thread) | |
| 5595 | + if (!ring || !drm_sched_wqueue_ready(&ring->sched)) | |
| 5596 | 5596 | continue; |
| 5597 | 5597 | |
| 5598 | 5598 | drm_sched_stop(&ring->sched, job ? &job->base : NULL); |
| @@ -5668,7 +5668,7 @@ skip_hw_reset: · 69 unmodified lines | ||
| 5668 | 5668 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { |
| 5669 | 5669 | struct amdgpu_ring *ring = tmp_adev->rings[i]; |
| 5670 | 5670 | |
| 5671 | − if (!ring || !ring->sched.thread) | |
| 5671 | + if (!ring || !drm_sched_wqueue_ready(&ring->sched)) | |
| 5672 | 5672 | continue; |
| 5673 | 5673 | |
| 5674 | 5674 | drm_sched_start(&ring->sched, true); |
| @@ -5991,7 +5991,7 @@ pci_ers_result_t amdgpu_pci_error_detected(struct pci_dev *pdev, pci_channel_sta · 316 unmodified lines | ||
| 5991 | 5991 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { |
| 5992 | 5992 | struct amdgpu_ring *ring = adev->rings[i]; |
| 5993 | 5993 | |
| 5994 | − if (!ring || !ring->sched.thread) | |
| 5994 | + if (!ring || !drm_sched_wqueue_ready(&ring->sched)) | |
| 5995 | 5995 | continue; |
| 5996 | 5996 | |
| 5997 | 5997 | drm_sched_stop(&ring->sched, NULL); |
| @@ -6119,7 +6119,7 @@ void amdgpu_pci_resume(struct pci_dev *pdev) · 121 unmodified lines | ||
| 6119 | 6119 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { |
| 6120 | 6120 | struct amdgpu_ring *ring = adev->rings[i]; |
| 6121 | 6121 | |
| 6122 | − if (!ring || !ring->sched.thread) | |
| 6122 | + if (!ring || !drm_sched_wqueue_ready(&ring->sched)) | |
| 6123 | 6123 | continue; |
| 6124 | 6124 | |
| 6125 | 6125 | drm_sched_start(&ring->sched, true); |
| @@ -207,6 +207,7 @@ int amdgpu_user_partt_mode = AMDGPU_AUTO_COMPUTE_PARTITION_MODE; · 206 unmodified lines | ||
| 207 | 207 | int amdgpu_umsch_mm; |
| 208 | 208 | int amdgpu_seamless = -1; /* auto */ |
| 209 | 209 | uint amdgpu_debug_mask; |
| 210 | +int amdgpu_agp = -1; /* auto */ | |
| 210 | 211 | |
| 211 | 212 | static void amdgpu_drv_delayed_reset_work_handler(struct work_struct *work); |
| 212 | 213 | |
| @@ -961,6 +962,15 @@ module_param_named(seamless, amdgpu_seamless, int, 0444); · 748 unmodified lines | ||
| 961 | 962 | MODULE_PARM_DESC(debug_mask, "debug options for amdgpu, disabled by default"); |
| 962 | 963 | module_param_named(debug_mask, amdgpu_debug_mask, uint, 0444); |
| 963 | 964 | |
| 965 | +/** | |
| 966 | + * DOC: agp (int) | |
| 967 | + * Enable the AGP aperture. This provides an aperture in the GPU's internal | |
| 968 | + * address space for direct access to system memory. Note that these accesses | |
| 969 | + * are non-snooped, so they are only used for access to uncached memory. | |
| 970 | + */ | |
| 971 | +MODULE_PARM_DESC(agp, "AGP (-1 = auto (default), 0 = disable, 1 = enable)"); | |
| 972 | +module_param_named(agp, amdgpu_agp, int, 0444); | |
| 973 | + | |
| 964 | 974 | /* These devices are not supported by amdgpu. |
| 965 | 975 | * They are supported by the mach64, r128, radeon drivers |
| 966 | 976 | */ |
| @@ -115,7 +115,7 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 114 unmodified lines | ||
| 115 | 115 | if (!entity) |
| 116 | 116 | return 0; |
| 117 | 117 | |
| 118 | − return drm_sched_job_init(&(*job)->base, entity, owner); | |
| 118 | + return drm_sched_job_init(&(*job)->base, entity, 1, owner); | |
| 119 | 119 | } |
| 120 | 120 | |
| 121 | 121 | int amdgpu_job_alloc_with_ib(struct amdgpu_device *adev, |
| @@ -325,7 +325,7 @@ void amdgpu_job_stop_all_jobs_on_sched(struct drm_gpu_scheduler *sched) · 203 unmodified lines | ||
| 325 | 325 | int i; |
| 326 | 326 | |
| 327 | 327 | /* Signal all jobs not yet scheduled */ |
| 328 | − for (i = sched->num_rqs - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) { | |
| 328 | + for (i = DRM_SCHED_PRIORITY_KERNEL; i < sched->num_rqs; i++) { | |
| 329 | 329 | struct drm_sched_rq *rq = sched->sched_rq[i]; |
| 330 | 330 | spin_lock(&rq->lock); |
| 331 | 331 | list_for_each_entry(s_entity, &rq->entities, list) { |
| @@ -1473,6 +1473,11 @@ int psp_xgmi_get_topology_info(struct psp_context *psp, · 1472 unmodified lines | ||
| 1473 | 1473 | topology->nodes[i].num_links = (requires_reflection && topology->nodes[i].num_links) ? |
| 1474 | 1474 | topology->nodes[i].num_links : node_num_links; |
| 1475 | 1475 | } |
| 1476 | + /* popluate the connected port num info if supported and available */ | |
| 1477 | + if (ta_port_num_support && topology->nodes[i].num_links) { | |
| 1478 | + memcpy(topology->nodes[i].port_num, link_extend_info_output->nodes[i].port_num, | |
| 1479 | + sizeof(struct xgmi_connected_port_num) * TA_XGMI__MAX_PORT_NUM); | |
| 1480 | + } | |
| 1476 | 1481 | |
| 1477 | 1482 | /* reflect the topology information for bi-directionality */ |
| 1478 | 1483 | if (requires_reflection && topology->nodes[i].num_hops) |
| @@ -150,6 +150,7 @@ struct psp_xgmi_node_info { · 149 unmodified lines | ||
| 150 | 150 | uint8_t is_sharing_enabled; |
| 151 | 151 | enum ta_xgmi_assigned_sdma_engine sdma_engine; |
| 152 | 152 | uint8_t num_links; |
| 153 | + struct xgmi_connected_port_num port_num[TA_XGMI__MAX_PORT_NUM]; | |
| 153 | 154 | }; |
| 154 | 155 | |
| 155 | 156 | struct psp_xgmi_topology_info { |
| @@ -1188,7 +1188,7 @@ static int amdgpu_ras_query_error_status_helper(struct amdgpu_device *adev, · 1187 unmodified lines | ||
| 1188 | 1188 | } |
| 1189 | 1189 | |
| 1190 | 1190 | if (block_obj->hw_ops->query_ras_error_count) |
| 1191 | − block_obj->hw_ops->query_ras_error_count(adev, &err_data); | |
| 1191 | + block_obj->hw_ops->query_ras_error_count(adev, err_data); | |
| 1192 | 1192 | |
| 1193 | 1193 | if ((info->head.block == AMDGPU_RAS_BLOCK__SDMA) || |
| 1194 | 1194 | (info->head.block == AMDGPU_RAS_BLOCK__GFX) || |
| @@ -398,6 +398,7 @@ int amdgpu_uvd_sw_fini(struct amdgpu_device *adev) · 397 unmodified lines | ||
| 398 | 398 | * amdgpu_uvd_entity_init - init entity |
| 399 | 399 | * |
| 400 | 400 | * @adev: amdgpu_device pointer |
| 401 | + * @ring: amdgpu_ring pointer to check | |
| 401 | 402 | * |
| 402 | 403 | * Initialize the entity used for handle management in the kernel driver. |
| 403 | 404 | */ |
| @@ -230,6 +230,7 @@ int amdgpu_vce_sw_fini(struct amdgpu_device *adev) · 229 unmodified lines | ||
| 230 | 230 | * amdgpu_vce_entity_init - init entity |
| 231 | 231 | * |
| 232 | 232 | * @adev: amdgpu_device pointer |
| 233 | + * @ring: amdgpu_ring pointer to check | |
| 233 | 234 | * |
| 234 | 235 | * Initialize the entity used for handle management in the kernel driver. |
| 235 | 236 | */ |
| @@ -675,7 +675,7 @@ static void gmc_v10_0_vram_gtt_location(struct amdgpu_device *adev, · 674 unmodified lines | ||
| 675 | 675 | amdgpu_gmc_set_agp_default(adev, mc); |
| 676 | 676 | amdgpu_gmc_vram_location(adev, &adev->gmc, base); |
| 677 | 677 | amdgpu_gmc_gart_location(adev, mc, AMDGPU_GART_PLACEMENT_BEST_FIT); |
| 678 | − if (!amdgpu_sriov_vf(adev)) | |
| 678 | + if (!amdgpu_sriov_vf(adev) && (amdgpu_agp == 1)) | |
| 679 | 679 | amdgpu_gmc_agp_location(adev, mc); |
| 680 | 680 | |
| 681 | 681 | /* base offset of vram pages */ |
| @@ -640,8 +640,9 @@ static void gmc_v11_0_vram_gtt_location(struct amdgpu_device *adev, · 639 unmodified lines | ||
| 640 | 640 | amdgpu_gmc_set_agp_default(adev, mc); |
| 641 | 641 | amdgpu_gmc_vram_location(adev, &adev->gmc, base); |
| 642 | 642 | amdgpu_gmc_gart_location(adev, mc, AMDGPU_GART_PLACEMENT_HIGH); |
| 643 | − if (!amdgpu_sriov_vf(adev) || | |
| 644 | − (amdgpu_ip_version(adev, GC_HWIP, 0) < IP_VERSION(11, 5, 0))) | |
| 643 | + if (!amdgpu_sriov_vf(adev) && | |
| 644 | + (amdgpu_ip_version(adev, GC_HWIP, 0) < IP_VERSION(11, 5, 0)) && | |
| 645 | + (amdgpu_agp == 1)) | |
| 645 | 646 | amdgpu_gmc_agp_location(adev, mc); |
| 646 | 647 | |
| 647 | 648 | /* base offset of vram pages */ |
| @@ -1630,7 +1630,7 @@ static void gmc_v9_0_vram_gtt_location(struct amdgpu_device *adev, · 1629 unmodified lines | ||
| 1630 | 1630 | } else { |
| 1631 | 1631 | amdgpu_gmc_vram_location(adev, mc, base); |
| 1632 | 1632 | amdgpu_gmc_gart_location(adev, mc, AMDGPU_GART_PLACEMENT_BEST_FIT); |
| 1633 | − if (!amdgpu_sriov_vf(adev)) | |
| 1633 | + if (!amdgpu_sriov_vf(adev) && (amdgpu_agp == 1)) | |
| 1634 | 1634 | amdgpu_gmc_agp_location(adev, mc); |
| 1635 | 1635 | } |
| 1636 | 1636 | /* base offset of vram pages */ |
| @@ -2170,8 +2170,6 @@ static int gmc_v9_0_sw_fini(void *handle) · 533 unmodified lines | ||
| 2170 | 2170 | |
| 2171 | 2171 | if (amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 4, 3)) |
| 2172 | 2172 | amdgpu_gmc_sysfs_fini(adev); |
| 2173 | − adev->gmc.num_mem_partitions = 0; | |
| 2174 | − kfree(adev->gmc.mem_partitions); | |
| 2175 | 2173 | |
| 2176 | 2174 | amdgpu_gmc_ras_fini(adev); |
| 2177 | 2175 | amdgpu_gem_force_release(adev); |
| @@ -2185,6 +2183,9 @@ static int gmc_v9_0_sw_fini(void *handle) · 7 unmodified lines | ||
| 2185 | 2183 | amdgpu_bo_free_kernel(&adev->gmc.pdb0_bo, NULL, &adev->gmc.ptr_pdb0); |
| 2186 | 2184 | amdgpu_bo_fini(adev); |
| 2187 | 2185 | |
| 2186 | + adev->gmc.num_mem_partitions = 0; | |
| 2187 | + kfree(adev->gmc.mem_partitions); | |
| 2188 | + | |
| 2188 | 2189 | return 0; |
| 2189 | 2190 | } |
| 2190 | 2191 | |
| @@ -130,6 +130,9 @@ static void mmhub_v1_8_init_system_aperture_regs(struct amdgpu_device *adev) · 129 unmodified lines | ||
| 130 | 130 | uint64_t value; |
| 131 | 131 | int i; |
| 132 | 132 | |
| 133 | + if (amdgpu_sriov_vf(adev)) | |
| 134 | + return; | |
| 135 | + | |
| 133 | 136 | inst_mask = adev->aid_mask; |
| 134 | 137 | for_each_inst(i, inst_mask) { |
| 135 | 138 | /* Program the AGP BAR */ |
| @@ -139,9 +142,6 @@ static void mmhub_v1_8_init_system_aperture_regs(struct amdgpu_device *adev) · 3 unmodified lines | ||
| 139 | 142 | WREG32_SOC15(MMHUB, i, regMC_VM_AGP_TOP, |
| 140 | 143 | adev->gmc.agp_end >> 24); |
| 141 | 144 | |
| 142 | − if (amdgpu_sriov_vf(adev)) | |
| 143 | − return; | |
| 144 | − | |
| 145 | 145 | /* Program the system aperture low logical page number. */ |
| 146 | 146 | WREG32_SOC15(MMHUB, i, regMC_VM_SYSTEM_APERTURE_LOW_ADDR, |
| 147 | 147 | min(adev->gmc.fb_start, adev->gmc.agp_start) >> 18); |
| @@ -85,12 +85,13 @@ · 84 unmodified lines | ||
| 85 | 85 | #include <drm/drm_atomic_uapi.h> |
| 86 | 86 | #include <drm/drm_atomic_helper.h> |
| 87 | 87 | #include <drm/drm_blend.h> |
| 88 | +#include <drm/drm_fixed.h> | |
| 88 | 89 | #include <drm/drm_fourcc.h> |
| 89 | 90 | #include <drm/drm_edid.h> |
| 91 | +#include <drm/drm_eld.h> | |
| 90 | 92 | #include <drm/drm_vblank.h> |
| 91 | 93 | #include <drm/drm_audio_component.h> |
| 92 | 94 | #include <drm/drm_gem_atomic_helper.h> |
| 93 | −#include <drm/drm_plane_helper.h> | |
| 94 | 95 | |
| 95 | 96 | #include <acpi/video.h> |
| 96 | 97 | |
| @@ -2079,7 +2080,7 @@ static int dm_dmub_sw_init(struct amdgpu_device *adev) · 1982 unmodified lines | ||
| 2079 | 2080 | struct dmub_srv_create_params create_params; |
| 2080 | 2081 | struct dmub_srv_region_params region_params; |
| 2081 | 2082 | struct dmub_srv_region_info region_info; |
| 2082 | − struct dmub_srv_fb_params fb_params; | |
| 2083 | + struct dmub_srv_memory_params memory_params; | |
| 2083 | 2084 | struct dmub_srv_fb_info *fb_info; |
| 2084 | 2085 | struct dmub_srv *dmub_srv; |
| 2085 | 2086 | const struct dmcub_firmware_header_v1_0 *hdr; |
| @@ -2182,6 +2183,7 @@ static int dm_dmub_sw_init(struct amdgpu_device *adev) · 96 unmodified lines | ||
| 2182 | 2183 | adev->dm.dmub_fw->data + |
| 2183 | 2184 | le32_to_cpu(hdr->header.ucode_array_offset_bytes) + |
| 2184 | 2185 | PSP_HEADER_BYTES; |
| 2186 | + region_params.is_mailbox_in_inbox = false; | |
| 2185 | 2187 | |
| 2186 | 2188 | status = dmub_srv_calc_region_info(dmub_srv, ®ion_params, |
| 2187 | 2189 | ®ion_info); |
| @@ -2205,10 +2207,10 @@ static int dm_dmub_sw_init(struct amdgpu_device *adev) · 17 unmodified lines | ||
| 2205 | 2207 | return r; |
| 2206 | 2208 | |
| 2207 | 2209 | /* Rebase the regions on the framebuffer address. */ |
| 2208 | − memset(&fb_params, 0, sizeof(fb_params)); | |
| 2209 | − fb_params.cpu_addr = adev->dm.dmub_bo_cpu_addr; | |
| 2210 | − fb_params.gpu_addr = adev->dm.dmub_bo_gpu_addr; | |
| 2211 | − fb_params.region_info = ®ion_info; | |
| 2210 | + memset(&memory_params, 0, sizeof(memory_params)); | |
| 2211 | + memory_params.cpu_fb_addr = adev->dm.dmub_bo_cpu_addr; | |
| 2212 | + memory_params.gpu_fb_addr = adev->dm.dmub_bo_gpu_addr; | |
| 2213 | + memory_params.region_info = ®ion_info; | |
| 2212 | 2214 | |
| 2213 | 2215 | adev->dm.dmub_fb_info = |
| 2214 | 2216 | kzalloc(sizeof(*adev->dm.dmub_fb_info), GFP_KERNEL); |
| @@ -2220,7 +2222,7 @@ static int dm_dmub_sw_init(struct amdgpu_device *adev) · 5 unmodified lines | ||
| 2220 | 2222 | return -ENOMEM; |
| 2221 | 2223 | } |
| 2222 | 2224 | |
| 2223 | − status = dmub_srv_calc_fb_info(dmub_srv, &fb_params, fb_info); | |
| 2225 | + status = dmub_srv_calc_mem_info(dmub_srv, &memory_params, fb_info); | |
| 2224 | 2226 | if (status != DMUB_STATUS_OK) { |
| 2225 | 2227 | DRM_ERROR("Error calculating DMUB FB info: %d\n", status); |
| 2226 | 2228 | return -EINVAL; |
| @@ -6908,8 +6910,8 @@ static int dm_encoder_helper_atomic_check(struct drm_encoder *encoder, · 4681 unmodified lines | ||
| 6908 | 6910 | if (IS_ERR(mst_state)) |
| 6909 | 6911 | return PTR_ERR(mst_state); |
| 6910 | 6912 | |
| 6911 | − if (!mst_state->pbn_div) | |
| 6912 | − mst_state->pbn_div = dm_mst_get_pbn_divider(aconnector->mst_root->dc_link); | |
| 6913 | + if (!mst_state->pbn_div.full) | |
| 6914 | + mst_state->pbn_div.full = dfixed_const(dm_mst_get_pbn_divider(aconnector->mst_root->dc_link)); | |
| 6913 | 6915 | |
| 6914 | 6916 | if (!state->duplicated) { |
| 6915 | 6917 | int max_bpc = conn_state->max_requested_bpc; |
| @@ -6921,7 +6923,7 @@ static int dm_encoder_helper_atomic_check(struct drm_encoder *encoder, · 5 unmodified lines | ||
| 6921 | 6923 | max_bpc); |
| 6922 | 6924 | bpp = convert_dc_color_depth_into_bpc(color_depth) * 3; |
| 6923 | 6925 | clock = adjusted_mode->clock; |
| 6924 | − dm_new_connector_state->pbn = drm_dp_calc_pbn_mode(clock, bpp, false); | |
| 6926 | + dm_new_connector_state->pbn = drm_dp_calc_pbn_mode(clock, bpp << 4); | |
| 6925 | 6927 | } |
| 6926 | 6928 | |
| 6927 | 6929 | dm_new_connector_state->vcpi_slots = |
| @@ -7481,6 +7483,9 @@ static int amdgpu_dm_i2c_xfer(struct i2c_adapter *i2c_adap, · 553 unmodified lines | ||
| 7481 | 7483 | int i; |
| 7482 | 7484 | int result = -EIO; |
| 7483 | 7485 | |
| 7486 | + if (!ddc_service->ddc_pin || !ddc_service->ddc_pin->hw_info.hw_supported) | |
| 7487 | + return result; | |
| 7488 | + | |
| 7484 | 7489 | cmd.payloads = kcalloc(num, sizeof(struct i2c_payload), GFP_KERNEL); |
| 7485 | 7490 | |
| 7486 | 7491 | if (!cmd.payloads) |
| @@ -9603,14 +9608,14 @@ static bool should_reset_plane(struct drm_atomic_state *state, · 2116 unmodified lines | ||
| 9603 | 9608 | struct drm_plane *other; |
| 9604 | 9609 | struct drm_plane_state *old_other_state, *new_other_state; |
| 9605 | 9610 | struct drm_crtc_state *new_crtc_state; |
| 9611 | + struct amdgpu_device *adev = drm_to_adev(plane->dev); | |
| 9606 | 9612 | int i; |
| 9607 | 9613 | |
| 9608 | 9614 | /* |
| 9609 | − * TODO: Remove this hack once the checks below are sufficient | |
| 9610 | − * enough to determine when we need to reset all the planes on | |
| 9611 | − * the stream. | |
| 9615 | + * TODO: Remove this hack for all asics once it proves that the | |
| 9616 | + * fast updates works fine on DCN3.2+. | |
| 9612 | 9617 | */ |
| 9613 | − if (state->allow_modeset) | |
| 9618 | + if (adev->ip_versions[DCE_HWIP][0] < IP_VERSION(3, 2, 0) && state->allow_modeset) | |
| 9614 | 9619 | return true; |
| 9615 | 9620 | |
| 9616 | 9621 | /* Exit early if we know that we're adding or removing the plane. */ |
| @@ -31,6 +31,7 @@ · 30 unmodified lines | ||
| 31 | 31 | #include <drm/drm_probe_helper.h> |
| 32 | 32 | #include <drm/amdgpu_drm.h> |
| 33 | 33 | #include <drm/drm_edid.h> |
| 34 | +#include <drm/drm_fixed.h> | |
| 34 | 35 | |
| 35 | 36 | #include "dm_services.h" |
| 36 | 37 | #include "amdgpu.h" |
| @@ -210,7 +211,7 @@ static void dm_helpers_construct_old_payload( · 173 unmodified lines | ||
| 210 | 211 | struct drm_dp_mst_atomic_payload *old_payload) |
| 211 | 212 | { |
| 212 | 213 | struct drm_dp_mst_atomic_payload *pos; |
| 213 | − int pbn_per_slot = mst_state->pbn_div; | |
| 214 | + int pbn_per_slot = dfixed_trunc(mst_state->pbn_div); | |
| 214 | 215 | u8 next_payload_vc_start = mgr->next_start_slot; |
| 215 | 216 | u8 payload_vc_start = new_payload->vc_start_slot; |
| 216 | 217 | u8 allocated_time_slots; |
| @@ -536,11 +537,8 @@ bool dm_helpers_dp_read_dpcd( · 319 unmodified lines | ||
| 536 | 537 | |
| 537 | 538 | struct amdgpu_dm_connector *aconnector = link->priv; |
| 538 | 539 | |
| 539 | − if (!aconnector) { | |
| 540 | − drm_dbg_dp(aconnector->base.dev, | |
| 541 | − "Failed to find connector for link!\n"); | |
| 540 | + if (!aconnector) | |
| 542 | 541 | return false; |
| 543 | − } | |
| 544 | 542 | |
| 545 | 543 | return drm_dp_dpcd_read(&aconnector->dm_dp_aux.aux, address, data, |
| 546 | 544 | size) == size; |
| @@ -27,6 +27,7 @@ · 26 unmodified lines | ||
| 27 | 27 | #include <drm/display/drm_dp_mst_helper.h> |
| 28 | 28 | #include <drm/drm_atomic.h> |
| 29 | 29 | #include <drm/drm_atomic_helper.h> |
| 30 | +#include <drm/drm_fixed.h> | |
| 30 | 31 | #include "dm_services.h" |
| 31 | 32 | #include "amdgpu.h" |
| 32 | 33 | #include "amdgpu_dm.h" |
| @@ -941,10 +942,10 @@ static int increase_dsc_bpp(struct drm_atomic_state *state, · 908 unmodified lines | ||
| 941 | 942 | link_timeslots_used = 0; |
| 942 | 943 | |
| 943 | 944 | for (i = 0; i < count; i++) |
| 944 | − link_timeslots_used += DIV_ROUND_UP(vars[i + k].pbn, mst_state->pbn_div); | |
| 945 | + link_timeslots_used += DIV_ROUND_UP(vars[i + k].pbn, dfixed_trunc(mst_state->pbn_div)); | |
| 945 | 946 | |
| 946 | 947 | fair_pbn_alloc = |
| 947 | − (63 - link_timeslots_used) / remaining_to_increase * mst_state->pbn_div; | |
| 948 | + (63 - link_timeslots_used) / remaining_to_increase * dfixed_trunc(mst_state->pbn_div); | |
| 948 | 949 | |
| 949 | 950 | if (initial_slack[next_index] > fair_pbn_alloc) { |
| 950 | 951 | vars[next_index].pbn += fair_pbn_alloc; |
| @@ -1604,31 +1605,31 @@ enum dc_status dm_dp_mst_is_port_support_mode( · 653 unmodified lines | ||
| 1604 | 1605 | unsigned int upper_link_bw_in_kbps = 0, down_link_bw_in_kbps = 0; |
| 1605 | 1606 | unsigned int max_compressed_bw_in_kbps = 0; |
| 1606 | 1607 | struct dc_dsc_bw_range bw_range = {0}; |
| 1607 | − struct drm_dp_mst_topology_mgr *mst_mgr; | |
| 1608 | + uint16_t full_pbn = aconnector->mst_output_port->full_pbn; | |
| 1608 | 1609 | |
| 1609 | 1610 | /* |
| 1610 | − * check if the mode could be supported if DSC pass-through is supported | |
| 1611 | − * AND check if there enough bandwidth available to support the mode | |
| 1612 | − * with DSC enabled. | |
| 1611 | + * Consider the case with the depth of the mst topology tree is equal or less than 2 | |
| 1612 | + * A. When dsc bitstream can be transmitted along the entire path | |
| 1613 | + * 1. dsc is possible between source and branch/leaf device (common dsc params is possible), AND | |
| 1614 | + * 2. dsc passthrough supported at MST branch, or | |
| 1615 | + * 3. dsc decoding supported at leaf MST device | |
| 1616 | + * Use maximum dsc compression as bw constraint | |
| 1617 | + * B. When dsc bitstream cannot be transmitted along the entire path | |
| 1618 | + * Use native bw as bw constraint | |
| 1613 | 1619 | */ |
| 1614 | 1620 | if (is_dsc_common_config_possible(stream, &bw_range) && |
| 1615 | − aconnector->mst_output_port->passthrough_aux) { | |
| 1616 | − mst_mgr = aconnector->mst_output_port->mgr; | |
| 1617 | − mutex_lock(&mst_mgr->lock); | |
| 1618 | − | |
| 1621 | + (aconnector->mst_output_port->passthrough_aux || | |
| 1622 | + aconnector->dsc_aux == &aconnector->mst_output_port->aux)) { | |
| 1619 | 1623 | cur_link_settings = stream->link->verified_link_cap; |
| 1620 | 1624 | |
| 1621 | 1625 | upper_link_bw_in_kbps = dc_link_bandwidth_kbps(aconnector->dc_link, |
| 1622 | − &cur_link_settings | |
| 1623 | − ); | |
| 1624 | − down_link_bw_in_kbps = kbps_from_pbn(aconnector->mst_output_port->full_pbn); | |
| 1626 | + &cur_link_settings); | |
| 1627 | + down_link_bw_in_kbps = kbps_from_pbn(full_pbn); | |
| 1625 | 1628 | |
| 1626 | 1629 | /* pick the bottleneck */ |
| 1627 | 1630 | end_to_end_bw_in_kbps = min(upper_link_bw_in_kbps, |
| 1628 | 1631 | down_link_bw_in_kbps); |
| 1629 | 1632 | |
| 1630 | − mutex_unlock(&mst_mgr->lock); | |
| 1631 | − | |
| 1632 | 1633 | /* |
| 1633 | 1634 | * use the maximum dsc compression bandwidth as the required |
| 1634 | 1635 | * bandwidth for the mode |
| @@ -1642,9 +1643,8 @@ enum dc_status dm_dp_mst_is_port_support_mode( · 7 unmodified lines | ||
| 1642 | 1643 | } else { |
| 1643 | 1644 | /* check if mode could be supported within full_pbn */ |
| 1644 | 1645 | bpp = convert_dc_color_depth_into_bpc(stream->timing.display_color_depth) * 3; |
| 1645 | − pbn = drm_dp_calc_pbn_mode(stream->timing.pix_clk_100hz / 10, bpp, false); | |
| 1646 | − | |
| 1647 | − if (pbn > aconnector->mst_output_port->full_pbn) | |
| 1646 | + pbn = drm_dp_calc_pbn_mode(stream->timing.pix_clk_100hz / 10, bpp << 4); | |
| 1647 | + if (pbn > full_pbn) | |
| 1648 | 1648 | return DC_FAIL_BANDWIDTH_VALIDATE; |
| 1649 | 1649 | } |
| 1650 | 1650 | |
| @@ -820,22 +820,22 @@ static void dcn35_set_idle_state(struct clk_mgr *clk_mgr_base, bool allow_idle) · 819 unmodified lines | ||
| 820 | 820 | |
| 821 | 821 | if (dc->config.disable_ips == DMUB_IPS_ENABLE || |
| 822 | 822 | dc->config.disable_ips == DMUB_IPS_DISABLE_DYNAMIC) { |
| 823 | − val |= DMUB_IPS1_ALLOW_MASK; | |
| 824 | − val |= DMUB_IPS2_ALLOW_MASK; | |
| 825 | − } else if (dc->config.disable_ips == DMUB_IPS_DISABLE_IPS1) { | |
| 826 | 823 | val = val & ~DMUB_IPS1_ALLOW_MASK; |
| 827 | 824 | val = val & ~DMUB_IPS2_ALLOW_MASK; |
| 828 | − } else if (dc->config.disable_ips == DMUB_IPS_DISABLE_IPS2) { | |
| 829 | − val |= DMUB_IPS1_ALLOW_MASK; | |
| 830 | − val = val & ~DMUB_IPS2_ALLOW_MASK; | |
| 831 | − } else if (dc->config.disable_ips == DMUB_IPS_DISABLE_IPS2_Z10) { | |
| 825 | + } else if (dc->config.disable_ips == DMUB_IPS_DISABLE_IPS1) { | |
| 832 | 826 | val |= DMUB_IPS1_ALLOW_MASK; |
| 833 | 827 | val |= DMUB_IPS2_ALLOW_MASK; |
| 828 | + } else if (dc->config.disable_ips == DMUB_IPS_DISABLE_IPS2) { | |
| 829 | + val = val & ~DMUB_IPS1_ALLOW_MASK; | |
| 830 | + val |= DMUB_IPS2_ALLOW_MASK; | |
| 831 | + } else if (dc->config.disable_ips == DMUB_IPS_DISABLE_IPS2_Z10) { | |
| 832 | + val = val & ~DMUB_IPS1_ALLOW_MASK; | |
| 833 | + val = val & ~DMUB_IPS2_ALLOW_MASK; | |
| 834 | 834 | } |
| 835 | 835 | |
| 836 | 836 | if (!allow_idle) { |
| 837 | − val = val & ~DMUB_IPS1_ALLOW_MASK; | |
| 838 | − val = val & ~DMUB_IPS2_ALLOW_MASK; | |
| 837 | + val |= DMUB_IPS1_ALLOW_MASK; | |
| 838 | + val |= DMUB_IPS2_ALLOW_MASK; | |
| 839 | 839 | } |
| 840 | 840 | |
| 841 | 841 | dcn35_smu_write_ips_scratch(clk_mgr, val); |
| @@ -3178,7 +3178,7 @@ static bool update_planes_and_stream_state(struct dc *dc, · 3177 unmodified lines | ||
| 3178 | 3178 | struct pipe_ctx *otg_master = resource_get_otg_master_for_stream(&context->res_ctx, |
| 3179 | 3179 | context->streams[i]); |
| 3180 | 3180 | |
| 3181 | − if (otg_master->stream->test_pattern.type != DP_TEST_PATTERN_VIDEO_MODE) | |
| 3181 | + if (otg_master && otg_master->stream->test_pattern.type != DP_TEST_PATTERN_VIDEO_MODE) | |
| 3182 | 3182 | resource_build_test_pattern_params(&context->res_ctx, otg_master); |
| 3183 | 3183 | } |
| 3184 | 3184 | } |
| @@ -4934,8 +4934,8 @@ bool dc_dmub_is_ips_idle_state(struct dc *dc) · 1749 unmodified lines | ||
| 4934 | 4934 | if (dc->hwss.get_idle_state) |
| 4935 | 4935 | idle_state = dc->hwss.get_idle_state(dc); |
| 4936 | 4936 | |
| 4937 | − if ((idle_state & DMUB_IPS1_ALLOW_MASK) || | |
| 4938 | − (idle_state & DMUB_IPS2_ALLOW_MASK)) | |
| 4937 | + if (!(idle_state & DMUB_IPS1_ALLOW_MASK) || | |
| 4938 | + !(idle_state & DMUB_IPS2_ALLOW_MASK)) | |
| 4939 | 4939 | return true; |
| 4940 | 4940 | |
| 4941 | 4941 | return false; |
| @@ -5190,6 +5190,9 @@ bool dc_resource_acquire_secondary_pipe_for_mpc_odm_legacy( · 5189 unmodified lines | ||
| 5190 | 5190 | sec_next = sec_pipe->next_odm_pipe; |
| 5191 | 5191 | sec_prev = sec_pipe->prev_odm_pipe; |
| 5192 | 5192 | |
| 5193 | + if (pri_pipe == NULL) | |
| 5194 | + return false; | |
| 5195 | + | |
| 5193 | 5196 | *sec_pipe = *pri_pipe; |
| 5194 | 5197 | |
| 5195 | 5198 | sec_pipe->top_pipe = sec_top; |
| @@ -1202,11 +1202,11 @@ void dc_dmub_srv_exit_low_power_state(const struct dc *dc) · 1201 unmodified lines | ||
| 1202 | 1202 | allow_state = dc->hwss.get_idle_state(dc); |
| 1203 | 1203 | dc->hwss.set_idle_state(dc, false); |
| 1204 | 1204 | |
| 1205 | − if (allow_state & DMUB_IPS2_ALLOW_MASK) { | |
| 1205 | + if (!(allow_state & DMUB_IPS2_ALLOW_MASK)) { | |
| 1206 | 1206 | // Wait for evaluation time |
| 1207 | 1207 | udelay(dc->debug.ips2_eval_delay_us); |
| 1208 | 1208 | commit_state = dc->hwss.get_idle_state(dc); |
| 1209 | − if (commit_state & DMUB_IPS2_COMMIT_MASK) { | |
| 1209 | + if (!(commit_state & DMUB_IPS2_COMMIT_MASK)) { | |
| 1210 | 1210 | // Tell PMFW to exit low power state |
| 1211 | 1211 | dc->clk_mgr->funcs->exit_low_power_state(dc->clk_mgr); |
| 1212 | 1212 | |
| @@ -1216,7 +1216,7 @@ void dc_dmub_srv_exit_low_power_state(const struct dc *dc) · 3 unmodified lines | ||
| 1216 | 1216 | |
| 1217 | 1217 | for (i = 0; i < max_num_polls; ++i) { |
| 1218 | 1218 | commit_state = dc->hwss.get_idle_state(dc); |
| 1219 | − if (!(commit_state & DMUB_IPS2_COMMIT_MASK)) | |
| 1219 | + if (commit_state & DMUB_IPS2_COMMIT_MASK) | |
| 1220 | 1220 | break; |
| 1221 | 1221 | |
| 1222 | 1222 | udelay(1); |
| @@ -1235,10 +1235,10 @@ void dc_dmub_srv_exit_low_power_state(const struct dc *dc) · 12 unmodified lines | ||
| 1235 | 1235 | } |
| 1236 | 1236 | |
| 1237 | 1237 | dc_dmub_srv_notify_idle(dc, false); |
| 1238 | − if (allow_state & DMUB_IPS1_ALLOW_MASK) { | |
| 1238 | + if (!(allow_state & DMUB_IPS1_ALLOW_MASK)) { | |
| 1239 | 1239 | for (i = 0; i < max_num_polls; ++i) { |
| 1240 | 1240 | commit_state = dc->hwss.get_idle_state(dc); |
| 1241 | − if (!(commit_state & DMUB_IPS1_COMMIT_MASK)) | |
| 1241 | + if (commit_state & DMUB_IPS1_COMMIT_MASK) | |
| 1242 | 1242 | break; |
| 1243 | 1243 | |
| 1244 | 1244 | udelay(1); |
| @@ -177,6 +177,7 @@ struct dc_panel_patch { · 176 unmodified lines | ||
| 177 | 177 | unsigned int disable_fams; |
| 178 | 178 | unsigned int skip_avmute; |
| 179 | 179 | unsigned int mst_start_top_delay; |
| 180 | + unsigned int remove_sink_ext_caps; | |
| 180 | 181 | }; |
| 181 | 182 | |
| 182 | 183 | struct dc_edid_caps { |
| @@ -261,12 +261,6 @@ static void enc35_stream_encoder_enable( · 260 unmodified lines | ||
| 261 | 261 | /* invalid mode ! */ |
| 262 | 262 | ASSERT_CRITICAL(false); |
| 263 | 263 | } |
| 264 | − | |
| 265 | − REG_UPDATE(DIG_FE_CLK_CNTL, DIG_FE_CLK_EN, 1); | |
| 266 | − REG_UPDATE(DIG_FE_EN_CNTL, DIG_FE_ENABLE, 1); | |
| 267 | − } else { | |
| 268 | − REG_UPDATE(DIG_FE_EN_CNTL, DIG_FE_ENABLE, 0); | |
| 269 | − REG_UPDATE(DIG_FE_CLK_CNTL, DIG_FE_CLK_EN, 0); | |
| 270 | 264 | } |
| 271 | 265 | } |
| 272 | 266 | |
| @@ -436,6 +430,8 @@ static void enc35_disable_fifo(struct stream_encoder *enc) · 163 unmodified lines | ||
| 436 | 430 | struct dcn10_stream_encoder *enc1 = DCN10STRENC_FROM_STRENC(enc); |
| 437 | 431 | |
| 438 | 432 | REG_UPDATE(DIG_FIFO_CTRL0, DIG_FIFO_ENABLE, 0); |
| 433 | + REG_UPDATE(DIG_FE_EN_CNTL, DIG_FE_ENABLE, 0); | |
| 434 | + REG_UPDATE(DIG_FE_CLK_CNTL, DIG_FE_CLK_EN, 0); | |
| 439 | 435 | } |
| 440 | 436 | |
| 441 | 437 | static void enc35_enable_fifo(struct stream_encoder *enc) |
| @@ -443,6 +439,8 @@ static void enc35_enable_fifo(struct stream_encoder *enc) · 1 unmodified line | ||
| 443 | 439 | struct dcn10_stream_encoder *enc1 = DCN10STRENC_FROM_STRENC(enc); |
| 444 | 440 | |
| 445 | 441 | REG_UPDATE(DIG_FIFO_CTRL0, DIG_FIFO_READ_START_LEVEL, 0x7); |
| 442 | + REG_UPDATE(DIG_FE_CLK_CNTL, DIG_FE_CLK_EN, 1); | |
| 443 | + REG_UPDATE(DIG_FE_EN_CNTL, DIG_FE_ENABLE, 1); | |
| 446 | 444 | |
| 447 | 445 | enc35_reset_fifo(enc, true); |
| 448 | 446 | enc35_reset_fifo(enc, false); |
| @@ -1088,6 +1088,9 @@ static bool detect_link_and_local_sink(struct dc_link *link, · 1087 unmodified lines | ||
| 1088 | 1088 | if (sink->edid_caps.panel_patch.skip_scdc_overwrite) |
| 1089 | 1089 | link->ctx->dc->debug.hdmi20_disable = true; |
| 1090 | 1090 | |
| 1091 | + if (sink->edid_caps.panel_patch.remove_sink_ext_caps) | |
| 1092 | + link->dpcd_sink_ext_caps.raw = 0; | |
| 1093 | + | |
| 1091 | 1094 | if (dc_is_hdmi_signal(link->connector_signal)) |
| 1092 | 1095 | read_scdc_caps(link->ddc, link->local_sink); |
| 1093 | 1096 | |
| @@ -195,6 +195,7 @@ struct dmub_srv_region_params { · 194 unmodified lines | ||
| 195 | 195 | uint32_t vbios_size; |
| 196 | 196 | const uint8_t *fw_inst_const; |
| 197 | 197 | const uint8_t *fw_bss_data; |
| 198 | + bool is_mailbox_in_inbox; | |
| 198 | 199 | }; |
| 199 | 200 | |
| 200 | 201 | /** |
| @@ -214,20 +215,25 @@ struct dmub_srv_region_params { · 13 unmodified lines | ||
| 214 | 215 | */ |
| 215 | 216 | struct dmub_srv_region_info { |
| 216 | 217 | uint32_t fb_size; |
| 218 | + uint32_t inbox_size; | |
| 217 | 219 | uint8_t num_regions; |
| 218 | 220 | struct dmub_region regions[DMUB_WINDOW_TOTAL]; |
| 219 | 221 | }; |
| 220 | 222 | |
| 221 | 223 | /** |
| 222 | − * struct dmub_srv_fb_params - parameters used for driver fb setup | |
| 224 | + * struct dmub_srv_memory_params - parameters used for driver fb setup | |
| 223 | 225 | * @region_info: region info calculated by dmub service |
| 224 | − * @cpu_addr: base cpu address for the framebuffer | |
| 225 | − * @gpu_addr: base gpu virtual address for the framebuffer | |
| 226 | + * @cpu_fb_addr: base cpu address for the framebuffer | |
| 227 | + * @cpu_inbox_addr: base cpu address for the gart | |
| 228 | + * @gpu_fb_addr: base gpu virtual address for the framebuffer | |
| 229 | + * @gpu_inbox_addr: base gpu virtual address for the gart | |
| 226 | 230 | */ |
| 227 | −struct dmub_srv_fb_params { | |
| 231 | +struct dmub_srv_memory_params { | |
| 228 | 232 | const struct dmub_srv_region_info *region_info; |
| 229 | − void *cpu_addr; | |
| 230 | − uint64_t gpu_addr; | |
| 233 | + void *cpu_fb_addr; | |
| 234 | + void *cpu_inbox_addr; | |
| 235 | + uint64_t gpu_fb_addr; | |
| 236 | + uint64_t gpu_inbox_addr; | |
| 231 | 237 | }; |
| 232 | 238 | |
| 233 | 239 | /** |
| @@ -563,8 +569,8 @@ dmub_srv_calc_region_info(struct dmub_srv *dmub, · 329 unmodified lines | ||
| 563 | 569 | * DMUB_STATUS_OK - success |
| 564 | 570 | * DMUB_STATUS_INVALID - unspecified error |
| 565 | 571 | */ |
| 566 | −enum dmub_status dmub_srv_calc_fb_info(struct dmub_srv *dmub, | |
| 567 | − const struct dmub_srv_fb_params *params, | |
| 572 | +enum dmub_status dmub_srv_calc_mem_info(struct dmub_srv *dmub, | |
| 573 | + const struct dmub_srv_memory_params *params, | |
| 568 | 574 | struct dmub_srv_fb_info *out); |
| 569 | 575 | |
| 570 | 576 | /** |
| @@ -434,7 +434,7 @@ dmub_srv_calc_region_info(struct dmub_srv *dmub, · 433 unmodified lines | ||
| 434 | 434 | uint32_t fw_state_size = DMUB_FW_STATE_SIZE; |
| 435 | 435 | uint32_t trace_buffer_size = DMUB_TRACE_BUFFER_SIZE; |
| 436 | 436 | uint32_t scratch_mem_size = DMUB_SCRATCH_MEM_SIZE; |
| 437 | − | |
| 437 | + uint32_t previous_top = 0; | |
| 438 | 438 | if (!dmub->sw_init) |
| 439 | 439 | return DMUB_STATUS_INVALID; |
| 440 | 440 | |
| @@ -459,8 +459,15 @@ dmub_srv_calc_region_info(struct dmub_srv *dmub, · 18 unmodified lines | ||
| 459 | 459 | bios->base = dmub_align(stack->top, 256); |
| 460 | 460 | bios->top = bios->base + params->vbios_size; |
| 461 | 461 | |
| 462 | − mail->base = dmub_align(bios->top, 256); | |
| 463 | − mail->top = mail->base + DMUB_MAILBOX_SIZE; | |
| 462 | + if (params->is_mailbox_in_inbox) { | |
| 463 | + mail->base = 0; | |
| 464 | + mail->top = mail->base + DMUB_MAILBOX_SIZE; | |
| 465 | + previous_top = bios->top; | |
| 466 | + } else { | |
| 467 | + mail->base = dmub_align(bios->top, 256); | |
| 468 | + mail->top = mail->base + DMUB_MAILBOX_SIZE; | |
| 469 | + previous_top = mail->top; | |
| 470 | + } | |
| 464 | 471 | |
| 465 | 472 | fw_info = dmub_get_fw_meta_info(params); |
| 466 | 473 | |
| @@ -479,7 +486,7 @@ dmub_srv_calc_region_info(struct dmub_srv *dmub, · 12 unmodified lines | ||
| 479 | 486 | dmub->fw_version = fw_info->fw_version; |
| 480 | 487 | } |
| 481 | 488 | |
| 482 | − trace_buff->base = dmub_align(mail->top, 256); | |
| 489 | + trace_buff->base = dmub_align(previous_top, 256); | |
| 483 | 490 | trace_buff->top = trace_buff->base + dmub_align(trace_buffer_size, 64); |
| 484 | 491 | |
| 485 | 492 | fw_state->base = dmub_align(trace_buff->top, 256); |
| @@ -490,11 +497,14 @@ dmub_srv_calc_region_info(struct dmub_srv *dmub, · 4 unmodified lines | ||
| 490 | 497 | |
| 491 | 498 | out->fb_size = dmub_align(scratch_mem->top, 4096); |
| 492 | 499 | |
| 500 | + if (params->is_mailbox_in_inbox) | |
| 501 | + out->inbox_size = dmub_align(mail->top, 4096); | |
| 502 | + | |
| 493 | 503 | return DMUB_STATUS_OK; |
| 494 | 504 | } |
| 495 | 505 | |
| 496 | −enum dmub_status dmub_srv_calc_fb_info(struct dmub_srv *dmub, | |
| 497 | − const struct dmub_srv_fb_params *params, | |
| 506 | +enum dmub_status dmub_srv_calc_mem_info(struct dmub_srv *dmub, | |
| 507 | + const struct dmub_srv_memory_params *params, | |
| 498 | 508 | struct dmub_srv_fb_info *out) |
| 499 | 509 | { |
| 500 | 510 | uint8_t *cpu_base; |
| @@ -509,8 +519,8 @@ enum dmub_status dmub_srv_calc_fb_info(struct dmub_srv *dmub, · 8 unmodified lines | ||
| 509 | 519 | if (params->region_info->num_regions != DMUB_NUM_WINDOWS) |
| 510 | 520 | return DMUB_STATUS_INVALID; |
| 511 | 521 | |
| 512 | − cpu_base = (uint8_t *)params->cpu_addr; | |
| 513 | − gpu_base = params->gpu_addr; | |
| 522 | + cpu_base = (uint8_t *)params->cpu_fb_addr; | |
| 523 | + gpu_base = params->gpu_fb_addr; | |
| 514 | 524 | |
| 515 | 525 | for (i = 0; i < DMUB_NUM_WINDOWS; ++i) { |
| 516 | 526 | const struct dmub_region *reg = |
| @@ -518,6 +528,12 @@ enum dmub_status dmub_srv_calc_fb_info(struct dmub_srv *dmub, · 1 unmodified line | ||
| 518 | 528 | |
| 519 | 529 | out->fb[i].cpu_addr = cpu_base + reg->base; |
| 520 | 530 | out->fb[i].gpu_addr = gpu_base + reg->base; |
| 531 | + | |
| 532 | + if (i == DMUB_WINDOW_4_MAILBOX && params->cpu_inbox_addr != 0) { | |
| 533 | + out->fb[i].cpu_addr = (uint8_t *)params->cpu_inbox_addr + reg->base; | |
| 534 | + out->fb[i].gpu_addr = params->gpu_inbox_addr + reg->base; | |
| 535 | + } | |
| 536 | + | |
| 521 | 537 | out->fb[i].size = reg->top - reg->base; |
| 522 | 538 | } |
| 523 | 539 | |
| @@ -707,9 +723,16 @@ enum dmub_status dmub_srv_sync_inbox1(struct dmub_srv *dmub) · 183 unmodified lines | ||
| 707 | 723 | return DMUB_STATUS_INVALID; |
| 708 | 724 | |
| 709 | 725 | if (dmub->hw_funcs.get_inbox1_rptr && dmub->hw_funcs.get_inbox1_wptr) { |
| 710 | − dmub->inbox1_rb.rptr = dmub->hw_funcs.get_inbox1_rptr(dmub); | |
| 711 | − dmub->inbox1_rb.wrpt = dmub->hw_funcs.get_inbox1_wptr(dmub); | |
| 712 | − dmub->inbox1_last_wptr = dmub->inbox1_rb.wrpt; | |
| 726 | + uint32_t rptr = dmub->hw_funcs.get_inbox1_rptr(dmub); | |
| 727 | + uint32_t wptr = dmub->hw_funcs.get_inbox1_wptr(dmub); | |
| 728 | + | |
| 729 | + if (rptr > dmub->inbox1_rb.capacity || wptr > dmub->inbox1_rb.capacity) { | |
| 730 | + return DMUB_STATUS_HW_FAILURE; | |
| 731 | + } else { | |
| 732 | + dmub->inbox1_rb.rptr = rptr; | |
| 733 | + dmub->inbox1_rb.wrpt = wptr; | |
| 734 | + dmub->inbox1_last_wptr = dmub->inbox1_rb.wrpt; | |
| 735 | + } | |
| 713 | 736 | } |
| 714 | 737 | |
| 715 | 738 | return DMUB_STATUS_OK; |
| @@ -743,6 +766,11 @@ enum dmub_status dmub_srv_cmd_queue(struct dmub_srv *dmub, · 27 unmodified lines | ||
| 743 | 766 | if (!dmub->hw_init) |
| 744 | 767 | return DMUB_STATUS_INVALID; |
| 745 | 768 | |
| 769 | + if (dmub->inbox1_rb.rptr > dmub->inbox1_rb.capacity || | |
| 770 | + dmub->inbox1_rb.wrpt > dmub->inbox1_rb.capacity) { | |
| 771 | + return DMUB_STATUS_HW_FAILURE; | |
| 772 | + } | |
| 773 | + | |
| 746 | 774 | if (dmub_rb_push_front(&dmub->inbox1_rb, cmd)) |
| 747 | 775 | return DMUB_STATUS_OK; |
| 748 | 776 | |
| @@ -123,7 +123,7 @@ typedef enum { · 122 unmodified lines | ||
| 123 | 123 | VOLTAGE_GUARDBAND_COUNT |
| 124 | 124 | } GFX_GUARDBAND_e; |
| 125 | 125 | |
| 126 | −#define SMU_METRICS_TABLE_VERSION 0x8 | |
| 126 | +#define SMU_METRICS_TABLE_VERSION 0x9 | |
| 127 | 127 | |
| 128 | 128 | typedef struct __attribute__((packed, aligned(4))) { |
| 129 | 129 | uint32_t AccumulationCounter; |
| @@ -211,6 +211,14 @@ typedef struct __attribute__((packed, aligned(4))) { · 81 unmodified lines | ||
| 211 | 211 | //XGMI Data tranfser size |
| 212 | 212 | uint64_t XgmiReadDataSizeAcc[8];//in KByte |
| 213 | 213 | uint64_t XgmiWriteDataSizeAcc[8];//in KByte |
| 214 | + | |
| 215 | + //PCIE BW Data and error count | |
| 216 | + uint32_t PcieBandwidth[4]; | |
| 217 | + uint32_t PCIeL0ToRecoveryCountAcc; // The Pcie counter itself is accumulated | |
| 218 | + uint32_t PCIenReplayAAcc; // The Pcie counter itself is accumulated | |
| 219 | + uint32_t PCIenReplayARolloverCountAcc; // The Pcie counter itself is accumulated | |
| 220 | + uint32_t PCIeNAKSentCountAcc; // The Pcie counter itself is accumulated | |
| 221 | + uint32_t PCIeNAKReceivedCountAcc; // The Pcie counter itself is accumulated | |
| 214 | 222 | } MetricsTable_t; |
| 215 | 223 | |
| 216 | 224 | #define SMU_VF_METRICS_TABLE_VERSION 0x3 |
| @@ -1454,7 +1454,7 @@ static int smu_v13_0_6_register_irq_handler(struct smu_context *smu) · 1453 unmodified lines | ||
| 1454 | 1454 | |
| 1455 | 1455 | static int smu_v13_0_6_notify_unload(struct smu_context *smu) |
| 1456 | 1456 | { |
| 1457 | − if (smu->smc_fw_version <= 0x553500) | |
| 1457 | + if (amdgpu_in_reset(smu->adev)) | |
| 1458 | 1458 | return 0; |
| 1459 | 1459 | |
| 1460 | 1460 | dev_dbg(smu->adev->dev, "Notify PMFW about driver unload"); |
| @@ -2095,6 +2095,14 @@ static ssize_t smu_v13_0_6_get_gpu_metrics(struct smu_context *smu, void **table · 634 unmodified lines | ||
| 2095 | 2095 | smu_v13_0_6_get_current_pcie_link_speed(smu); |
| 2096 | 2096 | gpu_metrics->pcie_bandwidth_acc = |
| 2097 | 2097 | SMUQ10_ROUND(metrics->PcieBandwidthAcc[0]); |
| 2098 | + gpu_metrics->pcie_bandwidth_inst = | |
| 2099 | + SMUQ10_ROUND(metrics->PcieBandwidth[0]); | |
| 2100 | + gpu_metrics->pcie_l0_to_recov_count_acc = | |
| 2101 | + metrics->PCIeL0ToRecoveryCountAcc; | |
| 2102 | + gpu_metrics->pcie_replay_count_acc = | |
| 2103 | + metrics->PCIenReplayAAcc; | |
| 2104 | + gpu_metrics->pcie_replay_rover_count_acc = | |
| 2105 | + metrics->PCIenReplayARolloverCountAcc; | |
| 2098 | 2106 | } |
| 2099 | 2107 | |
| 2100 | 2108 | gpu_metrics->system_clock_counter = ktime_get_boottime_ns(); |
| @@ -7,8 +7,9 @@ · 6 unmodified lines | ||
| 7 | 7 | #include <linux/clk.h> |
| 8 | 8 | #include <linux/component.h> |
| 9 | 9 | #include <linux/module.h> |
| 10 | −#include <linux/of_device.h> | |
| 10 | +#include <linux/of.h> | |
| 11 | 11 | #include <linux/platform_device.h> |
| 12 | +#include <linux/property.h> | |
| 12 | 13 | |
| 13 | 14 | #include <drm/drm_atomic.h> |
| 14 | 15 | #include <drm/drm_atomic_helper.h> |
| @@ -1012,26 +1013,17 @@ armada_lcd_bind(struct device *dev, struct device *master, void *data) · 997 unmodified lines | ||
| 1012 | 1013 | int irq = platform_get_irq(pdev, 0); |
| 1013 | 1014 | const struct armada_variant *variant; |
| 1014 | 1015 | struct device_node *port = NULL; |
| 1016 | + struct device_node *np, *parent = dev->of_node; | |
| 1015 | 1017 | |
| 1016 | 1018 | if (irq < 0) |
| 1017 | 1019 | return irq; |
| 1018 | 1020 | |
| 1019 | − if (!dev->of_node) { | |
| 1020 | − const struct platform_device_id *id; | |
| 1021 | 1021 | |
| 1022 | − id = platform_get_device_id(pdev); | |
| 1023 | − if (!id) | |
| 1024 | − return -ENXIO; | |
| 1025 | − | |
| 1026 | − variant = (const struct armada_variant *)id->driver_data; | |
| 1027 | − } else { | |
| 1028 | − const struct of_device_id *match; | |
| 1029 | − struct device_node *np, *parent = dev->of_node; | |
| 1030 | − | |
| 1031 | − match = of_match_device(dev->driver->of_match_table, dev); | |
| 1032 | − if (!match) | |
| 1033 | − return -ENXIO; | |
| 1022 | + variant = device_get_match_data(dev); | |
| 1023 | + if (!variant) | |
| 1024 | + return -ENXIO; | |
| 1034 | 1025 | |
| 1026 | + if (parent) { | |
| 1035 | 1027 | np = of_get_child_by_name(parent, "ports"); |
| 1036 | 1028 | if (np) |
| 1037 | 1029 | parent = np; |
| @@ -1041,8 +1033,6 @@ armada_lcd_bind(struct device *dev, struct device *master, void *data) · 3 unmodified lines | ||
| 1041 | 1033 | dev_err(dev, "no port node found in %pOF\n", parent); |
| 1042 | 1034 | return -ENXIO; |
| 1043 | 1035 | } |
| 1044 | − | |
| 1045 | − variant = match->data; | |
| 1046 | 1036 | } |
| 1047 | 1037 | |
| 1048 | 1038 | return armada_drm_crtc_create(drm, dev, res, irq, variant, port); |
| @@ -1066,10 +1056,9 @@ static int armada_lcd_probe(struct platform_device *pdev) · 17 unmodified lines | ||
| 1066 | 1056 | return component_add(&pdev->dev, &armada_lcd_ops); |
| 1067 | 1057 | } |
| 1068 | 1058 | |
| 1069 | −static int armada_lcd_remove(struct platform_device *pdev) | |
| 1059 | +static void armada_lcd_remove(struct platform_device *pdev) | |
| 1070 | 1060 | { |
| 1071 | 1061 | component_del(&pdev->dev, &armada_lcd_ops); |
| 1072 | − return 0; | |
| 1073 | 1062 | } |
| 1074 | 1063 | |
| 1075 | 1064 | static const struct of_device_id armada_lcd_of_match[] = { |
| @@ -1095,7 +1084,7 @@ MODULE_DEVICE_TABLE(platform, armada_lcd_platform_ids); · 19 unmodified lines | ||
| 1095 | 1084 | |
| 1096 | 1085 | struct platform_driver armada_lcd_platform_driver = { |
| 1097 | 1086 | .probe = armada_lcd_probe, |
| 1098 | − .remove = armada_lcd_remove, | |
| 1087 | + .remove_new = armada_lcd_remove, | |
| 1099 | 1088 | .driver = { |
| 1100 | 1089 | .name = "armada-lcd", |
| 1101 | 1090 | .owner = THIS_MODULE, |
| @@ -226,10 +226,9 @@ static int armada_drm_probe(struct platform_device *pdev) · 225 unmodified lines | ||
| 226 | 226 | match); |
| 227 | 227 | } |
| 228 | 228 | |
| 229 | −static int armada_drm_remove(struct platform_device *pdev) | |
| 229 | +static void armada_drm_remove(struct platform_device *pdev) | |
| 230 | 230 | { |
| 231 | 231 | component_master_del(&pdev->dev, &armada_master_ops); |
| 232 | − return 0; | |
| 233 | 232 | } |
| 234 | 233 | |
| 235 | 234 | static void armada_drm_shutdown(struct platform_device *pdev) |
| @@ -249,7 +248,7 @@ MODULE_DEVICE_TABLE(platform, armada_drm_platform_ids); · 13 unmodified lines | ||
| 249 | 248 | |
| 250 | 249 | static struct platform_driver armada_drm_platform_driver = { |
| 251 | 250 | .probe = armada_drm_probe, |
| 252 | − .remove = armada_drm_remove, | |
| 251 | + .remove_new = armada_drm_remove, | |
| 253 | 252 | .shutdown = armada_drm_shutdown, |
| 254 | 253 | .driver = { |
| 255 | 254 | .name = "armada-drm", |
| @@ -6,10 +6,10 @@ · 5 unmodified lines | ||
| 6 | 6 | #include <linux/irq.h> |
| 7 | 7 | #include <linux/mfd/syscon.h> |
| 8 | 8 | #include <linux/module.h> |
| 9 | −#include <linux/of.h> | |
| 10 | −#include <linux/of_device.h> | |
| 9 | +#include <linux/mod_devicetable.h> | |
| 11 | 10 | #include <linux/of_reserved_mem.h> |
| 12 | 11 | #include <linux/platform_device.h> |
| 12 | +#include <linux/property.h> | |
| 13 | 13 | #include <linux/regmap.h> |
| 14 | 14 | #include <linux/reset.h> |
| 15 | 15 | |
| @@ -143,7 +143,6 @@ static int aspeed_gfx_load(struct drm_device *drm) · 127 unmodified lines | ||
| 143 | 143 | struct aspeed_gfx *priv = to_aspeed_gfx(drm); |
| 144 | 144 | struct device_node *np = pdev->dev.of_node; |
| 145 | 145 | const struct aspeed_gfx_config *config; |
| 146 | − const struct of_device_id *match; | |
| 147 | 146 | struct resource *res; |
| 148 | 147 | int ret; |
| 149 | 148 | |
| @@ -152,10 +151,9 @@ static int aspeed_gfx_load(struct drm_device *drm) · 2 unmodified lines | ||
| 152 | 151 | if (IS_ERR(priv->base)) |
| 153 | 152 | return PTR_ERR(priv->base); |
| 154 | 153 | |
| 155 | − match = of_match_device(aspeed_gfx_match, &pdev->dev); | |
| 156 | − if (!match) | |
| 154 | + config = device_get_match_data(&pdev->dev); | |
| 155 | + if (!config) | |
| 157 | 156 | return -EINVAL; |
| 158 | − config = match->data; | |
| 159 | 157 | |
| 160 | 158 | priv->dac_reg = config->dac_reg; |
| 161 | 159 | priv->int_clr_reg = config->int_clear_reg; |
| @@ -89,11 +89,194 @@ static const struct pci_device_id ast_pciidlist[] = { · 88 unmodified lines | ||
| 89 | 89 | |
| 90 | 90 | MODULE_DEVICE_TABLE(pci, ast_pciidlist); |
| 91 | 91 | |
| 92 | +static bool ast_is_vga_enabled(void __iomem *ioregs) | |
| 93 | +{ | |
| 94 | + u8 vgaer = __ast_read8(ioregs, AST_IO_VGAER); | |
| 95 | + | |
| 96 | + return vgaer & AST_IO_VGAER_VGA_ENABLE; | |
| 97 | +} | |
| 98 | + | |
| 99 | +static void ast_enable_vga(void __iomem *ioregs) | |
| 100 | +{ | |
| 101 | + __ast_write8(ioregs, AST_IO_VGAER, AST_IO_VGAER_VGA_ENABLE); | |
| 102 | + __ast_write8(ioregs, AST_IO_VGAMR_W, AST_IO_VGAMR_IOSEL); | |
| 103 | +} | |
| 104 | + | |
| 105 | +/* | |
| 106 | + * Run this function as part of the HW device cleanup; not | |
| 107 | + * when the DRM device gets released. | |
| 108 | + */ | |
| 109 | +static void ast_enable_mmio_release(void *data) | |
| 110 | +{ | |
| 111 | + void __iomem *ioregs = (void __force __iomem *)data; | |
| 112 | + | |
| 113 | + /* enable standard VGA decode */ | |
| 114 | + __ast_write8_i(ioregs, AST_IO_VGACRI, 0xa1, AST_IO_VGACRA1_MMIO_ENABLED); | |
| 115 | +} | |
| 116 | + | |
| 117 | +static int ast_enable_mmio(struct device *dev, void __iomem *ioregs) | |
| 118 | +{ | |
| 119 | + void *data = (void __force *)ioregs; | |
| 120 | + | |
| 121 | + __ast_write8_i(ioregs, AST_IO_VGACRI, 0xa1, | |
| 122 | + AST_IO_VGACRA1_MMIO_ENABLED | | |
| 123 | + AST_IO_VGACRA1_VGAIO_DISABLED); | |
| 124 | + | |
| 125 | + return devm_add_action_or_reset(dev, ast_enable_mmio_release, data); | |
| 126 | +} | |
| 127 | + | |
| 128 | +static void ast_open_key(void __iomem *ioregs) | |
| 129 | +{ | |
| 130 | + __ast_write8_i(ioregs, AST_IO_VGACRI, 0x80, AST_IO_VGACR80_PASSWORD); | |
| 131 | +} | |
| 132 | + | |
| 133 | +static int ast_detect_chip(struct pci_dev *pdev, | |
| 134 | + void __iomem *regs, void __iomem *ioregs, | |
| 135 | + enum ast_chip *chip_out, | |
| 136 | + enum ast_config_mode *config_mode_out) | |
| 137 | +{ | |
| 138 | + struct device *dev = &pdev->dev; | |
| 139 | + struct device_node *np = dev->of_node; | |
| 140 | + enum ast_config_mode config_mode = ast_use_defaults; | |
| 141 | + uint32_t scu_rev = 0xffffffff; | |
| 142 | + enum ast_chip chip; | |
| 143 | + u32 data; | |
| 144 | + u8 vgacrd0, vgacrd1; | |
| 145 | + | |
| 146 | + /* | |
| 147 | + * Find configuration mode and read SCU revision | |
| 148 | + */ | |
| 149 | + | |
| 150 | + /* Check if we have device-tree properties */ | |
| 151 | + if (np && !of_property_read_u32(np, "aspeed,scu-revision-id", &data)) { | |
| 152 | + /* We do, disable P2A access */ | |
| 153 | + config_mode = ast_use_dt; | |
| 154 | + scu_rev = data; | |
| 155 | + } else if (pdev->device == PCI_CHIP_AST2000) { // Not all families have a P2A bridge | |
| 156 | + /* | |
| 157 | + * The BMC will set SCU 0x40 D[12] to 1 if the P2 bridge | |
| 158 | + * is disabled. We force using P2A if VGA only mode bit | |
| 159 | + * is set D[7] | |
| 160 | + */ | |
| 161 | + vgacrd0 = __ast_read8_i(ioregs, AST_IO_VGACRI, 0xd0); | |
| 162 | + vgacrd1 = __ast_read8_i(ioregs, AST_IO_VGACRI, 0xd1); | |
| 163 | + if (!(vgacrd0 & 0x80) || !(vgacrd1 & 0x10)) { | |
| 164 | + | |
| 165 | + /* | |
| 166 | + * We have a P2A bridge and it is enabled. | |
| 167 | + */ | |
| 168 | + | |
| 169 | + /* Patch AST2500/AST2510 */ | |
| 170 | + if ((pdev->revision & 0xf0) == 0x40) { | |
| 171 | + if (!(vgacrd0 & AST_VRAM_INIT_STATUS_MASK)) | |
| 172 | + ast_patch_ahb_2500(regs); | |
| 173 | + } | |
| 174 | + | |
| 175 | + /* Double check that it's actually working */ | |
| 176 | + data = __ast_read32(regs, 0xf004); | |
| 177 | + if ((data != 0xffffffff) && (data != 0x00)) { | |
| 178 | + config_mode = ast_use_p2a; | |
| 179 | + | |
| 180 | + /* Read SCU7c (silicon revision register) */ | |
| 181 | + __ast_write32(regs, 0xf004, 0x1e6e0000); | |
| 182 | + __ast_write32(regs, 0xf000, 0x1); | |
| 183 | + scu_rev = __ast_read32(regs, 0x1207c); | |
| 184 | + } | |
| 185 | + } | |
| 186 | + } | |
| 187 | + | |
| 188 | + switch (config_mode) { | |
| 189 | + case ast_use_defaults: | |
| 190 | + dev_info(dev, "Using default configuration\n"); | |
| 191 | + break; | |
| 192 | + case ast_use_dt: | |
| 193 | + dev_info(dev, "Using device-tree for configuration\n"); | |
| 194 | + break; | |
| 195 | + case ast_use_p2a: | |
| 196 | + dev_info(dev, "Using P2A bridge for configuration\n"); | |
| 197 | + break; | |
| 198 | + } | |
| 199 | + | |
| 200 | + /* | |
| 201 | + * Identify chipset | |
| 202 | + */ | |
| 203 | + | |
| 204 | + if (pdev->revision >= 0x50) { | |
| 205 | + chip = AST2600; | |
| 206 | + dev_info(dev, "AST 2600 detected\n"); | |
| 207 | + } else if (pdev->revision >= 0x40) { | |
| 208 | + switch (scu_rev & 0x300) { | |
| 209 | + case 0x0100: | |
| 210 | + chip = AST2510; | |
| 211 | + dev_info(dev, "AST 2510 detected\n"); | |
| 212 | + break; | |
| 213 | + default: | |
| 214 | + chip = AST2500; | |
| 215 | + dev_info(dev, "AST 2500 detected\n"); | |
| 216 | + break; | |
| 217 | + } | |
| 218 | + } else if (pdev->revision >= 0x30) { | |
| 219 | + switch (scu_rev & 0x300) { | |
| 220 | + case 0x0100: | |
| 221 | + chip = AST1400; | |
| 222 | + dev_info(dev, "AST 1400 detected\n"); | |
| 223 | + break; | |
| 224 | + default: | |
| 225 | + chip = AST2400; | |
| 226 | + dev_info(dev, "AST 2400 detected\n"); | |
| 227 | + break; | |
| 228 | + } | |
| 229 | + } else if (pdev->revision >= 0x20) { | |
| 230 | + switch (scu_rev & 0x300) { | |
| 231 | + case 0x0000: | |
| 232 | + chip = AST1300; | |
| 233 | + dev_info(dev, "AST 1300 detected\n"); | |
| 234 | + break; | |
| 235 | + default: | |
| 236 | + chip = AST2300; | |
| 237 | + dev_info(dev, "AST 2300 detected\n"); | |
| 238 | + break; | |
| 239 | + } | |
| 240 | + } else if (pdev->revision >= 0x10) { | |
| 241 | + switch (scu_rev & 0x0300) { | |
| 242 | + case 0x0200: | |
| 243 | + chip = AST1100; | |
| 244 | + dev_info(dev, "AST 1100 detected\n"); | |
| 245 | + break; | |
| 246 | + case 0x0100: | |
| 247 | + chip = AST2200; | |
| 248 | + dev_info(dev, "AST 2200 detected\n"); | |
| 249 | + break; | |
| 250 | + case 0x0000: | |
| 251 | + chip = AST2150; | |
| 252 | + dev_info(dev, "AST 2150 detected\n"); | |
| 253 | + break; | |
| 254 | + default: | |
| 255 | + chip = AST2100; | |
| 256 | + dev_info(dev, "AST 2100 detected\n"); | |
| 257 | + break; | |
| 258 | + } | |
| 259 | + } else { | |
| 260 | + chip = AST2000; | |
| 261 | + dev_info(dev, "AST 2000 detected\n"); | |
| 262 | + } | |
| 263 | + | |
| 264 | + *chip_out = chip; | |
| 265 | + *config_mode_out = config_mode; | |
| 266 | + | |
| 267 | + return 0; | |
| 268 | +} | |
| 269 | + | |
| 92 | 270 | static int ast_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) |
| 93 | 271 | { |
| 94 | − struct ast_device *ast; | |
| 95 | − struct drm_device *dev; | |
| 272 | + struct device *dev = &pdev->dev; | |
| 96 | 273 | int ret; |
| 274 | + void __iomem *regs; | |
| 275 | + void __iomem *ioregs; | |
| 276 | + enum ast_config_mode config_mode; | |
| 277 | + enum ast_chip chip; | |
| 278 | + struct drm_device *drm; | |
| 279 | + bool need_post = false; | |
| 97 | 280 | |
| 98 | 281 | ret = drm_aperture_remove_conflicting_pci_framebuffers(pdev, &ast_driver); |
| 99 | 282 | if (ret) |
| @@ -103,16 +286,80 @@ static int ast_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 3 unmodified lines | ||
| 103 | 286 | if (ret) |
| 104 | 287 | return ret; |
| 105 | 288 | |
| 106 | − ast = ast_device_create(&ast_driver, pdev, ent->driver_data); | |
| 107 | − if (IS_ERR(ast)) | |
| 108 | − return PTR_ERR(ast); | |
| 109 | − dev = &ast->base; | |
| 289 | + regs = pcim_iomap(pdev, 1, 0); | |
| 290 | + if (!regs) | |
| 291 | + return -EIO; | |
| 292 | + | |
| 293 | + if (pdev->revision >= 0x40) { | |
| 294 | + /* | |
| 295 | + * On AST2500 and later models, MMIO is enabled by | |
| 296 | + * default. Adopt it to be compatible with ARM. | |
| 297 | + */ | |
| 298 | + resource_size_t len = pci_resource_len(pdev, 1); | |
| 299 | + | |
| 300 | + if (len < AST_IO_MM_OFFSET) | |
| 301 | + return -EIO; | |
| 302 | + if ((len - AST_IO_MM_OFFSET) < AST_IO_MM_LENGTH) | |
| 303 | + return -EIO; | |
| 304 | + ioregs = regs + AST_IO_MM_OFFSET; | |
| 305 | + } else if (pci_resource_flags(pdev, 2) & IORESOURCE_IO) { | |
| 306 | + /* | |
| 307 | + * Map I/O registers if we have a PCI BAR for I/O. | |
| 308 | + */ | |
| 309 | + resource_size_t len = pci_resource_len(pdev, 2); | |
| 310 | + | |
| 311 | + if (len < AST_IO_MM_LENGTH) | |
| 312 | + return -EIO; | |
| 313 | + ioregs = pcim_iomap(pdev, 2, 0); | |
| 314 | + if (!ioregs) | |
| 315 | + return -EIO; | |
| 316 | + } else { | |
| 317 | + /* | |
| 318 | + * Anything else is best effort. | |
| 319 | + */ | |
| 320 | + resource_size_t len = pci_resource_len(pdev, 1); | |
| 321 | + | |
| 322 | + if (len < AST_IO_MM_OFFSET) | |
| 323 | + return -EIO; | |
| 324 | + if ((len - AST_IO_MM_OFFSET) < AST_IO_MM_LENGTH) | |
| 325 | + return -EIO; | |
| 326 | + ioregs = regs + AST_IO_MM_OFFSET; | |
| 327 | + | |
| 328 | + dev_info(dev, "Platform has no I/O space, using MMIO\n"); | |
| 329 | + } | |
| 330 | + | |
| 331 | + if (!ast_is_vga_enabled(ioregs)) { | |
| 332 | + dev_info(dev, "VGA not enabled on entry, requesting chip POST\n"); | |
| 333 | + need_post = true; | |
| 334 | + } | |
| 335 | + | |
| 336 | + /* | |
| 337 | + * If VGA isn't enabled, we need to enable now or subsequent | |
| 338 | + * access to the scratch registers will fail. | |
| 339 | + */ | |
| 340 | + if (need_post) | |
| 341 | + ast_enable_vga(ioregs); | |
| 342 | + /* Enable extended register access */ | |
| 343 | + ast_open_key(ioregs); | |
| 344 | + | |
| 345 | + ret = ast_enable_mmio(dev, ioregs); | |
| 346 | + if (ret) | |
| 347 | + return ret; | |
| 348 | + | |
| 349 | + ret = ast_detect_chip(pdev, regs, ioregs, &chip, &config_mode); | |
| 350 | + if (ret) | |
| 351 | + return ret; | |
| 352 | + | |
| 353 | + drm = ast_device_create(pdev, &ast_driver, chip, config_mode, regs, ioregs, need_post); | |
| 354 | + if (IS_ERR(drm)) | |
| 355 | + return PTR_ERR(drm); | |
| 356 | + pci_set_drvdata(pdev, drm); | |
| 110 | 357 | |
| 111 | − ret = drm_dev_register(dev, ent->driver_data); | |
| 358 | + ret = drm_dev_register(drm, ent->driver_data); | |
| 112 | 359 | if (ret) |
| 113 | 360 | return ret; |
| 114 | 361 | |
| 115 | − drm_fbdev_generic_setup(dev, 32); | |
| 362 | + drm_fbdev_generic_setup(drm, 32); | |
| 116 | 363 | |
| 117 | 364 | return 0; |
| 118 | 365 | } |
| @@ -98,6 +98,12 @@ enum ast_tx_chip { · 97 unmodified lines | ||
| 98 | 98 | #define AST_TX_DP501_BIT BIT(AST_TX_DP501) |
| 99 | 99 | #define AST_TX_ASTDP_BIT BIT(AST_TX_ASTDP) |
| 100 | 100 | |
| 101 | +enum ast_config_mode { | |
| 102 | + ast_use_p2a, | |
| 103 | + ast_use_dt, | |
| 104 | + ast_use_defaults | |
| 105 | +}; | |
| 106 | + | |
| 101 | 107 | #define AST_DRAM_512Mx16 0 |
| 102 | 108 | #define AST_DRAM_1Gx16 1 |
| 103 | 109 | #define AST_DRAM_512Mx32 2 |
| @@ -174,6 +180,17 @@ to_ast_sil164_connector(struct drm_connector *connector) · 70 unmodified lines | ||
| 174 | 180 | return container_of(connector, struct ast_sil164_connector, base); |
| 175 | 181 | } |
| 176 | 182 | |
| 183 | +struct ast_bmc_connector { | |
| 184 | + struct drm_connector base; | |
| 185 | + struct drm_connector *physical_connector; | |
| 186 | +}; | |
| 187 | + | |
| 188 | +static inline struct ast_bmc_connector * | |
| 189 | +to_ast_bmc_connector(struct drm_connector *connector) | |
| 190 | +{ | |
| 191 | + return container_of(connector, struct ast_bmc_connector, base); | |
| 192 | +} | |
| 193 | + | |
| 177 | 194 | /* |
| 178 | 195 | * Device |
| 179 | 196 | */ |
| @@ -181,12 +198,13 @@ to_ast_sil164_connector(struct drm_connector *connector) · 1 unmodified line | ||
| 181 | 198 | struct ast_device { |
| 182 | 199 | struct drm_device base; |
| 183 | 200 | |
| 184 | − struct mutex ioregs_lock; /* Protects access to I/O registers in ioregs */ | |
| 185 | 201 | void __iomem *regs; |
| 186 | 202 | void __iomem *ioregs; |
| 187 | 203 | void __iomem *dp501_fw_buf; |
| 188 | 204 | |
| 205 | + enum ast_config_mode config_mode; | |
| 189 | 206 | enum ast_chip chip; |
| 207 | + | |
| 190 | 208 | uint32_t dram_bus_width; |
| 191 | 209 | uint32_t dram_type; |
| 192 | 210 | uint32_t mclk; |
| @@ -196,6 +214,8 @@ struct ast_device { · 3 unmodified lines | ||
| 196 | 214 | unsigned long vram_size; |
| 197 | 215 | unsigned long vram_fb_available; |
| 198 | 216 | |
| 217 | + struct mutex modeset_lock; /* Protects access to modeset I/O registers in ioregs */ | |
| 218 | + | |
| 199 | 219 | struct ast_plane primary_plane; |
| 200 | 220 | struct ast_plane cursor_plane; |
| 201 | 221 | struct drm_crtc crtc; |
| @@ -218,16 +238,11 @@ struct ast_device { · 16 unmodified lines | ||
| 218 | 238 | } astdp; |
| 219 | 239 | struct { |
| 220 | 240 | struct drm_encoder encoder; |
| 221 | − struct drm_connector connector; | |
| 241 | + struct ast_bmc_connector bmc_connector; | |
| 222 | 242 | } bmc; |
| 223 | 243 | } output; |
| 224 | 244 | |
| 225 | 245 | bool support_wide_screen; |
| 226 | − enum { | |
| 227 | − ast_use_p2a, | |
| 228 | − ast_use_dt, | |
| 229 | − ast_use_defaults | |
| 230 | − } config_mode; | |
| 231 | 246 | |
| 232 | 247 | unsigned long tx_chip_types; /* bitfield of enum ast_chip_type */ |
| 233 | 248 | u8 *dp501_fw_addr; |
| @@ -239,9 +254,13 @@ static inline struct ast_device *to_ast_device(struct drm_device *dev) · 5 unmodified lines | ||
| 239 | 254 | return container_of(dev, struct ast_device, base); |
| 240 | 255 | } |
| 241 | 256 | |
| 242 | −struct ast_device *ast_device_create(const struct drm_driver *drv, | |
| 243 | − struct pci_dev *pdev, | |
| 244 | − unsigned long flags); | |
| 257 | +struct drm_device *ast_device_create(struct pci_dev *pdev, | |
| 258 | + const struct drm_driver *drv, | |
| 259 | + enum ast_chip chip, | |
| 260 | + enum ast_config_mode config_mode, | |
| 261 | + void __iomem *regs, | |
| 262 | + void __iomem *ioregs, | |
| 263 | + bool need_post); | |
| 245 | 264 | |
| 246 | 265 | static inline unsigned long __ast_gen(struct ast_device *ast) |
| 247 | 266 | { |
| @@ -261,55 +280,94 @@ static inline bool __ast_gen_is_eq(struct ast_device *ast, unsigned long gen) · 13 unmodified lines | ||
| 261 | 280 | #define IS_AST_GEN6(__ast) __ast_gen_is_eq(__ast, 6) |
| 262 | 281 | #define IS_AST_GEN7(__ast) __ast_gen_is_eq(__ast, 7) |
| 263 | 282 | |
| 283 | +static inline u8 __ast_read8(const void __iomem *addr, u32 reg) | |
| 284 | +{ | |
| 285 | + return ioread8(addr + reg); | |
| 286 | +} | |
| 287 | + | |
| 288 | +static inline u32 __ast_read32(const void __iomem *addr, u32 reg) | |
| 289 | +{ | |
| 290 | + return ioread32(addr + reg); | |
| 291 | +} | |
| 292 | + | |
| 293 | +static inline void __ast_write8(void __iomem *addr, u32 reg, u8 val) | |
| 294 | +{ | |
| 295 | + iowrite8(val, addr + reg); | |
| 296 | +} | |
| 297 | + | |
| 298 | +static inline void __ast_write32(void __iomem *addr, u32 reg, u32 val) | |
| 299 | +{ | |
| 300 | + iowrite32(val, addr + reg); | |
| 301 | +} | |
| 302 | + | |
| 303 | +static inline u8 __ast_read8_i(void __iomem *addr, u32 reg, u8 index) | |
| 304 | +{ | |
| 305 | + __ast_write8(addr, reg, index); | |
| 306 | + return __ast_read8(addr, reg + 1); | |
| 307 | +} | |
| 308 | + | |
| 309 | +static inline u8 __ast_read8_i_masked(void __iomem *addr, u32 reg, u8 index, u8 read_mask) | |
| 310 | +{ | |
| 311 | + u8 val = __ast_read8_i(addr, reg, index); | |
| 312 | + | |
| 313 | + return val & read_mask; | |
| 314 | +} | |
| 315 | + | |
| 316 | +static inline void __ast_write8_i(void __iomem *addr, u32 reg, u8 index, u8 val) | |
| 317 | +{ | |
| 318 | + __ast_write8(addr, reg, index); | |
| 319 | + __ast_write8(addr, reg + 1, val); | |
| 320 | +} | |
| 321 | + | |
| 322 | +static inline void __ast_write8_i_masked(void __iomem *addr, u32 reg, u8 index, u8 read_mask, | |
| 323 | + u8 val) | |
| 324 | +{ | |
| 325 | + u8 tmp = __ast_read8_i_masked(addr, reg, index, read_mask); | |
| 326 | + | |
| 327 | + tmp |= val; | |
| 328 | + __ast_write8_i(addr, reg, index, tmp); | |
| 329 | +} | |
| 330 | + | |
| 264 | 331 | static inline u32 ast_read32(struct ast_device *ast, u32 reg) |
| 265 | 332 | { |
| 266 | − return ioread32(ast->regs + reg); | |
| 333 | + return __ast_read32(ast->regs, reg); | |
| 267 | 334 | } |
| 268 | 335 | |
| 269 | 336 | static inline void ast_write32(struct ast_device *ast, u32 reg, u32 val) |
| 270 | 337 | { |
| 271 | − iowrite32(val, ast->regs + reg); | |
| 338 | + __ast_write32(ast->regs, reg, val); | |
| 272 | 339 | } |
| 273 | 340 | |
| 274 | 341 | static inline u8 ast_io_read8(struct ast_device *ast, u32 reg) |
| 275 | 342 | { |
| 276 | − return ioread8(ast->ioregs + reg); | |
| 343 | + return __ast_read8(ast->ioregs, reg); | |
| 277 | 344 | } |
| 278 | 345 | |
| 279 | 346 | static inline void ast_io_write8(struct ast_device *ast, u32 reg, u8 val) |
| 280 | 347 | { |
| 281 | − iowrite8(val, ast->ioregs + reg); | |
| 348 | + __ast_write8(ast->ioregs, reg, val); | |
| 282 | 349 | } |
| 283 | 350 | |
| 284 | 351 | static inline u8 ast_get_index_reg(struct ast_device *ast, u32 base, u8 index) |
| 285 | 352 | { |
| 286 | − ast_io_write8(ast, base, index); | |
| 287 | − ++base; | |
| 288 | − return ast_io_read8(ast, base); | |
| 353 | + return __ast_read8_i(ast->ioregs, base, index); | |
| 289 | 354 | } |
| 290 | 355 | |
| 291 | 356 | static inline u8 ast_get_index_reg_mask(struct ast_device *ast, u32 base, u8 index, |
| 292 | 357 | u8 preserve_mask) |
| 293 | 358 | { |
| 294 | − u8 val = ast_get_index_reg(ast, base, index); | |
| 295 | − | |
| 296 | − return val & preserve_mask; | |
| 359 | + return __ast_read8_i_masked(ast->ioregs, base, index, preserve_mask); | |
| 297 | 360 | } |
| 298 | 361 | |
| 299 | 362 | static inline void ast_set_index_reg(struct ast_device *ast, u32 base, u8 index, u8 val) |
| 300 | 363 | { |
| 301 | − ast_io_write8(ast, base, index); | |
| 302 | − ++base; | |
| 303 | − ast_io_write8(ast, base, val); | |
| 364 | + __ast_write8_i(ast->ioregs, base, index, val); | |
| 304 | 365 | } |
| 305 | 366 | |
| 306 | 367 | static inline void ast_set_index_reg_mask(struct ast_device *ast, u32 base, u8 index, |
| 307 | 368 | u8 preserve_mask, u8 val) |
| 308 | 369 | { |
| 309 | − u8 tmp = ast_get_index_reg_mask(ast, base, index, preserve_mask); | |
| 310 | − | |
| 311 | − tmp |= val; | |
| 312 | − ast_set_index_reg(ast, base, index, tmp); | |
| 370 | + __ast_write8_i_masked(ast->ioregs, base, index, preserve_mask, val); | |
| 313 | 371 | } |
| 314 | 372 | |
| 315 | 373 | #define AST_VIDMEM_SIZE_8M 0x00800000 |
| @@ -431,7 +489,7 @@ int ast_mm_init(struct ast_device *ast); · 115 unmodified lines | ||
| 431 | 489 | void ast_post_gpu(struct drm_device *dev); |
| 432 | 490 | u32 ast_mindwm(struct ast_device *ast, u32 r); |
| 433 | 491 | void ast_moutdwm(struct ast_device *ast, u32 r, u32 v); |
| 434 | −void ast_patch_ahb_2500(struct ast_device *ast); | |
| 492 | +void ast_patch_ahb_2500(void __iomem *regs); | |
| 435 | 493 | /* ast dp501 */ |
| 436 | 494 | void ast_set_dp501_video_output(struct drm_device *dev, u8 mode); |
| 437 | 495 | bool ast_backup_fw(struct drm_device *dev, u8 *addr, u32 size); |
| @@ -35,180 +35,6 @@ · 34 unmodified lines | ||
| 35 | 35 | |
| 36 | 36 | #include "ast_drv.h" |
| 37 | 37 | |
| 38 | −static bool ast_is_vga_enabled(struct drm_device *dev) | |
| 39 | −{ | |
| 40 | − struct ast_device *ast = to_ast_device(dev); | |
| 41 | − u8 ch; | |
| 42 | − | |
| 43 | − ch = ast_io_read8(ast, AST_IO_VGAER); | |
| 44 | − | |
| 45 | − return !!(ch & 0x01); | |
| 46 | −} | |
| 47 | − | |
| 48 | −static void ast_enable_vga(struct drm_device *dev) | |
| 49 | −{ | |
| 50 | − struct ast_device *ast = to_ast_device(dev); | |
| 51 | − | |
| 52 | − ast_io_write8(ast, AST_IO_VGAER, 0x01); | |
| 53 | − ast_io_write8(ast, AST_IO_VGAMR_W, 0x01); | |
| 54 | −} | |
| 55 | − | |
| 56 | −/* | |
| 57 | − * Run this function as part of the HW device cleanup; not | |
| 58 | − * when the DRM device gets released. | |
| 59 | − */ | |
| 60 | −static void ast_enable_mmio_release(void *data) | |
| 61 | −{ | |
| 62 | − struct ast_device *ast = data; | |
| 63 | − | |
| 64 | − /* enable standard VGA decode */ | |
| 65 | − ast_set_index_reg(ast, AST_IO_VGACRI, 0xa1, 0x04); | |
| 66 | −} | |
| 67 | − | |
| 68 | −static int ast_enable_mmio(struct ast_device *ast) | |
| 69 | −{ | |
| 70 | − struct drm_device *dev = &ast->base; | |
| 71 | − | |
| 72 | − ast_set_index_reg(ast, AST_IO_VGACRI, 0xa1, 0x06); | |
| 73 | − | |
| 74 | − return devm_add_action_or_reset(dev->dev, ast_enable_mmio_release, ast); | |
| 75 | −} | |
| 76 | − | |
| 77 | −static void ast_open_key(struct ast_device *ast) | |
| 78 | −{ | |
| 79 | − ast_set_index_reg(ast, AST_IO_VGACRI, 0x80, 0xA8); | |
| 80 | −} | |
| 81 | − | |
| 82 | −static int ast_device_config_init(struct ast_device *ast) | |
| 83 | −{ | |
| 84 | − struct drm_device *dev = &ast->base; | |
| 85 | − struct pci_dev *pdev = to_pci_dev(dev->dev); | |
| 86 | − struct device_node *np = dev->dev->of_node; | |
| 87 | − uint32_t scu_rev = 0xffffffff; | |
| 88 | − u32 data; | |
| 89 | − u8 jregd0, jregd1; | |
| 90 | − | |
| 91 | − /* | |
| 92 | − * Find configuration mode and read SCU revision | |
| 93 | − */ | |
| 94 | − | |
| 95 | − ast->config_mode = ast_use_defaults; | |
| 96 | − | |
| 97 | − /* Check if we have device-tree properties */ | |
| 98 | − if (np && !of_property_read_u32(np, "aspeed,scu-revision-id", &data)) { | |
| 99 | − /* We do, disable P2A access */ | |
| 100 | − ast->config_mode = ast_use_dt; | |
| 101 | − scu_rev = data; | |
| 102 | − } else if (pdev->device == PCI_CHIP_AST2000) { // Not all families have a P2A bridge | |
| 103 | − /* | |
| 104 | − * The BMC will set SCU 0x40 D[12] to 1 if the P2 bridge | |
| 105 | − * is disabled. We force using P2A if VGA only mode bit | |
| 106 | − * is set D[7] | |
| 107 | − */ | |
| 108 | − jregd0 = ast_get_index_reg_mask(ast, AST_IO_VGACRI, 0xd0, 0xff); | |
| 109 | − jregd1 = ast_get_index_reg_mask(ast, AST_IO_VGACRI, 0xd1, 0xff); | |
| 110 | − if (!(jregd0 & 0x80) || !(jregd1 & 0x10)) { | |
| 111 | − | |
| 112 | − /* | |
| 113 | − * We have a P2A bridge and it is enabled. | |
| 114 | − */ | |
| 115 | − | |
| 116 | − /* Patch AST2500/AST2510 */ | |
| 117 | − if ((pdev->revision & 0xf0) == 0x40) { | |
| 118 | − if (!(jregd0 & AST_VRAM_INIT_STATUS_MASK)) | |
| 119 | − ast_patch_ahb_2500(ast); | |
| 120 | − } | |
| 121 | − | |
| 122 | − /* Double check that it's actually working */ | |
| 123 | − data = ast_read32(ast, 0xf004); | |
| 124 | − if ((data != 0xffffffff) && (data != 0x00)) { | |
| 125 | − ast->config_mode = ast_use_p2a; | |
| 126 | − | |
| 127 | − /* Read SCU7c (silicon revision register) */ | |
| 128 | − ast_write32(ast, 0xf004, 0x1e6e0000); | |
| 129 | − ast_write32(ast, 0xf000, 0x1); | |
| 130 | − scu_rev = ast_read32(ast, 0x1207c); | |
| 131 | − } | |
| 132 | − } | |
| 133 | − } | |
| 134 | − | |
| 135 | − switch (ast->config_mode) { | |
| 136 | − case ast_use_defaults: | |
| 137 | − drm_info(dev, "Using default configuration\n"); | |
| 138 | − break; | |
| 139 | − case ast_use_dt: | |
| 140 | − drm_info(dev, "Using device-tree for configuration\n"); | |
| 141 | − break; | |
| 142 | − case ast_use_p2a: | |
| 143 | − drm_info(dev, "Using P2A bridge for configuration\n"); | |
| 144 | − break; | |
| 145 | − } | |
| 146 | − | |
| 147 | − /* | |
| 148 | − * Identify chipset | |
| 149 | − */ | |
| 150 | − | |
| 151 | − if (pdev->revision >= 0x50) { | |
| 152 | − ast->chip = AST2600; | |
| 153 | − drm_info(dev, "AST 2600 detected\n"); | |
| 154 | − } else if (pdev->revision >= 0x40) { | |
| 155 | − switch (scu_rev & 0x300) { | |
| 156 | − case 0x0100: | |
| 157 | − ast->chip = AST2510; | |
| 158 | − drm_info(dev, "AST 2510 detected\n"); | |
| 159 | − break; | |
| 160 | − default: | |
| 161 | − ast->chip = AST2500; | |
| 162 | − drm_info(dev, "AST 2500 detected\n"); | |
| 163 | − } | |
| 164 | − } else if (pdev->revision >= 0x30) { | |
| 165 | − switch (scu_rev & 0x300) { | |
| 166 | − case 0x0100: | |
| 167 | − ast->chip = AST1400; | |
| 168 | − drm_info(dev, "AST 1400 detected\n"); | |
| 169 | − break; | |
| 170 | − default: | |
| 171 | − ast->chip = AST2400; | |
| 172 | − drm_info(dev, "AST 2400 detected\n"); | |
| 173 | − } | |
| 174 | − } else if (pdev->revision >= 0x20) { | |
| 175 | − switch (scu_rev & 0x300) { | |
| 176 | − case 0x0000: | |
| 177 | − ast->chip = AST1300; | |
| 178 | − drm_info(dev, "AST 1300 detected\n"); | |
| 179 | − break; | |
| 180 | − default: | |
| 181 | − ast->chip = AST2300; | |
| 182 | − drm_info(dev, "AST 2300 detected\n"); | |
| 183 | − break; | |
| 184 | − } | |
| 185 | − } else if (pdev->revision >= 0x10) { | |
| 186 | − switch (scu_rev & 0x0300) { | |
| 187 | − case 0x0200: | |
| 188 | − ast->chip = AST1100; | |
| 189 | − drm_info(dev, "AST 1100 detected\n"); | |
| 190 | − break; | |
| 191 | − case 0x0100: | |
| 192 | − ast->chip = AST2200; | |
| 193 | − drm_info(dev, "AST 2200 detected\n"); | |
| 194 | − break; | |
| 195 | − case 0x0000: | |
| 196 | − ast->chip = AST2150; | |
| 197 | − drm_info(dev, "AST 2150 detected\n"); | |
| 198 | − break; | |
| 199 | − default: | |
| 200 | − ast->chip = AST2100; | |
| 201 | − drm_info(dev, "AST 2100 detected\n"); | |
| 202 | − break; | |
| 203 | − } | |
| 204 | − } else { | |
| 205 | − ast->chip = AST2000; | |
| 206 | − drm_info(dev, "AST 2000 detected\n"); | |
| 207 | − } | |
| 208 | − | |
| 209 | − return 0; | |
| 210 | −} | |
| 211 | − | |
| 212 | 38 | static void ast_detect_widescreen(struct ast_device *ast) |
| 213 | 39 | { |
| 214 | 40 | u8 jreg; |
| @@ -424,69 +250,27 @@ static int ast_get_dram_info(struct drm_device *dev) · 209 unmodified lines | ||
| 424 | 250 | return 0; |
| 425 | 251 | } |
| 426 | 252 | |
| 427 | −struct ast_device *ast_device_create(const struct drm_driver *drv, | |
| 428 | − struct pci_dev *pdev, | |
| 429 | − unsigned long flags) | |
| 253 | +struct drm_device *ast_device_create(struct pci_dev *pdev, | |
| 254 | + const struct drm_driver *drv, | |
| 255 | + enum ast_chip chip, | |
| 256 | + enum ast_config_mode config_mode, | |
| 257 | + void __iomem *regs, | |
| 258 | + void __iomem *ioregs, | |
| 259 | + bool need_post) | |
| 430 | 260 | { |
| 431 | 261 | struct drm_device *dev; |
| 432 | 262 | struct ast_device *ast; |
| 433 | − bool need_post = false; | |
| 434 | − int ret = 0; | |
| 263 | + int ret; | |
| 435 | 264 | |
| 436 | 265 | ast = devm_drm_dev_alloc(&pdev->dev, drv, struct ast_device, base); |
| 437 | 266 | if (IS_ERR(ast)) |
| 438 | − return ast; | |
| 267 | + return ERR_CAST(ast); | |
| 439 | 268 | dev = &ast->base; |
| 440 | 269 | |
| 441 | − pci_set_drvdata(pdev, dev); | |
| 442 | − | |
| 443 | − ret = drmm_mutex_init(dev, &ast->ioregs_lock); | |
| 444 | − if (ret) | |
| 445 | − return ERR_PTR(ret); | |
| 446 | − | |
| 447 | − ast->regs = pcim_iomap(pdev, 1, 0); | |
| 448 | − if (!ast->regs) | |
| 449 | − return ERR_PTR(-EIO); | |
| 450 | − | |
| 451 | − /* | |
| 452 | − * After AST2500, MMIO is enabled by default, and it should be adopted | |
| 453 | − * to be compatible with Arm. | |
| 454 | − */ | |
| 455 | − if (pdev->revision >= 0x40) { | |
| 456 | − ast->ioregs = ast->regs + AST_IO_MM_OFFSET; | |
| 457 | − } else if (!(pci_resource_flags(pdev, 2) & IORESOURCE_IO)) { | |
| 458 | − drm_info(dev, "platform has no IO space, trying MMIO\n"); | |
| 459 | − ast->ioregs = ast->regs + AST_IO_MM_OFFSET; | |
| 460 | − } | |
| 461 | − | |
| 462 | − /* "map" IO regs if the above hasn't done so already */ | |
| 463 | − if (!ast->ioregs) { | |
| 464 | − ast->ioregs = pcim_iomap(pdev, 2, 0); | |
| 465 | − if (!ast->ioregs) | |
| 466 | − return ERR_PTR(-EIO); | |
| 467 | − } | |
| 468 | − | |
| 469 | − if (!ast_is_vga_enabled(dev)) { | |
| 470 | − drm_info(dev, "VGA not enabled on entry, requesting chip POST\n"); | |
| 471 | − need_post = true; | |
| 472 | − } | |
| 473 | − | |
| 474 | − /* | |
| 475 | − * If VGA isn't enabled, we need to enable now or subsequent | |
| 476 | − * access to the scratch registers will fail. | |
| 477 | − */ | |
| 478 | − if (need_post) | |
| 479 | − ast_enable_vga(dev); | |
| 480 | − | |
| 481 | − /* Enable extended register access */ | |
| 482 | − ast_open_key(ast); | |
| 483 | − ret = ast_enable_mmio(ast); | |
| 484 | − if (ret) | |
| 485 | − return ERR_PTR(ret); | |
| 486 | − | |
| 487 | − ret = ast_device_config_init(ast); | |
| 488 | − if (ret) | |
| 489 | − return ERR_PTR(ret); | |
| 270 | + ast->chip = chip; | |
| 271 | + ast->config_mode = config_mode; | |
| 272 | + ast->regs = regs; | |
| 273 | + ast->ioregs = ioregs; | |
| 490 | 274 | |
| 491 | 275 | ast_detect_widescreen(ast); |
| 492 | 276 | ast_detect_tx_chip(ast, need_post); |
| @@ -517,5 +301,5 @@ struct ast_device *ast_device_create(const struct drm_driver *drv, · 24 unmodified lines | ||
| 517 | 301 | if (ret) |
| 518 | 302 | return ERR_PTR(ret); |
| 519 | 303 | |
| 520 | − return ast; | |
| 304 | + return dev; | |
| 521 | 305 | } |
| @@ -1358,13 +1358,13 @@ static int ast_vga_connector_helper_get_modes(struct drm_connector *connector) · 1357 unmodified lines | ||
| 1358 | 1358 | * Protect access to I/O registers from concurrent modesetting |
| 1359 | 1359 | * by acquiring the I/O-register lock. |
| 1360 | 1360 | */ |
| 1361 | − mutex_lock(&ast->ioregs_lock); | |
| 1361 | + mutex_lock(&ast->modeset_lock); | |
| 1362 | 1362 | |
| 1363 | 1363 | edid = drm_get_edid(connector, &ast_vga_connector->i2c->adapter); |
| 1364 | 1364 | if (!edid) |
| 1365 | 1365 | goto err_mutex_unlock; |
| 1366 | 1366 | |
| 1367 | − mutex_unlock(&ast->ioregs_lock); | |
| 1367 | + mutex_unlock(&ast->modeset_lock); | |
| 1368 | 1368 | |
| 1369 | 1369 | count = drm_add_edid_modes(connector, edid); |
| 1370 | 1370 | kfree(edid); |
| @@ -1372,7 +1372,7 @@ static int ast_vga_connector_helper_get_modes(struct drm_connector *connector) · 1 unmodified line | ||
| 1372 | 1372 | return count; |
| 1373 | 1373 | |
| 1374 | 1374 | err_mutex_unlock: |
| 1375 | − mutex_unlock(&ast->ioregs_lock); | |
| 1375 | + mutex_unlock(&ast->modeset_lock); | |
| 1376 | 1376 | err_drm_connector_update_edid_property: |
| 1377 | 1377 | drm_connector_update_edid_property(connector, NULL); |
| 1378 | 1378 | return 0; |
| @@ -1464,13 +1464,13 @@ static int ast_sil164_connector_helper_get_modes(struct drm_connector *connector · 85 unmodified lines | ||
| 1464 | 1464 | * Protect access to I/O registers from concurrent modesetting |
| 1465 | 1465 | * by acquiring the I/O-register lock. |
| 1466 | 1466 | */ |
| 1467 | − mutex_lock(&ast->ioregs_lock); | |
| 1467 | + mutex_lock(&ast->modeset_lock); | |
| 1468 | 1468 | |
| 1469 | 1469 | edid = drm_get_edid(connector, &ast_sil164_connector->i2c->adapter); |
| 1470 | 1470 | if (!edid) |
| 1471 | 1471 | goto err_mutex_unlock; |
| 1472 | 1472 | |
| 1473 | − mutex_unlock(&ast->ioregs_lock); | |
| 1473 | + mutex_unlock(&ast->modeset_lock); | |
| 1474 | 1474 | |
| 1475 | 1475 | count = drm_add_edid_modes(connector, edid); |
| 1476 | 1476 | kfree(edid); |
| @@ -1478,7 +1478,7 @@ static int ast_sil164_connector_helper_get_modes(struct drm_connector *connector · 1 unmodified line | ||
| 1478 | 1478 | return count; |
| 1479 | 1479 | |
| 1480 | 1480 | err_mutex_unlock: |
| 1481 | − mutex_unlock(&ast->ioregs_lock); | |
| 1481 | + mutex_unlock(&ast->modeset_lock); | |
| 1482 | 1482 | err_drm_connector_update_edid_property: |
| 1483 | 1483 | drm_connector_update_edid_property(connector, NULL); |
| 1484 | 1484 | return 0; |
| @@ -1670,13 +1670,13 @@ static int ast_astdp_connector_helper_get_modes(struct drm_connector *connector) · 185 unmodified lines | ||
| 1670 | 1670 | * Protect access to I/O registers from concurrent modesetting |
| 1671 | 1671 | * by acquiring the I/O-register lock. |
| 1672 | 1672 | */ |
| 1673 | − mutex_lock(&ast->ioregs_lock); | |
| 1673 | + mutex_lock(&ast->modeset_lock); | |
| 1674 | 1674 | |
| 1675 | 1675 | succ = ast_astdp_read_edid(connector->dev, edid); |
| 1676 | 1676 | if (succ < 0) |
| 1677 | 1677 | goto err_mutex_unlock; |
| 1678 | 1678 | |
| 1679 | − mutex_unlock(&ast->ioregs_lock); | |
| 1679 | + mutex_unlock(&ast->modeset_lock); | |
| 1680 | 1680 | |
| 1681 | 1681 | drm_connector_update_edid_property(connector, edid); |
| 1682 | 1682 | count = drm_add_edid_modes(connector, edid); |
| @@ -1685,7 +1685,7 @@ static int ast_astdp_connector_helper_get_modes(struct drm_connector *connector) · 2 unmodified lines | ||
| 1685 | 1685 | return count; |
| 1686 | 1686 | |
| 1687 | 1687 | err_mutex_unlock: |
| 1688 | − mutex_unlock(&ast->ioregs_lock); | |
| 1688 | + mutex_unlock(&ast->modeset_lock); | |
| 1689 | 1689 | kfree(edid); |
| 1690 | 1690 | err_drm_connector_update_edid_property: |
| 1691 | 1691 | drm_connector_update_edid_property(connector, NULL); |
| @@ -1767,6 +1767,30 @@ static const struct drm_encoder_funcs ast_bmc_encoder_funcs = { · 75 unmodified lines | ||
| 1767 | 1767 | .destroy = drm_encoder_cleanup, |
| 1768 | 1768 | }; |
| 1769 | 1769 | |
| 1770 | +static int ast_bmc_connector_helper_detect_ctx(struct drm_connector *connector, | |
| 1771 | + struct drm_modeset_acquire_ctx *ctx, | |
| 1772 | + bool force) | |
| 1773 | +{ | |
| 1774 | + struct ast_bmc_connector *bmc_connector = to_ast_bmc_connector(connector); | |
| 1775 | + struct drm_connector *physical_connector = bmc_connector->physical_connector; | |
| 1776 | + | |
| 1777 | + /* | |
| 1778 | + * Most user-space compositors cannot handle more than one connected | |
| 1779 | + * connector per CRTC. Hence, we only mark the BMC as connected if the | |
| 1780 | + * physical connector is disconnected. If the physical connector's status | |
| 1781 | + * is connected or unknown, the BMC remains disconnected. This has no | |
| 1782 | + * effect on the output of the BMC. | |
| 1783 | + * | |
| 1784 | + * FIXME: Remove this logic once user-space compositors can handle more | |
| 1785 | + * than one connector per CRTC. The BMC should always be connected. | |
| 1786 | + */ | |
| 1787 | + | |
| 1788 | + if (physical_connector && physical_connector->status == connector_status_disconnected) | |
| 1789 | + return connector_status_connected; | |
| 1790 | + | |
| 1791 | + return connector_status_disconnected; | |
| 1792 | +} | |
| 1793 | + | |
| 1770 | 1794 | static int ast_bmc_connector_helper_get_modes(struct drm_connector *connector) |
| 1771 | 1795 | { |
| 1772 | 1796 | return drm_add_modes_noedid(connector, 4096, 4096); |
| @@ -1774,6 +1798,7 @@ static int ast_bmc_connector_helper_get_modes(struct drm_connector *connector) · 1 unmodified line | ||
| 1774 | 1798 | |
| 1775 | 1799 | static const struct drm_connector_helper_funcs ast_bmc_connector_helper_funcs = { |
| 1776 | 1800 | .get_modes = ast_bmc_connector_helper_get_modes, |
| 1801 | + .detect_ctx = ast_bmc_connector_helper_detect_ctx, | |
| 1777 | 1802 | }; |
| 1778 | 1803 | |
| 1779 | 1804 | static const struct drm_connector_funcs ast_bmc_connector_funcs = { |
| @@ -1784,12 +1809,33 @@ static const struct drm_connector_funcs ast_bmc_connector_funcs = { · 4 unmodified lines | ||
| 1784 | 1809 | .atomic_destroy_state = drm_atomic_helper_connector_destroy_state, |
| 1785 | 1810 | }; |
| 1786 | 1811 | |
| 1787 | −static int ast_bmc_output_init(struct ast_device *ast) | |
| 1812 | +static int ast_bmc_connector_init(struct drm_device *dev, | |
| 1813 | + struct ast_bmc_connector *bmc_connector, | |
| 1814 | + struct drm_connector *physical_connector) | |
| 1815 | +{ | |
| 1816 | + struct drm_connector *connector = &bmc_connector->base; | |
| 1817 | + int ret; | |
| 1818 | + | |
| 1819 | + ret = drm_connector_init(dev, connector, &ast_bmc_connector_funcs, | |
| 1820 | + DRM_MODE_CONNECTOR_VIRTUAL); | |
| 1821 | + if (ret) | |
| 1822 | + return ret; | |
| 1823 | + | |
| 1824 | + drm_connector_helper_add(connector, &ast_bmc_connector_helper_funcs); | |
| 1825 | + | |
| 1826 | + bmc_connector->physical_connector = physical_connector; | |
| 1827 | + | |
| 1828 | + return 0; | |
| 1829 | +} | |
| 1830 | + | |
| 1831 | +static int ast_bmc_output_init(struct ast_device *ast, | |
| 1832 | + struct drm_connector *physical_connector) | |
| 1788 | 1833 | { |
| 1789 | 1834 | struct drm_device *dev = &ast->base; |
| 1790 | 1835 | struct drm_crtc *crtc = &ast->crtc; |
| 1791 | 1836 | struct drm_encoder *encoder = &ast->output.bmc.encoder; |
| 1792 | − struct drm_connector *connector = &ast->output.bmc.connector; | |
| 1837 | + struct ast_bmc_connector *bmc_connector = &ast->output.bmc.bmc_connector; | |
| 1838 | + struct drm_connector *connector = &bmc_connector->base; | |
| 1793 | 1839 | int ret; |
| 1794 | 1840 | |
| 1795 | 1841 | ret = drm_encoder_init(dev, encoder, |
| @@ -1799,13 +1845,10 @@ static int ast_bmc_output_init(struct ast_device *ast) · 3 unmodified lines | ||
| 1799 | 1845 | return ret; |
| 1800 | 1846 | encoder->possible_crtcs = drm_crtc_mask(crtc); |
| 1801 | 1847 | |
| 1802 | − ret = drm_connector_init(dev, connector, &ast_bmc_connector_funcs, | |
| 1803 | − DRM_MODE_CONNECTOR_VIRTUAL); | |
| 1848 | + ret = ast_bmc_connector_init(dev, bmc_connector, physical_connector); | |
| 1804 | 1849 | if (ret) |
| 1805 | 1850 | return ret; |
| 1806 | 1851 | |
| 1807 | − drm_connector_helper_add(connector, &ast_bmc_connector_helper_funcs); | |
| 1808 | − | |
| 1809 | 1852 | ret = drm_connector_attach_encoder(connector, encoder); |
| 1810 | 1853 | if (ret) |
| 1811 | 1854 | return ret; |
| @@ -1827,9 +1870,9 @@ static void ast_mode_config_helper_atomic_commit_tail(struct drm_atomic_state *s · 15 unmodified lines | ||
| 1827 | 1870 | * display modes. Protect access to I/O registers by acquiring |
| 1828 | 1871 | * the I/O-register lock. Released in atomic_flush(). |
| 1829 | 1872 | */ |
| 1830 | − mutex_lock(&ast->ioregs_lock); | |
| 1873 | + mutex_lock(&ast->modeset_lock); | |
| 1831 | 1874 | drm_atomic_helper_commit_tail_rpm(state); |
| 1832 | − mutex_unlock(&ast->ioregs_lock); | |
| 1875 | + mutex_unlock(&ast->modeset_lock); | |
| 1833 | 1876 | } |
| 1834 | 1877 | |
| 1835 | 1878 | static const struct drm_mode_config_helper_funcs ast_mode_config_helper_funcs = { |
| @@ -1864,8 +1907,13 @@ static const struct drm_mode_config_funcs ast_mode_config_funcs = { · 28 unmodified lines | ||
| 1864 | 1907 | int ast_mode_config_init(struct ast_device *ast) |
| 1865 | 1908 | { |
| 1866 | 1909 | struct drm_device *dev = &ast->base; |
| 1910 | + struct drm_connector *physical_connector = NULL; | |
| 1867 | 1911 | int ret; |
| 1868 | 1912 | |
| 1913 | + ret = drmm_mutex_init(dev, &ast->modeset_lock); | |
| 1914 | + if (ret) | |
| 1915 | + return ret; | |
| 1916 | + | |
| 1869 | 1917 | ret = drmm_mode_config_init(dev); |
| 1870 | 1918 | if (ret) |
| 1871 | 1919 | return ret; |
| @@ -1904,23 +1952,27 @@ int ast_mode_config_init(struct ast_device *ast) · 32 unmodified lines | ||
| 1904 | 1952 | ret = ast_vga_output_init(ast); |
| 1905 | 1953 | if (ret) |
| 1906 | 1954 | return ret; |
| 1955 | + physical_connector = &ast->output.vga.vga_connector.base; | |
| 1907 | 1956 | } |
| 1908 | 1957 | if (ast->tx_chip_types & AST_TX_SIL164_BIT) { |
| 1909 | 1958 | ret = ast_sil164_output_init(ast); |
| 1910 | 1959 | if (ret) |
| 1911 | 1960 | return ret; |
| 1961 | + physical_connector = &ast->output.sil164.sil164_connector.base; | |
| 1912 | 1962 | } |
| 1913 | 1963 | if (ast->tx_chip_types & AST_TX_DP501_BIT) { |
| 1914 | 1964 | ret = ast_dp501_output_init(ast); |
| 1915 | 1965 | if (ret) |
| 1916 | 1966 | return ret; |
| 1967 | + physical_connector = &ast->output.dp501.connector; | |
| 1917 | 1968 | } |
| 1918 | 1969 | if (ast->tx_chip_types & AST_TX_ASTDP_BIT) { |
| 1919 | 1970 | ret = ast_astdp_output_init(ast); |
| 1920 | 1971 | if (ret) |
| 1921 | 1972 | return ret; |
| 1973 | + physical_connector = &ast->output.astdp.connector; | |
| 1922 | 1974 | } |
| 1923 | − ret = ast_bmc_output_init(ast); | |
| 1975 | + ret = ast_bmc_output_init(ast, physical_connector); | |
| 1924 | 1976 | if (ret) |
| 1925 | 1977 | return ret; |
| 1926 | 1978 | |
| @@ -77,28 +77,42 @@ ast_set_def_ext_reg(struct drm_device *dev) · 76 unmodified lines | ||
| 77 | 77 | ast_set_index_reg_mask(ast, AST_IO_VGACRI, 0xb6, 0xff, reg); |
| 78 | 78 | } |
| 79 | 79 | |
| 80 | −u32 ast_mindwm(struct ast_device *ast, u32 r) | |
| 80 | +static u32 __ast_mindwm(void __iomem *regs, u32 r) | |
| 81 | 81 | { |
| 82 | − uint32_t data; | |
| 82 | + u32 data; | |
| 83 | 83 | |
| 84 | − ast_write32(ast, 0xf004, r & 0xffff0000); | |
| 85 | − ast_write32(ast, 0xf000, 0x1); | |
| 84 | + __ast_write32(regs, 0xf004, r & 0xffff0000); | |
| 85 | + __ast_write32(regs, 0xf000, 0x1); | |
| 86 | 86 | |
| 87 | 87 | do { |
| 88 | − data = ast_read32(ast, 0xf004) & 0xffff0000; | |
| 88 | + data = __ast_read32(regs, 0xf004) & 0xffff0000; | |
| 89 | 89 | } while (data != (r & 0xffff0000)); |
| 90 | − return ast_read32(ast, 0x10000 + (r & 0x0000ffff)); | |
| 90 | + | |
| 91 | + return __ast_read32(regs, 0x10000 + (r & 0x0000ffff)); | |
| 91 | 92 | } |
| 92 | 93 | |
| 93 | −void ast_moutdwm(struct ast_device *ast, u32 r, u32 v) | |
| 94 | +static void __ast_moutdwm(void __iomem *regs, u32 r, u32 v) | |
| 94 | 95 | { |
| 95 | − uint32_t data; | |
| 96 | − ast_write32(ast, 0xf004, r & 0xffff0000); | |
| 97 | − ast_write32(ast, 0xf000, 0x1); | |
| 96 | + u32 data; | |
| 97 | + | |
| 98 | + __ast_write32(regs, 0xf004, r & 0xffff0000); | |
| 99 | + __ast_write32(regs, 0xf000, 0x1); | |
| 100 | + | |
| 98 | 101 | do { |
| 99 | − data = ast_read32(ast, 0xf004) & 0xffff0000; | |
| 102 | + data = __ast_read32(regs, 0xf004) & 0xffff0000; | |
| 100 | 103 | } while (data != (r & 0xffff0000)); |
| 101 | − ast_write32(ast, 0x10000 + (r & 0x0000ffff), v); | |
| 104 | + | |
| 105 | + __ast_write32(regs, 0x10000 + (r & 0x0000ffff), v); | |
| 106 | +} | |
| 107 | + | |
| 108 | +u32 ast_mindwm(struct ast_device *ast, u32 r) | |
| 109 | +{ | |
| 110 | + return __ast_mindwm(ast->regs, r); | |
| 111 | +} | |
| 112 | + | |
| 113 | +void ast_moutdwm(struct ast_device *ast, u32 r, u32 v) | |
| 114 | +{ | |
| 115 | + __ast_moutdwm(ast->regs, r, v); | |
| 102 | 116 | } |
| 103 | 117 | |
| 104 | 118 | /* |
| @@ -1987,17 +2001,18 @@ static bool ast_dram_init_2500(struct ast_device *ast) · 1882 unmodified lines | ||
| 1987 | 2001 | return true; |
| 1988 | 2002 | } |
| 1989 | 2003 | |
| 1990 | −void ast_patch_ahb_2500(struct ast_device *ast) | |
| 2004 | +void ast_patch_ahb_2500(void __iomem *regs) | |
| 1991 | 2005 | { |
| 1992 | − u32 data; | |
| 2006 | + u32 data; | |
| 1993 | 2007 | |
| 1994 | 2008 | /* Clear bus lock condition */ |
| 1995 | − ast_moutdwm(ast, 0x1e600000, 0xAEED1A03); | |
| 1996 | − ast_moutdwm(ast, 0x1e600084, 0x00010000); | |
| 1997 | − ast_moutdwm(ast, 0x1e600088, 0x00000000); | |
| 1998 | − ast_moutdwm(ast, 0x1e6e2000, 0x1688A8A8); | |
| 1999 | − data = ast_mindwm(ast, 0x1e6e2070); | |
| 2000 | − if (data & 0x08000000) { /* check fast reset */ | |
| 2009 | + __ast_moutdwm(regs, 0x1e600000, 0xAEED1A03); | |
| 2010 | + __ast_moutdwm(regs, 0x1e600084, 0x00010000); | |
| 2011 | + __ast_moutdwm(regs, 0x1e600088, 0x00000000); | |
| 2012 | + __ast_moutdwm(regs, 0x1e6e2000, 0x1688A8A8); | |
| 2013 | + | |
| 2014 | + data = __ast_mindwm(regs, 0x1e6e2070); | |
| 2015 | + if (data & 0x08000000) { /* check fast reset */ | |
| 2001 | 2016 | /* |
| 2002 | 2017 | * If "Fast restet" is enabled for ARM-ICE debugger, |
| 2003 | 2018 | * then WDT needs to enable, that |
| @@ -2009,16 +2024,18 @@ void ast_patch_ahb_2500(struct ast_device *ast) · 5 unmodified lines | ||
| 2009 | 2024 | * [1]:= 1:WDT will be cleeared and disabled after timeout occurs |
| 2010 | 2025 | * [0]:= 1:WDT enable |
| 2011 | 2026 | */ |
| 2012 | − ast_moutdwm(ast, 0x1E785004, 0x00000010); | |
| 2013 | − ast_moutdwm(ast, 0x1E785008, 0x00004755); | |
| 2014 | − ast_moutdwm(ast, 0x1E78500c, 0x00000033); | |
| 2027 | + __ast_moutdwm(regs, 0x1E785004, 0x00000010); | |
| 2028 | + __ast_moutdwm(regs, 0x1E785008, 0x00004755); | |
| 2029 | + __ast_moutdwm(regs, 0x1E78500c, 0x00000033); | |
| 2015 | 2030 | udelay(1000); |
| 2016 | 2031 | } |
| 2032 | + | |
| 2017 | 2033 | do { |
| 2018 | − ast_moutdwm(ast, 0x1e6e2000, 0x1688A8A8); | |
| 2019 | − data = ast_mindwm(ast, 0x1e6e2000); | |
| 2020 | − } while (data != 1); | |
| 2021 | − ast_moutdwm(ast, 0x1e6e207c, 0x08000000); /* clear fast reset */ | |
| 2034 | + __ast_moutdwm(regs, 0x1e6e2000, 0x1688A8A8); | |
| 2035 | + data = __ast_mindwm(regs, 0x1e6e2000); | |
| 2036 | + } while (data != 1); | |
| 2037 | + | |
| 2038 | + __ast_moutdwm(regs, 0x1e6e207c, 0x08000000); /* clear fast reset */ | |
| 2022 | 2039 | } |
| 2023 | 2040 | |
| 2024 | 2041 | void ast_post_chip_2500(struct drm_device *dev) |
| @@ -2030,7 +2047,7 @@ void ast_post_chip_2500(struct drm_device *dev) · 5 unmodified lines | ||
| 2030 | 2047 | reg = ast_get_index_reg_mask(ast, AST_IO_VGACRI, 0xd0, 0xff); |
| 2031 | 2048 | if ((reg & AST_VRAM_INIT_STATUS_MASK) == 0) {/* vga only */ |
| 2032 | 2049 | /* Clear bus lock condition */ |
| 2033 | − ast_patch_ahb_2500(ast); | |
| 2050 | + ast_patch_ahb_2500(ast->regs); | |
| 2034 | 2051 | |
| 2035 | 2052 | /* Disable watchdog */ |
| 2036 | 2053 | ast_moutdwm(ast, 0x1E78502C, 0x00000000); |
| @@ -10,10 +10,17 @@ · 9 unmodified lines | ||
| 10 | 10 | */ |
| 11 | 11 | |
| 12 | 12 | #define AST_IO_MM_OFFSET (0x380) |
| 13 | +#define AST_IO_MM_LENGTH (128) | |
| 13 | 14 | |
| 14 | 15 | #define AST_IO_VGAARI_W (0x40) |
| 16 | + | |
| 15 | 17 | #define AST_IO_VGAMR_W (0x42) |
| 18 | +#define AST_IO_VGAMR_R (0x4c) | |
| 19 | +#define AST_IO_VGAMR_IOSEL BIT(0) | |
| 20 | + | |
| 16 | 21 | #define AST_IO_VGAER (0x43) |
| 22 | +#define AST_IO_VGAER_VGA_ENABLE BIT(0) | |
| 23 | + | |
| 17 | 24 | #define AST_IO_VGASRI (0x44) |
| 18 | 25 | #define AST_IO_VGADRR (0x47) |
| 19 | 26 | #define AST_IO_VGADWR (0x48) |
| @@ -21,14 +28,15 @@ · 1 unmodified line | ||
| 21 | 28 | #define AST_IO_VGAGRI (0x4E) |
| 22 | 29 | |
| 23 | 30 | #define AST_IO_VGACRI (0x54) |
| 31 | +#define AST_IO_VGACR80_PASSWORD (0xa8) | |
| 32 | +#define AST_IO_VGACRA1_VGAIO_DISABLED BIT(1) | |
| 33 | +#define AST_IO_VGACRA1_MMIO_ENABLED BIT(2) | |
| 24 | 34 | #define AST_IO_VGACRCB_HWC_16BPP BIT(0) /* set: ARGB4444, cleared: 2bpp palette */ |
| 25 | 35 | #define AST_IO_VGACRCB_HWC_ENABLED BIT(1) |
| 26 | 36 | |
| 27 | 37 | #define AST_IO_VGAIR1_R (0x5A) |
| 28 | 38 | #define AST_IO_VGAIR1_VREFRESH BIT(3) |
| 29 | 39 | |
| 30 | −#define AST_IO_VGAMR_R (0x4C) | |
| 31 | − | |
| 32 | 40 | /* |
| 33 | 41 | * Display Transmitter Type |
| 34 | 42 | */ |
| @@ -12,6 +12,23 @@ config DRM_PANEL_BRIDGE · 11 unmodified lines | ||
| 12 | 12 | help |
| 13 | 13 | DRM bridge wrapper of DRM panels |
| 14 | 14 | |
| 15 | +config DRM_AUX_BRIDGE | |
| 16 | + tristate | |
| 17 | + depends on DRM_BRIDGE && OF | |
| 18 | + select AUXILIARY_BUS | |
| 19 | + select DRM_PANEL_BRIDGE | |
| 20 | + help | |
| 21 | + Simple transparent bridge that is used by several non-DRM drivers to | |
| 22 | + build bridges chain. | |
| 23 | + | |
| 24 | +config DRM_AUX_HPD_BRIDGE | |
| 25 | + tristate | |
| 26 | + depends on DRM_BRIDGE && OF | |
| 27 | + select AUXILIARY_BUS | |
| 28 | + help | |
| 29 | + Simple bridge that terminates the bridge chain and provides HPD | |
| 30 | + support. | |
| 31 | + | |
| 15 | 32 | menu "Display Interface Bridges" |
| 16 | 33 | depends on DRM && DRM_BRIDGE |
| 17 | 34 | |
| @@ -1,4 +1,6 @@ | ||
| 1 | 1 | # SPDX-License-Identifier: GPL-2.0 |
| 2 | +obj-$(CONFIG_DRM_AUX_BRIDGE) += aux-bridge.o | |
| 3 | +obj-$(CONFIG_DRM_AUX_HPD_BRIDGE) += aux-hpd-bridge.o | |
| 2 | 4 | obj-$(CONFIG_DRM_CHIPONE_ICN6211) += chipone-icn6211.o |
| 3 | 5 | obj-$(CONFIG_DRM_CHRONTEL_CH7033) += chrontel-ch7033.o |
| 4 | 6 | obj-$(CONFIG_DRM_CROS_EC_ANX7688) += cros-ec-anx7688.o |
| @@ -1298,10 +1298,32 @@ static void anx7625_config(struct anx7625_data *ctx) · 1297 unmodified lines | ||
| 1298 | 1298 | XTAL_FRQ_SEL, XTAL_FRQ_27M); |
| 1299 | 1299 | } |
| 1300 | 1300 | |
| 1301 | +static int anx7625_hpd_timer_config(struct anx7625_data *ctx) | |
| 1302 | +{ | |
| 1303 | + int ret; | |
| 1304 | + | |
| 1305 | + /* Set irq detect window to 2ms */ | |
| 1306 | + ret = anx7625_reg_write(ctx, ctx->i2c.tx_p2_client, | |
| 1307 | + HPD_DET_TIMER_BIT0_7, HPD_TIME & 0xFF); | |
| 1308 | + ret |= anx7625_reg_write(ctx, ctx->i2c.tx_p2_client, | |
| 1309 | + HPD_DET_TIMER_BIT8_15, | |
| 1310 | + (HPD_TIME >> 8) & 0xFF); | |
| 1311 | + ret |= anx7625_reg_write(ctx, ctx->i2c.tx_p2_client, | |
| 1312 | + HPD_DET_TIMER_BIT16_23, | |
| 1313 | + (HPD_TIME >> 16) & 0xFF); | |
| 1314 | + | |
| 1315 | + return ret; | |
| 1316 | +} | |
| 1317 | + | |
| 1318 | +static int anx7625_read_hpd_gpio_config_status(struct anx7625_data *ctx) | |
| 1319 | +{ | |
| 1320 | + return anx7625_reg_read(ctx, ctx->i2c.rx_p0_client, GPIO_CTRL_2); | |
| 1321 | +} | |
| 1322 | + | |
| 1301 | 1323 | static void anx7625_disable_pd_protocol(struct anx7625_data *ctx) |
| 1302 | 1324 | { |
| 1303 | 1325 | struct device *dev = ctx->dev; |
| 1304 | − int ret; | |
| 1326 | + int ret, val; | |
| 1305 | 1327 | |
| 1306 | 1328 | /* Reset main ocm */ |
| 1307 | 1329 | ret = anx7625_reg_write(ctx, ctx->i2c.rx_p0_client, 0x88, 0x40); |
| @@ -1315,6 +1337,19 @@ static void anx7625_disable_pd_protocol(struct anx7625_data *ctx) · 7 unmodified lines | ||
| 1315 | 1337 | DRM_DEV_DEBUG_DRIVER(dev, "disable PD feature fail.\n"); |
| 1316 | 1338 | else |
| 1317 | 1339 | DRM_DEV_DEBUG_DRIVER(dev, "disable PD feature succeeded.\n"); |
| 1340 | + | |
| 1341 | + /* | |
| 1342 | + * Make sure the HPD GPIO already be configured after OCM release before | |
| 1343 | + * setting HPD detect window register. Here we poll the status register | |
| 1344 | + * at maximum 40ms, then config HPD irq detect window register | |
| 1345 | + */ | |
| 1346 | + readx_poll_timeout(anx7625_read_hpd_gpio_config_status, | |
| 1347 | + ctx, val, | |
| 1348 | + ((val & HPD_SOURCE) || (val < 0)), | |
| 1349 | + 2000, 2000 * 20); | |
| 1350 | + | |
| 1351 | + /* Set HPD irq detect window to 2ms */ | |
| 1352 | + anx7625_hpd_timer_config(ctx); | |
| 1318 | 1353 | } |
| 1319 | 1354 | |
| 1320 | 1355 | static int anx7625_ocm_loading_check(struct anx7625_data *ctx) |
| @@ -1437,20 +1472,6 @@ static void anx7625_start_dp_work(struct anx7625_data *ctx) · 116 unmodified lines | ||
| 1437 | 1472 | |
| 1438 | 1473 | static int anx7625_read_hpd_status_p0(struct anx7625_data *ctx) |
| 1439 | 1474 | { |
| 1440 | − int ret; | |
| 1441 | − | |
| 1442 | − /* Set irq detect window to 2ms */ | |
| 1443 | − ret = anx7625_reg_write(ctx, ctx->i2c.tx_p2_client, | |
| 1444 | − HPD_DET_TIMER_BIT0_7, HPD_TIME & 0xFF); | |
| 1445 | − ret |= anx7625_reg_write(ctx, ctx->i2c.tx_p2_client, | |
| 1446 | − HPD_DET_TIMER_BIT8_15, | |
| 1447 | − (HPD_TIME >> 8) & 0xFF); | |
| 1448 | − ret |= anx7625_reg_write(ctx, ctx->i2c.tx_p2_client, | |
| 1449 | − HPD_DET_TIMER_BIT16_23, | |
| 1450 | − (HPD_TIME >> 16) & 0xFF); | |
| 1451 | − if (ret < 0) | |
| 1452 | − return ret; | |
| 1453 | − | |
| 1454 | 1475 | return anx7625_reg_read(ctx, ctx->i2c.rx_p0_client, SYSTEM_STSTUS); |
| 1455 | 1476 | } |
| 1456 | 1477 | |
| @@ -1464,9 +1485,6 @@ static int _anx7625_hpd_polling(struct anx7625_data *ctx, · 7 unmodified lines | ||
| 1464 | 1485 | if (ctx->pdata.intp_irq) |
| 1465 | 1486 | return 0; |
| 1466 | 1487 | |
| 1467 | − /* Delay 200ms for FW HPD de-bounce */ | |
| 1468 | − msleep(200); | |
| 1469 | − | |
| 1470 | 1488 | ret = readx_poll_timeout(anx7625_read_hpd_status_p0, |
| 1471 | 1489 | ctx, val, |
| 1472 | 1490 | ((val & HPD_STATUS) || (val < 0)), |
| @@ -259,6 +259,10 @@ · 258 unmodified lines | ||
| 259 | 259 | #define AP_MIPI_RX_EN BIT(5) /* 1: MIPI RX input in 0: no RX in */ |
| 260 | 260 | #define AP_DISABLE_PD BIT(6) |
| 261 | 261 | #define AP_DISABLE_DISPLAY BIT(7) |
| 262 | + | |
| 263 | +#define GPIO_CTRL_2 0x49 | |
| 264 | +#define HPD_SOURCE BIT(6) | |
| 265 | + | |
| 262 | 266 | /***************************************************************/ |
| 263 | 267 | /* Register definition of device address 0x84 */ |
| 264 | 268 | #define MIPI_PHY_CONTROL_3 0x03 |
| @@ -0,0 +1,140 @@ | ||
| 1 | +// SPDX-License-Identifier: GPL-2.0+ | |
| 2 | +/* | |
| 3 | + * Copyright (C) 2023 Linaro Ltd. | |
| 4 | + * | |
| 5 | + * Author: Dmitry Baryshkov <dmitry.baryshkov@linaro.org> | |
| 6 | + */ | |
| 7 | +#include <linux/auxiliary_bus.h> | |
| 8 | +#include <linux/module.h> | |
| 9 | + | |
| 10 | +#include <drm/drm_bridge.h> | |
| 11 | +#include <drm/bridge/aux-bridge.h> | |
| 12 | + | |
| 13 | +static DEFINE_IDA(drm_aux_bridge_ida); | |
| 14 | + | |
| 15 | +static void drm_aux_bridge_release(struct device *dev) | |
| 16 | +{ | |
| 17 | + struct auxiliary_device *adev = to_auxiliary_dev(dev); | |
| 18 | + | |
| 19 | + ida_free(&drm_aux_bridge_ida, adev->id); | |
| 20 | + | |
| 21 | + kfree(adev); | |
| 22 | +} | |
| 23 | + | |
| 24 | +static void drm_aux_bridge_unregister_adev(void *_adev) | |
| 25 | +{ | |
| 26 | + struct auxiliary_device *adev = _adev; | |
| 27 | + | |
| 28 | + auxiliary_device_delete(adev); | |
| 29 | + auxiliary_device_uninit(adev); | |
| 30 | +} | |
| 31 | + | |
| 32 | +/** | |
| 33 | + * drm_aux_bridge_register - Create a simple bridge device to link the chain | |
| 34 | + * @parent: device instance providing this bridge | |
| 35 | + * | |
| 36 | + * Creates a simple DRM bridge that doesn't implement any drm_bridge | |
| 37 | + * operations. Such bridges merely fill a place in the bridge chain linking | |
| 38 | + * surrounding DRM bridges. | |
| 39 | + * | |
| 40 | + * Return: zero on success, negative error code on failure | |
| 41 | + */ | |
| 42 | +int drm_aux_bridge_register(struct device *parent) | |
| 43 | +{ | |
| 44 | + struct auxiliary_device *adev; | |
| 45 | + int ret; | |
| 46 | + | |
| 47 | + adev = kzalloc(sizeof(*adev), GFP_KERNEL); | |
| 48 | + if (!adev) | |
| 49 | + return -ENOMEM; | |
| 50 | + | |
| 51 | + ret = ida_alloc(&drm_aux_bridge_ida, GFP_KERNEL); | |
| 52 | + if (ret < 0) { | |
| 53 | + kfree(adev); | |
| 54 | + return ret; | |
| 55 | + } | |
| 56 | + | |
| 57 | + adev->id = ret; | |
| 58 | + adev->name = "aux_bridge"; | |
| 59 | + adev->dev.parent = parent; | |
| 60 | + adev->dev.of_node = parent->of_node; | |
| 61 | + adev->dev.release = drm_aux_bridge_release; | |
| 62 | + | |
| 63 | + ret = auxiliary_device_init(adev); | |
| 64 | + if (ret) { | |
| 65 | + ida_free(&drm_aux_bridge_ida, adev->id); | |
| 66 | + kfree(adev); | |
| 67 | + return ret; | |
| 68 | + } | |
| 69 | + | |
| 70 | + ret = auxiliary_device_add(adev); | |
| 71 | + if (ret) { | |
| 72 | + auxiliary_device_uninit(adev); | |
| 73 | + return ret; | |
| 74 | + } | |
| 75 | + | |
| 76 | + return devm_add_action_or_reset(parent, drm_aux_bridge_unregister_adev, adev); | |
| 77 | +} | |
| 78 | +EXPORT_SYMBOL_GPL(drm_aux_bridge_register); | |
| 79 | + | |
| 80 | +struct drm_aux_bridge_data { | |
| 81 | + struct drm_bridge bridge; | |
| 82 | + struct drm_bridge *next_bridge; | |
| 83 | + struct device *dev; | |
| 84 | +}; | |
| 85 | + | |
| 86 | +static int drm_aux_bridge_attach(struct drm_bridge *bridge, | |
| 87 | + enum drm_bridge_attach_flags flags) | |
| 88 | +{ | |
| 89 | + struct drm_aux_bridge_data *data; | |
| 90 | + | |
| 91 | + if (!(flags & DRM_BRIDGE_ATTACH_NO_CONNECTOR)) | |
| 92 | + return -EINVAL; | |
| 93 | + | |
| 94 | + data = container_of(bridge, struct drm_aux_bridge_data, bridge); | |
| 95 | + | |
| 96 | + return drm_bridge_attach(bridge->encoder, data->next_bridge, bridge, | |
| 97 | + DRM_BRIDGE_ATTACH_NO_CONNECTOR); | |
| 98 | +} | |
| 99 | + | |
| 100 | +static const struct drm_bridge_funcs drm_aux_bridge_funcs = { | |
| 101 | + .attach = drm_aux_bridge_attach, | |
| 102 | +}; | |
| 103 | + | |
| 104 | +static int drm_aux_bridge_probe(struct auxiliary_device *auxdev, | |
| 105 | + const struct auxiliary_device_id *id) | |
| 106 | +{ | |
| 107 | + struct drm_aux_bridge_data *data; | |
| 108 | + | |
| 109 | + data = devm_kzalloc(&auxdev->dev, sizeof(*data), GFP_KERNEL); | |
| 110 | + if (!data) | |
| 111 | + return -ENOMEM; | |
| 112 | + | |
| 113 | + data->dev = &auxdev->dev; | |
| 114 | + data->next_bridge = devm_drm_of_get_bridge(&auxdev->dev, auxdev->dev.of_node, 0, 0); | |
| 115 | + if (IS_ERR(data->next_bridge)) | |
| 116 | + return dev_err_probe(&auxdev->dev, PTR_ERR(data->next_bridge), | |
| 117 | + "failed to acquire drm_bridge\n"); | |
| 118 | + | |
| 119 | + data->bridge.funcs = &drm_aux_bridge_funcs; | |
| 120 | + data->bridge.of_node = data->dev->of_node; | |
| 121 | + | |
| 122 | + return devm_drm_bridge_add(data->dev, &data->bridge); | |
| 123 | +} | |
| 124 | + | |
| 125 | +static const struct auxiliary_device_id drm_aux_bridge_table[] = { | |
| 126 | + { .name = KBUILD_MODNAME ".aux_bridge" }, | |
| 127 | + {}, | |
| 128 | +}; | |
| 129 | +MODULE_DEVICE_TABLE(auxiliary, drm_aux_bridge_table); | |
| 130 | + | |
| 131 | +static struct auxiliary_driver drm_aux_bridge_drv = { | |
| 132 | + .name = "aux_bridge", | |
| 133 | + .id_table = drm_aux_bridge_table, | |
| 134 | + .probe = drm_aux_bridge_probe, | |
| 135 | +}; | |
| 136 | +module_auxiliary_driver(drm_aux_bridge_drv); | |
| 137 | + | |
| 138 | +MODULE_AUTHOR("Dmitry Baryshkov <dmitry.baryshkov@linaro.org>"); | |
| 139 | +MODULE_DESCRIPTION("DRM transparent bridge"); | |
| 140 | +MODULE_LICENSE("GPL"); | |
| @@ -0,0 +1,163 @@ | ||
| 1 | +// SPDX-License-Identifier: GPL-2.0+ | |
| 2 | +/* | |
| 3 | + * Copyright (C) 2023 Linaro Ltd. | |
| 4 | + * | |
| 5 | + * Author: Dmitry Baryshkov <dmitry.baryshkov@linaro.org> | |
| 6 | + */ | |
| 7 | +#include <linux/auxiliary_bus.h> | |
| 8 | +#include <linux/module.h> | |
| 9 | +#include <linux/of.h> | |
| 10 | + | |
| 11 | +#include <drm/drm_bridge.h> | |
| 12 | +#include <drm/bridge/aux-bridge.h> | |
| 13 | + | |
| 14 | +static DEFINE_IDA(drm_aux_hpd_bridge_ida); | |
| 15 | + | |
| 16 | +struct drm_aux_hpd_bridge_data { | |
| 17 | + struct drm_bridge bridge; | |
| 18 | + struct device *dev; | |
| 19 | +}; | |
| 20 | + | |
| 21 | +static void drm_aux_hpd_bridge_release(struct device *dev) | |
| 22 | +{ | |
| 23 | + struct auxiliary_device *adev = to_auxiliary_dev(dev); | |
| 24 | + | |
| 25 | + ida_free(&drm_aux_hpd_bridge_ida, adev->id); | |
| 26 | + | |
| 27 | + of_node_put(adev->dev.platform_data); | |
| 28 | + | |
| 29 | + kfree(adev); | |
| 30 | +} | |
| 31 | + | |
| 32 | +static void drm_aux_hpd_bridge_unregister_adev(void *_adev) | |
| 33 | +{ | |
| 34 | + struct auxiliary_device *adev = _adev; | |
| 35 | + | |
| 36 | + auxiliary_device_delete(adev); | |
| 37 | + auxiliary_device_uninit(adev); | |
| 38 | +} | |
| 39 | + | |
| 40 | +/** | |
| 41 | + * drm_dp_hpd_bridge_register - Create a simple HPD DisplayPort bridge | |
| 42 | + * @parent: device instance providing this bridge | |
| 43 | + * @np: device node pointer corresponding to this bridge instance | |
| 44 | + * | |
| 45 | + * Creates a simple DRM bridge with the type set to | |
| 46 | + * DRM_MODE_CONNECTOR_DisplayPort, which terminates the bridge chain and is | |
| 47 | + * able to send the HPD events. | |
| 48 | + * | |
| 49 | + * Return: device instance that will handle created bridge or an error code | |
| 50 | + * encoded into the pointer. | |
| 51 | + */ | |
| 52 | +struct device *drm_dp_hpd_bridge_register(struct device *parent, | |
| 53 | + struct device_node *np) | |
| 54 | +{ | |
| 55 | + struct auxiliary_device *adev; | |
| 56 | + int ret; | |
| 57 | + | |
| 58 | + adev = kzalloc(sizeof(*adev), GFP_KERNEL); | |
| 59 | + if (!adev) | |
| 60 | + return ERR_PTR(-ENOMEM); | |
| 61 | + | |
| 62 | + ret = ida_alloc(&drm_aux_hpd_bridge_ida, GFP_KERNEL); | |
| 63 | + if (ret < 0) { | |
| 64 | + kfree(adev); | |
| 65 | + return ERR_PTR(ret); | |
| 66 | + } | |
| 67 | + | |
| 68 | + adev->id = ret; | |
| 69 | + adev->name = "dp_hpd_bridge"; | |
| 70 | + adev->dev.parent = parent; | |
| 71 | + adev->dev.of_node = parent->of_node; | |
| 72 | + adev->dev.release = drm_aux_hpd_bridge_release; | |
| 73 | + adev->dev.platform_data = np; | |
| 74 | + | |
| 75 | + ret = auxiliary_device_init(adev); | |
| 76 | + if (ret) { | |
| 77 | + ida_free(&drm_aux_hpd_bridge_ida, adev->id); | |
| 78 | + kfree(adev); | |
| 79 | + return ERR_PTR(ret); | |
| 80 | + } | |
| 81 | + | |
| 82 | + ret = auxiliary_device_add(adev); | |
| 83 | + if (ret) { | |
| 84 | + auxiliary_device_uninit(adev); | |
| 85 | + return ERR_PTR(ret); | |
| 86 | + } | |
| 87 | + | |
| 88 | + ret = devm_add_action_or_reset(parent, drm_aux_hpd_bridge_unregister_adev, adev); | |
| 89 | + if (ret) | |
| 90 | + return ERR_PTR(ret); | |
| 91 | + | |
| 92 | + return &adev->dev; | |
| 93 | +} | |
| 94 | +EXPORT_SYMBOL_GPL(drm_dp_hpd_bridge_register); | |
| 95 | + | |
| 96 | +/** | |
| 97 | + * drm_aux_hpd_bridge_notify - notify hot plug detection events | |
| 98 | + * @dev: device created for the HPD bridge | |
| 99 | + * @status: output connection status | |
| 100 | + * | |
| 101 | + * A wrapper around drm_bridge_hpd_notify() that is used to report hot plug | |
| 102 | + * detection events for bridges created via drm_dp_hpd_bridge_register(). | |
| 103 | + * | |
| 104 | + * This function shall be called in a context that can sleep. | |
| 105 | + */ | |
| 106 | +void drm_aux_hpd_bridge_notify(struct device *dev, enum drm_connector_status status) | |
| 107 | +{ | |
| 108 | + struct auxiliary_device *adev = to_auxiliary_dev(dev); | |
| 109 | + struct drm_aux_hpd_bridge_data *data = auxiliary_get_drvdata(adev); | |
| 110 | + | |
| 111 | + if (!data) | |
| 112 | + return; | |
| 113 | + | |
| 114 | + drm_bridge_hpd_notify(&data->bridge, status); | |
| 115 | +} | |
| 116 | +EXPORT_SYMBOL_GPL(drm_aux_hpd_bridge_notify); | |
| 117 | + | |
| 118 | +static int drm_aux_hpd_bridge_attach(struct drm_bridge *bridge, | |
| 119 | + enum drm_bridge_attach_flags flags) | |
| 120 | +{ | |
| 121 | + return flags & DRM_BRIDGE_ATTACH_NO_CONNECTOR ? 0 : -EINVAL; | |
| 122 | +} | |
| 123 | + | |
| 124 | +static const struct drm_bridge_funcs drm_aux_hpd_bridge_funcs = { | |
| 125 | + .attach = drm_aux_hpd_bridge_attach, | |
| 126 | +}; | |
| 127 | + | |
| 128 | +static int drm_aux_hpd_bridge_probe(struct auxiliary_device *auxdev, | |
| 129 | + const struct auxiliary_device_id *id) | |
| 130 | +{ | |
| 131 | + struct drm_aux_hpd_bridge_data *data; | |
| 132 | + | |
| 133 | + data = devm_kzalloc(&auxdev->dev, sizeof(*data), GFP_KERNEL); | |
| 134 | + if (!data) | |
| 135 | + return -ENOMEM; | |
| 136 | + | |
| 137 | + data->dev = &auxdev->dev; | |
| 138 | + data->bridge.funcs = &drm_aux_hpd_bridge_funcs; | |
| 139 | + data->bridge.of_node = dev_get_platdata(data->dev); | |
| 140 | + data->bridge.ops = DRM_BRIDGE_OP_HPD; | |
| 141 | + data->bridge.type = id->driver_data; | |
| 142 | + | |
| 143 | + auxiliary_set_drvdata(auxdev, data); | |
| 144 | + | |
| 145 | + return devm_drm_bridge_add(data->dev, &data->bridge); | |
| 146 | +} | |
| 147 | + | |
| 148 | +static const struct auxiliary_device_id drm_aux_hpd_bridge_table[] = { | |
| 149 | + { .name = KBUILD_MODNAME ".dp_hpd_bridge", .driver_data = DRM_MODE_CONNECTOR_DisplayPort, }, | |
| 150 | + {}, | |
| 151 | +}; | |
| 152 | +MODULE_DEVICE_TABLE(auxiliary, drm_aux_hpd_bridge_table); | |
| 153 | + | |
| 154 | +static struct auxiliary_driver drm_aux_hpd_bridge_drv = { | |
| 155 | + .name = "aux_hpd_bridge", | |
| 156 | + .id_table = drm_aux_hpd_bridge_table, | |
| 157 | + .probe = drm_aux_hpd_bridge_probe, | |
| 158 | +}; | |
| 159 | +module_auxiliary_driver(drm_aux_hpd_bridge_drv); | |
| 160 | + | |
| 161 | +MODULE_AUTHOR("Dmitry Baryshkov <dmitry.baryshkov@linaro.org>"); | |
| 162 | +MODULE_DESCRIPTION("DRM HPD bridge"); | |
| 163 | +MODULE_LICENSE("GPL"); | |
| @@ -2596,11 +2596,10 @@ clk_disable: · 2595 unmodified lines | ||
| 2596 | 2596 | return ret; |
| 2597 | 2597 | } |
| 2598 | 2598 | |
| 2599 | −static int cdns_mhdp_remove(struct platform_device *pdev) | |
| 2599 | +static void cdns_mhdp_remove(struct platform_device *pdev) | |
| 2600 | 2600 | { |
| 2601 | 2601 | struct cdns_mhdp_device *mhdp = platform_get_drvdata(pdev); |
| 2602 | 2602 | unsigned long timeout = msecs_to_jiffies(100); |
| 2603 | − bool stop_fw = false; | |
| 2604 | 2603 | int ret; |
| 2605 | 2604 | |
| 2606 | 2605 | drm_bridge_remove(&mhdp->bridge); |
| @@ -2608,18 +2607,19 @@ static int cdns_mhdp_remove(struct platform_device *pdev) · 1 unmodified line | ||
| 2608 | 2607 | ret = wait_event_timeout(mhdp->fw_load_wq, |
| 2609 | 2608 | mhdp->hw_state == MHDP_HW_READY, |
| 2610 | 2609 | timeout); |
| 2611 | − if (ret == 0) | |
| 2612 | − dev_err(mhdp->dev, "%s: Timeout waiting for fw loading\n", | |
| 2613 | − __func__); | |
| 2614 | − else | |
| 2615 | − stop_fw = true; | |
| 2616 | − | |
| 2617 | 2610 | spin_lock(&mhdp->start_lock); |
| 2618 | 2611 | mhdp->hw_state = MHDP_HW_STOPPED; |
| 2619 | 2612 | spin_unlock(&mhdp->start_lock); |
| 2620 | 2613 | |
| 2621 | − if (stop_fw) | |
| 2614 | + if (ret == 0) { | |
| 2615 | + dev_err(mhdp->dev, "%s: Timeout waiting for fw loading\n", | |
| 2616 | + __func__); | |
| 2617 | + } else { | |
| 2622 | 2618 | ret = cdns_mhdp_set_firmware_active(mhdp, false); |
| 2619 | + if (ret) | |
| 2620 | + dev_err(mhdp->dev, "Failed to stop firmware (%pe)\n", | |
| 2621 | + ERR_PTR(ret)); | |
| 2622 | + } | |
| 2623 | 2623 | |
| 2624 | 2624 | phy_exit(mhdp->phy); |
| 2625 | 2625 | |
| @@ -2634,8 +2634,6 @@ static int cdns_mhdp_remove(struct platform_device *pdev) · 8 unmodified lines | ||
| 2634 | 2634 | /* Ignoring mhdp->hdcp.check_work and mhdp->hdcp.prop_work here. */ |
| 2635 | 2635 | |
| 2636 | 2636 | clk_disable_unprepare(mhdp->clk); |
| 2637 | − | |
| 2638 | − return ret; | |
| 2639 | 2637 | } |
| 2640 | 2638 | |
| 2641 | 2639 | static const struct of_device_id mhdp_ids[] = { |
| @@ -2658,7 +2656,7 @@ static struct platform_driver mhdp_driver = { · 16 unmodified lines | ||
| 2658 | 2656 | .of_match_table = mhdp_ids, |
| 2659 | 2657 | }, |
| 2660 | 2658 | .probe = cdns_mhdp_probe, |
| 2661 | − .remove = cdns_mhdp_remove, | |
| 2659 | + .remove_new = cdns_mhdp_remove, | |
| 2662 | 2660 | }; |
| 2663 | 2661 | module_platform_driver(mhdp_driver); |
| 2664 | 2662 | |
| @@ -403,7 +403,8 @@ static int _cdns_mhdp_hdcp_disable(struct cdns_mhdp_device *mhdp) · 402 unmodified lines | ||
| 403 | 403 | |
| 404 | 404 | static int _cdns_mhdp_hdcp_enable(struct cdns_mhdp_device *mhdp, u8 content_type) |
| 405 | 405 | { |
| 406 | − int ret, tries = 3; | |
| 406 | + int ret = -EINVAL; | |
| 407 | + int tries = 3; | |
| 407 | 408 | u32 i; |
| 408 | 409 | |
| 409 | 410 | for (i = 0; i < tries; i++) { |
| @@ -226,8 +226,8 @@ dphy_pll_get_configure_from_opts(struct imx93_dsi *dsi, · 225 unmodified lines | ||
| 226 | 226 | unsigned long fout; |
| 227 | 227 | unsigned long best_fout = 0; |
| 228 | 228 | unsigned int fvco_div; |
| 229 | − unsigned int min_n, max_n, n, best_n; | |
| 230 | − unsigned long m, best_m; | |
| 229 | + unsigned int min_n, max_n, n, best_n = UINT_MAX; | |
| 230 | + unsigned long m, best_m = 0; | |
| 231 | 231 | unsigned long min_delta = ULONG_MAX; |
| 232 | 232 | unsigned long delta; |
| 233 | 233 | u64 tmp; |
| @@ -43,6 +43,8 @@ struct lt8912 { · 42 unmodified lines | ||
| 43 | 43 | |
| 44 | 44 | struct videomode mode; |
| 45 | 45 | |
| 46 | + struct regulator_bulk_data supplies[7]; | |
| 47 | + | |
| 46 | 48 | u8 data_lanes; |
| 47 | 49 | bool is_power_on; |
| 48 | 50 | }; |
| @@ -257,6 +259,12 @@ static int lt8912_free_i2c(struct lt8912 *lt) · 208 unmodified lines | ||
| 257 | 259 | |
| 258 | 260 | static int lt8912_hard_power_on(struct lt8912 *lt) |
| 259 | 261 | { |
| 262 | + int ret; | |
| 263 | + | |
| 264 | + ret = regulator_bulk_enable(ARRAY_SIZE(lt->supplies), lt->supplies); | |
| 265 | + if (ret) | |
| 266 | + return ret; | |
| 267 | + | |
| 260 | 268 | gpiod_set_value_cansleep(lt->gp_reset, 0); |
| 261 | 269 | msleep(20); |
| 262 | 270 | |
| @@ -267,6 +275,9 @@ static void lt8912_hard_power_off(struct lt8912 *lt) · 4 unmodified lines | ||
| 267 | 275 | { |
| 268 | 276 | gpiod_set_value_cansleep(lt->gp_reset, 1); |
| 269 | 277 | msleep(20); |
| 278 | + | |
| 279 | + regulator_bulk_disable(ARRAY_SIZE(lt->supplies), lt->supplies); | |
| 280 | + | |
| 270 | 281 | lt->is_power_on = false; |
| 271 | 282 | } |
| 272 | 283 | |
| @@ -634,6 +645,48 @@ static const struct drm_bridge_funcs lt8912_bridge_funcs = { · 361 unmodified lines | ||
| 634 | 645 | .get_edid = lt8912_bridge_get_edid, |
| 635 | 646 | }; |
| 636 | 647 | |
| 648 | +static int lt8912_bridge_resume(struct device *dev) | |
| 649 | +{ | |
| 650 | + struct lt8912 *lt = dev_get_drvdata(dev); | |
| 651 | + int ret; | |
| 652 | + | |
| 653 | + ret = lt8912_hard_power_on(lt); | |
| 654 | + if (ret) | |
| 655 | + return ret; | |
| 656 | + | |
| 657 | + ret = lt8912_soft_power_on(lt); | |
| 658 | + if (ret) | |
| 659 | + return ret; | |
| 660 | + | |
| 661 | + return lt8912_video_on(lt); | |
| 662 | +} | |
| 663 | + | |
| 664 | +static int lt8912_bridge_suspend(struct device *dev) | |
| 665 | +{ | |
| 666 | + struct lt8912 *lt = dev_get_drvdata(dev); | |
| 667 | + | |
| 668 | + lt8912_hard_power_off(lt); | |
| 669 | + | |
| 670 | + return 0; | |
| 671 | +} | |
| 672 | + | |
| 673 | +static DEFINE_SIMPLE_DEV_PM_OPS(lt8912_bridge_pm_ops, lt8912_bridge_suspend, lt8912_bridge_resume); | |
| 674 | + | |
| 675 | +static int lt8912_get_regulators(struct lt8912 *lt) | |
| 676 | +{ | |
| 677 | + unsigned int i; | |
| 678 | + const char * const supply_names[] = { | |
| 679 | + "vdd", "vccmipirx", "vccsysclk", "vcclvdstx", | |
| 680 | + "vcchdmitx", "vcclvdspll", "vcchdmipll" | |
| 681 | + }; | |
| 682 | + | |
| 683 | + for (i = 0; i < ARRAY_SIZE(lt->supplies); i++) | |
| 684 | + lt->supplies[i].supply = supply_names[i]; | |
| 685 | + | |
| 686 | + return devm_regulator_bulk_get(lt->dev, ARRAY_SIZE(lt->supplies), | |
| 687 | + lt->supplies); | |
| 688 | +} | |
| 689 | + | |
| 637 | 690 | static int lt8912_parse_dt(struct lt8912 *lt) |
| 638 | 691 | { |
| 639 | 692 | struct gpio_desc *gp_reset; |
| @@ -685,6 +738,10 @@ static int lt8912_parse_dt(struct lt8912 *lt) · 45 unmodified lines | ||
| 685 | 738 | goto err_free_host_node; |
| 686 | 739 | } |
| 687 | 740 | |
| 741 | + ret = lt8912_get_regulators(lt); | |
| 742 | + if (ret) | |
| 743 | + goto err_free_host_node; | |
| 744 | + | |
| 688 | 745 | of_node_put(port_node); |
| 689 | 746 | return 0; |
| 690 | 747 | |
| @@ -770,6 +827,7 @@ static struct i2c_driver lt8912_i2c_driver = { · 79 unmodified lines | ||
| 770 | 827 | .driver = { |
| 771 | 828 | .name = "lt8912", |
| 772 | 829 | .of_match_table = lt8912_dt_match, |
| 830 | + .pm = pm_sleep_ptr(<8912_bridge_pm_ops), | |
| 773 | 831 | }, |
| 774 | 832 | .probe = lt8912_probe, |
| 775 | 833 | .remove = lt8912_remove, |
| @@ -54,13 +54,13 @@ static int ptn3460_read_bytes(struct ptn3460_bridge *ptn_bridge, char addr, · 53 unmodified lines | ||
| 54 | 54 | int ret; |
| 55 | 55 | |
| 56 | 56 | ret = i2c_master_send(ptn_bridge->client, &addr, 1); |
| 57 | − if (ret <= 0) { | |
| 57 | + if (ret < 0) { | |
| 58 | 58 | DRM_ERROR("Failed to send i2c command, ret=%d\n", ret); |
| 59 | 59 | return ret; |
| 60 | 60 | } |
| 61 | 61 | |
| 62 | 62 | ret = i2c_master_recv(ptn_bridge->client, buf, len); |
| 63 | − if (ret <= 0) { | |
| 63 | + if (ret < 0) { | |
| 64 | 64 | DRM_ERROR("Failed to recv i2c data, ret=%d\n", ret); |
| 65 | 65 | return ret; |
| 66 | 66 | } |
| @@ -78,7 +78,7 @@ static int ptn3460_write_byte(struct ptn3460_bridge *ptn_bridge, char addr, · 11 unmodified lines | ||
| 78 | 78 | buf[1] = val; |
| 79 | 79 | |
| 80 | 80 | ret = i2c_master_send(ptn_bridge->client, buf, ARRAY_SIZE(buf)); |
| 81 | − if (ret <= 0) { | |
| 81 | + if (ret < 0) { | |
| 82 | 82 | DRM_ERROR("Failed to send i2c command, ret=%d\n", ret); |
| 83 | 83 | return ret; |
| 84 | 84 | } |
| @@ -2273,7 +2273,7 @@ static int tc_probe(struct i2c_client *client) · 2272 unmodified lines | ||
| 2273 | 2273 | } else { |
| 2274 | 2274 | if (tc->hpd_pin < 0 || tc->hpd_pin > 1) { |
| 2275 | 2275 | dev_err(dev, "failed to parse HPD number\n"); |
| 2276 | − return ret; | |
| 2276 | + return -EINVAL; | |
| 2277 | 2277 | } |
| 2278 | 2278 | } |
| 2279 | 2279 | |
| @@ -1413,11 +1413,9 @@ static int ti_sn_pwm_apply(struct pwm_chip *chip, struct pwm_device *pwm, · 1412 unmodified lines | ||
| 1413 | 1413 | int ret; |
| 1414 | 1414 | |
| 1415 | 1415 | if (!pdata->pwm_enabled) { |
| 1416 | − ret = pm_runtime_get_sync(pdata->dev); | |
| 1417 | − if (ret < 0) { | |
| 1418 | − pm_runtime_put_sync(pdata->dev); | |
| 1416 | + ret = pm_runtime_resume_and_get(pdata->dev); | |
| 1417 | + if (ret < 0) | |
| 1419 | 1418 | return ret; |
| 1420 | − } | |
| 1421 | 1419 | } |
| 1422 | 1420 | |
| 1423 | 1421 | if (state->enabled) { |
| @@ -179,13 +179,11 @@ static int tpd12s015_probe(struct platform_device *pdev) · 178 unmodified lines | ||
| 179 | 179 | return 0; |
| 180 | 180 | } |
| 181 | 181 | |
| 182 | −static int __exit tpd12s015_remove(struct platform_device *pdev) | |
| 182 | +static void tpd12s015_remove(struct platform_device *pdev) | |
| 183 | 183 | { |
| 184 | 184 | struct tpd12s015_device *tpd = platform_get_drvdata(pdev); |
| 185 | 185 | |
| 186 | 186 | drm_bridge_remove(&tpd->bridge); |
| 187 | − | |
| 188 | − return 0; | |
| 189 | 187 | } |
| 190 | 188 | |
| 191 | 189 | static const struct of_device_id tpd12s015_of_match[] = { |
| @@ -197,7 +195,7 @@ MODULE_DEVICE_TABLE(of, tpd12s015_of_match); · 5 unmodified lines | ||
| 197 | 195 | |
| 198 | 196 | static struct platform_driver tpd12s015_driver = { |
| 199 | 197 | .probe = tpd12s015_probe, |
| 200 | − .remove = __exit_p(tpd12s015_remove), | |
| 198 | + .remove_new = tpd12s015_remove, | |
| 201 | 199 | .driver = { |
| 202 | 200 | .name = "tpd12s015", |
| 203 | 201 | .of_match_table = tpd12s015_of_match, |
| @@ -5,7 +5,7 @@ termcolor==2.3.0 · 4 unmodified lines | ||
| 5 | 5 | certifi==2023.7.22 |
| 6 | 6 | charset-normalizer==3.2.0 |
| 7 | 7 | idna==3.4 |
| 8 | −pip==23.2.1 | |
| 8 | +pip==23.3 | |
| 9 | 9 | python-gitlab==3.15.0 |
| 10 | 10 | requests==2.31.0 |
| 11 | 11 | requests-toolbelt==1.0.0 |
| @@ -13,5 +13,5 @@ ruamel.yaml==0.17.32 · 1 unmodified line | ||
| 13 | 13 | ruamel.yaml.clib==0.2.7 |
| 14 | 14 | setuptools==68.0.0 |
| 15 | 15 | tenacity==8.2.3 |
| 16 | −urllib3==2.0.4 | |
| 17 | −wheel==0.41.1 | |
| 16 | +urllib3==2.0.7 | |
| 17 | +wheel==0.41.1 | |
| @@ -2245,6 +2245,8 @@ static const struct dpcd_quirk dpcd_quirk_list[] = { · 2244 unmodified lines | ||
| 2245 | 2245 | { OUI(0x00, 0x00, 0x00), DEVICE_ID('C', 'H', '7', '5', '1', '1'), false, BIT(DP_DPCD_QUIRK_NO_SINK_COUNT) }, |
| 2246 | 2246 | /* Synaptics DP1.4 MST hubs can support DSC without virtual DPCD */ |
| 2247 | 2247 | { OUI(0x90, 0xCC, 0x24), DEVICE_ID_ANY, true, BIT(DP_DPCD_QUIRK_DSC_WITHOUT_VIRTUAL_DPCD) }, |
| 2248 | + /* Synaptics DP1.4 MST hubs require DSC for some modes on which it applies HBLANK expansion. */ | |
| 2249 | + { OUI(0x90, 0xCC, 0x24), DEVICE_ID_ANY, true, BIT(DP_DPCD_QUIRK_HBLANK_EXPANSION_REQUIRES_DSC) }, | |
| 2248 | 2250 | /* Apple MacBookPro 2017 15 inch eDP Retina panel reports too low DP_MAX_LINK_RATE */ |
| 2249 | 2251 | { OUI(0x00, 0x10, 0xfa), DEVICE_ID(101, 68, 21, 101, 98, 97), false, BIT(DP_DPCD_QUIRK_CAN_DO_MAX_LINK_RATE_3_24_GBPS) }, |
| 2250 | 2252 | }; |
| @@ -2326,6 +2328,33 @@ int drm_dp_read_desc(struct drm_dp_aux *aux, struct drm_dp_desc *desc, · 75 unmodified lines | ||
| 2326 | 2328 | } |
| 2327 | 2329 | EXPORT_SYMBOL(drm_dp_read_desc); |
| 2328 | 2330 | |
| 2331 | +/** | |
| 2332 | + * drm_dp_dsc_sink_bpp_incr() - Get bits per pixel increment | |
| 2333 | + * @dsc_dpcd: DSC capabilities from DPCD | |
| 2334 | + * | |
| 2335 | + * Returns the bpp precision supported by the DP sink. | |
| 2336 | + */ | |
| 2337 | +u8 drm_dp_dsc_sink_bpp_incr(const u8 dsc_dpcd[DP_DSC_RECEIVER_CAP_SIZE]) | |
| 2338 | +{ | |
| 2339 | + u8 bpp_increment_dpcd = dsc_dpcd[DP_DSC_BITS_PER_PIXEL_INC - DP_DSC_SUPPORT]; | |
| 2340 | + | |
| 2341 | + switch (bpp_increment_dpcd) { | |
| 2342 | + case DP_DSC_BITS_PER_PIXEL_1_16: | |
| 2343 | + return 16; | |
| 2344 | + case DP_DSC_BITS_PER_PIXEL_1_8: | |
| 2345 | + return 8; | |
| 2346 | + case DP_DSC_BITS_PER_PIXEL_1_4: | |
| 2347 | + return 4; | |
| 2348 | + case DP_DSC_BITS_PER_PIXEL_1_2: | |
| 2349 | + return 2; | |
| 2350 | + case DP_DSC_BITS_PER_PIXEL_1_1: | |
| 2351 | + return 1; | |
| 2352 | + } | |
| 2353 | + | |
| 2354 | + return 0; | |
| 2355 | +} | |
| 2356 | +EXPORT_SYMBOL(drm_dp_dsc_sink_bpp_incr); | |
| 2357 | + | |
| 2329 | 2358 | /** |
| 2330 | 2359 | * drm_dp_dsc_sink_max_slice_count() - Get the max slice count |
| 2331 | 2360 | * supported by the DSC sink. |
| @@ -3898,3 +3927,135 @@ int drm_panel_dp_aux_backlight(struct drm_panel *panel, struct drm_dp_aux *aux) · 1566 unmodified lines | ||
| 3898 | 3927 | EXPORT_SYMBOL(drm_panel_dp_aux_backlight); |
| 3899 | 3928 | |
| 3900 | 3929 | #endif |
| 3930 | + | |
| 3931 | +/* See DP Standard v2.1 2.6.4.4.1.1, 2.8.4.4, 2.8.7 */ | |
| 3932 | +static int drm_dp_link_symbol_cycles(int lane_count, int pixels, int bpp_x16, | |
| 3933 | + int symbol_size, bool is_mst) | |
| 3934 | +{ | |
| 3935 | + int cycles = DIV_ROUND_UP(pixels * bpp_x16, 16 * symbol_size * lane_count); | |
| 3936 | + int align = is_mst ? 4 / lane_count : 1; | |
| 3937 | + | |
| 3938 | + return ALIGN(cycles, align); | |
| 3939 | +} | |
| 3940 | + | |
| 3941 | +static int drm_dp_link_dsc_symbol_cycles(int lane_count, int pixels, int slice_count, | |
| 3942 | + int bpp_x16, int symbol_size, bool is_mst) | |
| 3943 | +{ | |
| 3944 | + int slice_pixels = DIV_ROUND_UP(pixels, slice_count); | |
| 3945 | + int slice_data_cycles = drm_dp_link_symbol_cycles(lane_count, slice_pixels, | |
| 3946 | + bpp_x16, symbol_size, is_mst); | |
| 3947 | + int slice_eoc_cycles = is_mst ? 4 / lane_count : 1; | |
| 3948 | + | |
| 3949 | + return slice_count * (slice_data_cycles + slice_eoc_cycles); | |
| 3950 | +} | |
| 3951 | + | |
| 3952 | +/** | |
| 3953 | + * drm_dp_bw_overhead - Calculate the BW overhead of a DP link stream | |
| 3954 | + * @lane_count: DP link lane count | |
| 3955 | + * @hactive: pixel count of the active period in one scanline of the stream | |
| 3956 | + * @dsc_slice_count: DSC slice count if @flags/DRM_DP_LINK_BW_OVERHEAD_DSC is set | |
| 3957 | + * @bpp_x16: bits per pixel in .4 binary fixed point | |
| 3958 | + * @flags: DRM_DP_OVERHEAD_x flags | |
| 3959 | + * | |
| 3960 | + * Calculate the BW allocation overhead of a DP link stream, depending | |
| 3961 | + * on the link's | |
| 3962 | + * - @lane_count | |
| 3963 | + * - SST/MST mode (@flags / %DRM_DP_OVERHEAD_MST) | |
| 3964 | + * - symbol size (@flags / %DRM_DP_OVERHEAD_UHBR) | |
| 3965 | + * - FEC mode (@flags / %DRM_DP_OVERHEAD_FEC) | |
| 3966 | + * - SSC/REF_CLK mode (@flags / %DRM_DP_OVERHEAD_SSC_REF_CLK) | |
| 3967 | + * as well as the stream's | |
| 3968 | + * - @hactive timing | |
| 3969 | + * - @bpp_x16 color depth | |
| 3970 | + * - compression mode (@flags / %DRM_DP_OVERHEAD_DSC). | |
| 3971 | + * Note that this overhead doesn't account for the 8b/10b, 128b/132b | |
| 3972 | + * channel coding efficiency, for that see | |
| 3973 | + * @drm_dp_link_bw_channel_coding_efficiency(). | |
| 3974 | + * | |
| 3975 | + * Returns the overhead as 100% + overhead% in 1ppm units. | |
| 3976 | + */ | |
| 3977 | +int drm_dp_bw_overhead(int lane_count, int hactive, | |
| 3978 | + int dsc_slice_count, | |
| 3979 | + int bpp_x16, unsigned long flags) | |
| 3980 | +{ | |
| 3981 | + int symbol_size = flags & DRM_DP_BW_OVERHEAD_UHBR ? 32 : 8; | |
| 3982 | + bool is_mst = flags & DRM_DP_BW_OVERHEAD_MST; | |
| 3983 | + u32 overhead = 1000000; | |
| 3984 | + int symbol_cycles; | |
| 3985 | + | |
| 3986 | + /* | |
| 3987 | + * DP Standard v2.1 2.6.4.1 | |
| 3988 | + * SSC downspread and ref clock variation margin: | |
| 3989 | + * 5300ppm + 300ppm ~ 0.6% | |
| 3990 | + */ | |
| 3991 | + if (flags & DRM_DP_BW_OVERHEAD_SSC_REF_CLK) | |
| 3992 | + overhead += 6000; | |
| 3993 | + | |
| 3994 | + /* | |
| 3995 | + * DP Standard v2.1 2.6.4.1.1, 3.5.1.5.4: | |
| 3996 | + * FEC symbol insertions for 8b/10b channel coding: | |
| 3997 | + * After each 250 data symbols on 2-4 lanes: | |
| 3998 | + * 250 LL + 5 FEC_PARITY_PH + 1 CD_ADJ (256 byte FEC block) | |
| 3999 | + * After each 2 x 250 data symbols on 1 lane: | |
| 4000 | + * 2 * 250 LL + 11 FEC_PARITY_PH + 1 CD_ADJ (512 byte FEC block) | |
| 4001 | + * After 256 (2-4 lanes) or 128 (1 lane) FEC blocks: | |
| 4002 | + * 256 * 256 bytes + 1 FEC_PM | |
| 4003 | + * or | |
| 4004 | + * 128 * 512 bytes + 1 FEC_PM | |
| 4005 | + * (256 * 6 + 1) / (256 * 250) = 2.4015625 % | |
| 4006 | + */ | |
| 4007 | + if (flags & DRM_DP_BW_OVERHEAD_FEC) | |
| 4008 | + overhead += 24016; | |
| 4009 | + | |
| 4010 | + /* | |
| 4011 | + * DP Standard v2.1 2.7.9, 5.9.7 | |
| 4012 | + * The FEC overhead for UHBR is accounted for in its 96.71% channel | |
| 4013 | + * coding efficiency. | |
| 4014 | + */ | |
| 4015 | + WARN_ON((flags & DRM_DP_BW_OVERHEAD_UHBR) && | |
| 4016 | + (flags & DRM_DP_BW_OVERHEAD_FEC)); | |
| 4017 | + | |
| 4018 | + if (flags & DRM_DP_BW_OVERHEAD_DSC) | |
| 4019 | + symbol_cycles = drm_dp_link_dsc_symbol_cycles(lane_count, hactive, | |
| 4020 | + dsc_slice_count, | |
| 4021 | + bpp_x16, symbol_size, | |
| 4022 | + is_mst); | |
| 4023 | + else | |
| 4024 | + symbol_cycles = drm_dp_link_symbol_cycles(lane_count, hactive, | |
| 4025 | + bpp_x16, symbol_size, | |
| 4026 | + is_mst); | |
| 4027 | + | |
| 4028 | + return DIV_ROUND_UP_ULL(mul_u32_u32(symbol_cycles * symbol_size * lane_count, | |
| 4029 | + overhead * 16), | |
| 4030 | + hactive * bpp_x16); | |
| 4031 | +} | |
| 4032 | +EXPORT_SYMBOL(drm_dp_bw_overhead); | |
| 4033 | + | |
| 4034 | +/** | |
| 4035 | + * drm_dp_bw_channel_coding_efficiency - Get a DP link's channel coding efficiency | |
| 4036 | + * @is_uhbr: Whether the link has a 128b/132b channel coding | |
| 4037 | + * | |
| 4038 | + * Return the channel coding efficiency of the given DP link type, which is | |
| 4039 | + * either 8b/10b or 128b/132b (aka UHBR). The corresponding overhead includes | |
| 4040 | + * the 8b -> 10b, 128b -> 132b pixel data to link symbol conversion overhead | |
| 4041 | + * and for 128b/132b any link or PHY level control symbol insertion overhead | |
| 4042 | + * (LLCP, FEC, PHY sync, see DP Standard v2.1 3.5.2.18). For 8b/10b the | |
| 4043 | + * corresponding FEC overhead is BW allocation specific, included in the value | |
| 4044 | + * returned by drm_dp_bw_overhead(). | |
| 4045 | + * | |
| 4046 | + * Returns the efficiency in the 100%/coding-overhead% ratio in | |
| 4047 | + * 1ppm units. | |
| 4048 | + */ | |
| 4049 | +int drm_dp_bw_channel_coding_efficiency(bool is_uhbr) | |
| 4050 | +{ | |
| 4051 | + if (is_uhbr) | |
| 4052 | + return 967100; | |
| 4053 | + else | |
| 4054 | + /* | |
| 4055 | + * Note that on 8b/10b MST the efficiency is only | |
| 4056 | + * 78.75% due to the 1 out of 64 MTPH packet overhead, | |
| 4057 | + * not accounted for here. | |
| 4058 | + */ | |
| 4059 | + return 800000; | |
| 4060 | +} | |
| 4061 | +EXPORT_SYMBOL(drm_dp_bw_channel_coding_efficiency); | |
| @@ -43,6 +43,7 @@ · 42 unmodified lines | ||
| 43 | 43 | #include <drm/drm_atomic_helper.h> |
| 44 | 44 | #include <drm/drm_drv.h> |
| 45 | 45 | #include <drm/drm_edid.h> |
| 46 | +#include <drm/drm_fixed.h> | |
| 46 | 47 | #include <drm/drm_print.h> |
| 47 | 48 | #include <drm/drm_probe_helper.h> |
| 48 | 49 | |
| @@ -3578,16 +3579,26 @@ static int drm_dp_send_up_ack_reply(struct drm_dp_mst_topology_mgr *mgr, · 3529 unmodified lines | ||
| 3578 | 3579 | * value is in units of PBNs/(timeslots/1 MTP). This value can be used to |
| 3579 | 3580 | * convert the number of PBNs required for a given stream to the number of |
| 3580 | 3581 | * timeslots this stream requires in each MTP. |
| 3582 | + * | |
| 3583 | + * Returns the BW / timeslot value in 20.12 fixed point format. | |
| 3581 | 3584 | */ |
| 3582 | −int drm_dp_get_vc_payload_bw(const struct drm_dp_mst_topology_mgr *mgr, | |
| 3583 | − int link_rate, int link_lane_count) | |
| 3585 | +fixed20_12 drm_dp_get_vc_payload_bw(const struct drm_dp_mst_topology_mgr *mgr, | |
| 3586 | + int link_rate, int link_lane_count) | |
| 3584 | 3587 | { |
| 3588 | + int ch_coding_efficiency = | |
| 3589 | + drm_dp_bw_channel_coding_efficiency(drm_dp_is_uhbr_rate(link_rate)); | |
| 3590 | + fixed20_12 ret; | |
| 3591 | + | |
| 3585 | 3592 | if (link_rate == 0 || link_lane_count == 0) |
| 3586 | 3593 | drm_dbg_kms(mgr->dev, "invalid link rate/lane count: (%d / %d)\n", |
| 3587 | 3594 | link_rate, link_lane_count); |
| 3588 | 3595 | |
| 3589 | − /* See DP v2.0 2.6.4.2, VCPayload_Bandwidth_for_OneTimeSlotPer_MTP_Allocation */ | |
| 3590 | − return link_rate * link_lane_count / 54000; | |
| 3596 | + /* See DP v2.0 2.6.4.2, 2.7.6.3 VCPayload_Bandwidth_for_OneTimeSlotPer_MTP_Allocation */ | |
| 3597 | + ret.full = DIV_ROUND_DOWN_ULL(mul_u32_u32(link_rate * link_lane_count, | |
| 3598 | + ch_coding_efficiency), | |
| 3599 | + (1000000ULL * 8 * 5400) >> 12); | |
| 3600 | + | |
| 3601 | + return ret; | |
| 3591 | 3602 | } |
| 3592 | 3603 | EXPORT_SYMBOL(drm_dp_get_vc_payload_bw); |
| 3593 | 3604 | |
| @@ -4335,7 +4346,7 @@ int drm_dp_atomic_find_time_slots(struct drm_atomic_state *state, · 741 unmodified lines | ||
| 4335 | 4346 | } |
| 4336 | 4347 | } |
| 4337 | 4348 | |
| 4338 | − req_slots = DIV_ROUND_UP(pbn, topology_state->pbn_div); | |
| 4349 | + req_slots = DIV_ROUND_UP(dfixed_const(pbn), topology_state->pbn_div.full); | |
| 4339 | 4350 | |
| 4340 | 4351 | drm_dbg_atomic(mgr->dev, "[CONNECTOR:%d:%s] [MST PORT:%p] TU %d -> %d\n", |
| 4341 | 4352 | port->connector->base.id, port->connector->name, |
| @@ -4718,35 +4729,36 @@ EXPORT_SYMBOL(drm_dp_check_act_status); · 376 unmodified lines | ||
| 4718 | 4729 | |
| 4719 | 4730 | /** |
| 4720 | 4731 | * drm_dp_calc_pbn_mode() - Calculate the PBN for a mode. |
| 4721 | − * @clock: dot clock for the mode | |
| 4722 | − * @bpp: bpp for the mode. | |
| 4723 | − * @dsc: DSC mode. If true, bpp has units of 1/16 of a bit per pixel | |
| 4732 | + * @clock: dot clock | |
| 4733 | + * @bpp: bpp as .4 binary fixed point | |
| 4724 | 4734 | * |
| 4725 | 4735 | * This uses the formula in the spec to calculate the PBN value for a mode. |
| 4726 | 4736 | */ |
| 4727 | −int drm_dp_calc_pbn_mode(int clock, int bpp, bool dsc) | |
| 4737 | +int drm_dp_calc_pbn_mode(int clock, int bpp) | |
| 4728 | 4738 | { |
| 4729 | 4739 | /* |
| 4730 | − * margin 5300ppm + 300ppm ~ 0.6% as per spec, factor is 1.006 | |
| 4731 | 4740 | * The unit of 54/64Mbytes/sec is an arbitrary unit chosen based on |
| 4732 | 4741 | * common multiplier to render an integer PBN for all link rate/lane |
| 4733 | 4742 | * counts combinations |
| 4734 | 4743 | * calculate |
| 4735 | − * peak_kbps *= (1006/1000) | |
| 4736 | − * peak_kbps *= (64/54) | |
| 4737 | − * peak_kbps *= 8 convert to bytes | |
| 4738 | − * | |
| 4739 | − * If the bpp is in units of 1/16, further divide by 16. Put this | |
| 4740 | − * factor in the numerator rather than the denominator to avoid | |
| 4741 | − * integer overflow | |
| 4744 | + * peak_kbps = clock * bpp / 16 | |
| 4745 | + * peak_kbps *= SSC overhead / 1000000 | |
| 4746 | + * peak_kbps /= 8 convert to Kbytes | |
| 4747 | + * peak_kBps *= (64/54) / 1000 convert to PBN | |
| 4742 | 4748 | */ |
| 4749 | + /* | |
| 4750 | + * TODO: Use the actual link and mode parameters to calculate | |
| 4751 | + * the overhead. For now it's assumed that these are | |
| 4752 | + * 4 link lanes, 4096 hactive pixels, which don't add any | |
| 4753 | + * significant data padding overhead and that there is no DSC | |
| 4754 | + * or FEC overhead. | |
| 4755 | + */ | |
| 4756 | + int overhead = drm_dp_bw_overhead(4, 4096, 0, bpp, | |
| 4757 | + DRM_DP_BW_OVERHEAD_MST | | |
| 4758 | + DRM_DP_BW_OVERHEAD_SSC_REF_CLK); | |
| 4743 | 4759 | |
| 4744 | − if (dsc) | |
| 4745 | − return DIV_ROUND_UP_ULL(mul_u32_u32(clock * (bpp / 16), 64 * 1006), | |
| 4746 | − 8 * 54 * 1000 * 1000); | |
| 4747 | − | |
| 4748 | − return DIV_ROUND_UP_ULL(mul_u32_u32(clock * bpp, 64 * 1006), | |
| 4749 | − 8 * 54 * 1000 * 1000); | |
| 4760 | + return DIV64_U64_ROUND_UP(mul_u32_u32(clock * bpp, 64 * overhead >> 4), | |
| 4761 | + 1000000ULL * 8 * 54 * 1000); | |
| 4750 | 4762 | } |
| 4751 | 4763 | EXPORT_SYMBOL(drm_dp_calc_pbn_mode); |
| 4752 | 4764 | |
| @@ -4871,7 +4883,8 @@ void drm_dp_mst_dump_topology(struct seq_file *m, · 118 unmodified lines | ||
| 4871 | 4883 | state = to_drm_dp_mst_topology_state(mgr->base.state); |
| 4872 | 4884 | seq_printf(m, "\n*** Atomic state info ***\n"); |
| 4873 | 4885 | seq_printf(m, "payload_mask: %x, max_payloads: %d, start_slot: %u, pbn_div: %d\n", |
| 4874 | − state->payload_mask, mgr->max_payloads, state->start_slot, state->pbn_div); | |
| 4886 | + state->payload_mask, mgr->max_payloads, state->start_slot, | |
| 4887 | + dfixed_trunc(state->pbn_div)); | |
| 4875 | 4888 | |
| 4876 | 4889 | seq_printf(m, "\n| idx | port | vcpi | slots | pbn | dsc | status | sink name |\n"); |
| 4877 | 4890 | for (i = 0; i < mgr->max_payloads; i++) { |
| @@ -5136,13 +5149,67 @@ static bool drm_dp_mst_port_downstream_of_branch(struct drm_dp_mst_port *port, · 258 unmodified lines | ||
| 5136 | 5149 | return false; |
| 5137 | 5150 | } |
| 5138 | 5151 | |
| 5152 | +static bool | |
| 5153 | +drm_dp_mst_port_downstream_of_parent_locked(struct drm_dp_mst_topology_mgr *mgr, | |
| 5154 | + struct drm_dp_mst_port *port, | |
| 5155 | + struct drm_dp_mst_port *parent) | |
| 5156 | +{ | |
| 5157 | + if (!mgr->mst_primary) | |
| 5158 | + return false; | |
| 5159 | + | |
| 5160 | + port = drm_dp_mst_topology_get_port_validated_locked(mgr->mst_primary, | |
| 5161 | + port); | |
| 5162 | + if (!port) | |
| 5163 | + return false; | |
| 5164 | + | |
| 5165 | + if (!parent) | |
| 5166 | + return true; | |
| 5167 | + | |
| 5168 | + parent = drm_dp_mst_topology_get_port_validated_locked(mgr->mst_primary, | |
| 5169 | + parent); | |
| 5170 | + if (!parent) | |
| 5171 | + return false; | |
| 5172 | + | |
| 5173 | + if (!parent->mstb) | |
| 5174 | + return false; | |
| 5175 | + | |
| 5176 | + return drm_dp_mst_port_downstream_of_branch(port, parent->mstb); | |
| 5177 | +} | |
| 5178 | + | |
| 5179 | +/** | |
| 5180 | + * drm_dp_mst_port_downstream_of_parent - check if a port is downstream of a parent port | |
| 5181 | + * @mgr: MST topology manager | |
| 5182 | + * @port: the port being looked up | |
| 5183 | + * @parent: the parent port | |
| 5184 | + * | |
| 5185 | + * The function returns %true if @port is downstream of @parent. If @parent is | |
| 5186 | + * %NULL - denoting the root port - the function returns %true if @port is in | |
| 5187 | + * @mgr's topology. | |
| 5188 | + */ | |
| 5189 | +bool | |
| 5190 | +drm_dp_mst_port_downstream_of_parent(struct drm_dp_mst_topology_mgr *mgr, | |
| 5191 | + struct drm_dp_mst_port *port, | |
| 5192 | + struct drm_dp_mst_port *parent) | |
| 5193 | +{ | |
| 5194 | + bool ret; | |
| 5195 | + | |
| 5196 | + mutex_lock(&mgr->lock); | |
| 5197 | + ret = drm_dp_mst_port_downstream_of_parent_locked(mgr, port, parent); | |
| 5198 | + mutex_unlock(&mgr->lock); | |
| 5199 | + | |
| 5200 | + return ret; | |
| 5201 | +} | |
| 5202 | +EXPORT_SYMBOL(drm_dp_mst_port_downstream_of_parent); | |
| 5203 | + | |
| 5139 | 5204 | static int |
| 5140 | 5205 | drm_dp_mst_atomic_check_port_bw_limit(struct drm_dp_mst_port *port, |
| 5141 | − struct drm_dp_mst_topology_state *state); | |
| 5206 | + struct drm_dp_mst_topology_state *state, | |
| 5207 | + struct drm_dp_mst_port **failing_port); | |
| 5142 | 5208 | |
| 5143 | 5209 | static int |
| 5144 | 5210 | drm_dp_mst_atomic_check_mstb_bw_limit(struct drm_dp_mst_branch *mstb, |
| 5145 | − struct drm_dp_mst_topology_state *state) | |
| 5211 | + struct drm_dp_mst_topology_state *state, | |
| 5212 | + struct drm_dp_mst_port **failing_port) | |
| 5146 | 5213 | { |
| 5147 | 5214 | struct drm_dp_mst_atomic_payload *payload; |
| 5148 | 5215 | struct drm_dp_mst_port *port; |
| @@ -5171,7 +5238,7 @@ drm_dp_mst_atomic_check_mstb_bw_limit(struct drm_dp_mst_branch *mstb, · 22 unmodified lines | ||
| 5171 | 5238 | drm_dbg_atomic(mstb->mgr->dev, "[MSTB:%p] Checking bandwidth limits\n", mstb); |
| 5172 | 5239 | |
| 5173 | 5240 | list_for_each_entry(port, &mstb->ports, next) { |
| 5174 | − ret = drm_dp_mst_atomic_check_port_bw_limit(port, state); | |
| 5241 | + ret = drm_dp_mst_atomic_check_port_bw_limit(port, state, failing_port); | |
| 5175 | 5242 | if (ret < 0) |
| 5176 | 5243 | return ret; |
| 5177 | 5244 | |
| @@ -5183,7 +5250,8 @@ drm_dp_mst_atomic_check_mstb_bw_limit(struct drm_dp_mst_branch *mstb, · 5 unmodified lines | ||
| 5183 | 5250 | |
| 5184 | 5251 | static int |
| 5185 | 5252 | drm_dp_mst_atomic_check_port_bw_limit(struct drm_dp_mst_port *port, |
| 5186 | − struct drm_dp_mst_topology_state *state) | |
| 5253 | + struct drm_dp_mst_topology_state *state, | |
| 5254 | + struct drm_dp_mst_port **failing_port) | |
| 5187 | 5255 | { |
| 5188 | 5256 | struct drm_dp_mst_atomic_payload *payload; |
| 5189 | 5257 | int pbn_used = 0; |
| @@ -5204,13 +5272,15 @@ drm_dp_mst_atomic_check_port_bw_limit(struct drm_dp_mst_port *port, · 14 unmodified lines | ||
| 5204 | 5272 | drm_dbg_atomic(port->mgr->dev, |
| 5205 | 5273 | "[MSTB:%p] [MST PORT:%p] no BW available for the port\n", |
| 5206 | 5274 | port->parent, port); |
| 5275 | + *failing_port = port; | |
| 5207 | 5276 | return -EINVAL; |
| 5208 | 5277 | } |
| 5209 | 5278 | |
| 5210 | 5279 | pbn_used = payload->pbn; |
| 5211 | 5280 | } else { |
| 5212 | 5281 | pbn_used = drm_dp_mst_atomic_check_mstb_bw_limit(port->mstb, |
| 5213 | − state); | |
| 5282 | + state, | |
| 5283 | + failing_port); | |
| 5214 | 5284 | if (pbn_used <= 0) |
| 5215 | 5285 | return pbn_used; |
| 5216 | 5286 | } |
| @@ -5219,6 +5289,7 @@ drm_dp_mst_atomic_check_port_bw_limit(struct drm_dp_mst_port *port, · 2 unmodified lines | ||
| 5219 | 5289 | drm_dbg_atomic(port->mgr->dev, |
| 5220 | 5290 | "[MSTB:%p] [MST PORT:%p] required PBN of %d exceeds port limit of %d\n", |
| 5221 | 5291 | port->parent, port, pbn_used, port->full_pbn); |
| 5292 | + *failing_port = port; | |
| 5222 | 5293 | return -ENOSPC; |
| 5223 | 5294 | } |
| 5224 | 5295 | |
| @@ -5271,10 +5342,10 @@ drm_dp_mst_atomic_check_payload_alloc_limits(struct drm_dp_mst_topology_mgr *mgr · 46 unmodified lines | ||
| 5271 | 5342 | } |
| 5272 | 5343 | |
| 5273 | 5344 | if (!payload_count) |
| 5274 | − mst_state->pbn_div = 0; | |
| 5345 | + mst_state->pbn_div.full = dfixed_const(0); | |
| 5275 | 5346 | |
| 5276 | 5347 | drm_dbg_atomic(mgr->dev, "[MST MGR:%p] mst state %p TU pbn_div=%d avail=%d used=%d\n", |
| 5277 | − mgr, mst_state, mst_state->pbn_div, avail_slots, | |
| 5348 | + mgr, mst_state, dfixed_trunc(mst_state->pbn_div), avail_slots, | |
| 5278 | 5349 | mst_state->total_avail_slots - avail_slots); |
| 5279 | 5350 | |
| 5280 | 5351 | return 0; |
| @@ -5396,20 +5467,82 @@ int drm_dp_mst_atomic_enable_dsc(struct drm_atomic_state *state, · 115 unmodified lines | ||
| 5396 | 5467 | } |
| 5397 | 5468 | EXPORT_SYMBOL(drm_dp_mst_atomic_enable_dsc); |
| 5398 | 5469 | |
| 5470 | +/** | |
| 5471 | + * drm_dp_mst_atomic_check_mgr - Check the atomic state of an MST topology manager | |
| 5472 | + * @state: The global atomic state | |
| 5473 | + * @mgr: Manager to check | |
| 5474 | + * @mst_state: The MST atomic state for @mgr | |
| 5475 | + * @failing_port: Returns the port with a BW limitation | |
| 5476 | + * | |
| 5477 | + * Checks the given MST manager's topology state for an atomic update to ensure | |
| 5478 | + * that it's valid. This includes checking whether there's enough bandwidth to | |
| 5479 | + * support the new timeslot allocations in the atomic update. | |
| 5480 | + * | |
| 5481 | + * Any atomic drivers supporting DP MST must make sure to call this or | |
| 5482 | + * the drm_dp_mst_atomic_check() function after checking the rest of their state | |
| 5483 | + * in their &drm_mode_config_funcs.atomic_check() callback. | |
| 5484 | + * | |
| 5485 | + * See also: | |
| 5486 | + * drm_dp_mst_atomic_check() | |
| 5487 | + * drm_dp_atomic_find_time_slots() | |
| 5488 | + * drm_dp_atomic_release_time_slots() | |
| 5489 | + * | |
| 5490 | + * Returns: | |
| 5491 | + * - 0 if the new state is valid | |
| 5492 | + * - %-ENOSPC, if the new state is invalid, because of BW limitation | |
| 5493 | + * @failing_port is set to: | |
| 5494 | + * - The non-root port where a BW limit check failed | |
| 5495 | + * with all the ports downstream of @failing_port passing | |
| 5496 | + * the BW limit check. | |
| 5497 | + * The returned port pointer is valid until at least | |
| 5498 | + * one payload downstream of it exists. | |
| 5499 | + * - %NULL if the BW limit check failed at the root port | |
| 5500 | + * with all the ports downstream of the root port passing | |
| 5501 | + * the BW limit check. | |
| 5502 | + * - %-EINVAL, if the new state is invalid, because the root port has | |
| 5503 | + * too many payloads. | |
| 5504 | + */ | |
| 5505 | +int drm_dp_mst_atomic_check_mgr(struct drm_atomic_state *state, | |
| 5506 | + struct drm_dp_mst_topology_mgr *mgr, | |
| 5507 | + struct drm_dp_mst_topology_state *mst_state, | |
| 5508 | + struct drm_dp_mst_port **failing_port) | |
| 5509 | +{ | |
| 5510 | + int ret; | |
| 5511 | + | |
| 5512 | + *failing_port = NULL; | |
| 5513 | + | |
| 5514 | + if (!mgr->mst_state) | |
| 5515 | + return 0; | |
| 5516 | + | |
| 5517 | + mutex_lock(&mgr->lock); | |
| 5518 | + ret = drm_dp_mst_atomic_check_mstb_bw_limit(mgr->mst_primary, | |
| 5519 | + mst_state, | |
| 5520 | + failing_port); | |
| 5521 | + mutex_unlock(&mgr->lock); | |
| 5522 | + | |
| 5523 | + if (ret < 0) | |
| 5524 | + return ret; | |
| 5525 | + | |
| 5526 | + return drm_dp_mst_atomic_check_payload_alloc_limits(mgr, mst_state); | |
| 5527 | +} | |
| 5528 | +EXPORT_SYMBOL(drm_dp_mst_atomic_check_mgr); | |
| 5529 | + | |
| 5399 | 5530 | /** |
| 5400 | 5531 | * drm_dp_mst_atomic_check - Check that the new state of an MST topology in an |
| 5401 | 5532 | * atomic update is valid |
| 5402 | 5533 | * @state: Pointer to the new &struct drm_dp_mst_topology_state |
| 5403 | 5534 | * |
| 5404 | 5535 | * Checks the given topology state for an atomic update to ensure that it's |
| 5405 | − * valid. This includes checking whether there's enough bandwidth to support | |
| 5406 | − * the new timeslot allocations in the atomic update. | |
| 5536 | + * valid, calling drm_dp_mst_atomic_check_mgr() for all MST manager in the | |
| 5537 | + * atomic state. This includes checking whether there's enough bandwidth to | |
| 5538 | + * support the new timeslot allocations in the atomic update. | |
| 5407 | 5539 | * |
| 5408 | 5540 | * Any atomic drivers supporting DP MST must make sure to call this after |
| 5409 | 5541 | * checking the rest of their state in their |
| 5410 | 5542 | * &drm_mode_config_funcs.atomic_check() callback. |
| 5411 | 5543 | * |
| 5412 | 5544 | * See also: |
| 5545 | + * drm_dp_mst_atomic_check_mgr() | |
| 5413 | 5546 | * drm_dp_atomic_find_time_slots() |
| 5414 | 5547 | * drm_dp_atomic_release_time_slots() |
| 5415 | 5548 | * |
| @@ -5424,21 +5557,11 @@ int drm_dp_mst_atomic_check(struct drm_atomic_state *state) · 8 unmodified lines | ||
| 5424 | 5557 | int i, ret = 0; |
| 5425 | 5558 | |
| 5426 | 5559 | for_each_new_mst_mgr_in_state(state, mgr, mst_state, i) { |
| 5427 | − if (!mgr->mst_state) | |
| 5428 | − continue; | |
| 5560 | + struct drm_dp_mst_port *tmp_port; | |
| 5429 | 5561 | |
| 5430 | − ret = drm_dp_mst_atomic_check_payload_alloc_limits(mgr, mst_state); | |
| 5562 | + ret = drm_dp_mst_atomic_check_mgr(state, mgr, mst_state, &tmp_port); | |
| 5431 | 5563 | if (ret) |
| 5432 | 5564 | break; |
| 5433 | − | |
| 5434 | − mutex_lock(&mgr->lock); | |
| 5435 | − ret = drm_dp_mst_atomic_check_mstb_bw_limit(mgr->mst_primary, | |
| 5436 | − mst_state); | |
| 5437 | − mutex_unlock(&mgr->lock); | |
| 5438 | − if (ret < 0) | |
| 5439 | − break; | |
| 5440 | − else | |
| 5441 | − ret = 0; | |
| 5442 | 5565 | } |
| 5443 | 5566 | |
| 5444 | 5567 | return ret; |
| @@ -5894,6 +6017,7 @@ static bool drm_dp_mst_is_virtual_dpcd(struct drm_dp_mst_port *port) · 449 unmodified lines | ||
| 5894 | 6017 | struct drm_dp_aux *drm_dp_mst_dsc_aux_for_port(struct drm_dp_mst_port *port) |
| 5895 | 6018 | { |
| 5896 | 6019 | struct drm_dp_mst_port *immediate_upstream_port; |
| 6020 | + struct drm_dp_aux *immediate_upstream_aux; | |
| 5897 | 6021 | struct drm_dp_mst_port *fec_port; |
| 5898 | 6022 | struct drm_dp_desc desc = {}; |
| 5899 | 6023 | u8 endpoint_fec; |
| @@ -5958,21 +6082,25 @@ struct drm_dp_aux *drm_dp_mst_dsc_aux_for_port(struct drm_dp_mst_port *port) · 58 unmodified lines | ||
| 5958 | 6082 | * - Port is on primary branch device |
| 5959 | 6083 | * - Not a VGA adapter (DP_DWN_STRM_PORT_TYPE_ANALOG) |
| 5960 | 6084 | */ |
| 5961 | − if (drm_dp_read_desc(port->mgr->aux, &desc, true)) | |
| 6085 | + if (immediate_upstream_port) | |
| 6086 | + immediate_upstream_aux = &immediate_upstream_port->aux; | |
| 6087 | + else | |
| 6088 | + immediate_upstream_aux = port->mgr->aux; | |
| 6089 | + | |
| 6090 | + if (drm_dp_read_desc(immediate_upstream_aux, &desc, true)) | |
| 5962 | 6091 | return NULL; |
| 5963 | 6092 | |
| 5964 | − if (drm_dp_has_quirk(&desc, DP_DPCD_QUIRK_DSC_WITHOUT_VIRTUAL_DPCD) && | |
| 5965 | − port->mgr->dpcd[DP_DPCD_REV] >= DP_DPCD_REV_14 && | |
| 5966 | − port->parent == port->mgr->mst_primary) { | |
| 6093 | + if (drm_dp_has_quirk(&desc, DP_DPCD_QUIRK_DSC_WITHOUT_VIRTUAL_DPCD)) { | |
| 5967 | 6094 | u8 dpcd_ext[DP_RECEIVER_CAP_SIZE]; |
| 5968 | 6095 | |
| 5969 | − if (drm_dp_read_dpcd_caps(port->mgr->aux, dpcd_ext) < 0) | |
| 6096 | + if (drm_dp_read_dpcd_caps(immediate_upstream_aux, dpcd_ext) < 0) | |
| 5970 | 6097 | return NULL; |
| 5971 | 6098 | |
| 5972 | − if ((dpcd_ext[DP_DOWNSTREAMPORT_PRESENT] & DP_DWN_STRM_PORT_PRESENT) && | |
| 6099 | + if (dpcd_ext[DP_DPCD_REV] >= DP_DPCD_REV_14 && | |
| 6100 | + ((dpcd_ext[DP_DOWNSTREAMPORT_PRESENT] & DP_DWN_STRM_PORT_PRESENT) && | |
| 5973 | 6101 | ((dpcd_ext[DP_DOWNSTREAMPORT_PRESENT] & DP_DWN_STRM_PORT_TYPE_MASK) |
| 5974 | − != DP_DWN_STRM_PORT_TYPE_ANALOG)) | |
| 5975 | − return port->mgr->aux; | |
| 6102 | + != DP_DWN_STRM_PORT_TYPE_ANALOG))) | |
| 6103 | + return immediate_upstream_aux; | |
| 5976 | 6104 | } |
| 5977 | 6105 | |
| 5978 | 6106 | /* |
| @@ -1,451 +0,0 @@ | ||
| 1 | −/* | |
| 2 | − * \file drm_agpsupport.c | |
| 3 | − * DRM support for AGP/GART backend | |
| 4 | − * | |
| 5 | − * \author Rickard E. (Rik) Faith <faith@valinux.com> | |
| 6 | − * \author Gareth Hughes <gareth@valinux.com> | |
| 7 | − */ | |
| 8 | − | |
| 9 | −/* | |
| 10 | − * Copyright 1999 Precision Insight, Inc., Cedar Park, Texas. | |
| 11 | − * Copyright 2000 VA Linux Systems, Inc., Sunnyvale, California. | |
| 12 | − * All Rights Reserved. | |
| 13 | − * | |
| 14 | − * Permission is hereby granted, free of charge, to any person obtaining a | |
| 15 | − * copy of this software and associated documentation files (the "Software"), | |
| 16 | − * to deal in the Software without restriction, including without limitation | |
| 17 | − * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 18 | − * and/or sell copies of the Software, and to permit persons to whom the | |
| 19 | − * Software is furnished to do so, subject to the following conditions: | |
| 20 | − * | |
| 21 | − * The above copyright notice and this permission notice (including the next | |
| 22 | − * paragraph) shall be included in all copies or substantial portions of the | |
| 23 | − * Software. | |
| 24 | − * | |
| 25 | − * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 26 | − * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 27 | − * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 28 | − * VA LINUX SYSTEMS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 29 | − * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 30 | − * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 31 | − * OTHER DEALINGS IN THE SOFTWARE. | |
| 32 | − */ | |
| 33 | − | |
| 34 | −#include <linux/module.h> | |
| 35 | −#include <linux/pci.h> | |
| 36 | −#include <linux/slab.h> | |
| 37 | − | |
| 38 | −#if IS_ENABLED(CONFIG_AGP) | |
| 39 | −#include <asm/agp.h> | |
| 40 | −#endif | |
| 41 | − | |
| 42 | −#include <drm/drm_device.h> | |
| 43 | −#include <drm/drm_drv.h> | |
| 44 | −#include <drm/drm_file.h> | |
| 45 | −#include <drm/drm_print.h> | |
| 46 | − | |
| 47 | −#include "drm_legacy.h" | |
| 48 | − | |
| 49 | −#if IS_ENABLED(CONFIG_AGP) | |
| 50 | − | |
| 51 | −/* | |
| 52 | − * Get AGP information. | |
| 53 | − * | |
| 54 | − * \return zero on success or a negative number on failure. | |
| 55 | − * | |
| 56 | − * Verifies the AGP device has been initialized and acquired and fills in the | |
| 57 | − * drm_agp_info structure with the information in drm_agp_head::agp_info. | |
| 58 | − */ | |
| 59 | −int drm_legacy_agp_info(struct drm_device *dev, struct drm_agp_info *info) | |
| 60 | −{ | |
| 61 | − struct agp_kern_info *kern; | |
| 62 | − | |
| 63 | − if (!dev->agp || !dev->agp->acquired) | |
| 64 | − return -EINVAL; | |
| 65 | − | |
| 66 | − kern = &dev->agp->agp_info; | |
| 67 | − info->agp_version_major = kern->version.major; | |
| 68 | − info->agp_version_minor = kern->version.minor; | |
| 69 | − info->mode = kern->mode; | |
| 70 | − info->aperture_base = kern->aper_base; | |
| 71 | − info->aperture_size = kern->aper_size * 1024 * 1024; | |
| 72 | − info->memory_allowed = kern->max_memory << PAGE_SHIFT; | |
| 73 | − info->memory_used = kern->current_memory << PAGE_SHIFT; | |
| 74 | − info->id_vendor = kern->device->vendor; | |
| 75 | − info->id_device = kern->device->device; | |
| 76 | − | |
| 77 | − return 0; | |
| 78 | −} | |
| 79 | −EXPORT_SYMBOL(drm_legacy_agp_info); | |
| 80 | − | |
| 81 | −int drm_legacy_agp_info_ioctl(struct drm_device *dev, void *data, | |
| 82 | − struct drm_file *file_priv) | |
| 83 | −{ | |
| 84 | − struct drm_agp_info *info = data; | |
| 85 | − int err; | |
| 86 | − | |
| 87 | − err = drm_legacy_agp_info(dev, info); | |
| 88 | − if (err) | |
| 89 | − return err; | |
| 90 | − | |
| 91 | − return 0; | |
| 92 | −} | |
| 93 | − | |
| 94 | −/* | |
| 95 | − * Acquire the AGP device. | |
| 96 | − * | |
| 97 | − * \param dev DRM device that is to acquire AGP. | |
| 98 | − * \return zero on success or a negative number on failure. | |
| 99 | − * | |
| 100 | − * Verifies the AGP device hasn't been acquired before and calls | |
| 101 | − * \c agp_backend_acquire. | |
| 102 | − */ | |
| 103 | −int drm_legacy_agp_acquire(struct drm_device *dev) | |
| 104 | −{ | |
| 105 | − struct pci_dev *pdev = to_pci_dev(dev->dev); | |
| 106 | − | |
| 107 | − if (!dev->agp) | |
| 108 | − return -ENODEV; | |
| 109 | − if (dev->agp->acquired) | |
| 110 | − return -EBUSY; | |
| 111 | − dev->agp->bridge = agp_backend_acquire(pdev); | |
| 112 | − if (!dev->agp->bridge) | |
| 113 | − return -ENODEV; | |
| 114 | − dev->agp->acquired = 1; | |
| 115 | − return 0; | |
| 116 | −} | |
| 117 | −EXPORT_SYMBOL(drm_legacy_agp_acquire); | |
| 118 | − | |
| 119 | −/* | |
| 120 | − * Acquire the AGP device (ioctl). | |
| 121 | − * | |
| 122 | − * \return zero on success or a negative number on failure. | |
| 123 | − * | |
| 124 | − * Verifies the AGP device hasn't been acquired before and calls | |
| 125 | − * \c agp_backend_acquire. | |
| 126 | − */ | |
| 127 | −int drm_legacy_agp_acquire_ioctl(struct drm_device *dev, void *data, | |
| 128 | − struct drm_file *file_priv) | |
| 129 | −{ | |
| 130 | − return drm_legacy_agp_acquire((struct drm_device *)file_priv->minor->dev); | |
| 131 | −} | |
| 132 | − | |
| 133 | −/* | |
| 134 | − * Release the AGP device. | |
| 135 | − * | |
| 136 | − * \param dev DRM device that is to release AGP. | |
| 137 | − * \return zero on success or a negative number on failure. | |
| 138 | − * | |
| 139 | − * Verifies the AGP device has been acquired and calls \c agp_backend_release. | |
| 140 | − */ | |
| 141 | −int drm_legacy_agp_release(struct drm_device *dev) | |
| 142 | −{ | |
| 143 | − if (!dev->agp || !dev->agp->acquired) | |
| 144 | − return -EINVAL; | |
| 145 | − agp_backend_release(dev->agp->bridge); | |
| 146 | − dev->agp->acquired = 0; | |
| 147 | − return 0; | |
| 148 | −} | |
| 149 | −EXPORT_SYMBOL(drm_legacy_agp_release); | |
| 150 | − | |
| 151 | −int drm_legacy_agp_release_ioctl(struct drm_device *dev, void *data, | |
| 152 | − struct drm_file *file_priv) | |
| 153 | −{ | |
| 154 | − return drm_legacy_agp_release(dev); | |
| 155 | −} | |
| 156 | − | |
| 157 | −/* | |
| 158 | − * Enable the AGP bus. | |
| 159 | − * | |
| 160 | − * \param dev DRM device that has previously acquired AGP. | |
| 161 | − * \param mode Requested AGP mode. | |
| 162 | − * \return zero on success or a negative number on failure. | |
| 163 | − * | |
| 164 | − * Verifies the AGP device has been acquired but not enabled, and calls | |
| 165 | − * \c agp_enable. | |
| 166 | − */ | |
| 167 | −int drm_legacy_agp_enable(struct drm_device *dev, struct drm_agp_mode mode) | |
| 168 | −{ | |
| 169 | − if (!dev->agp || !dev->agp->acquired) | |
| 170 | − return -EINVAL; | |
| 171 | − | |
| 172 | − dev->agp->mode = mode.mode; | |
| 173 | − agp_enable(dev->agp->bridge, mode.mode); | |
| 174 | − dev->agp->enabled = 1; | |
| 175 | − return 0; | |
| 176 | −} | |
| 177 | −EXPORT_SYMBOL(drm_legacy_agp_enable); | |
| 178 | − | |
| 179 | −int drm_legacy_agp_enable_ioctl(struct drm_device *dev, void *data, | |
| 180 | − struct drm_file *file_priv) | |
| 181 | −{ | |
| 182 | − struct drm_agp_mode *mode = data; | |
| 183 | − | |
| 184 | − return drm_legacy_agp_enable(dev, *mode); | |
| 185 | −} | |
| 186 | − | |
| 187 | −/* | |
| 188 | − * Allocate AGP memory. | |
| 189 | − * | |
| 190 | − * \return zero on success or a negative number on failure. | |
| 191 | − * | |
| 192 | − * Verifies the AGP device is present and has been acquired, allocates the | |
| 193 | − * memory via agp_allocate_memory() and creates a drm_agp_mem entry for it. | |
| 194 | − */ | |
| 195 | −int drm_legacy_agp_alloc(struct drm_device *dev, struct drm_agp_buffer *request) | |
| 196 | −{ | |
| 197 | − struct drm_agp_mem *entry; | |
| 198 | − struct agp_memory *memory; | |
| 199 | − unsigned long pages; | |
| 200 | − u32 type; | |
| 201 | − | |
| 202 | − if (!dev->agp || !dev->agp->acquired) | |
| 203 | − return -EINVAL; | |
| 204 | − entry = kzalloc(sizeof(*entry), GFP_KERNEL); | |
| 205 | − if (!entry) | |
| 206 | − return -ENOMEM; | |
| 207 | − | |
| 208 | − pages = DIV_ROUND_UP(request->size, PAGE_SIZE); | |
| 209 | − type = (u32) request->type; | |
| 210 | − memory = agp_allocate_memory(dev->agp->bridge, pages, type); | |
| 211 | − if (!memory) { | |
| 212 | − kfree(entry); | |
| 213 | − return -ENOMEM; | |
| 214 | − } | |
| 215 | − | |
| 216 | − entry->handle = (unsigned long)memory->key + 1; | |
| 217 | − entry->memory = memory; | |
| 218 | − entry->bound = 0; | |
| 219 | − entry->pages = pages; | |
| 220 | − list_add(&entry->head, &dev->agp->memory); | |
| 221 | − | |
| 222 | − request->handle = entry->handle; | |
| 223 | − request->physical = memory->physical; | |
| 224 | − | |
| 225 | − return 0; | |
| 226 | −} | |
| 227 | −EXPORT_SYMBOL(drm_legacy_agp_alloc); | |
| 228 | − | |
| 229 | − | |
| 230 | −int drm_legacy_agp_alloc_ioctl(struct drm_device *dev, void *data, | |
| 231 | − struct drm_file *file_priv) | |
| 232 | −{ | |
| 233 | − struct drm_agp_buffer *request = data; | |
| 234 | − | |
| 235 | − return drm_legacy_agp_alloc(dev, request); | |
| 236 | −} | |
| 237 | − | |
| 238 | −/* | |
| 239 | − * Search for the AGP memory entry associated with a handle. | |
| 240 | − * | |
| 241 | − * \param dev DRM device structure. | |
| 242 | − * \param handle AGP memory handle. | |
| 243 | − * \return pointer to the drm_agp_mem structure associated with \p handle. | |
| 244 | − * | |
| 245 | − * Walks through drm_agp_head::memory until finding a matching handle. | |
| 246 | − */ | |
| 247 | −static struct drm_agp_mem *drm_legacy_agp_lookup_entry(struct drm_device *dev, | |
| 248 | − unsigned long handle) | |
| 249 | −{ | |
| 250 | − struct drm_agp_mem *entry; | |
| 251 | − | |
| 252 | − list_for_each_entry(entry, &dev->agp->memory, head) { | |
| 253 | − if (entry->handle == handle) | |
| 254 | − return entry; | |
| 255 | − } | |
| 256 | − return NULL; | |
| 257 | −} | |
| 258 | − | |
| 259 | −/* | |
| 260 | − * Unbind AGP memory from the GATT (ioctl). | |
| 261 | − * | |
| 262 | − * \return zero on success or a negative number on failure. | |
| 263 | − * | |
| 264 | − * Verifies the AGP device is present and acquired, looks-up the AGP memory | |
| 265 | − * entry and passes it to the unbind_agp() function. | |
| 266 | − */ | |
| 267 | −int drm_legacy_agp_unbind(struct drm_device *dev, struct drm_agp_binding *request) | |
| 268 | −{ | |
| 269 | − struct drm_agp_mem *entry; | |
| 270 | − int ret; | |
| 271 | − | |
| 272 | − if (!dev->agp || !dev->agp->acquired) | |
| 273 | − return -EINVAL; | |
| 274 | − entry = drm_legacy_agp_lookup_entry(dev, request->handle); | |
| 275 | − if (!entry || !entry->bound) | |
| 276 | − return -EINVAL; | |
| 277 | − ret = agp_unbind_memory(entry->memory); | |
| 278 | − if (ret == 0) | |
| 279 | − entry->bound = 0; | |
| 280 | − return ret; | |
| 281 | −} | |
| 282 | −EXPORT_SYMBOL(drm_legacy_agp_unbind); | |
| 283 | − | |
| 284 | − | |
| 285 | −int drm_legacy_agp_unbind_ioctl(struct drm_device *dev, void *data, | |
| 286 | − struct drm_file *file_priv) | |
| 287 | −{ | |
| 288 | − struct drm_agp_binding *request = data; | |
| 289 | − | |
| 290 | − return drm_legacy_agp_unbind(dev, request); | |
| 291 | −} | |
| 292 | − | |
| 293 | −/* | |
| 294 | − * Bind AGP memory into the GATT (ioctl) | |
| 295 | − * | |
| 296 | − * \return zero on success or a negative number on failure. | |
| 297 | − * | |
| 298 | − * Verifies the AGP device is present and has been acquired and that no memory | |
| 299 | − * is currently bound into the GATT. Looks-up the AGP memory entry and passes | |
| 300 | − * it to bind_agp() function. | |
| 301 | − */ | |
| 302 | −int drm_legacy_agp_bind(struct drm_device *dev, struct drm_agp_binding *request) | |
| 303 | −{ | |
| 304 | − struct drm_agp_mem *entry; | |
| 305 | − int retcode; | |
| 306 | − int page; | |
| 307 | − | |
| 308 | − if (!dev->agp || !dev->agp->acquired) | |
| 309 | − return -EINVAL; | |
| 310 | − entry = drm_legacy_agp_lookup_entry(dev, request->handle); | |
| 311 | − if (!entry || entry->bound) | |
| 312 | − return -EINVAL; | |
| 313 | − page = DIV_ROUND_UP(request->offset, PAGE_SIZE); | |
| 314 | − retcode = agp_bind_memory(entry->memory, page); | |
| 315 | − if (retcode) | |
| 316 | − return retcode; | |
| 317 | − entry->bound = dev->agp->base + (page << PAGE_SHIFT); | |
| 318 | − DRM_DEBUG("base = 0x%lx entry->bound = 0x%lx\n", | |
| 319 | − dev->agp->base, entry->bound); | |
| 320 | − return 0; | |
| 321 | −} | |
| 322 | −EXPORT_SYMBOL(drm_legacy_agp_bind); | |
| 323 | − | |
| 324 | − | |
| 325 | −int drm_legacy_agp_bind_ioctl(struct drm_device *dev, void *data, | |
| 326 | − struct drm_file *file_priv) | |
| 327 | −{ | |
| 328 | − struct drm_agp_binding *request = data; | |
| 329 | − | |
| 330 | − return drm_legacy_agp_bind(dev, request); | |
| 331 | −} | |
| 332 | − | |
| 333 | −/* | |
| 334 | − * Free AGP memory (ioctl). | |
| 335 | − * | |
| 336 | − * \return zero on success or a negative number on failure. | |
| 337 | − * | |
| 338 | − * Verifies the AGP device is present and has been acquired and looks up the | |
| 339 | − * AGP memory entry. If the memory is currently bound, unbind it via | |
| 340 | − * unbind_agp(). Frees it via free_agp() as well as the entry itself | |
| 341 | − * and unlinks from the doubly linked list it's inserted in. | |
| 342 | − */ | |
| 343 | −int drm_legacy_agp_free(struct drm_device *dev, struct drm_agp_buffer *request) | |
| 344 | −{ | |
| 345 | − struct drm_agp_mem *entry; | |
| 346 | − | |
| 347 | − if (!dev->agp || !dev->agp->acquired) | |
| 348 | − return -EINVAL; | |
| 349 | − entry = drm_legacy_agp_lookup_entry(dev, request->handle); | |
| 350 | − if (!entry) | |
| 351 | − return -EINVAL; | |
| 352 | − if (entry->bound) | |
| 353 | − agp_unbind_memory(entry->memory); | |
| 354 | − | |
| 355 | − list_del(&entry->head); | |
| 356 | − | |
| 357 | − agp_free_memory(entry->memory); | |
| 358 | − kfree(entry); | |
| 359 | − return 0; | |
| 360 | −} | |
| 361 | −EXPORT_SYMBOL(drm_legacy_agp_free); | |
| 362 | − | |
| 363 | − | |
| 364 | −int drm_legacy_agp_free_ioctl(struct drm_device *dev, void *data, | |
| 365 | − struct drm_file *file_priv) | |
| 366 | −{ | |
| 367 | − struct drm_agp_buffer *request = data; | |
| 368 | − | |
| 369 | − return drm_legacy_agp_free(dev, request); | |
| 370 | −} | |
| 371 | − | |
| 372 | −/* | |
| 373 | − * Initialize the AGP resources. | |
| 374 | − * | |
| 375 | − * \return pointer to a drm_agp_head structure. | |
| 376 | − * | |
| 377 | − * Gets the drm_agp_t structure which is made available by the agpgart module | |
| 378 | − * via the inter_module_* functions. Creates and initializes a drm_agp_head | |
| 379 | − * structure. | |
| 380 | − * | |
| 381 | − * Note that final cleanup of the kmalloced structure is directly done in | |
| 382 | − * drm_pci_agp_destroy. | |
| 383 | − */ | |
| 384 | −struct drm_agp_head *drm_legacy_agp_init(struct drm_device *dev) | |
| 385 | −{ | |
| 386 | − struct pci_dev *pdev = to_pci_dev(dev->dev); | |
| 387 | − struct drm_agp_head *head = NULL; | |
| 388 | − | |
| 389 | − head = kzalloc(sizeof(*head), GFP_KERNEL); | |
| 390 | − if (!head) | |
| 391 | − return NULL; | |
| 392 | − head->bridge = agp_find_bridge(pdev); | |
| 393 | − if (!head->bridge) { | |
| 394 | − head->bridge = agp_backend_acquire(pdev); | |
| 395 | − if (!head->bridge) { | |
| 396 | − kfree(head); | |
| 397 | − return NULL; | |
| 398 | − } | |
| 399 | − agp_copy_info(head->bridge, &head->agp_info); | |
| 400 | − agp_backend_release(head->bridge); | |
| 401 | − } else { | |
| 402 | − agp_copy_info(head->bridge, &head->agp_info); | |
| 403 | − } | |
| 404 | − if (head->agp_info.chipset == NOT_SUPPORTED) { | |
| 405 | − kfree(head); | |
| 406 | − return NULL; | |
| 407 | − } | |
| 408 | − INIT_LIST_HEAD(&head->memory); | |
| 409 | − head->cant_use_aperture = head->agp_info.cant_use_aperture; | |
| 410 | − head->page_mask = head->agp_info.page_mask; | |
| 411 | − head->base = head->agp_info.aper_base; | |
| 412 | − return head; | |
| 413 | −} | |
| 414 | −/* Only exported for i810.ko */ | |
| 415 | −EXPORT_SYMBOL(drm_legacy_agp_init); | |
| 416 | − | |
| 417 | −/** | |
| 418 | − * drm_legacy_agp_clear - Clear AGP resource list | |
| 419 | − * @dev: DRM device | |
| 420 | − * | |
| 421 | − * Iterate over all AGP resources and remove them. But keep the AGP head | |
| 422 | − * intact so it can still be used. It is safe to call this if AGP is disabled or | |
| 423 | − * was already removed. | |
| 424 | − * | |
| 425 | − * Cleanup is only done for drivers who have DRIVER_LEGACY set. | |
| 426 | − */ | |
| 427 | −void drm_legacy_agp_clear(struct drm_device *dev) | |
| 428 | −{ | |
| 429 | − struct drm_agp_mem *entry, *tempe; | |
| 430 | − | |
| 431 | − if (!dev->agp) | |
| 432 | − return; | |
| 433 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 434 | − return; | |
| 435 | − | |
| 436 | − list_for_each_entry_safe(entry, tempe, &dev->agp->memory, head) { | |
| 437 | − if (entry->bound) | |
| 438 | − agp_unbind_memory(entry->memory); | |
| 439 | − agp_free_memory(entry->memory); | |
| 440 | − kfree(entry); | |
| 441 | − } | |
| 442 | − INIT_LIST_HEAD(&dev->agp->memory); | |
| 443 | − | |
| 444 | − if (dev->agp->acquired) | |
| 445 | − drm_legacy_agp_release(dev); | |
| 446 | − | |
| 447 | − dev->agp->acquired = 0; | |
| 448 | − dev->agp->enabled = 0; | |
| 449 | −} | |
| 450 | − | |
| 451 | −#endif | |
| @@ -1773,6 +1773,7 @@ static void __drm_state_dump(struct drm_device *dev, struct drm_printer *p, · 1772 unmodified lines | ||
| 1773 | 1773 | struct drm_crtc *crtc; |
| 1774 | 1774 | struct drm_connector *connector; |
| 1775 | 1775 | struct drm_connector_list_iter conn_iter; |
| 1776 | + struct drm_private_obj *obj; | |
| 1776 | 1777 | |
| 1777 | 1778 | if (!drm_drv_uses_atomic_modeset(dev)) |
| 1778 | 1779 | return; |
| @@ -1801,6 +1802,14 @@ static void __drm_state_dump(struct drm_device *dev, struct drm_printer *p, · 22 unmodified lines | ||
| 1801 | 1802 | if (take_locks) |
| 1802 | 1803 | drm_modeset_unlock(&dev->mode_config.connection_mutex); |
| 1803 | 1804 | drm_connector_list_iter_end(&conn_iter); |
| 1805 | + | |
| 1806 | + list_for_each_entry(obj, &config->privobj_list, head) { | |
| 1807 | + if (take_locks) | |
| 1808 | + drm_modeset_lock(&obj->lock, NULL); | |
| 1809 | + drm_atomic_private_obj_print_state(p, obj->state); | |
| 1810 | + if (take_locks) | |
| 1811 | + drm_modeset_unlock(&obj->lock); | |
| 1812 | + } | |
| 1804 | 1813 | } |
| 1805 | 1814 | |
| 1806 | 1815 | /** |
| @@ -795,9 +795,9 @@ drm_atomic_helper_check_modeset(struct drm_device *dev, · 794 unmodified lines | ||
| 795 | 795 | EXPORT_SYMBOL(drm_atomic_helper_check_modeset); |
| 796 | 796 | |
| 797 | 797 | /** |
| 798 | − * drm_atomic_helper_check_wb_encoder_state() - Check writeback encoder state | |
| 799 | − * @encoder: encoder state to check | |
| 800 | − * @conn_state: connector state to check | |
| 798 | + * drm_atomic_helper_check_wb_connector_state() - Check writeback connector state | |
| 799 | + * @connector: corresponding connector | |
| 800 | + * @state: the driver state object | |
| 801 | 801 | * |
| 802 | 802 | * Checks if the writeback connector state is valid, and returns an error if it |
| 803 | 803 | * isn't. |
| @@ -806,9 +806,11 @@ EXPORT_SYMBOL(drm_atomic_helper_check_modeset); · 2 unmodified lines | ||
| 806 | 806 | * Zero for success or -errno |
| 807 | 807 | */ |
| 808 | 808 | int |
| 809 | −drm_atomic_helper_check_wb_encoder_state(struct drm_encoder *encoder, | |
| 810 | − struct drm_connector_state *conn_state) | |
| 809 | +drm_atomic_helper_check_wb_connector_state(struct drm_connector *connector, | |
| 810 | + struct drm_atomic_state *state) | |
| 811 | 811 | { |
| 812 | + struct drm_connector_state *conn_state = | |
| 813 | + drm_atomic_get_new_connector_state(state, connector); | |
| 812 | 814 | struct drm_writeback_job *wb_job = conn_state->writeback_job; |
| 813 | 815 | struct drm_property_blob *pixel_format_blob; |
| 814 | 816 | struct drm_framebuffer *fb; |
| @@ -827,11 +829,11 @@ drm_atomic_helper_check_wb_encoder_state(struct drm_encoder *encoder, · 12 unmodified lines | ||
| 827 | 829 | if (fb->format->format == formats[i]) |
| 828 | 830 | return 0; |
| 829 | 831 | |
| 830 | − drm_dbg_kms(encoder->dev, "Invalid pixel format %p4cc\n", &fb->format->format); | |
| 832 | + drm_dbg_kms(connector->dev, "Invalid pixel format %p4cc\n", &fb->format->format); | |
| 831 | 833 | |
| 832 | 834 | return -EINVAL; |
| 833 | 835 | } |
| 834 | −EXPORT_SYMBOL(drm_atomic_helper_check_wb_encoder_state); | |
| 836 | +EXPORT_SYMBOL(drm_atomic_helper_check_wb_connector_state); | |
| 835 | 837 | |
| 836 | 838 | /** |
| 837 | 839 | * drm_atomic_helper_check_plane_state() - Check plane state for validity |
| @@ -2382,10 +2384,10 @@ int drm_atomic_helper_setup_commit(struct drm_atomic_state *state, · 1544 unmodified lines | ||
| 2382 | 2384 | EXPORT_SYMBOL(drm_atomic_helper_setup_commit); |
| 2383 | 2385 | |
| 2384 | 2386 | /** |
| 2385 | − * drm_atomic_helper_wait_for_dependencies - wait for required preceeding commits | |
| 2387 | + * drm_atomic_helper_wait_for_dependencies - wait for required preceding commits | |
| 2386 | 2388 | * @old_state: atomic state object with old state structures |
| 2387 | 2389 | * |
| 2388 | − * This function waits for all preceeding commits that touch the same CRTC as | |
| 2390 | + * This function waits for all preceding commits that touch the same CRTC as | |
| 2389 | 2391 | * @old_state to both be committed to the hardware (as signalled by |
| 2390 | 2392 | * drm_atomic_helper_commit_hw_done()) and executed by the hardware (as signalled |
| 2391 | 2393 | * by calling drm_crtc_send_vblank_event() on the &drm_crtc_state.event). |
| @@ -275,6 +275,20 @@ void __drm_atomic_helper_plane_state_reset(struct drm_plane_state *plane_state, · 274 unmodified lines | ||
| 275 | 275 | plane_state->normalized_zpos = val; |
| 276 | 276 | } |
| 277 | 277 | } |
| 278 | + | |
| 279 | + if (plane->hotspot_x_property) { | |
| 280 | + if (!drm_object_property_get_default_value(&plane->base, | |
| 281 | + plane->hotspot_x_property, | |
| 282 | + &val)) | |
| 283 | + plane_state->hotspot_x = val; | |
| 284 | + } | |
| 285 | + | |
| 286 | + if (plane->hotspot_y_property) { | |
| 287 | + if (!drm_object_property_get_default_value(&plane->base, | |
| 288 | + plane->hotspot_y_property, | |
| 289 | + &val)) | |
| 290 | + plane_state->hotspot_y = val; | |
| 291 | + } | |
| 278 | 292 | } |
| 279 | 293 | EXPORT_SYMBOL(__drm_atomic_helper_plane_state_reset); |
| 280 | 294 | |
| @@ -593,6 +593,22 @@ static int drm_atomic_plane_set_property(struct drm_plane *plane, · 592 unmodified lines | ||
| 593 | 593 | } else if (plane->funcs->atomic_set_property) { |
| 594 | 594 | return plane->funcs->atomic_set_property(plane, state, |
| 595 | 595 | property, val); |
| 596 | + } else if (property == plane->hotspot_x_property) { | |
| 597 | + if (plane->type != DRM_PLANE_TYPE_CURSOR) { | |
| 598 | + drm_dbg_atomic(plane->dev, | |
| 599 | + "[PLANE:%d:%s] is not a cursor plane: 0x%llx\n", | |
| 600 | + plane->base.id, plane->name, val); | |
| 601 | + return -EINVAL; | |
| 602 | + } | |
| 603 | + state->hotspot_x = val; | |
| 604 | + } else if (property == plane->hotspot_y_property) { | |
| 605 | + if (plane->type != DRM_PLANE_TYPE_CURSOR) { | |
| 606 | + drm_dbg_atomic(plane->dev, | |
| 607 | + "[PLANE:%d:%s] is not a cursor plane: 0x%llx\n", | |
| 608 | + plane->base.id, plane->name, val); | |
| 609 | + return -EINVAL; | |
| 610 | + } | |
| 611 | + state->hotspot_y = val; | |
| 596 | 612 | } else { |
| 597 | 613 | drm_dbg_atomic(plane->dev, |
| 598 | 614 | "[PLANE:%d:%s] unknown property [PROP:%d:%s]\n", |
| @@ -653,6 +669,10 @@ drm_atomic_plane_get_property(struct drm_plane *plane, · 54 unmodified lines | ||
| 653 | 669 | *val = state->scaling_filter; |
| 654 | 670 | } else if (plane->funcs->atomic_get_property) { |
| 655 | 671 | return plane->funcs->atomic_get_property(plane, state, property, val); |
| 672 | + } else if (property == plane->hotspot_x_property) { | |
| 673 | + *val = state->hotspot_x; | |
| 674 | + } else if (property == plane->hotspot_y_property) { | |
| 675 | + *val = state->hotspot_y; | |
| 656 | 676 | } else { |
| 657 | 677 | drm_dbg_atomic(dev, |
| 658 | 678 | "[PLANE:%d:%s] unknown property [PROP:%d:%s]\n", |
| @@ -1006,13 +1026,28 @@ out: · 347 unmodified lines | ||
| 1006 | 1026 | return ret; |
| 1007 | 1027 | } |
| 1008 | 1028 | |
| 1029 | +static int drm_atomic_check_prop_changes(int ret, uint64_t old_val, uint64_t prop_value, | |
| 1030 | + struct drm_property *prop) | |
| 1031 | +{ | |
| 1032 | + if (ret != 0 || old_val != prop_value) { | |
| 1033 | + drm_dbg_atomic(prop->dev, | |
| 1034 | + "[PROP:%d:%s] No prop can be changed during async flip\n", | |
| 1035 | + prop->base.id, prop->name); | |
| 1036 | + return -EINVAL; | |
| 1037 | + } | |
| 1038 | + | |
| 1039 | + return 0; | |
| 1040 | +} | |
| 1041 | + | |
| 1009 | 1042 | int drm_atomic_set_property(struct drm_atomic_state *state, |
| 1010 | 1043 | struct drm_file *file_priv, |
| 1011 | 1044 | struct drm_mode_object *obj, |
| 1012 | 1045 | struct drm_property *prop, |
| 1013 | − uint64_t prop_value) | |
| 1046 | + u64 prop_value, | |
| 1047 | + bool async_flip) | |
| 1014 | 1048 | { |
| 1015 | 1049 | struct drm_mode_object *ref; |
| 1050 | + u64 old_val; | |
| 1016 | 1051 | int ret; |
| 1017 | 1052 | |
| 1018 | 1053 | if (!drm_property_change_valid_get(prop, prop_value, &ref)) |
| @@ -1029,6 +1064,13 @@ int drm_atomic_set_property(struct drm_atomic_state *state, · 10 unmodified lines | ||
| 1029 | 1064 | break; |
| 1030 | 1065 | } |
| 1031 | 1066 | |
| 1067 | + if (async_flip) { | |
| 1068 | + ret = drm_atomic_connector_get_property(connector, connector_state, | |
| 1069 | + prop, &old_val); | |
| 1070 | + ret = drm_atomic_check_prop_changes(ret, old_val, prop_value, prop); | |
| 1071 | + break; | |
| 1072 | + } | |
| 1073 | + | |
| 1032 | 1074 | ret = drm_atomic_connector_set_property(connector, |
| 1033 | 1075 | connector_state, file_priv, |
| 1034 | 1076 | prop, prop_value); |
| @@ -1044,6 +1086,13 @@ int drm_atomic_set_property(struct drm_atomic_state *state, · 9 unmodified lines | ||
| 1044 | 1086 | break; |
| 1045 | 1087 | } |
| 1046 | 1088 | |
| 1089 | + if (async_flip) { | |
| 1090 | + ret = drm_atomic_crtc_get_property(crtc, crtc_state, | |
| 1091 | + prop, &old_val); | |
| 1092 | + ret = drm_atomic_check_prop_changes(ret, old_val, prop_value, prop); | |
| 1093 | + break; | |
| 1094 | + } | |
| 1095 | + | |
| 1047 | 1096 | ret = drm_atomic_crtc_set_property(crtc, |
| 1048 | 1097 | crtc_state, prop, prop_value); |
| 1049 | 1098 | break; |
| @@ -1051,6 +1100,7 @@ int drm_atomic_set_property(struct drm_atomic_state *state, · 1 unmodified line | ||
| 1051 | 1100 | case DRM_MODE_OBJECT_PLANE: { |
| 1052 | 1101 | struct drm_plane *plane = obj_to_plane(obj); |
| 1053 | 1102 | struct drm_plane_state *plane_state; |
| 1103 | + struct drm_mode_config *config = &plane->dev->mode_config; | |
| 1054 | 1104 | |
| 1055 | 1105 | plane_state = drm_atomic_get_plane_state(state, plane); |
| 1056 | 1106 | if (IS_ERR(plane_state)) { |
| @@ -1058,6 +1108,21 @@ int drm_atomic_set_property(struct drm_atomic_state *state, · 1 unmodified line | ||
| 1058 | 1108 | break; |
| 1059 | 1109 | } |
| 1060 | 1110 | |
| 1111 | + if (async_flip && prop != config->prop_fb_id) { | |
| 1112 | + ret = drm_atomic_plane_get_property(plane, plane_state, | |
| 1113 | + prop, &old_val); | |
| 1114 | + ret = drm_atomic_check_prop_changes(ret, old_val, prop_value, prop); | |
| 1115 | + break; | |
| 1116 | + } | |
| 1117 | + | |
| 1118 | + if (async_flip && plane_state->plane->type != DRM_PLANE_TYPE_PRIMARY) { | |
| 1119 | + drm_dbg_atomic(prop->dev, | |
| 1120 | + "[OBJECT:%d] Only primary planes can be changed during async flip\n", | |
| 1121 | + obj->id); | |
| 1122 | + ret = -EINVAL; | |
| 1123 | + break; | |
| 1124 | + } | |
| 1125 | + | |
| 1061 | 1126 | ret = drm_atomic_plane_set_property(plane, |
| 1062 | 1127 | plane_state, file_priv, |
| 1063 | 1128 | prop, prop_value); |
| @@ -1323,6 +1388,18 @@ static void complete_signaling(struct drm_device *dev, · 259 unmodified lines | ||
| 1323 | 1388 | kfree(fence_state); |
| 1324 | 1389 | } |
| 1325 | 1390 | |
| 1391 | +static void | |
| 1392 | +set_async_flip(struct drm_atomic_state *state) | |
| 1393 | +{ | |
| 1394 | + struct drm_crtc *crtc; | |
| 1395 | + struct drm_crtc_state *crtc_state; | |
| 1396 | + int i; | |
| 1397 | + | |
| 1398 | + for_each_new_crtc_in_state(state, crtc, crtc_state, i) { | |
| 1399 | + crtc_state->async_flip = true; | |
| 1400 | + } | |
| 1401 | +} | |
| 1402 | + | |
| 1326 | 1403 | int drm_mode_atomic_ioctl(struct drm_device *dev, |
| 1327 | 1404 | void *data, struct drm_file *file_priv) |
| 1328 | 1405 | { |
| @@ -1337,6 +1414,7 @@ int drm_mode_atomic_ioctl(struct drm_device *dev, · 8 unmodified lines | ||
| 1337 | 1414 | struct drm_out_fence_state *fence_state; |
| 1338 | 1415 | int ret = 0; |
| 1339 | 1416 | unsigned int i, j, num_fences; |
| 1417 | + bool async_flip = false; | |
| 1340 | 1418 | |
| 1341 | 1419 | /* disallow for drivers not supporting atomic: */ |
| 1342 | 1420 | if (!drm_core_check_feature(dev, DRIVER_ATOMIC)) |
| @@ -1363,9 +1441,13 @@ int drm_mode_atomic_ioctl(struct drm_device *dev, · 20 unmodified lines | ||
| 1363 | 1441 | } |
| 1364 | 1442 | |
| 1365 | 1443 | if (arg->flags & DRM_MODE_PAGE_FLIP_ASYNC) { |
| 1366 | − drm_dbg_atomic(dev, | |
| 1367 | − "commit failed: invalid flag DRM_MODE_PAGE_FLIP_ASYNC\n"); | |
| 1368 | − return -EINVAL; | |
| 1444 | + if (!dev->mode_config.async_page_flip) { | |
| 1445 | + drm_dbg_atomic(dev, | |
| 1446 | + "commit failed: DRM_MODE_PAGE_FLIP_ASYNC not supported\n"); | |
| 1447 | + return -EINVAL; | |
| 1448 | + } | |
| 1449 | + | |
| 1450 | + async_flip = true; | |
| 1369 | 1451 | } |
| 1370 | 1452 | |
| 1371 | 1453 | /* can't test and expect an event at the same time. */ |
| @@ -1450,8 +1532,8 @@ retry: · 78 unmodified lines | ||
| 1450 | 1532 | goto out; |
| 1451 | 1533 | } |
| 1452 | 1534 | |
| 1453 | − ret = drm_atomic_set_property(state, file_priv, | |
| 1454 | − obj, prop, prop_value); | |
| 1535 | + ret = drm_atomic_set_property(state, file_priv, obj, | |
| 1536 | + prop, prop_value, async_flip); | |
| 1455 | 1537 | if (ret) { |
| 1456 | 1538 | drm_mode_object_put(obj); |
| 1457 | 1539 | goto out; |
| @@ -1468,6 +1550,9 @@ retry: · 10 unmodified lines | ||
| 1468 | 1550 | if (ret) |
| 1469 | 1551 | goto out; |
| 1470 | 1552 | |
| 1553 | + if (arg->flags & DRM_MODE_PAGE_FLIP_ASYNC) | |
| 1554 | + set_async_flip(state); | |
| 1555 | + | |
| 1471 | 1556 | if (arg->flags & DRM_MODE_ATOMIC_TEST_ONLY) { |
| 1472 | 1557 | ret = drm_atomic_check_only(state); |
| 1473 | 1558 | } else if (arg->flags & DRM_MODE_ATOMIC_NONBLOCK) { |
| @@ -37,13 +37,12 @@ · 36 unmodified lines | ||
| 37 | 37 | #include <drm/drm_print.h> |
| 38 | 38 | |
| 39 | 39 | #include "drm_internal.h" |
| 40 | −#include "drm_legacy.h" | |
| 41 | 40 | |
| 42 | 41 | /** |
| 43 | 42 | * DOC: master and authentication |
| 44 | 43 | * |
| 45 | 44 | * &struct drm_master is used to track groups of clients with open |
| 46 | − * primary/legacy device nodes. For every &struct drm_file which has had at | |
| 45 | + * primary device nodes. For every &struct drm_file which has had at | |
| 47 | 46 | * least once successfully became the device master (either through the |
| 48 | 47 | * SET_MASTER IOCTL, or implicitly through opening the primary device node when |
| 49 | 48 | * no one else is the current master that time) there exists one &drm_master. |
| @@ -139,7 +138,6 @@ struct drm_master *drm_master_create(struct drm_device *dev) · 89 unmodified lines | ||
| 139 | 138 | return NULL; |
| 140 | 139 | |
| 141 | 140 | kref_init(&master->refcount); |
| 142 | − drm_master_legacy_init(master); | |
| 143 | 141 | idr_init_base(&master->magic_map, 1); |
| 144 | 142 | master->dev = dev; |
| 145 | 143 | |
| @@ -365,8 +363,6 @@ void drm_master_release(struct drm_file *file_priv) · 219 unmodified lines | ||
| 365 | 363 | if (!drm_is_current_master_locked(file_priv)) |
| 366 | 364 | goto out; |
| 367 | 365 | |
| 368 | − drm_legacy_lock_master_cleanup(dev, master); | |
| 369 | − | |
| 370 | 366 | if (dev->master == file_priv->master) |
| 371 | 367 | drm_drop_master(dev, file_priv); |
| 372 | 368 | out: |
| @@ -429,8 +425,6 @@ static void drm_master_destroy(struct kref *kref) · 56 unmodified lines | ||
| 429 | 425 | if (drm_core_check_feature(dev, DRIVER_MODESET)) |
| 430 | 426 | drm_lease_destroy(master); |
| 431 | 427 | |
| 432 | − drm_legacy_master_rmmaps(dev, master); | |
| 433 | − | |
| 434 | 428 | idr_destroy(&master->magic_map); |
| 435 | 429 | idr_destroy(&master->leases); |
| 436 | 430 | idr_destroy(&master->lessee_idr); |
| @@ -1347,50 +1347,6 @@ struct drm_bridge *of_drm_find_bridge(struct device_node *np) · 1346 unmodified lines | ||
| 1347 | 1347 | EXPORT_SYMBOL(of_drm_find_bridge); |
| 1348 | 1348 | #endif |
| 1349 | 1349 | |
| 1350 | −#ifdef CONFIG_DEBUG_FS | |
| 1351 | −static int drm_bridge_chains_info(struct seq_file *m, void *data) | |
| 1352 | −{ | |
| 1353 | − struct drm_debugfs_entry *entry = m->private; | |
| 1354 | − struct drm_device *dev = entry->dev; | |
| 1355 | − struct drm_printer p = drm_seq_file_printer(m); | |
| 1356 | − struct drm_mode_config *config = &dev->mode_config; | |
| 1357 | − struct drm_encoder *encoder; | |
| 1358 | − unsigned int bridge_idx = 0; | |
| 1359 | − | |
| 1360 | − list_for_each_entry(encoder, &config->encoder_list, head) { | |
| 1361 | − struct drm_bridge *bridge; | |
| 1362 | − | |
| 1363 | − drm_printf(&p, "encoder[%u]\n", encoder->base.id); | |
| 1364 | − | |
| 1365 | − drm_for_each_bridge_in_chain(encoder, bridge) { | |
| 1366 | − drm_printf(&p, "\tbridge[%u] type: %u, ops: %#x", | |
| 1367 | − bridge_idx, bridge->type, bridge->ops); | |
| 1368 | − | |
| 1369 | −#ifdef CONFIG_OF | |
| 1370 | − if (bridge->of_node) | |
| 1371 | − drm_printf(&p, ", OF: %pOFfc", bridge->of_node); | |
| 1372 | −#endif | |
| 1373 | − | |
| 1374 | − drm_printf(&p, "\n"); | |
| 1375 | − | |
| 1376 | − bridge_idx++; | |
| 1377 | − } | |
| 1378 | − } | |
| 1379 | − | |
| 1380 | − return 0; | |
| 1381 | −} | |
| 1382 | − | |
| 1383 | −static const struct drm_debugfs_info drm_bridge_debugfs_list[] = { | |
| 1384 | − { "bridge_chains", drm_bridge_chains_info, 0 }, | |
| 1385 | −}; | |
| 1386 | − | |
| 1387 | −void drm_bridge_debugfs_init(struct drm_device *dev) | |
| 1388 | −{ | |
| 1389 | − drm_debugfs_add_files(dev, drm_bridge_debugfs_list, | |
| 1390 | − ARRAY_SIZE(drm_bridge_debugfs_list)); | |
| 1391 | −} | |
| 1392 | −#endif | |
| 1393 | − | |
| 1394 | 1350 | MODULE_AUTHOR("Ajay Kumar <ajaykumar.rs@samsung.com>"); |
| 1395 | 1351 | MODULE_DESCRIPTION("DRM bridge infrastructure"); |
| 1396 | 1352 | MODULE_LICENSE("GPL and additional rights"); |
| @@ -198,12 +198,6 @@ static void drm_bridge_connector_destroy(struct drm_connector *connector) · 197 unmodified lines | ||
| 198 | 198 | struct drm_bridge_connector *bridge_connector = |
| 199 | 199 | to_drm_bridge_connector(connector); |
| 200 | 200 | |
| 201 | − if (bridge_connector->bridge_hpd) { | |
| 202 | − struct drm_bridge *hpd = bridge_connector->bridge_hpd; | |
| 203 | − | |
| 204 | − drm_bridge_hpd_disable(hpd); | |
| 205 | − } | |
| 206 | − | |
| 207 | 201 | drm_connector_unregister(connector); |
| 208 | 202 | drm_connector_cleanup(connector); |
| 209 | 203 | |
| @@ -1,1627 +0,0 @@ | ||
| 1 | −/* | |
| 2 | − * Legacy: Generic DRM Buffer Management | |
| 3 | − * | |
| 4 | − * Copyright 1999, 2000 Precision Insight, Inc., Cedar Park, Texas. | |
| 5 | − * Copyright 2000 VA Linux Systems, Inc., Sunnyvale, California. | |
| 6 | − * All Rights Reserved. | |
| 7 | − * | |
| 8 | − * Author: Rickard E. (Rik) Faith <faith@valinux.com> | |
| 9 | − * Author: Gareth Hughes <gareth@valinux.com> | |
| 10 | − * | |
| 11 | − * Permission is hereby granted, free of charge, to any person obtaining a | |
| 12 | − * copy of this software and associated documentation files (the "Software"), | |
| 13 | − * to deal in the Software without restriction, including without limitation | |
| 14 | − * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 15 | − * and/or sell copies of the Software, and to permit persons to whom the | |
| 16 | − * Software is furnished to do so, subject to the following conditions: | |
| 17 | − * | |
| 18 | − * The above copyright notice and this permission notice (including the next | |
| 19 | − * paragraph) shall be included in all copies or substantial portions of the | |
| 20 | − * Software. | |
| 21 | − * | |
| 22 | − * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 23 | − * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 24 | − * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 25 | − * VA LINUX SYSTEMS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 26 | − * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 27 | − * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 28 | − * OTHER DEALINGS IN THE SOFTWARE. | |
| 29 | − */ | |
| 30 | − | |
| 31 | −#include <linux/export.h> | |
| 32 | −#include <linux/log2.h> | |
| 33 | −#include <linux/mm.h> | |
| 34 | −#include <linux/mman.h> | |
| 35 | −#include <linux/nospec.h> | |
| 36 | −#include <linux/pci.h> | |
| 37 | −#include <linux/slab.h> | |
| 38 | −#include <linux/uaccess.h> | |
| 39 | −#include <linux/vmalloc.h> | |
| 40 | − | |
| 41 | −#include <asm/shmparam.h> | |
| 42 | − | |
| 43 | −#include <drm/drm_device.h> | |
| 44 | −#include <drm/drm_drv.h> | |
| 45 | −#include <drm/drm_file.h> | |
| 46 | −#include <drm/drm_print.h> | |
| 47 | − | |
| 48 | −#include "drm_legacy.h" | |
| 49 | − | |
| 50 | − | |
| 51 | −static struct drm_map_list *drm_find_matching_map(struct drm_device *dev, | |
| 52 | − struct drm_local_map *map) | |
| 53 | −{ | |
| 54 | − struct drm_map_list *entry; | |
| 55 | − | |
| 56 | − list_for_each_entry(entry, &dev->maplist, head) { | |
| 57 | − /* | |
| 58 | − * Because the kernel-userspace ABI is fixed at a 32-bit offset | |
| 59 | − * while PCI resources may live above that, we only compare the | |
| 60 | − * lower 32 bits of the map offset for maps of type | |
| 61 | − * _DRM_FRAMEBUFFER or _DRM_REGISTERS. | |
| 62 | − * It is assumed that if a driver have more than one resource | |
| 63 | − * of each type, the lower 32 bits are different. | |
| 64 | − */ | |
| 65 | − if (!entry->map || | |
| 66 | − map->type != entry->map->type || | |
| 67 | − entry->master != dev->master) | |
| 68 | − continue; | |
| 69 | − switch (map->type) { | |
| 70 | − case _DRM_SHM: | |
| 71 | − if (map->flags != _DRM_CONTAINS_LOCK) | |
| 72 | − break; | |
| 73 | − return entry; | |
| 74 | − case _DRM_REGISTERS: | |
| 75 | − case _DRM_FRAME_BUFFER: | |
| 76 | − if ((entry->map->offset & 0xffffffff) == | |
| 77 | − (map->offset & 0xffffffff)) | |
| 78 | − return entry; | |
| 79 | − break; | |
| 80 | − default: /* Make gcc happy */ | |
| 81 | − break; | |
| 82 | − } | |
| 83 | − if (entry->map->offset == map->offset) | |
| 84 | − return entry; | |
| 85 | − } | |
| 86 | − | |
| 87 | − return NULL; | |
| 88 | −} | |
| 89 | − | |
| 90 | −static int drm_map_handle(struct drm_device *dev, struct drm_hash_item *hash, | |
| 91 | − unsigned long user_token, int hashed_handle, int shm) | |
| 92 | −{ | |
| 93 | − int use_hashed_handle, shift; | |
| 94 | − unsigned long add; | |
| 95 | − | |
| 96 | −#if (BITS_PER_LONG == 64) | |
| 97 | − use_hashed_handle = ((user_token & 0xFFFFFFFF00000000UL) || hashed_handle); | |
| 98 | −#elif (BITS_PER_LONG == 32) | |
| 99 | − use_hashed_handle = hashed_handle; | |
| 100 | −#else | |
| 101 | −#error Unsupported long size. Neither 64 nor 32 bits. | |
| 102 | −#endif | |
| 103 | − | |
| 104 | − if (!use_hashed_handle) { | |
| 105 | − int ret; | |
| 106 | − | |
| 107 | − hash->key = user_token >> PAGE_SHIFT; | |
| 108 | − ret = drm_ht_insert_item(&dev->map_hash, hash); | |
| 109 | − if (ret != -EINVAL) | |
| 110 | − return ret; | |
| 111 | − } | |
| 112 | − | |
| 113 | − shift = 0; | |
| 114 | − add = DRM_MAP_HASH_OFFSET >> PAGE_SHIFT; | |
| 115 | − if (shm && (SHMLBA > PAGE_SIZE)) { | |
| 116 | − int bits = ilog2(SHMLBA >> PAGE_SHIFT) + 1; | |
| 117 | − | |
| 118 | − /* For shared memory, we have to preserve the SHMLBA | |
| 119 | − * bits of the eventual vma->vm_pgoff value during | |
| 120 | − * mmap(). Otherwise we run into cache aliasing problems | |
| 121 | − * on some platforms. On these platforms, the pgoff of | |
| 122 | − * a mmap() request is used to pick a suitable virtual | |
| 123 | − * address for the mmap() region such that it will not | |
| 124 | − * cause cache aliasing problems. | |
| 125 | − * | |
| 126 | − * Therefore, make sure the SHMLBA relevant bits of the | |
| 127 | − * hash value we use are equal to those in the original | |
| 128 | − * kernel virtual address. | |
| 129 | − */ | |
| 130 | − shift = bits; | |
| 131 | − add |= ((user_token >> PAGE_SHIFT) & ((1UL << bits) - 1UL)); | |
| 132 | − } | |
| 133 | − | |
| 134 | − return drm_ht_just_insert_please(&dev->map_hash, hash, | |
| 135 | − user_token, 32 - PAGE_SHIFT - 3, | |
| 136 | − shift, add); | |
| 137 | −} | |
| 138 | − | |
| 139 | −/* | |
| 140 | − * Core function to create a range of memory available for mapping by a | |
| 141 | − * non-root process. | |
| 142 | − * | |
| 143 | − * Adjusts the memory offset to its absolute value according to the mapping | |
| 144 | − * type. Adds the map to the map list drm_device::maplist. Adds MTRR's where | |
| 145 | − * applicable and if supported by the kernel. | |
| 146 | − */ | |
| 147 | −static int drm_addmap_core(struct drm_device *dev, resource_size_t offset, | |
| 148 | − unsigned int size, enum drm_map_type type, | |
| 149 | − enum drm_map_flags flags, | |
| 150 | − struct drm_map_list **maplist) | |
| 151 | −{ | |
| 152 | − struct drm_local_map *map; | |
| 153 | − struct drm_map_list *list; | |
| 154 | − unsigned long user_token; | |
| 155 | − int ret; | |
| 156 | − | |
| 157 | − map = kmalloc(sizeof(*map), GFP_KERNEL); | |
| 158 | − if (!map) | |
| 159 | − return -ENOMEM; | |
| 160 | − | |
| 161 | − map->offset = offset; | |
| 162 | − map->size = size; | |
| 163 | − map->flags = flags; | |
| 164 | − map->type = type; | |
| 165 | − | |
| 166 | − /* Only allow shared memory to be removable since we only keep enough | |
| 167 | − * book keeping information about shared memory to allow for removal | |
| 168 | − * when processes fork. | |
| 169 | − */ | |
| 170 | − if ((map->flags & _DRM_REMOVABLE) && map->type != _DRM_SHM) { | |
| 171 | − kfree(map); | |
| 172 | − return -EINVAL; | |
| 173 | − } | |
| 174 | − DRM_DEBUG("offset = 0x%08llx, size = 0x%08lx, type = %d\n", | |
| 175 | − (unsigned long long)map->offset, map->size, map->type); | |
| 176 | − | |
| 177 | − /* page-align _DRM_SHM maps. They are allocated here so there is no security | |
| 178 | − * hole created by that and it works around various broken drivers that use | |
| 179 | − * a non-aligned quantity to map the SAREA. --BenH | |
| 180 | − */ | |
| 181 | − if (map->type == _DRM_SHM) | |
| 182 | − map->size = PAGE_ALIGN(map->size); | |
| 183 | − | |
| 184 | − if ((map->offset & (~(resource_size_t)PAGE_MASK)) || (map->size & (~PAGE_MASK))) { | |
| 185 | − kfree(map); | |
| 186 | − return -EINVAL; | |
| 187 | − } | |
| 188 | − map->mtrr = -1; | |
| 189 | − map->handle = NULL; | |
| 190 | − | |
| 191 | − switch (map->type) { | |
| 192 | − case _DRM_REGISTERS: | |
| 193 | − case _DRM_FRAME_BUFFER: | |
| 194 | −#if !defined(__sparc__) && !defined(__alpha__) && !defined(__ia64__) && !defined(__powerpc64__) && !defined(__x86_64__) && !defined(__arm__) | |
| 195 | − if (map->offset + (map->size-1) < map->offset || | |
| 196 | − map->offset < virt_to_phys(high_memory)) { | |
| 197 | − kfree(map); | |
| 198 | − return -EINVAL; | |
| 199 | − } | |
| 200 | −#endif | |
| 201 | − /* Some drivers preinitialize some maps, without the X Server | |
| 202 | − * needing to be aware of it. Therefore, we just return success | |
| 203 | − * when the server tries to create a duplicate map. | |
| 204 | − */ | |
| 205 | − list = drm_find_matching_map(dev, map); | |
| 206 | − if (list != NULL) { | |
| 207 | − if (list->map->size != map->size) { | |
| 208 | − DRM_DEBUG("Matching maps of type %d with " | |
| 209 | − "mismatched sizes, (%ld vs %ld)\n", | |
| 210 | − map->type, map->size, | |
| 211 | − list->map->size); | |
| 212 | − list->map->size = map->size; | |
| 213 | − } | |
| 214 | − | |
| 215 | − kfree(map); | |
| 216 | − *maplist = list; | |
| 217 | − return 0; | |
| 218 | − } | |
| 219 | − | |
| 220 | − if (map->type == _DRM_FRAME_BUFFER || | |
| 221 | − (map->flags & _DRM_WRITE_COMBINING)) { | |
| 222 | − map->mtrr = | |
| 223 | − arch_phys_wc_add(map->offset, map->size); | |
| 224 | − } | |
| 225 | − if (map->type == _DRM_REGISTERS) { | |
| 226 | − if (map->flags & _DRM_WRITE_COMBINING) | |
| 227 | − map->handle = ioremap_wc(map->offset, | |
| 228 | − map->size); | |
| 229 | − else | |
| 230 | − map->handle = ioremap(map->offset, map->size); | |
| 231 | − if (!map->handle) { | |
| 232 | − kfree(map); | |
| 233 | − return -ENOMEM; | |
| 234 | − } | |
| 235 | − } | |
| 236 | − | |
| 237 | − break; | |
| 238 | − case _DRM_SHM: | |
| 239 | − list = drm_find_matching_map(dev, map); | |
| 240 | − if (list != NULL) { | |
| 241 | − if (list->map->size != map->size) { | |
| 242 | − DRM_DEBUG("Matching maps of type %d with " | |
| 243 | − "mismatched sizes, (%ld vs %ld)\n", | |
| 244 | − map->type, map->size, list->map->size); | |
| 245 | − list->map->size = map->size; | |
| 246 | − } | |
| 247 | − | |
| 248 | − kfree(map); | |
| 249 | − *maplist = list; | |
| 250 | − return 0; | |
| 251 | − } | |
| 252 | − map->handle = vmalloc_user(map->size); | |
| 253 | − DRM_DEBUG("%lu %d %p\n", | |
| 254 | − map->size, order_base_2(map->size), map->handle); | |
| 255 | − if (!map->handle) { | |
| 256 | − kfree(map); | |
| 257 | − return -ENOMEM; | |
| 258 | − } | |
| 259 | − map->offset = (unsigned long)map->handle; | |
| 260 | − if (map->flags & _DRM_CONTAINS_LOCK) { | |
| 261 | − /* Prevent a 2nd X Server from creating a 2nd lock */ | |
| 262 | − if (dev->master->lock.hw_lock != NULL) { | |
| 263 | − vfree(map->handle); | |
| 264 | − kfree(map); | |
| 265 | − return -EBUSY; | |
| 266 | − } | |
| 267 | − dev->sigdata.lock = dev->master->lock.hw_lock = map->handle; /* Pointer to lock */ | |
| 268 | − } | |
| 269 | − break; | |
| 270 | − case _DRM_AGP: { | |
| 271 | − struct drm_agp_mem *entry; | |
| 272 | − int valid = 0; | |
| 273 | − | |
| 274 | − if (!dev->agp) { | |
| 275 | − kfree(map); | |
| 276 | − return -EINVAL; | |
| 277 | − } | |
| 278 | −#ifdef __alpha__ | |
| 279 | − map->offset += dev->hose->mem_space->start; | |
| 280 | −#endif | |
| 281 | − /* In some cases (i810 driver), user space may have already | |
| 282 | − * added the AGP base itself, because dev->agp->base previously | |
| 283 | − * only got set during AGP enable. So, only add the base | |
| 284 | − * address if the map's offset isn't already within the | |
| 285 | − * aperture. | |
| 286 | − */ | |
| 287 | − if (map->offset < dev->agp->base || | |
| 288 | − map->offset > dev->agp->base + | |
| 289 | − dev->agp->agp_info.aper_size * 1024 * 1024 - 1) { | |
| 290 | − map->offset += dev->agp->base; | |
| 291 | − } | |
| 292 | − map->mtrr = dev->agp->agp_mtrr; /* for getmap */ | |
| 293 | − | |
| 294 | − /* This assumes the DRM is in total control of AGP space. | |
| 295 | − * It's not always the case as AGP can be in the control | |
| 296 | − * of user space (i.e. i810 driver). So this loop will get | |
| 297 | − * skipped and we double check that dev->agp->memory is | |
| 298 | − * actually set as well as being invalid before EPERM'ing | |
| 299 | − */ | |
| 300 | − list_for_each_entry(entry, &dev->agp->memory, head) { | |
| 301 | − if ((map->offset >= entry->bound) && | |
| 302 | − (map->offset + map->size <= entry->bound + entry->pages * PAGE_SIZE)) { | |
| 303 | − valid = 1; | |
| 304 | − break; | |
| 305 | − } | |
| 306 | − } | |
| 307 | − if (!list_empty(&dev->agp->memory) && !valid) { | |
| 308 | − kfree(map); | |
| 309 | − return -EPERM; | |
| 310 | − } | |
| 311 | − DRM_DEBUG("AGP offset = 0x%08llx, size = 0x%08lx\n", | |
| 312 | − (unsigned long long)map->offset, map->size); | |
| 313 | − | |
| 314 | − break; | |
| 315 | − } | |
| 316 | − case _DRM_SCATTER_GATHER: | |
| 317 | − if (!dev->sg) { | |
| 318 | − kfree(map); | |
| 319 | − return -EINVAL; | |
| 320 | − } | |
| 321 | − map->offset += (unsigned long)dev->sg->virtual; | |
| 322 | − break; | |
| 323 | − case _DRM_CONSISTENT: | |
| 324 | − /* dma_addr_t is 64bit on i386 with CONFIG_HIGHMEM64G, | |
| 325 | − * As we're limiting the address to 2^32-1 (or less), | |
| 326 | − * casting it down to 32 bits is no problem, but we | |
| 327 | − * need to point to a 64bit variable first. | |
| 328 | − */ | |
| 329 | − map->handle = dma_alloc_coherent(dev->dev, | |
| 330 | − map->size, | |
| 331 | − &map->offset, | |
| 332 | − GFP_KERNEL); | |
| 333 | − if (!map->handle) { | |
| 334 | − kfree(map); | |
| 335 | − return -ENOMEM; | |
| 336 | − } | |
| 337 | − break; | |
| 338 | − default: | |
| 339 | − kfree(map); | |
| 340 | − return -EINVAL; | |
| 341 | − } | |
| 342 | − | |
| 343 | − list = kzalloc(sizeof(*list), GFP_KERNEL); | |
| 344 | − if (!list) { | |
| 345 | − if (map->type == _DRM_REGISTERS) | |
| 346 | − iounmap(map->handle); | |
| 347 | − kfree(map); | |
| 348 | − return -EINVAL; | |
| 349 | − } | |
| 350 | − list->map = map; | |
| 351 | − | |
| 352 | − mutex_lock(&dev->struct_mutex); | |
| 353 | − list_add(&list->head, &dev->maplist); | |
| 354 | − | |
| 355 | − /* Assign a 32-bit handle */ | |
| 356 | − /* We do it here so that dev->struct_mutex protects the increment */ | |
| 357 | − user_token = (map->type == _DRM_SHM) ? (unsigned long)map->handle : | |
| 358 | − map->offset; | |
| 359 | − ret = drm_map_handle(dev, &list->hash, user_token, 0, | |
| 360 | − (map->type == _DRM_SHM)); | |
| 361 | − if (ret) { | |
| 362 | − if (map->type == _DRM_REGISTERS) | |
| 363 | − iounmap(map->handle); | |
| 364 | − kfree(map); | |
| 365 | − kfree(list); | |
| 366 | − mutex_unlock(&dev->struct_mutex); | |
| 367 | − return ret; | |
| 368 | − } | |
| 369 | − | |
| 370 | − list->user_token = list->hash.key << PAGE_SHIFT; | |
| 371 | − mutex_unlock(&dev->struct_mutex); | |
| 372 | − | |
| 373 | − if (!(map->flags & _DRM_DRIVER)) | |
| 374 | − list->master = dev->master; | |
| 375 | − *maplist = list; | |
| 376 | − return 0; | |
| 377 | −} | |
| 378 | − | |
| 379 | −int drm_legacy_addmap(struct drm_device *dev, resource_size_t offset, | |
| 380 | − unsigned int size, enum drm_map_type type, | |
| 381 | − enum drm_map_flags flags, struct drm_local_map **map_ptr) | |
| 382 | −{ | |
| 383 | − struct drm_map_list *list; | |
| 384 | − int rc; | |
| 385 | − | |
| 386 | − rc = drm_addmap_core(dev, offset, size, type, flags, &list); | |
| 387 | − if (!rc) | |
| 388 | − *map_ptr = list->map; | |
| 389 | − return rc; | |
| 390 | −} | |
| 391 | −EXPORT_SYMBOL(drm_legacy_addmap); | |
| 392 | − | |
| 393 | −struct drm_local_map *drm_legacy_findmap(struct drm_device *dev, | |
| 394 | − unsigned int token) | |
| 395 | −{ | |
| 396 | − struct drm_map_list *_entry; | |
| 397 | − | |
| 398 | − list_for_each_entry(_entry, &dev->maplist, head) | |
| 399 | − if (_entry->user_token == token) | |
| 400 | − return _entry->map; | |
| 401 | − return NULL; | |
| 402 | −} | |
| 403 | −EXPORT_SYMBOL(drm_legacy_findmap); | |
| 404 | − | |
| 405 | −/* | |
| 406 | − * Ioctl to specify a range of memory that is available for mapping by a | |
| 407 | − * non-root process. | |
| 408 | − * | |
| 409 | − * \param inode device inode. | |
| 410 | − * \param file_priv DRM file private. | |
| 411 | − * \param cmd command. | |
| 412 | − * \param arg pointer to a drm_map structure. | |
| 413 | − * \return zero on success or a negative value on error. | |
| 414 | − * | |
| 415 | − */ | |
| 416 | −int drm_legacy_addmap_ioctl(struct drm_device *dev, void *data, | |
| 417 | − struct drm_file *file_priv) | |
| 418 | −{ | |
| 419 | − struct drm_map *map = data; | |
| 420 | − struct drm_map_list *maplist; | |
| 421 | − int err; | |
| 422 | − | |
| 423 | − if (!(capable(CAP_SYS_ADMIN) || map->type == _DRM_AGP || map->type == _DRM_SHM)) | |
| 424 | − return -EPERM; | |
| 425 | − | |
| 426 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 427 | − return -EOPNOTSUPP; | |
| 428 | − | |
| 429 | − err = drm_addmap_core(dev, map->offset, map->size, map->type, | |
| 430 | − map->flags, &maplist); | |
| 431 | − | |
| 432 | − if (err) | |
| 433 | − return err; | |
| 434 | − | |
| 435 | − /* avoid a warning on 64-bit, this casting isn't very nice, but the API is set so too late */ | |
| 436 | − map->handle = (void *)(unsigned long)maplist->user_token; | |
| 437 | − | |
| 438 | − /* | |
| 439 | − * It appears that there are no users of this value whatsoever -- | |
| 440 | − * drmAddMap just discards it. Let's not encourage its use. | |
| 441 | − * (Keeping drm_addmap_core's returned mtrr value would be wrong -- | |
| 442 | − * it's not a real mtrr index anymore.) | |
| 443 | − */ | |
| 444 | − map->mtrr = -1; | |
| 445 | − | |
| 446 | − return 0; | |
| 447 | −} | |
| 448 | − | |
| 449 | −/* | |
| 450 | − * Get a mapping information. | |
| 451 | − * | |
| 452 | − * \param inode device inode. | |
| 453 | − * \param file_priv DRM file private. | |
| 454 | − * \param cmd command. | |
| 455 | − * \param arg user argument, pointing to a drm_map structure. | |
| 456 | − * | |
| 457 | − * \return zero on success or a negative number on failure. | |
| 458 | − * | |
| 459 | − * Searches for the mapping with the specified offset and copies its information | |
| 460 | − * into userspace | |
| 461 | − */ | |
| 462 | −int drm_legacy_getmap_ioctl(struct drm_device *dev, void *data, | |
| 463 | − struct drm_file *file_priv) | |
| 464 | −{ | |
| 465 | − struct drm_map *map = data; | |
| 466 | − struct drm_map_list *r_list = NULL; | |
| 467 | − struct list_head *list; | |
| 468 | − int idx; | |
| 469 | − int i; | |
| 470 | − | |
| 471 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 472 | − return -EOPNOTSUPP; | |
| 473 | − | |
| 474 | − idx = map->offset; | |
| 475 | − if (idx < 0) | |
| 476 | − return -EINVAL; | |
| 477 | − | |
| 478 | − i = 0; | |
| 479 | − mutex_lock(&dev->struct_mutex); | |
| 480 | − list_for_each(list, &dev->maplist) { | |
| 481 | − if (i == idx) { | |
| 482 | − r_list = list_entry(list, struct drm_map_list, head); | |
| 483 | − break; | |
| 484 | − } | |
| 485 | − i++; | |
| 486 | − } | |
| 487 | − if (!r_list || !r_list->map) { | |
| 488 | − mutex_unlock(&dev->struct_mutex); | |
| 489 | − return -EINVAL; | |
| 490 | − } | |
| 491 | − | |
| 492 | − map->offset = r_list->map->offset; | |
| 493 | − map->size = r_list->map->size; | |
| 494 | − map->type = r_list->map->type; | |
| 495 | − map->flags = r_list->map->flags; | |
| 496 | − map->handle = (void *)(unsigned long) r_list->user_token; | |
| 497 | − map->mtrr = arch_phys_wc_index(r_list->map->mtrr); | |
| 498 | − | |
| 499 | − mutex_unlock(&dev->struct_mutex); | |
| 500 | − | |
| 501 | − return 0; | |
| 502 | −} | |
| 503 | − | |
| 504 | −/* | |
| 505 | − * Remove a map private from list and deallocate resources if the mapping | |
| 506 | − * isn't in use. | |
| 507 | − * | |
| 508 | − * Searches the map on drm_device::maplist, removes it from the list, see if | |
| 509 | − * it's being used, and free any associated resource (such as MTRR's) if it's not | |
| 510 | − * being on use. | |
| 511 | − * | |
| 512 | − * \sa drm_legacy_addmap | |
| 513 | − */ | |
| 514 | −int drm_legacy_rmmap_locked(struct drm_device *dev, struct drm_local_map *map) | |
| 515 | −{ | |
| 516 | − struct drm_map_list *r_list = NULL, *list_t; | |
| 517 | − int found = 0; | |
| 518 | − struct drm_master *master; | |
| 519 | − | |
| 520 | − /* Find the list entry for the map and remove it */ | |
| 521 | − list_for_each_entry_safe(r_list, list_t, &dev->maplist, head) { | |
| 522 | − if (r_list->map == map) { | |
| 523 | − master = r_list->master; | |
| 524 | − list_del(&r_list->head); | |
| 525 | − drm_ht_remove_key(&dev->map_hash, | |
| 526 | − r_list->user_token >> PAGE_SHIFT); | |
| 527 | − kfree(r_list); | |
| 528 | − found = 1; | |
| 529 | − break; | |
| 530 | − } | |
| 531 | − } | |
| 532 | − | |
| 533 | − if (!found) | |
| 534 | − return -EINVAL; | |
| 535 | − | |
| 536 | − switch (map->type) { | |
| 537 | − case _DRM_REGISTERS: | |
| 538 | − iounmap(map->handle); | |
| 539 | − fallthrough; | |
| 540 | − case _DRM_FRAME_BUFFER: | |
| 541 | − arch_phys_wc_del(map->mtrr); | |
| 542 | − break; | |
| 543 | − case _DRM_SHM: | |
| 544 | − vfree(map->handle); | |
| 545 | − if (master) { | |
| 546 | − if (dev->sigdata.lock == master->lock.hw_lock) | |
| 547 | − dev->sigdata.lock = NULL; | |
| 548 | − master->lock.hw_lock = NULL; /* SHM removed */ | |
| 549 | − master->lock.file_priv = NULL; | |
| 550 | − wake_up_interruptible_all(&master->lock.lock_queue); | |
| 551 | − } | |
| 552 | − break; | |
| 553 | − case _DRM_AGP: | |
| 554 | − case _DRM_SCATTER_GATHER: | |
| 555 | − break; | |
| 556 | − case _DRM_CONSISTENT: | |
| 557 | − dma_free_coherent(dev->dev, | |
| 558 | − map->size, | |
| 559 | − map->handle, | |
| 560 | − map->offset); | |
| 561 | − break; | |
| 562 | − } | |
| 563 | − kfree(map); | |
| 564 | − | |
| 565 | − return 0; | |
| 566 | −} | |
| 567 | −EXPORT_SYMBOL(drm_legacy_rmmap_locked); | |
| 568 | − | |
| 569 | −void drm_legacy_rmmap(struct drm_device *dev, struct drm_local_map *map) | |
| 570 | −{ | |
| 571 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 572 | − return; | |
| 573 | − | |
| 574 | − mutex_lock(&dev->struct_mutex); | |
| 575 | − drm_legacy_rmmap_locked(dev, map); | |
| 576 | − mutex_unlock(&dev->struct_mutex); | |
| 577 | −} | |
| 578 | −EXPORT_SYMBOL(drm_legacy_rmmap); | |
| 579 | − | |
| 580 | −void drm_legacy_master_rmmaps(struct drm_device *dev, struct drm_master *master) | |
| 581 | −{ | |
| 582 | − struct drm_map_list *r_list, *list_temp; | |
| 583 | − | |
| 584 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 585 | − return; | |
| 586 | − | |
| 587 | − mutex_lock(&dev->struct_mutex); | |
| 588 | − list_for_each_entry_safe(r_list, list_temp, &dev->maplist, head) { | |
| 589 | − if (r_list->master == master) { | |
| 590 | − drm_legacy_rmmap_locked(dev, r_list->map); | |
| 591 | − r_list = NULL; | |
| 592 | − } | |
| 593 | − } | |
| 594 | − mutex_unlock(&dev->struct_mutex); | |
| 595 | −} | |
| 596 | − | |
| 597 | −void drm_legacy_rmmaps(struct drm_device *dev) | |
| 598 | −{ | |
| 599 | − struct drm_map_list *r_list, *list_temp; | |
| 600 | − | |
| 601 | − list_for_each_entry_safe(r_list, list_temp, &dev->maplist, head) | |
| 602 | − drm_legacy_rmmap(dev, r_list->map); | |
| 603 | −} | |
| 604 | − | |
| 605 | −/* The rmmap ioctl appears to be unnecessary. All mappings are torn down on | |
| 606 | − * the last close of the device, and this is necessary for cleanup when things | |
| 607 | − * exit uncleanly. Therefore, having userland manually remove mappings seems | |
| 608 | − * like a pointless exercise since they're going away anyway. | |
| 609 | − * | |
| 610 | − * One use case might be after addmap is allowed for normal users for SHM and | |
| 611 | − * gets used by drivers that the server doesn't need to care about. This seems | |
| 612 | − * unlikely. | |
| 613 | − * | |
| 614 | − * \param inode device inode. | |
| 615 | − * \param file_priv DRM file private. | |
| 616 | − * \param cmd command. | |
| 617 | − * \param arg pointer to a struct drm_map structure. | |
| 618 | − * \return zero on success or a negative value on error. | |
| 619 | − */ | |
| 620 | −int drm_legacy_rmmap_ioctl(struct drm_device *dev, void *data, | |
| 621 | − struct drm_file *file_priv) | |
| 622 | −{ | |
| 623 | − struct drm_map *request = data; | |
| 624 | − struct drm_local_map *map = NULL; | |
| 625 | − struct drm_map_list *r_list; | |
| 626 | − int ret; | |
| 627 | − | |
| 628 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 629 | − return -EOPNOTSUPP; | |
| 630 | − | |
| 631 | − mutex_lock(&dev->struct_mutex); | |
| 632 | − list_for_each_entry(r_list, &dev->maplist, head) { | |
| 633 | − if (r_list->map && | |
| 634 | − r_list->user_token == (unsigned long)request->handle && | |
| 635 | − r_list->map->flags & _DRM_REMOVABLE) { | |
| 636 | − map = r_list->map; | |
| 637 | − break; | |
| 638 | − } | |
| 639 | − } | |
| 640 | − | |
| 641 | − /* List has wrapped around to the head pointer, or it's empty we didn't | |
| 642 | − * find anything. | |
| 643 | − */ | |
| 644 | − if (list_empty(&dev->maplist) || !map) { | |
| 645 | − mutex_unlock(&dev->struct_mutex); | |
| 646 | − return -EINVAL; | |
| 647 | − } | |
| 648 | − | |
| 649 | − /* Register and framebuffer maps are permanent */ | |
| 650 | − if ((map->type == _DRM_REGISTERS) || (map->type == _DRM_FRAME_BUFFER)) { | |
| 651 | − mutex_unlock(&dev->struct_mutex); | |
| 652 | − return 0; | |
| 653 | − } | |
| 654 | − | |
| 655 | − ret = drm_legacy_rmmap_locked(dev, map); | |
| 656 | − | |
| 657 | − mutex_unlock(&dev->struct_mutex); | |
| 658 | − | |
| 659 | − return ret; | |
| 660 | −} | |
| 661 | − | |
| 662 | −/* | |
| 663 | − * Cleanup after an error on one of the addbufs() functions. | |
| 664 | − * | |
| 665 | − * \param dev DRM device. | |
| 666 | − * \param entry buffer entry where the error occurred. | |
| 667 | − * | |
| 668 | − * Frees any pages and buffers associated with the given entry. | |
| 669 | − */ | |
| 670 | −static void drm_cleanup_buf_error(struct drm_device *dev, | |
| 671 | − struct drm_buf_entry *entry) | |
| 672 | −{ | |
| 673 | − drm_dma_handle_t *dmah; | |
| 674 | − int i; | |
| 675 | − | |
| 676 | − if (entry->seg_count) { | |
| 677 | − for (i = 0; i < entry->seg_count; i++) { | |
| 678 | − if (entry->seglist[i]) { | |
| 679 | − dmah = entry->seglist[i]; | |
| 680 | − dma_free_coherent(dev->dev, | |
| 681 | − dmah->size, | |
| 682 | − dmah->vaddr, | |
| 683 | − dmah->busaddr); | |
| 684 | − kfree(dmah); | |
| 685 | − } | |
| 686 | − } | |
| 687 | − kfree(entry->seglist); | |
| 688 | − | |
| 689 | − entry->seg_count = 0; | |
| 690 | − } | |
| 691 | − | |
| 692 | − if (entry->buf_count) { | |
| 693 | − for (i = 0; i < entry->buf_count; i++) { | |
| 694 | − kfree(entry->buflist[i].dev_private); | |
| 695 | − } | |
| 696 | − kfree(entry->buflist); | |
| 697 | − | |
| 698 | − entry->buf_count = 0; | |
| 699 | − } | |
| 700 | −} | |
| 701 | − | |
| 702 | −#if IS_ENABLED(CONFIG_AGP) | |
| 703 | −/* | |
| 704 | − * Add AGP buffers for DMA transfers. | |
| 705 | − * | |
| 706 | − * \param dev struct drm_device to which the buffers are to be added. | |
| 707 | − * \param request pointer to a struct drm_buf_desc describing the request. | |
| 708 | − * \return zero on success or a negative number on failure. | |
| 709 | − * | |
| 710 | − * After some sanity checks creates a drm_buf structure for each buffer and | |
| 711 | − * reallocates the buffer list of the same size order to accommodate the new | |
| 712 | − * buffers. | |
| 713 | − */ | |
| 714 | −int drm_legacy_addbufs_agp(struct drm_device *dev, | |
| 715 | − struct drm_buf_desc *request) | |
| 716 | −{ | |
| 717 | − struct drm_device_dma *dma = dev->dma; | |
| 718 | − struct drm_buf_entry *entry; | |
| 719 | − struct drm_agp_mem *agp_entry; | |
| 720 | − struct drm_buf *buf; | |
| 721 | − unsigned long offset; | |
| 722 | − unsigned long agp_offset; | |
| 723 | − int count; | |
| 724 | − int order; | |
| 725 | − int size; | |
| 726 | − int alignment; | |
| 727 | − int page_order; | |
| 728 | − int total; | |
| 729 | − int byte_count; | |
| 730 | − int i, valid; | |
| 731 | − struct drm_buf **temp_buflist; | |
| 732 | − | |
| 733 | − if (!dma) | |
| 734 | − return -EINVAL; | |
| 735 | − | |
| 736 | − count = request->count; | |
| 737 | − order = order_base_2(request->size); | |
| 738 | − size = 1 << order; | |
| 739 | − | |
| 740 | − alignment = (request->flags & _DRM_PAGE_ALIGN) | |
| 741 | − ? PAGE_ALIGN(size) : size; | |
| 742 | − page_order = order - PAGE_SHIFT > 0 ? order - PAGE_SHIFT : 0; | |
| 743 | − total = PAGE_SIZE << page_order; | |
| 744 | − | |
| 745 | − byte_count = 0; | |
| 746 | − agp_offset = dev->agp->base + request->agp_start; | |
| 747 | − | |
| 748 | − DRM_DEBUG("count: %d\n", count); | |
| 749 | − DRM_DEBUG("order: %d\n", order); | |
| 750 | − DRM_DEBUG("size: %d\n", size); | |
| 751 | − DRM_DEBUG("agp_offset: %lx\n", agp_offset); | |
| 752 | − DRM_DEBUG("alignment: %d\n", alignment); | |
| 753 | − DRM_DEBUG("page_order: %d\n", page_order); | |
| 754 | − DRM_DEBUG("total: %d\n", total); | |
| 755 | − | |
| 756 | − if (order < DRM_MIN_ORDER || order > DRM_MAX_ORDER) | |
| 757 | − return -EINVAL; | |
| 758 | − | |
| 759 | − /* Make sure buffers are located in AGP memory that we own */ | |
| 760 | − valid = 0; | |
| 761 | − list_for_each_entry(agp_entry, &dev->agp->memory, head) { | |
| 762 | − if ((agp_offset >= agp_entry->bound) && | |
| 763 | − (agp_offset + total * count <= agp_entry->bound + agp_entry->pages * PAGE_SIZE)) { | |
| 764 | − valid = 1; | |
| 765 | − break; | |
| 766 | − } | |
| 767 | − } | |
| 768 | − if (!list_empty(&dev->agp->memory) && !valid) { | |
| 769 | − DRM_DEBUG("zone invalid\n"); | |
| 770 | − return -EINVAL; | |
| 771 | − } | |
| 772 | − spin_lock(&dev->buf_lock); | |
| 773 | − if (dev->buf_use) { | |
| 774 | − spin_unlock(&dev->buf_lock); | |
| 775 | − return -EBUSY; | |
| 776 | − } | |
| 777 | − atomic_inc(&dev->buf_alloc); | |
| 778 | − spin_unlock(&dev->buf_lock); | |
| 779 | − | |
| 780 | − mutex_lock(&dev->struct_mutex); | |
| 781 | − entry = &dma->bufs[order]; | |
| 782 | − if (entry->buf_count) { | |
| 783 | − mutex_unlock(&dev->struct_mutex); | |
| 784 | − atomic_dec(&dev->buf_alloc); | |
| 785 | − return -ENOMEM; /* May only call once for each order */ | |
| 786 | − } | |
| 787 | − | |
| 788 | − if (count < 0 || count > 4096) { | |
| 789 | − mutex_unlock(&dev->struct_mutex); | |
| 790 | − atomic_dec(&dev->buf_alloc); | |
| 791 | − return -EINVAL; | |
| 792 | − } | |
| 793 | − | |
| 794 | − entry->buflist = kcalloc(count, sizeof(*entry->buflist), GFP_KERNEL); | |
| 795 | − if (!entry->buflist) { | |
| 796 | − mutex_unlock(&dev->struct_mutex); | |
| 797 | − atomic_dec(&dev->buf_alloc); | |
| 798 | − return -ENOMEM; | |
| 799 | − } | |
| 800 | − | |
| 801 | − entry->buf_size = size; | |
| 802 | − entry->page_order = page_order; | |
| 803 | − | |
| 804 | − offset = 0; | |
| 805 | − | |
| 806 | − while (entry->buf_count < count) { | |
| 807 | − buf = &entry->buflist[entry->buf_count]; | |
| 808 | − buf->idx = dma->buf_count + entry->buf_count; | |
| 809 | − buf->total = alignment; | |
| 810 | − buf->order = order; | |
| 811 | − buf->used = 0; | |
| 812 | − | |
| 813 | − buf->offset = (dma->byte_count + offset); | |
| 814 | − buf->bus_address = agp_offset + offset; | |
| 815 | − buf->address = (void *)(agp_offset + offset); | |
| 816 | − buf->next = NULL; | |
| 817 | − buf->waiting = 0; | |
| 818 | − buf->pending = 0; | |
| 819 | − buf->file_priv = NULL; | |
| 820 | − | |
| 821 | − buf->dev_priv_size = dev->driver->dev_priv_size; | |
| 822 | − buf->dev_private = kzalloc(buf->dev_priv_size, GFP_KERNEL); | |
| 823 | − if (!buf->dev_private) { | |
| 824 | − /* Set count correctly so we free the proper amount. */ | |
| 825 | − entry->buf_count = count; | |
| 826 | − drm_cleanup_buf_error(dev, entry); | |
| 827 | − mutex_unlock(&dev->struct_mutex); | |
| 828 | − atomic_dec(&dev->buf_alloc); | |
| 829 | − return -ENOMEM; | |
| 830 | − } | |
| 831 | − | |
| 832 | − DRM_DEBUG("buffer %d @ %p\n", entry->buf_count, buf->address); | |
| 833 | − | |
| 834 | − offset += alignment; | |
| 835 | − entry->buf_count++; | |
| 836 | − byte_count += PAGE_SIZE << page_order; | |
| 837 | − } | |
| 838 | − | |
| 839 | − DRM_DEBUG("byte_count: %d\n", byte_count); | |
| 840 | − | |
| 841 | − temp_buflist = krealloc(dma->buflist, | |
| 842 | − (dma->buf_count + entry->buf_count) * | |
| 843 | − sizeof(*dma->buflist), GFP_KERNEL); | |
| 844 | − if (!temp_buflist) { | |
| 845 | − /* Free the entry because it isn't valid */ | |
| 846 | − drm_cleanup_buf_error(dev, entry); | |
| 847 | − mutex_unlock(&dev->struct_mutex); | |
| 848 | − atomic_dec(&dev->buf_alloc); | |
| 849 | − return -ENOMEM; | |
| 850 | − } | |
| 851 | − dma->buflist = temp_buflist; | |
| 852 | − | |
| 853 | − for (i = 0; i < entry->buf_count; i++) { | |
| 854 | − dma->buflist[i + dma->buf_count] = &entry->buflist[i]; | |
| 855 | − } | |
| 856 | − | |
| 857 | − dma->buf_count += entry->buf_count; | |
| 858 | − dma->seg_count += entry->seg_count; | |
| 859 | − dma->page_count += byte_count >> PAGE_SHIFT; | |
| 860 | − dma->byte_count += byte_count; | |
| 861 | − | |
| 862 | − DRM_DEBUG("dma->buf_count : %d\n", dma->buf_count); | |
| 863 | − DRM_DEBUG("entry->buf_count : %d\n", entry->buf_count); | |
| 864 | − | |
| 865 | − mutex_unlock(&dev->struct_mutex); | |
| 866 | − | |
| 867 | − request->count = entry->buf_count; | |
| 868 | − request->size = size; | |
| 869 | − | |
| 870 | − dma->flags = _DRM_DMA_USE_AGP; | |
| 871 | − | |
| 872 | − atomic_dec(&dev->buf_alloc); | |
| 873 | − return 0; | |
| 874 | −} | |
| 875 | −EXPORT_SYMBOL(drm_legacy_addbufs_agp); | |
| 876 | −#endif /* CONFIG_AGP */ | |
| 877 | − | |
| 878 | −int drm_legacy_addbufs_pci(struct drm_device *dev, | |
| 879 | − struct drm_buf_desc *request) | |
| 880 | −{ | |
| 881 | − struct drm_device_dma *dma = dev->dma; | |
| 882 | − int count; | |
| 883 | − int order; | |
| 884 | − int size; | |
| 885 | − int total; | |
| 886 | − int page_order; | |
| 887 | − struct drm_buf_entry *entry; | |
| 888 | − drm_dma_handle_t *dmah; | |
| 889 | − struct drm_buf *buf; | |
| 890 | − int alignment; | |
| 891 | − unsigned long offset; | |
| 892 | − int i; | |
| 893 | − int byte_count; | |
| 894 | − int page_count; | |
| 895 | − unsigned long *temp_pagelist; | |
| 896 | − struct drm_buf **temp_buflist; | |
| 897 | − | |
| 898 | − if (!drm_core_check_feature(dev, DRIVER_PCI_DMA)) | |
| 899 | − return -EOPNOTSUPP; | |
| 900 | − | |
| 901 | − if (!dma) | |
| 902 | − return -EINVAL; | |
| 903 | − | |
| 904 | − if (!capable(CAP_SYS_ADMIN)) | |
| 905 | − return -EPERM; | |
| 906 | − | |
| 907 | − count = request->count; | |
| 908 | − order = order_base_2(request->size); | |
| 909 | − size = 1 << order; | |
| 910 | − | |
| 911 | − DRM_DEBUG("count=%d, size=%d (%d), order=%d\n", | |
| 912 | − request->count, request->size, size, order); | |
| 913 | − | |
| 914 | − if (order < DRM_MIN_ORDER || order > DRM_MAX_ORDER) | |
| 915 | − return -EINVAL; | |
| 916 | − | |
| 917 | − alignment = (request->flags & _DRM_PAGE_ALIGN) | |
| 918 | − ? PAGE_ALIGN(size) : size; | |
| 919 | − page_order = order - PAGE_SHIFT > 0 ? order - PAGE_SHIFT : 0; | |
| 920 | − total = PAGE_SIZE << page_order; | |
| 921 | − | |
| 922 | − spin_lock(&dev->buf_lock); | |
| 923 | − if (dev->buf_use) { | |
| 924 | − spin_unlock(&dev->buf_lock); | |
| 925 | − return -EBUSY; | |
| 926 | − } | |
| 927 | − atomic_inc(&dev->buf_alloc); | |
| 928 | − spin_unlock(&dev->buf_lock); | |
| 929 | − | |
| 930 | − mutex_lock(&dev->struct_mutex); | |
| 931 | − entry = &dma->bufs[order]; | |
| 932 | − if (entry->buf_count) { | |
| 933 | − mutex_unlock(&dev->struct_mutex); | |
| 934 | − atomic_dec(&dev->buf_alloc); | |
| 935 | − return -ENOMEM; /* May only call once for each order */ | |
| 936 | − } | |
| 937 | − | |
| 938 | − if (count < 0 || count > 4096) { | |
| 939 | − mutex_unlock(&dev->struct_mutex); | |
| 940 | − atomic_dec(&dev->buf_alloc); | |
| 941 | − return -EINVAL; | |
| 942 | − } | |
| 943 | − | |
| 944 | − entry->buflist = kcalloc(count, sizeof(*entry->buflist), GFP_KERNEL); | |
| 945 | − if (!entry->buflist) { | |
| 946 | − mutex_unlock(&dev->struct_mutex); | |
| 947 | − atomic_dec(&dev->buf_alloc); | |
| 948 | − return -ENOMEM; | |
| 949 | − } | |
| 950 | − | |
| 951 | − entry->seglist = kcalloc(count, sizeof(*entry->seglist), GFP_KERNEL); | |
| 952 | − if (!entry->seglist) { | |
| 953 | − kfree(entry->buflist); | |
| 954 | − mutex_unlock(&dev->struct_mutex); | |
| 955 | − atomic_dec(&dev->buf_alloc); | |
| 956 | − return -ENOMEM; | |
| 957 | − } | |
| 958 | − | |
| 959 | − /* Keep the original pagelist until we know all the allocations | |
| 960 | − * have succeeded | |
| 961 | − */ | |
| 962 | − temp_pagelist = kmalloc_array(dma->page_count + (count << page_order), | |
| 963 | − sizeof(*dma->pagelist), | |
| 964 | − GFP_KERNEL); | |
| 965 | − if (!temp_pagelist) { | |
| 966 | − kfree(entry->buflist); | |
| 967 | − kfree(entry->seglist); | |
| 968 | − mutex_unlock(&dev->struct_mutex); | |
| 969 | − atomic_dec(&dev->buf_alloc); | |
| 970 | − return -ENOMEM; | |
| 971 | − } | |
| 972 | − memcpy(temp_pagelist, | |
| 973 | − dma->pagelist, dma->page_count * sizeof(*dma->pagelist)); | |
| 974 | − DRM_DEBUG("pagelist: %d entries\n", | |
| 975 | − dma->page_count + (count << page_order)); | |
| 976 | − | |
| 977 | − entry->buf_size = size; | |
| 978 | − entry->page_order = page_order; | |
| 979 | − byte_count = 0; | |
| 980 | − page_count = 0; | |
| 981 | − | |
| 982 | − while (entry->buf_count < count) { | |
| 983 | − dmah = kmalloc(sizeof(drm_dma_handle_t), GFP_KERNEL); | |
| 984 | − if (!dmah) { | |
| 985 | − /* Set count correctly so we free the proper amount. */ | |
| 986 | − entry->buf_count = count; | |
| 987 | − entry->seg_count = count; | |
| 988 | − drm_cleanup_buf_error(dev, entry); | |
| 989 | − kfree(temp_pagelist); | |
| 990 | − mutex_unlock(&dev->struct_mutex); | |
| 991 | − atomic_dec(&dev->buf_alloc); | |
| 992 | − return -ENOMEM; | |
| 993 | − } | |
| 994 | − | |
| 995 | − dmah->size = total; | |
| 996 | − dmah->vaddr = dma_alloc_coherent(dev->dev, | |
| 997 | − dmah->size, | |
| 998 | − &dmah->busaddr, | |
| 999 | − GFP_KERNEL); | |
| 1000 | − if (!dmah->vaddr) { | |
| 1001 | − kfree(dmah); | |
| 1002 | − | |
| 1003 | − /* Set count correctly so we free the proper amount. */ | |
| 1004 | − entry->buf_count = count; | |
| 1005 | − entry->seg_count = count; | |
| 1006 | − drm_cleanup_buf_error(dev, entry); | |
| 1007 | − kfree(temp_pagelist); | |
| 1008 | − mutex_unlock(&dev->struct_mutex); | |
| 1009 | − atomic_dec(&dev->buf_alloc); | |
| 1010 | − return -ENOMEM; | |
| 1011 | − } | |
| 1012 | − entry->seglist[entry->seg_count++] = dmah; | |
| 1013 | − for (i = 0; i < (1 << page_order); i++) { | |
| 1014 | − DRM_DEBUG("page %d @ 0x%08lx\n", | |
| 1015 | − dma->page_count + page_count, | |
| 1016 | − (unsigned long)dmah->vaddr + PAGE_SIZE * i); | |
| 1017 | − temp_pagelist[dma->page_count + page_count++] | |
| 1018 | − = (unsigned long)dmah->vaddr + PAGE_SIZE * i; | |
| 1019 | − } | |
| 1020 | − for (offset = 0; | |
| 1021 | − offset + size <= total && entry->buf_count < count; | |
| 1022 | − offset += alignment, ++entry->buf_count) { | |
| 1023 | − buf = &entry->buflist[entry->buf_count]; | |
| 1024 | − buf->idx = dma->buf_count + entry->buf_count; | |
| 1025 | − buf->total = alignment; | |
| 1026 | − buf->order = order; | |
| 1027 | − buf->used = 0; | |
| 1028 | − buf->offset = (dma->byte_count + byte_count + offset); | |
| 1029 | − buf->address = (void *)(dmah->vaddr + offset); | |
| 1030 | − buf->bus_address = dmah->busaddr + offset; | |
| 1031 | − buf->next = NULL; | |
| 1032 | − buf->waiting = 0; | |
| 1033 | − buf->pending = 0; | |
| 1034 | − buf->file_priv = NULL; | |
| 1035 | − | |
| 1036 | − buf->dev_priv_size = dev->driver->dev_priv_size; | |
| 1037 | − buf->dev_private = kzalloc(buf->dev_priv_size, | |
| 1038 | − GFP_KERNEL); | |
| 1039 | − if (!buf->dev_private) { | |
| 1040 | − /* Set count correctly so we free the proper amount. */ | |
| 1041 | − entry->buf_count = count; | |
| 1042 | − entry->seg_count = count; | |
| 1043 | − drm_cleanup_buf_error(dev, entry); | |
| 1044 | − kfree(temp_pagelist); | |
| 1045 | − mutex_unlock(&dev->struct_mutex); | |
| 1046 | − atomic_dec(&dev->buf_alloc); | |
| 1047 | − return -ENOMEM; | |
| 1048 | − } | |
| 1049 | − | |
| 1050 | − DRM_DEBUG("buffer %d @ %p\n", | |
| 1051 | − entry->buf_count, buf->address); | |
| 1052 | − } | |
| 1053 | − byte_count += PAGE_SIZE << page_order; | |
| 1054 | − } | |
| 1055 | − | |
| 1056 | − temp_buflist = krealloc(dma->buflist, | |
| 1057 | − (dma->buf_count + entry->buf_count) * | |
| 1058 | − sizeof(*dma->buflist), GFP_KERNEL); | |
| 1059 | − if (!temp_buflist) { | |
| 1060 | − /* Free the entry because it isn't valid */ | |
| 1061 | − drm_cleanup_buf_error(dev, entry); | |
| 1062 | − kfree(temp_pagelist); | |
| 1063 | − mutex_unlock(&dev->struct_mutex); | |
| 1064 | − atomic_dec(&dev->buf_alloc); | |
| 1065 | − return -ENOMEM; | |
| 1066 | − } | |
| 1067 | − dma->buflist = temp_buflist; | |
| 1068 | − | |
| 1069 | − for (i = 0; i < entry->buf_count; i++) { | |
| 1070 | − dma->buflist[i + dma->buf_count] = &entry->buflist[i]; | |
| 1071 | − } | |
| 1072 | − | |
| 1073 | − /* No allocations failed, so now we can replace the original pagelist | |
| 1074 | − * with the new one. | |
| 1075 | − */ | |
| 1076 | − if (dma->page_count) { | |
| 1077 | − kfree(dma->pagelist); | |
| 1078 | − } | |
| 1079 | − dma->pagelist = temp_pagelist; | |
| 1080 | − | |
| 1081 | − dma->buf_count += entry->buf_count; | |
| 1082 | − dma->seg_count += entry->seg_count; | |
| 1083 | − dma->page_count += entry->seg_count << page_order; | |
| 1084 | − dma->byte_count += PAGE_SIZE * (entry->seg_count << page_order); | |
| 1085 | − | |
| 1086 | − mutex_unlock(&dev->struct_mutex); | |
| 1087 | − | |
| 1088 | − request->count = entry->buf_count; | |
| 1089 | − request->size = size; | |
| 1090 | − | |
| 1091 | − if (request->flags & _DRM_PCI_BUFFER_RO) | |
| 1092 | − dma->flags = _DRM_DMA_USE_PCI_RO; | |
| 1093 | − | |
| 1094 | − atomic_dec(&dev->buf_alloc); | |
| 1095 | − return 0; | |
| 1096 | − | |
| 1097 | −} | |
| 1098 | −EXPORT_SYMBOL(drm_legacy_addbufs_pci); | |
| 1099 | − | |
| 1100 | −static int drm_legacy_addbufs_sg(struct drm_device *dev, | |
| 1101 | − struct drm_buf_desc *request) | |
| 1102 | −{ | |
| 1103 | − struct drm_device_dma *dma = dev->dma; | |
| 1104 | − struct drm_buf_entry *entry; | |
| 1105 | − struct drm_buf *buf; | |
| 1106 | − unsigned long offset; | |
| 1107 | − unsigned long agp_offset; | |
| 1108 | − int count; | |
| 1109 | − int order; | |
| 1110 | − int size; | |
| 1111 | − int alignment; | |
| 1112 | − int page_order; | |
| 1113 | − int total; | |
| 1114 | − int byte_count; | |
| 1115 | − int i; | |
| 1116 | − struct drm_buf **temp_buflist; | |
| 1117 | − | |
| 1118 | − if (!drm_core_check_feature(dev, DRIVER_SG)) | |
| 1119 | − return -EOPNOTSUPP; | |
| 1120 | − | |
| 1121 | − if (!dma) | |
| 1122 | − return -EINVAL; | |
| 1123 | − | |
| 1124 | − if (!capable(CAP_SYS_ADMIN)) | |
| 1125 | − return -EPERM; | |
| 1126 | − | |
| 1127 | − count = request->count; | |
| 1128 | − order = order_base_2(request->size); | |
| 1129 | − size = 1 << order; | |
| 1130 | − | |
| 1131 | − alignment = (request->flags & _DRM_PAGE_ALIGN) | |
| 1132 | − ? PAGE_ALIGN(size) : size; | |
| 1133 | − page_order = order - PAGE_SHIFT > 0 ? order - PAGE_SHIFT : 0; | |
| 1134 | − total = PAGE_SIZE << page_order; | |
| 1135 | − | |
| 1136 | − byte_count = 0; | |
| 1137 | − agp_offset = request->agp_start; | |
| 1138 | − | |
| 1139 | − DRM_DEBUG("count: %d\n", count); | |
| 1140 | − DRM_DEBUG("order: %d\n", order); | |
| 1141 | − DRM_DEBUG("size: %d\n", size); | |
| 1142 | − DRM_DEBUG("agp_offset: %lu\n", agp_offset); | |
| 1143 | − DRM_DEBUG("alignment: %d\n", alignment); | |
| 1144 | − DRM_DEBUG("page_order: %d\n", page_order); | |
| 1145 | − DRM_DEBUG("total: %d\n", total); | |
| 1146 | − | |
| 1147 | − if (order < DRM_MIN_ORDER || order > DRM_MAX_ORDER) | |
| 1148 | − return -EINVAL; | |
| 1149 | − | |
| 1150 | − spin_lock(&dev->buf_lock); | |
| 1151 | − if (dev->buf_use) { | |
| 1152 | − spin_unlock(&dev->buf_lock); | |
| 1153 | − return -EBUSY; | |
| 1154 | − } | |
| 1155 | − atomic_inc(&dev->buf_alloc); | |
| 1156 | − spin_unlock(&dev->buf_lock); | |
| 1157 | − | |
| 1158 | − mutex_lock(&dev->struct_mutex); | |
| 1159 | − entry = &dma->bufs[order]; | |
| 1160 | − if (entry->buf_count) { | |
| 1161 | − mutex_unlock(&dev->struct_mutex); | |
| 1162 | − atomic_dec(&dev->buf_alloc); | |
| 1163 | − return -ENOMEM; /* May only call once for each order */ | |
| 1164 | − } | |
| 1165 | − | |
| 1166 | − if (count < 0 || count > 4096) { | |
| 1167 | − mutex_unlock(&dev->struct_mutex); | |
| 1168 | − atomic_dec(&dev->buf_alloc); | |
| 1169 | − return -EINVAL; | |
| 1170 | − } | |
| 1171 | − | |
| 1172 | − entry->buflist = kcalloc(count, sizeof(*entry->buflist), GFP_KERNEL); | |
| 1173 | − if (!entry->buflist) { | |
| 1174 | − mutex_unlock(&dev->struct_mutex); | |
| 1175 | − atomic_dec(&dev->buf_alloc); | |
| 1176 | − return -ENOMEM; | |
| 1177 | − } | |
| 1178 | − | |
| 1179 | − entry->buf_size = size; | |
| 1180 | − entry->page_order = page_order; | |
| 1181 | − | |
| 1182 | − offset = 0; | |
| 1183 | − | |
| 1184 | − while (entry->buf_count < count) { | |
| 1185 | − buf = &entry->buflist[entry->buf_count]; | |
| 1186 | − buf->idx = dma->buf_count + entry->buf_count; | |
| 1187 | − buf->total = alignment; | |
| 1188 | − buf->order = order; | |
| 1189 | − buf->used = 0; | |
| 1190 | − | |
| 1191 | − buf->offset = (dma->byte_count + offset); | |
| 1192 | − buf->bus_address = agp_offset + offset; | |
| 1193 | − buf->address = (void *)(agp_offset + offset | |
| 1194 | − + (unsigned long)dev->sg->virtual); | |
| 1195 | − buf->next = NULL; | |
| 1196 | − buf->waiting = 0; | |
| 1197 | − buf->pending = 0; | |
| 1198 | − buf->file_priv = NULL; | |
| 1199 | − | |
| 1200 | − buf->dev_priv_size = dev->driver->dev_priv_size; | |
| 1201 | − buf->dev_private = kzalloc(buf->dev_priv_size, GFP_KERNEL); | |
| 1202 | − if (!buf->dev_private) { | |
| 1203 | − /* Set count correctly so we free the proper amount. */ | |
| 1204 | − entry->buf_count = count; | |
| 1205 | − drm_cleanup_buf_error(dev, entry); | |
| 1206 | − mutex_unlock(&dev->struct_mutex); | |
| 1207 | − atomic_dec(&dev->buf_alloc); | |
| 1208 | − return -ENOMEM; | |
| 1209 | − } | |
| 1210 | − | |
| 1211 | − DRM_DEBUG("buffer %d @ %p\n", entry->buf_count, buf->address); | |
| 1212 | − | |
| 1213 | − offset += alignment; | |
| 1214 | − entry->buf_count++; | |
| 1215 | − byte_count += PAGE_SIZE << page_order; | |
| 1216 | − } | |
| 1217 | − | |
| 1218 | − DRM_DEBUG("byte_count: %d\n", byte_count); | |
| 1219 | − | |
| 1220 | − temp_buflist = krealloc(dma->buflist, | |
| 1221 | − (dma->buf_count + entry->buf_count) * | |
| 1222 | − sizeof(*dma->buflist), GFP_KERNEL); | |
| 1223 | − if (!temp_buflist) { | |
| 1224 | − /* Free the entry because it isn't valid */ | |
| 1225 | − drm_cleanup_buf_error(dev, entry); | |
| 1226 | − mutex_unlock(&dev->struct_mutex); | |
| 1227 | − atomic_dec(&dev->buf_alloc); | |
| 1228 | − return -ENOMEM; | |
| 1229 | − } | |
| 1230 | − dma->buflist = temp_buflist; | |
| 1231 | − | |
| 1232 | − for (i = 0; i < entry->buf_count; i++) { | |
| 1233 | − dma->buflist[i + dma->buf_count] = &entry->buflist[i]; | |
| 1234 | − } | |
| 1235 | − | |
| 1236 | − dma->buf_count += entry->buf_count; | |
| 1237 | − dma->seg_count += entry->seg_count; | |
| 1238 | − dma->page_count += byte_count >> PAGE_SHIFT; | |
| 1239 | − dma->byte_count += byte_count; | |
| 1240 | − | |
| 1241 | − DRM_DEBUG("dma->buf_count : %d\n", dma->buf_count); | |
| 1242 | − DRM_DEBUG("entry->buf_count : %d\n", entry->buf_count); | |
| 1243 | − | |
| 1244 | − mutex_unlock(&dev->struct_mutex); | |
| 1245 | − | |
| 1246 | − request->count = entry->buf_count; | |
| 1247 | − request->size = size; | |
| 1248 | − | |
| 1249 | − dma->flags = _DRM_DMA_USE_SG; | |
| 1250 | − | |
| 1251 | − atomic_dec(&dev->buf_alloc); | |
| 1252 | − return 0; | |
| 1253 | −} | |
| 1254 | − | |
| 1255 | −/* | |
| 1256 | − * Add buffers for DMA transfers (ioctl). | |
| 1257 | − * | |
| 1258 | − * \param inode device inode. | |
| 1259 | − * \param file_priv DRM file private. | |
| 1260 | − * \param cmd command. | |
| 1261 | − * \param arg pointer to a struct drm_buf_desc request. | |
| 1262 | − * \return zero on success or a negative number on failure. | |
| 1263 | − * | |
| 1264 | − * According with the memory type specified in drm_buf_desc::flags and the | |
| 1265 | − * build options, it dispatches the call either to addbufs_agp(), | |
| 1266 | − * addbufs_sg() or addbufs_pci() for AGP, scatter-gather or consistent | |
| 1267 | − * PCI memory respectively. | |
| 1268 | − */ | |
| 1269 | −int drm_legacy_addbufs(struct drm_device *dev, void *data, | |
| 1270 | − struct drm_file *file_priv) | |
| 1271 | −{ | |
| 1272 | − struct drm_buf_desc *request = data; | |
| 1273 | − int ret; | |
| 1274 | − | |
| 1275 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 1276 | − return -EOPNOTSUPP; | |
| 1277 | − | |
| 1278 | − if (!drm_core_check_feature(dev, DRIVER_HAVE_DMA)) | |
| 1279 | − return -EOPNOTSUPP; | |
| 1280 | − | |
| 1281 | −#if IS_ENABLED(CONFIG_AGP) | |
| 1282 | − if (request->flags & _DRM_AGP_BUFFER) | |
| 1283 | − ret = drm_legacy_addbufs_agp(dev, request); | |
| 1284 | − else | |
| 1285 | −#endif | |
| 1286 | − if (request->flags & _DRM_SG_BUFFER) | |
| 1287 | − ret = drm_legacy_addbufs_sg(dev, request); | |
| 1288 | − else if (request->flags & _DRM_FB_BUFFER) | |
| 1289 | − ret = -EINVAL; | |
| 1290 | − else | |
| 1291 | − ret = drm_legacy_addbufs_pci(dev, request); | |
| 1292 | − | |
| 1293 | − return ret; | |
| 1294 | −} | |
| 1295 | − | |
| 1296 | −/* | |
| 1297 | − * Get information about the buffer mappings. | |
| 1298 | − * | |
| 1299 | − * This was originally mean for debugging purposes, or by a sophisticated | |
| 1300 | − * client library to determine how best to use the available buffers (e.g., | |
| 1301 | − * large buffers can be used for image transfer). | |
| 1302 | − * | |
| 1303 | − * \param inode device inode. | |
| 1304 | − * \param file_priv DRM file private. | |
| 1305 | − * \param cmd command. | |
| 1306 | − * \param arg pointer to a drm_buf_info structure. | |
| 1307 | − * \return zero on success or a negative number on failure. | |
| 1308 | − * | |
| 1309 | − * Increments drm_device::buf_use while holding the drm_device::buf_lock | |
| 1310 | − * lock, preventing of allocating more buffers after this call. Information | |
| 1311 | − * about each requested buffer is then copied into user space. | |
| 1312 | − */ | |
| 1313 | −int __drm_legacy_infobufs(struct drm_device *dev, | |
| 1314 | − void *data, int *p, | |
| 1315 | − int (*f)(void *, int, struct drm_buf_entry *)) | |
| 1316 | −{ | |
| 1317 | − struct drm_device_dma *dma = dev->dma; | |
| 1318 | − int i; | |
| 1319 | − int count; | |
| 1320 | − | |
| 1321 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 1322 | − return -EOPNOTSUPP; | |
| 1323 | − | |
| 1324 | − if (!drm_core_check_feature(dev, DRIVER_HAVE_DMA)) | |
| 1325 | − return -EOPNOTSUPP; | |
| 1326 | − | |
| 1327 | − if (!dma) | |
| 1328 | − return -EINVAL; | |
| 1329 | − | |
| 1330 | − spin_lock(&dev->buf_lock); | |
| 1331 | − if (atomic_read(&dev->buf_alloc)) { | |
| 1332 | − spin_unlock(&dev->buf_lock); | |
| 1333 | − return -EBUSY; | |
| 1334 | − } | |
| 1335 | − ++dev->buf_use; /* Can't allocate more after this call */ | |
| 1336 | − spin_unlock(&dev->buf_lock); | |
| 1337 | − | |
| 1338 | − for (i = 0, count = 0; i < DRM_MAX_ORDER + 1; i++) { | |
| 1339 | − if (dma->bufs[i].buf_count) | |
| 1340 | − ++count; | |
| 1341 | − } | |
| 1342 | − | |
| 1343 | − DRM_DEBUG("count = %d\n", count); | |
| 1344 | − | |
| 1345 | − if (*p >= count) { | |
| 1346 | − for (i = 0, count = 0; i < DRM_MAX_ORDER + 1; i++) { | |
| 1347 | − struct drm_buf_entry *from = &dma->bufs[i]; | |
| 1348 | − | |
| 1349 | − if (from->buf_count) { | |
| 1350 | − if (f(data, count, from) < 0) | |
| 1351 | − return -EFAULT; | |
| 1352 | − DRM_DEBUG("%d %d %d %d %d\n", | |
| 1353 | − i, | |
| 1354 | − dma->bufs[i].buf_count, | |
| 1355 | − dma->bufs[i].buf_size, | |
| 1356 | − dma->bufs[i].low_mark, | |
| 1357 | − dma->bufs[i].high_mark); | |
| 1358 | − ++count; | |
| 1359 | − } | |
| 1360 | − } | |
| 1361 | − } | |
| 1362 | − *p = count; | |
| 1363 | − | |
| 1364 | − return 0; | |
| 1365 | −} | |
| 1366 | − | |
| 1367 | −static int copy_one_buf(void *data, int count, struct drm_buf_entry *from) | |
| 1368 | −{ | |
| 1369 | − struct drm_buf_info *request = data; | |
| 1370 | − struct drm_buf_desc __user *to = &request->list[count]; | |
| 1371 | − struct drm_buf_desc v = {.count = from->buf_count, | |
| 1372 | − .size = from->buf_size, | |
| 1373 | − .low_mark = from->low_mark, | |
| 1374 | − .high_mark = from->high_mark}; | |
| 1375 | − | |
| 1376 | − if (copy_to_user(to, &v, offsetof(struct drm_buf_desc, flags))) | |
| 1377 | − return -EFAULT; | |
| 1378 | − return 0; | |
| 1379 | −} | |
| 1380 | − | |
| 1381 | −int drm_legacy_infobufs(struct drm_device *dev, void *data, | |
| 1382 | − struct drm_file *file_priv) | |
| 1383 | −{ | |
| 1384 | − struct drm_buf_info *request = data; | |
| 1385 | − | |
| 1386 | − return __drm_legacy_infobufs(dev, data, &request->count, copy_one_buf); | |
| 1387 | −} | |
| 1388 | − | |
| 1389 | −/* | |
| 1390 | − * Specifies a low and high water mark for buffer allocation | |
| 1391 | − * | |
| 1392 | − * \param inode device inode. | |
| 1393 | − * \param file_priv DRM file private. | |
| 1394 | − * \param cmd command. | |
| 1395 | − * \param arg a pointer to a drm_buf_desc structure. | |
| 1396 | − * \return zero on success or a negative number on failure. | |
| 1397 | − * | |
| 1398 | − * Verifies that the size order is bounded between the admissible orders and | |
| 1399 | − * updates the respective drm_device_dma::bufs entry low and high water mark. | |
| 1400 | − * | |
| 1401 | − * \note This ioctl is deprecated and mostly never used. | |
| 1402 | − */ | |
| 1403 | −int drm_legacy_markbufs(struct drm_device *dev, void *data, | |
| 1404 | − struct drm_file *file_priv) | |
| 1405 | −{ | |
| 1406 | − struct drm_device_dma *dma = dev->dma; | |
| 1407 | − struct drm_buf_desc *request = data; | |
| 1408 | − int order; | |
| 1409 | − struct drm_buf_entry *entry; | |
| 1410 | − | |
| 1411 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 1412 | − return -EOPNOTSUPP; | |
| 1413 | − | |
| 1414 | − if (!drm_core_check_feature(dev, DRIVER_HAVE_DMA)) | |
| 1415 | − return -EOPNOTSUPP; | |
| 1416 | − | |
| 1417 | − if (!dma) | |
| 1418 | − return -EINVAL; | |
| 1419 | − | |
| 1420 | − DRM_DEBUG("%d, %d, %d\n", | |
| 1421 | − request->size, request->low_mark, request->high_mark); | |
| 1422 | − order = order_base_2(request->size); | |
| 1423 | − if (order < DRM_MIN_ORDER || order > DRM_MAX_ORDER) | |
| 1424 | − return -EINVAL; | |
| 1425 | − entry = &dma->bufs[order]; | |
| 1426 | − | |
| 1427 | − if (request->low_mark < 0 || request->low_mark > entry->buf_count) | |
| 1428 | − return -EINVAL; | |
| 1429 | − if (request->high_mark < 0 || request->high_mark > entry->buf_count) | |
| 1430 | − return -EINVAL; | |
| 1431 | − | |
| 1432 | − entry->low_mark = request->low_mark; | |
| 1433 | − entry->high_mark = request->high_mark; | |
| 1434 | − | |
| 1435 | − return 0; | |
| 1436 | −} | |
| 1437 | − | |
| 1438 | −/* | |
| 1439 | − * Unreserve the buffers in list, previously reserved using drmDMA. | |
| 1440 | − * | |
| 1441 | − * \param inode device inode. | |
| 1442 | − * \param file_priv DRM file private. | |
| 1443 | − * \param cmd command. | |
| 1444 | − * \param arg pointer to a drm_buf_free structure. | |
| 1445 | − * \return zero on success or a negative number on failure. | |
| 1446 | − * | |
| 1447 | − * Calls free_buffer() for each used buffer. | |
| 1448 | − * This function is primarily used for debugging. | |
| 1449 | − */ | |
| 1450 | −int drm_legacy_freebufs(struct drm_device *dev, void *data, | |
| 1451 | − struct drm_file *file_priv) | |
| 1452 | −{ | |
| 1453 | − struct drm_device_dma *dma = dev->dma; | |
| 1454 | − struct drm_buf_free *request = data; | |
| 1455 | − int i; | |
| 1456 | − int idx; | |
| 1457 | − struct drm_buf *buf; | |
| 1458 | − | |
| 1459 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 1460 | − return -EOPNOTSUPP; | |
| 1461 | − | |
| 1462 | − if (!drm_core_check_feature(dev, DRIVER_HAVE_DMA)) | |
| 1463 | − return -EOPNOTSUPP; | |
| 1464 | − | |
| 1465 | − if (!dma) | |
| 1466 | − return -EINVAL; | |
| 1467 | − | |
| 1468 | − DRM_DEBUG("%d\n", request->count); | |
| 1469 | − for (i = 0; i < request->count; i++) { | |
| 1470 | − if (copy_from_user(&idx, &request->list[i], sizeof(idx))) | |
| 1471 | − return -EFAULT; | |
| 1472 | − if (idx < 0 || idx >= dma->buf_count) { | |
| 1473 | − DRM_ERROR("Index %d (of %d max)\n", | |
| 1474 | − idx, dma->buf_count - 1); | |
| 1475 | − return -EINVAL; | |
| 1476 | − } | |
| 1477 | − idx = array_index_nospec(idx, dma->buf_count); | |
| 1478 | − buf = dma->buflist[idx]; | |
| 1479 | − if (buf->file_priv != file_priv) { | |
| 1480 | − DRM_ERROR("Process %d freeing buffer not owned\n", | |
| 1481 | − task_pid_nr(current)); | |
| 1482 | − return -EINVAL; | |
| 1483 | − } | |
| 1484 | − drm_legacy_free_buffer(dev, buf); | |
| 1485 | − } | |
| 1486 | − | |
| 1487 | − return 0; | |
| 1488 | −} | |
| 1489 | − | |
| 1490 | −/* | |
| 1491 | − * Maps all of the DMA buffers into client-virtual space (ioctl). | |
| 1492 | − * | |
| 1493 | − * \param inode device inode. | |
| 1494 | − * \param file_priv DRM file private. | |
| 1495 | − * \param cmd command. | |
| 1496 | − * \param arg pointer to a drm_buf_map structure. | |
| 1497 | − * \return zero on success or a negative number on failure. | |
| 1498 | − * | |
| 1499 | − * Maps the AGP, SG or PCI buffer region with vm_mmap(), and copies information | |
| 1500 | − * about each buffer into user space. For PCI buffers, it calls vm_mmap() with | |
| 1501 | − * offset equal to 0, which drm_mmap() interprets as PCI buffers and calls | |
| 1502 | − * drm_mmap_dma(). | |
| 1503 | − */ | |
| 1504 | −int __drm_legacy_mapbufs(struct drm_device *dev, void *data, int *p, | |
| 1505 | − void __user **v, | |
| 1506 | − int (*f)(void *, int, unsigned long, | |
| 1507 | − struct drm_buf *), | |
| 1508 | − struct drm_file *file_priv) | |
| 1509 | −{ | |
| 1510 | − struct drm_device_dma *dma = dev->dma; | |
| 1511 | − int retcode = 0; | |
| 1512 | − unsigned long virtual; | |
| 1513 | − int i; | |
| 1514 | − | |
| 1515 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 1516 | − return -EOPNOTSUPP; | |
| 1517 | − | |
| 1518 | − if (!drm_core_check_feature(dev, DRIVER_HAVE_DMA)) | |
| 1519 | − return -EOPNOTSUPP; | |
| 1520 | − | |
| 1521 | − if (!dma) | |
| 1522 | − return -EINVAL; | |
| 1523 | − | |
| 1524 | − spin_lock(&dev->buf_lock); | |
| 1525 | − if (atomic_read(&dev->buf_alloc)) { | |
| 1526 | − spin_unlock(&dev->buf_lock); | |
| 1527 | − return -EBUSY; | |
| 1528 | − } | |
| 1529 | − dev->buf_use++; /* Can't allocate more after this call */ | |
| 1530 | − spin_unlock(&dev->buf_lock); | |
| 1531 | − | |
| 1532 | − if (*p >= dma->buf_count) { | |
| 1533 | − if ((dev->agp && (dma->flags & _DRM_DMA_USE_AGP)) | |
| 1534 | − || (drm_core_check_feature(dev, DRIVER_SG) | |
| 1535 | − && (dma->flags & _DRM_DMA_USE_SG))) { | |
| 1536 | − struct drm_local_map *map = dev->agp_buffer_map; | |
| 1537 | − unsigned long token = dev->agp_buffer_token; | |
| 1538 | − | |
| 1539 | − if (!map) { | |
| 1540 | − retcode = -EINVAL; | |
| 1541 | − goto done; | |
| 1542 | − } | |
| 1543 | − virtual = vm_mmap(file_priv->filp, 0, map->size, | |
| 1544 | − PROT_READ | PROT_WRITE, | |
| 1545 | − MAP_SHARED, | |
| 1546 | − token); | |
| 1547 | − } else { | |
| 1548 | − virtual = vm_mmap(file_priv->filp, 0, dma->byte_count, | |
| 1549 | − PROT_READ | PROT_WRITE, | |
| 1550 | − MAP_SHARED, 0); | |
| 1551 | − } | |
| 1552 | − if (virtual > -1024UL) { | |
| 1553 | − /* Real error */ | |
| 1554 | − retcode = (signed long)virtual; | |
| 1555 | − goto done; | |
| 1556 | − } | |
| 1557 | − *v = (void __user *)virtual; | |
| 1558 | − | |
| 1559 | − for (i = 0; i < dma->buf_count; i++) { | |
| 1560 | − if (f(data, i, virtual, dma->buflist[i]) < 0) { | |
| 1561 | − retcode = -EFAULT; | |
| 1562 | − goto done; | |
| 1563 | − } | |
| 1564 | − } | |
| 1565 | − } | |
| 1566 | − done: | |
| 1567 | − *p = dma->buf_count; | |
| 1568 | − DRM_DEBUG("%d buffers, retcode = %d\n", *p, retcode); | |
| 1569 | − | |
| 1570 | − return retcode; | |
| 1571 | −} | |
| 1572 | − | |
| 1573 | −static int map_one_buf(void *data, int idx, unsigned long virtual, | |
| 1574 | − struct drm_buf *buf) | |
| 1575 | −{ | |
| 1576 | − struct drm_buf_map *request = data; | |
| 1577 | − unsigned long address = virtual + buf->offset; /* *** */ | |
| 1578 | − | |
| 1579 | − if (copy_to_user(&request->list[idx].idx, &buf->idx, | |
| 1580 | − sizeof(request->list[0].idx))) | |
| 1581 | − return -EFAULT; | |
| 1582 | − if (copy_to_user(&request->list[idx].total, &buf->total, | |
| 1583 | − sizeof(request->list[0].total))) | |
| 1584 | − return -EFAULT; | |
| 1585 | − if (clear_user(&request->list[idx].used, sizeof(int))) | |
| 1586 | − return -EFAULT; | |
| 1587 | − if (copy_to_user(&request->list[idx].address, &address, | |
| 1588 | − sizeof(address))) | |
| 1589 | − return -EFAULT; | |
| 1590 | − return 0; | |
| 1591 | −} | |
| 1592 | − | |
| 1593 | −int drm_legacy_mapbufs(struct drm_device *dev, void *data, | |
| 1594 | − struct drm_file *file_priv) | |
| 1595 | −{ | |
| 1596 | − struct drm_buf_map *request = data; | |
| 1597 | − | |
| 1598 | − return __drm_legacy_mapbufs(dev, data, &request->count, | |
| 1599 | − &request->virtual, map_one_buf, | |
| 1600 | − file_priv); | |
| 1601 | −} | |
| 1602 | − | |
| 1603 | −int drm_legacy_dma_ioctl(struct drm_device *dev, void *data, | |
| 1604 | − struct drm_file *file_priv) | |
| 1605 | −{ | |
| 1606 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 1607 | − return -EOPNOTSUPP; | |
| 1608 | − | |
| 1609 | − if (dev->driver->dma_ioctl) | |
| 1610 | − return dev->driver->dma_ioctl(dev, data, file_priv); | |
| 1611 | − else | |
| 1612 | − return -EINVAL; | |
| 1613 | −} | |
| 1614 | − | |
| 1615 | −struct drm_local_map *drm_legacy_getsarea(struct drm_device *dev) | |
| 1616 | −{ | |
| 1617 | − struct drm_map_list *entry; | |
| 1618 | − | |
| 1619 | − list_for_each_entry(entry, &dev->maplist, head) { | |
| 1620 | − if (entry->map && entry->map->type == _DRM_SHM && | |
| 1621 | − (entry->map->flags & _DRM_CONTAINS_LOCK)) { | |
| 1622 | − return entry->map; | |
| 1623 | − } | |
| 1624 | − } | |
| 1625 | − return NULL; | |
| 1626 | −} | |
| 1627 | −EXPORT_SYMBOL(drm_legacy_getsarea); | |
| @@ -5,7 +5,6 @@ · 4 unmodified lines | ||
| 5 | 5 | |
| 6 | 6 | #include <linux/iosys-map.h> |
| 7 | 7 | #include <linux/list.h> |
| 8 | −#include <linux/module.h> | |
| 9 | 8 | #include <linux/mutex.h> |
| 10 | 9 | #include <linux/seq_file.h> |
| 11 | 10 | #include <linux/slab.h> |
| @@ -84,16 +83,13 @@ int drm_client_init(struct drm_device *dev, struct drm_client_dev *client, · 72 unmodified lines | ||
| 84 | 83 | if (!drm_core_check_feature(dev, DRIVER_MODESET) || !dev->driver->dumb_create) |
| 85 | 84 | return -EOPNOTSUPP; |
| 86 | 85 | |
| 87 | − if (funcs && !try_module_get(funcs->owner)) | |
| 88 | − return -ENODEV; | |
| 89 | − | |
| 90 | 86 | client->dev = dev; |
| 91 | 87 | client->name = name; |
| 92 | 88 | client->funcs = funcs; |
| 93 | 89 | |
| 94 | 90 | ret = drm_client_modeset_create(client); |
| 95 | 91 | if (ret) |
| 96 | − goto err_put_module; | |
| 92 | + return ret; | |
| 97 | 93 | |
| 98 | 94 | ret = drm_client_open(client); |
| 99 | 95 | if (ret) |
| @@ -105,10 +101,6 @@ int drm_client_init(struct drm_device *dev, struct drm_client_dev *client, · 5 unmodified lines | ||
| 105 | 101 | |
| 106 | 102 | err_free: |
| 107 | 103 | drm_client_modeset_free(client); |
| 108 | −err_put_module: | |
| 109 | − if (funcs) | |
| 110 | − module_put(funcs->owner); | |
| 111 | − | |
| 112 | 104 | return ret; |
| 113 | 105 | } |
| 114 | 106 | EXPORT_SYMBOL(drm_client_init); |
| @@ -177,8 +169,6 @@ void drm_client_release(struct drm_client_dev *client) · 62 unmodified lines | ||
| 177 | 169 | drm_client_modeset_free(client); |
| 178 | 170 | drm_client_close(client); |
| 179 | 171 | drm_dev_put(dev); |
| 180 | − if (client->funcs) | |
| 181 | − module_put(client->funcs->owner); | |
| 182 | 172 | } |
| 183 | 173 | EXPORT_SYMBOL(drm_client_release); |
| 184 | 174 | |
| @@ -1198,6 +1198,12 @@ static const u32 dp_colorspaces = · 1197 unmodified lines | ||
| 1198 | 1198 | * drm_connector_set_path_property(), in the case of DP MST with the |
| 1199 | 1199 | * path property the MST manager created. Userspace cannot change this |
| 1200 | 1200 | * property. |
| 1201 | + * | |
| 1202 | + * In the case of DP MST, the property has the format | |
| 1203 | + * ``mst:<parent>-<ports>`` where ``<parent>`` is the KMS object ID of the | |
| 1204 | + * parent connector and ``<ports>`` is a hyphen-separated list of DP MST | |
| 1205 | + * port numbers. Note, KMS object IDs are not guaranteed to be stable | |
| 1206 | + * across reboots. | |
| 1201 | 1207 | * TILE: |
| 1202 | 1208 | * Connector tile group property to indicate how a set of DRM connector |
| 1203 | 1209 | * compose together into one logical screen. This is used by both high-res |
| @@ -1,513 +0,0 @@ | ||
| 1 | −/* | |
| 2 | − * Legacy: Generic DRM Contexts | |
| 3 | − * | |
| 4 | − * Copyright 1999, 2000 Precision Insight, Inc., Cedar Park, Texas. | |
| 5 | − * Copyright 2000 VA Linux Systems, Inc., Sunnyvale, California. | |
| 6 | − * All Rights Reserved. | |
| 7 | − * | |
| 8 | − * Author: Rickard E. (Rik) Faith <faith@valinux.com> | |
| 9 | − * Author: Gareth Hughes <gareth@valinux.com> | |
| 10 | − * | |
| 11 | − * Permission is hereby granted, free of charge, to any person obtaining a | |
| 12 | − * copy of this software and associated documentation files (the "Software"), | |
| 13 | − * to deal in the Software without restriction, including without limitation | |
| 14 | − * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 15 | − * and/or sell copies of the Software, and to permit persons to whom the | |
| 16 | − * Software is furnished to do so, subject to the following conditions: | |
| 17 | − * | |
| 18 | − * The above copyright notice and this permission notice (including the next | |
| 19 | − * paragraph) shall be included in all copies or substantial portions of the | |
| 20 | − * Software. | |
| 21 | − * | |
| 22 | − * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 23 | − * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 24 | − * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 25 | − * VA LINUX SYSTEMS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 26 | − * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 27 | − * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 28 | − * OTHER DEALINGS IN THE SOFTWARE. | |
| 29 | − */ | |
| 30 | − | |
| 31 | −#include <linux/slab.h> | |
| 32 | −#include <linux/uaccess.h> | |
| 33 | − | |
| 34 | −#include <drm/drm_drv.h> | |
| 35 | −#include <drm/drm_file.h> | |
| 36 | −#include <drm/drm_print.h> | |
| 37 | − | |
| 38 | −#include "drm_legacy.h" | |
| 39 | − | |
| 40 | −struct drm_ctx_list { | |
| 41 | − struct list_head head; | |
| 42 | − drm_context_t handle; | |
| 43 | − struct drm_file *tag; | |
| 44 | −}; | |
| 45 | − | |
| 46 | −/******************************************************************/ | |
| 47 | −/** \name Context bitmap support */ | |
| 48 | −/*@{*/ | |
| 49 | − | |
| 50 | −/* | |
| 51 | − * Free a handle from the context bitmap. | |
| 52 | − * | |
| 53 | − * \param dev DRM device. | |
| 54 | − * \param ctx_handle context handle. | |
| 55 | − * | |
| 56 | − * Clears the bit specified by \p ctx_handle in drm_device::ctx_bitmap and the entry | |
| 57 | − * in drm_device::ctx_idr, while holding the drm_device::struct_mutex | |
| 58 | − * lock. | |
| 59 | − */ | |
| 60 | −void drm_legacy_ctxbitmap_free(struct drm_device * dev, int ctx_handle) | |
| 61 | −{ | |
| 62 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 63 | − return; | |
| 64 | − | |
| 65 | − mutex_lock(&dev->struct_mutex); | |
| 66 | − idr_remove(&dev->ctx_idr, ctx_handle); | |
| 67 | − mutex_unlock(&dev->struct_mutex); | |
| 68 | −} | |
| 69 | − | |
| 70 | −/* | |
| 71 | − * Context bitmap allocation. | |
| 72 | − * | |
| 73 | − * \param dev DRM device. | |
| 74 | − * \return (non-negative) context handle on success or a negative number on failure. | |
| 75 | − * | |
| 76 | − * Allocate a new idr from drm_device::ctx_idr while holding the | |
| 77 | − * drm_device::struct_mutex lock. | |
| 78 | − */ | |
| 79 | −static int drm_legacy_ctxbitmap_next(struct drm_device * dev) | |
| 80 | −{ | |
| 81 | − int ret; | |
| 82 | − | |
| 83 | − mutex_lock(&dev->struct_mutex); | |
| 84 | − ret = idr_alloc(&dev->ctx_idr, NULL, DRM_RESERVED_CONTEXTS, 0, | |
| 85 | − GFP_KERNEL); | |
| 86 | − mutex_unlock(&dev->struct_mutex); | |
| 87 | − return ret; | |
| 88 | −} | |
| 89 | − | |
| 90 | −/* | |
| 91 | − * Context bitmap initialization. | |
| 92 | − * | |
| 93 | − * \param dev DRM device. | |
| 94 | − * | |
| 95 | − * Initialise the drm_device::ctx_idr | |
| 96 | − */ | |
| 97 | −void drm_legacy_ctxbitmap_init(struct drm_device * dev) | |
| 98 | −{ | |
| 99 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 100 | − return; | |
| 101 | − | |
| 102 | − idr_init(&dev->ctx_idr); | |
| 103 | −} | |
| 104 | − | |
| 105 | −/* | |
| 106 | − * Context bitmap cleanup. | |
| 107 | − * | |
| 108 | − * \param dev DRM device. | |
| 109 | − * | |
| 110 | − * Free all idr members using drm_ctx_sarea_free helper function | |
| 111 | − * while holding the drm_device::struct_mutex lock. | |
| 112 | − */ | |
| 113 | −void drm_legacy_ctxbitmap_cleanup(struct drm_device * dev) | |
| 114 | −{ | |
| 115 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 116 | − return; | |
| 117 | − | |
| 118 | − mutex_lock(&dev->struct_mutex); | |
| 119 | − idr_destroy(&dev->ctx_idr); | |
| 120 | − mutex_unlock(&dev->struct_mutex); | |
| 121 | −} | |
| 122 | − | |
| 123 | −/** | |
| 124 | − * drm_legacy_ctxbitmap_flush() - Flush all contexts owned by a file | |
| 125 | − * @dev: DRM device to operate on | |
| 126 | − * @file: Open file to flush contexts for | |
| 127 | − * | |
| 128 | − * This iterates over all contexts on @dev and drops them if they're owned by | |
| 129 | − * @file. Note that after this call returns, new contexts might be added if | |
| 130 | − * the file is still alive. | |
| 131 | − */ | |
| 132 | −void drm_legacy_ctxbitmap_flush(struct drm_device *dev, struct drm_file *file) | |
| 133 | −{ | |
| 134 | − struct drm_ctx_list *pos, *tmp; | |
| 135 | − | |
| 136 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 137 | − return; | |
| 138 | − | |
| 139 | − mutex_lock(&dev->ctxlist_mutex); | |
| 140 | − | |
| 141 | − list_for_each_entry_safe(pos, tmp, &dev->ctxlist, head) { | |
| 142 | − if (pos->tag == file && | |
| 143 | − pos->handle != DRM_KERNEL_CONTEXT) { | |
| 144 | − if (dev->driver->context_dtor) | |
| 145 | − dev->driver->context_dtor(dev, pos->handle); | |
| 146 | − | |
| 147 | − drm_legacy_ctxbitmap_free(dev, pos->handle); | |
| 148 | − list_del(&pos->head); | |
| 149 | − kfree(pos); | |
| 150 | − } | |
| 151 | − } | |
| 152 | − | |
| 153 | − mutex_unlock(&dev->ctxlist_mutex); | |
| 154 | −} | |
| 155 | − | |
| 156 | −/*@}*/ | |
| 157 | − | |
| 158 | −/******************************************************************/ | |
| 159 | −/** \name Per Context SAREA Support */ | |
| 160 | −/*@{*/ | |
| 161 | − | |
| 162 | −/* | |
| 163 | − * Get per-context SAREA. | |
| 164 | − * | |
| 165 | − * \param inode device inode. | |
| 166 | − * \param file_priv DRM file private. | |
| 167 | − * \param cmd command. | |
| 168 | − * \param arg user argument pointing to a drm_ctx_priv_map structure. | |
| 169 | − * \return zero on success or a negative number on failure. | |
| 170 | − * | |
| 171 | − * Gets the map from drm_device::ctx_idr with the handle specified and | |
| 172 | − * returns its handle. | |
| 173 | − */ | |
| 174 | −int drm_legacy_getsareactx(struct drm_device *dev, void *data, | |
| 175 | − struct drm_file *file_priv) | |
| 176 | −{ | |
| 177 | − struct drm_ctx_priv_map *request = data; | |
| 178 | − struct drm_local_map *map; | |
| 179 | − struct drm_map_list *_entry; | |
| 180 | − | |
| 181 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 182 | − return -EOPNOTSUPP; | |
| 183 | − | |
| 184 | − mutex_lock(&dev->struct_mutex); | |
| 185 | − | |
| 186 | − map = idr_find(&dev->ctx_idr, request->ctx_id); | |
| 187 | − if (!map) { | |
| 188 | − mutex_unlock(&dev->struct_mutex); | |
| 189 | − return -EINVAL; | |
| 190 | − } | |
| 191 | − | |
| 192 | − request->handle = NULL; | |
| 193 | − list_for_each_entry(_entry, &dev->maplist, head) { | |
| 194 | − if (_entry->map == map) { | |
| 195 | − request->handle = | |
| 196 | − (void *)(unsigned long)_entry->user_token; | |
| 197 | − break; | |
| 198 | − } | |
| 199 | − } | |
| 200 | − | |
| 201 | − mutex_unlock(&dev->struct_mutex); | |
| 202 | − | |
| 203 | − if (request->handle == NULL) | |
| 204 | − return -EINVAL; | |
| 205 | − | |
| 206 | − return 0; | |
| 207 | −} | |
| 208 | − | |
| 209 | −/* | |
| 210 | − * Set per-context SAREA. | |
| 211 | − * | |
| 212 | − * \param inode device inode. | |
| 213 | − * \param file_priv DRM file private. | |
| 214 | − * \param cmd command. | |
| 215 | − * \param arg user argument pointing to a drm_ctx_priv_map structure. | |
| 216 | − * \return zero on success or a negative number on failure. | |
| 217 | − * | |
| 218 | − * Searches the mapping specified in \p arg and update the entry in | |
| 219 | − * drm_device::ctx_idr with it. | |
| 220 | − */ | |
| 221 | −int drm_legacy_setsareactx(struct drm_device *dev, void *data, | |
| 222 | − struct drm_file *file_priv) | |
| 223 | −{ | |
| 224 | − struct drm_ctx_priv_map *request = data; | |
| 225 | − struct drm_local_map *map = NULL; | |
| 226 | − struct drm_map_list *r_list = NULL; | |
| 227 | − | |
| 228 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 229 | − return -EOPNOTSUPP; | |
| 230 | − | |
| 231 | − mutex_lock(&dev->struct_mutex); | |
| 232 | − list_for_each_entry(r_list, &dev->maplist, head) { | |
| 233 | − if (r_list->map | |
| 234 | − && r_list->user_token == (unsigned long) request->handle) | |
| 235 | − goto found; | |
| 236 | − } | |
| 237 | − bad: | |
| 238 | − mutex_unlock(&dev->struct_mutex); | |
| 239 | − return -EINVAL; | |
| 240 | − | |
| 241 | − found: | |
| 242 | − map = r_list->map; | |
| 243 | − if (!map) | |
| 244 | − goto bad; | |
| 245 | − | |
| 246 | − if (IS_ERR(idr_replace(&dev->ctx_idr, map, request->ctx_id))) | |
| 247 | − goto bad; | |
| 248 | − | |
| 249 | − mutex_unlock(&dev->struct_mutex); | |
| 250 | − | |
| 251 | − return 0; | |
| 252 | −} | |
| 253 | − | |
| 254 | −/*@}*/ | |
| 255 | − | |
| 256 | −/******************************************************************/ | |
| 257 | −/** \name The actual DRM context handling routines */ | |
| 258 | −/*@{*/ | |
| 259 | − | |
| 260 | −/* | |
| 261 | − * Switch context. | |
| 262 | − * | |
| 263 | − * \param dev DRM device. | |
| 264 | − * \param old old context handle. | |
| 265 | − * \param new new context handle. | |
| 266 | − * \return zero on success or a negative number on failure. | |
| 267 | − * | |
| 268 | − * Attempt to set drm_device::context_flag. | |
| 269 | − */ | |
| 270 | −static int drm_context_switch(struct drm_device * dev, int old, int new) | |
| 271 | −{ | |
| 272 | − if (test_and_set_bit(0, &dev->context_flag)) { | |
| 273 | − DRM_ERROR("Reentering -- FIXME\n"); | |
| 274 | − return -EBUSY; | |
| 275 | − } | |
| 276 | − | |
| 277 | − DRM_DEBUG("Context switch from %d to %d\n", old, new); | |
| 278 | − | |
| 279 | − if (new == dev->last_context) { | |
| 280 | − clear_bit(0, &dev->context_flag); | |
| 281 | − return 0; | |
| 282 | − } | |
| 283 | − | |
| 284 | − return 0; | |
| 285 | −} | |
| 286 | − | |
| 287 | −/* | |
| 288 | − * Complete context switch. | |
| 289 | − * | |
| 290 | − * \param dev DRM device. | |
| 291 | − * \param new new context handle. | |
| 292 | − * \return zero on success or a negative number on failure. | |
| 293 | − * | |
| 294 | − * Updates drm_device::last_context and drm_device::last_switch. Verifies the | |
| 295 | − * hardware lock is held, clears the drm_device::context_flag and wakes up | |
| 296 | − * drm_device::context_wait. | |
| 297 | − */ | |
| 298 | −static int drm_context_switch_complete(struct drm_device *dev, | |
| 299 | − struct drm_file *file_priv, int new) | |
| 300 | −{ | |
| 301 | − dev->last_context = new; /* PRE/POST: This is the _only_ writer. */ | |
| 302 | − | |
| 303 | − if (!_DRM_LOCK_IS_HELD(file_priv->master->lock.hw_lock->lock)) { | |
| 304 | − DRM_ERROR("Lock isn't held after context switch\n"); | |
| 305 | − } | |
| 306 | − | |
| 307 | − /* If a context switch is ever initiated | |
| 308 | − when the kernel holds the lock, release | |
| 309 | − that lock here. | |
| 310 | − */ | |
| 311 | − clear_bit(0, &dev->context_flag); | |
| 312 | − | |
| 313 | − return 0; | |
| 314 | −} | |
| 315 | − | |
| 316 | −/* | |
| 317 | − * Reserve contexts. | |
| 318 | − * | |
| 319 | − * \param inode device inode. | |
| 320 | − * \param file_priv DRM file private. | |
| 321 | − * \param cmd command. | |
| 322 | − * \param arg user argument pointing to a drm_ctx_res structure. | |
| 323 | − * \return zero on success or a negative number on failure. | |
| 324 | − */ | |
| 325 | −int drm_legacy_resctx(struct drm_device *dev, void *data, | |
| 326 | − struct drm_file *file_priv) | |
| 327 | −{ | |
| 328 | − struct drm_ctx_res *res = data; | |
| 329 | − struct drm_ctx ctx; | |
| 330 | − int i; | |
| 331 | − | |
| 332 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 333 | − return -EOPNOTSUPP; | |
| 334 | − | |
| 335 | − if (res->count >= DRM_RESERVED_CONTEXTS) { | |
| 336 | − memset(&ctx, 0, sizeof(ctx)); | |
| 337 | − for (i = 0; i < DRM_RESERVED_CONTEXTS; i++) { | |
| 338 | − ctx.handle = i; | |
| 339 | − if (copy_to_user(&res->contexts[i], &ctx, sizeof(ctx))) | |
| 340 | − return -EFAULT; | |
| 341 | − } | |
| 342 | − } | |
| 343 | − res->count = DRM_RESERVED_CONTEXTS; | |
| 344 | − | |
| 345 | − return 0; | |
| 346 | −} | |
| 347 | − | |
| 348 | −/* | |
| 349 | − * Add context. | |
| 350 | − * | |
| 351 | − * \param inode device inode. | |
| 352 | − * \param file_priv DRM file private. | |
| 353 | − * \param cmd command. | |
| 354 | − * \param arg user argument pointing to a drm_ctx structure. | |
| 355 | − * \return zero on success or a negative number on failure. | |
| 356 | − * | |
| 357 | − * Get a new handle for the context and copy to userspace. | |
| 358 | − */ | |
| 359 | −int drm_legacy_addctx(struct drm_device *dev, void *data, | |
| 360 | − struct drm_file *file_priv) | |
| 361 | −{ | |
| 362 | − struct drm_ctx_list *ctx_entry; | |
| 363 | − struct drm_ctx *ctx = data; | |
| 364 | − int tmp_handle; | |
| 365 | − | |
| 366 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 367 | − return -EOPNOTSUPP; | |
| 368 | − | |
| 369 | − tmp_handle = drm_legacy_ctxbitmap_next(dev); | |
| 370 | − if (tmp_handle == DRM_KERNEL_CONTEXT) { | |
| 371 | − /* Skip kernel's context and get a new one. */ | |
| 372 | − tmp_handle = drm_legacy_ctxbitmap_next(dev); | |
| 373 | − } | |
| 374 | − DRM_DEBUG("%d\n", tmp_handle); | |
| 375 | − if (tmp_handle < 0) { | |
| 376 | − DRM_DEBUG("Not enough free contexts.\n"); | |
| 377 | − /* Should this return -EBUSY instead? */ | |
| 378 | − return tmp_handle; | |
| 379 | − } | |
| 380 | − | |
| 381 | − ctx->handle = tmp_handle; | |
| 382 | − | |
| 383 | − ctx_entry = kmalloc(sizeof(*ctx_entry), GFP_KERNEL); | |
| 384 | − if (!ctx_entry) { | |
| 385 | − DRM_DEBUG("out of memory\n"); | |
| 386 | − return -ENOMEM; | |
| 387 | − } | |
| 388 | − | |
| 389 | − INIT_LIST_HEAD(&ctx_entry->head); | |
| 390 | − ctx_entry->handle = ctx->handle; | |
| 391 | − ctx_entry->tag = file_priv; | |
| 392 | − | |
| 393 | − mutex_lock(&dev->ctxlist_mutex); | |
| 394 | − list_add(&ctx_entry->head, &dev->ctxlist); | |
| 395 | − mutex_unlock(&dev->ctxlist_mutex); | |
| 396 | − | |
| 397 | − return 0; | |
| 398 | −} | |
| 399 | − | |
| 400 | −/* | |
| 401 | − * Get context. | |
| 402 | − * | |
| 403 | − * \param inode device inode. | |
| 404 | − * \param file_priv DRM file private. | |
| 405 | − * \param cmd command. | |
| 406 | − * \param arg user argument pointing to a drm_ctx structure. | |
| 407 | − * \return zero on success or a negative number on failure. | |
| 408 | − */ | |
| 409 | −int drm_legacy_getctx(struct drm_device *dev, void *data, | |
| 410 | − struct drm_file *file_priv) | |
| 411 | −{ | |
| 412 | − struct drm_ctx *ctx = data; | |
| 413 | − | |
| 414 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 415 | − return -EOPNOTSUPP; | |
| 416 | − | |
| 417 | − /* This is 0, because we don't handle any context flags */ | |
| 418 | − ctx->flags = 0; | |
| 419 | − | |
| 420 | − return 0; | |
| 421 | −} | |
| 422 | − | |
| 423 | −/* | |
| 424 | − * Switch context. | |
| 425 | − * | |
| 426 | − * \param inode device inode. | |
| 427 | − * \param file_priv DRM file private. | |
| 428 | − * \param cmd command. | |
| 429 | − * \param arg user argument pointing to a drm_ctx structure. | |
| 430 | − * \return zero on success or a negative number on failure. | |
| 431 | − * | |
| 432 | − * Calls context_switch(). | |
| 433 | − */ | |
| 434 | −int drm_legacy_switchctx(struct drm_device *dev, void *data, | |
| 435 | − struct drm_file *file_priv) | |
| 436 | −{ | |
| 437 | − struct drm_ctx *ctx = data; | |
| 438 | − | |
| 439 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 440 | − return -EOPNOTSUPP; | |
| 441 | − | |
| 442 | − DRM_DEBUG("%d\n", ctx->handle); | |
| 443 | − return drm_context_switch(dev, dev->last_context, ctx->handle); | |
| 444 | −} | |
| 445 | − | |
| 446 | −/* | |
| 447 | − * New context. | |
| 448 | − * | |
| 449 | − * \param inode device inode. | |
| 450 | − * \param file_priv DRM file private. | |
| 451 | − * \param cmd command. | |
| 452 | − * \param arg user argument pointing to a drm_ctx structure. | |
| 453 | − * \return zero on success or a negative number on failure. | |
| 454 | − * | |
| 455 | − * Calls context_switch_complete(). | |
| 456 | − */ | |
| 457 | −int drm_legacy_newctx(struct drm_device *dev, void *data, | |
| 458 | − struct drm_file *file_priv) | |
| 459 | −{ | |
| 460 | − struct drm_ctx *ctx = data; | |
| 461 | − | |
| 462 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 463 | − return -EOPNOTSUPP; | |
| 464 | − | |
| 465 | − DRM_DEBUG("%d\n", ctx->handle); | |
| 466 | − drm_context_switch_complete(dev, file_priv, ctx->handle); | |
| 467 | − | |
| 468 | − return 0; | |
| 469 | −} | |
| 470 | − | |
| 471 | −/* | |
| 472 | − * Remove context. | |
| 473 | − * | |
| 474 | − * \param inode device inode. | |
| 475 | − * \param file_priv DRM file private. | |
| 476 | − * \param cmd command. | |
| 477 | − * \param arg user argument pointing to a drm_ctx structure. | |
| 478 | − * \return zero on success or a negative number on failure. | |
| 479 | − * | |
| 480 | − * If not the special kernel context, calls ctxbitmap_free() to free the specified context. | |
| 481 | − */ | |
| 482 | −int drm_legacy_rmctx(struct drm_device *dev, void *data, | |
| 483 | − struct drm_file *file_priv) | |
| 484 | −{ | |
| 485 | − struct drm_ctx *ctx = data; | |
| 486 | − | |
| 487 | − if (!drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 488 | − return -EOPNOTSUPP; | |
| 489 | − | |
| 490 | − DRM_DEBUG("%d\n", ctx->handle); | |
| 491 | − if (ctx->handle != DRM_KERNEL_CONTEXT) { | |
| 492 | − if (dev->driver->context_dtor) | |
| 493 | − dev->driver->context_dtor(dev, ctx->handle); | |
| 494 | − drm_legacy_ctxbitmap_free(dev, ctx->handle); | |
| 495 | − } | |
| 496 | − | |
| 497 | − mutex_lock(&dev->ctxlist_mutex); | |
| 498 | − if (!list_empty(&dev->ctxlist)) { | |
| 499 | − struct drm_ctx_list *pos, *n; | |
| 500 | − | |
| 501 | − list_for_each_entry_safe(pos, n, &dev->ctxlist, head) { | |
| 502 | − if (pos->handle == ctx->handle) { | |
| 503 | − list_del(&pos->head); | |
| 504 | − kfree(pos); | |
| 505 | − } | |
| 506 | − } | |
| 507 | − } | |
| 508 | − mutex_unlock(&dev->ctxlist_mutex); | |
| 509 | − | |
| 510 | − return 0; | |
| 511 | −} | |
| 512 | − | |
| 513 | −/*@}*/ | |
| @@ -439,11 +439,8 @@ EXPORT_SYMBOL(drm_crtc_helper_set_mode); · 438 unmodified lines | ||
| 439 | 439 | * @state: atomic state object |
| 440 | 440 | * |
| 441 | 441 | * Provides a default CRTC-state check handler for CRTCs that only have |
| 442 | − * one primary plane attached to it. | |
| 443 | − * | |
| 444 | − * This is often the case for the CRTC of simple framebuffers. See also | |
| 445 | − * drm_plane_helper_atomic_check() for the respective plane-state check | |
| 446 | − * helper function. | |
| 442 | + * one primary plane attached to it. This is often the case for the CRTC | |
| 443 | + * of simple framebuffers. | |
| 447 | 444 | * |
| 448 | 445 | * RETURNS: |
| 449 | 446 | * Zero on success, or an errno code otherwise. |
| @@ -222,6 +222,8 @@ int drm_mode_addfb2_ioctl(struct drm_device *dev, · 221 unmodified lines | ||
| 222 | 222 | void *data, struct drm_file *file_priv); |
| 223 | 223 | int drm_mode_rmfb_ioctl(struct drm_device *dev, |
| 224 | 224 | void *data, struct drm_file *file_priv); |
| 225 | +int drm_mode_closefb_ioctl(struct drm_device *dev, | |
| 226 | + void *data, struct drm_file *file_priv); | |
| 225 | 227 | int drm_mode_getfb(struct drm_device *dev, |
| 226 | 228 | void *data, struct drm_file *file_priv); |
| 227 | 229 | int drm_mode_getfb2_ioctl(struct drm_device *dev, |
| @@ -251,7 +253,7 @@ int drm_atomic_set_property(struct drm_atomic_state *state, · 23 unmodified lines | ||
| 251 | 253 | struct drm_file *file_priv, |
| 252 | 254 | struct drm_mode_object *obj, |
| 253 | 255 | struct drm_property *prop, |
| 254 | − uint64_t prop_value); | |
| 256 | + u64 prop_value, bool async_flip); | |
| 255 | 257 | int drm_atomic_get_property(struct drm_mode_object *obj, |
| 256 | 258 | struct drm_property *property, uint64_t *val); |
| 257 | 259 | |
| @@ -241,7 +241,8 @@ drm_atomic_helper_damage_iter_init(struct drm_atomic_helper_damage_iter *iter, · 240 unmodified lines | ||
| 241 | 241 | iter->plane_src.x2 = (src.x2 >> 16) + !!(src.x2 & 0xFFFF); |
| 242 | 242 | iter->plane_src.y2 = (src.y2 >> 16) + !!(src.y2 & 0xFFFF); |
| 243 | 243 | |
| 244 | − if (!iter->clips || !drm_rect_equals(&state->src, &old_state->src)) { | |
| 244 | + if (!iter->clips || state->ignore_damage_clips || | |
| 245 | + !drm_rect_equals(&state->src, &old_state->src)) { | |
| 245 | 246 | iter->clips = NULL; |
| 246 | 247 | iter->num_clips = 0; |
| 247 | 248 | iter->full_update = true; |
| @@ -314,10 +314,8 @@ void drm_debugfs_dev_register(struct drm_device *dev) · 313 unmodified lines | ||
| 314 | 314 | drm_framebuffer_debugfs_init(dev); |
| 315 | 315 | drm_client_debugfs_init(dev); |
| 316 | 316 | } |
| 317 | − if (drm_drv_uses_atomic_modeset(dev)) { | |
| 317 | + if (drm_drv_uses_atomic_modeset(dev)) | |
| 318 | 318 | drm_atomic_debugfs_init(dev); |
| 319 | − drm_bridge_debugfs_init(dev); | |
| 320 | − } | |
| 321 | 319 | } |
| 322 | 320 | |
| 323 | 321 | int drm_debugfs_register(struct drm_minor *minor, int minor_id, |
| @@ -589,4 +587,65 @@ void drm_debugfs_crtc_remove(struct drm_crtc *crtc) · 265 unmodified lines | ||
| 589 | 587 | crtc->debugfs_entry = NULL; |
| 590 | 588 | } |
| 591 | 589 | |
| 590 | +static int bridges_show(struct seq_file *m, void *data) | |
| 591 | +{ | |
| 592 | + struct drm_encoder *encoder = m->private; | |
| 593 | + struct drm_printer p = drm_seq_file_printer(m); | |
| 594 | + struct drm_bridge *bridge; | |
| 595 | + unsigned int idx = 0; | |
| 596 | + | |
| 597 | + drm_for_each_bridge_in_chain(encoder, bridge) { | |
| 598 | + drm_printf(&p, "bridge[%d]: %ps\n", idx++, bridge->funcs); | |
| 599 | + drm_printf(&p, "\ttype: [%d] %s\n", | |
| 600 | + bridge->type, | |
| 601 | + drm_get_connector_type_name(bridge->type)); | |
| 602 | +#ifdef CONFIG_OF | |
| 603 | + if (bridge->of_node) | |
| 604 | + drm_printf(&p, "\tOF: %pOFfc\n", bridge->of_node); | |
| 605 | +#endif | |
| 606 | + drm_printf(&p, "\tops: [0x%x]", bridge->ops); | |
| 607 | + if (bridge->ops & DRM_BRIDGE_OP_DETECT) | |
| 608 | + drm_puts(&p, " detect"); | |
| 609 | + if (bridge->ops & DRM_BRIDGE_OP_EDID) | |
| 610 | + drm_puts(&p, " edid"); | |
| 611 | + if (bridge->ops & DRM_BRIDGE_OP_HPD) | |
| 612 | + drm_puts(&p, " hpd"); | |
| 613 | + if (bridge->ops & DRM_BRIDGE_OP_MODES) | |
| 614 | + drm_puts(&p, " modes"); | |
| 615 | + drm_puts(&p, "\n"); | |
| 616 | + } | |
| 617 | + | |
| 618 | + return 0; | |
| 619 | +} | |
| 620 | +DEFINE_SHOW_ATTRIBUTE(bridges); | |
| 621 | + | |
| 622 | +void drm_debugfs_encoder_add(struct drm_encoder *encoder) | |
| 623 | +{ | |
| 624 | + struct drm_minor *minor = encoder->dev->primary; | |
| 625 | + struct dentry *root; | |
| 626 | + char *name; | |
| 627 | + | |
| 628 | + name = kasprintf(GFP_KERNEL, "encoder-%d", encoder->index); | |
| 629 | + if (!name) | |
| 630 | + return; | |
| 631 | + | |
| 632 | + root = debugfs_create_dir(name, minor->debugfs_root); | |
| 633 | + kfree(name); | |
| 634 | + | |
| 635 | + encoder->debugfs_entry = root; | |
| 636 | + | |
| 637 | + /* bridges list */ | |
| 638 | + debugfs_create_file("bridges", 0444, root, encoder, | |
| 639 | + &bridges_fops); | |
| 640 | + | |
| 641 | + if (encoder->funcs && encoder->funcs->debugfs_init) | |
| 642 | + encoder->funcs->debugfs_init(encoder, root); | |
| 643 | +} | |
| 644 | + | |
| 645 | +void drm_debugfs_encoder_remove(struct drm_encoder *encoder) | |
| 646 | +{ | |
| 647 | + debugfs_remove_recursive(encoder->debugfs_entry); | |
| 648 | + encoder->debugfs_entry = NULL; | |
| 649 | +} | |
| 650 | + | |
| 592 | 651 | #endif /* CONFIG_DEBUG_FS */ |
| @@ -1,178 +0,0 @@ | ||
| 1 | −/* | |
| 2 | − * \file drm_dma.c | |
| 3 | − * DMA IOCTL and function support | |
| 4 | − * | |
| 5 | − * \author Rickard E. (Rik) Faith <faith@valinux.com> | |
| 6 | − * \author Gareth Hughes <gareth@valinux.com> | |
| 7 | − */ | |
| 8 | − | |
| 9 | −/* | |
| 10 | − * Created: Fri Mar 19 14:30:16 1999 by faith@valinux.com | |
| 11 | − * | |
| 12 | − * Copyright 1999, 2000 Precision Insight, Inc., Cedar Park, Texas. | |
| 13 | − * Copyright 2000 VA Linux Systems, Inc., Sunnyvale, California. | |
| 14 | − * All Rights Reserved. | |
| 15 | − * | |
| 16 | − * Permission is hereby granted, free of charge, to any person obtaining a | |
| 17 | − * copy of this software and associated documentation files (the "Software"), | |
| 18 | − * to deal in the Software without restriction, including without limitation | |
| 19 | − * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 20 | − * and/or sell copies of the Software, and to permit persons to whom the | |
| 21 | − * Software is furnished to do so, subject to the following conditions: | |
| 22 | − * | |
| 23 | − * The above copyright notice and this permission notice (including the next | |
| 24 | − * paragraph) shall be included in all copies or substantial portions of the | |
| 25 | − * Software. | |
| 26 | − * | |
| 27 | − * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 28 | − * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 29 | − * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 30 | − * VA LINUX SYSTEMS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 31 | − * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 32 | − * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 33 | − * OTHER DEALINGS IN THE SOFTWARE. | |
| 34 | − */ | |
| 35 | − | |
| 36 | −#include <linux/export.h> | |
| 37 | −#include <linux/pci.h> | |
| 38 | − | |
| 39 | −#include <drm/drm_drv.h> | |
| 40 | −#include <drm/drm_print.h> | |
| 41 | − | |
| 42 | −#include "drm_legacy.h" | |
| 43 | − | |
| 44 | −/** | |
| 45 | − * drm_legacy_dma_setup() - Initialize the DMA data. | |
| 46 | − * | |
| 47 | − * @dev: DRM device. | |
| 48 | − * Return: zero on success or a negative value on failure. | |
| 49 | − * | |
| 50 | − * Allocate and initialize a drm_device_dma structure. | |
| 51 | − */ | |
| 52 | −int drm_legacy_dma_setup(struct drm_device *dev) | |
| 53 | −{ | |
| 54 | − int i; | |
| 55 | − | |
| 56 | − if (!drm_core_check_feature(dev, DRIVER_HAVE_DMA) || | |
| 57 | − !drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 58 | − return 0; | |
| 59 | − | |
| 60 | − dev->buf_use = 0; | |
| 61 | − atomic_set(&dev->buf_alloc, 0); | |
| 62 | − | |
| 63 | − dev->dma = kzalloc(sizeof(*dev->dma), GFP_KERNEL); | |
| 64 | − if (!dev->dma) | |
| 65 | − return -ENOMEM; | |
| 66 | − | |
| 67 | − for (i = 0; i <= DRM_MAX_ORDER; i++) | |
| 68 | − memset(&dev->dma->bufs[i], 0, sizeof(dev->dma->bufs[0])); | |
| 69 | − | |
| 70 | − return 0; | |
| 71 | −} | |
| 72 | − | |
| 73 | −/** | |
| 74 | − * drm_legacy_dma_takedown() - Cleanup the DMA resources. | |
| 75 | − * | |
| 76 | − * @dev: DRM device. | |
| 77 | − * | |
| 78 | − * Free all pages associated with DMA buffers, the buffers and pages lists, and | |
| 79 | − * finally the drm_device::dma structure itself. | |
| 80 | − */ | |
| 81 | −void drm_legacy_dma_takedown(struct drm_device *dev) | |
| 82 | −{ | |
| 83 | − struct drm_device_dma *dma = dev->dma; | |
| 84 | − drm_dma_handle_t *dmah; | |
| 85 | − int i, j; | |
| 86 | − | |
| 87 | − if (!drm_core_check_feature(dev, DRIVER_HAVE_DMA) || | |
| 88 | − !drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 89 | − return; | |
| 90 | − | |
| 91 | − if (!dma) | |
| 92 | − return; | |
| 93 | − | |
| 94 | − /* Clear dma buffers */ | |
| 95 | − for (i = 0; i <= DRM_MAX_ORDER; i++) { | |
| 96 | − if (dma->bufs[i].seg_count) { | |
| 97 | − DRM_DEBUG("order %d: buf_count = %d," | |
| 98 | − " seg_count = %d\n", | |
| 99 | − i, | |
| 100 | − dma->bufs[i].buf_count, | |
| 101 | − dma->bufs[i].seg_count); | |
| 102 | − for (j = 0; j < dma->bufs[i].seg_count; j++) { | |
| 103 | − if (dma->bufs[i].seglist[j]) { | |
| 104 | − dmah = dma->bufs[i].seglist[j]; | |
| 105 | − dma_free_coherent(dev->dev, | |
| 106 | − dmah->size, | |
| 107 | − dmah->vaddr, | |
| 108 | − dmah->busaddr); | |
| 109 | − kfree(dmah); | |
| 110 | − } | |
| 111 | − } | |
| 112 | − kfree(dma->bufs[i].seglist); | |
| 113 | − } | |
| 114 | − if (dma->bufs[i].buf_count) { | |
| 115 | − for (j = 0; j < dma->bufs[i].buf_count; j++) { | |
| 116 | − kfree(dma->bufs[i].buflist[j].dev_private); | |
| 117 | − } | |
| 118 | − kfree(dma->bufs[i].buflist); | |
| 119 | − } | |
| 120 | − } | |
| 121 | − | |
| 122 | − kfree(dma->buflist); | |
| 123 | − kfree(dma->pagelist); | |
| 124 | − kfree(dev->dma); | |
| 125 | − dev->dma = NULL; | |
| 126 | −} | |
| 127 | − | |
| 128 | −/** | |
| 129 | − * drm_legacy_free_buffer() - Free a buffer. | |
| 130 | − * | |
| 131 | − * @dev: DRM device. | |
| 132 | − * @buf: buffer to free. | |
| 133 | − * | |
| 134 | − * Resets the fields of \p buf. | |
| 135 | − */ | |
| 136 | −void drm_legacy_free_buffer(struct drm_device *dev, struct drm_buf * buf) | |
| 137 | −{ | |
| 138 | − if (!buf) | |
| 139 | − return; | |
| 140 | − | |
| 141 | − buf->waiting = 0; | |
| 142 | − buf->pending = 0; | |
| 143 | − buf->file_priv = NULL; | |
| 144 | − buf->used = 0; | |
| 145 | −} | |
| 146 | − | |
| 147 | −/** | |
| 148 | − * drm_legacy_reclaim_buffers() - Reclaim the buffers. | |
| 149 | − * | |
| 150 | − * @dev: DRM device. | |
| 151 | − * @file_priv: DRM file private. | |
| 152 | − * | |
| 153 | − * Frees each buffer associated with \p file_priv not already on the hardware. | |
| 154 | − */ | |
| 155 | −void drm_legacy_reclaim_buffers(struct drm_device *dev, | |
| 156 | − struct drm_file *file_priv) | |
| 157 | −{ | |
| 158 | − struct drm_device_dma *dma = dev->dma; | |
| 159 | − int i; | |
| 160 | − | |
| 161 | − if (!dma) | |
| 162 | − return; | |
| 163 | − for (i = 0; i < dma->buf_count; i++) { | |
| 164 | − if (dma->buflist[i]->file_priv == file_priv) { | |
| 165 | − switch (dma->buflist[i]->list) { | |
| 166 | − case DRM_LIST_NONE: | |
| 167 | − drm_legacy_free_buffer(dev, dma->buflist[i]); | |
| 168 | − break; | |
| 169 | − case DRM_LIST_WAIT: | |
| 170 | − dma->buflist[i]->list = DRM_LIST_RECLAIM; | |
| 171 | − break; | |
| 172 | − default: | |
| 173 | − /* Buffer already on hardware. */ | |
| 174 | − break; | |
| 175 | − } | |
| 176 | − } | |
| 177 | − } | |
| 178 | −} | |
| @@ -48,7 +48,6 @@ · 47 unmodified lines | ||
| 48 | 48 | |
| 49 | 49 | #include "drm_crtc_internal.h" |
| 50 | 50 | #include "drm_internal.h" |
| 51 | −#include "drm_legacy.h" | |
| 52 | 51 | |
| 53 | 52 | MODULE_AUTHOR("Gareth Hughes, Leif Delgass, José Fonseca, Jon Smirl"); |
| 54 | 53 | MODULE_DESCRIPTION("DRM shared core routines"); |
| @@ -585,8 +584,6 @@ static void drm_fs_inode_free(struct inode *inode) · 530 unmodified lines | ||
| 585 | 584 | |
| 586 | 585 | static void drm_dev_init_release(struct drm_device *dev, void *res) |
| 587 | 586 | { |
| 588 | − drm_legacy_ctxbitmap_cleanup(dev); | |
| 589 | − drm_legacy_remove_map_hash(dev); | |
| 590 | 587 | drm_fs_inode_free(dev->anon_inode); |
| 591 | 588 | |
| 592 | 589 | put_device(dev->dev); |
| @@ -597,7 +594,6 @@ static void drm_dev_init_release(struct drm_device *dev, void *res) · 4 unmodified lines | ||
| 597 | 594 | mutex_destroy(&dev->clientlist_mutex); |
| 598 | 595 | mutex_destroy(&dev->filelist_mutex); |
| 599 | 596 | mutex_destroy(&dev->struct_mutex); |
| 600 | − drm_legacy_destroy_members(dev); | |
| 601 | 597 | } |
| 602 | 598 | |
| 603 | 599 | static int drm_dev_init(struct drm_device *dev, |
| @@ -632,7 +628,6 @@ static int drm_dev_init(struct drm_device *dev, · 28 unmodified lines | ||
| 632 | 628 | return -EINVAL; |
| 633 | 629 | } |
| 634 | 630 | |
| 635 | − drm_legacy_init_members(dev); | |
| 636 | 631 | INIT_LIST_HEAD(&dev->filelist); |
| 637 | 632 | INIT_LIST_HEAD(&dev->filelist_internal); |
| 638 | 633 | INIT_LIST_HEAD(&dev->clientlist); |
| @@ -673,12 +668,6 @@ static int drm_dev_init(struct drm_device *dev, · 34 unmodified lines | ||
| 673 | 668 | goto err; |
| 674 | 669 | } |
| 675 | 670 | |
| 676 | − ret = drm_legacy_create_map_hash(dev); | |
| 677 | − if (ret) | |
| 678 | − goto err; | |
| 679 | − | |
| 680 | − drm_legacy_ctxbitmap_init(dev); | |
| 681 | − | |
| 682 | 671 | if (drm_core_check_feature(dev, DRIVER_GEM)) { |
| 683 | 672 | ret = drm_gem_init(dev); |
| 684 | 673 | if (ret) { |
| @@ -949,8 +938,11 @@ int drm_dev_register(struct drm_device *dev, unsigned long flags) · 264 unmodified lines | ||
| 949 | 938 | goto err_minors; |
| 950 | 939 | } |
| 951 | 940 | |
| 952 | − if (drm_core_check_feature(dev, DRIVER_MODESET)) | |
| 953 | − drm_modeset_register_all(dev); | |
| 941 | + if (drm_core_check_feature(dev, DRIVER_MODESET)) { | |
| 942 | + ret = drm_modeset_register_all(dev); | |
| 943 | + if (ret) | |
| 944 | + goto err_unload; | |
| 945 | + } | |
| 954 | 946 | |
| 955 | 947 | DRM_INFO("Initialized %s %d.%d.%d %s for %s on minor %d\n", |
| 956 | 948 | driver->name, driver->major, driver->minor, |
| @@ -960,6 +952,9 @@ int drm_dev_register(struct drm_device *dev, unsigned long flags) · 3 unmodified lines | ||
| 960 | 952 | |
| 961 | 953 | goto out_unlock; |
| 962 | 954 | |
| 955 | +err_unload: | |
| 956 | + if (dev->driver->unload) | |
| 957 | + dev->driver->unload(dev); | |
| 963 | 958 | err_minors: |
| 964 | 959 | remove_compat_control_link(dev); |
| 965 | 960 | drm_minor_unregister(dev, DRM_MINOR_ACCEL); |
| @@ -990,9 +985,6 @@ EXPORT_SYMBOL(drm_dev_register); · 24 unmodified lines | ||
| 990 | 985 | */ |
| 991 | 986 | void drm_dev_unregister(struct drm_device *dev) |
| 992 | 987 | { |
| 993 | − if (drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 994 | − drm_lastclose(dev); | |
| 995 | − | |
| 996 | 988 | dev->registered = false; |
| 997 | 989 | |
| 998 | 990 | drm_client_dev_unregister(dev); |
| @@ -1003,9 +995,6 @@ void drm_dev_unregister(struct drm_device *dev) · 4 unmodified lines | ||
| 1003 | 995 | if (dev->driver->unload) |
| 1004 | 996 | dev->driver->unload(dev); |
| 1005 | 997 | |
| 1006 | − drm_legacy_pci_agp_destroy(dev); | |
| 1007 | − drm_legacy_rmmaps(dev); | |
| 1008 | − | |
| 1009 | 998 | remove_compat_control_link(dev); |
| 1010 | 999 | drm_minor_unregister(dev, DRM_MINOR_ACCEL); |
| 1011 | 1000 | drm_minor_unregister(dev, DRM_MINOR_PRIMARY); |
| @@ -41,10 +41,12 @@ · 40 unmodified lines | ||
| 41 | 41 | #include <drm/drm_displayid.h> |
| 42 | 42 | #include <drm/drm_drv.h> |
| 43 | 43 | #include <drm/drm_edid.h> |
| 44 | +#include <drm/drm_eld.h> | |
| 44 | 45 | #include <drm/drm_encoder.h> |
| 45 | 46 | #include <drm/drm_print.h> |
| 46 | 47 | |
| 47 | 48 | #include "drm_crtc_internal.h" |
| 49 | +#include "drm_internal.h" | |
| 48 | 50 | |
| 49 | 51 | static int oui(u8 first, u8 second, u8 third) |
| 50 | 52 | { |
| @@ -5509,6 +5511,27 @@ static void clear_eld(struct drm_connector *connector) · 5458 unmodified lines | ||
| 5509 | 5511 | connector->audio_latency[1] = 0; |
| 5510 | 5512 | } |
| 5511 | 5513 | |
| 5514 | +/* | |
| 5515 | + * Get 3-byte SAD buffer from struct cea_sad. | |
| 5516 | + */ | |
| 5517 | +void drm_edid_cta_sad_get(const struct cea_sad *cta_sad, u8 *sad) | |
| 5518 | +{ | |
| 5519 | + sad[0] = cta_sad->format << 3 | cta_sad->channels; | |
| 5520 | + sad[1] = cta_sad->freq; | |
| 5521 | + sad[2] = cta_sad->byte2; | |
| 5522 | +} | |
| 5523 | + | |
| 5524 | +/* | |
| 5525 | + * Set struct cea_sad from 3-byte SAD buffer. | |
| 5526 | + */ | |
| 5527 | +void drm_edid_cta_sad_set(struct cea_sad *cta_sad, const u8 *sad) | |
| 5528 | +{ | |
| 5529 | + cta_sad->format = (sad[0] & 0x78) >> 3; | |
| 5530 | + cta_sad->channels = sad[0] & 0x07; | |
| 5531 | + cta_sad->freq = sad[1] & 0x7f; | |
| 5532 | + cta_sad->byte2 = sad[2]; | |
| 5533 | +} | |
| 5534 | + | |
| 5512 | 5535 | /* |
| 5513 | 5536 | * drm_edid_to_eld - build ELD from EDID |
| 5514 | 5537 | * @connector: connector corresponding to the HDMI/DP sink |
| @@ -5593,7 +5616,7 @@ static void drm_edid_to_eld(struct drm_connector *connector, · 78 unmodified lines | ||
| 5593 | 5616 | } |
| 5594 | 5617 | |
| 5595 | 5618 | static int _drm_edid_to_sad(const struct drm_edid *drm_edid, |
| 5596 | − struct cea_sad **sads) | |
| 5619 | + struct cea_sad **psads) | |
| 5597 | 5620 | { |
| 5598 | 5621 | const struct cea_db *db; |
| 5599 | 5622 | struct cea_db_iter iter; |
| @@ -5602,20 +5625,16 @@ static int _drm_edid_to_sad(const struct drm_edid *drm_edid, · 2 unmodified lines | ||
| 5602 | 5625 | cea_db_iter_edid_begin(drm_edid, &iter); |
| 5603 | 5626 | cea_db_iter_for_each(db, &iter) { |
| 5604 | 5627 | if (cea_db_tag(db) == CTA_DB_AUDIO) { |
| 5605 | − int j; | |
| 5628 | + struct cea_sad *sads; | |
| 5629 | + int i; | |
| 5606 | 5630 | |
| 5607 | 5631 | count = cea_db_payload_len(db) / 3; /* SAD is 3B */ |
| 5608 | − *sads = kcalloc(count, sizeof(**sads), GFP_KERNEL); | |
| 5609 | − if (!*sads) | |
| 5632 | + sads = kcalloc(count, sizeof(*sads), GFP_KERNEL); | |
| 5633 | + *psads = sads; | |
| 5634 | + if (!sads) | |
| 5610 | 5635 | return -ENOMEM; |
| 5611 | − for (j = 0; j < count; j++) { | |
| 5612 | − const u8 *sad = &db->data[j * 3]; | |
| 5613 | − | |
| 5614 | − (*sads)[j].format = (sad[0] & 0x78) >> 3; | |
| 5615 | − (*sads)[j].channels = sad[0] & 0x7; | |
| 5616 | − (*sads)[j].freq = sad[1] & 0x7F; | |
| 5617 | − (*sads)[j].byte2 = sad[2]; | |
| 5618 | − } | |
| 5636 | + for (i = 0; i < count; i++) | |
| 5637 | + drm_edid_cta_sad_set(&sads[i], &db->data[i * 3]); | |
| 5619 | 5638 | break; |
| 5620 | 5639 | } |
| 5621 | 5640 | } |
| @@ -23,22 +23,6 @@ module_param_string(edid_firmware, edid_firmware, sizeof(edid_firmware), 0644); · 22 unmodified lines | ||
| 23 | 23 | MODULE_PARM_DESC(edid_firmware, "Do not probe monitor, use specified EDID blob " |
| 24 | 24 | "from built-in data or /lib/firmware instead. "); |
| 25 | 25 | |
| 26 | −/* Use only for backward compatibility with drm_kms_helper.edid_firmware */ | |
| 27 | −int __drm_set_edid_firmware_path(const char *path) | |
| 28 | −{ | |
| 29 | − scnprintf(edid_firmware, sizeof(edid_firmware), "%s", path); | |
| 30 | − | |
| 31 | − return 0; | |
| 32 | −} | |
| 33 | −EXPORT_SYMBOL(__drm_set_edid_firmware_path); | |
| 34 | − | |
| 35 | −/* Use only for backward compatibility with drm_kms_helper.edid_firmware */ | |
| 36 | −int __drm_get_edid_firmware_path(char *buf, size_t bufsize) | |
| 37 | −{ | |
| 38 | − return scnprintf(buf, bufsize, "%s", edid_firmware); | |
| 39 | −} | |
| 40 | −EXPORT_SYMBOL(__drm_get_edid_firmware_path); | |
| 41 | − | |
| 42 | 26 | #define GENERIC_EDIDS 6 |
| 43 | 27 | static const char * const generic_edid_name[GENERIC_EDIDS] = { |
| 44 | 28 | "edid/800x600.bin", |
| @@ -0,0 +1,55 @@ | ||
| 1 | +// SPDX-License-Identifier: MIT | |
| 2 | +/* | |
| 3 | + * Copyright © 2023 Intel Corporation | |
| 4 | + */ | |
| 5 | + | |
| 6 | +#include <drm/drm_edid.h> | |
| 7 | +#include <drm/drm_eld.h> | |
| 8 | + | |
| 9 | +#include "drm_internal.h" | |
| 10 | + | |
| 11 | +/** | |
| 12 | + * drm_eld_sad_get - get SAD from ELD to struct cea_sad | |
| 13 | + * @eld: ELD buffer | |
| 14 | + * @sad_index: SAD index | |
| 15 | + * @cta_sad: destination struct cea_sad | |
| 16 | + * | |
| 17 | + * @return: 0 on success, or negative on errors | |
| 18 | + */ | |
| 19 | +int drm_eld_sad_get(const u8 *eld, int sad_index, struct cea_sad *cta_sad) | |
| 20 | +{ | |
| 21 | + const u8 *sad; | |
| 22 | + | |
| 23 | + if (sad_index >= drm_eld_sad_count(eld)) | |
| 24 | + return -EINVAL; | |
| 25 | + | |
| 26 | + sad = eld + DRM_ELD_CEA_SAD(drm_eld_mnl(eld), sad_index); | |
| 27 | + | |
| 28 | + drm_edid_cta_sad_set(cta_sad, sad); | |
| 29 | + | |
| 30 | + return 0; | |
| 31 | +} | |
| 32 | +EXPORT_SYMBOL(drm_eld_sad_get); | |
| 33 | + | |
| 34 | +/** | |
| 35 | + * drm_eld_sad_set - set SAD to ELD from struct cea_sad | |
| 36 | + * @eld: ELD buffer | |
| 37 | + * @sad_index: SAD index | |
| 38 | + * @cta_sad: source struct cea_sad | |
| 39 | + * | |
| 40 | + * @return: 0 on success, or negative on errors | |
| 41 | + */ | |
| 42 | +int drm_eld_sad_set(u8 *eld, int sad_index, const struct cea_sad *cta_sad) | |
| 43 | +{ | |
| 44 | + u8 *sad; | |
| 45 | + | |
| 46 | + if (sad_index >= drm_eld_sad_count(eld)) | |
| 47 | + return -EINVAL; | |
| 48 | + | |
| 49 | + sad = eld + DRM_ELD_CEA_SAD(drm_eld_mnl(eld), sad_index); | |
| 50 | + | |
| 51 | + drm_edid_cta_sad_get(cta_sad, sad); | |
| 52 | + | |
| 53 | + return 0; | |
| 54 | +} | |
| 55 | +EXPORT_SYMBOL(drm_eld_sad_set); | |
| @@ -30,6 +30,7 @@ · 29 unmodified lines | ||
| 30 | 30 | #include <drm/drm_print.h> |
| 31 | 31 | |
| 32 | 32 | #include "drm_crtc_internal.h" |
| 33 | +#include "drm_internal.h" | |
| 33 | 34 | |
| 34 | 35 | /** |
| 35 | 36 | * DOC: overview |
| @@ -74,6 +75,8 @@ int drm_encoder_register_all(struct drm_device *dev) · 38 unmodified lines | ||
| 74 | 75 | int ret = 0; |
| 75 | 76 | |
| 76 | 77 | drm_for_each_encoder(encoder, dev) { |
| 78 | + drm_debugfs_encoder_add(encoder); | |
| 79 | + | |
| 77 | 80 | if (encoder->funcs && encoder->funcs->late_register) |
| 78 | 81 | ret = encoder->funcs->late_register(encoder); |
| 79 | 82 | if (ret) |
| @@ -90,6 +93,7 @@ void drm_encoder_unregister_all(struct drm_device *dev) · 10 unmodified lines | ||
| 90 | 93 | drm_for_each_encoder(encoder, dev) { |
| 91 | 94 | if (encoder->funcs && encoder->funcs->early_unregister) |
| 92 | 95 | encoder->funcs->early_unregister(encoder); |
| 96 | + drm_debugfs_encoder_remove(encoder); | |
| 93 | 97 | } |
| 94 | 98 | } |
| 95 | 99 | |
| @@ -47,21 +47,12 @@ · 46 unmodified lines | ||
| 47 | 47 | |
| 48 | 48 | #include "drm_crtc_internal.h" |
| 49 | 49 | #include "drm_internal.h" |
| 50 | −#include "drm_legacy.h" | |
| 51 | 50 | |
| 52 | 51 | /* from BKL pushdown */ |
| 53 | 52 | DEFINE_MUTEX(drm_global_mutex); |
| 54 | 53 | |
| 55 | 54 | bool drm_dev_needs_global_mutex(struct drm_device *dev) |
| 56 | 55 | { |
| 57 | − /* | |
| 58 | − * Legacy drivers rely on all kinds of BKL locking semantics, don't | |
| 59 | − * bother. They also still need BKL locking for their ioctls, so better | |
| 60 | − * safe than sorry. | |
| 61 | − */ | |
| 62 | − if (drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 63 | − return true; | |
| 64 | − | |
| 65 | 56 | /* |
| 66 | 57 | * The deprecated ->load callback must be called after the driver is |
| 67 | 58 | * already registered. This means such drivers rely on the BKL to make |
| @@ -107,9 +98,7 @@ bool drm_dev_needs_global_mutex(struct drm_device *dev) · 39 unmodified lines | ||
| 107 | 98 | * drm_send_event() as the main starting points. |
| 108 | 99 | * |
| 109 | 100 | * The memory mapping implementation will vary depending on how the driver |
| 110 | − * manages memory. Legacy drivers will use the deprecated drm_legacy_mmap() | |
| 111 | − * function, modern drivers should use one of the provided memory-manager | |
| 112 | − * specific implementations. For GEM-based drivers this is drm_gem_mmap(). | |
| 101 | + * manages memory. For GEM-based drivers this is drm_gem_mmap(). | |
| 113 | 102 | * |
| 114 | 103 | * No other file operations are supported by the DRM userspace API. Overall the |
| 115 | 104 | * following is an example &file_operations structure:: |
| @@ -254,18 +243,6 @@ void drm_file_free(struct drm_file *file) · 138 unmodified lines | ||
| 254 | 243 | (long)old_encode_dev(file->minor->kdev->devt), |
| 255 | 244 | atomic_read(&dev->open_count)); |
| 256 | 245 | |
| 257 | −#ifdef CONFIG_DRM_LEGACY | |
| 258 | − if (drm_core_check_feature(dev, DRIVER_LEGACY) && | |
| 259 | − dev->driver->preclose) | |
| 260 | − dev->driver->preclose(dev, file); | |
| 261 | −#endif | |
| 262 | − | |
| 263 | − if (drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 264 | − drm_legacy_lock_release(dev, file->filp); | |
| 265 | − | |
| 266 | − if (drm_core_check_feature(dev, DRIVER_HAVE_DMA)) | |
| 267 | − drm_legacy_reclaim_buffers(dev, file); | |
| 268 | − | |
| 269 | 246 | drm_events_release(file); |
| 270 | 247 | |
| 271 | 248 | if (drm_core_check_feature(dev, DRIVER_MODESET)) { |
| @@ -279,8 +256,6 @@ void drm_file_free(struct drm_file *file) · 7 unmodified lines | ||
| 279 | 256 | if (drm_core_check_feature(dev, DRIVER_GEM)) |
| 280 | 257 | drm_gem_release(dev, file); |
| 281 | 258 | |
| 282 | − drm_legacy_ctxbitmap_flush(dev, file); | |
| 283 | − | |
| 284 | 259 | if (drm_is_primary_client(file)) |
| 285 | 260 | drm_master_release(file); |
| 286 | 261 | |
| @@ -367,29 +342,6 @@ int drm_open_helper(struct file *filp, struct drm_minor *minor) · 80 unmodified lines | ||
| 367 | 342 | list_add(&priv->lhead, &dev->filelist); |
| 368 | 343 | mutex_unlock(&dev->filelist_mutex); |
| 369 | 344 | |
| 370 | −#ifdef CONFIG_DRM_LEGACY | |
| 371 | −#ifdef __alpha__ | |
| 372 | − /* | |
| 373 | − * Default the hose | |
| 374 | − */ | |
| 375 | − if (!dev->hose) { | |
| 376 | − struct pci_dev *pci_dev; | |
| 377 | − | |
| 378 | − pci_dev = pci_get_class(PCI_CLASS_DISPLAY_VGA << 8, NULL); | |
| 379 | − if (pci_dev) { | |
| 380 | − dev->hose = pci_dev->sysdata; | |
| 381 | − pci_dev_put(pci_dev); | |
| 382 | − } | |
| 383 | − if (!dev->hose) { | |
| 384 | − struct pci_bus *b = list_entry(pci_root_buses.next, | |
| 385 | − struct pci_bus, node); | |
| 386 | − if (b) | |
| 387 | − dev->hose = b->sysdata; | |
| 388 | − } | |
| 389 | − } | |
| 390 | −#endif | |
| 391 | −#endif | |
| 392 | − | |
| 393 | 345 | return 0; |
| 394 | 346 | } |
| 395 | 347 | |
| @@ -411,7 +363,6 @@ int drm_open(struct inode *inode, struct file *filp) · 15 unmodified lines | ||
| 411 | 363 | struct drm_device *dev; |
| 412 | 364 | struct drm_minor *minor; |
| 413 | 365 | int retcode; |
| 414 | − int need_setup = 0; | |
| 415 | 366 | |
| 416 | 367 | minor = drm_minor_acquire(iminor(inode)); |
| 417 | 368 | if (IS_ERR(minor)) |
| @@ -421,8 +372,7 @@ int drm_open(struct inode *inode, struct file *filp) · 3 unmodified lines | ||
| 421 | 372 | if (drm_dev_needs_global_mutex(dev)) |
| 422 | 373 | mutex_lock(&drm_global_mutex); |
| 423 | 374 | |
| 424 | − if (!atomic_fetch_inc(&dev->open_count)) | |
| 425 | − need_setup = 1; | |
| 375 | + atomic_fetch_inc(&dev->open_count); | |
| 426 | 376 | |
| 427 | 377 | /* share address_space across all char-devs of a single device */ |
| 428 | 378 | filp->f_mapping = dev->anon_inode->i_mapping; |
| @@ -430,13 +380,6 @@ int drm_open(struct inode *inode, struct file *filp) · 1 unmodified line | ||
| 430 | 380 | retcode = drm_open_helper(filp, minor); |
| 431 | 381 | if (retcode) |
| 432 | 382 | goto err_undo; |
| 433 | − if (need_setup) { | |
| 434 | − retcode = drm_legacy_setup(dev); | |
| 435 | − if (retcode) { | |
| 436 | − drm_close_helper(filp); | |
| 437 | − goto err_undo; | |
| 438 | − } | |
| 439 | − } | |
| 440 | 383 | |
| 441 | 384 | if (drm_dev_needs_global_mutex(dev)) |
| 442 | 385 | mutex_unlock(&drm_global_mutex); |
| @@ -460,9 +403,6 @@ void drm_lastclose(struct drm_device * dev) · 17 unmodified lines | ||
| 460 | 403 | dev->driver->lastclose(dev); |
| 461 | 404 | drm_dbg_core(dev, "driver lastclose completed\n"); |
| 462 | 405 | |
| 463 | − if (drm_core_check_feature(dev, DRIVER_LEGACY)) | |
| 464 | − drm_legacy_dev_reinit(dev); | |
| 465 | − | |
| 466 | 406 | drm_client_dev_restore(dev); |
| 467 | 407 | } |
| 468 | 408 | |
| @@ -913,7 +853,7 @@ static void print_size(struct drm_printer *p, const char *stat, · 444 unmodified lines | ||
| 913 | 853 | unsigned u; |
| 914 | 854 | |
| 915 | 855 | for (u = 0; u < ARRAY_SIZE(units) - 1; u++) { |
| 916 | − if (sz < SZ_1K) | |
| 856 | + if (sz == 0 || !IS_ALIGNED(sz, SZ_1K)) | |
| 917 | 857 | break; |
| 918 | 858 | sz = div_u64(sz, SZ_1K); |
| 919 | 859 | } |
| @@ -958,7 +898,7 @@ void drm_show_memory_stats(struct drm_printer *p, struct drm_file *file) · 38 unmodified lines | ||
| 958 | 898 | { |
| 959 | 899 | struct drm_gem_object *obj; |
| 960 | 900 | struct drm_memory_stats status = {}; |
| 961 | − enum drm_gem_object_status supported_status; | |
| 901 | + enum drm_gem_object_status supported_status = 0; | |
| 962 | 902 | int id; |
| 963 | 903 | |
| 964 | 904 | spin_lock(&file->table_lock); |
| @@ -27,14 +27,12 @@ · 26 unmodified lines | ||
| 27 | 27 | #include <drm/drm_print.h> |
| 28 | 28 | #include <drm/drm_util.h> |
| 29 | 29 | |
| 30 | −/** | |
| 31 | − * drm_flip_work_allocate_task - allocate a flip-work task | |
| 32 | − * @data: data associated to the task | |
| 33 | − * @flags: allocator flags | |
| 34 | − * | |
| 35 | − * Allocate a drm_flip_task object and attach private data to it. | |
| 36 | − */ | |
| 37 | −struct drm_flip_task *drm_flip_work_allocate_task(void *data, gfp_t flags) | |
| 30 | +struct drm_flip_task { | |
| 31 | + struct list_head node; | |
| 32 | + void *data; | |
| 33 | +}; | |
| 34 | + | |
| 35 | +static struct drm_flip_task *drm_flip_work_allocate_task(void *data, gfp_t flags) | |
| 38 | 36 | { |
| 39 | 37 | struct drm_flip_task *task; |
| 40 | 38 | |
| @@ -44,18 +42,8 @@ struct drm_flip_task *drm_flip_work_allocate_task(void *data, gfp_t flags) · 3 unmodified lines | ||
| 44 | 42 | |
| 45 | 43 | return task; |
| 46 | 44 | } |
| 47 | −EXPORT_SYMBOL(drm_flip_work_allocate_task); | |
| 48 | 45 | |
| 49 | −/** | |
| 50 | − * drm_flip_work_queue_task - queue a specific task | |
| 51 | − * @work: the flip-work | |
| 52 | − * @task: the task to handle | |
| 53 | − * | |
| 54 | − * Queues task, that will later be run (passed back to drm_flip_func_t | |
| 55 | − * func) on a work queue after drm_flip_work_commit() is called. | |
| 56 | − */ | |
| 57 | −void drm_flip_work_queue_task(struct drm_flip_work *work, | |
| 58 | − struct drm_flip_task *task) | |
| 46 | +static void drm_flip_work_queue_task(struct drm_flip_work *work, struct drm_flip_task *task) | |
| 59 | 47 | { |
| 60 | 48 | unsigned long flags; |
| 61 | 49 | |
| @@ -63,7 +51,6 @@ void drm_flip_work_queue_task(struct drm_flip_work *work, · 1 unmodified line | ||
| 63 | 51 | list_add_tail(&task->node, &work->queued); |
| 64 | 52 | spin_unlock_irqrestore(&work->lock, flags); |
| 65 | 53 | } |
| 66 | −EXPORT_SYMBOL(drm_flip_work_queue_task); | |
| 67 | 54 | |
| 68 | 55 | /** |
| 69 | 56 | * drm_flip_work_queue - queue work |
| @@ -20,6 +20,97 @@ · 19 unmodified lines | ||
| 20 | 20 | #include <drm/drm_print.h> |
| 21 | 21 | #include <drm/drm_rect.h> |
| 22 | 22 | |
| 23 | +/** | |
| 24 | + * drm_format_conv_state_init - Initialize format-conversion state | |
| 25 | + * @state: The state to initialize | |
| 26 | + * | |
| 27 | + * Clears all fields in struct drm_format_conv_state. The state will | |
| 28 | + * be empty with no preallocated resources. | |
| 29 | + */ | |
| 30 | +void drm_format_conv_state_init(struct drm_format_conv_state *state) | |
| 31 | +{ | |
| 32 | + state->tmp.mem = NULL; | |
| 33 | + state->tmp.size = 0; | |
| 34 | + state->tmp.preallocated = false; | |
| 35 | +} | |
| 36 | +EXPORT_SYMBOL(drm_format_conv_state_init); | |
| 37 | + | |
| 38 | +/** | |
| 39 | + * drm_format_conv_state_copy - Copy format-conversion state | |
| 40 | + * @state: Destination state | |
| 41 | + * @old_state: Source state | |
| 42 | + * | |
| 43 | + * Copies format-conversion state from @old_state to @state; except for | |
| 44 | + * temporary storage. | |
| 45 | + */ | |
| 46 | +void drm_format_conv_state_copy(struct drm_format_conv_state *state, | |
| 47 | + const struct drm_format_conv_state *old_state) | |
| 48 | +{ | |
| 49 | + /* | |
| 50 | + * So far, there's only temporary storage here, which we don't | |
| 51 | + * duplicate. Just clear the fields. | |
| 52 | + */ | |
| 53 | + state->tmp.mem = NULL; | |
| 54 | + state->tmp.size = 0; | |
| 55 | + state->tmp.preallocated = false; | |
| 56 | +} | |
| 57 | +EXPORT_SYMBOL(drm_format_conv_state_copy); | |
| 58 | + | |
| 59 | +/** | |
| 60 | + * drm_format_conv_state_reserve - Allocates storage for format conversion | |
| 61 | + * @state: The format-conversion state | |
| 62 | + * @new_size: The minimum allocation size | |
| 63 | + * @flags: Flags for kmalloc() | |
| 64 | + * | |
| 65 | + * Allocates at least @new_size bytes and returns a pointer to the memory | |
| 66 | + * range. After calling this function, previously returned memory blocks | |
| 67 | + * are invalid. It's best to collect all memory requirements of a format | |
| 68 | + * conversion and call this function once to allocate the range. | |
| 69 | + * | |
| 70 | + * Returns: | |
| 71 | + * A pointer to the allocated memory range, or NULL otherwise. | |
| 72 | + */ | |
| 73 | +void *drm_format_conv_state_reserve(struct drm_format_conv_state *state, | |
| 74 | + size_t new_size, gfp_t flags) | |
| 75 | +{ | |
| 76 | + void *mem; | |
| 77 | + | |
| 78 | + if (new_size <= state->tmp.size) | |
| 79 | + goto out; | |
| 80 | + else if (state->tmp.preallocated) | |
| 81 | + return NULL; | |
| 82 | + | |
| 83 | + mem = krealloc(state->tmp.mem, new_size, flags); | |
| 84 | + if (!mem) | |
| 85 | + return NULL; | |
| 86 | + | |
| 87 | + state->tmp.mem = mem; | |
| 88 | + state->tmp.size = new_size; | |
| 89 | + | |
| 90 | +out: | |
| 91 | + return state->tmp.mem; | |
| 92 | +} | |
| 93 | +EXPORT_SYMBOL(drm_format_conv_state_reserve); | |
| 94 | + | |
| 95 | +/** | |
| 96 | + * drm_format_conv_state_release - Releases an format-conversion storage | |
| 97 | + * @state: The format-conversion state | |
| 98 | + * | |
| 99 | + * Releases the memory range references by the format-conversion state. | |
| 100 | + * After this call, all pointers to the memory are invalid. Prefer | |
| 101 | + * drm_format_conv_state_init() for cleaning up and unloading a driver. | |
| 102 | + */ | |
| 103 | +void drm_format_conv_state_release(struct drm_format_conv_state *state) | |
| 104 | +{ | |
| 105 | + if (state->tmp.preallocated) | |
| 106 | + return; | |
| 107 | + | |
| 108 | + kfree(state->tmp.mem); | |
| 109 | + state->tmp.mem = NULL; | |
| 110 | + state->tmp.size = 0; | |
| 111 | +} | |
| 112 | +EXPORT_SYMBOL(drm_format_conv_state_release); | |
| 113 | + | |
| 23 | 114 | static unsigned int clip_offset(const struct drm_rect *clip, unsigned int pitch, unsigned int cpp) |
| 24 | 115 | { |
| 25 | 116 | return clip->y1 * pitch + clip->x1 * cpp; |
| @@ -45,6 +136,7 @@ EXPORT_SYMBOL(drm_fb_clip_offset); · 19 unmodified lines | ||
| 45 | 136 | static int __drm_fb_xfrm(void *dst, unsigned long dst_pitch, unsigned long dst_pixsize, |
| 46 | 137 | const void *vaddr, const struct drm_framebuffer *fb, |
| 47 | 138 | const struct drm_rect *clip, bool vaddr_cached_hint, |
| 139 | + struct drm_format_conv_state *state, | |
| 48 | 140 | void (*xfrm_line)(void *dbuf, const void *sbuf, unsigned int npixels)) |
| 49 | 141 | { |
| 50 | 142 | unsigned long linepixels = drm_rect_width(clip); |
| @@ -60,7 +152,7 @@ static int __drm_fb_xfrm(void *dst, unsigned long dst_pitch, unsigned long dst_p · 9 unmodified lines | ||
| 60 | 152 | * one line at a time. |
| 61 | 153 | */ |
| 62 | 154 | if (!vaddr_cached_hint) { |
| 63 | − stmp = kmalloc(sbuf_len, GFP_KERNEL); | |
| 155 | + stmp = drm_format_conv_state_reserve(state, sbuf_len, GFP_KERNEL); | |
| 64 | 156 | if (!stmp) |
| 65 | 157 | return -ENOMEM; |
| 66 | 158 | } |
| @@ -79,8 +171,6 @@ static int __drm_fb_xfrm(void *dst, unsigned long dst_pitch, unsigned long dst_p · 12 unmodified lines | ||
| 79 | 171 | dst += dst_pitch; |
| 80 | 172 | } |
| 81 | 173 | |
| 82 | − kfree(stmp); | |
| 83 | − | |
| 84 | 174 | return 0; |
| 85 | 175 | } |
| 86 | 176 | |
| @@ -88,6 +178,7 @@ static int __drm_fb_xfrm(void *dst, unsigned long dst_pitch, unsigned long dst_p · 1 unmodified line | ||
| 88 | 178 | static int __drm_fb_xfrm_toio(void __iomem *dst, unsigned long dst_pitch, unsigned long dst_pixsize, |
| 89 | 179 | const void *vaddr, const struct drm_framebuffer *fb, |
| 90 | 180 | const struct drm_rect *clip, bool vaddr_cached_hint, |
| 181 | + struct drm_format_conv_state *state, | |
| 91 | 182 | void (*xfrm_line)(void *dbuf, const void *sbuf, unsigned int npixels)) |
| 92 | 183 | { |
| 93 | 184 | unsigned long linepixels = drm_rect_width(clip); |
| @@ -101,9 +192,9 @@ static int __drm_fb_xfrm_toio(void __iomem *dst, unsigned long dst_pitch, unsign · 7 unmodified lines | ||
| 101 | 192 | void *dbuf; |
| 102 | 193 | |
| 103 | 194 | if (vaddr_cached_hint) { |
| 104 | − dbuf = kmalloc(dbuf_len, GFP_KERNEL); | |
| 195 | + dbuf = drm_format_conv_state_reserve(state, dbuf_len, GFP_KERNEL); | |
| 105 | 196 | } else { |
| 106 | − dbuf = kmalloc(stmp_off + sbuf_len, GFP_KERNEL); | |
| 197 | + dbuf = drm_format_conv_state_reserve(state, stmp_off + sbuf_len, GFP_KERNEL); | |
| 107 | 198 | stmp = dbuf + stmp_off; |
| 108 | 199 | } |
| 109 | 200 | if (!dbuf) |
| @@ -124,8 +215,6 @@ static int __drm_fb_xfrm_toio(void __iomem *dst, unsigned long dst_pitch, unsign · 14 unmodified lines | ||
| 124 | 215 | dst += dst_pitch; |
| 125 | 216 | } |
| 126 | 217 | |
| 127 | − kfree(dbuf); | |
| 128 | − | |
| 129 | 218 | return 0; |
| 130 | 219 | } |
| 131 | 220 | |
| @@ -134,6 +223,7 @@ static int drm_fb_xfrm(struct iosys_map *dst, · 2 unmodified lines | ||
| 134 | 223 | const unsigned int *dst_pitch, const u8 *dst_pixsize, |
| 135 | 224 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 136 | 225 | const struct drm_rect *clip, bool vaddr_cached_hint, |
| 226 | + struct drm_format_conv_state *state, | |
| 137 | 227 | void (*xfrm_line)(void *dbuf, const void *sbuf, unsigned int npixels)) |
| 138 | 228 | { |
| 139 | 229 | static const unsigned int default_dst_pitch[DRM_FORMAT_MAX_PLANES] = { |
| @@ -146,10 +236,12 @@ static int drm_fb_xfrm(struct iosys_map *dst, · 6 unmodified lines | ||
| 146 | 236 | /* TODO: handle src in I/O memory here */ |
| 147 | 237 | if (dst[0].is_iomem) |
| 148 | 238 | return __drm_fb_xfrm_toio(dst[0].vaddr_iomem, dst_pitch[0], dst_pixsize[0], |
| 149 | − src[0].vaddr, fb, clip, vaddr_cached_hint, xfrm_line); | |
| 239 | + src[0].vaddr, fb, clip, vaddr_cached_hint, state, | |
| 240 | + xfrm_line); | |
| 150 | 241 | else |
| 151 | 242 | return __drm_fb_xfrm(dst[0].vaddr, dst_pitch[0], dst_pixsize[0], |
| 152 | − src[0].vaddr, fb, clip, vaddr_cached_hint, xfrm_line); | |
| 243 | + src[0].vaddr, fb, clip, vaddr_cached_hint, state, | |
| 244 | + xfrm_line); | |
| 153 | 245 | } |
| 154 | 246 | |
| 155 | 247 | /** |
| @@ -235,6 +327,7 @@ static void drm_fb_swab32_line(void *dbuf, const void *sbuf, unsigned int pixels · 79 unmodified lines | ||
| 235 | 327 | * @fb: DRM framebuffer |
| 236 | 328 | * @clip: Clip rectangle area to copy |
| 237 | 329 | * @cached: Source buffer is mapped cached (eg. not write-combined) |
| 330 | + * @state: Transform and conversion state | |
| 238 | 331 | * |
| 239 | 332 | * This function copies parts of a framebuffer to display memory and swaps per-pixel |
| 240 | 333 | * bytes during the process. Destination and framebuffer formats must match. The |
| @@ -249,7 +342,8 @@ static void drm_fb_swab32_line(void *dbuf, const void *sbuf, unsigned int pixels · 8 unmodified lines | ||
| 249 | 342 | */ |
| 250 | 343 | void drm_fb_swab(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 251 | 344 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 252 | − const struct drm_rect *clip, bool cached) | |
| 345 | + const struct drm_rect *clip, bool cached, | |
| 346 | + struct drm_format_conv_state *state) | |
| 253 | 347 | { |
| 254 | 348 | const struct drm_format_info *format = fb->format; |
| 255 | 349 | u8 cpp = DIV_ROUND_UP(drm_format_info_bpp(format, 0), 8); |
| @@ -268,7 +362,7 @@ void drm_fb_swab(struct iosys_map *dst, const unsigned int *dst_pitch, · 12 unmodified lines | ||
| 268 | 362 | return; |
| 269 | 363 | } |
| 270 | 364 | |
| 271 | − drm_fb_xfrm(dst, dst_pitch, &cpp, src, fb, clip, cached, swab_line); | |
| 365 | + drm_fb_xfrm(dst, dst_pitch, &cpp, src, fb, clip, cached, state, swab_line); | |
| 272 | 366 | } |
| 273 | 367 | EXPORT_SYMBOL(drm_fb_swab); |
| 274 | 368 | |
| @@ -295,6 +389,7 @@ static void drm_fb_xrgb8888_to_rgb332_line(void *dbuf, const void *sbuf, unsigne · 20 unmodified lines | ||
| 295 | 389 | * @src: Array of XRGB8888 source buffers |
| 296 | 390 | * @fb: DRM framebuffer |
| 297 | 391 | * @clip: Clip rectangle area to copy |
| 392 | + * @state: Transform and conversion state | |
| 298 | 393 | * |
| 299 | 394 | * This function copies parts of a framebuffer to display memory and converts the |
| 300 | 395 | * color format during the process. Destination and framebuffer formats must match. The |
| @@ -309,13 +404,13 @@ static void drm_fb_xrgb8888_to_rgb332_line(void *dbuf, const void *sbuf, unsigne · 8 unmodified lines | ||
| 309 | 404 | */ |
| 310 | 405 | void drm_fb_xrgb8888_to_rgb332(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 311 | 406 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 312 | − const struct drm_rect *clip) | |
| 407 | + const struct drm_rect *clip, struct drm_format_conv_state *state) | |
| 313 | 408 | { |
| 314 | 409 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 315 | 410 | 1, |
| 316 | 411 | }; |
| 317 | 412 | |
| 318 | − drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | |
| 413 | + drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, | |
| 319 | 414 | drm_fb_xrgb8888_to_rgb332_line); |
| 320 | 415 | } |
| 321 | 416 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb332); |
| @@ -364,6 +459,7 @@ static void drm_fb_xrgb8888_to_rgb565_swab_line(void *dbuf, const void *sbuf, · 42 unmodified lines | ||
| 364 | 459 | * @src: Array of XRGB8888 source buffer |
| 365 | 460 | * @fb: DRM framebuffer |
| 366 | 461 | * @clip: Clip rectangle area to copy |
| 462 | + * @state: Transform and conversion state | |
| 367 | 463 | * @swab: Swap bytes |
| 368 | 464 | * |
| 369 | 465 | * This function copies parts of a framebuffer to display memory and converts the |
| @@ -379,7 +475,8 @@ static void drm_fb_xrgb8888_to_rgb565_swab_line(void *dbuf, const void *sbuf, · 9 unmodified lines | ||
| 379 | 475 | */ |
| 380 | 476 | void drm_fb_xrgb8888_to_rgb565(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 381 | 477 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 382 | − const struct drm_rect *clip, bool swab) | |
| 478 | + const struct drm_rect *clip, struct drm_format_conv_state *state, | |
| 479 | + bool swab) | |
| 383 | 480 | { |
| 384 | 481 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 385 | 482 | 2, |
| @@ -392,7 +489,7 @@ void drm_fb_xrgb8888_to_rgb565(struct iosys_map *dst, const unsigned int *dst_pi · 6 unmodified lines | ||
| 392 | 489 | else |
| 393 | 490 | xfrm_line = drm_fb_xrgb8888_to_rgb565_line; |
| 394 | 491 | |
| 395 | − drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, xfrm_line); | |
| 492 | + drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, xfrm_line); | |
| 396 | 493 | } |
| 397 | 494 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb565); |
| 398 | 495 | |
| @@ -421,6 +518,7 @@ static void drm_fb_xrgb8888_to_xrgb1555_line(void *dbuf, const void *sbuf, unsig · 22 unmodified lines | ||
| 421 | 518 | * @src: Array of XRGB8888 source buffer |
| 422 | 519 | * @fb: DRM framebuffer |
| 423 | 520 | * @clip: Clip rectangle area to copy |
| 521 | + * @state: Transform and conversion state | |
| 424 | 522 | * |
| 425 | 523 | * This function copies parts of a framebuffer to display memory and converts |
| 426 | 524 | * the color format during the process. The parameters @dst, @dst_pitch and |
| @@ -436,13 +534,13 @@ static void drm_fb_xrgb8888_to_xrgb1555_line(void *dbuf, const void *sbuf, unsig · 9 unmodified lines | ||
| 436 | 534 | */ |
| 437 | 535 | void drm_fb_xrgb8888_to_xrgb1555(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 438 | 536 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 439 | − const struct drm_rect *clip) | |
| 537 | + const struct drm_rect *clip, struct drm_format_conv_state *state) | |
| 440 | 538 | { |
| 441 | 539 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 442 | 540 | 2, |
| 443 | 541 | }; |
| 444 | 542 | |
| 445 | − drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | |
| 543 | + drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, | |
| 446 | 544 | drm_fb_xrgb8888_to_xrgb1555_line); |
| 447 | 545 | } |
| 448 | 546 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_xrgb1555); |
| @@ -473,6 +571,7 @@ static void drm_fb_xrgb8888_to_argb1555_line(void *dbuf, const void *sbuf, unsig · 24 unmodified lines | ||
| 473 | 571 | * @src: Array of XRGB8888 source buffer |
| 474 | 572 | * @fb: DRM framebuffer |
| 475 | 573 | * @clip: Clip rectangle area to copy |
| 574 | + * @state: Transform and conversion state | |
| 476 | 575 | * |
| 477 | 576 | * This function copies parts of a framebuffer to display memory and converts |
| 478 | 577 | * the color format during the process. The parameters @dst, @dst_pitch and |
| @@ -488,13 +587,13 @@ static void drm_fb_xrgb8888_to_argb1555_line(void *dbuf, const void *sbuf, unsig · 9 unmodified lines | ||
| 488 | 587 | */ |
| 489 | 588 | void drm_fb_xrgb8888_to_argb1555(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 490 | 589 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 491 | − const struct drm_rect *clip) | |
| 590 | + const struct drm_rect *clip, struct drm_format_conv_state *state) | |
| 492 | 591 | { |
| 493 | 592 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 494 | 593 | 2, |
| 495 | 594 | }; |
| 496 | 595 | |
| 497 | − drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | |
| 596 | + drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, | |
| 498 | 597 | drm_fb_xrgb8888_to_argb1555_line); |
| 499 | 598 | } |
| 500 | 599 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_argb1555); |
| @@ -525,6 +624,7 @@ static void drm_fb_xrgb8888_to_rgba5551_line(void *dbuf, const void *sbuf, unsig · 24 unmodified lines | ||
| 525 | 624 | * @src: Array of XRGB8888 source buffer |
| 526 | 625 | * @fb: DRM framebuffer |
| 527 | 626 | * @clip: Clip rectangle area to copy |
| 627 | + * @state: Transform and conversion state | |
| 528 | 628 | * |
| 529 | 629 | * This function copies parts of a framebuffer to display memory and converts |
| 530 | 630 | * the color format during the process. The parameters @dst, @dst_pitch and |
| @@ -540,13 +640,13 @@ static void drm_fb_xrgb8888_to_rgba5551_line(void *dbuf, const void *sbuf, unsig · 9 unmodified lines | ||
| 540 | 640 | */ |
| 541 | 641 | void drm_fb_xrgb8888_to_rgba5551(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 542 | 642 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 543 | − const struct drm_rect *clip) | |
| 643 | + const struct drm_rect *clip, struct drm_format_conv_state *state) | |
| 544 | 644 | { |
| 545 | 645 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 546 | 646 | 2, |
| 547 | 647 | }; |
| 548 | 648 | |
| 549 | − drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | |
| 649 | + drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, | |
| 550 | 650 | drm_fb_xrgb8888_to_rgba5551_line); |
| 551 | 651 | } |
| 552 | 652 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgba5551); |
| @@ -575,6 +675,7 @@ static void drm_fb_xrgb8888_to_rgb888_line(void *dbuf, const void *sbuf, unsigne · 22 unmodified lines | ||
| 575 | 675 | * @src: Array of XRGB8888 source buffers |
| 576 | 676 | * @fb: DRM framebuffer |
| 577 | 677 | * @clip: Clip rectangle area to copy |
| 678 | + * @state: Transform and conversion state | |
| 578 | 679 | * |
| 579 | 680 | * This function copies parts of a framebuffer to display memory and converts the |
| 580 | 681 | * color format during the process. Destination and framebuffer formats must match. The |
| @@ -590,13 +691,13 @@ static void drm_fb_xrgb8888_to_rgb888_line(void *dbuf, const void *sbuf, unsigne · 9 unmodified lines | ||
| 590 | 691 | */ |
| 591 | 692 | void drm_fb_xrgb8888_to_rgb888(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 592 | 693 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 593 | − const struct drm_rect *clip) | |
| 694 | + const struct drm_rect *clip, struct drm_format_conv_state *state) | |
| 594 | 695 | { |
| 595 | 696 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 596 | 697 | 3, |
| 597 | 698 | }; |
| 598 | 699 | |
| 599 | − drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | |
| 700 | + drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, | |
| 600 | 701 | drm_fb_xrgb8888_to_rgb888_line); |
| 601 | 702 | } |
| 602 | 703 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_rgb888); |
| @@ -623,6 +724,7 @@ static void drm_fb_xrgb8888_to_argb8888_line(void *dbuf, const void *sbuf, unsig · 20 unmodified lines | ||
| 623 | 724 | * @src: Array of XRGB8888 source buffer |
| 624 | 725 | * @fb: DRM framebuffer |
| 625 | 726 | * @clip: Clip rectangle area to copy |
| 727 | + * @state: Transform and conversion state | |
| 626 | 728 | * |
| 627 | 729 | * This function copies parts of a framebuffer to display memory and converts the |
| 628 | 730 | * color format during the process. The parameters @dst, @dst_pitch and @src refer |
| @@ -638,13 +740,13 @@ static void drm_fb_xrgb8888_to_argb8888_line(void *dbuf, const void *sbuf, unsig · 9 unmodified lines | ||
| 638 | 740 | */ |
| 639 | 741 | void drm_fb_xrgb8888_to_argb8888(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 640 | 742 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 641 | − const struct drm_rect *clip) | |
| 743 | + const struct drm_rect *clip, struct drm_format_conv_state *state) | |
| 642 | 744 | { |
| 643 | 745 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 644 | 746 | 4, |
| 645 | 747 | }; |
| 646 | 748 | |
| 647 | − drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | |
| 749 | + drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, | |
| 648 | 750 | drm_fb_xrgb8888_to_argb8888_line); |
| 649 | 751 | } |
| 650 | 752 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_argb8888); |
| @@ -669,13 +771,14 @@ static void drm_fb_xrgb8888_to_abgr8888_line(void *dbuf, const void *sbuf, unsig · 18 unmodified lines | ||
| 669 | 771 | static void drm_fb_xrgb8888_to_abgr8888(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 670 | 772 | const struct iosys_map *src, |
| 671 | 773 | const struct drm_framebuffer *fb, |
| 672 | − const struct drm_rect *clip) | |
| 774 | + const struct drm_rect *clip, | |
| 775 | + struct drm_format_conv_state *state) | |
| 673 | 776 | { |
| 674 | 777 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 675 | 778 | 4, |
| 676 | 779 | }; |
| 677 | 780 | |
| 678 | − drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | |
| 781 | + drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, | |
| 679 | 782 | drm_fb_xrgb8888_to_abgr8888_line); |
| 680 | 783 | } |
| 681 | 784 | |
| @@ -699,13 +802,14 @@ static void drm_fb_xrgb8888_to_xbgr8888_line(void *dbuf, const void *sbuf, unsig · 17 unmodified lines | ||
| 699 | 802 | static void drm_fb_xrgb8888_to_xbgr8888(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 700 | 803 | const struct iosys_map *src, |
| 701 | 804 | const struct drm_framebuffer *fb, |
| 702 | − const struct drm_rect *clip) | |
| 805 | + const struct drm_rect *clip, | |
| 806 | + struct drm_format_conv_state *state) | |
| 703 | 807 | { |
| 704 | 808 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 705 | 809 | 4, |
| 706 | 810 | }; |
| 707 | 811 | |
| 708 | − drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | |
| 812 | + drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, | |
| 709 | 813 | drm_fb_xrgb8888_to_xbgr8888_line); |
| 710 | 814 | } |
| 711 | 815 | |
| @@ -735,6 +839,7 @@ static void drm_fb_xrgb8888_to_xrgb2101010_line(void *dbuf, const void *sbuf, un · 23 unmodified lines | ||
| 735 | 839 | * @src: Array of XRGB8888 source buffers |
| 736 | 840 | * @fb: DRM framebuffer |
| 737 | 841 | * @clip: Clip rectangle area to copy |
| 842 | + * @state: Transform and conversion state | |
| 738 | 843 | * |
| 739 | 844 | * This function copies parts of a framebuffer to display memory and converts the |
| 740 | 845 | * color format during the process. Destination and framebuffer formats must match. The |
| @@ -750,13 +855,14 @@ static void drm_fb_xrgb8888_to_xrgb2101010_line(void *dbuf, const void *sbuf, un · 9 unmodified lines | ||
| 750 | 855 | */ |
| 751 | 856 | void drm_fb_xrgb8888_to_xrgb2101010(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 752 | 857 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 753 | − const struct drm_rect *clip) | |
| 858 | + const struct drm_rect *clip, | |
| 859 | + struct drm_format_conv_state *state) | |
| 754 | 860 | { |
| 755 | 861 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 756 | 862 | 4, |
| 757 | 863 | }; |
| 758 | 864 | |
| 759 | − drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | |
| 865 | + drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, | |
| 760 | 866 | drm_fb_xrgb8888_to_xrgb2101010_line); |
| 761 | 867 | } |
| 762 | 868 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_xrgb2101010); |
| @@ -788,6 +894,7 @@ static void drm_fb_xrgb8888_to_argb2101010_line(void *dbuf, const void *sbuf, un · 25 unmodified lines | ||
| 788 | 894 | * @src: Array of XRGB8888 source buffers |
| 789 | 895 | * @fb: DRM framebuffer |
| 790 | 896 | * @clip: Clip rectangle area to copy |
| 897 | + * @state: Transform and conversion state | |
| 791 | 898 | * |
| 792 | 899 | * This function copies parts of a framebuffer to display memory and converts |
| 793 | 900 | * the color format during the process. The parameters @dst, @dst_pitch and |
| @@ -803,13 +910,14 @@ static void drm_fb_xrgb8888_to_argb2101010_line(void *dbuf, const void *sbuf, un · 9 unmodified lines | ||
| 803 | 910 | */ |
| 804 | 911 | void drm_fb_xrgb8888_to_argb2101010(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 805 | 912 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 806 | − const struct drm_rect *clip) | |
| 913 | + const struct drm_rect *clip, | |
| 914 | + struct drm_format_conv_state *state) | |
| 807 | 915 | { |
| 808 | 916 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 809 | 917 | 4, |
| 810 | 918 | }; |
| 811 | 919 | |
| 812 | − drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | |
| 920 | + drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, | |
| 813 | 921 | drm_fb_xrgb8888_to_argb2101010_line); |
| 814 | 922 | } |
| 815 | 923 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_argb2101010); |
| @@ -839,6 +947,7 @@ static void drm_fb_xrgb8888_to_gray8_line(void *dbuf, const void *sbuf, unsigned · 23 unmodified lines | ||
| 839 | 947 | * @src: Array of XRGB8888 source buffers |
| 840 | 948 | * @fb: DRM framebuffer |
| 841 | 949 | * @clip: Clip rectangle area to copy |
| 950 | + * @state: Transform and conversion state | |
| 842 | 951 | * |
| 843 | 952 | * This function copies parts of a framebuffer to display memory and converts the |
| 844 | 953 | * color format during the process. Destination and framebuffer formats must match. The |
| @@ -858,13 +967,13 @@ static void drm_fb_xrgb8888_to_gray8_line(void *dbuf, const void *sbuf, unsigned · 13 unmodified lines | ||
| 858 | 967 | */ |
| 859 | 968 | void drm_fb_xrgb8888_to_gray8(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 860 | 969 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 861 | − const struct drm_rect *clip) | |
| 970 | + const struct drm_rect *clip, struct drm_format_conv_state *state) | |
| 862 | 971 | { |
| 863 | 972 | static const u8 dst_pixsize[DRM_FORMAT_MAX_PLANES] = { |
| 864 | 973 | 1, |
| 865 | 974 | }; |
| 866 | 975 | |
| 867 | − drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, | |
| 976 | + drm_fb_xfrm(dst, dst_pitch, dst_pixsize, src, fb, clip, false, state, | |
| 868 | 977 | drm_fb_xrgb8888_to_gray8_line); |
| 869 | 978 | } |
| 870 | 979 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_gray8); |
| @@ -878,6 +987,7 @@ EXPORT_SYMBOL(drm_fb_xrgb8888_to_gray8); · 7 unmodified lines | ||
| 878 | 987 | * @src: The framebuffer memory to copy from |
| 879 | 988 | * @fb: The framebuffer to copy from |
| 880 | 989 | * @clip: Clip rectangle area to copy |
| 990 | + * @state: Transform and conversion state | |
| 881 | 991 | * |
| 882 | 992 | * This function copies parts of a framebuffer to display memory. If the |
| 883 | 993 | * formats of the display and the framebuffer mismatch, the blit function |
| @@ -896,7 +1006,7 @@ EXPORT_SYMBOL(drm_fb_xrgb8888_to_gray8); · 12 unmodified lines | ||
| 896 | 1006 | */ |
| 897 | 1007 | int drm_fb_blit(struct iosys_map *dst, const unsigned int *dst_pitch, uint32_t dst_format, |
| 898 | 1008 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 899 | − const struct drm_rect *clip) | |
| 1009 | + const struct drm_rect *clip, struct drm_format_conv_state *state) | |
| 900 | 1010 | { |
| 901 | 1011 | uint32_t fb_format = fb->format->format; |
| 902 | 1012 | |
| @@ -904,44 +1014,44 @@ int drm_fb_blit(struct iosys_map *dst, const unsigned int *dst_pitch, uint32_t d · 1 unmodified line | ||
| 904 | 1014 | drm_fb_memcpy(dst, dst_pitch, src, fb, clip); |
| 905 | 1015 | return 0; |
| 906 | 1016 | } else if (fb_format == (dst_format | DRM_FORMAT_BIG_ENDIAN)) { |
| 907 | − drm_fb_swab(dst, dst_pitch, src, fb, clip, false); | |
| 1017 | + drm_fb_swab(dst, dst_pitch, src, fb, clip, false, state); | |
| 908 | 1018 | return 0; |
| 909 | 1019 | } else if (fb_format == (dst_format & ~DRM_FORMAT_BIG_ENDIAN)) { |
| 910 | − drm_fb_swab(dst, dst_pitch, src, fb, clip, false); | |
| 1020 | + drm_fb_swab(dst, dst_pitch, src, fb, clip, false, state); | |
| 911 | 1021 | return 0; |
| 912 | 1022 | } else if (fb_format == DRM_FORMAT_XRGB8888) { |
| 913 | 1023 | if (dst_format == DRM_FORMAT_RGB565) { |
| 914 | − drm_fb_xrgb8888_to_rgb565(dst, dst_pitch, src, fb, clip, false); | |
| 1024 | + drm_fb_xrgb8888_to_rgb565(dst, dst_pitch, src, fb, clip, state, false); | |
| 915 | 1025 | return 0; |
| 916 | 1026 | } else if (dst_format == DRM_FORMAT_XRGB1555) { |
| 917 | − drm_fb_xrgb8888_to_xrgb1555(dst, dst_pitch, src, fb, clip); | |
| 1027 | + drm_fb_xrgb8888_to_xrgb1555(dst, dst_pitch, src, fb, clip, state); | |
| 918 | 1028 | return 0; |
| 919 | 1029 | } else if (dst_format == DRM_FORMAT_ARGB1555) { |
| 920 | − drm_fb_xrgb8888_to_argb1555(dst, dst_pitch, src, fb, clip); | |
| 1030 | + drm_fb_xrgb8888_to_argb1555(dst, dst_pitch, src, fb, clip, state); | |
| 921 | 1031 | return 0; |
| 922 | 1032 | } else if (dst_format == DRM_FORMAT_RGBA5551) { |
| 923 | − drm_fb_xrgb8888_to_rgba5551(dst, dst_pitch, src, fb, clip); | |
| 1033 | + drm_fb_xrgb8888_to_rgba5551(dst, dst_pitch, src, fb, clip, state); | |
| 924 | 1034 | return 0; |
| 925 | 1035 | } else if (dst_format == DRM_FORMAT_RGB888) { |
| 926 | − drm_fb_xrgb8888_to_rgb888(dst, dst_pitch, src, fb, clip); | |
| 1036 | + drm_fb_xrgb8888_to_rgb888(dst, dst_pitch, src, fb, clip, state); | |
| 927 | 1037 | return 0; |
| 928 | 1038 | } else if (dst_format == DRM_FORMAT_ARGB8888) { |
| 929 | − drm_fb_xrgb8888_to_argb8888(dst, dst_pitch, src, fb, clip); | |
| 1039 | + drm_fb_xrgb8888_to_argb8888(dst, dst_pitch, src, fb, clip, state); | |
| 930 | 1040 | return 0; |
| 931 | 1041 | } else if (dst_format == DRM_FORMAT_XBGR8888) { |
| 932 | − drm_fb_xrgb8888_to_xbgr8888(dst, dst_pitch, src, fb, clip); | |
| 1042 | + drm_fb_xrgb8888_to_xbgr8888(dst, dst_pitch, src, fb, clip, state); | |
| 933 | 1043 | return 0; |
| 934 | 1044 | } else if (dst_format == DRM_FORMAT_ABGR8888) { |
| 935 | − drm_fb_xrgb8888_to_abgr8888(dst, dst_pitch, src, fb, clip); | |
| 1045 | + drm_fb_xrgb8888_to_abgr8888(dst, dst_pitch, src, fb, clip, state); | |
| 936 | 1046 | return 0; |
| 937 | 1047 | } else if (dst_format == DRM_FORMAT_XRGB2101010) { |
| 938 | − drm_fb_xrgb8888_to_xrgb2101010(dst, dst_pitch, src, fb, clip); | |
| 1048 | + drm_fb_xrgb8888_to_xrgb2101010(dst, dst_pitch, src, fb, clip, state); | |
| 939 | 1049 | return 0; |
| 940 | 1050 | } else if (dst_format == DRM_FORMAT_ARGB2101010) { |
| 941 | − drm_fb_xrgb8888_to_argb2101010(dst, dst_pitch, src, fb, clip); | |
| 1051 | + drm_fb_xrgb8888_to_argb2101010(dst, dst_pitch, src, fb, clip, state); | |
| 942 | 1052 | return 0; |
| 943 | 1053 | } else if (dst_format == DRM_FORMAT_BGRX8888) { |
| 944 | − drm_fb_swab(dst, dst_pitch, src, fb, clip, false); | |
| 1054 | + drm_fb_swab(dst, dst_pitch, src, fb, clip, false, state); | |
| 945 | 1055 | return 0; |
| 946 | 1056 | } |
| 947 | 1057 | } |
| @@ -978,6 +1088,7 @@ static void drm_fb_gray8_to_mono_line(void *dbuf, const void *sbuf, unsigned int · 30 unmodified lines | ||
| 978 | 1088 | * @src: Array of XRGB8888 source buffers |
| 979 | 1089 | * @fb: DRM framebuffer |
| 980 | 1090 | * @clip: Clip rectangle area to copy |
| 1091 | + * @state: Transform and conversion state | |
| 981 | 1092 | * |
| 982 | 1093 | * This function copies parts of a framebuffer to display memory and converts the |
| 983 | 1094 | * color format during the process. Destination and framebuffer formats must match. The |
| @@ -1002,7 +1113,7 @@ static void drm_fb_gray8_to_mono_line(void *dbuf, const void *sbuf, unsigned int · 18 unmodified lines | ||
| 1002 | 1113 | */ |
| 1003 | 1114 | void drm_fb_xrgb8888_to_mono(struct iosys_map *dst, const unsigned int *dst_pitch, |
| 1004 | 1115 | const struct iosys_map *src, const struct drm_framebuffer *fb, |
| 1005 | − const struct drm_rect *clip) | |
| 1116 | + const struct drm_rect *clip, struct drm_format_conv_state *state) | |
| 1006 | 1117 | { |
| 1007 | 1118 | static const unsigned int default_dst_pitch[DRM_FORMAT_MAX_PLANES] = { |
| 1008 | 1119 | 0, 0, 0, 0 |
| @@ -1042,7 +1153,7 @@ void drm_fb_xrgb8888_to_mono(struct iosys_map *dst, const unsigned int *dst_pitc · 33 unmodified lines | ||
| 1042 | 1153 | * Allocate a buffer to be used for both copying from the cma |
| 1043 | 1154 | * memory and to store the intermediate grayscale line pixels. |
| 1044 | 1155 | */ |
| 1045 | − src32 = kmalloc(len_src32 + linepixels, GFP_KERNEL); | |
| 1156 | + src32 = drm_format_conv_state_reserve(state, len_src32 + linepixels, GFP_KERNEL); | |
| 1046 | 1157 | if (!src32) |
| 1047 | 1158 | return; |
| 1048 | 1159 | |
| @@ -1056,8 +1167,6 @@ void drm_fb_xrgb8888_to_mono(struct iosys_map *dst, const unsigned int *dst_pitc · 7 unmodified lines | ||
| 1056 | 1167 | vaddr += fb->pitches[0]; |
| 1057 | 1168 | mono += dst_pitch_0; |
| 1058 | 1169 | } |
| 1059 | − | |
| 1060 | − kfree(src32); | |
| 1061 | 1170 | } |
| 1062 | 1171 | EXPORT_SYMBOL(drm_fb_xrgb8888_to_mono); |
| 1063 | 1172 | |
| @@ -394,6 +394,31 @@ static void drm_mode_rmfb_work_fn(struct work_struct *w) · 393 unmodified lines | ||
| 394 | 394 | } |
| 395 | 395 | } |
| 396 | 396 | |
| 397 | +static int drm_mode_closefb(struct drm_framebuffer *fb, | |
| 398 | + struct drm_file *file_priv) | |
| 399 | +{ | |
| 400 | + struct drm_framebuffer *fbl; | |
| 401 | + bool found = false; | |
| 402 | + | |
| 403 | + mutex_lock(&file_priv->fbs_lock); | |
| 404 | + list_for_each_entry(fbl, &file_priv->fbs, filp_head) | |
| 405 | + if (fb == fbl) | |
| 406 | + found = true; | |
| 407 | + | |
| 408 | + if (!found) { | |
| 409 | + mutex_unlock(&file_priv->fbs_lock); | |
| 410 | + return -ENOENT; | |
| 411 | + } | |
| 412 | + | |
| 413 | + list_del_init(&fb->filp_head); | |
| 414 | + mutex_unlock(&file_priv->fbs_lock); | |
| 415 | + | |
| 416 | + /* Drop the reference that was stored in the fbs list */ | |
| 417 | + drm_framebuffer_put(fb); | |
| 418 | + | |
| 419 | + return 0; | |
| 420 | +} | |
| 421 | + | |
| 397 | 422 | /** |
| 398 | 423 | * drm_mode_rmfb - remove an FB from the configuration |
| 399 | 424 | * @dev: drm device |
| @@ -410,9 +435,8 @@ static void drm_mode_rmfb_work_fn(struct work_struct *w) · 10 unmodified lines | ||
| 410 | 435 | int drm_mode_rmfb(struct drm_device *dev, u32 fb_id, |
| 411 | 436 | struct drm_file *file_priv) |
| 412 | 437 | { |
| 413 | − struct drm_framebuffer *fb = NULL; | |
| 414 | − struct drm_framebuffer *fbl = NULL; | |
| 415 | − int found = 0; | |
| 438 | + struct drm_framebuffer *fb; | |
| 439 | + int ret; | |
| 416 | 440 | |
| 417 | 441 | if (!drm_core_check_feature(dev, DRIVER_MODESET)) |
| 418 | 442 | return -EOPNOTSUPP; |
| @@ -421,24 +445,13 @@ int drm_mode_rmfb(struct drm_device *dev, u32 fb_id, · 2 unmodified lines | ||
| 421 | 445 | if (!fb) |
| 422 | 446 | return -ENOENT; |
| 423 | 447 | |
| 424 | − mutex_lock(&file_priv->fbs_lock); | |
| 425 | − list_for_each_entry(fbl, &file_priv->fbs, filp_head) | |
| 426 | − if (fb == fbl) | |
| 427 | − found = 1; | |
| 428 | − if (!found) { | |
| 429 | − mutex_unlock(&file_priv->fbs_lock); | |
| 430 | − goto fail_unref; | |
| 448 | + ret = drm_mode_closefb(fb, file_priv); | |
| 449 | + if (ret != 0) { | |
| 450 | + drm_framebuffer_put(fb); | |
| 451 | + return ret; | |
| 431 | 452 | } |
| 432 | 453 | |
| 433 | − list_del_init(&fb->filp_head); | |
| 434 | − mutex_unlock(&file_priv->fbs_lock); | |
| 435 | − | |
| 436 | − /* drop the reference we picked up in framebuffer lookup */ | |
| 437 | − drm_framebuffer_put(fb); | |
| 438 | − | |
| 439 | 454 | /* |
| 440 | − * we now own the reference that was stored in the fbs list | |
| 441 | − * | |
| 442 | 455 | * drm_framebuffer_remove may fail with -EINTR on pending signals, |
| 443 | 456 | * so run this in a separate stack as there's no way to correctly |
| 444 | 457 | * handle this after the fb is already removed from the lookup table. |
| @@ -457,10 +470,6 @@ int drm_mode_rmfb(struct drm_device *dev, u32 fb_id, · 12 unmodified lines | ||
| 457 | 470 | drm_framebuffer_put(fb); |
| 458 | 471 | |
| 459 | 472 | return 0; |
| 460 | − | |
| 461 | −fail_unref: | |
| 462 | − drm_framebuffer_put(fb); | |
| 463 | − return -ENOENT; | |
| 464 | 473 | } |
| 465 | 474 | |
| 466 | 475 | int drm_mode_rmfb_ioctl(struct drm_device *dev, |
| @@ -471,6 +480,28 @@ int drm_mode_rmfb_ioctl(struct drm_device *dev, · 4 unmodified lines | ||
| 471 | 480 | return drm_mode_rmfb(dev, *fb_id, file_priv); |
| 472 | 481 | } |
| 473 | 482 | |
| 483 | +int drm_mode_closefb_ioctl(struct drm_device *dev, | |
| 484 | + void *data, struct drm_file *file_priv) | |
| 485 | +{ | |
| 486 | + struct drm_mode_closefb *r = data; | |
| 487 | + struct drm_framebuffer *fb; | |
| 488 | + int ret; | |
| 489 | + | |
| 490 | + if (!drm_core_check_feature(dev, DRIVER_MODESET)) | |
| 491 | + return -EOPNOTSUPP; | |
| 492 | + | |
| 493 | + if (r->pad) | |
| 494 | + return -EINVAL; | |
| 495 | + | |
| 496 | + fb = drm_framebuffer_lookup(dev, file_priv, r->fb_id); | |
| 497 | + if (!fb) | |
| 498 | + return -ENOENT; | |
| 499 | + | |
| 500 | + ret = drm_mode_closefb(fb, file_priv); | |
| 501 | + drm_framebuffer_put(fb); | |
| 502 | + return ret; | |
| 503 | +} | |
| 504 | + | |
| 474 | 505 | /** |
| 475 | 506 | * drm_mode_getfb - get FB info |
| 476 | 507 | * @dev: drm device for the ioctl |
| @@ -552,7 +583,7 @@ int drm_mode_getfb2_ioctl(struct drm_device *dev, · 75 unmodified lines | ||
| 552 | 583 | struct drm_mode_fb_cmd2 *r = data; |
| 553 | 584 | struct drm_framebuffer *fb; |
| 554 | 585 | unsigned int i; |
| 555 | − int ret; | |
| 586 | + int ret = 0; | |
| 556 | 587 | |
| 557 | 588 | if (!drm_core_check_feature(dev, DRIVER_MODESET)) |
| 558 | 589 | return -EINVAL; |
| @@ -218,7 +218,14 @@ void · 217 unmodified lines | ||
| 218 | 218 | __drm_gem_duplicate_shadow_plane_state(struct drm_plane *plane, |
| 219 | 219 | struct drm_shadow_plane_state *new_shadow_plane_state) |
| 220 | 220 | { |
| 221 | + struct drm_plane_state *plane_state = plane->state; | |
| 222 | + struct drm_shadow_plane_state *shadow_plane_state = | |
| 223 | + to_drm_shadow_plane_state(plane_state); | |
| 224 | + | |
| 221 | 225 | __drm_atomic_helper_plane_duplicate_state(plane, &new_shadow_plane_state->base); |
| 226 | + | |
| 227 | + drm_format_conv_state_copy(&shadow_plane_state->fmtcnv_state, | |
| 228 | + &new_shadow_plane_state->fmtcnv_state); | |
| 222 | 229 | } |
| 223 | 230 | EXPORT_SYMBOL(__drm_gem_duplicate_shadow_plane_state); |
| 224 | 231 | |
| @@ -266,6 +273,7 @@ EXPORT_SYMBOL(drm_gem_duplicate_shadow_plane_state); · 41 unmodified lines | ||
| 266 | 273 | */ |
| 267 | 274 | void __drm_gem_destroy_shadow_plane_state(struct drm_shadow_plane_state *shadow_plane_state) |
| 268 | 275 | { |
| 276 | + drm_format_conv_state_release(&shadow_plane_state->fmtcnv_state); | |
| 269 | 277 | __drm_atomic_helper_plane_destroy_state(&shadow_plane_state->base); |
| 270 | 278 | } |
| 271 | 279 | EXPORT_SYMBOL(__drm_gem_destroy_shadow_plane_state); |
| @@ -302,6 +310,7 @@ void __drm_gem_reset_shadow_plane(struct drm_plane *plane, · 30 unmodified lines | ||
| 302 | 310 | struct drm_shadow_plane_state *shadow_plane_state) |
| 303 | 311 | { |
| 304 | 312 | __drm_atomic_helper_plane_reset(plane, &shadow_plane_state->base); |
| 313 | + drm_format_conv_state_init(&shadow_plane_state->fmtcnv_state); | |
| 305 | 314 | } |
| 306 | 315 | EXPORT_SYMBOL(__drm_gem_reset_shadow_plane); |
| 307 | 316 | |
| @@ -61,6 +61,42 @@ · 60 unmodified lines | ||
| 61 | 61 | * contained within struct drm_gpuva already. Hence, for inserting &drm_gpuva |
| 62 | 62 | * entries from within dma-fence signalling critical sections it is enough to |
| 63 | 63 | * pre-allocate the &drm_gpuva structures. |
| 64 | + * | |
| 65 | + * &drm_gem_objects which are private to a single VM can share a common | |
| 66 | + * &dma_resv in order to improve locking efficiency (e.g. with &drm_exec). | |
| 67 | + * For this purpose drivers must pass a &drm_gem_object to drm_gpuvm_init(), in | |
| 68 | + * the following called 'resv object', which serves as the container of the | |
| 69 | + * GPUVM's shared &dma_resv. This resv object can be a driver specific | |
| 70 | + * &drm_gem_object, such as the &drm_gem_object containing the root page table, | |
| 71 | + * but it can also be a 'dummy' object, which can be allocated with | |
| 72 | + * drm_gpuvm_resv_object_alloc(). | |
| 73 | + * | |
| 74 | + * In order to connect a struct drm_gpuva its backing &drm_gem_object each | |
| 75 | + * &drm_gem_object maintains a list of &drm_gpuvm_bo structures, and each | |
| 76 | + * &drm_gpuvm_bo contains a list of &drm_gpuva structures. | |
| 77 | + * | |
| 78 | + * A &drm_gpuvm_bo is an abstraction that represents a combination of a | |
| 79 | + * &drm_gpuvm and a &drm_gem_object. Every such combination should be unique. | |
| 80 | + * This is ensured by the API through drm_gpuvm_bo_obtain() and | |
| 81 | + * drm_gpuvm_bo_obtain_prealloc() which first look into the corresponding | |
| 82 | + * &drm_gem_object list of &drm_gpuvm_bos for an existing instance of this | |
| 83 | + * particular combination. If not existent a new instance is created and linked | |
| 84 | + * to the &drm_gem_object. | |
| 85 | + * | |
| 86 | + * &drm_gpuvm_bo structures, since unique for a given &drm_gpuvm, are also used | |
| 87 | + * as entry for the &drm_gpuvm's lists of external and evicted objects. Those | |
| 88 | + * lists are maintained in order to accelerate locking of dma-resv locks and | |
| 89 | + * validation of evicted objects bound in a &drm_gpuvm. For instance, all | |
| 90 | + * &drm_gem_object's &dma_resv of a given &drm_gpuvm can be locked by calling | |
| 91 | + * drm_gpuvm_exec_lock(). Once locked drivers can call drm_gpuvm_validate() in | |
| 92 | + * order to validate all evicted &drm_gem_objects. It is also possible to lock | |
| 93 | + * additional &drm_gem_objects by providing the corresponding parameters to | |
| 94 | + * drm_gpuvm_exec_lock() as well as open code the &drm_exec loop while making | |
| 95 | + * use of helper functions such as drm_gpuvm_prepare_range() or | |
| 96 | + * drm_gpuvm_prepare_objects(). | |
| 97 | + * | |
| 98 | + * Every bound &drm_gem_object is treated as external object when its &dma_resv | |
| 99 | + * structure is different than the &drm_gpuvm's common &dma_resv structure. | |
| 64 | 100 | */ |
| 65 | 101 | |
| 66 | 102 | /** |
| @@ -386,21 +422,42 @@ · 319 unmodified lines | ||
| 386 | 422 | /** |
| 387 | 423 | * DOC: Locking |
| 388 | 424 | * |
| 389 | − * Generally, the GPU VA manager does not take care of locking itself, it is | |
| 390 | − * the drivers responsibility to take care about locking. Drivers might want to | |
| 391 | − * protect the following operations: inserting, removing and iterating | |
| 392 | − * &drm_gpuva objects as well as generating all kinds of operations, such as | |
| 393 | − * split / merge or prefetch. | |
| 394 | − * | |
| 395 | − * The GPU VA manager also does not take care of the locking of the backing | |
| 396 | − * &drm_gem_object buffers GPU VA lists by itself; drivers are responsible to | |
| 397 | − * enforce mutual exclusion using either the GEMs dma_resv lock or alternatively | |
| 398 | − * a driver specific external lock. For the latter see also | |
| 399 | − * drm_gem_gpuva_set_lock(). | |
| 400 | − * | |
| 401 | − * However, the GPU VA manager contains lockdep checks to ensure callers of its | |
| 402 | − * API hold the corresponding lock whenever the &drm_gem_objects GPU VA list is | |
| 403 | − * accessed by functions such as drm_gpuva_link() or drm_gpuva_unlink(). | |
| 425 | + * In terms of managing &drm_gpuva entries DRM GPUVM does not take care of | |
| 426 | + * locking itself, it is the drivers responsibility to take care about locking. | |
| 427 | + * Drivers might want to protect the following operations: inserting, removing | |
| 428 | + * and iterating &drm_gpuva objects as well as generating all kinds of | |
| 429 | + * operations, such as split / merge or prefetch. | |
| 430 | + * | |
| 431 | + * DRM GPUVM also does not take care of the locking of the backing | |
| 432 | + * &drm_gem_object buffers GPU VA lists and &drm_gpuvm_bo abstractions by | |
| 433 | + * itself; drivers are responsible to enforce mutual exclusion using either the | |
| 434 | + * GEMs dma_resv lock or alternatively a driver specific external lock. For the | |
| 435 | + * latter see also drm_gem_gpuva_set_lock(). | |
| 436 | + * | |
| 437 | + * However, DRM GPUVM contains lockdep checks to ensure callers of its API hold | |
| 438 | + * the corresponding lock whenever the &drm_gem_objects GPU VA list is accessed | |
| 439 | + * by functions such as drm_gpuva_link() or drm_gpuva_unlink(), but also | |
| 440 | + * drm_gpuvm_bo_obtain() and drm_gpuvm_bo_put(). | |
| 441 | + * | |
| 442 | + * The latter is required since on creation and destruction of a &drm_gpuvm_bo | |
| 443 | + * the &drm_gpuvm_bo is attached / removed from the &drm_gem_objects gpuva list. | |
| 444 | + * Subsequent calls to drm_gpuvm_bo_obtain() for the same &drm_gpuvm and | |
| 445 | + * &drm_gem_object must be able to observe previous creations and destructions | |
| 446 | + * of &drm_gpuvm_bos in order to keep instances unique. | |
| 447 | + * | |
| 448 | + * The &drm_gpuvm's lists for keeping track of external and evicted objects are | |
| 449 | + * protected against concurrent insertion / removal and iteration internally. | |
| 450 | + * | |
| 451 | + * However, drivers still need ensure to protect concurrent calls to functions | |
| 452 | + * iterating those lists, namely drm_gpuvm_prepare_objects() and | |
| 453 | + * drm_gpuvm_validate(). | |
| 454 | + * | |
| 455 | + * Alternatively, drivers can set the &DRM_GPUVM_RESV_PROTECTED flag to indicate | |
| 456 | + * that the corresponding &dma_resv locks are held in order to protect the | |
| 457 | + * lists. If &DRM_GPUVM_RESV_PROTECTED is set, internal locking is disabled and | |
| 458 | + * the corresponding lockdep checks are enabled. This is an optimization for | |
| 459 | + * drivers which are capable of taking the corresponding &dma_resv locks and | |
| 460 | + * hence do not require internal locking. | |
| 404 | 461 | */ |
| 405 | 462 | |
| 406 | 463 | /** |
| @@ -430,6 +487,7 @@ · 23 unmodified lines | ||
| 430 | 487 | * { |
| 431 | 488 | * struct drm_gpuva_ops *ops; |
| 432 | 489 | * struct drm_gpuva_op *op |
| 490 | + * struct drm_gpuvm_bo *vm_bo; | |
| 433 | 491 | * |
| 434 | 492 | * driver_lock_va_space(); |
| 435 | 493 | * ops = drm_gpuvm_sm_map_ops_create(gpuvm, addr, range, |
| @@ -437,6 +495,10 @@ · 1 unmodified line | ||
| 437 | 495 | * if (IS_ERR(ops)) |
| 438 | 496 | * return PTR_ERR(ops); |
| 439 | 497 | * |
| 498 | + * vm_bo = drm_gpuvm_bo_obtain(gpuvm, obj); | |
| 499 | + * if (IS_ERR(vm_bo)) | |
| 500 | + * return PTR_ERR(vm_bo); | |
| 501 | + * | |
| 440 | 502 | * drm_gpuva_for_each_op(op, ops) { |
| 441 | 503 | * struct drm_gpuva *va; |
| 442 | 504 | * |
| @@ -449,7 +511,7 @@ · 6 unmodified lines | ||
| 449 | 511 | * |
| 450 | 512 | * driver_vm_map(); |
| 451 | 513 | * drm_gpuva_map(gpuvm, va, &op->map); |
| 452 | − * drm_gpuva_link(va); | |
| 514 | + * drm_gpuva_link(va, vm_bo); | |
| 453 | 515 | * |
| 454 | 516 | * break; |
| 455 | 517 | * case DRM_GPUVA_OP_REMAP: { |
| @@ -476,11 +538,11 @@ · 20 unmodified lines | ||
| 476 | 538 | * driver_vm_remap(); |
| 477 | 539 | * drm_gpuva_remap(prev, next, &op->remap); |
| 478 | 540 | * |
| 479 | − * drm_gpuva_unlink(va); | |
| 480 | 541 | * if (prev) |
| 481 | − * drm_gpuva_link(prev); | |
| 542 | + * drm_gpuva_link(prev, va->vm_bo); | |
| 482 | 543 | * if (next) |
| 483 | − * drm_gpuva_link(next); | |
| 544 | + * drm_gpuva_link(next, va->vm_bo); | |
| 545 | + * drm_gpuva_unlink(va); | |
| 484 | 546 | * |
| 485 | 547 | * break; |
| 486 | 548 | * } |
| @@ -496,6 +558,7 @@ · 9 unmodified lines | ||
| 496 | 558 | * break; |
| 497 | 559 | * } |
| 498 | 560 | * } |
| 561 | + * drm_gpuvm_bo_put(vm_bo); | |
| 499 | 562 | * driver_unlock_va_space(); |
| 500 | 563 | * |
| 501 | 564 | * return 0; |
| @@ -505,6 +568,7 @@ · 3 unmodified lines | ||
| 505 | 568 | * |
| 506 | 569 | * struct driver_context { |
| 507 | 570 | * struct drm_gpuvm *gpuvm; |
| 571 | + * struct drm_gpuvm_bo *vm_bo; | |
| 508 | 572 | * struct drm_gpuva *new_va; |
| 509 | 573 | * struct drm_gpuva *prev_va; |
| 510 | 574 | * struct drm_gpuva *next_va; |
| @@ -525,6 +589,7 @@ · 14 unmodified lines | ||
| 525 | 589 | * struct drm_gem_object *obj, u64 offset) |
| 526 | 590 | * { |
| 527 | 591 | * struct driver_context ctx; |
| 592 | + * struct drm_gpuvm_bo *vm_bo; | |
| 528 | 593 | * struct drm_gpuva_ops *ops; |
| 529 | 594 | * struct drm_gpuva_op *op; |
| 530 | 595 | * int ret = 0; |
| @@ -534,16 +599,23 @@ · 3 unmodified lines | ||
| 534 | 599 | * ctx.new_va = kzalloc(sizeof(*ctx.new_va), GFP_KERNEL); |
| 535 | 600 | * ctx.prev_va = kzalloc(sizeof(*ctx.prev_va), GFP_KERNEL); |
| 536 | 601 | * ctx.next_va = kzalloc(sizeof(*ctx.next_va), GFP_KERNEL); |
| 537 | − * if (!ctx.new_va || !ctx.prev_va || !ctx.next_va) { | |
| 602 | + * ctx.vm_bo = drm_gpuvm_bo_create(gpuvm, obj); | |
| 603 | + * if (!ctx.new_va || !ctx.prev_va || !ctx.next_va || !vm_bo) { | |
| 538 | 604 | * ret = -ENOMEM; |
| 539 | 605 | * goto out; |
| 540 | 606 | * } |
| 541 | 607 | * |
| 608 | + * // Typically protected with a driver specific GEM gpuva lock | |
| 609 | + * // used in the fence signaling path for drm_gpuva_link() and | |
| 610 | + * // drm_gpuva_unlink(), hence pre-allocate. | |
| 611 | + * ctx.vm_bo = drm_gpuvm_bo_obtain_prealloc(ctx.vm_bo); | |
| 612 | + * | |
| 542 | 613 | * driver_lock_va_space(); |
| 543 | 614 | * ret = drm_gpuvm_sm_map(gpuvm, &ctx, addr, range, obj, offset); |
| 544 | 615 | * driver_unlock_va_space(); |
| 545 | 616 | * |
| 546 | 617 | * out: |
| 618 | + * drm_gpuvm_bo_put(ctx.vm_bo); | |
| 547 | 619 | * kfree(ctx.new_va); |
| 548 | 620 | * kfree(ctx.prev_va); |
| 549 | 621 | * kfree(ctx.next_va); |
| @@ -556,7 +628,7 @@ · 6 unmodified lines | ||
| 556 | 628 | * |
| 557 | 629 | * drm_gpuva_map(ctx->vm, ctx->new_va, &op->map); |
| 558 | 630 | * |
| 559 | − * drm_gpuva_link(ctx->new_va); | |
| 631 | + * drm_gpuva_link(ctx->new_va, ctx->vm_bo); | |
| 560 | 632 | * |
| 561 | 633 | * // prevent the new GPUVA from being freed in |
| 562 | 634 | * // driver_mapping_create() |
| @@ -568,22 +640,23 @@ · 5 unmodified lines | ||
| 568 | 640 | * int driver_gpuva_remap(struct drm_gpuva_op *op, void *__ctx) |
| 569 | 641 | * { |
| 570 | 642 | * struct driver_context *ctx = __ctx; |
| 643 | + * struct drm_gpuva *va = op->remap.unmap->va; | |
| 571 | 644 | * |
| 572 | 645 | * drm_gpuva_remap(ctx->prev_va, ctx->next_va, &op->remap); |
| 573 | 646 | * |
| 574 | − * drm_gpuva_unlink(op->remap.unmap->va); | |
| 575 | − * kfree(op->remap.unmap->va); | |
| 576 | − * | |
| 577 | 647 | * if (op->remap.prev) { |
| 578 | − * drm_gpuva_link(ctx->prev_va); | |
| 648 | + * drm_gpuva_link(ctx->prev_va, va->vm_bo); | |
| 579 | 649 | * ctx->prev_va = NULL; |
| 580 | 650 | * } |
| 581 | 651 | * |
| 582 | 652 | * if (op->remap.next) { |
| 583 | − * drm_gpuva_link(ctx->next_va); | |
| 653 | + * drm_gpuva_link(ctx->next_va, va->vm_bo); | |
| 584 | 654 | * ctx->next_va = NULL; |
| 585 | 655 | * } |
| 586 | 656 | * |
| 657 | + * drm_gpuva_unlink(va); | |
| 658 | + * kfree(va); | |
| 659 | + * | |
| 587 | 660 | * return 0; |
| 588 | 661 | * } |
| 589 | 662 | * |
| @@ -597,6 +670,201 @@ · 7 unmodified lines | ||
| 597 | 670 | * } |
| 598 | 671 | */ |
| 599 | 672 | |
| 673 | +/** | |
| 674 | + * get_next_vm_bo_from_list() - get the next vm_bo element | |
| 675 | + * @__gpuvm: the &drm_gpuvm | |
| 676 | + * @__list_name: the name of the list we're iterating on | |
| 677 | + * @__local_list: a pointer to the local list used to store already iterated items | |
| 678 | + * @__prev_vm_bo: the previous element we got from get_next_vm_bo_from_list() | |
| 679 | + * | |
| 680 | + * This helper is here to provide lockless list iteration. Lockless as in, the | |
| 681 | + * iterator releases the lock immediately after picking the first element from | |
| 682 | + * the list, so list insertion deletion can happen concurrently. | |
| 683 | + * | |
| 684 | + * Elements popped from the original list are kept in a local list, so removal | |
| 685 | + * and is_empty checks can still happen while we're iterating the list. | |
| 686 | + */ | |
| 687 | +#define get_next_vm_bo_from_list(__gpuvm, __list_name, __local_list, __prev_vm_bo) \ | |
| 688 | + ({ \ | |
| 689 | + struct drm_gpuvm_bo *__vm_bo = NULL; \ | |
| 690 | + \ | |
| 691 | + drm_gpuvm_bo_put(__prev_vm_bo); \ | |
| 692 | + \ | |
| 693 | + spin_lock(&(__gpuvm)->__list_name.lock); \ | |
| 694 | + if (!(__gpuvm)->__list_name.local_list) \ | |
| 695 | + (__gpuvm)->__list_name.local_list = __local_list; \ | |
| 696 | + else \ | |
| 697 | + drm_WARN_ON((__gpuvm)->drm, \ | |
| 698 | + (__gpuvm)->__list_name.local_list != __local_list); \ | |
| 699 | + \ | |
| 700 | + while (!list_empty(&(__gpuvm)->__list_name.list)) { \ | |
| 701 | + __vm_bo = list_first_entry(&(__gpuvm)->__list_name.list, \ | |
| 702 | + struct drm_gpuvm_bo, \ | |
| 703 | + list.entry.__list_name); \ | |
| 704 | + if (kref_get_unless_zero(&__vm_bo->kref)) { \ | |
| 705 | + list_move_tail(&(__vm_bo)->list.entry.__list_name, \ | |
| 706 | + __local_list); \ | |
| 707 | + break; \ | |
| 708 | + } else { \ | |
| 709 | + list_del_init(&(__vm_bo)->list.entry.__list_name); \ | |
| 710 | + __vm_bo = NULL; \ | |
| 711 | + } \ | |
| 712 | + } \ | |
| 713 | + spin_unlock(&(__gpuvm)->__list_name.lock); \ | |
| 714 | + \ | |
| 715 | + __vm_bo; \ | |
| 716 | + }) | |
| 717 | + | |
| 718 | +/** | |
| 719 | + * for_each_vm_bo_in_list() - internal vm_bo list iterator | |
| 720 | + * @__gpuvm: the &drm_gpuvm | |
| 721 | + * @__list_name: the name of the list we're iterating on | |
| 722 | + * @__local_list: a pointer to the local list used to store already iterated items | |
| 723 | + * @__vm_bo: the struct drm_gpuvm_bo to assign in each iteration step | |
| 724 | + * | |
| 725 | + * This helper is here to provide lockless list iteration. Lockless as in, the | |
| 726 | + * iterator releases the lock immediately after picking the first element from the | |
| 727 | + * list, hence list insertion and deletion can happen concurrently. | |
| 728 | + * | |
| 729 | + * It is not allowed to re-assign the vm_bo pointer from inside this loop. | |
| 730 | + * | |
| 731 | + * Typical use: | |
| 732 | + * | |
| 733 | + * struct drm_gpuvm_bo *vm_bo; | |
| 734 | + * LIST_HEAD(my_local_list); | |
| 735 | + * | |
| 736 | + * ret = 0; | |
| 737 | + * for_each_vm_bo_in_list(gpuvm, <list_name>, &my_local_list, vm_bo) { | |
| 738 | + * ret = do_something_with_vm_bo(..., vm_bo); | |
| 739 | + * if (ret) | |
| 740 | + * break; | |
| 741 | + * } | |
| 742 | + * // Drop ref in case we break out of the loop. | |
| 743 | + * drm_gpuvm_bo_put(vm_bo); | |
| 744 | + * restore_vm_bo_list(gpuvm, <list_name>, &my_local_list); | |
| 745 | + * | |
| 746 | + * | |
| 747 | + * Only used for internal list iterations, not meant to be exposed to the outside | |
| 748 | + * world. | |
| 749 | + */ | |
| 750 | +#define for_each_vm_bo_in_list(__gpuvm, __list_name, __local_list, __vm_bo) \ | |
| 751 | + for (__vm_bo = get_next_vm_bo_from_list(__gpuvm, __list_name, \ | |
| 752 | + __local_list, NULL); \ | |
| 753 | + __vm_bo; \ | |
| 754 | + __vm_bo = get_next_vm_bo_from_list(__gpuvm, __list_name, \ | |
| 755 | + __local_list, __vm_bo)) | |
| 756 | + | |
| 757 | +static void | |
| 758 | +__restore_vm_bo_list(struct drm_gpuvm *gpuvm, spinlock_t *lock, | |
| 759 | + struct list_head *list, struct list_head **local_list) | |
| 760 | +{ | |
| 761 | + /* Merge back the two lists, moving local list elements to the | |
| 762 | + * head to preserve previous ordering, in case it matters. | |
| 763 | + */ | |
| 764 | + spin_lock(lock); | |
| 765 | + if (*local_list) { | |
| 766 | + list_splice(*local_list, list); | |
| 767 | + *local_list = NULL; | |
| 768 | + } | |
| 769 | + spin_unlock(lock); | |
| 770 | +} | |
| 771 | + | |
| 772 | +/** | |
| 773 | + * restore_vm_bo_list() - move vm_bo elements back to their original list | |
| 774 | + * @__gpuvm: the &drm_gpuvm | |
| 775 | + * @__list_name: the name of the list we're iterating on | |
| 776 | + * | |
| 777 | + * When we're done iterating a vm_bo list, we should call restore_vm_bo_list() | |
| 778 | + * to restore the original state and let new iterations take place. | |
| 779 | + */ | |
| 780 | +#define restore_vm_bo_list(__gpuvm, __list_name) \ | |
| 781 | + __restore_vm_bo_list((__gpuvm), &(__gpuvm)->__list_name.lock, \ | |
| 782 | + &(__gpuvm)->__list_name.list, \ | |
| 783 | + &(__gpuvm)->__list_name.local_list) | |
| 784 | + | |
| 785 | +static void | |
| 786 | +cond_spin_lock(spinlock_t *lock, bool cond) | |
| 787 | +{ | |
| 788 | + if (cond) | |
| 789 | + spin_lock(lock); | |
| 790 | +} | |
| 791 | + | |
| 792 | +static void | |
| 793 | +cond_spin_unlock(spinlock_t *lock, bool cond) | |
| 794 | +{ | |
| 795 | + if (cond) | |
| 796 | + spin_unlock(lock); | |
| 797 | +} | |
| 798 | + | |
| 799 | +static void | |
| 800 | +__drm_gpuvm_bo_list_add(struct drm_gpuvm *gpuvm, spinlock_t *lock, | |
| 801 | + struct list_head *entry, struct list_head *list) | |
| 802 | +{ | |
| 803 | + cond_spin_lock(lock, !!lock); | |
| 804 | + if (list_empty(entry)) | |
| 805 | + list_add_tail(entry, list); | |
| 806 | + cond_spin_unlock(lock, !!lock); | |
| 807 | +} | |
| 808 | + | |
| 809 | +/** | |
| 810 | + * drm_gpuvm_bo_list_add() - insert a vm_bo into the given list | |
| 811 | + * @__vm_bo: the &drm_gpuvm_bo | |
| 812 | + * @__list_name: the name of the list to insert into | |
| 813 | + * @__lock: whether to lock with the internal spinlock | |
| 814 | + * | |
| 815 | + * Inserts the given @__vm_bo into the list specified by @__list_name. | |
| 816 | + */ | |
| 817 | +#define drm_gpuvm_bo_list_add(__vm_bo, __list_name, __lock) \ | |
| 818 | + __drm_gpuvm_bo_list_add((__vm_bo)->vm, \ | |
| 819 | + __lock ? &(__vm_bo)->vm->__list_name.lock : \ | |
| 820 | + NULL, \ | |
| 821 | + &(__vm_bo)->list.entry.__list_name, \ | |
| 822 | + &(__vm_bo)->vm->__list_name.list) | |
| 823 | + | |
| 824 | +static void | |
| 825 | +__drm_gpuvm_bo_list_del(struct drm_gpuvm *gpuvm, spinlock_t *lock, | |
| 826 | + struct list_head *entry, bool init) | |
| 827 | +{ | |
| 828 | + cond_spin_lock(lock, !!lock); | |
| 829 | + if (init) { | |
| 830 | + if (!list_empty(entry)) | |
| 831 | + list_del_init(entry); | |
| 832 | + } else { | |
| 833 | + list_del(entry); | |
| 834 | + } | |
| 835 | + cond_spin_unlock(lock, !!lock); | |
| 836 | +} | |
| 837 | + | |
| 838 | +/** | |
| 839 | + * drm_gpuvm_bo_list_del_init() - remove a vm_bo from the given list | |
| 840 | + * @__vm_bo: the &drm_gpuvm_bo | |
| 841 | + * @__list_name: the name of the list to insert into | |
| 842 | + * @__lock: whether to lock with the internal spinlock | |
| 843 | + * | |
| 844 | + * Removes the given @__vm_bo from the list specified by @__list_name. | |
| 845 | + */ | |
| 846 | +#define drm_gpuvm_bo_list_del_init(__vm_bo, __list_name, __lock) \ | |
| 847 | + __drm_gpuvm_bo_list_del((__vm_bo)->vm, \ | |
| 848 | + __lock ? &(__vm_bo)->vm->__list_name.lock : \ | |
| 849 | + NULL, \ | |
| 850 | + &(__vm_bo)->list.entry.__list_name, \ | |
| 851 | + true) | |
| 852 | + | |
| 853 | +/** | |
| 854 | + * drm_gpuvm_bo_list_del() - remove a vm_bo from the given list | |
| 855 | + * @__vm_bo: the &drm_gpuvm_bo | |
| 856 | + * @__list_name: the name of the list to insert into | |
| 857 | + * @__lock: whether to lock with the internal spinlock | |
| 858 | + * | |
| 859 | + * Removes the given @__vm_bo from the list specified by @__list_name. | |
| 860 | + */ | |
| 861 | +#define drm_gpuvm_bo_list_del(__vm_bo, __list_name, __lock) \ | |
| 862 | + __drm_gpuvm_bo_list_del((__vm_bo)->vm, \ | |
| 863 | + __lock ? &(__vm_bo)->vm->__list_name.lock : \ | |
| 864 | + NULL, \ | |
| 865 | + &(__vm_bo)->list.entry.__list_name, \ | |
| 866 | + false) | |
| 867 | + | |
| 600 | 868 | #define to_drm_gpuva(__node) container_of((__node), struct drm_gpuva, rb.node) |
| 601 | 869 | |
| 602 | 870 | #define GPUVA_START(node) ((node)->va.addr) |
| @@ -618,8 +886,14 @@ drm_gpuvm_check_overflow(u64 addr, u64 range) · 15 unmodified lines | ||
| 618 | 886 | { |
| 619 | 887 | u64 end; |
| 620 | 888 | |
| 621 | − return WARN(check_add_overflow(addr, range, &end), | |
| 622 | − "GPUVA address limited to %zu bytes.\n", sizeof(end)); | |
| 889 | + return check_add_overflow(addr, range, &end); | |
| 890 | +} | |
| 891 | + | |
| 892 | +static bool | |
| 893 | +drm_gpuvm_warn_check_overflow(struct drm_gpuvm *gpuvm, u64 addr, u64 range) | |
| 894 | +{ | |
| 895 | + return drm_WARN(gpuvm->drm, drm_gpuvm_check_overflow(addr, range), | |
| 896 | + "GPUVA address limited to %zu bytes.\n", sizeof(addr)); | |
| 623 | 897 | } |
| 624 | 898 | |
| 625 | 899 | static bool |
| @@ -643,7 +917,18 @@ drm_gpuvm_in_kernel_node(struct drm_gpuvm *gpuvm, u64 addr, u64 range) · 17 unmodified lines | ||
| 643 | 917 | return krange && addr < kend && kstart < end; |
| 644 | 918 | } |
| 645 | 919 | |
| 646 | −static bool | |
| 920 | +/** | |
| 921 | + * drm_gpuvm_range_valid() - checks whether the given range is valid for the | |
| 922 | + * given &drm_gpuvm | |
| 923 | + * @gpuvm: the GPUVM to check the range for | |
| 924 | + * @addr: the base address | |
| 925 | + * @range: the range starting from the base address | |
| 926 | + * | |
| 927 | + * Checks whether the range is within the GPUVM's managed boundaries. | |
| 928 | + * | |
| 929 | + * Returns: true for a valid range, false otherwise | |
| 930 | + */ | |
| 931 | +bool | |
| 647 | 932 | drm_gpuvm_range_valid(struct drm_gpuvm *gpuvm, |
| 648 | 933 | u64 addr, u64 range) |
| 649 | 934 | { |
| @@ -651,11 +936,52 @@ drm_gpuvm_range_valid(struct drm_gpuvm *gpuvm, · 1 unmodified line | ||
| 651 | 936 | drm_gpuvm_in_mm_range(gpuvm, addr, range) && |
| 652 | 937 | !drm_gpuvm_in_kernel_node(gpuvm, addr, range); |
| 653 | 938 | } |
| 939 | +EXPORT_SYMBOL_GPL(drm_gpuvm_range_valid); | |
| 940 | + | |
| 941 | +static void | |
| 942 | +drm_gpuvm_gem_object_free(struct drm_gem_object *obj) | |
| 943 | +{ | |
| 944 | + drm_gem_object_release(obj); | |
| 945 | + kfree(obj); | |
| 946 | +} | |
| 947 | + | |
| 948 | +static const struct drm_gem_object_funcs drm_gpuvm_object_funcs = { | |
| 949 | + .free = drm_gpuvm_gem_object_free, | |
| 950 | +}; | |
| 951 | + | |
| 952 | +/** | |
| 953 | + * drm_gpuvm_resv_object_alloc() - allocate a dummy &drm_gem_object | |
| 954 | + * @drm: the drivers &drm_device | |
| 955 | + * | |
| 956 | + * Allocates a dummy &drm_gem_object which can be passed to drm_gpuvm_init() in | |
| 957 | + * order to serve as root GEM object providing the &drm_resv shared across | |
| 958 | + * &drm_gem_objects local to a single GPUVM. | |
| 959 | + * | |
| 960 | + * Returns: the &drm_gem_object on success, NULL on failure | |
| 961 | + */ | |
| 962 | +struct drm_gem_object * | |
| 963 | +drm_gpuvm_resv_object_alloc(struct drm_device *drm) | |
| 964 | +{ | |
| 965 | + struct drm_gem_object *obj; | |
| 966 | + | |
| 967 | + obj = kzalloc(sizeof(*obj), GFP_KERNEL); | |
| 968 | + if (!obj) | |
| 969 | + return NULL; | |
| 970 | + | |
| 971 | + obj->funcs = &drm_gpuvm_object_funcs; | |
| 972 | + drm_gem_private_object_init(drm, obj, 0); | |
| 973 | + | |
| 974 | + return obj; | |
| 975 | +} | |
| 976 | +EXPORT_SYMBOL_GPL(drm_gpuvm_resv_object_alloc); | |
| 654 | 977 | |
| 655 | 978 | /** |
| 656 | 979 | * drm_gpuvm_init() - initialize a &drm_gpuvm |
| 657 | 980 | * @gpuvm: pointer to the &drm_gpuvm to initialize |
| 658 | 981 | * @name: the name of the GPU VA space |
| 982 | + * @flags: the &drm_gpuvm_flags for this GPUVM | |
| 983 | + * @drm: the &drm_device this VM resides in | |
| 984 | + * @r_obj: the resv &drm_gem_object providing the GPUVM's common &dma_resv | |
| 659 | 985 | * @start_offset: the start offset of the GPU VA space |
| 660 | 986 | * @range: the size of the GPU VA space |
| 661 | 987 | * @reserve_offset: the start of the kernel reserved GPU VA area |
| @@ -668,8 +994,10 @@ drm_gpuvm_range_valid(struct drm_gpuvm *gpuvm, · 6 unmodified lines | ||
| 668 | 994 | * &name is expected to be managed by the surrounding driver structures. |
| 669 | 995 | */ |
| 670 | 996 | void |
| 671 | −drm_gpuvm_init(struct drm_gpuvm *gpuvm, | |
| 672 | − const char *name, | |
| 997 | +drm_gpuvm_init(struct drm_gpuvm *gpuvm, const char *name, | |
| 998 | + enum drm_gpuvm_flags flags, | |
| 999 | + struct drm_device *drm, | |
| 1000 | + struct drm_gem_object *r_obj, | |
| 673 | 1001 | u64 start_offset, u64 range, |
| 674 | 1002 | u64 reserve_offset, u64 reserve_range, |
| 675 | 1003 | const struct drm_gpuvm_ops *ops) |
| @@ -677,45 +1005,713 @@ drm_gpuvm_init(struct drm_gpuvm *gpuvm, · 1 unmodified line | ||
| 677 | 1005 | gpuvm->rb.tree = RB_ROOT_CACHED; |
| 678 | 1006 | INIT_LIST_HEAD(&gpuvm->rb.list); |
| 679 | 1007 | |
| 680 | − drm_gpuvm_check_overflow(start_offset, range); | |
| 681 | − gpuvm->mm_start = start_offset; | |
| 682 | − gpuvm->mm_range = range; | |
| 1008 | + INIT_LIST_HEAD(&gpuvm->extobj.list); | |
| 1009 | + spin_lock_init(&gpuvm->extobj.lock); | |
| 1010 | + | |
| 1011 | + INIT_LIST_HEAD(&gpuvm->evict.list); | |
| 1012 | + spin_lock_init(&gpuvm->evict.lock); | |
| 1013 | + | |
| 1014 | + kref_init(&gpuvm->kref); | |
| 683 | 1015 | |
| 684 | 1016 | gpuvm->name = name ? name : "unknown"; |
| 1017 | + gpuvm->flags = flags; | |
| 685 | 1018 | gpuvm->ops = ops; |
| 1019 | + gpuvm->drm = drm; | |
| 1020 | + gpuvm->r_obj = r_obj; | |
| 686 | 1021 | |
| 687 | − memset(&gpuvm->kernel_alloc_node, 0, sizeof(struct drm_gpuva)); | |
| 1022 | + drm_gem_object_get(r_obj); | |
| 1023 | + | |
| 1024 | + drm_gpuvm_warn_check_overflow(gpuvm, start_offset, range); | |
| 1025 | + gpuvm->mm_start = start_offset; | |
| 1026 | + gpuvm->mm_range = range; | |
| 688 | 1027 | |
| 1028 | + memset(&gpuvm->kernel_alloc_node, 0, sizeof(struct drm_gpuva)); | |
| 689 | 1029 | if (reserve_range) { |
| 690 | 1030 | gpuvm->kernel_alloc_node.va.addr = reserve_offset; |
| 691 | 1031 | gpuvm->kernel_alloc_node.va.range = reserve_range; |
| 692 | 1032 | |
| 693 | − if (likely(!drm_gpuvm_check_overflow(reserve_offset, | |
| 694 | − reserve_range))) | |
| 1033 | + if (likely(!drm_gpuvm_warn_check_overflow(gpuvm, reserve_offset, | |
| 1034 | + reserve_range))) | |
| 695 | 1035 | __drm_gpuva_insert(gpuvm, &gpuvm->kernel_alloc_node); |
| 696 | 1036 | } |
| 697 | 1037 | } |
| 698 | 1038 | EXPORT_SYMBOL_GPL(drm_gpuvm_init); |
| 699 | 1039 | |
| 1040 | +static void | |
| 1041 | +drm_gpuvm_fini(struct drm_gpuvm *gpuvm) | |
| 1042 | +{ | |
| 1043 | + gpuvm->name = NULL; | |
| 1044 | + | |
| 1045 | + if (gpuvm->kernel_alloc_node.va.range) | |
| 1046 | + __drm_gpuva_remove(&gpuvm->kernel_alloc_node); | |
| 1047 | + | |
| 1048 | + drm_WARN(gpuvm->drm, !RB_EMPTY_ROOT(&gpuvm->rb.tree.rb_root), | |
| 1049 | + "GPUVA tree is not empty, potentially leaking memory.\n"); | |
| 1050 | + | |
| 1051 | + drm_WARN(gpuvm->drm, !list_empty(&gpuvm->extobj.list), | |
| 1052 | + "Extobj list should be empty.\n"); | |
| 1053 | + drm_WARN(gpuvm->drm, !list_empty(&gpuvm->evict.list), | |
| 1054 | + "Evict list should be empty.\n"); | |
| 1055 | + | |
| 1056 | + drm_gem_object_put(gpuvm->r_obj); | |
| 1057 | +} | |
| 1058 | + | |
| 1059 | +static void | |
| 1060 | +drm_gpuvm_free(struct kref *kref) | |
| 1061 | +{ | |
| 1062 | + struct drm_gpuvm *gpuvm = container_of(kref, struct drm_gpuvm, kref); | |
| 1063 | + | |
| 1064 | + drm_gpuvm_fini(gpuvm); | |
| 1065 | + | |
| 1066 | + if (drm_WARN_ON(gpuvm->drm, !gpuvm->ops->vm_free)) | |
| 1067 | + return; | |
| 1068 | + | |
| 1069 | + gpuvm->ops->vm_free(gpuvm); | |
| 1070 | +} | |
| 1071 | + | |
| 700 | 1072 | /** |
| 701 | − * drm_gpuvm_destroy() - cleanup a &drm_gpuvm | |
| 702 | − * @gpuvm: pointer to the &drm_gpuvm to clean up | |
| 1073 | + * drm_gpuvm_put() - drop a struct drm_gpuvm reference | |
| 1074 | + * @gpuvm: the &drm_gpuvm to release the reference of | |
| 1075 | + * | |
| 1076 | + * This releases a reference to @gpuvm. | |
| 703 | 1077 | * |
| 704 | − * Note that it is a bug to call this function on a manager that still | |
| 705 | − * holds GPU VA mappings. | |
| 1078 | + * This function may be called from atomic context. | |
| 706 | 1079 | */ |
| 707 | 1080 | void |
| 708 | −drm_gpuvm_destroy(struct drm_gpuvm *gpuvm) | |
| 1081 | +drm_gpuvm_put(struct drm_gpuvm *gpuvm) | |
| 709 | 1082 | { |
| 710 | − gpuvm->name = NULL; | |
| 1083 | + if (gpuvm) | |
| 1084 | + kref_put(&gpuvm->kref, drm_gpuvm_free); | |
| 1085 | +} | |
| 1086 | +EXPORT_SYMBOL_GPL(drm_gpuvm_put); | |
| 711 | 1087 | |
| 712 | − if (gpuvm->kernel_alloc_node.va.range) | |
| 713 | − __drm_gpuva_remove(&gpuvm->kernel_alloc_node); | |
| 1088 | +static int | |
| 1089 | +exec_prepare_obj(struct drm_exec *exec, struct drm_gem_object *obj, | |
| 1090 | + unsigned int num_fences) | |
| 1091 | +{ | |
| 1092 | + return num_fences ? drm_exec_prepare_obj(exec, obj, num_fences) : | |
| 1093 | + drm_exec_lock_obj(exec, obj); | |
| 1094 | +} | |
| 714 | 1095 | |
| 715 | − WARN(!RB_EMPTY_ROOT(&gpuvm->rb.tree.rb_root), | |
| 716 | − "GPUVA tree is not empty, potentially leaking memory."); | |
| 1096 | +/** | |
| 1097 | + * drm_gpuvm_prepare_vm() - prepare the GPUVMs common dma-resv | |
| 1098 | + * @gpuvm: the &drm_gpuvm | |
| 1099 | + * @exec: the &drm_exec context | |
| 1100 | + * @num_fences: the amount of &dma_fences to reserve | |
| 1101 | + * | |
| 1102 | + * Calls drm_exec_prepare_obj() for the GPUVMs dummy &drm_gem_object; if | |
| 1103 | + * @num_fences is zero drm_exec_lock_obj() is called instead. | |
| 1104 | + * | |
| 1105 | + * Using this function directly, it is the drivers responsibility to call | |
| 1106 | + * drm_exec_init() and drm_exec_fini() accordingly. | |
| 1107 | + * | |
| 1108 | + * Returns: 0 on success, negative error code on failure. | |
| 1109 | + */ | |
| 1110 | +int | |
| 1111 | +drm_gpuvm_prepare_vm(struct drm_gpuvm *gpuvm, | |
| 1112 | + struct drm_exec *exec, | |
| 1113 | + unsigned int num_fences) | |
| 1114 | +{ | |
| 1115 | + return exec_prepare_obj(exec, gpuvm->r_obj, num_fences); | |
| 1116 | +} | |
| 1117 | +EXPORT_SYMBOL_GPL(drm_gpuvm_prepare_vm); | |
| 1118 | + | |
| 1119 | +static int | |
| 1120 | +__drm_gpuvm_prepare_objects(struct drm_gpuvm *gpuvm, | |
| 1121 | + struct drm_exec *exec, | |
| 1122 | + unsigned int num_fences) | |
| 1123 | +{ | |
| 1124 | + struct drm_gpuvm_bo *vm_bo; | |
| 1125 | + LIST_HEAD(extobjs); | |
| 1126 | + int ret = 0; | |
| 1127 | + | |
| 1128 | + for_each_vm_bo_in_list(gpuvm, extobj, &extobjs, vm_bo) { | |
| 1129 | + ret = exec_prepare_obj(exec, vm_bo->obj, num_fences); | |
| 1130 | + if (ret) | |
| 1131 | + break; | |
| 1132 | + } | |
| 1133 | + /* Drop ref in case we break out of the loop. */ | |
| 1134 | + drm_gpuvm_bo_put(vm_bo); | |
| 1135 | + restore_vm_bo_list(gpuvm, extobj); | |
| 1136 | + | |
| 1137 | + return ret; | |
| 717 | 1138 | } |
| 718 | −EXPORT_SYMBOL_GPL(drm_gpuvm_destroy); | |
| 1139 | + | |
| 1140 | +static int | |
| 1141 | +drm_gpuvm_prepare_objects_locked(struct drm_gpuvm *gpuvm, | |
| 1142 | + struct drm_exec *exec, | |
| 1143 | + unsigned int num_fences) | |
| 1144 | +{ | |
| 1145 | + struct drm_gpuvm_bo *vm_bo; | |
| 1146 | + int ret = 0; | |
| 1147 | + | |
| 1148 | + drm_gpuvm_resv_assert_held(gpuvm); | |
| 1149 | + list_for_each_entry(vm_bo, &gpuvm->extobj.list, list.entry.extobj) { | |
| 1150 | + ret = exec_prepare_obj(exec, vm_bo->obj, num_fences); | |
| 1151 | + if (ret) | |
| 1152 | + break; | |
| 1153 | + | |
| 1154 | + if (vm_bo->evicted) | |
| 1155 | + drm_gpuvm_bo_list_add(vm_bo, evict, false); | |
| 1156 | + } | |
| 1157 | + | |
| 1158 | + return ret; | |
| 1159 | +} | |
| 1160 | + | |
| 1161 | +/** | |
| 1162 | + * drm_gpuvm_prepare_objects() - prepare all assoiciated BOs | |
| 1163 | + * @gpuvm: the &drm_gpuvm | |
| 1164 | + * @exec: the &drm_exec locking context | |
| 1165 | + * @num_fences: the amount of &dma_fences to reserve | |
| 1166 | + * | |
| 1167 | + * Calls drm_exec_prepare_obj() for all &drm_gem_objects the given | |
| 1168 | + * &drm_gpuvm contains mappings of; if @num_fences is zero drm_exec_lock_obj() | |
| 1169 | + * is called instead. | |
| 1170 | + * | |
| 1171 | + * Using this function directly, it is the drivers responsibility to call | |
| 1172 | + * drm_exec_init() and drm_exec_fini() accordingly. | |
| 1173 | + * | |
| 1174 | + * Note: This function is safe against concurrent insertion and removal of | |
| 1175 | + * external objects, however it is not safe against concurrent usage itself. | |
| 1176 | + * | |
| 1177 | + * Drivers need to make sure to protect this case with either an outer VM lock | |
| 1178 | + * or by calling drm_gpuvm_prepare_vm() before this function within the | |
| 1179 | + * drm_exec_until_all_locked() loop, such that the GPUVM's dma-resv lock ensures | |
| 1180 | + * mutual exclusion. | |
| 1181 | + * | |
| 1182 | + * Returns: 0 on success, negative error code on failure. | |
| 1183 | + */ | |
| 1184 | +int | |
| 1185 | +drm_gpuvm_prepare_objects(struct drm_gpuvm *gpuvm, | |
| 1186 | + struct drm_exec *exec, | |
| 1187 | + unsigned int num_fences) | |
| 1188 | +{ | |
| 1189 | + if (drm_gpuvm_resv_protected(gpuvm)) | |
| 1190 | + return drm_gpuvm_prepare_objects_locked(gpuvm, exec, | |
| 1191 | + num_fences); | |
| 1192 | + else | |
| 1193 | + return __drm_gpuvm_prepare_objects(gpuvm, exec, num_fences); | |
| 1194 | +} | |
| 1195 | +EXPORT_SYMBOL_GPL(drm_gpuvm_prepare_objects); | |
| 1196 | + | |
| 1197 | +/** | |
| 1198 | + * drm_gpuvm_prepare_range() - prepare all BOs mapped within a given range | |
| 1199 | + * @gpuvm: the &drm_gpuvm | |
| 1200 | + * @exec: the &drm_exec locking context | |
| 1201 | + * @addr: the start address within the VA space | |
| 1202 | + * @range: the range to iterate within the VA space | |
| 1203 | + * @num_fences: the amount of &dma_fences to reserve | |
| 1204 | + * | |
| 1205 | + * Calls drm_exec_prepare_obj() for all &drm_gem_objects mapped between @addr | |
| 1206 | + * and @addr + @range; if @num_fences is zero drm_exec_lock_obj() is called | |
| 1207 | + * instead. | |
| 1208 | + * | |
| 1209 | + * Returns: 0 on success, negative error code on failure. | |
| 1210 | + */ | |
| 1211 | +int | |
| 1212 | +drm_gpuvm_prepare_range(struct drm_gpuvm *gpuvm, struct drm_exec *exec, | |
| 1213 | + u64 addr, u64 range, unsigned int num_fences) | |
| 1214 | +{ | |
| 1215 | + struct drm_gpuva *va; | |
| 1216 | + u64 end = addr + range; | |
| 1217 | + int ret; | |
| 1218 | + | |
| 1219 | + drm_gpuvm_for_each_va_range(va, gpuvm, addr, end) { | |
| 1220 | + struct drm_gem_object *obj = va->gem.obj; | |
| 1221 | + | |
| 1222 | + ret = exec_prepare_obj(exec, obj, num_fences); | |
| 1223 | + if (ret) | |
| 1224 | + return ret; | |
| 1225 | + } | |
| 1226 | + | |
| 1227 | + return 0; | |
| 1228 | +} | |
| 1229 | +EXPORT_SYMBOL_GPL(drm_gpuvm_prepare_range); | |
| 1230 | + | |
| 1231 | +/** | |
| 1232 | + * drm_gpuvm_exec_lock() - lock all dma-resv of all assoiciated BOs | |
| 1233 | + * @vm_exec: the &drm_gpuvm_exec wrapper | |
| 1234 | + * | |
| 1235 | + * Acquires all dma-resv locks of all &drm_gem_objects the given | |
| 1236 | + * &drm_gpuvm contains mappings of. | |
| 1237 | + * | |
| 1238 | + * Addionally, when calling this function with struct drm_gpuvm_exec::extra | |
| 1239 | + * being set the driver receives the given @fn callback to lock additional | |
| 1240 | + * dma-resv in the context of the &drm_gpuvm_exec instance. Typically, drivers | |
| 1241 | + * would call drm_exec_prepare_obj() from within this callback. | |
| 1242 | + * | |
| 1243 | + * Returns: 0 on success, negative error code on failure. | |
| 1244 | + */ | |
| 1245 | +int | |
| 1246 | +drm_gpuvm_exec_lock(struct drm_gpuvm_exec *vm_exec) | |
| 1247 | +{ | |
| 1248 | + struct drm_gpuvm *gpuvm = vm_exec->vm; | |
| 1249 | + struct drm_exec *exec = &vm_exec->exec; | |
| 1250 | + unsigned int num_fences = vm_exec->num_fences; | |
| 1251 | + int ret; | |
| 1252 | + | |
| 1253 | + drm_exec_init(exec, vm_exec->flags); | |
| 1254 | + | |
| 1255 | + drm_exec_until_all_locked(exec) { | |
| 1256 | + ret = drm_gpuvm_prepare_vm(gpuvm, exec, num_fences); | |
| 1257 | + drm_exec_retry_on_contention(exec); | |
| 1258 | + if (ret) | |
| 1259 | + goto err; | |
| 1260 | + | |
| 1261 | + ret = drm_gpuvm_prepare_objects(gpuvm, exec, num_fences); | |
| 1262 | + drm_exec_retry_on_contention(exec); | |
| 1263 | + if (ret) | |
| 1264 | + goto err; | |
| 1265 | + | |
| 1266 | + if (vm_exec->extra.fn) { | |
| 1267 | + ret = vm_exec->extra.fn(vm_exec); | |
| 1268 | + drm_exec_retry_on_contention(exec); | |
| 1269 | + if (ret) | |
| 1270 | + goto err; | |
| 1271 | + } | |
| 1272 | + } | |
| 1273 | + | |
| 1274 | + return 0; | |
| 1275 | + | |
| 1276 | +err: | |
| 1277 | + drm_exec_fini(exec); | |
| 1278 | + return ret; | |
| 1279 | +} | |
| 1280 | +EXPORT_SYMBOL_GPL(drm_gpuvm_exec_lock); | |
| 1281 | + | |
| 1282 | +static int | |
| 1283 | +fn_lock_array(struct drm_gpuvm_exec *vm_exec) | |
| 1284 | +{ | |
| 1285 | + struct { | |
| 1286 | + struct drm_gem_object **objs; | |
| 1287 | + unsigned int num_objs; | |
| 1288 | + } *args = vm_exec->extra.priv; | |
| 1289 | + | |
| 1290 | + return drm_exec_prepare_array(&vm_exec->exec, args->objs, | |
| 1291 | + args->num_objs, vm_exec->num_fences); | |
| 1292 | +} | |
| 1293 | + | |
| 1294 | +/** | |
| 1295 | + * drm_gpuvm_exec_lock_array() - lock all dma-resv of all assoiciated BOs | |
| 1296 | + * @vm_exec: the &drm_gpuvm_exec wrapper | |
| 1297 | + * @objs: additional &drm_gem_objects to lock | |
| 1298 | + * @num_objs: the number of additional &drm_gem_objects to lock | |
| 1299 | + * | |
| 1300 | + * Acquires all dma-resv locks of all &drm_gem_objects the given &drm_gpuvm | |
| 1301 | + * contains mappings of, plus the ones given through @objs. | |
| 1302 | + * | |
| 1303 | + * Returns: 0 on success, negative error code on failure. | |
| 1304 | + */ | |
| 1305 | +int | |
| 1306 | +drm_gpuvm_exec_lock_array(struct drm_gpuvm_exec *vm_exec, | |
| 1307 | + struct drm_gem_object **objs, | |
| 1308 | + unsigned int num_objs) | |
| 1309 | +{ | |
| 1310 | + struct { | |
| 1311 | + struct drm_gem_object **objs; | |
| 1312 | + unsigned int num_objs; | |
| 1313 | + } args; | |
| 1314 | + | |
| 1315 | + args.objs = objs; | |
| 1316 | + args.num_objs = num_objs; | |
| 1317 | + | |
| 1318 | + vm_exec->extra.fn = fn_lock_array; | |
| 1319 | + vm_exec->extra.priv = &args; | |
| 1320 | + | |
| 1321 | + return drm_gpuvm_exec_lock(vm_exec); | |
| 1322 | +} | |
| 1323 | +EXPORT_SYMBOL_GPL(drm_gpuvm_exec_lock_array); | |
| 1324 | + | |
| 1325 | +/** | |
| 1326 | + * drm_gpuvm_exec_lock_range() - prepare all BOs mapped within a given range | |
| 1327 | + * @vm_exec: the &drm_gpuvm_exec wrapper | |
| 1328 | + * @addr: the start address within the VA space | |
| 1329 | + * @range: the range to iterate within the VA space | |
| 1330 | + * | |
| 1331 | + * Acquires all dma-resv locks of all &drm_gem_objects mapped between @addr and | |
| 1332 | + * @addr + @range. | |
| 1333 | + * | |
| 1334 | + * Returns: 0 on success, negative error code on failure. | |
| 1335 | + */ | |
| 1336 | +int | |
| 1337 | +drm_gpuvm_exec_lock_range(struct drm_gpuvm_exec *vm_exec, | |
| 1338 | + u64 addr, u64 range) | |
| 1339 | +{ | |
| 1340 | + struct drm_gpuvm *gpuvm = vm_exec->vm; | |
| 1341 | + struct drm_exec *exec = &vm_exec->exec; | |
| 1342 | + int ret; | |
| 1343 | + | |
| 1344 | + drm_exec_init(exec, vm_exec->flags); | |
| 1345 | + | |
| 1346 | + drm_exec_until_all_locked(exec) { | |
| 1347 | + ret = drm_gpuvm_prepare_range(gpuvm, exec, addr, range, | |
| 1348 | + vm_exec->num_fences); | |
| 1349 | + drm_exec_retry_on_contention(exec); | |
| 1350 | + if (ret) | |
| 1351 | + goto err; | |
| 1352 | + } | |
| 1353 | + | |
| 1354 | + return ret; | |
| 1355 | + | |
| 1356 | +err: | |
| 1357 | + drm_exec_fini(exec); | |
| 1358 | + return ret; | |
| 1359 | +} | |
| 1360 | +EXPORT_SYMBOL_GPL(drm_gpuvm_exec_lock_range); | |
| 1361 | + | |
| 1362 | +static int | |
| 1363 | +__drm_gpuvm_validate(struct drm_gpuvm *gpuvm, struct drm_exec *exec) | |
| 1364 | +{ | |
| 1365 | + const struct drm_gpuvm_ops *ops = gpuvm->ops; | |
| 1366 | + struct drm_gpuvm_bo *vm_bo; | |
| 1367 | + LIST_HEAD(evict); | |
| 1368 | + int ret = 0; | |
| 1369 | + | |
| 1370 | + for_each_vm_bo_in_list(gpuvm, evict, &evict, vm_bo) { | |
| 1371 | + ret = ops->vm_bo_validate(vm_bo, exec); | |
| 1372 | + if (ret) | |
| 1373 | + break; | |
| 1374 | + } | |
| 1375 | + /* Drop ref in case we break out of the loop. */ | |
| 1376 | + drm_gpuvm_bo_put(vm_bo); | |
| 1377 | + restore_vm_bo_list(gpuvm, evict); | |
| 1378 | + | |
| 1379 | + return ret; | |
| 1380 | +} | |
| 1381 | + | |
| 1382 | +static int | |
| 1383 | +drm_gpuvm_validate_locked(struct drm_gpuvm *gpuvm, struct drm_exec *exec) | |
| 1384 | +{ | |
| 1385 | + const struct drm_gpuvm_ops *ops = gpuvm->ops; | |
| 1386 | + struct drm_gpuvm_bo *vm_bo, *next; | |
| 1387 | + int ret = 0; | |
| 1388 | + | |
| 1389 | + drm_gpuvm_resv_assert_held(gpuvm); | |
| 1390 | + | |
| 1391 | + list_for_each_entry_safe(vm_bo, next, &gpuvm->evict.list, | |
| 1392 | + list.entry.evict) { | |
| 1393 | + ret = ops->vm_bo_validate(vm_bo, exec); | |
| 1394 | + if (ret) | |
| 1395 | + break; | |
| 1396 | + | |
| 1397 | + dma_resv_assert_held(vm_bo->obj->resv); | |
| 1398 | + if (!vm_bo->evicted) | |
| 1399 | + drm_gpuvm_bo_list_del_init(vm_bo, evict, false); | |
| 1400 | + } | |
| 1401 | + | |
| 1402 | + return ret; | |
| 1403 | +} | |
| 1404 | + | |
| 1405 | +/** | |
| 1406 | + * drm_gpuvm_validate() - validate all BOs marked as evicted | |
| 1407 | + * @gpuvm: the &drm_gpuvm to validate evicted BOs | |
| 1408 | + * @exec: the &drm_exec instance used for locking the GPUVM | |
| 1409 | + * | |
| 1410 | + * Calls the &drm_gpuvm_ops::vm_bo_validate callback for all evicted buffer | |
| 1411 | + * objects being mapped in the given &drm_gpuvm. | |
| 1412 | + * | |
| 1413 | + * Returns: 0 on success, negative error code on failure. | |
| 1414 | + */ | |
| 1415 | +int | |
| 1416 | +drm_gpuvm_validate(struct drm_gpuvm *gpuvm, struct drm_exec *exec) | |
| 1417 | +{ | |
| 1418 | + const struct drm_gpuvm_ops *ops = gpuvm->ops; | |
| 1419 | + | |
| 1420 | + if (unlikely(!ops || !ops->vm_bo_validate)) | |
| 1421 | + return -EOPNOTSUPP; | |
| 1422 | + | |
| 1423 | + if (drm_gpuvm_resv_protected(gpuvm)) | |
| 1424 | + return drm_gpuvm_validate_locked(gpuvm, exec); | |
| 1425 | + else | |
| 1426 | + return __drm_gpuvm_validate(gpuvm, exec); | |
| 1427 | +} | |
| 1428 | +EXPORT_SYMBOL_GPL(drm_gpuvm_validate); | |
| 1429 | + | |
| 1430 | +/** | |
| 1431 | + * drm_gpuvm_resv_add_fence - add fence to private and all extobj | |
| 1432 | + * dma-resv | |
| 1433 | + * @gpuvm: the &drm_gpuvm to add a fence to | |
| 1434 | + * @exec: the &drm_exec locking context | |
| 1435 | + * @fence: fence to add | |
| 1436 | + * @private_usage: private dma-resv usage | |
| 1437 | + * @extobj_usage: extobj dma-resv usage | |
| 1438 | + */ | |
| 1439 | +void | |
| 1440 | +drm_gpuvm_resv_add_fence(struct drm_gpuvm *gpuvm, | |
| 1441 | + struct drm_exec *exec, | |
| 1442 | + struct dma_fence *fence, | |
| 1443 | + enum dma_resv_usage private_usage, | |
| 1444 | + enum dma_resv_usage extobj_usage) | |
| 1445 | +{ | |
| 1446 | + struct drm_gem_object *obj; | |
| 1447 | + unsigned long index; | |
| 1448 | + | |
| 1449 | + drm_exec_for_each_locked_object(exec, index, obj) { | |
| 1450 | + dma_resv_assert_held(obj->resv); | |
| 1451 | + dma_resv_add_fence(obj->resv, fence, | |
| 1452 | + drm_gpuvm_is_extobj(gpuvm, obj) ? | |
| 1453 | + extobj_usage : private_usage); | |
| 1454 | + } | |
| 1455 | +} | |
| 1456 | +EXPORT_SYMBOL_GPL(drm_gpuvm_resv_add_fence); | |
| 1457 | + | |
| 1458 | +/** | |
| 1459 | + * drm_gpuvm_bo_create() - create a new instance of struct drm_gpuvm_bo | |
| 1460 | + * @gpuvm: The &drm_gpuvm the @obj is mapped in. | |
| 1461 | + * @obj: The &drm_gem_object being mapped in the @gpuvm. | |
| 1462 | + * | |
| 1463 | + * If provided by the driver, this function uses the &drm_gpuvm_ops | |
| 1464 | + * vm_bo_alloc() callback to allocate. | |
| 1465 | + * | |
| 1466 | + * Returns: a pointer to the &drm_gpuvm_bo on success, NULL on failure | |
| 1467 | + */ | |
| 1468 | +struct drm_gpuvm_bo * | |
| 1469 | +drm_gpuvm_bo_create(struct drm_gpuvm *gpuvm, | |
| 1470 | + struct drm_gem_object *obj) | |
| 1471 | +{ | |
| 1472 | + const struct drm_gpuvm_ops *ops = gpuvm->ops; | |
| 1473 | + struct drm_gpuvm_bo *vm_bo; | |
| 1474 | + | |
| 1475 | + if (ops && ops->vm_bo_alloc) | |
| 1476 | + vm_bo = ops->vm_bo_alloc(); | |
| 1477 | + else | |
| 1478 | + vm_bo = kzalloc(sizeof(*vm_bo), GFP_KERNEL); | |
| 1479 | + | |
| 1480 | + if (unlikely(!vm_bo)) | |
| 1481 | + return NULL; | |
| 1482 | + | |
| 1483 | + vm_bo->vm = drm_gpuvm_get(gpuvm); | |
| 1484 | + vm_bo->obj = obj; | |
| 1485 | + drm_gem_object_get(obj); | |
| 1486 | + | |
| 1487 | + kref_init(&vm_bo->kref); | |
| 1488 | + INIT_LIST_HEAD(&vm_bo->list.gpuva); | |
| 1489 | + INIT_LIST_HEAD(&vm_bo->list.entry.gem); | |
| 1490 | + | |
| 1491 | + INIT_LIST_HEAD(&vm_bo->list.entry.extobj); | |
| 1492 | + INIT_LIST_HEAD(&vm_bo->list.entry.evict); | |
| 1493 | + | |
| 1494 | + return vm_bo; | |
| 1495 | +} | |
| 1496 | +EXPORT_SYMBOL_GPL(drm_gpuvm_bo_create); | |
| 1497 | + | |
| 1498 | +static void | |
| 1499 | +drm_gpuvm_bo_destroy(struct kref *kref) | |
| 1500 | +{ | |
| 1501 | + struct drm_gpuvm_bo *vm_bo = container_of(kref, struct drm_gpuvm_bo, | |
| 1502 | + kref); | |
| 1503 | + struct drm_gpuvm *gpuvm = vm_bo->vm; | |
| 1504 | + const struct drm_gpuvm_ops *ops = gpuvm->ops; | |
| 1505 | + struct drm_gem_object *obj = vm_bo->obj; | |
| 1506 | + bool lock = !drm_gpuvm_resv_protected(gpuvm); | |
| 1507 | + | |
| 1508 | + if (!lock) | |
| 1509 | + drm_gpuvm_resv_assert_held(gpuvm); | |
| 1510 | + | |
| 1511 | + drm_gpuvm_bo_list_del(vm_bo, extobj, lock); | |
| 1512 | + drm_gpuvm_bo_list_del(vm_bo, evict, lock); | |
| 1513 | + | |
| 1514 | + drm_gem_gpuva_assert_lock_held(obj); | |
| 1515 | + list_del(&vm_bo->list.entry.gem); | |
| 1516 | + | |
| 1517 | + if (ops && ops->vm_bo_free) | |
| 1518 | + ops->vm_bo_free(vm_bo); | |
| 1519 | + else | |
| 1520 | + kfree(vm_bo); | |
| 1521 | + | |
| 1522 | + drm_gpuvm_put(gpuvm); | |
| 1523 | + drm_gem_object_put(obj); | |
| 1524 | +} | |
| 1525 | + | |
| 1526 | +/** | |
| 1527 | + * drm_gpuvm_bo_put() - drop a struct drm_gpuvm_bo reference | |
| 1528 | + * @vm_bo: the &drm_gpuvm_bo to release the reference of | |
| 1529 | + * | |
| 1530 | + * This releases a reference to @vm_bo. | |
| 1531 | + * | |
| 1532 | + * If the reference count drops to zero, the &gpuvm_bo is destroyed, which | |
| 1533 | + * includes removing it from the GEMs gpuva list. Hence, if a call to this | |
| 1534 | + * function can potentially let the reference count drop to zero the caller must | |
| 1535 | + * hold the dma-resv or driver specific GEM gpuva lock. | |
| 1536 | + * | |
| 1537 | + * This function may only be called from non-atomic context. | |
| 1538 | + * | |
| 1539 | + * Returns: true if vm_bo was destroyed, false otherwise. | |
| 1540 | + */ | |
| 1541 | +bool | |
| 1542 | +drm_gpuvm_bo_put(struct drm_gpuvm_bo *vm_bo) | |
| 1543 | +{ | |
| 1544 | + might_sleep(); | |
| 1545 | + | |
| 1546 | + if (vm_bo) | |
| 1547 | + return !!kref_put(&vm_bo->kref, drm_gpuvm_bo_destroy); | |
| 1548 | + | |
| 1549 | + return false; | |
| 1550 | +} | |
| 1551 | +EXPORT_SYMBOL_GPL(drm_gpuvm_bo_put); | |
| 1552 | + | |
| 1553 | +static struct drm_gpuvm_bo * | |
| 1554 | +__drm_gpuvm_bo_find(struct drm_gpuvm *gpuvm, | |
| 1555 | + struct drm_gem_object *obj) | |
| 1556 | +{ | |
| 1557 | + struct drm_gpuvm_bo *vm_bo; | |
| 1558 | + | |
| 1559 | + drm_gem_gpuva_assert_lock_held(obj); | |
| 1560 | + drm_gem_for_each_gpuvm_bo(vm_bo, obj) | |
| 1561 | + if (vm_bo->vm == gpuvm) | |
| 1562 | + return vm_bo; | |
| 1563 | + | |
| 1564 | + return NULL; | |
| 1565 | +} | |
| 1566 | + | |
| 1567 | +/** | |
| 1568 | + * drm_gpuvm_bo_find() - find the &drm_gpuvm_bo for the given | |
| 1569 | + * &drm_gpuvm and &drm_gem_object | |
| 1570 | + * @gpuvm: The &drm_gpuvm the @obj is mapped in. | |
| 1571 | + * @obj: The &drm_gem_object being mapped in the @gpuvm. | |
| 1572 | + * | |
| 1573 | + * Find the &drm_gpuvm_bo representing the combination of the given | |
| 1574 | + * &drm_gpuvm and &drm_gem_object. If found, increases the reference | |
| 1575 | + * count of the &drm_gpuvm_bo accordingly. | |
| 1576 | + * | |
| 1577 | + * Returns: a pointer to the &drm_gpuvm_bo on success, NULL on failure | |
| 1578 | + */ | |
| 1579 | +struct drm_gpuvm_bo * | |
| 1580 | +drm_gpuvm_bo_find(struct drm_gpuvm *gpuvm, | |
| 1581 | + struct drm_gem_object *obj) | |
| 1582 | +{ | |
| 1583 | + struct drm_gpuvm_bo *vm_bo = __drm_gpuvm_bo_find(gpuvm, obj); | |
| 1584 | + | |
| 1585 | + return vm_bo ? drm_gpuvm_bo_get(vm_bo) : NULL; | |
| 1586 | +} | |
| 1587 | +EXPORT_SYMBOL_GPL(drm_gpuvm_bo_find); | |
| 1588 | + | |
| 1589 | +/** | |
| 1590 | + * drm_gpuvm_bo_obtain() - obtains and instance of the &drm_gpuvm_bo for the | |
| 1591 | + * given &drm_gpuvm and &drm_gem_object | |
| 1592 | + * @gpuvm: The &drm_gpuvm the @obj is mapped in. | |
| 1593 | + * @obj: The &drm_gem_object being mapped in the @gpuvm. | |
| 1594 | + * | |
| 1595 | + * Find the &drm_gpuvm_bo representing the combination of the given | |
| 1596 | + * &drm_gpuvm and &drm_gem_object. If found, increases the reference | |
| 1597 | + * count of the &drm_gpuvm_bo accordingly. If not found, allocates a new | |
| 1598 | + * &drm_gpuvm_bo. | |
| 1599 | + * | |
| 1600 | + * A new &drm_gpuvm_bo is added to the GEMs gpuva list. | |
| 1601 | + * | |
| 1602 | + * Returns: a pointer to the &drm_gpuvm_bo on success, an ERR_PTR on failure | |
| 1603 | + */ | |
| 1604 | +struct drm_gpuvm_bo * | |
| 1605 | +drm_gpuvm_bo_obtain(struct drm_gpuvm *gpuvm, | |
| 1606 | + struct drm_gem_object *obj) | |
| 1607 | +{ | |
| 1608 | + struct drm_gpuvm_bo *vm_bo; | |
| 1609 | + | |
| 1610 | + vm_bo = drm_gpuvm_bo_find(gpuvm, obj); | |
| 1611 | + if (vm_bo) | |
| 1612 | + return vm_bo; | |
| 1613 | + | |
| 1614 | + vm_bo = drm_gpuvm_bo_create(gpuvm, obj); | |
| 1615 | + if (!vm_bo) | |
| 1616 | + return ERR_PTR(-ENOMEM); | |
| 1617 | + | |
| 1618 | + drm_gem_gpuva_assert_lock_held(obj); | |
| 1619 | + list_add_tail(&vm_bo->list.entry.gem, &obj->gpuva.list); | |
| 1620 | + | |
| 1621 | + return vm_bo; | |
| 1622 | +} | |
| 1623 | +EXPORT_SYMBOL_GPL(drm_gpuvm_bo_obtain); | |
| 1624 | + | |
| 1625 | +/** | |
| 1626 | + * drm_gpuvm_bo_obtain_prealloc() - obtains and instance of the &drm_gpuvm_bo | |
| 1627 | + * for the given &drm_gpuvm and &drm_gem_object | |
| 1628 | + * @__vm_bo: A pre-allocated struct drm_gpuvm_bo. | |
| 1629 | + * | |
| 1630 | + * Find the &drm_gpuvm_bo representing the combination of the given | |
| 1631 | + * &drm_gpuvm and &drm_gem_object. If found, increases the reference | |
| 1632 | + * count of the found &drm_gpuvm_bo accordingly, while the @__vm_bo reference | |
| 1633 | + * count is decreased. If not found @__vm_bo is returned without further | |
| 1634 | + * increase of the reference count. | |
| 1635 | + * | |
| 1636 | + * A new &drm_gpuvm_bo is added to the GEMs gpuva list. | |
| 1637 | + * | |
| 1638 | + * Returns: a pointer to the found &drm_gpuvm_bo or @__vm_bo if no existing | |
| 1639 | + * &drm_gpuvm_bo was found | |
| 1640 | + */ | |
| 1641 | +struct drm_gpuvm_bo * | |
| 1642 | +drm_gpuvm_bo_obtain_prealloc(struct drm_gpuvm_bo *__vm_bo) | |
| 1643 | +{ | |
| 1644 | + struct drm_gpuvm *gpuvm = __vm_bo->vm; | |
| 1645 | + struct drm_gem_object *obj = __vm_bo->obj; | |
| 1646 | + struct drm_gpuvm_bo *vm_bo; | |
| 1647 | + | |
| 1648 | + vm_bo = drm_gpuvm_bo_find(gpuvm, obj); | |
| 1649 | + if (vm_bo) { | |
| 1650 | + drm_gpuvm_bo_put(__vm_bo); | |
| 1651 | + return vm_bo; | |
| 1652 | + } | |
| 1653 | + | |
| 1654 | + drm_gem_gpuva_assert_lock_held(obj); | |
| 1655 | + list_add_tail(&__vm_bo->list.entry.gem, &obj->gpuva.list); | |
| 1656 | + | |
| 1657 | + return __vm_bo; | |
| 1658 | +} | |
| 1659 | +EXPORT_SYMBOL_GPL(drm_gpuvm_bo_obtain_prealloc); | |
| 1660 | + | |
| 1661 | +/** | |
| 1662 | + * drm_gpuvm_bo_extobj_add() - adds the &drm_gpuvm_bo to its &drm_gpuvm's | |
| 1663 | + * extobj list | |
| 1664 | + * @vm_bo: The &drm_gpuvm_bo to add to its &drm_gpuvm's the extobj list. | |
| 1665 | + * | |
| 1666 | + * Adds the given @vm_bo to its &drm_gpuvm's extobj list if not on the list | |
| 1667 | + * already and if the corresponding &drm_gem_object is an external object, | |
| 1668 | + * actually. | |
| 1669 | + */ | |
| 1670 | +void | |
| 1671 | +drm_gpuvm_bo_extobj_add(struct drm_gpuvm_bo *vm_bo) | |
| 1672 | +{ | |
| 1673 | + struct drm_gpuvm *gpuvm = vm_bo->vm; | |
| 1674 | + bool lock = !drm_gpuvm_resv_protected(gpuvm); | |
| 1675 | + | |
| 1676 | + if (!lock) | |
| 1677 | + drm_gpuvm_resv_assert_held(gpuvm); | |
| 1678 | + | |
| 1679 | + if (drm_gpuvm_is_extobj(gpuvm, vm_bo->obj)) | |
| 1680 | + drm_gpuvm_bo_list_add(vm_bo, extobj, lock); | |
| 1681 | +} | |
| 1682 | +EXPORT_SYMBOL_GPL(drm_gpuvm_bo_extobj_add); | |
| 1683 | + | |
| 1684 | +/** | |
| 1685 | + * drm_gpuvm_bo_evict() - add / remove a &drm_gpuvm_bo to / from the &drm_gpuvms | |
| 1686 | + * evicted list | |
| 1687 | + * @vm_bo: the &drm_gpuvm_bo to add or remove | |
| 1688 | + * @evict: indicates whether the object is evicted | |
| 1689 | + * | |
| 1690 | + * Adds a &drm_gpuvm_bo to or removes it from the &drm_gpuvms evicted list. | |
| 1691 | + */ | |
| 1692 | +void | |
| 1693 | +drm_gpuvm_bo_evict(struct drm_gpuvm_bo *vm_bo, bool evict) | |
| 1694 | +{ | |
| 1695 | + struct drm_gpuvm *gpuvm = vm_bo->vm; | |
| 1696 | + struct drm_gem_object *obj = vm_bo->obj; | |
| 1697 | + bool lock = !drm_gpuvm_resv_protected(gpuvm); | |
| 1698 | + | |
| 1699 | + dma_resv_assert_held(obj->resv); | |
| 1700 | + vm_bo->evicted = evict; | |
| 1701 | + | |
| 1702 | + /* Can't add external objects to the evicted list directly if not using | |
| 1703 | + * internal spinlocks, since in this case the evicted list is protected | |
| 1704 | + * with the VM's common dma-resv lock. | |
| 1705 | + */ | |
| 1706 | + if (drm_gpuvm_is_extobj(gpuvm, obj) && !lock) | |
| 1707 | + return; | |
| 1708 | + | |
| 1709 | + if (evict) | |
| 1710 | + drm_gpuvm_bo_list_add(vm_bo, evict, lock); | |
| 1711 | + else | |
| 1712 | + drm_gpuvm_bo_list_del_init(vm_bo, evict, lock); | |
| 1713 | +} | |
| 1714 | +EXPORT_SYMBOL_GPL(drm_gpuvm_bo_evict); | |
| 719 | 1715 | |
| 720 | 1716 | static int |
| 721 | 1717 | __drm_gpuva_insert(struct drm_gpuvm *gpuvm, |
| @@ -764,11 +1760,21 @@ drm_gpuva_insert(struct drm_gpuvm *gpuvm, · 42 unmodified lines | ||
| 764 | 1760 | { |
| 765 | 1761 | u64 addr = va->va.addr; |
| 766 | 1762 | u64 range = va->va.range; |
| 1763 | + int ret; | |
| 767 | 1764 | |
| 768 | 1765 | if (unlikely(!drm_gpuvm_range_valid(gpuvm, addr, range))) |
| 769 | 1766 | return -EINVAL; |
| 770 | 1767 | |
| 771 | − return __drm_gpuva_insert(gpuvm, va); | |
| 1768 | + ret = __drm_gpuva_insert(gpuvm, va); | |
| 1769 | + if (likely(!ret)) | |
| 1770 | + /* Take a reference of the GPUVM for the successfully inserted | |
| 1771 | + * drm_gpuva. We can't take the reference in | |
| 1772 | + * __drm_gpuva_insert() itself, since we don't want to increse | |
| 1773 | + * the reference count for the GPUVM's kernel_alloc_node. | |
| 1774 | + */ | |
| 1775 | + drm_gpuvm_get(gpuvm); | |
| 1776 | + | |
| 1777 | + return ret; | |
| 772 | 1778 | } |
| 773 | 1779 | EXPORT_SYMBOL_GPL(drm_gpuva_insert); |
| 774 | 1780 | |
| @@ -795,35 +1801,46 @@ drm_gpuva_remove(struct drm_gpuva *va) · 20 unmodified lines | ||
| 795 | 1801 | struct drm_gpuvm *gpuvm = va->vm; |
| 796 | 1802 | |
| 797 | 1803 | if (unlikely(va == &gpuvm->kernel_alloc_node)) { |
| 798 | − WARN(1, "Can't destroy kernel reserved node.\n"); | |
| 1804 | + drm_WARN(gpuvm->drm, 1, | |
| 1805 | + "Can't destroy kernel reserved node.\n"); | |
| 799 | 1806 | return; |
| 800 | 1807 | } |
| 801 | 1808 | |
| 802 | 1809 | __drm_gpuva_remove(va); |
| 1810 | + drm_gpuvm_put(va->vm); | |
| 803 | 1811 | } |
| 804 | 1812 | EXPORT_SYMBOL_GPL(drm_gpuva_remove); |
| 805 | 1813 | |
| 806 | 1814 | /** |
| 807 | 1815 | * drm_gpuva_link() - link a &drm_gpuva |
| 808 | 1816 | * @va: the &drm_gpuva to link |
| 1817 | + * @vm_bo: the &drm_gpuvm_bo to add the &drm_gpuva to | |
| 809 | 1818 | * |
| 810 | − * This adds the given &va to the GPU VA list of the &drm_gem_object it is | |
| 811 | − * associated with. | |
| 1819 | + * This adds the given &va to the GPU VA list of the &drm_gpuvm_bo and the | |
| 1820 | + * &drm_gpuvm_bo to the &drm_gem_object it is associated with. | |
| 1821 | + * | |
| 1822 | + * For every &drm_gpuva entry added to the &drm_gpuvm_bo an additional | |
| 1823 | + * reference of the latter is taken. | |
| 812 | 1824 | * |
| 813 | 1825 | * This function expects the caller to protect the GEM's GPUVA list against |
| 814 | − * concurrent access using the GEMs dma_resv lock. | |
| 1826 | + * concurrent access using either the GEMs dma_resv lock or a driver specific | |
| 1827 | + * lock set through drm_gem_gpuva_set_lock(). | |
| 815 | 1828 | */ |
| 816 | 1829 | void |
| 817 | −drm_gpuva_link(struct drm_gpuva *va) | |
| 1830 | +drm_gpuva_link(struct drm_gpuva *va, struct drm_gpuvm_bo *vm_bo) | |
| 818 | 1831 | { |
| 819 | 1832 | struct drm_gem_object *obj = va->gem.obj; |
| 1833 | + struct drm_gpuvm *gpuvm = va->vm; | |
| 820 | 1834 | |
| 821 | 1835 | if (unlikely(!obj)) |
| 822 | 1836 | return; |
| 823 | 1837 | |
| 824 | − drm_gem_gpuva_assert_lock_held(obj); | |
| 1838 | + drm_WARN_ON(gpuvm->drm, obj != vm_bo->obj); | |
| 825 | 1839 | |
| 826 | − list_add_tail(&va->gem.entry, &obj->gpuva.list); | |
| 1840 | + va->vm_bo = drm_gpuvm_bo_get(vm_bo); | |
| 1841 | + | |
| 1842 | + drm_gem_gpuva_assert_lock_held(obj); | |
| 1843 | + list_add_tail(&va->gem.entry, &vm_bo->list.gpuva); | |
| 827 | 1844 | } |
| 828 | 1845 | EXPORT_SYMBOL_GPL(drm_gpuva_link); |
| 829 | 1846 | |
| @@ -834,20 +1851,31 @@ EXPORT_SYMBOL_GPL(drm_gpuva_link); · 4 unmodified lines | ||
| 834 | 1851 | * This removes the given &va from the GPU VA list of the &drm_gem_object it is |
| 835 | 1852 | * associated with. |
| 836 | 1853 | * |
| 1854 | + * This removes the given &va from the GPU VA list of the &drm_gpuvm_bo and | |
| 1855 | + * the &drm_gpuvm_bo from the &drm_gem_object it is associated with in case | |
| 1856 | + * this call unlinks the last &drm_gpuva from the &drm_gpuvm_bo. | |
| 1857 | + * | |
| 1858 | + * For every &drm_gpuva entry removed from the &drm_gpuvm_bo a reference of | |
| 1859 | + * the latter is dropped. | |
| 1860 | + * | |
| 837 | 1861 | * This function expects the caller to protect the GEM's GPUVA list against |
| 838 | − * concurrent access using the GEMs dma_resv lock. | |
| 1862 | + * concurrent access using either the GEMs dma_resv lock or a driver specific | |
| 1863 | + * lock set through drm_gem_gpuva_set_lock(). | |
| 839 | 1864 | */ |
| 840 | 1865 | void |
| 841 | 1866 | drm_gpuva_unlink(struct drm_gpuva *va) |
| 842 | 1867 | { |
| 843 | 1868 | struct drm_gem_object *obj = va->gem.obj; |
| 1869 | + struct drm_gpuvm_bo *vm_bo = va->vm_bo; | |
| 844 | 1870 | |
| 845 | 1871 | if (unlikely(!obj)) |
| 846 | 1872 | return; |
| 847 | 1873 | |
| 848 | 1874 | drm_gem_gpuva_assert_lock_held(obj); |
| 849 | − | |
| 850 | 1875 | list_del_init(&va->gem.entry); |
| 1876 | + | |
| 1877 | + va->vm_bo = NULL; | |
| 1878 | + drm_gpuvm_bo_put(vm_bo); | |
| 851 | 1879 | } |
| 852 | 1880 | EXPORT_SYMBOL_GPL(drm_gpuva_unlink); |
| 853 | 1881 | |
| @@ -992,10 +2020,10 @@ drm_gpuva_remap(struct drm_gpuva *prev, · 138 unmodified lines | ||
| 992 | 2020 | struct drm_gpuva *next, |
| 993 | 2021 | struct drm_gpuva_op_remap *op) |
| 994 | 2022 | { |
| 995 | − struct drm_gpuva *curr = op->unmap->va; | |
| 996 | − struct drm_gpuvm *gpuvm = curr->vm; | |
| 2023 | + struct drm_gpuva *va = op->unmap->va; | |
| 2024 | + struct drm_gpuvm *gpuvm = va->vm; | |
| 997 | 2025 | |
| 998 | − drm_gpuva_remove(curr); | |
| 2026 | + drm_gpuva_remove(va); | |
| 999 | 2027 | |
| 1000 | 2028 | if (op->prev) { |
| 1001 | 2029 | drm_gpuva_init_from_op(prev, op->prev); |
| @@ -1637,9 +2665,8 @@ err_free_ops: · 635 unmodified lines | ||
| 1637 | 2665 | EXPORT_SYMBOL_GPL(drm_gpuvm_prefetch_ops_create); |
| 1638 | 2666 | |
| 1639 | 2667 | /** |
| 1640 | − * drm_gpuvm_gem_unmap_ops_create() - creates the &drm_gpuva_ops to unmap a GEM | |
| 1641 | − * @gpuvm: the &drm_gpuvm representing the GPU VA space | |
| 1642 | − * @obj: the &drm_gem_object to unmap | |
| 2668 | + * drm_gpuvm_bo_unmap_ops_create() - creates the &drm_gpuva_ops to unmap a GEM | |
| 2669 | + * @vm_bo: the &drm_gpuvm_bo abstraction | |
| 1643 | 2670 | * |
| 1644 | 2671 | * This function creates a list of operations to perform unmapping for every |
| 1645 | 2672 | * GPUVA attached to a GEM. |
| @@ -1656,15 +2683,14 @@ EXPORT_SYMBOL_GPL(drm_gpuvm_prefetch_ops_create); · 10 unmodified lines | ||
| 1656 | 2683 | * Returns: a pointer to the &drm_gpuva_ops on success, an ERR_PTR on failure |
| 1657 | 2684 | */ |
| 1658 | 2685 | struct drm_gpuva_ops * |
| 1659 | −drm_gpuvm_gem_unmap_ops_create(struct drm_gpuvm *gpuvm, | |
| 1660 | − struct drm_gem_object *obj) | |
| 2686 | +drm_gpuvm_bo_unmap_ops_create(struct drm_gpuvm_bo *vm_bo) | |
| 1661 | 2687 | { |
| 1662 | 2688 | struct drm_gpuva_ops *ops; |
| 1663 | 2689 | struct drm_gpuva_op *op; |
| 1664 | 2690 | struct drm_gpuva *va; |
| 1665 | 2691 | int ret; |
| 1666 | 2692 | |
| 1667 | − drm_gem_gpuva_assert_lock_held(obj); | |
| 2693 | + drm_gem_gpuva_assert_lock_held(vm_bo->obj); | |
| 1668 | 2694 | |
| 1669 | 2695 | ops = kzalloc(sizeof(*ops), GFP_KERNEL); |
| 1670 | 2696 | if (!ops) |
| @@ -1672,8 +2698,8 @@ drm_gpuvm_gem_unmap_ops_create(struct drm_gpuvm *gpuvm, · 1 unmodified line | ||
| 1672 | 2698 | |
| 1673 | 2699 | INIT_LIST_HEAD(&ops->list); |
| 1674 | 2700 | |
| 1675 | − drm_gem_for_each_gpuva(va, obj) { | |
| 1676 | − op = gpuva_op_alloc(gpuvm); | |
| 2701 | + drm_gpuvm_bo_for_each_va(va, vm_bo) { | |
| 2702 | + op = gpuva_op_alloc(vm_bo->vm); | |
| 1677 | 2703 | if (!op) { |
| 1678 | 2704 | ret = -ENOMEM; |
| 1679 | 2705 | goto err_free_ops; |
| @@ -1687,10 +2713,10 @@ drm_gpuvm_gem_unmap_ops_create(struct drm_gpuvm *gpuvm, · 7 unmodified lines | ||
| 1687 | 2713 | return ops; |
| 1688 | 2714 | |
| 1689 | 2715 | err_free_ops: |
| 1690 | − drm_gpuva_ops_free(gpuvm, ops); | |
| 2716 | + drm_gpuva_ops_free(vm_bo->vm, ops); | |
| 1691 | 2717 | return ERR_PTR(ret); |
| 1692 | 2718 | } |
| 1693 | −EXPORT_SYMBOL_GPL(drm_gpuvm_gem_unmap_ops_create); | |
| 2719 | +EXPORT_SYMBOL_GPL(drm_gpuvm_bo_unmap_ops_create); | |
| 1694 | 2720 | |
| 1695 | 2721 | /** |
| 1696 | 2722 | * drm_gpuva_ops_free() - free the given &drm_gpuva_ops |
| @@ -1,203 +0,0 @@ | ||
| 1 | −/************************************************************************** | |
| 2 | − * | |
| 3 | − * Copyright 2006 Tungsten Graphics, Inc., Bismarck, ND. USA. | |
| 4 | − * All Rights Reserved. | |
| 5 | − * | |
| 6 | − * Permission is hereby granted, free of charge, to any person obtaining a | |
| 7 | − * copy of this software and associated documentation files (the | |
| 8 | − * "Software"), to deal in the Software without restriction, including | |
| 9 | − * without limitation the rights to use, copy, modify, merge, publish, | |
| 10 | − * distribute, sub license, and/or sell copies of the Software, and to | |
| 11 | − * permit persons to whom the Software is furnished to do so, subject to | |
| 12 | − * the following conditions: | |
| 13 | − * | |
| 14 | − * The above copyright notice and this permission notice (including the | |
| 15 | − * next paragraph) shall be included in all copies or substantial portions | |
| 16 | − * of the Software. | |
| 17 | − * | |
| 18 | − * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 19 | − * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 20 | − * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL | |
| 21 | − * THE COPYRIGHT HOLDERS, AUTHORS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, | |
| 22 | − * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR | |
| 23 | − * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE | |
| 24 | − * USE OR OTHER DEALINGS IN THE SOFTWARE. | |
| 25 | − * | |
| 26 | − * | |
| 27 | − **************************************************************************/ | |
| 28 | −/* | |
| 29 | − * Simple open hash tab implementation. | |
| 30 | − * | |
| 31 | − * Authors: | |
| 32 | − * Thomas Hellström <thomas-at-tungstengraphics-dot-com> | |
| 33 | − */ | |
| 34 | − | |
| 35 | −#include <linux/hash.h> | |
| 36 | −#include <linux/mm.h> | |
| 37 | −#include <linux/rculist.h> | |
| 38 | −#include <linux/slab.h> | |
| 39 | −#include <linux/vmalloc.h> | |
| 40 | − | |
| 41 | −#include <drm/drm_print.h> | |
| 42 | − | |
| 43 | −#include "drm_legacy.h" | |
| 44 | − | |
| 45 | −int drm_ht_create(struct drm_open_hash *ht, unsigned int order) | |
| 46 | −{ | |
| 47 | − unsigned int size = 1 << order; | |
| 48 | − | |
| 49 | − ht->order = order; | |
| 50 | − ht->table = NULL; | |
| 51 | − if (size <= PAGE_SIZE / sizeof(*ht->table)) | |
| 52 | − ht->table = kcalloc(size, sizeof(*ht->table), GFP_KERNEL); | |
| 53 | − else | |
| 54 | − ht->table = vzalloc(array_size(size, sizeof(*ht->table))); | |
| 55 | − if (!ht->table) { | |
| 56 | − DRM_ERROR("Out of memory for hash table\n"); | |
| 57 | − return -ENOMEM; | |
| 58 | − } | |
| 59 | − return 0; | |
| 60 | −} | |
| 61 | − | |
| 62 | −void drm_ht_verbose_list(struct drm_open_hash *ht, unsigned long key) | |
| 63 | −{ | |
| 64 | − struct drm_hash_item *entry; | |
| 65 | − struct hlist_head *h_list; | |
| 66 | − unsigned int hashed_key; | |
| 67 | − int count = 0; | |
| 68 | − | |
| 69 | − hashed_key = hash_long(key, ht->order); | |
| 70 | − DRM_DEBUG("Key is 0x%08lx, Hashed key is 0x%08x\n", key, hashed_key); | |
| 71 | − h_list = &ht->table[hashed_key]; | |
| 72 | − hlist_for_each_entry(entry, h_list, head) | |
| 73 | − DRM_DEBUG("count %d, key: 0x%08lx\n", count++, entry->key); | |
| 74 | −} | |
| 75 | − | |
| 76 | −static struct hlist_node *drm_ht_find_key(struct drm_open_hash *ht, | |
| 77 | − unsigned long key) | |
| 78 | −{ | |
| 79 | − struct drm_hash_item *entry; | |
| 80 | − struct hlist_head *h_list; | |
| 81 | − unsigned int hashed_key; | |
| 82 | − | |
| 83 | − hashed_key = hash_long(key, ht->order); | |
| 84 | − h_list = &ht->table[hashed_key]; | |
| 85 | − hlist_for_each_entry(entry, h_list, head) { | |
| 86 | − if (entry->key == key) | |
| 87 | − return &entry->head; | |
| 88 | − if (entry->key > key) | |
| 89 | − break; | |
| 90 | − } | |
| 91 | − return NULL; | |
| 92 | −} | |
| 93 | − | |
| 94 | −static struct hlist_node *drm_ht_find_key_rcu(struct drm_open_hash *ht, | |
| 95 | − unsigned long key) | |
| 96 | −{ | |
| 97 | − struct drm_hash_item *entry; | |
| 98 | − struct hlist_head *h_list; | |
| 99 | − unsigned int hashed_key; | |
| 100 | − | |
| 101 | − hashed_key = hash_long(key, ht->order); | |
| 102 | − h_list = &ht->table[hashed_key]; | |
| 103 | − hlist_for_each_entry_rcu(entry, h_list, head) { | |
| 104 | − if (entry->key == key) | |
| 105 | − return &entry->head; | |
| 106 | − if (entry->key > key) | |
| 107 | − break; | |
| 108 | − } | |
| 109 | − return NULL; | |
| 110 | −} | |
| 111 | − | |
| 112 | −int drm_ht_insert_item(struct drm_open_hash *ht, struct drm_hash_item *item) | |
| 113 | −{ | |
| 114 | − struct drm_hash_item *entry; | |
| 115 | − struct hlist_head *h_list; | |
| 116 | − struct hlist_node *parent; | |
| 117 | − unsigned int hashed_key; | |
| 118 | − unsigned long key = item->key; | |
| 119 | − | |
84 more lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 656 lines aren’t shown here.
This diff is large: its 165 lines aren’t shown here.
This diff is large: its 205 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 291 lines aren’t shown here.
This diff is large: its 106 lines aren’t shown here.
This diff is large: its 374 lines aren’t shown here.
This diff is large: its 139 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 220 lines aren’t shown here.
This diff is large: its 185 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 221 lines aren’t shown here.
This diff is large: its 152 lines aren’t shown here.
This diff is large: its 136 lines aren’t shown here.
This diff is large: its 666 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 299 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 147 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 126 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 229 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 361 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 960 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 147 lines aren’t shown here.
This diff is large: its 177 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 218 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 90 lines aren’t shown here.
This diff is large: its 859 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 276 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 957 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 448 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 475 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 313 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 255 lines aren’t shown here.
This diff is large: its 83 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 511 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 728 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 129 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 646 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 268 lines aren’t shown here.
This diff is large: its 111 lines aren’t shown here.
This diff is large: its 465 lines aren’t shown here.
This diff is large: its 206 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 659 lines aren’t shown here.
This diff is large: its 722 lines aren’t shown here.
This diff is large: its 256 lines aren’t shown here.
This diff is large: its 187 lines aren’t shown here.
This diff is large: its 1502 lines aren’t shown here.
This diff is large: its 130 lines aren’t shown here.
This diff is large: its 626 lines aren’t shown here.
This diff is large: its 196 lines aren’t shown here.
This diff is large: its 1490 lines aren’t shown here.
This diff is large: its 510 lines aren’t shown here.
This diff is large: its 136 lines aren’t shown here.
This diff is large: its 556 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 253 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 307 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 472 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 415 lines aren’t shown here.
This diff is large: its 171 lines aren’t shown here.
This diff is large: its 550 lines aren’t shown here.
This diff is large: its 167 lines aren’t shown here.
This diff is large: its 787 lines aren’t shown here.
This diff is large: its 162 lines aren’t shown here.
This diff is large: its 2641 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 148 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 434 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 1433 lines aren’t shown here.
This diff is large: its 170 lines aren’t shown here.
This diff is large: its 6194 lines aren’t shown here.
This diff is large: its 160 lines aren’t shown here.
This diff is large: its 180 lines aren’t shown here.
This diff is large: its 2189 lines aren’t shown here.
This diff is large: its 494 lines aren’t shown here.
This diff is large: its 374 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 1649 lines aren’t shown here.
This diff is large: its 259 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 357 lines aren’t shown here.
This diff is large: its 336 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 137 lines aren’t shown here.
This diff is large: its 286 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 352 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 290 lines aren’t shown here.
This diff is large: its 85 lines aren’t shown here.
This diff is large: its 1093 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 238 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 132 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 315 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 680 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 241 lines aren’t shown here.
This diff is large: its 107 lines aren’t shown here.
This diff is large: its 280 lines aren’t shown here.
This diff is large: its 258 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 188 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 801 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 508 lines aren’t shown here.
This diff is large: its 206 lines aren’t shown here.
This diff is large: its 265 lines aren’t shown here.
This diff is large: its 384 lines aren’t shown here.
This diff is large: its 2107 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 210 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 245 lines aren’t shown here.
This diff is large: its 115 lines aren’t shown here.
This diff is large: its 250 lines aren’t shown here.
This diff is large: its 816 lines aren’t shown here.
This diff is large: its 172 lines aren’t shown here.
This diff is large: its 235 lines aren’t shown here.
This diff is large: its 555 lines aren’t shown here.
This diff is large: its 1321 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 117 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 205 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 215 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 123 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 130 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 244 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 124 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 198 lines aren’t shown here.
This diff is large: its 132 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 135 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 103 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 115 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 4 lines aren’t shown here.
This diff is large: its 1326 lines aren’t shown here.
This diff is large: its 762 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 154 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 163 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 129 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 151 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 103 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 196 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 1254 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 153 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 130 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 251 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 102 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 167 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 133 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 214 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 202 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 357 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 180 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 174 lines aren’t shown here.
This diff is large: its 165 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 637 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 332 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 150 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 1296 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 294 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 859 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 214 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 4073 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 498 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 220 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 243 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 132 lines aren’t shown here.
This diff is large: its 148 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.