Picking up v5.0 + missed misc-fixes from last release
Unified · Split
| @@ -842,10 +842,9 @@ D: ax25-utils maintainer. · 841 unmodified lines | ||
| 842 | 842 | |
| 843 | 843 | N: Helge Deller |
| 844 | 844 | E: deller@gmx.de |
| 845 | −E: hdeller@redhat.de | |
| 846 | −D: PA-RISC Linux hacker, LASI-, ASP-, WAX-, LCD/LED-driver | |
| 847 | −S: Schimmelsrain 1 | |
| 848 | −S: D-69231 Rauenberg | |
| 845 | +W: http://www.parisc-linux.org/ | |
| 846 | +D: PA-RISC Linux architecture maintainer | |
| 847 | +D: LASI-, ASP-, WAX-, LCD/LED-driver | |
| 849 | 848 | S: Germany |
| 850 | 849 | |
| 851 | 850 | N: Jean Delvare |
| @@ -1361,7 +1360,7 @@ S: Stellenbosch, Western Cape · 509 unmodified lines | ||
| 1361 | 1360 | S: South Africa |
| 1362 | 1361 | |
| 1363 | 1362 | N: Grant Grundler |
| 1364 | −E: grundler@parisc-linux.org | |
| 1363 | +E: grantgrundler@gmail.com | |
| 1365 | 1364 | W: http://obmouse.sourceforge.net/ |
| 1366 | 1365 | W: http://www.parisc-linux.org/ |
| 1367 | 1366 | D: obmouse - rewrote Olivier Florent's Omnibook 600 "pop-up" mouse driver |
| @@ -2492,7 +2491,7 @@ S: Syracuse, New York 13206 · 1124 unmodified lines | ||
| 2492 | 2491 | S: USA |
| 2493 | 2492 | |
| 2494 | 2493 | N: Kyle McMartin |
| 2495 | −E: kyle@parisc-linux.org | |
| 2494 | +E: kyle@mcmartin.ca | |
| 2496 | 2495 | D: Linux/PARISC hacker |
| 2497 | 2496 | D: AD1889 sound driver |
| 2498 | 2497 | S: Ottawa, Canada |
| @@ -3780,14 +3779,13 @@ S: 21513 Conradia Ct · 1281 unmodified lines | ||
| 3780 | 3779 | S: Cupertino, CA 95014 |
| 3781 | 3780 | S: USA |
| 3782 | 3781 | |
| 3783 | −N: Thibaut Varene | |
| 3784 | −E: T-Bone@parisc-linux.org | |
| 3785 | −W: http://www.parisc-linux.org/~varenet/ | |
| 3786 | −P: 1024D/B7D2F063 E67C 0D43 A75E 12A5 BB1C FA2F 1E32 C3DA B7D2 F063 | |
| 3782 | +N: Thibaut Varène | |
| 3783 | +E: hacks+kernel@slashdirt.org | |
| 3784 | +W: http://hacks.slashdirt.org/ | |
| 3787 | 3785 | D: PA-RISC port minion, PDC and GSCPS2 drivers, debuglocks and other bits |
| 3788 | 3786 | D: Some ARM at91rm9200 bits, S1D13XXX FB driver, random patches here and there |
| 3789 | 3787 | D: AD1889 sound driver |
| 3790 | −S: Paris, France | |
| 3788 | +S: France | |
| 3791 | 3789 | |
| 3792 | 3790 | N: Heikki Vatiainen |
| 3793 | 3791 | E: hessu@cs.tut.fi |
| @@ -1,9 +1,9 @@ | ||
| 1 | 1 | .. _readme: |
| 2 | 2 | |
| 3 | −Linux kernel release 4.x <http://kernel.org/> | |
| 3 | +Linux kernel release 5.x <http://kernel.org/> | |
| 4 | 4 | ============================================= |
| 5 | 5 | |
| 6 | −These are the release notes for Linux version 4. Read them carefully, | |
| 6 | +These are the release notes for Linux version 5. Read them carefully, | |
| 7 | 7 | as they tell you what this is all about, explain how to install the |
| 8 | 8 | kernel, and what to do if something goes wrong. |
| 9 | 9 | |
| @@ -63,7 +63,7 @@ Installing the kernel source · 53 unmodified lines | ||
| 63 | 63 | directory where you have permissions (e.g. your home directory) and |
| 64 | 64 | unpack it:: |
| 65 | 65 | |
| 66 | − xz -cd linux-4.X.tar.xz | tar xvf - | |
| 66 | + xz -cd linux-5.x.tar.xz | tar xvf - | |
| 67 | 67 | |
| 68 | 68 | Replace "X" with the version number of the latest kernel. |
| 69 | 69 | |
| @@ -72,26 +72,26 @@ Installing the kernel source · 2 unmodified lines | ||
| 72 | 72 | files. They should match the library, and not get messed up by |
| 73 | 73 | whatever the kernel-du-jour happens to be. |
| 74 | 74 | |
| 75 | − - You can also upgrade between 4.x releases by patching. Patches are | |
| 75 | + - You can also upgrade between 5.x releases by patching. Patches are | |
| 76 | 76 | distributed in the xz format. To install by patching, get all the |
| 77 | 77 | newer patch files, enter the top level directory of the kernel source |
| 78 | − (linux-4.X) and execute:: | |
| 78 | + (linux-5.x) and execute:: | |
| 79 | 79 | |
| 80 | − xz -cd ../patch-4.x.xz | patch -p1 | |
| 80 | + xz -cd ../patch-5.x.xz | patch -p1 | |
| 81 | 81 | |
| 82 | − Replace "x" for all versions bigger than the version "X" of your current | |
| 82 | + Replace "x" for all versions bigger than the version "x" of your current | |
| 83 | 83 | source tree, **in_order**, and you should be ok. You may want to remove |
| 84 | 84 | the backup files (some-file-name~ or some-file-name.orig), and make sure |
| 85 | 85 | that there are no failed patches (some-file-name# or some-file-name.rej). |
| 86 | 86 | If there are, either you or I have made a mistake. |
| 87 | 87 | |
| 88 | − Unlike patches for the 4.x kernels, patches for the 4.x.y kernels | |
| 88 | + Unlike patches for the 5.x kernels, patches for the 5.x.y kernels | |
| 89 | 89 | (also known as the -stable kernels) are not incremental but instead apply |
| 90 | − directly to the base 4.x kernel. For example, if your base kernel is 4.0 | |
| 91 | − and you want to apply the 4.0.3 patch, you must not first apply the 4.0.1 | |
| 92 | − and 4.0.2 patches. Similarly, if you are running kernel version 4.0.2 and | |
| 93 | − want to jump to 4.0.3, you must first reverse the 4.0.2 patch (that is, | |
| 94 | − patch -R) **before** applying the 4.0.3 patch. You can read more on this in | |
| 90 | + directly to the base 5.x kernel. For example, if your base kernel is 5.0 | |
| 91 | + and you want to apply the 5.0.3 patch, you must not first apply the 5.0.1 | |
| 92 | + and 5.0.2 patches. Similarly, if you are running kernel version 5.0.2 and | |
| 93 | + want to jump to 5.0.3, you must first reverse the 5.0.2 patch (that is, | |
| 94 | + patch -R) **before** applying the 5.0.3 patch. You can read more on this in | |
| 95 | 95 | :ref:`Documentation/process/applying-patches.rst <applying_patches>`. |
| 96 | 96 | |
| 97 | 97 | Alternatively, the script patch-kernel can be used to automate this |
| @@ -114,7 +114,7 @@ Installing the kernel source · 16 unmodified lines | ||
| 114 | 114 | Software requirements |
| 115 | 115 | --------------------- |
| 116 | 116 | |
| 117 | − Compiling and running the 4.x kernels requires up-to-date | |
| 117 | + Compiling and running the 5.x kernels requires up-to-date | |
| 118 | 118 | versions of various software packages. Consult |
| 119 | 119 | :ref:`Documentation/process/changes.rst <changes>` for the minimum version numbers |
| 120 | 120 | required and how to get updates for these packages. Beware that using |
| @@ -132,12 +132,12 @@ Build directory for the kernel · 11 unmodified lines | ||
| 132 | 132 | place for the output files (including .config). |
| 133 | 133 | Example:: |
| 134 | 134 | |
| 135 | − kernel source code: /usr/src/linux-4.X | |
| 135 | + kernel source code: /usr/src/linux-5.x | |
| 136 | 136 | build directory: /home/name/build/kernel |
| 137 | 137 | |
| 138 | 138 | To configure and build the kernel, use:: |
| 139 | 139 | |
| 140 | − cd /usr/src/linux-4.X | |
| 140 | + cd /usr/src/linux-5.x | |
| 141 | 141 | make O=/home/name/build/kernel menuconfig |
| 142 | 142 | make O=/home/name/build/kernel |
| 143 | 143 | sudo make O=/home/name/build/kernel modules_install install |
| @@ -0,0 +1,59 @@ | ||
| 1 | +Qualcomm adreno/snapdragon GMU (Graphics management unit) | |
| 2 | + | |
| 3 | +The GMU is a programmable power controller for the GPU. the CPU controls the | |
| 4 | +GMU which in turn handles power controls for the GPU. | |
| 5 | + | |
| 6 | +Required properties: | |
| 7 | +- compatible: "qcom,adreno-gmu-XYZ.W", "qcom,adreno-gmu" | |
| 8 | + for example: "qcom,adreno-gmu-630.2", "qcom,adreno-gmu" | |
| 9 | + Note that you need to list the less specific "qcom,adreno-gmu" | |
| 10 | + for generic matches and the more specific identifier to identify | |
| 11 | + the specific device. | |
| 12 | +- reg: Physical base address and length of the GMU registers. | |
| 13 | +- reg-names: Matching names for the register regions | |
| 14 | + * "gmu" | |
| 15 | + * "gmu_pdc" | |
| 16 | + * "gmu_pdc_seg" | |
| 17 | +- interrupts: The interrupt signals from the GMU. | |
| 18 | +- interrupt-names: Matching names for the interrupts | |
| 19 | + * "hfi" | |
| 20 | + * "gmu" | |
| 21 | +- clocks: phandles to the device clocks | |
| 22 | +- clock-names: Matching names for the clocks | |
| 23 | + * "gmu" | |
| 24 | + * "cxo" | |
| 25 | + * "axi" | |
| 26 | + * "mnoc" | |
| 27 | +- power-domains: should be <&clock_gpucc GPU_CX_GDSC> | |
| 28 | +- iommus: phandle to the adreno iommu | |
| 29 | +- operating-points-v2: phandle to the OPP operating points | |
| 30 | + | |
| 31 | +Example: | |
| 32 | + | |
| 33 | +/ { | |
| 34 | + ... | |
| 35 | + | |
| 36 | + gmu: gmu@506a000 { | |
| 37 | + compatible="qcom,adreno-gmu-630.2", "qcom,adreno-gmu"; | |
| 38 | + | |
| 39 | + reg = <0x506a000 0x30000>, | |
| 40 | + <0xb280000 0x10000>, | |
| 41 | + <0xb480000 0x10000>; | |
| 42 | + reg-names = "gmu", "gmu_pdc", "gmu_pdc_seq"; | |
| 43 | + | |
| 44 | + interrupts = <GIC_SPI 304 IRQ_TYPE_LEVEL_HIGH>, | |
| 45 | + <GIC_SPI 305 IRQ_TYPE_LEVEL_HIGH>; | |
| 46 | + interrupt-names = "hfi", "gmu"; | |
| 47 | + | |
| 48 | + clocks = <&gpucc GPU_CC_CX_GMU_CLK>, | |
| 49 | + <&gpucc GPU_CC_CXO_CLK>, | |
| 50 | + <&gcc GCC_DDRSS_GPU_AXI_CLK>, | |
| 51 | + <&gcc GCC_GPU_MEMNOC_GFX_CLK>; | |
| 52 | + clock-names = "gmu", "cxo", "axi", "memnoc"; | |
| 53 | + | |
| 54 | + power-domains = <&gpucc GPU_CX_GDSC>; | |
| 55 | + iommus = <&adreno_smmu 5>; | |
| 56 | + | |
| 57 | + operating-points-v2 = <&gmu_opp_table>; | |
| 58 | + }; | |
| 59 | +}; | |
| @@ -10,14 +10,23 @@ Required properties: · 9 unmodified lines | ||
| 10 | 10 | If "amd,imageon" is used, there should be no top level msm device. |
| 11 | 11 | - reg: Physical base address and length of the controller's registers. |
| 12 | 12 | - interrupts: The interrupt signal from the gpu. |
| 13 | −- clocks: device clocks | |
| 13 | +- clocks: device clocks (if applicable) | |
| 14 | 14 | See ../clocks/clock-bindings.txt for details. |
| 15 | −- clock-names: the following clocks are required: | |
| 15 | +- clock-names: the following clocks are required by a3xx, a4xx and a5xx | |
| 16 | + cores: | |
| 16 | 17 | * "core" |
| 17 | 18 | * "iface" |
| 18 | 19 | * "mem_iface" |
| 20 | + For GMU attached devices the GPU clocks are not used and are not required. The | |
| 21 | + following devices should not list clocks: | |
| 22 | + - qcom,adreno-630.2 | |
| 23 | +- iommus: optional phandle to an adreno iommu instance | |
| 24 | +- operating-points-v2: optional phandle to the OPP operating points | |
| 25 | +- qcom,gmu: For GMU attached devices a phandle to the GMU device that will | |
| 26 | + control the power for the GPU. Applicable targets: | |
| 27 | + - qcom,adreno-630.2 | |
| 19 | 28 | |
| 20 | −Example: | |
| 29 | +Example 3xx/4xx/a5xx: | |
| 21 | 30 | |
| 22 | 31 | / { |
| 23 | 32 | ... |
| @@ -37,3 +46,30 @@ Example: · 13 unmodified lines | ||
| 37 | 46 | <&mmcc MMSS_IMEM_AHB_CLK>; |
| 38 | 47 | }; |
| 39 | 48 | }; |
| 49 | + | |
| 50 | +Example a6xx (with GMU): | |
| 51 | + | |
| 52 | +/ { | |
| 53 | + ... | |
| 54 | + | |
| 55 | + gpu@5000000 { | |
| 56 | + compatible = "qcom,adreno-630.2", "qcom,adreno"; | |
| 57 | + #stream-id-cells = <16>; | |
| 58 | + | |
| 59 | + reg = <0x5000000 0x40000>, <0x509e000 0x10>; | |
| 60 | + reg-names = "kgsl_3d0_reg_memory", "cx_mem"; | |
| 61 | + | |
| 62 | + /* | |
| 63 | + * Look ma, no clocks! The GPU clocks and power are | |
| 64 | + * controlled entirely by the GMU | |
| 65 | + */ | |
| 66 | + | |
| 67 | + interrupts = <GIC_SPI 300 IRQ_TYPE_LEVEL_HIGH>; | |
| 68 | + | |
| 69 | + iommus = <&adreno_smmu 0>; | |
| 70 | + | |
| 71 | + operating-points-v2 = <&gpu_opp_table>; | |
| 72 | + | |
| 73 | + qcom,gmu = <&gmu>; | |
| 74 | + }; | |
| 75 | +}; | |
| @@ -533,16 +533,12 @@ Bridge VLAN filtering · 532 unmodified lines | ||
| 533 | 533 | function that the driver has to call for each VLAN the given port is a member |
| 534 | 534 | of. A switchdev object is used to carry the VID and bridge flags. |
| 535 | 535 | |
| 536 | −- port_fdb_prepare: bridge layer function invoked when the bridge prepares the | |
| 537 | − installation of a Forwarding Database entry. If the operation is not | |
| 538 | − supported, this function should return -EOPNOTSUPP to inform the bridge code | |
| 539 | − to fallback to a software implementation. No hardware setup must be done in | |
| 540 | − this function. See port_fdb_add for this and details. | |
| 541 | − | |
| 542 | 536 | - port_fdb_add: bridge layer function invoked when the bridge wants to install a |
| 543 | 537 | Forwarding Database entry, the switch hardware should be programmed with the |
| 544 | 538 | specified address in the specified VLAN Id in the forwarding database |
| 545 | − associated with this VLAN ID | |
| 539 | + associated with this VLAN ID. If the operation is not supported, this | |
| 540 | + function should return -EOPNOTSUPP to inform the bridge code to fallback to | |
| 541 | + a software implementation. | |
| 546 | 542 | |
| 547 | 543 | Note: VLAN ID 0 corresponds to the port private database, which, in the context |
| 548 | 544 | of DSA, would be the its port-based VLAN, used by the associated bridge device. |
| @@ -7,7 +7,7 @@ Intro · 6 unmodified lines | ||
| 7 | 7 | ===== |
| 8 | 8 | |
| 9 | 9 | The MSG_ZEROCOPY flag enables copy avoidance for socket send calls. |
| 10 | −The feature is currently implemented for TCP sockets. | |
| 10 | +The feature is currently implemented for TCP and UDP sockets. | |
| 11 | 11 | |
| 12 | 12 | |
| 13 | 13 | Opportunity and Caveats |
| @@ -92,11 +92,11 @@ device. · 91 unmodified lines | ||
| 92 | 92 | Switch ID |
| 93 | 93 | ^^^^^^^^^ |
| 94 | 94 | |
| 95 | −The switchdev driver must implement the switchdev op switchdev_port_attr_get | |
| 96 | −for SWITCHDEV_ATTR_ID_PORT_PARENT_ID for each port netdev, returning the same | |
| 97 | −physical ID for each port of a switch. The ID must be unique between switches | |
| 98 | −on the same system. The ID does not need to be unique between switches on | |
| 99 | −different systems. | |
| 95 | +The switchdev driver must implement the net_device operation | |
| 96 | +ndo_get_port_parent_id for each port netdev, returning the same physical ID for | |
| 97 | +each port of a switch. The ID must be unique between switches on the same | |
| 98 | +system. The ID does not need to be unique between switches on different | |
| 99 | +systems. | |
| 100 | 100 | |
| 101 | 101 | The switch ID is used to locate ports on a switch and to know if aggregated |
| 102 | 102 | ports belong to the same switch. |
| @@ -216,14 +216,14 @@ You can use the ``interdiff`` program (http://cyberelk.net/tim/patchutils/) to · 215 unmodified lines | ||
| 216 | 216 | generate a patch representing the differences between two patches and then |
| 217 | 217 | apply the result. |
| 218 | 218 | |
| 219 | −This will let you move from something like 4.7.2 to 4.7.3 in a single | |
| 219 | +This will let you move from something like 5.7.2 to 5.7.3 in a single | |
| 220 | 220 | step. The -z flag to interdiff will even let you feed it patches in gzip or |
| 221 | 221 | bzip2 compressed form directly without the use of zcat or bzcat or manual |
| 222 | 222 | decompression. |
| 223 | 223 | |
| 224 | −Here's how you'd go from 4.7.2 to 4.7.3 in a single step:: | |
| 224 | +Here's how you'd go from 5.7.2 to 5.7.3 in a single step:: | |
| 225 | 225 | |
| 226 | − interdiff -z ../patch-4.7.2.gz ../patch-4.7.3.gz | patch -p1 | |
| 226 | + interdiff -z ../patch-5.7.2.gz ../patch-5.7.3.gz | patch -p1 | |
| 227 | 227 | |
| 228 | 228 | Although interdiff may save you a step or two you are generally advised to |
| 229 | 229 | do the additional steps since interdiff can get things wrong in some cases. |
| @@ -245,62 +245,67 @@ The patches are available at http://kernel.org/ · 15 unmodified lines | ||
| 245 | 245 | Most recent patches are linked from the front page, but they also have |
| 246 | 246 | specific homes. |
| 247 | 247 | |
| 248 | −The 4.x.y (-stable) and 4.x patches live at | |
| 248 | +The 5.x.y (-stable) and 5.x patches live at | |
| 249 | 249 | |
| 250 | − https://www.kernel.org/pub/linux/kernel/v4.x/ | |
| 250 | + https://www.kernel.org/pub/linux/kernel/v5.x/ | |
| 251 | 251 | |
| 252 | −The -rc patches live at | |
| 252 | +The -rc patches are not stored on the webserver but are generated on | |
| 253 | +demand from git tags such as | |
| 253 | 254 | |
| 254 | − https://www.kernel.org/pub/linux/kernel/v4.x/testing/ | |
| 255 | + https://git.kernel.org/torvalds/p/v5.1-rc1/v5.0 | |
| 255 | 256 | |
| 257 | +The stable -rc patches live at | |
| 256 | 258 | |
| 257 | −The 4.x kernels | |
| 259 | + https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/ | |
| 260 | + | |
| 261 | + | |
| 262 | +The 5.x kernels | |
| 258 | 263 | =============== |
| 259 | 264 | |
| 260 | 265 | These are the base stable releases released by Linus. The highest numbered |
| 261 | 266 | release is the most recent. |
| 262 | 267 | |
| 263 | 268 | If regressions or other serious flaws are found, then a -stable fix patch |
| 264 | −will be released (see below) on top of this base. Once a new 4.x base | |
| 269 | +will be released (see below) on top of this base. Once a new 5.x base | |
| 265 | 270 | kernel is released, a patch is made available that is a delta between the |
| 266 | −previous 4.x kernel and the new one. | |
| 271 | +previous 5.x kernel and the new one. | |
| 267 | 272 | |
| 268 | −To apply a patch moving from 4.6 to 4.7, you'd do the following (note | |
| 269 | −that such patches do **NOT** apply on top of 4.x.y kernels but on top of the | |
| 270 | −base 4.x kernel -- if you need to move from 4.x.y to 4.x+1 you need to | |
| 271 | −first revert the 4.x.y patch). | |
| 273 | +To apply a patch moving from 5.6 to 5.7, you'd do the following (note | |
| 274 | +that such patches do **NOT** apply on top of 5.x.y kernels but on top of the | |
| 275 | +base 5.x kernel -- if you need to move from 5.x.y to 5.x+1 you need to | |
| 276 | +first revert the 5.x.y patch). | |
| 272 | 277 | |
| 273 | 278 | Here are some examples:: |
| 274 | 279 | |
| 275 | − # moving from 4.6 to 4.7 | |
| 280 | + # moving from 5.6 to 5.7 | |
| 276 | 281 | |
| 277 | − $ cd ~/linux-4.6 # change to kernel source dir | |
| 278 | − $ patch -p1 < ../patch-4.7 # apply the 4.7 patch | |
| 282 | + $ cd ~/linux-5.6 # change to kernel source dir | |
| 283 | + $ patch -p1 < ../patch-5.7 # apply the 5.7 patch | |
| 279 | 284 | $ cd .. |
| 280 | − $ mv linux-4.6 linux-4.7 # rename source dir | |
| 285 | + $ mv linux-5.6 linux-5.7 # rename source dir | |
| 281 | 286 | |
| 282 | − # moving from 4.6.1 to 4.7 | |
| 287 | + # moving from 5.6.1 to 5.7 | |
| 283 | 288 | |
| 284 | − $ cd ~/linux-4.6.1 # change to kernel source dir | |
| 285 | − $ patch -p1 -R < ../patch-4.6.1 # revert the 4.6.1 patch | |
| 286 | − # source dir is now 4.6 | |
| 287 | − $ patch -p1 < ../patch-4.7 # apply new 4.7 patch | |
| 289 | + $ cd ~/linux-5.6.1 # change to kernel source dir | |
| 290 | + $ patch -p1 -R < ../patch-5.6.1 # revert the 5.6.1 patch | |
| 291 | + # source dir is now 5.6 | |
| 292 | + $ patch -p1 < ../patch-5.7 # apply new 5.7 patch | |
| 288 | 293 | $ cd .. |
| 289 | − $ mv linux-4.6.1 linux-4.7 # rename source dir | |
| 294 | + $ mv linux-5.6.1 linux-5.7 # rename source dir | |
| 290 | 295 | |
| 291 | 296 | |
| 292 | −The 4.x.y kernels | |
| 297 | +The 5.x.y kernels | |
| 293 | 298 | ================= |
| 294 | 299 | |
| 295 | 300 | Kernels with 3-digit versions are -stable kernels. They contain small(ish) |
| 296 | 301 | critical fixes for security problems or significant regressions discovered |
| 297 | −in a given 4.x kernel. | |
| 302 | +in a given 5.x kernel. | |
| 298 | 303 | |
| 299 | 304 | This is the recommended branch for users who want the most recent stable |
| 300 | 305 | kernel and are not interested in helping test development/experimental |
| 301 | 306 | versions. |
| 302 | 307 | |
| 303 | −If no 4.x.y kernel is available, then the highest numbered 4.x kernel is | |
| 308 | +If no 5.x.y kernel is available, then the highest numbered 5.x kernel is | |
| 304 | 309 | the current stable kernel. |
| 305 | 310 | |
| 306 | 311 | .. note:: |
| @@ -308,23 +313,23 @@ the current stable kernel. · 1 unmodified line | ||
| 308 | 313 | The -stable team usually do make incremental patches available as well |
| 309 | 314 | as patches against the latest mainline release, but I only cover the |
| 310 | 315 | non-incremental ones below. The incremental ones can be found at |
| 311 | − https://www.kernel.org/pub/linux/kernel/v4.x/incr/ | |
| 316 | + https://www.kernel.org/pub/linux/kernel/v5.x/incr/ | |
| 312 | 317 | |
| 313 | −These patches are not incremental, meaning that for example the 4.7.3 | |
| 314 | −patch does not apply on top of the 4.7.2 kernel source, but rather on top | |
| 315 | −of the base 4.7 kernel source. | |
| 318 | +These patches are not incremental, meaning that for example the 5.7.3 | |
| 319 | +patch does not apply on top of the 5.7.2 kernel source, but rather on top | |
| 320 | +of the base 5.7 kernel source. | |
| 316 | 321 | |
| 317 | −So, in order to apply the 4.7.3 patch to your existing 4.7.2 kernel | |
| 318 | −source you have to first back out the 4.7.2 patch (so you are left with a | |
| 319 | −base 4.7 kernel source) and then apply the new 4.7.3 patch. | |
| 322 | +So, in order to apply the 5.7.3 patch to your existing 5.7.2 kernel | |
| 323 | +source you have to first back out the 5.7.2 patch (so you are left with a | |
| 324 | +base 5.7 kernel source) and then apply the new 5.7.3 patch. | |
| 320 | 325 | |
| 321 | 326 | Here's a small example:: |
| 322 | 327 | |
| 323 | − $ cd ~/linux-4.7.2 # change to the kernel source dir | |
| 324 | − $ patch -p1 -R < ../patch-4.7.2 # revert the 4.7.2 patch | |
| 325 | − $ patch -p1 < ../patch-4.7.3 # apply the new 4.7.3 patch | |
| 328 | + $ cd ~/linux-5.7.2 # change to the kernel source dir | |
| 329 | + $ patch -p1 -R < ../patch-5.7.2 # revert the 5.7.2 patch | |
| 330 | + $ patch -p1 < ../patch-5.7.3 # apply the new 5.7.3 patch | |
| 326 | 331 | $ cd .. |
| 327 | − $ mv linux-4.7.2 linux-4.7.3 # rename the kernel source dir | |
| 332 | + $ mv linux-5.7.2 linux-5.7.3 # rename the kernel source dir | |
| 328 | 333 | |
| 329 | 334 | The -rc kernels |
| 330 | 335 | =============== |
| @@ -343,38 +348,38 @@ This is a good branch to run for people who want to help out testing · 12 unmodified lines | ||
| 343 | 348 | development kernels but do not want to run some of the really experimental |
| 344 | 349 | stuff (such people should see the sections about -next and -mm kernels below). |
| 345 | 350 | |
| 346 | −The -rc patches are not incremental, they apply to a base 4.x kernel, just | |
| 347 | −like the 4.x.y patches described above. The kernel version before the -rcN | |
| 351 | +The -rc patches are not incremental, they apply to a base 5.x kernel, just | |
| 352 | +like the 5.x.y patches described above. The kernel version before the -rcN | |
| 348 | 353 | suffix denotes the version of the kernel that this -rc kernel will eventually |
| 349 | 354 | turn into. |
| 350 | 355 | |
| 351 | −So, 4.8-rc5 means that this is the fifth release candidate for the 4.8 | |
| 352 | −kernel and the patch should be applied on top of the 4.7 kernel source. | |
| 356 | +So, 5.8-rc5 means that this is the fifth release candidate for the 5.8 | |
| 357 | +kernel and the patch should be applied on top of the 5.7 kernel source. | |
| 353 | 358 | |
| 354 | 359 | Here are 3 examples of how to apply these patches:: |
| 355 | 360 | |
| 356 | − # first an example of moving from 4.7 to 4.8-rc3 | |
| 361 | + # first an example of moving from 5.7 to 5.8-rc3 | |
| 357 | 362 | |
| 358 | − $ cd ~/linux-4.7 # change to the 4.7 source dir | |
| 359 | − $ patch -p1 < ../patch-4.8-rc3 # apply the 4.8-rc3 patch | |
| 363 | + $ cd ~/linux-5.7 # change to the 5.7 source dir | |
| 364 | + $ patch -p1 < ../patch-5.8-rc3 # apply the 5.8-rc3 patch | |
| 360 | 365 | $ cd .. |
| 361 | − $ mv linux-4.7 linux-4.8-rc3 # rename the source dir | |
| 366 | + $ mv linux-5.7 linux-5.8-rc3 # rename the source dir | |
| 362 | 367 | |
| 363 | − # now let's move from 4.8-rc3 to 4.8-rc5 | |
| 368 | + # now let's move from 5.8-rc3 to 5.8-rc5 | |
| 364 | 369 | |
| 365 | − $ cd ~/linux-4.8-rc3 # change to the 4.8-rc3 dir | |
| 366 | − $ patch -p1 -R < ../patch-4.8-rc3 # revert the 4.8-rc3 patch | |
| 367 | − $ patch -p1 < ../patch-4.8-rc5 # apply the new 4.8-rc5 patch | |
| 370 | + $ cd ~/linux-5.8-rc3 # change to the 5.8-rc3 dir | |
| 371 | + $ patch -p1 -R < ../patch-5.8-rc3 # revert the 5.8-rc3 patch | |
| 372 | + $ patch -p1 < ../patch-5.8-rc5 # apply the new 5.8-rc5 patch | |
| 368 | 373 | $ cd .. |
| 369 | − $ mv linux-4.8-rc3 linux-4.8-rc5 # rename the source dir | |
| 374 | + $ mv linux-5.8-rc3 linux-5.8-rc5 # rename the source dir | |
| 370 | 375 | |
| 371 | − # finally let's try and move from 4.7.3 to 4.8-rc5 | |
| 376 | + # finally let's try and move from 5.7.3 to 5.8-rc5 | |
| 372 | 377 | |
| 373 | − $ cd ~/linux-4.7.3 # change to the kernel source dir | |
| 374 | − $ patch -p1 -R < ../patch-4.7.3 # revert the 4.7.3 patch | |
| 375 | − $ patch -p1 < ../patch-4.8-rc5 # apply new 4.8-rc5 patch | |
| 378 | + $ cd ~/linux-5.7.3 # change to the kernel source dir | |
| 379 | + $ patch -p1 -R < ../patch-5.7.3 # revert the 5.7.3 patch | |
| 380 | + $ patch -p1 < ../patch-5.8-rc5 # apply new 5.8-rc5 patch | |
| 376 | 381 | $ cd .. |
| 377 | − $ mv linux-4.7.3 linux-4.8-rc5 # rename the kernel source dir | |
| 382 | + $ mv linux-5.7.3 linux-5.8-rc5 # rename the kernel source dir | |
| 378 | 383 | |
| 379 | 384 | |
| 380 | 385 | The -mm patches and the linux-next tree |
| @@ -4,7 +4,7 @@ · 3 unmodified lines | ||
| 4 | 4 | |
| 5 | 5 | .. _it_readme: |
| 6 | 6 | |
| 7 | −Rilascio del kernel Linux 4.x <http://kernel.org/> | |
| 7 | +Rilascio del kernel Linux 5.x <http://kernel.org/> | |
| 8 | 8 | =================================================== |
| 9 | 9 | |
| 10 | 10 | .. warning:: |
| @@ -409,8 +409,7 @@ F: drivers/platform/x86/wmi.c · 408 unmodified lines | ||
| 409 | 409 | F: include/uapi/linux/wmi.h |
| 410 | 410 | |
| 411 | 411 | AD1889 ALSA SOUND DRIVER |
| 412 | −M: Thibaut Varene <T-Bone@parisc-linux.org> | |
| 413 | −W: http://wiki.parisc-linux.org/AD1889 | |
| 412 | +W: https://parisc.wiki.kernel.org/index.php/AD1889 | |
| 414 | 413 | L: linux-parisc@vger.kernel.org |
| 415 | 414 | S: Maintained |
| 416 | 415 | F: sound/pci/ad1889.* |
| @@ -2865,7 +2864,7 @@ R: Martin KaFai Lau <kafai@fb.com> · 2448 unmodified lines | ||
| 2865 | 2864 | R: Song Liu <songliubraving@fb.com> |
| 2866 | 2865 | R: Yonghong Song <yhs@fb.com> |
| 2867 | 2866 | L: netdev@vger.kernel.org |
| 2868 | −L: linux-kernel@vger.kernel.org | |
| 2867 | +L: bpf@vger.kernel.org | |
| 2869 | 2868 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git |
| 2870 | 2869 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git |
| 2871 | 2870 | Q: https://patchwork.ozlabs.org/project/netdev/list/?delegate=77147 |
| @@ -2895,6 +2894,7 @@ N: bpf · 23 unmodified lines | ||
| 2895 | 2894 | BPF JIT for ARM |
| 2896 | 2895 | M: Shubham Bansal <illusionist.neo@gmail.com> |
| 2897 | 2896 | L: netdev@vger.kernel.org |
| 2897 | +L: bpf@vger.kernel.org | |
| 2898 | 2898 | S: Maintained |
| 2899 | 2899 | F: arch/arm/net/ |
| 2900 | 2900 | |
| @@ -2903,18 +2903,21 @@ M: Daniel Borkmann <daniel@iogearbox.net> · 2 unmodified lines | ||
| 2903 | 2903 | M: Alexei Starovoitov <ast@kernel.org> |
| 2904 | 2904 | M: Zi Shen Lim <zlim.lnx@gmail.com> |
| 2905 | 2905 | L: netdev@vger.kernel.org |
| 2906 | +L: bpf@vger.kernel.org | |
| 2906 | 2907 | S: Supported |
| 2907 | 2908 | F: arch/arm64/net/ |
| 2908 | 2909 | |
| 2909 | 2910 | BPF JIT for MIPS (32-BIT AND 64-BIT) |
| 2910 | 2911 | M: Paul Burton <paul.burton@mips.com> |
| 2911 | 2912 | L: netdev@vger.kernel.org |
| 2913 | +L: bpf@vger.kernel.org | |
| 2912 | 2914 | S: Maintained |
| 2913 | 2915 | F: arch/mips/net/ |
| 2914 | 2916 | |
| 2915 | 2917 | BPF JIT for NFP NICs |
| 2916 | 2918 | M: Jakub Kicinski <jakub.kicinski@netronome.com> |
| 2917 | 2919 | L: netdev@vger.kernel.org |
| 2920 | +L: bpf@vger.kernel.org | |
| 2918 | 2921 | S: Supported |
| 2919 | 2922 | F: drivers/net/ethernet/netronome/nfp/bpf/ |
| 2920 | 2923 | |
| @@ -2922,6 +2925,7 @@ BPF JIT for POWERPC (32-BIT AND 64-BIT) · 1 unmodified line | ||
| 2922 | 2925 | M: Naveen N. Rao <naveen.n.rao@linux.ibm.com> |
| 2923 | 2926 | M: Sandipan Das <sandipan@linux.ibm.com> |
| 2924 | 2927 | L: netdev@vger.kernel.org |
| 2928 | +L: bpf@vger.kernel.org | |
| 2925 | 2929 | S: Maintained |
| 2926 | 2930 | F: arch/powerpc/net/ |
| 2927 | 2931 | |
| @@ -2929,6 +2933,7 @@ BPF JIT for S390 · 1 unmodified line | ||
| 2929 | 2933 | M: Martin Schwidefsky <schwidefsky@de.ibm.com> |
| 2930 | 2934 | M: Heiko Carstens <heiko.carstens@de.ibm.com> |
| 2931 | 2935 | L: netdev@vger.kernel.org |
| 2936 | +L: bpf@vger.kernel.org | |
| 2932 | 2937 | S: Maintained |
| 2933 | 2938 | F: arch/s390/net/ |
| 2934 | 2939 | X: arch/s390/net/pnet.c |
| @@ -2936,12 +2941,14 @@ X: arch/s390/net/pnet.c · 1 unmodified line | ||
| 2936 | 2941 | BPF JIT for SPARC (32-BIT AND 64-BIT) |
| 2937 | 2942 | M: David S. Miller <davem@davemloft.net> |
| 2938 | 2943 | L: netdev@vger.kernel.org |
| 2944 | +L: bpf@vger.kernel.org | |
| 2939 | 2945 | S: Maintained |
| 2940 | 2946 | F: arch/sparc/net/ |
| 2941 | 2947 | |
| 2942 | 2948 | BPF JIT for X86 32-BIT |
| 2943 | 2949 | M: Wang YanQing <udknight@gmail.com> |
| 2944 | 2950 | L: netdev@vger.kernel.org |
| 2951 | +L: bpf@vger.kernel.org | |
| 2945 | 2952 | S: Maintained |
| 2946 | 2953 | F: arch/x86/net/bpf_jit_comp32.c |
| 2947 | 2954 | |
| @@ -2949,6 +2956,7 @@ BPF JIT for X86 64-BIT · 1 unmodified line | ||
| 2949 | 2956 | M: Alexei Starovoitov <ast@kernel.org> |
| 2950 | 2957 | M: Daniel Borkmann <daniel@iogearbox.net> |
| 2951 | 2958 | L: netdev@vger.kernel.org |
| 2959 | +L: bpf@vger.kernel.org | |
| 2952 | 2960 | S: Supported |
| 2953 | 2961 | F: arch/x86/net/ |
| 2954 | 2962 | X: arch/x86/net/bpf_jit_comp32.c |
| @@ -3403,9 +3411,8 @@ F: Documentation/media/v4l-drivers/cafe_ccic* · 448 unmodified lines | ||
| 3403 | 3411 | F: drivers/media/platform/marvell-ccic/ |
| 3404 | 3412 | |
| 3405 | 3413 | CAIF NETWORK LAYER |
| 3406 | −M: Dmitry Tarnyagin <dmitry.tarnyagin@lockless.no> | |
| 3407 | 3414 | L: netdev@vger.kernel.org |
| 3408 | −S: Supported | |
| 3415 | +S: Orphan | |
| 3409 | 3416 | F: Documentation/networking/caif/ |
| 3410 | 3417 | F: drivers/net/caif/ |
| 3411 | 3418 | F: include/uapi/linux/caif/ |
| @@ -4851,10 +4858,11 @@ F: Documentation/devicetree/bindings/display/multi-inno,mi0283qt.txt · 1439 unmodified lines | ||
| 4851 | 4858 | |
| 4852 | 4859 | DRM DRIVER FOR MSM ADRENO GPU |
| 4853 | 4860 | M: Rob Clark <robdclark@gmail.com> |
| 4861 | +M: Sean Paul <sean@poorly.run> | |
| 4854 | 4862 | L: linux-arm-msm@vger.kernel.org |
| 4855 | 4863 | L: dri-devel@lists.freedesktop.org |
| 4856 | 4864 | L: freedreno@lists.freedesktop.org |
| 4857 | −T: git git://people.freedesktop.org/~robclark/linux | |
| 4865 | +T: git https://gitlab.freedesktop.org/drm/msm.git | |
| 4858 | 4866 | S: Maintained |
| 4859 | 4867 | F: drivers/gpu/drm/msm/ |
| 4860 | 4868 | F: include/uapi/drm/msm_drm.h |
| @@ -8523,6 +8531,7 @@ L7 BPF FRAMEWORK · 3662 unmodified lines | ||
| 8523 | 8531 | M: John Fastabend <john.fastabend@gmail.com> |
| 8524 | 8532 | M: Daniel Borkmann <daniel@iogearbox.net> |
| 8525 | 8533 | L: netdev@vger.kernel.org |
| 8534 | +L: bpf@vger.kernel.org | |
| 8526 | 8535 | S: Maintained |
| 8527 | 8536 | F: include/linux/skmsg.h |
| 8528 | 8537 | F: net/core/skmsg.c |
| @@ -11524,7 +11533,7 @@ F: Documentation/blockdev/paride.txt · 2995 unmodified lines | ||
| 11524 | 11533 | F: drivers/block/paride/ |
| 11525 | 11534 | |
| 11526 | 11535 | PARISC ARCHITECTURE |
| 11527 | −M: "James E.J. Bottomley" <jejb@parisc-linux.org> | |
| 11536 | +M: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com> | |
| 11528 | 11537 | M: Helge Deller <deller@gmx.de> |
| 11529 | 11538 | L: linux-parisc@vger.kernel.org |
| 11530 | 11539 | W: http://www.parisc-linux.org/ |
| @@ -16750,6 +16759,7 @@ M: Jesper Dangaard Brouer <hawk@kernel.org> · 5219 unmodified lines | ||
| 16750 | 16759 | M: John Fastabend <john.fastabend@gmail.com> |
| 16751 | 16760 | L: netdev@vger.kernel.org |
| 16752 | 16761 | L: xdp-newbies@vger.kernel.org |
| 16762 | +L: bpf@vger.kernel.org | |
| 16753 | 16763 | S: Supported |
| 16754 | 16764 | F: net/core/xdp.c |
| 16755 | 16765 | F: include/net/xdp.h |
| @@ -16763,6 +16773,7 @@ XDP SOCKETS (AF_XDP) · 7 unmodified lines | ||
| 16763 | 16773 | M: Björn Töpel <bjorn.topel@intel.com> |
| 16764 | 16774 | M: Magnus Karlsson <magnus.karlsson@intel.com> |
| 16765 | 16775 | L: netdev@vger.kernel.org |
| 16776 | +L: bpf@vger.kernel.org | |
| 16766 | 16777 | S: Maintained |
| 16767 | 16778 | F: kernel/bpf/xskmap.c |
| 16768 | 16779 | F: net/xdp/ |
| @@ -2,7 +2,7 @@ · 1 unmodified line | ||
| 2 | 2 | VERSION = 5 |
| 3 | 3 | PATCHLEVEL = 0 |
| 4 | 4 | SUBLEVEL = 0 |
| 5 | −EXTRAVERSION = -rc7 | |
| 5 | +EXTRAVERSION = | |
| 6 | 6 | NAME = Shy Crocodile |
| 7 | 7 | |
| 8 | 8 | # *DOCUMENTATION* |
| @@ -191,7 +191,6 @@ config NR_CPUS · 190 unmodified lines | ||
| 191 | 191 | |
| 192 | 192 | config ARC_SMP_HALT_ON_RESET |
| 193 | 193 | bool "Enable Halt-on-reset boot mode" |
| 194 | − default y if ARC_UBOOT_SUPPORT | |
| 195 | 194 | help |
| 196 | 195 | In SMP configuration cores can be configured as Halt-on-reset |
| 197 | 196 | or they could all start at same time. For Halt-on-reset, non |
| @@ -407,6 +406,14 @@ config ARC_HAS_ACCL_REGS · 209 unmodified lines | ||
| 407 | 406 | (also referred to as r58:r59). These can also be used by gcc as GPR so |
| 408 | 407 | kernel needs to save/restore per process |
| 409 | 408 | |
| 409 | +config ARC_IRQ_NO_AUTOSAVE | |
| 410 | + bool "Disable hardware autosave regfile on interrupts" | |
| 411 | + default n | |
| 412 | + help | |
| 413 | + On HS cores, taken interrupt auto saves the regfile on stack. | |
| 414 | + This is programmable and can be optionally disabled in which case | |
| 415 | + software INTERRUPT_PROLOGUE/EPILGUE do the needed work | |
| 416 | + | |
| 410 | 417 | endif # ISA_ARCV2 |
| 411 | 418 | |
| 412 | 419 | endmenu # "ARC CPU Configuration" |
| @@ -515,17 +522,6 @@ config ARC_DBG_TLB_PARANOIA · 102 unmodified lines | ||
| 515 | 522 | |
| 516 | 523 | endif |
| 517 | 524 | |
| 518 | −config ARC_UBOOT_SUPPORT | |
| 519 | − bool "Support uboot arg Handling" | |
| 520 | − help | |
| 521 | − ARC Linux by default checks for uboot provided args as pointers to | |
| 522 | − external cmdline or DTB. This however breaks in absence of uboot, | |
| 523 | − when booting from Metaware debugger directly, as the registers are | |
| 524 | − not zeroed out on reset by mdb and/or ARCv2 based cores. The bogus | |
| 525 | − registers look like uboot args to kernel which then chokes. | |
| 526 | − So only enable the uboot arg checking/processing if users are sure | |
| 527 | − of uboot being in play. | |
| 528 | − | |
| 529 | 525 | config ARC_BUILTIN_DTB_NAME |
| 530 | 526 | string "Built in DTB" |
| 531 | 527 | help |
| @@ -31,7 +31,6 @@ CONFIG_ARC_CACHE_LINE_SHIFT=5 · 30 unmodified lines | ||
| 31 | 31 | # CONFIG_ARC_HAS_LLSC is not set |
| 32 | 32 | CONFIG_ARC_KVADDR_SIZE=402 |
| 33 | 33 | CONFIG_ARC_EMUL_UNALIGNED=y |
| 34 | −CONFIG_ARC_UBOOT_SUPPORT=y | |
| 35 | 34 | CONFIG_PREEMPT=y |
| 36 | 35 | CONFIG_NET=y |
| 37 | 36 | CONFIG_UNIX=y |
| @@ -13,7 +13,6 @@ CONFIG_PARTITION_ADVANCED=y · 12 unmodified lines | ||
| 13 | 13 | CONFIG_ARC_PLAT_AXS10X=y |
| 14 | 14 | CONFIG_AXS103=y |
| 15 | 15 | CONFIG_ISA_ARCV2=y |
| 16 | −CONFIG_ARC_UBOOT_SUPPORT=y | |
| 17 | 16 | CONFIG_ARC_BUILTIN_DTB_NAME="vdk_hs38" |
| 18 | 17 | CONFIG_PREEMPT=y |
| 19 | 18 | CONFIG_NET=y |
| @@ -15,8 +15,6 @@ CONFIG_AXS103=y · 14 unmodified lines | ||
| 15 | 15 | CONFIG_ISA_ARCV2=y |
| 16 | 16 | CONFIG_SMP=y |
| 17 | 17 | # CONFIG_ARC_TIMERS_64BIT is not set |
| 18 | −# CONFIG_ARC_SMP_HALT_ON_RESET is not set | |
| 19 | −CONFIG_ARC_UBOOT_SUPPORT=y | |
| 20 | 18 | CONFIG_ARC_BUILTIN_DTB_NAME="vdk_hs38_smp" |
| 21 | 19 | CONFIG_PREEMPT=y |
| 22 | 20 | CONFIG_NET=y |
| @@ -151,6 +151,14 @@ struct bcr_isa_arcv2 { · 150 unmodified lines | ||
| 151 | 151 | #endif |
| 152 | 152 | }; |
| 153 | 153 | |
| 154 | +struct bcr_uarch_build_arcv2 { | |
| 155 | +#ifdef CONFIG_CPU_BIG_ENDIAN | |
| 156 | + unsigned int pad:8, prod:8, maj:8, min:8; | |
| 157 | +#else | |
| 158 | + unsigned int min:8, maj:8, prod:8, pad:8; | |
| 159 | +#endif | |
| 160 | +}; | |
| 161 | + | |
| 154 | 162 | struct bcr_mpy { |
| 155 | 163 | #ifdef CONFIG_CPU_BIG_ENDIAN |
| 156 | 164 | unsigned int pad:8, x1616:8, dsp:4, cycles:2, type:2, ver:8; |
| @@ -52,6 +52,17 @@ · 51 unmodified lines | ||
| 52 | 52 | #define cache_line_size() SMP_CACHE_BYTES |
| 53 | 53 | #define ARCH_DMA_MINALIGN SMP_CACHE_BYTES |
| 54 | 54 | |
| 55 | +/* | |
| 56 | + * Make sure slab-allocated buffers are 64-bit aligned when atomic64_t uses | |
| 57 | + * ARCv2 64-bit atomics (LLOCKD/SCONDD). This guarantess runtime 64-bit | |
| 58 | + * alignment for any atomic64_t embedded in buffer. | |
| 59 | + * Default ARCH_SLAB_MINALIGN is __alignof__(long long) which has a relaxed | |
| 60 | + * value of 4 (and not 8) in ARC ABI. | |
| 61 | + */ | |
| 62 | +#if defined(CONFIG_ARC_HAS_LL64) && defined(CONFIG_ARC_HAS_LLSC) | |
| 63 | +#define ARCH_SLAB_MINALIGN 8 | |
| 64 | +#endif | |
| 65 | + | |
| 55 | 66 | extern void arc_cache_init(void); |
| 56 | 67 | extern char *arc_cache_mumbojumbo(int cpu_id, char *buf, int len); |
| 57 | 68 | extern void read_decode_cache_bcr(void); |
| @@ -17,6 +17,33 @@ · 16 unmodified lines | ||
| 17 | 17 | ; |
| 18 | 18 | ; Now manually save: r12, sp, fp, gp, r25 |
| 19 | 19 | |
| 20 | +#ifdef CONFIG_ARC_IRQ_NO_AUTOSAVE | |
| 21 | +.ifnc \called_from, exception | |
| 22 | + st.as r9, [sp, -10] ; save r9 in it's final stack slot | |
| 23 | + sub sp, sp, 12 ; skip JLI, LDI, EI | |
| 24 | + | |
| 25 | + PUSH lp_count | |
| 26 | + PUSHAX lp_start | |
| 27 | + PUSHAX lp_end | |
| 28 | + PUSH blink | |
| 29 | + | |
| 30 | + PUSH r11 | |
| 31 | + PUSH r10 | |
| 32 | + | |
| 33 | + sub sp, sp, 4 ; skip r9 | |
| 34 | + | |
| 35 | + PUSH r8 | |
| 36 | + PUSH r7 | |
| 37 | + PUSH r6 | |
| 38 | + PUSH r5 | |
| 39 | + PUSH r4 | |
| 40 | + PUSH r3 | |
| 41 | + PUSH r2 | |
| 42 | + PUSH r1 | |
| 43 | + PUSH r0 | |
| 44 | +.endif | |
| 45 | +#endif | |
| 46 | + | |
| 20 | 47 | #ifdef CONFIG_ARC_HAS_ACCL_REGS |
| 21 | 48 | PUSH r59 |
| 22 | 49 | PUSH r58 |
| @@ -86,6 +113,33 @@ · 63 unmodified lines | ||
| 86 | 113 | POP r59 |
| 87 | 114 | #endif |
| 88 | 115 | |
| 116 | +#ifdef CONFIG_ARC_IRQ_NO_AUTOSAVE | |
| 117 | +.ifnc \called_from, exception | |
| 118 | + POP r0 | |
| 119 | + POP r1 | |
| 120 | + POP r2 | |
| 121 | + POP r3 | |
| 122 | + POP r4 | |
| 123 | + POP r5 | |
| 124 | + POP r6 | |
| 125 | + POP r7 | |
| 126 | + POP r8 | |
| 127 | + POP r9 | |
| 128 | + POP r10 | |
| 129 | + POP r11 | |
| 130 | + | |
| 131 | + POP blink | |
| 132 | + POPAX lp_end | |
| 133 | + POPAX lp_start | |
| 134 | + | |
| 135 | + POP r9 | |
| 136 | + mov lp_count, r9 | |
| 137 | + | |
| 138 | + add sp, sp, 12 ; skip JLI, LDI, EI | |
| 139 | + ld.as r9, [sp, -10] ; reload r9 which got clobbered | |
| 140 | +.endif | |
| 141 | +#endif | |
| 142 | + | |
| 89 | 143 | .endm |
| 90 | 144 | |
| 91 | 145 | /*------------------------------------------------------------------------*/ |
| @@ -207,7 +207,7 @@ raw_copy_from_user(void *to, const void __user *from, unsigned long n) · 206 unmodified lines | ||
| 207 | 207 | */ |
| 208 | 208 | "=&r" (tmp), "+r" (to), "+r" (from) |
| 209 | 209 | : |
| 210 | − : "lp_count", "lp_start", "lp_end", "memory"); | |
| 210 | + : "lp_count", "memory"); | |
| 211 | 211 | |
| 212 | 212 | return n; |
| 213 | 213 | } |
| @@ -433,7 +433,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n) · 219 unmodified lines | ||
| 433 | 433 | */ |
| 434 | 434 | "=&r" (tmp), "+r" (to), "+r" (from) |
| 435 | 435 | : |
| 436 | − : "lp_count", "lp_start", "lp_end", "memory"); | |
| 436 | + : "lp_count", "memory"); | |
| 437 | 437 | |
| 438 | 438 | return n; |
| 439 | 439 | } |
| @@ -653,7 +653,7 @@ static inline unsigned long __arc_clear_user(void __user *to, unsigned long n) · 213 unmodified lines | ||
| 653 | 653 | " .previous \n" |
| 654 | 654 | : "+r"(d_char), "+r"(res) |
| 655 | 655 | : "i"(0) |
| 656 | − : "lp_count", "lp_start", "lp_end", "memory"); | |
| 656 | + : "lp_count", "memory"); | |
| 657 | 657 | |
| 658 | 658 | return res; |
| 659 | 659 | } |
| @@ -686,7 +686,7 @@ __arc_strncpy_from_user(char *dst, const char __user *src, long count) · 26 unmodified lines | ||
| 686 | 686 | " .previous \n" |
| 687 | 687 | : "+r"(res), "+r"(dst), "+r"(src), "=r"(val) |
| 688 | 688 | : "g"(-EFAULT), "r"(count) |
| 689 | − : "lp_count", "lp_start", "lp_end", "memory"); | |
| 689 | + : "lp_count", "memory"); | |
| 690 | 690 | |
| 691 | 691 | return res; |
| 692 | 692 | } |
| @@ -209,7 +209,9 @@ restore_regs: · 208 unmodified lines | ||
| 209 | 209 | ;####### Return from Intr ####### |
| 210 | 210 | |
| 211 | 211 | debug_marker_l1: |
| 212 | − bbit1.nt r0, STATUS_DE_BIT, .Lintr_ret_to_delay_slot | |
| 212 | + ; bbit1.nt r0, STATUS_DE_BIT, .Lintr_ret_to_delay_slot | |
| 213 | + btst r0, STATUS_DE_BIT ; Z flag set if bit clear | |
| 214 | + bnz .Lintr_ret_to_delay_slot ; branch if STATUS_DE_BIT set | |
| 213 | 215 | |
| 214 | 216 | .Lisr_ret_fast_path: |
| 215 | 217 | ; Handle special case #1: (Entry via Exception, Return via IRQ) |
| @@ -17,6 +17,7 @@ · 16 unmodified lines | ||
| 17 | 17 | #include <asm/entry.h> |
| 18 | 18 | #include <asm/arcregs.h> |
| 19 | 19 | #include <asm/cache.h> |
| 20 | +#include <asm/irqflags.h> | |
| 20 | 21 | |
| 21 | 22 | .macro CPU_EARLY_SETUP |
| 22 | 23 | |
| @@ -47,6 +48,15 @@ · 24 unmodified lines | ||
| 47 | 48 | sr r5, [ARC_REG_DC_CTRL] |
| 48 | 49 | |
| 49 | 50 | 1: |
| 51 | + | |
| 52 | +#ifdef CONFIG_ISA_ARCV2 | |
| 53 | + ; Unaligned access is disabled at reset, so re-enable early as | |
| 54 | + ; gcc 7.3.1 (ARC GNU 2018.03) onwards generates unaligned access | |
| 55 | + ; by default | |
| 56 | + lr r5, [status32] | |
| 57 | + bset r5, r5, STATUS_AD_BIT | |
| 58 | + kflag r5 | |
| 59 | +#endif | |
| 50 | 60 | .endm |
| 51 | 61 | |
| 52 | 62 | .section .init.text, "ax",@progbits |
| @@ -90,15 +100,13 @@ ENTRY(stext) · 37 unmodified lines | ||
| 90 | 100 | st.ab 0, [r5, 4] |
| 91 | 101 | 1: |
| 92 | 102 | |
| 93 | −#ifdef CONFIG_ARC_UBOOT_SUPPORT | |
| 94 | 103 | ; Uboot - kernel ABI |
| 95 | 104 | ; r0 = [0] No uboot interaction, [1] cmdline in r2, [2] DTB in r2 |
| 96 | − ; r1 = magic number (board identity, unused as of now | |
| 105 | + ; r1 = magic number (always zero as of now) | |
| 97 | 106 | ; r2 = pointer to uboot provided cmdline or external DTB in mem |
| 98 | − ; These are handled later in setup_arch() | |
| 107 | + ; These are handled later in handle_uboot_args() | |
| 99 | 108 | st r0, [@uboot_tag] |
| 100 | 109 | st r2, [@uboot_arg] |
| 101 | −#endif | |
| 102 | 110 | |
| 103 | 111 | ; setup "current" tsk and optionally cache it in dedicated r25 |
| 104 | 112 | mov r9, @init_task |
| @@ -49,11 +49,13 @@ void arc_init_IRQ(void) · 48 unmodified lines | ||
| 49 | 49 | |
| 50 | 50 | *(unsigned int *)&ictrl = 0; |
| 51 | 51 | |
| 52 | +#ifndef CONFIG_ARC_IRQ_NO_AUTOSAVE | |
| 52 | 53 | ictrl.save_nr_gpr_pairs = 6; /* r0 to r11 (r12 saved manually) */ |
| 53 | 54 | ictrl.save_blink = 1; |
| 54 | 55 | ictrl.save_lp_regs = 1; /* LP_COUNT, LP_START, LP_END */ |
| 55 | 56 | ictrl.save_u_to_u = 0; /* user ctxt saved on kernel stack */ |
| 56 | 57 | ictrl.save_idx_regs = 1; /* JLI, LDI, EI */ |
| 58 | +#endif | |
| 57 | 59 | |
| 58 | 60 | WRITE_AUX(AUX_IRQ_CTRL, ictrl); |
| 59 | 61 | |
| @@ -199,20 +199,36 @@ static void read_arc_build_cfg_regs(void) · 198 unmodified lines | ||
| 199 | 199 | cpu->bpu.ret_stk = 4 << bpu.rse; |
| 200 | 200 | |
| 201 | 201 | if (cpu->core.family >= 0x54) { |
| 202 | − unsigned int exec_ctrl; | |
| 203 | 202 | |
| 204 | − READ_BCR(AUX_EXEC_CTRL, exec_ctrl); | |
| 205 | − cpu->extn.dual_enb = !(exec_ctrl & 1); | |
| 203 | + struct bcr_uarch_build_arcv2 uarch; | |
| 206 | 204 | |
| 207 | − /* dual issue always present for this core */ | |
| 208 | − cpu->extn.dual = 1; | |
| 205 | + /* | |
| 206 | + * The first 0x54 core (uarch maj:min 0:1 or 0:2) was | |
| 207 | + * dual issue only (HS4x). But next uarch rev (1:0) | |
| 208 | + * allows it be configured for single issue (HS3x) | |
| 209 | + * Ensure we fiddle with dual issue only on HS4x | |
| 210 | + */ | |
| 211 | + READ_BCR(ARC_REG_MICRO_ARCH_BCR, uarch); | |
| 212 | + | |
| 213 | + if (uarch.prod == 4) { | |
| 214 | + unsigned int exec_ctrl; | |
| 215 | + | |
| 216 | + /* dual issue hardware always present */ | |
| 217 | + cpu->extn.dual = 1; | |
| 218 | + | |
| 219 | + READ_BCR(AUX_EXEC_CTRL, exec_ctrl); | |
| 220 | + | |
| 221 | + /* dual issue hardware enabled ? */ | |
| 222 | + cpu->extn.dual_enb = !(exec_ctrl & 1); | |
| 223 | + | |
| 224 | + } | |
| 209 | 225 | } |
| 210 | 226 | } |
| 211 | 227 | |
| 212 | 228 | READ_BCR(ARC_REG_AP_BCR, ap); |
| 213 | 229 | if (ap.ver) { |
| 214 | 230 | cpu->extn.ap_num = 2 << ap.num; |
| 215 | − cpu->extn.ap_full = !!ap.min; | |
| 231 | + cpu->extn.ap_full = !ap.min; | |
| 216 | 232 | } |
| 217 | 233 | |
| 218 | 234 | READ_BCR(ARC_REG_SMART_BCR, bcr); |
| @@ -462,43 +478,78 @@ void setup_processor(void) · 243 unmodified lines | ||
| 462 | 478 | arc_chk_core_config(); |
| 463 | 479 | } |
| 464 | 480 | |
| 465 | −static inline int is_kernel(unsigned long addr) | |
| 481 | +static inline bool uboot_arg_invalid(unsigned long addr) | |
| 466 | 482 | { |
| 467 | − if (addr >= (unsigned long)_stext && addr <= (unsigned long)_end) | |
| 468 | − return 1; | |
| 469 | − return 0; | |
| 483 | + /* | |
| 484 | + * Check that it is a untranslated address (although MMU is not enabled | |
| 485 | + * yet, it being a high address ensures this is not by fluke) | |
| 486 | + */ | |
| 487 | + if (addr < PAGE_OFFSET) | |
| 488 | + return true; | |
| 489 | + | |
| 490 | + /* Check that address doesn't clobber resident kernel image */ | |
| 491 | + return addr >= (unsigned long)_stext && addr <= (unsigned long)_end; | |
| 470 | 492 | } |
| 471 | 493 | |
| 472 | −void __init setup_arch(char **cmdline_p) | |
| 494 | +#define IGNORE_ARGS "Ignore U-boot args: " | |
| 495 | + | |
| 496 | +/* uboot_tag values for U-boot - kernel ABI revision 0; see head.S */ | |
| 497 | +#define UBOOT_TAG_NONE 0 | |
| 498 | +#define UBOOT_TAG_CMDLINE 1 | |
| 499 | +#define UBOOT_TAG_DTB 2 | |
| 500 | + | |
| 501 | +void __init handle_uboot_args(void) | |
| 473 | 502 | { |
| 474 | −#ifdef CONFIG_ARC_UBOOT_SUPPORT | |
| 475 | − /* make sure that uboot passed pointer to cmdline/dtb is valid */ | |
| 476 | − if (uboot_tag && is_kernel((unsigned long)uboot_arg)) | |
| 477 | − panic("Invalid uboot arg\n"); | |
| 478 | − | |
| 479 | − /* See if u-boot passed an external Device Tree blob */ | |
| 480 | − machine_desc = setup_machine_fdt(uboot_arg); /* uboot_tag == 2 */ | |
| 481 | − if (!machine_desc) | |
| 482 | −#endif | |
| 483 | − { | |
| 484 | − /* No, so try the embedded one */ | |
| 503 | + bool use_embedded_dtb = true; | |
| 504 | + bool append_cmdline = false; | |
| 505 | + | |
| 506 | + /* check that we know this tag */ | |
| 507 | + if (uboot_tag != UBOOT_TAG_NONE && | |
| 508 | + uboot_tag != UBOOT_TAG_CMDLINE && | |
| 509 | + uboot_tag != UBOOT_TAG_DTB) { | |
| 510 | + pr_warn(IGNORE_ARGS "invalid uboot tag: '%08x'\n", uboot_tag); | |
| 511 | + goto ignore_uboot_args; | |
| 512 | + } | |
| 513 | + | |
| 514 | + if (uboot_tag != UBOOT_TAG_NONE && | |
| 515 | + uboot_arg_invalid((unsigned long)uboot_arg)) { | |
| 516 | + pr_warn(IGNORE_ARGS "invalid uboot arg: '%px'\n", uboot_arg); | |
| 517 | + goto ignore_uboot_args; | |
| 518 | + } | |
| 519 | + | |
| 520 | + /* see if U-boot passed an external Device Tree blob */ | |
| 521 | + if (uboot_tag == UBOOT_TAG_DTB) { | |
| 522 | + machine_desc = setup_machine_fdt((void *)uboot_arg); | |
| 523 | + | |
| 524 | + /* external Device Tree blob is invalid - use embedded one */ | |
| 525 | + use_embedded_dtb = !machine_desc; | |
| 526 | + } | |
| 527 | + | |
| 528 | + if (uboot_tag == UBOOT_TAG_CMDLINE) | |
| 529 | + append_cmdline = true; | |
| 530 | + | |
| 531 | +ignore_uboot_args: | |
| 532 | + | |
| 533 | + if (use_embedded_dtb) { | |
| 485 | 534 | machine_desc = setup_machine_fdt(__dtb_start); |
| 486 | 535 | if (!machine_desc) |
| 487 | 536 | panic("Embedded DT invalid\n"); |
| 537 | + } | |
| 488 | 538 | |
| 489 | − /* | |
| 490 | − * If we are here, it is established that @uboot_arg didn't | |
| 491 | − * point to DT blob. Instead if u-boot says it is cmdline, | |
| 492 | − * append to embedded DT cmdline. | |
| 493 | − * setup_machine_fdt() would have populated @boot_command_line | |
| 494 | − */ | |
| 495 | − if (uboot_tag == 1) { | |
| 496 | − /* Ensure a whitespace between the 2 cmdlines */ | |
| 497 | − strlcat(boot_command_line, " ", COMMAND_LINE_SIZE); | |
| 498 | − strlcat(boot_command_line, uboot_arg, | |
| 499 | − COMMAND_LINE_SIZE); | |
| 500 | − } | |
| 539 | + /* | |
| 540 | + * NOTE: @boot_command_line is populated by setup_machine_fdt() so this | |
| 541 | + * append processing can only happen after. | |
| 542 | + */ | |
| 543 | + if (append_cmdline) { | |
| 544 | + /* Ensure a whitespace between the 2 cmdlines */ | |
| 545 | + strlcat(boot_command_line, " ", COMMAND_LINE_SIZE); | |
| 546 | + strlcat(boot_command_line, uboot_arg, COMMAND_LINE_SIZE); | |
| 501 | 547 | } |
| 548 | +} | |
| 549 | + | |
| 550 | +void __init setup_arch(char **cmdline_p) | |
| 551 | +{ | |
| 552 | + handle_uboot_args(); | |
| 502 | 553 | |
| 503 | 554 | /* Save unparsed command line copy for /proc/cmdline */ |
| 504 | 555 | *cmdline_p = boot_command_line; |
| @@ -25,15 +25,11 @@ · 24 unmodified lines | ||
| 25 | 25 | #endif |
| 26 | 26 | |
| 27 | 27 | #ifdef CONFIG_ARC_HAS_LL64 |
| 28 | −# define PREFETCH_READ(RX) prefetch [RX, 56] | |
| 29 | −# define PREFETCH_WRITE(RX) prefetchw [RX, 64] | |
| 30 | 28 | # define LOADX(DST,RX) ldd.ab DST, [RX, 8] |
| 31 | 29 | # define STOREX(SRC,RX) std.ab SRC, [RX, 8] |
| 32 | 30 | # define ZOLSHFT 5 |
| 33 | 31 | # define ZOLAND 0x1F |
| 34 | 32 | #else |
| 35 | −# define PREFETCH_READ(RX) prefetch [RX, 28] | |
| 36 | −# define PREFETCH_WRITE(RX) prefetchw [RX, 32] | |
| 37 | 33 | # define LOADX(DST,RX) ld.ab DST, [RX, 4] |
| 38 | 34 | # define STOREX(SRC,RX) st.ab SRC, [RX, 4] |
| 39 | 35 | # define ZOLSHFT 4 |
| @@ -41,8 +37,6 @@ · 1 unmodified line | ||
| 41 | 37 | #endif |
| 42 | 38 | |
| 43 | 39 | ENTRY_CFI(memcpy) |
| 44 | − prefetch [r1] ; Prefetch the read location | |
| 45 | − prefetchw [r0] ; Prefetch the write location | |
| 46 | 40 | mov.f 0, r2 |
| 47 | 41 | ;;; if size is zero |
| 48 | 42 | jz.d [blink] |
| @@ -72,8 +66,6 @@ ENTRY_CFI(memcpy) · 23 unmodified lines | ||
| 72 | 66 | lpnz @.Lcopy32_64bytes |
| 73 | 67 | ;; LOOP START |
| 74 | 68 | LOADX (r6, r1) |
| 75 | − PREFETCH_READ (r1) | |
| 76 | − PREFETCH_WRITE (r3) | |
| 77 | 69 | LOADX (r8, r1) |
| 78 | 70 | LOADX (r10, r1) |
| 79 | 71 | LOADX (r4, r1) |
| @@ -117,9 +109,7 @@ ENTRY_CFI(memcpy) · 37 unmodified lines | ||
| 117 | 109 | lpnz @.Lcopy8bytes_1 |
| 118 | 110 | ;; LOOP START |
| 119 | 111 | ld.ab r6, [r1, 4] |
| 120 | − prefetch [r1, 28] ;Prefetch the next read location | |
| 121 | 112 | ld.ab r8, [r1,4] |
| 122 | − prefetchw [r3, 32] ;Prefetch the next write location | |
| 123 | 113 | |
| 124 | 114 | SHIFT_1 (r7, r6, 24) |
| 125 | 115 | or r7, r7, r5 |
| @@ -162,9 +152,7 @@ ENTRY_CFI(memcpy) · 36 unmodified lines | ||
| 162 | 152 | lpnz @.Lcopy8bytes_2 |
| 163 | 153 | ;; LOOP START |
| 164 | 154 | ld.ab r6, [r1, 4] |
| 165 | − prefetch [r1, 28] ;Prefetch the next read location | |
| 166 | 155 | ld.ab r8, [r1,4] |
| 167 | − prefetchw [r3, 32] ;Prefetch the next write location | |
| 168 | 156 | |
| 169 | 157 | SHIFT_1 (r7, r6, 16) |
| 170 | 158 | or r7, r7, r5 |
| @@ -204,9 +192,7 @@ ENTRY_CFI(memcpy) · 33 unmodified lines | ||
| 204 | 192 | lpnz @.Lcopy8bytes_3 |
| 205 | 193 | ;; LOOP START |
| 206 | 194 | ld.ab r6, [r1, 4] |
| 207 | − prefetch [r1, 28] ;Prefetch the next read location | |
| 208 | 195 | ld.ab r8, [r1,4] |
| 209 | − prefetchw [r3, 32] ;Prefetch the next write location | |
| 210 | 196 | |
| 211 | 197 | SHIFT_1 (r7, r6, 8) |
| 212 | 198 | or r7, r7, r5 |
| @@ -9,6 +9,7 @@ menuconfig ARC_SOC_HSDK · 8 unmodified lines | ||
| 9 | 9 | bool "ARC HS Development Kit SOC" |
| 10 | 10 | depends on ISA_ARCV2 |
| 11 | 11 | select ARC_HAS_ACCL_REGS |
| 12 | + select ARC_IRQ_NO_AUTOSAVE | |
| 12 | 13 | select CLK_HSDK |
| 13 | 14 | select RESET_HSDK |
| 14 | 15 | select HAVE_PCI |
| @@ -1400,6 +1400,7 @@ config NR_CPUS · 1399 unmodified lines | ||
| 1400 | 1400 | config HOTPLUG_CPU |
| 1401 | 1401 | bool "Support for hot-pluggable CPUs" |
| 1402 | 1402 | depends on SMP |
| 1403 | + select GENERIC_IRQ_MIGRATION | |
| 1403 | 1404 | help |
| 1404 | 1405 | Say Y here to experiment with turning CPUs off and on. CPUs |
| 1405 | 1406 | can be controlled through /sys/devices/system/cpu. |
| @@ -729,7 +729,7 @@ · 728 unmodified lines | ||
| 729 | 729 | |
| 730 | 730 | &cpsw_emac0 { |
| 731 | 731 | phy-handle = <ðphy0>; |
| 732 | − phy-mode = "rgmii-txid"; | |
| 732 | + phy-mode = "rgmii-id"; | |
| 733 | 733 | }; |
| 734 | 734 | |
| 735 | 735 | &tscadc { |
| @@ -651,13 +651,13 @@ · 650 unmodified lines | ||
| 651 | 651 | |
| 652 | 652 | &cpsw_emac0 { |
| 653 | 653 | phy-handle = <ðphy0>; |
| 654 | − phy-mode = "rgmii-txid"; | |
| 654 | + phy-mode = "rgmii-id"; | |
| 655 | 655 | dual_emac_res_vlan = <1>; |
| 656 | 656 | }; |
| 657 | 657 | |
| 658 | 658 | &cpsw_emac1 { |
| 659 | 659 | phy-handle = <ðphy1>; |
| 660 | − phy-mode = "rgmii-txid"; | |
| 660 | + phy-mode = "rgmii-id"; | |
| 661 | 661 | dual_emac_res_vlan = <2>; |
| 662 | 662 | }; |
| 663 | 663 | |
| @@ -144,30 +144,32 @@ · 143 unmodified lines | ||
| 144 | 144 | status = "okay"; |
| 145 | 145 | }; |
| 146 | 146 | |
| 147 | − nand@d0000 { | |
| 147 | + nand-controller@d0000 { | |
| 148 | 148 | status = "okay"; |
| 149 | − label = "pxa3xx_nand-0"; | |
| 150 | − num-cs = <1>; | |
| 151 | − marvell,nand-keep-config; | |
| 152 | − nand-on-flash-bbt; | |
| 153 | − | |
| 154 | − partitions { | |
| 155 | − compatible = "fixed-partitions"; | |
| 156 | − #address-cells = <1>; | |
| 157 | − #size-cells = <1>; | |
| 158 | − | |
| 159 | − partition@0 { | |
| 160 | − label = "U-Boot"; | |
| 161 | − reg = <0 0x800000>; | |
| 162 | − }; | |
| 163 | − partition@800000 { | |
| 164 | − label = "Linux"; | |
| 165 | − reg = <0x800000 0x800000>; | |
| 166 | − }; | |
| 167 | − partition@1000000 { | |
| 168 | − label = "Filesystem"; | |
| 169 | − reg = <0x1000000 0x3f000000>; | |
| 170 | 149 | |
| 150 | + nand@0 { | |
| 151 | + reg = <0>; | |
| 152 | + label = "pxa3xx_nand-0"; | |
| 153 | + nand-rb = <0>; | |
| 154 | + nand-on-flash-bbt; | |
| 155 | + | |
| 156 | + partitions { | |
| 157 | + compatible = "fixed-partitions"; | |
| 158 | + #address-cells = <1>; | |
| 159 | + #size-cells = <1>; | |
| 160 | + | |
| 161 | + partition@0 { | |
| 162 | + label = "U-Boot"; | |
| 163 | + reg = <0 0x800000>; | |
| 164 | + }; | |
| 165 | + partition@800000 { | |
| 166 | + label = "Linux"; | |
| 167 | + reg = <0x800000 0x800000>; | |
| 168 | + }; | |
| 169 | + partition@1000000 { | |
| 170 | + label = "Filesystem"; | |
| 171 | + reg = <0x1000000 0x3f000000>; | |
| 172 | + }; | |
| 171 | 173 | }; |
| 172 | 174 | }; |
| 173 | 175 | }; |
| @@ -160,12 +160,15 @@ · 159 unmodified lines | ||
| 160 | 160 | status = "okay"; |
| 161 | 161 | }; |
| 162 | 162 | |
| 163 | − nand@d0000 { | |
| 163 | + nand-controller@d0000 { | |
| 164 | 164 | status = "okay"; |
| 165 | − label = "pxa3xx_nand-0"; | |
| 166 | − num-cs = <1>; | |
| 167 | − marvell,nand-keep-config; | |
| 168 | − nand-on-flash-bbt; | |
| 165 | + | |
| 166 | + nand@0 { | |
| 167 | + reg = <0>; | |
| 168 | + label = "pxa3xx_nand-0"; | |
| 169 | + nand-rb = <0>; | |
| 170 | + nand-on-flash-bbt; | |
| 171 | + }; | |
| 169 | 172 | }; |
| 170 | 173 | }; |
| 171 | 174 | |
| @@ -81,49 +81,52 @@ · 80 unmodified lines | ||
| 81 | 81 | |
| 82 | 82 | }; |
| 83 | 83 | |
| 84 | − nand@d0000 { | |
| 84 | + nand-controller@d0000 { | |
| 85 | 85 | status = "okay"; |
| 86 | − label = "pxa3xx_nand-0"; | |
| 87 | − num-cs = <1>; | |
| 88 | − marvell,nand-keep-config; | |
| 89 | − nand-on-flash-bbt; | |
| 90 | − | |
| 91 | − partitions { | |
| 92 | − compatible = "fixed-partitions"; | |
| 93 | − #address-cells = <1>; | |
| 94 | − #size-cells = <1>; | |
| 95 | − | |
| 96 | − partition@0 { | |
| 97 | − label = "u-boot"; | |
| 98 | − reg = <0x00000000 0x000e0000>; | |
| 99 | − read-only; | |
| 100 | − }; | |
| 101 | − | |
| 102 | − partition@e0000 { | |
| 103 | − label = "u-boot-env"; | |
| 104 | − reg = <0x000e0000 0x00020000>; | |
| 105 | − read-only; | |
| 106 | − }; | |
| 107 | − | |
| 108 | − partition@100000 { | |
| 109 | − label = "u-boot-env2"; | |
| 110 | − reg = <0x00100000 0x00020000>; | |
| 111 | − read-only; | |
| 112 | − }; | |
| 113 | − | |
| 114 | − partition@120000 { | |
| 115 | − label = "zImage"; | |
| 116 | − reg = <0x00120000 0x00400000>; | |
| 117 | − }; | |
| 118 | − | |
| 119 | − partition@520000 { | |
| 120 | − label = "initrd"; | |
| 121 | − reg = <0x00520000 0x00400000>; | |
| 122 | − }; | |
| 123 | 86 | |
| 124 | − partition@e00000 { | |
| 125 | − label = "boot"; | |
| 126 | − reg = <0x00e00000 0x3f200000>; | |
| 87 | + nand@0 { | |
| 88 | + reg = <0>; | |
| 89 | + label = "pxa3xx_nand-0"; | |
| 90 | + nand-rb = <0>; | |
| 91 | + nand-on-flash-bbt; | |
| 92 | + | |
| 93 | + partitions { | |
| 94 | + compatible = "fixed-partitions"; | |
| 95 | + #address-cells = <1>; | |
| 96 | + #size-cells = <1>; | |
| 97 | + | |
| 98 | + partition@0 { | |
| 99 | + label = "u-boot"; | |
| 100 | + reg = <0x00000000 0x000e0000>; | |
| 101 | + read-only; | |
| 102 | + }; | |
| 103 | + | |
| 104 | + partition@e0000 { | |
| 105 | + label = "u-boot-env"; | |
| 106 | + reg = <0x000e0000 0x00020000>; | |
| 107 | + read-only; | |
| 108 | + }; | |
| 109 | + | |
| 110 | + partition@100000 { | |
| 111 | + label = "u-boot-env2"; | |
| 112 | + reg = <0x00100000 0x00020000>; | |
| 113 | + read-only; | |
| 114 | + }; | |
| 115 | + | |
| 116 | + partition@120000 { | |
| 117 | + label = "zImage"; | |
| 118 | + reg = <0x00120000 0x00400000>; | |
| 119 | + }; | |
| 120 | + | |
| 121 | + partition@520000 { | |
| 122 | + label = "initrd"; | |
| 123 | + reg = <0x00520000 0x00400000>; | |
| 124 | + }; | |
| 125 | + | |
| 126 | + partition@e00000 { | |
| 127 | + label = "boot"; | |
| 128 | + reg = <0x00e00000 0x3f200000>; | |
| 129 | + }; | |
| 127 | 130 | }; |
| 128 | 131 | }; |
| 129 | 132 | }; |
| @@ -443,7 +443,7 @@ · 442 unmodified lines | ||
| 443 | 443 | }; |
| 444 | 444 | |
| 445 | 445 | display-controller@6a000000 { |
| 446 | − status = "disabled"; | |
| 446 | + status = "okay"; | |
| 447 | 447 | |
| 448 | 448 | port@0 { |
| 449 | 449 | reg = <0>; |
| @@ -13,10 +13,25 @@ · 12 unmodified lines | ||
| 13 | 13 | stdout-path = "serial0:115200n8"; |
| 14 | 14 | }; |
| 15 | 15 | |
| 16 | − memory@80000000 { | |
| 16 | + /* | |
| 17 | + * Note that recent version of the device tree compiler (starting with | |
| 18 | + * version 1.4.2) warn about this node containing a reg property, but | |
| 19 | + * missing a unit-address. However, the bootloader on these Chromebook | |
| 20 | + * devices relies on the full name of this node to be exactly /memory. | |
| 21 | + * Adding the unit-address causes the bootloader to create a /memory | |
| 22 | + * node and write the memory bank configuration to that node, which in | |
| 23 | + * turn leads the kernel to believe that the device has 2 GiB of | |
| 24 | + * memory instead of the amount detected by the bootloader. | |
| 25 | + * | |
| 26 | + * The name of this node is effectively ABI and must not be changed. | |
| 27 | + */ | |
| 28 | + memory { | |
| 29 | + device_type = "memory"; | |
| 17 | 30 | reg = <0x0 0x80000000 0x0 0x80000000>; |
| 18 | 31 | }; |
| 19 | 32 | |
| 33 | + /delete-node/ memory@80000000; | |
| 34 | + | |
| 20 | 35 | host1x@50000000 { |
| 21 | 36 | hdmi@54280000 { |
| 22 | 37 | status = "okay"; |
| @@ -212,10 +212,11 @@ K256: · 211 unmodified lines | ||
| 212 | 212 | .global sha256_block_data_order |
| 213 | 213 | .type sha256_block_data_order,%function |
| 214 | 214 | sha256_block_data_order: |
| 215 | +.Lsha256_block_data_order: | |
| 215 | 216 | #if __ARM_ARCH__<7 |
| 216 | 217 | sub r3,pc,#8 @ sha256_block_data_order |
| 217 | 218 | #else |
| 218 | − adr r3,sha256_block_data_order | |
| 219 | + adr r3,.Lsha256_block_data_order | |
| 219 | 220 | #endif |
| 220 | 221 | #if __ARM_MAX_ARCH__>=7 && !defined(__KERNEL__) |
| 221 | 222 | ldr r12,.LOPENSSL_armcap |
| @@ -93,10 +93,11 @@ K256: · 92 unmodified lines | ||
| 93 | 93 | .global sha256_block_data_order |
| 94 | 94 | .type sha256_block_data_order,%function |
| 95 | 95 | sha256_block_data_order: |
| 96 | +.Lsha256_block_data_order: | |
| 96 | 97 | #if __ARM_ARCH__<7 |
| 97 | 98 | sub r3,pc,#8 @ sha256_block_data_order |
| 98 | 99 | #else |
| 99 | − adr r3,sha256_block_data_order | |
| 100 | + adr r3,.Lsha256_block_data_order | |
| 100 | 101 | #endif |
| 101 | 102 | #if __ARM_MAX_ARCH__>=7 && !defined(__KERNEL__) |
| 102 | 103 | ldr r12,.LOPENSSL_armcap |
| @@ -274,10 +274,11 @@ WORD64(0x5fcb6fab,0x3ad6faec, 0x6c44198c,0x4a475817) · 273 unmodified lines | ||
| 274 | 274 | .global sha512_block_data_order |
| 275 | 275 | .type sha512_block_data_order,%function |
| 276 | 276 | sha512_block_data_order: |
| 277 | +.Lsha512_block_data_order: | |
| 277 | 278 | #if __ARM_ARCH__<7 |
| 278 | 279 | sub r3,pc,#8 @ sha512_block_data_order |
| 279 | 280 | #else |
| 280 | − adr r3,sha512_block_data_order | |
| 281 | + adr r3,.Lsha512_block_data_order | |
| 281 | 282 | #endif |
| 282 | 283 | #if __ARM_MAX_ARCH__>=7 && !defined(__KERNEL__) |
| 283 | 284 | ldr r12,.LOPENSSL_armcap |
| @@ -141,10 +141,11 @@ WORD64(0x5fcb6fab,0x3ad6faec, 0x6c44198c,0x4a475817) · 140 unmodified lines | ||
| 141 | 141 | .global sha512_block_data_order |
| 142 | 142 | .type sha512_block_data_order,%function |
| 143 | 143 | sha512_block_data_order: |
| 144 | +.Lsha512_block_data_order: | |
| 144 | 145 | #if __ARM_ARCH__<7 |
| 145 | 146 | sub r3,pc,#8 @ sha512_block_data_order |
| 146 | 147 | #else |
| 147 | − adr r3,sha512_block_data_order | |
| 148 | + adr r3,.Lsha512_block_data_order | |
| 148 | 149 | #endif |
| 149 | 150 | #if __ARM_MAX_ARCH__>=7 && !defined(__KERNEL__) |
| 150 | 151 | ldr r12,.LOPENSSL_armcap |
| @@ -25,7 +25,6 @@ · 24 unmodified lines | ||
| 25 | 25 | #ifndef __ASSEMBLY__ |
| 26 | 26 | struct irqaction; |
| 27 | 27 | struct pt_regs; |
| 28 | −extern void migrate_irqs(void); | |
| 29 | 28 | |
| 30 | 29 | extern void asm_do_IRQ(unsigned int, struct pt_regs *); |
| 31 | 30 | void handle_IRQ(unsigned int, struct pt_regs *); |
| @@ -31,7 +31,6 @@ · 30 unmodified lines | ||
| 31 | 31 | #include <linux/smp.h> |
| 32 | 32 | #include <linux/init.h> |
| 33 | 33 | #include <linux/seq_file.h> |
| 34 | −#include <linux/ratelimit.h> | |
| 35 | 34 | #include <linux/errno.h> |
| 36 | 35 | #include <linux/list.h> |
| 37 | 36 | #include <linux/kallsyms.h> |
| @@ -109,64 +108,3 @@ int __init arch_probe_nr_irqs(void) · 71 unmodified lines | ||
| 109 | 108 | return nr_irqs; |
| 110 | 109 | } |
| 111 | 110 | #endif |
| 112 | − | |
| 113 | −#ifdef CONFIG_HOTPLUG_CPU | |
| 114 | −static bool migrate_one_irq(struct irq_desc *desc) | |
| 115 | −{ | |
| 116 | − struct irq_data *d = irq_desc_get_irq_data(desc); | |
| 117 | − const struct cpumask *affinity = irq_data_get_affinity_mask(d); | |
| 118 | − struct irq_chip *c; | |
| 119 | − bool ret = false; | |
| 120 | − | |
| 121 | − /* | |
| 122 | − * If this is a per-CPU interrupt, or the affinity does not | |
| 123 | − * include this CPU, then we have nothing to do. | |
| 124 | − */ | |
| 125 | − if (irqd_is_per_cpu(d) || !cpumask_test_cpu(smp_processor_id(), affinity)) | |
| 126 | − return false; | |
| 127 | − | |
| 128 | − if (cpumask_any_and(affinity, cpu_online_mask) >= nr_cpu_ids) { | |
| 129 | − affinity = cpu_online_mask; | |
| 130 | − ret = true; | |
| 131 | − } | |
| 132 | − | |
| 133 | − c = irq_data_get_irq_chip(d); | |
| 134 | − if (!c->irq_set_affinity) | |
| 135 | − pr_debug("IRQ%u: unable to set affinity\n", d->irq); | |
| 136 | − else if (c->irq_set_affinity(d, affinity, false) == IRQ_SET_MASK_OK && ret) | |
| 137 | − cpumask_copy(irq_data_get_affinity_mask(d), affinity); | |
| 138 | − | |
| 139 | − return ret; | |
| 140 | −} | |
| 141 | − | |
| 142 | −/* | |
| 143 | − * The current CPU has been marked offline. Migrate IRQs off this CPU. | |
| 144 | − * If the affinity settings do not allow other CPUs, force them onto any | |
| 145 | − * available CPU. | |
| 146 | − * | |
| 147 | − * Note: we must iterate over all IRQs, whether they have an attached | |
| 148 | − * action structure or not, as we need to get chained interrupts too. | |
| 149 | − */ | |
| 150 | −void migrate_irqs(void) | |
| 151 | −{ | |
| 152 | − unsigned int i; | |
| 153 | − struct irq_desc *desc; | |
| 154 | − unsigned long flags; | |
| 155 | − | |
| 156 | − local_irq_save(flags); | |
| 157 | − | |
| 158 | − for_each_irq_desc(i, desc) { | |
| 159 | − bool affinity_broken; | |
| 160 | − | |
| 161 | − raw_spin_lock(&desc->lock); | |
| 162 | − affinity_broken = migrate_one_irq(desc); | |
| 163 | − raw_spin_unlock(&desc->lock); | |
| 164 | − | |
| 165 | − if (affinity_broken) | |
| 166 | − pr_warn_ratelimited("IRQ%u no longer affine to CPU%u\n", | |
| 167 | − i, smp_processor_id()); | |
| 168 | − } | |
| 169 | − | |
| 170 | − local_irq_restore(flags); | |
| 171 | −} | |
| 172 | −#endif /* CONFIG_HOTPLUG_CPU */ | |
| @@ -254,7 +254,7 @@ int __cpu_disable(void) · 253 unmodified lines | ||
| 254 | 254 | /* |
| 255 | 255 | * OK - migrate IRQs away from this CPU |
| 256 | 256 | */ |
| 257 | − migrate_irqs(); | |
| 257 | + irq_migrate_all_off_this_cpu(); | |
| 258 | 258 | |
| 259 | 259 | /* |
| 260 | 260 | * Flush user cache and TLB mappings, and then remove this CPU |
| @@ -2390,4 +2390,6 @@ void arch_teardown_dma_ops(struct device *dev) · 2389 unmodified lines | ||
| 2390 | 2390 | return; |
| 2391 | 2391 | |
| 2392 | 2392 | arm_teardown_iommu_dma_ops(dev); |
| 2393 | + /* Let arch_setup_dma_ops() start again from scratch upon re-probe */ | |
| 2394 | + set_dma_ops(dev, NULL); | |
| 2393 | 2395 | } |
| @@ -247,7 +247,7 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct kprobe *or · 246 unmodified lines | ||
| 247 | 247 | } |
| 248 | 248 | |
| 249 | 249 | /* Copy arch-dep-instance from template. */ |
| 250 | − memcpy(code, (unsigned char *)optprobe_template_entry, | |
| 250 | + memcpy(code, (unsigned long *)&optprobe_template_entry, | |
| 251 | 251 | TMPL_END_IDX * sizeof(kprobe_opcode_t)); |
| 252 | 252 | |
| 253 | 253 | /* Adjust buffer according to instruction. */ |
| @@ -351,7 +351,7 @@ · 350 unmodified lines | ||
| 351 | 351 | reg = <0>; |
| 352 | 352 | pinctrl-names = "default"; |
| 353 | 353 | pinctrl-0 = <&cp0_copper_eth_phy_reset>; |
| 354 | − reset-gpios = <&cp1_gpio1 11 GPIO_ACTIVE_LOW>; | |
| 354 | + reset-gpios = <&cp0_gpio2 11 GPIO_ACTIVE_LOW>; | |
| 355 | 355 | reset-assert-us = <10000>; |
| 356 | 356 | }; |
| 357 | 357 | |
| @@ -37,7 +37,7 @@ · 36 unmodified lines | ||
| 37 | 37 | }; |
| 38 | 38 | |
| 39 | 39 | memory@86200000 { |
| 40 | − reg = <0x0 0x86200000 0x0 0x2600000>; | |
| 40 | + reg = <0x0 0x86200000 0x0 0x2d00000>; | |
| 41 | 41 | no-map; |
| 42 | 42 | }; |
| 43 | 43 | |
| @@ -158,8 +158,8 @@ ENTRY(hchacha_block_neon) · 157 unmodified lines | ||
| 158 | 158 | mov w3, w2 |
| 159 | 159 | bl chacha_permute |
| 160 | 160 | |
| 161 | − st1 {v0.16b}, [x1], #16 | |
| 162 | − st1 {v3.16b}, [x1] | |
| 161 | + st1 {v0.4s}, [x1], #16 | |
| 162 | + st1 {v3.4s}, [x1] | |
| 163 | 163 | |
| 164 | 164 | ldp x29, x30, [sp], #16 |
| 165 | 165 | ret |
| @@ -532,6 +532,10 @@ ENTRY(chacha_4block_xor_neon) · 366 unmodified lines | ||
| 532 | 532 | add v3.4s, v3.4s, v19.4s |
| 533 | 533 | add a2, a2, w8 |
| 534 | 534 | add a3, a3, w9 |
| 535 | +CPU_BE( rev a0, a0 ) | |
| 536 | +CPU_BE( rev a1, a1 ) | |
| 537 | +CPU_BE( rev a2, a2 ) | |
| 538 | +CPU_BE( rev a3, a3 ) | |
| 535 | 539 | |
| 536 | 540 | ld4r {v24.4s-v27.4s}, [x0], #16 |
| 537 | 541 | ld4r {v28.4s-v31.4s}, [x0] |
| @@ -552,6 +556,10 @@ ENTRY(chacha_4block_xor_neon) · 14 unmodified lines | ||
| 552 | 556 | add v7.4s, v7.4s, v23.4s |
| 553 | 557 | add a6, a6, w8 |
| 554 | 558 | add a7, a7, w9 |
| 559 | +CPU_BE( rev a4, a4 ) | |
| 560 | +CPU_BE( rev a5, a5 ) | |
| 561 | +CPU_BE( rev a6, a6 ) | |
| 562 | +CPU_BE( rev a7, a7 ) | |
| 555 | 563 | |
| 556 | 564 | // x8[0-3] += s2[0] |
| 557 | 565 | // x9[0-3] += s2[1] |
| @@ -569,6 +577,10 @@ ENTRY(chacha_4block_xor_neon) · 11 unmodified lines | ||
| 569 | 577 | add v11.4s, v11.4s, v27.4s |
| 570 | 578 | add a10, a10, w8 |
| 571 | 579 | add a11, a11, w9 |
| 580 | +CPU_BE( rev a8, a8 ) | |
| 581 | +CPU_BE( rev a9, a9 ) | |
| 582 | +CPU_BE( rev a10, a10 ) | |
| 583 | +CPU_BE( rev a11, a11 ) | |
| 572 | 584 | |
| 573 | 585 | // x12[0-3] += s3[0] |
| 574 | 586 | // x13[0-3] += s3[1] |
| @@ -586,6 +598,10 @@ ENTRY(chacha_4block_xor_neon) · 11 unmodified lines | ||
| 586 | 598 | add v15.4s, v15.4s, v31.4s |
| 587 | 599 | add a14, a14, w8 |
| 588 | 600 | add a15, a15, w9 |
| 601 | +CPU_BE( rev a12, a12 ) | |
| 602 | +CPU_BE( rev a13, a13 ) | |
| 603 | +CPU_BE( rev a14, a14 ) | |
| 604 | +CPU_BE( rev a15, a15 ) | |
| 589 | 605 | |
| 590 | 606 | // interleave 32-bit words in state n, n+1 |
| 591 | 607 | ldp w6, w7, [x2], #64 |
| @@ -36,4 +36,8 @@ · 35 unmodified lines | ||
| 36 | 36 | #include <arm_neon.h> |
| 37 | 37 | #endif |
| 38 | 38 | |
| 39 | +#ifdef CONFIG_CC_IS_CLANG | |
| 40 | +#pragma clang diagnostic ignored "-Wincompatible-pointer-types" | |
| 41 | +#endif | |
| 42 | + | |
| 39 | 43 | #endif /* __ASM_NEON_INTRINSICS_H */ |
| @@ -539,8 +539,7 @@ set_hcr: · 538 unmodified lines | ||
| 539 | 539 | /* GICv3 system register access */ |
| 540 | 540 | mrs x0, id_aa64pfr0_el1 |
| 541 | 541 | ubfx x0, x0, #24, #4 |
| 542 | − cmp x0, #1 | |
| 543 | − b.ne 3f | |
| 542 | + cbz x0, 3f | |
| 544 | 543 | |
| 545 | 544 | mrs_s x0, SYS_ICC_SRE_EL2 |
| 546 | 545 | orr x0, x0, #ICC_SRE_EL2_SRE // Set ICC_SRE_EL2.SRE==1 |
| @@ -1702,19 +1702,20 @@ void syscall_trace_exit(struct pt_regs *regs) · 1701 unmodified lines | ||
| 1702 | 1702 | } |
| 1703 | 1703 | |
| 1704 | 1704 | /* |
| 1705 | − * SPSR_ELx bits which are always architecturally RES0 per ARM DDI 0487C.a | |
| 1706 | − * We also take into account DIT (bit 24), which is not yet documented, and | |
| 1707 | − * treat PAN and UAO as RES0 bits, as they are meaningless at EL0, and may be | |
| 1708 | − * allocated an EL0 meaning in future. | |
| 1705 | + * SPSR_ELx bits which are always architecturally RES0 per ARM DDI 0487D.a. | |
| 1706 | + * We permit userspace to set SSBS (AArch64 bit 12, AArch32 bit 23) which is | |
| 1707 | + * not described in ARM DDI 0487D.a. | |
| 1708 | + * We treat PAN and UAO as RES0 bits, as they are meaningless at EL0, and may | |
| 1709 | + * be allocated an EL0 meaning in future. | |
| 1709 | 1710 | * Userspace cannot use these until they have an architectural meaning. |
| 1710 | 1711 | * Note that this follows the SPSR_ELx format, not the AArch32 PSR format. |
| 1711 | 1712 | * We also reserve IL for the kernel; SS is handled dynamically. |
| 1712 | 1713 | */ |
| 1713 | 1714 | #define SPSR_EL1_AARCH64_RES0_BITS \ |
| 1714 | − (GENMASK_ULL(63,32) | GENMASK_ULL(27, 25) | GENMASK_ULL(23, 22) | \ | |
| 1715 | − GENMASK_ULL(20, 10) | GENMASK_ULL(5, 5)) | |
| 1715 | + (GENMASK_ULL(63, 32) | GENMASK_ULL(27, 25) | GENMASK_ULL(23, 22) | \ | |
| 1716 | + GENMASK_ULL(20, 13) | GENMASK_ULL(11, 10) | GENMASK_ULL(5, 5)) | |
| 1716 | 1717 | #define SPSR_EL1_AARCH32_RES0_BITS \ |
| 1717 | − (GENMASK_ULL(63,32) | GENMASK_ULL(23, 22) | GENMASK_ULL(20,20)) | |
| 1718 | + (GENMASK_ULL(63, 32) | GENMASK_ULL(22, 22) | GENMASK_ULL(20, 20)) | |
| 1718 | 1719 | |
| 1719 | 1720 | static int valid_compat_regs(struct user_pt_regs *regs) |
| 1720 | 1721 | { |
| @@ -339,6 +339,9 @@ void __init setup_arch(char **cmdline_p) · 338 unmodified lines | ||
| 339 | 339 | smp_init_cpus(); |
| 340 | 340 | smp_build_mpidr_hash(); |
| 341 | 341 | |
| 342 | + /* Init percpu seeds for random tags after cpus are set up. */ | |
| 343 | + kasan_init_tags(); | |
| 344 | + | |
| 342 | 345 | #ifdef CONFIG_ARM64_SW_TTBR0_PAN |
| 343 | 346 | /* |
| 344 | 347 | * Make sure init_thread_info.ttbr0 always generates translation |
| @@ -252,8 +252,6 @@ void __init kasan_init(void) · 251 unmodified lines | ||
| 252 | 252 | memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE); |
| 253 | 253 | cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); |
| 254 | 254 | |
| 255 | − kasan_init_tags(); | |
| 256 | − | |
| 257 | 255 | /* At this point kasan is fully initialized. Enable error messages */ |
| 258 | 256 | init_task.kasan_depth = 0; |
| 259 | 257 | pr_info("KernelAddressSanitizer initialized\n"); |
| @@ -70,6 +70,8 @@ static struct platform_device bcm63xx_enet_shared_device = { · 69 unmodified lines | ||
| 70 | 70 | |
| 71 | 71 | static int shared_device_registered; |
| 72 | 72 | |
| 73 | +static u64 enet_dmamask = DMA_BIT_MASK(32); | |
| 74 | + | |
| 73 | 75 | static struct resource enet0_res[] = { |
| 74 | 76 | { |
| 75 | 77 | .start = -1, /* filled at runtime */ |
| @@ -99,6 +101,8 @@ static struct platform_device bcm63xx_enet0_device = { · 23 unmodified lines | ||
| 99 | 101 | .resource = enet0_res, |
| 100 | 102 | .dev = { |
| 101 | 103 | .platform_data = &enet0_pd, |
| 104 | + .dma_mask = &enet_dmamask, | |
| 105 | + .coherent_dma_mask = DMA_BIT_MASK(32), | |
| 102 | 106 | }, |
| 103 | 107 | }; |
| 104 | 108 | |
| @@ -131,6 +135,8 @@ static struct platform_device bcm63xx_enet1_device = { · 26 unmodified lines | ||
| 131 | 135 | .resource = enet1_res, |
| 132 | 136 | .dev = { |
| 133 | 137 | .platform_data = &enet1_pd, |
| 138 | + .dma_mask = &enet_dmamask, | |
| 139 | + .coherent_dma_mask = DMA_BIT_MASK(32), | |
| 134 | 140 | }, |
| 135 | 141 | }; |
| 136 | 142 | |
| @@ -157,6 +163,8 @@ static struct platform_device bcm63xx_enetsw_device = { · 20 unmodified lines | ||
| 157 | 163 | .resource = enetsw_res, |
| 158 | 164 | .dev = { |
| 159 | 165 | .platform_data = &enetsw_pd, |
| 166 | + .dma_mask = &enet_dmamask, | |
| 167 | + .coherent_dma_mask = DMA_BIT_MASK(32), | |
| 160 | 168 | }, |
| 161 | 169 | }; |
| 162 | 170 | |
| @@ -54,10 +54,9 @@ unsigned long __xchg_small(volatile void *ptr, unsigned long val, unsigned int s · 53 unmodified lines | ||
| 54 | 54 | unsigned long __cmpxchg_small(volatile void *ptr, unsigned long old, |
| 55 | 55 | unsigned long new, unsigned int size) |
| 56 | 56 | { |
| 57 | − u32 mask, old32, new32, load32; | |
| 57 | + u32 mask, old32, new32, load32, load; | |
| 58 | 58 | volatile u32 *ptr32; |
| 59 | 59 | unsigned int shift; |
| 60 | − u8 load; | |
| 61 | 60 | |
| 62 | 61 | /* Check that ptr is naturally aligned */ |
| 63 | 62 | WARN_ON((unsigned long)ptr & (size - 1)); |
| @@ -384,7 +384,8 @@ static void __init bootmem_init(void) · 383 unmodified lines | ||
| 384 | 384 | init_initrd(); |
| 385 | 385 | reserved_end = (unsigned long) PFN_UP(__pa_symbol(&_end)); |
| 386 | 386 | |
| 387 | − memblock_reserve(PHYS_OFFSET, reserved_end << PAGE_SHIFT); | |
| 387 | + memblock_reserve(PHYS_OFFSET, | |
| 388 | + (reserved_end << PAGE_SHIFT) - PHYS_OFFSET); | |
| 388 | 389 | |
| 389 | 390 | /* |
| 390 | 391 | * max_low_pfn is not a number of pages. The number of pages |
| @@ -31,8 +31,8 @@ static int vmmc_probe(struct platform_device *pdev) · 30 unmodified lines | ||
| 31 | 31 | dma_addr_t dma; |
| 32 | 32 | |
| 33 | 33 | cp1_base = |
| 34 | − (void *) CPHYSADDR(dma_alloc_coherent(NULL, CP1_SIZE, | |
| 35 | − &dma, GFP_ATOMIC)); | |
| 34 | + (void *) CPHYSADDR(dma_alloc_coherent(&pdev->dev, CP1_SIZE, | |
| 35 | + &dma, GFP_KERNEL)); | |
| 36 | 36 | |
| 37 | 37 | gpio_count = of_gpio_count(pdev->dev.of_node); |
| 38 | 38 | while (gpio_count > 0) { |
| @@ -79,8 +79,6 @@ enum reg_val_type { · 78 unmodified lines | ||
| 79 | 79 | REG_64BIT_32BIT, |
| 80 | 80 | /* 32-bit compatible, need truncation for 64-bit ops. */ |
| 81 | 81 | REG_32BIT, |
| 82 | − /* 32-bit zero extended. */ | |
| 83 | − REG_32BIT_ZERO_EX, | |
| 84 | 82 | /* 32-bit no sign/zero extension needed. */ |
| 85 | 83 | REG_32BIT_POS |
| 86 | 84 | }; |
| @@ -343,12 +341,15 @@ static int build_int_epilogue(struct jit_ctx *ctx, int dest_reg) · 256 unmodified lines | ||
| 343 | 341 | const struct bpf_prog *prog = ctx->skf; |
| 344 | 342 | int stack_adjust = ctx->stack_size; |
| 345 | 343 | int store_offset = stack_adjust - 8; |
| 344 | + enum reg_val_type td; | |
| 346 | 345 | int r0 = MIPS_R_V0; |
| 347 | 346 | |
| 348 | − if (dest_reg == MIPS_R_RA && | |
| 349 | − get_reg_val_type(ctx, prog->len, BPF_REG_0) == REG_32BIT_ZERO_EX) | |
| 347 | + if (dest_reg == MIPS_R_RA) { | |
| 350 | 348 | /* Don't let zero extended value escape. */ |
| 351 | − emit_instr(ctx, sll, r0, r0, 0); | |
| 349 | + td = get_reg_val_type(ctx, prog->len, BPF_REG_0); | |
| 350 | + if (td == REG_64BIT) | |
| 351 | + emit_instr(ctx, sll, r0, r0, 0); | |
| 352 | + } | |
| 352 | 353 | |
| 353 | 354 | if (ctx->flags & EBPF_SAVE_RA) { |
| 354 | 355 | emit_instr(ctx, ld, MIPS_R_RA, store_offset, MIPS_R_SP); |
| @@ -692,7 +693,7 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, · 337 unmodified lines | ||
| 692 | 693 | if (dst < 0) |
| 693 | 694 | return dst; |
| 694 | 695 | td = get_reg_val_type(ctx, this_idx, insn->dst_reg); |
| 695 | − if (td == REG_64BIT || td == REG_32BIT_ZERO_EX) { | |
| 696 | + if (td == REG_64BIT) { | |
| 696 | 697 | /* sign extend */ |
| 697 | 698 | emit_instr(ctx, sll, dst, dst, 0); |
| 698 | 699 | } |
| @@ -707,7 +708,7 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, · 8 unmodified lines | ||
| 707 | 708 | if (dst < 0) |
| 708 | 709 | return dst; |
| 709 | 710 | td = get_reg_val_type(ctx, this_idx, insn->dst_reg); |
| 710 | − if (td == REG_64BIT || td == REG_32BIT_ZERO_EX) { | |
| 711 | + if (td == REG_64BIT) { | |
| 711 | 712 | /* sign extend */ |
| 712 | 713 | emit_instr(ctx, sll, dst, dst, 0); |
| 713 | 714 | } |
| @@ -721,7 +722,7 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, · 7 unmodified lines | ||
| 721 | 722 | if (dst < 0) |
| 722 | 723 | return dst; |
| 723 | 724 | td = get_reg_val_type(ctx, this_idx, insn->dst_reg); |
| 724 | − if (td == REG_64BIT || td == REG_32BIT_ZERO_EX) | |
| 725 | + if (td == REG_64BIT) | |
| 725 | 726 | /* sign extend */ |
| 726 | 727 | emit_instr(ctx, sll, dst, dst, 0); |
| 727 | 728 | if (insn->imm == 1) { |
| @@ -860,13 +861,13 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, · 132 unmodified lines | ||
| 860 | 861 | if (src < 0 || dst < 0) |
| 861 | 862 | return -EINVAL; |
| 862 | 863 | td = get_reg_val_type(ctx, this_idx, insn->dst_reg); |
| 863 | − if (td == REG_64BIT || td == REG_32BIT_ZERO_EX) { | |
| 864 | + if (td == REG_64BIT) { | |
| 864 | 865 | /* sign extend */ |
| 865 | 866 | emit_instr(ctx, sll, dst, dst, 0); |
| 866 | 867 | } |
| 867 | 868 | did_move = false; |
| 868 | 869 | ts = get_reg_val_type(ctx, this_idx, insn->src_reg); |
| 869 | − if (ts == REG_64BIT || ts == REG_32BIT_ZERO_EX) { | |
| 870 | + if (ts == REG_64BIT) { | |
| 870 | 871 | int tmp_reg = MIPS_R_AT; |
| 871 | 872 | |
| 872 | 873 | if (bpf_op == BPF_MOV) { |
| @@ -1254,8 +1255,7 @@ jeq_common: · 381 unmodified lines | ||
| 1254 | 1255 | if (insn->imm == 64 && td == REG_32BIT) |
| 1255 | 1256 | emit_instr(ctx, dinsu, dst, MIPS_R_ZERO, 32, 32); |
| 1256 | 1257 | |
| 1257 | − if (insn->imm != 64 && | |
| 1258 | − (td == REG_64BIT || td == REG_32BIT_ZERO_EX)) { | |
| 1258 | + if (insn->imm != 64 && td == REG_64BIT) { | |
| 1259 | 1259 | /* sign extend */ |
| 1260 | 1260 | emit_instr(ctx, sll, dst, dst, 0); |
| 1261 | 1261 | } |
| @@ -1819,7 +1819,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) · 557 unmodified lines | ||
| 1819 | 1819 | |
| 1820 | 1820 | /* Update the icache */ |
| 1821 | 1821 | flush_icache_range((unsigned long)ctx.target, |
| 1822 | − (unsigned long)(ctx.target + ctx.idx * sizeof(u32))); | |
| 1822 | + (unsigned long)&ctx.target[ctx.idx]); | |
| 1823 | 1823 | |
| 1824 | 1824 | if (bpf_jit_enable > 1) |
| 1825 | 1825 | /* Dump JIT code */ |
| @@ -308,15 +308,29 @@ long compat_arch_ptrace(struct task_struct *child, compat_long_t request, · 307 unmodified lines | ||
| 308 | 308 | |
| 309 | 309 | long do_syscall_trace_enter(struct pt_regs *regs) |
| 310 | 310 | { |
| 311 | − if (test_thread_flag(TIF_SYSCALL_TRACE) && | |
| 312 | − tracehook_report_syscall_entry(regs)) { | |
| 311 | + if (test_thread_flag(TIF_SYSCALL_TRACE)) { | |
| 312 | + int rc = tracehook_report_syscall_entry(regs); | |
| 313 | + | |
| 313 | 314 | /* |
| 314 | − * Tracing decided this syscall should not happen or the | |
| 315 | − * debugger stored an invalid system call number. Skip | |
| 316 | − * the system call and the system call restart handling. | |
| 315 | + * As tracesys_next does not set %r28 to -ENOSYS | |
| 316 | + * when %r20 is set to -1, initialize it here. | |
| 317 | 317 | */ |
| 318 | − regs->gr[20] = -1UL; | |
| 319 | − goto out; | |
| 318 | + regs->gr[28] = -ENOSYS; | |
| 319 | + | |
| 320 | + if (rc) { | |
| 321 | + /* | |
| 322 | + * A nonzero return code from | |
| 323 | + * tracehook_report_syscall_entry() tells us | |
| 324 | + * to prevent the syscall execution. Skip | |
| 325 | + * the syscall call and the syscall restart handling. | |
| 326 | + * | |
| 327 | + * Note that the tracer may also just change | |
| 328 | + * regs->gr[20] to an invalid syscall number, | |
| 329 | + * that is handled by tracesys_next. | |
| 330 | + */ | |
| 331 | + regs->gr[20] = -1UL; | |
| 332 | + return -1; | |
| 333 | + } | |
| 320 | 334 | } |
| 321 | 335 | |
| 322 | 336 | /* Do the secure computing check after ptrace. */ |
| @@ -340,7 +354,6 @@ long do_syscall_trace_enter(struct pt_regs *regs) · 17 unmodified lines | ||
| 340 | 354 | regs->gr[24] & 0xffffffff, |
| 341 | 355 | regs->gr[23] & 0xffffffff); |
| 342 | 356 | |
| 343 | −out: | |
| 344 | 357 | /* |
| 345 | 358 | * Sign extend the syscall number to 64bit since it may have been |
| 346 | 359 | * modified by a compat ptrace call |
| @@ -1593,6 +1593,8 @@ static void pnv_ioda_setup_vf_PE(struct pci_dev *pdev, u16 num_vfs) · 1592 unmodified lines | ||
| 1593 | 1593 | |
| 1594 | 1594 | pnv_pci_ioda2_setup_dma_pe(phb, pe); |
| 1595 | 1595 | #ifdef CONFIG_IOMMU_API |
| 1596 | + iommu_register_group(&pe->table_group, | |
| 1597 | + pe->phb->hose->global_number, pe->pe_number); | |
| 1596 | 1598 | pnv_ioda_setup_bus_iommu_group(pe, &pe->table_group, NULL); |
| 1597 | 1599 | #endif |
| 1598 | 1600 | } |
| @@ -1147,6 +1147,8 @@ static int pnv_tce_iommu_bus_notifier(struct notifier_block *nb, · 1146 unmodified lines | ||
| 1147 | 1147 | return 0; |
| 1148 | 1148 | |
| 1149 | 1149 | pe = &phb->ioda.pe_array[pdn->pe_number]; |
| 1150 | + if (!pe->table_group.group) | |
| 1151 | + return 0; | |
| 1150 | 1152 | iommu_add_device(&pe->table_group, dev); |
| 1151 | 1153 | return 0; |
| 1152 | 1154 | case BUS_NOTIFY_DEL_DEVICE: |
| @@ -297,7 +297,7 @@ static int shadow_crycb(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page) · 296 unmodified lines | ||
| 297 | 297 | scb_s->crycbd = 0; |
| 298 | 298 | |
| 299 | 299 | apie_h = vcpu->arch.sie_block->eca & ECA_APIE; |
| 300 | − if (!apie_h && !key_msk) | |
| 300 | + if (!apie_h && (!key_msk || fmt_o == CRYCB_FORMAT0)) | |
| 301 | 301 | return 0; |
| 302 | 302 | |
| 303 | 303 | if (!crycb_addr) |
| @@ -1,3 +1,3 @@ | ||
| 1 | 1 | ifneq ($(CONFIG_BUILTIN_DTB_SOURCE),"") |
| 2 | −obj-y += $(patsubst "%",%,$(CONFIG_BUILTIN_DTB_SOURCE)).dtb.o | |
| 2 | +obj-$(CONFIG_USE_BUILTIN_DTB) += $(patsubst "%",%,$(CONFIG_BUILTIN_DTB_SOURCE)).dtb.o | |
| 3 | 3 | endif |
| @@ -841,7 +841,7 @@ union hv_gpa_page_range { · 840 unmodified lines | ||
| 841 | 841 | * count is equal with how many entries of union hv_gpa_page_range can |
| 842 | 842 | * be populated into the input parameter page. |
| 843 | 843 | */ |
| 844 | −#define HV_MAX_FLUSH_REP_COUNT (PAGE_SIZE - 2 * sizeof(u64) / \ | |
| 844 | +#define HV_MAX_FLUSH_REP_COUNT ((PAGE_SIZE - 2 * sizeof(u64)) / \ | |
| 845 | 845 | sizeof(union hv_gpa_page_range)) |
| 846 | 846 | |
| 847 | 847 | struct hv_guest_mapping_flush_list { |
| @@ -299,6 +299,7 @@ union kvm_mmu_extended_role { · 298 unmodified lines | ||
| 299 | 299 | unsigned int cr4_smap:1; |
| 300 | 300 | unsigned int cr4_smep:1; |
| 301 | 301 | unsigned int cr4_la57:1; |
| 302 | + unsigned int maxphyaddr:6; | |
| 302 | 303 | }; |
| 303 | 304 | }; |
| 304 | 305 | |
| @@ -397,6 +398,7 @@ struct kvm_mmu { · 92 unmodified lines | ||
| 397 | 398 | void (*update_pte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, |
| 398 | 399 | u64 *spte, const void *pte); |
| 399 | 400 | hpa_t root_hpa; |
| 401 | + gpa_t root_cr3; | |
| 400 | 402 | union kvm_mmu_role mmu_role; |
| 401 | 403 | u8 root_level; |
| 402 | 404 | u8 shadow_root_level; |
| @@ -284,7 +284,7 @@ do { \ · 283 unmodified lines | ||
| 284 | 284 | __put_user_goto(x, ptr, "l", "k", "ir", label); \ |
| 285 | 285 | break; \ |
| 286 | 286 | case 8: \ |
| 287 | − __put_user_goto_u64((__typeof__(*ptr))(x), ptr, label); \ | |
| 287 | + __put_user_goto_u64(x, ptr, label); \ | |
| 288 | 288 | break; \ |
| 289 | 289 | default: \ |
| 290 | 290 | __put_user_bad(); \ |
| @@ -431,8 +431,10 @@ do { \ · 140 unmodified lines | ||
| 431 | 431 | ({ \ |
| 432 | 432 | __label__ __pu_label; \ |
| 433 | 433 | int __pu_err = -EFAULT; \ |
| 434 | + __typeof__(*(ptr)) __pu_val; \ | |
| 435 | + __pu_val = x; \ | |
| 434 | 436 | __uaccess_begin(); \ |
| 435 | − __put_user_size((x), (ptr), (size), __pu_label); \ | |
| 437 | + __put_user_size(__pu_val, (ptr), (size), __pu_label); \ | |
| 436 | 438 | __pu_err = 0; \ |
| 437 | 439 | __pu_label: \ |
| 438 | 440 | __uaccess_end(); \ |
| @@ -335,6 +335,7 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function, · 334 unmodified lines | ||
| 335 | 335 | unsigned f_xsaves = kvm_x86_ops->xsaves_supported() ? F(XSAVES) : 0; |
| 336 | 336 | unsigned f_umip = kvm_x86_ops->umip_emulated() ? F(UMIP) : 0; |
| 337 | 337 | unsigned f_intel_pt = kvm_x86_ops->pt_supported() ? F(INTEL_PT) : 0; |
| 338 | + unsigned f_la57 = 0; | |
| 338 | 339 | |
| 339 | 340 | /* cpuid 1.edx */ |
| 340 | 341 | const u32 kvm_cpuid_1_edx_x86_features = |
| @@ -489,7 +490,10 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function, · 148 unmodified lines | ||
| 489 | 490 | // TSC_ADJUST is emulated |
| 490 | 491 | entry->ebx |= F(TSC_ADJUST); |
| 491 | 492 | entry->ecx &= kvm_cpuid_7_0_ecx_x86_features; |
| 493 | + f_la57 = entry->ecx & F(LA57); | |
| 492 | 494 | cpuid_mask(&entry->ecx, CPUID_7_ECX); |
| 495 | + /* Set LA57 based on hardware capability. */ | |
| 496 | + entry->ecx |= f_la57; | |
| 493 | 497 | entry->ecx |= f_umip; |
| 494 | 498 | /* PKU is not yet implemented for shadow paging. */ |
| 495 | 499 | if (!tdp_enabled || !boot_cpu_has(X86_FEATURE_OSPKE)) |
| @@ -3555,6 +3555,7 @@ void kvm_mmu_free_roots(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, · 3554 unmodified lines | ||
| 3555 | 3555 | &invalid_list); |
| 3556 | 3556 | mmu->root_hpa = INVALID_PAGE; |
| 3557 | 3557 | } |
| 3558 | + mmu->root_cr3 = 0; | |
| 3558 | 3559 | } |
| 3559 | 3560 | |
| 3560 | 3561 | kvm_mmu_commit_zap_page(vcpu->kvm, &invalid_list); |
| @@ -3610,6 +3611,7 @@ static int mmu_alloc_direct_roots(struct kvm_vcpu *vcpu) · 49 unmodified lines | ||
| 3610 | 3611 | vcpu->arch.mmu->root_hpa = __pa(vcpu->arch.mmu->pae_root); |
| 3611 | 3612 | } else |
| 3612 | 3613 | BUG(); |
| 3614 | + vcpu->arch.mmu->root_cr3 = vcpu->arch.mmu->get_cr3(vcpu); | |
| 3613 | 3615 | |
| 3614 | 3616 | return 0; |
| 3615 | 3617 | } |
| @@ -3618,10 +3620,11 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) · 2 unmodified lines | ||
| 3618 | 3620 | { |
| 3619 | 3621 | struct kvm_mmu_page *sp; |
| 3620 | 3622 | u64 pdptr, pm_mask; |
| 3621 | − gfn_t root_gfn; | |
| 3623 | + gfn_t root_gfn, root_cr3; | |
| 3622 | 3624 | int i; |
| 3623 | 3625 | |
| 3624 | − root_gfn = vcpu->arch.mmu->get_cr3(vcpu) >> PAGE_SHIFT; | |
| 3626 | + root_cr3 = vcpu->arch.mmu->get_cr3(vcpu); | |
| 3627 | + root_gfn = root_cr3 >> PAGE_SHIFT; | |
| 3625 | 3628 | |
| 3626 | 3629 | if (mmu_check_root(vcpu, root_gfn)) |
| 3627 | 3630 | return 1; |
| @@ -3646,7 +3649,7 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) · 18 unmodified lines | ||
| 3646 | 3649 | ++sp->root_count; |
| 3647 | 3650 | spin_unlock(&vcpu->kvm->mmu_lock); |
| 3648 | 3651 | vcpu->arch.mmu->root_hpa = root; |
| 3649 | − return 0; | |
| 3652 | + goto set_root_cr3; | |
| 3650 | 3653 | } |
| 3651 | 3654 | |
| 3652 | 3655 | /* |
| @@ -3712,6 +3715,9 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) · 59 unmodified lines | ||
| 3712 | 3715 | vcpu->arch.mmu->root_hpa = __pa(vcpu->arch.mmu->lm_root); |
| 3713 | 3716 | } |
| 3714 | 3717 | |
| 3718 | +set_root_cr3: | |
| 3719 | + vcpu->arch.mmu->root_cr3 = root_cr3; | |
| 3720 | + | |
| 3715 | 3721 | return 0; |
| 3716 | 3722 | } |
| 3717 | 3723 | |
| @@ -4163,7 +4169,7 @@ static bool cached_root_available(struct kvm_vcpu *vcpu, gpa_t new_cr3, · 445 unmodified lines | ||
| 4163 | 4169 | struct kvm_mmu_root_info root; |
| 4164 | 4170 | struct kvm_mmu *mmu = vcpu->arch.mmu; |
| 4165 | 4171 | |
| 4166 | − root.cr3 = mmu->get_cr3(vcpu); | |
| 4172 | + root.cr3 = mmu->root_cr3; | |
| 4167 | 4173 | root.hpa = mmu->root_hpa; |
| 4168 | 4174 | |
| 4169 | 4175 | for (i = 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) { |
| @@ -4176,6 +4182,7 @@ static bool cached_root_available(struct kvm_vcpu *vcpu, gpa_t new_cr3, · 6 unmodified lines | ||
| 4176 | 4182 | } |
| 4177 | 4183 | |
| 4178 | 4184 | mmu->root_hpa = root.hpa; |
| 4185 | + mmu->root_cr3 = root.cr3; | |
| 4179 | 4186 | |
| 4180 | 4187 | return i < KVM_MMU_NUM_PREV_ROOTS; |
| 4181 | 4188 | } |
| @@ -4770,6 +4777,7 @@ static union kvm_mmu_extended_role kvm_calc_mmu_role_ext(struct kvm_vcpu *vcpu) · 588 unmodified lines | ||
| 4770 | 4777 | ext.cr4_pse = !!is_pse(vcpu); |
| 4771 | 4778 | ext.cr4_pke = !!kvm_read_cr4_bits(vcpu, X86_CR4_PKE); |
| 4772 | 4779 | ext.cr4_la57 = !!kvm_read_cr4_bits(vcpu, X86_CR4_LA57); |
| 4780 | + ext.maxphyaddr = cpuid_maxphyaddr(vcpu); | |
| 4773 | 4781 | |
| 4774 | 4782 | ext.valid = 1; |
| 4775 | 4783 | |
| @@ -5516,11 +5524,13 @@ int kvm_mmu_create(struct kvm_vcpu *vcpu) · 740 unmodified lines | ||
| 5516 | 5524 | vcpu->arch.walk_mmu = &vcpu->arch.root_mmu; |
| 5517 | 5525 | |
| 5518 | 5526 | vcpu->arch.root_mmu.root_hpa = INVALID_PAGE; |
| 5527 | + vcpu->arch.root_mmu.root_cr3 = 0; | |
| 5519 | 5528 | vcpu->arch.root_mmu.translate_gpa = translate_gpa; |
| 5520 | 5529 | for (i = 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) |
| 5521 | 5530 | vcpu->arch.root_mmu.prev_roots[i] = KVM_MMU_ROOT_INFO_INVALID; |
| 5522 | 5531 | |
| 5523 | 5532 | vcpu->arch.guest_mmu.root_hpa = INVALID_PAGE; |
| 5533 | + vcpu->arch.guest_mmu.root_cr3 = 0; | |
| 5524 | 5534 | vcpu->arch.guest_mmu.translate_gpa = translate_gpa; |
| 5525 | 5535 | for (i = 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) |
| 5526 | 5536 | vcpu->arch.guest_mmu.prev_roots[i] = KVM_MMU_ROOT_INFO_INVALID; |
| @@ -117,67 +117,11 @@ __visible bool ex_handler_fprestore(const struct exception_table_entry *fixup, · 116 unmodified lines | ||
| 117 | 117 | } |
| 118 | 118 | EXPORT_SYMBOL_GPL(ex_handler_fprestore); |
| 119 | 119 | |
| 120 | −/* Helper to check whether a uaccess fault indicates a kernel bug. */ | |
| 121 | −static bool bogus_uaccess(struct pt_regs *regs, int trapnr, | |
| 122 | − unsigned long fault_addr) | |
| 123 | −{ | |
| 124 | − /* This is the normal case: #PF with a fault address in userspace. */ | |
| 125 | − if (trapnr == X86_TRAP_PF && fault_addr < TASK_SIZE_MAX) | |
| 126 | − return false; | |
| 127 | − | |
| 128 | − /* | |
| 129 | − * This code can be reached for machine checks, but only if the #MC | |
| 130 | − * handler has already decided that it looks like a candidate for fixup. | |
| 131 | − * This e.g. happens when attempting to access userspace memory which | |
| 132 | − * the CPU can't access because of uncorrectable bad memory. | |
| 133 | − */ | |
| 134 | − if (trapnr == X86_TRAP_MC) | |
| 135 | − return false; | |
| 136 | − | |
| 137 | − /* | |
| 138 | − * There are two remaining exception types we might encounter here: | |
| 139 | − * - #PF for faulting accesses to kernel addresses | |
| 140 | − * - #GP for faulting accesses to noncanonical addresses | |
| 141 | − * Complain about anything else. | |
| 142 | − */ | |
| 143 | − if (trapnr != X86_TRAP_PF && trapnr != X86_TRAP_GP) { | |
| 144 | − WARN(1, "unexpected trap %d in uaccess\n", trapnr); | |
| 145 | − return false; | |
| 146 | − } | |
| 147 | − | |
| 148 | − /* | |
| 149 | − * This is a faulting memory access in kernel space, on a kernel | |
| 150 | − * address, in a usercopy function. This can e.g. be caused by improper | |
| 151 | − * use of helpers like __put_user and by improper attempts to access | |
| 152 | − * userspace addresses in KERNEL_DS regions. | |
| 153 | − * The one (semi-)legitimate exception are probe_kernel_{read,write}(), | |
| 154 | − * which can be invoked from places like kgdb, /dev/mem (for reading) | |
| 155 | − * and privileged BPF code (for reading). | |
| 156 | − * The probe_kernel_*() functions set the kernel_uaccess_faults_ok flag | |
| 157 | − * to tell us that faulting on kernel addresses, and even noncanonical | |
| 158 | − * addresses, in a userspace accessor does not necessarily imply a | |
| 159 | − * kernel bug, root might just be doing weird stuff. | |
| 160 | − */ | |
| 161 | − if (current->kernel_uaccess_faults_ok) | |
| 162 | − return false; | |
| 163 | − | |
| 164 | − /* This is bad. Refuse the fixup so that we go into die(). */ | |
| 165 | − if (trapnr == X86_TRAP_PF) { | |
| 166 | − pr_emerg("BUG: pagefault on kernel address 0x%lx in non-whitelisted uaccess\n", | |
| 167 | − fault_addr); | |
| 168 | − } else { | |
| 169 | − pr_emerg("BUG: GPF in non-whitelisted uaccess (non-canonical address?)\n"); | |
| 170 | − } | |
| 171 | − return true; | |
| 172 | −} | |
| 173 | − | |
| 174 | 120 | __visible bool ex_handler_uaccess(const struct exception_table_entry *fixup, |
| 175 | 121 | struct pt_regs *regs, int trapnr, |
| 176 | 122 | unsigned long error_code, |
| 177 | 123 | unsigned long fault_addr) |
| 178 | 124 | { |
| 179 | − if (bogus_uaccess(regs, trapnr, fault_addr)) | |
| 180 | − return false; | |
| 181 | 125 | regs->ip = ex_fixup_addr(fixup); |
| 182 | 126 | return true; |
| 183 | 127 | } |
| @@ -188,8 +132,6 @@ __visible bool ex_handler_ext(const struct exception_table_entry *fixup, · 4 unmodified lines | ||
| 188 | 132 | unsigned long error_code, |
| 189 | 133 | unsigned long fault_addr) |
| 190 | 134 | { |
| 191 | − if (bogus_uaccess(regs, trapnr, fault_addr)) | |
| 192 | − return false; | |
| 193 | 135 | /* Special hack for uaccess_err */ |
| 194 | 136 | current->thread.uaccess_err = 1; |
| 195 | 137 | regs->ip = ex_fixup_addr(fixup); |
| @@ -122,8 +122,10 @@ static void alg_do_release(const struct af_alg_type *type, void *private) · 121 unmodified lines | ||
| 122 | 122 | |
| 123 | 123 | int af_alg_release(struct socket *sock) |
| 124 | 124 | { |
| 125 | − if (sock->sk) | |
| 125 | + if (sock->sk) { | |
| 126 | 126 | sock_put(sock->sk); |
| 127 | + sock->sk = NULL; | |
| 128 | + } | |
| 127 | 129 | return 0; |
| 128 | 130 | } |
| 129 | 131 | EXPORT_SYMBOL_GPL(af_alg_release); |
| @@ -95,7 +95,7 @@ static void __update_runtime_status(struct device *dev, enum rpm_status status) · 94 unmodified lines | ||
| 95 | 95 | static void pm_runtime_deactivate_timer(struct device *dev) |
| 96 | 96 | { |
| 97 | 97 | if (dev->power.timer_expires > 0) { |
| 98 | − hrtimer_cancel(&dev->power.suspend_timer); | |
| 98 | + hrtimer_try_to_cancel(&dev->power.suspend_timer); | |
| 99 | 99 | dev->power.timer_expires = 0; |
| 100 | 100 | } |
| 101 | 101 | } |
| @@ -144,8 +144,7 @@ static void __init at91sam9x5_pmc_setup(struct device_node *np, · 143 unmodified lines | ||
| 144 | 144 | return; |
| 145 | 145 | |
| 146 | 146 | at91sam9x5_pmc = pmc_data_allocate(PMC_MAIN + 1, |
| 147 | − nck(at91sam9x5_systemck), | |
| 148 | − nck(at91sam9x35_periphck), 0); | |
| 147 | + nck(at91sam9x5_systemck), 31, 0); | |
| 149 | 148 | if (!at91sam9x5_pmc) |
| 150 | 149 | return; |
| 151 | 150 | |
| @@ -210,7 +209,7 @@ static void __init at91sam9x5_pmc_setup(struct device_node *np, · 58 unmodified lines | ||
| 210 | 209 | parent_names[1] = "mainck"; |
| 211 | 210 | parent_names[2] = "plladivck"; |
| 212 | 211 | parent_names[3] = "utmick"; |
| 213 | − parent_names[4] = "mck"; | |
| 212 | + parent_names[4] = "masterck"; | |
| 214 | 213 | for (i = 0; i < 2; i++) { |
| 215 | 214 | char name[6]; |
| 216 | 215 | |
| @@ -240,7 +240,7 @@ static void __init sama5d2_pmc_setup(struct device_node *np) · 239 unmodified lines | ||
| 240 | 240 | parent_names[1] = "mainck"; |
| 241 | 241 | parent_names[2] = "plladivck"; |
| 242 | 242 | parent_names[3] = "utmick"; |
| 243 | − parent_names[4] = "mck"; | |
| 243 | + parent_names[4] = "masterck"; | |
| 244 | 244 | for (i = 0; i < 3; i++) { |
| 245 | 245 | char name[6]; |
| 246 | 246 | |
| @@ -291,7 +291,7 @@ static void __init sama5d2_pmc_setup(struct device_node *np) · 44 unmodified lines | ||
| 291 | 291 | parent_names[1] = "mainck"; |
| 292 | 292 | parent_names[2] = "plladivck"; |
| 293 | 293 | parent_names[3] = "utmick"; |
| 294 | − parent_names[4] = "mck"; | |
| 294 | + parent_names[4] = "masterck"; | |
| 295 | 295 | parent_names[5] = "audiopll_pmcck"; |
| 296 | 296 | for (i = 0; i < ARRAY_SIZE(sama5d2_gck); i++) { |
| 297 | 297 | hw = at91_clk_register_generated(regmap, &pmc_pcr_lock, |
| @@ -207,7 +207,7 @@ static void __init sama5d4_pmc_setup(struct device_node *np) · 206 unmodified lines | ||
| 207 | 207 | parent_names[1] = "mainck"; |
| 208 | 208 | parent_names[2] = "plladivck"; |
| 209 | 209 | parent_names[3] = "utmick"; |
| 210 | − parent_names[4] = "mck"; | |
| 210 | + parent_names[4] = "masterck"; | |
| 211 | 211 | for (i = 0; i < 3; i++) { |
| 212 | 212 | char name[6]; |
| 213 | 213 | |
| @@ -264,9 +264,9 @@ static SUNXI_CCU_GATE(ahb1_mmc1_clk, "ahb1-mmc1", "ahb1", · 263 unmodified lines | ||
| 264 | 264 | static SUNXI_CCU_GATE(ahb1_mmc2_clk, "ahb1-mmc2", "ahb1", |
| 265 | 265 | 0x060, BIT(10), 0); |
| 266 | 266 | static SUNXI_CCU_GATE(ahb1_mmc3_clk, "ahb1-mmc3", "ahb1", |
| 267 | − 0x060, BIT(12), 0); | |
| 267 | + 0x060, BIT(11), 0); | |
| 268 | 268 | static SUNXI_CCU_GATE(ahb1_nand1_clk, "ahb1-nand1", "ahb1", |
| 269 | − 0x060, BIT(13), 0); | |
| 269 | + 0x060, BIT(12), 0); | |
| 270 | 270 | static SUNXI_CCU_GATE(ahb1_nand0_clk, "ahb1-nand0", "ahb1", |
| 271 | 271 | 0x060, BIT(13), 0); |
| 272 | 272 | static SUNXI_CCU_GATE(ahb1_sdram_clk, "ahb1-sdram", "ahb1", |
| @@ -542,7 +542,7 @@ static struct ccu_reset_map sun8i_v3s_ccu_resets[] = { · 541 unmodified lines | ||
| 542 | 542 | [RST_BUS_OHCI0] = { 0x2c0, BIT(29) }, |
| 543 | 543 | |
| 544 | 544 | [RST_BUS_VE] = { 0x2c4, BIT(0) }, |
| 545 | − [RST_BUS_TCON0] = { 0x2c4, BIT(3) }, | |
| 545 | + [RST_BUS_TCON0] = { 0x2c4, BIT(4) }, | |
| 546 | 546 | [RST_BUS_CSI] = { 0x2c4, BIT(8) }, |
| 547 | 547 | [RST_BUS_DE] = { 0x2c4, BIT(12) }, |
| 548 | 548 | [RST_BUS_DBG] = { 0x2c4, BIT(31) }, |
| @@ -187,8 +187,8 @@ static int scmi_cpufreq_exit(struct cpufreq_policy *policy) · 186 unmodified lines | ||
| 187 | 187 | |
| 188 | 188 | cpufreq_cooling_unregister(priv->cdev); |
| 189 | 189 | dev_pm_opp_free_cpufreq_table(priv->cpu_dev, &policy->freq_table); |
| 190 | − kfree(priv); | |
| 191 | 190 | dev_pm_opp_remove_all_dynamic(priv->cpu_dev); |
| 191 | + kfree(priv); | |
| 192 | 192 | |
| 193 | 193 | return 0; |
| 194 | 194 | } |
| @@ -30,7 +30,7 @@ static inline int cc_pm_init(struct cc_drvdata *drvdata) · 29 unmodified lines | ||
| 30 | 30 | return 0; |
| 31 | 31 | } |
| 32 | 32 | |
| 33 | −static void cc_pm_go(struct cc_drvdata *drvdata) {} | |
| 33 | +static inline void cc_pm_go(struct cc_drvdata *drvdata) {} | |
| 34 | 34 | |
| 35 | 35 | static inline void cc_pm_fini(struct cc_drvdata *drvdata) {} |
| 36 | 36 | |
| @@ -30,6 +30,7 @@ · 29 unmodified lines | ||
| 30 | 30 | #define GPIO_REG_EDGE 0xA0 |
| 31 | 31 | |
| 32 | 32 | struct mtk_gc { |
| 33 | + struct irq_chip irq_chip; | |
| 33 | 34 | struct gpio_chip chip; |
| 34 | 35 | spinlock_t lock; |
| 35 | 36 | int bank; |
| @@ -189,13 +190,6 @@ mediatek_gpio_irq_type(struct irq_data *d, unsigned int type) · 153 unmodified lines | ||
| 189 | 190 | return 0; |
| 190 | 191 | } |
| 191 | 192 | |
| 192 | −static struct irq_chip mediatek_gpio_irq_chip = { | |
| 193 | − .irq_unmask = mediatek_gpio_irq_unmask, | |
| 194 | − .irq_mask = mediatek_gpio_irq_mask, | |
| 195 | − .irq_mask_ack = mediatek_gpio_irq_mask, | |
| 196 | − .irq_set_type = mediatek_gpio_irq_type, | |
| 197 | −}; | |
| 198 | − | |
| 199 | 193 | static int |
| 200 | 194 | mediatek_gpio_xlate(struct gpio_chip *chip, |
| 201 | 195 | const struct of_phandle_args *spec, u32 *flags) |
| @@ -254,6 +248,13 @@ mediatek_gpio_bank_probe(struct device *dev, · 52 unmodified lines | ||
| 254 | 248 | return ret; |
| 255 | 249 | } |
| 256 | 250 | |
| 251 | + rg->irq_chip.name = dev_name(dev); | |
| 252 | + rg->irq_chip.parent_device = dev; | |
| 253 | + rg->irq_chip.irq_unmask = mediatek_gpio_irq_unmask; | |
| 254 | + rg->irq_chip.irq_mask = mediatek_gpio_irq_mask; | |
| 255 | + rg->irq_chip.irq_mask_ack = mediatek_gpio_irq_mask; | |
| 256 | + rg->irq_chip.irq_set_type = mediatek_gpio_irq_type; | |
| 257 | + | |
| 257 | 258 | if (mtk->gpio_irq) { |
| 258 | 259 | /* |
| 259 | 260 | * Manually request the irq here instead of passing |
| @@ -270,14 +271,14 @@ mediatek_gpio_bank_probe(struct device *dev, · 10 unmodified lines | ||
| 270 | 271 | return ret; |
| 271 | 272 | } |
| 272 | 273 | |
| 273 | − ret = gpiochip_irqchip_add(&rg->chip, &mediatek_gpio_irq_chip, | |
| 274 | + ret = gpiochip_irqchip_add(&rg->chip, &rg->irq_chip, | |
| 274 | 275 | 0, handle_simple_irq, IRQ_TYPE_NONE); |
| 275 | 276 | if (ret) { |
| 276 | 277 | dev_err(dev, "failed to add gpiochip_irqchip\n"); |
| 277 | 278 | return ret; |
| 278 | 279 | } |
| 279 | 280 | |
| 280 | − gpiochip_set_chained_irqchip(&rg->chip, &mediatek_gpio_irq_chip, | |
| 281 | + gpiochip_set_chained_irqchip(&rg->chip, &rg->irq_chip, | |
| 281 | 282 | mtk->gpio_irq, NULL); |
| 282 | 283 | } |
| 283 | 284 | |
| @@ -310,7 +311,6 @@ mediatek_gpio_probe(struct platform_device *pdev) · 26 unmodified lines | ||
| 310 | 311 | mtk->gpio_irq = irq_of_parse_and_map(np, 0); |
| 311 | 312 | mtk->dev = dev; |
| 312 | 313 | platform_set_drvdata(pdev, mtk); |
| 313 | − mediatek_gpio_irq_chip.name = dev_name(dev); | |
| 314 | 314 | |
| 315 | 315 | for (i = 0; i < MTK_BANK_CNT; i++) { |
| 316 | 316 | ret = mediatek_gpio_bank_probe(dev, np, i); |
| @@ -245,6 +245,7 @@ static bool pxa_gpio_has_pinctrl(void) · 244 unmodified lines | ||
| 245 | 245 | { |
| 246 | 246 | switch (gpio_type) { |
| 247 | 247 | case PXA3XX_GPIO: |
| 248 | + case MMP2_GPIO: | |
| 248 | 249 | return false; |
| 249 | 250 | |
| 250 | 251 | default: |
| @@ -411,6 +411,8 @@ struct amdgpu_fpriv { · 410 unmodified lines | ||
| 411 | 411 | struct amdgpu_ctx_mgr ctx_mgr; |
| 412 | 412 | }; |
| 413 | 413 | |
| 414 | +int amdgpu_file_to_fpriv(struct file *filp, struct amdgpu_fpriv **fpriv); | |
| 415 | + | |
| 414 | 416 | int amdgpu_ib_get(struct amdgpu_device *adev, struct amdgpu_vm *vm, |
| 415 | 417 | unsigned size, struct amdgpu_ib *ib); |
| 416 | 418 | void amdgpu_ib_free(struct amdgpu_device *adev, struct amdgpu_ib *ib, |
| @@ -131,7 +131,7 @@ static void amdgpu_doorbell_get_kfd_info(struct amdgpu_device *adev, · 130 unmodified lines | ||
| 131 | 131 | |
| 132 | 132 | void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) |
| 133 | 133 | { |
| 134 | − int i, n; | |
| 134 | + int i; | |
| 135 | 135 | int last_valid_bit; |
| 136 | 136 | |
| 137 | 137 | if (adev->kfd.dev) { |
| @@ -142,7 +142,9 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) · 4 unmodified lines | ||
| 142 | 142 | .gpuvm_size = min(adev->vm_manager.max_pfn |
| 143 | 143 | << AMDGPU_GPU_PAGE_SHIFT, |
| 144 | 144 | AMDGPU_GMC_HOLE_START), |
| 145 | − .drm_render_minor = adev->ddev->render->index | |
| 145 | + .drm_render_minor = adev->ddev->render->index, | |
| 146 | + .sdma_doorbell_idx = adev->doorbell_index.sdma_engine, | |
| 147 | + | |
| 146 | 148 | }; |
| 147 | 149 | |
| 148 | 150 | /* this is going to have a few of the MSBs set that we need to |
| @@ -172,35 +174,20 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) · 23 unmodified lines | ||
| 172 | 174 | &gpu_resources.doorbell_aperture_size, |
| 173 | 175 | &gpu_resources.doorbell_start_offset); |
| 174 | 176 | |
| 175 | − if (adev->asic_type < CHIP_VEGA10) { | |
| 176 | − kgd2kfd_device_init(adev->kfd.dev, &gpu_resources); | |
| 177 | − return; | |
| 178 | − } | |
| 179 | − | |
| 180 | − n = (adev->asic_type < CHIP_VEGA20) ? 2 : 8; | |
| 181 | − | |
| 182 | − for (i = 0; i < n; i += 2) { | |
| 183 | − /* On SOC15 the BIF is involved in routing | |
| 184 | − * doorbells using the low 12 bits of the | |
| 185 | − * address. Communicate the assignments to | |
| 186 | − * KFD. KFD uses two doorbell pages per | |
| 187 | − * process in case of 64-bit doorbells so we | |
| 188 | − * can use each doorbell assignment twice. | |
| 189 | − */ | |
| 190 | − gpu_resources.sdma_doorbell[0][i] = | |
| 191 | − adev->doorbell_index.sdma_engine[0] + (i >> 1); | |
| 192 | − gpu_resources.sdma_doorbell[0][i+1] = | |
| 193 | − adev->doorbell_index.sdma_engine[0] + 0x200 + (i >> 1); | |
| 194 | − gpu_resources.sdma_doorbell[1][i] = | |
| 195 | − adev->doorbell_index.sdma_engine[1] + (i >> 1); | |
| 196 | − gpu_resources.sdma_doorbell[1][i+1] = | |
| 197 | − adev->doorbell_index.sdma_engine[1] + 0x200 + (i >> 1); | |
| 198 | − } | |
| 199 | − /* Doorbells 0x0e0-0ff and 0x2e0-2ff are reserved for | |
| 200 | − * SDMA, IH and VCN. So don't use them for the CP. | |
| 177 | + /* Since SOC15, BIF starts to statically use the | |
| 178 | + * lower 12 bits of doorbell addresses for routing | |
| 179 | + * based on settings in registers like | |
| 180 | + * SDMA0_DOORBELL_RANGE etc.. | |
| 181 | + * In order to route a doorbell to CP engine, the lower | |
| 182 | + * 12 bits of its address has to be outside the range | |
| 183 | + * set for SDMA, VCN, and IH blocks. | |
| 201 | 184 | */ |
| 202 | − gpu_resources.reserved_doorbell_mask = 0x1e0; | |
| 203 | − gpu_resources.reserved_doorbell_val = 0x0e0; | |
| 185 | + if (adev->asic_type >= CHIP_VEGA10) { | |
| 186 | + gpu_resources.non_cp_doorbells_start = | |
| 187 | + adev->doorbell_index.first_non_cp; | |
| 188 | + gpu_resources.non_cp_doorbells_end = | |
| 189 | + adev->doorbell_index.last_non_cp; | |
| 190 | + } | |
| 204 | 191 | |
| 205 | 192 | kgd2kfd_device_init(adev->kfd.dev, &gpu_resources); |
| 206 | 193 | } |
| @@ -204,38 +204,25 @@ void amdgpu_amdkfd_unreserve_memory_limit(struct amdgpu_bo *bo) · 203 unmodified lines | ||
| 204 | 204 | } |
| 205 | 205 | |
| 206 | 206 | |
| 207 | −/* amdgpu_amdkfd_remove_eviction_fence - Removes eviction fence(s) from BO's | |
| 207 | +/* amdgpu_amdkfd_remove_eviction_fence - Removes eviction fence from BO's | |
| 208 | 208 | * reservation object. |
| 209 | 209 | * |
| 210 | 210 | * @bo: [IN] Remove eviction fence(s) from this BO |
| 211 | − * @ef: [IN] If ef is specified, then this eviction fence is removed if it | |
| 211 | + * @ef: [IN] This eviction fence is removed if it | |
| 212 | 212 | * is present in the shared list. |
| 213 | − * @ef_list: [OUT] Returns list of eviction fences. These fences are removed | |
| 214 | − * from BO's reservation object shared list. | |
| 215 | − * @ef_count: [OUT] Number of fences in ef_list. | |
| 216 | 213 | * |
| 217 | − * NOTE: If called with ef_list, then amdgpu_amdkfd_add_eviction_fence must be | |
| 218 | − * called to restore the eviction fences and to avoid memory leak. This is | |
| 219 | − * useful for shared BOs. | |
| 220 | 214 | * NOTE: Must be called with BO reserved i.e. bo->tbo.resv->lock held. |
| 221 | 215 | */ |
| 222 | 216 | static int amdgpu_amdkfd_remove_eviction_fence(struct amdgpu_bo *bo, |
| 223 | − struct amdgpu_amdkfd_fence *ef, | |
| 224 | − struct amdgpu_amdkfd_fence ***ef_list, | |
| 225 | − unsigned int *ef_count) | |
| 217 | + struct amdgpu_amdkfd_fence *ef) | |
| 226 | 218 | { |
| 227 | 219 | struct reservation_object *resv = bo->tbo.resv; |
| 228 | 220 | struct reservation_object_list *old, *new; |
| 229 | 221 | unsigned int i, j, k; |
| 230 | 222 | |
| 231 | − if (!ef && !ef_list) | |
| 223 | + if (!ef) | |
| 232 | 224 | return -EINVAL; |
| 233 | 225 | |
| 234 | − if (ef_list) { | |
| 235 | − *ef_list = NULL; | |
| 236 | − *ef_count = 0; | |
| 237 | − } | |
| 238 | − | |
| 239 | 226 | old = reservation_object_get_list(resv); |
| 240 | 227 | if (!old) |
| 241 | 228 | return 0; |
| @@ -254,8 +241,7 @@ static int amdgpu_amdkfd_remove_eviction_fence(struct amdgpu_bo *bo, · 12 unmodified lines | ||
| 254 | 241 | f = rcu_dereference_protected(old->shared[i], |
| 255 | 242 | reservation_object_held(resv)); |
| 256 | 243 | |
| 257 | − if ((ef && f->context == ef->base.context) || | |
| 258 | − (!ef && to_amdgpu_amdkfd_fence(f))) | |
| 244 | + if (f->context == ef->base.context) | |
| 259 | 245 | RCU_INIT_POINTER(new->shared[--j], f); |
| 260 | 246 | else |
| 261 | 247 | RCU_INIT_POINTER(new->shared[k++], f); |
| @@ -263,21 +249,6 @@ static int amdgpu_amdkfd_remove_eviction_fence(struct amdgpu_bo *bo, · 1 unmodified line | ||
| 263 | 249 | new->shared_max = old->shared_max; |
| 264 | 250 | new->shared_count = k; |
| 265 | 251 | |
| 266 | − if (!ef) { | |
| 267 | − unsigned int count = old->shared_count - j; | |
| 268 | − | |
| 269 | − /* Alloc memory for count number of eviction fence pointers. | |
| 270 | − * Fill the ef_list array and ef_count | |
| 271 | − */ | |
| 272 | − *ef_list = kcalloc(count, sizeof(**ef_list), GFP_KERNEL); | |
| 273 | − *ef_count = count; | |
| 274 | − | |
| 275 | − if (!*ef_list) { | |
| 276 | − kfree(new); | |
| 277 | − return -ENOMEM; | |
| 278 | − } | |
| 279 | − } | |
| 280 | − | |
| 281 | 252 | /* Install the new fence list, seqcount provides the barriers */ |
| 282 | 253 | preempt_disable(); |
| 283 | 254 | write_seqcount_begin(&resv->seq); |
| @@ -291,46 +262,13 @@ static int amdgpu_amdkfd_remove_eviction_fence(struct amdgpu_bo *bo, · 7 unmodified lines | ||
| 291 | 262 | |
| 292 | 263 | f = rcu_dereference_protected(new->shared[i], |
| 293 | 264 | reservation_object_held(resv)); |
| 294 | − if (!ef) | |
| 295 | − (*ef_list)[k++] = to_amdgpu_amdkfd_fence(f); | |
| 296 | − else | |
| 297 | − dma_fence_put(f); | |
| 265 | + dma_fence_put(f); | |
| 298 | 266 | } |
| 299 | 267 | kfree_rcu(old, rcu); |
| 300 | 268 | |
| 301 | 269 | return 0; |
| 302 | 270 | } |
| 303 | 271 | |
| 304 | −/* amdgpu_amdkfd_add_eviction_fence - Adds eviction fence(s) back into BO's | |
| 305 | − * reservation object. | |
| 306 | − * | |
| 307 | − * @bo: [IN] Add eviction fences to this BO | |
| 308 | − * @ef_list: [IN] List of eviction fences to be added | |
| 309 | − * @ef_count: [IN] Number of fences in ef_list. | |
| 310 | − * | |
| 311 | − * NOTE: Must call amdgpu_amdkfd_remove_eviction_fence before calling this | |
| 312 | − * function. | |
| 313 | − */ | |
| 314 | −static void amdgpu_amdkfd_add_eviction_fence(struct amdgpu_bo *bo, | |
| 315 | − struct amdgpu_amdkfd_fence **ef_list, | |
| 316 | − unsigned int ef_count) | |
| 317 | −{ | |
| 318 | − int i; | |
| 319 | − | |
| 320 | − if (!ef_list || !ef_count) | |
| 321 | − return; | |
| 322 | − | |
| 323 | − for (i = 0; i < ef_count; i++) { | |
| 324 | − amdgpu_bo_fence(bo, &ef_list[i]->base, true); | |
| 325 | − /* Re-adding the fence takes an additional reference. Drop that | |
| 326 | − * reference. | |
| 327 | − */ | |
| 328 | − dma_fence_put(&ef_list[i]->base); | |
| 329 | − } | |
| 330 | − | |
| 331 | − kfree(ef_list); | |
| 332 | −} | |
| 333 | − | |
| 334 | 272 | static int amdgpu_amdkfd_bo_validate(struct amdgpu_bo *bo, uint32_t domain, |
| 335 | 273 | bool wait) |
| 336 | 274 | { |
| @@ -346,18 +284,8 @@ static int amdgpu_amdkfd_bo_validate(struct amdgpu_bo *bo, uint32_t domain, · 9 unmodified lines | ||
| 346 | 284 | ret = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); |
| 347 | 285 | if (ret) |
| 348 | 286 | goto validate_fail; |
| 349 | − if (wait) { | |
| 350 | − struct amdgpu_amdkfd_fence **ef_list; | |
| 351 | − unsigned int ef_count; | |
| 352 | − | |
| 353 | − ret = amdgpu_amdkfd_remove_eviction_fence(bo, NULL, &ef_list, | |
| 354 | − &ef_count); | |
| 355 | − if (ret) | |
| 356 | − goto validate_fail; | |
| 357 | − | |
| 358 | − ttm_bo_wait(&bo->tbo, false, false); | |
| 359 | − amdgpu_amdkfd_add_eviction_fence(bo, ef_list, ef_count); | |
| 360 | − } | |
| 287 | + if (wait) | |
| 288 | + amdgpu_bo_sync_wait(bo, AMDGPU_FENCE_OWNER_KFD, false); | |
| 361 | 289 | |
| 362 | 290 | validate_fail: |
| 363 | 291 | return ret; |
| @@ -444,7 +372,6 @@ static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, · 80 unmodified lines | ||
| 444 | 372 | { |
| 445 | 373 | int ret; |
| 446 | 374 | struct kfd_bo_va_list *bo_va_entry; |
| 447 | − struct amdgpu_bo *pd = vm->root.base.bo; | |
| 448 | 375 | struct amdgpu_bo *bo = mem->bo; |
| 449 | 376 | uint64_t va = mem->va; |
| 450 | 377 | struct list_head *list_bo_va = &mem->bo_va_list; |
| @@ -484,14 +411,8 @@ static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, · 33 unmodified lines | ||
| 484 | 411 | *p_bo_va_entry = bo_va_entry; |
| 485 | 412 | |
| 486 | 413 | /* Allocate new page tables if needed and validate |
| 487 | − * them. Clearing of new page tables and validate need to wait | |
| 488 | − * on move fences. We don't want that to trigger the eviction | |
| 489 | − * fence, so remove it temporarily. | |
| 414 | + * them. | |
| 490 | 415 | */ |
| 491 | − amdgpu_amdkfd_remove_eviction_fence(pd, | |
| 492 | − vm->process_info->eviction_fence, | |
| 493 | − NULL, NULL); | |
| 494 | − | |
| 495 | 416 | ret = amdgpu_vm_alloc_pts(adev, vm, va, amdgpu_bo_size(bo)); |
| 496 | 417 | if (ret) { |
| 497 | 418 | pr_err("Failed to allocate pts, err=%d\n", ret); |
| @@ -504,13 +425,9 @@ static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, · 6 unmodified lines | ||
| 504 | 425 | goto err_alloc_pts; |
| 505 | 426 | } |
| 506 | 427 | |
| 507 | − /* Add the eviction fence back */ | |
| 508 | − amdgpu_bo_fence(pd, &vm->process_info->eviction_fence->base, true); | |
| 509 | − | |
| 510 | 428 | return 0; |
| 511 | 429 | |
| 512 | 430 | err_alloc_pts: |
| 513 | − amdgpu_bo_fence(pd, &vm->process_info->eviction_fence->base, true); | |
| 514 | 431 | amdgpu_vm_bo_rmv(adev, bo_va_entry->bo_va); |
| 515 | 432 | list_del(&bo_va_entry->bo_list); |
| 516 | 433 | err_vmadd: |
| @@ -809,24 +726,11 @@ static int unmap_bo_from_gpuvm(struct amdgpu_device *adev, · 292 unmodified lines | ||
| 809 | 726 | { |
| 810 | 727 | struct amdgpu_bo_va *bo_va = entry->bo_va; |
| 811 | 728 | struct amdgpu_vm *vm = bo_va->base.vm; |
| 812 | − struct amdgpu_bo *pd = vm->root.base.bo; | |
| 813 | 729 | |
| 814 | − /* Remove eviction fence from PD (and thereby from PTs too as | |
| 815 | − * they share the resv. object). Otherwise during PT update | |
| 816 | − * job (see amdgpu_vm_bo_update_mapping), eviction fence would | |
| 817 | − * get added to job->sync object and job execution would | |
| 818 | − * trigger the eviction fence. | |
| 819 | − */ | |
| 820 | − amdgpu_amdkfd_remove_eviction_fence(pd, | |
| 821 | − vm->process_info->eviction_fence, | |
| 822 | − NULL, NULL); | |
| 823 | 730 | amdgpu_vm_bo_unmap(adev, bo_va, entry->va); |
| 824 | 731 | |
| 825 | 732 | amdgpu_vm_clear_freed(adev, vm, &bo_va->last_pt_update); |
| 826 | 733 | |
| 827 | − /* Add the eviction fence back */ | |
| 828 | − amdgpu_bo_fence(pd, &vm->process_info->eviction_fence->base, true); | |
| 829 | − | |
| 830 | 734 | amdgpu_sync_fence(NULL, sync, bo_va->last_pt_update, false); |
| 831 | 735 | |
| 832 | 736 | return 0; |
| @@ -1002,7 +906,7 @@ static int init_kfd_vm(struct amdgpu_vm *vm, void **process_info, · 169 unmodified lines | ||
| 1002 | 906 | pr_err("validate_pt_pd_bos() failed\n"); |
| 1003 | 907 | goto validate_pd_fail; |
| 1004 | 908 | } |
| 1005 | − ret = ttm_bo_wait(&vm->root.base.bo->tbo, false, false); | |
| 909 | + amdgpu_bo_sync_wait(vm->root.base.bo, AMDGPU_FENCE_OWNER_KFD, false); | |
| 1006 | 910 | if (ret) |
| 1007 | 911 | goto wait_pd_fail; |
| 1008 | 912 | amdgpu_bo_fence(vm->root.base.bo, |
| @@ -1389,8 +1293,7 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( · 380 unmodified lines | ||
| 1389 | 1293 | * attached |
| 1390 | 1294 | */ |
| 1391 | 1295 | amdgpu_amdkfd_remove_eviction_fence(mem->bo, |
| 1392 | − process_info->eviction_fence, | |
| 1393 | − NULL, NULL); | |
| 1296 | + process_info->eviction_fence); | |
| 1394 | 1297 | pr_debug("Release VA 0x%llx - 0x%llx\n", mem->va, |
| 1395 | 1298 | mem->va + bo_size * (1 + mem->aql_queue)); |
| 1396 | 1299 | |
| @@ -1617,8 +1520,7 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( · 220 unmodified lines | ||
| 1617 | 1520 | if (mem->mapped_to_gpu_memory == 0 && |
| 1618 | 1521 | !amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm) && !mem->bo->pin_count) |
| 1619 | 1522 | amdgpu_amdkfd_remove_eviction_fence(mem->bo, |
| 1620 | − process_info->eviction_fence, | |
| 1621 | − NULL, NULL); | |
| 1523 | + process_info->eviction_fence); | |
| 1622 | 1524 | |
| 1623 | 1525 | unreserve_out: |
| 1624 | 1526 | unreserve_bo_and_vms(&ctx, false, false); |
| @@ -1679,7 +1581,7 @@ int amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(struct kgd_dev *kgd, · 54 unmodified lines | ||
| 1679 | 1581 | } |
| 1680 | 1582 | |
| 1681 | 1583 | amdgpu_amdkfd_remove_eviction_fence( |
| 1682 | − bo, mem->process_info->eviction_fence, NULL, NULL); | |
| 1584 | + bo, mem->process_info->eviction_fence); | |
| 1683 | 1585 | list_del_init(&mem->validate_list.head); |
| 1684 | 1586 | |
| 1685 | 1587 | if (size) |
| @@ -1945,16 +1847,6 @@ static int validate_invalid_user_pages(struct amdkfd_process_info *process_info) · 259 unmodified lines | ||
| 1945 | 1847 | |
| 1946 | 1848 | amdgpu_sync_create(&sync); |
| 1947 | 1849 | |
| 1948 | − /* Avoid triggering eviction fences when unmapping invalid | |
| 1949 | − * userptr BOs (waits for all fences, doesn't use | |
| 1950 | − * FENCE_OWNER_VM) | |
| 1951 | − */ | |
| 1952 | − list_for_each_entry(peer_vm, &process_info->vm_list_head, | |
| 1953 | − vm_list_node) | |
| 1954 | − amdgpu_amdkfd_remove_eviction_fence(peer_vm->root.base.bo, | |
| 1955 | − process_info->eviction_fence, | |
| 1956 | − NULL, NULL); | |
| 1957 | − | |
| 1958 | 1850 | ret = process_validate_vms(process_info); |
| 1959 | 1851 | if (ret) |
| 1960 | 1852 | goto unreserve_out; |
| @@ -2015,10 +1907,6 @@ static int validate_invalid_user_pages(struct amdkfd_process_info *process_info) · 54 unmodified lines | ||
| 2015 | 1907 | ret = process_update_pds(process_info, &sync); |
| 2016 | 1908 | |
| 2017 | 1909 | unreserve_out: |
| 2018 | − list_for_each_entry(peer_vm, &process_info->vm_list_head, | |
| 2019 | − vm_list_node) | |
| 2020 | − amdgpu_bo_fence(peer_vm->root.base.bo, | |
| 2021 | − &process_info->eviction_fence->base, true); | |
| 2022 | 1910 | ttm_eu_backoff_reservation(&ticket, &resv_list); |
| 2023 | 1911 | amdgpu_sync_wait(&sync, false); |
| 2024 | 1912 | amdgpu_sync_free(&sync); |
| @@ -124,6 +124,7 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, · 123 unmodified lines | ||
| 124 | 124 | struct amdgpu_ring *rings[AMDGPU_MAX_RINGS]; |
| 125 | 125 | struct drm_sched_rq *rqs[AMDGPU_MAX_RINGS]; |
| 126 | 126 | unsigned num_rings; |
| 127 | + unsigned num_rqs = 0; | |
| 127 | 128 | |
| 128 | 129 | switch (i) { |
| 129 | 130 | case AMDGPU_HW_IP_GFX: |
| @@ -166,12 +167,16 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, · 36 unmodified lines | ||
| 166 | 167 | break; |
| 167 | 168 | } |
| 168 | 169 | |
| 169 | − for (j = 0; j < num_rings; ++j) | |
| 170 | − rqs[j] = &rings[j]->sched.sched_rq[priority]; | |
| 170 | + for (j = 0; j < num_rings; ++j) { | |
| 171 | + if (!rings[j]->adev) | |
| 172 | + continue; | |
| 173 | + | |
| 174 | + rqs[num_rqs++] = &rings[j]->sched.sched_rq[priority]; | |
| 175 | + } | |
| 171 | 176 | |
| 172 | 177 | for (j = 0; j < amdgpu_ctx_num_entities[i]; ++j) |
| 173 | 178 | r = drm_sched_entity_init(&ctx->entities[i][j].entity, |
| 174 | − rqs, num_rings, &ctx->guilty); | |
| 179 | + rqs, num_rqs, &ctx->guilty); | |
| 175 | 180 | if (r) |
| 176 | 181 | goto error_cleanup_entities; |
| 177 | 182 | } |
| @@ -158,9 +158,6 @@ static int amdgpu_debugfs_process_reg_op(bool read, struct file *f, · 157 unmodified lines | ||
| 158 | 158 | while (size) { |
| 159 | 159 | uint32_t value; |
| 160 | 160 | |
| 161 | − if (*pos > adev->rmmio_size) | |
| 162 | − goto end; | |
| 163 | − | |
| 164 | 161 | if (read) { |
| 165 | 162 | value = RREG32(*pos >> 2); |
| 166 | 163 | r = put_user(value, (uint32_t *)buf); |
| @@ -71,6 +71,8 @@ struct amdgpu_doorbell_index { · 70 unmodified lines | ||
| 71 | 71 | uint32_t vce_ring6_7; |
| 72 | 72 | } uvd_vce; |
| 73 | 73 | }; |
| 74 | + uint32_t first_non_cp; | |
| 75 | + uint32_t last_non_cp; | |
| 74 | 76 | uint32_t max_assignment; |
| 75 | 77 | /* Per engine SDMA doorbell size in dword */ |
| 76 | 78 | uint32_t sdma_doorbell_range; |
| @@ -143,6 +145,10 @@ typedef enum _AMDGPU_VEGA20_DOORBELL_ASSIGNMENT · 66 unmodified lines | ||
| 143 | 145 | AMDGPU_VEGA20_DOORBELL64_VCE_RING2_3 = 0x18D, |
| 144 | 146 | AMDGPU_VEGA20_DOORBELL64_VCE_RING4_5 = 0x18E, |
| 145 | 147 | AMDGPU_VEGA20_DOORBELL64_VCE_RING6_7 = 0x18F, |
| 148 | + | |
| 149 | + AMDGPU_VEGA20_DOORBELL64_FIRST_NON_CP = AMDGPU_VEGA20_DOORBELL_sDMA_ENGINE0, | |
| 150 | + AMDGPU_VEGA20_DOORBELL64_LAST_NON_CP = AMDGPU_VEGA20_DOORBELL64_VCE_RING6_7, | |
| 151 | + | |
| 146 | 152 | AMDGPU_VEGA20_DOORBELL_MAX_ASSIGNMENT = 0x18F, |
| 147 | 153 | AMDGPU_VEGA20_DOORBELL_INVALID = 0xFFFF |
| 148 | 154 | } AMDGPU_VEGA20_DOORBELL_ASSIGNMENT; |
| @@ -222,6 +228,9 @@ typedef enum _AMDGPU_DOORBELL64_ASSIGNMENT · 73 unmodified lines | ||
| 222 | 228 | AMDGPU_DOORBELL64_VCE_RING4_5 = 0xFE, |
| 223 | 229 | AMDGPU_DOORBELL64_VCE_RING6_7 = 0xFF, |
| 224 | 230 | |
| 231 | + AMDGPU_DOORBELL64_FIRST_NON_CP = AMDGPU_DOORBELL64_sDMA_ENGINE0, | |
| 232 | + AMDGPU_DOORBELL64_LAST_NON_CP = AMDGPU_DOORBELL64_VCE_RING6_7, | |
| 233 | + | |
| 225 | 234 | AMDGPU_DOORBELL64_MAX_ASSIGNMENT = 0xFF, |
| 226 | 235 | AMDGPU_DOORBELL64_INVALID = 0xFFFF |
| 227 | 236 | } AMDGPU_DOORBELL64_ASSIGNMENT; |
| @@ -184,61 +184,6 @@ u32 amdgpu_dpm_get_vrefresh(struct amdgpu_device *adev) · 183 unmodified lines | ||
| 184 | 184 | return vrefresh; |
| 185 | 185 | } |
| 186 | 186 | |
| 187 | −void amdgpu_calculate_u_and_p(u32 i, u32 r_c, u32 p_b, | |
| 188 | − u32 *p, u32 *u) | |
| 189 | −{ | |
| 190 | − u32 b_c = 0; | |
| 191 | − u32 i_c; | |
| 192 | − u32 tmp; | |
| 193 | − | |
| 194 | − i_c = (i * r_c) / 100; | |
| 195 | − tmp = i_c >> p_b; | |
| 196 | − | |
| 197 | − while (tmp) { | |
| 198 | − b_c++; | |
| 199 | − tmp >>= 1; | |
| 200 | − } | |
| 201 | − | |
| 202 | − *u = (b_c + 1) / 2; | |
| 203 | − *p = i_c / (1 << (2 * (*u))); | |
| 204 | −} | |
| 205 | − | |
| 206 | −int amdgpu_calculate_at(u32 t, u32 h, u32 fh, u32 fl, u32 *tl, u32 *th) | |
| 207 | −{ | |
| 208 | − u32 k, a, ah, al; | |
| 209 | − u32 t1; | |
| 210 | − | |
| 211 | − if ((fl == 0) || (fh == 0) || (fl > fh)) | |
| 212 | − return -EINVAL; | |
| 213 | − | |
| 214 | − k = (100 * fh) / fl; | |
| 215 | − t1 = (t * (k - 100)); | |
| 216 | − a = (1000 * (100 * h + t1)) / (10000 + (t1 / 100)); | |
| 217 | − a = (a + 5) / 10; | |
| 218 | − ah = ((a * t) + 5000) / 10000; | |
| 219 | − al = a - ah; | |
| 220 | − | |
| 221 | − *th = t - ah; | |
| 222 | − *tl = t + al; | |
| 223 | − | |
| 224 | − return 0; | |
| 225 | −} | |
| 226 | − | |
| 227 | −bool amdgpu_is_uvd_state(u32 class, u32 class2) | |
| 228 | −{ | |
| 229 | − if (class & ATOM_PPLIB_CLASSIFICATION_UVDSTATE) | |
| 230 | − return true; | |
| 231 | − if (class & ATOM_PPLIB_CLASSIFICATION_HD2STATE) | |
| 232 | − return true; | |
| 233 | − if (class & ATOM_PPLIB_CLASSIFICATION_HDSTATE) | |
| 234 | − return true; | |
| 235 | − if (class & ATOM_PPLIB_CLASSIFICATION_SDSTATE) | |
| 236 | − return true; | |
| 237 | − if (class2 & ATOM_PPLIB_CLASSIFICATION2_MVC) | |
| 238 | − return true; | |
| 239 | − return false; | |
| 240 | −} | |
| 241 | − | |
| 242 | 187 | bool amdgpu_is_internal_thermal_sensor(enum amdgpu_int_thermal_type sensor) |
| 243 | 188 | { |
| 244 | 189 | switch (sensor) { |
| @@ -949,39 +894,6 @@ enum amdgpu_pcie_gen amdgpu_get_pcie_gen_support(struct amdgpu_device *adev, · 704 unmodified lines | ||
| 949 | 894 | return AMDGPU_PCIE_GEN1; |
| 950 | 895 | } |
| 951 | 896 | |
| 952 | −u16 amdgpu_get_pcie_lane_support(struct amdgpu_device *adev, | |
| 953 | − u16 asic_lanes, | |
| 954 | − u16 default_lanes) | |
| 955 | −{ | |
| 956 | − switch (asic_lanes) { | |
| 957 | − case 0: | |
| 958 | − default: | |
| 959 | − return default_lanes; | |
| 960 | − case 1: | |
| 961 | − return 1; | |
| 962 | − case 2: | |
| 963 | − return 2; | |
| 964 | − case 4: | |
| 965 | − return 4; | |
| 966 | − case 8: | |
| 967 | − return 8; | |
| 968 | − case 12: | |
| 969 | − return 12; | |
| 970 | − case 16: | |
| 971 | − return 16; | |
| 972 | − } | |
| 973 | −} | |
| 974 | − | |
| 975 | −u8 amdgpu_encode_pci_lane_width(u32 lanes) | |
| 976 | −{ | |
| 977 | − u8 encoded_lanes[] = { 0, 1, 2, 0, 3, 0, 0, 0, 4, 0, 0, 0, 5, 0, 0, 0, 6 }; | |
| 978 | − | |
| 979 | − if (lanes > 16) | |
| 980 | − return 0; | |
| 981 | − | |
| 982 | − return encoded_lanes[lanes]; | |
| 983 | −} | |
| 984 | − | |
| 985 | 897 | struct amd_vce_state* |
| 986 | 898 | amdgpu_get_vce_clock_state(void *handle, u32 idx) |
| 987 | 899 | { |
| @@ -486,10 +486,6 @@ void amdgpu_dpm_print_ps_status(struct amdgpu_device *adev, · 485 unmodified lines | ||
| 486 | 486 | u32 amdgpu_dpm_get_vblank_time(struct amdgpu_device *adev); |
| 487 | 487 | u32 amdgpu_dpm_get_vrefresh(struct amdgpu_device *adev); |
| 488 | 488 | void amdgpu_dpm_get_active_displays(struct amdgpu_device *adev); |
| 489 | −bool amdgpu_is_uvd_state(u32 class, u32 class2); | |
| 490 | −void amdgpu_calculate_u_and_p(u32 i, u32 r_c, u32 p_b, | |
| 491 | − u32 *p, u32 *u); | |
| 492 | −int amdgpu_calculate_at(u32 t, u32 h, u32 fh, u32 fl, u32 *tl, u32 *th); | |
| 493 | 489 | |
| 494 | 490 | bool amdgpu_is_internal_thermal_sensor(enum amdgpu_int_thermal_type sensor); |
| 495 | 491 | |
| @@ -505,11 +501,6 @@ enum amdgpu_pcie_gen amdgpu_get_pcie_gen_support(struct amdgpu_device *adev, · 9 unmodified lines | ||
| 505 | 501 | enum amdgpu_pcie_gen asic_gen, |
| 506 | 502 | enum amdgpu_pcie_gen default_gen); |
| 507 | 503 | |
| 508 | −u16 amdgpu_get_pcie_lane_support(struct amdgpu_device *adev, | |
| 509 | − u16 asic_lanes, | |
| 510 | − u16 default_lanes); | |
| 511 | −u8 amdgpu_encode_pci_lane_width(u32 lanes); | |
| 512 | − | |
| 513 | 504 | struct amd_vce_state* |
| 514 | 505 | amdgpu_get_vce_clock_state(void *handle, u32 idx); |
| 515 | 506 | |
| @@ -73,9 +73,10 @@ · 72 unmodified lines | ||
| 73 | 73 | * - 3.27.0 - Add new chunk to to AMDGPU_CS to enable BO_LIST creation. |
| 74 | 74 | * - 3.28.0 - Add AMDGPU_CHUNK_ID_SCHEDULED_DEPENDENCIES |
| 75 | 75 | * - 3.29.0 - Add AMDGPU_IB_FLAG_RESET_GDS_MAX_WAVE_ID |
| 76 | + * - 3.30.0 - Add AMDGPU_SCHED_OP_CONTEXT_PRIORITY_OVERRIDE. | |
| 76 | 77 | */ |
| 77 | 78 | #define KMS_DRIVER_MAJOR 3 |
| 78 | −#define KMS_DRIVER_MINOR 29 | |
| 79 | +#define KMS_DRIVER_MINOR 30 | |
| 79 | 80 | #define KMS_DRIVER_PATCHLEVEL 0 |
| 80 | 81 | |
| 81 | 82 | int amdgpu_vram_limit = 0; |
| @@ -1179,6 +1180,22 @@ static const struct file_operations amdgpu_driver_kms_fops = { · 1097 unmodified lines | ||
| 1179 | 1180 | #endif |
| 1180 | 1181 | }; |
| 1181 | 1182 | |
| 1183 | +int amdgpu_file_to_fpriv(struct file *filp, struct amdgpu_fpriv **fpriv) | |
| 1184 | +{ | |
| 1185 | + struct drm_file *file; | |
| 1186 | + | |
| 1187 | + if (!filp) | |
| 1188 | + return -EINVAL; | |
| 1189 | + | |
| 1190 | + if (filp->f_op != &amdgpu_driver_kms_fops) { | |
| 1191 | + return -EINVAL; | |
| 1192 | + } | |
| 1193 | + | |
| 1194 | + file = filp->private_data; | |
| 1195 | + *fpriv = file->driver_priv; | |
| 1196 | + return 0; | |
| 1197 | +} | |
| 1198 | + | |
| 1182 | 1199 | static bool |
| 1183 | 1200 | amdgpu_get_crtc_scanout_position(struct drm_device *dev, unsigned int pipe, |
| 1184 | 1201 | bool in_vblank_irq, int *vpos, int *hpos, |
| @@ -140,9 +140,7 @@ void amdgpu_ih_ring_fini(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih) · 139 unmodified lines | ||
| 140 | 140 | * Interrupt hander (VI), walk the IH ring. |
| 141 | 141 | * Returns irq process return code. |
| 142 | 142 | */ |
| 143 | −int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, | |
| 144 | − void (*callback)(struct amdgpu_device *adev, | |
| 145 | − struct amdgpu_ih_ring *ih)) | |
| 143 | +int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih) | |
| 146 | 144 | { |
| 147 | 145 | u32 wptr; |
| 148 | 146 | |
| @@ -162,7 +160,7 @@ restart_ih: · 13 unmodified lines | ||
| 162 | 160 | rmb(); |
| 163 | 161 | |
| 164 | 162 | while (ih->rptr != wptr) { |
| 165 | − callback(adev, ih); | |
| 163 | + amdgpu_irq_dispatch(adev, ih); | |
| 166 | 164 | ih->rptr &= ih->ptr_mask; |
| 167 | 165 | } |
| 168 | 166 | |
| @@ -69,8 +69,6 @@ struct amdgpu_ih_funcs { · 68 unmodified lines | ||
| 69 | 69 | int amdgpu_ih_ring_init(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, |
| 70 | 70 | unsigned ring_size, bool use_bus_addr); |
| 71 | 71 | void amdgpu_ih_ring_fini(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih); |
| 72 | −int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, | |
| 73 | − void (*callback)(struct amdgpu_device *adev, | |
| 74 | − struct amdgpu_ih_ring *ih)); | |
| 72 | +int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih); | |
| 75 | 73 | |
| 76 | 74 | #endif |
| @@ -130,29 +130,6 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev) · 129 unmodified lines | ||
| 130 | 130 | spin_unlock_irqrestore(&adev->irq.lock, irqflags); |
| 131 | 131 | } |
| 132 | 132 | |
| 133 | −/** | |
| 134 | − * amdgpu_irq_callback - callback from the IH ring | |
| 135 | − * | |
| 136 | − * @adev: amdgpu device pointer | |
| 137 | − * @ih: amdgpu ih ring | |
| 138 | − * | |
| 139 | − * Callback from IH ring processing to handle the entry at the current position | |
| 140 | − * and advance the read pointer. | |
| 141 | − */ | |
| 142 | −static void amdgpu_irq_callback(struct amdgpu_device *adev, | |
| 143 | − struct amdgpu_ih_ring *ih) | |
| 144 | −{ | |
| 145 | − u32 ring_index = ih->rptr >> 2; | |
| 146 | − struct amdgpu_iv_entry entry; | |
| 147 | − | |
| 148 | − entry.iv_entry = (const uint32_t *)&ih->ring[ring_index]; | |
| 149 | − amdgpu_ih_decode_iv(adev, &entry); | |
| 150 | − | |
| 151 | − trace_amdgpu_iv(ih - &adev->irq.ih, &entry); | |
| 152 | − | |
| 153 | − amdgpu_irq_dispatch(adev, &entry); | |
| 154 | −} | |
| 155 | − | |
| 156 | 133 | /** |
| 157 | 134 | * amdgpu_irq_handler - IRQ handler |
| 158 | 135 | * |
| @@ -170,7 +147,7 @@ irqreturn_t amdgpu_irq_handler(int irq, void *arg) · 11 unmodified lines | ||
| 170 | 147 | struct amdgpu_device *adev = dev->dev_private; |
| 171 | 148 | irqreturn_t ret; |
| 172 | 149 | |
| 173 | − ret = amdgpu_ih_process(adev, &adev->irq.ih, amdgpu_irq_callback); | |
| 150 | + ret = amdgpu_ih_process(adev, &adev->irq.ih); | |
| 174 | 151 | if (ret == IRQ_HANDLED) |
| 175 | 152 | pm_runtime_mark_last_busy(dev->dev); |
| 176 | 153 | return ret; |
| @@ -188,7 +165,7 @@ static void amdgpu_irq_handle_ih1(struct work_struct *work) · 11 unmodified lines | ||
| 188 | 165 | struct amdgpu_device *adev = container_of(work, struct amdgpu_device, |
| 189 | 166 | irq.ih1_work); |
| 190 | 167 | |
| 191 | − amdgpu_ih_process(adev, &adev->irq.ih1, amdgpu_irq_callback); | |
| 168 | + amdgpu_ih_process(adev, &adev->irq.ih1); | |
| 192 | 169 | } |
| 193 | 170 | |
| 194 | 171 | /** |
| @@ -203,7 +180,7 @@ static void amdgpu_irq_handle_ih2(struct work_struct *work) · 8 unmodified lines | ||
| 203 | 180 | struct amdgpu_device *adev = container_of(work, struct amdgpu_device, |
| 204 | 181 | irq.ih2_work); |
| 205 | 182 | |
| 206 | − amdgpu_ih_process(adev, &adev->irq.ih2, amdgpu_irq_callback); | |
| 183 | + amdgpu_ih_process(adev, &adev->irq.ih2); | |
| 207 | 184 | } |
| 208 | 185 | |
| 209 | 186 | /** |
| @@ -394,14 +371,23 @@ int amdgpu_irq_add_id(struct amdgpu_device *adev, · 184 unmodified lines | ||
| 394 | 371 | * Dispatches IRQ to IP blocks. |
| 395 | 372 | */ |
| 396 | 373 | void amdgpu_irq_dispatch(struct amdgpu_device *adev, |
| 397 | − struct amdgpu_iv_entry *entry) | |
| 374 | + struct amdgpu_ih_ring *ih) | |
| 398 | 375 | { |
| 399 | − unsigned client_id = entry->client_id; | |
| 400 | − unsigned src_id = entry->src_id; | |
| 376 | + u32 ring_index = ih->rptr >> 2; | |
| 377 | + struct amdgpu_iv_entry entry; | |
| 378 | + unsigned client_id, src_id; | |
| 401 | 379 | struct amdgpu_irq_src *src; |
| 402 | 380 | bool handled = false; |
| 403 | 381 | int r; |
| 404 | 382 | |
| 383 | + entry.iv_entry = (const uint32_t *)&ih->ring[ring_index]; | |
| 384 | + amdgpu_ih_decode_iv(adev, &entry); | |
| 385 | + | |
| 386 | + trace_amdgpu_iv(ih - &adev->irq.ih, &entry); | |
| 387 | + | |
| 388 | + client_id = entry.client_id; | |
| 389 | + src_id = entry.src_id; | |
| 390 | + | |
| 405 | 391 | if (client_id >= AMDGPU_IRQ_CLIENTID_MAX) { |
| 406 | 392 | DRM_DEBUG("Invalid client_id in IV: %d\n", client_id); |
| 407 | 393 | |
| @@ -416,7 +402,7 @@ void amdgpu_irq_dispatch(struct amdgpu_device *adev, · 8 unmodified lines | ||
| 416 | 402 | client_id, src_id); |
| 417 | 403 | |
| 418 | 404 | } else if ((src = adev->irq.client[client_id].sources[src_id])) { |
| 419 | − r = src->funcs->process(adev, src, entry); | |
| 405 | + r = src->funcs->process(adev, src, &entry); | |
| 420 | 406 | if (r < 0) |
| 421 | 407 | DRM_ERROR("error processing interrupt (%d)\n", r); |
| 422 | 408 | else if (r) |
| @@ -428,7 +414,7 @@ void amdgpu_irq_dispatch(struct amdgpu_device *adev, · 5 unmodified lines | ||
| 428 | 414 | |
| 429 | 415 | /* Send it to amdkfd as well if it isn't already handled */ |
| 430 | 416 | if (!handled) |
| 431 | − amdgpu_amdkfd_interrupt(adev, entry->iv_entry); | |
| 417 | + amdgpu_amdkfd_interrupt(adev, entry.iv_entry); | |
| 432 | 418 | } |
| 433 | 419 | |
| 434 | 420 | /** |
| @@ -108,7 +108,7 @@ int amdgpu_irq_add_id(struct amdgpu_device *adev, · 107 unmodified lines | ||
| 108 | 108 | unsigned client_id, unsigned src_id, |
| 109 | 109 | struct amdgpu_irq_src *source); |
| 110 | 110 | void amdgpu_irq_dispatch(struct amdgpu_device *adev, |
| 111 | − struct amdgpu_iv_entry *entry); | |
| 111 | + struct amdgpu_ih_ring *ih); | |
| 112 | 112 | int amdgpu_irq_update(struct amdgpu_device *adev, struct amdgpu_irq_src *src, |
| 113 | 113 | unsigned type); |
| 114 | 114 | int amdgpu_irq_get(struct amdgpu_device *adev, struct amdgpu_irq_src *src, |
| @@ -207,11 +207,12 @@ int amdgpu_driver_load_kms(struct drm_device *dev, unsigned long flags) · 206 unmodified lines | ||
| 207 | 207 | if (!r) { |
| 208 | 208 | acpi_status = amdgpu_acpi_init(adev); |
| 209 | 209 | if (acpi_status) |
| 210 | − dev_dbg(&dev->pdev->dev, | |
| 210 | + dev_dbg(&dev->pdev->dev, | |
| 211 | 211 | "Error during ACPI methods call\n"); |
| 212 | 212 | } |
| 213 | 213 | |
| 214 | 214 | if (amdgpu_device_is_px(dev)) { |
| 215 | + dev_pm_set_driver_flags(dev->dev, DPM_FLAG_NEVER_SKIP); | |
| 215 | 216 | pm_runtime_use_autosuspend(dev->dev); |
| 216 | 217 | pm_runtime_set_autosuspend_delay(dev->dev, 5000); |
| 217 | 218 | pm_runtime_set_active(dev->dev); |
| @@ -406,6 +406,7 @@ struct amdgpu_crtc { · 405 unmodified lines | ||
| 406 | 406 | struct amdgpu_flip_work *pflip_works; |
| 407 | 407 | enum amdgpu_flip_status pflip_status; |
| 408 | 408 | int deferred_flip_completion; |
| 409 | + u64 last_flip_vblank; | |
| 409 | 410 | /* pll sharing */ |
| 410 | 411 | struct amdgpu_atom_ss ss; |
| 411 | 412 | bool ss_enabled; |
| @@ -1284,6 +1284,30 @@ void amdgpu_bo_fence(struct amdgpu_bo *bo, struct dma_fence *fence, · 1283 unmodified lines | ||
| 1284 | 1284 | reservation_object_add_excl_fence(resv, fence); |
| 1285 | 1285 | } |
| 1286 | 1286 | |
| 1287 | +/** | |
| 1288 | + * amdgpu_sync_wait_resv - Wait for BO reservation fences | |
| 1289 | + * | |
| 1290 | + * @bo: buffer object | |
| 1291 | + * @owner: fence owner | |
| 1292 | + * @intr: Whether the wait is interruptible | |
| 1293 | + * | |
| 1294 | + * Returns: | |
| 1295 | + * 0 on success, errno otherwise. | |
| 1296 | + */ | |
| 1297 | +int amdgpu_bo_sync_wait(struct amdgpu_bo *bo, void *owner, bool intr) | |
| 1298 | +{ | |
| 1299 | + struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); | |
| 1300 | + struct amdgpu_sync sync; | |
| 1301 | + int r; | |
| 1302 | + | |
| 1303 | + amdgpu_sync_create(&sync); | |
| 1304 | + amdgpu_sync_resv(adev, &sync, bo->tbo.resv, owner, false); | |
| 1305 | + r = amdgpu_sync_wait(&sync, intr); | |
| 1306 | + amdgpu_sync_free(&sync); | |
| 1307 | + | |
| 1308 | + return r; | |
| 1309 | +} | |
| 1310 | + | |
| 1287 | 1311 | /** |
| 1288 | 1312 | * amdgpu_bo_gpu_offset - return GPU offset of bo |
| 1289 | 1313 | * @bo: amdgpu object for which we query the offset |
| @@ -266,6 +266,7 @@ void amdgpu_bo_move_notify(struct ttm_buffer_object *bo, · 265 unmodified lines | ||
| 266 | 266 | int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo); |
| 267 | 267 | void amdgpu_bo_fence(struct amdgpu_bo *bo, struct dma_fence *fence, |
| 268 | 268 | bool shared); |
| 269 | +int amdgpu_bo_sync_wait(struct amdgpu_bo *bo, void *owner, bool intr); | |
| 269 | 270 | u64 amdgpu_bo_gpu_offset(struct amdgpu_bo *bo); |
| 270 | 271 | int amdgpu_bo_validate(struct amdgpu_bo *bo); |
| 271 | 272 | int amdgpu_bo_restore_shadow(struct amdgpu_bo *shadow, |
| @@ -54,16 +54,20 @@ static int amdgpu_sched_process_priority_override(struct amdgpu_device *adev, · 53 unmodified lines | ||
| 54 | 54 | enum drm_sched_priority priority) |
| 55 | 55 | { |
| 56 | 56 | struct file *filp = fget(fd); |
| 57 | − struct drm_file *file; | |
| 58 | 57 | struct amdgpu_fpriv *fpriv; |
| 59 | 58 | struct amdgpu_ctx *ctx; |
| 60 | 59 | uint32_t id; |
| 60 | + int r; | |
| 61 | 61 | |
| 62 | 62 | if (!filp) |
| 63 | 63 | return -EINVAL; |
| 64 | 64 | |
| 65 | − file = filp->private_data; | |
| 66 | − fpriv = file->driver_priv; | |
| 65 | + r = amdgpu_file_to_fpriv(filp, &fpriv); | |
| 66 | + if (r) { | |
| 67 | + fput(filp); | |
| 68 | + return r; | |
| 69 | + } | |
| 70 | + | |
| 67 | 71 | idr_for_each_entry(&fpriv->ctx_mgr.ctx_handles, ctx, id) |
| 68 | 72 | amdgpu_ctx_priority_override(ctx, priority); |
| 69 | 73 | |
| @@ -72,6 +76,39 @@ static int amdgpu_sched_process_priority_override(struct amdgpu_device *adev, · 2 unmodified lines | ||
| 72 | 76 | return 0; |
| 73 | 77 | } |
| 74 | 78 | |
| 79 | +static int amdgpu_sched_context_priority_override(struct amdgpu_device *adev, | |
| 80 | + int fd, | |
| 81 | + unsigned ctx_id, | |
| 82 | + enum drm_sched_priority priority) | |
| 83 | +{ | |
| 84 | + struct file *filp = fget(fd); | |
| 85 | + struct amdgpu_fpriv *fpriv; | |
| 86 | + struct amdgpu_ctx *ctx; | |
| 87 | + int r; | |
| 88 | + | |
| 89 | + if (!filp) | |
| 90 | + return -EINVAL; | |
| 91 | + | |
| 92 | + r = amdgpu_file_to_fpriv(filp, &fpriv); | |
| 93 | + if (r) { | |
| 94 | + fput(filp); | |
| 95 | + return r; | |
| 96 | + } | |
| 97 | + | |
| 98 | + ctx = amdgpu_ctx_get(fpriv, ctx_id); | |
| 99 | + | |
| 100 | + if (!ctx) { | |
| 101 | + fput(filp); | |
| 102 | + return -EINVAL; | |
| 103 | + } | |
| 104 | + | |
| 105 | + amdgpu_ctx_priority_override(ctx, priority); | |
| 106 | + amdgpu_ctx_put(ctx); | |
| 107 | + fput(filp); | |
| 108 | + | |
| 109 | + return 0; | |
| 110 | +} | |
| 111 | + | |
| 75 | 112 | int amdgpu_sched_ioctl(struct drm_device *dev, void *data, |
| 76 | 113 | struct drm_file *filp) |
| 77 | 114 | { |
| @@ -81,7 +118,7 @@ int amdgpu_sched_ioctl(struct drm_device *dev, void *data, · 3 unmodified lines | ||
| 81 | 118 | int r; |
| 82 | 119 | |
| 83 | 120 | priority = amdgpu_to_sched_priority(args->in.priority); |
| 84 | − if (args->in.flags || priority == DRM_SCHED_PRIORITY_INVALID) | |
| 121 | + if (priority == DRM_SCHED_PRIORITY_INVALID) | |
| 85 | 122 | return -EINVAL; |
| 86 | 123 | |
| 87 | 124 | switch (args->in.op) { |
| @@ -90,6 +127,12 @@ int amdgpu_sched_ioctl(struct drm_device *dev, void *data, · 2 unmodified lines | ||
| 90 | 127 | args->in.fd, |
| 91 | 128 | priority); |
| 92 | 129 | break; |
| 130 | + case AMDGPU_SCHED_OP_CONTEXT_PRIORITY_OVERRIDE: | |
| 131 | + r = amdgpu_sched_context_priority_override(adev, | |
| 132 | + args->in.fd, | |
| 133 | + args->in.ctx_id, | |
| 134 | + priority); | |
| 135 | + break; | |
| 93 | 136 | default: |
| 94 | 137 | DRM_ERROR("Invalid sched op specified: %d\n", args->in.op); |
| 95 | 138 | r = -EINVAL; |
| @@ -652,12 +652,14 @@ void amdgpu_vm_move_to_lru_tail(struct amdgpu_device *adev, · 651 unmodified lines | ||
| 652 | 652 | struct ttm_bo_global *glob = adev->mman.bdev.glob; |
| 653 | 653 | struct amdgpu_vm_bo_base *bo_base; |
| 654 | 654 | |
| 655 | +#if 0 | |
| 655 | 656 | if (vm->bulk_moveable) { |
| 656 | 657 | spin_lock(&glob->lru_lock); |
| 657 | 658 | ttm_bo_bulk_move_lru_tail(&vm->lru_bulk_move); |
| 658 | 659 | spin_unlock(&glob->lru_lock); |
| 659 | 660 | return; |
| 660 | 661 | } |
| 662 | +#endif | |
| 661 | 663 | |
| 662 | 664 | memset(&vm->lru_bulk_move, 0, sizeof(vm->lru_bulk_move)); |
| 663 | 665 | |
| @@ -698,8 +700,6 @@ int amdgpu_vm_validate_pt_bos(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 34 unmodified lines | ||
| 698 | 700 | struct amdgpu_vm_bo_base *bo_base, *tmp; |
| 699 | 701 | int r = 0; |
| 700 | 702 | |
| 701 | − vm->bulk_moveable &= list_empty(&vm->evicted); | |
| 702 | − | |
| 703 | 703 | list_for_each_entry_safe(bo_base, tmp, &vm->evicted, vm_status) { |
| 704 | 704 | struct amdgpu_bo *bo = bo_base->bo; |
| 705 | 705 | |
| @@ -828,7 +828,7 @@ static int amdgpu_vm_clear_bo(struct amdgpu_device *adev, · 122 unmodified lines | ||
| 828 | 828 | |
| 829 | 829 | WARN_ON(job->ibs[0].length_dw > 64); |
| 830 | 830 | r = amdgpu_sync_resv(adev, &job->sync, bo->tbo.resv, |
| 831 | − AMDGPU_FENCE_OWNER_UNDEFINED, false); | |
| 831 | + AMDGPU_FENCE_OWNER_KFD, false); | |
| 832 | 832 | if (r) |
| 833 | 833 | goto error_free; |
| 834 | 834 | |
| @@ -1332,31 +1332,6 @@ static void amdgpu_vm_cpu_set_ptes(struct amdgpu_pte_update_params *params, · 497 unmodified lines | ||
| 1332 | 1332 | } |
| 1333 | 1333 | } |
| 1334 | 1334 | |
| 1335 | − | |
| 1336 | −/** | |
| 1337 | − * amdgpu_vm_wait_pd - Wait for PT BOs to be free. | |
| 1338 | − * | |
| 1339 | − * @adev: amdgpu_device pointer | |
| 1340 | − * @vm: related vm | |
| 1341 | − * @owner: fence owner | |
| 1342 | − * | |
| 1343 | − * Returns: | |
| 1344 | − * 0 on success, errno otherwise. | |
| 1345 | − */ | |
| 1346 | −static int amdgpu_vm_wait_pd(struct amdgpu_device *adev, struct amdgpu_vm *vm, | |
| 1347 | − void *owner) | |
| 1348 | −{ | |
| 1349 | − struct amdgpu_sync sync; | |
| 1350 | − int r; | |
| 1351 | − | |
| 1352 | − amdgpu_sync_create(&sync); | |
| 1353 | − amdgpu_sync_resv(adev, &sync, vm->root.base.bo->tbo.resv, owner, false); | |
| 1354 | − r = amdgpu_sync_wait(&sync, true); | |
| 1355 | − amdgpu_sync_free(&sync); | |
| 1356 | − | |
| 1357 | − return r; | |
| 1358 | −} | |
| 1359 | − | |
| 1360 | 1335 | /** |
| 1361 | 1336 | * amdgpu_vm_update_func - helper to call update function |
| 1362 | 1337 | * |
| @@ -1451,7 +1426,8 @@ restart: · 88 unmodified lines | ||
| 1451 | 1426 | params.adev = adev; |
| 1452 | 1427 | |
| 1453 | 1428 | if (vm->use_cpu_for_update) { |
| 1454 | − r = amdgpu_vm_wait_pd(adev, vm, AMDGPU_FENCE_OWNER_VM); | |
| 1429 | + r = amdgpu_bo_sync_wait(vm->root.base.bo, | |
| 1430 | + AMDGPU_FENCE_OWNER_VM, true); | |
| 1455 | 1431 | if (unlikely(r)) |
| 1456 | 1432 | return r; |
| 1457 | 1433 | |
| @@ -1772,9 +1748,9 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, · 314 unmodified lines | ||
| 1772 | 1748 | params.adev = adev; |
| 1773 | 1749 | params.vm = vm; |
| 1774 | 1750 | |
| 1775 | − /* sync to everything on unmapping */ | |
| 1751 | + /* sync to everything except eviction fences on unmapping */ | |
| 1776 | 1752 | if (!(flags & AMDGPU_PTE_VALID)) |
| 1777 | − owner = AMDGPU_FENCE_OWNER_UNDEFINED; | |
| 1753 | + owner = AMDGPU_FENCE_OWNER_KFD; | |
| 1778 | 1754 | |
| 1779 | 1755 | if (vm->use_cpu_for_update) { |
| 1780 | 1756 | /* params.src is used as flag to indicate system Memory */ |
| @@ -1784,7 +1760,7 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, · 3 unmodified lines | ||
| 1784 | 1760 | /* Wait for PT BOs to be idle. PTs share the same resv. object |
| 1785 | 1761 | * as the root PD BO |
| 1786 | 1762 | */ |
| 1787 | − r = amdgpu_vm_wait_pd(adev, vm, owner); | |
| 1763 | + r = amdgpu_bo_sync_wait(vm->root.base.bo, owner, true); | |
| 1788 | 1764 | if (unlikely(r)) |
| 1789 | 1765 | return r; |
| 1790 | 1766 | |
| @@ -2980,7 +2980,7 @@ static int dce_v6_0_pageflip_irq(struct amdgpu_device *adev, · 2979 unmodified lines | ||
| 2980 | 2980 | struct amdgpu_irq_src *source, |
| 2981 | 2981 | struct amdgpu_iv_entry *entry) |
| 2982 | 2982 | { |
| 2983 | − unsigned long flags; | |
| 2983 | + unsigned long flags; | |
| 2984 | 2984 | unsigned crtc_id; |
| 2985 | 2985 | struct amdgpu_crtc *amdgpu_crtc; |
| 2986 | 2986 | struct amdgpu_flip_work *works; |
| @@ -266,7 +266,8 @@ flr_done: · 265 unmodified lines | ||
| 266 | 266 | } |
| 267 | 267 | |
| 268 | 268 | /* Trigger recovery for world switch failure if no TDR */ |
| 269 | − if (amdgpu_device_should_recover_gpu(adev)) | |
| 269 | + if (amdgpu_device_should_recover_gpu(adev) | |
| 270 | + && amdgpu_lockup_timeout == MAX_SCHEDULE_TIMEOUT) | |
| 270 | 271 | amdgpu_device_gpu_recover(adev, NULL); |
| 271 | 272 | } |
| 272 | 273 | |
| @@ -32,7 +32,7 @@ · 31 unmodified lines | ||
| 32 | 32 | |
| 33 | 33 | static u32 nbio_v7_4_get_rev_id(struct amdgpu_device *adev) |
| 34 | 34 | { |
| 35 | − u32 tmp = RREG32_SOC15(NBIO, 0, mmRCC_DEV0_EPF0_STRAP0); | |
| 35 | + u32 tmp = RREG32_SOC15(NBIO, 0, mmRCC_DEV0_EPF0_STRAP0); | |
| 36 | 36 | |
| 37 | 37 | tmp &= RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0_MASK; |
| 38 | 38 | tmp >>= RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0__SHIFT; |
| @@ -128,7 +128,7 @@ static const struct soc15_reg_golden golden_settings_sdma0_4_2_init[] = { · 127 unmodified lines | ||
| 128 | 128 | |
| 129 | 129 | static const struct soc15_reg_golden golden_settings_sdma0_4_2[] = |
| 130 | 130 | { |
| 131 | − SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_CHICKEN_BITS, 0xfe931f07, 0x02831d07), | |
| 131 | + SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_CHICKEN_BITS, 0xfe931f07, 0x02831f07), | |
| 132 | 132 | SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_CLK_CTRL, 0xffffffff, 0x3f000100), |
| 133 | 133 | SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_GB_ADDR_CONFIG, 0x0000773f, 0x00004002), |
| 134 | 134 | SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_GB_ADDR_CONFIG_READ, 0x0000773f, 0x00004002), |
| @@ -158,7 +158,7 @@ static const struct soc15_reg_golden golden_settings_sdma0_4_2[] = · 23 unmodified lines | ||
| 158 | 158 | }; |
| 159 | 159 | |
| 160 | 160 | static const struct soc15_reg_golden golden_settings_sdma1_4_2[] = { |
| 161 | − SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_CHICKEN_BITS, 0xfe931f07, 0x02831d07), | |
| 161 | + SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_CHICKEN_BITS, 0xfe931f07, 0x02831f07), | |
| 162 | 162 | SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_CLK_CTRL, 0xffffffff, 0x3f000100), |
| 163 | 163 | SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_GB_ADDR_CONFIG, 0x0000773f, 0x00004002), |
| 164 | 164 | SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_GB_ADDR_CONFIG_READ, 0x0000773f, 0x00004002), |
| @@ -1436,7 +1436,7 @@ static int si_common_early_init(void *handle) · 1435 unmodified lines | ||
| 1436 | 1436 | AMD_CG_SUPPORT_UVD_MGCG | |
| 1437 | 1437 | AMD_CG_SUPPORT_HDP_LS | |
| 1438 | 1438 | AMD_CG_SUPPORT_HDP_MGCG; |
| 1439 | − adev->pg_flags = 0; | |
| 1439 | + adev->pg_flags = 0; | |
| 1440 | 1440 | adev->external_rev_id = (adev->rev_id == 0) ? 1 : |
| 1441 | 1441 | (adev->rev_id == 1) ? 5 : 6; |
| 1442 | 1442 | break; |
| @@ -6216,10 +6216,12 @@ static void si_request_link_speed_change_before_state_change(struct amdgpu_devic · 6215 unmodified lines | ||
| 6216 | 6216 | si_pi->force_pcie_gen = AMDGPU_PCIE_GEN2; |
| 6217 | 6217 | if (current_link_speed == AMDGPU_PCIE_GEN2) |
| 6218 | 6218 | break; |
| 6219 | + /* fall through */ | |
| 6219 | 6220 | case AMDGPU_PCIE_GEN2: |
| 6220 | 6221 | if (amdgpu_acpi_pcie_performance_request(adev, PCIE_PERF_REQ_PECI_GEN2, false) == 0) |
| 6221 | 6222 | break; |
| 6222 | 6223 | #endif |
| 6224 | + /* fall through */ | |
| 6223 | 6225 | default: |
| 6224 | 6226 | si_pi->force_pcie_gen = si_get_current_pcie_speed(adev); |
| 6225 | 6227 | break; |
| @@ -81,6 +81,10 @@ void vega10_doorbell_index_init(struct amdgpu_device *adev) · 80 unmodified lines | ||
| 81 | 81 | adev->doorbell_index.uvd_vce.vce_ring2_3 = AMDGPU_DOORBELL64_VCE_RING2_3; |
| 82 | 82 | adev->doorbell_index.uvd_vce.vce_ring4_5 = AMDGPU_DOORBELL64_VCE_RING4_5; |
| 83 | 83 | adev->doorbell_index.uvd_vce.vce_ring6_7 = AMDGPU_DOORBELL64_VCE_RING6_7; |
| 84 | + | |
| 85 | + adev->doorbell_index.first_non_cp = AMDGPU_DOORBELL64_FIRST_NON_CP; | |
| 86 | + adev->doorbell_index.last_non_cp = AMDGPU_DOORBELL64_LAST_NON_CP; | |
| 87 | + | |
| 84 | 88 | /* In unit of dword doorbell */ |
| 85 | 89 | adev->doorbell_index.max_assignment = AMDGPU_DOORBELL64_MAX_ASSIGNMENT << 1; |
| 86 | 90 | adev->doorbell_index.sdma_doorbell_range = 4; |
| @@ -85,6 +85,10 @@ void vega20_doorbell_index_init(struct amdgpu_device *adev) · 84 unmodified lines | ||
| 85 | 85 | adev->doorbell_index.uvd_vce.vce_ring2_3 = AMDGPU_VEGA20_DOORBELL64_VCE_RING2_3; |
| 86 | 86 | adev->doorbell_index.uvd_vce.vce_ring4_5 = AMDGPU_VEGA20_DOORBELL64_VCE_RING4_5; |
| 87 | 87 | adev->doorbell_index.uvd_vce.vce_ring6_7 = AMDGPU_VEGA20_DOORBELL64_VCE_RING6_7; |
| 88 | + | |
| 89 | + adev->doorbell_index.first_non_cp = AMDGPU_VEGA20_DOORBELL64_FIRST_NON_CP; | |
| 90 | + adev->doorbell_index.last_non_cp = AMDGPU_VEGA20_DOORBELL64_LAST_NON_CP; | |
| 91 | + | |
| 88 | 92 | adev->doorbell_index.max_assignment = AMDGPU_VEGA20_DOORBELL_MAX_ASSIGNMENT << 1; |
| 89 | 93 | adev->doorbell_index.sdma_doorbell_range = 20; |
| 90 | 94 | } |
| @@ -134,12 +134,18 @@ static int allocate_doorbell(struct qcm_process_device *qpd, struct queue *q) · 133 unmodified lines | ||
| 134 | 134 | */ |
| 135 | 135 | q->doorbell_id = q->properties.queue_id; |
| 136 | 136 | } else if (q->properties.type == KFD_QUEUE_TYPE_SDMA) { |
| 137 | − /* For SDMA queues on SOC15, use static doorbell | |
| 138 | − * assignments based on the engine and queue. | |
| 137 | + /* For SDMA queues on SOC15 with 8-byte doorbell, use static | |
| 138 | + * doorbell assignments based on the engine and queue id. | |
| 139 | + * The doobell index distance between RLC (2*i) and (2*i+1) | |
| 140 | + * for a SDMA engine is 512. | |
| 139 | 141 | */ |
| 140 | − q->doorbell_id = dev->shared_resources.sdma_doorbell | |
| 141 | − [q->properties.sdma_engine_id] | |
| 142 | − [q->properties.sdma_queue_id]; | |
| 142 | + uint32_t *idx_offset = | |
| 143 | + dev->shared_resources.sdma_doorbell_idx; | |
| 144 | + | |
| 145 | + q->doorbell_id = idx_offset[q->properties.sdma_engine_id] | |
| 146 | + + (q->properties.sdma_queue_id & 1) | |
| 147 | + * KFD_QUEUE_DOORBELL_MIRROR_OFFSET | |
| 148 | + + (q->properties.sdma_queue_id >> 1); | |
| 143 | 149 | } else { |
| 144 | 150 | /* For CP queues on SOC15 reserve a free doorbell ID */ |
| 145 | 151 | unsigned int found; |
| @@ -97,17 +97,29 @@ · 96 unmodified lines | ||
| 97 | 97 | #define KFD_CWSR_TBA_TMA_SIZE (PAGE_SIZE * 2) |
| 98 | 98 | #define KFD_CWSR_TMA_OFFSET PAGE_SIZE |
| 99 | 99 | |
| 100 | +#define KFD_MAX_NUM_OF_QUEUES_PER_DEVICE \ | |
| 101 | + (KFD_MAX_NUM_OF_PROCESSES * \ | |
| 102 | + KFD_MAX_NUM_OF_QUEUES_PER_PROCESS) | |
| 103 | + | |
| 104 | +#define KFD_KERNEL_QUEUE_SIZE 2048 | |
| 105 | + | |
| 106 | +/* | |
| 107 | + * 512 = 0x200 | |
| 108 | + * The doorbell index distance between SDMA RLC (2*i) and (2*i+1) in the | |
| 109 | + * same SDMA engine on SOC15, which has 8-byte doorbells for SDMA. | |
| 110 | + * 512 8-byte doorbell distance (i.e. one page away) ensures that SDMA RLC | |
| 111 | + * (2*i+1) doorbells (in terms of the lower 12 bit address) lie exactly in | |
| 112 | + * the OFFSET and SIZE set in registers like BIF_SDMA0_DOORBELL_RANGE. | |
| 113 | + */ | |
| 114 | +#define KFD_QUEUE_DOORBELL_MIRROR_OFFSET 512 | |
| 115 | + | |
| 116 | + | |
| 100 | 117 | /* |
| 101 | 118 | * Kernel module parameter to specify maximum number of supported queues per |
| 102 | 119 | * device |
| 103 | 120 | */ |
| 104 | 121 | extern int max_num_of_queues_per_device; |
| 105 | 122 | |
| 106 | −#define KFD_MAX_NUM_OF_QUEUES_PER_DEVICE \ | |
| 107 | − (KFD_MAX_NUM_OF_PROCESSES * \ | |
| 108 | − KFD_MAX_NUM_OF_QUEUES_PER_PROCESS) | |
| 109 | − | |
| 110 | −#define KFD_KERNEL_QUEUE_SIZE 2048 | |
| 111 | 123 | |
| 112 | 124 | /* Kernel module parameter to specify the scheduling policy */ |
| 113 | 125 | extern int sched_policy; |
| @@ -607,13 +607,17 @@ static int init_doorbell_bitmap(struct qcm_process_device *qpd, · 606 unmodified lines | ||
| 607 | 607 | if (!qpd->doorbell_bitmap) |
| 608 | 608 | return -ENOMEM; |
| 609 | 609 | |
| 610 | − /* Mask out any reserved doorbells */ | |
| 611 | − for (i = 0; i < KFD_MAX_NUM_OF_QUEUES_PER_PROCESS; i++) | |
| 612 | − if ((dev->shared_resources.reserved_doorbell_mask & i) == | |
| 613 | − dev->shared_resources.reserved_doorbell_val) { | |
| 610 | + /* Mask out doorbells reserved for SDMA, IH, and VCN on SOC15. */ | |
| 611 | + for (i = 0; i < KFD_MAX_NUM_OF_QUEUES_PER_PROCESS / 2; i++) { | |
| 612 | + if (i >= dev->shared_resources.non_cp_doorbells_start | |
| 613 | + && i <= dev->shared_resources.non_cp_doorbells_end) { | |
| 614 | 614 | set_bit(i, qpd->doorbell_bitmap); |
| 615 | − pr_debug("reserved doorbell 0x%03x\n", i); | |
| 615 | + set_bit(i + KFD_QUEUE_DOORBELL_MIRROR_OFFSET, | |
| 616 | + qpd->doorbell_bitmap); | |
| 617 | + pr_debug("reserved doorbell 0x%03x and 0x%03x\n", i, | |
| 618 | + i + KFD_QUEUE_DOORBELL_MIRROR_OFFSET); | |
| 616 | 619 | } |
| 620 | + } | |
| 617 | 621 | |
| 618 | 622 | return 0; |
| 619 | 623 | } |
| @@ -303,12 +303,11 @@ static void dm_pflip_high_irq(void *interrupt_params) · 302 unmodified lines | ||
| 303 | 303 | return; |
| 304 | 304 | } |
| 305 | 305 | |
| 306 | + /* Update to correct count(s) if racing with vblank irq */ | |
| 307 | + amdgpu_crtc->last_flip_vblank = drm_crtc_accurate_vblank_count(&amdgpu_crtc->base); | |
| 306 | 308 | |
| 307 | 309 | /* wake up userspace */ |
| 308 | 310 | if (amdgpu_crtc->event) { |
| 309 | − /* Update to correct count(s) if racing with vblank irq */ | |
| 310 | − drm_crtc_accurate_vblank_count(&amdgpu_crtc->base); | |
| 311 | − | |
| 312 | 311 | drm_crtc_send_vblank_event(&amdgpu_crtc->base, amdgpu_crtc->event); |
| 313 | 312 | |
| 314 | 313 | /* page flip completed. clean up */ |
| @@ -786,12 +785,13 @@ static int dm_suspend(void *handle) · 471 unmodified lines | ||
| 786 | 785 | struct amdgpu_display_manager *dm = &adev->dm; |
| 787 | 786 | int ret = 0; |
| 788 | 787 | |
| 788 | + WARN_ON(adev->dm.cached_state); | |
| 789 | + adev->dm.cached_state = drm_atomic_helper_suspend(adev->ddev); | |
| 790 | + | |
| 789 | 791 | s3_handle_mst(adev->ddev, true); |
| 790 | 792 | |
| 791 | 793 | amdgpu_dm_irq_suspend(adev); |
| 792 | 794 | |
| 793 | − WARN_ON(adev->dm.cached_state); | |
| 794 | − adev->dm.cached_state = drm_atomic_helper_suspend(adev->ddev); | |
| 795 | 795 | |
| 796 | 796 | dc_set_power_state(dm->dc, DC_ACPI_CM_POWER_STATE_D3); |
| 797 | 797 | |
| @@ -3790,7 +3790,6 @@ static const struct drm_plane_helper_funcs dm_plane_helper_funcs = { · 2992 unmodified lines | ||
| 3790 | 3790 | * check will succeed, and let DC implement proper check |
| 3791 | 3791 | */ |
| 3792 | 3792 | static const uint32_t rgb_formats[] = { |
| 3793 | − DRM_FORMAT_RGB888, | |
| 3794 | 3793 | DRM_FORMAT_XRGB8888, |
| 3795 | 3794 | DRM_FORMAT_ARGB8888, |
| 3796 | 3795 | DRM_FORMAT_RGBA8888, |
| @@ -4646,6 +4645,8 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state, · 849 unmodified lines | ||
| 4646 | 4645 | struct amdgpu_bo *abo; |
| 4647 | 4646 | uint64_t tiling_flags, dcc_address; |
| 4648 | 4647 | uint32_t target, target_vblank; |
| 4648 | + uint64_t last_flip_vblank; | |
| 4649 | + bool vrr_active = acrtc_state->freesync_config.state == VRR_STATE_ACTIVE_VARIABLE; | |
| 4649 | 4650 | |
| 4650 | 4651 | struct { |
| 4651 | 4652 | struct dc_surface_update surface_updates[MAX_SURFACES]; |
| @@ -4678,10 +4679,9 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state, · 26 unmodified lines | ||
| 4678 | 4679 | struct dc_plane_state *dc_plane; |
| 4679 | 4680 | struct dm_plane_state *dm_new_plane_state = to_dm_plane_state(new_plane_state); |
| 4680 | 4681 | |
| 4681 | − if (plane->type == DRM_PLANE_TYPE_CURSOR) { | |
| 4682 | − handle_cursor_update(plane, old_plane_state); | |
| 4682 | + /* Cursor plane is handled after stream updates */ | |
| 4683 | + if (plane->type == DRM_PLANE_TYPE_CURSOR) | |
| 4683 | 4684 | continue; |
| 4684 | − } | |
| 4685 | 4685 | |
| 4686 | 4686 | if (!fb || !crtc || pcrtc != crtc) |
| 4687 | 4687 | continue; |
| @@ -4712,14 +4712,21 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state, · 24 unmodified lines | ||
| 4712 | 4712 | */ |
| 4713 | 4713 | abo = gem_to_amdgpu_bo(fb->obj[0]); |
| 4714 | 4714 | r = amdgpu_bo_reserve(abo, true); |
| 4715 | − if (unlikely(r != 0)) { | |
| 4715 | + if (unlikely(r != 0)) | |
| 4716 | 4716 | DRM_ERROR("failed to reserve buffer before flip\n"); |
| 4717 | − WARN_ON(1); | |
| 4718 | − } | |
| 4719 | 4717 | |
| 4720 | − /* Wait for all fences on this FB */ | |
| 4721 | − WARN_ON(reservation_object_wait_timeout_rcu(abo->tbo.resv, true, false, | |
| 4722 | − MAX_SCHEDULE_TIMEOUT) < 0); | |
| 4718 | + /* | |
| 4719 | + * Wait for all fences on this FB. Do limited wait to avoid | |
| 4720 | + * deadlock during GPU reset when this fence will not signal | |
| 4721 | + * but we hold reservation lock for the BO. | |
| 4722 | + */ | |
| 4723 | + r = reservation_object_wait_timeout_rcu(abo->tbo.resv, | |
| 4724 | + true, false, | |
| 4725 | + msecs_to_jiffies(5000)); | |
| 4726 | + if (unlikely(r == 0)) | |
| 4727 | + DRM_ERROR("Waiting for fences timed out."); | |
| 4728 | + | |
| 4729 | + | |
| 4723 | 4730 | |
| 4724 | 4731 | amdgpu_bo_get_tiling_flags(abo, &tiling_flags); |
| 4725 | 4732 | |
| @@ -4799,7 +4806,31 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state, · 73 unmodified lines | ||
| 4799 | 4806 | * hopefully eliminating dc_*_update structs in their entirety. |
| 4800 | 4807 | */ |
| 4801 | 4808 | if (flip_count) { |
| 4802 | − target = (uint32_t)drm_crtc_vblank_count(pcrtc) + *wait_for_vblank; | |
| 4809 | + if (!vrr_active) { | |
| 4810 | + /* Use old throttling in non-vrr fixed refresh rate mode | |
| 4811 | + * to keep flip scheduling based on target vblank counts | |
| 4812 | + * working in a backwards compatible way, e.g., for | |
| 4813 | + * clients using the GLX_OML_sync_control extension or | |
| 4814 | + * DRI3/Present extension with defined target_msc. | |
| 4815 | + */ | |
| 4816 | + last_flip_vblank = drm_crtc_vblank_count(pcrtc); | |
| 4817 | + } | |
| 4818 | + else { | |
| 4819 | + /* For variable refresh rate mode only: | |
| 4820 | + * Get vblank of last completed flip to avoid > 1 vrr | |
| 4821 | + * flips per video frame by use of throttling, but allow | |
| 4822 | + * flip programming anywhere in the possibly large | |
| 4823 | + * variable vrr vblank interval for fine-grained flip | |
| 4824 | + * timing control and more opportunity to avoid stutter | |
| 4825 | + * on late submission of flips. | |
| 4826 | + */ | |
| 4827 | + spin_lock_irqsave(&pcrtc->dev->event_lock, flags); | |
| 4828 | + last_flip_vblank = acrtc_attach->last_flip_vblank; | |
| 4829 | + spin_unlock_irqrestore(&pcrtc->dev->event_lock, flags); | |
| 4830 | + } | |
| 4831 | + | |
| 4832 | + target = (uint32_t)last_flip_vblank + *wait_for_vblank; | |
| 4833 | + | |
| 4803 | 4834 | /* Prepare wait for target vblank early - before the fence-waits */ |
| 4804 | 4835 | target_vblank = target - (uint32_t)drm_crtc_vblank_count(pcrtc) + |
| 4805 | 4836 | amdgpu_get_vblank_counter_kms(pcrtc->dev, acrtc_attach->crtc_id); |
| @@ -4874,6 +4905,10 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state, · 68 unmodified lines | ||
| 4874 | 4905 | mutex_unlock(&dm->dc_lock); |
| 4875 | 4906 | } |
| 4876 | 4907 | |
| 4908 | + for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, i) | |
| 4909 | + if (plane->type == DRM_PLANE_TYPE_CURSOR) | |
| 4910 | + handle_cursor_update(plane, old_plane_state); | |
| 4911 | + | |
| 4877 | 4912 | cleanup: |
| 4878 | 4913 | kfree(flip); |
| 4879 | 4914 | kfree(full); |
| @@ -5799,14 +5834,13 @@ dm_determine_update_type_for_commit(struct dc *dc, · 919 unmodified lines | ||
| 5799 | 5834 | old_dm_crtc_state = to_dm_crtc_state(old_crtc_state); |
| 5800 | 5835 | num_plane = 0; |
| 5801 | 5836 | |
| 5802 | − if (!new_dm_crtc_state->stream) { | |
| 5803 | − if (!new_dm_crtc_state->stream && old_dm_crtc_state->stream) { | |
| 5804 | − update_type = UPDATE_TYPE_FULL; | |
| 5805 | − goto cleanup; | |
| 5806 | − } | |
| 5837 | + if (new_dm_crtc_state->stream != old_dm_crtc_state->stream) { | |
| 5838 | + update_type = UPDATE_TYPE_FULL; | |
| 5839 | + goto cleanup; | |
| 5840 | + } | |
| 5807 | 5841 | |
| 5842 | + if (!new_dm_crtc_state->stream) | |
| 5808 | 5843 | continue; |
| 5809 | − } | |
| 5810 | 5844 | |
| 5811 | 5845 | for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, j) { |
| 5812 | 5846 | new_plane_crtc = new_plane_state->crtc; |
| @@ -5817,6 +5851,11 @@ dm_determine_update_type_for_commit(struct dc *dc, · 4 unmodified lines | ||
| 5817 | 5851 | if (plane->type == DRM_PLANE_TYPE_CURSOR) |
| 5818 | 5852 | continue; |
| 5819 | 5853 | |
| 5854 | + if (new_dm_plane_state->dc_state != old_dm_plane_state->dc_state) { | |
| 5855 | + update_type = UPDATE_TYPE_FULL; | |
| 5856 | + goto cleanup; | |
| 5857 | + } | |
| 5858 | + | |
| 5820 | 5859 | if (!state->allow_modeset) |
| 5821 | 5860 | continue; |
| 5822 | 5861 | |
| @@ -5955,6 +5994,42 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev, · 132 unmodified lines | ||
| 5955 | 5994 | goto fail; |
| 5956 | 5995 | } |
| 5957 | 5996 | |
| 5997 | + /* | |
| 5998 | + * Add all primary and overlay planes on the CRTC to the state | |
| 5999 | + * whenever a plane is enabled to maintain correct z-ordering | |
| 6000 | + * and to enable fast surface updates. | |
| 6001 | + */ | |
| 6002 | + drm_for_each_crtc(crtc, dev) { | |
| 6003 | + bool modified = false; | |
| 6004 | + | |
| 6005 | + for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, i) { | |
| 6006 | + if (plane->type == DRM_PLANE_TYPE_CURSOR) | |
| 6007 | + continue; | |
| 6008 | + | |
| 6009 | + if (new_plane_state->crtc == crtc || | |
| 6010 | + old_plane_state->crtc == crtc) { | |
| 6011 | + modified = true; | |
| 6012 | + break; | |
| 6013 | + } | |
| 6014 | + } | |
| 6015 | + | |
| 6016 | + if (!modified) | |
| 6017 | + continue; | |
| 6018 | + | |
| 6019 | + drm_for_each_plane_mask(plane, state->dev, crtc->state->plane_mask) { | |
| 6020 | + if (plane->type == DRM_PLANE_TYPE_CURSOR) | |
| 6021 | + continue; | |
| 6022 | + | |
| 6023 | + new_plane_state = | |
| 6024 | + drm_atomic_get_plane_state(state, plane); | |
| 6025 | + | |
| 6026 | + if (IS_ERR(new_plane_state)) { | |
| 6027 | + ret = PTR_ERR(new_plane_state); | |
| 6028 | + goto fail; | |
| 6029 | + } | |
| 6030 | + } | |
| 6031 | + } | |
| 6032 | + | |
| 5958 | 6033 | /* Remove exiting planes if they are modified */ |
| 5959 | 6034 | for_each_oldnew_plane_in_state_reverse(state, plane, old_plane_state, new_plane_state, i) { |
| 5960 | 6035 | ret = dm_update_plane_state(dc, state, plane, |
| @@ -265,6 +265,7 @@ static struct atom_display_object_path_v2 *get_bios_object( · 264 unmodified lines | ||
| 265 | 265 | && id.enum_id == obj_id.enum_id) |
| 266 | 266 | return &bp->object_info_tbl.v1_4->display_path[i]; |
| 267 | 267 | } |
| 268 | + /* fall through */ | |
| 268 | 269 | case OBJECT_TYPE_CONNECTOR: |
| 269 | 270 | case OBJECT_TYPE_GENERIC: |
| 270 | 271 | /* Both Generic and Connector Object ID |
| @@ -277,6 +278,7 @@ static struct atom_display_object_path_v2 *get_bios_object( · 6 unmodified lines | ||
| 277 | 278 | && id.enum_id == obj_id.enum_id) |
| 278 | 279 | return &bp->object_info_tbl.v1_4->display_path[i]; |
| 279 | 280 | } |
| 281 | + /* fall through */ | |
| 280 | 282 | default: |
| 281 | 283 | return NULL; |
| 282 | 284 | } |
| @@ -1138,6 +1138,9 @@ static enum dc_status dc_commit_state_no_check(struct dc *dc, struct dc_state *c · 1137 unmodified lines | ||
| 1138 | 1138 | /* pplib is notified if disp_num changed */ |
| 1139 | 1139 | dc->hwss.optimize_bandwidth(dc, context); |
| 1140 | 1140 | |
| 1141 | + for (i = 0; i < context->stream_count; i++) | |
| 1142 | + context->streams[i]->mode_changed = false; | |
| 1143 | + | |
| 1141 | 1144 | dc_release_state(dc->current_state); |
| 1142 | 1145 | |
| 1143 | 1146 | dc->current_state = context; |
| @@ -1623,13 +1626,13 @@ static void commit_planes_do_stream_update(struct dc *dc, · 479 unmodified lines | ||
| 1623 | 1626 | stream_update->adjust->v_total_min, |
| 1624 | 1627 | stream_update->adjust->v_total_max); |
| 1625 | 1628 | |
| 1626 | − if (stream_update->periodic_vsync_config && pipe_ctx->stream_res.tg->funcs->program_vline_interrupt) | |
| 1627 | − pipe_ctx->stream_res.tg->funcs->program_vline_interrupt( | |
| 1628 | − pipe_ctx->stream_res.tg, &pipe_ctx->stream->timing, VLINE0, &stream->periodic_vsync_config); | |
| 1629 | + if (stream_update->periodic_interrupt0 && | |
| 1630 | + dc->hwss.setup_periodic_interrupt) | |
| 1631 | + dc->hwss.setup_periodic_interrupt(pipe_ctx, VLINE0); | |
| 1629 | 1632 | |
| 1630 | − if (stream_update->enhanced_sync_config && pipe_ctx->stream_res.tg->funcs->program_vline_interrupt) | |
| 1631 | − pipe_ctx->stream_res.tg->funcs->program_vline_interrupt( | |
| 1632 | − pipe_ctx->stream_res.tg, &pipe_ctx->stream->timing, VLINE1, &stream->enhanced_sync_config); | |
| 1633 | + if (stream_update->periodic_interrupt1 && | |
| 1634 | + dc->hwss.setup_periodic_interrupt) | |
| 1635 | + dc->hwss.setup_periodic_interrupt(pipe_ctx, VLINE1); | |
| 1633 | 1636 | |
| 1634 | 1637 | if ((stream_update->hdr_static_metadata && !stream->use_dynamic_meta) || |
| 1635 | 1638 | stream_update->vrr_infopacket || |
| @@ -51,9 +51,19 @@ struct freesync_context { · 50 unmodified lines | ||
| 51 | 51 | bool dummy; |
| 52 | 52 | }; |
| 53 | 53 | |
| 54 | −union vline_config { | |
| 55 | − unsigned int line_number; | |
| 56 | − unsigned long long delta_in_ns; | |
| 54 | +enum vertical_interrupt_ref_point { | |
| 55 | + START_V_UPDATE = 0, | |
| 56 | + START_V_SYNC, | |
| 57 | + INVALID_POINT | |
| 58 | + | |
| 59 | + //For now, only v_update interrupt is used. | |
| 60 | + //START_V_BLANK, | |
| 61 | + //START_V_ACTIVE | |
| 62 | +}; | |
| 63 | + | |
| 64 | +struct periodic_interrupt_config { | |
| 65 | + enum vertical_interrupt_ref_point ref_point; | |
| 66 | + int lines_offset; | |
| 57 | 67 | }; |
| 58 | 68 | |
| 59 | 69 | |
| @@ -106,8 +116,8 @@ struct dc_stream_state { · 46 unmodified lines | ||
| 106 | 116 | /* DMCU info */ |
| 107 | 117 | unsigned int abm_level; |
| 108 | 118 | |
| 109 | − union vline_config periodic_vsync_config; | |
| 110 | − union vline_config enhanced_sync_config; | |
| 119 | + struct periodic_interrupt_config periodic_interrupt0; | |
| 120 | + struct periodic_interrupt_config periodic_interrupt1; | |
| 111 | 121 | |
| 112 | 122 | /* from core_stream struct */ |
| 113 | 123 | struct dc_context *ctx; |
| @@ -158,8 +168,8 @@ struct dc_stream_update { · 44 unmodified lines | ||
| 158 | 168 | struct dc_info_packet *hdr_static_metadata; |
| 159 | 169 | unsigned int *abm_level; |
| 160 | 170 | |
| 161 | − union vline_config *periodic_vsync_config; | |
| 162 | − union vline_config *enhanced_sync_config; | |
| 171 | + struct periodic_interrupt_config *periodic_interrupt0; | |
| 172 | + struct periodic_interrupt_config *periodic_interrupt1; | |
| 163 | 173 | |
| 164 | 174 | struct dc_crtc_timing_adjust *adjust; |
| 165 | 175 | struct dc_info_packet *vrr_infopacket; |
| @@ -53,6 +53,27 @@ · 52 unmodified lines | ||
| 53 | 53 | |
| 54 | 54 | #define MCP_DISABLE_ABM_IMMEDIATELY 255 |
| 55 | 55 | |
| 56 | +static bool dce_abm_set_pipe(struct abm *abm, uint32_t controller_id) | |
| 57 | +{ | |
| 58 | + struct dce_abm *abm_dce = TO_DCE_ABM(abm); | |
| 59 | + uint32_t rampingBoundary = 0xFFFF; | |
| 60 | + | |
| 61 | + REG_WAIT(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, 0, | |
| 62 | + 1, 80000); | |
| 63 | + | |
| 64 | + /* set ramping boundary */ | |
| 65 | + REG_WRITE(MASTER_COMM_DATA_REG1, rampingBoundary); | |
| 66 | + | |
| 67 | + /* setDMCUParam_Pipe */ | |
| 68 | + REG_UPDATE_2(MASTER_COMM_CMD_REG, | |
| 69 | + MASTER_COMM_CMD_REG_BYTE0, MCP_ABM_PIPE_SET, | |
| 70 | + MASTER_COMM_CMD_REG_BYTE1, controller_id); | |
| 71 | + | |
| 72 | + /* notifyDMCUMsg */ | |
| 73 | + REG_UPDATE(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, 1); | |
| 74 | + | |
| 75 | + return true; | |
| 76 | +} | |
| 56 | 77 | |
| 57 | 78 | static unsigned int calculate_16_bit_backlight_from_pwm(struct dce_abm *abm_dce) |
| 58 | 79 | { |
| @@ -175,7 +196,6 @@ static void dmcu_set_backlight_level( · 116 unmodified lines | ||
| 175 | 196 | uint32_t controller_id) |
| 176 | 197 | { |
| 177 | 198 | unsigned int backlight_8_bit = 0; |
| 178 | − uint32_t rampingBoundary = 0xFFFF; | |
| 179 | 199 | uint32_t s2; |
| 180 | 200 | |
| 181 | 201 | if (backlight_pwm_u16_16 & 0x10000) |
| @@ -185,16 +205,7 @@ static void dmcu_set_backlight_level( · 3 unmodified lines | ||
| 185 | 205 | // Take MSB of fractional part since backlight is not max |
| 186 | 206 | backlight_8_bit = (backlight_pwm_u16_16 >> 8) & 0xFF; |
| 187 | 207 | |
| 188 | − /* set ramping boundary */ | |
| 189 | − REG_WRITE(MASTER_COMM_DATA_REG1, rampingBoundary); | |
| 190 | − | |
| 191 | − /* setDMCUParam_Pipe */ | |
| 192 | − REG_UPDATE_2(MASTER_COMM_CMD_REG, | |
| 193 | − MASTER_COMM_CMD_REG_BYTE0, MCP_ABM_PIPE_SET, | |
| 194 | − MASTER_COMM_CMD_REG_BYTE1, controller_id); | |
| 195 | − | |
| 196 | − /* notifyDMCUMsg */ | |
| 197 | − REG_UPDATE(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, 1); | |
| 208 | + dce_abm_set_pipe(&abm_dce->base, controller_id); | |
| 198 | 209 | |
| 199 | 210 | /* waitDMCUReadyForCmd */ |
| 200 | 211 | REG_WAIT(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, |
| @@ -309,16 +320,7 @@ static bool dce_abm_immediate_disable(struct abm *abm) · 108 unmodified lines | ||
| 309 | 320 | { |
| 310 | 321 | struct dce_abm *abm_dce = TO_DCE_ABM(abm); |
| 311 | 322 | |
| 312 | − REG_WAIT(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, 0, | |
| 313 | − 1, 80000); | |
| 314 | − | |
| 315 | − /* setDMCUParam_ABMLevel */ | |
| 316 | − REG_UPDATE_2(MASTER_COMM_CMD_REG, | |
| 317 | − MASTER_COMM_CMD_REG_BYTE0, MCP_ABM_PIPE_SET, | |
| 318 | − MASTER_COMM_CMD_REG_BYTE1, MCP_DISABLE_ABM_IMMEDIATELY); | |
| 319 | − | |
| 320 | − /* notifyDMCUMsg */ | |
| 321 | − REG_UPDATE(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, 1); | |
| 323 | + dce_abm_set_pipe(abm, MCP_DISABLE_ABM_IMMEDIATELY); | |
| 322 | 324 | |
| 323 | 325 | abm->stored_backlight_registers.BL_PWM_CNTL = |
| 324 | 326 | REG_READ(BL_PWM_CNTL); |
| @@ -419,6 +421,7 @@ static const struct abm_funcs dce_funcs = { · 94 unmodified lines | ||
| 419 | 421 | .abm_init = dce_abm_init, |
| 420 | 422 | .set_abm_level = dce_abm_set_level, |
| 421 | 423 | .init_backlight = dce_abm_init_backlight, |
| 424 | + .set_pipe = dce_abm_set_pipe, | |
| 422 | 425 | .set_backlight_level_pwm = dce_abm_set_backlight_level_pwm, |
| 423 | 426 | .get_current_backlight = dce_abm_get_current_backlight, |
| 424 | 427 | .get_target_backlight = dce_abm_get_target_backlight, |
| @@ -696,6 +696,11 @@ static void dce11_update_clocks(struct clk_mgr *clk_mgr, · 695 unmodified lines | ||
| 696 | 696 | { |
| 697 | 697 | struct dce_clk_mgr *clk_mgr_dce = TO_DCE_CLK_MGR(clk_mgr); |
| 698 | 698 | struct dm_pp_power_level_change_request level_change_req; |
| 699 | + int patched_disp_clk = context->bw.dce.dispclk_khz; | |
| 700 | + | |
| 701 | + /*TODO: W/A for dal3 linux, investigate why this works */ | |
| 702 | + if (!clk_mgr_dce->dfs_bypass_active) | |
| 703 | + patched_disp_clk = patched_disp_clk * 115 / 100; | |
| 699 | 704 | |
| 700 | 705 | level_change_req.power_level = dce_get_required_clocks_state(clk_mgr, context); |
| 701 | 706 | /* get max clock state from PPLIB */ |
| @@ -705,9 +710,9 @@ static void dce11_update_clocks(struct clk_mgr *clk_mgr, · 3 unmodified lines | ||
| 705 | 710 | clk_mgr_dce->cur_min_clks_state = level_change_req.power_level; |
| 706 | 711 | } |
| 707 | 712 | |
| 708 | − if (should_set_clock(safe_to_lower, context->bw.dce.dispclk_khz, clk_mgr->clks.dispclk_khz)) { | |
| 709 | − context->bw.dce.dispclk_khz = dce_set_clock(clk_mgr, context->bw.dce.dispclk_khz); | |
| 710 | − clk_mgr->clks.dispclk_khz = context->bw.dce.dispclk_khz; | |
| 713 | + if (should_set_clock(safe_to_lower, patched_disp_clk, clk_mgr->clks.dispclk_khz)) { | |
| 714 | + context->bw.dce.dispclk_khz = dce_set_clock(clk_mgr, patched_disp_clk); | |
| 715 | + clk_mgr->clks.dispclk_khz = patched_disp_clk; | |
| 711 | 716 | } |
| 712 | 717 | dce11_pplib_apply_display_requirements(clk_mgr->ctx->dc, context); |
| 713 | 718 | } |
| @@ -479,7 +479,7 @@ static void program_grph_pixel_format( · 478 unmodified lines | ||
| 479 | 479 | case SURFACE_PIXEL_FORMAT_GRPH_ABGR16161616F: |
| 480 | 480 | sign = 1; |
| 481 | 481 | floating = 1; |
| 482 | − /* no break */ | |
| 482 | + /* fall through */ | |
| 483 | 483 | case SURFACE_PIXEL_FORMAT_GRPH_ARGB16161616F: /* shouldn't this get float too? */ |
| 484 | 484 | case SURFACE_PIXEL_FORMAT_GRPH_ARGB16161616: |
| 485 | 485 | grph_depth = 3; |
| @@ -37,6 +37,10 @@ void dce100_prepare_bandwidth( · 36 unmodified lines | ||
| 37 | 37 | struct dc *dc, |
| 38 | 38 | struct dc_state *context); |
| 39 | 39 | |
| 40 | +void dce100_optimize_bandwidth( | |
| 41 | + struct dc *dc, | |
| 42 | + struct dc_state *context); | |
| 43 | + | |
| 40 | 44 | bool dce100_enable_display_power_gating(struct dc *dc, uint8_t controller_id, |
| 41 | 45 | struct dc_bios *dcb, |
| 42 | 46 | enum pipe_gating_control power_gating); |
| @@ -1300,6 +1300,10 @@ static enum dc_status apply_single_controller_ctx_to_hw( · 1299 unmodified lines | ||
| 1300 | 1300 | struct drr_params params = {0}; |
| 1301 | 1301 | unsigned int event_triggers = 0; |
| 1302 | 1302 | |
| 1303 | + if (dc->hwss.disable_stream_gating) { | |
| 1304 | + dc->hwss.disable_stream_gating(dc, pipe_ctx); | |
| 1305 | + } | |
| 1306 | + | |
| 1303 | 1307 | if (pipe_ctx->stream_res.audio != NULL) { |
| 1304 | 1308 | struct audio_output audio_output; |
| 1305 | 1309 | |
| @@ -1329,10 +1333,8 @@ static enum dc_status apply_single_controller_ctx_to_hw( · 23 unmodified lines | ||
| 1329 | 1333 | if (!pipe_ctx->stream->apply_seamless_boot_optimization) |
| 1330 | 1334 | dc->hwss.enable_stream_timing(pipe_ctx, context, dc); |
| 1331 | 1335 | |
| 1332 | − if (pipe_ctx->stream_res.tg->funcs->program_vupdate_interrupt) | |
| 1333 | − pipe_ctx->stream_res.tg->funcs->program_vupdate_interrupt( | |
| 1334 | − pipe_ctx->stream_res.tg, | |
| 1335 | − &stream->timing); | |
| 1336 | + if (dc->hwss.setup_vupdate_interrupt) | |
| 1337 | + dc->hwss.setup_vupdate_interrupt(pipe_ctx); | |
| 1336 | 1338 | |
| 1337 | 1339 | params.vertical_total_min = stream->adjust.v_total_min; |
| 1338 | 1340 | params.vertical_total_max = stream->adjust.v_total_max; |
| @@ -1521,6 +1523,14 @@ void dce110_enable_accelerated_mode(struct dc *dc, struct dc_state *context) · 182 unmodified lines | ||
| 1521 | 1523 | struct dc_link *edp_link = get_link_for_edp(dc); |
| 1522 | 1524 | bool can_edp_fast_boot_optimize = false; |
| 1523 | 1525 | bool apply_edp_fast_boot_optimization = false; |
| 1526 | + bool can_apply_seamless_boot = false; | |
| 1527 | + | |
| 1528 | + for (i = 0; i < context->stream_count; i++) { | |
| 1529 | + if (context->streams[i]->apply_seamless_boot_optimization) { | |
| 1530 | + can_apply_seamless_boot = true; | |
| 1531 | + break; | |
| 1532 | + } | |
| 1533 | + } | |
| 1524 | 1534 | |
| 1525 | 1535 | if (edp_link) { |
| 1526 | 1536 | /* this seems to cause blank screens on DCE8 */ |
| @@ -1549,7 +1559,7 @@ void dce110_enable_accelerated_mode(struct dc *dc, struct dc_state *context) · 22 unmodified lines | ||
| 1549 | 1559 | } |
| 1550 | 1560 | } |
| 1551 | 1561 | |
| 1552 | − if (!apply_edp_fast_boot_optimization) { | |
| 1562 | + if (!apply_edp_fast_boot_optimization && !can_apply_seamless_boot) { | |
| 1553 | 1563 | if (edp_link_to_turnoff) { |
| 1554 | 1564 | /*turn off backlight before DP_blank and encoder powered down*/ |
| 1555 | 1565 | dc->hwss.edp_backlight_control(edp_link_to_turnoff, false); |
| @@ -2676,6 +2686,8 @@ static const struct hw_sequencer_funcs dce110_funcs = { · 1120 unmodified lines | ||
| 2676 | 2686 | .set_static_screen_control = set_static_screen_control, |
| 2677 | 2687 | .reset_hw_ctx_wrap = dce110_reset_hw_ctx_wrap, |
| 2678 | 2688 | .enable_stream_timing = dce110_enable_stream_timing, |
| 2689 | + .disable_stream_gating = NULL, | |
| 2690 | + .enable_stream_gating = NULL, | |
| 2679 | 2691 | .setup_stereo = NULL, |
| 2680 | 2692 | .set_avmute = dce110_set_avmute, |
| 2681 | 2693 | .wait_for_mpcc_disconnect = dce110_wait_for_mpcc_disconnect, |
| @@ -77,6 +77,6 @@ void dce80_hw_sequencer_construct(struct dc *dc) · 76 unmodified lines | ||
| 77 | 77 | dc->hwss.enable_display_power_gating = dce100_enable_display_power_gating; |
| 78 | 78 | dc->hwss.pipe_control_lock = dce_pipe_control_lock; |
| 79 | 79 | dc->hwss.prepare_bandwidth = dce100_prepare_bandwidth; |
| 80 | − dc->hwss.optimize_bandwidth = dce100_prepare_bandwidth; | |
| 80 | + dc->hwss.optimize_bandwidth = dce100_optimize_bandwidth; | |
| 81 | 81 | } |
| 82 | 82 | |
| @@ -792,9 +792,22 @@ bool dce80_validate_bandwidth( · 791 unmodified lines | ||
| 792 | 792 | struct dc *dc, |
| 793 | 793 | struct dc_state *context) |
| 794 | 794 | { |
| 795 | − /* TODO implement when needed but for now hardcode max value*/ | |
| 796 | − context->bw.dce.dispclk_khz = 681000; | |
| 797 | − context->bw.dce.yclk_khz = 250000 * MEMORY_TYPE_MULTIPLIER_CZ; | |
| 795 | + int i; | |
| 796 | + bool at_least_one_pipe = false; | |
| 797 | + | |
| 798 | + for (i = 0; i < dc->res_pool->pipe_count; i++) { | |
| 799 | + if (context->res_ctx.pipe_ctx[i].stream) | |
| 800 | + at_least_one_pipe = true; | |
| 801 | + } | |
| 802 | + | |
| 803 | + if (at_least_one_pipe) { | |
| 804 | + /* TODO implement when needed but for now hardcode max value*/ | |
| 805 | + context->bw.dce.dispclk_khz = 681000; | |
| 806 | + context->bw.dce.yclk_khz = 250000 * MEMORY_TYPE_MULTIPLIER_CZ; | |
| 807 | + } else { | |
| 808 | + context->bw.dce.dispclk_khz = 0; | |
| 809 | + context->bw.dce.yclk_khz = 0; | |
| 810 | + } | |
| 798 | 811 | |
| 799 | 812 | return true; |
| 800 | 813 | } |
| @@ -959,9 +959,25 @@ static void dcn10_disable_plane(struct dc *dc, struct pipe_ctx *pipe_ctx) · 958 unmodified lines | ||
| 959 | 959 | static void dcn10_init_pipes(struct dc *dc, struct dc_state *context) |
| 960 | 960 | { |
| 961 | 961 | int i; |
| 962 | + bool can_apply_seamless_boot = false; | |
| 963 | + | |
| 964 | + for (i = 0; i < context->stream_count; i++) { | |
| 965 | + if (context->streams[i]->apply_seamless_boot_optimization) { | |
| 966 | + can_apply_seamless_boot = true; | |
| 967 | + break; | |
| 968 | + } | |
| 969 | + } | |
| 962 | 970 | |
| 963 | 971 | for (i = 0; i < dc->res_pool->pipe_count; i++) { |
| 964 | 972 | struct timing_generator *tg = dc->res_pool->timing_generators[i]; |
| 973 | + struct pipe_ctx *pipe_ctx = &context->res_ctx.pipe_ctx[i]; | |
| 974 | + | |
| 975 | + /* There is assumption that pipe_ctx is not mapping irregularly | |
| 976 | + * to non-preferred front end. If pipe_ctx->stream is not NULL, | |
| 977 | + * we will use the pipe, so don't disable | |
| 978 | + */ | |
| 979 | + if (pipe_ctx->stream != NULL) | |
| 980 | + continue; | |
| 965 | 981 | |
| 966 | 982 | if (tg->funcs->is_tg_enabled(tg)) |
| 967 | 983 | tg->funcs->lock(tg); |
| @@ -975,7 +991,9 @@ static void dcn10_init_pipes(struct dc *dc, struct dc_state *context) · 7 unmodified lines | ||
| 975 | 991 | } |
| 976 | 992 | } |
| 977 | 993 | |
| 978 | − dc->res_pool->mpc->funcs->mpc_init(dc->res_pool->mpc); | |
| 994 | + /* Cannot reset the MPC mux if seamless boot */ | |
| 995 | + if (!can_apply_seamless_boot) | |
| 996 | + dc->res_pool->mpc->funcs->mpc_init(dc->res_pool->mpc); | |
| 979 | 997 | |
| 980 | 998 | for (i = 0; i < dc->res_pool->pipe_count; i++) { |
| 981 | 999 | struct timing_generator *tg = dc->res_pool->timing_generators[i]; |
| @@ -983,6 +1001,16 @@ static void dcn10_init_pipes(struct dc *dc, struct dc_state *context) · 1 unmodified line | ||
| 983 | 1001 | struct dpp *dpp = dc->res_pool->dpps[i]; |
| 984 | 1002 | struct pipe_ctx *pipe_ctx = &context->res_ctx.pipe_ctx[i]; |
| 985 | 1003 | |
| 1004 | + // W/A for issue with dc_post_update_surfaces_to_stream | |
| 1005 | + hubp->power_gated = true; | |
| 1006 | + | |
| 1007 | + /* There is assumption that pipe_ctx is not mapping irregularly | |
| 1008 | + * to non-preferred front end. If pipe_ctx->stream is not NULL, | |
| 1009 | + * we will use the pipe, so don't disable | |
| 1010 | + */ | |
| 1011 | + if (pipe_ctx->stream != NULL) | |
| 1012 | + continue; | |
| 1013 | + | |
| 986 | 1014 | dpp->funcs->dpp_reset(dpp); |
| 987 | 1015 | |
| 988 | 1016 | pipe_ctx->stream_res.tg = tg; |
| @@ -1137,11 +1165,13 @@ static void reset_hw_ctx_wrap( · 148 unmodified lines | ||
| 1137 | 1165 | struct clock_source *old_clk = pipe_ctx_old->clock_source; |
| 1138 | 1166 | |
| 1139 | 1167 | reset_back_end_for_pipe(dc, pipe_ctx_old, dc->current_state); |
| 1168 | + if (dc->hwss.enable_stream_gating) { | |
| 1169 | + dc->hwss.enable_stream_gating(dc, pipe_ctx); | |
| 1170 | + } | |
| 1140 | 1171 | if (old_clk) |
| 1141 | 1172 | old_clk->funcs->cs_power_down(old_clk); |
| 1142 | 1173 | } |
| 1143 | 1174 | } |
| 1144 | − | |
| 1145 | 1175 | } |
| 1146 | 1176 | |
| 1147 | 1177 | static bool patch_address_for_sbs_tb_stereo( |
| @@ -2162,8 +2192,10 @@ static void dcn10_blank_pixel_data( · 1014 unmodified lines | ||
| 2162 | 2192 | if (!blank) { |
| 2163 | 2193 | if (stream_res->tg->funcs->set_blank) |
| 2164 | 2194 | stream_res->tg->funcs->set_blank(stream_res->tg, blank); |
| 2165 | − if (stream_res->abm) | |
| 2195 | + if (stream_res->abm) { | |
| 2196 | + stream_res->abm->funcs->set_pipe(stream_res->abm, stream_res->tg->inst + 1); | |
| 2166 | 2197 | stream_res->abm->funcs->set_abm_level(stream_res->abm, stream->abm_level); |
| 2198 | + } | |
| 2167 | 2199 | } else if (blank) { |
| 2168 | 2200 | if (stream_res->abm) |
| 2169 | 2201 | stream_res->abm->funcs->set_abm_immediate_disable(stream_res->abm); |
| @@ -2661,8 +2693,8 @@ static void dcn10_set_cursor_position(struct pipe_ctx *pipe_ctx) · 491 unmodified lines | ||
| 2661 | 2693 | .mirror = pipe_ctx->plane_state->horizontal_mirror |
| 2662 | 2694 | }; |
| 2663 | 2695 | |
| 2664 | − pos_cpy.x -= pipe_ctx->plane_state->dst_rect.x; | |
| 2665 | − pos_cpy.y -= pipe_ctx->plane_state->dst_rect.y; | |
| 2696 | + pos_cpy.x_hotspot += pipe_ctx->plane_state->dst_rect.x; | |
| 2697 | + pos_cpy.y_hotspot += pipe_ctx->plane_state->dst_rect.y; | |
| 2666 | 2698 | |
| 2667 | 2699 | if (pipe_ctx->plane_state->address.type |
| 2668 | 2700 | == PLN_ADDR_TYPE_VIDEO_PROGRESSIVE) |
| @@ -2709,6 +2741,147 @@ static void dcn10_set_cursor_sdr_white_level(struct pipe_ctx *pipe_ctx) · 40 unmodified lines | ||
| 2709 | 2741 | pipe_ctx->plane_res.dpp, &opt_attr); |
| 2710 | 2742 | } |
| 2711 | 2743 | |
| 2744 | +/** | |
| 2745 | +* apply_front_porch_workaround TODO FPGA still need? | |
| 2746 | +* | |
| 2747 | +* This is a workaround for a bug that has existed since R5xx and has not been | |
| 2748 | +* fixed keep Front porch at minimum 2 for Interlaced mode or 1 for progressive. | |
| 2749 | +*/ | |
| 2750 | +static void apply_front_porch_workaround( | |
| 2751 | + struct dc_crtc_timing *timing) | |
| 2752 | +{ | |
| 2753 | + if (timing->flags.INTERLACE == 1) { | |
| 2754 | + if (timing->v_front_porch < 2) | |
| 2755 | + timing->v_front_porch = 2; | |
| 2756 | + } else { | |
| 2757 | + if (timing->v_front_porch < 1) | |
| 2758 | + timing->v_front_porch = 1; | |
| 2759 | + } | |
| 2760 | +} | |
| 2761 | + | |
| 2762 | +int get_vupdate_offset_from_vsync(struct pipe_ctx *pipe_ctx) | |
| 2763 | +{ | |
| 2764 | + struct timing_generator *optc = pipe_ctx->stream_res.tg; | |
| 2765 | + const struct dc_crtc_timing *dc_crtc_timing = &pipe_ctx->stream->timing; | |
| 2766 | + struct dc_crtc_timing patched_crtc_timing; | |
| 2767 | + int vesa_sync_start; | |
| 2768 | + int asic_blank_end; | |
| 2769 | + int interlace_factor; | |
| 2770 | + int vertical_line_start; | |
| 2771 | + | |
| 2772 | + patched_crtc_timing = *dc_crtc_timing; | |
| 2773 | + apply_front_porch_workaround(&patched_crtc_timing); | |
| 2774 | + | |
| 2775 | + interlace_factor = patched_crtc_timing.flags.INTERLACE ? 2 : 1; | |
| 2776 | + | |
| 2777 | + vesa_sync_start = patched_crtc_timing.v_addressable + | |
| 2778 | + patched_crtc_timing.v_border_bottom + | |
| 2779 | + patched_crtc_timing.v_front_porch; | |
| 2780 | + | |
| 2781 | + asic_blank_end = (patched_crtc_timing.v_total - | |
| 2782 | + vesa_sync_start - | |
| 2783 | + patched_crtc_timing.v_border_top) | |
| 2784 | + * interlace_factor; | |
| 2785 | + | |
| 2786 | + vertical_line_start = asic_blank_end - | |
| 2787 | + optc->dlg_otg_param.vstartup_start + 1; | |
| 2788 | + | |
| 2789 | + return vertical_line_start; | |
| 2790 | +} | |
| 2791 | + | |
| 2792 | +static void calc_vupdate_position( | |
| 2793 | + struct pipe_ctx *pipe_ctx, | |
| 2794 | + uint32_t *start_line, | |
| 2795 | + uint32_t *end_line) | |
| 2796 | +{ | |
| 2797 | + const struct dc_crtc_timing *dc_crtc_timing = &pipe_ctx->stream->timing; | |
| 2798 | + int vline_int_offset_from_vupdate = | |
| 2799 | + pipe_ctx->stream->periodic_interrupt0.lines_offset; | |
| 2800 | + int vupdate_offset_from_vsync = get_vupdate_offset_from_vsync(pipe_ctx); | |
| 2801 | + int start_position; | |
| 2802 | + | |
| 2803 | + if (vline_int_offset_from_vupdate > 0) | |
| 2804 | + vline_int_offset_from_vupdate--; | |
| 2805 | + else if (vline_int_offset_from_vupdate < 0) | |
| 2806 | + vline_int_offset_from_vupdate++; | |
| 2807 | + | |
| 2808 | + start_position = vline_int_offset_from_vupdate + vupdate_offset_from_vsync; | |
| 2809 | + | |
| 2810 | + if (start_position >= 0) | |
| 2811 | + *start_line = start_position; | |
| 2812 | + else | |
| 2813 | + *start_line = dc_crtc_timing->v_total + start_position - 1; | |
| 2814 | + | |
| 2815 | + *end_line = *start_line + 2; | |
| 2816 | + | |
| 2817 | + if (*end_line >= dc_crtc_timing->v_total) | |
| 2818 | + *end_line = 2; | |
| 2819 | +} | |
| 2820 | + | |
| 2821 | +static void cal_vline_position( | |
| 2822 | + struct pipe_ctx *pipe_ctx, | |
| 2823 | + enum vline_select vline, | |
| 2824 | + uint32_t *start_line, | |
| 2825 | + uint32_t *end_line) | |
| 2826 | +{ | |
| 2827 | + enum vertical_interrupt_ref_point ref_point = INVALID_POINT; | |
| 2828 | + | |
| 2829 | + if (vline == VLINE0) | |
| 2830 | + ref_point = pipe_ctx->stream->periodic_interrupt0.ref_point; | |
| 2831 | + else if (vline == VLINE1) | |
| 2832 | + ref_point = pipe_ctx->stream->periodic_interrupt1.ref_point; | |
| 2833 | + | |
| 2834 | + switch (ref_point) { | |
| 2835 | + case START_V_UPDATE: | |
| 2836 | + calc_vupdate_position( | |
| 2837 | + pipe_ctx, | |
| 2838 | + start_line, | |
| 2839 | + end_line); | |
| 2840 | + break; | |
| 2841 | + case START_V_SYNC: | |
| 2842 | + // Suppose to do nothing because vsync is 0; | |
| 2843 | + break; | |
| 2844 | + default: | |
| 2845 | + ASSERT(0); | |
| 2846 | + break; | |
| 2847 | + } | |
| 2848 | +} | |
| 2849 | + | |
| 2850 | +static void dcn10_setup_periodic_interrupt( | |
| 2851 | + struct pipe_ctx *pipe_ctx, | |
| 2852 | + enum vline_select vline) | |
| 2853 | +{ | |
| 2854 | + struct timing_generator *tg = pipe_ctx->stream_res.tg; | |
| 2855 | + | |
| 2856 | + if (vline == VLINE0) { | |
| 2857 | + uint32_t start_line = 0; | |
| 2858 | + uint32_t end_line = 0; | |
| 2859 | + | |
| 2860 | + cal_vline_position(pipe_ctx, vline, &start_line, &end_line); | |
| 2861 | + | |
| 2862 | + tg->funcs->setup_vertical_interrupt0(tg, start_line, end_line); | |
| 2863 | + | |
| 2864 | + } else if (vline == VLINE1) { | |
| 2865 | + pipe_ctx->stream_res.tg->funcs->setup_vertical_interrupt1( | |
| 2866 | + tg, | |
| 2867 | + pipe_ctx->stream->periodic_interrupt1.lines_offset); | |
| 2868 | + } | |
| 2869 | +} | |
| 2870 | + | |
| 2871 | +static void dcn10_setup_vupdate_interrupt(struct pipe_ctx *pipe_ctx) | |
| 2872 | +{ | |
| 2873 | + struct timing_generator *tg = pipe_ctx->stream_res.tg; | |
| 2874 | + int start_line = get_vupdate_offset_from_vsync(pipe_ctx); | |
| 2875 | + | |
| 2876 | + if (start_line < 0) { | |
| 2877 | + ASSERT(0); | |
| 2878 | + start_line = 0; | |
| 2879 | + } | |
| 2880 | + | |
| 2881 | + if (tg->funcs->setup_vertical_interrupt2) | |
| 2882 | + tg->funcs->setup_vertical_interrupt2(tg, start_line); | |
| 2883 | +} | |
| 2884 | + | |
| 2712 | 2885 | static const struct hw_sequencer_funcs dcn10_funcs = { |
| 2713 | 2886 | .program_gamut_remap = program_gamut_remap, |
| 2714 | 2887 | .init_hw = dcn10_init_hw, |
| @@ -2756,7 +2929,11 @@ static const struct hw_sequencer_funcs dcn10_funcs = { · 41 unmodified lines | ||
| 2756 | 2929 | .edp_wait_for_hpd_ready = hwss_edp_wait_for_hpd_ready, |
| 2757 | 2930 | .set_cursor_position = dcn10_set_cursor_position, |
| 2758 | 2931 | .set_cursor_attribute = dcn10_set_cursor_attribute, |
| 2759 | − .set_cursor_sdr_white_level = dcn10_set_cursor_sdr_white_level | |
| 2932 | + .set_cursor_sdr_white_level = dcn10_set_cursor_sdr_white_level, | |
| 2933 | + .disable_stream_gating = NULL, | |
| 2934 | + .enable_stream_gating = NULL, | |
| 2935 | + .setup_periodic_interrupt = dcn10_setup_periodic_interrupt, | |
| 2936 | + .setup_vupdate_interrupt = dcn10_setup_vupdate_interrupt | |
| 2760 | 2937 | }; |
| 2761 | 2938 | |
| 2762 | 2939 | |
| @@ -81,4 +81,6 @@ struct pipe_ctx *find_top_pipe_for_stream( · 80 unmodified lines | ||
| 81 | 81 | struct dc_state *context, |
| 82 | 82 | const struct dc_stream_state *stream); |
| 83 | 83 | |
| 84 | +int get_vupdate_offset_from_vsync(struct pipe_ctx *pipe_ctx); | |
| 85 | + | |
| 84 | 86 | #endif /* __DC_HWSS_DCN10_H__ */ |
| @@ -92,134 +92,36 @@ static void optc1_disable_stereo(struct timing_generator *optc) · 91 unmodified lines | ||
| 92 | 92 | OTG_3D_STRUCTURE_STEREO_SEL_OVR, 0); |
| 93 | 93 | } |
| 94 | 94 | |
| 95 | −static uint32_t get_start_vline(struct timing_generator *optc, const struct dc_crtc_timing *dc_crtc_timing) | |
| 96 | −{ | |
| 97 | − struct dc_crtc_timing patched_crtc_timing; | |
| 98 | − int vesa_sync_start; | |
| 99 | − int asic_blank_end; | |
| 100 | − int interlace_factor; | |
| 101 | − int vertical_line_start; | |
| 102 | − | |
| 103 | − patched_crtc_timing = *dc_crtc_timing; | |
| 104 | − optc1_apply_front_porch_workaround(optc, &patched_crtc_timing); | |
| 105 | − | |
| 106 | − vesa_sync_start = patched_crtc_timing.h_addressable + | |
| 107 | − patched_crtc_timing.h_border_right + | |
| 108 | − patched_crtc_timing.h_front_porch; | |
| 109 | − | |
| 110 | − asic_blank_end = patched_crtc_timing.h_total - | |
| 111 | − vesa_sync_start - | |
| 112 | − patched_crtc_timing.h_border_left; | |
| 113 | − | |
| 114 | − interlace_factor = patched_crtc_timing.flags.INTERLACE ? 2 : 1; | |
| 115 | − | |
| 116 | − vesa_sync_start = patched_crtc_timing.v_addressable + | |
| 117 | − patched_crtc_timing.v_border_bottom + | |
| 118 | − patched_crtc_timing.v_front_porch; | |
| 119 | − | |
| 120 | − asic_blank_end = (patched_crtc_timing.v_total - | |
| 121 | − vesa_sync_start - | |
| 122 | − patched_crtc_timing.v_border_top) | |
| 123 | − * interlace_factor; | |
| 124 | − | |
| 125 | − vertical_line_start = asic_blank_end - optc->dlg_otg_param.vstartup_start + 1; | |
| 126 | − if (vertical_line_start < 0) { | |
| 127 | − ASSERT(0); | |
| 128 | − vertical_line_start = 0; | |
| 129 | − } | |
| 130 | − | |
| 131 | − return vertical_line_start; | |
| 132 | −} | |
| 133 | − | |
| 134 | −static void calc_vline_position( | |
| 95 | +void optc1_setup_vertical_interrupt0( | |
| 135 | 96 | struct timing_generator *optc, |
| 136 | − const struct dc_crtc_timing *dc_crtc_timing, | |
| 137 | − unsigned long long vsync_delta, | |
| 138 | − uint32_t *start_line, | |
| 139 | − uint32_t *end_line) | |
| 97 | + uint32_t start_line, | |
| 98 | + uint32_t end_line) | |
| 140 | 99 | { |
| 141 | − unsigned long long req_delta_tens_of_usec = div64_u64((vsync_delta + 9999), 10000); | |
| 142 | − unsigned long long pix_clk_hundreds_khz = div64_u64((dc_crtc_timing->pix_clk_100hz + 999), 1000); | |
| 143 | − uint32_t req_delta_lines = (uint32_t) div64_u64( | |
| 144 | − (req_delta_tens_of_usec * pix_clk_hundreds_khz + dc_crtc_timing->h_total - 1), | |
| 145 | − dc_crtc_timing->h_total); | |
| 146 | − | |
| 147 | − uint32_t vsync_line = get_start_vline(optc, dc_crtc_timing); | |
| 148 | − | |
| 149 | − if (req_delta_lines != 0) | |
| 150 | − req_delta_lines--; | |
| 151 | − | |
| 152 | − if (req_delta_lines > vsync_line) | |
| 153 | − *start_line = dc_crtc_timing->v_total - (req_delta_lines - vsync_line) - 1; | |
| 154 | − else | |
| 155 | − *start_line = vsync_line - req_delta_lines; | |
| 156 | − | |
| 157 | − *end_line = *start_line + 2; | |
| 100 | + struct optc *optc1 = DCN10TG_FROM_TG(optc); | |
| 158 | 101 | |
| 159 | − if (*end_line >= dc_crtc_timing->v_total) | |
| 160 | − *end_line = 2; | |
| 102 | + REG_SET_2(OTG_VERTICAL_INTERRUPT0_POSITION, 0, | |
| 103 | + OTG_VERTICAL_INTERRUPT0_LINE_START, start_line, | |
| 104 | + OTG_VERTICAL_INTERRUPT0_LINE_END, end_line); | |
| 161 | 105 | } |
| 162 | 106 | |
| 163 | −void optc1_program_vline_interrupt( | |
| 107 | +void optc1_setup_vertical_interrupt1( | |
| 164 | 108 | struct timing_generator *optc, |
| 165 | − const struct dc_crtc_timing *dc_crtc_timing, | |
| 166 | − enum vline_select vline, | |
| 167 | − const union vline_config *vline_config) | |
| 109 | + uint32_t start_line) | |
| 168 | 110 | { |
| 169 | 111 | struct optc *optc1 = DCN10TG_FROM_TG(optc); |
| 170 | − uint32_t start_line = 0; | |
| 171 | − uint32_t end_line = 0; | |
| 172 | − | |
| 173 | − switch (vline) { | |
| 174 | − case VLINE0: | |
| 175 | − calc_vline_position(optc, dc_crtc_timing, vline_config->delta_in_ns, &start_line, &end_line); | |
| 176 | − REG_SET_2(OTG_VERTICAL_INTERRUPT0_POSITION, 0, | |
| 177 | − OTG_VERTICAL_INTERRUPT0_LINE_START, start_line, | |
| 178 | − OTG_VERTICAL_INTERRUPT0_LINE_END, end_line); | |
| 179 | − break; | |
| 180 | − case VLINE1: | |
| 181 | − REG_SET(OTG_VERTICAL_INTERRUPT1_POSITION, 0, | |
| 182 | − OTG_VERTICAL_INTERRUPT1_LINE_START, vline_config->line_number); | |
| 183 | − break; | |
| 184 | − default: | |
| 185 | − break; | |
| 186 | − } | |
| 112 | + | |
| 113 | + REG_SET(OTG_VERTICAL_INTERRUPT1_POSITION, 0, | |
| 114 | + OTG_VERTICAL_INTERRUPT1_LINE_START, start_line); | |
| 187 | 115 | } |
| 188 | 116 | |
| 189 | −void optc1_program_vupdate_interrupt( | |
| 117 | +void optc1_setup_vertical_interrupt2( | |
| 190 | 118 | struct timing_generator *optc, |
| 191 | − const struct dc_crtc_timing *dc_crtc_timing) | |
| 119 | + uint32_t start_line) | |
| 192 | 120 | { |
| 193 | 121 | struct optc *optc1 = DCN10TG_FROM_TG(optc); |
| 194 | − int32_t vertical_line_start; | |
| 195 | − uint32_t asic_blank_end; | |
| 196 | − uint32_t vesa_sync_start; | |
| 197 | − struct dc_crtc_timing patched_crtc_timing; | |
| 198 | − | |
| 199 | − patched_crtc_timing = *dc_crtc_timing; | |
| 200 | − optc1_apply_front_porch_workaround(optc, &patched_crtc_timing); | |
| 201 | − | |
| 202 | − /* asic_h_blank_end = HsyncWidth + HbackPorch = | |
| 203 | − * vesa. usHorizontalTotal - vesa. usHorizontalSyncStart - | |
| 204 | − * vesa.h_left_border | |
| 205 | − */ | |
| 206 | − vesa_sync_start = patched_crtc_timing.h_addressable + | |
| 207 | − patched_crtc_timing.h_border_right + | |
| 208 | − patched_crtc_timing.h_front_porch; | |
| 209 | − | |
| 210 | − asic_blank_end = patched_crtc_timing.h_total - | |
| 211 | − vesa_sync_start - | |
| 212 | − patched_crtc_timing.h_border_left; | |
| 213 | − | |
| 214 | − /* Use OTG_VERTICAL_INTERRUPT2 replace VUPDATE interrupt, | |
| 215 | − * program the reg for interrupt postition. | |
| 216 | − */ | |
| 217 | − vertical_line_start = asic_blank_end - optc->dlg_otg_param.vstartup_start + 1; | |
| 218 | − if (vertical_line_start < 0) | |
| 219 | − vertical_line_start = 0; | |
| 220 | 122 | |
| 221 | 123 | REG_SET(OTG_VERTICAL_INTERRUPT2_POSITION, 0, |
| 222 | − OTG_VERTICAL_INTERRUPT2_LINE_START, vertical_line_start); | |
| 124 | + OTG_VERTICAL_INTERRUPT2_LINE_START, start_line); | |
| 223 | 125 | } |
| 224 | 126 | |
| 225 | 127 | /** |
| @@ -1480,8 +1382,9 @@ bool optc1_get_crc(struct timing_generator *optc, · 1254 unmodified lines | ||
| 1480 | 1382 | static const struct timing_generator_funcs dcn10_tg_funcs = { |
| 1481 | 1383 | .validate_timing = optc1_validate_timing, |
| 1482 | 1384 | .program_timing = optc1_program_timing, |
| 1483 | − .program_vline_interrupt = optc1_program_vline_interrupt, | |
| 1484 | − .program_vupdate_interrupt = optc1_program_vupdate_interrupt, | |
| 1385 | + .setup_vertical_interrupt0 = optc1_setup_vertical_interrupt0, | |
| 1386 | + .setup_vertical_interrupt1 = optc1_setup_vertical_interrupt1, | |
| 1387 | + .setup_vertical_interrupt2 = optc1_setup_vertical_interrupt2, | |
| 1485 | 1388 | .program_global_sync = optc1_program_global_sync, |
| 1486 | 1389 | .enable_crtc = optc1_enable_crtc, |
| 1487 | 1390 | .disable_crtc = optc1_disable_crtc, |
| @@ -483,11 +483,16 @@ void optc1_program_timing( · 482 unmodified lines | ||
| 483 | 483 | const struct dc_crtc_timing *dc_crtc_timing, |
| 484 | 484 | bool use_vbios); |
| 485 | 485 | |
| 486 | −void optc1_program_vline_interrupt( | |
| 486 | +void optc1_setup_vertical_interrupt0( | |
| 487 | 487 | struct timing_generator *optc, |
| 488 | − const struct dc_crtc_timing *dc_crtc_timing, | |
| 489 | − enum vline_select vline, | |
| 490 | − const union vline_config *vline_config); | |
| 488 | + uint32_t start_line, | |
| 489 | + uint32_t end_line); | |
| 490 | +void optc1_setup_vertical_interrupt1( | |
| 491 | + struct timing_generator *optc, | |
| 492 | + uint32_t start_line); | |
| 493 | +void optc1_setup_vertical_interrupt2( | |
| 494 | + struct timing_generator *optc, | |
| 495 | + uint32_t start_line); | |
| 491 | 496 | |
| 492 | 497 | void optc1_program_global_sync( |
| 493 | 498 | struct timing_generator *optc); |
| @@ -46,6 +46,7 @@ struct abm_funcs { · 45 unmodified lines | ||
| 46 | 46 | void (*abm_init)(struct abm *abm); |
| 47 | 47 | bool (*set_abm_level)(struct abm *abm, unsigned int abm_level); |
| 48 | 48 | bool (*set_abm_immediate_disable)(struct abm *abm); |
| 49 | + bool (*set_pipe)(struct abm *abm, unsigned int controller_id); | |
| 49 | 50 | bool (*init_backlight)(struct abm *abm); |
| 50 | 51 | |
| 51 | 52 | /* backlight_pwm_u16_16 is unsigned 32 bit, |
| @@ -134,14 +134,6 @@ struct dc_crtc_timing; · 133 unmodified lines | ||
| 134 | 134 | |
| 135 | 135 | struct drr_params; |
| 136 | 136 | |
| 137 | −union vline_config; | |
| 138 | − | |
| 139 | − | |
| 140 | −enum vline_select { | |
| 141 | − VLINE0, | |
| 142 | − VLINE1, | |
| 143 | − VLINE2 | |
| 144 | −}; | |
| 145 | 137 | |
| 146 | 138 | struct timing_generator_funcs { |
| 147 | 139 | bool (*validate_timing)(struct timing_generator *tg, |
| @@ -149,14 +141,17 @@ struct timing_generator_funcs { · 1 unmodified line | ||
| 149 | 141 | void (*program_timing)(struct timing_generator *tg, |
| 150 | 142 | const struct dc_crtc_timing *timing, |
| 151 | 143 | bool use_vbios); |
| 152 | − void (*program_vline_interrupt)( | |
| 144 | + void (*setup_vertical_interrupt0)( | |
| 145 | + struct timing_generator *optc, | |
| 146 | + uint32_t start_line, | |
| 147 | + uint32_t end_line); | |
| 148 | + void (*setup_vertical_interrupt1)( | |
| 149 | + struct timing_generator *optc, | |
| 150 | + uint32_t start_line); | |
| 151 | + void (*setup_vertical_interrupt2)( | |
| 153 | 152 | struct timing_generator *optc, |
| 154 | − const struct dc_crtc_timing *dc_crtc_timing, | |
| 155 | − enum vline_select vline, | |
| 156 | − const union vline_config *vline_config); | |
| 153 | + uint32_t start_line); | |
| 157 | 154 | |
| 158 | − void (*program_vupdate_interrupt)(struct timing_generator *optc, | |
| 159 | − const struct dc_crtc_timing *dc_crtc_timing); | |
| 160 | 155 | bool (*enable_crtc)(struct timing_generator *tg); |
| 161 | 156 | bool (*disable_crtc)(struct timing_generator *tg); |
| 162 | 157 | bool (*is_counter_moving)(struct timing_generator *tg); |
| @@ -38,6 +38,11 @@ enum pipe_gating_control { · 37 unmodified lines | ||
| 38 | 38 | PIPE_GATING_CONTROL_INIT |
| 39 | 39 | }; |
| 40 | 40 | |
| 41 | +enum vline_select { | |
| 42 | + VLINE0, | |
| 43 | + VLINE1 | |
| 44 | +}; | |
| 45 | + | |
| 41 | 46 | struct dce_hwseq_wa { |
| 42 | 47 | bool blnd_crtc_trigger; |
| 43 | 48 | bool DEGVIDCN10_253; |
| @@ -68,6 +73,10 @@ struct stream_resource; · 24 unmodified lines | ||
| 68 | 73 | |
| 69 | 74 | struct hw_sequencer_funcs { |
| 70 | 75 | |
| 76 | + void (*disable_stream_gating)(struct dc *dc, struct pipe_ctx *pipe_ctx); | |
| 77 | + | |
| 78 | + void (*enable_stream_gating)(struct dc *dc, struct pipe_ctx *pipe_ctx); | |
| 79 | + | |
| 71 | 80 | void (*init_hw)(struct dc *dc); |
| 72 | 81 | |
| 73 | 82 | void (*init_pipes)(struct dc *dc, struct dc_state *context); |
| @@ -220,6 +229,9 @@ struct hw_sequencer_funcs { · 146 unmodified lines | ||
| 220 | 229 | void (*set_cursor_attribute)(struct pipe_ctx *pipe); |
| 221 | 230 | void (*set_cursor_sdr_white_level)(struct pipe_ctx *pipe); |
| 222 | 231 | |
| 232 | + void (*setup_periodic_interrupt)(struct pipe_ctx *pipe_ctx, enum vline_select vline); | |
| 233 | + void (*setup_vupdate_interrupt)(struct pipe_ctx *pipe_ctx); | |
| 234 | + | |
| 223 | 235 | }; |
| 224 | 236 | |
| 225 | 237 | void color_space_to_black_color( |
| @@ -131,6 +131,7 @@ · 130 unmodified lines | ||
| 131 | 131 | #define INTERNAL_REV_RAVEN_A0 0x00 /* First spin of Raven */ |
| 132 | 132 | #define RAVEN_A0 0x01 |
| 133 | 133 | #define RAVEN_B0 0x21 |
| 134 | +#define PICASSO_A0 0x41 | |
| 134 | 135 | #if defined(CONFIG_DRM_AMD_DC_DCN1_01) |
| 135 | 136 | /* DCN1_01 */ |
| 136 | 137 | #define RAVEN2_A0 0x81 |
| @@ -165,4 +166,6 @@ · 28 unmodified lines | ||
| 165 | 166 | |
| 166 | 167 | #define FAMILY_UNKNOWN 0xFF |
| 167 | 168 | |
| 169 | + | |
| 170 | + | |
| 168 | 171 | #endif /* __DAL_ASIC_ID_H__ */ |
| @@ -165,18 +165,11 @@ struct iram_table_v_2_2 { · 164 unmodified lines | ||
| 165 | 165 | }; |
| 166 | 166 | #pragma pack(pop) |
| 167 | 167 | |
| 168 | −static uint16_t backlight_8_to_16(unsigned int backlight_8bit) | |
| 169 | −{ | |
| 170 | − return (uint16_t)(backlight_8bit * 0x101); | |
| 171 | −} | |
| 172 | − | |
| 173 | 168 | static void fill_backlight_transform_table(struct dmcu_iram_parameters params, |
| 174 | 169 | struct iram_table_v_2 *table) |
| 175 | 170 | { |
| 176 | 171 | unsigned int i; |
| 177 | 172 | unsigned int num_entries = NUM_BL_CURVE_SEGS; |
| 178 | − unsigned int query_input_8bit; | |
| 179 | − unsigned int query_output_8bit; | |
| 180 | 173 | unsigned int lut_index; |
| 181 | 174 | |
| 182 | 175 | table->backlight_thresholds[0] = 0; |
| @@ -194,16 +187,13 @@ static void fill_backlight_transform_table(struct dmcu_iram_parameters params, · 11 unmodified lines | ||
| 194 | 187 | * format U4.10. |
| 195 | 188 | */ |
| 196 | 189 | for (i = 1; i+1 < num_entries; i++) { |
| 197 | − query_input_8bit = DIV_ROUNDUP((i * 256), num_entries); | |
| 198 | − | |
| 199 | 190 | lut_index = (params.backlight_lut_array_size - 1) * i / (num_entries - 1); |
| 200 | 191 | ASSERT(lut_index < params.backlight_lut_array_size); |
| 201 | − query_output_8bit = params.backlight_lut_array[lut_index] >> 8; | |
| 202 | 192 | |
| 203 | 193 | table->backlight_thresholds[i] = |
| 204 | − backlight_8_to_16(query_input_8bit); | |
| 194 | + cpu_to_be16(DIV_ROUNDUP((i * 65536), num_entries)); | |
| 205 | 195 | table->backlight_offsets[i] = |
| 206 | − backlight_8_to_16(query_output_8bit); | |
| 196 | + cpu_to_be16(params.backlight_lut_array[lut_index]); | |
| 207 | 197 | } |
| 208 | 198 | } |
| 209 | 199 | |
| @@ -212,8 +202,6 @@ static void fill_backlight_transform_table_v_2_2(struct dmcu_iram_parameters par · 2 unmodified lines | ||
| 212 | 202 | { |
| 213 | 203 | unsigned int i; |
| 214 | 204 | unsigned int num_entries = NUM_BL_CURVE_SEGS; |
| 215 | − unsigned int query_input_8bit; | |
| 216 | − unsigned int query_output_8bit; | |
| 217 | 205 | unsigned int lut_index; |
| 218 | 206 | |
| 219 | 207 | table->backlight_thresholds[0] = 0; |
| @@ -231,16 +219,13 @@ static void fill_backlight_transform_table_v_2_2(struct dmcu_iram_parameters par · 11 unmodified lines | ||
| 231 | 219 | * format U4.10. |
| 232 | 220 | */ |
| 233 | 221 | for (i = 1; i+1 < num_entries; i++) { |
| 234 | − query_input_8bit = DIV_ROUNDUP((i * 256), num_entries); | |
| 235 | − | |
| 236 | 222 | lut_index = (params.backlight_lut_array_size - 1) * i / (num_entries - 1); |
| 237 | 223 | ASSERT(lut_index < params.backlight_lut_array_size); |
| 238 | − query_output_8bit = params.backlight_lut_array[lut_index] >> 8; | |
| 239 | 224 | |
| 240 | 225 | table->backlight_thresholds[i] = |
| 241 | − backlight_8_to_16(query_input_8bit); | |
| 226 | + cpu_to_be16(DIV_ROUNDUP((i * 65536), num_entries)); | |
| 242 | 227 | table->backlight_offsets[i] = |
| 243 | − backlight_8_to_16(query_output_8bit); | |
| 228 | + cpu_to_be16(params.backlight_lut_array[lut_index]); | |
| 244 | 229 | } |
| 245 | 230 | } |
| 246 | 231 | |
| @@ -137,20 +137,17 @@ struct kgd2kfd_shared_resources { · 136 unmodified lines | ||
| 137 | 137 | /* Bit n == 1 means Queue n is available for KFD */ |
| 138 | 138 | DECLARE_BITMAP(queue_bitmap, KGD_MAX_QUEUES); |
| 139 | 139 | |
| 140 | − /* Doorbell assignments (SOC15 and later chips only). Only | |
| 140 | + /* SDMA doorbell assignments (SOC15 and later chips only). Only | |
| 141 | 141 | * specific doorbells are routed to each SDMA engine. Others |
| 142 | 142 | * are routed to IH and VCN. They are not usable by the CP. |
| 143 | − * | |
| 144 | − * Any doorbell number D that satisfies the following condition | |
| 145 | − * is reserved: (D & reserved_doorbell_mask) == reserved_doorbell_val | |
| 146 | − * | |
| 147 | − * KFD currently uses 1024 (= 0x3ff) doorbells per process. If | |
| 148 | − * doorbells 0x0e0-0x0ff and 0x2e0-0x2ff are reserved, that means | |
| 149 | − * mask would be set to 0x1e0 and val set to 0x0e0. | |
| 150 | 143 | */ |
| 151 | − unsigned int sdma_doorbell[2][8]; | |
| 152 | − unsigned int reserved_doorbell_mask; | |
| 153 | − unsigned int reserved_doorbell_val; | |
| 144 | + uint32_t *sdma_doorbell_idx; | |
| 145 | + | |
| 146 | + /* From SOC15 onward, the doorbell index range not usable for CP | |
| 147 | + * queues. | |
| 148 | + */ | |
| 149 | + uint32_t non_cp_doorbells_start; | |
| 150 | + uint32_t non_cp_doorbells_end; | |
| 154 | 151 | |
| 155 | 152 | /* Base address of doorbell aperture. */ |
| 156 | 153 | phys_addr_t doorbell_physical_address; |
| @@ -139,12 +139,10 @@ static int smu10_construct_max_power_limits_table(struct pp_hwmgr *hwmgr, · 138 unmodified lines | ||
| 139 | 139 | static int smu10_init_dynamic_state_adjustment_rule_settings( |
| 140 | 140 | struct pp_hwmgr *hwmgr) |
| 141 | 141 | { |
| 142 | − uint32_t table_size = | |
| 143 | − sizeof(struct phm_clock_voltage_dependency_table) + | |
| 144 | − (7 * sizeof(struct phm_clock_voltage_dependency_record)); | |
| 142 | + struct phm_clock_voltage_dependency_table *table_clk_vlt; | |
| 145 | 143 | |
| 146 | − struct phm_clock_voltage_dependency_table *table_clk_vlt = | |
| 147 | − kzalloc(table_size, GFP_KERNEL); | |
| 144 | + table_clk_vlt = kzalloc(struct_size(table_clk_vlt, entries, 7), | |
| 145 | + GFP_KERNEL); | |
| 148 | 146 | |
| 149 | 147 | if (NULL == table_clk_vlt) { |
| 150 | 148 | pr_err("Can not allocate memory!\n"); |
| @@ -3681,10 +3681,12 @@ static int smu7_request_link_speed_change_before_state_change( · 3680 unmodified lines | ||
| 3681 | 3681 | data->force_pcie_gen = PP_PCIEGen2; |
| 3682 | 3682 | if (current_link_speed == PP_PCIEGen2) |
| 3683 | 3683 | break; |
| 3684 | + /* fall through */ | |
| 3684 | 3685 | case PP_PCIEGen2: |
| 3685 | 3686 | if (0 == amdgpu_acpi_pcie_performance_request(hwmgr->adev, PCIE_PERF_REQ_GEN2, false)) |
| 3686 | 3687 | break; |
| 3687 | 3688 | #endif |
| 3689 | + /* fall through */ | |
| 3688 | 3690 | default: |
| 3689 | 3691 | data->force_pcie_gen = smu7_get_current_pcie_speed(hwmgr); |
| 3690 | 3692 | break; |
| @@ -1211,7 +1211,7 @@ int smu7_power_control_set_level(struct pp_hwmgr *hwmgr) · 1210 unmodified lines | ||
| 1211 | 1211 | hwmgr->platform_descriptor.TDPAdjustment : |
| 1212 | 1212 | (-1 * hwmgr->platform_descriptor.TDPAdjustment); |
| 1213 | 1213 | |
| 1214 | − if (hwmgr->chip_id > CHIP_TONGA) | |
| 1214 | + if (hwmgr->chip_id > CHIP_TONGA) | |
| 1215 | 1215 | target_tdp = ((100 + adjust_percent) * (int)(cac_table->usTDP * 256)) / 100; |
| 1216 | 1216 | else |
| 1217 | 1217 | target_tdp = ((100 + adjust_percent) * (int)(cac_table->usConfigurableTDP * 256)) / 100; |
| @@ -272,12 +272,10 @@ static int smu8_init_dynamic_state_adjustment_rule_settings( · 271 unmodified lines | ||
| 272 | 272 | struct pp_hwmgr *hwmgr, |
| 273 | 273 | ATOM_CLK_VOLT_CAPABILITY *disp_voltage_table) |
| 274 | 274 | { |
| 275 | − uint32_t table_size = | |
| 276 | − sizeof(struct phm_clock_voltage_dependency_table) + | |
| 277 | − (7 * sizeof(struct phm_clock_voltage_dependency_record)); | |
| 275 | + struct phm_clock_voltage_dependency_table *table_clk_vlt; | |
| 278 | 276 | |
| 279 | − struct phm_clock_voltage_dependency_table *table_clk_vlt = | |
| 280 | − kzalloc(table_size, GFP_KERNEL); | |
| 277 | + table_clk_vlt = kzalloc(struct_size(table_clk_vlt, entries, 7), | |
| 278 | + GFP_KERNEL); | |
| 281 | 279 | |
| 282 | 280 | if (NULL == table_clk_vlt) { |
| 283 | 281 | pr_err("Can not allocate memory!\n"); |
| @@ -1,3 +1,25 @@ | ||
| 1 | +/* | |
| 2 | + * Copyright 2018 Advanced Micro Devices, Inc. | |
| 3 | + * | |
| 4 | + * Permission is hereby granted, free of charge, to any person obtaining a | |
| 5 | + * copy of this software and associated documentation files (the "Software"), | |
| 6 | + * to deal in the Software without restriction, including without limitation | |
| 7 | + * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 8 | + * and/or sell copies of the Software, and to permit persons to whom the | |
| 9 | + * Software is furnished to do so, subject to the following conditions: | |
| 10 | + * | |
| 11 | + * The above copyright notice and this permission notice shall be included in | |
| 12 | + * all copies or substantial portions of the Software. | |
| 13 | + * | |
| 14 | + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 15 | + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 16 | + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 17 | + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 18 | + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 19 | + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 20 | + * OTHER DEALINGS IN THE SOFTWARE. | |
| 21 | + * | |
| 22 | + */ | |
| 1 | 23 | #include "amdgpu.h" |
| 2 | 24 | #include "soc15.h" |
| 3 | 25 | #include "soc15_hw_ip.h" |
| @@ -114,7 +136,7 @@ int vega10_baco_set_state(struct pp_hwmgr *hwmgr, enum BACO_STATE state) · 110 unmodified lines | ||
| 114 | 136 | if (soc15_baco_program_registers(hwmgr, pre_baco_tbl, |
| 115 | 137 | ARRAY_SIZE(pre_baco_tbl))) { |
| 116 | 138 | if (smum_send_msg_to_smc(hwmgr, PPSMC_MSG_EnterBaco)) |
| 117 | − return -1; | |
| 139 | + return -EINVAL; | |
| 118 | 140 | |
| 119 | 141 | if (soc15_baco_program_registers(hwmgr, enter_baco_tbl, |
| 120 | 142 | ARRAY_SIZE(enter_baco_tbl))) |
| @@ -132,5 +154,5 @@ int vega10_baco_set_state(struct pp_hwmgr *hwmgr, enum BACO_STATE state) · 11 unmodified lines | ||
| 132 | 154 | } |
| 133 | 155 | } |
| 134 | 156 | |
| 135 | − return -1; | |
| 157 | + return -EINVAL; | |
| 136 | 158 | } |
| @@ -20,8 +20,8 @@ · 19 unmodified lines | ||
| 20 | 20 | * OTHER DEALINGS IN THE SOFTWARE. |
| 21 | 21 | * |
| 22 | 22 | */ |
| 23 | −#ifndef __VEGA10_BOCO_H__ | |
| 24 | −#define __VEGA10_BOCO_H__ | |
| 23 | +#ifndef __VEGA10_BACO_H__ | |
| 24 | +#define __VEGA10_BACO_H__ | |
| 25 | 25 | #include "hwmgr.h" |
| 26 | 26 | #include "common_baco.h" |
| 27 | 27 | |
| @@ -1,3 +1,25 @@ | ||
| 1 | +/* | |
| 2 | + * Copyright 2018 Advanced Micro Devices, Inc. | |
| 3 | + * | |
| 4 | + * Permission is hereby granted, free of charge, to any person obtaining a | |
| 5 | + * copy of this software and associated documentation files (the "Software"), | |
| 6 | + * to deal in the Software without restriction, including without limitation | |
| 7 | + * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 8 | + * and/or sell copies of the Software, and to permit persons to whom the | |
| 9 | + * Software is furnished to do so, subject to the following conditions: | |
| 10 | + * | |
| 11 | + * The above copyright notice and this permission notice shall be included in | |
| 12 | + * all copies or substantial portions of the Software. | |
| 13 | + * | |
| 14 | + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 15 | + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 16 | + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 17 | + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 18 | + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 19 | + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 20 | + * OTHER DEALINGS IN THE SOFTWARE. | |
| 21 | + * | |
| 22 | + */ | |
| 1 | 23 | #include "amdgpu.h" |
| 2 | 24 | #include "soc15.h" |
| 3 | 25 | #include "soc15_hw_ip.h" |
| @@ -67,14 +89,14 @@ int vega20_baco_set_state(struct pp_hwmgr *hwmgr, enum BACO_STATE state) · 63 unmodified lines | ||
| 67 | 89 | |
| 68 | 90 | |
| 69 | 91 | if(smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_EnterBaco, 0)) |
| 70 | − return -1; | |
| 92 | + return -EINVAL; | |
| 71 | 93 | |
| 72 | 94 | } else if (state == BACO_STATE_OUT) { |
| 73 | 95 | if (smum_send_msg_to_smc(hwmgr, PPSMC_MSG_ExitBaco)) |
| 74 | − return -1; | |
| 96 | + return -EINVAL; | |
| 75 | 97 | if (!soc15_baco_program_registers(hwmgr, clean_baco_tbl, |
| 76 | 98 | ARRAY_SIZE(clean_baco_tbl))) |
| 77 | − return -1; | |
| 99 | + return -EINVAL; | |
| 78 | 100 | } |
| 79 | 101 | |
| 80 | 102 | return 0; |
| @@ -20,8 +20,8 @@ · 19 unmodified lines | ||
| 20 | 20 | * OTHER DEALINGS IN THE SOFTWARE. |
| 21 | 21 | * |
| 22 | 22 | */ |
| 23 | −#ifndef __VEGA20_BOCO_H__ | |
| 24 | −#define __VEGA20_BOCO_H__ | |
| 23 | +#ifndef __VEGA20_BACO_H__ | |
| 24 | +#define __VEGA20_BACO_H__ | |
| 25 | 25 | #include "hwmgr.h" |
| 26 | 26 | #include "common_baco.h" |
| 27 | 27 | |
| @@ -3456,7 +3456,7 @@ static int vega20_apply_clocks_adjust_rules(struct pp_hwmgr *hwmgr) · 3455 unmodified lines | ||
| 3456 | 3456 | disable_mclk_switching = ((1 < hwmgr->display_config->num_display) && |
| 3457 | 3457 | !hwmgr->display_config->multi_monitor_in_sync) || |
| 3458 | 3458 | vblank_too_short; |
| 3459 | − latency = hwmgr->display_config->dce_tolerable_mclk_in_active_latency; | |
| 3459 | + latency = hwmgr->display_config->dce_tolerable_mclk_in_active_latency; | |
| 3460 | 3460 | |
| 3461 | 3461 | /* gfxclk */ |
| 3462 | 3462 | dpm_table = &(data->dpm_table.gfx_table); |
| @@ -29,6 +29,10 @@ · 28 unmodified lines | ||
| 29 | 29 | #include <drm/amdgpu_drm.h> |
| 30 | 30 | #include "smumgr.h" |
| 31 | 31 | |
| 32 | +MODULE_FIRMWARE("amdgpu/bonaire_smc.bin"); | |
| 33 | +MODULE_FIRMWARE("amdgpu/bonaire_k_smc.bin"); | |
| 34 | +MODULE_FIRMWARE("amdgpu/hawaii_smc.bin"); | |
| 35 | +MODULE_FIRMWARE("amdgpu/hawaii_k_smc.bin"); | |
| 32 | 36 | MODULE_FIRMWARE("amdgpu/topaz_smc.bin"); |
| 33 | 37 | MODULE_FIRMWARE("amdgpu/topaz_k_smc.bin"); |
| 34 | 38 | MODULE_FIRMWARE("amdgpu/tonga_smc.bin"); |
| @@ -145,6 +145,10 @@ static int bochs_pci_probe(struct pci_dev *pdev, · 144 unmodified lines | ||
| 145 | 145 | if (IS_ERR(dev)) |
| 146 | 146 | return PTR_ERR(dev); |
| 147 | 147 | |
| 148 | + ret = pci_enable_device(pdev); | |
| 149 | + if (ret) | |
| 150 | + goto err_free_dev; | |
| 151 | + | |
| 148 | 152 | dev->pdev = pdev; |
| 149 | 153 | pci_set_drvdata(pdev, dev); |
| 150 | 154 | |
| @@ -1608,6 +1608,15 @@ int drm_atomic_helper_async_check(struct drm_device *dev, · 1607 unmodified lines | ||
| 1608 | 1608 | old_plane_state->crtc != new_plane_state->crtc) |
| 1609 | 1609 | return -EINVAL; |
| 1610 | 1610 | |
| 1611 | + /* | |
| 1612 | + * FIXME: Since prepare_fb and cleanup_fb are always called on | |
| 1613 | + * the new_plane_state for async updates we need to block framebuffer | |
| 1614 | + * changes. This prevents use of a fb that's been cleaned up and | |
| 1615 | + * double cleanups from occuring. | |
| 1616 | + */ | |
| 1617 | + if (old_plane_state->fb != new_plane_state->fb) | |
| 1618 | + return -EINVAL; | |
| 1619 | + | |
| 1611 | 1620 | funcs = plane->helper_private; |
| 1612 | 1621 | if (!funcs->atomic_async_update) |
| 1613 | 1622 | return -EINVAL; |
| @@ -262,6 +262,18 @@ void drm_file_free(struct drm_file *file) · 261 unmodified lines | ||
| 262 | 262 | kfree(file); |
| 263 | 263 | } |
| 264 | 264 | |
| 265 | +static void drm_close_helper(struct file *filp) | |
| 266 | +{ | |
| 267 | + struct drm_file *file_priv = filp->private_data; | |
| 268 | + struct drm_device *dev = file_priv->minor->dev; | |
| 269 | + | |
| 270 | + mutex_lock(&dev->filelist_mutex); | |
| 271 | + list_del(&file_priv->lhead); | |
| 272 | + mutex_unlock(&dev->filelist_mutex); | |
| 273 | + | |
| 274 | + drm_file_free(file_priv); | |
| 275 | +} | |
| 276 | + | |
| 265 | 277 | static int drm_setup(struct drm_device * dev) |
| 266 | 278 | { |
| 267 | 279 | int ret; |
| @@ -318,8 +330,10 @@ int drm_open(struct inode *inode, struct file *filp) · 50 unmodified lines | ||
| 318 | 330 | goto err_undo; |
| 319 | 331 | if (need_setup) { |
| 320 | 332 | retcode = drm_setup(dev); |
| 321 | − if (retcode) | |
| 333 | + if (retcode) { | |
| 334 | + drm_close_helper(filp); | |
| 322 | 335 | goto err_undo; |
| 336 | + } | |
| 323 | 337 | } |
| 324 | 338 | return 0; |
| 325 | 339 | |
| @@ -473,11 +487,7 @@ int drm_release(struct inode *inode, struct file *filp) · 147 unmodified lines | ||
| 473 | 487 | |
| 474 | 488 | DRM_DEBUG("open_count = %d\n", dev->open_count); |
| 475 | 489 | |
| 476 | − mutex_lock(&dev->filelist_mutex); | |
| 477 | − list_del(&file_priv->lhead); | |
| 478 | − mutex_unlock(&dev->filelist_mutex); | |
| 479 | − | |
| 480 | − drm_file_free(file_priv); | |
| 490 | + drm_close_helper(filp); | |
| 481 | 491 | |
| 482 | 492 | if (!--dev->open_count) |
| 483 | 493 | drm_lastclose(dev); |
| @@ -508,6 +508,13 @@ int drm_version(struct drm_device *dev, void *data, · 507 unmodified lines | ||
| 508 | 508 | return err; |
| 509 | 509 | } |
| 510 | 510 | |
| 511 | +static inline bool | |
| 512 | +drm_render_driver_and_ioctl(const struct drm_device *dev, u32 flags) | |
| 513 | +{ | |
| 514 | + return drm_core_check_feature(dev, DRIVER_RENDER) && | |
| 515 | + (flags & DRM_RENDER_ALLOW); | |
| 516 | +} | |
| 517 | + | |
| 511 | 518 | /** |
| 512 | 519 | * drm_ioctl_permit - Check ioctl permissions against caller |
| 513 | 520 | * |
| @@ -522,14 +529,19 @@ int drm_version(struct drm_device *dev, void *data, · 8 unmodified lines | ||
| 522 | 529 | */ |
| 523 | 530 | int drm_ioctl_permit(u32 flags, struct drm_file *file_priv) |
| 524 | 531 | { |
| 532 | + const struct drm_device *dev = file_priv->minor->dev; | |
| 533 | + | |
| 525 | 534 | /* ROOT_ONLY is only for CAP_SYS_ADMIN */ |
| 526 | 535 | if (unlikely((flags & DRM_ROOT_ONLY) && !capable(CAP_SYS_ADMIN))) |
| 527 | 536 | return -EACCES; |
| 528 | 537 | |
| 529 | − /* AUTH is only for authenticated or render client */ | |
| 530 | − if (unlikely((flags & DRM_AUTH) && !drm_is_render_client(file_priv) && | |
| 531 | − !file_priv->authenticated)) | |
| 532 | − return -EACCES; | |
| 538 | + /* AUTH is only for master ... */ | |
| 539 | + if (unlikely((flags & DRM_AUTH) && drm_is_primary_client(file_priv))) { | |
| 540 | + /* authenticated ones, or render capable on DRM_RENDER_ALLOW. */ | |
| 541 | + if (!file_priv->authenticated && | |
| 542 | + !drm_render_driver_and_ioctl(dev, flags)) | |
| 543 | + return -EACCES; | |
| 544 | + } | |
| 533 | 545 | |
| 534 | 546 | /* MASTER is only for master or control clients */ |
| 535 | 547 | if (unlikely((flags & DRM_MASTER) && |
| @@ -570,7 +582,7 @@ static const struct drm_ioctl_desc drm_ioctls[] = { · 34 unmodified lines | ||
| 570 | 582 | DRM_IOCTL_DEF(DRM_IOCTL_SET_UNIQUE, drm_invalid_op, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY), |
| 571 | 583 | DRM_IOCTL_DEF(DRM_IOCTL_BLOCK, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY), |
| 572 | 584 | DRM_IOCTL_DEF(DRM_IOCTL_UNBLOCK, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY), |
| 573 | − DRM_IOCTL_DEF(DRM_IOCTL_AUTH_MAGIC, drm_authmagic, DRM_AUTH|DRM_UNLOCKED|DRM_MASTER), | |
| 585 | + DRM_IOCTL_DEF(DRM_IOCTL_AUTH_MAGIC, drm_authmagic, DRM_UNLOCKED|DRM_MASTER), | |
| 574 | 586 | |
| 575 | 587 | DRM_IOCTL_DEF(DRM_IOCTL_ADD_MAP, drm_legacy_addmap_ioctl, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY), |
| 576 | 588 | DRM_IOCTL_DEF(DRM_IOCTL_RM_MAP, drm_legacy_rmmap_ioctl, DRM_AUTH), |
| @@ -338,8 +338,8 @@ static bool intel_fb_initial_config(struct drm_fb_helper *fb_helper, · 337 unmodified lines | ||
| 338 | 338 | bool *enabled, int width, int height) |
| 339 | 339 | { |
| 340 | 340 | struct drm_i915_private *dev_priv = to_i915(fb_helper->dev); |
| 341 | − unsigned long conn_configured, conn_seq, mask; | |
| 342 | 341 | unsigned int count = min(fb_helper->connector_count, BITS_PER_LONG); |
| 342 | + unsigned long conn_configured, conn_seq; | |
| 343 | 343 | int i, j; |
| 344 | 344 | bool *save_enabled; |
| 345 | 345 | bool fallback = true, ret = true; |
| @@ -357,10 +357,9 @@ static bool intel_fb_initial_config(struct drm_fb_helper *fb_helper, · 11 unmodified lines | ||
| 357 | 357 | drm_modeset_backoff(&ctx); |
| 358 | 358 | |
| 359 | 359 | memcpy(save_enabled, enabled, count); |
| 360 | − mask = GENMASK(count - 1, 0); | |
| 360 | + conn_seq = GENMASK(count - 1, 0); | |
| 361 | 361 | conn_configured = 0; |
| 362 | 362 | retry: |
| 363 | − conn_seq = conn_configured; | |
| 364 | 363 | for (i = 0; i < count; i++) { |
| 365 | 364 | struct drm_fb_helper_connector *fb_conn; |
| 366 | 365 | struct drm_connector *connector; |
| @@ -373,7 +372,8 @@ retry: · 6 unmodified lines | ||
| 373 | 372 | if (conn_configured & BIT(i)) |
| 374 | 373 | continue; |
| 375 | 374 | |
| 376 | − if (conn_seq == 0 && !connector->has_tile) | |
| 375 | + /* First pass, only consider tiled connectors */ | |
| 376 | + if (conn_seq == GENMASK(count - 1, 0) && !connector->has_tile) | |
| 377 | 377 | continue; |
| 378 | 378 | |
| 379 | 379 | if (connector->status == connector_status_connected) |
| @@ -477,8 +477,10 @@ retry: · 97 unmodified lines | ||
| 477 | 477 | conn_configured |= BIT(i); |
| 478 | 478 | } |
| 479 | 479 | |
| 480 | − if ((conn_configured & mask) != mask && conn_configured != conn_seq) | |
| 480 | + if (conn_configured != conn_seq) { /* repeat until no more are found */ | |
| 481 | + conn_seq = conn_configured; | |
| 481 | 482 | goto retry; |
| 483 | + } | |
| 482 | 484 | |
| 483 | 485 | /* |
| 484 | 486 | * If the BIOS didn't enable everything it could, fall back to have the |
| @@ -4,7 +4,7 @@ config DRM_IMX · 3 unmodified lines | ||
| 4 | 4 | select VIDEOMODE_HELPERS |
| 5 | 5 | select DRM_GEM_CMA_HELPER |
| 6 | 6 | select DRM_KMS_CMA_HELPER |
| 7 | − depends on DRM && (ARCH_MXC || ARCH_MULTIPLATFORM) | |
| 7 | + depends on DRM && (ARCH_MXC || ARCH_MULTIPLATFORM || COMPILE_TEST) | |
| 8 | 8 | depends on IMX_IPUV3_CORE |
| 9 | 9 | help |
| 10 | 10 | enable i.MX graphics support |
| @@ -18,6 +18,7 @@ config DRM_IMX_PARALLEL_DISPLAY · 7 unmodified lines | ||
| 18 | 18 | config DRM_IMX_TVE |
| 19 | 19 | tristate "Support for TV and VGA displays" |
| 20 | 20 | depends on DRM_IMX |
| 21 | + depends on COMMON_CLK | |
| 21 | 22 | select REGMAP_MMIO |
| 22 | 23 | help |
| 23 | 24 | Choose this to enable the internal Television Encoder (TVe) |
| @@ -49,11 +49,7 @@ static int imx_drm_atomic_check(struct drm_device *dev, · 48 unmodified lines | ||
| 49 | 49 | { |
| 50 | 50 | int ret; |
| 51 | 51 | |
| 52 | − ret = drm_atomic_helper_check_modeset(dev, state); | |
| 53 | − if (ret) | |
| 54 | − return ret; | |
| 55 | − | |
| 56 | − ret = drm_atomic_helper_check_planes(dev, state); | |
| 52 | + ret = drm_atomic_helper_check(dev, state); | |
| 57 | 53 | if (ret) |
| 58 | 54 | return ret; |
| 59 | 55 | |
| @@ -229,6 +225,7 @@ static int imx_drm_bind(struct device *dev) · 169 unmodified lines | ||
| 229 | 225 | drm->mode_config.funcs = &imx_drm_mode_config_funcs; |
| 230 | 226 | drm->mode_config.helper_private = &imx_drm_mode_config_helpers; |
| 231 | 227 | drm->mode_config.allow_fb_modifiers = true; |
| 228 | + drm->mode_config.normalize_zpos = true; | |
| 232 | 229 | |
| 233 | 230 | drm_mode_config_init(drm); |
| 234 | 231 | |
| @@ -34,6 +34,7 @@ struct ipu_crtc { · 33 unmodified lines | ||
| 34 | 34 | struct ipu_dc *dc; |
| 35 | 35 | struct ipu_di *di; |
| 36 | 36 | int irq; |
| 37 | + struct drm_pending_vblank_event *event; | |
| 37 | 38 | }; |
| 38 | 39 | |
| 39 | 40 | static inline struct ipu_crtc *to_ipu_crtc(struct drm_crtc *crtc) |
| @@ -173,8 +174,31 @@ static const struct drm_crtc_funcs ipu_crtc_funcs = { · 133 unmodified lines | ||
| 173 | 174 | static irqreturn_t ipu_irq_handler(int irq, void *dev_id) |
| 174 | 175 | { |
| 175 | 176 | struct ipu_crtc *ipu_crtc = dev_id; |
| 177 | + struct drm_crtc *crtc = &ipu_crtc->base; | |
| 178 | + unsigned long flags; | |
| 179 | + int i; | |
| 180 | + | |
| 181 | + drm_crtc_handle_vblank(crtc); | |
| 182 | + | |
| 183 | + if (ipu_crtc->event) { | |
| 184 | + for (i = 0; i < ARRAY_SIZE(ipu_crtc->plane); i++) { | |
| 185 | + struct ipu_plane *plane = ipu_crtc->plane[i]; | |
| 176 | 186 | |
| 177 | − drm_crtc_handle_vblank(&ipu_crtc->base); | |
| 187 | + if (!plane) | |
| 188 | + continue; | |
| 189 | + | |
| 190 | + if (ipu_plane_atomic_update_pending(&plane->base)) | |
| 191 | + break; | |
| 192 | + } | |
| 193 | + | |
| 194 | + if (i == ARRAY_SIZE(ipu_crtc->plane)) { | |
| 195 | + spin_lock_irqsave(&crtc->dev->event_lock, flags); | |
| 196 | + drm_crtc_send_vblank_event(crtc, ipu_crtc->event); | |
| 197 | + ipu_crtc->event = NULL; | |
| 198 | + drm_crtc_vblank_put(crtc); | |
| 199 | + spin_unlock_irqrestore(&crtc->dev->event_lock, flags); | |
| 200 | + } | |
| 201 | + } | |
| 178 | 202 | |
| 179 | 203 | return IRQ_HANDLED; |
| 180 | 204 | } |
| @@ -223,8 +247,10 @@ static void ipu_crtc_atomic_flush(struct drm_crtc *crtc, · 42 unmodified lines | ||
| 223 | 247 | { |
| 224 | 248 | spin_lock_irq(&crtc->dev->event_lock); |
| 225 | 249 | if (crtc->state->event) { |
| 250 | + struct ipu_crtc *ipu_crtc = to_ipu_crtc(crtc); | |
| 251 | + | |
| 226 | 252 | WARN_ON(drm_crtc_vblank_get(crtc)); |
| 227 | − drm_crtc_arm_vblank_event(crtc, crtc->state->event); | |
| 253 | + ipu_crtc->event = crtc->state->event; | |
| 228 | 254 | crtc->state->event = NULL; |
| 229 | 255 | } |
| 230 | 256 | spin_unlock_irq(&crtc->dev->event_lock); |
| @@ -273,6 +273,7 @@ static void ipu_plane_destroy(struct drm_plane *plane) · 272 unmodified lines | ||
| 273 | 273 | |
| 274 | 274 | static void ipu_plane_state_reset(struct drm_plane *plane) |
| 275 | 275 | { |
| 276 | + unsigned int zpos = (plane->type == DRM_PLANE_TYPE_PRIMARY) ? 0 : 1; | |
| 276 | 277 | struct ipu_plane_state *ipu_state; |
| 277 | 278 | |
| 278 | 279 | if (plane->state) { |
| @@ -284,8 +285,11 @@ static void ipu_plane_state_reset(struct drm_plane *plane) · 5 unmodified lines | ||
| 284 | 285 | |
| 285 | 286 | ipu_state = kzalloc(sizeof(*ipu_state), GFP_KERNEL); |
| 286 | 287 | |
| 287 | − if (ipu_state) | |
| 288 | + if (ipu_state) { | |
| 288 | 289 | __drm_atomic_helper_plane_reset(plane, &ipu_state->base); |
| 290 | + ipu_state->base.zpos = zpos; | |
| 291 | + ipu_state->base.normalized_zpos = zpos; | |
| 292 | + } | |
| 289 | 293 | } |
| 290 | 294 | |
| 291 | 295 | static struct drm_plane_state * |
| @@ -560,6 +564,25 @@ static void ipu_plane_atomic_update(struct drm_plane *plane, · 268 unmodified lines | ||
| 560 | 564 | if (ipu_plane->dp_flow == IPU_DP_FLOW_SYNC_FG) |
| 561 | 565 | ipu_dp_set_window_pos(ipu_plane->dp, dst->x1, dst->y1); |
| 562 | 566 | |
| 567 | + switch (ipu_plane->dp_flow) { | |
| 568 | + case IPU_DP_FLOW_SYNC_BG: | |
| 569 | + if (state->normalized_zpos == 1) { | |
| 570 | + ipu_dp_set_global_alpha(ipu_plane->dp, | |
| 571 | + !fb->format->has_alpha, 0xff, | |
| 572 | + true); | |
| 573 | + } else { | |
| 574 | + ipu_dp_set_global_alpha(ipu_plane->dp, true, 0, true); | |
| 575 | + } | |
| 576 | + break; | |
| 577 | + case IPU_DP_FLOW_SYNC_FG: | |
| 578 | + if (state->normalized_zpos == 1) { | |
| 579 | + ipu_dp_set_global_alpha(ipu_plane->dp, | |
| 580 | + !fb->format->has_alpha, 0xff, | |
| 581 | + false); | |
| 582 | + } | |
| 583 | + break; | |
| 584 | + } | |
| 585 | + | |
| 563 | 586 | eba = drm_plane_state_to_eba(state, 0); |
| 564 | 587 | |
| 565 | 588 | /* |
| @@ -582,6 +605,7 @@ static void ipu_plane_atomic_update(struct drm_plane *plane, · 16 unmodified lines | ||
| 582 | 605 | active = ipu_idmac_get_current_buffer(ipu_plane->ipu_ch); |
| 583 | 606 | ipu_cpmem_set_buffer(ipu_plane->ipu_ch, !active, eba); |
| 584 | 607 | ipu_idmac_select_buffer(ipu_plane->ipu_ch, !active); |
| 608 | + ipu_plane->next_buf = !active; | |
| 585 | 609 | if (ipu_plane_separate_alpha(ipu_plane)) { |
| 586 | 610 | active = ipu_idmac_get_current_buffer(ipu_plane->alpha_ch); |
| 587 | 611 | ipu_cpmem_set_buffer(ipu_plane->alpha_ch, !active, |
| @@ -595,34 +619,11 @@ static void ipu_plane_atomic_update(struct drm_plane *plane, · 7 unmodified lines | ||
| 595 | 619 | switch (ipu_plane->dp_flow) { |
| 596 | 620 | case IPU_DP_FLOW_SYNC_BG: |
| 597 | 621 | ipu_dp_setup_channel(ipu_plane->dp, ics, IPUV3_COLORSPACE_RGB); |
| 598 | − ipu_dp_set_global_alpha(ipu_plane->dp, true, 0, true); | |
| 599 | 622 | break; |
| 600 | 623 | case IPU_DP_FLOW_SYNC_FG: |
| 601 | 624 | ipu_dp_setup_channel(ipu_plane->dp, ics, |
| 602 | 625 | IPUV3_COLORSPACE_UNKNOWN); |
| 603 | − /* Enable local alpha on partial plane */ | |
| 604 | − switch (fb->format->format) { | |
| 605 | − case DRM_FORMAT_ARGB1555: | |
| 606 | − case DRM_FORMAT_ABGR1555: | |
| 607 | − case DRM_FORMAT_RGBA5551: | |
| 608 | − case DRM_FORMAT_BGRA5551: | |
| 609 | − case DRM_FORMAT_ARGB4444: | |
| 610 | − case DRM_FORMAT_ARGB8888: | |
| 611 | − case DRM_FORMAT_ABGR8888: | |
| 612 | − case DRM_FORMAT_RGBA8888: | |
| 613 | − case DRM_FORMAT_BGRA8888: | |
| 614 | − case DRM_FORMAT_RGB565_A8: | |
| 615 | − case DRM_FORMAT_BGR565_A8: | |
| 616 | − case DRM_FORMAT_RGB888_A8: | |
| 617 | − case DRM_FORMAT_BGR888_A8: | |
| 618 | − case DRM_FORMAT_RGBX8888_A8: | |
| 619 | − case DRM_FORMAT_BGRX8888_A8: | |
| 620 | − ipu_dp_set_global_alpha(ipu_plane->dp, false, 0, false); | |
| 621 | − break; | |
| 622 | − default: | |
| 623 | − ipu_dp_set_global_alpha(ipu_plane->dp, true, 0, true); | |
| 624 | − break; | |
| 625 | − } | |
| 626 | + break; | |
| 626 | 627 | } |
| 627 | 628 | |
| 628 | 629 | ipu_dmfc_config_wait4eot(ipu_plane->dmfc, drm_rect_width(dst)); |
| @@ -709,6 +710,7 @@ static void ipu_plane_atomic_update(struct drm_plane *plane, · 80 unmodified lines | ||
| 709 | 710 | ipu_cpmem_set_buffer(ipu_plane->ipu_ch, 1, eba); |
| 710 | 711 | ipu_idmac_lock_enable(ipu_plane->ipu_ch, num_bursts); |
| 711 | 712 | ipu_plane_enable(ipu_plane); |
| 713 | + ipu_plane->next_buf = -1; | |
| 712 | 714 | } |
| 713 | 715 | |
| 714 | 716 | static const struct drm_plane_helper_funcs ipu_plane_helper_funcs = { |
| @@ -718,6 +720,24 @@ static const struct drm_plane_helper_funcs ipu_plane_helper_funcs = { · 3 unmodified lines | ||
| 718 | 720 | .atomic_update = ipu_plane_atomic_update, |
| 719 | 721 | }; |
| 720 | 722 | |
| 723 | +bool ipu_plane_atomic_update_pending(struct drm_plane *plane) | |
| 724 | +{ | |
| 725 | + struct ipu_plane *ipu_plane = to_ipu_plane(plane); | |
| 726 | + struct drm_plane_state *state = plane->state; | |
| 727 | + struct ipu_plane_state *ipu_state = to_ipu_plane_state(state); | |
| 728 | + | |
| 729 | + /* disabled crtcs must not block the update */ | |
| 730 | + if (!state->crtc) | |
| 731 | + return false; | |
| 732 | + | |
| 733 | + if (ipu_state->use_pre) | |
| 734 | + return ipu_prg_channel_configure_pending(ipu_plane->ipu_ch); | |
| 735 | + else if (ipu_plane->next_buf >= 0) | |
| 736 | + return ipu_idmac_get_current_buffer(ipu_plane->ipu_ch) != | |
| 737 | + ipu_plane->next_buf; | |
| 738 | + | |
| 739 | + return false; | |
| 740 | +} | |
| 721 | 741 | int ipu_planes_assign_pre(struct drm_device *dev, |
| 722 | 742 | struct drm_atomic_state *state) |
| 723 | 743 | { |
| @@ -806,6 +826,7 @@ struct ipu_plane *ipu_plane_init(struct drm_device *dev, struct ipu_soc *ipu, · 82 unmodified lines | ||
| 806 | 826 | { |
| 807 | 827 | struct ipu_plane *ipu_plane; |
| 808 | 828 | const uint64_t *modifiers = ipu_format_modifiers; |
| 829 | + unsigned int zpos = (type == DRM_PLANE_TYPE_PRIMARY) ? 0 : 1; | |
| 809 | 830 | int ret; |
| 810 | 831 | |
| 811 | 832 | DRM_DEBUG_KMS("channel %d, dp flow %d, possible_crtcs=0x%x\n", |
| @@ -836,5 +857,10 @@ struct ipu_plane *ipu_plane_init(struct drm_device *dev, struct ipu_soc *ipu, · 24 unmodified lines | ||
| 836 | 857 | |
| 837 | 858 | drm_plane_helper_add(&ipu_plane->base, &ipu_plane_helper_funcs); |
| 838 | 859 | |
| 860 | + if (dp == IPU_DP_FLOW_SYNC_BG || dp == IPU_DP_FLOW_SYNC_FG) | |
| 861 | + drm_plane_create_zpos_property(&ipu_plane->base, zpos, 0, 1); | |
| 862 | + else | |
| 863 | + drm_plane_create_zpos_immutable_property(&ipu_plane->base, 0); | |
| 864 | + | |
| 839 | 865 | return ipu_plane; |
| 840 | 866 | } |
| @@ -27,6 +27,7 @@ struct ipu_plane { · 26 unmodified lines | ||
| 27 | 27 | int dp_flow; |
| 28 | 28 | |
| 29 | 29 | bool disabling; |
| 30 | + int next_buf; | |
| 30 | 31 | }; |
| 31 | 32 | |
| 32 | 33 | struct ipu_plane *ipu_plane_init(struct drm_device *dev, struct ipu_soc *ipu, |
| @@ -48,5 +49,6 @@ int ipu_plane_irq(struct ipu_plane *plane); · 15 unmodified lines | ||
| 48 | 49 | |
| 49 | 50 | void ipu_plane_disable(struct ipu_plane *ipu_plane, bool disable_dp_channel); |
| 50 | 51 | void ipu_plane_disable_deferred(struct drm_plane *plane); |
| 52 | +bool ipu_plane_atomic_update_pending(struct drm_plane *plane); | |
| 51 | 53 | |
| 52 | 54 | #endif |
| @@ -465,8 +465,6 @@ static void _dpu_crtc_setup_mixer_for_encoder( · 464 unmodified lines | ||
| 465 | 465 | return; |
| 466 | 466 | } |
| 467 | 467 | |
| 468 | − mixer->encoder = enc; | |
| 469 | − | |
| 470 | 468 | cstate->num_mixers++; |
| 471 | 469 | DPU_DEBUG("setup mixer %d: lm %d\n", |
| 472 | 470 | i, mixer->hw_lm->idx - LM_0); |
| @@ -718,11 +716,8 @@ void dpu_crtc_commit_kickoff(struct drm_crtc *crtc, bool async) · 245 unmodified lines | ||
| 718 | 716 | * may delay and flush at an irq event (e.g. ppdone) |
| 719 | 717 | */ |
| 720 | 718 | drm_for_each_encoder_mask(encoder, crtc->dev, |
| 721 | − crtc->state->encoder_mask) { | |
| 722 | − struct dpu_encoder_kickoff_params params = { 0 }; | |
| 723 | − dpu_encoder_prepare_for_kickoff(encoder, ¶ms, async); | |
| 724 | − } | |
| 725 | − | |
| 719 | + crtc->state->encoder_mask) | |
| 720 | + dpu_encoder_prepare_for_kickoff(encoder, async); | |
| 726 | 721 | |
| 727 | 722 | if (!async) { |
| 728 | 723 | /* wait for frame_event_done completion */ |
| @@ -84,14 +84,12 @@ struct dpu_crtc_smmu_state_data { · 83 unmodified lines | ||
| 84 | 84 | * struct dpu_crtc_mixer: stores the map for each virtual pipeline in the CRTC |
| 85 | 85 | * @hw_lm: LM HW Driver context |
| 86 | 86 | * @lm_ctl: CTL Path HW driver context |
| 87 | − * @encoder: Encoder attached to this lm & ctl | |
| 88 | 87 | * @mixer_op_mode: mixer blending operation mode |
| 89 | 88 | * @flush_mask: mixer flush mask for ctl, mixer and pipe |
| 90 | 89 | */ |
| 91 | 90 | struct dpu_crtc_mixer { |
| 92 | 91 | struct dpu_hw_mixer *hw_lm; |
| 93 | 92 | struct dpu_hw_ctl *lm_ctl; |
| 94 | − struct drm_encoder *encoder; | |
| 95 | 93 | u32 mixer_op_mode; |
| 96 | 94 | u32 flush_mask; |
| 97 | 95 | }; |
| @@ -205,7 +205,7 @@ struct dpu_encoder_virt { · 204 unmodified lines | ||
| 205 | 205 | bool idle_pc_supported; |
| 206 | 206 | struct mutex rc_lock; |
| 207 | 207 | enum dpu_enc_rc_states rc_state; |
| 208 | − struct kthread_delayed_work delayed_off_work; | |
| 208 | + struct delayed_work delayed_off_work; | |
| 209 | 209 | struct kthread_work vsync_event_work; |
| 210 | 210 | struct msm_display_topology topology; |
| 211 | 211 | bool mode_set_complete; |
| @@ -742,7 +742,6 @@ static int dpu_encoder_resource_control(struct drm_encoder *drm_enc, · 530 unmodified lines | ||
| 742 | 742 | { |
| 743 | 743 | struct dpu_encoder_virt *dpu_enc; |
| 744 | 744 | struct msm_drm_private *priv; |
| 745 | − struct msm_drm_thread *disp_thread; | |
| 746 | 745 | bool is_vid_mode = false; |
| 747 | 746 | |
| 748 | 747 | if (!drm_enc || !drm_enc->dev || !drm_enc->dev->dev_private || |
| @@ -755,12 +754,6 @@ static int dpu_encoder_resource_control(struct drm_encoder *drm_enc, · 6 unmodified lines | ||
| 755 | 754 | is_vid_mode = dpu_enc->disp_info.capabilities & |
| 756 | 755 | MSM_DISPLAY_CAP_VID_MODE; |
| 757 | 756 | |
| 758 | − if (drm_enc->crtc->index >= ARRAY_SIZE(priv->disp_thread)) { | |
| 759 | − DPU_ERROR("invalid crtc index\n"); | |
| 760 | − return -EINVAL; | |
| 761 | − } | |
| 762 | − disp_thread = &priv->disp_thread[drm_enc->crtc->index]; | |
| 763 | − | |
| 764 | 757 | /* |
| 765 | 758 | * when idle_pc is not supported, process only KICKOFF, STOP and MODESET |
| 766 | 759 | * events and return early for other events (ie wb display). |
| @@ -777,8 +770,7 @@ static int dpu_encoder_resource_control(struct drm_encoder *drm_enc, · 10 unmodified lines | ||
| 777 | 770 | switch (sw_event) { |
| 778 | 771 | case DPU_ENC_RC_EVENT_KICKOFF: |
| 779 | 772 | /* cancel delayed off work, if any */ |
| 780 | − if (kthread_cancel_delayed_work_sync( | |
| 781 | − &dpu_enc->delayed_off_work)) | |
| 773 | + if (cancel_delayed_work_sync(&dpu_enc->delayed_off_work)) | |
| 782 | 774 | DPU_DEBUG_ENC(dpu_enc, "sw_event:%d, work cancelled\n", |
| 783 | 775 | sw_event); |
| 784 | 776 | |
| @@ -837,10 +829,8 @@ static int dpu_encoder_resource_control(struct drm_encoder *drm_enc, · 52 unmodified lines | ||
| 837 | 829 | return 0; |
| 838 | 830 | } |
| 839 | 831 | |
| 840 | − kthread_queue_delayed_work( | |
| 841 | − &disp_thread->worker, | |
| 842 | − &dpu_enc->delayed_off_work, | |
| 843 | − msecs_to_jiffies(dpu_enc->idle_timeout)); | |
| 832 | + queue_delayed_work(priv->wq, &dpu_enc->delayed_off_work, | |
| 833 | + msecs_to_jiffies(dpu_enc->idle_timeout)); | |
| 844 | 834 | |
| 845 | 835 | trace_dpu_enc_rc(DRMID(drm_enc), sw_event, |
| 846 | 836 | dpu_enc->idle_pc_supported, dpu_enc->rc_state, |
| @@ -849,8 +839,7 @@ static int dpu_encoder_resource_control(struct drm_encoder *drm_enc, · 2 unmodified lines | ||
| 849 | 839 | |
| 850 | 840 | case DPU_ENC_RC_EVENT_PRE_STOP: |
| 851 | 841 | /* cancel delayed off work, if any */ |
| 852 | − if (kthread_cancel_delayed_work_sync( | |
| 853 | − &dpu_enc->delayed_off_work)) | |
| 842 | + if (cancel_delayed_work_sync(&dpu_enc->delayed_off_work)) | |
| 854 | 843 | DPU_DEBUG_ENC(dpu_enc, "sw_event:%d, work cancelled\n", |
| 855 | 844 | sw_event); |
| 856 | 845 | |
| @@ -1368,7 +1357,7 @@ static void dpu_encoder_frame_done_callback( · 511 unmodified lines | ||
| 1368 | 1357 | } |
| 1369 | 1358 | } |
| 1370 | 1359 | |
| 1371 | −static void dpu_encoder_off_work(struct kthread_work *work) | |
| 1360 | +static void dpu_encoder_off_work(struct work_struct *work) | |
| 1372 | 1361 | { |
| 1373 | 1362 | struct dpu_encoder_virt *dpu_enc = container_of(work, |
| 1374 | 1363 | struct dpu_encoder_virt, delayed_off_work.work); |
| @@ -1756,15 +1745,14 @@ static void dpu_encoder_vsync_event_work_handler(struct kthread_work *work) · 381 unmodified lines | ||
| 1756 | 1745 | nsecs_to_jiffies(ktime_to_ns(wakeup_time))); |
| 1757 | 1746 | } |
| 1758 | 1747 | |
| 1759 | −void dpu_encoder_prepare_for_kickoff(struct drm_encoder *drm_enc, | |
| 1760 | − struct dpu_encoder_kickoff_params *params, bool async) | |
| 1748 | +void dpu_encoder_prepare_for_kickoff(struct drm_encoder *drm_enc, bool async) | |
| 1761 | 1749 | { |
| 1762 | 1750 | struct dpu_encoder_virt *dpu_enc; |
| 1763 | 1751 | struct dpu_encoder_phys *phys; |
| 1764 | 1752 | bool needs_hw_reset = false; |
| 1765 | 1753 | unsigned int i; |
| 1766 | 1754 | |
| 1767 | − if (!drm_enc || !params) { | |
| 1755 | + if (!drm_enc) { | |
| 1768 | 1756 | DPU_ERROR("invalid args\n"); |
| 1769 | 1757 | return; |
| 1770 | 1758 | } |
| @@ -1778,7 +1766,7 @@ void dpu_encoder_prepare_for_kickoff(struct drm_encoder *drm_enc, · 7 unmodified lines | ||
| 1778 | 1766 | phys = dpu_enc->phys_encs[i]; |
| 1779 | 1767 | if (phys) { |
| 1780 | 1768 | if (phys->ops.prepare_for_kickoff) |
| 1781 | − phys->ops.prepare_for_kickoff(phys, params); | |
| 1769 | + phys->ops.prepare_for_kickoff(phys); | |
| 1782 | 1770 | if (phys->enable_state == DPU_ENC_ERR_NEEDS_HW_RESET) |
| 1783 | 1771 | needs_hw_reset = true; |
| 1784 | 1772 | } |
| @@ -2193,7 +2181,7 @@ int dpu_encoder_setup(struct drm_device *dev, struct drm_encoder *enc, · 408 unmodified lines | ||
| 2193 | 2181 | |
| 2194 | 2182 | |
| 2195 | 2183 | mutex_init(&dpu_enc->rc_lock); |
| 2196 | − kthread_init_delayed_work(&dpu_enc->delayed_off_work, | |
| 2184 | + INIT_DELAYED_WORK(&dpu_enc->delayed_off_work, | |
| 2197 | 2185 | dpu_encoder_off_work); |
| 2198 | 2186 | dpu_enc->idle_timeout = IDLE_TIMEOUT; |
| 2199 | 2187 | |
| @@ -37,15 +37,6 @@ struct dpu_encoder_hw_resources { · 36 unmodified lines | ||
| 37 | 37 | enum dpu_intf_mode intfs[INTF_MAX]; |
| 38 | 38 | }; |
| 39 | 39 | |
| 40 | −/** | |
| 41 | − * dpu_encoder_kickoff_params - info encoder requires at kickoff | |
| 42 | − * @affected_displays: bitmask, bit set means the ROI of the commit lies within | |
| 43 | − * the bounds of the physical display at the bit index | |
| 44 | − */ | |
| 45 | −struct dpu_encoder_kickoff_params { | |
| 46 | − unsigned long affected_displays; | |
| 47 | −}; | |
| 48 | − | |
| 49 | 40 | /** |
| 50 | 41 | * dpu_encoder_get_hw_resources - Populate table of required hardware resources |
| 51 | 42 | * @encoder: encoder pointer |
| @@ -88,11 +79,9 @@ void dpu_encoder_register_frame_event_callback(struct drm_encoder *encoder, · 36 unmodified lines | ||
| 88 | 79 | * Immediately: if no previous commit is outstanding. |
| 89 | 80 | * Delayed: Block until next trigger can be issued. |
| 90 | 81 | * @encoder: encoder pointer |
| 91 | − * @params: kickoff time parameters | |
| 92 | 82 | * @async: true if this is an asynchronous commit |
| 93 | 83 | */ |
| 94 | −void dpu_encoder_prepare_for_kickoff(struct drm_encoder *encoder, | |
| 95 | − struct dpu_encoder_kickoff_params *params, bool async); | |
| 84 | +void dpu_encoder_prepare_for_kickoff(struct drm_encoder *encoder, bool async); | |
| 96 | 85 | |
| 97 | 86 | /** |
| 98 | 87 | * dpu_encoder_trigger_kickoff_pending - Clear the flush bits from previous |
| @@ -144,8 +144,7 @@ struct dpu_encoder_phys_ops { · 143 unmodified lines | ||
| 144 | 144 | int (*wait_for_commit_done)(struct dpu_encoder_phys *phys_enc); |
| 145 | 145 | int (*wait_for_tx_complete)(struct dpu_encoder_phys *phys_enc); |
| 146 | 146 | int (*wait_for_vblank)(struct dpu_encoder_phys *phys_enc); |
| 147 | − void (*prepare_for_kickoff)(struct dpu_encoder_phys *phys_enc, | |
| 148 | − struct dpu_encoder_kickoff_params *params); | |
| 147 | + void (*prepare_for_kickoff)(struct dpu_encoder_phys *phys_enc); | |
| 149 | 148 | void (*handle_post_kickoff)(struct dpu_encoder_phys *phys_enc); |
| 150 | 149 | void (*trigger_start)(struct dpu_encoder_phys *phys_enc); |
| 151 | 150 | bool (*needs_single_flush)(struct dpu_encoder_phys *phys_enc); |
| @@ -594,8 +594,7 @@ static void dpu_encoder_phys_cmd_get_hw_resources( · 593 unmodified lines | ||
| 594 | 594 | } |
| 595 | 595 | |
| 596 | 596 | static void dpu_encoder_phys_cmd_prepare_for_kickoff( |
| 597 | − struct dpu_encoder_phys *phys_enc, | |
| 598 | − struct dpu_encoder_kickoff_params *params) | |
| 597 | + struct dpu_encoder_phys *phys_enc) | |
| 599 | 598 | { |
| 600 | 599 | struct dpu_encoder_phys_cmd *cmd_enc = |
| 601 | 600 | to_dpu_encoder_phys_cmd(phys_enc); |
| @@ -693,7 +692,7 @@ static int dpu_encoder_phys_cmd_wait_for_commit_done( · 91 unmodified lines | ||
| 693 | 692 | |
| 694 | 693 | /* required for both controllers */ |
| 695 | 694 | if (!rc && cmd_enc->serialize_wait4pp) |
| 696 | − dpu_encoder_phys_cmd_prepare_for_kickoff(phys_enc, NULL); | |
| 695 | + dpu_encoder_phys_cmd_prepare_for_kickoff(phys_enc); | |
| 697 | 696 | |
| 698 | 697 | return rc; |
| 699 | 698 | } |
| @@ -587,14 +587,13 @@ static int dpu_encoder_phys_vid_wait_for_vblank( · 586 unmodified lines | ||
| 587 | 587 | } |
| 588 | 588 | |
| 589 | 589 | static void dpu_encoder_phys_vid_prepare_for_kickoff( |
| 590 | − struct dpu_encoder_phys *phys_enc, | |
| 591 | − struct dpu_encoder_kickoff_params *params) | |
| 590 | + struct dpu_encoder_phys *phys_enc) | |
| 592 | 591 | { |
| 593 | 592 | struct dpu_encoder_phys_vid *vid_enc; |
| 594 | 593 | struct dpu_hw_ctl *ctl; |
| 595 | 594 | int rc; |
| 596 | 595 | |
| 597 | − if (!phys_enc || !params) { | |
| 596 | + if (!phys_enc) { | |
| 598 | 597 | DPU_ERROR("invalid encoder/parameters\n"); |
| 599 | 598 | return; |
| 600 | 599 | } |
| @@ -263,13 +263,13 @@ static const struct dpu_format dpu_format_map[] = { · 262 unmodified lines | ||
| 263 | 263 | |
| 264 | 264 | INTERLEAVED_RGB_FMT(RGB565, |
| 265 | 265 | 0, COLOR_5BIT, COLOR_6BIT, COLOR_5BIT, |
| 266 | − C1_B_Cb, C0_G_Y, C2_R_Cr, 0, 3, | |
| 266 | + C2_R_Cr, C0_G_Y, C1_B_Cb, 0, 3, | |
| 267 | 267 | false, 2, 0, |
| 268 | 268 | DPU_FETCH_LINEAR, 1), |
| 269 | 269 | |
| 270 | 270 | INTERLEAVED_RGB_FMT(BGR565, |
| 271 | 271 | 0, COLOR_5BIT, COLOR_6BIT, COLOR_5BIT, |
| 272 | − C2_R_Cr, C0_G_Y, C1_B_Cb, 0, 3, | |
| 272 | + C1_B_Cb, C0_G_Y, C2_R_Cr, 0, 3, | |
| 273 | 273 | false, 2, 0, |
| 274 | 274 | DPU_FETCH_LINEAR, 1), |
| 275 | 275 | |
| @@ -1137,36 +1137,3 @@ const struct msm_format *dpu_get_msm_format( · 861 unmodified lines | ||
| 1137 | 1137 | return &fmt->base; |
| 1138 | 1138 | return NULL; |
| 1139 | 1139 | } |
| 1140 | − | |
| 1141 | −uint32_t dpu_populate_formats( | |
| 1142 | − const struct dpu_format_extended *format_list, | |
| 1143 | − uint32_t *pixel_formats, | |
| 1144 | − uint64_t *pixel_modifiers, | |
| 1145 | − uint32_t pixel_formats_max) | |
| 1146 | −{ | |
| 1147 | − uint32_t i, fourcc_format; | |
| 1148 | − | |
| 1149 | − if (!format_list || !pixel_formats) | |
| 1150 | − return 0; | |
| 1151 | − | |
| 1152 | − for (i = 0, fourcc_format = 0; | |
| 1153 | − format_list->fourcc_format && i < pixel_formats_max; | |
| 1154 | − ++format_list) { | |
| 1155 | − /* verify if listed format is in dpu_format_map? */ | |
| 1156 | − | |
| 1157 | − /* optionally return modified formats */ | |
| 1158 | − if (pixel_modifiers) { | |
| 1159 | − /* assume same modifier for all fb planes */ | |
| 1160 | − pixel_formats[i] = format_list->fourcc_format; | |
| 1161 | − pixel_modifiers[i++] = format_list->modifier; | |
| 1162 | − } else { | |
| 1163 | − /* assume base formats grouped together */ | |
| 1164 | − if (fourcc_format != format_list->fourcc_format) { | |
| 1165 | − fourcc_format = format_list->fourcc_format; | |
| 1166 | − pixel_formats[i++] = fourcc_format; | |
| 1167 | − } | |
| 1168 | − } | |
| 1169 | − } | |
| 1170 | − | |
| 1171 | − return i; | |
| 1172 | −} | |
| @@ -40,20 +40,6 @@ const struct msm_format *dpu_get_msm_format( · 39 unmodified lines | ||
| 40 | 40 | const uint32_t format, |
| 41 | 41 | const uint64_t modifiers); |
| 42 | 42 | |
| 43 | −/** | |
| 44 | − * dpu_populate_formats - populate the given array with fourcc codes supported | |
| 45 | − * @format_list: pointer to list of possible formats | |
| 46 | − * @pixel_formats: array to populate with fourcc codes | |
| 47 | − * @pixel_modifiers: array to populate with drm modifiers, can be NULL | |
| 48 | − * @pixel_formats_max: length of pixel formats array | |
| 49 | − * Return: number of elements populated | |
| 50 | − */ | |
| 51 | −uint32_t dpu_populate_formats( | |
| 52 | − const struct dpu_format_extended *format_list, | |
| 53 | − uint32_t *pixel_formats, | |
| 54 | − uint64_t *pixel_modifiers, | |
| 55 | − uint32_t pixel_formats_max); | |
| 56 | − | |
| 57 | 43 | /** |
| 58 | 44 | * dpu_format_check_modified_format - validate format and buffers for |
| 59 | 45 | * dpu non-standard, i.e. modified format |
| @@ -151,7 +151,9 @@ static const struct dpu_sspp_blks_common sdm845_sspp_common = { · 150 unmodified lines | ||
| 151 | 151 | .id = DPU_SSPP_CSC_10BIT, \ |
| 152 | 152 | .base = 0x1a00, .len = 0x100,}, \ |
| 153 | 153 | .format_list = plane_formats_yuv, \ |
| 154 | + .num_formats = ARRAY_SIZE(plane_formats_yuv), \ | |
| 154 | 155 | .virt_format_list = plane_formats, \ |
| 156 | + .virt_num_formats = ARRAY_SIZE(plane_formats), \ | |
| 155 | 157 | } |
| 156 | 158 | |
| 157 | 159 | #define _DMA_SBLK(num, sdma_pri) \ |
| @@ -163,7 +165,9 @@ static const struct dpu_sspp_blks_common sdm845_sspp_common = { · 5 unmodified lines | ||
| 163 | 165 | .src_blk = {.name = STRCAT("sspp_src_", num), \ |
| 164 | 166 | .id = DPU_SSPP_SRC, .base = 0x00, .len = 0x150,}, \ |
| 165 | 167 | .format_list = plane_formats, \ |
| 168 | + .num_formats = ARRAY_SIZE(plane_formats), \ | |
| 166 | 169 | .virt_format_list = plane_formats, \ |
| 170 | + .virt_num_formats = ARRAY_SIZE(plane_formats), \ | |
| 167 | 171 | } |
| 168 | 172 | |
| 169 | 173 | static const struct dpu_sspp_sub_blks sdm845_vig_sblk_0 = _VIG_SBLK("0", 5); |
| @@ -251,17 +251,6 @@ struct dpu_pp_blk { · 250 unmodified lines | ||
| 251 | 251 | u32 version; |
| 252 | 252 | }; |
| 253 | 253 | |
| 254 | −/** | |
| 255 | − * struct dpu_format_extended - define dpu specific pixel format+modifier | |
| 256 | − * @fourcc_format: Base FOURCC pixel format code | |
| 257 | − * @modifier: 64-bit drm format modifier, same modifier must be applied to all | |
| 258 | − * framebuffer planes | |
| 259 | − */ | |
| 260 | −struct dpu_format_extended { | |
| 261 | − uint32_t fourcc_format; | |
| 262 | − uint64_t modifier; | |
| 263 | −}; | |
| 264 | − | |
| 265 | 254 | /** |
| 266 | 255 | * enum dpu_qos_lut_usage - define QoS LUT use cases |
| 267 | 256 | */ |
| @@ -348,7 +337,9 @@ struct dpu_sspp_blks_common { · 80 unmodified lines | ||
| 348 | 337 | * @pcc_blk: |
| 349 | 338 | * @igc_blk: |
| 350 | 339 | * @format_list: Pointer to list of supported formats |
| 340 | + * @num_formats: Number of supported formats | |
| 351 | 341 | * @virt_format_list: Pointer to list of supported formats for virtual planes |
| 342 | + * @virt_num_formats: Number of supported formats for virtual planes | |
| 352 | 343 | */ |
| 353 | 344 | struct dpu_sspp_sub_blks { |
| 354 | 345 | const struct dpu_sspp_blks_common *common; |
| @@ -366,8 +357,10 @@ struct dpu_sspp_sub_blks { · 11 unmodified lines | ||
| 366 | 357 | struct dpu_pp_blk pcc_blk; |
| 367 | 358 | struct dpu_pp_blk igc_blk; |
| 368 | 359 | |
| 369 | − const struct dpu_format_extended *format_list; | |
| 370 | − const struct dpu_format_extended *virt_format_list; | |
| 360 | + const u32 *format_list; | |
| 361 | + u32 num_formats; | |
| 362 | + const u32 *virt_format_list; | |
| 363 | + u32 virt_num_formats; | |
| 371 | 364 | }; |
| 372 | 365 | |
| 373 | 366 | /** |
| @@ -12,157 +12,81 @@ · 11 unmodified lines | ||
| 12 | 12 | |
| 13 | 13 | #include "dpu_hw_mdss.h" |
| 14 | 14 | |
| 15 | −static const struct dpu_format_extended plane_formats[] = { | |
| 16 | − {DRM_FORMAT_ARGB8888, 0}, | |
| 17 | − {DRM_FORMAT_ABGR8888, 0}, | |
| 18 | − {DRM_FORMAT_RGBA8888, 0}, | |
| 19 | − {DRM_FORMAT_ABGR8888, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | |
| 20 | − {DRM_FORMAT_BGRA8888, 0}, | |
| 21 | − {DRM_FORMAT_XRGB8888, 0}, | |
| 22 | − {DRM_FORMAT_RGBX8888, 0}, | |
| 23 | − {DRM_FORMAT_BGRX8888, 0}, | |
| 24 | − {DRM_FORMAT_XBGR8888, 0}, | |
| 25 | − {DRM_FORMAT_XBGR8888, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | |
| 26 | − {DRM_FORMAT_RGB888, 0}, | |
| 27 | − {DRM_FORMAT_BGR888, 0}, | |
| 28 | − {DRM_FORMAT_RGB565, 0}, | |
| 29 | − {DRM_FORMAT_BGR565, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | |
| 30 | − {DRM_FORMAT_BGR565, 0}, | |
| 31 | − {DRM_FORMAT_ARGB1555, 0}, | |
| 32 | − {DRM_FORMAT_ABGR1555, 0}, | |
| 33 | − {DRM_FORMAT_RGBA5551, 0}, | |
| 34 | − {DRM_FORMAT_BGRA5551, 0}, | |
| 35 | − {DRM_FORMAT_XRGB1555, 0}, | |
| 36 | − {DRM_FORMAT_XBGR1555, 0}, | |
| 37 | − {DRM_FORMAT_RGBX5551, 0}, | |
| 38 | − {DRM_FORMAT_BGRX5551, 0}, | |
| 39 | − {DRM_FORMAT_ARGB4444, 0}, | |
| 40 | − {DRM_FORMAT_ABGR4444, 0}, | |
| 41 | − {DRM_FORMAT_RGBA4444, 0}, | |
| 42 | − {DRM_FORMAT_BGRA4444, 0}, | |
| 43 | − {DRM_FORMAT_XRGB4444, 0}, | |
| 44 | − {DRM_FORMAT_XBGR4444, 0}, | |
| 45 | − {DRM_FORMAT_RGBX4444, 0}, | |
| 46 | − {DRM_FORMAT_BGRX4444, 0}, | |
| 47 | − {0, 0}, | |
| 15 | +static const uint32_t qcom_compressed_supported_formats[] = { | |
| 16 | + DRM_FORMAT_ABGR8888, | |
| 17 | + DRM_FORMAT_XBGR8888, | |
| 18 | + DRM_FORMAT_BGR565, | |
| 48 | 19 | }; |
| 49 | 20 | |
| 50 | −static const struct dpu_format_extended plane_formats_yuv[] = { | |
| 51 | − {DRM_FORMAT_ARGB8888, 0}, | |
| 52 | − {DRM_FORMAT_ABGR8888, 0}, | |
| 53 | − {DRM_FORMAT_RGBA8888, 0}, | |
| 54 | − {DRM_FORMAT_BGRX8888, 0}, | |
| 55 | − {DRM_FORMAT_ABGR8888, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | |
| 56 | − {DRM_FORMAT_BGRA8888, 0}, | |
| 57 | − {DRM_FORMAT_XRGB8888, 0}, | |
| 58 | − {DRM_FORMAT_XBGR8888, 0}, | |
| 59 | − {DRM_FORMAT_RGBX8888, 0}, | |
| 60 | − {DRM_FORMAT_XBGR8888, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | |
| 61 | − {DRM_FORMAT_RGB888, 0}, | |
| 62 | − {DRM_FORMAT_BGR888, 0}, | |
| 63 | − {DRM_FORMAT_RGB565, 0}, | |
| 64 | − {DRM_FORMAT_BGR565, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | |
| 65 | − {DRM_FORMAT_BGR565, 0}, | |
| 66 | − {DRM_FORMAT_ARGB1555, 0}, | |
| 67 | − {DRM_FORMAT_ABGR1555, 0}, | |
| 68 | − {DRM_FORMAT_RGBA5551, 0}, | |
| 69 | − {DRM_FORMAT_BGRA5551, 0}, | |
| 70 | − {DRM_FORMAT_XRGB1555, 0}, | |
| 71 | − {DRM_FORMAT_XBGR1555, 0}, | |
| 72 | − {DRM_FORMAT_RGBX5551, 0}, | |
| 73 | − {DRM_FORMAT_BGRX5551, 0}, | |
| 74 | − {DRM_FORMAT_ARGB4444, 0}, | |
| 75 | − {DRM_FORMAT_ABGR4444, 0}, | |
| 76 | − {DRM_FORMAT_RGBA4444, 0}, | |
| 77 | − {DRM_FORMAT_BGRA4444, 0}, | |
| 78 | − {DRM_FORMAT_XRGB4444, 0}, | |
| 79 | − {DRM_FORMAT_XBGR4444, 0}, | |
| 80 | − {DRM_FORMAT_RGBX4444, 0}, | |
| 81 | − {DRM_FORMAT_BGRX4444, 0}, | |
| 82 | − | |
| 83 | − {DRM_FORMAT_NV12, 0}, | |
| 84 | − {DRM_FORMAT_NV12, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | |
| 85 | − {DRM_FORMAT_NV21, 0}, | |
| 86 | − {DRM_FORMAT_NV16, 0}, | |
| 87 | − {DRM_FORMAT_NV61, 0}, | |
| 88 | − {DRM_FORMAT_VYUY, 0}, | |
| 89 | − {DRM_FORMAT_UYVY, 0}, | |
| 90 | − {DRM_FORMAT_YUYV, 0}, | |
| 91 | − {DRM_FORMAT_YVYU, 0}, | |
| 92 | − {DRM_FORMAT_YUV420, 0}, | |
| 93 | − {DRM_FORMAT_YVU420, 0}, | |
| 94 | − {0, 0}, | |
| 95 | −}; | |
| 96 | − | |
| 97 | −static const struct dpu_format_extended cursor_formats[] = { | |
| 98 | − {DRM_FORMAT_ARGB8888, 0}, | |
| 99 | − {DRM_FORMAT_ABGR8888, 0}, | |
| 100 | − {DRM_FORMAT_RGBA8888, 0}, | |
| 101 | − {DRM_FORMAT_BGRA8888, 0}, | |
| 102 | − {DRM_FORMAT_XRGB8888, 0}, | |
| 103 | − {DRM_FORMAT_ARGB1555, 0}, | |
| 104 | − {DRM_FORMAT_ABGR1555, 0}, | |
| 105 | − {DRM_FORMAT_RGBA5551, 0}, | |
| 106 | − {DRM_FORMAT_BGRA5551, 0}, | |
| 107 | − {DRM_FORMAT_ARGB4444, 0}, | |
| 108 | − {DRM_FORMAT_ABGR4444, 0}, | |
| 109 | − {DRM_FORMAT_RGBA4444, 0}, | |
| 110 | − {DRM_FORMAT_BGRA4444, 0}, | |
| 111 | − {0, 0}, | |
| 21 | +static const uint32_t plane_formats[] = { | |
| 22 | + DRM_FORMAT_ARGB8888, | |
| 23 | + DRM_FORMAT_ABGR8888, | |
| 24 | + DRM_FORMAT_RGBA8888, | |
| 25 | + DRM_FORMAT_BGRA8888, | |
| 26 | + DRM_FORMAT_XRGB8888, | |
| 27 | + DRM_FORMAT_RGBX8888, | |
| 28 | + DRM_FORMAT_BGRX8888, | |
| 29 | + DRM_FORMAT_XBGR8888, | |
| 30 | + DRM_FORMAT_RGB888, | |
| 31 | + DRM_FORMAT_BGR888, | |
| 32 | + DRM_FORMAT_RGB565, | |
| 33 | + DRM_FORMAT_BGR565, | |
| 34 | + DRM_FORMAT_ARGB1555, | |
| 35 | + DRM_FORMAT_ABGR1555, | |
| 36 | + DRM_FORMAT_RGBA5551, | |
| 37 | + DRM_FORMAT_BGRA5551, | |
| 38 | + DRM_FORMAT_XRGB1555, | |
| 39 | + DRM_FORMAT_XBGR1555, | |
| 40 | + DRM_FORMAT_RGBX5551, | |
| 41 | + DRM_FORMAT_BGRX5551, | |
| 42 | + DRM_FORMAT_ARGB4444, | |
| 43 | + DRM_FORMAT_ABGR4444, | |
| 44 | + DRM_FORMAT_RGBA4444, | |
| 45 | + DRM_FORMAT_BGRA4444, | |
| 46 | + DRM_FORMAT_XRGB4444, | |
| 47 | + DRM_FORMAT_XBGR4444, | |
| 48 | + DRM_FORMAT_RGBX4444, | |
| 49 | + DRM_FORMAT_BGRX4444, | |
| 112 | 50 | }; |
| 113 | 51 | |
| 114 | −static const struct dpu_format_extended wb2_formats[] = { | |
| 115 | − {DRM_FORMAT_RGB565, 0}, | |
| 116 | − {DRM_FORMAT_BGR565, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | |
| 117 | − {DRM_FORMAT_RGB888, 0}, | |
| 118 | − {DRM_FORMAT_ARGB8888, 0}, | |
| 119 | − {DRM_FORMAT_RGBA8888, 0}, | |
| 120 | − {DRM_FORMAT_ABGR8888, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | |
| 121 | − {DRM_FORMAT_XRGB8888, 0}, | |
| 122 | − {DRM_FORMAT_RGBX8888, 0}, | |
| 123 | − {DRM_FORMAT_XBGR8888, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | |
| 124 | − {DRM_FORMAT_ARGB1555, 0}, | |
| 125 | − {DRM_FORMAT_RGBA5551, 0}, | |
| 126 | − {DRM_FORMAT_XRGB1555, 0}, | |
| 127 | − {DRM_FORMAT_RGBX5551, 0}, | |
| 128 | − {DRM_FORMAT_ARGB4444, 0}, | |
| 129 | − {DRM_FORMAT_RGBA4444, 0}, | |
| 130 | − {DRM_FORMAT_RGBX4444, 0}, | |
| 131 | − {DRM_FORMAT_XRGB4444, 0}, | |
| 132 | − | |
| 133 | − {DRM_FORMAT_BGR565, 0}, | |
| 134 | − {DRM_FORMAT_BGR888, 0}, | |
| 135 | − {DRM_FORMAT_ABGR8888, 0}, | |
| 136 | − {DRM_FORMAT_BGRA8888, 0}, | |
| 137 | − {DRM_FORMAT_BGRX8888, 0}, | |
| 138 | − {DRM_FORMAT_XBGR8888, 0}, | |
| 139 | − {DRM_FORMAT_ABGR1555, 0}, | |
| 140 | − {DRM_FORMAT_BGRA5551, 0}, | |
| 141 | − {DRM_FORMAT_XBGR1555, 0}, | |
| 142 | − {DRM_FORMAT_BGRX5551, 0}, | |
| 143 | − {DRM_FORMAT_ABGR4444, 0}, | |
| 144 | − {DRM_FORMAT_BGRA4444, 0}, | |
| 145 | − {DRM_FORMAT_BGRX4444, 0}, | |
| 146 | − {DRM_FORMAT_XBGR4444, 0}, | |
| 147 | − | |
| 148 | − {DRM_FORMAT_YUV420, 0}, | |
| 149 | − {DRM_FORMAT_NV12, 0}, | |
| 150 | − {DRM_FORMAT_NV12, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | |
| 151 | − {DRM_FORMAT_NV16, 0}, | |
| 152 | − {DRM_FORMAT_YUYV, 0}, | |
| 153 | − | |
| 154 | − {0, 0}, | |
| 155 | −}; | |
| 52 | +static const uint32_t plane_formats_yuv[] = { | |
| 53 | + DRM_FORMAT_ARGB8888, | |
| 54 | + DRM_FORMAT_ABGR8888, | |
| 55 | + DRM_FORMAT_RGBA8888, | |
| 56 | + DRM_FORMAT_BGRX8888, | |
| 57 | + DRM_FORMAT_BGRA8888, | |
| 58 | + DRM_FORMAT_XRGB8888, | |
| 59 | + DRM_FORMAT_XBGR8888, | |
| 60 | + DRM_FORMAT_RGBX8888, | |
| 61 | + DRM_FORMAT_RGB888, | |
| 62 | + DRM_FORMAT_BGR888, | |
| 63 | + DRM_FORMAT_RGB565, | |
| 64 | + DRM_FORMAT_BGR565, | |
| 65 | + DRM_FORMAT_ARGB1555, | |
| 66 | + DRM_FORMAT_ABGR1555, | |
| 67 | + DRM_FORMAT_RGBA5551, | |
| 68 | + DRM_FORMAT_BGRA5551, | |
| 69 | + DRM_FORMAT_XRGB1555, | |
| 70 | + DRM_FORMAT_XBGR1555, | |
| 71 | + DRM_FORMAT_RGBX5551, | |
| 72 | + DRM_FORMAT_BGRX5551, | |
| 73 | + DRM_FORMAT_ARGB4444, | |
| 74 | + DRM_FORMAT_ABGR4444, | |
| 75 | + DRM_FORMAT_RGBA4444, | |
| 76 | + DRM_FORMAT_BGRA4444, | |
| 77 | + DRM_FORMAT_XRGB4444, | |
| 78 | + DRM_FORMAT_XBGR4444, | |
| 79 | + DRM_FORMAT_RGBX4444, | |
| 80 | + DRM_FORMAT_BGRX4444, | |
| 156 | 81 | |
| 157 | −static const struct dpu_format_extended rgb_10bit_formats[] = { | |
| 158 | − {DRM_FORMAT_BGRA1010102, 0}, | |
| 159 | − {DRM_FORMAT_BGRX1010102, 0}, | |
| 160 | − {DRM_FORMAT_RGBA1010102, 0}, | |
| 161 | − {DRM_FORMAT_RGBX1010102, 0}, | |
| 162 | − {DRM_FORMAT_ABGR2101010, 0}, | |
| 163 | − {DRM_FORMAT_ABGR2101010, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | |
| 164 | − {DRM_FORMAT_XBGR2101010, 0}, | |
| 165 | − {DRM_FORMAT_XBGR2101010, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | |
| 166 | − {DRM_FORMAT_ARGB2101010, 0}, | |
| 167 | − {DRM_FORMAT_XRGB2101010, 0}, | |
| 82 | + DRM_FORMAT_NV12, | |
| 83 | + DRM_FORMAT_NV21, | |
| 84 | + DRM_FORMAT_NV16, | |
| 85 | + DRM_FORMAT_NV61, | |
| 86 | + DRM_FORMAT_VYUY, | |
| 87 | + DRM_FORMAT_UYVY, | |
| 88 | + DRM_FORMAT_YUYV, | |
| 89 | + DRM_FORMAT_YVYU, | |
| 90 | + DRM_FORMAT_YUV420, | |
| 91 | + DRM_FORMAT_YVU420, | |
| 168 | 92 | }; |
| @@ -170,10 +170,6 @@ · 169 unmodified lines | ||
| 170 | 170 | /** |
| 171 | 171 | * AD4 interrupt status bit definitions |
| 172 | 172 | */ |
| 173 | −#define DPU_INTR_BRIGHTPR_UPDATED BIT(4) | |
| 174 | −#define DPU_INTR_DARKENH_UPDATED BIT(3) | |
| 175 | −#define DPU_INTR_STREN_OUTROI_UPDATED BIT(2) | |
| 176 | −#define DPU_INTR_STREN_INROI_UPDATED BIT(1) | |
| 177 | 173 | #define DPU_INTR_BACKLIGHT_UPDATED BIT(0) |
| 178 | 174 | /** |
| 179 | 175 | * struct dpu_intr_reg - array of DPU register sets |
| @@ -782,18 +778,6 @@ static int dpu_hw_intr_irqidx_lookup(enum dpu_intr_type intr_type, · 602 unmodified lines | ||
| 782 | 778 | return -EINVAL; |
| 783 | 779 | } |
| 784 | 780 | |
| 785 | −static void dpu_hw_intr_set_mask(struct dpu_hw_intr *intr, uint32_t reg_off, | |
| 786 | − uint32_t mask) | |
| 787 | −{ | |
| 788 | − if (!intr) | |
| 789 | − return; | |
| 790 | − | |
| 791 | − DPU_REG_WRITE(&intr->hw, reg_off, mask); | |
| 792 | − | |
| 793 | − /* ensure register writes go through */ | |
| 794 | − wmb(); | |
| 795 | −} | |
| 796 | − | |
| 797 | 781 | static void dpu_hw_intr_dispatch_irq(struct dpu_hw_intr *intr, |
| 798 | 782 | void (*cbfunc)(void *, int), |
| 799 | 783 | void *arg) |
| @@ -1004,18 +988,6 @@ static int dpu_hw_intr_disable_irqs(struct dpu_hw_intr *intr) · 204 unmodified lines | ||
| 1004 | 988 | return 0; |
| 1005 | 989 | } |
| 1006 | 990 | |
| 1007 | −static int dpu_hw_intr_get_valid_interrupts(struct dpu_hw_intr *intr, | |
| 1008 | − uint32_t *mask) | |
| 1009 | −{ | |
| 1010 | − if (!intr || !mask) | |
| 1011 | − return -EINVAL; | |
| 1012 | − | |
| 1013 | − *mask = IRQ_SOURCE_MDP | IRQ_SOURCE_DSI0 | IRQ_SOURCE_DSI1 | |
| 1014 | − | IRQ_SOURCE_HDMI | IRQ_SOURCE_EDP; | |
| 1015 | − | |
| 1016 | − return 0; | |
| 1017 | −} | |
| 1018 | − | |
| 1019 | 991 | static void dpu_hw_intr_get_interrupt_statuses(struct dpu_hw_intr *intr) |
| 1020 | 992 | { |
| 1021 | 993 | int i; |
| @@ -1065,19 +1037,6 @@ static void dpu_hw_intr_clear_intr_status_nolock(struct dpu_hw_intr *intr, · 43 unmodified lines | ||
| 1065 | 1037 | wmb(); |
| 1066 | 1038 | } |
| 1067 | 1039 | |
| 1068 | −static void dpu_hw_intr_clear_interrupt_status(struct dpu_hw_intr *intr, | |
| 1069 | − int irq_idx) | |
| 1070 | −{ | |
| 1071 | − unsigned long irq_flags; | |
| 1072 | − | |
| 1073 | − if (!intr) | |
| 1074 | − return; | |
| 1075 | − | |
| 1076 | − spin_lock_irqsave(&intr->irq_lock, irq_flags); | |
| 1077 | − dpu_hw_intr_clear_intr_status_nolock(intr, irq_idx); | |
| 1078 | − spin_unlock_irqrestore(&intr->irq_lock, irq_flags); | |
| 1079 | −} | |
| 1080 | − | |
| 1081 | 1040 | static u32 dpu_hw_intr_get_interrupt_status(struct dpu_hw_intr *intr, |
| 1082 | 1041 | int irq_idx, bool clear) |
| 1083 | 1042 | { |
| @@ -1113,16 +1072,13 @@ static u32 dpu_hw_intr_get_interrupt_status(struct dpu_hw_intr *intr, · 29 unmodified lines | ||
| 1113 | 1072 | |
| 1114 | 1073 | static void __setup_intr_ops(struct dpu_hw_intr_ops *ops) |
| 1115 | 1074 | { |
| 1116 | − ops->set_mask = dpu_hw_intr_set_mask; | |
| 1117 | 1075 | ops->irq_idx_lookup = dpu_hw_intr_irqidx_lookup; |
| 1118 | 1076 | ops->enable_irq = dpu_hw_intr_enable_irq; |
| 1119 | 1077 | ops->disable_irq = dpu_hw_intr_disable_irq; |
| 1120 | 1078 | ops->dispatch_irqs = dpu_hw_intr_dispatch_irq; |
| 1121 | 1079 | ops->clear_all_irqs = dpu_hw_intr_clear_irqs; |
| 1122 | 1080 | ops->disable_all_irqs = dpu_hw_intr_disable_irqs; |
| 1123 | − ops->get_valid_interrupts = dpu_hw_intr_get_valid_interrupts; | |
| 1124 | 1081 | ops->get_interrupt_statuses = dpu_hw_intr_get_interrupt_statuses; |
| 1125 | − ops->clear_interrupt_status = dpu_hw_intr_clear_interrupt_status; | |
| 1126 | 1082 | ops->clear_intr_status_nolock = dpu_hw_intr_clear_intr_status_nolock; |
| 1127 | 1083 | ops->get_interrupt_status = dpu_hw_intr_get_interrupt_status; |
| 1128 | 1084 | } |
| @@ -20,13 +20,6 @@ · 19 unmodified lines | ||
| 20 | 20 | #include "dpu_hw_util.h" |
| 21 | 21 | #include "dpu_hw_mdss.h" |
| 22 | 22 | |
| 23 | −#define IRQ_SOURCE_MDP BIT(0) | |
| 24 | −#define IRQ_SOURCE_DSI0 BIT(4) | |
| 25 | −#define IRQ_SOURCE_DSI1 BIT(5) | |
| 26 | −#define IRQ_SOURCE_HDMI BIT(8) | |
| 27 | −#define IRQ_SOURCE_EDP BIT(12) | |
| 28 | −#define IRQ_SOURCE_MHL BIT(16) | |
| 29 | − | |
| 30 | 23 | /** |
| 31 | 24 | * dpu_intr_type - HW Interrupt Type |
| 32 | 25 | * @DPU_IRQ_TYPE_WB_ROT_COMP: WB rotator done |
| @@ -95,18 +88,6 @@ struct dpu_hw_intr; · 62 unmodified lines | ||
| 95 | 88 | * Interrupt operations. |
| 96 | 89 | */ |
| 97 | 90 | struct dpu_hw_intr_ops { |
| 98 | − /** | |
| 99 | − * set_mask - Programs the given interrupt register with the | |
| 100 | − * given interrupt mask. Register value will get overwritten. | |
| 101 | − * @intr: HW interrupt handle | |
| 102 | − * @reg_off: MDSS HW register offset | |
| 103 | − * @irqmask: IRQ mask value | |
| 104 | − */ | |
| 105 | − void (*set_mask)( | |
| 106 | − struct dpu_hw_intr *intr, | |
| 107 | − uint32_t reg, | |
| 108 | − uint32_t irqmask); | |
| 109 | − | |
| 110 | 91 | /** |
| 111 | 92 | * irq_idx_lookup - Lookup IRQ index on the HW interrupt type |
| 112 | 93 | * Used for all irq related ops |
| @@ -176,16 +157,6 @@ struct dpu_hw_intr_ops { · 63 unmodified lines | ||
| 176 | 157 | void (*get_interrupt_statuses)( |
| 177 | 158 | struct dpu_hw_intr *intr); |
| 178 | 159 | |
| 179 | − /** | |
| 180 | − * clear_interrupt_status - Clears HW interrupt status based on given | |
| 181 | − * lookup IRQ index. | |
| 182 | − * @intr: HW interrupt handle | |
| 183 | − * @irq_idx: Lookup irq index return from irq_idx_lookup | |
| 184 | − */ | |
| 185 | − void (*clear_interrupt_status)( | |
| 186 | − struct dpu_hw_intr *intr, | |
| 187 | − int irq_idx); | |
| 188 | − | |
| 189 | 160 | /** |
| 190 | 161 | * clear_intr_status_nolock() - clears the HW interrupts without lock |
| 191 | 162 | * @intr: HW interrupt handle |
| @@ -206,21 +177,6 @@ struct dpu_hw_intr_ops { · 14 unmodified lines | ||
| 206 | 177 | struct dpu_hw_intr *intr, |
| 207 | 178 | int irq_idx, |
| 208 | 179 | bool clear); |
| 209 | − | |
| 210 | − /** | |
| 211 | − * get_valid_interrupts - Gets a mask of all valid interrupt sources | |
| 212 | − * within DPU. These are actually status bits | |
| 213 | − * within interrupt registers that specify the | |
| 214 | − * source of the interrupt in IRQs. For example, | |
| 215 | − * valid interrupt sources can be MDP, DSI, | |
| 216 | − * HDMI etc. | |
| 217 | − * @intr: HW interrupt handle | |
| 218 | − * @mask: Returning the interrupt source MASK | |
| 219 | − * @return: 0 for success, otherwise failure | |
| 220 | − */ | |
| 221 | − int (*get_valid_interrupts)( | |
| 222 | − struct dpu_hw_intr *intr, | |
| 223 | − uint32_t *mask); | |
| 224 | 180 | }; |
| 225 | 181 | |
| 226 | 182 | /** |
| @@ -258,12 +258,6 @@ enum dpu_vbif { · 257 unmodified lines | ||
| 258 | 258 | VBIF_NRT = VBIF_1 |
| 259 | 259 | }; |
| 260 | 260 | |
| 261 | −enum dpu_iommu_domain { | |
| 262 | − DPU_IOMMU_DOMAIN_UNSECURE, | |
| 263 | − DPU_IOMMU_DOMAIN_SECURE, | |
| 264 | − DPU_IOMMU_DOMAIN_MAX | |
| 265 | −}; | |
| 266 | − | |
| 267 | 261 | /** |
| 268 | 262 | * DPU HW,Component order color map |
| 269 | 263 | */ |
| @@ -358,7 +352,6 @@ enum dpu_3d_blend_mode { · 88 unmodified lines | ||
| 358 | 352 | * @alpha_enable: whether the format has an alpha channel |
| 359 | 353 | * @num_planes: number of planes (including meta data planes) |
| 360 | 354 | * @fetch_mode: linear, tiled, or ubwc hw fetch behavior |
| 361 | − * @is_yuv: is format a yuv variant | |
| 362 | 355 | * @flag: usage bit flags |
| 363 | 356 | * @tile_width: format tile width |
| 364 | 357 | * @tile_height: format tile height |
| @@ -18,7 +18,6 @@ · 17 unmodified lines | ||
| 18 | 18 | #include "dpu_hw_mdss.h" |
| 19 | 19 | |
| 20 | 20 | #define REG_MASK(n) ((BIT(n)) - 1) |
| 21 | −struct dpu_format_extended; | |
| 22 | 21 | |
| 23 | 22 | /* |
| 24 | 23 | * This is the common struct maintained by each sub block |
| @@ -405,35 +405,38 @@ static void dpu_kms_wait_for_commit_done(struct msm_kms *kms, · 404 unmodified lines | ||
| 405 | 405 | } |
| 406 | 406 | } |
| 407 | 407 | |
| 408 | −static void _dpu_kms_initialize_dsi(struct drm_device *dev, | |
| 408 | +static int _dpu_kms_initialize_dsi(struct drm_device *dev, | |
| 409 | 409 | struct msm_drm_private *priv, |
| 410 | 410 | struct dpu_kms *dpu_kms) |
| 411 | 411 | { |
| 412 | 412 | struct drm_encoder *encoder = NULL; |
| 413 | − int i, rc; | |
| 413 | + int i, rc = 0; | |
| 414 | + | |
| 415 | + if (!(priv->dsi[0] || priv->dsi[1])) | |
| 416 | + return rc; | |
| 414 | 417 | |
| 415 | 418 | /*TODO: Support two independent DSI connectors */ |
| 416 | 419 | encoder = dpu_encoder_init(dev, DRM_MODE_ENCODER_DSI); |
| 417 | − if (IS_ERR_OR_NULL(encoder)) { | |
| 420 | + if (IS_ERR(encoder)) { | |
| 418 | 421 | DPU_ERROR("encoder init failed for dsi display\n"); |
| 419 | − return; | |
| 422 | + return PTR_ERR(encoder); | |
| 420 | 423 | } |
| 421 | 424 | |
| 422 | 425 | priv->encoders[priv->num_encoders++] = encoder; |
| 423 | 426 | |
| 424 | 427 | for (i = 0; i < ARRAY_SIZE(priv->dsi); i++) { |
| 425 | − if (!priv->dsi[i]) { | |
| 426 | − DPU_DEBUG("invalid msm_dsi for ctrl %d\n", i); | |
| 427 | − return; | |
| 428 | − } | |
| 428 | + if (!priv->dsi[i]) | |
| 429 | + continue; | |
| 429 | 430 | |
| 430 | 431 | rc = msm_dsi_modeset_init(priv->dsi[i], dev, encoder); |
| 431 | 432 | if (rc) { |
| 432 | 433 | DPU_ERROR("modeset_init failed for dsi[%d], rc = %d\n", |
| 433 | 434 | i, rc); |
| 434 | − continue; | |
| 435 | + break; | |
| 435 | 436 | } |
| 436 | 437 | } |
| 438 | + | |
| 439 | + return rc; | |
| 437 | 440 | } |
| 438 | 441 | |
| 439 | 442 | /** |
| @@ -444,16 +447,16 @@ static void _dpu_kms_initialize_dsi(struct drm_device *dev, · 4 unmodified lines | ||
| 444 | 447 | * @dpu_kms: Pointer to dpu kms structure |
| 445 | 448 | * Returns: Zero on success |
| 446 | 449 | */ |
| 447 | −static void _dpu_kms_setup_displays(struct drm_device *dev, | |
| 450 | +static int _dpu_kms_setup_displays(struct drm_device *dev, | |
| 448 | 451 | struct msm_drm_private *priv, |
| 449 | 452 | struct dpu_kms *dpu_kms) |
| 450 | 453 | { |
| 451 | − _dpu_kms_initialize_dsi(dev, priv, dpu_kms); | |
| 452 | − | |
| 453 | 454 | /** |
| 454 | 455 | * Extend this function to initialize other |
| 455 | 456 | * types of displays |
| 456 | 457 | */ |
| 458 | + | |
| 459 | + return _dpu_kms_initialize_dsi(dev, priv, dpu_kms); | |
| 457 | 460 | } |
| 458 | 461 | |
| 459 | 462 | static void _dpu_kms_drm_obj_destroy(struct dpu_kms *dpu_kms) |
| @@ -516,7 +519,9 @@ static int _dpu_kms_drm_obj_init(struct dpu_kms *dpu_kms) · 56 unmodified lines | ||
| 516 | 519 | * Create encoder and query display drivers to create |
| 517 | 520 | * bridges and connectors |
| 518 | 521 | */ |
| 519 | − _dpu_kms_setup_displays(dev, priv, dpu_kms); | |
| 522 | + ret = _dpu_kms_setup_displays(dev, priv, dpu_kms); | |
| 523 | + if (ret) | |
| 524 | + goto fail; | |
| 520 | 525 | |
| 521 | 526 | max_crtc_count = min(catalog->mixer_count, priv->num_encoders); |
| 522 | 527 | |
| @@ -627,6 +632,10 @@ static void _dpu_kms_hw_destroy(struct dpu_kms *dpu_kms) · 104 unmodified lines | ||
| 627 | 632 | devm_iounmap(&dpu_kms->pdev->dev, dpu_kms->vbif[VBIF_RT]); |
| 628 | 633 | dpu_kms->vbif[VBIF_RT] = NULL; |
| 629 | 634 | |
| 635 | + if (dpu_kms->hw_mdp) | |
| 636 | + dpu_hw_mdp_destroy(dpu_kms->hw_mdp); | |
| 637 | + dpu_kms->hw_mdp = NULL; | |
| 638 | + | |
| 630 | 639 | if (dpu_kms->mmio) |
| 631 | 640 | devm_iounmap(&dpu_kms->pdev->dev, dpu_kms->mmio); |
| 632 | 641 | dpu_kms->mmio = NULL; |
| @@ -877,8 +886,7 @@ static int dpu_kms_hw_init(struct msm_kms *kms) · 244 unmodified lines | ||
| 877 | 886 | goto power_error; |
| 878 | 887 | } |
| 879 | 888 | |
| 880 | − rc = dpu_rm_init(&dpu_kms->rm, dpu_kms->catalog, dpu_kms->mmio, | |
| 881 | − dpu_kms->dev); | |
| 889 | + rc = dpu_rm_init(&dpu_kms->rm, dpu_kms->catalog, dpu_kms->mmio); | |
| 882 | 890 | if (rc) { |
| 883 | 891 | DPU_ERROR("rm init failed: %d\n", rc); |
| 884 | 892 | goto power_error; |
| @@ -886,11 +894,10 @@ static int dpu_kms_hw_init(struct msm_kms *kms) · 1 unmodified line | ||
| 886 | 894 | |
| 887 | 895 | dpu_kms->rm_init = true; |
| 888 | 896 | |
| 889 | − dpu_kms->hw_mdp = dpu_rm_get_mdp(&dpu_kms->rm); | |
| 890 | − if (IS_ERR_OR_NULL(dpu_kms->hw_mdp)) { | |
| 897 | + dpu_kms->hw_mdp = dpu_hw_mdptop_init(MDP_TOP, dpu_kms->mmio, | |
| 898 | + dpu_kms->catalog); | |
| 899 | + if (IS_ERR(dpu_kms->hw_mdp)) { | |
| 891 | 900 | rc = PTR_ERR(dpu_kms->hw_mdp); |
| 892 | − if (!dpu_kms->hw_mdp) | |
| 893 | − rc = -EINVAL; | |
| 894 | 901 | DPU_ERROR("failed to get hw_mdp: %d\n", rc); |
| 895 | 902 | dpu_kms->hw_mdp = NULL; |
| 896 | 903 | goto power_error; |
| @@ -926,16 +933,6 @@ static int dpu_kms_hw_init(struct msm_kms *kms) · 29 unmodified lines | ||
| 926 | 933 | goto hw_intr_init_err; |
| 927 | 934 | } |
| 928 | 935 | |
| 929 | − /* | |
| 930 | − * _dpu_kms_drm_obj_init should create the DRM related objects | |
| 931 | − * i.e. CRTCs, planes, encoders, connectors and so forth | |
| 932 | − */ | |
| 933 | − rc = _dpu_kms_drm_obj_init(dpu_kms); | |
| 934 | − if (rc) { | |
| 935 | − DPU_ERROR("modeset init failed: %d\n", rc); | |
| 936 | − goto drm_obj_init_err; | |
| 937 | − } | |
| 938 | − | |
| 939 | 936 | dev->mode_config.min_width = 0; |
| 940 | 937 | dev->mode_config.min_height = 0; |
| 941 | 938 | |
| @@ -952,6 +949,16 @@ static int dpu_kms_hw_init(struct msm_kms *kms) · 10 unmodified lines | ||
| 952 | 949 | */ |
| 953 | 950 | dev->mode_config.allow_fb_modifiers = true; |
| 954 | 951 | |
| 952 | + /* | |
| 953 | + * _dpu_kms_drm_obj_init should create the DRM related objects | |
| 954 | + * i.e. CRTCs, planes, encoders, connectors and so forth | |
| 955 | + */ | |
| 956 | + rc = _dpu_kms_drm_obj_init(dpu_kms); | |
| 957 | + if (rc) { | |
| 958 | + DPU_ERROR("modeset init failed: %d\n", rc); | |
| 959 | + goto drm_obj_init_err; | |
| 960 | + } | |
| 961 | + | |
| 955 | 962 | dpu_vbif_init_memtypes(dpu_kms); |
| 956 | 963 | |
| 957 | 964 | pm_runtime_put_sync(&dpu_kms->pdev->dev); |
| @@ -23,11 +23,14 @@ struct dpu_mdss { · 22 unmodified lines | ||
| 23 | 23 | struct dpu_irq_controller irq_controller; |
| 24 | 24 | }; |
| 25 | 25 | |
| 26 | −static irqreturn_t dpu_mdss_irq(int irq, void *arg) | |
| 26 | +static void dpu_mdss_irq(struct irq_desc *desc) | |
| 27 | 27 | { |
| 28 | − struct dpu_mdss *dpu_mdss = arg; | |
| 28 | + struct dpu_mdss *dpu_mdss = irq_desc_get_handler_data(desc); | |
| 29 | + struct irq_chip *chip = irq_desc_get_chip(desc); | |
| 29 | 30 | u32 interrupts; |
| 30 | 31 | |
| 32 | + chained_irq_enter(chip, desc); | |
| 33 | + | |
| 31 | 34 | interrupts = readl_relaxed(dpu_mdss->mmio + HW_INTR_STATUS); |
| 32 | 35 | |
| 33 | 36 | while (interrupts) { |
| @@ -39,20 +42,20 @@ static irqreturn_t dpu_mdss_irq(int irq, void *arg) · 5 unmodified lines | ||
| 39 | 42 | hwirq); |
| 40 | 43 | if (mapping == 0) { |
| 41 | 44 | DRM_ERROR("couldn't find irq mapping for %lu\n", hwirq); |
| 42 | − return IRQ_NONE; | |
| 45 | + break; | |
| 43 | 46 | } |
| 44 | 47 | |
| 45 | 48 | rc = generic_handle_irq(mapping); |
| 46 | 49 | if (rc < 0) { |
| 47 | 50 | DRM_ERROR("handle irq fail: irq=%lu mapping=%u rc=%d\n", |
| 48 | 51 | hwirq, mapping, rc); |
| 49 | − return IRQ_NONE; | |
| 52 | + break; | |
| 50 | 53 | } |
| 51 | 54 | |
| 52 | 55 | interrupts &= ~(1 << hwirq); |
| 53 | 56 | } |
| 54 | 57 | |
| 55 | − return IRQ_HANDLED; | |
| 58 | + chained_irq_exit(chip, desc); | |
| 56 | 59 | } |
| 57 | 60 | |
| 58 | 61 | static void dpu_mdss_irq_mask(struct irq_data *irqd) |
| @@ -83,16 +86,16 @@ static struct irq_chip dpu_mdss_irq_chip = { · 24 unmodified lines | ||
| 83 | 86 | .irq_unmask = dpu_mdss_irq_unmask, |
| 84 | 87 | }; |
| 85 | 88 | |
| 89 | +static struct lock_class_key dpu_mdss_lock_key, dpu_mdss_request_key; | |
| 90 | + | |
| 86 | 91 | static int dpu_mdss_irqdomain_map(struct irq_domain *domain, |
| 87 | 92 | unsigned int irq, irq_hw_number_t hwirq) |
| 88 | 93 | { |
| 89 | 94 | struct dpu_mdss *dpu_mdss = domain->host_data; |
| 90 | − int ret; | |
| 91 | 95 | |
| 96 | + irq_set_lockdep_class(irq, &dpu_mdss_lock_key, &dpu_mdss_request_key); | |
| 92 | 97 | irq_set_chip_and_handler(irq, &dpu_mdss_irq_chip, handle_level_irq); |
| 93 | − ret = irq_set_chip_data(irq, dpu_mdss); | |
| 94 | − | |
| 95 | − return ret; | |
| 98 | + return irq_set_chip_data(irq, dpu_mdss); | |
| 96 | 99 | } |
| 97 | 100 | |
| 98 | 101 | static const struct irq_domain_ops dpu_mdss_irqdomain_ops = { |
| @@ -159,11 +162,13 @@ static void dpu_mdss_destroy(struct drm_device *dev) · 60 unmodified lines | ||
| 159 | 162 | struct msm_drm_private *priv = dev->dev_private; |
| 160 | 163 | struct dpu_mdss *dpu_mdss = to_dpu_mdss(priv->mdss); |
| 161 | 164 | struct dss_module_power *mp = &dpu_mdss->mp; |
| 165 | + int irq; | |
| 162 | 166 | |
| 163 | 167 | pm_runtime_suspend(dev->dev); |
| 164 | 168 | pm_runtime_disable(dev->dev); |
| 165 | 169 | _dpu_mdss_irq_domain_fini(dpu_mdss); |
| 166 | − free_irq(platform_get_irq(pdev, 0), dpu_mdss); | |
| 170 | + irq = platform_get_irq(pdev, 0); | |
| 171 | + irq_set_chained_handler_and_data(irq, NULL, NULL); | |
| 167 | 172 | msm_dss_put_clk(mp->clk_config, mp->num_clk); |
| 168 | 173 | devm_kfree(&pdev->dev, mp->clk_config); |
| 169 | 174 | |
| @@ -187,6 +192,7 @@ int dpu_mdss_init(struct drm_device *dev) · 17 unmodified lines | ||
| 187 | 192 | struct dpu_mdss *dpu_mdss; |
| 188 | 193 | struct dss_module_power *mp; |
| 189 | 194 | int ret = 0; |
| 195 | + int irq; | |
| 190 | 196 | |
| 191 | 197 | dpu_mdss = devm_kzalloc(dev->dev, sizeof(*dpu_mdss), GFP_KERNEL); |
| 192 | 198 | if (!dpu_mdss) |
| @@ -219,12 +225,12 @@ int dpu_mdss_init(struct drm_device *dev) · 26 unmodified lines | ||
| 219 | 225 | if (ret) |
| 220 | 226 | goto irq_domain_error; |
| 221 | 227 | |
| 222 | − ret = request_irq(platform_get_irq(pdev, 0), | |
| 223 | − dpu_mdss_irq, 0, "dpu_mdss_isr", dpu_mdss); | |
| 224 | − if (ret) { | |
| 225 | − DPU_ERROR("failed to init irq: %d\n", ret); | |
| 228 | + irq = platform_get_irq(pdev, 0); | |
| 229 | + if (irq < 0) | |
| 226 | 230 | goto irq_error; |
| 227 | − } | |
| 231 | + | |
| 232 | + irq_set_chained_handler_and_data(irq, dpu_mdss_irq, | |
| 233 | + dpu_mdss); | |
| 228 | 234 | |
| 229 | 235 | pm_runtime_enable(dev->dev); |
| 230 | 236 | |
| @@ -95,8 +95,6 @@ struct dpu_plane { · 94 unmodified lines | ||
| 95 | 95 | |
| 96 | 96 | enum dpu_sspp pipe; |
| 97 | 97 | uint32_t features; /* capabilities from catalog */ |
| 98 | − uint32_t nformats; | |
| 99 | − uint32_t formats[64]; | |
| 100 | 98 | |
| 101 | 99 | struct dpu_hw_pipe *pipe_hw; |
| 102 | 100 | struct dpu_hw_pipe_cfg pipe_cfg; |
| @@ -121,6 +119,12 @@ struct dpu_plane { · 18 unmodified lines | ||
| 121 | 119 | bool debugfs_default_scale; |
| 122 | 120 | }; |
| 123 | 121 | |
| 122 | +static const uint64_t supported_format_modifiers[] = { | |
| 123 | + DRM_FORMAT_MOD_QCOM_COMPRESSED, | |
| 124 | + DRM_FORMAT_MOD_LINEAR, | |
| 125 | + DRM_FORMAT_MOD_INVALID | |
| 126 | +}; | |
| 127 | + | |
| 124 | 128 | #define to_dpu_plane(x) container_of(x, struct dpu_plane, base) |
| 125 | 129 | |
| 126 | 130 | static struct dpu_kms *_dpu_plane_get_kms(struct drm_plane *plane) |
| @@ -1410,6 +1414,23 @@ static void dpu_plane_early_unregister(struct drm_plane *plane) · 1283 unmodified lines | ||
| 1410 | 1414 | debugfs_remove_recursive(pdpu->debugfs_root); |
| 1411 | 1415 | } |
| 1412 | 1416 | |
| 1417 | +static bool dpu_plane_format_mod_supported(struct drm_plane *plane, | |
| 1418 | + uint32_t format, uint64_t modifier) | |
| 1419 | +{ | |
| 1420 | + if (modifier == DRM_FORMAT_MOD_LINEAR) | |
| 1421 | + return true; | |
| 1422 | + | |
| 1423 | + if (modifier == DRM_FORMAT_MOD_QCOM_COMPRESSED) { | |
| 1424 | + int i; | |
| 1425 | + for (i = 0; i < ARRAY_SIZE(qcom_compressed_supported_formats); i++) { | |
| 1426 | + if (format == qcom_compressed_supported_formats[i]) | |
| 1427 | + return true; | |
| 1428 | + } | |
| 1429 | + } | |
| 1430 | + | |
| 1431 | + return false; | |
| 1432 | +} | |
| 1433 | + | |
| 1413 | 1434 | static const struct drm_plane_funcs dpu_plane_funcs = { |
| 1414 | 1435 | .update_plane = drm_atomic_helper_update_plane, |
| 1415 | 1436 | .disable_plane = drm_atomic_helper_disable_plane, |
| @@ -1419,6 +1440,7 @@ static const struct drm_plane_funcs dpu_plane_funcs = { · 3 unmodified lines | ||
| 1419 | 1440 | .atomic_destroy_state = dpu_plane_destroy_state, |
| 1420 | 1441 | .late_register = dpu_plane_late_register, |
| 1421 | 1442 | .early_unregister = dpu_plane_early_unregister, |
| 1443 | + .format_mod_supported = dpu_plane_format_mod_supported, | |
| 1422 | 1444 | }; |
| 1423 | 1445 | |
| 1424 | 1446 | static const struct drm_plane_helper_funcs dpu_plane_helper_funcs = { |
| @@ -1444,11 +1466,12 @@ struct drm_plane *dpu_plane_init(struct drm_device *dev, · 19 unmodified lines | ||
| 1444 | 1466 | unsigned long possible_crtcs, u32 master_plane_id) |
| 1445 | 1467 | { |
| 1446 | 1468 | struct drm_plane *plane = NULL, *master_plane = NULL; |
| 1447 | − const struct dpu_format_extended *format_list; | |
| 1469 | + const uint32_t *format_list; | |
| 1448 | 1470 | struct dpu_plane *pdpu; |
| 1449 | 1471 | struct msm_drm_private *priv = dev->dev_private; |
| 1450 | 1472 | struct dpu_kms *kms = to_dpu_kms(priv->kms); |
| 1451 | 1473 | int zpos_max = DPU_ZPOS_MAX; |
| 1474 | + uint32_t num_formats; | |
| 1452 | 1475 | int ret = -EINVAL; |
| 1453 | 1476 | |
| 1454 | 1477 | /* create and zero local structure */ |
| @@ -1491,24 +1514,18 @@ struct drm_plane *dpu_plane_init(struct drm_device *dev, · 36 unmodified lines | ||
| 1491 | 1514 | goto clean_sspp; |
| 1492 | 1515 | } |
| 1493 | 1516 | |
| 1494 | − if (!master_plane_id) | |
| 1495 | − format_list = pdpu->pipe_sblk->format_list; | |
| 1496 | − else | |
| 1517 | + if (pdpu->is_virtual) { | |
| 1497 | 1518 | format_list = pdpu->pipe_sblk->virt_format_list; |
| 1498 | − | |
| 1499 | − pdpu->nformats = dpu_populate_formats(format_list, | |
| 1500 | − pdpu->formats, | |
| 1501 | − 0, | |
| 1502 | − ARRAY_SIZE(pdpu->formats)); | |
| 1503 | − | |
| 1504 | − if (!pdpu->nformats) { | |
| 1505 | − DPU_ERROR("[%u]no valid formats for plane\n", pipe); | |
| 1506 | − goto clean_sspp; | |
| 1519 | + num_formats = pdpu->pipe_sblk->virt_num_formats; | |
| 1520 | + } | |
| 1521 | + else { | |
| 1522 | + format_list = pdpu->pipe_sblk->format_list; | |
| 1523 | + num_formats = pdpu->pipe_sblk->num_formats; | |
| 1507 | 1524 | } |
| 1508 | 1525 | |
| 1509 | 1526 | ret = drm_universal_plane_init(dev, plane, 0xff, &dpu_plane_funcs, |
| 1510 | − pdpu->formats, pdpu->nformats, | |
| 1511 | − NULL, type, NULL); | |
| 1527 | + format_list, num_formats, | |
| 1528 | + supported_format_modifiers, type, NULL); | |
| 1512 | 1529 | if (ret) |
| 1513 | 1530 | goto clean_sspp; |
| 1514 | 1531 | |
| @@ -28,23 +28,18 @@ · 27 unmodified lines | ||
| 28 | 28 | /** |
| 29 | 29 | * struct dpu_plane_state: Define dpu extension of drm plane state object |
| 30 | 30 | * @base: base drm plane state object |
| 31 | − * @property_state: Local storage for msm_prop properties | |
| 32 | − * @property_values: cached plane property values | |
| 33 | 31 | * @aspace: pointer to address space for input/output buffers |
| 34 | − * @input_fence: dereferenced input fence pointer | |
| 35 | 32 | * @stage: assigned by crtc blender |
| 36 | 33 | * @multirect_index: index of the rectangle of SSPP |
| 37 | 34 | * @multirect_mode: parallel or time multiplex multirect mode |
| 38 | 35 | * @pending: whether the current update is still pending |
| 39 | 36 | * @scaler3_cfg: configuration data for scaler3 |
| 40 | 37 | * @pixel_ext: configuration data for pixel extensions |
| 41 | − * @scaler_check_state: indicates status of user provided pixel extension data | |
| 42 | 38 | * @cdp_cfg: CDP configuration |
| 43 | 39 | */ |
| 44 | 40 | struct dpu_plane_state { |
| 45 | 41 | struct drm_plane_state base; |
| 46 | 42 | struct msm_gem_address_space *aspace; |
| 47 | − void *input_fence; | |
| 48 | 43 | enum dpu_stage stage; |
| 49 | 44 | uint32_t multirect_index; |
| 50 | 45 | uint32_t multirect_mode; |
| @@ -106,12 +101,6 @@ void dpu_plane_restore(struct drm_plane *plane); · 55 unmodified lines | ||
| 106 | 101 | */ |
| 107 | 102 | void dpu_plane_flush(struct drm_plane *plane); |
| 108 | 103 | |
| 109 | −/** | |
| 110 | − * dpu_plane_kickoff - final plane operations before commit kickoff | |
| 111 | − * @plane: Pointer to drm plane structure | |
| 112 | − */ | |
| 113 | −void dpu_plane_kickoff(struct drm_plane *plane); | |
| 114 | − | |
| 115 | 104 | /** |
| 116 | 105 | * dpu_plane_set_error: enable/disable error condition |
| 117 | 106 | * @plane: pointer to drm_plane structure |
| @@ -146,14 +135,6 @@ int dpu_plane_validate_multirect_v2(struct dpu_multirect_plane_states *plane); · 28 unmodified lines | ||
| 146 | 135 | */ |
| 147 | 136 | void dpu_plane_clear_multirect(const struct drm_plane_state *drm_state); |
| 148 | 137 | |
| 149 | −/** | |
| 150 | − * dpu_plane_wait_input_fence - wait for input fence object | |
| 151 | − * @plane: Pointer to DRM plane object | |
| 152 | − * @wait_ms: Wait timeout value | |
| 153 | − * Returns: Zero on success | |
| 154 | − */ | |
| 155 | −int dpu_plane_wait_input_fence(struct drm_plane *plane, uint32_t wait_ms); | |
| 156 | − | |
| 157 | 138 | /** |
| 158 | 139 | * dpu_plane_color_fill - enables color fill on plane |
| 159 | 140 | * @plane: Pointer to DRM plane object |
| @@ -164,12 +145,4 @@ int dpu_plane_wait_input_fence(struct drm_plane *plane, uint32_t wait_ms); · 4 unmodified lines | ||
| 164 | 145 | int dpu_plane_color_fill(struct drm_plane *plane, |
| 165 | 146 | uint32_t color, uint32_t alpha); |
| 166 | 147 | |
| 167 | −/** | |
| 168 | − * dpu_plane_set_revalidate - sets revalidate flag which forces a full | |
| 169 | − * validation of the plane properties in the next atomic check | |
| 170 | − * @plane: Pointer to DRM plane object | |
| 171 | − * @enable: Boolean to set/unset the flag | |
| 172 | − */ | |
| 173 | −void dpu_plane_set_revalidate(struct drm_plane *plane, bool enable); | |
| 174 | − | |
| 175 | 148 | #endif /* _DPU_PLANE_H_ */ |
| @@ -21,8 +21,8 @@ · 20 unmodified lines | ||
| 21 | 21 | #include "dpu_encoder.h" |
| 22 | 22 | #include "dpu_trace.h" |
| 23 | 23 | |
| 24 | −#define RESERVED_BY_OTHER(h, r) \ | |
| 25 | − ((h)->rsvp && ((h)->rsvp->enc_id != (r)->enc_id)) | |
| 24 | +#define RESERVED_BY_OTHER(h, r) \ | |
| 25 | + ((h)->enc_id && (h)->enc_id != r) | |
| 26 | 26 | |
| 27 | 27 | /** |
| 28 | 28 | * struct dpu_rm_requirements - Reservation requirements parameter bundle |
| @@ -34,90 +34,21 @@ struct dpu_rm_requirements { · 5 unmodified lines | ||
| 34 | 34 | struct dpu_encoder_hw_resources hw_res; |
| 35 | 35 | }; |
| 36 | 36 | |
| 37 | −/** | |
| 38 | − * struct dpu_rm_rsvp - Use Case Reservation tagging structure | |
| 39 | − * Used to tag HW blocks as reserved by a CRTC->Encoder->Connector chain | |
| 40 | − * By using as a tag, rather than lists of pointers to HW blocks used | |
| 41 | − * we can avoid some list management since we don't know how many blocks | |
| 42 | − * of each type a given use case may require. | |
| 43 | − * @list: List head for list of all reservations | |
| 44 | − * @seq: Global RSVP sequence number for debugging, especially for | |
| 45 | − * differentiating differenct allocations for same encoder. | |
| 46 | − * @enc_id: Reservations are tracked by Encoder DRM object ID. | |
| 47 | − * CRTCs may be connected to multiple Encoders. | |
| 48 | − * An encoder or connector id identifies the display path. | |
| 49 | − */ | |
| 50 | −struct dpu_rm_rsvp { | |
| 51 | − struct list_head list; | |
| 52 | − uint32_t seq; | |
| 53 | − uint32_t enc_id; | |
| 54 | −}; | |
| 55 | 37 | |
| 56 | 38 | /** |
| 57 | 39 | * struct dpu_rm_hw_blk - hardware block tracking list member |
| 58 | 40 | * @list: List head for list of all hardware blocks tracking items |
| 59 | − * @rsvp: Pointer to use case reservation if reserved by a client | |
| 60 | − * @rsvp_nxt: Temporary pointer used during reservation to the incoming | |
| 61 | − * request. Will be swapped into rsvp if proposal is accepted | |
| 62 | − * @type: Type of hardware block this structure tracks | |
| 63 | 41 | * @id: Hardware ID number, within it's own space, ie. LM_X |
| 64 | − * @catalog: Pointer to the hardware catalog entry for this block | |
| 42 | + * @enc_id: Encoder id to which this blk is binded | |
| 65 | 43 | * @hw: Pointer to the hardware register access object for this block |
| 66 | 44 | */ |
| 67 | 45 | struct dpu_rm_hw_blk { |
| 68 | 46 | struct list_head list; |
| 69 | − struct dpu_rm_rsvp *rsvp; | |
| 70 | − struct dpu_rm_rsvp *rsvp_nxt; | |
| 71 | − enum dpu_hw_blk_type type; | |
| 72 | 47 | uint32_t id; |
| 48 | + uint32_t enc_id; | |
| 73 | 49 | struct dpu_hw_blk *hw; |
| 74 | 50 | }; |
| 75 | 51 | |
| 76 | −/** | |
| 77 | − * dpu_rm_dbg_rsvp_stage - enum of steps in making reservation for event logging | |
| 78 | − */ | |
| 79 | −enum dpu_rm_dbg_rsvp_stage { | |
| 80 | − DPU_RM_STAGE_BEGIN, | |
| 81 | − DPU_RM_STAGE_AFTER_CLEAR, | |
| 82 | − DPU_RM_STAGE_AFTER_RSVPNEXT, | |
| 83 | − DPU_RM_STAGE_FINAL | |
| 84 | −}; | |
| 85 | − | |
| 86 | −static void _dpu_rm_print_rsvps( | |
| 87 | − struct dpu_rm *rm, | |
| 88 | − enum dpu_rm_dbg_rsvp_stage stage) | |
| 89 | −{ | |
| 90 | − struct dpu_rm_rsvp *rsvp; | |
| 91 | − struct dpu_rm_hw_blk *blk; | |
| 92 | − enum dpu_hw_blk_type type; | |
| 93 | − | |
| 94 | − DPU_DEBUG("%d\n", stage); | |
| 95 | − | |
| 96 | − list_for_each_entry(rsvp, &rm->rsvps, list) { | |
| 97 | − DRM_DEBUG_KMS("%d rsvp[s%ue%u]\n", stage, rsvp->seq, | |
| 98 | − rsvp->enc_id); | |
| 99 | − } | |
| 100 | − | |
| 101 | − for (type = 0; type < DPU_HW_BLK_MAX; type++) { | |
| 102 | − list_for_each_entry(blk, &rm->hw_blks[type], list) { | |
| 103 | − if (!blk->rsvp && !blk->rsvp_nxt) | |
| 104 | − continue; | |
| 105 | − | |
| 106 | − DRM_DEBUG_KMS("%d rsvp[s%ue%u->s%ue%u] %d %d\n", stage, | |
| 107 | − (blk->rsvp) ? blk->rsvp->seq : 0, | |
| 108 | − (blk->rsvp) ? blk->rsvp->enc_id : 0, | |
| 109 | − (blk->rsvp_nxt) ? blk->rsvp_nxt->seq : 0, | |
| 110 | − (blk->rsvp_nxt) ? blk->rsvp_nxt->enc_id : 0, | |
| 111 | − blk->type, blk->id); | |
| 112 | − } | |
| 113 | − } | |
| 114 | −} | |
| 115 | − | |
| 116 | −struct dpu_hw_mdp *dpu_rm_get_mdp(struct dpu_rm *rm) | |
| 117 | −{ | |
| 118 | − return rm->hw_mdp; | |
| 119 | −} | |
| 120 | − | |
| 121 | 52 | void dpu_rm_init_hw_iter( |
| 122 | 53 | struct dpu_rm_hw_iter *iter, |
| 123 | 54 | uint32_t enc_id, |
| @@ -148,15 +79,7 @@ static bool _dpu_rm_get_hw_locked(struct dpu_rm *rm, struct dpu_rm_hw_iter *i) · 24 unmodified lines | ||
| 148 | 79 | i->blk = list_prepare_entry(i->blk, blk_list, list); |
| 149 | 80 | |
| 150 | 81 | list_for_each_entry_continue(i->blk, blk_list, list) { |
| 151 | − struct dpu_rm_rsvp *rsvp = i->blk->rsvp; | |
| 152 | − | |
| 153 | − if (i->blk->type != i->type) { | |
| 154 | − DPU_ERROR("found incorrect block type %d on %d list\n", | |
| 155 | − i->blk->type, i->type); | |
| 156 | − return false; | |
| 157 | − } | |
| 158 | − | |
| 159 | − if ((i->enc_id == 0) || (rsvp && rsvp->enc_id == i->enc_id)) { | |
| 82 | + if (i->enc_id == i->blk->enc_id) { | |
| 160 | 83 | i->hw = i->blk->hw; |
| 161 | 84 | DPU_DEBUG("found type %d id %d for enc %d\n", |
| 162 | 85 | i->type, i->blk->id, i->enc_id); |
| @@ -208,34 +131,18 @@ static void _dpu_rm_hw_destroy(enum dpu_hw_blk_type type, void *hw) · 45 unmodified lines | ||
| 208 | 131 | |
| 209 | 132 | int dpu_rm_destroy(struct dpu_rm *rm) |
| 210 | 133 | { |
| 211 | − | |
| 212 | − struct dpu_rm_rsvp *rsvp_cur, *rsvp_nxt; | |
| 213 | 134 | struct dpu_rm_hw_blk *hw_cur, *hw_nxt; |
| 214 | 135 | enum dpu_hw_blk_type type; |
| 215 | 136 | |
| 216 | − if (!rm) { | |
| 217 | − DPU_ERROR("invalid rm\n"); | |
| 218 | − return -EINVAL; | |
| 219 | − } | |
| 220 | − | |
| 221 | − list_for_each_entry_safe(rsvp_cur, rsvp_nxt, &rm->rsvps, list) { | |
| 222 | − list_del(&rsvp_cur->list); | |
| 223 | − kfree(rsvp_cur); | |
| 224 | − } | |
| 225 | − | |
| 226 | − | |
| 227 | 137 | for (type = 0; type < DPU_HW_BLK_MAX; type++) { |
| 228 | 138 | list_for_each_entry_safe(hw_cur, hw_nxt, &rm->hw_blks[type], |
| 229 | 139 | list) { |
| 230 | 140 | list_del(&hw_cur->list); |
| 231 | − _dpu_rm_hw_destroy(hw_cur->type, hw_cur->hw); | |
| 141 | + _dpu_rm_hw_destroy(type, hw_cur->hw); | |
| 232 | 142 | kfree(hw_cur); |
| 233 | 143 | } |
| 234 | 144 | } |
| 235 | 145 | |
| 236 | − dpu_hw_mdp_destroy(rm->hw_mdp); | |
| 237 | − rm->hw_mdp = NULL; | |
| 238 | − | |
| 239 | 146 | mutex_destroy(&rm->rm_lock); |
| 240 | 147 | |
| 241 | 148 | return 0; |
| @@ -250,11 +157,8 @@ static int _dpu_rm_hw_blk_create( · 8 unmodified lines | ||
| 250 | 157 | void *hw_catalog_info) |
| 251 | 158 | { |
| 252 | 159 | struct dpu_rm_hw_blk *blk; |
| 253 | − struct dpu_hw_mdp *hw_mdp; | |
| 254 | 160 | void *hw; |
| 255 | 161 | |
| 256 | − hw_mdp = rm->hw_mdp; | |
| 257 | − | |
| 258 | 162 | switch (type) { |
| 259 | 163 | case DPU_HW_BLK_LM: |
| 260 | 164 | hw = dpu_hw_lm_init(id, mmio, cat); |
| @@ -290,9 +194,9 @@ static int _dpu_rm_hw_blk_create( · 29 unmodified lines | ||
| 290 | 194 | return -ENOMEM; |
| 291 | 195 | } |
| 292 | 196 | |
| 293 | − blk->type = type; | |
| 294 | 197 | blk->id = id; |
| 295 | 198 | blk->hw = hw; |
| 199 | + blk->enc_id = 0; | |
| 296 | 200 | list_add_tail(&blk->list, &rm->hw_blks[type]); |
| 297 | 201 | |
| 298 | 202 | return 0; |
| @@ -300,13 +204,12 @@ static int _dpu_rm_hw_blk_create( · 1 unmodified line | ||
| 300 | 204 | |
| 301 | 205 | int dpu_rm_init(struct dpu_rm *rm, |
| 302 | 206 | struct dpu_mdss_cfg *cat, |
| 303 | − void __iomem *mmio, | |
| 304 | − struct drm_device *dev) | |
| 207 | + void __iomem *mmio) | |
| 305 | 208 | { |
| 306 | 209 | int rc, i; |
| 307 | 210 | enum dpu_hw_blk_type type; |
| 308 | 211 | |
| 309 | − if (!rm || !cat || !mmio || !dev) { | |
| 212 | + if (!rm || !cat || !mmio) { | |
| 310 | 213 | DPU_ERROR("invalid kms\n"); |
| 311 | 214 | return -EINVAL; |
| 312 | 215 | } |
| @@ -316,21 +219,9 @@ int dpu_rm_init(struct dpu_rm *rm, · 3 unmodified lines | ||
| 316 | 219 | |
| 317 | 220 | mutex_init(&rm->rm_lock); |
| 318 | 221 | |
| 319 | − INIT_LIST_HEAD(&rm->rsvps); | |
| 320 | 222 | for (type = 0; type < DPU_HW_BLK_MAX; type++) |
| 321 | 223 | INIT_LIST_HEAD(&rm->hw_blks[type]); |
| 322 | 224 | |
| 323 | − rm->dev = dev; | |
| 324 | − | |
| 325 | − /* Some of the sub-blocks require an mdptop to be created */ | |
| 326 | − rm->hw_mdp = dpu_hw_mdptop_init(MDP_TOP, mmio, cat); | |
| 327 | − if (IS_ERR_OR_NULL(rm->hw_mdp)) { | |
| 328 | − rc = PTR_ERR(rm->hw_mdp); | |
| 329 | − rm->hw_mdp = NULL; | |
| 330 | − DPU_ERROR("failed: mdp hw not available\n"); | |
| 331 | − goto fail; | |
| 332 | − } | |
| 333 | − | |
| 334 | 225 | /* Interrogate HW catalog and create tracking items for hw blocks */ |
| 335 | 226 | for (i = 0; i < cat->mixer_count; i++) { |
| 336 | 227 | struct dpu_lm_cfg *lm = &cat->mixer[i]; |
| @@ -410,7 +301,7 @@ static bool _dpu_rm_needs_split_display(const struct msm_display_topology *top) · 73 unmodified lines | ||
| 410 | 301 | * proposed use case requirements, incl. hardwired dependent blocks like |
| 411 | 302 | * pingpong |
| 412 | 303 | * @rm: dpu resource manager handle |
| 413 | − * @rsvp: reservation currently being created | |
| 304 | + * @enc_id: encoder id requesting for allocation | |
| 414 | 305 | * @reqs: proposed use case requirements |
| 415 | 306 | * @lm: proposed layer mixer, function checks if lm, and all other hardwired |
| 416 | 307 | * blocks connected to the lm (pp) is available and appropriate |
| @@ -422,7 +313,7 @@ static bool _dpu_rm_needs_split_display(const struct msm_display_topology *top) · 5 unmodified lines | ||
| 422 | 313 | */ |
| 423 | 314 | static bool _dpu_rm_check_lm_and_get_connected_blks( |
| 424 | 315 | struct dpu_rm *rm, |
| 425 | − struct dpu_rm_rsvp *rsvp, | |
| 316 | + uint32_t enc_id, | |
| 426 | 317 | struct dpu_rm_requirements *reqs, |
| 427 | 318 | struct dpu_rm_hw_blk *lm, |
| 428 | 319 | struct dpu_rm_hw_blk **pp, |
| @@ -449,7 +340,7 @@ static bool _dpu_rm_check_lm_and_get_connected_blks( · 20 unmodified lines | ||
| 449 | 340 | } |
| 450 | 341 | |
| 451 | 342 | /* Already reserved? */ |
| 452 | − if (RESERVED_BY_OTHER(lm, rsvp)) { | |
| 343 | + if (RESERVED_BY_OTHER(lm, enc_id)) { | |
| 453 | 344 | DPU_DEBUG("lm %d already reserved\n", lm_cfg->id); |
| 454 | 345 | return false; |
| 455 | 346 | } |
| @@ -467,7 +358,7 @@ static bool _dpu_rm_check_lm_and_get_connected_blks( · 11 unmodified lines | ||
| 467 | 358 | return false; |
| 468 | 359 | } |
| 469 | 360 | |
| 470 | − if (RESERVED_BY_OTHER(*pp, rsvp)) { | |
| 361 | + if (RESERVED_BY_OTHER(*pp, enc_id)) { | |
| 471 | 362 | DPU_DEBUG("lm %d pp %d already reserved\n", lm->id, |
| 472 | 363 | (*pp)->id); |
| 473 | 364 | return false; |
| @@ -476,10 +367,8 @@ static bool _dpu_rm_check_lm_and_get_connected_blks( · 2 unmodified lines | ||
| 476 | 367 | return true; |
| 477 | 368 | } |
| 478 | 369 | |
| 479 | −static int _dpu_rm_reserve_lms( | |
| 480 | − struct dpu_rm *rm, | |
| 481 | − struct dpu_rm_rsvp *rsvp, | |
| 482 | − struct dpu_rm_requirements *reqs) | |
| 370 | +static int _dpu_rm_reserve_lms(struct dpu_rm *rm, uint32_t enc_id, | |
| 371 | + struct dpu_rm_requirements *reqs) | |
| 483 | 372 | |
| 484 | 373 | { |
| 485 | 374 | struct dpu_rm_hw_blk *lm[MAX_BLOCKS]; |
| @@ -504,7 +393,7 @@ static int _dpu_rm_reserve_lms( · 18 unmodified lines | ||
| 504 | 393 | lm[lm_count] = iter_i.blk; |
| 505 | 394 | |
| 506 | 395 | if (!_dpu_rm_check_lm_and_get_connected_blks( |
| 507 | − rm, rsvp, reqs, lm[lm_count], | |
| 396 | + rm, enc_id, reqs, lm[lm_count], | |
| 508 | 397 | &pp[lm_count], NULL)) |
| 509 | 398 | continue; |
| 510 | 399 | |
| @@ -519,7 +408,7 @@ static int _dpu_rm_reserve_lms( · 8 unmodified lines | ||
| 519 | 408 | continue; |
| 520 | 409 | |
| 521 | 410 | if (!_dpu_rm_check_lm_and_get_connected_blks( |
| 522 | − rm, rsvp, reqs, iter_j.blk, | |
| 411 | + rm, enc_id, reqs, iter_j.blk, | |
| 523 | 412 | &pp[lm_count], iter_i.blk)) |
| 524 | 413 | continue; |
| 525 | 414 | |
| @@ -537,11 +426,10 @@ static int _dpu_rm_reserve_lms( · 11 unmodified lines | ||
| 537 | 426 | if (!lm[i]) |
| 538 | 427 | break; |
| 539 | 428 | |
| 540 | − lm[i]->rsvp_nxt = rsvp; | |
| 541 | − pp[i]->rsvp_nxt = rsvp; | |
| 429 | + lm[i]->enc_id = enc_id; | |
| 430 | + pp[i]->enc_id = enc_id; | |
| 542 | 431 | |
| 543 | − trace_dpu_rm_reserve_lms(lm[i]->id, lm[i]->type, rsvp->enc_id, | |
| 544 | − pp[i]->id); | |
| 432 | + trace_dpu_rm_reserve_lms(lm[i]->id, enc_id, pp[i]->id); | |
| 545 | 433 | } |
| 546 | 434 | |
| 547 | 435 | return rc; |
| @@ -549,7 +437,7 @@ static int _dpu_rm_reserve_lms( · 1 unmodified line | ||
| 549 | 437 | |
| 550 | 438 | static int _dpu_rm_reserve_ctls( |
| 551 | 439 | struct dpu_rm *rm, |
| 552 | − struct dpu_rm_rsvp *rsvp, | |
| 440 | + uint32_t enc_id, | |
| 553 | 441 | const struct msm_display_topology *top) |
| 554 | 442 | { |
| 555 | 443 | struct dpu_rm_hw_blk *ctls[MAX_BLOCKS]; |
| @@ -570,7 +458,7 @@ static int _dpu_rm_reserve_ctls( · 14 unmodified lines | ||
| 570 | 458 | unsigned long features = ctl->caps->features; |
| 571 | 459 | bool has_split_display; |
| 572 | 460 | |
| 573 | − if (RESERVED_BY_OTHER(iter.blk, rsvp)) | |
| 461 | + if (RESERVED_BY_OTHER(iter.blk, enc_id)) | |
| 574 | 462 | continue; |
| 575 | 463 | |
| 576 | 464 | has_split_display = BIT(DPU_CTL_SPLIT_DISPLAY) & features; |
| @@ -591,9 +479,8 @@ static int _dpu_rm_reserve_ctls( · 14 unmodified lines | ||
| 591 | 479 | return -ENAVAIL; |
| 592 | 480 | |
| 593 | 481 | for (i = 0; i < ARRAY_SIZE(ctls) && i < num_ctls; i++) { |
| 594 | − ctls[i]->rsvp_nxt = rsvp; | |
| 595 | − trace_dpu_rm_reserve_ctls(ctls[i]->id, ctls[i]->type, | |
| 596 | − rsvp->enc_id); | |
| 482 | + ctls[i]->enc_id = enc_id; | |
| 483 | + trace_dpu_rm_reserve_ctls(ctls[i]->id, enc_id); | |
| 597 | 484 | } |
| 598 | 485 | |
| 599 | 486 | return 0; |
| @@ -601,7 +488,7 @@ static int _dpu_rm_reserve_ctls( · 1 unmodified line | ||
| 601 | 488 | |
| 602 | 489 | static int _dpu_rm_reserve_intf( |
| 603 | 490 | struct dpu_rm *rm, |
| 604 | − struct dpu_rm_rsvp *rsvp, | |
| 491 | + uint32_t enc_id, | |
| 605 | 492 | uint32_t id, |
| 606 | 493 | enum dpu_hw_blk_type type) |
| 607 | 494 | { |
| @@ -614,14 +501,13 @@ static int _dpu_rm_reserve_intf( · 6 unmodified lines | ||
| 614 | 501 | if (iter.blk->id != id) |
| 615 | 502 | continue; |
| 616 | 503 | |
| 617 | − if (RESERVED_BY_OTHER(iter.blk, rsvp)) { | |
| 504 | + if (RESERVED_BY_OTHER(iter.blk, enc_id)) { | |
| 618 | 505 | DPU_ERROR("type %d id %d already reserved\n", type, id); |
| 619 | 506 | return -ENAVAIL; |
| 620 | 507 | } |
| 621 | 508 | |
| 622 | − iter.blk->rsvp_nxt = rsvp; | |
| 623 | − trace_dpu_rm_reserve_intf(iter.blk->id, iter.blk->type, | |
| 624 | − rsvp->enc_id); | |
| 509 | + iter.blk->enc_id = enc_id; | |
| 510 | + trace_dpu_rm_reserve_intf(iter.blk->id, enc_id); | |
| 625 | 511 | break; |
| 626 | 512 | } |
| 627 | 513 | |
| @@ -636,7 +522,7 @@ static int _dpu_rm_reserve_intf( · 8 unmodified lines | ||
| 636 | 522 | |
| 637 | 523 | static int _dpu_rm_reserve_intf_related_hw( |
| 638 | 524 | struct dpu_rm *rm, |
| 639 | − struct dpu_rm_rsvp *rsvp, | |
| 525 | + uint32_t enc_id, | |
| 640 | 526 | struct dpu_encoder_hw_resources *hw_res) |
| 641 | 527 | { |
| 642 | 528 | int i, ret = 0; |
| @@ -646,7 +532,7 @@ static int _dpu_rm_reserve_intf_related_hw( · 3 unmodified lines | ||
| 646 | 532 | if (hw_res->intfs[i] == INTF_MODE_NONE) |
| 647 | 533 | continue; |
| 648 | 534 | id = i + INTF_0; |
| 649 | − ret = _dpu_rm_reserve_intf(rm, rsvp, id, | |
| 535 | + ret = _dpu_rm_reserve_intf(rm, enc_id, id, | |
| 650 | 536 | DPU_HW_BLK_INTF); |
| 651 | 537 | if (ret) |
| 652 | 538 | return ret; |
| @@ -655,33 +541,27 @@ static int _dpu_rm_reserve_intf_related_hw( · 2 unmodified lines | ||
| 655 | 541 | return ret; |
| 656 | 542 | } |
| 657 | 543 | |
| 658 | −static int _dpu_rm_make_next_rsvp( | |
| 544 | +static int _dpu_rm_make_reservation( | |
| 659 | 545 | struct dpu_rm *rm, |
| 660 | 546 | struct drm_encoder *enc, |
| 661 | 547 | struct drm_crtc_state *crtc_state, |
| 662 | − struct dpu_rm_rsvp *rsvp, | |
| 663 | 548 | struct dpu_rm_requirements *reqs) |
| 664 | 549 | { |
| 665 | 550 | int ret; |
| 666 | 551 | |
| 667 | − /* Create reservation info, tag reserved blocks with it as we go */ | |
| 668 | − rsvp->seq = ++rm->rsvp_next_seq; | |
| 669 | − rsvp->enc_id = enc->base.id; | |
| 670 | − list_add_tail(&rsvp->list, &rm->rsvps); | |
| 671 | − | |
| 672 | − ret = _dpu_rm_reserve_lms(rm, rsvp, reqs); | |
| 552 | + ret = _dpu_rm_reserve_lms(rm, enc->base.id, reqs); | |
| 673 | 553 | if (ret) { |
| 674 | 554 | DPU_ERROR("unable to find appropriate mixers\n"); |
| 675 | 555 | return ret; |
| 676 | 556 | } |
| 677 | 557 | |
| 678 | − ret = _dpu_rm_reserve_ctls(rm, rsvp, &reqs->topology); | |
| 558 | + ret = _dpu_rm_reserve_ctls(rm, enc->base.id, &reqs->topology); | |
| 679 | 559 | if (ret) { |
| 680 | 560 | DPU_ERROR("unable to find appropriate CTL\n"); |
| 681 | 561 | return ret; |
| 682 | 562 | } |
| 683 | 563 | |
| 684 | − ret = _dpu_rm_reserve_intf_related_hw(rm, rsvp, &reqs->hw_res); | |
| 564 | + ret = _dpu_rm_reserve_intf_related_hw(rm, enc->base.id, &reqs->hw_res); | |
| 685 | 565 | if (ret) |
| 686 | 566 | return ret; |
| 687 | 567 | |
| @@ -706,108 +586,31 @@ static int _dpu_rm_populate_requirements( · 18 unmodified lines | ||
| 706 | 586 | return 0; |
| 707 | 587 | } |
| 708 | 588 | |
| 709 | −static struct dpu_rm_rsvp *_dpu_rm_get_rsvp( | |
| 710 | − struct dpu_rm *rm, | |
| 711 | − struct drm_encoder *enc) | |
| 589 | +static void _dpu_rm_release_reservation(struct dpu_rm *rm, uint32_t enc_id) | |
| 712 | 590 | { |
| 713 | − struct dpu_rm_rsvp *i; | |
| 714 | − | |
| 715 | − if (!rm || !enc) { | |
| 716 | − DPU_ERROR("invalid params\n"); | |
| 717 | − return NULL; | |
| 718 | − } | |
| 719 | − | |
| 720 | − if (list_empty(&rm->rsvps)) | |
| 721 | − return NULL; | |
| 722 | − | |
| 723 | − list_for_each_entry(i, &rm->rsvps, list) | |
| 724 | − if (i->enc_id == enc->base.id) | |
| 725 | − return i; | |
| 726 | − | |
| 727 | − return NULL; | |
| 728 | −} | |
| 729 | − | |
| 730 | −/** | |
| 731 | − * _dpu_rm_release_rsvp - release resources and release a reservation | |
| 732 | − * @rm: KMS handle | |
| 733 | − * @rsvp: RSVP pointer to release and release resources for | |
| 734 | − */ | |
| 735 | −static void _dpu_rm_release_rsvp(struct dpu_rm *rm, struct dpu_rm_rsvp *rsvp) | |
| 736 | −{ | |
| 737 | − struct dpu_rm_rsvp *rsvp_c, *rsvp_n; | |
| 738 | 591 | struct dpu_rm_hw_blk *blk; |
| 739 | 592 | enum dpu_hw_blk_type type; |
| 740 | 593 | |
| 741 | − if (!rsvp) | |
| 742 | − return; | |
| 743 | − | |
| 744 | − DPU_DEBUG("rel rsvp %d enc %d\n", rsvp->seq, rsvp->enc_id); | |
| 745 | − | |
| 746 | − list_for_each_entry_safe(rsvp_c, rsvp_n, &rm->rsvps, list) { | |
| 747 | − if (rsvp == rsvp_c) { | |
| 748 | − list_del(&rsvp_c->list); | |
| 749 | − break; | |
| 750 | − } | |
| 751 | − } | |
| 752 | − | |
| 753 | 594 | for (type = 0; type < DPU_HW_BLK_MAX; type++) { |
| 754 | 595 | list_for_each_entry(blk, &rm->hw_blks[type], list) { |
| 755 | − if (blk->rsvp == rsvp) { | |
| 756 | − blk->rsvp = NULL; | |
| 757 | − DPU_DEBUG("rel rsvp %d enc %d %d %d\n", | |
| 758 | − rsvp->seq, rsvp->enc_id, | |
| 759 | − blk->type, blk->id); | |
| 760 | − } | |
| 761 | − if (blk->rsvp_nxt == rsvp) { | |
| 762 | − blk->rsvp_nxt = NULL; | |
| 763 | − DPU_DEBUG("rel rsvp_nxt %d enc %d %d %d\n", | |
| 764 | − rsvp->seq, rsvp->enc_id, | |
| 765 | − blk->type, blk->id); | |
| 596 | + if (blk->enc_id == enc_id) { | |
| 597 | + blk->enc_id = 0; | |
| 598 | + DPU_DEBUG("rel enc %d %d %d\n", enc_id, | |
| 599 | + type, blk->id); | |
| 766 | 600 | } |
| 767 | 601 | } |
| 768 | 602 | } |
| 769 | − | |
| 770 | − kfree(rsvp); | |
| 771 | 603 | } |
| 772 | 604 | |
| 773 | 605 | void dpu_rm_release(struct dpu_rm *rm, struct drm_encoder *enc) |
| 774 | 606 | { |
| 775 | − struct dpu_rm_rsvp *rsvp; | |
| 776 | − | |
| 777 | − if (!rm || !enc) { | |
| 778 | − DPU_ERROR("invalid params\n"); | |
| 779 | − return; | |
| 780 | − } | |
| 781 | − | |
| 782 | 607 | mutex_lock(&rm->rm_lock); |
| 783 | 608 | |
| 784 | − rsvp = _dpu_rm_get_rsvp(rm, enc); | |
| 785 | − if (!rsvp) { | |
| 786 | − DPU_ERROR("failed to find rsvp for enc %d\n", enc->base.id); | |
| 787 | − goto end; | |
| 788 | − } | |
| 609 | + _dpu_rm_release_reservation(rm, enc->base.id); | |
| 789 | 610 | |
| 790 | − _dpu_rm_release_rsvp(rm, rsvp); | |
| 791 | −end: | |
| 792 | 611 | mutex_unlock(&rm->rm_lock); |
| 793 | 612 | } |
| 794 | 613 | |
| 795 | −static void _dpu_rm_commit_rsvp(struct dpu_rm *rm, struct dpu_rm_rsvp *rsvp) | |
| 796 | −{ | |
| 797 | − struct dpu_rm_hw_blk *blk; | |
| 798 | − enum dpu_hw_blk_type type; | |
| 799 | − | |
| 800 | − /* Swap next rsvp to be the active */ | |
| 801 | − for (type = 0; type < DPU_HW_BLK_MAX; type++) { | |
| 802 | − list_for_each_entry(blk, &rm->hw_blks[type], list) { | |
| 803 | − if (blk->rsvp_nxt) { | |
| 804 | − blk->rsvp = blk->rsvp_nxt; | |
| 805 | − blk->rsvp_nxt = NULL; | |
| 806 | − } | |
| 807 | − } | |
| 808 | − } | |
| 809 | −} | |
| 810 | − | |
| 811 | 614 | int dpu_rm_reserve( |
| 812 | 615 | struct dpu_rm *rm, |
| 813 | 616 | struct drm_encoder *enc, |
| @@ -815,7 +618,6 @@ int dpu_rm_reserve( · 1 unmodified line | ||
| 815 | 618 | struct msm_display_topology topology, |
| 816 | 619 | bool test_only) |
| 817 | 620 | { |
| 818 | − struct dpu_rm_rsvp *rsvp_cur, *rsvp_nxt; | |
| 819 | 621 | struct dpu_rm_requirements reqs; |
| 820 | 622 | int ret; |
| 821 | 623 | |
| @@ -828,8 +630,6 @@ int dpu_rm_reserve( · 6 unmodified lines | ||
| 828 | 630 | |
| 829 | 631 | mutex_lock(&rm->rm_lock); |
| 830 | 632 | |
| 831 | − _dpu_rm_print_rsvps(rm, DPU_RM_STAGE_BEGIN); | |
| 832 | − | |
| 833 | 633 | ret = _dpu_rm_populate_requirements(rm, enc, crtc_state, &reqs, |
| 834 | 634 | topology); |
| 835 | 635 | if (ret) { |
| @@ -837,50 +637,17 @@ int dpu_rm_reserve( · 1 unmodified line | ||
| 837 | 637 | goto end; |
| 838 | 638 | } |
| 839 | 639 | |
| 840 | − /* | |
| 841 | − * We only support one active reservation per-hw-block. But to implement | |
| 842 | − * transactional semantics for test-only, and for allowing failure while | |
| 843 | − * modifying your existing reservation, over the course of this | |
| 844 | − * function we can have two reservations: | |
| 845 | − * Current: Existing reservation | |
| 846 | − * Next: Proposed reservation. The proposed reservation may fail, or may | |
| 847 | − * be discarded if in test-only mode. | |
| 848 | − * If reservation is successful, and we're not in test-only, then we | |
| 849 | − * replace the current with the next. | |
| 850 | − */ | |
| 851 | − rsvp_nxt = kzalloc(sizeof(*rsvp_nxt), GFP_KERNEL); | |
| 852 | − if (!rsvp_nxt) { | |
| 853 | − ret = -ENOMEM; | |
| 854 | − goto end; | |
| 855 | − } | |
| 856 | − | |
| 857 | − rsvp_cur = _dpu_rm_get_rsvp(rm, enc); | |
| 858 | − | |
| 859 | − /* Check the proposed reservation, store it in hw's "next" field */ | |
| 860 | − ret = _dpu_rm_make_next_rsvp(rm, enc, crtc_state, rsvp_nxt, &reqs); | |
| 861 | − | |
| 862 | − _dpu_rm_print_rsvps(rm, DPU_RM_STAGE_AFTER_RSVPNEXT); | |
| 863 | − | |
| 640 | + ret = _dpu_rm_make_reservation(rm, enc, crtc_state, &reqs); | |
| 864 | 641 | if (ret) { |
| 865 | 642 | DPU_ERROR("failed to reserve hw resources: %d\n", ret); |
| 866 | − _dpu_rm_release_rsvp(rm, rsvp_nxt); | |
| 643 | + _dpu_rm_release_reservation(rm, enc->base.id); | |
| 867 | 644 | } else if (test_only) { |
| 868 | − /* | |
| 869 | − * Normally, if test_only, test the reservation and then undo | |
| 870 | − * However, if the user requests LOCK, then keep the reservation | |
| 871 | − * made during the atomic_check phase. | |
| 872 | − */ | |
| 873 | − DPU_DEBUG("test_only: discard test rsvp[s%de%d]\n", | |
| 874 | − rsvp_nxt->seq, rsvp_nxt->enc_id); | |
| 875 | − _dpu_rm_release_rsvp(rm, rsvp_nxt); | |
| 876 | − } else { | |
| 877 | − _dpu_rm_release_rsvp(rm, rsvp_cur); | |
| 878 | − | |
| 879 | − _dpu_rm_commit_rsvp(rm, rsvp_nxt); | |
| 645 | + /* test_only: test the reservation and then undo */ | |
| 646 | + DPU_DEBUG("test_only: discard test [enc: %d]\n", | |
| 647 | + enc->base.id); | |
| 648 | + _dpu_rm_release_reservation(rm, enc->base.id); | |
| 880 | 649 | } |
| 881 | 650 | |
| 882 | − _dpu_rm_print_rsvps(rm, DPU_RM_STAGE_FINAL); | |
| 883 | − | |
| 884 | 651 | end: |
| 885 | 652 | mutex_unlock(&rm->rm_lock); |
| 886 | 653 | |
| @@ -22,22 +22,14 @@ · 21 unmodified lines | ||
| 22 | 22 | |
| 23 | 23 | /** |
| 24 | 24 | * struct dpu_rm - DPU dynamic hardware resource manager |
| 25 | − * @dev: device handle for event logging purposes | |
| 26 | − * @rsvps: list of hardware reservations by each crtc->encoder->connector | |
| 27 | 25 | * @hw_blks: array of lists of hardware resources present in the system, one |
| 28 | 26 | * list per type of hardware block |
| 29 | − * @hw_mdp: hardware object for mdp_top | |
| 30 | 27 | * @lm_max_width: cached layer mixer maximum width |
| 31 | − * @rsvp_next_seq: sequence number for next reservation for debugging purposes | |
| 32 | 28 | * @rm_lock: resource manager mutex |
| 33 | 29 | */ |
| 34 | 30 | struct dpu_rm { |
| 35 | − struct drm_device *dev; | |
| 36 | − struct list_head rsvps; | |
| 37 | 31 | struct list_head hw_blks[DPU_HW_BLK_MAX]; |
| 38 | − struct dpu_hw_mdp *hw_mdp; | |
| 39 | 32 | uint32_t lm_max_width; |
| 40 | − uint32_t rsvp_next_seq; | |
| 41 | 33 | struct mutex rm_lock; |
| 42 | 34 | }; |
| 43 | 35 | |
| @@ -67,13 +59,11 @@ struct dpu_rm_hw_iter { · 23 unmodified lines | ||
| 67 | 59 | * @rm: DPU Resource Manager handle |
| 68 | 60 | * @cat: Pointer to hardware catalog |
| 69 | 61 | * @mmio: mapped register io address of MDP |
| 70 | − * @dev: device handle for event logging purposes | |
| 71 | 62 | * @Return: 0 on Success otherwise -ERROR |
| 72 | 63 | */ |
| 73 | 64 | int dpu_rm_init(struct dpu_rm *rm, |
| 74 | 65 | struct dpu_mdss_cfg *cat, |
| 75 | − void __iomem *mmio, | |
| 76 | − struct drm_device *dev); | |
| 66 | + void __iomem *mmio); | |
| 77 | 67 | |
| 78 | 68 | /** |
| 79 | 69 | * dpu_rm_destroy - Free all memory allocated by dpu_rm_init |
| @@ -111,14 +101,6 @@ int dpu_rm_reserve(struct dpu_rm *rm, · 31 unmodified lines | ||
| 111 | 101 | */ |
| 112 | 102 | void dpu_rm_release(struct dpu_rm *rm, struct drm_encoder *enc); |
| 113 | 103 | |
| 114 | −/** | |
| 115 | − * dpu_rm_get_mdp - Retrieve HW block for MDP TOP. | |
| 116 | − * This is never reserved, and is usable by any display. | |
| 117 | − * @rm: DPU Resource Manager handle | |
| 118 | − * @Return: Pointer to hw block or NULL | |
| 119 | − */ | |
| 120 | −struct dpu_hw_mdp *dpu_rm_get_mdp(struct dpu_rm *rm); | |
| 121 | − | |
| 122 | 104 | /** |
| 123 | 105 | * dpu_rm_init_hw_iter - setup given iterator for new iteration over hw list |
| 124 | 106 | * using dpu_rm_get_hw |
| @@ -144,12 +126,4 @@ void dpu_rm_init_hw_iter( · 19 unmodified lines | ||
| 144 | 126 | * @Return: true on match found, false on no match found |
| 145 | 127 | */ |
| 146 | 128 | bool dpu_rm_get_hw(struct dpu_rm *rm, struct dpu_rm_hw_iter *iter); |
| 147 | − | |
| 148 | −/** | |
| 149 | − * dpu_rm_check_property_topctl - validate property bitmask before it is set | |
| 150 | − * @val: user's proposed topology control bitmask | |
| 151 | − * @Return: 0 on success or error | |
| 152 | − */ | |
| 153 | −int dpu_rm_check_property_topctl(uint64_t val); | |
| 154 | − | |
| 155 | 129 | #endif /* __DPU_RM_H__ */ |
| @@ -831,48 +831,42 @@ TRACE_EVENT(dpu_plane_disable, · 830 unmodified lines | ||
| 831 | 831 | ); |
| 832 | 832 | |
| 833 | 833 | DECLARE_EVENT_CLASS(dpu_rm_iter_template, |
| 834 | − TP_PROTO(uint32_t id, enum dpu_hw_blk_type type, uint32_t enc_id), | |
| 835 | − TP_ARGS(id, type, enc_id), | |
| 834 | + TP_PROTO(uint32_t id, uint32_t enc_id), | |
| 835 | + TP_ARGS(id, enc_id), | |
| 836 | 836 | TP_STRUCT__entry( |
| 837 | 837 | __field( uint32_t, id ) |
| 838 | − __field( enum dpu_hw_blk_type, type ) | |
| 839 | 838 | __field( uint32_t, enc_id ) |
| 840 | 839 | ), |
| 841 | 840 | TP_fast_assign( |
| 842 | 841 | __entry->id = id; |
| 843 | − __entry->type = type; | |
| 844 | 842 | __entry->enc_id = enc_id; |
| 845 | 843 | ), |
| 846 | − TP_printk("id:%d type:%d enc_id:%u", __entry->id, __entry->type, | |
| 847 | − __entry->enc_id) | |
| 844 | + TP_printk("id:%d enc_id:%u", __entry->id, __entry->enc_id) | |
| 848 | 845 | ); |
| 849 | 846 | DEFINE_EVENT(dpu_rm_iter_template, dpu_rm_reserve_intf, |
| 850 | − TP_PROTO(uint32_t id, enum dpu_hw_blk_type type, uint32_t enc_id), | |
| 851 | − TP_ARGS(id, type, enc_id) | |
| 847 | + TP_PROTO(uint32_t id, uint32_t enc_id), | |
| 848 | + TP_ARGS(id, enc_id) | |
| 852 | 849 | ); |
| 853 | 850 | DEFINE_EVENT(dpu_rm_iter_template, dpu_rm_reserve_ctls, |
| 854 | − TP_PROTO(uint32_t id, enum dpu_hw_blk_type type, uint32_t enc_id), | |
| 855 | − TP_ARGS(id, type, enc_id) | |
| 851 | + TP_PROTO(uint32_t id, uint32_t enc_id), | |
| 852 | + TP_ARGS(id, enc_id) | |
| 856 | 853 | ); |
| 857 | 854 | |
| 858 | 855 | TRACE_EVENT(dpu_rm_reserve_lms, |
| 859 | − TP_PROTO(uint32_t id, enum dpu_hw_blk_type type, uint32_t enc_id, | |
| 860 | − uint32_t pp_id), | |
| 861 | − TP_ARGS(id, type, enc_id, pp_id), | |
| 856 | + TP_PROTO(uint32_t id, uint32_t enc_id, uint32_t pp_id), | |
| 857 | + TP_ARGS(id, enc_id, pp_id), | |
| 862 | 858 | TP_STRUCT__entry( |
| 863 | 859 | __field( uint32_t, id ) |
| 864 | − __field( enum dpu_hw_blk_type, type ) | |
| 865 | 860 | __field( uint32_t, enc_id ) |
| 866 | 861 | __field( uint32_t, pp_id ) |
| 867 | 862 | ), |
| 868 | 863 | TP_fast_assign( |
| 869 | 864 | __entry->id = id; |
| 870 | − __entry->type = type; | |
| 871 | 865 | __entry->enc_id = enc_id; |
| 872 | 866 | __entry->pp_id = pp_id; |
| 873 | 867 | ), |
| 874 | − TP_printk("id:%d type:%d enc_id:%u pp_id:%u", __entry->id, | |
| 875 | − __entry->type, __entry->enc_id, __entry->pp_id) | |
| 868 | + TP_printk("id:%d enc_id:%u pp_id:%u", __entry->id, | |
| 869 | + __entry->enc_id, __entry->pp_id) | |
| 876 | 870 | ); |
| 877 | 871 | |
| 878 | 872 | TRACE_EVENT(dpu_vbif_wait_xin_halt_fail, |
| @@ -207,62 +207,44 @@ u32 msm_readl(const void __iomem *addr) · 206 unmodified lines | ||
| 207 | 207 | return val; |
| 208 | 208 | } |
| 209 | 209 | |
| 210 | −struct vblank_event { | |
| 211 | − struct list_head node; | |
| 210 | +struct msm_vblank_work { | |
| 211 | + struct work_struct work; | |
| 212 | 212 | int crtc_id; |
| 213 | 213 | bool enable; |
| 214 | + struct msm_drm_private *priv; | |
| 214 | 215 | }; |
| 215 | 216 | |
| 216 | −static void vblank_ctrl_worker(struct kthread_work *work) | |
| 217 | +static void vblank_ctrl_worker(struct work_struct *work) | |
| 217 | 218 | { |
| 218 | − struct msm_vblank_ctrl *vbl_ctrl = container_of(work, | |
| 219 | − struct msm_vblank_ctrl, work); | |
| 220 | − struct msm_drm_private *priv = container_of(vbl_ctrl, | |
| 221 | − struct msm_drm_private, vblank_ctrl); | |
| 219 | + struct msm_vblank_work *vbl_work = container_of(work, | |
| 220 | + struct msm_vblank_work, work); | |
| 221 | + struct msm_drm_private *priv = vbl_work->priv; | |
| 222 | 222 | struct msm_kms *kms = priv->kms; |
| 223 | − struct vblank_event *vbl_ev, *tmp; | |
| 224 | − unsigned long flags; | |
| 225 | − | |
| 226 | − spin_lock_irqsave(&vbl_ctrl->lock, flags); | |
| 227 | − list_for_each_entry_safe(vbl_ev, tmp, &vbl_ctrl->event_list, node) { | |
| 228 | − list_del(&vbl_ev->node); | |
| 229 | − spin_unlock_irqrestore(&vbl_ctrl->lock, flags); | |
| 230 | − | |
| 231 | − if (vbl_ev->enable) | |
| 232 | − kms->funcs->enable_vblank(kms, | |
| 233 | − priv->crtcs[vbl_ev->crtc_id]); | |
| 234 | − else | |
| 235 | − kms->funcs->disable_vblank(kms, | |
| 236 | − priv->crtcs[vbl_ev->crtc_id]); | |
| 237 | − | |
| 238 | − kfree(vbl_ev); | |
| 239 | 223 | |
| 240 | − spin_lock_irqsave(&vbl_ctrl->lock, flags); | |
| 241 | − } | |
| 224 | + if (vbl_work->enable) | |
| 225 | + kms->funcs->enable_vblank(kms, priv->crtcs[vbl_work->crtc_id]); | |
| 226 | + else | |
| 227 | + kms->funcs->disable_vblank(kms, priv->crtcs[vbl_work->crtc_id]); | |
| 242 | 228 | |
| 243 | − spin_unlock_irqrestore(&vbl_ctrl->lock, flags); | |
| 229 | + kfree(vbl_work); | |
| 244 | 230 | } |
| 245 | 231 | |
| 246 | 232 | static int vblank_ctrl_queue_work(struct msm_drm_private *priv, |
| 247 | 233 | int crtc_id, bool enable) |
| 248 | 234 | { |
| 249 | − struct msm_vblank_ctrl *vbl_ctrl = &priv->vblank_ctrl; | |
| 250 | − struct vblank_event *vbl_ev; | |
| 251 | − unsigned long flags; | |
| 235 | + struct msm_vblank_work *vbl_work; | |
| 252 | 236 | |
| 253 | − vbl_ev = kzalloc(sizeof(*vbl_ev), GFP_ATOMIC); | |
| 254 | − if (!vbl_ev) | |
| 237 | + vbl_work = kzalloc(sizeof(*vbl_work), GFP_ATOMIC); | |
| 238 | + if (!vbl_work) | |
| 255 | 239 | return -ENOMEM; |
| 256 | 240 | |
| 257 | − vbl_ev->crtc_id = crtc_id; | |
| 258 | − vbl_ev->enable = enable; | |
| 241 | + INIT_WORK(&vbl_work->work, vblank_ctrl_worker); | |
| 259 | 242 | |
| 260 | − spin_lock_irqsave(&vbl_ctrl->lock, flags); | |
| 261 | − list_add_tail(&vbl_ev->node, &vbl_ctrl->event_list); | |
| 262 | − spin_unlock_irqrestore(&vbl_ctrl->lock, flags); | |
| 243 | + vbl_work->crtc_id = crtc_id; | |
| 244 | + vbl_work->enable = enable; | |
| 245 | + vbl_work->priv = priv; | |
| 263 | 246 | |
| 264 | − kthread_queue_work(&priv->disp_thread[crtc_id].worker, | |
| 265 | − &vbl_ctrl->work); | |
| 247 | + queue_work(priv->wq, &vbl_work->work); | |
| 266 | 248 | |
| 267 | 249 | return 0; |
| 268 | 250 | } |
| @@ -274,31 +256,20 @@ static int msm_drm_uninit(struct device *dev) · 5 unmodified lines | ||
| 274 | 256 | struct msm_drm_private *priv = ddev->dev_private; |
| 275 | 257 | struct msm_kms *kms = priv->kms; |
| 276 | 258 | struct msm_mdss *mdss = priv->mdss; |
| 277 | − struct msm_vblank_ctrl *vbl_ctrl = &priv->vblank_ctrl; | |
| 278 | − struct vblank_event *vbl_ev, *tmp; | |
| 279 | 259 | int i; |
| 280 | 260 | |
| 281 | 261 | /* We must cancel and cleanup any pending vblank enable/disable |
| 282 | 262 | * work before drm_irq_uninstall() to avoid work re-enabling an |
| 283 | 263 | * irq after uninstall has disabled it. |
| 284 | 264 | */ |
| 285 | − kthread_flush_work(&vbl_ctrl->work); | |
| 286 | − list_for_each_entry_safe(vbl_ev, tmp, &vbl_ctrl->event_list, node) { | |
| 287 | − list_del(&vbl_ev->node); | |
| 288 | − kfree(vbl_ev); | |
| 289 | − } | |
| 290 | 265 | |
| 291 | − /* clean up display commit/event worker threads */ | |
| 292 | − for (i = 0; i < priv->num_crtcs; i++) { | |
| 293 | − if (priv->disp_thread[i].thread) { | |
| 294 | − kthread_flush_worker(&priv->disp_thread[i].worker); | |
| 295 | − kthread_stop(priv->disp_thread[i].thread); | |
| 296 | − priv->disp_thread[i].thread = NULL; | |
| 297 | − } | |
| 266 | + flush_workqueue(priv->wq); | |
| 267 | + destroy_workqueue(priv->wq); | |
| 298 | 268 | |
| 269 | + /* clean up event worker threads */ | |
| 270 | + for (i = 0; i < priv->num_crtcs; i++) { | |
| 299 | 271 | if (priv->event_thread[i].thread) { |
| 300 | − kthread_flush_worker(&priv->event_thread[i].worker); | |
| 301 | − kthread_stop(priv->event_thread[i].thread); | |
| 272 | + kthread_destroy_worker(&priv->event_thread[i].worker); | |
| 302 | 273 | priv->event_thread[i].thread = NULL; |
| 303 | 274 | } |
| 304 | 275 | } |
| @@ -323,9 +294,6 @@ static int msm_drm_uninit(struct device *dev) · 18 unmodified lines | ||
| 323 | 294 | drm_irq_uninstall(ddev); |
| 324 | 295 | pm_runtime_put_sync(dev); |
| 325 | 296 | |
| 326 | − flush_workqueue(priv->wq); | |
| 327 | − destroy_workqueue(priv->wq); | |
| 328 | − | |
| 329 | 297 | if (kms && kms->funcs) |
| 330 | 298 | kms->funcs->destroy(kms); |
| 331 | 299 | |
| @@ -490,9 +458,6 @@ static int msm_drm_init(struct device *dev, struct drm_driver *drv) · 158 unmodified lines | ||
| 490 | 458 | priv->wq = alloc_ordered_workqueue("msm", 0); |
| 491 | 459 | |
| 492 | 460 | INIT_LIST_HEAD(&priv->inactive_list); |
| 493 | − INIT_LIST_HEAD(&priv->vblank_ctrl.event_list); | |
| 494 | − kthread_init_work(&priv->vblank_ctrl.work, vblank_ctrl_worker); | |
| 495 | − spin_lock_init(&priv->vblank_ctrl.lock); | |
| 496 | 461 | |
| 497 | 462 | drm_mode_config_init(ddev); |
| 498 | 463 | |
| @@ -554,27 +519,6 @@ static int msm_drm_init(struct device *dev, struct drm_driver *drv) · 55 unmodified lines | ||
| 554 | 519 | */ |
| 555 | 520 | param.sched_priority = 16; |
| 556 | 521 | for (i = 0; i < priv->num_crtcs; i++) { |
| 557 | − | |
| 558 | − /* initialize display thread */ | |
| 559 | − priv->disp_thread[i].crtc_id = priv->crtcs[i]->base.id; | |
| 560 | − kthread_init_worker(&priv->disp_thread[i].worker); | |
| 561 | − priv->disp_thread[i].dev = ddev; | |
| 562 | − priv->disp_thread[i].thread = | |
| 563 | − kthread_run(kthread_worker_fn, | |
| 564 | − &priv->disp_thread[i].worker, | |
| 565 | − "crtc_commit:%d", priv->disp_thread[i].crtc_id); | |
| 566 | − if (IS_ERR(priv->disp_thread[i].thread)) { | |
| 567 | − DRM_DEV_ERROR(dev, "failed to create crtc_commit kthread\n"); | |
| 568 | − priv->disp_thread[i].thread = NULL; | |
| 569 | − goto err_msm_uninit; | |
| 570 | − } | |
| 571 | − | |
| 572 | − ret = sched_setscheduler(priv->disp_thread[i].thread, | |
| 573 | − SCHED_FIFO, ¶m); | |
| 574 | − if (ret) | |
| 575 | − dev_warn(dev, "disp_thread set priority failed: %d\n", | |
| 576 | − ret); | |
| 577 | − | |
| 578 | 522 | /* initialize event thread */ |
| 579 | 523 | priv->event_thread[i].crtc_id = priv->crtcs[i]->base.id; |
| 580 | 524 | kthread_init_worker(&priv->event_thread[i].worker); |
| @@ -589,13 +533,6 @@ static int msm_drm_init(struct device *dev, struct drm_driver *drv) · 8 unmodified lines | ||
| 589 | 533 | goto err_msm_uninit; |
| 590 | 534 | } |
| 591 | 535 | |
| 592 | − /** | |
| 593 | − * event thread should also run at same priority as disp_thread | |
| 594 | − * because it is handling frame_done events. A lower priority | |
| 595 | − * event thread and higher priority disp_thread can causes | |
| 596 | − * frame_pending counters beyond 2. This can lead to commit | |
| 597 | − * failure at crtc commit level. | |
| 598 | − */ | |
| 599 | 536 | ret = sched_setscheduler(priv->event_thread[i].thread, |
| 600 | 537 | SCHED_FIFO, ¶m); |
| 601 | 538 | if (ret) |
| @@ -914,8 +851,12 @@ static int msm_ioctl_gem_info(struct drm_device *dev, void *data, · 312 unmodified lines | ||
| 914 | 851 | ret = -EINVAL; |
| 915 | 852 | break; |
| 916 | 853 | } |
| 917 | − ret = copy_from_user(msm_obj->name, | |
| 918 | − u64_to_user_ptr(args->value), args->len); | |
| 854 | + if (copy_from_user(msm_obj->name, u64_to_user_ptr(args->value), | |
| 855 | + args->len)) { | |
| 856 | + msm_obj->name[0] = '\0'; | |
| 857 | + ret = -EFAULT; | |
| 858 | + break; | |
| 859 | + } | |
| 919 | 860 | msm_obj->name[args->len] = '\0'; |
| 920 | 861 | for (i = 0; i < args->len; i++) { |
| 921 | 862 | if (!isprint(msm_obj->name[i])) { |
| @@ -931,8 +872,9 @@ static int msm_ioctl_gem_info(struct drm_device *dev, void *data, · 9 unmodified lines | ||
| 931 | 872 | } |
| 932 | 873 | args->len = strlen(msm_obj->name); |
| 933 | 874 | if (args->value) { |
| 934 | − ret = copy_to_user(u64_to_user_ptr(args->value), | |
| 935 | − msm_obj->name, args->len); | |
| 875 | + if (copy_to_user(u64_to_user_ptr(args->value), | |
| 876 | + msm_obj->name, args->len)) | |
| 877 | + ret = -EFAULT; | |
| 936 | 878 | } |
| 937 | 879 | break; |
| 938 | 880 | } |
| @@ -77,12 +77,6 @@ enum msm_mdp_plane_property { · 76 unmodified lines | ||
| 77 | 77 | PLANE_PROP_MAX_NUM |
| 78 | 78 | }; |
| 79 | 79 | |
| 80 | −struct msm_vblank_ctrl { | |
| 81 | − struct kthread_work work; | |
| 82 | − struct list_head event_list; | |
| 83 | − spinlock_t lock; | |
| 84 | −}; | |
| 85 | − | |
| 86 | 80 | #define MSM_GPU_MAX_RINGS 4 |
| 87 | 81 | #define MAX_H_TILES_PER_DISPLAY 2 |
| 88 | 82 | |
| @@ -126,7 +120,7 @@ struct msm_display_topology { · 37 unmodified lines | ||
| 126 | 120 | |
| 127 | 121 | /** |
| 128 | 122 | * struct msm_display_info - defines display properties |
| 129 | − * @intf_type: DRM_MODE_CONNECTOR_ display type | |
| 123 | + * @intf_type: DRM_MODE_ENCODER_ type | |
| 130 | 124 | * @capabilities: Bitmask of display flags |
| 131 | 125 | * @num_of_h_tiles: Number of horizontal tiles in case of split interface |
| 132 | 126 | * @h_tile_instance: Controller instance used per tile. Number of elements is |
| @@ -199,7 +193,6 @@ struct msm_drm_private { · 66 unmodified lines | ||
| 199 | 193 | unsigned int num_crtcs; |
| 200 | 194 | struct drm_crtc *crtcs[MAX_CRTCS]; |
| 201 | 195 | |
| 202 | − struct msm_drm_thread disp_thread[MAX_CRTCS]; | |
| 203 | 196 | struct msm_drm_thread event_thread[MAX_CRTCS]; |
| 204 | 197 | |
| 205 | 198 | unsigned int num_encoders; |
| @@ -228,7 +221,6 @@ struct msm_drm_private { · 22 unmodified lines | ||
| 228 | 221 | struct notifier_block vmap_notifier; |
| 229 | 222 | struct shrinker shrinker; |
| 230 | 223 | |
| 231 | − struct msm_vblank_ctrl vblank_ctrl; | |
| 232 | 224 | struct drm_atomic_state *pm_state; |
| 233 | 225 | }; |
| 234 | 226 | |
| @@ -30,6 +30,8 @@ nouveau-y += nouveau_vga.o · 29 unmodified lines | ||
| 30 | 30 | # DRM - memory management |
| 31 | 31 | nouveau-y += nouveau_bo.o |
| 32 | 32 | nouveau-y += nouveau_gem.o |
| 33 | +nouveau-$(CONFIG_DRM_NOUVEAU_SVM) += nouveau_svm.o | |
| 34 | +nouveau-$(CONFIG_DRM_NOUVEAU_SVM) += nouveau_dmem.o | |
| 33 | 35 | nouveau-y += nouveau_mem.o |
| 34 | 36 | nouveau-y += nouveau_prime.o |
| 35 | 37 | nouveau-y += nouveau_sgdma.o |
| @@ -71,3 +71,15 @@ config DRM_NOUVEAU_BACKLIGHT · 70 unmodified lines | ||
| 71 | 71 | help |
| 72 | 72 | Say Y here if you want to control the backlight of your display |
| 73 | 73 | (e.g. a laptop panel). |
| 74 | + | |
| 75 | +config DRM_NOUVEAU_SVM | |
| 76 | + bool "(EXPERIMENTAL) Enable SVM (Shared Virtual Memory) support" | |
| 77 | + depends on ARCH_HAS_HMM | |
| 78 | + depends on DRM_NOUVEAU | |
| 79 | + depends on STAGING | |
| 80 | + select HMM_MIRROR | |
| 81 | + select DEVICE_PRIVATE | |
| 82 | + default n | |
| 83 | + help | |
| 84 | + Say Y here if you want to enable experimental support for | |
| 85 | + Shared Virtual Memory (SVM). | |
| @@ -40,6 +40,7 @@ · 39 unmodified lines | ||
| 40 | 40 | #include "nvreg.h" |
| 41 | 41 | #include "nouveau_fbcon.h" |
| 42 | 42 | #include "disp.h" |
| 43 | +#include "nouveau_dma.h" | |
| 43 | 44 | |
| 44 | 45 | #include <subdev/bios/pll.h> |
| 45 | 46 | #include <subdev/clk.h> |
| @@ -1077,12 +1078,223 @@ nouveau_crtc_set_config(struct drm_mode_set *set, · 1031 unmodified lines | ||
| 1077 | 1078 | return ret; |
| 1078 | 1079 | } |
| 1079 | 1080 | |
| 1081 | +struct nv04_page_flip_state { | |
| 1082 | + struct list_head head; | |
| 1083 | + struct drm_pending_vblank_event *event; | |
| 1084 | + struct drm_crtc *crtc; | |
| 1085 | + int bpp, pitch; | |
| 1086 | + u64 offset; | |
| 1087 | +}; | |
| 1088 | + | |
| 1089 | +static int | |
| 1090 | +nv04_finish_page_flip(struct nouveau_channel *chan, | |
| 1091 | + struct nv04_page_flip_state *ps) | |
| 1092 | +{ | |
| 1093 | + struct nouveau_fence_chan *fctx = chan->fence; | |
| 1094 | + struct nouveau_drm *drm = chan->drm; | |
| 1095 | + struct drm_device *dev = drm->dev; | |
| 1096 | + struct nv04_page_flip_state *s; | |
| 1097 | + unsigned long flags; | |
| 1098 | + | |
| 1099 | + spin_lock_irqsave(&dev->event_lock, flags); | |
| 1100 | + | |
| 1101 | + if (list_empty(&fctx->flip)) { | |
| 1102 | + NV_ERROR(drm, "unexpected pageflip\n"); | |
| 1103 | + spin_unlock_irqrestore(&dev->event_lock, flags); | |
| 1104 | + return -EINVAL; | |
| 1105 | + } | |
| 1106 | + | |
| 1107 | + s = list_first_entry(&fctx->flip, struct nv04_page_flip_state, head); | |
| 1108 | + if (s->event) { | |
| 1109 | + drm_crtc_arm_vblank_event(s->crtc, s->event); | |
| 1110 | + } else { | |
| 1111 | + /* Give up ownership of vblank for page-flipped crtc */ | |
| 1112 | + drm_crtc_vblank_put(s->crtc); | |
| 1113 | + } | |
| 1114 | + | |
| 1115 | + list_del(&s->head); | |
| 1116 | + if (ps) | |
| 1117 | + *ps = *s; | |
| 1118 | + kfree(s); | |
| 1119 | + | |
| 1120 | + spin_unlock_irqrestore(&dev->event_lock, flags); | |
| 1121 | + return 0; | |
| 1122 | +} | |
| 1123 | + | |
| 1124 | +int | |
| 1125 | +nv04_flip_complete(struct nvif_notify *notify) | |
| 1126 | +{ | |
| 1127 | + struct nouveau_cli *cli = (void *)notify->object->client; | |
| 1128 | + struct nouveau_drm *drm = cli->drm; | |
| 1129 | + struct nouveau_channel *chan = drm->channel; | |
| 1130 | + struct nv04_page_flip_state state; | |
| 1131 | + | |
| 1132 | + if (!nv04_finish_page_flip(chan, &state)) { | |
| 1133 | + nv_set_crtc_base(drm->dev, drm_crtc_index(state.crtc), | |
| 1134 | + state.offset + state.crtc->y * | |
| 1135 | + state.pitch + state.crtc->x * | |
| 1136 | + state.bpp / 8); | |
| 1137 | + } | |
| 1138 | + | |
| 1139 | + return NVIF_NOTIFY_KEEP; | |
| 1140 | +} | |
| 1141 | + | |
| 1142 | +static int | |
| 1143 | +nv04_page_flip_emit(struct nouveau_channel *chan, | |
| 1144 | + struct nouveau_bo *old_bo, | |
| 1145 | + struct nouveau_bo *new_bo, | |
| 1146 | + struct nv04_page_flip_state *s, | |
| 1147 | + struct nouveau_fence **pfence) | |
| 1148 | +{ | |
| 1149 | + struct nouveau_fence_chan *fctx = chan->fence; | |
| 1150 | + struct nouveau_drm *drm = chan->drm; | |
| 1151 | + struct drm_device *dev = drm->dev; | |
| 1152 | + unsigned long flags; | |
| 1153 | + int ret; | |
| 1154 | + | |
| 1155 | + /* Queue it to the pending list */ | |
| 1156 | + spin_lock_irqsave(&dev->event_lock, flags); | |
| 1157 | + list_add_tail(&s->head, &fctx->flip); | |
| 1158 | + spin_unlock_irqrestore(&dev->event_lock, flags); | |
| 1159 | + | |
| 1160 | + /* Synchronize with the old framebuffer */ | |
| 1161 | + ret = nouveau_fence_sync(old_bo, chan, false, false); | |
| 1162 | + if (ret) | |
| 1163 | + goto fail; | |
| 1164 | + | |
| 1165 | + /* Emit the pageflip */ | |
| 1166 | + ret = RING_SPACE(chan, 2); | |
| 1167 | + if (ret) | |
| 1168 | + goto fail; | |
| 1169 | + | |
| 1170 | + BEGIN_NV04(chan, NvSubSw, NV_SW_PAGE_FLIP, 1); | |
| 1171 | + OUT_RING (chan, 0x00000000); | |
| 1172 | + FIRE_RING (chan); | |
| 1173 | + | |
| 1174 | + ret = nouveau_fence_new(chan, false, pfence); | |
| 1175 | + if (ret) | |
| 1176 | + goto fail; | |
| 1177 | + | |
| 1178 | + return 0; | |
| 1179 | +fail: | |
| 1180 | + spin_lock_irqsave(&dev->event_lock, flags); | |
| 1181 | + list_del(&s->head); | |
| 1182 | + spin_unlock_irqrestore(&dev->event_lock, flags); | |
| 1183 | + return ret; | |
| 1184 | +} | |
| 1185 | + | |
| 1186 | +static int | |
| 1187 | +nv04_crtc_page_flip(struct drm_crtc *crtc, struct drm_framebuffer *fb, | |
| 1188 | + struct drm_pending_vblank_event *event, u32 flags, | |
| 1189 | + struct drm_modeset_acquire_ctx *ctx) | |
| 1190 | +{ | |
| 1191 | + const int swap_interval = (flags & DRM_MODE_PAGE_FLIP_ASYNC) ? 0 : 1; | |
| 1192 | + struct drm_device *dev = crtc->dev; | |
| 1193 | + struct nouveau_drm *drm = nouveau_drm(dev); | |
| 1194 | + struct nouveau_bo *old_bo = nouveau_framebuffer(crtc->primary->fb)->nvbo; | |
| 1195 | + struct nouveau_bo *new_bo = nouveau_framebuffer(fb)->nvbo; | |
| 1196 | + struct nv04_page_flip_state *s; | |
| 1197 | + struct nouveau_channel *chan; | |
| 1198 | + struct nouveau_cli *cli; | |
| 1199 | + struct nouveau_fence *fence; | |
| 1200 | + struct nv04_display *dispnv04 = nv04_display(dev); | |
| 1201 | + int head = nouveau_crtc(crtc)->index; | |
| 1202 | + int ret; | |
| 1203 | + | |
| 1204 | + chan = drm->channel; | |
| 1205 | + if (!chan) | |
| 1206 | + return -ENODEV; | |
| 1207 | + cli = (void *)chan->user.client; | |
| 1208 | + | |
| 1209 | + s = kzalloc(sizeof(*s), GFP_KERNEL); | |
| 1210 | + if (!s) | |
| 1211 | + return -ENOMEM; | |
| 1212 | + | |
| 1213 | + if (new_bo != old_bo) { | |
| 1214 | + ret = nouveau_bo_pin(new_bo, TTM_PL_FLAG_VRAM, true); | |
| 1215 | + if (ret) | |
| 1216 | + goto fail_free; | |
| 1217 | + } | |
| 1218 | + | |
| 1219 | + mutex_lock(&cli->mutex); | |
| 1220 | + ret = ttm_bo_reserve(&new_bo->bo, true, false, NULL); | |
| 1221 | + if (ret) | |
| 1222 | + goto fail_unpin; | |
| 1223 | + | |
| 1224 | + /* synchronise rendering channel with the kernel's channel */ | |
| 1225 | + ret = nouveau_fence_sync(new_bo, chan, false, true); | |
| 1226 | + if (ret) { | |
| 1227 | + ttm_bo_unreserve(&new_bo->bo); | |
| 1228 | + goto fail_unpin; | |
| 1229 | + } | |
| 1230 | + | |
| 1231 | + if (new_bo != old_bo) { | |
| 1232 | + ttm_bo_unreserve(&new_bo->bo); | |
| 1233 | + | |
| 1234 | + ret = ttm_bo_reserve(&old_bo->bo, true, false, NULL); | |
| 1235 | + if (ret) | |
| 1236 | + goto fail_unpin; | |
| 1237 | + } | |
| 1238 | + | |
| 1239 | + /* Initialize a page flip struct */ | |
| 1240 | + *s = (struct nv04_page_flip_state) | |
| 1241 | + { { }, event, crtc, fb->format->cpp[0] * 8, fb->pitches[0], | |
| 1242 | + new_bo->bo.offset }; | |
| 1243 | + | |
| 1244 | + /* Keep vblanks on during flip, for the target crtc of this flip */ | |
| 1245 | + drm_crtc_vblank_get(crtc); | |
| 1246 | + | |
| 1247 | + /* Emit a page flip */ | |
| 1248 | + if (swap_interval) { | |
| 1249 | + ret = RING_SPACE(chan, 8); | |
| 1250 | + if (ret) | |
| 1251 | + goto fail_unreserve; | |
| 1252 | + | |
| 1253 | + BEGIN_NV04(chan, NvSubImageBlit, 0x012c, 1); | |
| 1254 | + OUT_RING (chan, 0); | |
| 1255 | + BEGIN_NV04(chan, NvSubImageBlit, 0x0134, 1); | |
| 1256 | + OUT_RING (chan, head); | |
| 1257 | + BEGIN_NV04(chan, NvSubImageBlit, 0x0100, 1); | |
| 1258 | + OUT_RING (chan, 0); | |
| 1259 | + BEGIN_NV04(chan, NvSubImageBlit, 0x0130, 1); | |
| 1260 | + OUT_RING (chan, 0); | |
| 1261 | + } | |
| 1262 | + | |
| 1263 | + nouveau_bo_ref(new_bo, &dispnv04->image[head]); | |
| 1264 | + | |
| 1265 | + ret = nv04_page_flip_emit(chan, old_bo, new_bo, s, &fence); | |
| 1266 | + if (ret) | |
| 1267 | + goto fail_unreserve; | |
| 1268 | + mutex_unlock(&cli->mutex); | |
| 1269 | + | |
| 1270 | + /* Update the crtc struct and cleanup */ | |
| 1271 | + crtc->primary->fb = fb; | |
| 1272 | + | |
| 1273 | + nouveau_bo_fence(old_bo, fence, false); | |
| 1274 | + ttm_bo_unreserve(&old_bo->bo); | |
| 1275 | + if (old_bo != new_bo) | |
| 1276 | + nouveau_bo_unpin(old_bo); | |
| 1277 | + nouveau_fence_unref(&fence); | |
| 1278 | + return 0; | |
| 1279 | + | |
| 1280 | +fail_unreserve: | |
| 1281 | + drm_crtc_vblank_put(crtc); | |
| 1282 | + ttm_bo_unreserve(&old_bo->bo); | |
| 1283 | +fail_unpin: | |
| 1284 | + mutex_unlock(&cli->mutex); | |
| 1285 | + if (old_bo != new_bo) | |
| 1286 | + nouveau_bo_unpin(new_bo); | |
| 1287 | +fail_free: | |
| 1288 | + kfree(s); | |
| 1289 | + return ret; | |
| 1290 | +} | |
| 1291 | + | |
| 1080 | 1292 | static const struct drm_crtc_funcs nv04_crtc_funcs = { |
| 1081 | 1293 | .cursor_set = nv04_crtc_cursor_set, |
| 1082 | 1294 | .cursor_move = nv04_crtc_cursor_move, |
| 1083 | 1295 | .gamma_set = nv_crtc_gamma_set, |
| 1084 | 1296 | .set_config = nouveau_crtc_set_config, |
| 1085 | − .page_flip = nouveau_crtc_page_flip, | |
| 1297 | + .page_flip = nv04_crtc_page_flip, | |
| 1086 | 1298 | .destroy = nv_crtc_destroy, |
| 1087 | 1299 | }; |
| 1088 | 1300 | |
| @@ -30,6 +30,160 @@ · 29 unmodified lines | ||
| 30 | 30 | #include "hw.h" |
| 31 | 31 | #include "nouveau_encoder.h" |
| 32 | 32 | #include "nouveau_connector.h" |
| 33 | +#include "nouveau_bo.h" | |
| 34 | + | |
| 35 | +#include <nvif/if0004.h> | |
| 36 | + | |
| 37 | +static void | |
| 38 | +nv04_display_fini(struct drm_device *dev, bool suspend) | |
| 39 | +{ | |
| 40 | + struct nv04_display *disp = nv04_display(dev); | |
| 41 | + struct drm_crtc *crtc; | |
| 42 | + | |
| 43 | + /* Disable flip completion events. */ | |
| 44 | + nvif_notify_put(&disp->flip); | |
| 45 | + | |
| 46 | + /* Disable vblank interrupts. */ | |
| 47 | + NVWriteCRTC(dev, 0, NV_PCRTC_INTR_EN_0, 0); | |
| 48 | + if (nv_two_heads(dev)) | |
| 49 | + NVWriteCRTC(dev, 1, NV_PCRTC_INTR_EN_0, 0); | |
| 50 | + | |
| 51 | + if (!suspend) | |
| 52 | + return; | |
| 53 | + | |
| 54 | + /* Un-pin FB and cursors so they'll be evicted to system memory. */ | |
| 55 | + list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | |
| 56 | + struct nouveau_framebuffer *nouveau_fb; | |
| 57 | + | |
| 58 | + nouveau_fb = nouveau_framebuffer(crtc->primary->fb); | |
| 59 | + if (!nouveau_fb || !nouveau_fb->nvbo) | |
| 60 | + continue; | |
| 61 | + | |
| 62 | + nouveau_bo_unpin(nouveau_fb->nvbo); | |
| 63 | + } | |
| 64 | + | |
| 65 | + list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | |
| 66 | + struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | |
| 67 | + if (nv_crtc->cursor.nvbo) { | |
| 68 | + if (nv_crtc->cursor.set_offset) | |
| 69 | + nouveau_bo_unmap(nv_crtc->cursor.nvbo); | |
| 70 | + nouveau_bo_unpin(nv_crtc->cursor.nvbo); | |
| 71 | + } | |
| 72 | + } | |
| 73 | +} | |
| 74 | + | |
| 75 | +static int | |
| 76 | +nv04_display_init(struct drm_device *dev, bool resume, bool runtime) | |
| 77 | +{ | |
| 78 | + struct nv04_display *disp = nv04_display(dev); | |
| 79 | + struct nouveau_drm *drm = nouveau_drm(dev); | |
| 80 | + struct nouveau_encoder *encoder; | |
| 81 | + struct drm_crtc *crtc; | |
| 82 | + int ret; | |
| 83 | + | |
| 84 | + /* meh.. modeset apparently doesn't setup all the regs and depends | |
| 85 | + * on pre-existing state, for now load the state of the card *before* | |
| 86 | + * nouveau was loaded, and then do a modeset. | |
| 87 | + * | |
| 88 | + * best thing to do probably is to make save/restore routines not | |
| 89 | + * save/restore "pre-load" state, but more general so we can save | |
| 90 | + * on suspend too. | |
| 91 | + */ | |
| 92 | + list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | |
| 93 | + struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | |
| 94 | + nv_crtc->save(&nv_crtc->base); | |
| 95 | + } | |
| 96 | + | |
| 97 | + list_for_each_entry(encoder, &dev->mode_config.encoder_list, base.base.head) | |
| 98 | + encoder->enc_save(&encoder->base.base); | |
| 99 | + | |
| 100 | + /* Enable flip completion events. */ | |
| 101 | + nvif_notify_get(&disp->flip); | |
| 102 | + | |
| 103 | + if (!resume) | |
| 104 | + return 0; | |
| 105 | + | |
| 106 | + /* Re-pin FB/cursors. */ | |
| 107 | + list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | |
| 108 | + struct nouveau_framebuffer *nouveau_fb; | |
| 109 | + | |
| 110 | + nouveau_fb = nouveau_framebuffer(crtc->primary->fb); | |
| 111 | + if (!nouveau_fb || !nouveau_fb->nvbo) | |
| 112 | + continue; | |
| 113 | + | |
| 114 | + ret = nouveau_bo_pin(nouveau_fb->nvbo, TTM_PL_FLAG_VRAM, true); | |
| 115 | + if (ret) | |
| 116 | + NV_ERROR(drm, "Could not pin framebuffer\n"); | |
| 117 | + } | |
| 118 | + | |
| 119 | + list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | |
| 120 | + struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | |
| 121 | + if (!nv_crtc->cursor.nvbo) | |
| 122 | + continue; | |
| 123 | + | |
| 124 | + ret = nouveau_bo_pin(nv_crtc->cursor.nvbo, TTM_PL_FLAG_VRAM, true); | |
| 125 | + if (!ret && nv_crtc->cursor.set_offset) | |
| 126 | + ret = nouveau_bo_map(nv_crtc->cursor.nvbo); | |
| 127 | + if (ret) | |
| 128 | + NV_ERROR(drm, "Could not pin/map cursor.\n"); | |
| 129 | + } | |
| 130 | + | |
| 131 | + /* Force CLUT to get re-loaded during modeset. */ | |
| 132 | + list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | |
| 133 | + struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | |
| 134 | + | |
| 135 | + nv_crtc->lut.depth = 0; | |
| 136 | + } | |
| 137 | + | |
| 138 | + /* This should ensure we don't hit a locking problem when someone | |
| 139 | + * wakes us up via a connector. We should never go into suspend | |
| 140 | + * while the display is on anyways. | |
| 141 | + */ | |
| 142 | + if (runtime) | |
| 143 | + return 0; | |
| 144 | + | |
| 145 | + /* Restore mode. */ | |
| 146 | + drm_helper_resume_force_mode(dev); | |
| 147 | + | |
| 148 | + list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | |
| 149 | + struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | |
| 150 | + | |
| 151 | + if (!nv_crtc->cursor.nvbo) | |
| 152 | + continue; | |
| 153 | + | |
| 154 | + if (nv_crtc->cursor.set_offset) | |
| 155 | + nv_crtc->cursor.set_offset(nv_crtc, nv_crtc->cursor.nvbo->bo.offset); | |
| 156 | + nv_crtc->cursor.set_pos(nv_crtc, nv_crtc->cursor_saved_x, | |
| 157 | + nv_crtc->cursor_saved_y); | |
| 158 | + } | |
| 159 | + | |
| 160 | + return 0; | |
| 161 | +} | |
| 162 | + | |
| 163 | +static void | |
| 164 | +nv04_display_destroy(struct drm_device *dev) | |
| 165 | +{ | |
| 166 | + struct nv04_display *disp = nv04_display(dev); | |
| 167 | + struct nouveau_drm *drm = nouveau_drm(dev); | |
| 168 | + struct nouveau_encoder *encoder; | |
| 169 | + struct nouveau_crtc *nv_crtc; | |
| 170 | + | |
| 171 | + /* Restore state */ | |
| 172 | + list_for_each_entry(encoder, &dev->mode_config.encoder_list, base.base.head) | |
| 173 | + encoder->enc_restore(&encoder->base.base); | |
| 174 | + | |
| 175 | + list_for_each_entry(nv_crtc, &dev->mode_config.crtc_list, base.head) | |
| 176 | + nv_crtc->restore(&nv_crtc->base); | |
| 177 | + | |
| 178 | + nouveau_hw_save_vga_fonts(dev, 0); | |
| 179 | + | |
| 180 | + nvif_notify_fini(&disp->flip); | |
| 181 | + | |
| 182 | + nouveau_display(dev)->priv = NULL; | |
| 183 | + kfree(disp); | |
| 184 | + | |
| 185 | + nvif_object_unmap(&drm->client.device.object); | |
| 186 | +} | |
| 33 | 187 | |
| 34 | 188 | int |
| 35 | 189 | nv04_display_create(struct drm_device *dev) |
| @@ -58,6 +212,13 @@ nv04_display_create(struct drm_device *dev) · 22 unmodified lines | ||
| 58 | 212 | /* Pre-nv50 doesn't support atomic, so don't expose the ioctls */ |
| 59 | 213 | dev->driver->driver_features &= ~DRIVER_ATOMIC; |
| 60 | 214 | |
| 215 | + /* Request page flip completion event. */ | |
| 216 | + if (drm->nvsw.client) { | |
| 217 | + nvif_notify_init(&drm->nvsw, nv04_flip_complete, | |
| 218 | + false, NV04_NVSW_NTFY_UEVENT, | |
| 219 | + NULL, 0, 0, &disp->flip); | |
| 220 | + } | |
| 221 | + | |
| 61 | 222 | nouveau_hw_save_vga_fonts(dev, 1); |
| 62 | 223 | |
| 63 | 224 | nv04_crtc_create(dev, 0); |
| @@ -121,58 +282,3 @@ nv04_display_create(struct drm_device *dev) · 57 unmodified lines | ||
| 121 | 282 | |
| 122 | 283 | return 0; |
| 123 | 284 | } |
| 124 | − | |
| 125 | −void | |
| 126 | −nv04_display_destroy(struct drm_device *dev) | |
| 127 | −{ | |
| 128 | − struct nv04_display *disp = nv04_display(dev); | |
| 129 | − struct nouveau_drm *drm = nouveau_drm(dev); | |
| 130 | − struct nouveau_encoder *encoder; | |
| 131 | − struct nouveau_crtc *nv_crtc; | |
| 132 | − | |
| 133 | − /* Restore state */ | |
| 134 | − list_for_each_entry(encoder, &dev->mode_config.encoder_list, base.base.head) | |
| 135 | − encoder->enc_restore(&encoder->base.base); | |
| 136 | − | |
| 137 | − list_for_each_entry(nv_crtc, &dev->mode_config.crtc_list, base.head) | |
| 138 | − nv_crtc->restore(&nv_crtc->base); | |
| 139 | − | |
| 140 | − nouveau_hw_save_vga_fonts(dev, 0); | |
| 141 | − | |
| 142 | − nouveau_display(dev)->priv = NULL; | |
| 143 | − kfree(disp); | |
| 144 | − | |
| 145 | − nvif_object_unmap(&drm->client.device.object); | |
| 146 | −} | |
| 147 | − | |
| 148 | −int | |
| 149 | −nv04_display_init(struct drm_device *dev) | |
| 150 | −{ | |
| 151 | − struct nouveau_encoder *encoder; | |
| 152 | − struct nouveau_crtc *crtc; | |
| 153 | − | |
| 154 | − /* meh.. modeset apparently doesn't setup all the regs and depends | |
| 155 | − * on pre-existing state, for now load the state of the card *before* | |
| 156 | − * nouveau was loaded, and then do a modeset. | |
| 157 | − * | |
| 158 | − * best thing to do probably is to make save/restore routines not | |
| 159 | − * save/restore "pre-load" state, but more general so we can save | |
| 160 | − * on suspend too. | |
| 161 | − */ | |
| 162 | − list_for_each_entry(crtc, &dev->mode_config.crtc_list, base.head) | |
| 163 | − crtc->save(&crtc->base); | |
| 164 | − | |
| 165 | − list_for_each_entry(encoder, &dev->mode_config.encoder_list, base.base.head) | |
| 166 | − encoder->enc_save(&encoder->base.base); | |
| 167 | − | |
| 168 | − return 0; | |
| 169 | −} | |
| 170 | − | |
| 171 | −void | |
| 172 | −nv04_display_fini(struct drm_device *dev) | |
| 173 | −{ | |
| 174 | − /* disable vblank interrupts */ | |
| 175 | − NVWriteCRTC(dev, 0, NV_PCRTC_INTR_EN_0, 0); | |
| 176 | − if (nv_two_heads(dev)) | |
| 177 | − NVWriteCRTC(dev, 1, NV_PCRTC_INTR_EN_0, 0); | |
| 178 | −} | |
| @@ -82,6 +82,7 @@ struct nv04_display { · 81 unmodified lines | ||
| 82 | 82 | uint32_t saved_vga_font[4][16384]; |
| 83 | 83 | uint32_t dac_users[4]; |
| 84 | 84 | struct nouveau_bo *image[2]; |
| 85 | + struct nvif_notify flip; | |
| 85 | 86 | }; |
| 86 | 87 | |
| 87 | 88 | static inline struct nv04_display * |
| @@ -92,9 +93,6 @@ nv04_display(struct drm_device *dev) · 4 unmodified lines | ||
| 92 | 93 | |
| 93 | 94 | /* nv04_display.c */ |
| 94 | 95 | int nv04_display_create(struct drm_device *); |
| 95 | −void nv04_display_destroy(struct drm_device *); | |
| 96 | −int nv04_display_init(struct drm_device *); | |
| 97 | −void nv04_display_fini(struct drm_device *); | |
| 98 | 96 | |
| 99 | 97 | /* nv04_crtc.c */ |
| 100 | 98 | int nv04_crtc_create(struct drm_device *, int index); |
| @@ -176,4 +174,5 @@ nouveau_bios_run_init_table(struct drm_device *dev, u16 table, · 75 unmodified lines | ||
| 176 | 174 | ); |
| 177 | 175 | } |
| 178 | 176 | |
| 177 | +int nv04_flip_complete(struct nvif_notify *); | |
| 179 | 178 | #endif |
| @@ -42,7 +42,7 @@ nv50_core_new(struct nouveau_drm *drm, struct nv50_core **pcore) · 41 unmodified lines | ||
| 42 | 42 | int version; |
| 43 | 43 | int (*new)(struct nouveau_drm *, s32, struct nv50_core **); |
| 44 | 44 | } cores[] = { |
| 45 | − { TU104_DISP_CORE_CHANNEL_DMA, 0, corec57d_new }, | |
| 45 | + { TU102_DISP_CORE_CHANNEL_DMA, 0, corec57d_new }, | |
| 46 | 46 | { GV100_DISP_CORE_CHANNEL_DMA, 0, corec37d_new }, |
| 47 | 47 | { GP102_DISP_CORE_CHANNEL_DMA, 0, core917d_new }, |
| 48 | 48 | { GP100_DISP_CORE_CHANNEL_DMA, 0, core917d_new }, |
| @@ -31,7 +31,7 @@ nv50_curs_new(struct nouveau_drm *drm, int head, struct nv50_wndw **pwndw) · 30 unmodified lines | ||
| 31 | 31 | int version; |
| 32 | 32 | int (*new)(struct nouveau_drm *, int, s32, struct nv50_wndw **); |
| 33 | 33 | } curses[] = { |
| 34 | − { TU104_DISP_CURSOR, 0, cursc37a_new }, | |
| 34 | + { TU102_DISP_CURSOR, 0, cursc37a_new }, | |
| 35 | 35 | { GV100_DISP_CURSOR, 0, cursc37a_new }, |
| 36 | 36 | { GK104_DISP_CURSOR, 0, curs907a_new }, |
| 37 | 37 | { GF110_DISP_CURSOR, 0, curs907a_new }, |
| @@ -817,7 +817,8 @@ nv50_msto_enable(struct drm_encoder *encoder) · 816 unmodified lines | ||
| 817 | 817 | |
| 818 | 818 | r = drm_dp_mst_allocate_vcpi(&mstm->mgr, mstc->port, armh->dp.pbn, |
| 819 | 819 | armh->dp.tu); |
| 820 | − WARN_ON(!r); | |
| 820 | + if (!r) | |
| 821 | + DRM_DEBUG_KMS("Failed to allocate VCPI\n"); | |
| 821 | 822 | |
| 822 | 823 | if (!mstm->links++) |
| 823 | 824 | nv50_outp_acquire(mstm->outp); |
| @@ -2220,8 +2221,8 @@ nv50_disp_func = { · 1396 unmodified lines | ||
| 2220 | 2221 | * Init |
| 2221 | 2222 | *****************************************************************************/ |
| 2222 | 2223 | |
| 2223 | −void | |
| 2224 | −nv50_display_fini(struct drm_device *dev) | |
| 2224 | +static void | |
| 2225 | +nv50_display_fini(struct drm_device *dev, bool suspend) | |
| 2225 | 2226 | { |
| 2226 | 2227 | struct nouveau_encoder *nv_encoder; |
| 2227 | 2228 | struct drm_encoder *encoder; |
| @@ -2242,8 +2243,8 @@ nv50_display_fini(struct drm_device *dev) · 14 unmodified lines | ||
| 2242 | 2243 | } |
| 2243 | 2244 | } |
| 2244 | 2245 | |
| 2245 | −int | |
| 2246 | −nv50_display_init(struct drm_device *dev) | |
| 2246 | +static int | |
| 2247 | +nv50_display_init(struct drm_device *dev, bool resume, bool runtime) | |
| 2247 | 2248 | { |
| 2248 | 2249 | struct nv50_core *core = nv50_disp(dev)->core; |
| 2249 | 2250 | struct drm_encoder *encoder; |
| @@ -2269,7 +2270,7 @@ nv50_display_init(struct drm_device *dev) · 19 unmodified lines | ||
| 2269 | 2270 | return 0; |
| 2270 | 2271 | } |
| 2271 | 2272 | |
| 2272 | −void | |
| 2273 | +static void | |
| 2273 | 2274 | nv50_display_destroy(struct drm_device *dev) |
| 2274 | 2275 | { |
| 2275 | 2276 | struct nv50_disp *disp = nv50_disp(dev); |
| @@ -31,7 +31,7 @@ nv50_wimm_init(struct nouveau_drm *drm, struct nv50_wndw *wndw) · 30 unmodified lines | ||
| 31 | 31 | int version; |
| 32 | 32 | int (*init)(struct nouveau_drm *, s32, struct nv50_wndw *); |
| 33 | 33 | } wimms[] = { |
| 34 | − { TU104_DISP_WINDOW_IMM_CHANNEL_DMA, 0, wimmc37b_init }, | |
| 34 | + { TU102_DISP_WINDOW_IMM_CHANNEL_DMA, 0, wimmc37b_init }, | |
| 35 | 35 | { GV100_DISP_WINDOW_IMM_CHANNEL_DMA, 0, wimmc37b_init }, |
| 36 | 36 | {} |
| 37 | 37 | }; |
| @@ -626,7 +626,7 @@ nv50_wndw_new(struct nouveau_drm *drm, enum drm_plane_type type, int index, · 625 unmodified lines | ||
| 626 | 626 | int (*new)(struct nouveau_drm *, enum drm_plane_type, |
| 627 | 627 | int, s32, struct nv50_wndw **); |
| 628 | 628 | } wndws[] = { |
| 629 | − { TU104_DISP_WINDOW_CHANNEL_DMA, 0, wndwc57e_new }, | |
| 629 | + { TU102_DISP_WINDOW_CHANNEL_DMA, 0, wndwc57e_new }, | |
| 630 | 630 | { GV100_DISP_WINDOW_CHANNEL_DMA, 0, wndwc37e_new }, |
| 631 | 631 | {} |
| 632 | 632 | }; |
| @@ -54,6 +54,9 @@ · 53 unmodified lines | ||
| 54 | 54 | |
| 55 | 55 | #define VOLTA_USERMODE_A 0x0000c361 |
| 56 | 56 | |
| 57 | +#define MAXWELL_FAULT_BUFFER_A /* clb069.h */ 0x0000b069 | |
| 58 | +#define VOLTA_FAULT_BUFFER_A /* clb069.h */ 0x0000c369 | |
| 59 | + | |
| 57 | 60 | #define NV03_CHANNEL_DMA /* cl506b.h */ 0x0000006b |
| 58 | 61 | #define NV10_CHANNEL_DMA /* cl506b.h */ 0x0000006e |
| 59 | 62 | #define NV17_CHANNEL_DMA /* cl506b.h */ 0x0000176e |
| @@ -84,7 +87,7 @@ · 24 unmodified lines | ||
| 84 | 87 | #define GP100_DISP /* cl5070.h */ 0x00009770 |
| 85 | 88 | #define GP102_DISP /* cl5070.h */ 0x00009870 |
| 86 | 89 | #define GV100_DISP /* cl5070.h */ 0x0000c370 |
| 87 | −#define TU104_DISP /* cl5070.h */ 0x0000c570 | |
| 90 | +#define TU102_DISP /* cl5070.h */ 0x0000c570 | |
| 88 | 91 | |
| 89 | 92 | #define NV31_MPEG 0x00003174 |
| 90 | 93 | #define G82_MPEG 0x00008274 |
| @@ -97,7 +100,7 @@ · 6 unmodified lines | ||
| 97 | 100 | #define GF110_DISP_CURSOR /* cl507a.h */ 0x0000907a |
| 98 | 101 | #define GK104_DISP_CURSOR /* cl507a.h */ 0x0000917a |
| 99 | 102 | #define GV100_DISP_CURSOR /* cl507a.h */ 0x0000c37a |
| 100 | −#define TU104_DISP_CURSOR /* cl507a.h */ 0x0000c57a | |
| 103 | +#define TU102_DISP_CURSOR /* cl507a.h */ 0x0000c57a | |
| 101 | 104 | |
| 102 | 105 | #define NV50_DISP_OVERLAY /* cl507b.h */ 0x0000507b |
| 103 | 106 | #define G82_DISP_OVERLAY /* cl507b.h */ 0x0000827b |
| @@ -106,7 +109,7 @@ · 2 unmodified lines | ||
| 106 | 109 | #define GK104_DISP_OVERLAY /* cl507b.h */ 0x0000917b |
| 107 | 110 | |
| 108 | 111 | #define GV100_DISP_WINDOW_IMM_CHANNEL_DMA /* clc37b.h */ 0x0000c37b |
| 109 | −#define TU104_DISP_WINDOW_IMM_CHANNEL_DMA /* clc37b.h */ 0x0000c57b | |
| 112 | +#define TU102_DISP_WINDOW_IMM_CHANNEL_DMA /* clc37b.h */ 0x0000c57b | |
| 110 | 113 | |
| 111 | 114 | #define NV50_DISP_BASE_CHANNEL_DMA /* cl507c.h */ 0x0000507c |
| 112 | 115 | #define G82_DISP_BASE_CHANNEL_DMA /* cl507c.h */ 0x0000827c |
| @@ -129,7 +132,7 @@ · 16 unmodified lines | ||
| 129 | 132 | #define GP100_DISP_CORE_CHANNEL_DMA /* cl507d.h */ 0x0000977d |
| 130 | 133 | #define GP102_DISP_CORE_CHANNEL_DMA /* cl507d.h */ 0x0000987d |
| 131 | 134 | #define GV100_DISP_CORE_CHANNEL_DMA /* cl507d.h */ 0x0000c37d |
| 132 | −#define TU104_DISP_CORE_CHANNEL_DMA /* cl507d.h */ 0x0000c57d | |
| 135 | +#define TU102_DISP_CORE_CHANNEL_DMA /* cl507d.h */ 0x0000c57d | |
| 133 | 136 | |
| 134 | 137 | #define NV50_DISP_OVERLAY_CHANNEL_DMA /* cl507e.h */ 0x0000507e |
| 135 | 138 | #define G82_DISP_OVERLAY_CHANNEL_DMA /* cl507e.h */ 0x0000827e |
| @@ -139,7 +142,7 @@ · 3 unmodified lines | ||
| 139 | 142 | #define GK104_DISP_OVERLAY_CONTROL_DMA /* cl507e.h */ 0x0000917e |
| 140 | 143 | |
| 141 | 144 | #define GV100_DISP_WINDOW_CHANNEL_DMA /* clc37e.h */ 0x0000c37e |
| 142 | −#define TU104_DISP_WINDOW_CHANNEL_DMA /* clc37e.h */ 0x0000c57e | |
| 145 | +#define TU102_DISP_WINDOW_CHANNEL_DMA /* clc37e.h */ 0x0000c57e | |
| 143 | 146 | |
| 144 | 147 | #define NV50_TESLA 0x00005097 |
| 145 | 148 | #define G82_TESLA 0x00008297 |
| @@ -0,0 +1,12 @@ | ||
| 1 | +#ifndef __NVIF_CLB069_H__ | |
| 2 | +#define __NVIF_CLB069_H__ | |
| 3 | +struct nvif_clb069_v0 { | |
| 4 | + __u8 version; | |
| 5 | + __u8 pad01[3]; | |
| 6 | + __u32 entries; | |
| 7 | + __u32 get; | |
| 8 | + __u32 put; | |
| 9 | +}; | |
| 10 | + | |
| 11 | +#define NVB069_V0_NTFY_FAULT 0x00 | |
| 12 | +#endif | |
| @@ -3,7 +3,8 @@ · 2 unmodified lines | ||
| 3 | 3 | struct nvif_vmm_v0 { |
| 4 | 4 | __u8 version; |
| 5 | 5 | __u8 page_nr; |
| 6 | − __u8 pad02[6]; | |
| 6 | + __u8 managed; | |
| 7 | + __u8 pad03[5]; | |
| 7 | 8 | __u64 addr; |
| 8 | 9 | __u64 size; |
| 9 | 10 | __u8 data[]; |
| @@ -14,6 +15,9 @@ struct nvif_vmm_v0 { · 4 unmodified lines | ||
| 14 | 15 | #define NVIF_VMM_V0_PUT 0x02 |
| 15 | 16 | #define NVIF_VMM_V0_MAP 0x03 |
| 16 | 17 | #define NVIF_VMM_V0_UNMAP 0x04 |
| 18 | +#define NVIF_VMM_V0_PFNMAP 0x05 | |
| 19 | +#define NVIF_VMM_V0_PFNCLR 0x06 | |
| 20 | +#define NVIF_VMM_V0_MTHD(i) ((i) + 0x80) | |
| 17 | 21 | |
| 18 | 22 | struct nvif_vmm_page_v0 { |
| 19 | 23 | __u8 version; |
| @@ -61,4 +65,28 @@ struct nvif_vmm_unmap_v0 { · 41 unmodified lines | ||
| 61 | 65 | __u8 pad01[7]; |
| 62 | 66 | __u64 addr; |
| 63 | 67 | }; |
| 68 | + | |
| 69 | +struct nvif_vmm_pfnmap_v0 { | |
| 70 | + __u8 version; | |
| 71 | + __u8 page; | |
| 72 | + __u8 pad02[6]; | |
| 73 | + __u64 addr; | |
| 74 | + __u64 size; | |
| 75 | +#define NVIF_VMM_PFNMAP_V0_ADDR 0xfffffffffffff000ULL | |
| 76 | +#define NVIF_VMM_PFNMAP_V0_ADDR_SHIFT 12 | |
| 77 | +#define NVIF_VMM_PFNMAP_V0_APER 0x00000000000000f0ULL | |
| 78 | +#define NVIF_VMM_PFNMAP_V0_HOST 0x0000000000000000ULL | |
| 79 | +#define NVIF_VMM_PFNMAP_V0_VRAM 0x0000000000000010ULL | |
| 80 | +#define NVIF_VMM_PFNMAP_V0_W 0x0000000000000002ULL | |
| 81 | +#define NVIF_VMM_PFNMAP_V0_V 0x0000000000000001ULL | |
| 82 | +#define NVIF_VMM_PFNMAP_V0_NONE 0x0000000000000000ULL | |
| 83 | + __u64 phys[]; | |
| 84 | +}; | |
| 85 | + | |
| 86 | +struct nvif_vmm_pfnclr_v0 { | |
| 87 | + __u8 version; | |
| 88 | + __u8 pad01[7]; | |
| 89 | + __u64 addr; | |
| 90 | + __u64 size; | |
| 91 | +}; | |
| 64 | 92 | #endif |
| @@ -6,6 +6,12 @@ struct gp100_vmm_vn { · 5 unmodified lines | ||
| 6 | 6 | /* nvif_vmm_vX ... */ |
| 7 | 7 | }; |
| 8 | 8 | |
| 9 | +struct gp100_vmm_v0 { | |
| 10 | + /* nvif_vmm_vX ... */ | |
| 11 | + __u8 version; | |
| 12 | + __u8 fault_replay; | |
| 13 | +}; | |
| 14 | + | |
| 9 | 15 | struct gp100_vmm_map_vn { |
| 10 | 16 | /* nvif_vmm_map_vX ... */ |
| 11 | 17 | }; |
| @@ -18,4 +24,19 @@ struct gp100_vmm_map_v0 { · 6 unmodified lines | ||
| 18 | 24 | __u8 priv; |
| 19 | 25 | __u8 kind; |
| 20 | 26 | }; |
| 27 | + | |
| 28 | +#define GP100_VMM_VN_FAULT_REPLAY NVIF_VMM_V0_MTHD(0x00) | |
| 29 | +#define GP100_VMM_VN_FAULT_CANCEL NVIF_VMM_V0_MTHD(0x01) | |
| 30 | + | |
| 31 | +struct gp100_vmm_fault_replay_vn { | |
| 32 | +}; | |
| 33 | + | |
| 34 | +struct gp100_vmm_fault_cancel_v0 { | |
| 35 | + __u8 version; | |
| 36 | + __u8 hub; | |
| 37 | + __u8 gpc; | |
| 38 | + __u8 client; | |
| 39 | + __u8 pad04[4]; | |
| 40 | + __u64 inst; | |
| 41 | +}; | |
| 21 | 42 | #endif |
| @@ -30,8 +30,8 @@ struct nvif_vmm { · 29 unmodified lines | ||
| 30 | 30 | int page_nr; |
| 31 | 31 | }; |
| 32 | 32 | |
| 33 | −int nvif_vmm_init(struct nvif_mmu *, s32 oclass, u64 addr, u64 size, | |
| 34 | − void *argv, u32 argc, struct nvif_vmm *); | |
| 33 | +int nvif_vmm_init(struct nvif_mmu *, s32 oclass, bool managed, u64 addr, | |
| 34 | + u64 size, void *argv, u32 argc, struct nvif_vmm *); | |
| 35 | 35 | void nvif_vmm_fini(struct nvif_vmm *); |
| 36 | 36 | int nvif_vmm_get(struct nvif_vmm *, enum nvif_vmm_get, bool sparse, |
| 37 | 37 | u8 page, u8 align, u64 size, struct nvif_vma *); |
| @@ -28,6 +28,7 @@ enum nvkm_devidx { · 27 unmodified lines | ||
| 28 | 28 | NVKM_SUBDEV_ICCSENSE, |
| 29 | 29 | NVKM_SUBDEV_THERM, |
| 30 | 30 | NVKM_SUBDEV_CLK, |
| 31 | + NVKM_SUBDEV_GSP, | |
| 31 | 32 | NVKM_SUBDEV_SECBOOT, |
| 32 | 33 | |
| 33 | 34 | NVKM_ENGINE_BSP, |
| @@ -137,6 +138,7 @@ struct nvkm_device { · 103 unmodified lines | ||
| 137 | 138 | struct nvkm_fb *fb; |
| 138 | 139 | struct nvkm_fuse *fuse; |
| 139 | 140 | struct nvkm_gpio *gpio; |
| 141 | + struct nvkm_gsp *gsp; | |
| 140 | 142 | struct nvkm_i2c *i2c; |
| 141 | 143 | struct nvkm_subdev *ibus; |
| 142 | 144 | struct nvkm_iccsense *iccsense; |
| @@ -209,6 +211,7 @@ struct nvkm_device_chip { · 66 unmodified lines | ||
| 209 | 211 | int (*fb )(struct nvkm_device *, int idx, struct nvkm_fb **); |
| 210 | 212 | int (*fuse )(struct nvkm_device *, int idx, struct nvkm_fuse **); |
| 211 | 213 | int (*gpio )(struct nvkm_device *, int idx, struct nvkm_gpio **); |
| 214 | + int (*gsp )(struct nvkm_device *, int idx, struct nvkm_gsp **); | |
| 212 | 215 | int (*i2c )(struct nvkm_device *, int idx, struct nvkm_i2c **); |
| 213 | 216 | int (*ibus )(struct nvkm_device *, int idx, struct nvkm_subdev **); |
| 214 | 217 | int (*iccsense)(struct nvkm_device *, int idx, struct nvkm_iccsense **); |
| @@ -11,5 +11,5 @@ int gm200_ce_new(struct nvkm_device *, int, struct nvkm_engine **); · 10 unmodified lines | ||
| 11 | 11 | int gp100_ce_new(struct nvkm_device *, int, struct nvkm_engine **); |
| 12 | 12 | int gp102_ce_new(struct nvkm_device *, int, struct nvkm_engine **); |
| 13 | 13 | int gv100_ce_new(struct nvkm_device *, int, struct nvkm_engine **); |
| 14 | −int tu104_ce_new(struct nvkm_device *, int, struct nvkm_engine **); | |
| 14 | +int tu102_ce_new(struct nvkm_device *, int, struct nvkm_engine **); | |
| 15 | 15 | #endif |
| @@ -36,5 +36,5 @@ int gm200_disp_new(struct nvkm_device *, int, struct nvkm_disp **); · 35 unmodified lines | ||
| 36 | 36 | int gp100_disp_new(struct nvkm_device *, int, struct nvkm_disp **); |
| 37 | 37 | int gp102_disp_new(struct nvkm_device *, int, struct nvkm_disp **); |
| 38 | 38 | int gv100_disp_new(struct nvkm_device *, int, struct nvkm_disp **); |
| 39 | −int tu104_disp_new(struct nvkm_device *, int, struct nvkm_disp **); | |
| 39 | +int tu102_disp_new(struct nvkm_device *, int, struct nvkm_disp **); | |
| 40 | 40 | #endif |
| @@ -74,5 +74,5 @@ int gm20b_fifo_new(struct nvkm_device *, int, struct nvkm_fifo **); · 73 unmodified lines | ||
| 74 | 74 | int gp100_fifo_new(struct nvkm_device *, int, struct nvkm_fifo **); |
| 75 | 75 | int gp10b_fifo_new(struct nvkm_device *, int, struct nvkm_fifo **); |
| 76 | 76 | int gv100_fifo_new(struct nvkm_device *, int, struct nvkm_fifo **); |
| 77 | −int tu104_fifo_new(struct nvkm_device *, int, struct nvkm_fifo **); | |
| 77 | +int tu102_fifo_new(struct nvkm_device *, int, struct nvkm_fifo **); | |
| 78 | 78 | #endif |
| @@ -10,6 +10,9 @@ struct nvkm_gr { · 9 unmodified lines | ||
| 10 | 10 | |
| 11 | 11 | u64 nvkm_gr_units(struct nvkm_gr *); |
| 12 | 12 | int nvkm_gr_tlb_flush(struct nvkm_gr *); |
| 13 | +int nvkm_gr_ctxsw_pause(struct nvkm_device *); | |
| 14 | +int nvkm_gr_ctxsw_resume(struct nvkm_device *); | |
| 15 | +u32 nvkm_gr_ctxsw_inst(struct nvkm_device *); | |
| 13 | 16 | |
| 14 | 17 | int nv04_gr_new(struct nvkm_device *, int, struct nvkm_gr **); |
| 15 | 18 | int nv10_gr_new(struct nvkm_device *, int, struct nvkm_gr **); |
| @@ -6,6 +6,8 @@ · 5 unmodified lines | ||
| 6 | 6 | |
| 7 | 7 | struct nvkm_nvdec { |
| 8 | 8 | struct nvkm_engine engine; |
| 9 | + u32 addr; | |
| 10 | + | |
| 9 | 11 | struct nvkm_falcon *falcon; |
| 10 | 12 | }; |
| 11 | 13 | |
| @@ -5,10 +5,13 @@ · 4 unmodified lines | ||
| 5 | 5 | |
| 6 | 6 | struct nvkm_sec2 { |
| 7 | 7 | struct nvkm_engine engine; |
| 8 | + u32 addr; | |
| 9 | + | |
| 8 | 10 | struct nvkm_falcon *falcon; |
| 9 | 11 | struct nvkm_msgqueue *queue; |
| 10 | 12 | struct work_struct work; |
| 11 | 13 | }; |
| 12 | 14 | |
| 13 | 15 | int gp102_sec2_new(struct nvkm_device *, int, struct nvkm_sec2 **); |
| 16 | +int tu102_sec2_new(struct nvkm_device *, int, struct nvkm_sec2 **); | |
| 14 | 17 | #endif |
| @@ -29,5 +29,5 @@ int gf100_bar_new(struct nvkm_device *, int, struct nvkm_bar **); · 28 unmodified lines | ||
| 29 | 29 | int gk20a_bar_new(struct nvkm_device *, int, struct nvkm_bar **); |
| 30 | 30 | int gm107_bar_new(struct nvkm_device *, int, struct nvkm_bar **); |
| 31 | 31 | int gm20b_bar_new(struct nvkm_device *, int, struct nvkm_bar **); |
| 32 | −int tu104_bar_new(struct nvkm_device *, int, struct nvkm_bar **); | |
| 32 | +int tu102_bar_new(struct nvkm_device *, int, struct nvkm_bar **); | |
| 33 | 33 | #endif |
| @@ -31,5 +31,5 @@ int gf100_devinit_new(struct nvkm_device *, int, struct nvkm_devinit **); · 30 unmodified lines | ||
| 31 | 31 | int gm107_devinit_new(struct nvkm_device *, int, struct nvkm_devinit **); |
| 32 | 32 | int gm200_devinit_new(struct nvkm_device *, int, struct nvkm_devinit **); |
| 33 | 33 | int gv100_devinit_new(struct nvkm_device *, int, struct nvkm_devinit **); |
| 34 | −int tu104_devinit_new(struct nvkm_device *, int, struct nvkm_devinit **); | |
| 34 | +int tu102_devinit_new(struct nvkm_device *, int, struct nvkm_devinit **); | |
| 35 | 35 | #endif |
| @@ -13,6 +13,8 @@ struct nvkm_fault { · 12 unmodified lines | ||
| 13 | 13 | struct nvkm_event event; |
| 14 | 14 | |
| 15 | 15 | struct nvkm_notify nrpfb; |
| 16 | + | |
| 17 | + struct nvkm_device_oclass user; | |
| 16 | 18 | }; |
| 17 | 19 | |
| 18 | 20 | struct nvkm_fault_data { |
| @@ -30,5 +32,5 @@ struct nvkm_fault_data { · 11 unmodified lines | ||
| 30 | 32 | |
| 31 | 33 | int gp100_fault_new(struct nvkm_device *, int, struct nvkm_fault **); |
| 32 | 34 | int gv100_fault_new(struct nvkm_device *, int, struct nvkm_fault **); |
| 33 | −int tu104_fault_new(struct nvkm_device *, int, struct nvkm_fault **); | |
| 35 | +int tu102_fault_new(struct nvkm_device *, int, struct nvkm_fault **); | |
| 34 | 36 | #endif |
| @@ -0,0 +1,14 @@ | ||
| 1 | +#ifndef __NVKM_GSP_H__ | |
| 2 | +#define __NVKM_GSP_H__ | |
| 3 | +#define nvkm_gsp(p) container_of((p), struct nvkm_gsp, subdev) | |
| 4 | +#include <core/subdev.h> | |
| 5 | + | |
| 6 | +struct nvkm_gsp { | |
| 7 | + struct nvkm_subdev subdev; | |
| 8 | + u32 addr; | |
| 9 | + | |
| 10 | + struct nvkm_falcon *falcon; | |
| 11 | +}; | |
| 12 | + | |
| 13 | +int gv100_gsp_new(struct nvkm_device *, int, struct nvkm_gsp **); | |
| 14 | +#endif | |
| @@ -31,5 +31,5 @@ int gk104_mc_new(struct nvkm_device *, int, struct nvkm_mc **); · 30 unmodified lines | ||
| 31 | 31 | int gk20a_mc_new(struct nvkm_device *, int, struct nvkm_mc **); |
| 32 | 32 | int gp100_mc_new(struct nvkm_device *, int, struct nvkm_mc **); |
| 33 | 33 | int gp10b_mc_new(struct nvkm_device *, int, struct nvkm_mc **); |
| 34 | −int tu104_mc_new(struct nvkm_device *, int, struct nvkm_mc **); | |
| 34 | +int tu102_mc_new(struct nvkm_device *, int, struct nvkm_mc **); | |
| 35 | 35 | #endif |
| @@ -17,6 +17,7 @@ struct nvkm_vma { · 16 unmodified lines | ||
| 17 | 17 | bool part:1; /* Region was split from an allocated region by map(). */ |
| 18 | 18 | bool user:1; /* Region user-allocated. */ |
| 19 | 19 | bool busy:1; /* Region busy (for temporarily preventing user access). */ |
| 20 | + bool mapped:1; /* Region contains valid pages. */ | |
| 20 | 21 | struct nvkm_memory *memory; /* Memory currently mapped into VMA. */ |
| 21 | 22 | struct nvkm_tags *tags; /* Compression tag reference. */ |
| 22 | 23 | }; |
| @@ -44,6 +45,8 @@ struct nvkm_vmm { · 21 unmodified lines | ||
| 44 | 45 | |
| 45 | 46 | dma_addr_t null; |
| 46 | 47 | void *nullp; |
| 48 | + | |
| 49 | + bool replay; | |
| 47 | 50 | }; |
| 48 | 51 | |
| 49 | 52 | int nvkm_vmm_new(struct nvkm_device *, u64 addr, u64 size, void *argv, u32 argc, |
| @@ -63,6 +66,7 @@ struct nvkm_vmm_map { · 13 unmodified lines | ||
| 63 | 66 | struct nvkm_mm_node *mem; |
| 64 | 67 | struct scatterlist *sgl; |
| 65 | 68 | dma_addr_t *dma; |
| 69 | + u64 *pfn; | |
| 66 | 70 | u64 off; |
| 67 | 71 | |
| 68 | 72 | const struct nvkm_vmm_page *page; |
| @@ -130,5 +134,5 @@ int gm20b_mmu_new(struct nvkm_device *, int, struct nvkm_mmu **); · 61 unmodified lines | ||
| 130 | 134 | int gp100_mmu_new(struct nvkm_device *, int, struct nvkm_mmu **); |
| 131 | 135 | int gp10b_mmu_new(struct nvkm_device *, int, struct nvkm_mmu **); |
| 132 | 136 | int gv100_mmu_new(struct nvkm_device *, int, struct nvkm_mmu **); |
| 133 | −int tu104_mmu_new(struct nvkm_device *, int, struct nvkm_mmu **); | |
| 137 | +int tu102_mmu_new(struct nvkm_device *, int, struct nvkm_mmu **); | |
| 134 | 138 | #endif |
| @@ -9,6 +9,7 @@ struct nvkm_top { · 8 unmodified lines | ||
| 9 | 9 | struct list_head device; |
| 10 | 10 | }; |
| 11 | 11 | |
| 12 | +u32 nvkm_top_addr(struct nvkm_device *, enum nvkm_devidx); | |
| 12 | 13 | u32 nvkm_top_reset(struct nvkm_device *, enum nvkm_devidx); |
| 13 | 14 | u32 nvkm_top_intr(struct nvkm_device *, u32 intr, u64 *subdevs); |
| 14 | 15 | u32 nvkm_top_intr_mask(struct nvkm_device *, enum nvkm_devidx); |
| @@ -38,6 +38,7 @@ int nvkm_volt_set_id(struct nvkm_volt *, u8 id, u8 min_id, u8 temp, · 37 unmodified lines | ||
| 38 | 38 | |
| 39 | 39 | int nv40_volt_new(struct nvkm_device *, int, struct nvkm_volt **); |
| 40 | 40 | int gf100_volt_new(struct nvkm_device *, int, struct nvkm_volt **); |
| 41 | +int gf117_volt_new(struct nvkm_device *, int, struct nvkm_volt **); | |
| 41 | 42 | int gk104_volt_new(struct nvkm_device *, int, struct nvkm_volt **); |
| 42 | 43 | int gk20a_volt_new(struct nvkm_device *, int, struct nvkm_volt **); |
| 43 | 44 | int gm20b_volt_new(struct nvkm_device *, int, struct nvkm_volt **); |
| @@ -214,6 +214,7 @@ nouveau_abi16_ioctl_getparam(ABI16_IOCTL_ARGS) · 213 unmodified lines | ||
| 214 | 214 | WARN_ON(1); |
| 215 | 215 | break; |
| 216 | 216 | } |
| 217 | + break; | |
| 217 | 218 | case NOUVEAU_GETPARAM_FB_SIZE: |
| 218 | 219 | getparam->value = drm->gem.vram_available; |
| 219 | 220 | break; |
| @@ -338,7 +339,8 @@ nouveau_abi16_ioctl_channel_alloc(ABI16_IOCTL_ARGS) · 118 unmodified lines | ||
| 338 | 339 | goto done; |
| 339 | 340 | |
| 340 | 341 | if (device->info.family >= NV_DEVICE_INFO_V0_TESLA) { |
| 341 | − ret = nouveau_vma_new(chan->ntfy, &cli->vmm, &chan->ntfy_vma); | |
| 342 | + ret = nouveau_vma_new(chan->ntfy, chan->chan->vmm, | |
| 343 | + &chan->ntfy_vma); | |
| 342 | 344 | if (ret) |
| 343 | 345 | goto done; |
| 344 | 346 | } |
| @@ -194,7 +194,7 @@ nouveau_bo_new(struct nouveau_cli *cli, u64 size, int align, · 193 unmodified lines | ||
| 194 | 194 | struct nouveau_drm *drm = cli->drm; |
| 195 | 195 | struct nouveau_bo *nvbo; |
| 196 | 196 | struct nvif_mmu *mmu = &cli->mmu; |
| 197 | − struct nvif_vmm *vmm = &cli->vmm.vmm; | |
| 197 | + struct nvif_vmm *vmm = cli->svm.cli ? &cli->svm.vmm : &cli->vmm.vmm; | |
| 198 | 198 | size_t acc_size; |
| 199 | 199 | int type = ttm_bo_type_device; |
| 200 | 200 | int ret, i, pi = -1; |
| @@ -1434,7 +1434,7 @@ nouveau_ttm_io_mem_reserve(struct ttm_bo_device *bdev, struct ttm_mem_reg *reg) · 1233 unmodified lines | ||
| 1434 | 1434 | if (drm->client.mem->oclass < NVIF_CLASS_MEM_NV50 || !mem->kind) |
| 1435 | 1435 | /* untiled */ |
| 1436 | 1436 | break; |
| 1437 | − /* fallthrough, tiled memory */ | |
| 1437 | + /* fall through - tiled memory */ | |
| 1438 | 1438 | case TTM_PL_VRAM: |
| 1439 | 1439 | reg->bus.offset = reg->start << PAGE_SHIFT; |
| 1440 | 1440 | reg->bus.base = device->func->resource_addr(device, 1); |
| @@ -42,6 +42,7 @@ · 41 unmodified lines | ||
| 42 | 42 | #include "nouveau_fence.h" |
| 43 | 43 | #include "nouveau_abi16.h" |
| 44 | 44 | #include "nouveau_vmm.h" |
| 45 | +#include "nouveau_svm.h" | |
| 45 | 46 | |
| 46 | 47 | MODULE_PARM_DESC(vram_pushbuf, "Create DMA push buffers in VRAM"); |
| 47 | 48 | int nouveau_vram_pushbuf; |
| @@ -95,6 +96,10 @@ nouveau_channel_del(struct nouveau_channel **pchan) · 47 unmodified lines | ||
| 95 | 96 | |
| 96 | 97 | if (chan->fence) |
| 97 | 98 | nouveau_fence(chan->drm)->context_del(chan); |
| 99 | + | |
| 100 | + if (cli) | |
| 101 | + nouveau_svmm_part(chan->vmm->svmm, chan->inst); | |
| 102 | + | |
| 98 | 103 | nvif_object_fini(&chan->nvsw); |
| 99 | 104 | nvif_object_fini(&chan->gart); |
| 100 | 105 | nvif_object_fini(&chan->vram); |
| @@ -130,6 +135,7 @@ nouveau_channel_prep(struct nouveau_drm *drm, struct nvif_device *device, · 29 unmodified lines | ||
| 130 | 135 | |
| 131 | 136 | chan->device = device; |
| 132 | 137 | chan->drm = drm; |
| 138 | + chan->vmm = cli->svm.cli ? &cli->svm : &cli->vmm; | |
| 133 | 139 | atomic_set(&chan->killed, 0); |
| 134 | 140 | |
| 135 | 141 | /* allocate memory for dma push buffer */ |
| @@ -157,7 +163,7 @@ nouveau_channel_prep(struct nouveau_drm *drm, struct nvif_device *device, · 21 unmodified lines | ||
| 157 | 163 | chan->push.addr = chan->push.buffer->bo.offset; |
| 158 | 164 | |
| 159 | 165 | if (device->info.family >= NV_DEVICE_INFO_V0_TESLA) { |
| 160 | − ret = nouveau_vma_new(chan->push.buffer, &cli->vmm, | |
| 166 | + ret = nouveau_vma_new(chan->push.buffer, chan->vmm, | |
| 161 | 167 | &chan->push.vma); |
| 162 | 168 | if (ret) { |
| 163 | 169 | nouveau_channel_del(pchan); |
| @@ -172,7 +178,7 @@ nouveau_channel_prep(struct nouveau_drm *drm, struct nvif_device *device, · 8 unmodified lines | ||
| 172 | 178 | args.target = NV_DMA_V0_TARGET_VM; |
| 173 | 179 | args.access = NV_DMA_V0_ACCESS_VM; |
| 174 | 180 | args.start = 0; |
| 175 | − args.limit = cli->vmm.vmm.limit - 1; | |
| 181 | + args.limit = chan->vmm->vmm.limit - 1; | |
| 176 | 182 | } else |
| 177 | 183 | if (chan->push.buffer->bo.mem.mem_type == TTM_PL_VRAM) { |
| 178 | 184 | if (device->info.family == NV_DEVICE_INFO_V0_TNT) { |
| @@ -202,7 +208,7 @@ nouveau_channel_prep(struct nouveau_drm *drm, struct nvif_device *device, · 23 unmodified lines | ||
| 202 | 208 | args.target = NV_DMA_V0_TARGET_VM; |
| 203 | 209 | args.access = NV_DMA_V0_ACCESS_RDWR; |
| 204 | 210 | args.start = 0; |
| 205 | − args.limit = cli->vmm.vmm.limit - 1; | |
| 211 | + args.limit = chan->vmm->vmm.limit - 1; | |
| 206 | 212 | } |
| 207 | 213 | } |
| 208 | 214 | |
| @@ -220,7 +226,6 @@ static int · 11 unmodified lines | ||
| 220 | 226 | nouveau_channel_ind(struct nouveau_drm *drm, struct nvif_device *device, |
| 221 | 227 | u64 runlist, bool priv, struct nouveau_channel **pchan) |
| 222 | 228 | { |
| 223 | − struct nouveau_cli *cli = (void *)device->object.client; | |
| 224 | 229 | static const u16 oclasses[] = { TURING_CHANNEL_GPFIFO_A, |
| 225 | 230 | VOLTA_CHANNEL_GPFIFO_A, |
| 226 | 231 | PASCAL_CHANNEL_GPFIFO_A, |
| @@ -255,7 +260,7 @@ nouveau_channel_ind(struct nouveau_drm *drm, struct nvif_device *device, · 28 unmodified lines | ||
| 255 | 260 | args.volta.ilength = 0x02000; |
| 256 | 261 | args.volta.ioffset = 0x10000 + chan->push.addr; |
| 257 | 262 | args.volta.runlist = runlist; |
| 258 | − args.volta.vmm = nvif_handle(&cli->vmm.vmm.object); | |
| 263 | + args.volta.vmm = nvif_handle(&chan->vmm->vmm.object); | |
| 259 | 264 | args.volta.priv = priv; |
| 260 | 265 | size = sizeof(args.volta); |
| 261 | 266 | } else |
| @@ -264,7 +269,7 @@ nouveau_channel_ind(struct nouveau_drm *drm, struct nvif_device *device, · 2 unmodified lines | ||
| 264 | 269 | args.kepler.ilength = 0x02000; |
| 265 | 270 | args.kepler.ioffset = 0x10000 + chan->push.addr; |
| 266 | 271 | args.kepler.runlist = runlist; |
| 267 | − args.kepler.vmm = nvif_handle(&cli->vmm.vmm.object); | |
| 272 | + args.kepler.vmm = nvif_handle(&chan->vmm->vmm.object); | |
| 268 | 273 | args.kepler.priv = priv; |
| 269 | 274 | size = sizeof(args.kepler); |
| 270 | 275 | } else |
| @@ -272,14 +277,14 @@ nouveau_channel_ind(struct nouveau_drm *drm, struct nvif_device *device, · 1 unmodified line | ||
| 272 | 277 | args.fermi.version = 0; |
| 273 | 278 | args.fermi.ilength = 0x02000; |
| 274 | 279 | args.fermi.ioffset = 0x10000 + chan->push.addr; |
| 275 | − args.fermi.vmm = nvif_handle(&cli->vmm.vmm.object); | |
| 280 | + args.fermi.vmm = nvif_handle(&chan->vmm->vmm.object); | |
| 276 | 281 | size = sizeof(args.fermi); |
| 277 | 282 | } else { |
| 278 | 283 | args.nv50.version = 0; |
| 279 | 284 | args.nv50.ilength = 0x02000; |
| 280 | 285 | args.nv50.ioffset = 0x10000 + chan->push.addr; |
| 281 | 286 | args.nv50.pushbuf = nvif_handle(&chan->push.ctxdma); |
| 282 | − args.nv50.vmm = nvif_handle(&cli->vmm.vmm.object); | |
| 287 | + args.nv50.vmm = nvif_handle(&chan->vmm->vmm.object); | |
| 283 | 288 | size = sizeof(args.nv50); |
| 284 | 289 | } |
| 285 | 290 | |
| @@ -350,7 +355,6 @@ static int · 64 unmodified lines | ||
| 350 | 355 | nouveau_channel_init(struct nouveau_channel *chan, u32 vram, u32 gart) |
| 351 | 356 | { |
| 352 | 357 | struct nvif_device *device = chan->device; |
| 353 | − struct nouveau_cli *cli = (void *)chan->user.client; | |
| 354 | 358 | struct nouveau_drm *drm = chan->drm; |
| 355 | 359 | struct nv_dma_v0 args = {}; |
| 356 | 360 | int ret, i; |
| @@ -376,7 +380,7 @@ nouveau_channel_init(struct nouveau_channel *chan, u32 vram, u32 gart) · 19 unmodified lines | ||
| 376 | 380 | args.target = NV_DMA_V0_TARGET_VM; |
| 377 | 381 | args.access = NV_DMA_V0_ACCESS_VM; |
| 378 | 382 | args.start = 0; |
| 379 | − args.limit = cli->vmm.vmm.limit - 1; | |
| 383 | + args.limit = chan->vmm->vmm.limit - 1; | |
| 380 | 384 | } else { |
| 381 | 385 | args.target = NV_DMA_V0_TARGET_VRAM; |
| 382 | 386 | args.access = NV_DMA_V0_ACCESS_RDWR; |
| @@ -393,7 +397,7 @@ nouveau_channel_init(struct nouveau_channel *chan, u32 vram, u32 gart) · 10 unmodified lines | ||
| 393 | 397 | args.target = NV_DMA_V0_TARGET_VM; |
| 394 | 398 | args.access = NV_DMA_V0_ACCESS_VM; |
| 395 | 399 | args.start = 0; |
| 396 | − args.limit = cli->vmm.vmm.limit - 1; | |
| 400 | + args.limit = chan->vmm->vmm.limit - 1; | |
| 397 | 401 | } else |
| 398 | 402 | if (chan->drm->agp.bridge) { |
| 399 | 403 | args.target = NV_DMA_V0_TARGET_AGP; |
| @@ -405,7 +409,7 @@ nouveau_channel_init(struct nouveau_channel *chan, u32 vram, u32 gart) · 5 unmodified lines | ||
| 405 | 409 | args.target = NV_DMA_V0_TARGET_VM; |
| 406 | 410 | args.access = NV_DMA_V0_ACCESS_RDWR; |
| 407 | 411 | args.start = 0; |
| 408 | − args.limit = cli->vmm.vmm.limit - 1; | |
| 412 | + args.limit = chan->vmm->vmm.limit - 1; | |
| 409 | 413 | } |
| 410 | 414 | |
| 411 | 415 | ret = nvif_object_init(&chan->user, gart, NV_DMA_IN_MEMORY, |
| @@ -495,6 +499,10 @@ nouveau_channel_new(struct nouveau_drm *drm, struct nvif_device *device, · 83 unmodified lines | ||
| 495 | 499 | nouveau_channel_del(pchan); |
| 496 | 500 | } |
| 497 | 501 | |
| 502 | + ret = nouveau_svmm_join((*pchan)->vmm->svmm, (*pchan)->inst); | |
| 503 | + if (ret) | |
| 504 | + nouveau_channel_del(pchan); | |
| 505 | + | |
| 498 | 506 | done: |
| 499 | 507 | cli->base.super = super; |
| 500 | 508 | return ret; |
| @@ -8,6 +8,7 @@ struct nvif_device; · 7 unmodified lines | ||
| 8 | 8 | struct nouveau_channel { |
| 9 | 9 | struct nvif_device *device; |
| 10 | 10 | struct nouveau_drm *drm; |
| 11 | + struct nouveau_vmm *vmm; | |
| 11 | 12 | |
| 12 | 13 | int chid; |
| 13 | 14 | u64 inst; |
| @@ -32,18 +32,13 @@ · 31 unmodified lines | ||
| 32 | 32 | #include <drm/drm_fb_helper.h> |
| 33 | 33 | #include <drm/drm_probe_helper.h> |
| 34 | 34 | |
| 35 | −#include <nvif/class.h> | |
| 36 | − | |
| 37 | 35 | #include "nouveau_fbcon.h" |
| 38 | −#include "dispnv04/hw.h" | |
| 39 | 36 | #include "nouveau_crtc.h" |
| 40 | −#include "nouveau_dma.h" | |
| 41 | 37 | #include "nouveau_gem.h" |
| 42 | 38 | #include "nouveau_connector.h" |
| 43 | 39 | #include "nv50_display.h" |
| 44 | 40 | |
| 45 | −#include "nouveau_fence.h" | |
| 46 | − | |
| 41 | +#include <nvif/class.h> | |
| 47 | 42 | #include <nvif/cl0046.h> |
| 48 | 43 | #include <nvif/event.h> |
| 49 | 44 | |
| @@ -412,15 +407,14 @@ nouveau_display_acpi_ntfy(struct notifier_block *nb, unsigned long val, · 362 unmodified lines | ||
| 412 | 407 | #endif |
| 413 | 408 | |
| 414 | 409 | int |
| 415 | −nouveau_display_init(struct drm_device *dev) | |
| 410 | +nouveau_display_init(struct drm_device *dev, bool resume, bool runtime) | |
| 416 | 411 | { |
| 417 | 412 | struct nouveau_display *disp = nouveau_display(dev); |
| 418 | − struct nouveau_drm *drm = nouveau_drm(dev); | |
| 419 | 413 | struct drm_connector *connector; |
| 420 | 414 | struct drm_connector_list_iter conn_iter; |
| 421 | 415 | int ret; |
| 422 | 416 | |
| 423 | − ret = disp->init(dev); | |
| 417 | + ret = disp->init(dev, resume, runtime); | |
| 424 | 418 | if (ret) |
| 425 | 419 | return ret; |
| 426 | 420 | |
| @@ -437,8 +431,6 @@ nouveau_display_init(struct drm_device *dev) · 10 unmodified lines | ||
| 437 | 431 | } |
| 438 | 432 | drm_connector_list_iter_end(&conn_iter); |
| 439 | 433 | |
| 440 | − /* enable flip completion events */ | |
| 441 | − nvif_notify_get(&drm->flip); | |
| 442 | 434 | return ret; |
| 443 | 435 | } |
| 444 | 436 | |
| @@ -457,9 +449,6 @@ nouveau_display_fini(struct drm_device *dev, bool suspend, bool runtime) · 12 unmodified lines | ||
| 457 | 449 | drm_helper_force_disable_all(dev); |
| 458 | 450 | } |
| 459 | 451 | |
| 460 | − /* disable flip completion events */ | |
| 461 | − nvif_notify_put(&drm->flip); | |
| 462 | − | |
| 463 | 452 | /* disable hotplug interrupts */ |
| 464 | 453 | drm_connector_list_iter_begin(dev, &conn_iter); |
| 465 | 454 | nouveau_for_each_non_mst_connector_iter(connector, &conn_iter) { |
| @@ -472,7 +461,7 @@ nouveau_display_fini(struct drm_device *dev, bool suspend, bool runtime) · 6 unmodified lines | ||
| 472 | 461 | cancel_work_sync(&drm->hpd_work); |
| 473 | 462 | |
| 474 | 463 | drm_kms_helper_poll_disable(dev); |
| 475 | − disp->fini(dev); | |
| 464 | + disp->fini(dev, suspend); | |
| 476 | 465 | } |
| 477 | 466 | |
| 478 | 467 | static void |
| @@ -625,7 +614,6 @@ int · 146 unmodified lines | ||
| 625 | 614 | nouveau_display_suspend(struct drm_device *dev, bool runtime) |
| 626 | 615 | { |
| 627 | 616 | struct nouveau_display *disp = nouveau_display(dev); |
| 628 | − struct drm_crtc *crtc; | |
| 629 | 617 | |
| 630 | 618 | if (drm_drv_uses_atomic_modeset(dev)) { |
| 631 | 619 | if (!runtime) { |
| @@ -636,32 +624,9 @@ nouveau_display_suspend(struct drm_device *dev, bool runtime) · 4 unmodified lines | ||
| 636 | 624 | return ret; |
| 637 | 625 | } |
| 638 | 626 | } |
| 639 | − | |
| 640 | − nouveau_display_fini(dev, true, runtime); | |
| 641 | − return 0; | |
| 642 | 627 | } |
| 643 | 628 | |
| 644 | 629 | nouveau_display_fini(dev, true, runtime); |
| 645 | − | |
| 646 | − list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | |
| 647 | − struct nouveau_framebuffer *nouveau_fb; | |
| 648 | − | |
| 649 | − nouveau_fb = nouveau_framebuffer(crtc->primary->fb); | |
| 650 | − if (!nouveau_fb || !nouveau_fb->nvbo) | |
| 651 | − continue; | |
| 652 | − | |
| 653 | − nouveau_bo_unpin(nouveau_fb->nvbo); | |
| 654 | − } | |
| 655 | − | |
| 656 | − list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | |
| 657 | − struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | |
| 658 | − if (nv_crtc->cursor.nvbo) { | |
| 659 | − if (nv_crtc->cursor.set_offset) | |
| 660 | − nouveau_bo_unmap(nv_crtc->cursor.nvbo); | |
| 661 | − nouveau_bo_unpin(nv_crtc->cursor.nvbo); | |
| 662 | − } | |
| 663 | − } | |
| 664 | − | |
| 665 | 630 | return 0; |
| 666 | 631 | } |
| 667 | 632 | |
| @@ -669,275 +634,16 @@ void · 1 unmodified line | ||
| 669 | 634 | nouveau_display_resume(struct drm_device *dev, bool runtime) |
| 670 | 635 | { |
| 671 | 636 | struct nouveau_display *disp = nouveau_display(dev); |
| 672 | − struct nouveau_drm *drm = nouveau_drm(dev); | |
| 673 | − struct drm_crtc *crtc; | |
| 674 | − int ret; | |
| 637 | + | |
| 638 | + nouveau_display_init(dev, true, runtime); | |
| 675 | 639 | |
| 676 | 640 | if (drm_drv_uses_atomic_modeset(dev)) { |
| 677 | − nouveau_display_init(dev); | |
| 678 | 641 | if (disp->suspend) { |
| 679 | 642 | drm_atomic_helper_resume(dev, disp->suspend); |
| 680 | 643 | disp->suspend = NULL; |
| 681 | 644 | } |
| 682 | 645 | return; |
| 683 | 646 | } |
| 684 | − | |
| 685 | − /* re-pin fb/cursors */ | |
| 686 | − list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | |
| 687 | − struct nouveau_framebuffer *nouveau_fb; | |
| 688 | − | |
| 689 | − nouveau_fb = nouveau_framebuffer(crtc->primary->fb); | |
| 690 | − if (!nouveau_fb || !nouveau_fb->nvbo) | |
| 691 | − continue; | |
| 692 | − | |
| 693 | − ret = nouveau_bo_pin(nouveau_fb->nvbo, TTM_PL_FLAG_VRAM, true); | |
| 694 | − if (ret) | |
| 695 | − NV_ERROR(drm, "Could not pin framebuffer\n"); | |
| 696 | − } | |
| 697 | − | |
| 698 | − list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | |
| 699 | − struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | |
| 700 | − if (!nv_crtc->cursor.nvbo) | |
| 701 | − continue; | |
| 702 | − | |
| 703 | − ret = nouveau_bo_pin(nv_crtc->cursor.nvbo, TTM_PL_FLAG_VRAM, true); | |
| 704 | − if (!ret && nv_crtc->cursor.set_offset) | |
| 705 | − ret = nouveau_bo_map(nv_crtc->cursor.nvbo); | |
| 706 | − if (ret) | |
| 707 | − NV_ERROR(drm, "Could not pin/map cursor.\n"); | |
| 708 | − } | |
| 709 | − | |
| 710 | − nouveau_display_init(dev); | |
| 711 | − | |
| 712 | − /* Force CLUT to get re-loaded during modeset */ | |
| 713 | − list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | |
| 714 | − struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | |
| 715 | − | |
| 716 | − nv_crtc->lut.depth = 0; | |
| 717 | − } | |
| 718 | − | |
| 719 | − /* This should ensure we don't hit a locking problem when someone | |
| 720 | − * wakes us up via a connector. We should never go into suspend | |
| 721 | − * while the display is on anyways. | |
| 722 | − */ | |
| 723 | − if (runtime) | |
| 724 | − return; | |
| 725 | − | |
| 726 | − drm_helper_resume_force_mode(dev); | |
| 727 | − | |
| 728 | − list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | |
| 729 | − struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | |
| 730 | − | |
| 731 | − if (!nv_crtc->cursor.nvbo) | |
| 732 | − continue; | |
| 733 | − | |
| 734 | − if (nv_crtc->cursor.set_offset) | |
| 735 | − nv_crtc->cursor.set_offset(nv_crtc, nv_crtc->cursor.nvbo->bo.offset); | |
| 736 | − nv_crtc->cursor.set_pos(nv_crtc, nv_crtc->cursor_saved_x, | |
| 737 | − nv_crtc->cursor_saved_y); | |
| 738 | − } | |
| 739 | −} | |
| 740 | − | |
| 741 | −static int | |
| 742 | −nouveau_page_flip_emit(struct nouveau_channel *chan, | |
| 743 | − struct nouveau_bo *old_bo, | |
| 744 | − struct nouveau_bo *new_bo, | |
| 745 | − struct nouveau_page_flip_state *s, | |
| 746 | − struct nouveau_fence **pfence) | |
| 747 | −{ | |
| 748 | − struct nouveau_fence_chan *fctx = chan->fence; | |
| 749 | − struct nouveau_drm *drm = chan->drm; | |
| 750 | − struct drm_device *dev = drm->dev; | |
| 751 | − unsigned long flags; | |
| 752 | − int ret; | |
| 753 | − | |
| 754 | − /* Queue it to the pending list */ | |
| 755 | − spin_lock_irqsave(&dev->event_lock, flags); | |
| 756 | − list_add_tail(&s->head, &fctx->flip); | |
| 757 | − spin_unlock_irqrestore(&dev->event_lock, flags); | |
| 758 | − | |
| 759 | − /* Synchronize with the old framebuffer */ | |
| 760 | − ret = nouveau_fence_sync(old_bo, chan, false, false); | |
| 761 | − if (ret) | |
| 762 | − goto fail; | |
| 763 | − | |
| 764 | − /* Emit the pageflip */ | |
| 765 | − ret = RING_SPACE(chan, 2); | |
| 766 | − if (ret) | |
| 767 | − goto fail; | |
| 768 | − | |
| 769 | − BEGIN_NV04(chan, NvSubSw, NV_SW_PAGE_FLIP, 1); | |
| 770 | − OUT_RING (chan, 0x00000000); | |
| 771 | − FIRE_RING (chan); | |
| 772 | − | |
| 773 | − ret = nouveau_fence_new(chan, false, pfence); | |
| 774 | − if (ret) | |
| 775 | − goto fail; | |
| 776 | − | |
| 777 | − return 0; | |
| 778 | −fail: | |
| 779 | − spin_lock_irqsave(&dev->event_lock, flags); | |
| 780 | − list_del(&s->head); | |
| 781 | − spin_unlock_irqrestore(&dev->event_lock, flags); | |
| 782 | − return ret; | |
| 783 | −} | |
| 784 | − | |
| 785 | −int | |
| 786 | −nouveau_crtc_page_flip(struct drm_crtc *crtc, struct drm_framebuffer *fb, | |
| 787 | − struct drm_pending_vblank_event *event, u32 flags, | |
| 788 | − struct drm_modeset_acquire_ctx *ctx) | |
| 789 | −{ | |
| 790 | − const int swap_interval = (flags & DRM_MODE_PAGE_FLIP_ASYNC) ? 0 : 1; | |
| 791 | − struct drm_device *dev = crtc->dev; | |
| 792 | − struct nouveau_drm *drm = nouveau_drm(dev); | |
| 793 | − struct nouveau_bo *old_bo = nouveau_framebuffer(crtc->primary->fb)->nvbo; | |
| 794 | − struct nouveau_bo *new_bo = nouveau_framebuffer(fb)->nvbo; | |
| 795 | − struct nouveau_page_flip_state *s; | |
| 796 | − struct nouveau_channel *chan; | |
| 797 | − struct nouveau_cli *cli; | |
| 798 | − struct nouveau_fence *fence; | |
| 799 | − struct nv04_display *dispnv04 = nv04_display(dev); | |
| 800 | − int head = nouveau_crtc(crtc)->index; | |
| 801 | − int ret; | |
| 802 | − | |
| 803 | − chan = drm->channel; | |
| 804 | − if (!chan) | |
| 805 | − return -ENODEV; | |
| 806 | − cli = (void *)chan->user.client; | |
| 807 | − | |
| 808 | − s = kzalloc(sizeof(*s), GFP_KERNEL); | |
| 809 | − if (!s) | |
| 810 | − return -ENOMEM; | |
| 811 | − | |
| 812 | − if (new_bo != old_bo) { | |
| 813 | − ret = nouveau_bo_pin(new_bo, TTM_PL_FLAG_VRAM, true); | |
| 814 | − if (ret) | |
| 815 | − goto fail_free; | |
| 816 | − } | |
| 817 | − | |
| 818 | − mutex_lock(&cli->mutex); | |
| 819 | − ret = ttm_bo_reserve(&new_bo->bo, true, false, NULL); | |
| 820 | − if (ret) | |
| 821 | − goto fail_unpin; | |
| 822 | − | |
| 823 | − /* synchronise rendering channel with the kernel's channel */ | |
| 824 | − ret = nouveau_fence_sync(new_bo, chan, false, true); | |
| 825 | − if (ret) { | |
| 826 | − ttm_bo_unreserve(&new_bo->bo); | |
| 827 | − goto fail_unpin; | |
| 828 | − } | |
| 829 | − | |
| 830 | − if (new_bo != old_bo) { | |
| 831 | − ttm_bo_unreserve(&new_bo->bo); | |
| 832 | − | |
| 833 | − ret = ttm_bo_reserve(&old_bo->bo, true, false, NULL); | |
| 834 | − if (ret) | |
| 835 | − goto fail_unpin; | |
| 836 | − } | |
| 837 | − | |
| 838 | − /* Initialize a page flip struct */ | |
| 839 | − *s = (struct nouveau_page_flip_state) | |
| 840 | − { { }, event, crtc, fb->format->cpp[0] * 8, fb->pitches[0], | |
| 841 | − new_bo->bo.offset }; | |
| 842 | − | |
| 843 | − /* Keep vblanks on during flip, for the target crtc of this flip */ | |
| 844 | − drm_crtc_vblank_get(crtc); | |
| 845 | − | |
| 846 | − /* Emit a page flip */ | |
| 847 | − if (swap_interval) { | |
| 848 | − ret = RING_SPACE(chan, 8); | |
| 849 | − if (ret) | |
| 850 | − goto fail_unreserve; | |
| 851 | − | |
| 852 | − BEGIN_NV04(chan, NvSubImageBlit, 0x012c, 1); | |
| 853 | − OUT_RING (chan, 0); | |
| 854 | − BEGIN_NV04(chan, NvSubImageBlit, 0x0134, 1); | |
| 855 | − OUT_RING (chan, head); | |
| 856 | − BEGIN_NV04(chan, NvSubImageBlit, 0x0100, 1); | |
| 857 | − OUT_RING (chan, 0); | |
| 858 | − BEGIN_NV04(chan, NvSubImageBlit, 0x0130, 1); | |
| 859 | − OUT_RING (chan, 0); | |
| 860 | − } | |
| 861 | − | |
| 862 | − nouveau_bo_ref(new_bo, &dispnv04->image[head]); | |
| 863 | − | |
| 864 | − ret = nouveau_page_flip_emit(chan, old_bo, new_bo, s, &fence); | |
| 865 | − if (ret) | |
| 866 | − goto fail_unreserve; | |
| 867 | − mutex_unlock(&cli->mutex); | |
| 868 | − | |
| 869 | − /* Update the crtc struct and cleanup */ | |
| 870 | − crtc->primary->fb = fb; | |
| 871 | − | |
| 872 | − nouveau_bo_fence(old_bo, fence, false); | |
| 873 | − ttm_bo_unreserve(&old_bo->bo); | |
| 874 | − if (old_bo != new_bo) | |
| 875 | − nouveau_bo_unpin(old_bo); | |
| 876 | − nouveau_fence_unref(&fence); | |
| 877 | − return 0; | |
| 878 | − | |
| 879 | −fail_unreserve: | |
| 880 | − drm_crtc_vblank_put(crtc); | |
| 881 | − ttm_bo_unreserve(&old_bo->bo); | |
| 882 | −fail_unpin: | |
| 883 | − mutex_unlock(&cli->mutex); | |
| 884 | − if (old_bo != new_bo) | |
| 885 | − nouveau_bo_unpin(new_bo); | |
| 886 | −fail_free: | |
| 887 | − kfree(s); | |
| 888 | − return ret; | |
| 889 | −} | |
| 890 | − | |
| 891 | −int | |
| 892 | −nouveau_finish_page_flip(struct nouveau_channel *chan, | |
| 893 | − struct nouveau_page_flip_state *ps) | |
| 894 | −{ | |
| 895 | − struct nouveau_fence_chan *fctx = chan->fence; | |
| 896 | − struct nouveau_drm *drm = chan->drm; | |
| 897 | − struct drm_device *dev = drm->dev; | |
| 898 | − struct nouveau_page_flip_state *s; | |
| 899 | − unsigned long flags; | |
| 900 | − | |
| 901 | − spin_lock_irqsave(&dev->event_lock, flags); | |
| 902 | − | |
| 903 | − if (list_empty(&fctx->flip)) { | |
| 904 | − NV_ERROR(drm, "unexpected pageflip\n"); | |
| 905 | − spin_unlock_irqrestore(&dev->event_lock, flags); | |
| 906 | − return -EINVAL; | |
| 907 | − } | |
| 908 | − | |
| 909 | − s = list_first_entry(&fctx->flip, struct nouveau_page_flip_state, head); | |
| 910 | − if (s->event) { | |
| 911 | − drm_crtc_arm_vblank_event(s->crtc, s->event); | |
| 912 | − } else { | |
| 913 | − /* Give up ownership of vblank for page-flipped crtc */ | |
| 914 | − drm_crtc_vblank_put(s->crtc); | |
| 915 | − } | |
| 916 | − | |
| 917 | − list_del(&s->head); | |
| 918 | − if (ps) | |
| 919 | − *ps = *s; | |
| 920 | − kfree(s); | |
| 921 | − | |
| 922 | − spin_unlock_irqrestore(&dev->event_lock, flags); | |
| 923 | − return 0; | |
| 924 | −} | |
| 925 | − | |
| 926 | −int | |
| 927 | −nouveau_flip_complete(struct nvif_notify *notify) | |
| 928 | −{ | |
| 929 | − struct nouveau_drm *drm = container_of(notify, typeof(*drm), flip); | |
| 930 | − struct nouveau_channel *chan = drm->channel; | |
| 931 | − struct nouveau_page_flip_state state; | |
| 932 | − | |
| 933 | − if (!nouveau_finish_page_flip(chan, &state)) { | |
| 934 | − nv_set_crtc_base(drm->dev, drm_crtc_index(state.crtc), | |
| 935 | − state.offset + state.crtc->y * | |
| 936 | − state.pitch + state.crtc->x * | |
| 937 | − state.bpp / 8); | |
| 938 | − } | |
| 939 | − | |
| 940 | − return NVIF_NOTIFY_KEEP; | |
| 941 | 647 | } |
| 942 | 648 | |
| 943 | 649 | int |
| @@ -25,19 +25,11 @@ int nouveau_framebuffer_new(struct drm_device *, · 24 unmodified lines | ||
| 25 | 25 | const struct drm_mode_fb_cmd2 *, |
| 26 | 26 | struct nouveau_bo *, struct nouveau_framebuffer **); |
| 27 | 27 | |
| 28 | −struct nouveau_page_flip_state { | |
| 29 | − struct list_head head; | |
| 30 | − struct drm_pending_vblank_event *event; | |
| 31 | − struct drm_crtc *crtc; | |
| 32 | − int bpp, pitch; | |
| 33 | − u64 offset; | |
| 34 | −}; | |
| 35 | − | |
| 36 | 28 | struct nouveau_display { |
| 37 | 29 | void *priv; |
| 38 | 30 | void (*dtor)(struct drm_device *); |
| 39 | − int (*init)(struct drm_device *); | |
| 40 | − void (*fini)(struct drm_device *); | |
| 31 | + int (*init)(struct drm_device *, bool resume, bool runtime); | |
| 32 | + void (*fini)(struct drm_device *, bool suspend); | |
| 41 | 33 | |
| 42 | 34 | struct nvif_disp disp; |
| 43 | 35 | |
| @@ -61,7 +53,7 @@ nouveau_display(struct drm_device *dev) · 17 unmodified lines | ||
| 61 | 53 | |
| 62 | 54 | int nouveau_display_create(struct drm_device *dev); |
| 63 | 55 | void nouveau_display_destroy(struct drm_device *dev); |
| 64 | −int nouveau_display_init(struct drm_device *dev); | |
| 56 | +int nouveau_display_init(struct drm_device *dev, bool resume, bool runtime); | |
| 65 | 57 | void nouveau_display_fini(struct drm_device *dev, bool suspend, bool runtime); |
| 66 | 58 | int nouveau_display_suspend(struct drm_device *dev, bool runtime); |
| 67 | 59 | void nouveau_display_resume(struct drm_device *dev, bool runtime); |
| @@ -71,13 +63,6 @@ bool nouveau_display_scanoutpos(struct drm_device *, unsigned int, · 3 unmodified lines | ||
| 71 | 63 | bool, int *, int *, ktime_t *, |
| 72 | 64 | ktime_t *, const struct drm_display_mode *); |
| 73 | 65 | |
| 74 | −int nouveau_crtc_page_flip(struct drm_crtc *crtc, struct drm_framebuffer *fb, | |
| 75 | − struct drm_pending_vblank_event *event, | |
| 76 | − uint32_t page_flip_flags, | |
| 77 | − struct drm_modeset_acquire_ctx *ctx); | |
| 78 | −int nouveau_finish_page_flip(struct nouveau_channel *, | |
| 79 | − struct nouveau_page_flip_state *); | |
| 80 | − | |
| 81 | 66 | int nouveau_display_dumb_create(struct drm_file *, struct drm_device *, |
| 82 | 67 | struct drm_mode_create_dumb *args); |
| 83 | 68 | int nouveau_display_dumb_map_offset(struct drm_file *, struct drm_device *, |
| @@ -0,0 +1,887 @@ | ||
| 1 | +/* | |
| 2 | + * Copyright 2018 Red Hat Inc. | |
| 3 | + * | |
| 4 | + * Permission is hereby granted, free of charge, to any person obtaining a | |
| 5 | + * copy of this software and associated documentation files (the "Software"), | |
| 6 | + * to deal in the Software without restriction, including without limitation | |
| 7 | + * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 8 | + * and/or sell copies of the Software, and to permit persons to whom the | |
| 9 | + * Software is furnished to do so, subject to the following conditions: | |
| 10 | + * | |
| 11 | + * The above copyright notice and this permission notice shall be included in | |
| 12 | + * all copies or substantial portions of the Software. | |
| 13 | + * | |
| 14 | + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 15 | + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 16 | + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 17 | + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 18 | + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 19 | + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 20 | + * OTHER DEALINGS IN THE SOFTWARE. | |
| 21 | + */ | |
| 22 | +#include "nouveau_dmem.h" | |
| 23 | +#include "nouveau_drv.h" | |
| 24 | +#include "nouveau_chan.h" | |
| 25 | +#include "nouveau_dma.h" | |
| 26 | +#include "nouveau_mem.h" | |
| 27 | +#include "nouveau_bo.h" | |
| 28 | + | |
| 29 | +#include <nvif/class.h> | |
| 30 | +#include <nvif/object.h> | |
| 31 | +#include <nvif/if500b.h> | |
| 32 | +#include <nvif/if900b.h> | |
| 33 | + | |
| 34 | +#include <linux/sched/mm.h> | |
| 35 | +#include <linux/hmm.h> | |
| 36 | + | |
| 37 | +/* | |
| 38 | + * FIXME: this is ugly right now we are using TTM to allocate vram and we pin | |
| 39 | + * it in vram while in use. We likely want to overhaul memory management for | |
| 40 | + * nouveau to be more page like (not necessarily with system page size but a | |
| 41 | + * bigger page size) at lowest level and have some shim layer on top that would | |
| 42 | + * provide the same functionality as TTM. | |
| 43 | + */ | |
| 44 | +#define DMEM_CHUNK_SIZE (2UL << 20) | |
| 45 | +#define DMEM_CHUNK_NPAGES (DMEM_CHUNK_SIZE >> PAGE_SHIFT) | |
| 46 | + | |
| 47 | +struct nouveau_migrate; | |
| 48 | + | |
| 49 | +enum nouveau_aper { | |
| 50 | + NOUVEAU_APER_VIRT, | |
| 51 | + NOUVEAU_APER_VRAM, | |
| 52 | + NOUVEAU_APER_HOST, | |
| 53 | +}; | |
| 54 | + | |
| 55 | +typedef int (*nouveau_migrate_copy_t)(struct nouveau_drm *drm, u64 npages, | |
| 56 | + enum nouveau_aper, u64 dst_addr, | |
| 57 | + enum nouveau_aper, u64 src_addr); | |
| 58 | + | |
| 59 | +struct nouveau_dmem_chunk { | |
| 60 | + struct list_head list; | |
| 61 | + struct nouveau_bo *bo; | |
| 62 | + struct nouveau_drm *drm; | |
| 63 | + unsigned long pfn_first; | |
| 64 | + unsigned long callocated; | |
| 65 | + unsigned long bitmap[BITS_TO_LONGS(DMEM_CHUNK_NPAGES)]; | |
| 66 | + spinlock_t lock; | |
| 67 | +}; | |
| 68 | + | |
| 69 | +struct nouveau_dmem_migrate { | |
| 70 | + nouveau_migrate_copy_t copy_func; | |
| 71 | + struct nouveau_channel *chan; | |
| 72 | +}; | |
| 73 | + | |
| 74 | +struct nouveau_dmem { | |
| 75 | + struct hmm_devmem *devmem; | |
| 76 | + struct nouveau_dmem_migrate migrate; | |
| 77 | + struct list_head chunk_free; | |
| 78 | + struct list_head chunk_full; | |
| 79 | + struct list_head chunk_empty; | |
| 80 | + struct mutex mutex; | |
| 81 | +}; | |
| 82 | + | |
| 83 | +struct nouveau_dmem_fault { | |
| 84 | + struct nouveau_drm *drm; | |
| 85 | + struct nouveau_fence *fence; | |
| 86 | + dma_addr_t *dma; | |
| 87 | + unsigned long npages; | |
| 88 | +}; | |
| 89 | + | |
| 90 | +struct nouveau_migrate { | |
| 91 | + struct vm_area_struct *vma; | |
| 92 | + struct nouveau_drm *drm; | |
| 93 | + struct nouveau_fence *fence; | |
| 94 | + unsigned long npages; | |
| 95 | + dma_addr_t *dma; | |
| 96 | + unsigned long dma_nr; | |
| 97 | +}; | |
| 98 | + | |
| 99 | +static void | |
| 100 | +nouveau_dmem_free(struct hmm_devmem *devmem, struct page *page) | |
| 101 | +{ | |
| 102 | + struct nouveau_dmem_chunk *chunk; | |
| 103 | + struct nouveau_drm *drm; | |
| 104 | + unsigned long idx; | |
| 105 | + | |
| 106 | + chunk = (void *)hmm_devmem_page_get_drvdata(page); | |
| 107 | + idx = page_to_pfn(page) - chunk->pfn_first; | |
| 108 | + drm = chunk->drm; | |
| 109 | + | |
| 110 | + /* | |
| 111 | + * FIXME: | |
| 112 | + * | |
| 113 | + * This is really a bad example, we need to overhaul nouveau memory | |
| 114 | + * management to be more page focus and allow lighter locking scheme | |
| 115 | + * to be use in the process. | |
| 116 | + */ | |
| 117 | + spin_lock(&chunk->lock); | |
| 118 | + clear_bit(idx, chunk->bitmap); | |
| 119 | + WARN_ON(!chunk->callocated); | |
| 120 | + chunk->callocated--; | |
| 121 | + /* | |
| 122 | + * FIXME when chunk->callocated reach 0 we should add the chunk to | |
| 123 | + * a reclaim list so that it can be freed in case of memory pressure. | |
| 124 | + */ | |
| 125 | + spin_unlock(&chunk->lock); | |
| 126 | +} | |
| 127 | + | |
| 128 | +static void | |
| 129 | +nouveau_dmem_fault_alloc_and_copy(struct vm_area_struct *vma, | |
| 130 | + const unsigned long *src_pfns, | |
| 131 | + unsigned long *dst_pfns, | |
| 132 | + unsigned long start, | |
| 133 | + unsigned long end, | |
| 134 | + void *private) | |
| 135 | +{ | |
| 136 | + struct nouveau_dmem_fault *fault = private; | |
| 137 | + struct nouveau_drm *drm = fault->drm; | |
| 138 | + struct device *dev = drm->dev->dev; | |
| 139 | + unsigned long addr, i, npages = 0; | |
| 140 | + nouveau_migrate_copy_t copy; | |
| 141 | + int ret; | |
| 142 | + | |
| 143 | + | |
| 144 | + /* First allocate new memory */ | |
| 145 | + for (addr = start, i = 0; addr < end; addr += PAGE_SIZE, i++) { | |
| 146 | + struct page *dpage, *spage; | |
| 147 | + | |
| 148 | + dst_pfns[i] = 0; | |
| 149 | + spage = migrate_pfn_to_page(src_pfns[i]); | |
| 150 | + if (!spage || !(src_pfns[i] & MIGRATE_PFN_MIGRATE)) | |
| 151 | + continue; | |
| 152 | + | |
| 153 | + dpage = hmm_vma_alloc_locked_page(vma, addr); | |
| 154 | + if (!dpage) { | |
| 155 | + dst_pfns[i] = MIGRATE_PFN_ERROR; | |
| 156 | + continue; | |
| 157 | + } | |
| 158 | + | |
| 159 | + dst_pfns[i] = migrate_pfn(page_to_pfn(dpage)) | | |
| 160 | + MIGRATE_PFN_LOCKED; | |
| 161 | + npages++; | |
| 162 | + } | |
| 163 | + | |
| 164 | + /* Allocate storage for DMA addresses, so we can unmap later. */ | |
| 165 | + fault->dma = kmalloc(sizeof(*fault->dma) * npages, GFP_KERNEL); | |
| 166 | + if (!fault->dma) | |
| 167 | + goto error; | |
| 168 | + | |
| 169 | + /* Copy things over */ | |
| 170 | + copy = drm->dmem->migrate.copy_func; | |
| 171 | + for (addr = start, i = 0; addr < end; addr += PAGE_SIZE, i++) { | |
| 172 | + struct nouveau_dmem_chunk *chunk; | |
| 173 | + struct page *spage, *dpage; | |
| 174 | + u64 src_addr, dst_addr; | |
| 175 | + | |
| 176 | + dpage = migrate_pfn_to_page(dst_pfns[i]); | |
| 177 | + if (!dpage || dst_pfns[i] == MIGRATE_PFN_ERROR) | |
| 178 | + continue; | |
| 179 | + | |
| 180 | + spage = migrate_pfn_to_page(src_pfns[i]); | |
| 181 | + if (!spage || !(src_pfns[i] & MIGRATE_PFN_MIGRATE)) { | |
| 182 | + dst_pfns[i] = MIGRATE_PFN_ERROR; | |
| 183 | + __free_page(dpage); | |
| 184 | + continue; | |
| 185 | + } | |
| 186 | + | |
| 187 | + fault->dma[fault->npages] = | |
| 188 | + dma_map_page_attrs(dev, dpage, 0, PAGE_SIZE, | |
| 189 | + PCI_DMA_BIDIRECTIONAL, | |
| 190 | + DMA_ATTR_SKIP_CPU_SYNC); | |
| 191 | + if (dma_mapping_error(dev, fault->dma[fault->npages])) { | |
| 192 | + dst_pfns[i] = MIGRATE_PFN_ERROR; | |
| 193 | + __free_page(dpage); | |
| 194 | + continue; | |
| 195 | + } | |
| 196 | + | |
| 197 | + dst_addr = fault->dma[fault->npages++]; | |
| 198 | + | |
| 199 | + chunk = (void *)hmm_devmem_page_get_drvdata(spage); | |
| 200 | + src_addr = page_to_pfn(spage) - chunk->pfn_first; | |
| 201 | + src_addr = (src_addr << PAGE_SHIFT) + chunk->bo->bo.offset; | |
| 202 | + | |
| 203 | + ret = copy(drm, 1, NOUVEAU_APER_HOST, dst_addr, | |
| 204 | + NOUVEAU_APER_VRAM, src_addr); | |
| 205 | + if (ret) { | |
| 206 | + dst_pfns[i] = MIGRATE_PFN_ERROR; | |
| 207 | + __free_page(dpage); | |
| 208 | + continue; | |
| 209 | + } | |
| 210 | + } | |
| 211 | + | |
| 212 | + nouveau_fence_new(drm->dmem->migrate.chan, false, &fault->fence); | |
| 213 | + | |
| 214 | + return; | |
| 215 | + | |
| 216 | +error: | |
| 217 | + for (addr = start, i = 0; addr < end; addr += PAGE_SIZE, ++i) { | |
| 218 | + struct page *page; | |
| 219 | + | |
| 220 | + if (!dst_pfns[i] || dst_pfns[i] == MIGRATE_PFN_ERROR) | |
| 221 | + continue; | |
| 222 | + | |
| 223 | + page = migrate_pfn_to_page(dst_pfns[i]); | |
| 224 | + dst_pfns[i] = MIGRATE_PFN_ERROR; | |
| 225 | + if (page == NULL) | |
| 226 | + continue; | |
| 227 | + | |
| 228 | + __free_page(page); | |
| 229 | + } | |
| 230 | +} | |
| 231 | + | |
| 232 | +void nouveau_dmem_fault_finalize_and_map(struct vm_area_struct *vma, | |
| 233 | + const unsigned long *src_pfns, | |
| 234 | + const unsigned long *dst_pfns, | |
| 235 | + unsigned long start, | |
| 236 | + unsigned long end, | |
| 237 | + void *private) | |
| 238 | +{ | |
| 239 | + struct nouveau_dmem_fault *fault = private; | |
| 240 | + struct nouveau_drm *drm = fault->drm; | |
| 241 | + | |
| 242 | + if (fault->fence) { | |
| 243 | + nouveau_fence_wait(fault->fence, true, false); | |
| 244 | + nouveau_fence_unref(&fault->fence); | |
| 245 | + } else { | |
| 246 | + /* | |
| 247 | + * FIXME wait for channel to be IDLE before calling finalizing | |
| 248 | + * the hmem object below (nouveau_migrate_hmem_fini()). | |
| 249 | + */ | |
| 250 | + } | |
| 251 | + | |
| 252 | + while (fault->npages--) { | |
| 253 | + dma_unmap_page(drm->dev->dev, fault->dma[fault->npages], | |
| 254 | + PAGE_SIZE, PCI_DMA_BIDIRECTIONAL); | |
| 255 | + } | |
| 256 | + kfree(fault->dma); | |
| 257 | +} | |
| 258 | + | |
| 259 | +static const struct migrate_vma_ops nouveau_dmem_fault_migrate_ops = { | |
| 260 | + .alloc_and_copy = nouveau_dmem_fault_alloc_and_copy, | |
| 261 | + .finalize_and_map = nouveau_dmem_fault_finalize_and_map, | |
| 262 | +}; | |
| 263 | + | |
| 264 | +static int | |
| 265 | +nouveau_dmem_fault(struct hmm_devmem *devmem, | |
| 266 | + struct vm_area_struct *vma, | |
| 267 | + unsigned long addr, | |
| 268 | + const struct page *page, | |
| 269 | + unsigned int flags, | |
| 270 | + pmd_t *pmdp) | |
| 271 | +{ | |
| 272 | + struct drm_device *drm_dev = dev_get_drvdata(devmem->device); | |
| 273 | + unsigned long src[1] = {0}, dst[1] = {0}; | |
| 274 | + struct nouveau_dmem_fault fault = {0}; | |
| 275 | + int ret; | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + /* | |
| 280 | + * FIXME what we really want is to find some heuristic to migrate more | |
| 281 | + * than just one page on CPU fault. When such fault happens it is very | |
| 282 | + * likely that more surrounding page will CPU fault too. | |
| 283 | + */ | |
| 284 | + fault.drm = nouveau_drm(drm_dev); | |
| 285 | + ret = migrate_vma(&nouveau_dmem_fault_migrate_ops, vma, addr, | |
| 286 | + addr + PAGE_SIZE, src, dst, &fault); | |
| 287 | + if (ret) | |
| 288 | + return VM_FAULT_SIGBUS; | |
| 289 | + | |
| 290 | + if (dst[0] == MIGRATE_PFN_ERROR) | |
| 291 | + return VM_FAULT_SIGBUS; | |
| 292 | + | |
| 293 | + return 0; | |
| 294 | +} | |
| 295 | + | |
| 296 | +static const struct hmm_devmem_ops | |
| 297 | +nouveau_dmem_devmem_ops = { | |
| 298 | + .free = nouveau_dmem_free, | |
| 299 | + .fault = nouveau_dmem_fault, | |
| 300 | +}; | |
| 301 | + | |
| 302 | +static int | |
| 303 | +nouveau_dmem_chunk_alloc(struct nouveau_drm *drm) | |
| 304 | +{ | |
| 305 | + struct nouveau_dmem_chunk *chunk; | |
| 306 | + int ret; | |
| 307 | + | |
| 308 | + if (drm->dmem == NULL) | |
| 309 | + return -EINVAL; | |
| 310 | + | |
| 311 | + mutex_lock(&drm->dmem->mutex); | |
| 312 | + chunk = list_first_entry_or_null(&drm->dmem->chunk_empty, | |
| 313 | + struct nouveau_dmem_chunk, | |
| 314 | + list); | |
| 315 | + if (chunk == NULL) { | |
| 316 | + mutex_unlock(&drm->dmem->mutex); | |
| 317 | + return -ENOMEM; | |
| 318 | + } | |
| 319 | + | |
| 320 | + list_del(&chunk->list); | |
| 321 | + mutex_unlock(&drm->dmem->mutex); | |
| 322 | + | |
| 323 | + ret = nouveau_bo_new(&drm->client, DMEM_CHUNK_SIZE, 0, | |
| 324 | + TTM_PL_FLAG_VRAM, 0, 0, NULL, NULL, | |
| 325 | + &chunk->bo); | |
| 326 | + if (ret) | |
| 327 | + goto out; | |
| 328 | + | |
| 329 | + ret = nouveau_bo_pin(chunk->bo, TTM_PL_FLAG_VRAM, false); | |
| 330 | + if (ret) { | |
| 331 | + nouveau_bo_ref(NULL, &chunk->bo); | |
| 332 | + goto out; | |
| 333 | + } | |
| 334 | + | |
| 335 | + bitmap_zero(chunk->bitmap, DMEM_CHUNK_NPAGES); | |
| 336 | + spin_lock_init(&chunk->lock); | |
| 337 | + | |
| 338 | +out: | |
| 339 | + mutex_lock(&drm->dmem->mutex); | |
| 340 | + if (chunk->bo) | |
| 341 | + list_add(&chunk->list, &drm->dmem->chunk_empty); | |
| 342 | + else | |
| 343 | + list_add_tail(&chunk->list, &drm->dmem->chunk_empty); | |
| 344 | + mutex_unlock(&drm->dmem->mutex); | |
| 345 | + | |
| 346 | + return ret; | |
| 347 | +} | |
| 348 | + | |
| 349 | +static struct nouveau_dmem_chunk * | |
| 350 | +nouveau_dmem_chunk_first_free_locked(struct nouveau_drm *drm) | |
| 351 | +{ | |
| 352 | + struct nouveau_dmem_chunk *chunk; | |
| 353 | + | |
| 354 | + chunk = list_first_entry_or_null(&drm->dmem->chunk_free, | |
| 355 | + struct nouveau_dmem_chunk, | |
| 356 | + list); | |
| 357 | + if (chunk) | |
| 358 | + return chunk; | |
| 359 | + | |
| 360 | + chunk = list_first_entry_or_null(&drm->dmem->chunk_empty, | |
| 361 | + struct nouveau_dmem_chunk, | |
| 362 | + list); | |
| 363 | + if (chunk->bo) | |
| 364 | + return chunk; | |
| 365 | + | |
| 366 | + return NULL; | |
| 367 | +} | |
| 368 | + | |
| 369 | +static int | |
| 370 | +nouveau_dmem_pages_alloc(struct nouveau_drm *drm, | |
| 371 | + unsigned long npages, | |
| 372 | + unsigned long *pages) | |
| 373 | +{ | |
| 374 | + struct nouveau_dmem_chunk *chunk; | |
| 375 | + unsigned long c; | |
| 376 | + int ret; | |
| 377 | + | |
| 378 | + memset(pages, 0xff, npages * sizeof(*pages)); | |
| 379 | + | |
| 380 | + mutex_lock(&drm->dmem->mutex); | |
| 381 | + for (c = 0; c < npages;) { | |
| 382 | + unsigned long i; | |
| 383 | + | |
| 384 | + chunk = nouveau_dmem_chunk_first_free_locked(drm); | |
| 385 | + if (chunk == NULL) { | |
| 386 | + mutex_unlock(&drm->dmem->mutex); | |
| 387 | + ret = nouveau_dmem_chunk_alloc(drm); | |
| 388 | + if (ret) { | |
| 389 | + if (c) | |
| 390 | + break; | |
| 391 | + return ret; | |
| 392 | + } | |
| 393 | + continue; | |
| 394 | + } | |
| 395 | + | |
| 396 | + spin_lock(&chunk->lock); | |
| 397 | + i = find_first_zero_bit(chunk->bitmap, DMEM_CHUNK_NPAGES); | |
| 398 | + while (i < DMEM_CHUNK_NPAGES && c < npages) { | |
| 399 | + pages[c] = chunk->pfn_first + i; | |
| 400 | + set_bit(i, chunk->bitmap); | |
| 401 | + chunk->callocated++; | |
| 402 | + c++; | |
| 403 | + | |
| 404 | + i = find_next_zero_bit(chunk->bitmap, | |
| 405 | + DMEM_CHUNK_NPAGES, i); | |
| 406 | + } | |
| 407 | + spin_unlock(&chunk->lock); | |
| 408 | + } | |
| 409 | + mutex_unlock(&drm->dmem->mutex); | |
| 410 | + | |
| 411 | + return 0; | |
| 412 | +} | |
| 413 | + | |
| 414 | +static struct page * | |
| 415 | +nouveau_dmem_page_alloc_locked(struct nouveau_drm *drm) | |
| 416 | +{ | |
| 417 | + unsigned long pfns[1]; | |
| 418 | + struct page *page; | |
| 419 | + int ret; | |
| 420 | + | |
| 421 | + /* FIXME stop all the miss-match API ... */ | |
| 422 | + ret = nouveau_dmem_pages_alloc(drm, 1, pfns); | |
| 423 | + if (ret) | |
| 424 | + return NULL; | |
| 425 | + | |
| 426 | + page = pfn_to_page(pfns[0]); | |
| 427 | + get_page(page); | |
| 428 | + lock_page(page); | |
| 429 | + return page; | |
| 430 | +} | |
| 431 | + | |
| 432 | +static void | |
| 433 | +nouveau_dmem_page_free_locked(struct nouveau_drm *drm, struct page *page) | |
| 434 | +{ | |
| 435 | + unlock_page(page); | |
| 436 | + put_page(page); | |
| 437 | +} | |
| 438 | + | |
| 439 | +void | |
| 440 | +nouveau_dmem_resume(struct nouveau_drm *drm) | |
| 441 | +{ | |
| 442 | + struct nouveau_dmem_chunk *chunk; | |
| 443 | + int ret; | |
| 444 | + | |
| 445 | + if (drm->dmem == NULL) | |
| 446 | + return; | |
| 447 | + | |
| 448 | + mutex_lock(&drm->dmem->mutex); | |
| 449 | + list_for_each_entry (chunk, &drm->dmem->chunk_free, list) { | |
| 450 | + ret = nouveau_bo_pin(chunk->bo, TTM_PL_FLAG_VRAM, false); | |
| 451 | + /* FIXME handle pin failure */ | |
| 452 | + WARN_ON(ret); | |
| 453 | + } | |
| 454 | + list_for_each_entry (chunk, &drm->dmem->chunk_full, list) { | |
| 455 | + ret = nouveau_bo_pin(chunk->bo, TTM_PL_FLAG_VRAM, false); | |
| 456 | + /* FIXME handle pin failure */ | |
| 457 | + WARN_ON(ret); | |
| 458 | + } | |
| 459 | + list_for_each_entry (chunk, &drm->dmem->chunk_empty, list) { | |
| 460 | + ret = nouveau_bo_pin(chunk->bo, TTM_PL_FLAG_VRAM, false); | |
| 461 | + /* FIXME handle pin failure */ | |
| 462 | + WARN_ON(ret); | |
| 463 | + } | |
| 464 | + mutex_unlock(&drm->dmem->mutex); | |
| 465 | +} | |
| 466 | + | |
| 467 | +void | |
| 468 | +nouveau_dmem_suspend(struct nouveau_drm *drm) | |
| 469 | +{ | |
| 470 | + struct nouveau_dmem_chunk *chunk; | |
| 471 | + | |
| 472 | + if (drm->dmem == NULL) | |
| 473 | + return; | |
| 474 | + | |
| 475 | + mutex_lock(&drm->dmem->mutex); | |
| 476 | + list_for_each_entry (chunk, &drm->dmem->chunk_free, list) { | |
| 477 | + nouveau_bo_unpin(chunk->bo); | |
| 478 | + } | |
| 479 | + list_for_each_entry (chunk, &drm->dmem->chunk_full, list) { | |
| 480 | + nouveau_bo_unpin(chunk->bo); | |
| 481 | + } | |
| 482 | + list_for_each_entry (chunk, &drm->dmem->chunk_empty, list) { | |
| 483 | + nouveau_bo_unpin(chunk->bo); | |
| 484 | + } | |
| 485 | + mutex_unlock(&drm->dmem->mutex); | |
| 486 | +} | |
| 487 | + | |
| 488 | +void | |
| 489 | +nouveau_dmem_fini(struct nouveau_drm *drm) | |
| 490 | +{ | |
| 491 | + struct nouveau_dmem_chunk *chunk, *tmp; | |
| 492 | + | |
| 493 | + if (drm->dmem == NULL) | |
| 494 | + return; | |
| 495 | + | |
| 496 | + mutex_lock(&drm->dmem->mutex); | |
| 497 | + | |
| 498 | + WARN_ON(!list_empty(&drm->dmem->chunk_free)); | |
| 499 | + WARN_ON(!list_empty(&drm->dmem->chunk_full)); | |
| 500 | + | |
| 501 | + list_for_each_entry_safe (chunk, tmp, &drm->dmem->chunk_empty, list) { | |
| 502 | + if (chunk->bo) { | |
| 503 | + nouveau_bo_unpin(chunk->bo); | |
| 504 | + nouveau_bo_ref(NULL, &chunk->bo); | |
| 505 | + } | |
| 506 | + list_del(&chunk->list); | |
| 507 | + kfree(chunk); | |
| 508 | + } | |
| 509 | + | |
| 510 | + mutex_unlock(&drm->dmem->mutex); | |
| 511 | +} | |
| 512 | + | |
| 513 | +static int | |
| 514 | +nvc0b5_migrate_copy(struct nouveau_drm *drm, u64 npages, | |
| 515 | + enum nouveau_aper dst_aper, u64 dst_addr, | |
| 516 | + enum nouveau_aper src_aper, u64 src_addr) | |
| 517 | +{ | |
| 518 | + struct nouveau_channel *chan = drm->dmem->migrate.chan; | |
| 519 | + u32 launch_dma = (1 << 9) /* MULTI_LINE_ENABLE. */ | | |
| 520 | + (1 << 8) /* DST_MEMORY_LAYOUT_PITCH. */ | | |
| 521 | + (1 << 7) /* SRC_MEMORY_LAYOUT_PITCH. */ | | |
| 522 | + (1 << 2) /* FLUSH_ENABLE_TRUE. */ | | |
| 523 | + (2 << 0) /* DATA_TRANSFER_TYPE_NON_PIPELINED. */; | |
| 524 | + int ret; | |
| 525 | + | |
| 526 | + ret = RING_SPACE(chan, 13); | |
| 527 | + if (ret) | |
| 528 | + return ret; | |
| 529 | + | |
| 530 | + if (src_aper != NOUVEAU_APER_VIRT) { | |
| 531 | + switch (src_aper) { | |
| 532 | + case NOUVEAU_APER_VRAM: | |
| 533 | + BEGIN_IMC0(chan, NvSubCopy, 0x0260, 0); | |
| 534 | + break; | |
| 535 | + case NOUVEAU_APER_HOST: | |
| 536 | + BEGIN_IMC0(chan, NvSubCopy, 0x0260, 1); | |
| 537 | + break; | |
| 538 | + default: | |
| 539 | + return -EINVAL; | |
| 540 | + } | |
| 541 | + launch_dma |= 0x00001000; /* SRC_TYPE_PHYSICAL. */ | |
| 542 | + } | |
| 543 | + | |
| 544 | + if (dst_aper != NOUVEAU_APER_VIRT) { | |
| 545 | + switch (dst_aper) { | |
| 546 | + case NOUVEAU_APER_VRAM: | |
| 547 | + BEGIN_IMC0(chan, NvSubCopy, 0x0264, 0); | |
| 548 | + break; | |
| 549 | + case NOUVEAU_APER_HOST: | |
| 550 | + BEGIN_IMC0(chan, NvSubCopy, 0x0264, 1); | |
| 551 | + break; | |
| 552 | + default: | |
| 553 | + return -EINVAL; | |
| 554 | + } | |
| 555 | + launch_dma |= 0x00002000; /* DST_TYPE_PHYSICAL. */ | |
| 556 | + } | |
| 557 | + | |
| 558 | + BEGIN_NVC0(chan, NvSubCopy, 0x0400, 8); | |
| 559 | + OUT_RING (chan, upper_32_bits(src_addr)); | |
| 560 | + OUT_RING (chan, lower_32_bits(src_addr)); | |
| 561 | + OUT_RING (chan, upper_32_bits(dst_addr)); | |
| 562 | + OUT_RING (chan, lower_32_bits(dst_addr)); | |
| 563 | + OUT_RING (chan, PAGE_SIZE); | |
| 564 | + OUT_RING (chan, PAGE_SIZE); | |
| 565 | + OUT_RING (chan, PAGE_SIZE); | |
| 566 | + OUT_RING (chan, npages); | |
| 567 | + BEGIN_NVC0(chan, NvSubCopy, 0x0300, 1); | |
| 568 | + OUT_RING (chan, launch_dma); | |
| 569 | + return 0; | |
| 570 | +} | |
| 571 | + | |
| 572 | +static int | |
| 573 | +nouveau_dmem_migrate_init(struct nouveau_drm *drm) | |
| 574 | +{ | |
| 575 | + switch (drm->ttm.copy.oclass) { | |
| 576 | + case PASCAL_DMA_COPY_A: | |
| 577 | + case PASCAL_DMA_COPY_B: | |
| 578 | + case VOLTA_DMA_COPY_A: | |
| 579 | + case TURING_DMA_COPY_A: | |
| 580 | + drm->dmem->migrate.copy_func = nvc0b5_migrate_copy; | |
| 581 | + drm->dmem->migrate.chan = drm->ttm.chan; | |
| 582 | + return 0; | |
| 583 | + default: | |
| 584 | + break; | |
| 585 | + } | |
| 586 | + return -ENODEV; | |
| 587 | +} | |
| 588 | + | |
| 589 | +void | |
| 590 | +nouveau_dmem_init(struct nouveau_drm *drm) | |
| 591 | +{ | |
| 592 | + struct device *device = drm->dev->dev; | |
| 593 | + unsigned long i, size; | |
| 594 | + int ret; | |
| 595 | + | |
| 596 | + /* This only make sense on PASCAL or newer */ | |
| 597 | + if (drm->client.device.info.family < NV_DEVICE_INFO_V0_PASCAL) | |
| 598 | + return; | |
| 599 | + | |
| 600 | + if (!(drm->dmem = kzalloc(sizeof(*drm->dmem), GFP_KERNEL))) | |
| 601 | + return; | |
| 602 | + | |
| 603 | + mutex_init(&drm->dmem->mutex); | |
| 604 | + INIT_LIST_HEAD(&drm->dmem->chunk_free); | |
| 605 | + INIT_LIST_HEAD(&drm->dmem->chunk_full); | |
| 606 | + INIT_LIST_HEAD(&drm->dmem->chunk_empty); | |
| 607 | + | |
| 608 | + size = ALIGN(drm->client.device.info.ram_user, DMEM_CHUNK_SIZE); | |
| 609 | + | |
| 610 | + /* Initialize migration dma helpers before registering memory */ | |
| 611 | + ret = nouveau_dmem_migrate_init(drm); | |
| 612 | + if (ret) { | |
| 613 | + kfree(drm->dmem); | |
| 614 | + drm->dmem = NULL; | |
| 615 | + return; | |
| 616 | + } | |
| 617 | + | |
| 618 | + /* | |
| 619 | + * FIXME we need some kind of policy to decide how much VRAM we | |
| 620 | + * want to register with HMM. For now just register everything | |
| 621 | + * and latter if we want to do thing like over commit then we | |
| 622 | + * could revisit this. | |
| 623 | + */ | |
| 624 | + drm->dmem->devmem = hmm_devmem_add(&nouveau_dmem_devmem_ops, | |
| 625 | + device, size); | |
| 626 | + if (drm->dmem->devmem == NULL) { | |
| 627 | + kfree(drm->dmem); | |
| 628 | + drm->dmem = NULL; | |
| 629 | + return; | |
| 630 | + } | |
| 631 | + | |
| 632 | + for (i = 0; i < (size / DMEM_CHUNK_SIZE); ++i) { | |
| 633 | + struct nouveau_dmem_chunk *chunk; | |
| 634 | + struct page *page; | |
| 635 | + unsigned long j; | |
| 636 | + | |
| 637 | + chunk = kzalloc(sizeof(*chunk), GFP_KERNEL); | |
| 638 | + if (chunk == NULL) { | |
| 639 | + nouveau_dmem_fini(drm); | |
| 640 | + return; | |
| 641 | + } | |
| 642 | + | |
| 643 | + chunk->drm = drm; | |
| 644 | + chunk->pfn_first = drm->dmem->devmem->pfn_first; | |
| 645 | + chunk->pfn_first += (i * DMEM_CHUNK_NPAGES); | |
| 646 | + list_add_tail(&chunk->list, &drm->dmem->chunk_empty); | |
| 647 | + | |
| 648 | + page = pfn_to_page(chunk->pfn_first); | |
| 649 | + for (j = 0; j < DMEM_CHUNK_NPAGES; ++j, ++page) { | |
| 650 | + hmm_devmem_page_set_drvdata(page, (long)chunk); | |
| 651 | + } | |
| 652 | + } | |
| 653 | + | |
| 654 | + NV_INFO(drm, "DMEM: registered %ldMB of device memory\n", size >> 20); | |
| 655 | +} | |
| 656 | + | |
| 657 | +static void | |
| 658 | +nouveau_dmem_migrate_alloc_and_copy(struct vm_area_struct *vma, | |
| 659 | + const unsigned long *src_pfns, | |
| 660 | + unsigned long *dst_pfns, | |
| 661 | + unsigned long start, | |
| 662 | + unsigned long end, | |
| 663 | + void *private) | |
| 664 | +{ | |
| 665 | + struct nouveau_migrate *migrate = private; | |
| 666 | + struct nouveau_drm *drm = migrate->drm; | |
| 667 | + struct device *dev = drm->dev->dev; | |
| 668 | + unsigned long addr, i, npages = 0; | |
| 669 | + nouveau_migrate_copy_t copy; | |
| 670 | + int ret; | |
| 671 | + | |
| 672 | + /* First allocate new memory */ | |
| 673 | + for (addr = start, i = 0; addr < end; addr += PAGE_SIZE, i++) { | |
| 674 | + struct page *dpage, *spage; | |
| 675 | + | |
| 676 | + dst_pfns[i] = 0; | |
| 677 | + spage = migrate_pfn_to_page(src_pfns[i]); | |
| 678 | + if (!spage || !(src_pfns[i] & MIGRATE_PFN_MIGRATE)) | |
| 679 | + continue; | |
| 680 | + | |
| 681 | + dpage = nouveau_dmem_page_alloc_locked(drm); | |
| 682 | + if (!dpage) | |
| 683 | + continue; | |
| 684 | + | |
| 685 | + dst_pfns[i] = migrate_pfn(page_to_pfn(dpage)) | | |
| 686 | + MIGRATE_PFN_LOCKED | | |
| 687 | + MIGRATE_PFN_DEVICE; | |
| 688 | + npages++; | |
| 689 | + } | |
| 690 | + | |
| 691 | + if (!npages) | |
| 692 | + return; | |
| 693 | + | |
| 694 | + /* Allocate storage for DMA addresses, so we can unmap later. */ | |
| 695 | + migrate->dma = kmalloc(sizeof(*migrate->dma) * npages, GFP_KERNEL); | |
| 696 | + if (!migrate->dma) | |
| 697 | + goto error; | |
| 698 | + | |
| 699 | + /* Copy things over */ | |
| 700 | + copy = drm->dmem->migrate.copy_func; | |
| 701 | + for (addr = start, i = 0; addr < end; addr += PAGE_SIZE, i++) { | |
| 702 | + struct nouveau_dmem_chunk *chunk; | |
| 703 | + struct page *spage, *dpage; | |
| 704 | + u64 src_addr, dst_addr; | |
| 705 | + | |
| 706 | + dpage = migrate_pfn_to_page(dst_pfns[i]); | |
| 707 | + if (!dpage || dst_pfns[i] == MIGRATE_PFN_ERROR) | |
| 708 | + continue; | |
| 709 | + | |
| 710 | + chunk = (void *)hmm_devmem_page_get_drvdata(dpage); | |
| 711 | + dst_addr = page_to_pfn(dpage) - chunk->pfn_first; | |
| 712 | + dst_addr = (dst_addr << PAGE_SHIFT) + chunk->bo->bo.offset; | |
| 713 | + | |
| 714 | + spage = migrate_pfn_to_page(src_pfns[i]); | |
| 715 | + if (!spage || !(src_pfns[i] & MIGRATE_PFN_MIGRATE)) { | |
| 716 | + nouveau_dmem_page_free_locked(drm, dpage); | |
| 717 | + dst_pfns[i] = 0; | |
| 718 | + continue; | |
| 719 | + } | |
| 720 | + | |
| 721 | + migrate->dma[migrate->dma_nr] = | |
| 722 | + dma_map_page_attrs(dev, spage, 0, PAGE_SIZE, | |
| 723 | + PCI_DMA_BIDIRECTIONAL, | |
| 724 | + DMA_ATTR_SKIP_CPU_SYNC); | |
| 725 | + if (dma_mapping_error(dev, migrate->dma[migrate->dma_nr])) { | |
| 726 | + nouveau_dmem_page_free_locked(drm, dpage); | |
| 727 | + dst_pfns[i] = 0; | |
| 728 | + continue; | |
| 729 | + } | |
| 730 | + | |
| 731 | + src_addr = migrate->dma[migrate->dma_nr++]; | |
| 732 | + | |
| 733 | + ret = copy(drm, 1, NOUVEAU_APER_VRAM, dst_addr, | |
| 734 | + NOUVEAU_APER_HOST, src_addr); | |
| 735 | + if (ret) { | |
| 736 | + nouveau_dmem_page_free_locked(drm, dpage); | |
| 737 | + dst_pfns[i] = 0; | |
| 738 | + continue; | |
| 739 | + } | |
| 740 | + } | |
| 741 | + | |
| 742 | + nouveau_fence_new(drm->dmem->migrate.chan, false, &migrate->fence); | |
| 743 | + | |
| 744 | + return; | |
| 745 | + | |
| 746 | +error: | |
| 747 | + for (addr = start, i = 0; addr < end; addr += PAGE_SIZE, ++i) { | |
| 748 | + struct page *page; | |
| 749 | + | |
| 750 | + if (!dst_pfns[i] || dst_pfns[i] == MIGRATE_PFN_ERROR) | |
| 751 | + continue; | |
| 752 | + | |
| 753 | + page = migrate_pfn_to_page(dst_pfns[i]); | |
| 754 | + dst_pfns[i] = MIGRATE_PFN_ERROR; | |
| 755 | + if (page == NULL) | |
| 756 | + continue; | |
| 757 | + | |
| 758 | + __free_page(page); | |
| 759 | + } | |
| 760 | +} | |
| 761 | + | |
| 762 | +void nouveau_dmem_migrate_finalize_and_map(struct vm_area_struct *vma, | |
| 763 | + const unsigned long *src_pfns, | |
| 764 | + const unsigned long *dst_pfns, | |
| 765 | + unsigned long start, | |
| 766 | + unsigned long end, | |
| 767 | + void *private) | |
| 768 | +{ | |
| 769 | + struct nouveau_migrate *migrate = private; | |
| 770 | + struct nouveau_drm *drm = migrate->drm; | |
| 771 | + | |
| 772 | + if (migrate->fence) { | |
| 773 | + nouveau_fence_wait(migrate->fence, true, false); | |
| 774 | + nouveau_fence_unref(&migrate->fence); | |
| 775 | + } else { | |
| 776 | + /* | |
| 777 | + * FIXME wait for channel to be IDLE before finalizing | |
| 778 | + * the hmem object below (nouveau_migrate_hmem_fini()) ? | |
| 779 | + */ | |
| 780 | + } | |
| 781 | + | |
| 782 | + while (migrate->dma_nr--) { | |
| 783 | + dma_unmap_page(drm->dev->dev, migrate->dma[migrate->dma_nr], | |
| 784 | + PAGE_SIZE, PCI_DMA_BIDIRECTIONAL); | |
| 785 | + } | |
| 786 | + kfree(migrate->dma); | |
| 787 | + | |
| 788 | + /* | |
| 789 | + * FIXME optimization: update GPU page table to point to newly | |
| 790 | + * migrated memory. | |
| 791 | + */ | |
| 792 | +} | |
| 793 | + | |
| 794 | +static const struct migrate_vma_ops nouveau_dmem_migrate_ops = { | |
| 795 | + .alloc_and_copy = nouveau_dmem_migrate_alloc_and_copy, | |
| 796 | + .finalize_and_map = nouveau_dmem_migrate_finalize_and_map, | |
| 797 | +}; | |
| 798 | + | |
| 799 | +int | |
| 800 | +nouveau_dmem_migrate_vma(struct nouveau_drm *drm, | |
| 801 | + struct vm_area_struct *vma, | |
| 802 | + unsigned long start, | |
| 803 | + unsigned long end) | |
| 804 | +{ | |
| 805 | + unsigned long *src_pfns, *dst_pfns, npages; | |
| 806 | + struct nouveau_migrate migrate = {0}; | |
| 807 | + unsigned long i, c, max; | |
| 808 | + int ret = 0; | |
| 809 | + | |
| 810 | + npages = (end - start) >> PAGE_SHIFT; | |
| 811 | + max = min(SG_MAX_SINGLE_ALLOC, npages); | |
| 812 | + src_pfns = kzalloc(sizeof(long) * max, GFP_KERNEL); | |
| 813 | + if (src_pfns == NULL) | |
| 814 | + return -ENOMEM; | |
| 815 | + dst_pfns = kzalloc(sizeof(long) * max, GFP_KERNEL); | |
| 816 | + if (dst_pfns == NULL) { | |
| 817 | + kfree(src_pfns); | |
| 818 | + return -ENOMEM; | |
| 819 | + } | |
| 820 | + | |
| 821 | + migrate.drm = drm; | |
| 822 | + migrate.vma = vma; | |
| 823 | + migrate.npages = npages; | |
| 824 | + for (i = 0; i < npages; i += c) { | |
| 825 | + unsigned long next; | |
| 826 | + | |
| 827 | + c = min(SG_MAX_SINGLE_ALLOC, npages); | |
| 828 | + next = start + (c << PAGE_SHIFT); | |
| 829 | + ret = migrate_vma(&nouveau_dmem_migrate_ops, vma, start, | |
| 830 | + next, src_pfns, dst_pfns, &migrate); | |
| 831 | + if (ret) | |
| 832 | + goto out; | |
| 833 | + start = next; | |
| 834 | + } | |
| 835 | + | |
| 836 | +out: | |
| 837 | + kfree(dst_pfns); | |
| 838 | + kfree(src_pfns); | |
| 839 | + return ret; | |
| 840 | +} | |
| 841 | + | |
| 842 | +static inline bool | |
| 843 | +nouveau_dmem_page(struct nouveau_drm *drm, struct page *page) | |
| 844 | +{ | |
| 845 | + if (!is_device_private_page(page)) | |
| 846 | + return false; | |
| 847 | + | |
| 848 | + if (drm->dmem->devmem != page->pgmap->data) | |
| 849 | + return false; | |
| 850 | + | |
| 851 | + return true; | |
| 852 | +} | |
| 853 | + | |
| 854 | +void | |
| 855 | +nouveau_dmem_convert_pfn(struct nouveau_drm *drm, | |
| 856 | + struct hmm_range *range) | |
| 857 | +{ | |
| 858 | + unsigned long i, npages; | |
| 859 | + | |
| 860 | + npages = (range->end - range->start) >> PAGE_SHIFT; | |
| 861 | + for (i = 0; i < npages; ++i) { | |
| 862 | + struct nouveau_dmem_chunk *chunk; | |
| 863 | + struct page *page; | |
| 864 | + uint64_t addr; | |
| 865 | + | |
| 866 | + page = hmm_pfn_to_page(range, range->pfns[i]); | |
| 867 | + if (page == NULL) | |
| 868 | + continue; | |
| 869 | + | |
| 870 | + if (!(range->pfns[i] & range->flags[HMM_PFN_DEVICE_PRIVATE])) { | |
| 871 | + continue; | |
| 872 | + } | |
| 873 | + | |
| 874 | + if (!nouveau_dmem_page(drm, page)) { | |
| 875 | + WARN(1, "Some unknown device memory !\n"); | |
| 876 | + range->pfns[i] = 0; | |
| 877 | + continue; | |
| 878 | + } | |
| 879 | + | |
| 880 | + chunk = (void *)hmm_devmem_page_get_drvdata(page); | |
| 881 | + addr = page_to_pfn(page) - chunk->pfn_first; | |
| 882 | + addr = (addr + chunk->bo->bo.mem.start) << PAGE_SHIFT; | |
| 883 | + | |
| 884 | + range->pfns[i] &= ((1UL << range->pfn_shift) - 1); | |
| 885 | + range->pfns[i] |= (addr >> PAGE_SHIFT) << range->pfn_shift; | |
| 886 | + } | |
| 887 | +} | |
| @@ -0,0 +1,60 @@ | ||
| 1 | +/* | |
| 2 | + * Copyright 2018 Red Hat Inc. | |
| 3 | + * | |
| 4 | + * Permission is hereby granted, free of charge, to any person obtaining a | |
| 5 | + * copy of this software and associated documentation files (the "Software"), | |
| 6 | + * to deal in the Software without restriction, including without limitation | |
| 7 | + * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 8 | + * and/or sell copies of the Software, and to permit persons to whom the | |
| 9 | + * Software is furnished to do so, subject to the following conditions: | |
| 10 | + * | |
| 11 | + * The above copyright notice and this permission notice shall be included in | |
| 12 | + * all copies or substantial portions of the Software. | |
| 13 | + * | |
| 14 | + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 15 | + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 16 | + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 17 | + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 18 | + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 19 | + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 20 | + * OTHER DEALINGS IN THE SOFTWARE. | |
| 21 | + */ | |
| 22 | +#ifndef __NOUVEAU_DMEM_H__ | |
| 23 | +#define __NOUVEAU_DMEM_H__ | |
| 24 | +#include <nvif/os.h> | |
| 25 | +struct drm_device; | |
| 26 | +struct drm_file; | |
| 27 | +struct nouveau_drm; | |
| 28 | +struct hmm_range; | |
| 29 | + | |
| 30 | +#if IS_ENABLED(CONFIG_DRM_NOUVEAU_SVM) | |
| 31 | +void nouveau_dmem_init(struct nouveau_drm *); | |
| 32 | +void nouveau_dmem_fini(struct nouveau_drm *); | |
| 33 | +void nouveau_dmem_suspend(struct nouveau_drm *); | |
| 34 | +void nouveau_dmem_resume(struct nouveau_drm *); | |
| 35 | + | |
| 36 | +int nouveau_dmem_migrate_vma(struct nouveau_drm *drm, | |
| 37 | + struct vm_area_struct *vma, | |
| 38 | + unsigned long start, | |
| 39 | + unsigned long end); | |
| 40 | + | |
| 41 | +void nouveau_dmem_convert_pfn(struct nouveau_drm *drm, | |
| 42 | + struct hmm_range *range); | |
| 43 | +#else /* IS_ENABLED(CONFIG_DRM_NOUVEAU_SVM) */ | |
| 44 | +static inline void nouveau_dmem_init(struct nouveau_drm *drm) {} | |
| 45 | +static inline void nouveau_dmem_fini(struct nouveau_drm *drm) {} | |
| 46 | +static inline void nouveau_dmem_suspend(struct nouveau_drm *drm) {} | |
| 47 | +static inline void nouveau_dmem_resume(struct nouveau_drm *drm) {} | |
| 48 | + | |
| 49 | +static inline int nouveau_dmem_migrate_vma(struct nouveau_drm *drm, | |
| 50 | + struct vm_area_struct *vma, | |
| 51 | + unsigned long start, | |
| 52 | + unsigned long end) | |
| 53 | +{ | |
| 54 | + return 0; | |
| 55 | +} | |
| 56 | + | |
| 57 | +static inline void nouveau_dmem_convert_pfn(struct nouveau_drm *drm, | |
| 58 | + struct hmm_range *range) {} | |
| 59 | +#endif /* IS_ENABLED(CONFIG_DRM_NOUVEAU_SVM) */ | |
| 60 | +#endif | |
| @@ -44,7 +44,6 @@ · 43 unmodified lines | ||
| 44 | 44 | #include <nvif/class.h> |
| 45 | 45 | #include <nvif/cl0002.h> |
| 46 | 46 | #include <nvif/cla06f.h> |
| 47 | −#include <nvif/if0004.h> | |
| 48 | 47 | |
| 49 | 48 | #include "nouveau_drv.h" |
| 50 | 49 | #include "nouveau_dma.h" |
| @@ -63,6 +62,8 @@ · 12 unmodified lines | ||
| 63 | 62 | #include "nouveau_usif.h" |
| 64 | 63 | #include "nouveau_connector.h" |
| 65 | 64 | #include "nouveau_platform.h" |
| 65 | +#include "nouveau_svm.h" | |
| 66 | +#include "nouveau_dmem.h" | |
| 66 | 67 | |
| 67 | 68 | MODULE_PARM_DESC(config, "option string to pass to driver core"); |
| 68 | 69 | static char *nouveau_config; |
| @@ -173,6 +174,7 @@ nouveau_cli_fini(struct nouveau_cli *cli) · 104 unmodified lines | ||
| 173 | 174 | WARN_ON(!list_empty(&cli->worker)); |
| 174 | 175 | |
| 175 | 176 | usif_client_fini(cli); |
| 177 | + nouveau_vmm_fini(&cli->svm); | |
| 176 | 178 | nouveau_vmm_fini(&cli->vmm); |
| 177 | 179 | nvif_mmu_fini(&cli->mmu); |
| 178 | 180 | nvif_device_fini(&cli->device); |
| @@ -283,19 +285,134 @@ done: · 104 unmodified lines | ||
| 283 | 285 | } |
| 284 | 286 | |
| 285 | 287 | static void |
| 286 | −nouveau_accel_fini(struct nouveau_drm *drm) | |
| 288 | +nouveau_accel_ce_fini(struct nouveau_drm *drm) | |
| 289 | +{ | |
| 290 | + nouveau_channel_idle(drm->cechan); | |
| 291 | + nvif_object_fini(&drm->ttm.copy); | |
| 292 | + nouveau_channel_del(&drm->cechan); | |
| 293 | +} | |
| 294 | + | |
| 295 | +static void | |
| 296 | +nouveau_accel_ce_init(struct nouveau_drm *drm) | |
| 297 | +{ | |
| 298 | + struct nvif_device *device = &drm->client.device; | |
| 299 | + int ret = 0; | |
| 300 | + | |
| 301 | + /* Allocate channel that has access to a (preferably async) copy | |
| 302 | + * engine, to use for TTM buffer moves. | |
| 303 | + */ | |
| 304 | + if (device->info.family >= NV_DEVICE_INFO_V0_KEPLER) { | |
| 305 | + ret = nouveau_channel_new(drm, device, | |
| 306 | + nvif_fifo_runlist_ce(device), 0, | |
| 307 | + true, &drm->cechan); | |
| 308 | + } else | |
| 309 | + if (device->info.chipset >= 0xa3 && | |
| 310 | + device->info.chipset != 0xaa && | |
| 311 | + device->info.chipset != 0xac) { | |
| 312 | + /* Prior to Kepler, there's only a single runlist, so all | |
| 313 | + * engines can be accessed from any channel. | |
| 314 | + * | |
| 315 | + * We still want to use a separate channel though. | |
| 316 | + */ | |
| 317 | + ret = nouveau_channel_new(drm, device, NvDmaFB, NvDmaTT, false, | |
| 318 | + &drm->cechan); | |
| 319 | + } | |
| 320 | + | |
| 321 | + if (ret) | |
| 322 | + NV_ERROR(drm, "failed to create ce channel, %d\n", ret); | |
| 323 | +} | |
| 324 | + | |
| 325 | +static void | |
| 326 | +nouveau_accel_gr_fini(struct nouveau_drm *drm) | |
| 287 | 327 | { |
| 288 | 328 | nouveau_channel_idle(drm->channel); |
| 289 | 329 | nvif_object_fini(&drm->ntfy); |
| 290 | 330 | nvkm_gpuobj_del(&drm->notify); |
| 291 | − nvif_notify_fini(&drm->flip); | |
| 292 | 331 | nvif_object_fini(&drm->nvsw); |
| 293 | 332 | nouveau_channel_del(&drm->channel); |
| 333 | +} | |
| 294 | 334 | |
| 295 | − nouveau_channel_idle(drm->cechan); | |
| 296 | − nvif_object_fini(&drm->ttm.copy); | |
| 297 | − nouveau_channel_del(&drm->cechan); | |
| 335 | +static void | |
| 336 | +nouveau_accel_gr_init(struct nouveau_drm *drm) | |
| 337 | +{ | |
| 338 | + struct nvif_device *device = &drm->client.device; | |
| 339 | + u32 arg0, arg1; | |
| 340 | + int ret; | |
| 341 | + | |
| 342 | + /* Allocate channel that has access to the graphics engine. */ | |
| 343 | + if (device->info.family >= NV_DEVICE_INFO_V0_KEPLER) { | |
| 344 | + arg0 = nvif_fifo_runlist(device, NV_DEVICE_INFO_ENGINE_GR); | |
| 345 | + arg1 = 1; | |
| 346 | + } else { | |
| 347 | + arg0 = NvDmaFB; | |
| 348 | + arg1 = NvDmaTT; | |
| 349 | + } | |
| 298 | 350 | |
| 351 | + ret = nouveau_channel_new(drm, device, arg0, arg1, false, | |
| 352 | + &drm->channel); | |
| 353 | + if (ret) { | |
| 354 | + NV_ERROR(drm, "failed to create kernel channel, %d\n", ret); | |
| 355 | + nouveau_accel_gr_fini(drm); | |
| 356 | + return; | |
| 357 | + } | |
| 358 | + | |
| 359 | + /* A SW class is used on pre-NV50 HW to assist with handling the | |
| 360 | + * synchronisation of page flips, as well as to implement fences | |
| 361 | + * on TNT/TNT2 HW that lacks any kind of support in host. | |
| 362 | + */ | |
| 363 | + if (device->info.family < NV_DEVICE_INFO_V0_TESLA) { | |
| 364 | + ret = nvif_object_init(&drm->channel->user, NVDRM_NVSW, | |
| 365 | + nouveau_abi16_swclass(drm), NULL, 0, | |
| 366 | + &drm->nvsw); | |
| 367 | + if (ret == 0) { | |
| 368 | + ret = RING_SPACE(drm->channel, 2); | |
| 369 | + if (ret == 0) { | |
| 370 | + BEGIN_NV04(drm->channel, NvSubSw, 0, 1); | |
| 371 | + OUT_RING (drm->channel, drm->nvsw.handle); | |
| 372 | + } | |
| 373 | + } | |
| 374 | + | |
| 375 | + if (ret) { | |
| 376 | + NV_ERROR(drm, "failed to allocate sw class, %d\n", ret); | |
| 377 | + nouveau_accel_gr_fini(drm); | |
| 378 | + return; | |
| 379 | + } | |
| 380 | + } | |
| 381 | + | |
| 382 | + /* NvMemoryToMemoryFormat requires a notifier ctxdma for some reason, | |
| 383 | + * even if notification is never requested, so, allocate a ctxdma on | |
| 384 | + * any GPU where it's possible we'll end up using M2MF for BO moves. | |
| 385 | + */ | |
| 386 | + if (device->info.family < NV_DEVICE_INFO_V0_FERMI) { | |
| 387 | + ret = nvkm_gpuobj_new(nvxx_device(device), 32, 0, false, NULL, | |
| 388 | + &drm->notify); | |
| 389 | + if (ret) { | |
| 390 | + NV_ERROR(drm, "failed to allocate notifier, %d\n", ret); | |
| 391 | + nouveau_accel_gr_fini(drm); | |
| 392 | + return; | |
| 393 | + } | |
| 394 | + | |
| 395 | + ret = nvif_object_init(&drm->channel->user, NvNotify0, | |
| 396 | + NV_DMA_IN_MEMORY, | |
| 397 | + &(struct nv_dma_v0) { | |
| 398 | + .target = NV_DMA_V0_TARGET_VRAM, | |
| 399 | + .access = NV_DMA_V0_ACCESS_RDWR, | |
| 400 | + .start = drm->notify->addr, | |
| 401 | + .limit = drm->notify->addr + 31 | |
| 402 | + }, sizeof(struct nv_dma_v0), | |
| 403 | + &drm->ntfy); | |
| 404 | + if (ret) { | |
| 405 | + nouveau_accel_gr_fini(drm); | |
| 406 | + return; | |
| 407 | + } | |
| 408 | + } | |
| 409 | +} | |
| 410 | + | |
| 411 | +static void | |
| 412 | +nouveau_accel_fini(struct nouveau_drm *drm) | |
| 413 | +{ | |
| 414 | + nouveau_accel_ce_fini(drm); | |
| 415 | + nouveau_accel_gr_fini(drm); | |
| 299 | 416 | if (drm->fence) |
| 300 | 417 | nouveau_fence(drm)->dtor(drm); |
| 301 | 418 | } |
| @@ -305,23 +422,16 @@ nouveau_accel_init(struct nouveau_drm *drm) · 3 unmodified lines | ||
| 305 | 422 | { |
| 306 | 423 | struct nvif_device *device = &drm->client.device; |
| 307 | 424 | struct nvif_sclass *sclass; |
| 308 | − u32 arg0, arg1; | |
| 309 | 425 | int ret, i, n; |
| 310 | 426 | |
| 311 | 427 | if (nouveau_noaccel) |
| 312 | 428 | return; |
| 313 | 429 | |
| 430 | + /* Initialise global support for channels, and synchronisation. */ | |
| 314 | 431 | ret = nouveau_channels_init(drm); |
| 315 | 432 | if (ret) |
| 316 | 433 | return; |
| 317 | 434 | |
| 318 | − if (drm->client.device.info.family >= NV_DEVICE_INFO_V0_VOLTA) { | |
| 319 | − ret = nvif_user_init(device); | |
| 320 | − if (ret) | |
| 321 | − return; | |
| 322 | − } | |
| 323 | − | |
| 324 | − /* initialise synchronisation routines */ | |
| 325 | 435 | /*XXX: this is crap, but the fence/channel stuff is a little |
| 326 | 436 | * backwards in some places. this will be fixed. |
| 327 | 437 | */ |
| @@ -368,95 +478,18 @@ nouveau_accel_init(struct nouveau_drm *drm) · 40 unmodified lines | ||
| 368 | 478 | return; |
| 369 | 479 | } |
| 370 | 480 | |
| 371 | − if (device->info.family >= NV_DEVICE_INFO_V0_KEPLER) { | |
| 372 | − ret = nouveau_channel_new(drm, &drm->client.device, | |
| 373 | − nvif_fifo_runlist_ce(device), 0, | |
| 374 | − true, &drm->cechan); | |
| 375 | − if (ret) | |
| 376 | − NV_ERROR(drm, "failed to create ce channel, %d\n", ret); | |
| 377 | − | |
| 378 | − arg0 = nvif_fifo_runlist(device, NV_DEVICE_INFO_ENGINE_GR); | |
| 379 | − arg1 = 1; | |
| 380 | − } else | |
| 381 | − if (device->info.chipset >= 0xa3 && | |
| 382 | − device->info.chipset != 0xaa && | |
| 383 | − device->info.chipset != 0xac) { | |
| 384 | − ret = nouveau_channel_new(drm, &drm->client.device, | |
| 385 | − NvDmaFB, NvDmaTT, false, | |
| 386 | − &drm->cechan); | |
| 481 | + /* Volta requires access to a doorbell register for kickoff. */ | |
| 482 | + if (drm->client.device.info.family >= NV_DEVICE_INFO_V0_VOLTA) { | |
| 483 | + ret = nvif_user_init(device); | |
| 387 | 484 | if (ret) |
| 388 | − NV_ERROR(drm, "failed to create ce channel, %d\n", ret); | |
| 389 | − | |
| 390 | − arg0 = NvDmaFB; | |
| 391 | − arg1 = NvDmaTT; | |
| 392 | − } else { | |
| 393 | − arg0 = NvDmaFB; | |
| 394 | − arg1 = NvDmaTT; | |
| 395 | − } | |
| 396 | − | |
| 397 | − ret = nouveau_channel_new(drm, &drm->client.device, | |
| 398 | − arg0, arg1, false, &drm->channel); | |
| 399 | − if (ret) { | |
| 400 | − NV_ERROR(drm, "failed to create kernel channel, %d\n", ret); | |
| 401 | − nouveau_accel_fini(drm); | |
| 402 | − return; | |
| 403 | − } | |
| 404 | − | |
| 405 | − if (device->info.family < NV_DEVICE_INFO_V0_TESLA) { | |
| 406 | − ret = nvif_object_init(&drm->channel->user, NVDRM_NVSW, | |
| 407 | − nouveau_abi16_swclass(drm), NULL, 0, | |
| 408 | − &drm->nvsw); | |
| 409 | − if (ret == 0) { | |
| 410 | − ret = RING_SPACE(drm->channel, 2); | |
| 411 | − if (ret == 0) { | |
| 412 | − BEGIN_NV04(drm->channel, NvSubSw, 0, 1); | |
| 413 | − OUT_RING (drm->channel, drm->nvsw.handle); | |
| 414 | − } | |
| 415 | − | |
| 416 | − ret = nvif_notify_init(&drm->nvsw, | |
| 417 | − nouveau_flip_complete, | |
| 418 | − false, NV04_NVSW_NTFY_UEVENT, | |
| 419 | − NULL, 0, 0, &drm->flip); | |
| 420 | − if (ret == 0) | |
| 421 | − ret = nvif_notify_get(&drm->flip); | |
| 422 | − if (ret) { | |
| 423 | − nouveau_accel_fini(drm); | |
| 424 | − return; | |
| 425 | − } | |
| 426 | − } | |
| 427 | − | |
| 428 | − if (ret) { | |
| 429 | − NV_ERROR(drm, "failed to allocate sw class, %d\n", ret); | |
| 430 | − nouveau_accel_fini(drm); | |
| 431 | − return; | |
| 432 | − } | |
| 433 | − } | |
| 434 | − | |
| 435 | − if (device->info.family < NV_DEVICE_INFO_V0_FERMI) { | |
| 436 | − ret = nvkm_gpuobj_new(nvxx_device(&drm->client.device), 32, 0, | |
| 437 | − false, NULL, &drm->notify); | |
| 438 | − if (ret) { | |
| 439 | − NV_ERROR(drm, "failed to allocate notifier, %d\n", ret); | |
| 440 | − nouveau_accel_fini(drm); | |
| 441 | 485 | return; |
| 442 | − } | |
| 443 | − | |
| 444 | − ret = nvif_object_init(&drm->channel->user, NvNotify0, | |
| 445 | − NV_DMA_IN_MEMORY, | |
| 446 | − &(struct nv_dma_v0) { | |
| 447 | − .target = NV_DMA_V0_TARGET_VRAM, | |
| 448 | − .access = NV_DMA_V0_ACCESS_RDWR, | |
| 449 | − .start = drm->notify->addr, | |
| 450 | − .limit = drm->notify->addr + 31 | |
| 451 | − }, sizeof(struct nv_dma_v0), | |
| 452 | − &drm->ntfy); | |
| 453 | − if (ret) { | |
| 454 | − nouveau_accel_fini(drm); | |
| 455 | − return; | |
| 456 | − } | |
| 457 | 486 | } |
| 458 | 487 | |
| 488 | + /* Allocate channels we need to support various functions. */ | |
| 489 | + nouveau_accel_gr_init(drm); | |
| 490 | + nouveau_accel_ce_init(drm); | |
| 459 | 491 | |
| 492 | + /* Initialise accelerated TTM buffer moves. */ | |
| 460 | 493 | nouveau_bo_move_init(drm); |
| 461 | 494 | } |
| 462 | 495 | |
| @@ -504,19 +537,22 @@ nouveau_drm_device_init(struct drm_device *dev) · 41 unmodified lines | ||
| 504 | 537 | if (ret) |
| 505 | 538 | goto fail_bios; |
| 506 | 539 | |
| 540 | + nouveau_accel_init(drm); | |
| 541 | + | |
| 507 | 542 | ret = nouveau_display_create(dev); |
| 508 | 543 | if (ret) |
| 509 | 544 | goto fail_dispctor; |
| 510 | 545 | |
| 511 | 546 | if (dev->mode_config.num_crtc) { |
| 512 | − ret = nouveau_display_init(dev); | |
| 547 | + ret = nouveau_display_init(dev, false, false); | |
| 513 | 548 | if (ret) |
| 514 | 549 | goto fail_dispinit; |
| 515 | 550 | } |
| 516 | 551 | |
| 517 | 552 | nouveau_debugfs_init(drm); |
| 518 | 553 | nouveau_hwmon_init(dev); |
| 519 | − nouveau_accel_init(drm); | |
| 554 | + nouveau_svm_init(drm); | |
| 555 | + nouveau_dmem_init(drm); | |
| 520 | 556 | nouveau_fbcon_init(dev); |
| 521 | 557 | nouveau_led_init(dev); |
| 522 | 558 | |
| @@ -534,6 +570,7 @@ nouveau_drm_device_init(struct drm_device *dev) · 11 unmodified lines | ||
| 534 | 570 | fail_dispinit: |
| 535 | 571 | nouveau_display_destroy(dev); |
| 536 | 572 | fail_dispctor: |
| 573 | + nouveau_accel_fini(drm); | |
| 537 | 574 | nouveau_bios_takedown(dev); |
| 538 | 575 | fail_bios: |
| 539 | 576 | nouveau_ttm_fini(drm); |
| @@ -559,7 +596,8 @@ nouveau_drm_device_fini(struct drm_device *dev) · 19 unmodified lines | ||
| 559 | 596 | |
| 560 | 597 | nouveau_led_fini(dev); |
| 561 | 598 | nouveau_fbcon_fini(dev); |
| 562 | − nouveau_accel_fini(drm); | |
| 599 | + nouveau_dmem_fini(drm); | |
| 600 | + nouveau_svm_fini(drm); | |
| 563 | 601 | nouveau_hwmon_fini(dev); |
| 564 | 602 | nouveau_debugfs_fini(drm); |
| 565 | 603 | |
| @@ -567,6 +605,7 @@ nouveau_drm_device_fini(struct drm_device *dev) · 1 unmodified line | ||
| 567 | 605 | nouveau_display_fini(dev, false, false); |
| 568 | 606 | nouveau_display_destroy(dev); |
| 569 | 607 | |
| 608 | + nouveau_accel_fini(drm); | |
| 570 | 609 | nouveau_bios_takedown(dev); |
| 571 | 610 | |
| 572 | 611 | nouveau_ttm_fini(drm); |
| @@ -704,6 +743,8 @@ nouveau_do_suspend(struct drm_device *dev, bool runtime) · 131 unmodified lines | ||
| 704 | 743 | struct nouveau_drm *drm = nouveau_drm(dev); |
| 705 | 744 | int ret; |
| 706 | 745 | |
| 746 | + nouveau_svm_suspend(drm); | |
| 747 | + nouveau_dmem_suspend(drm); | |
| 707 | 748 | nouveau_led_suspend(dev); |
| 708 | 749 | |
| 709 | 750 | if (dev->mode_config.num_crtc) { |
| @@ -780,7 +821,8 @@ nouveau_do_resume(struct drm_device *dev, bool runtime) · 70 unmodified lines | ||
| 780 | 821 | } |
| 781 | 822 | |
| 782 | 823 | nouveau_led_resume(dev); |
| 783 | − | |
| 824 | + nouveau_dmem_resume(drm); | |
| 825 | + nouveau_svm_resume(drm); | |
| 784 | 826 | return 0; |
| 785 | 827 | } |
| 786 | 828 | |
| @@ -1000,6 +1042,8 @@ nouveau_ioctls[] = { · 213 unmodified lines | ||
| 1000 | 1042 | DRM_IOCTL_DEF_DRV(NOUVEAU_GROBJ_ALLOC, nouveau_abi16_ioctl_grobj_alloc, DRM_AUTH|DRM_RENDER_ALLOW), |
| 1001 | 1043 | DRM_IOCTL_DEF_DRV(NOUVEAU_NOTIFIEROBJ_ALLOC, nouveau_abi16_ioctl_notifierobj_alloc, DRM_AUTH|DRM_RENDER_ALLOW), |
| 1002 | 1044 | DRM_IOCTL_DEF_DRV(NOUVEAU_GPUOBJ_FREE, nouveau_abi16_ioctl_gpuobj_free, DRM_AUTH|DRM_RENDER_ALLOW), |
| 1045 | + DRM_IOCTL_DEF_DRV(NOUVEAU_SVM_INIT, nouveau_svmm_init, DRM_AUTH|DRM_RENDER_ALLOW), | |
| 1046 | + DRM_IOCTL_DEF_DRV(NOUVEAU_SVM_BIND, nouveau_svmm_bind, DRM_AUTH|DRM_RENDER_ALLOW), | |
| 1003 | 1047 | DRM_IOCTL_DEF_DRV(NOUVEAU_GEM_NEW, nouveau_gem_ioctl_new, DRM_AUTH|DRM_RENDER_ALLOW), |
| 1004 | 1048 | DRM_IOCTL_DEF_DRV(NOUVEAU_GEM_PUSHBUF, nouveau_gem_ioctl_pushbuf, DRM_AUTH|DRM_RENDER_ALLOW), |
| 1005 | 1049 | DRM_IOCTL_DEF_DRV(NOUVEAU_GEM_CPU_PREP, nouveau_gem_ioctl_cpu_prep, DRM_AUTH|DRM_RENDER_ALLOW), |
| @@ -96,6 +96,7 @@ struct nouveau_cli { · 95 unmodified lines | ||
| 96 | 96 | struct nvif_device device; |
| 97 | 97 | struct nvif_mmu mmu; |
| 98 | 98 | struct nouveau_vmm vmm; |
| 99 | + struct nouveau_vmm svm; | |
| 99 | 100 | const struct nvif_mclass *mem; |
| 100 | 101 | |
| 101 | 102 | struct list_head head; |
| @@ -181,7 +182,6 @@ struct nouveau_drm { · 79 unmodified lines | ||
| 181 | 182 | struct nouveau_fbdev *fbcon; |
| 182 | 183 | struct nvif_object nvsw; |
| 183 | 184 | struct nvif_object ntfy; |
| 184 | − struct nvif_notify flip; | |
| 185 | 185 | |
| 186 | 186 | /* nv10-nv40 tiling regions */ |
| 187 | 187 | struct { |
| @@ -210,6 +210,10 @@ struct nouveau_drm { · 22 unmodified lines | ||
| 210 | 210 | bool have_disp_power_ref; |
| 211 | 211 | |
| 212 | 212 | struct dev_pm_domain vga_pm_domain; |
| 213 | + | |
| 214 | + struct nouveau_svm *svm; | |
| 215 | + | |
| 216 | + struct nouveau_dmem *dmem; | |
| 213 | 217 | }; |
| 214 | 218 | |
| 215 | 219 | static inline struct nouveau_drm * |
| @@ -353,7 +353,7 @@ nouveau_fbcon_create(struct drm_fb_helper *helper, · 352 unmodified lines | ||
| 353 | 353 | |
| 354 | 354 | chan = nouveau_nofbaccel ? NULL : drm->channel; |
| 355 | 355 | if (chan && device->info.family >= NV_DEVICE_INFO_V0_TESLA) { |
| 356 | − ret = nouveau_vma_new(nvbo, &drm->client.vmm, &fb->vma); | |
| 356 | + ret = nouveau_vma_new(nvbo, chan->vmm, &fb->vma); | |
| 357 | 357 | if (ret) { |
| 358 | 358 | NV_ERROR(drm, "failed to map fb into chan: %d\n", ret); |
| 359 | 359 | chan = NULL; |
| @@ -82,8 +82,6 @@ int nv50_fence_create(struct nouveau_drm *); · 81 unmodified lines | ||
| 82 | 82 | int nv84_fence_create(struct nouveau_drm *); |
| 83 | 83 | int nvc0_fence_create(struct nouveau_drm *); |
| 84 | 84 | |
| 85 | −int nouveau_flip_complete(struct nvif_notify *); | |
| 86 | − | |
| 87 | 85 | struct nv84_fence_chan { |
| 88 | 86 | struct nouveau_fence_chan base; |
| 89 | 87 | struct nouveau_vma *vma; |
| @@ -68,10 +68,11 @@ nouveau_gem_object_open(struct drm_gem_object *gem, struct drm_file *file_priv) · 67 unmodified lines | ||
| 68 | 68 | struct nouveau_bo *nvbo = nouveau_gem_object(gem); |
| 69 | 69 | struct nouveau_drm *drm = nouveau_bdev(nvbo->bo.bdev); |
| 70 | 70 | struct device *dev = drm->dev->dev; |
| 71 | + struct nouveau_vmm *vmm = cli->svm.cli ? &cli->svm : &cli->vmm; | |
| 71 | 72 | struct nouveau_vma *vma; |
| 72 | 73 | int ret; |
| 73 | 74 | |
| 74 | − if (cli->vmm.vmm.object.oclass < NVIF_CLASS_VMM_NV50) | |
| 75 | + if (vmm->vmm.object.oclass < NVIF_CLASS_VMM_NV50) | |
| 75 | 76 | return 0; |
| 76 | 77 | |
| 77 | 78 | ret = ttm_bo_reserve(&nvbo->bo, false, false, NULL); |
| @@ -82,7 +83,7 @@ nouveau_gem_object_open(struct drm_gem_object *gem, struct drm_file *file_priv) · 4 unmodified lines | ||
| 82 | 83 | if (ret < 0 && ret != -EACCES) |
| 83 | 84 | goto out; |
| 84 | 85 | |
| 85 | − ret = nouveau_vma_new(nvbo, &cli->vmm, &vma); | |
| 86 | + ret = nouveau_vma_new(nvbo, vmm, &vma); | |
| 86 | 87 | pm_runtime_mark_last_busy(dev); |
| 87 | 88 | pm_runtime_put_autosuspend(dev); |
| 88 | 89 | out: |
| @@ -142,17 +143,18 @@ nouveau_gem_object_close(struct drm_gem_object *gem, struct drm_file *file_priv) · 53 unmodified lines | ||
| 142 | 143 | struct nouveau_bo *nvbo = nouveau_gem_object(gem); |
| 143 | 144 | struct nouveau_drm *drm = nouveau_bdev(nvbo->bo.bdev); |
| 144 | 145 | struct device *dev = drm->dev->dev; |
| 146 | + struct nouveau_vmm *vmm = cli->svm.cli ? &cli->svm : & cli->vmm; | |
| 145 | 147 | struct nouveau_vma *vma; |
| 146 | 148 | int ret; |
| 147 | 149 | |
| 148 | − if (cli->vmm.vmm.object.oclass < NVIF_CLASS_VMM_NV50) | |
| 150 | + if (vmm->vmm.object.oclass < NVIF_CLASS_VMM_NV50) | |
| 149 | 151 | return; |
| 150 | 152 | |
| 151 | 153 | ret = ttm_bo_reserve(&nvbo->bo, false, false, NULL); |
| 152 | 154 | if (ret) |
| 153 | 155 | return; |
| 154 | 156 | |
| 155 | − vma = nouveau_vma_find(nvbo, &cli->vmm); | |
| 157 | + vma = nouveau_vma_find(nvbo, vmm); | |
| 156 | 158 | if (vma) { |
| 157 | 159 | if (--vma->refs == 0) { |
| 158 | 160 | ret = pm_runtime_get_sync(dev); |
| @@ -219,6 +221,7 @@ nouveau_gem_info(struct drm_file *file_priv, struct drm_gem_object *gem, · 60 unmodified lines | ||
| 219 | 221 | { |
| 220 | 222 | struct nouveau_cli *cli = nouveau_cli(file_priv); |
| 221 | 223 | struct nouveau_bo *nvbo = nouveau_gem_object(gem); |
| 224 | + struct nouveau_vmm *vmm = cli->svm.cli ? &cli->svm : &cli->vmm; | |
| 222 | 225 | struct nouveau_vma *vma; |
| 223 | 226 | |
| 224 | 227 | if (is_power_of_2(nvbo->valid_domains)) |
| @@ -228,8 +231,8 @@ nouveau_gem_info(struct drm_file *file_priv, struct drm_gem_object *gem, · 3 unmodified lines | ||
| 228 | 231 | else |
| 229 | 232 | rep->domain = NOUVEAU_GEM_DOMAIN_VRAM; |
| 230 | 233 | rep->offset = nvbo->bo.offset; |
| 231 | − if (cli->vmm.vmm.object.oclass >= NVIF_CLASS_VMM_NV50) { | |
| 232 | − vma = nouveau_vma_find(nvbo, &cli->vmm); | |
| 234 | + if (vmm->vmm.object.oclass >= NVIF_CLASS_VMM_NV50) { | |
| 235 | + vma = nouveau_vma_find(nvbo, vmm); | |
| 233 | 236 | if (!vma) |
| 234 | 237 | return -EINVAL; |
| 235 | 238 | |
| @@ -321,7 +324,8 @@ struct validate_op { · 85 unmodified lines | ||
| 321 | 324 | }; |
| 322 | 325 | |
| 323 | 326 | static void |
| 324 | −validate_fini_no_ticket(struct validate_op *op, struct nouveau_fence *fence, | |
| 327 | +validate_fini_no_ticket(struct validate_op *op, struct nouveau_channel *chan, | |
| 328 | + struct nouveau_fence *fence, | |
| 325 | 329 | struct drm_nouveau_gem_pushbuf_bo *pbbo) |
| 326 | 330 | { |
| 327 | 331 | struct nouveau_bo *nvbo; |
| @@ -332,13 +336,11 @@ validate_fini_no_ticket(struct validate_op *op, struct nouveau_fence *fence, · 4 unmodified lines | ||
| 332 | 336 | b = &pbbo[nvbo->pbbo_index]; |
| 333 | 337 | |
| 334 | 338 | if (likely(fence)) { |
| 335 | − struct nouveau_drm *drm = nouveau_bdev(nvbo->bo.bdev); | |
| 336 | − struct nouveau_vma *vma; | |
| 337 | − | |
| 338 | 339 | nouveau_bo_fence(nvbo, fence, !!b->write_domains); |
| 339 | 340 | |
| 340 | − if (drm->client.vmm.vmm.object.oclass >= NVIF_CLASS_VMM_NV50) { | |
| 341 | − vma = (void *)(unsigned long)b->user_priv; | |
| 341 | + if (chan->vmm->vmm.object.oclass >= NVIF_CLASS_VMM_NV50) { | |
| 342 | + struct nouveau_vma *vma = | |
| 343 | + (void *)(unsigned long)b->user_priv; | |
| 342 | 344 | nouveau_fence_unref(&vma->fence); |
| 343 | 345 | dma_fence_get(&fence->base); |
| 344 | 346 | vma->fence = fence; |
| @@ -358,10 +360,11 @@ validate_fini_no_ticket(struct validate_op *op, struct nouveau_fence *fence, · 13 unmodified lines | ||
| 358 | 360 | } |
| 359 | 361 | |
| 360 | 362 | static void |
| 361 | −validate_fini(struct validate_op *op, struct nouveau_fence *fence, | |
| 363 | +validate_fini(struct validate_op *op, struct nouveau_channel *chan, | |
| 364 | + struct nouveau_fence *fence, | |
| 362 | 365 | struct drm_nouveau_gem_pushbuf_bo *pbbo) |
| 363 | 366 | { |
| 364 | − validate_fini_no_ticket(op, fence, pbbo); | |
| 367 | + validate_fini_no_ticket(op, chan, fence, pbbo); | |
| 365 | 368 | ww_acquire_fini(&op->ticket); |
| 366 | 369 | } |
| 367 | 370 | |
| @@ -416,7 +419,7 @@ retry: · 48 unmodified lines | ||
| 416 | 419 | list_splice_tail_init(&vram_list, &op->list); |
| 417 | 420 | list_splice_tail_init(&gart_list, &op->list); |
| 418 | 421 | list_splice_tail_init(&both_list, &op->list); |
| 419 | − validate_fini_no_ticket(op, NULL, NULL); | |
| 422 | + validate_fini_no_ticket(op, chan, NULL, NULL); | |
| 420 | 423 | if (unlikely(ret == -EDEADLK)) { |
| 421 | 424 | ret = ttm_bo_reserve_slowpath(&nvbo->bo, true, |
| 422 | 425 | &op->ticket); |
| @@ -430,8 +433,8 @@ retry: · 7 unmodified lines | ||
| 430 | 433 | } |
| 431 | 434 | } |
| 432 | 435 | |
| 433 | − if (cli->vmm.vmm.object.oclass >= NVIF_CLASS_VMM_NV50) { | |
| 434 | − struct nouveau_vmm *vmm = &cli->vmm; | |
| 436 | + if (chan->vmm->vmm.object.oclass >= NVIF_CLASS_VMM_NV50) { | |
| 437 | + struct nouveau_vmm *vmm = chan->vmm; | |
| 435 | 438 | struct nouveau_vma *vma = nouveau_vma_find(nvbo, vmm); |
| 436 | 439 | if (!vma) { |
| 437 | 440 | NV_PRINTK(err, cli, "vma not found!\n"); |
| @@ -471,7 +474,7 @@ retry: · 33 unmodified lines | ||
| 471 | 474 | list_splice_tail(&gart_list, &op->list); |
| 472 | 475 | list_splice_tail(&both_list, &op->list); |
| 473 | 476 | if (ret) |
| 474 | − validate_fini(op, NULL, NULL); | |
| 477 | + validate_fini(op, chan, NULL, NULL); | |
| 475 | 478 | return ret; |
| 476 | 479 | |
| 477 | 480 | } |
| @@ -563,7 +566,7 @@ nouveau_gem_pushbuf_validate(struct nouveau_channel *chan, · 85 unmodified lines | ||
| 563 | 566 | if (unlikely(ret < 0)) { |
| 564 | 567 | if (ret != -ERESTARTSYS) |
| 565 | 568 | NV_PRINTK(err, cli, "validating bo list\n"); |
| 566 | − validate_fini(op, NULL, NULL); | |
| 569 | + validate_fini(op, chan, NULL, NULL); | |
| 567 | 570 | return ret; |
| 568 | 571 | } |
| 569 | 572 | *apply_relocs = ret; |
| @@ -842,7 +845,7 @@ nouveau_gem_ioctl_pushbuf(struct drm_device *dev, void *data, · 272 unmodified lines | ||
| 842 | 845 | } |
| 843 | 846 | |
| 844 | 847 | out: |
| 845 | − validate_fini(&op, fence, bo); | |
| 848 | + validate_fini(&op, chan, fence, bo); | |
| 846 | 849 | nouveau_fence_unref(&fence); |
| 847 | 850 | |
| 848 | 851 | out_prevalid: |
| @@ -0,0 +1,835 @@ | ||
| 1 | +/* | |
| 2 | + * Copyright 2018 Red Hat Inc. | |
| 3 | + * | |
| 4 | + * Permission is hereby granted, free of charge, to any person obtaining a | |
| 5 | + * copy of this software and associated documentation files (the "Software"), | |
| 6 | + * to deal in the Software without restriction, including without limitation | |
| 7 | + * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 8 | + * and/or sell copies of the Software, and to permit persons to whom the | |
| 9 | + * Software is furnished to do so, subject to the following conditions: | |
| 10 | + * | |
| 11 | + * The above copyright notice and this permission notice shall be included in | |
| 12 | + * all copies or substantial portions of the Software. | |
| 13 | + * | |
| 14 | + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 15 | + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 16 | + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 17 | + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 18 | + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 19 | + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 20 | + * OTHER DEALINGS IN THE SOFTWARE. | |
| 21 | + */ | |
| 22 | +#include "nouveau_svm.h" | |
| 23 | +#include "nouveau_drv.h" | |
| 24 | +#include "nouveau_chan.h" | |
| 25 | +#include "nouveau_dmem.h" | |
| 26 | + | |
| 27 | +#include <nvif/notify.h> | |
| 28 | +#include <nvif/object.h> | |
| 29 | +#include <nvif/vmm.h> | |
| 30 | + | |
| 31 | +#include <nvif/class.h> | |
| 32 | +#include <nvif/clb069.h> | |
| 33 | +#include <nvif/ifc00d.h> | |
| 34 | + | |
| 35 | +#include <linux/sched/mm.h> | |
| 36 | +#include <linux/sort.h> | |
| 37 | +#include <linux/hmm.h> | |
| 38 | + | |
| 39 | +struct nouveau_svm { | |
| 40 | + struct nouveau_drm *drm; | |
| 41 | + struct mutex mutex; | |
| 42 | + struct list_head inst; | |
| 43 | + | |
| 44 | + struct nouveau_svm_fault_buffer { | |
| 45 | + int id; | |
| 46 | + struct nvif_object object; | |
| 47 | + u32 entries; | |
| 48 | + u32 getaddr; | |
| 49 | + u32 putaddr; | |
| 50 | + u32 get; | |
| 51 | + u32 put; | |
| 52 | + struct nvif_notify notify; | |
| 53 | + | |
| 54 | + struct nouveau_svm_fault { | |
| 55 | + u64 inst; | |
| 56 | + u64 addr; | |
| 57 | + u64 time; | |
| 58 | + u32 engine; | |
| 59 | + u8 gpc; | |
| 60 | + u8 hub; | |
| 61 | + u8 access; | |
| 62 | + u8 client; | |
| 63 | + u8 fault; | |
| 64 | + struct nouveau_svmm *svmm; | |
| 65 | + } **fault; | |
| 66 | + int fault_nr; | |
| 67 | + } buffer[1]; | |
| 68 | +}; | |
| 69 | + | |
| 70 | +#define SVM_DBG(s,f,a...) NV_DEBUG((s)->drm, "svm: "f"\n", ##a) | |
| 71 | +#define SVM_ERR(s,f,a...) NV_WARN((s)->drm, "svm: "f"\n", ##a) | |
| 72 | + | |
| 73 | +struct nouveau_ivmm { | |
| 74 | + struct nouveau_svmm *svmm; | |
| 75 | + u64 inst; | |
| 76 | + struct list_head head; | |
| 77 | +}; | |
| 78 | + | |
| 79 | +static struct nouveau_ivmm * | |
| 80 | +nouveau_ivmm_find(struct nouveau_svm *svm, u64 inst) | |
| 81 | +{ | |
| 82 | + struct nouveau_ivmm *ivmm; | |
| 83 | + list_for_each_entry(ivmm, &svm->inst, head) { | |
| 84 | + if (ivmm->inst == inst) | |
| 85 | + return ivmm; | |
| 86 | + } | |
| 87 | + return NULL; | |
| 88 | +} | |
| 89 | + | |
| 90 | +struct nouveau_svmm { | |
| 91 | + struct nouveau_vmm *vmm; | |
| 92 | + struct { | |
| 93 | + unsigned long start; | |
| 94 | + unsigned long limit; | |
| 95 | + } unmanaged; | |
| 96 | + | |
| 97 | + struct mutex mutex; | |
| 98 | + | |
| 99 | + struct mm_struct *mm; | |
| 100 | + struct hmm_mirror mirror; | |
| 101 | +}; | |
| 102 | + | |
| 103 | +#define SVMM_DBG(s,f,a...) \ | |
| 104 | + NV_DEBUG((s)->vmm->cli->drm, "svm-%p: "f"\n", (s), ##a) | |
| 105 | +#define SVMM_ERR(s,f,a...) \ | |
| 106 | + NV_WARN((s)->vmm->cli->drm, "svm-%p: "f"\n", (s), ##a) | |
| 107 | + | |
| 108 | +int | |
| 109 | +nouveau_svmm_bind(struct drm_device *dev, void *data, | |
| 110 | + struct drm_file *file_priv) | |
| 111 | +{ | |
| 112 | + struct nouveau_cli *cli = nouveau_cli(file_priv); | |
| 113 | + struct drm_nouveau_svm_bind *args = data; | |
| 114 | + unsigned target, cmd, priority; | |
| 115 | + unsigned long addr, end, size; | |
| 116 | + struct mm_struct *mm; | |
| 117 | + | |
| 118 | + args->va_start &= PAGE_MASK; | |
| 119 | + args->va_end &= PAGE_MASK; | |
| 120 | + | |
| 121 | + /* Sanity check arguments */ | |
| 122 | + if (args->reserved0 || args->reserved1) | |
| 123 | + return -EINVAL; | |
| 124 | + if (args->header & (~NOUVEAU_SVM_BIND_VALID_MASK)) | |
| 125 | + return -EINVAL; | |
| 126 | + if (args->va_start >= args->va_end) | |
| 127 | + return -EINVAL; | |
| 128 | + if (!args->npages) | |
| 129 | + return -EINVAL; | |
| 130 | + | |
| 131 | + cmd = args->header >> NOUVEAU_SVM_BIND_COMMAND_SHIFT; | |
| 132 | + cmd &= NOUVEAU_SVM_BIND_COMMAND_MASK; | |
| 133 | + switch (cmd) { | |
| 134 | + case NOUVEAU_SVM_BIND_COMMAND__MIGRATE: | |
| 135 | + break; | |
| 136 | + default: | |
| 137 | + return -EINVAL; | |
| 138 | + } | |
| 139 | + | |
| 140 | + priority = args->header >> NOUVEAU_SVM_BIND_PRIORITY_SHIFT; | |
| 141 | + priority &= NOUVEAU_SVM_BIND_PRIORITY_MASK; | |
| 142 | + | |
| 143 | + /* FIXME support CPU target ie all target value < GPU_VRAM */ | |
| 144 | + target = args->header >> NOUVEAU_SVM_BIND_TARGET_SHIFT; | |
| 145 | + target &= NOUVEAU_SVM_BIND_TARGET_MASK; | |
| 146 | + switch (target) { | |
| 147 | + case NOUVEAU_SVM_BIND_TARGET__GPU_VRAM: | |
| 148 | + break; | |
| 149 | + default: | |
| 150 | + return -EINVAL; | |
| 151 | + } | |
| 152 | + | |
| 153 | + /* | |
| 154 | + * FIXME: For now refuse non 0 stride, we need to change the migrate | |
| 155 | + * kernel function to handle stride to avoid to create a mess within | |
| 156 | + * each device driver. | |
| 157 | + */ | |
| 158 | + if (args->stride) | |
| 159 | + return -EINVAL; | |
| 160 | + | |
| 161 | + size = ((unsigned long)args->npages) << PAGE_SHIFT; | |
| 162 | + if ((args->va_start + size) <= args->va_start) | |
| 163 | + return -EINVAL; | |
| 164 | + if ((args->va_start + size) > args->va_end) | |
| 165 | + return -EINVAL; | |
| 166 | + | |
| 167 | + /* | |
| 168 | + * Ok we are ask to do something sane, for now we only support migrate | |
| 169 | + * commands but we will add things like memory policy (what to do on | |
| 170 | + * page fault) and maybe some other commands. | |
| 171 | + */ | |
| 172 | + | |
| 173 | + mm = get_task_mm(current); | |
| 174 | + down_read(&mm->mmap_sem); | |
| 175 | + | |
| 176 | + for (addr = args->va_start, end = args->va_start + size; addr < end;) { | |
| 177 | + struct vm_area_struct *vma; | |
| 178 | + unsigned long next; | |
| 179 | + | |
| 180 | + vma = find_vma_intersection(mm, addr, end); | |
| 181 | + if (!vma) | |
| 182 | + break; | |
| 183 | + | |
| 184 | + next = min(vma->vm_end, end); | |
| 185 | + /* This is a best effort so we ignore errors */ | |
| 186 | + nouveau_dmem_migrate_vma(cli->drm, vma, addr, next); | |
| 187 | + addr = next; | |
| 188 | + } | |
| 189 | + | |
| 190 | + /* | |
| 191 | + * FIXME Return the number of page we have migrated, again we need to | |
| 192 | + * update the migrate API to return that information so that we can | |
| 193 | + * report it to user space. | |
| 194 | + */ | |
| 195 | + args->result = 0; | |
| 196 | + | |
| 197 | + up_read(&mm->mmap_sem); | |
| 198 | + mmput(mm); | |
| 199 | + | |
| 200 | + return 0; | |
| 201 | +} | |
| 202 | + | |
| 203 | +/* Unlink channel instance from SVMM. */ | |
| 204 | +void | |
| 205 | +nouveau_svmm_part(struct nouveau_svmm *svmm, u64 inst) | |
| 206 | +{ | |
| 207 | + struct nouveau_ivmm *ivmm; | |
| 208 | + if (svmm) { | |
| 209 | + mutex_lock(&svmm->vmm->cli->drm->svm->mutex); | |
| 210 | + ivmm = nouveau_ivmm_find(svmm->vmm->cli->drm->svm, inst); | |
| 211 | + if (ivmm) { | |
| 212 | + list_del(&ivmm->head); | |
| 213 | + kfree(ivmm); | |
| 214 | + } | |
| 215 | + mutex_unlock(&svmm->vmm->cli->drm->svm->mutex); | |
| 216 | + } | |
| 217 | +} | |
| 218 | + | |
| 219 | +/* Link channel instance to SVMM. */ | |
| 220 | +int | |
| 221 | +nouveau_svmm_join(struct nouveau_svmm *svmm, u64 inst) | |
| 222 | +{ | |
| 223 | + struct nouveau_ivmm *ivmm; | |
| 224 | + if (svmm) { | |
| 225 | + if (!(ivmm = kmalloc(sizeof(*ivmm), GFP_KERNEL))) | |
| 226 | + return -ENOMEM; | |
| 227 | + ivmm->svmm = svmm; | |
| 228 | + ivmm->inst = inst; | |
| 229 | + | |
| 230 | + mutex_lock(&svmm->vmm->cli->drm->svm->mutex); | |
| 231 | + list_add(&ivmm->head, &svmm->vmm->cli->drm->svm->inst); | |
| 232 | + mutex_unlock(&svmm->vmm->cli->drm->svm->mutex); | |
| 233 | + } | |
| 234 | + return 0; | |
| 235 | +} | |
| 236 | + | |
| 237 | +/* Invalidate SVMM address-range on GPU. */ | |
| 238 | +static void | |
| 239 | +nouveau_svmm_invalidate(struct nouveau_svmm *svmm, u64 start, u64 limit) | |
| 240 | +{ | |
| 241 | + if (limit > start) { | |
| 242 | + bool super = svmm->vmm->vmm.object.client->super; | |
| 243 | + svmm->vmm->vmm.object.client->super = true; | |
| 244 | + nvif_object_mthd(&svmm->vmm->vmm.object, NVIF_VMM_V0_PFNCLR, | |
| 245 | + &(struct nvif_vmm_pfnclr_v0) { | |
| 246 | + .addr = start, | |
| 247 | + .size = limit - start, | |
| 248 | + }, sizeof(struct nvif_vmm_pfnclr_v0)); | |
| 249 | + svmm->vmm->vmm.object.client->super = super; | |
| 250 | + } | |
| 251 | +} | |
| 252 | + | |
| 253 | +static int | |
| 254 | +nouveau_svmm_sync_cpu_device_pagetables(struct hmm_mirror *mirror, | |
| 255 | + const struct hmm_update *update) | |
| 256 | +{ | |
| 257 | + struct nouveau_svmm *svmm = container_of(mirror, typeof(*svmm), mirror); | |
| 258 | + unsigned long start = update->start; | |
| 259 | + unsigned long limit = update->end; | |
| 260 | + | |
| 261 | + if (!update->blockable) | |
| 262 | + return -EAGAIN; | |
| 263 | + | |
| 264 | + SVMM_DBG(svmm, "invalidate %016lx-%016lx", start, limit); | |
| 265 | + | |
| 266 | + mutex_lock(&svmm->mutex); | |
| 267 | + if (limit > svmm->unmanaged.start && start < svmm->unmanaged.limit) { | |
| 268 | + if (start < svmm->unmanaged.start) { | |
| 269 | + nouveau_svmm_invalidate(svmm, start, | |
| 270 | + svmm->unmanaged.limit); | |
| 271 | + } | |
| 272 | + start = svmm->unmanaged.limit; | |
| 273 | + } | |
| 274 | + | |
| 275 | + nouveau_svmm_invalidate(svmm, start, limit); | |
| 276 | + mutex_unlock(&svmm->mutex); | |
| 277 | + return 0; | |
| 278 | +} | |
| 279 | + | |
| 280 | +static void | |
| 281 | +nouveau_svmm_release(struct hmm_mirror *mirror) | |
| 282 | +{ | |
| 283 | +} | |
| 284 | + | |
| 285 | +static const struct hmm_mirror_ops | |
| 286 | +nouveau_svmm = { | |
| 287 | + .sync_cpu_device_pagetables = nouveau_svmm_sync_cpu_device_pagetables, | |
| 288 | + .release = nouveau_svmm_release, | |
| 289 | +}; | |
| 290 | + | |
| 291 | +void | |
| 292 | +nouveau_svmm_fini(struct nouveau_svmm **psvmm) | |
| 293 | +{ | |
| 294 | + struct nouveau_svmm *svmm = *psvmm; | |
| 295 | + if (svmm) { | |
| 296 | + hmm_mirror_unregister(&svmm->mirror); | |
| 297 | + kfree(*psvmm); | |
| 298 | + *psvmm = NULL; | |
| 299 | + } | |
| 300 | +} | |
| 301 | + | |
| 302 | +int | |
| 303 | +nouveau_svmm_init(struct drm_device *dev, void *data, | |
| 304 | + struct drm_file *file_priv) | |
| 305 | +{ | |
| 306 | + struct nouveau_cli *cli = nouveau_cli(file_priv); | |
| 307 | + struct nouveau_svmm *svmm; | |
| 308 | + struct drm_nouveau_svm_init *args = data; | |
| 309 | + int ret; | |
| 310 | + | |
| 311 | + /* Allocate tracking for SVM-enabled VMM. */ | |
| 312 | + if (!(svmm = kzalloc(sizeof(*svmm), GFP_KERNEL))) | |
| 313 | + return -ENOMEM; | |
| 314 | + svmm->vmm = &cli->svm; | |
| 315 | + svmm->unmanaged.start = args->unmanaged_addr; | |
| 316 | + svmm->unmanaged.limit = args->unmanaged_addr + args->unmanaged_size; | |
| 317 | + mutex_init(&svmm->mutex); | |
| 318 | + | |
| 319 | + /* Check that SVM isn't already enabled for the client. */ | |
| 320 | + mutex_lock(&cli->mutex); | |
| 321 | + if (cli->svm.cli) { | |
| 322 | + ret = -EBUSY; | |
| 323 | + goto done; | |
| 324 | + } | |
| 325 | + | |
| 326 | + /* Allocate a new GPU VMM that can support SVM (managed by the | |
| 327 | + * client, with replayable faults enabled). | |
| 328 | + * | |
| 329 | + * All future channel/memory allocations will make use of this | |
| 330 | + * VMM instead of the standard one. | |
| 331 | + */ | |
| 332 | + ret = nvif_vmm_init(&cli->mmu, cli->vmm.vmm.object.oclass, true, | |
| 333 | + args->unmanaged_addr, args->unmanaged_size, | |
| 334 | + &(struct gp100_vmm_v0) { | |
| 335 | + .fault_replay = true, | |
| 336 | + }, sizeof(struct gp100_vmm_v0), &cli->svm.vmm); | |
| 337 | + if (ret) | |
| 338 | + goto done; | |
| 339 | + | |
| 340 | + /* Enable HMM mirroring of CPU address-space to VMM. */ | |
| 341 | + svmm->mm = get_task_mm(current); | |
| 342 | + down_write(&svmm->mm->mmap_sem); | |
| 343 | + svmm->mirror.ops = &nouveau_svmm; | |
| 344 | + ret = hmm_mirror_register(&svmm->mirror, svmm->mm); | |
| 345 | + if (ret == 0) { | |
| 346 | + cli->svm.svmm = svmm; | |
| 347 | + cli->svm.cli = cli; | |
| 348 | + } | |
| 349 | + up_write(&svmm->mm->mmap_sem); | |
| 350 | + mmput(svmm->mm); | |
| 351 | + | |
| 352 | +done: | |
| 353 | + if (ret) | |
| 354 | + nouveau_svmm_fini(&svmm); | |
| 355 | + mutex_unlock(&cli->mutex); | |
| 356 | + return ret; | |
| 357 | +} | |
| 358 | + | |
| 359 | +static const u64 | |
| 360 | +nouveau_svm_pfn_flags[HMM_PFN_FLAG_MAX] = { | |
| 361 | + [HMM_PFN_VALID ] = NVIF_VMM_PFNMAP_V0_V, | |
| 362 | + [HMM_PFN_WRITE ] = NVIF_VMM_PFNMAP_V0_W, | |
| 363 | + [HMM_PFN_DEVICE_PRIVATE] = NVIF_VMM_PFNMAP_V0_VRAM, | |
| 364 | +}; | |
| 365 | + | |
| 366 | +static const u64 | |
| 367 | +nouveau_svm_pfn_values[HMM_PFN_VALUE_MAX] = { | |
| 368 | + [HMM_PFN_ERROR ] = ~NVIF_VMM_PFNMAP_V0_V, | |
| 369 | + [HMM_PFN_NONE ] = NVIF_VMM_PFNMAP_V0_NONE, | |
| 370 | + [HMM_PFN_SPECIAL] = ~NVIF_VMM_PFNMAP_V0_V, | |
| 371 | +}; | |
| 372 | + | |
| 373 | +/* Issue fault replay for GPU to retry accesses that faulted previously. */ | |
| 374 | +static void | |
| 375 | +nouveau_svm_fault_replay(struct nouveau_svm *svm) | |
| 376 | +{ | |
| 377 | + SVM_DBG(svm, "replay"); | |
| 378 | + WARN_ON(nvif_object_mthd(&svm->drm->client.vmm.vmm.object, | |
| 379 | + GP100_VMM_VN_FAULT_REPLAY, | |
| 380 | + &(struct gp100_vmm_fault_replay_vn) {}, | |
| 381 | + sizeof(struct gp100_vmm_fault_replay_vn))); | |
| 382 | +} | |
| 383 | + | |
| 384 | +/* Cancel a replayable fault that could not be handled. | |
| 385 | + * | |
| 386 | + * Cancelling the fault will trigger recovery to reset the engine | |
| 387 | + * and kill the offending channel (ie. GPU SIGSEGV). | |
| 388 | + */ | |
| 389 | +static void | |
| 390 | +nouveau_svm_fault_cancel(struct nouveau_svm *svm, | |
| 391 | + u64 inst, u8 hub, u8 gpc, u8 client) | |
| 392 | +{ | |
| 393 | + SVM_DBG(svm, "cancel %016llx %d %02x %02x", inst, hub, gpc, client); | |
| 394 | + WARN_ON(nvif_object_mthd(&svm->drm->client.vmm.vmm.object, | |
| 395 | + GP100_VMM_VN_FAULT_CANCEL, | |
| 396 | + &(struct gp100_vmm_fault_cancel_v0) { | |
| 397 | + .hub = hub, | |
| 398 | + .gpc = gpc, | |
| 399 | + .client = client, | |
| 400 | + .inst = inst, | |
| 401 | + }, sizeof(struct gp100_vmm_fault_cancel_v0))); | |
| 402 | +} | |
| 403 | + | |
| 404 | +static void | |
| 405 | +nouveau_svm_fault_cancel_fault(struct nouveau_svm *svm, | |
| 406 | + struct nouveau_svm_fault *fault) | |
| 407 | +{ | |
| 408 | + nouveau_svm_fault_cancel(svm, fault->inst, | |
| 409 | + fault->hub, | |
| 410 | + fault->gpc, | |
| 411 | + fault->client); | |
| 412 | +} | |
| 413 | + | |
| 414 | +static int | |
| 415 | +nouveau_svm_fault_cmp(const void *a, const void *b) | |
| 416 | +{ | |
| 417 | + const struct nouveau_svm_fault *fa = *(struct nouveau_svm_fault **)a; | |
| 418 | + const struct nouveau_svm_fault *fb = *(struct nouveau_svm_fault **)b; | |
| 419 | + int ret; | |
| 420 | + if ((ret = (s64)fa->inst - fb->inst)) | |
| 421 | + return ret; | |
| 422 | + if ((ret = (s64)fa->addr - fb->addr)) | |
| 423 | + return ret; | |
| 424 | + /*XXX: atomic? */ | |
| 425 | + return (fa->access == 0 || fa->access == 3) - | |
| 426 | + (fb->access == 0 || fb->access == 3); | |
| 427 | +} | |
| 428 | + | |
| 429 | +static void | |
| 430 | +nouveau_svm_fault_cache(struct nouveau_svm *svm, | |
| 431 | + struct nouveau_svm_fault_buffer *buffer, u32 offset) | |
| 432 | +{ | |
| 433 | + struct nvif_object *memory = &buffer->object; | |
| 434 | + const u32 instlo = nvif_rd32(memory, offset + 0x00); | |
| 435 | + const u32 insthi = nvif_rd32(memory, offset + 0x04); | |
| 436 | + const u32 addrlo = nvif_rd32(memory, offset + 0x08); | |
| 437 | + const u32 addrhi = nvif_rd32(memory, offset + 0x0c); | |
| 438 | + const u32 timelo = nvif_rd32(memory, offset + 0x10); | |
| 439 | + const u32 timehi = nvif_rd32(memory, offset + 0x14); | |
| 440 | + const u32 engine = nvif_rd32(memory, offset + 0x18); | |
| 441 | + const u32 info = nvif_rd32(memory, offset + 0x1c); | |
| 442 | + const u64 inst = (u64)insthi << 32 | instlo; | |
| 443 | + const u8 gpc = (info & 0x1f000000) >> 24; | |
| 444 | + const u8 hub = (info & 0x00100000) >> 20; | |
| 445 | + const u8 client = (info & 0x00007f00) >> 8; | |
| 446 | + struct nouveau_svm_fault *fault; | |
| 447 | + | |
| 448 | + //XXX: i think we're supposed to spin waiting */ | |
| 449 | + if (WARN_ON(!(info & 0x80000000))) | |
| 450 | + return; | |
| 451 | + | |
| 452 | + nvif_mask(memory, offset + 0x1c, 0x80000000, 0x00000000); | |
| 453 | + | |
| 454 | + if (!buffer->fault[buffer->fault_nr]) { | |
| 455 | + fault = kmalloc(sizeof(*fault), GFP_KERNEL); | |
| 456 | + if (WARN_ON(!fault)) { | |
| 457 | + nouveau_svm_fault_cancel(svm, inst, hub, gpc, client); | |
| 458 | + return; | |
| 459 | + } | |
| 460 | + buffer->fault[buffer->fault_nr] = fault; | |
| 461 | + } | |
| 462 | + | |
| 463 | + fault = buffer->fault[buffer->fault_nr++]; | |
| 464 | + fault->inst = inst; | |
| 465 | + fault->addr = (u64)addrhi << 32 | addrlo; | |
| 466 | + fault->time = (u64)timehi << 32 | timelo; | |
| 467 | + fault->engine = engine; | |
| 468 | + fault->gpc = gpc; | |
| 469 | + fault->hub = hub; | |
| 470 | + fault->access = (info & 0x000f0000) >> 16; | |
| 471 | + fault->client = client; | |
| 472 | + fault->fault = (info & 0x0000001f); | |
| 473 | + | |
| 474 | + SVM_DBG(svm, "fault %016llx %016llx %02x", | |
| 475 | + fault->inst, fault->addr, fault->access); | |
| 476 | +} | |
| 477 | + | |
| 478 | +static int | |
| 479 | +nouveau_svm_fault(struct nvif_notify *notify) | |
| 480 | +{ | |
| 481 | + struct nouveau_svm_fault_buffer *buffer = | |
| 482 | + container_of(notify, typeof(*buffer), notify); | |
| 483 | + struct nouveau_svm *svm = | |
| 484 | + container_of(buffer, typeof(*svm), buffer[buffer->id]); | |
| 485 | + struct nvif_object *device = &svm->drm->client.device.object; | |
| 486 | + struct nouveau_svmm *svmm; | |
| 487 | + struct { | |
| 488 | + struct { | |
| 489 | + struct nvif_ioctl_v0 i; | |
| 490 | + struct nvif_ioctl_mthd_v0 m; | |
| 491 | + struct nvif_vmm_pfnmap_v0 p; | |
| 492 | + } i; | |
| 493 | + u64 phys[16]; | |
| 494 | + } args; | |
| 495 | + struct hmm_range range; | |
| 496 | + struct vm_area_struct *vma; | |
| 497 | + u64 inst, start, limit; | |
| 498 | + int fi, fn, pi, fill; | |
| 499 | + int replay = 0, ret; | |
| 500 | + | |
| 501 | + /* Parse available fault buffer entries into a cache, and update | |
| 502 | + * the GET pointer so HW can reuse the entries. | |
| 503 | + */ | |
| 504 | + SVM_DBG(svm, "fault handler"); | |
| 505 | + if (buffer->get == buffer->put) { | |
| 506 | + buffer->put = nvif_rd32(device, buffer->putaddr); | |
| 507 | + buffer->get = nvif_rd32(device, buffer->getaddr); | |
| 508 | + if (buffer->get == buffer->put) | |
| 509 | + return NVIF_NOTIFY_KEEP; | |
| 510 | + } | |
| 511 | + buffer->fault_nr = 0; | |
| 512 | + | |
| 513 | + SVM_DBG(svm, "get %08x put %08x", buffer->get, buffer->put); | |
| 514 | + while (buffer->get != buffer->put) { | |
| 515 | + nouveau_svm_fault_cache(svm, buffer, buffer->get * 0x20); | |
| 516 | + if (++buffer->get == buffer->entries) | |
| 517 | + buffer->get = 0; | |
| 518 | + } | |
| 519 | + nvif_wr32(device, buffer->getaddr, buffer->get); | |
| 520 | + SVM_DBG(svm, "%d fault(s) pending", buffer->fault_nr); | |
| 521 | + | |
| 522 | + /* Sort parsed faults by instance pointer to prevent unnecessary | |
| 523 | + * instance to SVMM translations, followed by address and access | |
| 524 | + * type to reduce the amount of work when handling the faults. | |
| 525 | + */ | |
| 526 | + sort(buffer->fault, buffer->fault_nr, sizeof(*buffer->fault), | |
| 527 | + nouveau_svm_fault_cmp, NULL); | |
| 528 | + | |
| 529 | + /* Lookup SVMM structure for each unique instance pointer. */ | |
| 530 | + mutex_lock(&svm->mutex); | |
| 531 | + for (fi = 0, svmm = NULL; fi < buffer->fault_nr; fi++) { | |
| 532 | + if (!svmm || buffer->fault[fi]->inst != inst) { | |
| 533 | + struct nouveau_ivmm *ivmm = | |
| 534 | + nouveau_ivmm_find(svm, buffer->fault[fi]->inst); | |
| 535 | + svmm = ivmm ? ivmm->svmm : NULL; | |
| 536 | + inst = buffer->fault[fi]->inst; | |
| 537 | + SVM_DBG(svm, "inst %016llx -> svm-%p", inst, svmm); | |
| 538 | + } | |
| 539 | + buffer->fault[fi]->svmm = svmm; | |
| 540 | + } | |
| 541 | + mutex_unlock(&svm->mutex); | |
| 542 | + | |
| 543 | + /* Process list of faults. */ | |
| 544 | + args.i.i.version = 0; | |
| 545 | + args.i.i.type = NVIF_IOCTL_V0_MTHD; | |
| 546 | + args.i.m.version = 0; | |
| 547 | + args.i.m.method = NVIF_VMM_V0_PFNMAP; | |
| 548 | + args.i.p.version = 0; | |
| 549 | + | |
| 550 | + for (fi = 0; fn = fi + 1, fi < buffer->fault_nr; fi = fn) { | |
| 551 | + /* Cancel any faults from non-SVM channels. */ | |
| 552 | + if (!(svmm = buffer->fault[fi]->svmm)) { | |
| 553 | + nouveau_svm_fault_cancel_fault(svm, buffer->fault[fi]); | |
| 554 | + continue; | |
| 555 | + } | |
| 556 | + SVMM_DBG(svmm, "addr %016llx", buffer->fault[fi]->addr); | |
| 557 | + | |
| 558 | + /* We try and group handling of faults within a small | |
| 559 | + * window into a single update. | |
| 560 | + */ | |
| 561 | + start = buffer->fault[fi]->addr; | |
| 562 | + limit = start + (ARRAY_SIZE(args.phys) << PAGE_SHIFT); | |
| 563 | + if (start < svmm->unmanaged.limit) | |
| 564 | + limit = min_t(u64, limit, svmm->unmanaged.start); | |
| 565 | + else | |
| 566 | + if (limit > svmm->unmanaged.start) | |
| 567 | + start = max_t(u64, start, svmm->unmanaged.limit); | |
| 568 | + SVMM_DBG(svmm, "wndw %016llx-%016llx", start, limit); | |
| 569 | + | |
| 570 | + /* Intersect fault window with the CPU VMA, cancelling | |
| 571 | + * the fault if the address is invalid. | |
| 572 | + */ | |
| 573 | + down_read(&svmm->mm->mmap_sem); | |
| 574 | + vma = find_vma_intersection(svmm->mm, start, limit); | |
| 575 | + if (!vma) { | |
| 576 | + SVMM_ERR(svmm, "wndw %016llx-%016llx", start, limit); | |
| 577 | + up_read(&svmm->mm->mmap_sem); | |
| 578 | + nouveau_svm_fault_cancel_fault(svm, buffer->fault[fi]); | |
| 579 | + continue; | |
| 580 | + } | |
| 581 | + start = max_t(u64, start, vma->vm_start); | |
| 582 | + limit = min_t(u64, limit, vma->vm_end); | |
| 583 | + SVMM_DBG(svmm, "wndw %016llx-%016llx", start, limit); | |
| 584 | + | |
| 585 | + if (buffer->fault[fi]->addr != start) { | |
| 586 | + SVMM_ERR(svmm, "addr %016llx", buffer->fault[fi]->addr); | |
| 587 | + up_read(&svmm->mm->mmap_sem); | |
| 588 | + nouveau_svm_fault_cancel_fault(svm, buffer->fault[fi]); | |
| 589 | + continue; | |
| 590 | + } | |
| 591 | + | |
| 592 | + /* Prepare the GPU-side update of all pages within the | |
| 593 | + * fault window, determining required pages and access | |
| 594 | + * permissions based on pending faults. | |
| 595 | + */ | |
| 596 | + args.i.p.page = PAGE_SHIFT; | |
| 597 | + args.i.p.addr = start; | |
| 598 | + for (fn = fi, pi = 0;;) { | |
| 599 | + /* Determine required permissions based on GPU fault | |
| 600 | + * access flags. | |
| 601 | + *XXX: atomic? | |
| 602 | + */ | |
| 603 | + if (buffer->fault[fn]->access != 0 /* READ. */ && | |
| 604 | + buffer->fault[fn]->access != 3 /* PREFETCH. */) { | |
| 605 | + args.phys[pi++] = NVIF_VMM_PFNMAP_V0_V | | |
| 606 | + NVIF_VMM_PFNMAP_V0_W; | |
| 607 | + } else { | |
| 608 | + args.phys[pi++] = NVIF_VMM_PFNMAP_V0_V; | |
| 609 | + } | |
| 610 | + args.i.p.size = pi << PAGE_SHIFT; | |
| 611 | + | |
| 612 | + /* It's okay to skip over duplicate addresses from the | |
| 613 | + * same SVMM as faults are ordered by access type such | |
| 614 | + * that only the first one needs to be handled. | |
| 615 | + * | |
| 616 | + * ie. WRITE faults appear first, thus any handling of | |
| 617 | + * pending READ faults will already be satisfied. | |
| 618 | + */ | |
| 619 | + while (++fn < buffer->fault_nr && | |
| 620 | + buffer->fault[fn]->svmm == svmm && | |
| 621 | + buffer->fault[fn ]->addr == | |
| 622 | + buffer->fault[fn - 1]->addr); | |
| 623 | + | |
| 624 | + /* If the next fault is outside the window, or all GPU | |
| 625 | + * faults have been dealt with, we're done here. | |
| 626 | + */ | |
| 627 | + if (fn >= buffer->fault_nr || | |
| 628 | + buffer->fault[fn]->svmm != svmm || | |
| 629 | + buffer->fault[fn]->addr >= limit) | |
| 630 | + break; | |
| 631 | + | |
| 632 | + /* Fill in the gap between this fault and the next. */ | |
| 633 | + fill = (buffer->fault[fn ]->addr - | |
| 634 | + buffer->fault[fn - 1]->addr) >> PAGE_SHIFT; | |
| 635 | + while (--fill) | |
| 636 | + args.phys[pi++] = NVIF_VMM_PFNMAP_V0_NONE; | |
| 637 | + } | |
| 638 | + | |
| 639 | + SVMM_DBG(svmm, "wndw %016llx-%016llx covering %d fault(s)", | |
| 640 | + args.i.p.addr, | |
| 641 | + args.i.p.addr + args.i.p.size, fn - fi); | |
| 642 | + | |
| 643 | + /* Have HMM fault pages within the fault window to the GPU. */ | |
| 644 | + range.vma = vma; | |
| 645 | + range.start = args.i.p.addr; | |
| 646 | + range.end = args.i.p.addr + args.i.p.size; | |
| 647 | + range.pfns = args.phys; | |
| 648 | + range.flags = nouveau_svm_pfn_flags; | |
| 649 | + range.values = nouveau_svm_pfn_values; | |
| 650 | + range.pfn_shift = NVIF_VMM_PFNMAP_V0_ADDR_SHIFT; | |
| 651 | +again: | |
| 652 | + ret = hmm_vma_fault(&range, true); | |
| 653 | + if (ret == 0) { | |
| 654 | + mutex_lock(&svmm->mutex); | |
| 655 | + if (!hmm_vma_range_done(&range)) { | |
| 656 | + mutex_unlock(&svmm->mutex); | |
| 657 | + goto again; | |
| 658 | + } | |
| 659 | + | |
| 660 | + nouveau_dmem_convert_pfn(svm->drm, &range); | |
| 661 | + | |
| 662 | + svmm->vmm->vmm.object.client->super = true; | |
| 663 | + ret = nvif_object_ioctl(&svmm->vmm->vmm.object, | |
| 664 | + &args, sizeof(args.i) + | |
| 665 | + pi * sizeof(args.phys[0]), | |
| 666 | + NULL); | |
| 667 | + svmm->vmm->vmm.object.client->super = false; | |
| 668 | + mutex_unlock(&svmm->mutex); | |
| 669 | + } | |
| 670 | + up_read(&svmm->mm->mmap_sem); | |
| 671 | + | |
| 672 | + /* Cancel any faults in the window whose pages didn't manage | |
| 673 | + * to keep their valid bit, or stay writeable when required. | |
| 674 | + * | |
| 675 | + * If handling failed completely, cancel all faults. | |
| 676 | + */ | |
| 677 | + while (fi < fn) { | |
| 678 | + struct nouveau_svm_fault *fault = buffer->fault[fi++]; | |
| 679 | + pi = (fault->addr - range.start) >> PAGE_SHIFT; | |
| 680 | + if (ret || | |
| 681 | + !(range.pfns[pi] & NVIF_VMM_PFNMAP_V0_V) || | |
| 682 | + (!(range.pfns[pi] & NVIF_VMM_PFNMAP_V0_W) && | |
| 683 | + fault->access != 0 && fault->access != 3)) { | |
| 684 | + nouveau_svm_fault_cancel_fault(svm, fault); | |
| 685 | + continue; | |
| 686 | + } | |
| 687 | + replay++; | |
| 688 | + } | |
| 689 | + } | |
| 690 | + | |
| 691 | + /* Issue fault replay to the GPU. */ | |
| 692 | + if (replay) | |
| 693 | + nouveau_svm_fault_replay(svm); | |
| 694 | + return NVIF_NOTIFY_KEEP; | |
| 695 | +} | |
| 696 | + | |
| 697 | +static void | |
| 698 | +nouveau_svm_fault_buffer_fini(struct nouveau_svm *svm, int id) | |
| 699 | +{ | |
| 700 | + struct nouveau_svm_fault_buffer *buffer = &svm->buffer[id]; | |
| 701 | + nvif_notify_put(&buffer->notify); | |
| 702 | +} | |
| 703 | + | |
| 704 | +static int | |
| 705 | +nouveau_svm_fault_buffer_init(struct nouveau_svm *svm, int id) | |
| 706 | +{ | |
| 707 | + struct nouveau_svm_fault_buffer *buffer = &svm->buffer[id]; | |
| 708 | + struct nvif_object *device = &svm->drm->client.device.object; | |
| 709 | + buffer->get = nvif_rd32(device, buffer->getaddr); | |
| 710 | + buffer->put = nvif_rd32(device, buffer->putaddr); | |
| 711 | + SVM_DBG(svm, "get %08x put %08x (init)", buffer->get, buffer->put); | |
| 712 | + return nvif_notify_get(&buffer->notify); | |
| 713 | +} | |
| 714 | + | |
| 715 | +static void | |
| 716 | +nouveau_svm_fault_buffer_dtor(struct nouveau_svm *svm, int id) | |
| 717 | +{ | |
| 718 | + struct nouveau_svm_fault_buffer *buffer = &svm->buffer[id]; | |
| 719 | + int i; | |
| 720 | + | |
| 721 | + if (buffer->fault) { | |
| 722 | + for (i = 0; buffer->fault[i] && i < buffer->entries; i++) | |
| 723 | + kfree(buffer->fault[i]); | |
| 724 | + kvfree(buffer->fault); | |
| 725 | + } | |
| 726 | + | |
| 727 | + nouveau_svm_fault_buffer_fini(svm, id); | |
| 728 | + | |
| 729 | + nvif_notify_fini(&buffer->notify); | |
| 730 | + nvif_object_fini(&buffer->object); | |
| 731 | +} | |
| 732 | + | |
| 733 | +static int | |
| 734 | +nouveau_svm_fault_buffer_ctor(struct nouveau_svm *svm, s32 oclass, int id) | |
| 735 | +{ | |
| 736 | + struct nouveau_svm_fault_buffer *buffer = &svm->buffer[id]; | |
| 737 | + struct nouveau_drm *drm = svm->drm; | |
| 738 | + struct nvif_object *device = &drm->client.device.object; | |
| 739 | + struct nvif_clb069_v0 args = {}; | |
| 740 | + int ret; | |
| 741 | + | |
| 742 | + buffer->id = id; | |
| 743 | + | |
| 744 | + ret = nvif_object_init(device, 0, oclass, &args, sizeof(args), | |
| 745 | + &buffer->object); | |
| 746 | + if (ret < 0) { | |
| 747 | + SVM_ERR(svm, "Fault buffer allocation failed: %d", ret); | |
| 748 | + return ret; | |
| 749 | + } | |
| 750 | + | |
| 751 | + nvif_object_map(&buffer->object, NULL, 0); | |
| 752 | + buffer->entries = args.entries; | |
| 753 | + buffer->getaddr = args.get; | |
| 754 | + buffer->putaddr = args.put; | |
| 755 | + | |
| 756 | + ret = nvif_notify_init(&buffer->object, nouveau_svm_fault, true, | |
| 757 | + NVB069_V0_NTFY_FAULT, NULL, 0, 0, | |
| 758 | + &buffer->notify); | |
| 759 | + if (ret) | |
| 760 | + return ret; | |
| 761 | + | |
| 762 | + buffer->fault = kvzalloc(sizeof(*buffer->fault) * buffer->entries, GFP_KERNEL); | |
| 763 | + if (!buffer->fault) | |
| 764 | + return -ENOMEM; | |
| 765 | + | |
| 766 | + return nouveau_svm_fault_buffer_init(svm, id); | |
| 767 | +} | |
| 768 | + | |
| 769 | +void | |
| 770 | +nouveau_svm_resume(struct nouveau_drm *drm) | |
| 771 | +{ | |
| 772 | + struct nouveau_svm *svm = drm->svm; | |
| 773 | + if (svm) | |
| 774 | + nouveau_svm_fault_buffer_init(svm, 0); | |
| 775 | +} | |
| 776 | + | |
| 777 | +void | |
| 778 | +nouveau_svm_suspend(struct nouveau_drm *drm) | |
| 779 | +{ | |
| 780 | + struct nouveau_svm *svm = drm->svm; | |
| 781 | + if (svm) | |
| 782 | + nouveau_svm_fault_buffer_fini(svm, 0); | |
| 783 | +} | |
| 784 | + | |
| 785 | +void | |
| 786 | +nouveau_svm_fini(struct nouveau_drm *drm) | |
| 787 | +{ | |
| 788 | + struct nouveau_svm *svm = drm->svm; | |
| 789 | + if (svm) { | |
| 790 | + nouveau_svm_fault_buffer_dtor(svm, 0); | |
| 791 | + kfree(drm->svm); | |
| 792 | + drm->svm = NULL; | |
| 793 | + } | |
| 794 | +} | |
| 795 | + | |
| 796 | +void | |
| 797 | +nouveau_svm_init(struct nouveau_drm *drm) | |
| 798 | +{ | |
| 799 | + static const struct nvif_mclass buffers[] = { | |
| 800 | + { VOLTA_FAULT_BUFFER_A, 0 }, | |
| 801 | + { MAXWELL_FAULT_BUFFER_A, 0 }, | |
| 802 | + {} | |
| 803 | + }; | |
| 804 | + struct nouveau_svm *svm; | |
| 805 | + int ret; | |
| 806 | + | |
| 807 | + /* Disable on Volta and newer until channel recovery is fixed, | |
| 808 | + * otherwise clients will have a trivial way to trash the GPU | |
| 809 | + * for everyone. | |
| 810 | + */ | |
| 811 | + if (drm->client.device.info.family > NV_DEVICE_INFO_V0_PASCAL) | |
| 812 | + return; | |
| 813 | + | |
| 814 | + if (!(drm->svm = svm = kzalloc(sizeof(*drm->svm), GFP_KERNEL))) | |
| 815 | + return; | |
| 816 | + | |
| 817 | + drm->svm->drm = drm; | |
| 818 | + mutex_init(&drm->svm->mutex); | |
| 819 | + INIT_LIST_HEAD(&drm->svm->inst); | |
| 820 | + | |
| 821 | + ret = nvif_mclass(&drm->client.device.object, buffers); | |
| 822 | + if (ret < 0) { | |
| 823 | + SVM_DBG(svm, "No supported fault buffer class"); | |
| 824 | + nouveau_svm_fini(drm); | |
| 825 | + return; | |
| 826 | + } | |
| 827 | + | |
| 828 | + ret = nouveau_svm_fault_buffer_ctor(svm, buffers[ret].oclass, 0); | |
| 829 | + if (ret) { | |
| 830 | + nouveau_svm_fini(drm); | |
| 831 | + return; | |
| 832 | + } | |
| 833 | + | |
| 834 | + SVM_DBG(svm, "Initialised"); | |
| 835 | +} | |
| @@ -0,0 +1,48 @@ | ||
| 1 | +#ifndef __NOUVEAU_SVM_H__ | |
| 2 | +#define __NOUVEAU_SVM_H__ | |
| 3 | +#include <nvif/os.h> | |
| 4 | +struct drm_device; | |
| 5 | +struct drm_file; | |
| 6 | +struct nouveau_drm; | |
| 7 | + | |
| 8 | +struct nouveau_svmm; | |
| 9 | + | |
| 10 | +#if IS_ENABLED(CONFIG_DRM_NOUVEAU_SVM) | |
| 11 | +void nouveau_svm_init(struct nouveau_drm *); | |
| 12 | +void nouveau_svm_fini(struct nouveau_drm *); | |
| 13 | +void nouveau_svm_suspend(struct nouveau_drm *); | |
| 14 | +void nouveau_svm_resume(struct nouveau_drm *); | |
| 15 | + | |
| 16 | +int nouveau_svmm_init(struct drm_device *, void *, struct drm_file *); | |
| 17 | +void nouveau_svmm_fini(struct nouveau_svmm **); | |
| 18 | +int nouveau_svmm_join(struct nouveau_svmm *, u64 inst); | |
| 19 | +void nouveau_svmm_part(struct nouveau_svmm *, u64 inst); | |
| 20 | +int nouveau_svmm_bind(struct drm_device *, void *, struct drm_file *); | |
| 21 | +#else /* IS_ENABLED(CONFIG_DRM_NOUVEAU_SVM) */ | |
| 22 | +static inline void nouveau_svm_init(struct nouveau_drm *drm) {} | |
| 23 | +static inline void nouveau_svm_fini(struct nouveau_drm *drm) {} | |
| 24 | +static inline void nouveau_svm_suspend(struct nouveau_drm *drm) {} | |
| 25 | +static inline void nouveau_svm_resume(struct nouveau_drm *drm) {} | |
| 26 | + | |
| 27 | +static inline int nouveau_svmm_init(struct drm_device *device, void *p, | |
| 28 | + struct drm_file *file) | |
| 29 | +{ | |
| 30 | + return -ENOSYS; | |
| 31 | +} | |
| 32 | + | |
| 33 | +static inline void nouveau_svmm_fini(struct nouveau_svmm **svmmp) {} | |
| 34 | + | |
| 35 | +static inline int nouveau_svmm_join(struct nouveau_svmm *svmm, u64 inst) | |
| 36 | +{ | |
| 37 | + return 0; | |
| 38 | +} | |
| 39 | + | |
| 40 | +static inline void nouveau_svmm_part(struct nouveau_svmm *svmm, u64 inst) {} | |
| 41 | + | |
| 42 | +static inline int nouveau_svmm_bind(struct drm_device *device, void *p, | |
| 43 | + struct drm_file *file) | |
| 44 | +{ | |
| 45 | + return -ENOSYS; | |
| 46 | +} | |
| 47 | +#endif /* IS_ENABLED(CONFIG_DRM_NOUVEAU_SVM) */ | |
| 48 | +#endif | |
| @@ -22,6 +22,7 @@ · 21 unmodified lines | ||
| 22 | 22 | #include "nouveau_vmm.h" |
| 23 | 23 | #include "nouveau_drv.h" |
| 24 | 24 | #include "nouveau_bo.h" |
| 25 | +#include "nouveau_svm.h" | |
| 25 | 26 | #include "nouveau_mem.h" |
| 26 | 27 | |
| 27 | 28 | void |
| @@ -119,6 +120,7 @@ done: · 91 unmodified lines | ||
| 119 | 120 | void |
| 120 | 121 | nouveau_vmm_fini(struct nouveau_vmm *vmm) |
| 121 | 122 | { |
| 123 | + nouveau_svmm_fini(&vmm->svmm); | |
| 122 | 124 | nvif_vmm_fini(&vmm->vmm); |
| 123 | 125 | vmm->cli = NULL; |
| 124 | 126 | } |
| @@ -126,7 +128,7 @@ nouveau_vmm_fini(struct nouveau_vmm *vmm) · 1 unmodified line | ||
| 126 | 128 | int |
| 127 | 129 | nouveau_vmm_init(struct nouveau_cli *cli, s32 oclass, struct nouveau_vmm *vmm) |
| 128 | 130 | { |
| 129 | − int ret = nvif_vmm_init(&cli->mmu, oclass, PAGE_SIZE, 0, NULL, 0, | |
| 131 | + int ret = nvif_vmm_init(&cli->mmu, oclass, false, PAGE_SIZE, 0, NULL, 0, | |
| 130 | 132 | &vmm->vmm); |
| 131 | 133 | if (ret) |
| 132 | 134 | return ret; |
| @@ -25,6 +25,7 @@ void nouveau_vma_unmap(struct nouveau_vma *); · 24 unmodified lines | ||
| 25 | 25 | struct nouveau_vmm { |
| 26 | 26 | struct nouveau_cli *cli; |
| 27 | 27 | struct nvif_vmm vmm; |
| 28 | + struct nouveau_svmm *svmm; | |
| 28 | 29 | }; |
| 29 | 30 | |
| 30 | 31 | int nouveau_vmm_init(struct nouveau_cli *, s32 oclass, struct nouveau_vmm *); |
| @@ -109,7 +109,6 @@ nv84_fence_context_del(struct nouveau_channel *chan) · 108 unmodified lines | ||
| 109 | 109 | int |
| 110 | 110 | nv84_fence_context_new(struct nouveau_channel *chan) |
| 111 | 111 | { |
| 112 | − struct nouveau_cli *cli = (void *)chan->user.client; | |
| 113 | 112 | struct nv84_fence_priv *priv = chan->drm->fence; |
| 114 | 113 | struct nv84_fence_chan *fctx; |
| 115 | 114 | int ret; |
| @@ -127,7 +126,7 @@ nv84_fence_context_new(struct nouveau_channel *chan) · 11 unmodified lines | ||
| 127 | 126 | fctx->base.sequence = nv84_fence_read(chan); |
| 128 | 127 | |
| 129 | 128 | mutex_lock(&priv->mutex); |
| 130 | − ret = nouveau_vma_new(priv->bo, &cli->vmm, &fctx->vma); | |
| 129 | + ret = nouveau_vma_new(priv->bo, chan->vmm, &fctx->vma); | |
| 131 | 130 | mutex_unlock(&priv->mutex); |
| 132 | 131 | |
| 133 | 132 | if (ret) |
| @@ -34,7 +34,7 @@ int · 33 unmodified lines | ||
| 34 | 34 | nvif_disp_ctor(struct nvif_device *device, s32 oclass, struct nvif_disp *disp) |
| 35 | 35 | { |
| 36 | 36 | static const struct nvif_mclass disps[] = { |
| 37 | − { TU104_DISP, -1 }, | |
| 37 | + { TU102_DISP, -1 }, | |
| 38 | 38 | { GV100_DISP, -1 }, |
| 39 | 39 | { GP102_DISP, -1 }, |
| 40 | 40 | { GP100_DISP, -1 }, |
| @@ -112,8 +112,8 @@ nvif_vmm_fini(struct nvif_vmm *vmm) · 111 unmodified lines | ||
| 112 | 112 | } |
| 113 | 113 | |
| 114 | 114 | int |
| 115 | −nvif_vmm_init(struct nvif_mmu *mmu, s32 oclass, u64 addr, u64 size, | |
| 116 | − void *argv, u32 argc, struct nvif_vmm *vmm) | |
| 115 | +nvif_vmm_init(struct nvif_mmu *mmu, s32 oclass, bool managed, u64 addr, | |
| 116 | + u64 size, void *argv, u32 argc, struct nvif_vmm *vmm) | |
| 117 | 117 | { |
| 118 | 118 | struct nvif_vmm_v0 *args; |
| 119 | 119 | u32 argn = sizeof(*args) + argc; |
| @@ -125,6 +125,7 @@ nvif_vmm_init(struct nvif_mmu *mmu, s32 oclass, u64 addr, u64 size, · 5 unmodified lines | ||
| 125 | 125 | if (!(args = kmalloc(argn, GFP_KERNEL))) |
| 126 | 126 | return -ENOMEM; |
| 127 | 127 | args->version = 0; |
| 128 | + args->managed = managed; | |
| 128 | 129 | args->addr = addr; |
| 129 | 130 | args->size = size; |
| 130 | 131 | memcpy(args->data, argv, argc); |
| @@ -39,6 +39,7 @@ nvkm_subdev_name[NVKM_SUBDEV_NR] = { · 38 unmodified lines | ||
| 39 | 39 | [NVKM_SUBDEV_FB ] = "fb", |
| 40 | 40 | [NVKM_SUBDEV_FUSE ] = "fuse", |
| 41 | 41 | [NVKM_SUBDEV_GPIO ] = "gpio", |
| 42 | + [NVKM_SUBDEV_GSP ] = "gsp", | |
| 42 | 43 | [NVKM_SUBDEV_I2C ] = "i2c", |
| 43 | 44 | [NVKM_SUBDEV_IBUS ] = "priv", |
| 44 | 45 | [NVKM_SUBDEV_ICCSENSE] = "iccsense", |
| @@ -6,4 +6,4 @@ nvkm-y += nvkm/engine/ce/gm200.o · 5 unmodified lines | ||
| 6 | 6 | nvkm-y += nvkm/engine/ce/gp100.o |
| 7 | 7 | nvkm-y += nvkm/engine/ce/gp102.o |
| 8 | 8 | nvkm-y += nvkm/engine/ce/gv100.o |
| 9 | −nvkm-y += nvkm/engine/ce/tu104.o | |
| 9 | +nvkm-y += nvkm/engine/ce/tu102.o | |
| @@ -24,7 +24,7 @@ · 23 unmodified lines | ||
| 24 | 24 | #include <nvif/class.h> |
| 25 | 25 | |
| 26 | 26 | static const struct nvkm_engine_func |
| 27 | −tu104_ce = { | |
| 27 | +tu102_ce = { | |
| 28 | 28 | .intr = gp100_ce_intr, |
| 29 | 29 | .sclass = { |
| 30 | 30 | { -1, -1, TURING_DMA_COPY_A }, |
| @@ -33,8 +33,8 @@ tu104_ce = { · 2 unmodified lines | ||
| 33 | 33 | }; |
| 34 | 34 | |
| 35 | 35 | int |
| 36 | −tu104_ce_new(struct nvkm_device *device, int index, | |
| 36 | +tu102_ce_new(struct nvkm_device *device, int index, | |
| 37 | 37 | struct nvkm_engine **pengine) |
| 38 | 38 | { |
| 39 | − return nvkm_engine_new_(&tu104_ce, device, index, true, pengine); | |
| 39 | + return nvkm_engine_new_(&tu102_ce, device, index, true, pengine); | |
| 40 | 40 | } |
| @@ -1613,7 +1613,7 @@ nvd7_chipset = { · 1612 unmodified lines | ||
| 1613 | 1613 | .pci = gf106_pci_new, |
| 1614 | 1614 | .therm = gf119_therm_new, |
| 1615 | 1615 | .timer = nv41_timer_new, |
| 1616 | − .volt = gf100_volt_new, | |
| 1616 | + .volt = gf117_volt_new, | |
| 1617 | 1617 | .ce[0] = gf100_ce_new, |
| 1618 | 1618 | .disp = gf119_disp_new, |
| 1619 | 1619 | .dma = gf119_dma_new, |
| @@ -2405,6 +2405,7 @@ nv140_chipset = { · 785 unmodified lines | ||
| 2405 | 2405 | .fb = gv100_fb_new, |
| 2406 | 2406 | .fuse = gm107_fuse_new, |
| 2407 | 2407 | .gpio = gk104_gpio_new, |
| 2408 | + .gsp = gv100_gsp_new, | |
| 2408 | 2409 | .i2c = gm200_i2c_new, |
| 2409 | 2410 | .ibus = gm200_ibus_new, |
| 2410 | 2411 | .imem = nv50_instmem_new, |
| @@ -2437,97 +2438,106 @@ nv140_chipset = { · 26 unmodified lines | ||
| 2437 | 2438 | static const struct nvkm_device_chip |
| 2438 | 2439 | nv162_chipset = { |
| 2439 | 2440 | .name = "TU102", |
| 2440 | − .bar = tu104_bar_new, | |
| 2441 | + .bar = tu102_bar_new, | |
| 2441 | 2442 | .bios = nvkm_bios_new, |
| 2442 | 2443 | .bus = gf100_bus_new, |
| 2443 | − .devinit = tu104_devinit_new, | |
| 2444 | − .fault = tu104_fault_new, | |
| 2444 | + .devinit = tu102_devinit_new, | |
| 2445 | + .fault = tu102_fault_new, | |
| 2445 | 2446 | .fb = gv100_fb_new, |
| 2446 | 2447 | .fuse = gm107_fuse_new, |
| 2447 | 2448 | .gpio = gk104_gpio_new, |
| 2449 | + .gsp = gv100_gsp_new, | |
| 2448 | 2450 | .i2c = gm200_i2c_new, |
| 2449 | 2451 | .ibus = gm200_ibus_new, |
| 2450 | 2452 | .imem = nv50_instmem_new, |
| 2451 | 2453 | .ltc = gp102_ltc_new, |
| 2452 | − .mc = tu104_mc_new, | |
| 2453 | − .mmu = tu104_mmu_new, | |
| 2454 | + .mc = tu102_mc_new, | |
| 2455 | + .mmu = tu102_mmu_new, | |
| 2454 | 2456 | .pci = gp100_pci_new, |
| 2455 | 2457 | .pmu = gp102_pmu_new, |
| 2456 | 2458 | .therm = gp100_therm_new, |
| 2457 | 2459 | .timer = gk20a_timer_new, |
| 2458 | 2460 | .top = gk104_top_new, |
| 2459 | − .ce[0] = tu104_ce_new, | |
| 2460 | − .ce[1] = tu104_ce_new, | |
| 2461 | − .ce[2] = tu104_ce_new, | |
| 2462 | − .ce[3] = tu104_ce_new, | |
| 2463 | − .ce[4] = tu104_ce_new, | |
| 2464 | − .disp = tu104_disp_new, | |
| 2461 | + .ce[0] = tu102_ce_new, | |
| 2462 | + .ce[1] = tu102_ce_new, | |
| 2463 | + .ce[2] = tu102_ce_new, | |
| 2464 | + .ce[3] = tu102_ce_new, | |
| 2465 | + .ce[4] = tu102_ce_new, | |
| 2466 | + .disp = tu102_disp_new, | |
| 2465 | 2467 | .dma = gv100_dma_new, |
| 2466 | − .fifo = tu104_fifo_new, | |
| 2468 | + .fifo = tu102_fifo_new, | |
| 2469 | + .nvdec[0] = gp102_nvdec_new, | |
| 2470 | + .sec2 = tu102_sec2_new, | |
| 2467 | 2471 | }; |
| 2468 | 2472 | |
| 2469 | 2473 | static const struct nvkm_device_chip |
| 2470 | 2474 | nv164_chipset = { |
| 2471 | 2475 | .name = "TU104", |
| 2472 | − .bar = tu104_bar_new, | |
| 2476 | + .bar = tu102_bar_new, | |
| 2473 | 2477 | .bios = nvkm_bios_new, |
| 2474 | 2478 | .bus = gf100_bus_new, |
| 2475 | − .devinit = tu104_devinit_new, | |
| 2476 | − .fault = tu104_fault_new, | |
| 2479 | + .devinit = tu102_devinit_new, | |
| 2480 | + .fault = tu102_fault_new, | |
| 2477 | 2481 | .fb = gv100_fb_new, |
| 2478 | 2482 | .fuse = gm107_fuse_new, |
| 2479 | 2483 | .gpio = gk104_gpio_new, |
| 2484 | + .gsp = gv100_gsp_new, | |
| 2480 | 2485 | .i2c = gm200_i2c_new, |
| 2481 | 2486 | .ibus = gm200_ibus_new, |
| 2482 | 2487 | .imem = nv50_instmem_new, |
| 2483 | 2488 | .ltc = gp102_ltc_new, |
| 2484 | − .mc = tu104_mc_new, | |
| 2485 | − .mmu = tu104_mmu_new, | |
| 2489 | + .mc = tu102_mc_new, | |
| 2490 | + .mmu = tu102_mmu_new, | |
| 2486 | 2491 | .pci = gp100_pci_new, |
| 2487 | 2492 | .pmu = gp102_pmu_new, |
| 2488 | 2493 | .therm = gp100_therm_new, |
| 2489 | 2494 | .timer = gk20a_timer_new, |
| 2490 | 2495 | .top = gk104_top_new, |
| 2491 | − .ce[0] = tu104_ce_new, | |
| 2492 | − .ce[1] = tu104_ce_new, | |
| 2493 | − .ce[2] = tu104_ce_new, | |
| 2494 | − .ce[3] = tu104_ce_new, | |
| 2495 | − .ce[4] = tu104_ce_new, | |
| 2496 | − .disp = tu104_disp_new, | |
| 2496 | + .ce[0] = tu102_ce_new, | |
| 2497 | + .ce[1] = tu102_ce_new, | |
| 2498 | + .ce[2] = tu102_ce_new, | |
| 2499 | + .ce[3] = tu102_ce_new, | |
| 2500 | + .ce[4] = tu102_ce_new, | |
| 2501 | + .disp = tu102_disp_new, | |
| 2497 | 2502 | .dma = gv100_dma_new, |
| 2498 | − .fifo = tu104_fifo_new, | |
| 2503 | + .fifo = tu102_fifo_new, | |
| 2504 | + .nvdec[0] = gp102_nvdec_new, | |
| 2505 | + .sec2 = tu102_sec2_new, | |
| 2499 | 2506 | }; |
| 2500 | 2507 | |
| 2501 | 2508 | static const struct nvkm_device_chip |
| 2502 | 2509 | nv166_chipset = { |
| 2503 | 2510 | .name = "TU106", |
| 2504 | − .bar = tu104_bar_new, | |
| 2511 | + .bar = tu102_bar_new, | |
| 2505 | 2512 | .bios = nvkm_bios_new, |
| 2506 | 2513 | .bus = gf100_bus_new, |
| 2507 | − .devinit = tu104_devinit_new, | |
| 2508 | − .fault = tu104_fault_new, | |
| 2514 | + .devinit = tu102_devinit_new, | |
| 2515 | + .fault = tu102_fault_new, | |
| 2509 | 2516 | .fb = gv100_fb_new, |
| 2510 | 2517 | .fuse = gm107_fuse_new, |
| 2511 | 2518 | .gpio = gk104_gpio_new, |
| 2519 | + .gsp = gv100_gsp_new, | |
| 2512 | 2520 | .i2c = gm200_i2c_new, |
| 2513 | 2521 | .ibus = gm200_ibus_new, |
| 2514 | 2522 | .imem = nv50_instmem_new, |
| 2515 | 2523 | .ltc = gp102_ltc_new, |
| 2516 | − .mc = tu104_mc_new, | |
| 2517 | − .mmu = tu104_mmu_new, | |
| 2524 | + .mc = tu102_mc_new, | |
| 2525 | + .mmu = tu102_mmu_new, | |
| 2518 | 2526 | .pci = gp100_pci_new, |
| 2519 | 2527 | .pmu = gp102_pmu_new, |
| 2520 | 2528 | .therm = gp100_therm_new, |
| 2521 | 2529 | .timer = gk20a_timer_new, |
| 2522 | 2530 | .top = gk104_top_new, |
| 2523 | − .ce[0] = tu104_ce_new, | |
| 2524 | − .ce[1] = tu104_ce_new, | |
| 2525 | − .ce[2] = tu104_ce_new, | |
| 2526 | − .ce[3] = tu104_ce_new, | |
| 2527 | − .ce[4] = tu104_ce_new, | |
| 2528 | − .disp = tu104_disp_new, | |
| 2531 | + .ce[0] = tu102_ce_new, | |
| 2532 | + .ce[1] = tu102_ce_new, | |
| 2533 | + .ce[2] = tu102_ce_new, | |
| 2534 | + .ce[3] = tu102_ce_new, | |
| 2535 | + .ce[4] = tu102_ce_new, | |
| 2536 | + .disp = tu102_disp_new, | |
| 2529 | 2537 | .dma = gv100_dma_new, |
| 2530 | − .fifo = tu104_fifo_new, | |
| 2538 | + .fifo = tu102_fifo_new, | |
| 2539 | + .nvdec[0] = gp102_nvdec_new, | |
| 2540 | + .sec2 = tu102_sec2_new, | |
| 2531 | 2541 | }; |
| 2532 | 2542 | |
| 2533 | 2543 | static int |
| @@ -2567,6 +2577,7 @@ nvkm_device_subdev(struct nvkm_device *device, int index) · 33 unmodified lines | ||
| 2567 | 2577 | _(FB , device->fb , &device->fb->subdev); |
| 2568 | 2578 | _(FUSE , device->fuse , &device->fuse->subdev); |
| 2569 | 2579 | _(GPIO , device->gpio , &device->gpio->subdev); |
| 2580 | + _(GSP , device->gsp , &device->gsp->subdev); | |
| 2570 | 2581 | _(I2C , device->i2c , &device->i2c->subdev); |
| 2571 | 2582 | _(IBUS , device->ibus , device->ibus); |
| 2572 | 2583 | _(ICCSENSE, device->iccsense, &device->iccsense->subdev); |
| @@ -3050,6 +3061,7 @@ nvkm_device_ctor(const struct nvkm_device_func *func, · 477 unmodified lines | ||
| 3050 | 3061 | _(NVKM_SUBDEV_FB , fb); |
| 3051 | 3062 | _(NVKM_SUBDEV_FUSE , fuse); |
| 3052 | 3063 | _(NVKM_SUBDEV_GPIO , gpio); |
| 3064 | + _(NVKM_SUBDEV_GSP , gsp); | |
| 3053 | 3065 | _(NVKM_SUBDEV_I2C , i2c); |
| 3054 | 3066 | _(NVKM_SUBDEV_IBUS , ibus); |
| 3055 | 3067 | _(NVKM_SUBDEV_ICCSENSE, iccsense); |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | #include <subdev/fb.h> |
| 13 | 13 | #include <subdev/fuse.h> |
| 14 | 14 | #include <subdev/gpio.h> |
| 15 | +#include <subdev/gsp.h> | |
| 15 | 16 | #include <subdev/i2c.h> |
| 16 | 17 | #include <subdev/ibus.h> |
| 17 | 18 | #include <subdev/iccsense.h> |
| @@ -365,16 +365,15 @@ nvkm_udevice_child_get(struct nvkm_object *object, int index, · 364 unmodified lines | ||
| 365 | 365 | } |
| 366 | 366 | |
| 367 | 367 | if (!sclass) { |
| 368 | − switch (index) { | |
| 369 | − case 0: sclass = &nvkm_control_oclass; break; | |
| 370 | − case 1: | |
| 371 | − if (!device->mmu) | |
| 372 | − return -EINVAL; | |
| 368 | + if (index-- == 0) | |
| 369 | + sclass = &nvkm_control_oclass; | |
| 370 | + else if (device->mmu && index-- == 0) | |
| 373 | 371 | sclass = &device->mmu->user; |
| 374 | − break; | |
| 375 | − default: | |
| 372 | + else if (device->fault && index-- == 0) | |
| 373 | + sclass = &device->fault->user; | |
| 374 | + else | |
| 376 | 375 | return -EINVAL; |
| 377 | − } | |
| 376 | + | |
| 378 | 377 | oclass->base = sclass->base; |
| 379 | 378 | } |
| 380 | 379 | |
| @@ -15,7 +15,7 @@ nvkm-y += nvkm/engine/disp/gm200.o · 14 unmodified lines | ||
| 15 | 15 | nvkm-y += nvkm/engine/disp/gp100.o |
| 16 | 16 | nvkm-y += nvkm/engine/disp/gp102.o |
| 17 | 17 | nvkm-y += nvkm/engine/disp/gv100.o |
| 18 | −nvkm-y += nvkm/engine/disp/tu104.o | |
| 18 | +nvkm-y += nvkm/engine/disp/tu102.o | |
| 19 | 19 | nvkm-y += nvkm/engine/disp/vga.o |
| 20 | 20 | |
| 21 | 21 | nvkm-y += nvkm/engine/disp/head.o |
| @@ -39,7 +39,7 @@ nvkm-y += nvkm/engine/disp/sorgk104.o · 17 unmodified lines | ||
| 39 | 39 | nvkm-y += nvkm/engine/disp/sorgm107.o |
| 40 | 40 | nvkm-y += nvkm/engine/disp/sorgm200.o |
| 41 | 41 | nvkm-y += nvkm/engine/disp/sorgv100.o |
| 42 | −nvkm-y += nvkm/engine/disp/sortu104.o | |
| 42 | +nvkm-y += nvkm/engine/disp/sortu102.o | |
| 43 | 43 | |
| 44 | 44 | nvkm-y += nvkm/engine/disp/outp.o |
| 45 | 45 | nvkm-y += nvkm/engine/disp/dp.o |
| @@ -71,7 +71,7 @@ nvkm-y += nvkm/engine/disp/rootgm200.o · 25 unmodified lines | ||
| 71 | 71 | nvkm-y += nvkm/engine/disp/rootgp100.o |
| 72 | 72 | nvkm-y += nvkm/engine/disp/rootgp102.o |
| 73 | 73 | nvkm-y += nvkm/engine/disp/rootgv100.o |
| 74 | −nvkm-y += nvkm/engine/disp/roottu104.o | |
| 74 | +nvkm-y += nvkm/engine/disp/roottu102.o | |
| 75 | 75 | |
| 76 | 76 | nvkm-y += nvkm/engine/disp/channv50.o |
| 77 | 77 | nvkm-y += nvkm/engine/disp/changf119.o |
| @@ -91,15 +91,21 @@ gf119_disp_intr_error(struct nv50_disp *disp, int chid) · 90 unmodified lines | ||
| 91 | 91 | { |
| 92 | 92 | struct nvkm_subdev *subdev = &disp->base.engine.subdev; |
| 93 | 93 | struct nvkm_device *device = subdev->device; |
| 94 | − u32 mthd = nvkm_rd32(device, 0x6101f0 + (chid * 12)); | |
| 94 | + u32 stat = nvkm_rd32(device, 0x6101f0 + (chid * 12)); | |
| 95 | + u32 type = (stat & 0x00007000) >> 12; | |
| 96 | + u32 mthd = (stat & 0x00000ffc); | |
| 95 | 97 | u32 data = nvkm_rd32(device, 0x6101f4 + (chid * 12)); |
| 96 | − u32 unkn = nvkm_rd32(device, 0x6101f8 + (chid * 12)); | |
| 98 | + u32 code = nvkm_rd32(device, 0x6101f8 + (chid * 12)); | |
| 99 | + const struct nvkm_enum *reason = | |
| 100 | + nvkm_enum_find(nv50_disp_intr_error_type, type); | |
| 97 | 101 | |
| 98 | − nvkm_error(subdev, "chid %d mthd %04x data %08x %08x %08x\n", | |
| 99 | − chid, (mthd & 0x0000ffc), data, mthd, unkn); | |
| 102 | + nvkm_error(subdev, "chid %d stat %08x reason %d [%s] mthd %04x " | |
| 103 | + "data %08x code %08x\n", | |
| 104 | + chid, stat, type, reason ? reason->name : "", | |
| 105 | + mthd, data, code); | |
| 100 | 106 | |
| 101 | 107 | if (chid < ARRAY_SIZE(disp->chan)) { |
| 102 | − switch (mthd & 0xffc) { | |
| 108 | + switch (mthd) { | |
| 103 | 109 | case 0x0080: |
| 104 | 110 | nv50_disp_chan_mthd(disp->chan[chid], NV_DBG_ERROR); |
| 105 | 111 | break; |
| @@ -103,10 +103,13 @@ gv100_disp_exception(struct nv50_disp *disp, int chid) · 102 unmodified lines | ||
| 103 | 103 | u32 mthd = (stat & 0x00000fff) << 2; |
| 104 | 104 | u32 data = nvkm_rd32(device, 0x611024 + (chid * 12)); |
| 105 | 105 | u32 code = nvkm_rd32(device, 0x611028 + (chid * 12)); |
| 106 | + const struct nvkm_enum *reason = | |
| 107 | + nvkm_enum_find(nv50_disp_intr_error_type, type); | |
| 106 | 108 | |
| 107 | − nvkm_error(subdev, "chid %d %08x [type %d mthd %04x] " | |
| 109 | + nvkm_error(subdev, "chid %d stat %08x reason %d [%s] mthd %04x " | |
| 108 | 110 | "data %08x code %08x\n", |
| 109 | − chid, stat, type, mthd, data, code); | |
| 111 | + chid, stat, type, reason ? reason->name : "", | |
| 112 | + mthd, data, code); | |
| 110 | 113 | |
| 111 | 114 | if (chid < ARRAY_SIZE(disp->chan) && disp->chan[chid]) { |
| 112 | 115 | switch (mthd) { |
| @@ -201,5 +201,5 @@ int gm200_sor_new(struct nvkm_disp *, int); · 200 unmodified lines | ||
| 201 | 201 | int gv100_sor_cnt(struct nvkm_disp *, unsigned long *); |
| 202 | 202 | int gv100_sor_new(struct nvkm_disp *, int); |
| 203 | 203 | |
| 204 | −int tu104_sor_new(struct nvkm_disp *, int); | |
| 204 | +int tu102_sor_new(struct nvkm_disp *, int); | |
| 205 | 205 | #endif |
| @@ -28,7 +28,6 @@ · 27 unmodified lines | ||
| 28 | 28 | #include "rootnv50.h" |
| 29 | 29 | |
| 30 | 30 | #include <core/client.h> |
| 31 | −#include <core/enum.h> | |
| 32 | 31 | #include <core/ramht.h> |
| 33 | 32 | #include <subdev/bios.h> |
| 34 | 33 | #include <subdev/bios/disp.h> |
| @@ -593,12 +592,15 @@ nv50_disp_super(struct work_struct *work) · 558 unmodified lines | ||
| 593 | 592 | nvkm_wr32(device, 0x610030, 0x80000000); |
| 594 | 593 | } |
| 595 | 594 | |
| 596 | −static const struct nvkm_enum | |
| 595 | +const struct nvkm_enum | |
| 597 | 596 | nv50_disp_intr_error_type[] = { |
| 598 | − { 3, "ILLEGAL_MTHD" }, | |
| 599 | − { 4, "INVALID_VALUE" }, | |
| 597 | + { 0, "NONE" }, | |
| 598 | + { 1, "PUSHBUFFER_ERR" }, | |
| 599 | + { 2, "TRAP" }, | |
| 600 | + { 3, "RESERVED_METHOD" }, | |
| 601 | + { 4, "INVALID_ARG" }, | |
| 600 | 602 | { 5, "INVALID_STATE" }, |
| 601 | − { 7, "INVALID_HANDLE" }, | |
| 603 | + { 7, "UNRESOLVABLE_HANDLE" }, | |
| 602 | 604 | {} |
| 603 | 605 | }; |
| 604 | 606 | |
| @@ -5,6 +5,8 @@ · 4 unmodified lines | ||
| 5 | 5 | #include "priv.h" |
| 6 | 6 | struct nvkm_head; |
| 7 | 7 | |
| 8 | +#include <core/enum.h> | |
| 9 | + | |
| 8 | 10 | struct nv50_disp { |
| 9 | 11 | const struct nv50_disp_func *func; |
| 10 | 12 | struct nvkm_disp base; |
| @@ -71,6 +73,7 @@ int nv50_disp_init(struct nv50_disp *); · 60 unmodified lines | ||
| 71 | 73 | void nv50_disp_fini(struct nv50_disp *); |
| 72 | 74 | void nv50_disp_intr(struct nv50_disp *); |
| 73 | 75 | void nv50_disp_super(struct work_struct *); |
| 76 | +extern const struct nvkm_enum nv50_disp_intr_error_type[]; | |
| 74 | 77 | |
| 75 | 78 | int gf119_disp_init(struct nv50_disp *); |
| 76 | 79 | void gf119_disp_fini(struct nv50_disp *); |
| @@ -37,5 +37,5 @@ extern const struct nvkm_disp_oclass gm200_disp_root_oclass; · 36 unmodified lines | ||
| 37 | 37 | extern const struct nvkm_disp_oclass gp100_disp_root_oclass; |
| 38 | 38 | extern const struct nvkm_disp_oclass gp102_disp_root_oclass; |
| 39 | 39 | extern const struct nvkm_disp_oclass gv100_disp_root_oclass; |
| 40 | −extern const struct nvkm_disp_oclass tu104_disp_root_oclass; | |
| 40 | +extern const struct nvkm_disp_oclass tu102_disp_root_oclass; | |
| 41 | 41 | #endif |
| @@ -25,28 +25,28 @@ · 24 unmodified lines | ||
| 25 | 25 | #include <nvif/class.h> |
| 26 | 26 | |
| 27 | 27 | static const struct nv50_disp_root_func |
| 28 | −tu104_disp_root = { | |
| 28 | +tu102_disp_root = { | |
| 29 | 29 | .user = { |
| 30 | − {{0,0,TU104_DISP_CURSOR }, gv100_disp_curs_new }, | |
| 31 | − {{0,0,TU104_DISP_WINDOW_IMM_CHANNEL_DMA}, gv100_disp_wimm_new }, | |
| 32 | − {{0,0,TU104_DISP_CORE_CHANNEL_DMA }, gv100_disp_core_new }, | |
| 33 | − {{0,0,TU104_DISP_WINDOW_CHANNEL_DMA }, gv100_disp_wndw_new }, | |
| 30 | + {{0,0,TU102_DISP_CURSOR }, gv100_disp_curs_new }, | |
| 31 | + {{0,0,TU102_DISP_WINDOW_IMM_CHANNEL_DMA}, gv100_disp_wimm_new }, | |
| 32 | + {{0,0,TU102_DISP_CORE_CHANNEL_DMA }, gv100_disp_core_new }, | |
| 33 | + {{0,0,TU102_DISP_WINDOW_CHANNEL_DMA }, gv100_disp_wndw_new }, | |
| 34 | 34 | {} |
| 35 | 35 | }, |
| 36 | 36 | }; |
| 37 | 37 | |
| 38 | 38 | static int |
| 39 | −tu104_disp_root_new(struct nvkm_disp *disp, const struct nvkm_oclass *oclass, | |
| 39 | +tu102_disp_root_new(struct nvkm_disp *disp, const struct nvkm_oclass *oclass, | |
| 40 | 40 | void *data, u32 size, struct nvkm_object **pobject) |
| 41 | 41 | { |
| 42 | − return nv50_disp_root_new_(&tu104_disp_root, disp, oclass, | |
| 42 | + return nv50_disp_root_new_(&tu102_disp_root, disp, oclass, | |
| 43 | 43 | data, size, pobject); |
| 44 | 44 | } |
| 45 | 45 | |
| 46 | 46 | const struct nvkm_disp_oclass |
| 47 | −tu104_disp_root_oclass = { | |
| 48 | − .base.oclass = TU104_DISP, | |
| 47 | +tu102_disp_root_oclass = { | |
| 48 | + .base.oclass = TU102_DISP, | |
| 49 | 49 | .base.minver = -1, |
| 50 | 50 | .base.maxver = -1, |
| 51 | − .ctor = tu104_disp_root_new, | |
| 51 | + .ctor = tu102_disp_root_new, | |
| 52 | 52 | }; |
| @@ -24,7 +24,7 @@ · 23 unmodified lines | ||
| 24 | 24 | #include <subdev/timer.h> |
| 25 | 25 | |
| 26 | 26 | static void |
| 27 | −tu104_sor_dp_vcpi(struct nvkm_ior *sor, int head, | |
| 27 | +tu102_sor_dp_vcpi(struct nvkm_ior *sor, int head, | |
| 28 | 28 | u8 slot, u8 slot_nr, u16 pbn, u16 aligned) |
| 29 | 29 | { |
| 30 | 30 | struct nvkm_device *device = sor->disp->engine.subdev.device; |
| @@ -35,7 +35,7 @@ tu104_sor_dp_vcpi(struct nvkm_ior *sor, int head, · 4 unmodified lines | ||
| 35 | 35 | } |
| 36 | 36 | |
| 37 | 37 | static int |
| 38 | −tu104_sor_dp_links(struct nvkm_ior *sor, struct nvkm_i2c_aux *aux) | |
| 38 | +tu102_sor_dp_links(struct nvkm_ior *sor, struct nvkm_i2c_aux *aux) | |
| 39 | 39 | { |
| 40 | 40 | struct nvkm_device *device = sor->disp->engine.subdev.device; |
| 41 | 41 | const u32 soff = nv50_ior_base(sor); |
| @@ -62,7 +62,7 @@ tu104_sor_dp_links(struct nvkm_ior *sor, struct nvkm_i2c_aux *aux) · 20 unmodified lines | ||
| 62 | 62 | } |
| 63 | 63 | |
| 64 | 64 | static const struct nvkm_ior_func |
| 65 | −tu104_sor = { | |
| 65 | +tu102_sor = { | |
| 66 | 66 | .route = { |
| 67 | 67 | .get = gm200_sor_route_get, |
| 68 | 68 | .set = gm200_sor_route_set, |
| @@ -75,11 +75,11 @@ tu104_sor = { · 6 unmodified lines | ||
| 75 | 75 | }, |
| 76 | 76 | .dp = { |
| 77 | 77 | .lanes = { 0, 1, 2, 3 }, |
| 78 | − .links = tu104_sor_dp_links, | |
| 78 | + .links = tu102_sor_dp_links, | |
| 79 | 79 | .power = g94_sor_dp_power, |
| 80 | 80 | .pattern = gm107_sor_dp_pattern, |
| 81 | 81 | .drive = gm200_sor_dp_drive, |
| 82 | − .vcpi = tu104_sor_dp_vcpi, | |
| 82 | + .vcpi = tu102_sor_dp_vcpi, | |
| 83 | 83 | .audio = gv100_sor_dp_audio, |
| 84 | 84 | .audio_sym = gv100_sor_dp_audio_sym, |
| 85 | 85 | .watermark = gv100_sor_dp_watermark, |
| @@ -91,7 +91,7 @@ tu104_sor = { · 5 unmodified lines | ||
| 91 | 91 | }; |
| 92 | 92 | |
| 93 | 93 | int |
| 94 | −tu104_sor_new(struct nvkm_disp *disp, int id) | |
| 94 | +tu102_sor_new(struct nvkm_disp *disp, int id) | |
| 95 | 95 | { |
| 96 | − return nvkm_ior_new_(&tu104_sor, disp, SOR, id); | |
| 96 | + return nvkm_ior_new_(&tu102_sor, disp, SOR, id); | |
| 97 | 97 | } |
| @@ -29,7 +29,7 @@ · 28 unmodified lines | ||
| 29 | 29 | #include <subdev/timer.h> |
| 30 | 30 | |
| 31 | 31 | static int |
| 32 | −tu104_disp_init(struct nv50_disp *disp) | |
| 32 | +tu102_disp_init(struct nv50_disp *disp) | |
| 33 | 33 | { |
| 34 | 34 | struct nvkm_device *device = disp->base.engine.subdev.device; |
| 35 | 35 | struct nvkm_head *head; |
| @@ -132,21 +132,21 @@ tu104_disp_init(struct nv50_disp *disp) · 96 unmodified lines | ||
| 132 | 132 | } |
| 133 | 133 | |
| 134 | 134 | static const struct nv50_disp_func |
| 135 | −tu104_disp = { | |
| 136 | − .init = tu104_disp_init, | |
| 135 | +tu102_disp = { | |
| 136 | + .init = tu102_disp_init, | |
| 137 | 137 | .fini = gv100_disp_fini, |
| 138 | 138 | .intr = gv100_disp_intr, |
| 139 | 139 | .uevent = &gv100_disp_chan_uevent, |
| 140 | 140 | .super = gv100_disp_super, |
| 141 | − .root = &tu104_disp_root_oclass, | |
| 141 | + .root = &tu102_disp_root_oclass, | |
| 142 | 142 | .wndw = { .cnt = gv100_disp_wndw_cnt }, |
| 143 | 143 | .head = { .cnt = gv100_head_cnt, .new = gv100_head_new }, |
| 144 | − .sor = { .cnt = gv100_sor_cnt, .new = tu104_sor_new }, | |
| 144 | + .sor = { .cnt = gv100_sor_cnt, .new = tu102_sor_new }, | |
| 145 | 145 | .ramht_size = 0x2000, |
| 146 | 146 | }; |
| 147 | 147 | |
| 148 | 148 | int |
| 149 | −tu104_disp_new(struct nvkm_device *device, int index, struct nvkm_disp **pdisp) | |
| 149 | +tu102_disp_new(struct nvkm_device *device, int index, struct nvkm_disp **pdisp) | |
| 150 | 150 | { |
| 151 | − return nv50_disp_new_(&tu104_disp, device, index, pdisp); | |
| 151 | + return nv50_disp_new_(&tu102_disp, device, index, pdisp); | |
| 152 | 152 | } |
| @@ -16,7 +16,7 @@ nvkm-y += nvkm/engine/fifo/gm20b.o · 15 unmodified lines | ||
| 16 | 16 | nvkm-y += nvkm/engine/fifo/gp100.o |
| 17 | 17 | nvkm-y += nvkm/engine/fifo/gp10b.o |
| 18 | 18 | nvkm-y += nvkm/engine/fifo/gv100.o |
| 19 | −nvkm-y += nvkm/engine/fifo/tu104.o | |
| 19 | +nvkm-y += nvkm/engine/fifo/tu102.o | |
| 20 | 20 | |
| 21 | 21 | nvkm-y += nvkm/engine/fifo/chan.o |
| 22 | 22 | nvkm-y += nvkm/engine/fifo/channv50.o |
| @@ -34,7 +34,7 @@ nvkm-y += nvkm/engine/fifo/gpfifog84.o · 11 unmodified lines | ||
| 34 | 34 | nvkm-y += nvkm/engine/fifo/gpfifogf100.o |
| 35 | 35 | nvkm-y += nvkm/engine/fifo/gpfifogk104.o |
| 36 | 36 | nvkm-y += nvkm/engine/fifo/gpfifogv100.o |
| 37 | −nvkm-y += nvkm/engine/fifo/gpfifotu104.o | |
| 37 | +nvkm-y += nvkm/engine/fifo/gpfifotu102.o | |
| 38 | 38 | |
| 39 | 39 | nvkm-y += nvkm/engine/fifo/usergv100.o |
| 40 | −nvkm-y += nvkm/engine/fifo/usertu104.o | |
| 40 | +nvkm-y += nvkm/engine/fifo/usertu102.o | |
| @@ -47,6 +47,6 @@ int gv100_fifo_gpfifo_engine_init(struct nvkm_fifo_chan *, · 46 unmodified lines | ||
| 47 | 47 | int gv100_fifo_gpfifo_engine_fini(struct nvkm_fifo_chan *, |
| 48 | 48 | struct nvkm_engine *, bool); |
| 49 | 49 | |
| 50 | −int tu104_fifo_gpfifo_new(struct gk104_fifo *, const struct nvkm_oclass *, | |
| 50 | +int tu102_fifo_gpfifo_new(struct gk104_fifo *, const struct nvkm_oclass *, | |
| 51 | 51 | void *data, u32 size, struct nvkm_object **); |
| 52 | 52 | #endif |
| @@ -29,14 +29,14 @@ · 28 unmodified lines | ||
| 29 | 29 | #include <nvif/unpack.h> |
| 30 | 30 | |
| 31 | 31 | static u32 |
| 32 | −tu104_fifo_gpfifo_submit_token(struct nvkm_fifo_chan *base) | |
| 32 | +tu102_fifo_gpfifo_submit_token(struct nvkm_fifo_chan *base) | |
| 33 | 33 | { |
| 34 | 34 | struct gk104_fifo_chan *chan = gk104_fifo_chan(base); |
| 35 | 35 | return (chan->runl << 16) | chan->base.chid; |
| 36 | 36 | } |
| 37 | 37 | |
| 38 | 38 | static const struct nvkm_fifo_chan_func |
| 39 | −tu104_fifo_gpfifo = { | |
| 39 | +tu102_fifo_gpfifo = { | |
| 40 | 40 | .dtor = gk104_fifo_gpfifo_dtor, |
| 41 | 41 | .init = gk104_fifo_gpfifo_init, |
| 42 | 42 | .fini = gk104_fifo_gpfifo_fini, |
| @@ -45,11 +45,11 @@ tu104_fifo_gpfifo = { · 2 unmodified lines | ||
| 45 | 45 | .engine_dtor = gk104_fifo_gpfifo_engine_dtor, |
| 46 | 46 | .engine_init = gv100_fifo_gpfifo_engine_init, |
| 47 | 47 | .engine_fini = gv100_fifo_gpfifo_engine_fini, |
| 48 | − .submit_token = tu104_fifo_gpfifo_submit_token, | |
| 48 | + .submit_token = tu102_fifo_gpfifo_submit_token, | |
| 49 | 49 | }; |
| 50 | 50 | |
| 51 | 51 | int |
| 52 | −tu104_fifo_gpfifo_new(struct gk104_fifo *fifo, const struct nvkm_oclass *oclass, | |
| 52 | +tu102_fifo_gpfifo_new(struct gk104_fifo *fifo, const struct nvkm_oclass *oclass, | |
| 53 | 53 | void *data, u32 size, struct nvkm_object **pobject) |
| 54 | 54 | { |
| 55 | 55 | struct nvkm_object *parent = oclass->parent; |
| @@ -67,7 +67,7 @@ tu104_fifo_gpfifo_new(struct gk104_fifo *fifo, const struct nvkm_oclass *oclass, · 11 unmodified lines | ||
| 67 | 67 | args->v0.ilength, args->v0.runlist, args->v0.priv); |
| 68 | 68 | if (args->v0.priv && !oclass->client->super) |
| 69 | 69 | return -EINVAL; |
| 70 | − return gv100_fifo_gpfifo_new_(&tu104_fifo_gpfifo, fifo, | |
| 70 | + return gv100_fifo_gpfifo_new_(&tu102_fifo_gpfifo, fifo, | |
| 71 | 71 | &args->v0.runlist, |
| 72 | 72 | &args->v0.chid, |
| 73 | 73 | args->v0.vmm, |
| @@ -29,7 +29,7 @@ · 28 unmodified lines | ||
| 29 | 29 | #include <nvif/class.h> |
| 30 | 30 | |
| 31 | 31 | static void |
| 32 | −tu104_fifo_runlist_commit(struct gk104_fifo *fifo, int runl, | |
| 32 | +tu102_fifo_runlist_commit(struct gk104_fifo *fifo, int runl, | |
| 33 | 33 | struct nvkm_memory *mem, int nr) |
| 34 | 34 | { |
| 35 | 35 | struct nvkm_device *device = fifo->base.engine.subdev.device; |
| @@ -44,15 +44,15 @@ tu104_fifo_runlist_commit(struct gk104_fifo *fifo, int runl, · 8 unmodified lines | ||
| 44 | 44 | } |
| 45 | 45 | |
| 46 | 46 | const struct gk104_fifo_runlist_func |
| 47 | −tu104_fifo_runlist = { | |
| 47 | +tu102_fifo_runlist = { | |
| 48 | 48 | .size = 16, |
| 49 | 49 | .cgrp = gv100_fifo_runlist_cgrp, |
| 50 | 50 | .chan = gv100_fifo_runlist_chan, |
| 51 | − .commit = tu104_fifo_runlist_commit, | |
| 51 | + .commit = tu102_fifo_runlist_commit, | |
| 52 | 52 | }; |
| 53 | 53 | |
| 54 | 54 | static const struct nvkm_enum |
| 55 | −tu104_fifo_fault_engine[] = { | |
| 55 | +tu102_fifo_fault_engine[] = { | |
| 56 | 56 | { 0x01, "DISPLAY" }, |
| 57 | 57 | { 0x03, "PTP" }, |
| 58 | 58 | { 0x06, "PWR_PMU" }, |
| @@ -80,7 +80,7 @@ tu104_fifo_fault_engine[] = { · 21 unmodified lines | ||
| 80 | 80 | }; |
| 81 | 81 | |
| 82 | 82 | static void |
| 83 | −tu104_fifo_pbdma_init(struct gk104_fifo *fifo) | |
| 83 | +tu102_fifo_pbdma_init(struct gk104_fifo *fifo) | |
| 84 | 84 | { |
| 85 | 85 | struct nvkm_device *device = fifo->base.engine.subdev.device; |
| 86 | 86 | const u32 mask = (1 << fifo->pbdma_nr) - 1; |
| @@ -89,28 +89,28 @@ tu104_fifo_pbdma_init(struct gk104_fifo *fifo) · 2 unmodified lines | ||
| 89 | 89 | } |
| 90 | 90 | |
| 91 | 91 | static const struct gk104_fifo_pbdma_func |
| 92 | −tu104_fifo_pbdma = { | |
| 92 | +tu102_fifo_pbdma = { | |
| 93 | 93 | .nr = gm200_fifo_pbdma_nr, |
| 94 | − .init = tu104_fifo_pbdma_init, | |
| 94 | + .init = tu102_fifo_pbdma_init, | |
| 95 | 95 | .init_timeout = gk208_fifo_pbdma_init_timeout, |
| 96 | 96 | }; |
| 97 | 97 | |
| 98 | 98 | static const struct gk104_fifo_func |
| 99 | −tu104_fifo = { | |
| 100 | − .pbdma = &tu104_fifo_pbdma, | |
| 99 | +tu102_fifo = { | |
| 100 | + .pbdma = &tu102_fifo_pbdma, | |
| 101 | 101 | .fault.access = gv100_fifo_fault_access, |
| 102 | − .fault.engine = tu104_fifo_fault_engine, | |
| 102 | + .fault.engine = tu102_fifo_fault_engine, | |
| 103 | 103 | .fault.reason = gv100_fifo_fault_reason, |
| 104 | 104 | .fault.hubclient = gv100_fifo_fault_hubclient, |
| 105 | 105 | .fault.gpcclient = gv100_fifo_fault_gpcclient, |
| 106 | − .runlist = &tu104_fifo_runlist, | |
| 107 | − .user = {{-1,-1,VOLTA_USERMODE_A }, tu104_fifo_user_new }, | |
| 108 | − .chan = {{ 0, 0,TURING_CHANNEL_GPFIFO_A}, tu104_fifo_gpfifo_new }, | |
| 106 | + .runlist = &tu102_fifo_runlist, | |
| 107 | + .user = {{-1,-1,VOLTA_USERMODE_A }, tu102_fifo_user_new }, | |
| 108 | + .chan = {{ 0, 0,TURING_CHANNEL_GPFIFO_A}, tu102_fifo_gpfifo_new }, | |
| 109 | 109 | .cgrp_force = true, |
| 110 | 110 | }; |
| 111 | 111 | |
| 112 | 112 | int |
| 113 | −tu104_fifo_new(struct nvkm_device *device, int index, struct nvkm_fifo **pfifo) | |
| 113 | +tu102_fifo_new(struct nvkm_device *device, int index, struct nvkm_fifo **pfifo) | |
| 114 | 114 | { |
| 115 | − return gk104_fifo_new_(&tu104_fifo, device, index, 4096, pfifo); | |
| 115 | + return gk104_fifo_new_(&tu102_fifo, device, index, 4096, pfifo); | |
| 116 | 116 | } |
| @@ -3,6 +3,6 @@ · 2 unmodified lines | ||
| 3 | 3 | #include "priv.h" |
| 4 | 4 | int gv100_fifo_user_new(const struct nvkm_oclass *, void *, u32, |
| 5 | 5 | struct nvkm_object **); |
| 6 | −int tu104_fifo_user_new(const struct nvkm_oclass *, void *, u32, | |
| 6 | +int tu102_fifo_user_new(const struct nvkm_oclass *, void *, u32, | |
| 7 | 7 | struct nvkm_object **); |
| 8 | 8 | #endif |
| @@ -22,7 +22,7 @@ · 21 unmodified lines | ||
| 22 | 22 | #include "user.h" |
| 23 | 23 | |
| 24 | 24 | static int |
| 25 | −tu104_fifo_user_map(struct nvkm_object *object, void *argv, u32 argc, | |
| 25 | +tu102_fifo_user_map(struct nvkm_object *object, void *argv, u32 argc, | |
| 26 | 26 | enum nvkm_object_map *type, u64 *addr, u64 *size) |
| 27 | 27 | { |
| 28 | 28 | struct nvkm_device *device = object->engine->subdev.device; |
| @@ -33,13 +33,13 @@ tu104_fifo_user_map(struct nvkm_object *object, void *argv, u32 argc, · 4 unmodified lines | ||
| 33 | 33 | } |
| 34 | 34 | |
| 35 | 35 | static const struct nvkm_object_func |
| 36 | −tu104_fifo_user = { | |
| 37 | − .map = tu104_fifo_user_map, | |
| 36 | +tu102_fifo_user = { | |
| 37 | + .map = tu102_fifo_user_map, | |
| 38 | 38 | }; |
| 39 | 39 | |
| 40 | 40 | int |
| 41 | −tu104_fifo_user_new(const struct nvkm_oclass *oclass, void *argv, u32 argc, | |
| 41 | +tu102_fifo_user_new(const struct nvkm_oclass *oclass, void *argv, u32 argc, | |
| 42 | 42 | struct nvkm_object **pobject) |
| 43 | 43 | { |
| 44 | − return nvkm_object_new_(&tu104_fifo_user, oclass, argv, argc, pobject); | |
| 44 | + return nvkm_object_new_(&tu102_fifo_user, oclass, argv, argc, pobject); | |
| 45 | 45 | } |
| @@ -25,6 +25,33 @@ · 24 unmodified lines | ||
| 25 | 25 | |
| 26 | 26 | #include <engine/fifo.h> |
| 27 | 27 | |
| 28 | +u32 | |
| 29 | +nvkm_gr_ctxsw_inst(struct nvkm_device *device) | |
| 30 | +{ | |
| 31 | + struct nvkm_gr *gr = device->gr; | |
| 32 | + if (gr && gr->func->ctxsw.inst) | |
| 33 | + return gr->func->ctxsw.inst(gr); | |
| 34 | + return 0; | |
| 35 | +} | |
| 36 | + | |
| 37 | +int | |
| 38 | +nvkm_gr_ctxsw_resume(struct nvkm_device *device) | |
| 39 | +{ | |
| 40 | + struct nvkm_gr *gr = device->gr; | |
| 41 | + if (gr && gr->func->ctxsw.resume) | |
| 42 | + return gr->func->ctxsw.resume(gr); | |
| 43 | + return 0; | |
| 44 | +} | |
| 45 | + | |
| 46 | +int | |
| 47 | +nvkm_gr_ctxsw_pause(struct nvkm_device *device) | |
| 48 | +{ | |
| 49 | + struct nvkm_gr *gr = device->gr; | |
| 50 | + if (gr && gr->func->ctxsw.pause) | |
| 51 | + return gr->func->ctxsw.pause(gr); | |
| 52 | + return 0; | |
| 53 | +} | |
| 54 | + | |
| 28 | 55 | static bool |
| 29 | 56 | nvkm_gr_chsw_load(struct nvkm_engine *engine) |
| 30 | 57 | { |
| @@ -1523,13 +1523,9 @@ gf100_grctx_generate(struct gf100_gr *gr) · 1522 unmodified lines | ||
| 1523 | 1523 | /* Make channel current. */ |
| 1524 | 1524 | addr = nvkm_memory_addr(inst) >> 12; |
| 1525 | 1525 | if (gr->firmware) { |
| 1526 | − nvkm_wr32(device, 0x409840, 0x00000030); | |
| 1527 | − nvkm_wr32(device, 0x409500, 0x80000000 | addr); | |
| 1528 | − nvkm_wr32(device, 0x409504, 0x00000003); | |
| 1529 | − nvkm_msec(device, 2000, | |
| 1530 | − if (nvkm_rd32(device, 0x409800) & 0x00000010) | |
| 1531 | − break; | |
| 1532 | − ); | |
| 1526 | + ret = gf100_gr_fecs_bind_pointer(gr, 0x80000000 | addr); | |
| 1527 | + if (ret) | |
| 1528 | + goto done; | |
| 1533 | 1529 | |
| 1534 | 1530 | nvkm_kmap(data); |
| 1535 | 1531 | nvkm_wo32(data, 0x1c, 1); |
| @@ -715,6 +715,211 @@ gf100_gr_pack_mmio[] = { · 714 unmodified lines | ||
| 715 | 715 | * PGRAPH engine/subdev functions |
| 716 | 716 | ******************************************************************************/ |
| 717 | 717 | |
| 718 | +static u32 | |
| 719 | +gf100_gr_ctxsw_inst(struct nvkm_gr *gr) | |
| 720 | +{ | |
| 721 | + return nvkm_rd32(gr->engine.subdev.device, 0x409b00); | |
| 722 | +} | |
| 723 | + | |
| 724 | +static int | |
| 725 | +gf100_gr_fecs_ctrl_ctxsw(struct gf100_gr *gr, u32 mthd) | |
| 726 | +{ | |
| 727 | + struct nvkm_device *device = gr->base.engine.subdev.device; | |
| 728 | + | |
| 729 | + nvkm_wr32(device, 0x409804, 0xffffffff); | |
| 730 | + nvkm_wr32(device, 0x409840, 0xffffffff); | |
| 731 | + nvkm_wr32(device, 0x409500, 0xffffffff); | |
| 732 | + nvkm_wr32(device, 0x409504, mthd); | |
| 733 | + nvkm_msec(device, 2000, | |
| 734 | + u32 stat = nvkm_rd32(device, 0x409804); | |
| 735 | + if (stat == 0x00000002) | |
| 736 | + return -EIO; | |
| 737 | + if (stat == 0x00000001) | |
| 738 | + return 0; | |
| 739 | + ); | |
| 740 | + | |
| 741 | + return -ETIMEDOUT; | |
| 742 | +} | |
| 743 | + | |
| 744 | +int | |
| 745 | +gf100_gr_fecs_start_ctxsw(struct nvkm_gr *base) | |
| 746 | +{ | |
| 747 | + struct gf100_gr *gr = gf100_gr(base); | |
| 748 | + int ret = 0; | |
| 749 | + | |
| 750 | + mutex_lock(&gr->fecs.mutex); | |
| 751 | + if (!--gr->fecs.disable) { | |
| 752 | + if (WARN_ON(ret = gf100_gr_fecs_ctrl_ctxsw(gr, 0x39))) | |
| 753 | + gr->fecs.disable++; | |
| 754 | + } | |
| 755 | + mutex_unlock(&gr->fecs.mutex); | |
| 756 | + return ret; | |
| 757 | +} | |
| 758 | + | |
| 759 | +int | |
| 760 | +gf100_gr_fecs_stop_ctxsw(struct nvkm_gr *base) | |
| 761 | +{ | |
| 762 | + struct gf100_gr *gr = gf100_gr(base); | |
| 763 | + int ret = 0; | |
| 764 | + | |
| 765 | + mutex_lock(&gr->fecs.mutex); | |
| 766 | + if (!gr->fecs.disable++) { | |
| 767 | + if (WARN_ON(ret = gf100_gr_fecs_ctrl_ctxsw(gr, 0x38))) | |
| 768 | + gr->fecs.disable--; | |
| 769 | + } | |
| 770 | + mutex_unlock(&gr->fecs.mutex); | |
| 771 | + return ret; | |
| 772 | +} | |
| 773 | + | |
| 774 | +int | |
| 775 | +gf100_gr_fecs_bind_pointer(struct gf100_gr *gr, u32 inst) | |
| 776 | +{ | |
| 777 | + struct nvkm_device *device = gr->base.engine.subdev.device; | |
| 778 | + | |
| 779 | + nvkm_wr32(device, 0x409840, 0x00000030); | |
| 780 | + nvkm_wr32(device, 0x409500, inst); | |
| 781 | + nvkm_wr32(device, 0x409504, 0x00000003); | |
| 782 | + nvkm_msec(device, 2000, | |
| 783 | + u32 stat = nvkm_rd32(device, 0x409800); | |
| 784 | + if (stat & 0x00000020) | |
| 785 | + return -EIO; | |
| 786 | + if (stat & 0x00000010) | |
| 787 | + return 0; | |
| 788 | + ); | |
| 789 | + | |
| 790 | + return -ETIMEDOUT; | |
| 791 | +} | |
| 792 | + | |
| 793 | +static int | |
| 794 | +gf100_gr_fecs_set_reglist_virtual_address(struct gf100_gr *gr, u64 addr) | |
| 795 | +{ | |
| 796 | + struct nvkm_device *device = gr->base.engine.subdev.device; | |
| 797 | + | |
| 798 | + nvkm_wr32(device, 0x409810, addr >> 8); | |
| 799 | + nvkm_wr32(device, 0x409800, 0x00000000); | |
| 800 | + nvkm_wr32(device, 0x409500, 0x00000001); | |
| 801 | + nvkm_wr32(device, 0x409504, 0x00000032); | |
| 802 | + nvkm_msec(device, 2000, | |
| 803 | + if (nvkm_rd32(device, 0x409800) == 0x00000001) | |
| 804 | + return 0; | |
| 805 | + ); | |
| 806 | + | |
| 807 | + return -ETIMEDOUT; | |
| 808 | +} | |
| 809 | + | |
| 810 | +static int | |
| 811 | +gf100_gr_fecs_set_reglist_bind_instance(struct gf100_gr *gr, u32 inst) | |
| 812 | +{ | |
| 813 | + struct nvkm_device *device = gr->base.engine.subdev.device; | |
| 814 | + | |
| 815 | + nvkm_wr32(device, 0x409810, inst); | |
| 816 | + nvkm_wr32(device, 0x409800, 0x00000000); | |
| 817 | + nvkm_wr32(device, 0x409500, 0x00000001); | |
| 818 | + nvkm_wr32(device, 0x409504, 0x00000031); | |
| 819 | + nvkm_msec(device, 2000, | |
| 820 | + if (nvkm_rd32(device, 0x409800) == 0x00000001) | |
| 821 | + return 0; | |
| 822 | + ); | |
| 823 | + | |
| 824 | + return -ETIMEDOUT; | |
| 825 | +} | |
| 826 | + | |
| 827 | +static int | |
| 828 | +gf100_gr_fecs_discover_reglist_image_size(struct gf100_gr *gr, u32 *psize) | |
| 829 | +{ | |
| 830 | + struct nvkm_device *device = gr->base.engine.subdev.device; | |
| 831 | + | |
| 832 | + nvkm_wr32(device, 0x409800, 0x00000000); | |
| 833 | + nvkm_wr32(device, 0x409500, 0x00000001); | |
| 834 | + nvkm_wr32(device, 0x409504, 0x00000030); | |
| 835 | + nvkm_msec(device, 2000, | |
| 836 | + if ((*psize = nvkm_rd32(device, 0x409800))) | |
| 837 | + return 0; | |
| 838 | + ); | |
| 839 | + | |
| 840 | + return -ETIMEDOUT; | |
| 841 | +} | |
| 842 | + | |
| 843 | +static int | |
| 844 | +gf100_gr_fecs_elpg_bind(struct gf100_gr *gr) | |
| 845 | +{ | |
| 846 | + u32 size; | |
| 847 | + int ret; | |
| 848 | + | |
| 849 | + ret = gf100_gr_fecs_discover_reglist_image_size(gr, &size); | |
| 850 | + if (ret) | |
| 851 | + return ret; | |
| 852 | + | |
| 853 | + /*XXX: We need to allocate + map the above into PMU's inst block, | |
| 854 | + * which which means we probably need a proper PMU before we | |
| 855 | + * even bother. | |
| 856 | + */ | |
| 857 | + | |
| 858 | + ret = gf100_gr_fecs_set_reglist_bind_instance(gr, 0); | |
| 859 | + if (ret) | |
| 860 | + return ret; | |
| 861 | + | |
| 862 | + return gf100_gr_fecs_set_reglist_virtual_address(gr, 0); | |
| 863 | +} | |
| 864 | + | |
| 865 | +static int | |
| 866 | +gf100_gr_fecs_discover_pm_image_size(struct gf100_gr *gr, u32 *psize) | |
| 867 | +{ | |
| 868 | + struct nvkm_device *device = gr->base.engine.subdev.device; | |
| 869 | + | |
| 870 | + nvkm_wr32(device, 0x409840, 0xffffffff); | |
| 871 | + nvkm_wr32(device, 0x409500, 0x00000000); | |
| 872 | + nvkm_wr32(device, 0x409504, 0x00000025); | |
| 873 | + nvkm_msec(device, 2000, | |
| 874 | + if ((*psize = nvkm_rd32(device, 0x409800))) | |
| 875 | + return 0; | |
| 876 | + ); | |
| 877 | + | |
| 878 | + return -ETIMEDOUT; | |
| 879 | +} | |
| 880 | + | |
| 881 | +static int | |
| 882 | +gf100_gr_fecs_discover_zcull_image_size(struct gf100_gr *gr, u32 *psize) | |
| 883 | +{ | |
| 884 | + struct nvkm_device *device = gr->base.engine.subdev.device; | |
| 885 | + | |
| 886 | + nvkm_wr32(device, 0x409840, 0xffffffff); | |
| 887 | + nvkm_wr32(device, 0x409500, 0x00000000); | |
| 888 | + nvkm_wr32(device, 0x409504, 0x00000016); | |
| 889 | + nvkm_msec(device, 2000, | |
| 890 | + if ((*psize = nvkm_rd32(device, 0x409800))) | |
| 891 | + return 0; | |
| 892 | + ); | |
| 893 | + | |
| 894 | + return -ETIMEDOUT; | |
| 895 | +} | |
| 896 | + | |
| 897 | +static int | |
| 898 | +gf100_gr_fecs_discover_image_size(struct gf100_gr *gr, u32 *psize) | |
| 899 | +{ | |
| 900 | + struct nvkm_device *device = gr->base.engine.subdev.device; | |
| 901 | + | |
| 902 | + nvkm_wr32(device, 0x409840, 0xffffffff); | |
| 903 | + nvkm_wr32(device, 0x409500, 0x00000000); | |
| 904 | + nvkm_wr32(device, 0x409504, 0x00000010); | |
| 905 | + nvkm_msec(device, 2000, | |
| 906 | + if ((*psize = nvkm_rd32(device, 0x409800))) | |
| 907 | + return 0; | |
| 908 | + ); | |
| 909 | + | |
| 910 | + return -ETIMEDOUT; | |
| 911 | +} | |
| 912 | + | |
| 913 | +static void | |
| 914 | +gf100_gr_fecs_set_watchdog_timeout(struct gf100_gr *gr, u32 timeout) | |
| 915 | +{ | |
| 916 | + struct nvkm_device *device = gr->base.engine.subdev.device; | |
| 917 | + | |
| 918 | + nvkm_wr32(device, 0x409840, 0xffffffff); | |
| 919 | + nvkm_wr32(device, 0x409500, timeout); | |
| 920 | + nvkm_wr32(device, 0x409504, 0x00000021); | |
| 921 | +} | |
| 922 | + | |
| 718 | 923 | static bool |
| 719 | 924 | gf100_gr_chsw_load(struct nvkm_gr *base) |
| 720 | 925 | { |
| @@ -1487,6 +1692,7 @@ gf100_gr_init_ctxctl_ext(struct gf100_gr *gr) · 766 unmodified lines | ||
| 1487 | 1692 | struct nvkm_device *device = subdev->device; |
| 1488 | 1693 | struct nvkm_secboot *sb = device->secboot; |
| 1489 | 1694 | u32 secboot_mask = 0; |
| 1695 | + int ret; | |
| 1490 | 1696 | |
| 1491 | 1697 | /* load fuc microcode */ |
| 1492 | 1698 | nvkm_mc_unk260(device, 0); |
| @@ -1495,12 +1701,12 @@ gf100_gr_init_ctxctl_ext(struct gf100_gr *gr) · 2 unmodified lines | ||
| 1495 | 1701 | if (nvkm_secboot_is_managed(sb, NVKM_SECBOOT_FALCON_FECS)) |
| 1496 | 1702 | secboot_mask |= BIT(NVKM_SECBOOT_FALCON_FECS); |
| 1497 | 1703 | else |
| 1498 | − gf100_gr_init_fw(gr->fecs, &gr->fuc409c, &gr->fuc409d); | |
| 1704 | + gf100_gr_init_fw(gr->fecs.falcon, &gr->fuc409c, &gr->fuc409d); | |
| 1499 | 1705 | |
| 1500 | 1706 | if (nvkm_secboot_is_managed(sb, NVKM_SECBOOT_FALCON_GPCCS)) |
| 1501 | 1707 | secboot_mask |= BIT(NVKM_SECBOOT_FALCON_GPCCS); |
| 1502 | 1708 | else |
| 1503 | − gf100_gr_init_fw(gr->gpccs, &gr->fuc41ac, &gr->fuc41ad); | |
| 1709 | + gf100_gr_init_fw(gr->gpccs.falcon, &gr->fuc41ac, &gr->fuc41ad); | |
| 1504 | 1710 | |
| 1505 | 1711 | if (secboot_mask != 0) { |
| 1506 | 1712 | int ret = nvkm_secboot_reset(sb, secboot_mask); |
| @@ -1515,8 +1721,8 @@ gf100_gr_init_ctxctl_ext(struct gf100_gr *gr) · 8 unmodified lines | ||
| 1515 | 1721 | nvkm_wr32(device, 0x41a10c, 0x00000000); |
| 1516 | 1722 | nvkm_wr32(device, 0x40910c, 0x00000000); |
| 1517 | 1723 | |
| 1518 | − nvkm_falcon_start(gr->gpccs); | |
| 1519 | − nvkm_falcon_start(gr->fecs); | |
| 1724 | + nvkm_falcon_start(gr->gpccs.falcon); | |
| 1725 | + nvkm_falcon_start(gr->fecs.falcon); | |
| 1520 | 1726 | |
| 1521 | 1727 | if (nvkm_msec(device, 2000, |
| 1522 | 1728 | if (nvkm_rd32(device, 0x409800) & 0x00000001) |
| @@ -1524,72 +1730,36 @@ gf100_gr_init_ctxctl_ext(struct gf100_gr *gr) · 1 unmodified line | ||
| 1524 | 1730 | ) < 0) |
| 1525 | 1731 | return -EBUSY; |
| 1526 | 1732 | |
| 1527 | − nvkm_wr32(device, 0x409840, 0xffffffff); | |
| 1528 | − nvkm_wr32(device, 0x409500, 0x7fffffff); | |
| 1529 | − nvkm_wr32(device, 0x409504, 0x00000021); | |
| 1733 | + gf100_gr_fecs_set_watchdog_timeout(gr, 0x7fffffff); | |
| 1530 | 1734 | |
| 1531 | − nvkm_wr32(device, 0x409840, 0xffffffff); | |
| 1532 | − nvkm_wr32(device, 0x409500, 0x00000000); | |
| 1533 | − nvkm_wr32(device, 0x409504, 0x00000010); | |
| 1534 | − if (nvkm_msec(device, 2000, | |
| 1535 | − if ((gr->size = nvkm_rd32(device, 0x409800))) | |
| 1536 | − break; | |
| 1537 | − ) < 0) | |
| 1538 | − return -EBUSY; | |
| 1539 | − | |
| 1540 | − nvkm_wr32(device, 0x409840, 0xffffffff); | |
| 1541 | − nvkm_wr32(device, 0x409500, 0x00000000); | |
| 1542 | − nvkm_wr32(device, 0x409504, 0x00000016); | |
| 1543 | − if (nvkm_msec(device, 2000, | |
| 1544 | − if (nvkm_rd32(device, 0x409800)) | |
| 1545 | − break; | |
| 1546 | − ) < 0) | |
| 1547 | − return -EBUSY; | |
| 1735 | + /* Determine how much memory is required to store main context image. */ | |
| 1736 | + ret = gf100_gr_fecs_discover_image_size(gr, &gr->size); | |
| 1737 | + if (ret) | |
| 1738 | + return ret; | |
| 1548 | 1739 | |
| 1549 | − nvkm_wr32(device, 0x409840, 0xffffffff); | |
| 1550 | − nvkm_wr32(device, 0x409500, 0x00000000); | |
| 1551 | − nvkm_wr32(device, 0x409504, 0x00000025); | |
| 1552 | − if (nvkm_msec(device, 2000, | |
| 1553 | − if (nvkm_rd32(device, 0x409800)) | |
| 1554 | − break; | |
| 1555 | − ) < 0) | |
| 1556 | − return -EBUSY; | |
| 1740 | + /* Determine how much memory is required to store ZCULL image. */ | |
| 1741 | + ret = gf100_gr_fecs_discover_zcull_image_size(gr, &gr->size_zcull); | |
| 1742 | + if (ret) | |
| 1743 | + return ret; | |
| 1557 | 1744 | |
| 1558 | − if (device->chipset >= 0xe0) { | |
| 1559 | − nvkm_wr32(device, 0x409800, 0x00000000); | |
| 1560 | − nvkm_wr32(device, 0x409500, 0x00000001); | |
| 1561 | − nvkm_wr32(device, 0x409504, 0x00000030); | |
| 1562 | − if (nvkm_msec(device, 2000, | |
| 1563 | − if (nvkm_rd32(device, 0x409800)) | |
| 1564 | − break; | |
| 1565 | − ) < 0) | |
| 1566 | − return -EBUSY; | |
| 1567 | − | |
| 1568 | − nvkm_wr32(device, 0x409810, 0xb00095c8); | |
| 1569 | − nvkm_wr32(device, 0x409800, 0x00000000); | |
| 1570 | − nvkm_wr32(device, 0x409500, 0x00000001); | |
| 1571 | − nvkm_wr32(device, 0x409504, 0x00000031); | |
| 1572 | − if (nvkm_msec(device, 2000, | |
| 1573 | − if (nvkm_rd32(device, 0x409800)) | |
| 1574 | − break; | |
| 1575 | − ) < 0) | |
| 1576 | − return -EBUSY; | |
| 1577 | − | |
| 1578 | − nvkm_wr32(device, 0x409810, 0x00080420); | |
| 1579 | − nvkm_wr32(device, 0x409800, 0x00000000); | |
| 1580 | − nvkm_wr32(device, 0x409500, 0x00000001); | |
| 1581 | − nvkm_wr32(device, 0x409504, 0x00000032); | |
| 1582 | − if (nvkm_msec(device, 2000, | |
| 1583 | − if (nvkm_rd32(device, 0x409800)) | |
| 1584 | − break; | |
| 1585 | − ) < 0) | |
| 1586 | − return -EBUSY; | |
| 1745 | + /* Determine how much memory is required to store PerfMon image. */ | |
| 1746 | + ret = gf100_gr_fecs_discover_pm_image_size(gr, &gr->size_pm); | |
| 1747 | + if (ret) | |
| 1748 | + return ret; | |
| 1587 | 1749 | |
| 1588 | − nvkm_wr32(device, 0x409614, 0x00000070); | |
| 1589 | − nvkm_wr32(device, 0x409614, 0x00000770); | |
| 1590 | − nvkm_wr32(device, 0x40802c, 0x00000001); | |
| 1750 | + /*XXX: We (likely) require PMU support to even bother with this. | |
| 1751 | + * | |
| 1752 | + * Also, it seems like not all GPUs support ELPG. Traces I | |
| 1753 | + * have here show RM enabling it on Kepler/Turing, but none | |
| 1754 | + * of the GPUs between those. NVGPU decides this by PCIID. | |
| 1755 | + */ | |
| 1756 | + if (0) { | |
| 1757 | + ret = gf100_gr_fecs_elpg_bind(gr); | |
| 1758 | + if (ret) | |
| 1759 | + return ret; | |
| 1591 | 1760 | } |
| 1592 | 1761 | |
| 1762 | + /* Generate golden context image. */ | |
| 1593 | 1763 | if (gr->data == NULL) { |
| 1594 | 1764 | int ret = gf100_grctx_generate(gr); |
| 1595 | 1765 | if (ret) { |
| @@ -1614,15 +1784,19 @@ gf100_gr_init_ctxctl_int(struct gf100_gr *gr) · 18 unmodified lines | ||
| 1614 | 1784 | |
| 1615 | 1785 | /* load HUB microcode */ |
| 1616 | 1786 | nvkm_mc_unk260(device, 0); |
| 1617 | − nvkm_falcon_load_dmem(gr->fecs, gr->func->fecs.ucode->data.data, 0x0, | |
| 1787 | + nvkm_falcon_load_dmem(gr->fecs.falcon, | |
| 1788 | + gr->func->fecs.ucode->data.data, 0x0, | |
| 1618 | 1789 | gr->func->fecs.ucode->data.size, 0); |
| 1619 | − nvkm_falcon_load_imem(gr->fecs, gr->func->fecs.ucode->code.data, 0x0, | |
| 1790 | + nvkm_falcon_load_imem(gr->fecs.falcon, | |
| 1791 | + gr->func->fecs.ucode->code.data, 0x0, | |
| 1620 | 1792 | gr->func->fecs.ucode->code.size, 0, 0, false); |
| 1621 | 1793 | |
| 1622 | 1794 | /* load GPC microcode */ |
| 1623 | − nvkm_falcon_load_dmem(gr->gpccs, gr->func->gpccs.ucode->data.data, 0x0, | |
| 1795 | + nvkm_falcon_load_dmem(gr->gpccs.falcon, | |
| 1796 | + gr->func->gpccs.ucode->data.data, 0x0, | |
| 1624 | 1797 | gr->func->gpccs.ucode->data.size, 0); |
| 1625 | − nvkm_falcon_load_imem(gr->gpccs, gr->func->gpccs.ucode->code.data, 0x0, | |
| 1798 | + nvkm_falcon_load_imem(gr->gpccs.falcon, | |
| 1799 | + gr->func->gpccs.ucode->code.data, 0x0, | |
| 1626 | 1800 | gr->func->gpccs.ucode->code.size, 0, 0, false); |
| 1627 | 1801 | nvkm_mc_unk260(device, 1); |
| 1628 | 1802 | |
| @@ -1769,11 +1943,13 @@ gf100_gr_oneinit(struct nvkm_gr *base) · 140 unmodified lines | ||
| 1769 | 1943 | int i, j; |
| 1770 | 1944 | int ret; |
| 1771 | 1945 | |
| 1772 | − ret = nvkm_falcon_v1_new(subdev, "FECS", 0x409000, &gr->fecs); | |
| 1946 | + ret = nvkm_falcon_v1_new(subdev, "FECS", 0x409000, &gr->fecs.falcon); | |
| 1773 | 1947 | if (ret) |
| 1774 | 1948 | return ret; |
| 1775 | 1949 | |
| 1776 | − ret = nvkm_falcon_v1_new(subdev, "GPCCS", 0x41a000, &gr->gpccs); | |
| 1950 | + mutex_init(&gr->fecs.mutex); | |
| 1951 | + | |
| 1952 | + ret = nvkm_falcon_v1_new(subdev, "GPCCS", 0x41a000, &gr->gpccs.falcon); | |
| 1777 | 1953 | if (ret) |
| 1778 | 1954 | return ret; |
| 1779 | 1955 | |
| @@ -1816,11 +1992,11 @@ gf100_gr_init_(struct nvkm_gr *base) · 36 unmodified lines | ||
| 1816 | 1992 | |
| 1817 | 1993 | nvkm_pmu_pgob(gr->base.engine.subdev.device->pmu, false); |
| 1818 | 1994 | |
| 1819 | − ret = nvkm_falcon_get(gr->fecs, subdev); | |
| 1995 | + ret = nvkm_falcon_get(gr->fecs.falcon, subdev); | |
| 1820 | 1996 | if (ret) |
| 1821 | 1997 | return ret; |
| 1822 | 1998 | |
| 1823 | − ret = nvkm_falcon_get(gr->gpccs, subdev); | |
| 1999 | + ret = nvkm_falcon_get(gr->gpccs.falcon, subdev); | |
| 1824 | 2000 | if (ret) |
| 1825 | 2001 | return ret; |
| 1826 | 2002 | |
| @@ -1832,8 +2008,8 @@ gf100_gr_fini_(struct nvkm_gr *base, bool suspend) · 5 unmodified lines | ||
| 1832 | 2008 | { |
| 1833 | 2009 | struct gf100_gr *gr = gf100_gr(base); |
| 1834 | 2010 | struct nvkm_subdev *subdev = &gr->base.engine.subdev; |
| 1835 | − nvkm_falcon_put(gr->gpccs, subdev); | |
| 1836 | − nvkm_falcon_put(gr->fecs, subdev); | |
| 2011 | + nvkm_falcon_put(gr->gpccs.falcon, subdev); | |
| 2012 | + nvkm_falcon_put(gr->fecs.falcon, subdev); | |
| 1837 | 2013 | return 0; |
| 1838 | 2014 | } |
| 1839 | 2015 | |
| @@ -1859,8 +2035,8 @@ gf100_gr_dtor(struct nvkm_gr *base) · 19 unmodified lines | ||
| 1859 | 2035 | gr->func->dtor(gr); |
| 1860 | 2036 | kfree(gr->data); |
| 1861 | 2037 | |
| 1862 | − nvkm_falcon_del(&gr->gpccs); | |
| 1863 | − nvkm_falcon_del(&gr->fecs); | |
| 2038 | + nvkm_falcon_del(&gr->gpccs.falcon); | |
| 2039 | + nvkm_falcon_del(&gr->fecs.falcon); | |
| 1864 | 2040 | |
| 1865 | 2041 | gf100_gr_dtor_fw(&gr->fuc409c); |
| 1866 | 2042 | gf100_gr_dtor_fw(&gr->fuc409d); |
| @@ -1886,6 +2062,9 @@ gf100_gr_ = { · 19 unmodified lines | ||
| 1886 | 2062 | .chan_new = gf100_gr_chan_new, |
| 1887 | 2063 | .object_get = gf100_gr_object_get, |
| 1888 | 2064 | .chsw_load = gf100_gr_chsw_load, |
| 2065 | + .ctxsw.pause = gf100_gr_fecs_stop_ctxsw, | |
| 2066 | + .ctxsw.resume = gf100_gr_fecs_start_ctxsw, | |
| 2067 | + .ctxsw.inst = gf100_gr_ctxsw_inst, | |
| 1889 | 2068 | }; |
| 1890 | 2069 | |
| 1891 | 2070 | int |
| @@ -82,8 +82,16 @@ struct gf100_gr { · 81 unmodified lines | ||
| 82 | 82 | const struct gf100_gr_func *func; |
| 83 | 83 | struct nvkm_gr base; |
| 84 | 84 | |
| 85 | − struct nvkm_falcon *fecs; | |
| 86 | − struct nvkm_falcon *gpccs; | |
| 85 | + struct { | |
| 86 | + struct nvkm_falcon *falcon; | |
| 87 | + struct mutex mutex; | |
| 88 | + u32 disable; | |
| 89 | + } fecs; | |
| 90 | + | |
| 91 | + struct { | |
| 92 | + struct nvkm_falcon *falcon; | |
| 93 | + } gpccs; | |
| 94 | + | |
| 87 | 95 | struct gf100_gr_fuc fuc409c; |
| 88 | 96 | struct gf100_gr_fuc fuc409d; |
| 89 | 97 | struct gf100_gr_fuc fuc41ac; |
| @@ -128,6 +136,8 @@ struct gf100_gr { · 38 unmodified lines | ||
| 128 | 136 | struct gf100_gr_mmio mmio_list[4096/8]; |
| 129 | 137 | u32 size; |
| 130 | 138 | u32 *data; |
| 139 | + u32 size_zcull; | |
| 140 | + u32 size_pm; | |
| 131 | 141 | }; |
| 132 | 142 | |
| 133 | 143 | int gf100_gr_ctor(const struct gf100_gr_func *, struct nvkm_device *, |
| @@ -136,6 +146,8 @@ int gf100_gr_new_(const struct gf100_gr_func *, struct nvkm_device *, · 2 unmodified lines | ||
| 136 | 146 | int, struct nvkm_gr **); |
| 137 | 147 | void *gf100_gr_dtor(struct nvkm_gr *); |
| 138 | 148 | |
| 149 | +int gf100_gr_fecs_bind_pointer(struct gf100_gr *, u32 inst); | |
| 150 | + | |
| 139 | 151 | struct gf100_gr_func_zbc { |
| 140 | 152 | void (*clear_color)(struct gf100_gr *, int zbc); |
| 141 | 153 | void (*clear_depth)(struct gf100_gr *, int zbc); |
| @@ -27,6 +27,11 @@ struct nvkm_gr_func { · 26 unmodified lines | ||
| 27 | 27 | */ |
| 28 | 28 | u64 (*units)(struct nvkm_gr *); |
| 29 | 29 | bool (*chsw_load)(struct nvkm_gr *); |
| 30 | + struct { | |
| 31 | + int (*pause)(struct nvkm_gr *); | |
| 32 | + int (*resume)(struct nvkm_gr *); | |
| 33 | + u32 (*inst)(struct nvkm_gr *); | |
| 34 | + } ctxsw; | |
| 30 | 35 | struct nvkm_sclass sclass[]; |
| 31 | 36 | }; |
| 32 | 37 | |
| @@ -21,13 +21,21 @@ · 20 unmodified lines | ||
| 21 | 21 | */ |
| 22 | 22 | #include "priv.h" |
| 23 | 23 | |
| 24 | +#include <subdev/top.h> | |
| 24 | 25 | #include <engine/falcon.h> |
| 25 | 26 | |
| 26 | 27 | static int |
| 27 | 28 | nvkm_nvdec_oneinit(struct nvkm_engine *engine) |
| 28 | 29 | { |
| 29 | 30 | struct nvkm_nvdec *nvdec = nvkm_nvdec(engine); |
| 30 | − return nvkm_falcon_v1_new(&nvdec->engine.subdev, "NVDEC", 0x84000, | |
| 31 | + struct nvkm_subdev *subdev = &nvdec->engine.subdev; | |
| 32 | + | |
| 33 | + nvdec->addr = nvkm_top_addr(subdev->device, subdev->index); | |
| 34 | + if (!nvdec->addr) | |
| 35 | + return -EINVAL; | |
| 36 | + | |
| 37 | + /*XXX: fix naming of this when adding support for multiple-NVDEC */ | |
| 38 | + return nvkm_falcon_v1_new(subdev, "NVDEC", nvdec->addr, | |
| 31 | 39 | &nvdec->falcon); |
| 32 | 40 | } |
| 33 | 41 | |
| @@ -1,2 +1,3 @@ | ||
| 1 | 1 | nvkm-y += nvkm/engine/sec2/base.o |
| 2 | 2 | nvkm-y += nvkm/engine/sec2/gp102.o |
| 3 | +nvkm-y += nvkm/engine/sec2/tu102.o | |
| @@ -22,6 +22,7 @@ · 21 unmodified lines | ||
| 22 | 22 | #include "priv.h" |
| 23 | 23 | |
| 24 | 24 | #include <core/msgqueue.h> |
| 25 | +#include <subdev/top.h> | |
| 25 | 26 | #include <engine/falcon.h> |
| 26 | 27 | |
| 27 | 28 | static void * |
| @@ -39,18 +40,18 @@ nvkm_sec2_intr(struct nvkm_engine *engine) · 11 unmodified lines | ||
| 39 | 40 | struct nvkm_sec2 *sec2 = nvkm_sec2(engine); |
| 40 | 41 | struct nvkm_subdev *subdev = &engine->subdev; |
| 41 | 42 | struct nvkm_device *device = subdev->device; |
| 42 | − u32 disp = nvkm_rd32(device, 0x8701c); | |
| 43 | − u32 intr = nvkm_rd32(device, 0x87008) & disp & ~(disp >> 16); | |
| 43 | + u32 disp = nvkm_rd32(device, sec2->addr + 0x01c); | |
| 44 | + u32 intr = nvkm_rd32(device, sec2->addr + 0x008) & disp & ~(disp >> 16); | |
| 44 | 45 | |
| 45 | 46 | if (intr & 0x00000040) { |
| 46 | 47 | schedule_work(&sec2->work); |
| 47 | − nvkm_wr32(device, 0x87004, 0x00000040); | |
| 48 | + nvkm_wr32(device, sec2->addr + 0x004, 0x00000040); | |
| 48 | 49 | intr &= ~0x00000040; |
| 49 | 50 | } |
| 50 | 51 | |
| 51 | 52 | if (intr) { |
| 52 | 53 | nvkm_error(subdev, "unhandled intr %08x\n", intr); |
| 53 | − nvkm_wr32(device, 0x87004, intr); | |
| 54 | + nvkm_wr32(device, sec2->addr + 0x004, intr); | |
| 54 | 55 | |
| 55 | 56 | } |
| 56 | 57 | } |
| @@ -74,8 +75,15 @@ static int · 17 unmodified lines | ||
| 74 | 75 | nvkm_sec2_oneinit(struct nvkm_engine *engine) |
| 75 | 76 | { |
| 76 | 77 | struct nvkm_sec2 *sec2 = nvkm_sec2(engine); |
| 77 | − return nvkm_falcon_v1_new(&sec2->engine.subdev, "SEC2", 0x87000, | |
| 78 | − &sec2->falcon); | |
| 78 | + struct nvkm_subdev *subdev = &sec2->engine.subdev; | |
| 79 | + | |
| 80 | + if (!sec2->addr) { | |
| 81 | + sec2->addr = nvkm_top_addr(subdev->device, subdev->index); | |
| 82 | + if (WARN_ON(!sec2->addr)) | |
| 83 | + return -EINVAL; | |
| 84 | + } | |
| 85 | + | |
| 86 | + return nvkm_falcon_v1_new(subdev, "SEC2", sec2->addr, &sec2->falcon); | |
| 79 | 87 | } |
| 80 | 88 | |
| 81 | 89 | static int |
| @@ -95,13 +103,14 @@ nvkm_sec2 = { · 13 unmodified lines | ||
| 95 | 103 | }; |
| 96 | 104 | |
| 97 | 105 | int |
| 98 | −nvkm_sec2_new_(struct nvkm_device *device, int index, | |
| 106 | +nvkm_sec2_new_(struct nvkm_device *device, int index, u32 addr, | |
| 99 | 107 | struct nvkm_sec2 **psec2) |
| 100 | 108 | { |
| 101 | 109 | struct nvkm_sec2 *sec2; |
| 102 | 110 | |
| 103 | 111 | if (!(sec2 = *psec2 = kzalloc(sizeof(*sec2), GFP_KERNEL))) |
| 104 | 112 | return -ENOMEM; |
| 113 | + sec2->addr = addr; | |
| 105 | 114 | INIT_WORK(&sec2->work, nvkm_sec2_recv); |
| 106 | 115 | |
| 107 | 116 | return nvkm_engine_ctor(&nvkm_sec2, device, index, true, &sec2->engine); |
| @@ -26,5 +26,5 @@ int · 25 unmodified lines | ||
| 26 | 26 | gp102_sec2_new(struct nvkm_device *device, int index, |
| 27 | 27 | struct nvkm_sec2 **psec2) |
| 28 | 28 | { |
| 29 | − return nvkm_sec2_new_(device, index, psec2); | |
| 29 | + return nvkm_sec2_new_(device, index, 0, psec2); | |
| 30 | 30 | } |
| @@ -5,6 +5,5 @@ · 4 unmodified lines | ||
| 5 | 5 | |
| 6 | 6 | #define nvkm_sec2(p) container_of((p), struct nvkm_sec2, engine) |
| 7 | 7 | |
| 8 | −int nvkm_sec2_new_(struct nvkm_device *, int, struct nvkm_sec2 **); | |
| 9 | − | |
| 8 | +int nvkm_sec2_new_(struct nvkm_device *, int, u32 addr, struct nvkm_sec2 **); | |
| 10 | 9 | #endif |
| @@ -0,0 +1,33 @@ | ||
| 1 | +/* | |
| 2 | + * Copyright 2019 Red Hat Inc. | |
| 3 | + * | |
| 4 | + * Permission is hereby granted, free of charge, to any person obtaining a | |
| 5 | + * copy of this software and associated documentation files (the "Software"), | |
| 6 | + * to deal in the Software without restriction, including without limitation | |
| 7 | + * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 8 | + * and/or sell copies of the Software, and to permit persons to whom the | |
| 9 | + * Software is furnished to do so, subject to the following conditions: | |
| 10 | + * | |
| 11 | + * The above copyright notice and this permission notice shall be included in | |
| 12 | + * all copies or substantial portions of the Software. | |
| 13 | + * | |
| 14 | + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 15 | + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 16 | + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 17 | + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 18 | + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 19 | + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 20 | + * OTHER DEALINGS IN THE SOFTWARE. | |
| 21 | + */ | |
| 22 | + | |
| 23 | +#include "priv.h" | |
| 24 | + | |
| 25 | +int | |
| 26 | +tu102_sec2_new(struct nvkm_device *device, int index, | |
| 27 | + struct nvkm_sec2 **psec2) | |
| 28 | +{ | |
| 29 | + /* TOP info wasn't updated on Turing to reflect the PRI | |
| 30 | + * address change for some reason. We override it here. | |
| 31 | + */ | |
| 32 | + return nvkm_sec2_new_(device, index, 0x840000, psec2); | |
| 33 | +} | |
| @@ -204,6 +204,9 @@ nvkm_falcon_ctor(const struct nvkm_falcon_func *func, · 203 unmodified lines | ||
| 204 | 204 | debug_reg = 0x408; |
| 205 | 205 | falcon->has_emem = true; |
| 206 | 206 | break; |
| 207 | + case NVKM_SUBDEV_GSP: | |
| 208 | + debug_reg = 0x0; /*XXX*/ | |
| 209 | + break; | |
| 207 | 210 | default: |
| 208 | 211 | nvkm_warn(subdev, "unsupported falcon %s!\n", |
| 209 | 212 | nvkm_subdev_name[subdev->index]); |
| @@ -269,7 +269,7 @@ cmd_write(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_hdr *cmd, · 268 unmodified lines | ||
| 269 | 269 | commit = false; |
| 270 | 270 | } |
| 271 | 271 | |
| 272 | − cmd_queue_close(priv, queue, commit); | |
| 272 | + cmd_queue_close(priv, queue, commit); | |
| 273 | 273 | |
| 274 | 274 | return ret; |
| 275 | 275 | } |
| @@ -347,7 +347,7 @@ nvkm_msgqueue_post(struct nvkm_msgqueue *priv, enum msgqueue_msg_priority prio, · 71 unmodified lines | ||
| 347 | 347 | ret = cmd_write(priv, cmd, queue); |
| 348 | 348 | if (ret) { |
| 349 | 349 | seq->state = SEQ_STATE_PENDING; |
| 350 | − msgqueue_seq_release(priv, seq); | |
| 350 | + msgqueue_seq_release(priv, seq); | |
| 351 | 351 | } |
| 352 | 352 | |
| 353 | 353 | return ret; |
| @@ -373,7 +373,7 @@ msgqueue_msg_handle(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_hdr *hdr) · 19 unmodified lines | ||
| 373 | 373 | if (seq->completion) |
| 374 | 374 | complete(seq->completion); |
| 375 | 375 | |
| 376 | − msgqueue_seq_release(priv, seq); | |
| 376 | + msgqueue_seq_release(priv, seq); | |
| 377 | 377 | |
| 378 | 378 | return 0; |
| 379 | 379 | } |
| @@ -7,6 +7,7 @@ include $(src)/nvkm/subdev/fault/Kbuild · 6 unmodified lines | ||
| 7 | 7 | include $(src)/nvkm/subdev/fb/Kbuild |
| 8 | 8 | include $(src)/nvkm/subdev/fuse/Kbuild |
| 9 | 9 | include $(src)/nvkm/subdev/gpio/Kbuild |
| 10 | +include $(src)/nvkm/subdev/gsp/Kbuild | |
| 10 | 11 | include $(src)/nvkm/subdev/i2c/Kbuild |
| 11 | 12 | include $(src)/nvkm/subdev/ibus/Kbuild |
| 12 | 13 | include $(src)/nvkm/subdev/iccsense/Kbuild |
| @@ -5,4 +5,4 @@ nvkm-y += nvkm/subdev/bar/gf100.o · 4 unmodified lines | ||
| 5 | 5 | nvkm-y += nvkm/subdev/bar/gk20a.o |
| 6 | 6 | nvkm-y += nvkm/subdev/bar/gm107.o |
| 7 | 7 | nvkm-y += nvkm/subdev/bar/gm20b.o |
| 8 | −nvkm-y += nvkm/subdev/bar/tu104.o | |
| 8 | +nvkm-y += nvkm/subdev/bar/tu102.o | |
| @@ -25,7 +25,7 @@ · 24 unmodified lines | ||
| 25 | 25 | #include <subdev/timer.h> |
| 26 | 26 | |
| 27 | 27 | static void |
| 28 | −tu104_bar_bar2_wait(struct nvkm_bar *bar) | |
| 28 | +tu102_bar_bar2_wait(struct nvkm_bar *bar) | |
| 29 | 29 | { |
| 30 | 30 | struct nvkm_device *device = bar->subdev.device; |
| 31 | 31 | nvkm_msec(device, 2000, |
| @@ -35,13 +35,13 @@ tu104_bar_bar2_wait(struct nvkm_bar *bar) · 3 unmodified lines | ||
| 35 | 35 | } |
| 36 | 36 | |
| 37 | 37 | static void |
| 38 | −tu104_bar_bar2_fini(struct nvkm_bar *bar) | |
| 38 | +tu102_bar_bar2_fini(struct nvkm_bar *bar) | |
| 39 | 39 | { |
| 40 | 40 | nvkm_mask(bar->subdev.device, 0xb80f48, 0x80000000, 0x00000000); |
| 41 | 41 | } |
| 42 | 42 | |
| 43 | 43 | static void |
| 44 | −tu104_bar_bar2_init(struct nvkm_bar *base) | |
| 44 | +tu102_bar_bar2_init(struct nvkm_bar *base) | |
| 45 | 45 | { |
| 46 | 46 | struct nvkm_device *device = base->subdev.device; |
| 47 | 47 | struct gf100_bar *bar = gf100_bar(base); |
| @@ -52,7 +52,7 @@ tu104_bar_bar2_init(struct nvkm_bar *base) · 4 unmodified lines | ||
| 52 | 52 | } |
| 53 | 53 | |
| 54 | 54 | static void |
| 55 | −tu104_bar_bar1_wait(struct nvkm_bar *bar) | |
| 55 | +tu102_bar_bar1_wait(struct nvkm_bar *bar) | |
| 56 | 56 | { |
| 57 | 57 | struct nvkm_device *device = bar->subdev.device; |
| 58 | 58 | nvkm_msec(device, 2000, |
| @@ -62,13 +62,13 @@ tu104_bar_bar1_wait(struct nvkm_bar *bar) · 3 unmodified lines | ||
| 62 | 62 | } |
| 63 | 63 | |
| 64 | 64 | static void |
| 65 | −tu104_bar_bar1_fini(struct nvkm_bar *bar) | |
| 65 | +tu102_bar_bar1_fini(struct nvkm_bar *bar) | |
| 66 | 66 | { |
| 67 | 67 | nvkm_mask(bar->subdev.device, 0xb80f40, 0x80000000, 0x00000000); |
| 68 | 68 | } |
| 69 | 69 | |
| 70 | 70 | static void |
| 71 | −tu104_bar_bar1_init(struct nvkm_bar *base) | |
| 71 | +tu102_bar_bar1_init(struct nvkm_bar *base) | |
| 72 | 72 | { |
| 73 | 73 | struct nvkm_device *device = base->subdev.device; |
| 74 | 74 | struct gf100_bar *bar = gf100_bar(base); |
| @@ -77,22 +77,22 @@ tu104_bar_bar1_init(struct nvkm_bar *base) · 2 unmodified lines | ||
| 77 | 77 | } |
| 78 | 78 | |
| 79 | 79 | static const struct nvkm_bar_func |
| 80 | −tu104_bar = { | |
| 80 | +tu102_bar = { | |
| 81 | 81 | .dtor = gf100_bar_dtor, |
| 82 | 82 | .oneinit = gf100_bar_oneinit, |
| 83 | − .bar1.init = tu104_bar_bar1_init, | |
| 84 | − .bar1.fini = tu104_bar_bar1_fini, | |
| 85 | − .bar1.wait = tu104_bar_bar1_wait, | |
| 83 | + .bar1.init = tu102_bar_bar1_init, | |
| 84 | + .bar1.fini = tu102_bar_bar1_fini, | |
| 85 | + .bar1.wait = tu102_bar_bar1_wait, | |
| 86 | 86 | .bar1.vmm = gf100_bar_bar1_vmm, |
| 87 | − .bar2.init = tu104_bar_bar2_init, | |
| 88 | − .bar2.fini = tu104_bar_bar2_fini, | |
| 89 | − .bar2.wait = tu104_bar_bar2_wait, | |
| 87 | + .bar2.init = tu102_bar_bar2_init, | |
| 88 | + .bar2.fini = tu102_bar_bar2_fini, | |
| 89 | + .bar2.wait = tu102_bar_bar2_wait, | |
| 90 | 90 | .bar2.vmm = gf100_bar_bar2_vmm, |
| 91 | 91 | .flush = g84_bar_flush, |
| 92 | 92 | }; |
| 93 | 93 | |
| 94 | 94 | int |
| 95 | −tu104_bar_new(struct nvkm_device *device, int index, struct nvkm_bar **pbar) | |
| 95 | +tu102_bar_new(struct nvkm_device *device, int index, struct nvkm_bar **pbar) | |
| 96 | 96 | { |
| 97 | − return gf100_bar_new_(&tu104_bar, device, index, pbar); | |
| 97 | + return gf100_bar_new_(&tu102_bar, device, index, pbar); | |
| 98 | 98 | } |
| @@ -212,7 +212,7 @@ nvbios_dpcfg_match(struct nvkm_bios *bios, u16 outp, u8 pc, u8 vs, u8 pe, · 211 unmodified lines | ||
| 212 | 212 | u16 data; |
| 213 | 213 | |
| 214 | 214 | if (*ver >= 0x30) { |
| 215 | − const u8 vsoff[] = { 0, 4, 7, 9 }; | |
| 215 | + static const u8 vsoff[] = { 0, 4, 7, 9 }; | |
| 216 | 216 | idx = (pc * 10) + vsoff[vs] + pe; |
| 217 | 217 | if (*ver >= 0x40 && *ver <= 0x41 && *hdr >= 0x12) |
| 218 | 218 | idx += nvbios_rd08(bios, outp + 0x11) * 40; |
| @@ -806,12 +806,12 @@ init_generic_condition(struct nvbios_init *init) · 805 unmodified lines | ||
| 806 | 806 | init->offset += 3; |
| 807 | 807 | |
| 808 | 808 | switch (cond) { |
| 809 | − case 0: | |
| 809 | + case 0: /* CONDITION_ID_INT_DP. */ | |
| 810 | 810 | if (init_conn(init) != DCB_CONNECTOR_eDP) |
| 811 | 811 | init_exec_set(init, false); |
| 812 | 812 | break; |
| 813 | − case 1: | |
| 814 | − case 2: | |
| 813 | + case 1: /* CONDITION_ID_USE_SPPLL0. */ | |
| 814 | + case 2: /* CONDITION_ID_USE_SPPLL1. */ | |
| 815 | 815 | if ( init->outp && |
| 816 | 816 | (data = nvbios_dpout_match(bios, DCB_OUTPUT_DP, |
| 817 | 817 | (init->outp->or << 0) | |
| @@ -826,10 +826,13 @@ init_generic_condition(struct nvbios_init *init) · 8 unmodified lines | ||
| 826 | 826 | if (init_exec(init)) |
| 827 | 827 | warn("script needs dp output table data\n"); |
| 828 | 828 | break; |
| 829 | − case 5: | |
| 829 | + case 5: /* CONDITION_ID_ASSR_SUPPORT. */ | |
| 830 | 830 | if (!(init_rdauxr(init, 0x0d) & 1)) |
| 831 | 831 | init_exec_set(init, false); |
| 832 | 832 | break; |
| 833 | + case 7: /* CONDITION_ID_NO_PANEL_SEQ_DELAYS. */ | |
| 834 | + init_exec_set(init, false); | |
| 835 | + break; | |
| 833 | 836 | default: |
| 834 | 837 | warn("INIT_GENERIC_CONDITON: unknown 0x%02x\n", cond); |
| 835 | 838 | init->offset += size; |
| @@ -13,4 +13,4 @@ nvkm-y += nvkm/subdev/devinit/gf100.o · 12 unmodified lines | ||
| 13 | 13 | nvkm-y += nvkm/subdev/devinit/gm107.o |
| 14 | 14 | nvkm-y += nvkm/subdev/devinit/gm200.o |
| 15 | 15 | nvkm-y += nvkm/subdev/devinit/gv100.o |
| 16 | −nvkm-y += nvkm/subdev/devinit/tu104.o | |
| 16 | +nvkm-y += nvkm/subdev/devinit/tu102.o | |
| @@ -26,7 +26,7 @@ · 25 unmodified lines | ||
| 26 | 26 | #include <subdev/clk/pll.h> |
| 27 | 27 | |
| 28 | 28 | static int |
| 29 | −tu104_devinit_pll_set(struct nvkm_devinit *init, u32 type, u32 freq) | |
| 29 | +tu102_devinit_pll_set(struct nvkm_devinit *init, u32 type, u32 freq) | |
| 30 | 30 | { |
| 31 | 31 | struct nvkm_subdev *subdev = &init->subdev; |
| 32 | 32 | struct nvkm_device *device = subdev->device; |
| @@ -66,7 +66,7 @@ tu104_devinit_pll_set(struct nvkm_devinit *init, u32 type, u32 freq) · 33 unmodified lines | ||
| 66 | 66 | } |
| 67 | 67 | |
| 68 | 68 | static int |
| 69 | −tu104_devinit_post(struct nvkm_devinit *base, bool post) | |
| 69 | +tu102_devinit_post(struct nvkm_devinit *base, bool post) | |
| 70 | 70 | { |
| 71 | 71 | struct nv50_devinit *init = nv50_devinit(base); |
| 72 | 72 | gm200_devinit_preos(init, post); |
| @@ -74,16 +74,16 @@ tu104_devinit_post(struct nvkm_devinit *base, bool post) · 1 unmodified line | ||
| 74 | 74 | } |
| 75 | 75 | |
| 76 | 76 | static const struct nvkm_devinit_func |
| 77 | −tu104_devinit = { | |
| 77 | +tu102_devinit = { | |
| 78 | 78 | .init = nv50_devinit_init, |
| 79 | − .post = tu104_devinit_post, | |
| 80 | − .pll_set = tu104_devinit_pll_set, | |
| 79 | + .post = tu102_devinit_post, | |
| 80 | + .pll_set = tu102_devinit_pll_set, | |
| 81 | 81 | .disable = gm107_devinit_disable, |
| 82 | 82 | }; |
| 83 | 83 | |
| 84 | 84 | int |
| 85 | −tu104_devinit_new(struct nvkm_device *device, int index, | |
| 85 | +tu102_devinit_new(struct nvkm_device *device, int index, | |
| 86 | 86 | struct nvkm_devinit **pinit) |
| 87 | 87 | { |
| 88 | − return nv50_devinit_new_(&tu104_devinit, device, index, pinit); | |
| 88 | + return nv50_devinit_new_(&tu102_devinit, device, index, pinit); | |
| 89 | 89 | } |
| @@ -1,4 +1,5 @@ | ||
| 1 | 1 | nvkm-y += nvkm/subdev/fault/base.o |
| 2 | +nvkm-y += nvkm/subdev/fault/user.o | |
| 2 | 3 | nvkm-y += nvkm/subdev/fault/gp100.o |
| 3 | 4 | nvkm-y += nvkm/subdev/fault/gv100.o |
| 4 | −nvkm-y += nvkm/subdev/fault/tu104.o | |
| 5 | +nvkm-y += nvkm/subdev/fault/tu102.o | |
| @@ -176,5 +176,7 @@ nvkm_fault_new_(const struct nvkm_fault_func *func, struct nvkm_device *device, · 175 unmodified lines | ||
| 176 | 176 | return -ENOMEM; |
| 177 | 177 | nvkm_subdev_ctor(&nvkm_fault, device, index, &fault->subdev); |
| 178 | 178 | fault->func = func; |
| 179 | + fault->user.ctor = nvkm_ufault_new; | |
| 180 | + fault->user.base = func->user.base; | |
| 179 | 181 | return 0; |
| 180 | 182 | } |
| @@ -23,6 +23,8 @@ · 22 unmodified lines | ||
| 23 | 23 | |
| 24 | 24 | #include <subdev/mc.h> |
| 25 | 25 | |
| 26 | +#include <nvif/class.h> | |
| 27 | + | |
| 26 | 28 | static void |
| 27 | 29 | gp100_fault_buffer_intr(struct nvkm_fault_buffer *buffer, bool enable) |
| 28 | 30 | { |
| @@ -69,6 +71,7 @@ gp100_fault = { · 40 unmodified lines | ||
| 69 | 71 | .buffer.init = gp100_fault_buffer_init, |
| 70 | 72 | .buffer.fini = gp100_fault_buffer_fini, |
| 71 | 73 | .buffer.intr = gp100_fault_buffer_intr, |
| 74 | + .user = { { 0, 0, MAXWELL_FAULT_BUFFER_A }, 0 }, | |
| 72 | 75 | }; |
| 73 | 76 | |
| 74 | 77 | int |
| @@ -25,6 +25,8 @@ · 24 unmodified lines | ||
| 25 | 25 | #include <subdev/mmu.h> |
| 26 | 26 | #include <engine/fifo.h> |
| 27 | 27 | |
| 28 | +#include <nvif/class.h> | |
| 29 | + | |
| 28 | 30 | static void |
| 29 | 31 | gv100_fault_buffer_process(struct nvkm_fault_buffer *buffer) |
| 30 | 32 | { |
| @@ -166,6 +168,13 @@ gv100_fault_intr(struct nvkm_fault *fault) · 135 unmodified lines | ||
| 166 | 168 | } |
| 167 | 169 | } |
| 168 | 170 | |
| 171 | + if (stat & 0x08000000) { | |
| 172 | + if (fault->buffer[1]) { | |
| 173 | + nvkm_event_send(&fault->event, 1, 1, NULL, 0); | |
| 174 | + stat &= ~0x08000000; | |
| 175 | + } | |
| 176 | + } | |
| 177 | + | |
| 169 | 178 | if (stat) { |
| 170 | 179 | nvkm_debug(subdev, "intr %08x\n", stat); |
| 171 | 180 | } |
| @@ -208,6 +217,13 @@ gv100_fault = { · 36 unmodified lines | ||
| 208 | 217 | .buffer.init = gv100_fault_buffer_init, |
| 209 | 218 | .buffer.fini = gv100_fault_buffer_fini, |
| 210 | 219 | .buffer.intr = gv100_fault_buffer_intr, |
| 220 | + /*TODO: Figure out how to expose non-replayable fault buffer, which, | |
| 221 | + * for some reason, is where recoverable CE faults appear... | |
| 222 | + * | |
| 223 | + * It's a bit tricky, as both NVKM and SVM will need access to | |
| 224 | + * the non-replayable fault buffer. | |
| 225 | + */ | |
| 226 | + .user = { { 0, 0, VOLTA_FAULT_BUFFER_A }, 1 }, | |
| 211 | 227 | }; |
| 212 | 228 | |
| 213 | 229 | int |
| @@ -34,7 +34,14 @@ struct nvkm_fault_func { · 33 unmodified lines | ||
| 34 | 34 | void (*fini)(struct nvkm_fault_buffer *); |
| 35 | 35 | void (*intr)(struct nvkm_fault_buffer *, bool enable); |
| 36 | 36 | } buffer; |
| 37 | + struct { | |
| 38 | + struct nvkm_sclass base; | |
| 39 | + int rp; | |
| 40 | + } user; | |
| 37 | 41 | }; |
| 38 | 42 | |
| 39 | 43 | int gv100_fault_oneinit(struct nvkm_fault *); |
| 44 | + | |
| 45 | +int nvkm_ufault_new(struct nvkm_device *, const struct nvkm_oclass *, | |
| 46 | + void *, u32, struct nvkm_object **); | |
| 40 | 47 | #endif |
| @@ -28,7 +28,7 @@ · 27 unmodified lines | ||
| 28 | 28 | #include <nvif/class.h> |
| 29 | 29 | |
| 30 | 30 | static void |
| 31 | −tu104_fault_buffer_intr(struct nvkm_fault_buffer *buffer, bool enable) | |
| 31 | +tu102_fault_buffer_intr(struct nvkm_fault_buffer *buffer, bool enable) | |
| 32 | 32 | { |
| 33 | 33 | /*XXX: Earlier versions of RM touched the old regs on Turing, |
| 34 | 34 | * which don't appear to actually work anymore, but newer |
| @@ -37,7 +37,7 @@ tu104_fault_buffer_intr(struct nvkm_fault_buffer *buffer, bool enable) · 2 unmodified lines | ||
| 37 | 37 | } |
| 38 | 38 | |
| 39 | 39 | static void |
| 40 | −tu104_fault_buffer_fini(struct nvkm_fault_buffer *buffer) | |
| 40 | +tu102_fault_buffer_fini(struct nvkm_fault_buffer *buffer) | |
| 41 | 41 | { |
| 42 | 42 | struct nvkm_device *device = buffer->fault->subdev.device; |
| 43 | 43 | const u32 foff = buffer->id * 0x20; |
| @@ -45,7 +45,7 @@ tu104_fault_buffer_fini(struct nvkm_fault_buffer *buffer) · 1 unmodified line | ||
| 45 | 45 | } |
| 46 | 46 | |
| 47 | 47 | static void |
| 48 | −tu104_fault_buffer_init(struct nvkm_fault_buffer *buffer) | |
| 48 | +tu102_fault_buffer_init(struct nvkm_fault_buffer *buffer) | |
| 49 | 49 | { |
| 50 | 50 | struct nvkm_device *device = buffer->fault->subdev.device; |
| 51 | 51 | const u32 foff = buffer->id * 0x20; |
| @@ -57,7 +57,7 @@ tu104_fault_buffer_init(struct nvkm_fault_buffer *buffer) · 5 unmodified lines | ||
| 57 | 57 | } |
| 58 | 58 | |
| 59 | 59 | static void |
| 60 | −tu104_fault_buffer_info(struct nvkm_fault_buffer *buffer) | |
| 60 | +tu102_fault_buffer_info(struct nvkm_fault_buffer *buffer) | |
| 61 | 61 | { |
| 62 | 62 | struct nvkm_device *device = buffer->fault->subdev.device; |
| 63 | 63 | const u32 foff = buffer->id * 0x20; |
| @@ -70,7 +70,7 @@ tu104_fault_buffer_info(struct nvkm_fault_buffer *buffer) · 6 unmodified lines | ||
| 70 | 70 | } |
| 71 | 71 | |
| 72 | 72 | static void |
| 73 | −tu104_fault_intr_fault(struct nvkm_fault *fault) | |
| 73 | +tu102_fault_intr_fault(struct nvkm_fault *fault) | |
| 74 | 74 | { |
| 75 | 75 | struct nvkm_subdev *subdev = &fault->subdev; |
| 76 | 76 | struct nvkm_device *device = subdev->device; |
| @@ -96,14 +96,14 @@ tu104_fault_intr_fault(struct nvkm_fault *fault) · 19 unmodified lines | ||
| 96 | 96 | } |
| 97 | 97 | |
| 98 | 98 | static void |
| 99 | −tu104_fault_intr(struct nvkm_fault *fault) | |
| 99 | +tu102_fault_intr(struct nvkm_fault *fault) | |
| 100 | 100 | { |
| 101 | 101 | struct nvkm_subdev *subdev = &fault->subdev; |
| 102 | 102 | struct nvkm_device *device = subdev->device; |
| 103 | 103 | u32 stat = nvkm_rd32(device, 0xb83094); |
| 104 | 104 | |
| 105 | 105 | if (stat & 0x80000000) { |
| 106 | − tu104_fault_intr_fault(fault); | |
| 106 | + tu102_fault_intr_fault(fault); | |
| 107 | 107 | nvkm_wr32(device, 0xb83094, 0x80000000); |
| 108 | 108 | stat &= ~0x80000000; |
| 109 | 109 | } |
| @@ -129,7 +129,7 @@ tu104_fault_intr(struct nvkm_fault *fault) · 19 unmodified lines | ||
| 129 | 129 | } |
| 130 | 130 | |
| 131 | 131 | static void |
| 132 | −tu104_fault_fini(struct nvkm_fault *fault) | |
| 132 | +tu102_fault_fini(struct nvkm_fault *fault) | |
| 133 | 133 | { |
| 134 | 134 | nvkm_notify_put(&fault->nrpfb); |
| 135 | 135 | if (fault->buffer[0]) |
| @@ -138,7 +138,7 @@ tu104_fault_fini(struct nvkm_fault *fault) · 2 unmodified lines | ||
| 138 | 138 | } |
| 139 | 139 | |
| 140 | 140 | static void |
| 141 | −tu104_fault_init(struct nvkm_fault *fault) | |
| 141 | +tu102_fault_init(struct nvkm_fault *fault) | |
| 142 | 142 | { |
| 143 | 143 | /*XXX: enable priv faults */ |
| 144 | 144 | fault->func->buffer.init(fault->buffer[0]); |
| @@ -146,22 +146,23 @@ tu104_fault_init(struct nvkm_fault *fault) · 1 unmodified line | ||
| 146 | 146 | } |
| 147 | 147 | |
| 148 | 148 | static const struct nvkm_fault_func |
| 149 | −tu104_fault = { | |
| 149 | +tu102_fault = { | |
| 150 | 150 | .oneinit = gv100_fault_oneinit, |
| 151 | − .init = tu104_fault_init, | |
| 152 | − .fini = tu104_fault_fini, | |
| 153 | − .intr = tu104_fault_intr, | |
| 151 | + .init = tu102_fault_init, | |
| 152 | + .fini = tu102_fault_fini, | |
| 153 | + .intr = tu102_fault_intr, | |
| 154 | 154 | .buffer.nr = 2, |
| 155 | 155 | .buffer.entry_size = 32, |
| 156 | − .buffer.info = tu104_fault_buffer_info, | |
| 157 | − .buffer.init = tu104_fault_buffer_init, | |
| 158 | − .buffer.fini = tu104_fault_buffer_fini, | |
| 159 | − .buffer.intr = tu104_fault_buffer_intr, | |
| 156 | + .buffer.info = tu102_fault_buffer_info, | |
| 157 | + .buffer.init = tu102_fault_buffer_init, | |
| 158 | + .buffer.fini = tu102_fault_buffer_fini, | |
| 159 | + .buffer.intr = tu102_fault_buffer_intr, | |
| 160 | + .user = { { 0, 0, VOLTA_FAULT_BUFFER_A }, 1 }, | |
| 160 | 161 | }; |
| 161 | 162 | |
| 162 | 163 | int |
| 163 | −tu104_fault_new(struct nvkm_device *device, int index, | |
| 164 | +tu102_fault_new(struct nvkm_device *device, int index, | |
| 164 | 165 | struct nvkm_fault **pfault) |
| 165 | 166 | { |
| 166 | − return nvkm_fault_new_(&tu104_fault, device, index, pfault); | |
| 167 | + return nvkm_fault_new_(&tu102_fault, device, index, pfault); | |
| 167 | 168 | } |
| @@ -0,0 +1,106 @@ | ||
| 1 | +/* | |
| 2 | + * Copyright 2018 Red Hat Inc. | |
| 3 | + * | |
| 4 | + * Permission is hereby granted, free of charge, to any person obtaining a | |
| 5 | + * copy of this software and associated documentation files (the "Software"), | |
| 6 | + * to deal in the Software without restriction, including without limitation | |
| 7 | + * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 8 | + * and/or sell copies of the Software, and to permit persons to whom the | |
| 9 | + * Software is furnished to do so, subject to the following conditions: | |
| 10 | + * | |
| 11 | + * The above copyright notice and this permission notice shall be included in | |
| 12 | + * all copies or substantial portions of the Software. | |
| 13 | + * | |
| 14 | + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 15 | + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 16 | + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 17 | + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 18 | + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 19 | + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 20 | + * OTHER DEALINGS IN THE SOFTWARE. | |
| 21 | + */ | |
| 22 | +#include "priv.h" | |
| 23 | + | |
| 24 | +#include <core/memory.h> | |
| 25 | +#include <subdev/mmu.h> | |
| 26 | + | |
| 27 | +#include <nvif/clb069.h> | |
| 28 | +#include <nvif/unpack.h> | |
| 29 | + | |
| 30 | +static int | |
| 31 | +nvkm_ufault_map(struct nvkm_object *object, void *argv, u32 argc, | |
| 32 | + enum nvkm_object_map *type, u64 *addr, u64 *size) | |
| 33 | +{ | |
| 34 | + struct nvkm_fault_buffer *buffer = nvkm_fault_buffer(object); | |
| 35 | + struct nvkm_device *device = buffer->fault->subdev.device; | |
| 36 | + *type = NVKM_OBJECT_MAP_IO; | |
| 37 | + *addr = device->func->resource_addr(device, 3) + buffer->addr; | |
| 38 | + *size = nvkm_memory_size(buffer->mem); | |
| 39 | + return 0; | |
| 40 | +} | |
| 41 | + | |
| 42 | +static int | |
| 43 | +nvkm_ufault_ntfy(struct nvkm_object *object, u32 type, | |
| 44 | + struct nvkm_event **pevent) | |
| 45 | +{ | |
| 46 | + struct nvkm_fault_buffer *buffer = nvkm_fault_buffer(object); | |
| 47 | + if (type == NVB069_V0_NTFY_FAULT) { | |
| 48 | + *pevent = &buffer->fault->event; | |
| 49 | + return 0; | |
| 50 | + } | |
| 51 | + return -EINVAL; | |
| 52 | +} | |
| 53 | + | |
| 54 | +static int | |
| 55 | +nvkm_ufault_fini(struct nvkm_object *object, bool suspend) | |
| 56 | +{ | |
| 57 | + struct nvkm_fault_buffer *buffer = nvkm_fault_buffer(object); | |
| 58 | + buffer->fault->func->buffer.fini(buffer); | |
| 59 | + return 0; | |
| 60 | +} | |
| 61 | + | |
| 62 | +static int | |
| 63 | +nvkm_ufault_init(struct nvkm_object *object) | |
| 64 | +{ | |
| 65 | + struct nvkm_fault_buffer *buffer = nvkm_fault_buffer(object); | |
| 66 | + buffer->fault->func->buffer.init(buffer); | |
| 67 | + return 0; | |
| 68 | +} | |
| 69 | + | |
| 70 | +static void * | |
| 71 | +nvkm_ufault_dtor(struct nvkm_object *object) | |
| 72 | +{ | |
| 73 | + return NULL; | |
| 74 | +} | |
| 75 | + | |
| 76 | +static const struct nvkm_object_func | |
| 77 | +nvkm_ufault = { | |
| 78 | + .dtor = nvkm_ufault_dtor, | |
| 79 | + .init = nvkm_ufault_init, | |
| 80 | + .fini = nvkm_ufault_fini, | |
| 81 | + .ntfy = nvkm_ufault_ntfy, | |
| 82 | + .map = nvkm_ufault_map, | |
| 83 | +}; | |
| 84 | + | |
| 85 | +int | |
| 86 | +nvkm_ufault_new(struct nvkm_device *device, const struct nvkm_oclass *oclass, | |
| 87 | + void *argv, u32 argc, struct nvkm_object **pobject) | |
| 88 | +{ | |
| 89 | + union { | |
| 90 | + struct nvif_clb069_v0 v0; | |
| 91 | + } *args = argv; | |
| 92 | + struct nvkm_fault *fault = device->fault; | |
| 93 | + struct nvkm_fault_buffer *buffer = fault->buffer[fault->func->user.rp]; | |
| 94 | + int ret = -ENOSYS; | |
| 95 | + | |
| 96 | + if (!(ret = nvif_unpack(ret, &argv, &argc, args->v0, 0, 0, false))) { | |
| 97 | + args->v0.entries = buffer->entries; | |
| 98 | + args->v0.get = buffer->get; | |
| 99 | + args->v0.put = buffer->put; | |
| 100 | + } else | |
| 101 | + return ret; | |
| 102 | + | |
| 103 | + nvkm_object_ctor(&nvkm_ufault, oclass, &buffer->object); | |
| 104 | + *pobject = &buffer->object; | |
| 105 | + return 0; | |
| 106 | +} | |
| @@ -87,7 +87,7 @@ nvkm_gddr3_calc(struct nvkm_ram *ram) · 86 unmodified lines | ||
| 87 | 87 | WR = (ram->next->bios.timing[2] & 0x007f0000) >> 16; |
| 88 | 88 | /* XXX: Get these values from the VBIOS instead */ |
| 89 | 89 | DLL = !(ram->mr[1] & 0x1); |
| 90 | − RON = !(ram->mr[1] & 0x300) >> 8; | |
| 90 | + RON = !((ram->mr[1] & 0x300) >> 8); | |
| 91 | 91 | break; |
| 92 | 92 | default: |
| 93 | 93 | return -ENOSYS; |
| @@ -0,0 +1 @@ | ||
| 1 | +nvkm-y += nvkm/subdev/gsp/gv100.o | |
| @@ -0,0 +1,62 @@ | ||
| 1 | +/* | |
| 2 | + * Copyright 2019 Red Hat Inc. | |
| 3 | + * | |
| 4 | + * Permission is hereby granted, free of charge, to any person obtaining a | |
| 5 | + * copy of this software and associated documentation files (the "Software"), | |
| 6 | + * to deal in the Software without restriction, including without limitation | |
| 7 | + * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 8 | + * and/or sell copies of the Software, and to permit persons to whom the | |
| 9 | + * Software is furnished to do so, subject to the following conditions: | |
| 10 | + * | |
| 11 | + * The above copyright notice and this permission notice shall be included in | |
| 12 | + * all copies or substantial portions of the Software. | |
| 13 | + * | |
| 14 | + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 15 | + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 16 | + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 17 | + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 18 | + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 19 | + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 20 | + * OTHER DEALINGS IN THE SOFTWARE. | |
| 21 | + */ | |
| 22 | +#include <subdev/gsp.h> | |
| 23 | +#include <subdev/top.h> | |
| 24 | +#include <engine/falcon.h> | |
| 25 | + | |
| 26 | +static int | |
| 27 | +gv100_gsp_oneinit(struct nvkm_subdev *subdev) | |
| 28 | +{ | |
| 29 | + struct nvkm_gsp *gsp = nvkm_gsp(subdev); | |
| 30 | + | |
| 31 | + gsp->addr = nvkm_top_addr(subdev->device, subdev->index); | |
| 32 | + if (!gsp->addr) | |
| 33 | + return -EINVAL; | |
| 34 | + | |
| 35 | + return nvkm_falcon_v1_new(subdev, "GSP", gsp->addr, &gsp->falcon); | |
| 36 | +} | |
| 37 | + | |
| 38 | +static void * | |
| 39 | +gv100_gsp_dtor(struct nvkm_subdev *subdev) | |
| 40 | +{ | |
| 41 | + struct nvkm_gsp *gsp = nvkm_gsp(subdev); | |
| 42 | + nvkm_falcon_del(&gsp->falcon); | |
| 43 | + return gsp; | |
| 44 | +} | |
| 45 | + | |
| 46 | +static const struct nvkm_subdev_func | |
| 47 | +gv100_gsp = { | |
| 48 | + .dtor = gv100_gsp_dtor, | |
| 49 | + .oneinit = gv100_gsp_oneinit, | |
| 50 | +}; | |
| 51 | + | |
| 52 | +int | |
| 53 | +gv100_gsp_new(struct nvkm_device *device, int index, struct nvkm_gsp **pgsp) | |
| 54 | +{ | |
| 55 | + struct nvkm_gsp *gsp; | |
| 56 | + | |
| 57 | + if (!(gsp = *pgsp = kzalloc(sizeof(*gsp), GFP_KERNEL))) | |
| 58 | + return -ENOMEM; | |
| 59 | + | |
| 60 | + nvkm_subdev_ctor(&gv100_gsp, device, index, &gsp->subdev); | |
| 61 | + return 0; | |
| 62 | +} | |
| @@ -12,4 +12,4 @@ nvkm-y += nvkm/subdev/mc/gk104.o · 11 unmodified lines | ||
| 12 | 12 | nvkm-y += nvkm/subdev/mc/gk20a.o |
| 13 | 13 | nvkm-y += nvkm/subdev/mc/gp100.o |
| 14 | 14 | nvkm-y += nvkm/subdev/mc/gp10b.o |
| 15 | −nvkm-y += nvkm/subdev/mc/tu104.o | |
| 15 | +nvkm-y += nvkm/subdev/mc/tu102.o | |
| @@ -22,7 +22,7 @@ · 21 unmodified lines | ||
| 22 | 22 | #include "priv.h" |
| 23 | 23 | |
| 24 | 24 | static void |
| 25 | −tu104_mc_intr_hack(struct nvkm_mc *mc, bool *handled) | |
| 25 | +tu102_mc_intr_hack(struct nvkm_mc *mc, bool *handled) | |
| 26 | 26 | { |
| 27 | 27 | struct nvkm_device *device = mc->subdev.device; |
| 28 | 28 | u32 stat = nvkm_rd32(device, 0xb81010); |
| @@ -37,19 +37,19 @@ tu104_mc_intr_hack(struct nvkm_mc *mc, bool *handled) · 8 unmodified lines | ||
| 37 | 37 | } |
| 38 | 38 | |
| 39 | 39 | static const struct nvkm_mc_func |
| 40 | −tu104_mc = { | |
| 40 | +tu102_mc = { | |
| 41 | 41 | .init = nv50_mc_init, |
| 42 | 42 | .intr = gp100_mc_intr, |
| 43 | 43 | .intr_unarm = gp100_mc_intr_unarm, |
| 44 | 44 | .intr_rearm = gp100_mc_intr_rearm, |
| 45 | 45 | .intr_mask = gp100_mc_intr_mask, |
| 46 | 46 | .intr_stat = gf100_mc_intr_stat, |
| 47 | − .intr_hack = tu104_mc_intr_hack, | |
| 47 | + .intr_hack = tu102_mc_intr_hack, | |
| 48 | 48 | .reset = gk104_mc_reset, |
| 49 | 49 | }; |
| 50 | 50 | |
| 51 | 51 | int |
| 52 | −tu104_mc_new(struct nvkm_device *device, int index, struct nvkm_mc **pmc) | |
| 52 | +tu102_mc_new(struct nvkm_device *device, int index, struct nvkm_mc **pmc) | |
| 53 | 53 | { |
| 54 | − return gp100_mc_new_(&tu104_mc, device, index, pmc); | |
| 54 | + return gp100_mc_new_(&tu102_mc, device, index, pmc); | |
| 55 | 55 | } |
| @@ -13,7 +13,7 @@ nvkm-y += nvkm/subdev/mmu/gm20b.o · 12 unmodified lines | ||
| 13 | 13 | nvkm-y += nvkm/subdev/mmu/gp100.o |
| 14 | 14 | nvkm-y += nvkm/subdev/mmu/gp10b.o |
| 15 | 15 | nvkm-y += nvkm/subdev/mmu/gv100.o |
| 16 | −nvkm-y += nvkm/subdev/mmu/tu104.o | |
| 16 | +nvkm-y += nvkm/subdev/mmu/tu102.o | |
| 17 | 17 | |
| 18 | 18 | nvkm-y += nvkm/subdev/mmu/mem.o |
| 19 | 19 | nvkm-y += nvkm/subdev/mmu/memnv04.o |
| @@ -34,7 +34,7 @@ nvkm-y += nvkm/subdev/mmu/vmmgm20b.o · 14 unmodified lines | ||
| 34 | 34 | nvkm-y += nvkm/subdev/mmu/vmmgp100.o |
| 35 | 35 | nvkm-y += nvkm/subdev/mmu/vmmgp10b.o |
| 36 | 36 | nvkm-y += nvkm/subdev/mmu/vmmgv100.o |
| 37 | −nvkm-y += nvkm/subdev/mmu/vmmtu104.o | |
| 37 | +nvkm-y += nvkm/subdev/mmu/vmmtu102.o | |
| 38 | 38 | |
| 39 | 39 | nvkm-y += nvkm/subdev/mmu/umem.o |
| 40 | 40 | nvkm-y += nvkm/subdev/mmu/ummu.o |
| @@ -31,7 +31,7 @@ gp100_mmu = { · 30 unmodified lines | ||
| 31 | 31 | .dma_bits = 47, |
| 32 | 32 | .mmu = {{ -1, -1, NVIF_CLASS_MMU_GF100}}, |
| 33 | 33 | .mem = {{ -1, 0, NVIF_CLASS_MEM_GF100}, gf100_mem_new, gf100_mem_map }, |
| 34 | − .vmm = {{ -1, -1, NVIF_CLASS_VMM_GP100}, gp100_vmm_new }, | |
| 34 | + .vmm = {{ -1, 0, NVIF_CLASS_VMM_GP100}, gp100_vmm_new }, | |
| 35 | 35 | .kind = gm200_mmu_kind, |
| 36 | 36 | .kind_sys = true, |
| 37 | 37 | }; |
| @@ -31,7 +31,7 @@ gp10b_mmu = { · 30 unmodified lines | ||
| 31 | 31 | .dma_bits = 47, |
| 32 | 32 | .mmu = {{ -1, -1, NVIF_CLASS_MMU_GF100}}, |
| 33 | 33 | .mem = {{ -1, -1, NVIF_CLASS_MEM_GF100}, .umap = gf100_mem_map }, |
| 34 | − .vmm = {{ -1, -1, NVIF_CLASS_VMM_GP100}, gp10b_vmm_new }, | |
| 34 | + .vmm = {{ -1, 0, NVIF_CLASS_VMM_GP100}, gp10b_vmm_new }, | |
| 35 | 35 | .kind = gm200_mmu_kind, |
| 36 | 36 | .kind_sys = true, |
| 37 | 37 | }; |
| @@ -31,7 +31,7 @@ gv100_mmu = { · 30 unmodified lines | ||
| 31 | 31 | .dma_bits = 47, |
| 32 | 32 | .mmu = {{ -1, -1, NVIF_CLASS_MMU_GF100}}, |
| 33 | 33 | .mem = {{ -1, 0, NVIF_CLASS_MEM_GF100}, gf100_mem_new, gf100_mem_map }, |
| 34 | − .vmm = {{ -1, -1, NVIF_CLASS_VMM_GP100}, gv100_vmm_new }, | |
| 34 | + .vmm = {{ -1, 0, NVIF_CLASS_VMM_GP100}, gv100_vmm_new }, | |
| 35 | 35 | .kind = gm200_mmu_kind, |
| 36 | 36 | .kind_sys = true, |
| 37 | 37 | }; |
| @@ -28,7 +28,7 @@ struct nvkm_mmu_func { · 27 unmodified lines | ||
| 28 | 28 | |
| 29 | 29 | struct { |
| 30 | 30 | struct nvkm_sclass user; |
| 31 | − int (*ctor)(struct nvkm_mmu *, u64 addr, u64 size, | |
| 31 | + int (*ctor)(struct nvkm_mmu *, bool managed, u64 addr, u64 size, | |
| 32 | 32 | void *argv, u32 argc, struct lock_class_key *, |
| 33 | 33 | const char *name, struct nvkm_vmm **); |
| 34 | 34 | bool global; |
| @@ -27,17 +27,17 @@ · 26 unmodified lines | ||
| 27 | 27 | #include <nvif/class.h> |
| 28 | 28 | |
| 29 | 29 | static const struct nvkm_mmu_func |
| 30 | −tu104_mmu = { | |
| 30 | +tu102_mmu = { | |
| 31 | 31 | .dma_bits = 47, |
| 32 | 32 | .mmu = {{ -1, -1, NVIF_CLASS_MMU_GF100}}, |
| 33 | 33 | .mem = {{ -1, 0, NVIF_CLASS_MEM_GF100}, gf100_mem_new, gf100_mem_map }, |
| 34 | − .vmm = {{ -1, 0, NVIF_CLASS_VMM_GP100}, tu104_vmm_new }, | |
| 34 | + .vmm = {{ -1, 0, NVIF_CLASS_VMM_GP100}, tu102_vmm_new }, | |
| 35 | 35 | .kind = gm200_mmu_kind, |
| 36 | 36 | .kind_sys = true, |
| 37 | 37 | }; |
| 38 | 38 | |
| 39 | 39 | int |
| 40 | −tu104_mmu_new(struct nvkm_device *device, int index, struct nvkm_mmu **pmmu) | |
| 40 | +tu102_mmu_new(struct nvkm_device *device, int index, struct nvkm_mmu **pmmu) | |
| 41 | 41 | { |
| 42 | − return nvkm_mmu_new_(&tu104_mmu, device, index, pmmu); | |
| 42 | + return nvkm_mmu_new_(&tu102_mmu, device, index, pmmu); | |
| 43 | 43 | } |
| @@ -42,6 +42,69 @@ nvkm_uvmm_search(struct nvkm_client *client, u64 handle) · 41 unmodified lines | ||
| 42 | 42 | return nvkm_uvmm(object)->vmm; |
| 43 | 43 | } |
| 44 | 44 | |
| 45 | +static int | |
| 46 | +nvkm_uvmm_mthd_pfnclr(struct nvkm_uvmm *uvmm, void *argv, u32 argc) | |
| 47 | +{ | |
| 48 | + struct nvkm_client *client = uvmm->object.client; | |
| 49 | + union { | |
| 50 | + struct nvif_vmm_pfnclr_v0 v0; | |
| 51 | + } *args = argv; | |
| 52 | + struct nvkm_vmm *vmm = uvmm->vmm; | |
| 53 | + int ret = -ENOSYS; | |
| 54 | + u64 addr, size; | |
| 55 | + | |
| 56 | + if (!(ret = nvif_unpack(ret, &argv, &argc, args->v0, 0, 0, false))) { | |
| 57 | + addr = args->v0.addr; | |
| 58 | + size = args->v0.size; | |
| 59 | + } else | |
| 60 | + return ret; | |
| 61 | + | |
| 62 | + if (!client->super) | |
| 63 | + return -ENOENT; | |
| 64 | + | |
| 65 | + if (size) { | |
| 66 | + mutex_lock(&vmm->mutex); | |
| 67 | + ret = nvkm_vmm_pfn_unmap(vmm, addr, size); | |
| 68 | + mutex_unlock(&vmm->mutex); | |
| 69 | + } | |
| 70 | + | |
| 71 | + return ret; | |
| 72 | +} | |
| 73 | + | |
| 74 | +static int | |
| 75 | +nvkm_uvmm_mthd_pfnmap(struct nvkm_uvmm *uvmm, void *argv, u32 argc) | |
| 76 | +{ | |
| 77 | + struct nvkm_client *client = uvmm->object.client; | |
| 78 | + union { | |
| 79 | + struct nvif_vmm_pfnmap_v0 v0; | |
| 80 | + } *args = argv; | |
| 81 | + struct nvkm_vmm *vmm = uvmm->vmm; | |
| 82 | + int ret = -ENOSYS; | |
| 83 | + u64 addr, size, *phys; | |
| 84 | + u8 page; | |
| 85 | + | |
| 86 | + if (!(ret = nvif_unpack(ret, &argv, &argc, args->v0, 0, 0, true))) { | |
| 87 | + page = args->v0.page; | |
| 88 | + addr = args->v0.addr; | |
| 89 | + size = args->v0.size; | |
| 90 | + phys = args->v0.phys; | |
| 91 | + if (argc != (size >> page) * sizeof(args->v0.phys[0])) | |
| 92 | + return -EINVAL; | |
| 93 | + } else | |
| 94 | + return ret; | |
| 95 | + | |
| 96 | + if (!client->super) | |
| 97 | + return -ENOENT; | |
| 98 | + | |
| 99 | + if (size) { | |
| 100 | + mutex_lock(&vmm->mutex); | |
| 101 | + ret = nvkm_vmm_pfn_map(vmm, page, addr, size, phys); | |
| 102 | + mutex_unlock(&vmm->mutex); | |
| 103 | + } | |
| 104 | + | |
| 105 | + return ret; | |
| 106 | +} | |
| 107 | + | |
| 45 | 108 | static int |
| 46 | 109 | nvkm_uvmm_mthd_unmap(struct nvkm_uvmm *uvmm, void *argv, u32 argc) |
| 47 | 110 | { |
| @@ -78,7 +141,7 @@ nvkm_uvmm_mthd_unmap(struct nvkm_uvmm *uvmm, void *argv, u32 argc) · 30 unmodified lines | ||
| 78 | 141 | goto done; |
| 79 | 142 | } |
| 80 | 143 | |
| 81 | − nvkm_vmm_unmap_locked(vmm, vma); | |
| 144 | + nvkm_vmm_unmap_locked(vmm, vma, false); | |
| 82 | 145 | ret = 0; |
| 83 | 146 | done: |
| 84 | 147 | mutex_unlock(&vmm->mutex); |
| @@ -124,6 +187,11 @@ nvkm_uvmm_mthd_map(struct nvkm_uvmm *uvmm, void *argv, u32 argc) · 39 unmodified lines | ||
| 124 | 187 | goto fail; |
| 125 | 188 | } |
| 126 | 189 | |
| 190 | + if (ret = -EINVAL, vma->mapped && !vma->memory) { | |
| 191 | + VMM_DEBUG(vmm, "pfnmap %016llx", addr); | |
| 192 | + goto fail; | |
| 193 | + } | |
| 194 | + | |
| 127 | 195 | if (ret = -EINVAL, vma->addr != addr || vma->size != size) { |
| 128 | 196 | if (addr + size > vma->addr + vma->size || vma->memory || |
| 129 | 197 | (vma->refd == NVKM_VMA_PAGE_NONE && !vma->mapref)) { |
| @@ -271,6 +339,15 @@ nvkm_uvmm_mthd(struct nvkm_object *object, u32 mthd, void *argv, u32 argc) · 141 unmodified lines | ||
| 271 | 339 | case NVIF_VMM_V0_PUT : return nvkm_uvmm_mthd_put (uvmm, argv, argc); |
| 272 | 340 | case NVIF_VMM_V0_MAP : return nvkm_uvmm_mthd_map (uvmm, argv, argc); |
| 273 | 341 | case NVIF_VMM_V0_UNMAP : return nvkm_uvmm_mthd_unmap (uvmm, argv, argc); |
| 342 | + case NVIF_VMM_V0_PFNMAP: return nvkm_uvmm_mthd_pfnmap(uvmm, argv, argc); | |
| 343 | + case NVIF_VMM_V0_PFNCLR: return nvkm_uvmm_mthd_pfnclr(uvmm, argv, argc); | |
| 344 | + case NVIF_VMM_V0_MTHD(0x00) ... NVIF_VMM_V0_MTHD(0x7f): | |
| 345 | + if (uvmm->vmm->func->mthd) { | |
| 346 | + return uvmm->vmm->func->mthd(uvmm->vmm, | |
| 347 | + uvmm->object.client, | |
| 348 | + mthd, argv, argc); | |
| 349 | + } | |
| 350 | + break; | |
| 274 | 351 | default: |
| 275 | 352 | break; |
| 276 | 353 | } |
| @@ -304,8 +381,10 @@ nvkm_uvmm_new(const struct nvkm_oclass *oclass, void *argv, u32 argc, · 27 unmodified lines | ||
| 304 | 381 | struct nvkm_uvmm *uvmm; |
| 305 | 382 | int ret = -ENOSYS; |
| 306 | 383 | u64 addr, size; |
| 384 | + bool managed; | |
| 307 | 385 | |
| 308 | 386 | if (!(ret = nvif_unpack(ret, &argv, &argc, args->v0, 0, 0, more))) { |
| 387 | + managed = args->v0.managed != 0; | |
| 309 | 388 | addr = args->v0.addr; |
| 310 | 389 | size = args->v0.size; |
| 311 | 390 | } else |
| @@ -317,7 +396,7 @@ nvkm_uvmm_new(const struct nvkm_oclass *oclass, void *argv, u32 argc, · 5 unmodified lines | ||
| 317 | 396 | *pobject = &uvmm->object; |
| 318 | 397 | |
| 319 | 398 | if (!mmu->vmm) { |
| 320 | − ret = mmu->func->vmm.ctor(mmu, addr, size, argv, argc, | |
| 399 | + ret = mmu->func->vmm.ctor(mmu, managed, addr, size, argv, argc, | |
| 321 | 400 | NULL, "user", &uvmm->vmm); |
| 322 | 401 | if (ret) |
| 323 | 402 | return ret; |
| @@ -255,11 +255,23 @@ nvkm_vmm_unref_sptes(struct nvkm_vmm_iter *it, struct nvkm_vmm_pt *pgt, · 254 unmodified lines | ||
| 255 | 255 | } |
| 256 | 256 | |
| 257 | 257 | static bool |
| 258 | −nvkm_vmm_unref_ptes(struct nvkm_vmm_iter *it, u32 ptei, u32 ptes) | |
| 258 | +nvkm_vmm_unref_ptes(struct nvkm_vmm_iter *it, bool pfn, u32 ptei, u32 ptes) | |
| 259 | 259 | { |
| 260 | 260 | const struct nvkm_vmm_desc *desc = it->desc; |
| 261 | 261 | const int type = desc->type == SPT; |
| 262 | 262 | struct nvkm_vmm_pt *pgt = it->pt[0]; |
| 263 | + bool dma; | |
| 264 | + | |
| 265 | + if (pfn) { | |
| 266 | + /* Need to clear PTE valid bits before we dma_unmap_page(). */ | |
| 267 | + dma = desc->func->pfn_clear(it->vmm, pgt->pt[type], ptei, ptes); | |
| 268 | + if (dma) { | |
| 269 | + /* GPU may have cached the PT, flush before unmap. */ | |
| 270 | + nvkm_vmm_flush_mark(it); | |
| 271 | + nvkm_vmm_flush(it); | |
| 272 | + desc->func->pfn_unmap(it->vmm, pgt->pt[type], ptei, ptes); | |
| 273 | + } | |
| 274 | + } | |
| 263 | 275 | |
| 264 | 276 | /* Drop PTE references. */ |
| 265 | 277 | pgt->refs[type] -= ptes; |
| @@ -349,7 +361,7 @@ nvkm_vmm_ref_sptes(struct nvkm_vmm_iter *it, struct nvkm_vmm_pt *pgt, · 83 unmodified lines | ||
| 349 | 361 | } |
| 350 | 362 | |
| 351 | 363 | static bool |
| 352 | −nvkm_vmm_ref_ptes(struct nvkm_vmm_iter *it, u32 ptei, u32 ptes) | |
| 364 | +nvkm_vmm_ref_ptes(struct nvkm_vmm_iter *it, bool pfn, u32 ptei, u32 ptes) | |
| 353 | 365 | { |
| 354 | 366 | const struct nvkm_vmm_desc *desc = it->desc; |
| 355 | 367 | const int type = desc->type == SPT; |
| @@ -379,7 +391,7 @@ nvkm_vmm_sparse_ptes(const struct nvkm_vmm_desc *desc, · 23 unmodified lines | ||
| 379 | 391 | } |
| 380 | 392 | |
| 381 | 393 | static bool |
| 382 | −nvkm_vmm_sparse_unref_ptes(struct nvkm_vmm_iter *it, u32 ptei, u32 ptes) | |
| 394 | +nvkm_vmm_sparse_unref_ptes(struct nvkm_vmm_iter *it, bool pfn, u32 ptei, u32 ptes) | |
| 383 | 395 | { |
| 384 | 396 | struct nvkm_vmm_pt *pt = it->pt[0]; |
| 385 | 397 | if (it->desc->type == PGD) |
| @@ -387,14 +399,14 @@ nvkm_vmm_sparse_unref_ptes(struct nvkm_vmm_iter *it, u32 ptei, u32 ptes) · 1 unmodified line | ||
| 387 | 399 | else |
| 388 | 400 | if (it->desc->type == LPT) |
| 389 | 401 | memset(&pt->pte[ptei], 0x00, sizeof(pt->pte[0]) * ptes); |
| 390 | − return nvkm_vmm_unref_ptes(it, ptei, ptes); | |
| 402 | + return nvkm_vmm_unref_ptes(it, pfn, ptei, ptes); | |
| 391 | 403 | } |
| 392 | 404 | |
| 393 | 405 | static bool |
| 394 | −nvkm_vmm_sparse_ref_ptes(struct nvkm_vmm_iter *it, u32 ptei, u32 ptes) | |
| 406 | +nvkm_vmm_sparse_ref_ptes(struct nvkm_vmm_iter *it, bool pfn, u32 ptei, u32 ptes) | |
| 395 | 407 | { |
| 396 | 408 | nvkm_vmm_sparse_ptes(it->desc, it->pt[0], ptei, ptes); |
| 397 | − return nvkm_vmm_ref_ptes(it, ptei, ptes); | |
| 409 | + return nvkm_vmm_ref_ptes(it, pfn, ptei, ptes); | |
| 398 | 410 | } |
| 399 | 411 | |
| 400 | 412 | static bool |
| @@ -487,8 +499,8 @@ nvkm_vmm_ref_swpt(struct nvkm_vmm_iter *it, struct nvkm_vmm_pt *pgd, u32 pdei) · 86 unmodified lines | ||
| 487 | 499 | |
| 488 | 500 | static inline u64 |
| 489 | 501 | nvkm_vmm_iter(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, |
| 490 | − u64 addr, u64 size, const char *name, bool ref, | |
| 491 | − bool (*REF_PTES)(struct nvkm_vmm_iter *, u32, u32), | |
| 502 | + u64 addr, u64 size, const char *name, bool ref, bool pfn, | |
| 503 | + bool (*REF_PTES)(struct nvkm_vmm_iter *, bool pfn, u32, u32), | |
| 492 | 504 | nvkm_vmm_pte_func MAP_PTES, struct nvkm_vmm_map *map, |
| 493 | 505 | nvkm_vmm_pxe_func CLR_PTES) |
| 494 | 506 | { |
| @@ -548,7 +560,7 @@ nvkm_vmm_iter(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, · 53 unmodified lines | ||
| 548 | 560 | } |
| 549 | 561 | |
| 550 | 562 | /* Handle PTE updates. */ |
| 551 | − if (!REF_PTES || REF_PTES(&it, ptei, ptes)) { | |
| 563 | + if (!REF_PTES || REF_PTES(&it, pfn, ptei, ptes)) { | |
| 552 | 564 | struct nvkm_mmu_pt *pt = pgt->pt[type]; |
| 553 | 565 | if (MAP_PTES || CLR_PTES) { |
| 554 | 566 | if (MAP_PTES) |
| @@ -590,7 +602,7 @@ static void · 35 unmodified lines | ||
| 590 | 602 | nvkm_vmm_ptes_sparse_put(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, |
| 591 | 603 | u64 addr, u64 size) |
| 592 | 604 | { |
| 593 | − nvkm_vmm_iter(vmm, page, addr, size, "sparse unref", false, | |
| 605 | + nvkm_vmm_iter(vmm, page, addr, size, "sparse unref", false, false, | |
| 594 | 606 | nvkm_vmm_sparse_unref_ptes, NULL, NULL, |
| 595 | 607 | page->desc->func->invalid ? |
| 596 | 608 | page->desc->func->invalid : page->desc->func->unmap); |
| @@ -602,8 +614,8 @@ nvkm_vmm_ptes_sparse_get(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, · 5 unmodified lines | ||
| 602 | 614 | { |
| 603 | 615 | if ((page->type & NVKM_VMM_PAGE_SPARSE)) { |
| 604 | 616 | u64 fail = nvkm_vmm_iter(vmm, page, addr, size, "sparse ref", |
| 605 | − true, nvkm_vmm_sparse_ref_ptes, NULL, | |
| 606 | − NULL, page->desc->func->sparse); | |
| 617 | + true, false, nvkm_vmm_sparse_ref_ptes, | |
| 618 | + NULL, NULL, page->desc->func->sparse); | |
| 607 | 619 | if (fail != ~0ULL) { |
| 608 | 620 | if ((size = fail - addr)) |
| 609 | 621 | nvkm_vmm_ptes_sparse_put(vmm, page, addr, size); |
| @@ -666,11 +678,11 @@ nvkm_vmm_ptes_sparse(struct nvkm_vmm *vmm, u64 addr, u64 size, bool ref) · 56 unmodified lines | ||
| 666 | 678 | |
| 667 | 679 | static void |
| 668 | 680 | nvkm_vmm_ptes_unmap_put(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, |
| 669 | − u64 addr, u64 size, bool sparse) | |
| 681 | + u64 addr, u64 size, bool sparse, bool pfn) | |
| 670 | 682 | { |
| 671 | 683 | const struct nvkm_vmm_desc_func *func = page->desc->func; |
| 672 | 684 | nvkm_vmm_iter(vmm, page, addr, size, "unmap + unref", |
| 673 | − false, nvkm_vmm_unref_ptes, NULL, NULL, | |
| 685 | + false, pfn, nvkm_vmm_unref_ptes, NULL, NULL, | |
| 674 | 686 | sparse ? func->sparse : func->invalid ? func->invalid : |
| 675 | 687 | func->unmap); |
| 676 | 688 | } |
| @@ -681,10 +693,10 @@ nvkm_vmm_ptes_get_map(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, · 4 unmodified lines | ||
| 681 | 693 | nvkm_vmm_pte_func func) |
| 682 | 694 | { |
| 683 | 695 | u64 fail = nvkm_vmm_iter(vmm, page, addr, size, "ref + map", true, |
| 684 | − nvkm_vmm_ref_ptes, func, map, NULL); | |
| 696 | + false, nvkm_vmm_ref_ptes, func, map, NULL); | |
| 685 | 697 | if (fail != ~0ULL) { |
| 686 | 698 | if ((size = fail - addr)) |
| 687 | − nvkm_vmm_ptes_unmap_put(vmm, page, addr, size, false); | |
| 699 | + nvkm_vmm_ptes_unmap_put(vmm, page, addr, size, false, false); | |
| 688 | 700 | return -ENOMEM; |
| 689 | 701 | } |
| 690 | 702 | return 0; |
| @@ -692,10 +704,11 @@ nvkm_vmm_ptes_get_map(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, · 1 unmodified line | ||
| 692 | 704 | |
| 693 | 705 | static void |
| 694 | 706 | nvkm_vmm_ptes_unmap(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, |
| 695 | − u64 addr, u64 size, bool sparse) | |
| 707 | + u64 addr, u64 size, bool sparse, bool pfn) | |
| 696 | 708 | { |
| 697 | 709 | const struct nvkm_vmm_desc_func *func = page->desc->func; |
| 698 | − nvkm_vmm_iter(vmm, page, addr, size, "unmap", false, NULL, NULL, NULL, | |
| 710 | + nvkm_vmm_iter(vmm, page, addr, size, "unmap", false, pfn, | |
| 711 | + NULL, NULL, NULL, | |
| 699 | 712 | sparse ? func->sparse : func->invalid ? func->invalid : |
| 700 | 713 | func->unmap); |
| 701 | 714 | } |
| @@ -705,7 +718,7 @@ nvkm_vmm_ptes_map(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, · 3 unmodified lines | ||
| 705 | 718 | u64 addr, u64 size, struct nvkm_vmm_map *map, |
| 706 | 719 | nvkm_vmm_pte_func func) |
| 707 | 720 | { |
| 708 | − nvkm_vmm_iter(vmm, page, addr, size, "map", false, | |
| 721 | + nvkm_vmm_iter(vmm, page, addr, size, "map", false, false, | |
| 709 | 722 | NULL, func, map, NULL); |
| 710 | 723 | } |
| 711 | 724 | |
| @@ -713,7 +726,7 @@ static void · 1 unmodified line | ||
| 713 | 726 | nvkm_vmm_ptes_put(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, |
| 714 | 727 | u64 addr, u64 size) |
| 715 | 728 | { |
| 716 | − nvkm_vmm_iter(vmm, page, addr, size, "unref", false, | |
| 729 | + nvkm_vmm_iter(vmm, page, addr, size, "unref", false, false, | |
| 717 | 730 | nvkm_vmm_unref_ptes, NULL, NULL, NULL); |
| 718 | 731 | } |
| 719 | 732 | |
| @@ -721,7 +734,7 @@ static int · 1 unmodified line | ||
| 721 | 734 | nvkm_vmm_ptes_get(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, |
| 722 | 735 | u64 addr, u64 size) |
| 723 | 736 | { |
| 724 | − u64 fail = nvkm_vmm_iter(vmm, page, addr, size, "ref", true, | |
| 737 | + u64 fail = nvkm_vmm_iter(vmm, page, addr, size, "ref", true, false, | |
| 725 | 738 | nvkm_vmm_ref_ptes, NULL, NULL, NULL); |
| 726 | 739 | if (fail != ~0ULL) { |
| 727 | 740 | if (fail != addr) |
| @@ -763,6 +776,7 @@ nvkm_vma_tail(struct nvkm_vma *vma, u64 tail) · 35 unmodified lines | ||
| 763 | 776 | new->part = vma->part; |
| 764 | 777 | new->user = vma->user; |
| 765 | 778 | new->busy = vma->busy; |
| 779 | + new->mapped = vma->mapped; | |
| 766 | 780 | list_add(&new->head, &vma->head); |
| 767 | 781 | return new; |
| 768 | 782 | } |
| @@ -934,12 +948,41 @@ nvkm_vmm_node_split(struct nvkm_vmm *vmm, · 165 unmodified lines | ||
| 934 | 948 | return vma; |
| 935 | 949 | } |
| 936 | 950 | |
| 951 | +static void | |
| 952 | +nvkm_vma_dump(struct nvkm_vma *vma) | |
| 953 | +{ | |
| 954 | + printk(KERN_ERR "%016llx %016llx %c%c%c%c%c%c%c%c%c %p\n", | |
| 955 | + vma->addr, (u64)vma->size, | |
| 956 | + vma->used ? '-' : 'F', | |
| 957 | + vma->mapref ? 'R' : '-', | |
| 958 | + vma->sparse ? 'S' : '-', | |
| 959 | + vma->page != NVKM_VMA_PAGE_NONE ? '0' + vma->page : '-', | |
| 960 | + vma->refd != NVKM_VMA_PAGE_NONE ? '0' + vma->refd : '-', | |
| 961 | + vma->part ? 'P' : '-', | |
| 962 | + vma->user ? 'U' : '-', | |
| 963 | + vma->busy ? 'B' : '-', | |
| 964 | + vma->mapped ? 'M' : '-', | |
| 965 | + vma->memory); | |
| 966 | +} | |
| 967 | + | |
| 968 | +static void | |
| 969 | +nvkm_vmm_dump(struct nvkm_vmm *vmm) | |
| 970 | +{ | |
| 971 | + struct nvkm_vma *vma; | |
| 972 | + list_for_each_entry(vma, &vmm->list, head) { | |
| 973 | + nvkm_vma_dump(vma); | |
| 974 | + } | |
| 975 | +} | |
| 976 | + | |
| 937 | 977 | static void |
| 938 | 978 | nvkm_vmm_dtor(struct nvkm_vmm *vmm) |
| 939 | 979 | { |
| 940 | 980 | struct nvkm_vma *vma; |
| 941 | 981 | struct rb_node *node; |
| 942 | 982 | |
| 983 | + if (0) | |
| 984 | + nvkm_vmm_dump(vmm); | |
| 985 | + | |
| 943 | 986 | while ((node = rb_first(&vmm->root))) { |
| 944 | 987 | struct nvkm_vma *vma = rb_entry(node, typeof(*vma), tree); |
| 945 | 988 | nvkm_vmm_put(vmm, &vma); |
| @@ -972,16 +1015,32 @@ nvkm_vmm_dtor(struct nvkm_vmm *vmm) · 26 unmodified lines | ||
| 972 | 1015 | } |
| 973 | 1016 | } |
| 974 | 1017 | |
| 1018 | +static int | |
| 1019 | +nvkm_vmm_ctor_managed(struct nvkm_vmm *vmm, u64 addr, u64 size) | |
| 1020 | +{ | |
| 1021 | + struct nvkm_vma *vma; | |
| 1022 | + if (!(vma = nvkm_vma_new(addr, size))) | |
| 1023 | + return -ENOMEM; | |
| 1024 | + vma->mapref = true; | |
| 1025 | + vma->sparse = false; | |
| 1026 | + vma->used = true; | |
| 1027 | + vma->user = true; | |
| 1028 | + nvkm_vmm_node_insert(vmm, vma); | |
| 1029 | + list_add_tail(&vma->head, &vmm->list); | |
| 1030 | + return 0; | |
| 1031 | +} | |
| 1032 | + | |
| 975 | 1033 | int |
| 976 | 1034 | nvkm_vmm_ctor(const struct nvkm_vmm_func *func, struct nvkm_mmu *mmu, |
| 977 | − u32 pd_header, u64 addr, u64 size, struct lock_class_key *key, | |
| 978 | − const char *name, struct nvkm_vmm *vmm) | |
| 1035 | + u32 pd_header, bool managed, u64 addr, u64 size, | |
| 1036 | + struct lock_class_key *key, const char *name, | |
| 1037 | + struct nvkm_vmm *vmm) | |
| 979 | 1038 | { |
| 980 | 1039 | static struct lock_class_key _key; |
| 981 | 1040 | const struct nvkm_vmm_page *page = func->page; |
| 982 | 1041 | const struct nvkm_vmm_desc *desc; |
| 983 | 1042 | struct nvkm_vma *vma; |
| 984 | − int levels, bits = 0; | |
| 1043 | + int levels, bits = 0, ret; | |
| 985 | 1044 | |
| 986 | 1045 | vmm->func = func; |
| 987 | 1046 | vmm->mmu = mmu; |
| @@ -1009,11 +1068,6 @@ nvkm_vmm_ctor(const struct nvkm_vmm_func *func, struct nvkm_mmu *mmu, · 21 unmodified lines | ||
| 1009 | 1068 | if (WARN_ON(levels > NVKM_VMM_LEVELS_MAX)) |
| 1010 | 1069 | return -EINVAL; |
| 1011 | 1070 | |
| 1012 | − vmm->start = addr; | |
| 1013 | − vmm->limit = size ? (addr + size) : (1ULL << bits); | |
| 1014 | − if (vmm->start > vmm->limit || vmm->limit > (1ULL << bits)) | |
| 1015 | − return -EINVAL; | |
| 1016 | − | |
| 1017 | 1071 | /* Allocate top-level page table. */ |
| 1018 | 1072 | vmm->pd = nvkm_vmm_pt_new(desc, false, NULL); |
| 1019 | 1073 | if (!vmm->pd) |
| @@ -1036,50 +1090,273 @@ nvkm_vmm_ctor(const struct nvkm_vmm_func *func, struct nvkm_mmu *mmu, · 16 unmodified lines | ||
| 1036 | 1090 | vmm->free = RB_ROOT; |
| 1037 | 1091 | vmm->root = RB_ROOT; |
| 1038 | 1092 | |
| 1039 | − if (!(vma = nvkm_vma_new(vmm->start, vmm->limit - vmm->start))) | |
| 1040 | − return -ENOMEM; | |
| 1093 | + if (managed) { | |
| 1094 | + /* Address-space will be managed by the client for the most | |
| 1095 | + * part, except for a specified area where NVKM allocations | |
| 1096 | + * are allowed to be placed. | |
| 1097 | + */ | |
| 1098 | + vmm->start = 0; | |
| 1099 | + vmm->limit = 1ULL << bits; | |
| 1100 | + if (addr + size < addr || addr + size > vmm->limit) | |
| 1101 | + return -EINVAL; | |
| 1102 | + | |
| 1103 | + /* Client-managed area before the NVKM-managed area. */ | |
| 1104 | + if (addr && (ret = nvkm_vmm_ctor_managed(vmm, 0, addr))) | |
| 1105 | + return ret; | |
| 1106 | + | |
| 1107 | + /* NVKM-managed area. */ | |
| 1108 | + if (size) { | |
| 1109 | + if (!(vma = nvkm_vma_new(addr, size))) | |
| 1110 | + return -ENOMEM; | |
| 1111 | + nvkm_vmm_free_insert(vmm, vma); | |
| 1112 | + list_add_tail(&vma->head, &vmm->list); | |
| 1113 | + } | |
| 1114 | + | |
| 1115 | + /* Client-managed area after the NVKM-managed area. */ | |
| 1116 | + addr = addr + size; | |
| 1117 | + size = vmm->limit - addr; | |
| 1118 | + if (size && (ret = nvkm_vmm_ctor_managed(vmm, addr, size))) | |
| 1119 | + return ret; | |
| 1120 | + } else { | |
| 1121 | + /* Address-space fully managed by NVKM, requiring calls to | |
| 1122 | + * nvkm_vmm_get()/nvkm_vmm_put() to allocate address-space. | |
| 1123 | + */ | |
| 1124 | + vmm->start = addr; | |
| 1125 | + vmm->limit = size ? (addr + size) : (1ULL << bits); | |
| 1126 | + if (vmm->start > vmm->limit || vmm->limit > (1ULL << bits)) | |
| 1127 | + return -EINVAL; | |
| 1128 | + | |
| 1129 | + if (!(vma = nvkm_vma_new(vmm->start, vmm->limit - vmm->start))) | |
| 1130 | + return -ENOMEM; | |
| 1131 | + | |
| 1132 | + nvkm_vmm_free_insert(vmm, vma); | |
| 1133 | + list_add(&vma->head, &vmm->list); | |
| 1134 | + } | |
| 1041 | 1135 | |
| 1042 | − nvkm_vmm_free_insert(vmm, vma); | |
| 1043 | − list_add(&vma->head, &vmm->list); | |
| 1044 | 1136 | return 0; |
| 1045 | 1137 | } |
| 1046 | 1138 | |
| 1047 | 1139 | int |
| 1048 | 1140 | nvkm_vmm_new_(const struct nvkm_vmm_func *func, struct nvkm_mmu *mmu, |
| 1049 | − u32 hdr, u64 addr, u64 size, struct lock_class_key *key, | |
| 1050 | − const char *name, struct nvkm_vmm **pvmm) | |
| 1141 | + u32 hdr, bool managed, u64 addr, u64 size, | |
| 1142 | + struct lock_class_key *key, const char *name, | |
| 1143 | + struct nvkm_vmm **pvmm) | |
| 1051 | 1144 | { |
| 1052 | 1145 | if (!(*pvmm = kzalloc(sizeof(**pvmm), GFP_KERNEL))) |
| 1053 | 1146 | return -ENOMEM; |
| 1054 | − return nvkm_vmm_ctor(func, mmu, hdr, addr, size, key, name, *pvmm); | |
| 1147 | + return nvkm_vmm_ctor(func, mmu, hdr, managed, addr, size, key, name, *pvmm); | |
| 1148 | +} | |
| 1149 | + | |
| 1150 | +static struct nvkm_vma * | |
| 1151 | +nvkm_vmm_pfn_split_merge(struct nvkm_vmm *vmm, struct nvkm_vma *vma, | |
| 1152 | + u64 addr, u64 size, u8 page, bool map) | |
| 1153 | +{ | |
| 1154 | + struct nvkm_vma *prev = NULL; | |
| 1155 | + struct nvkm_vma *next = NULL; | |
| 1156 | + | |
| 1157 | + if (vma->addr == addr && vma->part && (prev = node(vma, prev))) { | |
| 1158 | + if (prev->memory || prev->mapped != map) | |
| 1159 | + prev = NULL; | |
| 1160 | + } | |
| 1161 | + | |
| 1162 | + if (vma->addr + vma->size == addr + size && (next = node(vma, next))) { | |
| 1163 | + if (!next->part || | |
| 1164 | + next->memory || next->mapped != map) | |
| 1165 | + next = NULL; | |
| 1166 | + } | |
| 1167 | + | |
| 1168 | + if (prev || next) | |
| 1169 | + return nvkm_vmm_node_merge(vmm, prev, vma, next, size); | |
| 1170 | + return nvkm_vmm_node_split(vmm, vma, addr, size); | |
| 1171 | +} | |
| 1172 | + | |
| 1173 | +int | |
| 1174 | +nvkm_vmm_pfn_unmap(struct nvkm_vmm *vmm, u64 addr, u64 size) | |
| 1175 | +{ | |
| 1176 | + struct nvkm_vma *vma = nvkm_vmm_node_search(vmm, addr); | |
| 1177 | + struct nvkm_vma *next; | |
| 1178 | + u64 limit = addr + size; | |
| 1179 | + u64 start = addr; | |
| 1180 | + | |
| 1181 | + if (!vma) | |
| 1182 | + return -EINVAL; | |
| 1183 | + | |
| 1184 | + do { | |
| 1185 | + if (!vma->mapped || vma->memory) | |
| 1186 | + continue; | |
| 1187 | + | |
| 1188 | + size = min(limit - start, vma->size - (start - vma->addr)); | |
| 1189 | + | |
| 1190 | + nvkm_vmm_ptes_unmap_put(vmm, &vmm->func->page[vma->refd], | |
| 1191 | + start, size, false, true); | |
| 1192 | + | |
| 1193 | + next = nvkm_vmm_pfn_split_merge(vmm, vma, start, size, 0, false); | |
| 1194 | + if (!WARN_ON(!next)) { | |
| 1195 | + vma = next; | |
| 1196 | + vma->refd = NVKM_VMA_PAGE_NONE; | |
| 1197 | + vma->mapped = false; | |
| 1198 | + } | |
| 1199 | + } while ((vma = node(vma, next)) && (start = vma->addr) < limit); | |
| 1200 | + | |
| 1201 | + return 0; | |
| 1202 | +} | |
| 1203 | + | |
| 1204 | +/*TODO: | |
| 1205 | + * - Avoid PT readback (for dma_unmap etc), this might end up being dealt | |
| 1206 | + * with inside HMM, which would be a lot nicer for us to deal with. | |
| 1207 | + * - Multiple page sizes (particularly for huge page support). | |
| 1208 | + * - Support for systems without a 4KiB page size. | |
| 1209 | + */ | |
| 1210 | +int | |
| 1211 | +nvkm_vmm_pfn_map(struct nvkm_vmm *vmm, u8 shift, u64 addr, u64 size, u64 *pfn) | |
| 1212 | +{ | |
| 1213 | + const struct nvkm_vmm_page *page = vmm->func->page; | |
| 1214 | + struct nvkm_vma *vma, *tmp; | |
| 1215 | + u64 limit = addr + size; | |
| 1216 | + u64 start = addr; | |
| 1217 | + int pm = size >> shift; | |
| 1218 | + int pi = 0; | |
| 1219 | + | |
| 1220 | + /* Only support mapping where the page size of the incoming page | |
| 1221 | + * array matches a page size available for direct mapping. | |
| 1222 | + */ | |
| 1223 | + while (page->shift && page->shift != shift && | |
| 1224 | + page->desc->func->pfn == NULL) | |
| 1225 | + page++; | |
| 1226 | + | |
| 1227 | + if (!page->shift || !IS_ALIGNED(addr, 1ULL << shift) || | |
| 1228 | + !IS_ALIGNED(size, 1ULL << shift) || | |
| 1229 | + addr + size < addr || addr + size > vmm->limit) { | |
| 1230 | + VMM_DEBUG(vmm, "paged map %d %d %016llx %016llx\n", | |
| 1231 | + shift, page->shift, addr, size); | |
| 1232 | + return -EINVAL; | |
| 1233 | + } | |
| 1234 | + | |
| 1235 | + if (!(vma = nvkm_vmm_node_search(vmm, addr))) | |
| 1236 | + return -ENOENT; | |
| 1237 | + | |
| 1238 | + do { | |
| 1239 | + bool map = !!(pfn[pi] & NVKM_VMM_PFN_V); | |
| 1240 | + bool mapped = vma->mapped; | |
| 1241 | + u64 size = limit - start; | |
| 1242 | + u64 addr = start; | |
| 1243 | + int pn, ret = 0; | |
| 1244 | + | |
| 1245 | + /* Narrow the operation window to cover a single action (page | |
| 1246 | + * should be mapped or not) within a single VMA. | |
| 1247 | + */ | |
| 1248 | + for (pn = 0; pi + pn < pm; pn++) { | |
| 1249 | + if (map != !!(pfn[pi + pn] & NVKM_VMM_PFN_V)) | |
| 1250 | + break; | |
| 1251 | + } | |
| 1252 | + size = min_t(u64, size, pn << page->shift); | |
| 1253 | + size = min_t(u64, size, vma->size + vma->addr - addr); | |
| 1254 | + | |
| 1255 | + /* Reject any operation to unmanaged regions, and areas that | |
| 1256 | + * have nvkm_memory objects mapped in them already. | |
| 1257 | + */ | |
| 1258 | + if (!vma->mapref || vma->memory) { | |
| 1259 | + ret = -EINVAL; | |
| 1260 | + goto next; | |
| 1261 | + } | |
| 1262 | + | |
| 1263 | + /* In order to both properly refcount GPU page tables, and | |
| 1264 | + * prevent "normal" mappings and these direct mappings from | |
| 1265 | + * interfering with each other, we need to track contiguous | |
| 1266 | + * ranges that have been mapped with this interface. | |
| 1267 | + * | |
| 1268 | + * Here we attempt to either split an existing VMA so we're | |
| 1269 | + * able to flag the region as either unmapped/mapped, or to | |
| 1270 | + * merge with adjacent VMAs that are already compatible. | |
| 1271 | + * | |
| 1272 | + * If the region is already compatible, nothing is required. | |
| 1273 | + */ | |
| 1274 | + if (map != mapped) { | |
| 1275 | + tmp = nvkm_vmm_pfn_split_merge(vmm, vma, addr, size, | |
| 1276 | + page - | |
| 1277 | + vmm->func->page, map); | |
| 1278 | + if (WARN_ON(!tmp)) { | |
| 1279 | + ret = -ENOMEM; | |
| 1280 | + goto next; | |
| 1281 | + } | |
| 1282 | + | |
| 1283 | + if ((tmp->mapped = map)) | |
| 1284 | + tmp->refd = page - vmm->func->page; | |
| 1285 | + else | |
| 1286 | + tmp->refd = NVKM_VMA_PAGE_NONE; | |
| 1287 | + vma = tmp; | |
| 1288 | + } | |
| 1289 | + | |
| 1290 | + /* Update HW page tables. */ | |
| 1291 | + if (map) { | |
| 1292 | + struct nvkm_vmm_map args; | |
| 1293 | + args.page = page; | |
| 1294 | + args.pfn = &pfn[pi]; | |
| 1295 | + | |
| 1296 | + if (!mapped) { | |
| 1297 | + ret = nvkm_vmm_ptes_get_map(vmm, page, addr, | |
| 1298 | + size, &args, page-> | |
| 1299 | + desc->func->pfn); | |
| 1300 | + } else { | |
| 1301 | + nvkm_vmm_ptes_map(vmm, page, addr, size, &args, | |
| 1302 | + page->desc->func->pfn); | |
| 1303 | + } | |
| 1304 | + } else { | |
| 1305 | + if (mapped) { | |
| 1306 | + nvkm_vmm_ptes_unmap_put(vmm, page, addr, size, | |
| 1307 | + false, true); | |
| 1308 | + } | |
| 1309 | + } | |
| 1310 | + | |
| 1311 | +next: | |
| 1312 | + /* Iterate to next operation. */ | |
| 1313 | + if (vma->addr + vma->size == addr + size) | |
| 1314 | + vma = node(vma, next); | |
| 1315 | + start += size; | |
| 1316 | + | |
| 1317 | + if (ret) { | |
| 1318 | + /* Failure is signalled by clearing the valid bit on | |
| 1319 | + * any PFN that couldn't be modified as requested. | |
| 1320 | + */ | |
| 1321 | + while (size) { | |
| 1322 | + pfn[pi++] = NVKM_VMM_PFN_NONE; | |
| 1323 | + size -= 1 << page->shift; | |
| 1324 | + } | |
| 1325 | + } else { | |
| 1326 | + pi += size >> page->shift; | |
| 1327 | + } | |
| 1328 | + } while (vma && start < limit); | |
| 1329 | + | |
| 1330 | + return 0; | |
| 1055 | 1331 | } |
| 1056 | 1332 | |
| 1057 | 1333 | void |
| 1058 | 1334 | nvkm_vmm_unmap_region(struct nvkm_vmm *vmm, struct nvkm_vma *vma) |
| 1059 | 1335 | { |
| 1060 | − struct nvkm_vma *next = node(vma, next); | |
| 1061 | 1336 | struct nvkm_vma *prev = NULL; |
| 1337 | + struct nvkm_vma *next; | |
| 1062 | 1338 | |
| 1063 | 1339 | nvkm_memory_tags_put(vma->memory, vmm->mmu->subdev.device, &vma->tags); |
| 1064 | 1340 | nvkm_memory_unref(&vma->memory); |
| 1341 | + vma->mapped = false; | |
| 1065 | 1342 | |
| 1066 | − if (!vma->part || ((prev = node(vma, prev)), prev->memory)) | |
| 1343 | + if (vma->part && (prev = node(vma, prev)) && prev->mapped) | |
| 1067 | 1344 | prev = NULL; |
| 1068 | − if (!next->part || next->memory) | |
| 1345 | + if ((next = node(vma, next)) && (!next->part || next->mapped)) | |
| 1069 | 1346 | next = NULL; |
| 1070 | 1347 | nvkm_vmm_node_merge(vmm, prev, vma, next, vma->size); |
| 1071 | 1348 | } |
| 1072 | 1349 | |
| 1073 | 1350 | void |
| 1074 | −nvkm_vmm_unmap_locked(struct nvkm_vmm *vmm, struct nvkm_vma *vma) | |
| 1351 | +nvkm_vmm_unmap_locked(struct nvkm_vmm *vmm, struct nvkm_vma *vma, bool pfn) | |
| 1075 | 1352 | { |
| 1076 | 1353 | const struct nvkm_vmm_page *page = &vmm->func->page[vma->refd]; |
| 1077 | 1354 | |
| 1078 | 1355 | if (vma->mapref) { |
| 1079 | − nvkm_vmm_ptes_unmap_put(vmm, page, vma->addr, vma->size, vma->sparse); | |
| 1356 | + nvkm_vmm_ptes_unmap_put(vmm, page, vma->addr, vma->size, vma->sparse, pfn); | |
| 1080 | 1357 | vma->refd = NVKM_VMA_PAGE_NONE; |
| 1081 | 1358 | } else { |
| 1082 | − nvkm_vmm_ptes_unmap(vmm, page, vma->addr, vma->size, vma->sparse); | |
| 1359 | + nvkm_vmm_ptes_unmap(vmm, page, vma->addr, vma->size, vma->sparse, pfn); | |
| 1083 | 1360 | } |
| 1084 | 1361 | |
| 1085 | 1362 | nvkm_vmm_unmap_region(vmm, vma); |
| @@ -1090,7 +1367,7 @@ nvkm_vmm_unmap(struct nvkm_vmm *vmm, struct nvkm_vma *vma) · 4 unmodified lines | ||
| 1090 | 1367 | { |
| 1091 | 1368 | if (vma->memory) { |
| 1092 | 1369 | mutex_lock(&vmm->mutex); |
| 1093 | − nvkm_vmm_unmap_locked(vmm, vma); | |
| 1370 | + nvkm_vmm_unmap_locked(vmm, vma, false); | |
| 1094 | 1371 | mutex_unlock(&vmm->mutex); |
| 1095 | 1372 | } |
| 1096 | 1373 | } |
| @@ -1224,6 +1501,7 @@ nvkm_vmm_map_locked(struct nvkm_vmm *vmm, struct nvkm_vma *vma, · 127 unmodified lines | ||
| 1224 | 1501 | nvkm_memory_tags_put(vma->memory, vmm->mmu->subdev.device, &vma->tags); |
| 1225 | 1502 | nvkm_memory_unref(&vma->memory); |
| 1226 | 1503 | vma->memory = nvkm_memory_ref(map->memory); |
| 1504 | + vma->mapped = true; | |
| 1227 | 1505 | vma->tags = map->tags; |
| 1228 | 1506 | return 0; |
| 1229 | 1507 | } |
| @@ -1269,14 +1547,16 @@ nvkm_vmm_put_locked(struct nvkm_vmm *vmm, struct nvkm_vma *vma) · 39 unmodified lines | ||
| 1269 | 1547 | |
| 1270 | 1548 | if (vma->mapref || !vma->sparse) { |
| 1271 | 1549 | do { |
| 1272 | − const bool map = next->memory != NULL; | |
| 1550 | + const bool mem = next->memory != NULL; | |
| 1551 | + const bool map = next->mapped; | |
| 1273 | 1552 | const u8 refd = next->refd; |
| 1274 | 1553 | const u64 addr = next->addr; |
| 1275 | 1554 | u64 size = next->size; |
| 1276 | 1555 | |
| 1277 | 1556 | /* Merge regions that are in the same state. */ |
| 1278 | 1557 | while ((next = node(next, next)) && next->part && |
| 1279 | − (next->memory != NULL) == map && | |
| 1558 | + (next->mapped == map) && | |
| 1559 | + (next->memory != NULL) == mem && | |
| 1280 | 1560 | (next->refd == refd)) |
| 1281 | 1561 | size += next->size; |
| 1282 | 1562 | |
| @@ -1286,7 +1566,8 @@ nvkm_vmm_put_locked(struct nvkm_vmm *vmm, struct nvkm_vma *vma) · 3 unmodified lines | ||
| 1286 | 1566 | * the page tree. |
| 1287 | 1567 | */ |
| 1288 | 1568 | nvkm_vmm_ptes_unmap_put(vmm, &page[refd], addr, |
| 1289 | − size, vma->sparse); | |
| 1569 | + size, vma->sparse, | |
| 1570 | + !mem); | |
| 1290 | 1571 | } else |
| 1291 | 1572 | if (refd != NVKM_VMA_PAGE_NONE) { |
| 1292 | 1573 | /* Drop allocation-time PTE references. */ |
| @@ -1301,7 +1582,7 @@ nvkm_vmm_put_locked(struct nvkm_vmm *vmm, struct nvkm_vma *vma) · 8 unmodified lines | ||
| 1301 | 1582 | */ |
| 1302 | 1583 | next = vma; |
| 1303 | 1584 | do { |
| 1304 | − if (next->memory) | |
| 1585 | + if (next->mapped) | |
| 1305 | 1586 | nvkm_vmm_unmap_region(vmm, next); |
| 1306 | 1587 | } while ((next = node(vma, next)) && next->part); |
| 1307 | 1588 | |
| @@ -1522,7 +1803,7 @@ nvkm_vmm_join(struct nvkm_vmm *vmm, struct nvkm_memory *inst) · 214 unmodified lines | ||
| 1522 | 1803 | } |
| 1523 | 1804 | |
| 1524 | 1805 | static bool |
| 1525 | −nvkm_vmm_boot_ptes(struct nvkm_vmm_iter *it, u32 ptei, u32 ptes) | |
| 1806 | +nvkm_vmm_boot_ptes(struct nvkm_vmm_iter *it, bool pfn, u32 ptei, u32 ptes) | |
| 1526 | 1807 | { |
| 1527 | 1808 | const struct nvkm_vmm_desc *desc = it->desc; |
| 1528 | 1809 | const int type = desc->type == SPT; |
| @@ -1544,7 +1825,7 @@ nvkm_vmm_boot(struct nvkm_vmm *vmm) · 15 unmodified lines | ||
| 1544 | 1825 | if (ret) |
| 1545 | 1826 | return ret; |
| 1546 | 1827 | |
| 1547 | − nvkm_vmm_iter(vmm, page, vmm->start, limit, "bootstrap", false, | |
| 1828 | + nvkm_vmm_iter(vmm, page, vmm->start, limit, "bootstrap", false, false, | |
| 1548 | 1829 | nvkm_vmm_boot_ptes, NULL, NULL, NULL); |
| 1549 | 1830 | vmm->bootstrapped = true; |
| 1550 | 1831 | return 0; |
| @@ -1584,7 +1865,8 @@ nvkm_vmm_new(struct nvkm_device *device, u64 addr, u64 size, void *argv, · 33 unmodified lines | ||
| 1584 | 1865 | struct nvkm_mmu *mmu = device->mmu; |
| 1585 | 1866 | struct nvkm_vmm *vmm = NULL; |
| 1586 | 1867 | int ret; |
| 1587 | − ret = mmu->func->vmm.ctor(mmu, addr, size, argv, argc, key, name, &vmm); | |
| 1868 | + ret = mmu->func->vmm.ctor(mmu, false, addr, size, argv, argc, | |
| 1869 | + key, name, &vmm); | |
| 1588 | 1870 | if (ret) |
| 1589 | 1871 | nvkm_vmm_unref(&vmm); |
| 1590 | 1872 | *pvmm = vmm; |
| @@ -67,6 +67,10 @@ struct nvkm_vmm_desc_func { · 66 unmodified lines | ||
| 67 | 67 | nvkm_vmm_pte_func mem; |
| 68 | 68 | nvkm_vmm_pte_func dma; |
| 69 | 69 | nvkm_vmm_pte_func sgl; |
| 70 | + | |
| 71 | + nvkm_vmm_pte_func pfn; | |
| 72 | + bool (*pfn_clear)(struct nvkm_vmm *, struct nvkm_mmu_pt *, u32 ptei, u32 ptes); | |
| 73 | + nvkm_vmm_pxe_func pfn_unmap; | |
| 70 | 74 | }; |
| 71 | 75 | |
| 72 | 76 | extern const struct nvkm_vmm_desc_func gf100_vmm_pgd; |
| @@ -141,6 +145,11 @@ struct nvkm_vmm_func { · 68 unmodified lines | ||
| 141 | 145 | struct nvkm_vmm_map *); |
| 142 | 146 | void (*flush)(struct nvkm_vmm *, int depth); |
| 143 | 147 | |
| 148 | + int (*mthd)(struct nvkm_vmm *, struct nvkm_client *, | |
| 149 | + u32 mthd, void *argv, u32 argc); | |
| 150 | + | |
| 151 | + void (*invalidate_pdb)(struct nvkm_vmm *, u64 addr); | |
| 152 | + | |
| 144 | 153 | u64 page_block; |
| 145 | 154 | const struct nvkm_vmm_page page[]; |
| 146 | 155 | }; |
| @@ -151,11 +160,12 @@ struct nvkm_vmm_join { · 4 unmodified lines | ||
| 151 | 160 | }; |
| 152 | 161 | |
| 153 | 162 | int nvkm_vmm_new_(const struct nvkm_vmm_func *, struct nvkm_mmu *, |
| 154 | − u32 pd_header, u64 addr, u64 size, struct lock_class_key *, | |
| 155 | − const char *name, struct nvkm_vmm **); | |
| 163 | + u32 pd_header, bool managed, u64 addr, u64 size, | |
| 164 | + struct lock_class_key *, const char *name, | |
| 165 | + struct nvkm_vmm **); | |
| 156 | 166 | int nvkm_vmm_ctor(const struct nvkm_vmm_func *, struct nvkm_mmu *, |
| 157 | − u32 pd_header, u64 addr, u64 size, struct lock_class_key *, | |
| 158 | − const char *name, struct nvkm_vmm *); | |
| 167 | + u32 pd_header, bool managed, u64 addr, u64 size, | |
| 168 | + struct lock_class_key *, const char *name, struct nvkm_vmm *); | |
| 159 | 169 | struct nvkm_vma *nvkm_vmm_node_search(struct nvkm_vmm *, u64 addr); |
| 160 | 170 | struct nvkm_vma *nvkm_vmm_node_split(struct nvkm_vmm *, struct nvkm_vma *, |
| 161 | 171 | u64 addr, u64 size); |
| @@ -163,13 +173,25 @@ int nvkm_vmm_get_locked(struct nvkm_vmm *, bool getref, bool mapref, · 1 unmodified line | ||
| 163 | 173 | bool sparse, u8 page, u8 align, u64 size, |
| 164 | 174 | struct nvkm_vma **pvma); |
| 165 | 175 | void nvkm_vmm_put_locked(struct nvkm_vmm *, struct nvkm_vma *); |
| 166 | −void nvkm_vmm_unmap_locked(struct nvkm_vmm *, struct nvkm_vma *); | |
| 167 | −void nvkm_vmm_unmap_region(struct nvkm_vmm *vmm, struct nvkm_vma *vma); | |
| 176 | +void nvkm_vmm_unmap_locked(struct nvkm_vmm *, struct nvkm_vma *, bool pfn); | |
| 177 | +void nvkm_vmm_unmap_region(struct nvkm_vmm *, struct nvkm_vma *); | |
| 178 | + | |
| 179 | +#define NVKM_VMM_PFN_ADDR 0xfffffffffffff000ULL | |
| 180 | +#define NVKM_VMM_PFN_ADDR_SHIFT 12 | |
| 181 | +#define NVKM_VMM_PFN_APER 0x00000000000000f0ULL | |
| 182 | +#define NVKM_VMM_PFN_HOST 0x0000000000000000ULL | |
| 183 | +#define NVKM_VMM_PFN_VRAM 0x0000000000000010ULL | |
| 184 | +#define NVKM_VMM_PFN_W 0x0000000000000002ULL | |
| 185 | +#define NVKM_VMM_PFN_V 0x0000000000000001ULL | |
| 186 | +#define NVKM_VMM_PFN_NONE 0x0000000000000000ULL | |
| 187 | + | |
| 188 | +int nvkm_vmm_pfn_map(struct nvkm_vmm *, u8 page, u64 addr, u64 size, u64 *pfn); | |
| 189 | +int nvkm_vmm_pfn_unmap(struct nvkm_vmm *, u64 addr, u64 size); | |
| 168 | 190 | |
| 169 | 191 | struct nvkm_vma *nvkm_vma_tail(struct nvkm_vma *, u64 tail); |
| 170 | 192 | |
| 171 | 193 | int nv04_vmm_new_(const struct nvkm_vmm_func *, struct nvkm_mmu *, u32, |
| 172 | − u64, u64, void *, u32, struct lock_class_key *, | |
| 194 | + bool, u64, u64, void *, u32, struct lock_class_key *, | |
| 173 | 195 | const char *, struct nvkm_vmm **); |
| 174 | 196 | int nv04_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *); |
| 175 | 197 | |
| @@ -179,70 +201,76 @@ int nv50_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *); · 3 unmodified lines | ||
| 179 | 201 | void nv50_vmm_flush(struct nvkm_vmm *, int); |
| 180 | 202 | |
| 181 | 203 | int gf100_vmm_new_(const struct nvkm_vmm_func *, const struct nvkm_vmm_func *, |
| 182 | − struct nvkm_mmu *, u64, u64, void *, u32, | |
| 204 | + struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 183 | 205 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 184 | 206 | int gf100_vmm_join_(struct nvkm_vmm *, struct nvkm_memory *, u64 base); |
| 185 | 207 | int gf100_vmm_join(struct nvkm_vmm *, struct nvkm_memory *); |
| 186 | 208 | void gf100_vmm_part(struct nvkm_vmm *, struct nvkm_memory *); |
| 187 | 209 | int gf100_vmm_aper(enum nvkm_memory_target); |
| 188 | 210 | int gf100_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *); |
| 189 | −void gf100_vmm_flush_(struct nvkm_vmm *, int); | |
| 190 | 211 | void gf100_vmm_flush(struct nvkm_vmm *, int); |
| 212 | +void gf100_vmm_invalidate(struct nvkm_vmm *, u32 type); | |
| 213 | +void gf100_vmm_invalidate_pdb(struct nvkm_vmm *, u64 addr); | |
| 191 | 214 | |
| 192 | 215 | int gk20a_vmm_aper(enum nvkm_memory_target); |
| 193 | 216 | |
| 194 | 217 | int gm200_vmm_new_(const struct nvkm_vmm_func *, const struct nvkm_vmm_func *, |
| 195 | − struct nvkm_mmu *, u64, u64, void *, u32, | |
| 218 | + struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 196 | 219 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 197 | 220 | int gm200_vmm_join_(struct nvkm_vmm *, struct nvkm_memory *, u64 base); |
| 198 | 221 | int gm200_vmm_join(struct nvkm_vmm *, struct nvkm_memory *); |
| 199 | 222 | |
| 223 | +int gp100_vmm_new_(const struct nvkm_vmm_func *, | |
| 224 | + struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 225 | + struct lock_class_key *, const char *, struct nvkm_vmm **); | |
| 200 | 226 | int gp100_vmm_join(struct nvkm_vmm *, struct nvkm_memory *); |
| 201 | 227 | int gp100_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *); |
| 202 | 228 | void gp100_vmm_flush(struct nvkm_vmm *, int); |
| 229 | +int gp100_vmm_mthd(struct nvkm_vmm *, struct nvkm_client *, u32, void *, u32); | |
| 230 | +void gp100_vmm_invalidate_pdb(struct nvkm_vmm *, u64 addr); | |
| 203 | 231 | |
| 204 | 232 | int gv100_vmm_join(struct nvkm_vmm *, struct nvkm_memory *); |
| 205 | 233 | |
| 206 | −int nv04_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 234 | +int nv04_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 207 | 235 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 208 | −int nv41_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 236 | +int nv41_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 209 | 237 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 210 | −int nv44_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 238 | +int nv44_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 211 | 239 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 212 | −int nv50_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 240 | +int nv50_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 213 | 241 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 214 | −int mcp77_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 242 | +int mcp77_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 215 | 243 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 216 | −int g84_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 244 | +int g84_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 217 | 245 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 218 | −int gf100_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 246 | +int gf100_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 219 | 247 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 220 | −int gk104_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 248 | +int gk104_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 221 | 249 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 222 | −int gk20a_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 250 | +int gk20a_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 223 | 251 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 224 | −int gm200_vmm_new_fixed(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 252 | +int gm200_vmm_new_fixed(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 225 | 253 | struct lock_class_key *, const char *, |
| 226 | 254 | struct nvkm_vmm **); |
| 227 | −int gm200_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 255 | +int gm200_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 228 | 256 | struct lock_class_key *, const char *, |
| 229 | 257 | struct nvkm_vmm **); |
| 230 | −int gm20b_vmm_new_fixed(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 258 | +int gm20b_vmm_new_fixed(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 231 | 259 | struct lock_class_key *, const char *, |
| 232 | 260 | struct nvkm_vmm **); |
| 233 | −int gm20b_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 261 | +int gm20b_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 234 | 262 | struct lock_class_key *, const char *, |
| 235 | 263 | struct nvkm_vmm **); |
| 236 | −int gp100_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 264 | +int gp100_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 237 | 265 | struct lock_class_key *, const char *, |
| 238 | 266 | struct nvkm_vmm **); |
| 239 | −int gp10b_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 267 | +int gp10b_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 240 | 268 | struct lock_class_key *, const char *, |
| 241 | 269 | struct nvkm_vmm **); |
| 242 | −int gv100_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 270 | +int gv100_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 243 | 271 | struct lock_class_key *, const char *, |
| 244 | 272 | struct nvkm_vmm **); |
| 245 | −int tu104_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | |
| 273 | +int tu102_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, | |
| 246 | 274 | struct lock_class_key *, const char *, |
| 247 | 275 | struct nvkm_vmm **); |
| 248 | 276 | |
| @@ -178,15 +178,19 @@ gf100_vmm_desc_16_16[] = { · 177 unmodified lines | ||
| 178 | 178 | }; |
| 179 | 179 | |
| 180 | 180 | void |
| 181 | −gf100_vmm_flush_(struct nvkm_vmm *vmm, int depth) | |
| 181 | +gf100_vmm_invalidate_pdb(struct nvkm_vmm *vmm, u64 addr) | |
| 182 | +{ | |
| 183 | + struct nvkm_device *device = vmm->mmu->subdev.device; | |
| 184 | + nvkm_wr32(device, 0x100cb8, addr); | |
| 185 | +} | |
| 186 | + | |
| 187 | +void | |
| 188 | +gf100_vmm_invalidate(struct nvkm_vmm *vmm, u32 type) | |
| 182 | 189 | { |
| 183 | 190 | struct nvkm_subdev *subdev = &vmm->mmu->subdev; |
| 184 | 191 | struct nvkm_device *device = subdev->device; |
| 185 | − u32 type = depth << 24; | |
| 186 | − | |
| 187 | − type = 0x00000001; /* PAGE_ALL */ | |
| 188 | − if (atomic_read(&vmm->engref[NVKM_SUBDEV_BAR])) | |
| 189 | − type |= 0x00000004; /* HUB_ONLY */ | |
| 192 | + struct nvkm_mmu_pt *pd = vmm->pd->pt[0]; | |
| 193 | + u64 addr = 0; | |
| 190 | 194 | |
| 191 | 195 | mutex_lock(&subdev->mutex); |
| 192 | 196 | /* Looks like maybe a "free flush slots" counter, the |
| @@ -197,7 +201,20 @@ gf100_vmm_flush_(struct nvkm_vmm *vmm, int depth) · 4 unmodified lines | ||
| 197 | 201 | break; |
| 198 | 202 | ); |
| 199 | 203 | |
| 200 | − nvkm_wr32(device, 0x100cb8, vmm->pd->pt[0]->addr >> 8); | |
| 204 | + if (!(type & 0x00000002) /* ALL_PDB. */) { | |
| 205 | + switch (nvkm_memory_target(pd->memory)) { | |
| 206 | + case NVKM_MEM_TARGET_VRAM: addr |= 0x00000000; break; | |
| 207 | + case NVKM_MEM_TARGET_HOST: addr |= 0x00000002; break; | |
| 208 | + case NVKM_MEM_TARGET_NCOH: addr |= 0x00000003; break; | |
| 209 | + default: | |
| 210 | + WARN_ON(1); | |
| 211 | + break; | |
| 212 | + } | |
| 213 | + addr |= (vmm->pd->pt[0]->addr >> 12) << 4; | |
| 214 | + | |
| 215 | + vmm->func->invalidate_pdb(vmm, addr); | |
| 216 | + } | |
| 217 | + | |
| 201 | 218 | nvkm_wr32(device, 0x100cbc, 0x80000000 | type); |
| 202 | 219 | |
| 203 | 220 | /* Wait for flush to be queued? */ |
| @@ -211,7 +228,10 @@ gf100_vmm_flush_(struct nvkm_vmm *vmm, int depth) · 7 unmodified lines | ||
| 211 | 228 | void |
| 212 | 229 | gf100_vmm_flush(struct nvkm_vmm *vmm, int depth) |
| 213 | 230 | { |
| 214 | − gf100_vmm_flush_(vmm, 0); | |
| 231 | + u32 type = 0x00000001; /* PAGE_ALL */ | |
| 232 | + if (atomic_read(&vmm->engref[NVKM_SUBDEV_BAR])) | |
| 233 | + type |= 0x00000004; /* HUB_ONLY */ | |
| 234 | + gf100_vmm_invalidate(vmm, type); | |
| 215 | 235 | } |
| 216 | 236 | |
| 217 | 237 | int |
| @@ -354,6 +374,7 @@ gf100_vmm_17 = { · 136 unmodified lines | ||
| 354 | 374 | .aper = gf100_vmm_aper, |
| 355 | 375 | .valid = gf100_vmm_valid, |
| 356 | 376 | .flush = gf100_vmm_flush, |
| 377 | + .invalidate_pdb = gf100_vmm_invalidate_pdb, | |
| 357 | 378 | .page = { |
| 358 | 379 | { 17, &gf100_vmm_desc_17_17[0], NVKM_VMM_PAGE_xVxC }, |
| 359 | 380 | { 12, &gf100_vmm_desc_17_12[0], NVKM_VMM_PAGE_xVHx }, |
| @@ -368,6 +389,7 @@ gf100_vmm_16 = { · 8 unmodified lines | ||
| 368 | 389 | .aper = gf100_vmm_aper, |
| 369 | 390 | .valid = gf100_vmm_valid, |
| 370 | 391 | .flush = gf100_vmm_flush, |
| 392 | + .invalidate_pdb = gf100_vmm_invalidate_pdb, | |
| 371 | 393 | .page = { |
| 372 | 394 | { 16, &gf100_vmm_desc_16_16[0], NVKM_VMM_PAGE_xVxC }, |
| 373 | 395 | { 12, &gf100_vmm_desc_16_12[0], NVKM_VMM_PAGE_xVHx }, |
| @@ -378,14 +400,14 @@ gf100_vmm_16 = { · 4 unmodified lines | ||
| 378 | 400 | int |
| 379 | 401 | gf100_vmm_new_(const struct nvkm_vmm_func *func_16, |
| 380 | 402 | const struct nvkm_vmm_func *func_17, |
| 381 | − struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | |
| 382 | − struct lock_class_key *key, const char *name, | |
| 383 | − struct nvkm_vmm **pvmm) | |
| 403 | + struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 404 | + void *argv, u32 argc, struct lock_class_key *key, | |
| 405 | + const char *name, struct nvkm_vmm **pvmm) | |
| 384 | 406 | { |
| 385 | 407 | switch (mmu->subdev.device->fb->page) { |
| 386 | − case 16: return nv04_vmm_new_(func_16, mmu, 0, addr, size, | |
| 408 | + case 16: return nv04_vmm_new_(func_16, mmu, 0, managed, addr, size, | |
| 387 | 409 | argv, argc, key, name, pvmm); |
| 388 | − case 17: return nv04_vmm_new_(func_17, mmu, 0, addr, size, | |
| 410 | + case 17: return nv04_vmm_new_(func_17, mmu, 0, managed, addr, size, | |
| 389 | 411 | argv, argc, key, name, pvmm); |
| 390 | 412 | default: |
| 391 | 413 | WARN_ON(1); |
| @@ -394,10 +416,10 @@ gf100_vmm_new_(const struct nvkm_vmm_func *func_16, · 2 unmodified lines | ||
| 394 | 416 | } |
| 395 | 417 | |
| 396 | 418 | int |
| 397 | −gf100_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | |
| 398 | − struct lock_class_key *key, const char *name, | |
| 399 | − struct nvkm_vmm **pvmm) | |
| 419 | +gf100_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 420 | + void *argv, u32 argc, struct lock_class_key *key, | |
| 421 | + const char *name, struct nvkm_vmm **pvmm) | |
| 400 | 422 | { |
| 401 | − return gf100_vmm_new_(&gf100_vmm_16, &gf100_vmm_17, mmu, addr, | |
| 423 | + return gf100_vmm_new_(&gf100_vmm_16, &gf100_vmm_17, mmu, managed, addr, | |
| 402 | 424 | size, argv, argc, key, name, pvmm); |
| 403 | 425 | } |
| @@ -71,6 +71,7 @@ gk104_vmm_17 = { · 70 unmodified lines | ||
| 71 | 71 | .aper = gf100_vmm_aper, |
| 72 | 72 | .valid = gf100_vmm_valid, |
| 73 | 73 | .flush = gf100_vmm_flush, |
| 74 | + .invalidate_pdb = gf100_vmm_invalidate_pdb, | |
| 74 | 75 | .page = { |
| 75 | 76 | { 17, &gk104_vmm_desc_17_17[0], NVKM_VMM_PAGE_xVxC }, |
| 76 | 77 | { 12, &gk104_vmm_desc_17_12[0], NVKM_VMM_PAGE_xVHx }, |
| @@ -85,6 +86,7 @@ gk104_vmm_16 = { · 8 unmodified lines | ||
| 85 | 86 | .aper = gf100_vmm_aper, |
| 86 | 87 | .valid = gf100_vmm_valid, |
| 87 | 88 | .flush = gf100_vmm_flush, |
| 89 | + .invalidate_pdb = gf100_vmm_invalidate_pdb, | |
| 88 | 90 | .page = { |
| 89 | 91 | { 16, &gk104_vmm_desc_16_16[0], NVKM_VMM_PAGE_xVxC }, |
| 90 | 92 | { 12, &gk104_vmm_desc_16_12[0], NVKM_VMM_PAGE_xVHx }, |
| @@ -93,10 +95,10 @@ gk104_vmm_16 = { · 2 unmodified lines | ||
| 93 | 95 | }; |
| 94 | 96 | |
| 95 | 97 | int |
| 96 | −gk104_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | |
| 97 | − struct lock_class_key *key, const char *name, | |
| 98 | − struct nvkm_vmm **pvmm) | |
| 98 | +gk104_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 99 | + void *argv, u32 argc, struct lock_class_key *key, | |
| 100 | + const char *name, struct nvkm_vmm **pvmm) | |
| 99 | 101 | { |
| 100 | − return gf100_vmm_new_(&gk104_vmm_16, &gk104_vmm_17, mmu, addr, | |
| 102 | + return gf100_vmm_new_(&gk104_vmm_16, &gk104_vmm_17, mmu, managed, addr, | |
| 101 | 103 | size, argv, argc, key, name, pvmm); |
| 102 | 104 | } |
| @@ -40,6 +40,7 @@ gk20a_vmm_17 = { · 39 unmodified lines | ||
| 40 | 40 | .aper = gf100_vmm_aper, |
| 41 | 41 | .valid = gf100_vmm_valid, |
| 42 | 42 | .flush = gf100_vmm_flush, |
| 43 | + .invalidate_pdb = gf100_vmm_invalidate_pdb, | |
| 43 | 44 | .page = { |
| 44 | 45 | { 17, &gk104_vmm_desc_17_17[0], NVKM_VMM_PAGE_xxHC }, |
| 45 | 46 | { 12, &gk104_vmm_desc_17_12[0], NVKM_VMM_PAGE_xxHx }, |
| @@ -54,6 +55,7 @@ gk20a_vmm_16 = { · 8 unmodified lines | ||
| 54 | 55 | .aper = gf100_vmm_aper, |
| 55 | 56 | .valid = gf100_vmm_valid, |
| 56 | 57 | .flush = gf100_vmm_flush, |
| 58 | + .invalidate_pdb = gf100_vmm_invalidate_pdb, | |
| 57 | 59 | .page = { |
| 58 | 60 | { 16, &gk104_vmm_desc_16_16[0], NVKM_VMM_PAGE_xxHC }, |
| 59 | 61 | { 12, &gk104_vmm_desc_16_12[0], NVKM_VMM_PAGE_xxHx }, |
| @@ -62,10 +64,10 @@ gk20a_vmm_16 = { · 2 unmodified lines | ||
| 62 | 64 | }; |
| 63 | 65 | |
| 64 | 66 | int |
| 65 | −gk20a_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | |
| 66 | − struct lock_class_key *key, const char *name, | |
| 67 | − struct nvkm_vmm **pvmm) | |
| 67 | +gk20a_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 68 | + void *argv, u32 argc, struct lock_class_key *key, | |
| 69 | + const char *name, struct nvkm_vmm **pvmm) | |
| 68 | 70 | { |
| 69 | − return gf100_vmm_new_(&gk20a_vmm_16, &gk20a_vmm_17, mmu, addr, | |
| 71 | + return gf100_vmm_new_(&gk20a_vmm_16, &gk20a_vmm_17, mmu, managed, addr, | |
| 70 | 72 | size, argv, argc, key, name, pvmm); |
| 71 | 73 | } |
| @@ -113,6 +113,7 @@ gm200_vmm_17 = { · 112 unmodified lines | ||
| 113 | 113 | .aper = gf100_vmm_aper, |
| 114 | 114 | .valid = gf100_vmm_valid, |
| 115 | 115 | .flush = gf100_vmm_flush, |
| 116 | + .invalidate_pdb = gf100_vmm_invalidate_pdb, | |
| 116 | 117 | .page = { |
| 117 | 118 | { 27, &gm200_vmm_desc_17_17[1], NVKM_VMM_PAGE_Sxxx }, |
| 118 | 119 | { 17, &gm200_vmm_desc_17_17[0], NVKM_VMM_PAGE_SVxC }, |
| @@ -128,6 +129,7 @@ gm200_vmm_16 = { · 9 unmodified lines | ||
| 128 | 129 | .aper = gf100_vmm_aper, |
| 129 | 130 | .valid = gf100_vmm_valid, |
| 130 | 131 | .flush = gf100_vmm_flush, |
| 132 | + .invalidate_pdb = gf100_vmm_invalidate_pdb, | |
| 131 | 133 | .page = { |
| 132 | 134 | { 27, &gm200_vmm_desc_16_16[1], NVKM_VMM_PAGE_Sxxx }, |
| 133 | 135 | { 16, &gm200_vmm_desc_16_16[0], NVKM_VMM_PAGE_SVxC }, |
| @@ -139,9 +141,9 @@ gm200_vmm_16 = { · 5 unmodified lines | ||
| 139 | 141 | int |
| 140 | 142 | gm200_vmm_new_(const struct nvkm_vmm_func *func_16, |
| 141 | 143 | const struct nvkm_vmm_func *func_17, |
| 142 | − struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | |
| 143 | − struct lock_class_key *key, const char *name, | |
| 144 | − struct nvkm_vmm **pvmm) | |
| 144 | + struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 145 | + void *argv, u32 argc, struct lock_class_key *key, | |
| 146 | + const char *name, struct nvkm_vmm **pvmm) | |
| 145 | 147 | { |
| 146 | 148 | const struct nvkm_vmm_func *func; |
| 147 | 149 | union { |
| @@ -163,23 +165,23 @@ gm200_vmm_new_(const struct nvkm_vmm_func *func_16, · 15 unmodified lines | ||
| 163 | 165 | } else |
| 164 | 166 | return ret; |
| 165 | 167 | |
| 166 | − return nvkm_vmm_new_(func, mmu, 0, addr, size, key, name, pvmm); | |
| 168 | + return nvkm_vmm_new_(func, mmu, 0, managed, addr, size, key, name, pvmm); | |
| 167 | 169 | } |
| 168 | 170 | |
| 169 | 171 | int |
| 170 | −gm200_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | |
| 171 | − struct lock_class_key *key, const char *name, | |
| 172 | − struct nvkm_vmm **pvmm) | |
| 172 | +gm200_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 173 | + void *argv, u32 argc, struct lock_class_key *key, | |
| 174 | + const char *name, struct nvkm_vmm **pvmm) | |
| 173 | 175 | { |
| 174 | − return gm200_vmm_new_(&gm200_vmm_16, &gm200_vmm_17, mmu, addr, | |
| 176 | + return gm200_vmm_new_(&gm200_vmm_16, &gm200_vmm_17, mmu, managed, addr, | |
| 175 | 177 | size, argv, argc, key, name, pvmm); |
| 176 | 178 | } |
| 177 | 179 | |
| 178 | 180 | int |
| 179 | −gm200_vmm_new_fixed(struct nvkm_mmu *mmu, u64 addr, u64 size, | |
| 181 | +gm200_vmm_new_fixed(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 180 | 182 | void *argv, u32 argc, struct lock_class_key *key, |
| 181 | 183 | const char *name, struct nvkm_vmm **pvmm) |
| 182 | 184 | { |
| 183 | − return gf100_vmm_new_(&gm200_vmm_16, &gm200_vmm_17, mmu, addr, | |
| 185 | + return gf100_vmm_new_(&gm200_vmm_16, &gm200_vmm_17, mmu, managed, addr, | |
| 184 | 186 | size, argv, argc, key, name, pvmm); |
| 185 | 187 | } |
| @@ -28,6 +28,7 @@ gm20b_vmm_17 = { · 27 unmodified lines | ||
| 28 | 28 | .aper = gk20a_vmm_aper, |
| 29 | 29 | .valid = gf100_vmm_valid, |
| 30 | 30 | .flush = gf100_vmm_flush, |
| 31 | + .invalidate_pdb = gf100_vmm_invalidate_pdb, | |
| 31 | 32 | .page = { |
| 32 | 33 | { 27, &gm200_vmm_desc_17_17[1], NVKM_VMM_PAGE_Sxxx }, |
| 33 | 34 | { 17, &gm200_vmm_desc_17_17[0], NVKM_VMM_PAGE_SxHC }, |
| @@ -43,6 +44,7 @@ gm20b_vmm_16 = { · 9 unmodified lines | ||
| 43 | 44 | .aper = gk20a_vmm_aper, |
| 44 | 45 | .valid = gf100_vmm_valid, |
| 45 | 46 | .flush = gf100_vmm_flush, |
| 47 | + .invalidate_pdb = gf100_vmm_invalidate_pdb, | |
| 46 | 48 | .page = { |
| 47 | 49 | { 27, &gm200_vmm_desc_16_16[1], NVKM_VMM_PAGE_Sxxx }, |
| 48 | 50 | { 16, &gm200_vmm_desc_16_16[0], NVKM_VMM_PAGE_SxHC }, |
| @@ -52,19 +54,19 @@ gm20b_vmm_16 = { · 3 unmodified lines | ||
| 52 | 54 | }; |
| 53 | 55 | |
| 54 | 56 | int |
| 55 | −gm20b_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | |
| 56 | − struct lock_class_key *key, const char *name, | |
| 57 | − struct nvkm_vmm **pvmm) | |
| 57 | +gm20b_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 58 | + void *argv, u32 argc, struct lock_class_key *key, | |
| 59 | + const char *name, struct nvkm_vmm **pvmm) | |
| 58 | 60 | { |
| 59 | − return gm200_vmm_new_(&gm20b_vmm_16, &gm20b_vmm_17, mmu, addr, | |
| 61 | + return gm200_vmm_new_(&gm20b_vmm_16, &gm20b_vmm_17, mmu, managed, addr, | |
| 60 | 62 | size, argv, argc, key, name, pvmm); |
| 61 | 63 | } |
| 62 | 64 | |
| 63 | 65 | int |
| 64 | −gm20b_vmm_new_fixed(struct nvkm_mmu *mmu, u64 addr, u64 size, | |
| 66 | +gm20b_vmm_new_fixed(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 65 | 67 | void *argv, u32 argc, struct lock_class_key *key, |
| 66 | 68 | const char *name, struct nvkm_vmm **pvmm) |
| 67 | 69 | { |
| 68 | − return gf100_vmm_new_(&gm20b_vmm_16, &gm20b_vmm_17, mmu, addr, | |
| 70 | + return gf100_vmm_new_(&gm20b_vmm_16, &gm20b_vmm_17, mmu, managed, addr, | |
| 69 | 71 | size, argv, argc, key, name, pvmm); |
| 70 | 72 | } |
| @@ -21,12 +21,90 @@ · 20 unmodified lines | ||
| 21 | 21 | */ |
| 22 | 22 | #include "vmm.h" |
| 23 | 23 | |
| 24 | +#include <core/client.h> | |
| 24 | 25 | #include <subdev/fb.h> |
| 25 | 26 | #include <subdev/ltc.h> |
| 27 | +#include <subdev/timer.h> | |
| 28 | +#include <engine/gr.h> | |
| 26 | 29 | |
| 27 | 30 | #include <nvif/ifc00d.h> |
| 28 | 31 | #include <nvif/unpack.h> |
| 29 | 32 | |
| 33 | +static void | |
| 34 | +gp100_vmm_pfn_unmap(struct nvkm_vmm *vmm, | |
| 35 | + struct nvkm_mmu_pt *pt, u32 ptei, u32 ptes) | |
| 36 | +{ | |
| 37 | + struct device *dev = vmm->mmu->subdev.device->dev; | |
| 38 | + dma_addr_t addr; | |
| 39 | + | |
| 40 | + nvkm_kmap(pt->memory); | |
| 41 | + while (ptes--) { | |
| 42 | + u32 datalo = nvkm_ro32(pt->memory, pt->base + ptei * 8 + 0); | |
| 43 | + u32 datahi = nvkm_ro32(pt->memory, pt->base + ptei * 8 + 4); | |
| 44 | + u64 data = (u64)datahi << 32 | datalo; | |
| 45 | + if ((data & (3ULL << 1)) != 0) { | |
| 46 | + addr = (data >> 8) << 12; | |
| 47 | + dma_unmap_page(dev, addr, PAGE_SIZE, DMA_BIDIRECTIONAL); | |
| 48 | + } | |
| 49 | + ptei++; | |
| 50 | + } | |
| 51 | + nvkm_done(pt->memory); | |
| 52 | +} | |
| 53 | + | |
| 54 | +static bool | |
| 55 | +gp100_vmm_pfn_clear(struct nvkm_vmm *vmm, | |
| 56 | + struct nvkm_mmu_pt *pt, u32 ptei, u32 ptes) | |
| 57 | +{ | |
| 58 | + bool dma = false; | |
| 59 | + nvkm_kmap(pt->memory); | |
| 60 | + while (ptes--) { | |
| 61 | + u32 datalo = nvkm_ro32(pt->memory, pt->base + ptei * 8 + 0); | |
| 62 | + u32 datahi = nvkm_ro32(pt->memory, pt->base + ptei * 8 + 4); | |
| 63 | + u64 data = (u64)datahi << 32 | datalo; | |
| 64 | + if ((data & BIT_ULL(0)) && (data & (3ULL << 1)) != 0) { | |
| 65 | + VMM_WO064(pt, vmm, ptei * 8, data & ~BIT_ULL(0)); | |
| 66 | + dma = true; | |
| 67 | + } | |
| 68 | + ptei++; | |
| 69 | + } | |
| 70 | + nvkm_done(pt->memory); | |
| 71 | + return dma; | |
| 72 | +} | |
| 73 | + | |
| 74 | +static void | |
| 75 | +gp100_vmm_pgt_pfn(struct nvkm_vmm *vmm, struct nvkm_mmu_pt *pt, | |
| 76 | + u32 ptei, u32 ptes, struct nvkm_vmm_map *map) | |
| 77 | +{ | |
| 78 | + struct device *dev = vmm->mmu->subdev.device->dev; | |
| 79 | + dma_addr_t addr; | |
| 80 | + | |
| 81 | + nvkm_kmap(pt->memory); | |
| 82 | + while (ptes--) { | |
| 83 | + u64 data = 0; | |
| 84 | + if (!(*map->pfn & NVKM_VMM_PFN_W)) | |
| 85 | + data |= BIT_ULL(6); /* RO. */ | |
| 86 | + | |
| 87 | + if (!(*map->pfn & NVKM_VMM_PFN_VRAM)) { | |
| 88 | + addr = *map->pfn >> NVKM_VMM_PFN_ADDR_SHIFT; | |
| 89 | + addr = dma_map_page(dev, pfn_to_page(addr), 0, | |
| 90 | + PAGE_SIZE, DMA_BIDIRECTIONAL); | |
| 91 | + if (!WARN_ON(dma_mapping_error(dev, addr))) { | |
| 92 | + data |= addr >> 4; | |
| 93 | + data |= 2ULL << 1; /* SYSTEM_COHERENT_MEMORY. */ | |
| 94 | + data |= BIT_ULL(3); /* VOL. */ | |
| 95 | + data |= BIT_ULL(0); /* VALID. */ | |
| 96 | + } | |
| 97 | + } else { | |
| 98 | + data |= (*map->pfn & NVKM_VMM_PFN_ADDR) >> 4; | |
| 99 | + data |= BIT_ULL(0); /* VALID. */ | |
| 100 | + } | |
| 101 | + | |
| 102 | + VMM_WO064(pt, vmm, ptei++ * 8, data); | |
| 103 | + map->pfn++; | |
| 104 | + } | |
| 105 | + nvkm_done(pt->memory); | |
| 106 | +} | |
| 107 | + | |
| 30 | 108 | static inline void |
| 31 | 109 | gp100_vmm_pgt_pte(struct nvkm_vmm *vmm, struct nvkm_mmu_pt *pt, |
| 32 | 110 | u32 ptei, u32 ptes, struct nvkm_vmm_map *map, u64 addr) |
| @@ -89,6 +167,9 @@ gp100_vmm_desc_spt = { · 56 unmodified lines | ||
| 89 | 167 | .mem = gp100_vmm_pgt_mem, |
| 90 | 168 | .dma = gp100_vmm_pgt_dma, |
| 91 | 169 | .sgl = gp100_vmm_pgt_sgl, |
| 170 | + .pfn = gp100_vmm_pgt_pfn, | |
| 171 | + .pfn_clear = gp100_vmm_pfn_clear, | |
| 172 | + .pfn_unmap = gp100_vmm_pfn_unmap, | |
| 92 | 173 | }; |
| 93 | 174 | |
| 94 | 175 | static void |
| @@ -306,16 +387,100 @@ gp100_vmm_valid(struct nvkm_vmm *vmm, void *argv, u32 argc, · 211 unmodified lines | ||
| 306 | 387 | return 0; |
| 307 | 388 | } |
| 308 | 389 | |
| 390 | +static int | |
| 391 | +gp100_vmm_fault_cancel(struct nvkm_vmm *vmm, void *argv, u32 argc) | |
| 392 | +{ | |
| 393 | + struct nvkm_device *device = vmm->mmu->subdev.device; | |
| 394 | + union { | |
| 395 | + struct gp100_vmm_fault_cancel_v0 v0; | |
| 396 | + } *args = argv; | |
| 397 | + int ret = -ENOSYS; | |
| 398 | + u32 inst, aper; | |
| 399 | + | |
| 400 | + if ((ret = nvif_unpack(ret, &argv, &argc, args->v0, 0, 0, false))) | |
| 401 | + return ret; | |
| 402 | + | |
| 403 | + /* Translate MaxwellFaultBufferA instance pointer to the same | |
| 404 | + * format as the NV_GR_FECS_CURRENT_CTX register. | |
| 405 | + */ | |
| 406 | + aper = (args->v0.inst >> 8) & 3; | |
| 407 | + args->v0.inst >>= 12; | |
| 408 | + args->v0.inst |= aper << 28; | |
| 409 | + args->v0.inst |= 0x80000000; | |
| 410 | + | |
| 411 | + if (!WARN_ON(nvkm_gr_ctxsw_pause(device))) { | |
| 412 | + if ((inst = nvkm_gr_ctxsw_inst(device)) == args->v0.inst) { | |
| 413 | + gf100_vmm_invalidate(vmm, 0x0000001b | |
| 414 | + /* CANCEL_TARGETED. */ | | |
| 415 | + (args->v0.hub << 20) | | |
| 416 | + (args->v0.gpc << 15) | | |
| 417 | + (args->v0.client << 9)); | |
| 418 | + } | |
| 419 | + WARN_ON(nvkm_gr_ctxsw_resume(device)); | |
| 420 | + } | |
| 421 | + | |
| 422 | + return 0; | |
| 423 | +} | |
| 424 | + | |
| 425 | +static int | |
| 426 | +gp100_vmm_fault_replay(struct nvkm_vmm *vmm, void *argv, u32 argc) | |
| 427 | +{ | |
| 428 | + union { | |
| 429 | + struct gp100_vmm_fault_replay_vn vn; | |
| 430 | + } *args = argv; | |
| 431 | + int ret = -ENOSYS; | |
| 432 | + | |
| 433 | + if (!(ret = nvif_unvers(ret, &argv, &argc, args->vn))) { | |
| 434 | + gf100_vmm_invalidate(vmm, 0x0000000b); /* REPLAY_GLOBAL. */ | |
| 435 | + } | |
| 436 | + | |
| 437 | + return ret; | |
| 438 | +} | |
| 439 | + | |
| 440 | +int | |
| 441 | +gp100_vmm_mthd(struct nvkm_vmm *vmm, | |
| 442 | + struct nvkm_client *client, u32 mthd, void *argv, u32 argc) | |
| 443 | +{ | |
| 444 | + if (client->super) { | |
| 445 | + switch (mthd) { | |
| 446 | + case GP100_VMM_VN_FAULT_REPLAY: | |
| 447 | + return gp100_vmm_fault_replay(vmm, argv, argc); | |
| 448 | + case GP100_VMM_VN_FAULT_CANCEL: | |
| 449 | + return gp100_vmm_fault_cancel(vmm, argv, argc); | |
| 450 | + default: | |
| 451 | + break; | |
| 452 | + } | |
| 453 | + } | |
| 454 | + return -EINVAL; | |
| 455 | +} | |
| 456 | + | |
| 457 | +void | |
| 458 | +gp100_vmm_invalidate_pdb(struct nvkm_vmm *vmm, u64 addr) | |
| 459 | +{ | |
| 460 | + struct nvkm_device *device = vmm->mmu->subdev.device; | |
| 461 | + nvkm_wr32(device, 0x100cb8, lower_32_bits(addr)); | |
| 462 | + nvkm_wr32(device, 0x100cec, upper_32_bits(addr)); | |
| 463 | +} | |
| 464 | + | |
| 309 | 465 | void |
| 310 | 466 | gp100_vmm_flush(struct nvkm_vmm *vmm, int depth) |
| 311 | 467 | { |
| 312 | − gf100_vmm_flush_(vmm, 5 /* CACHE_LEVEL_UP_TO_PDE3 */ - depth); | |
| 468 | + u32 type = (5 /* CACHE_LEVEL_UP_TO_PDE3 */ - depth) << 24; | |
| 469 | + type = 0; /*XXX: need to confirm stuff works with depth enabled... */ | |
| 470 | + if (atomic_read(&vmm->engref[NVKM_SUBDEV_BAR])) | |
| 471 | + type |= 0x00000004; /* HUB_ONLY */ | |
| 472 | + type |= 0x00000001; /* PAGE_ALL */ | |
| 473 | + gf100_vmm_invalidate(vmm, type); | |
| 313 | 474 | } |
| 314 | 475 | |
| 315 | 476 | int |
| 316 | 477 | gp100_vmm_join(struct nvkm_vmm *vmm, struct nvkm_memory *inst) |
| 317 | 478 | { |
| 318 | − const u64 base = BIT_ULL(10) /* VER2 */ | BIT_ULL(11); /* 64KiB */ | |
| 479 | + u64 base = BIT_ULL(10) /* VER2 */ | BIT_ULL(11) /* 64KiB */; | |
| 480 | + if (vmm->replay) { | |
| 481 | + base |= BIT_ULL(4); /* FAULT_REPLAY_TEX */ | |
| 482 | + base |= BIT_ULL(5); /* FAULT_REPLAY_GCC */ | |
| 483 | + } | |
| 319 | 484 | return gf100_vmm_join_(vmm, inst, base); |
| 320 | 485 | } |
| 321 | 486 | |
| @@ -326,6 +491,8 @@ gp100_vmm = { · 4 unmodified lines | ||
| 326 | 491 | .aper = gf100_vmm_aper, |
| 327 | 492 | .valid = gp100_vmm_valid, |
| 328 | 493 | .flush = gp100_vmm_flush, |
| 494 | + .mthd = gp100_vmm_mthd, | |
| 495 | + .invalidate_pdb = gp100_vmm_invalidate_pdb, | |
| 329 | 496 | .page = { |
| 330 | 497 | { 47, &gp100_vmm_desc_16[4], NVKM_VMM_PAGE_Sxxx }, |
| 331 | 498 | { 38, &gp100_vmm_desc_16[3], NVKM_VMM_PAGE_Sxxx }, |
| @@ -338,10 +505,39 @@ gp100_vmm = { · 6 unmodified lines | ||
| 338 | 505 | }; |
| 339 | 506 | |
| 340 | 507 | int |
| 341 | −gp100_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | |
| 342 | − struct lock_class_key *key, const char *name, | |
| 343 | − struct nvkm_vmm **pvmm) | |
| 508 | +gp100_vmm_new_(const struct nvkm_vmm_func *func, | |
| 509 | + struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 510 | + void *argv, u32 argc, struct lock_class_key *key, | |
| 511 | + const char *name, struct nvkm_vmm **pvmm) | |
| 512 | +{ | |
| 513 | + union { | |
| 514 | + struct gp100_vmm_vn vn; | |
| 515 | + struct gp100_vmm_v0 v0; | |
| 516 | + } *args = argv; | |
| 517 | + int ret = -ENOSYS; | |
| 518 | + bool replay; | |
| 519 | + | |
| 520 | + if (!(ret = nvif_unpack(ret, &argv, &argc, args->v0, 0, 0, false))) { | |
| 521 | + replay = args->v0.fault_replay != 0; | |
| 522 | + } else | |
| 523 | + if (!(ret = nvif_unvers(ret, &argv, &argc, args->vn))) { | |
| 524 | + replay = false; | |
| 525 | + } else | |
| 526 | + return ret; | |
| 527 | + | |
| 528 | + ret = nvkm_vmm_new_(func, mmu, 0, managed, addr, size, key, name, pvmm); | |
| 529 | + if (ret) | |
| 530 | + return ret; | |
| 531 | + | |
| 532 | + (*pvmm)->replay = replay; | |
| 533 | + return 0; | |
| 534 | +} | |
| 535 | + | |
| 536 | +int | |
| 537 | +gp100_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 538 | + void *argv, u32 argc, struct lock_class_key *key, | |
| 539 | + const char *name, struct nvkm_vmm **pvmm) | |
| 344 | 540 | { |
| 345 | − return nv04_vmm_new_(&gp100_vmm, mmu, 0, addr, size, | |
| 346 | − argv, argc, key, name, pvmm); | |
| 541 | + return gp100_vmm_new_(&gp100_vmm, mmu, managed, addr, size, | |
| 542 | + argv, argc, key, name, pvmm); | |
| 347 | 543 | } |
| @@ -28,6 +28,8 @@ gp10b_vmm = { · 27 unmodified lines | ||
| 28 | 28 | .aper = gk20a_vmm_aper, |
| 29 | 29 | .valid = gp100_vmm_valid, |
| 30 | 30 | .flush = gp100_vmm_flush, |
| 31 | + .mthd = gp100_vmm_mthd, | |
| 32 | + .invalidate_pdb = gp100_vmm_invalidate_pdb, | |
| 31 | 33 | .page = { |
| 32 | 34 | { 47, &gp100_vmm_desc_16[4], NVKM_VMM_PAGE_Sxxx }, |
| 33 | 35 | { 38, &gp100_vmm_desc_16[3], NVKM_VMM_PAGE_Sxxx }, |
| @@ -40,10 +42,10 @@ gp10b_vmm = { · 6 unmodified lines | ||
| 40 | 42 | }; |
| 41 | 43 | |
| 42 | 44 | int |
| 43 | −gp10b_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | |
| 44 | − struct lock_class_key *key, const char *name, | |
| 45 | − struct nvkm_vmm **pvmm) | |
| 45 | +gp10b_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 46 | + void *argv, u32 argc, struct lock_class_key *key, | |
| 47 | + const char *name, struct nvkm_vmm **pvmm) | |
| 46 | 48 | { |
| 47 | − return nv04_vmm_new_(&gp10b_vmm, mmu, 0, addr, size, | |
| 48 | − argv, argc, key, name, pvmm); | |
| 49 | + return gp100_vmm_new_(&gp10b_vmm, mmu, managed, addr, size, | |
| 50 | + argv, argc, key, name, pvmm); | |
| 49 | 51 | } |
| @@ -66,6 +66,8 @@ gv100_vmm = { · 65 unmodified lines | ||
| 66 | 66 | .aper = gf100_vmm_aper, |
| 67 | 67 | .valid = gp100_vmm_valid, |
| 68 | 68 | .flush = gp100_vmm_flush, |
| 69 | + .mthd = gp100_vmm_mthd, | |
| 70 | + .invalidate_pdb = gp100_vmm_invalidate_pdb, | |
| 69 | 71 | .page = { |
| 70 | 72 | { 47, &gp100_vmm_desc_16[4], NVKM_VMM_PAGE_Sxxx }, |
| 71 | 73 | { 38, &gp100_vmm_desc_16[3], NVKM_VMM_PAGE_Sxxx }, |
| @@ -78,10 +80,10 @@ gv100_vmm = { · 6 unmodified lines | ||
| 78 | 80 | }; |
| 79 | 81 | |
| 80 | 82 | int |
| 81 | −gv100_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | |
| 82 | − struct lock_class_key *key, const char *name, | |
| 83 | − struct nvkm_vmm **pvmm) | |
| 83 | +gv100_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 84 | + void *argv, u32 argc, struct lock_class_key *key, | |
| 85 | + const char *name, struct nvkm_vmm **pvmm) | |
| 84 | 86 | { |
| 85 | − return nv04_vmm_new_(&gv100_vmm, mmu, 0, addr, size, | |
| 86 | − argv, argc, key, name, pvmm); | |
| 87 | + return gp100_vmm_new_(&gv100_vmm, mmu, managed, addr, size, | |
| 88 | + argv, argc, key, name, pvmm); | |
| 87 | 89 | } |
| @@ -36,10 +36,10 @@ mcp77_vmm = { · 35 unmodified lines | ||
| 36 | 36 | }; |
| 37 | 37 | |
| 38 | 38 | int |
| 39 | −mcp77_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | |
| 40 | − struct lock_class_key *key, const char *name, | |
| 41 | − struct nvkm_vmm **pvmm) | |
| 39 | +mcp77_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 40 | + void *argv, u32 argc, struct lock_class_key *key, | |
| 41 | + const char *name, struct nvkm_vmm **pvmm) | |
| 42 | 42 | { |
| 43 | − return nv04_vmm_new_(&mcp77_vmm, mmu, 0, addr, size, | |
| 43 | + return nv04_vmm_new_(&mcp77_vmm, mmu, 0, managed, addr, size, | |
| 44 | 44 | argv, argc, key, name, pvmm); |
| 45 | 45 | } |
| @@ -100,16 +100,17 @@ nv04_vmm = { · 99 unmodified lines | ||
| 100 | 100 | |
| 101 | 101 | int |
| 102 | 102 | nv04_vmm_new_(const struct nvkm_vmm_func *func, struct nvkm_mmu *mmu, |
| 103 | − u32 pd_header, u64 addr, u64 size, void *argv, u32 argc, | |
| 104 | − struct lock_class_key *key, const char *name, | |
| 105 | − struct nvkm_vmm **pvmm) | |
| 103 | + u32 pd_header, bool managed, u64 addr, u64 size, | |
| 104 | + void *argv, u32 argc, struct lock_class_key *key, | |
| 105 | + const char *name, struct nvkm_vmm **pvmm) | |
| 106 | 106 | { |
| 107 | 107 | union { |
| 108 | 108 | struct nv04_vmm_vn vn; |
| 109 | 109 | } *args = argv; |
| 110 | 110 | int ret; |
| 111 | 111 | |
| 112 | − ret = nvkm_vmm_new_(func, mmu, pd_header, addr, size, key, name, pvmm); | |
| 112 | + ret = nvkm_vmm_new_(func, mmu, pd_header, managed, addr, size, | |
| 113 | + key, name, pvmm); | |
| 113 | 114 | if (ret) |
| 114 | 115 | return ret; |
| 115 | 116 | |
| @@ -117,15 +118,15 @@ nv04_vmm_new_(const struct nvkm_vmm_func *func, struct nvkm_mmu *mmu, · 1 unmodified line | ||
| 117 | 118 | } |
| 118 | 119 | |
| 119 | 120 | int |
| 120 | −nv04_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | |
| 121 | − struct lock_class_key *key, const char *name, | |
| 121 | +nv04_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 122 | + void *argv, u32 argc, struct lock_class_key *key, const char *name, | |
| 122 | 123 | struct nvkm_vmm **pvmm) |
| 123 | 124 | { |
| 124 | 125 | struct nvkm_memory *mem; |
| 125 | 126 | struct nvkm_vmm *vmm; |
| 126 | 127 | int ret; |
| 127 | 128 | |
| 128 | − ret = nv04_vmm_new_(&nv04_vmm, mmu, 8, addr, size, | |
| 129 | + ret = nv04_vmm_new_(&nv04_vmm, mmu, 8, managed, addr, size, | |
| 129 | 130 | argv, argc, key, name, &vmm); |
| 130 | 131 | *pvmm = vmm; |
| 131 | 132 | if (ret) |
| @@ -104,10 +104,10 @@ nv41_vmm = { · 103 unmodified lines | ||
| 104 | 104 | }; |
| 105 | 105 | |
| 106 | 106 | int |
| 107 | −nv41_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | |
| 108 | − struct lock_class_key *key, const char *name, | |
| 107 | +nv41_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 108 | + void *argv, u32 argc, struct lock_class_key *key, const char *name, | |
| 109 | 109 | struct nvkm_vmm **pvmm) |
| 110 | 110 | { |
| 111 | − return nv04_vmm_new_(&nv41_vmm, mmu, 0, addr, size, | |
| 111 | + return nv04_vmm_new_(&nv41_vmm, mmu, 0, managed, addr, size, | |
| 112 | 112 | argv, argc, key, name, pvmm); |
| 113 | 113 | } |
| @@ -205,15 +205,15 @@ nv44_vmm = { · 204 unmodified lines | ||
| 205 | 205 | }; |
| 206 | 206 | |
| 207 | 207 | int |
| 208 | −nv44_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | |
| 209 | − struct lock_class_key *key, const char *name, | |
| 208 | +nv44_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 209 | + void *argv, u32 argc, struct lock_class_key *key, const char *name, | |
| 210 | 210 | struct nvkm_vmm **pvmm) |
| 211 | 211 | { |
| 212 | 212 | struct nvkm_subdev *subdev = &mmu->subdev; |
| 213 | 213 | struct nvkm_vmm *vmm; |
| 214 | 214 | int ret; |
| 215 | 215 | |
| 216 | − ret = nv04_vmm_new_(&nv44_vmm, mmu, 0, addr, size, | |
| 216 | + ret = nv04_vmm_new_(&nv44_vmm, mmu, 0, managed, addr, size, | |
| 217 | 217 | argv, argc, key, name, &vmm); |
| 218 | 218 | *pvmm = vmm; |
| 219 | 219 | if (ret) |
| @@ -376,10 +376,10 @@ nv50_vmm = { · 375 unmodified lines | ||
| 376 | 376 | }; |
| 377 | 377 | |
| 378 | 378 | int |
| 379 | −nv50_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | |
| 380 | − struct lock_class_key *key, const char *name, | |
| 379 | +nv50_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 380 | + void *argv, u32 argc, struct lock_class_key *key, const char *name, | |
| 381 | 381 | struct nvkm_vmm **pvmm) |
| 382 | 382 | { |
| 383 | − return nv04_vmm_new_(&nv50_vmm, mmu, 0, addr, size, | |
| 383 | + return nv04_vmm_new_(&nv50_vmm, mmu, 0, managed, addr, size, | |
| 384 | 384 | argv, argc, key, name, pvmm); |
| 385 | 385 | } |
| @@ -24,7 +24,7 @@ · 23 unmodified lines | ||
| 24 | 24 | #include <subdev/timer.h> |
| 25 | 25 | |
| 26 | 26 | static void |
| 27 | −tu104_vmm_flush(struct nvkm_vmm *vmm, int depth) | |
| 27 | +tu102_vmm_flush(struct nvkm_vmm *vmm, int depth) | |
| 28 | 28 | { |
| 29 | 29 | struct nvkm_subdev *subdev = &vmm->mmu->subdev; |
| 30 | 30 | struct nvkm_device *device = subdev->device; |
| @@ -50,12 +50,13 @@ tu104_vmm_flush(struct nvkm_vmm *vmm, int depth) · 19 unmodified lines | ||
| 50 | 50 | } |
| 51 | 51 | |
| 52 | 52 | static const struct nvkm_vmm_func |
| 53 | −tu104_vmm = { | |
| 53 | +tu102_vmm = { | |
| 54 | 54 | .join = gv100_vmm_join, |
| 55 | 55 | .part = gf100_vmm_part, |
| 56 | 56 | .aper = gf100_vmm_aper, |
| 57 | 57 | .valid = gp100_vmm_valid, |
| 58 | − .flush = tu104_vmm_flush, | |
| 58 | + .flush = tu102_vmm_flush, | |
| 59 | + .mthd = gp100_vmm_mthd, | |
| 59 | 60 | .page = { |
| 60 | 61 | { 47, &gp100_vmm_desc_16[4], NVKM_VMM_PAGE_Sxxx }, |
| 61 | 62 | { 38, &gp100_vmm_desc_16[3], NVKM_VMM_PAGE_Sxxx }, |
| @@ -68,10 +69,10 @@ tu104_vmm = { · 6 unmodified lines | ||
| 68 | 69 | }; |
| 69 | 70 | |
| 70 | 71 | int |
| 71 | −tu104_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, | |
| 72 | +tu102_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | |
| 72 | 73 | void *argv, u32 argc, struct lock_class_key *key, |
| 73 | 74 | const char *name, struct nvkm_vmm **pvmm) |
| 74 | 75 | { |
| 75 | − return nv04_vmm_new_(&tu104_vmm, mmu, 0, addr, size, | |
| 76 | − argv, argc, key, name, pvmm); | |
| 76 | + return gp100_vmm_new_(&tu102_vmm, mmu, managed, addr, size, | |
| 77 | + argv, argc, key, name, pvmm); | |
| 77 | 78 | } |
| @@ -88,10 +88,10 @@ nvkm_memx_fini(struct nvkm_memx **pmemx, bool exec) · 87 unmodified lines | ||
| 88 | 88 | if (exec) { |
| 89 | 89 | nvkm_pmu_send(pmu, reply, PROC_MEMX, MEMX_MSG_EXEC, |
| 90 | 90 | memx->base, finish); |
| 91 | + nvkm_debug(subdev, "Exec took %uns, PMU_IN %08x\n", | |
| 92 | + reply[0], reply[1]); | |
| 91 | 93 | } |
| 92 | 94 | |
| 93 | − nvkm_debug(subdev, "Exec took %uns, PMU_IN %08x\n", | |
| 94 | − reply[0], reply[1]); | |
| 95 | 95 | kfree(memx); |
| 96 | 96 | return 0; |
| 97 | 97 | } |
| @@ -853,7 +853,7 @@ acr_r352_shutdown(struct acr_r352 *acr, struct nvkm_secboot *sb) · 852 unmodified lines | ||
| 853 | 853 | * and the expected behavior on RM as well |
| 854 | 854 | */ |
| 855 | 855 | if (ret && ret != 0x1d) { |
| 856 | − nvkm_error(subdev, "HS unload failed, ret 0x%08x", ret); | |
| 856 | + nvkm_error(subdev, "HS unload failed, ret 0x%08x\n", ret); | |
| 857 | 857 | return -EINVAL; |
| 858 | 858 | } |
| 859 | 859 | nvkm_debug(subdev, "HS unload blob completed\n"); |
| @@ -922,7 +922,7 @@ acr_r352_bootstrap(struct acr_r352 *acr, struct nvkm_secboot *sb) · 62 unmodified lines | ||
| 922 | 922 | if (ret < 0) { |
| 923 | 923 | return ret; |
| 924 | 924 | } else if (ret > 0) { |
| 925 | − nvkm_error(subdev, "HS load failed, ret 0x%08x", ret); | |
| 925 | + nvkm_error(subdev, "HS load failed, ret 0x%08x\n", ret); | |
| 926 | 926 | return -EINVAL; |
| 927 | 927 | } |
| 928 | 928 | nvkm_debug(subdev, "HS load blob completed\n"); |
| @@ -40,6 +40,22 @@ nvkm_top_device_new(struct nvkm_top *top) · 39 unmodified lines | ||
| 40 | 40 | return info; |
| 41 | 41 | } |
| 42 | 42 | |
| 43 | +u32 | |
| 44 | +nvkm_top_addr(struct nvkm_device *device, enum nvkm_devidx index) | |
| 45 | +{ | |
| 46 | + struct nvkm_top *top = device->top; | |
| 47 | + struct nvkm_top_device *info; | |
| 48 | + | |
| 49 | + if (top) { | |
| 50 | + list_for_each_entry(info, &top->device, head) { | |
| 51 | + if (info->index == index) | |
| 52 | + return info->addr; | |
| 53 | + } | |
| 54 | + } | |
| 55 | + | |
| 56 | + return 0; | |
| 57 | +} | |
| 58 | + | |
| 43 | 59 | u32 |
| 44 | 60 | nvkm_top_reset(struct nvkm_device *device, enum nvkm_devidx index) |
| 45 | 61 | { |
| @@ -73,6 +73,7 @@ gk104_top_oneinit(struct nvkm_top *top) · 72 unmodified lines | ||
| 73 | 73 | #define A_(A) if (inst == 0) info->index = NVKM_ENGINE_##A |
| 74 | 74 | #define B_(A) if (inst + NVKM_ENGINE_##A##0 < NVKM_ENGINE_##A##_LAST + 1) \ |
| 75 | 75 | info->index = NVKM_ENGINE_##A##0 + inst |
| 76 | +#define C_(A) if (inst == 0) info->index = NVKM_SUBDEV_##A | |
| 76 | 77 | switch (type) { |
| 77 | 78 | case 0x00000000: A_(GR ); break; |
| 78 | 79 | case 0x00000001: A_(CE0 ); break; |
| @@ -88,6 +89,7 @@ gk104_top_oneinit(struct nvkm_top *top) · 9 unmodified lines | ||
| 88 | 89 | case 0x0000000f: A_(NVENC1); break; |
| 89 | 90 | case 0x00000010: B_(NVDEC ); break; |
| 90 | 91 | case 0x00000013: B_(CE ); break; |
| 92 | + case 0x00000014: C_(GSP ); break; | |
| 91 | 93 | break; |
| 92 | 94 | default: |
| 93 | 95 | break; |
| @@ -2,6 +2,7 @@ nvkm-y += nvkm/subdev/volt/base.o · 1 unmodified line | ||
| 2 | 2 | nvkm-y += nvkm/subdev/volt/gpio.o |
| 3 | 3 | nvkm-y += nvkm/subdev/volt/nv40.o |
| 4 | 4 | nvkm-y += nvkm/subdev/volt/gf100.o |
| 5 | +nvkm-y += nvkm/subdev/volt/gf117.o | |
| 5 | 6 | nvkm-y += nvkm/subdev/volt/gk104.o |
| 6 | 7 | nvkm-y += nvkm/subdev/volt/gk20a.o |
| 7 | 8 | nvkm-y += nvkm/subdev/volt/gm20b.o |
| @@ -0,0 +1,60 @@ | ||
| 1 | +/* | |
| 2 | + * Copyright 2019 Ilia Mirkin | |
| 3 | + * | |
| 4 | + * Permission is hereby granted, free of charge, to any person obtaining a | |
| 5 | + * copy of this software and associated documentation files (the "Software"), | |
| 6 | + * to deal in the Software without restriction, including without limitation | |
| 7 | + * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 8 | + * and/or sell copies of the Software, and to permit persons to whom the | |
| 9 | + * Software is furnished to do so, subject to the following conditions: | |
| 10 | + * | |
| 11 | + * The above copyright notice and this permission notice shall be included in | |
| 12 | + * all copies or substantial portions of the Software. | |
| 13 | + * | |
| 14 | + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 15 | + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 16 | + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 17 | + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 18 | + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 19 | + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 20 | + * OTHER DEALINGS IN THE SOFTWARE. | |
| 21 | + * | |
| 22 | + * Authors: Ilia Mirkin | |
| 23 | + */ | |
| 24 | +#include "priv.h" | |
| 25 | + | |
| 26 | +#include <subdev/fuse.h> | |
| 27 | + | |
| 28 | +static int | |
| 29 | +gf117_volt_speedo_read(struct nvkm_volt *volt) | |
| 30 | +{ | |
| 31 | + struct nvkm_device *device = volt->subdev.device; | |
| 32 | + struct nvkm_fuse *fuse = device->fuse; | |
| 33 | + | |
| 34 | + if (!fuse) | |
| 35 | + return -EINVAL; | |
| 36 | + | |
| 37 | + return nvkm_fuse_read(fuse, 0x3a8); | |
| 38 | +} | |
| 39 | + | |
| 40 | +static const struct nvkm_volt_func | |
| 41 | +gf117_volt = { | |
| 42 | + .oneinit = gf100_volt_oneinit, | |
| 43 | + .vid_get = nvkm_voltgpio_get, | |
| 44 | + .vid_set = nvkm_voltgpio_set, | |
| 45 | + .speedo_read = gf117_volt_speedo_read, | |
| 46 | +}; | |
| 47 | + | |
| 48 | +int | |
| 49 | +gf117_volt_new(struct nvkm_device *device, int index, struct nvkm_volt **pvolt) | |
| 50 | +{ | |
| 51 | + struct nvkm_volt *volt; | |
| 52 | + int ret; | |
| 53 | + | |
| 54 | + ret = nvkm_volt_new_(&gf117_volt, device, index, &volt); | |
| 55 | + *pvolt = volt; | |
| 56 | + if (ret) | |
| 57 | + return ret; | |
| 58 | + | |
| 59 | + return nvkm_voltgpio_init(volt); | |
| 60 | +} | |
| @@ -4869,10 +4869,12 @@ static void ci_request_link_speed_change_before_state_change(struct radeon_devic · 4868 unmodified lines | ||
| 4869 | 4869 | pi->force_pcie_gen = RADEON_PCIE_GEN2; |
| 4870 | 4870 | if (current_link_speed == RADEON_PCIE_GEN2) |
| 4871 | 4871 | break; |
| 4872 | + /* fall through */ | |
| 4872 | 4873 | case RADEON_PCIE_GEN2: |
| 4873 | 4874 | if (radeon_acpi_pcie_performance_request(rdev, PCIE_PERF_REQ_PECI_GEN2, false) == 0) |
| 4874 | 4875 | break; |
| 4875 | 4876 | #endif |
| 4877 | + /* fall through */ | |
| 4876 | 4878 | default: |
| 4877 | 4879 | pi->force_pcie_gen = ci_get_current_pcie_speed(rdev); |
| 4878 | 4880 | break; |
| @@ -1299,6 +1299,7 @@ static int evergreen_cs_handle_reg(struct radeon_cs_parser *p, u32 reg, u32 idx) · 1298 unmodified lines | ||
| 1299 | 1299 | return -EINVAL; |
| 1300 | 1300 | } |
| 1301 | 1301 | ib[idx] += (u32)((reloc->gpu_offset >> 8) & 0xffffffff); |
| 1302 | + break; | |
| 1302 | 1303 | case CB_TARGET_MASK: |
| 1303 | 1304 | track->cb_target_mask = radeon_get_ib_value(p, idx); |
| 1304 | 1305 | track->cb_dirty = true; |
| @@ -172,6 +172,7 @@ int radeon_driver_load_kms(struct drm_device *dev, unsigned long flags) · 171 unmodified lines | ||
| 172 | 172 | } |
| 173 | 173 | |
| 174 | 174 | if (radeon_is_px(dev)) { |
| 175 | + dev_pm_set_driver_flags(dev->dev, DPM_FLAG_NEVER_SKIP); | |
| 175 | 176 | pm_runtime_use_autosuspend(dev->dev); |
| 176 | 177 | pm_runtime_set_autosuspend_delay(dev->dev, 5000); |
| 177 | 178 | pm_runtime_set_active(dev->dev); |
| @@ -5762,10 +5762,12 @@ static void si_request_link_speed_change_before_state_change(struct radeon_devic · 5761 unmodified lines | ||
| 5762 | 5762 | si_pi->force_pcie_gen = RADEON_PCIE_GEN2; |
| 5763 | 5763 | if (current_link_speed == RADEON_PCIE_GEN2) |
| 5764 | 5764 | break; |
| 5765 | + /* fall through */ | |
| 5765 | 5766 | case RADEON_PCIE_GEN2: |
| 5766 | 5767 | if (radeon_acpi_pcie_performance_request(rdev, PCIE_PERF_REQ_PECI_GEN2, false) == 0) |
| 5767 | 5768 | break; |
| 5768 | 5769 | #endif |
| 5770 | + /* fall through */ | |
| 5769 | 5771 | default: |
| 5770 | 5772 | si_pi->force_pcie_gen = si_get_current_pcie_speed(rdev); |
| 5771 | 5773 | break; |
| @@ -52,12 +52,12 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, · 51 unmodified lines | ||
| 52 | 52 | { |
| 53 | 53 | int i; |
| 54 | 54 | |
| 55 | − if (!(entity && rq_list && num_rq_list > 0 && rq_list[0])) | |
| 55 | + if (!(entity && rq_list && (num_rq_list == 0 || rq_list[0]))) | |
| 56 | 56 | return -EINVAL; |
| 57 | 57 | |
| 58 | 58 | memset(entity, 0, sizeof(struct drm_sched_entity)); |
| 59 | 59 | INIT_LIST_HEAD(&entity->list); |
| 60 | − entity->rq = rq_list[0]; | |
| 60 | + entity->rq = NULL; | |
| 61 | 61 | entity->guilty = guilty; |
| 62 | 62 | entity->num_rq_list = num_rq_list; |
| 63 | 63 | entity->rq_list = kcalloc(num_rq_list, sizeof(struct drm_sched_rq *), |
| @@ -67,6 +67,10 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, · 3 unmodified lines | ||
| 67 | 67 | |
| 68 | 68 | for (i = 0; i < num_rq_list; ++i) |
| 69 | 69 | entity->rq_list[i] = rq_list[i]; |
| 70 | + | |
| 71 | + if (num_rq_list) | |
| 72 | + entity->rq = rq_list[0]; | |
| 73 | + | |
| 70 | 74 | entity->last_scheduled = NULL; |
| 71 | 75 | |
| 72 | 76 | spin_lock_init(&entity->rq_lock); |
| @@ -165,6 +169,9 @@ long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout) · 92 unmodified lines | ||
| 165 | 169 | struct task_struct *last_user; |
| 166 | 170 | long ret = timeout; |
| 167 | 171 | |
| 172 | + if (!entity->rq) | |
| 173 | + return 0; | |
| 174 | + | |
| 168 | 175 | sched = entity->rq->sched; |
| 169 | 176 | /** |
| 170 | 177 | * The client will not queue more IBs during this fini, consume existing |
| @@ -264,20 +271,24 @@ static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity) · 93 unmodified lines | ||
| 264 | 271 | */ |
| 265 | 272 | void drm_sched_entity_fini(struct drm_sched_entity *entity) |
| 266 | 273 | { |
| 267 | − struct drm_gpu_scheduler *sched; | |
| 274 | + struct drm_gpu_scheduler *sched = NULL; | |
| 268 | 275 | |
| 269 | − sched = entity->rq->sched; | |
| 270 | − drm_sched_rq_remove_entity(entity->rq, entity); | |
| 276 | + if (entity->rq) { | |
| 277 | + sched = entity->rq->sched; | |
| 278 | + drm_sched_rq_remove_entity(entity->rq, entity); | |
| 279 | + } | |
| 271 | 280 | |
| 272 | 281 | /* Consumption of existing IBs wasn't completed. Forcefully |
| 273 | 282 | * remove them here. |
| 274 | 283 | */ |
| 275 | 284 | if (spsc_queue_peek(&entity->job_queue)) { |
| 276 | − /* Park the kernel for a moment to make sure it isn't processing | |
| 277 | − * our enity. | |
| 278 | − */ | |
| 279 | − kthread_park(sched->thread); | |
| 280 | − kthread_unpark(sched->thread); | |
| 285 | + if (sched) { | |
| 286 | + /* Park the kernel for a moment to make sure it isn't processing | |
| 287 | + * our enity. | |
| 288 | + */ | |
| 289 | + kthread_park(sched->thread); | |
| 290 | + kthread_unpark(sched->thread); | |
| 291 | + } | |
| 281 | 292 | if (entity->dependency) { |
| 282 | 293 | dma_fence_remove_callback(entity->dependency, |
| 283 | 294 | &entity->cb); |
| @@ -362,9 +373,11 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity, · 78 unmodified lines | ||
| 362 | 373 | for (i = 0; i < entity->num_rq_list; ++i) |
| 363 | 374 | drm_sched_entity_set_rq_priority(&entity->rq_list[i], priority); |
| 364 | 375 | |
| 365 | − drm_sched_rq_remove_entity(entity->rq, entity); | |
| 366 | − drm_sched_entity_set_rq_priority(&entity->rq, priority); | |
| 367 | − drm_sched_rq_add_entity(entity->rq, entity); | |
| 376 | + if (entity->rq) { | |
| 377 | + drm_sched_rq_remove_entity(entity->rq, entity); | |
| 378 | + drm_sched_entity_set_rq_priority(&entity->rq, priority); | |
| 379 | + drm_sched_rq_add_entity(entity->rq, entity); | |
| 380 | + } | |
| 368 | 381 | |
| 369 | 382 | spin_unlock(&entity->rq_lock); |
| 370 | 383 | } |
| @@ -265,6 +265,12 @@ void ipu_pre_update(struct ipu_pre *pre, unsigned int bufaddr) · 264 unmodified lines | ||
| 265 | 265 | writel(IPU_PRE_CTRL_SDW_UPDATE, pre->regs + IPU_PRE_CTRL_SET); |
| 266 | 266 | } |
| 267 | 267 | |
| 268 | +bool ipu_pre_update_pending(struct ipu_pre *pre) | |
| 269 | +{ | |
| 270 | + return !!(readl_relaxed(pre->regs + IPU_PRE_CTRL) & | |
| 271 | + IPU_PRE_CTRL_SDW_UPDATE); | |
| 272 | +} | |
| 273 | + | |
| 268 | 274 | u32 ipu_pre_get_baddr(struct ipu_pre *pre) |
| 269 | 275 | { |
| 270 | 276 | return (u32)pre->buffer_paddr; |
| @@ -347,6 +347,22 @@ int ipu_prg_channel_configure(struct ipuv3_channel *ipu_chan, · 346 unmodified lines | ||
| 347 | 347 | } |
| 348 | 348 | EXPORT_SYMBOL_GPL(ipu_prg_channel_configure); |
| 349 | 349 | |
| 350 | +bool ipu_prg_channel_configure_pending(struct ipuv3_channel *ipu_chan) | |
| 351 | +{ | |
| 352 | + int prg_chan = ipu_prg_ipu_to_prg_chan(ipu_chan->num); | |
| 353 | + struct ipu_prg *prg = ipu_chan->ipu->prg_priv; | |
| 354 | + struct ipu_prg_channel *chan; | |
| 355 | + | |
| 356 | + if (prg_chan < 0) | |
| 357 | + return false; | |
| 358 | + | |
| 359 | + chan = &prg->chan[prg_chan]; | |
| 360 | + WARN_ON(!chan->enabled); | |
| 361 | + | |
| 362 | + return ipu_pre_update_pending(prg->pres[chan->used_pre]); | |
| 363 | +} | |
| 364 | +EXPORT_SYMBOL_GPL(ipu_prg_channel_configure_pending); | |
| 365 | + | |
| 350 | 366 | static int ipu_prg_probe(struct platform_device *pdev) |
| 351 | 367 | { |
| 352 | 368 | struct device *dev = &pdev->dev; |
| @@ -272,6 +272,7 @@ void ipu_pre_configure(struct ipu_pre *pre, unsigned int width, · 271 unmodified lines | ||
| 272 | 272 | unsigned int height, unsigned int stride, u32 format, |
| 273 | 273 | uint64_t modifier, unsigned int bufaddr); |
| 274 | 274 | void ipu_pre_update(struct ipu_pre *pre, unsigned int bufaddr); |
| 275 | +bool ipu_pre_update_pending(struct ipu_pre *pre); | |
| 275 | 276 | |
| 276 | 277 | struct ipu_prg *ipu_prg_lookup_by_phandle(struct device *dev, const char *name, |
| 277 | 278 | int ipu_id); |
| @@ -783,6 +783,7 @@ void c4iw_init_dev_ucontext(struct c4iw_rdev *rdev, · 782 unmodified lines | ||
| 783 | 783 | static int c4iw_rdev_open(struct c4iw_rdev *rdev) |
| 784 | 784 | { |
| 785 | 785 | int err; |
| 786 | + unsigned int factor; | |
| 786 | 787 | |
| 787 | 788 | c4iw_init_dev_ucontext(rdev, &rdev->uctx); |
| 788 | 789 | |
| @@ -806,8 +807,18 @@ static int c4iw_rdev_open(struct c4iw_rdev *rdev) · 17 unmodified lines | ||
| 806 | 807 | return -EINVAL; |
| 807 | 808 | } |
| 808 | 809 | |
| 809 | − rdev->qpmask = rdev->lldi.udb_density - 1; | |
| 810 | − rdev->cqmask = rdev->lldi.ucq_density - 1; | |
| 810 | + /* This implementation requires a sge_host_page_size <= PAGE_SIZE. */ | |
| 811 | + if (rdev->lldi.sge_host_page_size > PAGE_SIZE) { | |
| 812 | + pr_err("%s: unsupported sge host page size %u\n", | |
| 813 | + pci_name(rdev->lldi.pdev), | |
| 814 | + rdev->lldi.sge_host_page_size); | |
| 815 | + return -EINVAL; | |
| 816 | + } | |
| 817 | + | |
| 818 | + factor = PAGE_SIZE / rdev->lldi.sge_host_page_size; | |
| 819 | + rdev->qpmask = (rdev->lldi.udb_density * factor) - 1; | |
| 820 | + rdev->cqmask = (rdev->lldi.ucq_density * factor) - 1; | |
| 821 | + | |
| 811 | 822 | pr_debug("dev %s stag start 0x%0x size 0x%0x num stags %d pbl start 0x%0x size 0x%0x rq start 0x%0x size 0x%0x qp qid start %u size %u cq qid start %u size %u srq size %u\n", |
| 812 | 823 | pci_name(rdev->lldi.pdev), rdev->lldi.vr->stag.start, |
| 813 | 824 | rdev->lldi.vr->stag.size, c4iw_num_stags(rdev), |
| @@ -3032,7 +3032,6 @@ static int srp_reset_device(struct scsi_cmnd *scmnd) · 3031 unmodified lines | ||
| 3032 | 3032 | { |
| 3033 | 3033 | struct srp_target_port *target = host_to_target(scmnd->device->host); |
| 3034 | 3034 | struct srp_rdma_ch *ch; |
| 3035 | − int i, j; | |
| 3036 | 3035 | u8 status; |
| 3037 | 3036 | |
| 3038 | 3037 | shost_printk(KERN_ERR, target->scsi_host, "SRP reset_device called\n"); |
| @@ -3044,15 +3043,6 @@ static int srp_reset_device(struct scsi_cmnd *scmnd) · 5 unmodified lines | ||
| 3044 | 3043 | if (status) |
| 3045 | 3044 | return FAILED; |
| 3046 | 3045 | |
| 3047 | − for (i = 0; i < target->ch_count; i++) { | |
| 3048 | − ch = &target->ch[i]; | |
| 3049 | − for (j = 0; j < target->req_ring_size; ++j) { | |
| 3050 | − struct srp_request *req = &ch->req_ring[j]; | |
| 3051 | − | |
| 3052 | − srp_finish_req(ch, req, scmnd->device, DID_RESET << 16); | |
| 3053 | − } | |
| 3054 | − } | |
| 3055 | − | |
| 3056 | 3046 | return SUCCESS; |
| 3057 | 3047 | } |
| 3058 | 3048 | |
| @@ -144,7 +144,7 @@ dmar_alloc_pci_notify_info(struct pci_dev *dev, unsigned long event) · 143 unmodified lines | ||
| 144 | 144 | for (tmp = dev; tmp; tmp = tmp->bus->self) |
| 145 | 145 | level++; |
| 146 | 146 | |
| 147 | − size = sizeof(*info) + level * sizeof(struct acpi_dmar_pci_path); | |
| 147 | + size = sizeof(*info) + level * sizeof(info->path[0]); | |
| 148 | 148 | if (size <= sizeof(dmar_pci_notify_info_buf)) { |
| 149 | 149 | info = (struct dmar_pci_notify_info *)dmar_pci_notify_info_buf; |
| 150 | 150 | } else { |
| @@ -1396,9 +1396,9 @@ static void flexrm_shutdown(struct mbox_chan *chan) · 1395 unmodified lines | ||
| 1396 | 1396 | |
| 1397 | 1397 | /* Clear ring flush state */ |
| 1398 | 1398 | timeout = 1000; /* timeout of 1s */ |
| 1399 | − writel_relaxed(0x0, ring + RING_CONTROL); | |
| 1399 | + writel_relaxed(0x0, ring->regs + RING_CONTROL); | |
| 1400 | 1400 | do { |
| 1401 | − if (!(readl_relaxed(ring + RING_FLUSH_DONE) & | |
| 1401 | + if (!(readl_relaxed(ring->regs + RING_FLUSH_DONE) & | |
| 1402 | 1402 | FLUSH_DONE_MASK)) |
| 1403 | 1403 | break; |
| 1404 | 1404 | mdelay(1); |
| @@ -310,6 +310,7 @@ int mbox_flush(struct mbox_chan *chan, unsigned long timeout) · 309 unmodified lines | ||
| 310 | 310 | |
| 311 | 311 | return ret; |
| 312 | 312 | } |
| 313 | +EXPORT_SYMBOL_GPL(mbox_flush); | |
| 313 | 314 | |
| 314 | 315 | /** |
| 315 | 316 | * mbox_request_channel - Request a mailbox channel. |
| @@ -2380,12 +2380,6 @@ static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card, · 2379 unmodified lines | ||
| 2380 | 2380 | snprintf(md->disk->disk_name, sizeof(md->disk->disk_name), |
| 2381 | 2381 | "mmcblk%u%s", card->host->index, subname ? subname : ""); |
| 2382 | 2382 | |
| 2383 | − if (mmc_card_mmc(card)) | |
| 2384 | − blk_queue_logical_block_size(md->queue.queue, | |
| 2385 | − card->ext_csd.data_sector_size); | |
| 2386 | − else | |
| 2387 | − blk_queue_logical_block_size(md->queue.queue, 512); | |
| 2388 | − | |
| 2389 | 2383 | set_capacity(md->disk, size); |
| 2390 | 2384 | |
| 2391 | 2385 | if (mmc_host_cmd23(card->host)) { |
| @@ -95,7 +95,7 @@ static void mmc_should_fail_request(struct mmc_host *host, · 94 unmodified lines | ||
| 95 | 95 | if (!data) |
| 96 | 96 | return; |
| 97 | 97 | |
| 98 | − if (cmd->error || data->error || | |
| 98 | + if ((cmd && cmd->error) || data->error || | |
| 99 | 99 | !should_fail(&host->fail_mmc_request, data->blksz * data->blocks)) |
| 100 | 100 | return; |
| 101 | 101 | |
| @@ -355,6 +355,7 @@ static void mmc_setup_queue(struct mmc_queue *mq, struct mmc_card *card) · 354 unmodified lines | ||
| 355 | 355 | { |
| 356 | 356 | struct mmc_host *host = card->host; |
| 357 | 357 | u64 limit = BLK_BOUNCE_HIGH; |
| 358 | + unsigned block_size = 512; | |
| 358 | 359 | |
| 359 | 360 | if (mmc_dev(host)->dma_mask && *mmc_dev(host)->dma_mask) |
| 360 | 361 | limit = (u64)dma_max_pfn(mmc_dev(host)) << PAGE_SHIFT; |
| @@ -368,7 +369,13 @@ static void mmc_setup_queue(struct mmc_queue *mq, struct mmc_card *card) · 7 unmodified lines | ||
| 368 | 369 | blk_queue_max_hw_sectors(mq->queue, |
| 369 | 370 | min(host->max_blk_count, host->max_req_size / 512)); |
| 370 | 371 | blk_queue_max_segments(mq->queue, host->max_segs); |
| 371 | − blk_queue_max_segment_size(mq->queue, host->max_seg_size); | |
| 372 | + | |
| 373 | + if (mmc_card_mmc(card)) | |
| 374 | + block_size = card->ext_csd.data_sector_size; | |
| 375 | + | |
| 376 | + blk_queue_logical_block_size(mq->queue, block_size); | |
| 377 | + blk_queue_max_segment_size(mq->queue, | |
| 378 | + round_down(host->max_seg_size, block_size)); | |
| 372 | 379 | |
| 373 | 380 | INIT_WORK(&mq->recovery_work, mmc_mq_recovery_handler); |
| 374 | 381 | INIT_WORK(&mq->complete_work, mmc_blk_mq_complete_work); |
| @@ -201,7 +201,7 @@ static int cqhci_host_alloc_tdl(struct cqhci_host *cq_host) · 200 unmodified lines | ||
| 201 | 201 | cq_host->desc_size = cq_host->slot_sz * cq_host->num_slots; |
| 202 | 202 | |
| 203 | 203 | cq_host->data_size = cq_host->trans_desc_len * cq_host->mmc->max_segs * |
| 204 | − (cq_host->num_slots - 1); | |
| 204 | + cq_host->mmc->cqe_qdepth; | |
| 205 | 205 | |
| 206 | 206 | pr_debug("%s: cqhci: desc_size: %zu data_sz: %zu slot-sz: %d\n", |
| 207 | 207 | mmc_hostname(cq_host->mmc), cq_host->desc_size, cq_host->data_size, |
| @@ -217,12 +217,21 @@ static int cqhci_host_alloc_tdl(struct cqhci_host *cq_host) · 9 unmodified lines | ||
| 217 | 217 | cq_host->desc_size, |
| 218 | 218 | &cq_host->desc_dma_base, |
| 219 | 219 | GFP_KERNEL); |
| 220 | + if (!cq_host->desc_base) | |
| 221 | + return -ENOMEM; | |
| 222 | + | |
| 220 | 223 | cq_host->trans_desc_base = dmam_alloc_coherent(mmc_dev(cq_host->mmc), |
| 221 | 224 | cq_host->data_size, |
| 222 | 225 | &cq_host->trans_desc_dma_base, |
| 223 | 226 | GFP_KERNEL); |
| 224 | − if (!cq_host->desc_base || !cq_host->trans_desc_base) | |
| 227 | + if (!cq_host->trans_desc_base) { | |
| 228 | + dmam_free_coherent(mmc_dev(cq_host->mmc), cq_host->desc_size, | |
| 229 | + cq_host->desc_base, | |
| 230 | + cq_host->desc_dma_base); | |
| 231 | + cq_host->desc_base = NULL; | |
| 232 | + cq_host->desc_dma_base = 0; | |
| 225 | 233 | return -ENOMEM; |
| 234 | + } | |
| 226 | 235 | |
| 227 | 236 | pr_debug("%s: cqhci: desc-base: 0x%p trans-base: 0x%p\n desc_dma 0x%llx trans_dma: 0x%llx\n", |
| 228 | 237 | mmc_hostname(cq_host->mmc), cq_host->desc_base, cq_host->trans_desc_base, |
| @@ -1450,6 +1450,7 @@ static int mmc_spi_probe(struct spi_device *spi) · 1449 unmodified lines | ||
| 1450 | 1450 | mmc->caps &= ~MMC_CAP_NEEDS_POLL; |
| 1451 | 1451 | mmc_gpiod_request_cd_irq(mmc); |
| 1452 | 1452 | } |
| 1453 | + mmc_detect_change(mmc, 0); | |
| 1453 | 1454 | |
| 1454 | 1455 | /* Index 1 is write protect/read only */ |
| 1455 | 1456 | status = mmc_gpiod_request_ro(mmc, NULL, 1, false, 0, NULL); |
| @@ -65,6 +65,7 @@ static const struct renesas_sdhi_of_data of_rcar_gen2_compatible = { · 64 unmodified lines | ||
| 65 | 65 | .scc_offset = 0x0300, |
| 66 | 66 | .taps = rcar_gen2_scc_taps, |
| 67 | 67 | .taps_num = ARRAY_SIZE(rcar_gen2_scc_taps), |
| 68 | + .max_blk_count = 0xffffffff, | |
| 68 | 69 | }; |
| 69 | 70 | |
| 70 | 71 | /* Definitions for sampling clocks */ |
| @@ -1095,11 +1095,12 @@ static void sdhci_esdhc_imx_hwinit(struct sdhci_host *host) · 1094 unmodified lines | ||
| 1095 | 1095 | writel(readl(host->ioaddr + SDHCI_HOST_CONTROL) |
| 1096 | 1096 | | ESDHC_BURST_LEN_EN_INCR, |
| 1097 | 1097 | host->ioaddr + SDHCI_HOST_CONTROL); |
| 1098 | + | |
| 1098 | 1099 | /* |
| 1099 | − * erratum ESDHC_FLAG_ERR004536 fix for MX6Q TO1.2 and MX6DL | |
| 1100 | − * TO1.1, it's harmless for MX6SL | |
| 1101 | − */ | |
| 1102 | − writel(readl(host->ioaddr + 0x6c) | BIT(7), | |
| 1100 | + * erratum ESDHC_FLAG_ERR004536 fix for MX6Q TO1.2 and MX6DL | |
| 1101 | + * TO1.1, it's harmless for MX6SL | |
| 1102 | + */ | |
| 1103 | + writel(readl(host->ioaddr + 0x6c) & ~BIT(7), | |
| 1103 | 1104 | host->ioaddr + 0x6c); |
| 1104 | 1105 | |
| 1105 | 1106 | /* disable DLL_CTRL delay line settings */ |
| @@ -277,6 +277,11 @@ static inline void sd_ctrl_write32_as_16_and_16(struct tmio_mmc_host *host, · 276 unmodified lines | ||
| 277 | 277 | iowrite16(val >> 16, host->ctl + ((addr + 2) << host->bus_shift)); |
| 278 | 278 | } |
| 279 | 279 | |
| 280 | +static inline void sd_ctrl_write32(struct tmio_mmc_host *host, int addr, u32 val) | |
| 281 | +{ | |
| 282 | + iowrite32(val, host->ctl + (addr << host->bus_shift)); | |
| 283 | +} | |
| 284 | + | |
| 280 | 285 | static inline void sd_ctrl_write32_rep(struct tmio_mmc_host *host, int addr, |
| 281 | 286 | const u32 *buf, int count) |
| 282 | 287 | { |
| @@ -43,6 +43,7 @@ · 42 unmodified lines | ||
| 43 | 43 | #include <linux/regulator/consumer.h> |
| 44 | 44 | #include <linux/mmc/sdio.h> |
| 45 | 45 | #include <linux/scatterlist.h> |
| 46 | +#include <linux/sizes.h> | |
| 46 | 47 | #include <linux/spinlock.h> |
| 47 | 48 | #include <linux/swiotlb.h> |
| 48 | 49 | #include <linux/workqueue.h> |
| @@ -629,7 +630,7 @@ static bool __tmio_mmc_sdcard_irq(struct tmio_mmc_host *host, int ireg, · 580 unmodified lines | ||
| 629 | 630 | return false; |
| 630 | 631 | } |
| 631 | 632 | |
| 632 | −static void __tmio_mmc_sdio_irq(struct tmio_mmc_host *host) | |
| 633 | +static bool __tmio_mmc_sdio_irq(struct tmio_mmc_host *host) | |
| 633 | 634 | { |
| 634 | 635 | struct mmc_host *mmc = host->mmc; |
| 635 | 636 | struct tmio_mmc_data *pdata = host->pdata; |
| @@ -637,7 +638,7 @@ static void __tmio_mmc_sdio_irq(struct tmio_mmc_host *host) · 1 unmodified line | ||
| 637 | 638 | unsigned int sdio_status; |
| 638 | 639 | |
| 639 | 640 | if (!(pdata->flags & TMIO_MMC_SDIO_IRQ)) |
| 640 | − return; | |
| 641 | + return false; | |
| 641 | 642 | |
| 642 | 643 | status = sd_ctrl_read16(host, CTL_SDIO_STATUS); |
| 643 | 644 | ireg = status & TMIO_SDIO_MASK_ALL & ~host->sdio_irq_mask; |
| @@ -650,6 +651,8 @@ static void __tmio_mmc_sdio_irq(struct tmio_mmc_host *host) · 6 unmodified lines | ||
| 650 | 651 | |
| 651 | 652 | if (mmc->caps & MMC_CAP_SDIO_IRQ && ireg & TMIO_SDIO_STAT_IOIRQ) |
| 652 | 653 | mmc_signal_sdio_irq(mmc); |
| 654 | + | |
| 655 | + return ireg; | |
| 653 | 656 | } |
| 654 | 657 | |
| 655 | 658 | irqreturn_t tmio_mmc_irq(int irq, void *devid) |
| @@ -668,9 +671,10 @@ irqreturn_t tmio_mmc_irq(int irq, void *devid) · 12 unmodified lines | ||
| 668 | 671 | if (__tmio_mmc_sdcard_irq(host, ireg, status)) |
| 669 | 672 | return IRQ_HANDLED; |
| 670 | 673 | |
| 671 | − __tmio_mmc_sdio_irq(host); | |
| 674 | + if (__tmio_mmc_sdio_irq(host)) | |
| 675 | + return IRQ_HANDLED; | |
| 672 | 676 | |
| 673 | − return IRQ_HANDLED; | |
| 677 | + return IRQ_NONE; | |
| 674 | 678 | } |
| 675 | 679 | EXPORT_SYMBOL_GPL(tmio_mmc_irq); |
| 676 | 680 | |
| @@ -700,7 +704,10 @@ static int tmio_mmc_start_data(struct tmio_mmc_host *host, · 23 unmodified lines | ||
| 700 | 704 | |
| 701 | 705 | /* Set transfer length / blocksize */ |
| 702 | 706 | sd_ctrl_write16(host, CTL_SD_XFER_LEN, data->blksz); |
| 703 | − sd_ctrl_write16(host, CTL_XFER_BLK_COUNT, data->blocks); | |
| 707 | + if (host->mmc->max_blk_count >= SZ_64K) | |
| 708 | + sd_ctrl_write32(host, CTL_XFER_BLK_COUNT, data->blocks); | |
| 709 | + else | |
| 710 | + sd_ctrl_write16(host, CTL_XFER_BLK_COUNT, data->blocks); | |
| 704 | 711 | |
| 705 | 712 | tmio_mmc_start_dma(host, data); |
| 706 | 713 | |
| @@ -212,7 +212,7 @@ static int powernv_flash_set_driver_info(struct device *dev, · 211 unmodified lines | ||
| 212 | 212 | * Going to have to check what details I need to set and how to |
| 213 | 213 | * get them |
| 214 | 214 | */ |
| 215 | − mtd->name = devm_kasprintf(dev, GFP_KERNEL, "%pOFn", dev->of_node); | |
| 215 | + mtd->name = devm_kasprintf(dev, GFP_KERNEL, "%pOFP", dev->of_node); | |
| 216 | 216 | mtd->type = MTD_NORFLASH; |
| 217 | 217 | mtd->flags = MTD_WRITEABLE; |
| 218 | 218 | mtd->size = size; |
| @@ -507,6 +507,7 @@ static int mtd_nvmem_add(struct mtd_info *mtd) · 506 unmodified lines | ||
| 507 | 507 | { |
| 508 | 508 | struct nvmem_config config = {}; |
| 509 | 509 | |
| 510 | + config.id = -1; | |
| 510 | 511 | config.dev = &mtd->dev; |
| 511 | 512 | config.name = mtd->name; |
| 512 | 513 | config.owner = THIS_MODULE; |
| @@ -1183,29 +1183,22 @@ static rx_handler_result_t bond_handle_frame(struct sk_buff **pskb) · 1182 unmodified lines | ||
| 1183 | 1183 | } |
| 1184 | 1184 | } |
| 1185 | 1185 | |
| 1186 | − /* Link-local multicast packets should be passed to the | |
| 1187 | − * stack on the link they arrive as well as pass them to the | |
| 1188 | − * bond-master device. These packets are mostly usable when | |
| 1189 | − * stack receives it with the link on which they arrive | |
| 1190 | − * (e.g. LLDP) they also must be available on master. Some of | |
| 1191 | − * the use cases include (but are not limited to): LLDP agents | |
| 1192 | − * that must be able to operate both on enslaved interfaces as | |
| 1193 | − * well as on bonds themselves; linux bridges that must be able | |
| 1194 | − * to process/pass BPDUs from attached bonds when any kind of | |
| 1195 | − * STP version is enabled on the network. | |
| 1186 | + /* | |
| 1187 | + * For packets determined by bond_should_deliver_exact_match() call to | |
| 1188 | + * be suppressed we want to make an exception for link-local packets. | |
| 1189 | + * This is necessary for e.g. LLDP daemons to be able to monitor | |
| 1190 | + * inactive slave links without being forced to bind to them | |
| 1191 | + * explicitly. | |
| 1192 | + * | |
| 1193 | + * At the same time, packets that are passed to the bonding master | |
| 1194 | + * (including link-local ones) can have their originating interface | |
| 1195 | + * determined via PACKET_ORIGDEV socket option. | |
| 1196 | 1196 | */ |
| 1197 | − if (is_link_local_ether_addr(eth_hdr(skb)->h_dest)) { | |
| 1198 | − struct sk_buff *nskb = skb_clone(skb, GFP_ATOMIC); | |
| 1199 | − | |
| 1200 | − if (nskb) { | |
| 1201 | − nskb->dev = bond->dev; | |
| 1202 | − nskb->queue_mapping = 0; | |
| 1203 | − netif_rx(nskb); | |
| 1204 | − } | |
| 1205 | − return RX_HANDLER_PASS; | |
| 1206 | − } | |
| 1207 | − if (bond_should_deliver_exact_match(skb, slave, bond)) | |
| 1197 | + if (bond_should_deliver_exact_match(skb, slave, bond)) { | |
| 1198 | + if (is_link_local_ether_addr(eth_hdr(skb)->h_dest)) | |
| 1199 | + return RX_HANDLER_PASS; | |
| 1208 | 1200 | return RX_HANDLER_EXACT; |
| 1201 | + } | |
| 1209 | 1202 | |
| 1210 | 1203 | skb->dev = bond->dev; |
| 1211 | 1204 | |
| @@ -344,7 +344,8 @@ static void b53_set_forwarding(struct b53_device *dev, int enable) · 343 unmodified lines | ||
| 344 | 344 | b53_write8(dev, B53_CTRL_PAGE, B53_SWITCH_CTRL, mgmt); |
| 345 | 345 | } |
| 346 | 346 | |
| 347 | −static void b53_enable_vlan(struct b53_device *dev, bool enable) | |
| 347 | +static void b53_enable_vlan(struct b53_device *dev, bool enable, | |
| 348 | + bool enable_filtering) | |
| 348 | 349 | { |
| 349 | 350 | u8 mgmt, vc0, vc1, vc4 = 0, vc5; |
| 350 | 351 | |
| @@ -369,8 +370,13 @@ static void b53_enable_vlan(struct b53_device *dev, bool enable) · 18 unmodified lines | ||
| 369 | 370 | vc0 |= VC0_VLAN_EN | VC0_VID_CHK_EN | VC0_VID_HASH_VID; |
| 370 | 371 | vc1 |= VC1_RX_MCST_UNTAG_EN | VC1_RX_MCST_FWD_EN; |
| 371 | 372 | vc4 &= ~VC4_ING_VID_CHECK_MASK; |
| 372 | − vc4 |= VC4_ING_VID_VIO_DROP << VC4_ING_VID_CHECK_S; | |
| 373 | − vc5 |= VC5_DROP_VTABLE_MISS; | |
| 373 | + if (enable_filtering) { | |
| 374 | + vc4 |= VC4_ING_VID_VIO_DROP << VC4_ING_VID_CHECK_S; | |
| 375 | + vc5 |= VC5_DROP_VTABLE_MISS; | |
| 376 | + } else { | |
| 377 | + vc4 |= VC4_ING_VID_VIO_FWD << VC4_ING_VID_CHECK_S; | |
| 378 | + vc5 &= ~VC5_DROP_VTABLE_MISS; | |
| 379 | + } | |
| 374 | 380 | |
| 375 | 381 | if (is5325(dev)) |
| 376 | 382 | vc0 &= ~VC0_RESERVED_1; |
| @@ -420,6 +426,9 @@ static void b53_enable_vlan(struct b53_device *dev, bool enable) · 43 unmodified lines | ||
| 420 | 426 | } |
| 421 | 427 | |
| 422 | 428 | b53_write8(dev, B53_CTRL_PAGE, B53_SWITCH_MODE, mgmt); |
| 429 | + | |
| 430 | + dev->vlan_enabled = enable; | |
| 431 | + dev->vlan_filtering_enabled = enable_filtering; | |
| 423 | 432 | } |
| 424 | 433 | |
| 425 | 434 | static int b53_set_jumbo(struct b53_device *dev, bool enable, bool allow_10_100) |
| @@ -632,25 +641,35 @@ static void b53_enable_mib(struct b53_device *dev) · 206 unmodified lines | ||
| 632 | 641 | b53_write8(dev, B53_MGMT_PAGE, B53_GLOBAL_CONFIG, gc); |
| 633 | 642 | } |
| 634 | 643 | |
| 644 | +static u16 b53_default_pvid(struct b53_device *dev) | |
| 645 | +{ | |
| 646 | + if (is5325(dev) || is5365(dev)) | |
| 647 | + return 1; | |
| 648 | + else | |
| 649 | + return 0; | |
| 650 | +} | |
| 651 | + | |
| 635 | 652 | int b53_configure_vlan(struct dsa_switch *ds) |
| 636 | 653 | { |
| 637 | 654 | struct b53_device *dev = ds->priv; |
| 638 | 655 | struct b53_vlan vl = { 0 }; |
| 639 | − int i; | |
| 656 | + int i, def_vid; | |
| 657 | + | |
| 658 | + def_vid = b53_default_pvid(dev); | |
| 640 | 659 | |
| 641 | 660 | /* clear all vlan entries */ |
| 642 | 661 | if (is5325(dev) || is5365(dev)) { |
| 643 | − for (i = 1; i < dev->num_vlans; i++) | |
| 662 | + for (i = def_vid; i < dev->num_vlans; i++) | |
| 644 | 663 | b53_set_vlan_entry(dev, i, &vl); |
| 645 | 664 | } else { |
| 646 | 665 | b53_do_vlan_op(dev, VTA_CMD_CLEAR); |
| 647 | 666 | } |
| 648 | 667 | |
| 649 | − b53_enable_vlan(dev, false); | |
| 668 | + b53_enable_vlan(dev, false, dev->vlan_filtering_enabled); | |
| 650 | 669 | |
| 651 | 670 | b53_for_each_port(dev, i) |
| 652 | 671 | b53_write16(dev, B53_VLAN_PAGE, |
| 653 | − B53_VLAN_PORT_DEF_TAG(i), 1); | |
| 672 | + B53_VLAN_PORT_DEF_TAG(i), def_vid); | |
| 654 | 673 | |
| 655 | 674 | if (!is5325(dev) && !is5365(dev)) |
| 656 | 675 | b53_set_jumbo(dev, dev->enable_jumbo, false); |
| @@ -1255,6 +1274,46 @@ EXPORT_SYMBOL(b53_phylink_mac_link_up); · 598 unmodified lines | ||
| 1255 | 1274 | |
| 1256 | 1275 | int b53_vlan_filtering(struct dsa_switch *ds, int port, bool vlan_filtering) |
| 1257 | 1276 | { |
| 1277 | + struct b53_device *dev = ds->priv; | |
| 1278 | + struct net_device *bridge_dev; | |
| 1279 | + unsigned int i; | |
| 1280 | + u16 pvid, new_pvid; | |
| 1281 | + | |
| 1282 | + /* Handle the case were multiple bridges span the same switch device | |
| 1283 | + * and one of them has a different setting than what is being requested | |
| 1284 | + * which would be breaking filtering semantics for any of the other | |
| 1285 | + * bridge devices. | |
| 1286 | + */ | |
| 1287 | + b53_for_each_port(dev, i) { | |
| 1288 | + bridge_dev = dsa_to_port(ds, i)->bridge_dev; | |
| 1289 | + if (bridge_dev && | |
| 1290 | + bridge_dev != dsa_to_port(ds, port)->bridge_dev && | |
| 1291 | + br_vlan_enabled(bridge_dev) != vlan_filtering) { | |
| 1292 | + netdev_err(bridge_dev, | |
| 1293 | + "VLAN filtering is global to the switch!\n"); | |
| 1294 | + return -EINVAL; | |
| 1295 | + } | |
| 1296 | + } | |
| 1297 | + | |
| 1298 | + b53_read16(dev, B53_VLAN_PAGE, B53_VLAN_PORT_DEF_TAG(port), &pvid); | |
| 1299 | + new_pvid = pvid; | |
| 1300 | + if (dev->vlan_filtering_enabled && !vlan_filtering) { | |
| 1301 | + /* Filtering is currently enabled, use the default PVID since | |
| 1302 | + * the bridge does not expect tagging anymore | |
| 1303 | + */ | |
| 1304 | + dev->ports[port].pvid = pvid; | |
| 1305 | + new_pvid = b53_default_pvid(dev); | |
| 1306 | + } else if (!dev->vlan_filtering_enabled && vlan_filtering) { | |
| 1307 | + /* Filtering is currently disabled, restore the previous PVID */ | |
| 1308 | + new_pvid = dev->ports[port].pvid; | |
| 1309 | + } | |
| 1310 | + | |
| 1311 | + if (pvid != new_pvid) | |
| 1312 | + b53_write16(dev, B53_VLAN_PAGE, B53_VLAN_PORT_DEF_TAG(port), | |
| 1313 | + new_pvid); | |
| 1314 | + | |
| 1315 | + b53_enable_vlan(dev, dev->vlan_enabled, vlan_filtering); | |
| 1316 | + | |
| 1258 | 1317 | return 0; |
| 1259 | 1318 | } |
| 1260 | 1319 | EXPORT_SYMBOL(b53_vlan_filtering); |
| @@ -1270,7 +1329,7 @@ int b53_vlan_prepare(struct dsa_switch *ds, int port, · 9 unmodified lines | ||
| 1270 | 1329 | if (vlan->vid_end > dev->num_vlans) |
| 1271 | 1330 | return -ERANGE; |
| 1272 | 1331 | |
| 1273 | − b53_enable_vlan(dev, true); | |
| 1332 | + b53_enable_vlan(dev, true, dev->vlan_filtering_enabled); | |
| 1274 | 1333 | |
| 1275 | 1334 | return 0; |
| 1276 | 1335 | } |
| @@ -1300,7 +1359,7 @@ void b53_vlan_add(struct dsa_switch *ds, int port, · 23 unmodified lines | ||
| 1300 | 1359 | b53_fast_age_vlan(dev, vid); |
| 1301 | 1360 | } |
| 1302 | 1361 | |
| 1303 | − if (pvid) { | |
| 1362 | + if (pvid && !dsa_is_cpu_port(ds, port)) { | |
| 1304 | 1363 | b53_write16(dev, B53_VLAN_PAGE, B53_VLAN_PORT_DEF_TAG(port), |
| 1305 | 1364 | vlan->vid_end); |
| 1306 | 1365 | b53_fast_age_vlan(dev, vid); |
| @@ -1326,12 +1385,8 @@ int b53_vlan_del(struct dsa_switch *ds, int port, · 19 unmodified lines | ||
| 1326 | 1385 | |
| 1327 | 1386 | vl->members &= ~BIT(port); |
| 1328 | 1387 | |
| 1329 | − if (pvid == vid) { | |
| 1330 | − if (is5325(dev) || is5365(dev)) | |
| 1331 | − pvid = 1; | |
| 1332 | − else | |
| 1333 | − pvid = 0; | |
| 1334 | − } | |
| 1388 | + if (pvid == vid) | |
| 1389 | + pvid = b53_default_pvid(dev); | |
| 1335 | 1390 | |
| 1336 | 1391 | if (untagged && !dsa_is_cpu_port(ds, port)) |
| 1337 | 1392 | vl->untag &= ~(BIT(port)); |
| @@ -1644,10 +1699,7 @@ void b53_br_leave(struct dsa_switch *ds, int port, struct net_device *br) · 306 unmodified lines | ||
| 1644 | 1699 | b53_write16(dev, B53_PVLAN_PAGE, B53_PVLAN_PORT_MASK(port), pvlan); |
| 1645 | 1700 | dev->ports[port].vlan_ctl_mask = pvlan; |
| 1646 | 1701 | |
| 1647 | − if (is5325(dev) || is5365(dev)) | |
| 1648 | − pvid = 1; | |
| 1649 | − else | |
| 1650 | − pvid = 0; | |
| 1702 | + pvid = b53_default_pvid(dev); | |
| 1651 | 1703 | |
| 1652 | 1704 | /* Make this port join all VLANs without VLAN entries */ |
| 1653 | 1705 | if (is58xx(dev)) { |
| @@ -91,6 +91,7 @@ enum { · 90 unmodified lines | ||
| 91 | 91 | struct b53_port { |
| 92 | 92 | u16 vlan_ctl_mask; |
| 93 | 93 | struct ethtool_eee eee; |
| 94 | + u16 pvid; | |
| 94 | 95 | }; |
| 95 | 96 | |
| 96 | 97 | struct b53_vlan { |
| @@ -137,6 +138,8 @@ struct b53_device { · 40 unmodified lines | ||
| 137 | 138 | |
| 138 | 139 | unsigned int num_vlans; |
| 139 | 140 | struct b53_vlan *vlans; |
| 141 | + bool vlan_enabled; | |
| 142 | + bool vlan_filtering_enabled; | |
| 140 | 143 | unsigned int num_ports; |
| 141 | 144 | struct b53_port *ports; |
| 142 | 145 | }; |
| @@ -726,10 +726,11 @@ static void bcm_sf2_sw_get_wol(struct dsa_switch *ds, int port, · 725 unmodified lines | ||
| 726 | 726 | { |
| 727 | 727 | struct net_device *p = ds->ports[port].cpu_dp->master; |
| 728 | 728 | struct bcm_sf2_priv *priv = bcm_sf2_to_priv(ds); |
| 729 | − struct ethtool_wolinfo pwol; | |
| 729 | + struct ethtool_wolinfo pwol = { }; | |
| 730 | 730 | |
| 731 | 731 | /* Get the parent device WoL settings */ |
| 732 | − p->ethtool_ops->get_wol(p, &pwol); | |
| 732 | + if (p->ethtool_ops->get_wol) | |
| 733 | + p->ethtool_ops->get_wol(p, &pwol); | |
| 733 | 734 | |
| 734 | 735 | /* Advertise the parent device supported settings */ |
| 735 | 736 | wol->supported = pwol.supported; |
| @@ -750,9 +751,10 @@ static int bcm_sf2_sw_set_wol(struct dsa_switch *ds, int port, · 14 unmodified lines | ||
| 750 | 751 | struct net_device *p = ds->ports[port].cpu_dp->master; |
| 751 | 752 | struct bcm_sf2_priv *priv = bcm_sf2_to_priv(ds); |
| 752 | 753 | s8 cpu_port = ds->ports[port].cpu_dp->index; |
| 753 | − struct ethtool_wolinfo pwol; | |
| 754 | + struct ethtool_wolinfo pwol = { }; | |
| 754 | 755 | |
| 755 | − p->ethtool_ops->get_wol(p, &pwol); | |
| 756 | + if (p->ethtool_ops->get_wol) | |
| 757 | + p->ethtool_ops->get_wol(p, &pwol); | |
| 756 | 758 | if (wol->wolopts & ~pwol.supported) |
| 757 | 759 | return -EINVAL; |
| 758 | 760 | |
| @@ -1162,6 +1162,12 @@ static struct platform_driver gswip_driver = { · 1161 unmodified lines | ||
| 1162 | 1162 | |
| 1163 | 1163 | module_platform_driver(gswip_driver); |
| 1164 | 1164 | |
| 1165 | +MODULE_FIRMWARE("lantiq/xrx300_phy11g_a21.bin"); | |
| 1166 | +MODULE_FIRMWARE("lantiq/xrx300_phy22f_a21.bin"); | |
| 1167 | +MODULE_FIRMWARE("lantiq/xrx200_phy11g_a14.bin"); | |
| 1168 | +MODULE_FIRMWARE("lantiq/xrx200_phy11g_a22.bin"); | |
| 1169 | +MODULE_FIRMWARE("lantiq/xrx200_phy22f_a14.bin"); | |
| 1170 | +MODULE_FIRMWARE("lantiq/xrx200_phy22f_a22.bin"); | |
| 1165 | 1171 | MODULE_AUTHOR("Hauke Mehrtens <hauke@hauke-m.de>"); |
| 1166 | 1172 | MODULE_DESCRIPTION("Lantiq / Intel GSWIP driver"); |
| 1167 | 1173 | MODULE_LICENSE("GPL v2"); |
| @@ -896,7 +896,7 @@ static uint64_t _mv88e6xxx_get_ethtool_stat(struct mv88e6xxx_chip *chip, · 895 unmodified lines | ||
| 896 | 896 | default: |
| 897 | 897 | return U64_MAX; |
| 898 | 898 | } |
| 899 | − value = (((u64)high) << 16) | low; | |
| 899 | + value = (((u64)high) << 32) | low; | |
| 900 | 900 | return value; |
| 901 | 901 | } |
| 902 | 902 | |
| @@ -3093,7 +3093,7 @@ static const struct mv88e6xxx_ops mv88e6161_ops = { · 2190 unmodified lines | ||
| 3093 | 3093 | .port_disable_pri_override = mv88e6xxx_port_disable_pri_override, |
| 3094 | 3094 | .port_link_state = mv88e6352_port_link_state, |
| 3095 | 3095 | .port_get_cmode = mv88e6185_port_get_cmode, |
| 3096 | − .stats_snapshot = mv88e6320_g1_stats_snapshot, | |
| 3096 | + .stats_snapshot = mv88e6xxx_g1_stats_snapshot, | |
| 3097 | 3097 | .stats_set_histogram = mv88e6095_g1_stats_set_histogram, |
| 3098 | 3098 | .stats_get_sset_count = mv88e6095_stats_get_sset_count, |
| 3099 | 3099 | .stats_get_strings = mv88e6095_stats_get_strings, |
| @@ -4595,6 +4595,14 @@ static int mv88e6xxx_smi_init(struct mv88e6xxx_chip *chip, · 1495 unmodified lines | ||
| 4595 | 4595 | return 0; |
| 4596 | 4596 | } |
| 4597 | 4597 | |
| 4598 | +static void mv88e6xxx_ports_cmode_init(struct mv88e6xxx_chip *chip) | |
| 4599 | +{ | |
| 4600 | + int i; | |
| 4601 | + | |
| 4602 | + for (i = 0; i < mv88e6xxx_num_ports(chip); i++) | |
| 4603 | + chip->ports[i].cmode = MV88E6XXX_PORT_STS_CMODE_INVALID; | |
| 4604 | +} | |
| 4605 | + | |
| 4598 | 4606 | static enum dsa_tag_protocol mv88e6xxx_get_tag_protocol(struct dsa_switch *ds, |
| 4599 | 4607 | int port) |
| 4600 | 4608 | { |
| @@ -4631,6 +4639,8 @@ static const char *mv88e6xxx_drv_probe(struct device *dsa_dev, · 30 unmodified lines | ||
| 4631 | 4639 | if (err) |
| 4632 | 4640 | goto free; |
| 4633 | 4641 | |
| 4642 | + mv88e6xxx_ports_cmode_init(chip); | |
| 4643 | + | |
| 4634 | 4644 | mutex_lock(&chip->reg_lock); |
| 4635 | 4645 | err = mv88e6xxx_switch_reset(chip); |
| 4636 | 4646 | mutex_unlock(&chip->reg_lock); |
| @@ -398,6 +398,10 @@ int mv88e6390x_port_set_cmode(struct mv88e6xxx_chip *chip, int port, · 397 unmodified lines | ||
| 398 | 398 | cmode = 0; |
| 399 | 399 | } |
| 400 | 400 | |
| 401 | + /* cmode doesn't change, nothing to do for us */ | |
| 402 | + if (cmode == chip->ports[port].cmode) | |
| 403 | + return 0; | |
| 404 | + | |
| 401 | 405 | lane = mv88e6390x_serdes_get_lane(chip, port); |
| 402 | 406 | if (lane < 0) |
| 403 | 407 | return lane; |
| @@ -408,7 +412,7 @@ int mv88e6390x_port_set_cmode(struct mv88e6xxx_chip *chip, int port, · 4 unmodified lines | ||
| 408 | 412 | return err; |
| 409 | 413 | } |
| 410 | 414 | |
| 411 | − err = mv88e6390_serdes_power(chip, port, false); | |
| 415 | + err = mv88e6390x_serdes_power(chip, port, false); | |
| 412 | 416 | if (err) |
| 413 | 417 | return err; |
| 414 | 418 | |
| @@ -424,7 +428,7 @@ int mv88e6390x_port_set_cmode(struct mv88e6xxx_chip *chip, int port, · 9 unmodified lines | ||
| 424 | 428 | if (err) |
| 425 | 429 | return err; |
| 426 | 430 | |
| 427 | − err = mv88e6390_serdes_power(chip, port, true); | |
| 431 | + err = mv88e6390x_serdes_power(chip, port, true); | |
| 428 | 432 | if (err) |
| 429 | 433 | return err; |
| 430 | 434 | |
| @@ -52,6 +52,7 @@ · 51 unmodified lines | ||
| 52 | 52 | #define MV88E6185_PORT_STS_CMODE_1000BASE_X 0x0005 |
| 53 | 53 | #define MV88E6185_PORT_STS_CMODE_PHY 0x0006 |
| 54 | 54 | #define MV88E6185_PORT_STS_CMODE_DISABLED 0x0007 |
| 55 | +#define MV88E6XXX_PORT_STS_CMODE_INVALID 0xff | |
| 55 | 56 | |
| 56 | 57 | /* Offset 0x01: MAC (or PCS or Physical) Control Register */ |
| 57 | 58 | #define MV88E6XXX_PORT_MAC_CTL 0x01 |
| @@ -275,6 +275,9 @@ static int hw_atl_b0_hw_offload_set(struct aq_hw_s *self, · 274 unmodified lines | ||
| 275 | 275 | |
| 276 | 276 | static int hw_atl_b0_hw_init_tx_path(struct aq_hw_s *self) |
| 277 | 277 | { |
| 278 | + /* Tx TC/Queue number config */ | |
| 279 | + hw_atl_rpb_tps_tx_tc_mode_set(self, 1U); | |
| 280 | + | |
| 278 | 281 | hw_atl_thm_lso_tcp_flag_of_first_pkt_set(self, 0x0FF6U); |
| 279 | 282 | hw_atl_thm_lso_tcp_flag_of_middle_pkt_set(self, 0x0FF6U); |
| 280 | 283 | hw_atl_thm_lso_tcp_flag_of_last_pkt_set(self, 0x0F7FU); |
| @@ -1274,6 +1274,15 @@ void hw_atl_tpb_tx_buff_en_set(struct aq_hw_s *aq_hw, u32 tx_buff_en) · 1273 unmodified lines | ||
| 1274 | 1274 | HW_ATL_TPB_TX_BUF_EN_SHIFT, tx_buff_en); |
| 1275 | 1275 | } |
| 1276 | 1276 | |
| 1277 | +void hw_atl_rpb_tps_tx_tc_mode_set(struct aq_hw_s *aq_hw, | |
| 1278 | + u32 tx_traf_class_mode) | |
| 1279 | +{ | |
| 1280 | + aq_hw_write_reg_bit(aq_hw, HW_ATL_TPB_TX_TC_MODE_ADDR, | |
| 1281 | + HW_ATL_TPB_TX_TC_MODE_MSK, | |
| 1282 | + HW_ATL_TPB_TX_TC_MODE_SHIFT, | |
| 1283 | + tx_traf_class_mode); | |
| 1284 | +} | |
| 1285 | + | |
| 1277 | 1286 | void hw_atl_tpb_tx_buff_hi_threshold_per_tc_set(struct aq_hw_s *aq_hw, |
| 1278 | 1287 | u32 tx_buff_hi_threshold_per_tc, |
| 1279 | 1288 | u32 buffer) |
| @@ -605,6 +605,10 @@ void hw_atl_thm_lso_tcp_flag_of_middle_pkt_set(struct aq_hw_s *aq_hw, · 604 unmodified lines | ||
| 605 | 605 | |
| 606 | 606 | /* tpb */ |
| 607 | 607 | |
| 608 | +/* set TX Traffic Class Mode */ | |
| 609 | +void hw_atl_rpb_tps_tx_tc_mode_set(struct aq_hw_s *aq_hw, | |
| 610 | + u32 tx_traf_class_mode); | |
| 611 | + | |
| 608 | 612 | /* set tx buffer enable */ |
| 609 | 613 | void hw_atl_tpb_tx_buff_en_set(struct aq_hw_s *aq_hw, u32 tx_buff_en); |
| 610 | 614 | |
| @@ -1948,6 +1948,19 @@ · 1947 unmodified lines | ||
| 1948 | 1948 | /* default value of bitfield tx_buf_en */ |
| 1949 | 1949 | #define HW_ATL_TPB_TX_BUF_EN_DEFAULT 0x0 |
| 1950 | 1950 | |
| 1951 | +/* register address for bitfield tx_tc_mode */ | |
| 1952 | +#define HW_ATL_TPB_TX_TC_MODE_ADDR 0x00007900 | |
| 1953 | +/* bitmask for bitfield tx_tc_mode */ | |
| 1954 | +#define HW_ATL_TPB_TX_TC_MODE_MSK 0x00000100 | |
| 1955 | +/* inverted bitmask for bitfield tx_tc_mode */ | |
| 1956 | +#define HW_ATL_TPB_TX_TC_MODE_MSKN 0xFFFFFEFF | |
| 1957 | +/* lower bit position of bitfield tx_tc_mode */ | |
| 1958 | +#define HW_ATL_TPB_TX_TC_MODE_SHIFT 8 | |
| 1959 | +/* width of bitfield tx_tc_mode */ | |
| 1960 | +#define HW_ATL_TPB_TX_TC_MODE_WIDTH 1 | |
| 1961 | +/* default value of bitfield tx_tc_mode */ | |
| 1962 | +#define HW_ATL_TPB_TX_TC_MODE_DEFAULT 0x0 | |
| 1963 | + | |
| 1951 | 1964 | /* tx tx{b}_hi_thresh[c:0] bitfield definitions |
| 1952 | 1965 | * preprocessor definitions for the bitfield "tx{b}_hi_thresh[c:0]". |
| 1953 | 1966 | * parameter: buffer {b} | stride size 0x10 | range [0, 7] |
| @@ -1335,13 +1335,11 @@ static int atl2_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 1334 unmodified lines | ||
| 1335 | 1335 | { |
| 1336 | 1336 | struct net_device *netdev; |
| 1337 | 1337 | struct atl2_adapter *adapter; |
| 1338 | − static int cards_found; | |
| 1338 | + static int cards_found = 0; | |
| 1339 | 1339 | unsigned long mmio_start; |
| 1340 | 1340 | int mmio_len; |
| 1341 | 1341 | int err; |
| 1342 | 1342 | |
| 1343 | − cards_found = 0; | |
| 1344 | − | |
| 1345 | 1343 | err = pci_enable_device(pdev); |
| 1346 | 1344 | if (err) |
| 1347 | 1345 | return err; |
| @@ -134,6 +134,10 @@ static void bcm_sysport_set_rx_csum(struct net_device *dev, · 133 unmodified lines | ||
| 134 | 134 | |
| 135 | 135 | priv->rx_chk_en = !!(wanted & NETIF_F_RXCSUM); |
| 136 | 136 | reg = rxchk_readl(priv, RXCHK_CONTROL); |
| 137 | + /* Clear L2 header checks, which would prevent BPDUs | |
| 138 | + * from being received. | |
| 139 | + */ | |
| 140 | + reg &= ~RXCHK_L2_HDR_DIS; | |
| 137 | 141 | if (priv->rx_chk_en) |
| 138 | 142 | reg |= RXCHK_EN; |
| 139 | 143 | else |
| @@ -500,6 +500,12 @@ normal_tx: · 499 unmodified lines | ||
| 500 | 500 | } |
| 501 | 501 | |
| 502 | 502 | length >>= 9; |
| 503 | + if (unlikely(length >= ARRAY_SIZE(bnxt_lhint_arr))) { | |
| 504 | + dev_warn_ratelimited(&pdev->dev, "Dropped oversize %d bytes TX packet.\n", | |
| 505 | + skb->len); | |
| 506 | + i = 0; | |
| 507 | + goto tx_dma_error; | |
| 508 | + } | |
| 503 | 509 | flags |= bnxt_lhint_arr[length]; |
| 504 | 510 | txbd->tx_bd_len_flags_type = cpu_to_le32(flags); |
| 505 | 511 | |
| @@ -3903,7 +3909,7 @@ static int bnxt_hwrm_do_send_msg(struct bnxt *bp, void *msg, u32 msg_len, · 3397 unmodified lines | ||
| 3903 | 3909 | if (len) |
| 3904 | 3910 | break; |
| 3905 | 3911 | /* on first few passes, just barely sleep */ |
| 3906 | − if (i < DFLT_HWRM_CMD_TIMEOUT) | |
| 3912 | + if (i < HWRM_SHORT_TIMEOUT_COUNTER) | |
| 3907 | 3913 | usleep_range(HWRM_SHORT_MIN_TIMEOUT, |
| 3908 | 3914 | HWRM_SHORT_MAX_TIMEOUT); |
| 3909 | 3915 | else |
| @@ -3926,7 +3932,7 @@ static int bnxt_hwrm_do_send_msg(struct bnxt *bp, void *msg, u32 msg_len, · 16 unmodified lines | ||
| 3926 | 3932 | dma_rmb(); |
| 3927 | 3933 | if (*valid) |
| 3928 | 3934 | break; |
| 3929 | − udelay(1); | |
| 3935 | + usleep_range(1, 5); | |
| 3930 | 3936 | } |
| 3931 | 3937 | |
| 3932 | 3938 | if (j >= HWRM_VALID_BIT_DELAY_USEC) { |
| @@ -582,7 +582,7 @@ struct nqe_cn { · 581 unmodified lines | ||
| 582 | 582 | (HWRM_SHORT_TIMEOUT_COUNTER * HWRM_SHORT_MIN_TIMEOUT + \ |
| 583 | 583 | ((n) - HWRM_SHORT_TIMEOUT_COUNTER) * HWRM_MIN_TIMEOUT)) |
| 584 | 584 | |
| 585 | −#define HWRM_VALID_BIT_DELAY_USEC 20 | |
| 585 | +#define HWRM_VALID_BIT_DELAY_USEC 150 | |
| 586 | 586 | |
| 587 | 587 | #define BNXT_HWRM_CHNL_CHIMP 0 |
| 588 | 588 | #define BNXT_HWRM_CHNL_KONG 1 |
| @@ -271,7 +271,7 @@ struct xcast_addr_list { · 270 unmodified lines | ||
| 271 | 271 | }; |
| 272 | 272 | |
| 273 | 273 | struct nicvf_work { |
| 274 | − struct delayed_work work; | |
| 274 | + struct work_struct work; | |
| 275 | 275 | u8 mode; |
| 276 | 276 | struct xcast_addr_list *mc; |
| 277 | 277 | }; |
| @@ -327,7 +327,11 @@ struct nicvf { · 49 unmodified lines | ||
| 327 | 327 | struct nicvf_work rx_mode_work; |
| 328 | 328 | /* spinlock to protect workqueue arguments from concurrent access */ |
| 329 | 329 | spinlock_t rx_mode_wq_lock; |
| 330 | − | |
| 330 | + /* workqueue for handling kernel ndo_set_rx_mode() calls */ | |
| 331 | + struct workqueue_struct *nicvf_rx_mode_wq; | |
| 332 | + /* mutex to protect VF's mailbox contents from concurrent access */ | |
| 333 | + struct mutex rx_mode_mtx; | |
| 334 | + struct delayed_work link_change_work; | |
| 331 | 335 | /* PTP timestamp */ |
| 332 | 336 | struct cavium_ptp *ptp_clock; |
| 333 | 337 | /* Inbound timestamping is on */ |
| @@ -575,10 +579,8 @@ struct set_ptp { · 241 unmodified lines | ||
| 575 | 579 | |
| 576 | 580 | struct xcast { |
| 577 | 581 | u8 msg; |
| 578 | − union { | |
| 579 | − u8 mode; | |
| 580 | − u64 mac; | |
| 581 | − } data; | |
| 582 | + u8 mode; | |
| 583 | + u64 mac:48; | |
| 582 | 584 | }; |
| 583 | 585 | |
| 584 | 586 | /* 128 bit shared memory between PF and each VF */ |
| @@ -57,14 +57,8 @@ struct nicpf { · 56 unmodified lines | ||
| 57 | 57 | #define NIC_GET_BGX_FROM_VF_LMAC_MAP(map) ((map >> 4) & 0xF) |
| 58 | 58 | #define NIC_GET_LMAC_FROM_VF_LMAC_MAP(map) (map & 0xF) |
| 59 | 59 | u8 *vf_lmac_map; |
| 60 | − struct delayed_work dwork; | |
| 61 | − struct workqueue_struct *check_link; | |
| 62 | − u8 *link; | |
| 63 | − u8 *duplex; | |
| 64 | − u32 *speed; | |
| 65 | 60 | u16 cpi_base[MAX_NUM_VFS_SUPPORTED]; |
| 66 | 61 | u16 rssi_base[MAX_NUM_VFS_SUPPORTED]; |
| 67 | − bool mbx_lock[MAX_NUM_VFS_SUPPORTED]; | |
| 68 | 62 | |
| 69 | 63 | /* MSI-X */ |
| 70 | 64 | u8 num_vec; |
| @@ -929,6 +923,35 @@ static void nic_config_timestamp(struct nicpf *nic, int vf, struct set_ptp *ptp) · 858 unmodified lines | ||
| 929 | 923 | nic_reg_write(nic, NIC_PF_PKIND_0_15_CFG | (pkind_idx << 3), pkind_val); |
| 930 | 924 | } |
| 931 | 925 | |
| 926 | +/* Get BGX LMAC link status and update corresponding VF | |
| 927 | + * if there is a change, valid only if internal L2 switch | |
| 928 | + * is not present otherwise VF link is always treated as up | |
| 929 | + */ | |
| 930 | +static void nic_link_status_get(struct nicpf *nic, u8 vf) | |
| 931 | +{ | |
| 932 | + union nic_mbx mbx = {}; | |
| 933 | + struct bgx_link_status link; | |
| 934 | + u8 bgx, lmac; | |
| 935 | + | |
| 936 | + mbx.link_status.msg = NIC_MBOX_MSG_BGX_LINK_CHANGE; | |
| 937 | + | |
| 938 | + /* Get BGX, LMAC indices for the VF */ | |
| 939 | + bgx = NIC_GET_BGX_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); | |
| 940 | + lmac = NIC_GET_LMAC_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); | |
| 941 | + | |
| 942 | + /* Get interface link status */ | |
| 943 | + bgx_get_lmac_link_state(nic->node, bgx, lmac, &link); | |
| 944 | + | |
| 945 | + /* Send a mbox message to VF with current link status */ | |
| 946 | + mbx.link_status.link_up = link.link_up; | |
| 947 | + mbx.link_status.duplex = link.duplex; | |
| 948 | + mbx.link_status.speed = link.speed; | |
| 949 | + mbx.link_status.mac_type = link.mac_type; | |
| 950 | + | |
| 951 | + /* reply with link status */ | |
| 952 | + nic_send_msg_to_vf(nic, vf, &mbx); | |
| 953 | +} | |
| 954 | + | |
| 932 | 955 | /* Interrupt handler to handle mailbox messages from VFs */ |
| 933 | 956 | static void nic_handle_mbx_intr(struct nicpf *nic, int vf) |
| 934 | 957 | { |
| @@ -941,8 +964,6 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 6 unmodified lines | ||
| 941 | 964 | int i; |
| 942 | 965 | int ret = 0; |
| 943 | 966 | |
| 944 | − nic->mbx_lock[vf] = true; | |
| 945 | − | |
| 946 | 967 | mbx_addr = nic_get_mbx_addr(vf); |
| 947 | 968 | mbx_data = (u64 *)&mbx; |
| 948 | 969 | |
| @@ -957,12 +978,7 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 8 unmodified lines | ||
| 957 | 978 | switch (mbx.msg.msg) { |
| 958 | 979 | case NIC_MBOX_MSG_READY: |
| 959 | 980 | nic_mbx_send_ready(nic, vf); |
| 960 | − if (vf < nic->num_vf_en) { | |
| 961 | − nic->link[vf] = 0; | |
| 962 | − nic->duplex[vf] = 0; | |
| 963 | − nic->speed[vf] = 0; | |
| 964 | − } | |
| 965 | − goto unlock; | |
| 981 | + return; | |
| 966 | 982 | case NIC_MBOX_MSG_QS_CFG: |
| 967 | 983 | reg_addr = NIC_PF_QSET_0_127_CFG | |
| 968 | 984 | (mbx.qs.num << NIC_QS_ID_SHIFT); |
| @@ -1031,7 +1047,7 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 62 unmodified lines | ||
| 1031 | 1047 | break; |
| 1032 | 1048 | case NIC_MBOX_MSG_RSS_SIZE: |
| 1033 | 1049 | nic_send_rss_size(nic, vf); |
| 1034 | − goto unlock; | |
| 1050 | + return; | |
| 1035 | 1051 | case NIC_MBOX_MSG_RSS_CFG: |
| 1036 | 1052 | case NIC_MBOX_MSG_RSS_CFG_CONT: |
| 1037 | 1053 | nic_config_rss(nic, &mbx.rss_cfg); |
| @@ -1039,7 +1055,7 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 1 unmodified line | ||
| 1039 | 1055 | case NIC_MBOX_MSG_CFG_DONE: |
| 1040 | 1056 | /* Last message of VF config msg sequence */ |
| 1041 | 1057 | nic_enable_vf(nic, vf, true); |
| 1042 | − goto unlock; | |
| 1058 | + break; | |
| 1043 | 1059 | case NIC_MBOX_MSG_SHUTDOWN: |
| 1044 | 1060 | /* First msg in VF teardown sequence */ |
| 1045 | 1061 | if (vf >= nic->num_vf_en) |
| @@ -1049,19 +1065,19 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 3 unmodified lines | ||
| 1049 | 1065 | break; |
| 1050 | 1066 | case NIC_MBOX_MSG_ALLOC_SQS: |
| 1051 | 1067 | nic_alloc_sqs(nic, &mbx.sqs_alloc); |
| 1052 | − goto unlock; | |
| 1068 | + return; | |
| 1053 | 1069 | case NIC_MBOX_MSG_NICVF_PTR: |
| 1054 | 1070 | nic->nicvf[vf] = mbx.nicvf.nicvf; |
| 1055 | 1071 | break; |
| 1056 | 1072 | case NIC_MBOX_MSG_PNICVF_PTR: |
| 1057 | 1073 | nic_send_pnicvf(nic, vf); |
| 1058 | − goto unlock; | |
| 1074 | + return; | |
| 1059 | 1075 | case NIC_MBOX_MSG_SNICVF_PTR: |
| 1060 | 1076 | nic_send_snicvf(nic, &mbx.nicvf); |
| 1061 | − goto unlock; | |
| 1077 | + return; | |
| 1062 | 1078 | case NIC_MBOX_MSG_BGX_STATS: |
| 1063 | 1079 | nic_get_bgx_stats(nic, &mbx.bgx_stats); |
| 1064 | − goto unlock; | |
| 1080 | + return; | |
| 1065 | 1081 | case NIC_MBOX_MSG_LOOPBACK: |
| 1066 | 1082 | ret = nic_config_loopback(nic, &mbx.lbk); |
| 1067 | 1083 | break; |
| @@ -1070,7 +1086,7 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 2 unmodified lines | ||
| 1070 | 1086 | break; |
| 1071 | 1087 | case NIC_MBOX_MSG_PFC: |
| 1072 | 1088 | nic_pause_frame(nic, vf, &mbx.pfc); |
| 1073 | − goto unlock; | |
| 1089 | + return; | |
| 1074 | 1090 | case NIC_MBOX_MSG_PTP_CFG: |
| 1075 | 1091 | nic_config_timestamp(nic, vf, &mbx.ptp); |
| 1076 | 1092 | break; |
| @@ -1094,7 +1110,7 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 17 unmodified lines | ||
| 1094 | 1110 | bgx = NIC_GET_BGX_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); |
| 1095 | 1111 | lmac = NIC_GET_LMAC_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); |
| 1096 | 1112 | bgx_set_dmac_cam_filter(nic->node, bgx, lmac, |
| 1097 | − mbx.xcast.data.mac, | |
| 1113 | + mbx.xcast.mac, | |
| 1098 | 1114 | vf < NIC_VF_PER_MBX_REG ? vf : |
| 1099 | 1115 | vf - NIC_VF_PER_MBX_REG); |
| 1100 | 1116 | break; |
| @@ -1106,8 +1122,15 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 5 unmodified lines | ||
| 1106 | 1122 | } |
| 1107 | 1123 | bgx = NIC_GET_BGX_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); |
| 1108 | 1124 | lmac = NIC_GET_LMAC_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); |
| 1109 | − bgx_set_xcast_mode(nic->node, bgx, lmac, mbx.xcast.data.mode); | |
| 1125 | + bgx_set_xcast_mode(nic->node, bgx, lmac, mbx.xcast.mode); | |
| 1110 | 1126 | break; |
| 1127 | + case NIC_MBOX_MSG_BGX_LINK_CHANGE: | |
| 1128 | + if (vf >= nic->num_vf_en) { | |
| 1129 | + ret = -1; /* NACK */ | |
| 1130 | + break; | |
| 1131 | + } | |
| 1132 | + nic_link_status_get(nic, vf); | |
| 1133 | + return; | |
| 1111 | 1134 | default: |
| 1112 | 1135 | dev_err(&nic->pdev->dev, |
| 1113 | 1136 | "Invalid msg from VF%d, msg 0x%x\n", vf, mbx.msg.msg); |
| @@ -1121,8 +1144,6 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 7 unmodified lines | ||
| 1121 | 1144 | mbx.msg.msg, vf); |
| 1122 | 1145 | nic_mbx_send_nack(nic, vf); |
| 1123 | 1146 | } |
| 1124 | −unlock: | |
| 1125 | − nic->mbx_lock[vf] = false; | |
| 1126 | 1147 | } |
| 1127 | 1148 | |
| 1128 | 1149 | static irqreturn_t nic_mbx_intr_handler(int irq, void *nic_irq) |
| @@ -1270,52 +1291,6 @@ static int nic_sriov_init(struct pci_dev *pdev, struct nicpf *nic) · 141 unmodified lines | ||
| 1270 | 1291 | return 0; |
| 1271 | 1292 | } |
| 1272 | 1293 | |
| 1273 | −/* Poll for BGX LMAC link status and update corresponding VF | |
| 1274 | − * if there is a change, valid only if internal L2 switch | |
| 1275 | − * is not present otherwise VF link is always treated as up | |
| 1276 | − */ | |
| 1277 | −static void nic_poll_for_link(struct work_struct *work) | |
| 1278 | −{ | |
| 1279 | − union nic_mbx mbx = {}; | |
| 1280 | − struct nicpf *nic; | |
| 1281 | − struct bgx_link_status link; | |
| 1282 | − u8 vf, bgx, lmac; | |
| 1283 | − | |
| 1284 | − nic = container_of(work, struct nicpf, dwork.work); | |
| 1285 | − | |
| 1286 | − mbx.link_status.msg = NIC_MBOX_MSG_BGX_LINK_CHANGE; | |
| 1287 | − | |
| 1288 | − for (vf = 0; vf < nic->num_vf_en; vf++) { | |
| 1289 | − /* Poll only if VF is UP */ | |
| 1290 | − if (!nic->vf_enabled[vf]) | |
| 1291 | − continue; | |
| 1292 | − | |
| 1293 | − /* Get BGX, LMAC indices for the VF */ | |
| 1294 | − bgx = NIC_GET_BGX_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); | |
| 1295 | − lmac = NIC_GET_LMAC_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); | |
| 1296 | − /* Get interface link status */ | |
| 1297 | − bgx_get_lmac_link_state(nic->node, bgx, lmac, &link); | |
| 1298 | − | |
| 1299 | − /* Inform VF only if link status changed */ | |
| 1300 | − if (nic->link[vf] == link.link_up) | |
| 1301 | − continue; | |
| 1302 | − | |
| 1303 | − if (!nic->mbx_lock[vf]) { | |
| 1304 | − nic->link[vf] = link.link_up; | |
| 1305 | − nic->duplex[vf] = link.duplex; | |
| 1306 | − nic->speed[vf] = link.speed; | |
| 1307 | − | |
| 1308 | − /* Send a mbox message to VF with current link status */ | |
| 1309 | − mbx.link_status.link_up = link.link_up; | |
| 1310 | − mbx.link_status.duplex = link.duplex; | |
| 1311 | − mbx.link_status.speed = link.speed; | |
| 1312 | − mbx.link_status.mac_type = link.mac_type; | |
| 1313 | − nic_send_msg_to_vf(nic, vf, &mbx); | |
| 1314 | − } | |
| 1315 | − } | |
| 1316 | − queue_delayed_work(nic->check_link, &nic->dwork, HZ * 2); | |
| 1317 | −} | |
| 1318 | − | |
| 1319 | 1294 | static int nic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) |
| 1320 | 1295 | { |
| 1321 | 1296 | struct device *dev = &pdev->dev; |
| @@ -1384,18 +1359,6 @@ static int nic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 62 unmodified lines | ||
| 1384 | 1359 | if (!nic->vf_lmac_map) |
| 1385 | 1360 | goto err_release_regions; |
| 1386 | 1361 | |
| 1387 | − nic->link = devm_kmalloc_array(dev, max_lmac, sizeof(u8), GFP_KERNEL); | |
| 1388 | − if (!nic->link) | |
| 1389 | − goto err_release_regions; | |
| 1390 | − | |
| 1391 | − nic->duplex = devm_kmalloc_array(dev, max_lmac, sizeof(u8), GFP_KERNEL); | |
| 1392 | − if (!nic->duplex) | |
| 1393 | − goto err_release_regions; | |
| 1394 | − | |
| 1395 | − nic->speed = devm_kmalloc_array(dev, max_lmac, sizeof(u32), GFP_KERNEL); | |
| 1396 | − if (!nic->speed) | |
| 1397 | − goto err_release_regions; | |
| 1398 | − | |
| 1399 | 1362 | /* Initialize hardware */ |
| 1400 | 1363 | nic_init_hw(nic); |
| 1401 | 1364 | |
| @@ -1411,22 +1374,8 @@ static int nic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 9 unmodified lines | ||
| 1411 | 1374 | if (err) |
| 1412 | 1375 | goto err_unregister_interrupts; |
| 1413 | 1376 | |
| 1414 | − /* Register a physical link status poll fn() */ | |
| 1415 | − nic->check_link = alloc_workqueue("check_link_status", | |
| 1416 | − WQ_UNBOUND | WQ_MEM_RECLAIM, 1); | |
| 1417 | − if (!nic->check_link) { | |
| 1418 | − err = -ENOMEM; | |
| 1419 | − goto err_disable_sriov; | |
| 1420 | − } | |
| 1421 | − | |
| 1422 | − INIT_DELAYED_WORK(&nic->dwork, nic_poll_for_link); | |
| 1423 | − queue_delayed_work(nic->check_link, &nic->dwork, 0); | |
| 1424 | − | |
| 1425 | 1377 | return 0; |
| 1426 | 1378 | |
| 1427 | −err_disable_sriov: | |
| 1428 | − if (nic->flags & NIC_SRIOV_ENABLED) | |
| 1429 | − pci_disable_sriov(pdev); | |
| 1430 | 1379 | err_unregister_interrupts: |
| 1431 | 1380 | nic_unregister_interrupts(nic); |
| 1432 | 1381 | err_release_regions: |
| @@ -1447,12 +1396,6 @@ static void nic_remove(struct pci_dev *pdev) · 14 unmodified lines | ||
| 1447 | 1396 | if (nic->flags & NIC_SRIOV_ENABLED) |
| 1448 | 1397 | pci_disable_sriov(pdev); |
| 1449 | 1398 | |
| 1450 | − if (nic->check_link) { | |
| 1451 | − /* Destroy work Queue */ | |
| 1452 | − cancel_delayed_work_sync(&nic->dwork); | |
| 1453 | − destroy_workqueue(nic->check_link); | |
| 1454 | − } | |
| 1455 | − | |
| 1456 | 1399 | nic_unregister_interrupts(nic); |
| 1457 | 1400 | pci_release_regions(pdev); |
| 1458 | 1401 | |
| @@ -68,9 +68,6 @@ module_param(cpi_alg, int, 0444); · 67 unmodified lines | ||
| 68 | 68 | MODULE_PARM_DESC(cpi_alg, |
| 69 | 69 | "PFC algorithm (0=none, 1=VLAN, 2=VLAN16, 3=IP Diffserv)"); |
| 70 | 70 | |
| 71 | −/* workqueue for handling kernel ndo_set_rx_mode() calls */ | |
| 72 | −static struct workqueue_struct *nicvf_rx_mode_wq; | |
| 73 | − | |
| 74 | 71 | static inline u8 nicvf_netdev_qidx(struct nicvf *nic, u8 qidx) |
| 75 | 72 | { |
| 76 | 73 | if (nic->sqs_mode) |
| @@ -127,6 +124,9 @@ int nicvf_send_msg_to_pf(struct nicvf *nic, union nic_mbx *mbx) · 50 unmodified lines | ||
| 127 | 124 | { |
| 128 | 125 | int timeout = NIC_MBOX_MSG_TIMEOUT; |
| 129 | 126 | int sleep = 10; |
| 127 | + int ret = 0; | |
| 128 | + | |
| 129 | + mutex_lock(&nic->rx_mode_mtx); | |
| 130 | 130 | |
| 131 | 131 | nic->pf_acked = false; |
| 132 | 132 | nic->pf_nacked = false; |
| @@ -139,7 +139,8 @@ int nicvf_send_msg_to_pf(struct nicvf *nic, union nic_mbx *mbx) · 6 unmodified lines | ||
| 139 | 139 | netdev_err(nic->netdev, |
| 140 | 140 | "PF NACK to mbox msg 0x%02x from VF%d\n", |
| 141 | 141 | (mbx->msg.msg & 0xFF), nic->vf_id); |
| 142 | − return -EINVAL; | |
| 142 | + ret = -EINVAL; | |
| 143 | + break; | |
| 143 | 144 | } |
| 144 | 145 | msleep(sleep); |
| 145 | 146 | if (nic->pf_acked) |
| @@ -149,10 +150,12 @@ int nicvf_send_msg_to_pf(struct nicvf *nic, union nic_mbx *mbx) · 3 unmodified lines | ||
| 149 | 150 | netdev_err(nic->netdev, |
| 150 | 151 | "PF didn't ACK to mbox msg 0x%02x from VF%d\n", |
| 151 | 152 | (mbx->msg.msg & 0xFF), nic->vf_id); |
| 152 | − return -EBUSY; | |
| 153 | + ret = -EBUSY; | |
| 154 | + break; | |
| 153 | 155 | } |
| 154 | 156 | } |
| 155 | − return 0; | |
| 157 | + mutex_unlock(&nic->rx_mode_mtx); | |
| 158 | + return ret; | |
| 156 | 159 | } |
| 157 | 160 | |
| 158 | 161 | /* Checks if VF is able to comminicate with PF |
| @@ -172,6 +175,17 @@ static int nicvf_check_pf_ready(struct nicvf *nic) · 13 unmodified lines | ||
| 172 | 175 | return 1; |
| 173 | 176 | } |
| 174 | 177 | |
| 178 | +static void nicvf_send_cfg_done(struct nicvf *nic) | |
| 179 | +{ | |
| 180 | + union nic_mbx mbx = {}; | |
| 181 | + | |
| 182 | + mbx.msg.msg = NIC_MBOX_MSG_CFG_DONE; | |
| 183 | + if (nicvf_send_msg_to_pf(nic, &mbx)) { | |
| 184 | + netdev_err(nic->netdev, | |
| 185 | + "PF didn't respond to CFG DONE msg\n"); | |
| 186 | + } | |
| 187 | +} | |
| 188 | + | |
| 175 | 189 | static void nicvf_read_bgx_stats(struct nicvf *nic, struct bgx_stats_msg *bgx) |
| 176 | 190 | { |
| 177 | 191 | if (bgx->rx) |
| @@ -228,21 +242,24 @@ static void nicvf_handle_mbx_intr(struct nicvf *nic) · 50 unmodified lines | ||
| 228 | 242 | break; |
| 229 | 243 | case NIC_MBOX_MSG_BGX_LINK_CHANGE: |
| 230 | 244 | nic->pf_acked = true; |
| 231 | − nic->link_up = mbx.link_status.link_up; | |
| 232 | − nic->duplex = mbx.link_status.duplex; | |
| 233 | − nic->speed = mbx.link_status.speed; | |
| 234 | − nic->mac_type = mbx.link_status.mac_type; | |
| 235 | − if (nic->link_up) { | |
| 236 | − netdev_info(nic->netdev, "Link is Up %d Mbps %s duplex\n", | |
| 237 | − nic->speed, | |
| 238 | − nic->duplex == DUPLEX_FULL ? | |
| 239 | − "Full" : "Half"); | |
| 240 | − netif_carrier_on(nic->netdev); | |
| 241 | − netif_tx_start_all_queues(nic->netdev); | |
| 242 | − } else { | |
| 243 | − netdev_info(nic->netdev, "Link is Down\n"); | |
| 244 | − netif_carrier_off(nic->netdev); | |
| 245 | − netif_tx_stop_all_queues(nic->netdev); | |
| 245 | + if (nic->link_up != mbx.link_status.link_up) { | |
| 246 | + nic->link_up = mbx.link_status.link_up; | |
| 247 | + nic->duplex = mbx.link_status.duplex; | |
| 248 | + nic->speed = mbx.link_status.speed; | |
| 249 | + nic->mac_type = mbx.link_status.mac_type; | |
| 250 | + if (nic->link_up) { | |
| 251 | + netdev_info(nic->netdev, | |
| 252 | + "Link is Up %d Mbps %s duplex\n", | |
| 253 | + nic->speed, | |
| 254 | + nic->duplex == DUPLEX_FULL ? | |
| 255 | + "Full" : "Half"); | |
| 256 | + netif_carrier_on(nic->netdev); | |
| 257 | + netif_tx_start_all_queues(nic->netdev); | |
| 258 | + } else { | |
| 259 | + netdev_info(nic->netdev, "Link is Down\n"); | |
| 260 | + netif_carrier_off(nic->netdev); | |
| 261 | + netif_tx_stop_all_queues(nic->netdev); | |
| 262 | + } | |
| 246 | 263 | } |
| 247 | 264 | break; |
| 248 | 265 | case NIC_MBOX_MSG_ALLOC_SQS: |
| @@ -1311,6 +1328,11 @@ int nicvf_stop(struct net_device *netdev) · 1062 unmodified lines | ||
| 1311 | 1328 | struct nicvf_cq_poll *cq_poll = NULL; |
| 1312 | 1329 | union nic_mbx mbx = {}; |
| 1313 | 1330 | |
| 1331 | + cancel_delayed_work_sync(&nic->link_change_work); | |
| 1332 | + | |
| 1333 | + /* wait till all queued set_rx_mode tasks completes */ | |
| 1334 | + drain_workqueue(nic->nicvf_rx_mode_wq); | |
| 1335 | + | |
| 1314 | 1336 | mbx.msg.msg = NIC_MBOX_MSG_SHUTDOWN; |
| 1315 | 1337 | nicvf_send_msg_to_pf(nic, &mbx); |
| 1316 | 1338 | |
| @@ -1410,13 +1432,27 @@ static int nicvf_update_hw_max_frs(struct nicvf *nic, int mtu) · 93 unmodified lines | ||
| 1410 | 1432 | return nicvf_send_msg_to_pf(nic, &mbx); |
| 1411 | 1433 | } |
| 1412 | 1434 | |
| 1435 | +static void nicvf_link_status_check_task(struct work_struct *work_arg) | |
| 1436 | +{ | |
| 1437 | + struct nicvf *nic = container_of(work_arg, | |
| 1438 | + struct nicvf, | |
| 1439 | + link_change_work.work); | |
| 1440 | + union nic_mbx mbx = {}; | |
| 1441 | + mbx.msg.msg = NIC_MBOX_MSG_BGX_LINK_CHANGE; | |
| 1442 | + nicvf_send_msg_to_pf(nic, &mbx); | |
| 1443 | + queue_delayed_work(nic->nicvf_rx_mode_wq, | |
| 1444 | + &nic->link_change_work, 2 * HZ); | |
| 1445 | +} | |
| 1446 | + | |
| 1413 | 1447 | int nicvf_open(struct net_device *netdev) |
| 1414 | 1448 | { |
| 1415 | 1449 | int cpu, err, qidx; |
| 1416 | 1450 | struct nicvf *nic = netdev_priv(netdev); |
| 1417 | 1451 | struct queue_set *qs = nic->qs; |
| 1418 | 1452 | struct nicvf_cq_poll *cq_poll = NULL; |
| 1419 | − union nic_mbx mbx = {}; | |
| 1453 | + | |
| 1454 | + /* wait till all queued set_rx_mode tasks completes if any */ | |
| 1455 | + drain_workqueue(nic->nicvf_rx_mode_wq); | |
| 1420 | 1456 | |
| 1421 | 1457 | netif_carrier_off(netdev); |
| 1422 | 1458 | |
| @@ -1512,8 +1548,12 @@ int nicvf_open(struct net_device *netdev) · 89 unmodified lines | ||
| 1512 | 1548 | nicvf_enable_intr(nic, NICVF_INTR_RBDR, qidx); |
| 1513 | 1549 | |
| 1514 | 1550 | /* Send VF config done msg to PF */ |
| 1515 | − mbx.msg.msg = NIC_MBOX_MSG_CFG_DONE; | |
| 1516 | − nicvf_write_to_mbx(nic, &mbx); | |
| 1551 | + nicvf_send_cfg_done(nic); | |
| 1552 | + | |
| 1553 | + INIT_DELAYED_WORK(&nic->link_change_work, | |
| 1554 | + nicvf_link_status_check_task); | |
| 1555 | + queue_delayed_work(nic->nicvf_rx_mode_wq, | |
| 1556 | + &nic->link_change_work, 0); | |
| 1517 | 1557 | |
| 1518 | 1558 | return 0; |
| 1519 | 1559 | cleanup: |
| @@ -1941,15 +1981,17 @@ static void __nicvf_set_rx_mode_task(u8 mode, struct xcast_addr_list *mc_addrs, · 421 unmodified lines | ||
| 1941 | 1981 | |
| 1942 | 1982 | /* flush DMAC filters and reset RX mode */ |
| 1943 | 1983 | mbx.xcast.msg = NIC_MBOX_MSG_RESET_XCAST; |
| 1944 | − nicvf_send_msg_to_pf(nic, &mbx); | |
| 1984 | + if (nicvf_send_msg_to_pf(nic, &mbx) < 0) | |
| 1985 | + goto free_mc; | |
| 1945 | 1986 | |
| 1946 | 1987 | if (mode & BGX_XCAST_MCAST_FILTER) { |
| 1947 | 1988 | /* once enabling filtering, we need to signal to PF to add |
| 1948 | 1989 | * its' own LMAC to the filter to accept packets for it. |
| 1949 | 1990 | */ |
| 1950 | 1991 | mbx.xcast.msg = NIC_MBOX_MSG_ADD_MCAST; |
| 1951 | − mbx.xcast.data.mac = 0; | |
| 1952 | − nicvf_send_msg_to_pf(nic, &mbx); | |
| 1992 | + mbx.xcast.mac = 0; | |
| 1993 | + if (nicvf_send_msg_to_pf(nic, &mbx) < 0) | |
| 1994 | + goto free_mc; | |
| 1953 | 1995 | } |
| 1954 | 1996 | |
| 1955 | 1997 | /* check if we have any specific MACs to be added to PF DMAC filter */ |
| @@ -1957,23 +1999,25 @@ static void __nicvf_set_rx_mode_task(u8 mode, struct xcast_addr_list *mc_addrs, · 1 unmodified line | ||
| 1957 | 1999 | /* now go through kernel list of MACs and add them one by one */ |
| 1958 | 2000 | for (idx = 0; idx < mc_addrs->count; idx++) { |
| 1959 | 2001 | mbx.xcast.msg = NIC_MBOX_MSG_ADD_MCAST; |
| 1960 | − mbx.xcast.data.mac = mc_addrs->mc[idx]; | |
| 1961 | − nicvf_send_msg_to_pf(nic, &mbx); | |
| 2002 | + mbx.xcast.mac = mc_addrs->mc[idx]; | |
| 2003 | + if (nicvf_send_msg_to_pf(nic, &mbx) < 0) | |
| 2004 | + goto free_mc; | |
| 1962 | 2005 | } |
| 1963 | − kfree(mc_addrs); | |
| 1964 | 2006 | } |
| 1965 | 2007 | |
| 1966 | 2008 | /* and finally set rx mode for PF accordingly */ |
| 1967 | 2009 | mbx.xcast.msg = NIC_MBOX_MSG_SET_XCAST; |
| 1968 | − mbx.xcast.data.mode = mode; | |
| 2010 | + mbx.xcast.mode = mode; | |
| 1969 | 2011 | |
| 1970 | 2012 | nicvf_send_msg_to_pf(nic, &mbx); |
| 2013 | +free_mc: | |
| 2014 | + kfree(mc_addrs); | |
| 1971 | 2015 | } |
| 1972 | 2016 | |
| 1973 | 2017 | static void nicvf_set_rx_mode_task(struct work_struct *work_arg) |
| 1974 | 2018 | { |
| 1975 | 2019 | struct nicvf_work *vf_work = container_of(work_arg, struct nicvf_work, |
| 1976 | − work.work); | |
| 2020 | + work); | |
| 1977 | 2021 | struct nicvf *nic = container_of(vf_work, struct nicvf, rx_mode_work); |
| 1978 | 2022 | u8 mode; |
| 1979 | 2023 | struct xcast_addr_list *mc; |
| @@ -2030,7 +2074,7 @@ static void nicvf_set_rx_mode(struct net_device *netdev) · 50 unmodified lines | ||
| 2030 | 2074 | kfree(nic->rx_mode_work.mc); |
| 2031 | 2075 | nic->rx_mode_work.mc = mc_list; |
| 2032 | 2076 | nic->rx_mode_work.mode = mode; |
| 2033 | − queue_delayed_work(nicvf_rx_mode_wq, &nic->rx_mode_work.work, 0); | |
| 2077 | + queue_work(nic->nicvf_rx_mode_wq, &nic->rx_mode_work.work); | |
| 2034 | 2078 | spin_unlock(&nic->rx_mode_wq_lock); |
| 2035 | 2079 | } |
| 2036 | 2080 | |
| @@ -2187,8 +2231,12 @@ static int nicvf_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 150 unmodified lines | ||
| 2187 | 2231 | |
| 2188 | 2232 | INIT_WORK(&nic->reset_task, nicvf_reset_task); |
| 2189 | 2233 | |
| 2190 | − INIT_DELAYED_WORK(&nic->rx_mode_work.work, nicvf_set_rx_mode_task); | |
| 2234 | + nic->nicvf_rx_mode_wq = alloc_ordered_workqueue("nicvf_rx_mode_wq_VF%d", | |
| 2235 | + WQ_MEM_RECLAIM, | |
| 2236 | + nic->vf_id); | |
| 2237 | + INIT_WORK(&nic->rx_mode_work.work, nicvf_set_rx_mode_task); | |
| 2191 | 2238 | spin_lock_init(&nic->rx_mode_wq_lock); |
| 2239 | + mutex_init(&nic->rx_mode_mtx); | |
| 2192 | 2240 | |
| 2193 | 2241 | err = register_netdev(netdev); |
| 2194 | 2242 | if (err) { |
| @@ -2228,13 +2276,15 @@ static void nicvf_remove(struct pci_dev *pdev) · 33 unmodified lines | ||
| 2228 | 2276 | nic = netdev_priv(netdev); |
| 2229 | 2277 | pnetdev = nic->pnicvf->netdev; |
| 2230 | 2278 | |
| 2231 | − cancel_delayed_work_sync(&nic->rx_mode_work.work); | |
| 2232 | − | |
| 2233 | 2279 | /* Check if this Qset is assigned to different VF. |
| 2234 | 2280 | * If yes, clean primary and all secondary Qsets. |
| 2235 | 2281 | */ |
| 2236 | 2282 | if (pnetdev && (pnetdev->reg_state == NETREG_REGISTERED)) |
| 2237 | 2283 | unregister_netdev(pnetdev); |
| 2284 | + if (nic->nicvf_rx_mode_wq) { | |
| 2285 | + destroy_workqueue(nic->nicvf_rx_mode_wq); | |
| 2286 | + nic->nicvf_rx_mode_wq = NULL; | |
| 2287 | + } | |
| 2238 | 2288 | nicvf_unregister_interrupts(nic); |
| 2239 | 2289 | pci_set_drvdata(pdev, NULL); |
| 2240 | 2290 | if (nic->drv_stats) |
| @@ -2261,17 +2311,11 @@ static struct pci_driver nicvf_driver = { · 20 unmodified lines | ||
| 2261 | 2311 | static int __init nicvf_init_module(void) |
| 2262 | 2312 | { |
| 2263 | 2313 | pr_info("%s, ver %s\n", DRV_NAME, DRV_VERSION); |
| 2264 | − nicvf_rx_mode_wq = alloc_ordered_workqueue("nicvf_generic", | |
| 2265 | − WQ_MEM_RECLAIM); | |
| 2266 | 2314 | return pci_register_driver(&nicvf_driver); |
| 2267 | 2315 | } |
| 2268 | 2316 | |
| 2269 | 2317 | static void __exit nicvf_cleanup_module(void) |
| 2270 | 2318 | { |
| 2271 | − if (nicvf_rx_mode_wq) { | |
| 2272 | − destroy_workqueue(nicvf_rx_mode_wq); | |
| 2273 | − nicvf_rx_mode_wq = NULL; | |
| 2274 | − } | |
| 2275 | 2319 | pci_unregister_driver(&nicvf_driver); |
| 2276 | 2320 | } |
| 2277 | 2321 | |
| @@ -1217,7 +1217,7 @@ static void bgx_init_hw(struct bgx *bgx) · 1216 unmodified lines | ||
| 1217 | 1217 | |
| 1218 | 1218 | /* Disable MAC steering (NCSI traffic) */ |
| 1219 | 1219 | for (i = 0; i < RX_TRAFFIC_STEER_RULE_COUNT; i++) |
| 1220 | − bgx_reg_write(bgx, 0, BGX_CMR_RX_STREERING + (i * 8), 0x00); | |
| 1220 | + bgx_reg_write(bgx, 0, BGX_CMR_RX_STEERING + (i * 8), 0x00); | |
| 1221 | 1221 | } |
| 1222 | 1222 | |
| 1223 | 1223 | static u8 bgx_get_lane2sds_cfg(struct bgx *bgx, struct lmac *lmac) |
| @@ -60,7 +60,7 @@ · 59 unmodified lines | ||
| 60 | 60 | #define RX_DMACX_CAM_EN BIT_ULL(48) |
| 61 | 61 | #define RX_DMACX_CAM_LMACID(x) (((u64)x) << 49) |
| 62 | 62 | #define RX_DMAC_COUNT 32 |
| 63 | −#define BGX_CMR_RX_STREERING 0x300 | |
| 63 | +#define BGX_CMR_RX_STEERING 0x300 | |
| 64 | 64 | #define RX_TRAFFIC_STEER_RULE_COUNT 8 |
| 65 | 65 | #define BGX_CMR_CHAN_MSK_AND 0x450 |
| 66 | 66 | #define BGX_CMR_BIST_STATUS 0x460 |
| @@ -660,6 +660,7 @@ static void uld_init(struct adapter *adap, struct cxgb4_lld_info *lld) · 659 unmodified lines | ||
| 660 | 660 | lld->cclk_ps = 1000000000 / adap->params.vpd.cclk; |
| 661 | 661 | lld->udb_density = 1 << adap->params.sge.eq_qpp; |
| 662 | 662 | lld->ucq_density = 1 << adap->params.sge.iq_qpp; |
| 663 | + lld->sge_host_page_size = 1 << (adap->params.sge.hps + 10); | |
| 663 | 664 | lld->filt_mode = adap->params.tp.vlan_pri_map; |
| 664 | 665 | /* MODQ_REQ_MAP sets queues 0-3 to chan 0-3 */ |
| 665 | 666 | for (i = 0; i < NCHAN; i++) |
| @@ -336,6 +336,7 @@ struct cxgb4_lld_info { · 335 unmodified lines | ||
| 336 | 336 | unsigned int cclk_ps; /* Core clock period in psec */ |
| 337 | 337 | unsigned short udb_density; /* # of user DB/page */ |
| 338 | 338 | unsigned short ucq_density; /* # of user CQs/page */ |
| 339 | + unsigned int sge_host_page_size; /* SGE host page size */ | |
| 339 | 340 | unsigned short filt_mode; /* filter optional components */ |
| 340 | 341 | unsigned short tx_modq[NCHAN]; /* maps each tx channel to a */ |
| 341 | 342 | /* scheduler queue */ |
| @@ -3128,6 +3128,9 @@ int hns_dsaf_roce_reset(struct fwnode_handle *dsaf_fwnode, bool dereset) · 3127 unmodified lines | ||
| 3128 | 3128 | dsaf_set_bit(credit, DSAF_SBM_ROCEE_CFG_CRD_EN_B, 1); |
| 3129 | 3129 | dsaf_write_dev(dsaf_dev, DSAF_SBM_ROCEE_CFG_REG_REG, credit); |
| 3130 | 3130 | } |
| 3131 | + | |
| 3132 | + put_device(&pdev->dev); | |
| 3133 | + | |
| 3131 | 3134 | return 0; |
| 3132 | 3135 | } |
| 3133 | 3136 | EXPORT_SYMBOL(hns_dsaf_roce_reset); |
| @@ -3289,8 +3289,11 @@ static int i40e_configure_rx_ring(struct i40e_ring *ring) · 3288 unmodified lines | ||
| 3289 | 3289 | i40e_alloc_rx_buffers_zc(ring, I40E_DESC_UNUSED(ring)) : |
| 3290 | 3290 | !i40e_alloc_rx_buffers(ring, I40E_DESC_UNUSED(ring)); |
| 3291 | 3291 | if (!ok) { |
| 3292 | + /* Log this in case the user has forgotten to give the kernel | |
| 3293 | + * any buffers, even later in the application. | |
| 3294 | + */ | |
| 3292 | 3295 | dev_info(&vsi->back->pdev->dev, |
| 3293 | − "Failed allocate some buffers on %sRx ring %d (pf_q %d)\n", | |
| 3296 | + "Failed to allocate some buffers on %sRx ring %d (pf_q %d)\n", | |
| 3294 | 3297 | ring->xsk_umem ? "UMEM enabled " : "", |
| 3295 | 3298 | ring->queue_index, pf_q); |
| 3296 | 3299 | } |
| @@ -6725,8 +6728,13 @@ void i40e_down(struct i40e_vsi *vsi) · 3428 unmodified lines | ||
| 6725 | 6728 | |
| 6726 | 6729 | for (i = 0; i < vsi->num_queue_pairs; i++) { |
| 6727 | 6730 | i40e_clean_tx_ring(vsi->tx_rings[i]); |
| 6728 | − if (i40e_enabled_xdp_vsi(vsi)) | |
| 6731 | + if (i40e_enabled_xdp_vsi(vsi)) { | |
| 6732 | + /* Make sure that in-progress ndo_xdp_xmit | |
| 6733 | + * calls are completed. | |
| 6734 | + */ | |
| 6735 | + synchronize_rcu(); | |
| 6729 | 6736 | i40e_clean_tx_ring(vsi->xdp_rings[i]); |
| 6737 | + } | |
| 6730 | 6738 | i40e_clean_rx_ring(vsi->rx_rings[i]); |
| 6731 | 6739 | } |
| 6732 | 6740 | |
| @@ -11895,6 +11903,14 @@ static int i40e_xdp_setup(struct i40e_vsi *vsi, · 5162 unmodified lines | ||
| 11895 | 11903 | if (old_prog) |
| 11896 | 11904 | bpf_prog_put(old_prog); |
| 11897 | 11905 | |
| 11906 | + /* Kick start the NAPI context if there is an AF_XDP socket open | |
| 11907 | + * on that queue id. This so that receiving will start. | |
| 11908 | + */ | |
| 11909 | + if (need_reset && prog) | |
| 11910 | + for (i = 0; i < vsi->num_queue_pairs; i++) | |
| 11911 | + if (vsi->xdp_rings[i]->xsk_umem) | |
| 11912 | + (void)i40e_xsk_async_xmit(vsi->netdev, i); | |
| 11913 | + | |
| 11898 | 11914 | return 0; |
| 11899 | 11915 | } |
| 11900 | 11916 | |
| @@ -11955,8 +11971,13 @@ static void i40e_queue_pair_reset_stats(struct i40e_vsi *vsi, int queue_pair) · 54 unmodified lines | ||
| 11955 | 11971 | static void i40e_queue_pair_clean_rings(struct i40e_vsi *vsi, int queue_pair) |
| 11956 | 11972 | { |
| 11957 | 11973 | i40e_clean_tx_ring(vsi->tx_rings[queue_pair]); |
| 11958 | − if (i40e_enabled_xdp_vsi(vsi)) | |
| 11974 | + if (i40e_enabled_xdp_vsi(vsi)) { | |
| 11975 | + /* Make sure that in-progress ndo_xdp_xmit calls are | |
| 11976 | + * completed. | |
| 11977 | + */ | |
| 11978 | + synchronize_rcu(); | |
| 11959 | 11979 | i40e_clean_tx_ring(vsi->xdp_rings[queue_pair]); |
| 11980 | + } | |
| 11960 | 11981 | i40e_clean_rx_ring(vsi->rx_rings[queue_pair]); |
| 11961 | 11982 | } |
| 11962 | 11983 | |
| @@ -3709,6 +3709,7 @@ int i40e_xdp_xmit(struct net_device *dev, int n, struct xdp_frame **frames, · 3708 unmodified lines | ||
| 3709 | 3709 | struct i40e_netdev_priv *np = netdev_priv(dev); |
| 3710 | 3710 | unsigned int queue_index = smp_processor_id(); |
| 3711 | 3711 | struct i40e_vsi *vsi = np->vsi; |
| 3712 | + struct i40e_pf *pf = vsi->back; | |
| 3712 | 3713 | struct i40e_ring *xdp_ring; |
| 3713 | 3714 | int drops = 0; |
| 3714 | 3715 | int i; |
| @@ -3716,7 +3717,8 @@ int i40e_xdp_xmit(struct net_device *dev, int n, struct xdp_frame **frames, · 1 unmodified line | ||
| 3716 | 3717 | if (test_bit(__I40E_VSI_DOWN, vsi->state)) |
| 3717 | 3718 | return -ENETDOWN; |
| 3718 | 3719 | |
| 3719 | − if (!i40e_enabled_xdp_vsi(vsi) || queue_index >= vsi->num_queue_pairs) | |
| 3720 | + if (!i40e_enabled_xdp_vsi(vsi) || queue_index >= vsi->num_queue_pairs || | |
| 3721 | + test_bit(__I40E_CONFIG_BUSY, pf->state)) | |
| 3720 | 3722 | return -ENXIO; |
| 3721 | 3723 | |
| 3722 | 3724 | if (unlikely(flags & ~XDP_XMIT_FLAGS_MASK)) |
| @@ -183,6 +183,11 @@ static int i40e_xsk_umem_enable(struct i40e_vsi *vsi, struct xdp_umem *umem, · 182 unmodified lines | ||
| 183 | 183 | err = i40e_queue_pair_enable(vsi, qid); |
| 184 | 184 | if (err) |
| 185 | 185 | return err; |
| 186 | + | |
| 187 | + /* Kick start the NAPI context so that receiving will start */ | |
| 188 | + err = i40e_xsk_async_xmit(vsi->netdev, qid); | |
| 189 | + if (err) | |
| 190 | + return err; | |
| 186 | 191 | } |
| 187 | 192 | |
| 188 | 193 | return 0; |
| @@ -3953,8 +3953,11 @@ static void ixgbe_setup_mrqc(struct ixgbe_adapter *adapter) · 3952 unmodified lines | ||
| 3953 | 3953 | else |
| 3954 | 3954 | mrqc = IXGBE_MRQC_VMDQRSS64EN; |
| 3955 | 3955 | |
| 3956 | − /* Enable L3/L4 for Tx Switched packets */ | |
| 3957 | − mrqc |= IXGBE_MRQC_L3L4TXSWEN; | |
| 3956 | + /* Enable L3/L4 for Tx Switched packets only for X550, | |
| 3957 | + * older devices do not support this feature | |
| 3958 | + */ | |
| 3959 | + if (hw->mac.type >= ixgbe_mac_X550) | |
| 3960 | + mrqc |= IXGBE_MRQC_L3L4TXSWEN; | |
| 3958 | 3961 | } else { |
| 3959 | 3962 | if (tcs > 4) |
| 3960 | 3963 | mrqc = IXGBE_MRQC_RTRSS8TCEN; |
| @@ -10225,6 +10228,7 @@ static int ixgbe_xdp_setup(struct net_device *dev, struct bpf_prog *prog) · 6264 unmodified lines | ||
| 10225 | 10228 | int i, frame_size = dev->mtu + ETH_HLEN + ETH_FCS_LEN + VLAN_HLEN; |
| 10226 | 10229 | struct ixgbe_adapter *adapter = netdev_priv(dev); |
| 10227 | 10230 | struct bpf_prog *old_prog; |
| 10231 | + bool need_reset; | |
| 10228 | 10232 | |
| 10229 | 10233 | if (adapter->flags & IXGBE_FLAG_SRIOV_ENABLED) |
| 10230 | 10234 | return -EINVAL; |
| @@ -10247,9 +10251,10 @@ static int ixgbe_xdp_setup(struct net_device *dev, struct bpf_prog *prog) · 16 unmodified lines | ||
| 10247 | 10251 | return -ENOMEM; |
| 10248 | 10252 | |
| 10249 | 10253 | old_prog = xchg(&adapter->xdp_prog, prog); |
| 10254 | + need_reset = (!!prog != !!old_prog); | |
| 10250 | 10255 | |
| 10251 | 10256 | /* If transitioning XDP modes reconfigure rings */ |
| 10252 | − if (!!prog != !!old_prog) { | |
| 10257 | + if (need_reset) { | |
| 10253 | 10258 | int err = ixgbe_setup_tc(dev, adapter->hw_tcs); |
| 10254 | 10259 | |
| 10255 | 10260 | if (err) { |
| @@ -10265,6 +10270,14 @@ static int ixgbe_xdp_setup(struct net_device *dev, struct bpf_prog *prog) · 9 unmodified lines | ||
| 10265 | 10270 | if (old_prog) |
| 10266 | 10271 | bpf_prog_put(old_prog); |
| 10267 | 10272 | |
| 10273 | + /* Kick start the NAPI context if there is an AF_XDP socket open | |
| 10274 | + * on that queue id. This so that receiving will start. | |
| 10275 | + */ | |
| 10276 | + if (need_reset && prog) | |
| 10277 | + for (i = 0; i < adapter->num_rx_queues; i++) | |
| 10278 | + if (adapter->xdp_ring[i]->xsk_umem) | |
| 10279 | + (void)ixgbe_xsk_async_xmit(adapter->netdev, i); | |
| 10280 | + | |
| 10268 | 10281 | return 0; |
| 10269 | 10282 | } |
| 10270 | 10283 | |
| @@ -144,11 +144,19 @@ static int ixgbe_xsk_umem_enable(struct ixgbe_adapter *adapter, · 143 unmodified lines | ||
| 144 | 144 | ixgbe_txrx_ring_disable(adapter, qid); |
| 145 | 145 | |
| 146 | 146 | err = ixgbe_add_xsk_umem(adapter, umem, qid); |
| 147 | + if (err) | |
| 148 | + return err; | |
| 147 | 149 | |
| 148 | − if (if_running) | |
| 150 | + if (if_running) { | |
| 149 | 151 | ixgbe_txrx_ring_enable(adapter, qid); |
| 150 | 152 | |
| 151 | − return err; | |
| 153 | + /* Kick start the NAPI context so that receiving will start */ | |
| 154 | + err = ixgbe_xsk_async_xmit(adapter->netdev, qid); | |
| 155 | + if (err) | |
| 156 | + return err; | |
| 157 | + } | |
| 158 | + | |
| 159 | + return 0; | |
| 152 | 160 | } |
| 153 | 161 | |
| 154 | 162 | static int ixgbe_xsk_umem_disable(struct ixgbe_adapter *adapter, u16 qid) |
| @@ -634,7 +642,8 @@ static bool ixgbe_xmit_zc(struct ixgbe_ring *xdp_ring, unsigned int budget) · 479 unmodified lines | ||
| 634 | 642 | dma_addr_t dma; |
| 635 | 643 | |
| 636 | 644 | while (budget-- > 0) { |
| 637 | − if (unlikely(!ixgbe_desc_unused(xdp_ring))) { | |
| 645 | + if (unlikely(!ixgbe_desc_unused(xdp_ring)) || | |
| 646 | + !netif_carrier_ok(xdp_ring->netdev)) { | |
| 638 | 647 | work_done = false; |
| 639 | 648 | break; |
| 640 | 649 | } |
| @@ -2879,7 +2879,7 @@ static int mv643xx_eth_shared_probe(struct platform_device *pdev) · 2878 unmodified lines | ||
| 2879 | 2879 | |
| 2880 | 2880 | ret = mv643xx_eth_shared_of_probe(pdev); |
| 2881 | 2881 | if (ret) |
| 2882 | − return ret; | |
| 2882 | + goto err_put_clk; | |
| 2883 | 2883 | pd = dev_get_platdata(&pdev->dev); |
| 2884 | 2884 | |
| 2885 | 2885 | msp->tx_csum_limit = (pd != NULL && pd->tx_csum_limit) ? |
| @@ -2887,6 +2887,11 @@ static int mv643xx_eth_shared_probe(struct platform_device *pdev) · 1 unmodified line | ||
| 2887 | 2887 | infer_hw_params(msp); |
| 2888 | 2888 | |
| 2889 | 2889 | return 0; |
| 2890 | + | |
| 2891 | +err_put_clk: | |
| 2892 | + if (!IS_ERR(msp->clk)) | |
| 2893 | + clk_disable_unprepare(msp->clk); | |
| 2894 | + return ret; | |
| 2890 | 2895 | } |
| 2891 | 2896 | |
| 2892 | 2897 | static int mv643xx_eth_shared_remove(struct platform_device *pdev) |
| @@ -2146,7 +2146,7 @@ err_drop_frame: · 2145 unmodified lines | ||
| 2146 | 2146 | if (unlikely(!skb)) |
| 2147 | 2147 | goto err_drop_frame_ret_pool; |
| 2148 | 2148 | |
| 2149 | − dma_sync_single_range_for_cpu(dev->dev.parent, | |
| 2149 | + dma_sync_single_range_for_cpu(&pp->bm_priv->pdev->dev, | |
| 2150 | 2150 | rx_desc->buf_phys_addr, |
| 2151 | 2151 | MVNETA_MH_SIZE + NET_SKB_PAD, |
| 2152 | 2152 | rx_bytes, |
| @@ -5073,7 +5073,7 @@ static int sky2_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 5072 unmodified lines | ||
| 5073 | 5073 | INIT_WORK(&hw->restart_work, sky2_restart); |
| 5074 | 5074 | |
| 5075 | 5075 | pci_set_drvdata(pdev, hw); |
| 5076 | − pdev->d3_delay = 200; | |
| 5076 | + pdev->d3_delay = 300; | |
| 5077 | 5077 | |
| 5078 | 5078 | return 0; |
| 5079 | 5079 | |
| @@ -3360,7 +3360,7 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port, · 3359 unmodified lines | ||
| 3360 | 3360 | dev->addr_len = ETH_ALEN; |
| 3361 | 3361 | mlx4_en_u64_to_mac(dev->dev_addr, mdev->dev->caps.def_mac[priv->port]); |
| 3362 | 3362 | if (!is_valid_ether_addr(dev->dev_addr)) { |
| 3363 | − en_err(priv, "Port: %d, invalid mac burned: %pM, quiting\n", | |
| 3363 | + en_err(priv, "Port: %d, invalid mac burned: %pM, quitting\n", | |
| 3364 | 3364 | priv->port, dev->dev_addr); |
| 3365 | 3365 | err = -EINVAL; |
| 3366 | 3366 | goto out; |
| @@ -862,8 +862,9 @@ int __mlxsw_sp_port_headroom_set(struct mlxsw_sp_port *mlxsw_sp_port, int mtu, · 861 unmodified lines | ||
| 862 | 862 | for (i = 0; i < IEEE_8021QAZ_MAX_TCS; i++) { |
| 863 | 863 | bool configure = false; |
| 864 | 864 | bool pfc = false; |
| 865 | + u16 thres_cells; | |
| 866 | + u16 delay_cells; | |
| 865 | 867 | bool lossy; |
| 866 | − u16 thres; | |
| 867 | 868 | |
| 868 | 869 | for (j = 0; j < IEEE_8021QAZ_MAX_TCS; j++) { |
| 869 | 870 | if (prio_tc[j] == i) { |
| @@ -877,10 +878,11 @@ int __mlxsw_sp_port_headroom_set(struct mlxsw_sp_port *mlxsw_sp_port, int mtu, · 7 unmodified lines | ||
| 877 | 878 | continue; |
| 878 | 879 | |
| 879 | 880 | lossy = !(pfc || pause_en); |
| 880 | − thres = mlxsw_sp_pg_buf_threshold_get(mlxsw_sp, mtu); | |
| 881 | − delay = mlxsw_sp_pg_buf_delay_get(mlxsw_sp, mtu, delay, pfc, | |
| 882 | − pause_en); | |
| 883 | − mlxsw_sp_pg_buf_pack(pbmc_pl, i, thres + delay, thres, lossy); | |
| 881 | + thres_cells = mlxsw_sp_pg_buf_threshold_get(mlxsw_sp, mtu); | |
| 882 | + delay_cells = mlxsw_sp_pg_buf_delay_get(mlxsw_sp, mtu, delay, | |
| 883 | + pfc, pause_en); | |
| 884 | + mlxsw_sp_pg_buf_pack(pbmc_pl, i, thres_cells + delay_cells, | |
| 885 | + thres_cells, lossy); | |
| 884 | 886 | } |
| 885 | 887 | |
| 886 | 888 | return mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(pbmc), pbmc_pl); |
| @@ -1681,5 +1681,5 @@ MODULE_DESCRIPTION(DRV_NAME " ethernet driver"); · 1680 unmodified lines | ||
| 1681 | 1681 | MODULE_AUTHOR("Claudio Lanconelli <lanconelli.claudio@eptar.com>"); |
| 1682 | 1682 | MODULE_LICENSE("GPL"); |
| 1683 | 1683 | module_param_named(debug, debug.msg_enable, int, 0); |
| 1684 | −MODULE_PARM_DESC(debug, "Debug verbosity level (0=none, ..., ffff=all)"); | |
| 1684 | +MODULE_PARM_DESC(debug, "Debug verbosity level in amount of bits set (0=none, ..., 31=all)"); | |
| 1685 | 1685 | MODULE_ALIAS("spi:" DRV_NAME); |
| @@ -1400,7 +1400,8 @@ static int lan743x_tx_frame_start(struct lan743x_tx *tx, · 1399 unmodified lines | ||
| 1400 | 1400 | } |
| 1401 | 1401 | |
| 1402 | 1402 | static void lan743x_tx_frame_add_lso(struct lan743x_tx *tx, |
| 1403 | − unsigned int frame_length) | |
| 1403 | + unsigned int frame_length, | |
| 1404 | + int nr_frags) | |
| 1404 | 1405 | { |
| 1405 | 1406 | /* called only from within lan743x_tx_xmit_frame. |
| 1406 | 1407 | * assuming tx->ring_lock has already been acquired. |
| @@ -1410,6 +1411,10 @@ static void lan743x_tx_frame_add_lso(struct lan743x_tx *tx, · 3 unmodified lines | ||
| 1410 | 1411 | |
| 1411 | 1412 | /* wrap up previous descriptor */ |
| 1412 | 1413 | tx->frame_data0 |= TX_DESC_DATA0_EXT_; |
| 1414 | + if (nr_frags <= 0) { | |
| 1415 | + tx->frame_data0 |= TX_DESC_DATA0_LS_; | |
| 1416 | + tx->frame_data0 |= TX_DESC_DATA0_IOC_; | |
| 1417 | + } | |
| 1413 | 1418 | tx_descriptor = &tx->ring_cpu_ptr[tx->frame_tail]; |
| 1414 | 1419 | tx_descriptor->data0 = tx->frame_data0; |
| 1415 | 1420 | |
| @@ -1514,8 +1519,11 @@ static void lan743x_tx_frame_end(struct lan743x_tx *tx, · 98 unmodified lines | ||
| 1514 | 1519 | u32 tx_tail_flags = 0; |
| 1515 | 1520 | |
| 1516 | 1521 | /* wrap up previous descriptor */ |
| 1517 | − tx->frame_data0 |= TX_DESC_DATA0_LS_; | |
| 1518 | − tx->frame_data0 |= TX_DESC_DATA0_IOC_; | |
| 1522 | + if ((tx->frame_data0 & TX_DESC_DATA0_DTYPE_MASK_) == | |
| 1523 | + TX_DESC_DATA0_DTYPE_DATA_) { | |
| 1524 | + tx->frame_data0 |= TX_DESC_DATA0_LS_; | |
| 1525 | + tx->frame_data0 |= TX_DESC_DATA0_IOC_; | |
| 1526 | + } | |
| 1519 | 1527 | |
| 1520 | 1528 | tx_descriptor = &tx->ring_cpu_ptr[tx->frame_tail]; |
| 1521 | 1529 | buffer_info = &tx->buffer_info[tx->frame_tail]; |
| @@ -1600,7 +1608,7 @@ static netdev_tx_t lan743x_tx_xmit_frame(struct lan743x_tx *tx, · 78 unmodified lines | ||
| 1600 | 1608 | } |
| 1601 | 1609 | |
| 1602 | 1610 | if (gso) |
| 1603 | − lan743x_tx_frame_add_lso(tx, frame_length); | |
| 1611 | + lan743x_tx_frame_add_lso(tx, frame_length, nr_frags); | |
| 1604 | 1612 | |
| 1605 | 1613 | if (nr_frags <= 0) |
| 1606 | 1614 | goto finish; |
| @@ -1291,15 +1291,10 @@ wrp_alu64_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, · 1290 unmodified lines | ||
| 1291 | 1291 | |
| 1292 | 1292 | static int |
| 1293 | 1293 | wrp_alu32_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, |
| 1294 | − enum alu_op alu_op, bool skip) | |
| 1294 | + enum alu_op alu_op) | |
| 1295 | 1295 | { |
| 1296 | 1296 | const struct bpf_insn *insn = &meta->insn; |
| 1297 | 1297 | |
| 1298 | − if (skip) { | |
| 1299 | − meta->skip = true; | |
| 1300 | − return 0; | |
| 1301 | − } | |
| 1302 | − | |
| 1303 | 1298 | wrp_alu_imm(nfp_prog, insn->dst_reg * 2, alu_op, insn->imm); |
| 1304 | 1299 | wrp_immed(nfp_prog, reg_both(insn->dst_reg * 2 + 1), 0); |
| 1305 | 1300 | |
| @@ -2309,7 +2304,7 @@ static int xor_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) · 1003 unmodified lines | ||
| 2309 | 2304 | |
| 2310 | 2305 | static int xor_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| 2311 | 2306 | { |
| 2312 | − return wrp_alu32_imm(nfp_prog, meta, ALU_OP_XOR, !~meta->insn.imm); | |
| 2307 | + return wrp_alu32_imm(nfp_prog, meta, ALU_OP_XOR); | |
| 2313 | 2308 | } |
| 2314 | 2309 | |
| 2315 | 2310 | static int and_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| @@ -2319,7 +2314,7 @@ static int and_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) · 3 unmodified lines | ||
| 2319 | 2314 | |
| 2320 | 2315 | static int and_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| 2321 | 2316 | { |
| 2322 | − return wrp_alu32_imm(nfp_prog, meta, ALU_OP_AND, !~meta->insn.imm); | |
| 2317 | + return wrp_alu32_imm(nfp_prog, meta, ALU_OP_AND); | |
| 2323 | 2318 | } |
| 2324 | 2319 | |
| 2325 | 2320 | static int or_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| @@ -2329,7 +2324,7 @@ static int or_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) · 3 unmodified lines | ||
| 2329 | 2324 | |
| 2330 | 2325 | static int or_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| 2331 | 2326 | { |
| 2332 | − return wrp_alu32_imm(nfp_prog, meta, ALU_OP_OR, !meta->insn.imm); | |
| 2327 | + return wrp_alu32_imm(nfp_prog, meta, ALU_OP_OR); | |
| 2333 | 2328 | } |
| 2334 | 2329 | |
| 2335 | 2330 | static int add_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| @@ -2339,7 +2334,7 @@ static int add_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) · 3 unmodified lines | ||
| 2339 | 2334 | |
| 2340 | 2335 | static int add_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| 2341 | 2336 | { |
| 2342 | − return wrp_alu32_imm(nfp_prog, meta, ALU_OP_ADD, !meta->insn.imm); | |
| 2337 | + return wrp_alu32_imm(nfp_prog, meta, ALU_OP_ADD); | |
| 2343 | 2338 | } |
| 2344 | 2339 | |
| 2345 | 2340 | static int sub_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| @@ -2349,7 +2344,7 @@ static int sub_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) · 3 unmodified lines | ||
| 2349 | 2344 | |
| 2350 | 2345 | static int sub_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| 2351 | 2346 | { |
| 2352 | − return wrp_alu32_imm(nfp_prog, meta, ALU_OP_SUB, !meta->insn.imm); | |
| 2347 | + return wrp_alu32_imm(nfp_prog, meta, ALU_OP_SUB); | |
| 2353 | 2348 | } |
| 2354 | 2349 | |
| 2355 | 2350 | static int mul_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| @@ -1688,6 +1688,15 @@ qed_iwarp_parse_rx_pkt(struct qed_hwfn *p_hwfn, · 1687 unmodified lines | ||
| 1688 | 1688 | |
| 1689 | 1689 | eth_hlen = ETH_HLEN + (vlan_valid ? sizeof(u32) : 0); |
| 1690 | 1690 | |
| 1691 | + if (!ether_addr_equal(ethh->h_dest, | |
| 1692 | + p_hwfn->p_rdma_info->iwarp.mac_addr)) { | |
| 1693 | + DP_VERBOSE(p_hwfn, | |
| 1694 | + QED_MSG_RDMA, | |
| 1695 | + "Got unexpected mac %pM instead of %pM\n", | |
| 1696 | + ethh->h_dest, p_hwfn->p_rdma_info->iwarp.mac_addr); | |
| 1697 | + return -EINVAL; | |
| 1698 | + } | |
| 1699 | + | |
| 1691 | 1700 | ether_addr_copy(remote_mac_addr, ethh->h_source); |
| 1692 | 1701 | ether_addr_copy(local_mac_addr, ethh->h_dest); |
| 1693 | 1702 | |
| @@ -2605,7 +2614,7 @@ qed_iwarp_ll2_start(struct qed_hwfn *p_hwfn, · 911 unmodified lines | ||
| 2605 | 2614 | struct qed_iwarp_info *iwarp_info; |
| 2606 | 2615 | struct qed_ll2_acquire_data data; |
| 2607 | 2616 | struct qed_ll2_cbs cbs; |
| 2608 | − u32 mpa_buff_size; | |
| 2617 | + u32 buff_size; | |
| 2609 | 2618 | u16 n_ooo_bufs; |
| 2610 | 2619 | int rc = 0; |
| 2611 | 2620 | int i; |
| @@ -2632,7 +2641,7 @@ qed_iwarp_ll2_start(struct qed_hwfn *p_hwfn, · 20 unmodified lines | ||
| 2632 | 2641 | |
| 2633 | 2642 | memset(&data, 0, sizeof(data)); |
| 2634 | 2643 | data.input.conn_type = QED_LL2_TYPE_IWARP; |
| 2635 | − data.input.mtu = QED_IWARP_MAX_SYN_PKT_SIZE; | |
| 2644 | + data.input.mtu = params->max_mtu; | |
| 2636 | 2645 | data.input.rx_num_desc = QED_IWARP_LL2_SYN_RX_SIZE; |
| 2637 | 2646 | data.input.tx_num_desc = QED_IWARP_LL2_SYN_TX_SIZE; |
| 2638 | 2647 | data.input.tx_max_bds_per_packet = 1; /* will never be fragmented */ |
| @@ -2654,9 +2663,10 @@ qed_iwarp_ll2_start(struct qed_hwfn *p_hwfn, · 15 unmodified lines | ||
| 2654 | 2663 | goto err; |
| 2655 | 2664 | } |
| 2656 | 2665 | |
| 2666 | + buff_size = QED_IWARP_MAX_BUF_SIZE(params->max_mtu); | |
| 2657 | 2667 | rc = qed_iwarp_ll2_alloc_buffers(p_hwfn, |
| 2658 | 2668 | QED_IWARP_LL2_SYN_RX_SIZE, |
| 2659 | − QED_IWARP_MAX_SYN_PKT_SIZE, | |
| 2669 | + buff_size, | |
| 2660 | 2670 | iwarp_info->ll2_syn_handle); |
| 2661 | 2671 | if (rc) |
| 2662 | 2672 | goto err; |
| @@ -2710,10 +2720,9 @@ qed_iwarp_ll2_start(struct qed_hwfn *p_hwfn, · 47 unmodified lines | ||
| 2710 | 2720 | if (rc) |
| 2711 | 2721 | goto err; |
| 2712 | 2722 | |
| 2713 | − mpa_buff_size = QED_IWARP_MAX_BUF_SIZE(params->max_mtu); | |
| 2714 | 2723 | rc = qed_iwarp_ll2_alloc_buffers(p_hwfn, |
| 2715 | 2724 | data.input.rx_num_desc, |
| 2716 | − mpa_buff_size, | |
| 2725 | + buff_size, | |
| 2717 | 2726 | iwarp_info->ll2_mpa_handle); |
| 2718 | 2727 | if (rc) |
| 2719 | 2728 | goto err; |
| @@ -2726,7 +2735,7 @@ qed_iwarp_ll2_start(struct qed_hwfn *p_hwfn, · 6 unmodified lines | ||
| 2726 | 2735 | |
| 2727 | 2736 | iwarp_info->max_num_partial_fpdus = (u16)p_hwfn->p_rdma_info->num_qps; |
| 2728 | 2737 | |
| 2729 | − iwarp_info->mpa_intermediate_buf = kzalloc(mpa_buff_size, GFP_KERNEL); | |
| 2738 | + iwarp_info->mpa_intermediate_buf = kzalloc(buff_size, GFP_KERNEL); | |
| 2730 | 2739 | if (!iwarp_info->mpa_intermediate_buf) |
| 2731 | 2740 | goto err; |
| 2732 | 2741 | |
| @@ -46,7 +46,6 @@ enum qed_iwarp_qp_state qed_roce2iwarp_state(enum qed_roce_qp_state state); · 45 unmodified lines | ||
| 46 | 46 | |
| 47 | 47 | #define QED_IWARP_LL2_SYN_TX_SIZE (128) |
| 48 | 48 | #define QED_IWARP_LL2_SYN_RX_SIZE (256) |
| 49 | −#define QED_IWARP_MAX_SYN_PKT_SIZE (128) | |
| 50 | 49 | |
| 51 | 50 | #define QED_IWARP_LL2_OOO_DEF_TX_SIZE (256) |
| 52 | 51 | #define QED_IWARP_MAX_OOO (16) |
| @@ -241,15 +241,18 @@ static inline void dwmac4_get_timestamp(void *desc, u32 ats, u64 *ts) · 240 unmodified lines | ||
| 241 | 241 | static int dwmac4_rx_check_timestamp(void *desc) |
| 242 | 242 | { |
| 243 | 243 | struct dma_desc *p = (struct dma_desc *)desc; |
| 244 | + unsigned int rdes0 = le32_to_cpu(p->des0); | |
| 245 | + unsigned int rdes1 = le32_to_cpu(p->des1); | |
| 246 | + unsigned int rdes3 = le32_to_cpu(p->des3); | |
| 244 | 247 | u32 own, ctxt; |
| 245 | 248 | int ret = 1; |
| 246 | 249 | |
| 247 | − own = p->des3 & RDES3_OWN; | |
| 248 | − ctxt = ((p->des3 & RDES3_CONTEXT_DESCRIPTOR) | |
| 250 | + own = rdes3 & RDES3_OWN; | |
| 251 | + ctxt = ((rdes3 & RDES3_CONTEXT_DESCRIPTOR) | |
| 249 | 252 | >> RDES3_CONTEXT_DESCRIPTOR_SHIFT); |
| 250 | 253 | |
| 251 | 254 | if (likely(!own && ctxt)) { |
| 252 | − if ((p->des0 == 0xffffffff) && (p->des1 == 0xffffffff)) | |
| 255 | + if ((rdes0 == 0xffffffff) && (rdes1 == 0xffffffff)) | |
| 253 | 256 | /* Corrupted value */ |
| 254 | 257 | ret = -EINVAL; |
| 255 | 258 | else |
| @@ -696,25 +696,27 @@ static int stmmac_ethtool_op_set_eee(struct net_device *dev, · 695 unmodified lines | ||
| 696 | 696 | struct ethtool_eee *edata) |
| 697 | 697 | { |
| 698 | 698 | struct stmmac_priv *priv = netdev_priv(dev); |
| 699 | + int ret; | |
| 699 | 700 | |
| 700 | − priv->eee_enabled = edata->eee_enabled; | |
| 701 | − | |
| 702 | − if (!priv->eee_enabled) | |
| 701 | + if (!edata->eee_enabled) { | |
| 703 | 702 | stmmac_disable_eee_mode(priv); |
| 704 | − else { | |
| 703 | + } else { | |
| 705 | 704 | /* We are asking for enabling the EEE but it is safe |
| 706 | 705 | * to verify all by invoking the eee_init function. |
| 707 | 706 | * In case of failure it will return an error. |
| 708 | 707 | */ |
| 709 | − priv->eee_enabled = stmmac_eee_init(priv); | |
| 710 | − if (!priv->eee_enabled) | |
| 708 | + edata->eee_enabled = stmmac_eee_init(priv); | |
| 709 | + if (!edata->eee_enabled) | |
| 711 | 710 | return -EOPNOTSUPP; |
| 712 | − | |
| 713 | − /* Do not change tx_lpi_timer in case of failure */ | |
| 714 | − priv->tx_lpi_timer = edata->tx_lpi_timer; | |
| 715 | 711 | } |
| 716 | 712 | |
| 717 | − return phy_ethtool_set_eee(dev->phydev, edata); | |
| 713 | + ret = phy_ethtool_set_eee(dev->phydev, edata); | |
| 714 | + if (ret) | |
| 715 | + return ret; | |
| 716 | + | |
| 717 | + priv->eee_enabled = edata->eee_enabled; | |
| 718 | + priv->tx_lpi_timer = edata->tx_lpi_timer; | |
| 719 | + return 0; | |
| 718 | 720 | } |
| 719 | 721 | |
| 720 | 722 | static u32 stmmac_usec2riwt(u32 usec, struct stmmac_priv *priv) |
| @@ -259,7 +259,7 @@ static int netcp_module_probe(struct netcp_device *netcp_device, · 258 unmodified lines | ||
| 259 | 259 | const char *name; |
| 260 | 260 | char node_name[32]; |
| 261 | 261 | |
| 262 | − if (of_property_read_string(node, "label", &name) < 0) { | |
| 262 | + if (of_property_read_string(child, "label", &name) < 0) { | |
| 263 | 263 | snprintf(node_name, sizeof(node_name), "%pOFn", child); |
| 264 | 264 | name = node_name; |
| 265 | 265 | } |
| @@ -692,15 +692,20 @@ out: · 691 unmodified lines | ||
| 692 | 692 | static int geneve_open(struct net_device *dev) |
| 693 | 693 | { |
| 694 | 694 | struct geneve_dev *geneve = netdev_priv(dev); |
| 695 | − bool ipv6 = !!(geneve->info.mode & IP_TUNNEL_INFO_IPV6); | |
| 696 | 695 | bool metadata = geneve->collect_md; |
| 696 | + bool ipv4, ipv6; | |
| 697 | 697 | int ret = 0; |
| 698 | 698 | |
| 699 | + ipv6 = geneve->info.mode & IP_TUNNEL_INFO_IPV6 || metadata; | |
| 700 | + ipv4 = !ipv6 || metadata; | |
| 699 | 701 | #if IS_ENABLED(CONFIG_IPV6) |
| 700 | − if (ipv6 || metadata) | |
| 702 | + if (ipv6) { | |
| 701 | 703 | ret = geneve_sock_add(geneve, true); |
| 704 | + if (ret < 0 && ret != -EAFNOSUPPORT) | |
| 705 | + ipv4 = false; | |
| 706 | + } | |
| 702 | 707 | #endif |
| 703 | − if (!ret && (!ipv6 || metadata)) | |
| 708 | + if (ipv4) | |
| 704 | 709 | ret = geneve_sock_add(geneve, false); |
| 705 | 710 | if (ret < 0) |
| 706 | 711 | geneve_sock_release(geneve); |
| @@ -744,6 +744,14 @@ void netvsc_linkstatus_callback(struct net_device *net, · 743 unmodified lines | ||
| 744 | 744 | schedule_delayed_work(&ndev_ctx->dwork, 0); |
| 745 | 745 | } |
| 746 | 746 | |
| 747 | +static void netvsc_comp_ipcsum(struct sk_buff *skb) | |
| 748 | +{ | |
| 749 | + struct iphdr *iph = (struct iphdr *)skb->data; | |
| 750 | + | |
| 751 | + iph->check = 0; | |
| 752 | + iph->check = ip_fast_csum(iph, iph->ihl); | |
| 753 | +} | |
| 754 | + | |
| 747 | 755 | static struct sk_buff *netvsc_alloc_recv_skb(struct net_device *net, |
| 748 | 756 | struct netvsc_channel *nvchan) |
| 749 | 757 | { |
| @@ -770,9 +778,17 @@ static struct sk_buff *netvsc_alloc_recv_skb(struct net_device *net, · 20 unmodified lines | ||
| 770 | 778 | /* skb is already created with CHECKSUM_NONE */ |
| 771 | 779 | skb_checksum_none_assert(skb); |
| 772 | 780 | |
| 773 | − /* | |
| 774 | − * In Linux, the IP checksum is always checked. | |
| 775 | − * Do L4 checksum offload if enabled and present. | |
| 781 | + /* Incoming packets may have IP header checksum verified by the host. | |
| 782 | + * They may not have IP header checksum computed after coalescing. | |
| 783 | + * We compute it here if the flags are set, because on Linux, the IP | |
| 784 | + * checksum is always checked. | |
| 785 | + */ | |
| 786 | + if (csum_info && csum_info->receive.ip_checksum_value_invalid && | |
| 787 | + csum_info->receive.ip_checksum_succeeded && | |
| 788 | + skb->protocol == htons(ETH_P_IP)) | |
| 789 | + netvsc_comp_ipcsum(skb); | |
| 790 | + | |
| 791 | + /* Do L4 checksum offload if enabled and present. | |
| 776 | 792 | */ |
| 777 | 793 | if (csum_info && (net->features & NETIF_F_RXCSUM)) { |
| 778 | 794 | if (csum_info->receive.tcp_checksum_succeeded || |
| @@ -499,6 +499,8 @@ static int ipvlan_nl_changelink(struct net_device *dev, · 498 unmodified lines | ||
| 499 | 499 | |
| 500 | 500 | if (!data) |
| 501 | 501 | return 0; |
| 502 | + if (!ns_capable(dev_net(ipvlan->phy_dev)->user_ns, CAP_NET_ADMIN)) | |
| 503 | + return -EPERM; | |
| 502 | 504 | |
| 503 | 505 | if (data[IFLA_IPVLAN_MODE]) { |
| 504 | 506 | u16 nmode = nla_get_u16(data[IFLA_IPVLAN_MODE]); |
| @@ -601,6 +603,8 @@ int ipvlan_link_new(struct net *src_net, struct net_device *dev, · 96 unmodified lines | ||
| 601 | 603 | struct ipvl_dev *tmp = netdev_priv(phy_dev); |
| 602 | 604 | |
| 603 | 605 | phy_dev = tmp->phy_dev; |
| 606 | + if (!ns_capable(dev_net(phy_dev)->user_ns, CAP_NET_ADMIN)) | |
| 607 | + return -EPERM; | |
| 604 | 608 | } else if (!netif_is_ipvlan_port(phy_dev)) { |
| 605 | 609 | /* Exit early if the underlying link is invalid or busy */ |
| 606 | 610 | if (phy_dev->type != ARPHRD_ETHER || |
| @@ -19,6 +19,7 @@ · 18 unmodified lines | ||
| 19 | 19 | #include <linux/module.h> |
| 20 | 20 | #include <linux/of.h> |
| 21 | 21 | #include <linux/phy.h> |
| 22 | +#include <linux/delay.h> | |
| 22 | 23 | |
| 23 | 24 | #include <dt-bindings/net/ti-dp83867.h> |
| 24 | 25 | |
| @@ -325,6 +326,8 @@ static int dp83867_phy_reset(struct phy_device *phydev) · 300 unmodified lines | ||
| 325 | 326 | if (err < 0) |
| 326 | 327 | return err; |
| 327 | 328 | |
| 329 | + usleep_range(10, 20); | |
| 330 | + | |
| 328 | 331 | return dp83867_config_init(phydev); |
| 329 | 332 | } |
| 330 | 333 | |
| @@ -26,6 +26,8 @@ · 25 unmodified lines | ||
| 26 | 26 | #include <linux/marvell_phy.h> |
| 27 | 27 | #include <linux/phy.h> |
| 28 | 28 | |
| 29 | +#define MDIO_AN_10GBT_CTRL_ADV_NBT_MASK 0x01e0 | |
| 30 | + | |
| 29 | 31 | enum { |
| 30 | 32 | MV_PCS_BASE_T = 0x0000, |
| 31 | 33 | MV_PCS_BASE_R = 0x1000, |
| @@ -386,8 +388,10 @@ static int mv3310_config_aneg(struct phy_device *phydev) · 354 unmodified lines | ||
| 386 | 388 | else |
| 387 | 389 | reg = 0; |
| 388 | 390 | |
| 391 | + /* Make sure we clear unsupported 2.5G/5G advertising */ | |
| 389 | 392 | ret = mv3310_modify(phydev, MDIO_MMD_AN, MDIO_AN_10GBT_CTRL, |
| 390 | − MDIO_AN_10GBT_CTRL_ADV10G, reg); | |
| 393 | + MDIO_AN_10GBT_CTRL_ADV10G | | |
| 394 | + MDIO_AN_10GBT_CTRL_ADV_NBT_MASK, reg); | |
| 391 | 395 | if (ret < 0) |
| 392 | 396 | return ret; |
| 393 | 397 | if (ret > 0) |
| @@ -379,7 +379,6 @@ int __mdiobus_register(struct mii_bus *bus, struct module *owner) · 378 unmodified lines | ||
| 379 | 379 | err = device_register(&bus->dev); |
| 380 | 380 | if (err) { |
| 381 | 381 | pr_err("mii_bus %s failed to register\n", bus->id); |
| 382 | − put_device(&bus->dev); | |
| 383 | 382 | return -EINVAL; |
| 384 | 383 | } |
| 385 | 384 | |
| @@ -344,6 +344,17 @@ static int ksz8041_config_aneg(struct phy_device *phydev) · 343 unmodified lines | ||
| 344 | 344 | return genphy_config_aneg(phydev); |
| 345 | 345 | } |
| 346 | 346 | |
| 347 | +static int ksz8061_config_init(struct phy_device *phydev) | |
| 348 | +{ | |
| 349 | + int ret; | |
| 350 | + | |
| 351 | + ret = phy_write_mmd(phydev, MDIO_MMD_PMAPMD, MDIO_DEVID1, 0xB61A); | |
| 352 | + if (ret) | |
| 353 | + return ret; | |
| 354 | + | |
| 355 | + return kszphy_config_init(phydev); | |
| 356 | +} | |
| 357 | + | |
| 347 | 358 | static int ksz9021_load_values_from_of(struct phy_device *phydev, |
| 348 | 359 | const struct device_node *of_node, |
| 349 | 360 | u16 reg, |
| @@ -1040,7 +1051,7 @@ static struct phy_driver ksphy_driver[] = { · 690 unmodified lines | ||
| 1040 | 1051 | .name = "Micrel KSZ8061", |
| 1041 | 1052 | .phy_id_mask = MICREL_PHY_ID_MASK, |
| 1042 | 1053 | .features = PHY_BASIC_FEATURES, |
| 1043 | − .config_init = kszphy_config_init, | |
| 1054 | + .config_init = ksz8061_config_init, | |
| 1044 | 1055 | .ack_interrupt = kszphy_ack_interrupt, |
| 1045 | 1056 | .config_intr = kszphy_config_intr, |
| 1046 | 1057 | .suspend = genphy_suspend, |
| @@ -320,6 +320,10 @@ static int phylink_get_mac_state(struct phylink *pl, struct phylink_link_state * · 319 unmodified lines | ||
| 320 | 320 | linkmode_zero(state->lp_advertising); |
| 321 | 321 | state->interface = pl->link_config.interface; |
| 322 | 322 | state->an_enabled = pl->link_config.an_enabled; |
| 323 | + state->speed = SPEED_UNKNOWN; | |
| 324 | + state->duplex = DUPLEX_UNKNOWN; | |
| 325 | + state->pause = MLO_PAUSE_NONE; | |
| 326 | + state->an_complete = 0; | |
| 323 | 327 | state->link = 1; |
| 324 | 328 | |
| 325 | 329 | return pl->ops->mac_link_state(ndev, state); |
| @@ -282,6 +282,13 @@ static struct phy_driver realtek_drvs[] = { · 281 unmodified lines | ||
| 282 | 282 | .name = "RTL8366RB Gigabit Ethernet", |
| 283 | 283 | .features = PHY_GBIT_FEATURES, |
| 284 | 284 | .config_init = &rtl8366rb_config_init, |
| 285 | + /* These interrupts are handled by the irq controller | |
| 286 | + * embedded inside the RTL8366RB, they get unmasked when the | |
| 287 | + * irq is requested and ACKed by reading the status register, | |
| 288 | + * which is done by the irqchip code. | |
| 289 | + */ | |
| 290 | + .ack_interrupt = genphy_no_ack_interrupt, | |
| 291 | + .config_intr = genphy_no_config_intr, | |
| 285 | 292 | .suspend = genphy_suspend, |
| 286 | 293 | .resume = genphy_resume, |
| 287 | 294 | }, |
| @@ -44,7 +44,10 @@ static int xgmiitorgmii_read_status(struct phy_device *phydev) · 43 unmodified lines | ||
| 44 | 44 | u16 val = 0; |
| 45 | 45 | int err; |
| 46 | 46 | |
| 47 | − err = priv->phy_drv->read_status(phydev); | |
| 47 | + if (priv->phy_drv->read_status) | |
| 48 | + err = priv->phy_drv->read_status(phydev); | |
| 49 | + else | |
| 50 | + err = genphy_read_status(phydev); | |
| 48 | 51 | if (err < 0) |
| 49 | 52 | return err; |
| 50 | 53 | |
| @@ -1256,7 +1256,7 @@ static int team_port_add(struct team *team, struct net_device *port_dev, · 1255 unmodified lines | ||
| 1256 | 1256 | list_add_tail_rcu(&port->list, &team->port_list); |
| 1257 | 1257 | team_port_enable(team, port); |
| 1258 | 1258 | __team_compute_features(team); |
| 1259 | − __team_port_change_port_added(port, !!netif_carrier_ok(port_dev)); | |
| 1259 | + __team_port_change_port_added(port, !!netif_oper_up(port_dev)); | |
| 1260 | 1260 | __team_options_change_check(team); |
| 1261 | 1261 | |
| 1262 | 1262 | netdev_info(dev, "Port device %s added\n", portname); |
| @@ -2915,7 +2915,7 @@ static int team_device_event(struct notifier_block *unused, · 1652 unmodified lines | ||
| 2915 | 2915 | |
| 2916 | 2916 | switch (event) { |
| 2917 | 2917 | case NETDEV_UP: |
| 2918 | − if (netif_carrier_ok(dev)) | |
| 2918 | + if (netif_oper_up(dev)) | |
| 2919 | 2919 | team_port_change_check(port, true); |
| 2920 | 2920 | break; |
| 2921 | 2921 | case NETDEV_DOWN: |
| @@ -2167,9 +2167,9 @@ static void *tun_ring_recv(struct tun_file *tfile, int noblock, int *err) · 2166 unmodified lines | ||
| 2167 | 2167 | } |
| 2168 | 2168 | |
| 2169 | 2169 | add_wait_queue(&tfile->wq.wait, &wait); |
| 2170 | − current->state = TASK_INTERRUPTIBLE; | |
| 2171 | 2170 | |
| 2172 | 2171 | while (1) { |
| 2172 | + set_current_state(TASK_INTERRUPTIBLE); | |
| 2173 | 2173 | ptr = ptr_ring_consume(&tfile->tx_ring); |
| 2174 | 2174 | if (ptr) |
| 2175 | 2175 | break; |
| @@ -2185,7 +2185,7 @@ static void *tun_ring_recv(struct tun_file *tfile, int noblock, int *err) · 9 unmodified lines | ||
| 2185 | 2185 | schedule(); |
| 2186 | 2186 | } |
| 2187 | 2187 | |
| 2188 | − current->state = TASK_RUNNING; | |
| 2188 | + __set_current_state(TASK_RUNNING); | |
| 2189 | 2189 | remove_wait_queue(&tfile->wq.wait, &wait); |
| 2190 | 2190 | |
| 2191 | 2191 | out: |
| @@ -1201,8 +1201,8 @@ static const struct usb_device_id products[] = { · 1200 unmodified lines | ||
| 1201 | 1201 | {QMI_FIXED_INTF(0x114f, 0x68a2, 8)}, /* Sierra Wireless MC7750 */ |
| 1202 | 1202 | {QMI_FIXED_INTF(0x1199, 0x68a2, 8)}, /* Sierra Wireless MC7710 in QMI mode */ |
| 1203 | 1203 | {QMI_FIXED_INTF(0x1199, 0x68a2, 19)}, /* Sierra Wireless MC7710 in QMI mode */ |
| 1204 | − {QMI_FIXED_INTF(0x1199, 0x68c0, 8)}, /* Sierra Wireless MC7304/MC7354 */ | |
| 1205 | − {QMI_FIXED_INTF(0x1199, 0x68c0, 10)}, /* Sierra Wireless MC7304/MC7354 */ | |
| 1204 | + {QMI_QUIRK_SET_DTR(0x1199, 0x68c0, 8)}, /* Sierra Wireless MC7304/MC7354, WP76xx */ | |
| 1205 | + {QMI_QUIRK_SET_DTR(0x1199, 0x68c0, 10)},/* Sierra Wireless MC7304/MC7354 */ | |
| 1206 | 1206 | {QMI_FIXED_INTF(0x1199, 0x901c, 8)}, /* Sierra Wireless EM7700 */ |
| 1207 | 1207 | {QMI_FIXED_INTF(0x1199, 0x901f, 8)}, /* Sierra Wireless EM7355 */ |
| 1208 | 1208 | {QMI_FIXED_INTF(0x1199, 0x9041, 8)}, /* Sierra Wireless MC7305/MC7355 */ |
| @@ -557,6 +557,7 @@ enum spd_duplex { · 556 unmodified lines | ||
| 557 | 557 | /* MAC PASSTHRU */ |
| 558 | 558 | #define AD_MASK 0xfee0 |
| 559 | 559 | #define BND_MASK 0x0004 |
| 560 | +#define BD_MASK 0x0001 | |
| 560 | 561 | #define EFUSE 0xcfdb |
| 561 | 562 | #define PASS_THRU_MASK 0x1 |
| 562 | 563 | |
| @@ -1176,9 +1177,9 @@ static int vendor_mac_passthru_addr_read(struct r8152 *tp, struct sockaddr *sa) · 613 unmodified lines | ||
| 1176 | 1177 | return -ENODEV; |
| 1177 | 1178 | } |
| 1178 | 1179 | } else { |
| 1179 | − /* test for RTL8153-BND */ | |
| 1180 | + /* test for RTL8153-BND and RTL8153-BD */ | |
| 1180 | 1181 | ocp_data = ocp_read_byte(tp, MCU_TYPE_USB, USB_MISC_1); |
| 1181 | − if ((ocp_data & BND_MASK) == 0) { | |
| 1182 | + if ((ocp_data & BND_MASK) == 0 && (ocp_data & BD_MASK) == 0) { | |
| 1182 | 1183 | netif_dbg(tp, probe, tp->netdev, |
| 1183 | 1184 | "Invalid variant for MAC pass through\n"); |
| 1184 | 1185 | return -ENODEV; |
| @@ -1273,6 +1273,9 @@ static void vrf_setup(struct net_device *dev) · 1272 unmodified lines | ||
| 1273 | 1273 | |
| 1274 | 1274 | /* default to no qdisc; user can add if desired */ |
| 1275 | 1275 | dev->priv_flags |= IFF_NO_QUEUE; |
| 1276 | + | |
| 1277 | + dev->min_mtu = 0; | |
| 1278 | + dev->max_mtu = 0; | |
| 1276 | 1279 | } |
| 1277 | 1280 | |
| 1278 | 1281 | static int vrf_validate(struct nlattr *tb[], struct nlattr *data[], |
| @@ -3554,7 +3554,7 @@ static int hwsim_get_radio_nl(struct sk_buff *msg, struct genl_info *info) · 3553 unmodified lines | ||
| 3554 | 3554 | goto out_err; |
| 3555 | 3555 | } |
| 3556 | 3556 | |
| 3557 | − genlmsg_reply(skb, info); | |
| 3557 | + res = genlmsg_reply(skb, info); | |
| 3558 | 3558 | break; |
| 3559 | 3559 | } |
| 3560 | 3560 | |
| @@ -158,39 +158,49 @@ static const struct ieee80211_ops mt76x0u_ops = { · 157 unmodified lines | ||
| 158 | 158 | .get_txpower = mt76x02_get_txpower, |
| 159 | 159 | }; |
| 160 | 160 | |
| 161 | −static int mt76x0u_register_device(struct mt76x02_dev *dev) | |
| 161 | +static int mt76x0u_init_hardware(struct mt76x02_dev *dev) | |
| 162 | 162 | { |
| 163 | − struct ieee80211_hw *hw = dev->mt76.hw; | |
| 164 | 163 | int err; |
| 165 | 164 | |
| 166 | − err = mt76u_alloc_queues(&dev->mt76); | |
| 167 | − if (err < 0) | |
| 168 | − goto out_err; | |
| 169 | − | |
| 170 | − err = mt76u_mcu_init_rx(&dev->mt76); | |
| 171 | − if (err < 0) | |
| 172 | − goto out_err; | |
| 173 | − | |
| 174 | 165 | mt76x0_chip_onoff(dev, true, true); |
| 175 | − if (!mt76x02_wait_for_mac(&dev->mt76)) { | |
| 176 | − err = -ETIMEDOUT; | |
| 177 | − goto out_err; | |
| 178 | − } | |
| 166 | + | |
| 167 | + if (!mt76x02_wait_for_mac(&dev->mt76)) | |
| 168 | + return -ETIMEDOUT; | |
| 179 | 169 | |
| 180 | 170 | err = mt76x0u_mcu_init(dev); |
| 181 | 171 | if (err < 0) |
| 182 | − goto out_err; | |
| 172 | + return err; | |
| 183 | 173 | |
| 184 | 174 | mt76x0_init_usb_dma(dev); |
| 185 | 175 | err = mt76x0_init_hardware(dev); |
| 186 | 176 | if (err < 0) |
| 187 | − goto out_err; | |
| 177 | + return err; | |
| 188 | 178 | |
| 189 | 179 | mt76_rmw(dev, MT_US_CYC_CFG, MT_US_CYC_CNT, 0x1e); |
| 190 | 180 | mt76_wr(dev, MT_TXOP_CTRL_CFG, |
| 191 | 181 | FIELD_PREP(MT_TXOP_TRUN_EN, 0x3f) | |
| 192 | 182 | FIELD_PREP(MT_TXOP_EXT_CCA_DLY, 0x58)); |
| 193 | 183 | |
| 184 | + return 0; | |
| 185 | +} | |
| 186 | + | |
| 187 | +static int mt76x0u_register_device(struct mt76x02_dev *dev) | |
| 188 | +{ | |
| 189 | + struct ieee80211_hw *hw = dev->mt76.hw; | |
| 190 | + int err; | |
| 191 | + | |
| 192 | + err = mt76u_alloc_queues(&dev->mt76); | |
| 193 | + if (err < 0) | |
| 194 | + goto out_err; | |
| 195 | + | |
| 196 | + err = mt76u_mcu_init_rx(&dev->mt76); | |
| 197 | + if (err < 0) | |
| 198 | + goto out_err; | |
| 199 | + | |
| 200 | + err = mt76x0u_init_hardware(dev); | |
| 201 | + if (err < 0) | |
| 202 | + goto out_err; | |
| 203 | + | |
| 194 | 204 | err = mt76x0_register_device(dev); |
| 195 | 205 | if (err < 0) |
| 196 | 206 | goto out_err; |
| @@ -301,6 +311,8 @@ static int __maybe_unused mt76x0_suspend(struct usb_interface *usb_intf, · 104 unmodified lines | ||
| 301 | 311 | |
| 302 | 312 | mt76u_stop_queues(&dev->mt76); |
| 303 | 313 | mt76x0u_mac_stop(dev); |
| 314 | + clear_bit(MT76_STATE_MCU_RUNNING, &dev->mt76.state); | |
| 315 | + mt76x0_chip_onoff(dev, false, false); | |
| 304 | 316 | usb_kill_urb(usb->mcu.res.urb); |
| 305 | 317 | |
| 306 | 318 | return 0; |
| @@ -328,7 +340,7 @@ static int __maybe_unused mt76x0_resume(struct usb_interface *usb_intf) · 21 unmodified lines | ||
| 328 | 340 | tasklet_enable(&usb->rx_tasklet); |
| 329 | 341 | tasklet_enable(&usb->tx_tasklet); |
| 330 | 342 | |
| 331 | − ret = mt76x0_init_hardware(dev); | |
| 343 | + ret = mt76x0u_init_hardware(dev); | |
| 332 | 344 | if (ret) |
| 333 | 345 | goto err; |
| 334 | 346 | |
| @@ -454,6 +454,8 @@ void xenvif_init_hash(struct xenvif *vif) · 453 unmodified lines | ||
| 454 | 454 | if (xenvif_hash_cache_size == 0) |
| 455 | 455 | return; |
| 456 | 456 | |
| 457 | + BUG_ON(vif->hash.cache.count); | |
| 458 | + | |
| 457 | 459 | spin_lock_init(&vif->hash.cache.lock); |
| 458 | 460 | INIT_LIST_HEAD(&vif->hash.cache.list); |
| 459 | 461 | } |
| @@ -153,6 +153,13 @@ static u16 xenvif_select_queue(struct net_device *dev, struct sk_buff *skb, · 152 unmodified lines | ||
| 153 | 153 | { |
| 154 | 154 | struct xenvif *vif = netdev_priv(dev); |
| 155 | 155 | unsigned int size = vif->hash.size; |
| 156 | + unsigned int num_queues; | |
| 157 | + | |
| 158 | + /* If queues are not set up internally - always return 0 | |
| 159 | + * as the packet going to be dropped anyway */ | |
| 160 | + num_queues = READ_ONCE(vif->num_queues); | |
| 161 | + if (num_queues < 1) | |
| 162 | + return 0; | |
| 156 | 163 | |
| 157 | 164 | if (vif->hash.alg == XEN_NETIF_CTRL_HASH_ALGORITHM_NONE) |
| 158 | 165 | return fallback(dev, skb, NULL) % dev->real_num_tx_queues; |
| @@ -1072,11 +1072,6 @@ static int xenvif_handle_frag_list(struct xenvif_queue *queue, struct sk_buff *s · 1071 unmodified lines | ||
| 1072 | 1072 | skb_frag_size_set(&frags[i], len); |
| 1073 | 1073 | } |
| 1074 | 1074 | |
| 1075 | − /* Copied all the bits from the frag list -- free it. */ | |
| 1076 | − skb_frag_list_init(skb); | |
| 1077 | − xenvif_skb_zerocopy_prepare(queue, nskb); | |
| 1078 | − kfree_skb(nskb); | |
| 1079 | − | |
| 1080 | 1075 | /* Release all the original (foreign) frags. */ |
| 1081 | 1076 | for (f = 0; f < skb_shinfo(skb)->nr_frags; f++) |
| 1082 | 1077 | skb_frag_unref(skb, f); |
| @@ -1145,6 +1140,8 @@ static int xenvif_tx_submit(struct xenvif_queue *queue) · 62 unmodified lines | ||
| 1145 | 1140 | xenvif_fill_frags(queue, skb); |
| 1146 | 1141 | |
| 1147 | 1142 | if (unlikely(skb_has_frag_list(skb))) { |
| 1143 | + struct sk_buff *nskb = skb_shinfo(skb)->frag_list; | |
| 1144 | + xenvif_skb_zerocopy_prepare(queue, nskb); | |
| 1148 | 1145 | if (xenvif_handle_frag_list(queue, skb)) { |
| 1149 | 1146 | if (net_ratelimit()) |
| 1150 | 1147 | netdev_err(queue->vif->dev, |
| @@ -1153,6 +1150,9 @@ static int xenvif_tx_submit(struct xenvif_queue *queue) · 2 unmodified lines | ||
| 1153 | 1150 | kfree_skb(skb); |
| 1154 | 1151 | continue; |
| 1155 | 1152 | } |
| 1153 | + /* Copied all the bits from the frag list -- free it. */ | |
| 1154 | + skb_frag_list_init(skb); | |
| 1155 | + kfree_skb(nskb); | |
| 1156 | 1156 | } |
| 1157 | 1157 | |
| 1158 | 1158 | skb->dev = queue->vif->dev; |
| @@ -693,7 +693,7 @@ static const char * const sd_a_groups[] = { · 692 unmodified lines | ||
| 693 | 693 | |
| 694 | 694 | static const char * const sdxc_a_groups[] = { |
| 695 | 695 | "sdxc_d0_0_a", "sdxc_d13_0_a", "sdxc_d47_a", "sdxc_clk_a", |
| 696 | − "sdxc_cmd_a", "sdxc_d0_1_a", "sdxc_d0_13_1_a" | |
| 696 | + "sdxc_cmd_a", "sdxc_d0_1_a", "sdxc_d13_1_a" | |
| 697 | 697 | }; |
| 698 | 698 | |
| 699 | 699 | static const char * const pcm_a_groups[] = { |
| @@ -79,7 +79,7 @@ enum { · 78 unmodified lines | ||
| 79 | 79 | .intr_cfg_reg = 0, \ |
| 80 | 80 | .intr_status_reg = 0, \ |
| 81 | 81 | .intr_target_reg = 0, \ |
| 82 | − .tile = NORTH, \ | |
| 82 | + .tile = SOUTH, \ | |
| 83 | 83 | .mux_bit = -1, \ |
| 84 | 84 | .pull_bit = pull, \ |
| 85 | 85 | .drv_bit = drv, \ |
| @@ -2009,7 +2009,7 @@ static int twa_probe(struct pci_dev *pdev, const struct pci_device_id *dev_id) · 2008 unmodified lines | ||
| 2009 | 2009 | struct Scsi_Host *host = NULL; |
| 2010 | 2010 | TW_Device_Extension *tw_dev; |
| 2011 | 2011 | unsigned long mem_addr, mem_len; |
| 2012 | − int retval = -ENODEV; | |
| 2012 | + int retval; | |
| 2013 | 2013 | |
| 2014 | 2014 | retval = pci_enable_device(pdev); |
| 2015 | 2015 | if (retval) { |
| @@ -2020,8 +2020,10 @@ static int twa_probe(struct pci_dev *pdev, const struct pci_device_id *dev_id) · 4 unmodified lines | ||
| 2020 | 2020 | pci_set_master(pdev); |
| 2021 | 2021 | pci_try_set_mwi(pdev); |
| 2022 | 2022 | |
| 2023 | − if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | |
| 2024 | − dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) { | |
| 2023 | + retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); | |
| 2024 | + if (retval) | |
| 2025 | + retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | |
| 2026 | + if (retval) { | |
| 2025 | 2027 | TW_PRINTK(host, TW_DRIVER, 0x23, "Failed to set dma mask"); |
| 2026 | 2028 | retval = -ENODEV; |
| 2027 | 2029 | goto out_disable_device; |
| @@ -2240,8 +2242,10 @@ static int twa_resume(struct pci_dev *pdev) · 212 unmodified lines | ||
| 2240 | 2242 | pci_set_master(pdev); |
| 2241 | 2243 | pci_try_set_mwi(pdev); |
| 2242 | 2244 | |
| 2243 | − if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | |
| 2244 | − dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) { | |
| 2245 | + retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); | |
| 2246 | + if (retval) | |
| 2247 | + retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | |
| 2248 | + if (retval) { | |
| 2245 | 2249 | TW_PRINTK(host, TW_DRIVER, 0x40, "Failed to set dma mask during resume"); |
| 2246 | 2250 | retval = -ENODEV; |
| 2247 | 2251 | goto out_disable_device; |
| @@ -1573,8 +1573,10 @@ static int twl_probe(struct pci_dev *pdev, const struct pci_device_id *dev_id) · 1572 unmodified lines | ||
| 1573 | 1573 | pci_set_master(pdev); |
| 1574 | 1574 | pci_try_set_mwi(pdev); |
| 1575 | 1575 | |
| 1576 | − if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | |
| 1577 | − dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) { | |
| 1576 | + retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); | |
| 1577 | + if (retval) | |
| 1578 | + retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | |
| 1579 | + if (retval) { | |
| 1578 | 1580 | TW_PRINTK(host, TW_DRIVER, 0x18, "Failed to set dma mask"); |
| 1579 | 1581 | retval = -ENODEV; |
| 1580 | 1582 | goto out_disable_device; |
| @@ -1805,8 +1807,10 @@ static int twl_resume(struct pci_dev *pdev) · 224 unmodified lines | ||
| 1805 | 1807 | pci_set_master(pdev); |
| 1806 | 1808 | pci_try_set_mwi(pdev); |
| 1807 | 1809 | |
| 1808 | − if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | |
| 1809 | − dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) { | |
| 1810 | + retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); | |
| 1811 | + if (retval) | |
| 1812 | + retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | |
| 1813 | + if (retval) { | |
| 1810 | 1814 | TW_PRINTK(host, TW_DRIVER, 0x25, "Failed to set dma mask during resume"); |
| 1811 | 1815 | retval = -ENODEV; |
| 1812 | 1816 | goto out_disable_device; |
| @@ -769,9 +769,11 @@ static int asd_pci_probe(struct pci_dev *dev, const struct pci_device_id *id) · 768 unmodified lines | ||
| 769 | 769 | if (err) |
| 770 | 770 | goto Err_remove; |
| 771 | 771 | |
| 772 | − err = -ENODEV; | |
| 773 | − if (dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(64)) || | |
| 774 | − dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(32))) { | |
| 772 | + err = dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(64)); | |
| 773 | + if (err) | |
| 774 | + err = dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(32)); | |
| 775 | + if (err) { | |
| 776 | + err = -ENODEV; | |
| 775 | 777 | asd_printk("no suitable DMA mask for %s\n", pci_name(dev)); |
| 776 | 778 | goto Err_remove; |
| 777 | 779 | } |
| @@ -727,7 +727,7 @@ bfad_init_timer(struct bfad_s *bfad) · 726 unmodified lines | ||
| 727 | 727 | int |
| 728 | 728 | bfad_pci_init(struct pci_dev *pdev, struct bfad_s *bfad) |
| 729 | 729 | { |
| 730 | − int rc = -ENODEV; | |
| 730 | + int rc = -ENODEV; | |
| 731 | 731 | |
| 732 | 732 | if (pci_enable_device(pdev)) { |
| 733 | 733 | printk(KERN_ERR "pci_enable_device fail %p\n", pdev); |
| @@ -739,8 +739,12 @@ bfad_pci_init(struct pci_dev *pdev, struct bfad_s *bfad) · 5 unmodified lines | ||
| 739 | 739 | |
| 740 | 740 | pci_set_master(pdev); |
| 741 | 741 | |
| 742 | − if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | |
| 743 | − dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) { | |
| 742 | + rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); | |
| 743 | + if (rc) | |
| 744 | + rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | |
| 745 | + | |
| 746 | + if (rc) { | |
| 747 | + rc = -ENODEV; | |
| 744 | 748 | printk(KERN_ERR "dma_set_mask_and_coherent fail %p\n", pdev); |
| 745 | 749 | goto out_release_region; |
| 746 | 750 | } |
| @@ -1534,6 +1538,7 @@ bfad_pci_slot_reset(struct pci_dev *pdev) · 787 unmodified lines | ||
| 1534 | 1538 | { |
| 1535 | 1539 | struct bfad_s *bfad = pci_get_drvdata(pdev); |
| 1536 | 1540 | u8 byte; |
| 1541 | + int rc; | |
| 1537 | 1542 | |
| 1538 | 1543 | dev_printk(KERN_ERR, &pdev->dev, |
| 1539 | 1544 | "bfad_pci_slot_reset flags: 0x%x\n", bfad->bfad_flags); |
| @@ -1561,8 +1566,11 @@ bfad_pci_slot_reset(struct pci_dev *pdev) · 21 unmodified lines | ||
| 1561 | 1566 | pci_save_state(pdev); |
| 1562 | 1567 | pci_set_master(pdev); |
| 1563 | 1568 | |
| 1564 | − if (dma_set_mask_and_coherent(&bfad->pcidev->dev, DMA_BIT_MASK(64)) || | |
| 1565 | − dma_set_mask_and_coherent(&bfad->pcidev->dev, DMA_BIT_MASK(32))) | |
| 1569 | + rc = dma_set_mask_and_coherent(&bfad->pcidev->dev, DMA_BIT_MASK(64)); | |
| 1570 | + if (rc) | |
| 1571 | + rc = dma_set_mask_and_coherent(&bfad->pcidev->dev, | |
| 1572 | + DMA_BIT_MASK(32)); | |
| 1573 | + if (rc) | |
| 1566 | 1574 | goto out_disable_device; |
| 1567 | 1575 | |
| 1568 | 1576 | if (restart_bfa(bfad) == -1) |
| @@ -210,8 +210,11 @@ csio_pci_init(struct pci_dev *pdev, int *bars) · 209 unmodified lines | ||
| 210 | 210 | pci_set_master(pdev); |
| 211 | 211 | pci_try_set_mwi(pdev); |
| 212 | 212 | |
| 213 | − if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | |
| 214 | − dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) { | |
| 213 | + rv = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); | |
| 214 | + if (rv) | |
| 215 | + rv = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | |
| 216 | + if (rv) { | |
| 217 | + rv = -ENODEV; | |
| 215 | 218 | dev_err(&pdev->dev, "No suitable DMA available.\n"); |
| 216 | 219 | goto err_release_regions; |
| 217 | 220 | } |
| @@ -2323,6 +2323,7 @@ static struct Scsi_Host *hisi_sas_shost_alloc(struct platform_device *pdev, · 2322 unmodified lines | ||
| 2323 | 2323 | struct Scsi_Host *shost; |
| 2324 | 2324 | struct hisi_hba *hisi_hba; |
| 2325 | 2325 | struct device *dev = &pdev->dev; |
| 2326 | + int error; | |
| 2326 | 2327 | |
| 2327 | 2328 | shost = scsi_host_alloc(hw->sht, sizeof(*hisi_hba)); |
| 2328 | 2329 | if (!shost) { |
| @@ -2343,8 +2344,11 @@ static struct Scsi_Host *hisi_sas_shost_alloc(struct platform_device *pdev, · 14 unmodified lines | ||
| 2343 | 2344 | if (hisi_sas_get_fw_info(hisi_hba) < 0) |
| 2344 | 2345 | goto err_out; |
| 2345 | 2346 | |
| 2346 | − if (dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64)) && | |
| 2347 | − dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32))) { | |
| 2347 | + error = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64)); | |
| 2348 | + if (error) | |
| 2349 | + error = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32)); | |
| 2350 | + | |
| 2351 | + if (error) { | |
| 2348 | 2352 | dev_err(dev, "No usable DMA addressing method\n"); |
| 2349 | 2353 | goto err_out; |
| 2350 | 2354 | } |
| @@ -2447,10 +2447,12 @@ hisi_sas_v3_probe(struct pci_dev *pdev, const struct pci_device_id *id) · 2446 unmodified lines | ||
| 2447 | 2447 | if (rc) |
| 2448 | 2448 | goto err_out_disable_device; |
| 2449 | 2449 | |
| 2450 | − if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | |
| 2451 | − dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) { | |
| 2450 | + rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); | |
| 2451 | + if (rc) | |
| 2452 | + rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | |
| 2453 | + if (rc) { | |
| 2452 | 2454 | dev_err(dev, "No usable DMA addressing method\n"); |
| 2453 | − rc = -EIO; | |
| 2455 | + rc = -ENODEV; | |
| 2454 | 2456 | goto err_out_regions; |
| 2455 | 2457 | } |
| 2456 | 2458 | |
| @@ -1292,6 +1292,7 @@ static int hptiop_probe(struct pci_dev *pcidev, const struct pci_device_id *id) · 1291 unmodified lines | ||
| 1292 | 1292 | dma_addr_t start_phy; |
| 1293 | 1293 | void *start_virt; |
| 1294 | 1294 | u32 offset, i, req_size; |
| 1295 | + int rc; | |
| 1295 | 1296 | |
| 1296 | 1297 | dprintk("hptiop_probe(%p)\n", pcidev); |
| 1297 | 1298 | |
| @@ -1308,9 +1309,12 @@ static int hptiop_probe(struct pci_dev *pcidev, const struct pci_device_id *id) · 10 unmodified lines | ||
| 1308 | 1309 | |
| 1309 | 1310 | /* Enable 64bit DMA if possible */ |
| 1310 | 1311 | iop_ops = (struct hptiop_adapter_ops *)id->driver_data; |
| 1311 | − if (dma_set_mask(&pcidev->dev, | |
| 1312 | − DMA_BIT_MASK(iop_ops->hw_dma_bit_mask)) || | |
| 1313 | − dma_set_mask(&pcidev->dev, DMA_BIT_MASK(32))) { | |
| 1312 | + rc = dma_set_mask(&pcidev->dev, | |
| 1313 | + DMA_BIT_MASK(iop_ops->hw_dma_bit_mask)); | |
| 1314 | + if (rc) | |
| 1315 | + rc = dma_set_mask(&pcidev->dev, DMA_BIT_MASK(32)); | |
| 1316 | + | |
| 1317 | + if (rc) { | |
| 1314 | 1318 | printk(KERN_ERR "hptiop: fail to set dma_mask\n"); |
| 1315 | 1319 | goto disable_pci_device; |
| 1316 | 1320 | } |
| @@ -1459,7 +1459,13 @@ static int iscsi_xmit_task(struct iscsi_conn *conn) · 1458 unmodified lines | ||
| 1459 | 1459 | if (test_bit(ISCSI_SUSPEND_BIT, &conn->suspend_tx)) |
| 1460 | 1460 | return -ENODATA; |
| 1461 | 1461 | |
| 1462 | + spin_lock_bh(&conn->session->back_lock); | |
| 1463 | + if (conn->task == NULL) { | |
| 1464 | + spin_unlock_bh(&conn->session->back_lock); | |
| 1465 | + return -ENODATA; | |
| 1466 | + } | |
| 1462 | 1467 | __iscsi_get_task(task); |
| 1468 | + spin_unlock_bh(&conn->session->back_lock); | |
| 1463 | 1469 | spin_unlock_bh(&conn->session->frwd_lock); |
| 1464 | 1470 | rc = conn->session->tt->xmit_task(task); |
| 1465 | 1471 | spin_lock_bh(&conn->session->frwd_lock); |
| @@ -828,6 +828,7 @@ static struct domain_device *sas_ex_discover_end_dev( · 827 unmodified lines | ||
| 828 | 828 | rphy = sas_end_device_alloc(phy->port); |
| 829 | 829 | if (!rphy) |
| 830 | 830 | goto out_free; |
| 831 | + rphy->identify.phy_identifier = phy_id; | |
| 831 | 832 | |
| 832 | 833 | child->rphy = rphy; |
| 833 | 834 | get_device(&rphy->dev); |
| @@ -854,6 +855,7 @@ static struct domain_device *sas_ex_discover_end_dev( · 20 unmodified lines | ||
| 854 | 855 | |
| 855 | 856 | child->rphy = rphy; |
| 856 | 857 | get_device(&rphy->dev); |
| 858 | + rphy->identify.phy_identifier = phy_id; | |
| 857 | 859 | sas_fill_in_rphy(child, rphy); |
| 858 | 860 | |
| 859 | 861 | list_add_tail(&child->disco_list_node, &parent->port->disco_list); |
| @@ -7361,15 +7361,18 @@ lpfc_sli_pci_mem_setup(struct lpfc_hba *phba) · 7360 unmodified lines | ||
| 7361 | 7361 | unsigned long bar0map_len, bar2map_len; |
| 7362 | 7362 | int i, hbq_count; |
| 7363 | 7363 | void *ptr; |
| 7364 | − int error = -ENODEV; | |
| 7364 | + int error; | |
| 7365 | 7365 | |
| 7366 | 7366 | if (!pdev) |
| 7367 | − return error; | |
| 7367 | + return -ENODEV; | |
| 7368 | 7368 | |
| 7369 | 7369 | /* Set the device DMA mask size */ |
| 7370 | − if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | |
| 7371 | − dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) | |
| 7370 | + error = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); | |
| 7371 | + if (error) | |
| 7372 | + error = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | |
| 7373 | + if (error) | |
| 7372 | 7374 | return error; |
| 7375 | + error = -ENODEV; | |
| 7373 | 7376 | |
| 7374 | 7377 | /* Get the bus address of Bar0 and Bar2 and the number of bytes |
| 7375 | 7378 | * required by each mapping. |
| @@ -9742,11 +9745,13 @@ lpfc_sli4_pci_mem_setup(struct lpfc_hba *phba) · 2366 unmodified lines | ||
| 9742 | 9745 | uint32_t if_type; |
| 9743 | 9746 | |
| 9744 | 9747 | if (!pdev) |
| 9745 | − return error; | |
| 9748 | + return -ENODEV; | |
| 9746 | 9749 | |
| 9747 | 9750 | /* Set the device DMA mask size */ |
| 9748 | − if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | |
| 9749 | − dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) | |
| 9751 | + error = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); | |
| 9752 | + if (error) | |
| 9753 | + error = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | |
| 9754 | + if (error) | |
| 9750 | 9755 | return error; |
| 9751 | 9756 | |
| 9752 | 9757 | /* |
| @@ -655,6 +655,7 @@ static blk_status_t scsi_result_to_blk_status(struct scsi_cmnd *cmd, int result) · 654 unmodified lines | ||
| 655 | 655 | set_host_byte(cmd, DID_OK); |
| 656 | 656 | return BLK_STS_TARGET; |
| 657 | 657 | case DID_NEXUS_FAILURE: |
| 658 | + set_host_byte(cmd, DID_OK); | |
| 658 | 659 | return BLK_STS_NEXUS; |
| 659 | 660 | case DID_ALLOC_FAILURE: |
| 660 | 661 | set_host_byte(cmd, DID_OK); |
| @@ -2597,7 +2598,6 @@ void scsi_device_resume(struct scsi_device *sdev) · 1936 unmodified lines | ||
| 2597 | 2598 | * device deleted during suspend) |
| 2598 | 2599 | */ |
| 2599 | 2600 | mutex_lock(&sdev->state_mutex); |
| 2600 | − WARN_ON_ONCE(!sdev->quiesced_by); | |
| 2601 | 2601 | sdev->quiesced_by = NULL; |
| 2602 | 2602 | blk_clear_pm_only(sdev->request_queue); |
| 2603 | 2603 | if (sdev->sdev_state == SDEV_QUIESCE) |
| @@ -142,10 +142,12 @@ int sd_zbc_report_zones(struct gendisk *disk, sector_t sector, · 141 unmodified lines | ||
| 142 | 142 | return -EOPNOTSUPP; |
| 143 | 143 | |
| 144 | 144 | /* |
| 145 | − * Get a reply buffer for the number of requested zones plus a header. | |
| 146 | − * For ATA, buffers must be aligned to 512B. | |
| 145 | + * Get a reply buffer for the number of requested zones plus a header, | |
| 146 | + * without exceeding the device maximum command size. For ATA disks, | |
| 147 | + * buffers must be aligned to 512B. | |
| 147 | 148 | */ |
| 148 | − buflen = roundup((nrz + 1) * 64, 512); | |
| 149 | + buflen = min(queue_max_hw_sectors(disk->queue) << 9, | |
| 150 | + roundup((nrz + 1) * 64, 512)); | |
| 149 | 151 | buf = kmalloc(buflen, gfp_mask); |
| 150 | 152 | if (!buf) |
| 151 | 153 | return -ENOMEM; |
| @@ -699,8 +699,10 @@ static int __init optee_driver_init(void) · 698 unmodified lines | ||
| 699 | 699 | return -ENODEV; |
| 700 | 700 | |
| 701 | 701 | np = of_find_matching_node(fw_np, optee_match); |
| 702 | − if (!np || !of_device_is_available(np)) | |
| 702 | + if (!np || !of_device_is_available(np)) { | |
| 703 | + of_node_put(np); | |
| 703 | 704 | return -ENODEV; |
| 705 | + } | |
| 704 | 706 | |
| 705 | 707 | optee = optee_probe(np); |
| 706 | 708 | of_node_put(np); |
| @@ -1788,7 +1788,7 @@ static int log_used(struct vhost_virtqueue *vq, u64 used_offset, u64 len) · 1787 unmodified lines | ||
| 1788 | 1788 | |
| 1789 | 1789 | ret = translate_desc(vq, (uintptr_t)vq->used + used_offset, |
| 1790 | 1790 | len, iov, 64, VHOST_ACCESS_WO); |
| 1791 | − if (ret) | |
| 1791 | + if (ret < 0) | |
| 1792 | 1792 | return ret; |
| 1793 | 1793 | |
| 1794 | 1794 | for (i = 0; i < ret; i++) { |
| @@ -173,6 +173,7 @@ static struct afs_cell *afs_alloc_cell(struct afs_net *net, · 172 unmodified lines | ||
| 173 | 173 | |
| 174 | 174 | rcu_assign_pointer(cell->vl_servers, vllist); |
| 175 | 175 | cell->dns_expiry = TIME64_MAX; |
| 176 | + __clear_bit(AFS_CELL_FL_NO_LOOKUP_YET, &cell->flags); | |
| 176 | 177 | } else { |
| 177 | 178 | cell->dns_expiry = ktime_get_real_seconds(); |
| 178 | 179 | } |
| @@ -14,13 +14,30 @@ · 13 unmodified lines | ||
| 14 | 14 | #include <linux/err.h> |
| 15 | 15 | #include <linux/fs.h> |
| 16 | 16 | |
| 17 | +static inline bool spacetab(char c) { return c == ' ' || c == '\t'; } | |
| 18 | +static inline char *next_non_spacetab(char *first, const char *last) | |
| 19 | +{ | |
| 20 | + for (; first <= last; first++) | |
| 21 | + if (!spacetab(*first)) | |
| 22 | + return first; | |
| 23 | + return NULL; | |
| 24 | +} | |
| 25 | +static inline char *next_terminator(char *first, const char *last) | |
| 26 | +{ | |
| 27 | + for (; first <= last; first++) | |
| 28 | + if (spacetab(*first) || !*first) | |
| 29 | + return first; | |
| 30 | + return NULL; | |
| 31 | +} | |
| 32 | + | |
| 17 | 33 | static int load_script(struct linux_binprm *bprm) |
| 18 | 34 | { |
| 19 | 35 | const char *i_arg, *i_name; |
| 20 | − char *cp; | |
| 36 | + char *cp, *buf_end; | |
| 21 | 37 | struct file *file; |
| 22 | 38 | int retval; |
| 23 | 39 | |
| 40 | + /* Not ours to exec if we don't start with "#!". */ | |
| 24 | 41 | if ((bprm->buf[0] != '#') || (bprm->buf[1] != '!')) |
| 25 | 42 | return -ENOEXEC; |
| 26 | 43 | |
| @@ -33,18 +50,40 @@ static int load_script(struct linux_binprm *bprm) · 6 unmodified lines | ||
| 33 | 50 | if (bprm->interp_flags & BINPRM_FLAGS_PATH_INACCESSIBLE) |
| 34 | 51 | return -ENOENT; |
| 35 | 52 | |
| 36 | − /* | |
| 37 | − * This section does the #! interpretation. | |
| 38 | − * Sorta complicated, but hopefully it will work. -TYT | |
| 39 | − */ | |
| 40 | − | |
| 53 | + /* Release since we are not mapping a binary into memory. */ | |
| 41 | 54 | allow_write_access(bprm->file); |
| 42 | 55 | fput(bprm->file); |
| 43 | 56 | bprm->file = NULL; |
| 44 | 57 | |
| 45 | − bprm->buf[BINPRM_BUF_SIZE - 1] = '\0'; | |
| 46 | − if ((cp = strchr(bprm->buf, '\n')) == NULL) | |
| 47 | − cp = bprm->buf+BINPRM_BUF_SIZE-1; | |
| 58 | + /* | |
| 59 | + * This section handles parsing the #! line into separate | |
| 60 | + * interpreter path and argument strings. We must be careful | |
| 61 | + * because bprm->buf is not yet guaranteed to be NUL-terminated | |
| 62 | + * (though the buffer will have trailing NUL padding when the | |
| 63 | + * file size was smaller than the buffer size). | |
| 64 | + * | |
| 65 | + * We do not want to exec a truncated interpreter path, so either | |
| 66 | + * we find a newline (which indicates nothing is truncated), or | |
| 67 | + * we find a space/tab/NUL after the interpreter path (which | |
| 68 | + * itself may be preceded by spaces/tabs). Truncating the | |
| 69 | + * arguments is fine: the interpreter can re-read the script to | |
| 70 | + * parse them on its own. | |
| 71 | + */ | |
| 72 | + buf_end = bprm->buf + sizeof(bprm->buf) - 1; | |
| 73 | + cp = strnchr(bprm->buf, sizeof(bprm->buf), '\n'); | |
| 74 | + if (!cp) { | |
| 75 | + cp = next_non_spacetab(bprm->buf + 2, buf_end); | |
| 76 | + if (!cp) | |
| 77 | + return -ENOEXEC; /* Entire buf is spaces/tabs */ | |
| 78 | + /* | |
| 79 | + * If there is no later space/tab/NUL we must assume the | |
| 80 | + * interpreter path is truncated. | |
| 81 | + */ | |
| 82 | + if (!next_terminator(cp, buf_end)) | |
| 83 | + return -ENOEXEC; | |
| 84 | + cp = buf_end; | |
| 85 | + } | |
| 86 | + /* NUL-terminate the buffer and any trailing spaces/tabs. */ | |
| 48 | 87 | *cp = '\0'; |
| 49 | 88 | while (cp > bprm->buf) { |
| 50 | 89 | cp--; |
| @@ -616,7 +616,8 @@ int __ceph_finish_cap_snap(struct ceph_inode_info *ci, · 615 unmodified lines | ||
| 616 | 616 | capsnap->size); |
| 617 | 617 | |
| 618 | 618 | spin_lock(&mdsc->snap_flush_lock); |
| 619 | − list_add_tail(&ci->i_snap_flush_item, &mdsc->snap_flush_list); | |
| 619 | + if (list_empty(&ci->i_snap_flush_item)) | |
| 620 | + list_add_tail(&ci->i_snap_flush_item, &mdsc->snap_flush_list); | |
| 620 | 621 | spin_unlock(&mdsc->snap_flush_lock); |
| 621 | 622 | return 1; /* caller may want to ceph_flush_snaps */ |
| 622 | 623 | } |
| @@ -859,6 +859,18 @@ static int hugetlbfs_migrate_page(struct address_space *mapping, · 858 unmodified lines | ||
| 859 | 859 | rc = migrate_huge_page_move_mapping(mapping, newpage, page); |
| 860 | 860 | if (rc != MIGRATEPAGE_SUCCESS) |
| 861 | 861 | return rc; |
| 862 | + | |
| 863 | + /* | |
| 864 | + * page_private is subpool pointer in hugetlb pages. Transfer to | |
| 865 | + * new page. PagePrivate is not associated with page_private for | |
| 866 | + * hugetlb pages and can not be set here as only page_huge_active | |
| 867 | + * pages can be migrated. | |
| 868 | + */ | |
| 869 | + if (page_private(page)) { | |
| 870 | + set_page_private(newpage, page_private(page)); | |
| 871 | + set_page_private(page, 0); | |
| 872 | + } | |
| 873 | + | |
| 862 | 874 | if (mode != MIGRATE_SYNC_NO_COPY) |
| 863 | 875 | migrate_page_copy(newpage, page); |
| 864 | 876 | else |
| @@ -2698,7 +2698,6 @@ static long exact_copy_from_user(void *to, const void __user * from, · 2697 unmodified lines | ||
| 2698 | 2698 | if (!access_ok(from, n)) |
| 2699 | 2699 | return n; |
| 2700 | 2700 | |
| 2701 | − current->kernel_uaccess_faults_ok++; | |
| 2702 | 2701 | while (n) { |
| 2703 | 2702 | if (__get_user(c, f)) { |
| 2704 | 2703 | memset(t, 0, n); |
| @@ -2708,7 +2707,6 @@ static long exact_copy_from_user(void *to, const void __user * from, · 3 unmodified lines | ||
| 2708 | 2707 | f++; |
| 2709 | 2708 | n--; |
| 2710 | 2709 | } |
| 2711 | − current->kernel_uaccess_faults_ok--; | |
| 2712 | 2710 | return n; |
| 2713 | 2711 | } |
| 2714 | 2712 | |
| @@ -44,6 +44,7 @@ · 43 unmodified lines | ||
| 44 | 44 | #include <linux/keyctl.h> |
| 45 | 45 | #include <linux/key-type.h> |
| 46 | 46 | #include <keys/user-type.h> |
| 47 | +#include <keys/request_key_auth-type.h> | |
| 47 | 48 | #include <linux/module.h> |
| 48 | 49 | |
| 49 | 50 | #include "internal.h" |
| @@ -59,7 +60,7 @@ static struct key_type key_type_id_resolver_legacy; · 9 unmodified lines | ||
| 59 | 60 | struct idmap_legacy_upcalldata { |
| 60 | 61 | struct rpc_pipe_msg pipe_msg; |
| 61 | 62 | struct idmap_msg idmap_msg; |
| 62 | − struct key_construction *key_cons; | |
| 63 | + struct key *authkey; | |
| 63 | 64 | struct idmap *idmap; |
| 64 | 65 | }; |
| 65 | 66 | |
| @@ -384,7 +385,7 @@ static const match_table_t nfs_idmap_tokens = { · 318 unmodified lines | ||
| 384 | 385 | { Opt_find_err, NULL } |
| 385 | 386 | }; |
| 386 | 387 | |
| 387 | −static int nfs_idmap_legacy_upcall(struct key_construction *, const char *, void *); | |
| 388 | +static int nfs_idmap_legacy_upcall(struct key *, void *); | |
| 388 | 389 | static ssize_t idmap_pipe_downcall(struct file *, const char __user *, |
| 389 | 390 | size_t); |
| 390 | 391 | static void idmap_release_pipe(struct inode *); |
| @@ -549,11 +550,12 @@ nfs_idmap_prepare_pipe_upcall(struct idmap *idmap, · 158 unmodified lines | ||
| 549 | 550 | static void |
| 550 | 551 | nfs_idmap_complete_pipe_upcall_locked(struct idmap *idmap, int ret) |
| 551 | 552 | { |
| 552 | − struct key_construction *cons = idmap->idmap_upcall_data->key_cons; | |
| 553 | + struct key *authkey = idmap->idmap_upcall_data->authkey; | |
| 553 | 554 | |
| 554 | 555 | kfree(idmap->idmap_upcall_data); |
| 555 | 556 | idmap->idmap_upcall_data = NULL; |
| 556 | − complete_request_key(cons, ret); | |
| 557 | + complete_request_key(authkey, ret); | |
| 558 | + key_put(authkey); | |
| 557 | 559 | } |
| 558 | 560 | |
| 559 | 561 | static void |
| @@ -563,15 +565,14 @@ nfs_idmap_abort_pipe_upcall(struct idmap *idmap, int ret) · 3 unmodified lines | ||
| 563 | 565 | nfs_idmap_complete_pipe_upcall_locked(idmap, ret); |
| 564 | 566 | } |
| 565 | 567 | |
| 566 | −static int nfs_idmap_legacy_upcall(struct key_construction *cons, | |
| 567 | − const char *op, | |
| 568 | − void *aux) | |
| 568 | +static int nfs_idmap_legacy_upcall(struct key *authkey, void *aux) | |
| 569 | 569 | { |
| 570 | 570 | struct idmap_legacy_upcalldata *data; |
| 571 | + struct request_key_auth *rka = get_request_key_auth(authkey); | |
| 571 | 572 | struct rpc_pipe_msg *msg; |
| 572 | 573 | struct idmap_msg *im; |
| 573 | 574 | struct idmap *idmap = (struct idmap *)aux; |
| 574 | − struct key *key = cons->key; | |
| 575 | + struct key *key = rka->target_key; | |
| 575 | 576 | int ret = -ENOKEY; |
| 576 | 577 | |
| 577 | 578 | if (!aux) |
| @@ -586,7 +587,7 @@ static int nfs_idmap_legacy_upcall(struct key_construction *cons, · 8 unmodified lines | ||
| 586 | 587 | msg = &data->pipe_msg; |
| 587 | 588 | im = &data->idmap_msg; |
| 588 | 589 | data->idmap = idmap; |
| 589 | − data->key_cons = cons; | |
| 590 | + data->authkey = key_get(authkey); | |
| 590 | 591 | |
| 591 | 592 | ret = nfs_idmap_prepare_message(key->description, idmap, im, msg); |
| 592 | 593 | if (ret < 0) |
| @@ -604,7 +605,7 @@ static int nfs_idmap_legacy_upcall(struct key_construction *cons, · 11 unmodified lines | ||
| 604 | 605 | out2: |
| 605 | 606 | kfree(data); |
| 606 | 607 | out1: |
| 607 | − complete_request_key(cons, ret); | |
| 608 | + complete_request_key(authkey, ret); | |
| 608 | 609 | return ret; |
| 609 | 610 | } |
| 610 | 611 | |
| @@ -651,9 +652,10 @@ out: · 40 unmodified lines | ||
| 651 | 652 | static ssize_t |
| 652 | 653 | idmap_pipe_downcall(struct file *filp, const char __user *src, size_t mlen) |
| 653 | 654 | { |
| 655 | + struct request_key_auth *rka; | |
| 654 | 656 | struct rpc_inode *rpci = RPC_I(file_inode(filp)); |
| 655 | 657 | struct idmap *idmap = (struct idmap *)rpci->private; |
| 656 | − struct key_construction *cons; | |
| 658 | + struct key *authkey; | |
| 657 | 659 | struct idmap_msg im; |
| 658 | 660 | size_t namelen_in; |
| 659 | 661 | int ret = -ENOKEY; |
| @@ -665,7 +667,8 @@ idmap_pipe_downcall(struct file *filp, const char __user *src, size_t mlen) · 5 unmodified lines | ||
| 665 | 667 | if (idmap->idmap_upcall_data == NULL) |
| 666 | 668 | goto out_noupcall; |
| 667 | 669 | |
| 668 | − cons = idmap->idmap_upcall_data->key_cons; | |
| 670 | + authkey = idmap->idmap_upcall_data->authkey; | |
| 671 | + rka = get_request_key_auth(authkey); | |
| 669 | 672 | |
| 670 | 673 | if (mlen != sizeof(im)) { |
| 671 | 674 | ret = -ENOSPC; |
| @@ -690,9 +693,9 @@ idmap_pipe_downcall(struct file *filp, const char __user *src, size_t mlen) · 18 unmodified lines | ||
| 690 | 693 | |
| 691 | 694 | ret = nfs_idmap_read_and_verify_message(&im, |
| 692 | 695 | &idmap->idmap_upcall_data->idmap_msg, |
| 693 | − cons->key, cons->authkey); | |
| 696 | + rka->target_key, authkey); | |
| 694 | 697 | if (ret >= 0) { |
| 695 | − key_set_timeout(cons->key, nfs_idmap_cache_timeout); | |
| 698 | + key_set_timeout(rka->target_key, nfs_idmap_cache_timeout); | |
| 696 | 699 | ret = mlen; |
| 697 | 700 | } |
| 698 | 701 | |
| @@ -398,8 +398,6 @@ static ssize_t orangefs_file_read_iter(struct kiocb *iocb, struct iov_iter *iter · 397 unmodified lines | ||
| 398 | 398 | loff_t pos = iocb->ki_pos; |
| 399 | 399 | ssize_t rc = 0; |
| 400 | 400 | |
| 401 | − BUG_ON(iocb->private); | |
| 402 | − | |
| 403 | 401 | gossip_debug(GOSSIP_FILE_DEBUG, "orangefs_file_read_iter\n"); |
| 404 | 402 | |
| 405 | 403 | orangefs_stats.reads++; |
| @@ -416,8 +414,6 @@ static ssize_t orangefs_file_write_iter(struct kiocb *iocb, struct iov_iter *ite · 10 unmodified lines | ||
| 416 | 414 | loff_t pos; |
| 417 | 415 | ssize_t rc; |
| 418 | 416 | |
| 419 | − BUG_ON(iocb->private); | |
| 420 | − | |
| 421 | 417 | gossip_debug(GOSSIP_FILE_DEBUG, "orangefs_file_write_iter\n"); |
| 422 | 418 | |
| 423 | 419 | inode_lock(file->f_mapping->host); |
| @@ -1086,10 +1086,6 @@ static int __set_oom_adj(struct file *file, int oom_adj, bool legacy) · 1085 unmodified lines | ||
| 1086 | 1086 | |
| 1087 | 1087 | task_lock(p); |
| 1088 | 1088 | if (!p->vfork_done && process_shares_mm(p, mm)) { |
| 1089 | − pr_info("updating oom_score_adj for %d (%s) from %d to %d because it shares mm with %d (%s). Report if this is unexpected.\n", | |
| 1090 | − task_pid_nr(p), p->comm, | |
| 1091 | − p->signal->oom_score_adj, oom_adj, | |
| 1092 | − task_pid_nr(task), task->comm); | |
| 1093 | 1089 | p->signal->oom_score_adj = oom_adj; |
| 1094 | 1090 | if (!legacy && has_capability_noaudit(current, CAP_SYS_RESOURCE)) |
| 1095 | 1091 | p->signal->oom_score_adj_min = (short)oom_adj; |
| @@ -560,6 +560,8 @@ · 559 unmodified lines | ||
| 560 | 560 | # define DP_TEST_LINK_EDID_READ (1 << 2) |
| 561 | 561 | # define DP_TEST_LINK_PHY_TEST_PATTERN (1 << 3) /* DPCD >= 1.1 */ |
| 562 | 562 | # define DP_TEST_LINK_FAUX_PATTERN (1 << 4) /* DPCD >= 1.2 */ |
| 563 | +# define DP_TEST_LINK_AUDIO_PATTERN (1 << 5) /* DPCD >= 1.2 */ | |
| 564 | +# define DP_TEST_LINK_AUDIO_DISABLED_VIDEO (1 << 6) /* DPCD >= 1.2 */ | |
| 563 | 565 | |
| 564 | 566 | #define DP_TEST_LINK_RATE 0x219 |
| 565 | 567 | # define DP_LINK_RATE_162 (0x6) |
| @@ -608,6 +610,7 @@ · 42 unmodified lines | ||
| 608 | 610 | # define DP_COLOR_FORMAT_RGB (0 << 1) |
| 609 | 611 | # define DP_COLOR_FORMAT_YCbCr422 (1 << 1) |
| 610 | 612 | # define DP_COLOR_FORMAT_YCbCr444 (2 << 1) |
| 613 | +# define DP_TEST_DYNAMIC_RANGE_VESA (0 << 3) | |
| 611 | 614 | # define DP_TEST_DYNAMIC_RANGE_CEA (1 << 3) |
| 612 | 615 | # define DP_TEST_YCBCR_COEFFICIENTS (1 << 4) |
| 613 | 616 | # define DP_YCBCR_COEFFICIENTS_ITU601 (0 << 4) |
| @@ -657,6 +660,16 @@ · 43 unmodified lines | ||
| 657 | 660 | |
| 658 | 661 | #define DP_TEST_SINK 0x270 |
| 659 | 662 | # define DP_TEST_SINK_START (1 << 0) |
| 663 | +#define DP_TEST_AUDIO_MODE 0x271 | |
| 664 | +#define DP_TEST_AUDIO_PATTERN_TYPE 0x272 | |
| 665 | +#define DP_TEST_AUDIO_PERIOD_CH1 0x273 | |
| 666 | +#define DP_TEST_AUDIO_PERIOD_CH2 0x274 | |
| 667 | +#define DP_TEST_AUDIO_PERIOD_CH3 0x275 | |
| 668 | +#define DP_TEST_AUDIO_PERIOD_CH4 0x276 | |
| 669 | +#define DP_TEST_AUDIO_PERIOD_CH5 0x277 | |
| 670 | +#define DP_TEST_AUDIO_PERIOD_CH6 0x278 | |
| 671 | +#define DP_TEST_AUDIO_PERIOD_CH7 0x279 | |
| 672 | +#define DP_TEST_AUDIO_PERIOD_CH8 0x27A | |
| 660 | 673 | |
| 661 | 674 | #define DP_FEC_STATUS 0x280 /* 1.4 */ |
| 662 | 675 | # define DP_FEC_DECODE_EN_DETECTED (1 << 0) |
| @@ -770,7 +770,7 @@ static inline bool drm_dev_is_unplugged(struct drm_device *dev) · 769 unmodified lines | ||
| 770 | 770 | * |
| 771 | 771 | * Returns true if the @feature is supported, false otherwise. |
| 772 | 772 | */ |
| 773 | −static inline bool drm_core_check_feature(struct drm_device *dev, u32 feature) | |
| 773 | +static inline bool drm_core_check_feature(const struct drm_device *dev, u32 feature) | |
| 774 | 774 | { |
| 775 | 775 | return dev->driver->driver_features & dev->driver_features & feature; |
| 776 | 776 | } |
| @@ -0,0 +1,36 @@ | ||
| 1 | +/* request_key authorisation token key type | |
| 2 | + * | |
| 3 | + * Copyright (C) 2005 Red Hat, Inc. All Rights Reserved. | |
| 4 | + * Written by David Howells (dhowells@redhat.com) | |
| 5 | + * | |
| 6 | + * This program is free software; you can redistribute it and/or | |
| 7 | + * modify it under the terms of the GNU General Public Licence | |
| 8 | + * as published by the Free Software Foundation; either version | |
| 9 | + * 2 of the Licence, or (at your option) any later version. | |
| 10 | + */ | |
| 11 | + | |
| 12 | +#ifndef _KEYS_REQUEST_KEY_AUTH_TYPE_H | |
| 13 | +#define _KEYS_REQUEST_KEY_AUTH_TYPE_H | |
| 14 | + | |
| 15 | +#include <linux/key.h> | |
| 16 | + | |
| 17 | +/* | |
| 18 | + * Authorisation record for request_key(). | |
| 19 | + */ | |
| 20 | +struct request_key_auth { | |
| 21 | + struct key *target_key; | |
| 22 | + struct key *dest_keyring; | |
| 23 | + const struct cred *cred; | |
| 24 | + void *callout_info; | |
| 25 | + size_t callout_len; | |
| 26 | + pid_t pid; | |
| 27 | + char op[8]; | |
| 28 | +} __randomize_layout; | |
| 29 | + | |
| 30 | +static inline struct request_key_auth *get_request_key_auth(const struct key *key) | |
| 31 | +{ | |
| 32 | + return key->payload.data[0]; | |
| 33 | +} | |
| 34 | + | |
| 35 | + | |
| 36 | +#endif /* _KEYS_REQUEST_KEY_AUTH_TYPE_H */ | |
| @@ -31,7 +31,7 @@ · 30 unmodified lines | ||
| 31 | 31 | struct user_key_payload { |
| 32 | 32 | struct rcu_head rcu; /* RCU destructor */ |
| 33 | 33 | unsigned short datalen; /* length of this data */ |
| 34 | − char data[0]; /* actual data */ | |
| 34 | + char data[0] __aligned(__alignof__(u64)); /* actual data */ | |
| 35 | 35 | }; |
| 36 | 36 | |
| 37 | 37 | extern struct key_type key_type_user; |
| @@ -20,15 +20,6 @@ · 19 unmodified lines | ||
| 20 | 20 | struct kernel_pkey_query; |
| 21 | 21 | struct kernel_pkey_params; |
| 22 | 22 | |
| 23 | −/* | |
| 24 | − * key under-construction record | |
| 25 | − * - passed to the request_key actor if supplied | |
| 26 | − */ | |
| 27 | −struct key_construction { | |
| 28 | − struct key *key; /* key being constructed */ | |
| 29 | − struct key *authkey;/* authorisation for key being constructed */ | |
| 30 | −}; | |
| 31 | − | |
| 32 | 23 | /* |
| 33 | 24 | * Pre-parsed payload, used by key add, update and instantiate. |
| 34 | 25 | * |
| @@ -50,8 +41,7 @@ struct key_preparsed_payload { · 15 unmodified lines | ||
| 50 | 41 | time64_t expiry; /* Expiry time of key */ |
| 51 | 42 | } __randomize_layout; |
| 52 | 43 | |
| 53 | −typedef int (*request_key_actor_t)(struct key_construction *key, | |
| 54 | − const char *op, void *aux); | |
| 44 | +typedef int (*request_key_actor_t)(struct key *auth_key, void *aux); | |
| 55 | 45 | |
| 56 | 46 | /* |
| 57 | 47 | * Preparsed matching criterion. |
| @@ -181,20 +171,20 @@ extern int key_instantiate_and_link(struct key *key, · 123 unmodified lines | ||
| 181 | 171 | const void *data, |
| 182 | 172 | size_t datalen, |
| 183 | 173 | struct key *keyring, |
| 184 | − struct key *instkey); | |
| 174 | + struct key *authkey); | |
| 185 | 175 | extern int key_reject_and_link(struct key *key, |
| 186 | 176 | unsigned timeout, |
| 187 | 177 | unsigned error, |
| 188 | 178 | struct key *keyring, |
| 189 | − struct key *instkey); | |
| 190 | −extern void complete_request_key(struct key_construction *cons, int error); | |
| 179 | + struct key *authkey); | |
| 180 | +extern void complete_request_key(struct key *authkey, int error); | |
| 191 | 181 | |
| 192 | 182 | static inline int key_negate_and_link(struct key *key, |
| 193 | 183 | unsigned timeout, |
| 194 | 184 | struct key *keyring, |
| 195 | − struct key *instkey) | |
| 185 | + struct key *authkey) | |
| 196 | 186 | { |
| 197 | − return key_reject_and_link(key, timeout, ENOKEY, keyring, instkey); | |
| 187 | + return key_reject_and_link(key, timeout, ENOKEY, keyring, authkey); | |
| 198 | 188 | } |
| 199 | 189 | |
| 200 | 190 | extern int generic_key_instantiate(struct key *key, struct key_preparsed_payload *prep); |
| @@ -11,6 +11,8 @@ · 10 unmodified lines | ||
| 11 | 11 | #define _LINUX_NETDEV_FEATURES_H |
| 12 | 12 | |
| 13 | 13 | #include <linux/types.h> |
| 14 | +#include <linux/bitops.h> | |
| 15 | +#include <asm/byteorder.h> | |
| 14 | 16 | |
| 15 | 17 | typedef u64 netdev_features_t; |
| 16 | 18 | |
| @@ -154,8 +156,26 @@ enum { · 137 unmodified lines | ||
| 154 | 156 | #define NETIF_F_HW_TLS_TX __NETIF_F(HW_TLS_TX) |
| 155 | 157 | #define NETIF_F_HW_TLS_RX __NETIF_F(HW_TLS_RX) |
| 156 | 158 | |
| 157 | −#define for_each_netdev_feature(mask_addr, bit) \ | |
| 158 | − for_each_set_bit(bit, (unsigned long *)mask_addr, NETDEV_FEATURE_COUNT) | |
| 159 | +/* Finds the next feature with the highest number of the range of start till 0. | |
| 160 | + */ | |
| 161 | +static inline int find_next_netdev_feature(u64 feature, unsigned long start) | |
| 162 | +{ | |
| 163 | + /* like BITMAP_LAST_WORD_MASK() for u64 | |
| 164 | + * this sets the most significant 64 - start to 0. | |
| 165 | + */ | |
| 166 | + feature &= ~0ULL >> (-start & ((sizeof(feature) * 8) - 1)); | |
| 167 | + | |
| 168 | + return fls64(feature) - 1; | |
| 169 | +} | |
| 170 | + | |
| 171 | +/* This goes for the MSB to the LSB through the set feature bits, | |
| 172 | + * mask_addr should be a u64 and bit an int | |
| 173 | + */ | |
| 174 | +#define for_each_netdev_feature(mask_addr, bit) \ | |
| 175 | + for ((bit) = find_next_netdev_feature((mask_addr), \ | |
| 176 | + NETDEV_FEATURE_COUNT); \ | |
| 177 | + (bit) >= 0; \ | |
| 178 | + (bit) = find_next_netdev_feature((mask_addr), (bit) - 1)) | |
| 159 | 179 | |
| 160 | 180 | /* Features valid for ethtool to change */ |
| 161 | 181 | /* = all defined minus driver/device-class-related */ |
| @@ -3861,7 +3861,7 @@ static inline u32 netif_msg_init(int debug_value, int default_msg_enable_bits) · 3860 unmodified lines | ||
| 3861 | 3861 | if (debug_value == 0) /* no output */ |
| 3862 | 3862 | return 0; |
| 3863 | 3863 | /* set low N bits */ |
| 3864 | − return (1 << debug_value) - 1; | |
| 3864 | + return (1U << debug_value) - 1; | |
| 3865 | 3865 | } |
| 3866 | 3866 | |
| 3867 | 3867 | static inline void __netif_tx_lock(struct netdev_queue *txq, int cpu) |
| @@ -992,6 +992,14 @@ static inline int genphy_no_soft_reset(struct phy_device *phydev) · 991 unmodified lines | ||
| 992 | 992 | { |
| 993 | 993 | return 0; |
| 994 | 994 | } |
| 995 | +static inline int genphy_no_ack_interrupt(struct phy_device *phydev) | |
| 996 | +{ | |
| 997 | + return 0; | |
| 998 | +} | |
| 999 | +static inline int genphy_no_config_intr(struct phy_device *phydev) | |
| 1000 | +{ | |
| 1001 | + return 0; | |
| 1002 | +} | |
| 995 | 1003 | int genphy_read_mmd_unsupported(struct phy_device *phdev, int devad, |
| 996 | 1004 | u16 regnum); |
| 997 | 1005 | int genphy_write_mmd_unsupported(struct phy_device *phdev, int devnum, |
| @@ -739,12 +739,6 @@ struct task_struct { · 738 unmodified lines | ||
| 739 | 739 | unsigned use_memdelay:1; |
| 740 | 740 | #endif |
| 741 | 741 | |
| 742 | − /* | |
| 743 | − * May usercopy functions fault on kernel addresses? | |
| 744 | − * This is not just a single bit because this can potentially nest. | |
| 745 | − */ | |
| 746 | − unsigned int kernel_uaccess_faults_ok; | |
| 747 | − | |
| 748 | 742 | unsigned long atomic_flags; /* Flags requiring atomic access. */ |
| 749 | 743 | |
| 750 | 744 | struct restart_block restart_block; |
| @@ -2434,7 +2434,7 @@ static inline void skb_probe_transport_header(struct sk_buff *skb, · 2433 unmodified lines | ||
| 2434 | 2434 | |
| 2435 | 2435 | if (skb_flow_dissect_flow_keys_basic(skb, &keys, NULL, 0, 0, 0, 0)) |
| 2436 | 2436 | skb_set_transport_header(skb, keys.control.thoff); |
| 2437 | − else | |
| 2437 | + else if (offset_hint >= 0) | |
| 2438 | 2438 | skb_set_transport_header(skb, offset_hint); |
| 2439 | 2439 | } |
| 2440 | 2440 | |
| @@ -4212,6 +4212,12 @@ static inline bool skb_is_gso_sctp(const struct sk_buff *skb) · 1771 unmodified lines | ||
| 4212 | 4212 | return skb_shinfo(skb)->gso_type & SKB_GSO_SCTP; |
| 4213 | 4213 | } |
| 4214 | 4214 | |
| 4215 | +static inline bool skb_is_gso_tcp(const struct sk_buff *skb) | |
| 4216 | +{ | |
| 4217 | + return skb_is_gso(skb) && | |
| 4218 | + skb_shinfo(skb)->gso_type & (SKB_GSO_TCPV4 | SKB_GSO_TCPV6); | |
| 4219 | +} | |
| 4220 | + | |
| 4215 | 4221 | static inline void skb_gso_reset(struct sk_buff *skb) |
| 4216 | 4222 | { |
| 4217 | 4223 | skb_shinfo(skb)->gso_size = 0; |
| @@ -57,6 +57,25 @@ static inline int virtio_net_hdr_to_skb(struct sk_buff *skb, · 56 unmodified lines | ||
| 57 | 57 | |
| 58 | 58 | if (!skb_partial_csum_set(skb, start, off)) |
| 59 | 59 | return -EINVAL; |
| 60 | + } else { | |
| 61 | + /* gso packets without NEEDS_CSUM do not set transport_offset. | |
| 62 | + * probe and drop if does not match one of the above types. | |
| 63 | + */ | |
| 64 | + if (gso_type && skb->network_header) { | |
| 65 | + if (!skb->protocol) | |
| 66 | + virtio_net_hdr_set_proto(skb, hdr); | |
| 67 | +retry: | |
| 68 | + skb_probe_transport_header(skb, -1); | |
| 69 | + if (!skb_transport_header_was_set(skb)) { | |
| 70 | + /* UFO does not specify ipv4 or 6: try both */ | |
| 71 | + if (gso_type & SKB_GSO_UDP && | |
| 72 | + skb->protocol == htons(ETH_P_IP)) { | |
| 73 | + skb->protocol = htons(ETH_P_IPV6); | |
| 74 | + goto retry; | |
| 75 | + } | |
| 76 | + return -EINVAL; | |
| 77 | + } | |
| 78 | + } | |
| 60 | 79 | } |
| 61 | 80 | |
| 62 | 81 | if (hdr->gso_type != VIRTIO_NET_HDR_GSO_NONE) { |
| @@ -22,6 +22,7 @@ · 21 unmodified lines | ||
| 22 | 22 | |
| 23 | 23 | #include <net/inet_sock.h> |
| 24 | 24 | #include <net/snmp.h> |
| 25 | +#include <net/ip.h> | |
| 25 | 26 | |
| 26 | 27 | struct icmp_err { |
| 27 | 28 | int errno; |
| @@ -39,7 +40,13 @@ struct net_proto_family; · 11 unmodified lines | ||
| 39 | 40 | struct sk_buff; |
| 40 | 41 | struct net; |
| 41 | 42 | |
| 42 | −void icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info); | |
| 43 | +void __icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info, | |
| 44 | + const struct ip_options *opt); | |
| 45 | +static inline void icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info) | |
| 46 | +{ | |
| 47 | + __icmp_send(skb_in, type, code, info, &IPCB(skb_in)->opt); | |
| 48 | +} | |
| 49 | + | |
| 43 | 50 | int icmp_rcv(struct sk_buff *skb); |
| 44 | 51 | int icmp_err(struct sk_buff *skb, u32 info); |
| 45 | 52 | int icmp_init(void); |
| @@ -667,6 +667,8 @@ static inline int ip_options_echo(struct net *net, struct ip_options *dopt, · 666 unmodified lines | ||
| 667 | 667 | } |
| 668 | 668 | |
| 669 | 669 | void ip_options_fragment(struct sk_buff *skb); |
| 670 | +int __ip_options_compile(struct net *net, struct ip_options *opt, | |
| 671 | + struct sk_buff *skb, __be32 *info); | |
| 670 | 672 | int ip_options_compile(struct net *net, struct ip_options *opt, |
| 671 | 673 | struct sk_buff *skb); |
| 672 | 674 | int ip_options_get(struct net *net, struct ip_options_rcu **optp, |
| @@ -716,7 +718,7 @@ extern int sysctl_icmp_msgs_burst; · 43 unmodified lines | ||
| 716 | 718 | int ip_misc_proc_init(void); |
| 717 | 719 | #endif |
| 718 | 720 | |
| 719 | −int rtm_getroute_parse_ip_proto(struct nlattr *attr, u8 *ip_proto, | |
| 721 | +int rtm_getroute_parse_ip_proto(struct nlattr *attr, u8 *ip_proto, u8 family, | |
| 720 | 722 | struct netlink_ext_ack *extack); |
| 721 | 723 | |
| 722 | 724 | #endif /* _IP_H */ |
| @@ -63,10 +63,11 @@ struct pnpipehdr { · 62 unmodified lines | ||
| 63 | 63 | u8 state_after_reset; /* reset request */ |
| 64 | 64 | u8 error_code; /* any response */ |
| 65 | 65 | u8 pep_type; /* status indication */ |
| 66 | − u8 data[1]; | |
| 66 | + u8 data0; /* anything else */ | |
| 67 | 67 | }; |
| 68 | + u8 data[]; | |
| 68 | 69 | }; |
| 69 | −#define other_pep_type data[1] | |
| 70 | +#define other_pep_type data[0] | |
| 70 | 71 | |
| 71 | 72 | static inline struct pnpipehdr *pnp_hdr(struct sk_buff *skb) |
| 72 | 73 | { |
| @@ -853,7 +853,7 @@ static inline void xfrm_pols_put(struct xfrm_policy **pols, int npols) · 852 unmodified lines | ||
| 853 | 853 | xfrm_pol_put(pols[i]); |
| 854 | 854 | } |
| 855 | 855 | |
| 856 | −void __xfrm_state_destroy(struct xfrm_state *); | |
| 856 | +void __xfrm_state_destroy(struct xfrm_state *, bool); | |
| 857 | 857 | |
| 858 | 858 | static inline void __xfrm_state_put(struct xfrm_state *x) |
| 859 | 859 | { |
| @@ -863,7 +863,13 @@ static inline void __xfrm_state_put(struct xfrm_state *x) · 3 unmodified lines | ||
| 863 | 863 | static inline void xfrm_state_put(struct xfrm_state *x) |
| 864 | 864 | { |
| 865 | 865 | if (refcount_dec_and_test(&x->refcnt)) |
| 866 | − __xfrm_state_destroy(x); | |
| 866 | + __xfrm_state_destroy(x, false); | |
| 867 | +} | |
| 868 | + | |
| 869 | +static inline void xfrm_state_put_sync(struct xfrm_state *x) | |
| 870 | +{ | |
| 871 | + if (refcount_dec_and_test(&x->refcnt)) | |
| 872 | + __xfrm_state_destroy(x, true); | |
| 867 | 873 | } |
| 868 | 874 | |
| 869 | 875 | static inline void xfrm_state_hold(struct xfrm_state *x) |
| @@ -1590,7 +1596,7 @@ struct xfrmk_spdinfo { · 720 unmodified lines | ||
| 1590 | 1596 | |
| 1591 | 1597 | struct xfrm_state *xfrm_find_acq_byseq(struct net *net, u32 mark, u32 seq); |
| 1592 | 1598 | int xfrm_state_delete(struct xfrm_state *x); |
| 1593 | −int xfrm_state_flush(struct net *net, u8 proto, bool task_valid); | |
| 1599 | +int xfrm_state_flush(struct net *net, u8 proto, bool task_valid, bool sync); | |
| 1594 | 1600 | int xfrm_dev_state_flush(struct net *net, struct net_device *dev, bool task_valid); |
| 1595 | 1601 | void xfrm_sad_getinfo(struct net *net, struct xfrmk_sadinfo *si); |
| 1596 | 1602 | void xfrm_spd_getinfo(struct net *net, struct xfrmk_spdinfo *si); |
| @@ -272,13 +272,14 @@ union drm_amdgpu_vm { · 271 unmodified lines | ||
| 272 | 272 | |
| 273 | 273 | /* sched ioctl */ |
| 274 | 274 | #define AMDGPU_SCHED_OP_PROCESS_PRIORITY_OVERRIDE 1 |
| 275 | +#define AMDGPU_SCHED_OP_CONTEXT_PRIORITY_OVERRIDE 2 | |
| 275 | 276 | |
| 276 | 277 | struct drm_amdgpu_sched_in { |
| 277 | 278 | /* AMDGPU_SCHED_OP_* */ |
| 278 | 279 | __u32 op; |
| 279 | 280 | __u32 fd; |
| 280 | 281 | __s32 priority; |
| 281 | − __u32 flags; | |
| 282 | + __u32 ctx_id; | |
| 282 | 283 | }; |
| 283 | 284 | |
| 284 | 285 | union drm_amdgpu_sched { |
| @@ -133,12 +133,63 @@ struct drm_nouveau_gem_cpu_fini { · 132 unmodified lines | ||
| 133 | 133 | #define DRM_NOUVEAU_NOTIFIEROBJ_ALLOC 0x05 /* deprecated */ |
| 134 | 134 | #define DRM_NOUVEAU_GPUOBJ_FREE 0x06 /* deprecated */ |
| 135 | 135 | #define DRM_NOUVEAU_NVIF 0x07 |
| 136 | +#define DRM_NOUVEAU_SVM_INIT 0x08 | |
| 137 | +#define DRM_NOUVEAU_SVM_BIND 0x09 | |
| 136 | 138 | #define DRM_NOUVEAU_GEM_NEW 0x40 |
| 137 | 139 | #define DRM_NOUVEAU_GEM_PUSHBUF 0x41 |
| 138 | 140 | #define DRM_NOUVEAU_GEM_CPU_PREP 0x42 |
| 139 | 141 | #define DRM_NOUVEAU_GEM_CPU_FINI 0x43 |
| 140 | 142 | #define DRM_NOUVEAU_GEM_INFO 0x44 |
| 141 | 143 | |
| 144 | +struct drm_nouveau_svm_init { | |
| 145 | + __u64 unmanaged_addr; | |
| 146 | + __u64 unmanaged_size; | |
| 147 | +}; | |
| 148 | + | |
| 149 | +struct drm_nouveau_svm_bind { | |
| 150 | + __u64 header; | |
| 151 | + __u64 va_start; | |
| 152 | + __u64 va_end; | |
| 153 | + __u64 npages; | |
| 154 | + __u64 stride; | |
| 155 | + __u64 result; | |
| 156 | + __u64 reserved0; | |
| 157 | + __u64 reserved1; | |
| 158 | +}; | |
| 159 | + | |
| 160 | +#define NOUVEAU_SVM_BIND_COMMAND_SHIFT 0 | |
| 161 | +#define NOUVEAU_SVM_BIND_COMMAND_BITS 8 | |
| 162 | +#define NOUVEAU_SVM_BIND_COMMAND_MASK ((1 << 8) - 1) | |
| 163 | +#define NOUVEAU_SVM_BIND_PRIORITY_SHIFT 8 | |
| 164 | +#define NOUVEAU_SVM_BIND_PRIORITY_BITS 8 | |
| 165 | +#define NOUVEAU_SVM_BIND_PRIORITY_MASK ((1 << 8) - 1) | |
| 166 | +#define NOUVEAU_SVM_BIND_TARGET_SHIFT 16 | |
| 167 | +#define NOUVEAU_SVM_BIND_TARGET_BITS 32 | |
| 168 | +#define NOUVEAU_SVM_BIND_TARGET_MASK 0xffffffff | |
| 169 | + | |
| 170 | +/* | |
| 171 | + * Below is use to validate ioctl argument, userspace can also use it to make | |
| 172 | + * sure that no bit are set beyond known fields for a given kernel version. | |
| 173 | + */ | |
| 174 | +#define NOUVEAU_SVM_BIND_VALID_BITS 48 | |
| 175 | +#define NOUVEAU_SVM_BIND_VALID_MASK ((1ULL << NOUVEAU_SVM_BIND_VALID_BITS) - 1) | |
| 176 | + | |
| 177 | + | |
| 178 | +/* | |
| 179 | + * NOUVEAU_BIND_COMMAND__MIGRATE: synchronous migrate to target memory. | |
| 180 | + * result: number of page successfuly migrate to the target memory. | |
| 181 | + */ | |
| 182 | +#define NOUVEAU_SVM_BIND_COMMAND__MIGRATE 0 | |
| 183 | + | |
| 184 | +/* | |
| 185 | + * NOUVEAU_SVM_BIND_HEADER_TARGET__GPU_VRAM: target the GPU VRAM memory. | |
| 186 | + */ | |
| 187 | +#define NOUVEAU_SVM_BIND_TARGET__GPU_VRAM (1UL << 31) | |
| 188 | + | |
| 189 | + | |
| 190 | +#define DRM_IOCTL_NOUVEAU_SVM_INIT DRM_IOWR(DRM_COMMAND_BASE + DRM_NOUVEAU_SVM_INIT, struct drm_nouveau_svm_init) | |
| 191 | +#define DRM_IOCTL_NOUVEAU_SVM_BIND DRM_IOWR(DRM_COMMAND_BASE + DRM_NOUVEAU_SVM_BIND, struct drm_nouveau_svm_bind) | |
| 192 | + | |
| 142 | 193 | #define DRM_IOCTL_NOUVEAU_GEM_NEW DRM_IOWR(DRM_COMMAND_BASE + DRM_NOUVEAU_GEM_NEW, struct drm_nouveau_gem_new) |
| 143 | 194 | #define DRM_IOCTL_NOUVEAU_GEM_PUSHBUF DRM_IOWR(DRM_COMMAND_BASE + DRM_NOUVEAU_GEM_PUSHBUF, struct drm_nouveau_gem_pushbuf) |
| 144 | 195 | #define DRM_IOCTL_NOUVEAU_GEM_CPU_PREP DRM_IOW (DRM_COMMAND_BASE + DRM_NOUVEAU_GEM_CPU_PREP, struct drm_nouveau_gem_cpu_prep) |
| @@ -348,6 +348,7 @@ int ipu_prg_channel_configure(struct ipuv3_channel *ipu_chan, · 347 unmodified lines | ||
| 348 | 348 | unsigned int axi_id, unsigned int width, |
| 349 | 349 | unsigned int height, unsigned int stride, |
| 350 | 350 | u32 format, uint64_t modifier, unsigned long *eba); |
| 351 | +bool ipu_prg_channel_configure_pending(struct ipuv3_channel *ipu_chan); | |
| 351 | 352 | |
| 352 | 353 | /* |
| 353 | 354 | * IPU CMOS Sensor Interface (csi) functions |
| @@ -550,6 +550,7 @@ skip: · 549 unmodified lines | ||
| 550 | 550 | initrd_end = 0; |
| 551 | 551 | } |
| 552 | 552 | |
| 553 | +#ifdef CONFIG_BLK_DEV_RAM | |
| 553 | 554 | #define BUF_SIZE 1024 |
| 554 | 555 | static void __init clean_rootfs(void) |
| 555 | 556 | { |
| @@ -596,6 +597,7 @@ static void __init clean_rootfs(void) · 40 unmodified lines | ||
| 596 | 597 | ksys_close(fd); |
| 597 | 598 | kfree(buf); |
| 598 | 599 | } |
| 600 | +#endif | |
| 599 | 601 | |
| 600 | 602 | static int __init populate_rootfs(void) |
| 601 | 603 | { |
| @@ -638,10 +640,8 @@ static int __init populate_rootfs(void) · 36 unmodified lines | ||
| 638 | 640 | printk(KERN_INFO "Unpacking initramfs...\n"); |
| 639 | 641 | err = unpack_to_rootfs((char *)initrd_start, |
| 640 | 642 | initrd_end - initrd_start); |
| 641 | − if (err) { | |
| 643 | + if (err) | |
| 642 | 644 | printk(KERN_EMERG "Initramfs unpacking failed: %s\n", err); |
| 643 | − clean_rootfs(); | |
| 644 | − } | |
| 645 | 645 | free_initrd(); |
| 646 | 646 | #endif |
| 647 | 647 | } |
| @@ -471,6 +471,7 @@ static int trie_delete_elem(struct bpf_map *map, void *_key) · 470 unmodified lines | ||
| 471 | 471 | } |
| 472 | 472 | |
| 473 | 473 | if (!node || node->prefixlen != key->prefixlen || |
| 474 | + node->prefixlen != matchlen || | |
| 474 | 475 | (node->flags & LPM_TREE_NODE_FLAG_IM)) { |
| 475 | 476 | ret = -ENOENT; |
| 476 | 477 | goto out; |
| @@ -44,7 +44,7 @@ static void do_up_read(struct irq_work *entry) · 43 unmodified lines | ||
| 44 | 44 | struct stack_map_irq_work *work; |
| 45 | 45 | |
| 46 | 46 | work = container_of(entry, struct stack_map_irq_work, irq_work); |
| 47 | − up_read(work->sem); | |
| 47 | + up_read_non_owner(work->sem); | |
| 48 | 48 | work->sem = NULL; |
| 49 | 49 | } |
| 50 | 50 | |
| @@ -338,6 +338,12 @@ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs, · 287 unmodified lines | ||
| 338 | 338 | } else { |
| 339 | 339 | work->sem = ¤t->mm->mmap_sem; |
| 340 | 340 | irq_work_queue(&work->irq_work); |
| 341 | + /* | |
| 342 | + * The irq_work will release the mmap_sem with | |
| 343 | + * up_read_non_owner(). The rwsem_release() is called | |
| 344 | + * here to release the lock from lockdep's perspective. | |
| 345 | + */ | |
| 346 | + rwsem_release(¤t->mm->mmap_sem.dep_map, 1, _RET_IP_); | |
| 341 | 347 | } |
| 342 | 348 | } |
| 343 | 349 | |
| @@ -559,12 +559,12 @@ static int map_create(union bpf_attr *attr) · 558 unmodified lines | ||
| 559 | 559 | err = bpf_map_new_fd(map, f_flags); |
| 560 | 560 | if (err < 0) { |
| 561 | 561 | /* failed to allocate fd. |
| 562 | − * bpf_map_put() is needed because the above | |
| 562 | + * bpf_map_put_with_uref() is needed because the above | |
| 563 | 563 | * bpf_map_alloc_id() has published the map |
| 564 | 564 | * to the userspace and the userspace may |
| 565 | 565 | * have refcnt-ed it through BPF_MAP_GET_FD_BY_ID. |
| 566 | 566 | */ |
| 567 | − bpf_map_put(map); | |
| 567 | + bpf_map_put_with_uref(map); | |
| 568 | 568 | return err; |
| 569 | 569 | } |
| 570 | 570 | |
| @@ -1986,7 +1986,7 @@ static int bpf_map_get_fd_by_id(const union bpf_attr *attr) · 1415 unmodified lines | ||
| 1986 | 1986 | |
| 1987 | 1987 | fd = bpf_map_new_fd(map, f_flags); |
| 1988 | 1988 | if (fd < 0) |
| 1989 | − bpf_map_put(map); | |
| 1989 | + bpf_map_put_with_uref(map); | |
| 1990 | 1990 | |
| 1991 | 1991 | return fd; |
| 1992 | 1992 | } |
| @@ -1617,12 +1617,13 @@ static int check_flow_keys_access(struct bpf_verifier_env *env, int off, · 1616 unmodified lines | ||
| 1617 | 1617 | return 0; |
| 1618 | 1618 | } |
| 1619 | 1619 | |
| 1620 | −static int check_sock_access(struct bpf_verifier_env *env, u32 regno, int off, | |
| 1621 | − int size, enum bpf_access_type t) | |
| 1620 | +static int check_sock_access(struct bpf_verifier_env *env, int insn_idx, | |
| 1621 | + u32 regno, int off, int size, | |
| 1622 | + enum bpf_access_type t) | |
| 1622 | 1623 | { |
| 1623 | 1624 | struct bpf_reg_state *regs = cur_regs(env); |
| 1624 | 1625 | struct bpf_reg_state *reg = ®s[regno]; |
| 1625 | − struct bpf_insn_access_aux info; | |
| 1626 | + struct bpf_insn_access_aux info = {}; | |
| 1626 | 1627 | |
| 1627 | 1628 | if (reg->smin_value < 0) { |
| 1628 | 1629 | verbose(env, "R%d min value is negative, either use unsigned index or do a if (index >=0) check.\n", |
| @@ -1636,6 +1637,8 @@ static int check_sock_access(struct bpf_verifier_env *env, u32 regno, int off, · 7 unmodified lines | ||
| 1636 | 1637 | return -EACCES; |
| 1637 | 1638 | } |
| 1638 | 1639 | |
| 1640 | + env->insn_aux_data[insn_idx].ctx_field_size = info.ctx_field_size; | |
| 1641 | + | |
| 1639 | 1642 | return 0; |
| 1640 | 1643 | } |
| 1641 | 1644 | |
| @@ -2032,7 +2035,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn · 390 unmodified lines | ||
| 2032 | 2035 | verbose(env, "cannot write into socket\n"); |
| 2033 | 2036 | return -EACCES; |
| 2034 | 2037 | } |
| 2035 | − err = check_sock_access(env, regno, off, size, t); | |
| 2038 | + err = check_sock_access(env, insn_idx, regno, off, size, t); | |
| 2036 | 2039 | if (!err && value_regno >= 0) |
| 2037 | 2040 | mark_reg_unknown(env, regs, value_regno); |
| 2038 | 2041 | } else { |
| @@ -6917,7 +6920,8 @@ static int fixup_bpf_calls(struct bpf_verifier_env *env) · 4878 unmodified lines | ||
| 6917 | 6920 | u32 off_reg; |
| 6918 | 6921 | |
| 6919 | 6922 | aux = &env->insn_aux_data[i + delta]; |
| 6920 | − if (!aux->alu_state) | |
| 6923 | + if (!aux->alu_state || | |
| 6924 | + aux->alu_state == BPF_ALU_NON_POINTER) | |
| 6921 | 6925 | continue; |
| 6922 | 6926 | |
| 6923 | 6927 | isneg = aux->alu_state & BPF_ALU_NEG_VALUE; |
| @@ -322,7 +322,7 @@ static bool update_stats(struct psi_group *group) · 321 unmodified lines | ||
| 322 | 322 | expires = group->next_update; |
| 323 | 323 | if (now < expires) |
| 324 | 324 | goto out; |
| 325 | − if (now - expires > psi_period) | |
| 325 | + if (now - expires >= psi_period) | |
| 326 | 326 | missed_periods = div_u64(now - expires, psi_period); |
| 327 | 327 | |
| 328 | 328 | /* |
| @@ -3384,6 +3384,8 @@ static void print_func_help_header_irq(struct trace_buffer *buf, struct seq_file · 3383 unmodified lines | ||
| 3384 | 3384 | const char tgid_space[] = " "; |
| 3385 | 3385 | const char space[] = " "; |
| 3386 | 3386 | |
| 3387 | + print_event_info(buf, m); | |
| 3388 | + | |
| 3387 | 3389 | seq_printf(m, "# %s _-----=> irqs-off\n", |
| 3388 | 3390 | tgid ? tgid_space : space); |
| 3389 | 3391 | seq_printf(m, "# %s / _----=> need-resched\n", |
| @@ -861,22 +861,14 @@ static const struct file_operations kprobe_profile_ops = { · 860 unmodified lines | ||
| 861 | 861 | static nokprobe_inline int |
| 862 | 862 | fetch_store_strlen(unsigned long addr) |
| 863 | 863 | { |
| 864 | − mm_segment_t old_fs; | |
| 865 | 864 | int ret, len = 0; |
| 866 | 865 | u8 c; |
| 867 | 866 | |
| 868 | − old_fs = get_fs(); | |
| 869 | − set_fs(KERNEL_DS); | |
| 870 | − pagefault_disable(); | |
| 871 | − | |
| 872 | 867 | do { |
| 873 | − ret = __copy_from_user_inatomic(&c, (u8 *)addr + len, 1); | |
| 868 | + ret = probe_mem_read(&c, (u8 *)addr + len, 1); | |
| 874 | 869 | len++; |
| 875 | 870 | } while (c && ret == 0 && len < MAX_STRING_SIZE); |
| 876 | 871 | |
| 877 | − pagefault_enable(); | |
| 878 | − set_fs(old_fs); | |
| 879 | − | |
| 880 | 872 | return (ret < 0) ? ret : len; |
| 881 | 873 | } |
| 882 | 874 | |
| @@ -113,6 +113,28 @@ config KASAN_INLINE · 112 unmodified lines | ||
| 113 | 113 | |
| 114 | 114 | endchoice |
| 115 | 115 | |
| 116 | +config KASAN_STACK_ENABLE | |
| 117 | + bool "Enable stack instrumentation (unsafe)" if CC_IS_CLANG && !COMPILE_TEST | |
| 118 | + default !(CLANG_VERSION < 90000) | |
| 119 | + depends on KASAN | |
| 120 | + help | |
| 121 | + The LLVM stack address sanitizer has a know problem that | |
| 122 | + causes excessive stack usage in a lot of functions, see | |
| 123 | + https://bugs.llvm.org/show_bug.cgi?id=38809 | |
| 124 | + Disabling asan-stack makes it safe to run kernels build | |
| 125 | + with clang-8 with KASAN enabled, though it loses some of | |
| 126 | + the functionality. | |
| 127 | + This feature is always disabled when compile-testing with clang-8 | |
| 128 | + or earlier to avoid cluttering the output in stack overflow | |
| 129 | + warnings, but clang-8 users can still enable it for builds without | |
| 130 | + CONFIG_COMPILE_TEST. On gcc and later clang versions it is | |
| 131 | + assumed to always be safe to use and enabled by default. | |
| 132 | + | |
| 133 | +config KASAN_STACK | |
| 134 | + int | |
| 135 | + default 1 if KASAN_STACK_ENABLE || CC_IS_GCC | |
| 136 | + default 0 | |
| 137 | + | |
| 116 | 138 | config KASAN_S390_4_LEVEL_PAGING |
| 117 | 139 | bool "KASan: use 4-level paging" |
| 118 | 140 | depends on KASAN && S390 |
| @@ -768,9 +768,11 @@ all_leaves_cluster_together: · 767 unmodified lines | ||
| 768 | 768 | new_s0->index_key[i] = |
| 769 | 769 | ops->get_key_chunk(index_key, i * ASSOC_ARRAY_KEY_CHUNK_SIZE); |
| 770 | 770 | |
| 771 | − blank = ULONG_MAX << (level & ASSOC_ARRAY_KEY_CHUNK_MASK); | |
| 772 | − pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, level, blank); | |
| 773 | − new_s0->index_key[keylen - 1] &= ~blank; | |
| 771 | + if (level & ASSOC_ARRAY_KEY_CHUNK_MASK) { | |
| 772 | + blank = ULONG_MAX << (level & ASSOC_ARRAY_KEY_CHUNK_MASK); | |
| 773 | + pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, level, blank); | |
| 774 | + new_s0->index_key[keylen - 1] &= ~blank; | |
| 775 | + } | |
| 774 | 776 | |
| 775 | 777 | /* This now reduces to a node splitting exercise for which we'll need |
| 776 | 778 | * to regenerate the disparity table. |
| @@ -44,7 +44,7 @@ const struct trace_print_flags vmaflag_names[] = { · 43 unmodified lines | ||
| 44 | 44 | |
| 45 | 45 | void __dump_page(struct page *page, const char *reason) |
| 46 | 46 | { |
| 47 | − struct address_space *mapping = page_mapping(page); | |
| 47 | + struct address_space *mapping; | |
| 48 | 48 | bool page_poisoned = PagePoisoned(page); |
| 49 | 49 | int mapcount; |
| 50 | 50 | |
| @@ -58,6 +58,8 @@ void __dump_page(struct page *page, const char *reason) · 7 unmodified lines | ||
| 58 | 58 | goto hex_only; |
| 59 | 59 | } |
| 60 | 60 | |
| 61 | + mapping = page_mapping(page); | |
| 62 | + | |
| 61 | 63 | /* |
| 62 | 64 | * Avoid VM_BUG_ON() in page_mapcount(). |
| 63 | 65 | * page->_mapcount space in struct page is used by sl[aou]b pages to |
| @@ -3624,7 +3624,6 @@ retry_avoidcopy: · 3623 unmodified lines | ||
| 3624 | 3624 | copy_user_huge_page(new_page, old_page, address, vma, |
| 3625 | 3625 | pages_per_huge_page(h)); |
| 3626 | 3626 | __SetPageUptodate(new_page); |
| 3627 | − set_page_huge_active(new_page); | |
| 3628 | 3627 | |
| 3629 | 3628 | mmu_notifier_range_init(&range, mm, haddr, haddr + huge_page_size(h)); |
| 3630 | 3629 | mmu_notifier_invalidate_range_start(&range); |
| @@ -3645,6 +3644,7 @@ retry_avoidcopy: · 14 unmodified lines | ||
| 3645 | 3644 | make_huge_pte(vma, new_page, 1)); |
| 3646 | 3645 | page_remove_rmap(old_page, true); |
| 3647 | 3646 | hugepage_add_new_anon_rmap(new_page, vma, haddr); |
| 3647 | + set_page_huge_active(new_page); | |
| 3648 | 3648 | /* Make the old page be freed below */ |
| 3649 | 3649 | new_page = old_page; |
| 3650 | 3650 | } |
| @@ -3729,6 +3729,7 @@ static vm_fault_t hugetlb_no_page(struct mm_struct *mm, · 78 unmodified lines | ||
| 3729 | 3729 | pte_t new_pte; |
| 3730 | 3730 | spinlock_t *ptl; |
| 3731 | 3731 | unsigned long haddr = address & huge_page_mask(h); |
| 3732 | + bool new_page = false; | |
| 3732 | 3733 | |
| 3733 | 3734 | /* |
| 3734 | 3735 | * Currently, we are forced to kill the process in the event the |
| @@ -3790,7 +3791,7 @@ retry: · 55 unmodified lines | ||
| 3790 | 3791 | } |
| 3791 | 3792 | clear_huge_page(page, address, pages_per_huge_page(h)); |
| 3792 | 3793 | __SetPageUptodate(page); |
| 3793 | − set_page_huge_active(page); | |
| 3794 | + new_page = true; | |
| 3794 | 3795 | |
| 3795 | 3796 | if (vma->vm_flags & VM_MAYSHARE) { |
| 3796 | 3797 | int err = huge_add_to_page_cache(page, mapping, idx); |
| @@ -3861,6 +3862,15 @@ retry: · 64 unmodified lines | ||
| 3861 | 3862 | } |
| 3862 | 3863 | |
| 3863 | 3864 | spin_unlock(ptl); |
| 3865 | + | |
| 3866 | + /* | |
| 3867 | + * Only make newly allocated pages active. Existing pages found | |
| 3868 | + * in the pagecache could be !page_huge_active() if they have been | |
| 3869 | + * isolated for migration. | |
| 3870 | + */ | |
| 3871 | + if (new_page) | |
| 3872 | + set_page_huge_active(page); | |
| 3873 | + | |
| 3864 | 3874 | unlock_page(page); |
| 3865 | 3875 | out: |
| 3866 | 3876 | return ret; |
| @@ -4095,7 +4105,6 @@ int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, · 228 unmodified lines | ||
| 4095 | 4105 | * the set_pte_at() write. |
| 4096 | 4106 | */ |
| 4097 | 4107 | __SetPageUptodate(page); |
| 4098 | − set_page_huge_active(page); | |
| 4099 | 4108 | |
| 4100 | 4109 | mapping = dst_vma->vm_file->f_mapping; |
| 4101 | 4110 | idx = vma_hugecache_offset(h, dst_vma, dst_addr); |
| @@ -4163,6 +4172,7 @@ int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, · 61 unmodified lines | ||
| 4163 | 4172 | update_mmu_cache(dst_vma, dst_addr, dst_pte); |
| 4164 | 4173 | |
| 4165 | 4174 | spin_unlock(ptl); |
| 4175 | + set_page_huge_active(page); | |
| 4166 | 4176 | if (vm_shared) |
| 4167 | 4177 | unlock_page(page); |
| 4168 | 4178 | ret = 0; |
| @@ -7,6 +7,8 @@ KCOV_INSTRUMENT := n · 6 unmodified lines | ||
| 7 | 7 | |
| 8 | 8 | CFLAGS_REMOVE_common.o = -pg |
| 9 | 9 | CFLAGS_REMOVE_generic.o = -pg |
| 10 | +CFLAGS_REMOVE_tags.o = -pg | |
| 11 | + | |
| 10 | 12 | # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 |
| 11 | 13 | # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 |
| 12 | 14 | |
| @@ -361,10 +361,15 @@ void kasan_poison_object_data(struct kmem_cache *cache, void *object) · 360 unmodified lines | ||
| 361 | 361 | * get different tags. |
| 362 | 362 | */ |
| 363 | 363 | static u8 assign_tag(struct kmem_cache *cache, const void *object, |
| 364 | − bool init, bool krealloc) | |
| 364 | + bool init, bool keep_tag) | |
| 365 | 365 | { |
| 366 | − /* Reuse the same tag for krealloc'ed objects. */ | |
| 367 | − if (krealloc) | |
| 366 | + /* | |
| 367 | + * 1. When an object is kmalloc()'ed, two hooks are called: | |
| 368 | + * kasan_slab_alloc() and kasan_kmalloc(). We assign the | |
| 369 | + * tag only in the first one. | |
| 370 | + * 2. We reuse the same tag for krealloc'ed objects. | |
| 371 | + */ | |
| 372 | + if (keep_tag) | |
| 368 | 373 | return get_tag(object); |
| 369 | 374 | |
| 370 | 375 | /* |
| @@ -405,12 +410,6 @@ void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, · 34 unmodified lines | ||
| 405 | 410 | return (void *)object; |
| 406 | 411 | } |
| 407 | 412 | |
| 408 | −void * __must_check kasan_slab_alloc(struct kmem_cache *cache, void *object, | |
| 409 | − gfp_t flags) | |
| 410 | −{ | |
| 411 | − return kasan_kmalloc(cache, object, cache->object_size, flags); | |
| 412 | −} | |
| 413 | − | |
| 414 | 413 | static inline bool shadow_invalid(u8 tag, s8 shadow_byte) |
| 415 | 414 | { |
| 416 | 415 | if (IS_ENABLED(CONFIG_KASAN_GENERIC)) |
| @@ -467,7 +466,7 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) · 50 unmodified lines | ||
| 467 | 466 | } |
| 468 | 467 | |
| 469 | 468 | static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, |
| 470 | − size_t size, gfp_t flags, bool krealloc) | |
| 469 | + size_t size, gfp_t flags, bool keep_tag) | |
| 471 | 470 | { |
| 472 | 471 | unsigned long redzone_start; |
| 473 | 472 | unsigned long redzone_end; |
| @@ -485,7 +484,7 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, · 11 unmodified lines | ||
| 485 | 484 | KASAN_SHADOW_SCALE_SIZE); |
| 486 | 485 | |
| 487 | 486 | if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) |
| 488 | − tag = assign_tag(cache, object, false, krealloc); | |
| 487 | + tag = assign_tag(cache, object, false, keep_tag); | |
| 489 | 488 | |
| 490 | 489 | /* Tag is ignored in set_tag without CONFIG_KASAN_SW_TAGS */ |
| 491 | 490 | kasan_unpoison_shadow(set_tag(object, tag), size); |
| @@ -498,10 +497,16 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, · 6 unmodified lines | ||
| 498 | 497 | return set_tag(object, tag); |
| 499 | 498 | } |
| 500 | 499 | |
| 500 | +void * __must_check kasan_slab_alloc(struct kmem_cache *cache, void *object, | |
| 501 | + gfp_t flags) | |
| 502 | +{ | |
| 503 | + return __kasan_kmalloc(cache, object, cache->object_size, flags, false); | |
| 504 | +} | |
| 505 | + | |
| 501 | 506 | void * __must_check kasan_kmalloc(struct kmem_cache *cache, const void *object, |
| 502 | 507 | size_t size, gfp_t flags) |
| 503 | 508 | { |
| 504 | − return __kasan_kmalloc(cache, object, size, flags, false); | |
| 509 | + return __kasan_kmalloc(cache, object, size, flags, true); | |
| 505 | 510 | } |
| 506 | 511 | EXPORT_SYMBOL(kasan_kmalloc); |
| 507 | 512 | |
| @@ -46,7 +46,7 @@ void kasan_init_tags(void) · 45 unmodified lines | ||
| 46 | 46 | int cpu; |
| 47 | 47 | |
| 48 | 48 | for_each_possible_cpu(cpu) |
| 49 | − per_cpu(prng_state, cpu) = get_random_u32(); | |
| 49 | + per_cpu(prng_state, cpu) = (u32)get_cycles(); | |
| 50 | 50 | } |
| 51 | 51 | |
| 52 | 52 | /* |
| @@ -574,6 +574,7 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size, · 573 unmodified lines | ||
| 574 | 574 | unsigned long flags; |
| 575 | 575 | struct kmemleak_object *object, *parent; |
| 576 | 576 | struct rb_node **link, *rb_parent; |
| 577 | + unsigned long untagged_ptr; | |
| 577 | 578 | |
| 578 | 579 | object = kmem_cache_alloc(object_cache, gfp_kmemleak_mask(gfp)); |
| 579 | 580 | if (!object) { |
| @@ -619,8 +620,9 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size, · 39 unmodified lines | ||
| 619 | 620 | |
| 620 | 621 | write_lock_irqsave(&kmemleak_lock, flags); |
| 621 | 622 | |
| 622 | − min_addr = min(min_addr, ptr); | |
| 623 | − max_addr = max(max_addr, ptr + size); | |
| 623 | + untagged_ptr = (unsigned long)kasan_reset_tag((void *)ptr); | |
| 624 | + min_addr = min(min_addr, untagged_ptr); | |
| 625 | + max_addr = max(max_addr, untagged_ptr + size); | |
| 624 | 626 | link = &object_tree_root.rb_node; |
| 625 | 627 | rb_parent = NULL; |
| 626 | 628 | while (*link) { |
| @@ -1333,6 +1335,7 @@ static void scan_block(void *_start, void *_end, · 706 unmodified lines | ||
| 1333 | 1335 | unsigned long *start = PTR_ALIGN(_start, BYTES_PER_POINTER); |
| 1334 | 1336 | unsigned long *end = _end - (BYTES_PER_POINTER - 1); |
| 1335 | 1337 | unsigned long flags; |
| 1338 | + unsigned long untagged_ptr; | |
| 1336 | 1339 | |
| 1337 | 1340 | read_lock_irqsave(&kmemleak_lock, flags); |
| 1338 | 1341 | for (ptr = start; ptr < end; ptr++) { |
| @@ -1347,7 +1350,8 @@ static void scan_block(void *_start, void *_end, · 8 unmodified lines | ||
| 1347 | 1350 | pointer = *ptr; |
| 1348 | 1351 | kasan_enable_current(); |
| 1349 | 1352 | |
| 1350 | − if (pointer < min_addr || pointer >= max_addr) | |
| 1353 | + untagged_ptr = (unsigned long)kasan_reset_tag((void *)pointer); | |
| 1354 | + if (untagged_ptr < min_addr || untagged_ptr >= max_addr) | |
| 1351 | 1355 | continue; |
| 1352 | 1356 | |
| 1353 | 1357 | /* |
| @@ -30,10 +30,8 @@ long __probe_kernel_read(void *dst, const void *src, size_t size) · 29 unmodified lines | ||
| 30 | 30 | |
| 31 | 31 | set_fs(KERNEL_DS); |
| 32 | 32 | pagefault_disable(); |
| 33 | − current->kernel_uaccess_faults_ok++; | |
| 34 | 33 | ret = __copy_from_user_inatomic(dst, |
| 35 | 34 | (__force const void __user *)src, size); |
| 36 | − current->kernel_uaccess_faults_ok--; | |
| 37 | 35 | pagefault_enable(); |
| 38 | 36 | set_fs(old_fs); |
| 39 | 37 | |
| @@ -60,9 +58,7 @@ long __probe_kernel_write(void *dst, const void *src, size_t size) · 20 unmodified lines | ||
| 60 | 58 | |
| 61 | 59 | set_fs(KERNEL_DS); |
| 62 | 60 | pagefault_disable(); |
| 63 | − current->kernel_uaccess_faults_ok++; | |
| 64 | 61 | ret = __copy_to_user_inatomic((__force void __user *)dst, src, size); |
| 65 | − current->kernel_uaccess_faults_ok--; | |
| 66 | 62 | pagefault_enable(); |
| 67 | 63 | set_fs(old_fs); |
| 68 | 64 | |
| @@ -98,13 +94,11 @@ long strncpy_from_unsafe(char *dst, const void *unsafe_addr, long count) · 29 unmodified lines | ||
| 98 | 94 | |
| 99 | 95 | set_fs(KERNEL_DS); |
| 100 | 96 | pagefault_disable(); |
| 101 | − current->kernel_uaccess_faults_ok++; | |
| 102 | 97 | |
| 103 | 98 | do { |
| 104 | 99 | ret = __get_user(*dst++, (const char __user __force *)src++); |
| 105 | 100 | } while (dst[-1] && ret == 0 && src - unsafe_addr < count); |
| 106 | 101 | |
| 107 | − current->kernel_uaccess_faults_ok--; | |
| 108 | 102 | dst[-1] = '\0'; |
| 109 | 103 | pagefault_enable(); |
| 110 | 104 | set_fs(old_fs); |
| @@ -1188,11 +1188,13 @@ static inline int pageblock_free(struct page *page) · 1187 unmodified lines | ||
| 1188 | 1188 | return PageBuddy(page) && page_order(page) >= pageblock_order; |
| 1189 | 1189 | } |
| 1190 | 1190 | |
| 1191 | −/* Return the start of the next active pageblock after a given page */ | |
| 1192 | −static struct page *next_active_pageblock(struct page *page) | |
| 1191 | +/* Return the pfn of the start of the next active pageblock after a given pfn */ | |
| 1192 | +static unsigned long next_active_pageblock(unsigned long pfn) | |
| 1193 | 1193 | { |
| 1194 | + struct page *page = pfn_to_page(pfn); | |
| 1195 | + | |
| 1194 | 1196 | /* Ensure the starting page is pageblock-aligned */ |
| 1195 | − BUG_ON(page_to_pfn(page) & (pageblock_nr_pages - 1)); | |
| 1197 | + BUG_ON(pfn & (pageblock_nr_pages - 1)); | |
| 1196 | 1198 | |
| 1197 | 1199 | /* If the entire pageblock is free, move to the end of free page */ |
| 1198 | 1200 | if (pageblock_free(page)) { |
| @@ -1200,16 +1202,16 @@ static struct page *next_active_pageblock(struct page *page) · 1 unmodified line | ||
| 1200 | 1202 | /* be careful. we don't have locks, page_order can be changed.*/ |
| 1201 | 1203 | order = page_order(page); |
| 1202 | 1204 | if ((order < MAX_ORDER) && (order >= pageblock_order)) |
| 1203 | − return page + (1 << order); | |
| 1205 | + return pfn + (1 << order); | |
| 1204 | 1206 | } |
| 1205 | 1207 | |
| 1206 | − return page + pageblock_nr_pages; | |
| 1208 | + return pfn + pageblock_nr_pages; | |
| 1207 | 1209 | } |
| 1208 | 1210 | |
| 1209 | −static bool is_pageblock_removable_nolock(struct page *page) | |
| 1211 | +static bool is_pageblock_removable_nolock(unsigned long pfn) | |
| 1210 | 1212 | { |
| 1213 | + struct page *page = pfn_to_page(pfn); | |
| 1211 | 1214 | struct zone *zone; |
| 1212 | − unsigned long pfn; | |
| 1213 | 1215 | |
| 1214 | 1216 | /* |
| 1215 | 1217 | * We have to be careful here because we are iterating over memory |
| @@ -1232,13 +1234,14 @@ static bool is_pageblock_removable_nolock(struct page *page) · 16 unmodified lines | ||
| 1232 | 1234 | /* Checks if this range of memory is likely to be hot-removable. */ |
| 1233 | 1235 | bool is_mem_section_removable(unsigned long start_pfn, unsigned long nr_pages) |
| 1234 | 1236 | { |
| 1235 | − struct page *page = pfn_to_page(start_pfn); | |
| 1236 | − unsigned long end_pfn = min(start_pfn + nr_pages, zone_end_pfn(page_zone(page))); | |
| 1237 | − struct page *end_page = pfn_to_page(end_pfn); | |
| 1237 | + unsigned long end_pfn, pfn; | |
| 1238 | + | |
| 1239 | + end_pfn = min(start_pfn + nr_pages, | |
| 1240 | + zone_end_pfn(page_zone(pfn_to_page(start_pfn)))); | |
| 1238 | 1241 | |
| 1239 | 1242 | /* Check the starting page of each pageblock within the range */ |
| 1240 | − for (; page < end_page; page = next_active_pageblock(page)) { | |
| 1241 | − if (!is_pageblock_removable_nolock(page)) | |
| 1243 | + for (pfn = start_pfn; pfn < end_pfn; pfn = next_active_pageblock(pfn)) { | |
| 1244 | + if (!is_pageblock_removable_nolock(pfn)) | |
| 1242 | 1245 | return false; |
| 1243 | 1246 | cond_resched(); |
| 1244 | 1247 | } |
| @@ -1314,7 +1314,7 @@ static int copy_nodes_to_user(unsigned long __user *mask, unsigned long maxnode, · 1313 unmodified lines | ||
| 1314 | 1314 | nodemask_t *nodes) |
| 1315 | 1315 | { |
| 1316 | 1316 | unsigned long copy = ALIGN(maxnode-1, 64) / 8; |
| 1317 | − const int nbytes = BITS_TO_LONGS(MAX_NUMNODES) * sizeof(long); | |
| 1317 | + unsigned int nbytes = BITS_TO_LONGS(nr_node_ids) * sizeof(long); | |
| 1318 | 1318 | |
| 1319 | 1319 | if (copy > nbytes) { |
| 1320 | 1320 | if (copy > PAGE_SIZE) |
| @@ -1491,7 +1491,7 @@ static int kernel_get_mempolicy(int __user *policy, · 170 unmodified lines | ||
| 1491 | 1491 | int uninitialized_var(pval); |
| 1492 | 1492 | nodemask_t nodes; |
| 1493 | 1493 | |
| 1494 | − if (nmask != NULL && maxnode < MAX_NUMNODES) | |
| 1494 | + if (nmask != NULL && maxnode < nr_node_ids) | |
| 1495 | 1495 | return -EINVAL; |
| 1496 | 1496 | |
| 1497 | 1497 | err = do_get_mempolicy(&pval, &nodes, addr, flags); |
| @@ -1527,7 +1527,7 @@ COMPAT_SYSCALL_DEFINE5(get_mempolicy, int __user *, policy, · 29 unmodified lines | ||
| 1527 | 1527 | unsigned long nr_bits, alloc_size; |
| 1528 | 1528 | DECLARE_BITMAP(bm, MAX_NUMNODES); |
| 1529 | 1529 | |
| 1530 | − nr_bits = min_t(unsigned long, maxnode-1, MAX_NUMNODES); | |
| 1530 | + nr_bits = min_t(unsigned long, maxnode-1, nr_node_ids); | |
| 1531 | 1531 | alloc_size = ALIGN(nr_bits, BITS_PER_LONG) / 8; |
| 1532 | 1532 | |
| 1533 | 1533 | if (nmask) |
| @@ -1315,6 +1315,16 @@ static int unmap_and_move_huge_page(new_page_t get_new_page, · 1314 unmodified lines | ||
| 1315 | 1315 | lock_page(hpage); |
| 1316 | 1316 | } |
| 1317 | 1317 | |
| 1318 | + /* | |
| 1319 | + * Check for pages which are in the process of being freed. Without | |
| 1320 | + * page_mapping() set, hugetlbfs specific move page routine will not | |
| 1321 | + * be called and we could leak usage counts for subpools. | |
| 1322 | + */ | |
| 1323 | + if (page_private(hpage) && !page_mapping(hpage)) { | |
| 1324 | + rc = -EBUSY; | |
| 1325 | + goto out_unlock; | |
| 1326 | + } | |
| 1327 | + | |
| 1318 | 1328 | if (PageAnon(hpage)) |
| 1319 | 1329 | anon_vma = page_get_anon_vma(hpage); |
| 1320 | 1330 | |
| @@ -1345,6 +1355,7 @@ put_anon: · 24 unmodified lines | ||
| 1345 | 1355 | put_new_page = NULL; |
| 1346 | 1356 | } |
| 1347 | 1357 | |
| 1358 | +out_unlock: | |
| 1348 | 1359 | unlock_page(hpage); |
| 1349 | 1360 | out: |
| 1350 | 1361 | if (rc != -EAGAIN) |
| @@ -2426,12 +2426,11 @@ int expand_downwards(struct vm_area_struct *vma, · 2425 unmodified lines | ||
| 2426 | 2426 | { |
| 2427 | 2427 | struct mm_struct *mm = vma->vm_mm; |
| 2428 | 2428 | struct vm_area_struct *prev; |
| 2429 | − int error; | |
| 2429 | + int error = 0; | |
| 2430 | 2430 | |
| 2431 | 2431 | address &= PAGE_MASK; |
| 2432 | − error = security_mmap_addr(address); | |
| 2433 | − if (error) | |
| 2434 | − return error; | |
| 2432 | + if (address < mmap_min_addr) | |
| 2433 | + return -EPERM; | |
| 2435 | 2434 | |
| 2436 | 2435 | /* Enforce stack_guard_gap */ |
| 2437 | 2436 | prev = vma->vm_prev; |
| @@ -2170,6 +2170,18 @@ static inline void boost_watermark(struct zone *zone) · 2169 unmodified lines | ||
| 2170 | 2170 | |
| 2171 | 2171 | max_boost = mult_frac(zone->_watermark[WMARK_HIGH], |
| 2172 | 2172 | watermark_boost_factor, 10000); |
| 2173 | + | |
| 2174 | + /* | |
| 2175 | + * high watermark may be uninitialised if fragmentation occurs | |
| 2176 | + * very early in boot so do not boost. We do not fall | |
| 2177 | + * through and boost by pageblock_nr_pages as failing | |
| 2178 | + * allocations that early means that reclaim is not going | |
| 2179 | + * to help and it may even be impossible to reclaim the | |
| 2180 | + * boosted watermark resulting in a hang. | |
| 2181 | + */ | |
| 2182 | + if (!max_boost) | |
| 2183 | + return; | |
| 2184 | + | |
| 2173 | 2185 | max_boost = max(pageblock_nr_pages, max_boost); |
| 2174 | 2186 | |
| 2175 | 2187 | zone->watermark_boost = min(zone->watermark_boost + pageblock_nr_pages, |
| @@ -4675,11 +4687,11 @@ refill: · 2499 unmodified lines | ||
| 4675 | 4687 | /* Even if we own the page, we do not use atomic_set(). |
| 4676 | 4688 | * This would break get_page_unless_zero() users. |
| 4677 | 4689 | */ |
| 4678 | − page_ref_add(page, size); | |
| 4690 | + page_ref_add(page, PAGE_FRAG_CACHE_MAX_SIZE); | |
| 4679 | 4691 | |
| 4680 | 4692 | /* reset page count bias and offset to start of new frag */ |
| 4681 | 4693 | nc->pfmemalloc = page_is_pfmemalloc(page); |
| 4682 | − nc->pagecnt_bias = size + 1; | |
| 4694 | + nc->pagecnt_bias = PAGE_FRAG_CACHE_MAX_SIZE + 1; | |
| 4683 | 4695 | nc->offset = size; |
| 4684 | 4696 | } |
| 4685 | 4697 | |
| @@ -4695,10 +4707,10 @@ refill: · 9 unmodified lines | ||
| 4695 | 4707 | size = nc->size; |
| 4696 | 4708 | #endif |
| 4697 | 4709 | /* OK, page count is 0, we can safely set it */ |
| 4698 | − set_page_count(page, size + 1); | |
| 4710 | + set_page_count(page, PAGE_FRAG_CACHE_MAX_SIZE + 1); | |
| 4699 | 4711 | |
| 4700 | 4712 | /* reset page count bias and offset to start of new frag */ |
| 4701 | − nc->pagecnt_bias = size + 1; | |
| 4713 | + nc->pagecnt_bias = PAGE_FRAG_CACHE_MAX_SIZE + 1; | |
| 4702 | 4714 | offset = size - fragsz; |
| 4703 | 4715 | } |
| 4704 | 4716 | |
| @@ -2848,16 +2848,20 @@ static int shmem_create(struct inode *dir, struct dentry *dentry, umode_t mode, · 2847 unmodified lines | ||
| 2848 | 2848 | static int shmem_link(struct dentry *old_dentry, struct inode *dir, struct dentry *dentry) |
| 2849 | 2849 | { |
| 2850 | 2850 | struct inode *inode = d_inode(old_dentry); |
| 2851 | − int ret; | |
| 2851 | + int ret = 0; | |
| 2852 | 2852 | |
| 2853 | 2853 | /* |
| 2854 | 2854 | * No ordinary (disk based) filesystem counts links as inodes; |
| 2855 | 2855 | * but each new link needs a new dentry, pinning lowmem, and |
| 2856 | 2856 | * tmpfs dentries cannot be pruned until they are unlinked. |
| 2857 | + * But if an O_TMPFILE file is linked into the tmpfs, the | |
| 2858 | + * first link must skip that, to get the accounting right. | |
| 2857 | 2859 | */ |
| 2858 | − ret = shmem_reserve_inode(inode->i_sb); | |
| 2859 | − if (ret) | |
| 2860 | − goto out; | |
| 2860 | + if (inode->i_nlink) { | |
| 2861 | + ret = shmem_reserve_inode(inode->i_sb); | |
| 2862 | + if (ret) | |
| 2863 | + goto out; | |
| 2864 | + } | |
| 2861 | 2865 | |
| 2862 | 2866 | dir->i_size += BOGO_DIRENT_SIZE; |
| 2863 | 2867 | inode->i_ctime = dir->i_ctime = dir->i_mtime = current_time(inode); |
| @@ -2359,7 +2359,7 @@ static void *alloc_slabmgmt(struct kmem_cache *cachep, · 2358 unmodified lines | ||
| 2359 | 2359 | void *freelist; |
| 2360 | 2360 | void *addr = page_address(page); |
| 2361 | 2361 | |
| 2362 | − page->s_mem = kasan_reset_tag(addr) + colour_off; | |
| 2362 | + page->s_mem = addr + colour_off; | |
| 2363 | 2363 | page->active = 0; |
| 2364 | 2364 | |
| 2365 | 2365 | if (OBJFREELIST_SLAB(cachep)) |
| @@ -2368,6 +2368,7 @@ static void *alloc_slabmgmt(struct kmem_cache *cachep, · 2 unmodified lines | ||
| 2368 | 2368 | /* Slab management obj is off-slab. */ |
| 2369 | 2369 | freelist = kmem_cache_alloc_node(cachep->freelist_cache, |
| 2370 | 2370 | local_flags, nodeid); |
| 2371 | + freelist = kasan_reset_tag(freelist); | |
| 2371 | 2372 | if (!freelist) |
| 2372 | 2373 | return NULL; |
| 2373 | 2374 | } else { |
| @@ -2681,6 +2682,13 @@ static struct page *cache_grow_begin(struct kmem_cache *cachep, · 307 unmodified lines | ||
| 2681 | 2682 | |
| 2682 | 2683 | offset *= cachep->colour_off; |
| 2683 | 2684 | |
| 2685 | + /* | |
| 2686 | + * Call kasan_poison_slab() before calling alloc_slabmgmt(), so | |
| 2687 | + * page_address() in the latter returns a non-tagged pointer, | |
| 2688 | + * as it should be for slab pages. | |
| 2689 | + */ | |
| 2690 | + kasan_poison_slab(page); | |
| 2691 | + | |
| 2684 | 2692 | /* Get slab management. */ |
| 2685 | 2693 | freelist = alloc_slabmgmt(cachep, page, offset, |
| 2686 | 2694 | local_flags & ~GFP_CONSTRAINT_MASK, page_node); |
| @@ -2689,7 +2697,6 @@ static struct page *cache_grow_begin(struct kmem_cache *cachep, · 2 unmodified lines | ||
| 2689 | 2697 | |
| 2690 | 2698 | slab_map_pages(cachep, page, freelist); |
| 2691 | 2699 | |
| 2692 | − kasan_poison_slab(page); | |
| 2693 | 2700 | cache_init_objs(cachep, page); |
| 2694 | 2701 | |
| 2695 | 2702 | if (gfpflags_allow_blocking(local_flags)) |
| @@ -3540,7 +3547,6 @@ void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags) · 844 unmodified lines | ||
| 3540 | 3547 | { |
| 3541 | 3548 | void *ret = slab_alloc(cachep, flags, _RET_IP_); |
| 3542 | 3549 | |
| 3543 | − ret = kasan_slab_alloc(cachep, ret, flags); | |
| 3544 | 3550 | trace_kmem_cache_alloc(_RET_IP_, ret, |
| 3545 | 3551 | cachep->object_size, cachep->size, flags); |
| 3546 | 3552 | |
| @@ -3630,7 +3636,6 @@ void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid) · 83 unmodified lines | ||
| 3630 | 3636 | { |
| 3631 | 3637 | void *ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); |
| 3632 | 3638 | |
| 3633 | − ret = kasan_slab_alloc(cachep, ret, flags); | |
| 3634 | 3639 | trace_kmem_cache_alloc_node(_RET_IP_, ret, |
| 3635 | 3640 | cachep->object_size, cachep->size, |
| 3636 | 3641 | flags, nodeid); |
| @@ -4408,6 +4413,8 @@ void __check_heap_object(const void *ptr, unsigned long n, struct page *page, · 771 unmodified lines | ||
| 4408 | 4413 | unsigned int objnr; |
| 4409 | 4414 | unsigned long offset; |
| 4410 | 4415 | |
| 4416 | + ptr = kasan_reset_tag(ptr); | |
| 4417 | + | |
| 4411 | 4418 | /* Find and validate object. */ |
| 4412 | 4419 | cachep = page->slab_cache; |
| 4413 | 4420 | objnr = obj_to_index(cachep, page, (void *)ptr); |
| @@ -437,11 +437,10 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags, · 436 unmodified lines | ||
| 437 | 437 | |
| 438 | 438 | flags &= gfp_allowed_mask; |
| 439 | 439 | for (i = 0; i < size; i++) { |
| 440 | − void *object = p[i]; | |
| 441 | − | |
| 442 | − kmemleak_alloc_recursive(object, s->object_size, 1, | |
| 440 | + p[i] = kasan_slab_alloc(s, p[i], flags); | |
| 441 | + /* As p[i] might get tagged, call kmemleak hook after KASAN. */ | |
| 442 | + kmemleak_alloc_recursive(p[i], s->object_size, 1, | |
| 443 | 443 | s->flags, flags); |
| 444 | − p[i] = kasan_slab_alloc(s, object, flags); | |
| 445 | 444 | } |
| 446 | 445 | |
| 447 | 446 | if (memcg_kmem_enabled()) |
| @@ -1228,8 +1228,9 @@ void *kmalloc_order(size_t size, gfp_t flags, unsigned int order) · 1227 unmodified lines | ||
| 1228 | 1228 | flags |= __GFP_COMP; |
| 1229 | 1229 | page = alloc_pages(flags, order); |
| 1230 | 1230 | ret = page ? page_address(page) : NULL; |
| 1231 | − kmemleak_alloc(ret, size, 1, flags); | |
| 1232 | 1231 | ret = kasan_kmalloc_large(ret, size, flags); |
| 1232 | + /* As ret might get tagged, call kmemleak hook after KASAN. */ | |
| 1233 | + kmemleak_alloc(ret, size, 1, flags); | |
| 1233 | 1234 | return ret; |
| 1234 | 1235 | } |
| 1235 | 1236 | EXPORT_SYMBOL(kmalloc_order); |
| @@ -249,7 +249,18 @@ static inline void *freelist_ptr(const struct kmem_cache *s, void *ptr, · 248 unmodified lines | ||
| 249 | 249 | unsigned long ptr_addr) |
| 250 | 250 | { |
| 251 | 251 | #ifdef CONFIG_SLAB_FREELIST_HARDENED |
| 252 | − return (void *)((unsigned long)ptr ^ s->random ^ ptr_addr); | |
| 252 | + /* | |
| 253 | + * When CONFIG_KASAN_SW_TAGS is enabled, ptr_addr might be tagged. | |
| 254 | + * Normally, this doesn't cause any issues, as both set_freepointer() | |
| 255 | + * and get_freepointer() are called with a pointer with the same tag. | |
| 256 | + * However, there are some issues with CONFIG_SLUB_DEBUG code. For | |
| 257 | + * example, when __free_slub() iterates over objects in a cache, it | |
| 258 | + * passes untagged pointers to check_object(). check_object() in turns | |
| 259 | + * calls get_freepointer() with an untagged pointer, which causes the | |
| 260 | + * freepointer to be restored incorrectly. | |
| 261 | + */ | |
| 262 | + return (void *)((unsigned long)ptr ^ s->random ^ | |
| 263 | + (unsigned long)kasan_reset_tag((void *)ptr_addr)); | |
| 253 | 264 | #else |
| 254 | 265 | return ptr; |
| 255 | 266 | #endif |
| @@ -303,15 +314,10 @@ static inline void set_freepointer(struct kmem_cache *s, void *object, void *fp) · 47 unmodified lines | ||
| 303 | 314 | __p < (__addr) + (__objects) * (__s)->size; \ |
| 304 | 315 | __p += (__s)->size) |
| 305 | 316 | |
| 306 | −#define for_each_object_idx(__p, __idx, __s, __addr, __objects) \ | |
| 307 | − for (__p = fixup_red_left(__s, __addr), __idx = 1; \ | |
| 308 | − __idx <= __objects; \ | |
| 309 | − __p += (__s)->size, __idx++) | |
| 310 | − | |
| 311 | 317 | /* Determine object index from a given position */ |
| 312 | 318 | static inline unsigned int slab_index(void *p, struct kmem_cache *s, void *addr) |
| 313 | 319 | { |
| 314 | − return (p - addr) / s->size; | |
| 320 | + return (kasan_reset_tag(p) - addr) / s->size; | |
| 315 | 321 | } |
| 316 | 322 | |
| 317 | 323 | static inline unsigned int order_objects(unsigned int order, unsigned int size) |
| @@ -507,6 +513,7 @@ static inline int check_valid_pointer(struct kmem_cache *s, · 189 unmodified lines | ||
| 507 | 513 | return 1; |
| 508 | 514 | |
| 509 | 515 | base = page_address(page); |
| 516 | + object = kasan_reset_tag(object); | |
| 510 | 517 | object = restore_red_left(s, object); |
| 511 | 518 | if (object < base || object >= base + page->objects * s->size || |
| 512 | 519 | (object - base) % s->size) { |
| @@ -1075,6 +1082,16 @@ static void setup_object_debug(struct kmem_cache *s, struct page *page, · 562 unmodified lines | ||
| 1075 | 1082 | init_tracking(s, object); |
| 1076 | 1083 | } |
| 1077 | 1084 | |
| 1085 | +static void setup_page_debug(struct kmem_cache *s, void *addr, int order) | |
| 1086 | +{ | |
| 1087 | + if (!(s->flags & SLAB_POISON)) | |
| 1088 | + return; | |
| 1089 | + | |
| 1090 | + metadata_access_enable(); | |
| 1091 | + memset(addr, POISON_INUSE, PAGE_SIZE << order); | |
| 1092 | + metadata_access_disable(); | |
| 1093 | +} | |
| 1094 | + | |
| 1078 | 1095 | static inline int alloc_consistency_checks(struct kmem_cache *s, |
| 1079 | 1096 | struct page *page, |
| 1080 | 1097 | void *object, unsigned long addr) |
| @@ -1330,6 +1347,8 @@ slab_flags_t kmem_cache_flags(unsigned int object_size, · 249 unmodified lines | ||
| 1330 | 1347 | #else /* !CONFIG_SLUB_DEBUG */ |
| 1331 | 1348 | static inline void setup_object_debug(struct kmem_cache *s, |
| 1332 | 1349 | struct page *page, void *object) {} |
| 1350 | +static inline void setup_page_debug(struct kmem_cache *s, | |
| 1351 | + void *addr, int order) {} | |
| 1333 | 1352 | |
| 1334 | 1353 | static inline int alloc_debug_processing(struct kmem_cache *s, |
| 1335 | 1354 | struct page *page, void *object, unsigned long addr) { return 0; } |
| @@ -1374,8 +1393,10 @@ static inline void dec_slabs_node(struct kmem_cache *s, int node, · 38 unmodified lines | ||
| 1374 | 1393 | */ |
| 1375 | 1394 | static inline void *kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) |
| 1376 | 1395 | { |
| 1396 | + ptr = kasan_kmalloc_large(ptr, size, flags); | |
| 1397 | + /* As ptr might get tagged, call kmemleak hook after KASAN. */ | |
| 1377 | 1398 | kmemleak_alloc(ptr, size, 1, flags); |
| 1378 | − return kasan_kmalloc_large(ptr, size, flags); | |
| 1399 | + return ptr; | |
| 1379 | 1400 | } |
| 1380 | 1401 | |
| 1381 | 1402 | static __always_inline void kfree_hook(void *x) |
| @@ -1641,27 +1662,25 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) · 259 unmodified lines | ||
| 1641 | 1662 | if (page_is_pfmemalloc(page)) |
| 1642 | 1663 | SetPageSlabPfmemalloc(page); |
| 1643 | 1664 | |
| 1665 | + kasan_poison_slab(page); | |
| 1666 | + | |
| 1644 | 1667 | start = page_address(page); |
| 1645 | 1668 | |
| 1646 | − if (unlikely(s->flags & SLAB_POISON)) | |
| 1647 | − memset(start, POISON_INUSE, PAGE_SIZE << order); | |
| 1648 | − | |
| 1649 | − kasan_poison_slab(page); | |
| 1669 | + setup_page_debug(s, start, order); | |
| 1650 | 1670 | |
| 1651 | 1671 | shuffle = shuffle_freelist(s, page); |
| 1652 | 1672 | |
| 1653 | 1673 | if (!shuffle) { |
| 1654 | − for_each_object_idx(p, idx, s, start, page->objects) { | |
| 1655 | − if (likely(idx < page->objects)) { | |
| 1656 | − next = p + s->size; | |
| 1657 | − next = setup_object(s, page, next); | |
| 1658 | − set_freepointer(s, p, next); | |
| 1659 | − } else | |
| 1660 | − set_freepointer(s, p, NULL); | |
| 1661 | − } | |
| 1662 | 1674 | start = fixup_red_left(s, start); |
| 1663 | 1675 | start = setup_object(s, page, start); |
| 1664 | 1676 | page->freelist = start; |
| 1677 | + for (idx = 0, p = start; idx < page->objects - 1; idx++) { | |
| 1678 | + next = p + s->size; | |
| 1679 | + next = setup_object(s, page, next); | |
| 1680 | + set_freepointer(s, p, next); | |
| 1681 | + p = next; | |
| 1682 | + } | |
| 1683 | + set_freepointer(s, p, NULL); | |
| 1665 | 1684 | } |
| 1666 | 1685 | |
| 1667 | 1686 | page->inuse = page->objects; |
| @@ -320,11 +320,6 @@ static inline void activate_page_drain(int cpu) · 319 unmodified lines | ||
| 320 | 320 | { |
| 321 | 321 | } |
| 322 | 322 | |
| 323 | −static bool need_activate_page_drain(int cpu) | |
| 324 | −{ | |
| 325 | − return false; | |
| 326 | −} | |
| 327 | − | |
| 328 | 323 | void activate_page(struct page *page) |
| 329 | 324 | { |
| 330 | 325 | struct zone *zone = page_zone(page); |
| @@ -653,13 +648,15 @@ void lru_add_drain(void) · 322 unmodified lines | ||
| 653 | 648 | put_cpu(); |
| 654 | 649 | } |
| 655 | 650 | |
| 651 | +#ifdef CONFIG_SMP | |
| 652 | + | |
| 653 | +static DEFINE_PER_CPU(struct work_struct, lru_add_drain_work); | |
| 654 | + | |
| 656 | 655 | static void lru_add_drain_per_cpu(struct work_struct *dummy) |
| 657 | 656 | { |
| 658 | 657 | lru_add_drain(); |
| 659 | 658 | } |
| 660 | 659 | |
| 661 | −static DEFINE_PER_CPU(struct work_struct, lru_add_drain_work); | |
| 662 | − | |
| 663 | 660 | /* |
| 664 | 661 | * Doesn't need any cpu hotplug locking because we do rely on per-cpu |
| 665 | 662 | * kworkers being shut down before our page_alloc_cpu_dead callback is |
| @@ -702,6 +699,12 @@ void lru_add_drain_all(void) · 36 unmodified lines | ||
| 702 | 699 | |
| 703 | 700 | mutex_unlock(&lock); |
| 704 | 701 | } |
| 702 | +#else | |
| 703 | +void lru_add_drain_all(void) | |
| 704 | +{ | |
| 705 | + lru_add_drain(); | |
| 706 | +} | |
| 707 | +#endif | |
| 705 | 708 | |
| 706 | 709 | /** |
| 707 | 710 | * release_pages - batched put_page() |
| @@ -150,7 +150,7 @@ void *memdup_user(const void __user *src, size_t len) · 149 unmodified lines | ||
| 150 | 150 | { |
| 151 | 151 | void *p; |
| 152 | 152 | |
| 153 | − p = kmalloc_track_caller(len, GFP_USER); | |
| 153 | + p = kmalloc_track_caller(len, GFP_USER | __GFP_NOWARN); | |
| 154 | 154 | if (!p) |
| 155 | 155 | return ERR_PTR(-ENOMEM); |
| 156 | 156 | |
| @@ -13,27 +13,13 @@ · 12 unmodified lines | ||
| 13 | 13 | #include <net/sock.h> |
| 14 | 14 | #include <net/tcp.h> |
| 15 | 15 | |
| 16 | −static __always_inline u32 bpf_test_run_one(struct bpf_prog *prog, void *ctx, | |
| 17 | − struct bpf_cgroup_storage *storage[MAX_BPF_CGROUP_STORAGE_TYPE]) | |
| 18 | −{ | |
| 19 | − u32 ret; | |
| 20 | − | |
| 21 | − preempt_disable(); | |
| 22 | − rcu_read_lock(); | |
| 23 | − bpf_cgroup_storage_set(storage); | |
| 24 | − ret = BPF_PROG_RUN(prog, ctx); | |
| 25 | − rcu_read_unlock(); | |
| 26 | − preempt_enable(); | |
| 27 | − | |
| 28 | − return ret; | |
| 29 | −} | |
| 30 | − | |
| 31 | −static int bpf_test_run(struct bpf_prog *prog, void *ctx, u32 repeat, u32 *ret, | |
| 32 | − u32 *time) | |
| 16 | +static int bpf_test_run(struct bpf_prog *prog, void *ctx, u32 repeat, | |
| 17 | + u32 *retval, u32 *time) | |
| 33 | 18 | { |
| 34 | 19 | struct bpf_cgroup_storage *storage[MAX_BPF_CGROUP_STORAGE_TYPE] = { 0 }; |
| 35 | 20 | enum bpf_cgroup_storage_type stype; |
| 36 | 21 | u64 time_start, time_spent = 0; |
| 22 | + int ret = 0; | |
| 37 | 23 | u32 i; |
| 38 | 24 | |
| 39 | 25 | for_each_cgroup_storage_type(stype) { |
| @@ -48,25 +34,42 @@ static int bpf_test_run(struct bpf_prog *prog, void *ctx, u32 repeat, u32 *ret, · 8 unmodified lines | ||
| 48 | 34 | |
| 49 | 35 | if (!repeat) |
| 50 | 36 | repeat = 1; |
| 37 | + | |
| 38 | + rcu_read_lock(); | |
| 39 | + preempt_disable(); | |
| 51 | 40 | time_start = ktime_get_ns(); |
| 52 | 41 | for (i = 0; i < repeat; i++) { |
| 53 | − *ret = bpf_test_run_one(prog, ctx, storage); | |
| 42 | + bpf_cgroup_storage_set(storage); | |
| 43 | + *retval = BPF_PROG_RUN(prog, ctx); | |
| 44 | + | |
| 45 | + if (signal_pending(current)) { | |
| 46 | + ret = -EINTR; | |
| 47 | + break; | |
| 48 | + } | |
| 49 | + | |
| 54 | 50 | if (need_resched()) { |
| 55 | − if (signal_pending(current)) | |
| 56 | − break; | |
| 57 | 51 | time_spent += ktime_get_ns() - time_start; |
| 52 | + preempt_enable(); | |
| 53 | + rcu_read_unlock(); | |
| 54 | + | |
| 58 | 55 | cond_resched(); |
| 56 | + | |
| 57 | + rcu_read_lock(); | |
| 58 | + preempt_disable(); | |
| 59 | 59 | time_start = ktime_get_ns(); |
| 60 | 60 | } |
| 61 | 61 | } |
| 62 | 62 | time_spent += ktime_get_ns() - time_start; |
| 63 | + preempt_enable(); | |
| 64 | + rcu_read_unlock(); | |
| 65 | + | |
| 63 | 66 | do_div(time_spent, repeat); |
| 64 | 67 | *time = time_spent > U32_MAX ? U32_MAX : (u32)time_spent; |
| 65 | 68 | |
| 66 | 69 | for_each_cgroup_storage_type(stype) |
| 67 | 70 | bpf_cgroup_storage_free(storage[stype]); |
| 68 | 71 | |
| 69 | − return 0; | |
| 72 | + return ret; | |
| 70 | 73 | } |
| 71 | 74 | |
| 72 | 75 | static int bpf_test_finish(const union bpf_attr *kattr, |
| @@ -1204,14 +1204,7 @@ static void br_multicast_query_received(struct net_bridge *br, · 1203 unmodified lines | ||
| 1204 | 1204 | return; |
| 1205 | 1205 | |
| 1206 | 1206 | br_multicast_update_query_timer(br, query, max_delay); |
| 1207 | − | |
| 1208 | − /* Based on RFC4541, section 2.1.1 IGMP Forwarding Rules, | |
| 1209 | − * the arrival port for IGMP Queries where the source address | |
| 1210 | − * is 0.0.0.0 should not be added to router port list. | |
| 1211 | − */ | |
| 1212 | − if ((saddr->proto == htons(ETH_P_IP) && saddr->u.ip4) || | |
| 1213 | − saddr->proto == htons(ETH_P_IPV6)) | |
| 1214 | − br_multicast_mark_router(br, port); | |
| 1207 | + br_multicast_mark_router(br, port); | |
| 1215 | 1208 | } |
| 1216 | 1209 | |
| 1217 | 1210 | static void br_ip4_multicast_query(struct net_bridge *br, |
| @@ -2058,6 +2058,8 @@ static int process_connect(struct ceph_connection *con) · 2057 unmodified lines | ||
| 2058 | 2058 | dout("process_connect on %p tag %d\n", con, (int)con->in_tag); |
| 2059 | 2059 | |
| 2060 | 2060 | if (con->auth) { |
| 2061 | + int len = le32_to_cpu(con->in_reply.authorizer_len); | |
| 2062 | + | |
| 2061 | 2063 | /* |
| 2062 | 2064 | * Any connection that defines ->get_authorizer() |
| 2063 | 2065 | * should also define ->add_authorizer_challenge() and |
| @@ -2067,8 +2069,7 @@ static int process_connect(struct ceph_connection *con) · 3 unmodified lines | ||
| 2067 | 2069 | */ |
| 2068 | 2070 | if (con->in_reply.tag == CEPH_MSGR_TAG_CHALLENGE_AUTHORIZER) { |
| 2069 | 2071 | ret = con->ops->add_authorizer_challenge( |
| 2070 | − con, con->auth->authorizer_reply_buf, | |
| 2071 | − le32_to_cpu(con->in_reply.authorizer_len)); | |
| 2072 | + con, con->auth->authorizer_reply_buf, len); | |
| 2072 | 2073 | if (ret < 0) |
| 2073 | 2074 | return ret; |
| 2074 | 2075 | |
| @@ -2078,10 +2079,12 @@ static int process_connect(struct ceph_connection *con) · 3 unmodified lines | ||
| 2078 | 2079 | return 0; |
| 2079 | 2080 | } |
| 2080 | 2081 | |
| 2081 | − ret = con->ops->verify_authorizer_reply(con); | |
| 2082 | − if (ret < 0) { | |
| 2083 | − con->error_msg = "bad authorize reply"; | |
| 2084 | − return ret; | |
| 2082 | + if (len) { | |
| 2083 | + ret = con->ops->verify_authorizer_reply(con); | |
| 2084 | + if (ret < 0) { | |
| 2085 | + con->error_msg = "bad authorize reply"; | |
| 2086 | + return ret; | |
| 2087 | + } | |
| 2085 | 2088 | } |
| 2086 | 2089 | } |
| 2087 | 2090 | |
| @@ -388,8 +388,12 @@ static int __compat_sys_setsockopt(int fd, int level, int optname, · 387 unmodified lines | ||
| 388 | 388 | char __user *optval, unsigned int optlen) |
| 389 | 389 | { |
| 390 | 390 | int err; |
| 391 | − struct socket *sock = sockfd_lookup(fd, &err); | |
| 391 | + struct socket *sock; | |
| 392 | + | |
| 393 | + if (optlen > INT_MAX) | |
| 394 | + return -EINVAL; | |
| 392 | 395 | |
| 396 | + sock = sockfd_lookup(fd, &err); | |
| 393 | 397 | if (sock) { |
| 394 | 398 | err = security_socket_setsockopt(sock, level, optname); |
| 395 | 399 | if (err) { |
| @@ -8152,7 +8152,7 @@ static netdev_features_t netdev_sync_upper_features(struct net_device *lower, · 8151 unmodified lines | ||
| 8152 | 8152 | netdev_features_t feature; |
| 8153 | 8153 | int feature_bit; |
| 8154 | 8154 | |
| 8155 | − for_each_netdev_feature(&upper_disables, feature_bit) { | |
| 8155 | + for_each_netdev_feature(upper_disables, feature_bit) { | |
| 8156 | 8156 | feature = __NETIF_F_BIT(feature_bit); |
| 8157 | 8157 | if (!(upper->wanted_features & feature) |
| 8158 | 8158 | && (features & feature)) { |
| @@ -8172,7 +8172,7 @@ static void netdev_sync_lower_features(struct net_device *upper, · 13 unmodified lines | ||
| 8172 | 8172 | netdev_features_t feature; |
| 8173 | 8173 | int feature_bit; |
| 8174 | 8174 | |
| 8175 | − for_each_netdev_feature(&upper_disables, feature_bit) { | |
| 8175 | + for_each_netdev_feature(upper_disables, feature_bit) { | |
| 8176 | 8176 | feature = __NETIF_F_BIT(feature_bit); |
| 8177 | 8177 | if (!(features & feature) && (lower->features & feature)) { |
| 8178 | 8178 | netdev_dbg(upper, "Disabling feature %pNF on lower dev %s.\n", |
| @@ -2789,8 +2789,7 @@ static int bpf_skb_proto_4_to_6(struct sk_buff *skb) · 2788 unmodified lines | ||
| 2789 | 2789 | u32 off = skb_mac_header_len(skb); |
| 2790 | 2790 | int ret; |
| 2791 | 2791 | |
| 2792 | − /* SCTP uses GSO_BY_FRAGS, thus cannot adjust it. */ | |
| 2793 | − if (skb_is_gso(skb) && unlikely(skb_is_gso_sctp(skb))) | |
| 2792 | + if (!skb_is_gso_tcp(skb)) | |
| 2794 | 2793 | return -ENOTSUPP; |
| 2795 | 2794 | |
| 2796 | 2795 | ret = skb_cow(skb, len_diff); |
| @@ -2831,8 +2830,7 @@ static int bpf_skb_proto_6_to_4(struct sk_buff *skb) · 34 unmodified lines | ||
| 2831 | 2830 | u32 off = skb_mac_header_len(skb); |
| 2832 | 2831 | int ret; |
| 2833 | 2832 | |
| 2834 | − /* SCTP uses GSO_BY_FRAGS, thus cannot adjust it. */ | |
| 2835 | − if (skb_is_gso(skb) && unlikely(skb_is_gso_sctp(skb))) | |
| 2833 | + if (!skb_is_gso_tcp(skb)) | |
| 2836 | 2834 | return -ENOTSUPP; |
| 2837 | 2835 | |
| 2838 | 2836 | ret = skb_unclone(skb, GFP_ATOMIC); |
| @@ -2957,8 +2955,7 @@ static int bpf_skb_net_grow(struct sk_buff *skb, u32 len_diff) · 118 unmodified lines | ||
| 2957 | 2955 | u32 off = skb_mac_header_len(skb) + bpf_skb_net_base_len(skb); |
| 2958 | 2956 | int ret; |
| 2959 | 2957 | |
| 2960 | − /* SCTP uses GSO_BY_FRAGS, thus cannot adjust it. */ | |
| 2961 | − if (skb_is_gso(skb) && unlikely(skb_is_gso_sctp(skb))) | |
| 2958 | + if (!skb_is_gso_tcp(skb)) | |
| 2962 | 2959 | return -ENOTSUPP; |
| 2963 | 2960 | |
| 2964 | 2961 | ret = skb_cow(skb, len_diff); |
| @@ -2987,8 +2984,7 @@ static int bpf_skb_net_shrink(struct sk_buff *skb, u32 len_diff) · 22 unmodified lines | ||
| 2987 | 2984 | u32 off = skb_mac_header_len(skb) + bpf_skb_net_base_len(skb); |
| 2988 | 2985 | int ret; |
| 2989 | 2986 | |
| 2990 | − /* SCTP uses GSO_BY_FRAGS, thus cannot adjust it. */ | |
| 2991 | − if (skb_is_gso(skb) && unlikely(skb_is_gso_sctp(skb))) | |
| 2987 | + if (!skb_is_gso_tcp(skb)) | |
| 2992 | 2988 | return -ENOTSUPP; |
| 2993 | 2989 | |
| 2994 | 2990 | ret = skb_unclone(skb, GFP_ATOMIC); |
| @@ -356,6 +356,8 @@ static void *__netdev_alloc_frag(unsigned int fragsz, gfp_t gfp_mask) · 355 unmodified lines | ||
| 356 | 356 | */ |
| 357 | 357 | void *netdev_alloc_frag(unsigned int fragsz) |
| 358 | 358 | { |
| 359 | + fragsz = SKB_DATA_ALIGN(fragsz); | |
| 360 | + | |
| 359 | 361 | return __netdev_alloc_frag(fragsz, GFP_ATOMIC); |
| 360 | 362 | } |
| 361 | 363 | EXPORT_SYMBOL(netdev_alloc_frag); |
| @@ -369,6 +371,8 @@ static void *__napi_alloc_frag(unsigned int fragsz, gfp_t gfp_mask) · 7 unmodified lines | ||
| 369 | 371 | |
| 370 | 372 | void *napi_alloc_frag(unsigned int fragsz) |
| 371 | 373 | { |
| 374 | + fragsz = SKB_DATA_ALIGN(fragsz); | |
| 375 | + | |
| 372 | 376 | return __napi_alloc_frag(fragsz, GFP_ATOMIC); |
| 373 | 377 | } |
| 374 | 378 | EXPORT_SYMBOL(napi_alloc_frag); |
| @@ -612,8 +612,8 @@ static int dsa_switch_parse_ports_of(struct dsa_switch *ds, · 611 unmodified lines | ||
| 612 | 612 | { |
| 613 | 613 | struct device_node *ports, *port; |
| 614 | 614 | struct dsa_port *dp; |
| 615 | + int err = 0; | |
| 615 | 616 | u32 reg; |
| 616 | − int err; | |
| 617 | 617 | |
| 618 | 618 | ports = of_get_child_by_name(dn, "ports"); |
| 619 | 619 | if (!ports) { |
| @@ -624,19 +624,23 @@ static int dsa_switch_parse_ports_of(struct dsa_switch *ds, · 4 unmodified lines | ||
| 624 | 624 | for_each_available_child_of_node(ports, port) { |
| 625 | 625 | err = of_property_read_u32(port, "reg", ®); |
| 626 | 626 | if (err) |
| 627 | − return err; | |
| 627 | + goto out_put_node; | |
| 628 | 628 | |
| 629 | − if (reg >= ds->num_ports) | |
| 630 | − return -EINVAL; | |
| 629 | + if (reg >= ds->num_ports) { | |
| 630 | + err = -EINVAL; | |
| 631 | + goto out_put_node; | |
| 632 | + } | |
| 631 | 633 | |
| 632 | 634 | dp = &ds->ports[reg]; |
| 633 | 635 | |
| 634 | 636 | err = dsa_port_parse_of(dp, port); |
| 635 | 637 | if (err) |
| 636 | − return err; | |
| 638 | + goto out_put_node; | |
| 637 | 639 | } |
| 638 | 640 | |
| 639 | − return 0; | |
| 641 | +out_put_node: | |
| 642 | + of_node_put(ports); | |
| 643 | + return err; | |
| 640 | 644 | } |
| 641 | 645 | |
| 642 | 646 | static int dsa_switch_parse_member_of(struct dsa_switch *ds, |
| @@ -69,7 +69,6 @@ static void dsa_port_set_state_now(struct dsa_port *dp, u8 state) · 68 unmodified lines | ||
| 69 | 69 | |
| 70 | 70 | int dsa_port_enable(struct dsa_port *dp, struct phy_device *phy) |
| 71 | 71 | { |
| 72 | − u8 stp_state = dp->bridge_dev ? BR_STATE_BLOCKING : BR_STATE_FORWARDING; | |
| 73 | 72 | struct dsa_switch *ds = dp->ds; |
| 74 | 73 | int port = dp->index; |
| 75 | 74 | int err; |
| @@ -80,7 +79,8 @@ int dsa_port_enable(struct dsa_port *dp, struct phy_device *phy) · 4 unmodified lines | ||
| 80 | 79 | return err; |
| 81 | 80 | } |
| 82 | 81 | |
| 83 | − dsa_port_set_state_now(dp, stp_state); | |
| 82 | + if (!dp->bridge_dev) | |
| 83 | + dsa_port_set_state_now(dp, BR_STATE_FORWARDING); | |
| 84 | 84 | |
| 85 | 85 | return 0; |
| 86 | 86 | } |
| @@ -90,7 +90,8 @@ void dsa_port_disable(struct dsa_port *dp, struct phy_device *phy) · 3 unmodified lines | ||
| 90 | 90 | struct dsa_switch *ds = dp->ds; |
| 91 | 91 | int port = dp->index; |
| 92 | 92 | |
| 93 | − dsa_port_set_state_now(dp, BR_STATE_DISABLED); | |
| 93 | + if (!dp->bridge_dev) | |
| 94 | + dsa_port_set_state_now(dp, BR_STATE_DISABLED); | |
| 94 | 95 | |
| 95 | 96 | if (ds->ops->port_disable) |
| 96 | 97 | ds->ops->port_disable(ds, port, phy); |
| @@ -291,6 +292,7 @@ static struct phy_device *dsa_port_get_phy_device(struct dsa_port *dp) · 194 unmodified lines | ||
| 291 | 292 | return ERR_PTR(-EPROBE_DEFER); |
| 292 | 293 | } |
| 293 | 294 | |
| 295 | + of_node_put(phy_dn); | |
| 294 | 296 | return phydev; |
| 295 | 297 | } |
| 296 | 298 | |
| @@ -667,7 +667,8 @@ static int cipso_v4_map_lvl_valid(const struct cipso_v4_doi *doi_def, u8 level) · 666 unmodified lines | ||
| 667 | 667 | case CIPSO_V4_MAP_PASS: |
| 668 | 668 | return 0; |
| 669 | 669 | case CIPSO_V4_MAP_TRANS: |
| 670 | − if (doi_def->map.std->lvl.cipso[level] < CIPSO_V4_INV_LVL) | |
| 670 | + if ((level < doi_def->map.std->lvl.cipso_size) && | |
| 671 | + (doi_def->map.std->lvl.cipso[level] < CIPSO_V4_INV_LVL)) | |
| 671 | 672 | return 0; |
| 672 | 673 | break; |
| 673 | 674 | } |
| @@ -1735,13 +1736,26 @@ validate_return: · 1061 unmodified lines | ||
| 1735 | 1736 | */ |
| 1736 | 1737 | void cipso_v4_error(struct sk_buff *skb, int error, u32 gateway) |
| 1737 | 1738 | { |
| 1739 | + unsigned char optbuf[sizeof(struct ip_options) + 40]; | |
| 1740 | + struct ip_options *opt = (struct ip_options *)optbuf; | |
| 1741 | + | |
| 1738 | 1742 | if (ip_hdr(skb)->protocol == IPPROTO_ICMP || error != -EACCES) |
| 1739 | 1743 | return; |
| 1740 | 1744 | |
| 1745 | + /* | |
| 1746 | + * We might be called above the IP layer, | |
| 1747 | + * so we can not use icmp_send and IPCB here. | |
| 1748 | + */ | |
| 1749 | + | |
| 1750 | + memset(opt, 0, sizeof(struct ip_options)); | |
| 1751 | + opt->optlen = ip_hdr(skb)->ihl*4 - sizeof(struct iphdr); | |
12 more lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 122 lines aren’t shown here.
This diff is large: its 116 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 285 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 129 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 83 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 158 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 199 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 120 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.