Picking up v5.0 + missed misc-fixes from last release
Unified · Split
| @@ -842,10 +842,9 @@ D: ax25-utils maintainer. · 841 unmodified lines | |||
| 842 | 842 | ||
| 843 | N: Helge Deller | 843 | N: Helge Deller |
| 844 | E: deller@gmx.de | 844 | E: deller@gmx.de |
| 845 | E: hdeller@redhat.de | 845 | W: http://www.parisc-linux.org/ |
| 846 | D: PA-RISC Linux hacker, LASI-, ASP-, WAX-, LCD/LED-driver | 846 | D: PA-RISC Linux architecture maintainer |
| 847 | S: Schimmelsrain 1 | 847 | D: LASI-, ASP-, WAX-, LCD/LED-driver |
| 848 | S: D-69231 Rauenberg | ||
| 849 | S: Germany | 848 | S: Germany |
| 850 | 849 | ||
| 851 | N: Jean Delvare | 850 | N: Jean Delvare |
| @@ -1361,7 +1360,7 @@ S: Stellenbosch, Western Cape · 509 unmodified lines | |||
| 1361 | S: South Africa | 1360 | S: South Africa |
| 1362 | 1361 | ||
| 1363 | N: Grant Grundler | 1362 | N: Grant Grundler |
| 1364 | E: grundler@parisc-linux.org | 1363 | E: grantgrundler@gmail.com |
| 1365 | W: http://obmouse.sourceforge.net/ | 1364 | W: http://obmouse.sourceforge.net/ |
| 1366 | W: http://www.parisc-linux.org/ | 1365 | W: http://www.parisc-linux.org/ |
| 1367 | D: obmouse - rewrote Olivier Florent's Omnibook 600 "pop-up" mouse driver | 1366 | D: obmouse - rewrote Olivier Florent's Omnibook 600 "pop-up" mouse driver |
| @@ -2492,7 +2491,7 @@ S: Syracuse, New York 13206 · 1124 unmodified lines | |||
| 2492 | S: USA | 2491 | S: USA |
| 2493 | 2492 | ||
| 2494 | N: Kyle McMartin | 2493 | N: Kyle McMartin |
| 2495 | E: kyle@parisc-linux.org | 2494 | E: kyle@mcmartin.ca |
| 2496 | D: Linux/PARISC hacker | 2495 | D: Linux/PARISC hacker |
| 2497 | D: AD1889 sound driver | 2496 | D: AD1889 sound driver |
| 2498 | S: Ottawa, Canada | 2497 | S: Ottawa, Canada |
| @@ -3780,14 +3779,13 @@ S: 21513 Conradia Ct · 1281 unmodified lines | |||
| 3780 | S: Cupertino, CA 95014 | 3779 | S: Cupertino, CA 95014 |
| 3781 | S: USA | 3780 | S: USA |
| 3782 | 3781 | ||
| 3783 | N: Thibaut Varene | 3782 | N: Thibaut Varène |
| 3784 | E: T-Bone@parisc-linux.org | 3783 | E: hacks+kernel@slashdirt.org |
| 3785 | W: http://www.parisc-linux.org/~varenet/ | 3784 | W: http://hacks.slashdirt.org/ |
| 3786 | P: 1024D/B7D2F063 E67C 0D43 A75E 12A5 BB1C FA2F 1E32 C3DA B7D2 F063 | ||
| 3787 | D: PA-RISC port minion, PDC and GSCPS2 drivers, debuglocks and other bits | 3785 | D: PA-RISC port minion, PDC and GSCPS2 drivers, debuglocks and other bits |
| 3788 | D: Some ARM at91rm9200 bits, S1D13XXX FB driver, random patches here and there | 3786 | D: Some ARM at91rm9200 bits, S1D13XXX FB driver, random patches here and there |
| 3789 | D: AD1889 sound driver | 3787 | D: AD1889 sound driver |
| 3790 | S: Paris, France | 3788 | S: France |
| 3791 | 3789 | ||
| 3792 | N: Heikki Vatiainen | 3790 | N: Heikki Vatiainen |
| 3793 | E: hessu@cs.tut.fi | 3791 | E: hessu@cs.tut.fi |
| @@ -1,9 +1,9 @@ | |||
| 1 | .. _readme: | 1 | .. _readme: |
| 2 | 2 | ||
| 3 | Linux kernel release 4.x <http://kernel.org/> | 3 | Linux kernel release 5.x <http://kernel.org/> |
| 4 | ============================================= | 4 | ============================================= |
| 5 | 5 | ||
| 6 | These are the release notes for Linux version 4. Read them carefully, | 6 | These are the release notes for Linux version 5. Read them carefully, |
| 7 | as they tell you what this is all about, explain how to install the | 7 | as they tell you what this is all about, explain how to install the |
| 8 | kernel, and what to do if something goes wrong. | 8 | kernel, and what to do if something goes wrong. |
| 9 | 9 | ||
| @@ -63,7 +63,7 @@ Installing the kernel source · 53 unmodified lines | |||
| 63 | directory where you have permissions (e.g. your home directory) and | 63 | directory where you have permissions (e.g. your home directory) and |
| 64 | unpack it:: | 64 | unpack it:: |
| 65 | 65 | ||
| 66 | xz -cd linux-4.X.tar.xz | tar xvf - | 66 | xz -cd linux-5.x.tar.xz | tar xvf - |
| 67 | 67 | ||
| 68 | Replace "X" with the version number of the latest kernel. | 68 | Replace "X" with the version number of the latest kernel. |
| 69 | 69 | ||
| @@ -72,26 +72,26 @@ Installing the kernel source · 2 unmodified lines | |||
| 72 | files. They should match the library, and not get messed up by | 72 | files. They should match the library, and not get messed up by |
| 73 | whatever the kernel-du-jour happens to be. | 73 | whatever the kernel-du-jour happens to be. |
| 74 | 74 | ||
| 75 | - You can also upgrade between 4.x releases by patching. Patches are | 75 | - You can also upgrade between 5.x releases by patching. Patches are |
| 76 | distributed in the xz format. To install by patching, get all the | 76 | distributed in the xz format. To install by patching, get all the |
| 77 | newer patch files, enter the top level directory of the kernel source | 77 | newer patch files, enter the top level directory of the kernel source |
| 78 | (linux-4.X) and execute:: | 78 | (linux-5.x) and execute:: |
| 79 | 79 | ||
| 80 | xz -cd ../patch-4.x.xz | patch -p1 | 80 | xz -cd ../patch-5.x.xz | patch -p1 |
| 81 | 81 | ||
| 82 | Replace "x" for all versions bigger than the version "X" of your current | 82 | Replace "x" for all versions bigger than the version "x" of your current |
| 83 | source tree, **in_order**, and you should be ok. You may want to remove | 83 | source tree, **in_order**, and you should be ok. You may want to remove |
| 84 | the backup files (some-file-name~ or some-file-name.orig), and make sure | 84 | the backup files (some-file-name~ or some-file-name.orig), and make sure |
| 85 | that there are no failed patches (some-file-name# or some-file-name.rej). | 85 | that there are no failed patches (some-file-name# or some-file-name.rej). |
| 86 | If there are, either you or I have made a mistake. | 86 | If there are, either you or I have made a mistake. |
| 87 | 87 | ||
| 88 | Unlike patches for the 4.x kernels, patches for the 4.x.y kernels | 88 | Unlike patches for the 5.x kernels, patches for the 5.x.y kernels |
| 89 | (also known as the -stable kernels) are not incremental but instead apply | 89 | (also known as the -stable kernels) are not incremental but instead apply |
| 90 | directly to the base 4.x kernel. For example, if your base kernel is 4.0 | 90 | directly to the base 5.x kernel. For example, if your base kernel is 5.0 |
| 91 | and you want to apply the 4.0.3 patch, you must not first apply the 4.0.1 | 91 | and you want to apply the 5.0.3 patch, you must not first apply the 5.0.1 |
| 92 | and 4.0.2 patches. Similarly, if you are running kernel version 4.0.2 and | 92 | and 5.0.2 patches. Similarly, if you are running kernel version 5.0.2 and |
| 93 | want to jump to 4.0.3, you must first reverse the 4.0.2 patch (that is, | 93 | want to jump to 5.0.3, you must first reverse the 5.0.2 patch (that is, |
| 94 | patch -R) **before** applying the 4.0.3 patch. You can read more on this in | 94 | patch -R) **before** applying the 5.0.3 patch. You can read more on this in |
| 95 | :ref:`Documentation/process/applying-patches.rst <applying_patches>`. | 95 | :ref:`Documentation/process/applying-patches.rst <applying_patches>`. |
| 96 | 96 | ||
| 97 | Alternatively, the script patch-kernel can be used to automate this | 97 | Alternatively, the script patch-kernel can be used to automate this |
| @@ -114,7 +114,7 @@ Installing the kernel source · 16 unmodified lines | |||
| 114 | Software requirements | 114 | Software requirements |
| 115 | --------------------- | 115 | --------------------- |
| 116 | 116 | ||
| 117 | Compiling and running the 4.x kernels requires up-to-date | 117 | Compiling and running the 5.x kernels requires up-to-date |
| 118 | versions of various software packages. Consult | 118 | versions of various software packages. Consult |
| 119 | :ref:`Documentation/process/changes.rst <changes>` for the minimum version numbers | 119 | :ref:`Documentation/process/changes.rst <changes>` for the minimum version numbers |
| 120 | required and how to get updates for these packages. Beware that using | 120 | required and how to get updates for these packages. Beware that using |
| @@ -132,12 +132,12 @@ Build directory for the kernel · 11 unmodified lines | |||
| 132 | place for the output files (including .config). | 132 | place for the output files (including .config). |
| 133 | Example:: | 133 | Example:: |
| 134 | 134 | ||
| 135 | kernel source code: /usr/src/linux-4.X | 135 | kernel source code: /usr/src/linux-5.x |
| 136 | build directory: /home/name/build/kernel | 136 | build directory: /home/name/build/kernel |
| 137 | 137 | ||
| 138 | To configure and build the kernel, use:: | 138 | To configure and build the kernel, use:: |
| 139 | 139 | ||
| 140 | cd /usr/src/linux-4.X | 140 | cd /usr/src/linux-5.x |
| 141 | make O=/home/name/build/kernel menuconfig | 141 | make O=/home/name/build/kernel menuconfig |
| 142 | make O=/home/name/build/kernel | 142 | make O=/home/name/build/kernel |
| 143 | sudo make O=/home/name/build/kernel modules_install install | 143 | sudo make O=/home/name/build/kernel modules_install install |
| @@ -0,0 +1,59 @@ | |||
| 1 | Qualcomm adreno/snapdragon GMU (Graphics management unit) | ||
| 2 | |||
| 3 | The GMU is a programmable power controller for the GPU. the CPU controls the | ||
| 4 | GMU which in turn handles power controls for the GPU. | ||
| 5 | |||
| 6 | Required properties: | ||
| 7 | - compatible: "qcom,adreno-gmu-XYZ.W", "qcom,adreno-gmu" | ||
| 8 | for example: "qcom,adreno-gmu-630.2", "qcom,adreno-gmu" | ||
| 9 | Note that you need to list the less specific "qcom,adreno-gmu" | ||
| 10 | for generic matches and the more specific identifier to identify | ||
| 11 | the specific device. | ||
| 12 | - reg: Physical base address and length of the GMU registers. | ||
| 13 | - reg-names: Matching names for the register regions | ||
| 14 | * "gmu" | ||
| 15 | * "gmu_pdc" | ||
| 16 | * "gmu_pdc_seg" | ||
| 17 | - interrupts: The interrupt signals from the GMU. | ||
| 18 | - interrupt-names: Matching names for the interrupts | ||
| 19 | * "hfi" | ||
| 20 | * "gmu" | ||
| 21 | - clocks: phandles to the device clocks | ||
| 22 | - clock-names: Matching names for the clocks | ||
| 23 | * "gmu" | ||
| 24 | * "cxo" | ||
| 25 | * "axi" | ||
| 26 | * "mnoc" | ||
| 27 | - power-domains: should be <&clock_gpucc GPU_CX_GDSC> | ||
| 28 | - iommus: phandle to the adreno iommu | ||
| 29 | - operating-points-v2: phandle to the OPP operating points | ||
| 30 | |||
| 31 | Example: | ||
| 32 | |||
| 33 | / { | ||
| 34 | ... | ||
| 35 | |||
| 36 | gmu: gmu@506a000 { | ||
| 37 | compatible="qcom,adreno-gmu-630.2", "qcom,adreno-gmu"; | ||
| 38 | |||
| 39 | reg = <0x506a000 0x30000>, | ||
| 40 | <0xb280000 0x10000>, | ||
| 41 | <0xb480000 0x10000>; | ||
| 42 | reg-names = "gmu", "gmu_pdc", "gmu_pdc_seq"; | ||
| 43 | |||
| 44 | interrupts = <GIC_SPI 304 IRQ_TYPE_LEVEL_HIGH>, | ||
| 45 | <GIC_SPI 305 IRQ_TYPE_LEVEL_HIGH>; | ||
| 46 | interrupt-names = "hfi", "gmu"; | ||
| 47 | |||
| 48 | clocks = <&gpucc GPU_CC_CX_GMU_CLK>, | ||
| 49 | <&gpucc GPU_CC_CXO_CLK>, | ||
| 50 | <&gcc GCC_DDRSS_GPU_AXI_CLK>, | ||
| 51 | <&gcc GCC_GPU_MEMNOC_GFX_CLK>; | ||
| 52 | clock-names = "gmu", "cxo", "axi", "memnoc"; | ||
| 53 | |||
| 54 | power-domains = <&gpucc GPU_CX_GDSC>; | ||
| 55 | iommus = <&adreno_smmu 5>; | ||
| 56 | |||
| 57 | operating-points-v2 = <&gmu_opp_table>; | ||
| 58 | }; | ||
| 59 | }; | ||
| @@ -10,14 +10,23 @@ Required properties: · 9 unmodified lines | |||
| 10 | If "amd,imageon" is used, there should be no top level msm device. | 10 | If "amd,imageon" is used, there should be no top level msm device. |
| 11 | - reg: Physical base address and length of the controller's registers. | 11 | - reg: Physical base address and length of the controller's registers. |
| 12 | - interrupts: The interrupt signal from the gpu. | 12 | - interrupts: The interrupt signal from the gpu. |
| 13 | - clocks: device clocks | 13 | - clocks: device clocks (if applicable) |
| 14 | See ../clocks/clock-bindings.txt for details. | 14 | See ../clocks/clock-bindings.txt for details. |
| 15 | - clock-names: the following clocks are required: | 15 | - clock-names: the following clocks are required by a3xx, a4xx and a5xx |
| 16 | cores: | ||
| 16 | * "core" | 17 | * "core" |
| 17 | * "iface" | 18 | * "iface" |
| 18 | * "mem_iface" | 19 | * "mem_iface" |
| 20 | For GMU attached devices the GPU clocks are not used and are not required. The | ||
| 21 | following devices should not list clocks: | ||
| 22 | - qcom,adreno-630.2 | ||
| 23 | - iommus: optional phandle to an adreno iommu instance | ||
| 24 | - operating-points-v2: optional phandle to the OPP operating points | ||
| 25 | - qcom,gmu: For GMU attached devices a phandle to the GMU device that will | ||
| 26 | control the power for the GPU. Applicable targets: | ||
| 27 | - qcom,adreno-630.2 | ||
| 19 | 28 | ||
| 20 | Example: | 29 | Example 3xx/4xx/a5xx: |
| 21 | 30 | ||
| 22 | / { | 31 | / { |
| 23 | ... | 32 | ... |
| @@ -37,3 +46,30 @@ Example: · 13 unmodified lines | |||
| 37 | <&mmcc MMSS_IMEM_AHB_CLK>; | 46 | <&mmcc MMSS_IMEM_AHB_CLK>; |
| 38 | }; | 47 | }; |
| 39 | }; | 48 | }; |
| 49 | |||
| 50 | Example a6xx (with GMU): | ||
| 51 | |||
| 52 | / { | ||
| 53 | ... | ||
| 54 | |||
| 55 | gpu@5000000 { | ||
| 56 | compatible = "qcom,adreno-630.2", "qcom,adreno"; | ||
| 57 | #stream-id-cells = <16>; | ||
| 58 | |||
| 59 | reg = <0x5000000 0x40000>, <0x509e000 0x10>; | ||
| 60 | reg-names = "kgsl_3d0_reg_memory", "cx_mem"; | ||
| 61 | |||
| 62 | /* | ||
| 63 | * Look ma, no clocks! The GPU clocks and power are | ||
| 64 | * controlled entirely by the GMU | ||
| 65 | */ | ||
| 66 | |||
| 67 | interrupts = <GIC_SPI 300 IRQ_TYPE_LEVEL_HIGH>; | ||
| 68 | |||
| 69 | iommus = <&adreno_smmu 0>; | ||
| 70 | |||
| 71 | operating-points-v2 = <&gpu_opp_table>; | ||
| 72 | |||
| 73 | qcom,gmu = <&gmu>; | ||
| 74 | }; | ||
| 75 | }; | ||
| @@ -533,16 +533,12 @@ Bridge VLAN filtering · 532 unmodified lines | |||
| 533 | function that the driver has to call for each VLAN the given port is a member | 533 | function that the driver has to call for each VLAN the given port is a member |
| 534 | of. A switchdev object is used to carry the VID and bridge flags. | 534 | of. A switchdev object is used to carry the VID and bridge flags. |
| 535 | 535 | ||
| 536 | - port_fdb_prepare: bridge layer function invoked when the bridge prepares the | ||
| 537 | installation of a Forwarding Database entry. If the operation is not | ||
| 538 | supported, this function should return -EOPNOTSUPP to inform the bridge code | ||
| 539 | to fallback to a software implementation. No hardware setup must be done in | ||
| 540 | this function. See port_fdb_add for this and details. | ||
| 541 | |||
| 542 | - port_fdb_add: bridge layer function invoked when the bridge wants to install a | 536 | - port_fdb_add: bridge layer function invoked when the bridge wants to install a |
| 543 | Forwarding Database entry, the switch hardware should be programmed with the | 537 | Forwarding Database entry, the switch hardware should be programmed with the |
| 544 | specified address in the specified VLAN Id in the forwarding database | 538 | specified address in the specified VLAN Id in the forwarding database |
| 545 | associated with this VLAN ID | 539 | associated with this VLAN ID. If the operation is not supported, this |
| 540 | function should return -EOPNOTSUPP to inform the bridge code to fallback to | ||
| 541 | a software implementation. | ||
| 546 | 542 | ||
| 547 | Note: VLAN ID 0 corresponds to the port private database, which, in the context | 543 | Note: VLAN ID 0 corresponds to the port private database, which, in the context |
| 548 | of DSA, would be the its port-based VLAN, used by the associated bridge device. | 544 | of DSA, would be the its port-based VLAN, used by the associated bridge device. |
| @@ -7,7 +7,7 @@ Intro · 6 unmodified lines | |||
| 7 | ===== | 7 | ===== |
| 8 | 8 | ||
| 9 | The MSG_ZEROCOPY flag enables copy avoidance for socket send calls. | 9 | The MSG_ZEROCOPY flag enables copy avoidance for socket send calls. |
| 10 | The feature is currently implemented for TCP sockets. | 10 | The feature is currently implemented for TCP and UDP sockets. |
| 11 | 11 | ||
| 12 | 12 | ||
| 13 | Opportunity and Caveats | 13 | Opportunity and Caveats |
| @@ -92,11 +92,11 @@ device. · 91 unmodified lines | |||
| 92 | Switch ID | 92 | Switch ID |
| 93 | ^^^^^^^^^ | 93 | ^^^^^^^^^ |
| 94 | 94 | ||
| 95 | The switchdev driver must implement the switchdev op switchdev_port_attr_get | 95 | The switchdev driver must implement the net_device operation |
| 96 | for SWITCHDEV_ATTR_ID_PORT_PARENT_ID for each port netdev, returning the same | 96 | ndo_get_port_parent_id for each port netdev, returning the same physical ID for |
| 97 | physical ID for each port of a switch. The ID must be unique between switches | 97 | each port of a switch. The ID must be unique between switches on the same |
| 98 | on the same system. The ID does not need to be unique between switches on | 98 | system. The ID does not need to be unique between switches on different |
| 99 | different systems. | 99 | systems. |
| 100 | 100 | ||
| 101 | The switch ID is used to locate ports on a switch and to know if aggregated | 101 | The switch ID is used to locate ports on a switch and to know if aggregated |
| 102 | ports belong to the same switch. | 102 | ports belong to the same switch. |
| @@ -216,14 +216,14 @@ You can use the ``interdiff`` program (http://cyberelk.net/tim/patchutils/) to · 215 unmodified lines | |||
| 216 | generate a patch representing the differences between two patches and then | 216 | generate a patch representing the differences between two patches and then |
| 217 | apply the result. | 217 | apply the result. |
| 218 | 218 | ||
| 219 | This will let you move from something like 4.7.2 to 4.7.3 in a single | 219 | This will let you move from something like 5.7.2 to 5.7.3 in a single |
| 220 | step. The -z flag to interdiff will even let you feed it patches in gzip or | 220 | step. The -z flag to interdiff will even let you feed it patches in gzip or |
| 221 | bzip2 compressed form directly without the use of zcat or bzcat or manual | 221 | bzip2 compressed form directly without the use of zcat or bzcat or manual |
| 222 | decompression. | 222 | decompression. |
| 223 | 223 | ||
| 224 | Here's how you'd go from 4.7.2 to 4.7.3 in a single step:: | 224 | Here's how you'd go from 5.7.2 to 5.7.3 in a single step:: |
| 225 | 225 | ||
| 226 | interdiff -z ../patch-4.7.2.gz ../patch-4.7.3.gz | patch -p1 | 226 | interdiff -z ../patch-5.7.2.gz ../patch-5.7.3.gz | patch -p1 |
| 227 | 227 | ||
| 228 | Although interdiff may save you a step or two you are generally advised to | 228 | Although interdiff may save you a step or two you are generally advised to |
| 229 | do the additional steps since interdiff can get things wrong in some cases. | 229 | do the additional steps since interdiff can get things wrong in some cases. |
| @@ -245,62 +245,67 @@ The patches are available at http://kernel.org/ · 15 unmodified lines | |||
| 245 | Most recent patches are linked from the front page, but they also have | 245 | Most recent patches are linked from the front page, but they also have |
| 246 | specific homes. | 246 | specific homes. |
| 247 | 247 | ||
| 248 | The 4.x.y (-stable) and 4.x patches live at | 248 | The 5.x.y (-stable) and 5.x patches live at |
| 249 | 249 | ||
| 250 | https://www.kernel.org/pub/linux/kernel/v4.x/ | 250 | https://www.kernel.org/pub/linux/kernel/v5.x/ |
| 251 | 251 | ||
| 252 | The -rc patches live at | 252 | The -rc patches are not stored on the webserver but are generated on |
| 253 | demand from git tags such as | ||
| 253 | 254 | ||
| 254 | https://www.kernel.org/pub/linux/kernel/v4.x/testing/ | 255 | https://git.kernel.org/torvalds/p/v5.1-rc1/v5.0 |
| 255 | 256 | ||
| 257 | The stable -rc patches live at | ||
| 256 | 258 | ||
| 257 | The 4.x kernels | 259 | https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/ |
| 260 | |||
| 261 | |||
| 262 | The 5.x kernels | ||
| 258 | =============== | 263 | =============== |
| 259 | 264 | ||
| 260 | These are the base stable releases released by Linus. The highest numbered | 265 | These are the base stable releases released by Linus. The highest numbered |
| 261 | release is the most recent. | 266 | release is the most recent. |
| 262 | 267 | ||
| 263 | If regressions or other serious flaws are found, then a -stable fix patch | 268 | If regressions or other serious flaws are found, then a -stable fix patch |
| 264 | will be released (see below) on top of this base. Once a new 4.x base | 269 | will be released (see below) on top of this base. Once a new 5.x base |
| 265 | kernel is released, a patch is made available that is a delta between the | 270 | kernel is released, a patch is made available that is a delta between the |
| 266 | previous 4.x kernel and the new one. | 271 | previous 5.x kernel and the new one. |
| 267 | 272 | ||
| 268 | To apply a patch moving from 4.6 to 4.7, you'd do the following (note | 273 | To apply a patch moving from 5.6 to 5.7, you'd do the following (note |
| 269 | that such patches do **NOT** apply on top of 4.x.y kernels but on top of the | 274 | that such patches do **NOT** apply on top of 5.x.y kernels but on top of the |
| 270 | base 4.x kernel -- if you need to move from 4.x.y to 4.x+1 you need to | 275 | base 5.x kernel -- if you need to move from 5.x.y to 5.x+1 you need to |
| 271 | first revert the 4.x.y patch). | 276 | first revert the 5.x.y patch). |
| 272 | 277 | ||
| 273 | Here are some examples:: | 278 | Here are some examples:: |
| 274 | 279 | ||
| 275 | # moving from 4.6 to 4.7 | 280 | # moving from 5.6 to 5.7 |
| 276 | 281 | ||
| 277 | $ cd ~/linux-4.6 # change to kernel source dir | 282 | $ cd ~/linux-5.6 # change to kernel source dir |
| 278 | $ patch -p1 < ../patch-4.7 # apply the 4.7 patch | 283 | $ patch -p1 < ../patch-5.7 # apply the 5.7 patch |
| 279 | $ cd .. | 284 | $ cd .. |
| 280 | $ mv linux-4.6 linux-4.7 # rename source dir | 285 | $ mv linux-5.6 linux-5.7 # rename source dir |
| 281 | 286 | ||
| 282 | # moving from 4.6.1 to 4.7 | 287 | # moving from 5.6.1 to 5.7 |
| 283 | 288 | ||
| 284 | $ cd ~/linux-4.6.1 # change to kernel source dir | 289 | $ cd ~/linux-5.6.1 # change to kernel source dir |
| 285 | $ patch -p1 -R < ../patch-4.6.1 # revert the 4.6.1 patch | 290 | $ patch -p1 -R < ../patch-5.6.1 # revert the 5.6.1 patch |
| 286 | # source dir is now 4.6 | 291 | # source dir is now 5.6 |
| 287 | $ patch -p1 < ../patch-4.7 # apply new 4.7 patch | 292 | $ patch -p1 < ../patch-5.7 # apply new 5.7 patch |
| 288 | $ cd .. | 293 | $ cd .. |
| 289 | $ mv linux-4.6.1 linux-4.7 # rename source dir | 294 | $ mv linux-5.6.1 linux-5.7 # rename source dir |
| 290 | 295 | ||
| 291 | 296 | ||
| 292 | The 4.x.y kernels | 297 | The 5.x.y kernels |
| 293 | ================= | 298 | ================= |
| 294 | 299 | ||
| 295 | Kernels with 3-digit versions are -stable kernels. They contain small(ish) | 300 | Kernels with 3-digit versions are -stable kernels. They contain small(ish) |
| 296 | critical fixes for security problems or significant regressions discovered | 301 | critical fixes for security problems or significant regressions discovered |
| 297 | in a given 4.x kernel. | 302 | in a given 5.x kernel. |
| 298 | 303 | ||
| 299 | This is the recommended branch for users who want the most recent stable | 304 | This is the recommended branch for users who want the most recent stable |
| 300 | kernel and are not interested in helping test development/experimental | 305 | kernel and are not interested in helping test development/experimental |
| 301 | versions. | 306 | versions. |
| 302 | 307 | ||
| 303 | If no 4.x.y kernel is available, then the highest numbered 4.x kernel is | 308 | If no 5.x.y kernel is available, then the highest numbered 5.x kernel is |
| 304 | the current stable kernel. | 309 | the current stable kernel. |
| 305 | 310 | ||
| 306 | .. note:: | 311 | .. note:: |
| @@ -308,23 +313,23 @@ the current stable kernel. · 1 unmodified line | |||
| 308 | The -stable team usually do make incremental patches available as well | 313 | The -stable team usually do make incremental patches available as well |
| 309 | as patches against the latest mainline release, but I only cover the | 314 | as patches against the latest mainline release, but I only cover the |
| 310 | non-incremental ones below. The incremental ones can be found at | 315 | non-incremental ones below. The incremental ones can be found at |
| 311 | https://www.kernel.org/pub/linux/kernel/v4.x/incr/ | 316 | https://www.kernel.org/pub/linux/kernel/v5.x/incr/ |
| 312 | 317 | ||
| 313 | These patches are not incremental, meaning that for example the 4.7.3 | 318 | These patches are not incremental, meaning that for example the 5.7.3 |
| 314 | patch does not apply on top of the 4.7.2 kernel source, but rather on top | 319 | patch does not apply on top of the 5.7.2 kernel source, but rather on top |
| 315 | of the base 4.7 kernel source. | 320 | of the base 5.7 kernel source. |
| 316 | 321 | ||
| 317 | So, in order to apply the 4.7.3 patch to your existing 4.7.2 kernel | 322 | So, in order to apply the 5.7.3 patch to your existing 5.7.2 kernel |
| 318 | source you have to first back out the 4.7.2 patch (so you are left with a | 323 | source you have to first back out the 5.7.2 patch (so you are left with a |
| 319 | base 4.7 kernel source) and then apply the new 4.7.3 patch. | 324 | base 5.7 kernel source) and then apply the new 5.7.3 patch. |
| 320 | 325 | ||
| 321 | Here's a small example:: | 326 | Here's a small example:: |
| 322 | 327 | ||
| 323 | $ cd ~/linux-4.7.2 # change to the kernel source dir | 328 | $ cd ~/linux-5.7.2 # change to the kernel source dir |
| 324 | $ patch -p1 -R < ../patch-4.7.2 # revert the 4.7.2 patch | 329 | $ patch -p1 -R < ../patch-5.7.2 # revert the 5.7.2 patch |
| 325 | $ patch -p1 < ../patch-4.7.3 # apply the new 4.7.3 patch | 330 | $ patch -p1 < ../patch-5.7.3 # apply the new 5.7.3 patch |
| 326 | $ cd .. | 331 | $ cd .. |
| 327 | $ mv linux-4.7.2 linux-4.7.3 # rename the kernel source dir | 332 | $ mv linux-5.7.2 linux-5.7.3 # rename the kernel source dir |
| 328 | 333 | ||
| 329 | The -rc kernels | 334 | The -rc kernels |
| 330 | =============== | 335 | =============== |
| @@ -343,38 +348,38 @@ This is a good branch to run for people who want to help out testing · 12 unmodified lines | |||
| 343 | development kernels but do not want to run some of the really experimental | 348 | development kernels but do not want to run some of the really experimental |
| 344 | stuff (such people should see the sections about -next and -mm kernels below). | 349 | stuff (such people should see the sections about -next and -mm kernels below). |
| 345 | 350 | ||
| 346 | The -rc patches are not incremental, they apply to a base 4.x kernel, just | 351 | The -rc patches are not incremental, they apply to a base 5.x kernel, just |
| 347 | like the 4.x.y patches described above. The kernel version before the -rcN | 352 | like the 5.x.y patches described above. The kernel version before the -rcN |
| 348 | suffix denotes the version of the kernel that this -rc kernel will eventually | 353 | suffix denotes the version of the kernel that this -rc kernel will eventually |
| 349 | turn into. | 354 | turn into. |
| 350 | 355 | ||
| 351 | So, 4.8-rc5 means that this is the fifth release candidate for the 4.8 | 356 | So, 5.8-rc5 means that this is the fifth release candidate for the 5.8 |
| 352 | kernel and the patch should be applied on top of the 4.7 kernel source. | 357 | kernel and the patch should be applied on top of the 5.7 kernel source. |
| 353 | 358 | ||
| 354 | Here are 3 examples of how to apply these patches:: | 359 | Here are 3 examples of how to apply these patches:: |
| 355 | 360 | ||
| 356 | # first an example of moving from 4.7 to 4.8-rc3 | 361 | # first an example of moving from 5.7 to 5.8-rc3 |
| 357 | 362 | ||
| 358 | $ cd ~/linux-4.7 # change to the 4.7 source dir | 363 | $ cd ~/linux-5.7 # change to the 5.7 source dir |
| 359 | $ patch -p1 < ../patch-4.8-rc3 # apply the 4.8-rc3 patch | 364 | $ patch -p1 < ../patch-5.8-rc3 # apply the 5.8-rc3 patch |
| 360 | $ cd .. | 365 | $ cd .. |
| 361 | $ mv linux-4.7 linux-4.8-rc3 # rename the source dir | 366 | $ mv linux-5.7 linux-5.8-rc3 # rename the source dir |
| 362 | 367 | ||
| 363 | # now let's move from 4.8-rc3 to 4.8-rc5 | 368 | # now let's move from 5.8-rc3 to 5.8-rc5 |
| 364 | 369 | ||
| 365 | $ cd ~/linux-4.8-rc3 # change to the 4.8-rc3 dir | 370 | $ cd ~/linux-5.8-rc3 # change to the 5.8-rc3 dir |
| 366 | $ patch -p1 -R < ../patch-4.8-rc3 # revert the 4.8-rc3 patch | 371 | $ patch -p1 -R < ../patch-5.8-rc3 # revert the 5.8-rc3 patch |
| 367 | $ patch -p1 < ../patch-4.8-rc5 # apply the new 4.8-rc5 patch | 372 | $ patch -p1 < ../patch-5.8-rc5 # apply the new 5.8-rc5 patch |
| 368 | $ cd .. | 373 | $ cd .. |
| 369 | $ mv linux-4.8-rc3 linux-4.8-rc5 # rename the source dir | 374 | $ mv linux-5.8-rc3 linux-5.8-rc5 # rename the source dir |
| 370 | 375 | ||
| 371 | # finally let's try and move from 4.7.3 to 4.8-rc5 | 376 | # finally let's try and move from 5.7.3 to 5.8-rc5 |
| 372 | 377 | ||
| 373 | $ cd ~/linux-4.7.3 # change to the kernel source dir | 378 | $ cd ~/linux-5.7.3 # change to the kernel source dir |
| 374 | $ patch -p1 -R < ../patch-4.7.3 # revert the 4.7.3 patch | 379 | $ patch -p1 -R < ../patch-5.7.3 # revert the 5.7.3 patch |
| 375 | $ patch -p1 < ../patch-4.8-rc5 # apply new 4.8-rc5 patch | 380 | $ patch -p1 < ../patch-5.8-rc5 # apply new 5.8-rc5 patch |
| 376 | $ cd .. | 381 | $ cd .. |
| 377 | $ mv linux-4.7.3 linux-4.8-rc5 # rename the kernel source dir | 382 | $ mv linux-5.7.3 linux-5.8-rc5 # rename the kernel source dir |
| 378 | 383 | ||
| 379 | 384 | ||
| 380 | The -mm patches and the linux-next tree | 385 | The -mm patches and the linux-next tree |
| @@ -4,7 +4,7 @@ · 3 unmodified lines | |||
| 4 | 4 | ||
| 5 | .. _it_readme: | 5 | .. _it_readme: |
| 6 | 6 | ||
| 7 | Rilascio del kernel Linux 4.x <http://kernel.org/> | 7 | Rilascio del kernel Linux 5.x <http://kernel.org/> |
| 8 | =================================================== | 8 | =================================================== |
| 9 | 9 | ||
| 10 | .. warning:: | 10 | .. warning:: |
| @@ -409,8 +409,7 @@ F: drivers/platform/x86/wmi.c · 408 unmodified lines | |||
| 409 | F: include/uapi/linux/wmi.h | 409 | F: include/uapi/linux/wmi.h |
| 410 | 410 | ||
| 411 | AD1889 ALSA SOUND DRIVER | 411 | AD1889 ALSA SOUND DRIVER |
| 412 | M: Thibaut Varene <T-Bone@parisc-linux.org> | 412 | W: https://parisc.wiki.kernel.org/index.php/AD1889 |
| 413 | W: http://wiki.parisc-linux.org/AD1889 | ||
| 414 | L: linux-parisc@vger.kernel.org | 413 | L: linux-parisc@vger.kernel.org |
| 415 | S: Maintained | 414 | S: Maintained |
| 416 | F: sound/pci/ad1889.* | 415 | F: sound/pci/ad1889.* |
| @@ -2865,7 +2864,7 @@ R: Martin KaFai Lau <kafai@fb.com> · 2448 unmodified lines | |||
| 2865 | R: Song Liu <songliubraving@fb.com> | 2864 | R: Song Liu <songliubraving@fb.com> |
| 2866 | R: Yonghong Song <yhs@fb.com> | 2865 | R: Yonghong Song <yhs@fb.com> |
| 2867 | L: netdev@vger.kernel.org | 2866 | L: netdev@vger.kernel.org |
| 2868 | L: linux-kernel@vger.kernel.org | 2867 | L: bpf@vger.kernel.org |
| 2869 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git | 2868 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git |
| 2870 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git | 2869 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git |
| 2871 | Q: https://patchwork.ozlabs.org/project/netdev/list/?delegate=77147 | 2870 | Q: https://patchwork.ozlabs.org/project/netdev/list/?delegate=77147 |
| @@ -2895,6 +2894,7 @@ N: bpf · 23 unmodified lines | |||
| 2895 | BPF JIT for ARM | 2894 | BPF JIT for ARM |
| 2896 | M: Shubham Bansal <illusionist.neo@gmail.com> | 2895 | M: Shubham Bansal <illusionist.neo@gmail.com> |
| 2897 | L: netdev@vger.kernel.org | 2896 | L: netdev@vger.kernel.org |
| 2897 | L: bpf@vger.kernel.org | ||
| 2898 | S: Maintained | 2898 | S: Maintained |
| 2899 | F: arch/arm/net/ | 2899 | F: arch/arm/net/ |
| 2900 | 2900 | ||
| @@ -2903,18 +2903,21 @@ M: Daniel Borkmann <daniel@iogearbox.net> · 2 unmodified lines | |||
| 2903 | M: Alexei Starovoitov <ast@kernel.org> | 2903 | M: Alexei Starovoitov <ast@kernel.org> |
| 2904 | M: Zi Shen Lim <zlim.lnx@gmail.com> | 2904 | M: Zi Shen Lim <zlim.lnx@gmail.com> |
| 2905 | L: netdev@vger.kernel.org | 2905 | L: netdev@vger.kernel.org |
| 2906 | L: bpf@vger.kernel.org | ||
| 2906 | S: Supported | 2907 | S: Supported |
| 2907 | F: arch/arm64/net/ | 2908 | F: arch/arm64/net/ |
| 2908 | 2909 | ||
| 2909 | BPF JIT for MIPS (32-BIT AND 64-BIT) | 2910 | BPF JIT for MIPS (32-BIT AND 64-BIT) |
| 2910 | M: Paul Burton <paul.burton@mips.com> | 2911 | M: Paul Burton <paul.burton@mips.com> |
| 2911 | L: netdev@vger.kernel.org | 2912 | L: netdev@vger.kernel.org |
| 2913 | L: bpf@vger.kernel.org | ||
| 2912 | S: Maintained | 2914 | S: Maintained |
| 2913 | F: arch/mips/net/ | 2915 | F: arch/mips/net/ |
| 2914 | 2916 | ||
| 2915 | BPF JIT for NFP NICs | 2917 | BPF JIT for NFP NICs |
| 2916 | M: Jakub Kicinski <jakub.kicinski@netronome.com> | 2918 | M: Jakub Kicinski <jakub.kicinski@netronome.com> |
| 2917 | L: netdev@vger.kernel.org | 2919 | L: netdev@vger.kernel.org |
| 2920 | L: bpf@vger.kernel.org | ||
| 2918 | S: Supported | 2921 | S: Supported |
| 2919 | F: drivers/net/ethernet/netronome/nfp/bpf/ | 2922 | F: drivers/net/ethernet/netronome/nfp/bpf/ |
| 2920 | 2923 | ||
| @@ -2922,6 +2925,7 @@ BPF JIT for POWERPC (32-BIT AND 64-BIT) · 1 unmodified line | |||
| 2922 | M: Naveen N. Rao <naveen.n.rao@linux.ibm.com> | 2925 | M: Naveen N. Rao <naveen.n.rao@linux.ibm.com> |
| 2923 | M: Sandipan Das <sandipan@linux.ibm.com> | 2926 | M: Sandipan Das <sandipan@linux.ibm.com> |
| 2924 | L: netdev@vger.kernel.org | 2927 | L: netdev@vger.kernel.org |
| 2928 | L: bpf@vger.kernel.org | ||
| 2925 | S: Maintained | 2929 | S: Maintained |
| 2926 | F: arch/powerpc/net/ | 2930 | F: arch/powerpc/net/ |
| 2927 | 2931 | ||
| @@ -2929,6 +2933,7 @@ BPF JIT for S390 · 1 unmodified line | |||
| 2929 | M: Martin Schwidefsky <schwidefsky@de.ibm.com> | 2933 | M: Martin Schwidefsky <schwidefsky@de.ibm.com> |
| 2930 | M: Heiko Carstens <heiko.carstens@de.ibm.com> | 2934 | M: Heiko Carstens <heiko.carstens@de.ibm.com> |
| 2931 | L: netdev@vger.kernel.org | 2935 | L: netdev@vger.kernel.org |
| 2936 | L: bpf@vger.kernel.org | ||
| 2932 | S: Maintained | 2937 | S: Maintained |
| 2933 | F: arch/s390/net/ | 2938 | F: arch/s390/net/ |
| 2934 | X: arch/s390/net/pnet.c | 2939 | X: arch/s390/net/pnet.c |
| @@ -2936,12 +2941,14 @@ X: arch/s390/net/pnet.c · 1 unmodified line | |||
| 2936 | BPF JIT for SPARC (32-BIT AND 64-BIT) | 2941 | BPF JIT for SPARC (32-BIT AND 64-BIT) |
| 2937 | M: David S. Miller <davem@davemloft.net> | 2942 | M: David S. Miller <davem@davemloft.net> |
| 2938 | L: netdev@vger.kernel.org | 2943 | L: netdev@vger.kernel.org |
| 2944 | L: bpf@vger.kernel.org | ||
| 2939 | S: Maintained | 2945 | S: Maintained |
| 2940 | F: arch/sparc/net/ | 2946 | F: arch/sparc/net/ |
| 2941 | 2947 | ||
| 2942 | BPF JIT for X86 32-BIT | 2948 | BPF JIT for X86 32-BIT |
| 2943 | M: Wang YanQing <udknight@gmail.com> | 2949 | M: Wang YanQing <udknight@gmail.com> |
| 2944 | L: netdev@vger.kernel.org | 2950 | L: netdev@vger.kernel.org |
| 2951 | L: bpf@vger.kernel.org | ||
| 2945 | S: Maintained | 2952 | S: Maintained |
| 2946 | F: arch/x86/net/bpf_jit_comp32.c | 2953 | F: arch/x86/net/bpf_jit_comp32.c |
| 2947 | 2954 | ||
| @@ -2949,6 +2956,7 @@ BPF JIT for X86 64-BIT · 1 unmodified line | |||
| 2949 | M: Alexei Starovoitov <ast@kernel.org> | 2956 | M: Alexei Starovoitov <ast@kernel.org> |
| 2950 | M: Daniel Borkmann <daniel@iogearbox.net> | 2957 | M: Daniel Borkmann <daniel@iogearbox.net> |
| 2951 | L: netdev@vger.kernel.org | 2958 | L: netdev@vger.kernel.org |
| 2959 | L: bpf@vger.kernel.org | ||
| 2952 | S: Supported | 2960 | S: Supported |
| 2953 | F: arch/x86/net/ | 2961 | F: arch/x86/net/ |
| 2954 | X: arch/x86/net/bpf_jit_comp32.c | 2962 | X: arch/x86/net/bpf_jit_comp32.c |
| @@ -3403,9 +3411,8 @@ F: Documentation/media/v4l-drivers/cafe_ccic* · 448 unmodified lines | |||
| 3403 | F: drivers/media/platform/marvell-ccic/ | 3411 | F: drivers/media/platform/marvell-ccic/ |
| 3404 | 3412 | ||
| 3405 | CAIF NETWORK LAYER | 3413 | CAIF NETWORK LAYER |
| 3406 | M: Dmitry Tarnyagin <dmitry.tarnyagin@lockless.no> | ||
| 3407 | L: netdev@vger.kernel.org | 3414 | L: netdev@vger.kernel.org |
| 3408 | S: Supported | 3415 | S: Orphan |
| 3409 | F: Documentation/networking/caif/ | 3416 | F: Documentation/networking/caif/ |
| 3410 | F: drivers/net/caif/ | 3417 | F: drivers/net/caif/ |
| 3411 | F: include/uapi/linux/caif/ | 3418 | F: include/uapi/linux/caif/ |
| @@ -4851,10 +4858,11 @@ F: Documentation/devicetree/bindings/display/multi-inno,mi0283qt.txt · 1439 unmodified lines | |||
| 4851 | 4858 | ||
| 4852 | DRM DRIVER FOR MSM ADRENO GPU | 4859 | DRM DRIVER FOR MSM ADRENO GPU |
| 4853 | M: Rob Clark <robdclark@gmail.com> | 4860 | M: Rob Clark <robdclark@gmail.com> |
| 4861 | M: Sean Paul <sean@poorly.run> | ||
| 4854 | L: linux-arm-msm@vger.kernel.org | 4862 | L: linux-arm-msm@vger.kernel.org |
| 4855 | L: dri-devel@lists.freedesktop.org | 4863 | L: dri-devel@lists.freedesktop.org |
| 4856 | L: freedreno@lists.freedesktop.org | 4864 | L: freedreno@lists.freedesktop.org |
| 4857 | T: git git://people.freedesktop.org/~robclark/linux | 4865 | T: git https://gitlab.freedesktop.org/drm/msm.git |
| 4858 | S: Maintained | 4866 | S: Maintained |
| 4859 | F: drivers/gpu/drm/msm/ | 4867 | F: drivers/gpu/drm/msm/ |
| 4860 | F: include/uapi/drm/msm_drm.h | 4868 | F: include/uapi/drm/msm_drm.h |
| @@ -8523,6 +8531,7 @@ L7 BPF FRAMEWORK · 3662 unmodified lines | |||
| 8523 | M: John Fastabend <john.fastabend@gmail.com> | 8531 | M: John Fastabend <john.fastabend@gmail.com> |
| 8524 | M: Daniel Borkmann <daniel@iogearbox.net> | 8532 | M: Daniel Borkmann <daniel@iogearbox.net> |
| 8525 | L: netdev@vger.kernel.org | 8533 | L: netdev@vger.kernel.org |
| 8534 | L: bpf@vger.kernel.org | ||
| 8526 | S: Maintained | 8535 | S: Maintained |
| 8527 | F: include/linux/skmsg.h | 8536 | F: include/linux/skmsg.h |
| 8528 | F: net/core/skmsg.c | 8537 | F: net/core/skmsg.c |
| @@ -11524,7 +11533,7 @@ F: Documentation/blockdev/paride.txt · 2995 unmodified lines | |||
| 11524 | F: drivers/block/paride/ | 11533 | F: drivers/block/paride/ |
| 11525 | 11534 | ||
| 11526 | PARISC ARCHITECTURE | 11535 | PARISC ARCHITECTURE |
| 11527 | M: "James E.J. Bottomley" <jejb@parisc-linux.org> | 11536 | M: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com> |
| 11528 | M: Helge Deller <deller@gmx.de> | 11537 | M: Helge Deller <deller@gmx.de> |
| 11529 | L: linux-parisc@vger.kernel.org | 11538 | L: linux-parisc@vger.kernel.org |
| 11530 | W: http://www.parisc-linux.org/ | 11539 | W: http://www.parisc-linux.org/ |
| @@ -16750,6 +16759,7 @@ M: Jesper Dangaard Brouer <hawk@kernel.org> · 5219 unmodified lines | |||
| 16750 | M: John Fastabend <john.fastabend@gmail.com> | 16759 | M: John Fastabend <john.fastabend@gmail.com> |
| 16751 | L: netdev@vger.kernel.org | 16760 | L: netdev@vger.kernel.org |
| 16752 | L: xdp-newbies@vger.kernel.org | 16761 | L: xdp-newbies@vger.kernel.org |
| 16762 | L: bpf@vger.kernel.org | ||
| 16753 | S: Supported | 16763 | S: Supported |
| 16754 | F: net/core/xdp.c | 16764 | F: net/core/xdp.c |
| 16755 | F: include/net/xdp.h | 16765 | F: include/net/xdp.h |
| @@ -16763,6 +16773,7 @@ XDP SOCKETS (AF_XDP) · 7 unmodified lines | |||
| 16763 | M: Björn Töpel <bjorn.topel@intel.com> | 16773 | M: Björn Töpel <bjorn.topel@intel.com> |
| 16764 | M: Magnus Karlsson <magnus.karlsson@intel.com> | 16774 | M: Magnus Karlsson <magnus.karlsson@intel.com> |
| 16765 | L: netdev@vger.kernel.org | 16775 | L: netdev@vger.kernel.org |
| 16776 | L: bpf@vger.kernel.org | ||
| 16766 | S: Maintained | 16777 | S: Maintained |
| 16767 | F: kernel/bpf/xskmap.c | 16778 | F: kernel/bpf/xskmap.c |
| 16768 | F: net/xdp/ | 16779 | F: net/xdp/ |
| @@ -2,7 +2,7 @@ · 1 unmodified line | |||
| 2 | VERSION = 5 | 2 | VERSION = 5 |
| 3 | PATCHLEVEL = 0 | 3 | PATCHLEVEL = 0 |
| 4 | SUBLEVEL = 0 | 4 | SUBLEVEL = 0 |
| 5 | EXTRAVERSION = -rc7 | 5 | EXTRAVERSION = |
| 6 | NAME = Shy Crocodile | 6 | NAME = Shy Crocodile |
| 7 | 7 | ||
| 8 | # *DOCUMENTATION* | 8 | # *DOCUMENTATION* |
| @@ -191,7 +191,6 @@ config NR_CPUS · 190 unmodified lines | |||
| 191 | 191 | ||
| 192 | config ARC_SMP_HALT_ON_RESET | 192 | config ARC_SMP_HALT_ON_RESET |
| 193 | bool "Enable Halt-on-reset boot mode" | 193 | bool "Enable Halt-on-reset boot mode" |
| 194 | default y if ARC_UBOOT_SUPPORT | ||
| 195 | help | 194 | help |
| 196 | In SMP configuration cores can be configured as Halt-on-reset | 195 | In SMP configuration cores can be configured as Halt-on-reset |
| 197 | or they could all start at same time. For Halt-on-reset, non | 196 | or they could all start at same time. For Halt-on-reset, non |
| @@ -407,6 +406,14 @@ config ARC_HAS_ACCL_REGS · 209 unmodified lines | |||
| 407 | (also referred to as r58:r59). These can also be used by gcc as GPR so | 406 | (also referred to as r58:r59). These can also be used by gcc as GPR so |
| 408 | kernel needs to save/restore per process | 407 | kernel needs to save/restore per process |
| 409 | 408 | ||
| 409 | config ARC_IRQ_NO_AUTOSAVE | ||
| 410 | bool "Disable hardware autosave regfile on interrupts" | ||
| 411 | default n | ||
| 412 | help | ||
| 413 | On HS cores, taken interrupt auto saves the regfile on stack. | ||
| 414 | This is programmable and can be optionally disabled in which case | ||
| 415 | software INTERRUPT_PROLOGUE/EPILGUE do the needed work | ||
| 416 | |||
| 410 | endif # ISA_ARCV2 | 417 | endif # ISA_ARCV2 |
| 411 | 418 | ||
| 412 | endmenu # "ARC CPU Configuration" | 419 | endmenu # "ARC CPU Configuration" |
| @@ -515,17 +522,6 @@ config ARC_DBG_TLB_PARANOIA · 102 unmodified lines | |||
| 515 | 522 | ||
| 516 | endif | 523 | endif |
| 517 | 524 | ||
| 518 | config ARC_UBOOT_SUPPORT | ||
| 519 | bool "Support uboot arg Handling" | ||
| 520 | help | ||
| 521 | ARC Linux by default checks for uboot provided args as pointers to | ||
| 522 | external cmdline or DTB. This however breaks in absence of uboot, | ||
| 523 | when booting from Metaware debugger directly, as the registers are | ||
| 524 | not zeroed out on reset by mdb and/or ARCv2 based cores. The bogus | ||
| 525 | registers look like uboot args to kernel which then chokes. | ||
| 526 | So only enable the uboot arg checking/processing if users are sure | ||
| 527 | of uboot being in play. | ||
| 528 | |||
| 529 | config ARC_BUILTIN_DTB_NAME | 525 | config ARC_BUILTIN_DTB_NAME |
| 530 | string "Built in DTB" | 526 | string "Built in DTB" |
| 531 | help | 527 | help |
| @@ -31,7 +31,6 @@ CONFIG_ARC_CACHE_LINE_SHIFT=5 · 30 unmodified lines | |||
| 31 | # CONFIG_ARC_HAS_LLSC is not set | 31 | # CONFIG_ARC_HAS_LLSC is not set |
| 32 | CONFIG_ARC_KVADDR_SIZE=402 | 32 | CONFIG_ARC_KVADDR_SIZE=402 |
| 33 | CONFIG_ARC_EMUL_UNALIGNED=y | 33 | CONFIG_ARC_EMUL_UNALIGNED=y |
| 34 | CONFIG_ARC_UBOOT_SUPPORT=y | ||
| 35 | CONFIG_PREEMPT=y | 34 | CONFIG_PREEMPT=y |
| 36 | CONFIG_NET=y | 35 | CONFIG_NET=y |
| 37 | CONFIG_UNIX=y | 36 | CONFIG_UNIX=y |
| @@ -13,7 +13,6 @@ CONFIG_PARTITION_ADVANCED=y · 12 unmodified lines | |||
| 13 | CONFIG_ARC_PLAT_AXS10X=y | 13 | CONFIG_ARC_PLAT_AXS10X=y |
| 14 | CONFIG_AXS103=y | 14 | CONFIG_AXS103=y |
| 15 | CONFIG_ISA_ARCV2=y | 15 | CONFIG_ISA_ARCV2=y |
| 16 | CONFIG_ARC_UBOOT_SUPPORT=y | ||
| 17 | CONFIG_ARC_BUILTIN_DTB_NAME="vdk_hs38" | 16 | CONFIG_ARC_BUILTIN_DTB_NAME="vdk_hs38" |
| 18 | CONFIG_PREEMPT=y | 17 | CONFIG_PREEMPT=y |
| 19 | CONFIG_NET=y | 18 | CONFIG_NET=y |
| @@ -15,8 +15,6 @@ CONFIG_AXS103=y · 14 unmodified lines | |||
| 15 | CONFIG_ISA_ARCV2=y | 15 | CONFIG_ISA_ARCV2=y |
| 16 | CONFIG_SMP=y | 16 | CONFIG_SMP=y |
| 17 | # CONFIG_ARC_TIMERS_64BIT is not set | 17 | # CONFIG_ARC_TIMERS_64BIT is not set |
| 18 | # CONFIG_ARC_SMP_HALT_ON_RESET is not set | ||
| 19 | CONFIG_ARC_UBOOT_SUPPORT=y | ||
| 20 | CONFIG_ARC_BUILTIN_DTB_NAME="vdk_hs38_smp" | 18 | CONFIG_ARC_BUILTIN_DTB_NAME="vdk_hs38_smp" |
| 21 | CONFIG_PREEMPT=y | 19 | CONFIG_PREEMPT=y |
| 22 | CONFIG_NET=y | 20 | CONFIG_NET=y |
| @@ -151,6 +151,14 @@ struct bcr_isa_arcv2 { · 150 unmodified lines | |||
| 151 | #endif | 151 | #endif |
| 152 | }; | 152 | }; |
| 153 | 153 | ||
| 154 | struct bcr_uarch_build_arcv2 { | ||
| 155 | #ifdef CONFIG_CPU_BIG_ENDIAN | ||
| 156 | unsigned int pad:8, prod:8, maj:8, min:8; | ||
| 157 | #else | ||
| 158 | unsigned int min:8, maj:8, prod:8, pad:8; | ||
| 159 | #endif | ||
| 160 | }; | ||
| 161 | |||
| 154 | struct bcr_mpy { | 162 | struct bcr_mpy { |
| 155 | #ifdef CONFIG_CPU_BIG_ENDIAN | 163 | #ifdef CONFIG_CPU_BIG_ENDIAN |
| 156 | unsigned int pad:8, x1616:8, dsp:4, cycles:2, type:2, ver:8; | 164 | unsigned int pad:8, x1616:8, dsp:4, cycles:2, type:2, ver:8; |
| @@ -52,6 +52,17 @@ · 51 unmodified lines | |||
| 52 | #define cache_line_size() SMP_CACHE_BYTES | 52 | #define cache_line_size() SMP_CACHE_BYTES |
| 53 | #define ARCH_DMA_MINALIGN SMP_CACHE_BYTES | 53 | #define ARCH_DMA_MINALIGN SMP_CACHE_BYTES |
| 54 | 54 | ||
| 55 | /* | ||
| 56 | * Make sure slab-allocated buffers are 64-bit aligned when atomic64_t uses | ||
| 57 | * ARCv2 64-bit atomics (LLOCKD/SCONDD). This guarantess runtime 64-bit | ||
| 58 | * alignment for any atomic64_t embedded in buffer. | ||
| 59 | * Default ARCH_SLAB_MINALIGN is __alignof__(long long) which has a relaxed | ||
| 60 | * value of 4 (and not 8) in ARC ABI. | ||
| 61 | */ | ||
| 62 | #if defined(CONFIG_ARC_HAS_LL64) && defined(CONFIG_ARC_HAS_LLSC) | ||
| 63 | #define ARCH_SLAB_MINALIGN 8 | ||
| 64 | #endif | ||
| 65 | |||
| 55 | extern void arc_cache_init(void); | 66 | extern void arc_cache_init(void); |
| 56 | extern char *arc_cache_mumbojumbo(int cpu_id, char *buf, int len); | 67 | extern char *arc_cache_mumbojumbo(int cpu_id, char *buf, int len); |
| 57 | extern void read_decode_cache_bcr(void); | 68 | extern void read_decode_cache_bcr(void); |
| @@ -17,6 +17,33 @@ · 16 unmodified lines | |||
| 17 | ; | 17 | ; |
| 18 | ; Now manually save: r12, sp, fp, gp, r25 | 18 | ; Now manually save: r12, sp, fp, gp, r25 |
| 19 | 19 | ||
| 20 | #ifdef CONFIG_ARC_IRQ_NO_AUTOSAVE | ||
| 21 | .ifnc \called_from, exception | ||
| 22 | st.as r9, [sp, -10] ; save r9 in it's final stack slot | ||
| 23 | sub sp, sp, 12 ; skip JLI, LDI, EI | ||
| 24 | |||
| 25 | PUSH lp_count | ||
| 26 | PUSHAX lp_start | ||
| 27 | PUSHAX lp_end | ||
| 28 | PUSH blink | ||
| 29 | |||
| 30 | PUSH r11 | ||
| 31 | PUSH r10 | ||
| 32 | |||
| 33 | sub sp, sp, 4 ; skip r9 | ||
| 34 | |||
| 35 | PUSH r8 | ||
| 36 | PUSH r7 | ||
| 37 | PUSH r6 | ||
| 38 | PUSH r5 | ||
| 39 | PUSH r4 | ||
| 40 | PUSH r3 | ||
| 41 | PUSH r2 | ||
| 42 | PUSH r1 | ||
| 43 | PUSH r0 | ||
| 44 | .endif | ||
| 45 | #endif | ||
| 46 | |||
| 20 | #ifdef CONFIG_ARC_HAS_ACCL_REGS | 47 | #ifdef CONFIG_ARC_HAS_ACCL_REGS |
| 21 | PUSH r59 | 48 | PUSH r59 |
| 22 | PUSH r58 | 49 | PUSH r58 |
| @@ -86,6 +113,33 @@ · 63 unmodified lines | |||
| 86 | POP r59 | 113 | POP r59 |
| 87 | #endif | 114 | #endif |
| 88 | 115 | ||
| 116 | #ifdef CONFIG_ARC_IRQ_NO_AUTOSAVE | ||
| 117 | .ifnc \called_from, exception | ||
| 118 | POP r0 | ||
| 119 | POP r1 | ||
| 120 | POP r2 | ||
| 121 | POP r3 | ||
| 122 | POP r4 | ||
| 123 | POP r5 | ||
| 124 | POP r6 | ||
| 125 | POP r7 | ||
| 126 | POP r8 | ||
| 127 | POP r9 | ||
| 128 | POP r10 | ||
| 129 | POP r11 | ||
| 130 | |||
| 131 | POP blink | ||
| 132 | POPAX lp_end | ||
| 133 | POPAX lp_start | ||
| 134 | |||
| 135 | POP r9 | ||
| 136 | mov lp_count, r9 | ||
| 137 | |||
| 138 | add sp, sp, 12 ; skip JLI, LDI, EI | ||
| 139 | ld.as r9, [sp, -10] ; reload r9 which got clobbered | ||
| 140 | .endif | ||
| 141 | #endif | ||
| 142 | |||
| 89 | .endm | 143 | .endm |
| 90 | 144 | ||
| 91 | /*------------------------------------------------------------------------*/ | 145 | /*------------------------------------------------------------------------*/ |
| @@ -207,7 +207,7 @@ raw_copy_from_user(void *to, const void __user *from, unsigned long n) · 206 unmodified lines | |||
| 207 | */ | 207 | */ |
| 208 | "=&r" (tmp), "+r" (to), "+r" (from) | 208 | "=&r" (tmp), "+r" (to), "+r" (from) |
| 209 | : | 209 | : |
| 210 | : "lp_count", "lp_start", "lp_end", "memory"); | 210 | : "lp_count", "memory"); |
| 211 | 211 | ||
| 212 | return n; | 212 | return n; |
| 213 | } | 213 | } |
| @@ -433,7 +433,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n) · 219 unmodified lines | |||
| 433 | */ | 433 | */ |
| 434 | "=&r" (tmp), "+r" (to), "+r" (from) | 434 | "=&r" (tmp), "+r" (to), "+r" (from) |
| 435 | : | 435 | : |
| 436 | : "lp_count", "lp_start", "lp_end", "memory"); | 436 | : "lp_count", "memory"); |
| 437 | 437 | ||
| 438 | return n; | 438 | return n; |
| 439 | } | 439 | } |
| @@ -653,7 +653,7 @@ static inline unsigned long __arc_clear_user(void __user *to, unsigned long n) · 213 unmodified lines | |||
| 653 | " .previous \n" | 653 | " .previous \n" |
| 654 | : "+r"(d_char), "+r"(res) | 654 | : "+r"(d_char), "+r"(res) |
| 655 | : "i"(0) | 655 | : "i"(0) |
| 656 | : "lp_count", "lp_start", "lp_end", "memory"); | 656 | : "lp_count", "memory"); |
| 657 | 657 | ||
| 658 | return res; | 658 | return res; |
| 659 | } | 659 | } |
| @@ -686,7 +686,7 @@ __arc_strncpy_from_user(char *dst, const char __user *src, long count) · 26 unmodified lines | |||
| 686 | " .previous \n" | 686 | " .previous \n" |
| 687 | : "+r"(res), "+r"(dst), "+r"(src), "=r"(val) | 687 | : "+r"(res), "+r"(dst), "+r"(src), "=r"(val) |
| 688 | : "g"(-EFAULT), "r"(count) | 688 | : "g"(-EFAULT), "r"(count) |
| 689 | : "lp_count", "lp_start", "lp_end", "memory"); | 689 | : "lp_count", "memory"); |
| 690 | 690 | ||
| 691 | return res; | 691 | return res; |
| 692 | } | 692 | } |
| @@ -209,7 +209,9 @@ restore_regs: · 208 unmodified lines | |||
| 209 | ;####### Return from Intr ####### | 209 | ;####### Return from Intr ####### |
| 210 | 210 | ||
| 211 | debug_marker_l1: | 211 | debug_marker_l1: |
| 212 | bbit1.nt r0, STATUS_DE_BIT, .Lintr_ret_to_delay_slot | 212 | ; bbit1.nt r0, STATUS_DE_BIT, .Lintr_ret_to_delay_slot |
| 213 | btst r0, STATUS_DE_BIT ; Z flag set if bit clear | ||
| 214 | bnz .Lintr_ret_to_delay_slot ; branch if STATUS_DE_BIT set | ||
| 213 | 215 | ||
| 214 | .Lisr_ret_fast_path: | 216 | .Lisr_ret_fast_path: |
| 215 | ; Handle special case #1: (Entry via Exception, Return via IRQ) | 217 | ; Handle special case #1: (Entry via Exception, Return via IRQ) |
| @@ -17,6 +17,7 @@ · 16 unmodified lines | |||
| 17 | #include <asm/entry.h> | 17 | #include <asm/entry.h> |
| 18 | #include <asm/arcregs.h> | 18 | #include <asm/arcregs.h> |
| 19 | #include <asm/cache.h> | 19 | #include <asm/cache.h> |
| 20 | #include <asm/irqflags.h> | ||
| 20 | 21 | ||
| 21 | .macro CPU_EARLY_SETUP | 22 | .macro CPU_EARLY_SETUP |
| 22 | 23 | ||
| @@ -47,6 +48,15 @@ · 24 unmodified lines | |||
| 47 | sr r5, [ARC_REG_DC_CTRL] | 48 | sr r5, [ARC_REG_DC_CTRL] |
| 48 | 49 | ||
| 49 | 1: | 50 | 1: |
| 51 | |||
| 52 | #ifdef CONFIG_ISA_ARCV2 | ||
| 53 | ; Unaligned access is disabled at reset, so re-enable early as | ||
| 54 | ; gcc 7.3.1 (ARC GNU 2018.03) onwards generates unaligned access | ||
| 55 | ; by default | ||
| 56 | lr r5, [status32] | ||
| 57 | bset r5, r5, STATUS_AD_BIT | ||
| 58 | kflag r5 | ||
| 59 | #endif | ||
| 50 | .endm | 60 | .endm |
| 51 | 61 | ||
| 52 | .section .init.text, "ax",@progbits | 62 | .section .init.text, "ax",@progbits |
| @@ -90,15 +100,13 @@ ENTRY(stext) · 37 unmodified lines | |||
| 90 | st.ab 0, [r5, 4] | 100 | st.ab 0, [r5, 4] |
| 91 | 1: | 101 | 1: |
| 92 | 102 | ||
| 93 | #ifdef CONFIG_ARC_UBOOT_SUPPORT | ||
| 94 | ; Uboot - kernel ABI | 103 | ; Uboot - kernel ABI |
| 95 | ; r0 = [0] No uboot interaction, [1] cmdline in r2, [2] DTB in r2 | 104 | ; r0 = [0] No uboot interaction, [1] cmdline in r2, [2] DTB in r2 |
| 96 | ; r1 = magic number (board identity, unused as of now | 105 | ; r1 = magic number (always zero as of now) |
| 97 | ; r2 = pointer to uboot provided cmdline or external DTB in mem | 106 | ; r2 = pointer to uboot provided cmdline or external DTB in mem |
| 98 | ; These are handled later in setup_arch() | 107 | ; These are handled later in handle_uboot_args() |
| 99 | st r0, [@uboot_tag] | 108 | st r0, [@uboot_tag] |
| 100 | st r2, [@uboot_arg] | 109 | st r2, [@uboot_arg] |
| 101 | #endif | ||
| 102 | 110 | ||
| 103 | ; setup "current" tsk and optionally cache it in dedicated r25 | 111 | ; setup "current" tsk and optionally cache it in dedicated r25 |
| 104 | mov r9, @init_task | 112 | mov r9, @init_task |
| @@ -49,11 +49,13 @@ void arc_init_IRQ(void) · 48 unmodified lines | |||
| 49 | 49 | ||
| 50 | *(unsigned int *)&ictrl = 0; | 50 | *(unsigned int *)&ictrl = 0; |
| 51 | 51 | ||
| 52 | #ifndef CONFIG_ARC_IRQ_NO_AUTOSAVE | ||
| 52 | ictrl.save_nr_gpr_pairs = 6; /* r0 to r11 (r12 saved manually) */ | 53 | ictrl.save_nr_gpr_pairs = 6; /* r0 to r11 (r12 saved manually) */ |
| 53 | ictrl.save_blink = 1; | 54 | ictrl.save_blink = 1; |
| 54 | ictrl.save_lp_regs = 1; /* LP_COUNT, LP_START, LP_END */ | 55 | ictrl.save_lp_regs = 1; /* LP_COUNT, LP_START, LP_END */ |
| 55 | ictrl.save_u_to_u = 0; /* user ctxt saved on kernel stack */ | 56 | ictrl.save_u_to_u = 0; /* user ctxt saved on kernel stack */ |
| 56 | ictrl.save_idx_regs = 1; /* JLI, LDI, EI */ | 57 | ictrl.save_idx_regs = 1; /* JLI, LDI, EI */ |
| 58 | #endif | ||
| 57 | 59 | ||
| 58 | WRITE_AUX(AUX_IRQ_CTRL, ictrl); | 60 | WRITE_AUX(AUX_IRQ_CTRL, ictrl); |
| 59 | 61 | ||
| @@ -199,20 +199,36 @@ static void read_arc_build_cfg_regs(void) · 198 unmodified lines | |||
| 199 | cpu->bpu.ret_stk = 4 << bpu.rse; | 199 | cpu->bpu.ret_stk = 4 << bpu.rse; |
| 200 | 200 | ||
| 201 | if (cpu->core.family >= 0x54) { | 201 | if (cpu->core.family >= 0x54) { |
| 202 | unsigned int exec_ctrl; | ||
| 203 | 202 | ||
| 204 | READ_BCR(AUX_EXEC_CTRL, exec_ctrl); | 203 | struct bcr_uarch_build_arcv2 uarch; |
| 205 | cpu->extn.dual_enb = !(exec_ctrl & 1); | ||
| 206 | 204 | ||
| 207 | /* dual issue always present for this core */ | 205 | /* |
| 208 | cpu->extn.dual = 1; | 206 | * The first 0x54 core (uarch maj:min 0:1 or 0:2) was |
| 207 | * dual issue only (HS4x). But next uarch rev (1:0) | ||
| 208 | * allows it be configured for single issue (HS3x) | ||
| 209 | * Ensure we fiddle with dual issue only on HS4x | ||
| 210 | */ | ||
| 211 | READ_BCR(ARC_REG_MICRO_ARCH_BCR, uarch); | ||
| 212 | |||
| 213 | if (uarch.prod == 4) { | ||
| 214 | unsigned int exec_ctrl; | ||
| 215 | |||
| 216 | /* dual issue hardware always present */ | ||
| 217 | cpu->extn.dual = 1; | ||
| 218 | |||
| 219 | READ_BCR(AUX_EXEC_CTRL, exec_ctrl); | ||
| 220 | |||
| 221 | /* dual issue hardware enabled ? */ | ||
| 222 | cpu->extn.dual_enb = !(exec_ctrl & 1); | ||
| 223 | |||
| 224 | } | ||
| 209 | } | 225 | } |
| 210 | } | 226 | } |
| 211 | 227 | ||
| 212 | READ_BCR(ARC_REG_AP_BCR, ap); | 228 | READ_BCR(ARC_REG_AP_BCR, ap); |
| 213 | if (ap.ver) { | 229 | if (ap.ver) { |
| 214 | cpu->extn.ap_num = 2 << ap.num; | 230 | cpu->extn.ap_num = 2 << ap.num; |
| 215 | cpu->extn.ap_full = !!ap.min; | 231 | cpu->extn.ap_full = !ap.min; |
| 216 | } | 232 | } |
| 217 | 233 | ||
| 218 | READ_BCR(ARC_REG_SMART_BCR, bcr); | 234 | READ_BCR(ARC_REG_SMART_BCR, bcr); |
| @@ -462,43 +478,78 @@ void setup_processor(void) · 243 unmodified lines | |||
| 462 | arc_chk_core_config(); | 478 | arc_chk_core_config(); |
| 463 | } | 479 | } |
| 464 | 480 | ||
| 465 | static inline int is_kernel(unsigned long addr) | 481 | static inline bool uboot_arg_invalid(unsigned long addr) |
| 466 | { | 482 | { |
| 467 | if (addr >= (unsigned long)_stext && addr <= (unsigned long)_end) | 483 | /* |
| 468 | return 1; | 484 | * Check that it is a untranslated address (although MMU is not enabled |
| 469 | return 0; | 485 | * yet, it being a high address ensures this is not by fluke) |
| 486 | */ | ||
| 487 | if (addr < PAGE_OFFSET) | ||
| 488 | return true; | ||
| 489 | |||
| 490 | /* Check that address doesn't clobber resident kernel image */ | ||
| 491 | return addr >= (unsigned long)_stext && addr <= (unsigned long)_end; | ||
| 470 | } | 492 | } |
| 471 | 493 | ||
| 472 | void __init setup_arch(char **cmdline_p) | 494 | #define IGNORE_ARGS "Ignore U-boot args: " |
| 495 | |||
| 496 | /* uboot_tag values for U-boot - kernel ABI revision 0; see head.S */ | ||
| 497 | #define UBOOT_TAG_NONE 0 | ||
| 498 | #define UBOOT_TAG_CMDLINE 1 | ||
| 499 | #define UBOOT_TAG_DTB 2 | ||
| 500 | |||
| 501 | void __init handle_uboot_args(void) | ||
| 473 | { | 502 | { |
| 474 | #ifdef CONFIG_ARC_UBOOT_SUPPORT | 503 | bool use_embedded_dtb = true; |
| 475 | /* make sure that uboot passed pointer to cmdline/dtb is valid */ | 504 | bool append_cmdline = false; |
| 476 | if (uboot_tag && is_kernel((unsigned long)uboot_arg)) | 505 | |
| 477 | panic("Invalid uboot arg\n"); | 506 | /* check that we know this tag */ |
| 478 | 507 | if (uboot_tag != UBOOT_TAG_NONE && | |
| 479 | /* See if u-boot passed an external Device Tree blob */ | 508 | uboot_tag != UBOOT_TAG_CMDLINE && |
| 480 | machine_desc = setup_machine_fdt(uboot_arg); /* uboot_tag == 2 */ | 509 | uboot_tag != UBOOT_TAG_DTB) { |
| 481 | if (!machine_desc) | 510 | pr_warn(IGNORE_ARGS "invalid uboot tag: '%08x'\n", uboot_tag); |
| 482 | #endif | 511 | goto ignore_uboot_args; |
| 483 | { | 512 | } |
| 484 | /* No, so try the embedded one */ | 513 | |
| 514 | if (uboot_tag != UBOOT_TAG_NONE && | ||
| 515 | uboot_arg_invalid((unsigned long)uboot_arg)) { | ||
| 516 | pr_warn(IGNORE_ARGS "invalid uboot arg: '%px'\n", uboot_arg); | ||
| 517 | goto ignore_uboot_args; | ||
| 518 | } | ||
| 519 | |||
| 520 | /* see if U-boot passed an external Device Tree blob */ | ||
| 521 | if (uboot_tag == UBOOT_TAG_DTB) { | ||
| 522 | machine_desc = setup_machine_fdt((void *)uboot_arg); | ||
| 523 | |||
| 524 | /* external Device Tree blob is invalid - use embedded one */ | ||
| 525 | use_embedded_dtb = !machine_desc; | ||
| 526 | } | ||
| 527 | |||
| 528 | if (uboot_tag == UBOOT_TAG_CMDLINE) | ||
| 529 | append_cmdline = true; | ||
| 530 | |||
| 531 | ignore_uboot_args: | ||
| 532 | |||
| 533 | if (use_embedded_dtb) { | ||
| 485 | machine_desc = setup_machine_fdt(__dtb_start); | 534 | machine_desc = setup_machine_fdt(__dtb_start); |
| 486 | if (!machine_desc) | 535 | if (!machine_desc) |
| 487 | panic("Embedded DT invalid\n"); | 536 | panic("Embedded DT invalid\n"); |
| 537 | } | ||
| 488 | 538 | ||
| 489 | /* | 539 | /* |
| 490 | * If we are here, it is established that @uboot_arg didn't | 540 | * NOTE: @boot_command_line is populated by setup_machine_fdt() so this |
| 491 | * point to DT blob. Instead if u-boot says it is cmdline, | 541 | * append processing can only happen after. |
| 492 | * append to embedded DT cmdline. | 542 | */ |
| 493 | * setup_machine_fdt() would have populated @boot_command_line | 543 | if (append_cmdline) { |
| 494 | */ | 544 | /* Ensure a whitespace between the 2 cmdlines */ |
| 495 | if (uboot_tag == 1) { | 545 | strlcat(boot_command_line, " ", COMMAND_LINE_SIZE); |
| 496 | /* Ensure a whitespace between the 2 cmdlines */ | 546 | strlcat(boot_command_line, uboot_arg, COMMAND_LINE_SIZE); |
| 497 | strlcat(boot_command_line, " ", COMMAND_LINE_SIZE); | ||
| 498 | strlcat(boot_command_line, uboot_arg, | ||
| 499 | COMMAND_LINE_SIZE); | ||
| 500 | } | ||
| 501 | } | 547 | } |
| 548 | } | ||
| 549 | |||
| 550 | void __init setup_arch(char **cmdline_p) | ||
| 551 | { | ||
| 552 | handle_uboot_args(); | ||
| 502 | 553 | ||
| 503 | /* Save unparsed command line copy for /proc/cmdline */ | 554 | /* Save unparsed command line copy for /proc/cmdline */ |
| 504 | *cmdline_p = boot_command_line; | 555 | *cmdline_p = boot_command_line; |
| @@ -25,15 +25,11 @@ · 24 unmodified lines | |||
| 25 | #endif | 25 | #endif |
| 26 | 26 | ||
| 27 | #ifdef CONFIG_ARC_HAS_LL64 | 27 | #ifdef CONFIG_ARC_HAS_LL64 |
| 28 | # define PREFETCH_READ(RX) prefetch [RX, 56] | ||
| 29 | # define PREFETCH_WRITE(RX) prefetchw [RX, 64] | ||
| 30 | # define LOADX(DST,RX) ldd.ab DST, [RX, 8] | 28 | # define LOADX(DST,RX) ldd.ab DST, [RX, 8] |
| 31 | # define STOREX(SRC,RX) std.ab SRC, [RX, 8] | 29 | # define STOREX(SRC,RX) std.ab SRC, [RX, 8] |
| 32 | # define ZOLSHFT 5 | 30 | # define ZOLSHFT 5 |
| 33 | # define ZOLAND 0x1F | 31 | # define ZOLAND 0x1F |
| 34 | #else | 32 | #else |
| 35 | # define PREFETCH_READ(RX) prefetch [RX, 28] | ||
| 36 | # define PREFETCH_WRITE(RX) prefetchw [RX, 32] | ||
| 37 | # define LOADX(DST,RX) ld.ab DST, [RX, 4] | 33 | # define LOADX(DST,RX) ld.ab DST, [RX, 4] |
| 38 | # define STOREX(SRC,RX) st.ab SRC, [RX, 4] | 34 | # define STOREX(SRC,RX) st.ab SRC, [RX, 4] |
| 39 | # define ZOLSHFT 4 | 35 | # define ZOLSHFT 4 |
| @@ -41,8 +37,6 @@ · 1 unmodified line | |||
| 41 | #endif | 37 | #endif |
| 42 | 38 | ||
| 43 | ENTRY_CFI(memcpy) | 39 | ENTRY_CFI(memcpy) |
| 44 | prefetch [r1] ; Prefetch the read location | ||
| 45 | prefetchw [r0] ; Prefetch the write location | ||
| 46 | mov.f 0, r2 | 40 | mov.f 0, r2 |
| 47 | ;;; if size is zero | 41 | ;;; if size is zero |
| 48 | jz.d [blink] | 42 | jz.d [blink] |
| @@ -72,8 +66,6 @@ ENTRY_CFI(memcpy) · 23 unmodified lines | |||
| 72 | lpnz @.Lcopy32_64bytes | 66 | lpnz @.Lcopy32_64bytes |
| 73 | ;; LOOP START | 67 | ;; LOOP START |
| 74 | LOADX (r6, r1) | 68 | LOADX (r6, r1) |
| 75 | PREFETCH_READ (r1) | ||
| 76 | PREFETCH_WRITE (r3) | ||
| 77 | LOADX (r8, r1) | 69 | LOADX (r8, r1) |
| 78 | LOADX (r10, r1) | 70 | LOADX (r10, r1) |
| 79 | LOADX (r4, r1) | 71 | LOADX (r4, r1) |
| @@ -117,9 +109,7 @@ ENTRY_CFI(memcpy) · 37 unmodified lines | |||
| 117 | lpnz @.Lcopy8bytes_1 | 109 | lpnz @.Lcopy8bytes_1 |
| 118 | ;; LOOP START | 110 | ;; LOOP START |
| 119 | ld.ab r6, [r1, 4] | 111 | ld.ab r6, [r1, 4] |
| 120 | prefetch [r1, 28] ;Prefetch the next read location | ||
| 121 | ld.ab r8, [r1,4] | 112 | ld.ab r8, [r1,4] |
| 122 | prefetchw [r3, 32] ;Prefetch the next write location | ||
| 123 | 113 | ||
| 124 | SHIFT_1 (r7, r6, 24) | 114 | SHIFT_1 (r7, r6, 24) |
| 125 | or r7, r7, r5 | 115 | or r7, r7, r5 |
| @@ -162,9 +152,7 @@ ENTRY_CFI(memcpy) · 36 unmodified lines | |||
| 162 | lpnz @.Lcopy8bytes_2 | 152 | lpnz @.Lcopy8bytes_2 |
| 163 | ;; LOOP START | 153 | ;; LOOP START |
| 164 | ld.ab r6, [r1, 4] | 154 | ld.ab r6, [r1, 4] |
| 165 | prefetch [r1, 28] ;Prefetch the next read location | ||
| 166 | ld.ab r8, [r1,4] | 155 | ld.ab r8, [r1,4] |
| 167 | prefetchw [r3, 32] ;Prefetch the next write location | ||
| 168 | 156 | ||
| 169 | SHIFT_1 (r7, r6, 16) | 157 | SHIFT_1 (r7, r6, 16) |
| 170 | or r7, r7, r5 | 158 | or r7, r7, r5 |
| @@ -204,9 +192,7 @@ ENTRY_CFI(memcpy) · 33 unmodified lines | |||
| 204 | lpnz @.Lcopy8bytes_3 | 192 | lpnz @.Lcopy8bytes_3 |
| 205 | ;; LOOP START | 193 | ;; LOOP START |
| 206 | ld.ab r6, [r1, 4] | 194 | ld.ab r6, [r1, 4] |
| 207 | prefetch [r1, 28] ;Prefetch the next read location | ||
| 208 | ld.ab r8, [r1,4] | 195 | ld.ab r8, [r1,4] |
| 209 | prefetchw [r3, 32] ;Prefetch the next write location | ||
| 210 | 196 | ||
| 211 | SHIFT_1 (r7, r6, 8) | 197 | SHIFT_1 (r7, r6, 8) |
| 212 | or r7, r7, r5 | 198 | or r7, r7, r5 |
| @@ -9,6 +9,7 @@ menuconfig ARC_SOC_HSDK · 8 unmodified lines | |||
| 9 | bool "ARC HS Development Kit SOC" | 9 | bool "ARC HS Development Kit SOC" |
| 10 | depends on ISA_ARCV2 | 10 | depends on ISA_ARCV2 |
| 11 | select ARC_HAS_ACCL_REGS | 11 | select ARC_HAS_ACCL_REGS |
| 12 | select ARC_IRQ_NO_AUTOSAVE | ||
| 12 | select CLK_HSDK | 13 | select CLK_HSDK |
| 13 | select RESET_HSDK | 14 | select RESET_HSDK |
| 14 | select HAVE_PCI | 15 | select HAVE_PCI |
| @@ -1400,6 +1400,7 @@ config NR_CPUS · 1399 unmodified lines | |||
| 1400 | config HOTPLUG_CPU | 1400 | config HOTPLUG_CPU |
| 1401 | bool "Support for hot-pluggable CPUs" | 1401 | bool "Support for hot-pluggable CPUs" |
| 1402 | depends on SMP | 1402 | depends on SMP |
| 1403 | select GENERIC_IRQ_MIGRATION | ||
| 1403 | help | 1404 | help |
| 1404 | Say Y here to experiment with turning CPUs off and on. CPUs | 1405 | Say Y here to experiment with turning CPUs off and on. CPUs |
| 1405 | can be controlled through /sys/devices/system/cpu. | 1406 | can be controlled through /sys/devices/system/cpu. |
| @@ -729,7 +729,7 @@ · 728 unmodified lines | |||
| 729 | 729 | ||
| 730 | &cpsw_emac0 { | 730 | &cpsw_emac0 { |
| 731 | phy-handle = <ðphy0>; | 731 | phy-handle = <ðphy0>; |
| 732 | phy-mode = "rgmii-txid"; | 732 | phy-mode = "rgmii-id"; |
| 733 | }; | 733 | }; |
| 734 | 734 | ||
| 735 | &tscadc { | 735 | &tscadc { |
| @@ -651,13 +651,13 @@ · 650 unmodified lines | |||
| 651 | 651 | ||
| 652 | &cpsw_emac0 { | 652 | &cpsw_emac0 { |
| 653 | phy-handle = <ðphy0>; | 653 | phy-handle = <ðphy0>; |
| 654 | phy-mode = "rgmii-txid"; | 654 | phy-mode = "rgmii-id"; |
| 655 | dual_emac_res_vlan = <1>; | 655 | dual_emac_res_vlan = <1>; |
| 656 | }; | 656 | }; |
| 657 | 657 | ||
| 658 | &cpsw_emac1 { | 658 | &cpsw_emac1 { |
| 659 | phy-handle = <ðphy1>; | 659 | phy-handle = <ðphy1>; |
| 660 | phy-mode = "rgmii-txid"; | 660 | phy-mode = "rgmii-id"; |
| 661 | dual_emac_res_vlan = <2>; | 661 | dual_emac_res_vlan = <2>; |
| 662 | }; | 662 | }; |
| 663 | 663 | ||
| @@ -144,30 +144,32 @@ · 143 unmodified lines | |||
| 144 | status = "okay"; | 144 | status = "okay"; |
| 145 | }; | 145 | }; |
| 146 | 146 | ||
| 147 | nand@d0000 { | 147 | nand-controller@d0000 { |
| 148 | status = "okay"; | 148 | status = "okay"; |
| 149 | label = "pxa3xx_nand-0"; | ||
| 150 | num-cs = <1>; | ||
| 151 | marvell,nand-keep-config; | ||
| 152 | nand-on-flash-bbt; | ||
| 153 | |||
| 154 | partitions { | ||
| 155 | compatible = "fixed-partitions"; | ||
| 156 | #address-cells = <1>; | ||
| 157 | #size-cells = <1>; | ||
| 158 | |||
| 159 | partition@0 { | ||
| 160 | label = "U-Boot"; | ||
| 161 | reg = <0 0x800000>; | ||
| 162 | }; | ||
| 163 | partition@800000 { | ||
| 164 | label = "Linux"; | ||
| 165 | reg = <0x800000 0x800000>; | ||
| 166 | }; | ||
| 167 | partition@1000000 { | ||
| 168 | label = "Filesystem"; | ||
| 169 | reg = <0x1000000 0x3f000000>; | ||
| 170 | 149 | ||
| 150 | nand@0 { | ||
| 151 | reg = <0>; | ||
| 152 | label = "pxa3xx_nand-0"; | ||
| 153 | nand-rb = <0>; | ||
| 154 | nand-on-flash-bbt; | ||
| 155 | |||
| 156 | partitions { | ||
| 157 | compatible = "fixed-partitions"; | ||
| 158 | #address-cells = <1>; | ||
| 159 | #size-cells = <1>; | ||
| 160 | |||
| 161 | partition@0 { | ||
| 162 | label = "U-Boot"; | ||
| 163 | reg = <0 0x800000>; | ||
| 164 | }; | ||
| 165 | partition@800000 { | ||
| 166 | label = "Linux"; | ||
| 167 | reg = <0x800000 0x800000>; | ||
| 168 | }; | ||
| 169 | partition@1000000 { | ||
| 170 | label = "Filesystem"; | ||
| 171 | reg = <0x1000000 0x3f000000>; | ||
| 172 | }; | ||
| 171 | }; | 173 | }; |
| 172 | }; | 174 | }; |
| 173 | }; | 175 | }; |
| @@ -160,12 +160,15 @@ · 159 unmodified lines | |||
| 160 | status = "okay"; | 160 | status = "okay"; |
| 161 | }; | 161 | }; |
| 162 | 162 | ||
| 163 | nand@d0000 { | 163 | nand-controller@d0000 { |
| 164 | status = "okay"; | 164 | status = "okay"; |
| 165 | label = "pxa3xx_nand-0"; | 165 | |
| 166 | num-cs = <1>; | 166 | nand@0 { |
| 167 | marvell,nand-keep-config; | 167 | reg = <0>; |
| 168 | nand-on-flash-bbt; | 168 | label = "pxa3xx_nand-0"; |
| 169 | nand-rb = <0>; | ||
| 170 | nand-on-flash-bbt; | ||
| 171 | }; | ||
| 169 | }; | 172 | }; |
| 170 | }; | 173 | }; |
| 171 | 174 | ||
| @@ -81,49 +81,52 @@ · 80 unmodified lines | |||
| 81 | 81 | ||
| 82 | }; | 82 | }; |
| 83 | 83 | ||
| 84 | nand@d0000 { | 84 | nand-controller@d0000 { |
| 85 | status = "okay"; | 85 | status = "okay"; |
| 86 | label = "pxa3xx_nand-0"; | ||
| 87 | num-cs = <1>; | ||
| 88 | marvell,nand-keep-config; | ||
| 89 | nand-on-flash-bbt; | ||
| 90 | |||
| 91 | partitions { | ||
| 92 | compatible = "fixed-partitions"; | ||
| 93 | #address-cells = <1>; | ||
| 94 | #size-cells = <1>; | ||
| 95 | |||
| 96 | partition@0 { | ||
| 97 | label = "u-boot"; | ||
| 98 | reg = <0x00000000 0x000e0000>; | ||
| 99 | read-only; | ||
| 100 | }; | ||
| 101 | |||
| 102 | partition@e0000 { | ||
| 103 | label = "u-boot-env"; | ||
| 104 | reg = <0x000e0000 0x00020000>; | ||
| 105 | read-only; | ||
| 106 | }; | ||
| 107 | |||
| 108 | partition@100000 { | ||
| 109 | label = "u-boot-env2"; | ||
| 110 | reg = <0x00100000 0x00020000>; | ||
| 111 | read-only; | ||
| 112 | }; | ||
| 113 | |||
| 114 | partition@120000 { | ||
| 115 | label = "zImage"; | ||
| 116 | reg = <0x00120000 0x00400000>; | ||
| 117 | }; | ||
| 118 | |||
| 119 | partition@520000 { | ||
| 120 | label = "initrd"; | ||
| 121 | reg = <0x00520000 0x00400000>; | ||
| 122 | }; | ||
| 123 | 86 | ||
| 124 | partition@e00000 { | 87 | nand@0 { |
| 125 | label = "boot"; | 88 | reg = <0>; |
| 126 | reg = <0x00e00000 0x3f200000>; | 89 | label = "pxa3xx_nand-0"; |
| 90 | nand-rb = <0>; | ||
| 91 | nand-on-flash-bbt; | ||
| 92 | |||
| 93 | partitions { | ||
| 94 | compatible = "fixed-partitions"; | ||
| 95 | #address-cells = <1>; | ||
| 96 | #size-cells = <1>; | ||
| 97 | |||
| 98 | partition@0 { | ||
| 99 | label = "u-boot"; | ||
| 100 | reg = <0x00000000 0x000e0000>; | ||
| 101 | read-only; | ||
| 102 | }; | ||
| 103 | |||
| 104 | partition@e0000 { | ||
| 105 | label = "u-boot-env"; | ||
| 106 | reg = <0x000e0000 0x00020000>; | ||
| 107 | read-only; | ||
| 108 | }; | ||
| 109 | |||
| 110 | partition@100000 { | ||
| 111 | label = "u-boot-env2"; | ||
| 112 | reg = <0x00100000 0x00020000>; | ||
| 113 | read-only; | ||
| 114 | }; | ||
| 115 | |||
| 116 | partition@120000 { | ||
| 117 | label = "zImage"; | ||
| 118 | reg = <0x00120000 0x00400000>; | ||
| 119 | }; | ||
| 120 | |||
| 121 | partition@520000 { | ||
| 122 | label = "initrd"; | ||
| 123 | reg = <0x00520000 0x00400000>; | ||
| 124 | }; | ||
| 125 | |||
| 126 | partition@e00000 { | ||
| 127 | label = "boot"; | ||
| 128 | reg = <0x00e00000 0x3f200000>; | ||
| 129 | }; | ||
| 127 | }; | 130 | }; |
| 128 | }; | 131 | }; |
| 129 | }; | 132 | }; |
| @@ -443,7 +443,7 @@ · 442 unmodified lines | |||
| 443 | }; | 443 | }; |
| 444 | 444 | ||
| 445 | display-controller@6a000000 { | 445 | display-controller@6a000000 { |
| 446 | status = "disabled"; | 446 | status = "okay"; |
| 447 | 447 | ||
| 448 | port@0 { | 448 | port@0 { |
| 449 | reg = <0>; | 449 | reg = <0>; |
| @@ -13,10 +13,25 @@ · 12 unmodified lines | |||
| 13 | stdout-path = "serial0:115200n8"; | 13 | stdout-path = "serial0:115200n8"; |
| 14 | }; | 14 | }; |
| 15 | 15 | ||
| 16 | memory@80000000 { | 16 | /* |
| 17 | * Note that recent version of the device tree compiler (starting with | ||
| 18 | * version 1.4.2) warn about this node containing a reg property, but | ||
| 19 | * missing a unit-address. However, the bootloader on these Chromebook | ||
| 20 | * devices relies on the full name of this node to be exactly /memory. | ||
| 21 | * Adding the unit-address causes the bootloader to create a /memory | ||
| 22 | * node and write the memory bank configuration to that node, which in | ||
| 23 | * turn leads the kernel to believe that the device has 2 GiB of | ||
| 24 | * memory instead of the amount detected by the bootloader. | ||
| 25 | * | ||
| 26 | * The name of this node is effectively ABI and must not be changed. | ||
| 27 | */ | ||
| 28 | memory { | ||
| 29 | device_type = "memory"; | ||
| 17 | reg = <0x0 0x80000000 0x0 0x80000000>; | 30 | reg = <0x0 0x80000000 0x0 0x80000000>; |
| 18 | }; | 31 | }; |
| 19 | 32 | ||
| 33 | /delete-node/ memory@80000000; | ||
| 34 | |||
| 20 | host1x@50000000 { | 35 | host1x@50000000 { |
| 21 | hdmi@54280000 { | 36 | hdmi@54280000 { |
| 22 | status = "okay"; | 37 | status = "okay"; |
| @@ -212,10 +212,11 @@ K256: · 211 unmodified lines | |||
| 212 | .global sha256_block_data_order | 212 | .global sha256_block_data_order |
| 213 | .type sha256_block_data_order,%function | 213 | .type sha256_block_data_order,%function |
| 214 | sha256_block_data_order: | 214 | sha256_block_data_order: |
| 215 | .Lsha256_block_data_order: | ||
| 215 | #if __ARM_ARCH__<7 | 216 | #if __ARM_ARCH__<7 |
| 216 | sub r3,pc,#8 @ sha256_block_data_order | 217 | sub r3,pc,#8 @ sha256_block_data_order |
| 217 | #else | 218 | #else |
| 218 | adr r3,sha256_block_data_order | 219 | adr r3,.Lsha256_block_data_order |
| 219 | #endif | 220 | #endif |
| 220 | #if __ARM_MAX_ARCH__>=7 && !defined(__KERNEL__) | 221 | #if __ARM_MAX_ARCH__>=7 && !defined(__KERNEL__) |
| 221 | ldr r12,.LOPENSSL_armcap | 222 | ldr r12,.LOPENSSL_armcap |
| @@ -93,10 +93,11 @@ K256: · 92 unmodified lines | |||
| 93 | .global sha256_block_data_order | 93 | .global sha256_block_data_order |
| 94 | .type sha256_block_data_order,%function | 94 | .type sha256_block_data_order,%function |
| 95 | sha256_block_data_order: | 95 | sha256_block_data_order: |
| 96 | .Lsha256_block_data_order: | ||
| 96 | #if __ARM_ARCH__<7 | 97 | #if __ARM_ARCH__<7 |
| 97 | sub r3,pc,#8 @ sha256_block_data_order | 98 | sub r3,pc,#8 @ sha256_block_data_order |
| 98 | #else | 99 | #else |
| 99 | adr r3,sha256_block_data_order | 100 | adr r3,.Lsha256_block_data_order |
| 100 | #endif | 101 | #endif |
| 101 | #if __ARM_MAX_ARCH__>=7 && !defined(__KERNEL__) | 102 | #if __ARM_MAX_ARCH__>=7 && !defined(__KERNEL__) |
| 102 | ldr r12,.LOPENSSL_armcap | 103 | ldr r12,.LOPENSSL_armcap |
| @@ -274,10 +274,11 @@ WORD64(0x5fcb6fab,0x3ad6faec, 0x6c44198c,0x4a475817) · 273 unmodified lines | |||
| 274 | .global sha512_block_data_order | 274 | .global sha512_block_data_order |
| 275 | .type sha512_block_data_order,%function | 275 | .type sha512_block_data_order,%function |
| 276 | sha512_block_data_order: | 276 | sha512_block_data_order: |
| 277 | .Lsha512_block_data_order: | ||
| 277 | #if __ARM_ARCH__<7 | 278 | #if __ARM_ARCH__<7 |
| 278 | sub r3,pc,#8 @ sha512_block_data_order | 279 | sub r3,pc,#8 @ sha512_block_data_order |
| 279 | #else | 280 | #else |
| 280 | adr r3,sha512_block_data_order | 281 | adr r3,.Lsha512_block_data_order |
| 281 | #endif | 282 | #endif |
| 282 | #if __ARM_MAX_ARCH__>=7 && !defined(__KERNEL__) | 283 | #if __ARM_MAX_ARCH__>=7 && !defined(__KERNEL__) |
| 283 | ldr r12,.LOPENSSL_armcap | 284 | ldr r12,.LOPENSSL_armcap |
| @@ -141,10 +141,11 @@ WORD64(0x5fcb6fab,0x3ad6faec, 0x6c44198c,0x4a475817) · 140 unmodified lines | |||
| 141 | .global sha512_block_data_order | 141 | .global sha512_block_data_order |
| 142 | .type sha512_block_data_order,%function | 142 | .type sha512_block_data_order,%function |
| 143 | sha512_block_data_order: | 143 | sha512_block_data_order: |
| 144 | .Lsha512_block_data_order: | ||
| 144 | #if __ARM_ARCH__<7 | 145 | #if __ARM_ARCH__<7 |
| 145 | sub r3,pc,#8 @ sha512_block_data_order | 146 | sub r3,pc,#8 @ sha512_block_data_order |
| 146 | #else | 147 | #else |
| 147 | adr r3,sha512_block_data_order | 148 | adr r3,.Lsha512_block_data_order |
| 148 | #endif | 149 | #endif |
| 149 | #if __ARM_MAX_ARCH__>=7 && !defined(__KERNEL__) | 150 | #if __ARM_MAX_ARCH__>=7 && !defined(__KERNEL__) |
| 150 | ldr r12,.LOPENSSL_armcap | 151 | ldr r12,.LOPENSSL_armcap |
| @@ -25,7 +25,6 @@ · 24 unmodified lines | |||
| 25 | #ifndef __ASSEMBLY__ | 25 | #ifndef __ASSEMBLY__ |
| 26 | struct irqaction; | 26 | struct irqaction; |
| 27 | struct pt_regs; | 27 | struct pt_regs; |
| 28 | extern void migrate_irqs(void); | ||
| 29 | 28 | ||
| 30 | extern void asm_do_IRQ(unsigned int, struct pt_regs *); | 29 | extern void asm_do_IRQ(unsigned int, struct pt_regs *); |
| 31 | void handle_IRQ(unsigned int, struct pt_regs *); | 30 | void handle_IRQ(unsigned int, struct pt_regs *); |
| @@ -31,7 +31,6 @@ · 30 unmodified lines | |||
| 31 | #include <linux/smp.h> | 31 | #include <linux/smp.h> |
| 32 | #include <linux/init.h> | 32 | #include <linux/init.h> |
| 33 | #include <linux/seq_file.h> | 33 | #include <linux/seq_file.h> |
| 34 | #include <linux/ratelimit.h> | ||
| 35 | #include <linux/errno.h> | 34 | #include <linux/errno.h> |
| 36 | #include <linux/list.h> | 35 | #include <linux/list.h> |
| 37 | #include <linux/kallsyms.h> | 36 | #include <linux/kallsyms.h> |
| @@ -109,64 +108,3 @@ int __init arch_probe_nr_irqs(void) · 71 unmodified lines | |||
| 109 | return nr_irqs; | 108 | return nr_irqs; |
| 110 | } | 109 | } |
| 111 | #endif | 110 | #endif |
| 112 | |||
| 113 | #ifdef CONFIG_HOTPLUG_CPU | ||
| 114 | static bool migrate_one_irq(struct irq_desc *desc) | ||
| 115 | { | ||
| 116 | struct irq_data *d = irq_desc_get_irq_data(desc); | ||
| 117 | const struct cpumask *affinity = irq_data_get_affinity_mask(d); | ||
| 118 | struct irq_chip *c; | ||
| 119 | bool ret = false; | ||
| 120 | |||
| 121 | /* | ||
| 122 | * If this is a per-CPU interrupt, or the affinity does not | ||
| 123 | * include this CPU, then we have nothing to do. | ||
| 124 | */ | ||
| 125 | if (irqd_is_per_cpu(d) || !cpumask_test_cpu(smp_processor_id(), affinity)) | ||
| 126 | return false; | ||
| 127 | |||
| 128 | if (cpumask_any_and(affinity, cpu_online_mask) >= nr_cpu_ids) { | ||
| 129 | affinity = cpu_online_mask; | ||
| 130 | ret = true; | ||
| 131 | } | ||
| 132 | |||
| 133 | c = irq_data_get_irq_chip(d); | ||
| 134 | if (!c->irq_set_affinity) | ||
| 135 | pr_debug("IRQ%u: unable to set affinity\n", d->irq); | ||
| 136 | else if (c->irq_set_affinity(d, affinity, false) == IRQ_SET_MASK_OK && ret) | ||
| 137 | cpumask_copy(irq_data_get_affinity_mask(d), affinity); | ||
| 138 | |||
| 139 | return ret; | ||
| 140 | } | ||
| 141 | |||
| 142 | /* | ||
| 143 | * The current CPU has been marked offline. Migrate IRQs off this CPU. | ||
| 144 | * If the affinity settings do not allow other CPUs, force them onto any | ||
| 145 | * available CPU. | ||
| 146 | * | ||
| 147 | * Note: we must iterate over all IRQs, whether they have an attached | ||
| 148 | * action structure or not, as we need to get chained interrupts too. | ||
| 149 | */ | ||
| 150 | void migrate_irqs(void) | ||
| 151 | { | ||
| 152 | unsigned int i; | ||
| 153 | struct irq_desc *desc; | ||
| 154 | unsigned long flags; | ||
| 155 | |||
| 156 | local_irq_save(flags); | ||
| 157 | |||
| 158 | for_each_irq_desc(i, desc) { | ||
| 159 | bool affinity_broken; | ||
| 160 | |||
| 161 | raw_spin_lock(&desc->lock); | ||
| 162 | affinity_broken = migrate_one_irq(desc); | ||
| 163 | raw_spin_unlock(&desc->lock); | ||
| 164 | |||
| 165 | if (affinity_broken) | ||
| 166 | pr_warn_ratelimited("IRQ%u no longer affine to CPU%u\n", | ||
| 167 | i, smp_processor_id()); | ||
| 168 | } | ||
| 169 | |||
| 170 | local_irq_restore(flags); | ||
| 171 | } | ||
| 172 | #endif /* CONFIG_HOTPLUG_CPU */ | ||
| @@ -254,7 +254,7 @@ int __cpu_disable(void) · 253 unmodified lines | |||
| 254 | /* | 254 | /* |
| 255 | * OK - migrate IRQs away from this CPU | 255 | * OK - migrate IRQs away from this CPU |
| 256 | */ | 256 | */ |
| 257 | migrate_irqs(); | 257 | irq_migrate_all_off_this_cpu(); |
| 258 | 258 | ||
| 259 | /* | 259 | /* |
| 260 | * Flush user cache and TLB mappings, and then remove this CPU | 260 | * Flush user cache and TLB mappings, and then remove this CPU |
| @@ -2390,4 +2390,6 @@ void arch_teardown_dma_ops(struct device *dev) · 2389 unmodified lines | |||
| 2390 | return; | 2390 | return; |
| 2391 | 2391 | ||
| 2392 | arm_teardown_iommu_dma_ops(dev); | 2392 | arm_teardown_iommu_dma_ops(dev); |
| 2393 | /* Let arch_setup_dma_ops() start again from scratch upon re-probe */ | ||
| 2394 | set_dma_ops(dev, NULL); | ||
| 2393 | } | 2395 | } |
| @@ -247,7 +247,7 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct kprobe *or · 246 unmodified lines | |||
| 247 | } | 247 | } |
| 248 | 248 | ||
| 249 | /* Copy arch-dep-instance from template. */ | 249 | /* Copy arch-dep-instance from template. */ |
| 250 | memcpy(code, (unsigned char *)optprobe_template_entry, | 250 | memcpy(code, (unsigned long *)&optprobe_template_entry, |
| 251 | TMPL_END_IDX * sizeof(kprobe_opcode_t)); | 251 | TMPL_END_IDX * sizeof(kprobe_opcode_t)); |
| 252 | 252 | ||
| 253 | /* Adjust buffer according to instruction. */ | 253 | /* Adjust buffer according to instruction. */ |
| @@ -351,7 +351,7 @@ · 350 unmodified lines | |||
| 351 | reg = <0>; | 351 | reg = <0>; |
| 352 | pinctrl-names = "default"; | 352 | pinctrl-names = "default"; |
| 353 | pinctrl-0 = <&cp0_copper_eth_phy_reset>; | 353 | pinctrl-0 = <&cp0_copper_eth_phy_reset>; |
| 354 | reset-gpios = <&cp1_gpio1 11 GPIO_ACTIVE_LOW>; | 354 | reset-gpios = <&cp0_gpio2 11 GPIO_ACTIVE_LOW>; |
| 355 | reset-assert-us = <10000>; | 355 | reset-assert-us = <10000>; |
| 356 | }; | 356 | }; |
| 357 | 357 | ||
| @@ -37,7 +37,7 @@ · 36 unmodified lines | |||
| 37 | }; | 37 | }; |
| 38 | 38 | ||
| 39 | memory@86200000 { | 39 | memory@86200000 { |
| 40 | reg = <0x0 0x86200000 0x0 0x2600000>; | 40 | reg = <0x0 0x86200000 0x0 0x2d00000>; |
| 41 | no-map; | 41 | no-map; |
| 42 | }; | 42 | }; |
| 43 | 43 | ||
| @@ -158,8 +158,8 @@ ENTRY(hchacha_block_neon) · 157 unmodified lines | |||
| 158 | mov w3, w2 | 158 | mov w3, w2 |
| 159 | bl chacha_permute | 159 | bl chacha_permute |
| 160 | 160 | ||
| 161 | st1 {v0.16b}, [x1], #16 | 161 | st1 {v0.4s}, [x1], #16 |
| 162 | st1 {v3.16b}, [x1] | 162 | st1 {v3.4s}, [x1] |
| 163 | 163 | ||
| 164 | ldp x29, x30, [sp], #16 | 164 | ldp x29, x30, [sp], #16 |
| 165 | ret | 165 | ret |
| @@ -532,6 +532,10 @@ ENTRY(chacha_4block_xor_neon) · 366 unmodified lines | |||
| 532 | add v3.4s, v3.4s, v19.4s | 532 | add v3.4s, v3.4s, v19.4s |
| 533 | add a2, a2, w8 | 533 | add a2, a2, w8 |
| 534 | add a3, a3, w9 | 534 | add a3, a3, w9 |
| 535 | CPU_BE( rev a0, a0 ) | ||
| 536 | CPU_BE( rev a1, a1 ) | ||
| 537 | CPU_BE( rev a2, a2 ) | ||
| 538 | CPU_BE( rev a3, a3 ) | ||
| 535 | 539 | ||
| 536 | ld4r {v24.4s-v27.4s}, [x0], #16 | 540 | ld4r {v24.4s-v27.4s}, [x0], #16 |
| 537 | ld4r {v28.4s-v31.4s}, [x0] | 541 | ld4r {v28.4s-v31.4s}, [x0] |
| @@ -552,6 +556,10 @@ ENTRY(chacha_4block_xor_neon) · 14 unmodified lines | |||
| 552 | add v7.4s, v7.4s, v23.4s | 556 | add v7.4s, v7.4s, v23.4s |
| 553 | add a6, a6, w8 | 557 | add a6, a6, w8 |
| 554 | add a7, a7, w9 | 558 | add a7, a7, w9 |
| 559 | CPU_BE( rev a4, a4 ) | ||
| 560 | CPU_BE( rev a5, a5 ) | ||
| 561 | CPU_BE( rev a6, a6 ) | ||
| 562 | CPU_BE( rev a7, a7 ) | ||
| 555 | 563 | ||
| 556 | // x8[0-3] += s2[0] | 564 | // x8[0-3] += s2[0] |
| 557 | // x9[0-3] += s2[1] | 565 | // x9[0-3] += s2[1] |
| @@ -569,6 +577,10 @@ ENTRY(chacha_4block_xor_neon) · 11 unmodified lines | |||
| 569 | add v11.4s, v11.4s, v27.4s | 577 | add v11.4s, v11.4s, v27.4s |
| 570 | add a10, a10, w8 | 578 | add a10, a10, w8 |
| 571 | add a11, a11, w9 | 579 | add a11, a11, w9 |
| 580 | CPU_BE( rev a8, a8 ) | ||
| 581 | CPU_BE( rev a9, a9 ) | ||
| 582 | CPU_BE( rev a10, a10 ) | ||
| 583 | CPU_BE( rev a11, a11 ) | ||
| 572 | 584 | ||
| 573 | // x12[0-3] += s3[0] | 585 | // x12[0-3] += s3[0] |
| 574 | // x13[0-3] += s3[1] | 586 | // x13[0-3] += s3[1] |
| @@ -586,6 +598,10 @@ ENTRY(chacha_4block_xor_neon) · 11 unmodified lines | |||
| 586 | add v15.4s, v15.4s, v31.4s | 598 | add v15.4s, v15.4s, v31.4s |
| 587 | add a14, a14, w8 | 599 | add a14, a14, w8 |
| 588 | add a15, a15, w9 | 600 | add a15, a15, w9 |
| 601 | CPU_BE( rev a12, a12 ) | ||
| 602 | CPU_BE( rev a13, a13 ) | ||
| 603 | CPU_BE( rev a14, a14 ) | ||
| 604 | CPU_BE( rev a15, a15 ) | ||
| 589 | 605 | ||
| 590 | // interleave 32-bit words in state n, n+1 | 606 | // interleave 32-bit words in state n, n+1 |
| 591 | ldp w6, w7, [x2], #64 | 607 | ldp w6, w7, [x2], #64 |
| @@ -36,4 +36,8 @@ · 35 unmodified lines | |||
| 36 | #include <arm_neon.h> | 36 | #include <arm_neon.h> |
| 37 | #endif | 37 | #endif |
| 38 | 38 | ||
| 39 | #ifdef CONFIG_CC_IS_CLANG | ||
| 40 | #pragma clang diagnostic ignored "-Wincompatible-pointer-types" | ||
| 41 | #endif | ||
| 42 | |||
| 39 | #endif /* __ASM_NEON_INTRINSICS_H */ | 43 | #endif /* __ASM_NEON_INTRINSICS_H */ |
| @@ -539,8 +539,7 @@ set_hcr: · 538 unmodified lines | |||
| 539 | /* GICv3 system register access */ | 539 | /* GICv3 system register access */ |
| 540 | mrs x0, id_aa64pfr0_el1 | 540 | mrs x0, id_aa64pfr0_el1 |
| 541 | ubfx x0, x0, #24, #4 | 541 | ubfx x0, x0, #24, #4 |
| 542 | cmp x0, #1 | 542 | cbz x0, 3f |
| 543 | b.ne 3f | ||
| 544 | 543 | ||
| 545 | mrs_s x0, SYS_ICC_SRE_EL2 | 544 | mrs_s x0, SYS_ICC_SRE_EL2 |
| 546 | orr x0, x0, #ICC_SRE_EL2_SRE // Set ICC_SRE_EL2.SRE==1 | 545 | orr x0, x0, #ICC_SRE_EL2_SRE // Set ICC_SRE_EL2.SRE==1 |
| @@ -1702,19 +1702,20 @@ void syscall_trace_exit(struct pt_regs *regs) · 1701 unmodified lines | |||
| 1702 | } | 1702 | } |
| 1703 | 1703 | ||
| 1704 | /* | 1704 | /* |
| 1705 | * SPSR_ELx bits which are always architecturally RES0 per ARM DDI 0487C.a | 1705 | * SPSR_ELx bits which are always architecturally RES0 per ARM DDI 0487D.a. |
| 1706 | * We also take into account DIT (bit 24), which is not yet documented, and | 1706 | * We permit userspace to set SSBS (AArch64 bit 12, AArch32 bit 23) which is |
| 1707 | * treat PAN and UAO as RES0 bits, as they are meaningless at EL0, and may be | 1707 | * not described in ARM DDI 0487D.a. |
| 1708 | * allocated an EL0 meaning in future. | 1708 | * We treat PAN and UAO as RES0 bits, as they are meaningless at EL0, and may |
| 1709 | * be allocated an EL0 meaning in future. | ||
| 1709 | * Userspace cannot use these until they have an architectural meaning. | 1710 | * Userspace cannot use these until they have an architectural meaning. |
| 1710 | * Note that this follows the SPSR_ELx format, not the AArch32 PSR format. | 1711 | * Note that this follows the SPSR_ELx format, not the AArch32 PSR format. |
| 1711 | * We also reserve IL for the kernel; SS is handled dynamically. | 1712 | * We also reserve IL for the kernel; SS is handled dynamically. |
| 1712 | */ | 1713 | */ |
| 1713 | #define SPSR_EL1_AARCH64_RES0_BITS \ | 1714 | #define SPSR_EL1_AARCH64_RES0_BITS \ |
| 1714 | (GENMASK_ULL(63,32) | GENMASK_ULL(27, 25) | GENMASK_ULL(23, 22) | \ | 1715 | (GENMASK_ULL(63, 32) | GENMASK_ULL(27, 25) | GENMASK_ULL(23, 22) | \ |
| 1715 | GENMASK_ULL(20, 10) | GENMASK_ULL(5, 5)) | 1716 | GENMASK_ULL(20, 13) | GENMASK_ULL(11, 10) | GENMASK_ULL(5, 5)) |
| 1716 | #define SPSR_EL1_AARCH32_RES0_BITS \ | 1717 | #define SPSR_EL1_AARCH32_RES0_BITS \ |
| 1717 | (GENMASK_ULL(63,32) | GENMASK_ULL(23, 22) | GENMASK_ULL(20,20)) | 1718 | (GENMASK_ULL(63, 32) | GENMASK_ULL(22, 22) | GENMASK_ULL(20, 20)) |
| 1718 | 1719 | ||
| 1719 | static int valid_compat_regs(struct user_pt_regs *regs) | 1720 | static int valid_compat_regs(struct user_pt_regs *regs) |
| 1720 | { | 1721 | { |
| @@ -339,6 +339,9 @@ void __init setup_arch(char **cmdline_p) · 338 unmodified lines | |||
| 339 | smp_init_cpus(); | 339 | smp_init_cpus(); |
| 340 | smp_build_mpidr_hash(); | 340 | smp_build_mpidr_hash(); |
| 341 | 341 | ||
| 342 | /* Init percpu seeds for random tags after cpus are set up. */ | ||
| 343 | kasan_init_tags(); | ||
| 344 | |||
| 342 | #ifdef CONFIG_ARM64_SW_TTBR0_PAN | 345 | #ifdef CONFIG_ARM64_SW_TTBR0_PAN |
| 343 | /* | 346 | /* |
| 344 | * Make sure init_thread_info.ttbr0 always generates translation | 347 | * Make sure init_thread_info.ttbr0 always generates translation |
| @@ -252,8 +252,6 @@ void __init kasan_init(void) · 251 unmodified lines | |||
| 252 | memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE); | 252 | memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE); |
| 253 | cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); | 253 | cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); |
| 254 | 254 | ||
| 255 | kasan_init_tags(); | ||
| 256 | |||
| 257 | /* At this point kasan is fully initialized. Enable error messages */ | 255 | /* At this point kasan is fully initialized. Enable error messages */ |
| 258 | init_task.kasan_depth = 0; | 256 | init_task.kasan_depth = 0; |
| 259 | pr_info("KernelAddressSanitizer initialized\n"); | 257 | pr_info("KernelAddressSanitizer initialized\n"); |
| @@ -70,6 +70,8 @@ static struct platform_device bcm63xx_enet_shared_device = { · 69 unmodified lines | |||
| 70 | 70 | ||
| 71 | static int shared_device_registered; | 71 | static int shared_device_registered; |
| 72 | 72 | ||
| 73 | static u64 enet_dmamask = DMA_BIT_MASK(32); | ||
| 74 | |||
| 73 | static struct resource enet0_res[] = { | 75 | static struct resource enet0_res[] = { |
| 74 | { | 76 | { |
| 75 | .start = -1, /* filled at runtime */ | 77 | .start = -1, /* filled at runtime */ |
| @@ -99,6 +101,8 @@ static struct platform_device bcm63xx_enet0_device = { · 23 unmodified lines | |||
| 99 | .resource = enet0_res, | 101 | .resource = enet0_res, |
| 100 | .dev = { | 102 | .dev = { |
| 101 | .platform_data = &enet0_pd, | 103 | .platform_data = &enet0_pd, |
| 104 | .dma_mask = &enet_dmamask, | ||
| 105 | .coherent_dma_mask = DMA_BIT_MASK(32), | ||
| 102 | }, | 106 | }, |
| 103 | }; | 107 | }; |
| 104 | 108 | ||
| @@ -131,6 +135,8 @@ static struct platform_device bcm63xx_enet1_device = { · 26 unmodified lines | |||
| 131 | .resource = enet1_res, | 135 | .resource = enet1_res, |
| 132 | .dev = { | 136 | .dev = { |
| 133 | .platform_data = &enet1_pd, | 137 | .platform_data = &enet1_pd, |
| 138 | .dma_mask = &enet_dmamask, | ||
| 139 | .coherent_dma_mask = DMA_BIT_MASK(32), | ||
| 134 | }, | 140 | }, |
| 135 | }; | 141 | }; |
| 136 | 142 | ||
| @@ -157,6 +163,8 @@ static struct platform_device bcm63xx_enetsw_device = { · 20 unmodified lines | |||
| 157 | .resource = enetsw_res, | 163 | .resource = enetsw_res, |
| 158 | .dev = { | 164 | .dev = { |
| 159 | .platform_data = &enetsw_pd, | 165 | .platform_data = &enetsw_pd, |
| 166 | .dma_mask = &enet_dmamask, | ||
| 167 | .coherent_dma_mask = DMA_BIT_MASK(32), | ||
| 160 | }, | 168 | }, |
| 161 | }; | 169 | }; |
| 162 | 170 | ||
| @@ -54,10 +54,9 @@ unsigned long __xchg_small(volatile void *ptr, unsigned long val, unsigned int s · 53 unmodified lines | |||
| 54 | unsigned long __cmpxchg_small(volatile void *ptr, unsigned long old, | 54 | unsigned long __cmpxchg_small(volatile void *ptr, unsigned long old, |
| 55 | unsigned long new, unsigned int size) | 55 | unsigned long new, unsigned int size) |
| 56 | { | 56 | { |
| 57 | u32 mask, old32, new32, load32; | 57 | u32 mask, old32, new32, load32, load; |
| 58 | volatile u32 *ptr32; | 58 | volatile u32 *ptr32; |
| 59 | unsigned int shift; | 59 | unsigned int shift; |
| 60 | u8 load; | ||
| 61 | 60 | ||
| 62 | /* Check that ptr is naturally aligned */ | 61 | /* Check that ptr is naturally aligned */ |
| 63 | WARN_ON((unsigned long)ptr & (size - 1)); | 62 | WARN_ON((unsigned long)ptr & (size - 1)); |
| @@ -384,7 +384,8 @@ static void __init bootmem_init(void) · 383 unmodified lines | |||
| 384 | init_initrd(); | 384 | init_initrd(); |
| 385 | reserved_end = (unsigned long) PFN_UP(__pa_symbol(&_end)); | 385 | reserved_end = (unsigned long) PFN_UP(__pa_symbol(&_end)); |
| 386 | 386 | ||
| 387 | memblock_reserve(PHYS_OFFSET, reserved_end << PAGE_SHIFT); | 387 | memblock_reserve(PHYS_OFFSET, |
| 388 | (reserved_end << PAGE_SHIFT) - PHYS_OFFSET); | ||
| 388 | 389 | ||
| 389 | /* | 390 | /* |
| 390 | * max_low_pfn is not a number of pages. The number of pages | 391 | * max_low_pfn is not a number of pages. The number of pages |
| @@ -31,8 +31,8 @@ static int vmmc_probe(struct platform_device *pdev) · 30 unmodified lines | |||
| 31 | dma_addr_t dma; | 31 | dma_addr_t dma; |
| 32 | 32 | ||
| 33 | cp1_base = | 33 | cp1_base = |
| 34 | (void *) CPHYSADDR(dma_alloc_coherent(NULL, CP1_SIZE, | 34 | (void *) CPHYSADDR(dma_alloc_coherent(&pdev->dev, CP1_SIZE, |
| 35 | &dma, GFP_ATOMIC)); | 35 | &dma, GFP_KERNEL)); |
| 36 | 36 | ||
| 37 | gpio_count = of_gpio_count(pdev->dev.of_node); | 37 | gpio_count = of_gpio_count(pdev->dev.of_node); |
| 38 | while (gpio_count > 0) { | 38 | while (gpio_count > 0) { |
| @@ -79,8 +79,6 @@ enum reg_val_type { · 78 unmodified lines | |||
| 79 | REG_64BIT_32BIT, | 79 | REG_64BIT_32BIT, |
| 80 | /* 32-bit compatible, need truncation for 64-bit ops. */ | 80 | /* 32-bit compatible, need truncation for 64-bit ops. */ |
| 81 | REG_32BIT, | 81 | REG_32BIT, |
| 82 | /* 32-bit zero extended. */ | ||
| 83 | REG_32BIT_ZERO_EX, | ||
| 84 | /* 32-bit no sign/zero extension needed. */ | 82 | /* 32-bit no sign/zero extension needed. */ |
| 85 | REG_32BIT_POS | 83 | REG_32BIT_POS |
| 86 | }; | 84 | }; |
| @@ -343,12 +341,15 @@ static int build_int_epilogue(struct jit_ctx *ctx, int dest_reg) · 256 unmodified lines | |||
| 343 | const struct bpf_prog *prog = ctx->skf; | 341 | const struct bpf_prog *prog = ctx->skf; |
| 344 | int stack_adjust = ctx->stack_size; | 342 | int stack_adjust = ctx->stack_size; |
| 345 | int store_offset = stack_adjust - 8; | 343 | int store_offset = stack_adjust - 8; |
| 344 | enum reg_val_type td; | ||
| 346 | int r0 = MIPS_R_V0; | 345 | int r0 = MIPS_R_V0; |
| 347 | 346 | ||
| 348 | if (dest_reg == MIPS_R_RA && | 347 | if (dest_reg == MIPS_R_RA) { |
| 349 | get_reg_val_type(ctx, prog->len, BPF_REG_0) == REG_32BIT_ZERO_EX) | ||
| 350 | /* Don't let zero extended value escape. */ | 348 | /* Don't let zero extended value escape. */ |
| 351 | emit_instr(ctx, sll, r0, r0, 0); | 349 | td = get_reg_val_type(ctx, prog->len, BPF_REG_0); |
| 350 | if (td == REG_64BIT) | ||
| 351 | emit_instr(ctx, sll, r0, r0, 0); | ||
| 352 | } | ||
| 352 | 353 | ||
| 353 | if (ctx->flags & EBPF_SAVE_RA) { | 354 | if (ctx->flags & EBPF_SAVE_RA) { |
| 354 | emit_instr(ctx, ld, MIPS_R_RA, store_offset, MIPS_R_SP); | 355 | emit_instr(ctx, ld, MIPS_R_RA, store_offset, MIPS_R_SP); |
| @@ -692,7 +693,7 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, · 337 unmodified lines | |||
| 692 | if (dst < 0) | 693 | if (dst < 0) |
| 693 | return dst; | 694 | return dst; |
| 694 | td = get_reg_val_type(ctx, this_idx, insn->dst_reg); | 695 | td = get_reg_val_type(ctx, this_idx, insn->dst_reg); |
| 695 | if (td == REG_64BIT || td == REG_32BIT_ZERO_EX) { | 696 | if (td == REG_64BIT) { |
| 696 | /* sign extend */ | 697 | /* sign extend */ |
| 697 | emit_instr(ctx, sll, dst, dst, 0); | 698 | emit_instr(ctx, sll, dst, dst, 0); |
| 698 | } | 699 | } |
| @@ -707,7 +708,7 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, · 8 unmodified lines | |||
| 707 | if (dst < 0) | 708 | if (dst < 0) |
| 708 | return dst; | 709 | return dst; |
| 709 | td = get_reg_val_type(ctx, this_idx, insn->dst_reg); | 710 | td = get_reg_val_type(ctx, this_idx, insn->dst_reg); |
| 710 | if (td == REG_64BIT || td == REG_32BIT_ZERO_EX) { | 711 | if (td == REG_64BIT) { |
| 711 | /* sign extend */ | 712 | /* sign extend */ |
| 712 | emit_instr(ctx, sll, dst, dst, 0); | 713 | emit_instr(ctx, sll, dst, dst, 0); |
| 713 | } | 714 | } |
| @@ -721,7 +722,7 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, · 7 unmodified lines | |||
| 721 | if (dst < 0) | 722 | if (dst < 0) |
| 722 | return dst; | 723 | return dst; |
| 723 | td = get_reg_val_type(ctx, this_idx, insn->dst_reg); | 724 | td = get_reg_val_type(ctx, this_idx, insn->dst_reg); |
| 724 | if (td == REG_64BIT || td == REG_32BIT_ZERO_EX) | 725 | if (td == REG_64BIT) |
| 725 | /* sign extend */ | 726 | /* sign extend */ |
| 726 | emit_instr(ctx, sll, dst, dst, 0); | 727 | emit_instr(ctx, sll, dst, dst, 0); |
| 727 | if (insn->imm == 1) { | 728 | if (insn->imm == 1) { |
| @@ -860,13 +861,13 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, · 132 unmodified lines | |||
| 860 | if (src < 0 || dst < 0) | 861 | if (src < 0 || dst < 0) |
| 861 | return -EINVAL; | 862 | return -EINVAL; |
| 862 | td = get_reg_val_type(ctx, this_idx, insn->dst_reg); | 863 | td = get_reg_val_type(ctx, this_idx, insn->dst_reg); |
| 863 | if (td == REG_64BIT || td == REG_32BIT_ZERO_EX) { | 864 | if (td == REG_64BIT) { |
| 864 | /* sign extend */ | 865 | /* sign extend */ |
| 865 | emit_instr(ctx, sll, dst, dst, 0); | 866 | emit_instr(ctx, sll, dst, dst, 0); |
| 866 | } | 867 | } |
| 867 | did_move = false; | 868 | did_move = false; |
| 868 | ts = get_reg_val_type(ctx, this_idx, insn->src_reg); | 869 | ts = get_reg_val_type(ctx, this_idx, insn->src_reg); |
| 869 | if (ts == REG_64BIT || ts == REG_32BIT_ZERO_EX) { | 870 | if (ts == REG_64BIT) { |
| 870 | int tmp_reg = MIPS_R_AT; | 871 | int tmp_reg = MIPS_R_AT; |
| 871 | 872 | ||
| 872 | if (bpf_op == BPF_MOV) { | 873 | if (bpf_op == BPF_MOV) { |
| @@ -1254,8 +1255,7 @@ jeq_common: · 381 unmodified lines | |||
| 1254 | if (insn->imm == 64 && td == REG_32BIT) | 1255 | if (insn->imm == 64 && td == REG_32BIT) |
| 1255 | emit_instr(ctx, dinsu, dst, MIPS_R_ZERO, 32, 32); | 1256 | emit_instr(ctx, dinsu, dst, MIPS_R_ZERO, 32, 32); |
| 1256 | 1257 | ||
| 1257 | if (insn->imm != 64 && | 1258 | if (insn->imm != 64 && td == REG_64BIT) { |
| 1258 | (td == REG_64BIT || td == REG_32BIT_ZERO_EX)) { | ||
| 1259 | /* sign extend */ | 1259 | /* sign extend */ |
| 1260 | emit_instr(ctx, sll, dst, dst, 0); | 1260 | emit_instr(ctx, sll, dst, dst, 0); |
| 1261 | } | 1261 | } |
| @@ -1819,7 +1819,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) · 557 unmodified lines | |||
| 1819 | 1819 | ||
| 1820 | /* Update the icache */ | 1820 | /* Update the icache */ |
| 1821 | flush_icache_range((unsigned long)ctx.target, | 1821 | flush_icache_range((unsigned long)ctx.target, |
| 1822 | (unsigned long)(ctx.target + ctx.idx * sizeof(u32))); | 1822 | (unsigned long)&ctx.target[ctx.idx]); |
| 1823 | 1823 | ||
| 1824 | if (bpf_jit_enable > 1) | 1824 | if (bpf_jit_enable > 1) |
| 1825 | /* Dump JIT code */ | 1825 | /* Dump JIT code */ |
| @@ -308,15 +308,29 @@ long compat_arch_ptrace(struct task_struct *child, compat_long_t request, · 307 unmodified lines | |||
| 308 | 308 | ||
| 309 | long do_syscall_trace_enter(struct pt_regs *regs) | 309 | long do_syscall_trace_enter(struct pt_regs *regs) |
| 310 | { | 310 | { |
| 311 | if (test_thread_flag(TIF_SYSCALL_TRACE) && | 311 | if (test_thread_flag(TIF_SYSCALL_TRACE)) { |
| 312 | tracehook_report_syscall_entry(regs)) { | 312 | int rc = tracehook_report_syscall_entry(regs); |
| 313 | |||
| 313 | /* | 314 | /* |
| 314 | * Tracing decided this syscall should not happen or the | 315 | * As tracesys_next does not set %r28 to -ENOSYS |
| 315 | * debugger stored an invalid system call number. Skip | 316 | * when %r20 is set to -1, initialize it here. |
| 316 | * the system call and the system call restart handling. | ||
| 317 | */ | 317 | */ |
| 318 | regs->gr[20] = -1UL; | 318 | regs->gr[28] = -ENOSYS; |
| 319 | goto out; | 319 | |
| 320 | if (rc) { | ||
| 321 | /* | ||
| 322 | * A nonzero return code from | ||
| 323 | * tracehook_report_syscall_entry() tells us | ||
| 324 | * to prevent the syscall execution. Skip | ||
| 325 | * the syscall call and the syscall restart handling. | ||
| 326 | * | ||
| 327 | * Note that the tracer may also just change | ||
| 328 | * regs->gr[20] to an invalid syscall number, | ||
| 329 | * that is handled by tracesys_next. | ||
| 330 | */ | ||
| 331 | regs->gr[20] = -1UL; | ||
| 332 | return -1; | ||
| 333 | } | ||
| 320 | } | 334 | } |
| 321 | 335 | ||
| 322 | /* Do the secure computing check after ptrace. */ | 336 | /* Do the secure computing check after ptrace. */ |
| @@ -340,7 +354,6 @@ long do_syscall_trace_enter(struct pt_regs *regs) · 17 unmodified lines | |||
| 340 | regs->gr[24] & 0xffffffff, | 354 | regs->gr[24] & 0xffffffff, |
| 341 | regs->gr[23] & 0xffffffff); | 355 | regs->gr[23] & 0xffffffff); |
| 342 | 356 | ||
| 343 | out: | ||
| 344 | /* | 357 | /* |
| 345 | * Sign extend the syscall number to 64bit since it may have been | 358 | * Sign extend the syscall number to 64bit since it may have been |
| 346 | * modified by a compat ptrace call | 359 | * modified by a compat ptrace call |
| @@ -1593,6 +1593,8 @@ static void pnv_ioda_setup_vf_PE(struct pci_dev *pdev, u16 num_vfs) · 1592 unmodified lines | |||
| 1593 | 1593 | ||
| 1594 | pnv_pci_ioda2_setup_dma_pe(phb, pe); | 1594 | pnv_pci_ioda2_setup_dma_pe(phb, pe); |
| 1595 | #ifdef CONFIG_IOMMU_API | 1595 | #ifdef CONFIG_IOMMU_API |
| 1596 | iommu_register_group(&pe->table_group, | ||
| 1597 | pe->phb->hose->global_number, pe->pe_number); | ||
| 1596 | pnv_ioda_setup_bus_iommu_group(pe, &pe->table_group, NULL); | 1598 | pnv_ioda_setup_bus_iommu_group(pe, &pe->table_group, NULL); |
| 1597 | #endif | 1599 | #endif |
| 1598 | } | 1600 | } |
| @@ -1147,6 +1147,8 @@ static int pnv_tce_iommu_bus_notifier(struct notifier_block *nb, · 1146 unmodified lines | |||
| 1147 | return 0; | 1147 | return 0; |
| 1148 | 1148 | ||
| 1149 | pe = &phb->ioda.pe_array[pdn->pe_number]; | 1149 | pe = &phb->ioda.pe_array[pdn->pe_number]; |
| 1150 | if (!pe->table_group.group) | ||
| 1151 | return 0; | ||
| 1150 | iommu_add_device(&pe->table_group, dev); | 1152 | iommu_add_device(&pe->table_group, dev); |
| 1151 | return 0; | 1153 | return 0; |
| 1152 | case BUS_NOTIFY_DEL_DEVICE: | 1154 | case BUS_NOTIFY_DEL_DEVICE: |
| @@ -297,7 +297,7 @@ static int shadow_crycb(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page) · 296 unmodified lines | |||
| 297 | scb_s->crycbd = 0; | 297 | scb_s->crycbd = 0; |
| 298 | 298 | ||
| 299 | apie_h = vcpu->arch.sie_block->eca & ECA_APIE; | 299 | apie_h = vcpu->arch.sie_block->eca & ECA_APIE; |
| 300 | if (!apie_h && !key_msk) | 300 | if (!apie_h && (!key_msk || fmt_o == CRYCB_FORMAT0)) |
| 301 | return 0; | 301 | return 0; |
| 302 | 302 | ||
| 303 | if (!crycb_addr) | 303 | if (!crycb_addr) |
| @@ -1,3 +1,3 @@ | |||
| 1 | ifneq ($(CONFIG_BUILTIN_DTB_SOURCE),"") | 1 | ifneq ($(CONFIG_BUILTIN_DTB_SOURCE),"") |
| 2 | obj-y += $(patsubst "%",%,$(CONFIG_BUILTIN_DTB_SOURCE)).dtb.o | 2 | obj-$(CONFIG_USE_BUILTIN_DTB) += $(patsubst "%",%,$(CONFIG_BUILTIN_DTB_SOURCE)).dtb.o |
| 3 | endif | 3 | endif |
| @@ -841,7 +841,7 @@ union hv_gpa_page_range { · 840 unmodified lines | |||
| 841 | * count is equal with how many entries of union hv_gpa_page_range can | 841 | * count is equal with how many entries of union hv_gpa_page_range can |
| 842 | * be populated into the input parameter page. | 842 | * be populated into the input parameter page. |
| 843 | */ | 843 | */ |
| 844 | #define HV_MAX_FLUSH_REP_COUNT (PAGE_SIZE - 2 * sizeof(u64) / \ | 844 | #define HV_MAX_FLUSH_REP_COUNT ((PAGE_SIZE - 2 * sizeof(u64)) / \ |
| 845 | sizeof(union hv_gpa_page_range)) | 845 | sizeof(union hv_gpa_page_range)) |
| 846 | 846 | ||
| 847 | struct hv_guest_mapping_flush_list { | 847 | struct hv_guest_mapping_flush_list { |
| @@ -299,6 +299,7 @@ union kvm_mmu_extended_role { · 298 unmodified lines | |||
| 299 | unsigned int cr4_smap:1; | 299 | unsigned int cr4_smap:1; |
| 300 | unsigned int cr4_smep:1; | 300 | unsigned int cr4_smep:1; |
| 301 | unsigned int cr4_la57:1; | 301 | unsigned int cr4_la57:1; |
| 302 | unsigned int maxphyaddr:6; | ||
| 302 | }; | 303 | }; |
| 303 | }; | 304 | }; |
| 304 | 305 | ||
| @@ -397,6 +398,7 @@ struct kvm_mmu { · 92 unmodified lines | |||
| 397 | void (*update_pte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, | 398 | void (*update_pte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, |
| 398 | u64 *spte, const void *pte); | 399 | u64 *spte, const void *pte); |
| 399 | hpa_t root_hpa; | 400 | hpa_t root_hpa; |
| 401 | gpa_t root_cr3; | ||
| 400 | union kvm_mmu_role mmu_role; | 402 | union kvm_mmu_role mmu_role; |
| 401 | u8 root_level; | 403 | u8 root_level; |
| 402 | u8 shadow_root_level; | 404 | u8 shadow_root_level; |
| @@ -284,7 +284,7 @@ do { \ · 283 unmodified lines | |||
| 284 | __put_user_goto(x, ptr, "l", "k", "ir", label); \ | 284 | __put_user_goto(x, ptr, "l", "k", "ir", label); \ |
| 285 | break; \ | 285 | break; \ |
| 286 | case 8: \ | 286 | case 8: \ |
| 287 | __put_user_goto_u64((__typeof__(*ptr))(x), ptr, label); \ | 287 | __put_user_goto_u64(x, ptr, label); \ |
| 288 | break; \ | 288 | break; \ |
| 289 | default: \ | 289 | default: \ |
| 290 | __put_user_bad(); \ | 290 | __put_user_bad(); \ |
| @@ -431,8 +431,10 @@ do { \ · 140 unmodified lines | |||
| 431 | ({ \ | 431 | ({ \ |
| 432 | __label__ __pu_label; \ | 432 | __label__ __pu_label; \ |
| 433 | int __pu_err = -EFAULT; \ | 433 | int __pu_err = -EFAULT; \ |
| 434 | __typeof__(*(ptr)) __pu_val; \ | ||
| 435 | __pu_val = x; \ | ||
| 434 | __uaccess_begin(); \ | 436 | __uaccess_begin(); \ |
| 435 | __put_user_size((x), (ptr), (size), __pu_label); \ | 437 | __put_user_size(__pu_val, (ptr), (size), __pu_label); \ |
| 436 | __pu_err = 0; \ | 438 | __pu_err = 0; \ |
| 437 | __pu_label: \ | 439 | __pu_label: \ |
| 438 | __uaccess_end(); \ | 440 | __uaccess_end(); \ |
| @@ -335,6 +335,7 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function, · 334 unmodified lines | |||
| 335 | unsigned f_xsaves = kvm_x86_ops->xsaves_supported() ? F(XSAVES) : 0; | 335 | unsigned f_xsaves = kvm_x86_ops->xsaves_supported() ? F(XSAVES) : 0; |
| 336 | unsigned f_umip = kvm_x86_ops->umip_emulated() ? F(UMIP) : 0; | 336 | unsigned f_umip = kvm_x86_ops->umip_emulated() ? F(UMIP) : 0; |
| 337 | unsigned f_intel_pt = kvm_x86_ops->pt_supported() ? F(INTEL_PT) : 0; | 337 | unsigned f_intel_pt = kvm_x86_ops->pt_supported() ? F(INTEL_PT) : 0; |
| 338 | unsigned f_la57 = 0; | ||
| 338 | 339 | ||
| 339 | /* cpuid 1.edx */ | 340 | /* cpuid 1.edx */ |
| 340 | const u32 kvm_cpuid_1_edx_x86_features = | 341 | const u32 kvm_cpuid_1_edx_x86_features = |
| @@ -489,7 +490,10 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function, · 148 unmodified lines | |||
| 489 | // TSC_ADJUST is emulated | 490 | // TSC_ADJUST is emulated |
| 490 | entry->ebx |= F(TSC_ADJUST); | 491 | entry->ebx |= F(TSC_ADJUST); |
| 491 | entry->ecx &= kvm_cpuid_7_0_ecx_x86_features; | 492 | entry->ecx &= kvm_cpuid_7_0_ecx_x86_features; |
| 493 | f_la57 = entry->ecx & F(LA57); | ||
| 492 | cpuid_mask(&entry->ecx, CPUID_7_ECX); | 494 | cpuid_mask(&entry->ecx, CPUID_7_ECX); |
| 495 | /* Set LA57 based on hardware capability. */ | ||
| 496 | entry->ecx |= f_la57; | ||
| 493 | entry->ecx |= f_umip; | 497 | entry->ecx |= f_umip; |
| 494 | /* PKU is not yet implemented for shadow paging. */ | 498 | /* PKU is not yet implemented for shadow paging. */ |
| 495 | if (!tdp_enabled || !boot_cpu_has(X86_FEATURE_OSPKE)) | 499 | if (!tdp_enabled || !boot_cpu_has(X86_FEATURE_OSPKE)) |
| @@ -3555,6 +3555,7 @@ void kvm_mmu_free_roots(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, · 3554 unmodified lines | |||
| 3555 | &invalid_list); | 3555 | &invalid_list); |
| 3556 | mmu->root_hpa = INVALID_PAGE; | 3556 | mmu->root_hpa = INVALID_PAGE; |
| 3557 | } | 3557 | } |
| 3558 | mmu->root_cr3 = 0; | ||
| 3558 | } | 3559 | } |
| 3559 | 3560 | ||
| 3560 | kvm_mmu_commit_zap_page(vcpu->kvm, &invalid_list); | 3561 | kvm_mmu_commit_zap_page(vcpu->kvm, &invalid_list); |
| @@ -3610,6 +3611,7 @@ static int mmu_alloc_direct_roots(struct kvm_vcpu *vcpu) · 49 unmodified lines | |||
| 3610 | vcpu->arch.mmu->root_hpa = __pa(vcpu->arch.mmu->pae_root); | 3611 | vcpu->arch.mmu->root_hpa = __pa(vcpu->arch.mmu->pae_root); |
| 3611 | } else | 3612 | } else |
| 3612 | BUG(); | 3613 | BUG(); |
| 3614 | vcpu->arch.mmu->root_cr3 = vcpu->arch.mmu->get_cr3(vcpu); | ||
| 3613 | 3615 | ||
| 3614 | return 0; | 3616 | return 0; |
| 3615 | } | 3617 | } |
| @@ -3618,10 +3620,11 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) · 2 unmodified lines | |||
| 3618 | { | 3620 | { |
| 3619 | struct kvm_mmu_page *sp; | 3621 | struct kvm_mmu_page *sp; |
| 3620 | u64 pdptr, pm_mask; | 3622 | u64 pdptr, pm_mask; |
| 3621 | gfn_t root_gfn; | 3623 | gfn_t root_gfn, root_cr3; |
| 3622 | int i; | 3624 | int i; |
| 3623 | 3625 | ||
| 3624 | root_gfn = vcpu->arch.mmu->get_cr3(vcpu) >> PAGE_SHIFT; | 3626 | root_cr3 = vcpu->arch.mmu->get_cr3(vcpu); |
| 3627 | root_gfn = root_cr3 >> PAGE_SHIFT; | ||
| 3625 | 3628 | ||
| 3626 | if (mmu_check_root(vcpu, root_gfn)) | 3629 | if (mmu_check_root(vcpu, root_gfn)) |
| 3627 | return 1; | 3630 | return 1; |
| @@ -3646,7 +3649,7 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) · 18 unmodified lines | |||
| 3646 | ++sp->root_count; | 3649 | ++sp->root_count; |
| 3647 | spin_unlock(&vcpu->kvm->mmu_lock); | 3650 | spin_unlock(&vcpu->kvm->mmu_lock); |
| 3648 | vcpu->arch.mmu->root_hpa = root; | 3651 | vcpu->arch.mmu->root_hpa = root; |
| 3649 | return 0; | 3652 | goto set_root_cr3; |
| 3650 | } | 3653 | } |
| 3651 | 3654 | ||
| 3652 | /* | 3655 | /* |
| @@ -3712,6 +3715,9 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) · 59 unmodified lines | |||
| 3712 | vcpu->arch.mmu->root_hpa = __pa(vcpu->arch.mmu->lm_root); | 3715 | vcpu->arch.mmu->root_hpa = __pa(vcpu->arch.mmu->lm_root); |
| 3713 | } | 3716 | } |
| 3714 | 3717 | ||
| 3718 | set_root_cr3: | ||
| 3719 | vcpu->arch.mmu->root_cr3 = root_cr3; | ||
| 3720 | |||
| 3715 | return 0; | 3721 | return 0; |
| 3716 | } | 3722 | } |
| 3717 | 3723 | ||
| @@ -4163,7 +4169,7 @@ static bool cached_root_available(struct kvm_vcpu *vcpu, gpa_t new_cr3, · 445 unmodified lines | |||
| 4163 | struct kvm_mmu_root_info root; | 4169 | struct kvm_mmu_root_info root; |
| 4164 | struct kvm_mmu *mmu = vcpu->arch.mmu; | 4170 | struct kvm_mmu *mmu = vcpu->arch.mmu; |
| 4165 | 4171 | ||
| 4166 | root.cr3 = mmu->get_cr3(vcpu); | 4172 | root.cr3 = mmu->root_cr3; |
| 4167 | root.hpa = mmu->root_hpa; | 4173 | root.hpa = mmu->root_hpa; |
| 4168 | 4174 | ||
| 4169 | for (i = 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) { | 4175 | for (i = 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) { |
| @@ -4176,6 +4182,7 @@ static bool cached_root_available(struct kvm_vcpu *vcpu, gpa_t new_cr3, · 6 unmodified lines | |||
| 4176 | } | 4182 | } |
| 4177 | 4183 | ||
| 4178 | mmu->root_hpa = root.hpa; | 4184 | mmu->root_hpa = root.hpa; |
| 4185 | mmu->root_cr3 = root.cr3; | ||
| 4179 | 4186 | ||
| 4180 | return i < KVM_MMU_NUM_PREV_ROOTS; | 4187 | return i < KVM_MMU_NUM_PREV_ROOTS; |
| 4181 | } | 4188 | } |
| @@ -4770,6 +4777,7 @@ static union kvm_mmu_extended_role kvm_calc_mmu_role_ext(struct kvm_vcpu *vcpu) · 588 unmodified lines | |||
| 4770 | ext.cr4_pse = !!is_pse(vcpu); | 4777 | ext.cr4_pse = !!is_pse(vcpu); |
| 4771 | ext.cr4_pke = !!kvm_read_cr4_bits(vcpu, X86_CR4_PKE); | 4778 | ext.cr4_pke = !!kvm_read_cr4_bits(vcpu, X86_CR4_PKE); |
| 4772 | ext.cr4_la57 = !!kvm_read_cr4_bits(vcpu, X86_CR4_LA57); | 4779 | ext.cr4_la57 = !!kvm_read_cr4_bits(vcpu, X86_CR4_LA57); |
| 4780 | ext.maxphyaddr = cpuid_maxphyaddr(vcpu); | ||
| 4773 | 4781 | ||
| 4774 | ext.valid = 1; | 4782 | ext.valid = 1; |
| 4775 | 4783 | ||
| @@ -5516,11 +5524,13 @@ int kvm_mmu_create(struct kvm_vcpu *vcpu) · 740 unmodified lines | |||
| 5516 | vcpu->arch.walk_mmu = &vcpu->arch.root_mmu; | 5524 | vcpu->arch.walk_mmu = &vcpu->arch.root_mmu; |
| 5517 | 5525 | ||
| 5518 | vcpu->arch.root_mmu.root_hpa = INVALID_PAGE; | 5526 | vcpu->arch.root_mmu.root_hpa = INVALID_PAGE; |
| 5527 | vcpu->arch.root_mmu.root_cr3 = 0; | ||
| 5519 | vcpu->arch.root_mmu.translate_gpa = translate_gpa; | 5528 | vcpu->arch.root_mmu.translate_gpa = translate_gpa; |
| 5520 | for (i = 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) | 5529 | for (i = 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) |
| 5521 | vcpu->arch.root_mmu.prev_roots[i] = KVM_MMU_ROOT_INFO_INVALID; | 5530 | vcpu->arch.root_mmu.prev_roots[i] = KVM_MMU_ROOT_INFO_INVALID; |
| 5522 | 5531 | ||
| 5523 | vcpu->arch.guest_mmu.root_hpa = INVALID_PAGE; | 5532 | vcpu->arch.guest_mmu.root_hpa = INVALID_PAGE; |
| 5533 | vcpu->arch.guest_mmu.root_cr3 = 0; | ||
| 5524 | vcpu->arch.guest_mmu.translate_gpa = translate_gpa; | 5534 | vcpu->arch.guest_mmu.translate_gpa = translate_gpa; |
| 5525 | for (i = 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) | 5535 | for (i = 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) |
| 5526 | vcpu->arch.guest_mmu.prev_roots[i] = KVM_MMU_ROOT_INFO_INVALID; | 5536 | vcpu->arch.guest_mmu.prev_roots[i] = KVM_MMU_ROOT_INFO_INVALID; |
| @@ -117,67 +117,11 @@ __visible bool ex_handler_fprestore(const struct exception_table_entry *fixup, · 116 unmodified lines | |||
| 117 | } | 117 | } |
| 118 | EXPORT_SYMBOL_GPL(ex_handler_fprestore); | 118 | EXPORT_SYMBOL_GPL(ex_handler_fprestore); |
| 119 | 119 | ||
| 120 | /* Helper to check whether a uaccess fault indicates a kernel bug. */ | ||
| 121 | static bool bogus_uaccess(struct pt_regs *regs, int trapnr, | ||
| 122 | unsigned long fault_addr) | ||
| 123 | { | ||
| 124 | /* This is the normal case: #PF with a fault address in userspace. */ | ||
| 125 | if (trapnr == X86_TRAP_PF && fault_addr < TASK_SIZE_MAX) | ||
| 126 | return false; | ||
| 127 | |||
| 128 | /* | ||
| 129 | * This code can be reached for machine checks, but only if the #MC | ||
| 130 | * handler has already decided that it looks like a candidate for fixup. | ||
| 131 | * This e.g. happens when attempting to access userspace memory which | ||
| 132 | * the CPU can't access because of uncorrectable bad memory. | ||
| 133 | */ | ||
| 134 | if (trapnr == X86_TRAP_MC) | ||
| 135 | return false; | ||
| 136 | |||
| 137 | /* | ||
| 138 | * There are two remaining exception types we might encounter here: | ||
| 139 | * - #PF for faulting accesses to kernel addresses | ||
| 140 | * - #GP for faulting accesses to noncanonical addresses | ||
| 141 | * Complain about anything else. | ||
| 142 | */ | ||
| 143 | if (trapnr != X86_TRAP_PF && trapnr != X86_TRAP_GP) { | ||
| 144 | WARN(1, "unexpected trap %d in uaccess\n", trapnr); | ||
| 145 | return false; | ||
| 146 | } | ||
| 147 | |||
| 148 | /* | ||
| 149 | * This is a faulting memory access in kernel space, on a kernel | ||
| 150 | * address, in a usercopy function. This can e.g. be caused by improper | ||
| 151 | * use of helpers like __put_user and by improper attempts to access | ||
| 152 | * userspace addresses in KERNEL_DS regions. | ||
| 153 | * The one (semi-)legitimate exception are probe_kernel_{read,write}(), | ||
| 154 | * which can be invoked from places like kgdb, /dev/mem (for reading) | ||
| 155 | * and privileged BPF code (for reading). | ||
| 156 | * The probe_kernel_*() functions set the kernel_uaccess_faults_ok flag | ||
| 157 | * to tell us that faulting on kernel addresses, and even noncanonical | ||
| 158 | * addresses, in a userspace accessor does not necessarily imply a | ||
| 159 | * kernel bug, root might just be doing weird stuff. | ||
| 160 | */ | ||
| 161 | if (current->kernel_uaccess_faults_ok) | ||
| 162 | return false; | ||
| 163 | |||
| 164 | /* This is bad. Refuse the fixup so that we go into die(). */ | ||
| 165 | if (trapnr == X86_TRAP_PF) { | ||
| 166 | pr_emerg("BUG: pagefault on kernel address 0x%lx in non-whitelisted uaccess\n", | ||
| 167 | fault_addr); | ||
| 168 | } else { | ||
| 169 | pr_emerg("BUG: GPF in non-whitelisted uaccess (non-canonical address?)\n"); | ||
| 170 | } | ||
| 171 | return true; | ||
| 172 | } | ||
| 173 | |||
| 174 | __visible bool ex_handler_uaccess(const struct exception_table_entry *fixup, | 120 | __visible bool ex_handler_uaccess(const struct exception_table_entry *fixup, |
| 175 | struct pt_regs *regs, int trapnr, | 121 | struct pt_regs *regs, int trapnr, |
| 176 | unsigned long error_code, | 122 | unsigned long error_code, |
| 177 | unsigned long fault_addr) | 123 | unsigned long fault_addr) |
| 178 | { | 124 | { |
| 179 | if (bogus_uaccess(regs, trapnr, fault_addr)) | ||
| 180 | return false; | ||
| 181 | regs->ip = ex_fixup_addr(fixup); | 125 | regs->ip = ex_fixup_addr(fixup); |
| 182 | return true; | 126 | return true; |
| 183 | } | 127 | } |
| @@ -188,8 +132,6 @@ __visible bool ex_handler_ext(const struct exception_table_entry *fixup, · 4 unmodified lines | |||
| 188 | unsigned long error_code, | 132 | unsigned long error_code, |
| 189 | unsigned long fault_addr) | 133 | unsigned long fault_addr) |
| 190 | { | 134 | { |
| 191 | if (bogus_uaccess(regs, trapnr, fault_addr)) | ||
| 192 | return false; | ||
| 193 | /* Special hack for uaccess_err */ | 135 | /* Special hack for uaccess_err */ |
| 194 | current->thread.uaccess_err = 1; | 136 | current->thread.uaccess_err = 1; |
| 195 | regs->ip = ex_fixup_addr(fixup); | 137 | regs->ip = ex_fixup_addr(fixup); |
| @@ -122,8 +122,10 @@ static void alg_do_release(const struct af_alg_type *type, void *private) · 121 unmodified lines | |||
| 122 | 122 | ||
| 123 | int af_alg_release(struct socket *sock) | 123 | int af_alg_release(struct socket *sock) |
| 124 | { | 124 | { |
| 125 | if (sock->sk) | 125 | if (sock->sk) { |
| 126 | sock_put(sock->sk); | 126 | sock_put(sock->sk); |
| 127 | sock->sk = NULL; | ||
| 128 | } | ||
| 127 | return 0; | 129 | return 0; |
| 128 | } | 130 | } |
| 129 | EXPORT_SYMBOL_GPL(af_alg_release); | 131 | EXPORT_SYMBOL_GPL(af_alg_release); |
| @@ -95,7 +95,7 @@ static void __update_runtime_status(struct device *dev, enum rpm_status status) · 94 unmodified lines | |||
| 95 | static void pm_runtime_deactivate_timer(struct device *dev) | 95 | static void pm_runtime_deactivate_timer(struct device *dev) |
| 96 | { | 96 | { |
| 97 | if (dev->power.timer_expires > 0) { | 97 | if (dev->power.timer_expires > 0) { |
| 98 | hrtimer_cancel(&dev->power.suspend_timer); | 98 | hrtimer_try_to_cancel(&dev->power.suspend_timer); |
| 99 | dev->power.timer_expires = 0; | 99 | dev->power.timer_expires = 0; |
| 100 | } | 100 | } |
| 101 | } | 101 | } |
| @@ -144,8 +144,7 @@ static void __init at91sam9x5_pmc_setup(struct device_node *np, · 143 unmodified lines | |||
| 144 | return; | 144 | return; |
| 145 | 145 | ||
| 146 | at91sam9x5_pmc = pmc_data_allocate(PMC_MAIN + 1, | 146 | at91sam9x5_pmc = pmc_data_allocate(PMC_MAIN + 1, |
| 147 | nck(at91sam9x5_systemck), | 147 | nck(at91sam9x5_systemck), 31, 0); |
| 148 | nck(at91sam9x35_periphck), 0); | ||
| 149 | if (!at91sam9x5_pmc) | 148 | if (!at91sam9x5_pmc) |
| 150 | return; | 149 | return; |
| 151 | 150 | ||
| @@ -210,7 +209,7 @@ static void __init at91sam9x5_pmc_setup(struct device_node *np, · 58 unmodified lines | |||
| 210 | parent_names[1] = "mainck"; | 209 | parent_names[1] = "mainck"; |
| 211 | parent_names[2] = "plladivck"; | 210 | parent_names[2] = "plladivck"; |
| 212 | parent_names[3] = "utmick"; | 211 | parent_names[3] = "utmick"; |
| 213 | parent_names[4] = "mck"; | 212 | parent_names[4] = "masterck"; |
| 214 | for (i = 0; i < 2; i++) { | 213 | for (i = 0; i < 2; i++) { |
| 215 | char name[6]; | 214 | char name[6]; |
| 216 | 215 | ||
| @@ -240,7 +240,7 @@ static void __init sama5d2_pmc_setup(struct device_node *np) · 239 unmodified lines | |||
| 240 | parent_names[1] = "mainck"; | 240 | parent_names[1] = "mainck"; |
| 241 | parent_names[2] = "plladivck"; | 241 | parent_names[2] = "plladivck"; |
| 242 | parent_names[3] = "utmick"; | 242 | parent_names[3] = "utmick"; |
| 243 | parent_names[4] = "mck"; | 243 | parent_names[4] = "masterck"; |
| 244 | for (i = 0; i < 3; i++) { | 244 | for (i = 0; i < 3; i++) { |
| 245 | char name[6]; | 245 | char name[6]; |
| 246 | 246 | ||
| @@ -291,7 +291,7 @@ static void __init sama5d2_pmc_setup(struct device_node *np) · 44 unmodified lines | |||
| 291 | parent_names[1] = "mainck"; | 291 | parent_names[1] = "mainck"; |
| 292 | parent_names[2] = "plladivck"; | 292 | parent_names[2] = "plladivck"; |
| 293 | parent_names[3] = "utmick"; | 293 | parent_names[3] = "utmick"; |
| 294 | parent_names[4] = "mck"; | 294 | parent_names[4] = "masterck"; |
| 295 | parent_names[5] = "audiopll_pmcck"; | 295 | parent_names[5] = "audiopll_pmcck"; |
| 296 | for (i = 0; i < ARRAY_SIZE(sama5d2_gck); i++) { | 296 | for (i = 0; i < ARRAY_SIZE(sama5d2_gck); i++) { |
| 297 | hw = at91_clk_register_generated(regmap, &pmc_pcr_lock, | 297 | hw = at91_clk_register_generated(regmap, &pmc_pcr_lock, |
| @@ -207,7 +207,7 @@ static void __init sama5d4_pmc_setup(struct device_node *np) · 206 unmodified lines | |||
| 207 | parent_names[1] = "mainck"; | 207 | parent_names[1] = "mainck"; |
| 208 | parent_names[2] = "plladivck"; | 208 | parent_names[2] = "plladivck"; |
| 209 | parent_names[3] = "utmick"; | 209 | parent_names[3] = "utmick"; |
| 210 | parent_names[4] = "mck"; | 210 | parent_names[4] = "masterck"; |
| 211 | for (i = 0; i < 3; i++) { | 211 | for (i = 0; i < 3; i++) { |
| 212 | char name[6]; | 212 | char name[6]; |
| 213 | 213 | ||
| @@ -264,9 +264,9 @@ static SUNXI_CCU_GATE(ahb1_mmc1_clk, "ahb1-mmc1", "ahb1", · 263 unmodified lines | |||
| 264 | static SUNXI_CCU_GATE(ahb1_mmc2_clk, "ahb1-mmc2", "ahb1", | 264 | static SUNXI_CCU_GATE(ahb1_mmc2_clk, "ahb1-mmc2", "ahb1", |
| 265 | 0x060, BIT(10), 0); | 265 | 0x060, BIT(10), 0); |
| 266 | static SUNXI_CCU_GATE(ahb1_mmc3_clk, "ahb1-mmc3", "ahb1", | 266 | static SUNXI_CCU_GATE(ahb1_mmc3_clk, "ahb1-mmc3", "ahb1", |
| 267 | 0x060, BIT(12), 0); | 267 | 0x060, BIT(11), 0); |
| 268 | static SUNXI_CCU_GATE(ahb1_nand1_clk, "ahb1-nand1", "ahb1", | 268 | static SUNXI_CCU_GATE(ahb1_nand1_clk, "ahb1-nand1", "ahb1", |
| 269 | 0x060, BIT(13), 0); | 269 | 0x060, BIT(12), 0); |
| 270 | static SUNXI_CCU_GATE(ahb1_nand0_clk, "ahb1-nand0", "ahb1", | 270 | static SUNXI_CCU_GATE(ahb1_nand0_clk, "ahb1-nand0", "ahb1", |
| 271 | 0x060, BIT(13), 0); | 271 | 0x060, BIT(13), 0); |
| 272 | static SUNXI_CCU_GATE(ahb1_sdram_clk, "ahb1-sdram", "ahb1", | 272 | static SUNXI_CCU_GATE(ahb1_sdram_clk, "ahb1-sdram", "ahb1", |
| @@ -542,7 +542,7 @@ static struct ccu_reset_map sun8i_v3s_ccu_resets[] = { · 541 unmodified lines | |||
| 542 | [RST_BUS_OHCI0] = { 0x2c0, BIT(29) }, | 542 | [RST_BUS_OHCI0] = { 0x2c0, BIT(29) }, |
| 543 | 543 | ||
| 544 | [RST_BUS_VE] = { 0x2c4, BIT(0) }, | 544 | [RST_BUS_VE] = { 0x2c4, BIT(0) }, |
| 545 | [RST_BUS_TCON0] = { 0x2c4, BIT(3) }, | 545 | [RST_BUS_TCON0] = { 0x2c4, BIT(4) }, |
| 546 | [RST_BUS_CSI] = { 0x2c4, BIT(8) }, | 546 | [RST_BUS_CSI] = { 0x2c4, BIT(8) }, |
| 547 | [RST_BUS_DE] = { 0x2c4, BIT(12) }, | 547 | [RST_BUS_DE] = { 0x2c4, BIT(12) }, |
| 548 | [RST_BUS_DBG] = { 0x2c4, BIT(31) }, | 548 | [RST_BUS_DBG] = { 0x2c4, BIT(31) }, |
| @@ -187,8 +187,8 @@ static int scmi_cpufreq_exit(struct cpufreq_policy *policy) · 186 unmodified lines | |||
| 187 | 187 | ||
| 188 | cpufreq_cooling_unregister(priv->cdev); | 188 | cpufreq_cooling_unregister(priv->cdev); |
| 189 | dev_pm_opp_free_cpufreq_table(priv->cpu_dev, &policy->freq_table); | 189 | dev_pm_opp_free_cpufreq_table(priv->cpu_dev, &policy->freq_table); |
| 190 | kfree(priv); | ||
| 191 | dev_pm_opp_remove_all_dynamic(priv->cpu_dev); | 190 | dev_pm_opp_remove_all_dynamic(priv->cpu_dev); |
| 191 | kfree(priv); | ||
| 192 | 192 | ||
| 193 | return 0; | 193 | return 0; |
| 194 | } | 194 | } |
| @@ -30,7 +30,7 @@ static inline int cc_pm_init(struct cc_drvdata *drvdata) · 29 unmodified lines | |||
| 30 | return 0; | 30 | return 0; |
| 31 | } | 31 | } |
| 32 | 32 | ||
| 33 | static void cc_pm_go(struct cc_drvdata *drvdata) {} | 33 | static inline void cc_pm_go(struct cc_drvdata *drvdata) {} |
| 34 | 34 | ||
| 35 | static inline void cc_pm_fini(struct cc_drvdata *drvdata) {} | 35 | static inline void cc_pm_fini(struct cc_drvdata *drvdata) {} |
| 36 | 36 | ||
| @@ -30,6 +30,7 @@ · 29 unmodified lines | |||
| 30 | #define GPIO_REG_EDGE 0xA0 | 30 | #define GPIO_REG_EDGE 0xA0 |
| 31 | 31 | ||
| 32 | struct mtk_gc { | 32 | struct mtk_gc { |
| 33 | struct irq_chip irq_chip; | ||
| 33 | struct gpio_chip chip; | 34 | struct gpio_chip chip; |
| 34 | spinlock_t lock; | 35 | spinlock_t lock; |
| 35 | int bank; | 36 | int bank; |
| @@ -189,13 +190,6 @@ mediatek_gpio_irq_type(struct irq_data *d, unsigned int type) · 153 unmodified lines | |||
| 189 | return 0; | 190 | return 0; |
| 190 | } | 191 | } |
| 191 | 192 | ||
| 192 | static struct irq_chip mediatek_gpio_irq_chip = { | ||
| 193 | .irq_unmask = mediatek_gpio_irq_unmask, | ||
| 194 | .irq_mask = mediatek_gpio_irq_mask, | ||
| 195 | .irq_mask_ack = mediatek_gpio_irq_mask, | ||
| 196 | .irq_set_type = mediatek_gpio_irq_type, | ||
| 197 | }; | ||
| 198 | |||
| 199 | static int | 193 | static int |
| 200 | mediatek_gpio_xlate(struct gpio_chip *chip, | 194 | mediatek_gpio_xlate(struct gpio_chip *chip, |
| 201 | const struct of_phandle_args *spec, u32 *flags) | 195 | const struct of_phandle_args *spec, u32 *flags) |
| @@ -254,6 +248,13 @@ mediatek_gpio_bank_probe(struct device *dev, · 52 unmodified lines | |||
| 254 | return ret; | 248 | return ret; |
| 255 | } | 249 | } |
| 256 | 250 | ||
| 251 | rg->irq_chip.name = dev_name(dev); | ||
| 252 | rg->irq_chip.parent_device = dev; | ||
| 253 | rg->irq_chip.irq_unmask = mediatek_gpio_irq_unmask; | ||
| 254 | rg->irq_chip.irq_mask = mediatek_gpio_irq_mask; | ||
| 255 | rg->irq_chip.irq_mask_ack = mediatek_gpio_irq_mask; | ||
| 256 | rg->irq_chip.irq_set_type = mediatek_gpio_irq_type; | ||
| 257 | |||
| 257 | if (mtk->gpio_irq) { | 258 | if (mtk->gpio_irq) { |
| 258 | /* | 259 | /* |
| 259 | * Manually request the irq here instead of passing | 260 | * Manually request the irq here instead of passing |
| @@ -270,14 +271,14 @@ mediatek_gpio_bank_probe(struct device *dev, · 10 unmodified lines | |||
| 270 | return ret; | 271 | return ret; |
| 271 | } | 272 | } |
| 272 | 273 | ||
| 273 | ret = gpiochip_irqchip_add(&rg->chip, &mediatek_gpio_irq_chip, | 274 | ret = gpiochip_irqchip_add(&rg->chip, &rg->irq_chip, |
| 274 | 0, handle_simple_irq, IRQ_TYPE_NONE); | 275 | 0, handle_simple_irq, IRQ_TYPE_NONE); |
| 275 | if (ret) { | 276 | if (ret) { |
| 276 | dev_err(dev, "failed to add gpiochip_irqchip\n"); | 277 | dev_err(dev, "failed to add gpiochip_irqchip\n"); |
| 277 | return ret; | 278 | return ret; |
| 278 | } | 279 | } |
| 279 | 280 | ||
| 280 | gpiochip_set_chained_irqchip(&rg->chip, &mediatek_gpio_irq_chip, | 281 | gpiochip_set_chained_irqchip(&rg->chip, &rg->irq_chip, |
| 281 | mtk->gpio_irq, NULL); | 282 | mtk->gpio_irq, NULL); |
| 282 | } | 283 | } |
| 283 | 284 | ||
| @@ -310,7 +311,6 @@ mediatek_gpio_probe(struct platform_device *pdev) · 26 unmodified lines | |||
| 310 | mtk->gpio_irq = irq_of_parse_and_map(np, 0); | 311 | mtk->gpio_irq = irq_of_parse_and_map(np, 0); |
| 311 | mtk->dev = dev; | 312 | mtk->dev = dev; |
| 312 | platform_set_drvdata(pdev, mtk); | 313 | platform_set_drvdata(pdev, mtk); |
| 313 | mediatek_gpio_irq_chip.name = dev_name(dev); | ||
| 314 | 314 | ||
| 315 | for (i = 0; i < MTK_BANK_CNT; i++) { | 315 | for (i = 0; i < MTK_BANK_CNT; i++) { |
| 316 | ret = mediatek_gpio_bank_probe(dev, np, i); | 316 | ret = mediatek_gpio_bank_probe(dev, np, i); |
| @@ -245,6 +245,7 @@ static bool pxa_gpio_has_pinctrl(void) · 244 unmodified lines | |||
| 245 | { | 245 | { |
| 246 | switch (gpio_type) { | 246 | switch (gpio_type) { |
| 247 | case PXA3XX_GPIO: | 247 | case PXA3XX_GPIO: |
| 248 | case MMP2_GPIO: | ||
| 248 | return false; | 249 | return false; |
| 249 | 250 | ||
| 250 | default: | 251 | default: |
| @@ -411,6 +411,8 @@ struct amdgpu_fpriv { · 410 unmodified lines | |||
| 411 | struct amdgpu_ctx_mgr ctx_mgr; | 411 | struct amdgpu_ctx_mgr ctx_mgr; |
| 412 | }; | 412 | }; |
| 413 | 413 | ||
| 414 | int amdgpu_file_to_fpriv(struct file *filp, struct amdgpu_fpriv **fpriv); | ||
| 415 | |||
| 414 | int amdgpu_ib_get(struct amdgpu_device *adev, struct amdgpu_vm *vm, | 416 | int amdgpu_ib_get(struct amdgpu_device *adev, struct amdgpu_vm *vm, |
| 415 | unsigned size, struct amdgpu_ib *ib); | 417 | unsigned size, struct amdgpu_ib *ib); |
| 416 | void amdgpu_ib_free(struct amdgpu_device *adev, struct amdgpu_ib *ib, | 418 | void amdgpu_ib_free(struct amdgpu_device *adev, struct amdgpu_ib *ib, |
| @@ -131,7 +131,7 @@ static void amdgpu_doorbell_get_kfd_info(struct amdgpu_device *adev, · 130 unmodified lines | |||
| 131 | 131 | ||
| 132 | void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) | 132 | void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) |
| 133 | { | 133 | { |
| 134 | int i, n; | 134 | int i; |
| 135 | int last_valid_bit; | 135 | int last_valid_bit; |
| 136 | 136 | ||
| 137 | if (adev->kfd.dev) { | 137 | if (adev->kfd.dev) { |
| @@ -142,7 +142,9 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) · 4 unmodified lines | |||
| 142 | .gpuvm_size = min(adev->vm_manager.max_pfn | 142 | .gpuvm_size = min(adev->vm_manager.max_pfn |
| 143 | << AMDGPU_GPU_PAGE_SHIFT, | 143 | << AMDGPU_GPU_PAGE_SHIFT, |
| 144 | AMDGPU_GMC_HOLE_START), | 144 | AMDGPU_GMC_HOLE_START), |
| 145 | .drm_render_minor = adev->ddev->render->index | 145 | .drm_render_minor = adev->ddev->render->index, |
| 146 | .sdma_doorbell_idx = adev->doorbell_index.sdma_engine, | ||
| 147 | |||
| 146 | }; | 148 | }; |
| 147 | 149 | ||
| 148 | /* this is going to have a few of the MSBs set that we need to | 150 | /* this is going to have a few of the MSBs set that we need to |
| @@ -172,35 +174,20 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) · 23 unmodified lines | |||
| 172 | &gpu_resources.doorbell_aperture_size, | 174 | &gpu_resources.doorbell_aperture_size, |
| 173 | &gpu_resources.doorbell_start_offset); | 175 | &gpu_resources.doorbell_start_offset); |
| 174 | 176 | ||
| 175 | if (adev->asic_type < CHIP_VEGA10) { | 177 | /* Since SOC15, BIF starts to statically use the |
| 176 | kgd2kfd_device_init(adev->kfd.dev, &gpu_resources); | 178 | * lower 12 bits of doorbell addresses for routing |
| 177 | return; | 179 | * based on settings in registers like |
| 178 | } | 180 | * SDMA0_DOORBELL_RANGE etc.. |
| 179 | 181 | * In order to route a doorbell to CP engine, the lower | |
| 180 | n = (adev->asic_type < CHIP_VEGA20) ? 2 : 8; | 182 | * 12 bits of its address has to be outside the range |
| 181 | 183 | * set for SDMA, VCN, and IH blocks. | |
| 182 | for (i = 0; i < n; i += 2) { | ||
| 183 | /* On SOC15 the BIF is involved in routing | ||
| 184 | * doorbells using the low 12 bits of the | ||
| 185 | * address. Communicate the assignments to | ||
| 186 | * KFD. KFD uses two doorbell pages per | ||
| 187 | * process in case of 64-bit doorbells so we | ||
| 188 | * can use each doorbell assignment twice. | ||
| 189 | */ | ||
| 190 | gpu_resources.sdma_doorbell[0][i] = | ||
| 191 | adev->doorbell_index.sdma_engine[0] + (i >> 1); | ||
| 192 | gpu_resources.sdma_doorbell[0][i+1] = | ||
| 193 | adev->doorbell_index.sdma_engine[0] + 0x200 + (i >> 1); | ||
| 194 | gpu_resources.sdma_doorbell[1][i] = | ||
| 195 | adev->doorbell_index.sdma_engine[1] + (i >> 1); | ||
| 196 | gpu_resources.sdma_doorbell[1][i+1] = | ||
| 197 | adev->doorbell_index.sdma_engine[1] + 0x200 + (i >> 1); | ||
| 198 | } | ||
| 199 | /* Doorbells 0x0e0-0ff and 0x2e0-2ff are reserved for | ||
| 200 | * SDMA, IH and VCN. So don't use them for the CP. | ||
| 201 | */ | 184 | */ |
| 202 | gpu_resources.reserved_doorbell_mask = 0x1e0; | 185 | if (adev->asic_type >= CHIP_VEGA10) { |
| 203 | gpu_resources.reserved_doorbell_val = 0x0e0; | 186 | gpu_resources.non_cp_doorbells_start = |
| 187 | adev->doorbell_index.first_non_cp; | ||
| 188 | gpu_resources.non_cp_doorbells_end = | ||
| 189 | adev->doorbell_index.last_non_cp; | ||
| 190 | } | ||
| 204 | 191 | ||
| 205 | kgd2kfd_device_init(adev->kfd.dev, &gpu_resources); | 192 | kgd2kfd_device_init(adev->kfd.dev, &gpu_resources); |
| 206 | } | 193 | } |
| @@ -204,38 +204,25 @@ void amdgpu_amdkfd_unreserve_memory_limit(struct amdgpu_bo *bo) · 203 unmodified lines | |||
| 204 | } | 204 | } |
| 205 | 205 | ||
| 206 | 206 | ||
| 207 | /* amdgpu_amdkfd_remove_eviction_fence - Removes eviction fence(s) from BO's | 207 | /* amdgpu_amdkfd_remove_eviction_fence - Removes eviction fence from BO's |
| 208 | * reservation object. | 208 | * reservation object. |
| 209 | * | 209 | * |
| 210 | * @bo: [IN] Remove eviction fence(s) from this BO | 210 | * @bo: [IN] Remove eviction fence(s) from this BO |
| 211 | * @ef: [IN] If ef is specified, then this eviction fence is removed if it | 211 | * @ef: [IN] This eviction fence is removed if it |
| 212 | * is present in the shared list. | 212 | * is present in the shared list. |
| 213 | * @ef_list: [OUT] Returns list of eviction fences. These fences are removed | ||
| 214 | * from BO's reservation object shared list. | ||
| 215 | * @ef_count: [OUT] Number of fences in ef_list. | ||
| 216 | * | 213 | * |
| 217 | * NOTE: If called with ef_list, then amdgpu_amdkfd_add_eviction_fence must be | ||
| 218 | * called to restore the eviction fences and to avoid memory leak. This is | ||
| 219 | * useful for shared BOs. | ||
| 220 | * NOTE: Must be called with BO reserved i.e. bo->tbo.resv->lock held. | 214 | * NOTE: Must be called with BO reserved i.e. bo->tbo.resv->lock held. |
| 221 | */ | 215 | */ |
| 222 | static int amdgpu_amdkfd_remove_eviction_fence(struct amdgpu_bo *bo, | 216 | static int amdgpu_amdkfd_remove_eviction_fence(struct amdgpu_bo *bo, |
| 223 | struct amdgpu_amdkfd_fence *ef, | 217 | struct amdgpu_amdkfd_fence *ef) |
| 224 | struct amdgpu_amdkfd_fence ***ef_list, | ||
| 225 | unsigned int *ef_count) | ||
| 226 | { | 218 | { |
| 227 | struct reservation_object *resv = bo->tbo.resv; | 219 | struct reservation_object *resv = bo->tbo.resv; |
| 228 | struct reservation_object_list *old, *new; | 220 | struct reservation_object_list *old, *new; |
| 229 | unsigned int i, j, k; | 221 | unsigned int i, j, k; |
| 230 | 222 | ||
| 231 | if (!ef && !ef_list) | 223 | if (!ef) |
| 232 | return -EINVAL; | 224 | return -EINVAL; |
| 233 | 225 | ||
| 234 | if (ef_list) { | ||
| 235 | *ef_list = NULL; | ||
| 236 | *ef_count = 0; | ||
| 237 | } | ||
| 238 | |||
| 239 | old = reservation_object_get_list(resv); | 226 | old = reservation_object_get_list(resv); |
| 240 | if (!old) | 227 | if (!old) |
| 241 | return 0; | 228 | return 0; |
| @@ -254,8 +241,7 @@ static int amdgpu_amdkfd_remove_eviction_fence(struct amdgpu_bo *bo, · 12 unmodified lines | |||
| 254 | f = rcu_dereference_protected(old->shared[i], | 241 | f = rcu_dereference_protected(old->shared[i], |
| 255 | reservation_object_held(resv)); | 242 | reservation_object_held(resv)); |
| 256 | 243 | ||
| 257 | if ((ef && f->context == ef->base.context) || | 244 | if (f->context == ef->base.context) |
| 258 | (!ef && to_amdgpu_amdkfd_fence(f))) | ||
| 259 | RCU_INIT_POINTER(new->shared[--j], f); | 245 | RCU_INIT_POINTER(new->shared[--j], f); |
| 260 | else | 246 | else |
| 261 | RCU_INIT_POINTER(new->shared[k++], f); | 247 | RCU_INIT_POINTER(new->shared[k++], f); |
| @@ -263,21 +249,6 @@ static int amdgpu_amdkfd_remove_eviction_fence(struct amdgpu_bo *bo, · 1 unmodified line | |||
| 263 | new->shared_max = old->shared_max; | 249 | new->shared_max = old->shared_max; |
| 264 | new->shared_count = k; | 250 | new->shared_count = k; |
| 265 | 251 | ||
| 266 | if (!ef) { | ||
| 267 | unsigned int count = old->shared_count - j; | ||
| 268 | |||
| 269 | /* Alloc memory for count number of eviction fence pointers. | ||
| 270 | * Fill the ef_list array and ef_count | ||
| 271 | */ | ||
| 272 | *ef_list = kcalloc(count, sizeof(**ef_list), GFP_KERNEL); | ||
| 273 | *ef_count = count; | ||
| 274 | |||
| 275 | if (!*ef_list) { | ||
| 276 | kfree(new); | ||
| 277 | return -ENOMEM; | ||
| 278 | } | ||
| 279 | } | ||
| 280 | |||
| 281 | /* Install the new fence list, seqcount provides the barriers */ | 252 | /* Install the new fence list, seqcount provides the barriers */ |
| 282 | preempt_disable(); | 253 | preempt_disable(); |
| 283 | write_seqcount_begin(&resv->seq); | 254 | write_seqcount_begin(&resv->seq); |
| @@ -291,46 +262,13 @@ static int amdgpu_amdkfd_remove_eviction_fence(struct amdgpu_bo *bo, · 7 unmodified lines | |||
| 291 | 262 | ||
| 292 | f = rcu_dereference_protected(new->shared[i], | 263 | f = rcu_dereference_protected(new->shared[i], |
| 293 | reservation_object_held(resv)); | 264 | reservation_object_held(resv)); |
| 294 | if (!ef) | 265 | dma_fence_put(f); |
| 295 | (*ef_list)[k++] = to_amdgpu_amdkfd_fence(f); | ||
| 296 | else | ||
| 297 | dma_fence_put(f); | ||
| 298 | } | 266 | } |
| 299 | kfree_rcu(old, rcu); | 267 | kfree_rcu(old, rcu); |
| 300 | 268 | ||
| 301 | return 0; | 269 | return 0; |
| 302 | } | 270 | } |
| 303 | 271 | ||
| 304 | /* amdgpu_amdkfd_add_eviction_fence - Adds eviction fence(s) back into BO's | ||
| 305 | * reservation object. | ||
| 306 | * | ||
| 307 | * @bo: [IN] Add eviction fences to this BO | ||
| 308 | * @ef_list: [IN] List of eviction fences to be added | ||
| 309 | * @ef_count: [IN] Number of fences in ef_list. | ||
| 310 | * | ||
| 311 | * NOTE: Must call amdgpu_amdkfd_remove_eviction_fence before calling this | ||
| 312 | * function. | ||
| 313 | */ | ||
| 314 | static void amdgpu_amdkfd_add_eviction_fence(struct amdgpu_bo *bo, | ||
| 315 | struct amdgpu_amdkfd_fence **ef_list, | ||
| 316 | unsigned int ef_count) | ||
| 317 | { | ||
| 318 | int i; | ||
| 319 | |||
| 320 | if (!ef_list || !ef_count) | ||
| 321 | return; | ||
| 322 | |||
| 323 | for (i = 0; i < ef_count; i++) { | ||
| 324 | amdgpu_bo_fence(bo, &ef_list[i]->base, true); | ||
| 325 | /* Re-adding the fence takes an additional reference. Drop that | ||
| 326 | * reference. | ||
| 327 | */ | ||
| 328 | dma_fence_put(&ef_list[i]->base); | ||
| 329 | } | ||
| 330 | |||
| 331 | kfree(ef_list); | ||
| 332 | } | ||
| 333 | |||
| 334 | static int amdgpu_amdkfd_bo_validate(struct amdgpu_bo *bo, uint32_t domain, | 272 | static int amdgpu_amdkfd_bo_validate(struct amdgpu_bo *bo, uint32_t domain, |
| 335 | bool wait) | 273 | bool wait) |
| 336 | { | 274 | { |
| @@ -346,18 +284,8 @@ static int amdgpu_amdkfd_bo_validate(struct amdgpu_bo *bo, uint32_t domain, · 9 unmodified lines | |||
| 346 | ret = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); | 284 | ret = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); |
| 347 | if (ret) | 285 | if (ret) |
| 348 | goto validate_fail; | 286 | goto validate_fail; |
| 349 | if (wait) { | 287 | if (wait) |
| 350 | struct amdgpu_amdkfd_fence **ef_list; | 288 | amdgpu_bo_sync_wait(bo, AMDGPU_FENCE_OWNER_KFD, false); |
| 351 | unsigned int ef_count; | ||
| 352 | |||
| 353 | ret = amdgpu_amdkfd_remove_eviction_fence(bo, NULL, &ef_list, | ||
| 354 | &ef_count); | ||
| 355 | if (ret) | ||
| 356 | goto validate_fail; | ||
| 357 | |||
| 358 | ttm_bo_wait(&bo->tbo, false, false); | ||
| 359 | amdgpu_amdkfd_add_eviction_fence(bo, ef_list, ef_count); | ||
| 360 | } | ||
| 361 | 289 | ||
| 362 | validate_fail: | 290 | validate_fail: |
| 363 | return ret; | 291 | return ret; |
| @@ -444,7 +372,6 @@ static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, · 80 unmodified lines | |||
| 444 | { | 372 | { |
| 445 | int ret; | 373 | int ret; |
| 446 | struct kfd_bo_va_list *bo_va_entry; | 374 | struct kfd_bo_va_list *bo_va_entry; |
| 447 | struct amdgpu_bo *pd = vm->root.base.bo; | ||
| 448 | struct amdgpu_bo *bo = mem->bo; | 375 | struct amdgpu_bo *bo = mem->bo; |
| 449 | uint64_t va = mem->va; | 376 | uint64_t va = mem->va; |
| 450 | struct list_head *list_bo_va = &mem->bo_va_list; | 377 | struct list_head *list_bo_va = &mem->bo_va_list; |
| @@ -484,14 +411,8 @@ static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, · 33 unmodified lines | |||
| 484 | *p_bo_va_entry = bo_va_entry; | 411 | *p_bo_va_entry = bo_va_entry; |
| 485 | 412 | ||
| 486 | /* Allocate new page tables if needed and validate | 413 | /* Allocate new page tables if needed and validate |
| 487 | * them. Clearing of new page tables and validate need to wait | 414 | * them. |
| 488 | * on move fences. We don't want that to trigger the eviction | ||
| 489 | * fence, so remove it temporarily. | ||
| 490 | */ | 415 | */ |
| 491 | amdgpu_amdkfd_remove_eviction_fence(pd, | ||
| 492 | vm->process_info->eviction_fence, | ||
| 493 | NULL, NULL); | ||
| 494 | |||
| 495 | ret = amdgpu_vm_alloc_pts(adev, vm, va, amdgpu_bo_size(bo)); | 416 | ret = amdgpu_vm_alloc_pts(adev, vm, va, amdgpu_bo_size(bo)); |
| 496 | if (ret) { | 417 | if (ret) { |
| 497 | pr_err("Failed to allocate pts, err=%d\n", ret); | 418 | pr_err("Failed to allocate pts, err=%d\n", ret); |
| @@ -504,13 +425,9 @@ static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, · 6 unmodified lines | |||
| 504 | goto err_alloc_pts; | 425 | goto err_alloc_pts; |
| 505 | } | 426 | } |
| 506 | 427 | ||
| 507 | /* Add the eviction fence back */ | ||
| 508 | amdgpu_bo_fence(pd, &vm->process_info->eviction_fence->base, true); | ||
| 509 | |||
| 510 | return 0; | 428 | return 0; |
| 511 | 429 | ||
| 512 | err_alloc_pts: | 430 | err_alloc_pts: |
| 513 | amdgpu_bo_fence(pd, &vm->process_info->eviction_fence->base, true); | ||
| 514 | amdgpu_vm_bo_rmv(adev, bo_va_entry->bo_va); | 431 | amdgpu_vm_bo_rmv(adev, bo_va_entry->bo_va); |
| 515 | list_del(&bo_va_entry->bo_list); | 432 | list_del(&bo_va_entry->bo_list); |
| 516 | err_vmadd: | 433 | err_vmadd: |
| @@ -809,24 +726,11 @@ static int unmap_bo_from_gpuvm(struct amdgpu_device *adev, · 292 unmodified lines | |||
| 809 | { | 726 | { |
| 810 | struct amdgpu_bo_va *bo_va = entry->bo_va; | 727 | struct amdgpu_bo_va *bo_va = entry->bo_va; |
| 811 | struct amdgpu_vm *vm = bo_va->base.vm; | 728 | struct amdgpu_vm *vm = bo_va->base.vm; |
| 812 | struct amdgpu_bo *pd = vm->root.base.bo; | ||
| 813 | 729 | ||
| 814 | /* Remove eviction fence from PD (and thereby from PTs too as | ||
| 815 | * they share the resv. object). Otherwise during PT update | ||
| 816 | * job (see amdgpu_vm_bo_update_mapping), eviction fence would | ||
| 817 | * get added to job->sync object and job execution would | ||
| 818 | * trigger the eviction fence. | ||
| 819 | */ | ||
| 820 | amdgpu_amdkfd_remove_eviction_fence(pd, | ||
| 821 | vm->process_info->eviction_fence, | ||
| 822 | NULL, NULL); | ||
| 823 | amdgpu_vm_bo_unmap(adev, bo_va, entry->va); | 730 | amdgpu_vm_bo_unmap(adev, bo_va, entry->va); |
| 824 | 731 | ||
| 825 | amdgpu_vm_clear_freed(adev, vm, &bo_va->last_pt_update); | 732 | amdgpu_vm_clear_freed(adev, vm, &bo_va->last_pt_update); |
| 826 | 733 | ||
| 827 | /* Add the eviction fence back */ | ||
| 828 | amdgpu_bo_fence(pd, &vm->process_info->eviction_fence->base, true); | ||
| 829 | |||
| 830 | amdgpu_sync_fence(NULL, sync, bo_va->last_pt_update, false); | 734 | amdgpu_sync_fence(NULL, sync, bo_va->last_pt_update, false); |
| 831 | 735 | ||
| 832 | return 0; | 736 | return 0; |
| @@ -1002,7 +906,7 @@ static int init_kfd_vm(struct amdgpu_vm *vm, void **process_info, · 169 unmodified lines | |||
| 1002 | pr_err("validate_pt_pd_bos() failed\n"); | 906 | pr_err("validate_pt_pd_bos() failed\n"); |
| 1003 | goto validate_pd_fail; | 907 | goto validate_pd_fail; |
| 1004 | } | 908 | } |
| 1005 | ret = ttm_bo_wait(&vm->root.base.bo->tbo, false, false); | 909 | amdgpu_bo_sync_wait(vm->root.base.bo, AMDGPU_FENCE_OWNER_KFD, false); |
| 1006 | if (ret) | 910 | if (ret) |
| 1007 | goto wait_pd_fail; | 911 | goto wait_pd_fail; |
| 1008 | amdgpu_bo_fence(vm->root.base.bo, | 912 | amdgpu_bo_fence(vm->root.base.bo, |
| @@ -1389,8 +1293,7 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( · 380 unmodified lines | |||
| 1389 | * attached | 1293 | * attached |
| 1390 | */ | 1294 | */ |
| 1391 | amdgpu_amdkfd_remove_eviction_fence(mem->bo, | 1295 | amdgpu_amdkfd_remove_eviction_fence(mem->bo, |
| 1392 | process_info->eviction_fence, | 1296 | process_info->eviction_fence); |
| 1393 | NULL, NULL); | ||
| 1394 | pr_debug("Release VA 0x%llx - 0x%llx\n", mem->va, | 1297 | pr_debug("Release VA 0x%llx - 0x%llx\n", mem->va, |
| 1395 | mem->va + bo_size * (1 + mem->aql_queue)); | 1298 | mem->va + bo_size * (1 + mem->aql_queue)); |
| 1396 | 1299 | ||
| @@ -1617,8 +1520,7 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( · 220 unmodified lines | |||
| 1617 | if (mem->mapped_to_gpu_memory == 0 && | 1520 | if (mem->mapped_to_gpu_memory == 0 && |
| 1618 | !amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm) && !mem->bo->pin_count) | 1521 | !amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm) && !mem->bo->pin_count) |
| 1619 | amdgpu_amdkfd_remove_eviction_fence(mem->bo, | 1522 | amdgpu_amdkfd_remove_eviction_fence(mem->bo, |
| 1620 | process_info->eviction_fence, | 1523 | process_info->eviction_fence); |
| 1621 | NULL, NULL); | ||
| 1622 | 1524 | ||
| 1623 | unreserve_out: | 1525 | unreserve_out: |
| 1624 | unreserve_bo_and_vms(&ctx, false, false); | 1526 | unreserve_bo_and_vms(&ctx, false, false); |
| @@ -1679,7 +1581,7 @@ int amdgpu_amdkfd_gpuvm_map_gtt_bo_to_kernel(struct kgd_dev *kgd, · 54 unmodified lines | |||
| 1679 | } | 1581 | } |
| 1680 | 1582 | ||
| 1681 | amdgpu_amdkfd_remove_eviction_fence( | 1583 | amdgpu_amdkfd_remove_eviction_fence( |
| 1682 | bo, mem->process_info->eviction_fence, NULL, NULL); | 1584 | bo, mem->process_info->eviction_fence); |
| 1683 | list_del_init(&mem->validate_list.head); | 1585 | list_del_init(&mem->validate_list.head); |
| 1684 | 1586 | ||
| 1685 | if (size) | 1587 | if (size) |
| @@ -1945,16 +1847,6 @@ static int validate_invalid_user_pages(struct amdkfd_process_info *process_info) · 259 unmodified lines | |||
| 1945 | 1847 | ||
| 1946 | amdgpu_sync_create(&sync); | 1848 | amdgpu_sync_create(&sync); |
| 1947 | 1849 | ||
| 1948 | /* Avoid triggering eviction fences when unmapping invalid | ||
| 1949 | * userptr BOs (waits for all fences, doesn't use | ||
| 1950 | * FENCE_OWNER_VM) | ||
| 1951 | */ | ||
| 1952 | list_for_each_entry(peer_vm, &process_info->vm_list_head, | ||
| 1953 | vm_list_node) | ||
| 1954 | amdgpu_amdkfd_remove_eviction_fence(peer_vm->root.base.bo, | ||
| 1955 | process_info->eviction_fence, | ||
| 1956 | NULL, NULL); | ||
| 1957 | |||
| 1958 | ret = process_validate_vms(process_info); | 1850 | ret = process_validate_vms(process_info); |
| 1959 | if (ret) | 1851 | if (ret) |
| 1960 | goto unreserve_out; | 1852 | goto unreserve_out; |
| @@ -2015,10 +1907,6 @@ static int validate_invalid_user_pages(struct amdkfd_process_info *process_info) · 54 unmodified lines | |||
| 2015 | ret = process_update_pds(process_info, &sync); | 1907 | ret = process_update_pds(process_info, &sync); |
| 2016 | 1908 | ||
| 2017 | unreserve_out: | 1909 | unreserve_out: |
| 2018 | list_for_each_entry(peer_vm, &process_info->vm_list_head, | ||
| 2019 | vm_list_node) | ||
| 2020 | amdgpu_bo_fence(peer_vm->root.base.bo, | ||
| 2021 | &process_info->eviction_fence->base, true); | ||
| 2022 | ttm_eu_backoff_reservation(&ticket, &resv_list); | 1910 | ttm_eu_backoff_reservation(&ticket, &resv_list); |
| 2023 | amdgpu_sync_wait(&sync, false); | 1911 | amdgpu_sync_wait(&sync, false); |
| 2024 | amdgpu_sync_free(&sync); | 1912 | amdgpu_sync_free(&sync); |
| @@ -124,6 +124,7 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, · 123 unmodified lines | |||
| 124 | struct amdgpu_ring *rings[AMDGPU_MAX_RINGS]; | 124 | struct amdgpu_ring *rings[AMDGPU_MAX_RINGS]; |
| 125 | struct drm_sched_rq *rqs[AMDGPU_MAX_RINGS]; | 125 | struct drm_sched_rq *rqs[AMDGPU_MAX_RINGS]; |
| 126 | unsigned num_rings; | 126 | unsigned num_rings; |
| 127 | unsigned num_rqs = 0; | ||
| 127 | 128 | ||
| 128 | switch (i) { | 129 | switch (i) { |
| 129 | case AMDGPU_HW_IP_GFX: | 130 | case AMDGPU_HW_IP_GFX: |
| @@ -166,12 +167,16 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, · 36 unmodified lines | |||
| 166 | break; | 167 | break; |
| 167 | } | 168 | } |
| 168 | 169 | ||
| 169 | for (j = 0; j < num_rings; ++j) | 170 | for (j = 0; j < num_rings; ++j) { |
| 170 | rqs[j] = &rings[j]->sched.sched_rq[priority]; | 171 | if (!rings[j]->adev) |
| 172 | continue; | ||
| 173 | |||
| 174 | rqs[num_rqs++] = &rings[j]->sched.sched_rq[priority]; | ||
| 175 | } | ||
| 171 | 176 | ||
| 172 | for (j = 0; j < amdgpu_ctx_num_entities[i]; ++j) | 177 | for (j = 0; j < amdgpu_ctx_num_entities[i]; ++j) |
| 173 | r = drm_sched_entity_init(&ctx->entities[i][j].entity, | 178 | r = drm_sched_entity_init(&ctx->entities[i][j].entity, |
| 174 | rqs, num_rings, &ctx->guilty); | 179 | rqs, num_rqs, &ctx->guilty); |
| 175 | if (r) | 180 | if (r) |
| 176 | goto error_cleanup_entities; | 181 | goto error_cleanup_entities; |
| 177 | } | 182 | } |
| @@ -158,9 +158,6 @@ static int amdgpu_debugfs_process_reg_op(bool read, struct file *f, · 157 unmodified lines | |||
| 158 | while (size) { | 158 | while (size) { |
| 159 | uint32_t value; | 159 | uint32_t value; |
| 160 | 160 | ||
| 161 | if (*pos > adev->rmmio_size) | ||
| 162 | goto end; | ||
| 163 | |||
| 164 | if (read) { | 161 | if (read) { |
| 165 | value = RREG32(*pos >> 2); | 162 | value = RREG32(*pos >> 2); |
| 166 | r = put_user(value, (uint32_t *)buf); | 163 | r = put_user(value, (uint32_t *)buf); |
| @@ -71,6 +71,8 @@ struct amdgpu_doorbell_index { · 70 unmodified lines | |||
| 71 | uint32_t vce_ring6_7; | 71 | uint32_t vce_ring6_7; |
| 72 | } uvd_vce; | 72 | } uvd_vce; |
| 73 | }; | 73 | }; |
| 74 | uint32_t first_non_cp; | ||
| 75 | uint32_t last_non_cp; | ||
| 74 | uint32_t max_assignment; | 76 | uint32_t max_assignment; |
| 75 | /* Per engine SDMA doorbell size in dword */ | 77 | /* Per engine SDMA doorbell size in dword */ |
| 76 | uint32_t sdma_doorbell_range; | 78 | uint32_t sdma_doorbell_range; |
| @@ -143,6 +145,10 @@ typedef enum _AMDGPU_VEGA20_DOORBELL_ASSIGNMENT · 66 unmodified lines | |||
| 143 | AMDGPU_VEGA20_DOORBELL64_VCE_RING2_3 = 0x18D, | 145 | AMDGPU_VEGA20_DOORBELL64_VCE_RING2_3 = 0x18D, |
| 144 | AMDGPU_VEGA20_DOORBELL64_VCE_RING4_5 = 0x18E, | 146 | AMDGPU_VEGA20_DOORBELL64_VCE_RING4_5 = 0x18E, |
| 145 | AMDGPU_VEGA20_DOORBELL64_VCE_RING6_7 = 0x18F, | 147 | AMDGPU_VEGA20_DOORBELL64_VCE_RING6_7 = 0x18F, |
| 148 | |||
| 149 | AMDGPU_VEGA20_DOORBELL64_FIRST_NON_CP = AMDGPU_VEGA20_DOORBELL_sDMA_ENGINE0, | ||
| 150 | AMDGPU_VEGA20_DOORBELL64_LAST_NON_CP = AMDGPU_VEGA20_DOORBELL64_VCE_RING6_7, | ||
| 151 | |||
| 146 | AMDGPU_VEGA20_DOORBELL_MAX_ASSIGNMENT = 0x18F, | 152 | AMDGPU_VEGA20_DOORBELL_MAX_ASSIGNMENT = 0x18F, |
| 147 | AMDGPU_VEGA20_DOORBELL_INVALID = 0xFFFF | 153 | AMDGPU_VEGA20_DOORBELL_INVALID = 0xFFFF |
| 148 | } AMDGPU_VEGA20_DOORBELL_ASSIGNMENT; | 154 | } AMDGPU_VEGA20_DOORBELL_ASSIGNMENT; |
| @@ -222,6 +228,9 @@ typedef enum _AMDGPU_DOORBELL64_ASSIGNMENT · 73 unmodified lines | |||
| 222 | AMDGPU_DOORBELL64_VCE_RING4_5 = 0xFE, | 228 | AMDGPU_DOORBELL64_VCE_RING4_5 = 0xFE, |
| 223 | AMDGPU_DOORBELL64_VCE_RING6_7 = 0xFF, | 229 | AMDGPU_DOORBELL64_VCE_RING6_7 = 0xFF, |
| 224 | 230 | ||
| 231 | AMDGPU_DOORBELL64_FIRST_NON_CP = AMDGPU_DOORBELL64_sDMA_ENGINE0, | ||
| 232 | AMDGPU_DOORBELL64_LAST_NON_CP = AMDGPU_DOORBELL64_VCE_RING6_7, | ||
| 233 | |||
| 225 | AMDGPU_DOORBELL64_MAX_ASSIGNMENT = 0xFF, | 234 | AMDGPU_DOORBELL64_MAX_ASSIGNMENT = 0xFF, |
| 226 | AMDGPU_DOORBELL64_INVALID = 0xFFFF | 235 | AMDGPU_DOORBELL64_INVALID = 0xFFFF |
| 227 | } AMDGPU_DOORBELL64_ASSIGNMENT; | 236 | } AMDGPU_DOORBELL64_ASSIGNMENT; |
| @@ -184,61 +184,6 @@ u32 amdgpu_dpm_get_vrefresh(struct amdgpu_device *adev) · 183 unmodified lines | |||
| 184 | return vrefresh; | 184 | return vrefresh; |
| 185 | } | 185 | } |
| 186 | 186 | ||
| 187 | void amdgpu_calculate_u_and_p(u32 i, u32 r_c, u32 p_b, | ||
| 188 | u32 *p, u32 *u) | ||
| 189 | { | ||
| 190 | u32 b_c = 0; | ||
| 191 | u32 i_c; | ||
| 192 | u32 tmp; | ||
| 193 | |||
| 194 | i_c = (i * r_c) / 100; | ||
| 195 | tmp = i_c >> p_b; | ||
| 196 | |||
| 197 | while (tmp) { | ||
| 198 | b_c++; | ||
| 199 | tmp >>= 1; | ||
| 200 | } | ||
| 201 | |||
| 202 | *u = (b_c + 1) / 2; | ||
| 203 | *p = i_c / (1 << (2 * (*u))); | ||
| 204 | } | ||
| 205 | |||
| 206 | int amdgpu_calculate_at(u32 t, u32 h, u32 fh, u32 fl, u32 *tl, u32 *th) | ||
| 207 | { | ||
| 208 | u32 k, a, ah, al; | ||
| 209 | u32 t1; | ||
| 210 | |||
| 211 | if ((fl == 0) || (fh == 0) || (fl > fh)) | ||
| 212 | return -EINVAL; | ||
| 213 | |||
| 214 | k = (100 * fh) / fl; | ||
| 215 | t1 = (t * (k - 100)); | ||
| 216 | a = (1000 * (100 * h + t1)) / (10000 + (t1 / 100)); | ||
| 217 | a = (a + 5) / 10; | ||
| 218 | ah = ((a * t) + 5000) / 10000; | ||
| 219 | al = a - ah; | ||
| 220 | |||
| 221 | *th = t - ah; | ||
| 222 | *tl = t + al; | ||
| 223 | |||
| 224 | return 0; | ||
| 225 | } | ||
| 226 | |||
| 227 | bool amdgpu_is_uvd_state(u32 class, u32 class2) | ||
| 228 | { | ||
| 229 | if (class & ATOM_PPLIB_CLASSIFICATION_UVDSTATE) | ||
| 230 | return true; | ||
| 231 | if (class & ATOM_PPLIB_CLASSIFICATION_HD2STATE) | ||
| 232 | return true; | ||
| 233 | if (class & ATOM_PPLIB_CLASSIFICATION_HDSTATE) | ||
| 234 | return true; | ||
| 235 | if (class & ATOM_PPLIB_CLASSIFICATION_SDSTATE) | ||
| 236 | return true; | ||
| 237 | if (class2 & ATOM_PPLIB_CLASSIFICATION2_MVC) | ||
| 238 | return true; | ||
| 239 | return false; | ||
| 240 | } | ||
| 241 | |||
| 242 | bool amdgpu_is_internal_thermal_sensor(enum amdgpu_int_thermal_type sensor) | 187 | bool amdgpu_is_internal_thermal_sensor(enum amdgpu_int_thermal_type sensor) |
| 243 | { | 188 | { |
| 244 | switch (sensor) { | 189 | switch (sensor) { |
| @@ -949,39 +894,6 @@ enum amdgpu_pcie_gen amdgpu_get_pcie_gen_support(struct amdgpu_device *adev, · 704 unmodified lines | |||
| 949 | return AMDGPU_PCIE_GEN1; | 894 | return AMDGPU_PCIE_GEN1; |
| 950 | } | 895 | } |
| 951 | 896 | ||
| 952 | u16 amdgpu_get_pcie_lane_support(struct amdgpu_device *adev, | ||
| 953 | u16 asic_lanes, | ||
| 954 | u16 default_lanes) | ||
| 955 | { | ||
| 956 | switch (asic_lanes) { | ||
| 957 | case 0: | ||
| 958 | default: | ||
| 959 | return default_lanes; | ||
| 960 | case 1: | ||
| 961 | return 1; | ||
| 962 | case 2: | ||
| 963 | return 2; | ||
| 964 | case 4: | ||
| 965 | return 4; | ||
| 966 | case 8: | ||
| 967 | return 8; | ||
| 968 | case 12: | ||
| 969 | return 12; | ||
| 970 | case 16: | ||
| 971 | return 16; | ||
| 972 | } | ||
| 973 | } | ||
| 974 | |||
| 975 | u8 amdgpu_encode_pci_lane_width(u32 lanes) | ||
| 976 | { | ||
| 977 | u8 encoded_lanes[] = { 0, 1, 2, 0, 3, 0, 0, 0, 4, 0, 0, 0, 5, 0, 0, 0, 6 }; | ||
| 978 | |||
| 979 | if (lanes > 16) | ||
| 980 | return 0; | ||
| 981 | |||
| 982 | return encoded_lanes[lanes]; | ||
| 983 | } | ||
| 984 | |||
| 985 | struct amd_vce_state* | 897 | struct amd_vce_state* |
| 986 | amdgpu_get_vce_clock_state(void *handle, u32 idx) | 898 | amdgpu_get_vce_clock_state(void *handle, u32 idx) |
| 987 | { | 899 | { |
| @@ -486,10 +486,6 @@ void amdgpu_dpm_print_ps_status(struct amdgpu_device *adev, · 485 unmodified lines | |||
| 486 | u32 amdgpu_dpm_get_vblank_time(struct amdgpu_device *adev); | 486 | u32 amdgpu_dpm_get_vblank_time(struct amdgpu_device *adev); |
| 487 | u32 amdgpu_dpm_get_vrefresh(struct amdgpu_device *adev); | 487 | u32 amdgpu_dpm_get_vrefresh(struct amdgpu_device *adev); |
| 488 | void amdgpu_dpm_get_active_displays(struct amdgpu_device *adev); | 488 | void amdgpu_dpm_get_active_displays(struct amdgpu_device *adev); |
| 489 | bool amdgpu_is_uvd_state(u32 class, u32 class2); | ||
| 490 | void amdgpu_calculate_u_and_p(u32 i, u32 r_c, u32 p_b, | ||
| 491 | u32 *p, u32 *u); | ||
| 492 | int amdgpu_calculate_at(u32 t, u32 h, u32 fh, u32 fl, u32 *tl, u32 *th); | ||
| 493 | 489 | ||
| 494 | bool amdgpu_is_internal_thermal_sensor(enum amdgpu_int_thermal_type sensor); | 490 | bool amdgpu_is_internal_thermal_sensor(enum amdgpu_int_thermal_type sensor); |
| 495 | 491 | ||
| @@ -505,11 +501,6 @@ enum amdgpu_pcie_gen amdgpu_get_pcie_gen_support(struct amdgpu_device *adev, · 9 unmodified lines | |||
| 505 | enum amdgpu_pcie_gen asic_gen, | 501 | enum amdgpu_pcie_gen asic_gen, |
| 506 | enum amdgpu_pcie_gen default_gen); | 502 | enum amdgpu_pcie_gen default_gen); |
| 507 | 503 | ||
| 508 | u16 amdgpu_get_pcie_lane_support(struct amdgpu_device *adev, | ||
| 509 | u16 asic_lanes, | ||
| 510 | u16 default_lanes); | ||
| 511 | u8 amdgpu_encode_pci_lane_width(u32 lanes); | ||
| 512 | |||
| 513 | struct amd_vce_state* | 504 | struct amd_vce_state* |
| 514 | amdgpu_get_vce_clock_state(void *handle, u32 idx); | 505 | amdgpu_get_vce_clock_state(void *handle, u32 idx); |
| 515 | 506 | ||
| @@ -73,9 +73,10 @@ · 72 unmodified lines | |||
| 73 | * - 3.27.0 - Add new chunk to to AMDGPU_CS to enable BO_LIST creation. | 73 | * - 3.27.0 - Add new chunk to to AMDGPU_CS to enable BO_LIST creation. |
| 74 | * - 3.28.0 - Add AMDGPU_CHUNK_ID_SCHEDULED_DEPENDENCIES | 74 | * - 3.28.0 - Add AMDGPU_CHUNK_ID_SCHEDULED_DEPENDENCIES |
| 75 | * - 3.29.0 - Add AMDGPU_IB_FLAG_RESET_GDS_MAX_WAVE_ID | 75 | * - 3.29.0 - Add AMDGPU_IB_FLAG_RESET_GDS_MAX_WAVE_ID |
| 76 | * - 3.30.0 - Add AMDGPU_SCHED_OP_CONTEXT_PRIORITY_OVERRIDE. | ||
| 76 | */ | 77 | */ |
| 77 | #define KMS_DRIVER_MAJOR 3 | 78 | #define KMS_DRIVER_MAJOR 3 |
| 78 | #define KMS_DRIVER_MINOR 29 | 79 | #define KMS_DRIVER_MINOR 30 |
| 79 | #define KMS_DRIVER_PATCHLEVEL 0 | 80 | #define KMS_DRIVER_PATCHLEVEL 0 |
| 80 | 81 | ||
| 81 | int amdgpu_vram_limit = 0; | 82 | int amdgpu_vram_limit = 0; |
| @@ -1179,6 +1180,22 @@ static const struct file_operations amdgpu_driver_kms_fops = { · 1097 unmodified lines | |||
| 1179 | #endif | 1180 | #endif |
| 1180 | }; | 1181 | }; |
| 1181 | 1182 | ||
| 1183 | int amdgpu_file_to_fpriv(struct file *filp, struct amdgpu_fpriv **fpriv) | ||
| 1184 | { | ||
| 1185 | struct drm_file *file; | ||
| 1186 | |||
| 1187 | if (!filp) | ||
| 1188 | return -EINVAL; | ||
| 1189 | |||
| 1190 | if (filp->f_op != &amdgpu_driver_kms_fops) { | ||
| 1191 | return -EINVAL; | ||
| 1192 | } | ||
| 1193 | |||
| 1194 | file = filp->private_data; | ||
| 1195 | *fpriv = file->driver_priv; | ||
| 1196 | return 0; | ||
| 1197 | } | ||
| 1198 | |||
| 1182 | static bool | 1199 | static bool |
| 1183 | amdgpu_get_crtc_scanout_position(struct drm_device *dev, unsigned int pipe, | 1200 | amdgpu_get_crtc_scanout_position(struct drm_device *dev, unsigned int pipe, |
| 1184 | bool in_vblank_irq, int *vpos, int *hpos, | 1201 | bool in_vblank_irq, int *vpos, int *hpos, |
| @@ -140,9 +140,7 @@ void amdgpu_ih_ring_fini(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih) · 139 unmodified lines | |||
| 140 | * Interrupt hander (VI), walk the IH ring. | 140 | * Interrupt hander (VI), walk the IH ring. |
| 141 | * Returns irq process return code. | 141 | * Returns irq process return code. |
| 142 | */ | 142 | */ |
| 143 | int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, | 143 | int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih) |
| 144 | void (*callback)(struct amdgpu_device *adev, | ||
| 145 | struct amdgpu_ih_ring *ih)) | ||
| 146 | { | 144 | { |
| 147 | u32 wptr; | 145 | u32 wptr; |
| 148 | 146 | ||
| @@ -162,7 +160,7 @@ restart_ih: · 13 unmodified lines | |||
| 162 | rmb(); | 160 | rmb(); |
| 163 | 161 | ||
| 164 | while (ih->rptr != wptr) { | 162 | while (ih->rptr != wptr) { |
| 165 | callback(adev, ih); | 163 | amdgpu_irq_dispatch(adev, ih); |
| 166 | ih->rptr &= ih->ptr_mask; | 164 | ih->rptr &= ih->ptr_mask; |
| 167 | } | 165 | } |
| 168 | 166 | ||
| @@ -69,8 +69,6 @@ struct amdgpu_ih_funcs { · 68 unmodified lines | |||
| 69 | int amdgpu_ih_ring_init(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, | 69 | int amdgpu_ih_ring_init(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, |
| 70 | unsigned ring_size, bool use_bus_addr); | 70 | unsigned ring_size, bool use_bus_addr); |
| 71 | void amdgpu_ih_ring_fini(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih); | 71 | void amdgpu_ih_ring_fini(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih); |
| 72 | int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, | 72 | int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih); |
| 73 | void (*callback)(struct amdgpu_device *adev, | ||
| 74 | struct amdgpu_ih_ring *ih)); | ||
| 75 | 73 | ||
| 76 | #endif | 74 | #endif |
| @@ -130,29 +130,6 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev) · 129 unmodified lines | |||
| 130 | spin_unlock_irqrestore(&adev->irq.lock, irqflags); | 130 | spin_unlock_irqrestore(&adev->irq.lock, irqflags); |
| 131 | } | 131 | } |
| 132 | 132 | ||
| 133 | /** | ||
| 134 | * amdgpu_irq_callback - callback from the IH ring | ||
| 135 | * | ||
| 136 | * @adev: amdgpu device pointer | ||
| 137 | * @ih: amdgpu ih ring | ||
| 138 | * | ||
| 139 | * Callback from IH ring processing to handle the entry at the current position | ||
| 140 | * and advance the read pointer. | ||
| 141 | */ | ||
| 142 | static void amdgpu_irq_callback(struct amdgpu_device *adev, | ||
| 143 | struct amdgpu_ih_ring *ih) | ||
| 144 | { | ||
| 145 | u32 ring_index = ih->rptr >> 2; | ||
| 146 | struct amdgpu_iv_entry entry; | ||
| 147 | |||
| 148 | entry.iv_entry = (const uint32_t *)&ih->ring[ring_index]; | ||
| 149 | amdgpu_ih_decode_iv(adev, &entry); | ||
| 150 | |||
| 151 | trace_amdgpu_iv(ih - &adev->irq.ih, &entry); | ||
| 152 | |||
| 153 | amdgpu_irq_dispatch(adev, &entry); | ||
| 154 | } | ||
| 155 | |||
| 156 | /** | 133 | /** |
| 157 | * amdgpu_irq_handler - IRQ handler | 134 | * amdgpu_irq_handler - IRQ handler |
| 158 | * | 135 | * |
| @@ -170,7 +147,7 @@ irqreturn_t amdgpu_irq_handler(int irq, void *arg) · 11 unmodified lines | |||
| 170 | struct amdgpu_device *adev = dev->dev_private; | 147 | struct amdgpu_device *adev = dev->dev_private; |
| 171 | irqreturn_t ret; | 148 | irqreturn_t ret; |
| 172 | 149 | ||
| 173 | ret = amdgpu_ih_process(adev, &adev->irq.ih, amdgpu_irq_callback); | 150 | ret = amdgpu_ih_process(adev, &adev->irq.ih); |
| 174 | if (ret == IRQ_HANDLED) | 151 | if (ret == IRQ_HANDLED) |
| 175 | pm_runtime_mark_last_busy(dev->dev); | 152 | pm_runtime_mark_last_busy(dev->dev); |
| 176 | return ret; | 153 | return ret; |
| @@ -188,7 +165,7 @@ static void amdgpu_irq_handle_ih1(struct work_struct *work) · 11 unmodified lines | |||
| 188 | struct amdgpu_device *adev = container_of(work, struct amdgpu_device, | 165 | struct amdgpu_device *adev = container_of(work, struct amdgpu_device, |
| 189 | irq.ih1_work); | 166 | irq.ih1_work); |
| 190 | 167 | ||
| 191 | amdgpu_ih_process(adev, &adev->irq.ih1, amdgpu_irq_callback); | 168 | amdgpu_ih_process(adev, &adev->irq.ih1); |
| 192 | } | 169 | } |
| 193 | 170 | ||
| 194 | /** | 171 | /** |
| @@ -203,7 +180,7 @@ static void amdgpu_irq_handle_ih2(struct work_struct *work) · 8 unmodified lines | |||
| 203 | struct amdgpu_device *adev = container_of(work, struct amdgpu_device, | 180 | struct amdgpu_device *adev = container_of(work, struct amdgpu_device, |
| 204 | irq.ih2_work); | 181 | irq.ih2_work); |
| 205 | 182 | ||
| 206 | amdgpu_ih_process(adev, &adev->irq.ih2, amdgpu_irq_callback); | 183 | amdgpu_ih_process(adev, &adev->irq.ih2); |
| 207 | } | 184 | } |
| 208 | 185 | ||
| 209 | /** | 186 | /** |
| @@ -394,14 +371,23 @@ int amdgpu_irq_add_id(struct amdgpu_device *adev, · 184 unmodified lines | |||
| 394 | * Dispatches IRQ to IP blocks. | 371 | * Dispatches IRQ to IP blocks. |
| 395 | */ | 372 | */ |
| 396 | void amdgpu_irq_dispatch(struct amdgpu_device *adev, | 373 | void amdgpu_irq_dispatch(struct amdgpu_device *adev, |
| 397 | struct amdgpu_iv_entry *entry) | 374 | struct amdgpu_ih_ring *ih) |
| 398 | { | 375 | { |
| 399 | unsigned client_id = entry->client_id; | 376 | u32 ring_index = ih->rptr >> 2; |
| 400 | unsigned src_id = entry->src_id; | 377 | struct amdgpu_iv_entry entry; |
| 378 | unsigned client_id, src_id; | ||
| 401 | struct amdgpu_irq_src *src; | 379 | struct amdgpu_irq_src *src; |
| 402 | bool handled = false; | 380 | bool handled = false; |
| 403 | int r; | 381 | int r; |
| 404 | 382 | ||
| 383 | entry.iv_entry = (const uint32_t *)&ih->ring[ring_index]; | ||
| 384 | amdgpu_ih_decode_iv(adev, &entry); | ||
| 385 | |||
| 386 | trace_amdgpu_iv(ih - &adev->irq.ih, &entry); | ||
| 387 | |||
| 388 | client_id = entry.client_id; | ||
| 389 | src_id = entry.src_id; | ||
| 390 | |||
| 405 | if (client_id >= AMDGPU_IRQ_CLIENTID_MAX) { | 391 | if (client_id >= AMDGPU_IRQ_CLIENTID_MAX) { |
| 406 | DRM_DEBUG("Invalid client_id in IV: %d\n", client_id); | 392 | DRM_DEBUG("Invalid client_id in IV: %d\n", client_id); |
| 407 | 393 | ||
| @@ -416,7 +402,7 @@ void amdgpu_irq_dispatch(struct amdgpu_device *adev, · 8 unmodified lines | |||
| 416 | client_id, src_id); | 402 | client_id, src_id); |
| 417 | 403 | ||
| 418 | } else if ((src = adev->irq.client[client_id].sources[src_id])) { | 404 | } else if ((src = adev->irq.client[client_id].sources[src_id])) { |
| 419 | r = src->funcs->process(adev, src, entry); | 405 | r = src->funcs->process(adev, src, &entry); |
| 420 | if (r < 0) | 406 | if (r < 0) |
| 421 | DRM_ERROR("error processing interrupt (%d)\n", r); | 407 | DRM_ERROR("error processing interrupt (%d)\n", r); |
| 422 | else if (r) | 408 | else if (r) |
| @@ -428,7 +414,7 @@ void amdgpu_irq_dispatch(struct amdgpu_device *adev, · 5 unmodified lines | |||
| 428 | 414 | ||
| 429 | /* Send it to amdkfd as well if it isn't already handled */ | 415 | /* Send it to amdkfd as well if it isn't already handled */ |
| 430 | if (!handled) | 416 | if (!handled) |
| 431 | amdgpu_amdkfd_interrupt(adev, entry->iv_entry); | 417 | amdgpu_amdkfd_interrupt(adev, entry.iv_entry); |
| 432 | } | 418 | } |
| 433 | 419 | ||
| 434 | /** | 420 | /** |
| @@ -108,7 +108,7 @@ int amdgpu_irq_add_id(struct amdgpu_device *adev, · 107 unmodified lines | |||
| 108 | unsigned client_id, unsigned src_id, | 108 | unsigned client_id, unsigned src_id, |
| 109 | struct amdgpu_irq_src *source); | 109 | struct amdgpu_irq_src *source); |
| 110 | void amdgpu_irq_dispatch(struct amdgpu_device *adev, | 110 | void amdgpu_irq_dispatch(struct amdgpu_device *adev, |
| 111 | struct amdgpu_iv_entry *entry); | 111 | struct amdgpu_ih_ring *ih); |
| 112 | int amdgpu_irq_update(struct amdgpu_device *adev, struct amdgpu_irq_src *src, | 112 | int amdgpu_irq_update(struct amdgpu_device *adev, struct amdgpu_irq_src *src, |
| 113 | unsigned type); | 113 | unsigned type); |
| 114 | int amdgpu_irq_get(struct amdgpu_device *adev, struct amdgpu_irq_src *src, | 114 | int amdgpu_irq_get(struct amdgpu_device *adev, struct amdgpu_irq_src *src, |
| @@ -207,11 +207,12 @@ int amdgpu_driver_load_kms(struct drm_device *dev, unsigned long flags) · 206 unmodified lines | |||
| 207 | if (!r) { | 207 | if (!r) { |
| 208 | acpi_status = amdgpu_acpi_init(adev); | 208 | acpi_status = amdgpu_acpi_init(adev); |
| 209 | if (acpi_status) | 209 | if (acpi_status) |
| 210 | dev_dbg(&dev->pdev->dev, | 210 | dev_dbg(&dev->pdev->dev, |
| 211 | "Error during ACPI methods call\n"); | 211 | "Error during ACPI methods call\n"); |
| 212 | } | 212 | } |
| 213 | 213 | ||
| 214 | if (amdgpu_device_is_px(dev)) { | 214 | if (amdgpu_device_is_px(dev)) { |
| 215 | dev_pm_set_driver_flags(dev->dev, DPM_FLAG_NEVER_SKIP); | ||
| 215 | pm_runtime_use_autosuspend(dev->dev); | 216 | pm_runtime_use_autosuspend(dev->dev); |
| 216 | pm_runtime_set_autosuspend_delay(dev->dev, 5000); | 217 | pm_runtime_set_autosuspend_delay(dev->dev, 5000); |
| 217 | pm_runtime_set_active(dev->dev); | 218 | pm_runtime_set_active(dev->dev); |
| @@ -406,6 +406,7 @@ struct amdgpu_crtc { · 405 unmodified lines | |||
| 406 | struct amdgpu_flip_work *pflip_works; | 406 | struct amdgpu_flip_work *pflip_works; |
| 407 | enum amdgpu_flip_status pflip_status; | 407 | enum amdgpu_flip_status pflip_status; |
| 408 | int deferred_flip_completion; | 408 | int deferred_flip_completion; |
| 409 | u64 last_flip_vblank; | ||
| 409 | /* pll sharing */ | 410 | /* pll sharing */ |
| 410 | struct amdgpu_atom_ss ss; | 411 | struct amdgpu_atom_ss ss; |
| 411 | bool ss_enabled; | 412 | bool ss_enabled; |
| @@ -1284,6 +1284,30 @@ void amdgpu_bo_fence(struct amdgpu_bo *bo, struct dma_fence *fence, · 1283 unmodified lines | |||
| 1284 | reservation_object_add_excl_fence(resv, fence); | 1284 | reservation_object_add_excl_fence(resv, fence); |
| 1285 | } | 1285 | } |
| 1286 | 1286 | ||
| 1287 | /** | ||
| 1288 | * amdgpu_sync_wait_resv - Wait for BO reservation fences | ||
| 1289 | * | ||
| 1290 | * @bo: buffer object | ||
| 1291 | * @owner: fence owner | ||
| 1292 | * @intr: Whether the wait is interruptible | ||
| 1293 | * | ||
| 1294 | * Returns: | ||
| 1295 | * 0 on success, errno otherwise. | ||
| 1296 | */ | ||
| 1297 | int amdgpu_bo_sync_wait(struct amdgpu_bo *bo, void *owner, bool intr) | ||
| 1298 | { | ||
| 1299 | struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); | ||
| 1300 | struct amdgpu_sync sync; | ||
| 1301 | int r; | ||
| 1302 | |||
| 1303 | amdgpu_sync_create(&sync); | ||
| 1304 | amdgpu_sync_resv(adev, &sync, bo->tbo.resv, owner, false); | ||
| 1305 | r = amdgpu_sync_wait(&sync, intr); | ||
| 1306 | amdgpu_sync_free(&sync); | ||
| 1307 | |||
| 1308 | return r; | ||
| 1309 | } | ||
| 1310 | |||
| 1287 | /** | 1311 | /** |
| 1288 | * amdgpu_bo_gpu_offset - return GPU offset of bo | 1312 | * amdgpu_bo_gpu_offset - return GPU offset of bo |
| 1289 | * @bo: amdgpu object for which we query the offset | 1313 | * @bo: amdgpu object for which we query the offset |
| @@ -266,6 +266,7 @@ void amdgpu_bo_move_notify(struct ttm_buffer_object *bo, · 265 unmodified lines | |||
| 266 | int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo); | 266 | int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo); |
| 267 | void amdgpu_bo_fence(struct amdgpu_bo *bo, struct dma_fence *fence, | 267 | void amdgpu_bo_fence(struct amdgpu_bo *bo, struct dma_fence *fence, |
| 268 | bool shared); | 268 | bool shared); |
| 269 | int amdgpu_bo_sync_wait(struct amdgpu_bo *bo, void *owner, bool intr); | ||
| 269 | u64 amdgpu_bo_gpu_offset(struct amdgpu_bo *bo); | 270 | u64 amdgpu_bo_gpu_offset(struct amdgpu_bo *bo); |
| 270 | int amdgpu_bo_validate(struct amdgpu_bo *bo); | 271 | int amdgpu_bo_validate(struct amdgpu_bo *bo); |
| 271 | int amdgpu_bo_restore_shadow(struct amdgpu_bo *shadow, | 272 | int amdgpu_bo_restore_shadow(struct amdgpu_bo *shadow, |
| @@ -54,16 +54,20 @@ static int amdgpu_sched_process_priority_override(struct amdgpu_device *adev, · 53 unmodified lines | |||
| 54 | enum drm_sched_priority priority) | 54 | enum drm_sched_priority priority) |
| 55 | { | 55 | { |
| 56 | struct file *filp = fget(fd); | 56 | struct file *filp = fget(fd); |
| 57 | struct drm_file *file; | ||
| 58 | struct amdgpu_fpriv *fpriv; | 57 | struct amdgpu_fpriv *fpriv; |
| 59 | struct amdgpu_ctx *ctx; | 58 | struct amdgpu_ctx *ctx; |
| 60 | uint32_t id; | 59 | uint32_t id; |
| 60 | int r; | ||
| 61 | 61 | ||
| 62 | if (!filp) | 62 | if (!filp) |
| 63 | return -EINVAL; | 63 | return -EINVAL; |
| 64 | 64 | ||
| 65 | file = filp->private_data; | 65 | r = amdgpu_file_to_fpriv(filp, &fpriv); |
| 66 | fpriv = file->driver_priv; | 66 | if (r) { |
| 67 | fput(filp); | ||
| 68 | return r; | ||
| 69 | } | ||
| 70 | |||
| 67 | idr_for_each_entry(&fpriv->ctx_mgr.ctx_handles, ctx, id) | 71 | idr_for_each_entry(&fpriv->ctx_mgr.ctx_handles, ctx, id) |
| 68 | amdgpu_ctx_priority_override(ctx, priority); | 72 | amdgpu_ctx_priority_override(ctx, priority); |
| 69 | 73 | ||
| @@ -72,6 +76,39 @@ static int amdgpu_sched_process_priority_override(struct amdgpu_device *adev, · 2 unmodified lines | |||
| 72 | return 0; | 76 | return 0; |
| 73 | } | 77 | } |
| 74 | 78 | ||
| 79 | static int amdgpu_sched_context_priority_override(struct amdgpu_device *adev, | ||
| 80 | int fd, | ||
| 81 | unsigned ctx_id, | ||
| 82 | enum drm_sched_priority priority) | ||
| 83 | { | ||
| 84 | struct file *filp = fget(fd); | ||
| 85 | struct amdgpu_fpriv *fpriv; | ||
| 86 | struct amdgpu_ctx *ctx; | ||
| 87 | int r; | ||
| 88 | |||
| 89 | if (!filp) | ||
| 90 | return -EINVAL; | ||
| 91 | |||
| 92 | r = amdgpu_file_to_fpriv(filp, &fpriv); | ||
| 93 | if (r) { | ||
| 94 | fput(filp); | ||
| 95 | return r; | ||
| 96 | } | ||
| 97 | |||
| 98 | ctx = amdgpu_ctx_get(fpriv, ctx_id); | ||
| 99 | |||
| 100 | if (!ctx) { | ||
| 101 | fput(filp); | ||
| 102 | return -EINVAL; | ||
| 103 | } | ||
| 104 | |||
| 105 | amdgpu_ctx_priority_override(ctx, priority); | ||
| 106 | amdgpu_ctx_put(ctx); | ||
| 107 | fput(filp); | ||
| 108 | |||
| 109 | return 0; | ||
| 110 | } | ||
| 111 | |||
| 75 | int amdgpu_sched_ioctl(struct drm_device *dev, void *data, | 112 | int amdgpu_sched_ioctl(struct drm_device *dev, void *data, |
| 76 | struct drm_file *filp) | 113 | struct drm_file *filp) |
| 77 | { | 114 | { |
| @@ -81,7 +118,7 @@ int amdgpu_sched_ioctl(struct drm_device *dev, void *data, · 3 unmodified lines | |||
| 81 | int r; | 118 | int r; |
| 82 | 119 | ||
| 83 | priority = amdgpu_to_sched_priority(args->in.priority); | 120 | priority = amdgpu_to_sched_priority(args->in.priority); |
| 84 | if (args->in.flags || priority == DRM_SCHED_PRIORITY_INVALID) | 121 | if (priority == DRM_SCHED_PRIORITY_INVALID) |
| 85 | return -EINVAL; | 122 | return -EINVAL; |
| 86 | 123 | ||
| 87 | switch (args->in.op) { | 124 | switch (args->in.op) { |
| @@ -90,6 +127,12 @@ int amdgpu_sched_ioctl(struct drm_device *dev, void *data, · 2 unmodified lines | |||
| 90 | args->in.fd, | 127 | args->in.fd, |
| 91 | priority); | 128 | priority); |
| 92 | break; | 129 | break; |
| 130 | case AMDGPU_SCHED_OP_CONTEXT_PRIORITY_OVERRIDE: | ||
| 131 | r = amdgpu_sched_context_priority_override(adev, | ||
| 132 | args->in.fd, | ||
| 133 | args->in.ctx_id, | ||
| 134 | priority); | ||
| 135 | break; | ||
| 93 | default: | 136 | default: |
| 94 | DRM_ERROR("Invalid sched op specified: %d\n", args->in.op); | 137 | DRM_ERROR("Invalid sched op specified: %d\n", args->in.op); |
| 95 | r = -EINVAL; | 138 | r = -EINVAL; |
| @@ -652,12 +652,14 @@ void amdgpu_vm_move_to_lru_tail(struct amdgpu_device *adev, · 651 unmodified lines | |||
| 652 | struct ttm_bo_global *glob = adev->mman.bdev.glob; | 652 | struct ttm_bo_global *glob = adev->mman.bdev.glob; |
| 653 | struct amdgpu_vm_bo_base *bo_base; | 653 | struct amdgpu_vm_bo_base *bo_base; |
| 654 | 654 | ||
| 655 | #if 0 | ||
| 655 | if (vm->bulk_moveable) { | 656 | if (vm->bulk_moveable) { |
| 656 | spin_lock(&glob->lru_lock); | 657 | spin_lock(&glob->lru_lock); |
| 657 | ttm_bo_bulk_move_lru_tail(&vm->lru_bulk_move); | 658 | ttm_bo_bulk_move_lru_tail(&vm->lru_bulk_move); |
| 658 | spin_unlock(&glob->lru_lock); | 659 | spin_unlock(&glob->lru_lock); |
| 659 | return; | 660 | return; |
| 660 | } | 661 | } |
| 662 | #endif | ||
| 661 | 663 | ||
| 662 | memset(&vm->lru_bulk_move, 0, sizeof(vm->lru_bulk_move)); | 664 | memset(&vm->lru_bulk_move, 0, sizeof(vm->lru_bulk_move)); |
| 663 | 665 | ||
| @@ -698,8 +700,6 @@ int amdgpu_vm_validate_pt_bos(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 34 unmodified lines | |||
| 698 | struct amdgpu_vm_bo_base *bo_base, *tmp; | 700 | struct amdgpu_vm_bo_base *bo_base, *tmp; |
| 699 | int r = 0; | 701 | int r = 0; |
| 700 | 702 | ||
| 701 | vm->bulk_moveable &= list_empty(&vm->evicted); | ||
| 702 | |||
| 703 | list_for_each_entry_safe(bo_base, tmp, &vm->evicted, vm_status) { | 703 | list_for_each_entry_safe(bo_base, tmp, &vm->evicted, vm_status) { |
| 704 | struct amdgpu_bo *bo = bo_base->bo; | 704 | struct amdgpu_bo *bo = bo_base->bo; |
| 705 | 705 | ||
| @@ -828,7 +828,7 @@ static int amdgpu_vm_clear_bo(struct amdgpu_device *adev, · 122 unmodified lines | |||
| 828 | 828 | ||
| 829 | WARN_ON(job->ibs[0].length_dw > 64); | 829 | WARN_ON(job->ibs[0].length_dw > 64); |
| 830 | r = amdgpu_sync_resv(adev, &job->sync, bo->tbo.resv, | 830 | r = amdgpu_sync_resv(adev, &job->sync, bo->tbo.resv, |
| 831 | AMDGPU_FENCE_OWNER_UNDEFINED, false); | 831 | AMDGPU_FENCE_OWNER_KFD, false); |
| 832 | if (r) | 832 | if (r) |
| 833 | goto error_free; | 833 | goto error_free; |
| 834 | 834 | ||
| @@ -1332,31 +1332,6 @@ static void amdgpu_vm_cpu_set_ptes(struct amdgpu_pte_update_params *params, · 497 unmodified lines | |||
| 1332 | } | 1332 | } |
| 1333 | } | 1333 | } |
| 1334 | 1334 | ||
| 1335 | |||
| 1336 | /** | ||
| 1337 | * amdgpu_vm_wait_pd - Wait for PT BOs to be free. | ||
| 1338 | * | ||
| 1339 | * @adev: amdgpu_device pointer | ||
| 1340 | * @vm: related vm | ||
| 1341 | * @owner: fence owner | ||
| 1342 | * | ||
| 1343 | * Returns: | ||
| 1344 | * 0 on success, errno otherwise. | ||
| 1345 | */ | ||
| 1346 | static int amdgpu_vm_wait_pd(struct amdgpu_device *adev, struct amdgpu_vm *vm, | ||
| 1347 | void *owner) | ||
| 1348 | { | ||
| 1349 | struct amdgpu_sync sync; | ||
| 1350 | int r; | ||
| 1351 | |||
| 1352 | amdgpu_sync_create(&sync); | ||
| 1353 | amdgpu_sync_resv(adev, &sync, vm->root.base.bo->tbo.resv, owner, false); | ||
| 1354 | r = amdgpu_sync_wait(&sync, true); | ||
| 1355 | amdgpu_sync_free(&sync); | ||
| 1356 | |||
| 1357 | return r; | ||
| 1358 | } | ||
| 1359 | |||
| 1360 | /** | 1335 | /** |
| 1361 | * amdgpu_vm_update_func - helper to call update function | 1336 | * amdgpu_vm_update_func - helper to call update function |
| 1362 | * | 1337 | * |
| @@ -1451,7 +1426,8 @@ restart: · 88 unmodified lines | |||
| 1451 | params.adev = adev; | 1426 | params.adev = adev; |
| 1452 | 1427 | ||
| 1453 | if (vm->use_cpu_for_update) { | 1428 | if (vm->use_cpu_for_update) { |
| 1454 | r = amdgpu_vm_wait_pd(adev, vm, AMDGPU_FENCE_OWNER_VM); | 1429 | r = amdgpu_bo_sync_wait(vm->root.base.bo, |
| 1430 | AMDGPU_FENCE_OWNER_VM, true); | ||
| 1455 | if (unlikely(r)) | 1431 | if (unlikely(r)) |
| 1456 | return r; | 1432 | return r; |
| 1457 | 1433 | ||
| @@ -1772,9 +1748,9 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, · 314 unmodified lines | |||
| 1772 | params.adev = adev; | 1748 | params.adev = adev; |
| 1773 | params.vm = vm; | 1749 | params.vm = vm; |
| 1774 | 1750 | ||
| 1775 | /* sync to everything on unmapping */ | 1751 | /* sync to everything except eviction fences on unmapping */ |
| 1776 | if (!(flags & AMDGPU_PTE_VALID)) | 1752 | if (!(flags & AMDGPU_PTE_VALID)) |
| 1777 | owner = AMDGPU_FENCE_OWNER_UNDEFINED; | 1753 | owner = AMDGPU_FENCE_OWNER_KFD; |
| 1778 | 1754 | ||
| 1779 | if (vm->use_cpu_for_update) { | 1755 | if (vm->use_cpu_for_update) { |
| 1780 | /* params.src is used as flag to indicate system Memory */ | 1756 | /* params.src is used as flag to indicate system Memory */ |
| @@ -1784,7 +1760,7 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, · 3 unmodified lines | |||
| 1784 | /* Wait for PT BOs to be idle. PTs share the same resv. object | 1760 | /* Wait for PT BOs to be idle. PTs share the same resv. object |
| 1785 | * as the root PD BO | 1761 | * as the root PD BO |
| 1786 | */ | 1762 | */ |
| 1787 | r = amdgpu_vm_wait_pd(adev, vm, owner); | 1763 | r = amdgpu_bo_sync_wait(vm->root.base.bo, owner, true); |
| 1788 | if (unlikely(r)) | 1764 | if (unlikely(r)) |
| 1789 | return r; | 1765 | return r; |
| 1790 | 1766 | ||
| @@ -2980,7 +2980,7 @@ static int dce_v6_0_pageflip_irq(struct amdgpu_device *adev, · 2979 unmodified lines | |||
| 2980 | struct amdgpu_irq_src *source, | 2980 | struct amdgpu_irq_src *source, |
| 2981 | struct amdgpu_iv_entry *entry) | 2981 | struct amdgpu_iv_entry *entry) |
| 2982 | { | 2982 | { |
| 2983 | unsigned long flags; | 2983 | unsigned long flags; |
| 2984 | unsigned crtc_id; | 2984 | unsigned crtc_id; |
| 2985 | struct amdgpu_crtc *amdgpu_crtc; | 2985 | struct amdgpu_crtc *amdgpu_crtc; |
| 2986 | struct amdgpu_flip_work *works; | 2986 | struct amdgpu_flip_work *works; |
| @@ -266,7 +266,8 @@ flr_done: · 265 unmodified lines | |||
| 266 | } | 266 | } |
| 267 | 267 | ||
| 268 | /* Trigger recovery for world switch failure if no TDR */ | 268 | /* Trigger recovery for world switch failure if no TDR */ |
| 269 | if (amdgpu_device_should_recover_gpu(adev)) | 269 | if (amdgpu_device_should_recover_gpu(adev) |
| 270 | && amdgpu_lockup_timeout == MAX_SCHEDULE_TIMEOUT) | ||
| 270 | amdgpu_device_gpu_recover(adev, NULL); | 271 | amdgpu_device_gpu_recover(adev, NULL); |
| 271 | } | 272 | } |
| 272 | 273 | ||
| @@ -32,7 +32,7 @@ · 31 unmodified lines | |||
| 32 | 32 | ||
| 33 | static u32 nbio_v7_4_get_rev_id(struct amdgpu_device *adev) | 33 | static u32 nbio_v7_4_get_rev_id(struct amdgpu_device *adev) |
| 34 | { | 34 | { |
| 35 | u32 tmp = RREG32_SOC15(NBIO, 0, mmRCC_DEV0_EPF0_STRAP0); | 35 | u32 tmp = RREG32_SOC15(NBIO, 0, mmRCC_DEV0_EPF0_STRAP0); |
| 36 | 36 | ||
| 37 | tmp &= RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0_MASK; | 37 | tmp &= RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0_MASK; |
| 38 | tmp >>= RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0__SHIFT; | 38 | tmp >>= RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0__SHIFT; |
| @@ -128,7 +128,7 @@ static const struct soc15_reg_golden golden_settings_sdma0_4_2_init[] = { · 127 unmodified lines | |||
| 128 | 128 | ||
| 129 | static const struct soc15_reg_golden golden_settings_sdma0_4_2[] = | 129 | static const struct soc15_reg_golden golden_settings_sdma0_4_2[] = |
| 130 | { | 130 | { |
| 131 | SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_CHICKEN_BITS, 0xfe931f07, 0x02831d07), | 131 | SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_CHICKEN_BITS, 0xfe931f07, 0x02831f07), |
| 132 | SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_CLK_CTRL, 0xffffffff, 0x3f000100), | 132 | SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_CLK_CTRL, 0xffffffff, 0x3f000100), |
| 133 | SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_GB_ADDR_CONFIG, 0x0000773f, 0x00004002), | 133 | SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_GB_ADDR_CONFIG, 0x0000773f, 0x00004002), |
| 134 | SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_GB_ADDR_CONFIG_READ, 0x0000773f, 0x00004002), | 134 | SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_GB_ADDR_CONFIG_READ, 0x0000773f, 0x00004002), |
| @@ -158,7 +158,7 @@ static const struct soc15_reg_golden golden_settings_sdma0_4_2[] = · 23 unmodified lines | |||
| 158 | }; | 158 | }; |
| 159 | 159 | ||
| 160 | static const struct soc15_reg_golden golden_settings_sdma1_4_2[] = { | 160 | static const struct soc15_reg_golden golden_settings_sdma1_4_2[] = { |
| 161 | SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_CHICKEN_BITS, 0xfe931f07, 0x02831d07), | 161 | SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_CHICKEN_BITS, 0xfe931f07, 0x02831f07), |
| 162 | SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_CLK_CTRL, 0xffffffff, 0x3f000100), | 162 | SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_CLK_CTRL, 0xffffffff, 0x3f000100), |
| 163 | SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_GB_ADDR_CONFIG, 0x0000773f, 0x00004002), | 163 | SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_GB_ADDR_CONFIG, 0x0000773f, 0x00004002), |
| 164 | SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_GB_ADDR_CONFIG_READ, 0x0000773f, 0x00004002), | 164 | SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_GB_ADDR_CONFIG_READ, 0x0000773f, 0x00004002), |
| @@ -1436,7 +1436,7 @@ static int si_common_early_init(void *handle) · 1435 unmodified lines | |||
| 1436 | AMD_CG_SUPPORT_UVD_MGCG | | 1436 | AMD_CG_SUPPORT_UVD_MGCG | |
| 1437 | AMD_CG_SUPPORT_HDP_LS | | 1437 | AMD_CG_SUPPORT_HDP_LS | |
| 1438 | AMD_CG_SUPPORT_HDP_MGCG; | 1438 | AMD_CG_SUPPORT_HDP_MGCG; |
| 1439 | adev->pg_flags = 0; | 1439 | adev->pg_flags = 0; |
| 1440 | adev->external_rev_id = (adev->rev_id == 0) ? 1 : | 1440 | adev->external_rev_id = (adev->rev_id == 0) ? 1 : |
| 1441 | (adev->rev_id == 1) ? 5 : 6; | 1441 | (adev->rev_id == 1) ? 5 : 6; |
| 1442 | break; | 1442 | break; |
| @@ -6216,10 +6216,12 @@ static void si_request_link_speed_change_before_state_change(struct amdgpu_devic · 6215 unmodified lines | |||
| 6216 | si_pi->force_pcie_gen = AMDGPU_PCIE_GEN2; | 6216 | si_pi->force_pcie_gen = AMDGPU_PCIE_GEN2; |
| 6217 | if (current_link_speed == AMDGPU_PCIE_GEN2) | 6217 | if (current_link_speed == AMDGPU_PCIE_GEN2) |
| 6218 | break; | 6218 | break; |
| 6219 | /* fall through */ | ||
| 6219 | case AMDGPU_PCIE_GEN2: | 6220 | case AMDGPU_PCIE_GEN2: |
| 6220 | if (amdgpu_acpi_pcie_performance_request(adev, PCIE_PERF_REQ_PECI_GEN2, false) == 0) | 6221 | if (amdgpu_acpi_pcie_performance_request(adev, PCIE_PERF_REQ_PECI_GEN2, false) == 0) |
| 6221 | break; | 6222 | break; |
| 6222 | #endif | 6223 | #endif |
| 6224 | /* fall through */ | ||
| 6223 | default: | 6225 | default: |
| 6224 | si_pi->force_pcie_gen = si_get_current_pcie_speed(adev); | 6226 | si_pi->force_pcie_gen = si_get_current_pcie_speed(adev); |
| 6225 | break; | 6227 | break; |
| @@ -81,6 +81,10 @@ void vega10_doorbell_index_init(struct amdgpu_device *adev) · 80 unmodified lines | |||
| 81 | adev->doorbell_index.uvd_vce.vce_ring2_3 = AMDGPU_DOORBELL64_VCE_RING2_3; | 81 | adev->doorbell_index.uvd_vce.vce_ring2_3 = AMDGPU_DOORBELL64_VCE_RING2_3; |
| 82 | adev->doorbell_index.uvd_vce.vce_ring4_5 = AMDGPU_DOORBELL64_VCE_RING4_5; | 82 | adev->doorbell_index.uvd_vce.vce_ring4_5 = AMDGPU_DOORBELL64_VCE_RING4_5; |
| 83 | adev->doorbell_index.uvd_vce.vce_ring6_7 = AMDGPU_DOORBELL64_VCE_RING6_7; | 83 | adev->doorbell_index.uvd_vce.vce_ring6_7 = AMDGPU_DOORBELL64_VCE_RING6_7; |
| 84 | |||
| 85 | adev->doorbell_index.first_non_cp = AMDGPU_DOORBELL64_FIRST_NON_CP; | ||
| 86 | adev->doorbell_index.last_non_cp = AMDGPU_DOORBELL64_LAST_NON_CP; | ||
| 87 | |||
| 84 | /* In unit of dword doorbell */ | 88 | /* In unit of dword doorbell */ |
| 85 | adev->doorbell_index.max_assignment = AMDGPU_DOORBELL64_MAX_ASSIGNMENT << 1; | 89 | adev->doorbell_index.max_assignment = AMDGPU_DOORBELL64_MAX_ASSIGNMENT << 1; |
| 86 | adev->doorbell_index.sdma_doorbell_range = 4; | 90 | adev->doorbell_index.sdma_doorbell_range = 4; |
| @@ -85,6 +85,10 @@ void vega20_doorbell_index_init(struct amdgpu_device *adev) · 84 unmodified lines | |||
| 85 | adev->doorbell_index.uvd_vce.vce_ring2_3 = AMDGPU_VEGA20_DOORBELL64_VCE_RING2_3; | 85 | adev->doorbell_index.uvd_vce.vce_ring2_3 = AMDGPU_VEGA20_DOORBELL64_VCE_RING2_3; |
| 86 | adev->doorbell_index.uvd_vce.vce_ring4_5 = AMDGPU_VEGA20_DOORBELL64_VCE_RING4_5; | 86 | adev->doorbell_index.uvd_vce.vce_ring4_5 = AMDGPU_VEGA20_DOORBELL64_VCE_RING4_5; |
| 87 | adev->doorbell_index.uvd_vce.vce_ring6_7 = AMDGPU_VEGA20_DOORBELL64_VCE_RING6_7; | 87 | adev->doorbell_index.uvd_vce.vce_ring6_7 = AMDGPU_VEGA20_DOORBELL64_VCE_RING6_7; |
| 88 | |||
| 89 | adev->doorbell_index.first_non_cp = AMDGPU_VEGA20_DOORBELL64_FIRST_NON_CP; | ||
| 90 | adev->doorbell_index.last_non_cp = AMDGPU_VEGA20_DOORBELL64_LAST_NON_CP; | ||
| 91 | |||
| 88 | adev->doorbell_index.max_assignment = AMDGPU_VEGA20_DOORBELL_MAX_ASSIGNMENT << 1; | 92 | adev->doorbell_index.max_assignment = AMDGPU_VEGA20_DOORBELL_MAX_ASSIGNMENT << 1; |
| 89 | adev->doorbell_index.sdma_doorbell_range = 20; | 93 | adev->doorbell_index.sdma_doorbell_range = 20; |
| 90 | } | 94 | } |
| @@ -134,12 +134,18 @@ static int allocate_doorbell(struct qcm_process_device *qpd, struct queue *q) · 133 unmodified lines | |||
| 134 | */ | 134 | */ |
| 135 | q->doorbell_id = q->properties.queue_id; | 135 | q->doorbell_id = q->properties.queue_id; |
| 136 | } else if (q->properties.type == KFD_QUEUE_TYPE_SDMA) { | 136 | } else if (q->properties.type == KFD_QUEUE_TYPE_SDMA) { |
| 137 | /* For SDMA queues on SOC15, use static doorbell | 137 | /* For SDMA queues on SOC15 with 8-byte doorbell, use static |
| 138 | * assignments based on the engine and queue. | 138 | * doorbell assignments based on the engine and queue id. |
| 139 | * The doobell index distance between RLC (2*i) and (2*i+1) | ||
| 140 | * for a SDMA engine is 512. | ||
| 139 | */ | 141 | */ |
| 140 | q->doorbell_id = dev->shared_resources.sdma_doorbell | 142 | uint32_t *idx_offset = |
| 141 | [q->properties.sdma_engine_id] | 143 | dev->shared_resources.sdma_doorbell_idx; |
| 142 | [q->properties.sdma_queue_id]; | 144 | |
| 145 | q->doorbell_id = idx_offset[q->properties.sdma_engine_id] | ||
| 146 | + (q->properties.sdma_queue_id & 1) | ||
| 147 | * KFD_QUEUE_DOORBELL_MIRROR_OFFSET | ||
| 148 | + (q->properties.sdma_queue_id >> 1); | ||
| 143 | } else { | 149 | } else { |
| 144 | /* For CP queues on SOC15 reserve a free doorbell ID */ | 150 | /* For CP queues on SOC15 reserve a free doorbell ID */ |
| 145 | unsigned int found; | 151 | unsigned int found; |
| @@ -97,17 +97,29 @@ · 96 unmodified lines | |||
| 97 | #define KFD_CWSR_TBA_TMA_SIZE (PAGE_SIZE * 2) | 97 | #define KFD_CWSR_TBA_TMA_SIZE (PAGE_SIZE * 2) |
| 98 | #define KFD_CWSR_TMA_OFFSET PAGE_SIZE | 98 | #define KFD_CWSR_TMA_OFFSET PAGE_SIZE |
| 99 | 99 | ||
| 100 | #define KFD_MAX_NUM_OF_QUEUES_PER_DEVICE \ | ||
| 101 | (KFD_MAX_NUM_OF_PROCESSES * \ | ||
| 102 | KFD_MAX_NUM_OF_QUEUES_PER_PROCESS) | ||
| 103 | |||
| 104 | #define KFD_KERNEL_QUEUE_SIZE 2048 | ||
| 105 | |||
| 106 | /* | ||
| 107 | * 512 = 0x200 | ||
| 108 | * The doorbell index distance between SDMA RLC (2*i) and (2*i+1) in the | ||
| 109 | * same SDMA engine on SOC15, which has 8-byte doorbells for SDMA. | ||
| 110 | * 512 8-byte doorbell distance (i.e. one page away) ensures that SDMA RLC | ||
| 111 | * (2*i+1) doorbells (in terms of the lower 12 bit address) lie exactly in | ||
| 112 | * the OFFSET and SIZE set in registers like BIF_SDMA0_DOORBELL_RANGE. | ||
| 113 | */ | ||
| 114 | #define KFD_QUEUE_DOORBELL_MIRROR_OFFSET 512 | ||
| 115 | |||
| 116 | |||
| 100 | /* | 117 | /* |
| 101 | * Kernel module parameter to specify maximum number of supported queues per | 118 | * Kernel module parameter to specify maximum number of supported queues per |
| 102 | * device | 119 | * device |
| 103 | */ | 120 | */ |
| 104 | extern int max_num_of_queues_per_device; | 121 | extern int max_num_of_queues_per_device; |
| 105 | 122 | ||
| 106 | #define KFD_MAX_NUM_OF_QUEUES_PER_DEVICE \ | ||
| 107 | (KFD_MAX_NUM_OF_PROCESSES * \ | ||
| 108 | KFD_MAX_NUM_OF_QUEUES_PER_PROCESS) | ||
| 109 | |||
| 110 | #define KFD_KERNEL_QUEUE_SIZE 2048 | ||
| 111 | 123 | ||
| 112 | /* Kernel module parameter to specify the scheduling policy */ | 124 | /* Kernel module parameter to specify the scheduling policy */ |
| 113 | extern int sched_policy; | 125 | extern int sched_policy; |
| @@ -607,13 +607,17 @@ static int init_doorbell_bitmap(struct qcm_process_device *qpd, · 606 unmodified lines | |||
| 607 | if (!qpd->doorbell_bitmap) | 607 | if (!qpd->doorbell_bitmap) |
| 608 | return -ENOMEM; | 608 | return -ENOMEM; |
| 609 | 609 | ||
| 610 | /* Mask out any reserved doorbells */ | 610 | /* Mask out doorbells reserved for SDMA, IH, and VCN on SOC15. */ |
| 611 | for (i = 0; i < KFD_MAX_NUM_OF_QUEUES_PER_PROCESS; i++) | 611 | for (i = 0; i < KFD_MAX_NUM_OF_QUEUES_PER_PROCESS / 2; i++) { |
| 612 | if ((dev->shared_resources.reserved_doorbell_mask & i) == | 612 | if (i >= dev->shared_resources.non_cp_doorbells_start |
| 613 | dev->shared_resources.reserved_doorbell_val) { | 613 | && i <= dev->shared_resources.non_cp_doorbells_end) { |
| 614 | set_bit(i, qpd->doorbell_bitmap); | 614 | set_bit(i, qpd->doorbell_bitmap); |
| 615 | pr_debug("reserved doorbell 0x%03x\n", i); | 615 | set_bit(i + KFD_QUEUE_DOORBELL_MIRROR_OFFSET, |
| 616 | qpd->doorbell_bitmap); | ||
| 617 | pr_debug("reserved doorbell 0x%03x and 0x%03x\n", i, | ||
| 618 | i + KFD_QUEUE_DOORBELL_MIRROR_OFFSET); | ||
| 616 | } | 619 | } |
| 620 | } | ||
| 617 | 621 | ||
| 618 | return 0; | 622 | return 0; |
| 619 | } | 623 | } |
| @@ -303,12 +303,11 @@ static void dm_pflip_high_irq(void *interrupt_params) · 302 unmodified lines | |||
| 303 | return; | 303 | return; |
| 304 | } | 304 | } |
| 305 | 305 | ||
| 306 | /* Update to correct count(s) if racing with vblank irq */ | ||
| 307 | amdgpu_crtc->last_flip_vblank = drm_crtc_accurate_vblank_count(&amdgpu_crtc->base); | ||
| 306 | 308 | ||
| 307 | /* wake up userspace */ | 309 | /* wake up userspace */ |
| 308 | if (amdgpu_crtc->event) { | 310 | if (amdgpu_crtc->event) { |
| 309 | /* Update to correct count(s) if racing with vblank irq */ | ||
| 310 | drm_crtc_accurate_vblank_count(&amdgpu_crtc->base); | ||
| 311 | |||
| 312 | drm_crtc_send_vblank_event(&amdgpu_crtc->base, amdgpu_crtc->event); | 311 | drm_crtc_send_vblank_event(&amdgpu_crtc->base, amdgpu_crtc->event); |
| 313 | 312 | ||
| 314 | /* page flip completed. clean up */ | 313 | /* page flip completed. clean up */ |
| @@ -786,12 +785,13 @@ static int dm_suspend(void *handle) · 471 unmodified lines | |||
| 786 | struct amdgpu_display_manager *dm = &adev->dm; | 785 | struct amdgpu_display_manager *dm = &adev->dm; |
| 787 | int ret = 0; | 786 | int ret = 0; |
| 788 | 787 | ||
| 788 | WARN_ON(adev->dm.cached_state); | ||
| 789 | adev->dm.cached_state = drm_atomic_helper_suspend(adev->ddev); | ||
| 790 | |||
| 789 | s3_handle_mst(adev->ddev, true); | 791 | s3_handle_mst(adev->ddev, true); |
| 790 | 792 | ||
| 791 | amdgpu_dm_irq_suspend(adev); | 793 | amdgpu_dm_irq_suspend(adev); |
| 792 | 794 | ||
| 793 | WARN_ON(adev->dm.cached_state); | ||
| 794 | adev->dm.cached_state = drm_atomic_helper_suspend(adev->ddev); | ||
| 795 | 795 | ||
| 796 | dc_set_power_state(dm->dc, DC_ACPI_CM_POWER_STATE_D3); | 796 | dc_set_power_state(dm->dc, DC_ACPI_CM_POWER_STATE_D3); |
| 797 | 797 | ||
| @@ -3790,7 +3790,6 @@ static const struct drm_plane_helper_funcs dm_plane_helper_funcs = { · 2992 unmodified lines | |||
| 3790 | * check will succeed, and let DC implement proper check | 3790 | * check will succeed, and let DC implement proper check |
| 3791 | */ | 3791 | */ |
| 3792 | static const uint32_t rgb_formats[] = { | 3792 | static const uint32_t rgb_formats[] = { |
| 3793 | DRM_FORMAT_RGB888, | ||
| 3794 | DRM_FORMAT_XRGB8888, | 3793 | DRM_FORMAT_XRGB8888, |
| 3795 | DRM_FORMAT_ARGB8888, | 3794 | DRM_FORMAT_ARGB8888, |
| 3796 | DRM_FORMAT_RGBA8888, | 3795 | DRM_FORMAT_RGBA8888, |
| @@ -4646,6 +4645,8 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state, · 849 unmodified lines | |||
| 4646 | struct amdgpu_bo *abo; | 4645 | struct amdgpu_bo *abo; |
| 4647 | uint64_t tiling_flags, dcc_address; | 4646 | uint64_t tiling_flags, dcc_address; |
| 4648 | uint32_t target, target_vblank; | 4647 | uint32_t target, target_vblank; |
| 4648 | uint64_t last_flip_vblank; | ||
| 4649 | bool vrr_active = acrtc_state->freesync_config.state == VRR_STATE_ACTIVE_VARIABLE; | ||
| 4649 | 4650 | ||
| 4650 | struct { | 4651 | struct { |
| 4651 | struct dc_surface_update surface_updates[MAX_SURFACES]; | 4652 | struct dc_surface_update surface_updates[MAX_SURFACES]; |
| @@ -4678,10 +4679,9 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state, · 26 unmodified lines | |||
| 4678 | struct dc_plane_state *dc_plane; | 4679 | struct dc_plane_state *dc_plane; |
| 4679 | struct dm_plane_state *dm_new_plane_state = to_dm_plane_state(new_plane_state); | 4680 | struct dm_plane_state *dm_new_plane_state = to_dm_plane_state(new_plane_state); |
| 4680 | 4681 | ||
| 4681 | if (plane->type == DRM_PLANE_TYPE_CURSOR) { | 4682 | /* Cursor plane is handled after stream updates */ |
| 4682 | handle_cursor_update(plane, old_plane_state); | 4683 | if (plane->type == DRM_PLANE_TYPE_CURSOR) |
| 4683 | continue; | 4684 | continue; |
| 4684 | } | ||
| 4685 | 4685 | ||
| 4686 | if (!fb || !crtc || pcrtc != crtc) | 4686 | if (!fb || !crtc || pcrtc != crtc) |
| 4687 | continue; | 4687 | continue; |
| @@ -4712,14 +4712,21 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state, · 24 unmodified lines | |||
| 4712 | */ | 4712 | */ |
| 4713 | abo = gem_to_amdgpu_bo(fb->obj[0]); | 4713 | abo = gem_to_amdgpu_bo(fb->obj[0]); |
| 4714 | r = amdgpu_bo_reserve(abo, true); | 4714 | r = amdgpu_bo_reserve(abo, true); |
| 4715 | if (unlikely(r != 0)) { | 4715 | if (unlikely(r != 0)) |
| 4716 | DRM_ERROR("failed to reserve buffer before flip\n"); | 4716 | DRM_ERROR("failed to reserve buffer before flip\n"); |
| 4717 | WARN_ON(1); | ||
| 4718 | } | ||
| 4719 | 4717 | ||
| 4720 | /* Wait for all fences on this FB */ | 4718 | /* |
| 4721 | WARN_ON(reservation_object_wait_timeout_rcu(abo->tbo.resv, true, false, | 4719 | * Wait for all fences on this FB. Do limited wait to avoid |
| 4722 | MAX_SCHEDULE_TIMEOUT) < 0); | 4720 | * deadlock during GPU reset when this fence will not signal |
| 4721 | * but we hold reservation lock for the BO. | ||
| 4722 | */ | ||
| 4723 | r = reservation_object_wait_timeout_rcu(abo->tbo.resv, | ||
| 4724 | true, false, | ||
| 4725 | msecs_to_jiffies(5000)); | ||
| 4726 | if (unlikely(r == 0)) | ||
| 4727 | DRM_ERROR("Waiting for fences timed out."); | ||
| 4728 | |||
| 4729 | |||
| 4723 | 4730 | ||
| 4724 | amdgpu_bo_get_tiling_flags(abo, &tiling_flags); | 4731 | amdgpu_bo_get_tiling_flags(abo, &tiling_flags); |
| 4725 | 4732 | ||
| @@ -4799,7 +4806,31 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state, · 73 unmodified lines | |||
| 4799 | * hopefully eliminating dc_*_update structs in their entirety. | 4806 | * hopefully eliminating dc_*_update structs in their entirety. |
| 4800 | */ | 4807 | */ |
| 4801 | if (flip_count) { | 4808 | if (flip_count) { |
| 4802 | target = (uint32_t)drm_crtc_vblank_count(pcrtc) + *wait_for_vblank; | 4809 | if (!vrr_active) { |
| 4810 | /* Use old throttling in non-vrr fixed refresh rate mode | ||
| 4811 | * to keep flip scheduling based on target vblank counts | ||
| 4812 | * working in a backwards compatible way, e.g., for | ||
| 4813 | * clients using the GLX_OML_sync_control extension or | ||
| 4814 | * DRI3/Present extension with defined target_msc. | ||
| 4815 | */ | ||
| 4816 | last_flip_vblank = drm_crtc_vblank_count(pcrtc); | ||
| 4817 | } | ||
| 4818 | else { | ||
| 4819 | /* For variable refresh rate mode only: | ||
| 4820 | * Get vblank of last completed flip to avoid > 1 vrr | ||
| 4821 | * flips per video frame by use of throttling, but allow | ||
| 4822 | * flip programming anywhere in the possibly large | ||
| 4823 | * variable vrr vblank interval for fine-grained flip | ||
| 4824 | * timing control and more opportunity to avoid stutter | ||
| 4825 | * on late submission of flips. | ||
| 4826 | */ | ||
| 4827 | spin_lock_irqsave(&pcrtc->dev->event_lock, flags); | ||
| 4828 | last_flip_vblank = acrtc_attach->last_flip_vblank; | ||
| 4829 | spin_unlock_irqrestore(&pcrtc->dev->event_lock, flags); | ||
| 4830 | } | ||
| 4831 | |||
| 4832 | target = (uint32_t)last_flip_vblank + *wait_for_vblank; | ||
| 4833 | |||
| 4803 | /* Prepare wait for target vblank early - before the fence-waits */ | 4834 | /* Prepare wait for target vblank early - before the fence-waits */ |
| 4804 | target_vblank = target - (uint32_t)drm_crtc_vblank_count(pcrtc) + | 4835 | target_vblank = target - (uint32_t)drm_crtc_vblank_count(pcrtc) + |
| 4805 | amdgpu_get_vblank_counter_kms(pcrtc->dev, acrtc_attach->crtc_id); | 4836 | amdgpu_get_vblank_counter_kms(pcrtc->dev, acrtc_attach->crtc_id); |
| @@ -4874,6 +4905,10 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state, · 68 unmodified lines | |||
| 4874 | mutex_unlock(&dm->dc_lock); | 4905 | mutex_unlock(&dm->dc_lock); |
| 4875 | } | 4906 | } |
| 4876 | 4907 | ||
| 4908 | for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, i) | ||
| 4909 | if (plane->type == DRM_PLANE_TYPE_CURSOR) | ||
| 4910 | handle_cursor_update(plane, old_plane_state); | ||
| 4911 | |||
| 4877 | cleanup: | 4912 | cleanup: |
| 4878 | kfree(flip); | 4913 | kfree(flip); |
| 4879 | kfree(full); | 4914 | kfree(full); |
| @@ -5799,14 +5834,13 @@ dm_determine_update_type_for_commit(struct dc *dc, · 919 unmodified lines | |||
| 5799 | old_dm_crtc_state = to_dm_crtc_state(old_crtc_state); | 5834 | old_dm_crtc_state = to_dm_crtc_state(old_crtc_state); |
| 5800 | num_plane = 0; | 5835 | num_plane = 0; |
| 5801 | 5836 | ||
| 5802 | if (!new_dm_crtc_state->stream) { | 5837 | if (new_dm_crtc_state->stream != old_dm_crtc_state->stream) { |
| 5803 | if (!new_dm_crtc_state->stream && old_dm_crtc_state->stream) { | 5838 | update_type = UPDATE_TYPE_FULL; |
| 5804 | update_type = UPDATE_TYPE_FULL; | 5839 | goto cleanup; |
| 5805 | goto cleanup; | 5840 | } |
| 5806 | } | ||
| 5807 | 5841 | ||
| 5842 | if (!new_dm_crtc_state->stream) | ||
| 5808 | continue; | 5843 | continue; |
| 5809 | } | ||
| 5810 | 5844 | ||
| 5811 | for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, j) { | 5845 | for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, j) { |
| 5812 | new_plane_crtc = new_plane_state->crtc; | 5846 | new_plane_crtc = new_plane_state->crtc; |
| @@ -5817,6 +5851,11 @@ dm_determine_update_type_for_commit(struct dc *dc, · 4 unmodified lines | |||
| 5817 | if (plane->type == DRM_PLANE_TYPE_CURSOR) | 5851 | if (plane->type == DRM_PLANE_TYPE_CURSOR) |
| 5818 | continue; | 5852 | continue; |
| 5819 | 5853 | ||
| 5854 | if (new_dm_plane_state->dc_state != old_dm_plane_state->dc_state) { | ||
| 5855 | update_type = UPDATE_TYPE_FULL; | ||
| 5856 | goto cleanup; | ||
| 5857 | } | ||
| 5858 | |||
| 5820 | if (!state->allow_modeset) | 5859 | if (!state->allow_modeset) |
| 5821 | continue; | 5860 | continue; |
| 5822 | 5861 | ||
| @@ -5955,6 +5994,42 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev, · 132 unmodified lines | |||
| 5955 | goto fail; | 5994 | goto fail; |
| 5956 | } | 5995 | } |
| 5957 | 5996 | ||
| 5997 | /* | ||
| 5998 | * Add all primary and overlay planes on the CRTC to the state | ||
| 5999 | * whenever a plane is enabled to maintain correct z-ordering | ||
| 6000 | * and to enable fast surface updates. | ||
| 6001 | */ | ||
| 6002 | drm_for_each_crtc(crtc, dev) { | ||
| 6003 | bool modified = false; | ||
| 6004 | |||
| 6005 | for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, i) { | ||
| 6006 | if (plane->type == DRM_PLANE_TYPE_CURSOR) | ||
| 6007 | continue; | ||
| 6008 | |||
| 6009 | if (new_plane_state->crtc == crtc || | ||
| 6010 | old_plane_state->crtc == crtc) { | ||
| 6011 | modified = true; | ||
| 6012 | break; | ||
| 6013 | } | ||
| 6014 | } | ||
| 6015 | |||
| 6016 | if (!modified) | ||
| 6017 | continue; | ||
| 6018 | |||
| 6019 | drm_for_each_plane_mask(plane, state->dev, crtc->state->plane_mask) { | ||
| 6020 | if (plane->type == DRM_PLANE_TYPE_CURSOR) | ||
| 6021 | continue; | ||
| 6022 | |||
| 6023 | new_plane_state = | ||
| 6024 | drm_atomic_get_plane_state(state, plane); | ||
| 6025 | |||
| 6026 | if (IS_ERR(new_plane_state)) { | ||
| 6027 | ret = PTR_ERR(new_plane_state); | ||
| 6028 | goto fail; | ||
| 6029 | } | ||
| 6030 | } | ||
| 6031 | } | ||
| 6032 | |||
| 5958 | /* Remove exiting planes if they are modified */ | 6033 | /* Remove exiting planes if they are modified */ |
| 5959 | for_each_oldnew_plane_in_state_reverse(state, plane, old_plane_state, new_plane_state, i) { | 6034 | for_each_oldnew_plane_in_state_reverse(state, plane, old_plane_state, new_plane_state, i) { |
| 5960 | ret = dm_update_plane_state(dc, state, plane, | 6035 | ret = dm_update_plane_state(dc, state, plane, |
| @@ -265,6 +265,7 @@ static struct atom_display_object_path_v2 *get_bios_object( · 264 unmodified lines | |||
| 265 | && id.enum_id == obj_id.enum_id) | 265 | && id.enum_id == obj_id.enum_id) |
| 266 | return &bp->object_info_tbl.v1_4->display_path[i]; | 266 | return &bp->object_info_tbl.v1_4->display_path[i]; |
| 267 | } | 267 | } |
| 268 | /* fall through */ | ||
| 268 | case OBJECT_TYPE_CONNECTOR: | 269 | case OBJECT_TYPE_CONNECTOR: |
| 269 | case OBJECT_TYPE_GENERIC: | 270 | case OBJECT_TYPE_GENERIC: |
| 270 | /* Both Generic and Connector Object ID | 271 | /* Both Generic and Connector Object ID |
| @@ -277,6 +278,7 @@ static struct atom_display_object_path_v2 *get_bios_object( · 6 unmodified lines | |||
| 277 | && id.enum_id == obj_id.enum_id) | 278 | && id.enum_id == obj_id.enum_id) |
| 278 | return &bp->object_info_tbl.v1_4->display_path[i]; | 279 | return &bp->object_info_tbl.v1_4->display_path[i]; |
| 279 | } | 280 | } |
| 281 | /* fall through */ | ||
| 280 | default: | 282 | default: |
| 281 | return NULL; | 283 | return NULL; |
| 282 | } | 284 | } |
| @@ -1138,6 +1138,9 @@ static enum dc_status dc_commit_state_no_check(struct dc *dc, struct dc_state *c · 1137 unmodified lines | |||
| 1138 | /* pplib is notified if disp_num changed */ | 1138 | /* pplib is notified if disp_num changed */ |
| 1139 | dc->hwss.optimize_bandwidth(dc, context); | 1139 | dc->hwss.optimize_bandwidth(dc, context); |
| 1140 | 1140 | ||
| 1141 | for (i = 0; i < context->stream_count; i++) | ||
| 1142 | context->streams[i]->mode_changed = false; | ||
| 1143 | |||
| 1141 | dc_release_state(dc->current_state); | 1144 | dc_release_state(dc->current_state); |
| 1142 | 1145 | ||
| 1143 | dc->current_state = context; | 1146 | dc->current_state = context; |
| @@ -1623,13 +1626,13 @@ static void commit_planes_do_stream_update(struct dc *dc, · 479 unmodified lines | |||
| 1623 | stream_update->adjust->v_total_min, | 1626 | stream_update->adjust->v_total_min, |
| 1624 | stream_update->adjust->v_total_max); | 1627 | stream_update->adjust->v_total_max); |
| 1625 | 1628 | ||
| 1626 | if (stream_update->periodic_vsync_config && pipe_ctx->stream_res.tg->funcs->program_vline_interrupt) | 1629 | if (stream_update->periodic_interrupt0 && |
| 1627 | pipe_ctx->stream_res.tg->funcs->program_vline_interrupt( | 1630 | dc->hwss.setup_periodic_interrupt) |
| 1628 | pipe_ctx->stream_res.tg, &pipe_ctx->stream->timing, VLINE0, &stream->periodic_vsync_config); | 1631 | dc->hwss.setup_periodic_interrupt(pipe_ctx, VLINE0); |
| 1629 | 1632 | ||
| 1630 | if (stream_update->enhanced_sync_config && pipe_ctx->stream_res.tg->funcs->program_vline_interrupt) | 1633 | if (stream_update->periodic_interrupt1 && |
| 1631 | pipe_ctx->stream_res.tg->funcs->program_vline_interrupt( | 1634 | dc->hwss.setup_periodic_interrupt) |
| 1632 | pipe_ctx->stream_res.tg, &pipe_ctx->stream->timing, VLINE1, &stream->enhanced_sync_config); | 1635 | dc->hwss.setup_periodic_interrupt(pipe_ctx, VLINE1); |
| 1633 | 1636 | ||
| 1634 | if ((stream_update->hdr_static_metadata && !stream->use_dynamic_meta) || | 1637 | if ((stream_update->hdr_static_metadata && !stream->use_dynamic_meta) || |
| 1635 | stream_update->vrr_infopacket || | 1638 | stream_update->vrr_infopacket || |
| @@ -51,9 +51,19 @@ struct freesync_context { · 50 unmodified lines | |||
| 51 | bool dummy; | 51 | bool dummy; |
| 52 | }; | 52 | }; |
| 53 | 53 | ||
| 54 | union vline_config { | 54 | enum vertical_interrupt_ref_point { |
| 55 | unsigned int line_number; | 55 | START_V_UPDATE = 0, |
| 56 | unsigned long long delta_in_ns; | 56 | START_V_SYNC, |
| 57 | INVALID_POINT | ||
| 58 | |||
| 59 | //For now, only v_update interrupt is used. | ||
| 60 | //START_V_BLANK, | ||
| 61 | //START_V_ACTIVE | ||
| 62 | }; | ||
| 63 | |||
| 64 | struct periodic_interrupt_config { | ||
| 65 | enum vertical_interrupt_ref_point ref_point; | ||
| 66 | int lines_offset; | ||
| 57 | }; | 67 | }; |
| 58 | 68 | ||
| 59 | 69 | ||
| @@ -106,8 +116,8 @@ struct dc_stream_state { · 46 unmodified lines | |||
| 106 | /* DMCU info */ | 116 | /* DMCU info */ |
| 107 | unsigned int abm_level; | 117 | unsigned int abm_level; |
| 108 | 118 | ||
| 109 | union vline_config periodic_vsync_config; | 119 | struct periodic_interrupt_config periodic_interrupt0; |
| 110 | union vline_config enhanced_sync_config; | 120 | struct periodic_interrupt_config periodic_interrupt1; |
| 111 | 121 | ||
| 112 | /* from core_stream struct */ | 122 | /* from core_stream struct */ |
| 113 | struct dc_context *ctx; | 123 | struct dc_context *ctx; |
| @@ -158,8 +168,8 @@ struct dc_stream_update { · 44 unmodified lines | |||
| 158 | struct dc_info_packet *hdr_static_metadata; | 168 | struct dc_info_packet *hdr_static_metadata; |
| 159 | unsigned int *abm_level; | 169 | unsigned int *abm_level; |
| 160 | 170 | ||
| 161 | union vline_config *periodic_vsync_config; | 171 | struct periodic_interrupt_config *periodic_interrupt0; |
| 162 | union vline_config *enhanced_sync_config; | 172 | struct periodic_interrupt_config *periodic_interrupt1; |
| 163 | 173 | ||
| 164 | struct dc_crtc_timing_adjust *adjust; | 174 | struct dc_crtc_timing_adjust *adjust; |
| 165 | struct dc_info_packet *vrr_infopacket; | 175 | struct dc_info_packet *vrr_infopacket; |
| @@ -53,6 +53,27 @@ · 52 unmodified lines | |||
| 53 | 53 | ||
| 54 | #define MCP_DISABLE_ABM_IMMEDIATELY 255 | 54 | #define MCP_DISABLE_ABM_IMMEDIATELY 255 |
| 55 | 55 | ||
| 56 | static bool dce_abm_set_pipe(struct abm *abm, uint32_t controller_id) | ||
| 57 | { | ||
| 58 | struct dce_abm *abm_dce = TO_DCE_ABM(abm); | ||
| 59 | uint32_t rampingBoundary = 0xFFFF; | ||
| 60 | |||
| 61 | REG_WAIT(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, 0, | ||
| 62 | 1, 80000); | ||
| 63 | |||
| 64 | /* set ramping boundary */ | ||
| 65 | REG_WRITE(MASTER_COMM_DATA_REG1, rampingBoundary); | ||
| 66 | |||
| 67 | /* setDMCUParam_Pipe */ | ||
| 68 | REG_UPDATE_2(MASTER_COMM_CMD_REG, | ||
| 69 | MASTER_COMM_CMD_REG_BYTE0, MCP_ABM_PIPE_SET, | ||
| 70 | MASTER_COMM_CMD_REG_BYTE1, controller_id); | ||
| 71 | |||
| 72 | /* notifyDMCUMsg */ | ||
| 73 | REG_UPDATE(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, 1); | ||
| 74 | |||
| 75 | return true; | ||
| 76 | } | ||
| 56 | 77 | ||
| 57 | static unsigned int calculate_16_bit_backlight_from_pwm(struct dce_abm *abm_dce) | 78 | static unsigned int calculate_16_bit_backlight_from_pwm(struct dce_abm *abm_dce) |
| 58 | { | 79 | { |
| @@ -175,7 +196,6 @@ static void dmcu_set_backlight_level( · 116 unmodified lines | |||
| 175 | uint32_t controller_id) | 196 | uint32_t controller_id) |
| 176 | { | 197 | { |
| 177 | unsigned int backlight_8_bit = 0; | 198 | unsigned int backlight_8_bit = 0; |
| 178 | uint32_t rampingBoundary = 0xFFFF; | ||
| 179 | uint32_t s2; | 199 | uint32_t s2; |
| 180 | 200 | ||
| 181 | if (backlight_pwm_u16_16 & 0x10000) | 201 | if (backlight_pwm_u16_16 & 0x10000) |
| @@ -185,16 +205,7 @@ static void dmcu_set_backlight_level( · 3 unmodified lines | |||
| 185 | // Take MSB of fractional part since backlight is not max | 205 | // Take MSB of fractional part since backlight is not max |
| 186 | backlight_8_bit = (backlight_pwm_u16_16 >> 8) & 0xFF; | 206 | backlight_8_bit = (backlight_pwm_u16_16 >> 8) & 0xFF; |
| 187 | 207 | ||
| 188 | /* set ramping boundary */ | 208 | dce_abm_set_pipe(&abm_dce->base, controller_id); |
| 189 | REG_WRITE(MASTER_COMM_DATA_REG1, rampingBoundary); | ||
| 190 | |||
| 191 | /* setDMCUParam_Pipe */ | ||
| 192 | REG_UPDATE_2(MASTER_COMM_CMD_REG, | ||
| 193 | MASTER_COMM_CMD_REG_BYTE0, MCP_ABM_PIPE_SET, | ||
| 194 | MASTER_COMM_CMD_REG_BYTE1, controller_id); | ||
| 195 | |||
| 196 | /* notifyDMCUMsg */ | ||
| 197 | REG_UPDATE(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, 1); | ||
| 198 | 209 | ||
| 199 | /* waitDMCUReadyForCmd */ | 210 | /* waitDMCUReadyForCmd */ |
| 200 | REG_WAIT(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, | 211 | REG_WAIT(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, |
| @@ -309,16 +320,7 @@ static bool dce_abm_immediate_disable(struct abm *abm) · 108 unmodified lines | |||
| 309 | { | 320 | { |
| 310 | struct dce_abm *abm_dce = TO_DCE_ABM(abm); | 321 | struct dce_abm *abm_dce = TO_DCE_ABM(abm); |
| 311 | 322 | ||
| 312 | REG_WAIT(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, 0, | 323 | dce_abm_set_pipe(abm, MCP_DISABLE_ABM_IMMEDIATELY); |
| 313 | 1, 80000); | ||
| 314 | |||
| 315 | /* setDMCUParam_ABMLevel */ | ||
| 316 | REG_UPDATE_2(MASTER_COMM_CMD_REG, | ||
| 317 | MASTER_COMM_CMD_REG_BYTE0, MCP_ABM_PIPE_SET, | ||
| 318 | MASTER_COMM_CMD_REG_BYTE1, MCP_DISABLE_ABM_IMMEDIATELY); | ||
| 319 | |||
| 320 | /* notifyDMCUMsg */ | ||
| 321 | REG_UPDATE(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, 1); | ||
| 322 | 324 | ||
| 323 | abm->stored_backlight_registers.BL_PWM_CNTL = | 325 | abm->stored_backlight_registers.BL_PWM_CNTL = |
| 324 | REG_READ(BL_PWM_CNTL); | 326 | REG_READ(BL_PWM_CNTL); |
| @@ -419,6 +421,7 @@ static const struct abm_funcs dce_funcs = { · 94 unmodified lines | |||
| 419 | .abm_init = dce_abm_init, | 421 | .abm_init = dce_abm_init, |
| 420 | .set_abm_level = dce_abm_set_level, | 422 | .set_abm_level = dce_abm_set_level, |
| 421 | .init_backlight = dce_abm_init_backlight, | 423 | .init_backlight = dce_abm_init_backlight, |
| 424 | .set_pipe = dce_abm_set_pipe, | ||
| 422 | .set_backlight_level_pwm = dce_abm_set_backlight_level_pwm, | 425 | .set_backlight_level_pwm = dce_abm_set_backlight_level_pwm, |
| 423 | .get_current_backlight = dce_abm_get_current_backlight, | 426 | .get_current_backlight = dce_abm_get_current_backlight, |
| 424 | .get_target_backlight = dce_abm_get_target_backlight, | 427 | .get_target_backlight = dce_abm_get_target_backlight, |
| @@ -696,6 +696,11 @@ static void dce11_update_clocks(struct clk_mgr *clk_mgr, · 695 unmodified lines | |||
| 696 | { | 696 | { |
| 697 | struct dce_clk_mgr *clk_mgr_dce = TO_DCE_CLK_MGR(clk_mgr); | 697 | struct dce_clk_mgr *clk_mgr_dce = TO_DCE_CLK_MGR(clk_mgr); |
| 698 | struct dm_pp_power_level_change_request level_change_req; | 698 | struct dm_pp_power_level_change_request level_change_req; |
| 699 | int patched_disp_clk = context->bw.dce.dispclk_khz; | ||
| 700 | |||
| 701 | /*TODO: W/A for dal3 linux, investigate why this works */ | ||
| 702 | if (!clk_mgr_dce->dfs_bypass_active) | ||
| 703 | patched_disp_clk = patched_disp_clk * 115 / 100; | ||
| 699 | 704 | ||
| 700 | level_change_req.power_level = dce_get_required_clocks_state(clk_mgr, context); | 705 | level_change_req.power_level = dce_get_required_clocks_state(clk_mgr, context); |
| 701 | /* get max clock state from PPLIB */ | 706 | /* get max clock state from PPLIB */ |
| @@ -705,9 +710,9 @@ static void dce11_update_clocks(struct clk_mgr *clk_mgr, · 3 unmodified lines | |||
| 705 | clk_mgr_dce->cur_min_clks_state = level_change_req.power_level; | 710 | clk_mgr_dce->cur_min_clks_state = level_change_req.power_level; |
| 706 | } | 711 | } |
| 707 | 712 | ||
| 708 | if (should_set_clock(safe_to_lower, context->bw.dce.dispclk_khz, clk_mgr->clks.dispclk_khz)) { | 713 | if (should_set_clock(safe_to_lower, patched_disp_clk, clk_mgr->clks.dispclk_khz)) { |
| 709 | context->bw.dce.dispclk_khz = dce_set_clock(clk_mgr, context->bw.dce.dispclk_khz); | 714 | context->bw.dce.dispclk_khz = dce_set_clock(clk_mgr, patched_disp_clk); |
| 710 | clk_mgr->clks.dispclk_khz = context->bw.dce.dispclk_khz; | 715 | clk_mgr->clks.dispclk_khz = patched_disp_clk; |
| 711 | } | 716 | } |
| 712 | dce11_pplib_apply_display_requirements(clk_mgr->ctx->dc, context); | 717 | dce11_pplib_apply_display_requirements(clk_mgr->ctx->dc, context); |
| 713 | } | 718 | } |
| @@ -479,7 +479,7 @@ static void program_grph_pixel_format( · 478 unmodified lines | |||
| 479 | case SURFACE_PIXEL_FORMAT_GRPH_ABGR16161616F: | 479 | case SURFACE_PIXEL_FORMAT_GRPH_ABGR16161616F: |
| 480 | sign = 1; | 480 | sign = 1; |
| 481 | floating = 1; | 481 | floating = 1; |
| 482 | /* no break */ | 482 | /* fall through */ |
| 483 | case SURFACE_PIXEL_FORMAT_GRPH_ARGB16161616F: /* shouldn't this get float too? */ | 483 | case SURFACE_PIXEL_FORMAT_GRPH_ARGB16161616F: /* shouldn't this get float too? */ |
| 484 | case SURFACE_PIXEL_FORMAT_GRPH_ARGB16161616: | 484 | case SURFACE_PIXEL_FORMAT_GRPH_ARGB16161616: |
| 485 | grph_depth = 3; | 485 | grph_depth = 3; |
| @@ -37,6 +37,10 @@ void dce100_prepare_bandwidth( · 36 unmodified lines | |||
| 37 | struct dc *dc, | 37 | struct dc *dc, |
| 38 | struct dc_state *context); | 38 | struct dc_state *context); |
| 39 | 39 | ||
| 40 | void dce100_optimize_bandwidth( | ||
| 41 | struct dc *dc, | ||
| 42 | struct dc_state *context); | ||
| 43 | |||
| 40 | bool dce100_enable_display_power_gating(struct dc *dc, uint8_t controller_id, | 44 | bool dce100_enable_display_power_gating(struct dc *dc, uint8_t controller_id, |
| 41 | struct dc_bios *dcb, | 45 | struct dc_bios *dcb, |
| 42 | enum pipe_gating_control power_gating); | 46 | enum pipe_gating_control power_gating); |
| @@ -1300,6 +1300,10 @@ static enum dc_status apply_single_controller_ctx_to_hw( · 1299 unmodified lines | |||
| 1300 | struct drr_params params = {0}; | 1300 | struct drr_params params = {0}; |
| 1301 | unsigned int event_triggers = 0; | 1301 | unsigned int event_triggers = 0; |
| 1302 | 1302 | ||
| 1303 | if (dc->hwss.disable_stream_gating) { | ||
| 1304 | dc->hwss.disable_stream_gating(dc, pipe_ctx); | ||
| 1305 | } | ||
| 1306 | |||
| 1303 | if (pipe_ctx->stream_res.audio != NULL) { | 1307 | if (pipe_ctx->stream_res.audio != NULL) { |
| 1304 | struct audio_output audio_output; | 1308 | struct audio_output audio_output; |
| 1305 | 1309 | ||
| @@ -1329,10 +1333,8 @@ static enum dc_status apply_single_controller_ctx_to_hw( · 23 unmodified lines | |||
| 1329 | if (!pipe_ctx->stream->apply_seamless_boot_optimization) | 1333 | if (!pipe_ctx->stream->apply_seamless_boot_optimization) |
| 1330 | dc->hwss.enable_stream_timing(pipe_ctx, context, dc); | 1334 | dc->hwss.enable_stream_timing(pipe_ctx, context, dc); |
| 1331 | 1335 | ||
| 1332 | if (pipe_ctx->stream_res.tg->funcs->program_vupdate_interrupt) | 1336 | if (dc->hwss.setup_vupdate_interrupt) |
| 1333 | pipe_ctx->stream_res.tg->funcs->program_vupdate_interrupt( | 1337 | dc->hwss.setup_vupdate_interrupt(pipe_ctx); |
| 1334 | pipe_ctx->stream_res.tg, | ||
| 1335 | &stream->timing); | ||
| 1336 | 1338 | ||
| 1337 | params.vertical_total_min = stream->adjust.v_total_min; | 1339 | params.vertical_total_min = stream->adjust.v_total_min; |
| 1338 | params.vertical_total_max = stream->adjust.v_total_max; | 1340 | params.vertical_total_max = stream->adjust.v_total_max; |
| @@ -1521,6 +1523,14 @@ void dce110_enable_accelerated_mode(struct dc *dc, struct dc_state *context) · 182 unmodified lines | |||
| 1521 | struct dc_link *edp_link = get_link_for_edp(dc); | 1523 | struct dc_link *edp_link = get_link_for_edp(dc); |
| 1522 | bool can_edp_fast_boot_optimize = false; | 1524 | bool can_edp_fast_boot_optimize = false; |
| 1523 | bool apply_edp_fast_boot_optimization = false; | 1525 | bool apply_edp_fast_boot_optimization = false; |
| 1526 | bool can_apply_seamless_boot = false; | ||
| 1527 | |||
| 1528 | for (i = 0; i < context->stream_count; i++) { | ||
| 1529 | if (context->streams[i]->apply_seamless_boot_optimization) { | ||
| 1530 | can_apply_seamless_boot = true; | ||
| 1531 | break; | ||
| 1532 | } | ||
| 1533 | } | ||
| 1524 | 1534 | ||
| 1525 | if (edp_link) { | 1535 | if (edp_link) { |
| 1526 | /* this seems to cause blank screens on DCE8 */ | 1536 | /* this seems to cause blank screens on DCE8 */ |
| @@ -1549,7 +1559,7 @@ void dce110_enable_accelerated_mode(struct dc *dc, struct dc_state *context) · 22 unmodified lines | |||
| 1549 | } | 1559 | } |
| 1550 | } | 1560 | } |
| 1551 | 1561 | ||
| 1552 | if (!apply_edp_fast_boot_optimization) { | 1562 | if (!apply_edp_fast_boot_optimization && !can_apply_seamless_boot) { |
| 1553 | if (edp_link_to_turnoff) { | 1563 | if (edp_link_to_turnoff) { |
| 1554 | /*turn off backlight before DP_blank and encoder powered down*/ | 1564 | /*turn off backlight before DP_blank and encoder powered down*/ |
| 1555 | dc->hwss.edp_backlight_control(edp_link_to_turnoff, false); | 1565 | dc->hwss.edp_backlight_control(edp_link_to_turnoff, false); |
| @@ -2676,6 +2686,8 @@ static const struct hw_sequencer_funcs dce110_funcs = { · 1120 unmodified lines | |||
| 2676 | .set_static_screen_control = set_static_screen_control, | 2686 | .set_static_screen_control = set_static_screen_control, |
| 2677 | .reset_hw_ctx_wrap = dce110_reset_hw_ctx_wrap, | 2687 | .reset_hw_ctx_wrap = dce110_reset_hw_ctx_wrap, |
| 2678 | .enable_stream_timing = dce110_enable_stream_timing, | 2688 | .enable_stream_timing = dce110_enable_stream_timing, |
| 2689 | .disable_stream_gating = NULL, | ||
| 2690 | .enable_stream_gating = NULL, | ||
| 2679 | .setup_stereo = NULL, | 2691 | .setup_stereo = NULL, |
| 2680 | .set_avmute = dce110_set_avmute, | 2692 | .set_avmute = dce110_set_avmute, |
| 2681 | .wait_for_mpcc_disconnect = dce110_wait_for_mpcc_disconnect, | 2693 | .wait_for_mpcc_disconnect = dce110_wait_for_mpcc_disconnect, |
| @@ -77,6 +77,6 @@ void dce80_hw_sequencer_construct(struct dc *dc) · 76 unmodified lines | |||
| 77 | dc->hwss.enable_display_power_gating = dce100_enable_display_power_gating; | 77 | dc->hwss.enable_display_power_gating = dce100_enable_display_power_gating; |
| 78 | dc->hwss.pipe_control_lock = dce_pipe_control_lock; | 78 | dc->hwss.pipe_control_lock = dce_pipe_control_lock; |
| 79 | dc->hwss.prepare_bandwidth = dce100_prepare_bandwidth; | 79 | dc->hwss.prepare_bandwidth = dce100_prepare_bandwidth; |
| 80 | dc->hwss.optimize_bandwidth = dce100_prepare_bandwidth; | 80 | dc->hwss.optimize_bandwidth = dce100_optimize_bandwidth; |
| 81 | } | 81 | } |
| 82 | 82 | ||
| @@ -792,9 +792,22 @@ bool dce80_validate_bandwidth( · 791 unmodified lines | |||
| 792 | struct dc *dc, | 792 | struct dc *dc, |
| 793 | struct dc_state *context) | 793 | struct dc_state *context) |
| 794 | { | 794 | { |
| 795 | /* TODO implement when needed but for now hardcode max value*/ | 795 | int i; |
| 796 | context->bw.dce.dispclk_khz = 681000; | 796 | bool at_least_one_pipe = false; |
| 797 | context->bw.dce.yclk_khz = 250000 * MEMORY_TYPE_MULTIPLIER_CZ; | 797 | |
| 798 | for (i = 0; i < dc->res_pool->pipe_count; i++) { | ||
| 799 | if (context->res_ctx.pipe_ctx[i].stream) | ||
| 800 | at_least_one_pipe = true; | ||
| 801 | } | ||
| 802 | |||
| 803 | if (at_least_one_pipe) { | ||
| 804 | /* TODO implement when needed but for now hardcode max value*/ | ||
| 805 | context->bw.dce.dispclk_khz = 681000; | ||
| 806 | context->bw.dce.yclk_khz = 250000 * MEMORY_TYPE_MULTIPLIER_CZ; | ||
| 807 | } else { | ||
| 808 | context->bw.dce.dispclk_khz = 0; | ||
| 809 | context->bw.dce.yclk_khz = 0; | ||
| 810 | } | ||
| 798 | 811 | ||
| 799 | return true; | 812 | return true; |
| 800 | } | 813 | } |
| @@ -959,9 +959,25 @@ static void dcn10_disable_plane(struct dc *dc, struct pipe_ctx *pipe_ctx) · 958 unmodified lines | |||
| 959 | static void dcn10_init_pipes(struct dc *dc, struct dc_state *context) | 959 | static void dcn10_init_pipes(struct dc *dc, struct dc_state *context) |
| 960 | { | 960 | { |
| 961 | int i; | 961 | int i; |
| 962 | bool can_apply_seamless_boot = false; | ||
| 963 | |||
| 964 | for (i = 0; i < context->stream_count; i++) { | ||
| 965 | if (context->streams[i]->apply_seamless_boot_optimization) { | ||
| 966 | can_apply_seamless_boot = true; | ||
| 967 | break; | ||
| 968 | } | ||
| 969 | } | ||
| 962 | 970 | ||
| 963 | for (i = 0; i < dc->res_pool->pipe_count; i++) { | 971 | for (i = 0; i < dc->res_pool->pipe_count; i++) { |
| 964 | struct timing_generator *tg = dc->res_pool->timing_generators[i]; | 972 | struct timing_generator *tg = dc->res_pool->timing_generators[i]; |
| 973 | struct pipe_ctx *pipe_ctx = &context->res_ctx.pipe_ctx[i]; | ||
| 974 | |||
| 975 | /* There is assumption that pipe_ctx is not mapping irregularly | ||
| 976 | * to non-preferred front end. If pipe_ctx->stream is not NULL, | ||
| 977 | * we will use the pipe, so don't disable | ||
| 978 | */ | ||
| 979 | if (pipe_ctx->stream != NULL) | ||
| 980 | continue; | ||
| 965 | 981 | ||
| 966 | if (tg->funcs->is_tg_enabled(tg)) | 982 | if (tg->funcs->is_tg_enabled(tg)) |
| 967 | tg->funcs->lock(tg); | 983 | tg->funcs->lock(tg); |
| @@ -975,7 +991,9 @@ static void dcn10_init_pipes(struct dc *dc, struct dc_state *context) · 7 unmodified lines | |||
| 975 | } | 991 | } |
| 976 | } | 992 | } |
| 977 | 993 | ||
| 978 | dc->res_pool->mpc->funcs->mpc_init(dc->res_pool->mpc); | 994 | /* Cannot reset the MPC mux if seamless boot */ |
| 995 | if (!can_apply_seamless_boot) | ||
| 996 | dc->res_pool->mpc->funcs->mpc_init(dc->res_pool->mpc); | ||
| 979 | 997 | ||
| 980 | for (i = 0; i < dc->res_pool->pipe_count; i++) { | 998 | for (i = 0; i < dc->res_pool->pipe_count; i++) { |
| 981 | struct timing_generator *tg = dc->res_pool->timing_generators[i]; | 999 | struct timing_generator *tg = dc->res_pool->timing_generators[i]; |
| @@ -983,6 +1001,16 @@ static void dcn10_init_pipes(struct dc *dc, struct dc_state *context) · 1 unmodified line | |||
| 983 | struct dpp *dpp = dc->res_pool->dpps[i]; | 1001 | struct dpp *dpp = dc->res_pool->dpps[i]; |
| 984 | struct pipe_ctx *pipe_ctx = &context->res_ctx.pipe_ctx[i]; | 1002 | struct pipe_ctx *pipe_ctx = &context->res_ctx.pipe_ctx[i]; |
| 985 | 1003 | ||
| 1004 | // W/A for issue with dc_post_update_surfaces_to_stream | ||
| 1005 | hubp->power_gated = true; | ||
| 1006 | |||
| 1007 | /* There is assumption that pipe_ctx is not mapping irregularly | ||
| 1008 | * to non-preferred front end. If pipe_ctx->stream is not NULL, | ||
| 1009 | * we will use the pipe, so don't disable | ||
| 1010 | */ | ||
| 1011 | if (pipe_ctx->stream != NULL) | ||
| 1012 | continue; | ||
| 1013 | |||
| 986 | dpp->funcs->dpp_reset(dpp); | 1014 | dpp->funcs->dpp_reset(dpp); |
| 987 | 1015 | ||
| 988 | pipe_ctx->stream_res.tg = tg; | 1016 | pipe_ctx->stream_res.tg = tg; |
| @@ -1137,11 +1165,13 @@ static void reset_hw_ctx_wrap( · 148 unmodified lines | |||
| 1137 | struct clock_source *old_clk = pipe_ctx_old->clock_source; | 1165 | struct clock_source *old_clk = pipe_ctx_old->clock_source; |
| 1138 | 1166 | ||
| 1139 | reset_back_end_for_pipe(dc, pipe_ctx_old, dc->current_state); | 1167 | reset_back_end_for_pipe(dc, pipe_ctx_old, dc->current_state); |
| 1168 | if (dc->hwss.enable_stream_gating) { | ||
| 1169 | dc->hwss.enable_stream_gating(dc, pipe_ctx); | ||
| 1170 | } | ||
| 1140 | if (old_clk) | 1171 | if (old_clk) |
| 1141 | old_clk->funcs->cs_power_down(old_clk); | 1172 | old_clk->funcs->cs_power_down(old_clk); |
| 1142 | } | 1173 | } |
| 1143 | } | 1174 | } |
| 1144 | |||
| 1145 | } | 1175 | } |
| 1146 | 1176 | ||
| 1147 | static bool patch_address_for_sbs_tb_stereo( | 1177 | static bool patch_address_for_sbs_tb_stereo( |
| @@ -2162,8 +2192,10 @@ static void dcn10_blank_pixel_data( · 1014 unmodified lines | |||
| 2162 | if (!blank) { | 2192 | if (!blank) { |
| 2163 | if (stream_res->tg->funcs->set_blank) | 2193 | if (stream_res->tg->funcs->set_blank) |
| 2164 | stream_res->tg->funcs->set_blank(stream_res->tg, blank); | 2194 | stream_res->tg->funcs->set_blank(stream_res->tg, blank); |
| 2165 | if (stream_res->abm) | 2195 | if (stream_res->abm) { |
| 2196 | stream_res->abm->funcs->set_pipe(stream_res->abm, stream_res->tg->inst + 1); | ||
| 2166 | stream_res->abm->funcs->set_abm_level(stream_res->abm, stream->abm_level); | 2197 | stream_res->abm->funcs->set_abm_level(stream_res->abm, stream->abm_level); |
| 2198 | } | ||
| 2167 | } else if (blank) { | 2199 | } else if (blank) { |
| 2168 | if (stream_res->abm) | 2200 | if (stream_res->abm) |
| 2169 | stream_res->abm->funcs->set_abm_immediate_disable(stream_res->abm); | 2201 | stream_res->abm->funcs->set_abm_immediate_disable(stream_res->abm); |
| @@ -2661,8 +2693,8 @@ static void dcn10_set_cursor_position(struct pipe_ctx *pipe_ctx) · 491 unmodified lines | |||
| 2661 | .mirror = pipe_ctx->plane_state->horizontal_mirror | 2693 | .mirror = pipe_ctx->plane_state->horizontal_mirror |
| 2662 | }; | 2694 | }; |
| 2663 | 2695 | ||
| 2664 | pos_cpy.x -= pipe_ctx->plane_state->dst_rect.x; | 2696 | pos_cpy.x_hotspot += pipe_ctx->plane_state->dst_rect.x; |
| 2665 | pos_cpy.y -= pipe_ctx->plane_state->dst_rect.y; | 2697 | pos_cpy.y_hotspot += pipe_ctx->plane_state->dst_rect.y; |
| 2666 | 2698 | ||
| 2667 | if (pipe_ctx->plane_state->address.type | 2699 | if (pipe_ctx->plane_state->address.type |
| 2668 | == PLN_ADDR_TYPE_VIDEO_PROGRESSIVE) | 2700 | == PLN_ADDR_TYPE_VIDEO_PROGRESSIVE) |
| @@ -2709,6 +2741,147 @@ static void dcn10_set_cursor_sdr_white_level(struct pipe_ctx *pipe_ctx) · 40 unmodified lines | |||
| 2709 | pipe_ctx->plane_res.dpp, &opt_attr); | 2741 | pipe_ctx->plane_res.dpp, &opt_attr); |
| 2710 | } | 2742 | } |
| 2711 | 2743 | ||
| 2744 | /** | ||
| 2745 | * apply_front_porch_workaround TODO FPGA still need? | ||
| 2746 | * | ||
| 2747 | * This is a workaround for a bug that has existed since R5xx and has not been | ||
| 2748 | * fixed keep Front porch at minimum 2 for Interlaced mode or 1 for progressive. | ||
| 2749 | */ | ||
| 2750 | static void apply_front_porch_workaround( | ||
| 2751 | struct dc_crtc_timing *timing) | ||
| 2752 | { | ||
| 2753 | if (timing->flags.INTERLACE == 1) { | ||
| 2754 | if (timing->v_front_porch < 2) | ||
| 2755 | timing->v_front_porch = 2; | ||
| 2756 | } else { | ||
| 2757 | if (timing->v_front_porch < 1) | ||
| 2758 | timing->v_front_porch = 1; | ||
| 2759 | } | ||
| 2760 | } | ||
| 2761 | |||
| 2762 | int get_vupdate_offset_from_vsync(struct pipe_ctx *pipe_ctx) | ||
| 2763 | { | ||
| 2764 | struct timing_generator *optc = pipe_ctx->stream_res.tg; | ||
| 2765 | const struct dc_crtc_timing *dc_crtc_timing = &pipe_ctx->stream->timing; | ||
| 2766 | struct dc_crtc_timing patched_crtc_timing; | ||
| 2767 | int vesa_sync_start; | ||
| 2768 | int asic_blank_end; | ||
| 2769 | int interlace_factor; | ||
| 2770 | int vertical_line_start; | ||
| 2771 | |||
| 2772 | patched_crtc_timing = *dc_crtc_timing; | ||
| 2773 | apply_front_porch_workaround(&patched_crtc_timing); | ||
| 2774 | |||
| 2775 | interlace_factor = patched_crtc_timing.flags.INTERLACE ? 2 : 1; | ||
| 2776 | |||
| 2777 | vesa_sync_start = patched_crtc_timing.v_addressable + | ||
| 2778 | patched_crtc_timing.v_border_bottom + | ||
| 2779 | patched_crtc_timing.v_front_porch; | ||
| 2780 | |||
| 2781 | asic_blank_end = (patched_crtc_timing.v_total - | ||
| 2782 | vesa_sync_start - | ||
| 2783 | patched_crtc_timing.v_border_top) | ||
| 2784 | * interlace_factor; | ||
| 2785 | |||
| 2786 | vertical_line_start = asic_blank_end - | ||
| 2787 | optc->dlg_otg_param.vstartup_start + 1; | ||
| 2788 | |||
| 2789 | return vertical_line_start; | ||
| 2790 | } | ||
| 2791 | |||
| 2792 | static void calc_vupdate_position( | ||
| 2793 | struct pipe_ctx *pipe_ctx, | ||
| 2794 | uint32_t *start_line, | ||
| 2795 | uint32_t *end_line) | ||
| 2796 | { | ||
| 2797 | const struct dc_crtc_timing *dc_crtc_timing = &pipe_ctx->stream->timing; | ||
| 2798 | int vline_int_offset_from_vupdate = | ||
| 2799 | pipe_ctx->stream->periodic_interrupt0.lines_offset; | ||
| 2800 | int vupdate_offset_from_vsync = get_vupdate_offset_from_vsync(pipe_ctx); | ||
| 2801 | int start_position; | ||
| 2802 | |||
| 2803 | if (vline_int_offset_from_vupdate > 0) | ||
| 2804 | vline_int_offset_from_vupdate--; | ||
| 2805 | else if (vline_int_offset_from_vupdate < 0) | ||
| 2806 | vline_int_offset_from_vupdate++; | ||
| 2807 | |||
| 2808 | start_position = vline_int_offset_from_vupdate + vupdate_offset_from_vsync; | ||
| 2809 | |||
| 2810 | if (start_position >= 0) | ||
| 2811 | *start_line = start_position; | ||
| 2812 | else | ||
| 2813 | *start_line = dc_crtc_timing->v_total + start_position - 1; | ||
| 2814 | |||
| 2815 | *end_line = *start_line + 2; | ||
| 2816 | |||
| 2817 | if (*end_line >= dc_crtc_timing->v_total) | ||
| 2818 | *end_line = 2; | ||
| 2819 | } | ||
| 2820 | |||
| 2821 | static void cal_vline_position( | ||
| 2822 | struct pipe_ctx *pipe_ctx, | ||
| 2823 | enum vline_select vline, | ||
| 2824 | uint32_t *start_line, | ||
| 2825 | uint32_t *end_line) | ||
| 2826 | { | ||
| 2827 | enum vertical_interrupt_ref_point ref_point = INVALID_POINT; | ||
| 2828 | |||
| 2829 | if (vline == VLINE0) | ||
| 2830 | ref_point = pipe_ctx->stream->periodic_interrupt0.ref_point; | ||
| 2831 | else if (vline == VLINE1) | ||
| 2832 | ref_point = pipe_ctx->stream->periodic_interrupt1.ref_point; | ||
| 2833 | |||
| 2834 | switch (ref_point) { | ||
| 2835 | case START_V_UPDATE: | ||
| 2836 | calc_vupdate_position( | ||
| 2837 | pipe_ctx, | ||
| 2838 | start_line, | ||
| 2839 | end_line); | ||
| 2840 | break; | ||
| 2841 | case START_V_SYNC: | ||
| 2842 | // Suppose to do nothing because vsync is 0; | ||
| 2843 | break; | ||
| 2844 | default: | ||
| 2845 | ASSERT(0); | ||
| 2846 | break; | ||
| 2847 | } | ||
| 2848 | } | ||
| 2849 | |||
| 2850 | static void dcn10_setup_periodic_interrupt( | ||
| 2851 | struct pipe_ctx *pipe_ctx, | ||
| 2852 | enum vline_select vline) | ||
| 2853 | { | ||
| 2854 | struct timing_generator *tg = pipe_ctx->stream_res.tg; | ||
| 2855 | |||
| 2856 | if (vline == VLINE0) { | ||
| 2857 | uint32_t start_line = 0; | ||
| 2858 | uint32_t end_line = 0; | ||
| 2859 | |||
| 2860 | cal_vline_position(pipe_ctx, vline, &start_line, &end_line); | ||
| 2861 | |||
| 2862 | tg->funcs->setup_vertical_interrupt0(tg, start_line, end_line); | ||
| 2863 | |||
| 2864 | } else if (vline == VLINE1) { | ||
| 2865 | pipe_ctx->stream_res.tg->funcs->setup_vertical_interrupt1( | ||
| 2866 | tg, | ||
| 2867 | pipe_ctx->stream->periodic_interrupt1.lines_offset); | ||
| 2868 | } | ||
| 2869 | } | ||
| 2870 | |||
| 2871 | static void dcn10_setup_vupdate_interrupt(struct pipe_ctx *pipe_ctx) | ||
| 2872 | { | ||
| 2873 | struct timing_generator *tg = pipe_ctx->stream_res.tg; | ||
| 2874 | int start_line = get_vupdate_offset_from_vsync(pipe_ctx); | ||
| 2875 | |||
| 2876 | if (start_line < 0) { | ||
| 2877 | ASSERT(0); | ||
| 2878 | start_line = 0; | ||
| 2879 | } | ||
| 2880 | |||
| 2881 | if (tg->funcs->setup_vertical_interrupt2) | ||
| 2882 | tg->funcs->setup_vertical_interrupt2(tg, start_line); | ||
| 2883 | } | ||
| 2884 | |||
| 2712 | static const struct hw_sequencer_funcs dcn10_funcs = { | 2885 | static const struct hw_sequencer_funcs dcn10_funcs = { |
| 2713 | .program_gamut_remap = program_gamut_remap, | 2886 | .program_gamut_remap = program_gamut_remap, |
| 2714 | .init_hw = dcn10_init_hw, | 2887 | .init_hw = dcn10_init_hw, |
| @@ -2756,7 +2929,11 @@ static const struct hw_sequencer_funcs dcn10_funcs = { · 41 unmodified lines | |||
| 2756 | .edp_wait_for_hpd_ready = hwss_edp_wait_for_hpd_ready, | 2929 | .edp_wait_for_hpd_ready = hwss_edp_wait_for_hpd_ready, |
| 2757 | .set_cursor_position = dcn10_set_cursor_position, | 2930 | .set_cursor_position = dcn10_set_cursor_position, |
| 2758 | .set_cursor_attribute = dcn10_set_cursor_attribute, | 2931 | .set_cursor_attribute = dcn10_set_cursor_attribute, |
| 2759 | .set_cursor_sdr_white_level = dcn10_set_cursor_sdr_white_level | 2932 | .set_cursor_sdr_white_level = dcn10_set_cursor_sdr_white_level, |
| 2933 | .disable_stream_gating = NULL, | ||
| 2934 | .enable_stream_gating = NULL, | ||
| 2935 | .setup_periodic_interrupt = dcn10_setup_periodic_interrupt, | ||
| 2936 | .setup_vupdate_interrupt = dcn10_setup_vupdate_interrupt | ||
| 2760 | }; | 2937 | }; |
| 2761 | 2938 | ||
| 2762 | 2939 | ||
| @@ -81,4 +81,6 @@ struct pipe_ctx *find_top_pipe_for_stream( · 80 unmodified lines | |||
| 81 | struct dc_state *context, | 81 | struct dc_state *context, |
| 82 | const struct dc_stream_state *stream); | 82 | const struct dc_stream_state *stream); |
| 83 | 83 | ||
| 84 | int get_vupdate_offset_from_vsync(struct pipe_ctx *pipe_ctx); | ||
| 85 | |||
| 84 | #endif /* __DC_HWSS_DCN10_H__ */ | 86 | #endif /* __DC_HWSS_DCN10_H__ */ |
| @@ -92,134 +92,36 @@ static void optc1_disable_stereo(struct timing_generator *optc) · 91 unmodified lines | |||
| 92 | OTG_3D_STRUCTURE_STEREO_SEL_OVR, 0); | 92 | OTG_3D_STRUCTURE_STEREO_SEL_OVR, 0); |
| 93 | } | 93 | } |
| 94 | 94 | ||
| 95 | static uint32_t get_start_vline(struct timing_generator *optc, const struct dc_crtc_timing *dc_crtc_timing) | 95 | void optc1_setup_vertical_interrupt0( |
| 96 | { | ||
| 97 | struct dc_crtc_timing patched_crtc_timing; | ||
| 98 | int vesa_sync_start; | ||
| 99 | int asic_blank_end; | ||
| 100 | int interlace_factor; | ||
| 101 | int vertical_line_start; | ||
| 102 | |||
| 103 | patched_crtc_timing = *dc_crtc_timing; | ||
| 104 | optc1_apply_front_porch_workaround(optc, &patched_crtc_timing); | ||
| 105 | |||
| 106 | vesa_sync_start = patched_crtc_timing.h_addressable + | ||
| 107 | patched_crtc_timing.h_border_right + | ||
| 108 | patched_crtc_timing.h_front_porch; | ||
| 109 | |||
| 110 | asic_blank_end = patched_crtc_timing.h_total - | ||
| 111 | vesa_sync_start - | ||
| 112 | patched_crtc_timing.h_border_left; | ||
| 113 | |||
| 114 | interlace_factor = patched_crtc_timing.flags.INTERLACE ? 2 : 1; | ||
| 115 | |||
| 116 | vesa_sync_start = patched_crtc_timing.v_addressable + | ||
| 117 | patched_crtc_timing.v_border_bottom + | ||
| 118 | patched_crtc_timing.v_front_porch; | ||
| 119 | |||
| 120 | asic_blank_end = (patched_crtc_timing.v_total - | ||
| 121 | vesa_sync_start - | ||
| 122 | patched_crtc_timing.v_border_top) | ||
| 123 | * interlace_factor; | ||
| 124 | |||
| 125 | vertical_line_start = asic_blank_end - optc->dlg_otg_param.vstartup_start + 1; | ||
| 126 | if (vertical_line_start < 0) { | ||
| 127 | ASSERT(0); | ||
| 128 | vertical_line_start = 0; | ||
| 129 | } | ||
| 130 | |||
| 131 | return vertical_line_start; | ||
| 132 | } | ||
| 133 | |||
| 134 | static void calc_vline_position( | ||
| 135 | struct timing_generator *optc, | 96 | struct timing_generator *optc, |
| 136 | const struct dc_crtc_timing *dc_crtc_timing, | 97 | uint32_t start_line, |
| 137 | unsigned long long vsync_delta, | 98 | uint32_t end_line) |
| 138 | uint32_t *start_line, | ||
| 139 | uint32_t *end_line) | ||
| 140 | { | 99 | { |
| 141 | unsigned long long req_delta_tens_of_usec = div64_u64((vsync_delta + 9999), 10000); | 100 | struct optc *optc1 = DCN10TG_FROM_TG(optc); |
| 142 | unsigned long long pix_clk_hundreds_khz = div64_u64((dc_crtc_timing->pix_clk_100hz + 999), 1000); | ||
| 143 | uint32_t req_delta_lines = (uint32_t) div64_u64( | ||
| 144 | (req_delta_tens_of_usec * pix_clk_hundreds_khz + dc_crtc_timing->h_total - 1), | ||
| 145 | dc_crtc_timing->h_total); | ||
| 146 | |||
| 147 | uint32_t vsync_line = get_start_vline(optc, dc_crtc_timing); | ||
| 148 | |||
| 149 | if (req_delta_lines != 0) | ||
| 150 | req_delta_lines--; | ||
| 151 | |||
| 152 | if (req_delta_lines > vsync_line) | ||
| 153 | *start_line = dc_crtc_timing->v_total - (req_delta_lines - vsync_line) - 1; | ||
| 154 | else | ||
| 155 | *start_line = vsync_line - req_delta_lines; | ||
| 156 | |||
| 157 | *end_line = *start_line + 2; | ||
| 158 | 101 | ||
| 159 | if (*end_line >= dc_crtc_timing->v_total) | 102 | REG_SET_2(OTG_VERTICAL_INTERRUPT0_POSITION, 0, |
| 160 | *end_line = 2; | 103 | OTG_VERTICAL_INTERRUPT0_LINE_START, start_line, |
| 104 | OTG_VERTICAL_INTERRUPT0_LINE_END, end_line); | ||
| 161 | } | 105 | } |
| 162 | 106 | ||
| 163 | void optc1_program_vline_interrupt( | 107 | void optc1_setup_vertical_interrupt1( |
| 164 | struct timing_generator *optc, | 108 | struct timing_generator *optc, |
| 165 | const struct dc_crtc_timing *dc_crtc_timing, | 109 | uint32_t start_line) |
| 166 | enum vline_select vline, | ||
| 167 | const union vline_config *vline_config) | ||
| 168 | { | 110 | { |
| 169 | struct optc *optc1 = DCN10TG_FROM_TG(optc); | 111 | struct optc *optc1 = DCN10TG_FROM_TG(optc); |
| 170 | uint32_t start_line = 0; | 112 | |
| 171 | uint32_t end_line = 0; | 113 | REG_SET(OTG_VERTICAL_INTERRUPT1_POSITION, 0, |
| 172 | 114 | OTG_VERTICAL_INTERRUPT1_LINE_START, start_line); | |
| 173 | switch (vline) { | ||
| 174 | case VLINE0: | ||
| 175 | calc_vline_position(optc, dc_crtc_timing, vline_config->delta_in_ns, &start_line, &end_line); | ||
| 176 | REG_SET_2(OTG_VERTICAL_INTERRUPT0_POSITION, 0, | ||
| 177 | OTG_VERTICAL_INTERRUPT0_LINE_START, start_line, | ||
| 178 | OTG_VERTICAL_INTERRUPT0_LINE_END, end_line); | ||
| 179 | break; | ||
| 180 | case VLINE1: | ||
| 181 | REG_SET(OTG_VERTICAL_INTERRUPT1_POSITION, 0, | ||
| 182 | OTG_VERTICAL_INTERRUPT1_LINE_START, vline_config->line_number); | ||
| 183 | break; | ||
| 184 | default: | ||
| 185 | break; | ||
| 186 | } | ||
| 187 | } | 115 | } |
| 188 | 116 | ||
| 189 | void optc1_program_vupdate_interrupt( | 117 | void optc1_setup_vertical_interrupt2( |
| 190 | struct timing_generator *optc, | 118 | struct timing_generator *optc, |
| 191 | const struct dc_crtc_timing *dc_crtc_timing) | 119 | uint32_t start_line) |
| 192 | { | 120 | { |
| 193 | struct optc *optc1 = DCN10TG_FROM_TG(optc); | 121 | struct optc *optc1 = DCN10TG_FROM_TG(optc); |
| 194 | int32_t vertical_line_start; | ||
| 195 | uint32_t asic_blank_end; | ||
| 196 | uint32_t vesa_sync_start; | ||
| 197 | struct dc_crtc_timing patched_crtc_timing; | ||
| 198 | |||
| 199 | patched_crtc_timing = *dc_crtc_timing; | ||
| 200 | optc1_apply_front_porch_workaround(optc, &patched_crtc_timing); | ||
| 201 | |||
| 202 | /* asic_h_blank_end = HsyncWidth + HbackPorch = | ||
| 203 | * vesa. usHorizontalTotal - vesa. usHorizontalSyncStart - | ||
| 204 | * vesa.h_left_border | ||
| 205 | */ | ||
| 206 | vesa_sync_start = patched_crtc_timing.h_addressable + | ||
| 207 | patched_crtc_timing.h_border_right + | ||
| 208 | patched_crtc_timing.h_front_porch; | ||
| 209 | |||
| 210 | asic_blank_end = patched_crtc_timing.h_total - | ||
| 211 | vesa_sync_start - | ||
| 212 | patched_crtc_timing.h_border_left; | ||
| 213 | |||
| 214 | /* Use OTG_VERTICAL_INTERRUPT2 replace VUPDATE interrupt, | ||
| 215 | * program the reg for interrupt postition. | ||
| 216 | */ | ||
| 217 | vertical_line_start = asic_blank_end - optc->dlg_otg_param.vstartup_start + 1; | ||
| 218 | if (vertical_line_start < 0) | ||
| 219 | vertical_line_start = 0; | ||
| 220 | 122 | ||
| 221 | REG_SET(OTG_VERTICAL_INTERRUPT2_POSITION, 0, | 123 | REG_SET(OTG_VERTICAL_INTERRUPT2_POSITION, 0, |
| 222 | OTG_VERTICAL_INTERRUPT2_LINE_START, vertical_line_start); | 124 | OTG_VERTICAL_INTERRUPT2_LINE_START, start_line); |
| 223 | } | 125 | } |
| 224 | 126 | ||
| 225 | /** | 127 | /** |
| @@ -1480,8 +1382,9 @@ bool optc1_get_crc(struct timing_generator *optc, · 1254 unmodified lines | |||
| 1480 | static const struct timing_generator_funcs dcn10_tg_funcs = { | 1382 | static const struct timing_generator_funcs dcn10_tg_funcs = { |
| 1481 | .validate_timing = optc1_validate_timing, | 1383 | .validate_timing = optc1_validate_timing, |
| 1482 | .program_timing = optc1_program_timing, | 1384 | .program_timing = optc1_program_timing, |
| 1483 | .program_vline_interrupt = optc1_program_vline_interrupt, | 1385 | .setup_vertical_interrupt0 = optc1_setup_vertical_interrupt0, |
| 1484 | .program_vupdate_interrupt = optc1_program_vupdate_interrupt, | 1386 | .setup_vertical_interrupt1 = optc1_setup_vertical_interrupt1, |
| 1387 | .setup_vertical_interrupt2 = optc1_setup_vertical_interrupt2, | ||
| 1485 | .program_global_sync = optc1_program_global_sync, | 1388 | .program_global_sync = optc1_program_global_sync, |
| 1486 | .enable_crtc = optc1_enable_crtc, | 1389 | .enable_crtc = optc1_enable_crtc, |
| 1487 | .disable_crtc = optc1_disable_crtc, | 1390 | .disable_crtc = optc1_disable_crtc, |
| @@ -483,11 +483,16 @@ void optc1_program_timing( · 482 unmodified lines | |||
| 483 | const struct dc_crtc_timing *dc_crtc_timing, | 483 | const struct dc_crtc_timing *dc_crtc_timing, |
| 484 | bool use_vbios); | 484 | bool use_vbios); |
| 485 | 485 | ||
| 486 | void optc1_program_vline_interrupt( | 486 | void optc1_setup_vertical_interrupt0( |
| 487 | struct timing_generator *optc, | 487 | struct timing_generator *optc, |
| 488 | const struct dc_crtc_timing *dc_crtc_timing, | 488 | uint32_t start_line, |
| 489 | enum vline_select vline, | 489 | uint32_t end_line); |
| 490 | const union vline_config *vline_config); | 490 | void optc1_setup_vertical_interrupt1( |
| 491 | struct timing_generator *optc, | ||
| 492 | uint32_t start_line); | ||
| 493 | void optc1_setup_vertical_interrupt2( | ||
| 494 | struct timing_generator *optc, | ||
| 495 | uint32_t start_line); | ||
| 491 | 496 | ||
| 492 | void optc1_program_global_sync( | 497 | void optc1_program_global_sync( |
| 493 | struct timing_generator *optc); | 498 | struct timing_generator *optc); |
| @@ -46,6 +46,7 @@ struct abm_funcs { · 45 unmodified lines | |||
| 46 | void (*abm_init)(struct abm *abm); | 46 | void (*abm_init)(struct abm *abm); |
| 47 | bool (*set_abm_level)(struct abm *abm, unsigned int abm_level); | 47 | bool (*set_abm_level)(struct abm *abm, unsigned int abm_level); |
| 48 | bool (*set_abm_immediate_disable)(struct abm *abm); | 48 | bool (*set_abm_immediate_disable)(struct abm *abm); |
| 49 | bool (*set_pipe)(struct abm *abm, unsigned int controller_id); | ||
| 49 | bool (*init_backlight)(struct abm *abm); | 50 | bool (*init_backlight)(struct abm *abm); |
| 50 | 51 | ||
| 51 | /* backlight_pwm_u16_16 is unsigned 32 bit, | 52 | /* backlight_pwm_u16_16 is unsigned 32 bit, |
| @@ -134,14 +134,6 @@ struct dc_crtc_timing; · 133 unmodified lines | |||
| 134 | 134 | ||
| 135 | struct drr_params; | 135 | struct drr_params; |
| 136 | 136 | ||
| 137 | union vline_config; | ||
| 138 | |||
| 139 | |||
| 140 | enum vline_select { | ||
| 141 | VLINE0, | ||
| 142 | VLINE1, | ||
| 143 | VLINE2 | ||
| 144 | }; | ||
| 145 | 137 | ||
| 146 | struct timing_generator_funcs { | 138 | struct timing_generator_funcs { |
| 147 | bool (*validate_timing)(struct timing_generator *tg, | 139 | bool (*validate_timing)(struct timing_generator *tg, |
| @@ -149,14 +141,17 @@ struct timing_generator_funcs { · 1 unmodified line | |||
| 149 | void (*program_timing)(struct timing_generator *tg, | 141 | void (*program_timing)(struct timing_generator *tg, |
| 150 | const struct dc_crtc_timing *timing, | 142 | const struct dc_crtc_timing *timing, |
| 151 | bool use_vbios); | 143 | bool use_vbios); |
| 152 | void (*program_vline_interrupt)( | 144 | void (*setup_vertical_interrupt0)( |
| 145 | struct timing_generator *optc, | ||
| 146 | uint32_t start_line, | ||
| 147 | uint32_t end_line); | ||
| 148 | void (*setup_vertical_interrupt1)( | ||
| 149 | struct timing_generator *optc, | ||
| 150 | uint32_t start_line); | ||
| 151 | void (*setup_vertical_interrupt2)( | ||
| 153 | struct timing_generator *optc, | 152 | struct timing_generator *optc, |
| 154 | const struct dc_crtc_timing *dc_crtc_timing, | 153 | uint32_t start_line); |
| 155 | enum vline_select vline, | ||
| 156 | const union vline_config *vline_config); | ||
| 157 | 154 | ||
| 158 | void (*program_vupdate_interrupt)(struct timing_generator *optc, | ||
| 159 | const struct dc_crtc_timing *dc_crtc_timing); | ||
| 160 | bool (*enable_crtc)(struct timing_generator *tg); | 155 | bool (*enable_crtc)(struct timing_generator *tg); |
| 161 | bool (*disable_crtc)(struct timing_generator *tg); | 156 | bool (*disable_crtc)(struct timing_generator *tg); |
| 162 | bool (*is_counter_moving)(struct timing_generator *tg); | 157 | bool (*is_counter_moving)(struct timing_generator *tg); |
| @@ -38,6 +38,11 @@ enum pipe_gating_control { · 37 unmodified lines | |||
| 38 | PIPE_GATING_CONTROL_INIT | 38 | PIPE_GATING_CONTROL_INIT |
| 39 | }; | 39 | }; |
| 40 | 40 | ||
| 41 | enum vline_select { | ||
| 42 | VLINE0, | ||
| 43 | VLINE1 | ||
| 44 | }; | ||
| 45 | |||
| 41 | struct dce_hwseq_wa { | 46 | struct dce_hwseq_wa { |
| 42 | bool blnd_crtc_trigger; | 47 | bool blnd_crtc_trigger; |
| 43 | bool DEGVIDCN10_253; | 48 | bool DEGVIDCN10_253; |
| @@ -68,6 +73,10 @@ struct stream_resource; · 24 unmodified lines | |||
| 68 | 73 | ||
| 69 | struct hw_sequencer_funcs { | 74 | struct hw_sequencer_funcs { |
| 70 | 75 | ||
| 76 | void (*disable_stream_gating)(struct dc *dc, struct pipe_ctx *pipe_ctx); | ||
| 77 | |||
| 78 | void (*enable_stream_gating)(struct dc *dc, struct pipe_ctx *pipe_ctx); | ||
| 79 | |||
| 71 | void (*init_hw)(struct dc *dc); | 80 | void (*init_hw)(struct dc *dc); |
| 72 | 81 | ||
| 73 | void (*init_pipes)(struct dc *dc, struct dc_state *context); | 82 | void (*init_pipes)(struct dc *dc, struct dc_state *context); |
| @@ -220,6 +229,9 @@ struct hw_sequencer_funcs { · 146 unmodified lines | |||
| 220 | void (*set_cursor_attribute)(struct pipe_ctx *pipe); | 229 | void (*set_cursor_attribute)(struct pipe_ctx *pipe); |
| 221 | void (*set_cursor_sdr_white_level)(struct pipe_ctx *pipe); | 230 | void (*set_cursor_sdr_white_level)(struct pipe_ctx *pipe); |
| 222 | 231 | ||
| 232 | void (*setup_periodic_interrupt)(struct pipe_ctx *pipe_ctx, enum vline_select vline); | ||
| 233 | void (*setup_vupdate_interrupt)(struct pipe_ctx *pipe_ctx); | ||
| 234 | |||
| 223 | }; | 235 | }; |
| 224 | 236 | ||
| 225 | void color_space_to_black_color( | 237 | void color_space_to_black_color( |
| @@ -131,6 +131,7 @@ · 130 unmodified lines | |||
| 131 | #define INTERNAL_REV_RAVEN_A0 0x00 /* First spin of Raven */ | 131 | #define INTERNAL_REV_RAVEN_A0 0x00 /* First spin of Raven */ |
| 132 | #define RAVEN_A0 0x01 | 132 | #define RAVEN_A0 0x01 |
| 133 | #define RAVEN_B0 0x21 | 133 | #define RAVEN_B0 0x21 |
| 134 | #define PICASSO_A0 0x41 | ||
| 134 | #if defined(CONFIG_DRM_AMD_DC_DCN1_01) | 135 | #if defined(CONFIG_DRM_AMD_DC_DCN1_01) |
| 135 | /* DCN1_01 */ | 136 | /* DCN1_01 */ |
| 136 | #define RAVEN2_A0 0x81 | 137 | #define RAVEN2_A0 0x81 |
| @@ -165,4 +166,6 @@ · 28 unmodified lines | |||
| 165 | 166 | ||
| 166 | #define FAMILY_UNKNOWN 0xFF | 167 | #define FAMILY_UNKNOWN 0xFF |
| 167 | 168 | ||
| 169 | |||
| 170 | |||
| 168 | #endif /* __DAL_ASIC_ID_H__ */ | 171 | #endif /* __DAL_ASIC_ID_H__ */ |
| @@ -165,18 +165,11 @@ struct iram_table_v_2_2 { · 164 unmodified lines | |||
| 165 | }; | 165 | }; |
| 166 | #pragma pack(pop) | 166 | #pragma pack(pop) |
| 167 | 167 | ||
| 168 | static uint16_t backlight_8_to_16(unsigned int backlight_8bit) | ||
| 169 | { | ||
| 170 | return (uint16_t)(backlight_8bit * 0x101); | ||
| 171 | } | ||
| 172 | |||
| 173 | static void fill_backlight_transform_table(struct dmcu_iram_parameters params, | 168 | static void fill_backlight_transform_table(struct dmcu_iram_parameters params, |
| 174 | struct iram_table_v_2 *table) | 169 | struct iram_table_v_2 *table) |
| 175 | { | 170 | { |
| 176 | unsigned int i; | 171 | unsigned int i; |
| 177 | unsigned int num_entries = NUM_BL_CURVE_SEGS; | 172 | unsigned int num_entries = NUM_BL_CURVE_SEGS; |
| 178 | unsigned int query_input_8bit; | ||
| 179 | unsigned int query_output_8bit; | ||
| 180 | unsigned int lut_index; | 173 | unsigned int lut_index; |
| 181 | 174 | ||
| 182 | table->backlight_thresholds[0] = 0; | 175 | table->backlight_thresholds[0] = 0; |
| @@ -194,16 +187,13 @@ static void fill_backlight_transform_table(struct dmcu_iram_parameters params, · 11 unmodified lines | |||
| 194 | * format U4.10. | 187 | * format U4.10. |
| 195 | */ | 188 | */ |
| 196 | for (i = 1; i+1 < num_entries; i++) { | 189 | for (i = 1; i+1 < num_entries; i++) { |
| 197 | query_input_8bit = DIV_ROUNDUP((i * 256), num_entries); | ||
| 198 | |||
| 199 | lut_index = (params.backlight_lut_array_size - 1) * i / (num_entries - 1); | 190 | lut_index = (params.backlight_lut_array_size - 1) * i / (num_entries - 1); |
| 200 | ASSERT(lut_index < params.backlight_lut_array_size); | 191 | ASSERT(lut_index < params.backlight_lut_array_size); |
| 201 | query_output_8bit = params.backlight_lut_array[lut_index] >> 8; | ||
| 202 | 192 | ||
| 203 | table->backlight_thresholds[i] = | 193 | table->backlight_thresholds[i] = |
| 204 | backlight_8_to_16(query_input_8bit); | 194 | cpu_to_be16(DIV_ROUNDUP((i * 65536), num_entries)); |
| 205 | table->backlight_offsets[i] = | 195 | table->backlight_offsets[i] = |
| 206 | backlight_8_to_16(query_output_8bit); | 196 | cpu_to_be16(params.backlight_lut_array[lut_index]); |
| 207 | } | 197 | } |
| 208 | } | 198 | } |
| 209 | 199 | ||
| @@ -212,8 +202,6 @@ static void fill_backlight_transform_table_v_2_2(struct dmcu_iram_parameters par · 2 unmodified lines | |||
| 212 | { | 202 | { |
| 213 | unsigned int i; | 203 | unsigned int i; |
| 214 | unsigned int num_entries = NUM_BL_CURVE_SEGS; | 204 | unsigned int num_entries = NUM_BL_CURVE_SEGS; |
| 215 | unsigned int query_input_8bit; | ||
| 216 | unsigned int query_output_8bit; | ||
| 217 | unsigned int lut_index; | 205 | unsigned int lut_index; |
| 218 | 206 | ||
| 219 | table->backlight_thresholds[0] = 0; | 207 | table->backlight_thresholds[0] = 0; |
| @@ -231,16 +219,13 @@ static void fill_backlight_transform_table_v_2_2(struct dmcu_iram_parameters par · 11 unmodified lines | |||
| 231 | * format U4.10. | 219 | * format U4.10. |
| 232 | */ | 220 | */ |
| 233 | for (i = 1; i+1 < num_entries; i++) { | 221 | for (i = 1; i+1 < num_entries; i++) { |
| 234 | query_input_8bit = DIV_ROUNDUP((i * 256), num_entries); | ||
| 235 | |||
| 236 | lut_index = (params.backlight_lut_array_size - 1) * i / (num_entries - 1); | 222 | lut_index = (params.backlight_lut_array_size - 1) * i / (num_entries - 1); |
| 237 | ASSERT(lut_index < params.backlight_lut_array_size); | 223 | ASSERT(lut_index < params.backlight_lut_array_size); |
| 238 | query_output_8bit = params.backlight_lut_array[lut_index] >> 8; | ||
| 239 | 224 | ||
| 240 | table->backlight_thresholds[i] = | 225 | table->backlight_thresholds[i] = |
| 241 | backlight_8_to_16(query_input_8bit); | 226 | cpu_to_be16(DIV_ROUNDUP((i * 65536), num_entries)); |
| 242 | table->backlight_offsets[i] = | 227 | table->backlight_offsets[i] = |
| 243 | backlight_8_to_16(query_output_8bit); | 228 | cpu_to_be16(params.backlight_lut_array[lut_index]); |
| 244 | } | 229 | } |
| 245 | } | 230 | } |
| 246 | 231 | ||
| @@ -137,20 +137,17 @@ struct kgd2kfd_shared_resources { · 136 unmodified lines | |||
| 137 | /* Bit n == 1 means Queue n is available for KFD */ | 137 | /* Bit n == 1 means Queue n is available for KFD */ |
| 138 | DECLARE_BITMAP(queue_bitmap, KGD_MAX_QUEUES); | 138 | DECLARE_BITMAP(queue_bitmap, KGD_MAX_QUEUES); |
| 139 | 139 | ||
| 140 | /* Doorbell assignments (SOC15 and later chips only). Only | 140 | /* SDMA doorbell assignments (SOC15 and later chips only). Only |
| 141 | * specific doorbells are routed to each SDMA engine. Others | 141 | * specific doorbells are routed to each SDMA engine. Others |
| 142 | * are routed to IH and VCN. They are not usable by the CP. | 142 | * are routed to IH and VCN. They are not usable by the CP. |
| 143 | * | ||
| 144 | * Any doorbell number D that satisfies the following condition | ||
| 145 | * is reserved: (D & reserved_doorbell_mask) == reserved_doorbell_val | ||
| 146 | * | ||
| 147 | * KFD currently uses 1024 (= 0x3ff) doorbells per process. If | ||
| 148 | * doorbells 0x0e0-0x0ff and 0x2e0-0x2ff are reserved, that means | ||
| 149 | * mask would be set to 0x1e0 and val set to 0x0e0. | ||
| 150 | */ | 143 | */ |
| 151 | unsigned int sdma_doorbell[2][8]; | 144 | uint32_t *sdma_doorbell_idx; |
| 152 | unsigned int reserved_doorbell_mask; | 145 | |
| 153 | unsigned int reserved_doorbell_val; | 146 | /* From SOC15 onward, the doorbell index range not usable for CP |
| 147 | * queues. | ||
| 148 | */ | ||
| 149 | uint32_t non_cp_doorbells_start; | ||
| 150 | uint32_t non_cp_doorbells_end; | ||
| 154 | 151 | ||
| 155 | /* Base address of doorbell aperture. */ | 152 | /* Base address of doorbell aperture. */ |
| 156 | phys_addr_t doorbell_physical_address; | 153 | phys_addr_t doorbell_physical_address; |
| @@ -139,12 +139,10 @@ static int smu10_construct_max_power_limits_table(struct pp_hwmgr *hwmgr, · 138 unmodified lines | |||
| 139 | static int smu10_init_dynamic_state_adjustment_rule_settings( | 139 | static int smu10_init_dynamic_state_adjustment_rule_settings( |
| 140 | struct pp_hwmgr *hwmgr) | 140 | struct pp_hwmgr *hwmgr) |
| 141 | { | 141 | { |
| 142 | uint32_t table_size = | 142 | struct phm_clock_voltage_dependency_table *table_clk_vlt; |
| 143 | sizeof(struct phm_clock_voltage_dependency_table) + | ||
| 144 | (7 * sizeof(struct phm_clock_voltage_dependency_record)); | ||
| 145 | 143 | ||
| 146 | struct phm_clock_voltage_dependency_table *table_clk_vlt = | 144 | table_clk_vlt = kzalloc(struct_size(table_clk_vlt, entries, 7), |
| 147 | kzalloc(table_size, GFP_KERNEL); | 145 | GFP_KERNEL); |
| 148 | 146 | ||
| 149 | if (NULL == table_clk_vlt) { | 147 | if (NULL == table_clk_vlt) { |
| 150 | pr_err("Can not allocate memory!\n"); | 148 | pr_err("Can not allocate memory!\n"); |
| @@ -3681,10 +3681,12 @@ static int smu7_request_link_speed_change_before_state_change( · 3680 unmodified lines | |||
| 3681 | data->force_pcie_gen = PP_PCIEGen2; | 3681 | data->force_pcie_gen = PP_PCIEGen2; |
| 3682 | if (current_link_speed == PP_PCIEGen2) | 3682 | if (current_link_speed == PP_PCIEGen2) |
| 3683 | break; | 3683 | break; |
| 3684 | /* fall through */ | ||
| 3684 | case PP_PCIEGen2: | 3685 | case PP_PCIEGen2: |
| 3685 | if (0 == amdgpu_acpi_pcie_performance_request(hwmgr->adev, PCIE_PERF_REQ_GEN2, false)) | 3686 | if (0 == amdgpu_acpi_pcie_performance_request(hwmgr->adev, PCIE_PERF_REQ_GEN2, false)) |
| 3686 | break; | 3687 | break; |
| 3687 | #endif | 3688 | #endif |
| 3689 | /* fall through */ | ||
| 3688 | default: | 3690 | default: |
| 3689 | data->force_pcie_gen = smu7_get_current_pcie_speed(hwmgr); | 3691 | data->force_pcie_gen = smu7_get_current_pcie_speed(hwmgr); |
| 3690 | break; | 3692 | break; |
| @@ -1211,7 +1211,7 @@ int smu7_power_control_set_level(struct pp_hwmgr *hwmgr) · 1210 unmodified lines | |||
| 1211 | hwmgr->platform_descriptor.TDPAdjustment : | 1211 | hwmgr->platform_descriptor.TDPAdjustment : |
| 1212 | (-1 * hwmgr->platform_descriptor.TDPAdjustment); | 1212 | (-1 * hwmgr->platform_descriptor.TDPAdjustment); |
| 1213 | 1213 | ||
| 1214 | if (hwmgr->chip_id > CHIP_TONGA) | 1214 | if (hwmgr->chip_id > CHIP_TONGA) |
| 1215 | target_tdp = ((100 + adjust_percent) * (int)(cac_table->usTDP * 256)) / 100; | 1215 | target_tdp = ((100 + adjust_percent) * (int)(cac_table->usTDP * 256)) / 100; |
| 1216 | else | 1216 | else |
| 1217 | target_tdp = ((100 + adjust_percent) * (int)(cac_table->usConfigurableTDP * 256)) / 100; | 1217 | target_tdp = ((100 + adjust_percent) * (int)(cac_table->usConfigurableTDP * 256)) / 100; |
| @@ -272,12 +272,10 @@ static int smu8_init_dynamic_state_adjustment_rule_settings( · 271 unmodified lines | |||
| 272 | struct pp_hwmgr *hwmgr, | 272 | struct pp_hwmgr *hwmgr, |
| 273 | ATOM_CLK_VOLT_CAPABILITY *disp_voltage_table) | 273 | ATOM_CLK_VOLT_CAPABILITY *disp_voltage_table) |
| 274 | { | 274 | { |
| 275 | uint32_t table_size = | 275 | struct phm_clock_voltage_dependency_table *table_clk_vlt; |
| 276 | sizeof(struct phm_clock_voltage_dependency_table) + | ||
| 277 | (7 * sizeof(struct phm_clock_voltage_dependency_record)); | ||
| 278 | 276 | ||
| 279 | struct phm_clock_voltage_dependency_table *table_clk_vlt = | 277 | table_clk_vlt = kzalloc(struct_size(table_clk_vlt, entries, 7), |
| 280 | kzalloc(table_size, GFP_KERNEL); | 278 | GFP_KERNEL); |
| 281 | 279 | ||
| 282 | if (NULL == table_clk_vlt) { | 280 | if (NULL == table_clk_vlt) { |
| 283 | pr_err("Can not allocate memory!\n"); | 281 | pr_err("Can not allocate memory!\n"); |
| @@ -1,3 +1,25 @@ | |||
| 1 | /* | ||
| 2 | * Copyright 2018 Advanced Micro Devices, Inc. | ||
| 3 | * | ||
| 4 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 5 | * copy of this software and associated documentation files (the "Software"), | ||
| 6 | * to deal in the Software without restriction, including without limitation | ||
| 7 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 8 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 9 | * Software is furnished to do so, subject to the following conditions: | ||
| 10 | * | ||
| 11 | * The above copyright notice and this permission notice shall be included in | ||
| 12 | * all copies or substantial portions of the Software. | ||
| 13 | * | ||
| 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 17 | * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | ||
| 18 | * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | ||
| 19 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | ||
| 20 | * OTHER DEALINGS IN THE SOFTWARE. | ||
| 21 | * | ||
| 22 | */ | ||
| 1 | #include "amdgpu.h" | 23 | #include "amdgpu.h" |
| 2 | #include "soc15.h" | 24 | #include "soc15.h" |
| 3 | #include "soc15_hw_ip.h" | 25 | #include "soc15_hw_ip.h" |
| @@ -114,7 +136,7 @@ int vega10_baco_set_state(struct pp_hwmgr *hwmgr, enum BACO_STATE state) · 110 unmodified lines | |||
| 114 | if (soc15_baco_program_registers(hwmgr, pre_baco_tbl, | 136 | if (soc15_baco_program_registers(hwmgr, pre_baco_tbl, |
| 115 | ARRAY_SIZE(pre_baco_tbl))) { | 137 | ARRAY_SIZE(pre_baco_tbl))) { |
| 116 | if (smum_send_msg_to_smc(hwmgr, PPSMC_MSG_EnterBaco)) | 138 | if (smum_send_msg_to_smc(hwmgr, PPSMC_MSG_EnterBaco)) |
| 117 | return -1; | 139 | return -EINVAL; |
| 118 | 140 | ||
| 119 | if (soc15_baco_program_registers(hwmgr, enter_baco_tbl, | 141 | if (soc15_baco_program_registers(hwmgr, enter_baco_tbl, |
| 120 | ARRAY_SIZE(enter_baco_tbl))) | 142 | ARRAY_SIZE(enter_baco_tbl))) |
| @@ -132,5 +154,5 @@ int vega10_baco_set_state(struct pp_hwmgr *hwmgr, enum BACO_STATE state) · 11 unmodified lines | |||
| 132 | } | 154 | } |
| 133 | } | 155 | } |
| 134 | 156 | ||
| 135 | return -1; | 157 | return -EINVAL; |
| 136 | } | 158 | } |
| @@ -20,8 +20,8 @@ · 19 unmodified lines | |||
| 20 | * OTHER DEALINGS IN THE SOFTWARE. | 20 | * OTHER DEALINGS IN THE SOFTWARE. |
| 21 | * | 21 | * |
| 22 | */ | 22 | */ |
| 23 | #ifndef __VEGA10_BOCO_H__ | 23 | #ifndef __VEGA10_BACO_H__ |
| 24 | #define __VEGA10_BOCO_H__ | 24 | #define __VEGA10_BACO_H__ |
| 25 | #include "hwmgr.h" | 25 | #include "hwmgr.h" |
| 26 | #include "common_baco.h" | 26 | #include "common_baco.h" |
| 27 | 27 | ||
| @@ -1,3 +1,25 @@ | |||
| 1 | /* | ||
| 2 | * Copyright 2018 Advanced Micro Devices, Inc. | ||
| 3 | * | ||
| 4 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 5 | * copy of this software and associated documentation files (the "Software"), | ||
| 6 | * to deal in the Software without restriction, including without limitation | ||
| 7 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 8 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 9 | * Software is furnished to do so, subject to the following conditions: | ||
| 10 | * | ||
| 11 | * The above copyright notice and this permission notice shall be included in | ||
| 12 | * all copies or substantial portions of the Software. | ||
| 13 | * | ||
| 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 17 | * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | ||
| 18 | * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | ||
| 19 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | ||
| 20 | * OTHER DEALINGS IN THE SOFTWARE. | ||
| 21 | * | ||
| 22 | */ | ||
| 1 | #include "amdgpu.h" | 23 | #include "amdgpu.h" |
| 2 | #include "soc15.h" | 24 | #include "soc15.h" |
| 3 | #include "soc15_hw_ip.h" | 25 | #include "soc15_hw_ip.h" |
| @@ -67,14 +89,14 @@ int vega20_baco_set_state(struct pp_hwmgr *hwmgr, enum BACO_STATE state) · 63 unmodified lines | |||
| 67 | 89 | ||
| 68 | 90 | ||
| 69 | if(smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_EnterBaco, 0)) | 91 | if(smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_EnterBaco, 0)) |
| 70 | return -1; | 92 | return -EINVAL; |
| 71 | 93 | ||
| 72 | } else if (state == BACO_STATE_OUT) { | 94 | } else if (state == BACO_STATE_OUT) { |
| 73 | if (smum_send_msg_to_smc(hwmgr, PPSMC_MSG_ExitBaco)) | 95 | if (smum_send_msg_to_smc(hwmgr, PPSMC_MSG_ExitBaco)) |
| 74 | return -1; | 96 | return -EINVAL; |
| 75 | if (!soc15_baco_program_registers(hwmgr, clean_baco_tbl, | 97 | if (!soc15_baco_program_registers(hwmgr, clean_baco_tbl, |
| 76 | ARRAY_SIZE(clean_baco_tbl))) | 98 | ARRAY_SIZE(clean_baco_tbl))) |
| 77 | return -1; | 99 | return -EINVAL; |
| 78 | } | 100 | } |
| 79 | 101 | ||
| 80 | return 0; | 102 | return 0; |
| @@ -20,8 +20,8 @@ · 19 unmodified lines | |||
| 20 | * OTHER DEALINGS IN THE SOFTWARE. | 20 | * OTHER DEALINGS IN THE SOFTWARE. |
| 21 | * | 21 | * |
| 22 | */ | 22 | */ |
| 23 | #ifndef __VEGA20_BOCO_H__ | 23 | #ifndef __VEGA20_BACO_H__ |
| 24 | #define __VEGA20_BOCO_H__ | 24 | #define __VEGA20_BACO_H__ |
| 25 | #include "hwmgr.h" | 25 | #include "hwmgr.h" |
| 26 | #include "common_baco.h" | 26 | #include "common_baco.h" |
| 27 | 27 | ||
| @@ -3456,7 +3456,7 @@ static int vega20_apply_clocks_adjust_rules(struct pp_hwmgr *hwmgr) · 3455 unmodified lines | |||
| 3456 | disable_mclk_switching = ((1 < hwmgr->display_config->num_display) && | 3456 | disable_mclk_switching = ((1 < hwmgr->display_config->num_display) && |
| 3457 | !hwmgr->display_config->multi_monitor_in_sync) || | 3457 | !hwmgr->display_config->multi_monitor_in_sync) || |
| 3458 | vblank_too_short; | 3458 | vblank_too_short; |
| 3459 | latency = hwmgr->display_config->dce_tolerable_mclk_in_active_latency; | 3459 | latency = hwmgr->display_config->dce_tolerable_mclk_in_active_latency; |
| 3460 | 3460 | ||
| 3461 | /* gfxclk */ | 3461 | /* gfxclk */ |
| 3462 | dpm_table = &(data->dpm_table.gfx_table); | 3462 | dpm_table = &(data->dpm_table.gfx_table); |
| @@ -29,6 +29,10 @@ · 28 unmodified lines | |||
| 29 | #include <drm/amdgpu_drm.h> | 29 | #include <drm/amdgpu_drm.h> |
| 30 | #include "smumgr.h" | 30 | #include "smumgr.h" |
| 31 | 31 | ||
| 32 | MODULE_FIRMWARE("amdgpu/bonaire_smc.bin"); | ||
| 33 | MODULE_FIRMWARE("amdgpu/bonaire_k_smc.bin"); | ||
| 34 | MODULE_FIRMWARE("amdgpu/hawaii_smc.bin"); | ||
| 35 | MODULE_FIRMWARE("amdgpu/hawaii_k_smc.bin"); | ||
| 32 | MODULE_FIRMWARE("amdgpu/topaz_smc.bin"); | 36 | MODULE_FIRMWARE("amdgpu/topaz_smc.bin"); |
| 33 | MODULE_FIRMWARE("amdgpu/topaz_k_smc.bin"); | 37 | MODULE_FIRMWARE("amdgpu/topaz_k_smc.bin"); |
| 34 | MODULE_FIRMWARE("amdgpu/tonga_smc.bin"); | 38 | MODULE_FIRMWARE("amdgpu/tonga_smc.bin"); |
| @@ -145,6 +145,10 @@ static int bochs_pci_probe(struct pci_dev *pdev, · 144 unmodified lines | |||
| 145 | if (IS_ERR(dev)) | 145 | if (IS_ERR(dev)) |
| 146 | return PTR_ERR(dev); | 146 | return PTR_ERR(dev); |
| 147 | 147 | ||
| 148 | ret = pci_enable_device(pdev); | ||
| 149 | if (ret) | ||
| 150 | goto err_free_dev; | ||
| 151 | |||
| 148 | dev->pdev = pdev; | 152 | dev->pdev = pdev; |
| 149 | pci_set_drvdata(pdev, dev); | 153 | pci_set_drvdata(pdev, dev); |
| 150 | 154 | ||
| @@ -1608,6 +1608,15 @@ int drm_atomic_helper_async_check(struct drm_device *dev, · 1607 unmodified lines | |||
| 1608 | old_plane_state->crtc != new_plane_state->crtc) | 1608 | old_plane_state->crtc != new_plane_state->crtc) |
| 1609 | return -EINVAL; | 1609 | return -EINVAL; |
| 1610 | 1610 | ||
| 1611 | /* | ||
| 1612 | * FIXME: Since prepare_fb and cleanup_fb are always called on | ||
| 1613 | * the new_plane_state for async updates we need to block framebuffer | ||
| 1614 | * changes. This prevents use of a fb that's been cleaned up and | ||
| 1615 | * double cleanups from occuring. | ||
| 1616 | */ | ||
| 1617 | if (old_plane_state->fb != new_plane_state->fb) | ||
| 1618 | return -EINVAL; | ||
| 1619 | |||
| 1611 | funcs = plane->helper_private; | 1620 | funcs = plane->helper_private; |
| 1612 | if (!funcs->atomic_async_update) | 1621 | if (!funcs->atomic_async_update) |
| 1613 | return -EINVAL; | 1622 | return -EINVAL; |
| @@ -262,6 +262,18 @@ void drm_file_free(struct drm_file *file) · 261 unmodified lines | |||
| 262 | kfree(file); | 262 | kfree(file); |
| 263 | } | 263 | } |
| 264 | 264 | ||
| 265 | static void drm_close_helper(struct file *filp) | ||
| 266 | { | ||
| 267 | struct drm_file *file_priv = filp->private_data; | ||
| 268 | struct drm_device *dev = file_priv->minor->dev; | ||
| 269 | |||
| 270 | mutex_lock(&dev->filelist_mutex); | ||
| 271 | list_del(&file_priv->lhead); | ||
| 272 | mutex_unlock(&dev->filelist_mutex); | ||
| 273 | |||
| 274 | drm_file_free(file_priv); | ||
| 275 | } | ||
| 276 | |||
| 265 | static int drm_setup(struct drm_device * dev) | 277 | static int drm_setup(struct drm_device * dev) |
| 266 | { | 278 | { |
| 267 | int ret; | 279 | int ret; |
| @@ -318,8 +330,10 @@ int drm_open(struct inode *inode, struct file *filp) · 50 unmodified lines | |||
| 318 | goto err_undo; | 330 | goto err_undo; |
| 319 | if (need_setup) { | 331 | if (need_setup) { |
| 320 | retcode = drm_setup(dev); | 332 | retcode = drm_setup(dev); |
| 321 | if (retcode) | 333 | if (retcode) { |
| 334 | drm_close_helper(filp); | ||
| 322 | goto err_undo; | 335 | goto err_undo; |
| 336 | } | ||
| 323 | } | 337 | } |
| 324 | return 0; | 338 | return 0; |
| 325 | 339 | ||
| @@ -473,11 +487,7 @@ int drm_release(struct inode *inode, struct file *filp) · 147 unmodified lines | |||
| 473 | 487 | ||
| 474 | DRM_DEBUG("open_count = %d\n", dev->open_count); | 488 | DRM_DEBUG("open_count = %d\n", dev->open_count); |
| 475 | 489 | ||
| 476 | mutex_lock(&dev->filelist_mutex); | 490 | drm_close_helper(filp); |
| 477 | list_del(&file_priv->lhead); | ||
| 478 | mutex_unlock(&dev->filelist_mutex); | ||
| 479 | |||
| 480 | drm_file_free(file_priv); | ||
| 481 | 491 | ||
| 482 | if (!--dev->open_count) | 492 | if (!--dev->open_count) |
| 483 | drm_lastclose(dev); | 493 | drm_lastclose(dev); |
| @@ -508,6 +508,13 @@ int drm_version(struct drm_device *dev, void *data, · 507 unmodified lines | |||
| 508 | return err; | 508 | return err; |
| 509 | } | 509 | } |
| 510 | 510 | ||
| 511 | static inline bool | ||
| 512 | drm_render_driver_and_ioctl(const struct drm_device *dev, u32 flags) | ||
| 513 | { | ||
| 514 | return drm_core_check_feature(dev, DRIVER_RENDER) && | ||
| 515 | (flags & DRM_RENDER_ALLOW); | ||
| 516 | } | ||
| 517 | |||
| 511 | /** | 518 | /** |
| 512 | * drm_ioctl_permit - Check ioctl permissions against caller | 519 | * drm_ioctl_permit - Check ioctl permissions against caller |
| 513 | * | 520 | * |
| @@ -522,14 +529,19 @@ int drm_version(struct drm_device *dev, void *data, · 8 unmodified lines | |||
| 522 | */ | 529 | */ |
| 523 | int drm_ioctl_permit(u32 flags, struct drm_file *file_priv) | 530 | int drm_ioctl_permit(u32 flags, struct drm_file *file_priv) |
| 524 | { | 531 | { |
| 532 | const struct drm_device *dev = file_priv->minor->dev; | ||
| 533 | |||
| 525 | /* ROOT_ONLY is only for CAP_SYS_ADMIN */ | 534 | /* ROOT_ONLY is only for CAP_SYS_ADMIN */ |
| 526 | if (unlikely((flags & DRM_ROOT_ONLY) && !capable(CAP_SYS_ADMIN))) | 535 | if (unlikely((flags & DRM_ROOT_ONLY) && !capable(CAP_SYS_ADMIN))) |
| 527 | return -EACCES; | 536 | return -EACCES; |
| 528 | 537 | ||
| 529 | /* AUTH is only for authenticated or render client */ | 538 | /* AUTH is only for master ... */ |
| 530 | if (unlikely((flags & DRM_AUTH) && !drm_is_render_client(file_priv) && | 539 | if (unlikely((flags & DRM_AUTH) && drm_is_primary_client(file_priv))) { |
| 531 | !file_priv->authenticated)) | 540 | /* authenticated ones, or render capable on DRM_RENDER_ALLOW. */ |
| 532 | return -EACCES; | 541 | if (!file_priv->authenticated && |
| 542 | !drm_render_driver_and_ioctl(dev, flags)) | ||
| 543 | return -EACCES; | ||
| 544 | } | ||
| 533 | 545 | ||
| 534 | /* MASTER is only for master or control clients */ | 546 | /* MASTER is only for master or control clients */ |
| 535 | if (unlikely((flags & DRM_MASTER) && | 547 | if (unlikely((flags & DRM_MASTER) && |
| @@ -570,7 +582,7 @@ static const struct drm_ioctl_desc drm_ioctls[] = { · 34 unmodified lines | |||
| 570 | DRM_IOCTL_DEF(DRM_IOCTL_SET_UNIQUE, drm_invalid_op, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY), | 582 | DRM_IOCTL_DEF(DRM_IOCTL_SET_UNIQUE, drm_invalid_op, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY), |
| 571 | DRM_IOCTL_DEF(DRM_IOCTL_BLOCK, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY), | 583 | DRM_IOCTL_DEF(DRM_IOCTL_BLOCK, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY), |
| 572 | DRM_IOCTL_DEF(DRM_IOCTL_UNBLOCK, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY), | 584 | DRM_IOCTL_DEF(DRM_IOCTL_UNBLOCK, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY), |
| 573 | DRM_IOCTL_DEF(DRM_IOCTL_AUTH_MAGIC, drm_authmagic, DRM_AUTH|DRM_UNLOCKED|DRM_MASTER), | 585 | DRM_IOCTL_DEF(DRM_IOCTL_AUTH_MAGIC, drm_authmagic, DRM_UNLOCKED|DRM_MASTER), |
| 574 | 586 | ||
| 575 | DRM_IOCTL_DEF(DRM_IOCTL_ADD_MAP, drm_legacy_addmap_ioctl, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY), | 587 | DRM_IOCTL_DEF(DRM_IOCTL_ADD_MAP, drm_legacy_addmap_ioctl, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY), |
| 576 | DRM_IOCTL_DEF(DRM_IOCTL_RM_MAP, drm_legacy_rmmap_ioctl, DRM_AUTH), | 588 | DRM_IOCTL_DEF(DRM_IOCTL_RM_MAP, drm_legacy_rmmap_ioctl, DRM_AUTH), |
| @@ -338,8 +338,8 @@ static bool intel_fb_initial_config(struct drm_fb_helper *fb_helper, · 337 unmodified lines | |||
| 338 | bool *enabled, int width, int height) | 338 | bool *enabled, int width, int height) |
| 339 | { | 339 | { |
| 340 | struct drm_i915_private *dev_priv = to_i915(fb_helper->dev); | 340 | struct drm_i915_private *dev_priv = to_i915(fb_helper->dev); |
| 341 | unsigned long conn_configured, conn_seq, mask; | ||
| 342 | unsigned int count = min(fb_helper->connector_count, BITS_PER_LONG); | 341 | unsigned int count = min(fb_helper->connector_count, BITS_PER_LONG); |
| 342 | unsigned long conn_configured, conn_seq; | ||
| 343 | int i, j; | 343 | int i, j; |
| 344 | bool *save_enabled; | 344 | bool *save_enabled; |
| 345 | bool fallback = true, ret = true; | 345 | bool fallback = true, ret = true; |
| @@ -357,10 +357,9 @@ static bool intel_fb_initial_config(struct drm_fb_helper *fb_helper, · 11 unmodified lines | |||
| 357 | drm_modeset_backoff(&ctx); | 357 | drm_modeset_backoff(&ctx); |
| 358 | 358 | ||
| 359 | memcpy(save_enabled, enabled, count); | 359 | memcpy(save_enabled, enabled, count); |
| 360 | mask = GENMASK(count - 1, 0); | 360 | conn_seq = GENMASK(count - 1, 0); |
| 361 | conn_configured = 0; | 361 | conn_configured = 0; |
| 362 | retry: | 362 | retry: |
| 363 | conn_seq = conn_configured; | ||
| 364 | for (i = 0; i < count; i++) { | 363 | for (i = 0; i < count; i++) { |
| 365 | struct drm_fb_helper_connector *fb_conn; | 364 | struct drm_fb_helper_connector *fb_conn; |
| 366 | struct drm_connector *connector; | 365 | struct drm_connector *connector; |
| @@ -373,7 +372,8 @@ retry: · 6 unmodified lines | |||
| 373 | if (conn_configured & BIT(i)) | 372 | if (conn_configured & BIT(i)) |
| 374 | continue; | 373 | continue; |
| 375 | 374 | ||
| 376 | if (conn_seq == 0 && !connector->has_tile) | 375 | /* First pass, only consider tiled connectors */ |
| 376 | if (conn_seq == GENMASK(count - 1, 0) && !connector->has_tile) | ||
| 377 | continue; | 377 | continue; |
| 378 | 378 | ||
| 379 | if (connector->status == connector_status_connected) | 379 | if (connector->status == connector_status_connected) |
| @@ -477,8 +477,10 @@ retry: · 97 unmodified lines | |||
| 477 | conn_configured |= BIT(i); | 477 | conn_configured |= BIT(i); |
| 478 | } | 478 | } |
| 479 | 479 | ||
| 480 | if ((conn_configured & mask) != mask && conn_configured != conn_seq) | 480 | if (conn_configured != conn_seq) { /* repeat until no more are found */ |
| 481 | conn_seq = conn_configured; | ||
| 481 | goto retry; | 482 | goto retry; |
| 483 | } | ||
| 482 | 484 | ||
| 483 | /* | 485 | /* |
| 484 | * If the BIOS didn't enable everything it could, fall back to have the | 486 | * If the BIOS didn't enable everything it could, fall back to have the |
| @@ -4,7 +4,7 @@ config DRM_IMX · 3 unmodified lines | |||
| 4 | select VIDEOMODE_HELPERS | 4 | select VIDEOMODE_HELPERS |
| 5 | select DRM_GEM_CMA_HELPER | 5 | select DRM_GEM_CMA_HELPER |
| 6 | select DRM_KMS_CMA_HELPER | 6 | select DRM_KMS_CMA_HELPER |
| 7 | depends on DRM && (ARCH_MXC || ARCH_MULTIPLATFORM) | 7 | depends on DRM && (ARCH_MXC || ARCH_MULTIPLATFORM || COMPILE_TEST) |
| 8 | depends on IMX_IPUV3_CORE | 8 | depends on IMX_IPUV3_CORE |
| 9 | help | 9 | help |
| 10 | enable i.MX graphics support | 10 | enable i.MX graphics support |
| @@ -18,6 +18,7 @@ config DRM_IMX_PARALLEL_DISPLAY · 7 unmodified lines | |||
| 18 | config DRM_IMX_TVE | 18 | config DRM_IMX_TVE |
| 19 | tristate "Support for TV and VGA displays" | 19 | tristate "Support for TV and VGA displays" |
| 20 | depends on DRM_IMX | 20 | depends on DRM_IMX |
| 21 | depends on COMMON_CLK | ||
| 21 | select REGMAP_MMIO | 22 | select REGMAP_MMIO |
| 22 | help | 23 | help |
| 23 | Choose this to enable the internal Television Encoder (TVe) | 24 | Choose this to enable the internal Television Encoder (TVe) |
| @@ -49,11 +49,7 @@ static int imx_drm_atomic_check(struct drm_device *dev, · 48 unmodified lines | |||
| 49 | { | 49 | { |
| 50 | int ret; | 50 | int ret; |
| 51 | 51 | ||
| 52 | ret = drm_atomic_helper_check_modeset(dev, state); | 52 | ret = drm_atomic_helper_check(dev, state); |
| 53 | if (ret) | ||
| 54 | return ret; | ||
| 55 | |||
| 56 | ret = drm_atomic_helper_check_planes(dev, state); | ||
| 57 | if (ret) | 53 | if (ret) |
| 58 | return ret; | 54 | return ret; |
| 59 | 55 | ||
| @@ -229,6 +225,7 @@ static int imx_drm_bind(struct device *dev) · 169 unmodified lines | |||
| 229 | drm->mode_config.funcs = &imx_drm_mode_config_funcs; | 225 | drm->mode_config.funcs = &imx_drm_mode_config_funcs; |
| 230 | drm->mode_config.helper_private = &imx_drm_mode_config_helpers; | 226 | drm->mode_config.helper_private = &imx_drm_mode_config_helpers; |
| 231 | drm->mode_config.allow_fb_modifiers = true; | 227 | drm->mode_config.allow_fb_modifiers = true; |
| 228 | drm->mode_config.normalize_zpos = true; | ||
| 232 | 229 | ||
| 233 | drm_mode_config_init(drm); | 230 | drm_mode_config_init(drm); |
| 234 | 231 | ||
| @@ -34,6 +34,7 @@ struct ipu_crtc { · 33 unmodified lines | |||
| 34 | struct ipu_dc *dc; | 34 | struct ipu_dc *dc; |
| 35 | struct ipu_di *di; | 35 | struct ipu_di *di; |
| 36 | int irq; | 36 | int irq; |
| 37 | struct drm_pending_vblank_event *event; | ||
| 37 | }; | 38 | }; |
| 38 | 39 | ||
| 39 | static inline struct ipu_crtc *to_ipu_crtc(struct drm_crtc *crtc) | 40 | static inline struct ipu_crtc *to_ipu_crtc(struct drm_crtc *crtc) |
| @@ -173,8 +174,31 @@ static const struct drm_crtc_funcs ipu_crtc_funcs = { · 133 unmodified lines | |||
| 173 | static irqreturn_t ipu_irq_handler(int irq, void *dev_id) | 174 | static irqreturn_t ipu_irq_handler(int irq, void *dev_id) |
| 174 | { | 175 | { |
| 175 | struct ipu_crtc *ipu_crtc = dev_id; | 176 | struct ipu_crtc *ipu_crtc = dev_id; |
| 177 | struct drm_crtc *crtc = &ipu_crtc->base; | ||
| 178 | unsigned long flags; | ||
| 179 | int i; | ||
| 180 | |||
| 181 | drm_crtc_handle_vblank(crtc); | ||
| 182 | |||
| 183 | if (ipu_crtc->event) { | ||
| 184 | for (i = 0; i < ARRAY_SIZE(ipu_crtc->plane); i++) { | ||
| 185 | struct ipu_plane *plane = ipu_crtc->plane[i]; | ||
| 176 | 186 | ||
| 177 | drm_crtc_handle_vblank(&ipu_crtc->base); | 187 | if (!plane) |
| 188 | continue; | ||
| 189 | |||
| 190 | if (ipu_plane_atomic_update_pending(&plane->base)) | ||
| 191 | break; | ||
| 192 | } | ||
| 193 | |||
| 194 | if (i == ARRAY_SIZE(ipu_crtc->plane)) { | ||
| 195 | spin_lock_irqsave(&crtc->dev->event_lock, flags); | ||
| 196 | drm_crtc_send_vblank_event(crtc, ipu_crtc->event); | ||
| 197 | ipu_crtc->event = NULL; | ||
| 198 | drm_crtc_vblank_put(crtc); | ||
| 199 | spin_unlock_irqrestore(&crtc->dev->event_lock, flags); | ||
| 200 | } | ||
| 201 | } | ||
| 178 | 202 | ||
| 179 | return IRQ_HANDLED; | 203 | return IRQ_HANDLED; |
| 180 | } | 204 | } |
| @@ -223,8 +247,10 @@ static void ipu_crtc_atomic_flush(struct drm_crtc *crtc, · 42 unmodified lines | |||
| 223 | { | 247 | { |
| 224 | spin_lock_irq(&crtc->dev->event_lock); | 248 | spin_lock_irq(&crtc->dev->event_lock); |
| 225 | if (crtc->state->event) { | 249 | if (crtc->state->event) { |
| 250 | struct ipu_crtc *ipu_crtc = to_ipu_crtc(crtc); | ||
| 251 | |||
| 226 | WARN_ON(drm_crtc_vblank_get(crtc)); | 252 | WARN_ON(drm_crtc_vblank_get(crtc)); |
| 227 | drm_crtc_arm_vblank_event(crtc, crtc->state->event); | 253 | ipu_crtc->event = crtc->state->event; |
| 228 | crtc->state->event = NULL; | 254 | crtc->state->event = NULL; |
| 229 | } | 255 | } |
| 230 | spin_unlock_irq(&crtc->dev->event_lock); | 256 | spin_unlock_irq(&crtc->dev->event_lock); |
| @@ -273,6 +273,7 @@ static void ipu_plane_destroy(struct drm_plane *plane) · 272 unmodified lines | |||
| 273 | 273 | ||
| 274 | static void ipu_plane_state_reset(struct drm_plane *plane) | 274 | static void ipu_plane_state_reset(struct drm_plane *plane) |
| 275 | { | 275 | { |
| 276 | unsigned int zpos = (plane->type == DRM_PLANE_TYPE_PRIMARY) ? 0 : 1; | ||
| 276 | struct ipu_plane_state *ipu_state; | 277 | struct ipu_plane_state *ipu_state; |
| 277 | 278 | ||
| 278 | if (plane->state) { | 279 | if (plane->state) { |
| @@ -284,8 +285,11 @@ static void ipu_plane_state_reset(struct drm_plane *plane) · 5 unmodified lines | |||
| 284 | 285 | ||
| 285 | ipu_state = kzalloc(sizeof(*ipu_state), GFP_KERNEL); | 286 | ipu_state = kzalloc(sizeof(*ipu_state), GFP_KERNEL); |
| 286 | 287 | ||
| 287 | if (ipu_state) | 288 | if (ipu_state) { |
| 288 | __drm_atomic_helper_plane_reset(plane, &ipu_state->base); | 289 | __drm_atomic_helper_plane_reset(plane, &ipu_state->base); |
| 290 | ipu_state->base.zpos = zpos; | ||
| 291 | ipu_state->base.normalized_zpos = zpos; | ||
| 292 | } | ||
| 289 | } | 293 | } |
| 290 | 294 | ||
| 291 | static struct drm_plane_state * | 295 | static struct drm_plane_state * |
| @@ -560,6 +564,25 @@ static void ipu_plane_atomic_update(struct drm_plane *plane, · 268 unmodified lines | |||
| 560 | if (ipu_plane->dp_flow == IPU_DP_FLOW_SYNC_FG) | 564 | if (ipu_plane->dp_flow == IPU_DP_FLOW_SYNC_FG) |
| 561 | ipu_dp_set_window_pos(ipu_plane->dp, dst->x1, dst->y1); | 565 | ipu_dp_set_window_pos(ipu_plane->dp, dst->x1, dst->y1); |
| 562 | 566 | ||
| 567 | switch (ipu_plane->dp_flow) { | ||
| 568 | case IPU_DP_FLOW_SYNC_BG: | ||
| 569 | if (state->normalized_zpos == 1) { | ||
| 570 | ipu_dp_set_global_alpha(ipu_plane->dp, | ||
| 571 | !fb->format->has_alpha, 0xff, | ||
| 572 | true); | ||
| 573 | } else { | ||
| 574 | ipu_dp_set_global_alpha(ipu_plane->dp, true, 0, true); | ||
| 575 | } | ||
| 576 | break; | ||
| 577 | case IPU_DP_FLOW_SYNC_FG: | ||
| 578 | if (state->normalized_zpos == 1) { | ||
| 579 | ipu_dp_set_global_alpha(ipu_plane->dp, | ||
| 580 | !fb->format->has_alpha, 0xff, | ||
| 581 | false); | ||
| 582 | } | ||
| 583 | break; | ||
| 584 | } | ||
| 585 | |||
| 563 | eba = drm_plane_state_to_eba(state, 0); | 586 | eba = drm_plane_state_to_eba(state, 0); |
| 564 | 587 | ||
| 565 | /* | 588 | /* |
| @@ -582,6 +605,7 @@ static void ipu_plane_atomic_update(struct drm_plane *plane, · 16 unmodified lines | |||
| 582 | active = ipu_idmac_get_current_buffer(ipu_plane->ipu_ch); | 605 | active = ipu_idmac_get_current_buffer(ipu_plane->ipu_ch); |
| 583 | ipu_cpmem_set_buffer(ipu_plane->ipu_ch, !active, eba); | 606 | ipu_cpmem_set_buffer(ipu_plane->ipu_ch, !active, eba); |
| 584 | ipu_idmac_select_buffer(ipu_plane->ipu_ch, !active); | 607 | ipu_idmac_select_buffer(ipu_plane->ipu_ch, !active); |
| 608 | ipu_plane->next_buf = !active; | ||
| 585 | if (ipu_plane_separate_alpha(ipu_plane)) { | 609 | if (ipu_plane_separate_alpha(ipu_plane)) { |
| 586 | active = ipu_idmac_get_current_buffer(ipu_plane->alpha_ch); | 610 | active = ipu_idmac_get_current_buffer(ipu_plane->alpha_ch); |
| 587 | ipu_cpmem_set_buffer(ipu_plane->alpha_ch, !active, | 611 | ipu_cpmem_set_buffer(ipu_plane->alpha_ch, !active, |
| @@ -595,34 +619,11 @@ static void ipu_plane_atomic_update(struct drm_plane *plane, · 7 unmodified lines | |||
| 595 | switch (ipu_plane->dp_flow) { | 619 | switch (ipu_plane->dp_flow) { |
| 596 | case IPU_DP_FLOW_SYNC_BG: | 620 | case IPU_DP_FLOW_SYNC_BG: |
| 597 | ipu_dp_setup_channel(ipu_plane->dp, ics, IPUV3_COLORSPACE_RGB); | 621 | ipu_dp_setup_channel(ipu_plane->dp, ics, IPUV3_COLORSPACE_RGB); |
| 598 | ipu_dp_set_global_alpha(ipu_plane->dp, true, 0, true); | ||
| 599 | break; | 622 | break; |
| 600 | case IPU_DP_FLOW_SYNC_FG: | 623 | case IPU_DP_FLOW_SYNC_FG: |
| 601 | ipu_dp_setup_channel(ipu_plane->dp, ics, | 624 | ipu_dp_setup_channel(ipu_plane->dp, ics, |
| 602 | IPUV3_COLORSPACE_UNKNOWN); | 625 | IPUV3_COLORSPACE_UNKNOWN); |
| 603 | /* Enable local alpha on partial plane */ | 626 | break; |
| 604 | switch (fb->format->format) { | ||
| 605 | case DRM_FORMAT_ARGB1555: | ||
| 606 | case DRM_FORMAT_ABGR1555: | ||
| 607 | case DRM_FORMAT_RGBA5551: | ||
| 608 | case DRM_FORMAT_BGRA5551: | ||
| 609 | case DRM_FORMAT_ARGB4444: | ||
| 610 | case DRM_FORMAT_ARGB8888: | ||
| 611 | case DRM_FORMAT_ABGR8888: | ||
| 612 | case DRM_FORMAT_RGBA8888: | ||
| 613 | case DRM_FORMAT_BGRA8888: | ||
| 614 | case DRM_FORMAT_RGB565_A8: | ||
| 615 | case DRM_FORMAT_BGR565_A8: | ||
| 616 | case DRM_FORMAT_RGB888_A8: | ||
| 617 | case DRM_FORMAT_BGR888_A8: | ||
| 618 | case DRM_FORMAT_RGBX8888_A8: | ||
| 619 | case DRM_FORMAT_BGRX8888_A8: | ||
| 620 | ipu_dp_set_global_alpha(ipu_plane->dp, false, 0, false); | ||
| 621 | break; | ||
| 622 | default: | ||
| 623 | ipu_dp_set_global_alpha(ipu_plane->dp, true, 0, true); | ||
| 624 | break; | ||
| 625 | } | ||
| 626 | } | 627 | } |
| 627 | 628 | ||
| 628 | ipu_dmfc_config_wait4eot(ipu_plane->dmfc, drm_rect_width(dst)); | 629 | ipu_dmfc_config_wait4eot(ipu_plane->dmfc, drm_rect_width(dst)); |
| @@ -709,6 +710,7 @@ static void ipu_plane_atomic_update(struct drm_plane *plane, · 80 unmodified lines | |||
| 709 | ipu_cpmem_set_buffer(ipu_plane->ipu_ch, 1, eba); | 710 | ipu_cpmem_set_buffer(ipu_plane->ipu_ch, 1, eba); |
| 710 | ipu_idmac_lock_enable(ipu_plane->ipu_ch, num_bursts); | 711 | ipu_idmac_lock_enable(ipu_plane->ipu_ch, num_bursts); |
| 711 | ipu_plane_enable(ipu_plane); | 712 | ipu_plane_enable(ipu_plane); |
| 713 | ipu_plane->next_buf = -1; | ||
| 712 | } | 714 | } |
| 713 | 715 | ||
| 714 | static const struct drm_plane_helper_funcs ipu_plane_helper_funcs = { | 716 | static const struct drm_plane_helper_funcs ipu_plane_helper_funcs = { |
| @@ -718,6 +720,24 @@ static const struct drm_plane_helper_funcs ipu_plane_helper_funcs = { · 3 unmodified lines | |||
| 718 | .atomic_update = ipu_plane_atomic_update, | 720 | .atomic_update = ipu_plane_atomic_update, |
| 719 | }; | 721 | }; |
| 720 | 722 | ||
| 723 | bool ipu_plane_atomic_update_pending(struct drm_plane *plane) | ||
| 724 | { | ||
| 725 | struct ipu_plane *ipu_plane = to_ipu_plane(plane); | ||
| 726 | struct drm_plane_state *state = plane->state; | ||
| 727 | struct ipu_plane_state *ipu_state = to_ipu_plane_state(state); | ||
| 728 | |||
| 729 | /* disabled crtcs must not block the update */ | ||
| 730 | if (!state->crtc) | ||
| 731 | return false; | ||
| 732 | |||
| 733 | if (ipu_state->use_pre) | ||
| 734 | return ipu_prg_channel_configure_pending(ipu_plane->ipu_ch); | ||
| 735 | else if (ipu_plane->next_buf >= 0) | ||
| 736 | return ipu_idmac_get_current_buffer(ipu_plane->ipu_ch) != | ||
| 737 | ipu_plane->next_buf; | ||
| 738 | |||
| 739 | return false; | ||
| 740 | } | ||
| 721 | int ipu_planes_assign_pre(struct drm_device *dev, | 741 | int ipu_planes_assign_pre(struct drm_device *dev, |
| 722 | struct drm_atomic_state *state) | 742 | struct drm_atomic_state *state) |
| 723 | { | 743 | { |
| @@ -806,6 +826,7 @@ struct ipu_plane *ipu_plane_init(struct drm_device *dev, struct ipu_soc *ipu, · 82 unmodified lines | |||
| 806 | { | 826 | { |
| 807 | struct ipu_plane *ipu_plane; | 827 | struct ipu_plane *ipu_plane; |
| 808 | const uint64_t *modifiers = ipu_format_modifiers; | 828 | const uint64_t *modifiers = ipu_format_modifiers; |
| 829 | unsigned int zpos = (type == DRM_PLANE_TYPE_PRIMARY) ? 0 : 1; | ||
| 809 | int ret; | 830 | int ret; |
| 810 | 831 | ||
| 811 | DRM_DEBUG_KMS("channel %d, dp flow %d, possible_crtcs=0x%x\n", | 832 | DRM_DEBUG_KMS("channel %d, dp flow %d, possible_crtcs=0x%x\n", |
| @@ -836,5 +857,10 @@ struct ipu_plane *ipu_plane_init(struct drm_device *dev, struct ipu_soc *ipu, · 24 unmodified lines | |||
| 836 | 857 | ||
| 837 | drm_plane_helper_add(&ipu_plane->base, &ipu_plane_helper_funcs); | 858 | drm_plane_helper_add(&ipu_plane->base, &ipu_plane_helper_funcs); |
| 838 | 859 | ||
| 860 | if (dp == IPU_DP_FLOW_SYNC_BG || dp == IPU_DP_FLOW_SYNC_FG) | ||
| 861 | drm_plane_create_zpos_property(&ipu_plane->base, zpos, 0, 1); | ||
| 862 | else | ||
| 863 | drm_plane_create_zpos_immutable_property(&ipu_plane->base, 0); | ||
| 864 | |||
| 839 | return ipu_plane; | 865 | return ipu_plane; |
| 840 | } | 866 | } |
| @@ -27,6 +27,7 @@ struct ipu_plane { · 26 unmodified lines | |||
| 27 | int dp_flow; | 27 | int dp_flow; |
| 28 | 28 | ||
| 29 | bool disabling; | 29 | bool disabling; |
| 30 | int next_buf; | ||
| 30 | }; | 31 | }; |
| 31 | 32 | ||
| 32 | struct ipu_plane *ipu_plane_init(struct drm_device *dev, struct ipu_soc *ipu, | 33 | struct ipu_plane *ipu_plane_init(struct drm_device *dev, struct ipu_soc *ipu, |
| @@ -48,5 +49,6 @@ int ipu_plane_irq(struct ipu_plane *plane); · 15 unmodified lines | |||
| 48 | 49 | ||
| 49 | void ipu_plane_disable(struct ipu_plane *ipu_plane, bool disable_dp_channel); | 50 | void ipu_plane_disable(struct ipu_plane *ipu_plane, bool disable_dp_channel); |
| 50 | void ipu_plane_disable_deferred(struct drm_plane *plane); | 51 | void ipu_plane_disable_deferred(struct drm_plane *plane); |
| 52 | bool ipu_plane_atomic_update_pending(struct drm_plane *plane); | ||
| 51 | 53 | ||
| 52 | #endif | 54 | #endif |
| @@ -465,8 +465,6 @@ static void _dpu_crtc_setup_mixer_for_encoder( · 464 unmodified lines | |||
| 465 | return; | 465 | return; |
| 466 | } | 466 | } |
| 467 | 467 | ||
| 468 | mixer->encoder = enc; | ||
| 469 | |||
| 470 | cstate->num_mixers++; | 468 | cstate->num_mixers++; |
| 471 | DPU_DEBUG("setup mixer %d: lm %d\n", | 469 | DPU_DEBUG("setup mixer %d: lm %d\n", |
| 472 | i, mixer->hw_lm->idx - LM_0); | 470 | i, mixer->hw_lm->idx - LM_0); |
| @@ -718,11 +716,8 @@ void dpu_crtc_commit_kickoff(struct drm_crtc *crtc, bool async) · 245 unmodified lines | |||
| 718 | * may delay and flush at an irq event (e.g. ppdone) | 716 | * may delay and flush at an irq event (e.g. ppdone) |
| 719 | */ | 717 | */ |
| 720 | drm_for_each_encoder_mask(encoder, crtc->dev, | 718 | drm_for_each_encoder_mask(encoder, crtc->dev, |
| 721 | crtc->state->encoder_mask) { | 719 | crtc->state->encoder_mask) |
| 722 | struct dpu_encoder_kickoff_params params = { 0 }; | 720 | dpu_encoder_prepare_for_kickoff(encoder, async); |
| 723 | dpu_encoder_prepare_for_kickoff(encoder, ¶ms, async); | ||
| 724 | } | ||
| 725 | |||
| 726 | 721 | ||
| 727 | if (!async) { | 722 | if (!async) { |
| 728 | /* wait for frame_event_done completion */ | 723 | /* wait for frame_event_done completion */ |
| @@ -84,14 +84,12 @@ struct dpu_crtc_smmu_state_data { · 83 unmodified lines | |||
| 84 | * struct dpu_crtc_mixer: stores the map for each virtual pipeline in the CRTC | 84 | * struct dpu_crtc_mixer: stores the map for each virtual pipeline in the CRTC |
| 85 | * @hw_lm: LM HW Driver context | 85 | * @hw_lm: LM HW Driver context |
| 86 | * @lm_ctl: CTL Path HW driver context | 86 | * @lm_ctl: CTL Path HW driver context |
| 87 | * @encoder: Encoder attached to this lm & ctl | ||
| 88 | * @mixer_op_mode: mixer blending operation mode | 87 | * @mixer_op_mode: mixer blending operation mode |
| 89 | * @flush_mask: mixer flush mask for ctl, mixer and pipe | 88 | * @flush_mask: mixer flush mask for ctl, mixer and pipe |
| 90 | */ | 89 | */ |
| 91 | struct dpu_crtc_mixer { | 90 | struct dpu_crtc_mixer { |
| 92 | struct dpu_hw_mixer *hw_lm; | 91 | struct dpu_hw_mixer *hw_lm; |
| 93 | struct dpu_hw_ctl *lm_ctl; | 92 | struct dpu_hw_ctl *lm_ctl; |
| 94 | struct drm_encoder *encoder; | ||
| 95 | u32 mixer_op_mode; | 93 | u32 mixer_op_mode; |
| 96 | u32 flush_mask; | 94 | u32 flush_mask; |
| 97 | }; | 95 | }; |
| @@ -205,7 +205,7 @@ struct dpu_encoder_virt { · 204 unmodified lines | |||
| 205 | bool idle_pc_supported; | 205 | bool idle_pc_supported; |
| 206 | struct mutex rc_lock; | 206 | struct mutex rc_lock; |
| 207 | enum dpu_enc_rc_states rc_state; | 207 | enum dpu_enc_rc_states rc_state; |
| 208 | struct kthread_delayed_work delayed_off_work; | 208 | struct delayed_work delayed_off_work; |
| 209 | struct kthread_work vsync_event_work; | 209 | struct kthread_work vsync_event_work; |
| 210 | struct msm_display_topology topology; | 210 | struct msm_display_topology topology; |
| 211 | bool mode_set_complete; | 211 | bool mode_set_complete; |
| @@ -742,7 +742,6 @@ static int dpu_encoder_resource_control(struct drm_encoder *drm_enc, · 530 unmodified lines | |||
| 742 | { | 742 | { |
| 743 | struct dpu_encoder_virt *dpu_enc; | 743 | struct dpu_encoder_virt *dpu_enc; |
| 744 | struct msm_drm_private *priv; | 744 | struct msm_drm_private *priv; |
| 745 | struct msm_drm_thread *disp_thread; | ||
| 746 | bool is_vid_mode = false; | 745 | bool is_vid_mode = false; |
| 747 | 746 | ||
| 748 | if (!drm_enc || !drm_enc->dev || !drm_enc->dev->dev_private || | 747 | if (!drm_enc || !drm_enc->dev || !drm_enc->dev->dev_private || |
| @@ -755,12 +754,6 @@ static int dpu_encoder_resource_control(struct drm_encoder *drm_enc, · 6 unmodified lines | |||
| 755 | is_vid_mode = dpu_enc->disp_info.capabilities & | 754 | is_vid_mode = dpu_enc->disp_info.capabilities & |
| 756 | MSM_DISPLAY_CAP_VID_MODE; | 755 | MSM_DISPLAY_CAP_VID_MODE; |
| 757 | 756 | ||
| 758 | if (drm_enc->crtc->index >= ARRAY_SIZE(priv->disp_thread)) { | ||
| 759 | DPU_ERROR("invalid crtc index\n"); | ||
| 760 | return -EINVAL; | ||
| 761 | } | ||
| 762 | disp_thread = &priv->disp_thread[drm_enc->crtc->index]; | ||
| 763 | |||
| 764 | /* | 757 | /* |
| 765 | * when idle_pc is not supported, process only KICKOFF, STOP and MODESET | 758 | * when idle_pc is not supported, process only KICKOFF, STOP and MODESET |
| 766 | * events and return early for other events (ie wb display). | 759 | * events and return early for other events (ie wb display). |
| @@ -777,8 +770,7 @@ static int dpu_encoder_resource_control(struct drm_encoder *drm_enc, · 10 unmodified lines | |||
| 777 | switch (sw_event) { | 770 | switch (sw_event) { |
| 778 | case DPU_ENC_RC_EVENT_KICKOFF: | 771 | case DPU_ENC_RC_EVENT_KICKOFF: |
| 779 | /* cancel delayed off work, if any */ | 772 | /* cancel delayed off work, if any */ |
| 780 | if (kthread_cancel_delayed_work_sync( | 773 | if (cancel_delayed_work_sync(&dpu_enc->delayed_off_work)) |
| 781 | &dpu_enc->delayed_off_work)) | ||
| 782 | DPU_DEBUG_ENC(dpu_enc, "sw_event:%d, work cancelled\n", | 774 | DPU_DEBUG_ENC(dpu_enc, "sw_event:%d, work cancelled\n", |
| 783 | sw_event); | 775 | sw_event); |
| 784 | 776 | ||
| @@ -837,10 +829,8 @@ static int dpu_encoder_resource_control(struct drm_encoder *drm_enc, · 52 unmodified lines | |||
| 837 | return 0; | 829 | return 0; |
| 838 | } | 830 | } |
| 839 | 831 | ||
| 840 | kthread_queue_delayed_work( | 832 | queue_delayed_work(priv->wq, &dpu_enc->delayed_off_work, |
| 841 | &disp_thread->worker, | 833 | msecs_to_jiffies(dpu_enc->idle_timeout)); |
| 842 | &dpu_enc->delayed_off_work, | ||
| 843 | msecs_to_jiffies(dpu_enc->idle_timeout)); | ||
| 844 | 834 | ||
| 845 | trace_dpu_enc_rc(DRMID(drm_enc), sw_event, | 835 | trace_dpu_enc_rc(DRMID(drm_enc), sw_event, |
| 846 | dpu_enc->idle_pc_supported, dpu_enc->rc_state, | 836 | dpu_enc->idle_pc_supported, dpu_enc->rc_state, |
| @@ -849,8 +839,7 @@ static int dpu_encoder_resource_control(struct drm_encoder *drm_enc, · 2 unmodified lines | |||
| 849 | 839 | ||
| 850 | case DPU_ENC_RC_EVENT_PRE_STOP: | 840 | case DPU_ENC_RC_EVENT_PRE_STOP: |
| 851 | /* cancel delayed off work, if any */ | 841 | /* cancel delayed off work, if any */ |
| 852 | if (kthread_cancel_delayed_work_sync( | 842 | if (cancel_delayed_work_sync(&dpu_enc->delayed_off_work)) |
| 853 | &dpu_enc->delayed_off_work)) | ||
| 854 | DPU_DEBUG_ENC(dpu_enc, "sw_event:%d, work cancelled\n", | 843 | DPU_DEBUG_ENC(dpu_enc, "sw_event:%d, work cancelled\n", |
| 855 | sw_event); | 844 | sw_event); |
| 856 | 845 | ||
| @@ -1368,7 +1357,7 @@ static void dpu_encoder_frame_done_callback( · 511 unmodified lines | |||
| 1368 | } | 1357 | } |
| 1369 | } | 1358 | } |
| 1370 | 1359 | ||
| 1371 | static void dpu_encoder_off_work(struct kthread_work *work) | 1360 | static void dpu_encoder_off_work(struct work_struct *work) |
| 1372 | { | 1361 | { |
| 1373 | struct dpu_encoder_virt *dpu_enc = container_of(work, | 1362 | struct dpu_encoder_virt *dpu_enc = container_of(work, |
| 1374 | struct dpu_encoder_virt, delayed_off_work.work); | 1363 | struct dpu_encoder_virt, delayed_off_work.work); |
| @@ -1756,15 +1745,14 @@ static void dpu_encoder_vsync_event_work_handler(struct kthread_work *work) · 381 unmodified lines | |||
| 1756 | nsecs_to_jiffies(ktime_to_ns(wakeup_time))); | 1745 | nsecs_to_jiffies(ktime_to_ns(wakeup_time))); |
| 1757 | } | 1746 | } |
| 1758 | 1747 | ||
| 1759 | void dpu_encoder_prepare_for_kickoff(struct drm_encoder *drm_enc, | 1748 | void dpu_encoder_prepare_for_kickoff(struct drm_encoder *drm_enc, bool async) |
| 1760 | struct dpu_encoder_kickoff_params *params, bool async) | ||
| 1761 | { | 1749 | { |
| 1762 | struct dpu_encoder_virt *dpu_enc; | 1750 | struct dpu_encoder_virt *dpu_enc; |
| 1763 | struct dpu_encoder_phys *phys; | 1751 | struct dpu_encoder_phys *phys; |
| 1764 | bool needs_hw_reset = false; | 1752 | bool needs_hw_reset = false; |
| 1765 | unsigned int i; | 1753 | unsigned int i; |
| 1766 | 1754 | ||
| 1767 | if (!drm_enc || !params) { | 1755 | if (!drm_enc) { |
| 1768 | DPU_ERROR("invalid args\n"); | 1756 | DPU_ERROR("invalid args\n"); |
| 1769 | return; | 1757 | return; |
| 1770 | } | 1758 | } |
| @@ -1778,7 +1766,7 @@ void dpu_encoder_prepare_for_kickoff(struct drm_encoder *drm_enc, · 7 unmodified lines | |||
| 1778 | phys = dpu_enc->phys_encs[i]; | 1766 | phys = dpu_enc->phys_encs[i]; |
| 1779 | if (phys) { | 1767 | if (phys) { |
| 1780 | if (phys->ops.prepare_for_kickoff) | 1768 | if (phys->ops.prepare_for_kickoff) |
| 1781 | phys->ops.prepare_for_kickoff(phys, params); | 1769 | phys->ops.prepare_for_kickoff(phys); |
| 1782 | if (phys->enable_state == DPU_ENC_ERR_NEEDS_HW_RESET) | 1770 | if (phys->enable_state == DPU_ENC_ERR_NEEDS_HW_RESET) |
| 1783 | needs_hw_reset = true; | 1771 | needs_hw_reset = true; |
| 1784 | } | 1772 | } |
| @@ -2193,7 +2181,7 @@ int dpu_encoder_setup(struct drm_device *dev, struct drm_encoder *enc, · 408 unmodified lines | |||
| 2193 | 2181 | ||
| 2194 | 2182 | ||
| 2195 | mutex_init(&dpu_enc->rc_lock); | 2183 | mutex_init(&dpu_enc->rc_lock); |
| 2196 | kthread_init_delayed_work(&dpu_enc->delayed_off_work, | 2184 | INIT_DELAYED_WORK(&dpu_enc->delayed_off_work, |
| 2197 | dpu_encoder_off_work); | 2185 | dpu_encoder_off_work); |
| 2198 | dpu_enc->idle_timeout = IDLE_TIMEOUT; | 2186 | dpu_enc->idle_timeout = IDLE_TIMEOUT; |
| 2199 | 2187 | ||
| @@ -37,15 +37,6 @@ struct dpu_encoder_hw_resources { · 36 unmodified lines | |||
| 37 | enum dpu_intf_mode intfs[INTF_MAX]; | 37 | enum dpu_intf_mode intfs[INTF_MAX]; |
| 38 | }; | 38 | }; |
| 39 | 39 | ||
| 40 | /** | ||
| 41 | * dpu_encoder_kickoff_params - info encoder requires at kickoff | ||
| 42 | * @affected_displays: bitmask, bit set means the ROI of the commit lies within | ||
| 43 | * the bounds of the physical display at the bit index | ||
| 44 | */ | ||
| 45 | struct dpu_encoder_kickoff_params { | ||
| 46 | unsigned long affected_displays; | ||
| 47 | }; | ||
| 48 | |||
| 49 | /** | 40 | /** |
| 50 | * dpu_encoder_get_hw_resources - Populate table of required hardware resources | 41 | * dpu_encoder_get_hw_resources - Populate table of required hardware resources |
| 51 | * @encoder: encoder pointer | 42 | * @encoder: encoder pointer |
| @@ -88,11 +79,9 @@ void dpu_encoder_register_frame_event_callback(struct drm_encoder *encoder, · 36 unmodified lines | |||
| 88 | * Immediately: if no previous commit is outstanding. | 79 | * Immediately: if no previous commit is outstanding. |
| 89 | * Delayed: Block until next trigger can be issued. | 80 | * Delayed: Block until next trigger can be issued. |
| 90 | * @encoder: encoder pointer | 81 | * @encoder: encoder pointer |
| 91 | * @params: kickoff time parameters | ||
| 92 | * @async: true if this is an asynchronous commit | 82 | * @async: true if this is an asynchronous commit |
| 93 | */ | 83 | */ |
| 94 | void dpu_encoder_prepare_for_kickoff(struct drm_encoder *encoder, | 84 | void dpu_encoder_prepare_for_kickoff(struct drm_encoder *encoder, bool async); |
| 95 | struct dpu_encoder_kickoff_params *params, bool async); | ||
| 96 | 85 | ||
| 97 | /** | 86 | /** |
| 98 | * dpu_encoder_trigger_kickoff_pending - Clear the flush bits from previous | 87 | * dpu_encoder_trigger_kickoff_pending - Clear the flush bits from previous |
| @@ -144,8 +144,7 @@ struct dpu_encoder_phys_ops { · 143 unmodified lines | |||
| 144 | int (*wait_for_commit_done)(struct dpu_encoder_phys *phys_enc); | 144 | int (*wait_for_commit_done)(struct dpu_encoder_phys *phys_enc); |
| 145 | int (*wait_for_tx_complete)(struct dpu_encoder_phys *phys_enc); | 145 | int (*wait_for_tx_complete)(struct dpu_encoder_phys *phys_enc); |
| 146 | int (*wait_for_vblank)(struct dpu_encoder_phys *phys_enc); | 146 | int (*wait_for_vblank)(struct dpu_encoder_phys *phys_enc); |
| 147 | void (*prepare_for_kickoff)(struct dpu_encoder_phys *phys_enc, | 147 | void (*prepare_for_kickoff)(struct dpu_encoder_phys *phys_enc); |
| 148 | struct dpu_encoder_kickoff_params *params); | ||
| 149 | void (*handle_post_kickoff)(struct dpu_encoder_phys *phys_enc); | 148 | void (*handle_post_kickoff)(struct dpu_encoder_phys *phys_enc); |
| 150 | void (*trigger_start)(struct dpu_encoder_phys *phys_enc); | 149 | void (*trigger_start)(struct dpu_encoder_phys *phys_enc); |
| 151 | bool (*needs_single_flush)(struct dpu_encoder_phys *phys_enc); | 150 | bool (*needs_single_flush)(struct dpu_encoder_phys *phys_enc); |
| @@ -594,8 +594,7 @@ static void dpu_encoder_phys_cmd_get_hw_resources( · 593 unmodified lines | |||
| 594 | } | 594 | } |
| 595 | 595 | ||
| 596 | static void dpu_encoder_phys_cmd_prepare_for_kickoff( | 596 | static void dpu_encoder_phys_cmd_prepare_for_kickoff( |
| 597 | struct dpu_encoder_phys *phys_enc, | 597 | struct dpu_encoder_phys *phys_enc) |
| 598 | struct dpu_encoder_kickoff_params *params) | ||
| 599 | { | 598 | { |
| 600 | struct dpu_encoder_phys_cmd *cmd_enc = | 599 | struct dpu_encoder_phys_cmd *cmd_enc = |
| 601 | to_dpu_encoder_phys_cmd(phys_enc); | 600 | to_dpu_encoder_phys_cmd(phys_enc); |
| @@ -693,7 +692,7 @@ static int dpu_encoder_phys_cmd_wait_for_commit_done( · 91 unmodified lines | |||
| 693 | 692 | ||
| 694 | /* required for both controllers */ | 693 | /* required for both controllers */ |
| 695 | if (!rc && cmd_enc->serialize_wait4pp) | 694 | if (!rc && cmd_enc->serialize_wait4pp) |
| 696 | dpu_encoder_phys_cmd_prepare_for_kickoff(phys_enc, NULL); | 695 | dpu_encoder_phys_cmd_prepare_for_kickoff(phys_enc); |
| 697 | 696 | ||
| 698 | return rc; | 697 | return rc; |
| 699 | } | 698 | } |
| @@ -587,14 +587,13 @@ static int dpu_encoder_phys_vid_wait_for_vblank( · 586 unmodified lines | |||
| 587 | } | 587 | } |
| 588 | 588 | ||
| 589 | static void dpu_encoder_phys_vid_prepare_for_kickoff( | 589 | static void dpu_encoder_phys_vid_prepare_for_kickoff( |
| 590 | struct dpu_encoder_phys *phys_enc, | 590 | struct dpu_encoder_phys *phys_enc) |
| 591 | struct dpu_encoder_kickoff_params *params) | ||
| 592 | { | 591 | { |
| 593 | struct dpu_encoder_phys_vid *vid_enc; | 592 | struct dpu_encoder_phys_vid *vid_enc; |
| 594 | struct dpu_hw_ctl *ctl; | 593 | struct dpu_hw_ctl *ctl; |
| 595 | int rc; | 594 | int rc; |
| 596 | 595 | ||
| 597 | if (!phys_enc || !params) { | 596 | if (!phys_enc) { |
| 598 | DPU_ERROR("invalid encoder/parameters\n"); | 597 | DPU_ERROR("invalid encoder/parameters\n"); |
| 599 | return; | 598 | return; |
| 600 | } | 599 | } |
| @@ -263,13 +263,13 @@ static const struct dpu_format dpu_format_map[] = { · 262 unmodified lines | |||
| 263 | 263 | ||
| 264 | INTERLEAVED_RGB_FMT(RGB565, | 264 | INTERLEAVED_RGB_FMT(RGB565, |
| 265 | 0, COLOR_5BIT, COLOR_6BIT, COLOR_5BIT, | 265 | 0, COLOR_5BIT, COLOR_6BIT, COLOR_5BIT, |
| 266 | C1_B_Cb, C0_G_Y, C2_R_Cr, 0, 3, | 266 | C2_R_Cr, C0_G_Y, C1_B_Cb, 0, 3, |
| 267 | false, 2, 0, | 267 | false, 2, 0, |
| 268 | DPU_FETCH_LINEAR, 1), | 268 | DPU_FETCH_LINEAR, 1), |
| 269 | 269 | ||
| 270 | INTERLEAVED_RGB_FMT(BGR565, | 270 | INTERLEAVED_RGB_FMT(BGR565, |
| 271 | 0, COLOR_5BIT, COLOR_6BIT, COLOR_5BIT, | 271 | 0, COLOR_5BIT, COLOR_6BIT, COLOR_5BIT, |
| 272 | C2_R_Cr, C0_G_Y, C1_B_Cb, 0, 3, | 272 | C1_B_Cb, C0_G_Y, C2_R_Cr, 0, 3, |
| 273 | false, 2, 0, | 273 | false, 2, 0, |
| 274 | DPU_FETCH_LINEAR, 1), | 274 | DPU_FETCH_LINEAR, 1), |
| 275 | 275 | ||
| @@ -1137,36 +1137,3 @@ const struct msm_format *dpu_get_msm_format( · 861 unmodified lines | |||
| 1137 | return &fmt->base; | 1137 | return &fmt->base; |
| 1138 | return NULL; | 1138 | return NULL; |
| 1139 | } | 1139 | } |
| 1140 | |||
| 1141 | uint32_t dpu_populate_formats( | ||
| 1142 | const struct dpu_format_extended *format_list, | ||
| 1143 | uint32_t *pixel_formats, | ||
| 1144 | uint64_t *pixel_modifiers, | ||
| 1145 | uint32_t pixel_formats_max) | ||
| 1146 | { | ||
| 1147 | uint32_t i, fourcc_format; | ||
| 1148 | |||
| 1149 | if (!format_list || !pixel_formats) | ||
| 1150 | return 0; | ||
| 1151 | |||
| 1152 | for (i = 0, fourcc_format = 0; | ||
| 1153 | format_list->fourcc_format && i < pixel_formats_max; | ||
| 1154 | ++format_list) { | ||
| 1155 | /* verify if listed format is in dpu_format_map? */ | ||
| 1156 | |||
| 1157 | /* optionally return modified formats */ | ||
| 1158 | if (pixel_modifiers) { | ||
| 1159 | /* assume same modifier for all fb planes */ | ||
| 1160 | pixel_formats[i] = format_list->fourcc_format; | ||
| 1161 | pixel_modifiers[i++] = format_list->modifier; | ||
| 1162 | } else { | ||
| 1163 | /* assume base formats grouped together */ | ||
| 1164 | if (fourcc_format != format_list->fourcc_format) { | ||
| 1165 | fourcc_format = format_list->fourcc_format; | ||
| 1166 | pixel_formats[i++] = fourcc_format; | ||
| 1167 | } | ||
| 1168 | } | ||
| 1169 | } | ||
| 1170 | |||
| 1171 | return i; | ||
| 1172 | } | ||
| @@ -40,20 +40,6 @@ const struct msm_format *dpu_get_msm_format( · 39 unmodified lines | |||
| 40 | const uint32_t format, | 40 | const uint32_t format, |
| 41 | const uint64_t modifiers); | 41 | const uint64_t modifiers); |
| 42 | 42 | ||
| 43 | /** | ||
| 44 | * dpu_populate_formats - populate the given array with fourcc codes supported | ||
| 45 | * @format_list: pointer to list of possible formats | ||
| 46 | * @pixel_formats: array to populate with fourcc codes | ||
| 47 | * @pixel_modifiers: array to populate with drm modifiers, can be NULL | ||
| 48 | * @pixel_formats_max: length of pixel formats array | ||
| 49 | * Return: number of elements populated | ||
| 50 | */ | ||
| 51 | uint32_t dpu_populate_formats( | ||
| 52 | const struct dpu_format_extended *format_list, | ||
| 53 | uint32_t *pixel_formats, | ||
| 54 | uint64_t *pixel_modifiers, | ||
| 55 | uint32_t pixel_formats_max); | ||
| 56 | |||
| 57 | /** | 43 | /** |
| 58 | * dpu_format_check_modified_format - validate format and buffers for | 44 | * dpu_format_check_modified_format - validate format and buffers for |
| 59 | * dpu non-standard, i.e. modified format | 45 | * dpu non-standard, i.e. modified format |
| @@ -151,7 +151,9 @@ static const struct dpu_sspp_blks_common sdm845_sspp_common = { · 150 unmodified lines | |||
| 151 | .id = DPU_SSPP_CSC_10BIT, \ | 151 | .id = DPU_SSPP_CSC_10BIT, \ |
| 152 | .base = 0x1a00, .len = 0x100,}, \ | 152 | .base = 0x1a00, .len = 0x100,}, \ |
| 153 | .format_list = plane_formats_yuv, \ | 153 | .format_list = plane_formats_yuv, \ |
| 154 | .num_formats = ARRAY_SIZE(plane_formats_yuv), \ | ||
| 154 | .virt_format_list = plane_formats, \ | 155 | .virt_format_list = plane_formats, \ |
| 156 | .virt_num_formats = ARRAY_SIZE(plane_formats), \ | ||
| 155 | } | 157 | } |
| 156 | 158 | ||
| 157 | #define _DMA_SBLK(num, sdma_pri) \ | 159 | #define _DMA_SBLK(num, sdma_pri) \ |
| @@ -163,7 +165,9 @@ static const struct dpu_sspp_blks_common sdm845_sspp_common = { · 5 unmodified lines | |||
| 163 | .src_blk = {.name = STRCAT("sspp_src_", num), \ | 165 | .src_blk = {.name = STRCAT("sspp_src_", num), \ |
| 164 | .id = DPU_SSPP_SRC, .base = 0x00, .len = 0x150,}, \ | 166 | .id = DPU_SSPP_SRC, .base = 0x00, .len = 0x150,}, \ |
| 165 | .format_list = plane_formats, \ | 167 | .format_list = plane_formats, \ |
| 168 | .num_formats = ARRAY_SIZE(plane_formats), \ | ||
| 166 | .virt_format_list = plane_formats, \ | 169 | .virt_format_list = plane_formats, \ |
| 170 | .virt_num_formats = ARRAY_SIZE(plane_formats), \ | ||
| 167 | } | 171 | } |
| 168 | 172 | ||
| 169 | static const struct dpu_sspp_sub_blks sdm845_vig_sblk_0 = _VIG_SBLK("0", 5); | 173 | static const struct dpu_sspp_sub_blks sdm845_vig_sblk_0 = _VIG_SBLK("0", 5); |
| @@ -251,17 +251,6 @@ struct dpu_pp_blk { · 250 unmodified lines | |||
| 251 | u32 version; | 251 | u32 version; |
| 252 | }; | 252 | }; |
| 253 | 253 | ||
| 254 | /** | ||
| 255 | * struct dpu_format_extended - define dpu specific pixel format+modifier | ||
| 256 | * @fourcc_format: Base FOURCC pixel format code | ||
| 257 | * @modifier: 64-bit drm format modifier, same modifier must be applied to all | ||
| 258 | * framebuffer planes | ||
| 259 | */ | ||
| 260 | struct dpu_format_extended { | ||
| 261 | uint32_t fourcc_format; | ||
| 262 | uint64_t modifier; | ||
| 263 | }; | ||
| 264 | |||
| 265 | /** | 254 | /** |
| 266 | * enum dpu_qos_lut_usage - define QoS LUT use cases | 255 | * enum dpu_qos_lut_usage - define QoS LUT use cases |
| 267 | */ | 256 | */ |
| @@ -348,7 +337,9 @@ struct dpu_sspp_blks_common { · 80 unmodified lines | |||
| 348 | * @pcc_blk: | 337 | * @pcc_blk: |
| 349 | * @igc_blk: | 338 | * @igc_blk: |
| 350 | * @format_list: Pointer to list of supported formats | 339 | * @format_list: Pointer to list of supported formats |
| 340 | * @num_formats: Number of supported formats | ||
| 351 | * @virt_format_list: Pointer to list of supported formats for virtual planes | 341 | * @virt_format_list: Pointer to list of supported formats for virtual planes |
| 342 | * @virt_num_formats: Number of supported formats for virtual planes | ||
| 352 | */ | 343 | */ |
| 353 | struct dpu_sspp_sub_blks { | 344 | struct dpu_sspp_sub_blks { |
| 354 | const struct dpu_sspp_blks_common *common; | 345 | const struct dpu_sspp_blks_common *common; |
| @@ -366,8 +357,10 @@ struct dpu_sspp_sub_blks { · 11 unmodified lines | |||
| 366 | struct dpu_pp_blk pcc_blk; | 357 | struct dpu_pp_blk pcc_blk; |
| 367 | struct dpu_pp_blk igc_blk; | 358 | struct dpu_pp_blk igc_blk; |
| 368 | 359 | ||
| 369 | const struct dpu_format_extended *format_list; | 360 | const u32 *format_list; |
| 370 | const struct dpu_format_extended *virt_format_list; | 361 | u32 num_formats; |
| 362 | const u32 *virt_format_list; | ||
| 363 | u32 virt_num_formats; | ||
| 371 | }; | 364 | }; |
| 372 | 365 | ||
| 373 | /** | 366 | /** |
| @@ -12,157 +12,81 @@ · 11 unmodified lines | |||
| 12 | 12 | ||
| 13 | #include "dpu_hw_mdss.h" | 13 | #include "dpu_hw_mdss.h" |
| 14 | 14 | ||
| 15 | static const struct dpu_format_extended plane_formats[] = { | 15 | static const uint32_t qcom_compressed_supported_formats[] = { |
| 16 | {DRM_FORMAT_ARGB8888, 0}, | 16 | DRM_FORMAT_ABGR8888, |
| 17 | {DRM_FORMAT_ABGR8888, 0}, | 17 | DRM_FORMAT_XBGR8888, |
| 18 | {DRM_FORMAT_RGBA8888, 0}, | 18 | DRM_FORMAT_BGR565, |
| 19 | {DRM_FORMAT_ABGR8888, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | ||
| 20 | {DRM_FORMAT_BGRA8888, 0}, | ||
| 21 | {DRM_FORMAT_XRGB8888, 0}, | ||
| 22 | {DRM_FORMAT_RGBX8888, 0}, | ||
| 23 | {DRM_FORMAT_BGRX8888, 0}, | ||
| 24 | {DRM_FORMAT_XBGR8888, 0}, | ||
| 25 | {DRM_FORMAT_XBGR8888, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | ||
| 26 | {DRM_FORMAT_RGB888, 0}, | ||
| 27 | {DRM_FORMAT_BGR888, 0}, | ||
| 28 | {DRM_FORMAT_RGB565, 0}, | ||
| 29 | {DRM_FORMAT_BGR565, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | ||
| 30 | {DRM_FORMAT_BGR565, 0}, | ||
| 31 | {DRM_FORMAT_ARGB1555, 0}, | ||
| 32 | {DRM_FORMAT_ABGR1555, 0}, | ||
| 33 | {DRM_FORMAT_RGBA5551, 0}, | ||
| 34 | {DRM_FORMAT_BGRA5551, 0}, | ||
| 35 | {DRM_FORMAT_XRGB1555, 0}, | ||
| 36 | {DRM_FORMAT_XBGR1555, 0}, | ||
| 37 | {DRM_FORMAT_RGBX5551, 0}, | ||
| 38 | {DRM_FORMAT_BGRX5551, 0}, | ||
| 39 | {DRM_FORMAT_ARGB4444, 0}, | ||
| 40 | {DRM_FORMAT_ABGR4444, 0}, | ||
| 41 | {DRM_FORMAT_RGBA4444, 0}, | ||
| 42 | {DRM_FORMAT_BGRA4444, 0}, | ||
| 43 | {DRM_FORMAT_XRGB4444, 0}, | ||
| 44 | {DRM_FORMAT_XBGR4444, 0}, | ||
| 45 | {DRM_FORMAT_RGBX4444, 0}, | ||
| 46 | {DRM_FORMAT_BGRX4444, 0}, | ||
| 47 | {0, 0}, | ||
| 48 | }; | 19 | }; |
| 49 | 20 | ||
| 50 | static const struct dpu_format_extended plane_formats_yuv[] = { | 21 | static const uint32_t plane_formats[] = { |
| 51 | {DRM_FORMAT_ARGB8888, 0}, | 22 | DRM_FORMAT_ARGB8888, |
| 52 | {DRM_FORMAT_ABGR8888, 0}, | 23 | DRM_FORMAT_ABGR8888, |
| 53 | {DRM_FORMAT_RGBA8888, 0}, | 24 | DRM_FORMAT_RGBA8888, |
| 54 | {DRM_FORMAT_BGRX8888, 0}, | 25 | DRM_FORMAT_BGRA8888, |
| 55 | {DRM_FORMAT_ABGR8888, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | 26 | DRM_FORMAT_XRGB8888, |
| 56 | {DRM_FORMAT_BGRA8888, 0}, | 27 | DRM_FORMAT_RGBX8888, |
| 57 | {DRM_FORMAT_XRGB8888, 0}, | 28 | DRM_FORMAT_BGRX8888, |
| 58 | {DRM_FORMAT_XBGR8888, 0}, | 29 | DRM_FORMAT_XBGR8888, |
| 59 | {DRM_FORMAT_RGBX8888, 0}, | 30 | DRM_FORMAT_RGB888, |
| 60 | {DRM_FORMAT_XBGR8888, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | 31 | DRM_FORMAT_BGR888, |
| 61 | {DRM_FORMAT_RGB888, 0}, | 32 | DRM_FORMAT_RGB565, |
| 62 | {DRM_FORMAT_BGR888, 0}, | 33 | DRM_FORMAT_BGR565, |
| 63 | {DRM_FORMAT_RGB565, 0}, | 34 | DRM_FORMAT_ARGB1555, |
| 64 | {DRM_FORMAT_BGR565, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | 35 | DRM_FORMAT_ABGR1555, |
| 65 | {DRM_FORMAT_BGR565, 0}, | 36 | DRM_FORMAT_RGBA5551, |
| 66 | {DRM_FORMAT_ARGB1555, 0}, | 37 | DRM_FORMAT_BGRA5551, |
| 67 | {DRM_FORMAT_ABGR1555, 0}, | 38 | DRM_FORMAT_XRGB1555, |
| 68 | {DRM_FORMAT_RGBA5551, 0}, | 39 | DRM_FORMAT_XBGR1555, |
| 69 | {DRM_FORMAT_BGRA5551, 0}, | 40 | DRM_FORMAT_RGBX5551, |
| 70 | {DRM_FORMAT_XRGB1555, 0}, | 41 | DRM_FORMAT_BGRX5551, |
| 71 | {DRM_FORMAT_XBGR1555, 0}, | 42 | DRM_FORMAT_ARGB4444, |
| 72 | {DRM_FORMAT_RGBX5551, 0}, | 43 | DRM_FORMAT_ABGR4444, |
| 73 | {DRM_FORMAT_BGRX5551, 0}, | 44 | DRM_FORMAT_RGBA4444, |
| 74 | {DRM_FORMAT_ARGB4444, 0}, | 45 | DRM_FORMAT_BGRA4444, |
| 75 | {DRM_FORMAT_ABGR4444, 0}, | 46 | DRM_FORMAT_XRGB4444, |
| 76 | {DRM_FORMAT_RGBA4444, 0}, | 47 | DRM_FORMAT_XBGR4444, |
| 77 | {DRM_FORMAT_BGRA4444, 0}, | 48 | DRM_FORMAT_RGBX4444, |
| 78 | {DRM_FORMAT_XRGB4444, 0}, | 49 | DRM_FORMAT_BGRX4444, |
| 79 | {DRM_FORMAT_XBGR4444, 0}, | ||
| 80 | {DRM_FORMAT_RGBX4444, 0}, | ||
| 81 | {DRM_FORMAT_BGRX4444, 0}, | ||
| 82 | |||
| 83 | {DRM_FORMAT_NV12, 0}, | ||
| 84 | {DRM_FORMAT_NV12, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | ||
| 85 | {DRM_FORMAT_NV21, 0}, | ||
| 86 | {DRM_FORMAT_NV16, 0}, | ||
| 87 | {DRM_FORMAT_NV61, 0}, | ||
| 88 | {DRM_FORMAT_VYUY, 0}, | ||
| 89 | {DRM_FORMAT_UYVY, 0}, | ||
| 90 | {DRM_FORMAT_YUYV, 0}, | ||
| 91 | {DRM_FORMAT_YVYU, 0}, | ||
| 92 | {DRM_FORMAT_YUV420, 0}, | ||
| 93 | {DRM_FORMAT_YVU420, 0}, | ||
| 94 | {0, 0}, | ||
| 95 | }; | ||
| 96 | |||
| 97 | static const struct dpu_format_extended cursor_formats[] = { | ||
| 98 | {DRM_FORMAT_ARGB8888, 0}, | ||
| 99 | {DRM_FORMAT_ABGR8888, 0}, | ||
| 100 | {DRM_FORMAT_RGBA8888, 0}, | ||
| 101 | {DRM_FORMAT_BGRA8888, 0}, | ||
| 102 | {DRM_FORMAT_XRGB8888, 0}, | ||
| 103 | {DRM_FORMAT_ARGB1555, 0}, | ||
| 104 | {DRM_FORMAT_ABGR1555, 0}, | ||
| 105 | {DRM_FORMAT_RGBA5551, 0}, | ||
| 106 | {DRM_FORMAT_BGRA5551, 0}, | ||
| 107 | {DRM_FORMAT_ARGB4444, 0}, | ||
| 108 | {DRM_FORMAT_ABGR4444, 0}, | ||
| 109 | {DRM_FORMAT_RGBA4444, 0}, | ||
| 110 | {DRM_FORMAT_BGRA4444, 0}, | ||
| 111 | {0, 0}, | ||
| 112 | }; | 50 | }; |
| 113 | 51 | ||
| 114 | static const struct dpu_format_extended wb2_formats[] = { | 52 | static const uint32_t plane_formats_yuv[] = { |
| 115 | {DRM_FORMAT_RGB565, 0}, | 53 | DRM_FORMAT_ARGB8888, |
| 116 | {DRM_FORMAT_BGR565, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | 54 | DRM_FORMAT_ABGR8888, |
| 117 | {DRM_FORMAT_RGB888, 0}, | 55 | DRM_FORMAT_RGBA8888, |
| 118 | {DRM_FORMAT_ARGB8888, 0}, | 56 | DRM_FORMAT_BGRX8888, |
| 119 | {DRM_FORMAT_RGBA8888, 0}, | 57 | DRM_FORMAT_BGRA8888, |
| 120 | {DRM_FORMAT_ABGR8888, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | 58 | DRM_FORMAT_XRGB8888, |
| 121 | {DRM_FORMAT_XRGB8888, 0}, | 59 | DRM_FORMAT_XBGR8888, |
| 122 | {DRM_FORMAT_RGBX8888, 0}, | 60 | DRM_FORMAT_RGBX8888, |
| 123 | {DRM_FORMAT_XBGR8888, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | 61 | DRM_FORMAT_RGB888, |
| 124 | {DRM_FORMAT_ARGB1555, 0}, | 62 | DRM_FORMAT_BGR888, |
| 125 | {DRM_FORMAT_RGBA5551, 0}, | 63 | DRM_FORMAT_RGB565, |
| 126 | {DRM_FORMAT_XRGB1555, 0}, | 64 | DRM_FORMAT_BGR565, |
| 127 | {DRM_FORMAT_RGBX5551, 0}, | 65 | DRM_FORMAT_ARGB1555, |
| 128 | {DRM_FORMAT_ARGB4444, 0}, | 66 | DRM_FORMAT_ABGR1555, |
| 129 | {DRM_FORMAT_RGBA4444, 0}, | 67 | DRM_FORMAT_RGBA5551, |
| 130 | {DRM_FORMAT_RGBX4444, 0}, | 68 | DRM_FORMAT_BGRA5551, |
| 131 | {DRM_FORMAT_XRGB4444, 0}, | 69 | DRM_FORMAT_XRGB1555, |
| 132 | 70 | DRM_FORMAT_XBGR1555, | |
| 133 | {DRM_FORMAT_BGR565, 0}, | 71 | DRM_FORMAT_RGBX5551, |
| 134 | {DRM_FORMAT_BGR888, 0}, | 72 | DRM_FORMAT_BGRX5551, |
| 135 | {DRM_FORMAT_ABGR8888, 0}, | 73 | DRM_FORMAT_ARGB4444, |
| 136 | {DRM_FORMAT_BGRA8888, 0}, | 74 | DRM_FORMAT_ABGR4444, |
| 137 | {DRM_FORMAT_BGRX8888, 0}, | 75 | DRM_FORMAT_RGBA4444, |
| 138 | {DRM_FORMAT_XBGR8888, 0}, | 76 | DRM_FORMAT_BGRA4444, |
| 139 | {DRM_FORMAT_ABGR1555, 0}, | 77 | DRM_FORMAT_XRGB4444, |
| 140 | {DRM_FORMAT_BGRA5551, 0}, | 78 | DRM_FORMAT_XBGR4444, |
| 141 | {DRM_FORMAT_XBGR1555, 0}, | 79 | DRM_FORMAT_RGBX4444, |
| 142 | {DRM_FORMAT_BGRX5551, 0}, | 80 | DRM_FORMAT_BGRX4444, |
| 143 | {DRM_FORMAT_ABGR4444, 0}, | ||
| 144 | {DRM_FORMAT_BGRA4444, 0}, | ||
| 145 | {DRM_FORMAT_BGRX4444, 0}, | ||
| 146 | {DRM_FORMAT_XBGR4444, 0}, | ||
| 147 | |||
| 148 | {DRM_FORMAT_YUV420, 0}, | ||
| 149 | {DRM_FORMAT_NV12, 0}, | ||
| 150 | {DRM_FORMAT_NV12, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | ||
| 151 | {DRM_FORMAT_NV16, 0}, | ||
| 152 | {DRM_FORMAT_YUYV, 0}, | ||
| 153 | |||
| 154 | {0, 0}, | ||
| 155 | }; | ||
| 156 | 81 | ||
| 157 | static const struct dpu_format_extended rgb_10bit_formats[] = { | 82 | DRM_FORMAT_NV12, |
| 158 | {DRM_FORMAT_BGRA1010102, 0}, | 83 | DRM_FORMAT_NV21, |
| 159 | {DRM_FORMAT_BGRX1010102, 0}, | 84 | DRM_FORMAT_NV16, |
| 160 | {DRM_FORMAT_RGBA1010102, 0}, | 85 | DRM_FORMAT_NV61, |
| 161 | {DRM_FORMAT_RGBX1010102, 0}, | 86 | DRM_FORMAT_VYUY, |
| 162 | {DRM_FORMAT_ABGR2101010, 0}, | 87 | DRM_FORMAT_UYVY, |
| 163 | {DRM_FORMAT_ABGR2101010, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | 88 | DRM_FORMAT_YUYV, |
| 164 | {DRM_FORMAT_XBGR2101010, 0}, | 89 | DRM_FORMAT_YVYU, |
| 165 | {DRM_FORMAT_XBGR2101010, DRM_FORMAT_MOD_QCOM_COMPRESSED}, | 90 | DRM_FORMAT_YUV420, |
| 166 | {DRM_FORMAT_ARGB2101010, 0}, | 91 | DRM_FORMAT_YVU420, |
| 167 | {DRM_FORMAT_XRGB2101010, 0}, | ||
| 168 | }; | 92 | }; |
| @@ -170,10 +170,6 @@ · 169 unmodified lines | |||
| 170 | /** | 170 | /** |
| 171 | * AD4 interrupt status bit definitions | 171 | * AD4 interrupt status bit definitions |
| 172 | */ | 172 | */ |
| 173 | #define DPU_INTR_BRIGHTPR_UPDATED BIT(4) | ||
| 174 | #define DPU_INTR_DARKENH_UPDATED BIT(3) | ||
| 175 | #define DPU_INTR_STREN_OUTROI_UPDATED BIT(2) | ||
| 176 | #define DPU_INTR_STREN_INROI_UPDATED BIT(1) | ||
| 177 | #define DPU_INTR_BACKLIGHT_UPDATED BIT(0) | 173 | #define DPU_INTR_BACKLIGHT_UPDATED BIT(0) |
| 178 | /** | 174 | /** |
| 179 | * struct dpu_intr_reg - array of DPU register sets | 175 | * struct dpu_intr_reg - array of DPU register sets |
| @@ -782,18 +778,6 @@ static int dpu_hw_intr_irqidx_lookup(enum dpu_intr_type intr_type, · 602 unmodified lines | |||
| 782 | return -EINVAL; | 778 | return -EINVAL; |
| 783 | } | 779 | } |
| 784 | 780 | ||
| 785 | static void dpu_hw_intr_set_mask(struct dpu_hw_intr *intr, uint32_t reg_off, | ||
| 786 | uint32_t mask) | ||
| 787 | { | ||
| 788 | if (!intr) | ||
| 789 | return; | ||
| 790 | |||
| 791 | DPU_REG_WRITE(&intr->hw, reg_off, mask); | ||
| 792 | |||
| 793 | /* ensure register writes go through */ | ||
| 794 | wmb(); | ||
| 795 | } | ||
| 796 | |||
| 797 | static void dpu_hw_intr_dispatch_irq(struct dpu_hw_intr *intr, | 781 | static void dpu_hw_intr_dispatch_irq(struct dpu_hw_intr *intr, |
| 798 | void (*cbfunc)(void *, int), | 782 | void (*cbfunc)(void *, int), |
| 799 | void *arg) | 783 | void *arg) |
| @@ -1004,18 +988,6 @@ static int dpu_hw_intr_disable_irqs(struct dpu_hw_intr *intr) · 204 unmodified lines | |||
| 1004 | return 0; | 988 | return 0; |
| 1005 | } | 989 | } |
| 1006 | 990 | ||
| 1007 | static int dpu_hw_intr_get_valid_interrupts(struct dpu_hw_intr *intr, | ||
| 1008 | uint32_t *mask) | ||
| 1009 | { | ||
| 1010 | if (!intr || !mask) | ||
| 1011 | return -EINVAL; | ||
| 1012 | |||
| 1013 | *mask = IRQ_SOURCE_MDP | IRQ_SOURCE_DSI0 | IRQ_SOURCE_DSI1 | ||
| 1014 | | IRQ_SOURCE_HDMI | IRQ_SOURCE_EDP; | ||
| 1015 | |||
| 1016 | return 0; | ||
| 1017 | } | ||
| 1018 | |||
| 1019 | static void dpu_hw_intr_get_interrupt_statuses(struct dpu_hw_intr *intr) | 991 | static void dpu_hw_intr_get_interrupt_statuses(struct dpu_hw_intr *intr) |
| 1020 | { | 992 | { |
| 1021 | int i; | 993 | int i; |
| @@ -1065,19 +1037,6 @@ static void dpu_hw_intr_clear_intr_status_nolock(struct dpu_hw_intr *intr, · 43 unmodified lines | |||
| 1065 | wmb(); | 1037 | wmb(); |
| 1066 | } | 1038 | } |
| 1067 | 1039 | ||
| 1068 | static void dpu_hw_intr_clear_interrupt_status(struct dpu_hw_intr *intr, | ||
| 1069 | int irq_idx) | ||
| 1070 | { | ||
| 1071 | unsigned long irq_flags; | ||
| 1072 | |||
| 1073 | if (!intr) | ||
| 1074 | return; | ||
| 1075 | |||
| 1076 | spin_lock_irqsave(&intr->irq_lock, irq_flags); | ||
| 1077 | dpu_hw_intr_clear_intr_status_nolock(intr, irq_idx); | ||
| 1078 | spin_unlock_irqrestore(&intr->irq_lock, irq_flags); | ||
| 1079 | } | ||
| 1080 | |||
| 1081 | static u32 dpu_hw_intr_get_interrupt_status(struct dpu_hw_intr *intr, | 1040 | static u32 dpu_hw_intr_get_interrupt_status(struct dpu_hw_intr *intr, |
| 1082 | int irq_idx, bool clear) | 1041 | int irq_idx, bool clear) |
| 1083 | { | 1042 | { |
| @@ -1113,16 +1072,13 @@ static u32 dpu_hw_intr_get_interrupt_status(struct dpu_hw_intr *intr, · 29 unmodified lines | |||
| 1113 | 1072 | ||
| 1114 | static void __setup_intr_ops(struct dpu_hw_intr_ops *ops) | 1073 | static void __setup_intr_ops(struct dpu_hw_intr_ops *ops) |
| 1115 | { | 1074 | { |
| 1116 | ops->set_mask = dpu_hw_intr_set_mask; | ||
| 1117 | ops->irq_idx_lookup = dpu_hw_intr_irqidx_lookup; | 1075 | ops->irq_idx_lookup = dpu_hw_intr_irqidx_lookup; |
| 1118 | ops->enable_irq = dpu_hw_intr_enable_irq; | 1076 | ops->enable_irq = dpu_hw_intr_enable_irq; |
| 1119 | ops->disable_irq = dpu_hw_intr_disable_irq; | 1077 | ops->disable_irq = dpu_hw_intr_disable_irq; |
| 1120 | ops->dispatch_irqs = dpu_hw_intr_dispatch_irq; | 1078 | ops->dispatch_irqs = dpu_hw_intr_dispatch_irq; |
| 1121 | ops->clear_all_irqs = dpu_hw_intr_clear_irqs; | 1079 | ops->clear_all_irqs = dpu_hw_intr_clear_irqs; |
| 1122 | ops->disable_all_irqs = dpu_hw_intr_disable_irqs; | 1080 | ops->disable_all_irqs = dpu_hw_intr_disable_irqs; |
| 1123 | ops->get_valid_interrupts = dpu_hw_intr_get_valid_interrupts; | ||
| 1124 | ops->get_interrupt_statuses = dpu_hw_intr_get_interrupt_statuses; | 1081 | ops->get_interrupt_statuses = dpu_hw_intr_get_interrupt_statuses; |
| 1125 | ops->clear_interrupt_status = dpu_hw_intr_clear_interrupt_status; | ||
| 1126 | ops->clear_intr_status_nolock = dpu_hw_intr_clear_intr_status_nolock; | 1082 | ops->clear_intr_status_nolock = dpu_hw_intr_clear_intr_status_nolock; |
| 1127 | ops->get_interrupt_status = dpu_hw_intr_get_interrupt_status; | 1083 | ops->get_interrupt_status = dpu_hw_intr_get_interrupt_status; |
| 1128 | } | 1084 | } |
| @@ -20,13 +20,6 @@ · 19 unmodified lines | |||
| 20 | #include "dpu_hw_util.h" | 20 | #include "dpu_hw_util.h" |
| 21 | #include "dpu_hw_mdss.h" | 21 | #include "dpu_hw_mdss.h" |
| 22 | 22 | ||
| 23 | #define IRQ_SOURCE_MDP BIT(0) | ||
| 24 | #define IRQ_SOURCE_DSI0 BIT(4) | ||
| 25 | #define IRQ_SOURCE_DSI1 BIT(5) | ||
| 26 | #define IRQ_SOURCE_HDMI BIT(8) | ||
| 27 | #define IRQ_SOURCE_EDP BIT(12) | ||
| 28 | #define IRQ_SOURCE_MHL BIT(16) | ||
| 29 | |||
| 30 | /** | 23 | /** |
| 31 | * dpu_intr_type - HW Interrupt Type | 24 | * dpu_intr_type - HW Interrupt Type |
| 32 | * @DPU_IRQ_TYPE_WB_ROT_COMP: WB rotator done | 25 | * @DPU_IRQ_TYPE_WB_ROT_COMP: WB rotator done |
| @@ -95,18 +88,6 @@ struct dpu_hw_intr; · 62 unmodified lines | |||
| 95 | * Interrupt operations. | 88 | * Interrupt operations. |
| 96 | */ | 89 | */ |
| 97 | struct dpu_hw_intr_ops { | 90 | struct dpu_hw_intr_ops { |
| 98 | /** | ||
| 99 | * set_mask - Programs the given interrupt register with the | ||
| 100 | * given interrupt mask. Register value will get overwritten. | ||
| 101 | * @intr: HW interrupt handle | ||
| 102 | * @reg_off: MDSS HW register offset | ||
| 103 | * @irqmask: IRQ mask value | ||
| 104 | */ | ||
| 105 | void (*set_mask)( | ||
| 106 | struct dpu_hw_intr *intr, | ||
| 107 | uint32_t reg, | ||
| 108 | uint32_t irqmask); | ||
| 109 | |||
| 110 | /** | 91 | /** |
| 111 | * irq_idx_lookup - Lookup IRQ index on the HW interrupt type | 92 | * irq_idx_lookup - Lookup IRQ index on the HW interrupt type |
| 112 | * Used for all irq related ops | 93 | * Used for all irq related ops |
| @@ -176,16 +157,6 @@ struct dpu_hw_intr_ops { · 63 unmodified lines | |||
| 176 | void (*get_interrupt_statuses)( | 157 | void (*get_interrupt_statuses)( |
| 177 | struct dpu_hw_intr *intr); | 158 | struct dpu_hw_intr *intr); |
| 178 | 159 | ||
| 179 | /** | ||
| 180 | * clear_interrupt_status - Clears HW interrupt status based on given | ||
| 181 | * lookup IRQ index. | ||
| 182 | * @intr: HW interrupt handle | ||
| 183 | * @irq_idx: Lookup irq index return from irq_idx_lookup | ||
| 184 | */ | ||
| 185 | void (*clear_interrupt_status)( | ||
| 186 | struct dpu_hw_intr *intr, | ||
| 187 | int irq_idx); | ||
| 188 | |||
| 189 | /** | 160 | /** |
| 190 | * clear_intr_status_nolock() - clears the HW interrupts without lock | 161 | * clear_intr_status_nolock() - clears the HW interrupts without lock |
| 191 | * @intr: HW interrupt handle | 162 | * @intr: HW interrupt handle |
| @@ -206,21 +177,6 @@ struct dpu_hw_intr_ops { · 14 unmodified lines | |||
| 206 | struct dpu_hw_intr *intr, | 177 | struct dpu_hw_intr *intr, |
| 207 | int irq_idx, | 178 | int irq_idx, |
| 208 | bool clear); | 179 | bool clear); |
| 209 | |||
| 210 | /** | ||
| 211 | * get_valid_interrupts - Gets a mask of all valid interrupt sources | ||
| 212 | * within DPU. These are actually status bits | ||
| 213 | * within interrupt registers that specify the | ||
| 214 | * source of the interrupt in IRQs. For example, | ||
| 215 | * valid interrupt sources can be MDP, DSI, | ||
| 216 | * HDMI etc. | ||
| 217 | * @intr: HW interrupt handle | ||
| 218 | * @mask: Returning the interrupt source MASK | ||
| 219 | * @return: 0 for success, otherwise failure | ||
| 220 | */ | ||
| 221 | int (*get_valid_interrupts)( | ||
| 222 | struct dpu_hw_intr *intr, | ||
| 223 | uint32_t *mask); | ||
| 224 | }; | 180 | }; |
| 225 | 181 | ||
| 226 | /** | 182 | /** |
| @@ -258,12 +258,6 @@ enum dpu_vbif { · 257 unmodified lines | |||
| 258 | VBIF_NRT = VBIF_1 | 258 | VBIF_NRT = VBIF_1 |
| 259 | }; | 259 | }; |
| 260 | 260 | ||
| 261 | enum dpu_iommu_domain { | ||
| 262 | DPU_IOMMU_DOMAIN_UNSECURE, | ||
| 263 | DPU_IOMMU_DOMAIN_SECURE, | ||
| 264 | DPU_IOMMU_DOMAIN_MAX | ||
| 265 | }; | ||
| 266 | |||
| 267 | /** | 261 | /** |
| 268 | * DPU HW,Component order color map | 262 | * DPU HW,Component order color map |
| 269 | */ | 263 | */ |
| @@ -358,7 +352,6 @@ enum dpu_3d_blend_mode { · 88 unmodified lines | |||
| 358 | * @alpha_enable: whether the format has an alpha channel | 352 | * @alpha_enable: whether the format has an alpha channel |
| 359 | * @num_planes: number of planes (including meta data planes) | 353 | * @num_planes: number of planes (including meta data planes) |
| 360 | * @fetch_mode: linear, tiled, or ubwc hw fetch behavior | 354 | * @fetch_mode: linear, tiled, or ubwc hw fetch behavior |
| 361 | * @is_yuv: is format a yuv variant | ||
| 362 | * @flag: usage bit flags | 355 | * @flag: usage bit flags |
| 363 | * @tile_width: format tile width | 356 | * @tile_width: format tile width |
| 364 | * @tile_height: format tile height | 357 | * @tile_height: format tile height |
| @@ -18,7 +18,6 @@ · 17 unmodified lines | |||
| 18 | #include "dpu_hw_mdss.h" | 18 | #include "dpu_hw_mdss.h" |
| 19 | 19 | ||
| 20 | #define REG_MASK(n) ((BIT(n)) - 1) | 20 | #define REG_MASK(n) ((BIT(n)) - 1) |
| 21 | struct dpu_format_extended; | ||
| 22 | 21 | ||
| 23 | /* | 22 | /* |
| 24 | * This is the common struct maintained by each sub block | 23 | * This is the common struct maintained by each sub block |
| @@ -405,35 +405,38 @@ static void dpu_kms_wait_for_commit_done(struct msm_kms *kms, · 404 unmodified lines | |||
| 405 | } | 405 | } |
| 406 | } | 406 | } |
| 407 | 407 | ||
| 408 | static void _dpu_kms_initialize_dsi(struct drm_device *dev, | 408 | static int _dpu_kms_initialize_dsi(struct drm_device *dev, |
| 409 | struct msm_drm_private *priv, | 409 | struct msm_drm_private *priv, |
| 410 | struct dpu_kms *dpu_kms) | 410 | struct dpu_kms *dpu_kms) |
| 411 | { | 411 | { |
| 412 | struct drm_encoder *encoder = NULL; | 412 | struct drm_encoder *encoder = NULL; |
| 413 | int i, rc; | 413 | int i, rc = 0; |
| 414 | |||
| 415 | if (!(priv->dsi[0] || priv->dsi[1])) | ||
| 416 | return rc; | ||
| 414 | 417 | ||
| 415 | /*TODO: Support two independent DSI connectors */ | 418 | /*TODO: Support two independent DSI connectors */ |
| 416 | encoder = dpu_encoder_init(dev, DRM_MODE_ENCODER_DSI); | 419 | encoder = dpu_encoder_init(dev, DRM_MODE_ENCODER_DSI); |
| 417 | if (IS_ERR_OR_NULL(encoder)) { | 420 | if (IS_ERR(encoder)) { |
| 418 | DPU_ERROR("encoder init failed for dsi display\n"); | 421 | DPU_ERROR("encoder init failed for dsi display\n"); |
| 419 | return; | 422 | return PTR_ERR(encoder); |
| 420 | } | 423 | } |
| 421 | 424 | ||
| 422 | priv->encoders[priv->num_encoders++] = encoder; | 425 | priv->encoders[priv->num_encoders++] = encoder; |
| 423 | 426 | ||
| 424 | for (i = 0; i < ARRAY_SIZE(priv->dsi); i++) { | 427 | for (i = 0; i < ARRAY_SIZE(priv->dsi); i++) { |
| 425 | if (!priv->dsi[i]) { | 428 | if (!priv->dsi[i]) |
| 426 | DPU_DEBUG("invalid msm_dsi for ctrl %d\n", i); | 429 | continue; |
| 427 | return; | ||
| 428 | } | ||
| 429 | 430 | ||
| 430 | rc = msm_dsi_modeset_init(priv->dsi[i], dev, encoder); | 431 | rc = msm_dsi_modeset_init(priv->dsi[i], dev, encoder); |
| 431 | if (rc) { | 432 | if (rc) { |
| 432 | DPU_ERROR("modeset_init failed for dsi[%d], rc = %d\n", | 433 | DPU_ERROR("modeset_init failed for dsi[%d], rc = %d\n", |
| 433 | i, rc); | 434 | i, rc); |
| 434 | continue; | 435 | break; |
| 435 | } | 436 | } |
| 436 | } | 437 | } |
| 438 | |||
| 439 | return rc; | ||
| 437 | } | 440 | } |
| 438 | 441 | ||
| 439 | /** | 442 | /** |
| @@ -444,16 +447,16 @@ static void _dpu_kms_initialize_dsi(struct drm_device *dev, · 4 unmodified lines | |||
| 444 | * @dpu_kms: Pointer to dpu kms structure | 447 | * @dpu_kms: Pointer to dpu kms structure |
| 445 | * Returns: Zero on success | 448 | * Returns: Zero on success |
| 446 | */ | 449 | */ |
| 447 | static void _dpu_kms_setup_displays(struct drm_device *dev, | 450 | static int _dpu_kms_setup_displays(struct drm_device *dev, |
| 448 | struct msm_drm_private *priv, | 451 | struct msm_drm_private *priv, |
| 449 | struct dpu_kms *dpu_kms) | 452 | struct dpu_kms *dpu_kms) |
| 450 | { | 453 | { |
| 451 | _dpu_kms_initialize_dsi(dev, priv, dpu_kms); | ||
| 452 | |||
| 453 | /** | 454 | /** |
| 454 | * Extend this function to initialize other | 455 | * Extend this function to initialize other |
| 455 | * types of displays | 456 | * types of displays |
| 456 | */ | 457 | */ |
| 458 | |||
| 459 | return _dpu_kms_initialize_dsi(dev, priv, dpu_kms); | ||
| 457 | } | 460 | } |
| 458 | 461 | ||
| 459 | static void _dpu_kms_drm_obj_destroy(struct dpu_kms *dpu_kms) | 462 | static void _dpu_kms_drm_obj_destroy(struct dpu_kms *dpu_kms) |
| @@ -516,7 +519,9 @@ static int _dpu_kms_drm_obj_init(struct dpu_kms *dpu_kms) · 56 unmodified lines | |||
| 516 | * Create encoder and query display drivers to create | 519 | * Create encoder and query display drivers to create |
| 517 | * bridges and connectors | 520 | * bridges and connectors |
| 518 | */ | 521 | */ |
| 519 | _dpu_kms_setup_displays(dev, priv, dpu_kms); | 522 | ret = _dpu_kms_setup_displays(dev, priv, dpu_kms); |
| 523 | if (ret) | ||
| 524 | goto fail; | ||
| 520 | 525 | ||
| 521 | max_crtc_count = min(catalog->mixer_count, priv->num_encoders); | 526 | max_crtc_count = min(catalog->mixer_count, priv->num_encoders); |
| 522 | 527 | ||
| @@ -627,6 +632,10 @@ static void _dpu_kms_hw_destroy(struct dpu_kms *dpu_kms) · 104 unmodified lines | |||
| 627 | devm_iounmap(&dpu_kms->pdev->dev, dpu_kms->vbif[VBIF_RT]); | 632 | devm_iounmap(&dpu_kms->pdev->dev, dpu_kms->vbif[VBIF_RT]); |
| 628 | dpu_kms->vbif[VBIF_RT] = NULL; | 633 | dpu_kms->vbif[VBIF_RT] = NULL; |
| 629 | 634 | ||
| 635 | if (dpu_kms->hw_mdp) | ||
| 636 | dpu_hw_mdp_destroy(dpu_kms->hw_mdp); | ||
| 637 | dpu_kms->hw_mdp = NULL; | ||
| 638 | |||
| 630 | if (dpu_kms->mmio) | 639 | if (dpu_kms->mmio) |
| 631 | devm_iounmap(&dpu_kms->pdev->dev, dpu_kms->mmio); | 640 | devm_iounmap(&dpu_kms->pdev->dev, dpu_kms->mmio); |
| 632 | dpu_kms->mmio = NULL; | 641 | dpu_kms->mmio = NULL; |
| @@ -877,8 +886,7 @@ static int dpu_kms_hw_init(struct msm_kms *kms) · 244 unmodified lines | |||
| 877 | goto power_error; | 886 | goto power_error; |
| 878 | } | 887 | } |
| 879 | 888 | ||
| 880 | rc = dpu_rm_init(&dpu_kms->rm, dpu_kms->catalog, dpu_kms->mmio, | 889 | rc = dpu_rm_init(&dpu_kms->rm, dpu_kms->catalog, dpu_kms->mmio); |
| 881 | dpu_kms->dev); | ||
| 882 | if (rc) { | 890 | if (rc) { |
| 883 | DPU_ERROR("rm init failed: %d\n", rc); | 891 | DPU_ERROR("rm init failed: %d\n", rc); |
| 884 | goto power_error; | 892 | goto power_error; |
| @@ -886,11 +894,10 @@ static int dpu_kms_hw_init(struct msm_kms *kms) · 1 unmodified line | |||
| 886 | 894 | ||
| 887 | dpu_kms->rm_init = true; | 895 | dpu_kms->rm_init = true; |
| 888 | 896 | ||
| 889 | dpu_kms->hw_mdp = dpu_rm_get_mdp(&dpu_kms->rm); | 897 | dpu_kms->hw_mdp = dpu_hw_mdptop_init(MDP_TOP, dpu_kms->mmio, |
| 890 | if (IS_ERR_OR_NULL(dpu_kms->hw_mdp)) { | 898 | dpu_kms->catalog); |
| 899 | if (IS_ERR(dpu_kms->hw_mdp)) { | ||
| 891 | rc = PTR_ERR(dpu_kms->hw_mdp); | 900 | rc = PTR_ERR(dpu_kms->hw_mdp); |
| 892 | if (!dpu_kms->hw_mdp) | ||
| 893 | rc = -EINVAL; | ||
| 894 | DPU_ERROR("failed to get hw_mdp: %d\n", rc); | 901 | DPU_ERROR("failed to get hw_mdp: %d\n", rc); |
| 895 | dpu_kms->hw_mdp = NULL; | 902 | dpu_kms->hw_mdp = NULL; |
| 896 | goto power_error; | 903 | goto power_error; |
| @@ -926,16 +933,6 @@ static int dpu_kms_hw_init(struct msm_kms *kms) · 29 unmodified lines | |||
| 926 | goto hw_intr_init_err; | 933 | goto hw_intr_init_err; |
| 927 | } | 934 | } |
| 928 | 935 | ||
| 929 | /* | ||
| 930 | * _dpu_kms_drm_obj_init should create the DRM related objects | ||
| 931 | * i.e. CRTCs, planes, encoders, connectors and so forth | ||
| 932 | */ | ||
| 933 | rc = _dpu_kms_drm_obj_init(dpu_kms); | ||
| 934 | if (rc) { | ||
| 935 | DPU_ERROR("modeset init failed: %d\n", rc); | ||
| 936 | goto drm_obj_init_err; | ||
| 937 | } | ||
| 938 | |||
| 939 | dev->mode_config.min_width = 0; | 936 | dev->mode_config.min_width = 0; |
| 940 | dev->mode_config.min_height = 0; | 937 | dev->mode_config.min_height = 0; |
| 941 | 938 | ||
| @@ -952,6 +949,16 @@ static int dpu_kms_hw_init(struct msm_kms *kms) · 10 unmodified lines | |||
| 952 | */ | 949 | */ |
| 953 | dev->mode_config.allow_fb_modifiers = true; | 950 | dev->mode_config.allow_fb_modifiers = true; |
| 954 | 951 | ||
| 952 | /* | ||
| 953 | * _dpu_kms_drm_obj_init should create the DRM related objects | ||
| 954 | * i.e. CRTCs, planes, encoders, connectors and so forth | ||
| 955 | */ | ||
| 956 | rc = _dpu_kms_drm_obj_init(dpu_kms); | ||
| 957 | if (rc) { | ||
| 958 | DPU_ERROR("modeset init failed: %d\n", rc); | ||
| 959 | goto drm_obj_init_err; | ||
| 960 | } | ||
| 961 | |||
| 955 | dpu_vbif_init_memtypes(dpu_kms); | 962 | dpu_vbif_init_memtypes(dpu_kms); |
| 956 | 963 | ||
| 957 | pm_runtime_put_sync(&dpu_kms->pdev->dev); | 964 | pm_runtime_put_sync(&dpu_kms->pdev->dev); |
| @@ -23,11 +23,14 @@ struct dpu_mdss { · 22 unmodified lines | |||
| 23 | struct dpu_irq_controller irq_controller; | 23 | struct dpu_irq_controller irq_controller; |
| 24 | }; | 24 | }; |
| 25 | 25 | ||
| 26 | static irqreturn_t dpu_mdss_irq(int irq, void *arg) | 26 | static void dpu_mdss_irq(struct irq_desc *desc) |
| 27 | { | 27 | { |
| 28 | struct dpu_mdss *dpu_mdss = arg; | 28 | struct dpu_mdss *dpu_mdss = irq_desc_get_handler_data(desc); |
| 29 | struct irq_chip *chip = irq_desc_get_chip(desc); | ||
| 29 | u32 interrupts; | 30 | u32 interrupts; |
| 30 | 31 | ||
| 32 | chained_irq_enter(chip, desc); | ||
| 33 | |||
| 31 | interrupts = readl_relaxed(dpu_mdss->mmio + HW_INTR_STATUS); | 34 | interrupts = readl_relaxed(dpu_mdss->mmio + HW_INTR_STATUS); |
| 32 | 35 | ||
| 33 | while (interrupts) { | 36 | while (interrupts) { |
| @@ -39,20 +42,20 @@ static irqreturn_t dpu_mdss_irq(int irq, void *arg) · 5 unmodified lines | |||
| 39 | hwirq); | 42 | hwirq); |
| 40 | if (mapping == 0) { | 43 | if (mapping == 0) { |
| 41 | DRM_ERROR("couldn't find irq mapping for %lu\n", hwirq); | 44 | DRM_ERROR("couldn't find irq mapping for %lu\n", hwirq); |
| 42 | return IRQ_NONE; | 45 | break; |
| 43 | } | 46 | } |
| 44 | 47 | ||
| 45 | rc = generic_handle_irq(mapping); | 48 | rc = generic_handle_irq(mapping); |
| 46 | if (rc < 0) { | 49 | if (rc < 0) { |
| 47 | DRM_ERROR("handle irq fail: irq=%lu mapping=%u rc=%d\n", | 50 | DRM_ERROR("handle irq fail: irq=%lu mapping=%u rc=%d\n", |
| 48 | hwirq, mapping, rc); | 51 | hwirq, mapping, rc); |
| 49 | return IRQ_NONE; | 52 | break; |
| 50 | } | 53 | } |
| 51 | 54 | ||
| 52 | interrupts &= ~(1 << hwirq); | 55 | interrupts &= ~(1 << hwirq); |
| 53 | } | 56 | } |
| 54 | 57 | ||
| 55 | return IRQ_HANDLED; | 58 | chained_irq_exit(chip, desc); |
| 56 | } | 59 | } |
| 57 | 60 | ||
| 58 | static void dpu_mdss_irq_mask(struct irq_data *irqd) | 61 | static void dpu_mdss_irq_mask(struct irq_data *irqd) |
| @@ -83,16 +86,16 @@ static struct irq_chip dpu_mdss_irq_chip = { · 24 unmodified lines | |||
| 83 | .irq_unmask = dpu_mdss_irq_unmask, | 86 | .irq_unmask = dpu_mdss_irq_unmask, |
| 84 | }; | 87 | }; |
| 85 | 88 | ||
| 89 | static struct lock_class_key dpu_mdss_lock_key, dpu_mdss_request_key; | ||
| 90 | |||
| 86 | static int dpu_mdss_irqdomain_map(struct irq_domain *domain, | 91 | static int dpu_mdss_irqdomain_map(struct irq_domain *domain, |
| 87 | unsigned int irq, irq_hw_number_t hwirq) | 92 | unsigned int irq, irq_hw_number_t hwirq) |
| 88 | { | 93 | { |
| 89 | struct dpu_mdss *dpu_mdss = domain->host_data; | 94 | struct dpu_mdss *dpu_mdss = domain->host_data; |
| 90 | int ret; | ||
| 91 | 95 | ||
| 96 | irq_set_lockdep_class(irq, &dpu_mdss_lock_key, &dpu_mdss_request_key); | ||
| 92 | irq_set_chip_and_handler(irq, &dpu_mdss_irq_chip, handle_level_irq); | 97 | irq_set_chip_and_handler(irq, &dpu_mdss_irq_chip, handle_level_irq); |
| 93 | ret = irq_set_chip_data(irq, dpu_mdss); | 98 | return irq_set_chip_data(irq, dpu_mdss); |
| 94 | |||
| 95 | return ret; | ||
| 96 | } | 99 | } |
| 97 | 100 | ||
| 98 | static const struct irq_domain_ops dpu_mdss_irqdomain_ops = { | 101 | static const struct irq_domain_ops dpu_mdss_irqdomain_ops = { |
| @@ -159,11 +162,13 @@ static void dpu_mdss_destroy(struct drm_device *dev) · 60 unmodified lines | |||
| 159 | struct msm_drm_private *priv = dev->dev_private; | 162 | struct msm_drm_private *priv = dev->dev_private; |
| 160 | struct dpu_mdss *dpu_mdss = to_dpu_mdss(priv->mdss); | 163 | struct dpu_mdss *dpu_mdss = to_dpu_mdss(priv->mdss); |
| 161 | struct dss_module_power *mp = &dpu_mdss->mp; | 164 | struct dss_module_power *mp = &dpu_mdss->mp; |
| 165 | int irq; | ||
| 162 | 166 | ||
| 163 | pm_runtime_suspend(dev->dev); | 167 | pm_runtime_suspend(dev->dev); |
| 164 | pm_runtime_disable(dev->dev); | 168 | pm_runtime_disable(dev->dev); |
| 165 | _dpu_mdss_irq_domain_fini(dpu_mdss); | 169 | _dpu_mdss_irq_domain_fini(dpu_mdss); |
| 166 | free_irq(platform_get_irq(pdev, 0), dpu_mdss); | 170 | irq = platform_get_irq(pdev, 0); |
| 171 | irq_set_chained_handler_and_data(irq, NULL, NULL); | ||
| 167 | msm_dss_put_clk(mp->clk_config, mp->num_clk); | 172 | msm_dss_put_clk(mp->clk_config, mp->num_clk); |
| 168 | devm_kfree(&pdev->dev, mp->clk_config); | 173 | devm_kfree(&pdev->dev, mp->clk_config); |
| 169 | 174 | ||
| @@ -187,6 +192,7 @@ int dpu_mdss_init(struct drm_device *dev) · 17 unmodified lines | |||
| 187 | struct dpu_mdss *dpu_mdss; | 192 | struct dpu_mdss *dpu_mdss; |
| 188 | struct dss_module_power *mp; | 193 | struct dss_module_power *mp; |
| 189 | int ret = 0; | 194 | int ret = 0; |
| 195 | int irq; | ||
| 190 | 196 | ||
| 191 | dpu_mdss = devm_kzalloc(dev->dev, sizeof(*dpu_mdss), GFP_KERNEL); | 197 | dpu_mdss = devm_kzalloc(dev->dev, sizeof(*dpu_mdss), GFP_KERNEL); |
| 192 | if (!dpu_mdss) | 198 | if (!dpu_mdss) |
| @@ -219,12 +225,12 @@ int dpu_mdss_init(struct drm_device *dev) · 26 unmodified lines | |||
| 219 | if (ret) | 225 | if (ret) |
| 220 | goto irq_domain_error; | 226 | goto irq_domain_error; |
| 221 | 227 | ||
| 222 | ret = request_irq(platform_get_irq(pdev, 0), | 228 | irq = platform_get_irq(pdev, 0); |
| 223 | dpu_mdss_irq, 0, "dpu_mdss_isr", dpu_mdss); | 229 | if (irq < 0) |
| 224 | if (ret) { | ||
| 225 | DPU_ERROR("failed to init irq: %d\n", ret); | ||
| 226 | goto irq_error; | 230 | goto irq_error; |
| 227 | } | 231 | |
| 232 | irq_set_chained_handler_and_data(irq, dpu_mdss_irq, | ||
| 233 | dpu_mdss); | ||
| 228 | 234 | ||
| 229 | pm_runtime_enable(dev->dev); | 235 | pm_runtime_enable(dev->dev); |
| 230 | 236 | ||
| @@ -95,8 +95,6 @@ struct dpu_plane { · 94 unmodified lines | |||
| 95 | 95 | ||
| 96 | enum dpu_sspp pipe; | 96 | enum dpu_sspp pipe; |
| 97 | uint32_t features; /* capabilities from catalog */ | 97 | uint32_t features; /* capabilities from catalog */ |
| 98 | uint32_t nformats; | ||
| 99 | uint32_t formats[64]; | ||
| 100 | 98 | ||
| 101 | struct dpu_hw_pipe *pipe_hw; | 99 | struct dpu_hw_pipe *pipe_hw; |
| 102 | struct dpu_hw_pipe_cfg pipe_cfg; | 100 | struct dpu_hw_pipe_cfg pipe_cfg; |
| @@ -121,6 +119,12 @@ struct dpu_plane { · 18 unmodified lines | |||
| 121 | bool debugfs_default_scale; | 119 | bool debugfs_default_scale; |
| 122 | }; | 120 | }; |
| 123 | 121 | ||
| 122 | static const uint64_t supported_format_modifiers[] = { | ||
| 123 | DRM_FORMAT_MOD_QCOM_COMPRESSED, | ||
| 124 | DRM_FORMAT_MOD_LINEAR, | ||
| 125 | DRM_FORMAT_MOD_INVALID | ||
| 126 | }; | ||
| 127 | |||
| 124 | #define to_dpu_plane(x) container_of(x, struct dpu_plane, base) | 128 | #define to_dpu_plane(x) container_of(x, struct dpu_plane, base) |
| 125 | 129 | ||
| 126 | static struct dpu_kms *_dpu_plane_get_kms(struct drm_plane *plane) | 130 | static struct dpu_kms *_dpu_plane_get_kms(struct drm_plane *plane) |
| @@ -1410,6 +1414,23 @@ static void dpu_plane_early_unregister(struct drm_plane *plane) · 1283 unmodified lines | |||
| 1410 | debugfs_remove_recursive(pdpu->debugfs_root); | 1414 | debugfs_remove_recursive(pdpu->debugfs_root); |
| 1411 | } | 1415 | } |
| 1412 | 1416 | ||
| 1417 | static bool dpu_plane_format_mod_supported(struct drm_plane *plane, | ||
| 1418 | uint32_t format, uint64_t modifier) | ||
| 1419 | { | ||
| 1420 | if (modifier == DRM_FORMAT_MOD_LINEAR) | ||
| 1421 | return true; | ||
| 1422 | |||
| 1423 | if (modifier == DRM_FORMAT_MOD_QCOM_COMPRESSED) { | ||
| 1424 | int i; | ||
| 1425 | for (i = 0; i < ARRAY_SIZE(qcom_compressed_supported_formats); i++) { | ||
| 1426 | if (format == qcom_compressed_supported_formats[i]) | ||
| 1427 | return true; | ||
| 1428 | } | ||
| 1429 | } | ||
| 1430 | |||
| 1431 | return false; | ||
| 1432 | } | ||
| 1433 | |||
| 1413 | static const struct drm_plane_funcs dpu_plane_funcs = { | 1434 | static const struct drm_plane_funcs dpu_plane_funcs = { |
| 1414 | .update_plane = drm_atomic_helper_update_plane, | 1435 | .update_plane = drm_atomic_helper_update_plane, |
| 1415 | .disable_plane = drm_atomic_helper_disable_plane, | 1436 | .disable_plane = drm_atomic_helper_disable_plane, |
| @@ -1419,6 +1440,7 @@ static const struct drm_plane_funcs dpu_plane_funcs = { · 3 unmodified lines | |||
| 1419 | .atomic_destroy_state = dpu_plane_destroy_state, | 1440 | .atomic_destroy_state = dpu_plane_destroy_state, |
| 1420 | .late_register = dpu_plane_late_register, | 1441 | .late_register = dpu_plane_late_register, |
| 1421 | .early_unregister = dpu_plane_early_unregister, | 1442 | .early_unregister = dpu_plane_early_unregister, |
| 1443 | .format_mod_supported = dpu_plane_format_mod_supported, | ||
| 1422 | }; | 1444 | }; |
| 1423 | 1445 | ||
| 1424 | static const struct drm_plane_helper_funcs dpu_plane_helper_funcs = { | 1446 | static const struct drm_plane_helper_funcs dpu_plane_helper_funcs = { |
| @@ -1444,11 +1466,12 @@ struct drm_plane *dpu_plane_init(struct drm_device *dev, · 19 unmodified lines | |||
| 1444 | unsigned long possible_crtcs, u32 master_plane_id) | 1466 | unsigned long possible_crtcs, u32 master_plane_id) |
| 1445 | { | 1467 | { |
| 1446 | struct drm_plane *plane = NULL, *master_plane = NULL; | 1468 | struct drm_plane *plane = NULL, *master_plane = NULL; |
| 1447 | const struct dpu_format_extended *format_list; | 1469 | const uint32_t *format_list; |
| 1448 | struct dpu_plane *pdpu; | 1470 | struct dpu_plane *pdpu; |
| 1449 | struct msm_drm_private *priv = dev->dev_private; | 1471 | struct msm_drm_private *priv = dev->dev_private; |
| 1450 | struct dpu_kms *kms = to_dpu_kms(priv->kms); | 1472 | struct dpu_kms *kms = to_dpu_kms(priv->kms); |
| 1451 | int zpos_max = DPU_ZPOS_MAX; | 1473 | int zpos_max = DPU_ZPOS_MAX; |
| 1474 | uint32_t num_formats; | ||
| 1452 | int ret = -EINVAL; | 1475 | int ret = -EINVAL; |
| 1453 | 1476 | ||
| 1454 | /* create and zero local structure */ | 1477 | /* create and zero local structure */ |
| @@ -1491,24 +1514,18 @@ struct drm_plane *dpu_plane_init(struct drm_device *dev, · 36 unmodified lines | |||
| 1491 | goto clean_sspp; | 1514 | goto clean_sspp; |
| 1492 | } | 1515 | } |
| 1493 | 1516 | ||
| 1494 | if (!master_plane_id) | 1517 | if (pdpu->is_virtual) { |
| 1495 | format_list = pdpu->pipe_sblk->format_list; | ||
| 1496 | else | ||
| 1497 | format_list = pdpu->pipe_sblk->virt_format_list; | 1518 | format_list = pdpu->pipe_sblk->virt_format_list; |
| 1498 | 1519 | num_formats = pdpu->pipe_sblk->virt_num_formats; | |
| 1499 | pdpu->nformats = dpu_populate_formats(format_list, | 1520 | } |
| 1500 | pdpu->formats, | 1521 | else { |
| 1501 | 0, | 1522 | format_list = pdpu->pipe_sblk->format_list; |
| 1502 | ARRAY_SIZE(pdpu->formats)); | 1523 | num_formats = pdpu->pipe_sblk->num_formats; |
| 1503 | |||
| 1504 | if (!pdpu->nformats) { | ||
| 1505 | DPU_ERROR("[%u]no valid formats for plane\n", pipe); | ||
| 1506 | goto clean_sspp; | ||
| 1507 | } | 1524 | } |
| 1508 | 1525 | ||
| 1509 | ret = drm_universal_plane_init(dev, plane, 0xff, &dpu_plane_funcs, | 1526 | ret = drm_universal_plane_init(dev, plane, 0xff, &dpu_plane_funcs, |
| 1510 | pdpu->formats, pdpu->nformats, | 1527 | format_list, num_formats, |
| 1511 | NULL, type, NULL); | 1528 | supported_format_modifiers, type, NULL); |
| 1512 | if (ret) | 1529 | if (ret) |
| 1513 | goto clean_sspp; | 1530 | goto clean_sspp; |
| 1514 | 1531 | ||
| @@ -28,23 +28,18 @@ · 27 unmodified lines | |||
| 28 | /** | 28 | /** |
| 29 | * struct dpu_plane_state: Define dpu extension of drm plane state object | 29 | * struct dpu_plane_state: Define dpu extension of drm plane state object |
| 30 | * @base: base drm plane state object | 30 | * @base: base drm plane state object |
| 31 | * @property_state: Local storage for msm_prop properties | ||
| 32 | * @property_values: cached plane property values | ||
| 33 | * @aspace: pointer to address space for input/output buffers | 31 | * @aspace: pointer to address space for input/output buffers |
| 34 | * @input_fence: dereferenced input fence pointer | ||
| 35 | * @stage: assigned by crtc blender | 32 | * @stage: assigned by crtc blender |
| 36 | * @multirect_index: index of the rectangle of SSPP | 33 | * @multirect_index: index of the rectangle of SSPP |
| 37 | * @multirect_mode: parallel or time multiplex multirect mode | 34 | * @multirect_mode: parallel or time multiplex multirect mode |
| 38 | * @pending: whether the current update is still pending | 35 | * @pending: whether the current update is still pending |
| 39 | * @scaler3_cfg: configuration data for scaler3 | 36 | * @scaler3_cfg: configuration data for scaler3 |
| 40 | * @pixel_ext: configuration data for pixel extensions | 37 | * @pixel_ext: configuration data for pixel extensions |
| 41 | * @scaler_check_state: indicates status of user provided pixel extension data | ||
| 42 | * @cdp_cfg: CDP configuration | 38 | * @cdp_cfg: CDP configuration |
| 43 | */ | 39 | */ |
| 44 | struct dpu_plane_state { | 40 | struct dpu_plane_state { |
| 45 | struct drm_plane_state base; | 41 | struct drm_plane_state base; |
| 46 | struct msm_gem_address_space *aspace; | 42 | struct msm_gem_address_space *aspace; |
| 47 | void *input_fence; | ||
| 48 | enum dpu_stage stage; | 43 | enum dpu_stage stage; |
| 49 | uint32_t multirect_index; | 44 | uint32_t multirect_index; |
| 50 | uint32_t multirect_mode; | 45 | uint32_t multirect_mode; |
| @@ -106,12 +101,6 @@ void dpu_plane_restore(struct drm_plane *plane); · 55 unmodified lines | |||
| 106 | */ | 101 | */ |
| 107 | void dpu_plane_flush(struct drm_plane *plane); | 102 | void dpu_plane_flush(struct drm_plane *plane); |
| 108 | 103 | ||
| 109 | /** | ||
| 110 | * dpu_plane_kickoff - final plane operations before commit kickoff | ||
| 111 | * @plane: Pointer to drm plane structure | ||
| 112 | */ | ||
| 113 | void dpu_plane_kickoff(struct drm_plane *plane); | ||
| 114 | |||
| 115 | /** | 104 | /** |
| 116 | * dpu_plane_set_error: enable/disable error condition | 105 | * dpu_plane_set_error: enable/disable error condition |
| 117 | * @plane: pointer to drm_plane structure | 106 | * @plane: pointer to drm_plane structure |
| @@ -146,14 +135,6 @@ int dpu_plane_validate_multirect_v2(struct dpu_multirect_plane_states *plane); · 28 unmodified lines | |||
| 146 | */ | 135 | */ |
| 147 | void dpu_plane_clear_multirect(const struct drm_plane_state *drm_state); | 136 | void dpu_plane_clear_multirect(const struct drm_plane_state *drm_state); |
| 148 | 137 | ||
| 149 | /** | ||
| 150 | * dpu_plane_wait_input_fence - wait for input fence object | ||
| 151 | * @plane: Pointer to DRM plane object | ||
| 152 | * @wait_ms: Wait timeout value | ||
| 153 | * Returns: Zero on success | ||
| 154 | */ | ||
| 155 | int dpu_plane_wait_input_fence(struct drm_plane *plane, uint32_t wait_ms); | ||
| 156 | |||
| 157 | /** | 138 | /** |
| 158 | * dpu_plane_color_fill - enables color fill on plane | 139 | * dpu_plane_color_fill - enables color fill on plane |
| 159 | * @plane: Pointer to DRM plane object | 140 | * @plane: Pointer to DRM plane object |
| @@ -164,12 +145,4 @@ int dpu_plane_wait_input_fence(struct drm_plane *plane, uint32_t wait_ms); · 4 unmodified lines | |||
| 164 | int dpu_plane_color_fill(struct drm_plane *plane, | 145 | int dpu_plane_color_fill(struct drm_plane *plane, |
| 165 | uint32_t color, uint32_t alpha); | 146 | uint32_t color, uint32_t alpha); |
| 166 | 147 | ||
| 167 | /** | ||
| 168 | * dpu_plane_set_revalidate - sets revalidate flag which forces a full | ||
| 169 | * validation of the plane properties in the next atomic check | ||
| 170 | * @plane: Pointer to DRM plane object | ||
| 171 | * @enable: Boolean to set/unset the flag | ||
| 172 | */ | ||
| 173 | void dpu_plane_set_revalidate(struct drm_plane *plane, bool enable); | ||
| 174 | |||
| 175 | #endif /* _DPU_PLANE_H_ */ | 148 | #endif /* _DPU_PLANE_H_ */ |
| @@ -21,8 +21,8 @@ · 20 unmodified lines | |||
| 21 | #include "dpu_encoder.h" | 21 | #include "dpu_encoder.h" |
| 22 | #include "dpu_trace.h" | 22 | #include "dpu_trace.h" |
| 23 | 23 | ||
| 24 | #define RESERVED_BY_OTHER(h, r) \ | 24 | #define RESERVED_BY_OTHER(h, r) \ |
| 25 | ((h)->rsvp && ((h)->rsvp->enc_id != (r)->enc_id)) | 25 | ((h)->enc_id && (h)->enc_id != r) |
| 26 | 26 | ||
| 27 | /** | 27 | /** |
| 28 | * struct dpu_rm_requirements - Reservation requirements parameter bundle | 28 | * struct dpu_rm_requirements - Reservation requirements parameter bundle |
| @@ -34,90 +34,21 @@ struct dpu_rm_requirements { · 5 unmodified lines | |||
| 34 | struct dpu_encoder_hw_resources hw_res; | 34 | struct dpu_encoder_hw_resources hw_res; |
| 35 | }; | 35 | }; |
| 36 | 36 | ||
| 37 | /** | ||
| 38 | * struct dpu_rm_rsvp - Use Case Reservation tagging structure | ||
| 39 | * Used to tag HW blocks as reserved by a CRTC->Encoder->Connector chain | ||
| 40 | * By using as a tag, rather than lists of pointers to HW blocks used | ||
| 41 | * we can avoid some list management since we don't know how many blocks | ||
| 42 | * of each type a given use case may require. | ||
| 43 | * @list: List head for list of all reservations | ||
| 44 | * @seq: Global RSVP sequence number for debugging, especially for | ||
| 45 | * differentiating differenct allocations for same encoder. | ||
| 46 | * @enc_id: Reservations are tracked by Encoder DRM object ID. | ||
| 47 | * CRTCs may be connected to multiple Encoders. | ||
| 48 | * An encoder or connector id identifies the display path. | ||
| 49 | */ | ||
| 50 | struct dpu_rm_rsvp { | ||
| 51 | struct list_head list; | ||
| 52 | uint32_t seq; | ||
| 53 | uint32_t enc_id; | ||
| 54 | }; | ||
| 55 | 37 | ||
| 56 | /** | 38 | /** |
| 57 | * struct dpu_rm_hw_blk - hardware block tracking list member | 39 | * struct dpu_rm_hw_blk - hardware block tracking list member |
| 58 | * @list: List head for list of all hardware blocks tracking items | 40 | * @list: List head for list of all hardware blocks tracking items |
| 59 | * @rsvp: Pointer to use case reservation if reserved by a client | ||
| 60 | * @rsvp_nxt: Temporary pointer used during reservation to the incoming | ||
| 61 | * request. Will be swapped into rsvp if proposal is accepted | ||
| 62 | * @type: Type of hardware block this structure tracks | ||
| 63 | * @id: Hardware ID number, within it's own space, ie. LM_X | 41 | * @id: Hardware ID number, within it's own space, ie. LM_X |
| 64 | * @catalog: Pointer to the hardware catalog entry for this block | 42 | * @enc_id: Encoder id to which this blk is binded |
| 65 | * @hw: Pointer to the hardware register access object for this block | 43 | * @hw: Pointer to the hardware register access object for this block |
| 66 | */ | 44 | */ |
| 67 | struct dpu_rm_hw_blk { | 45 | struct dpu_rm_hw_blk { |
| 68 | struct list_head list; | 46 | struct list_head list; |
| 69 | struct dpu_rm_rsvp *rsvp; | ||
| 70 | struct dpu_rm_rsvp *rsvp_nxt; | ||
| 71 | enum dpu_hw_blk_type type; | ||
| 72 | uint32_t id; | 47 | uint32_t id; |
| 48 | uint32_t enc_id; | ||
| 73 | struct dpu_hw_blk *hw; | 49 | struct dpu_hw_blk *hw; |
| 74 | }; | 50 | }; |
| 75 | 51 | ||
| 76 | /** | ||
| 77 | * dpu_rm_dbg_rsvp_stage - enum of steps in making reservation for event logging | ||
| 78 | */ | ||
| 79 | enum dpu_rm_dbg_rsvp_stage { | ||
| 80 | DPU_RM_STAGE_BEGIN, | ||
| 81 | DPU_RM_STAGE_AFTER_CLEAR, | ||
| 82 | DPU_RM_STAGE_AFTER_RSVPNEXT, | ||
| 83 | DPU_RM_STAGE_FINAL | ||
| 84 | }; | ||
| 85 | |||
| 86 | static void _dpu_rm_print_rsvps( | ||
| 87 | struct dpu_rm *rm, | ||
| 88 | enum dpu_rm_dbg_rsvp_stage stage) | ||
| 89 | { | ||
| 90 | struct dpu_rm_rsvp *rsvp; | ||
| 91 | struct dpu_rm_hw_blk *blk; | ||
| 92 | enum dpu_hw_blk_type type; | ||
| 93 | |||
| 94 | DPU_DEBUG("%d\n", stage); | ||
| 95 | |||
| 96 | list_for_each_entry(rsvp, &rm->rsvps, list) { | ||
| 97 | DRM_DEBUG_KMS("%d rsvp[s%ue%u]\n", stage, rsvp->seq, | ||
| 98 | rsvp->enc_id); | ||
| 99 | } | ||
| 100 | |||
| 101 | for (type = 0; type < DPU_HW_BLK_MAX; type++) { | ||
| 102 | list_for_each_entry(blk, &rm->hw_blks[type], list) { | ||
| 103 | if (!blk->rsvp && !blk->rsvp_nxt) | ||
| 104 | continue; | ||
| 105 | |||
| 106 | DRM_DEBUG_KMS("%d rsvp[s%ue%u->s%ue%u] %d %d\n", stage, | ||
| 107 | (blk->rsvp) ? blk->rsvp->seq : 0, | ||
| 108 | (blk->rsvp) ? blk->rsvp->enc_id : 0, | ||
| 109 | (blk->rsvp_nxt) ? blk->rsvp_nxt->seq : 0, | ||
| 110 | (blk->rsvp_nxt) ? blk->rsvp_nxt->enc_id : 0, | ||
| 111 | blk->type, blk->id); | ||
| 112 | } | ||
| 113 | } | ||
| 114 | } | ||
| 115 | |||
| 116 | struct dpu_hw_mdp *dpu_rm_get_mdp(struct dpu_rm *rm) | ||
| 117 | { | ||
| 118 | return rm->hw_mdp; | ||
| 119 | } | ||
| 120 | |||
| 121 | void dpu_rm_init_hw_iter( | 52 | void dpu_rm_init_hw_iter( |
| 122 | struct dpu_rm_hw_iter *iter, | 53 | struct dpu_rm_hw_iter *iter, |
| 123 | uint32_t enc_id, | 54 | uint32_t enc_id, |
| @@ -148,15 +79,7 @@ static bool _dpu_rm_get_hw_locked(struct dpu_rm *rm, struct dpu_rm_hw_iter *i) · 24 unmodified lines | |||
| 148 | i->blk = list_prepare_entry(i->blk, blk_list, list); | 79 | i->blk = list_prepare_entry(i->blk, blk_list, list); |
| 149 | 80 | ||
| 150 | list_for_each_entry_continue(i->blk, blk_list, list) { | 81 | list_for_each_entry_continue(i->blk, blk_list, list) { |
| 151 | struct dpu_rm_rsvp *rsvp = i->blk->rsvp; | 82 | if (i->enc_id == i->blk->enc_id) { |
| 152 | |||
| 153 | if (i->blk->type != i->type) { | ||
| 154 | DPU_ERROR("found incorrect block type %d on %d list\n", | ||
| 155 | i->blk->type, i->type); | ||
| 156 | return false; | ||
| 157 | } | ||
| 158 | |||
| 159 | if ((i->enc_id == 0) || (rsvp && rsvp->enc_id == i->enc_id)) { | ||
| 160 | i->hw = i->blk->hw; | 83 | i->hw = i->blk->hw; |
| 161 | DPU_DEBUG("found type %d id %d for enc %d\n", | 84 | DPU_DEBUG("found type %d id %d for enc %d\n", |
| 162 | i->type, i->blk->id, i->enc_id); | 85 | i->type, i->blk->id, i->enc_id); |
| @@ -208,34 +131,18 @@ static void _dpu_rm_hw_destroy(enum dpu_hw_blk_type type, void *hw) · 45 unmodified lines | |||
| 208 | 131 | ||
| 209 | int dpu_rm_destroy(struct dpu_rm *rm) | 132 | int dpu_rm_destroy(struct dpu_rm *rm) |
| 210 | { | 133 | { |
| 211 | |||
| 212 | struct dpu_rm_rsvp *rsvp_cur, *rsvp_nxt; | ||
| 213 | struct dpu_rm_hw_blk *hw_cur, *hw_nxt; | 134 | struct dpu_rm_hw_blk *hw_cur, *hw_nxt; |
| 214 | enum dpu_hw_blk_type type; | 135 | enum dpu_hw_blk_type type; |
| 215 | 136 | ||
| 216 | if (!rm) { | ||
| 217 | DPU_ERROR("invalid rm\n"); | ||
| 218 | return -EINVAL; | ||
| 219 | } | ||
| 220 | |||
| 221 | list_for_each_entry_safe(rsvp_cur, rsvp_nxt, &rm->rsvps, list) { | ||
| 222 | list_del(&rsvp_cur->list); | ||
| 223 | kfree(rsvp_cur); | ||
| 224 | } | ||
| 225 | |||
| 226 | |||
| 227 | for (type = 0; type < DPU_HW_BLK_MAX; type++) { | 137 | for (type = 0; type < DPU_HW_BLK_MAX; type++) { |
| 228 | list_for_each_entry_safe(hw_cur, hw_nxt, &rm->hw_blks[type], | 138 | list_for_each_entry_safe(hw_cur, hw_nxt, &rm->hw_blks[type], |
| 229 | list) { | 139 | list) { |
| 230 | list_del(&hw_cur->list); | 140 | list_del(&hw_cur->list); |
| 231 | _dpu_rm_hw_destroy(hw_cur->type, hw_cur->hw); | 141 | _dpu_rm_hw_destroy(type, hw_cur->hw); |
| 232 | kfree(hw_cur); | 142 | kfree(hw_cur); |
| 233 | } | 143 | } |
| 234 | } | 144 | } |
| 235 | 145 | ||
| 236 | dpu_hw_mdp_destroy(rm->hw_mdp); | ||
| 237 | rm->hw_mdp = NULL; | ||
| 238 | |||
| 239 | mutex_destroy(&rm->rm_lock); | 146 | mutex_destroy(&rm->rm_lock); |
| 240 | 147 | ||
| 241 | return 0; | 148 | return 0; |
| @@ -250,11 +157,8 @@ static int _dpu_rm_hw_blk_create( · 8 unmodified lines | |||
| 250 | void *hw_catalog_info) | 157 | void *hw_catalog_info) |
| 251 | { | 158 | { |
| 252 | struct dpu_rm_hw_blk *blk; | 159 | struct dpu_rm_hw_blk *blk; |
| 253 | struct dpu_hw_mdp *hw_mdp; | ||
| 254 | void *hw; | 160 | void *hw; |
| 255 | 161 | ||
| 256 | hw_mdp = rm->hw_mdp; | ||
| 257 | |||
| 258 | switch (type) { | 162 | switch (type) { |
| 259 | case DPU_HW_BLK_LM: | 163 | case DPU_HW_BLK_LM: |
| 260 | hw = dpu_hw_lm_init(id, mmio, cat); | 164 | hw = dpu_hw_lm_init(id, mmio, cat); |
| @@ -290,9 +194,9 @@ static int _dpu_rm_hw_blk_create( · 29 unmodified lines | |||
| 290 | return -ENOMEM; | 194 | return -ENOMEM; |
| 291 | } | 195 | } |
| 292 | 196 | ||
| 293 | blk->type = type; | ||
| 294 | blk->id = id; | 197 | blk->id = id; |
| 295 | blk->hw = hw; | 198 | blk->hw = hw; |
| 199 | blk->enc_id = 0; | ||
| 296 | list_add_tail(&blk->list, &rm->hw_blks[type]); | 200 | list_add_tail(&blk->list, &rm->hw_blks[type]); |
| 297 | 201 | ||
| 298 | return 0; | 202 | return 0; |
| @@ -300,13 +204,12 @@ static int _dpu_rm_hw_blk_create( · 1 unmodified line | |||
| 300 | 204 | ||
| 301 | int dpu_rm_init(struct dpu_rm *rm, | 205 | int dpu_rm_init(struct dpu_rm *rm, |
| 302 | struct dpu_mdss_cfg *cat, | 206 | struct dpu_mdss_cfg *cat, |
| 303 | void __iomem *mmio, | 207 | void __iomem *mmio) |
| 304 | struct drm_device *dev) | ||
| 305 | { | 208 | { |
| 306 | int rc, i; | 209 | int rc, i; |
| 307 | enum dpu_hw_blk_type type; | 210 | enum dpu_hw_blk_type type; |
| 308 | 211 | ||
| 309 | if (!rm || !cat || !mmio || !dev) { | 212 | if (!rm || !cat || !mmio) { |
| 310 | DPU_ERROR("invalid kms\n"); | 213 | DPU_ERROR("invalid kms\n"); |
| 311 | return -EINVAL; | 214 | return -EINVAL; |
| 312 | } | 215 | } |
| @@ -316,21 +219,9 @@ int dpu_rm_init(struct dpu_rm *rm, · 3 unmodified lines | |||
| 316 | 219 | ||
| 317 | mutex_init(&rm->rm_lock); | 220 | mutex_init(&rm->rm_lock); |
| 318 | 221 | ||
| 319 | INIT_LIST_HEAD(&rm->rsvps); | ||
| 320 | for (type = 0; type < DPU_HW_BLK_MAX; type++) | 222 | for (type = 0; type < DPU_HW_BLK_MAX; type++) |
| 321 | INIT_LIST_HEAD(&rm->hw_blks[type]); | 223 | INIT_LIST_HEAD(&rm->hw_blks[type]); |
| 322 | 224 | ||
| 323 | rm->dev = dev; | ||
| 324 | |||
| 325 | /* Some of the sub-blocks require an mdptop to be created */ | ||
| 326 | rm->hw_mdp = dpu_hw_mdptop_init(MDP_TOP, mmio, cat); | ||
| 327 | if (IS_ERR_OR_NULL(rm->hw_mdp)) { | ||
| 328 | rc = PTR_ERR(rm->hw_mdp); | ||
| 329 | rm->hw_mdp = NULL; | ||
| 330 | DPU_ERROR("failed: mdp hw not available\n"); | ||
| 331 | goto fail; | ||
| 332 | } | ||
| 333 | |||
| 334 | /* Interrogate HW catalog and create tracking items for hw blocks */ | 225 | /* Interrogate HW catalog and create tracking items for hw blocks */ |
| 335 | for (i = 0; i < cat->mixer_count; i++) { | 226 | for (i = 0; i < cat->mixer_count; i++) { |
| 336 | struct dpu_lm_cfg *lm = &cat->mixer[i]; | 227 | struct dpu_lm_cfg *lm = &cat->mixer[i]; |
| @@ -410,7 +301,7 @@ static bool _dpu_rm_needs_split_display(const struct msm_display_topology *top) · 73 unmodified lines | |||
| 410 | * proposed use case requirements, incl. hardwired dependent blocks like | 301 | * proposed use case requirements, incl. hardwired dependent blocks like |
| 411 | * pingpong | 302 | * pingpong |
| 412 | * @rm: dpu resource manager handle | 303 | * @rm: dpu resource manager handle |
| 413 | * @rsvp: reservation currently being created | 304 | * @enc_id: encoder id requesting for allocation |
| 414 | * @reqs: proposed use case requirements | 305 | * @reqs: proposed use case requirements |
| 415 | * @lm: proposed layer mixer, function checks if lm, and all other hardwired | 306 | * @lm: proposed layer mixer, function checks if lm, and all other hardwired |
| 416 | * blocks connected to the lm (pp) is available and appropriate | 307 | * blocks connected to the lm (pp) is available and appropriate |
| @@ -422,7 +313,7 @@ static bool _dpu_rm_needs_split_display(const struct msm_display_topology *top) · 5 unmodified lines | |||
| 422 | */ | 313 | */ |
| 423 | static bool _dpu_rm_check_lm_and_get_connected_blks( | 314 | static bool _dpu_rm_check_lm_and_get_connected_blks( |
| 424 | struct dpu_rm *rm, | 315 | struct dpu_rm *rm, |
| 425 | struct dpu_rm_rsvp *rsvp, | 316 | uint32_t enc_id, |
| 426 | struct dpu_rm_requirements *reqs, | 317 | struct dpu_rm_requirements *reqs, |
| 427 | struct dpu_rm_hw_blk *lm, | 318 | struct dpu_rm_hw_blk *lm, |
| 428 | struct dpu_rm_hw_blk **pp, | 319 | struct dpu_rm_hw_blk **pp, |
| @@ -449,7 +340,7 @@ static bool _dpu_rm_check_lm_and_get_connected_blks( · 20 unmodified lines | |||
| 449 | } | 340 | } |
| 450 | 341 | ||
| 451 | /* Already reserved? */ | 342 | /* Already reserved? */ |
| 452 | if (RESERVED_BY_OTHER(lm, rsvp)) { | 343 | if (RESERVED_BY_OTHER(lm, enc_id)) { |
| 453 | DPU_DEBUG("lm %d already reserved\n", lm_cfg->id); | 344 | DPU_DEBUG("lm %d already reserved\n", lm_cfg->id); |
| 454 | return false; | 345 | return false; |
| 455 | } | 346 | } |
| @@ -467,7 +358,7 @@ static bool _dpu_rm_check_lm_and_get_connected_blks( · 11 unmodified lines | |||
| 467 | return false; | 358 | return false; |
| 468 | } | 359 | } |
| 469 | 360 | ||
| 470 | if (RESERVED_BY_OTHER(*pp, rsvp)) { | 361 | if (RESERVED_BY_OTHER(*pp, enc_id)) { |
| 471 | DPU_DEBUG("lm %d pp %d already reserved\n", lm->id, | 362 | DPU_DEBUG("lm %d pp %d already reserved\n", lm->id, |
| 472 | (*pp)->id); | 363 | (*pp)->id); |
| 473 | return false; | 364 | return false; |
| @@ -476,10 +367,8 @@ static bool _dpu_rm_check_lm_and_get_connected_blks( · 2 unmodified lines | |||
| 476 | return true; | 367 | return true; |
| 477 | } | 368 | } |
| 478 | 369 | ||
| 479 | static int _dpu_rm_reserve_lms( | 370 | static int _dpu_rm_reserve_lms(struct dpu_rm *rm, uint32_t enc_id, |
| 480 | struct dpu_rm *rm, | 371 | struct dpu_rm_requirements *reqs) |
| 481 | struct dpu_rm_rsvp *rsvp, | ||
| 482 | struct dpu_rm_requirements *reqs) | ||
| 483 | 372 | ||
| 484 | { | 373 | { |
| 485 | struct dpu_rm_hw_blk *lm[MAX_BLOCKS]; | 374 | struct dpu_rm_hw_blk *lm[MAX_BLOCKS]; |
| @@ -504,7 +393,7 @@ static int _dpu_rm_reserve_lms( · 18 unmodified lines | |||
| 504 | lm[lm_count] = iter_i.blk; | 393 | lm[lm_count] = iter_i.blk; |
| 505 | 394 | ||
| 506 | if (!_dpu_rm_check_lm_and_get_connected_blks( | 395 | if (!_dpu_rm_check_lm_and_get_connected_blks( |
| 507 | rm, rsvp, reqs, lm[lm_count], | 396 | rm, enc_id, reqs, lm[lm_count], |
| 508 | &pp[lm_count], NULL)) | 397 | &pp[lm_count], NULL)) |
| 509 | continue; | 398 | continue; |
| 510 | 399 | ||
| @@ -519,7 +408,7 @@ static int _dpu_rm_reserve_lms( · 8 unmodified lines | |||
| 519 | continue; | 408 | continue; |
| 520 | 409 | ||
| 521 | if (!_dpu_rm_check_lm_and_get_connected_blks( | 410 | if (!_dpu_rm_check_lm_and_get_connected_blks( |
| 522 | rm, rsvp, reqs, iter_j.blk, | 411 | rm, enc_id, reqs, iter_j.blk, |
| 523 | &pp[lm_count], iter_i.blk)) | 412 | &pp[lm_count], iter_i.blk)) |
| 524 | continue; | 413 | continue; |
| 525 | 414 | ||
| @@ -537,11 +426,10 @@ static int _dpu_rm_reserve_lms( · 11 unmodified lines | |||
| 537 | if (!lm[i]) | 426 | if (!lm[i]) |
| 538 | break; | 427 | break; |
| 539 | 428 | ||
| 540 | lm[i]->rsvp_nxt = rsvp; | 429 | lm[i]->enc_id = enc_id; |
| 541 | pp[i]->rsvp_nxt = rsvp; | 430 | pp[i]->enc_id = enc_id; |
| 542 | 431 | ||
| 543 | trace_dpu_rm_reserve_lms(lm[i]->id, lm[i]->type, rsvp->enc_id, | 432 | trace_dpu_rm_reserve_lms(lm[i]->id, enc_id, pp[i]->id); |
| 544 | pp[i]->id); | ||
| 545 | } | 433 | } |
| 546 | 434 | ||
| 547 | return rc; | 435 | return rc; |
| @@ -549,7 +437,7 @@ static int _dpu_rm_reserve_lms( · 1 unmodified line | |||
| 549 | 437 | ||
| 550 | static int _dpu_rm_reserve_ctls( | 438 | static int _dpu_rm_reserve_ctls( |
| 551 | struct dpu_rm *rm, | 439 | struct dpu_rm *rm, |
| 552 | struct dpu_rm_rsvp *rsvp, | 440 | uint32_t enc_id, |
| 553 | const struct msm_display_topology *top) | 441 | const struct msm_display_topology *top) |
| 554 | { | 442 | { |
| 555 | struct dpu_rm_hw_blk *ctls[MAX_BLOCKS]; | 443 | struct dpu_rm_hw_blk *ctls[MAX_BLOCKS]; |
| @@ -570,7 +458,7 @@ static int _dpu_rm_reserve_ctls( · 14 unmodified lines | |||
| 570 | unsigned long features = ctl->caps->features; | 458 | unsigned long features = ctl->caps->features; |
| 571 | bool has_split_display; | 459 | bool has_split_display; |
| 572 | 460 | ||
| 573 | if (RESERVED_BY_OTHER(iter.blk, rsvp)) | 461 | if (RESERVED_BY_OTHER(iter.blk, enc_id)) |
| 574 | continue; | 462 | continue; |
| 575 | 463 | ||
| 576 | has_split_display = BIT(DPU_CTL_SPLIT_DISPLAY) & features; | 464 | has_split_display = BIT(DPU_CTL_SPLIT_DISPLAY) & features; |
| @@ -591,9 +479,8 @@ static int _dpu_rm_reserve_ctls( · 14 unmodified lines | |||
| 591 | return -ENAVAIL; | 479 | return -ENAVAIL; |
| 592 | 480 | ||
| 593 | for (i = 0; i < ARRAY_SIZE(ctls) && i < num_ctls; i++) { | 481 | for (i = 0; i < ARRAY_SIZE(ctls) && i < num_ctls; i++) { |
| 594 | ctls[i]->rsvp_nxt = rsvp; | 482 | ctls[i]->enc_id = enc_id; |
| 595 | trace_dpu_rm_reserve_ctls(ctls[i]->id, ctls[i]->type, | 483 | trace_dpu_rm_reserve_ctls(ctls[i]->id, enc_id); |
| 596 | rsvp->enc_id); | ||
| 597 | } | 484 | } |
| 598 | 485 | ||
| 599 | return 0; | 486 | return 0; |
| @@ -601,7 +488,7 @@ static int _dpu_rm_reserve_ctls( · 1 unmodified line | |||
| 601 | 488 | ||
| 602 | static int _dpu_rm_reserve_intf( | 489 | static int _dpu_rm_reserve_intf( |
| 603 | struct dpu_rm *rm, | 490 | struct dpu_rm *rm, |
| 604 | struct dpu_rm_rsvp *rsvp, | 491 | uint32_t enc_id, |
| 605 | uint32_t id, | 492 | uint32_t id, |
| 606 | enum dpu_hw_blk_type type) | 493 | enum dpu_hw_blk_type type) |
| 607 | { | 494 | { |
| @@ -614,14 +501,13 @@ static int _dpu_rm_reserve_intf( · 6 unmodified lines | |||
| 614 | if (iter.blk->id != id) | 501 | if (iter.blk->id != id) |
| 615 | continue; | 502 | continue; |
| 616 | 503 | ||
| 617 | if (RESERVED_BY_OTHER(iter.blk, rsvp)) { | 504 | if (RESERVED_BY_OTHER(iter.blk, enc_id)) { |
| 618 | DPU_ERROR("type %d id %d already reserved\n", type, id); | 505 | DPU_ERROR("type %d id %d already reserved\n", type, id); |
| 619 | return -ENAVAIL; | 506 | return -ENAVAIL; |
| 620 | } | 507 | } |
| 621 | 508 | ||
| 622 | iter.blk->rsvp_nxt = rsvp; | 509 | iter.blk->enc_id = enc_id; |
| 623 | trace_dpu_rm_reserve_intf(iter.blk->id, iter.blk->type, | 510 | trace_dpu_rm_reserve_intf(iter.blk->id, enc_id); |
| 624 | rsvp->enc_id); | ||
| 625 | break; | 511 | break; |
| 626 | } | 512 | } |
| 627 | 513 | ||
| @@ -636,7 +522,7 @@ static int _dpu_rm_reserve_intf( · 8 unmodified lines | |||
| 636 | 522 | ||
| 637 | static int _dpu_rm_reserve_intf_related_hw( | 523 | static int _dpu_rm_reserve_intf_related_hw( |
| 638 | struct dpu_rm *rm, | 524 | struct dpu_rm *rm, |
| 639 | struct dpu_rm_rsvp *rsvp, | 525 | uint32_t enc_id, |
| 640 | struct dpu_encoder_hw_resources *hw_res) | 526 | struct dpu_encoder_hw_resources *hw_res) |
| 641 | { | 527 | { |
| 642 | int i, ret = 0; | 528 | int i, ret = 0; |
| @@ -646,7 +532,7 @@ static int _dpu_rm_reserve_intf_related_hw( · 3 unmodified lines | |||
| 646 | if (hw_res->intfs[i] == INTF_MODE_NONE) | 532 | if (hw_res->intfs[i] == INTF_MODE_NONE) |
| 647 | continue; | 533 | continue; |
| 648 | id = i + INTF_0; | 534 | id = i + INTF_0; |
| 649 | ret = _dpu_rm_reserve_intf(rm, rsvp, id, | 535 | ret = _dpu_rm_reserve_intf(rm, enc_id, id, |
| 650 | DPU_HW_BLK_INTF); | 536 | DPU_HW_BLK_INTF); |
| 651 | if (ret) | 537 | if (ret) |
| 652 | return ret; | 538 | return ret; |
| @@ -655,33 +541,27 @@ static int _dpu_rm_reserve_intf_related_hw( · 2 unmodified lines | |||
| 655 | return ret; | 541 | return ret; |
| 656 | } | 542 | } |
| 657 | 543 | ||
| 658 | static int _dpu_rm_make_next_rsvp( | 544 | static int _dpu_rm_make_reservation( |
| 659 | struct dpu_rm *rm, | 545 | struct dpu_rm *rm, |
| 660 | struct drm_encoder *enc, | 546 | struct drm_encoder *enc, |
| 661 | struct drm_crtc_state *crtc_state, | 547 | struct drm_crtc_state *crtc_state, |
| 662 | struct dpu_rm_rsvp *rsvp, | ||
| 663 | struct dpu_rm_requirements *reqs) | 548 | struct dpu_rm_requirements *reqs) |
| 664 | { | 549 | { |
| 665 | int ret; | 550 | int ret; |
| 666 | 551 | ||
| 667 | /* Create reservation info, tag reserved blocks with it as we go */ | 552 | ret = _dpu_rm_reserve_lms(rm, enc->base.id, reqs); |
| 668 | rsvp->seq = ++rm->rsvp_next_seq; | ||
| 669 | rsvp->enc_id = enc->base.id; | ||
| 670 | list_add_tail(&rsvp->list, &rm->rsvps); | ||
| 671 | |||
| 672 | ret = _dpu_rm_reserve_lms(rm, rsvp, reqs); | ||
| 673 | if (ret) { | 553 | if (ret) { |
| 674 | DPU_ERROR("unable to find appropriate mixers\n"); | 554 | DPU_ERROR("unable to find appropriate mixers\n"); |
| 675 | return ret; | 555 | return ret; |
| 676 | } | 556 | } |
| 677 | 557 | ||
| 678 | ret = _dpu_rm_reserve_ctls(rm, rsvp, &reqs->topology); | 558 | ret = _dpu_rm_reserve_ctls(rm, enc->base.id, &reqs->topology); |
| 679 | if (ret) { | 559 | if (ret) { |
| 680 | DPU_ERROR("unable to find appropriate CTL\n"); | 560 | DPU_ERROR("unable to find appropriate CTL\n"); |
| 681 | return ret; | 561 | return ret; |
| 682 | } | 562 | } |
| 683 | 563 | ||
| 684 | ret = _dpu_rm_reserve_intf_related_hw(rm, rsvp, &reqs->hw_res); | 564 | ret = _dpu_rm_reserve_intf_related_hw(rm, enc->base.id, &reqs->hw_res); |
| 685 | if (ret) | 565 | if (ret) |
| 686 | return ret; | 566 | return ret; |
| 687 | 567 | ||
| @@ -706,108 +586,31 @@ static int _dpu_rm_populate_requirements( · 18 unmodified lines | |||
| 706 | return 0; | 586 | return 0; |
| 707 | } | 587 | } |
| 708 | 588 | ||
| 709 | static struct dpu_rm_rsvp *_dpu_rm_get_rsvp( | 589 | static void _dpu_rm_release_reservation(struct dpu_rm *rm, uint32_t enc_id) |
| 710 | struct dpu_rm *rm, | ||
| 711 | struct drm_encoder *enc) | ||
| 712 | { | 590 | { |
| 713 | struct dpu_rm_rsvp *i; | ||
| 714 | |||
| 715 | if (!rm || !enc) { | ||
| 716 | DPU_ERROR("invalid params\n"); | ||
| 717 | return NULL; | ||
| 718 | } | ||
| 719 | |||
| 720 | if (list_empty(&rm->rsvps)) | ||
| 721 | return NULL; | ||
| 722 | |||
| 723 | list_for_each_entry(i, &rm->rsvps, list) | ||
| 724 | if (i->enc_id == enc->base.id) | ||
| 725 | return i; | ||
| 726 | |||
| 727 | return NULL; | ||
| 728 | } | ||
| 729 | |||
| 730 | /** | ||
| 731 | * _dpu_rm_release_rsvp - release resources and release a reservation | ||
| 732 | * @rm: KMS handle | ||
| 733 | * @rsvp: RSVP pointer to release and release resources for | ||
| 734 | */ | ||
| 735 | static void _dpu_rm_release_rsvp(struct dpu_rm *rm, struct dpu_rm_rsvp *rsvp) | ||
| 736 | { | ||
| 737 | struct dpu_rm_rsvp *rsvp_c, *rsvp_n; | ||
| 738 | struct dpu_rm_hw_blk *blk; | 591 | struct dpu_rm_hw_blk *blk; |
| 739 | enum dpu_hw_blk_type type; | 592 | enum dpu_hw_blk_type type; |
| 740 | 593 | ||
| 741 | if (!rsvp) | ||
| 742 | return; | ||
| 743 | |||
| 744 | DPU_DEBUG("rel rsvp %d enc %d\n", rsvp->seq, rsvp->enc_id); | ||
| 745 | |||
| 746 | list_for_each_entry_safe(rsvp_c, rsvp_n, &rm->rsvps, list) { | ||
| 747 | if (rsvp == rsvp_c) { | ||
| 748 | list_del(&rsvp_c->list); | ||
| 749 | break; | ||
| 750 | } | ||
| 751 | } | ||
| 752 | |||
| 753 | for (type = 0; type < DPU_HW_BLK_MAX; type++) { | 594 | for (type = 0; type < DPU_HW_BLK_MAX; type++) { |
| 754 | list_for_each_entry(blk, &rm->hw_blks[type], list) { | 595 | list_for_each_entry(blk, &rm->hw_blks[type], list) { |
| 755 | if (blk->rsvp == rsvp) { | 596 | if (blk->enc_id == enc_id) { |
| 756 | blk->rsvp = NULL; | 597 | blk->enc_id = 0; |
| 757 | DPU_DEBUG("rel rsvp %d enc %d %d %d\n", | 598 | DPU_DEBUG("rel enc %d %d %d\n", enc_id, |
| 758 | rsvp->seq, rsvp->enc_id, | 599 | type, blk->id); |
| 759 | blk->type, blk->id); | ||
| 760 | } | ||
| 761 | if (blk->rsvp_nxt == rsvp) { | ||
| 762 | blk->rsvp_nxt = NULL; | ||
| 763 | DPU_DEBUG("rel rsvp_nxt %d enc %d %d %d\n", | ||
| 764 | rsvp->seq, rsvp->enc_id, | ||
| 765 | blk->type, blk->id); | ||
| 766 | } | 600 | } |
| 767 | } | 601 | } |
| 768 | } | 602 | } |
| 769 | |||
| 770 | kfree(rsvp); | ||
| 771 | } | 603 | } |
| 772 | 604 | ||
| 773 | void dpu_rm_release(struct dpu_rm *rm, struct drm_encoder *enc) | 605 | void dpu_rm_release(struct dpu_rm *rm, struct drm_encoder *enc) |
| 774 | { | 606 | { |
| 775 | struct dpu_rm_rsvp *rsvp; | ||
| 776 | |||
| 777 | if (!rm || !enc) { | ||
| 778 | DPU_ERROR("invalid params\n"); | ||
| 779 | return; | ||
| 780 | } | ||
| 781 | |||
| 782 | mutex_lock(&rm->rm_lock); | 607 | mutex_lock(&rm->rm_lock); |
| 783 | 608 | ||
| 784 | rsvp = _dpu_rm_get_rsvp(rm, enc); | 609 | _dpu_rm_release_reservation(rm, enc->base.id); |
| 785 | if (!rsvp) { | ||
| 786 | DPU_ERROR("failed to find rsvp for enc %d\n", enc->base.id); | ||
| 787 | goto end; | ||
| 788 | } | ||
| 789 | 610 | ||
| 790 | _dpu_rm_release_rsvp(rm, rsvp); | ||
| 791 | end: | ||
| 792 | mutex_unlock(&rm->rm_lock); | 611 | mutex_unlock(&rm->rm_lock); |
| 793 | } | 612 | } |
| 794 | 613 | ||
| 795 | static void _dpu_rm_commit_rsvp(struct dpu_rm *rm, struct dpu_rm_rsvp *rsvp) | ||
| 796 | { | ||
| 797 | struct dpu_rm_hw_blk *blk; | ||
| 798 | enum dpu_hw_blk_type type; | ||
| 799 | |||
| 800 | /* Swap next rsvp to be the active */ | ||
| 801 | for (type = 0; type < DPU_HW_BLK_MAX; type++) { | ||
| 802 | list_for_each_entry(blk, &rm->hw_blks[type], list) { | ||
| 803 | if (blk->rsvp_nxt) { | ||
| 804 | blk->rsvp = blk->rsvp_nxt; | ||
| 805 | blk->rsvp_nxt = NULL; | ||
| 806 | } | ||
| 807 | } | ||
| 808 | } | ||
| 809 | } | ||
| 810 | |||
| 811 | int dpu_rm_reserve( | 614 | int dpu_rm_reserve( |
| 812 | struct dpu_rm *rm, | 615 | struct dpu_rm *rm, |
| 813 | struct drm_encoder *enc, | 616 | struct drm_encoder *enc, |
| @@ -815,7 +618,6 @@ int dpu_rm_reserve( · 1 unmodified line | |||
| 815 | struct msm_display_topology topology, | 618 | struct msm_display_topology topology, |
| 816 | bool test_only) | 619 | bool test_only) |
| 817 | { | 620 | { |
| 818 | struct dpu_rm_rsvp *rsvp_cur, *rsvp_nxt; | ||
| 819 | struct dpu_rm_requirements reqs; | 621 | struct dpu_rm_requirements reqs; |
| 820 | int ret; | 622 | int ret; |
| 821 | 623 | ||
| @@ -828,8 +630,6 @@ int dpu_rm_reserve( · 6 unmodified lines | |||
| 828 | 630 | ||
| 829 | mutex_lock(&rm->rm_lock); | 631 | mutex_lock(&rm->rm_lock); |
| 830 | 632 | ||
| 831 | _dpu_rm_print_rsvps(rm, DPU_RM_STAGE_BEGIN); | ||
| 832 | |||
| 833 | ret = _dpu_rm_populate_requirements(rm, enc, crtc_state, &reqs, | 633 | ret = _dpu_rm_populate_requirements(rm, enc, crtc_state, &reqs, |
| 834 | topology); | 634 | topology); |
| 835 | if (ret) { | 635 | if (ret) { |
| @@ -837,50 +637,17 @@ int dpu_rm_reserve( · 1 unmodified line | |||
| 837 | goto end; | 637 | goto end; |
| 838 | } | 638 | } |
| 839 | 639 | ||
| 840 | /* | 640 | ret = _dpu_rm_make_reservation(rm, enc, crtc_state, &reqs); |
| 841 | * We only support one active reservation per-hw-block. But to implement | ||
| 842 | * transactional semantics for test-only, and for allowing failure while | ||
| 843 | * modifying your existing reservation, over the course of this | ||
| 844 | * function we can have two reservations: | ||
| 845 | * Current: Existing reservation | ||
| 846 | * Next: Proposed reservation. The proposed reservation may fail, or may | ||
| 847 | * be discarded if in test-only mode. | ||
| 848 | * If reservation is successful, and we're not in test-only, then we | ||
| 849 | * replace the current with the next. | ||
| 850 | */ | ||
| 851 | rsvp_nxt = kzalloc(sizeof(*rsvp_nxt), GFP_KERNEL); | ||
| 852 | if (!rsvp_nxt) { | ||
| 853 | ret = -ENOMEM; | ||
| 854 | goto end; | ||
| 855 | } | ||
| 856 | |||
| 857 | rsvp_cur = _dpu_rm_get_rsvp(rm, enc); | ||
| 858 | |||
| 859 | /* Check the proposed reservation, store it in hw's "next" field */ | ||
| 860 | ret = _dpu_rm_make_next_rsvp(rm, enc, crtc_state, rsvp_nxt, &reqs); | ||
| 861 | |||
| 862 | _dpu_rm_print_rsvps(rm, DPU_RM_STAGE_AFTER_RSVPNEXT); | ||
| 863 | |||
| 864 | if (ret) { | 641 | if (ret) { |
| 865 | DPU_ERROR("failed to reserve hw resources: %d\n", ret); | 642 | DPU_ERROR("failed to reserve hw resources: %d\n", ret); |
| 866 | _dpu_rm_release_rsvp(rm, rsvp_nxt); | 643 | _dpu_rm_release_reservation(rm, enc->base.id); |
| 867 | } else if (test_only) { | 644 | } else if (test_only) { |
| 868 | /* | 645 | /* test_only: test the reservation and then undo */ |
| 869 | * Normally, if test_only, test the reservation and then undo | 646 | DPU_DEBUG("test_only: discard test [enc: %d]\n", |
| 870 | * However, if the user requests LOCK, then keep the reservation | 647 | enc->base.id); |
| 871 | * made during the atomic_check phase. | 648 | _dpu_rm_release_reservation(rm, enc->base.id); |
| 872 | */ | ||
| 873 | DPU_DEBUG("test_only: discard test rsvp[s%de%d]\n", | ||
| 874 | rsvp_nxt->seq, rsvp_nxt->enc_id); | ||
| 875 | _dpu_rm_release_rsvp(rm, rsvp_nxt); | ||
| 876 | } else { | ||
| 877 | _dpu_rm_release_rsvp(rm, rsvp_cur); | ||
| 878 | |||
| 879 | _dpu_rm_commit_rsvp(rm, rsvp_nxt); | ||
| 880 | } | 649 | } |
| 881 | 650 | ||
| 882 | _dpu_rm_print_rsvps(rm, DPU_RM_STAGE_FINAL); | ||
| 883 | |||
| 884 | end: | 651 | end: |
| 885 | mutex_unlock(&rm->rm_lock); | 652 | mutex_unlock(&rm->rm_lock); |
| 886 | 653 | ||
| @@ -22,22 +22,14 @@ · 21 unmodified lines | |||
| 22 | 22 | ||
| 23 | /** | 23 | /** |
| 24 | * struct dpu_rm - DPU dynamic hardware resource manager | 24 | * struct dpu_rm - DPU dynamic hardware resource manager |
| 25 | * @dev: device handle for event logging purposes | ||
| 26 | * @rsvps: list of hardware reservations by each crtc->encoder->connector | ||
| 27 | * @hw_blks: array of lists of hardware resources present in the system, one | 25 | * @hw_blks: array of lists of hardware resources present in the system, one |
| 28 | * list per type of hardware block | 26 | * list per type of hardware block |
| 29 | * @hw_mdp: hardware object for mdp_top | ||
| 30 | * @lm_max_width: cached layer mixer maximum width | 27 | * @lm_max_width: cached layer mixer maximum width |
| 31 | * @rsvp_next_seq: sequence number for next reservation for debugging purposes | ||
| 32 | * @rm_lock: resource manager mutex | 28 | * @rm_lock: resource manager mutex |
| 33 | */ | 29 | */ |
| 34 | struct dpu_rm { | 30 | struct dpu_rm { |
| 35 | struct drm_device *dev; | ||
| 36 | struct list_head rsvps; | ||
| 37 | struct list_head hw_blks[DPU_HW_BLK_MAX]; | 31 | struct list_head hw_blks[DPU_HW_BLK_MAX]; |
| 38 | struct dpu_hw_mdp *hw_mdp; | ||
| 39 | uint32_t lm_max_width; | 32 | uint32_t lm_max_width; |
| 40 | uint32_t rsvp_next_seq; | ||
| 41 | struct mutex rm_lock; | 33 | struct mutex rm_lock; |
| 42 | }; | 34 | }; |
| 43 | 35 | ||
| @@ -67,13 +59,11 @@ struct dpu_rm_hw_iter { · 23 unmodified lines | |||
| 67 | * @rm: DPU Resource Manager handle | 59 | * @rm: DPU Resource Manager handle |
| 68 | * @cat: Pointer to hardware catalog | 60 | * @cat: Pointer to hardware catalog |
| 69 | * @mmio: mapped register io address of MDP | 61 | * @mmio: mapped register io address of MDP |
| 70 | * @dev: device handle for event logging purposes | ||
| 71 | * @Return: 0 on Success otherwise -ERROR | 62 | * @Return: 0 on Success otherwise -ERROR |
| 72 | */ | 63 | */ |
| 73 | int dpu_rm_init(struct dpu_rm *rm, | 64 | int dpu_rm_init(struct dpu_rm *rm, |
| 74 | struct dpu_mdss_cfg *cat, | 65 | struct dpu_mdss_cfg *cat, |
| 75 | void __iomem *mmio, | 66 | void __iomem *mmio); |
| 76 | struct drm_device *dev); | ||
| 77 | 67 | ||
| 78 | /** | 68 | /** |
| 79 | * dpu_rm_destroy - Free all memory allocated by dpu_rm_init | 69 | * dpu_rm_destroy - Free all memory allocated by dpu_rm_init |
| @@ -111,14 +101,6 @@ int dpu_rm_reserve(struct dpu_rm *rm, · 31 unmodified lines | |||
| 111 | */ | 101 | */ |
| 112 | void dpu_rm_release(struct dpu_rm *rm, struct drm_encoder *enc); | 102 | void dpu_rm_release(struct dpu_rm *rm, struct drm_encoder *enc); |
| 113 | 103 | ||
| 114 | /** | ||
| 115 | * dpu_rm_get_mdp - Retrieve HW block for MDP TOP. | ||
| 116 | * This is never reserved, and is usable by any display. | ||
| 117 | * @rm: DPU Resource Manager handle | ||
| 118 | * @Return: Pointer to hw block or NULL | ||
| 119 | */ | ||
| 120 | struct dpu_hw_mdp *dpu_rm_get_mdp(struct dpu_rm *rm); | ||
| 121 | |||
| 122 | /** | 104 | /** |
| 123 | * dpu_rm_init_hw_iter - setup given iterator for new iteration over hw list | 105 | * dpu_rm_init_hw_iter - setup given iterator for new iteration over hw list |
| 124 | * using dpu_rm_get_hw | 106 | * using dpu_rm_get_hw |
| @@ -144,12 +126,4 @@ void dpu_rm_init_hw_iter( · 19 unmodified lines | |||
| 144 | * @Return: true on match found, false on no match found | 126 | * @Return: true on match found, false on no match found |
| 145 | */ | 127 | */ |
| 146 | bool dpu_rm_get_hw(struct dpu_rm *rm, struct dpu_rm_hw_iter *iter); | 128 | bool dpu_rm_get_hw(struct dpu_rm *rm, struct dpu_rm_hw_iter *iter); |
| 147 | |||
| 148 | /** | ||
| 149 | * dpu_rm_check_property_topctl - validate property bitmask before it is set | ||
| 150 | * @val: user's proposed topology control bitmask | ||
| 151 | * @Return: 0 on success or error | ||
| 152 | */ | ||
| 153 | int dpu_rm_check_property_topctl(uint64_t val); | ||
| 154 | |||
| 155 | #endif /* __DPU_RM_H__ */ | 129 | #endif /* __DPU_RM_H__ */ |
| @@ -831,48 +831,42 @@ TRACE_EVENT(dpu_plane_disable, · 830 unmodified lines | |||
| 831 | ); | 831 | ); |
| 832 | 832 | ||
| 833 | DECLARE_EVENT_CLASS(dpu_rm_iter_template, | 833 | DECLARE_EVENT_CLASS(dpu_rm_iter_template, |
| 834 | TP_PROTO(uint32_t id, enum dpu_hw_blk_type type, uint32_t enc_id), | 834 | TP_PROTO(uint32_t id, uint32_t enc_id), |
| 835 | TP_ARGS(id, type, enc_id), | 835 | TP_ARGS(id, enc_id), |
| 836 | TP_STRUCT__entry( | 836 | TP_STRUCT__entry( |
| 837 | __field( uint32_t, id ) | 837 | __field( uint32_t, id ) |
| 838 | __field( enum dpu_hw_blk_type, type ) | ||
| 839 | __field( uint32_t, enc_id ) | 838 | __field( uint32_t, enc_id ) |
| 840 | ), | 839 | ), |
| 841 | TP_fast_assign( | 840 | TP_fast_assign( |
| 842 | __entry->id = id; | 841 | __entry->id = id; |
| 843 | __entry->type = type; | ||
| 844 | __entry->enc_id = enc_id; | 842 | __entry->enc_id = enc_id; |
| 845 | ), | 843 | ), |
| 846 | TP_printk("id:%d type:%d enc_id:%u", __entry->id, __entry->type, | 844 | TP_printk("id:%d enc_id:%u", __entry->id, __entry->enc_id) |
| 847 | __entry->enc_id) | ||
| 848 | ); | 845 | ); |
| 849 | DEFINE_EVENT(dpu_rm_iter_template, dpu_rm_reserve_intf, | 846 | DEFINE_EVENT(dpu_rm_iter_template, dpu_rm_reserve_intf, |
| 850 | TP_PROTO(uint32_t id, enum dpu_hw_blk_type type, uint32_t enc_id), | 847 | TP_PROTO(uint32_t id, uint32_t enc_id), |
| 851 | TP_ARGS(id, type, enc_id) | 848 | TP_ARGS(id, enc_id) |
| 852 | ); | 849 | ); |
| 853 | DEFINE_EVENT(dpu_rm_iter_template, dpu_rm_reserve_ctls, | 850 | DEFINE_EVENT(dpu_rm_iter_template, dpu_rm_reserve_ctls, |
| 854 | TP_PROTO(uint32_t id, enum dpu_hw_blk_type type, uint32_t enc_id), | 851 | TP_PROTO(uint32_t id, uint32_t enc_id), |
| 855 | TP_ARGS(id, type, enc_id) | 852 | TP_ARGS(id, enc_id) |
| 856 | ); | 853 | ); |
| 857 | 854 | ||
| 858 | TRACE_EVENT(dpu_rm_reserve_lms, | 855 | TRACE_EVENT(dpu_rm_reserve_lms, |
| 859 | TP_PROTO(uint32_t id, enum dpu_hw_blk_type type, uint32_t enc_id, | 856 | TP_PROTO(uint32_t id, uint32_t enc_id, uint32_t pp_id), |
| 860 | uint32_t pp_id), | 857 | TP_ARGS(id, enc_id, pp_id), |
| 861 | TP_ARGS(id, type, enc_id, pp_id), | ||
| 862 | TP_STRUCT__entry( | 858 | TP_STRUCT__entry( |
| 863 | __field( uint32_t, id ) | 859 | __field( uint32_t, id ) |
| 864 | __field( enum dpu_hw_blk_type, type ) | ||
| 865 | __field( uint32_t, enc_id ) | 860 | __field( uint32_t, enc_id ) |
| 866 | __field( uint32_t, pp_id ) | 861 | __field( uint32_t, pp_id ) |
| 867 | ), | 862 | ), |
| 868 | TP_fast_assign( | 863 | TP_fast_assign( |
| 869 | __entry->id = id; | 864 | __entry->id = id; |
| 870 | __entry->type = type; | ||
| 871 | __entry->enc_id = enc_id; | 865 | __entry->enc_id = enc_id; |
| 872 | __entry->pp_id = pp_id; | 866 | __entry->pp_id = pp_id; |
| 873 | ), | 867 | ), |
| 874 | TP_printk("id:%d type:%d enc_id:%u pp_id:%u", __entry->id, | 868 | TP_printk("id:%d enc_id:%u pp_id:%u", __entry->id, |
| 875 | __entry->type, __entry->enc_id, __entry->pp_id) | 869 | __entry->enc_id, __entry->pp_id) |
| 876 | ); | 870 | ); |
| 877 | 871 | ||
| 878 | TRACE_EVENT(dpu_vbif_wait_xin_halt_fail, | 872 | TRACE_EVENT(dpu_vbif_wait_xin_halt_fail, |
| @@ -207,62 +207,44 @@ u32 msm_readl(const void __iomem *addr) · 206 unmodified lines | |||
| 207 | return val; | 207 | return val; |
| 208 | } | 208 | } |
| 209 | 209 | ||
| 210 | struct vblank_event { | 210 | struct msm_vblank_work { |
| 211 | struct list_head node; | 211 | struct work_struct work; |
| 212 | int crtc_id; | 212 | int crtc_id; |
| 213 | bool enable; | 213 | bool enable; |
| 214 | struct msm_drm_private *priv; | ||
| 214 | }; | 215 | }; |
| 215 | 216 | ||
| 216 | static void vblank_ctrl_worker(struct kthread_work *work) | 217 | static void vblank_ctrl_worker(struct work_struct *work) |
| 217 | { | 218 | { |
| 218 | struct msm_vblank_ctrl *vbl_ctrl = container_of(work, | 219 | struct msm_vblank_work *vbl_work = container_of(work, |
| 219 | struct msm_vblank_ctrl, work); | 220 | struct msm_vblank_work, work); |
| 220 | struct msm_drm_private *priv = container_of(vbl_ctrl, | 221 | struct msm_drm_private *priv = vbl_work->priv; |
| 221 | struct msm_drm_private, vblank_ctrl); | ||
| 222 | struct msm_kms *kms = priv->kms; | 222 | struct msm_kms *kms = priv->kms; |
| 223 | struct vblank_event *vbl_ev, *tmp; | ||
| 224 | unsigned long flags; | ||
| 225 | |||
| 226 | spin_lock_irqsave(&vbl_ctrl->lock, flags); | ||
| 227 | list_for_each_entry_safe(vbl_ev, tmp, &vbl_ctrl->event_list, node) { | ||
| 228 | list_del(&vbl_ev->node); | ||
| 229 | spin_unlock_irqrestore(&vbl_ctrl->lock, flags); | ||
| 230 | |||
| 231 | if (vbl_ev->enable) | ||
| 232 | kms->funcs->enable_vblank(kms, | ||
| 233 | priv->crtcs[vbl_ev->crtc_id]); | ||
| 234 | else | ||
| 235 | kms->funcs->disable_vblank(kms, | ||
| 236 | priv->crtcs[vbl_ev->crtc_id]); | ||
| 237 | |||
| 238 | kfree(vbl_ev); | ||
| 239 | 223 | ||
| 240 | spin_lock_irqsave(&vbl_ctrl->lock, flags); | 224 | if (vbl_work->enable) |
| 241 | } | 225 | kms->funcs->enable_vblank(kms, priv->crtcs[vbl_work->crtc_id]); |
| 226 | else | ||
| 227 | kms->funcs->disable_vblank(kms, priv->crtcs[vbl_work->crtc_id]); | ||
| 242 | 228 | ||
| 243 | spin_unlock_irqrestore(&vbl_ctrl->lock, flags); | 229 | kfree(vbl_work); |
| 244 | } | 230 | } |
| 245 | 231 | ||
| 246 | static int vblank_ctrl_queue_work(struct msm_drm_private *priv, | 232 | static int vblank_ctrl_queue_work(struct msm_drm_private *priv, |
| 247 | int crtc_id, bool enable) | 233 | int crtc_id, bool enable) |
| 248 | { | 234 | { |
| 249 | struct msm_vblank_ctrl *vbl_ctrl = &priv->vblank_ctrl; | 235 | struct msm_vblank_work *vbl_work; |
| 250 | struct vblank_event *vbl_ev; | ||
| 251 | unsigned long flags; | ||
| 252 | 236 | ||
| 253 | vbl_ev = kzalloc(sizeof(*vbl_ev), GFP_ATOMIC); | 237 | vbl_work = kzalloc(sizeof(*vbl_work), GFP_ATOMIC); |
| 254 | if (!vbl_ev) | 238 | if (!vbl_work) |
| 255 | return -ENOMEM; | 239 | return -ENOMEM; |
| 256 | 240 | ||
| 257 | vbl_ev->crtc_id = crtc_id; | 241 | INIT_WORK(&vbl_work->work, vblank_ctrl_worker); |
| 258 | vbl_ev->enable = enable; | ||
| 259 | 242 | ||
| 260 | spin_lock_irqsave(&vbl_ctrl->lock, flags); | 243 | vbl_work->crtc_id = crtc_id; |
| 261 | list_add_tail(&vbl_ev->node, &vbl_ctrl->event_list); | 244 | vbl_work->enable = enable; |
| 262 | spin_unlock_irqrestore(&vbl_ctrl->lock, flags); | 245 | vbl_work->priv = priv; |
| 263 | 246 | ||
| 264 | kthread_queue_work(&priv->disp_thread[crtc_id].worker, | 247 | queue_work(priv->wq, &vbl_work->work); |
| 265 | &vbl_ctrl->work); | ||
| 266 | 248 | ||
| 267 | return 0; | 249 | return 0; |
| 268 | } | 250 | } |
| @@ -274,31 +256,20 @@ static int msm_drm_uninit(struct device *dev) · 5 unmodified lines | |||
| 274 | struct msm_drm_private *priv = ddev->dev_private; | 256 | struct msm_drm_private *priv = ddev->dev_private; |
| 275 | struct msm_kms *kms = priv->kms; | 257 | struct msm_kms *kms = priv->kms; |
| 276 | struct msm_mdss *mdss = priv->mdss; | 258 | struct msm_mdss *mdss = priv->mdss; |
| 277 | struct msm_vblank_ctrl *vbl_ctrl = &priv->vblank_ctrl; | ||
| 278 | struct vblank_event *vbl_ev, *tmp; | ||
| 279 | int i; | 259 | int i; |
| 280 | 260 | ||
| 281 | /* We must cancel and cleanup any pending vblank enable/disable | 261 | /* We must cancel and cleanup any pending vblank enable/disable |
| 282 | * work before drm_irq_uninstall() to avoid work re-enabling an | 262 | * work before drm_irq_uninstall() to avoid work re-enabling an |
| 283 | * irq after uninstall has disabled it. | 263 | * irq after uninstall has disabled it. |
| 284 | */ | 264 | */ |
| 285 | kthread_flush_work(&vbl_ctrl->work); | ||
| 286 | list_for_each_entry_safe(vbl_ev, tmp, &vbl_ctrl->event_list, node) { | ||
| 287 | list_del(&vbl_ev->node); | ||
| 288 | kfree(vbl_ev); | ||
| 289 | } | ||
| 290 | 265 | ||
| 291 | /* clean up display commit/event worker threads */ | 266 | flush_workqueue(priv->wq); |
| 292 | for (i = 0; i < priv->num_crtcs; i++) { | 267 | destroy_workqueue(priv->wq); |
| 293 | if (priv->disp_thread[i].thread) { | ||
| 294 | kthread_flush_worker(&priv->disp_thread[i].worker); | ||
| 295 | kthread_stop(priv->disp_thread[i].thread); | ||
| 296 | priv->disp_thread[i].thread = NULL; | ||
| 297 | } | ||
| 298 | 268 | ||
| 269 | /* clean up event worker threads */ | ||
| 270 | for (i = 0; i < priv->num_crtcs; i++) { | ||
| 299 | if (priv->event_thread[i].thread) { | 271 | if (priv->event_thread[i].thread) { |
| 300 | kthread_flush_worker(&priv->event_thread[i].worker); | 272 | kthread_destroy_worker(&priv->event_thread[i].worker); |
| 301 | kthread_stop(priv->event_thread[i].thread); | ||
| 302 | priv->event_thread[i].thread = NULL; | 273 | priv->event_thread[i].thread = NULL; |
| 303 | } | 274 | } |
| 304 | } | 275 | } |
| @@ -323,9 +294,6 @@ static int msm_drm_uninit(struct device *dev) · 18 unmodified lines | |||
| 323 | drm_irq_uninstall(ddev); | 294 | drm_irq_uninstall(ddev); |
| 324 | pm_runtime_put_sync(dev); | 295 | pm_runtime_put_sync(dev); |
| 325 | 296 | ||
| 326 | flush_workqueue(priv->wq); | ||
| 327 | destroy_workqueue(priv->wq); | ||
| 328 | |||
| 329 | if (kms && kms->funcs) | 297 | if (kms && kms->funcs) |
| 330 | kms->funcs->destroy(kms); | 298 | kms->funcs->destroy(kms); |
| 331 | 299 | ||
| @@ -490,9 +458,6 @@ static int msm_drm_init(struct device *dev, struct drm_driver *drv) · 158 unmodified lines | |||
| 490 | priv->wq = alloc_ordered_workqueue("msm", 0); | 458 | priv->wq = alloc_ordered_workqueue("msm", 0); |
| 491 | 459 | ||
| 492 | INIT_LIST_HEAD(&priv->inactive_list); | 460 | INIT_LIST_HEAD(&priv->inactive_list); |
| 493 | INIT_LIST_HEAD(&priv->vblank_ctrl.event_list); | ||
| 494 | kthread_init_work(&priv->vblank_ctrl.work, vblank_ctrl_worker); | ||
| 495 | spin_lock_init(&priv->vblank_ctrl.lock); | ||
| 496 | 461 | ||
| 497 | drm_mode_config_init(ddev); | 462 | drm_mode_config_init(ddev); |
| 498 | 463 | ||
| @@ -554,27 +519,6 @@ static int msm_drm_init(struct device *dev, struct drm_driver *drv) · 55 unmodified lines | |||
| 554 | */ | 519 | */ |
| 555 | param.sched_priority = 16; | 520 | param.sched_priority = 16; |
| 556 | for (i = 0; i < priv->num_crtcs; i++) { | 521 | for (i = 0; i < priv->num_crtcs; i++) { |
| 557 | |||
| 558 | /* initialize display thread */ | ||
| 559 | priv->disp_thread[i].crtc_id = priv->crtcs[i]->base.id; | ||
| 560 | kthread_init_worker(&priv->disp_thread[i].worker); | ||
| 561 | priv->disp_thread[i].dev = ddev; | ||
| 562 | priv->disp_thread[i].thread = | ||
| 563 | kthread_run(kthread_worker_fn, | ||
| 564 | &priv->disp_thread[i].worker, | ||
| 565 | "crtc_commit:%d", priv->disp_thread[i].crtc_id); | ||
| 566 | if (IS_ERR(priv->disp_thread[i].thread)) { | ||
| 567 | DRM_DEV_ERROR(dev, "failed to create crtc_commit kthread\n"); | ||
| 568 | priv->disp_thread[i].thread = NULL; | ||
| 569 | goto err_msm_uninit; | ||
| 570 | } | ||
| 571 | |||
| 572 | ret = sched_setscheduler(priv->disp_thread[i].thread, | ||
| 573 | SCHED_FIFO, ¶m); | ||
| 574 | if (ret) | ||
| 575 | dev_warn(dev, "disp_thread set priority failed: %d\n", | ||
| 576 | ret); | ||
| 577 | |||
| 578 | /* initialize event thread */ | 522 | /* initialize event thread */ |
| 579 | priv->event_thread[i].crtc_id = priv->crtcs[i]->base.id; | 523 | priv->event_thread[i].crtc_id = priv->crtcs[i]->base.id; |
| 580 | kthread_init_worker(&priv->event_thread[i].worker); | 524 | kthread_init_worker(&priv->event_thread[i].worker); |
| @@ -589,13 +533,6 @@ static int msm_drm_init(struct device *dev, struct drm_driver *drv) · 8 unmodified lines | |||
| 589 | goto err_msm_uninit; | 533 | goto err_msm_uninit; |
| 590 | } | 534 | } |
| 591 | 535 | ||
| 592 | /** | ||
| 593 | * event thread should also run at same priority as disp_thread | ||
| 594 | * because it is handling frame_done events. A lower priority | ||
| 595 | * event thread and higher priority disp_thread can causes | ||
| 596 | * frame_pending counters beyond 2. This can lead to commit | ||
| 597 | * failure at crtc commit level. | ||
| 598 | */ | ||
| 599 | ret = sched_setscheduler(priv->event_thread[i].thread, | 536 | ret = sched_setscheduler(priv->event_thread[i].thread, |
| 600 | SCHED_FIFO, ¶m); | 537 | SCHED_FIFO, ¶m); |
| 601 | if (ret) | 538 | if (ret) |
| @@ -914,8 +851,12 @@ static int msm_ioctl_gem_info(struct drm_device *dev, void *data, · 312 unmodified lines | |||
| 914 | ret = -EINVAL; | 851 | ret = -EINVAL; |
| 915 | break; | 852 | break; |
| 916 | } | 853 | } |
| 917 | ret = copy_from_user(msm_obj->name, | 854 | if (copy_from_user(msm_obj->name, u64_to_user_ptr(args->value), |
| 918 | u64_to_user_ptr(args->value), args->len); | 855 | args->len)) { |
| 856 | msm_obj->name[0] = '\0'; | ||
| 857 | ret = -EFAULT; | ||
| 858 | break; | ||
| 859 | } | ||
| 919 | msm_obj->name[args->len] = '\0'; | 860 | msm_obj->name[args->len] = '\0'; |
| 920 | for (i = 0; i < args->len; i++) { | 861 | for (i = 0; i < args->len; i++) { |
| 921 | if (!isprint(msm_obj->name[i])) { | 862 | if (!isprint(msm_obj->name[i])) { |
| @@ -931,8 +872,9 @@ static int msm_ioctl_gem_info(struct drm_device *dev, void *data, · 9 unmodified lines | |||
| 931 | } | 872 | } |
| 932 | args->len = strlen(msm_obj->name); | 873 | args->len = strlen(msm_obj->name); |
| 933 | if (args->value) { | 874 | if (args->value) { |
| 934 | ret = copy_to_user(u64_to_user_ptr(args->value), | 875 | if (copy_to_user(u64_to_user_ptr(args->value), |
| 935 | msm_obj->name, args->len); | 876 | msm_obj->name, args->len)) |
| 877 | ret = -EFAULT; | ||
| 936 | } | 878 | } |
| 937 | break; | 879 | break; |
| 938 | } | 880 | } |
| @@ -77,12 +77,6 @@ enum msm_mdp_plane_property { · 76 unmodified lines | |||
| 77 | PLANE_PROP_MAX_NUM | 77 | PLANE_PROP_MAX_NUM |
| 78 | }; | 78 | }; |
| 79 | 79 | ||
| 80 | struct msm_vblank_ctrl { | ||
| 81 | struct kthread_work work; | ||
| 82 | struct list_head event_list; | ||
| 83 | spinlock_t lock; | ||
| 84 | }; | ||
| 85 | |||
| 86 | #define MSM_GPU_MAX_RINGS 4 | 80 | #define MSM_GPU_MAX_RINGS 4 |
| 87 | #define MAX_H_TILES_PER_DISPLAY 2 | 81 | #define MAX_H_TILES_PER_DISPLAY 2 |
| 88 | 82 | ||
| @@ -126,7 +120,7 @@ struct msm_display_topology { · 37 unmodified lines | |||
| 126 | 120 | ||
| 127 | /** | 121 | /** |
| 128 | * struct msm_display_info - defines display properties | 122 | * struct msm_display_info - defines display properties |
| 129 | * @intf_type: DRM_MODE_CONNECTOR_ display type | 123 | * @intf_type: DRM_MODE_ENCODER_ type |
| 130 | * @capabilities: Bitmask of display flags | 124 | * @capabilities: Bitmask of display flags |
| 131 | * @num_of_h_tiles: Number of horizontal tiles in case of split interface | 125 | * @num_of_h_tiles: Number of horizontal tiles in case of split interface |
| 132 | * @h_tile_instance: Controller instance used per tile. Number of elements is | 126 | * @h_tile_instance: Controller instance used per tile. Number of elements is |
| @@ -199,7 +193,6 @@ struct msm_drm_private { · 66 unmodified lines | |||
| 199 | unsigned int num_crtcs; | 193 | unsigned int num_crtcs; |
| 200 | struct drm_crtc *crtcs[MAX_CRTCS]; | 194 | struct drm_crtc *crtcs[MAX_CRTCS]; |
| 201 | 195 | ||
| 202 | struct msm_drm_thread disp_thread[MAX_CRTCS]; | ||
| 203 | struct msm_drm_thread event_thread[MAX_CRTCS]; | 196 | struct msm_drm_thread event_thread[MAX_CRTCS]; |
| 204 | 197 | ||
| 205 | unsigned int num_encoders; | 198 | unsigned int num_encoders; |
| @@ -228,7 +221,6 @@ struct msm_drm_private { · 22 unmodified lines | |||
| 228 | struct notifier_block vmap_notifier; | 221 | struct notifier_block vmap_notifier; |
| 229 | struct shrinker shrinker; | 222 | struct shrinker shrinker; |
| 230 | 223 | ||
| 231 | struct msm_vblank_ctrl vblank_ctrl; | ||
| 232 | struct drm_atomic_state *pm_state; | 224 | struct drm_atomic_state *pm_state; |
| 233 | }; | 225 | }; |
| 234 | 226 | ||
| @@ -30,6 +30,8 @@ nouveau-y += nouveau_vga.o · 29 unmodified lines | |||
| 30 | # DRM - memory management | 30 | # DRM - memory management |
| 31 | nouveau-y += nouveau_bo.o | 31 | nouveau-y += nouveau_bo.o |
| 32 | nouveau-y += nouveau_gem.o | 32 | nouveau-y += nouveau_gem.o |
| 33 | nouveau-$(CONFIG_DRM_NOUVEAU_SVM) += nouveau_svm.o | ||
| 34 | nouveau-$(CONFIG_DRM_NOUVEAU_SVM) += nouveau_dmem.o | ||
| 33 | nouveau-y += nouveau_mem.o | 35 | nouveau-y += nouveau_mem.o |
| 34 | nouveau-y += nouveau_prime.o | 36 | nouveau-y += nouveau_prime.o |
| 35 | nouveau-y += nouveau_sgdma.o | 37 | nouveau-y += nouveau_sgdma.o |
| @@ -71,3 +71,15 @@ config DRM_NOUVEAU_BACKLIGHT · 70 unmodified lines | |||
| 71 | help | 71 | help |
| 72 | Say Y here if you want to control the backlight of your display | 72 | Say Y here if you want to control the backlight of your display |
| 73 | (e.g. a laptop panel). | 73 | (e.g. a laptop panel). |
| 74 | |||
| 75 | config DRM_NOUVEAU_SVM | ||
| 76 | bool "(EXPERIMENTAL) Enable SVM (Shared Virtual Memory) support" | ||
| 77 | depends on ARCH_HAS_HMM | ||
| 78 | depends on DRM_NOUVEAU | ||
| 79 | depends on STAGING | ||
| 80 | select HMM_MIRROR | ||
| 81 | select DEVICE_PRIVATE | ||
| 82 | default n | ||
| 83 | help | ||
| 84 | Say Y here if you want to enable experimental support for | ||
| 85 | Shared Virtual Memory (SVM). | ||
| @@ -40,6 +40,7 @@ · 39 unmodified lines | |||
| 40 | #include "nvreg.h" | 40 | #include "nvreg.h" |
| 41 | #include "nouveau_fbcon.h" | 41 | #include "nouveau_fbcon.h" |
| 42 | #include "disp.h" | 42 | #include "disp.h" |
| 43 | #include "nouveau_dma.h" | ||
| 43 | 44 | ||
| 44 | #include <subdev/bios/pll.h> | 45 | #include <subdev/bios/pll.h> |
| 45 | #include <subdev/clk.h> | 46 | #include <subdev/clk.h> |
| @@ -1077,12 +1078,223 @@ nouveau_crtc_set_config(struct drm_mode_set *set, · 1031 unmodified lines | |||
| 1077 | return ret; | 1078 | return ret; |
| 1078 | } | 1079 | } |
| 1079 | 1080 | ||
| 1081 | struct nv04_page_flip_state { | ||
| 1082 | struct list_head head; | ||
| 1083 | struct drm_pending_vblank_event *event; | ||
| 1084 | struct drm_crtc *crtc; | ||
| 1085 | int bpp, pitch; | ||
| 1086 | u64 offset; | ||
| 1087 | }; | ||
| 1088 | |||
| 1089 | static int | ||
| 1090 | nv04_finish_page_flip(struct nouveau_channel *chan, | ||
| 1091 | struct nv04_page_flip_state *ps) | ||
| 1092 | { | ||
| 1093 | struct nouveau_fence_chan *fctx = chan->fence; | ||
| 1094 | struct nouveau_drm *drm = chan->drm; | ||
| 1095 | struct drm_device *dev = drm->dev; | ||
| 1096 | struct nv04_page_flip_state *s; | ||
| 1097 | unsigned long flags; | ||
| 1098 | |||
| 1099 | spin_lock_irqsave(&dev->event_lock, flags); | ||
| 1100 | |||
| 1101 | if (list_empty(&fctx->flip)) { | ||
| 1102 | NV_ERROR(drm, "unexpected pageflip\n"); | ||
| 1103 | spin_unlock_irqrestore(&dev->event_lock, flags); | ||
| 1104 | return -EINVAL; | ||
| 1105 | } | ||
| 1106 | |||
| 1107 | s = list_first_entry(&fctx->flip, struct nv04_page_flip_state, head); | ||
| 1108 | if (s->event) { | ||
| 1109 | drm_crtc_arm_vblank_event(s->crtc, s->event); | ||
| 1110 | } else { | ||
| 1111 | /* Give up ownership of vblank for page-flipped crtc */ | ||
| 1112 | drm_crtc_vblank_put(s->crtc); | ||
| 1113 | } | ||
| 1114 | |||
| 1115 | list_del(&s->head); | ||
| 1116 | if (ps) | ||
| 1117 | *ps = *s; | ||
| 1118 | kfree(s); | ||
| 1119 | |||
| 1120 | spin_unlock_irqrestore(&dev->event_lock, flags); | ||
| 1121 | return 0; | ||
| 1122 | } | ||
| 1123 | |||
| 1124 | int | ||
| 1125 | nv04_flip_complete(struct nvif_notify *notify) | ||
| 1126 | { | ||
| 1127 | struct nouveau_cli *cli = (void *)notify->object->client; | ||
| 1128 | struct nouveau_drm *drm = cli->drm; | ||
| 1129 | struct nouveau_channel *chan = drm->channel; | ||
| 1130 | struct nv04_page_flip_state state; | ||
| 1131 | |||
| 1132 | if (!nv04_finish_page_flip(chan, &state)) { | ||
| 1133 | nv_set_crtc_base(drm->dev, drm_crtc_index(state.crtc), | ||
| 1134 | state.offset + state.crtc->y * | ||
| 1135 | state.pitch + state.crtc->x * | ||
| 1136 | state.bpp / 8); | ||
| 1137 | } | ||
| 1138 | |||
| 1139 | return NVIF_NOTIFY_KEEP; | ||
| 1140 | } | ||
| 1141 | |||
| 1142 | static int | ||
| 1143 | nv04_page_flip_emit(struct nouveau_channel *chan, | ||
| 1144 | struct nouveau_bo *old_bo, | ||
| 1145 | struct nouveau_bo *new_bo, | ||
| 1146 | struct nv04_page_flip_state *s, | ||
| 1147 | struct nouveau_fence **pfence) | ||
| 1148 | { | ||
| 1149 | struct nouveau_fence_chan *fctx = chan->fence; | ||
| 1150 | struct nouveau_drm *drm = chan->drm; | ||
| 1151 | struct drm_device *dev = drm->dev; | ||
| 1152 | unsigned long flags; | ||
| 1153 | int ret; | ||
| 1154 | |||
| 1155 | /* Queue it to the pending list */ | ||
| 1156 | spin_lock_irqsave(&dev->event_lock, flags); | ||
| 1157 | list_add_tail(&s->head, &fctx->flip); | ||
| 1158 | spin_unlock_irqrestore(&dev->event_lock, flags); | ||
| 1159 | |||
| 1160 | /* Synchronize with the old framebuffer */ | ||
| 1161 | ret = nouveau_fence_sync(old_bo, chan, false, false); | ||
| 1162 | if (ret) | ||
| 1163 | goto fail; | ||
| 1164 | |||
| 1165 | /* Emit the pageflip */ | ||
| 1166 | ret = RING_SPACE(chan, 2); | ||
| 1167 | if (ret) | ||
| 1168 | goto fail; | ||
| 1169 | |||
| 1170 | BEGIN_NV04(chan, NvSubSw, NV_SW_PAGE_FLIP, 1); | ||
| 1171 | OUT_RING (chan, 0x00000000); | ||
| 1172 | FIRE_RING (chan); | ||
| 1173 | |||
| 1174 | ret = nouveau_fence_new(chan, false, pfence); | ||
| 1175 | if (ret) | ||
| 1176 | goto fail; | ||
| 1177 | |||
| 1178 | return 0; | ||
| 1179 | fail: | ||
| 1180 | spin_lock_irqsave(&dev->event_lock, flags); | ||
| 1181 | list_del(&s->head); | ||
| 1182 | spin_unlock_irqrestore(&dev->event_lock, flags); | ||
| 1183 | return ret; | ||
| 1184 | } | ||
| 1185 | |||
| 1186 | static int | ||
| 1187 | nv04_crtc_page_flip(struct drm_crtc *crtc, struct drm_framebuffer *fb, | ||
| 1188 | struct drm_pending_vblank_event *event, u32 flags, | ||
| 1189 | struct drm_modeset_acquire_ctx *ctx) | ||
| 1190 | { | ||
| 1191 | const int swap_interval = (flags & DRM_MODE_PAGE_FLIP_ASYNC) ? 0 : 1; | ||
| 1192 | struct drm_device *dev = crtc->dev; | ||
| 1193 | struct nouveau_drm *drm = nouveau_drm(dev); | ||
| 1194 | struct nouveau_bo *old_bo = nouveau_framebuffer(crtc->primary->fb)->nvbo; | ||
| 1195 | struct nouveau_bo *new_bo = nouveau_framebuffer(fb)->nvbo; | ||
| 1196 | struct nv04_page_flip_state *s; | ||
| 1197 | struct nouveau_channel *chan; | ||
| 1198 | struct nouveau_cli *cli; | ||
| 1199 | struct nouveau_fence *fence; | ||
| 1200 | struct nv04_display *dispnv04 = nv04_display(dev); | ||
| 1201 | int head = nouveau_crtc(crtc)->index; | ||
| 1202 | int ret; | ||
| 1203 | |||
| 1204 | chan = drm->channel; | ||
| 1205 | if (!chan) | ||
| 1206 | return -ENODEV; | ||
| 1207 | cli = (void *)chan->user.client; | ||
| 1208 | |||
| 1209 | s = kzalloc(sizeof(*s), GFP_KERNEL); | ||
| 1210 | if (!s) | ||
| 1211 | return -ENOMEM; | ||
| 1212 | |||
| 1213 | if (new_bo != old_bo) { | ||
| 1214 | ret = nouveau_bo_pin(new_bo, TTM_PL_FLAG_VRAM, true); | ||
| 1215 | if (ret) | ||
| 1216 | goto fail_free; | ||
| 1217 | } | ||
| 1218 | |||
| 1219 | mutex_lock(&cli->mutex); | ||
| 1220 | ret = ttm_bo_reserve(&new_bo->bo, true, false, NULL); | ||
| 1221 | if (ret) | ||
| 1222 | goto fail_unpin; | ||
| 1223 | |||
| 1224 | /* synchronise rendering channel with the kernel's channel */ | ||
| 1225 | ret = nouveau_fence_sync(new_bo, chan, false, true); | ||
| 1226 | if (ret) { | ||
| 1227 | ttm_bo_unreserve(&new_bo->bo); | ||
| 1228 | goto fail_unpin; | ||
| 1229 | } | ||
| 1230 | |||
| 1231 | if (new_bo != old_bo) { | ||
| 1232 | ttm_bo_unreserve(&new_bo->bo); | ||
| 1233 | |||
| 1234 | ret = ttm_bo_reserve(&old_bo->bo, true, false, NULL); | ||
| 1235 | if (ret) | ||
| 1236 | goto fail_unpin; | ||
| 1237 | } | ||
| 1238 | |||
| 1239 | /* Initialize a page flip struct */ | ||
| 1240 | *s = (struct nv04_page_flip_state) | ||
| 1241 | { { }, event, crtc, fb->format->cpp[0] * 8, fb->pitches[0], | ||
| 1242 | new_bo->bo.offset }; | ||
| 1243 | |||
| 1244 | /* Keep vblanks on during flip, for the target crtc of this flip */ | ||
| 1245 | drm_crtc_vblank_get(crtc); | ||
| 1246 | |||
| 1247 | /* Emit a page flip */ | ||
| 1248 | if (swap_interval) { | ||
| 1249 | ret = RING_SPACE(chan, 8); | ||
| 1250 | if (ret) | ||
| 1251 | goto fail_unreserve; | ||
| 1252 | |||
| 1253 | BEGIN_NV04(chan, NvSubImageBlit, 0x012c, 1); | ||
| 1254 | OUT_RING (chan, 0); | ||
| 1255 | BEGIN_NV04(chan, NvSubImageBlit, 0x0134, 1); | ||
| 1256 | OUT_RING (chan, head); | ||
| 1257 | BEGIN_NV04(chan, NvSubImageBlit, 0x0100, 1); | ||
| 1258 | OUT_RING (chan, 0); | ||
| 1259 | BEGIN_NV04(chan, NvSubImageBlit, 0x0130, 1); | ||
| 1260 | OUT_RING (chan, 0); | ||
| 1261 | } | ||
| 1262 | |||
| 1263 | nouveau_bo_ref(new_bo, &dispnv04->image[head]); | ||
| 1264 | |||
| 1265 | ret = nv04_page_flip_emit(chan, old_bo, new_bo, s, &fence); | ||
| 1266 | if (ret) | ||
| 1267 | goto fail_unreserve; | ||
| 1268 | mutex_unlock(&cli->mutex); | ||
| 1269 | |||
| 1270 | /* Update the crtc struct and cleanup */ | ||
| 1271 | crtc->primary->fb = fb; | ||
| 1272 | |||
| 1273 | nouveau_bo_fence(old_bo, fence, false); | ||
| 1274 | ttm_bo_unreserve(&old_bo->bo); | ||
| 1275 | if (old_bo != new_bo) | ||
| 1276 | nouveau_bo_unpin(old_bo); | ||
| 1277 | nouveau_fence_unref(&fence); | ||
| 1278 | return 0; | ||
| 1279 | |||
| 1280 | fail_unreserve: | ||
| 1281 | drm_crtc_vblank_put(crtc); | ||
| 1282 | ttm_bo_unreserve(&old_bo->bo); | ||
| 1283 | fail_unpin: | ||
| 1284 | mutex_unlock(&cli->mutex); | ||
| 1285 | if (old_bo != new_bo) | ||
| 1286 | nouveau_bo_unpin(new_bo); | ||
| 1287 | fail_free: | ||
| 1288 | kfree(s); | ||
| 1289 | return ret; | ||
| 1290 | } | ||
| 1291 | |||
| 1080 | static const struct drm_crtc_funcs nv04_crtc_funcs = { | 1292 | static const struct drm_crtc_funcs nv04_crtc_funcs = { |
| 1081 | .cursor_set = nv04_crtc_cursor_set, | 1293 | .cursor_set = nv04_crtc_cursor_set, |
| 1082 | .cursor_move = nv04_crtc_cursor_move, | 1294 | .cursor_move = nv04_crtc_cursor_move, |
| 1083 | .gamma_set = nv_crtc_gamma_set, | 1295 | .gamma_set = nv_crtc_gamma_set, |
| 1084 | .set_config = nouveau_crtc_set_config, | 1296 | .set_config = nouveau_crtc_set_config, |
| 1085 | .page_flip = nouveau_crtc_page_flip, | 1297 | .page_flip = nv04_crtc_page_flip, |
| 1086 | .destroy = nv_crtc_destroy, | 1298 | .destroy = nv_crtc_destroy, |
| 1087 | }; | 1299 | }; |
| 1088 | 1300 | ||
| @@ -30,6 +30,160 @@ · 29 unmodified lines | |||
| 30 | #include "hw.h" | 30 | #include "hw.h" |
| 31 | #include "nouveau_encoder.h" | 31 | #include "nouveau_encoder.h" |
| 32 | #include "nouveau_connector.h" | 32 | #include "nouveau_connector.h" |
| 33 | #include "nouveau_bo.h" | ||
| 34 | |||
| 35 | #include <nvif/if0004.h> | ||
| 36 | |||
| 37 | static void | ||
| 38 | nv04_display_fini(struct drm_device *dev, bool suspend) | ||
| 39 | { | ||
| 40 | struct nv04_display *disp = nv04_display(dev); | ||
| 41 | struct drm_crtc *crtc; | ||
| 42 | |||
| 43 | /* Disable flip completion events. */ | ||
| 44 | nvif_notify_put(&disp->flip); | ||
| 45 | |||
| 46 | /* Disable vblank interrupts. */ | ||
| 47 | NVWriteCRTC(dev, 0, NV_PCRTC_INTR_EN_0, 0); | ||
| 48 | if (nv_two_heads(dev)) | ||
| 49 | NVWriteCRTC(dev, 1, NV_PCRTC_INTR_EN_0, 0); | ||
| 50 | |||
| 51 | if (!suspend) | ||
| 52 | return; | ||
| 53 | |||
| 54 | /* Un-pin FB and cursors so they'll be evicted to system memory. */ | ||
| 55 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | ||
| 56 | struct nouveau_framebuffer *nouveau_fb; | ||
| 57 | |||
| 58 | nouveau_fb = nouveau_framebuffer(crtc->primary->fb); | ||
| 59 | if (!nouveau_fb || !nouveau_fb->nvbo) | ||
| 60 | continue; | ||
| 61 | |||
| 62 | nouveau_bo_unpin(nouveau_fb->nvbo); | ||
| 63 | } | ||
| 64 | |||
| 65 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | ||
| 66 | struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | ||
| 67 | if (nv_crtc->cursor.nvbo) { | ||
| 68 | if (nv_crtc->cursor.set_offset) | ||
| 69 | nouveau_bo_unmap(nv_crtc->cursor.nvbo); | ||
| 70 | nouveau_bo_unpin(nv_crtc->cursor.nvbo); | ||
| 71 | } | ||
| 72 | } | ||
| 73 | } | ||
| 74 | |||
| 75 | static int | ||
| 76 | nv04_display_init(struct drm_device *dev, bool resume, bool runtime) | ||
| 77 | { | ||
| 78 | struct nv04_display *disp = nv04_display(dev); | ||
| 79 | struct nouveau_drm *drm = nouveau_drm(dev); | ||
| 80 | struct nouveau_encoder *encoder; | ||
| 81 | struct drm_crtc *crtc; | ||
| 82 | int ret; | ||
| 83 | |||
| 84 | /* meh.. modeset apparently doesn't setup all the regs and depends | ||
| 85 | * on pre-existing state, for now load the state of the card *before* | ||
| 86 | * nouveau was loaded, and then do a modeset. | ||
| 87 | * | ||
| 88 | * best thing to do probably is to make save/restore routines not | ||
| 89 | * save/restore "pre-load" state, but more general so we can save | ||
| 90 | * on suspend too. | ||
| 91 | */ | ||
| 92 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | ||
| 93 | struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | ||
| 94 | nv_crtc->save(&nv_crtc->base); | ||
| 95 | } | ||
| 96 | |||
| 97 | list_for_each_entry(encoder, &dev->mode_config.encoder_list, base.base.head) | ||
| 98 | encoder->enc_save(&encoder->base.base); | ||
| 99 | |||
| 100 | /* Enable flip completion events. */ | ||
| 101 | nvif_notify_get(&disp->flip); | ||
| 102 | |||
| 103 | if (!resume) | ||
| 104 | return 0; | ||
| 105 | |||
| 106 | /* Re-pin FB/cursors. */ | ||
| 107 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | ||
| 108 | struct nouveau_framebuffer *nouveau_fb; | ||
| 109 | |||
| 110 | nouveau_fb = nouveau_framebuffer(crtc->primary->fb); | ||
| 111 | if (!nouveau_fb || !nouveau_fb->nvbo) | ||
| 112 | continue; | ||
| 113 | |||
| 114 | ret = nouveau_bo_pin(nouveau_fb->nvbo, TTM_PL_FLAG_VRAM, true); | ||
| 115 | if (ret) | ||
| 116 | NV_ERROR(drm, "Could not pin framebuffer\n"); | ||
| 117 | } | ||
| 118 | |||
| 119 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | ||
| 120 | struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | ||
| 121 | if (!nv_crtc->cursor.nvbo) | ||
| 122 | continue; | ||
| 123 | |||
| 124 | ret = nouveau_bo_pin(nv_crtc->cursor.nvbo, TTM_PL_FLAG_VRAM, true); | ||
| 125 | if (!ret && nv_crtc->cursor.set_offset) | ||
| 126 | ret = nouveau_bo_map(nv_crtc->cursor.nvbo); | ||
| 127 | if (ret) | ||
| 128 | NV_ERROR(drm, "Could not pin/map cursor.\n"); | ||
| 129 | } | ||
| 130 | |||
| 131 | /* Force CLUT to get re-loaded during modeset. */ | ||
| 132 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | ||
| 133 | struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | ||
| 134 | |||
| 135 | nv_crtc->lut.depth = 0; | ||
| 136 | } | ||
| 137 | |||
| 138 | /* This should ensure we don't hit a locking problem when someone | ||
| 139 | * wakes us up via a connector. We should never go into suspend | ||
| 140 | * while the display is on anyways. | ||
| 141 | */ | ||
| 142 | if (runtime) | ||
| 143 | return 0; | ||
| 144 | |||
| 145 | /* Restore mode. */ | ||
| 146 | drm_helper_resume_force_mode(dev); | ||
| 147 | |||
| 148 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | ||
| 149 | struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | ||
| 150 | |||
| 151 | if (!nv_crtc->cursor.nvbo) | ||
| 152 | continue; | ||
| 153 | |||
| 154 | if (nv_crtc->cursor.set_offset) | ||
| 155 | nv_crtc->cursor.set_offset(nv_crtc, nv_crtc->cursor.nvbo->bo.offset); | ||
| 156 | nv_crtc->cursor.set_pos(nv_crtc, nv_crtc->cursor_saved_x, | ||
| 157 | nv_crtc->cursor_saved_y); | ||
| 158 | } | ||
| 159 | |||
| 160 | return 0; | ||
| 161 | } | ||
| 162 | |||
| 163 | static void | ||
| 164 | nv04_display_destroy(struct drm_device *dev) | ||
| 165 | { | ||
| 166 | struct nv04_display *disp = nv04_display(dev); | ||
| 167 | struct nouveau_drm *drm = nouveau_drm(dev); | ||
| 168 | struct nouveau_encoder *encoder; | ||
| 169 | struct nouveau_crtc *nv_crtc; | ||
| 170 | |||
| 171 | /* Restore state */ | ||
| 172 | list_for_each_entry(encoder, &dev->mode_config.encoder_list, base.base.head) | ||
| 173 | encoder->enc_restore(&encoder->base.base); | ||
| 174 | |||
| 175 | list_for_each_entry(nv_crtc, &dev->mode_config.crtc_list, base.head) | ||
| 176 | nv_crtc->restore(&nv_crtc->base); | ||
| 177 | |||
| 178 | nouveau_hw_save_vga_fonts(dev, 0); | ||
| 179 | |||
| 180 | nvif_notify_fini(&disp->flip); | ||
| 181 | |||
| 182 | nouveau_display(dev)->priv = NULL; | ||
| 183 | kfree(disp); | ||
| 184 | |||
| 185 | nvif_object_unmap(&drm->client.device.object); | ||
| 186 | } | ||
| 33 | 187 | ||
| 34 | int | 188 | int |
| 35 | nv04_display_create(struct drm_device *dev) | 189 | nv04_display_create(struct drm_device *dev) |
| @@ -58,6 +212,13 @@ nv04_display_create(struct drm_device *dev) · 22 unmodified lines | |||
| 58 | /* Pre-nv50 doesn't support atomic, so don't expose the ioctls */ | 212 | /* Pre-nv50 doesn't support atomic, so don't expose the ioctls */ |
| 59 | dev->driver->driver_features &= ~DRIVER_ATOMIC; | 213 | dev->driver->driver_features &= ~DRIVER_ATOMIC; |
| 60 | 214 | ||
| 215 | /* Request page flip completion event. */ | ||
| 216 | if (drm->nvsw.client) { | ||
| 217 | nvif_notify_init(&drm->nvsw, nv04_flip_complete, | ||
| 218 | false, NV04_NVSW_NTFY_UEVENT, | ||
| 219 | NULL, 0, 0, &disp->flip); | ||
| 220 | } | ||
| 221 | |||
| 61 | nouveau_hw_save_vga_fonts(dev, 1); | 222 | nouveau_hw_save_vga_fonts(dev, 1); |
| 62 | 223 | ||
| 63 | nv04_crtc_create(dev, 0); | 224 | nv04_crtc_create(dev, 0); |
| @@ -121,58 +282,3 @@ nv04_display_create(struct drm_device *dev) · 57 unmodified lines | |||
| 121 | 282 | ||
| 122 | return 0; | 283 | return 0; |
| 123 | } | 284 | } |
| 124 | |||
| 125 | void | ||
| 126 | nv04_display_destroy(struct drm_device *dev) | ||
| 127 | { | ||
| 128 | struct nv04_display *disp = nv04_display(dev); | ||
| 129 | struct nouveau_drm *drm = nouveau_drm(dev); | ||
| 130 | struct nouveau_encoder *encoder; | ||
| 131 | struct nouveau_crtc *nv_crtc; | ||
| 132 | |||
| 133 | /* Restore state */ | ||
| 134 | list_for_each_entry(encoder, &dev->mode_config.encoder_list, base.base.head) | ||
| 135 | encoder->enc_restore(&encoder->base.base); | ||
| 136 | |||
| 137 | list_for_each_entry(nv_crtc, &dev->mode_config.crtc_list, base.head) | ||
| 138 | nv_crtc->restore(&nv_crtc->base); | ||
| 139 | |||
| 140 | nouveau_hw_save_vga_fonts(dev, 0); | ||
| 141 | |||
| 142 | nouveau_display(dev)->priv = NULL; | ||
| 143 | kfree(disp); | ||
| 144 | |||
| 145 | nvif_object_unmap(&drm->client.device.object); | ||
| 146 | } | ||
| 147 | |||
| 148 | int | ||
| 149 | nv04_display_init(struct drm_device *dev) | ||
| 150 | { | ||
| 151 | struct nouveau_encoder *encoder; | ||
| 152 | struct nouveau_crtc *crtc; | ||
| 153 | |||
| 154 | /* meh.. modeset apparently doesn't setup all the regs and depends | ||
| 155 | * on pre-existing state, for now load the state of the card *before* | ||
| 156 | * nouveau was loaded, and then do a modeset. | ||
| 157 | * | ||
| 158 | * best thing to do probably is to make save/restore routines not | ||
| 159 | * save/restore "pre-load" state, but more general so we can save | ||
| 160 | * on suspend too. | ||
| 161 | */ | ||
| 162 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, base.head) | ||
| 163 | crtc->save(&crtc->base); | ||
| 164 | |||
| 165 | list_for_each_entry(encoder, &dev->mode_config.encoder_list, base.base.head) | ||
| 166 | encoder->enc_save(&encoder->base.base); | ||
| 167 | |||
| 168 | return 0; | ||
| 169 | } | ||
| 170 | |||
| 171 | void | ||
| 172 | nv04_display_fini(struct drm_device *dev) | ||
| 173 | { | ||
| 174 | /* disable vblank interrupts */ | ||
| 175 | NVWriteCRTC(dev, 0, NV_PCRTC_INTR_EN_0, 0); | ||
| 176 | if (nv_two_heads(dev)) | ||
| 177 | NVWriteCRTC(dev, 1, NV_PCRTC_INTR_EN_0, 0); | ||
| 178 | } | ||
| @@ -82,6 +82,7 @@ struct nv04_display { · 81 unmodified lines | |||
| 82 | uint32_t saved_vga_font[4][16384]; | 82 | uint32_t saved_vga_font[4][16384]; |
| 83 | uint32_t dac_users[4]; | 83 | uint32_t dac_users[4]; |
| 84 | struct nouveau_bo *image[2]; | 84 | struct nouveau_bo *image[2]; |
| 85 | struct nvif_notify flip; | ||
| 85 | }; | 86 | }; |
| 86 | 87 | ||
| 87 | static inline struct nv04_display * | 88 | static inline struct nv04_display * |
| @@ -92,9 +93,6 @@ nv04_display(struct drm_device *dev) · 4 unmodified lines | |||
| 92 | 93 | ||
| 93 | /* nv04_display.c */ | 94 | /* nv04_display.c */ |
| 94 | int nv04_display_create(struct drm_device *); | 95 | int nv04_display_create(struct drm_device *); |
| 95 | void nv04_display_destroy(struct drm_device *); | ||
| 96 | int nv04_display_init(struct drm_device *); | ||
| 97 | void nv04_display_fini(struct drm_device *); | ||
| 98 | 96 | ||
| 99 | /* nv04_crtc.c */ | 97 | /* nv04_crtc.c */ |
| 100 | int nv04_crtc_create(struct drm_device *, int index); | 98 | int nv04_crtc_create(struct drm_device *, int index); |
| @@ -176,4 +174,5 @@ nouveau_bios_run_init_table(struct drm_device *dev, u16 table, · 75 unmodified lines | |||
| 176 | ); | 174 | ); |
| 177 | } | 175 | } |
| 178 | 176 | ||
| 177 | int nv04_flip_complete(struct nvif_notify *); | ||
| 179 | #endif | 178 | #endif |
| @@ -42,7 +42,7 @@ nv50_core_new(struct nouveau_drm *drm, struct nv50_core **pcore) · 41 unmodified lines | |||
| 42 | int version; | 42 | int version; |
| 43 | int (*new)(struct nouveau_drm *, s32, struct nv50_core **); | 43 | int (*new)(struct nouveau_drm *, s32, struct nv50_core **); |
| 44 | } cores[] = { | 44 | } cores[] = { |
| 45 | { TU104_DISP_CORE_CHANNEL_DMA, 0, corec57d_new }, | 45 | { TU102_DISP_CORE_CHANNEL_DMA, 0, corec57d_new }, |
| 46 | { GV100_DISP_CORE_CHANNEL_DMA, 0, corec37d_new }, | 46 | { GV100_DISP_CORE_CHANNEL_DMA, 0, corec37d_new }, |
| 47 | { GP102_DISP_CORE_CHANNEL_DMA, 0, core917d_new }, | 47 | { GP102_DISP_CORE_CHANNEL_DMA, 0, core917d_new }, |
| 48 | { GP100_DISP_CORE_CHANNEL_DMA, 0, core917d_new }, | 48 | { GP100_DISP_CORE_CHANNEL_DMA, 0, core917d_new }, |
| @@ -31,7 +31,7 @@ nv50_curs_new(struct nouveau_drm *drm, int head, struct nv50_wndw **pwndw) · 30 unmodified lines | |||
| 31 | int version; | 31 | int version; |
| 32 | int (*new)(struct nouveau_drm *, int, s32, struct nv50_wndw **); | 32 | int (*new)(struct nouveau_drm *, int, s32, struct nv50_wndw **); |
| 33 | } curses[] = { | 33 | } curses[] = { |
| 34 | { TU104_DISP_CURSOR, 0, cursc37a_new }, | 34 | { TU102_DISP_CURSOR, 0, cursc37a_new }, |
| 35 | { GV100_DISP_CURSOR, 0, cursc37a_new }, | 35 | { GV100_DISP_CURSOR, 0, cursc37a_new }, |
| 36 | { GK104_DISP_CURSOR, 0, curs907a_new }, | 36 | { GK104_DISP_CURSOR, 0, curs907a_new }, |
| 37 | { GF110_DISP_CURSOR, 0, curs907a_new }, | 37 | { GF110_DISP_CURSOR, 0, curs907a_new }, |
| @@ -817,7 +817,8 @@ nv50_msto_enable(struct drm_encoder *encoder) · 816 unmodified lines | |||
| 817 | 817 | ||
| 818 | r = drm_dp_mst_allocate_vcpi(&mstm->mgr, mstc->port, armh->dp.pbn, | 818 | r = drm_dp_mst_allocate_vcpi(&mstm->mgr, mstc->port, armh->dp.pbn, |
| 819 | armh->dp.tu); | 819 | armh->dp.tu); |
| 820 | WARN_ON(!r); | 820 | if (!r) |
| 821 | DRM_DEBUG_KMS("Failed to allocate VCPI\n"); | ||
| 821 | 822 | ||
| 822 | if (!mstm->links++) | 823 | if (!mstm->links++) |
| 823 | nv50_outp_acquire(mstm->outp); | 824 | nv50_outp_acquire(mstm->outp); |
| @@ -2220,8 +2221,8 @@ nv50_disp_func = { · 1396 unmodified lines | |||
| 2220 | * Init | 2221 | * Init |
| 2221 | *****************************************************************************/ | 2222 | *****************************************************************************/ |
| 2222 | 2223 | ||
| 2223 | void | 2224 | static void |
| 2224 | nv50_display_fini(struct drm_device *dev) | 2225 | nv50_display_fini(struct drm_device *dev, bool suspend) |
| 2225 | { | 2226 | { |
| 2226 | struct nouveau_encoder *nv_encoder; | 2227 | struct nouveau_encoder *nv_encoder; |
| 2227 | struct drm_encoder *encoder; | 2228 | struct drm_encoder *encoder; |
| @@ -2242,8 +2243,8 @@ nv50_display_fini(struct drm_device *dev) · 14 unmodified lines | |||
| 2242 | } | 2243 | } |
| 2243 | } | 2244 | } |
| 2244 | 2245 | ||
| 2245 | int | 2246 | static int |
| 2246 | nv50_display_init(struct drm_device *dev) | 2247 | nv50_display_init(struct drm_device *dev, bool resume, bool runtime) |
| 2247 | { | 2248 | { |
| 2248 | struct nv50_core *core = nv50_disp(dev)->core; | 2249 | struct nv50_core *core = nv50_disp(dev)->core; |
| 2249 | struct drm_encoder *encoder; | 2250 | struct drm_encoder *encoder; |
| @@ -2269,7 +2270,7 @@ nv50_display_init(struct drm_device *dev) · 19 unmodified lines | |||
| 2269 | return 0; | 2270 | return 0; |
| 2270 | } | 2271 | } |
| 2271 | 2272 | ||
| 2272 | void | 2273 | static void |
| 2273 | nv50_display_destroy(struct drm_device *dev) | 2274 | nv50_display_destroy(struct drm_device *dev) |
| 2274 | { | 2275 | { |
| 2275 | struct nv50_disp *disp = nv50_disp(dev); | 2276 | struct nv50_disp *disp = nv50_disp(dev); |
| @@ -31,7 +31,7 @@ nv50_wimm_init(struct nouveau_drm *drm, struct nv50_wndw *wndw) · 30 unmodified lines | |||
| 31 | int version; | 31 | int version; |
| 32 | int (*init)(struct nouveau_drm *, s32, struct nv50_wndw *); | 32 | int (*init)(struct nouveau_drm *, s32, struct nv50_wndw *); |
| 33 | } wimms[] = { | 33 | } wimms[] = { |
| 34 | { TU104_DISP_WINDOW_IMM_CHANNEL_DMA, 0, wimmc37b_init }, | 34 | { TU102_DISP_WINDOW_IMM_CHANNEL_DMA, 0, wimmc37b_init }, |
| 35 | { GV100_DISP_WINDOW_IMM_CHANNEL_DMA, 0, wimmc37b_init }, | 35 | { GV100_DISP_WINDOW_IMM_CHANNEL_DMA, 0, wimmc37b_init }, |
| 36 | {} | 36 | {} |
| 37 | }; | 37 | }; |
| @@ -626,7 +626,7 @@ nv50_wndw_new(struct nouveau_drm *drm, enum drm_plane_type type, int index, · 625 unmodified lines | |||
| 626 | int (*new)(struct nouveau_drm *, enum drm_plane_type, | 626 | int (*new)(struct nouveau_drm *, enum drm_plane_type, |
| 627 | int, s32, struct nv50_wndw **); | 627 | int, s32, struct nv50_wndw **); |
| 628 | } wndws[] = { | 628 | } wndws[] = { |
| 629 | { TU104_DISP_WINDOW_CHANNEL_DMA, 0, wndwc57e_new }, | 629 | { TU102_DISP_WINDOW_CHANNEL_DMA, 0, wndwc57e_new }, |
| 630 | { GV100_DISP_WINDOW_CHANNEL_DMA, 0, wndwc37e_new }, | 630 | { GV100_DISP_WINDOW_CHANNEL_DMA, 0, wndwc37e_new }, |
| 631 | {} | 631 | {} |
| 632 | }; | 632 | }; |
| @@ -54,6 +54,9 @@ · 53 unmodified lines | |||
| 54 | 54 | ||
| 55 | #define VOLTA_USERMODE_A 0x0000c361 | 55 | #define VOLTA_USERMODE_A 0x0000c361 |
| 56 | 56 | ||
| 57 | #define MAXWELL_FAULT_BUFFER_A /* clb069.h */ 0x0000b069 | ||
| 58 | #define VOLTA_FAULT_BUFFER_A /* clb069.h */ 0x0000c369 | ||
| 59 | |||
| 57 | #define NV03_CHANNEL_DMA /* cl506b.h */ 0x0000006b | 60 | #define NV03_CHANNEL_DMA /* cl506b.h */ 0x0000006b |
| 58 | #define NV10_CHANNEL_DMA /* cl506b.h */ 0x0000006e | 61 | #define NV10_CHANNEL_DMA /* cl506b.h */ 0x0000006e |
| 59 | #define NV17_CHANNEL_DMA /* cl506b.h */ 0x0000176e | 62 | #define NV17_CHANNEL_DMA /* cl506b.h */ 0x0000176e |
| @@ -84,7 +87,7 @@ · 24 unmodified lines | |||
| 84 | #define GP100_DISP /* cl5070.h */ 0x00009770 | 87 | #define GP100_DISP /* cl5070.h */ 0x00009770 |
| 85 | #define GP102_DISP /* cl5070.h */ 0x00009870 | 88 | #define GP102_DISP /* cl5070.h */ 0x00009870 |
| 86 | #define GV100_DISP /* cl5070.h */ 0x0000c370 | 89 | #define GV100_DISP /* cl5070.h */ 0x0000c370 |
| 87 | #define TU104_DISP /* cl5070.h */ 0x0000c570 | 90 | #define TU102_DISP /* cl5070.h */ 0x0000c570 |
| 88 | 91 | ||
| 89 | #define NV31_MPEG 0x00003174 | 92 | #define NV31_MPEG 0x00003174 |
| 90 | #define G82_MPEG 0x00008274 | 93 | #define G82_MPEG 0x00008274 |
| @@ -97,7 +100,7 @@ · 6 unmodified lines | |||
| 97 | #define GF110_DISP_CURSOR /* cl507a.h */ 0x0000907a | 100 | #define GF110_DISP_CURSOR /* cl507a.h */ 0x0000907a |
| 98 | #define GK104_DISP_CURSOR /* cl507a.h */ 0x0000917a | 101 | #define GK104_DISP_CURSOR /* cl507a.h */ 0x0000917a |
| 99 | #define GV100_DISP_CURSOR /* cl507a.h */ 0x0000c37a | 102 | #define GV100_DISP_CURSOR /* cl507a.h */ 0x0000c37a |
| 100 | #define TU104_DISP_CURSOR /* cl507a.h */ 0x0000c57a | 103 | #define TU102_DISP_CURSOR /* cl507a.h */ 0x0000c57a |
| 101 | 104 | ||
| 102 | #define NV50_DISP_OVERLAY /* cl507b.h */ 0x0000507b | 105 | #define NV50_DISP_OVERLAY /* cl507b.h */ 0x0000507b |
| 103 | #define G82_DISP_OVERLAY /* cl507b.h */ 0x0000827b | 106 | #define G82_DISP_OVERLAY /* cl507b.h */ 0x0000827b |
| @@ -106,7 +109,7 @@ · 2 unmodified lines | |||
| 106 | #define GK104_DISP_OVERLAY /* cl507b.h */ 0x0000917b | 109 | #define GK104_DISP_OVERLAY /* cl507b.h */ 0x0000917b |
| 107 | 110 | ||
| 108 | #define GV100_DISP_WINDOW_IMM_CHANNEL_DMA /* clc37b.h */ 0x0000c37b | 111 | #define GV100_DISP_WINDOW_IMM_CHANNEL_DMA /* clc37b.h */ 0x0000c37b |
| 109 | #define TU104_DISP_WINDOW_IMM_CHANNEL_DMA /* clc37b.h */ 0x0000c57b | 112 | #define TU102_DISP_WINDOW_IMM_CHANNEL_DMA /* clc37b.h */ 0x0000c57b |
| 110 | 113 | ||
| 111 | #define NV50_DISP_BASE_CHANNEL_DMA /* cl507c.h */ 0x0000507c | 114 | #define NV50_DISP_BASE_CHANNEL_DMA /* cl507c.h */ 0x0000507c |
| 112 | #define G82_DISP_BASE_CHANNEL_DMA /* cl507c.h */ 0x0000827c | 115 | #define G82_DISP_BASE_CHANNEL_DMA /* cl507c.h */ 0x0000827c |
| @@ -129,7 +132,7 @@ · 16 unmodified lines | |||
| 129 | #define GP100_DISP_CORE_CHANNEL_DMA /* cl507d.h */ 0x0000977d | 132 | #define GP100_DISP_CORE_CHANNEL_DMA /* cl507d.h */ 0x0000977d |
| 130 | #define GP102_DISP_CORE_CHANNEL_DMA /* cl507d.h */ 0x0000987d | 133 | #define GP102_DISP_CORE_CHANNEL_DMA /* cl507d.h */ 0x0000987d |
| 131 | #define GV100_DISP_CORE_CHANNEL_DMA /* cl507d.h */ 0x0000c37d | 134 | #define GV100_DISP_CORE_CHANNEL_DMA /* cl507d.h */ 0x0000c37d |
| 132 | #define TU104_DISP_CORE_CHANNEL_DMA /* cl507d.h */ 0x0000c57d | 135 | #define TU102_DISP_CORE_CHANNEL_DMA /* cl507d.h */ 0x0000c57d |
| 133 | 136 | ||
| 134 | #define NV50_DISP_OVERLAY_CHANNEL_DMA /* cl507e.h */ 0x0000507e | 137 | #define NV50_DISP_OVERLAY_CHANNEL_DMA /* cl507e.h */ 0x0000507e |
| 135 | #define G82_DISP_OVERLAY_CHANNEL_DMA /* cl507e.h */ 0x0000827e | 138 | #define G82_DISP_OVERLAY_CHANNEL_DMA /* cl507e.h */ 0x0000827e |
| @@ -139,7 +142,7 @@ · 3 unmodified lines | |||
| 139 | #define GK104_DISP_OVERLAY_CONTROL_DMA /* cl507e.h */ 0x0000917e | 142 | #define GK104_DISP_OVERLAY_CONTROL_DMA /* cl507e.h */ 0x0000917e |
| 140 | 143 | ||
| 141 | #define GV100_DISP_WINDOW_CHANNEL_DMA /* clc37e.h */ 0x0000c37e | 144 | #define GV100_DISP_WINDOW_CHANNEL_DMA /* clc37e.h */ 0x0000c37e |
| 142 | #define TU104_DISP_WINDOW_CHANNEL_DMA /* clc37e.h */ 0x0000c57e | 145 | #define TU102_DISP_WINDOW_CHANNEL_DMA /* clc37e.h */ 0x0000c57e |
| 143 | 146 | ||
| 144 | #define NV50_TESLA 0x00005097 | 147 | #define NV50_TESLA 0x00005097 |
| 145 | #define G82_TESLA 0x00008297 | 148 | #define G82_TESLA 0x00008297 |
| @@ -0,0 +1,12 @@ | |||
| 1 | #ifndef __NVIF_CLB069_H__ | ||
| 2 | #define __NVIF_CLB069_H__ | ||
| 3 | struct nvif_clb069_v0 { | ||
| 4 | __u8 version; | ||
| 5 | __u8 pad01[3]; | ||
| 6 | __u32 entries; | ||
| 7 | __u32 get; | ||
| 8 | __u32 put; | ||
| 9 | }; | ||
| 10 | |||
| 11 | #define NVB069_V0_NTFY_FAULT 0x00 | ||
| 12 | #endif | ||
| @@ -3,7 +3,8 @@ · 2 unmodified lines | |||
| 3 | struct nvif_vmm_v0 { | 3 | struct nvif_vmm_v0 { |
| 4 | __u8 version; | 4 | __u8 version; |
| 5 | __u8 page_nr; | 5 | __u8 page_nr; |
| 6 | __u8 pad02[6]; | 6 | __u8 managed; |
| 7 | __u8 pad03[5]; | ||
| 7 | __u64 addr; | 8 | __u64 addr; |
| 8 | __u64 size; | 9 | __u64 size; |
| 9 | __u8 data[]; | 10 | __u8 data[]; |
| @@ -14,6 +15,9 @@ struct nvif_vmm_v0 { · 4 unmodified lines | |||
| 14 | #define NVIF_VMM_V0_PUT 0x02 | 15 | #define NVIF_VMM_V0_PUT 0x02 |
| 15 | #define NVIF_VMM_V0_MAP 0x03 | 16 | #define NVIF_VMM_V0_MAP 0x03 |
| 16 | #define NVIF_VMM_V0_UNMAP 0x04 | 17 | #define NVIF_VMM_V0_UNMAP 0x04 |
| 18 | #define NVIF_VMM_V0_PFNMAP 0x05 | ||
| 19 | #define NVIF_VMM_V0_PFNCLR 0x06 | ||
| 20 | #define NVIF_VMM_V0_MTHD(i) ((i) + 0x80) | ||
| 17 | 21 | ||
| 18 | struct nvif_vmm_page_v0 { | 22 | struct nvif_vmm_page_v0 { |
| 19 | __u8 version; | 23 | __u8 version; |
| @@ -61,4 +65,28 @@ struct nvif_vmm_unmap_v0 { · 41 unmodified lines | |||
| 61 | __u8 pad01[7]; | 65 | __u8 pad01[7]; |
| 62 | __u64 addr; | 66 | __u64 addr; |
| 63 | }; | 67 | }; |
| 68 | |||
| 69 | struct nvif_vmm_pfnmap_v0 { | ||
| 70 | __u8 version; | ||
| 71 | __u8 page; | ||
| 72 | __u8 pad02[6]; | ||
| 73 | __u64 addr; | ||
| 74 | __u64 size; | ||
| 75 | #define NVIF_VMM_PFNMAP_V0_ADDR 0xfffffffffffff000ULL | ||
| 76 | #define NVIF_VMM_PFNMAP_V0_ADDR_SHIFT 12 | ||
| 77 | #define NVIF_VMM_PFNMAP_V0_APER 0x00000000000000f0ULL | ||
| 78 | #define NVIF_VMM_PFNMAP_V0_HOST 0x0000000000000000ULL | ||
| 79 | #define NVIF_VMM_PFNMAP_V0_VRAM 0x0000000000000010ULL | ||
| 80 | #define NVIF_VMM_PFNMAP_V0_W 0x0000000000000002ULL | ||
| 81 | #define NVIF_VMM_PFNMAP_V0_V 0x0000000000000001ULL | ||
| 82 | #define NVIF_VMM_PFNMAP_V0_NONE 0x0000000000000000ULL | ||
| 83 | __u64 phys[]; | ||
| 84 | }; | ||
| 85 | |||
| 86 | struct nvif_vmm_pfnclr_v0 { | ||
| 87 | __u8 version; | ||
| 88 | __u8 pad01[7]; | ||
| 89 | __u64 addr; | ||
| 90 | __u64 size; | ||
| 91 | }; | ||
| 64 | #endif | 92 | #endif |
| @@ -6,6 +6,12 @@ struct gp100_vmm_vn { · 5 unmodified lines | |||
| 6 | /* nvif_vmm_vX ... */ | 6 | /* nvif_vmm_vX ... */ |
| 7 | }; | 7 | }; |
| 8 | 8 | ||
| 9 | struct gp100_vmm_v0 { | ||
| 10 | /* nvif_vmm_vX ... */ | ||
| 11 | __u8 version; | ||
| 12 | __u8 fault_replay; | ||
| 13 | }; | ||
| 14 | |||
| 9 | struct gp100_vmm_map_vn { | 15 | struct gp100_vmm_map_vn { |
| 10 | /* nvif_vmm_map_vX ... */ | 16 | /* nvif_vmm_map_vX ... */ |
| 11 | }; | 17 | }; |
| @@ -18,4 +24,19 @@ struct gp100_vmm_map_v0 { · 6 unmodified lines | |||
| 18 | __u8 priv; | 24 | __u8 priv; |
| 19 | __u8 kind; | 25 | __u8 kind; |
| 20 | }; | 26 | }; |
| 27 | |||
| 28 | #define GP100_VMM_VN_FAULT_REPLAY NVIF_VMM_V0_MTHD(0x00) | ||
| 29 | #define GP100_VMM_VN_FAULT_CANCEL NVIF_VMM_V0_MTHD(0x01) | ||
| 30 | |||
| 31 | struct gp100_vmm_fault_replay_vn { | ||
| 32 | }; | ||
| 33 | |||
| 34 | struct gp100_vmm_fault_cancel_v0 { | ||
| 35 | __u8 version; | ||
| 36 | __u8 hub; | ||
| 37 | __u8 gpc; | ||
| 38 | __u8 client; | ||
| 39 | __u8 pad04[4]; | ||
| 40 | __u64 inst; | ||
| 41 | }; | ||
| 21 | #endif | 42 | #endif |
| @@ -30,8 +30,8 @@ struct nvif_vmm { · 29 unmodified lines | |||
| 30 | int page_nr; | 30 | int page_nr; |
| 31 | }; | 31 | }; |
| 32 | 32 | ||
| 33 | int nvif_vmm_init(struct nvif_mmu *, s32 oclass, u64 addr, u64 size, | 33 | int nvif_vmm_init(struct nvif_mmu *, s32 oclass, bool managed, u64 addr, |
| 34 | void *argv, u32 argc, struct nvif_vmm *); | 34 | u64 size, void *argv, u32 argc, struct nvif_vmm *); |
| 35 | void nvif_vmm_fini(struct nvif_vmm *); | 35 | void nvif_vmm_fini(struct nvif_vmm *); |
| 36 | int nvif_vmm_get(struct nvif_vmm *, enum nvif_vmm_get, bool sparse, | 36 | int nvif_vmm_get(struct nvif_vmm *, enum nvif_vmm_get, bool sparse, |
| 37 | u8 page, u8 align, u64 size, struct nvif_vma *); | 37 | u8 page, u8 align, u64 size, struct nvif_vma *); |
| @@ -28,6 +28,7 @@ enum nvkm_devidx { · 27 unmodified lines | |||
| 28 | NVKM_SUBDEV_ICCSENSE, | 28 | NVKM_SUBDEV_ICCSENSE, |
| 29 | NVKM_SUBDEV_THERM, | 29 | NVKM_SUBDEV_THERM, |
| 30 | NVKM_SUBDEV_CLK, | 30 | NVKM_SUBDEV_CLK, |
| 31 | NVKM_SUBDEV_GSP, | ||
| 31 | NVKM_SUBDEV_SECBOOT, | 32 | NVKM_SUBDEV_SECBOOT, |
| 32 | 33 | ||
| 33 | NVKM_ENGINE_BSP, | 34 | NVKM_ENGINE_BSP, |
| @@ -137,6 +138,7 @@ struct nvkm_device { · 103 unmodified lines | |||
| 137 | struct nvkm_fb *fb; | 138 | struct nvkm_fb *fb; |
| 138 | struct nvkm_fuse *fuse; | 139 | struct nvkm_fuse *fuse; |
| 139 | struct nvkm_gpio *gpio; | 140 | struct nvkm_gpio *gpio; |
| 141 | struct nvkm_gsp *gsp; | ||
| 140 | struct nvkm_i2c *i2c; | 142 | struct nvkm_i2c *i2c; |
| 141 | struct nvkm_subdev *ibus; | 143 | struct nvkm_subdev *ibus; |
| 142 | struct nvkm_iccsense *iccsense; | 144 | struct nvkm_iccsense *iccsense; |
| @@ -209,6 +211,7 @@ struct nvkm_device_chip { · 66 unmodified lines | |||
| 209 | int (*fb )(struct nvkm_device *, int idx, struct nvkm_fb **); | 211 | int (*fb )(struct nvkm_device *, int idx, struct nvkm_fb **); |
| 210 | int (*fuse )(struct nvkm_device *, int idx, struct nvkm_fuse **); | 212 | int (*fuse )(struct nvkm_device *, int idx, struct nvkm_fuse **); |
| 211 | int (*gpio )(struct nvkm_device *, int idx, struct nvkm_gpio **); | 213 | int (*gpio )(struct nvkm_device *, int idx, struct nvkm_gpio **); |
| 214 | int (*gsp )(struct nvkm_device *, int idx, struct nvkm_gsp **); | ||
| 212 | int (*i2c )(struct nvkm_device *, int idx, struct nvkm_i2c **); | 215 | int (*i2c )(struct nvkm_device *, int idx, struct nvkm_i2c **); |
| 213 | int (*ibus )(struct nvkm_device *, int idx, struct nvkm_subdev **); | 216 | int (*ibus )(struct nvkm_device *, int idx, struct nvkm_subdev **); |
| 214 | int (*iccsense)(struct nvkm_device *, int idx, struct nvkm_iccsense **); | 217 | int (*iccsense)(struct nvkm_device *, int idx, struct nvkm_iccsense **); |
| @@ -11,5 +11,5 @@ int gm200_ce_new(struct nvkm_device *, int, struct nvkm_engine **); · 10 unmodified lines | |||
| 11 | int gp100_ce_new(struct nvkm_device *, int, struct nvkm_engine **); | 11 | int gp100_ce_new(struct nvkm_device *, int, struct nvkm_engine **); |
| 12 | int gp102_ce_new(struct nvkm_device *, int, struct nvkm_engine **); | 12 | int gp102_ce_new(struct nvkm_device *, int, struct nvkm_engine **); |
| 13 | int gv100_ce_new(struct nvkm_device *, int, struct nvkm_engine **); | 13 | int gv100_ce_new(struct nvkm_device *, int, struct nvkm_engine **); |
| 14 | int tu104_ce_new(struct nvkm_device *, int, struct nvkm_engine **); | 14 | int tu102_ce_new(struct nvkm_device *, int, struct nvkm_engine **); |
| 15 | #endif | 15 | #endif |
| @@ -36,5 +36,5 @@ int gm200_disp_new(struct nvkm_device *, int, struct nvkm_disp **); · 35 unmodified lines | |||
| 36 | int gp100_disp_new(struct nvkm_device *, int, struct nvkm_disp **); | 36 | int gp100_disp_new(struct nvkm_device *, int, struct nvkm_disp **); |
| 37 | int gp102_disp_new(struct nvkm_device *, int, struct nvkm_disp **); | 37 | int gp102_disp_new(struct nvkm_device *, int, struct nvkm_disp **); |
| 38 | int gv100_disp_new(struct nvkm_device *, int, struct nvkm_disp **); | 38 | int gv100_disp_new(struct nvkm_device *, int, struct nvkm_disp **); |
| 39 | int tu104_disp_new(struct nvkm_device *, int, struct nvkm_disp **); | 39 | int tu102_disp_new(struct nvkm_device *, int, struct nvkm_disp **); |
| 40 | #endif | 40 | #endif |
| @@ -74,5 +74,5 @@ int gm20b_fifo_new(struct nvkm_device *, int, struct nvkm_fifo **); · 73 unmodified lines | |||
| 74 | int gp100_fifo_new(struct nvkm_device *, int, struct nvkm_fifo **); | 74 | int gp100_fifo_new(struct nvkm_device *, int, struct nvkm_fifo **); |
| 75 | int gp10b_fifo_new(struct nvkm_device *, int, struct nvkm_fifo **); | 75 | int gp10b_fifo_new(struct nvkm_device *, int, struct nvkm_fifo **); |
| 76 | int gv100_fifo_new(struct nvkm_device *, int, struct nvkm_fifo **); | 76 | int gv100_fifo_new(struct nvkm_device *, int, struct nvkm_fifo **); |
| 77 | int tu104_fifo_new(struct nvkm_device *, int, struct nvkm_fifo **); | 77 | int tu102_fifo_new(struct nvkm_device *, int, struct nvkm_fifo **); |
| 78 | #endif | 78 | #endif |
| @@ -10,6 +10,9 @@ struct nvkm_gr { · 9 unmodified lines | |||
| 10 | 10 | ||
| 11 | u64 nvkm_gr_units(struct nvkm_gr *); | 11 | u64 nvkm_gr_units(struct nvkm_gr *); |
| 12 | int nvkm_gr_tlb_flush(struct nvkm_gr *); | 12 | int nvkm_gr_tlb_flush(struct nvkm_gr *); |
| 13 | int nvkm_gr_ctxsw_pause(struct nvkm_device *); | ||
| 14 | int nvkm_gr_ctxsw_resume(struct nvkm_device *); | ||
| 15 | u32 nvkm_gr_ctxsw_inst(struct nvkm_device *); | ||
| 13 | 16 | ||
| 14 | int nv04_gr_new(struct nvkm_device *, int, struct nvkm_gr **); | 17 | int nv04_gr_new(struct nvkm_device *, int, struct nvkm_gr **); |
| 15 | int nv10_gr_new(struct nvkm_device *, int, struct nvkm_gr **); | 18 | int nv10_gr_new(struct nvkm_device *, int, struct nvkm_gr **); |
| @@ -6,6 +6,8 @@ · 5 unmodified lines | |||
| 6 | 6 | ||
| 7 | struct nvkm_nvdec { | 7 | struct nvkm_nvdec { |
| 8 | struct nvkm_engine engine; | 8 | struct nvkm_engine engine; |
| 9 | u32 addr; | ||
| 10 | |||
| 9 | struct nvkm_falcon *falcon; | 11 | struct nvkm_falcon *falcon; |
| 10 | }; | 12 | }; |
| 11 | 13 | ||
| @@ -5,10 +5,13 @@ · 4 unmodified lines | |||
| 5 | 5 | ||
| 6 | struct nvkm_sec2 { | 6 | struct nvkm_sec2 { |
| 7 | struct nvkm_engine engine; | 7 | struct nvkm_engine engine; |
| 8 | u32 addr; | ||
| 9 | |||
| 8 | struct nvkm_falcon *falcon; | 10 | struct nvkm_falcon *falcon; |
| 9 | struct nvkm_msgqueue *queue; | 11 | struct nvkm_msgqueue *queue; |
| 10 | struct work_struct work; | 12 | struct work_struct work; |
| 11 | }; | 13 | }; |
| 12 | 14 | ||
| 13 | int gp102_sec2_new(struct nvkm_device *, int, struct nvkm_sec2 **); | 15 | int gp102_sec2_new(struct nvkm_device *, int, struct nvkm_sec2 **); |
| 16 | int tu102_sec2_new(struct nvkm_device *, int, struct nvkm_sec2 **); | ||
| 14 | #endif | 17 | #endif |
| @@ -29,5 +29,5 @@ int gf100_bar_new(struct nvkm_device *, int, struct nvkm_bar **); · 28 unmodified lines | |||
| 29 | int gk20a_bar_new(struct nvkm_device *, int, struct nvkm_bar **); | 29 | int gk20a_bar_new(struct nvkm_device *, int, struct nvkm_bar **); |
| 30 | int gm107_bar_new(struct nvkm_device *, int, struct nvkm_bar **); | 30 | int gm107_bar_new(struct nvkm_device *, int, struct nvkm_bar **); |
| 31 | int gm20b_bar_new(struct nvkm_device *, int, struct nvkm_bar **); | 31 | int gm20b_bar_new(struct nvkm_device *, int, struct nvkm_bar **); |
| 32 | int tu104_bar_new(struct nvkm_device *, int, struct nvkm_bar **); | 32 | int tu102_bar_new(struct nvkm_device *, int, struct nvkm_bar **); |
| 33 | #endif | 33 | #endif |
| @@ -31,5 +31,5 @@ int gf100_devinit_new(struct nvkm_device *, int, struct nvkm_devinit **); · 30 unmodified lines | |||
| 31 | int gm107_devinit_new(struct nvkm_device *, int, struct nvkm_devinit **); | 31 | int gm107_devinit_new(struct nvkm_device *, int, struct nvkm_devinit **); |
| 32 | int gm200_devinit_new(struct nvkm_device *, int, struct nvkm_devinit **); | 32 | int gm200_devinit_new(struct nvkm_device *, int, struct nvkm_devinit **); |
| 33 | int gv100_devinit_new(struct nvkm_device *, int, struct nvkm_devinit **); | 33 | int gv100_devinit_new(struct nvkm_device *, int, struct nvkm_devinit **); |
| 34 | int tu104_devinit_new(struct nvkm_device *, int, struct nvkm_devinit **); | 34 | int tu102_devinit_new(struct nvkm_device *, int, struct nvkm_devinit **); |
| 35 | #endif | 35 | #endif |
| @@ -13,6 +13,8 @@ struct nvkm_fault { · 12 unmodified lines | |||
| 13 | struct nvkm_event event; | 13 | struct nvkm_event event; |
| 14 | 14 | ||
| 15 | struct nvkm_notify nrpfb; | 15 | struct nvkm_notify nrpfb; |
| 16 | |||
| 17 | struct nvkm_device_oclass user; | ||
| 16 | }; | 18 | }; |
| 17 | 19 | ||
| 18 | struct nvkm_fault_data { | 20 | struct nvkm_fault_data { |
| @@ -30,5 +32,5 @@ struct nvkm_fault_data { · 11 unmodified lines | |||
| 30 | 32 | ||
| 31 | int gp100_fault_new(struct nvkm_device *, int, struct nvkm_fault **); | 33 | int gp100_fault_new(struct nvkm_device *, int, struct nvkm_fault **); |
| 32 | int gv100_fault_new(struct nvkm_device *, int, struct nvkm_fault **); | 34 | int gv100_fault_new(struct nvkm_device *, int, struct nvkm_fault **); |
| 33 | int tu104_fault_new(struct nvkm_device *, int, struct nvkm_fault **); | 35 | int tu102_fault_new(struct nvkm_device *, int, struct nvkm_fault **); |
| 34 | #endif | 36 | #endif |
| @@ -0,0 +1,14 @@ | |||
| 1 | #ifndef __NVKM_GSP_H__ | ||
| 2 | #define __NVKM_GSP_H__ | ||
| 3 | #define nvkm_gsp(p) container_of((p), struct nvkm_gsp, subdev) | ||
| 4 | #include <core/subdev.h> | ||
| 5 | |||
| 6 | struct nvkm_gsp { | ||
| 7 | struct nvkm_subdev subdev; | ||
| 8 | u32 addr; | ||
| 9 | |||
| 10 | struct nvkm_falcon *falcon; | ||
| 11 | }; | ||
| 12 | |||
| 13 | int gv100_gsp_new(struct nvkm_device *, int, struct nvkm_gsp **); | ||
| 14 | #endif | ||
| @@ -31,5 +31,5 @@ int gk104_mc_new(struct nvkm_device *, int, struct nvkm_mc **); · 30 unmodified lines | |||
| 31 | int gk20a_mc_new(struct nvkm_device *, int, struct nvkm_mc **); | 31 | int gk20a_mc_new(struct nvkm_device *, int, struct nvkm_mc **); |
| 32 | int gp100_mc_new(struct nvkm_device *, int, struct nvkm_mc **); | 32 | int gp100_mc_new(struct nvkm_device *, int, struct nvkm_mc **); |
| 33 | int gp10b_mc_new(struct nvkm_device *, int, struct nvkm_mc **); | 33 | int gp10b_mc_new(struct nvkm_device *, int, struct nvkm_mc **); |
| 34 | int tu104_mc_new(struct nvkm_device *, int, struct nvkm_mc **); | 34 | int tu102_mc_new(struct nvkm_device *, int, struct nvkm_mc **); |
| 35 | #endif | 35 | #endif |
| @@ -17,6 +17,7 @@ struct nvkm_vma { · 16 unmodified lines | |||
| 17 | bool part:1; /* Region was split from an allocated region by map(). */ | 17 | bool part:1; /* Region was split from an allocated region by map(). */ |
| 18 | bool user:1; /* Region user-allocated. */ | 18 | bool user:1; /* Region user-allocated. */ |
| 19 | bool busy:1; /* Region busy (for temporarily preventing user access). */ | 19 | bool busy:1; /* Region busy (for temporarily preventing user access). */ |
| 20 | bool mapped:1; /* Region contains valid pages. */ | ||
| 20 | struct nvkm_memory *memory; /* Memory currently mapped into VMA. */ | 21 | struct nvkm_memory *memory; /* Memory currently mapped into VMA. */ |
| 21 | struct nvkm_tags *tags; /* Compression tag reference. */ | 22 | struct nvkm_tags *tags; /* Compression tag reference. */ |
| 22 | }; | 23 | }; |
| @@ -44,6 +45,8 @@ struct nvkm_vmm { · 21 unmodified lines | |||
| 44 | 45 | ||
| 45 | dma_addr_t null; | 46 | dma_addr_t null; |
| 46 | void *nullp; | 47 | void *nullp; |
| 48 | |||
| 49 | bool replay; | ||
| 47 | }; | 50 | }; |
| 48 | 51 | ||
| 49 | int nvkm_vmm_new(struct nvkm_device *, u64 addr, u64 size, void *argv, u32 argc, | 52 | int nvkm_vmm_new(struct nvkm_device *, u64 addr, u64 size, void *argv, u32 argc, |
| @@ -63,6 +66,7 @@ struct nvkm_vmm_map { · 13 unmodified lines | |||
| 63 | struct nvkm_mm_node *mem; | 66 | struct nvkm_mm_node *mem; |
| 64 | struct scatterlist *sgl; | 67 | struct scatterlist *sgl; |
| 65 | dma_addr_t *dma; | 68 | dma_addr_t *dma; |
| 69 | u64 *pfn; | ||
| 66 | u64 off; | 70 | u64 off; |
| 67 | 71 | ||
| 68 | const struct nvkm_vmm_page *page; | 72 | const struct nvkm_vmm_page *page; |
| @@ -130,5 +134,5 @@ int gm20b_mmu_new(struct nvkm_device *, int, struct nvkm_mmu **); · 61 unmodified lines | |||
| 130 | int gp100_mmu_new(struct nvkm_device *, int, struct nvkm_mmu **); | 134 | int gp100_mmu_new(struct nvkm_device *, int, struct nvkm_mmu **); |
| 131 | int gp10b_mmu_new(struct nvkm_device *, int, struct nvkm_mmu **); | 135 | int gp10b_mmu_new(struct nvkm_device *, int, struct nvkm_mmu **); |
| 132 | int gv100_mmu_new(struct nvkm_device *, int, struct nvkm_mmu **); | 136 | int gv100_mmu_new(struct nvkm_device *, int, struct nvkm_mmu **); |
| 133 | int tu104_mmu_new(struct nvkm_device *, int, struct nvkm_mmu **); | 137 | int tu102_mmu_new(struct nvkm_device *, int, struct nvkm_mmu **); |
| 134 | #endif | 138 | #endif |
| @@ -9,6 +9,7 @@ struct nvkm_top { · 8 unmodified lines | |||
| 9 | struct list_head device; | 9 | struct list_head device; |
| 10 | }; | 10 | }; |
| 11 | 11 | ||
| 12 | u32 nvkm_top_addr(struct nvkm_device *, enum nvkm_devidx); | ||
| 12 | u32 nvkm_top_reset(struct nvkm_device *, enum nvkm_devidx); | 13 | u32 nvkm_top_reset(struct nvkm_device *, enum nvkm_devidx); |
| 13 | u32 nvkm_top_intr(struct nvkm_device *, u32 intr, u64 *subdevs); | 14 | u32 nvkm_top_intr(struct nvkm_device *, u32 intr, u64 *subdevs); |
| 14 | u32 nvkm_top_intr_mask(struct nvkm_device *, enum nvkm_devidx); | 15 | u32 nvkm_top_intr_mask(struct nvkm_device *, enum nvkm_devidx); |
| @@ -38,6 +38,7 @@ int nvkm_volt_set_id(struct nvkm_volt *, u8 id, u8 min_id, u8 temp, · 37 unmodified lines | |||
| 38 | 38 | ||
| 39 | int nv40_volt_new(struct nvkm_device *, int, struct nvkm_volt **); | 39 | int nv40_volt_new(struct nvkm_device *, int, struct nvkm_volt **); |
| 40 | int gf100_volt_new(struct nvkm_device *, int, struct nvkm_volt **); | 40 | int gf100_volt_new(struct nvkm_device *, int, struct nvkm_volt **); |
| 41 | int gf117_volt_new(struct nvkm_device *, int, struct nvkm_volt **); | ||
| 41 | int gk104_volt_new(struct nvkm_device *, int, struct nvkm_volt **); | 42 | int gk104_volt_new(struct nvkm_device *, int, struct nvkm_volt **); |
| 42 | int gk20a_volt_new(struct nvkm_device *, int, struct nvkm_volt **); | 43 | int gk20a_volt_new(struct nvkm_device *, int, struct nvkm_volt **); |
| 43 | int gm20b_volt_new(struct nvkm_device *, int, struct nvkm_volt **); | 44 | int gm20b_volt_new(struct nvkm_device *, int, struct nvkm_volt **); |
| @@ -214,6 +214,7 @@ nouveau_abi16_ioctl_getparam(ABI16_IOCTL_ARGS) · 213 unmodified lines | |||
| 214 | WARN_ON(1); | 214 | WARN_ON(1); |
| 215 | break; | 215 | break; |
| 216 | } | 216 | } |
| 217 | break; | ||
| 217 | case NOUVEAU_GETPARAM_FB_SIZE: | 218 | case NOUVEAU_GETPARAM_FB_SIZE: |
| 218 | getparam->value = drm->gem.vram_available; | 219 | getparam->value = drm->gem.vram_available; |
| 219 | break; | 220 | break; |
| @@ -338,7 +339,8 @@ nouveau_abi16_ioctl_channel_alloc(ABI16_IOCTL_ARGS) · 118 unmodified lines | |||
| 338 | goto done; | 339 | goto done; |
| 339 | 340 | ||
| 340 | if (device->info.family >= NV_DEVICE_INFO_V0_TESLA) { | 341 | if (device->info.family >= NV_DEVICE_INFO_V0_TESLA) { |
| 341 | ret = nouveau_vma_new(chan->ntfy, &cli->vmm, &chan->ntfy_vma); | 342 | ret = nouveau_vma_new(chan->ntfy, chan->chan->vmm, |
| 343 | &chan->ntfy_vma); | ||
| 342 | if (ret) | 344 | if (ret) |
| 343 | goto done; | 345 | goto done; |
| 344 | } | 346 | } |
| @@ -194,7 +194,7 @@ nouveau_bo_new(struct nouveau_cli *cli, u64 size, int align, · 193 unmodified lines | |||
| 194 | struct nouveau_drm *drm = cli->drm; | 194 | struct nouveau_drm *drm = cli->drm; |
| 195 | struct nouveau_bo *nvbo; | 195 | struct nouveau_bo *nvbo; |
| 196 | struct nvif_mmu *mmu = &cli->mmu; | 196 | struct nvif_mmu *mmu = &cli->mmu; |
| 197 | struct nvif_vmm *vmm = &cli->vmm.vmm; | 197 | struct nvif_vmm *vmm = cli->svm.cli ? &cli->svm.vmm : &cli->vmm.vmm; |
| 198 | size_t acc_size; | 198 | size_t acc_size; |
| 199 | int type = ttm_bo_type_device; | 199 | int type = ttm_bo_type_device; |
| 200 | int ret, i, pi = -1; | 200 | int ret, i, pi = -1; |
| @@ -1434,7 +1434,7 @@ nouveau_ttm_io_mem_reserve(struct ttm_bo_device *bdev, struct ttm_mem_reg *reg) · 1233 unmodified lines | |||
| 1434 | if (drm->client.mem->oclass < NVIF_CLASS_MEM_NV50 || !mem->kind) | 1434 | if (drm->client.mem->oclass < NVIF_CLASS_MEM_NV50 || !mem->kind) |
| 1435 | /* untiled */ | 1435 | /* untiled */ |
| 1436 | break; | 1436 | break; |
| 1437 | /* fallthrough, tiled memory */ | 1437 | /* fall through - tiled memory */ |
| 1438 | case TTM_PL_VRAM: | 1438 | case TTM_PL_VRAM: |
| 1439 | reg->bus.offset = reg->start << PAGE_SHIFT; | 1439 | reg->bus.offset = reg->start << PAGE_SHIFT; |
| 1440 | reg->bus.base = device->func->resource_addr(device, 1); | 1440 | reg->bus.base = device->func->resource_addr(device, 1); |
| @@ -42,6 +42,7 @@ · 41 unmodified lines | |||
| 42 | #include "nouveau_fence.h" | 42 | #include "nouveau_fence.h" |
| 43 | #include "nouveau_abi16.h" | 43 | #include "nouveau_abi16.h" |
| 44 | #include "nouveau_vmm.h" | 44 | #include "nouveau_vmm.h" |
| 45 | #include "nouveau_svm.h" | ||
| 45 | 46 | ||
| 46 | MODULE_PARM_DESC(vram_pushbuf, "Create DMA push buffers in VRAM"); | 47 | MODULE_PARM_DESC(vram_pushbuf, "Create DMA push buffers in VRAM"); |
| 47 | int nouveau_vram_pushbuf; | 48 | int nouveau_vram_pushbuf; |
| @@ -95,6 +96,10 @@ nouveau_channel_del(struct nouveau_channel **pchan) · 47 unmodified lines | |||
| 95 | 96 | ||
| 96 | if (chan->fence) | 97 | if (chan->fence) |
| 97 | nouveau_fence(chan->drm)->context_del(chan); | 98 | nouveau_fence(chan->drm)->context_del(chan); |
| 99 | |||
| 100 | if (cli) | ||
| 101 | nouveau_svmm_part(chan->vmm->svmm, chan->inst); | ||
| 102 | |||
| 98 | nvif_object_fini(&chan->nvsw); | 103 | nvif_object_fini(&chan->nvsw); |
| 99 | nvif_object_fini(&chan->gart); | 104 | nvif_object_fini(&chan->gart); |
| 100 | nvif_object_fini(&chan->vram); | 105 | nvif_object_fini(&chan->vram); |
| @@ -130,6 +135,7 @@ nouveau_channel_prep(struct nouveau_drm *drm, struct nvif_device *device, · 29 unmodified lines | |||
| 130 | 135 | ||
| 131 | chan->device = device; | 136 | chan->device = device; |
| 132 | chan->drm = drm; | 137 | chan->drm = drm; |
| 138 | chan->vmm = cli->svm.cli ? &cli->svm : &cli->vmm; | ||
| 133 | atomic_set(&chan->killed, 0); | 139 | atomic_set(&chan->killed, 0); |
| 134 | 140 | ||
| 135 | /* allocate memory for dma push buffer */ | 141 | /* allocate memory for dma push buffer */ |
| @@ -157,7 +163,7 @@ nouveau_channel_prep(struct nouveau_drm *drm, struct nvif_device *device, · 21 unmodified lines | |||
| 157 | chan->push.addr = chan->push.buffer->bo.offset; | 163 | chan->push.addr = chan->push.buffer->bo.offset; |
| 158 | 164 | ||
| 159 | if (device->info.family >= NV_DEVICE_INFO_V0_TESLA) { | 165 | if (device->info.family >= NV_DEVICE_INFO_V0_TESLA) { |
| 160 | ret = nouveau_vma_new(chan->push.buffer, &cli->vmm, | 166 | ret = nouveau_vma_new(chan->push.buffer, chan->vmm, |
| 161 | &chan->push.vma); | 167 | &chan->push.vma); |
| 162 | if (ret) { | 168 | if (ret) { |
| 163 | nouveau_channel_del(pchan); | 169 | nouveau_channel_del(pchan); |
| @@ -172,7 +178,7 @@ nouveau_channel_prep(struct nouveau_drm *drm, struct nvif_device *device, · 8 unmodified lines | |||
| 172 | args.target = NV_DMA_V0_TARGET_VM; | 178 | args.target = NV_DMA_V0_TARGET_VM; |
| 173 | args.access = NV_DMA_V0_ACCESS_VM; | 179 | args.access = NV_DMA_V0_ACCESS_VM; |
| 174 | args.start = 0; | 180 | args.start = 0; |
| 175 | args.limit = cli->vmm.vmm.limit - 1; | 181 | args.limit = chan->vmm->vmm.limit - 1; |
| 176 | } else | 182 | } else |
| 177 | if (chan->push.buffer->bo.mem.mem_type == TTM_PL_VRAM) { | 183 | if (chan->push.buffer->bo.mem.mem_type == TTM_PL_VRAM) { |
| 178 | if (device->info.family == NV_DEVICE_INFO_V0_TNT) { | 184 | if (device->info.family == NV_DEVICE_INFO_V0_TNT) { |
| @@ -202,7 +208,7 @@ nouveau_channel_prep(struct nouveau_drm *drm, struct nvif_device *device, · 23 unmodified lines | |||
| 202 | args.target = NV_DMA_V0_TARGET_VM; | 208 | args.target = NV_DMA_V0_TARGET_VM; |
| 203 | args.access = NV_DMA_V0_ACCESS_RDWR; | 209 | args.access = NV_DMA_V0_ACCESS_RDWR; |
| 204 | args.start = 0; | 210 | args.start = 0; |
| 205 | args.limit = cli->vmm.vmm.limit - 1; | 211 | args.limit = chan->vmm->vmm.limit - 1; |
| 206 | } | 212 | } |
| 207 | } | 213 | } |
| 208 | 214 | ||
| @@ -220,7 +226,6 @@ static int · 11 unmodified lines | |||
| 220 | nouveau_channel_ind(struct nouveau_drm *drm, struct nvif_device *device, | 226 | nouveau_channel_ind(struct nouveau_drm *drm, struct nvif_device *device, |
| 221 | u64 runlist, bool priv, struct nouveau_channel **pchan) | 227 | u64 runlist, bool priv, struct nouveau_channel **pchan) |
| 222 | { | 228 | { |
| 223 | struct nouveau_cli *cli = (void *)device->object.client; | ||
| 224 | static const u16 oclasses[] = { TURING_CHANNEL_GPFIFO_A, | 229 | static const u16 oclasses[] = { TURING_CHANNEL_GPFIFO_A, |
| 225 | VOLTA_CHANNEL_GPFIFO_A, | 230 | VOLTA_CHANNEL_GPFIFO_A, |
| 226 | PASCAL_CHANNEL_GPFIFO_A, | 231 | PASCAL_CHANNEL_GPFIFO_A, |
| @@ -255,7 +260,7 @@ nouveau_channel_ind(struct nouveau_drm *drm, struct nvif_device *device, · 28 unmodified lines | |||
| 255 | args.volta.ilength = 0x02000; | 260 | args.volta.ilength = 0x02000; |
| 256 | args.volta.ioffset = 0x10000 + chan->push.addr; | 261 | args.volta.ioffset = 0x10000 + chan->push.addr; |
| 257 | args.volta.runlist = runlist; | 262 | args.volta.runlist = runlist; |
| 258 | args.volta.vmm = nvif_handle(&cli->vmm.vmm.object); | 263 | args.volta.vmm = nvif_handle(&chan->vmm->vmm.object); |
| 259 | args.volta.priv = priv; | 264 | args.volta.priv = priv; |
| 260 | size = sizeof(args.volta); | 265 | size = sizeof(args.volta); |
| 261 | } else | 266 | } else |
| @@ -264,7 +269,7 @@ nouveau_channel_ind(struct nouveau_drm *drm, struct nvif_device *device, · 2 unmodified lines | |||
| 264 | args.kepler.ilength = 0x02000; | 269 | args.kepler.ilength = 0x02000; |
| 265 | args.kepler.ioffset = 0x10000 + chan->push.addr; | 270 | args.kepler.ioffset = 0x10000 + chan->push.addr; |
| 266 | args.kepler.runlist = runlist; | 271 | args.kepler.runlist = runlist; |
| 267 | args.kepler.vmm = nvif_handle(&cli->vmm.vmm.object); | 272 | args.kepler.vmm = nvif_handle(&chan->vmm->vmm.object); |
| 268 | args.kepler.priv = priv; | 273 | args.kepler.priv = priv; |
| 269 | size = sizeof(args.kepler); | 274 | size = sizeof(args.kepler); |
| 270 | } else | 275 | } else |
| @@ -272,14 +277,14 @@ nouveau_channel_ind(struct nouveau_drm *drm, struct nvif_device *device, · 1 unmodified line | |||
| 272 | args.fermi.version = 0; | 277 | args.fermi.version = 0; |
| 273 | args.fermi.ilength = 0x02000; | 278 | args.fermi.ilength = 0x02000; |
| 274 | args.fermi.ioffset = 0x10000 + chan->push.addr; | 279 | args.fermi.ioffset = 0x10000 + chan->push.addr; |
| 275 | args.fermi.vmm = nvif_handle(&cli->vmm.vmm.object); | 280 | args.fermi.vmm = nvif_handle(&chan->vmm->vmm.object); |
| 276 | size = sizeof(args.fermi); | 281 | size = sizeof(args.fermi); |
| 277 | } else { | 282 | } else { |
| 278 | args.nv50.version = 0; | 283 | args.nv50.version = 0; |
| 279 | args.nv50.ilength = 0x02000; | 284 | args.nv50.ilength = 0x02000; |
| 280 | args.nv50.ioffset = 0x10000 + chan->push.addr; | 285 | args.nv50.ioffset = 0x10000 + chan->push.addr; |
| 281 | args.nv50.pushbuf = nvif_handle(&chan->push.ctxdma); | 286 | args.nv50.pushbuf = nvif_handle(&chan->push.ctxdma); |
| 282 | args.nv50.vmm = nvif_handle(&cli->vmm.vmm.object); | 287 | args.nv50.vmm = nvif_handle(&chan->vmm->vmm.object); |
| 283 | size = sizeof(args.nv50); | 288 | size = sizeof(args.nv50); |
| 284 | } | 289 | } |
| 285 | 290 | ||
| @@ -350,7 +355,6 @@ static int · 64 unmodified lines | |||
| 350 | nouveau_channel_init(struct nouveau_channel *chan, u32 vram, u32 gart) | 355 | nouveau_channel_init(struct nouveau_channel *chan, u32 vram, u32 gart) |
| 351 | { | 356 | { |
| 352 | struct nvif_device *device = chan->device; | 357 | struct nvif_device *device = chan->device; |
| 353 | struct nouveau_cli *cli = (void *)chan->user.client; | ||
| 354 | struct nouveau_drm *drm = chan->drm; | 358 | struct nouveau_drm *drm = chan->drm; |
| 355 | struct nv_dma_v0 args = {}; | 359 | struct nv_dma_v0 args = {}; |
| 356 | int ret, i; | 360 | int ret, i; |
| @@ -376,7 +380,7 @@ nouveau_channel_init(struct nouveau_channel *chan, u32 vram, u32 gart) · 19 unmodified lines | |||
| 376 | args.target = NV_DMA_V0_TARGET_VM; | 380 | args.target = NV_DMA_V0_TARGET_VM; |
| 377 | args.access = NV_DMA_V0_ACCESS_VM; | 381 | args.access = NV_DMA_V0_ACCESS_VM; |
| 378 | args.start = 0; | 382 | args.start = 0; |
| 379 | args.limit = cli->vmm.vmm.limit - 1; | 383 | args.limit = chan->vmm->vmm.limit - 1; |
| 380 | } else { | 384 | } else { |
| 381 | args.target = NV_DMA_V0_TARGET_VRAM; | 385 | args.target = NV_DMA_V0_TARGET_VRAM; |
| 382 | args.access = NV_DMA_V0_ACCESS_RDWR; | 386 | args.access = NV_DMA_V0_ACCESS_RDWR; |
| @@ -393,7 +397,7 @@ nouveau_channel_init(struct nouveau_channel *chan, u32 vram, u32 gart) · 10 unmodified lines | |||
| 393 | args.target = NV_DMA_V0_TARGET_VM; | 397 | args.target = NV_DMA_V0_TARGET_VM; |
| 394 | args.access = NV_DMA_V0_ACCESS_VM; | 398 | args.access = NV_DMA_V0_ACCESS_VM; |
| 395 | args.start = 0; | 399 | args.start = 0; |
| 396 | args.limit = cli->vmm.vmm.limit - 1; | 400 | args.limit = chan->vmm->vmm.limit - 1; |
| 397 | } else | 401 | } else |
| 398 | if (chan->drm->agp.bridge) { | 402 | if (chan->drm->agp.bridge) { |
| 399 | args.target = NV_DMA_V0_TARGET_AGP; | 403 | args.target = NV_DMA_V0_TARGET_AGP; |
| @@ -405,7 +409,7 @@ nouveau_channel_init(struct nouveau_channel *chan, u32 vram, u32 gart) · 5 unmodified lines | |||
| 405 | args.target = NV_DMA_V0_TARGET_VM; | 409 | args.target = NV_DMA_V0_TARGET_VM; |
| 406 | args.access = NV_DMA_V0_ACCESS_RDWR; | 410 | args.access = NV_DMA_V0_ACCESS_RDWR; |
| 407 | args.start = 0; | 411 | args.start = 0; |
| 408 | args.limit = cli->vmm.vmm.limit - 1; | 412 | args.limit = chan->vmm->vmm.limit - 1; |
| 409 | } | 413 | } |
| 410 | 414 | ||
| 411 | ret = nvif_object_init(&chan->user, gart, NV_DMA_IN_MEMORY, | 415 | ret = nvif_object_init(&chan->user, gart, NV_DMA_IN_MEMORY, |
| @@ -495,6 +499,10 @@ nouveau_channel_new(struct nouveau_drm *drm, struct nvif_device *device, · 83 unmodified lines | |||
| 495 | nouveau_channel_del(pchan); | 499 | nouveau_channel_del(pchan); |
| 496 | } | 500 | } |
| 497 | 501 | ||
| 502 | ret = nouveau_svmm_join((*pchan)->vmm->svmm, (*pchan)->inst); | ||
| 503 | if (ret) | ||
| 504 | nouveau_channel_del(pchan); | ||
| 505 | |||
| 498 | done: | 506 | done: |
| 499 | cli->base.super = super; | 507 | cli->base.super = super; |
| 500 | return ret; | 508 | return ret; |
| @@ -8,6 +8,7 @@ struct nvif_device; · 7 unmodified lines | |||
| 8 | struct nouveau_channel { | 8 | struct nouveau_channel { |
| 9 | struct nvif_device *device; | 9 | struct nvif_device *device; |
| 10 | struct nouveau_drm *drm; | 10 | struct nouveau_drm *drm; |
| 11 | struct nouveau_vmm *vmm; | ||
| 11 | 12 | ||
| 12 | int chid; | 13 | int chid; |
| 13 | u64 inst; | 14 | u64 inst; |
| @@ -32,18 +32,13 @@ · 31 unmodified lines | |||
| 32 | #include <drm/drm_fb_helper.h> | 32 | #include <drm/drm_fb_helper.h> |
| 33 | #include <drm/drm_probe_helper.h> | 33 | #include <drm/drm_probe_helper.h> |
| 34 | 34 | ||
| 35 | #include <nvif/class.h> | ||
| 36 | |||
| 37 | #include "nouveau_fbcon.h" | 35 | #include "nouveau_fbcon.h" |
| 38 | #include "dispnv04/hw.h" | ||
| 39 | #include "nouveau_crtc.h" | 36 | #include "nouveau_crtc.h" |
| 40 | #include "nouveau_dma.h" | ||
| 41 | #include "nouveau_gem.h" | 37 | #include "nouveau_gem.h" |
| 42 | #include "nouveau_connector.h" | 38 | #include "nouveau_connector.h" |
| 43 | #include "nv50_display.h" | 39 | #include "nv50_display.h" |
| 44 | 40 | ||
| 45 | #include "nouveau_fence.h" | 41 | #include <nvif/class.h> |
| 46 | |||
| 47 | #include <nvif/cl0046.h> | 42 | #include <nvif/cl0046.h> |
| 48 | #include <nvif/event.h> | 43 | #include <nvif/event.h> |
| 49 | 44 | ||
| @@ -412,15 +407,14 @@ nouveau_display_acpi_ntfy(struct notifier_block *nb, unsigned long val, · 362 unmodified lines | |||
| 412 | #endif | 407 | #endif |
| 413 | 408 | ||
| 414 | int | 409 | int |
| 415 | nouveau_display_init(struct drm_device *dev) | 410 | nouveau_display_init(struct drm_device *dev, bool resume, bool runtime) |
| 416 | { | 411 | { |
| 417 | struct nouveau_display *disp = nouveau_display(dev); | 412 | struct nouveau_display *disp = nouveau_display(dev); |
| 418 | struct nouveau_drm *drm = nouveau_drm(dev); | ||
| 419 | struct drm_connector *connector; | 413 | struct drm_connector *connector; |
| 420 | struct drm_connector_list_iter conn_iter; | 414 | struct drm_connector_list_iter conn_iter; |
| 421 | int ret; | 415 | int ret; |
| 422 | 416 | ||
| 423 | ret = disp->init(dev); | 417 | ret = disp->init(dev, resume, runtime); |
| 424 | if (ret) | 418 | if (ret) |
| 425 | return ret; | 419 | return ret; |
| 426 | 420 | ||
| @@ -437,8 +431,6 @@ nouveau_display_init(struct drm_device *dev) · 10 unmodified lines | |||
| 437 | } | 431 | } |
| 438 | drm_connector_list_iter_end(&conn_iter); | 432 | drm_connector_list_iter_end(&conn_iter); |
| 439 | 433 | ||
| 440 | /* enable flip completion events */ | ||
| 441 | nvif_notify_get(&drm->flip); | ||
| 442 | return ret; | 434 | return ret; |
| 443 | } | 435 | } |
| 444 | 436 | ||
| @@ -457,9 +449,6 @@ nouveau_display_fini(struct drm_device *dev, bool suspend, bool runtime) · 12 unmodified lines | |||
| 457 | drm_helper_force_disable_all(dev); | 449 | drm_helper_force_disable_all(dev); |
| 458 | } | 450 | } |
| 459 | 451 | ||
| 460 | /* disable flip completion events */ | ||
| 461 | nvif_notify_put(&drm->flip); | ||
| 462 | |||
| 463 | /* disable hotplug interrupts */ | 452 | /* disable hotplug interrupts */ |
| 464 | drm_connector_list_iter_begin(dev, &conn_iter); | 453 | drm_connector_list_iter_begin(dev, &conn_iter); |
| 465 | nouveau_for_each_non_mst_connector_iter(connector, &conn_iter) { | 454 | nouveau_for_each_non_mst_connector_iter(connector, &conn_iter) { |
| @@ -472,7 +461,7 @@ nouveau_display_fini(struct drm_device *dev, bool suspend, bool runtime) · 6 unmodified lines | |||
| 472 | cancel_work_sync(&drm->hpd_work); | 461 | cancel_work_sync(&drm->hpd_work); |
| 473 | 462 | ||
| 474 | drm_kms_helper_poll_disable(dev); | 463 | drm_kms_helper_poll_disable(dev); |
| 475 | disp->fini(dev); | 464 | disp->fini(dev, suspend); |
| 476 | } | 465 | } |
| 477 | 466 | ||
| 478 | static void | 467 | static void |
| @@ -625,7 +614,6 @@ int · 146 unmodified lines | |||
| 625 | nouveau_display_suspend(struct drm_device *dev, bool runtime) | 614 | nouveau_display_suspend(struct drm_device *dev, bool runtime) |
| 626 | { | 615 | { |
| 627 | struct nouveau_display *disp = nouveau_display(dev); | 616 | struct nouveau_display *disp = nouveau_display(dev); |
| 628 | struct drm_crtc *crtc; | ||
| 629 | 617 | ||
| 630 | if (drm_drv_uses_atomic_modeset(dev)) { | 618 | if (drm_drv_uses_atomic_modeset(dev)) { |
| 631 | if (!runtime) { | 619 | if (!runtime) { |
| @@ -636,32 +624,9 @@ nouveau_display_suspend(struct drm_device *dev, bool runtime) · 4 unmodified lines | |||
| 636 | return ret; | 624 | return ret; |
| 637 | } | 625 | } |
| 638 | } | 626 | } |
| 639 | |||
| 640 | nouveau_display_fini(dev, true, runtime); | ||
| 641 | return 0; | ||
| 642 | } | 627 | } |
| 643 | 628 | ||
| 644 | nouveau_display_fini(dev, true, runtime); | 629 | nouveau_display_fini(dev, true, runtime); |
| 645 | |||
| 646 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | ||
| 647 | struct nouveau_framebuffer *nouveau_fb; | ||
| 648 | |||
| 649 | nouveau_fb = nouveau_framebuffer(crtc->primary->fb); | ||
| 650 | if (!nouveau_fb || !nouveau_fb->nvbo) | ||
| 651 | continue; | ||
| 652 | |||
| 653 | nouveau_bo_unpin(nouveau_fb->nvbo); | ||
| 654 | } | ||
| 655 | |||
| 656 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | ||
| 657 | struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | ||
| 658 | if (nv_crtc->cursor.nvbo) { | ||
| 659 | if (nv_crtc->cursor.set_offset) | ||
| 660 | nouveau_bo_unmap(nv_crtc->cursor.nvbo); | ||
| 661 | nouveau_bo_unpin(nv_crtc->cursor.nvbo); | ||
| 662 | } | ||
| 663 | } | ||
| 664 | |||
| 665 | return 0; | 630 | return 0; |
| 666 | } | 631 | } |
| 667 | 632 | ||
| @@ -669,275 +634,16 @@ void · 1 unmodified line | |||
| 669 | nouveau_display_resume(struct drm_device *dev, bool runtime) | 634 | nouveau_display_resume(struct drm_device *dev, bool runtime) |
| 670 | { | 635 | { |
| 671 | struct nouveau_display *disp = nouveau_display(dev); | 636 | struct nouveau_display *disp = nouveau_display(dev); |
| 672 | struct nouveau_drm *drm = nouveau_drm(dev); | 637 | |
| 673 | struct drm_crtc *crtc; | 638 | nouveau_display_init(dev, true, runtime); |
| 674 | int ret; | ||
| 675 | 639 | ||
| 676 | if (drm_drv_uses_atomic_modeset(dev)) { | 640 | if (drm_drv_uses_atomic_modeset(dev)) { |
| 677 | nouveau_display_init(dev); | ||
| 678 | if (disp->suspend) { | 641 | if (disp->suspend) { |
| 679 | drm_atomic_helper_resume(dev, disp->suspend); | 642 | drm_atomic_helper_resume(dev, disp->suspend); |
| 680 | disp->suspend = NULL; | 643 | disp->suspend = NULL; |
| 681 | } | 644 | } |
| 682 | return; | 645 | return; |
| 683 | } | 646 | } |
| 684 | |||
| 685 | /* re-pin fb/cursors */ | ||
| 686 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | ||
| 687 | struct nouveau_framebuffer *nouveau_fb; | ||
| 688 | |||
| 689 | nouveau_fb = nouveau_framebuffer(crtc->primary->fb); | ||
| 690 | if (!nouveau_fb || !nouveau_fb->nvbo) | ||
| 691 | continue; | ||
| 692 | |||
| 693 | ret = nouveau_bo_pin(nouveau_fb->nvbo, TTM_PL_FLAG_VRAM, true); | ||
| 694 | if (ret) | ||
| 695 | NV_ERROR(drm, "Could not pin framebuffer\n"); | ||
| 696 | } | ||
| 697 | |||
| 698 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | ||
| 699 | struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | ||
| 700 | if (!nv_crtc->cursor.nvbo) | ||
| 701 | continue; | ||
| 702 | |||
| 703 | ret = nouveau_bo_pin(nv_crtc->cursor.nvbo, TTM_PL_FLAG_VRAM, true); | ||
| 704 | if (!ret && nv_crtc->cursor.set_offset) | ||
| 705 | ret = nouveau_bo_map(nv_crtc->cursor.nvbo); | ||
| 706 | if (ret) | ||
| 707 | NV_ERROR(drm, "Could not pin/map cursor.\n"); | ||
| 708 | } | ||
| 709 | |||
| 710 | nouveau_display_init(dev); | ||
| 711 | |||
| 712 | /* Force CLUT to get re-loaded during modeset */ | ||
| 713 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | ||
| 714 | struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | ||
| 715 | |||
| 716 | nv_crtc->lut.depth = 0; | ||
| 717 | } | ||
| 718 | |||
| 719 | /* This should ensure we don't hit a locking problem when someone | ||
| 720 | * wakes us up via a connector. We should never go into suspend | ||
| 721 | * while the display is on anyways. | ||
| 722 | */ | ||
| 723 | if (runtime) | ||
| 724 | return; | ||
| 725 | |||
| 726 | drm_helper_resume_force_mode(dev); | ||
| 727 | |||
| 728 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { | ||
| 729 | struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc); | ||
| 730 | |||
| 731 | if (!nv_crtc->cursor.nvbo) | ||
| 732 | continue; | ||
| 733 | |||
| 734 | if (nv_crtc->cursor.set_offset) | ||
| 735 | nv_crtc->cursor.set_offset(nv_crtc, nv_crtc->cursor.nvbo->bo.offset); | ||
| 736 | nv_crtc->cursor.set_pos(nv_crtc, nv_crtc->cursor_saved_x, | ||
| 737 | nv_crtc->cursor_saved_y); | ||
| 738 | } | ||
| 739 | } | ||
| 740 | |||
| 741 | static int | ||
| 742 | nouveau_page_flip_emit(struct nouveau_channel *chan, | ||
| 743 | struct nouveau_bo *old_bo, | ||
| 744 | struct nouveau_bo *new_bo, | ||
| 745 | struct nouveau_page_flip_state *s, | ||
| 746 | struct nouveau_fence **pfence) | ||
| 747 | { | ||
| 748 | struct nouveau_fence_chan *fctx = chan->fence; | ||
| 749 | struct nouveau_drm *drm = chan->drm; | ||
| 750 | struct drm_device *dev = drm->dev; | ||
| 751 | unsigned long flags; | ||
| 752 | int ret; | ||
| 753 | |||
| 754 | /* Queue it to the pending list */ | ||
| 755 | spin_lock_irqsave(&dev->event_lock, flags); | ||
| 756 | list_add_tail(&s->head, &fctx->flip); | ||
| 757 | spin_unlock_irqrestore(&dev->event_lock, flags); | ||
| 758 | |||
| 759 | /* Synchronize with the old framebuffer */ | ||
| 760 | ret = nouveau_fence_sync(old_bo, chan, false, false); | ||
| 761 | if (ret) | ||
| 762 | goto fail; | ||
| 763 | |||
| 764 | /* Emit the pageflip */ | ||
| 765 | ret = RING_SPACE(chan, 2); | ||
| 766 | if (ret) | ||
| 767 | goto fail; | ||
| 768 | |||
| 769 | BEGIN_NV04(chan, NvSubSw, NV_SW_PAGE_FLIP, 1); | ||
| 770 | OUT_RING (chan, 0x00000000); | ||
| 771 | FIRE_RING (chan); | ||
| 772 | |||
| 773 | ret = nouveau_fence_new(chan, false, pfence); | ||
| 774 | if (ret) | ||
| 775 | goto fail; | ||
| 776 | |||
| 777 | return 0; | ||
| 778 | fail: | ||
| 779 | spin_lock_irqsave(&dev->event_lock, flags); | ||
| 780 | list_del(&s->head); | ||
| 781 | spin_unlock_irqrestore(&dev->event_lock, flags); | ||
| 782 | return ret; | ||
| 783 | } | ||
| 784 | |||
| 785 | int | ||
| 786 | nouveau_crtc_page_flip(struct drm_crtc *crtc, struct drm_framebuffer *fb, | ||
| 787 | struct drm_pending_vblank_event *event, u32 flags, | ||
| 788 | struct drm_modeset_acquire_ctx *ctx) | ||
| 789 | { | ||
| 790 | const int swap_interval = (flags & DRM_MODE_PAGE_FLIP_ASYNC) ? 0 : 1; | ||
| 791 | struct drm_device *dev = crtc->dev; | ||
| 792 | struct nouveau_drm *drm = nouveau_drm(dev); | ||
| 793 | struct nouveau_bo *old_bo = nouveau_framebuffer(crtc->primary->fb)->nvbo; | ||
| 794 | struct nouveau_bo *new_bo = nouveau_framebuffer(fb)->nvbo; | ||
| 795 | struct nouveau_page_flip_state *s; | ||
| 796 | struct nouveau_channel *chan; | ||
| 797 | struct nouveau_cli *cli; | ||
| 798 | struct nouveau_fence *fence; | ||
| 799 | struct nv04_display *dispnv04 = nv04_display(dev); | ||
| 800 | int head = nouveau_crtc(crtc)->index; | ||
| 801 | int ret; | ||
| 802 | |||
| 803 | chan = drm->channel; | ||
| 804 | if (!chan) | ||
| 805 | return -ENODEV; | ||
| 806 | cli = (void *)chan->user.client; | ||
| 807 | |||
| 808 | s = kzalloc(sizeof(*s), GFP_KERNEL); | ||
| 809 | if (!s) | ||
| 810 | return -ENOMEM; | ||
| 811 | |||
| 812 | if (new_bo != old_bo) { | ||
| 813 | ret = nouveau_bo_pin(new_bo, TTM_PL_FLAG_VRAM, true); | ||
| 814 | if (ret) | ||
| 815 | goto fail_free; | ||
| 816 | } | ||
| 817 | |||
| 818 | mutex_lock(&cli->mutex); | ||
| 819 | ret = ttm_bo_reserve(&new_bo->bo, true, false, NULL); | ||
| 820 | if (ret) | ||
| 821 | goto fail_unpin; | ||
| 822 | |||
| 823 | /* synchronise rendering channel with the kernel's channel */ | ||
| 824 | ret = nouveau_fence_sync(new_bo, chan, false, true); | ||
| 825 | if (ret) { | ||
| 826 | ttm_bo_unreserve(&new_bo->bo); | ||
| 827 | goto fail_unpin; | ||
| 828 | } | ||
| 829 | |||
| 830 | if (new_bo != old_bo) { | ||
| 831 | ttm_bo_unreserve(&new_bo->bo); | ||
| 832 | |||
| 833 | ret = ttm_bo_reserve(&old_bo->bo, true, false, NULL); | ||
| 834 | if (ret) | ||
| 835 | goto fail_unpin; | ||
| 836 | } | ||
| 837 | |||
| 838 | /* Initialize a page flip struct */ | ||
| 839 | *s = (struct nouveau_page_flip_state) | ||
| 840 | { { }, event, crtc, fb->format->cpp[0] * 8, fb->pitches[0], | ||
| 841 | new_bo->bo.offset }; | ||
| 842 | |||
| 843 | /* Keep vblanks on during flip, for the target crtc of this flip */ | ||
| 844 | drm_crtc_vblank_get(crtc); | ||
| 845 | |||
| 846 | /* Emit a page flip */ | ||
| 847 | if (swap_interval) { | ||
| 848 | ret = RING_SPACE(chan, 8); | ||
| 849 | if (ret) | ||
| 850 | goto fail_unreserve; | ||
| 851 | |||
| 852 | BEGIN_NV04(chan, NvSubImageBlit, 0x012c, 1); | ||
| 853 | OUT_RING (chan, 0); | ||
| 854 | BEGIN_NV04(chan, NvSubImageBlit, 0x0134, 1); | ||
| 855 | OUT_RING (chan, head); | ||
| 856 | BEGIN_NV04(chan, NvSubImageBlit, 0x0100, 1); | ||
| 857 | OUT_RING (chan, 0); | ||
| 858 | BEGIN_NV04(chan, NvSubImageBlit, 0x0130, 1); | ||
| 859 | OUT_RING (chan, 0); | ||
| 860 | } | ||
| 861 | |||
| 862 | nouveau_bo_ref(new_bo, &dispnv04->image[head]); | ||
| 863 | |||
| 864 | ret = nouveau_page_flip_emit(chan, old_bo, new_bo, s, &fence); | ||
| 865 | if (ret) | ||
| 866 | goto fail_unreserve; | ||
| 867 | mutex_unlock(&cli->mutex); | ||
| 868 | |||
| 869 | /* Update the crtc struct and cleanup */ | ||
| 870 | crtc->primary->fb = fb; | ||
| 871 | |||
| 872 | nouveau_bo_fence(old_bo, fence, false); | ||
| 873 | ttm_bo_unreserve(&old_bo->bo); | ||
| 874 | if (old_bo != new_bo) | ||
| 875 | nouveau_bo_unpin(old_bo); | ||
| 876 | nouveau_fence_unref(&fence); | ||
| 877 | return 0; | ||
| 878 | |||
| 879 | fail_unreserve: | ||
| 880 | drm_crtc_vblank_put(crtc); | ||
| 881 | ttm_bo_unreserve(&old_bo->bo); | ||
| 882 | fail_unpin: | ||
| 883 | mutex_unlock(&cli->mutex); | ||
| 884 | if (old_bo != new_bo) | ||
| 885 | nouveau_bo_unpin(new_bo); | ||
| 886 | fail_free: | ||
| 887 | kfree(s); | ||
| 888 | return ret; | ||
| 889 | } | ||
| 890 | |||
| 891 | int | ||
| 892 | nouveau_finish_page_flip(struct nouveau_channel *chan, | ||
| 893 | struct nouveau_page_flip_state *ps) | ||
| 894 | { | ||
| 895 | struct nouveau_fence_chan *fctx = chan->fence; | ||
| 896 | struct nouveau_drm *drm = chan->drm; | ||
| 897 | struct drm_device *dev = drm->dev; | ||
| 898 | struct nouveau_page_flip_state *s; | ||
| 899 | unsigned long flags; | ||
| 900 | |||
| 901 | spin_lock_irqsave(&dev->event_lock, flags); | ||
| 902 | |||
| 903 | if (list_empty(&fctx->flip)) { | ||
| 904 | NV_ERROR(drm, "unexpected pageflip\n"); | ||
| 905 | spin_unlock_irqrestore(&dev->event_lock, flags); | ||
| 906 | return -EINVAL; | ||
| 907 | } | ||
| 908 | |||
| 909 | s = list_first_entry(&fctx->flip, struct nouveau_page_flip_state, head); | ||
| 910 | if (s->event) { | ||
| 911 | drm_crtc_arm_vblank_event(s->crtc, s->event); | ||
| 912 | } else { | ||
| 913 | /* Give up ownership of vblank for page-flipped crtc */ | ||
| 914 | drm_crtc_vblank_put(s->crtc); | ||
| 915 | } | ||
| 916 | |||
| 917 | list_del(&s->head); | ||
| 918 | if (ps) | ||
| 919 | *ps = *s; | ||
| 920 | kfree(s); | ||
| 921 | |||
| 922 | spin_unlock_irqrestore(&dev->event_lock, flags); | ||
| 923 | return 0; | ||
| 924 | } | ||
| 925 | |||
| 926 | int | ||
| 927 | nouveau_flip_complete(struct nvif_notify *notify) | ||
| 928 | { | ||
| 929 | struct nouveau_drm *drm = container_of(notify, typeof(*drm), flip); | ||
| 930 | struct nouveau_channel *chan = drm->channel; | ||
| 931 | struct nouveau_page_flip_state state; | ||
| 932 | |||
| 933 | if (!nouveau_finish_page_flip(chan, &state)) { | ||
| 934 | nv_set_crtc_base(drm->dev, drm_crtc_index(state.crtc), | ||
| 935 | state.offset + state.crtc->y * | ||
| 936 | state.pitch + state.crtc->x * | ||
| 937 | state.bpp / 8); | ||
| 938 | } | ||
| 939 | |||
| 940 | return NVIF_NOTIFY_KEEP; | ||
| 941 | } | 647 | } |
| 942 | 648 | ||
| 943 | int | 649 | int |
| @@ -25,19 +25,11 @@ int nouveau_framebuffer_new(struct drm_device *, · 24 unmodified lines | |||
| 25 | const struct drm_mode_fb_cmd2 *, | 25 | const struct drm_mode_fb_cmd2 *, |
| 26 | struct nouveau_bo *, struct nouveau_framebuffer **); | 26 | struct nouveau_bo *, struct nouveau_framebuffer **); |
| 27 | 27 | ||
| 28 | struct nouveau_page_flip_state { | ||
| 29 | struct list_head head; | ||
| 30 | struct drm_pending_vblank_event *event; | ||
| 31 | struct drm_crtc *crtc; | ||
| 32 | int bpp, pitch; | ||
| 33 | u64 offset; | ||
| 34 | }; | ||
| 35 | |||
| 36 | struct nouveau_display { | 28 | struct nouveau_display { |
| 37 | void *priv; | 29 | void *priv; |
| 38 | void (*dtor)(struct drm_device *); | 30 | void (*dtor)(struct drm_device *); |
| 39 | int (*init)(struct drm_device *); | 31 | int (*init)(struct drm_device *, bool resume, bool runtime); |
| 40 | void (*fini)(struct drm_device *); | 32 | void (*fini)(struct drm_device *, bool suspend); |
| 41 | 33 | ||
| 42 | struct nvif_disp disp; | 34 | struct nvif_disp disp; |
| 43 | 35 | ||
| @@ -61,7 +53,7 @@ nouveau_display(struct drm_device *dev) · 17 unmodified lines | |||
| 61 | 53 | ||
| 62 | int nouveau_display_create(struct drm_device *dev); | 54 | int nouveau_display_create(struct drm_device *dev); |
| 63 | void nouveau_display_destroy(struct drm_device *dev); | 55 | void nouveau_display_destroy(struct drm_device *dev); |
| 64 | int nouveau_display_init(struct drm_device *dev); | 56 | int nouveau_display_init(struct drm_device *dev, bool resume, bool runtime); |
| 65 | void nouveau_display_fini(struct drm_device *dev, bool suspend, bool runtime); | 57 | void nouveau_display_fini(struct drm_device *dev, bool suspend, bool runtime); |
| 66 | int nouveau_display_suspend(struct drm_device *dev, bool runtime); | 58 | int nouveau_display_suspend(struct drm_device *dev, bool runtime); |
| 67 | void nouveau_display_resume(struct drm_device *dev, bool runtime); | 59 | void nouveau_display_resume(struct drm_device *dev, bool runtime); |
| @@ -71,13 +63,6 @@ bool nouveau_display_scanoutpos(struct drm_device *, unsigned int, · 3 unmodified lines | |||
| 71 | bool, int *, int *, ktime_t *, | 63 | bool, int *, int *, ktime_t *, |
| 72 | ktime_t *, const struct drm_display_mode *); | 64 | ktime_t *, const struct drm_display_mode *); |
| 73 | 65 | ||
| 74 | int nouveau_crtc_page_flip(struct drm_crtc *crtc, struct drm_framebuffer *fb, | ||
| 75 | struct drm_pending_vblank_event *event, | ||
| 76 | uint32_t page_flip_flags, | ||
| 77 | struct drm_modeset_acquire_ctx *ctx); | ||
| 78 | int nouveau_finish_page_flip(struct nouveau_channel *, | ||
| 79 | struct nouveau_page_flip_state *); | ||
| 80 | |||
| 81 | int nouveau_display_dumb_create(struct drm_file *, struct drm_device *, | 66 | int nouveau_display_dumb_create(struct drm_file *, struct drm_device *, |
| 82 | struct drm_mode_create_dumb *args); | 67 | struct drm_mode_create_dumb *args); |
| 83 | int nouveau_display_dumb_map_offset(struct drm_file *, struct drm_device *, | 68 | int nouveau_display_dumb_map_offset(struct drm_file *, struct drm_device *, |
| @@ -0,0 +1,887 @@ | |||
| 1 | /* | ||
| 2 | * Copyright 2018 Red Hat Inc. | ||
| 3 | * | ||
| 4 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 5 | * copy of this software and associated documentation files (the "Software"), | ||
| 6 | * to deal in the Software without restriction, including without limitation | ||
| 7 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 8 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 9 | * Software is furnished to do so, subject to the following conditions: | ||
| 10 | * | ||
| 11 | * The above copyright notice and this permission notice shall be included in | ||
| 12 | * all copies or substantial portions of the Software. | ||
| 13 | * | ||
| 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 17 | * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | ||
| 18 | * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | ||
| 19 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | ||
| 20 | * OTHER DEALINGS IN THE SOFTWARE. | ||
| 21 | */ | ||
| 22 | #include "nouveau_dmem.h" | ||
| 23 | #include "nouveau_drv.h" | ||
| 24 | #include "nouveau_chan.h" | ||
| 25 | #include "nouveau_dma.h" | ||
| 26 | #include "nouveau_mem.h" | ||
| 27 | #include "nouveau_bo.h" | ||
| 28 | |||
| 29 | #include <nvif/class.h> | ||
| 30 | #include <nvif/object.h> | ||
| 31 | #include <nvif/if500b.h> | ||
| 32 | #include <nvif/if900b.h> | ||
| 33 | |||
| 34 | #include <linux/sched/mm.h> | ||
| 35 | #include <linux/hmm.h> | ||
| 36 | |||
| 37 | /* | ||
| 38 | * FIXME: this is ugly right now we are using TTM to allocate vram and we pin | ||
| 39 | * it in vram while in use. We likely want to overhaul memory management for | ||
| 40 | * nouveau to be more page like (not necessarily with system page size but a | ||
| 41 | * bigger page size) at lowest level and have some shim layer on top that would | ||
| 42 | * provide the same functionality as TTM. | ||
| 43 | */ | ||
| 44 | #define DMEM_CHUNK_SIZE (2UL << 20) | ||
| 45 | #define DMEM_CHUNK_NPAGES (DMEM_CHUNK_SIZE >> PAGE_SHIFT) | ||
| 46 | |||
| 47 | struct nouveau_migrate; | ||
| 48 | |||
| 49 | enum nouveau_aper { | ||
| 50 | NOUVEAU_APER_VIRT, | ||
| 51 | NOUVEAU_APER_VRAM, | ||
| 52 | NOUVEAU_APER_HOST, | ||
| 53 | }; | ||
| 54 | |||
| 55 | typedef int (*nouveau_migrate_copy_t)(struct nouveau_drm *drm, u64 npages, | ||
| 56 | enum nouveau_aper, u64 dst_addr, | ||
| 57 | enum nouveau_aper, u64 src_addr); | ||
| 58 | |||
| 59 | struct nouveau_dmem_chunk { | ||
| 60 | struct list_head list; | ||
| 61 | struct nouveau_bo *bo; | ||
| 62 | struct nouveau_drm *drm; | ||
| 63 | unsigned long pfn_first; | ||
| 64 | unsigned long callocated; | ||
| 65 | unsigned long bitmap[BITS_TO_LONGS(DMEM_CHUNK_NPAGES)]; | ||
| 66 | spinlock_t lock; | ||
| 67 | }; | ||
| 68 | |||
| 69 | struct nouveau_dmem_migrate { | ||
| 70 | nouveau_migrate_copy_t copy_func; | ||
| 71 | struct nouveau_channel *chan; | ||
| 72 | }; | ||
| 73 | |||
| 74 | struct nouveau_dmem { | ||
| 75 | struct hmm_devmem *devmem; | ||
| 76 | struct nouveau_dmem_migrate migrate; | ||
| 77 | struct list_head chunk_free; | ||
| 78 | struct list_head chunk_full; | ||
| 79 | struct list_head chunk_empty; | ||
| 80 | struct mutex mutex; | ||
| 81 | }; | ||
| 82 | |||
| 83 | struct nouveau_dmem_fault { | ||
| 84 | struct nouveau_drm *drm; | ||
| 85 | struct nouveau_fence *fence; | ||
| 86 | dma_addr_t *dma; | ||
| 87 | unsigned long npages; | ||
| 88 | }; | ||
| 89 | |||
| 90 | struct nouveau_migrate { | ||
| 91 | struct vm_area_struct *vma; | ||
| 92 | struct nouveau_drm *drm; | ||
| 93 | struct nouveau_fence *fence; | ||
| 94 | unsigned long npages; | ||
| 95 | dma_addr_t *dma; | ||
| 96 | unsigned long dma_nr; | ||
| 97 | }; | ||
| 98 | |||
| 99 | static void | ||
| 100 | nouveau_dmem_free(struct hmm_devmem *devmem, struct page *page) | ||
| 101 | { | ||
| 102 | struct nouveau_dmem_chunk *chunk; | ||
| 103 | struct nouveau_drm *drm; | ||
| 104 | unsigned long idx; | ||
| 105 | |||
| 106 | chunk = (void *)hmm_devmem_page_get_drvdata(page); | ||
| 107 | idx = page_to_pfn(page) - chunk->pfn_first; | ||
| 108 | drm = chunk->drm; | ||
| 109 | |||
| 110 | /* | ||
| 111 | * FIXME: | ||
| 112 | * | ||
| 113 | * This is really a bad example, we need to overhaul nouveau memory | ||
| 114 | * management to be more page focus and allow lighter locking scheme | ||
| 115 | * to be use in the process. | ||
| 116 | */ | ||
| 117 | spin_lock(&chunk->lock); | ||
| 118 | clear_bit(idx, chunk->bitmap); | ||
| 119 | WARN_ON(!chunk->callocated); | ||
| 120 | chunk->callocated--; | ||
| 121 | /* | ||
| 122 | * FIXME when chunk->callocated reach 0 we should add the chunk to | ||
| 123 | * a reclaim list so that it can be freed in case of memory pressure. | ||
| 124 | */ | ||
| 125 | spin_unlock(&chunk->lock); | ||
| 126 | } | ||
| 127 | |||
| 128 | static void | ||
| 129 | nouveau_dmem_fault_alloc_and_copy(struct vm_area_struct *vma, | ||
| 130 | const unsigned long *src_pfns, | ||
| 131 | unsigned long *dst_pfns, | ||
| 132 | unsigned long start, | ||
| 133 | unsigned long end, | ||
| 134 | void *private) | ||
| 135 | { | ||
| 136 | struct nouveau_dmem_fault *fault = private; | ||
| 137 | struct nouveau_drm *drm = fault->drm; | ||
| 138 | struct device *dev = drm->dev->dev; | ||
| 139 | unsigned long addr, i, npages = 0; | ||
| 140 | nouveau_migrate_copy_t copy; | ||
| 141 | int ret; | ||
| 142 | |||
| 143 | |||
| 144 | /* First allocate new memory */ | ||
| 145 | for (addr = start, i = 0; addr < end; addr += PAGE_SIZE, i++) { | ||
| 146 | struct page *dpage, *spage; | ||
| 147 | |||
| 148 | dst_pfns[i] = 0; | ||
| 149 | spage = migrate_pfn_to_page(src_pfns[i]); | ||
| 150 | if (!spage || !(src_pfns[i] & MIGRATE_PFN_MIGRATE)) | ||
| 151 | continue; | ||
| 152 | |||
| 153 | dpage = hmm_vma_alloc_locked_page(vma, addr); | ||
| 154 | if (!dpage) { | ||
| 155 | dst_pfns[i] = MIGRATE_PFN_ERROR; | ||
| 156 | continue; | ||
| 157 | } | ||
| 158 | |||
| 159 | dst_pfns[i] = migrate_pfn(page_to_pfn(dpage)) | | ||
| 160 | MIGRATE_PFN_LOCKED; | ||
| 161 | npages++; | ||
| 162 | } | ||
| 163 | |||
| 164 | /* Allocate storage for DMA addresses, so we can unmap later. */ | ||
| 165 | fault->dma = kmalloc(sizeof(*fault->dma) * npages, GFP_KERNEL); | ||
| 166 | if (!fault->dma) | ||
| 167 | goto error; | ||
| 168 | |||
| 169 | /* Copy things over */ | ||
| 170 | copy = drm->dmem->migrate.copy_func; | ||
| 171 | for (addr = start, i = 0; addr < end; addr += PAGE_SIZE, i++) { | ||
| 172 | struct nouveau_dmem_chunk *chunk; | ||
| 173 | struct page *spage, *dpage; | ||
| 174 | u64 src_addr, dst_addr; | ||
| 175 | |||
| 176 | dpage = migrate_pfn_to_page(dst_pfns[i]); | ||
| 177 | if (!dpage || dst_pfns[i] == MIGRATE_PFN_ERROR) | ||
| 178 | continue; | ||
| 179 | |||
| 180 | spage = migrate_pfn_to_page(src_pfns[i]); | ||
| 181 | if (!spage || !(src_pfns[i] & MIGRATE_PFN_MIGRATE)) { | ||
| 182 | dst_pfns[i] = MIGRATE_PFN_ERROR; | ||
| 183 | __free_page(dpage); | ||
| 184 | continue; | ||
| 185 | } | ||
| 186 | |||
| 187 | fault->dma[fault->npages] = | ||
| 188 | dma_map_page_attrs(dev, dpage, 0, PAGE_SIZE, | ||
| 189 | PCI_DMA_BIDIRECTIONAL, | ||
| 190 | DMA_ATTR_SKIP_CPU_SYNC); | ||
| 191 | if (dma_mapping_error(dev, fault->dma[fault->npages])) { | ||
| 192 | dst_pfns[i] = MIGRATE_PFN_ERROR; | ||
| 193 | __free_page(dpage); | ||
| 194 | continue; | ||
| 195 | } | ||
| 196 | |||
| 197 | dst_addr = fault->dma[fault->npages++]; | ||
| 198 | |||
| 199 | chunk = (void *)hmm_devmem_page_get_drvdata(spage); | ||
| 200 | src_addr = page_to_pfn(spage) - chunk->pfn_first; | ||
| 201 | src_addr = (src_addr << PAGE_SHIFT) + chunk->bo->bo.offset; | ||
| 202 | |||
| 203 | ret = copy(drm, 1, NOUVEAU_APER_HOST, dst_addr, | ||
| 204 | NOUVEAU_APER_VRAM, src_addr); | ||
| 205 | if (ret) { | ||
| 206 | dst_pfns[i] = MIGRATE_PFN_ERROR; | ||
| 207 | __free_page(dpage); | ||
| 208 | continue; | ||
| 209 | } | ||
| 210 | } | ||
| 211 | |||
| 212 | nouveau_fence_new(drm->dmem->migrate.chan, false, &fault->fence); | ||
| 213 | |||
| 214 | return; | ||
| 215 | |||
| 216 | error: | ||
| 217 | for (addr = start, i = 0; addr < end; addr += PAGE_SIZE, ++i) { | ||
| 218 | struct page *page; | ||
| 219 | |||
| 220 | if (!dst_pfns[i] || dst_pfns[i] == MIGRATE_PFN_ERROR) | ||
| 221 | continue; | ||
| 222 | |||
| 223 | page = migrate_pfn_to_page(dst_pfns[i]); | ||
| 224 | dst_pfns[i] = MIGRATE_PFN_ERROR; | ||
| 225 | if (page == NULL) | ||
| 226 | continue; | ||
| 227 | |||
| 228 | __free_page(page); | ||
| 229 | } | ||
| 230 | } | ||
| 231 | |||
| 232 | void nouveau_dmem_fault_finalize_and_map(struct vm_area_struct *vma, | ||
| 233 | const unsigned long *src_pfns, | ||
| 234 | const unsigned long *dst_pfns, | ||
| 235 | unsigned long start, | ||
| 236 | unsigned long end, | ||
| 237 | void *private) | ||
| 238 | { | ||
| 239 | struct nouveau_dmem_fault *fault = private; | ||
| 240 | struct nouveau_drm *drm = fault->drm; | ||
| 241 | |||
| 242 | if (fault->fence) { | ||
| 243 | nouveau_fence_wait(fault->fence, true, false); | ||
| 244 | nouveau_fence_unref(&fault->fence); | ||
| 245 | } else { | ||
| 246 | /* | ||
| 247 | * FIXME wait for channel to be IDLE before calling finalizing | ||
| 248 | * the hmem object below (nouveau_migrate_hmem_fini()). | ||
| 249 | */ | ||
| 250 | } | ||
| 251 | |||
| 252 | while (fault->npages--) { | ||
| 253 | dma_unmap_page(drm->dev->dev, fault->dma[fault->npages], | ||
| 254 | PAGE_SIZE, PCI_DMA_BIDIRECTIONAL); | ||
| 255 | } | ||
| 256 | kfree(fault->dma); | ||
| 257 | } | ||
| 258 | |||
| 259 | static const struct migrate_vma_ops nouveau_dmem_fault_migrate_ops = { | ||
| 260 | .alloc_and_copy = nouveau_dmem_fault_alloc_and_copy, | ||
| 261 | .finalize_and_map = nouveau_dmem_fault_finalize_and_map, | ||
| 262 | }; | ||
| 263 | |||
| 264 | static int | ||
| 265 | nouveau_dmem_fault(struct hmm_devmem *devmem, | ||
| 266 | struct vm_area_struct *vma, | ||
| 267 | unsigned long addr, | ||
| 268 | const struct page *page, | ||
| 269 | unsigned int flags, | ||
| 270 | pmd_t *pmdp) | ||
| 271 | { | ||
| 272 | struct drm_device *drm_dev = dev_get_drvdata(devmem->device); | ||
| 273 | unsigned long src[1] = {0}, dst[1] = {0}; | ||
| 274 | struct nouveau_dmem_fault fault = {0}; | ||
| 275 | int ret; | ||
| 276 | |||
| 277 | |||
| 278 | |||
| 279 | /* | ||
| 280 | * FIXME what we really want is to find some heuristic to migrate more | ||
| 281 | * than just one page on CPU fault. When such fault happens it is very | ||
| 282 | * likely that more surrounding page will CPU fault too. | ||
| 283 | */ | ||
| 284 | fault.drm = nouveau_drm(drm_dev); | ||
| 285 | ret = migrate_vma(&nouveau_dmem_fault_migrate_ops, vma, addr, | ||
| 286 | addr + PAGE_SIZE, src, dst, &fault); | ||
| 287 | if (ret) | ||
| 288 | return VM_FAULT_SIGBUS; | ||
| 289 | |||
| 290 | if (dst[0] == MIGRATE_PFN_ERROR) | ||
| 291 | return VM_FAULT_SIGBUS; | ||
| 292 | |||
| 293 | return 0; | ||
| 294 | } | ||
| 295 | |||
| 296 | static const struct hmm_devmem_ops | ||
| 297 | nouveau_dmem_devmem_ops = { | ||
| 298 | .free = nouveau_dmem_free, | ||
| 299 | .fault = nouveau_dmem_fault, | ||
| 300 | }; | ||
| 301 | |||
| 302 | static int | ||
| 303 | nouveau_dmem_chunk_alloc(struct nouveau_drm *drm) | ||
| 304 | { | ||
| 305 | struct nouveau_dmem_chunk *chunk; | ||
| 306 | int ret; | ||
| 307 | |||
| 308 | if (drm->dmem == NULL) | ||
| 309 | return -EINVAL; | ||
| 310 | |||
| 311 | mutex_lock(&drm->dmem->mutex); | ||
| 312 | chunk = list_first_entry_or_null(&drm->dmem->chunk_empty, | ||
| 313 | struct nouveau_dmem_chunk, | ||
| 314 | list); | ||
| 315 | if (chunk == NULL) { | ||
| 316 | mutex_unlock(&drm->dmem->mutex); | ||
| 317 | return -ENOMEM; | ||
| 318 | } | ||
| 319 | |||
| 320 | list_del(&chunk->list); | ||
| 321 | mutex_unlock(&drm->dmem->mutex); | ||
| 322 | |||
| 323 | ret = nouveau_bo_new(&drm->client, DMEM_CHUNK_SIZE, 0, | ||
| 324 | TTM_PL_FLAG_VRAM, 0, 0, NULL, NULL, | ||
| 325 | &chunk->bo); | ||
| 326 | if (ret) | ||
| 327 | goto out; | ||
| 328 | |||
| 329 | ret = nouveau_bo_pin(chunk->bo, TTM_PL_FLAG_VRAM, false); | ||
| 330 | if (ret) { | ||
| 331 | nouveau_bo_ref(NULL, &chunk->bo); | ||
| 332 | goto out; | ||
| 333 | } | ||
| 334 | |||
| 335 | bitmap_zero(chunk->bitmap, DMEM_CHUNK_NPAGES); | ||
| 336 | spin_lock_init(&chunk->lock); | ||
| 337 | |||
| 338 | out: | ||
| 339 | mutex_lock(&drm->dmem->mutex); | ||
| 340 | if (chunk->bo) | ||
| 341 | list_add(&chunk->list, &drm->dmem->chunk_empty); | ||
| 342 | else | ||
| 343 | list_add_tail(&chunk->list, &drm->dmem->chunk_empty); | ||
| 344 | mutex_unlock(&drm->dmem->mutex); | ||
| 345 | |||
| 346 | return ret; | ||
| 347 | } | ||
| 348 | |||
| 349 | static struct nouveau_dmem_chunk * | ||
| 350 | nouveau_dmem_chunk_first_free_locked(struct nouveau_drm *drm) | ||
| 351 | { | ||
| 352 | struct nouveau_dmem_chunk *chunk; | ||
| 353 | |||
| 354 | chunk = list_first_entry_or_null(&drm->dmem->chunk_free, | ||
| 355 | struct nouveau_dmem_chunk, | ||
| 356 | list); | ||
| 357 | if (chunk) | ||
| 358 | return chunk; | ||
| 359 | |||
| 360 | chunk = list_first_entry_or_null(&drm->dmem->chunk_empty, | ||
| 361 | struct nouveau_dmem_chunk, | ||
| 362 | list); | ||
| 363 | if (chunk->bo) | ||
| 364 | return chunk; | ||
| 365 | |||
| 366 | return NULL; | ||
| 367 | } | ||
| 368 | |||
| 369 | static int | ||
| 370 | nouveau_dmem_pages_alloc(struct nouveau_drm *drm, | ||
| 371 | unsigned long npages, | ||
| 372 | unsigned long *pages) | ||
| 373 | { | ||
| 374 | struct nouveau_dmem_chunk *chunk; | ||
| 375 | unsigned long c; | ||
| 376 | int ret; | ||
| 377 | |||
| 378 | memset(pages, 0xff, npages * sizeof(*pages)); | ||
| 379 | |||
| 380 | mutex_lock(&drm->dmem->mutex); | ||
| 381 | for (c = 0; c < npages;) { | ||
| 382 | unsigned long i; | ||
| 383 | |||
| 384 | chunk = nouveau_dmem_chunk_first_free_locked(drm); | ||
| 385 | if (chunk == NULL) { | ||
| 386 | mutex_unlock(&drm->dmem->mutex); | ||
| 387 | ret = nouveau_dmem_chunk_alloc(drm); | ||
| 388 | if (ret) { | ||
| 389 | if (c) | ||
| 390 | break; | ||
| 391 | return ret; | ||
| 392 | } | ||
| 393 | continue; | ||
| 394 | } | ||
| 395 | |||
| 396 | spin_lock(&chunk->lock); | ||
| 397 | i = find_first_zero_bit(chunk->bitmap, DMEM_CHUNK_NPAGES); | ||
| 398 | while (i < DMEM_CHUNK_NPAGES && c < npages) { | ||
| 399 | pages[c] = chunk->pfn_first + i; | ||
| 400 | set_bit(i, chunk->bitmap); | ||
| 401 | chunk->callocated++; | ||
| 402 | c++; | ||
| 403 | |||
| 404 | i = find_next_zero_bit(chunk->bitmap, | ||
| 405 | DMEM_CHUNK_NPAGES, i); | ||
| 406 | } | ||
| 407 | spin_unlock(&chunk->lock); | ||
| 408 | } | ||
| 409 | mutex_unlock(&drm->dmem->mutex); | ||
| 410 | |||
| 411 | return 0; | ||
| 412 | } | ||
| 413 | |||
| 414 | static struct page * | ||
| 415 | nouveau_dmem_page_alloc_locked(struct nouveau_drm *drm) | ||
| 416 | { | ||
| 417 | unsigned long pfns[1]; | ||
| 418 | struct page *page; | ||
| 419 | int ret; | ||
| 420 | |||
| 421 | /* FIXME stop all the miss-match API ... */ | ||
| 422 | ret = nouveau_dmem_pages_alloc(drm, 1, pfns); | ||
| 423 | if (ret) | ||
| 424 | return NULL; | ||
| 425 | |||
| 426 | page = pfn_to_page(pfns[0]); | ||
| 427 | get_page(page); | ||
| 428 | lock_page(page); | ||
| 429 | return page; | ||
| 430 | } | ||
| 431 | |||
| 432 | static void | ||
| 433 | nouveau_dmem_page_free_locked(struct nouveau_drm *drm, struct page *page) | ||
| 434 | { | ||
| 435 | unlock_page(page); | ||
| 436 | put_page(page); | ||
| 437 | } | ||
| 438 | |||
| 439 | void | ||
| 440 | nouveau_dmem_resume(struct nouveau_drm *drm) | ||
| 441 | { | ||
| 442 | struct nouveau_dmem_chunk *chunk; | ||
| 443 | int ret; | ||
| 444 | |||
| 445 | if (drm->dmem == NULL) | ||
| 446 | return; | ||
| 447 | |||
| 448 | mutex_lock(&drm->dmem->mutex); | ||
| 449 | list_for_each_entry (chunk, &drm->dmem->chunk_free, list) { | ||
| 450 | ret = nouveau_bo_pin(chunk->bo, TTM_PL_FLAG_VRAM, false); | ||
| 451 | /* FIXME handle pin failure */ | ||
| 452 | WARN_ON(ret); | ||
| 453 | } | ||
| 454 | list_for_each_entry (chunk, &drm->dmem->chunk_full, list) { | ||
| 455 | ret = nouveau_bo_pin(chunk->bo, TTM_PL_FLAG_VRAM, false); | ||
| 456 | /* FIXME handle pin failure */ | ||
| 457 | WARN_ON(ret); | ||
| 458 | } | ||
| 459 | list_for_each_entry (chunk, &drm->dmem->chunk_empty, list) { | ||
| 460 | ret = nouveau_bo_pin(chunk->bo, TTM_PL_FLAG_VRAM, false); | ||
| 461 | /* FIXME handle pin failure */ | ||
| 462 | WARN_ON(ret); | ||
| 463 | } | ||
| 464 | mutex_unlock(&drm->dmem->mutex); | ||
| 465 | } | ||
| 466 | |||
| 467 | void | ||
| 468 | nouveau_dmem_suspend(struct nouveau_drm *drm) | ||
| 469 | { | ||
| 470 | struct nouveau_dmem_chunk *chunk; | ||
| 471 | |||
| 472 | if (drm->dmem == NULL) | ||
| 473 | return; | ||
| 474 | |||
| 475 | mutex_lock(&drm->dmem->mutex); | ||
| 476 | list_for_each_entry (chunk, &drm->dmem->chunk_free, list) { | ||
| 477 | nouveau_bo_unpin(chunk->bo); | ||
| 478 | } | ||
| 479 | list_for_each_entry (chunk, &drm->dmem->chunk_full, list) { | ||
| 480 | nouveau_bo_unpin(chunk->bo); | ||
| 481 | } | ||
| 482 | list_for_each_entry (chunk, &drm->dmem->chunk_empty, list) { | ||
| 483 | nouveau_bo_unpin(chunk->bo); | ||
| 484 | } | ||
| 485 | mutex_unlock(&drm->dmem->mutex); | ||
| 486 | } | ||
| 487 | |||
| 488 | void | ||
| 489 | nouveau_dmem_fini(struct nouveau_drm *drm) | ||
| 490 | { | ||
| 491 | struct nouveau_dmem_chunk *chunk, *tmp; | ||
| 492 | |||
| 493 | if (drm->dmem == NULL) | ||
| 494 | return; | ||
| 495 | |||
| 496 | mutex_lock(&drm->dmem->mutex); | ||
| 497 | |||
| 498 | WARN_ON(!list_empty(&drm->dmem->chunk_free)); | ||
| 499 | WARN_ON(!list_empty(&drm->dmem->chunk_full)); | ||
| 500 | |||
| 501 | list_for_each_entry_safe (chunk, tmp, &drm->dmem->chunk_empty, list) { | ||
| 502 | if (chunk->bo) { | ||
| 503 | nouveau_bo_unpin(chunk->bo); | ||
| 504 | nouveau_bo_ref(NULL, &chunk->bo); | ||
| 505 | } | ||
| 506 | list_del(&chunk->list); | ||
| 507 | kfree(chunk); | ||
| 508 | } | ||
| 509 | |||
| 510 | mutex_unlock(&drm->dmem->mutex); | ||
| 511 | } | ||
| 512 | |||
| 513 | static int | ||
| 514 | nvc0b5_migrate_copy(struct nouveau_drm *drm, u64 npages, | ||
| 515 | enum nouveau_aper dst_aper, u64 dst_addr, | ||
| 516 | enum nouveau_aper src_aper, u64 src_addr) | ||
| 517 | { | ||
| 518 | struct nouveau_channel *chan = drm->dmem->migrate.chan; | ||
| 519 | u32 launch_dma = (1 << 9) /* MULTI_LINE_ENABLE. */ | | ||
| 520 | (1 << 8) /* DST_MEMORY_LAYOUT_PITCH. */ | | ||
| 521 | (1 << 7) /* SRC_MEMORY_LAYOUT_PITCH. */ | | ||
| 522 | (1 << 2) /* FLUSH_ENABLE_TRUE. */ | | ||
| 523 | (2 << 0) /* DATA_TRANSFER_TYPE_NON_PIPELINED. */; | ||
| 524 | int ret; | ||
| 525 | |||
| 526 | ret = RING_SPACE(chan, 13); | ||
| 527 | if (ret) | ||
| 528 | return ret; | ||
| 529 | |||
| 530 | if (src_aper != NOUVEAU_APER_VIRT) { | ||
| 531 | switch (src_aper) { | ||
| 532 | case NOUVEAU_APER_VRAM: | ||
| 533 | BEGIN_IMC0(chan, NvSubCopy, 0x0260, 0); | ||
| 534 | break; | ||
| 535 | case NOUVEAU_APER_HOST: | ||
| 536 | BEGIN_IMC0(chan, NvSubCopy, 0x0260, 1); | ||
| 537 | break; | ||
| 538 | default: | ||
| 539 | return -EINVAL; | ||
| 540 | } | ||
| 541 | launch_dma |= 0x00001000; /* SRC_TYPE_PHYSICAL. */ | ||
| 542 | } | ||
| 543 | |||
| 544 | if (dst_aper != NOUVEAU_APER_VIRT) { | ||
| 545 | switch (dst_aper) { | ||
| 546 | case NOUVEAU_APER_VRAM: | ||
| 547 | BEGIN_IMC0(chan, NvSubCopy, 0x0264, 0); | ||
| 548 | break; | ||
| 549 | case NOUVEAU_APER_HOST: | ||
| 550 | BEGIN_IMC0(chan, NvSubCopy, 0x0264, 1); | ||
| 551 | break; | ||
| 552 | default: | ||
| 553 | return -EINVAL; | ||
| 554 | } | ||
| 555 | launch_dma |= 0x00002000; /* DST_TYPE_PHYSICAL. */ | ||
| 556 | } | ||
| 557 | |||
| 558 | BEGIN_NVC0(chan, NvSubCopy, 0x0400, 8); | ||
| 559 | OUT_RING (chan, upper_32_bits(src_addr)); | ||
| 560 | OUT_RING (chan, lower_32_bits(src_addr)); | ||
| 561 | OUT_RING (chan, upper_32_bits(dst_addr)); | ||
| 562 | OUT_RING (chan, lower_32_bits(dst_addr)); | ||
| 563 | OUT_RING (chan, PAGE_SIZE); | ||
| 564 | OUT_RING (chan, PAGE_SIZE); | ||
| 565 | OUT_RING (chan, PAGE_SIZE); | ||
| 566 | OUT_RING (chan, npages); | ||
| 567 | BEGIN_NVC0(chan, NvSubCopy, 0x0300, 1); | ||
| 568 | OUT_RING (chan, launch_dma); | ||
| 569 | return 0; | ||
| 570 | } | ||
| 571 | |||
| 572 | static int | ||
| 573 | nouveau_dmem_migrate_init(struct nouveau_drm *drm) | ||
| 574 | { | ||
| 575 | switch (drm->ttm.copy.oclass) { | ||
| 576 | case PASCAL_DMA_COPY_A: | ||
| 577 | case PASCAL_DMA_COPY_B: | ||
| 578 | case VOLTA_DMA_COPY_A: | ||
| 579 | case TURING_DMA_COPY_A: | ||
| 580 | drm->dmem->migrate.copy_func = nvc0b5_migrate_copy; | ||
| 581 | drm->dmem->migrate.chan = drm->ttm.chan; | ||
| 582 | return 0; | ||
| 583 | default: | ||
| 584 | break; | ||
| 585 | } | ||
| 586 | return -ENODEV; | ||
| 587 | } | ||
| 588 | |||
| 589 | void | ||
| 590 | nouveau_dmem_init(struct nouveau_drm *drm) | ||
| 591 | { | ||
| 592 | struct device *device = drm->dev->dev; | ||
| 593 | unsigned long i, size; | ||
| 594 | int ret; | ||
| 595 | |||
| 596 | /* This only make sense on PASCAL or newer */ | ||
| 597 | if (drm->client.device.info.family < NV_DEVICE_INFO_V0_PASCAL) | ||
| 598 | return; | ||
| 599 | |||
| 600 | if (!(drm->dmem = kzalloc(sizeof(*drm->dmem), GFP_KERNEL))) | ||
| 601 | return; | ||
| 602 | |||
| 603 | mutex_init(&drm->dmem->mutex); | ||
| 604 | INIT_LIST_HEAD(&drm->dmem->chunk_free); | ||
| 605 | INIT_LIST_HEAD(&drm->dmem->chunk_full); | ||
| 606 | INIT_LIST_HEAD(&drm->dmem->chunk_empty); | ||
| 607 | |||
| 608 | size = ALIGN(drm->client.device.info.ram_user, DMEM_CHUNK_SIZE); | ||
| 609 | |||
| 610 | /* Initialize migration dma helpers before registering memory */ | ||
| 611 | ret = nouveau_dmem_migrate_init(drm); | ||
| 612 | if (ret) { | ||
| 613 | kfree(drm->dmem); | ||
| 614 | drm->dmem = NULL; | ||
| 615 | return; | ||
| 616 | } | ||
| 617 | |||
| 618 | /* | ||
| 619 | * FIXME we need some kind of policy to decide how much VRAM we | ||
| 620 | * want to register with HMM. For now just register everything | ||
| 621 | * and latter if we want to do thing like over commit then we | ||
| 622 | * could revisit this. | ||
| 623 | */ | ||
| 624 | drm->dmem->devmem = hmm_devmem_add(&nouveau_dmem_devmem_ops, | ||
| 625 | device, size); | ||
| 626 | if (drm->dmem->devmem == NULL) { | ||
| 627 | kfree(drm->dmem); | ||
| 628 | drm->dmem = NULL; | ||
| 629 | return; | ||
| 630 | } | ||
| 631 | |||
| 632 | for (i = 0; i < (size / DMEM_CHUNK_SIZE); ++i) { | ||
| 633 | struct nouveau_dmem_chunk *chunk; | ||
| 634 | struct page *page; | ||
| 635 | unsigned long j; | ||
| 636 | |||
| 637 | chunk = kzalloc(sizeof(*chunk), GFP_KERNEL); | ||
| 638 | if (chunk == NULL) { | ||
| 639 | nouveau_dmem_fini(drm); | ||
| 640 | return; | ||
| 641 | } | ||
| 642 | |||
| 643 | chunk->drm = drm; | ||
| 644 | chunk->pfn_first = drm->dmem->devmem->pfn_first; | ||
| 645 | chunk->pfn_first += (i * DMEM_CHUNK_NPAGES); | ||
| 646 | list_add_tail(&chunk->list, &drm->dmem->chunk_empty); | ||
| 647 | |||
| 648 | page = pfn_to_page(chunk->pfn_first); | ||
| 649 | for (j = 0; j < DMEM_CHUNK_NPAGES; ++j, ++page) { | ||
| 650 | hmm_devmem_page_set_drvdata(page, (long)chunk); | ||
| 651 | } | ||
| 652 | } | ||
| 653 | |||
| 654 | NV_INFO(drm, "DMEM: registered %ldMB of device memory\n", size >> 20); | ||
| 655 | } | ||
| 656 | |||
| 657 | static void | ||
| 658 | nouveau_dmem_migrate_alloc_and_copy(struct vm_area_struct *vma, | ||
| 659 | const unsigned long *src_pfns, | ||
| 660 | unsigned long *dst_pfns, | ||
| 661 | unsigned long start, | ||
| 662 | unsigned long end, | ||
| 663 | void *private) | ||
| 664 | { | ||
| 665 | struct nouveau_migrate *migrate = private; | ||
| 666 | struct nouveau_drm *drm = migrate->drm; | ||
| 667 | struct device *dev = drm->dev->dev; | ||
| 668 | unsigned long addr, i, npages = 0; | ||
| 669 | nouveau_migrate_copy_t copy; | ||
| 670 | int ret; | ||
| 671 | |||
| 672 | /* First allocate new memory */ | ||
| 673 | for (addr = start, i = 0; addr < end; addr += PAGE_SIZE, i++) { | ||
| 674 | struct page *dpage, *spage; | ||
| 675 | |||
| 676 | dst_pfns[i] = 0; | ||
| 677 | spage = migrate_pfn_to_page(src_pfns[i]); | ||
| 678 | if (!spage || !(src_pfns[i] & MIGRATE_PFN_MIGRATE)) | ||
| 679 | continue; | ||
| 680 | |||
| 681 | dpage = nouveau_dmem_page_alloc_locked(drm); | ||
| 682 | if (!dpage) | ||
| 683 | continue; | ||
| 684 | |||
| 685 | dst_pfns[i] = migrate_pfn(page_to_pfn(dpage)) | | ||
| 686 | MIGRATE_PFN_LOCKED | | ||
| 687 | MIGRATE_PFN_DEVICE; | ||
| 688 | npages++; | ||
| 689 | } | ||
| 690 | |||
| 691 | if (!npages) | ||
| 692 | return; | ||
| 693 | |||
| 694 | /* Allocate storage for DMA addresses, so we can unmap later. */ | ||
| 695 | migrate->dma = kmalloc(sizeof(*migrate->dma) * npages, GFP_KERNEL); | ||
| 696 | if (!migrate->dma) | ||
| 697 | goto error; | ||
| 698 | |||
| 699 | /* Copy things over */ | ||
| 700 | copy = drm->dmem->migrate.copy_func; | ||
| 701 | for (addr = start, i = 0; addr < end; addr += PAGE_SIZE, i++) { | ||
| 702 | struct nouveau_dmem_chunk *chunk; | ||
| 703 | struct page *spage, *dpage; | ||
| 704 | u64 src_addr, dst_addr; | ||
| 705 | |||
| 706 | dpage = migrate_pfn_to_page(dst_pfns[i]); | ||
| 707 | if (!dpage || dst_pfns[i] == MIGRATE_PFN_ERROR) | ||
| 708 | continue; | ||
| 709 | |||
| 710 | chunk = (void *)hmm_devmem_page_get_drvdata(dpage); | ||
| 711 | dst_addr = page_to_pfn(dpage) - chunk->pfn_first; | ||
| 712 | dst_addr = (dst_addr << PAGE_SHIFT) + chunk->bo->bo.offset; | ||
| 713 | |||
| 714 | spage = migrate_pfn_to_page(src_pfns[i]); | ||
| 715 | if (!spage || !(src_pfns[i] & MIGRATE_PFN_MIGRATE)) { | ||
| 716 | nouveau_dmem_page_free_locked(drm, dpage); | ||
| 717 | dst_pfns[i] = 0; | ||
| 718 | continue; | ||
| 719 | } | ||
| 720 | |||
| 721 | migrate->dma[migrate->dma_nr] = | ||
| 722 | dma_map_page_attrs(dev, spage, 0, PAGE_SIZE, | ||
| 723 | PCI_DMA_BIDIRECTIONAL, | ||
| 724 | DMA_ATTR_SKIP_CPU_SYNC); | ||
| 725 | if (dma_mapping_error(dev, migrate->dma[migrate->dma_nr])) { | ||
| 726 | nouveau_dmem_page_free_locked(drm, dpage); | ||
| 727 | dst_pfns[i] = 0; | ||
| 728 | continue; | ||
| 729 | } | ||
| 730 | |||
| 731 | src_addr = migrate->dma[migrate->dma_nr++]; | ||
| 732 | |||
| 733 | ret = copy(drm, 1, NOUVEAU_APER_VRAM, dst_addr, | ||
| 734 | NOUVEAU_APER_HOST, src_addr); | ||
| 735 | if (ret) { | ||
| 736 | nouveau_dmem_page_free_locked(drm, dpage); | ||
| 737 | dst_pfns[i] = 0; | ||
| 738 | continue; | ||
| 739 | } | ||
| 740 | } | ||
| 741 | |||
| 742 | nouveau_fence_new(drm->dmem->migrate.chan, false, &migrate->fence); | ||
| 743 | |||
| 744 | return; | ||
| 745 | |||
| 746 | error: | ||
| 747 | for (addr = start, i = 0; addr < end; addr += PAGE_SIZE, ++i) { | ||
| 748 | struct page *page; | ||
| 749 | |||
| 750 | if (!dst_pfns[i] || dst_pfns[i] == MIGRATE_PFN_ERROR) | ||
| 751 | continue; | ||
| 752 | |||
| 753 | page = migrate_pfn_to_page(dst_pfns[i]); | ||
| 754 | dst_pfns[i] = MIGRATE_PFN_ERROR; | ||
| 755 | if (page == NULL) | ||
| 756 | continue; | ||
| 757 | |||
| 758 | __free_page(page); | ||
| 759 | } | ||
| 760 | } | ||
| 761 | |||
| 762 | void nouveau_dmem_migrate_finalize_and_map(struct vm_area_struct *vma, | ||
| 763 | const unsigned long *src_pfns, | ||
| 764 | const unsigned long *dst_pfns, | ||
| 765 | unsigned long start, | ||
| 766 | unsigned long end, | ||
| 767 | void *private) | ||
| 768 | { | ||
| 769 | struct nouveau_migrate *migrate = private; | ||
| 770 | struct nouveau_drm *drm = migrate->drm; | ||
| 771 | |||
| 772 | if (migrate->fence) { | ||
| 773 | nouveau_fence_wait(migrate->fence, true, false); | ||
| 774 | nouveau_fence_unref(&migrate->fence); | ||
| 775 | } else { | ||
| 776 | /* | ||
| 777 | * FIXME wait for channel to be IDLE before finalizing | ||
| 778 | * the hmem object below (nouveau_migrate_hmem_fini()) ? | ||
| 779 | */ | ||
| 780 | } | ||
| 781 | |||
| 782 | while (migrate->dma_nr--) { | ||
| 783 | dma_unmap_page(drm->dev->dev, migrate->dma[migrate->dma_nr], | ||
| 784 | PAGE_SIZE, PCI_DMA_BIDIRECTIONAL); | ||
| 785 | } | ||
| 786 | kfree(migrate->dma); | ||
| 787 | |||
| 788 | /* | ||
| 789 | * FIXME optimization: update GPU page table to point to newly | ||
| 790 | * migrated memory. | ||
| 791 | */ | ||
| 792 | } | ||
| 793 | |||
| 794 | static const struct migrate_vma_ops nouveau_dmem_migrate_ops = { | ||
| 795 | .alloc_and_copy = nouveau_dmem_migrate_alloc_and_copy, | ||
| 796 | .finalize_and_map = nouveau_dmem_migrate_finalize_and_map, | ||
| 797 | }; | ||
| 798 | |||
| 799 | int | ||
| 800 | nouveau_dmem_migrate_vma(struct nouveau_drm *drm, | ||
| 801 | struct vm_area_struct *vma, | ||
| 802 | unsigned long start, | ||
| 803 | unsigned long end) | ||
| 804 | { | ||
| 805 | unsigned long *src_pfns, *dst_pfns, npages; | ||
| 806 | struct nouveau_migrate migrate = {0}; | ||
| 807 | unsigned long i, c, max; | ||
| 808 | int ret = 0; | ||
| 809 | |||
| 810 | npages = (end - start) >> PAGE_SHIFT; | ||
| 811 | max = min(SG_MAX_SINGLE_ALLOC, npages); | ||
| 812 | src_pfns = kzalloc(sizeof(long) * max, GFP_KERNEL); | ||
| 813 | if (src_pfns == NULL) | ||
| 814 | return -ENOMEM; | ||
| 815 | dst_pfns = kzalloc(sizeof(long) * max, GFP_KERNEL); | ||
| 816 | if (dst_pfns == NULL) { | ||
| 817 | kfree(src_pfns); | ||
| 818 | return -ENOMEM; | ||
| 819 | } | ||
| 820 | |||
| 821 | migrate.drm = drm; | ||
| 822 | migrate.vma = vma; | ||
| 823 | migrate.npages = npages; | ||
| 824 | for (i = 0; i < npages; i += c) { | ||
| 825 | unsigned long next; | ||
| 826 | |||
| 827 | c = min(SG_MAX_SINGLE_ALLOC, npages); | ||
| 828 | next = start + (c << PAGE_SHIFT); | ||
| 829 | ret = migrate_vma(&nouveau_dmem_migrate_ops, vma, start, | ||
| 830 | next, src_pfns, dst_pfns, &migrate); | ||
| 831 | if (ret) | ||
| 832 | goto out; | ||
| 833 | start = next; | ||
| 834 | } | ||
| 835 | |||
| 836 | out: | ||
| 837 | kfree(dst_pfns); | ||
| 838 | kfree(src_pfns); | ||
| 839 | return ret; | ||
| 840 | } | ||
| 841 | |||
| 842 | static inline bool | ||
| 843 | nouveau_dmem_page(struct nouveau_drm *drm, struct page *page) | ||
| 844 | { | ||
| 845 | if (!is_device_private_page(page)) | ||
| 846 | return false; | ||
| 847 | |||
| 848 | if (drm->dmem->devmem != page->pgmap->data) | ||
| 849 | return false; | ||
| 850 | |||
| 851 | return true; | ||
| 852 | } | ||
| 853 | |||
| 854 | void | ||
| 855 | nouveau_dmem_convert_pfn(struct nouveau_drm *drm, | ||
| 856 | struct hmm_range *range) | ||
| 857 | { | ||
| 858 | unsigned long i, npages; | ||
| 859 | |||
| 860 | npages = (range->end - range->start) >> PAGE_SHIFT; | ||
| 861 | for (i = 0; i < npages; ++i) { | ||
| 862 | struct nouveau_dmem_chunk *chunk; | ||
| 863 | struct page *page; | ||
| 864 | uint64_t addr; | ||
| 865 | |||
| 866 | page = hmm_pfn_to_page(range, range->pfns[i]); | ||
| 867 | if (page == NULL) | ||
| 868 | continue; | ||
| 869 | |||
| 870 | if (!(range->pfns[i] & range->flags[HMM_PFN_DEVICE_PRIVATE])) { | ||
| 871 | continue; | ||
| 872 | } | ||
| 873 | |||
| 874 | if (!nouveau_dmem_page(drm, page)) { | ||
| 875 | WARN(1, "Some unknown device memory !\n"); | ||
| 876 | range->pfns[i] = 0; | ||
| 877 | continue; | ||
| 878 | } | ||
| 879 | |||
| 880 | chunk = (void *)hmm_devmem_page_get_drvdata(page); | ||
| 881 | addr = page_to_pfn(page) - chunk->pfn_first; | ||
| 882 | addr = (addr + chunk->bo->bo.mem.start) << PAGE_SHIFT; | ||
| 883 | |||
| 884 | range->pfns[i] &= ((1UL << range->pfn_shift) - 1); | ||
| 885 | range->pfns[i] |= (addr >> PAGE_SHIFT) << range->pfn_shift; | ||
| 886 | } | ||
| 887 | } | ||
| @@ -0,0 +1,60 @@ | |||
| 1 | /* | ||
| 2 | * Copyright 2018 Red Hat Inc. | ||
| 3 | * | ||
| 4 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 5 | * copy of this software and associated documentation files (the "Software"), | ||
| 6 | * to deal in the Software without restriction, including without limitation | ||
| 7 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 8 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 9 | * Software is furnished to do so, subject to the following conditions: | ||
| 10 | * | ||
| 11 | * The above copyright notice and this permission notice shall be included in | ||
| 12 | * all copies or substantial portions of the Software. | ||
| 13 | * | ||
| 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 17 | * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | ||
| 18 | * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | ||
| 19 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | ||
| 20 | * OTHER DEALINGS IN THE SOFTWARE. | ||
| 21 | */ | ||
| 22 | #ifndef __NOUVEAU_DMEM_H__ | ||
| 23 | #define __NOUVEAU_DMEM_H__ | ||
| 24 | #include <nvif/os.h> | ||
| 25 | struct drm_device; | ||
| 26 | struct drm_file; | ||
| 27 | struct nouveau_drm; | ||
| 28 | struct hmm_range; | ||
| 29 | |||
| 30 | #if IS_ENABLED(CONFIG_DRM_NOUVEAU_SVM) | ||
| 31 | void nouveau_dmem_init(struct nouveau_drm *); | ||
| 32 | void nouveau_dmem_fini(struct nouveau_drm *); | ||
| 33 | void nouveau_dmem_suspend(struct nouveau_drm *); | ||
| 34 | void nouveau_dmem_resume(struct nouveau_drm *); | ||
| 35 | |||
| 36 | int nouveau_dmem_migrate_vma(struct nouveau_drm *drm, | ||
| 37 | struct vm_area_struct *vma, | ||
| 38 | unsigned long start, | ||
| 39 | unsigned long end); | ||
| 40 | |||
| 41 | void nouveau_dmem_convert_pfn(struct nouveau_drm *drm, | ||
| 42 | struct hmm_range *range); | ||
| 43 | #else /* IS_ENABLED(CONFIG_DRM_NOUVEAU_SVM) */ | ||
| 44 | static inline void nouveau_dmem_init(struct nouveau_drm *drm) {} | ||
| 45 | static inline void nouveau_dmem_fini(struct nouveau_drm *drm) {} | ||
| 46 | static inline void nouveau_dmem_suspend(struct nouveau_drm *drm) {} | ||
| 47 | static inline void nouveau_dmem_resume(struct nouveau_drm *drm) {} | ||
| 48 | |||
| 49 | static inline int nouveau_dmem_migrate_vma(struct nouveau_drm *drm, | ||
| 50 | struct vm_area_struct *vma, | ||
| 51 | unsigned long start, | ||
| 52 | unsigned long end) | ||
| 53 | { | ||
| 54 | return 0; | ||
| 55 | } | ||
| 56 | |||
| 57 | static inline void nouveau_dmem_convert_pfn(struct nouveau_drm *drm, | ||
| 58 | struct hmm_range *range) {} | ||
| 59 | #endif /* IS_ENABLED(CONFIG_DRM_NOUVEAU_SVM) */ | ||
| 60 | #endif | ||
| @@ -44,7 +44,6 @@ · 43 unmodified lines | |||
| 44 | #include <nvif/class.h> | 44 | #include <nvif/class.h> |
| 45 | #include <nvif/cl0002.h> | 45 | #include <nvif/cl0002.h> |
| 46 | #include <nvif/cla06f.h> | 46 | #include <nvif/cla06f.h> |
| 47 | #include <nvif/if0004.h> | ||
| 48 | 47 | ||
| 49 | #include "nouveau_drv.h" | 48 | #include "nouveau_drv.h" |
| 50 | #include "nouveau_dma.h" | 49 | #include "nouveau_dma.h" |
| @@ -63,6 +62,8 @@ · 12 unmodified lines | |||
| 63 | #include "nouveau_usif.h" | 62 | #include "nouveau_usif.h" |
| 64 | #include "nouveau_connector.h" | 63 | #include "nouveau_connector.h" |
| 65 | #include "nouveau_platform.h" | 64 | #include "nouveau_platform.h" |
| 65 | #include "nouveau_svm.h" | ||
| 66 | #include "nouveau_dmem.h" | ||
| 66 | 67 | ||
| 67 | MODULE_PARM_DESC(config, "option string to pass to driver core"); | 68 | MODULE_PARM_DESC(config, "option string to pass to driver core"); |
| 68 | static char *nouveau_config; | 69 | static char *nouveau_config; |
| @@ -173,6 +174,7 @@ nouveau_cli_fini(struct nouveau_cli *cli) · 104 unmodified lines | |||
| 173 | WARN_ON(!list_empty(&cli->worker)); | 174 | WARN_ON(!list_empty(&cli->worker)); |
| 174 | 175 | ||
| 175 | usif_client_fini(cli); | 176 | usif_client_fini(cli); |
| 177 | nouveau_vmm_fini(&cli->svm); | ||
| 176 | nouveau_vmm_fini(&cli->vmm); | 178 | nouveau_vmm_fini(&cli->vmm); |
| 177 | nvif_mmu_fini(&cli->mmu); | 179 | nvif_mmu_fini(&cli->mmu); |
| 178 | nvif_device_fini(&cli->device); | 180 | nvif_device_fini(&cli->device); |
| @@ -283,19 +285,134 @@ done: · 104 unmodified lines | |||
| 283 | } | 285 | } |
| 284 | 286 | ||
| 285 | static void | 287 | static void |
| 286 | nouveau_accel_fini(struct nouveau_drm *drm) | 288 | nouveau_accel_ce_fini(struct nouveau_drm *drm) |
| 289 | { | ||
| 290 | nouveau_channel_idle(drm->cechan); | ||
| 291 | nvif_object_fini(&drm->ttm.copy); | ||
| 292 | nouveau_channel_del(&drm->cechan); | ||
| 293 | } | ||
| 294 | |||
| 295 | static void | ||
| 296 | nouveau_accel_ce_init(struct nouveau_drm *drm) | ||
| 297 | { | ||
| 298 | struct nvif_device *device = &drm->client.device; | ||
| 299 | int ret = 0; | ||
| 300 | |||
| 301 | /* Allocate channel that has access to a (preferably async) copy | ||
| 302 | * engine, to use for TTM buffer moves. | ||
| 303 | */ | ||
| 304 | if (device->info.family >= NV_DEVICE_INFO_V0_KEPLER) { | ||
| 305 | ret = nouveau_channel_new(drm, device, | ||
| 306 | nvif_fifo_runlist_ce(device), 0, | ||
| 307 | true, &drm->cechan); | ||
| 308 | } else | ||
| 309 | if (device->info.chipset >= 0xa3 && | ||
| 310 | device->info.chipset != 0xaa && | ||
| 311 | device->info.chipset != 0xac) { | ||
| 312 | /* Prior to Kepler, there's only a single runlist, so all | ||
| 313 | * engines can be accessed from any channel. | ||
| 314 | * | ||
| 315 | * We still want to use a separate channel though. | ||
| 316 | */ | ||
| 317 | ret = nouveau_channel_new(drm, device, NvDmaFB, NvDmaTT, false, | ||
| 318 | &drm->cechan); | ||
| 319 | } | ||
| 320 | |||
| 321 | if (ret) | ||
| 322 | NV_ERROR(drm, "failed to create ce channel, %d\n", ret); | ||
| 323 | } | ||
| 324 | |||
| 325 | static void | ||
| 326 | nouveau_accel_gr_fini(struct nouveau_drm *drm) | ||
| 287 | { | 327 | { |
| 288 | nouveau_channel_idle(drm->channel); | 328 | nouveau_channel_idle(drm->channel); |
| 289 | nvif_object_fini(&drm->ntfy); | 329 | nvif_object_fini(&drm->ntfy); |
| 290 | nvkm_gpuobj_del(&drm->notify); | 330 | nvkm_gpuobj_del(&drm->notify); |
| 291 | nvif_notify_fini(&drm->flip); | ||
| 292 | nvif_object_fini(&drm->nvsw); | 331 | nvif_object_fini(&drm->nvsw); |
| 293 | nouveau_channel_del(&drm->channel); | 332 | nouveau_channel_del(&drm->channel); |
| 333 | } | ||
| 294 | 334 | ||
| 295 | nouveau_channel_idle(drm->cechan); | 335 | static void |
| 296 | nvif_object_fini(&drm->ttm.copy); | 336 | nouveau_accel_gr_init(struct nouveau_drm *drm) |
| 297 | nouveau_channel_del(&drm->cechan); | 337 | { |
| 338 | struct nvif_device *device = &drm->client.device; | ||
| 339 | u32 arg0, arg1; | ||
| 340 | int ret; | ||
| 341 | |||
| 342 | /* Allocate channel that has access to the graphics engine. */ | ||
| 343 | if (device->info.family >= NV_DEVICE_INFO_V0_KEPLER) { | ||
| 344 | arg0 = nvif_fifo_runlist(device, NV_DEVICE_INFO_ENGINE_GR); | ||
| 345 | arg1 = 1; | ||
| 346 | } else { | ||
| 347 | arg0 = NvDmaFB; | ||
| 348 | arg1 = NvDmaTT; | ||
| 349 | } | ||
| 298 | 350 | ||
| 351 | ret = nouveau_channel_new(drm, device, arg0, arg1, false, | ||
| 352 | &drm->channel); | ||
| 353 | if (ret) { | ||
| 354 | NV_ERROR(drm, "failed to create kernel channel, %d\n", ret); | ||
| 355 | nouveau_accel_gr_fini(drm); | ||
| 356 | return; | ||
| 357 | } | ||
| 358 | |||
| 359 | /* A SW class is used on pre-NV50 HW to assist with handling the | ||
| 360 | * synchronisation of page flips, as well as to implement fences | ||
| 361 | * on TNT/TNT2 HW that lacks any kind of support in host. | ||
| 362 | */ | ||
| 363 | if (device->info.family < NV_DEVICE_INFO_V0_TESLA) { | ||
| 364 | ret = nvif_object_init(&drm->channel->user, NVDRM_NVSW, | ||
| 365 | nouveau_abi16_swclass(drm), NULL, 0, | ||
| 366 | &drm->nvsw); | ||
| 367 | if (ret == 0) { | ||
| 368 | ret = RING_SPACE(drm->channel, 2); | ||
| 369 | if (ret == 0) { | ||
| 370 | BEGIN_NV04(drm->channel, NvSubSw, 0, 1); | ||
| 371 | OUT_RING (drm->channel, drm->nvsw.handle); | ||
| 372 | } | ||
| 373 | } | ||
| 374 | |||
| 375 | if (ret) { | ||
| 376 | NV_ERROR(drm, "failed to allocate sw class, %d\n", ret); | ||
| 377 | nouveau_accel_gr_fini(drm); | ||
| 378 | return; | ||
| 379 | } | ||
| 380 | } | ||
| 381 | |||
| 382 | /* NvMemoryToMemoryFormat requires a notifier ctxdma for some reason, | ||
| 383 | * even if notification is never requested, so, allocate a ctxdma on | ||
| 384 | * any GPU where it's possible we'll end up using M2MF for BO moves. | ||
| 385 | */ | ||
| 386 | if (device->info.family < NV_DEVICE_INFO_V0_FERMI) { | ||
| 387 | ret = nvkm_gpuobj_new(nvxx_device(device), 32, 0, false, NULL, | ||
| 388 | &drm->notify); | ||
| 389 | if (ret) { | ||
| 390 | NV_ERROR(drm, "failed to allocate notifier, %d\n", ret); | ||
| 391 | nouveau_accel_gr_fini(drm); | ||
| 392 | return; | ||
| 393 | } | ||
| 394 | |||
| 395 | ret = nvif_object_init(&drm->channel->user, NvNotify0, | ||
| 396 | NV_DMA_IN_MEMORY, | ||
| 397 | &(struct nv_dma_v0) { | ||
| 398 | .target = NV_DMA_V0_TARGET_VRAM, | ||
| 399 | .access = NV_DMA_V0_ACCESS_RDWR, | ||
| 400 | .start = drm->notify->addr, | ||
| 401 | .limit = drm->notify->addr + 31 | ||
| 402 | }, sizeof(struct nv_dma_v0), | ||
| 403 | &drm->ntfy); | ||
| 404 | if (ret) { | ||
| 405 | nouveau_accel_gr_fini(drm); | ||
| 406 | return; | ||
| 407 | } | ||
| 408 | } | ||
| 409 | } | ||
| 410 | |||
| 411 | static void | ||
| 412 | nouveau_accel_fini(struct nouveau_drm *drm) | ||
| 413 | { | ||
| 414 | nouveau_accel_ce_fini(drm); | ||
| 415 | nouveau_accel_gr_fini(drm); | ||
| 299 | if (drm->fence) | 416 | if (drm->fence) |
| 300 | nouveau_fence(drm)->dtor(drm); | 417 | nouveau_fence(drm)->dtor(drm); |
| 301 | } | 418 | } |
| @@ -305,23 +422,16 @@ nouveau_accel_init(struct nouveau_drm *drm) · 3 unmodified lines | |||
| 305 | { | 422 | { |
| 306 | struct nvif_device *device = &drm->client.device; | 423 | struct nvif_device *device = &drm->client.device; |
| 307 | struct nvif_sclass *sclass; | 424 | struct nvif_sclass *sclass; |
| 308 | u32 arg0, arg1; | ||
| 309 | int ret, i, n; | 425 | int ret, i, n; |
| 310 | 426 | ||
| 311 | if (nouveau_noaccel) | 427 | if (nouveau_noaccel) |
| 312 | return; | 428 | return; |
| 313 | 429 | ||
| 430 | /* Initialise global support for channels, and synchronisation. */ | ||
| 314 | ret = nouveau_channels_init(drm); | 431 | ret = nouveau_channels_init(drm); |
| 315 | if (ret) | 432 | if (ret) |
| 316 | return; | 433 | return; |
| 317 | 434 | ||
| 318 | if (drm->client.device.info.family >= NV_DEVICE_INFO_V0_VOLTA) { | ||
| 319 | ret = nvif_user_init(device); | ||
| 320 | if (ret) | ||
| 321 | return; | ||
| 322 | } | ||
| 323 | |||
| 324 | /* initialise synchronisation routines */ | ||
| 325 | /*XXX: this is crap, but the fence/channel stuff is a little | 435 | /*XXX: this is crap, but the fence/channel stuff is a little |
| 326 | * backwards in some places. this will be fixed. | 436 | * backwards in some places. this will be fixed. |
| 327 | */ | 437 | */ |
| @@ -368,95 +478,18 @@ nouveau_accel_init(struct nouveau_drm *drm) · 40 unmodified lines | |||
| 368 | return; | 478 | return; |
| 369 | } | 479 | } |
| 370 | 480 | ||
| 371 | if (device->info.family >= NV_DEVICE_INFO_V0_KEPLER) { | 481 | /* Volta requires access to a doorbell register for kickoff. */ |
| 372 | ret = nouveau_channel_new(drm, &drm->client.device, | 482 | if (drm->client.device.info.family >= NV_DEVICE_INFO_V0_VOLTA) { |
| 373 | nvif_fifo_runlist_ce(device), 0, | 483 | ret = nvif_user_init(device); |
| 374 | true, &drm->cechan); | ||
| 375 | if (ret) | ||
| 376 | NV_ERROR(drm, "failed to create ce channel, %d\n", ret); | ||
| 377 | |||
| 378 | arg0 = nvif_fifo_runlist(device, NV_DEVICE_INFO_ENGINE_GR); | ||
| 379 | arg1 = 1; | ||
| 380 | } else | ||
| 381 | if (device->info.chipset >= 0xa3 && | ||
| 382 | device->info.chipset != 0xaa && | ||
| 383 | device->info.chipset != 0xac) { | ||
| 384 | ret = nouveau_channel_new(drm, &drm->client.device, | ||
| 385 | NvDmaFB, NvDmaTT, false, | ||
| 386 | &drm->cechan); | ||
| 387 | if (ret) | 484 | if (ret) |
| 388 | NV_ERROR(drm, "failed to create ce channel, %d\n", ret); | ||
| 389 | |||
| 390 | arg0 = NvDmaFB; | ||
| 391 | arg1 = NvDmaTT; | ||
| 392 | } else { | ||
| 393 | arg0 = NvDmaFB; | ||
| 394 | arg1 = NvDmaTT; | ||
| 395 | } | ||
| 396 | |||
| 397 | ret = nouveau_channel_new(drm, &drm->client.device, | ||
| 398 | arg0, arg1, false, &drm->channel); | ||
| 399 | if (ret) { | ||
| 400 | NV_ERROR(drm, "failed to create kernel channel, %d\n", ret); | ||
| 401 | nouveau_accel_fini(drm); | ||
| 402 | return; | ||
| 403 | } | ||
| 404 | |||
| 405 | if (device->info.family < NV_DEVICE_INFO_V0_TESLA) { | ||
| 406 | ret = nvif_object_init(&drm->channel->user, NVDRM_NVSW, | ||
| 407 | nouveau_abi16_swclass(drm), NULL, 0, | ||
| 408 | &drm->nvsw); | ||
| 409 | if (ret == 0) { | ||
| 410 | ret = RING_SPACE(drm->channel, 2); | ||
| 411 | if (ret == 0) { | ||
| 412 | BEGIN_NV04(drm->channel, NvSubSw, 0, 1); | ||
| 413 | OUT_RING (drm->channel, drm->nvsw.handle); | ||
| 414 | } | ||
| 415 | |||
| 416 | ret = nvif_notify_init(&drm->nvsw, | ||
| 417 | nouveau_flip_complete, | ||
| 418 | false, NV04_NVSW_NTFY_UEVENT, | ||
| 419 | NULL, 0, 0, &drm->flip); | ||
| 420 | if (ret == 0) | ||
| 421 | ret = nvif_notify_get(&drm->flip); | ||
| 422 | if (ret) { | ||
| 423 | nouveau_accel_fini(drm); | ||
| 424 | return; | ||
| 425 | } | ||
| 426 | } | ||
| 427 | |||
| 428 | if (ret) { | ||
| 429 | NV_ERROR(drm, "failed to allocate sw class, %d\n", ret); | ||
| 430 | nouveau_accel_fini(drm); | ||
| 431 | return; | ||
| 432 | } | ||
| 433 | } | ||
| 434 | |||
| 435 | if (device->info.family < NV_DEVICE_INFO_V0_FERMI) { | ||
| 436 | ret = nvkm_gpuobj_new(nvxx_device(&drm->client.device), 32, 0, | ||
| 437 | false, NULL, &drm->notify); | ||
| 438 | if (ret) { | ||
| 439 | NV_ERROR(drm, "failed to allocate notifier, %d\n", ret); | ||
| 440 | nouveau_accel_fini(drm); | ||
| 441 | return; | 485 | return; |
| 442 | } | ||
| 443 | |||
| 444 | ret = nvif_object_init(&drm->channel->user, NvNotify0, | ||
| 445 | NV_DMA_IN_MEMORY, | ||
| 446 | &(struct nv_dma_v0) { | ||
| 447 | .target = NV_DMA_V0_TARGET_VRAM, | ||
| 448 | .access = NV_DMA_V0_ACCESS_RDWR, | ||
| 449 | .start = drm->notify->addr, | ||
| 450 | .limit = drm->notify->addr + 31 | ||
| 451 | }, sizeof(struct nv_dma_v0), | ||
| 452 | &drm->ntfy); | ||
| 453 | if (ret) { | ||
| 454 | nouveau_accel_fini(drm); | ||
| 455 | return; | ||
| 456 | } | ||
| 457 | } | 486 | } |
| 458 | 487 | ||
| 488 | /* Allocate channels we need to support various functions. */ | ||
| 489 | nouveau_accel_gr_init(drm); | ||
| 490 | nouveau_accel_ce_init(drm); | ||
| 459 | 491 | ||
| 492 | /* Initialise accelerated TTM buffer moves. */ | ||
| 460 | nouveau_bo_move_init(drm); | 493 | nouveau_bo_move_init(drm); |
| 461 | } | 494 | } |
| 462 | 495 | ||
| @@ -504,19 +537,22 @@ nouveau_drm_device_init(struct drm_device *dev) · 41 unmodified lines | |||
| 504 | if (ret) | 537 | if (ret) |
| 505 | goto fail_bios; | 538 | goto fail_bios; |
| 506 | 539 | ||
| 540 | nouveau_accel_init(drm); | ||
| 541 | |||
| 507 | ret = nouveau_display_create(dev); | 542 | ret = nouveau_display_create(dev); |
| 508 | if (ret) | 543 | if (ret) |
| 509 | goto fail_dispctor; | 544 | goto fail_dispctor; |
| 510 | 545 | ||
| 511 | if (dev->mode_config.num_crtc) { | 546 | if (dev->mode_config.num_crtc) { |
| 512 | ret = nouveau_display_init(dev); | 547 | ret = nouveau_display_init(dev, false, false); |
| 513 | if (ret) | 548 | if (ret) |
| 514 | goto fail_dispinit; | 549 | goto fail_dispinit; |
| 515 | } | 550 | } |
| 516 | 551 | ||
| 517 | nouveau_debugfs_init(drm); | 552 | nouveau_debugfs_init(drm); |
| 518 | nouveau_hwmon_init(dev); | 553 | nouveau_hwmon_init(dev); |
| 519 | nouveau_accel_init(drm); | 554 | nouveau_svm_init(drm); |
| 555 | nouveau_dmem_init(drm); | ||
| 520 | nouveau_fbcon_init(dev); | 556 | nouveau_fbcon_init(dev); |
| 521 | nouveau_led_init(dev); | 557 | nouveau_led_init(dev); |
| 522 | 558 | ||
| @@ -534,6 +570,7 @@ nouveau_drm_device_init(struct drm_device *dev) · 11 unmodified lines | |||
| 534 | fail_dispinit: | 570 | fail_dispinit: |
| 535 | nouveau_display_destroy(dev); | 571 | nouveau_display_destroy(dev); |
| 536 | fail_dispctor: | 572 | fail_dispctor: |
| 573 | nouveau_accel_fini(drm); | ||
| 537 | nouveau_bios_takedown(dev); | 574 | nouveau_bios_takedown(dev); |
| 538 | fail_bios: | 575 | fail_bios: |
| 539 | nouveau_ttm_fini(drm); | 576 | nouveau_ttm_fini(drm); |
| @@ -559,7 +596,8 @@ nouveau_drm_device_fini(struct drm_device *dev) · 19 unmodified lines | |||
| 559 | 596 | ||
| 560 | nouveau_led_fini(dev); | 597 | nouveau_led_fini(dev); |
| 561 | nouveau_fbcon_fini(dev); | 598 | nouveau_fbcon_fini(dev); |
| 562 | nouveau_accel_fini(drm); | 599 | nouveau_dmem_fini(drm); |
| 600 | nouveau_svm_fini(drm); | ||
| 563 | nouveau_hwmon_fini(dev); | 601 | nouveau_hwmon_fini(dev); |
| 564 | nouveau_debugfs_fini(drm); | 602 | nouveau_debugfs_fini(drm); |
| 565 | 603 | ||
| @@ -567,6 +605,7 @@ nouveau_drm_device_fini(struct drm_device *dev) · 1 unmodified line | |||
| 567 | nouveau_display_fini(dev, false, false); | 605 | nouveau_display_fini(dev, false, false); |
| 568 | nouveau_display_destroy(dev); | 606 | nouveau_display_destroy(dev); |
| 569 | 607 | ||
| 608 | nouveau_accel_fini(drm); | ||
| 570 | nouveau_bios_takedown(dev); | 609 | nouveau_bios_takedown(dev); |
| 571 | 610 | ||
| 572 | nouveau_ttm_fini(drm); | 611 | nouveau_ttm_fini(drm); |
| @@ -704,6 +743,8 @@ nouveau_do_suspend(struct drm_device *dev, bool runtime) · 131 unmodified lines | |||
| 704 | struct nouveau_drm *drm = nouveau_drm(dev); | 743 | struct nouveau_drm *drm = nouveau_drm(dev); |
| 705 | int ret; | 744 | int ret; |
| 706 | 745 | ||
| 746 | nouveau_svm_suspend(drm); | ||
| 747 | nouveau_dmem_suspend(drm); | ||
| 707 | nouveau_led_suspend(dev); | 748 | nouveau_led_suspend(dev); |
| 708 | 749 | ||
| 709 | if (dev->mode_config.num_crtc) { | 750 | if (dev->mode_config.num_crtc) { |
| @@ -780,7 +821,8 @@ nouveau_do_resume(struct drm_device *dev, bool runtime) · 70 unmodified lines | |||
| 780 | } | 821 | } |
| 781 | 822 | ||
| 782 | nouveau_led_resume(dev); | 823 | nouveau_led_resume(dev); |
| 783 | 824 | nouveau_dmem_resume(drm); | |
| 825 | nouveau_svm_resume(drm); | ||
| 784 | return 0; | 826 | return 0; |
| 785 | } | 827 | } |
| 786 | 828 | ||
| @@ -1000,6 +1042,8 @@ nouveau_ioctls[] = { · 213 unmodified lines | |||
| 1000 | DRM_IOCTL_DEF_DRV(NOUVEAU_GROBJ_ALLOC, nouveau_abi16_ioctl_grobj_alloc, DRM_AUTH|DRM_RENDER_ALLOW), | 1042 | DRM_IOCTL_DEF_DRV(NOUVEAU_GROBJ_ALLOC, nouveau_abi16_ioctl_grobj_alloc, DRM_AUTH|DRM_RENDER_ALLOW), |
| 1001 | DRM_IOCTL_DEF_DRV(NOUVEAU_NOTIFIEROBJ_ALLOC, nouveau_abi16_ioctl_notifierobj_alloc, DRM_AUTH|DRM_RENDER_ALLOW), | 1043 | DRM_IOCTL_DEF_DRV(NOUVEAU_NOTIFIEROBJ_ALLOC, nouveau_abi16_ioctl_notifierobj_alloc, DRM_AUTH|DRM_RENDER_ALLOW), |
| 1002 | DRM_IOCTL_DEF_DRV(NOUVEAU_GPUOBJ_FREE, nouveau_abi16_ioctl_gpuobj_free, DRM_AUTH|DRM_RENDER_ALLOW), | 1044 | DRM_IOCTL_DEF_DRV(NOUVEAU_GPUOBJ_FREE, nouveau_abi16_ioctl_gpuobj_free, DRM_AUTH|DRM_RENDER_ALLOW), |
| 1045 | DRM_IOCTL_DEF_DRV(NOUVEAU_SVM_INIT, nouveau_svmm_init, DRM_AUTH|DRM_RENDER_ALLOW), | ||
| 1046 | DRM_IOCTL_DEF_DRV(NOUVEAU_SVM_BIND, nouveau_svmm_bind, DRM_AUTH|DRM_RENDER_ALLOW), | ||
| 1003 | DRM_IOCTL_DEF_DRV(NOUVEAU_GEM_NEW, nouveau_gem_ioctl_new, DRM_AUTH|DRM_RENDER_ALLOW), | 1047 | DRM_IOCTL_DEF_DRV(NOUVEAU_GEM_NEW, nouveau_gem_ioctl_new, DRM_AUTH|DRM_RENDER_ALLOW), |
| 1004 | DRM_IOCTL_DEF_DRV(NOUVEAU_GEM_PUSHBUF, nouveau_gem_ioctl_pushbuf, DRM_AUTH|DRM_RENDER_ALLOW), | 1048 | DRM_IOCTL_DEF_DRV(NOUVEAU_GEM_PUSHBUF, nouveau_gem_ioctl_pushbuf, DRM_AUTH|DRM_RENDER_ALLOW), |
| 1005 | DRM_IOCTL_DEF_DRV(NOUVEAU_GEM_CPU_PREP, nouveau_gem_ioctl_cpu_prep, DRM_AUTH|DRM_RENDER_ALLOW), | 1049 | DRM_IOCTL_DEF_DRV(NOUVEAU_GEM_CPU_PREP, nouveau_gem_ioctl_cpu_prep, DRM_AUTH|DRM_RENDER_ALLOW), |
| @@ -96,6 +96,7 @@ struct nouveau_cli { · 95 unmodified lines | |||
| 96 | struct nvif_device device; | 96 | struct nvif_device device; |
| 97 | struct nvif_mmu mmu; | 97 | struct nvif_mmu mmu; |
| 98 | struct nouveau_vmm vmm; | 98 | struct nouveau_vmm vmm; |
| 99 | struct nouveau_vmm svm; | ||
| 99 | const struct nvif_mclass *mem; | 100 | const struct nvif_mclass *mem; |
| 100 | 101 | ||
| 101 | struct list_head head; | 102 | struct list_head head; |
| @@ -181,7 +182,6 @@ struct nouveau_drm { · 79 unmodified lines | |||
| 181 | struct nouveau_fbdev *fbcon; | 182 | struct nouveau_fbdev *fbcon; |
| 182 | struct nvif_object nvsw; | 183 | struct nvif_object nvsw; |
| 183 | struct nvif_object ntfy; | 184 | struct nvif_object ntfy; |
| 184 | struct nvif_notify flip; | ||
| 185 | 185 | ||
| 186 | /* nv10-nv40 tiling regions */ | 186 | /* nv10-nv40 tiling regions */ |
| 187 | struct { | 187 | struct { |
| @@ -210,6 +210,10 @@ struct nouveau_drm { · 22 unmodified lines | |||
| 210 | bool have_disp_power_ref; | 210 | bool have_disp_power_ref; |
| 211 | 211 | ||
| 212 | struct dev_pm_domain vga_pm_domain; | 212 | struct dev_pm_domain vga_pm_domain; |
| 213 | |||
| 214 | struct nouveau_svm *svm; | ||
| 215 | |||
| 216 | struct nouveau_dmem *dmem; | ||
| 213 | }; | 217 | }; |
| 214 | 218 | ||
| 215 | static inline struct nouveau_drm * | 219 | static inline struct nouveau_drm * |
| @@ -353,7 +353,7 @@ nouveau_fbcon_create(struct drm_fb_helper *helper, · 352 unmodified lines | |||
| 353 | 353 | ||
| 354 | chan = nouveau_nofbaccel ? NULL : drm->channel; | 354 | chan = nouveau_nofbaccel ? NULL : drm->channel; |
| 355 | if (chan && device->info.family >= NV_DEVICE_INFO_V0_TESLA) { | 355 | if (chan && device->info.family >= NV_DEVICE_INFO_V0_TESLA) { |
| 356 | ret = nouveau_vma_new(nvbo, &drm->client.vmm, &fb->vma); | 356 | ret = nouveau_vma_new(nvbo, chan->vmm, &fb->vma); |
| 357 | if (ret) { | 357 | if (ret) { |
| 358 | NV_ERROR(drm, "failed to map fb into chan: %d\n", ret); | 358 | NV_ERROR(drm, "failed to map fb into chan: %d\n", ret); |
| 359 | chan = NULL; | 359 | chan = NULL; |
| @@ -82,8 +82,6 @@ int nv50_fence_create(struct nouveau_drm *); · 81 unmodified lines | |||
| 82 | int nv84_fence_create(struct nouveau_drm *); | 82 | int nv84_fence_create(struct nouveau_drm *); |
| 83 | int nvc0_fence_create(struct nouveau_drm *); | 83 | int nvc0_fence_create(struct nouveau_drm *); |
| 84 | 84 | ||
| 85 | int nouveau_flip_complete(struct nvif_notify *); | ||
| 86 | |||
| 87 | struct nv84_fence_chan { | 85 | struct nv84_fence_chan { |
| 88 | struct nouveau_fence_chan base; | 86 | struct nouveau_fence_chan base; |
| 89 | struct nouveau_vma *vma; | 87 | struct nouveau_vma *vma; |
| @@ -68,10 +68,11 @@ nouveau_gem_object_open(struct drm_gem_object *gem, struct drm_file *file_priv) · 67 unmodified lines | |||
| 68 | struct nouveau_bo *nvbo = nouveau_gem_object(gem); | 68 | struct nouveau_bo *nvbo = nouveau_gem_object(gem); |
| 69 | struct nouveau_drm *drm = nouveau_bdev(nvbo->bo.bdev); | 69 | struct nouveau_drm *drm = nouveau_bdev(nvbo->bo.bdev); |
| 70 | struct device *dev = drm->dev->dev; | 70 | struct device *dev = drm->dev->dev; |
| 71 | struct nouveau_vmm *vmm = cli->svm.cli ? &cli->svm : &cli->vmm; | ||
| 71 | struct nouveau_vma *vma; | 72 | struct nouveau_vma *vma; |
| 72 | int ret; | 73 | int ret; |
| 73 | 74 | ||
| 74 | if (cli->vmm.vmm.object.oclass < NVIF_CLASS_VMM_NV50) | 75 | if (vmm->vmm.object.oclass < NVIF_CLASS_VMM_NV50) |
| 75 | return 0; | 76 | return 0; |
| 76 | 77 | ||
| 77 | ret = ttm_bo_reserve(&nvbo->bo, false, false, NULL); | 78 | ret = ttm_bo_reserve(&nvbo->bo, false, false, NULL); |
| @@ -82,7 +83,7 @@ nouveau_gem_object_open(struct drm_gem_object *gem, struct drm_file *file_priv) · 4 unmodified lines | |||
| 82 | if (ret < 0 && ret != -EACCES) | 83 | if (ret < 0 && ret != -EACCES) |
| 83 | goto out; | 84 | goto out; |
| 84 | 85 | ||
| 85 | ret = nouveau_vma_new(nvbo, &cli->vmm, &vma); | 86 | ret = nouveau_vma_new(nvbo, vmm, &vma); |
| 86 | pm_runtime_mark_last_busy(dev); | 87 | pm_runtime_mark_last_busy(dev); |
| 87 | pm_runtime_put_autosuspend(dev); | 88 | pm_runtime_put_autosuspend(dev); |
| 88 | out: | 89 | out: |
| @@ -142,17 +143,18 @@ nouveau_gem_object_close(struct drm_gem_object *gem, struct drm_file *file_priv) · 53 unmodified lines | |||
| 142 | struct nouveau_bo *nvbo = nouveau_gem_object(gem); | 143 | struct nouveau_bo *nvbo = nouveau_gem_object(gem); |
| 143 | struct nouveau_drm *drm = nouveau_bdev(nvbo->bo.bdev); | 144 | struct nouveau_drm *drm = nouveau_bdev(nvbo->bo.bdev); |
| 144 | struct device *dev = drm->dev->dev; | 145 | struct device *dev = drm->dev->dev; |
| 146 | struct nouveau_vmm *vmm = cli->svm.cli ? &cli->svm : & cli->vmm; | ||
| 145 | struct nouveau_vma *vma; | 147 | struct nouveau_vma *vma; |
| 146 | int ret; | 148 | int ret; |
| 147 | 149 | ||
| 148 | if (cli->vmm.vmm.object.oclass < NVIF_CLASS_VMM_NV50) | 150 | if (vmm->vmm.object.oclass < NVIF_CLASS_VMM_NV50) |
| 149 | return; | 151 | return; |
| 150 | 152 | ||
| 151 | ret = ttm_bo_reserve(&nvbo->bo, false, false, NULL); | 153 | ret = ttm_bo_reserve(&nvbo->bo, false, false, NULL); |
| 152 | if (ret) | 154 | if (ret) |
| 153 | return; | 155 | return; |
| 154 | 156 | ||
| 155 | vma = nouveau_vma_find(nvbo, &cli->vmm); | 157 | vma = nouveau_vma_find(nvbo, vmm); |
| 156 | if (vma) { | 158 | if (vma) { |
| 157 | if (--vma->refs == 0) { | 159 | if (--vma->refs == 0) { |
| 158 | ret = pm_runtime_get_sync(dev); | 160 | ret = pm_runtime_get_sync(dev); |
| @@ -219,6 +221,7 @@ nouveau_gem_info(struct drm_file *file_priv, struct drm_gem_object *gem, · 60 unmodified lines | |||
| 219 | { | 221 | { |
| 220 | struct nouveau_cli *cli = nouveau_cli(file_priv); | 222 | struct nouveau_cli *cli = nouveau_cli(file_priv); |
| 221 | struct nouveau_bo *nvbo = nouveau_gem_object(gem); | 223 | struct nouveau_bo *nvbo = nouveau_gem_object(gem); |
| 224 | struct nouveau_vmm *vmm = cli->svm.cli ? &cli->svm : &cli->vmm; | ||
| 222 | struct nouveau_vma *vma; | 225 | struct nouveau_vma *vma; |
| 223 | 226 | ||
| 224 | if (is_power_of_2(nvbo->valid_domains)) | 227 | if (is_power_of_2(nvbo->valid_domains)) |
| @@ -228,8 +231,8 @@ nouveau_gem_info(struct drm_file *file_priv, struct drm_gem_object *gem, · 3 unmodified lines | |||
| 228 | else | 231 | else |
| 229 | rep->domain = NOUVEAU_GEM_DOMAIN_VRAM; | 232 | rep->domain = NOUVEAU_GEM_DOMAIN_VRAM; |
| 230 | rep->offset = nvbo->bo.offset; | 233 | rep->offset = nvbo->bo.offset; |
| 231 | if (cli->vmm.vmm.object.oclass >= NVIF_CLASS_VMM_NV50) { | 234 | if (vmm->vmm.object.oclass >= NVIF_CLASS_VMM_NV50) { |
| 232 | vma = nouveau_vma_find(nvbo, &cli->vmm); | 235 | vma = nouveau_vma_find(nvbo, vmm); |
| 233 | if (!vma) | 236 | if (!vma) |
| 234 | return -EINVAL; | 237 | return -EINVAL; |
| 235 | 238 | ||
| @@ -321,7 +324,8 @@ struct validate_op { · 85 unmodified lines | |||
| 321 | }; | 324 | }; |
| 322 | 325 | ||
| 323 | static void | 326 | static void |
| 324 | validate_fini_no_ticket(struct validate_op *op, struct nouveau_fence *fence, | 327 | validate_fini_no_ticket(struct validate_op *op, struct nouveau_channel *chan, |
| 328 | struct nouveau_fence *fence, | ||
| 325 | struct drm_nouveau_gem_pushbuf_bo *pbbo) | 329 | struct drm_nouveau_gem_pushbuf_bo *pbbo) |
| 326 | { | 330 | { |
| 327 | struct nouveau_bo *nvbo; | 331 | struct nouveau_bo *nvbo; |
| @@ -332,13 +336,11 @@ validate_fini_no_ticket(struct validate_op *op, struct nouveau_fence *fence, · 4 unmodified lines | |||
| 332 | b = &pbbo[nvbo->pbbo_index]; | 336 | b = &pbbo[nvbo->pbbo_index]; |
| 333 | 337 | ||
| 334 | if (likely(fence)) { | 338 | if (likely(fence)) { |
| 335 | struct nouveau_drm *drm = nouveau_bdev(nvbo->bo.bdev); | ||
| 336 | struct nouveau_vma *vma; | ||
| 337 | |||
| 338 | nouveau_bo_fence(nvbo, fence, !!b->write_domains); | 339 | nouveau_bo_fence(nvbo, fence, !!b->write_domains); |
| 339 | 340 | ||
| 340 | if (drm->client.vmm.vmm.object.oclass >= NVIF_CLASS_VMM_NV50) { | 341 | if (chan->vmm->vmm.object.oclass >= NVIF_CLASS_VMM_NV50) { |
| 341 | vma = (void *)(unsigned long)b->user_priv; | 342 | struct nouveau_vma *vma = |
| 343 | (void *)(unsigned long)b->user_priv; | ||
| 342 | nouveau_fence_unref(&vma->fence); | 344 | nouveau_fence_unref(&vma->fence); |
| 343 | dma_fence_get(&fence->base); | 345 | dma_fence_get(&fence->base); |
| 344 | vma->fence = fence; | 346 | vma->fence = fence; |
| @@ -358,10 +360,11 @@ validate_fini_no_ticket(struct validate_op *op, struct nouveau_fence *fence, · 13 unmodified lines | |||
| 358 | } | 360 | } |
| 359 | 361 | ||
| 360 | static void | 362 | static void |
| 361 | validate_fini(struct validate_op *op, struct nouveau_fence *fence, | 363 | validate_fini(struct validate_op *op, struct nouveau_channel *chan, |
| 364 | struct nouveau_fence *fence, | ||
| 362 | struct drm_nouveau_gem_pushbuf_bo *pbbo) | 365 | struct drm_nouveau_gem_pushbuf_bo *pbbo) |
| 363 | { | 366 | { |
| 364 | validate_fini_no_ticket(op, fence, pbbo); | 367 | validate_fini_no_ticket(op, chan, fence, pbbo); |
| 365 | ww_acquire_fini(&op->ticket); | 368 | ww_acquire_fini(&op->ticket); |
| 366 | } | 369 | } |
| 367 | 370 | ||
| @@ -416,7 +419,7 @@ retry: · 48 unmodified lines | |||
| 416 | list_splice_tail_init(&vram_list, &op->list); | 419 | list_splice_tail_init(&vram_list, &op->list); |
| 417 | list_splice_tail_init(&gart_list, &op->list); | 420 | list_splice_tail_init(&gart_list, &op->list); |
| 418 | list_splice_tail_init(&both_list, &op->list); | 421 | list_splice_tail_init(&both_list, &op->list); |
| 419 | validate_fini_no_ticket(op, NULL, NULL); | 422 | validate_fini_no_ticket(op, chan, NULL, NULL); |
| 420 | if (unlikely(ret == -EDEADLK)) { | 423 | if (unlikely(ret == -EDEADLK)) { |
| 421 | ret = ttm_bo_reserve_slowpath(&nvbo->bo, true, | 424 | ret = ttm_bo_reserve_slowpath(&nvbo->bo, true, |
| 422 | &op->ticket); | 425 | &op->ticket); |
| @@ -430,8 +433,8 @@ retry: · 7 unmodified lines | |||
| 430 | } | 433 | } |
| 431 | } | 434 | } |
| 432 | 435 | ||
| 433 | if (cli->vmm.vmm.object.oclass >= NVIF_CLASS_VMM_NV50) { | 436 | if (chan->vmm->vmm.object.oclass >= NVIF_CLASS_VMM_NV50) { |
| 434 | struct nouveau_vmm *vmm = &cli->vmm; | 437 | struct nouveau_vmm *vmm = chan->vmm; |
| 435 | struct nouveau_vma *vma = nouveau_vma_find(nvbo, vmm); | 438 | struct nouveau_vma *vma = nouveau_vma_find(nvbo, vmm); |
| 436 | if (!vma) { | 439 | if (!vma) { |
| 437 | NV_PRINTK(err, cli, "vma not found!\n"); | 440 | NV_PRINTK(err, cli, "vma not found!\n"); |
| @@ -471,7 +474,7 @@ retry: · 33 unmodified lines | |||
| 471 | list_splice_tail(&gart_list, &op->list); | 474 | list_splice_tail(&gart_list, &op->list); |
| 472 | list_splice_tail(&both_list, &op->list); | 475 | list_splice_tail(&both_list, &op->list); |
| 473 | if (ret) | 476 | if (ret) |
| 474 | validate_fini(op, NULL, NULL); | 477 | validate_fini(op, chan, NULL, NULL); |
| 475 | return ret; | 478 | return ret; |
| 476 | 479 | ||
| 477 | } | 480 | } |
| @@ -563,7 +566,7 @@ nouveau_gem_pushbuf_validate(struct nouveau_channel *chan, · 85 unmodified lines | |||
| 563 | if (unlikely(ret < 0)) { | 566 | if (unlikely(ret < 0)) { |
| 564 | if (ret != -ERESTARTSYS) | 567 | if (ret != -ERESTARTSYS) |
| 565 | NV_PRINTK(err, cli, "validating bo list\n"); | 568 | NV_PRINTK(err, cli, "validating bo list\n"); |
| 566 | validate_fini(op, NULL, NULL); | 569 | validate_fini(op, chan, NULL, NULL); |
| 567 | return ret; | 570 | return ret; |
| 568 | } | 571 | } |
| 569 | *apply_relocs = ret; | 572 | *apply_relocs = ret; |
| @@ -842,7 +845,7 @@ nouveau_gem_ioctl_pushbuf(struct drm_device *dev, void *data, · 272 unmodified lines | |||
| 842 | } | 845 | } |
| 843 | 846 | ||
| 844 | out: | 847 | out: |
| 845 | validate_fini(&op, fence, bo); | 848 | validate_fini(&op, chan, fence, bo); |
| 846 | nouveau_fence_unref(&fence); | 849 | nouveau_fence_unref(&fence); |
| 847 | 850 | ||
| 848 | out_prevalid: | 851 | out_prevalid: |
| @@ -0,0 +1,835 @@ | |||
| 1 | /* | ||
| 2 | * Copyright 2018 Red Hat Inc. | ||
| 3 | * | ||
| 4 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 5 | * copy of this software and associated documentation files (the "Software"), | ||
| 6 | * to deal in the Software without restriction, including without limitation | ||
| 7 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 8 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 9 | * Software is furnished to do so, subject to the following conditions: | ||
| 10 | * | ||
| 11 | * The above copyright notice and this permission notice shall be included in | ||
| 12 | * all copies or substantial portions of the Software. | ||
| 13 | * | ||
| 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 17 | * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | ||
| 18 | * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | ||
| 19 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | ||
| 20 | * OTHER DEALINGS IN THE SOFTWARE. | ||
| 21 | */ | ||
| 22 | #include "nouveau_svm.h" | ||
| 23 | #include "nouveau_drv.h" | ||
| 24 | #include "nouveau_chan.h" | ||
| 25 | #include "nouveau_dmem.h" | ||
| 26 | |||
| 27 | #include <nvif/notify.h> | ||
| 28 | #include <nvif/object.h> | ||
| 29 | #include <nvif/vmm.h> | ||
| 30 | |||
| 31 | #include <nvif/class.h> | ||
| 32 | #include <nvif/clb069.h> | ||
| 33 | #include <nvif/ifc00d.h> | ||
| 34 | |||
| 35 | #include <linux/sched/mm.h> | ||
| 36 | #include <linux/sort.h> | ||
| 37 | #include <linux/hmm.h> | ||
| 38 | |||
| 39 | struct nouveau_svm { | ||
| 40 | struct nouveau_drm *drm; | ||
| 41 | struct mutex mutex; | ||
| 42 | struct list_head inst; | ||
| 43 | |||
| 44 | struct nouveau_svm_fault_buffer { | ||
| 45 | int id; | ||
| 46 | struct nvif_object object; | ||
| 47 | u32 entries; | ||
| 48 | u32 getaddr; | ||
| 49 | u32 putaddr; | ||
| 50 | u32 get; | ||
| 51 | u32 put; | ||
| 52 | struct nvif_notify notify; | ||
| 53 | |||
| 54 | struct nouveau_svm_fault { | ||
| 55 | u64 inst; | ||
| 56 | u64 addr; | ||
| 57 | u64 time; | ||
| 58 | u32 engine; | ||
| 59 | u8 gpc; | ||
| 60 | u8 hub; | ||
| 61 | u8 access; | ||
| 62 | u8 client; | ||
| 63 | u8 fault; | ||
| 64 | struct nouveau_svmm *svmm; | ||
| 65 | } **fault; | ||
| 66 | int fault_nr; | ||
| 67 | } buffer[1]; | ||
| 68 | }; | ||
| 69 | |||
| 70 | #define SVM_DBG(s,f,a...) NV_DEBUG((s)->drm, "svm: "f"\n", ##a) | ||
| 71 | #define SVM_ERR(s,f,a...) NV_WARN((s)->drm, "svm: "f"\n", ##a) | ||
| 72 | |||
| 73 | struct nouveau_ivmm { | ||
| 74 | struct nouveau_svmm *svmm; | ||
| 75 | u64 inst; | ||
| 76 | struct list_head head; | ||
| 77 | }; | ||
| 78 | |||
| 79 | static struct nouveau_ivmm * | ||
| 80 | nouveau_ivmm_find(struct nouveau_svm *svm, u64 inst) | ||
| 81 | { | ||
| 82 | struct nouveau_ivmm *ivmm; | ||
| 83 | list_for_each_entry(ivmm, &svm->inst, head) { | ||
| 84 | if (ivmm->inst == inst) | ||
| 85 | return ivmm; | ||
| 86 | } | ||
| 87 | return NULL; | ||
| 88 | } | ||
| 89 | |||
| 90 | struct nouveau_svmm { | ||
| 91 | struct nouveau_vmm *vmm; | ||
| 92 | struct { | ||
| 93 | unsigned long start; | ||
| 94 | unsigned long limit; | ||
| 95 | } unmanaged; | ||
| 96 | |||
| 97 | struct mutex mutex; | ||
| 98 | |||
| 99 | struct mm_struct *mm; | ||
| 100 | struct hmm_mirror mirror; | ||
| 101 | }; | ||
| 102 | |||
| 103 | #define SVMM_DBG(s,f,a...) \ | ||
| 104 | NV_DEBUG((s)->vmm->cli->drm, "svm-%p: "f"\n", (s), ##a) | ||
| 105 | #define SVMM_ERR(s,f,a...) \ | ||
| 106 | NV_WARN((s)->vmm->cli->drm, "svm-%p: "f"\n", (s), ##a) | ||
| 107 | |||
| 108 | int | ||
| 109 | nouveau_svmm_bind(struct drm_device *dev, void *data, | ||
| 110 | struct drm_file *file_priv) | ||
| 111 | { | ||
| 112 | struct nouveau_cli *cli = nouveau_cli(file_priv); | ||
| 113 | struct drm_nouveau_svm_bind *args = data; | ||
| 114 | unsigned target, cmd, priority; | ||
| 115 | unsigned long addr, end, size; | ||
| 116 | struct mm_struct *mm; | ||
| 117 | |||
| 118 | args->va_start &= PAGE_MASK; | ||
| 119 | args->va_end &= PAGE_MASK; | ||
| 120 | |||
| 121 | /* Sanity check arguments */ | ||
| 122 | if (args->reserved0 || args->reserved1) | ||
| 123 | return -EINVAL; | ||
| 124 | if (args->header & (~NOUVEAU_SVM_BIND_VALID_MASK)) | ||
| 125 | return -EINVAL; | ||
| 126 | if (args->va_start >= args->va_end) | ||
| 127 | return -EINVAL; | ||
| 128 | if (!args->npages) | ||
| 129 | return -EINVAL; | ||
| 130 | |||
| 131 | cmd = args->header >> NOUVEAU_SVM_BIND_COMMAND_SHIFT; | ||
| 132 | cmd &= NOUVEAU_SVM_BIND_COMMAND_MASK; | ||
| 133 | switch (cmd) { | ||
| 134 | case NOUVEAU_SVM_BIND_COMMAND__MIGRATE: | ||
| 135 | break; | ||
| 136 | default: | ||
| 137 | return -EINVAL; | ||
| 138 | } | ||
| 139 | |||
| 140 | priority = args->header >> NOUVEAU_SVM_BIND_PRIORITY_SHIFT; | ||
| 141 | priority &= NOUVEAU_SVM_BIND_PRIORITY_MASK; | ||
| 142 | |||
| 143 | /* FIXME support CPU target ie all target value < GPU_VRAM */ | ||
| 144 | target = args->header >> NOUVEAU_SVM_BIND_TARGET_SHIFT; | ||
| 145 | target &= NOUVEAU_SVM_BIND_TARGET_MASK; | ||
| 146 | switch (target) { | ||
| 147 | case NOUVEAU_SVM_BIND_TARGET__GPU_VRAM: | ||
| 148 | break; | ||
| 149 | default: | ||
| 150 | return -EINVAL; | ||
| 151 | } | ||
| 152 | |||
| 153 | /* | ||
| 154 | * FIXME: For now refuse non 0 stride, we need to change the migrate | ||
| 155 | * kernel function to handle stride to avoid to create a mess within | ||
| 156 | * each device driver. | ||
| 157 | */ | ||
| 158 | if (args->stride) | ||
| 159 | return -EINVAL; | ||
| 160 | |||
| 161 | size = ((unsigned long)args->npages) << PAGE_SHIFT; | ||
| 162 | if ((args->va_start + size) <= args->va_start) | ||
| 163 | return -EINVAL; | ||
| 164 | if ((args->va_start + size) > args->va_end) | ||
| 165 | return -EINVAL; | ||
| 166 | |||
| 167 | /* | ||
| 168 | * Ok we are ask to do something sane, for now we only support migrate | ||
| 169 | * commands but we will add things like memory policy (what to do on | ||
| 170 | * page fault) and maybe some other commands. | ||
| 171 | */ | ||
| 172 | |||
| 173 | mm = get_task_mm(current); | ||
| 174 | down_read(&mm->mmap_sem); | ||
| 175 | |||
| 176 | for (addr = args->va_start, end = args->va_start + size; addr < end;) { | ||
| 177 | struct vm_area_struct *vma; | ||
| 178 | unsigned long next; | ||
| 179 | |||
| 180 | vma = find_vma_intersection(mm, addr, end); | ||
| 181 | if (!vma) | ||
| 182 | break; | ||
| 183 | |||
| 184 | next = min(vma->vm_end, end); | ||
| 185 | /* This is a best effort so we ignore errors */ | ||
| 186 | nouveau_dmem_migrate_vma(cli->drm, vma, addr, next); | ||
| 187 | addr = next; | ||
| 188 | } | ||
| 189 | |||
| 190 | /* | ||
| 191 | * FIXME Return the number of page we have migrated, again we need to | ||
| 192 | * update the migrate API to return that information so that we can | ||
| 193 | * report it to user space. | ||
| 194 | */ | ||
| 195 | args->result = 0; | ||
| 196 | |||
| 197 | up_read(&mm->mmap_sem); | ||
| 198 | mmput(mm); | ||
| 199 | |||
| 200 | return 0; | ||
| 201 | } | ||
| 202 | |||
| 203 | /* Unlink channel instance from SVMM. */ | ||
| 204 | void | ||
| 205 | nouveau_svmm_part(struct nouveau_svmm *svmm, u64 inst) | ||
| 206 | { | ||
| 207 | struct nouveau_ivmm *ivmm; | ||
| 208 | if (svmm) { | ||
| 209 | mutex_lock(&svmm->vmm->cli->drm->svm->mutex); | ||
| 210 | ivmm = nouveau_ivmm_find(svmm->vmm->cli->drm->svm, inst); | ||
| 211 | if (ivmm) { | ||
| 212 | list_del(&ivmm->head); | ||
| 213 | kfree(ivmm); | ||
| 214 | } | ||
| 215 | mutex_unlock(&svmm->vmm->cli->drm->svm->mutex); | ||
| 216 | } | ||
| 217 | } | ||
| 218 | |||
| 219 | /* Link channel instance to SVMM. */ | ||
| 220 | int | ||
| 221 | nouveau_svmm_join(struct nouveau_svmm *svmm, u64 inst) | ||
| 222 | { | ||
| 223 | struct nouveau_ivmm *ivmm; | ||
| 224 | if (svmm) { | ||
| 225 | if (!(ivmm = kmalloc(sizeof(*ivmm), GFP_KERNEL))) | ||
| 226 | return -ENOMEM; | ||
| 227 | ivmm->svmm = svmm; | ||
| 228 | ivmm->inst = inst; | ||
| 229 | |||
| 230 | mutex_lock(&svmm->vmm->cli->drm->svm->mutex); | ||
| 231 | list_add(&ivmm->head, &svmm->vmm->cli->drm->svm->inst); | ||
| 232 | mutex_unlock(&svmm->vmm->cli->drm->svm->mutex); | ||
| 233 | } | ||
| 234 | return 0; | ||
| 235 | } | ||
| 236 | |||
| 237 | /* Invalidate SVMM address-range on GPU. */ | ||
| 238 | static void | ||
| 239 | nouveau_svmm_invalidate(struct nouveau_svmm *svmm, u64 start, u64 limit) | ||
| 240 | { | ||
| 241 | if (limit > start) { | ||
| 242 | bool super = svmm->vmm->vmm.object.client->super; | ||
| 243 | svmm->vmm->vmm.object.client->super = true; | ||
| 244 | nvif_object_mthd(&svmm->vmm->vmm.object, NVIF_VMM_V0_PFNCLR, | ||
| 245 | &(struct nvif_vmm_pfnclr_v0) { | ||
| 246 | .addr = start, | ||
| 247 | .size = limit - start, | ||
| 248 | }, sizeof(struct nvif_vmm_pfnclr_v0)); | ||
| 249 | svmm->vmm->vmm.object.client->super = super; | ||
| 250 | } | ||
| 251 | } | ||
| 252 | |||
| 253 | static int | ||
| 254 | nouveau_svmm_sync_cpu_device_pagetables(struct hmm_mirror *mirror, | ||
| 255 | const struct hmm_update *update) | ||
| 256 | { | ||
| 257 | struct nouveau_svmm *svmm = container_of(mirror, typeof(*svmm), mirror); | ||
| 258 | unsigned long start = update->start; | ||
| 259 | unsigned long limit = update->end; | ||
| 260 | |||
| 261 | if (!update->blockable) | ||
| 262 | return -EAGAIN; | ||
| 263 | |||
| 264 | SVMM_DBG(svmm, "invalidate %016lx-%016lx", start, limit); | ||
| 265 | |||
| 266 | mutex_lock(&svmm->mutex); | ||
| 267 | if (limit > svmm->unmanaged.start && start < svmm->unmanaged.limit) { | ||
| 268 | if (start < svmm->unmanaged.start) { | ||
| 269 | nouveau_svmm_invalidate(svmm, start, | ||
| 270 | svmm->unmanaged.limit); | ||
| 271 | } | ||
| 272 | start = svmm->unmanaged.limit; | ||
| 273 | } | ||
| 274 | |||
| 275 | nouveau_svmm_invalidate(svmm, start, limit); | ||
| 276 | mutex_unlock(&svmm->mutex); | ||
| 277 | return 0; | ||
| 278 | } | ||
| 279 | |||
| 280 | static void | ||
| 281 | nouveau_svmm_release(struct hmm_mirror *mirror) | ||
| 282 | { | ||
| 283 | } | ||
| 284 | |||
| 285 | static const struct hmm_mirror_ops | ||
| 286 | nouveau_svmm = { | ||
| 287 | .sync_cpu_device_pagetables = nouveau_svmm_sync_cpu_device_pagetables, | ||
| 288 | .release = nouveau_svmm_release, | ||
| 289 | }; | ||
| 290 | |||
| 291 | void | ||
| 292 | nouveau_svmm_fini(struct nouveau_svmm **psvmm) | ||
| 293 | { | ||
| 294 | struct nouveau_svmm *svmm = *psvmm; | ||
| 295 | if (svmm) { | ||
| 296 | hmm_mirror_unregister(&svmm->mirror); | ||
| 297 | kfree(*psvmm); | ||
| 298 | *psvmm = NULL; | ||
| 299 | } | ||
| 300 | } | ||
| 301 | |||
| 302 | int | ||
| 303 | nouveau_svmm_init(struct drm_device *dev, void *data, | ||
| 304 | struct drm_file *file_priv) | ||
| 305 | { | ||
| 306 | struct nouveau_cli *cli = nouveau_cli(file_priv); | ||
| 307 | struct nouveau_svmm *svmm; | ||
| 308 | struct drm_nouveau_svm_init *args = data; | ||
| 309 | int ret; | ||
| 310 | |||
| 311 | /* Allocate tracking for SVM-enabled VMM. */ | ||
| 312 | if (!(svmm = kzalloc(sizeof(*svmm), GFP_KERNEL))) | ||
| 313 | return -ENOMEM; | ||
| 314 | svmm->vmm = &cli->svm; | ||
| 315 | svmm->unmanaged.start = args->unmanaged_addr; | ||
| 316 | svmm->unmanaged.limit = args->unmanaged_addr + args->unmanaged_size; | ||
| 317 | mutex_init(&svmm->mutex); | ||
| 318 | |||
| 319 | /* Check that SVM isn't already enabled for the client. */ | ||
| 320 | mutex_lock(&cli->mutex); | ||
| 321 | if (cli->svm.cli) { | ||
| 322 | ret = -EBUSY; | ||
| 323 | goto done; | ||
| 324 | } | ||
| 325 | |||
| 326 | /* Allocate a new GPU VMM that can support SVM (managed by the | ||
| 327 | * client, with replayable faults enabled). | ||
| 328 | * | ||
| 329 | * All future channel/memory allocations will make use of this | ||
| 330 | * VMM instead of the standard one. | ||
| 331 | */ | ||
| 332 | ret = nvif_vmm_init(&cli->mmu, cli->vmm.vmm.object.oclass, true, | ||
| 333 | args->unmanaged_addr, args->unmanaged_size, | ||
| 334 | &(struct gp100_vmm_v0) { | ||
| 335 | .fault_replay = true, | ||
| 336 | }, sizeof(struct gp100_vmm_v0), &cli->svm.vmm); | ||
| 337 | if (ret) | ||
| 338 | goto done; | ||
| 339 | |||
| 340 | /* Enable HMM mirroring of CPU address-space to VMM. */ | ||
| 341 | svmm->mm = get_task_mm(current); | ||
| 342 | down_write(&svmm->mm->mmap_sem); | ||
| 343 | svmm->mirror.ops = &nouveau_svmm; | ||
| 344 | ret = hmm_mirror_register(&svmm->mirror, svmm->mm); | ||
| 345 | if (ret == 0) { | ||
| 346 | cli->svm.svmm = svmm; | ||
| 347 | cli->svm.cli = cli; | ||
| 348 | } | ||
| 349 | up_write(&svmm->mm->mmap_sem); | ||
| 350 | mmput(svmm->mm); | ||
| 351 | |||
| 352 | done: | ||
| 353 | if (ret) | ||
| 354 | nouveau_svmm_fini(&svmm); | ||
| 355 | mutex_unlock(&cli->mutex); | ||
| 356 | return ret; | ||
| 357 | } | ||
| 358 | |||
| 359 | static const u64 | ||
| 360 | nouveau_svm_pfn_flags[HMM_PFN_FLAG_MAX] = { | ||
| 361 | [HMM_PFN_VALID ] = NVIF_VMM_PFNMAP_V0_V, | ||
| 362 | [HMM_PFN_WRITE ] = NVIF_VMM_PFNMAP_V0_W, | ||
| 363 | [HMM_PFN_DEVICE_PRIVATE] = NVIF_VMM_PFNMAP_V0_VRAM, | ||
| 364 | }; | ||
| 365 | |||
| 366 | static const u64 | ||
| 367 | nouveau_svm_pfn_values[HMM_PFN_VALUE_MAX] = { | ||
| 368 | [HMM_PFN_ERROR ] = ~NVIF_VMM_PFNMAP_V0_V, | ||
| 369 | [HMM_PFN_NONE ] = NVIF_VMM_PFNMAP_V0_NONE, | ||
| 370 | [HMM_PFN_SPECIAL] = ~NVIF_VMM_PFNMAP_V0_V, | ||
| 371 | }; | ||
| 372 | |||
| 373 | /* Issue fault replay for GPU to retry accesses that faulted previously. */ | ||
| 374 | static void | ||
| 375 | nouveau_svm_fault_replay(struct nouveau_svm *svm) | ||
| 376 | { | ||
| 377 | SVM_DBG(svm, "replay"); | ||
| 378 | WARN_ON(nvif_object_mthd(&svm->drm->client.vmm.vmm.object, | ||
| 379 | GP100_VMM_VN_FAULT_REPLAY, | ||
| 380 | &(struct gp100_vmm_fault_replay_vn) {}, | ||
| 381 | sizeof(struct gp100_vmm_fault_replay_vn))); | ||
| 382 | } | ||
| 383 | |||
| 384 | /* Cancel a replayable fault that could not be handled. | ||
| 385 | * | ||
| 386 | * Cancelling the fault will trigger recovery to reset the engine | ||
| 387 | * and kill the offending channel (ie. GPU SIGSEGV). | ||
| 388 | */ | ||
| 389 | static void | ||
| 390 | nouveau_svm_fault_cancel(struct nouveau_svm *svm, | ||
| 391 | u64 inst, u8 hub, u8 gpc, u8 client) | ||
| 392 | { | ||
| 393 | SVM_DBG(svm, "cancel %016llx %d %02x %02x", inst, hub, gpc, client); | ||
| 394 | WARN_ON(nvif_object_mthd(&svm->drm->client.vmm.vmm.object, | ||
| 395 | GP100_VMM_VN_FAULT_CANCEL, | ||
| 396 | &(struct gp100_vmm_fault_cancel_v0) { | ||
| 397 | .hub = hub, | ||
| 398 | .gpc = gpc, | ||
| 399 | .client = client, | ||
| 400 | .inst = inst, | ||
| 401 | }, sizeof(struct gp100_vmm_fault_cancel_v0))); | ||
| 402 | } | ||
| 403 | |||
| 404 | static void | ||
| 405 | nouveau_svm_fault_cancel_fault(struct nouveau_svm *svm, | ||
| 406 | struct nouveau_svm_fault *fault) | ||
| 407 | { | ||
| 408 | nouveau_svm_fault_cancel(svm, fault->inst, | ||
| 409 | fault->hub, | ||
| 410 | fault->gpc, | ||
| 411 | fault->client); | ||
| 412 | } | ||
| 413 | |||
| 414 | static int | ||
| 415 | nouveau_svm_fault_cmp(const void *a, const void *b) | ||
| 416 | { | ||
| 417 | const struct nouveau_svm_fault *fa = *(struct nouveau_svm_fault **)a; | ||
| 418 | const struct nouveau_svm_fault *fb = *(struct nouveau_svm_fault **)b; | ||
| 419 | int ret; | ||
| 420 | if ((ret = (s64)fa->inst - fb->inst)) | ||
| 421 | return ret; | ||
| 422 | if ((ret = (s64)fa->addr - fb->addr)) | ||
| 423 | return ret; | ||
| 424 | /*XXX: atomic? */ | ||
| 425 | return (fa->access == 0 || fa->access == 3) - | ||
| 426 | (fb->access == 0 || fb->access == 3); | ||
| 427 | } | ||
| 428 | |||
| 429 | static void | ||
| 430 | nouveau_svm_fault_cache(struct nouveau_svm *svm, | ||
| 431 | struct nouveau_svm_fault_buffer *buffer, u32 offset) | ||
| 432 | { | ||
| 433 | struct nvif_object *memory = &buffer->object; | ||
| 434 | const u32 instlo = nvif_rd32(memory, offset + 0x00); | ||
| 435 | const u32 insthi = nvif_rd32(memory, offset + 0x04); | ||
| 436 | const u32 addrlo = nvif_rd32(memory, offset + 0x08); | ||
| 437 | const u32 addrhi = nvif_rd32(memory, offset + 0x0c); | ||
| 438 | const u32 timelo = nvif_rd32(memory, offset + 0x10); | ||
| 439 | const u32 timehi = nvif_rd32(memory, offset + 0x14); | ||
| 440 | const u32 engine = nvif_rd32(memory, offset + 0x18); | ||
| 441 | const u32 info = nvif_rd32(memory, offset + 0x1c); | ||
| 442 | const u64 inst = (u64)insthi << 32 | instlo; | ||
| 443 | const u8 gpc = (info & 0x1f000000) >> 24; | ||
| 444 | const u8 hub = (info & 0x00100000) >> 20; | ||
| 445 | const u8 client = (info & 0x00007f00) >> 8; | ||
| 446 | struct nouveau_svm_fault *fault; | ||
| 447 | |||
| 448 | //XXX: i think we're supposed to spin waiting */ | ||
| 449 | if (WARN_ON(!(info & 0x80000000))) | ||
| 450 | return; | ||
| 451 | |||
| 452 | nvif_mask(memory, offset + 0x1c, 0x80000000, 0x00000000); | ||
| 453 | |||
| 454 | if (!buffer->fault[buffer->fault_nr]) { | ||
| 455 | fault = kmalloc(sizeof(*fault), GFP_KERNEL); | ||
| 456 | if (WARN_ON(!fault)) { | ||
| 457 | nouveau_svm_fault_cancel(svm, inst, hub, gpc, client); | ||
| 458 | return; | ||
| 459 | } | ||
| 460 | buffer->fault[buffer->fault_nr] = fault; | ||
| 461 | } | ||
| 462 | |||
| 463 | fault = buffer->fault[buffer->fault_nr++]; | ||
| 464 | fault->inst = inst; | ||
| 465 | fault->addr = (u64)addrhi << 32 | addrlo; | ||
| 466 | fault->time = (u64)timehi << 32 | timelo; | ||
| 467 | fault->engine = engine; | ||
| 468 | fault->gpc = gpc; | ||
| 469 | fault->hub = hub; | ||
| 470 | fault->access = (info & 0x000f0000) >> 16; | ||
| 471 | fault->client = client; | ||
| 472 | fault->fault = (info & 0x0000001f); | ||
| 473 | |||
| 474 | SVM_DBG(svm, "fault %016llx %016llx %02x", | ||
| 475 | fault->inst, fault->addr, fault->access); | ||
| 476 | } | ||
| 477 | |||
| 478 | static int | ||
| 479 | nouveau_svm_fault(struct nvif_notify *notify) | ||
| 480 | { | ||
| 481 | struct nouveau_svm_fault_buffer *buffer = | ||
| 482 | container_of(notify, typeof(*buffer), notify); | ||
| 483 | struct nouveau_svm *svm = | ||
| 484 | container_of(buffer, typeof(*svm), buffer[buffer->id]); | ||
| 485 | struct nvif_object *device = &svm->drm->client.device.object; | ||
| 486 | struct nouveau_svmm *svmm; | ||
| 487 | struct { | ||
| 488 | struct { | ||
| 489 | struct nvif_ioctl_v0 i; | ||
| 490 | struct nvif_ioctl_mthd_v0 m; | ||
| 491 | struct nvif_vmm_pfnmap_v0 p; | ||
| 492 | } i; | ||
| 493 | u64 phys[16]; | ||
| 494 | } args; | ||
| 495 | struct hmm_range range; | ||
| 496 | struct vm_area_struct *vma; | ||
| 497 | u64 inst, start, limit; | ||
| 498 | int fi, fn, pi, fill; | ||
| 499 | int replay = 0, ret; | ||
| 500 | |||
| 501 | /* Parse available fault buffer entries into a cache, and update | ||
| 502 | * the GET pointer so HW can reuse the entries. | ||
| 503 | */ | ||
| 504 | SVM_DBG(svm, "fault handler"); | ||
| 505 | if (buffer->get == buffer->put) { | ||
| 506 | buffer->put = nvif_rd32(device, buffer->putaddr); | ||
| 507 | buffer->get = nvif_rd32(device, buffer->getaddr); | ||
| 508 | if (buffer->get == buffer->put) | ||
| 509 | return NVIF_NOTIFY_KEEP; | ||
| 510 | } | ||
| 511 | buffer->fault_nr = 0; | ||
| 512 | |||
| 513 | SVM_DBG(svm, "get %08x put %08x", buffer->get, buffer->put); | ||
| 514 | while (buffer->get != buffer->put) { | ||
| 515 | nouveau_svm_fault_cache(svm, buffer, buffer->get * 0x20); | ||
| 516 | if (++buffer->get == buffer->entries) | ||
| 517 | buffer->get = 0; | ||
| 518 | } | ||
| 519 | nvif_wr32(device, buffer->getaddr, buffer->get); | ||
| 520 | SVM_DBG(svm, "%d fault(s) pending", buffer->fault_nr); | ||
| 521 | |||
| 522 | /* Sort parsed faults by instance pointer to prevent unnecessary | ||
| 523 | * instance to SVMM translations, followed by address and access | ||
| 524 | * type to reduce the amount of work when handling the faults. | ||
| 525 | */ | ||
| 526 | sort(buffer->fault, buffer->fault_nr, sizeof(*buffer->fault), | ||
| 527 | nouveau_svm_fault_cmp, NULL); | ||
| 528 | |||
| 529 | /* Lookup SVMM structure for each unique instance pointer. */ | ||
| 530 | mutex_lock(&svm->mutex); | ||
| 531 | for (fi = 0, svmm = NULL; fi < buffer->fault_nr; fi++) { | ||
| 532 | if (!svmm || buffer->fault[fi]->inst != inst) { | ||
| 533 | struct nouveau_ivmm *ivmm = | ||
| 534 | nouveau_ivmm_find(svm, buffer->fault[fi]->inst); | ||
| 535 | svmm = ivmm ? ivmm->svmm : NULL; | ||
| 536 | inst = buffer->fault[fi]->inst; | ||
| 537 | SVM_DBG(svm, "inst %016llx -> svm-%p", inst, svmm); | ||
| 538 | } | ||
| 539 | buffer->fault[fi]->svmm = svmm; | ||
| 540 | } | ||
| 541 | mutex_unlock(&svm->mutex); | ||
| 542 | |||
| 543 | /* Process list of faults. */ | ||
| 544 | args.i.i.version = 0; | ||
| 545 | args.i.i.type = NVIF_IOCTL_V0_MTHD; | ||
| 546 | args.i.m.version = 0; | ||
| 547 | args.i.m.method = NVIF_VMM_V0_PFNMAP; | ||
| 548 | args.i.p.version = 0; | ||
| 549 | |||
| 550 | for (fi = 0; fn = fi + 1, fi < buffer->fault_nr; fi = fn) { | ||
| 551 | /* Cancel any faults from non-SVM channels. */ | ||
| 552 | if (!(svmm = buffer->fault[fi]->svmm)) { | ||
| 553 | nouveau_svm_fault_cancel_fault(svm, buffer->fault[fi]); | ||
| 554 | continue; | ||
| 555 | } | ||
| 556 | SVMM_DBG(svmm, "addr %016llx", buffer->fault[fi]->addr); | ||
| 557 | |||
| 558 | /* We try and group handling of faults within a small | ||
| 559 | * window into a single update. | ||
| 560 | */ | ||
| 561 | start = buffer->fault[fi]->addr; | ||
| 562 | limit = start + (ARRAY_SIZE(args.phys) << PAGE_SHIFT); | ||
| 563 | if (start < svmm->unmanaged.limit) | ||
| 564 | limit = min_t(u64, limit, svmm->unmanaged.start); | ||
| 565 | else | ||
| 566 | if (limit > svmm->unmanaged.start) | ||
| 567 | start = max_t(u64, start, svmm->unmanaged.limit); | ||
| 568 | SVMM_DBG(svmm, "wndw %016llx-%016llx", start, limit); | ||
| 569 | |||
| 570 | /* Intersect fault window with the CPU VMA, cancelling | ||
| 571 | * the fault if the address is invalid. | ||
| 572 | */ | ||
| 573 | down_read(&svmm->mm->mmap_sem); | ||
| 574 | vma = find_vma_intersection(svmm->mm, start, limit); | ||
| 575 | if (!vma) { | ||
| 576 | SVMM_ERR(svmm, "wndw %016llx-%016llx", start, limit); | ||
| 577 | up_read(&svmm->mm->mmap_sem); | ||
| 578 | nouveau_svm_fault_cancel_fault(svm, buffer->fault[fi]); | ||
| 579 | continue; | ||
| 580 | } | ||
| 581 | start = max_t(u64, start, vma->vm_start); | ||
| 582 | limit = min_t(u64, limit, vma->vm_end); | ||
| 583 | SVMM_DBG(svmm, "wndw %016llx-%016llx", start, limit); | ||
| 584 | |||
| 585 | if (buffer->fault[fi]->addr != start) { | ||
| 586 | SVMM_ERR(svmm, "addr %016llx", buffer->fault[fi]->addr); | ||
| 587 | up_read(&svmm->mm->mmap_sem); | ||
| 588 | nouveau_svm_fault_cancel_fault(svm, buffer->fault[fi]); | ||
| 589 | continue; | ||
| 590 | } | ||
| 591 | |||
| 592 | /* Prepare the GPU-side update of all pages within the | ||
| 593 | * fault window, determining required pages and access | ||
| 594 | * permissions based on pending faults. | ||
| 595 | */ | ||
| 596 | args.i.p.page = PAGE_SHIFT; | ||
| 597 | args.i.p.addr = start; | ||
| 598 | for (fn = fi, pi = 0;;) { | ||
| 599 | /* Determine required permissions based on GPU fault | ||
| 600 | * access flags. | ||
| 601 | *XXX: atomic? | ||
| 602 | */ | ||
| 603 | if (buffer->fault[fn]->access != 0 /* READ. */ && | ||
| 604 | buffer->fault[fn]->access != 3 /* PREFETCH. */) { | ||
| 605 | args.phys[pi++] = NVIF_VMM_PFNMAP_V0_V | | ||
| 606 | NVIF_VMM_PFNMAP_V0_W; | ||
| 607 | } else { | ||
| 608 | args.phys[pi++] = NVIF_VMM_PFNMAP_V0_V; | ||
| 609 | } | ||
| 610 | args.i.p.size = pi << PAGE_SHIFT; | ||
| 611 | |||
| 612 | /* It's okay to skip over duplicate addresses from the | ||
| 613 | * same SVMM as faults are ordered by access type such | ||
| 614 | * that only the first one needs to be handled. | ||
| 615 | * | ||
| 616 | * ie. WRITE faults appear first, thus any handling of | ||
| 617 | * pending READ faults will already be satisfied. | ||
| 618 | */ | ||
| 619 | while (++fn < buffer->fault_nr && | ||
| 620 | buffer->fault[fn]->svmm == svmm && | ||
| 621 | buffer->fault[fn ]->addr == | ||
| 622 | buffer->fault[fn - 1]->addr); | ||
| 623 | |||
| 624 | /* If the next fault is outside the window, or all GPU | ||
| 625 | * faults have been dealt with, we're done here. | ||
| 626 | */ | ||
| 627 | if (fn >= buffer->fault_nr || | ||
| 628 | buffer->fault[fn]->svmm != svmm || | ||
| 629 | buffer->fault[fn]->addr >= limit) | ||
| 630 | break; | ||
| 631 | |||
| 632 | /* Fill in the gap between this fault and the next. */ | ||
| 633 | fill = (buffer->fault[fn ]->addr - | ||
| 634 | buffer->fault[fn - 1]->addr) >> PAGE_SHIFT; | ||
| 635 | while (--fill) | ||
| 636 | args.phys[pi++] = NVIF_VMM_PFNMAP_V0_NONE; | ||
| 637 | } | ||
| 638 | |||
| 639 | SVMM_DBG(svmm, "wndw %016llx-%016llx covering %d fault(s)", | ||
| 640 | args.i.p.addr, | ||
| 641 | args.i.p.addr + args.i.p.size, fn - fi); | ||
| 642 | |||
| 643 | /* Have HMM fault pages within the fault window to the GPU. */ | ||
| 644 | range.vma = vma; | ||
| 645 | range.start = args.i.p.addr; | ||
| 646 | range.end = args.i.p.addr + args.i.p.size; | ||
| 647 | range.pfns = args.phys; | ||
| 648 | range.flags = nouveau_svm_pfn_flags; | ||
| 649 | range.values = nouveau_svm_pfn_values; | ||
| 650 | range.pfn_shift = NVIF_VMM_PFNMAP_V0_ADDR_SHIFT; | ||
| 651 | again: | ||
| 652 | ret = hmm_vma_fault(&range, true); | ||
| 653 | if (ret == 0) { | ||
| 654 | mutex_lock(&svmm->mutex); | ||
| 655 | if (!hmm_vma_range_done(&range)) { | ||
| 656 | mutex_unlock(&svmm->mutex); | ||
| 657 | goto again; | ||
| 658 | } | ||
| 659 | |||
| 660 | nouveau_dmem_convert_pfn(svm->drm, &range); | ||
| 661 | |||
| 662 | svmm->vmm->vmm.object.client->super = true; | ||
| 663 | ret = nvif_object_ioctl(&svmm->vmm->vmm.object, | ||
| 664 | &args, sizeof(args.i) + | ||
| 665 | pi * sizeof(args.phys[0]), | ||
| 666 | NULL); | ||
| 667 | svmm->vmm->vmm.object.client->super = false; | ||
| 668 | mutex_unlock(&svmm->mutex); | ||
| 669 | } | ||
| 670 | up_read(&svmm->mm->mmap_sem); | ||
| 671 | |||
| 672 | /* Cancel any faults in the window whose pages didn't manage | ||
| 673 | * to keep their valid bit, or stay writeable when required. | ||
| 674 | * | ||
| 675 | * If handling failed completely, cancel all faults. | ||
| 676 | */ | ||
| 677 | while (fi < fn) { | ||
| 678 | struct nouveau_svm_fault *fault = buffer->fault[fi++]; | ||
| 679 | pi = (fault->addr - range.start) >> PAGE_SHIFT; | ||
| 680 | if (ret || | ||
| 681 | !(range.pfns[pi] & NVIF_VMM_PFNMAP_V0_V) || | ||
| 682 | (!(range.pfns[pi] & NVIF_VMM_PFNMAP_V0_W) && | ||
| 683 | fault->access != 0 && fault->access != 3)) { | ||
| 684 | nouveau_svm_fault_cancel_fault(svm, fault); | ||
| 685 | continue; | ||
| 686 | } | ||
| 687 | replay++; | ||
| 688 | } | ||
| 689 | } | ||
| 690 | |||
| 691 | /* Issue fault replay to the GPU. */ | ||
| 692 | if (replay) | ||
| 693 | nouveau_svm_fault_replay(svm); | ||
| 694 | return NVIF_NOTIFY_KEEP; | ||
| 695 | } | ||
| 696 | |||
| 697 | static void | ||
| 698 | nouveau_svm_fault_buffer_fini(struct nouveau_svm *svm, int id) | ||
| 699 | { | ||
| 700 | struct nouveau_svm_fault_buffer *buffer = &svm->buffer[id]; | ||
| 701 | nvif_notify_put(&buffer->notify); | ||
| 702 | } | ||
| 703 | |||
| 704 | static int | ||
| 705 | nouveau_svm_fault_buffer_init(struct nouveau_svm *svm, int id) | ||
| 706 | { | ||
| 707 | struct nouveau_svm_fault_buffer *buffer = &svm->buffer[id]; | ||
| 708 | struct nvif_object *device = &svm->drm->client.device.object; | ||
| 709 | buffer->get = nvif_rd32(device, buffer->getaddr); | ||
| 710 | buffer->put = nvif_rd32(device, buffer->putaddr); | ||
| 711 | SVM_DBG(svm, "get %08x put %08x (init)", buffer->get, buffer->put); | ||
| 712 | return nvif_notify_get(&buffer->notify); | ||
| 713 | } | ||
| 714 | |||
| 715 | static void | ||
| 716 | nouveau_svm_fault_buffer_dtor(struct nouveau_svm *svm, int id) | ||
| 717 | { | ||
| 718 | struct nouveau_svm_fault_buffer *buffer = &svm->buffer[id]; | ||
| 719 | int i; | ||
| 720 | |||
| 721 | if (buffer->fault) { | ||
| 722 | for (i = 0; buffer->fault[i] && i < buffer->entries; i++) | ||
| 723 | kfree(buffer->fault[i]); | ||
| 724 | kvfree(buffer->fault); | ||
| 725 | } | ||
| 726 | |||
| 727 | nouveau_svm_fault_buffer_fini(svm, id); | ||
| 728 | |||
| 729 | nvif_notify_fini(&buffer->notify); | ||
| 730 | nvif_object_fini(&buffer->object); | ||
| 731 | } | ||
| 732 | |||
| 733 | static int | ||
| 734 | nouveau_svm_fault_buffer_ctor(struct nouveau_svm *svm, s32 oclass, int id) | ||
| 735 | { | ||
| 736 | struct nouveau_svm_fault_buffer *buffer = &svm->buffer[id]; | ||
| 737 | struct nouveau_drm *drm = svm->drm; | ||
| 738 | struct nvif_object *device = &drm->client.device.object; | ||
| 739 | struct nvif_clb069_v0 args = {}; | ||
| 740 | int ret; | ||
| 741 | |||
| 742 | buffer->id = id; | ||
| 743 | |||
| 744 | ret = nvif_object_init(device, 0, oclass, &args, sizeof(args), | ||
| 745 | &buffer->object); | ||
| 746 | if (ret < 0) { | ||
| 747 | SVM_ERR(svm, "Fault buffer allocation failed: %d", ret); | ||
| 748 | return ret; | ||
| 749 | } | ||
| 750 | |||
| 751 | nvif_object_map(&buffer->object, NULL, 0); | ||
| 752 | buffer->entries = args.entries; | ||
| 753 | buffer->getaddr = args.get; | ||
| 754 | buffer->putaddr = args.put; | ||
| 755 | |||
| 756 | ret = nvif_notify_init(&buffer->object, nouveau_svm_fault, true, | ||
| 757 | NVB069_V0_NTFY_FAULT, NULL, 0, 0, | ||
| 758 | &buffer->notify); | ||
| 759 | if (ret) | ||
| 760 | return ret; | ||
| 761 | |||
| 762 | buffer->fault = kvzalloc(sizeof(*buffer->fault) * buffer->entries, GFP_KERNEL); | ||
| 763 | if (!buffer->fault) | ||
| 764 | return -ENOMEM; | ||
| 765 | |||
| 766 | return nouveau_svm_fault_buffer_init(svm, id); | ||
| 767 | } | ||
| 768 | |||
| 769 | void | ||
| 770 | nouveau_svm_resume(struct nouveau_drm *drm) | ||
| 771 | { | ||
| 772 | struct nouveau_svm *svm = drm->svm; | ||
| 773 | if (svm) | ||
| 774 | nouveau_svm_fault_buffer_init(svm, 0); | ||
| 775 | } | ||
| 776 | |||
| 777 | void | ||
| 778 | nouveau_svm_suspend(struct nouveau_drm *drm) | ||
| 779 | { | ||
| 780 | struct nouveau_svm *svm = drm->svm; | ||
| 781 | if (svm) | ||
| 782 | nouveau_svm_fault_buffer_fini(svm, 0); | ||
| 783 | } | ||
| 784 | |||
| 785 | void | ||
| 786 | nouveau_svm_fini(struct nouveau_drm *drm) | ||
| 787 | { | ||
| 788 | struct nouveau_svm *svm = drm->svm; | ||
| 789 | if (svm) { | ||
| 790 | nouveau_svm_fault_buffer_dtor(svm, 0); | ||
| 791 | kfree(drm->svm); | ||
| 792 | drm->svm = NULL; | ||
| 793 | } | ||
| 794 | } | ||
| 795 | |||
| 796 | void | ||
| 797 | nouveau_svm_init(struct nouveau_drm *drm) | ||
| 798 | { | ||
| 799 | static const struct nvif_mclass buffers[] = { | ||
| 800 | { VOLTA_FAULT_BUFFER_A, 0 }, | ||
| 801 | { MAXWELL_FAULT_BUFFER_A, 0 }, | ||
| 802 | {} | ||
| 803 | }; | ||
| 804 | struct nouveau_svm *svm; | ||
| 805 | int ret; | ||
| 806 | |||
| 807 | /* Disable on Volta and newer until channel recovery is fixed, | ||
| 808 | * otherwise clients will have a trivial way to trash the GPU | ||
| 809 | * for everyone. | ||
| 810 | */ | ||
| 811 | if (drm->client.device.info.family > NV_DEVICE_INFO_V0_PASCAL) | ||
| 812 | return; | ||
| 813 | |||
| 814 | if (!(drm->svm = svm = kzalloc(sizeof(*drm->svm), GFP_KERNEL))) | ||
| 815 | return; | ||
| 816 | |||
| 817 | drm->svm->drm = drm; | ||
| 818 | mutex_init(&drm->svm->mutex); | ||
| 819 | INIT_LIST_HEAD(&drm->svm->inst); | ||
| 820 | |||
| 821 | ret = nvif_mclass(&drm->client.device.object, buffers); | ||
| 822 | if (ret < 0) { | ||
| 823 | SVM_DBG(svm, "No supported fault buffer class"); | ||
| 824 | nouveau_svm_fini(drm); | ||
| 825 | return; | ||
| 826 | } | ||
| 827 | |||
| 828 | ret = nouveau_svm_fault_buffer_ctor(svm, buffers[ret].oclass, 0); | ||
| 829 | if (ret) { | ||
| 830 | nouveau_svm_fini(drm); | ||
| 831 | return; | ||
| 832 | } | ||
| 833 | |||
| 834 | SVM_DBG(svm, "Initialised"); | ||
| 835 | } | ||
| @@ -0,0 +1,48 @@ | |||
| 1 | #ifndef __NOUVEAU_SVM_H__ | ||
| 2 | #define __NOUVEAU_SVM_H__ | ||
| 3 | #include <nvif/os.h> | ||
| 4 | struct drm_device; | ||
| 5 | struct drm_file; | ||
| 6 | struct nouveau_drm; | ||
| 7 | |||
| 8 | struct nouveau_svmm; | ||
| 9 | |||
| 10 | #if IS_ENABLED(CONFIG_DRM_NOUVEAU_SVM) | ||
| 11 | void nouveau_svm_init(struct nouveau_drm *); | ||
| 12 | void nouveau_svm_fini(struct nouveau_drm *); | ||
| 13 | void nouveau_svm_suspend(struct nouveau_drm *); | ||
| 14 | void nouveau_svm_resume(struct nouveau_drm *); | ||
| 15 | |||
| 16 | int nouveau_svmm_init(struct drm_device *, void *, struct drm_file *); | ||
| 17 | void nouveau_svmm_fini(struct nouveau_svmm **); | ||
| 18 | int nouveau_svmm_join(struct nouveau_svmm *, u64 inst); | ||
| 19 | void nouveau_svmm_part(struct nouveau_svmm *, u64 inst); | ||
| 20 | int nouveau_svmm_bind(struct drm_device *, void *, struct drm_file *); | ||
| 21 | #else /* IS_ENABLED(CONFIG_DRM_NOUVEAU_SVM) */ | ||
| 22 | static inline void nouveau_svm_init(struct nouveau_drm *drm) {} | ||
| 23 | static inline void nouveau_svm_fini(struct nouveau_drm *drm) {} | ||
| 24 | static inline void nouveau_svm_suspend(struct nouveau_drm *drm) {} | ||
| 25 | static inline void nouveau_svm_resume(struct nouveau_drm *drm) {} | ||
| 26 | |||
| 27 | static inline int nouveau_svmm_init(struct drm_device *device, void *p, | ||
| 28 | struct drm_file *file) | ||
| 29 | { | ||
| 30 | return -ENOSYS; | ||
| 31 | } | ||
| 32 | |||
| 33 | static inline void nouveau_svmm_fini(struct nouveau_svmm **svmmp) {} | ||
| 34 | |||
| 35 | static inline int nouveau_svmm_join(struct nouveau_svmm *svmm, u64 inst) | ||
| 36 | { | ||
| 37 | return 0; | ||
| 38 | } | ||
| 39 | |||
| 40 | static inline void nouveau_svmm_part(struct nouveau_svmm *svmm, u64 inst) {} | ||
| 41 | |||
| 42 | static inline int nouveau_svmm_bind(struct drm_device *device, void *p, | ||
| 43 | struct drm_file *file) | ||
| 44 | { | ||
| 45 | return -ENOSYS; | ||
| 46 | } | ||
| 47 | #endif /* IS_ENABLED(CONFIG_DRM_NOUVEAU_SVM) */ | ||
| 48 | #endif | ||
| @@ -22,6 +22,7 @@ · 21 unmodified lines | |||
| 22 | #include "nouveau_vmm.h" | 22 | #include "nouveau_vmm.h" |
| 23 | #include "nouveau_drv.h" | 23 | #include "nouveau_drv.h" |
| 24 | #include "nouveau_bo.h" | 24 | #include "nouveau_bo.h" |
| 25 | #include "nouveau_svm.h" | ||
| 25 | #include "nouveau_mem.h" | 26 | #include "nouveau_mem.h" |
| 26 | 27 | ||
| 27 | void | 28 | void |
| @@ -119,6 +120,7 @@ done: · 91 unmodified lines | |||
| 119 | void | 120 | void |
| 120 | nouveau_vmm_fini(struct nouveau_vmm *vmm) | 121 | nouveau_vmm_fini(struct nouveau_vmm *vmm) |
| 121 | { | 122 | { |
| 123 | nouveau_svmm_fini(&vmm->svmm); | ||
| 122 | nvif_vmm_fini(&vmm->vmm); | 124 | nvif_vmm_fini(&vmm->vmm); |
| 123 | vmm->cli = NULL; | 125 | vmm->cli = NULL; |
| 124 | } | 126 | } |
| @@ -126,7 +128,7 @@ nouveau_vmm_fini(struct nouveau_vmm *vmm) · 1 unmodified line | |||
| 126 | int | 128 | int |
| 127 | nouveau_vmm_init(struct nouveau_cli *cli, s32 oclass, struct nouveau_vmm *vmm) | 129 | nouveau_vmm_init(struct nouveau_cli *cli, s32 oclass, struct nouveau_vmm *vmm) |
| 128 | { | 130 | { |
| 129 | int ret = nvif_vmm_init(&cli->mmu, oclass, PAGE_SIZE, 0, NULL, 0, | 131 | int ret = nvif_vmm_init(&cli->mmu, oclass, false, PAGE_SIZE, 0, NULL, 0, |
| 130 | &vmm->vmm); | 132 | &vmm->vmm); |
| 131 | if (ret) | 133 | if (ret) |
| 132 | return ret; | 134 | return ret; |
| @@ -25,6 +25,7 @@ void nouveau_vma_unmap(struct nouveau_vma *); · 24 unmodified lines | |||
| 25 | struct nouveau_vmm { | 25 | struct nouveau_vmm { |
| 26 | struct nouveau_cli *cli; | 26 | struct nouveau_cli *cli; |
| 27 | struct nvif_vmm vmm; | 27 | struct nvif_vmm vmm; |
| 28 | struct nouveau_svmm *svmm; | ||
| 28 | }; | 29 | }; |
| 29 | 30 | ||
| 30 | int nouveau_vmm_init(struct nouveau_cli *, s32 oclass, struct nouveau_vmm *); | 31 | int nouveau_vmm_init(struct nouveau_cli *, s32 oclass, struct nouveau_vmm *); |
| @@ -109,7 +109,6 @@ nv84_fence_context_del(struct nouveau_channel *chan) · 108 unmodified lines | |||
| 109 | int | 109 | int |
| 110 | nv84_fence_context_new(struct nouveau_channel *chan) | 110 | nv84_fence_context_new(struct nouveau_channel *chan) |
| 111 | { | 111 | { |
| 112 | struct nouveau_cli *cli = (void *)chan->user.client; | ||
| 113 | struct nv84_fence_priv *priv = chan->drm->fence; | 112 | struct nv84_fence_priv *priv = chan->drm->fence; |
| 114 | struct nv84_fence_chan *fctx; | 113 | struct nv84_fence_chan *fctx; |
| 115 | int ret; | 114 | int ret; |
| @@ -127,7 +126,7 @@ nv84_fence_context_new(struct nouveau_channel *chan) · 11 unmodified lines | |||
| 127 | fctx->base.sequence = nv84_fence_read(chan); | 126 | fctx->base.sequence = nv84_fence_read(chan); |
| 128 | 127 | ||
| 129 | mutex_lock(&priv->mutex); | 128 | mutex_lock(&priv->mutex); |
| 130 | ret = nouveau_vma_new(priv->bo, &cli->vmm, &fctx->vma); | 129 | ret = nouveau_vma_new(priv->bo, chan->vmm, &fctx->vma); |
| 131 | mutex_unlock(&priv->mutex); | 130 | mutex_unlock(&priv->mutex); |
| 132 | 131 | ||
| 133 | if (ret) | 132 | if (ret) |
| @@ -34,7 +34,7 @@ int · 33 unmodified lines | |||
| 34 | nvif_disp_ctor(struct nvif_device *device, s32 oclass, struct nvif_disp *disp) | 34 | nvif_disp_ctor(struct nvif_device *device, s32 oclass, struct nvif_disp *disp) |
| 35 | { | 35 | { |
| 36 | static const struct nvif_mclass disps[] = { | 36 | static const struct nvif_mclass disps[] = { |
| 37 | { TU104_DISP, -1 }, | 37 | { TU102_DISP, -1 }, |
| 38 | { GV100_DISP, -1 }, | 38 | { GV100_DISP, -1 }, |
| 39 | { GP102_DISP, -1 }, | 39 | { GP102_DISP, -1 }, |
| 40 | { GP100_DISP, -1 }, | 40 | { GP100_DISP, -1 }, |
| @@ -112,8 +112,8 @@ nvif_vmm_fini(struct nvif_vmm *vmm) · 111 unmodified lines | |||
| 112 | } | 112 | } |
| 113 | 113 | ||
| 114 | int | 114 | int |
| 115 | nvif_vmm_init(struct nvif_mmu *mmu, s32 oclass, u64 addr, u64 size, | 115 | nvif_vmm_init(struct nvif_mmu *mmu, s32 oclass, bool managed, u64 addr, |
| 116 | void *argv, u32 argc, struct nvif_vmm *vmm) | 116 | u64 size, void *argv, u32 argc, struct nvif_vmm *vmm) |
| 117 | { | 117 | { |
| 118 | struct nvif_vmm_v0 *args; | 118 | struct nvif_vmm_v0 *args; |
| 119 | u32 argn = sizeof(*args) + argc; | 119 | u32 argn = sizeof(*args) + argc; |
| @@ -125,6 +125,7 @@ nvif_vmm_init(struct nvif_mmu *mmu, s32 oclass, u64 addr, u64 size, · 5 unmodified lines | |||
| 125 | if (!(args = kmalloc(argn, GFP_KERNEL))) | 125 | if (!(args = kmalloc(argn, GFP_KERNEL))) |
| 126 | return -ENOMEM; | 126 | return -ENOMEM; |
| 127 | args->version = 0; | 127 | args->version = 0; |
| 128 | args->managed = managed; | ||
| 128 | args->addr = addr; | 129 | args->addr = addr; |
| 129 | args->size = size; | 130 | args->size = size; |
| 130 | memcpy(args->data, argv, argc); | 131 | memcpy(args->data, argv, argc); |
| @@ -39,6 +39,7 @@ nvkm_subdev_name[NVKM_SUBDEV_NR] = { · 38 unmodified lines | |||
| 39 | [NVKM_SUBDEV_FB ] = "fb", | 39 | [NVKM_SUBDEV_FB ] = "fb", |
| 40 | [NVKM_SUBDEV_FUSE ] = "fuse", | 40 | [NVKM_SUBDEV_FUSE ] = "fuse", |
| 41 | [NVKM_SUBDEV_GPIO ] = "gpio", | 41 | [NVKM_SUBDEV_GPIO ] = "gpio", |
| 42 | [NVKM_SUBDEV_GSP ] = "gsp", | ||
| 42 | [NVKM_SUBDEV_I2C ] = "i2c", | 43 | [NVKM_SUBDEV_I2C ] = "i2c", |
| 43 | [NVKM_SUBDEV_IBUS ] = "priv", | 44 | [NVKM_SUBDEV_IBUS ] = "priv", |
| 44 | [NVKM_SUBDEV_ICCSENSE] = "iccsense", | 45 | [NVKM_SUBDEV_ICCSENSE] = "iccsense", |
| @@ -6,4 +6,4 @@ nvkm-y += nvkm/engine/ce/gm200.o · 5 unmodified lines | |||
| 6 | nvkm-y += nvkm/engine/ce/gp100.o | 6 | nvkm-y += nvkm/engine/ce/gp100.o |
| 7 | nvkm-y += nvkm/engine/ce/gp102.o | 7 | nvkm-y += nvkm/engine/ce/gp102.o |
| 8 | nvkm-y += nvkm/engine/ce/gv100.o | 8 | nvkm-y += nvkm/engine/ce/gv100.o |
| 9 | nvkm-y += nvkm/engine/ce/tu104.o | 9 | nvkm-y += nvkm/engine/ce/tu102.o |
| @@ -24,7 +24,7 @@ · 23 unmodified lines | |||
| 24 | #include <nvif/class.h> | 24 | #include <nvif/class.h> |
| 25 | 25 | ||
| 26 | static const struct nvkm_engine_func | 26 | static const struct nvkm_engine_func |
| 27 | tu104_ce = { | 27 | tu102_ce = { |
| 28 | .intr = gp100_ce_intr, | 28 | .intr = gp100_ce_intr, |
| 29 | .sclass = { | 29 | .sclass = { |
| 30 | { -1, -1, TURING_DMA_COPY_A }, | 30 | { -1, -1, TURING_DMA_COPY_A }, |
| @@ -33,8 +33,8 @@ tu104_ce = { · 2 unmodified lines | |||
| 33 | }; | 33 | }; |
| 34 | 34 | ||
| 35 | int | 35 | int |
| 36 | tu104_ce_new(struct nvkm_device *device, int index, | 36 | tu102_ce_new(struct nvkm_device *device, int index, |
| 37 | struct nvkm_engine **pengine) | 37 | struct nvkm_engine **pengine) |
| 38 | { | 38 | { |
| 39 | return nvkm_engine_new_(&tu104_ce, device, index, true, pengine); | 39 | return nvkm_engine_new_(&tu102_ce, device, index, true, pengine); |
| 40 | } | 40 | } |
| @@ -1613,7 +1613,7 @@ nvd7_chipset = { · 1612 unmodified lines | |||
| 1613 | .pci = gf106_pci_new, | 1613 | .pci = gf106_pci_new, |
| 1614 | .therm = gf119_therm_new, | 1614 | .therm = gf119_therm_new, |
| 1615 | .timer = nv41_timer_new, | 1615 | .timer = nv41_timer_new, |
| 1616 | .volt = gf100_volt_new, | 1616 | .volt = gf117_volt_new, |
| 1617 | .ce[0] = gf100_ce_new, | 1617 | .ce[0] = gf100_ce_new, |
| 1618 | .disp = gf119_disp_new, | 1618 | .disp = gf119_disp_new, |
| 1619 | .dma = gf119_dma_new, | 1619 | .dma = gf119_dma_new, |
| @@ -2405,6 +2405,7 @@ nv140_chipset = { · 785 unmodified lines | |||
| 2405 | .fb = gv100_fb_new, | 2405 | .fb = gv100_fb_new, |
| 2406 | .fuse = gm107_fuse_new, | 2406 | .fuse = gm107_fuse_new, |
| 2407 | .gpio = gk104_gpio_new, | 2407 | .gpio = gk104_gpio_new, |
| 2408 | .gsp = gv100_gsp_new, | ||
| 2408 | .i2c = gm200_i2c_new, | 2409 | .i2c = gm200_i2c_new, |
| 2409 | .ibus = gm200_ibus_new, | 2410 | .ibus = gm200_ibus_new, |
| 2410 | .imem = nv50_instmem_new, | 2411 | .imem = nv50_instmem_new, |
| @@ -2437,97 +2438,106 @@ nv140_chipset = { · 26 unmodified lines | |||
| 2437 | static const struct nvkm_device_chip | 2438 | static const struct nvkm_device_chip |
| 2438 | nv162_chipset = { | 2439 | nv162_chipset = { |
| 2439 | .name = "TU102", | 2440 | .name = "TU102", |
| 2440 | .bar = tu104_bar_new, | 2441 | .bar = tu102_bar_new, |
| 2441 | .bios = nvkm_bios_new, | 2442 | .bios = nvkm_bios_new, |
| 2442 | .bus = gf100_bus_new, | 2443 | .bus = gf100_bus_new, |
| 2443 | .devinit = tu104_devinit_new, | 2444 | .devinit = tu102_devinit_new, |
| 2444 | .fault = tu104_fault_new, | 2445 | .fault = tu102_fault_new, |
| 2445 | .fb = gv100_fb_new, | 2446 | .fb = gv100_fb_new, |
| 2446 | .fuse = gm107_fuse_new, | 2447 | .fuse = gm107_fuse_new, |
| 2447 | .gpio = gk104_gpio_new, | 2448 | .gpio = gk104_gpio_new, |
| 2449 | .gsp = gv100_gsp_new, | ||
| 2448 | .i2c = gm200_i2c_new, | 2450 | .i2c = gm200_i2c_new, |
| 2449 | .ibus = gm200_ibus_new, | 2451 | .ibus = gm200_ibus_new, |
| 2450 | .imem = nv50_instmem_new, | 2452 | .imem = nv50_instmem_new, |
| 2451 | .ltc = gp102_ltc_new, | 2453 | .ltc = gp102_ltc_new, |
| 2452 | .mc = tu104_mc_new, | 2454 | .mc = tu102_mc_new, |
| 2453 | .mmu = tu104_mmu_new, | 2455 | .mmu = tu102_mmu_new, |
| 2454 | .pci = gp100_pci_new, | 2456 | .pci = gp100_pci_new, |
| 2455 | .pmu = gp102_pmu_new, | 2457 | .pmu = gp102_pmu_new, |
| 2456 | .therm = gp100_therm_new, | 2458 | .therm = gp100_therm_new, |
| 2457 | .timer = gk20a_timer_new, | 2459 | .timer = gk20a_timer_new, |
| 2458 | .top = gk104_top_new, | 2460 | .top = gk104_top_new, |
| 2459 | .ce[0] = tu104_ce_new, | 2461 | .ce[0] = tu102_ce_new, |
| 2460 | .ce[1] = tu104_ce_new, | 2462 | .ce[1] = tu102_ce_new, |
| 2461 | .ce[2] = tu104_ce_new, | 2463 | .ce[2] = tu102_ce_new, |
| 2462 | .ce[3] = tu104_ce_new, | 2464 | .ce[3] = tu102_ce_new, |
| 2463 | .ce[4] = tu104_ce_new, | 2465 | .ce[4] = tu102_ce_new, |
| 2464 | .disp = tu104_disp_new, | 2466 | .disp = tu102_disp_new, |
| 2465 | .dma = gv100_dma_new, | 2467 | .dma = gv100_dma_new, |
| 2466 | .fifo = tu104_fifo_new, | 2468 | .fifo = tu102_fifo_new, |
| 2469 | .nvdec[0] = gp102_nvdec_new, | ||
| 2470 | .sec2 = tu102_sec2_new, | ||
| 2467 | }; | 2471 | }; |
| 2468 | 2472 | ||
| 2469 | static const struct nvkm_device_chip | 2473 | static const struct nvkm_device_chip |
| 2470 | nv164_chipset = { | 2474 | nv164_chipset = { |
| 2471 | .name = "TU104", | 2475 | .name = "TU104", |
| 2472 | .bar = tu104_bar_new, | 2476 | .bar = tu102_bar_new, |
| 2473 | .bios = nvkm_bios_new, | 2477 | .bios = nvkm_bios_new, |
| 2474 | .bus = gf100_bus_new, | 2478 | .bus = gf100_bus_new, |
| 2475 | .devinit = tu104_devinit_new, | 2479 | .devinit = tu102_devinit_new, |
| 2476 | .fault = tu104_fault_new, | 2480 | .fault = tu102_fault_new, |
| 2477 | .fb = gv100_fb_new, | 2481 | .fb = gv100_fb_new, |
| 2478 | .fuse = gm107_fuse_new, | 2482 | .fuse = gm107_fuse_new, |
| 2479 | .gpio = gk104_gpio_new, | 2483 | .gpio = gk104_gpio_new, |
| 2484 | .gsp = gv100_gsp_new, | ||
| 2480 | .i2c = gm200_i2c_new, | 2485 | .i2c = gm200_i2c_new, |
| 2481 | .ibus = gm200_ibus_new, | 2486 | .ibus = gm200_ibus_new, |
| 2482 | .imem = nv50_instmem_new, | 2487 | .imem = nv50_instmem_new, |
| 2483 | .ltc = gp102_ltc_new, | 2488 | .ltc = gp102_ltc_new, |
| 2484 | .mc = tu104_mc_new, | 2489 | .mc = tu102_mc_new, |
| 2485 | .mmu = tu104_mmu_new, | 2490 | .mmu = tu102_mmu_new, |
| 2486 | .pci = gp100_pci_new, | 2491 | .pci = gp100_pci_new, |
| 2487 | .pmu = gp102_pmu_new, | 2492 | .pmu = gp102_pmu_new, |
| 2488 | .therm = gp100_therm_new, | 2493 | .therm = gp100_therm_new, |
| 2489 | .timer = gk20a_timer_new, | 2494 | .timer = gk20a_timer_new, |
| 2490 | .top = gk104_top_new, | 2495 | .top = gk104_top_new, |
| 2491 | .ce[0] = tu104_ce_new, | 2496 | .ce[0] = tu102_ce_new, |
| 2492 | .ce[1] = tu104_ce_new, | 2497 | .ce[1] = tu102_ce_new, |
| 2493 | .ce[2] = tu104_ce_new, | 2498 | .ce[2] = tu102_ce_new, |
| 2494 | .ce[3] = tu104_ce_new, | 2499 | .ce[3] = tu102_ce_new, |
| 2495 | .ce[4] = tu104_ce_new, | 2500 | .ce[4] = tu102_ce_new, |
| 2496 | .disp = tu104_disp_new, | 2501 | .disp = tu102_disp_new, |
| 2497 | .dma = gv100_dma_new, | 2502 | .dma = gv100_dma_new, |
| 2498 | .fifo = tu104_fifo_new, | 2503 | .fifo = tu102_fifo_new, |
| 2504 | .nvdec[0] = gp102_nvdec_new, | ||
| 2505 | .sec2 = tu102_sec2_new, | ||
| 2499 | }; | 2506 | }; |
| 2500 | 2507 | ||
| 2501 | static const struct nvkm_device_chip | 2508 | static const struct nvkm_device_chip |
| 2502 | nv166_chipset = { | 2509 | nv166_chipset = { |
| 2503 | .name = "TU106", | 2510 | .name = "TU106", |
| 2504 | .bar = tu104_bar_new, | 2511 | .bar = tu102_bar_new, |
| 2505 | .bios = nvkm_bios_new, | 2512 | .bios = nvkm_bios_new, |
| 2506 | .bus = gf100_bus_new, | 2513 | .bus = gf100_bus_new, |
| 2507 | .devinit = tu104_devinit_new, | 2514 | .devinit = tu102_devinit_new, |
| 2508 | .fault = tu104_fault_new, | 2515 | .fault = tu102_fault_new, |
| 2509 | .fb = gv100_fb_new, | 2516 | .fb = gv100_fb_new, |
| 2510 | .fuse = gm107_fuse_new, | 2517 | .fuse = gm107_fuse_new, |
| 2511 | .gpio = gk104_gpio_new, | 2518 | .gpio = gk104_gpio_new, |
| 2519 | .gsp = gv100_gsp_new, | ||
| 2512 | .i2c = gm200_i2c_new, | 2520 | .i2c = gm200_i2c_new, |
| 2513 | .ibus = gm200_ibus_new, | 2521 | .ibus = gm200_ibus_new, |
| 2514 | .imem = nv50_instmem_new, | 2522 | .imem = nv50_instmem_new, |
| 2515 | .ltc = gp102_ltc_new, | 2523 | .ltc = gp102_ltc_new, |
| 2516 | .mc = tu104_mc_new, | 2524 | .mc = tu102_mc_new, |
| 2517 | .mmu = tu104_mmu_new, | 2525 | .mmu = tu102_mmu_new, |
| 2518 | .pci = gp100_pci_new, | 2526 | .pci = gp100_pci_new, |
| 2519 | .pmu = gp102_pmu_new, | 2527 | .pmu = gp102_pmu_new, |
| 2520 | .therm = gp100_therm_new, | 2528 | .therm = gp100_therm_new, |
| 2521 | .timer = gk20a_timer_new, | 2529 | .timer = gk20a_timer_new, |
| 2522 | .top = gk104_top_new, | 2530 | .top = gk104_top_new, |
| 2523 | .ce[0] = tu104_ce_new, | 2531 | .ce[0] = tu102_ce_new, |
| 2524 | .ce[1] = tu104_ce_new, | 2532 | .ce[1] = tu102_ce_new, |
| 2525 | .ce[2] = tu104_ce_new, | 2533 | .ce[2] = tu102_ce_new, |
| 2526 | .ce[3] = tu104_ce_new, | 2534 | .ce[3] = tu102_ce_new, |
| 2527 | .ce[4] = tu104_ce_new, | 2535 | .ce[4] = tu102_ce_new, |
| 2528 | .disp = tu104_disp_new, | 2536 | .disp = tu102_disp_new, |
| 2529 | .dma = gv100_dma_new, | 2537 | .dma = gv100_dma_new, |
| 2530 | .fifo = tu104_fifo_new, | 2538 | .fifo = tu102_fifo_new, |
| 2539 | .nvdec[0] = gp102_nvdec_new, | ||
| 2540 | .sec2 = tu102_sec2_new, | ||
| 2531 | }; | 2541 | }; |
| 2532 | 2542 | ||
| 2533 | static int | 2543 | static int |
| @@ -2567,6 +2577,7 @@ nvkm_device_subdev(struct nvkm_device *device, int index) · 33 unmodified lines | |||
| 2567 | _(FB , device->fb , &device->fb->subdev); | 2577 | _(FB , device->fb , &device->fb->subdev); |
| 2568 | _(FUSE , device->fuse , &device->fuse->subdev); | 2578 | _(FUSE , device->fuse , &device->fuse->subdev); |
| 2569 | _(GPIO , device->gpio , &device->gpio->subdev); | 2579 | _(GPIO , device->gpio , &device->gpio->subdev); |
| 2580 | _(GSP , device->gsp , &device->gsp->subdev); | ||
| 2570 | _(I2C , device->i2c , &device->i2c->subdev); | 2581 | _(I2C , device->i2c , &device->i2c->subdev); |
| 2571 | _(IBUS , device->ibus , device->ibus); | 2582 | _(IBUS , device->ibus , device->ibus); |
| 2572 | _(ICCSENSE, device->iccsense, &device->iccsense->subdev); | 2583 | _(ICCSENSE, device->iccsense, &device->iccsense->subdev); |
| @@ -3050,6 +3061,7 @@ nvkm_device_ctor(const struct nvkm_device_func *func, · 477 unmodified lines | |||
| 3050 | _(NVKM_SUBDEV_FB , fb); | 3061 | _(NVKM_SUBDEV_FB , fb); |
| 3051 | _(NVKM_SUBDEV_FUSE , fuse); | 3062 | _(NVKM_SUBDEV_FUSE , fuse); |
| 3052 | _(NVKM_SUBDEV_GPIO , gpio); | 3063 | _(NVKM_SUBDEV_GPIO , gpio); |
| 3064 | _(NVKM_SUBDEV_GSP , gsp); | ||
| 3053 | _(NVKM_SUBDEV_I2C , i2c); | 3065 | _(NVKM_SUBDEV_I2C , i2c); |
| 3054 | _(NVKM_SUBDEV_IBUS , ibus); | 3066 | _(NVKM_SUBDEV_IBUS , ibus); |
| 3055 | _(NVKM_SUBDEV_ICCSENSE, iccsense); | 3067 | _(NVKM_SUBDEV_ICCSENSE, iccsense); |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | |||
| 12 | #include <subdev/fb.h> | 12 | #include <subdev/fb.h> |
| 13 | #include <subdev/fuse.h> | 13 | #include <subdev/fuse.h> |
| 14 | #include <subdev/gpio.h> | 14 | #include <subdev/gpio.h> |
| 15 | #include <subdev/gsp.h> | ||
| 15 | #include <subdev/i2c.h> | 16 | #include <subdev/i2c.h> |
| 16 | #include <subdev/ibus.h> | 17 | #include <subdev/ibus.h> |
| 17 | #include <subdev/iccsense.h> | 18 | #include <subdev/iccsense.h> |
| @@ -365,16 +365,15 @@ nvkm_udevice_child_get(struct nvkm_object *object, int index, · 364 unmodified lines | |||
| 365 | } | 365 | } |
| 366 | 366 | ||
| 367 | if (!sclass) { | 367 | if (!sclass) { |
| 368 | switch (index) { | 368 | if (index-- == 0) |
| 369 | case 0: sclass = &nvkm_control_oclass; break; | 369 | sclass = &nvkm_control_oclass; |
| 370 | case 1: | 370 | else if (device->mmu && index-- == 0) |
| 371 | if (!device->mmu) | ||
| 372 | return -EINVAL; | ||
| 373 | sclass = &device->mmu->user; | 371 | sclass = &device->mmu->user; |
| 374 | break; | 372 | else if (device->fault && index-- == 0) |
| 375 | default: | 373 | sclass = &device->fault->user; |
| 374 | else | ||
| 376 | return -EINVAL; | 375 | return -EINVAL; |
| 377 | } | 376 | |
| 378 | oclass->base = sclass->base; | 377 | oclass->base = sclass->base; |
| 379 | } | 378 | } |
| 380 | 379 | ||
| @@ -15,7 +15,7 @@ nvkm-y += nvkm/engine/disp/gm200.o · 14 unmodified lines | |||
| 15 | nvkm-y += nvkm/engine/disp/gp100.o | 15 | nvkm-y += nvkm/engine/disp/gp100.o |
| 16 | nvkm-y += nvkm/engine/disp/gp102.o | 16 | nvkm-y += nvkm/engine/disp/gp102.o |
| 17 | nvkm-y += nvkm/engine/disp/gv100.o | 17 | nvkm-y += nvkm/engine/disp/gv100.o |
| 18 | nvkm-y += nvkm/engine/disp/tu104.o | 18 | nvkm-y += nvkm/engine/disp/tu102.o |
| 19 | nvkm-y += nvkm/engine/disp/vga.o | 19 | nvkm-y += nvkm/engine/disp/vga.o |
| 20 | 20 | ||
| 21 | nvkm-y += nvkm/engine/disp/head.o | 21 | nvkm-y += nvkm/engine/disp/head.o |
| @@ -39,7 +39,7 @@ nvkm-y += nvkm/engine/disp/sorgk104.o · 17 unmodified lines | |||
| 39 | nvkm-y += nvkm/engine/disp/sorgm107.o | 39 | nvkm-y += nvkm/engine/disp/sorgm107.o |
| 40 | nvkm-y += nvkm/engine/disp/sorgm200.o | 40 | nvkm-y += nvkm/engine/disp/sorgm200.o |
| 41 | nvkm-y += nvkm/engine/disp/sorgv100.o | 41 | nvkm-y += nvkm/engine/disp/sorgv100.o |
| 42 | nvkm-y += nvkm/engine/disp/sortu104.o | 42 | nvkm-y += nvkm/engine/disp/sortu102.o |
| 43 | 43 | ||
| 44 | nvkm-y += nvkm/engine/disp/outp.o | 44 | nvkm-y += nvkm/engine/disp/outp.o |
| 45 | nvkm-y += nvkm/engine/disp/dp.o | 45 | nvkm-y += nvkm/engine/disp/dp.o |
| @@ -71,7 +71,7 @@ nvkm-y += nvkm/engine/disp/rootgm200.o · 25 unmodified lines | |||
| 71 | nvkm-y += nvkm/engine/disp/rootgp100.o | 71 | nvkm-y += nvkm/engine/disp/rootgp100.o |
| 72 | nvkm-y += nvkm/engine/disp/rootgp102.o | 72 | nvkm-y += nvkm/engine/disp/rootgp102.o |
| 73 | nvkm-y += nvkm/engine/disp/rootgv100.o | 73 | nvkm-y += nvkm/engine/disp/rootgv100.o |
| 74 | nvkm-y += nvkm/engine/disp/roottu104.o | 74 | nvkm-y += nvkm/engine/disp/roottu102.o |
| 75 | 75 | ||
| 76 | nvkm-y += nvkm/engine/disp/channv50.o | 76 | nvkm-y += nvkm/engine/disp/channv50.o |
| 77 | nvkm-y += nvkm/engine/disp/changf119.o | 77 | nvkm-y += nvkm/engine/disp/changf119.o |
| @@ -91,15 +91,21 @@ gf119_disp_intr_error(struct nv50_disp *disp, int chid) · 90 unmodified lines | |||
| 91 | { | 91 | { |
| 92 | struct nvkm_subdev *subdev = &disp->base.engine.subdev; | 92 | struct nvkm_subdev *subdev = &disp->base.engine.subdev; |
| 93 | struct nvkm_device *device = subdev->device; | 93 | struct nvkm_device *device = subdev->device; |
| 94 | u32 mthd = nvkm_rd32(device, 0x6101f0 + (chid * 12)); | 94 | u32 stat = nvkm_rd32(device, 0x6101f0 + (chid * 12)); |
| 95 | u32 type = (stat & 0x00007000) >> 12; | ||
| 96 | u32 mthd = (stat & 0x00000ffc); | ||
| 95 | u32 data = nvkm_rd32(device, 0x6101f4 + (chid * 12)); | 97 | u32 data = nvkm_rd32(device, 0x6101f4 + (chid * 12)); |
| 96 | u32 unkn = nvkm_rd32(device, 0x6101f8 + (chid * 12)); | 98 | u32 code = nvkm_rd32(device, 0x6101f8 + (chid * 12)); |
| 99 | const struct nvkm_enum *reason = | ||
| 100 | nvkm_enum_find(nv50_disp_intr_error_type, type); | ||
| 97 | 101 | ||
| 98 | nvkm_error(subdev, "chid %d mthd %04x data %08x %08x %08x\n", | 102 | nvkm_error(subdev, "chid %d stat %08x reason %d [%s] mthd %04x " |
| 99 | chid, (mthd & 0x0000ffc), data, mthd, unkn); | 103 | "data %08x code %08x\n", |
| 104 | chid, stat, type, reason ? reason->name : "", | ||
| 105 | mthd, data, code); | ||
| 100 | 106 | ||
| 101 | if (chid < ARRAY_SIZE(disp->chan)) { | 107 | if (chid < ARRAY_SIZE(disp->chan)) { |
| 102 | switch (mthd & 0xffc) { | 108 | switch (mthd) { |
| 103 | case 0x0080: | 109 | case 0x0080: |
| 104 | nv50_disp_chan_mthd(disp->chan[chid], NV_DBG_ERROR); | 110 | nv50_disp_chan_mthd(disp->chan[chid], NV_DBG_ERROR); |
| 105 | break; | 111 | break; |
| @@ -103,10 +103,13 @@ gv100_disp_exception(struct nv50_disp *disp, int chid) · 102 unmodified lines | |||
| 103 | u32 mthd = (stat & 0x00000fff) << 2; | 103 | u32 mthd = (stat & 0x00000fff) << 2; |
| 104 | u32 data = nvkm_rd32(device, 0x611024 + (chid * 12)); | 104 | u32 data = nvkm_rd32(device, 0x611024 + (chid * 12)); |
| 105 | u32 code = nvkm_rd32(device, 0x611028 + (chid * 12)); | 105 | u32 code = nvkm_rd32(device, 0x611028 + (chid * 12)); |
| 106 | const struct nvkm_enum *reason = | ||
| 107 | nvkm_enum_find(nv50_disp_intr_error_type, type); | ||
| 106 | 108 | ||
| 107 | nvkm_error(subdev, "chid %d %08x [type %d mthd %04x] " | 109 | nvkm_error(subdev, "chid %d stat %08x reason %d [%s] mthd %04x " |
| 108 | "data %08x code %08x\n", | 110 | "data %08x code %08x\n", |
| 109 | chid, stat, type, mthd, data, code); | 111 | chid, stat, type, reason ? reason->name : "", |
| 112 | mthd, data, code); | ||
| 110 | 113 | ||
| 111 | if (chid < ARRAY_SIZE(disp->chan) && disp->chan[chid]) { | 114 | if (chid < ARRAY_SIZE(disp->chan) && disp->chan[chid]) { |
| 112 | switch (mthd) { | 115 | switch (mthd) { |
| @@ -201,5 +201,5 @@ int gm200_sor_new(struct nvkm_disp *, int); · 200 unmodified lines | |||
| 201 | int gv100_sor_cnt(struct nvkm_disp *, unsigned long *); | 201 | int gv100_sor_cnt(struct nvkm_disp *, unsigned long *); |
| 202 | int gv100_sor_new(struct nvkm_disp *, int); | 202 | int gv100_sor_new(struct nvkm_disp *, int); |
| 203 | 203 | ||
| 204 | int tu104_sor_new(struct nvkm_disp *, int); | 204 | int tu102_sor_new(struct nvkm_disp *, int); |
| 205 | #endif | 205 | #endif |
| @@ -28,7 +28,6 @@ · 27 unmodified lines | |||
| 28 | #include "rootnv50.h" | 28 | #include "rootnv50.h" |
| 29 | 29 | ||
| 30 | #include <core/client.h> | 30 | #include <core/client.h> |
| 31 | #include <core/enum.h> | ||
| 32 | #include <core/ramht.h> | 31 | #include <core/ramht.h> |
| 33 | #include <subdev/bios.h> | 32 | #include <subdev/bios.h> |
| 34 | #include <subdev/bios/disp.h> | 33 | #include <subdev/bios/disp.h> |
| @@ -593,12 +592,15 @@ nv50_disp_super(struct work_struct *work) · 558 unmodified lines | |||
| 593 | nvkm_wr32(device, 0x610030, 0x80000000); | 592 | nvkm_wr32(device, 0x610030, 0x80000000); |
| 594 | } | 593 | } |
| 595 | 594 | ||
| 596 | static const struct nvkm_enum | 595 | const struct nvkm_enum |
| 597 | nv50_disp_intr_error_type[] = { | 596 | nv50_disp_intr_error_type[] = { |
| 598 | { 3, "ILLEGAL_MTHD" }, | 597 | { 0, "NONE" }, |
| 599 | { 4, "INVALID_VALUE" }, | 598 | { 1, "PUSHBUFFER_ERR" }, |
| 599 | { 2, "TRAP" }, | ||
| 600 | { 3, "RESERVED_METHOD" }, | ||
| 601 | { 4, "INVALID_ARG" }, | ||
| 600 | { 5, "INVALID_STATE" }, | 602 | { 5, "INVALID_STATE" }, |
| 601 | { 7, "INVALID_HANDLE" }, | 603 | { 7, "UNRESOLVABLE_HANDLE" }, |
| 602 | {} | 604 | {} |
| 603 | }; | 605 | }; |
| 604 | 606 | ||
| @@ -5,6 +5,8 @@ · 4 unmodified lines | |||
| 5 | #include "priv.h" | 5 | #include "priv.h" |
| 6 | struct nvkm_head; | 6 | struct nvkm_head; |
| 7 | 7 | ||
| 8 | #include <core/enum.h> | ||
| 9 | |||
| 8 | struct nv50_disp { | 10 | struct nv50_disp { |
| 9 | const struct nv50_disp_func *func; | 11 | const struct nv50_disp_func *func; |
| 10 | struct nvkm_disp base; | 12 | struct nvkm_disp base; |
| @@ -71,6 +73,7 @@ int nv50_disp_init(struct nv50_disp *); · 60 unmodified lines | |||
| 71 | void nv50_disp_fini(struct nv50_disp *); | 73 | void nv50_disp_fini(struct nv50_disp *); |
| 72 | void nv50_disp_intr(struct nv50_disp *); | 74 | void nv50_disp_intr(struct nv50_disp *); |
| 73 | void nv50_disp_super(struct work_struct *); | 75 | void nv50_disp_super(struct work_struct *); |
| 76 | extern const struct nvkm_enum nv50_disp_intr_error_type[]; | ||
| 74 | 77 | ||
| 75 | int gf119_disp_init(struct nv50_disp *); | 78 | int gf119_disp_init(struct nv50_disp *); |
| 76 | void gf119_disp_fini(struct nv50_disp *); | 79 | void gf119_disp_fini(struct nv50_disp *); |
| @@ -37,5 +37,5 @@ extern const struct nvkm_disp_oclass gm200_disp_root_oclass; · 36 unmodified lines | |||
| 37 | extern const struct nvkm_disp_oclass gp100_disp_root_oclass; | 37 | extern const struct nvkm_disp_oclass gp100_disp_root_oclass; |
| 38 | extern const struct nvkm_disp_oclass gp102_disp_root_oclass; | 38 | extern const struct nvkm_disp_oclass gp102_disp_root_oclass; |
| 39 | extern const struct nvkm_disp_oclass gv100_disp_root_oclass; | 39 | extern const struct nvkm_disp_oclass gv100_disp_root_oclass; |
| 40 | extern const struct nvkm_disp_oclass tu104_disp_root_oclass; | 40 | extern const struct nvkm_disp_oclass tu102_disp_root_oclass; |
| 41 | #endif | 41 | #endif |
| @@ -25,28 +25,28 @@ · 24 unmodified lines | |||
| 25 | #include <nvif/class.h> | 25 | #include <nvif/class.h> |
| 26 | 26 | ||
| 27 | static const struct nv50_disp_root_func | 27 | static const struct nv50_disp_root_func |
| 28 | tu104_disp_root = { | 28 | tu102_disp_root = { |
| 29 | .user = { | 29 | .user = { |
| 30 | {{0,0,TU104_DISP_CURSOR }, gv100_disp_curs_new }, | 30 | {{0,0,TU102_DISP_CURSOR }, gv100_disp_curs_new }, |
| 31 | {{0,0,TU104_DISP_WINDOW_IMM_CHANNEL_DMA}, gv100_disp_wimm_new }, | 31 | {{0,0,TU102_DISP_WINDOW_IMM_CHANNEL_DMA}, gv100_disp_wimm_new }, |
| 32 | {{0,0,TU104_DISP_CORE_CHANNEL_DMA }, gv100_disp_core_new }, | 32 | {{0,0,TU102_DISP_CORE_CHANNEL_DMA }, gv100_disp_core_new }, |
| 33 | {{0,0,TU104_DISP_WINDOW_CHANNEL_DMA }, gv100_disp_wndw_new }, | 33 | {{0,0,TU102_DISP_WINDOW_CHANNEL_DMA }, gv100_disp_wndw_new }, |
| 34 | {} | 34 | {} |
| 35 | }, | 35 | }, |
| 36 | }; | 36 | }; |
| 37 | 37 | ||
| 38 | static int | 38 | static int |
| 39 | tu104_disp_root_new(struct nvkm_disp *disp, const struct nvkm_oclass *oclass, | 39 | tu102_disp_root_new(struct nvkm_disp *disp, const struct nvkm_oclass *oclass, |
| 40 | void *data, u32 size, struct nvkm_object **pobject) | 40 | void *data, u32 size, struct nvkm_object **pobject) |
| 41 | { | 41 | { |
| 42 | return nv50_disp_root_new_(&tu104_disp_root, disp, oclass, | 42 | return nv50_disp_root_new_(&tu102_disp_root, disp, oclass, |
| 43 | data, size, pobject); | 43 | data, size, pobject); |
| 44 | } | 44 | } |
| 45 | 45 | ||
| 46 | const struct nvkm_disp_oclass | 46 | const struct nvkm_disp_oclass |
| 47 | tu104_disp_root_oclass = { | 47 | tu102_disp_root_oclass = { |
| 48 | .base.oclass = TU104_DISP, | 48 | .base.oclass = TU102_DISP, |
| 49 | .base.minver = -1, | 49 | .base.minver = -1, |
| 50 | .base.maxver = -1, | 50 | .base.maxver = -1, |
| 51 | .ctor = tu104_disp_root_new, | 51 | .ctor = tu102_disp_root_new, |
| 52 | }; | 52 | }; |
| @@ -24,7 +24,7 @@ · 23 unmodified lines | |||
| 24 | #include <subdev/timer.h> | 24 | #include <subdev/timer.h> |
| 25 | 25 | ||
| 26 | static void | 26 | static void |
| 27 | tu104_sor_dp_vcpi(struct nvkm_ior *sor, int head, | 27 | tu102_sor_dp_vcpi(struct nvkm_ior *sor, int head, |
| 28 | u8 slot, u8 slot_nr, u16 pbn, u16 aligned) | 28 | u8 slot, u8 slot_nr, u16 pbn, u16 aligned) |
| 29 | { | 29 | { |
| 30 | struct nvkm_device *device = sor->disp->engine.subdev.device; | 30 | struct nvkm_device *device = sor->disp->engine.subdev.device; |
| @@ -35,7 +35,7 @@ tu104_sor_dp_vcpi(struct nvkm_ior *sor, int head, · 4 unmodified lines | |||
| 35 | } | 35 | } |
| 36 | 36 | ||
| 37 | static int | 37 | static int |
| 38 | tu104_sor_dp_links(struct nvkm_ior *sor, struct nvkm_i2c_aux *aux) | 38 | tu102_sor_dp_links(struct nvkm_ior *sor, struct nvkm_i2c_aux *aux) |
| 39 | { | 39 | { |
| 40 | struct nvkm_device *device = sor->disp->engine.subdev.device; | 40 | struct nvkm_device *device = sor->disp->engine.subdev.device; |
| 41 | const u32 soff = nv50_ior_base(sor); | 41 | const u32 soff = nv50_ior_base(sor); |
| @@ -62,7 +62,7 @@ tu104_sor_dp_links(struct nvkm_ior *sor, struct nvkm_i2c_aux *aux) · 20 unmodified lines | |||
| 62 | } | 62 | } |
| 63 | 63 | ||
| 64 | static const struct nvkm_ior_func | 64 | static const struct nvkm_ior_func |
| 65 | tu104_sor = { | 65 | tu102_sor = { |
| 66 | .route = { | 66 | .route = { |
| 67 | .get = gm200_sor_route_get, | 67 | .get = gm200_sor_route_get, |
| 68 | .set = gm200_sor_route_set, | 68 | .set = gm200_sor_route_set, |
| @@ -75,11 +75,11 @@ tu104_sor = { · 6 unmodified lines | |||
| 75 | }, | 75 | }, |
| 76 | .dp = { | 76 | .dp = { |
| 77 | .lanes = { 0, 1, 2, 3 }, | 77 | .lanes = { 0, 1, 2, 3 }, |
| 78 | .links = tu104_sor_dp_links, | 78 | .links = tu102_sor_dp_links, |
| 79 | .power = g94_sor_dp_power, | 79 | .power = g94_sor_dp_power, |
| 80 | .pattern = gm107_sor_dp_pattern, | 80 | .pattern = gm107_sor_dp_pattern, |
| 81 | .drive = gm200_sor_dp_drive, | 81 | .drive = gm200_sor_dp_drive, |
| 82 | .vcpi = tu104_sor_dp_vcpi, | 82 | .vcpi = tu102_sor_dp_vcpi, |
| 83 | .audio = gv100_sor_dp_audio, | 83 | .audio = gv100_sor_dp_audio, |
| 84 | .audio_sym = gv100_sor_dp_audio_sym, | 84 | .audio_sym = gv100_sor_dp_audio_sym, |
| 85 | .watermark = gv100_sor_dp_watermark, | 85 | .watermark = gv100_sor_dp_watermark, |
| @@ -91,7 +91,7 @@ tu104_sor = { · 5 unmodified lines | |||
| 91 | }; | 91 | }; |
| 92 | 92 | ||
| 93 | int | 93 | int |
| 94 | tu104_sor_new(struct nvkm_disp *disp, int id) | 94 | tu102_sor_new(struct nvkm_disp *disp, int id) |
| 95 | { | 95 | { |
| 96 | return nvkm_ior_new_(&tu104_sor, disp, SOR, id); | 96 | return nvkm_ior_new_(&tu102_sor, disp, SOR, id); |
| 97 | } | 97 | } |
| @@ -29,7 +29,7 @@ · 28 unmodified lines | |||
| 29 | #include <subdev/timer.h> | 29 | #include <subdev/timer.h> |
| 30 | 30 | ||
| 31 | static int | 31 | static int |
| 32 | tu104_disp_init(struct nv50_disp *disp) | 32 | tu102_disp_init(struct nv50_disp *disp) |
| 33 | { | 33 | { |
| 34 | struct nvkm_device *device = disp->base.engine.subdev.device; | 34 | struct nvkm_device *device = disp->base.engine.subdev.device; |
| 35 | struct nvkm_head *head; | 35 | struct nvkm_head *head; |
| @@ -132,21 +132,21 @@ tu104_disp_init(struct nv50_disp *disp) · 96 unmodified lines | |||
| 132 | } | 132 | } |
| 133 | 133 | ||
| 134 | static const struct nv50_disp_func | 134 | static const struct nv50_disp_func |
| 135 | tu104_disp = { | 135 | tu102_disp = { |
| 136 | .init = tu104_disp_init, | 136 | .init = tu102_disp_init, |
| 137 | .fini = gv100_disp_fini, | 137 | .fini = gv100_disp_fini, |
| 138 | .intr = gv100_disp_intr, | 138 | .intr = gv100_disp_intr, |
| 139 | .uevent = &gv100_disp_chan_uevent, | 139 | .uevent = &gv100_disp_chan_uevent, |
| 140 | .super = gv100_disp_super, | 140 | .super = gv100_disp_super, |
| 141 | .root = &tu104_disp_root_oclass, | 141 | .root = &tu102_disp_root_oclass, |
| 142 | .wndw = { .cnt = gv100_disp_wndw_cnt }, | 142 | .wndw = { .cnt = gv100_disp_wndw_cnt }, |
| 143 | .head = { .cnt = gv100_head_cnt, .new = gv100_head_new }, | 143 | .head = { .cnt = gv100_head_cnt, .new = gv100_head_new }, |
| 144 | .sor = { .cnt = gv100_sor_cnt, .new = tu104_sor_new }, | 144 | .sor = { .cnt = gv100_sor_cnt, .new = tu102_sor_new }, |
| 145 | .ramht_size = 0x2000, | 145 | .ramht_size = 0x2000, |
| 146 | }; | 146 | }; |
| 147 | 147 | ||
| 148 | int | 148 | int |
| 149 | tu104_disp_new(struct nvkm_device *device, int index, struct nvkm_disp **pdisp) | 149 | tu102_disp_new(struct nvkm_device *device, int index, struct nvkm_disp **pdisp) |
| 150 | { | 150 | { |
| 151 | return nv50_disp_new_(&tu104_disp, device, index, pdisp); | 151 | return nv50_disp_new_(&tu102_disp, device, index, pdisp); |
| 152 | } | 152 | } |
| @@ -16,7 +16,7 @@ nvkm-y += nvkm/engine/fifo/gm20b.o · 15 unmodified lines | |||
| 16 | nvkm-y += nvkm/engine/fifo/gp100.o | 16 | nvkm-y += nvkm/engine/fifo/gp100.o |
| 17 | nvkm-y += nvkm/engine/fifo/gp10b.o | 17 | nvkm-y += nvkm/engine/fifo/gp10b.o |
| 18 | nvkm-y += nvkm/engine/fifo/gv100.o | 18 | nvkm-y += nvkm/engine/fifo/gv100.o |
| 19 | nvkm-y += nvkm/engine/fifo/tu104.o | 19 | nvkm-y += nvkm/engine/fifo/tu102.o |
| 20 | 20 | ||
| 21 | nvkm-y += nvkm/engine/fifo/chan.o | 21 | nvkm-y += nvkm/engine/fifo/chan.o |
| 22 | nvkm-y += nvkm/engine/fifo/channv50.o | 22 | nvkm-y += nvkm/engine/fifo/channv50.o |
| @@ -34,7 +34,7 @@ nvkm-y += nvkm/engine/fifo/gpfifog84.o · 11 unmodified lines | |||
| 34 | nvkm-y += nvkm/engine/fifo/gpfifogf100.o | 34 | nvkm-y += nvkm/engine/fifo/gpfifogf100.o |
| 35 | nvkm-y += nvkm/engine/fifo/gpfifogk104.o | 35 | nvkm-y += nvkm/engine/fifo/gpfifogk104.o |
| 36 | nvkm-y += nvkm/engine/fifo/gpfifogv100.o | 36 | nvkm-y += nvkm/engine/fifo/gpfifogv100.o |
| 37 | nvkm-y += nvkm/engine/fifo/gpfifotu104.o | 37 | nvkm-y += nvkm/engine/fifo/gpfifotu102.o |
| 38 | 38 | ||
| 39 | nvkm-y += nvkm/engine/fifo/usergv100.o | 39 | nvkm-y += nvkm/engine/fifo/usergv100.o |
| 40 | nvkm-y += nvkm/engine/fifo/usertu104.o | 40 | nvkm-y += nvkm/engine/fifo/usertu102.o |
| @@ -47,6 +47,6 @@ int gv100_fifo_gpfifo_engine_init(struct nvkm_fifo_chan *, · 46 unmodified lines | |||
| 47 | int gv100_fifo_gpfifo_engine_fini(struct nvkm_fifo_chan *, | 47 | int gv100_fifo_gpfifo_engine_fini(struct nvkm_fifo_chan *, |
| 48 | struct nvkm_engine *, bool); | 48 | struct nvkm_engine *, bool); |
| 49 | 49 | ||
| 50 | int tu104_fifo_gpfifo_new(struct gk104_fifo *, const struct nvkm_oclass *, | 50 | int tu102_fifo_gpfifo_new(struct gk104_fifo *, const struct nvkm_oclass *, |
| 51 | void *data, u32 size, struct nvkm_object **); | 51 | void *data, u32 size, struct nvkm_object **); |
| 52 | #endif | 52 | #endif |
| @@ -29,14 +29,14 @@ · 28 unmodified lines | |||
| 29 | #include <nvif/unpack.h> | 29 | #include <nvif/unpack.h> |
| 30 | 30 | ||
| 31 | static u32 | 31 | static u32 |
| 32 | tu104_fifo_gpfifo_submit_token(struct nvkm_fifo_chan *base) | 32 | tu102_fifo_gpfifo_submit_token(struct nvkm_fifo_chan *base) |
| 33 | { | 33 | { |
| 34 | struct gk104_fifo_chan *chan = gk104_fifo_chan(base); | 34 | struct gk104_fifo_chan *chan = gk104_fifo_chan(base); |
| 35 | return (chan->runl << 16) | chan->base.chid; | 35 | return (chan->runl << 16) | chan->base.chid; |
| 36 | } | 36 | } |
| 37 | 37 | ||
| 38 | static const struct nvkm_fifo_chan_func | 38 | static const struct nvkm_fifo_chan_func |
| 39 | tu104_fifo_gpfifo = { | 39 | tu102_fifo_gpfifo = { |
| 40 | .dtor = gk104_fifo_gpfifo_dtor, | 40 | .dtor = gk104_fifo_gpfifo_dtor, |
| 41 | .init = gk104_fifo_gpfifo_init, | 41 | .init = gk104_fifo_gpfifo_init, |
| 42 | .fini = gk104_fifo_gpfifo_fini, | 42 | .fini = gk104_fifo_gpfifo_fini, |
| @@ -45,11 +45,11 @@ tu104_fifo_gpfifo = { · 2 unmodified lines | |||
| 45 | .engine_dtor = gk104_fifo_gpfifo_engine_dtor, | 45 | .engine_dtor = gk104_fifo_gpfifo_engine_dtor, |
| 46 | .engine_init = gv100_fifo_gpfifo_engine_init, | 46 | .engine_init = gv100_fifo_gpfifo_engine_init, |
| 47 | .engine_fini = gv100_fifo_gpfifo_engine_fini, | 47 | .engine_fini = gv100_fifo_gpfifo_engine_fini, |
| 48 | .submit_token = tu104_fifo_gpfifo_submit_token, | 48 | .submit_token = tu102_fifo_gpfifo_submit_token, |
| 49 | }; | 49 | }; |
| 50 | 50 | ||
| 51 | int | 51 | int |
| 52 | tu104_fifo_gpfifo_new(struct gk104_fifo *fifo, const struct nvkm_oclass *oclass, | 52 | tu102_fifo_gpfifo_new(struct gk104_fifo *fifo, const struct nvkm_oclass *oclass, |
| 53 | void *data, u32 size, struct nvkm_object **pobject) | 53 | void *data, u32 size, struct nvkm_object **pobject) |
| 54 | { | 54 | { |
| 55 | struct nvkm_object *parent = oclass->parent; | 55 | struct nvkm_object *parent = oclass->parent; |
| @@ -67,7 +67,7 @@ tu104_fifo_gpfifo_new(struct gk104_fifo *fifo, const struct nvkm_oclass *oclass, · 11 unmodified lines | |||
| 67 | args->v0.ilength, args->v0.runlist, args->v0.priv); | 67 | args->v0.ilength, args->v0.runlist, args->v0.priv); |
| 68 | if (args->v0.priv && !oclass->client->super) | 68 | if (args->v0.priv && !oclass->client->super) |
| 69 | return -EINVAL; | 69 | return -EINVAL; |
| 70 | return gv100_fifo_gpfifo_new_(&tu104_fifo_gpfifo, fifo, | 70 | return gv100_fifo_gpfifo_new_(&tu102_fifo_gpfifo, fifo, |
| 71 | &args->v0.runlist, | 71 | &args->v0.runlist, |
| 72 | &args->v0.chid, | 72 | &args->v0.chid, |
| 73 | args->v0.vmm, | 73 | args->v0.vmm, |
| @@ -29,7 +29,7 @@ · 28 unmodified lines | |||
| 29 | #include <nvif/class.h> | 29 | #include <nvif/class.h> |
| 30 | 30 | ||
| 31 | static void | 31 | static void |
| 32 | tu104_fifo_runlist_commit(struct gk104_fifo *fifo, int runl, | 32 | tu102_fifo_runlist_commit(struct gk104_fifo *fifo, int runl, |
| 33 | struct nvkm_memory *mem, int nr) | 33 | struct nvkm_memory *mem, int nr) |
| 34 | { | 34 | { |
| 35 | struct nvkm_device *device = fifo->base.engine.subdev.device; | 35 | struct nvkm_device *device = fifo->base.engine.subdev.device; |
| @@ -44,15 +44,15 @@ tu104_fifo_runlist_commit(struct gk104_fifo *fifo, int runl, · 8 unmodified lines | |||
| 44 | } | 44 | } |
| 45 | 45 | ||
| 46 | const struct gk104_fifo_runlist_func | 46 | const struct gk104_fifo_runlist_func |
| 47 | tu104_fifo_runlist = { | 47 | tu102_fifo_runlist = { |
| 48 | .size = 16, | 48 | .size = 16, |
| 49 | .cgrp = gv100_fifo_runlist_cgrp, | 49 | .cgrp = gv100_fifo_runlist_cgrp, |
| 50 | .chan = gv100_fifo_runlist_chan, | 50 | .chan = gv100_fifo_runlist_chan, |
| 51 | .commit = tu104_fifo_runlist_commit, | 51 | .commit = tu102_fifo_runlist_commit, |
| 52 | }; | 52 | }; |
| 53 | 53 | ||
| 54 | static const struct nvkm_enum | 54 | static const struct nvkm_enum |
| 55 | tu104_fifo_fault_engine[] = { | 55 | tu102_fifo_fault_engine[] = { |
| 56 | { 0x01, "DISPLAY" }, | 56 | { 0x01, "DISPLAY" }, |
| 57 | { 0x03, "PTP" }, | 57 | { 0x03, "PTP" }, |
| 58 | { 0x06, "PWR_PMU" }, | 58 | { 0x06, "PWR_PMU" }, |
| @@ -80,7 +80,7 @@ tu104_fifo_fault_engine[] = { · 21 unmodified lines | |||
| 80 | }; | 80 | }; |
| 81 | 81 | ||
| 82 | static void | 82 | static void |
| 83 | tu104_fifo_pbdma_init(struct gk104_fifo *fifo) | 83 | tu102_fifo_pbdma_init(struct gk104_fifo *fifo) |
| 84 | { | 84 | { |
| 85 | struct nvkm_device *device = fifo->base.engine.subdev.device; | 85 | struct nvkm_device *device = fifo->base.engine.subdev.device; |
| 86 | const u32 mask = (1 << fifo->pbdma_nr) - 1; | 86 | const u32 mask = (1 << fifo->pbdma_nr) - 1; |
| @@ -89,28 +89,28 @@ tu104_fifo_pbdma_init(struct gk104_fifo *fifo) · 2 unmodified lines | |||
| 89 | } | 89 | } |
| 90 | 90 | ||
| 91 | static const struct gk104_fifo_pbdma_func | 91 | static const struct gk104_fifo_pbdma_func |
| 92 | tu104_fifo_pbdma = { | 92 | tu102_fifo_pbdma = { |
| 93 | .nr = gm200_fifo_pbdma_nr, | 93 | .nr = gm200_fifo_pbdma_nr, |
| 94 | .init = tu104_fifo_pbdma_init, | 94 | .init = tu102_fifo_pbdma_init, |
| 95 | .init_timeout = gk208_fifo_pbdma_init_timeout, | 95 | .init_timeout = gk208_fifo_pbdma_init_timeout, |
| 96 | }; | 96 | }; |
| 97 | 97 | ||
| 98 | static const struct gk104_fifo_func | 98 | static const struct gk104_fifo_func |
| 99 | tu104_fifo = { | 99 | tu102_fifo = { |
| 100 | .pbdma = &tu104_fifo_pbdma, | 100 | .pbdma = &tu102_fifo_pbdma, |
| 101 | .fault.access = gv100_fifo_fault_access, | 101 | .fault.access = gv100_fifo_fault_access, |
| 102 | .fault.engine = tu104_fifo_fault_engine, | 102 | .fault.engine = tu102_fifo_fault_engine, |
| 103 | .fault.reason = gv100_fifo_fault_reason, | 103 | .fault.reason = gv100_fifo_fault_reason, |
| 104 | .fault.hubclient = gv100_fifo_fault_hubclient, | 104 | .fault.hubclient = gv100_fifo_fault_hubclient, |
| 105 | .fault.gpcclient = gv100_fifo_fault_gpcclient, | 105 | .fault.gpcclient = gv100_fifo_fault_gpcclient, |
| 106 | .runlist = &tu104_fifo_runlist, | 106 | .runlist = &tu102_fifo_runlist, |
| 107 | .user = {{-1,-1,VOLTA_USERMODE_A }, tu104_fifo_user_new }, | 107 | .user = {{-1,-1,VOLTA_USERMODE_A }, tu102_fifo_user_new }, |
| 108 | .chan = {{ 0, 0,TURING_CHANNEL_GPFIFO_A}, tu104_fifo_gpfifo_new }, | 108 | .chan = {{ 0, 0,TURING_CHANNEL_GPFIFO_A}, tu102_fifo_gpfifo_new }, |
| 109 | .cgrp_force = true, | 109 | .cgrp_force = true, |
| 110 | }; | 110 | }; |
| 111 | 111 | ||
| 112 | int | 112 | int |
| 113 | tu104_fifo_new(struct nvkm_device *device, int index, struct nvkm_fifo **pfifo) | 113 | tu102_fifo_new(struct nvkm_device *device, int index, struct nvkm_fifo **pfifo) |
| 114 | { | 114 | { |
| 115 | return gk104_fifo_new_(&tu104_fifo, device, index, 4096, pfifo); | 115 | return gk104_fifo_new_(&tu102_fifo, device, index, 4096, pfifo); |
| 116 | } | 116 | } |
| @@ -3,6 +3,6 @@ · 2 unmodified lines | |||
| 3 | #include "priv.h" | 3 | #include "priv.h" |
| 4 | int gv100_fifo_user_new(const struct nvkm_oclass *, void *, u32, | 4 | int gv100_fifo_user_new(const struct nvkm_oclass *, void *, u32, |
| 5 | struct nvkm_object **); | 5 | struct nvkm_object **); |
| 6 | int tu104_fifo_user_new(const struct nvkm_oclass *, void *, u32, | 6 | int tu102_fifo_user_new(const struct nvkm_oclass *, void *, u32, |
| 7 | struct nvkm_object **); | 7 | struct nvkm_object **); |
| 8 | #endif | 8 | #endif |
| @@ -22,7 +22,7 @@ · 21 unmodified lines | |||
| 22 | #include "user.h" | 22 | #include "user.h" |
| 23 | 23 | ||
| 24 | static int | 24 | static int |
| 25 | tu104_fifo_user_map(struct nvkm_object *object, void *argv, u32 argc, | 25 | tu102_fifo_user_map(struct nvkm_object *object, void *argv, u32 argc, |
| 26 | enum nvkm_object_map *type, u64 *addr, u64 *size) | 26 | enum nvkm_object_map *type, u64 *addr, u64 *size) |
| 27 | { | 27 | { |
| 28 | struct nvkm_device *device = object->engine->subdev.device; | 28 | struct nvkm_device *device = object->engine->subdev.device; |
| @@ -33,13 +33,13 @@ tu104_fifo_user_map(struct nvkm_object *object, void *argv, u32 argc, · 4 unmodified lines | |||
| 33 | } | 33 | } |
| 34 | 34 | ||
| 35 | static const struct nvkm_object_func | 35 | static const struct nvkm_object_func |
| 36 | tu104_fifo_user = { | 36 | tu102_fifo_user = { |
| 37 | .map = tu104_fifo_user_map, | 37 | .map = tu102_fifo_user_map, |
| 38 | }; | 38 | }; |
| 39 | 39 | ||
| 40 | int | 40 | int |
| 41 | tu104_fifo_user_new(const struct nvkm_oclass *oclass, void *argv, u32 argc, | 41 | tu102_fifo_user_new(const struct nvkm_oclass *oclass, void *argv, u32 argc, |
| 42 | struct nvkm_object **pobject) | 42 | struct nvkm_object **pobject) |
| 43 | { | 43 | { |
| 44 | return nvkm_object_new_(&tu104_fifo_user, oclass, argv, argc, pobject); | 44 | return nvkm_object_new_(&tu102_fifo_user, oclass, argv, argc, pobject); |
| 45 | } | 45 | } |
| @@ -25,6 +25,33 @@ · 24 unmodified lines | |||
| 25 | 25 | ||
| 26 | #include <engine/fifo.h> | 26 | #include <engine/fifo.h> |
| 27 | 27 | ||
| 28 | u32 | ||
| 29 | nvkm_gr_ctxsw_inst(struct nvkm_device *device) | ||
| 30 | { | ||
| 31 | struct nvkm_gr *gr = device->gr; | ||
| 32 | if (gr && gr->func->ctxsw.inst) | ||
| 33 | return gr->func->ctxsw.inst(gr); | ||
| 34 | return 0; | ||
| 35 | } | ||
| 36 | |||
| 37 | int | ||
| 38 | nvkm_gr_ctxsw_resume(struct nvkm_device *device) | ||
| 39 | { | ||
| 40 | struct nvkm_gr *gr = device->gr; | ||
| 41 | if (gr && gr->func->ctxsw.resume) | ||
| 42 | return gr->func->ctxsw.resume(gr); | ||
| 43 | return 0; | ||
| 44 | } | ||
| 45 | |||
| 46 | int | ||
| 47 | nvkm_gr_ctxsw_pause(struct nvkm_device *device) | ||
| 48 | { | ||
| 49 | struct nvkm_gr *gr = device->gr; | ||
| 50 | if (gr && gr->func->ctxsw.pause) | ||
| 51 | return gr->func->ctxsw.pause(gr); | ||
| 52 | return 0; | ||
| 53 | } | ||
| 54 | |||
| 28 | static bool | 55 | static bool |
| 29 | nvkm_gr_chsw_load(struct nvkm_engine *engine) | 56 | nvkm_gr_chsw_load(struct nvkm_engine *engine) |
| 30 | { | 57 | { |
| @@ -1523,13 +1523,9 @@ gf100_grctx_generate(struct gf100_gr *gr) · 1522 unmodified lines | |||
| 1523 | /* Make channel current. */ | 1523 | /* Make channel current. */ |
| 1524 | addr = nvkm_memory_addr(inst) >> 12; | 1524 | addr = nvkm_memory_addr(inst) >> 12; |
| 1525 | if (gr->firmware) { | 1525 | if (gr->firmware) { |
| 1526 | nvkm_wr32(device, 0x409840, 0x00000030); | 1526 | ret = gf100_gr_fecs_bind_pointer(gr, 0x80000000 | addr); |
| 1527 | nvkm_wr32(device, 0x409500, 0x80000000 | addr); | 1527 | if (ret) |
| 1528 | nvkm_wr32(device, 0x409504, 0x00000003); | 1528 | goto done; |
| 1529 | nvkm_msec(device, 2000, | ||
| 1530 | if (nvkm_rd32(device, 0x409800) & 0x00000010) | ||
| 1531 | break; | ||
| 1532 | ); | ||
| 1533 | 1529 | ||
| 1534 | nvkm_kmap(data); | 1530 | nvkm_kmap(data); |
| 1535 | nvkm_wo32(data, 0x1c, 1); | 1531 | nvkm_wo32(data, 0x1c, 1); |
| @@ -715,6 +715,211 @@ gf100_gr_pack_mmio[] = { · 714 unmodified lines | |||
| 715 | * PGRAPH engine/subdev functions | 715 | * PGRAPH engine/subdev functions |
| 716 | ******************************************************************************/ | 716 | ******************************************************************************/ |
| 717 | 717 | ||
| 718 | static u32 | ||
| 719 | gf100_gr_ctxsw_inst(struct nvkm_gr *gr) | ||
| 720 | { | ||
| 721 | return nvkm_rd32(gr->engine.subdev.device, 0x409b00); | ||
| 722 | } | ||
| 723 | |||
| 724 | static int | ||
| 725 | gf100_gr_fecs_ctrl_ctxsw(struct gf100_gr *gr, u32 mthd) | ||
| 726 | { | ||
| 727 | struct nvkm_device *device = gr->base.engine.subdev.device; | ||
| 728 | |||
| 729 | nvkm_wr32(device, 0x409804, 0xffffffff); | ||
| 730 | nvkm_wr32(device, 0x409840, 0xffffffff); | ||
| 731 | nvkm_wr32(device, 0x409500, 0xffffffff); | ||
| 732 | nvkm_wr32(device, 0x409504, mthd); | ||
| 733 | nvkm_msec(device, 2000, | ||
| 734 | u32 stat = nvkm_rd32(device, 0x409804); | ||
| 735 | if (stat == 0x00000002) | ||
| 736 | return -EIO; | ||
| 737 | if (stat == 0x00000001) | ||
| 738 | return 0; | ||
| 739 | ); | ||
| 740 | |||
| 741 | return -ETIMEDOUT; | ||
| 742 | } | ||
| 743 | |||
| 744 | int | ||
| 745 | gf100_gr_fecs_start_ctxsw(struct nvkm_gr *base) | ||
| 746 | { | ||
| 747 | struct gf100_gr *gr = gf100_gr(base); | ||
| 748 | int ret = 0; | ||
| 749 | |||
| 750 | mutex_lock(&gr->fecs.mutex); | ||
| 751 | if (!--gr->fecs.disable) { | ||
| 752 | if (WARN_ON(ret = gf100_gr_fecs_ctrl_ctxsw(gr, 0x39))) | ||
| 753 | gr->fecs.disable++; | ||
| 754 | } | ||
| 755 | mutex_unlock(&gr->fecs.mutex); | ||
| 756 | return ret; | ||
| 757 | } | ||
| 758 | |||
| 759 | int | ||
| 760 | gf100_gr_fecs_stop_ctxsw(struct nvkm_gr *base) | ||
| 761 | { | ||
| 762 | struct gf100_gr *gr = gf100_gr(base); | ||
| 763 | int ret = 0; | ||
| 764 | |||
| 765 | mutex_lock(&gr->fecs.mutex); | ||
| 766 | if (!gr->fecs.disable++) { | ||
| 767 | if (WARN_ON(ret = gf100_gr_fecs_ctrl_ctxsw(gr, 0x38))) | ||
| 768 | gr->fecs.disable--; | ||
| 769 | } | ||
| 770 | mutex_unlock(&gr->fecs.mutex); | ||
| 771 | return ret; | ||
| 772 | } | ||
| 773 | |||
| 774 | int | ||
| 775 | gf100_gr_fecs_bind_pointer(struct gf100_gr *gr, u32 inst) | ||
| 776 | { | ||
| 777 | struct nvkm_device *device = gr->base.engine.subdev.device; | ||
| 778 | |||
| 779 | nvkm_wr32(device, 0x409840, 0x00000030); | ||
| 780 | nvkm_wr32(device, 0x409500, inst); | ||
| 781 | nvkm_wr32(device, 0x409504, 0x00000003); | ||
| 782 | nvkm_msec(device, 2000, | ||
| 783 | u32 stat = nvkm_rd32(device, 0x409800); | ||
| 784 | if (stat & 0x00000020) | ||
| 785 | return -EIO; | ||
| 786 | if (stat & 0x00000010) | ||
| 787 | return 0; | ||
| 788 | ); | ||
| 789 | |||
| 790 | return -ETIMEDOUT; | ||
| 791 | } | ||
| 792 | |||
| 793 | static int | ||
| 794 | gf100_gr_fecs_set_reglist_virtual_address(struct gf100_gr *gr, u64 addr) | ||
| 795 | { | ||
| 796 | struct nvkm_device *device = gr->base.engine.subdev.device; | ||
| 797 | |||
| 798 | nvkm_wr32(device, 0x409810, addr >> 8); | ||
| 799 | nvkm_wr32(device, 0x409800, 0x00000000); | ||
| 800 | nvkm_wr32(device, 0x409500, 0x00000001); | ||
| 801 | nvkm_wr32(device, 0x409504, 0x00000032); | ||
| 802 | nvkm_msec(device, 2000, | ||
| 803 | if (nvkm_rd32(device, 0x409800) == 0x00000001) | ||
| 804 | return 0; | ||
| 805 | ); | ||
| 806 | |||
| 807 | return -ETIMEDOUT; | ||
| 808 | } | ||
| 809 | |||
| 810 | static int | ||
| 811 | gf100_gr_fecs_set_reglist_bind_instance(struct gf100_gr *gr, u32 inst) | ||
| 812 | { | ||
| 813 | struct nvkm_device *device = gr->base.engine.subdev.device; | ||
| 814 | |||
| 815 | nvkm_wr32(device, 0x409810, inst); | ||
| 816 | nvkm_wr32(device, 0x409800, 0x00000000); | ||
| 817 | nvkm_wr32(device, 0x409500, 0x00000001); | ||
| 818 | nvkm_wr32(device, 0x409504, 0x00000031); | ||
| 819 | nvkm_msec(device, 2000, | ||
| 820 | if (nvkm_rd32(device, 0x409800) == 0x00000001) | ||
| 821 | return 0; | ||
| 822 | ); | ||
| 823 | |||
| 824 | return -ETIMEDOUT; | ||
| 825 | } | ||
| 826 | |||
| 827 | static int | ||
| 828 | gf100_gr_fecs_discover_reglist_image_size(struct gf100_gr *gr, u32 *psize) | ||
| 829 | { | ||
| 830 | struct nvkm_device *device = gr->base.engine.subdev.device; | ||
| 831 | |||
| 832 | nvkm_wr32(device, 0x409800, 0x00000000); | ||
| 833 | nvkm_wr32(device, 0x409500, 0x00000001); | ||
| 834 | nvkm_wr32(device, 0x409504, 0x00000030); | ||
| 835 | nvkm_msec(device, 2000, | ||
| 836 | if ((*psize = nvkm_rd32(device, 0x409800))) | ||
| 837 | return 0; | ||
| 838 | ); | ||
| 839 | |||
| 840 | return -ETIMEDOUT; | ||
| 841 | } | ||
| 842 | |||
| 843 | static int | ||
| 844 | gf100_gr_fecs_elpg_bind(struct gf100_gr *gr) | ||
| 845 | { | ||
| 846 | u32 size; | ||
| 847 | int ret; | ||
| 848 | |||
| 849 | ret = gf100_gr_fecs_discover_reglist_image_size(gr, &size); | ||
| 850 | if (ret) | ||
| 851 | return ret; | ||
| 852 | |||
| 853 | /*XXX: We need to allocate + map the above into PMU's inst block, | ||
| 854 | * which which means we probably need a proper PMU before we | ||
| 855 | * even bother. | ||
| 856 | */ | ||
| 857 | |||
| 858 | ret = gf100_gr_fecs_set_reglist_bind_instance(gr, 0); | ||
| 859 | if (ret) | ||
| 860 | return ret; | ||
| 861 | |||
| 862 | return gf100_gr_fecs_set_reglist_virtual_address(gr, 0); | ||
| 863 | } | ||
| 864 | |||
| 865 | static int | ||
| 866 | gf100_gr_fecs_discover_pm_image_size(struct gf100_gr *gr, u32 *psize) | ||
| 867 | { | ||
| 868 | struct nvkm_device *device = gr->base.engine.subdev.device; | ||
| 869 | |||
| 870 | nvkm_wr32(device, 0x409840, 0xffffffff); | ||
| 871 | nvkm_wr32(device, 0x409500, 0x00000000); | ||
| 872 | nvkm_wr32(device, 0x409504, 0x00000025); | ||
| 873 | nvkm_msec(device, 2000, | ||
| 874 | if ((*psize = nvkm_rd32(device, 0x409800))) | ||
| 875 | return 0; | ||
| 876 | ); | ||
| 877 | |||
| 878 | return -ETIMEDOUT; | ||
| 879 | } | ||
| 880 | |||
| 881 | static int | ||
| 882 | gf100_gr_fecs_discover_zcull_image_size(struct gf100_gr *gr, u32 *psize) | ||
| 883 | { | ||
| 884 | struct nvkm_device *device = gr->base.engine.subdev.device; | ||
| 885 | |||
| 886 | nvkm_wr32(device, 0x409840, 0xffffffff); | ||
| 887 | nvkm_wr32(device, 0x409500, 0x00000000); | ||
| 888 | nvkm_wr32(device, 0x409504, 0x00000016); | ||
| 889 | nvkm_msec(device, 2000, | ||
| 890 | if ((*psize = nvkm_rd32(device, 0x409800))) | ||
| 891 | return 0; | ||
| 892 | ); | ||
| 893 | |||
| 894 | return -ETIMEDOUT; | ||
| 895 | } | ||
| 896 | |||
| 897 | static int | ||
| 898 | gf100_gr_fecs_discover_image_size(struct gf100_gr *gr, u32 *psize) | ||
| 899 | { | ||
| 900 | struct nvkm_device *device = gr->base.engine.subdev.device; | ||
| 901 | |||
| 902 | nvkm_wr32(device, 0x409840, 0xffffffff); | ||
| 903 | nvkm_wr32(device, 0x409500, 0x00000000); | ||
| 904 | nvkm_wr32(device, 0x409504, 0x00000010); | ||
| 905 | nvkm_msec(device, 2000, | ||
| 906 | if ((*psize = nvkm_rd32(device, 0x409800))) | ||
| 907 | return 0; | ||
| 908 | ); | ||
| 909 | |||
| 910 | return -ETIMEDOUT; | ||
| 911 | } | ||
| 912 | |||
| 913 | static void | ||
| 914 | gf100_gr_fecs_set_watchdog_timeout(struct gf100_gr *gr, u32 timeout) | ||
| 915 | { | ||
| 916 | struct nvkm_device *device = gr->base.engine.subdev.device; | ||
| 917 | |||
| 918 | nvkm_wr32(device, 0x409840, 0xffffffff); | ||
| 919 | nvkm_wr32(device, 0x409500, timeout); | ||
| 920 | nvkm_wr32(device, 0x409504, 0x00000021); | ||
| 921 | } | ||
| 922 | |||
| 718 | static bool | 923 | static bool |
| 719 | gf100_gr_chsw_load(struct nvkm_gr *base) | 924 | gf100_gr_chsw_load(struct nvkm_gr *base) |
| 720 | { | 925 | { |
| @@ -1487,6 +1692,7 @@ gf100_gr_init_ctxctl_ext(struct gf100_gr *gr) · 766 unmodified lines | |||
| 1487 | struct nvkm_device *device = subdev->device; | 1692 | struct nvkm_device *device = subdev->device; |
| 1488 | struct nvkm_secboot *sb = device->secboot; | 1693 | struct nvkm_secboot *sb = device->secboot; |
| 1489 | u32 secboot_mask = 0; | 1694 | u32 secboot_mask = 0; |
| 1695 | int ret; | ||
| 1490 | 1696 | ||
| 1491 | /* load fuc microcode */ | 1697 | /* load fuc microcode */ |
| 1492 | nvkm_mc_unk260(device, 0); | 1698 | nvkm_mc_unk260(device, 0); |
| @@ -1495,12 +1701,12 @@ gf100_gr_init_ctxctl_ext(struct gf100_gr *gr) · 2 unmodified lines | |||
| 1495 | if (nvkm_secboot_is_managed(sb, NVKM_SECBOOT_FALCON_FECS)) | 1701 | if (nvkm_secboot_is_managed(sb, NVKM_SECBOOT_FALCON_FECS)) |
| 1496 | secboot_mask |= BIT(NVKM_SECBOOT_FALCON_FECS); | 1702 | secboot_mask |= BIT(NVKM_SECBOOT_FALCON_FECS); |
| 1497 | else | 1703 | else |
| 1498 | gf100_gr_init_fw(gr->fecs, &gr->fuc409c, &gr->fuc409d); | 1704 | gf100_gr_init_fw(gr->fecs.falcon, &gr->fuc409c, &gr->fuc409d); |
| 1499 | 1705 | ||
| 1500 | if (nvkm_secboot_is_managed(sb, NVKM_SECBOOT_FALCON_GPCCS)) | 1706 | if (nvkm_secboot_is_managed(sb, NVKM_SECBOOT_FALCON_GPCCS)) |
| 1501 | secboot_mask |= BIT(NVKM_SECBOOT_FALCON_GPCCS); | 1707 | secboot_mask |= BIT(NVKM_SECBOOT_FALCON_GPCCS); |
| 1502 | else | 1708 | else |
| 1503 | gf100_gr_init_fw(gr->gpccs, &gr->fuc41ac, &gr->fuc41ad); | 1709 | gf100_gr_init_fw(gr->gpccs.falcon, &gr->fuc41ac, &gr->fuc41ad); |
| 1504 | 1710 | ||
| 1505 | if (secboot_mask != 0) { | 1711 | if (secboot_mask != 0) { |
| 1506 | int ret = nvkm_secboot_reset(sb, secboot_mask); | 1712 | int ret = nvkm_secboot_reset(sb, secboot_mask); |
| @@ -1515,8 +1721,8 @@ gf100_gr_init_ctxctl_ext(struct gf100_gr *gr) · 8 unmodified lines | |||
| 1515 | nvkm_wr32(device, 0x41a10c, 0x00000000); | 1721 | nvkm_wr32(device, 0x41a10c, 0x00000000); |
| 1516 | nvkm_wr32(device, 0x40910c, 0x00000000); | 1722 | nvkm_wr32(device, 0x40910c, 0x00000000); |
| 1517 | 1723 | ||
| 1518 | nvkm_falcon_start(gr->gpccs); | 1724 | nvkm_falcon_start(gr->gpccs.falcon); |
| 1519 | nvkm_falcon_start(gr->fecs); | 1725 | nvkm_falcon_start(gr->fecs.falcon); |
| 1520 | 1726 | ||
| 1521 | if (nvkm_msec(device, 2000, | 1727 | if (nvkm_msec(device, 2000, |
| 1522 | if (nvkm_rd32(device, 0x409800) & 0x00000001) | 1728 | if (nvkm_rd32(device, 0x409800) & 0x00000001) |
| @@ -1524,72 +1730,36 @@ gf100_gr_init_ctxctl_ext(struct gf100_gr *gr) · 1 unmodified line | |||
| 1524 | ) < 0) | 1730 | ) < 0) |
| 1525 | return -EBUSY; | 1731 | return -EBUSY; |
| 1526 | 1732 | ||
| 1527 | nvkm_wr32(device, 0x409840, 0xffffffff); | 1733 | gf100_gr_fecs_set_watchdog_timeout(gr, 0x7fffffff); |
| 1528 | nvkm_wr32(device, 0x409500, 0x7fffffff); | ||
| 1529 | nvkm_wr32(device, 0x409504, 0x00000021); | ||
| 1530 | 1734 | ||
| 1531 | nvkm_wr32(device, 0x409840, 0xffffffff); | 1735 | /* Determine how much memory is required to store main context image. */ |
| 1532 | nvkm_wr32(device, 0x409500, 0x00000000); | 1736 | ret = gf100_gr_fecs_discover_image_size(gr, &gr->size); |
| 1533 | nvkm_wr32(device, 0x409504, 0x00000010); | 1737 | if (ret) |
| 1534 | if (nvkm_msec(device, 2000, | 1738 | return ret; |
| 1535 | if ((gr->size = nvkm_rd32(device, 0x409800))) | ||
| 1536 | break; | ||
| 1537 | ) < 0) | ||
| 1538 | return -EBUSY; | ||
| 1539 | |||
| 1540 | nvkm_wr32(device, 0x409840, 0xffffffff); | ||
| 1541 | nvkm_wr32(device, 0x409500, 0x00000000); | ||
| 1542 | nvkm_wr32(device, 0x409504, 0x00000016); | ||
| 1543 | if (nvkm_msec(device, 2000, | ||
| 1544 | if (nvkm_rd32(device, 0x409800)) | ||
| 1545 | break; | ||
| 1546 | ) < 0) | ||
| 1547 | return -EBUSY; | ||
| 1548 | 1739 | ||
| 1549 | nvkm_wr32(device, 0x409840, 0xffffffff); | 1740 | /* Determine how much memory is required to store ZCULL image. */ |
| 1550 | nvkm_wr32(device, 0x409500, 0x00000000); | 1741 | ret = gf100_gr_fecs_discover_zcull_image_size(gr, &gr->size_zcull); |
| 1551 | nvkm_wr32(device, 0x409504, 0x00000025); | 1742 | if (ret) |
| 1552 | if (nvkm_msec(device, 2000, | 1743 | return ret; |
| 1553 | if (nvkm_rd32(device, 0x409800)) | ||
| 1554 | break; | ||
| 1555 | ) < 0) | ||
| 1556 | return -EBUSY; | ||
| 1557 | 1744 | ||
| 1558 | if (device->chipset >= 0xe0) { | 1745 | /* Determine how much memory is required to store PerfMon image. */ |
| 1559 | nvkm_wr32(device, 0x409800, 0x00000000); | 1746 | ret = gf100_gr_fecs_discover_pm_image_size(gr, &gr->size_pm); |
| 1560 | nvkm_wr32(device, 0x409500, 0x00000001); | 1747 | if (ret) |
| 1561 | nvkm_wr32(device, 0x409504, 0x00000030); | 1748 | return ret; |
| 1562 | if (nvkm_msec(device, 2000, | ||
| 1563 | if (nvkm_rd32(device, 0x409800)) | ||
| 1564 | break; | ||
| 1565 | ) < 0) | ||
| 1566 | return -EBUSY; | ||
| 1567 | |||
| 1568 | nvkm_wr32(device, 0x409810, 0xb00095c8); | ||
| 1569 | nvkm_wr32(device, 0x409800, 0x00000000); | ||
| 1570 | nvkm_wr32(device, 0x409500, 0x00000001); | ||
| 1571 | nvkm_wr32(device, 0x409504, 0x00000031); | ||
| 1572 | if (nvkm_msec(device, 2000, | ||
| 1573 | if (nvkm_rd32(device, 0x409800)) | ||
| 1574 | break; | ||
| 1575 | ) < 0) | ||
| 1576 | return -EBUSY; | ||
| 1577 | |||
| 1578 | nvkm_wr32(device, 0x409810, 0x00080420); | ||
| 1579 | nvkm_wr32(device, 0x409800, 0x00000000); | ||
| 1580 | nvkm_wr32(device, 0x409500, 0x00000001); | ||
| 1581 | nvkm_wr32(device, 0x409504, 0x00000032); | ||
| 1582 | if (nvkm_msec(device, 2000, | ||
| 1583 | if (nvkm_rd32(device, 0x409800)) | ||
| 1584 | break; | ||
| 1585 | ) < 0) | ||
| 1586 | return -EBUSY; | ||
| 1587 | 1749 | ||
| 1588 | nvkm_wr32(device, 0x409614, 0x00000070); | 1750 | /*XXX: We (likely) require PMU support to even bother with this. |
| 1589 | nvkm_wr32(device, 0x409614, 0x00000770); | 1751 | * |
| 1590 | nvkm_wr32(device, 0x40802c, 0x00000001); | 1752 | * Also, it seems like not all GPUs support ELPG. Traces I |
| 1753 | * have here show RM enabling it on Kepler/Turing, but none | ||
| 1754 | * of the GPUs between those. NVGPU decides this by PCIID. | ||
| 1755 | */ | ||
| 1756 | if (0) { | ||
| 1757 | ret = gf100_gr_fecs_elpg_bind(gr); | ||
| 1758 | if (ret) | ||
| 1759 | return ret; | ||
| 1591 | } | 1760 | } |
| 1592 | 1761 | ||
| 1762 | /* Generate golden context image. */ | ||
| 1593 | if (gr->data == NULL) { | 1763 | if (gr->data == NULL) { |
| 1594 | int ret = gf100_grctx_generate(gr); | 1764 | int ret = gf100_grctx_generate(gr); |
| 1595 | if (ret) { | 1765 | if (ret) { |
| @@ -1614,15 +1784,19 @@ gf100_gr_init_ctxctl_int(struct gf100_gr *gr) · 18 unmodified lines | |||
| 1614 | 1784 | ||
| 1615 | /* load HUB microcode */ | 1785 | /* load HUB microcode */ |
| 1616 | nvkm_mc_unk260(device, 0); | 1786 | nvkm_mc_unk260(device, 0); |
| 1617 | nvkm_falcon_load_dmem(gr->fecs, gr->func->fecs.ucode->data.data, 0x0, | 1787 | nvkm_falcon_load_dmem(gr->fecs.falcon, |
| 1788 | gr->func->fecs.ucode->data.data, 0x0, | ||
| 1618 | gr->func->fecs.ucode->data.size, 0); | 1789 | gr->func->fecs.ucode->data.size, 0); |
| 1619 | nvkm_falcon_load_imem(gr->fecs, gr->func->fecs.ucode->code.data, 0x0, | 1790 | nvkm_falcon_load_imem(gr->fecs.falcon, |
| 1791 | gr->func->fecs.ucode->code.data, 0x0, | ||
| 1620 | gr->func->fecs.ucode->code.size, 0, 0, false); | 1792 | gr->func->fecs.ucode->code.size, 0, 0, false); |
| 1621 | 1793 | ||
| 1622 | /* load GPC microcode */ | 1794 | /* load GPC microcode */ |
| 1623 | nvkm_falcon_load_dmem(gr->gpccs, gr->func->gpccs.ucode->data.data, 0x0, | 1795 | nvkm_falcon_load_dmem(gr->gpccs.falcon, |
| 1796 | gr->func->gpccs.ucode->data.data, 0x0, | ||
| 1624 | gr->func->gpccs.ucode->data.size, 0); | 1797 | gr->func->gpccs.ucode->data.size, 0); |
| 1625 | nvkm_falcon_load_imem(gr->gpccs, gr->func->gpccs.ucode->code.data, 0x0, | 1798 | nvkm_falcon_load_imem(gr->gpccs.falcon, |
| 1799 | gr->func->gpccs.ucode->code.data, 0x0, | ||
| 1626 | gr->func->gpccs.ucode->code.size, 0, 0, false); | 1800 | gr->func->gpccs.ucode->code.size, 0, 0, false); |
| 1627 | nvkm_mc_unk260(device, 1); | 1801 | nvkm_mc_unk260(device, 1); |
| 1628 | 1802 | ||
| @@ -1769,11 +1943,13 @@ gf100_gr_oneinit(struct nvkm_gr *base) · 140 unmodified lines | |||
| 1769 | int i, j; | 1943 | int i, j; |
| 1770 | int ret; | 1944 | int ret; |
| 1771 | 1945 | ||
| 1772 | ret = nvkm_falcon_v1_new(subdev, "FECS", 0x409000, &gr->fecs); | 1946 | ret = nvkm_falcon_v1_new(subdev, "FECS", 0x409000, &gr->fecs.falcon); |
| 1773 | if (ret) | 1947 | if (ret) |
| 1774 | return ret; | 1948 | return ret; |
| 1775 | 1949 | ||
| 1776 | ret = nvkm_falcon_v1_new(subdev, "GPCCS", 0x41a000, &gr->gpccs); | 1950 | mutex_init(&gr->fecs.mutex); |
| 1951 | |||
| 1952 | ret = nvkm_falcon_v1_new(subdev, "GPCCS", 0x41a000, &gr->gpccs.falcon); | ||
| 1777 | if (ret) | 1953 | if (ret) |
| 1778 | return ret; | 1954 | return ret; |
| 1779 | 1955 | ||
| @@ -1816,11 +1992,11 @@ gf100_gr_init_(struct nvkm_gr *base) · 36 unmodified lines | |||
| 1816 | 1992 | ||
| 1817 | nvkm_pmu_pgob(gr->base.engine.subdev.device->pmu, false); | 1993 | nvkm_pmu_pgob(gr->base.engine.subdev.device->pmu, false); |
| 1818 | 1994 | ||
| 1819 | ret = nvkm_falcon_get(gr->fecs, subdev); | 1995 | ret = nvkm_falcon_get(gr->fecs.falcon, subdev); |
| 1820 | if (ret) | 1996 | if (ret) |
| 1821 | return ret; | 1997 | return ret; |
| 1822 | 1998 | ||
| 1823 | ret = nvkm_falcon_get(gr->gpccs, subdev); | 1999 | ret = nvkm_falcon_get(gr->gpccs.falcon, subdev); |
| 1824 | if (ret) | 2000 | if (ret) |
| 1825 | return ret; | 2001 | return ret; |
| 1826 | 2002 | ||
| @@ -1832,8 +2008,8 @@ gf100_gr_fini_(struct nvkm_gr *base, bool suspend) · 5 unmodified lines | |||
| 1832 | { | 2008 | { |
| 1833 | struct gf100_gr *gr = gf100_gr(base); | 2009 | struct gf100_gr *gr = gf100_gr(base); |
| 1834 | struct nvkm_subdev *subdev = &gr->base.engine.subdev; | 2010 | struct nvkm_subdev *subdev = &gr->base.engine.subdev; |
| 1835 | nvkm_falcon_put(gr->gpccs, subdev); | 2011 | nvkm_falcon_put(gr->gpccs.falcon, subdev); |
| 1836 | nvkm_falcon_put(gr->fecs, subdev); | 2012 | nvkm_falcon_put(gr->fecs.falcon, subdev); |
| 1837 | return 0; | 2013 | return 0; |
| 1838 | } | 2014 | } |
| 1839 | 2015 | ||
| @@ -1859,8 +2035,8 @@ gf100_gr_dtor(struct nvkm_gr *base) · 19 unmodified lines | |||
| 1859 | gr->func->dtor(gr); | 2035 | gr->func->dtor(gr); |
| 1860 | kfree(gr->data); | 2036 | kfree(gr->data); |
| 1861 | 2037 | ||
| 1862 | nvkm_falcon_del(&gr->gpccs); | 2038 | nvkm_falcon_del(&gr->gpccs.falcon); |
| 1863 | nvkm_falcon_del(&gr->fecs); | 2039 | nvkm_falcon_del(&gr->fecs.falcon); |
| 1864 | 2040 | ||
| 1865 | gf100_gr_dtor_fw(&gr->fuc409c); | 2041 | gf100_gr_dtor_fw(&gr->fuc409c); |
| 1866 | gf100_gr_dtor_fw(&gr->fuc409d); | 2042 | gf100_gr_dtor_fw(&gr->fuc409d); |
| @@ -1886,6 +2062,9 @@ gf100_gr_ = { · 19 unmodified lines | |||
| 1886 | .chan_new = gf100_gr_chan_new, | 2062 | .chan_new = gf100_gr_chan_new, |
| 1887 | .object_get = gf100_gr_object_get, | 2063 | .object_get = gf100_gr_object_get, |
| 1888 | .chsw_load = gf100_gr_chsw_load, | 2064 | .chsw_load = gf100_gr_chsw_load, |
| 2065 | .ctxsw.pause = gf100_gr_fecs_stop_ctxsw, | ||
| 2066 | .ctxsw.resume = gf100_gr_fecs_start_ctxsw, | ||
| 2067 | .ctxsw.inst = gf100_gr_ctxsw_inst, | ||
| 1889 | }; | 2068 | }; |
| 1890 | 2069 | ||
| 1891 | int | 2070 | int |
| @@ -82,8 +82,16 @@ struct gf100_gr { · 81 unmodified lines | |||
| 82 | const struct gf100_gr_func *func; | 82 | const struct gf100_gr_func *func; |
| 83 | struct nvkm_gr base; | 83 | struct nvkm_gr base; |
| 84 | 84 | ||
| 85 | struct nvkm_falcon *fecs; | 85 | struct { |
| 86 | struct nvkm_falcon *gpccs; | 86 | struct nvkm_falcon *falcon; |
| 87 | struct mutex mutex; | ||
| 88 | u32 disable; | ||
| 89 | } fecs; | ||
| 90 | |||
| 91 | struct { | ||
| 92 | struct nvkm_falcon *falcon; | ||
| 93 | } gpccs; | ||
| 94 | |||
| 87 | struct gf100_gr_fuc fuc409c; | 95 | struct gf100_gr_fuc fuc409c; |
| 88 | struct gf100_gr_fuc fuc409d; | 96 | struct gf100_gr_fuc fuc409d; |
| 89 | struct gf100_gr_fuc fuc41ac; | 97 | struct gf100_gr_fuc fuc41ac; |
| @@ -128,6 +136,8 @@ struct gf100_gr { · 38 unmodified lines | |||
| 128 | struct gf100_gr_mmio mmio_list[4096/8]; | 136 | struct gf100_gr_mmio mmio_list[4096/8]; |
| 129 | u32 size; | 137 | u32 size; |
| 130 | u32 *data; | 138 | u32 *data; |
| 139 | u32 size_zcull; | ||
| 140 | u32 size_pm; | ||
| 131 | }; | 141 | }; |
| 132 | 142 | ||
| 133 | int gf100_gr_ctor(const struct gf100_gr_func *, struct nvkm_device *, | 143 | int gf100_gr_ctor(const struct gf100_gr_func *, struct nvkm_device *, |
| @@ -136,6 +146,8 @@ int gf100_gr_new_(const struct gf100_gr_func *, struct nvkm_device *, · 2 unmodified lines | |||
| 136 | int, struct nvkm_gr **); | 146 | int, struct nvkm_gr **); |
| 137 | void *gf100_gr_dtor(struct nvkm_gr *); | 147 | void *gf100_gr_dtor(struct nvkm_gr *); |
| 138 | 148 | ||
| 149 | int gf100_gr_fecs_bind_pointer(struct gf100_gr *, u32 inst); | ||
| 150 | |||
| 139 | struct gf100_gr_func_zbc { | 151 | struct gf100_gr_func_zbc { |
| 140 | void (*clear_color)(struct gf100_gr *, int zbc); | 152 | void (*clear_color)(struct gf100_gr *, int zbc); |
| 141 | void (*clear_depth)(struct gf100_gr *, int zbc); | 153 | void (*clear_depth)(struct gf100_gr *, int zbc); |
| @@ -27,6 +27,11 @@ struct nvkm_gr_func { · 26 unmodified lines | |||
| 27 | */ | 27 | */ |
| 28 | u64 (*units)(struct nvkm_gr *); | 28 | u64 (*units)(struct nvkm_gr *); |
| 29 | bool (*chsw_load)(struct nvkm_gr *); | 29 | bool (*chsw_load)(struct nvkm_gr *); |
| 30 | struct { | ||
| 31 | int (*pause)(struct nvkm_gr *); | ||
| 32 | int (*resume)(struct nvkm_gr *); | ||
| 33 | u32 (*inst)(struct nvkm_gr *); | ||
| 34 | } ctxsw; | ||
| 30 | struct nvkm_sclass sclass[]; | 35 | struct nvkm_sclass sclass[]; |
| 31 | }; | 36 | }; |
| 32 | 37 | ||
| @@ -21,13 +21,21 @@ · 20 unmodified lines | |||
| 21 | */ | 21 | */ |
| 22 | #include "priv.h" | 22 | #include "priv.h" |
| 23 | 23 | ||
| 24 | #include <subdev/top.h> | ||
| 24 | #include <engine/falcon.h> | 25 | #include <engine/falcon.h> |
| 25 | 26 | ||
| 26 | static int | 27 | static int |
| 27 | nvkm_nvdec_oneinit(struct nvkm_engine *engine) | 28 | nvkm_nvdec_oneinit(struct nvkm_engine *engine) |
| 28 | { | 29 | { |
| 29 | struct nvkm_nvdec *nvdec = nvkm_nvdec(engine); | 30 | struct nvkm_nvdec *nvdec = nvkm_nvdec(engine); |
| 30 | return nvkm_falcon_v1_new(&nvdec->engine.subdev, "NVDEC", 0x84000, | 31 | struct nvkm_subdev *subdev = &nvdec->engine.subdev; |
| 32 | |||
| 33 | nvdec->addr = nvkm_top_addr(subdev->device, subdev->index); | ||
| 34 | if (!nvdec->addr) | ||
| 35 | return -EINVAL; | ||
| 36 | |||
| 37 | /*XXX: fix naming of this when adding support for multiple-NVDEC */ | ||
| 38 | return nvkm_falcon_v1_new(subdev, "NVDEC", nvdec->addr, | ||
| 31 | &nvdec->falcon); | 39 | &nvdec->falcon); |
| 32 | } | 40 | } |
| 33 | 41 | ||
| @@ -1,2 +1,3 @@ | |||
| 1 | nvkm-y += nvkm/engine/sec2/base.o | 1 | nvkm-y += nvkm/engine/sec2/base.o |
| 2 | nvkm-y += nvkm/engine/sec2/gp102.o | 2 | nvkm-y += nvkm/engine/sec2/gp102.o |
| 3 | nvkm-y += nvkm/engine/sec2/tu102.o | ||
| @@ -22,6 +22,7 @@ · 21 unmodified lines | |||
| 22 | #include "priv.h" | 22 | #include "priv.h" |
| 23 | 23 | ||
| 24 | #include <core/msgqueue.h> | 24 | #include <core/msgqueue.h> |
| 25 | #include <subdev/top.h> | ||
| 25 | #include <engine/falcon.h> | 26 | #include <engine/falcon.h> |
| 26 | 27 | ||
| 27 | static void * | 28 | static void * |
| @@ -39,18 +40,18 @@ nvkm_sec2_intr(struct nvkm_engine *engine) · 11 unmodified lines | |||
| 39 | struct nvkm_sec2 *sec2 = nvkm_sec2(engine); | 40 | struct nvkm_sec2 *sec2 = nvkm_sec2(engine); |
| 40 | struct nvkm_subdev *subdev = &engine->subdev; | 41 | struct nvkm_subdev *subdev = &engine->subdev; |
| 41 | struct nvkm_device *device = subdev->device; | 42 | struct nvkm_device *device = subdev->device; |
| 42 | u32 disp = nvkm_rd32(device, 0x8701c); | 43 | u32 disp = nvkm_rd32(device, sec2->addr + 0x01c); |
| 43 | u32 intr = nvkm_rd32(device, 0x87008) & disp & ~(disp >> 16); | 44 | u32 intr = nvkm_rd32(device, sec2->addr + 0x008) & disp & ~(disp >> 16); |
| 44 | 45 | ||
| 45 | if (intr & 0x00000040) { | 46 | if (intr & 0x00000040) { |
| 46 | schedule_work(&sec2->work); | 47 | schedule_work(&sec2->work); |
| 47 | nvkm_wr32(device, 0x87004, 0x00000040); | 48 | nvkm_wr32(device, sec2->addr + 0x004, 0x00000040); |
| 48 | intr &= ~0x00000040; | 49 | intr &= ~0x00000040; |
| 49 | } | 50 | } |
| 50 | 51 | ||
| 51 | if (intr) { | 52 | if (intr) { |
| 52 | nvkm_error(subdev, "unhandled intr %08x\n", intr); | 53 | nvkm_error(subdev, "unhandled intr %08x\n", intr); |
| 53 | nvkm_wr32(device, 0x87004, intr); | 54 | nvkm_wr32(device, sec2->addr + 0x004, intr); |
| 54 | 55 | ||
| 55 | } | 56 | } |
| 56 | } | 57 | } |
| @@ -74,8 +75,15 @@ static int · 17 unmodified lines | |||
| 74 | nvkm_sec2_oneinit(struct nvkm_engine *engine) | 75 | nvkm_sec2_oneinit(struct nvkm_engine *engine) |
| 75 | { | 76 | { |
| 76 | struct nvkm_sec2 *sec2 = nvkm_sec2(engine); | 77 | struct nvkm_sec2 *sec2 = nvkm_sec2(engine); |
| 77 | return nvkm_falcon_v1_new(&sec2->engine.subdev, "SEC2", 0x87000, | 78 | struct nvkm_subdev *subdev = &sec2->engine.subdev; |
| 78 | &sec2->falcon); | 79 | |
| 80 | if (!sec2->addr) { | ||
| 81 | sec2->addr = nvkm_top_addr(subdev->device, subdev->index); | ||
| 82 | if (WARN_ON(!sec2->addr)) | ||
| 83 | return -EINVAL; | ||
| 84 | } | ||
| 85 | |||
| 86 | return nvkm_falcon_v1_new(subdev, "SEC2", sec2->addr, &sec2->falcon); | ||
| 79 | } | 87 | } |
| 80 | 88 | ||
| 81 | static int | 89 | static int |
| @@ -95,13 +103,14 @@ nvkm_sec2 = { · 13 unmodified lines | |||
| 95 | }; | 103 | }; |
| 96 | 104 | ||
| 97 | int | 105 | int |
| 98 | nvkm_sec2_new_(struct nvkm_device *device, int index, | 106 | nvkm_sec2_new_(struct nvkm_device *device, int index, u32 addr, |
| 99 | struct nvkm_sec2 **psec2) | 107 | struct nvkm_sec2 **psec2) |
| 100 | { | 108 | { |
| 101 | struct nvkm_sec2 *sec2; | 109 | struct nvkm_sec2 *sec2; |
| 102 | 110 | ||
| 103 | if (!(sec2 = *psec2 = kzalloc(sizeof(*sec2), GFP_KERNEL))) | 111 | if (!(sec2 = *psec2 = kzalloc(sizeof(*sec2), GFP_KERNEL))) |
| 104 | return -ENOMEM; | 112 | return -ENOMEM; |
| 113 | sec2->addr = addr; | ||
| 105 | INIT_WORK(&sec2->work, nvkm_sec2_recv); | 114 | INIT_WORK(&sec2->work, nvkm_sec2_recv); |
| 106 | 115 | ||
| 107 | return nvkm_engine_ctor(&nvkm_sec2, device, index, true, &sec2->engine); | 116 | return nvkm_engine_ctor(&nvkm_sec2, device, index, true, &sec2->engine); |
| @@ -26,5 +26,5 @@ int · 25 unmodified lines | |||
| 26 | gp102_sec2_new(struct nvkm_device *device, int index, | 26 | gp102_sec2_new(struct nvkm_device *device, int index, |
| 27 | struct nvkm_sec2 **psec2) | 27 | struct nvkm_sec2 **psec2) |
| 28 | { | 28 | { |
| 29 | return nvkm_sec2_new_(device, index, psec2); | 29 | return nvkm_sec2_new_(device, index, 0, psec2); |
| 30 | } | 30 | } |
| @@ -5,6 +5,5 @@ · 4 unmodified lines | |||
| 5 | 5 | ||
| 6 | #define nvkm_sec2(p) container_of((p), struct nvkm_sec2, engine) | 6 | #define nvkm_sec2(p) container_of((p), struct nvkm_sec2, engine) |
| 7 | 7 | ||
| 8 | int nvkm_sec2_new_(struct nvkm_device *, int, struct nvkm_sec2 **); | 8 | int nvkm_sec2_new_(struct nvkm_device *, int, u32 addr, struct nvkm_sec2 **); |
| 9 | |||
| 10 | #endif | 9 | #endif |
| @@ -0,0 +1,33 @@ | |||
| 1 | /* | ||
| 2 | * Copyright 2019 Red Hat Inc. | ||
| 3 | * | ||
| 4 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 5 | * copy of this software and associated documentation files (the "Software"), | ||
| 6 | * to deal in the Software without restriction, including without limitation | ||
| 7 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 8 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 9 | * Software is furnished to do so, subject to the following conditions: | ||
| 10 | * | ||
| 11 | * The above copyright notice and this permission notice shall be included in | ||
| 12 | * all copies or substantial portions of the Software. | ||
| 13 | * | ||
| 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 17 | * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | ||
| 18 | * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | ||
| 19 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | ||
| 20 | * OTHER DEALINGS IN THE SOFTWARE. | ||
| 21 | */ | ||
| 22 | |||
| 23 | #include "priv.h" | ||
| 24 | |||
| 25 | int | ||
| 26 | tu102_sec2_new(struct nvkm_device *device, int index, | ||
| 27 | struct nvkm_sec2 **psec2) | ||
| 28 | { | ||
| 29 | /* TOP info wasn't updated on Turing to reflect the PRI | ||
| 30 | * address change for some reason. We override it here. | ||
| 31 | */ | ||
| 32 | return nvkm_sec2_new_(device, index, 0x840000, psec2); | ||
| 33 | } | ||
| @@ -204,6 +204,9 @@ nvkm_falcon_ctor(const struct nvkm_falcon_func *func, · 203 unmodified lines | |||
| 204 | debug_reg = 0x408; | 204 | debug_reg = 0x408; |
| 205 | falcon->has_emem = true; | 205 | falcon->has_emem = true; |
| 206 | break; | 206 | break; |
| 207 | case NVKM_SUBDEV_GSP: | ||
| 208 | debug_reg = 0x0; /*XXX*/ | ||
| 209 | break; | ||
| 207 | default: | 210 | default: |
| 208 | nvkm_warn(subdev, "unsupported falcon %s!\n", | 211 | nvkm_warn(subdev, "unsupported falcon %s!\n", |
| 209 | nvkm_subdev_name[subdev->index]); | 212 | nvkm_subdev_name[subdev->index]); |
| @@ -269,7 +269,7 @@ cmd_write(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_hdr *cmd, · 268 unmodified lines | |||
| 269 | commit = false; | 269 | commit = false; |
| 270 | } | 270 | } |
| 271 | 271 | ||
| 272 | cmd_queue_close(priv, queue, commit); | 272 | cmd_queue_close(priv, queue, commit); |
| 273 | 273 | ||
| 274 | return ret; | 274 | return ret; |
| 275 | } | 275 | } |
| @@ -347,7 +347,7 @@ nvkm_msgqueue_post(struct nvkm_msgqueue *priv, enum msgqueue_msg_priority prio, · 71 unmodified lines | |||
| 347 | ret = cmd_write(priv, cmd, queue); | 347 | ret = cmd_write(priv, cmd, queue); |
| 348 | if (ret) { | 348 | if (ret) { |
| 349 | seq->state = SEQ_STATE_PENDING; | 349 | seq->state = SEQ_STATE_PENDING; |
| 350 | msgqueue_seq_release(priv, seq); | 350 | msgqueue_seq_release(priv, seq); |
| 351 | } | 351 | } |
| 352 | 352 | ||
| 353 | return ret; | 353 | return ret; |
| @@ -373,7 +373,7 @@ msgqueue_msg_handle(struct nvkm_msgqueue *priv, struct nvkm_msgqueue_hdr *hdr) · 19 unmodified lines | |||
| 373 | if (seq->completion) | 373 | if (seq->completion) |
| 374 | complete(seq->completion); | 374 | complete(seq->completion); |
| 375 | 375 | ||
| 376 | msgqueue_seq_release(priv, seq); | 376 | msgqueue_seq_release(priv, seq); |
| 377 | 377 | ||
| 378 | return 0; | 378 | return 0; |
| 379 | } | 379 | } |
| @@ -7,6 +7,7 @@ include $(src)/nvkm/subdev/fault/Kbuild · 6 unmodified lines | |||
| 7 | include $(src)/nvkm/subdev/fb/Kbuild | 7 | include $(src)/nvkm/subdev/fb/Kbuild |
| 8 | include $(src)/nvkm/subdev/fuse/Kbuild | 8 | include $(src)/nvkm/subdev/fuse/Kbuild |
| 9 | include $(src)/nvkm/subdev/gpio/Kbuild | 9 | include $(src)/nvkm/subdev/gpio/Kbuild |
| 10 | include $(src)/nvkm/subdev/gsp/Kbuild | ||
| 10 | include $(src)/nvkm/subdev/i2c/Kbuild | 11 | include $(src)/nvkm/subdev/i2c/Kbuild |
| 11 | include $(src)/nvkm/subdev/ibus/Kbuild | 12 | include $(src)/nvkm/subdev/ibus/Kbuild |
| 12 | include $(src)/nvkm/subdev/iccsense/Kbuild | 13 | include $(src)/nvkm/subdev/iccsense/Kbuild |
| @@ -5,4 +5,4 @@ nvkm-y += nvkm/subdev/bar/gf100.o · 4 unmodified lines | |||
| 5 | nvkm-y += nvkm/subdev/bar/gk20a.o | 5 | nvkm-y += nvkm/subdev/bar/gk20a.o |
| 6 | nvkm-y += nvkm/subdev/bar/gm107.o | 6 | nvkm-y += nvkm/subdev/bar/gm107.o |
| 7 | nvkm-y += nvkm/subdev/bar/gm20b.o | 7 | nvkm-y += nvkm/subdev/bar/gm20b.o |
| 8 | nvkm-y += nvkm/subdev/bar/tu104.o | 8 | nvkm-y += nvkm/subdev/bar/tu102.o |
| @@ -25,7 +25,7 @@ · 24 unmodified lines | |||
| 25 | #include <subdev/timer.h> | 25 | #include <subdev/timer.h> |
| 26 | 26 | ||
| 27 | static void | 27 | static void |
| 28 | tu104_bar_bar2_wait(struct nvkm_bar *bar) | 28 | tu102_bar_bar2_wait(struct nvkm_bar *bar) |
| 29 | { | 29 | { |
| 30 | struct nvkm_device *device = bar->subdev.device; | 30 | struct nvkm_device *device = bar->subdev.device; |
| 31 | nvkm_msec(device, 2000, | 31 | nvkm_msec(device, 2000, |
| @@ -35,13 +35,13 @@ tu104_bar_bar2_wait(struct nvkm_bar *bar) · 3 unmodified lines | |||
| 35 | } | 35 | } |
| 36 | 36 | ||
| 37 | static void | 37 | static void |
| 38 | tu104_bar_bar2_fini(struct nvkm_bar *bar) | 38 | tu102_bar_bar2_fini(struct nvkm_bar *bar) |
| 39 | { | 39 | { |
| 40 | nvkm_mask(bar->subdev.device, 0xb80f48, 0x80000000, 0x00000000); | 40 | nvkm_mask(bar->subdev.device, 0xb80f48, 0x80000000, 0x00000000); |
| 41 | } | 41 | } |
| 42 | 42 | ||
| 43 | static void | 43 | static void |
| 44 | tu104_bar_bar2_init(struct nvkm_bar *base) | 44 | tu102_bar_bar2_init(struct nvkm_bar *base) |
| 45 | { | 45 | { |
| 46 | struct nvkm_device *device = base->subdev.device; | 46 | struct nvkm_device *device = base->subdev.device; |
| 47 | struct gf100_bar *bar = gf100_bar(base); | 47 | struct gf100_bar *bar = gf100_bar(base); |
| @@ -52,7 +52,7 @@ tu104_bar_bar2_init(struct nvkm_bar *base) · 4 unmodified lines | |||
| 52 | } | 52 | } |
| 53 | 53 | ||
| 54 | static void | 54 | static void |
| 55 | tu104_bar_bar1_wait(struct nvkm_bar *bar) | 55 | tu102_bar_bar1_wait(struct nvkm_bar *bar) |
| 56 | { | 56 | { |
| 57 | struct nvkm_device *device = bar->subdev.device; | 57 | struct nvkm_device *device = bar->subdev.device; |
| 58 | nvkm_msec(device, 2000, | 58 | nvkm_msec(device, 2000, |
| @@ -62,13 +62,13 @@ tu104_bar_bar1_wait(struct nvkm_bar *bar) · 3 unmodified lines | |||
| 62 | } | 62 | } |
| 63 | 63 | ||
| 64 | static void | 64 | static void |
| 65 | tu104_bar_bar1_fini(struct nvkm_bar *bar) | 65 | tu102_bar_bar1_fini(struct nvkm_bar *bar) |
| 66 | { | 66 | { |
| 67 | nvkm_mask(bar->subdev.device, 0xb80f40, 0x80000000, 0x00000000); | 67 | nvkm_mask(bar->subdev.device, 0xb80f40, 0x80000000, 0x00000000); |
| 68 | } | 68 | } |
| 69 | 69 | ||
| 70 | static void | 70 | static void |
| 71 | tu104_bar_bar1_init(struct nvkm_bar *base) | 71 | tu102_bar_bar1_init(struct nvkm_bar *base) |
| 72 | { | 72 | { |
| 73 | struct nvkm_device *device = base->subdev.device; | 73 | struct nvkm_device *device = base->subdev.device; |
| 74 | struct gf100_bar *bar = gf100_bar(base); | 74 | struct gf100_bar *bar = gf100_bar(base); |
| @@ -77,22 +77,22 @@ tu104_bar_bar1_init(struct nvkm_bar *base) · 2 unmodified lines | |||
| 77 | } | 77 | } |
| 78 | 78 | ||
| 79 | static const struct nvkm_bar_func | 79 | static const struct nvkm_bar_func |
| 80 | tu104_bar = { | 80 | tu102_bar = { |
| 81 | .dtor = gf100_bar_dtor, | 81 | .dtor = gf100_bar_dtor, |
| 82 | .oneinit = gf100_bar_oneinit, | 82 | .oneinit = gf100_bar_oneinit, |
| 83 | .bar1.init = tu104_bar_bar1_init, | 83 | .bar1.init = tu102_bar_bar1_init, |
| 84 | .bar1.fini = tu104_bar_bar1_fini, | 84 | .bar1.fini = tu102_bar_bar1_fini, |
| 85 | .bar1.wait = tu104_bar_bar1_wait, | 85 | .bar1.wait = tu102_bar_bar1_wait, |
| 86 | .bar1.vmm = gf100_bar_bar1_vmm, | 86 | .bar1.vmm = gf100_bar_bar1_vmm, |
| 87 | .bar2.init = tu104_bar_bar2_init, | 87 | .bar2.init = tu102_bar_bar2_init, |
| 88 | .bar2.fini = tu104_bar_bar2_fini, | 88 | .bar2.fini = tu102_bar_bar2_fini, |
| 89 | .bar2.wait = tu104_bar_bar2_wait, | 89 | .bar2.wait = tu102_bar_bar2_wait, |
| 90 | .bar2.vmm = gf100_bar_bar2_vmm, | 90 | .bar2.vmm = gf100_bar_bar2_vmm, |
| 91 | .flush = g84_bar_flush, | 91 | .flush = g84_bar_flush, |
| 92 | }; | 92 | }; |
| 93 | 93 | ||
| 94 | int | 94 | int |
| 95 | tu104_bar_new(struct nvkm_device *device, int index, struct nvkm_bar **pbar) | 95 | tu102_bar_new(struct nvkm_device *device, int index, struct nvkm_bar **pbar) |
| 96 | { | 96 | { |
| 97 | return gf100_bar_new_(&tu104_bar, device, index, pbar); | 97 | return gf100_bar_new_(&tu102_bar, device, index, pbar); |
| 98 | } | 98 | } |
| @@ -212,7 +212,7 @@ nvbios_dpcfg_match(struct nvkm_bios *bios, u16 outp, u8 pc, u8 vs, u8 pe, · 211 unmodified lines | |||
| 212 | u16 data; | 212 | u16 data; |
| 213 | 213 | ||
| 214 | if (*ver >= 0x30) { | 214 | if (*ver >= 0x30) { |
| 215 | const u8 vsoff[] = { 0, 4, 7, 9 }; | 215 | static const u8 vsoff[] = { 0, 4, 7, 9 }; |
| 216 | idx = (pc * 10) + vsoff[vs] + pe; | 216 | idx = (pc * 10) + vsoff[vs] + pe; |
| 217 | if (*ver >= 0x40 && *ver <= 0x41 && *hdr >= 0x12) | 217 | if (*ver >= 0x40 && *ver <= 0x41 && *hdr >= 0x12) |
| 218 | idx += nvbios_rd08(bios, outp + 0x11) * 40; | 218 | idx += nvbios_rd08(bios, outp + 0x11) * 40; |
| @@ -806,12 +806,12 @@ init_generic_condition(struct nvbios_init *init) · 805 unmodified lines | |||
| 806 | init->offset += 3; | 806 | init->offset += 3; |
| 807 | 807 | ||
| 808 | switch (cond) { | 808 | switch (cond) { |
| 809 | case 0: | 809 | case 0: /* CONDITION_ID_INT_DP. */ |
| 810 | if (init_conn(init) != DCB_CONNECTOR_eDP) | 810 | if (init_conn(init) != DCB_CONNECTOR_eDP) |
| 811 | init_exec_set(init, false); | 811 | init_exec_set(init, false); |
| 812 | break; | 812 | break; |
| 813 | case 1: | 813 | case 1: /* CONDITION_ID_USE_SPPLL0. */ |
| 814 | case 2: | 814 | case 2: /* CONDITION_ID_USE_SPPLL1. */ |
| 815 | if ( init->outp && | 815 | if ( init->outp && |
| 816 | (data = nvbios_dpout_match(bios, DCB_OUTPUT_DP, | 816 | (data = nvbios_dpout_match(bios, DCB_OUTPUT_DP, |
| 817 | (init->outp->or << 0) | | 817 | (init->outp->or << 0) | |
| @@ -826,10 +826,13 @@ init_generic_condition(struct nvbios_init *init) · 8 unmodified lines | |||
| 826 | if (init_exec(init)) | 826 | if (init_exec(init)) |
| 827 | warn("script needs dp output table data\n"); | 827 | warn("script needs dp output table data\n"); |
| 828 | break; | 828 | break; |
| 829 | case 5: | 829 | case 5: /* CONDITION_ID_ASSR_SUPPORT. */ |
| 830 | if (!(init_rdauxr(init, 0x0d) & 1)) | 830 | if (!(init_rdauxr(init, 0x0d) & 1)) |
| 831 | init_exec_set(init, false); | 831 | init_exec_set(init, false); |
| 832 | break; | 832 | break; |
| 833 | case 7: /* CONDITION_ID_NO_PANEL_SEQ_DELAYS. */ | ||
| 834 | init_exec_set(init, false); | ||
| 835 | break; | ||
| 833 | default: | 836 | default: |
| 834 | warn("INIT_GENERIC_CONDITON: unknown 0x%02x\n", cond); | 837 | warn("INIT_GENERIC_CONDITON: unknown 0x%02x\n", cond); |
| 835 | init->offset += size; | 838 | init->offset += size; |
| @@ -13,4 +13,4 @@ nvkm-y += nvkm/subdev/devinit/gf100.o · 12 unmodified lines | |||
| 13 | nvkm-y += nvkm/subdev/devinit/gm107.o | 13 | nvkm-y += nvkm/subdev/devinit/gm107.o |
| 14 | nvkm-y += nvkm/subdev/devinit/gm200.o | 14 | nvkm-y += nvkm/subdev/devinit/gm200.o |
| 15 | nvkm-y += nvkm/subdev/devinit/gv100.o | 15 | nvkm-y += nvkm/subdev/devinit/gv100.o |
| 16 | nvkm-y += nvkm/subdev/devinit/tu104.o | 16 | nvkm-y += nvkm/subdev/devinit/tu102.o |
| @@ -26,7 +26,7 @@ · 25 unmodified lines | |||
| 26 | #include <subdev/clk/pll.h> | 26 | #include <subdev/clk/pll.h> |
| 27 | 27 | ||
| 28 | static int | 28 | static int |
| 29 | tu104_devinit_pll_set(struct nvkm_devinit *init, u32 type, u32 freq) | 29 | tu102_devinit_pll_set(struct nvkm_devinit *init, u32 type, u32 freq) |
| 30 | { | 30 | { |
| 31 | struct nvkm_subdev *subdev = &init->subdev; | 31 | struct nvkm_subdev *subdev = &init->subdev; |
| 32 | struct nvkm_device *device = subdev->device; | 32 | struct nvkm_device *device = subdev->device; |
| @@ -66,7 +66,7 @@ tu104_devinit_pll_set(struct nvkm_devinit *init, u32 type, u32 freq) · 33 unmodified lines | |||
| 66 | } | 66 | } |
| 67 | 67 | ||
| 68 | static int | 68 | static int |
| 69 | tu104_devinit_post(struct nvkm_devinit *base, bool post) | 69 | tu102_devinit_post(struct nvkm_devinit *base, bool post) |
| 70 | { | 70 | { |
| 71 | struct nv50_devinit *init = nv50_devinit(base); | 71 | struct nv50_devinit *init = nv50_devinit(base); |
| 72 | gm200_devinit_preos(init, post); | 72 | gm200_devinit_preos(init, post); |
| @@ -74,16 +74,16 @@ tu104_devinit_post(struct nvkm_devinit *base, bool post) · 1 unmodified line | |||
| 74 | } | 74 | } |
| 75 | 75 | ||
| 76 | static const struct nvkm_devinit_func | 76 | static const struct nvkm_devinit_func |
| 77 | tu104_devinit = { | 77 | tu102_devinit = { |
| 78 | .init = nv50_devinit_init, | 78 | .init = nv50_devinit_init, |
| 79 | .post = tu104_devinit_post, | 79 | .post = tu102_devinit_post, |
| 80 | .pll_set = tu104_devinit_pll_set, | 80 | .pll_set = tu102_devinit_pll_set, |
| 81 | .disable = gm107_devinit_disable, | 81 | .disable = gm107_devinit_disable, |
| 82 | }; | 82 | }; |
| 83 | 83 | ||
| 84 | int | 84 | int |
| 85 | tu104_devinit_new(struct nvkm_device *device, int index, | 85 | tu102_devinit_new(struct nvkm_device *device, int index, |
| 86 | struct nvkm_devinit **pinit) | 86 | struct nvkm_devinit **pinit) |
| 87 | { | 87 | { |
| 88 | return nv50_devinit_new_(&tu104_devinit, device, index, pinit); | 88 | return nv50_devinit_new_(&tu102_devinit, device, index, pinit); |
| 89 | } | 89 | } |
| @@ -1,4 +1,5 @@ | |||
| 1 | nvkm-y += nvkm/subdev/fault/base.o | 1 | nvkm-y += nvkm/subdev/fault/base.o |
| 2 | nvkm-y += nvkm/subdev/fault/user.o | ||
| 2 | nvkm-y += nvkm/subdev/fault/gp100.o | 3 | nvkm-y += nvkm/subdev/fault/gp100.o |
| 3 | nvkm-y += nvkm/subdev/fault/gv100.o | 4 | nvkm-y += nvkm/subdev/fault/gv100.o |
| 4 | nvkm-y += nvkm/subdev/fault/tu104.o | 5 | nvkm-y += nvkm/subdev/fault/tu102.o |
| @@ -176,5 +176,7 @@ nvkm_fault_new_(const struct nvkm_fault_func *func, struct nvkm_device *device, · 175 unmodified lines | |||
| 176 | return -ENOMEM; | 176 | return -ENOMEM; |
| 177 | nvkm_subdev_ctor(&nvkm_fault, device, index, &fault->subdev); | 177 | nvkm_subdev_ctor(&nvkm_fault, device, index, &fault->subdev); |
| 178 | fault->func = func; | 178 | fault->func = func; |
| 179 | fault->user.ctor = nvkm_ufault_new; | ||
| 180 | fault->user.base = func->user.base; | ||
| 179 | return 0; | 181 | return 0; |
| 180 | } | 182 | } |
| @@ -23,6 +23,8 @@ · 22 unmodified lines | |||
| 23 | 23 | ||
| 24 | #include <subdev/mc.h> | 24 | #include <subdev/mc.h> |
| 25 | 25 | ||
| 26 | #include <nvif/class.h> | ||
| 27 | |||
| 26 | static void | 28 | static void |
| 27 | gp100_fault_buffer_intr(struct nvkm_fault_buffer *buffer, bool enable) | 29 | gp100_fault_buffer_intr(struct nvkm_fault_buffer *buffer, bool enable) |
| 28 | { | 30 | { |
| @@ -69,6 +71,7 @@ gp100_fault = { · 40 unmodified lines | |||
| 69 | .buffer.init = gp100_fault_buffer_init, | 71 | .buffer.init = gp100_fault_buffer_init, |
| 70 | .buffer.fini = gp100_fault_buffer_fini, | 72 | .buffer.fini = gp100_fault_buffer_fini, |
| 71 | .buffer.intr = gp100_fault_buffer_intr, | 73 | .buffer.intr = gp100_fault_buffer_intr, |
| 74 | .user = { { 0, 0, MAXWELL_FAULT_BUFFER_A }, 0 }, | ||
| 72 | }; | 75 | }; |
| 73 | 76 | ||
| 74 | int | 77 | int |
| @@ -25,6 +25,8 @@ · 24 unmodified lines | |||
| 25 | #include <subdev/mmu.h> | 25 | #include <subdev/mmu.h> |
| 26 | #include <engine/fifo.h> | 26 | #include <engine/fifo.h> |
| 27 | 27 | ||
| 28 | #include <nvif/class.h> | ||
| 29 | |||
| 28 | static void | 30 | static void |
| 29 | gv100_fault_buffer_process(struct nvkm_fault_buffer *buffer) | 31 | gv100_fault_buffer_process(struct nvkm_fault_buffer *buffer) |
| 30 | { | 32 | { |
| @@ -166,6 +168,13 @@ gv100_fault_intr(struct nvkm_fault *fault) · 135 unmodified lines | |||
| 166 | } | 168 | } |
| 167 | } | 169 | } |
| 168 | 170 | ||
| 171 | if (stat & 0x08000000) { | ||
| 172 | if (fault->buffer[1]) { | ||
| 173 | nvkm_event_send(&fault->event, 1, 1, NULL, 0); | ||
| 174 | stat &= ~0x08000000; | ||
| 175 | } | ||
| 176 | } | ||
| 177 | |||
| 169 | if (stat) { | 178 | if (stat) { |
| 170 | nvkm_debug(subdev, "intr %08x\n", stat); | 179 | nvkm_debug(subdev, "intr %08x\n", stat); |
| 171 | } | 180 | } |
| @@ -208,6 +217,13 @@ gv100_fault = { · 36 unmodified lines | |||
| 208 | .buffer.init = gv100_fault_buffer_init, | 217 | .buffer.init = gv100_fault_buffer_init, |
| 209 | .buffer.fini = gv100_fault_buffer_fini, | 218 | .buffer.fini = gv100_fault_buffer_fini, |
| 210 | .buffer.intr = gv100_fault_buffer_intr, | 219 | .buffer.intr = gv100_fault_buffer_intr, |
| 220 | /*TODO: Figure out how to expose non-replayable fault buffer, which, | ||
| 221 | * for some reason, is where recoverable CE faults appear... | ||
| 222 | * | ||
| 223 | * It's a bit tricky, as both NVKM and SVM will need access to | ||
| 224 | * the non-replayable fault buffer. | ||
| 225 | */ | ||
| 226 | .user = { { 0, 0, VOLTA_FAULT_BUFFER_A }, 1 }, | ||
| 211 | }; | 227 | }; |
| 212 | 228 | ||
| 213 | int | 229 | int |
| @@ -34,7 +34,14 @@ struct nvkm_fault_func { · 33 unmodified lines | |||
| 34 | void (*fini)(struct nvkm_fault_buffer *); | 34 | void (*fini)(struct nvkm_fault_buffer *); |
| 35 | void (*intr)(struct nvkm_fault_buffer *, bool enable); | 35 | void (*intr)(struct nvkm_fault_buffer *, bool enable); |
| 36 | } buffer; | 36 | } buffer; |
| 37 | struct { | ||
| 38 | struct nvkm_sclass base; | ||
| 39 | int rp; | ||
| 40 | } user; | ||
| 37 | }; | 41 | }; |
| 38 | 42 | ||
| 39 | int gv100_fault_oneinit(struct nvkm_fault *); | 43 | int gv100_fault_oneinit(struct nvkm_fault *); |
| 44 | |||
| 45 | int nvkm_ufault_new(struct nvkm_device *, const struct nvkm_oclass *, | ||
| 46 | void *, u32, struct nvkm_object **); | ||
| 40 | #endif | 47 | #endif |
| @@ -28,7 +28,7 @@ · 27 unmodified lines | |||
| 28 | #include <nvif/class.h> | 28 | #include <nvif/class.h> |
| 29 | 29 | ||
| 30 | static void | 30 | static void |
| 31 | tu104_fault_buffer_intr(struct nvkm_fault_buffer *buffer, bool enable) | 31 | tu102_fault_buffer_intr(struct nvkm_fault_buffer *buffer, bool enable) |
| 32 | { | 32 | { |
| 33 | /*XXX: Earlier versions of RM touched the old regs on Turing, | 33 | /*XXX: Earlier versions of RM touched the old regs on Turing, |
| 34 | * which don't appear to actually work anymore, but newer | 34 | * which don't appear to actually work anymore, but newer |
| @@ -37,7 +37,7 @@ tu104_fault_buffer_intr(struct nvkm_fault_buffer *buffer, bool enable) · 2 unmodified lines | |||
| 37 | } | 37 | } |
| 38 | 38 | ||
| 39 | static void | 39 | static void |
| 40 | tu104_fault_buffer_fini(struct nvkm_fault_buffer *buffer) | 40 | tu102_fault_buffer_fini(struct nvkm_fault_buffer *buffer) |
| 41 | { | 41 | { |
| 42 | struct nvkm_device *device = buffer->fault->subdev.device; | 42 | struct nvkm_device *device = buffer->fault->subdev.device; |
| 43 | const u32 foff = buffer->id * 0x20; | 43 | const u32 foff = buffer->id * 0x20; |
| @@ -45,7 +45,7 @@ tu104_fault_buffer_fini(struct nvkm_fault_buffer *buffer) · 1 unmodified line | |||
| 45 | } | 45 | } |
| 46 | 46 | ||
| 47 | static void | 47 | static void |
| 48 | tu104_fault_buffer_init(struct nvkm_fault_buffer *buffer) | 48 | tu102_fault_buffer_init(struct nvkm_fault_buffer *buffer) |
| 49 | { | 49 | { |
| 50 | struct nvkm_device *device = buffer->fault->subdev.device; | 50 | struct nvkm_device *device = buffer->fault->subdev.device; |
| 51 | const u32 foff = buffer->id * 0x20; | 51 | const u32 foff = buffer->id * 0x20; |
| @@ -57,7 +57,7 @@ tu104_fault_buffer_init(struct nvkm_fault_buffer *buffer) · 5 unmodified lines | |||
| 57 | } | 57 | } |
| 58 | 58 | ||
| 59 | static void | 59 | static void |
| 60 | tu104_fault_buffer_info(struct nvkm_fault_buffer *buffer) | 60 | tu102_fault_buffer_info(struct nvkm_fault_buffer *buffer) |
| 61 | { | 61 | { |
| 62 | struct nvkm_device *device = buffer->fault->subdev.device; | 62 | struct nvkm_device *device = buffer->fault->subdev.device; |
| 63 | const u32 foff = buffer->id * 0x20; | 63 | const u32 foff = buffer->id * 0x20; |
| @@ -70,7 +70,7 @@ tu104_fault_buffer_info(struct nvkm_fault_buffer *buffer) · 6 unmodified lines | |||
| 70 | } | 70 | } |
| 71 | 71 | ||
| 72 | static void | 72 | static void |
| 73 | tu104_fault_intr_fault(struct nvkm_fault *fault) | 73 | tu102_fault_intr_fault(struct nvkm_fault *fault) |
| 74 | { | 74 | { |
| 75 | struct nvkm_subdev *subdev = &fault->subdev; | 75 | struct nvkm_subdev *subdev = &fault->subdev; |
| 76 | struct nvkm_device *device = subdev->device; | 76 | struct nvkm_device *device = subdev->device; |
| @@ -96,14 +96,14 @@ tu104_fault_intr_fault(struct nvkm_fault *fault) · 19 unmodified lines | |||
| 96 | } | 96 | } |
| 97 | 97 | ||
| 98 | static void | 98 | static void |
| 99 | tu104_fault_intr(struct nvkm_fault *fault) | 99 | tu102_fault_intr(struct nvkm_fault *fault) |
| 100 | { | 100 | { |
| 101 | struct nvkm_subdev *subdev = &fault->subdev; | 101 | struct nvkm_subdev *subdev = &fault->subdev; |
| 102 | struct nvkm_device *device = subdev->device; | 102 | struct nvkm_device *device = subdev->device; |
| 103 | u32 stat = nvkm_rd32(device, 0xb83094); | 103 | u32 stat = nvkm_rd32(device, 0xb83094); |
| 104 | 104 | ||
| 105 | if (stat & 0x80000000) { | 105 | if (stat & 0x80000000) { |
| 106 | tu104_fault_intr_fault(fault); | 106 | tu102_fault_intr_fault(fault); |
| 107 | nvkm_wr32(device, 0xb83094, 0x80000000); | 107 | nvkm_wr32(device, 0xb83094, 0x80000000); |
| 108 | stat &= ~0x80000000; | 108 | stat &= ~0x80000000; |
| 109 | } | 109 | } |
| @@ -129,7 +129,7 @@ tu104_fault_intr(struct nvkm_fault *fault) · 19 unmodified lines | |||
| 129 | } | 129 | } |
| 130 | 130 | ||
| 131 | static void | 131 | static void |
| 132 | tu104_fault_fini(struct nvkm_fault *fault) | 132 | tu102_fault_fini(struct nvkm_fault *fault) |
| 133 | { | 133 | { |
| 134 | nvkm_notify_put(&fault->nrpfb); | 134 | nvkm_notify_put(&fault->nrpfb); |
| 135 | if (fault->buffer[0]) | 135 | if (fault->buffer[0]) |
| @@ -138,7 +138,7 @@ tu104_fault_fini(struct nvkm_fault *fault) · 2 unmodified lines | |||
| 138 | } | 138 | } |
| 139 | 139 | ||
| 140 | static void | 140 | static void |
| 141 | tu104_fault_init(struct nvkm_fault *fault) | 141 | tu102_fault_init(struct nvkm_fault *fault) |
| 142 | { | 142 | { |
| 143 | /*XXX: enable priv faults */ | 143 | /*XXX: enable priv faults */ |
| 144 | fault->func->buffer.init(fault->buffer[0]); | 144 | fault->func->buffer.init(fault->buffer[0]); |
| @@ -146,22 +146,23 @@ tu104_fault_init(struct nvkm_fault *fault) · 1 unmodified line | |||
| 146 | } | 146 | } |
| 147 | 147 | ||
| 148 | static const struct nvkm_fault_func | 148 | static const struct nvkm_fault_func |
| 149 | tu104_fault = { | 149 | tu102_fault = { |
| 150 | .oneinit = gv100_fault_oneinit, | 150 | .oneinit = gv100_fault_oneinit, |
| 151 | .init = tu104_fault_init, | 151 | .init = tu102_fault_init, |
| 152 | .fini = tu104_fault_fini, | 152 | .fini = tu102_fault_fini, |
| 153 | .intr = tu104_fault_intr, | 153 | .intr = tu102_fault_intr, |
| 154 | .buffer.nr = 2, | 154 | .buffer.nr = 2, |
| 155 | .buffer.entry_size = 32, | 155 | .buffer.entry_size = 32, |
| 156 | .buffer.info = tu104_fault_buffer_info, | 156 | .buffer.info = tu102_fault_buffer_info, |
| 157 | .buffer.init = tu104_fault_buffer_init, | 157 | .buffer.init = tu102_fault_buffer_init, |
| 158 | .buffer.fini = tu104_fault_buffer_fini, | 158 | .buffer.fini = tu102_fault_buffer_fini, |
| 159 | .buffer.intr = tu104_fault_buffer_intr, | 159 | .buffer.intr = tu102_fault_buffer_intr, |
| 160 | .user = { { 0, 0, VOLTA_FAULT_BUFFER_A }, 1 }, | ||
| 160 | }; | 161 | }; |
| 161 | 162 | ||
| 162 | int | 163 | int |
| 163 | tu104_fault_new(struct nvkm_device *device, int index, | 164 | tu102_fault_new(struct nvkm_device *device, int index, |
| 164 | struct nvkm_fault **pfault) | 165 | struct nvkm_fault **pfault) |
| 165 | { | 166 | { |
| 166 | return nvkm_fault_new_(&tu104_fault, device, index, pfault); | 167 | return nvkm_fault_new_(&tu102_fault, device, index, pfault); |
| 167 | } | 168 | } |
| @@ -0,0 +1,106 @@ | |||
| 1 | /* | ||
| 2 | * Copyright 2018 Red Hat Inc. | ||
| 3 | * | ||
| 4 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 5 | * copy of this software and associated documentation files (the "Software"), | ||
| 6 | * to deal in the Software without restriction, including without limitation | ||
| 7 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 8 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 9 | * Software is furnished to do so, subject to the following conditions: | ||
| 10 | * | ||
| 11 | * The above copyright notice and this permission notice shall be included in | ||
| 12 | * all copies or substantial portions of the Software. | ||
| 13 | * | ||
| 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 17 | * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | ||
| 18 | * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | ||
| 19 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | ||
| 20 | * OTHER DEALINGS IN THE SOFTWARE. | ||
| 21 | */ | ||
| 22 | #include "priv.h" | ||
| 23 | |||
| 24 | #include <core/memory.h> | ||
| 25 | #include <subdev/mmu.h> | ||
| 26 | |||
| 27 | #include <nvif/clb069.h> | ||
| 28 | #include <nvif/unpack.h> | ||
| 29 | |||
| 30 | static int | ||
| 31 | nvkm_ufault_map(struct nvkm_object *object, void *argv, u32 argc, | ||
| 32 | enum nvkm_object_map *type, u64 *addr, u64 *size) | ||
| 33 | { | ||
| 34 | struct nvkm_fault_buffer *buffer = nvkm_fault_buffer(object); | ||
| 35 | struct nvkm_device *device = buffer->fault->subdev.device; | ||
| 36 | *type = NVKM_OBJECT_MAP_IO; | ||
| 37 | *addr = device->func->resource_addr(device, 3) + buffer->addr; | ||
| 38 | *size = nvkm_memory_size(buffer->mem); | ||
| 39 | return 0; | ||
| 40 | } | ||
| 41 | |||
| 42 | static int | ||
| 43 | nvkm_ufault_ntfy(struct nvkm_object *object, u32 type, | ||
| 44 | struct nvkm_event **pevent) | ||
| 45 | { | ||
| 46 | struct nvkm_fault_buffer *buffer = nvkm_fault_buffer(object); | ||
| 47 | if (type == NVB069_V0_NTFY_FAULT) { | ||
| 48 | *pevent = &buffer->fault->event; | ||
| 49 | return 0; | ||
| 50 | } | ||
| 51 | return -EINVAL; | ||
| 52 | } | ||
| 53 | |||
| 54 | static int | ||
| 55 | nvkm_ufault_fini(struct nvkm_object *object, bool suspend) | ||
| 56 | { | ||
| 57 | struct nvkm_fault_buffer *buffer = nvkm_fault_buffer(object); | ||
| 58 | buffer->fault->func->buffer.fini(buffer); | ||
| 59 | return 0; | ||
| 60 | } | ||
| 61 | |||
| 62 | static int | ||
| 63 | nvkm_ufault_init(struct nvkm_object *object) | ||
| 64 | { | ||
| 65 | struct nvkm_fault_buffer *buffer = nvkm_fault_buffer(object); | ||
| 66 | buffer->fault->func->buffer.init(buffer); | ||
| 67 | return 0; | ||
| 68 | } | ||
| 69 | |||
| 70 | static void * | ||
| 71 | nvkm_ufault_dtor(struct nvkm_object *object) | ||
| 72 | { | ||
| 73 | return NULL; | ||
| 74 | } | ||
| 75 | |||
| 76 | static const struct nvkm_object_func | ||
| 77 | nvkm_ufault = { | ||
| 78 | .dtor = nvkm_ufault_dtor, | ||
| 79 | .init = nvkm_ufault_init, | ||
| 80 | .fini = nvkm_ufault_fini, | ||
| 81 | .ntfy = nvkm_ufault_ntfy, | ||
| 82 | .map = nvkm_ufault_map, | ||
| 83 | }; | ||
| 84 | |||
| 85 | int | ||
| 86 | nvkm_ufault_new(struct nvkm_device *device, const struct nvkm_oclass *oclass, | ||
| 87 | void *argv, u32 argc, struct nvkm_object **pobject) | ||
| 88 | { | ||
| 89 | union { | ||
| 90 | struct nvif_clb069_v0 v0; | ||
| 91 | } *args = argv; | ||
| 92 | struct nvkm_fault *fault = device->fault; | ||
| 93 | struct nvkm_fault_buffer *buffer = fault->buffer[fault->func->user.rp]; | ||
| 94 | int ret = -ENOSYS; | ||
| 95 | |||
| 96 | if (!(ret = nvif_unpack(ret, &argv, &argc, args->v0, 0, 0, false))) { | ||
| 97 | args->v0.entries = buffer->entries; | ||
| 98 | args->v0.get = buffer->get; | ||
| 99 | args->v0.put = buffer->put; | ||
| 100 | } else | ||
| 101 | return ret; | ||
| 102 | |||
| 103 | nvkm_object_ctor(&nvkm_ufault, oclass, &buffer->object); | ||
| 104 | *pobject = &buffer->object; | ||
| 105 | return 0; | ||
| 106 | } | ||
| @@ -87,7 +87,7 @@ nvkm_gddr3_calc(struct nvkm_ram *ram) · 86 unmodified lines | |||
| 87 | WR = (ram->next->bios.timing[2] & 0x007f0000) >> 16; | 87 | WR = (ram->next->bios.timing[2] & 0x007f0000) >> 16; |
| 88 | /* XXX: Get these values from the VBIOS instead */ | 88 | /* XXX: Get these values from the VBIOS instead */ |
| 89 | DLL = !(ram->mr[1] & 0x1); | 89 | DLL = !(ram->mr[1] & 0x1); |
| 90 | RON = !(ram->mr[1] & 0x300) >> 8; | 90 | RON = !((ram->mr[1] & 0x300) >> 8); |
| 91 | break; | 91 | break; |
| 92 | default: | 92 | default: |
| 93 | return -ENOSYS; | 93 | return -ENOSYS; |
| @@ -0,0 +1 @@ | |||
| 1 | nvkm-y += nvkm/subdev/gsp/gv100.o | ||
| @@ -0,0 +1,62 @@ | |||
| 1 | /* | ||
| 2 | * Copyright 2019 Red Hat Inc. | ||
| 3 | * | ||
| 4 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 5 | * copy of this software and associated documentation files (the "Software"), | ||
| 6 | * to deal in the Software without restriction, including without limitation | ||
| 7 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 8 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 9 | * Software is furnished to do so, subject to the following conditions: | ||
| 10 | * | ||
| 11 | * The above copyright notice and this permission notice shall be included in | ||
| 12 | * all copies or substantial portions of the Software. | ||
| 13 | * | ||
| 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 17 | * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | ||
| 18 | * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | ||
| 19 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | ||
| 20 | * OTHER DEALINGS IN THE SOFTWARE. | ||
| 21 | */ | ||
| 22 | #include <subdev/gsp.h> | ||
| 23 | #include <subdev/top.h> | ||
| 24 | #include <engine/falcon.h> | ||
| 25 | |||
| 26 | static int | ||
| 27 | gv100_gsp_oneinit(struct nvkm_subdev *subdev) | ||
| 28 | { | ||
| 29 | struct nvkm_gsp *gsp = nvkm_gsp(subdev); | ||
| 30 | |||
| 31 | gsp->addr = nvkm_top_addr(subdev->device, subdev->index); | ||
| 32 | if (!gsp->addr) | ||
| 33 | return -EINVAL; | ||
| 34 | |||
| 35 | return nvkm_falcon_v1_new(subdev, "GSP", gsp->addr, &gsp->falcon); | ||
| 36 | } | ||
| 37 | |||
| 38 | static void * | ||
| 39 | gv100_gsp_dtor(struct nvkm_subdev *subdev) | ||
| 40 | { | ||
| 41 | struct nvkm_gsp *gsp = nvkm_gsp(subdev); | ||
| 42 | nvkm_falcon_del(&gsp->falcon); | ||
| 43 | return gsp; | ||
| 44 | } | ||
| 45 | |||
| 46 | static const struct nvkm_subdev_func | ||
| 47 | gv100_gsp = { | ||
| 48 | .dtor = gv100_gsp_dtor, | ||
| 49 | .oneinit = gv100_gsp_oneinit, | ||
| 50 | }; | ||
| 51 | |||
| 52 | int | ||
| 53 | gv100_gsp_new(struct nvkm_device *device, int index, struct nvkm_gsp **pgsp) | ||
| 54 | { | ||
| 55 | struct nvkm_gsp *gsp; | ||
| 56 | |||
| 57 | if (!(gsp = *pgsp = kzalloc(sizeof(*gsp), GFP_KERNEL))) | ||
| 58 | return -ENOMEM; | ||
| 59 | |||
| 60 | nvkm_subdev_ctor(&gv100_gsp, device, index, &gsp->subdev); | ||
| 61 | return 0; | ||
| 62 | } | ||
| @@ -12,4 +12,4 @@ nvkm-y += nvkm/subdev/mc/gk104.o · 11 unmodified lines | |||
| 12 | nvkm-y += nvkm/subdev/mc/gk20a.o | 12 | nvkm-y += nvkm/subdev/mc/gk20a.o |
| 13 | nvkm-y += nvkm/subdev/mc/gp100.o | 13 | nvkm-y += nvkm/subdev/mc/gp100.o |
| 14 | nvkm-y += nvkm/subdev/mc/gp10b.o | 14 | nvkm-y += nvkm/subdev/mc/gp10b.o |
| 15 | nvkm-y += nvkm/subdev/mc/tu104.o | 15 | nvkm-y += nvkm/subdev/mc/tu102.o |
| @@ -22,7 +22,7 @@ · 21 unmodified lines | |||
| 22 | #include "priv.h" | 22 | #include "priv.h" |
| 23 | 23 | ||
| 24 | static void | 24 | static void |
| 25 | tu104_mc_intr_hack(struct nvkm_mc *mc, bool *handled) | 25 | tu102_mc_intr_hack(struct nvkm_mc *mc, bool *handled) |
| 26 | { | 26 | { |
| 27 | struct nvkm_device *device = mc->subdev.device; | 27 | struct nvkm_device *device = mc->subdev.device; |
| 28 | u32 stat = nvkm_rd32(device, 0xb81010); | 28 | u32 stat = nvkm_rd32(device, 0xb81010); |
| @@ -37,19 +37,19 @@ tu104_mc_intr_hack(struct nvkm_mc *mc, bool *handled) · 8 unmodified lines | |||
| 37 | } | 37 | } |
| 38 | 38 | ||
| 39 | static const struct nvkm_mc_func | 39 | static const struct nvkm_mc_func |
| 40 | tu104_mc = { | 40 | tu102_mc = { |
| 41 | .init = nv50_mc_init, | 41 | .init = nv50_mc_init, |
| 42 | .intr = gp100_mc_intr, | 42 | .intr = gp100_mc_intr, |
| 43 | .intr_unarm = gp100_mc_intr_unarm, | 43 | .intr_unarm = gp100_mc_intr_unarm, |
| 44 | .intr_rearm = gp100_mc_intr_rearm, | 44 | .intr_rearm = gp100_mc_intr_rearm, |
| 45 | .intr_mask = gp100_mc_intr_mask, | 45 | .intr_mask = gp100_mc_intr_mask, |
| 46 | .intr_stat = gf100_mc_intr_stat, | 46 | .intr_stat = gf100_mc_intr_stat, |
| 47 | .intr_hack = tu104_mc_intr_hack, | 47 | .intr_hack = tu102_mc_intr_hack, |
| 48 | .reset = gk104_mc_reset, | 48 | .reset = gk104_mc_reset, |
| 49 | }; | 49 | }; |
| 50 | 50 | ||
| 51 | int | 51 | int |
| 52 | tu104_mc_new(struct nvkm_device *device, int index, struct nvkm_mc **pmc) | 52 | tu102_mc_new(struct nvkm_device *device, int index, struct nvkm_mc **pmc) |
| 53 | { | 53 | { |
| 54 | return gp100_mc_new_(&tu104_mc, device, index, pmc); | 54 | return gp100_mc_new_(&tu102_mc, device, index, pmc); |
| 55 | } | 55 | } |
| @@ -13,7 +13,7 @@ nvkm-y += nvkm/subdev/mmu/gm20b.o · 12 unmodified lines | |||
| 13 | nvkm-y += nvkm/subdev/mmu/gp100.o | 13 | nvkm-y += nvkm/subdev/mmu/gp100.o |
| 14 | nvkm-y += nvkm/subdev/mmu/gp10b.o | 14 | nvkm-y += nvkm/subdev/mmu/gp10b.o |
| 15 | nvkm-y += nvkm/subdev/mmu/gv100.o | 15 | nvkm-y += nvkm/subdev/mmu/gv100.o |
| 16 | nvkm-y += nvkm/subdev/mmu/tu104.o | 16 | nvkm-y += nvkm/subdev/mmu/tu102.o |
| 17 | 17 | ||
| 18 | nvkm-y += nvkm/subdev/mmu/mem.o | 18 | nvkm-y += nvkm/subdev/mmu/mem.o |
| 19 | nvkm-y += nvkm/subdev/mmu/memnv04.o | 19 | nvkm-y += nvkm/subdev/mmu/memnv04.o |
| @@ -34,7 +34,7 @@ nvkm-y += nvkm/subdev/mmu/vmmgm20b.o · 14 unmodified lines | |||
| 34 | nvkm-y += nvkm/subdev/mmu/vmmgp100.o | 34 | nvkm-y += nvkm/subdev/mmu/vmmgp100.o |
| 35 | nvkm-y += nvkm/subdev/mmu/vmmgp10b.o | 35 | nvkm-y += nvkm/subdev/mmu/vmmgp10b.o |
| 36 | nvkm-y += nvkm/subdev/mmu/vmmgv100.o | 36 | nvkm-y += nvkm/subdev/mmu/vmmgv100.o |
| 37 | nvkm-y += nvkm/subdev/mmu/vmmtu104.o | 37 | nvkm-y += nvkm/subdev/mmu/vmmtu102.o |
| 38 | 38 | ||
| 39 | nvkm-y += nvkm/subdev/mmu/umem.o | 39 | nvkm-y += nvkm/subdev/mmu/umem.o |
| 40 | nvkm-y += nvkm/subdev/mmu/ummu.o | 40 | nvkm-y += nvkm/subdev/mmu/ummu.o |
| @@ -31,7 +31,7 @@ gp100_mmu = { · 30 unmodified lines | |||
| 31 | .dma_bits = 47, | 31 | .dma_bits = 47, |
| 32 | .mmu = {{ -1, -1, NVIF_CLASS_MMU_GF100}}, | 32 | .mmu = {{ -1, -1, NVIF_CLASS_MMU_GF100}}, |
| 33 | .mem = {{ -1, 0, NVIF_CLASS_MEM_GF100}, gf100_mem_new, gf100_mem_map }, | 33 | .mem = {{ -1, 0, NVIF_CLASS_MEM_GF100}, gf100_mem_new, gf100_mem_map }, |
| 34 | .vmm = {{ -1, -1, NVIF_CLASS_VMM_GP100}, gp100_vmm_new }, | 34 | .vmm = {{ -1, 0, NVIF_CLASS_VMM_GP100}, gp100_vmm_new }, |
| 35 | .kind = gm200_mmu_kind, | 35 | .kind = gm200_mmu_kind, |
| 36 | .kind_sys = true, | 36 | .kind_sys = true, |
| 37 | }; | 37 | }; |
| @@ -31,7 +31,7 @@ gp10b_mmu = { · 30 unmodified lines | |||
| 31 | .dma_bits = 47, | 31 | .dma_bits = 47, |
| 32 | .mmu = {{ -1, -1, NVIF_CLASS_MMU_GF100}}, | 32 | .mmu = {{ -1, -1, NVIF_CLASS_MMU_GF100}}, |
| 33 | .mem = {{ -1, -1, NVIF_CLASS_MEM_GF100}, .umap = gf100_mem_map }, | 33 | .mem = {{ -1, -1, NVIF_CLASS_MEM_GF100}, .umap = gf100_mem_map }, |
| 34 | .vmm = {{ -1, -1, NVIF_CLASS_VMM_GP100}, gp10b_vmm_new }, | 34 | .vmm = {{ -1, 0, NVIF_CLASS_VMM_GP100}, gp10b_vmm_new }, |
| 35 | .kind = gm200_mmu_kind, | 35 | .kind = gm200_mmu_kind, |
| 36 | .kind_sys = true, | 36 | .kind_sys = true, |
| 37 | }; | 37 | }; |
| @@ -31,7 +31,7 @@ gv100_mmu = { · 30 unmodified lines | |||
| 31 | .dma_bits = 47, | 31 | .dma_bits = 47, |
| 32 | .mmu = {{ -1, -1, NVIF_CLASS_MMU_GF100}}, | 32 | .mmu = {{ -1, -1, NVIF_CLASS_MMU_GF100}}, |
| 33 | .mem = {{ -1, 0, NVIF_CLASS_MEM_GF100}, gf100_mem_new, gf100_mem_map }, | 33 | .mem = {{ -1, 0, NVIF_CLASS_MEM_GF100}, gf100_mem_new, gf100_mem_map }, |
| 34 | .vmm = {{ -1, -1, NVIF_CLASS_VMM_GP100}, gv100_vmm_new }, | 34 | .vmm = {{ -1, 0, NVIF_CLASS_VMM_GP100}, gv100_vmm_new }, |
| 35 | .kind = gm200_mmu_kind, | 35 | .kind = gm200_mmu_kind, |
| 36 | .kind_sys = true, | 36 | .kind_sys = true, |
| 37 | }; | 37 | }; |
| @@ -28,7 +28,7 @@ struct nvkm_mmu_func { · 27 unmodified lines | |||
| 28 | 28 | ||
| 29 | struct { | 29 | struct { |
| 30 | struct nvkm_sclass user; | 30 | struct nvkm_sclass user; |
| 31 | int (*ctor)(struct nvkm_mmu *, u64 addr, u64 size, | 31 | int (*ctor)(struct nvkm_mmu *, bool managed, u64 addr, u64 size, |
| 32 | void *argv, u32 argc, struct lock_class_key *, | 32 | void *argv, u32 argc, struct lock_class_key *, |
| 33 | const char *name, struct nvkm_vmm **); | 33 | const char *name, struct nvkm_vmm **); |
| 34 | bool global; | 34 | bool global; |
| @@ -27,17 +27,17 @@ · 26 unmodified lines | |||
| 27 | #include <nvif/class.h> | 27 | #include <nvif/class.h> |
| 28 | 28 | ||
| 29 | static const struct nvkm_mmu_func | 29 | static const struct nvkm_mmu_func |
| 30 | tu104_mmu = { | 30 | tu102_mmu = { |
| 31 | .dma_bits = 47, | 31 | .dma_bits = 47, |
| 32 | .mmu = {{ -1, -1, NVIF_CLASS_MMU_GF100}}, | 32 | .mmu = {{ -1, -1, NVIF_CLASS_MMU_GF100}}, |
| 33 | .mem = {{ -1, 0, NVIF_CLASS_MEM_GF100}, gf100_mem_new, gf100_mem_map }, | 33 | .mem = {{ -1, 0, NVIF_CLASS_MEM_GF100}, gf100_mem_new, gf100_mem_map }, |
| 34 | .vmm = {{ -1, 0, NVIF_CLASS_VMM_GP100}, tu104_vmm_new }, | 34 | .vmm = {{ -1, 0, NVIF_CLASS_VMM_GP100}, tu102_vmm_new }, |
| 35 | .kind = gm200_mmu_kind, | 35 | .kind = gm200_mmu_kind, |
| 36 | .kind_sys = true, | 36 | .kind_sys = true, |
| 37 | }; | 37 | }; |
| 38 | 38 | ||
| 39 | int | 39 | int |
| 40 | tu104_mmu_new(struct nvkm_device *device, int index, struct nvkm_mmu **pmmu) | 40 | tu102_mmu_new(struct nvkm_device *device, int index, struct nvkm_mmu **pmmu) |
| 41 | { | 41 | { |
| 42 | return nvkm_mmu_new_(&tu104_mmu, device, index, pmmu); | 42 | return nvkm_mmu_new_(&tu102_mmu, device, index, pmmu); |
| 43 | } | 43 | } |
| @@ -42,6 +42,69 @@ nvkm_uvmm_search(struct nvkm_client *client, u64 handle) · 41 unmodified lines | |||
| 42 | return nvkm_uvmm(object)->vmm; | 42 | return nvkm_uvmm(object)->vmm; |
| 43 | } | 43 | } |
| 44 | 44 | ||
| 45 | static int | ||
| 46 | nvkm_uvmm_mthd_pfnclr(struct nvkm_uvmm *uvmm, void *argv, u32 argc) | ||
| 47 | { | ||
| 48 | struct nvkm_client *client = uvmm->object.client; | ||
| 49 | union { | ||
| 50 | struct nvif_vmm_pfnclr_v0 v0; | ||
| 51 | } *args = argv; | ||
| 52 | struct nvkm_vmm *vmm = uvmm->vmm; | ||
| 53 | int ret = -ENOSYS; | ||
| 54 | u64 addr, size; | ||
| 55 | |||
| 56 | if (!(ret = nvif_unpack(ret, &argv, &argc, args->v0, 0, 0, false))) { | ||
| 57 | addr = args->v0.addr; | ||
| 58 | size = args->v0.size; | ||
| 59 | } else | ||
| 60 | return ret; | ||
| 61 | |||
| 62 | if (!client->super) | ||
| 63 | return -ENOENT; | ||
| 64 | |||
| 65 | if (size) { | ||
| 66 | mutex_lock(&vmm->mutex); | ||
| 67 | ret = nvkm_vmm_pfn_unmap(vmm, addr, size); | ||
| 68 | mutex_unlock(&vmm->mutex); | ||
| 69 | } | ||
| 70 | |||
| 71 | return ret; | ||
| 72 | } | ||
| 73 | |||
| 74 | static int | ||
| 75 | nvkm_uvmm_mthd_pfnmap(struct nvkm_uvmm *uvmm, void *argv, u32 argc) | ||
| 76 | { | ||
| 77 | struct nvkm_client *client = uvmm->object.client; | ||
| 78 | union { | ||
| 79 | struct nvif_vmm_pfnmap_v0 v0; | ||
| 80 | } *args = argv; | ||
| 81 | struct nvkm_vmm *vmm = uvmm->vmm; | ||
| 82 | int ret = -ENOSYS; | ||
| 83 | u64 addr, size, *phys; | ||
| 84 | u8 page; | ||
| 85 | |||
| 86 | if (!(ret = nvif_unpack(ret, &argv, &argc, args->v0, 0, 0, true))) { | ||
| 87 | page = args->v0.page; | ||
| 88 | addr = args->v0.addr; | ||
| 89 | size = args->v0.size; | ||
| 90 | phys = args->v0.phys; | ||
| 91 | if (argc != (size >> page) * sizeof(args->v0.phys[0])) | ||
| 92 | return -EINVAL; | ||
| 93 | } else | ||
| 94 | return ret; | ||
| 95 | |||
| 96 | if (!client->super) | ||
| 97 | return -ENOENT; | ||
| 98 | |||
| 99 | if (size) { | ||
| 100 | mutex_lock(&vmm->mutex); | ||
| 101 | ret = nvkm_vmm_pfn_map(vmm, page, addr, size, phys); | ||
| 102 | mutex_unlock(&vmm->mutex); | ||
| 103 | } | ||
| 104 | |||
| 105 | return ret; | ||
| 106 | } | ||
| 107 | |||
| 45 | static int | 108 | static int |
| 46 | nvkm_uvmm_mthd_unmap(struct nvkm_uvmm *uvmm, void *argv, u32 argc) | 109 | nvkm_uvmm_mthd_unmap(struct nvkm_uvmm *uvmm, void *argv, u32 argc) |
| 47 | { | 110 | { |
| @@ -78,7 +141,7 @@ nvkm_uvmm_mthd_unmap(struct nvkm_uvmm *uvmm, void *argv, u32 argc) · 30 unmodified lines | |||
| 78 | goto done; | 141 | goto done; |
| 79 | } | 142 | } |
| 80 | 143 | ||
| 81 | nvkm_vmm_unmap_locked(vmm, vma); | 144 | nvkm_vmm_unmap_locked(vmm, vma, false); |
| 82 | ret = 0; | 145 | ret = 0; |
| 83 | done: | 146 | done: |
| 84 | mutex_unlock(&vmm->mutex); | 147 | mutex_unlock(&vmm->mutex); |
| @@ -124,6 +187,11 @@ nvkm_uvmm_mthd_map(struct nvkm_uvmm *uvmm, void *argv, u32 argc) · 39 unmodified lines | |||
| 124 | goto fail; | 187 | goto fail; |
| 125 | } | 188 | } |
| 126 | 189 | ||
| 190 | if (ret = -EINVAL, vma->mapped && !vma->memory) { | ||
| 191 | VMM_DEBUG(vmm, "pfnmap %016llx", addr); | ||
| 192 | goto fail; | ||
| 193 | } | ||
| 194 | |||
| 127 | if (ret = -EINVAL, vma->addr != addr || vma->size != size) { | 195 | if (ret = -EINVAL, vma->addr != addr || vma->size != size) { |
| 128 | if (addr + size > vma->addr + vma->size || vma->memory || | 196 | if (addr + size > vma->addr + vma->size || vma->memory || |
| 129 | (vma->refd == NVKM_VMA_PAGE_NONE && !vma->mapref)) { | 197 | (vma->refd == NVKM_VMA_PAGE_NONE && !vma->mapref)) { |
| @@ -271,6 +339,15 @@ nvkm_uvmm_mthd(struct nvkm_object *object, u32 mthd, void *argv, u32 argc) · 141 unmodified lines | |||
| 271 | case NVIF_VMM_V0_PUT : return nvkm_uvmm_mthd_put (uvmm, argv, argc); | 339 | case NVIF_VMM_V0_PUT : return nvkm_uvmm_mthd_put (uvmm, argv, argc); |
| 272 | case NVIF_VMM_V0_MAP : return nvkm_uvmm_mthd_map (uvmm, argv, argc); | 340 | case NVIF_VMM_V0_MAP : return nvkm_uvmm_mthd_map (uvmm, argv, argc); |
| 273 | case NVIF_VMM_V0_UNMAP : return nvkm_uvmm_mthd_unmap (uvmm, argv, argc); | 341 | case NVIF_VMM_V0_UNMAP : return nvkm_uvmm_mthd_unmap (uvmm, argv, argc); |
| 342 | case NVIF_VMM_V0_PFNMAP: return nvkm_uvmm_mthd_pfnmap(uvmm, argv, argc); | ||
| 343 | case NVIF_VMM_V0_PFNCLR: return nvkm_uvmm_mthd_pfnclr(uvmm, argv, argc); | ||
| 344 | case NVIF_VMM_V0_MTHD(0x00) ... NVIF_VMM_V0_MTHD(0x7f): | ||
| 345 | if (uvmm->vmm->func->mthd) { | ||
| 346 | return uvmm->vmm->func->mthd(uvmm->vmm, | ||
| 347 | uvmm->object.client, | ||
| 348 | mthd, argv, argc); | ||
| 349 | } | ||
| 350 | break; | ||
| 274 | default: | 351 | default: |
| 275 | break; | 352 | break; |
| 276 | } | 353 | } |
| @@ -304,8 +381,10 @@ nvkm_uvmm_new(const struct nvkm_oclass *oclass, void *argv, u32 argc, · 27 unmodified lines | |||
| 304 | struct nvkm_uvmm *uvmm; | 381 | struct nvkm_uvmm *uvmm; |
| 305 | int ret = -ENOSYS; | 382 | int ret = -ENOSYS; |
| 306 | u64 addr, size; | 383 | u64 addr, size; |
| 384 | bool managed; | ||
| 307 | 385 | ||
| 308 | if (!(ret = nvif_unpack(ret, &argv, &argc, args->v0, 0, 0, more))) { | 386 | if (!(ret = nvif_unpack(ret, &argv, &argc, args->v0, 0, 0, more))) { |
| 387 | managed = args->v0.managed != 0; | ||
| 309 | addr = args->v0.addr; | 388 | addr = args->v0.addr; |
| 310 | size = args->v0.size; | 389 | size = args->v0.size; |
| 311 | } else | 390 | } else |
| @@ -317,7 +396,7 @@ nvkm_uvmm_new(const struct nvkm_oclass *oclass, void *argv, u32 argc, · 5 unmodified lines | |||
| 317 | *pobject = &uvmm->object; | 396 | *pobject = &uvmm->object; |
| 318 | 397 | ||
| 319 | if (!mmu->vmm) { | 398 | if (!mmu->vmm) { |
| 320 | ret = mmu->func->vmm.ctor(mmu, addr, size, argv, argc, | 399 | ret = mmu->func->vmm.ctor(mmu, managed, addr, size, argv, argc, |
| 321 | NULL, "user", &uvmm->vmm); | 400 | NULL, "user", &uvmm->vmm); |
| 322 | if (ret) | 401 | if (ret) |
| 323 | return ret; | 402 | return ret; |
| @@ -255,11 +255,23 @@ nvkm_vmm_unref_sptes(struct nvkm_vmm_iter *it, struct nvkm_vmm_pt *pgt, · 254 unmodified lines | |||
| 255 | } | 255 | } |
| 256 | 256 | ||
| 257 | static bool | 257 | static bool |
| 258 | nvkm_vmm_unref_ptes(struct nvkm_vmm_iter *it, u32 ptei, u32 ptes) | 258 | nvkm_vmm_unref_ptes(struct nvkm_vmm_iter *it, bool pfn, u32 ptei, u32 ptes) |
| 259 | { | 259 | { |
| 260 | const struct nvkm_vmm_desc *desc = it->desc; | 260 | const struct nvkm_vmm_desc *desc = it->desc; |
| 261 | const int type = desc->type == SPT; | 261 | const int type = desc->type == SPT; |
| 262 | struct nvkm_vmm_pt *pgt = it->pt[0]; | 262 | struct nvkm_vmm_pt *pgt = it->pt[0]; |
| 263 | bool dma; | ||
| 264 | |||
| 265 | if (pfn) { | ||
| 266 | /* Need to clear PTE valid bits before we dma_unmap_page(). */ | ||
| 267 | dma = desc->func->pfn_clear(it->vmm, pgt->pt[type], ptei, ptes); | ||
| 268 | if (dma) { | ||
| 269 | /* GPU may have cached the PT, flush before unmap. */ | ||
| 270 | nvkm_vmm_flush_mark(it); | ||
| 271 | nvkm_vmm_flush(it); | ||
| 272 | desc->func->pfn_unmap(it->vmm, pgt->pt[type], ptei, ptes); | ||
| 273 | } | ||
| 274 | } | ||
| 263 | 275 | ||
| 264 | /* Drop PTE references. */ | 276 | /* Drop PTE references. */ |
| 265 | pgt->refs[type] -= ptes; | 277 | pgt->refs[type] -= ptes; |
| @@ -349,7 +361,7 @@ nvkm_vmm_ref_sptes(struct nvkm_vmm_iter *it, struct nvkm_vmm_pt *pgt, · 83 unmodified lines | |||
| 349 | } | 361 | } |
| 350 | 362 | ||
| 351 | static bool | 363 | static bool |
| 352 | nvkm_vmm_ref_ptes(struct nvkm_vmm_iter *it, u32 ptei, u32 ptes) | 364 | nvkm_vmm_ref_ptes(struct nvkm_vmm_iter *it, bool pfn, u32 ptei, u32 ptes) |
| 353 | { | 365 | { |
| 354 | const struct nvkm_vmm_desc *desc = it->desc; | 366 | const struct nvkm_vmm_desc *desc = it->desc; |
| 355 | const int type = desc->type == SPT; | 367 | const int type = desc->type == SPT; |
| @@ -379,7 +391,7 @@ nvkm_vmm_sparse_ptes(const struct nvkm_vmm_desc *desc, · 23 unmodified lines | |||
| 379 | } | 391 | } |
| 380 | 392 | ||
| 381 | static bool | 393 | static bool |
| 382 | nvkm_vmm_sparse_unref_ptes(struct nvkm_vmm_iter *it, u32 ptei, u32 ptes) | 394 | nvkm_vmm_sparse_unref_ptes(struct nvkm_vmm_iter *it, bool pfn, u32 ptei, u32 ptes) |
| 383 | { | 395 | { |
| 384 | struct nvkm_vmm_pt *pt = it->pt[0]; | 396 | struct nvkm_vmm_pt *pt = it->pt[0]; |
| 385 | if (it->desc->type == PGD) | 397 | if (it->desc->type == PGD) |
| @@ -387,14 +399,14 @@ nvkm_vmm_sparse_unref_ptes(struct nvkm_vmm_iter *it, u32 ptei, u32 ptes) · 1 unmodified line | |||
| 387 | else | 399 | else |
| 388 | if (it->desc->type == LPT) | 400 | if (it->desc->type == LPT) |
| 389 | memset(&pt->pte[ptei], 0x00, sizeof(pt->pte[0]) * ptes); | 401 | memset(&pt->pte[ptei], 0x00, sizeof(pt->pte[0]) * ptes); |
| 390 | return nvkm_vmm_unref_ptes(it, ptei, ptes); | 402 | return nvkm_vmm_unref_ptes(it, pfn, ptei, ptes); |
| 391 | } | 403 | } |
| 392 | 404 | ||
| 393 | static bool | 405 | static bool |
| 394 | nvkm_vmm_sparse_ref_ptes(struct nvkm_vmm_iter *it, u32 ptei, u32 ptes) | 406 | nvkm_vmm_sparse_ref_ptes(struct nvkm_vmm_iter *it, bool pfn, u32 ptei, u32 ptes) |
| 395 | { | 407 | { |
| 396 | nvkm_vmm_sparse_ptes(it->desc, it->pt[0], ptei, ptes); | 408 | nvkm_vmm_sparse_ptes(it->desc, it->pt[0], ptei, ptes); |
| 397 | return nvkm_vmm_ref_ptes(it, ptei, ptes); | 409 | return nvkm_vmm_ref_ptes(it, pfn, ptei, ptes); |
| 398 | } | 410 | } |
| 399 | 411 | ||
| 400 | static bool | 412 | static bool |
| @@ -487,8 +499,8 @@ nvkm_vmm_ref_swpt(struct nvkm_vmm_iter *it, struct nvkm_vmm_pt *pgd, u32 pdei) · 86 unmodified lines | |||
| 487 | 499 | ||
| 488 | static inline u64 | 500 | static inline u64 |
| 489 | nvkm_vmm_iter(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, | 501 | nvkm_vmm_iter(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, |
| 490 | u64 addr, u64 size, const char *name, bool ref, | 502 | u64 addr, u64 size, const char *name, bool ref, bool pfn, |
| 491 | bool (*REF_PTES)(struct nvkm_vmm_iter *, u32, u32), | 503 | bool (*REF_PTES)(struct nvkm_vmm_iter *, bool pfn, u32, u32), |
| 492 | nvkm_vmm_pte_func MAP_PTES, struct nvkm_vmm_map *map, | 504 | nvkm_vmm_pte_func MAP_PTES, struct nvkm_vmm_map *map, |
| 493 | nvkm_vmm_pxe_func CLR_PTES) | 505 | nvkm_vmm_pxe_func CLR_PTES) |
| 494 | { | 506 | { |
| @@ -548,7 +560,7 @@ nvkm_vmm_iter(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, · 53 unmodified lines | |||
| 548 | } | 560 | } |
| 549 | 561 | ||
| 550 | /* Handle PTE updates. */ | 562 | /* Handle PTE updates. */ |
| 551 | if (!REF_PTES || REF_PTES(&it, ptei, ptes)) { | 563 | if (!REF_PTES || REF_PTES(&it, pfn, ptei, ptes)) { |
| 552 | struct nvkm_mmu_pt *pt = pgt->pt[type]; | 564 | struct nvkm_mmu_pt *pt = pgt->pt[type]; |
| 553 | if (MAP_PTES || CLR_PTES) { | 565 | if (MAP_PTES || CLR_PTES) { |
| 554 | if (MAP_PTES) | 566 | if (MAP_PTES) |
| @@ -590,7 +602,7 @@ static void · 35 unmodified lines | |||
| 590 | nvkm_vmm_ptes_sparse_put(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, | 602 | nvkm_vmm_ptes_sparse_put(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, |
| 591 | u64 addr, u64 size) | 603 | u64 addr, u64 size) |
| 592 | { | 604 | { |
| 593 | nvkm_vmm_iter(vmm, page, addr, size, "sparse unref", false, | 605 | nvkm_vmm_iter(vmm, page, addr, size, "sparse unref", false, false, |
| 594 | nvkm_vmm_sparse_unref_ptes, NULL, NULL, | 606 | nvkm_vmm_sparse_unref_ptes, NULL, NULL, |
| 595 | page->desc->func->invalid ? | 607 | page->desc->func->invalid ? |
| 596 | page->desc->func->invalid : page->desc->func->unmap); | 608 | page->desc->func->invalid : page->desc->func->unmap); |
| @@ -602,8 +614,8 @@ nvkm_vmm_ptes_sparse_get(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, · 5 unmodified lines | |||
| 602 | { | 614 | { |
| 603 | if ((page->type & NVKM_VMM_PAGE_SPARSE)) { | 615 | if ((page->type & NVKM_VMM_PAGE_SPARSE)) { |
| 604 | u64 fail = nvkm_vmm_iter(vmm, page, addr, size, "sparse ref", | 616 | u64 fail = nvkm_vmm_iter(vmm, page, addr, size, "sparse ref", |
| 605 | true, nvkm_vmm_sparse_ref_ptes, NULL, | 617 | true, false, nvkm_vmm_sparse_ref_ptes, |
| 606 | NULL, page->desc->func->sparse); | 618 | NULL, NULL, page->desc->func->sparse); |
| 607 | if (fail != ~0ULL) { | 619 | if (fail != ~0ULL) { |
| 608 | if ((size = fail - addr)) | 620 | if ((size = fail - addr)) |
| 609 | nvkm_vmm_ptes_sparse_put(vmm, page, addr, size); | 621 | nvkm_vmm_ptes_sparse_put(vmm, page, addr, size); |
| @@ -666,11 +678,11 @@ nvkm_vmm_ptes_sparse(struct nvkm_vmm *vmm, u64 addr, u64 size, bool ref) · 56 unmodified lines | |||
| 666 | 678 | ||
| 667 | static void | 679 | static void |
| 668 | nvkm_vmm_ptes_unmap_put(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, | 680 | nvkm_vmm_ptes_unmap_put(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, |
| 669 | u64 addr, u64 size, bool sparse) | 681 | u64 addr, u64 size, bool sparse, bool pfn) |
| 670 | { | 682 | { |
| 671 | const struct nvkm_vmm_desc_func *func = page->desc->func; | 683 | const struct nvkm_vmm_desc_func *func = page->desc->func; |
| 672 | nvkm_vmm_iter(vmm, page, addr, size, "unmap + unref", | 684 | nvkm_vmm_iter(vmm, page, addr, size, "unmap + unref", |
| 673 | false, nvkm_vmm_unref_ptes, NULL, NULL, | 685 | false, pfn, nvkm_vmm_unref_ptes, NULL, NULL, |
| 674 | sparse ? func->sparse : func->invalid ? func->invalid : | 686 | sparse ? func->sparse : func->invalid ? func->invalid : |
| 675 | func->unmap); | 687 | func->unmap); |
| 676 | } | 688 | } |
| @@ -681,10 +693,10 @@ nvkm_vmm_ptes_get_map(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, · 4 unmodified lines | |||
| 681 | nvkm_vmm_pte_func func) | 693 | nvkm_vmm_pte_func func) |
| 682 | { | 694 | { |
| 683 | u64 fail = nvkm_vmm_iter(vmm, page, addr, size, "ref + map", true, | 695 | u64 fail = nvkm_vmm_iter(vmm, page, addr, size, "ref + map", true, |
| 684 | nvkm_vmm_ref_ptes, func, map, NULL); | 696 | false, nvkm_vmm_ref_ptes, func, map, NULL); |
| 685 | if (fail != ~0ULL) { | 697 | if (fail != ~0ULL) { |
| 686 | if ((size = fail - addr)) | 698 | if ((size = fail - addr)) |
| 687 | nvkm_vmm_ptes_unmap_put(vmm, page, addr, size, false); | 699 | nvkm_vmm_ptes_unmap_put(vmm, page, addr, size, false, false); |
| 688 | return -ENOMEM; | 700 | return -ENOMEM; |
| 689 | } | 701 | } |
| 690 | return 0; | 702 | return 0; |
| @@ -692,10 +704,11 @@ nvkm_vmm_ptes_get_map(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, · 1 unmodified line | |||
| 692 | 704 | ||
| 693 | static void | 705 | static void |
| 694 | nvkm_vmm_ptes_unmap(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, | 706 | nvkm_vmm_ptes_unmap(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, |
| 695 | u64 addr, u64 size, bool sparse) | 707 | u64 addr, u64 size, bool sparse, bool pfn) |
| 696 | { | 708 | { |
| 697 | const struct nvkm_vmm_desc_func *func = page->desc->func; | 709 | const struct nvkm_vmm_desc_func *func = page->desc->func; |
| 698 | nvkm_vmm_iter(vmm, page, addr, size, "unmap", false, NULL, NULL, NULL, | 710 | nvkm_vmm_iter(vmm, page, addr, size, "unmap", false, pfn, |
| 711 | NULL, NULL, NULL, | ||
| 699 | sparse ? func->sparse : func->invalid ? func->invalid : | 712 | sparse ? func->sparse : func->invalid ? func->invalid : |
| 700 | func->unmap); | 713 | func->unmap); |
| 701 | } | 714 | } |
| @@ -705,7 +718,7 @@ nvkm_vmm_ptes_map(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, · 3 unmodified lines | |||
| 705 | u64 addr, u64 size, struct nvkm_vmm_map *map, | 718 | u64 addr, u64 size, struct nvkm_vmm_map *map, |
| 706 | nvkm_vmm_pte_func func) | 719 | nvkm_vmm_pte_func func) |
| 707 | { | 720 | { |
| 708 | nvkm_vmm_iter(vmm, page, addr, size, "map", false, | 721 | nvkm_vmm_iter(vmm, page, addr, size, "map", false, false, |
| 709 | NULL, func, map, NULL); | 722 | NULL, func, map, NULL); |
| 710 | } | 723 | } |
| 711 | 724 | ||
| @@ -713,7 +726,7 @@ static void · 1 unmodified line | |||
| 713 | nvkm_vmm_ptes_put(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, | 726 | nvkm_vmm_ptes_put(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, |
| 714 | u64 addr, u64 size) | 727 | u64 addr, u64 size) |
| 715 | { | 728 | { |
| 716 | nvkm_vmm_iter(vmm, page, addr, size, "unref", false, | 729 | nvkm_vmm_iter(vmm, page, addr, size, "unref", false, false, |
| 717 | nvkm_vmm_unref_ptes, NULL, NULL, NULL); | 730 | nvkm_vmm_unref_ptes, NULL, NULL, NULL); |
| 718 | } | 731 | } |
| 719 | 732 | ||
| @@ -721,7 +734,7 @@ static int · 1 unmodified line | |||
| 721 | nvkm_vmm_ptes_get(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, | 734 | nvkm_vmm_ptes_get(struct nvkm_vmm *vmm, const struct nvkm_vmm_page *page, |
| 722 | u64 addr, u64 size) | 735 | u64 addr, u64 size) |
| 723 | { | 736 | { |
| 724 | u64 fail = nvkm_vmm_iter(vmm, page, addr, size, "ref", true, | 737 | u64 fail = nvkm_vmm_iter(vmm, page, addr, size, "ref", true, false, |
| 725 | nvkm_vmm_ref_ptes, NULL, NULL, NULL); | 738 | nvkm_vmm_ref_ptes, NULL, NULL, NULL); |
| 726 | if (fail != ~0ULL) { | 739 | if (fail != ~0ULL) { |
| 727 | if (fail != addr) | 740 | if (fail != addr) |
| @@ -763,6 +776,7 @@ nvkm_vma_tail(struct nvkm_vma *vma, u64 tail) · 35 unmodified lines | |||
| 763 | new->part = vma->part; | 776 | new->part = vma->part; |
| 764 | new->user = vma->user; | 777 | new->user = vma->user; |
| 765 | new->busy = vma->busy; | 778 | new->busy = vma->busy; |
| 779 | new->mapped = vma->mapped; | ||
| 766 | list_add(&new->head, &vma->head); | 780 | list_add(&new->head, &vma->head); |
| 767 | return new; | 781 | return new; |
| 768 | } | 782 | } |
| @@ -934,12 +948,41 @@ nvkm_vmm_node_split(struct nvkm_vmm *vmm, · 165 unmodified lines | |||
| 934 | return vma; | 948 | return vma; |
| 935 | } | 949 | } |
| 936 | 950 | ||
| 951 | static void | ||
| 952 | nvkm_vma_dump(struct nvkm_vma *vma) | ||
| 953 | { | ||
| 954 | printk(KERN_ERR "%016llx %016llx %c%c%c%c%c%c%c%c%c %p\n", | ||
| 955 | vma->addr, (u64)vma->size, | ||
| 956 | vma->used ? '-' : 'F', | ||
| 957 | vma->mapref ? 'R' : '-', | ||
| 958 | vma->sparse ? 'S' : '-', | ||
| 959 | vma->page != NVKM_VMA_PAGE_NONE ? '0' + vma->page : '-', | ||
| 960 | vma->refd != NVKM_VMA_PAGE_NONE ? '0' + vma->refd : '-', | ||
| 961 | vma->part ? 'P' : '-', | ||
| 962 | vma->user ? 'U' : '-', | ||
| 963 | vma->busy ? 'B' : '-', | ||
| 964 | vma->mapped ? 'M' : '-', | ||
| 965 | vma->memory); | ||
| 966 | } | ||
| 967 | |||
| 968 | static void | ||
| 969 | nvkm_vmm_dump(struct nvkm_vmm *vmm) | ||
| 970 | { | ||
| 971 | struct nvkm_vma *vma; | ||
| 972 | list_for_each_entry(vma, &vmm->list, head) { | ||
| 973 | nvkm_vma_dump(vma); | ||
| 974 | } | ||
| 975 | } | ||
| 976 | |||
| 937 | static void | 977 | static void |
| 938 | nvkm_vmm_dtor(struct nvkm_vmm *vmm) | 978 | nvkm_vmm_dtor(struct nvkm_vmm *vmm) |
| 939 | { | 979 | { |
| 940 | struct nvkm_vma *vma; | 980 | struct nvkm_vma *vma; |
| 941 | struct rb_node *node; | 981 | struct rb_node *node; |
| 942 | 982 | ||
| 983 | if (0) | ||
| 984 | nvkm_vmm_dump(vmm); | ||
| 985 | |||
| 943 | while ((node = rb_first(&vmm->root))) { | 986 | while ((node = rb_first(&vmm->root))) { |
| 944 | struct nvkm_vma *vma = rb_entry(node, typeof(*vma), tree); | 987 | struct nvkm_vma *vma = rb_entry(node, typeof(*vma), tree); |
| 945 | nvkm_vmm_put(vmm, &vma); | 988 | nvkm_vmm_put(vmm, &vma); |
| @@ -972,16 +1015,32 @@ nvkm_vmm_dtor(struct nvkm_vmm *vmm) · 26 unmodified lines | |||
| 972 | } | 1015 | } |
| 973 | } | 1016 | } |
| 974 | 1017 | ||
| 1018 | static int | ||
| 1019 | nvkm_vmm_ctor_managed(struct nvkm_vmm *vmm, u64 addr, u64 size) | ||
| 1020 | { | ||
| 1021 | struct nvkm_vma *vma; | ||
| 1022 | if (!(vma = nvkm_vma_new(addr, size))) | ||
| 1023 | return -ENOMEM; | ||
| 1024 | vma->mapref = true; | ||
| 1025 | vma->sparse = false; | ||
| 1026 | vma->used = true; | ||
| 1027 | vma->user = true; | ||
| 1028 | nvkm_vmm_node_insert(vmm, vma); | ||
| 1029 | list_add_tail(&vma->head, &vmm->list); | ||
| 1030 | return 0; | ||
| 1031 | } | ||
| 1032 | |||
| 975 | int | 1033 | int |
| 976 | nvkm_vmm_ctor(const struct nvkm_vmm_func *func, struct nvkm_mmu *mmu, | 1034 | nvkm_vmm_ctor(const struct nvkm_vmm_func *func, struct nvkm_mmu *mmu, |
| 977 | u32 pd_header, u64 addr, u64 size, struct lock_class_key *key, | 1035 | u32 pd_header, bool managed, u64 addr, u64 size, |
| 978 | const char *name, struct nvkm_vmm *vmm) | 1036 | struct lock_class_key *key, const char *name, |
| 1037 | struct nvkm_vmm *vmm) | ||
| 979 | { | 1038 | { |
| 980 | static struct lock_class_key _key; | 1039 | static struct lock_class_key _key; |
| 981 | const struct nvkm_vmm_page *page = func->page; | 1040 | const struct nvkm_vmm_page *page = func->page; |
| 982 | const struct nvkm_vmm_desc *desc; | 1041 | const struct nvkm_vmm_desc *desc; |
| 983 | struct nvkm_vma *vma; | 1042 | struct nvkm_vma *vma; |
| 984 | int levels, bits = 0; | 1043 | int levels, bits = 0, ret; |
| 985 | 1044 | ||
| 986 | vmm->func = func; | 1045 | vmm->func = func; |
| 987 | vmm->mmu = mmu; | 1046 | vmm->mmu = mmu; |
| @@ -1009,11 +1068,6 @@ nvkm_vmm_ctor(const struct nvkm_vmm_func *func, struct nvkm_mmu *mmu, · 21 unmodified lines | |||
| 1009 | if (WARN_ON(levels > NVKM_VMM_LEVELS_MAX)) | 1068 | if (WARN_ON(levels > NVKM_VMM_LEVELS_MAX)) |
| 1010 | return -EINVAL; | 1069 | return -EINVAL; |
| 1011 | 1070 | ||
| 1012 | vmm->start = addr; | ||
| 1013 | vmm->limit = size ? (addr + size) : (1ULL << bits); | ||
| 1014 | if (vmm->start > vmm->limit || vmm->limit > (1ULL << bits)) | ||
| 1015 | return -EINVAL; | ||
| 1016 | |||
| 1017 | /* Allocate top-level page table. */ | 1071 | /* Allocate top-level page table. */ |
| 1018 | vmm->pd = nvkm_vmm_pt_new(desc, false, NULL); | 1072 | vmm->pd = nvkm_vmm_pt_new(desc, false, NULL); |
| 1019 | if (!vmm->pd) | 1073 | if (!vmm->pd) |
| @@ -1036,50 +1090,273 @@ nvkm_vmm_ctor(const struct nvkm_vmm_func *func, struct nvkm_mmu *mmu, · 16 unmodified lines | |||
| 1036 | vmm->free = RB_ROOT; | 1090 | vmm->free = RB_ROOT; |
| 1037 | vmm->root = RB_ROOT; | 1091 | vmm->root = RB_ROOT; |
| 1038 | 1092 | ||
| 1039 | if (!(vma = nvkm_vma_new(vmm->start, vmm->limit - vmm->start))) | 1093 | if (managed) { |
| 1040 | return -ENOMEM; | 1094 | /* Address-space will be managed by the client for the most |
| 1095 | * part, except for a specified area where NVKM allocations | ||
| 1096 | * are allowed to be placed. | ||
| 1097 | */ | ||
| 1098 | vmm->start = 0; | ||
| 1099 | vmm->limit = 1ULL << bits; | ||
| 1100 | if (addr + size < addr || addr + size > vmm->limit) | ||
| 1101 | return -EINVAL; | ||
| 1102 | |||
| 1103 | /* Client-managed area before the NVKM-managed area. */ | ||
| 1104 | if (addr && (ret = nvkm_vmm_ctor_managed(vmm, 0, addr))) | ||
| 1105 | return ret; | ||
| 1106 | |||
| 1107 | /* NVKM-managed area. */ | ||
| 1108 | if (size) { | ||
| 1109 | if (!(vma = nvkm_vma_new(addr, size))) | ||
| 1110 | return -ENOMEM; | ||
| 1111 | nvkm_vmm_free_insert(vmm, vma); | ||
| 1112 | list_add_tail(&vma->head, &vmm->list); | ||
| 1113 | } | ||
| 1114 | |||
| 1115 | /* Client-managed area after the NVKM-managed area. */ | ||
| 1116 | addr = addr + size; | ||
| 1117 | size = vmm->limit - addr; | ||
| 1118 | if (size && (ret = nvkm_vmm_ctor_managed(vmm, addr, size))) | ||
| 1119 | return ret; | ||
| 1120 | } else { | ||
| 1121 | /* Address-space fully managed by NVKM, requiring calls to | ||
| 1122 | * nvkm_vmm_get()/nvkm_vmm_put() to allocate address-space. | ||
| 1123 | */ | ||
| 1124 | vmm->start = addr; | ||
| 1125 | vmm->limit = size ? (addr + size) : (1ULL << bits); | ||
| 1126 | if (vmm->start > vmm->limit || vmm->limit > (1ULL << bits)) | ||
| 1127 | return -EINVAL; | ||
| 1128 | |||
| 1129 | if (!(vma = nvkm_vma_new(vmm->start, vmm->limit - vmm->start))) | ||
| 1130 | return -ENOMEM; | ||
| 1131 | |||
| 1132 | nvkm_vmm_free_insert(vmm, vma); | ||
| 1133 | list_add(&vma->head, &vmm->list); | ||
| 1134 | } | ||
| 1041 | 1135 | ||
| 1042 | nvkm_vmm_free_insert(vmm, vma); | ||
| 1043 | list_add(&vma->head, &vmm->list); | ||
| 1044 | return 0; | 1136 | return 0; |
| 1045 | } | 1137 | } |
| 1046 | 1138 | ||
| 1047 | int | 1139 | int |
| 1048 | nvkm_vmm_new_(const struct nvkm_vmm_func *func, struct nvkm_mmu *mmu, | 1140 | nvkm_vmm_new_(const struct nvkm_vmm_func *func, struct nvkm_mmu *mmu, |
| 1049 | u32 hdr, u64 addr, u64 size, struct lock_class_key *key, | 1141 | u32 hdr, bool managed, u64 addr, u64 size, |
| 1050 | const char *name, struct nvkm_vmm **pvmm) | 1142 | struct lock_class_key *key, const char *name, |
| 1143 | struct nvkm_vmm **pvmm) | ||
| 1051 | { | 1144 | { |
| 1052 | if (!(*pvmm = kzalloc(sizeof(**pvmm), GFP_KERNEL))) | 1145 | if (!(*pvmm = kzalloc(sizeof(**pvmm), GFP_KERNEL))) |
| 1053 | return -ENOMEM; | 1146 | return -ENOMEM; |
| 1054 | return nvkm_vmm_ctor(func, mmu, hdr, addr, size, key, name, *pvmm); | 1147 | return nvkm_vmm_ctor(func, mmu, hdr, managed, addr, size, key, name, *pvmm); |
| 1148 | } | ||
| 1149 | |||
| 1150 | static struct nvkm_vma * | ||
| 1151 | nvkm_vmm_pfn_split_merge(struct nvkm_vmm *vmm, struct nvkm_vma *vma, | ||
| 1152 | u64 addr, u64 size, u8 page, bool map) | ||
| 1153 | { | ||
| 1154 | struct nvkm_vma *prev = NULL; | ||
| 1155 | struct nvkm_vma *next = NULL; | ||
| 1156 | |||
| 1157 | if (vma->addr == addr && vma->part && (prev = node(vma, prev))) { | ||
| 1158 | if (prev->memory || prev->mapped != map) | ||
| 1159 | prev = NULL; | ||
| 1160 | } | ||
| 1161 | |||
| 1162 | if (vma->addr + vma->size == addr + size && (next = node(vma, next))) { | ||
| 1163 | if (!next->part || | ||
| 1164 | next->memory || next->mapped != map) | ||
| 1165 | next = NULL; | ||
| 1166 | } | ||
| 1167 | |||
| 1168 | if (prev || next) | ||
| 1169 | return nvkm_vmm_node_merge(vmm, prev, vma, next, size); | ||
| 1170 | return nvkm_vmm_node_split(vmm, vma, addr, size); | ||
| 1171 | } | ||
| 1172 | |||
| 1173 | int | ||
| 1174 | nvkm_vmm_pfn_unmap(struct nvkm_vmm *vmm, u64 addr, u64 size) | ||
| 1175 | { | ||
| 1176 | struct nvkm_vma *vma = nvkm_vmm_node_search(vmm, addr); | ||
| 1177 | struct nvkm_vma *next; | ||
| 1178 | u64 limit = addr + size; | ||
| 1179 | u64 start = addr; | ||
| 1180 | |||
| 1181 | if (!vma) | ||
| 1182 | return -EINVAL; | ||
| 1183 | |||
| 1184 | do { | ||
| 1185 | if (!vma->mapped || vma->memory) | ||
| 1186 | continue; | ||
| 1187 | |||
| 1188 | size = min(limit - start, vma->size - (start - vma->addr)); | ||
| 1189 | |||
| 1190 | nvkm_vmm_ptes_unmap_put(vmm, &vmm->func->page[vma->refd], | ||
| 1191 | start, size, false, true); | ||
| 1192 | |||
| 1193 | next = nvkm_vmm_pfn_split_merge(vmm, vma, start, size, 0, false); | ||
| 1194 | if (!WARN_ON(!next)) { | ||
| 1195 | vma = next; | ||
| 1196 | vma->refd = NVKM_VMA_PAGE_NONE; | ||
| 1197 | vma->mapped = false; | ||
| 1198 | } | ||
| 1199 | } while ((vma = node(vma, next)) && (start = vma->addr) < limit); | ||
| 1200 | |||
| 1201 | return 0; | ||
| 1202 | } | ||
| 1203 | |||
| 1204 | /*TODO: | ||
| 1205 | * - Avoid PT readback (for dma_unmap etc), this might end up being dealt | ||
| 1206 | * with inside HMM, which would be a lot nicer for us to deal with. | ||
| 1207 | * - Multiple page sizes (particularly for huge page support). | ||
| 1208 | * - Support for systems without a 4KiB page size. | ||
| 1209 | */ | ||
| 1210 | int | ||
| 1211 | nvkm_vmm_pfn_map(struct nvkm_vmm *vmm, u8 shift, u64 addr, u64 size, u64 *pfn) | ||
| 1212 | { | ||
| 1213 | const struct nvkm_vmm_page *page = vmm->func->page; | ||
| 1214 | struct nvkm_vma *vma, *tmp; | ||
| 1215 | u64 limit = addr + size; | ||
| 1216 | u64 start = addr; | ||
| 1217 | int pm = size >> shift; | ||
| 1218 | int pi = 0; | ||
| 1219 | |||
| 1220 | /* Only support mapping where the page size of the incoming page | ||
| 1221 | * array matches a page size available for direct mapping. | ||
| 1222 | */ | ||
| 1223 | while (page->shift && page->shift != shift && | ||
| 1224 | page->desc->func->pfn == NULL) | ||
| 1225 | page++; | ||
| 1226 | |||
| 1227 | if (!page->shift || !IS_ALIGNED(addr, 1ULL << shift) || | ||
| 1228 | !IS_ALIGNED(size, 1ULL << shift) || | ||
| 1229 | addr + size < addr || addr + size > vmm->limit) { | ||
| 1230 | VMM_DEBUG(vmm, "paged map %d %d %016llx %016llx\n", | ||
| 1231 | shift, page->shift, addr, size); | ||
| 1232 | return -EINVAL; | ||
| 1233 | } | ||
| 1234 | |||
| 1235 | if (!(vma = nvkm_vmm_node_search(vmm, addr))) | ||
| 1236 | return -ENOENT; | ||
| 1237 | |||
| 1238 | do { | ||
| 1239 | bool map = !!(pfn[pi] & NVKM_VMM_PFN_V); | ||
| 1240 | bool mapped = vma->mapped; | ||
| 1241 | u64 size = limit - start; | ||
| 1242 | u64 addr = start; | ||
| 1243 | int pn, ret = 0; | ||
| 1244 | |||
| 1245 | /* Narrow the operation window to cover a single action (page | ||
| 1246 | * should be mapped or not) within a single VMA. | ||
| 1247 | */ | ||
| 1248 | for (pn = 0; pi + pn < pm; pn++) { | ||
| 1249 | if (map != !!(pfn[pi + pn] & NVKM_VMM_PFN_V)) | ||
| 1250 | break; | ||
| 1251 | } | ||
| 1252 | size = min_t(u64, size, pn << page->shift); | ||
| 1253 | size = min_t(u64, size, vma->size + vma->addr - addr); | ||
| 1254 | |||
| 1255 | /* Reject any operation to unmanaged regions, and areas that | ||
| 1256 | * have nvkm_memory objects mapped in them already. | ||
| 1257 | */ | ||
| 1258 | if (!vma->mapref || vma->memory) { | ||
| 1259 | ret = -EINVAL; | ||
| 1260 | goto next; | ||
| 1261 | } | ||
| 1262 | |||
| 1263 | /* In order to both properly refcount GPU page tables, and | ||
| 1264 | * prevent "normal" mappings and these direct mappings from | ||
| 1265 | * interfering with each other, we need to track contiguous | ||
| 1266 | * ranges that have been mapped with this interface. | ||
| 1267 | * | ||
| 1268 | * Here we attempt to either split an existing VMA so we're | ||
| 1269 | * able to flag the region as either unmapped/mapped, or to | ||
| 1270 | * merge with adjacent VMAs that are already compatible. | ||
| 1271 | * | ||
| 1272 | * If the region is already compatible, nothing is required. | ||
| 1273 | */ | ||
| 1274 | if (map != mapped) { | ||
| 1275 | tmp = nvkm_vmm_pfn_split_merge(vmm, vma, addr, size, | ||
| 1276 | page - | ||
| 1277 | vmm->func->page, map); | ||
| 1278 | if (WARN_ON(!tmp)) { | ||
| 1279 | ret = -ENOMEM; | ||
| 1280 | goto next; | ||
| 1281 | } | ||
| 1282 | |||
| 1283 | if ((tmp->mapped = map)) | ||
| 1284 | tmp->refd = page - vmm->func->page; | ||
| 1285 | else | ||
| 1286 | tmp->refd = NVKM_VMA_PAGE_NONE; | ||
| 1287 | vma = tmp; | ||
| 1288 | } | ||
| 1289 | |||
| 1290 | /* Update HW page tables. */ | ||
| 1291 | if (map) { | ||
| 1292 | struct nvkm_vmm_map args; | ||
| 1293 | args.page = page; | ||
| 1294 | args.pfn = &pfn[pi]; | ||
| 1295 | |||
| 1296 | if (!mapped) { | ||
| 1297 | ret = nvkm_vmm_ptes_get_map(vmm, page, addr, | ||
| 1298 | size, &args, page-> | ||
| 1299 | desc->func->pfn); | ||
| 1300 | } else { | ||
| 1301 | nvkm_vmm_ptes_map(vmm, page, addr, size, &args, | ||
| 1302 | page->desc->func->pfn); | ||
| 1303 | } | ||
| 1304 | } else { | ||
| 1305 | if (mapped) { | ||
| 1306 | nvkm_vmm_ptes_unmap_put(vmm, page, addr, size, | ||
| 1307 | false, true); | ||
| 1308 | } | ||
| 1309 | } | ||
| 1310 | |||
| 1311 | next: | ||
| 1312 | /* Iterate to next operation. */ | ||
| 1313 | if (vma->addr + vma->size == addr + size) | ||
| 1314 | vma = node(vma, next); | ||
| 1315 | start += size; | ||
| 1316 | |||
| 1317 | if (ret) { | ||
| 1318 | /* Failure is signalled by clearing the valid bit on | ||
| 1319 | * any PFN that couldn't be modified as requested. | ||
| 1320 | */ | ||
| 1321 | while (size) { | ||
| 1322 | pfn[pi++] = NVKM_VMM_PFN_NONE; | ||
| 1323 | size -= 1 << page->shift; | ||
| 1324 | } | ||
| 1325 | } else { | ||
| 1326 | pi += size >> page->shift; | ||
| 1327 | } | ||
| 1328 | } while (vma && start < limit); | ||
| 1329 | |||
| 1330 | return 0; | ||
| 1055 | } | 1331 | } |
| 1056 | 1332 | ||
| 1057 | void | 1333 | void |
| 1058 | nvkm_vmm_unmap_region(struct nvkm_vmm *vmm, struct nvkm_vma *vma) | 1334 | nvkm_vmm_unmap_region(struct nvkm_vmm *vmm, struct nvkm_vma *vma) |
| 1059 | { | 1335 | { |
| 1060 | struct nvkm_vma *next = node(vma, next); | ||
| 1061 | struct nvkm_vma *prev = NULL; | 1336 | struct nvkm_vma *prev = NULL; |
| 1337 | struct nvkm_vma *next; | ||
| 1062 | 1338 | ||
| 1063 | nvkm_memory_tags_put(vma->memory, vmm->mmu->subdev.device, &vma->tags); | 1339 | nvkm_memory_tags_put(vma->memory, vmm->mmu->subdev.device, &vma->tags); |
| 1064 | nvkm_memory_unref(&vma->memory); | 1340 | nvkm_memory_unref(&vma->memory); |
| 1341 | vma->mapped = false; | ||
| 1065 | 1342 | ||
| 1066 | if (!vma->part || ((prev = node(vma, prev)), prev->memory)) | 1343 | if (vma->part && (prev = node(vma, prev)) && prev->mapped) |
| 1067 | prev = NULL; | 1344 | prev = NULL; |
| 1068 | if (!next->part || next->memory) | 1345 | if ((next = node(vma, next)) && (!next->part || next->mapped)) |
| 1069 | next = NULL; | 1346 | next = NULL; |
| 1070 | nvkm_vmm_node_merge(vmm, prev, vma, next, vma->size); | 1347 | nvkm_vmm_node_merge(vmm, prev, vma, next, vma->size); |
| 1071 | } | 1348 | } |
| 1072 | 1349 | ||
| 1073 | void | 1350 | void |
| 1074 | nvkm_vmm_unmap_locked(struct nvkm_vmm *vmm, struct nvkm_vma *vma) | 1351 | nvkm_vmm_unmap_locked(struct nvkm_vmm *vmm, struct nvkm_vma *vma, bool pfn) |
| 1075 | { | 1352 | { |
| 1076 | const struct nvkm_vmm_page *page = &vmm->func->page[vma->refd]; | 1353 | const struct nvkm_vmm_page *page = &vmm->func->page[vma->refd]; |
| 1077 | 1354 | ||
| 1078 | if (vma->mapref) { | 1355 | if (vma->mapref) { |
| 1079 | nvkm_vmm_ptes_unmap_put(vmm, page, vma->addr, vma->size, vma->sparse); | 1356 | nvkm_vmm_ptes_unmap_put(vmm, page, vma->addr, vma->size, vma->sparse, pfn); |
| 1080 | vma->refd = NVKM_VMA_PAGE_NONE; | 1357 | vma->refd = NVKM_VMA_PAGE_NONE; |
| 1081 | } else { | 1358 | } else { |
| 1082 | nvkm_vmm_ptes_unmap(vmm, page, vma->addr, vma->size, vma->sparse); | 1359 | nvkm_vmm_ptes_unmap(vmm, page, vma->addr, vma->size, vma->sparse, pfn); |
| 1083 | } | 1360 | } |
| 1084 | 1361 | ||
| 1085 | nvkm_vmm_unmap_region(vmm, vma); | 1362 | nvkm_vmm_unmap_region(vmm, vma); |
| @@ -1090,7 +1367,7 @@ nvkm_vmm_unmap(struct nvkm_vmm *vmm, struct nvkm_vma *vma) · 4 unmodified lines | |||
| 1090 | { | 1367 | { |
| 1091 | if (vma->memory) { | 1368 | if (vma->memory) { |
| 1092 | mutex_lock(&vmm->mutex); | 1369 | mutex_lock(&vmm->mutex); |
| 1093 | nvkm_vmm_unmap_locked(vmm, vma); | 1370 | nvkm_vmm_unmap_locked(vmm, vma, false); |
| 1094 | mutex_unlock(&vmm->mutex); | 1371 | mutex_unlock(&vmm->mutex); |
| 1095 | } | 1372 | } |
| 1096 | } | 1373 | } |
| @@ -1224,6 +1501,7 @@ nvkm_vmm_map_locked(struct nvkm_vmm *vmm, struct nvkm_vma *vma, · 127 unmodified lines | |||
| 1224 | nvkm_memory_tags_put(vma->memory, vmm->mmu->subdev.device, &vma->tags); | 1501 | nvkm_memory_tags_put(vma->memory, vmm->mmu->subdev.device, &vma->tags); |
| 1225 | nvkm_memory_unref(&vma->memory); | 1502 | nvkm_memory_unref(&vma->memory); |
| 1226 | vma->memory = nvkm_memory_ref(map->memory); | 1503 | vma->memory = nvkm_memory_ref(map->memory); |
| 1504 | vma->mapped = true; | ||
| 1227 | vma->tags = map->tags; | 1505 | vma->tags = map->tags; |
| 1228 | return 0; | 1506 | return 0; |
| 1229 | } | 1507 | } |
| @@ -1269,14 +1547,16 @@ nvkm_vmm_put_locked(struct nvkm_vmm *vmm, struct nvkm_vma *vma) · 39 unmodified lines | |||
| 1269 | 1547 | ||
| 1270 | if (vma->mapref || !vma->sparse) { | 1548 | if (vma->mapref || !vma->sparse) { |
| 1271 | do { | 1549 | do { |
| 1272 | const bool map = next->memory != NULL; | 1550 | const bool mem = next->memory != NULL; |
| 1551 | const bool map = next->mapped; | ||
| 1273 | const u8 refd = next->refd; | 1552 | const u8 refd = next->refd; |
| 1274 | const u64 addr = next->addr; | 1553 | const u64 addr = next->addr; |
| 1275 | u64 size = next->size; | 1554 | u64 size = next->size; |
| 1276 | 1555 | ||
| 1277 | /* Merge regions that are in the same state. */ | 1556 | /* Merge regions that are in the same state. */ |
| 1278 | while ((next = node(next, next)) && next->part && | 1557 | while ((next = node(next, next)) && next->part && |
| 1279 | (next->memory != NULL) == map && | 1558 | (next->mapped == map) && |
| 1559 | (next->memory != NULL) == mem && | ||
| 1280 | (next->refd == refd)) | 1560 | (next->refd == refd)) |
| 1281 | size += next->size; | 1561 | size += next->size; |
| 1282 | 1562 | ||
| @@ -1286,7 +1566,8 @@ nvkm_vmm_put_locked(struct nvkm_vmm *vmm, struct nvkm_vma *vma) · 3 unmodified lines | |||
| 1286 | * the page tree. | 1566 | * the page tree. |
| 1287 | */ | 1567 | */ |
| 1288 | nvkm_vmm_ptes_unmap_put(vmm, &page[refd], addr, | 1568 | nvkm_vmm_ptes_unmap_put(vmm, &page[refd], addr, |
| 1289 | size, vma->sparse); | 1569 | size, vma->sparse, |
| 1570 | !mem); | ||
| 1290 | } else | 1571 | } else |
| 1291 | if (refd != NVKM_VMA_PAGE_NONE) { | 1572 | if (refd != NVKM_VMA_PAGE_NONE) { |
| 1292 | /* Drop allocation-time PTE references. */ | 1573 | /* Drop allocation-time PTE references. */ |
| @@ -1301,7 +1582,7 @@ nvkm_vmm_put_locked(struct nvkm_vmm *vmm, struct nvkm_vma *vma) · 8 unmodified lines | |||
| 1301 | */ | 1582 | */ |
| 1302 | next = vma; | 1583 | next = vma; |
| 1303 | do { | 1584 | do { |
| 1304 | if (next->memory) | 1585 | if (next->mapped) |
| 1305 | nvkm_vmm_unmap_region(vmm, next); | 1586 | nvkm_vmm_unmap_region(vmm, next); |
| 1306 | } while ((next = node(vma, next)) && next->part); | 1587 | } while ((next = node(vma, next)) && next->part); |
| 1307 | 1588 | ||
| @@ -1522,7 +1803,7 @@ nvkm_vmm_join(struct nvkm_vmm *vmm, struct nvkm_memory *inst) · 214 unmodified lines | |||
| 1522 | } | 1803 | } |
| 1523 | 1804 | ||
| 1524 | static bool | 1805 | static bool |
| 1525 | nvkm_vmm_boot_ptes(struct nvkm_vmm_iter *it, u32 ptei, u32 ptes) | 1806 | nvkm_vmm_boot_ptes(struct nvkm_vmm_iter *it, bool pfn, u32 ptei, u32 ptes) |
| 1526 | { | 1807 | { |
| 1527 | const struct nvkm_vmm_desc *desc = it->desc; | 1808 | const struct nvkm_vmm_desc *desc = it->desc; |
| 1528 | const int type = desc->type == SPT; | 1809 | const int type = desc->type == SPT; |
| @@ -1544,7 +1825,7 @@ nvkm_vmm_boot(struct nvkm_vmm *vmm) · 15 unmodified lines | |||
| 1544 | if (ret) | 1825 | if (ret) |
| 1545 | return ret; | 1826 | return ret; |
| 1546 | 1827 | ||
| 1547 | nvkm_vmm_iter(vmm, page, vmm->start, limit, "bootstrap", false, | 1828 | nvkm_vmm_iter(vmm, page, vmm->start, limit, "bootstrap", false, false, |
| 1548 | nvkm_vmm_boot_ptes, NULL, NULL, NULL); | 1829 | nvkm_vmm_boot_ptes, NULL, NULL, NULL); |
| 1549 | vmm->bootstrapped = true; | 1830 | vmm->bootstrapped = true; |
| 1550 | return 0; | 1831 | return 0; |
| @@ -1584,7 +1865,8 @@ nvkm_vmm_new(struct nvkm_device *device, u64 addr, u64 size, void *argv, · 33 unmodified lines | |||
| 1584 | struct nvkm_mmu *mmu = device->mmu; | 1865 | struct nvkm_mmu *mmu = device->mmu; |
| 1585 | struct nvkm_vmm *vmm = NULL; | 1866 | struct nvkm_vmm *vmm = NULL; |
| 1586 | int ret; | 1867 | int ret; |
| 1587 | ret = mmu->func->vmm.ctor(mmu, addr, size, argv, argc, key, name, &vmm); | 1868 | ret = mmu->func->vmm.ctor(mmu, false, addr, size, argv, argc, |
| 1869 | key, name, &vmm); | ||
| 1588 | if (ret) | 1870 | if (ret) |
| 1589 | nvkm_vmm_unref(&vmm); | 1871 | nvkm_vmm_unref(&vmm); |
| 1590 | *pvmm = vmm; | 1872 | *pvmm = vmm; |
| @@ -67,6 +67,10 @@ struct nvkm_vmm_desc_func { · 66 unmodified lines | |||
| 67 | nvkm_vmm_pte_func mem; | 67 | nvkm_vmm_pte_func mem; |
| 68 | nvkm_vmm_pte_func dma; | 68 | nvkm_vmm_pte_func dma; |
| 69 | nvkm_vmm_pte_func sgl; | 69 | nvkm_vmm_pte_func sgl; |
| 70 | |||
| 71 | nvkm_vmm_pte_func pfn; | ||
| 72 | bool (*pfn_clear)(struct nvkm_vmm *, struct nvkm_mmu_pt *, u32 ptei, u32 ptes); | ||
| 73 | nvkm_vmm_pxe_func pfn_unmap; | ||
| 70 | }; | 74 | }; |
| 71 | 75 | ||
| 72 | extern const struct nvkm_vmm_desc_func gf100_vmm_pgd; | 76 | extern const struct nvkm_vmm_desc_func gf100_vmm_pgd; |
| @@ -141,6 +145,11 @@ struct nvkm_vmm_func { · 68 unmodified lines | |||
| 141 | struct nvkm_vmm_map *); | 145 | struct nvkm_vmm_map *); |
| 142 | void (*flush)(struct nvkm_vmm *, int depth); | 146 | void (*flush)(struct nvkm_vmm *, int depth); |
| 143 | 147 | ||
| 148 | int (*mthd)(struct nvkm_vmm *, struct nvkm_client *, | ||
| 149 | u32 mthd, void *argv, u32 argc); | ||
| 150 | |||
| 151 | void (*invalidate_pdb)(struct nvkm_vmm *, u64 addr); | ||
| 152 | |||
| 144 | u64 page_block; | 153 | u64 page_block; |
| 145 | const struct nvkm_vmm_page page[]; | 154 | const struct nvkm_vmm_page page[]; |
| 146 | }; | 155 | }; |
| @@ -151,11 +160,12 @@ struct nvkm_vmm_join { · 4 unmodified lines | |||
| 151 | }; | 160 | }; |
| 152 | 161 | ||
| 153 | int nvkm_vmm_new_(const struct nvkm_vmm_func *, struct nvkm_mmu *, | 162 | int nvkm_vmm_new_(const struct nvkm_vmm_func *, struct nvkm_mmu *, |
| 154 | u32 pd_header, u64 addr, u64 size, struct lock_class_key *, | 163 | u32 pd_header, bool managed, u64 addr, u64 size, |
| 155 | const char *name, struct nvkm_vmm **); | 164 | struct lock_class_key *, const char *name, |
| 165 | struct nvkm_vmm **); | ||
| 156 | int nvkm_vmm_ctor(const struct nvkm_vmm_func *, struct nvkm_mmu *, | 166 | int nvkm_vmm_ctor(const struct nvkm_vmm_func *, struct nvkm_mmu *, |
| 157 | u32 pd_header, u64 addr, u64 size, struct lock_class_key *, | 167 | u32 pd_header, bool managed, u64 addr, u64 size, |
| 158 | const char *name, struct nvkm_vmm *); | 168 | struct lock_class_key *, const char *name, struct nvkm_vmm *); |
| 159 | struct nvkm_vma *nvkm_vmm_node_search(struct nvkm_vmm *, u64 addr); | 169 | struct nvkm_vma *nvkm_vmm_node_search(struct nvkm_vmm *, u64 addr); |
| 160 | struct nvkm_vma *nvkm_vmm_node_split(struct nvkm_vmm *, struct nvkm_vma *, | 170 | struct nvkm_vma *nvkm_vmm_node_split(struct nvkm_vmm *, struct nvkm_vma *, |
| 161 | u64 addr, u64 size); | 171 | u64 addr, u64 size); |
| @@ -163,13 +173,25 @@ int nvkm_vmm_get_locked(struct nvkm_vmm *, bool getref, bool mapref, · 1 unmodified line | |||
| 163 | bool sparse, u8 page, u8 align, u64 size, | 173 | bool sparse, u8 page, u8 align, u64 size, |
| 164 | struct nvkm_vma **pvma); | 174 | struct nvkm_vma **pvma); |
| 165 | void nvkm_vmm_put_locked(struct nvkm_vmm *, struct nvkm_vma *); | 175 | void nvkm_vmm_put_locked(struct nvkm_vmm *, struct nvkm_vma *); |
| 166 | void nvkm_vmm_unmap_locked(struct nvkm_vmm *, struct nvkm_vma *); | 176 | void nvkm_vmm_unmap_locked(struct nvkm_vmm *, struct nvkm_vma *, bool pfn); |
| 167 | void nvkm_vmm_unmap_region(struct nvkm_vmm *vmm, struct nvkm_vma *vma); | 177 | void nvkm_vmm_unmap_region(struct nvkm_vmm *, struct nvkm_vma *); |
| 178 | |||
| 179 | #define NVKM_VMM_PFN_ADDR 0xfffffffffffff000ULL | ||
| 180 | #define NVKM_VMM_PFN_ADDR_SHIFT 12 | ||
| 181 | #define NVKM_VMM_PFN_APER 0x00000000000000f0ULL | ||
| 182 | #define NVKM_VMM_PFN_HOST 0x0000000000000000ULL | ||
| 183 | #define NVKM_VMM_PFN_VRAM 0x0000000000000010ULL | ||
| 184 | #define NVKM_VMM_PFN_W 0x0000000000000002ULL | ||
| 185 | #define NVKM_VMM_PFN_V 0x0000000000000001ULL | ||
| 186 | #define NVKM_VMM_PFN_NONE 0x0000000000000000ULL | ||
| 187 | |||
| 188 | int nvkm_vmm_pfn_map(struct nvkm_vmm *, u8 page, u64 addr, u64 size, u64 *pfn); | ||
| 189 | int nvkm_vmm_pfn_unmap(struct nvkm_vmm *, u64 addr, u64 size); | ||
| 168 | 190 | ||
| 169 | struct nvkm_vma *nvkm_vma_tail(struct nvkm_vma *, u64 tail); | 191 | struct nvkm_vma *nvkm_vma_tail(struct nvkm_vma *, u64 tail); |
| 170 | 192 | ||
| 171 | int nv04_vmm_new_(const struct nvkm_vmm_func *, struct nvkm_mmu *, u32, | 193 | int nv04_vmm_new_(const struct nvkm_vmm_func *, struct nvkm_mmu *, u32, |
| 172 | u64, u64, void *, u32, struct lock_class_key *, | 194 | bool, u64, u64, void *, u32, struct lock_class_key *, |
| 173 | const char *, struct nvkm_vmm **); | 195 | const char *, struct nvkm_vmm **); |
| 174 | int nv04_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *); | 196 | int nv04_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *); |
| 175 | 197 | ||
| @@ -179,70 +201,76 @@ int nv50_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *); · 3 unmodified lines | |||
| 179 | void nv50_vmm_flush(struct nvkm_vmm *, int); | 201 | void nv50_vmm_flush(struct nvkm_vmm *, int); |
| 180 | 202 | ||
| 181 | int gf100_vmm_new_(const struct nvkm_vmm_func *, const struct nvkm_vmm_func *, | 203 | int gf100_vmm_new_(const struct nvkm_vmm_func *, const struct nvkm_vmm_func *, |
| 182 | struct nvkm_mmu *, u64, u64, void *, u32, | 204 | struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 183 | struct lock_class_key *, const char *, struct nvkm_vmm **); | 205 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 184 | int gf100_vmm_join_(struct nvkm_vmm *, struct nvkm_memory *, u64 base); | 206 | int gf100_vmm_join_(struct nvkm_vmm *, struct nvkm_memory *, u64 base); |
| 185 | int gf100_vmm_join(struct nvkm_vmm *, struct nvkm_memory *); | 207 | int gf100_vmm_join(struct nvkm_vmm *, struct nvkm_memory *); |
| 186 | void gf100_vmm_part(struct nvkm_vmm *, struct nvkm_memory *); | 208 | void gf100_vmm_part(struct nvkm_vmm *, struct nvkm_memory *); |
| 187 | int gf100_vmm_aper(enum nvkm_memory_target); | 209 | int gf100_vmm_aper(enum nvkm_memory_target); |
| 188 | int gf100_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *); | 210 | int gf100_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *); |
| 189 | void gf100_vmm_flush_(struct nvkm_vmm *, int); | ||
| 190 | void gf100_vmm_flush(struct nvkm_vmm *, int); | 211 | void gf100_vmm_flush(struct nvkm_vmm *, int); |
| 212 | void gf100_vmm_invalidate(struct nvkm_vmm *, u32 type); | ||
| 213 | void gf100_vmm_invalidate_pdb(struct nvkm_vmm *, u64 addr); | ||
| 191 | 214 | ||
| 192 | int gk20a_vmm_aper(enum nvkm_memory_target); | 215 | int gk20a_vmm_aper(enum nvkm_memory_target); |
| 193 | 216 | ||
| 194 | int gm200_vmm_new_(const struct nvkm_vmm_func *, const struct nvkm_vmm_func *, | 217 | int gm200_vmm_new_(const struct nvkm_vmm_func *, const struct nvkm_vmm_func *, |
| 195 | struct nvkm_mmu *, u64, u64, void *, u32, | 218 | struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 196 | struct lock_class_key *, const char *, struct nvkm_vmm **); | 219 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 197 | int gm200_vmm_join_(struct nvkm_vmm *, struct nvkm_memory *, u64 base); | 220 | int gm200_vmm_join_(struct nvkm_vmm *, struct nvkm_memory *, u64 base); |
| 198 | int gm200_vmm_join(struct nvkm_vmm *, struct nvkm_memory *); | 221 | int gm200_vmm_join(struct nvkm_vmm *, struct nvkm_memory *); |
| 199 | 222 | ||
| 223 | int gp100_vmm_new_(const struct nvkm_vmm_func *, | ||
| 224 | struct nvkm_mmu *, bool, u64, u64, void *, u32, | ||
| 225 | struct lock_class_key *, const char *, struct nvkm_vmm **); | ||
| 200 | int gp100_vmm_join(struct nvkm_vmm *, struct nvkm_memory *); | 226 | int gp100_vmm_join(struct nvkm_vmm *, struct nvkm_memory *); |
| 201 | int gp100_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *); | 227 | int gp100_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *); |
| 202 | void gp100_vmm_flush(struct nvkm_vmm *, int); | 228 | void gp100_vmm_flush(struct nvkm_vmm *, int); |
| 229 | int gp100_vmm_mthd(struct nvkm_vmm *, struct nvkm_client *, u32, void *, u32); | ||
| 230 | void gp100_vmm_invalidate_pdb(struct nvkm_vmm *, u64 addr); | ||
| 203 | 231 | ||
| 204 | int gv100_vmm_join(struct nvkm_vmm *, struct nvkm_memory *); | 232 | int gv100_vmm_join(struct nvkm_vmm *, struct nvkm_memory *); |
| 205 | 233 | ||
| 206 | int nv04_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | 234 | int nv04_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 207 | struct lock_class_key *, const char *, struct nvkm_vmm **); | 235 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 208 | int nv41_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | 236 | int nv41_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 209 | struct lock_class_key *, const char *, struct nvkm_vmm **); | 237 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 210 | int nv44_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | 238 | int nv44_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 211 | struct lock_class_key *, const char *, struct nvkm_vmm **); | 239 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 212 | int nv50_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | 240 | int nv50_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 213 | struct lock_class_key *, const char *, struct nvkm_vmm **); | 241 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 214 | int mcp77_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | 242 | int mcp77_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 215 | struct lock_class_key *, const char *, struct nvkm_vmm **); | 243 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 216 | int g84_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | 244 | int g84_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 217 | struct lock_class_key *, const char *, struct nvkm_vmm **); | 245 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 218 | int gf100_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | 246 | int gf100_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 219 | struct lock_class_key *, const char *, struct nvkm_vmm **); | 247 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 220 | int gk104_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | 248 | int gk104_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 221 | struct lock_class_key *, const char *, struct nvkm_vmm **); | 249 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 222 | int gk20a_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | 250 | int gk20a_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 223 | struct lock_class_key *, const char *, struct nvkm_vmm **); | 251 | struct lock_class_key *, const char *, struct nvkm_vmm **); |
| 224 | int gm200_vmm_new_fixed(struct nvkm_mmu *, u64, u64, void *, u32, | 252 | int gm200_vmm_new_fixed(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 225 | struct lock_class_key *, const char *, | 253 | struct lock_class_key *, const char *, |
| 226 | struct nvkm_vmm **); | 254 | struct nvkm_vmm **); |
| 227 | int gm200_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | 255 | int gm200_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 228 | struct lock_class_key *, const char *, | 256 | struct lock_class_key *, const char *, |
| 229 | struct nvkm_vmm **); | 257 | struct nvkm_vmm **); |
| 230 | int gm20b_vmm_new_fixed(struct nvkm_mmu *, u64, u64, void *, u32, | 258 | int gm20b_vmm_new_fixed(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 231 | struct lock_class_key *, const char *, | 259 | struct lock_class_key *, const char *, |
| 232 | struct nvkm_vmm **); | 260 | struct nvkm_vmm **); |
| 233 | int gm20b_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | 261 | int gm20b_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 234 | struct lock_class_key *, const char *, | 262 | struct lock_class_key *, const char *, |
| 235 | struct nvkm_vmm **); | 263 | struct nvkm_vmm **); |
| 236 | int gp100_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | 264 | int gp100_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 237 | struct lock_class_key *, const char *, | 265 | struct lock_class_key *, const char *, |
| 238 | struct nvkm_vmm **); | 266 | struct nvkm_vmm **); |
| 239 | int gp10b_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | 267 | int gp10b_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 240 | struct lock_class_key *, const char *, | 268 | struct lock_class_key *, const char *, |
| 241 | struct nvkm_vmm **); | 269 | struct nvkm_vmm **); |
| 242 | int gv100_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | 270 | int gv100_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 243 | struct lock_class_key *, const char *, | 271 | struct lock_class_key *, const char *, |
| 244 | struct nvkm_vmm **); | 272 | struct nvkm_vmm **); |
| 245 | int tu104_vmm_new(struct nvkm_mmu *, u64, u64, void *, u32, | 273 | int tu102_vmm_new(struct nvkm_mmu *, bool, u64, u64, void *, u32, |
| 246 | struct lock_class_key *, const char *, | 274 | struct lock_class_key *, const char *, |
| 247 | struct nvkm_vmm **); | 275 | struct nvkm_vmm **); |
| 248 | 276 | ||
| @@ -178,15 +178,19 @@ gf100_vmm_desc_16_16[] = { · 177 unmodified lines | |||
| 178 | }; | 178 | }; |
| 179 | 179 | ||
| 180 | void | 180 | void |
| 181 | gf100_vmm_flush_(struct nvkm_vmm *vmm, int depth) | 181 | gf100_vmm_invalidate_pdb(struct nvkm_vmm *vmm, u64 addr) |
| 182 | { | ||
| 183 | struct nvkm_device *device = vmm->mmu->subdev.device; | ||
| 184 | nvkm_wr32(device, 0x100cb8, addr); | ||
| 185 | } | ||
| 186 | |||
| 187 | void | ||
| 188 | gf100_vmm_invalidate(struct nvkm_vmm *vmm, u32 type) | ||
| 182 | { | 189 | { |
| 183 | struct nvkm_subdev *subdev = &vmm->mmu->subdev; | 190 | struct nvkm_subdev *subdev = &vmm->mmu->subdev; |
| 184 | struct nvkm_device *device = subdev->device; | 191 | struct nvkm_device *device = subdev->device; |
| 185 | u32 type = depth << 24; | 192 | struct nvkm_mmu_pt *pd = vmm->pd->pt[0]; |
| 186 | 193 | u64 addr = 0; | |
| 187 | type = 0x00000001; /* PAGE_ALL */ | ||
| 188 | if (atomic_read(&vmm->engref[NVKM_SUBDEV_BAR])) | ||
| 189 | type |= 0x00000004; /* HUB_ONLY */ | ||
| 190 | 194 | ||
| 191 | mutex_lock(&subdev->mutex); | 195 | mutex_lock(&subdev->mutex); |
| 192 | /* Looks like maybe a "free flush slots" counter, the | 196 | /* Looks like maybe a "free flush slots" counter, the |
| @@ -197,7 +201,20 @@ gf100_vmm_flush_(struct nvkm_vmm *vmm, int depth) · 4 unmodified lines | |||
| 197 | break; | 201 | break; |
| 198 | ); | 202 | ); |
| 199 | 203 | ||
| 200 | nvkm_wr32(device, 0x100cb8, vmm->pd->pt[0]->addr >> 8); | 204 | if (!(type & 0x00000002) /* ALL_PDB. */) { |
| 205 | switch (nvkm_memory_target(pd->memory)) { | ||
| 206 | case NVKM_MEM_TARGET_VRAM: addr |= 0x00000000; break; | ||
| 207 | case NVKM_MEM_TARGET_HOST: addr |= 0x00000002; break; | ||
| 208 | case NVKM_MEM_TARGET_NCOH: addr |= 0x00000003; break; | ||
| 209 | default: | ||
| 210 | WARN_ON(1); | ||
| 211 | break; | ||
| 212 | } | ||
| 213 | addr |= (vmm->pd->pt[0]->addr >> 12) << 4; | ||
| 214 | |||
| 215 | vmm->func->invalidate_pdb(vmm, addr); | ||
| 216 | } | ||
| 217 | |||
| 201 | nvkm_wr32(device, 0x100cbc, 0x80000000 | type); | 218 | nvkm_wr32(device, 0x100cbc, 0x80000000 | type); |
| 202 | 219 | ||
| 203 | /* Wait for flush to be queued? */ | 220 | /* Wait for flush to be queued? */ |
| @@ -211,7 +228,10 @@ gf100_vmm_flush_(struct nvkm_vmm *vmm, int depth) · 7 unmodified lines | |||
| 211 | void | 228 | void |
| 212 | gf100_vmm_flush(struct nvkm_vmm *vmm, int depth) | 229 | gf100_vmm_flush(struct nvkm_vmm *vmm, int depth) |
| 213 | { | 230 | { |
| 214 | gf100_vmm_flush_(vmm, 0); | 231 | u32 type = 0x00000001; /* PAGE_ALL */ |
| 232 | if (atomic_read(&vmm->engref[NVKM_SUBDEV_BAR])) | ||
| 233 | type |= 0x00000004; /* HUB_ONLY */ | ||
| 234 | gf100_vmm_invalidate(vmm, type); | ||
| 215 | } | 235 | } |
| 216 | 236 | ||
| 217 | int | 237 | int |
| @@ -354,6 +374,7 @@ gf100_vmm_17 = { · 136 unmodified lines | |||
| 354 | .aper = gf100_vmm_aper, | 374 | .aper = gf100_vmm_aper, |
| 355 | .valid = gf100_vmm_valid, | 375 | .valid = gf100_vmm_valid, |
| 356 | .flush = gf100_vmm_flush, | 376 | .flush = gf100_vmm_flush, |
| 377 | .invalidate_pdb = gf100_vmm_invalidate_pdb, | ||
| 357 | .page = { | 378 | .page = { |
| 358 | { 17, &gf100_vmm_desc_17_17[0], NVKM_VMM_PAGE_xVxC }, | 379 | { 17, &gf100_vmm_desc_17_17[0], NVKM_VMM_PAGE_xVxC }, |
| 359 | { 12, &gf100_vmm_desc_17_12[0], NVKM_VMM_PAGE_xVHx }, | 380 | { 12, &gf100_vmm_desc_17_12[0], NVKM_VMM_PAGE_xVHx }, |
| @@ -368,6 +389,7 @@ gf100_vmm_16 = { · 8 unmodified lines | |||
| 368 | .aper = gf100_vmm_aper, | 389 | .aper = gf100_vmm_aper, |
| 369 | .valid = gf100_vmm_valid, | 390 | .valid = gf100_vmm_valid, |
| 370 | .flush = gf100_vmm_flush, | 391 | .flush = gf100_vmm_flush, |
| 392 | .invalidate_pdb = gf100_vmm_invalidate_pdb, | ||
| 371 | .page = { | 393 | .page = { |
| 372 | { 16, &gf100_vmm_desc_16_16[0], NVKM_VMM_PAGE_xVxC }, | 394 | { 16, &gf100_vmm_desc_16_16[0], NVKM_VMM_PAGE_xVxC }, |
| 373 | { 12, &gf100_vmm_desc_16_12[0], NVKM_VMM_PAGE_xVHx }, | 395 | { 12, &gf100_vmm_desc_16_12[0], NVKM_VMM_PAGE_xVHx }, |
| @@ -378,14 +400,14 @@ gf100_vmm_16 = { · 4 unmodified lines | |||
| 378 | int | 400 | int |
| 379 | gf100_vmm_new_(const struct nvkm_vmm_func *func_16, | 401 | gf100_vmm_new_(const struct nvkm_vmm_func *func_16, |
| 380 | const struct nvkm_vmm_func *func_17, | 402 | const struct nvkm_vmm_func *func_17, |
| 381 | struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | 403 | struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 382 | struct lock_class_key *key, const char *name, | 404 | void *argv, u32 argc, struct lock_class_key *key, |
| 383 | struct nvkm_vmm **pvmm) | 405 | const char *name, struct nvkm_vmm **pvmm) |
| 384 | { | 406 | { |
| 385 | switch (mmu->subdev.device->fb->page) { | 407 | switch (mmu->subdev.device->fb->page) { |
| 386 | case 16: return nv04_vmm_new_(func_16, mmu, 0, addr, size, | 408 | case 16: return nv04_vmm_new_(func_16, mmu, 0, managed, addr, size, |
| 387 | argv, argc, key, name, pvmm); | 409 | argv, argc, key, name, pvmm); |
| 388 | case 17: return nv04_vmm_new_(func_17, mmu, 0, addr, size, | 410 | case 17: return nv04_vmm_new_(func_17, mmu, 0, managed, addr, size, |
| 389 | argv, argc, key, name, pvmm); | 411 | argv, argc, key, name, pvmm); |
| 390 | default: | 412 | default: |
| 391 | WARN_ON(1); | 413 | WARN_ON(1); |
| @@ -394,10 +416,10 @@ gf100_vmm_new_(const struct nvkm_vmm_func *func_16, · 2 unmodified lines | |||
| 394 | } | 416 | } |
| 395 | 417 | ||
| 396 | int | 418 | int |
| 397 | gf100_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | 419 | gf100_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 398 | struct lock_class_key *key, const char *name, | 420 | void *argv, u32 argc, struct lock_class_key *key, |
| 399 | struct nvkm_vmm **pvmm) | 421 | const char *name, struct nvkm_vmm **pvmm) |
| 400 | { | 422 | { |
| 401 | return gf100_vmm_new_(&gf100_vmm_16, &gf100_vmm_17, mmu, addr, | 423 | return gf100_vmm_new_(&gf100_vmm_16, &gf100_vmm_17, mmu, managed, addr, |
| 402 | size, argv, argc, key, name, pvmm); | 424 | size, argv, argc, key, name, pvmm); |
| 403 | } | 425 | } |
| @@ -71,6 +71,7 @@ gk104_vmm_17 = { · 70 unmodified lines | |||
| 71 | .aper = gf100_vmm_aper, | 71 | .aper = gf100_vmm_aper, |
| 72 | .valid = gf100_vmm_valid, | 72 | .valid = gf100_vmm_valid, |
| 73 | .flush = gf100_vmm_flush, | 73 | .flush = gf100_vmm_flush, |
| 74 | .invalidate_pdb = gf100_vmm_invalidate_pdb, | ||
| 74 | .page = { | 75 | .page = { |
| 75 | { 17, &gk104_vmm_desc_17_17[0], NVKM_VMM_PAGE_xVxC }, | 76 | { 17, &gk104_vmm_desc_17_17[0], NVKM_VMM_PAGE_xVxC }, |
| 76 | { 12, &gk104_vmm_desc_17_12[0], NVKM_VMM_PAGE_xVHx }, | 77 | { 12, &gk104_vmm_desc_17_12[0], NVKM_VMM_PAGE_xVHx }, |
| @@ -85,6 +86,7 @@ gk104_vmm_16 = { · 8 unmodified lines | |||
| 85 | .aper = gf100_vmm_aper, | 86 | .aper = gf100_vmm_aper, |
| 86 | .valid = gf100_vmm_valid, | 87 | .valid = gf100_vmm_valid, |
| 87 | .flush = gf100_vmm_flush, | 88 | .flush = gf100_vmm_flush, |
| 89 | .invalidate_pdb = gf100_vmm_invalidate_pdb, | ||
| 88 | .page = { | 90 | .page = { |
| 89 | { 16, &gk104_vmm_desc_16_16[0], NVKM_VMM_PAGE_xVxC }, | 91 | { 16, &gk104_vmm_desc_16_16[0], NVKM_VMM_PAGE_xVxC }, |
| 90 | { 12, &gk104_vmm_desc_16_12[0], NVKM_VMM_PAGE_xVHx }, | 92 | { 12, &gk104_vmm_desc_16_12[0], NVKM_VMM_PAGE_xVHx }, |
| @@ -93,10 +95,10 @@ gk104_vmm_16 = { · 2 unmodified lines | |||
| 93 | }; | 95 | }; |
| 94 | 96 | ||
| 95 | int | 97 | int |
| 96 | gk104_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | 98 | gk104_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 97 | struct lock_class_key *key, const char *name, | 99 | void *argv, u32 argc, struct lock_class_key *key, |
| 98 | struct nvkm_vmm **pvmm) | 100 | const char *name, struct nvkm_vmm **pvmm) |
| 99 | { | 101 | { |
| 100 | return gf100_vmm_new_(&gk104_vmm_16, &gk104_vmm_17, mmu, addr, | 102 | return gf100_vmm_new_(&gk104_vmm_16, &gk104_vmm_17, mmu, managed, addr, |
| 101 | size, argv, argc, key, name, pvmm); | 103 | size, argv, argc, key, name, pvmm); |
| 102 | } | 104 | } |
| @@ -40,6 +40,7 @@ gk20a_vmm_17 = { · 39 unmodified lines | |||
| 40 | .aper = gf100_vmm_aper, | 40 | .aper = gf100_vmm_aper, |
| 41 | .valid = gf100_vmm_valid, | 41 | .valid = gf100_vmm_valid, |
| 42 | .flush = gf100_vmm_flush, | 42 | .flush = gf100_vmm_flush, |
| 43 | .invalidate_pdb = gf100_vmm_invalidate_pdb, | ||
| 43 | .page = { | 44 | .page = { |
| 44 | { 17, &gk104_vmm_desc_17_17[0], NVKM_VMM_PAGE_xxHC }, | 45 | { 17, &gk104_vmm_desc_17_17[0], NVKM_VMM_PAGE_xxHC }, |
| 45 | { 12, &gk104_vmm_desc_17_12[0], NVKM_VMM_PAGE_xxHx }, | 46 | { 12, &gk104_vmm_desc_17_12[0], NVKM_VMM_PAGE_xxHx }, |
| @@ -54,6 +55,7 @@ gk20a_vmm_16 = { · 8 unmodified lines | |||
| 54 | .aper = gf100_vmm_aper, | 55 | .aper = gf100_vmm_aper, |
| 55 | .valid = gf100_vmm_valid, | 56 | .valid = gf100_vmm_valid, |
| 56 | .flush = gf100_vmm_flush, | 57 | .flush = gf100_vmm_flush, |
| 58 | .invalidate_pdb = gf100_vmm_invalidate_pdb, | ||
| 57 | .page = { | 59 | .page = { |
| 58 | { 16, &gk104_vmm_desc_16_16[0], NVKM_VMM_PAGE_xxHC }, | 60 | { 16, &gk104_vmm_desc_16_16[0], NVKM_VMM_PAGE_xxHC }, |
| 59 | { 12, &gk104_vmm_desc_16_12[0], NVKM_VMM_PAGE_xxHx }, | 61 | { 12, &gk104_vmm_desc_16_12[0], NVKM_VMM_PAGE_xxHx }, |
| @@ -62,10 +64,10 @@ gk20a_vmm_16 = { · 2 unmodified lines | |||
| 62 | }; | 64 | }; |
| 63 | 65 | ||
| 64 | int | 66 | int |
| 65 | gk20a_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | 67 | gk20a_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 66 | struct lock_class_key *key, const char *name, | 68 | void *argv, u32 argc, struct lock_class_key *key, |
| 67 | struct nvkm_vmm **pvmm) | 69 | const char *name, struct nvkm_vmm **pvmm) |
| 68 | { | 70 | { |
| 69 | return gf100_vmm_new_(&gk20a_vmm_16, &gk20a_vmm_17, mmu, addr, | 71 | return gf100_vmm_new_(&gk20a_vmm_16, &gk20a_vmm_17, mmu, managed, addr, |
| 70 | size, argv, argc, key, name, pvmm); | 72 | size, argv, argc, key, name, pvmm); |
| 71 | } | 73 | } |
| @@ -113,6 +113,7 @@ gm200_vmm_17 = { · 112 unmodified lines | |||
| 113 | .aper = gf100_vmm_aper, | 113 | .aper = gf100_vmm_aper, |
| 114 | .valid = gf100_vmm_valid, | 114 | .valid = gf100_vmm_valid, |
| 115 | .flush = gf100_vmm_flush, | 115 | .flush = gf100_vmm_flush, |
| 116 | .invalidate_pdb = gf100_vmm_invalidate_pdb, | ||
| 116 | .page = { | 117 | .page = { |
| 117 | { 27, &gm200_vmm_desc_17_17[1], NVKM_VMM_PAGE_Sxxx }, | 118 | { 27, &gm200_vmm_desc_17_17[1], NVKM_VMM_PAGE_Sxxx }, |
| 118 | { 17, &gm200_vmm_desc_17_17[0], NVKM_VMM_PAGE_SVxC }, | 119 | { 17, &gm200_vmm_desc_17_17[0], NVKM_VMM_PAGE_SVxC }, |
| @@ -128,6 +129,7 @@ gm200_vmm_16 = { · 9 unmodified lines | |||
| 128 | .aper = gf100_vmm_aper, | 129 | .aper = gf100_vmm_aper, |
| 129 | .valid = gf100_vmm_valid, | 130 | .valid = gf100_vmm_valid, |
| 130 | .flush = gf100_vmm_flush, | 131 | .flush = gf100_vmm_flush, |
| 132 | .invalidate_pdb = gf100_vmm_invalidate_pdb, | ||
| 131 | .page = { | 133 | .page = { |
| 132 | { 27, &gm200_vmm_desc_16_16[1], NVKM_VMM_PAGE_Sxxx }, | 134 | { 27, &gm200_vmm_desc_16_16[1], NVKM_VMM_PAGE_Sxxx }, |
| 133 | { 16, &gm200_vmm_desc_16_16[0], NVKM_VMM_PAGE_SVxC }, | 135 | { 16, &gm200_vmm_desc_16_16[0], NVKM_VMM_PAGE_SVxC }, |
| @@ -139,9 +141,9 @@ gm200_vmm_16 = { · 5 unmodified lines | |||
| 139 | int | 141 | int |
| 140 | gm200_vmm_new_(const struct nvkm_vmm_func *func_16, | 142 | gm200_vmm_new_(const struct nvkm_vmm_func *func_16, |
| 141 | const struct nvkm_vmm_func *func_17, | 143 | const struct nvkm_vmm_func *func_17, |
| 142 | struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | 144 | struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 143 | struct lock_class_key *key, const char *name, | 145 | void *argv, u32 argc, struct lock_class_key *key, |
| 144 | struct nvkm_vmm **pvmm) | 146 | const char *name, struct nvkm_vmm **pvmm) |
| 145 | { | 147 | { |
| 146 | const struct nvkm_vmm_func *func; | 148 | const struct nvkm_vmm_func *func; |
| 147 | union { | 149 | union { |
| @@ -163,23 +165,23 @@ gm200_vmm_new_(const struct nvkm_vmm_func *func_16, · 15 unmodified lines | |||
| 163 | } else | 165 | } else |
| 164 | return ret; | 166 | return ret; |
| 165 | 167 | ||
| 166 | return nvkm_vmm_new_(func, mmu, 0, addr, size, key, name, pvmm); | 168 | return nvkm_vmm_new_(func, mmu, 0, managed, addr, size, key, name, pvmm); |
| 167 | } | 169 | } |
| 168 | 170 | ||
| 169 | int | 171 | int |
| 170 | gm200_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | 172 | gm200_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 171 | struct lock_class_key *key, const char *name, | 173 | void *argv, u32 argc, struct lock_class_key *key, |
| 172 | struct nvkm_vmm **pvmm) | 174 | const char *name, struct nvkm_vmm **pvmm) |
| 173 | { | 175 | { |
| 174 | return gm200_vmm_new_(&gm200_vmm_16, &gm200_vmm_17, mmu, addr, | 176 | return gm200_vmm_new_(&gm200_vmm_16, &gm200_vmm_17, mmu, managed, addr, |
| 175 | size, argv, argc, key, name, pvmm); | 177 | size, argv, argc, key, name, pvmm); |
| 176 | } | 178 | } |
| 177 | 179 | ||
| 178 | int | 180 | int |
| 179 | gm200_vmm_new_fixed(struct nvkm_mmu *mmu, u64 addr, u64 size, | 181 | gm200_vmm_new_fixed(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 180 | void *argv, u32 argc, struct lock_class_key *key, | 182 | void *argv, u32 argc, struct lock_class_key *key, |
| 181 | const char *name, struct nvkm_vmm **pvmm) | 183 | const char *name, struct nvkm_vmm **pvmm) |
| 182 | { | 184 | { |
| 183 | return gf100_vmm_new_(&gm200_vmm_16, &gm200_vmm_17, mmu, addr, | 185 | return gf100_vmm_new_(&gm200_vmm_16, &gm200_vmm_17, mmu, managed, addr, |
| 184 | size, argv, argc, key, name, pvmm); | 186 | size, argv, argc, key, name, pvmm); |
| 185 | } | 187 | } |
| @@ -28,6 +28,7 @@ gm20b_vmm_17 = { · 27 unmodified lines | |||
| 28 | .aper = gk20a_vmm_aper, | 28 | .aper = gk20a_vmm_aper, |
| 29 | .valid = gf100_vmm_valid, | 29 | .valid = gf100_vmm_valid, |
| 30 | .flush = gf100_vmm_flush, | 30 | .flush = gf100_vmm_flush, |
| 31 | .invalidate_pdb = gf100_vmm_invalidate_pdb, | ||
| 31 | .page = { | 32 | .page = { |
| 32 | { 27, &gm200_vmm_desc_17_17[1], NVKM_VMM_PAGE_Sxxx }, | 33 | { 27, &gm200_vmm_desc_17_17[1], NVKM_VMM_PAGE_Sxxx }, |
| 33 | { 17, &gm200_vmm_desc_17_17[0], NVKM_VMM_PAGE_SxHC }, | 34 | { 17, &gm200_vmm_desc_17_17[0], NVKM_VMM_PAGE_SxHC }, |
| @@ -43,6 +44,7 @@ gm20b_vmm_16 = { · 9 unmodified lines | |||
| 43 | .aper = gk20a_vmm_aper, | 44 | .aper = gk20a_vmm_aper, |
| 44 | .valid = gf100_vmm_valid, | 45 | .valid = gf100_vmm_valid, |
| 45 | .flush = gf100_vmm_flush, | 46 | .flush = gf100_vmm_flush, |
| 47 | .invalidate_pdb = gf100_vmm_invalidate_pdb, | ||
| 46 | .page = { | 48 | .page = { |
| 47 | { 27, &gm200_vmm_desc_16_16[1], NVKM_VMM_PAGE_Sxxx }, | 49 | { 27, &gm200_vmm_desc_16_16[1], NVKM_VMM_PAGE_Sxxx }, |
| 48 | { 16, &gm200_vmm_desc_16_16[0], NVKM_VMM_PAGE_SxHC }, | 50 | { 16, &gm200_vmm_desc_16_16[0], NVKM_VMM_PAGE_SxHC }, |
| @@ -52,19 +54,19 @@ gm20b_vmm_16 = { · 3 unmodified lines | |||
| 52 | }; | 54 | }; |
| 53 | 55 | ||
| 54 | int | 56 | int |
| 55 | gm20b_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | 57 | gm20b_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 56 | struct lock_class_key *key, const char *name, | 58 | void *argv, u32 argc, struct lock_class_key *key, |
| 57 | struct nvkm_vmm **pvmm) | 59 | const char *name, struct nvkm_vmm **pvmm) |
| 58 | { | 60 | { |
| 59 | return gm200_vmm_new_(&gm20b_vmm_16, &gm20b_vmm_17, mmu, addr, | 61 | return gm200_vmm_new_(&gm20b_vmm_16, &gm20b_vmm_17, mmu, managed, addr, |
| 60 | size, argv, argc, key, name, pvmm); | 62 | size, argv, argc, key, name, pvmm); |
| 61 | } | 63 | } |
| 62 | 64 | ||
| 63 | int | 65 | int |
| 64 | gm20b_vmm_new_fixed(struct nvkm_mmu *mmu, u64 addr, u64 size, | 66 | gm20b_vmm_new_fixed(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 65 | void *argv, u32 argc, struct lock_class_key *key, | 67 | void *argv, u32 argc, struct lock_class_key *key, |
| 66 | const char *name, struct nvkm_vmm **pvmm) | 68 | const char *name, struct nvkm_vmm **pvmm) |
| 67 | { | 69 | { |
| 68 | return gf100_vmm_new_(&gm20b_vmm_16, &gm20b_vmm_17, mmu, addr, | 70 | return gf100_vmm_new_(&gm20b_vmm_16, &gm20b_vmm_17, mmu, managed, addr, |
| 69 | size, argv, argc, key, name, pvmm); | 71 | size, argv, argc, key, name, pvmm); |
| 70 | } | 72 | } |
| @@ -21,12 +21,90 @@ · 20 unmodified lines | |||
| 21 | */ | 21 | */ |
| 22 | #include "vmm.h" | 22 | #include "vmm.h" |
| 23 | 23 | ||
| 24 | #include <core/client.h> | ||
| 24 | #include <subdev/fb.h> | 25 | #include <subdev/fb.h> |
| 25 | #include <subdev/ltc.h> | 26 | #include <subdev/ltc.h> |
| 27 | #include <subdev/timer.h> | ||
| 28 | #include <engine/gr.h> | ||
| 26 | 29 | ||
| 27 | #include <nvif/ifc00d.h> | 30 | #include <nvif/ifc00d.h> |
| 28 | #include <nvif/unpack.h> | 31 | #include <nvif/unpack.h> |
| 29 | 32 | ||
| 33 | static void | ||
| 34 | gp100_vmm_pfn_unmap(struct nvkm_vmm *vmm, | ||
| 35 | struct nvkm_mmu_pt *pt, u32 ptei, u32 ptes) | ||
| 36 | { | ||
| 37 | struct device *dev = vmm->mmu->subdev.device->dev; | ||
| 38 | dma_addr_t addr; | ||
| 39 | |||
| 40 | nvkm_kmap(pt->memory); | ||
| 41 | while (ptes--) { | ||
| 42 | u32 datalo = nvkm_ro32(pt->memory, pt->base + ptei * 8 + 0); | ||
| 43 | u32 datahi = nvkm_ro32(pt->memory, pt->base + ptei * 8 + 4); | ||
| 44 | u64 data = (u64)datahi << 32 | datalo; | ||
| 45 | if ((data & (3ULL << 1)) != 0) { | ||
| 46 | addr = (data >> 8) << 12; | ||
| 47 | dma_unmap_page(dev, addr, PAGE_SIZE, DMA_BIDIRECTIONAL); | ||
| 48 | } | ||
| 49 | ptei++; | ||
| 50 | } | ||
| 51 | nvkm_done(pt->memory); | ||
| 52 | } | ||
| 53 | |||
| 54 | static bool | ||
| 55 | gp100_vmm_pfn_clear(struct nvkm_vmm *vmm, | ||
| 56 | struct nvkm_mmu_pt *pt, u32 ptei, u32 ptes) | ||
| 57 | { | ||
| 58 | bool dma = false; | ||
| 59 | nvkm_kmap(pt->memory); | ||
| 60 | while (ptes--) { | ||
| 61 | u32 datalo = nvkm_ro32(pt->memory, pt->base + ptei * 8 + 0); | ||
| 62 | u32 datahi = nvkm_ro32(pt->memory, pt->base + ptei * 8 + 4); | ||
| 63 | u64 data = (u64)datahi << 32 | datalo; | ||
| 64 | if ((data & BIT_ULL(0)) && (data & (3ULL << 1)) != 0) { | ||
| 65 | VMM_WO064(pt, vmm, ptei * 8, data & ~BIT_ULL(0)); | ||
| 66 | dma = true; | ||
| 67 | } | ||
| 68 | ptei++; | ||
| 69 | } | ||
| 70 | nvkm_done(pt->memory); | ||
| 71 | return dma; | ||
| 72 | } | ||
| 73 | |||
| 74 | static void | ||
| 75 | gp100_vmm_pgt_pfn(struct nvkm_vmm *vmm, struct nvkm_mmu_pt *pt, | ||
| 76 | u32 ptei, u32 ptes, struct nvkm_vmm_map *map) | ||
| 77 | { | ||
| 78 | struct device *dev = vmm->mmu->subdev.device->dev; | ||
| 79 | dma_addr_t addr; | ||
| 80 | |||
| 81 | nvkm_kmap(pt->memory); | ||
| 82 | while (ptes--) { | ||
| 83 | u64 data = 0; | ||
| 84 | if (!(*map->pfn & NVKM_VMM_PFN_W)) | ||
| 85 | data |= BIT_ULL(6); /* RO. */ | ||
| 86 | |||
| 87 | if (!(*map->pfn & NVKM_VMM_PFN_VRAM)) { | ||
| 88 | addr = *map->pfn >> NVKM_VMM_PFN_ADDR_SHIFT; | ||
| 89 | addr = dma_map_page(dev, pfn_to_page(addr), 0, | ||
| 90 | PAGE_SIZE, DMA_BIDIRECTIONAL); | ||
| 91 | if (!WARN_ON(dma_mapping_error(dev, addr))) { | ||
| 92 | data |= addr >> 4; | ||
| 93 | data |= 2ULL << 1; /* SYSTEM_COHERENT_MEMORY. */ | ||
| 94 | data |= BIT_ULL(3); /* VOL. */ | ||
| 95 | data |= BIT_ULL(0); /* VALID. */ | ||
| 96 | } | ||
| 97 | } else { | ||
| 98 | data |= (*map->pfn & NVKM_VMM_PFN_ADDR) >> 4; | ||
| 99 | data |= BIT_ULL(0); /* VALID. */ | ||
| 100 | } | ||
| 101 | |||
| 102 | VMM_WO064(pt, vmm, ptei++ * 8, data); | ||
| 103 | map->pfn++; | ||
| 104 | } | ||
| 105 | nvkm_done(pt->memory); | ||
| 106 | } | ||
| 107 | |||
| 30 | static inline void | 108 | static inline void |
| 31 | gp100_vmm_pgt_pte(struct nvkm_vmm *vmm, struct nvkm_mmu_pt *pt, | 109 | gp100_vmm_pgt_pte(struct nvkm_vmm *vmm, struct nvkm_mmu_pt *pt, |
| 32 | u32 ptei, u32 ptes, struct nvkm_vmm_map *map, u64 addr) | 110 | u32 ptei, u32 ptes, struct nvkm_vmm_map *map, u64 addr) |
| @@ -89,6 +167,9 @@ gp100_vmm_desc_spt = { · 56 unmodified lines | |||
| 89 | .mem = gp100_vmm_pgt_mem, | 167 | .mem = gp100_vmm_pgt_mem, |
| 90 | .dma = gp100_vmm_pgt_dma, | 168 | .dma = gp100_vmm_pgt_dma, |
| 91 | .sgl = gp100_vmm_pgt_sgl, | 169 | .sgl = gp100_vmm_pgt_sgl, |
| 170 | .pfn = gp100_vmm_pgt_pfn, | ||
| 171 | .pfn_clear = gp100_vmm_pfn_clear, | ||
| 172 | .pfn_unmap = gp100_vmm_pfn_unmap, | ||
| 92 | }; | 173 | }; |
| 93 | 174 | ||
| 94 | static void | 175 | static void |
| @@ -306,16 +387,100 @@ gp100_vmm_valid(struct nvkm_vmm *vmm, void *argv, u32 argc, · 211 unmodified lines | |||
| 306 | return 0; | 387 | return 0; |
| 307 | } | 388 | } |
| 308 | 389 | ||
| 390 | static int | ||
| 391 | gp100_vmm_fault_cancel(struct nvkm_vmm *vmm, void *argv, u32 argc) | ||
| 392 | { | ||
| 393 | struct nvkm_device *device = vmm->mmu->subdev.device; | ||
| 394 | union { | ||
| 395 | struct gp100_vmm_fault_cancel_v0 v0; | ||
| 396 | } *args = argv; | ||
| 397 | int ret = -ENOSYS; | ||
| 398 | u32 inst, aper; | ||
| 399 | |||
| 400 | if ((ret = nvif_unpack(ret, &argv, &argc, args->v0, 0, 0, false))) | ||
| 401 | return ret; | ||
| 402 | |||
| 403 | /* Translate MaxwellFaultBufferA instance pointer to the same | ||
| 404 | * format as the NV_GR_FECS_CURRENT_CTX register. | ||
| 405 | */ | ||
| 406 | aper = (args->v0.inst >> 8) & 3; | ||
| 407 | args->v0.inst >>= 12; | ||
| 408 | args->v0.inst |= aper << 28; | ||
| 409 | args->v0.inst |= 0x80000000; | ||
| 410 | |||
| 411 | if (!WARN_ON(nvkm_gr_ctxsw_pause(device))) { | ||
| 412 | if ((inst = nvkm_gr_ctxsw_inst(device)) == args->v0.inst) { | ||
| 413 | gf100_vmm_invalidate(vmm, 0x0000001b | ||
| 414 | /* CANCEL_TARGETED. */ | | ||
| 415 | (args->v0.hub << 20) | | ||
| 416 | (args->v0.gpc << 15) | | ||
| 417 | (args->v0.client << 9)); | ||
| 418 | } | ||
| 419 | WARN_ON(nvkm_gr_ctxsw_resume(device)); | ||
| 420 | } | ||
| 421 | |||
| 422 | return 0; | ||
| 423 | } | ||
| 424 | |||
| 425 | static int | ||
| 426 | gp100_vmm_fault_replay(struct nvkm_vmm *vmm, void *argv, u32 argc) | ||
| 427 | { | ||
| 428 | union { | ||
| 429 | struct gp100_vmm_fault_replay_vn vn; | ||
| 430 | } *args = argv; | ||
| 431 | int ret = -ENOSYS; | ||
| 432 | |||
| 433 | if (!(ret = nvif_unvers(ret, &argv, &argc, args->vn))) { | ||
| 434 | gf100_vmm_invalidate(vmm, 0x0000000b); /* REPLAY_GLOBAL. */ | ||
| 435 | } | ||
| 436 | |||
| 437 | return ret; | ||
| 438 | } | ||
| 439 | |||
| 440 | int | ||
| 441 | gp100_vmm_mthd(struct nvkm_vmm *vmm, | ||
| 442 | struct nvkm_client *client, u32 mthd, void *argv, u32 argc) | ||
| 443 | { | ||
| 444 | if (client->super) { | ||
| 445 | switch (mthd) { | ||
| 446 | case GP100_VMM_VN_FAULT_REPLAY: | ||
| 447 | return gp100_vmm_fault_replay(vmm, argv, argc); | ||
| 448 | case GP100_VMM_VN_FAULT_CANCEL: | ||
| 449 | return gp100_vmm_fault_cancel(vmm, argv, argc); | ||
| 450 | default: | ||
| 451 | break; | ||
| 452 | } | ||
| 453 | } | ||
| 454 | return -EINVAL; | ||
| 455 | } | ||
| 456 | |||
| 457 | void | ||
| 458 | gp100_vmm_invalidate_pdb(struct nvkm_vmm *vmm, u64 addr) | ||
| 459 | { | ||
| 460 | struct nvkm_device *device = vmm->mmu->subdev.device; | ||
| 461 | nvkm_wr32(device, 0x100cb8, lower_32_bits(addr)); | ||
| 462 | nvkm_wr32(device, 0x100cec, upper_32_bits(addr)); | ||
| 463 | } | ||
| 464 | |||
| 309 | void | 465 | void |
| 310 | gp100_vmm_flush(struct nvkm_vmm *vmm, int depth) | 466 | gp100_vmm_flush(struct nvkm_vmm *vmm, int depth) |
| 311 | { | 467 | { |
| 312 | gf100_vmm_flush_(vmm, 5 /* CACHE_LEVEL_UP_TO_PDE3 */ - depth); | 468 | u32 type = (5 /* CACHE_LEVEL_UP_TO_PDE3 */ - depth) << 24; |
| 469 | type = 0; /*XXX: need to confirm stuff works with depth enabled... */ | ||
| 470 | if (atomic_read(&vmm->engref[NVKM_SUBDEV_BAR])) | ||
| 471 | type |= 0x00000004; /* HUB_ONLY */ | ||
| 472 | type |= 0x00000001; /* PAGE_ALL */ | ||
| 473 | gf100_vmm_invalidate(vmm, type); | ||
| 313 | } | 474 | } |
| 314 | 475 | ||
| 315 | int | 476 | int |
| 316 | gp100_vmm_join(struct nvkm_vmm *vmm, struct nvkm_memory *inst) | 477 | gp100_vmm_join(struct nvkm_vmm *vmm, struct nvkm_memory *inst) |
| 317 | { | 478 | { |
| 318 | const u64 base = BIT_ULL(10) /* VER2 */ | BIT_ULL(11); /* 64KiB */ | 479 | u64 base = BIT_ULL(10) /* VER2 */ | BIT_ULL(11) /* 64KiB */; |
| 480 | if (vmm->replay) { | ||
| 481 | base |= BIT_ULL(4); /* FAULT_REPLAY_TEX */ | ||
| 482 | base |= BIT_ULL(5); /* FAULT_REPLAY_GCC */ | ||
| 483 | } | ||
| 319 | return gf100_vmm_join_(vmm, inst, base); | 484 | return gf100_vmm_join_(vmm, inst, base); |
| 320 | } | 485 | } |
| 321 | 486 | ||
| @@ -326,6 +491,8 @@ gp100_vmm = { · 4 unmodified lines | |||
| 326 | .aper = gf100_vmm_aper, | 491 | .aper = gf100_vmm_aper, |
| 327 | .valid = gp100_vmm_valid, | 492 | .valid = gp100_vmm_valid, |
| 328 | .flush = gp100_vmm_flush, | 493 | .flush = gp100_vmm_flush, |
| 494 | .mthd = gp100_vmm_mthd, | ||
| 495 | .invalidate_pdb = gp100_vmm_invalidate_pdb, | ||
| 329 | .page = { | 496 | .page = { |
| 330 | { 47, &gp100_vmm_desc_16[4], NVKM_VMM_PAGE_Sxxx }, | 497 | { 47, &gp100_vmm_desc_16[4], NVKM_VMM_PAGE_Sxxx }, |
| 331 | { 38, &gp100_vmm_desc_16[3], NVKM_VMM_PAGE_Sxxx }, | 498 | { 38, &gp100_vmm_desc_16[3], NVKM_VMM_PAGE_Sxxx }, |
| @@ -338,10 +505,39 @@ gp100_vmm = { · 6 unmodified lines | |||
| 338 | }; | 505 | }; |
| 339 | 506 | ||
| 340 | int | 507 | int |
| 341 | gp100_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | 508 | gp100_vmm_new_(const struct nvkm_vmm_func *func, |
| 342 | struct lock_class_key *key, const char *name, | 509 | struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 343 | struct nvkm_vmm **pvmm) | 510 | void *argv, u32 argc, struct lock_class_key *key, |
| 511 | const char *name, struct nvkm_vmm **pvmm) | ||
| 512 | { | ||
| 513 | union { | ||
| 514 | struct gp100_vmm_vn vn; | ||
| 515 | struct gp100_vmm_v0 v0; | ||
| 516 | } *args = argv; | ||
| 517 | int ret = -ENOSYS; | ||
| 518 | bool replay; | ||
| 519 | |||
| 520 | if (!(ret = nvif_unpack(ret, &argv, &argc, args->v0, 0, 0, false))) { | ||
| 521 | replay = args->v0.fault_replay != 0; | ||
| 522 | } else | ||
| 523 | if (!(ret = nvif_unvers(ret, &argv, &argc, args->vn))) { | ||
| 524 | replay = false; | ||
| 525 | } else | ||
| 526 | return ret; | ||
| 527 | |||
| 528 | ret = nvkm_vmm_new_(func, mmu, 0, managed, addr, size, key, name, pvmm); | ||
| 529 | if (ret) | ||
| 530 | return ret; | ||
| 531 | |||
| 532 | (*pvmm)->replay = replay; | ||
| 533 | return 0; | ||
| 534 | } | ||
| 535 | |||
| 536 | int | ||
| 537 | gp100_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, | ||
| 538 | void *argv, u32 argc, struct lock_class_key *key, | ||
| 539 | const char *name, struct nvkm_vmm **pvmm) | ||
| 344 | { | 540 | { |
| 345 | return nv04_vmm_new_(&gp100_vmm, mmu, 0, addr, size, | 541 | return gp100_vmm_new_(&gp100_vmm, mmu, managed, addr, size, |
| 346 | argv, argc, key, name, pvmm); | 542 | argv, argc, key, name, pvmm); |
| 347 | } | 543 | } |
| @@ -28,6 +28,8 @@ gp10b_vmm = { · 27 unmodified lines | |||
| 28 | .aper = gk20a_vmm_aper, | 28 | .aper = gk20a_vmm_aper, |
| 29 | .valid = gp100_vmm_valid, | 29 | .valid = gp100_vmm_valid, |
| 30 | .flush = gp100_vmm_flush, | 30 | .flush = gp100_vmm_flush, |
| 31 | .mthd = gp100_vmm_mthd, | ||
| 32 | .invalidate_pdb = gp100_vmm_invalidate_pdb, | ||
| 31 | .page = { | 33 | .page = { |
| 32 | { 47, &gp100_vmm_desc_16[4], NVKM_VMM_PAGE_Sxxx }, | 34 | { 47, &gp100_vmm_desc_16[4], NVKM_VMM_PAGE_Sxxx }, |
| 33 | { 38, &gp100_vmm_desc_16[3], NVKM_VMM_PAGE_Sxxx }, | 35 | { 38, &gp100_vmm_desc_16[3], NVKM_VMM_PAGE_Sxxx }, |
| @@ -40,10 +42,10 @@ gp10b_vmm = { · 6 unmodified lines | |||
| 40 | }; | 42 | }; |
| 41 | 43 | ||
| 42 | int | 44 | int |
| 43 | gp10b_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | 45 | gp10b_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 44 | struct lock_class_key *key, const char *name, | 46 | void *argv, u32 argc, struct lock_class_key *key, |
| 45 | struct nvkm_vmm **pvmm) | 47 | const char *name, struct nvkm_vmm **pvmm) |
| 46 | { | 48 | { |
| 47 | return nv04_vmm_new_(&gp10b_vmm, mmu, 0, addr, size, | 49 | return gp100_vmm_new_(&gp10b_vmm, mmu, managed, addr, size, |
| 48 | argv, argc, key, name, pvmm); | 50 | argv, argc, key, name, pvmm); |
| 49 | } | 51 | } |
| @@ -66,6 +66,8 @@ gv100_vmm = { · 65 unmodified lines | |||
| 66 | .aper = gf100_vmm_aper, | 66 | .aper = gf100_vmm_aper, |
| 67 | .valid = gp100_vmm_valid, | 67 | .valid = gp100_vmm_valid, |
| 68 | .flush = gp100_vmm_flush, | 68 | .flush = gp100_vmm_flush, |
| 69 | .mthd = gp100_vmm_mthd, | ||
| 70 | .invalidate_pdb = gp100_vmm_invalidate_pdb, | ||
| 69 | .page = { | 71 | .page = { |
| 70 | { 47, &gp100_vmm_desc_16[4], NVKM_VMM_PAGE_Sxxx }, | 72 | { 47, &gp100_vmm_desc_16[4], NVKM_VMM_PAGE_Sxxx }, |
| 71 | { 38, &gp100_vmm_desc_16[3], NVKM_VMM_PAGE_Sxxx }, | 73 | { 38, &gp100_vmm_desc_16[3], NVKM_VMM_PAGE_Sxxx }, |
| @@ -78,10 +80,10 @@ gv100_vmm = { · 6 unmodified lines | |||
| 78 | }; | 80 | }; |
| 79 | 81 | ||
| 80 | int | 82 | int |
| 81 | gv100_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | 83 | gv100_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 82 | struct lock_class_key *key, const char *name, | 84 | void *argv, u32 argc, struct lock_class_key *key, |
| 83 | struct nvkm_vmm **pvmm) | 85 | const char *name, struct nvkm_vmm **pvmm) |
| 84 | { | 86 | { |
| 85 | return nv04_vmm_new_(&gv100_vmm, mmu, 0, addr, size, | 87 | return gp100_vmm_new_(&gv100_vmm, mmu, managed, addr, size, |
| 86 | argv, argc, key, name, pvmm); | 88 | argv, argc, key, name, pvmm); |
| 87 | } | 89 | } |
| @@ -36,10 +36,10 @@ mcp77_vmm = { · 35 unmodified lines | |||
| 36 | }; | 36 | }; |
| 37 | 37 | ||
| 38 | int | 38 | int |
| 39 | mcp77_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | 39 | mcp77_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 40 | struct lock_class_key *key, const char *name, | 40 | void *argv, u32 argc, struct lock_class_key *key, |
| 41 | struct nvkm_vmm **pvmm) | 41 | const char *name, struct nvkm_vmm **pvmm) |
| 42 | { | 42 | { |
| 43 | return nv04_vmm_new_(&mcp77_vmm, mmu, 0, addr, size, | 43 | return nv04_vmm_new_(&mcp77_vmm, mmu, 0, managed, addr, size, |
| 44 | argv, argc, key, name, pvmm); | 44 | argv, argc, key, name, pvmm); |
| 45 | } | 45 | } |
| @@ -100,16 +100,17 @@ nv04_vmm = { · 99 unmodified lines | |||
| 100 | 100 | ||
| 101 | int | 101 | int |
| 102 | nv04_vmm_new_(const struct nvkm_vmm_func *func, struct nvkm_mmu *mmu, | 102 | nv04_vmm_new_(const struct nvkm_vmm_func *func, struct nvkm_mmu *mmu, |
| 103 | u32 pd_header, u64 addr, u64 size, void *argv, u32 argc, | 103 | u32 pd_header, bool managed, u64 addr, u64 size, |
| 104 | struct lock_class_key *key, const char *name, | 104 | void *argv, u32 argc, struct lock_class_key *key, |
| 105 | struct nvkm_vmm **pvmm) | 105 | const char *name, struct nvkm_vmm **pvmm) |
| 106 | { | 106 | { |
| 107 | union { | 107 | union { |
| 108 | struct nv04_vmm_vn vn; | 108 | struct nv04_vmm_vn vn; |
| 109 | } *args = argv; | 109 | } *args = argv; |
| 110 | int ret; | 110 | int ret; |
| 111 | 111 | ||
| 112 | ret = nvkm_vmm_new_(func, mmu, pd_header, addr, size, key, name, pvmm); | 112 | ret = nvkm_vmm_new_(func, mmu, pd_header, managed, addr, size, |
| 113 | key, name, pvmm); | ||
| 113 | if (ret) | 114 | if (ret) |
| 114 | return ret; | 115 | return ret; |
| 115 | 116 | ||
| @@ -117,15 +118,15 @@ nv04_vmm_new_(const struct nvkm_vmm_func *func, struct nvkm_mmu *mmu, · 1 unmodified line | |||
| 117 | } | 118 | } |
| 118 | 119 | ||
| 119 | int | 120 | int |
| 120 | nv04_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | 121 | nv04_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 121 | struct lock_class_key *key, const char *name, | 122 | void *argv, u32 argc, struct lock_class_key *key, const char *name, |
| 122 | struct nvkm_vmm **pvmm) | 123 | struct nvkm_vmm **pvmm) |
| 123 | { | 124 | { |
| 124 | struct nvkm_memory *mem; | 125 | struct nvkm_memory *mem; |
| 125 | struct nvkm_vmm *vmm; | 126 | struct nvkm_vmm *vmm; |
| 126 | int ret; | 127 | int ret; |
| 127 | 128 | ||
| 128 | ret = nv04_vmm_new_(&nv04_vmm, mmu, 8, addr, size, | 129 | ret = nv04_vmm_new_(&nv04_vmm, mmu, 8, managed, addr, size, |
| 129 | argv, argc, key, name, &vmm); | 130 | argv, argc, key, name, &vmm); |
| 130 | *pvmm = vmm; | 131 | *pvmm = vmm; |
| 131 | if (ret) | 132 | if (ret) |
| @@ -104,10 +104,10 @@ nv41_vmm = { · 103 unmodified lines | |||
| 104 | }; | 104 | }; |
| 105 | 105 | ||
| 106 | int | 106 | int |
| 107 | nv41_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | 107 | nv41_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 108 | struct lock_class_key *key, const char *name, | 108 | void *argv, u32 argc, struct lock_class_key *key, const char *name, |
| 109 | struct nvkm_vmm **pvmm) | 109 | struct nvkm_vmm **pvmm) |
| 110 | { | 110 | { |
| 111 | return nv04_vmm_new_(&nv41_vmm, mmu, 0, addr, size, | 111 | return nv04_vmm_new_(&nv41_vmm, mmu, 0, managed, addr, size, |
| 112 | argv, argc, key, name, pvmm); | 112 | argv, argc, key, name, pvmm); |
| 113 | } | 113 | } |
| @@ -205,15 +205,15 @@ nv44_vmm = { · 204 unmodified lines | |||
| 205 | }; | 205 | }; |
| 206 | 206 | ||
| 207 | int | 207 | int |
| 208 | nv44_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | 208 | nv44_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 209 | struct lock_class_key *key, const char *name, | 209 | void *argv, u32 argc, struct lock_class_key *key, const char *name, |
| 210 | struct nvkm_vmm **pvmm) | 210 | struct nvkm_vmm **pvmm) |
| 211 | { | 211 | { |
| 212 | struct nvkm_subdev *subdev = &mmu->subdev; | 212 | struct nvkm_subdev *subdev = &mmu->subdev; |
| 213 | struct nvkm_vmm *vmm; | 213 | struct nvkm_vmm *vmm; |
| 214 | int ret; | 214 | int ret; |
| 215 | 215 | ||
| 216 | ret = nv04_vmm_new_(&nv44_vmm, mmu, 0, addr, size, | 216 | ret = nv04_vmm_new_(&nv44_vmm, mmu, 0, managed, addr, size, |
| 217 | argv, argc, key, name, &vmm); | 217 | argv, argc, key, name, &vmm); |
| 218 | *pvmm = vmm; | 218 | *pvmm = vmm; |
| 219 | if (ret) | 219 | if (ret) |
| @@ -376,10 +376,10 @@ nv50_vmm = { · 375 unmodified lines | |||
| 376 | }; | 376 | }; |
| 377 | 377 | ||
| 378 | int | 378 | int |
| 379 | nv50_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, void *argv, u32 argc, | 379 | nv50_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 380 | struct lock_class_key *key, const char *name, | 380 | void *argv, u32 argc, struct lock_class_key *key, const char *name, |
| 381 | struct nvkm_vmm **pvmm) | 381 | struct nvkm_vmm **pvmm) |
| 382 | { | 382 | { |
| 383 | return nv04_vmm_new_(&nv50_vmm, mmu, 0, addr, size, | 383 | return nv04_vmm_new_(&nv50_vmm, mmu, 0, managed, addr, size, |
| 384 | argv, argc, key, name, pvmm); | 384 | argv, argc, key, name, pvmm); |
| 385 | } | 385 | } |
| @@ -24,7 +24,7 @@ · 23 unmodified lines | |||
| 24 | #include <subdev/timer.h> | 24 | #include <subdev/timer.h> |
| 25 | 25 | ||
| 26 | static void | 26 | static void |
| 27 | tu104_vmm_flush(struct nvkm_vmm *vmm, int depth) | 27 | tu102_vmm_flush(struct nvkm_vmm *vmm, int depth) |
| 28 | { | 28 | { |
| 29 | struct nvkm_subdev *subdev = &vmm->mmu->subdev; | 29 | struct nvkm_subdev *subdev = &vmm->mmu->subdev; |
| 30 | struct nvkm_device *device = subdev->device; | 30 | struct nvkm_device *device = subdev->device; |
| @@ -50,12 +50,13 @@ tu104_vmm_flush(struct nvkm_vmm *vmm, int depth) · 19 unmodified lines | |||
| 50 | } | 50 | } |
| 51 | 51 | ||
| 52 | static const struct nvkm_vmm_func | 52 | static const struct nvkm_vmm_func |
| 53 | tu104_vmm = { | 53 | tu102_vmm = { |
| 54 | .join = gv100_vmm_join, | 54 | .join = gv100_vmm_join, |
| 55 | .part = gf100_vmm_part, | 55 | .part = gf100_vmm_part, |
| 56 | .aper = gf100_vmm_aper, | 56 | .aper = gf100_vmm_aper, |
| 57 | .valid = gp100_vmm_valid, | 57 | .valid = gp100_vmm_valid, |
| 58 | .flush = tu104_vmm_flush, | 58 | .flush = tu102_vmm_flush, |
| 59 | .mthd = gp100_vmm_mthd, | ||
| 59 | .page = { | 60 | .page = { |
| 60 | { 47, &gp100_vmm_desc_16[4], NVKM_VMM_PAGE_Sxxx }, | 61 | { 47, &gp100_vmm_desc_16[4], NVKM_VMM_PAGE_Sxxx }, |
| 61 | { 38, &gp100_vmm_desc_16[3], NVKM_VMM_PAGE_Sxxx }, | 62 | { 38, &gp100_vmm_desc_16[3], NVKM_VMM_PAGE_Sxxx }, |
| @@ -68,10 +69,10 @@ tu104_vmm = { · 6 unmodified lines | |||
| 68 | }; | 69 | }; |
| 69 | 70 | ||
| 70 | int | 71 | int |
| 71 | tu104_vmm_new(struct nvkm_mmu *mmu, u64 addr, u64 size, | 72 | tu102_vmm_new(struct nvkm_mmu *mmu, bool managed, u64 addr, u64 size, |
| 72 | void *argv, u32 argc, struct lock_class_key *key, | 73 | void *argv, u32 argc, struct lock_class_key *key, |
| 73 | const char *name, struct nvkm_vmm **pvmm) | 74 | const char *name, struct nvkm_vmm **pvmm) |
| 74 | { | 75 | { |
| 75 | return nv04_vmm_new_(&tu104_vmm, mmu, 0, addr, size, | 76 | return gp100_vmm_new_(&tu102_vmm, mmu, managed, addr, size, |
| 76 | argv, argc, key, name, pvmm); | 77 | argv, argc, key, name, pvmm); |
| 77 | } | 78 | } |
| @@ -88,10 +88,10 @@ nvkm_memx_fini(struct nvkm_memx **pmemx, bool exec) · 87 unmodified lines | |||
| 88 | if (exec) { | 88 | if (exec) { |
| 89 | nvkm_pmu_send(pmu, reply, PROC_MEMX, MEMX_MSG_EXEC, | 89 | nvkm_pmu_send(pmu, reply, PROC_MEMX, MEMX_MSG_EXEC, |
| 90 | memx->base, finish); | 90 | memx->base, finish); |
| 91 | nvkm_debug(subdev, "Exec took %uns, PMU_IN %08x\n", | ||
| 92 | reply[0], reply[1]); | ||
| 91 | } | 93 | } |
| 92 | 94 | ||
| 93 | nvkm_debug(subdev, "Exec took %uns, PMU_IN %08x\n", | ||
| 94 | reply[0], reply[1]); | ||
| 95 | kfree(memx); | 95 | kfree(memx); |
| 96 | return 0; | 96 | return 0; |
| 97 | } | 97 | } |
| @@ -853,7 +853,7 @@ acr_r352_shutdown(struct acr_r352 *acr, struct nvkm_secboot *sb) · 852 unmodified lines | |||
| 853 | * and the expected behavior on RM as well | 853 | * and the expected behavior on RM as well |
| 854 | */ | 854 | */ |
| 855 | if (ret && ret != 0x1d) { | 855 | if (ret && ret != 0x1d) { |
| 856 | nvkm_error(subdev, "HS unload failed, ret 0x%08x", ret); | 856 | nvkm_error(subdev, "HS unload failed, ret 0x%08x\n", ret); |
| 857 | return -EINVAL; | 857 | return -EINVAL; |
| 858 | } | 858 | } |
| 859 | nvkm_debug(subdev, "HS unload blob completed\n"); | 859 | nvkm_debug(subdev, "HS unload blob completed\n"); |
| @@ -922,7 +922,7 @@ acr_r352_bootstrap(struct acr_r352 *acr, struct nvkm_secboot *sb) · 62 unmodified lines | |||
| 922 | if (ret < 0) { | 922 | if (ret < 0) { |
| 923 | return ret; | 923 | return ret; |
| 924 | } else if (ret > 0) { | 924 | } else if (ret > 0) { |
| 925 | nvkm_error(subdev, "HS load failed, ret 0x%08x", ret); | 925 | nvkm_error(subdev, "HS load failed, ret 0x%08x\n", ret); |
| 926 | return -EINVAL; | 926 | return -EINVAL; |
| 927 | } | 927 | } |
| 928 | nvkm_debug(subdev, "HS load blob completed\n"); | 928 | nvkm_debug(subdev, "HS load blob completed\n"); |
| @@ -40,6 +40,22 @@ nvkm_top_device_new(struct nvkm_top *top) · 39 unmodified lines | |||
| 40 | return info; | 40 | return info; |
| 41 | } | 41 | } |
| 42 | 42 | ||
| 43 | u32 | ||
| 44 | nvkm_top_addr(struct nvkm_device *device, enum nvkm_devidx index) | ||
| 45 | { | ||
| 46 | struct nvkm_top *top = device->top; | ||
| 47 | struct nvkm_top_device *info; | ||
| 48 | |||
| 49 | if (top) { | ||
| 50 | list_for_each_entry(info, &top->device, head) { | ||
| 51 | if (info->index == index) | ||
| 52 | return info->addr; | ||
| 53 | } | ||
| 54 | } | ||
| 55 | |||
| 56 | return 0; | ||
| 57 | } | ||
| 58 | |||
| 43 | u32 | 59 | u32 |
| 44 | nvkm_top_reset(struct nvkm_device *device, enum nvkm_devidx index) | 60 | nvkm_top_reset(struct nvkm_device *device, enum nvkm_devidx index) |
| 45 | { | 61 | { |
| @@ -73,6 +73,7 @@ gk104_top_oneinit(struct nvkm_top *top) · 72 unmodified lines | |||
| 73 | #define A_(A) if (inst == 0) info->index = NVKM_ENGINE_##A | 73 | #define A_(A) if (inst == 0) info->index = NVKM_ENGINE_##A |
| 74 | #define B_(A) if (inst + NVKM_ENGINE_##A##0 < NVKM_ENGINE_##A##_LAST + 1) \ | 74 | #define B_(A) if (inst + NVKM_ENGINE_##A##0 < NVKM_ENGINE_##A##_LAST + 1) \ |
| 75 | info->index = NVKM_ENGINE_##A##0 + inst | 75 | info->index = NVKM_ENGINE_##A##0 + inst |
| 76 | #define C_(A) if (inst == 0) info->index = NVKM_SUBDEV_##A | ||
| 76 | switch (type) { | 77 | switch (type) { |
| 77 | case 0x00000000: A_(GR ); break; | 78 | case 0x00000000: A_(GR ); break; |
| 78 | case 0x00000001: A_(CE0 ); break; | 79 | case 0x00000001: A_(CE0 ); break; |
| @@ -88,6 +89,7 @@ gk104_top_oneinit(struct nvkm_top *top) · 9 unmodified lines | |||
| 88 | case 0x0000000f: A_(NVENC1); break; | 89 | case 0x0000000f: A_(NVENC1); break; |
| 89 | case 0x00000010: B_(NVDEC ); break; | 90 | case 0x00000010: B_(NVDEC ); break; |
| 90 | case 0x00000013: B_(CE ); break; | 91 | case 0x00000013: B_(CE ); break; |
| 92 | case 0x00000014: C_(GSP ); break; | ||
| 91 | break; | 93 | break; |
| 92 | default: | 94 | default: |
| 93 | break; | 95 | break; |
| @@ -2,6 +2,7 @@ nvkm-y += nvkm/subdev/volt/base.o · 1 unmodified line | |||
| 2 | nvkm-y += nvkm/subdev/volt/gpio.o | 2 | nvkm-y += nvkm/subdev/volt/gpio.o |
| 3 | nvkm-y += nvkm/subdev/volt/nv40.o | 3 | nvkm-y += nvkm/subdev/volt/nv40.o |
| 4 | nvkm-y += nvkm/subdev/volt/gf100.o | 4 | nvkm-y += nvkm/subdev/volt/gf100.o |
| 5 | nvkm-y += nvkm/subdev/volt/gf117.o | ||
| 5 | nvkm-y += nvkm/subdev/volt/gk104.o | 6 | nvkm-y += nvkm/subdev/volt/gk104.o |
| 6 | nvkm-y += nvkm/subdev/volt/gk20a.o | 7 | nvkm-y += nvkm/subdev/volt/gk20a.o |
| 7 | nvkm-y += nvkm/subdev/volt/gm20b.o | 8 | nvkm-y += nvkm/subdev/volt/gm20b.o |
| @@ -0,0 +1,60 @@ | |||
| 1 | /* | ||
| 2 | * Copyright 2019 Ilia Mirkin | ||
| 3 | * | ||
| 4 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 5 | * copy of this software and associated documentation files (the "Software"), | ||
| 6 | * to deal in the Software without restriction, including without limitation | ||
| 7 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 8 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 9 | * Software is furnished to do so, subject to the following conditions: | ||
| 10 | * | ||
| 11 | * The above copyright notice and this permission notice shall be included in | ||
| 12 | * all copies or substantial portions of the Software. | ||
| 13 | * | ||
| 14 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 15 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 16 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 17 | * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | ||
| 18 | * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | ||
| 19 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | ||
| 20 | * OTHER DEALINGS IN THE SOFTWARE. | ||
| 21 | * | ||
| 22 | * Authors: Ilia Mirkin | ||
| 23 | */ | ||
| 24 | #include "priv.h" | ||
| 25 | |||
| 26 | #include <subdev/fuse.h> | ||
| 27 | |||
| 28 | static int | ||
| 29 | gf117_volt_speedo_read(struct nvkm_volt *volt) | ||
| 30 | { | ||
| 31 | struct nvkm_device *device = volt->subdev.device; | ||
| 32 | struct nvkm_fuse *fuse = device->fuse; | ||
| 33 | |||
| 34 | if (!fuse) | ||
| 35 | return -EINVAL; | ||
| 36 | |||
| 37 | return nvkm_fuse_read(fuse, 0x3a8); | ||
| 38 | } | ||
| 39 | |||
| 40 | static const struct nvkm_volt_func | ||
| 41 | gf117_volt = { | ||
| 42 | .oneinit = gf100_volt_oneinit, | ||
| 43 | .vid_get = nvkm_voltgpio_get, | ||
| 44 | .vid_set = nvkm_voltgpio_set, | ||
| 45 | .speedo_read = gf117_volt_speedo_read, | ||
| 46 | }; | ||
| 47 | |||
| 48 | int | ||
| 49 | gf117_volt_new(struct nvkm_device *device, int index, struct nvkm_volt **pvolt) | ||
| 50 | { | ||
| 51 | struct nvkm_volt *volt; | ||
| 52 | int ret; | ||
| 53 | |||
| 54 | ret = nvkm_volt_new_(&gf117_volt, device, index, &volt); | ||
| 55 | *pvolt = volt; | ||
| 56 | if (ret) | ||
| 57 | return ret; | ||
| 58 | |||
| 59 | return nvkm_voltgpio_init(volt); | ||
| 60 | } | ||
| @@ -4869,10 +4869,12 @@ static void ci_request_link_speed_change_before_state_change(struct radeon_devic · 4868 unmodified lines | |||
| 4869 | pi->force_pcie_gen = RADEON_PCIE_GEN2; | 4869 | pi->force_pcie_gen = RADEON_PCIE_GEN2; |
| 4870 | if (current_link_speed == RADEON_PCIE_GEN2) | 4870 | if (current_link_speed == RADEON_PCIE_GEN2) |
| 4871 | break; | 4871 | break; |
| 4872 | /* fall through */ | ||
| 4872 | case RADEON_PCIE_GEN2: | 4873 | case RADEON_PCIE_GEN2: |
| 4873 | if (radeon_acpi_pcie_performance_request(rdev, PCIE_PERF_REQ_PECI_GEN2, false) == 0) | 4874 | if (radeon_acpi_pcie_performance_request(rdev, PCIE_PERF_REQ_PECI_GEN2, false) == 0) |
| 4874 | break; | 4875 | break; |
| 4875 | #endif | 4876 | #endif |
| 4877 | /* fall through */ | ||
| 4876 | default: | 4878 | default: |
| 4877 | pi->force_pcie_gen = ci_get_current_pcie_speed(rdev); | 4879 | pi->force_pcie_gen = ci_get_current_pcie_speed(rdev); |
| 4878 | break; | 4880 | break; |
| @@ -1299,6 +1299,7 @@ static int evergreen_cs_handle_reg(struct radeon_cs_parser *p, u32 reg, u32 idx) · 1298 unmodified lines | |||
| 1299 | return -EINVAL; | 1299 | return -EINVAL; |
| 1300 | } | 1300 | } |
| 1301 | ib[idx] += (u32)((reloc->gpu_offset >> 8) & 0xffffffff); | 1301 | ib[idx] += (u32)((reloc->gpu_offset >> 8) & 0xffffffff); |
| 1302 | break; | ||
| 1302 | case CB_TARGET_MASK: | 1303 | case CB_TARGET_MASK: |
| 1303 | track->cb_target_mask = radeon_get_ib_value(p, idx); | 1304 | track->cb_target_mask = radeon_get_ib_value(p, idx); |
| 1304 | track->cb_dirty = true; | 1305 | track->cb_dirty = true; |
| @@ -172,6 +172,7 @@ int radeon_driver_load_kms(struct drm_device *dev, unsigned long flags) · 171 unmodified lines | |||
| 172 | } | 172 | } |
| 173 | 173 | ||
| 174 | if (radeon_is_px(dev)) { | 174 | if (radeon_is_px(dev)) { |
| 175 | dev_pm_set_driver_flags(dev->dev, DPM_FLAG_NEVER_SKIP); | ||
| 175 | pm_runtime_use_autosuspend(dev->dev); | 176 | pm_runtime_use_autosuspend(dev->dev); |
| 176 | pm_runtime_set_autosuspend_delay(dev->dev, 5000); | 177 | pm_runtime_set_autosuspend_delay(dev->dev, 5000); |
| 177 | pm_runtime_set_active(dev->dev); | 178 | pm_runtime_set_active(dev->dev); |
| @@ -5762,10 +5762,12 @@ static void si_request_link_speed_change_before_state_change(struct radeon_devic · 5761 unmodified lines | |||
| 5762 | si_pi->force_pcie_gen = RADEON_PCIE_GEN2; | 5762 | si_pi->force_pcie_gen = RADEON_PCIE_GEN2; |
| 5763 | if (current_link_speed == RADEON_PCIE_GEN2) | 5763 | if (current_link_speed == RADEON_PCIE_GEN2) |
| 5764 | break; | 5764 | break; |
| 5765 | /* fall through */ | ||
| 5765 | case RADEON_PCIE_GEN2: | 5766 | case RADEON_PCIE_GEN2: |
| 5766 | if (radeon_acpi_pcie_performance_request(rdev, PCIE_PERF_REQ_PECI_GEN2, false) == 0) | 5767 | if (radeon_acpi_pcie_performance_request(rdev, PCIE_PERF_REQ_PECI_GEN2, false) == 0) |
| 5767 | break; | 5768 | break; |
| 5768 | #endif | 5769 | #endif |
| 5770 | /* fall through */ | ||
| 5769 | default: | 5771 | default: |
| 5770 | si_pi->force_pcie_gen = si_get_current_pcie_speed(rdev); | 5772 | si_pi->force_pcie_gen = si_get_current_pcie_speed(rdev); |
| 5771 | break; | 5773 | break; |
| @@ -52,12 +52,12 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, · 51 unmodified lines | |||
| 52 | { | 52 | { |
| 53 | int i; | 53 | int i; |
| 54 | 54 | ||
| 55 | if (!(entity && rq_list && num_rq_list > 0 && rq_list[0])) | 55 | if (!(entity && rq_list && (num_rq_list == 0 || rq_list[0]))) |
| 56 | return -EINVAL; | 56 | return -EINVAL; |
| 57 | 57 | ||
| 58 | memset(entity, 0, sizeof(struct drm_sched_entity)); | 58 | memset(entity, 0, sizeof(struct drm_sched_entity)); |
| 59 | INIT_LIST_HEAD(&entity->list); | 59 | INIT_LIST_HEAD(&entity->list); |
| 60 | entity->rq = rq_list[0]; | 60 | entity->rq = NULL; |
| 61 | entity->guilty = guilty; | 61 | entity->guilty = guilty; |
| 62 | entity->num_rq_list = num_rq_list; | 62 | entity->num_rq_list = num_rq_list; |
| 63 | entity->rq_list = kcalloc(num_rq_list, sizeof(struct drm_sched_rq *), | 63 | entity->rq_list = kcalloc(num_rq_list, sizeof(struct drm_sched_rq *), |
| @@ -67,6 +67,10 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, · 3 unmodified lines | |||
| 67 | 67 | ||
| 68 | for (i = 0; i < num_rq_list; ++i) | 68 | for (i = 0; i < num_rq_list; ++i) |
| 69 | entity->rq_list[i] = rq_list[i]; | 69 | entity->rq_list[i] = rq_list[i]; |
| 70 | |||
| 71 | if (num_rq_list) | ||
| 72 | entity->rq = rq_list[0]; | ||
| 73 | |||
| 70 | entity->last_scheduled = NULL; | 74 | entity->last_scheduled = NULL; |
| 71 | 75 | ||
| 72 | spin_lock_init(&entity->rq_lock); | 76 | spin_lock_init(&entity->rq_lock); |
| @@ -165,6 +169,9 @@ long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout) · 92 unmodified lines | |||
| 165 | struct task_struct *last_user; | 169 | struct task_struct *last_user; |
| 166 | long ret = timeout; | 170 | long ret = timeout; |
| 167 | 171 | ||
| 172 | if (!entity->rq) | ||
| 173 | return 0; | ||
| 174 | |||
| 168 | sched = entity->rq->sched; | 175 | sched = entity->rq->sched; |
| 169 | /** | 176 | /** |
| 170 | * The client will not queue more IBs during this fini, consume existing | 177 | * The client will not queue more IBs during this fini, consume existing |
| @@ -264,20 +271,24 @@ static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity) · 93 unmodified lines | |||
| 264 | */ | 271 | */ |
| 265 | void drm_sched_entity_fini(struct drm_sched_entity *entity) | 272 | void drm_sched_entity_fini(struct drm_sched_entity *entity) |
| 266 | { | 273 | { |
| 267 | struct drm_gpu_scheduler *sched; | 274 | struct drm_gpu_scheduler *sched = NULL; |
| 268 | 275 | ||
| 269 | sched = entity->rq->sched; | 276 | if (entity->rq) { |
| 270 | drm_sched_rq_remove_entity(entity->rq, entity); | 277 | sched = entity->rq->sched; |
| 278 | drm_sched_rq_remove_entity(entity->rq, entity); | ||
| 279 | } | ||
| 271 | 280 | ||
| 272 | /* Consumption of existing IBs wasn't completed. Forcefully | 281 | /* Consumption of existing IBs wasn't completed. Forcefully |
| 273 | * remove them here. | 282 | * remove them here. |
| 274 | */ | 283 | */ |
| 275 | if (spsc_queue_peek(&entity->job_queue)) { | 284 | if (spsc_queue_peek(&entity->job_queue)) { |
| 276 | /* Park the kernel for a moment to make sure it isn't processing | 285 | if (sched) { |
| 277 | * our enity. | 286 | /* Park the kernel for a moment to make sure it isn't processing |
| 278 | */ | 287 | * our enity. |
| 279 | kthread_park(sched->thread); | 288 | */ |
| 280 | kthread_unpark(sched->thread); | 289 | kthread_park(sched->thread); |
| 290 | kthread_unpark(sched->thread); | ||
| 291 | } | ||
| 281 | if (entity->dependency) { | 292 | if (entity->dependency) { |
| 282 | dma_fence_remove_callback(entity->dependency, | 293 | dma_fence_remove_callback(entity->dependency, |
| 283 | &entity->cb); | 294 | &entity->cb); |
| @@ -362,9 +373,11 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity, · 78 unmodified lines | |||
| 362 | for (i = 0; i < entity->num_rq_list; ++i) | 373 | for (i = 0; i < entity->num_rq_list; ++i) |
| 363 | drm_sched_entity_set_rq_priority(&entity->rq_list[i], priority); | 374 | drm_sched_entity_set_rq_priority(&entity->rq_list[i], priority); |
| 364 | 375 | ||
| 365 | drm_sched_rq_remove_entity(entity->rq, entity); | 376 | if (entity->rq) { |
| 366 | drm_sched_entity_set_rq_priority(&entity->rq, priority); | 377 | drm_sched_rq_remove_entity(entity->rq, entity); |
| 367 | drm_sched_rq_add_entity(entity->rq, entity); | 378 | drm_sched_entity_set_rq_priority(&entity->rq, priority); |
| 379 | drm_sched_rq_add_entity(entity->rq, entity); | ||
| 380 | } | ||
| 368 | 381 | ||
| 369 | spin_unlock(&entity->rq_lock); | 382 | spin_unlock(&entity->rq_lock); |
| 370 | } | 383 | } |
| @@ -265,6 +265,12 @@ void ipu_pre_update(struct ipu_pre *pre, unsigned int bufaddr) · 264 unmodified lines | |||
| 265 | writel(IPU_PRE_CTRL_SDW_UPDATE, pre->regs + IPU_PRE_CTRL_SET); | 265 | writel(IPU_PRE_CTRL_SDW_UPDATE, pre->regs + IPU_PRE_CTRL_SET); |
| 266 | } | 266 | } |
| 267 | 267 | ||
| 268 | bool ipu_pre_update_pending(struct ipu_pre *pre) | ||
| 269 | { | ||
| 270 | return !!(readl_relaxed(pre->regs + IPU_PRE_CTRL) & | ||
| 271 | IPU_PRE_CTRL_SDW_UPDATE); | ||
| 272 | } | ||
| 273 | |||
| 268 | u32 ipu_pre_get_baddr(struct ipu_pre *pre) | 274 | u32 ipu_pre_get_baddr(struct ipu_pre *pre) |
| 269 | { | 275 | { |
| 270 | return (u32)pre->buffer_paddr; | 276 | return (u32)pre->buffer_paddr; |
| @@ -347,6 +347,22 @@ int ipu_prg_channel_configure(struct ipuv3_channel *ipu_chan, · 346 unmodified lines | |||
| 347 | } | 347 | } |
| 348 | EXPORT_SYMBOL_GPL(ipu_prg_channel_configure); | 348 | EXPORT_SYMBOL_GPL(ipu_prg_channel_configure); |
| 349 | 349 | ||
| 350 | bool ipu_prg_channel_configure_pending(struct ipuv3_channel *ipu_chan) | ||
| 351 | { | ||
| 352 | int prg_chan = ipu_prg_ipu_to_prg_chan(ipu_chan->num); | ||
| 353 | struct ipu_prg *prg = ipu_chan->ipu->prg_priv; | ||
| 354 | struct ipu_prg_channel *chan; | ||
| 355 | |||
| 356 | if (prg_chan < 0) | ||
| 357 | return false; | ||
| 358 | |||
| 359 | chan = &prg->chan[prg_chan]; | ||
| 360 | WARN_ON(!chan->enabled); | ||
| 361 | |||
| 362 | return ipu_pre_update_pending(prg->pres[chan->used_pre]); | ||
| 363 | } | ||
| 364 | EXPORT_SYMBOL_GPL(ipu_prg_channel_configure_pending); | ||
| 365 | |||
| 350 | static int ipu_prg_probe(struct platform_device *pdev) | 366 | static int ipu_prg_probe(struct platform_device *pdev) |
| 351 | { | 367 | { |
| 352 | struct device *dev = &pdev->dev; | 368 | struct device *dev = &pdev->dev; |
| @@ -272,6 +272,7 @@ void ipu_pre_configure(struct ipu_pre *pre, unsigned int width, · 271 unmodified lines | |||
| 272 | unsigned int height, unsigned int stride, u32 format, | 272 | unsigned int height, unsigned int stride, u32 format, |
| 273 | uint64_t modifier, unsigned int bufaddr); | 273 | uint64_t modifier, unsigned int bufaddr); |
| 274 | void ipu_pre_update(struct ipu_pre *pre, unsigned int bufaddr); | 274 | void ipu_pre_update(struct ipu_pre *pre, unsigned int bufaddr); |
| 275 | bool ipu_pre_update_pending(struct ipu_pre *pre); | ||
| 275 | 276 | ||
| 276 | struct ipu_prg *ipu_prg_lookup_by_phandle(struct device *dev, const char *name, | 277 | struct ipu_prg *ipu_prg_lookup_by_phandle(struct device *dev, const char *name, |
| 277 | int ipu_id); | 278 | int ipu_id); |
| @@ -783,6 +783,7 @@ void c4iw_init_dev_ucontext(struct c4iw_rdev *rdev, · 782 unmodified lines | |||
| 783 | static int c4iw_rdev_open(struct c4iw_rdev *rdev) | 783 | static int c4iw_rdev_open(struct c4iw_rdev *rdev) |
| 784 | { | 784 | { |
| 785 | int err; | 785 | int err; |
| 786 | unsigned int factor; | ||
| 786 | 787 | ||
| 787 | c4iw_init_dev_ucontext(rdev, &rdev->uctx); | 788 | c4iw_init_dev_ucontext(rdev, &rdev->uctx); |
| 788 | 789 | ||
| @@ -806,8 +807,18 @@ static int c4iw_rdev_open(struct c4iw_rdev *rdev) · 17 unmodified lines | |||
| 806 | return -EINVAL; | 807 | return -EINVAL; |
| 807 | } | 808 | } |
| 808 | 809 | ||
| 809 | rdev->qpmask = rdev->lldi.udb_density - 1; | 810 | /* This implementation requires a sge_host_page_size <= PAGE_SIZE. */ |
| 810 | rdev->cqmask = rdev->lldi.ucq_density - 1; | 811 | if (rdev->lldi.sge_host_page_size > PAGE_SIZE) { |
| 812 | pr_err("%s: unsupported sge host page size %u\n", | ||
| 813 | pci_name(rdev->lldi.pdev), | ||
| 814 | rdev->lldi.sge_host_page_size); | ||
| 815 | return -EINVAL; | ||
| 816 | } | ||
| 817 | |||
| 818 | factor = PAGE_SIZE / rdev->lldi.sge_host_page_size; | ||
| 819 | rdev->qpmask = (rdev->lldi.udb_density * factor) - 1; | ||
| 820 | rdev->cqmask = (rdev->lldi.ucq_density * factor) - 1; | ||
| 821 | |||
| 811 | pr_debug("dev %s stag start 0x%0x size 0x%0x num stags %d pbl start 0x%0x size 0x%0x rq start 0x%0x size 0x%0x qp qid start %u size %u cq qid start %u size %u srq size %u\n", | 822 | pr_debug("dev %s stag start 0x%0x size 0x%0x num stags %d pbl start 0x%0x size 0x%0x rq start 0x%0x size 0x%0x qp qid start %u size %u cq qid start %u size %u srq size %u\n", |
| 812 | pci_name(rdev->lldi.pdev), rdev->lldi.vr->stag.start, | 823 | pci_name(rdev->lldi.pdev), rdev->lldi.vr->stag.start, |
| 813 | rdev->lldi.vr->stag.size, c4iw_num_stags(rdev), | 824 | rdev->lldi.vr->stag.size, c4iw_num_stags(rdev), |
| @@ -3032,7 +3032,6 @@ static int srp_reset_device(struct scsi_cmnd *scmnd) · 3031 unmodified lines | |||
| 3032 | { | 3032 | { |
| 3033 | struct srp_target_port *target = host_to_target(scmnd->device->host); | 3033 | struct srp_target_port *target = host_to_target(scmnd->device->host); |
| 3034 | struct srp_rdma_ch *ch; | 3034 | struct srp_rdma_ch *ch; |
| 3035 | int i, j; | ||
| 3036 | u8 status; | 3035 | u8 status; |
| 3037 | 3036 | ||
| 3038 | shost_printk(KERN_ERR, target->scsi_host, "SRP reset_device called\n"); | 3037 | shost_printk(KERN_ERR, target->scsi_host, "SRP reset_device called\n"); |
| @@ -3044,15 +3043,6 @@ static int srp_reset_device(struct scsi_cmnd *scmnd) · 5 unmodified lines | |||
| 3044 | if (status) | 3043 | if (status) |
| 3045 | return FAILED; | 3044 | return FAILED; |
| 3046 | 3045 | ||
| 3047 | for (i = 0; i < target->ch_count; i++) { | ||
| 3048 | ch = &target->ch[i]; | ||
| 3049 | for (j = 0; j < target->req_ring_size; ++j) { | ||
| 3050 | struct srp_request *req = &ch->req_ring[j]; | ||
| 3051 | |||
| 3052 | srp_finish_req(ch, req, scmnd->device, DID_RESET << 16); | ||
| 3053 | } | ||
| 3054 | } | ||
| 3055 | |||
| 3056 | return SUCCESS; | 3046 | return SUCCESS; |
| 3057 | } | 3047 | } |
| 3058 | 3048 | ||
| @@ -144,7 +144,7 @@ dmar_alloc_pci_notify_info(struct pci_dev *dev, unsigned long event) · 143 unmodified lines | |||
| 144 | for (tmp = dev; tmp; tmp = tmp->bus->self) | 144 | for (tmp = dev; tmp; tmp = tmp->bus->self) |
| 145 | level++; | 145 | level++; |
| 146 | 146 | ||
| 147 | size = sizeof(*info) + level * sizeof(struct acpi_dmar_pci_path); | 147 | size = sizeof(*info) + level * sizeof(info->path[0]); |
| 148 | if (size <= sizeof(dmar_pci_notify_info_buf)) { | 148 | if (size <= sizeof(dmar_pci_notify_info_buf)) { |
| 149 | info = (struct dmar_pci_notify_info *)dmar_pci_notify_info_buf; | 149 | info = (struct dmar_pci_notify_info *)dmar_pci_notify_info_buf; |
| 150 | } else { | 150 | } else { |
| @@ -1396,9 +1396,9 @@ static void flexrm_shutdown(struct mbox_chan *chan) · 1395 unmodified lines | |||
| 1396 | 1396 | ||
| 1397 | /* Clear ring flush state */ | 1397 | /* Clear ring flush state */ |
| 1398 | timeout = 1000; /* timeout of 1s */ | 1398 | timeout = 1000; /* timeout of 1s */ |
| 1399 | writel_relaxed(0x0, ring + RING_CONTROL); | 1399 | writel_relaxed(0x0, ring->regs + RING_CONTROL); |
| 1400 | do { | 1400 | do { |
| 1401 | if (!(readl_relaxed(ring + RING_FLUSH_DONE) & | 1401 | if (!(readl_relaxed(ring->regs + RING_FLUSH_DONE) & |
| 1402 | FLUSH_DONE_MASK)) | 1402 | FLUSH_DONE_MASK)) |
| 1403 | break; | 1403 | break; |
| 1404 | mdelay(1); | 1404 | mdelay(1); |
| @@ -310,6 +310,7 @@ int mbox_flush(struct mbox_chan *chan, unsigned long timeout) · 309 unmodified lines | |||
| 310 | 310 | ||
| 311 | return ret; | 311 | return ret; |
| 312 | } | 312 | } |
| 313 | EXPORT_SYMBOL_GPL(mbox_flush); | ||
| 313 | 314 | ||
| 314 | /** | 315 | /** |
| 315 | * mbox_request_channel - Request a mailbox channel. | 316 | * mbox_request_channel - Request a mailbox channel. |
| @@ -2380,12 +2380,6 @@ static struct mmc_blk_data *mmc_blk_alloc_req(struct mmc_card *card, · 2379 unmodified lines | |||
| 2380 | snprintf(md->disk->disk_name, sizeof(md->disk->disk_name), | 2380 | snprintf(md->disk->disk_name, sizeof(md->disk->disk_name), |
| 2381 | "mmcblk%u%s", card->host->index, subname ? subname : ""); | 2381 | "mmcblk%u%s", card->host->index, subname ? subname : ""); |
| 2382 | 2382 | ||
| 2383 | if (mmc_card_mmc(card)) | ||
| 2384 | blk_queue_logical_block_size(md->queue.queue, | ||
| 2385 | card->ext_csd.data_sector_size); | ||
| 2386 | else | ||
| 2387 | blk_queue_logical_block_size(md->queue.queue, 512); | ||
| 2388 | |||
| 2389 | set_capacity(md->disk, size); | 2383 | set_capacity(md->disk, size); |
| 2390 | 2384 | ||
| 2391 | if (mmc_host_cmd23(card->host)) { | 2385 | if (mmc_host_cmd23(card->host)) { |
| @@ -95,7 +95,7 @@ static void mmc_should_fail_request(struct mmc_host *host, · 94 unmodified lines | |||
| 95 | if (!data) | 95 | if (!data) |
| 96 | return; | 96 | return; |
| 97 | 97 | ||
| 98 | if (cmd->error || data->error || | 98 | if ((cmd && cmd->error) || data->error || |
| 99 | !should_fail(&host->fail_mmc_request, data->blksz * data->blocks)) | 99 | !should_fail(&host->fail_mmc_request, data->blksz * data->blocks)) |
| 100 | return; | 100 | return; |
| 101 | 101 | ||
| @@ -355,6 +355,7 @@ static void mmc_setup_queue(struct mmc_queue *mq, struct mmc_card *card) · 354 unmodified lines | |||
| 355 | { | 355 | { |
| 356 | struct mmc_host *host = card->host; | 356 | struct mmc_host *host = card->host; |
| 357 | u64 limit = BLK_BOUNCE_HIGH; | 357 | u64 limit = BLK_BOUNCE_HIGH; |
| 358 | unsigned block_size = 512; | ||
| 358 | 359 | ||
| 359 | if (mmc_dev(host)->dma_mask && *mmc_dev(host)->dma_mask) | 360 | if (mmc_dev(host)->dma_mask && *mmc_dev(host)->dma_mask) |
| 360 | limit = (u64)dma_max_pfn(mmc_dev(host)) << PAGE_SHIFT; | 361 | limit = (u64)dma_max_pfn(mmc_dev(host)) << PAGE_SHIFT; |
| @@ -368,7 +369,13 @@ static void mmc_setup_queue(struct mmc_queue *mq, struct mmc_card *card) · 7 unmodified lines | |||
| 368 | blk_queue_max_hw_sectors(mq->queue, | 369 | blk_queue_max_hw_sectors(mq->queue, |
| 369 | min(host->max_blk_count, host->max_req_size / 512)); | 370 | min(host->max_blk_count, host->max_req_size / 512)); |
| 370 | blk_queue_max_segments(mq->queue, host->max_segs); | 371 | blk_queue_max_segments(mq->queue, host->max_segs); |
| 371 | blk_queue_max_segment_size(mq->queue, host->max_seg_size); | 372 | |
| 373 | if (mmc_card_mmc(card)) | ||
| 374 | block_size = card->ext_csd.data_sector_size; | ||
| 375 | |||
| 376 | blk_queue_logical_block_size(mq->queue, block_size); | ||
| 377 | blk_queue_max_segment_size(mq->queue, | ||
| 378 | round_down(host->max_seg_size, block_size)); | ||
| 372 | 379 | ||
| 373 | INIT_WORK(&mq->recovery_work, mmc_mq_recovery_handler); | 380 | INIT_WORK(&mq->recovery_work, mmc_mq_recovery_handler); |
| 374 | INIT_WORK(&mq->complete_work, mmc_blk_mq_complete_work); | 381 | INIT_WORK(&mq->complete_work, mmc_blk_mq_complete_work); |
| @@ -201,7 +201,7 @@ static int cqhci_host_alloc_tdl(struct cqhci_host *cq_host) · 200 unmodified lines | |||
| 201 | cq_host->desc_size = cq_host->slot_sz * cq_host->num_slots; | 201 | cq_host->desc_size = cq_host->slot_sz * cq_host->num_slots; |
| 202 | 202 | ||
| 203 | cq_host->data_size = cq_host->trans_desc_len * cq_host->mmc->max_segs * | 203 | cq_host->data_size = cq_host->trans_desc_len * cq_host->mmc->max_segs * |
| 204 | (cq_host->num_slots - 1); | 204 | cq_host->mmc->cqe_qdepth; |
| 205 | 205 | ||
| 206 | pr_debug("%s: cqhci: desc_size: %zu data_sz: %zu slot-sz: %d\n", | 206 | pr_debug("%s: cqhci: desc_size: %zu data_sz: %zu slot-sz: %d\n", |
| 207 | mmc_hostname(cq_host->mmc), cq_host->desc_size, cq_host->data_size, | 207 | mmc_hostname(cq_host->mmc), cq_host->desc_size, cq_host->data_size, |
| @@ -217,12 +217,21 @@ static int cqhci_host_alloc_tdl(struct cqhci_host *cq_host) · 9 unmodified lines | |||
| 217 | cq_host->desc_size, | 217 | cq_host->desc_size, |
| 218 | &cq_host->desc_dma_base, | 218 | &cq_host->desc_dma_base, |
| 219 | GFP_KERNEL); | 219 | GFP_KERNEL); |
| 220 | if (!cq_host->desc_base) | ||
| 221 | return -ENOMEM; | ||
| 222 | |||
| 220 | cq_host->trans_desc_base = dmam_alloc_coherent(mmc_dev(cq_host->mmc), | 223 | cq_host->trans_desc_base = dmam_alloc_coherent(mmc_dev(cq_host->mmc), |
| 221 | cq_host->data_size, | 224 | cq_host->data_size, |
| 222 | &cq_host->trans_desc_dma_base, | 225 | &cq_host->trans_desc_dma_base, |
| 223 | GFP_KERNEL); | 226 | GFP_KERNEL); |
| 224 | if (!cq_host->desc_base || !cq_host->trans_desc_base) | 227 | if (!cq_host->trans_desc_base) { |
| 228 | dmam_free_coherent(mmc_dev(cq_host->mmc), cq_host->desc_size, | ||
| 229 | cq_host->desc_base, | ||
| 230 | cq_host->desc_dma_base); | ||
| 231 | cq_host->desc_base = NULL; | ||
| 232 | cq_host->desc_dma_base = 0; | ||
| 225 | return -ENOMEM; | 233 | return -ENOMEM; |
| 234 | } | ||
| 226 | 235 | ||
| 227 | pr_debug("%s: cqhci: desc-base: 0x%p trans-base: 0x%p\n desc_dma 0x%llx trans_dma: 0x%llx\n", | 236 | pr_debug("%s: cqhci: desc-base: 0x%p trans-base: 0x%p\n desc_dma 0x%llx trans_dma: 0x%llx\n", |
| 228 | mmc_hostname(cq_host->mmc), cq_host->desc_base, cq_host->trans_desc_base, | 237 | mmc_hostname(cq_host->mmc), cq_host->desc_base, cq_host->trans_desc_base, |
| @@ -1450,6 +1450,7 @@ static int mmc_spi_probe(struct spi_device *spi) · 1449 unmodified lines | |||
| 1450 | mmc->caps &= ~MMC_CAP_NEEDS_POLL; | 1450 | mmc->caps &= ~MMC_CAP_NEEDS_POLL; |
| 1451 | mmc_gpiod_request_cd_irq(mmc); | 1451 | mmc_gpiod_request_cd_irq(mmc); |
| 1452 | } | 1452 | } |
| 1453 | mmc_detect_change(mmc, 0); | ||
| 1453 | 1454 | ||
| 1454 | /* Index 1 is write protect/read only */ | 1455 | /* Index 1 is write protect/read only */ |
| 1455 | status = mmc_gpiod_request_ro(mmc, NULL, 1, false, 0, NULL); | 1456 | status = mmc_gpiod_request_ro(mmc, NULL, 1, false, 0, NULL); |
| @@ -65,6 +65,7 @@ static const struct renesas_sdhi_of_data of_rcar_gen2_compatible = { · 64 unmodified lines | |||
| 65 | .scc_offset = 0x0300, | 65 | .scc_offset = 0x0300, |
| 66 | .taps = rcar_gen2_scc_taps, | 66 | .taps = rcar_gen2_scc_taps, |
| 67 | .taps_num = ARRAY_SIZE(rcar_gen2_scc_taps), | 67 | .taps_num = ARRAY_SIZE(rcar_gen2_scc_taps), |
| 68 | .max_blk_count = 0xffffffff, | ||
| 68 | }; | 69 | }; |
| 69 | 70 | ||
| 70 | /* Definitions for sampling clocks */ | 71 | /* Definitions for sampling clocks */ |
| @@ -1095,11 +1095,12 @@ static void sdhci_esdhc_imx_hwinit(struct sdhci_host *host) · 1094 unmodified lines | |||
| 1095 | writel(readl(host->ioaddr + SDHCI_HOST_CONTROL) | 1095 | writel(readl(host->ioaddr + SDHCI_HOST_CONTROL) |
| 1096 | | ESDHC_BURST_LEN_EN_INCR, | 1096 | | ESDHC_BURST_LEN_EN_INCR, |
| 1097 | host->ioaddr + SDHCI_HOST_CONTROL); | 1097 | host->ioaddr + SDHCI_HOST_CONTROL); |
| 1098 | |||
| 1098 | /* | 1099 | /* |
| 1099 | * erratum ESDHC_FLAG_ERR004536 fix for MX6Q TO1.2 and MX6DL | 1100 | * erratum ESDHC_FLAG_ERR004536 fix for MX6Q TO1.2 and MX6DL |
| 1100 | * TO1.1, it's harmless for MX6SL | 1101 | * TO1.1, it's harmless for MX6SL |
| 1101 | */ | 1102 | */ |
| 1102 | writel(readl(host->ioaddr + 0x6c) | BIT(7), | 1103 | writel(readl(host->ioaddr + 0x6c) & ~BIT(7), |
| 1103 | host->ioaddr + 0x6c); | 1104 | host->ioaddr + 0x6c); |
| 1104 | 1105 | ||
| 1105 | /* disable DLL_CTRL delay line settings */ | 1106 | /* disable DLL_CTRL delay line settings */ |
| @@ -277,6 +277,11 @@ static inline void sd_ctrl_write32_as_16_and_16(struct tmio_mmc_host *host, · 276 unmodified lines | |||
| 277 | iowrite16(val >> 16, host->ctl + ((addr + 2) << host->bus_shift)); | 277 | iowrite16(val >> 16, host->ctl + ((addr + 2) << host->bus_shift)); |
| 278 | } | 278 | } |
| 279 | 279 | ||
| 280 | static inline void sd_ctrl_write32(struct tmio_mmc_host *host, int addr, u32 val) | ||
| 281 | { | ||
| 282 | iowrite32(val, host->ctl + (addr << host->bus_shift)); | ||
| 283 | } | ||
| 284 | |||
| 280 | static inline void sd_ctrl_write32_rep(struct tmio_mmc_host *host, int addr, | 285 | static inline void sd_ctrl_write32_rep(struct tmio_mmc_host *host, int addr, |
| 281 | const u32 *buf, int count) | 286 | const u32 *buf, int count) |
| 282 | { | 287 | { |
| @@ -43,6 +43,7 @@ · 42 unmodified lines | |||
| 43 | #include <linux/regulator/consumer.h> | 43 | #include <linux/regulator/consumer.h> |
| 44 | #include <linux/mmc/sdio.h> | 44 | #include <linux/mmc/sdio.h> |
| 45 | #include <linux/scatterlist.h> | 45 | #include <linux/scatterlist.h> |
| 46 | #include <linux/sizes.h> | ||
| 46 | #include <linux/spinlock.h> | 47 | #include <linux/spinlock.h> |
| 47 | #include <linux/swiotlb.h> | 48 | #include <linux/swiotlb.h> |
| 48 | #include <linux/workqueue.h> | 49 | #include <linux/workqueue.h> |
| @@ -629,7 +630,7 @@ static bool __tmio_mmc_sdcard_irq(struct tmio_mmc_host *host, int ireg, · 580 unmodified lines | |||
| 629 | return false; | 630 | return false; |
| 630 | } | 631 | } |
| 631 | 632 | ||
| 632 | static void __tmio_mmc_sdio_irq(struct tmio_mmc_host *host) | 633 | static bool __tmio_mmc_sdio_irq(struct tmio_mmc_host *host) |
| 633 | { | 634 | { |
| 634 | struct mmc_host *mmc = host->mmc; | 635 | struct mmc_host *mmc = host->mmc; |
| 635 | struct tmio_mmc_data *pdata = host->pdata; | 636 | struct tmio_mmc_data *pdata = host->pdata; |
| @@ -637,7 +638,7 @@ static void __tmio_mmc_sdio_irq(struct tmio_mmc_host *host) · 1 unmodified line | |||
| 637 | unsigned int sdio_status; | 638 | unsigned int sdio_status; |
| 638 | 639 | ||
| 639 | if (!(pdata->flags & TMIO_MMC_SDIO_IRQ)) | 640 | if (!(pdata->flags & TMIO_MMC_SDIO_IRQ)) |
| 640 | return; | 641 | return false; |
| 641 | 642 | ||
| 642 | status = sd_ctrl_read16(host, CTL_SDIO_STATUS); | 643 | status = sd_ctrl_read16(host, CTL_SDIO_STATUS); |
| 643 | ireg = status & TMIO_SDIO_MASK_ALL & ~host->sdio_irq_mask; | 644 | ireg = status & TMIO_SDIO_MASK_ALL & ~host->sdio_irq_mask; |
| @@ -650,6 +651,8 @@ static void __tmio_mmc_sdio_irq(struct tmio_mmc_host *host) · 6 unmodified lines | |||
| 650 | 651 | ||
| 651 | if (mmc->caps & MMC_CAP_SDIO_IRQ && ireg & TMIO_SDIO_STAT_IOIRQ) | 652 | if (mmc->caps & MMC_CAP_SDIO_IRQ && ireg & TMIO_SDIO_STAT_IOIRQ) |
| 652 | mmc_signal_sdio_irq(mmc); | 653 | mmc_signal_sdio_irq(mmc); |
| 654 | |||
| 655 | return ireg; | ||
| 653 | } | 656 | } |
| 654 | 657 | ||
| 655 | irqreturn_t tmio_mmc_irq(int irq, void *devid) | 658 | irqreturn_t tmio_mmc_irq(int irq, void *devid) |
| @@ -668,9 +671,10 @@ irqreturn_t tmio_mmc_irq(int irq, void *devid) · 12 unmodified lines | |||
| 668 | if (__tmio_mmc_sdcard_irq(host, ireg, status)) | 671 | if (__tmio_mmc_sdcard_irq(host, ireg, status)) |
| 669 | return IRQ_HANDLED; | 672 | return IRQ_HANDLED; |
| 670 | 673 | ||
| 671 | __tmio_mmc_sdio_irq(host); | 674 | if (__tmio_mmc_sdio_irq(host)) |
| 675 | return IRQ_HANDLED; | ||
| 672 | 676 | ||
| 673 | return IRQ_HANDLED; | 677 | return IRQ_NONE; |
| 674 | } | 678 | } |
| 675 | EXPORT_SYMBOL_GPL(tmio_mmc_irq); | 679 | EXPORT_SYMBOL_GPL(tmio_mmc_irq); |
| 676 | 680 | ||
| @@ -700,7 +704,10 @@ static int tmio_mmc_start_data(struct tmio_mmc_host *host, · 23 unmodified lines | |||
| 700 | 704 | ||
| 701 | /* Set transfer length / blocksize */ | 705 | /* Set transfer length / blocksize */ |
| 702 | sd_ctrl_write16(host, CTL_SD_XFER_LEN, data->blksz); | 706 | sd_ctrl_write16(host, CTL_SD_XFER_LEN, data->blksz); |
| 703 | sd_ctrl_write16(host, CTL_XFER_BLK_COUNT, data->blocks); | 707 | if (host->mmc->max_blk_count >= SZ_64K) |
| 708 | sd_ctrl_write32(host, CTL_XFER_BLK_COUNT, data->blocks); | ||
| 709 | else | ||
| 710 | sd_ctrl_write16(host, CTL_XFER_BLK_COUNT, data->blocks); | ||
| 704 | 711 | ||
| 705 | tmio_mmc_start_dma(host, data); | 712 | tmio_mmc_start_dma(host, data); |
| 706 | 713 | ||
| @@ -212,7 +212,7 @@ static int powernv_flash_set_driver_info(struct device *dev, · 211 unmodified lines | |||
| 212 | * Going to have to check what details I need to set and how to | 212 | * Going to have to check what details I need to set and how to |
| 213 | * get them | 213 | * get them |
| 214 | */ | 214 | */ |
| 215 | mtd->name = devm_kasprintf(dev, GFP_KERNEL, "%pOFn", dev->of_node); | 215 | mtd->name = devm_kasprintf(dev, GFP_KERNEL, "%pOFP", dev->of_node); |
| 216 | mtd->type = MTD_NORFLASH; | 216 | mtd->type = MTD_NORFLASH; |
| 217 | mtd->flags = MTD_WRITEABLE; | 217 | mtd->flags = MTD_WRITEABLE; |
| 218 | mtd->size = size; | 218 | mtd->size = size; |
| @@ -507,6 +507,7 @@ static int mtd_nvmem_add(struct mtd_info *mtd) · 506 unmodified lines | |||
| 507 | { | 507 | { |
| 508 | struct nvmem_config config = {}; | 508 | struct nvmem_config config = {}; |
| 509 | 509 | ||
| 510 | config.id = -1; | ||
| 510 | config.dev = &mtd->dev; | 511 | config.dev = &mtd->dev; |
| 511 | config.name = mtd->name; | 512 | config.name = mtd->name; |
| 512 | config.owner = THIS_MODULE; | 513 | config.owner = THIS_MODULE; |
| @@ -1183,29 +1183,22 @@ static rx_handler_result_t bond_handle_frame(struct sk_buff **pskb) · 1182 unmodified lines | |||
| 1183 | } | 1183 | } |
| 1184 | } | 1184 | } |
| 1185 | 1185 | ||
| 1186 | /* Link-local multicast packets should be passed to the | 1186 | /* |
| 1187 | * stack on the link they arrive as well as pass them to the | 1187 | * For packets determined by bond_should_deliver_exact_match() call to |
| 1188 | * bond-master device. These packets are mostly usable when | 1188 | * be suppressed we want to make an exception for link-local packets. |
| 1189 | * stack receives it with the link on which they arrive | 1189 | * This is necessary for e.g. LLDP daemons to be able to monitor |
| 1190 | * (e.g. LLDP) they also must be available on master. Some of | 1190 | * inactive slave links without being forced to bind to them |
| 1191 | * the use cases include (but are not limited to): LLDP agents | 1191 | * explicitly. |
| 1192 | * that must be able to operate both on enslaved interfaces as | 1192 | * |
| 1193 | * well as on bonds themselves; linux bridges that must be able | 1193 | * At the same time, packets that are passed to the bonding master |
| 1194 | * to process/pass BPDUs from attached bonds when any kind of | 1194 | * (including link-local ones) can have their originating interface |
| 1195 | * STP version is enabled on the network. | 1195 | * determined via PACKET_ORIGDEV socket option. |
| 1196 | */ | 1196 | */ |
| 1197 | if (is_link_local_ether_addr(eth_hdr(skb)->h_dest)) { | 1197 | if (bond_should_deliver_exact_match(skb, slave, bond)) { |
| 1198 | struct sk_buff *nskb = skb_clone(skb, GFP_ATOMIC); | 1198 | if (is_link_local_ether_addr(eth_hdr(skb)->h_dest)) |
| 1199 | 1199 | return RX_HANDLER_PASS; | |
| 1200 | if (nskb) { | ||
| 1201 | nskb->dev = bond->dev; | ||
| 1202 | nskb->queue_mapping = 0; | ||
| 1203 | netif_rx(nskb); | ||
| 1204 | } | ||
| 1205 | return RX_HANDLER_PASS; | ||
| 1206 | } | ||
| 1207 | if (bond_should_deliver_exact_match(skb, slave, bond)) | ||
| 1208 | return RX_HANDLER_EXACT; | 1200 | return RX_HANDLER_EXACT; |
| 1201 | } | ||
| 1209 | 1202 | ||
| 1210 | skb->dev = bond->dev; | 1203 | skb->dev = bond->dev; |
| 1211 | 1204 | ||
| @@ -344,7 +344,8 @@ static void b53_set_forwarding(struct b53_device *dev, int enable) · 343 unmodified lines | |||
| 344 | b53_write8(dev, B53_CTRL_PAGE, B53_SWITCH_CTRL, mgmt); | 344 | b53_write8(dev, B53_CTRL_PAGE, B53_SWITCH_CTRL, mgmt); |
| 345 | } | 345 | } |
| 346 | 346 | ||
| 347 | static void b53_enable_vlan(struct b53_device *dev, bool enable) | 347 | static void b53_enable_vlan(struct b53_device *dev, bool enable, |
| 348 | bool enable_filtering) | ||
| 348 | { | 349 | { |
| 349 | u8 mgmt, vc0, vc1, vc4 = 0, vc5; | 350 | u8 mgmt, vc0, vc1, vc4 = 0, vc5; |
| 350 | 351 | ||
| @@ -369,8 +370,13 @@ static void b53_enable_vlan(struct b53_device *dev, bool enable) · 18 unmodified lines | |||
| 369 | vc0 |= VC0_VLAN_EN | VC0_VID_CHK_EN | VC0_VID_HASH_VID; | 370 | vc0 |= VC0_VLAN_EN | VC0_VID_CHK_EN | VC0_VID_HASH_VID; |
| 370 | vc1 |= VC1_RX_MCST_UNTAG_EN | VC1_RX_MCST_FWD_EN; | 371 | vc1 |= VC1_RX_MCST_UNTAG_EN | VC1_RX_MCST_FWD_EN; |
| 371 | vc4 &= ~VC4_ING_VID_CHECK_MASK; | 372 | vc4 &= ~VC4_ING_VID_CHECK_MASK; |
| 372 | vc4 |= VC4_ING_VID_VIO_DROP << VC4_ING_VID_CHECK_S; | 373 | if (enable_filtering) { |
| 373 | vc5 |= VC5_DROP_VTABLE_MISS; | 374 | vc4 |= VC4_ING_VID_VIO_DROP << VC4_ING_VID_CHECK_S; |
| 375 | vc5 |= VC5_DROP_VTABLE_MISS; | ||
| 376 | } else { | ||
| 377 | vc4 |= VC4_ING_VID_VIO_FWD << VC4_ING_VID_CHECK_S; | ||
| 378 | vc5 &= ~VC5_DROP_VTABLE_MISS; | ||
| 379 | } | ||
| 374 | 380 | ||
| 375 | if (is5325(dev)) | 381 | if (is5325(dev)) |
| 376 | vc0 &= ~VC0_RESERVED_1; | 382 | vc0 &= ~VC0_RESERVED_1; |
| @@ -420,6 +426,9 @@ static void b53_enable_vlan(struct b53_device *dev, bool enable) · 43 unmodified lines | |||
| 420 | } | 426 | } |
| 421 | 427 | ||
| 422 | b53_write8(dev, B53_CTRL_PAGE, B53_SWITCH_MODE, mgmt); | 428 | b53_write8(dev, B53_CTRL_PAGE, B53_SWITCH_MODE, mgmt); |
| 429 | |||
| 430 | dev->vlan_enabled = enable; | ||
| 431 | dev->vlan_filtering_enabled = enable_filtering; | ||
| 423 | } | 432 | } |
| 424 | 433 | ||
| 425 | static int b53_set_jumbo(struct b53_device *dev, bool enable, bool allow_10_100) | 434 | static int b53_set_jumbo(struct b53_device *dev, bool enable, bool allow_10_100) |
| @@ -632,25 +641,35 @@ static void b53_enable_mib(struct b53_device *dev) · 206 unmodified lines | |||
| 632 | b53_write8(dev, B53_MGMT_PAGE, B53_GLOBAL_CONFIG, gc); | 641 | b53_write8(dev, B53_MGMT_PAGE, B53_GLOBAL_CONFIG, gc); |
| 633 | } | 642 | } |
| 634 | 643 | ||
| 644 | static u16 b53_default_pvid(struct b53_device *dev) | ||
| 645 | { | ||
| 646 | if (is5325(dev) || is5365(dev)) | ||
| 647 | return 1; | ||
| 648 | else | ||
| 649 | return 0; | ||
| 650 | } | ||
| 651 | |||
| 635 | int b53_configure_vlan(struct dsa_switch *ds) | 652 | int b53_configure_vlan(struct dsa_switch *ds) |
| 636 | { | 653 | { |
| 637 | struct b53_device *dev = ds->priv; | 654 | struct b53_device *dev = ds->priv; |
| 638 | struct b53_vlan vl = { 0 }; | 655 | struct b53_vlan vl = { 0 }; |
| 639 | int i; | 656 | int i, def_vid; |
| 657 | |||
| 658 | def_vid = b53_default_pvid(dev); | ||
| 640 | 659 | ||
| 641 | /* clear all vlan entries */ | 660 | /* clear all vlan entries */ |
| 642 | if (is5325(dev) || is5365(dev)) { | 661 | if (is5325(dev) || is5365(dev)) { |
| 643 | for (i = 1; i < dev->num_vlans; i++) | 662 | for (i = def_vid; i < dev->num_vlans; i++) |
| 644 | b53_set_vlan_entry(dev, i, &vl); | 663 | b53_set_vlan_entry(dev, i, &vl); |
| 645 | } else { | 664 | } else { |
| 646 | b53_do_vlan_op(dev, VTA_CMD_CLEAR); | 665 | b53_do_vlan_op(dev, VTA_CMD_CLEAR); |
| 647 | } | 666 | } |
| 648 | 667 | ||
| 649 | b53_enable_vlan(dev, false); | 668 | b53_enable_vlan(dev, false, dev->vlan_filtering_enabled); |
| 650 | 669 | ||
| 651 | b53_for_each_port(dev, i) | 670 | b53_for_each_port(dev, i) |
| 652 | b53_write16(dev, B53_VLAN_PAGE, | 671 | b53_write16(dev, B53_VLAN_PAGE, |
| 653 | B53_VLAN_PORT_DEF_TAG(i), 1); | 672 | B53_VLAN_PORT_DEF_TAG(i), def_vid); |
| 654 | 673 | ||
| 655 | if (!is5325(dev) && !is5365(dev)) | 674 | if (!is5325(dev) && !is5365(dev)) |
| 656 | b53_set_jumbo(dev, dev->enable_jumbo, false); | 675 | b53_set_jumbo(dev, dev->enable_jumbo, false); |
| @@ -1255,6 +1274,46 @@ EXPORT_SYMBOL(b53_phylink_mac_link_up); · 598 unmodified lines | |||
| 1255 | 1274 | ||
| 1256 | int b53_vlan_filtering(struct dsa_switch *ds, int port, bool vlan_filtering) | 1275 | int b53_vlan_filtering(struct dsa_switch *ds, int port, bool vlan_filtering) |
| 1257 | { | 1276 | { |
| 1277 | struct b53_device *dev = ds->priv; | ||
| 1278 | struct net_device *bridge_dev; | ||
| 1279 | unsigned int i; | ||
| 1280 | u16 pvid, new_pvid; | ||
| 1281 | |||
| 1282 | /* Handle the case were multiple bridges span the same switch device | ||
| 1283 | * and one of them has a different setting than what is being requested | ||
| 1284 | * which would be breaking filtering semantics for any of the other | ||
| 1285 | * bridge devices. | ||
| 1286 | */ | ||
| 1287 | b53_for_each_port(dev, i) { | ||
| 1288 | bridge_dev = dsa_to_port(ds, i)->bridge_dev; | ||
| 1289 | if (bridge_dev && | ||
| 1290 | bridge_dev != dsa_to_port(ds, port)->bridge_dev && | ||
| 1291 | br_vlan_enabled(bridge_dev) != vlan_filtering) { | ||
| 1292 | netdev_err(bridge_dev, | ||
| 1293 | "VLAN filtering is global to the switch!\n"); | ||
| 1294 | return -EINVAL; | ||
| 1295 | } | ||
| 1296 | } | ||
| 1297 | |||
| 1298 | b53_read16(dev, B53_VLAN_PAGE, B53_VLAN_PORT_DEF_TAG(port), &pvid); | ||
| 1299 | new_pvid = pvid; | ||
| 1300 | if (dev->vlan_filtering_enabled && !vlan_filtering) { | ||
| 1301 | /* Filtering is currently enabled, use the default PVID since | ||
| 1302 | * the bridge does not expect tagging anymore | ||
| 1303 | */ | ||
| 1304 | dev->ports[port].pvid = pvid; | ||
| 1305 | new_pvid = b53_default_pvid(dev); | ||
| 1306 | } else if (!dev->vlan_filtering_enabled && vlan_filtering) { | ||
| 1307 | /* Filtering is currently disabled, restore the previous PVID */ | ||
| 1308 | new_pvid = dev->ports[port].pvid; | ||
| 1309 | } | ||
| 1310 | |||
| 1311 | if (pvid != new_pvid) | ||
| 1312 | b53_write16(dev, B53_VLAN_PAGE, B53_VLAN_PORT_DEF_TAG(port), | ||
| 1313 | new_pvid); | ||
| 1314 | |||
| 1315 | b53_enable_vlan(dev, dev->vlan_enabled, vlan_filtering); | ||
| 1316 | |||
| 1258 | return 0; | 1317 | return 0; |
| 1259 | } | 1318 | } |
| 1260 | EXPORT_SYMBOL(b53_vlan_filtering); | 1319 | EXPORT_SYMBOL(b53_vlan_filtering); |
| @@ -1270,7 +1329,7 @@ int b53_vlan_prepare(struct dsa_switch *ds, int port, · 9 unmodified lines | |||
| 1270 | if (vlan->vid_end > dev->num_vlans) | 1329 | if (vlan->vid_end > dev->num_vlans) |
| 1271 | return -ERANGE; | 1330 | return -ERANGE; |
| 1272 | 1331 | ||
| 1273 | b53_enable_vlan(dev, true); | 1332 | b53_enable_vlan(dev, true, dev->vlan_filtering_enabled); |
| 1274 | 1333 | ||
| 1275 | return 0; | 1334 | return 0; |
| 1276 | } | 1335 | } |
| @@ -1300,7 +1359,7 @@ void b53_vlan_add(struct dsa_switch *ds, int port, · 23 unmodified lines | |||
| 1300 | b53_fast_age_vlan(dev, vid); | 1359 | b53_fast_age_vlan(dev, vid); |
| 1301 | } | 1360 | } |
| 1302 | 1361 | ||
| 1303 | if (pvid) { | 1362 | if (pvid && !dsa_is_cpu_port(ds, port)) { |
| 1304 | b53_write16(dev, B53_VLAN_PAGE, B53_VLAN_PORT_DEF_TAG(port), | 1363 | b53_write16(dev, B53_VLAN_PAGE, B53_VLAN_PORT_DEF_TAG(port), |
| 1305 | vlan->vid_end); | 1364 | vlan->vid_end); |
| 1306 | b53_fast_age_vlan(dev, vid); | 1365 | b53_fast_age_vlan(dev, vid); |
| @@ -1326,12 +1385,8 @@ int b53_vlan_del(struct dsa_switch *ds, int port, · 19 unmodified lines | |||
| 1326 | 1385 | ||
| 1327 | vl->members &= ~BIT(port); | 1386 | vl->members &= ~BIT(port); |
| 1328 | 1387 | ||
| 1329 | if (pvid == vid) { | 1388 | if (pvid == vid) |
| 1330 | if (is5325(dev) || is5365(dev)) | 1389 | pvid = b53_default_pvid(dev); |
| 1331 | pvid = 1; | ||
| 1332 | else | ||
| 1333 | pvid = 0; | ||
| 1334 | } | ||
| 1335 | 1390 | ||
| 1336 | if (untagged && !dsa_is_cpu_port(ds, port)) | 1391 | if (untagged && !dsa_is_cpu_port(ds, port)) |
| 1337 | vl->untag &= ~(BIT(port)); | 1392 | vl->untag &= ~(BIT(port)); |
| @@ -1644,10 +1699,7 @@ void b53_br_leave(struct dsa_switch *ds, int port, struct net_device *br) · 306 unmodified lines | |||
| 1644 | b53_write16(dev, B53_PVLAN_PAGE, B53_PVLAN_PORT_MASK(port), pvlan); | 1699 | b53_write16(dev, B53_PVLAN_PAGE, B53_PVLAN_PORT_MASK(port), pvlan); |
| 1645 | dev->ports[port].vlan_ctl_mask = pvlan; | 1700 | dev->ports[port].vlan_ctl_mask = pvlan; |
| 1646 | 1701 | ||
| 1647 | if (is5325(dev) || is5365(dev)) | 1702 | pvid = b53_default_pvid(dev); |
| 1648 | pvid = 1; | ||
| 1649 | else | ||
| 1650 | pvid = 0; | ||
| 1651 | 1703 | ||
| 1652 | /* Make this port join all VLANs without VLAN entries */ | 1704 | /* Make this port join all VLANs without VLAN entries */ |
| 1653 | if (is58xx(dev)) { | 1705 | if (is58xx(dev)) { |
| @@ -91,6 +91,7 @@ enum { · 90 unmodified lines | |||
| 91 | struct b53_port { | 91 | struct b53_port { |
| 92 | u16 vlan_ctl_mask; | 92 | u16 vlan_ctl_mask; |
| 93 | struct ethtool_eee eee; | 93 | struct ethtool_eee eee; |
| 94 | u16 pvid; | ||
| 94 | }; | 95 | }; |
| 95 | 96 | ||
| 96 | struct b53_vlan { | 97 | struct b53_vlan { |
| @@ -137,6 +138,8 @@ struct b53_device { · 40 unmodified lines | |||
| 137 | 138 | ||
| 138 | unsigned int num_vlans; | 139 | unsigned int num_vlans; |
| 139 | struct b53_vlan *vlans; | 140 | struct b53_vlan *vlans; |
| 141 | bool vlan_enabled; | ||
| 142 | bool vlan_filtering_enabled; | ||
| 140 | unsigned int num_ports; | 143 | unsigned int num_ports; |
| 141 | struct b53_port *ports; | 144 | struct b53_port *ports; |
| 142 | }; | 145 | }; |
| @@ -726,10 +726,11 @@ static void bcm_sf2_sw_get_wol(struct dsa_switch *ds, int port, · 725 unmodified lines | |||
| 726 | { | 726 | { |
| 727 | struct net_device *p = ds->ports[port].cpu_dp->master; | 727 | struct net_device *p = ds->ports[port].cpu_dp->master; |
| 728 | struct bcm_sf2_priv *priv = bcm_sf2_to_priv(ds); | 728 | struct bcm_sf2_priv *priv = bcm_sf2_to_priv(ds); |
| 729 | struct ethtool_wolinfo pwol; | 729 | struct ethtool_wolinfo pwol = { }; |
| 730 | 730 | ||
| 731 | /* Get the parent device WoL settings */ | 731 | /* Get the parent device WoL settings */ |
| 732 | p->ethtool_ops->get_wol(p, &pwol); | 732 | if (p->ethtool_ops->get_wol) |
| 733 | p->ethtool_ops->get_wol(p, &pwol); | ||
| 733 | 734 | ||
| 734 | /* Advertise the parent device supported settings */ | 735 | /* Advertise the parent device supported settings */ |
| 735 | wol->supported = pwol.supported; | 736 | wol->supported = pwol.supported; |
| @@ -750,9 +751,10 @@ static int bcm_sf2_sw_set_wol(struct dsa_switch *ds, int port, · 14 unmodified lines | |||
| 750 | struct net_device *p = ds->ports[port].cpu_dp->master; | 751 | struct net_device *p = ds->ports[port].cpu_dp->master; |
| 751 | struct bcm_sf2_priv *priv = bcm_sf2_to_priv(ds); | 752 | struct bcm_sf2_priv *priv = bcm_sf2_to_priv(ds); |
| 752 | s8 cpu_port = ds->ports[port].cpu_dp->index; | 753 | s8 cpu_port = ds->ports[port].cpu_dp->index; |
| 753 | struct ethtool_wolinfo pwol; | 754 | struct ethtool_wolinfo pwol = { }; |
| 754 | 755 | ||
| 755 | p->ethtool_ops->get_wol(p, &pwol); | 756 | if (p->ethtool_ops->get_wol) |
| 757 | p->ethtool_ops->get_wol(p, &pwol); | ||
| 756 | if (wol->wolopts & ~pwol.supported) | 758 | if (wol->wolopts & ~pwol.supported) |
| 757 | return -EINVAL; | 759 | return -EINVAL; |
| 758 | 760 | ||
| @@ -1162,6 +1162,12 @@ static struct platform_driver gswip_driver = { · 1161 unmodified lines | |||
| 1162 | 1162 | ||
| 1163 | module_platform_driver(gswip_driver); | 1163 | module_platform_driver(gswip_driver); |
| 1164 | 1164 | ||
| 1165 | MODULE_FIRMWARE("lantiq/xrx300_phy11g_a21.bin"); | ||
| 1166 | MODULE_FIRMWARE("lantiq/xrx300_phy22f_a21.bin"); | ||
| 1167 | MODULE_FIRMWARE("lantiq/xrx200_phy11g_a14.bin"); | ||
| 1168 | MODULE_FIRMWARE("lantiq/xrx200_phy11g_a22.bin"); | ||
| 1169 | MODULE_FIRMWARE("lantiq/xrx200_phy22f_a14.bin"); | ||
| 1170 | MODULE_FIRMWARE("lantiq/xrx200_phy22f_a22.bin"); | ||
| 1165 | MODULE_AUTHOR("Hauke Mehrtens <hauke@hauke-m.de>"); | 1171 | MODULE_AUTHOR("Hauke Mehrtens <hauke@hauke-m.de>"); |
| 1166 | MODULE_DESCRIPTION("Lantiq / Intel GSWIP driver"); | 1172 | MODULE_DESCRIPTION("Lantiq / Intel GSWIP driver"); |
| 1167 | MODULE_LICENSE("GPL v2"); | 1173 | MODULE_LICENSE("GPL v2"); |
| @@ -896,7 +896,7 @@ static uint64_t _mv88e6xxx_get_ethtool_stat(struct mv88e6xxx_chip *chip, · 895 unmodified lines | |||
| 896 | default: | 896 | default: |
| 897 | return U64_MAX; | 897 | return U64_MAX; |
| 898 | } | 898 | } |
| 899 | value = (((u64)high) << 16) | low; | 899 | value = (((u64)high) << 32) | low; |
| 900 | return value; | 900 | return value; |
| 901 | } | 901 | } |
| 902 | 902 | ||
| @@ -3093,7 +3093,7 @@ static const struct mv88e6xxx_ops mv88e6161_ops = { · 2190 unmodified lines | |||
| 3093 | .port_disable_pri_override = mv88e6xxx_port_disable_pri_override, | 3093 | .port_disable_pri_override = mv88e6xxx_port_disable_pri_override, |
| 3094 | .port_link_state = mv88e6352_port_link_state, | 3094 | .port_link_state = mv88e6352_port_link_state, |
| 3095 | .port_get_cmode = mv88e6185_port_get_cmode, | 3095 | .port_get_cmode = mv88e6185_port_get_cmode, |
| 3096 | .stats_snapshot = mv88e6320_g1_stats_snapshot, | 3096 | .stats_snapshot = mv88e6xxx_g1_stats_snapshot, |
| 3097 | .stats_set_histogram = mv88e6095_g1_stats_set_histogram, | 3097 | .stats_set_histogram = mv88e6095_g1_stats_set_histogram, |
| 3098 | .stats_get_sset_count = mv88e6095_stats_get_sset_count, | 3098 | .stats_get_sset_count = mv88e6095_stats_get_sset_count, |
| 3099 | .stats_get_strings = mv88e6095_stats_get_strings, | 3099 | .stats_get_strings = mv88e6095_stats_get_strings, |
| @@ -4595,6 +4595,14 @@ static int mv88e6xxx_smi_init(struct mv88e6xxx_chip *chip, · 1495 unmodified lines | |||
| 4595 | return 0; | 4595 | return 0; |
| 4596 | } | 4596 | } |
| 4597 | 4597 | ||
| 4598 | static void mv88e6xxx_ports_cmode_init(struct mv88e6xxx_chip *chip) | ||
| 4599 | { | ||
| 4600 | int i; | ||
| 4601 | |||
| 4602 | for (i = 0; i < mv88e6xxx_num_ports(chip); i++) | ||
| 4603 | chip->ports[i].cmode = MV88E6XXX_PORT_STS_CMODE_INVALID; | ||
| 4604 | } | ||
| 4605 | |||
| 4598 | static enum dsa_tag_protocol mv88e6xxx_get_tag_protocol(struct dsa_switch *ds, | 4606 | static enum dsa_tag_protocol mv88e6xxx_get_tag_protocol(struct dsa_switch *ds, |
| 4599 | int port) | 4607 | int port) |
| 4600 | { | 4608 | { |
| @@ -4631,6 +4639,8 @@ static const char *mv88e6xxx_drv_probe(struct device *dsa_dev, · 30 unmodified lines | |||
| 4631 | if (err) | 4639 | if (err) |
| 4632 | goto free; | 4640 | goto free; |
| 4633 | 4641 | ||
| 4642 | mv88e6xxx_ports_cmode_init(chip); | ||
| 4643 | |||
| 4634 | mutex_lock(&chip->reg_lock); | 4644 | mutex_lock(&chip->reg_lock); |
| 4635 | err = mv88e6xxx_switch_reset(chip); | 4645 | err = mv88e6xxx_switch_reset(chip); |
| 4636 | mutex_unlock(&chip->reg_lock); | 4646 | mutex_unlock(&chip->reg_lock); |
| @@ -398,6 +398,10 @@ int mv88e6390x_port_set_cmode(struct mv88e6xxx_chip *chip, int port, · 397 unmodified lines | |||
| 398 | cmode = 0; | 398 | cmode = 0; |
| 399 | } | 399 | } |
| 400 | 400 | ||
| 401 | /* cmode doesn't change, nothing to do for us */ | ||
| 402 | if (cmode == chip->ports[port].cmode) | ||
| 403 | return 0; | ||
| 404 | |||
| 401 | lane = mv88e6390x_serdes_get_lane(chip, port); | 405 | lane = mv88e6390x_serdes_get_lane(chip, port); |
| 402 | if (lane < 0) | 406 | if (lane < 0) |
| 403 | return lane; | 407 | return lane; |
| @@ -408,7 +412,7 @@ int mv88e6390x_port_set_cmode(struct mv88e6xxx_chip *chip, int port, · 4 unmodified lines | |||
| 408 | return err; | 412 | return err; |
| 409 | } | 413 | } |
| 410 | 414 | ||
| 411 | err = mv88e6390_serdes_power(chip, port, false); | 415 | err = mv88e6390x_serdes_power(chip, port, false); |
| 412 | if (err) | 416 | if (err) |
| 413 | return err; | 417 | return err; |
| 414 | 418 | ||
| @@ -424,7 +428,7 @@ int mv88e6390x_port_set_cmode(struct mv88e6xxx_chip *chip, int port, · 9 unmodified lines | |||
| 424 | if (err) | 428 | if (err) |
| 425 | return err; | 429 | return err; |
| 426 | 430 | ||
| 427 | err = mv88e6390_serdes_power(chip, port, true); | 431 | err = mv88e6390x_serdes_power(chip, port, true); |
| 428 | if (err) | 432 | if (err) |
| 429 | return err; | 433 | return err; |
| 430 | 434 | ||
| @@ -52,6 +52,7 @@ · 51 unmodified lines | |||
| 52 | #define MV88E6185_PORT_STS_CMODE_1000BASE_X 0x0005 | 52 | #define MV88E6185_PORT_STS_CMODE_1000BASE_X 0x0005 |
| 53 | #define MV88E6185_PORT_STS_CMODE_PHY 0x0006 | 53 | #define MV88E6185_PORT_STS_CMODE_PHY 0x0006 |
| 54 | #define MV88E6185_PORT_STS_CMODE_DISABLED 0x0007 | 54 | #define MV88E6185_PORT_STS_CMODE_DISABLED 0x0007 |
| 55 | #define MV88E6XXX_PORT_STS_CMODE_INVALID 0xff | ||
| 55 | 56 | ||
| 56 | /* Offset 0x01: MAC (or PCS or Physical) Control Register */ | 57 | /* Offset 0x01: MAC (or PCS or Physical) Control Register */ |
| 57 | #define MV88E6XXX_PORT_MAC_CTL 0x01 | 58 | #define MV88E6XXX_PORT_MAC_CTL 0x01 |
| @@ -275,6 +275,9 @@ static int hw_atl_b0_hw_offload_set(struct aq_hw_s *self, · 274 unmodified lines | |||
| 275 | 275 | ||
| 276 | static int hw_atl_b0_hw_init_tx_path(struct aq_hw_s *self) | 276 | static int hw_atl_b0_hw_init_tx_path(struct aq_hw_s *self) |
| 277 | { | 277 | { |
| 278 | /* Tx TC/Queue number config */ | ||
| 279 | hw_atl_rpb_tps_tx_tc_mode_set(self, 1U); | ||
| 280 | |||
| 278 | hw_atl_thm_lso_tcp_flag_of_first_pkt_set(self, 0x0FF6U); | 281 | hw_atl_thm_lso_tcp_flag_of_first_pkt_set(self, 0x0FF6U); |
| 279 | hw_atl_thm_lso_tcp_flag_of_middle_pkt_set(self, 0x0FF6U); | 282 | hw_atl_thm_lso_tcp_flag_of_middle_pkt_set(self, 0x0FF6U); |
| 280 | hw_atl_thm_lso_tcp_flag_of_last_pkt_set(self, 0x0F7FU); | 283 | hw_atl_thm_lso_tcp_flag_of_last_pkt_set(self, 0x0F7FU); |
| @@ -1274,6 +1274,15 @@ void hw_atl_tpb_tx_buff_en_set(struct aq_hw_s *aq_hw, u32 tx_buff_en) · 1273 unmodified lines | |||
| 1274 | HW_ATL_TPB_TX_BUF_EN_SHIFT, tx_buff_en); | 1274 | HW_ATL_TPB_TX_BUF_EN_SHIFT, tx_buff_en); |
| 1275 | } | 1275 | } |
| 1276 | 1276 | ||
| 1277 | void hw_atl_rpb_tps_tx_tc_mode_set(struct aq_hw_s *aq_hw, | ||
| 1278 | u32 tx_traf_class_mode) | ||
| 1279 | { | ||
| 1280 | aq_hw_write_reg_bit(aq_hw, HW_ATL_TPB_TX_TC_MODE_ADDR, | ||
| 1281 | HW_ATL_TPB_TX_TC_MODE_MSK, | ||
| 1282 | HW_ATL_TPB_TX_TC_MODE_SHIFT, | ||
| 1283 | tx_traf_class_mode); | ||
| 1284 | } | ||
| 1285 | |||
| 1277 | void hw_atl_tpb_tx_buff_hi_threshold_per_tc_set(struct aq_hw_s *aq_hw, | 1286 | void hw_atl_tpb_tx_buff_hi_threshold_per_tc_set(struct aq_hw_s *aq_hw, |
| 1278 | u32 tx_buff_hi_threshold_per_tc, | 1287 | u32 tx_buff_hi_threshold_per_tc, |
| 1279 | u32 buffer) | 1288 | u32 buffer) |
| @@ -605,6 +605,10 @@ void hw_atl_thm_lso_tcp_flag_of_middle_pkt_set(struct aq_hw_s *aq_hw, · 604 unmodified lines | |||
| 605 | 605 | ||
| 606 | /* tpb */ | 606 | /* tpb */ |
| 607 | 607 | ||
| 608 | /* set TX Traffic Class Mode */ | ||
| 609 | void hw_atl_rpb_tps_tx_tc_mode_set(struct aq_hw_s *aq_hw, | ||
| 610 | u32 tx_traf_class_mode); | ||
| 611 | |||
| 608 | /* set tx buffer enable */ | 612 | /* set tx buffer enable */ |
| 609 | void hw_atl_tpb_tx_buff_en_set(struct aq_hw_s *aq_hw, u32 tx_buff_en); | 613 | void hw_atl_tpb_tx_buff_en_set(struct aq_hw_s *aq_hw, u32 tx_buff_en); |
| 610 | 614 | ||
| @@ -1948,6 +1948,19 @@ · 1947 unmodified lines | |||
| 1948 | /* default value of bitfield tx_buf_en */ | 1948 | /* default value of bitfield tx_buf_en */ |
| 1949 | #define HW_ATL_TPB_TX_BUF_EN_DEFAULT 0x0 | 1949 | #define HW_ATL_TPB_TX_BUF_EN_DEFAULT 0x0 |
| 1950 | 1950 | ||
| 1951 | /* register address for bitfield tx_tc_mode */ | ||
| 1952 | #define HW_ATL_TPB_TX_TC_MODE_ADDR 0x00007900 | ||
| 1953 | /* bitmask for bitfield tx_tc_mode */ | ||
| 1954 | #define HW_ATL_TPB_TX_TC_MODE_MSK 0x00000100 | ||
| 1955 | /* inverted bitmask for bitfield tx_tc_mode */ | ||
| 1956 | #define HW_ATL_TPB_TX_TC_MODE_MSKN 0xFFFFFEFF | ||
| 1957 | /* lower bit position of bitfield tx_tc_mode */ | ||
| 1958 | #define HW_ATL_TPB_TX_TC_MODE_SHIFT 8 | ||
| 1959 | /* width of bitfield tx_tc_mode */ | ||
| 1960 | #define HW_ATL_TPB_TX_TC_MODE_WIDTH 1 | ||
| 1961 | /* default value of bitfield tx_tc_mode */ | ||
| 1962 | #define HW_ATL_TPB_TX_TC_MODE_DEFAULT 0x0 | ||
| 1963 | |||
| 1951 | /* tx tx{b}_hi_thresh[c:0] bitfield definitions | 1964 | /* tx tx{b}_hi_thresh[c:0] bitfield definitions |
| 1952 | * preprocessor definitions for the bitfield "tx{b}_hi_thresh[c:0]". | 1965 | * preprocessor definitions for the bitfield "tx{b}_hi_thresh[c:0]". |
| 1953 | * parameter: buffer {b} | stride size 0x10 | range [0, 7] | 1966 | * parameter: buffer {b} | stride size 0x10 | range [0, 7] |
| @@ -1335,13 +1335,11 @@ static int atl2_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 1334 unmodified lines | |||
| 1335 | { | 1335 | { |
| 1336 | struct net_device *netdev; | 1336 | struct net_device *netdev; |
| 1337 | struct atl2_adapter *adapter; | 1337 | struct atl2_adapter *adapter; |
| 1338 | static int cards_found; | 1338 | static int cards_found = 0; |
| 1339 | unsigned long mmio_start; | 1339 | unsigned long mmio_start; |
| 1340 | int mmio_len; | 1340 | int mmio_len; |
| 1341 | int err; | 1341 | int err; |
| 1342 | 1342 | ||
| 1343 | cards_found = 0; | ||
| 1344 | |||
| 1345 | err = pci_enable_device(pdev); | 1343 | err = pci_enable_device(pdev); |
| 1346 | if (err) | 1344 | if (err) |
| 1347 | return err; | 1345 | return err; |
| @@ -134,6 +134,10 @@ static void bcm_sysport_set_rx_csum(struct net_device *dev, · 133 unmodified lines | |||
| 134 | 134 | ||
| 135 | priv->rx_chk_en = !!(wanted & NETIF_F_RXCSUM); | 135 | priv->rx_chk_en = !!(wanted & NETIF_F_RXCSUM); |
| 136 | reg = rxchk_readl(priv, RXCHK_CONTROL); | 136 | reg = rxchk_readl(priv, RXCHK_CONTROL); |
| 137 | /* Clear L2 header checks, which would prevent BPDUs | ||
| 138 | * from being received. | ||
| 139 | */ | ||
| 140 | reg &= ~RXCHK_L2_HDR_DIS; | ||
| 137 | if (priv->rx_chk_en) | 141 | if (priv->rx_chk_en) |
| 138 | reg |= RXCHK_EN; | 142 | reg |= RXCHK_EN; |
| 139 | else | 143 | else |
| @@ -500,6 +500,12 @@ normal_tx: · 499 unmodified lines | |||
| 500 | } | 500 | } |
| 501 | 501 | ||
| 502 | length >>= 9; | 502 | length >>= 9; |
| 503 | if (unlikely(length >= ARRAY_SIZE(bnxt_lhint_arr))) { | ||
| 504 | dev_warn_ratelimited(&pdev->dev, "Dropped oversize %d bytes TX packet.\n", | ||
| 505 | skb->len); | ||
| 506 | i = 0; | ||
| 507 | goto tx_dma_error; | ||
| 508 | } | ||
| 503 | flags |= bnxt_lhint_arr[length]; | 509 | flags |= bnxt_lhint_arr[length]; |
| 504 | txbd->tx_bd_len_flags_type = cpu_to_le32(flags); | 510 | txbd->tx_bd_len_flags_type = cpu_to_le32(flags); |
| 505 | 511 | ||
| @@ -3903,7 +3909,7 @@ static int bnxt_hwrm_do_send_msg(struct bnxt *bp, void *msg, u32 msg_len, · 3397 unmodified lines | |||
| 3903 | if (len) | 3909 | if (len) |
| 3904 | break; | 3910 | break; |
| 3905 | /* on first few passes, just barely sleep */ | 3911 | /* on first few passes, just barely sleep */ |
| 3906 | if (i < DFLT_HWRM_CMD_TIMEOUT) | 3912 | if (i < HWRM_SHORT_TIMEOUT_COUNTER) |
| 3907 | usleep_range(HWRM_SHORT_MIN_TIMEOUT, | 3913 | usleep_range(HWRM_SHORT_MIN_TIMEOUT, |
| 3908 | HWRM_SHORT_MAX_TIMEOUT); | 3914 | HWRM_SHORT_MAX_TIMEOUT); |
| 3909 | else | 3915 | else |
| @@ -3926,7 +3932,7 @@ static int bnxt_hwrm_do_send_msg(struct bnxt *bp, void *msg, u32 msg_len, · 16 unmodified lines | |||
| 3926 | dma_rmb(); | 3932 | dma_rmb(); |
| 3927 | if (*valid) | 3933 | if (*valid) |
| 3928 | break; | 3934 | break; |
| 3929 | udelay(1); | 3935 | usleep_range(1, 5); |
| 3930 | } | 3936 | } |
| 3931 | 3937 | ||
| 3932 | if (j >= HWRM_VALID_BIT_DELAY_USEC) { | 3938 | if (j >= HWRM_VALID_BIT_DELAY_USEC) { |
| @@ -582,7 +582,7 @@ struct nqe_cn { · 581 unmodified lines | |||
| 582 | (HWRM_SHORT_TIMEOUT_COUNTER * HWRM_SHORT_MIN_TIMEOUT + \ | 582 | (HWRM_SHORT_TIMEOUT_COUNTER * HWRM_SHORT_MIN_TIMEOUT + \ |
| 583 | ((n) - HWRM_SHORT_TIMEOUT_COUNTER) * HWRM_MIN_TIMEOUT)) | 583 | ((n) - HWRM_SHORT_TIMEOUT_COUNTER) * HWRM_MIN_TIMEOUT)) |
| 584 | 584 | ||
| 585 | #define HWRM_VALID_BIT_DELAY_USEC 20 | 585 | #define HWRM_VALID_BIT_DELAY_USEC 150 |
| 586 | 586 | ||
| 587 | #define BNXT_HWRM_CHNL_CHIMP 0 | 587 | #define BNXT_HWRM_CHNL_CHIMP 0 |
| 588 | #define BNXT_HWRM_CHNL_KONG 1 | 588 | #define BNXT_HWRM_CHNL_KONG 1 |
| @@ -271,7 +271,7 @@ struct xcast_addr_list { · 270 unmodified lines | |||
| 271 | }; | 271 | }; |
| 272 | 272 | ||
| 273 | struct nicvf_work { | 273 | struct nicvf_work { |
| 274 | struct delayed_work work; | 274 | struct work_struct work; |
| 275 | u8 mode; | 275 | u8 mode; |
| 276 | struct xcast_addr_list *mc; | 276 | struct xcast_addr_list *mc; |
| 277 | }; | 277 | }; |
| @@ -327,7 +327,11 @@ struct nicvf { · 49 unmodified lines | |||
| 327 | struct nicvf_work rx_mode_work; | 327 | struct nicvf_work rx_mode_work; |
| 328 | /* spinlock to protect workqueue arguments from concurrent access */ | 328 | /* spinlock to protect workqueue arguments from concurrent access */ |
| 329 | spinlock_t rx_mode_wq_lock; | 329 | spinlock_t rx_mode_wq_lock; |
| 330 | 330 | /* workqueue for handling kernel ndo_set_rx_mode() calls */ | |
| 331 | struct workqueue_struct *nicvf_rx_mode_wq; | ||
| 332 | /* mutex to protect VF's mailbox contents from concurrent access */ | ||
| 333 | struct mutex rx_mode_mtx; | ||
| 334 | struct delayed_work link_change_work; | ||
| 331 | /* PTP timestamp */ | 335 | /* PTP timestamp */ |
| 332 | struct cavium_ptp *ptp_clock; | 336 | struct cavium_ptp *ptp_clock; |
| 333 | /* Inbound timestamping is on */ | 337 | /* Inbound timestamping is on */ |
| @@ -575,10 +579,8 @@ struct set_ptp { · 241 unmodified lines | |||
| 575 | 579 | ||
| 576 | struct xcast { | 580 | struct xcast { |
| 577 | u8 msg; | 581 | u8 msg; |
| 578 | union { | 582 | u8 mode; |
| 579 | u8 mode; | 583 | u64 mac:48; |
| 580 | u64 mac; | ||
| 581 | } data; | ||
| 582 | }; | 584 | }; |
| 583 | 585 | ||
| 584 | /* 128 bit shared memory between PF and each VF */ | 586 | /* 128 bit shared memory between PF and each VF */ |
| @@ -57,14 +57,8 @@ struct nicpf { · 56 unmodified lines | |||
| 57 | #define NIC_GET_BGX_FROM_VF_LMAC_MAP(map) ((map >> 4) & 0xF) | 57 | #define NIC_GET_BGX_FROM_VF_LMAC_MAP(map) ((map >> 4) & 0xF) |
| 58 | #define NIC_GET_LMAC_FROM_VF_LMAC_MAP(map) (map & 0xF) | 58 | #define NIC_GET_LMAC_FROM_VF_LMAC_MAP(map) (map & 0xF) |
| 59 | u8 *vf_lmac_map; | 59 | u8 *vf_lmac_map; |
| 60 | struct delayed_work dwork; | ||
| 61 | struct workqueue_struct *check_link; | ||
| 62 | u8 *link; | ||
| 63 | u8 *duplex; | ||
| 64 | u32 *speed; | ||
| 65 | u16 cpi_base[MAX_NUM_VFS_SUPPORTED]; | 60 | u16 cpi_base[MAX_NUM_VFS_SUPPORTED]; |
| 66 | u16 rssi_base[MAX_NUM_VFS_SUPPORTED]; | 61 | u16 rssi_base[MAX_NUM_VFS_SUPPORTED]; |
| 67 | bool mbx_lock[MAX_NUM_VFS_SUPPORTED]; | ||
| 68 | 62 | ||
| 69 | /* MSI-X */ | 63 | /* MSI-X */ |
| 70 | u8 num_vec; | 64 | u8 num_vec; |
| @@ -929,6 +923,35 @@ static void nic_config_timestamp(struct nicpf *nic, int vf, struct set_ptp *ptp) · 858 unmodified lines | |||
| 929 | nic_reg_write(nic, NIC_PF_PKIND_0_15_CFG | (pkind_idx << 3), pkind_val); | 923 | nic_reg_write(nic, NIC_PF_PKIND_0_15_CFG | (pkind_idx << 3), pkind_val); |
| 930 | } | 924 | } |
| 931 | 925 | ||
| 926 | /* Get BGX LMAC link status and update corresponding VF | ||
| 927 | * if there is a change, valid only if internal L2 switch | ||
| 928 | * is not present otherwise VF link is always treated as up | ||
| 929 | */ | ||
| 930 | static void nic_link_status_get(struct nicpf *nic, u8 vf) | ||
| 931 | { | ||
| 932 | union nic_mbx mbx = {}; | ||
| 933 | struct bgx_link_status link; | ||
| 934 | u8 bgx, lmac; | ||
| 935 | |||
| 936 | mbx.link_status.msg = NIC_MBOX_MSG_BGX_LINK_CHANGE; | ||
| 937 | |||
| 938 | /* Get BGX, LMAC indices for the VF */ | ||
| 939 | bgx = NIC_GET_BGX_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); | ||
| 940 | lmac = NIC_GET_LMAC_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); | ||
| 941 | |||
| 942 | /* Get interface link status */ | ||
| 943 | bgx_get_lmac_link_state(nic->node, bgx, lmac, &link); | ||
| 944 | |||
| 945 | /* Send a mbox message to VF with current link status */ | ||
| 946 | mbx.link_status.link_up = link.link_up; | ||
| 947 | mbx.link_status.duplex = link.duplex; | ||
| 948 | mbx.link_status.speed = link.speed; | ||
| 949 | mbx.link_status.mac_type = link.mac_type; | ||
| 950 | |||
| 951 | /* reply with link status */ | ||
| 952 | nic_send_msg_to_vf(nic, vf, &mbx); | ||
| 953 | } | ||
| 954 | |||
| 932 | /* Interrupt handler to handle mailbox messages from VFs */ | 955 | /* Interrupt handler to handle mailbox messages from VFs */ |
| 933 | static void nic_handle_mbx_intr(struct nicpf *nic, int vf) | 956 | static void nic_handle_mbx_intr(struct nicpf *nic, int vf) |
| 934 | { | 957 | { |
| @@ -941,8 +964,6 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 6 unmodified lines | |||
| 941 | int i; | 964 | int i; |
| 942 | int ret = 0; | 965 | int ret = 0; |
| 943 | 966 | ||
| 944 | nic->mbx_lock[vf] = true; | ||
| 945 | |||
| 946 | mbx_addr = nic_get_mbx_addr(vf); | 967 | mbx_addr = nic_get_mbx_addr(vf); |
| 947 | mbx_data = (u64 *)&mbx; | 968 | mbx_data = (u64 *)&mbx; |
| 948 | 969 | ||
| @@ -957,12 +978,7 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 8 unmodified lines | |||
| 957 | switch (mbx.msg.msg) { | 978 | switch (mbx.msg.msg) { |
| 958 | case NIC_MBOX_MSG_READY: | 979 | case NIC_MBOX_MSG_READY: |
| 959 | nic_mbx_send_ready(nic, vf); | 980 | nic_mbx_send_ready(nic, vf); |
| 960 | if (vf < nic->num_vf_en) { | 981 | return; |
| 961 | nic->link[vf] = 0; | ||
| 962 | nic->duplex[vf] = 0; | ||
| 963 | nic->speed[vf] = 0; | ||
| 964 | } | ||
| 965 | goto unlock; | ||
| 966 | case NIC_MBOX_MSG_QS_CFG: | 982 | case NIC_MBOX_MSG_QS_CFG: |
| 967 | reg_addr = NIC_PF_QSET_0_127_CFG | | 983 | reg_addr = NIC_PF_QSET_0_127_CFG | |
| 968 | (mbx.qs.num << NIC_QS_ID_SHIFT); | 984 | (mbx.qs.num << NIC_QS_ID_SHIFT); |
| @@ -1031,7 +1047,7 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 62 unmodified lines | |||
| 1031 | break; | 1047 | break; |
| 1032 | case NIC_MBOX_MSG_RSS_SIZE: | 1048 | case NIC_MBOX_MSG_RSS_SIZE: |
| 1033 | nic_send_rss_size(nic, vf); | 1049 | nic_send_rss_size(nic, vf); |
| 1034 | goto unlock; | 1050 | return; |
| 1035 | case NIC_MBOX_MSG_RSS_CFG: | 1051 | case NIC_MBOX_MSG_RSS_CFG: |
| 1036 | case NIC_MBOX_MSG_RSS_CFG_CONT: | 1052 | case NIC_MBOX_MSG_RSS_CFG_CONT: |
| 1037 | nic_config_rss(nic, &mbx.rss_cfg); | 1053 | nic_config_rss(nic, &mbx.rss_cfg); |
| @@ -1039,7 +1055,7 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 1 unmodified line | |||
| 1039 | case NIC_MBOX_MSG_CFG_DONE: | 1055 | case NIC_MBOX_MSG_CFG_DONE: |
| 1040 | /* Last message of VF config msg sequence */ | 1056 | /* Last message of VF config msg sequence */ |
| 1041 | nic_enable_vf(nic, vf, true); | 1057 | nic_enable_vf(nic, vf, true); |
| 1042 | goto unlock; | 1058 | break; |
| 1043 | case NIC_MBOX_MSG_SHUTDOWN: | 1059 | case NIC_MBOX_MSG_SHUTDOWN: |
| 1044 | /* First msg in VF teardown sequence */ | 1060 | /* First msg in VF teardown sequence */ |
| 1045 | if (vf >= nic->num_vf_en) | 1061 | if (vf >= nic->num_vf_en) |
| @@ -1049,19 +1065,19 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 3 unmodified lines | |||
| 1049 | break; | 1065 | break; |
| 1050 | case NIC_MBOX_MSG_ALLOC_SQS: | 1066 | case NIC_MBOX_MSG_ALLOC_SQS: |
| 1051 | nic_alloc_sqs(nic, &mbx.sqs_alloc); | 1067 | nic_alloc_sqs(nic, &mbx.sqs_alloc); |
| 1052 | goto unlock; | 1068 | return; |
| 1053 | case NIC_MBOX_MSG_NICVF_PTR: | 1069 | case NIC_MBOX_MSG_NICVF_PTR: |
| 1054 | nic->nicvf[vf] = mbx.nicvf.nicvf; | 1070 | nic->nicvf[vf] = mbx.nicvf.nicvf; |
| 1055 | break; | 1071 | break; |
| 1056 | case NIC_MBOX_MSG_PNICVF_PTR: | 1072 | case NIC_MBOX_MSG_PNICVF_PTR: |
| 1057 | nic_send_pnicvf(nic, vf); | 1073 | nic_send_pnicvf(nic, vf); |
| 1058 | goto unlock; | 1074 | return; |
| 1059 | case NIC_MBOX_MSG_SNICVF_PTR: | 1075 | case NIC_MBOX_MSG_SNICVF_PTR: |
| 1060 | nic_send_snicvf(nic, &mbx.nicvf); | 1076 | nic_send_snicvf(nic, &mbx.nicvf); |
| 1061 | goto unlock; | 1077 | return; |
| 1062 | case NIC_MBOX_MSG_BGX_STATS: | 1078 | case NIC_MBOX_MSG_BGX_STATS: |
| 1063 | nic_get_bgx_stats(nic, &mbx.bgx_stats); | 1079 | nic_get_bgx_stats(nic, &mbx.bgx_stats); |
| 1064 | goto unlock; | 1080 | return; |
| 1065 | case NIC_MBOX_MSG_LOOPBACK: | 1081 | case NIC_MBOX_MSG_LOOPBACK: |
| 1066 | ret = nic_config_loopback(nic, &mbx.lbk); | 1082 | ret = nic_config_loopback(nic, &mbx.lbk); |
| 1067 | break; | 1083 | break; |
| @@ -1070,7 +1086,7 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 2 unmodified lines | |||
| 1070 | break; | 1086 | break; |
| 1071 | case NIC_MBOX_MSG_PFC: | 1087 | case NIC_MBOX_MSG_PFC: |
| 1072 | nic_pause_frame(nic, vf, &mbx.pfc); | 1088 | nic_pause_frame(nic, vf, &mbx.pfc); |
| 1073 | goto unlock; | 1089 | return; |
| 1074 | case NIC_MBOX_MSG_PTP_CFG: | 1090 | case NIC_MBOX_MSG_PTP_CFG: |
| 1075 | nic_config_timestamp(nic, vf, &mbx.ptp); | 1091 | nic_config_timestamp(nic, vf, &mbx.ptp); |
| 1076 | break; | 1092 | break; |
| @@ -1094,7 +1110,7 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 17 unmodified lines | |||
| 1094 | bgx = NIC_GET_BGX_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); | 1110 | bgx = NIC_GET_BGX_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); |
| 1095 | lmac = NIC_GET_LMAC_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); | 1111 | lmac = NIC_GET_LMAC_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); |
| 1096 | bgx_set_dmac_cam_filter(nic->node, bgx, lmac, | 1112 | bgx_set_dmac_cam_filter(nic->node, bgx, lmac, |
| 1097 | mbx.xcast.data.mac, | 1113 | mbx.xcast.mac, |
| 1098 | vf < NIC_VF_PER_MBX_REG ? vf : | 1114 | vf < NIC_VF_PER_MBX_REG ? vf : |
| 1099 | vf - NIC_VF_PER_MBX_REG); | 1115 | vf - NIC_VF_PER_MBX_REG); |
| 1100 | break; | 1116 | break; |
| @@ -1106,8 +1122,15 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 5 unmodified lines | |||
| 1106 | } | 1122 | } |
| 1107 | bgx = NIC_GET_BGX_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); | 1123 | bgx = NIC_GET_BGX_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); |
| 1108 | lmac = NIC_GET_LMAC_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); | 1124 | lmac = NIC_GET_LMAC_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); |
| 1109 | bgx_set_xcast_mode(nic->node, bgx, lmac, mbx.xcast.data.mode); | 1125 | bgx_set_xcast_mode(nic->node, bgx, lmac, mbx.xcast.mode); |
| 1110 | break; | 1126 | break; |
| 1127 | case NIC_MBOX_MSG_BGX_LINK_CHANGE: | ||
| 1128 | if (vf >= nic->num_vf_en) { | ||
| 1129 | ret = -1; /* NACK */ | ||
| 1130 | break; | ||
| 1131 | } | ||
| 1132 | nic_link_status_get(nic, vf); | ||
| 1133 | return; | ||
| 1111 | default: | 1134 | default: |
| 1112 | dev_err(&nic->pdev->dev, | 1135 | dev_err(&nic->pdev->dev, |
| 1113 | "Invalid msg from VF%d, msg 0x%x\n", vf, mbx.msg.msg); | 1136 | "Invalid msg from VF%d, msg 0x%x\n", vf, mbx.msg.msg); |
| @@ -1121,8 +1144,6 @@ static void nic_handle_mbx_intr(struct nicpf *nic, int vf) · 7 unmodified lines | |||
| 1121 | mbx.msg.msg, vf); | 1144 | mbx.msg.msg, vf); |
| 1122 | nic_mbx_send_nack(nic, vf); | 1145 | nic_mbx_send_nack(nic, vf); |
| 1123 | } | 1146 | } |
| 1124 | unlock: | ||
| 1125 | nic->mbx_lock[vf] = false; | ||
| 1126 | } | 1147 | } |
| 1127 | 1148 | ||
| 1128 | static irqreturn_t nic_mbx_intr_handler(int irq, void *nic_irq) | 1149 | static irqreturn_t nic_mbx_intr_handler(int irq, void *nic_irq) |
| @@ -1270,52 +1291,6 @@ static int nic_sriov_init(struct pci_dev *pdev, struct nicpf *nic) · 141 unmodified lines | |||
| 1270 | return 0; | 1291 | return 0; |
| 1271 | } | 1292 | } |
| 1272 | 1293 | ||
| 1273 | /* Poll for BGX LMAC link status and update corresponding VF | ||
| 1274 | * if there is a change, valid only if internal L2 switch | ||
| 1275 | * is not present otherwise VF link is always treated as up | ||
| 1276 | */ | ||
| 1277 | static void nic_poll_for_link(struct work_struct *work) | ||
| 1278 | { | ||
| 1279 | union nic_mbx mbx = {}; | ||
| 1280 | struct nicpf *nic; | ||
| 1281 | struct bgx_link_status link; | ||
| 1282 | u8 vf, bgx, lmac; | ||
| 1283 | |||
| 1284 | nic = container_of(work, struct nicpf, dwork.work); | ||
| 1285 | |||
| 1286 | mbx.link_status.msg = NIC_MBOX_MSG_BGX_LINK_CHANGE; | ||
| 1287 | |||
| 1288 | for (vf = 0; vf < nic->num_vf_en; vf++) { | ||
| 1289 | /* Poll only if VF is UP */ | ||
| 1290 | if (!nic->vf_enabled[vf]) | ||
| 1291 | continue; | ||
| 1292 | |||
| 1293 | /* Get BGX, LMAC indices for the VF */ | ||
| 1294 | bgx = NIC_GET_BGX_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); | ||
| 1295 | lmac = NIC_GET_LMAC_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); | ||
| 1296 | /* Get interface link status */ | ||
| 1297 | bgx_get_lmac_link_state(nic->node, bgx, lmac, &link); | ||
| 1298 | |||
| 1299 | /* Inform VF only if link status changed */ | ||
| 1300 | if (nic->link[vf] == link.link_up) | ||
| 1301 | continue; | ||
| 1302 | |||
| 1303 | if (!nic->mbx_lock[vf]) { | ||
| 1304 | nic->link[vf] = link.link_up; | ||
| 1305 | nic->duplex[vf] = link.duplex; | ||
| 1306 | nic->speed[vf] = link.speed; | ||
| 1307 | |||
| 1308 | /* Send a mbox message to VF with current link status */ | ||
| 1309 | mbx.link_status.link_up = link.link_up; | ||
| 1310 | mbx.link_status.duplex = link.duplex; | ||
| 1311 | mbx.link_status.speed = link.speed; | ||
| 1312 | mbx.link_status.mac_type = link.mac_type; | ||
| 1313 | nic_send_msg_to_vf(nic, vf, &mbx); | ||
| 1314 | } | ||
| 1315 | } | ||
| 1316 | queue_delayed_work(nic->check_link, &nic->dwork, HZ * 2); | ||
| 1317 | } | ||
| 1318 | |||
| 1319 | static int nic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) | 1294 | static int nic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) |
| 1320 | { | 1295 | { |
| 1321 | struct device *dev = &pdev->dev; | 1296 | struct device *dev = &pdev->dev; |
| @@ -1384,18 +1359,6 @@ static int nic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 62 unmodified lines | |||
| 1384 | if (!nic->vf_lmac_map) | 1359 | if (!nic->vf_lmac_map) |
| 1385 | goto err_release_regions; | 1360 | goto err_release_regions; |
| 1386 | 1361 | ||
| 1387 | nic->link = devm_kmalloc_array(dev, max_lmac, sizeof(u8), GFP_KERNEL); | ||
| 1388 | if (!nic->link) | ||
| 1389 | goto err_release_regions; | ||
| 1390 | |||
| 1391 | nic->duplex = devm_kmalloc_array(dev, max_lmac, sizeof(u8), GFP_KERNEL); | ||
| 1392 | if (!nic->duplex) | ||
| 1393 | goto err_release_regions; | ||
| 1394 | |||
| 1395 | nic->speed = devm_kmalloc_array(dev, max_lmac, sizeof(u32), GFP_KERNEL); | ||
| 1396 | if (!nic->speed) | ||
| 1397 | goto err_release_regions; | ||
| 1398 | |||
| 1399 | /* Initialize hardware */ | 1362 | /* Initialize hardware */ |
| 1400 | nic_init_hw(nic); | 1363 | nic_init_hw(nic); |
| 1401 | 1364 | ||
| @@ -1411,22 +1374,8 @@ static int nic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 9 unmodified lines | |||
| 1411 | if (err) | 1374 | if (err) |
| 1412 | goto err_unregister_interrupts; | 1375 | goto err_unregister_interrupts; |
| 1413 | 1376 | ||
| 1414 | /* Register a physical link status poll fn() */ | ||
| 1415 | nic->check_link = alloc_workqueue("check_link_status", | ||
| 1416 | WQ_UNBOUND | WQ_MEM_RECLAIM, 1); | ||
| 1417 | if (!nic->check_link) { | ||
| 1418 | err = -ENOMEM; | ||
| 1419 | goto err_disable_sriov; | ||
| 1420 | } | ||
| 1421 | |||
| 1422 | INIT_DELAYED_WORK(&nic->dwork, nic_poll_for_link); | ||
| 1423 | queue_delayed_work(nic->check_link, &nic->dwork, 0); | ||
| 1424 | |||
| 1425 | return 0; | 1377 | return 0; |
| 1426 | 1378 | ||
| 1427 | err_disable_sriov: | ||
| 1428 | if (nic->flags & NIC_SRIOV_ENABLED) | ||
| 1429 | pci_disable_sriov(pdev); | ||
| 1430 | err_unregister_interrupts: | 1379 | err_unregister_interrupts: |
| 1431 | nic_unregister_interrupts(nic); | 1380 | nic_unregister_interrupts(nic); |
| 1432 | err_release_regions: | 1381 | err_release_regions: |
| @@ -1447,12 +1396,6 @@ static void nic_remove(struct pci_dev *pdev) · 14 unmodified lines | |||
| 1447 | if (nic->flags & NIC_SRIOV_ENABLED) | 1396 | if (nic->flags & NIC_SRIOV_ENABLED) |
| 1448 | pci_disable_sriov(pdev); | 1397 | pci_disable_sriov(pdev); |
| 1449 | 1398 | ||
| 1450 | if (nic->check_link) { | ||
| 1451 | /* Destroy work Queue */ | ||
| 1452 | cancel_delayed_work_sync(&nic->dwork); | ||
| 1453 | destroy_workqueue(nic->check_link); | ||
| 1454 | } | ||
| 1455 | |||
| 1456 | nic_unregister_interrupts(nic); | 1399 | nic_unregister_interrupts(nic); |
| 1457 | pci_release_regions(pdev); | 1400 | pci_release_regions(pdev); |
| 1458 | 1401 | ||
| @@ -68,9 +68,6 @@ module_param(cpi_alg, int, 0444); · 67 unmodified lines | |||
| 68 | MODULE_PARM_DESC(cpi_alg, | 68 | MODULE_PARM_DESC(cpi_alg, |
| 69 | "PFC algorithm (0=none, 1=VLAN, 2=VLAN16, 3=IP Diffserv)"); | 69 | "PFC algorithm (0=none, 1=VLAN, 2=VLAN16, 3=IP Diffserv)"); |
| 70 | 70 | ||
| 71 | /* workqueue for handling kernel ndo_set_rx_mode() calls */ | ||
| 72 | static struct workqueue_struct *nicvf_rx_mode_wq; | ||
| 73 | |||
| 74 | static inline u8 nicvf_netdev_qidx(struct nicvf *nic, u8 qidx) | 71 | static inline u8 nicvf_netdev_qidx(struct nicvf *nic, u8 qidx) |
| 75 | { | 72 | { |
| 76 | if (nic->sqs_mode) | 73 | if (nic->sqs_mode) |
| @@ -127,6 +124,9 @@ int nicvf_send_msg_to_pf(struct nicvf *nic, union nic_mbx *mbx) · 50 unmodified lines | |||
| 127 | { | 124 | { |
| 128 | int timeout = NIC_MBOX_MSG_TIMEOUT; | 125 | int timeout = NIC_MBOX_MSG_TIMEOUT; |
| 129 | int sleep = 10; | 126 | int sleep = 10; |
| 127 | int ret = 0; | ||
| 128 | |||
| 129 | mutex_lock(&nic->rx_mode_mtx); | ||
| 130 | 130 | ||
| 131 | nic->pf_acked = false; | 131 | nic->pf_acked = false; |
| 132 | nic->pf_nacked = false; | 132 | nic->pf_nacked = false; |
| @@ -139,7 +139,8 @@ int nicvf_send_msg_to_pf(struct nicvf *nic, union nic_mbx *mbx) · 6 unmodified lines | |||
| 139 | netdev_err(nic->netdev, | 139 | netdev_err(nic->netdev, |
| 140 | "PF NACK to mbox msg 0x%02x from VF%d\n", | 140 | "PF NACK to mbox msg 0x%02x from VF%d\n", |
| 141 | (mbx->msg.msg & 0xFF), nic->vf_id); | 141 | (mbx->msg.msg & 0xFF), nic->vf_id); |
| 142 | return -EINVAL; | 142 | ret = -EINVAL; |
| 143 | break; | ||
| 143 | } | 144 | } |
| 144 | msleep(sleep); | 145 | msleep(sleep); |
| 145 | if (nic->pf_acked) | 146 | if (nic->pf_acked) |
| @@ -149,10 +150,12 @@ int nicvf_send_msg_to_pf(struct nicvf *nic, union nic_mbx *mbx) · 3 unmodified lines | |||
| 149 | netdev_err(nic->netdev, | 150 | netdev_err(nic->netdev, |
| 150 | "PF didn't ACK to mbox msg 0x%02x from VF%d\n", | 151 | "PF didn't ACK to mbox msg 0x%02x from VF%d\n", |
| 151 | (mbx->msg.msg & 0xFF), nic->vf_id); | 152 | (mbx->msg.msg & 0xFF), nic->vf_id); |
| 152 | return -EBUSY; | 153 | ret = -EBUSY; |
| 154 | break; | ||
| 153 | } | 155 | } |
| 154 | } | 156 | } |
| 155 | return 0; | 157 | mutex_unlock(&nic->rx_mode_mtx); |
| 158 | return ret; | ||
| 156 | } | 159 | } |
| 157 | 160 | ||
| 158 | /* Checks if VF is able to comminicate with PF | 161 | /* Checks if VF is able to comminicate with PF |
| @@ -172,6 +175,17 @@ static int nicvf_check_pf_ready(struct nicvf *nic) · 13 unmodified lines | |||
| 172 | return 1; | 175 | return 1; |
| 173 | } | 176 | } |
| 174 | 177 | ||
| 178 | static void nicvf_send_cfg_done(struct nicvf *nic) | ||
| 179 | { | ||
| 180 | union nic_mbx mbx = {}; | ||
| 181 | |||
| 182 | mbx.msg.msg = NIC_MBOX_MSG_CFG_DONE; | ||
| 183 | if (nicvf_send_msg_to_pf(nic, &mbx)) { | ||
| 184 | netdev_err(nic->netdev, | ||
| 185 | "PF didn't respond to CFG DONE msg\n"); | ||
| 186 | } | ||
| 187 | } | ||
| 188 | |||
| 175 | static void nicvf_read_bgx_stats(struct nicvf *nic, struct bgx_stats_msg *bgx) | 189 | static void nicvf_read_bgx_stats(struct nicvf *nic, struct bgx_stats_msg *bgx) |
| 176 | { | 190 | { |
| 177 | if (bgx->rx) | 191 | if (bgx->rx) |
| @@ -228,21 +242,24 @@ static void nicvf_handle_mbx_intr(struct nicvf *nic) · 50 unmodified lines | |||
| 228 | break; | 242 | break; |
| 229 | case NIC_MBOX_MSG_BGX_LINK_CHANGE: | 243 | case NIC_MBOX_MSG_BGX_LINK_CHANGE: |
| 230 | nic->pf_acked = true; | 244 | nic->pf_acked = true; |
| 231 | nic->link_up = mbx.link_status.link_up; | 245 | if (nic->link_up != mbx.link_status.link_up) { |
| 232 | nic->duplex = mbx.link_status.duplex; | 246 | nic->link_up = mbx.link_status.link_up; |
| 233 | nic->speed = mbx.link_status.speed; | 247 | nic->duplex = mbx.link_status.duplex; |
| 234 | nic->mac_type = mbx.link_status.mac_type; | 248 | nic->speed = mbx.link_status.speed; |
| 235 | if (nic->link_up) { | 249 | nic->mac_type = mbx.link_status.mac_type; |
| 236 | netdev_info(nic->netdev, "Link is Up %d Mbps %s duplex\n", | 250 | if (nic->link_up) { |
| 237 | nic->speed, | 251 | netdev_info(nic->netdev, |
| 238 | nic->duplex == DUPLEX_FULL ? | 252 | "Link is Up %d Mbps %s duplex\n", |
| 239 | "Full" : "Half"); | 253 | nic->speed, |
| 240 | netif_carrier_on(nic->netdev); | 254 | nic->duplex == DUPLEX_FULL ? |
| 241 | netif_tx_start_all_queues(nic->netdev); | 255 | "Full" : "Half"); |
| 242 | } else { | 256 | netif_carrier_on(nic->netdev); |
| 243 | netdev_info(nic->netdev, "Link is Down\n"); | 257 | netif_tx_start_all_queues(nic->netdev); |
| 244 | netif_carrier_off(nic->netdev); | 258 | } else { |
| 245 | netif_tx_stop_all_queues(nic->netdev); | 259 | netdev_info(nic->netdev, "Link is Down\n"); |
| 260 | netif_carrier_off(nic->netdev); | ||
| 261 | netif_tx_stop_all_queues(nic->netdev); | ||
| 262 | } | ||
| 246 | } | 263 | } |
| 247 | break; | 264 | break; |
| 248 | case NIC_MBOX_MSG_ALLOC_SQS: | 265 | case NIC_MBOX_MSG_ALLOC_SQS: |
| @@ -1311,6 +1328,11 @@ int nicvf_stop(struct net_device *netdev) · 1062 unmodified lines | |||
| 1311 | struct nicvf_cq_poll *cq_poll = NULL; | 1328 | struct nicvf_cq_poll *cq_poll = NULL; |
| 1312 | union nic_mbx mbx = {}; | 1329 | union nic_mbx mbx = {}; |
| 1313 | 1330 | ||
| 1331 | cancel_delayed_work_sync(&nic->link_change_work); | ||
| 1332 | |||
| 1333 | /* wait till all queued set_rx_mode tasks completes */ | ||
| 1334 | drain_workqueue(nic->nicvf_rx_mode_wq); | ||
| 1335 | |||
| 1314 | mbx.msg.msg = NIC_MBOX_MSG_SHUTDOWN; | 1336 | mbx.msg.msg = NIC_MBOX_MSG_SHUTDOWN; |
| 1315 | nicvf_send_msg_to_pf(nic, &mbx); | 1337 | nicvf_send_msg_to_pf(nic, &mbx); |
| 1316 | 1338 | ||
| @@ -1410,13 +1432,27 @@ static int nicvf_update_hw_max_frs(struct nicvf *nic, int mtu) · 93 unmodified lines | |||
| 1410 | return nicvf_send_msg_to_pf(nic, &mbx); | 1432 | return nicvf_send_msg_to_pf(nic, &mbx); |
| 1411 | } | 1433 | } |
| 1412 | 1434 | ||
| 1435 | static void nicvf_link_status_check_task(struct work_struct *work_arg) | ||
| 1436 | { | ||
| 1437 | struct nicvf *nic = container_of(work_arg, | ||
| 1438 | struct nicvf, | ||
| 1439 | link_change_work.work); | ||
| 1440 | union nic_mbx mbx = {}; | ||
| 1441 | mbx.msg.msg = NIC_MBOX_MSG_BGX_LINK_CHANGE; | ||
| 1442 | nicvf_send_msg_to_pf(nic, &mbx); | ||
| 1443 | queue_delayed_work(nic->nicvf_rx_mode_wq, | ||
| 1444 | &nic->link_change_work, 2 * HZ); | ||
| 1445 | } | ||
| 1446 | |||
| 1413 | int nicvf_open(struct net_device *netdev) | 1447 | int nicvf_open(struct net_device *netdev) |
| 1414 | { | 1448 | { |
| 1415 | int cpu, err, qidx; | 1449 | int cpu, err, qidx; |
| 1416 | struct nicvf *nic = netdev_priv(netdev); | 1450 | struct nicvf *nic = netdev_priv(netdev); |
| 1417 | struct queue_set *qs = nic->qs; | 1451 | struct queue_set *qs = nic->qs; |
| 1418 | struct nicvf_cq_poll *cq_poll = NULL; | 1452 | struct nicvf_cq_poll *cq_poll = NULL; |
| 1419 | union nic_mbx mbx = {}; | 1453 | |
| 1454 | /* wait till all queued set_rx_mode tasks completes if any */ | ||
| 1455 | drain_workqueue(nic->nicvf_rx_mode_wq); | ||
| 1420 | 1456 | ||
| 1421 | netif_carrier_off(netdev); | 1457 | netif_carrier_off(netdev); |
| 1422 | 1458 | ||
| @@ -1512,8 +1548,12 @@ int nicvf_open(struct net_device *netdev) · 89 unmodified lines | |||
| 1512 | nicvf_enable_intr(nic, NICVF_INTR_RBDR, qidx); | 1548 | nicvf_enable_intr(nic, NICVF_INTR_RBDR, qidx); |
| 1513 | 1549 | ||
| 1514 | /* Send VF config done msg to PF */ | 1550 | /* Send VF config done msg to PF */ |
| 1515 | mbx.msg.msg = NIC_MBOX_MSG_CFG_DONE; | 1551 | nicvf_send_cfg_done(nic); |
| 1516 | nicvf_write_to_mbx(nic, &mbx); | 1552 | |
| 1553 | INIT_DELAYED_WORK(&nic->link_change_work, | ||
| 1554 | nicvf_link_status_check_task); | ||
| 1555 | queue_delayed_work(nic->nicvf_rx_mode_wq, | ||
| 1556 | &nic->link_change_work, 0); | ||
| 1517 | 1557 | ||
| 1518 | return 0; | 1558 | return 0; |
| 1519 | cleanup: | 1559 | cleanup: |
| @@ -1941,15 +1981,17 @@ static void __nicvf_set_rx_mode_task(u8 mode, struct xcast_addr_list *mc_addrs, · 421 unmodified lines | |||
| 1941 | 1981 | ||
| 1942 | /* flush DMAC filters and reset RX mode */ | 1982 | /* flush DMAC filters and reset RX mode */ |
| 1943 | mbx.xcast.msg = NIC_MBOX_MSG_RESET_XCAST; | 1983 | mbx.xcast.msg = NIC_MBOX_MSG_RESET_XCAST; |
| 1944 | nicvf_send_msg_to_pf(nic, &mbx); | 1984 | if (nicvf_send_msg_to_pf(nic, &mbx) < 0) |
| 1985 | goto free_mc; | ||
| 1945 | 1986 | ||
| 1946 | if (mode & BGX_XCAST_MCAST_FILTER) { | 1987 | if (mode & BGX_XCAST_MCAST_FILTER) { |
| 1947 | /* once enabling filtering, we need to signal to PF to add | 1988 | /* once enabling filtering, we need to signal to PF to add |
| 1948 | * its' own LMAC to the filter to accept packets for it. | 1989 | * its' own LMAC to the filter to accept packets for it. |
| 1949 | */ | 1990 | */ |
| 1950 | mbx.xcast.msg = NIC_MBOX_MSG_ADD_MCAST; | 1991 | mbx.xcast.msg = NIC_MBOX_MSG_ADD_MCAST; |
| 1951 | mbx.xcast.data.mac = 0; | 1992 | mbx.xcast.mac = 0; |
| 1952 | nicvf_send_msg_to_pf(nic, &mbx); | 1993 | if (nicvf_send_msg_to_pf(nic, &mbx) < 0) |
| 1994 | goto free_mc; | ||
| 1953 | } | 1995 | } |
| 1954 | 1996 | ||
| 1955 | /* check if we have any specific MACs to be added to PF DMAC filter */ | 1997 | /* check if we have any specific MACs to be added to PF DMAC filter */ |
| @@ -1957,23 +1999,25 @@ static void __nicvf_set_rx_mode_task(u8 mode, struct xcast_addr_list *mc_addrs, · 1 unmodified line | |||
| 1957 | /* now go through kernel list of MACs and add them one by one */ | 1999 | /* now go through kernel list of MACs and add them one by one */ |
| 1958 | for (idx = 0; idx < mc_addrs->count; idx++) { | 2000 | for (idx = 0; idx < mc_addrs->count; idx++) { |
| 1959 | mbx.xcast.msg = NIC_MBOX_MSG_ADD_MCAST; | 2001 | mbx.xcast.msg = NIC_MBOX_MSG_ADD_MCAST; |
| 1960 | mbx.xcast.data.mac = mc_addrs->mc[idx]; | 2002 | mbx.xcast.mac = mc_addrs->mc[idx]; |
| 1961 | nicvf_send_msg_to_pf(nic, &mbx); | 2003 | if (nicvf_send_msg_to_pf(nic, &mbx) < 0) |
| 2004 | goto free_mc; | ||
| 1962 | } | 2005 | } |
| 1963 | kfree(mc_addrs); | ||
| 1964 | } | 2006 | } |
| 1965 | 2007 | ||
| 1966 | /* and finally set rx mode for PF accordingly */ | 2008 | /* and finally set rx mode for PF accordingly */ |
| 1967 | mbx.xcast.msg = NIC_MBOX_MSG_SET_XCAST; | 2009 | mbx.xcast.msg = NIC_MBOX_MSG_SET_XCAST; |
| 1968 | mbx.xcast.data.mode = mode; | 2010 | mbx.xcast.mode = mode; |
| 1969 | 2011 | ||
| 1970 | nicvf_send_msg_to_pf(nic, &mbx); | 2012 | nicvf_send_msg_to_pf(nic, &mbx); |
| 2013 | free_mc: | ||
| 2014 | kfree(mc_addrs); | ||
| 1971 | } | 2015 | } |
| 1972 | 2016 | ||
| 1973 | static void nicvf_set_rx_mode_task(struct work_struct *work_arg) | 2017 | static void nicvf_set_rx_mode_task(struct work_struct *work_arg) |
| 1974 | { | 2018 | { |
| 1975 | struct nicvf_work *vf_work = container_of(work_arg, struct nicvf_work, | 2019 | struct nicvf_work *vf_work = container_of(work_arg, struct nicvf_work, |
| 1976 | work.work); | 2020 | work); |
| 1977 | struct nicvf *nic = container_of(vf_work, struct nicvf, rx_mode_work); | 2021 | struct nicvf *nic = container_of(vf_work, struct nicvf, rx_mode_work); |
| 1978 | u8 mode; | 2022 | u8 mode; |
| 1979 | struct xcast_addr_list *mc; | 2023 | struct xcast_addr_list *mc; |
| @@ -2030,7 +2074,7 @@ static void nicvf_set_rx_mode(struct net_device *netdev) · 50 unmodified lines | |||
| 2030 | kfree(nic->rx_mode_work.mc); | 2074 | kfree(nic->rx_mode_work.mc); |
| 2031 | nic->rx_mode_work.mc = mc_list; | 2075 | nic->rx_mode_work.mc = mc_list; |
| 2032 | nic->rx_mode_work.mode = mode; | 2076 | nic->rx_mode_work.mode = mode; |
| 2033 | queue_delayed_work(nicvf_rx_mode_wq, &nic->rx_mode_work.work, 0); | 2077 | queue_work(nic->nicvf_rx_mode_wq, &nic->rx_mode_work.work); |
| 2034 | spin_unlock(&nic->rx_mode_wq_lock); | 2078 | spin_unlock(&nic->rx_mode_wq_lock); |
| 2035 | } | 2079 | } |
| 2036 | 2080 | ||
| @@ -2187,8 +2231,12 @@ static int nicvf_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 150 unmodified lines | |||
| 2187 | 2231 | ||
| 2188 | INIT_WORK(&nic->reset_task, nicvf_reset_task); | 2232 | INIT_WORK(&nic->reset_task, nicvf_reset_task); |
| 2189 | 2233 | ||
| 2190 | INIT_DELAYED_WORK(&nic->rx_mode_work.work, nicvf_set_rx_mode_task); | 2234 | nic->nicvf_rx_mode_wq = alloc_ordered_workqueue("nicvf_rx_mode_wq_VF%d", |
| 2235 | WQ_MEM_RECLAIM, | ||
| 2236 | nic->vf_id); | ||
| 2237 | INIT_WORK(&nic->rx_mode_work.work, nicvf_set_rx_mode_task); | ||
| 2191 | spin_lock_init(&nic->rx_mode_wq_lock); | 2238 | spin_lock_init(&nic->rx_mode_wq_lock); |
| 2239 | mutex_init(&nic->rx_mode_mtx); | ||
| 2192 | 2240 | ||
| 2193 | err = register_netdev(netdev); | 2241 | err = register_netdev(netdev); |
| 2194 | if (err) { | 2242 | if (err) { |
| @@ -2228,13 +2276,15 @@ static void nicvf_remove(struct pci_dev *pdev) · 33 unmodified lines | |||
| 2228 | nic = netdev_priv(netdev); | 2276 | nic = netdev_priv(netdev); |
| 2229 | pnetdev = nic->pnicvf->netdev; | 2277 | pnetdev = nic->pnicvf->netdev; |
| 2230 | 2278 | ||
| 2231 | cancel_delayed_work_sync(&nic->rx_mode_work.work); | ||
| 2232 | |||
| 2233 | /* Check if this Qset is assigned to different VF. | 2279 | /* Check if this Qset is assigned to different VF. |
| 2234 | * If yes, clean primary and all secondary Qsets. | 2280 | * If yes, clean primary and all secondary Qsets. |
| 2235 | */ | 2281 | */ |
| 2236 | if (pnetdev && (pnetdev->reg_state == NETREG_REGISTERED)) | 2282 | if (pnetdev && (pnetdev->reg_state == NETREG_REGISTERED)) |
| 2237 | unregister_netdev(pnetdev); | 2283 | unregister_netdev(pnetdev); |
| 2284 | if (nic->nicvf_rx_mode_wq) { | ||
| 2285 | destroy_workqueue(nic->nicvf_rx_mode_wq); | ||
| 2286 | nic->nicvf_rx_mode_wq = NULL; | ||
| 2287 | } | ||
| 2238 | nicvf_unregister_interrupts(nic); | 2288 | nicvf_unregister_interrupts(nic); |
| 2239 | pci_set_drvdata(pdev, NULL); | 2289 | pci_set_drvdata(pdev, NULL); |
| 2240 | if (nic->drv_stats) | 2290 | if (nic->drv_stats) |
| @@ -2261,17 +2311,11 @@ static struct pci_driver nicvf_driver = { · 20 unmodified lines | |||
| 2261 | static int __init nicvf_init_module(void) | 2311 | static int __init nicvf_init_module(void) |
| 2262 | { | 2312 | { |
| 2263 | pr_info("%s, ver %s\n", DRV_NAME, DRV_VERSION); | 2313 | pr_info("%s, ver %s\n", DRV_NAME, DRV_VERSION); |
| 2264 | nicvf_rx_mode_wq = alloc_ordered_workqueue("nicvf_generic", | ||
| 2265 | WQ_MEM_RECLAIM); | ||
| 2266 | return pci_register_driver(&nicvf_driver); | 2314 | return pci_register_driver(&nicvf_driver); |
| 2267 | } | 2315 | } |
| 2268 | 2316 | ||
| 2269 | static void __exit nicvf_cleanup_module(void) | 2317 | static void __exit nicvf_cleanup_module(void) |
| 2270 | { | 2318 | { |
| 2271 | if (nicvf_rx_mode_wq) { | ||
| 2272 | destroy_workqueue(nicvf_rx_mode_wq); | ||
| 2273 | nicvf_rx_mode_wq = NULL; | ||
| 2274 | } | ||
| 2275 | pci_unregister_driver(&nicvf_driver); | 2319 | pci_unregister_driver(&nicvf_driver); |
| 2276 | } | 2320 | } |
| 2277 | 2321 | ||
| @@ -1217,7 +1217,7 @@ static void bgx_init_hw(struct bgx *bgx) · 1216 unmodified lines | |||
| 1217 | 1217 | ||
| 1218 | /* Disable MAC steering (NCSI traffic) */ | 1218 | /* Disable MAC steering (NCSI traffic) */ |
| 1219 | for (i = 0; i < RX_TRAFFIC_STEER_RULE_COUNT; i++) | 1219 | for (i = 0; i < RX_TRAFFIC_STEER_RULE_COUNT; i++) |
| 1220 | bgx_reg_write(bgx, 0, BGX_CMR_RX_STREERING + (i * 8), 0x00); | 1220 | bgx_reg_write(bgx, 0, BGX_CMR_RX_STEERING + (i * 8), 0x00); |
| 1221 | } | 1221 | } |
| 1222 | 1222 | ||
| 1223 | static u8 bgx_get_lane2sds_cfg(struct bgx *bgx, struct lmac *lmac) | 1223 | static u8 bgx_get_lane2sds_cfg(struct bgx *bgx, struct lmac *lmac) |
| @@ -60,7 +60,7 @@ · 59 unmodified lines | |||
| 60 | #define RX_DMACX_CAM_EN BIT_ULL(48) | 60 | #define RX_DMACX_CAM_EN BIT_ULL(48) |
| 61 | #define RX_DMACX_CAM_LMACID(x) (((u64)x) << 49) | 61 | #define RX_DMACX_CAM_LMACID(x) (((u64)x) << 49) |
| 62 | #define RX_DMAC_COUNT 32 | 62 | #define RX_DMAC_COUNT 32 |
| 63 | #define BGX_CMR_RX_STREERING 0x300 | 63 | #define BGX_CMR_RX_STEERING 0x300 |
| 64 | #define RX_TRAFFIC_STEER_RULE_COUNT 8 | 64 | #define RX_TRAFFIC_STEER_RULE_COUNT 8 |
| 65 | #define BGX_CMR_CHAN_MSK_AND 0x450 | 65 | #define BGX_CMR_CHAN_MSK_AND 0x450 |
| 66 | #define BGX_CMR_BIST_STATUS 0x460 | 66 | #define BGX_CMR_BIST_STATUS 0x460 |
| @@ -660,6 +660,7 @@ static void uld_init(struct adapter *adap, struct cxgb4_lld_info *lld) · 659 unmodified lines | |||
| 660 | lld->cclk_ps = 1000000000 / adap->params.vpd.cclk; | 660 | lld->cclk_ps = 1000000000 / adap->params.vpd.cclk; |
| 661 | lld->udb_density = 1 << adap->params.sge.eq_qpp; | 661 | lld->udb_density = 1 << adap->params.sge.eq_qpp; |
| 662 | lld->ucq_density = 1 << adap->params.sge.iq_qpp; | 662 | lld->ucq_density = 1 << adap->params.sge.iq_qpp; |
| 663 | lld->sge_host_page_size = 1 << (adap->params.sge.hps + 10); | ||
| 663 | lld->filt_mode = adap->params.tp.vlan_pri_map; | 664 | lld->filt_mode = adap->params.tp.vlan_pri_map; |
| 664 | /* MODQ_REQ_MAP sets queues 0-3 to chan 0-3 */ | 665 | /* MODQ_REQ_MAP sets queues 0-3 to chan 0-3 */ |
| 665 | for (i = 0; i < NCHAN; i++) | 666 | for (i = 0; i < NCHAN; i++) |
| @@ -336,6 +336,7 @@ struct cxgb4_lld_info { · 335 unmodified lines | |||
| 336 | unsigned int cclk_ps; /* Core clock period in psec */ | 336 | unsigned int cclk_ps; /* Core clock period in psec */ |
| 337 | unsigned short udb_density; /* # of user DB/page */ | 337 | unsigned short udb_density; /* # of user DB/page */ |
| 338 | unsigned short ucq_density; /* # of user CQs/page */ | 338 | unsigned short ucq_density; /* # of user CQs/page */ |
| 339 | unsigned int sge_host_page_size; /* SGE host page size */ | ||
| 339 | unsigned short filt_mode; /* filter optional components */ | 340 | unsigned short filt_mode; /* filter optional components */ |
| 340 | unsigned short tx_modq[NCHAN]; /* maps each tx channel to a */ | 341 | unsigned short tx_modq[NCHAN]; /* maps each tx channel to a */ |
| 341 | /* scheduler queue */ | 342 | /* scheduler queue */ |
| @@ -3128,6 +3128,9 @@ int hns_dsaf_roce_reset(struct fwnode_handle *dsaf_fwnode, bool dereset) · 3127 unmodified lines | |||
| 3128 | dsaf_set_bit(credit, DSAF_SBM_ROCEE_CFG_CRD_EN_B, 1); | 3128 | dsaf_set_bit(credit, DSAF_SBM_ROCEE_CFG_CRD_EN_B, 1); |
| 3129 | dsaf_write_dev(dsaf_dev, DSAF_SBM_ROCEE_CFG_REG_REG, credit); | 3129 | dsaf_write_dev(dsaf_dev, DSAF_SBM_ROCEE_CFG_REG_REG, credit); |
| 3130 | } | 3130 | } |
| 3131 | |||
| 3132 | put_device(&pdev->dev); | ||
| 3133 | |||
| 3131 | return 0; | 3134 | return 0; |
| 3132 | } | 3135 | } |
| 3133 | EXPORT_SYMBOL(hns_dsaf_roce_reset); | 3136 | EXPORT_SYMBOL(hns_dsaf_roce_reset); |
| @@ -3289,8 +3289,11 @@ static int i40e_configure_rx_ring(struct i40e_ring *ring) · 3288 unmodified lines | |||
| 3289 | i40e_alloc_rx_buffers_zc(ring, I40E_DESC_UNUSED(ring)) : | 3289 | i40e_alloc_rx_buffers_zc(ring, I40E_DESC_UNUSED(ring)) : |
| 3290 | !i40e_alloc_rx_buffers(ring, I40E_DESC_UNUSED(ring)); | 3290 | !i40e_alloc_rx_buffers(ring, I40E_DESC_UNUSED(ring)); |
| 3291 | if (!ok) { | 3291 | if (!ok) { |
| 3292 | /* Log this in case the user has forgotten to give the kernel | ||
| 3293 | * any buffers, even later in the application. | ||
| 3294 | */ | ||
| 3292 | dev_info(&vsi->back->pdev->dev, | 3295 | dev_info(&vsi->back->pdev->dev, |
| 3293 | "Failed allocate some buffers on %sRx ring %d (pf_q %d)\n", | 3296 | "Failed to allocate some buffers on %sRx ring %d (pf_q %d)\n", |
| 3294 | ring->xsk_umem ? "UMEM enabled " : "", | 3297 | ring->xsk_umem ? "UMEM enabled " : "", |
| 3295 | ring->queue_index, pf_q); | 3298 | ring->queue_index, pf_q); |
| 3296 | } | 3299 | } |
| @@ -6725,8 +6728,13 @@ void i40e_down(struct i40e_vsi *vsi) · 3428 unmodified lines | |||
| 6725 | 6728 | ||
| 6726 | for (i = 0; i < vsi->num_queue_pairs; i++) { | 6729 | for (i = 0; i < vsi->num_queue_pairs; i++) { |
| 6727 | i40e_clean_tx_ring(vsi->tx_rings[i]); | 6730 | i40e_clean_tx_ring(vsi->tx_rings[i]); |
| 6728 | if (i40e_enabled_xdp_vsi(vsi)) | 6731 | if (i40e_enabled_xdp_vsi(vsi)) { |
| 6732 | /* Make sure that in-progress ndo_xdp_xmit | ||
| 6733 | * calls are completed. | ||
| 6734 | */ | ||
| 6735 | synchronize_rcu(); | ||
| 6729 | i40e_clean_tx_ring(vsi->xdp_rings[i]); | 6736 | i40e_clean_tx_ring(vsi->xdp_rings[i]); |
| 6737 | } | ||
| 6730 | i40e_clean_rx_ring(vsi->rx_rings[i]); | 6738 | i40e_clean_rx_ring(vsi->rx_rings[i]); |
| 6731 | } | 6739 | } |
| 6732 | 6740 | ||
| @@ -11895,6 +11903,14 @@ static int i40e_xdp_setup(struct i40e_vsi *vsi, · 5162 unmodified lines | |||
| 11895 | if (old_prog) | 11903 | if (old_prog) |
| 11896 | bpf_prog_put(old_prog); | 11904 | bpf_prog_put(old_prog); |
| 11897 | 11905 | ||
| 11906 | /* Kick start the NAPI context if there is an AF_XDP socket open | ||
| 11907 | * on that queue id. This so that receiving will start. | ||
| 11908 | */ | ||
| 11909 | if (need_reset && prog) | ||
| 11910 | for (i = 0; i < vsi->num_queue_pairs; i++) | ||
| 11911 | if (vsi->xdp_rings[i]->xsk_umem) | ||
| 11912 | (void)i40e_xsk_async_xmit(vsi->netdev, i); | ||
| 11913 | |||
| 11898 | return 0; | 11914 | return 0; |
| 11899 | } | 11915 | } |
| 11900 | 11916 | ||
| @@ -11955,8 +11971,13 @@ static void i40e_queue_pair_reset_stats(struct i40e_vsi *vsi, int queue_pair) · 54 unmodified lines | |||
| 11955 | static void i40e_queue_pair_clean_rings(struct i40e_vsi *vsi, int queue_pair) | 11971 | static void i40e_queue_pair_clean_rings(struct i40e_vsi *vsi, int queue_pair) |
| 11956 | { | 11972 | { |
| 11957 | i40e_clean_tx_ring(vsi->tx_rings[queue_pair]); | 11973 | i40e_clean_tx_ring(vsi->tx_rings[queue_pair]); |
| 11958 | if (i40e_enabled_xdp_vsi(vsi)) | 11974 | if (i40e_enabled_xdp_vsi(vsi)) { |
| 11975 | /* Make sure that in-progress ndo_xdp_xmit calls are | ||
| 11976 | * completed. | ||
| 11977 | */ | ||
| 11978 | synchronize_rcu(); | ||
| 11959 | i40e_clean_tx_ring(vsi->xdp_rings[queue_pair]); | 11979 | i40e_clean_tx_ring(vsi->xdp_rings[queue_pair]); |
| 11980 | } | ||
| 11960 | i40e_clean_rx_ring(vsi->rx_rings[queue_pair]); | 11981 | i40e_clean_rx_ring(vsi->rx_rings[queue_pair]); |
| 11961 | } | 11982 | } |
| 11962 | 11983 | ||
| @@ -3709,6 +3709,7 @@ int i40e_xdp_xmit(struct net_device *dev, int n, struct xdp_frame **frames, · 3708 unmodified lines | |||
| 3709 | struct i40e_netdev_priv *np = netdev_priv(dev); | 3709 | struct i40e_netdev_priv *np = netdev_priv(dev); |
| 3710 | unsigned int queue_index = smp_processor_id(); | 3710 | unsigned int queue_index = smp_processor_id(); |
| 3711 | struct i40e_vsi *vsi = np->vsi; | 3711 | struct i40e_vsi *vsi = np->vsi; |
| 3712 | struct i40e_pf *pf = vsi->back; | ||
| 3712 | struct i40e_ring *xdp_ring; | 3713 | struct i40e_ring *xdp_ring; |
| 3713 | int drops = 0; | 3714 | int drops = 0; |
| 3714 | int i; | 3715 | int i; |
| @@ -3716,7 +3717,8 @@ int i40e_xdp_xmit(struct net_device *dev, int n, struct xdp_frame **frames, · 1 unmodified line | |||
| 3716 | if (test_bit(__I40E_VSI_DOWN, vsi->state)) | 3717 | if (test_bit(__I40E_VSI_DOWN, vsi->state)) |
| 3717 | return -ENETDOWN; | 3718 | return -ENETDOWN; |
| 3718 | 3719 | ||
| 3719 | if (!i40e_enabled_xdp_vsi(vsi) || queue_index >= vsi->num_queue_pairs) | 3720 | if (!i40e_enabled_xdp_vsi(vsi) || queue_index >= vsi->num_queue_pairs || |
| 3721 | test_bit(__I40E_CONFIG_BUSY, pf->state)) | ||
| 3720 | return -ENXIO; | 3722 | return -ENXIO; |
| 3721 | 3723 | ||
| 3722 | if (unlikely(flags & ~XDP_XMIT_FLAGS_MASK)) | 3724 | if (unlikely(flags & ~XDP_XMIT_FLAGS_MASK)) |
| @@ -183,6 +183,11 @@ static int i40e_xsk_umem_enable(struct i40e_vsi *vsi, struct xdp_umem *umem, · 182 unmodified lines | |||
| 183 | err = i40e_queue_pair_enable(vsi, qid); | 183 | err = i40e_queue_pair_enable(vsi, qid); |
| 184 | if (err) | 184 | if (err) |
| 185 | return err; | 185 | return err; |
| 186 | |||
| 187 | /* Kick start the NAPI context so that receiving will start */ | ||
| 188 | err = i40e_xsk_async_xmit(vsi->netdev, qid); | ||
| 189 | if (err) | ||
| 190 | return err; | ||
| 186 | } | 191 | } |
| 187 | 192 | ||
| 188 | return 0; | 193 | return 0; |
| @@ -3953,8 +3953,11 @@ static void ixgbe_setup_mrqc(struct ixgbe_adapter *adapter) · 3952 unmodified lines | |||
| 3953 | else | 3953 | else |
| 3954 | mrqc = IXGBE_MRQC_VMDQRSS64EN; | 3954 | mrqc = IXGBE_MRQC_VMDQRSS64EN; |
| 3955 | 3955 | ||
| 3956 | /* Enable L3/L4 for Tx Switched packets */ | 3956 | /* Enable L3/L4 for Tx Switched packets only for X550, |
| 3957 | mrqc |= IXGBE_MRQC_L3L4TXSWEN; | 3957 | * older devices do not support this feature |
| 3958 | */ | ||
| 3959 | if (hw->mac.type >= ixgbe_mac_X550) | ||
| 3960 | mrqc |= IXGBE_MRQC_L3L4TXSWEN; | ||
| 3958 | } else { | 3961 | } else { |
| 3959 | if (tcs > 4) | 3962 | if (tcs > 4) |
| 3960 | mrqc = IXGBE_MRQC_RTRSS8TCEN; | 3963 | mrqc = IXGBE_MRQC_RTRSS8TCEN; |
| @@ -10225,6 +10228,7 @@ static int ixgbe_xdp_setup(struct net_device *dev, struct bpf_prog *prog) · 6264 unmodified lines | |||
| 10225 | int i, frame_size = dev->mtu + ETH_HLEN + ETH_FCS_LEN + VLAN_HLEN; | 10228 | int i, frame_size = dev->mtu + ETH_HLEN + ETH_FCS_LEN + VLAN_HLEN; |
| 10226 | struct ixgbe_adapter *adapter = netdev_priv(dev); | 10229 | struct ixgbe_adapter *adapter = netdev_priv(dev); |
| 10227 | struct bpf_prog *old_prog; | 10230 | struct bpf_prog *old_prog; |
| 10231 | bool need_reset; | ||
| 10228 | 10232 | ||
| 10229 | if (adapter->flags & IXGBE_FLAG_SRIOV_ENABLED) | 10233 | if (adapter->flags & IXGBE_FLAG_SRIOV_ENABLED) |
| 10230 | return -EINVAL; | 10234 | return -EINVAL; |
| @@ -10247,9 +10251,10 @@ static int ixgbe_xdp_setup(struct net_device *dev, struct bpf_prog *prog) · 16 unmodified lines | |||
| 10247 | return -ENOMEM; | 10251 | return -ENOMEM; |
| 10248 | 10252 | ||
| 10249 | old_prog = xchg(&adapter->xdp_prog, prog); | 10253 | old_prog = xchg(&adapter->xdp_prog, prog); |
| 10254 | need_reset = (!!prog != !!old_prog); | ||
| 10250 | 10255 | ||
| 10251 | /* If transitioning XDP modes reconfigure rings */ | 10256 | /* If transitioning XDP modes reconfigure rings */ |
| 10252 | if (!!prog != !!old_prog) { | 10257 | if (need_reset) { |
| 10253 | int err = ixgbe_setup_tc(dev, adapter->hw_tcs); | 10258 | int err = ixgbe_setup_tc(dev, adapter->hw_tcs); |
| 10254 | 10259 | ||
| 10255 | if (err) { | 10260 | if (err) { |
| @@ -10265,6 +10270,14 @@ static int ixgbe_xdp_setup(struct net_device *dev, struct bpf_prog *prog) · 9 unmodified lines | |||
| 10265 | if (old_prog) | 10270 | if (old_prog) |
| 10266 | bpf_prog_put(old_prog); | 10271 | bpf_prog_put(old_prog); |
| 10267 | 10272 | ||
| 10273 | /* Kick start the NAPI context if there is an AF_XDP socket open | ||
| 10274 | * on that queue id. This so that receiving will start. | ||
| 10275 | */ | ||
| 10276 | if (need_reset && prog) | ||
| 10277 | for (i = 0; i < adapter->num_rx_queues; i++) | ||
| 10278 | if (adapter->xdp_ring[i]->xsk_umem) | ||
| 10279 | (void)ixgbe_xsk_async_xmit(adapter->netdev, i); | ||
| 10280 | |||
| 10268 | return 0; | 10281 | return 0; |
| 10269 | } | 10282 | } |
| 10270 | 10283 | ||
| @@ -144,11 +144,19 @@ static int ixgbe_xsk_umem_enable(struct ixgbe_adapter *adapter, · 143 unmodified lines | |||
| 144 | ixgbe_txrx_ring_disable(adapter, qid); | 144 | ixgbe_txrx_ring_disable(adapter, qid); |
| 145 | 145 | ||
| 146 | err = ixgbe_add_xsk_umem(adapter, umem, qid); | 146 | err = ixgbe_add_xsk_umem(adapter, umem, qid); |
| 147 | if (err) | ||
| 148 | return err; | ||
| 147 | 149 | ||
| 148 | if (if_running) | 150 | if (if_running) { |
| 149 | ixgbe_txrx_ring_enable(adapter, qid); | 151 | ixgbe_txrx_ring_enable(adapter, qid); |
| 150 | 152 | ||
| 151 | return err; | 153 | /* Kick start the NAPI context so that receiving will start */ |
| 154 | err = ixgbe_xsk_async_xmit(adapter->netdev, qid); | ||
| 155 | if (err) | ||
| 156 | return err; | ||
| 157 | } | ||
| 158 | |||
| 159 | return 0; | ||
| 152 | } | 160 | } |
| 153 | 161 | ||
| 154 | static int ixgbe_xsk_umem_disable(struct ixgbe_adapter *adapter, u16 qid) | 162 | static int ixgbe_xsk_umem_disable(struct ixgbe_adapter *adapter, u16 qid) |
| @@ -634,7 +642,8 @@ static bool ixgbe_xmit_zc(struct ixgbe_ring *xdp_ring, unsigned int budget) · 479 unmodified lines | |||
| 634 | dma_addr_t dma; | 642 | dma_addr_t dma; |
| 635 | 643 | ||
| 636 | while (budget-- > 0) { | 644 | while (budget-- > 0) { |
| 637 | if (unlikely(!ixgbe_desc_unused(xdp_ring))) { | 645 | if (unlikely(!ixgbe_desc_unused(xdp_ring)) || |
| 646 | !netif_carrier_ok(xdp_ring->netdev)) { | ||
| 638 | work_done = false; | 647 | work_done = false; |
| 639 | break; | 648 | break; |
| 640 | } | 649 | } |
| @@ -2879,7 +2879,7 @@ static int mv643xx_eth_shared_probe(struct platform_device *pdev) · 2878 unmodified lines | |||
| 2879 | 2879 | ||
| 2880 | ret = mv643xx_eth_shared_of_probe(pdev); | 2880 | ret = mv643xx_eth_shared_of_probe(pdev); |
| 2881 | if (ret) | 2881 | if (ret) |
| 2882 | return ret; | 2882 | goto err_put_clk; |
| 2883 | pd = dev_get_platdata(&pdev->dev); | 2883 | pd = dev_get_platdata(&pdev->dev); |
| 2884 | 2884 | ||
| 2885 | msp->tx_csum_limit = (pd != NULL && pd->tx_csum_limit) ? | 2885 | msp->tx_csum_limit = (pd != NULL && pd->tx_csum_limit) ? |
| @@ -2887,6 +2887,11 @@ static int mv643xx_eth_shared_probe(struct platform_device *pdev) · 1 unmodified line | |||
| 2887 | infer_hw_params(msp); | 2887 | infer_hw_params(msp); |
| 2888 | 2888 | ||
| 2889 | return 0; | 2889 | return 0; |
| 2890 | |||
| 2891 | err_put_clk: | ||
| 2892 | if (!IS_ERR(msp->clk)) | ||
| 2893 | clk_disable_unprepare(msp->clk); | ||
| 2894 | return ret; | ||
| 2890 | } | 2895 | } |
| 2891 | 2896 | ||
| 2892 | static int mv643xx_eth_shared_remove(struct platform_device *pdev) | 2897 | static int mv643xx_eth_shared_remove(struct platform_device *pdev) |
| @@ -2146,7 +2146,7 @@ err_drop_frame: · 2145 unmodified lines | |||
| 2146 | if (unlikely(!skb)) | 2146 | if (unlikely(!skb)) |
| 2147 | goto err_drop_frame_ret_pool; | 2147 | goto err_drop_frame_ret_pool; |
| 2148 | 2148 | ||
| 2149 | dma_sync_single_range_for_cpu(dev->dev.parent, | 2149 | dma_sync_single_range_for_cpu(&pp->bm_priv->pdev->dev, |
| 2150 | rx_desc->buf_phys_addr, | 2150 | rx_desc->buf_phys_addr, |
| 2151 | MVNETA_MH_SIZE + NET_SKB_PAD, | 2151 | MVNETA_MH_SIZE + NET_SKB_PAD, |
| 2152 | rx_bytes, | 2152 | rx_bytes, |
| @@ -5073,7 +5073,7 @@ static int sky2_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 5072 unmodified lines | |||
| 5073 | INIT_WORK(&hw->restart_work, sky2_restart); | 5073 | INIT_WORK(&hw->restart_work, sky2_restart); |
| 5074 | 5074 | ||
| 5075 | pci_set_drvdata(pdev, hw); | 5075 | pci_set_drvdata(pdev, hw); |
| 5076 | pdev->d3_delay = 200; | 5076 | pdev->d3_delay = 300; |
| 5077 | 5077 | ||
| 5078 | return 0; | 5078 | return 0; |
| 5079 | 5079 | ||
| @@ -3360,7 +3360,7 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port, · 3359 unmodified lines | |||
| 3360 | dev->addr_len = ETH_ALEN; | 3360 | dev->addr_len = ETH_ALEN; |
| 3361 | mlx4_en_u64_to_mac(dev->dev_addr, mdev->dev->caps.def_mac[priv->port]); | 3361 | mlx4_en_u64_to_mac(dev->dev_addr, mdev->dev->caps.def_mac[priv->port]); |
| 3362 | if (!is_valid_ether_addr(dev->dev_addr)) { | 3362 | if (!is_valid_ether_addr(dev->dev_addr)) { |
| 3363 | en_err(priv, "Port: %d, invalid mac burned: %pM, quiting\n", | 3363 | en_err(priv, "Port: %d, invalid mac burned: %pM, quitting\n", |
| 3364 | priv->port, dev->dev_addr); | 3364 | priv->port, dev->dev_addr); |
| 3365 | err = -EINVAL; | 3365 | err = -EINVAL; |
| 3366 | goto out; | 3366 | goto out; |
| @@ -862,8 +862,9 @@ int __mlxsw_sp_port_headroom_set(struct mlxsw_sp_port *mlxsw_sp_port, int mtu, · 861 unmodified lines | |||
| 862 | for (i = 0; i < IEEE_8021QAZ_MAX_TCS; i++) { | 862 | for (i = 0; i < IEEE_8021QAZ_MAX_TCS; i++) { |
| 863 | bool configure = false; | 863 | bool configure = false; |
| 864 | bool pfc = false; | 864 | bool pfc = false; |
| 865 | u16 thres_cells; | ||
| 866 | u16 delay_cells; | ||
| 865 | bool lossy; | 867 | bool lossy; |
| 866 | u16 thres; | ||
| 867 | 868 | ||
| 868 | for (j = 0; j < IEEE_8021QAZ_MAX_TCS; j++) { | 869 | for (j = 0; j < IEEE_8021QAZ_MAX_TCS; j++) { |
| 869 | if (prio_tc[j] == i) { | 870 | if (prio_tc[j] == i) { |
| @@ -877,10 +878,11 @@ int __mlxsw_sp_port_headroom_set(struct mlxsw_sp_port *mlxsw_sp_port, int mtu, · 7 unmodified lines | |||
| 877 | continue; | 878 | continue; |
| 878 | 879 | ||
| 879 | lossy = !(pfc || pause_en); | 880 | lossy = !(pfc || pause_en); |
| 880 | thres = mlxsw_sp_pg_buf_threshold_get(mlxsw_sp, mtu); | 881 | thres_cells = mlxsw_sp_pg_buf_threshold_get(mlxsw_sp, mtu); |
| 881 | delay = mlxsw_sp_pg_buf_delay_get(mlxsw_sp, mtu, delay, pfc, | 882 | delay_cells = mlxsw_sp_pg_buf_delay_get(mlxsw_sp, mtu, delay, |
| 882 | pause_en); | 883 | pfc, pause_en); |
| 883 | mlxsw_sp_pg_buf_pack(pbmc_pl, i, thres + delay, thres, lossy); | 884 | mlxsw_sp_pg_buf_pack(pbmc_pl, i, thres_cells + delay_cells, |
| 885 | thres_cells, lossy); | ||
| 884 | } | 886 | } |
| 885 | 887 | ||
| 886 | return mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(pbmc), pbmc_pl); | 888 | return mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(pbmc), pbmc_pl); |
| @@ -1681,5 +1681,5 @@ MODULE_DESCRIPTION(DRV_NAME " ethernet driver"); · 1680 unmodified lines | |||
| 1681 | MODULE_AUTHOR("Claudio Lanconelli <lanconelli.claudio@eptar.com>"); | 1681 | MODULE_AUTHOR("Claudio Lanconelli <lanconelli.claudio@eptar.com>"); |
| 1682 | MODULE_LICENSE("GPL"); | 1682 | MODULE_LICENSE("GPL"); |
| 1683 | module_param_named(debug, debug.msg_enable, int, 0); | 1683 | module_param_named(debug, debug.msg_enable, int, 0); |
| 1684 | MODULE_PARM_DESC(debug, "Debug verbosity level (0=none, ..., ffff=all)"); | 1684 | MODULE_PARM_DESC(debug, "Debug verbosity level in amount of bits set (0=none, ..., 31=all)"); |
| 1685 | MODULE_ALIAS("spi:" DRV_NAME); | 1685 | MODULE_ALIAS("spi:" DRV_NAME); |
| @@ -1400,7 +1400,8 @@ static int lan743x_tx_frame_start(struct lan743x_tx *tx, · 1399 unmodified lines | |||
| 1400 | } | 1400 | } |
| 1401 | 1401 | ||
| 1402 | static void lan743x_tx_frame_add_lso(struct lan743x_tx *tx, | 1402 | static void lan743x_tx_frame_add_lso(struct lan743x_tx *tx, |
| 1403 | unsigned int frame_length) | 1403 | unsigned int frame_length, |
| 1404 | int nr_frags) | ||
| 1404 | { | 1405 | { |
| 1405 | /* called only from within lan743x_tx_xmit_frame. | 1406 | /* called only from within lan743x_tx_xmit_frame. |
| 1406 | * assuming tx->ring_lock has already been acquired. | 1407 | * assuming tx->ring_lock has already been acquired. |
| @@ -1410,6 +1411,10 @@ static void lan743x_tx_frame_add_lso(struct lan743x_tx *tx, · 3 unmodified lines | |||
| 1410 | 1411 | ||
| 1411 | /* wrap up previous descriptor */ | 1412 | /* wrap up previous descriptor */ |
| 1412 | tx->frame_data0 |= TX_DESC_DATA0_EXT_; | 1413 | tx->frame_data0 |= TX_DESC_DATA0_EXT_; |
| 1414 | if (nr_frags <= 0) { | ||
| 1415 | tx->frame_data0 |= TX_DESC_DATA0_LS_; | ||
| 1416 | tx->frame_data0 |= TX_DESC_DATA0_IOC_; | ||
| 1417 | } | ||
| 1413 | tx_descriptor = &tx->ring_cpu_ptr[tx->frame_tail]; | 1418 | tx_descriptor = &tx->ring_cpu_ptr[tx->frame_tail]; |
| 1414 | tx_descriptor->data0 = tx->frame_data0; | 1419 | tx_descriptor->data0 = tx->frame_data0; |
| 1415 | 1420 | ||
| @@ -1514,8 +1519,11 @@ static void lan743x_tx_frame_end(struct lan743x_tx *tx, · 98 unmodified lines | |||
| 1514 | u32 tx_tail_flags = 0; | 1519 | u32 tx_tail_flags = 0; |
| 1515 | 1520 | ||
| 1516 | /* wrap up previous descriptor */ | 1521 | /* wrap up previous descriptor */ |
| 1517 | tx->frame_data0 |= TX_DESC_DATA0_LS_; | 1522 | if ((tx->frame_data0 & TX_DESC_DATA0_DTYPE_MASK_) == |
| 1518 | tx->frame_data0 |= TX_DESC_DATA0_IOC_; | 1523 | TX_DESC_DATA0_DTYPE_DATA_) { |
| 1524 | tx->frame_data0 |= TX_DESC_DATA0_LS_; | ||
| 1525 | tx->frame_data0 |= TX_DESC_DATA0_IOC_; | ||
| 1526 | } | ||
| 1519 | 1527 | ||
| 1520 | tx_descriptor = &tx->ring_cpu_ptr[tx->frame_tail]; | 1528 | tx_descriptor = &tx->ring_cpu_ptr[tx->frame_tail]; |
| 1521 | buffer_info = &tx->buffer_info[tx->frame_tail]; | 1529 | buffer_info = &tx->buffer_info[tx->frame_tail]; |
| @@ -1600,7 +1608,7 @@ static netdev_tx_t lan743x_tx_xmit_frame(struct lan743x_tx *tx, · 78 unmodified lines | |||
| 1600 | } | 1608 | } |
| 1601 | 1609 | ||
| 1602 | if (gso) | 1610 | if (gso) |
| 1603 | lan743x_tx_frame_add_lso(tx, frame_length); | 1611 | lan743x_tx_frame_add_lso(tx, frame_length, nr_frags); |
| 1604 | 1612 | ||
| 1605 | if (nr_frags <= 0) | 1613 | if (nr_frags <= 0) |
| 1606 | goto finish; | 1614 | goto finish; |
| @@ -1291,15 +1291,10 @@ wrp_alu64_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, · 1290 unmodified lines | |||
| 1291 | 1291 | ||
| 1292 | static int | 1292 | static int |
| 1293 | wrp_alu32_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, | 1293 | wrp_alu32_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, |
| 1294 | enum alu_op alu_op, bool skip) | 1294 | enum alu_op alu_op) |
| 1295 | { | 1295 | { |
| 1296 | const struct bpf_insn *insn = &meta->insn; | 1296 | const struct bpf_insn *insn = &meta->insn; |
| 1297 | 1297 | ||
| 1298 | if (skip) { | ||
| 1299 | meta->skip = true; | ||
| 1300 | return 0; | ||
| 1301 | } | ||
| 1302 | |||
| 1303 | wrp_alu_imm(nfp_prog, insn->dst_reg * 2, alu_op, insn->imm); | 1298 | wrp_alu_imm(nfp_prog, insn->dst_reg * 2, alu_op, insn->imm); |
| 1304 | wrp_immed(nfp_prog, reg_both(insn->dst_reg * 2 + 1), 0); | 1299 | wrp_immed(nfp_prog, reg_both(insn->dst_reg * 2 + 1), 0); |
| 1305 | 1300 | ||
| @@ -2309,7 +2304,7 @@ static int xor_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) · 1003 unmodified lines | |||
| 2309 | 2304 | ||
| 2310 | static int xor_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) | 2305 | static int xor_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| 2311 | { | 2306 | { |
| 2312 | return wrp_alu32_imm(nfp_prog, meta, ALU_OP_XOR, !~meta->insn.imm); | 2307 | return wrp_alu32_imm(nfp_prog, meta, ALU_OP_XOR); |
| 2313 | } | 2308 | } |
| 2314 | 2309 | ||
| 2315 | static int and_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) | 2310 | static int and_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| @@ -2319,7 +2314,7 @@ static int and_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) · 3 unmodified lines | |||
| 2319 | 2314 | ||
| 2320 | static int and_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) | 2315 | static int and_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| 2321 | { | 2316 | { |
| 2322 | return wrp_alu32_imm(nfp_prog, meta, ALU_OP_AND, !~meta->insn.imm); | 2317 | return wrp_alu32_imm(nfp_prog, meta, ALU_OP_AND); |
| 2323 | } | 2318 | } |
| 2324 | 2319 | ||
| 2325 | static int or_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) | 2320 | static int or_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| @@ -2329,7 +2324,7 @@ static int or_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) · 3 unmodified lines | |||
| 2329 | 2324 | ||
| 2330 | static int or_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) | 2325 | static int or_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| 2331 | { | 2326 | { |
| 2332 | return wrp_alu32_imm(nfp_prog, meta, ALU_OP_OR, !meta->insn.imm); | 2327 | return wrp_alu32_imm(nfp_prog, meta, ALU_OP_OR); |
| 2333 | } | 2328 | } |
| 2334 | 2329 | ||
| 2335 | static int add_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) | 2330 | static int add_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| @@ -2339,7 +2334,7 @@ static int add_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) · 3 unmodified lines | |||
| 2339 | 2334 | ||
| 2340 | static int add_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) | 2335 | static int add_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| 2341 | { | 2336 | { |
| 2342 | return wrp_alu32_imm(nfp_prog, meta, ALU_OP_ADD, !meta->insn.imm); | 2337 | return wrp_alu32_imm(nfp_prog, meta, ALU_OP_ADD); |
| 2343 | } | 2338 | } |
| 2344 | 2339 | ||
| 2345 | static int sub_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) | 2340 | static int sub_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| @@ -2349,7 +2344,7 @@ static int sub_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) · 3 unmodified lines | |||
| 2349 | 2344 | ||
| 2350 | static int sub_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) | 2345 | static int sub_imm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| 2351 | { | 2346 | { |
| 2352 | return wrp_alu32_imm(nfp_prog, meta, ALU_OP_SUB, !meta->insn.imm); | 2347 | return wrp_alu32_imm(nfp_prog, meta, ALU_OP_SUB); |
| 2353 | } | 2348 | } |
| 2354 | 2349 | ||
| 2355 | static int mul_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) | 2350 | static int mul_reg(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta) |
| @@ -1688,6 +1688,15 @@ qed_iwarp_parse_rx_pkt(struct qed_hwfn *p_hwfn, · 1687 unmodified lines | |||
| 1688 | 1688 | ||
| 1689 | eth_hlen = ETH_HLEN + (vlan_valid ? sizeof(u32) : 0); | 1689 | eth_hlen = ETH_HLEN + (vlan_valid ? sizeof(u32) : 0); |
| 1690 | 1690 | ||
| 1691 | if (!ether_addr_equal(ethh->h_dest, | ||
| 1692 | p_hwfn->p_rdma_info->iwarp.mac_addr)) { | ||
| 1693 | DP_VERBOSE(p_hwfn, | ||
| 1694 | QED_MSG_RDMA, | ||
| 1695 | "Got unexpected mac %pM instead of %pM\n", | ||
| 1696 | ethh->h_dest, p_hwfn->p_rdma_info->iwarp.mac_addr); | ||
| 1697 | return -EINVAL; | ||
| 1698 | } | ||
| 1699 | |||
| 1691 | ether_addr_copy(remote_mac_addr, ethh->h_source); | 1700 | ether_addr_copy(remote_mac_addr, ethh->h_source); |
| 1692 | ether_addr_copy(local_mac_addr, ethh->h_dest); | 1701 | ether_addr_copy(local_mac_addr, ethh->h_dest); |
| 1693 | 1702 | ||
| @@ -2605,7 +2614,7 @@ qed_iwarp_ll2_start(struct qed_hwfn *p_hwfn, · 911 unmodified lines | |||
| 2605 | struct qed_iwarp_info *iwarp_info; | 2614 | struct qed_iwarp_info *iwarp_info; |
| 2606 | struct qed_ll2_acquire_data data; | 2615 | struct qed_ll2_acquire_data data; |
| 2607 | struct qed_ll2_cbs cbs; | 2616 | struct qed_ll2_cbs cbs; |
| 2608 | u32 mpa_buff_size; | 2617 | u32 buff_size; |
| 2609 | u16 n_ooo_bufs; | 2618 | u16 n_ooo_bufs; |
| 2610 | int rc = 0; | 2619 | int rc = 0; |
| 2611 | int i; | 2620 | int i; |
| @@ -2632,7 +2641,7 @@ qed_iwarp_ll2_start(struct qed_hwfn *p_hwfn, · 20 unmodified lines | |||
| 2632 | 2641 | ||
| 2633 | memset(&data, 0, sizeof(data)); | 2642 | memset(&data, 0, sizeof(data)); |
| 2634 | data.input.conn_type = QED_LL2_TYPE_IWARP; | 2643 | data.input.conn_type = QED_LL2_TYPE_IWARP; |
| 2635 | data.input.mtu = QED_IWARP_MAX_SYN_PKT_SIZE; | 2644 | data.input.mtu = params->max_mtu; |
| 2636 | data.input.rx_num_desc = QED_IWARP_LL2_SYN_RX_SIZE; | 2645 | data.input.rx_num_desc = QED_IWARP_LL2_SYN_RX_SIZE; |
| 2637 | data.input.tx_num_desc = QED_IWARP_LL2_SYN_TX_SIZE; | 2646 | data.input.tx_num_desc = QED_IWARP_LL2_SYN_TX_SIZE; |
| 2638 | data.input.tx_max_bds_per_packet = 1; /* will never be fragmented */ | 2647 | data.input.tx_max_bds_per_packet = 1; /* will never be fragmented */ |
| @@ -2654,9 +2663,10 @@ qed_iwarp_ll2_start(struct qed_hwfn *p_hwfn, · 15 unmodified lines | |||
| 2654 | goto err; | 2663 | goto err; |
| 2655 | } | 2664 | } |
| 2656 | 2665 | ||
| 2666 | buff_size = QED_IWARP_MAX_BUF_SIZE(params->max_mtu); | ||
| 2657 | rc = qed_iwarp_ll2_alloc_buffers(p_hwfn, | 2667 | rc = qed_iwarp_ll2_alloc_buffers(p_hwfn, |
| 2658 | QED_IWARP_LL2_SYN_RX_SIZE, | 2668 | QED_IWARP_LL2_SYN_RX_SIZE, |
| 2659 | QED_IWARP_MAX_SYN_PKT_SIZE, | 2669 | buff_size, |
| 2660 | iwarp_info->ll2_syn_handle); | 2670 | iwarp_info->ll2_syn_handle); |
| 2661 | if (rc) | 2671 | if (rc) |
| 2662 | goto err; | 2672 | goto err; |
| @@ -2710,10 +2720,9 @@ qed_iwarp_ll2_start(struct qed_hwfn *p_hwfn, · 47 unmodified lines | |||
| 2710 | if (rc) | 2720 | if (rc) |
| 2711 | goto err; | 2721 | goto err; |
| 2712 | 2722 | ||
| 2713 | mpa_buff_size = QED_IWARP_MAX_BUF_SIZE(params->max_mtu); | ||
| 2714 | rc = qed_iwarp_ll2_alloc_buffers(p_hwfn, | 2723 | rc = qed_iwarp_ll2_alloc_buffers(p_hwfn, |
| 2715 | data.input.rx_num_desc, | 2724 | data.input.rx_num_desc, |
| 2716 | mpa_buff_size, | 2725 | buff_size, |
| 2717 | iwarp_info->ll2_mpa_handle); | 2726 | iwarp_info->ll2_mpa_handle); |
| 2718 | if (rc) | 2727 | if (rc) |
| 2719 | goto err; | 2728 | goto err; |
| @@ -2726,7 +2735,7 @@ qed_iwarp_ll2_start(struct qed_hwfn *p_hwfn, · 6 unmodified lines | |||
| 2726 | 2735 | ||
| 2727 | iwarp_info->max_num_partial_fpdus = (u16)p_hwfn->p_rdma_info->num_qps; | 2736 | iwarp_info->max_num_partial_fpdus = (u16)p_hwfn->p_rdma_info->num_qps; |
| 2728 | 2737 | ||
| 2729 | iwarp_info->mpa_intermediate_buf = kzalloc(mpa_buff_size, GFP_KERNEL); | 2738 | iwarp_info->mpa_intermediate_buf = kzalloc(buff_size, GFP_KERNEL); |
| 2730 | if (!iwarp_info->mpa_intermediate_buf) | 2739 | if (!iwarp_info->mpa_intermediate_buf) |
| 2731 | goto err; | 2740 | goto err; |
| 2732 | 2741 | ||
| @@ -46,7 +46,6 @@ enum qed_iwarp_qp_state qed_roce2iwarp_state(enum qed_roce_qp_state state); · 45 unmodified lines | |||
| 46 | 46 | ||
| 47 | #define QED_IWARP_LL2_SYN_TX_SIZE (128) | 47 | #define QED_IWARP_LL2_SYN_TX_SIZE (128) |
| 48 | #define QED_IWARP_LL2_SYN_RX_SIZE (256) | 48 | #define QED_IWARP_LL2_SYN_RX_SIZE (256) |
| 49 | #define QED_IWARP_MAX_SYN_PKT_SIZE (128) | ||
| 50 | 49 | ||
| 51 | #define QED_IWARP_LL2_OOO_DEF_TX_SIZE (256) | 50 | #define QED_IWARP_LL2_OOO_DEF_TX_SIZE (256) |
| 52 | #define QED_IWARP_MAX_OOO (16) | 51 | #define QED_IWARP_MAX_OOO (16) |
| @@ -241,15 +241,18 @@ static inline void dwmac4_get_timestamp(void *desc, u32 ats, u64 *ts) · 240 unmodified lines | |||
| 241 | static int dwmac4_rx_check_timestamp(void *desc) | 241 | static int dwmac4_rx_check_timestamp(void *desc) |
| 242 | { | 242 | { |
| 243 | struct dma_desc *p = (struct dma_desc *)desc; | 243 | struct dma_desc *p = (struct dma_desc *)desc; |
| 244 | unsigned int rdes0 = le32_to_cpu(p->des0); | ||
| 245 | unsigned int rdes1 = le32_to_cpu(p->des1); | ||
| 246 | unsigned int rdes3 = le32_to_cpu(p->des3); | ||
| 244 | u32 own, ctxt; | 247 | u32 own, ctxt; |
| 245 | int ret = 1; | 248 | int ret = 1; |
| 246 | 249 | ||
| 247 | own = p->des3 & RDES3_OWN; | 250 | own = rdes3 & RDES3_OWN; |
| 248 | ctxt = ((p->des3 & RDES3_CONTEXT_DESCRIPTOR) | 251 | ctxt = ((rdes3 & RDES3_CONTEXT_DESCRIPTOR) |
| 249 | >> RDES3_CONTEXT_DESCRIPTOR_SHIFT); | 252 | >> RDES3_CONTEXT_DESCRIPTOR_SHIFT); |
| 250 | 253 | ||
| 251 | if (likely(!own && ctxt)) { | 254 | if (likely(!own && ctxt)) { |
| 252 | if ((p->des0 == 0xffffffff) && (p->des1 == 0xffffffff)) | 255 | if ((rdes0 == 0xffffffff) && (rdes1 == 0xffffffff)) |
| 253 | /* Corrupted value */ | 256 | /* Corrupted value */ |
| 254 | ret = -EINVAL; | 257 | ret = -EINVAL; |
| 255 | else | 258 | else |
| @@ -696,25 +696,27 @@ static int stmmac_ethtool_op_set_eee(struct net_device *dev, · 695 unmodified lines | |||
| 696 | struct ethtool_eee *edata) | 696 | struct ethtool_eee *edata) |
| 697 | { | 697 | { |
| 698 | struct stmmac_priv *priv = netdev_priv(dev); | 698 | struct stmmac_priv *priv = netdev_priv(dev); |
| 699 | int ret; | ||
| 699 | 700 | ||
| 700 | priv->eee_enabled = edata->eee_enabled; | 701 | if (!edata->eee_enabled) { |
| 701 | |||
| 702 | if (!priv->eee_enabled) | ||
| 703 | stmmac_disable_eee_mode(priv); | 702 | stmmac_disable_eee_mode(priv); |
| 704 | else { | 703 | } else { |
| 705 | /* We are asking for enabling the EEE but it is safe | 704 | /* We are asking for enabling the EEE but it is safe |
| 706 | * to verify all by invoking the eee_init function. | 705 | * to verify all by invoking the eee_init function. |
| 707 | * In case of failure it will return an error. | 706 | * In case of failure it will return an error. |
| 708 | */ | 707 | */ |
| 709 | priv->eee_enabled = stmmac_eee_init(priv); | 708 | edata->eee_enabled = stmmac_eee_init(priv); |
| 710 | if (!priv->eee_enabled) | 709 | if (!edata->eee_enabled) |
| 711 | return -EOPNOTSUPP; | 710 | return -EOPNOTSUPP; |
| 712 | |||
| 713 | /* Do not change tx_lpi_timer in case of failure */ | ||
| 714 | priv->tx_lpi_timer = edata->tx_lpi_timer; | ||
| 715 | } | 711 | } |
| 716 | 712 | ||
| 717 | return phy_ethtool_set_eee(dev->phydev, edata); | 713 | ret = phy_ethtool_set_eee(dev->phydev, edata); |
| 714 | if (ret) | ||
| 715 | return ret; | ||
| 716 | |||
| 717 | priv->eee_enabled = edata->eee_enabled; | ||
| 718 | priv->tx_lpi_timer = edata->tx_lpi_timer; | ||
| 719 | return 0; | ||
| 718 | } | 720 | } |
| 719 | 721 | ||
| 720 | static u32 stmmac_usec2riwt(u32 usec, struct stmmac_priv *priv) | 722 | static u32 stmmac_usec2riwt(u32 usec, struct stmmac_priv *priv) |
| @@ -259,7 +259,7 @@ static int netcp_module_probe(struct netcp_device *netcp_device, · 258 unmodified lines | |||
| 259 | const char *name; | 259 | const char *name; |
| 260 | char node_name[32]; | 260 | char node_name[32]; |
| 261 | 261 | ||
| 262 | if (of_property_read_string(node, "label", &name) < 0) { | 262 | if (of_property_read_string(child, "label", &name) < 0) { |
| 263 | snprintf(node_name, sizeof(node_name), "%pOFn", child); | 263 | snprintf(node_name, sizeof(node_name), "%pOFn", child); |
| 264 | name = node_name; | 264 | name = node_name; |
| 265 | } | 265 | } |
| @@ -692,15 +692,20 @@ out: · 691 unmodified lines | |||
| 692 | static int geneve_open(struct net_device *dev) | 692 | static int geneve_open(struct net_device *dev) |
| 693 | { | 693 | { |
| 694 | struct geneve_dev *geneve = netdev_priv(dev); | 694 | struct geneve_dev *geneve = netdev_priv(dev); |
| 695 | bool ipv6 = !!(geneve->info.mode & IP_TUNNEL_INFO_IPV6); | ||
| 696 | bool metadata = geneve->collect_md; | 695 | bool metadata = geneve->collect_md; |
| 696 | bool ipv4, ipv6; | ||
| 697 | int ret = 0; | 697 | int ret = 0; |
| 698 | 698 | ||
| 699 | ipv6 = geneve->info.mode & IP_TUNNEL_INFO_IPV6 || metadata; | ||
| 700 | ipv4 = !ipv6 || metadata; | ||
| 699 | #if IS_ENABLED(CONFIG_IPV6) | 701 | #if IS_ENABLED(CONFIG_IPV6) |
| 700 | if (ipv6 || metadata) | 702 | if (ipv6) { |
| 701 | ret = geneve_sock_add(geneve, true); | 703 | ret = geneve_sock_add(geneve, true); |
| 704 | if (ret < 0 && ret != -EAFNOSUPPORT) | ||
| 705 | ipv4 = false; | ||
| 706 | } | ||
| 702 | #endif | 707 | #endif |
| 703 | if (!ret && (!ipv6 || metadata)) | 708 | if (ipv4) |
| 704 | ret = geneve_sock_add(geneve, false); | 709 | ret = geneve_sock_add(geneve, false); |
| 705 | if (ret < 0) | 710 | if (ret < 0) |
| 706 | geneve_sock_release(geneve); | 711 | geneve_sock_release(geneve); |
| @@ -744,6 +744,14 @@ void netvsc_linkstatus_callback(struct net_device *net, · 743 unmodified lines | |||
| 744 | schedule_delayed_work(&ndev_ctx->dwork, 0); | 744 | schedule_delayed_work(&ndev_ctx->dwork, 0); |
| 745 | } | 745 | } |
| 746 | 746 | ||
| 747 | static void netvsc_comp_ipcsum(struct sk_buff *skb) | ||
| 748 | { | ||
| 749 | struct iphdr *iph = (struct iphdr *)skb->data; | ||
| 750 | |||
| 751 | iph->check = 0; | ||
| 752 | iph->check = ip_fast_csum(iph, iph->ihl); | ||
| 753 | } | ||
| 754 | |||
| 747 | static struct sk_buff *netvsc_alloc_recv_skb(struct net_device *net, | 755 | static struct sk_buff *netvsc_alloc_recv_skb(struct net_device *net, |
| 748 | struct netvsc_channel *nvchan) | 756 | struct netvsc_channel *nvchan) |
| 749 | { | 757 | { |
| @@ -770,9 +778,17 @@ static struct sk_buff *netvsc_alloc_recv_skb(struct net_device *net, · 20 unmodified lines | |||
| 770 | /* skb is already created with CHECKSUM_NONE */ | 778 | /* skb is already created with CHECKSUM_NONE */ |
| 771 | skb_checksum_none_assert(skb); | 779 | skb_checksum_none_assert(skb); |
| 772 | 780 | ||
| 773 | /* | 781 | /* Incoming packets may have IP header checksum verified by the host. |
| 774 | * In Linux, the IP checksum is always checked. | 782 | * They may not have IP header checksum computed after coalescing. |
| 775 | * Do L4 checksum offload if enabled and present. | 783 | * We compute it here if the flags are set, because on Linux, the IP |
| 784 | * checksum is always checked. | ||
| 785 | */ | ||
| 786 | if (csum_info && csum_info->receive.ip_checksum_value_invalid && | ||
| 787 | csum_info->receive.ip_checksum_succeeded && | ||
| 788 | skb->protocol == htons(ETH_P_IP)) | ||
| 789 | netvsc_comp_ipcsum(skb); | ||
| 790 | |||
| 791 | /* Do L4 checksum offload if enabled and present. | ||
| 776 | */ | 792 | */ |
| 777 | if (csum_info && (net->features & NETIF_F_RXCSUM)) { | 793 | if (csum_info && (net->features & NETIF_F_RXCSUM)) { |
| 778 | if (csum_info->receive.tcp_checksum_succeeded || | 794 | if (csum_info->receive.tcp_checksum_succeeded || |
| @@ -499,6 +499,8 @@ static int ipvlan_nl_changelink(struct net_device *dev, · 498 unmodified lines | |||
| 499 | 499 | ||
| 500 | if (!data) | 500 | if (!data) |
| 501 | return 0; | 501 | return 0; |
| 502 | if (!ns_capable(dev_net(ipvlan->phy_dev)->user_ns, CAP_NET_ADMIN)) | ||
| 503 | return -EPERM; | ||
| 502 | 504 | ||
| 503 | if (data[IFLA_IPVLAN_MODE]) { | 505 | if (data[IFLA_IPVLAN_MODE]) { |
| 504 | u16 nmode = nla_get_u16(data[IFLA_IPVLAN_MODE]); | 506 | u16 nmode = nla_get_u16(data[IFLA_IPVLAN_MODE]); |
| @@ -601,6 +603,8 @@ int ipvlan_link_new(struct net *src_net, struct net_device *dev, · 96 unmodified lines | |||
| 601 | struct ipvl_dev *tmp = netdev_priv(phy_dev); | 603 | struct ipvl_dev *tmp = netdev_priv(phy_dev); |
| 602 | 604 | ||
| 603 | phy_dev = tmp->phy_dev; | 605 | phy_dev = tmp->phy_dev; |
| 606 | if (!ns_capable(dev_net(phy_dev)->user_ns, CAP_NET_ADMIN)) | ||
| 607 | return -EPERM; | ||
| 604 | } else if (!netif_is_ipvlan_port(phy_dev)) { | 608 | } else if (!netif_is_ipvlan_port(phy_dev)) { |
| 605 | /* Exit early if the underlying link is invalid or busy */ | 609 | /* Exit early if the underlying link is invalid or busy */ |
| 606 | if (phy_dev->type != ARPHRD_ETHER || | 610 | if (phy_dev->type != ARPHRD_ETHER || |
| @@ -19,6 +19,7 @@ · 18 unmodified lines | |||
| 19 | #include <linux/module.h> | 19 | #include <linux/module.h> |
| 20 | #include <linux/of.h> | 20 | #include <linux/of.h> |
| 21 | #include <linux/phy.h> | 21 | #include <linux/phy.h> |
| 22 | #include <linux/delay.h> | ||
| 22 | 23 | ||
| 23 | #include <dt-bindings/net/ti-dp83867.h> | 24 | #include <dt-bindings/net/ti-dp83867.h> |
| 24 | 25 | ||
| @@ -325,6 +326,8 @@ static int dp83867_phy_reset(struct phy_device *phydev) · 300 unmodified lines | |||
| 325 | if (err < 0) | 326 | if (err < 0) |
| 326 | return err; | 327 | return err; |
| 327 | 328 | ||
| 329 | usleep_range(10, 20); | ||
| 330 | |||
| 328 | return dp83867_config_init(phydev); | 331 | return dp83867_config_init(phydev); |
| 329 | } | 332 | } |
| 330 | 333 | ||
| @@ -26,6 +26,8 @@ · 25 unmodified lines | |||
| 26 | #include <linux/marvell_phy.h> | 26 | #include <linux/marvell_phy.h> |
| 27 | #include <linux/phy.h> | 27 | #include <linux/phy.h> |
| 28 | 28 | ||
| 29 | #define MDIO_AN_10GBT_CTRL_ADV_NBT_MASK 0x01e0 | ||
| 30 | |||
| 29 | enum { | 31 | enum { |
| 30 | MV_PCS_BASE_T = 0x0000, | 32 | MV_PCS_BASE_T = 0x0000, |
| 31 | MV_PCS_BASE_R = 0x1000, | 33 | MV_PCS_BASE_R = 0x1000, |
| @@ -386,8 +388,10 @@ static int mv3310_config_aneg(struct phy_device *phydev) · 354 unmodified lines | |||
| 386 | else | 388 | else |
| 387 | reg = 0; | 389 | reg = 0; |
| 388 | 390 | ||
| 391 | /* Make sure we clear unsupported 2.5G/5G advertising */ | ||
| 389 | ret = mv3310_modify(phydev, MDIO_MMD_AN, MDIO_AN_10GBT_CTRL, | 392 | ret = mv3310_modify(phydev, MDIO_MMD_AN, MDIO_AN_10GBT_CTRL, |
| 390 | MDIO_AN_10GBT_CTRL_ADV10G, reg); | 393 | MDIO_AN_10GBT_CTRL_ADV10G | |
| 394 | MDIO_AN_10GBT_CTRL_ADV_NBT_MASK, reg); | ||
| 391 | if (ret < 0) | 395 | if (ret < 0) |
| 392 | return ret; | 396 | return ret; |
| 393 | if (ret > 0) | 397 | if (ret > 0) |
| @@ -379,7 +379,6 @@ int __mdiobus_register(struct mii_bus *bus, struct module *owner) · 378 unmodified lines | |||
| 379 | err = device_register(&bus->dev); | 379 | err = device_register(&bus->dev); |
| 380 | if (err) { | 380 | if (err) { |
| 381 | pr_err("mii_bus %s failed to register\n", bus->id); | 381 | pr_err("mii_bus %s failed to register\n", bus->id); |
| 382 | put_device(&bus->dev); | ||
| 383 | return -EINVAL; | 382 | return -EINVAL; |
| 384 | } | 383 | } |
| 385 | 384 | ||
| @@ -344,6 +344,17 @@ static int ksz8041_config_aneg(struct phy_device *phydev) · 343 unmodified lines | |||
| 344 | return genphy_config_aneg(phydev); | 344 | return genphy_config_aneg(phydev); |
| 345 | } | 345 | } |
| 346 | 346 | ||
| 347 | static int ksz8061_config_init(struct phy_device *phydev) | ||
| 348 | { | ||
| 349 | int ret; | ||
| 350 | |||
| 351 | ret = phy_write_mmd(phydev, MDIO_MMD_PMAPMD, MDIO_DEVID1, 0xB61A); | ||
| 352 | if (ret) | ||
| 353 | return ret; | ||
| 354 | |||
| 355 | return kszphy_config_init(phydev); | ||
| 356 | } | ||
| 357 | |||
| 347 | static int ksz9021_load_values_from_of(struct phy_device *phydev, | 358 | static int ksz9021_load_values_from_of(struct phy_device *phydev, |
| 348 | const struct device_node *of_node, | 359 | const struct device_node *of_node, |
| 349 | u16 reg, | 360 | u16 reg, |
| @@ -1040,7 +1051,7 @@ static struct phy_driver ksphy_driver[] = { · 690 unmodified lines | |||
| 1040 | .name = "Micrel KSZ8061", | 1051 | .name = "Micrel KSZ8061", |
| 1041 | .phy_id_mask = MICREL_PHY_ID_MASK, | 1052 | .phy_id_mask = MICREL_PHY_ID_MASK, |
| 1042 | .features = PHY_BASIC_FEATURES, | 1053 | .features = PHY_BASIC_FEATURES, |
| 1043 | .config_init = kszphy_config_init, | 1054 | .config_init = ksz8061_config_init, |
| 1044 | .ack_interrupt = kszphy_ack_interrupt, | 1055 | .ack_interrupt = kszphy_ack_interrupt, |
| 1045 | .config_intr = kszphy_config_intr, | 1056 | .config_intr = kszphy_config_intr, |
| 1046 | .suspend = genphy_suspend, | 1057 | .suspend = genphy_suspend, |
| @@ -320,6 +320,10 @@ static int phylink_get_mac_state(struct phylink *pl, struct phylink_link_state * · 319 unmodified lines | |||
| 320 | linkmode_zero(state->lp_advertising); | 320 | linkmode_zero(state->lp_advertising); |
| 321 | state->interface = pl->link_config.interface; | 321 | state->interface = pl->link_config.interface; |
| 322 | state->an_enabled = pl->link_config.an_enabled; | 322 | state->an_enabled = pl->link_config.an_enabled; |
| 323 | state->speed = SPEED_UNKNOWN; | ||
| 324 | state->duplex = DUPLEX_UNKNOWN; | ||
| 325 | state->pause = MLO_PAUSE_NONE; | ||
| 326 | state->an_complete = 0; | ||
| 323 | state->link = 1; | 327 | state->link = 1; |
| 324 | 328 | ||
| 325 | return pl->ops->mac_link_state(ndev, state); | 329 | return pl->ops->mac_link_state(ndev, state); |
| @@ -282,6 +282,13 @@ static struct phy_driver realtek_drvs[] = { · 281 unmodified lines | |||
| 282 | .name = "RTL8366RB Gigabit Ethernet", | 282 | .name = "RTL8366RB Gigabit Ethernet", |
| 283 | .features = PHY_GBIT_FEATURES, | 283 | .features = PHY_GBIT_FEATURES, |
| 284 | .config_init = &rtl8366rb_config_init, | 284 | .config_init = &rtl8366rb_config_init, |
| 285 | /* These interrupts are handled by the irq controller | ||
| 286 | * embedded inside the RTL8366RB, they get unmasked when the | ||
| 287 | * irq is requested and ACKed by reading the status register, | ||
| 288 | * which is done by the irqchip code. | ||
| 289 | */ | ||
| 290 | .ack_interrupt = genphy_no_ack_interrupt, | ||
| 291 | .config_intr = genphy_no_config_intr, | ||
| 285 | .suspend = genphy_suspend, | 292 | .suspend = genphy_suspend, |
| 286 | .resume = genphy_resume, | 293 | .resume = genphy_resume, |
| 287 | }, | 294 | }, |
| @@ -44,7 +44,10 @@ static int xgmiitorgmii_read_status(struct phy_device *phydev) · 43 unmodified lines | |||
| 44 | u16 val = 0; | 44 | u16 val = 0; |
| 45 | int err; | 45 | int err; |
| 46 | 46 | ||
| 47 | err = priv->phy_drv->read_status(phydev); | 47 | if (priv->phy_drv->read_status) |
| 48 | err = priv->phy_drv->read_status(phydev); | ||
| 49 | else | ||
| 50 | err = genphy_read_status(phydev); | ||
| 48 | if (err < 0) | 51 | if (err < 0) |
| 49 | return err; | 52 | return err; |
| 50 | 53 | ||
| @@ -1256,7 +1256,7 @@ static int team_port_add(struct team *team, struct net_device *port_dev, · 1255 unmodified lines | |||
| 1256 | list_add_tail_rcu(&port->list, &team->port_list); | 1256 | list_add_tail_rcu(&port->list, &team->port_list); |
| 1257 | team_port_enable(team, port); | 1257 | team_port_enable(team, port); |
| 1258 | __team_compute_features(team); | 1258 | __team_compute_features(team); |
| 1259 | __team_port_change_port_added(port, !!netif_carrier_ok(port_dev)); | 1259 | __team_port_change_port_added(port, !!netif_oper_up(port_dev)); |
| 1260 | __team_options_change_check(team); | 1260 | __team_options_change_check(team); |
| 1261 | 1261 | ||
| 1262 | netdev_info(dev, "Port device %s added\n", portname); | 1262 | netdev_info(dev, "Port device %s added\n", portname); |
| @@ -2915,7 +2915,7 @@ static int team_device_event(struct notifier_block *unused, · 1652 unmodified lines | |||
| 2915 | 2915 | ||
| 2916 | switch (event) { | 2916 | switch (event) { |
| 2917 | case NETDEV_UP: | 2917 | case NETDEV_UP: |
| 2918 | if (netif_carrier_ok(dev)) | 2918 | if (netif_oper_up(dev)) |
| 2919 | team_port_change_check(port, true); | 2919 | team_port_change_check(port, true); |
| 2920 | break; | 2920 | break; |
| 2921 | case NETDEV_DOWN: | 2921 | case NETDEV_DOWN: |
| @@ -2167,9 +2167,9 @@ static void *tun_ring_recv(struct tun_file *tfile, int noblock, int *err) · 2166 unmodified lines | |||
| 2167 | } | 2167 | } |
| 2168 | 2168 | ||
| 2169 | add_wait_queue(&tfile->wq.wait, &wait); | 2169 | add_wait_queue(&tfile->wq.wait, &wait); |
| 2170 | current->state = TASK_INTERRUPTIBLE; | ||
| 2171 | 2170 | ||
| 2172 | while (1) { | 2171 | while (1) { |
| 2172 | set_current_state(TASK_INTERRUPTIBLE); | ||
| 2173 | ptr = ptr_ring_consume(&tfile->tx_ring); | 2173 | ptr = ptr_ring_consume(&tfile->tx_ring); |
| 2174 | if (ptr) | 2174 | if (ptr) |
| 2175 | break; | 2175 | break; |
| @@ -2185,7 +2185,7 @@ static void *tun_ring_recv(struct tun_file *tfile, int noblock, int *err) · 9 unmodified lines | |||
| 2185 | schedule(); | 2185 | schedule(); |
| 2186 | } | 2186 | } |
| 2187 | 2187 | ||
| 2188 | current->state = TASK_RUNNING; | 2188 | __set_current_state(TASK_RUNNING); |
| 2189 | remove_wait_queue(&tfile->wq.wait, &wait); | 2189 | remove_wait_queue(&tfile->wq.wait, &wait); |
| 2190 | 2190 | ||
| 2191 | out: | 2191 | out: |
| @@ -1201,8 +1201,8 @@ static const struct usb_device_id products[] = { · 1200 unmodified lines | |||
| 1201 | {QMI_FIXED_INTF(0x114f, 0x68a2, 8)}, /* Sierra Wireless MC7750 */ | 1201 | {QMI_FIXED_INTF(0x114f, 0x68a2, 8)}, /* Sierra Wireless MC7750 */ |
| 1202 | {QMI_FIXED_INTF(0x1199, 0x68a2, 8)}, /* Sierra Wireless MC7710 in QMI mode */ | 1202 | {QMI_FIXED_INTF(0x1199, 0x68a2, 8)}, /* Sierra Wireless MC7710 in QMI mode */ |
| 1203 | {QMI_FIXED_INTF(0x1199, 0x68a2, 19)}, /* Sierra Wireless MC7710 in QMI mode */ | 1203 | {QMI_FIXED_INTF(0x1199, 0x68a2, 19)}, /* Sierra Wireless MC7710 in QMI mode */ |
| 1204 | {QMI_FIXED_INTF(0x1199, 0x68c0, 8)}, /* Sierra Wireless MC7304/MC7354 */ | 1204 | {QMI_QUIRK_SET_DTR(0x1199, 0x68c0, 8)}, /* Sierra Wireless MC7304/MC7354, WP76xx */ |
| 1205 | {QMI_FIXED_INTF(0x1199, 0x68c0, 10)}, /* Sierra Wireless MC7304/MC7354 */ | 1205 | {QMI_QUIRK_SET_DTR(0x1199, 0x68c0, 10)},/* Sierra Wireless MC7304/MC7354 */ |
| 1206 | {QMI_FIXED_INTF(0x1199, 0x901c, 8)}, /* Sierra Wireless EM7700 */ | 1206 | {QMI_FIXED_INTF(0x1199, 0x901c, 8)}, /* Sierra Wireless EM7700 */ |
| 1207 | {QMI_FIXED_INTF(0x1199, 0x901f, 8)}, /* Sierra Wireless EM7355 */ | 1207 | {QMI_FIXED_INTF(0x1199, 0x901f, 8)}, /* Sierra Wireless EM7355 */ |
| 1208 | {QMI_FIXED_INTF(0x1199, 0x9041, 8)}, /* Sierra Wireless MC7305/MC7355 */ | 1208 | {QMI_FIXED_INTF(0x1199, 0x9041, 8)}, /* Sierra Wireless MC7305/MC7355 */ |
| @@ -557,6 +557,7 @@ enum spd_duplex { · 556 unmodified lines | |||
| 557 | /* MAC PASSTHRU */ | 557 | /* MAC PASSTHRU */ |
| 558 | #define AD_MASK 0xfee0 | 558 | #define AD_MASK 0xfee0 |
| 559 | #define BND_MASK 0x0004 | 559 | #define BND_MASK 0x0004 |
| 560 | #define BD_MASK 0x0001 | ||
| 560 | #define EFUSE 0xcfdb | 561 | #define EFUSE 0xcfdb |
| 561 | #define PASS_THRU_MASK 0x1 | 562 | #define PASS_THRU_MASK 0x1 |
| 562 | 563 | ||
| @@ -1176,9 +1177,9 @@ static int vendor_mac_passthru_addr_read(struct r8152 *tp, struct sockaddr *sa) · 613 unmodified lines | |||
| 1176 | return -ENODEV; | 1177 | return -ENODEV; |
| 1177 | } | 1178 | } |
| 1178 | } else { | 1179 | } else { |
| 1179 | /* test for RTL8153-BND */ | 1180 | /* test for RTL8153-BND and RTL8153-BD */ |
| 1180 | ocp_data = ocp_read_byte(tp, MCU_TYPE_USB, USB_MISC_1); | 1181 | ocp_data = ocp_read_byte(tp, MCU_TYPE_USB, USB_MISC_1); |
| 1181 | if ((ocp_data & BND_MASK) == 0) { | 1182 | if ((ocp_data & BND_MASK) == 0 && (ocp_data & BD_MASK) == 0) { |
| 1182 | netif_dbg(tp, probe, tp->netdev, | 1183 | netif_dbg(tp, probe, tp->netdev, |
| 1183 | "Invalid variant for MAC pass through\n"); | 1184 | "Invalid variant for MAC pass through\n"); |
| 1184 | return -ENODEV; | 1185 | return -ENODEV; |
| @@ -1273,6 +1273,9 @@ static void vrf_setup(struct net_device *dev) · 1272 unmodified lines | |||
| 1273 | 1273 | ||
| 1274 | /* default to no qdisc; user can add if desired */ | 1274 | /* default to no qdisc; user can add if desired */ |
| 1275 | dev->priv_flags |= IFF_NO_QUEUE; | 1275 | dev->priv_flags |= IFF_NO_QUEUE; |
| 1276 | |||
| 1277 | dev->min_mtu = 0; | ||
| 1278 | dev->max_mtu = 0; | ||
| 1276 | } | 1279 | } |
| 1277 | 1280 | ||
| 1278 | static int vrf_validate(struct nlattr *tb[], struct nlattr *data[], | 1281 | static int vrf_validate(struct nlattr *tb[], struct nlattr *data[], |
| @@ -3554,7 +3554,7 @@ static int hwsim_get_radio_nl(struct sk_buff *msg, struct genl_info *info) · 3553 unmodified lines | |||
| 3554 | goto out_err; | 3554 | goto out_err; |
| 3555 | } | 3555 | } |
| 3556 | 3556 | ||
| 3557 | genlmsg_reply(skb, info); | 3557 | res = genlmsg_reply(skb, info); |
| 3558 | break; | 3558 | break; |
| 3559 | } | 3559 | } |
| 3560 | 3560 | ||
| @@ -158,39 +158,49 @@ static const struct ieee80211_ops mt76x0u_ops = { · 157 unmodified lines | |||
| 158 | .get_txpower = mt76x02_get_txpower, | 158 | .get_txpower = mt76x02_get_txpower, |
| 159 | }; | 159 | }; |
| 160 | 160 | ||
| 161 | static int mt76x0u_register_device(struct mt76x02_dev *dev) | 161 | static int mt76x0u_init_hardware(struct mt76x02_dev *dev) |
| 162 | { | 162 | { |
| 163 | struct ieee80211_hw *hw = dev->mt76.hw; | ||
| 164 | int err; | 163 | int err; |
| 165 | 164 | ||
| 166 | err = mt76u_alloc_queues(&dev->mt76); | ||
| 167 | if (err < 0) | ||
| 168 | goto out_err; | ||
| 169 | |||
| 170 | err = mt76u_mcu_init_rx(&dev->mt76); | ||
| 171 | if (err < 0) | ||
| 172 | goto out_err; | ||
| 173 | |||
| 174 | mt76x0_chip_onoff(dev, true, true); | 165 | mt76x0_chip_onoff(dev, true, true); |
| 175 | if (!mt76x02_wait_for_mac(&dev->mt76)) { | 166 | |
| 176 | err = -ETIMEDOUT; | 167 | if (!mt76x02_wait_for_mac(&dev->mt76)) |
| 177 | goto out_err; | 168 | return -ETIMEDOUT; |
| 178 | } | ||
| 179 | 169 | ||
| 180 | err = mt76x0u_mcu_init(dev); | 170 | err = mt76x0u_mcu_init(dev); |
| 181 | if (err < 0) | 171 | if (err < 0) |
| 182 | goto out_err; | 172 | return err; |
| 183 | 173 | ||
| 184 | mt76x0_init_usb_dma(dev); | 174 | mt76x0_init_usb_dma(dev); |
| 185 | err = mt76x0_init_hardware(dev); | 175 | err = mt76x0_init_hardware(dev); |
| 186 | if (err < 0) | 176 | if (err < 0) |
| 187 | goto out_err; | 177 | return err; |
| 188 | 178 | ||
| 189 | mt76_rmw(dev, MT_US_CYC_CFG, MT_US_CYC_CNT, 0x1e); | 179 | mt76_rmw(dev, MT_US_CYC_CFG, MT_US_CYC_CNT, 0x1e); |
| 190 | mt76_wr(dev, MT_TXOP_CTRL_CFG, | 180 | mt76_wr(dev, MT_TXOP_CTRL_CFG, |
| 191 | FIELD_PREP(MT_TXOP_TRUN_EN, 0x3f) | | 181 | FIELD_PREP(MT_TXOP_TRUN_EN, 0x3f) | |
| 192 | FIELD_PREP(MT_TXOP_EXT_CCA_DLY, 0x58)); | 182 | FIELD_PREP(MT_TXOP_EXT_CCA_DLY, 0x58)); |
| 193 | 183 | ||
| 184 | return 0; | ||
| 185 | } | ||
| 186 | |||
| 187 | static int mt76x0u_register_device(struct mt76x02_dev *dev) | ||
| 188 | { | ||
| 189 | struct ieee80211_hw *hw = dev->mt76.hw; | ||
| 190 | int err; | ||
| 191 | |||
| 192 | err = mt76u_alloc_queues(&dev->mt76); | ||
| 193 | if (err < 0) | ||
| 194 | goto out_err; | ||
| 195 | |||
| 196 | err = mt76u_mcu_init_rx(&dev->mt76); | ||
| 197 | if (err < 0) | ||
| 198 | goto out_err; | ||
| 199 | |||
| 200 | err = mt76x0u_init_hardware(dev); | ||
| 201 | if (err < 0) | ||
| 202 | goto out_err; | ||
| 203 | |||
| 194 | err = mt76x0_register_device(dev); | 204 | err = mt76x0_register_device(dev); |
| 195 | if (err < 0) | 205 | if (err < 0) |
| 196 | goto out_err; | 206 | goto out_err; |
| @@ -301,6 +311,8 @@ static int __maybe_unused mt76x0_suspend(struct usb_interface *usb_intf, · 104 unmodified lines | |||
| 301 | 311 | ||
| 302 | mt76u_stop_queues(&dev->mt76); | 312 | mt76u_stop_queues(&dev->mt76); |
| 303 | mt76x0u_mac_stop(dev); | 313 | mt76x0u_mac_stop(dev); |
| 314 | clear_bit(MT76_STATE_MCU_RUNNING, &dev->mt76.state); | ||
| 315 | mt76x0_chip_onoff(dev, false, false); | ||
| 304 | usb_kill_urb(usb->mcu.res.urb); | 316 | usb_kill_urb(usb->mcu.res.urb); |
| 305 | 317 | ||
| 306 | return 0; | 318 | return 0; |
| @@ -328,7 +340,7 @@ static int __maybe_unused mt76x0_resume(struct usb_interface *usb_intf) · 21 unmodified lines | |||
| 328 | tasklet_enable(&usb->rx_tasklet); | 340 | tasklet_enable(&usb->rx_tasklet); |
| 329 | tasklet_enable(&usb->tx_tasklet); | 341 | tasklet_enable(&usb->tx_tasklet); |
| 330 | 342 | ||
| 331 | ret = mt76x0_init_hardware(dev); | 343 | ret = mt76x0u_init_hardware(dev); |
| 332 | if (ret) | 344 | if (ret) |
| 333 | goto err; | 345 | goto err; |
| 334 | 346 | ||
| @@ -454,6 +454,8 @@ void xenvif_init_hash(struct xenvif *vif) · 453 unmodified lines | |||
| 454 | if (xenvif_hash_cache_size == 0) | 454 | if (xenvif_hash_cache_size == 0) |
| 455 | return; | 455 | return; |
| 456 | 456 | ||
| 457 | BUG_ON(vif->hash.cache.count); | ||
| 458 | |||
| 457 | spin_lock_init(&vif->hash.cache.lock); | 459 | spin_lock_init(&vif->hash.cache.lock); |
| 458 | INIT_LIST_HEAD(&vif->hash.cache.list); | 460 | INIT_LIST_HEAD(&vif->hash.cache.list); |
| 459 | } | 461 | } |
| @@ -153,6 +153,13 @@ static u16 xenvif_select_queue(struct net_device *dev, struct sk_buff *skb, · 152 unmodified lines | |||
| 153 | { | 153 | { |
| 154 | struct xenvif *vif = netdev_priv(dev); | 154 | struct xenvif *vif = netdev_priv(dev); |
| 155 | unsigned int size = vif->hash.size; | 155 | unsigned int size = vif->hash.size; |
| 156 | unsigned int num_queues; | ||
| 157 | |||
| 158 | /* If queues are not set up internally - always return 0 | ||
| 159 | * as the packet going to be dropped anyway */ | ||
| 160 | num_queues = READ_ONCE(vif->num_queues); | ||
| 161 | if (num_queues < 1) | ||
| 162 | return 0; | ||
| 156 | 163 | ||
| 157 | if (vif->hash.alg == XEN_NETIF_CTRL_HASH_ALGORITHM_NONE) | 164 | if (vif->hash.alg == XEN_NETIF_CTRL_HASH_ALGORITHM_NONE) |
| 158 | return fallback(dev, skb, NULL) % dev->real_num_tx_queues; | 165 | return fallback(dev, skb, NULL) % dev->real_num_tx_queues; |
| @@ -1072,11 +1072,6 @@ static int xenvif_handle_frag_list(struct xenvif_queue *queue, struct sk_buff *s · 1071 unmodified lines | |||
| 1072 | skb_frag_size_set(&frags[i], len); | 1072 | skb_frag_size_set(&frags[i], len); |
| 1073 | } | 1073 | } |
| 1074 | 1074 | ||
| 1075 | /* Copied all the bits from the frag list -- free it. */ | ||
| 1076 | skb_frag_list_init(skb); | ||
| 1077 | xenvif_skb_zerocopy_prepare(queue, nskb); | ||
| 1078 | kfree_skb(nskb); | ||
| 1079 | |||
| 1080 | /* Release all the original (foreign) frags. */ | 1075 | /* Release all the original (foreign) frags. */ |
| 1081 | for (f = 0; f < skb_shinfo(skb)->nr_frags; f++) | 1076 | for (f = 0; f < skb_shinfo(skb)->nr_frags; f++) |
| 1082 | skb_frag_unref(skb, f); | 1077 | skb_frag_unref(skb, f); |
| @@ -1145,6 +1140,8 @@ static int xenvif_tx_submit(struct xenvif_queue *queue) · 62 unmodified lines | |||
| 1145 | xenvif_fill_frags(queue, skb); | 1140 | xenvif_fill_frags(queue, skb); |
| 1146 | 1141 | ||
| 1147 | if (unlikely(skb_has_frag_list(skb))) { | 1142 | if (unlikely(skb_has_frag_list(skb))) { |
| 1143 | struct sk_buff *nskb = skb_shinfo(skb)->frag_list; | ||
| 1144 | xenvif_skb_zerocopy_prepare(queue, nskb); | ||
| 1148 | if (xenvif_handle_frag_list(queue, skb)) { | 1145 | if (xenvif_handle_frag_list(queue, skb)) { |
| 1149 | if (net_ratelimit()) | 1146 | if (net_ratelimit()) |
| 1150 | netdev_err(queue->vif->dev, | 1147 | netdev_err(queue->vif->dev, |
| @@ -1153,6 +1150,9 @@ static int xenvif_tx_submit(struct xenvif_queue *queue) · 2 unmodified lines | |||
| 1153 | kfree_skb(skb); | 1150 | kfree_skb(skb); |
| 1154 | continue; | 1151 | continue; |
| 1155 | } | 1152 | } |
| 1153 | /* Copied all the bits from the frag list -- free it. */ | ||
| 1154 | skb_frag_list_init(skb); | ||
| 1155 | kfree_skb(nskb); | ||
| 1156 | } | 1156 | } |
| 1157 | 1157 | ||
| 1158 | skb->dev = queue->vif->dev; | 1158 | skb->dev = queue->vif->dev; |
| @@ -693,7 +693,7 @@ static const char * const sd_a_groups[] = { · 692 unmodified lines | |||
| 693 | 693 | ||
| 694 | static const char * const sdxc_a_groups[] = { | 694 | static const char * const sdxc_a_groups[] = { |
| 695 | "sdxc_d0_0_a", "sdxc_d13_0_a", "sdxc_d47_a", "sdxc_clk_a", | 695 | "sdxc_d0_0_a", "sdxc_d13_0_a", "sdxc_d47_a", "sdxc_clk_a", |
| 696 | "sdxc_cmd_a", "sdxc_d0_1_a", "sdxc_d0_13_1_a" | 696 | "sdxc_cmd_a", "sdxc_d0_1_a", "sdxc_d13_1_a" |
| 697 | }; | 697 | }; |
| 698 | 698 | ||
| 699 | static const char * const pcm_a_groups[] = { | 699 | static const char * const pcm_a_groups[] = { |
| @@ -79,7 +79,7 @@ enum { · 78 unmodified lines | |||
| 79 | .intr_cfg_reg = 0, \ | 79 | .intr_cfg_reg = 0, \ |
| 80 | .intr_status_reg = 0, \ | 80 | .intr_status_reg = 0, \ |
| 81 | .intr_target_reg = 0, \ | 81 | .intr_target_reg = 0, \ |
| 82 | .tile = NORTH, \ | 82 | .tile = SOUTH, \ |
| 83 | .mux_bit = -1, \ | 83 | .mux_bit = -1, \ |
| 84 | .pull_bit = pull, \ | 84 | .pull_bit = pull, \ |
| 85 | .drv_bit = drv, \ | 85 | .drv_bit = drv, \ |
| @@ -2009,7 +2009,7 @@ static int twa_probe(struct pci_dev *pdev, const struct pci_device_id *dev_id) · 2008 unmodified lines | |||
| 2009 | struct Scsi_Host *host = NULL; | 2009 | struct Scsi_Host *host = NULL; |
| 2010 | TW_Device_Extension *tw_dev; | 2010 | TW_Device_Extension *tw_dev; |
| 2011 | unsigned long mem_addr, mem_len; | 2011 | unsigned long mem_addr, mem_len; |
| 2012 | int retval = -ENODEV; | 2012 | int retval; |
| 2013 | 2013 | ||
| 2014 | retval = pci_enable_device(pdev); | 2014 | retval = pci_enable_device(pdev); |
| 2015 | if (retval) { | 2015 | if (retval) { |
| @@ -2020,8 +2020,10 @@ static int twa_probe(struct pci_dev *pdev, const struct pci_device_id *dev_id) · 4 unmodified lines | |||
| 2020 | pci_set_master(pdev); | 2020 | pci_set_master(pdev); |
| 2021 | pci_try_set_mwi(pdev); | 2021 | pci_try_set_mwi(pdev); |
| 2022 | 2022 | ||
| 2023 | if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | 2023 | retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); |
| 2024 | dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) { | 2024 | if (retval) |
| 2025 | retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | ||
| 2026 | if (retval) { | ||
| 2025 | TW_PRINTK(host, TW_DRIVER, 0x23, "Failed to set dma mask"); | 2027 | TW_PRINTK(host, TW_DRIVER, 0x23, "Failed to set dma mask"); |
| 2026 | retval = -ENODEV; | 2028 | retval = -ENODEV; |
| 2027 | goto out_disable_device; | 2029 | goto out_disable_device; |
| @@ -2240,8 +2242,10 @@ static int twa_resume(struct pci_dev *pdev) · 212 unmodified lines | |||
| 2240 | pci_set_master(pdev); | 2242 | pci_set_master(pdev); |
| 2241 | pci_try_set_mwi(pdev); | 2243 | pci_try_set_mwi(pdev); |
| 2242 | 2244 | ||
| 2243 | if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | 2245 | retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); |
| 2244 | dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) { | 2246 | if (retval) |
| 2247 | retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | ||
| 2248 | if (retval) { | ||
| 2245 | TW_PRINTK(host, TW_DRIVER, 0x40, "Failed to set dma mask during resume"); | 2249 | TW_PRINTK(host, TW_DRIVER, 0x40, "Failed to set dma mask during resume"); |
| 2246 | retval = -ENODEV; | 2250 | retval = -ENODEV; |
| 2247 | goto out_disable_device; | 2251 | goto out_disable_device; |
| @@ -1573,8 +1573,10 @@ static int twl_probe(struct pci_dev *pdev, const struct pci_device_id *dev_id) · 1572 unmodified lines | |||
| 1573 | pci_set_master(pdev); | 1573 | pci_set_master(pdev); |
| 1574 | pci_try_set_mwi(pdev); | 1574 | pci_try_set_mwi(pdev); |
| 1575 | 1575 | ||
| 1576 | if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | 1576 | retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); |
| 1577 | dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) { | 1577 | if (retval) |
| 1578 | retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | ||
| 1579 | if (retval) { | ||
| 1578 | TW_PRINTK(host, TW_DRIVER, 0x18, "Failed to set dma mask"); | 1580 | TW_PRINTK(host, TW_DRIVER, 0x18, "Failed to set dma mask"); |
| 1579 | retval = -ENODEV; | 1581 | retval = -ENODEV; |
| 1580 | goto out_disable_device; | 1582 | goto out_disable_device; |
| @@ -1805,8 +1807,10 @@ static int twl_resume(struct pci_dev *pdev) · 224 unmodified lines | |||
| 1805 | pci_set_master(pdev); | 1807 | pci_set_master(pdev); |
| 1806 | pci_try_set_mwi(pdev); | 1808 | pci_try_set_mwi(pdev); |
| 1807 | 1809 | ||
| 1808 | if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | 1810 | retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); |
| 1809 | dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) { | 1811 | if (retval) |
| 1812 | retval = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | ||
| 1813 | if (retval) { | ||
| 1810 | TW_PRINTK(host, TW_DRIVER, 0x25, "Failed to set dma mask during resume"); | 1814 | TW_PRINTK(host, TW_DRIVER, 0x25, "Failed to set dma mask during resume"); |
| 1811 | retval = -ENODEV; | 1815 | retval = -ENODEV; |
| 1812 | goto out_disable_device; | 1816 | goto out_disable_device; |
| @@ -769,9 +769,11 @@ static int asd_pci_probe(struct pci_dev *dev, const struct pci_device_id *id) · 768 unmodified lines | |||
| 769 | if (err) | 769 | if (err) |
| 770 | goto Err_remove; | 770 | goto Err_remove; |
| 771 | 771 | ||
| 772 | err = -ENODEV; | 772 | err = dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(64)); |
| 773 | if (dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(64)) || | 773 | if (err) |
| 774 | dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(32))) { | 774 | err = dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(32)); |
| 775 | if (err) { | ||
| 776 | err = -ENODEV; | ||
| 775 | asd_printk("no suitable DMA mask for %s\n", pci_name(dev)); | 777 | asd_printk("no suitable DMA mask for %s\n", pci_name(dev)); |
| 776 | goto Err_remove; | 778 | goto Err_remove; |
| 777 | } | 779 | } |
| @@ -727,7 +727,7 @@ bfad_init_timer(struct bfad_s *bfad) · 726 unmodified lines | |||
| 727 | int | 727 | int |
| 728 | bfad_pci_init(struct pci_dev *pdev, struct bfad_s *bfad) | 728 | bfad_pci_init(struct pci_dev *pdev, struct bfad_s *bfad) |
| 729 | { | 729 | { |
| 730 | int rc = -ENODEV; | 730 | int rc = -ENODEV; |
| 731 | 731 | ||
| 732 | if (pci_enable_device(pdev)) { | 732 | if (pci_enable_device(pdev)) { |
| 733 | printk(KERN_ERR "pci_enable_device fail %p\n", pdev); | 733 | printk(KERN_ERR "pci_enable_device fail %p\n", pdev); |
| @@ -739,8 +739,12 @@ bfad_pci_init(struct pci_dev *pdev, struct bfad_s *bfad) · 5 unmodified lines | |||
| 739 | 739 | ||
| 740 | pci_set_master(pdev); | 740 | pci_set_master(pdev); |
| 741 | 741 | ||
| 742 | if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | 742 | rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); |
| 743 | dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) { | 743 | if (rc) |
| 744 | rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | ||
| 745 | |||
| 746 | if (rc) { | ||
| 747 | rc = -ENODEV; | ||
| 744 | printk(KERN_ERR "dma_set_mask_and_coherent fail %p\n", pdev); | 748 | printk(KERN_ERR "dma_set_mask_and_coherent fail %p\n", pdev); |
| 745 | goto out_release_region; | 749 | goto out_release_region; |
| 746 | } | 750 | } |
| @@ -1534,6 +1538,7 @@ bfad_pci_slot_reset(struct pci_dev *pdev) · 787 unmodified lines | |||
| 1534 | { | 1538 | { |
| 1535 | struct bfad_s *bfad = pci_get_drvdata(pdev); | 1539 | struct bfad_s *bfad = pci_get_drvdata(pdev); |
| 1536 | u8 byte; | 1540 | u8 byte; |
| 1541 | int rc; | ||
| 1537 | 1542 | ||
| 1538 | dev_printk(KERN_ERR, &pdev->dev, | 1543 | dev_printk(KERN_ERR, &pdev->dev, |
| 1539 | "bfad_pci_slot_reset flags: 0x%x\n", bfad->bfad_flags); | 1544 | "bfad_pci_slot_reset flags: 0x%x\n", bfad->bfad_flags); |
| @@ -1561,8 +1566,11 @@ bfad_pci_slot_reset(struct pci_dev *pdev) · 21 unmodified lines | |||
| 1561 | pci_save_state(pdev); | 1566 | pci_save_state(pdev); |
| 1562 | pci_set_master(pdev); | 1567 | pci_set_master(pdev); |
| 1563 | 1568 | ||
| 1564 | if (dma_set_mask_and_coherent(&bfad->pcidev->dev, DMA_BIT_MASK(64)) || | 1569 | rc = dma_set_mask_and_coherent(&bfad->pcidev->dev, DMA_BIT_MASK(64)); |
| 1565 | dma_set_mask_and_coherent(&bfad->pcidev->dev, DMA_BIT_MASK(32))) | 1570 | if (rc) |
| 1571 | rc = dma_set_mask_and_coherent(&bfad->pcidev->dev, | ||
| 1572 | DMA_BIT_MASK(32)); | ||
| 1573 | if (rc) | ||
| 1566 | goto out_disable_device; | 1574 | goto out_disable_device; |
| 1567 | 1575 | ||
| 1568 | if (restart_bfa(bfad) == -1) | 1576 | if (restart_bfa(bfad) == -1) |
| @@ -210,8 +210,11 @@ csio_pci_init(struct pci_dev *pdev, int *bars) · 209 unmodified lines | |||
| 210 | pci_set_master(pdev); | 210 | pci_set_master(pdev); |
| 211 | pci_try_set_mwi(pdev); | 211 | pci_try_set_mwi(pdev); |
| 212 | 212 | ||
| 213 | if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | 213 | rv = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); |
| 214 | dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) { | 214 | if (rv) |
| 215 | rv = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | ||
| 216 | if (rv) { | ||
| 217 | rv = -ENODEV; | ||
| 215 | dev_err(&pdev->dev, "No suitable DMA available.\n"); | 218 | dev_err(&pdev->dev, "No suitable DMA available.\n"); |
| 216 | goto err_release_regions; | 219 | goto err_release_regions; |
| 217 | } | 220 | } |
| @@ -2323,6 +2323,7 @@ static struct Scsi_Host *hisi_sas_shost_alloc(struct platform_device *pdev, · 2322 unmodified lines | |||
| 2323 | struct Scsi_Host *shost; | 2323 | struct Scsi_Host *shost; |
| 2324 | struct hisi_hba *hisi_hba; | 2324 | struct hisi_hba *hisi_hba; |
| 2325 | struct device *dev = &pdev->dev; | 2325 | struct device *dev = &pdev->dev; |
| 2326 | int error; | ||
| 2326 | 2327 | ||
| 2327 | shost = scsi_host_alloc(hw->sht, sizeof(*hisi_hba)); | 2328 | shost = scsi_host_alloc(hw->sht, sizeof(*hisi_hba)); |
| 2328 | if (!shost) { | 2329 | if (!shost) { |
| @@ -2343,8 +2344,11 @@ static struct Scsi_Host *hisi_sas_shost_alloc(struct platform_device *pdev, · 14 unmodified lines | |||
| 2343 | if (hisi_sas_get_fw_info(hisi_hba) < 0) | 2344 | if (hisi_sas_get_fw_info(hisi_hba) < 0) |
| 2344 | goto err_out; | 2345 | goto err_out; |
| 2345 | 2346 | ||
| 2346 | if (dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64)) && | 2347 | error = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64)); |
| 2347 | dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32))) { | 2348 | if (error) |
| 2349 | error = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32)); | ||
| 2350 | |||
| 2351 | if (error) { | ||
| 2348 | dev_err(dev, "No usable DMA addressing method\n"); | 2352 | dev_err(dev, "No usable DMA addressing method\n"); |
| 2349 | goto err_out; | 2353 | goto err_out; |
| 2350 | } | 2354 | } |
| @@ -2447,10 +2447,12 @@ hisi_sas_v3_probe(struct pci_dev *pdev, const struct pci_device_id *id) · 2446 unmodified lines | |||
| 2447 | if (rc) | 2447 | if (rc) |
| 2448 | goto err_out_disable_device; | 2448 | goto err_out_disable_device; |
| 2449 | 2449 | ||
| 2450 | if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | 2450 | rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); |
| 2451 | dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) { | 2451 | if (rc) |
| 2452 | rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | ||
| 2453 | if (rc) { | ||
| 2452 | dev_err(dev, "No usable DMA addressing method\n"); | 2454 | dev_err(dev, "No usable DMA addressing method\n"); |
| 2453 | rc = -EIO; | 2455 | rc = -ENODEV; |
| 2454 | goto err_out_regions; | 2456 | goto err_out_regions; |
| 2455 | } | 2457 | } |
| 2456 | 2458 | ||
| @@ -1292,6 +1292,7 @@ static int hptiop_probe(struct pci_dev *pcidev, const struct pci_device_id *id) · 1291 unmodified lines | |||
| 1292 | dma_addr_t start_phy; | 1292 | dma_addr_t start_phy; |
| 1293 | void *start_virt; | 1293 | void *start_virt; |
| 1294 | u32 offset, i, req_size; | 1294 | u32 offset, i, req_size; |
| 1295 | int rc; | ||
| 1295 | 1296 | ||
| 1296 | dprintk("hptiop_probe(%p)\n", pcidev); | 1297 | dprintk("hptiop_probe(%p)\n", pcidev); |
| 1297 | 1298 | ||
| @@ -1308,9 +1309,12 @@ static int hptiop_probe(struct pci_dev *pcidev, const struct pci_device_id *id) · 10 unmodified lines | |||
| 1308 | 1309 | ||
| 1309 | /* Enable 64bit DMA if possible */ | 1310 | /* Enable 64bit DMA if possible */ |
| 1310 | iop_ops = (struct hptiop_adapter_ops *)id->driver_data; | 1311 | iop_ops = (struct hptiop_adapter_ops *)id->driver_data; |
| 1311 | if (dma_set_mask(&pcidev->dev, | 1312 | rc = dma_set_mask(&pcidev->dev, |
| 1312 | DMA_BIT_MASK(iop_ops->hw_dma_bit_mask)) || | 1313 | DMA_BIT_MASK(iop_ops->hw_dma_bit_mask)); |
| 1313 | dma_set_mask(&pcidev->dev, DMA_BIT_MASK(32))) { | 1314 | if (rc) |
| 1315 | rc = dma_set_mask(&pcidev->dev, DMA_BIT_MASK(32)); | ||
| 1316 | |||
| 1317 | if (rc) { | ||
| 1314 | printk(KERN_ERR "hptiop: fail to set dma_mask\n"); | 1318 | printk(KERN_ERR "hptiop: fail to set dma_mask\n"); |
| 1315 | goto disable_pci_device; | 1319 | goto disable_pci_device; |
| 1316 | } | 1320 | } |
| @@ -1459,7 +1459,13 @@ static int iscsi_xmit_task(struct iscsi_conn *conn) · 1458 unmodified lines | |||
| 1459 | if (test_bit(ISCSI_SUSPEND_BIT, &conn->suspend_tx)) | 1459 | if (test_bit(ISCSI_SUSPEND_BIT, &conn->suspend_tx)) |
| 1460 | return -ENODATA; | 1460 | return -ENODATA; |
| 1461 | 1461 | ||
| 1462 | spin_lock_bh(&conn->session->back_lock); | ||
| 1463 | if (conn->task == NULL) { | ||
| 1464 | spin_unlock_bh(&conn->session->back_lock); | ||
| 1465 | return -ENODATA; | ||
| 1466 | } | ||
| 1462 | __iscsi_get_task(task); | 1467 | __iscsi_get_task(task); |
| 1468 | spin_unlock_bh(&conn->session->back_lock); | ||
| 1463 | spin_unlock_bh(&conn->session->frwd_lock); | 1469 | spin_unlock_bh(&conn->session->frwd_lock); |
| 1464 | rc = conn->session->tt->xmit_task(task); | 1470 | rc = conn->session->tt->xmit_task(task); |
| 1465 | spin_lock_bh(&conn->session->frwd_lock); | 1471 | spin_lock_bh(&conn->session->frwd_lock); |
| @@ -828,6 +828,7 @@ static struct domain_device *sas_ex_discover_end_dev( · 827 unmodified lines | |||
| 828 | rphy = sas_end_device_alloc(phy->port); | 828 | rphy = sas_end_device_alloc(phy->port); |
| 829 | if (!rphy) | 829 | if (!rphy) |
| 830 | goto out_free; | 830 | goto out_free; |
| 831 | rphy->identify.phy_identifier = phy_id; | ||
| 831 | 832 | ||
| 832 | child->rphy = rphy; | 833 | child->rphy = rphy; |
| 833 | get_device(&rphy->dev); | 834 | get_device(&rphy->dev); |
| @@ -854,6 +855,7 @@ static struct domain_device *sas_ex_discover_end_dev( · 20 unmodified lines | |||
| 854 | 855 | ||
| 855 | child->rphy = rphy; | 856 | child->rphy = rphy; |
| 856 | get_device(&rphy->dev); | 857 | get_device(&rphy->dev); |
| 858 | rphy->identify.phy_identifier = phy_id; | ||
| 857 | sas_fill_in_rphy(child, rphy); | 859 | sas_fill_in_rphy(child, rphy); |
| 858 | 860 | ||
| 859 | list_add_tail(&child->disco_list_node, &parent->port->disco_list); | 861 | list_add_tail(&child->disco_list_node, &parent->port->disco_list); |
| @@ -7361,15 +7361,18 @@ lpfc_sli_pci_mem_setup(struct lpfc_hba *phba) · 7360 unmodified lines | |||
| 7361 | unsigned long bar0map_len, bar2map_len; | 7361 | unsigned long bar0map_len, bar2map_len; |
| 7362 | int i, hbq_count; | 7362 | int i, hbq_count; |
| 7363 | void *ptr; | 7363 | void *ptr; |
| 7364 | int error = -ENODEV; | 7364 | int error; |
| 7365 | 7365 | ||
| 7366 | if (!pdev) | 7366 | if (!pdev) |
| 7367 | return error; | 7367 | return -ENODEV; |
| 7368 | 7368 | ||
| 7369 | /* Set the device DMA mask size */ | 7369 | /* Set the device DMA mask size */ |
| 7370 | if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | 7370 | error = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); |
| 7371 | dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) | 7371 | if (error) |
| 7372 | error = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | ||
| 7373 | if (error) | ||
| 7372 | return error; | 7374 | return error; |
| 7375 | error = -ENODEV; | ||
| 7373 | 7376 | ||
| 7374 | /* Get the bus address of Bar0 and Bar2 and the number of bytes | 7377 | /* Get the bus address of Bar0 and Bar2 and the number of bytes |
| 7375 | * required by each mapping. | 7378 | * required by each mapping. |
| @@ -9742,11 +9745,13 @@ lpfc_sli4_pci_mem_setup(struct lpfc_hba *phba) · 2366 unmodified lines | |||
| 9742 | uint32_t if_type; | 9745 | uint32_t if_type; |
| 9743 | 9746 | ||
| 9744 | if (!pdev) | 9747 | if (!pdev) |
| 9745 | return error; | 9748 | return -ENODEV; |
| 9746 | 9749 | ||
| 9747 | /* Set the device DMA mask size */ | 9750 | /* Set the device DMA mask size */ |
| 9748 | if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)) || | 9751 | error = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); |
| 9749 | dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32))) | 9752 | if (error) |
| 9753 | error = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | ||
| 9754 | if (error) | ||
| 9750 | return error; | 9755 | return error; |
| 9751 | 9756 | ||
| 9752 | /* | 9757 | /* |
| @@ -655,6 +655,7 @@ static blk_status_t scsi_result_to_blk_status(struct scsi_cmnd *cmd, int result) · 654 unmodified lines | |||
| 655 | set_host_byte(cmd, DID_OK); | 655 | set_host_byte(cmd, DID_OK); |
| 656 | return BLK_STS_TARGET; | 656 | return BLK_STS_TARGET; |
| 657 | case DID_NEXUS_FAILURE: | 657 | case DID_NEXUS_FAILURE: |
| 658 | set_host_byte(cmd, DID_OK); | ||
| 658 | return BLK_STS_NEXUS; | 659 | return BLK_STS_NEXUS; |
| 659 | case DID_ALLOC_FAILURE: | 660 | case DID_ALLOC_FAILURE: |
| 660 | set_host_byte(cmd, DID_OK); | 661 | set_host_byte(cmd, DID_OK); |
| @@ -2597,7 +2598,6 @@ void scsi_device_resume(struct scsi_device *sdev) · 1936 unmodified lines | |||
| 2597 | * device deleted during suspend) | 2598 | * device deleted during suspend) |
| 2598 | */ | 2599 | */ |
| 2599 | mutex_lock(&sdev->state_mutex); | 2600 | mutex_lock(&sdev->state_mutex); |
| 2600 | WARN_ON_ONCE(!sdev->quiesced_by); | ||
| 2601 | sdev->quiesced_by = NULL; | 2601 | sdev->quiesced_by = NULL; |
| 2602 | blk_clear_pm_only(sdev->request_queue); | 2602 | blk_clear_pm_only(sdev->request_queue); |
| 2603 | if (sdev->sdev_state == SDEV_QUIESCE) | 2603 | if (sdev->sdev_state == SDEV_QUIESCE) |
| @@ -142,10 +142,12 @@ int sd_zbc_report_zones(struct gendisk *disk, sector_t sector, · 141 unmodified lines | |||
| 142 | return -EOPNOTSUPP; | 142 | return -EOPNOTSUPP; |
| 143 | 143 | ||
| 144 | /* | 144 | /* |
| 145 | * Get a reply buffer for the number of requested zones plus a header. | 145 | * Get a reply buffer for the number of requested zones plus a header, |
| 146 | * For ATA, buffers must be aligned to 512B. | 146 | * without exceeding the device maximum command size. For ATA disks, |
| 147 | * buffers must be aligned to 512B. | ||
| 147 | */ | 148 | */ |
| 148 | buflen = roundup((nrz + 1) * 64, 512); | 149 | buflen = min(queue_max_hw_sectors(disk->queue) << 9, |
| 150 | roundup((nrz + 1) * 64, 512)); | ||
| 149 | buf = kmalloc(buflen, gfp_mask); | 151 | buf = kmalloc(buflen, gfp_mask); |
| 150 | if (!buf) | 152 | if (!buf) |
| 151 | return -ENOMEM; | 153 | return -ENOMEM; |
| @@ -699,8 +699,10 @@ static int __init optee_driver_init(void) · 698 unmodified lines | |||
| 699 | return -ENODEV; | 699 | return -ENODEV; |
| 700 | 700 | ||
| 701 | np = of_find_matching_node(fw_np, optee_match); | 701 | np = of_find_matching_node(fw_np, optee_match); |
| 702 | if (!np || !of_device_is_available(np)) | 702 | if (!np || !of_device_is_available(np)) { |
| 703 | of_node_put(np); | ||
| 703 | return -ENODEV; | 704 | return -ENODEV; |
| 705 | } | ||
| 704 | 706 | ||
| 705 | optee = optee_probe(np); | 707 | optee = optee_probe(np); |
| 706 | of_node_put(np); | 708 | of_node_put(np); |
| @@ -1788,7 +1788,7 @@ static int log_used(struct vhost_virtqueue *vq, u64 used_offset, u64 len) · 1787 unmodified lines | |||
| 1788 | 1788 | ||
| 1789 | ret = translate_desc(vq, (uintptr_t)vq->used + used_offset, | 1789 | ret = translate_desc(vq, (uintptr_t)vq->used + used_offset, |
| 1790 | len, iov, 64, VHOST_ACCESS_WO); | 1790 | len, iov, 64, VHOST_ACCESS_WO); |
| 1791 | if (ret) | 1791 | if (ret < 0) |
| 1792 | return ret; | 1792 | return ret; |
| 1793 | 1793 | ||
| 1794 | for (i = 0; i < ret; i++) { | 1794 | for (i = 0; i < ret; i++) { |
| @@ -173,6 +173,7 @@ static struct afs_cell *afs_alloc_cell(struct afs_net *net, · 172 unmodified lines | |||
| 173 | 173 | ||
| 174 | rcu_assign_pointer(cell->vl_servers, vllist); | 174 | rcu_assign_pointer(cell->vl_servers, vllist); |
| 175 | cell->dns_expiry = TIME64_MAX; | 175 | cell->dns_expiry = TIME64_MAX; |
| 176 | __clear_bit(AFS_CELL_FL_NO_LOOKUP_YET, &cell->flags); | ||
| 176 | } else { | 177 | } else { |
| 177 | cell->dns_expiry = ktime_get_real_seconds(); | 178 | cell->dns_expiry = ktime_get_real_seconds(); |
| 178 | } | 179 | } |
| @@ -14,13 +14,30 @@ · 13 unmodified lines | |||
| 14 | #include <linux/err.h> | 14 | #include <linux/err.h> |
| 15 | #include <linux/fs.h> | 15 | #include <linux/fs.h> |
| 16 | 16 | ||
| 17 | static inline bool spacetab(char c) { return c == ' ' || c == '\t'; } | ||
| 18 | static inline char *next_non_spacetab(char *first, const char *last) | ||
| 19 | { | ||
| 20 | for (; first <= last; first++) | ||
| 21 | if (!spacetab(*first)) | ||
| 22 | return first; | ||
| 23 | return NULL; | ||
| 24 | } | ||
| 25 | static inline char *next_terminator(char *first, const char *last) | ||
| 26 | { | ||
| 27 | for (; first <= last; first++) | ||
| 28 | if (spacetab(*first) || !*first) | ||
| 29 | return first; | ||
| 30 | return NULL; | ||
| 31 | } | ||
| 32 | |||
| 17 | static int load_script(struct linux_binprm *bprm) | 33 | static int load_script(struct linux_binprm *bprm) |
| 18 | { | 34 | { |
| 19 | const char *i_arg, *i_name; | 35 | const char *i_arg, *i_name; |
| 20 | char *cp; | 36 | char *cp, *buf_end; |
| 21 | struct file *file; | 37 | struct file *file; |
| 22 | int retval; | 38 | int retval; |
| 23 | 39 | ||
| 40 | /* Not ours to exec if we don't start with "#!". */ | ||
| 24 | if ((bprm->buf[0] != '#') || (bprm->buf[1] != '!')) | 41 | if ((bprm->buf[0] != '#') || (bprm->buf[1] != '!')) |
| 25 | return -ENOEXEC; | 42 | return -ENOEXEC; |
| 26 | 43 | ||
| @@ -33,18 +50,40 @@ static int load_script(struct linux_binprm *bprm) · 6 unmodified lines | |||
| 33 | if (bprm->interp_flags & BINPRM_FLAGS_PATH_INACCESSIBLE) | 50 | if (bprm->interp_flags & BINPRM_FLAGS_PATH_INACCESSIBLE) |
| 34 | return -ENOENT; | 51 | return -ENOENT; |
| 35 | 52 | ||
| 36 | /* | 53 | /* Release since we are not mapping a binary into memory. */ |
| 37 | * This section does the #! interpretation. | ||
| 38 | * Sorta complicated, but hopefully it will work. -TYT | ||
| 39 | */ | ||
| 40 | |||
| 41 | allow_write_access(bprm->file); | 54 | allow_write_access(bprm->file); |
| 42 | fput(bprm->file); | 55 | fput(bprm->file); |
| 43 | bprm->file = NULL; | 56 | bprm->file = NULL; |
| 44 | 57 | ||
| 45 | bprm->buf[BINPRM_BUF_SIZE - 1] = '\0'; | 58 | /* |
| 46 | if ((cp = strchr(bprm->buf, '\n')) == NULL) | 59 | * This section handles parsing the #! line into separate |
| 47 | cp = bprm->buf+BINPRM_BUF_SIZE-1; | 60 | * interpreter path and argument strings. We must be careful |
| 61 | * because bprm->buf is not yet guaranteed to be NUL-terminated | ||
| 62 | * (though the buffer will have trailing NUL padding when the | ||
| 63 | * file size was smaller than the buffer size). | ||
| 64 | * | ||
| 65 | * We do not want to exec a truncated interpreter path, so either | ||
| 66 | * we find a newline (which indicates nothing is truncated), or | ||
| 67 | * we find a space/tab/NUL after the interpreter path (which | ||
| 68 | * itself may be preceded by spaces/tabs). Truncating the | ||
| 69 | * arguments is fine: the interpreter can re-read the script to | ||
| 70 | * parse them on its own. | ||
| 71 | */ | ||
| 72 | buf_end = bprm->buf + sizeof(bprm->buf) - 1; | ||
| 73 | cp = strnchr(bprm->buf, sizeof(bprm->buf), '\n'); | ||
| 74 | if (!cp) { | ||
| 75 | cp = next_non_spacetab(bprm->buf + 2, buf_end); | ||
| 76 | if (!cp) | ||
| 77 | return -ENOEXEC; /* Entire buf is spaces/tabs */ | ||
| 78 | /* | ||
| 79 | * If there is no later space/tab/NUL we must assume the | ||
| 80 | * interpreter path is truncated. | ||
| 81 | */ | ||
| 82 | if (!next_terminator(cp, buf_end)) | ||
| 83 | return -ENOEXEC; | ||
| 84 | cp = buf_end; | ||
| 85 | } | ||
| 86 | /* NUL-terminate the buffer and any trailing spaces/tabs. */ | ||
| 48 | *cp = '\0'; | 87 | *cp = '\0'; |
| 49 | while (cp > bprm->buf) { | 88 | while (cp > bprm->buf) { |
| 50 | cp--; | 89 | cp--; |
| @@ -616,7 +616,8 @@ int __ceph_finish_cap_snap(struct ceph_inode_info *ci, · 615 unmodified lines | |||
| 616 | capsnap->size); | 616 | capsnap->size); |
| 617 | 617 | ||
| 618 | spin_lock(&mdsc->snap_flush_lock); | 618 | spin_lock(&mdsc->snap_flush_lock); |
| 619 | list_add_tail(&ci->i_snap_flush_item, &mdsc->snap_flush_list); | 619 | if (list_empty(&ci->i_snap_flush_item)) |
| 620 | list_add_tail(&ci->i_snap_flush_item, &mdsc->snap_flush_list); | ||
| 620 | spin_unlock(&mdsc->snap_flush_lock); | 621 | spin_unlock(&mdsc->snap_flush_lock); |
| 621 | return 1; /* caller may want to ceph_flush_snaps */ | 622 | return 1; /* caller may want to ceph_flush_snaps */ |
| 622 | } | 623 | } |
| @@ -859,6 +859,18 @@ static int hugetlbfs_migrate_page(struct address_space *mapping, · 858 unmodified lines | |||
| 859 | rc = migrate_huge_page_move_mapping(mapping, newpage, page); | 859 | rc = migrate_huge_page_move_mapping(mapping, newpage, page); |
| 860 | if (rc != MIGRATEPAGE_SUCCESS) | 860 | if (rc != MIGRATEPAGE_SUCCESS) |
| 861 | return rc; | 861 | return rc; |
| 862 | |||
| 863 | /* | ||
| 864 | * page_private is subpool pointer in hugetlb pages. Transfer to | ||
| 865 | * new page. PagePrivate is not associated with page_private for | ||
| 866 | * hugetlb pages and can not be set here as only page_huge_active | ||
| 867 | * pages can be migrated. | ||
| 868 | */ | ||
| 869 | if (page_private(page)) { | ||
| 870 | set_page_private(newpage, page_private(page)); | ||
| 871 | set_page_private(page, 0); | ||
| 872 | } | ||
| 873 | |||
| 862 | if (mode != MIGRATE_SYNC_NO_COPY) | 874 | if (mode != MIGRATE_SYNC_NO_COPY) |
| 863 | migrate_page_copy(newpage, page); | 875 | migrate_page_copy(newpage, page); |
| 864 | else | 876 | else |
| @@ -2698,7 +2698,6 @@ static long exact_copy_from_user(void *to, const void __user * from, · 2697 unmodified lines | |||
| 2698 | if (!access_ok(from, n)) | 2698 | if (!access_ok(from, n)) |
| 2699 | return n; | 2699 | return n; |
| 2700 | 2700 | ||
| 2701 | current->kernel_uaccess_faults_ok++; | ||
| 2702 | while (n) { | 2701 | while (n) { |
| 2703 | if (__get_user(c, f)) { | 2702 | if (__get_user(c, f)) { |
| 2704 | memset(t, 0, n); | 2703 | memset(t, 0, n); |
| @@ -2708,7 +2707,6 @@ static long exact_copy_from_user(void *to, const void __user * from, · 3 unmodified lines | |||
| 2708 | f++; | 2707 | f++; |
| 2709 | n--; | 2708 | n--; |
| 2710 | } | 2709 | } |
| 2711 | current->kernel_uaccess_faults_ok--; | ||
| 2712 | return n; | 2710 | return n; |
| 2713 | } | 2711 | } |
| 2714 | 2712 | ||
| @@ -44,6 +44,7 @@ · 43 unmodified lines | |||
| 44 | #include <linux/keyctl.h> | 44 | #include <linux/keyctl.h> |
| 45 | #include <linux/key-type.h> | 45 | #include <linux/key-type.h> |
| 46 | #include <keys/user-type.h> | 46 | #include <keys/user-type.h> |
| 47 | #include <keys/request_key_auth-type.h> | ||
| 47 | #include <linux/module.h> | 48 | #include <linux/module.h> |
| 48 | 49 | ||
| 49 | #include "internal.h" | 50 | #include "internal.h" |
| @@ -59,7 +60,7 @@ static struct key_type key_type_id_resolver_legacy; · 9 unmodified lines | |||
| 59 | struct idmap_legacy_upcalldata { | 60 | struct idmap_legacy_upcalldata { |
| 60 | struct rpc_pipe_msg pipe_msg; | 61 | struct rpc_pipe_msg pipe_msg; |
| 61 | struct idmap_msg idmap_msg; | 62 | struct idmap_msg idmap_msg; |
| 62 | struct key_construction *key_cons; | 63 | struct key *authkey; |
| 63 | struct idmap *idmap; | 64 | struct idmap *idmap; |
| 64 | }; | 65 | }; |
| 65 | 66 | ||
| @@ -384,7 +385,7 @@ static const match_table_t nfs_idmap_tokens = { · 318 unmodified lines | |||
| 384 | { Opt_find_err, NULL } | 385 | { Opt_find_err, NULL } |
| 385 | }; | 386 | }; |
| 386 | 387 | ||
| 387 | static int nfs_idmap_legacy_upcall(struct key_construction *, const char *, void *); | 388 | static int nfs_idmap_legacy_upcall(struct key *, void *); |
| 388 | static ssize_t idmap_pipe_downcall(struct file *, const char __user *, | 389 | static ssize_t idmap_pipe_downcall(struct file *, const char __user *, |
| 389 | size_t); | 390 | size_t); |
| 390 | static void idmap_release_pipe(struct inode *); | 391 | static void idmap_release_pipe(struct inode *); |
| @@ -549,11 +550,12 @@ nfs_idmap_prepare_pipe_upcall(struct idmap *idmap, · 158 unmodified lines | |||
| 549 | static void | 550 | static void |
| 550 | nfs_idmap_complete_pipe_upcall_locked(struct idmap *idmap, int ret) | 551 | nfs_idmap_complete_pipe_upcall_locked(struct idmap *idmap, int ret) |
| 551 | { | 552 | { |
| 552 | struct key_construction *cons = idmap->idmap_upcall_data->key_cons; | 553 | struct key *authkey = idmap->idmap_upcall_data->authkey; |
| 553 | 554 | ||
| 554 | kfree(idmap->idmap_upcall_data); | 555 | kfree(idmap->idmap_upcall_data); |
| 555 | idmap->idmap_upcall_data = NULL; | 556 | idmap->idmap_upcall_data = NULL; |
| 556 | complete_request_key(cons, ret); | 557 | complete_request_key(authkey, ret); |
| 558 | key_put(authkey); | ||
| 557 | } | 559 | } |
| 558 | 560 | ||
| 559 | static void | 561 | static void |
| @@ -563,15 +565,14 @@ nfs_idmap_abort_pipe_upcall(struct idmap *idmap, int ret) · 3 unmodified lines | |||
| 563 | nfs_idmap_complete_pipe_upcall_locked(idmap, ret); | 565 | nfs_idmap_complete_pipe_upcall_locked(idmap, ret); |
| 564 | } | 566 | } |
| 565 | 567 | ||
| 566 | static int nfs_idmap_legacy_upcall(struct key_construction *cons, | 568 | static int nfs_idmap_legacy_upcall(struct key *authkey, void *aux) |
| 567 | const char *op, | ||
| 568 | void *aux) | ||
| 569 | { | 569 | { |
| 570 | struct idmap_legacy_upcalldata *data; | 570 | struct idmap_legacy_upcalldata *data; |
| 571 | struct request_key_auth *rka = get_request_key_auth(authkey); | ||
| 571 | struct rpc_pipe_msg *msg; | 572 | struct rpc_pipe_msg *msg; |
| 572 | struct idmap_msg *im; | 573 | struct idmap_msg *im; |
| 573 | struct idmap *idmap = (struct idmap *)aux; | 574 | struct idmap *idmap = (struct idmap *)aux; |
| 574 | struct key *key = cons->key; | 575 | struct key *key = rka->target_key; |
| 575 | int ret = -ENOKEY; | 576 | int ret = -ENOKEY; |
| 576 | 577 | ||
| 577 | if (!aux) | 578 | if (!aux) |
| @@ -586,7 +587,7 @@ static int nfs_idmap_legacy_upcall(struct key_construction *cons, · 8 unmodified lines | |||
| 586 | msg = &data->pipe_msg; | 587 | msg = &data->pipe_msg; |
| 587 | im = &data->idmap_msg; | 588 | im = &data->idmap_msg; |
| 588 | data->idmap = idmap; | 589 | data->idmap = idmap; |
| 589 | data->key_cons = cons; | 590 | data->authkey = key_get(authkey); |
| 590 | 591 | ||
| 591 | ret = nfs_idmap_prepare_message(key->description, idmap, im, msg); | 592 | ret = nfs_idmap_prepare_message(key->description, idmap, im, msg); |
| 592 | if (ret < 0) | 593 | if (ret < 0) |
| @@ -604,7 +605,7 @@ static int nfs_idmap_legacy_upcall(struct key_construction *cons, · 11 unmodified lines | |||
| 604 | out2: | 605 | out2: |
| 605 | kfree(data); | 606 | kfree(data); |
| 606 | out1: | 607 | out1: |
| 607 | complete_request_key(cons, ret); | 608 | complete_request_key(authkey, ret); |
| 608 | return ret; | 609 | return ret; |
| 609 | } | 610 | } |
| 610 | 611 | ||
| @@ -651,9 +652,10 @@ out: · 40 unmodified lines | |||
| 651 | static ssize_t | 652 | static ssize_t |
| 652 | idmap_pipe_downcall(struct file *filp, const char __user *src, size_t mlen) | 653 | idmap_pipe_downcall(struct file *filp, const char __user *src, size_t mlen) |
| 653 | { | 654 | { |
| 655 | struct request_key_auth *rka; | ||
| 654 | struct rpc_inode *rpci = RPC_I(file_inode(filp)); | 656 | struct rpc_inode *rpci = RPC_I(file_inode(filp)); |
| 655 | struct idmap *idmap = (struct idmap *)rpci->private; | 657 | struct idmap *idmap = (struct idmap *)rpci->private; |
| 656 | struct key_construction *cons; | 658 | struct key *authkey; |
| 657 | struct idmap_msg im; | 659 | struct idmap_msg im; |
| 658 | size_t namelen_in; | 660 | size_t namelen_in; |
| 659 | int ret = -ENOKEY; | 661 | int ret = -ENOKEY; |
| @@ -665,7 +667,8 @@ idmap_pipe_downcall(struct file *filp, const char __user *src, size_t mlen) · 5 unmodified lines | |||
| 665 | if (idmap->idmap_upcall_data == NULL) | 667 | if (idmap->idmap_upcall_data == NULL) |
| 666 | goto out_noupcall; | 668 | goto out_noupcall; |
| 667 | 669 | ||
| 668 | cons = idmap->idmap_upcall_data->key_cons; | 670 | authkey = idmap->idmap_upcall_data->authkey; |
| 671 | rka = get_request_key_auth(authkey); | ||
| 669 | 672 | ||
| 670 | if (mlen != sizeof(im)) { | 673 | if (mlen != sizeof(im)) { |
| 671 | ret = -ENOSPC; | 674 | ret = -ENOSPC; |
| @@ -690,9 +693,9 @@ idmap_pipe_downcall(struct file *filp, const char __user *src, size_t mlen) · 18 unmodified lines | |||
| 690 | 693 | ||
| 691 | ret = nfs_idmap_read_and_verify_message(&im, | 694 | ret = nfs_idmap_read_and_verify_message(&im, |
| 692 | &idmap->idmap_upcall_data->idmap_msg, | 695 | &idmap->idmap_upcall_data->idmap_msg, |
| 693 | cons->key, cons->authkey); | 696 | rka->target_key, authkey); |
| 694 | if (ret >= 0) { | 697 | if (ret >= 0) { |
| 695 | key_set_timeout(cons->key, nfs_idmap_cache_timeout); | 698 | key_set_timeout(rka->target_key, nfs_idmap_cache_timeout); |
| 696 | ret = mlen; | 699 | ret = mlen; |
| 697 | } | 700 | } |
| 698 | 701 | ||
| @@ -398,8 +398,6 @@ static ssize_t orangefs_file_read_iter(struct kiocb *iocb, struct iov_iter *iter · 397 unmodified lines | |||
| 398 | loff_t pos = iocb->ki_pos; | 398 | loff_t pos = iocb->ki_pos; |
| 399 | ssize_t rc = 0; | 399 | ssize_t rc = 0; |
| 400 | 400 | ||
| 401 | BUG_ON(iocb->private); | ||
| 402 | |||
| 403 | gossip_debug(GOSSIP_FILE_DEBUG, "orangefs_file_read_iter\n"); | 401 | gossip_debug(GOSSIP_FILE_DEBUG, "orangefs_file_read_iter\n"); |
| 404 | 402 | ||
| 405 | orangefs_stats.reads++; | 403 | orangefs_stats.reads++; |
| @@ -416,8 +414,6 @@ static ssize_t orangefs_file_write_iter(struct kiocb *iocb, struct iov_iter *ite · 10 unmodified lines | |||
| 416 | loff_t pos; | 414 | loff_t pos; |
| 417 | ssize_t rc; | 415 | ssize_t rc; |
| 418 | 416 | ||
| 419 | BUG_ON(iocb->private); | ||
| 420 | |||
| 421 | gossip_debug(GOSSIP_FILE_DEBUG, "orangefs_file_write_iter\n"); | 417 | gossip_debug(GOSSIP_FILE_DEBUG, "orangefs_file_write_iter\n"); |
| 422 | 418 | ||
| 423 | inode_lock(file->f_mapping->host); | 419 | inode_lock(file->f_mapping->host); |
| @@ -1086,10 +1086,6 @@ static int __set_oom_adj(struct file *file, int oom_adj, bool legacy) · 1085 unmodified lines | |||
| 1086 | 1086 | ||
| 1087 | task_lock(p); | 1087 | task_lock(p); |
| 1088 | if (!p->vfork_done && process_shares_mm(p, mm)) { | 1088 | if (!p->vfork_done && process_shares_mm(p, mm)) { |
| 1089 | pr_info("updating oom_score_adj for %d (%s) from %d to %d because it shares mm with %d (%s). Report if this is unexpected.\n", | ||
| 1090 | task_pid_nr(p), p->comm, | ||
| 1091 | p->signal->oom_score_adj, oom_adj, | ||
| 1092 | task_pid_nr(task), task->comm); | ||
| 1093 | p->signal->oom_score_adj = oom_adj; | 1089 | p->signal->oom_score_adj = oom_adj; |
| 1094 | if (!legacy && has_capability_noaudit(current, CAP_SYS_RESOURCE)) | 1090 | if (!legacy && has_capability_noaudit(current, CAP_SYS_RESOURCE)) |
| 1095 | p->signal->oom_score_adj_min = (short)oom_adj; | 1091 | p->signal->oom_score_adj_min = (short)oom_adj; |
| @@ -560,6 +560,8 @@ · 559 unmodified lines | |||
| 560 | # define DP_TEST_LINK_EDID_READ (1 << 2) | 560 | # define DP_TEST_LINK_EDID_READ (1 << 2) |
| 561 | # define DP_TEST_LINK_PHY_TEST_PATTERN (1 << 3) /* DPCD >= 1.1 */ | 561 | # define DP_TEST_LINK_PHY_TEST_PATTERN (1 << 3) /* DPCD >= 1.1 */ |
| 562 | # define DP_TEST_LINK_FAUX_PATTERN (1 << 4) /* DPCD >= 1.2 */ | 562 | # define DP_TEST_LINK_FAUX_PATTERN (1 << 4) /* DPCD >= 1.2 */ |
| 563 | # define DP_TEST_LINK_AUDIO_PATTERN (1 << 5) /* DPCD >= 1.2 */ | ||
| 564 | # define DP_TEST_LINK_AUDIO_DISABLED_VIDEO (1 << 6) /* DPCD >= 1.2 */ | ||
| 563 | 565 | ||
| 564 | #define DP_TEST_LINK_RATE 0x219 | 566 | #define DP_TEST_LINK_RATE 0x219 |
| 565 | # define DP_LINK_RATE_162 (0x6) | 567 | # define DP_LINK_RATE_162 (0x6) |
| @@ -608,6 +610,7 @@ · 42 unmodified lines | |||
| 608 | # define DP_COLOR_FORMAT_RGB (0 << 1) | 610 | # define DP_COLOR_FORMAT_RGB (0 << 1) |
| 609 | # define DP_COLOR_FORMAT_YCbCr422 (1 << 1) | 611 | # define DP_COLOR_FORMAT_YCbCr422 (1 << 1) |
| 610 | # define DP_COLOR_FORMAT_YCbCr444 (2 << 1) | 612 | # define DP_COLOR_FORMAT_YCbCr444 (2 << 1) |
| 613 | # define DP_TEST_DYNAMIC_RANGE_VESA (0 << 3) | ||
| 611 | # define DP_TEST_DYNAMIC_RANGE_CEA (1 << 3) | 614 | # define DP_TEST_DYNAMIC_RANGE_CEA (1 << 3) |
| 612 | # define DP_TEST_YCBCR_COEFFICIENTS (1 << 4) | 615 | # define DP_TEST_YCBCR_COEFFICIENTS (1 << 4) |
| 613 | # define DP_YCBCR_COEFFICIENTS_ITU601 (0 << 4) | 616 | # define DP_YCBCR_COEFFICIENTS_ITU601 (0 << 4) |
| @@ -657,6 +660,16 @@ · 43 unmodified lines | |||
| 657 | 660 | ||
| 658 | #define DP_TEST_SINK 0x270 | 661 | #define DP_TEST_SINK 0x270 |
| 659 | # define DP_TEST_SINK_START (1 << 0) | 662 | # define DP_TEST_SINK_START (1 << 0) |
| 663 | #define DP_TEST_AUDIO_MODE 0x271 | ||
| 664 | #define DP_TEST_AUDIO_PATTERN_TYPE 0x272 | ||
| 665 | #define DP_TEST_AUDIO_PERIOD_CH1 0x273 | ||
| 666 | #define DP_TEST_AUDIO_PERIOD_CH2 0x274 | ||
| 667 | #define DP_TEST_AUDIO_PERIOD_CH3 0x275 | ||
| 668 | #define DP_TEST_AUDIO_PERIOD_CH4 0x276 | ||
| 669 | #define DP_TEST_AUDIO_PERIOD_CH5 0x277 | ||
| 670 | #define DP_TEST_AUDIO_PERIOD_CH6 0x278 | ||
| 671 | #define DP_TEST_AUDIO_PERIOD_CH7 0x279 | ||
| 672 | #define DP_TEST_AUDIO_PERIOD_CH8 0x27A | ||
| 660 | 673 | ||
| 661 | #define DP_FEC_STATUS 0x280 /* 1.4 */ | 674 | #define DP_FEC_STATUS 0x280 /* 1.4 */ |
| 662 | # define DP_FEC_DECODE_EN_DETECTED (1 << 0) | 675 | # define DP_FEC_DECODE_EN_DETECTED (1 << 0) |
| @@ -770,7 +770,7 @@ static inline bool drm_dev_is_unplugged(struct drm_device *dev) · 769 unmodified lines | |||
| 770 | * | 770 | * |
| 771 | * Returns true if the @feature is supported, false otherwise. | 771 | * Returns true if the @feature is supported, false otherwise. |
| 772 | */ | 772 | */ |
| 773 | static inline bool drm_core_check_feature(struct drm_device *dev, u32 feature) | 773 | static inline bool drm_core_check_feature(const struct drm_device *dev, u32 feature) |
| 774 | { | 774 | { |
| 775 | return dev->driver->driver_features & dev->driver_features & feature; | 775 | return dev->driver->driver_features & dev->driver_features & feature; |
| 776 | } | 776 | } |
| @@ -0,0 +1,36 @@ | |||
| 1 | /* request_key authorisation token key type | ||
| 2 | * | ||
| 3 | * Copyright (C) 2005 Red Hat, Inc. All Rights Reserved. | ||
| 4 | * Written by David Howells (dhowells@redhat.com) | ||
| 5 | * | ||
| 6 | * This program is free software; you can redistribute it and/or | ||
| 7 | * modify it under the terms of the GNU General Public Licence | ||
| 8 | * as published by the Free Software Foundation; either version | ||
| 9 | * 2 of the Licence, or (at your option) any later version. | ||
| 10 | */ | ||
| 11 | |||
| 12 | #ifndef _KEYS_REQUEST_KEY_AUTH_TYPE_H | ||
| 13 | #define _KEYS_REQUEST_KEY_AUTH_TYPE_H | ||
| 14 | |||
| 15 | #include <linux/key.h> | ||
| 16 | |||
| 17 | /* | ||
| 18 | * Authorisation record for request_key(). | ||
| 19 | */ | ||
| 20 | struct request_key_auth { | ||
| 21 | struct key *target_key; | ||
| 22 | struct key *dest_keyring; | ||
| 23 | const struct cred *cred; | ||
| 24 | void *callout_info; | ||
| 25 | size_t callout_len; | ||
| 26 | pid_t pid; | ||
| 27 | char op[8]; | ||
| 28 | } __randomize_layout; | ||
| 29 | |||
| 30 | static inline struct request_key_auth *get_request_key_auth(const struct key *key) | ||
| 31 | { | ||
| 32 | return key->payload.data[0]; | ||
| 33 | } | ||
| 34 | |||
| 35 | |||
| 36 | #endif /* _KEYS_REQUEST_KEY_AUTH_TYPE_H */ | ||
| @@ -31,7 +31,7 @@ · 30 unmodified lines | |||
| 31 | struct user_key_payload { | 31 | struct user_key_payload { |
| 32 | struct rcu_head rcu; /* RCU destructor */ | 32 | struct rcu_head rcu; /* RCU destructor */ |
| 33 | unsigned short datalen; /* length of this data */ | 33 | unsigned short datalen; /* length of this data */ |
| 34 | char data[0]; /* actual data */ | 34 | char data[0] __aligned(__alignof__(u64)); /* actual data */ |
| 35 | }; | 35 | }; |
| 36 | 36 | ||
| 37 | extern struct key_type key_type_user; | 37 | extern struct key_type key_type_user; |
| @@ -20,15 +20,6 @@ · 19 unmodified lines | |||
| 20 | struct kernel_pkey_query; | 20 | struct kernel_pkey_query; |
| 21 | struct kernel_pkey_params; | 21 | struct kernel_pkey_params; |
| 22 | 22 | ||
| 23 | /* | ||
| 24 | * key under-construction record | ||
| 25 | * - passed to the request_key actor if supplied | ||
| 26 | */ | ||
| 27 | struct key_construction { | ||
| 28 | struct key *key; /* key being constructed */ | ||
| 29 | struct key *authkey;/* authorisation for key being constructed */ | ||
| 30 | }; | ||
| 31 | |||
| 32 | /* | 23 | /* |
| 33 | * Pre-parsed payload, used by key add, update and instantiate. | 24 | * Pre-parsed payload, used by key add, update and instantiate. |
| 34 | * | 25 | * |
| @@ -50,8 +41,7 @@ struct key_preparsed_payload { · 15 unmodified lines | |||
| 50 | time64_t expiry; /* Expiry time of key */ | 41 | time64_t expiry; /* Expiry time of key */ |
| 51 | } __randomize_layout; | 42 | } __randomize_layout; |
| 52 | 43 | ||
| 53 | typedef int (*request_key_actor_t)(struct key_construction *key, | 44 | typedef int (*request_key_actor_t)(struct key *auth_key, void *aux); |
| 54 | const char *op, void *aux); | ||
| 55 | 45 | ||
| 56 | /* | 46 | /* |
| 57 | * Preparsed matching criterion. | 47 | * Preparsed matching criterion. |
| @@ -181,20 +171,20 @@ extern int key_instantiate_and_link(struct key *key, · 123 unmodified lines | |||
| 181 | const void *data, | 171 | const void *data, |
| 182 | size_t datalen, | 172 | size_t datalen, |
| 183 | struct key *keyring, | 173 | struct key *keyring, |
| 184 | struct key *instkey); | 174 | struct key *authkey); |
| 185 | extern int key_reject_and_link(struct key *key, | 175 | extern int key_reject_and_link(struct key *key, |
| 186 | unsigned timeout, | 176 | unsigned timeout, |
| 187 | unsigned error, | 177 | unsigned error, |
| 188 | struct key *keyring, | 178 | struct key *keyring, |
| 189 | struct key *instkey); | 179 | struct key *authkey); |
| 190 | extern void complete_request_key(struct key_construction *cons, int error); | 180 | extern void complete_request_key(struct key *authkey, int error); |
| 191 | 181 | ||
| 192 | static inline int key_negate_and_link(struct key *key, | 182 | static inline int key_negate_and_link(struct key *key, |
| 193 | unsigned timeout, | 183 | unsigned timeout, |
| 194 | struct key *keyring, | 184 | struct key *keyring, |
| 195 | struct key *instkey) | 185 | struct key *authkey) |
| 196 | { | 186 | { |
| 197 | return key_reject_and_link(key, timeout, ENOKEY, keyring, instkey); | 187 | return key_reject_and_link(key, timeout, ENOKEY, keyring, authkey); |
| 198 | } | 188 | } |
| 199 | 189 | ||
| 200 | extern int generic_key_instantiate(struct key *key, struct key_preparsed_payload *prep); | 190 | extern int generic_key_instantiate(struct key *key, struct key_preparsed_payload *prep); |
| @@ -11,6 +11,8 @@ · 10 unmodified lines | |||
| 11 | #define _LINUX_NETDEV_FEATURES_H | 11 | #define _LINUX_NETDEV_FEATURES_H |
| 12 | 12 | ||
| 13 | #include <linux/types.h> | 13 | #include <linux/types.h> |
| 14 | #include <linux/bitops.h> | ||
| 15 | #include <asm/byteorder.h> | ||
| 14 | 16 | ||
| 15 | typedef u64 netdev_features_t; | 17 | typedef u64 netdev_features_t; |
| 16 | 18 | ||
| @@ -154,8 +156,26 @@ enum { · 137 unmodified lines | |||
| 154 | #define NETIF_F_HW_TLS_TX __NETIF_F(HW_TLS_TX) | 156 | #define NETIF_F_HW_TLS_TX __NETIF_F(HW_TLS_TX) |
| 155 | #define NETIF_F_HW_TLS_RX __NETIF_F(HW_TLS_RX) | 157 | #define NETIF_F_HW_TLS_RX __NETIF_F(HW_TLS_RX) |
| 156 | 158 | ||
| 157 | #define for_each_netdev_feature(mask_addr, bit) \ | 159 | /* Finds the next feature with the highest number of the range of start till 0. |
| 158 | for_each_set_bit(bit, (unsigned long *)mask_addr, NETDEV_FEATURE_COUNT) | 160 | */ |
| 161 | static inline int find_next_netdev_feature(u64 feature, unsigned long start) | ||
| 162 | { | ||
| 163 | /* like BITMAP_LAST_WORD_MASK() for u64 | ||
| 164 | * this sets the most significant 64 - start to 0. | ||
| 165 | */ | ||
| 166 | feature &= ~0ULL >> (-start & ((sizeof(feature) * 8) - 1)); | ||
| 167 | |||
| 168 | return fls64(feature) - 1; | ||
| 169 | } | ||
| 170 | |||
| 171 | /* This goes for the MSB to the LSB through the set feature bits, | ||
| 172 | * mask_addr should be a u64 and bit an int | ||
| 173 | */ | ||
| 174 | #define for_each_netdev_feature(mask_addr, bit) \ | ||
| 175 | for ((bit) = find_next_netdev_feature((mask_addr), \ | ||
| 176 | NETDEV_FEATURE_COUNT); \ | ||
| 177 | (bit) >= 0; \ | ||
| 178 | (bit) = find_next_netdev_feature((mask_addr), (bit) - 1)) | ||
| 159 | 179 | ||
| 160 | /* Features valid for ethtool to change */ | 180 | /* Features valid for ethtool to change */ |
| 161 | /* = all defined minus driver/device-class-related */ | 181 | /* = all defined minus driver/device-class-related */ |
| @@ -3861,7 +3861,7 @@ static inline u32 netif_msg_init(int debug_value, int default_msg_enable_bits) · 3860 unmodified lines | |||
| 3861 | if (debug_value == 0) /* no output */ | 3861 | if (debug_value == 0) /* no output */ |
| 3862 | return 0; | 3862 | return 0; |
| 3863 | /* set low N bits */ | 3863 | /* set low N bits */ |
| 3864 | return (1 << debug_value) - 1; | 3864 | return (1U << debug_value) - 1; |
| 3865 | } | 3865 | } |
| 3866 | 3866 | ||
| 3867 | static inline void __netif_tx_lock(struct netdev_queue *txq, int cpu) | 3867 | static inline void __netif_tx_lock(struct netdev_queue *txq, int cpu) |
| @@ -992,6 +992,14 @@ static inline int genphy_no_soft_reset(struct phy_device *phydev) · 991 unmodified lines | |||
| 992 | { | 992 | { |
| 993 | return 0; | 993 | return 0; |
| 994 | } | 994 | } |
| 995 | static inline int genphy_no_ack_interrupt(struct phy_device *phydev) | ||
| 996 | { | ||
| 997 | return 0; | ||
| 998 | } | ||
| 999 | static inline int genphy_no_config_intr(struct phy_device *phydev) | ||
| 1000 | { | ||
| 1001 | return 0; | ||
| 1002 | } | ||
| 995 | int genphy_read_mmd_unsupported(struct phy_device *phdev, int devad, | 1003 | int genphy_read_mmd_unsupported(struct phy_device *phdev, int devad, |
| 996 | u16 regnum); | 1004 | u16 regnum); |
| 997 | int genphy_write_mmd_unsupported(struct phy_device *phdev, int devnum, | 1005 | int genphy_write_mmd_unsupported(struct phy_device *phdev, int devnum, |
| @@ -739,12 +739,6 @@ struct task_struct { · 738 unmodified lines | |||
| 739 | unsigned use_memdelay:1; | 739 | unsigned use_memdelay:1; |
| 740 | #endif | 740 | #endif |
| 741 | 741 | ||
| 742 | /* | ||
| 743 | * May usercopy functions fault on kernel addresses? | ||
| 744 | * This is not just a single bit because this can potentially nest. | ||
| 745 | */ | ||
| 746 | unsigned int kernel_uaccess_faults_ok; | ||
| 747 | |||
| 748 | unsigned long atomic_flags; /* Flags requiring atomic access. */ | 742 | unsigned long atomic_flags; /* Flags requiring atomic access. */ |
| 749 | 743 | ||
| 750 | struct restart_block restart_block; | 744 | struct restart_block restart_block; |
| @@ -2434,7 +2434,7 @@ static inline void skb_probe_transport_header(struct sk_buff *skb, · 2433 unmodified lines | |||
| 2434 | 2434 | ||
| 2435 | if (skb_flow_dissect_flow_keys_basic(skb, &keys, NULL, 0, 0, 0, 0)) | 2435 | if (skb_flow_dissect_flow_keys_basic(skb, &keys, NULL, 0, 0, 0, 0)) |
| 2436 | skb_set_transport_header(skb, keys.control.thoff); | 2436 | skb_set_transport_header(skb, keys.control.thoff); |
| 2437 | else | 2437 | else if (offset_hint >= 0) |
| 2438 | skb_set_transport_header(skb, offset_hint); | 2438 | skb_set_transport_header(skb, offset_hint); |
| 2439 | } | 2439 | } |
| 2440 | 2440 | ||
| @@ -4212,6 +4212,12 @@ static inline bool skb_is_gso_sctp(const struct sk_buff *skb) · 1771 unmodified lines | |||
| 4212 | return skb_shinfo(skb)->gso_type & SKB_GSO_SCTP; | 4212 | return skb_shinfo(skb)->gso_type & SKB_GSO_SCTP; |
| 4213 | } | 4213 | } |
| 4214 | 4214 | ||
| 4215 | static inline bool skb_is_gso_tcp(const struct sk_buff *skb) | ||
| 4216 | { | ||
| 4217 | return skb_is_gso(skb) && | ||
| 4218 | skb_shinfo(skb)->gso_type & (SKB_GSO_TCPV4 | SKB_GSO_TCPV6); | ||
| 4219 | } | ||
| 4220 | |||
| 4215 | static inline void skb_gso_reset(struct sk_buff *skb) | 4221 | static inline void skb_gso_reset(struct sk_buff *skb) |
| 4216 | { | 4222 | { |
| 4217 | skb_shinfo(skb)->gso_size = 0; | 4223 | skb_shinfo(skb)->gso_size = 0; |
| @@ -57,6 +57,25 @@ static inline int virtio_net_hdr_to_skb(struct sk_buff *skb, · 56 unmodified lines | |||
| 57 | 57 | ||
| 58 | if (!skb_partial_csum_set(skb, start, off)) | 58 | if (!skb_partial_csum_set(skb, start, off)) |
| 59 | return -EINVAL; | 59 | return -EINVAL; |
| 60 | } else { | ||
| 61 | /* gso packets without NEEDS_CSUM do not set transport_offset. | ||
| 62 | * probe and drop if does not match one of the above types. | ||
| 63 | */ | ||
| 64 | if (gso_type && skb->network_header) { | ||
| 65 | if (!skb->protocol) | ||
| 66 | virtio_net_hdr_set_proto(skb, hdr); | ||
| 67 | retry: | ||
| 68 | skb_probe_transport_header(skb, -1); | ||
| 69 | if (!skb_transport_header_was_set(skb)) { | ||
| 70 | /* UFO does not specify ipv4 or 6: try both */ | ||
| 71 | if (gso_type & SKB_GSO_UDP && | ||
| 72 | skb->protocol == htons(ETH_P_IP)) { | ||
| 73 | skb->protocol = htons(ETH_P_IPV6); | ||
| 74 | goto retry; | ||
| 75 | } | ||
| 76 | return -EINVAL; | ||
| 77 | } | ||
| 78 | } | ||
| 60 | } | 79 | } |
| 61 | 80 | ||
| 62 | if (hdr->gso_type != VIRTIO_NET_HDR_GSO_NONE) { | 81 | if (hdr->gso_type != VIRTIO_NET_HDR_GSO_NONE) { |
| @@ -22,6 +22,7 @@ · 21 unmodified lines | |||
| 22 | 22 | ||
| 23 | #include <net/inet_sock.h> | 23 | #include <net/inet_sock.h> |
| 24 | #include <net/snmp.h> | 24 | #include <net/snmp.h> |
| 25 | #include <net/ip.h> | ||
| 25 | 26 | ||
| 26 | struct icmp_err { | 27 | struct icmp_err { |
| 27 | int errno; | 28 | int errno; |
| @@ -39,7 +40,13 @@ struct net_proto_family; · 11 unmodified lines | |||
| 39 | struct sk_buff; | 40 | struct sk_buff; |
| 40 | struct net; | 41 | struct net; |
| 41 | 42 | ||
| 42 | void icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info); | 43 | void __icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info, |
| 44 | const struct ip_options *opt); | ||
| 45 | static inline void icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info) | ||
| 46 | { | ||
| 47 | __icmp_send(skb_in, type, code, info, &IPCB(skb_in)->opt); | ||
| 48 | } | ||
| 49 | |||
| 43 | int icmp_rcv(struct sk_buff *skb); | 50 | int icmp_rcv(struct sk_buff *skb); |
| 44 | int icmp_err(struct sk_buff *skb, u32 info); | 51 | int icmp_err(struct sk_buff *skb, u32 info); |
| 45 | int icmp_init(void); | 52 | int icmp_init(void); |
| @@ -667,6 +667,8 @@ static inline int ip_options_echo(struct net *net, struct ip_options *dopt, · 666 unmodified lines | |||
| 667 | } | 667 | } |
| 668 | 668 | ||
| 669 | void ip_options_fragment(struct sk_buff *skb); | 669 | void ip_options_fragment(struct sk_buff *skb); |
| 670 | int __ip_options_compile(struct net *net, struct ip_options *opt, | ||
| 671 | struct sk_buff *skb, __be32 *info); | ||
| 670 | int ip_options_compile(struct net *net, struct ip_options *opt, | 672 | int ip_options_compile(struct net *net, struct ip_options *opt, |
| 671 | struct sk_buff *skb); | 673 | struct sk_buff *skb); |
| 672 | int ip_options_get(struct net *net, struct ip_options_rcu **optp, | 674 | int ip_options_get(struct net *net, struct ip_options_rcu **optp, |
| @@ -716,7 +718,7 @@ extern int sysctl_icmp_msgs_burst; · 43 unmodified lines | |||
| 716 | int ip_misc_proc_init(void); | 718 | int ip_misc_proc_init(void); |
| 717 | #endif | 719 | #endif |
| 718 | 720 | ||
| 719 | int rtm_getroute_parse_ip_proto(struct nlattr *attr, u8 *ip_proto, | 721 | int rtm_getroute_parse_ip_proto(struct nlattr *attr, u8 *ip_proto, u8 family, |
| 720 | struct netlink_ext_ack *extack); | 722 | struct netlink_ext_ack *extack); |
| 721 | 723 | ||
| 722 | #endif /* _IP_H */ | 724 | #endif /* _IP_H */ |
| @@ -63,10 +63,11 @@ struct pnpipehdr { · 62 unmodified lines | |||
| 63 | u8 state_after_reset; /* reset request */ | 63 | u8 state_after_reset; /* reset request */ |
| 64 | u8 error_code; /* any response */ | 64 | u8 error_code; /* any response */ |
| 65 | u8 pep_type; /* status indication */ | 65 | u8 pep_type; /* status indication */ |
| 66 | u8 data[1]; | 66 | u8 data0; /* anything else */ |
| 67 | }; | 67 | }; |
| 68 | u8 data[]; | ||
| 68 | }; | 69 | }; |
| 69 | #define other_pep_type data[1] | 70 | #define other_pep_type data[0] |
| 70 | 71 | ||
| 71 | static inline struct pnpipehdr *pnp_hdr(struct sk_buff *skb) | 72 | static inline struct pnpipehdr *pnp_hdr(struct sk_buff *skb) |
| 72 | { | 73 | { |
| @@ -853,7 +853,7 @@ static inline void xfrm_pols_put(struct xfrm_policy **pols, int npols) · 852 unmodified lines | |||
| 853 | xfrm_pol_put(pols[i]); | 853 | xfrm_pol_put(pols[i]); |
| 854 | } | 854 | } |
| 855 | 855 | ||
| 856 | void __xfrm_state_destroy(struct xfrm_state *); | 856 | void __xfrm_state_destroy(struct xfrm_state *, bool); |
| 857 | 857 | ||
| 858 | static inline void __xfrm_state_put(struct xfrm_state *x) | 858 | static inline void __xfrm_state_put(struct xfrm_state *x) |
| 859 | { | 859 | { |
| @@ -863,7 +863,13 @@ static inline void __xfrm_state_put(struct xfrm_state *x) · 3 unmodified lines | |||
| 863 | static inline void xfrm_state_put(struct xfrm_state *x) | 863 | static inline void xfrm_state_put(struct xfrm_state *x) |
| 864 | { | 864 | { |
| 865 | if (refcount_dec_and_test(&x->refcnt)) | 865 | if (refcount_dec_and_test(&x->refcnt)) |
| 866 | __xfrm_state_destroy(x); | 866 | __xfrm_state_destroy(x, false); |
| 867 | } | ||
| 868 | |||
| 869 | static inline void xfrm_state_put_sync(struct xfrm_state *x) | ||
| 870 | { | ||
| 871 | if (refcount_dec_and_test(&x->refcnt)) | ||
| 872 | __xfrm_state_destroy(x, true); | ||
| 867 | } | 873 | } |
| 868 | 874 | ||
| 869 | static inline void xfrm_state_hold(struct xfrm_state *x) | 875 | static inline void xfrm_state_hold(struct xfrm_state *x) |
| @@ -1590,7 +1596,7 @@ struct xfrmk_spdinfo { · 720 unmodified lines | |||
| 1590 | 1596 | ||
| 1591 | struct xfrm_state *xfrm_find_acq_byseq(struct net *net, u32 mark, u32 seq); | 1597 | struct xfrm_state *xfrm_find_acq_byseq(struct net *net, u32 mark, u32 seq); |
| 1592 | int xfrm_state_delete(struct xfrm_state *x); | 1598 | int xfrm_state_delete(struct xfrm_state *x); |
| 1593 | int xfrm_state_flush(struct net *net, u8 proto, bool task_valid); | 1599 | int xfrm_state_flush(struct net *net, u8 proto, bool task_valid, bool sync); |
| 1594 | int xfrm_dev_state_flush(struct net *net, struct net_device *dev, bool task_valid); | 1600 | int xfrm_dev_state_flush(struct net *net, struct net_device *dev, bool task_valid); |
| 1595 | void xfrm_sad_getinfo(struct net *net, struct xfrmk_sadinfo *si); | 1601 | void xfrm_sad_getinfo(struct net *net, struct xfrmk_sadinfo *si); |
| 1596 | void xfrm_spd_getinfo(struct net *net, struct xfrmk_spdinfo *si); | 1602 | void xfrm_spd_getinfo(struct net *net, struct xfrmk_spdinfo *si); |
| @@ -272,13 +272,14 @@ union drm_amdgpu_vm { · 271 unmodified lines | |||
| 272 | 272 | ||
| 273 | /* sched ioctl */ | 273 | /* sched ioctl */ |
| 274 | #define AMDGPU_SCHED_OP_PROCESS_PRIORITY_OVERRIDE 1 | 274 | #define AMDGPU_SCHED_OP_PROCESS_PRIORITY_OVERRIDE 1 |
| 275 | #define AMDGPU_SCHED_OP_CONTEXT_PRIORITY_OVERRIDE 2 | ||
| 275 | 276 | ||
| 276 | struct drm_amdgpu_sched_in { | 277 | struct drm_amdgpu_sched_in { |
| 277 | /* AMDGPU_SCHED_OP_* */ | 278 | /* AMDGPU_SCHED_OP_* */ |
| 278 | __u32 op; | 279 | __u32 op; |
| 279 | __u32 fd; | 280 | __u32 fd; |
| 280 | __s32 priority; | 281 | __s32 priority; |
| 281 | __u32 flags; | 282 | __u32 ctx_id; |
| 282 | }; | 283 | }; |
| 283 | 284 | ||
| 284 | union drm_amdgpu_sched { | 285 | union drm_amdgpu_sched { |
| @@ -133,12 +133,63 @@ struct drm_nouveau_gem_cpu_fini { · 132 unmodified lines | |||
| 133 | #define DRM_NOUVEAU_NOTIFIEROBJ_ALLOC 0x05 /* deprecated */ | 133 | #define DRM_NOUVEAU_NOTIFIEROBJ_ALLOC 0x05 /* deprecated */ |
| 134 | #define DRM_NOUVEAU_GPUOBJ_FREE 0x06 /* deprecated */ | 134 | #define DRM_NOUVEAU_GPUOBJ_FREE 0x06 /* deprecated */ |
| 135 | #define DRM_NOUVEAU_NVIF 0x07 | 135 | #define DRM_NOUVEAU_NVIF 0x07 |
| 136 | #define DRM_NOUVEAU_SVM_INIT 0x08 | ||
| 137 | #define DRM_NOUVEAU_SVM_BIND 0x09 | ||
| 136 | #define DRM_NOUVEAU_GEM_NEW 0x40 | 138 | #define DRM_NOUVEAU_GEM_NEW 0x40 |
| 137 | #define DRM_NOUVEAU_GEM_PUSHBUF 0x41 | 139 | #define DRM_NOUVEAU_GEM_PUSHBUF 0x41 |
| 138 | #define DRM_NOUVEAU_GEM_CPU_PREP 0x42 | 140 | #define DRM_NOUVEAU_GEM_CPU_PREP 0x42 |
| 139 | #define DRM_NOUVEAU_GEM_CPU_FINI 0x43 | 141 | #define DRM_NOUVEAU_GEM_CPU_FINI 0x43 |
| 140 | #define DRM_NOUVEAU_GEM_INFO 0x44 | 142 | #define DRM_NOUVEAU_GEM_INFO 0x44 |
| 141 | 143 | ||
| 144 | struct drm_nouveau_svm_init { | ||
| 145 | __u64 unmanaged_addr; | ||
| 146 | __u64 unmanaged_size; | ||
| 147 | }; | ||
| 148 | |||
| 149 | struct drm_nouveau_svm_bind { | ||
| 150 | __u64 header; | ||
| 151 | __u64 va_start; | ||
| 152 | __u64 va_end; | ||
| 153 | __u64 npages; | ||
| 154 | __u64 stride; | ||
| 155 | __u64 result; | ||
| 156 | __u64 reserved0; | ||
| 157 | __u64 reserved1; | ||
| 158 | }; | ||
| 159 | |||
| 160 | #define NOUVEAU_SVM_BIND_COMMAND_SHIFT 0 | ||
| 161 | #define NOUVEAU_SVM_BIND_COMMAND_BITS 8 | ||
| 162 | #define NOUVEAU_SVM_BIND_COMMAND_MASK ((1 << 8) - 1) | ||
| 163 | #define NOUVEAU_SVM_BIND_PRIORITY_SHIFT 8 | ||
| 164 | #define NOUVEAU_SVM_BIND_PRIORITY_BITS 8 | ||
| 165 | #define NOUVEAU_SVM_BIND_PRIORITY_MASK ((1 << 8) - 1) | ||
| 166 | #define NOUVEAU_SVM_BIND_TARGET_SHIFT 16 | ||
| 167 | #define NOUVEAU_SVM_BIND_TARGET_BITS 32 | ||
| 168 | #define NOUVEAU_SVM_BIND_TARGET_MASK 0xffffffff | ||
| 169 | |||
| 170 | /* | ||
| 171 | * Below is use to validate ioctl argument, userspace can also use it to make | ||
| 172 | * sure that no bit are set beyond known fields for a given kernel version. | ||
| 173 | */ | ||
| 174 | #define NOUVEAU_SVM_BIND_VALID_BITS 48 | ||
| 175 | #define NOUVEAU_SVM_BIND_VALID_MASK ((1ULL << NOUVEAU_SVM_BIND_VALID_BITS) - 1) | ||
| 176 | |||
| 177 | |||
| 178 | /* | ||
| 179 | * NOUVEAU_BIND_COMMAND__MIGRATE: synchronous migrate to target memory. | ||
| 180 | * result: number of page successfuly migrate to the target memory. | ||
| 181 | */ | ||
| 182 | #define NOUVEAU_SVM_BIND_COMMAND__MIGRATE 0 | ||
| 183 | |||
| 184 | /* | ||
| 185 | * NOUVEAU_SVM_BIND_HEADER_TARGET__GPU_VRAM: target the GPU VRAM memory. | ||
| 186 | */ | ||
| 187 | #define NOUVEAU_SVM_BIND_TARGET__GPU_VRAM (1UL << 31) | ||
| 188 | |||
| 189 | |||
| 190 | #define DRM_IOCTL_NOUVEAU_SVM_INIT DRM_IOWR(DRM_COMMAND_BASE + DRM_NOUVEAU_SVM_INIT, struct drm_nouveau_svm_init) | ||
| 191 | #define DRM_IOCTL_NOUVEAU_SVM_BIND DRM_IOWR(DRM_COMMAND_BASE + DRM_NOUVEAU_SVM_BIND, struct drm_nouveau_svm_bind) | ||
| 192 | |||
| 142 | #define DRM_IOCTL_NOUVEAU_GEM_NEW DRM_IOWR(DRM_COMMAND_BASE + DRM_NOUVEAU_GEM_NEW, struct drm_nouveau_gem_new) | 193 | #define DRM_IOCTL_NOUVEAU_GEM_NEW DRM_IOWR(DRM_COMMAND_BASE + DRM_NOUVEAU_GEM_NEW, struct drm_nouveau_gem_new) |
| 143 | #define DRM_IOCTL_NOUVEAU_GEM_PUSHBUF DRM_IOWR(DRM_COMMAND_BASE + DRM_NOUVEAU_GEM_PUSHBUF, struct drm_nouveau_gem_pushbuf) | 194 | #define DRM_IOCTL_NOUVEAU_GEM_PUSHBUF DRM_IOWR(DRM_COMMAND_BASE + DRM_NOUVEAU_GEM_PUSHBUF, struct drm_nouveau_gem_pushbuf) |
| 144 | #define DRM_IOCTL_NOUVEAU_GEM_CPU_PREP DRM_IOW (DRM_COMMAND_BASE + DRM_NOUVEAU_GEM_CPU_PREP, struct drm_nouveau_gem_cpu_prep) | 195 | #define DRM_IOCTL_NOUVEAU_GEM_CPU_PREP DRM_IOW (DRM_COMMAND_BASE + DRM_NOUVEAU_GEM_CPU_PREP, struct drm_nouveau_gem_cpu_prep) |
| @@ -348,6 +348,7 @@ int ipu_prg_channel_configure(struct ipuv3_channel *ipu_chan, · 347 unmodified lines | |||
| 348 | unsigned int axi_id, unsigned int width, | 348 | unsigned int axi_id, unsigned int width, |
| 349 | unsigned int height, unsigned int stride, | 349 | unsigned int height, unsigned int stride, |
| 350 | u32 format, uint64_t modifier, unsigned long *eba); | 350 | u32 format, uint64_t modifier, unsigned long *eba); |
| 351 | bool ipu_prg_channel_configure_pending(struct ipuv3_channel *ipu_chan); | ||
| 351 | 352 | ||
| 352 | /* | 353 | /* |
| 353 | * IPU CMOS Sensor Interface (csi) functions | 354 | * IPU CMOS Sensor Interface (csi) functions |
| @@ -550,6 +550,7 @@ skip: · 549 unmodified lines | |||
| 550 | initrd_end = 0; | 550 | initrd_end = 0; |
| 551 | } | 551 | } |
| 552 | 552 | ||
| 553 | #ifdef CONFIG_BLK_DEV_RAM | ||
| 553 | #define BUF_SIZE 1024 | 554 | #define BUF_SIZE 1024 |
| 554 | static void __init clean_rootfs(void) | 555 | static void __init clean_rootfs(void) |
| 555 | { | 556 | { |
| @@ -596,6 +597,7 @@ static void __init clean_rootfs(void) · 40 unmodified lines | |||
| 596 | ksys_close(fd); | 597 | ksys_close(fd); |
| 597 | kfree(buf); | 598 | kfree(buf); |
| 598 | } | 599 | } |
| 600 | #endif | ||
| 599 | 601 | ||
| 600 | static int __init populate_rootfs(void) | 602 | static int __init populate_rootfs(void) |
| 601 | { | 603 | { |
| @@ -638,10 +640,8 @@ static int __init populate_rootfs(void) · 36 unmodified lines | |||
| 638 | printk(KERN_INFO "Unpacking initramfs...\n"); | 640 | printk(KERN_INFO "Unpacking initramfs...\n"); |
| 639 | err = unpack_to_rootfs((char *)initrd_start, | 641 | err = unpack_to_rootfs((char *)initrd_start, |
| 640 | initrd_end - initrd_start); | 642 | initrd_end - initrd_start); |
| 641 | if (err) { | 643 | if (err) |
| 642 | printk(KERN_EMERG "Initramfs unpacking failed: %s\n", err); | 644 | printk(KERN_EMERG "Initramfs unpacking failed: %s\n", err); |
| 643 | clean_rootfs(); | ||
| 644 | } | ||
| 645 | free_initrd(); | 645 | free_initrd(); |
| 646 | #endif | 646 | #endif |
| 647 | } | 647 | } |
| @@ -471,6 +471,7 @@ static int trie_delete_elem(struct bpf_map *map, void *_key) · 470 unmodified lines | |||
| 471 | } | 471 | } |
| 472 | 472 | ||
| 473 | if (!node || node->prefixlen != key->prefixlen || | 473 | if (!node || node->prefixlen != key->prefixlen || |
| 474 | node->prefixlen != matchlen || | ||
| 474 | (node->flags & LPM_TREE_NODE_FLAG_IM)) { | 475 | (node->flags & LPM_TREE_NODE_FLAG_IM)) { |
| 475 | ret = -ENOENT; | 476 | ret = -ENOENT; |
| 476 | goto out; | 477 | goto out; |
| @@ -44,7 +44,7 @@ static void do_up_read(struct irq_work *entry) · 43 unmodified lines | |||
| 44 | struct stack_map_irq_work *work; | 44 | struct stack_map_irq_work *work; |
| 45 | 45 | ||
| 46 | work = container_of(entry, struct stack_map_irq_work, irq_work); | 46 | work = container_of(entry, struct stack_map_irq_work, irq_work); |
| 47 | up_read(work->sem); | 47 | up_read_non_owner(work->sem); |
| 48 | work->sem = NULL; | 48 | work->sem = NULL; |
| 49 | } | 49 | } |
| 50 | 50 | ||
| @@ -338,6 +338,12 @@ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs, · 287 unmodified lines | |||
| 338 | } else { | 338 | } else { |
| 339 | work->sem = ¤t->mm->mmap_sem; | 339 | work->sem = ¤t->mm->mmap_sem; |
| 340 | irq_work_queue(&work->irq_work); | 340 | irq_work_queue(&work->irq_work); |
| 341 | /* | ||
| 342 | * The irq_work will release the mmap_sem with | ||
| 343 | * up_read_non_owner(). The rwsem_release() is called | ||
| 344 | * here to release the lock from lockdep's perspective. | ||
| 345 | */ | ||
| 346 | rwsem_release(¤t->mm->mmap_sem.dep_map, 1, _RET_IP_); | ||
| 341 | } | 347 | } |
| 342 | } | 348 | } |
| 343 | 349 | ||
| @@ -559,12 +559,12 @@ static int map_create(union bpf_attr *attr) · 558 unmodified lines | |||
| 559 | err = bpf_map_new_fd(map, f_flags); | 559 | err = bpf_map_new_fd(map, f_flags); |
| 560 | if (err < 0) { | 560 | if (err < 0) { |
| 561 | /* failed to allocate fd. | 561 | /* failed to allocate fd. |
| 562 | * bpf_map_put() is needed because the above | 562 | * bpf_map_put_with_uref() is needed because the above |
| 563 | * bpf_map_alloc_id() has published the map | 563 | * bpf_map_alloc_id() has published the map |
| 564 | * to the userspace and the userspace may | 564 | * to the userspace and the userspace may |
| 565 | * have refcnt-ed it through BPF_MAP_GET_FD_BY_ID. | 565 | * have refcnt-ed it through BPF_MAP_GET_FD_BY_ID. |
| 566 | */ | 566 | */ |
| 567 | bpf_map_put(map); | 567 | bpf_map_put_with_uref(map); |
| 568 | return err; | 568 | return err; |
| 569 | } | 569 | } |
| 570 | 570 | ||
| @@ -1986,7 +1986,7 @@ static int bpf_map_get_fd_by_id(const union bpf_attr *attr) · 1415 unmodified lines | |||
| 1986 | 1986 | ||
| 1987 | fd = bpf_map_new_fd(map, f_flags); | 1987 | fd = bpf_map_new_fd(map, f_flags); |
| 1988 | if (fd < 0) | 1988 | if (fd < 0) |
| 1989 | bpf_map_put(map); | 1989 | bpf_map_put_with_uref(map); |
| 1990 | 1990 | ||
| 1991 | return fd; | 1991 | return fd; |
| 1992 | } | 1992 | } |
| @@ -1617,12 +1617,13 @@ static int check_flow_keys_access(struct bpf_verifier_env *env, int off, · 1616 unmodified lines | |||
| 1617 | return 0; | 1617 | return 0; |
| 1618 | } | 1618 | } |
| 1619 | 1619 | ||
| 1620 | static int check_sock_access(struct bpf_verifier_env *env, u32 regno, int off, | 1620 | static int check_sock_access(struct bpf_verifier_env *env, int insn_idx, |
| 1621 | int size, enum bpf_access_type t) | 1621 | u32 regno, int off, int size, |
| 1622 | enum bpf_access_type t) | ||
| 1622 | { | 1623 | { |
| 1623 | struct bpf_reg_state *regs = cur_regs(env); | 1624 | struct bpf_reg_state *regs = cur_regs(env); |
| 1624 | struct bpf_reg_state *reg = ®s[regno]; | 1625 | struct bpf_reg_state *reg = ®s[regno]; |
| 1625 | struct bpf_insn_access_aux info; | 1626 | struct bpf_insn_access_aux info = {}; |
| 1626 | 1627 | ||
| 1627 | if (reg->smin_value < 0) { | 1628 | if (reg->smin_value < 0) { |
| 1628 | verbose(env, "R%d min value is negative, either use unsigned index or do a if (index >=0) check.\n", | 1629 | verbose(env, "R%d min value is negative, either use unsigned index or do a if (index >=0) check.\n", |
| @@ -1636,6 +1637,8 @@ static int check_sock_access(struct bpf_verifier_env *env, u32 regno, int off, · 7 unmodified lines | |||
| 1636 | return -EACCES; | 1637 | return -EACCES; |
| 1637 | } | 1638 | } |
| 1638 | 1639 | ||
| 1640 | env->insn_aux_data[insn_idx].ctx_field_size = info.ctx_field_size; | ||
| 1641 | |||
| 1639 | return 0; | 1642 | return 0; |
| 1640 | } | 1643 | } |
| 1641 | 1644 | ||
| @@ -2032,7 +2035,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn · 390 unmodified lines | |||
| 2032 | verbose(env, "cannot write into socket\n"); | 2035 | verbose(env, "cannot write into socket\n"); |
| 2033 | return -EACCES; | 2036 | return -EACCES; |
| 2034 | } | 2037 | } |
| 2035 | err = check_sock_access(env, regno, off, size, t); | 2038 | err = check_sock_access(env, insn_idx, regno, off, size, t); |
| 2036 | if (!err && value_regno >= 0) | 2039 | if (!err && value_regno >= 0) |
| 2037 | mark_reg_unknown(env, regs, value_regno); | 2040 | mark_reg_unknown(env, regs, value_regno); |
| 2038 | } else { | 2041 | } else { |
| @@ -6917,7 +6920,8 @@ static int fixup_bpf_calls(struct bpf_verifier_env *env) · 4878 unmodified lines | |||
| 6917 | u32 off_reg; | 6920 | u32 off_reg; |
| 6918 | 6921 | ||
| 6919 | aux = &env->insn_aux_data[i + delta]; | 6922 | aux = &env->insn_aux_data[i + delta]; |
| 6920 | if (!aux->alu_state) | 6923 | if (!aux->alu_state || |
| 6924 | aux->alu_state == BPF_ALU_NON_POINTER) | ||
| 6921 | continue; | 6925 | continue; |
| 6922 | 6926 | ||
| 6923 | isneg = aux->alu_state & BPF_ALU_NEG_VALUE; | 6927 | isneg = aux->alu_state & BPF_ALU_NEG_VALUE; |
| @@ -322,7 +322,7 @@ static bool update_stats(struct psi_group *group) · 321 unmodified lines | |||
| 322 | expires = group->next_update; | 322 | expires = group->next_update; |
| 323 | if (now < expires) | 323 | if (now < expires) |
| 324 | goto out; | 324 | goto out; |
| 325 | if (now - expires > psi_period) | 325 | if (now - expires >= psi_period) |
| 326 | missed_periods = div_u64(now - expires, psi_period); | 326 | missed_periods = div_u64(now - expires, psi_period); |
| 327 | 327 | ||
| 328 | /* | 328 | /* |
| @@ -3384,6 +3384,8 @@ static void print_func_help_header_irq(struct trace_buffer *buf, struct seq_file · 3383 unmodified lines | |||
| 3384 | const char tgid_space[] = " "; | 3384 | const char tgid_space[] = " "; |
| 3385 | const char space[] = " "; | 3385 | const char space[] = " "; |
| 3386 | 3386 | ||
| 3387 | print_event_info(buf, m); | ||
| 3388 | |||
| 3387 | seq_printf(m, "# %s _-----=> irqs-off\n", | 3389 | seq_printf(m, "# %s _-----=> irqs-off\n", |
| 3388 | tgid ? tgid_space : space); | 3390 | tgid ? tgid_space : space); |
| 3389 | seq_printf(m, "# %s / _----=> need-resched\n", | 3391 | seq_printf(m, "# %s / _----=> need-resched\n", |
| @@ -861,22 +861,14 @@ static const struct file_operations kprobe_profile_ops = { · 860 unmodified lines | |||
| 861 | static nokprobe_inline int | 861 | static nokprobe_inline int |
| 862 | fetch_store_strlen(unsigned long addr) | 862 | fetch_store_strlen(unsigned long addr) |
| 863 | { | 863 | { |
| 864 | mm_segment_t old_fs; | ||
| 865 | int ret, len = 0; | 864 | int ret, len = 0; |
| 866 | u8 c; | 865 | u8 c; |
| 867 | 866 | ||
| 868 | old_fs = get_fs(); | ||
| 869 | set_fs(KERNEL_DS); | ||
| 870 | pagefault_disable(); | ||
| 871 | |||
| 872 | do { | 867 | do { |
| 873 | ret = __copy_from_user_inatomic(&c, (u8 *)addr + len, 1); | 868 | ret = probe_mem_read(&c, (u8 *)addr + len, 1); |
| 874 | len++; | 869 | len++; |
| 875 | } while (c && ret == 0 && len < MAX_STRING_SIZE); | 870 | } while (c && ret == 0 && len < MAX_STRING_SIZE); |
| 876 | 871 | ||
| 877 | pagefault_enable(); | ||
| 878 | set_fs(old_fs); | ||
| 879 | |||
| 880 | return (ret < 0) ? ret : len; | 872 | return (ret < 0) ? ret : len; |
| 881 | } | 873 | } |
| 882 | 874 | ||
| @@ -113,6 +113,28 @@ config KASAN_INLINE · 112 unmodified lines | |||
| 113 | 113 | ||
| 114 | endchoice | 114 | endchoice |
| 115 | 115 | ||
| 116 | config KASAN_STACK_ENABLE | ||
| 117 | bool "Enable stack instrumentation (unsafe)" if CC_IS_CLANG && !COMPILE_TEST | ||
| 118 | default !(CLANG_VERSION < 90000) | ||
| 119 | depends on KASAN | ||
| 120 | help | ||
| 121 | The LLVM stack address sanitizer has a know problem that | ||
| 122 | causes excessive stack usage in a lot of functions, see | ||
| 123 | https://bugs.llvm.org/show_bug.cgi?id=38809 | ||
| 124 | Disabling asan-stack makes it safe to run kernels build | ||
| 125 | with clang-8 with KASAN enabled, though it loses some of | ||
| 126 | the functionality. | ||
| 127 | This feature is always disabled when compile-testing with clang-8 | ||
| 128 | or earlier to avoid cluttering the output in stack overflow | ||
| 129 | warnings, but clang-8 users can still enable it for builds without | ||
| 130 | CONFIG_COMPILE_TEST. On gcc and later clang versions it is | ||
| 131 | assumed to always be safe to use and enabled by default. | ||
| 132 | |||
| 133 | config KASAN_STACK | ||
| 134 | int | ||
| 135 | default 1 if KASAN_STACK_ENABLE || CC_IS_GCC | ||
| 136 | default 0 | ||
| 137 | |||
| 116 | config KASAN_S390_4_LEVEL_PAGING | 138 | config KASAN_S390_4_LEVEL_PAGING |
| 117 | bool "KASan: use 4-level paging" | 139 | bool "KASan: use 4-level paging" |
| 118 | depends on KASAN && S390 | 140 | depends on KASAN && S390 |
| @@ -768,9 +768,11 @@ all_leaves_cluster_together: · 767 unmodified lines | |||
| 768 | new_s0->index_key[i] = | 768 | new_s0->index_key[i] = |
| 769 | ops->get_key_chunk(index_key, i * ASSOC_ARRAY_KEY_CHUNK_SIZE); | 769 | ops->get_key_chunk(index_key, i * ASSOC_ARRAY_KEY_CHUNK_SIZE); |
| 770 | 770 | ||
| 771 | blank = ULONG_MAX << (level & ASSOC_ARRAY_KEY_CHUNK_MASK); | 771 | if (level & ASSOC_ARRAY_KEY_CHUNK_MASK) { |
| 772 | pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, level, blank); | 772 | blank = ULONG_MAX << (level & ASSOC_ARRAY_KEY_CHUNK_MASK); |
| 773 | new_s0->index_key[keylen - 1] &= ~blank; | 773 | pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, level, blank); |
| 774 | new_s0->index_key[keylen - 1] &= ~blank; | ||
| 775 | } | ||
| 774 | 776 | ||
| 775 | /* This now reduces to a node splitting exercise for which we'll need | 777 | /* This now reduces to a node splitting exercise for which we'll need |
| 776 | * to regenerate the disparity table. | 778 | * to regenerate the disparity table. |
| @@ -44,7 +44,7 @@ const struct trace_print_flags vmaflag_names[] = { · 43 unmodified lines | |||
| 44 | 44 | ||
| 45 | void __dump_page(struct page *page, const char *reason) | 45 | void __dump_page(struct page *page, const char *reason) |
| 46 | { | 46 | { |
| 47 | struct address_space *mapping = page_mapping(page); | 47 | struct address_space *mapping; |
| 48 | bool page_poisoned = PagePoisoned(page); | 48 | bool page_poisoned = PagePoisoned(page); |
| 49 | int mapcount; | 49 | int mapcount; |
| 50 | 50 | ||
| @@ -58,6 +58,8 @@ void __dump_page(struct page *page, const char *reason) · 7 unmodified lines | |||
| 58 | goto hex_only; | 58 | goto hex_only; |
| 59 | } | 59 | } |
| 60 | 60 | ||
| 61 | mapping = page_mapping(page); | ||
| 62 | |||
| 61 | /* | 63 | /* |
| 62 | * Avoid VM_BUG_ON() in page_mapcount(). | 64 | * Avoid VM_BUG_ON() in page_mapcount(). |
| 63 | * page->_mapcount space in struct page is used by sl[aou]b pages to | 65 | * page->_mapcount space in struct page is used by sl[aou]b pages to |
| @@ -3624,7 +3624,6 @@ retry_avoidcopy: · 3623 unmodified lines | |||
| 3624 | copy_user_huge_page(new_page, old_page, address, vma, | 3624 | copy_user_huge_page(new_page, old_page, address, vma, |
| 3625 | pages_per_huge_page(h)); | 3625 | pages_per_huge_page(h)); |
| 3626 | __SetPageUptodate(new_page); | 3626 | __SetPageUptodate(new_page); |
| 3627 | set_page_huge_active(new_page); | ||
| 3628 | 3627 | ||
| 3629 | mmu_notifier_range_init(&range, mm, haddr, haddr + huge_page_size(h)); | 3628 | mmu_notifier_range_init(&range, mm, haddr, haddr + huge_page_size(h)); |
| 3630 | mmu_notifier_invalidate_range_start(&range); | 3629 | mmu_notifier_invalidate_range_start(&range); |
| @@ -3645,6 +3644,7 @@ retry_avoidcopy: · 14 unmodified lines | |||
| 3645 | make_huge_pte(vma, new_page, 1)); | 3644 | make_huge_pte(vma, new_page, 1)); |
| 3646 | page_remove_rmap(old_page, true); | 3645 | page_remove_rmap(old_page, true); |
| 3647 | hugepage_add_new_anon_rmap(new_page, vma, haddr); | 3646 | hugepage_add_new_anon_rmap(new_page, vma, haddr); |
| 3647 | set_page_huge_active(new_page); | ||
| 3648 | /* Make the old page be freed below */ | 3648 | /* Make the old page be freed below */ |
| 3649 | new_page = old_page; | 3649 | new_page = old_page; |
| 3650 | } | 3650 | } |
| @@ -3729,6 +3729,7 @@ static vm_fault_t hugetlb_no_page(struct mm_struct *mm, · 78 unmodified lines | |||
| 3729 | pte_t new_pte; | 3729 | pte_t new_pte; |
| 3730 | spinlock_t *ptl; | 3730 | spinlock_t *ptl; |
| 3731 | unsigned long haddr = address & huge_page_mask(h); | 3731 | unsigned long haddr = address & huge_page_mask(h); |
| 3732 | bool new_page = false; | ||
| 3732 | 3733 | ||
| 3733 | /* | 3734 | /* |
| 3734 | * Currently, we are forced to kill the process in the event the | 3735 | * Currently, we are forced to kill the process in the event the |
| @@ -3790,7 +3791,7 @@ retry: · 55 unmodified lines | |||
| 3790 | } | 3791 | } |
| 3791 | clear_huge_page(page, address, pages_per_huge_page(h)); | 3792 | clear_huge_page(page, address, pages_per_huge_page(h)); |
| 3792 | __SetPageUptodate(page); | 3793 | __SetPageUptodate(page); |
| 3793 | set_page_huge_active(page); | 3794 | new_page = true; |
| 3794 | 3795 | ||
| 3795 | if (vma->vm_flags & VM_MAYSHARE) { | 3796 | if (vma->vm_flags & VM_MAYSHARE) { |
| 3796 | int err = huge_add_to_page_cache(page, mapping, idx); | 3797 | int err = huge_add_to_page_cache(page, mapping, idx); |
| @@ -3861,6 +3862,15 @@ retry: · 64 unmodified lines | |||
| 3861 | } | 3862 | } |
| 3862 | 3863 | ||
| 3863 | spin_unlock(ptl); | 3864 | spin_unlock(ptl); |
| 3865 | |||
| 3866 | /* | ||
| 3867 | * Only make newly allocated pages active. Existing pages found | ||
| 3868 | * in the pagecache could be !page_huge_active() if they have been | ||
| 3869 | * isolated for migration. | ||
| 3870 | */ | ||
| 3871 | if (new_page) | ||
| 3872 | set_page_huge_active(page); | ||
| 3873 | |||
| 3864 | unlock_page(page); | 3874 | unlock_page(page); |
| 3865 | out: | 3875 | out: |
| 3866 | return ret; | 3876 | return ret; |
| @@ -4095,7 +4105,6 @@ int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, · 228 unmodified lines | |||
| 4095 | * the set_pte_at() write. | 4105 | * the set_pte_at() write. |
| 4096 | */ | 4106 | */ |
| 4097 | __SetPageUptodate(page); | 4107 | __SetPageUptodate(page); |
| 4098 | set_page_huge_active(page); | ||
| 4099 | 4108 | ||
| 4100 | mapping = dst_vma->vm_file->f_mapping; | 4109 | mapping = dst_vma->vm_file->f_mapping; |
| 4101 | idx = vma_hugecache_offset(h, dst_vma, dst_addr); | 4110 | idx = vma_hugecache_offset(h, dst_vma, dst_addr); |
| @@ -4163,6 +4172,7 @@ int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, · 61 unmodified lines | |||
| 4163 | update_mmu_cache(dst_vma, dst_addr, dst_pte); | 4172 | update_mmu_cache(dst_vma, dst_addr, dst_pte); |
| 4164 | 4173 | ||
| 4165 | spin_unlock(ptl); | 4174 | spin_unlock(ptl); |
| 4175 | set_page_huge_active(page); | ||
| 4166 | if (vm_shared) | 4176 | if (vm_shared) |
| 4167 | unlock_page(page); | 4177 | unlock_page(page); |
| 4168 | ret = 0; | 4178 | ret = 0; |
| @@ -7,6 +7,8 @@ KCOV_INSTRUMENT := n · 6 unmodified lines | |||
| 7 | 7 | ||
| 8 | CFLAGS_REMOVE_common.o = -pg | 8 | CFLAGS_REMOVE_common.o = -pg |
| 9 | CFLAGS_REMOVE_generic.o = -pg | 9 | CFLAGS_REMOVE_generic.o = -pg |
| 10 | CFLAGS_REMOVE_tags.o = -pg | ||
| 11 | |||
| 10 | # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 | 12 | # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 |
| 11 | # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 | 13 | # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 |
| 12 | 14 | ||
| @@ -361,10 +361,15 @@ void kasan_poison_object_data(struct kmem_cache *cache, void *object) · 360 unmodified lines | |||
| 361 | * get different tags. | 361 | * get different tags. |
| 362 | */ | 362 | */ |
| 363 | static u8 assign_tag(struct kmem_cache *cache, const void *object, | 363 | static u8 assign_tag(struct kmem_cache *cache, const void *object, |
| 364 | bool init, bool krealloc) | 364 | bool init, bool keep_tag) |
| 365 | { | 365 | { |
| 366 | /* Reuse the same tag for krealloc'ed objects. */ | 366 | /* |
| 367 | if (krealloc) | 367 | * 1. When an object is kmalloc()'ed, two hooks are called: |
| 368 | * kasan_slab_alloc() and kasan_kmalloc(). We assign the | ||
| 369 | * tag only in the first one. | ||
| 370 | * 2. We reuse the same tag for krealloc'ed objects. | ||
| 371 | */ | ||
| 372 | if (keep_tag) | ||
| 368 | return get_tag(object); | 373 | return get_tag(object); |
| 369 | 374 | ||
| 370 | /* | 375 | /* |
| @@ -405,12 +410,6 @@ void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, · 34 unmodified lines | |||
| 405 | return (void *)object; | 410 | return (void *)object; |
| 406 | } | 411 | } |
| 407 | 412 | ||
| 408 | void * __must_check kasan_slab_alloc(struct kmem_cache *cache, void *object, | ||
| 409 | gfp_t flags) | ||
| 410 | { | ||
| 411 | return kasan_kmalloc(cache, object, cache->object_size, flags); | ||
| 412 | } | ||
| 413 | |||
| 414 | static inline bool shadow_invalid(u8 tag, s8 shadow_byte) | 413 | static inline bool shadow_invalid(u8 tag, s8 shadow_byte) |
| 415 | { | 414 | { |
| 416 | if (IS_ENABLED(CONFIG_KASAN_GENERIC)) | 415 | if (IS_ENABLED(CONFIG_KASAN_GENERIC)) |
| @@ -467,7 +466,7 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) · 50 unmodified lines | |||
| 467 | } | 466 | } |
| 468 | 467 | ||
| 469 | static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, | 468 | static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, |
| 470 | size_t size, gfp_t flags, bool krealloc) | 469 | size_t size, gfp_t flags, bool keep_tag) |
| 471 | { | 470 | { |
| 472 | unsigned long redzone_start; | 471 | unsigned long redzone_start; |
| 473 | unsigned long redzone_end; | 472 | unsigned long redzone_end; |
| @@ -485,7 +484,7 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, · 11 unmodified lines | |||
| 485 | KASAN_SHADOW_SCALE_SIZE); | 484 | KASAN_SHADOW_SCALE_SIZE); |
| 486 | 485 | ||
| 487 | if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) | 486 | if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) |
| 488 | tag = assign_tag(cache, object, false, krealloc); | 487 | tag = assign_tag(cache, object, false, keep_tag); |
| 489 | 488 | ||
| 490 | /* Tag is ignored in set_tag without CONFIG_KASAN_SW_TAGS */ | 489 | /* Tag is ignored in set_tag without CONFIG_KASAN_SW_TAGS */ |
| 491 | kasan_unpoison_shadow(set_tag(object, tag), size); | 490 | kasan_unpoison_shadow(set_tag(object, tag), size); |
| @@ -498,10 +497,16 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, · 6 unmodified lines | |||
| 498 | return set_tag(object, tag); | 497 | return set_tag(object, tag); |
| 499 | } | 498 | } |
| 500 | 499 | ||
| 500 | void * __must_check kasan_slab_alloc(struct kmem_cache *cache, void *object, | ||
| 501 | gfp_t flags) | ||
| 502 | { | ||
| 503 | return __kasan_kmalloc(cache, object, cache->object_size, flags, false); | ||
| 504 | } | ||
| 505 | |||
| 501 | void * __must_check kasan_kmalloc(struct kmem_cache *cache, const void *object, | 506 | void * __must_check kasan_kmalloc(struct kmem_cache *cache, const void *object, |
| 502 | size_t size, gfp_t flags) | 507 | size_t size, gfp_t flags) |
| 503 | { | 508 | { |
| 504 | return __kasan_kmalloc(cache, object, size, flags, false); | 509 | return __kasan_kmalloc(cache, object, size, flags, true); |
| 505 | } | 510 | } |
| 506 | EXPORT_SYMBOL(kasan_kmalloc); | 511 | EXPORT_SYMBOL(kasan_kmalloc); |
| 507 | 512 | ||
| @@ -46,7 +46,7 @@ void kasan_init_tags(void) · 45 unmodified lines | |||
| 46 | int cpu; | 46 | int cpu; |
| 47 | 47 | ||
| 48 | for_each_possible_cpu(cpu) | 48 | for_each_possible_cpu(cpu) |
| 49 | per_cpu(prng_state, cpu) = get_random_u32(); | 49 | per_cpu(prng_state, cpu) = (u32)get_cycles(); |
| 50 | } | 50 | } |
| 51 | 51 | ||
| 52 | /* | 52 | /* |
| @@ -574,6 +574,7 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size, · 573 unmodified lines | |||
| 574 | unsigned long flags; | 574 | unsigned long flags; |
| 575 | struct kmemleak_object *object, *parent; | 575 | struct kmemleak_object *object, *parent; |
| 576 | struct rb_node **link, *rb_parent; | 576 | struct rb_node **link, *rb_parent; |
| 577 | unsigned long untagged_ptr; | ||
| 577 | 578 | ||
| 578 | object = kmem_cache_alloc(object_cache, gfp_kmemleak_mask(gfp)); | 579 | object = kmem_cache_alloc(object_cache, gfp_kmemleak_mask(gfp)); |
| 579 | if (!object) { | 580 | if (!object) { |
| @@ -619,8 +620,9 @@ static struct kmemleak_object *create_object(unsigned long ptr, size_t size, · 39 unmodified lines | |||
| 619 | 620 | ||
| 620 | write_lock_irqsave(&kmemleak_lock, flags); | 621 | write_lock_irqsave(&kmemleak_lock, flags); |
| 621 | 622 | ||
| 622 | min_addr = min(min_addr, ptr); | 623 | untagged_ptr = (unsigned long)kasan_reset_tag((void *)ptr); |
| 623 | max_addr = max(max_addr, ptr + size); | 624 | min_addr = min(min_addr, untagged_ptr); |
| 625 | max_addr = max(max_addr, untagged_ptr + size); | ||
| 624 | link = &object_tree_root.rb_node; | 626 | link = &object_tree_root.rb_node; |
| 625 | rb_parent = NULL; | 627 | rb_parent = NULL; |
| 626 | while (*link) { | 628 | while (*link) { |
| @@ -1333,6 +1335,7 @@ static void scan_block(void *_start, void *_end, · 706 unmodified lines | |||
| 1333 | unsigned long *start = PTR_ALIGN(_start, BYTES_PER_POINTER); | 1335 | unsigned long *start = PTR_ALIGN(_start, BYTES_PER_POINTER); |
| 1334 | unsigned long *end = _end - (BYTES_PER_POINTER - 1); | 1336 | unsigned long *end = _end - (BYTES_PER_POINTER - 1); |
| 1335 | unsigned long flags; | 1337 | unsigned long flags; |
| 1338 | unsigned long untagged_ptr; | ||
| 1336 | 1339 | ||
| 1337 | read_lock_irqsave(&kmemleak_lock, flags); | 1340 | read_lock_irqsave(&kmemleak_lock, flags); |
| 1338 | for (ptr = start; ptr < end; ptr++) { | 1341 | for (ptr = start; ptr < end; ptr++) { |
| @@ -1347,7 +1350,8 @@ static void scan_block(void *_start, void *_end, · 8 unmodified lines | |||
| 1347 | pointer = *ptr; | 1350 | pointer = *ptr; |
| 1348 | kasan_enable_current(); | 1351 | kasan_enable_current(); |
| 1349 | 1352 | ||
| 1350 | if (pointer < min_addr || pointer >= max_addr) | 1353 | untagged_ptr = (unsigned long)kasan_reset_tag((void *)pointer); |
| 1354 | if (untagged_ptr < min_addr || untagged_ptr >= max_addr) | ||
| 1351 | continue; | 1355 | continue; |
| 1352 | 1356 | ||
| 1353 | /* | 1357 | /* |
| @@ -30,10 +30,8 @@ long __probe_kernel_read(void *dst, const void *src, size_t size) · 29 unmodified lines | |||
| 30 | 30 | ||
| 31 | set_fs(KERNEL_DS); | 31 | set_fs(KERNEL_DS); |
| 32 | pagefault_disable(); | 32 | pagefault_disable(); |
| 33 | current->kernel_uaccess_faults_ok++; | ||
| 34 | ret = __copy_from_user_inatomic(dst, | 33 | ret = __copy_from_user_inatomic(dst, |
| 35 | (__force const void __user *)src, size); | 34 | (__force const void __user *)src, size); |
| 36 | current->kernel_uaccess_faults_ok--; | ||
| 37 | pagefault_enable(); | 35 | pagefault_enable(); |
| 38 | set_fs(old_fs); | 36 | set_fs(old_fs); |
| 39 | 37 | ||
| @@ -60,9 +58,7 @@ long __probe_kernel_write(void *dst, const void *src, size_t size) · 20 unmodified lines | |||
| 60 | 58 | ||
| 61 | set_fs(KERNEL_DS); | 59 | set_fs(KERNEL_DS); |
| 62 | pagefault_disable(); | 60 | pagefault_disable(); |
| 63 | current->kernel_uaccess_faults_ok++; | ||
| 64 | ret = __copy_to_user_inatomic((__force void __user *)dst, src, size); | 61 | ret = __copy_to_user_inatomic((__force void __user *)dst, src, size); |
| 65 | current->kernel_uaccess_faults_ok--; | ||
| 66 | pagefault_enable(); | 62 | pagefault_enable(); |
| 67 | set_fs(old_fs); | 63 | set_fs(old_fs); |
| 68 | 64 | ||
| @@ -98,13 +94,11 @@ long strncpy_from_unsafe(char *dst, const void *unsafe_addr, long count) · 29 unmodified lines | |||
| 98 | 94 | ||
| 99 | set_fs(KERNEL_DS); | 95 | set_fs(KERNEL_DS); |
| 100 | pagefault_disable(); | 96 | pagefault_disable(); |
| 101 | current->kernel_uaccess_faults_ok++; | ||
| 102 | 97 | ||
| 103 | do { | 98 | do { |
| 104 | ret = __get_user(*dst++, (const char __user __force *)src++); | 99 | ret = __get_user(*dst++, (const char __user __force *)src++); |
| 105 | } while (dst[-1] && ret == 0 && src - unsafe_addr < count); | 100 | } while (dst[-1] && ret == 0 && src - unsafe_addr < count); |
| 106 | 101 | ||
| 107 | current->kernel_uaccess_faults_ok--; | ||
| 108 | dst[-1] = '\0'; | 102 | dst[-1] = '\0'; |
| 109 | pagefault_enable(); | 103 | pagefault_enable(); |
| 110 | set_fs(old_fs); | 104 | set_fs(old_fs); |
| @@ -1188,11 +1188,13 @@ static inline int pageblock_free(struct page *page) · 1187 unmodified lines | |||
| 1188 | return PageBuddy(page) && page_order(page) >= pageblock_order; | 1188 | return PageBuddy(page) && page_order(page) >= pageblock_order; |
| 1189 | } | 1189 | } |
| 1190 | 1190 | ||
| 1191 | /* Return the start of the next active pageblock after a given page */ | 1191 | /* Return the pfn of the start of the next active pageblock after a given pfn */ |
| 1192 | static struct page *next_active_pageblock(struct page *page) | 1192 | static unsigned long next_active_pageblock(unsigned long pfn) |
| 1193 | { | 1193 | { |
| 1194 | struct page *page = pfn_to_page(pfn); | ||
| 1195 | |||
| 1194 | /* Ensure the starting page is pageblock-aligned */ | 1196 | /* Ensure the starting page is pageblock-aligned */ |
| 1195 | BUG_ON(page_to_pfn(page) & (pageblock_nr_pages - 1)); | 1197 | BUG_ON(pfn & (pageblock_nr_pages - 1)); |
| 1196 | 1198 | ||
| 1197 | /* If the entire pageblock is free, move to the end of free page */ | 1199 | /* If the entire pageblock is free, move to the end of free page */ |
| 1198 | if (pageblock_free(page)) { | 1200 | if (pageblock_free(page)) { |
| @@ -1200,16 +1202,16 @@ static struct page *next_active_pageblock(struct page *page) · 1 unmodified line | |||
| 1200 | /* be careful. we don't have locks, page_order can be changed.*/ | 1202 | /* be careful. we don't have locks, page_order can be changed.*/ |
| 1201 | order = page_order(page); | 1203 | order = page_order(page); |
| 1202 | if ((order < MAX_ORDER) && (order >= pageblock_order)) | 1204 | if ((order < MAX_ORDER) && (order >= pageblock_order)) |
| 1203 | return page + (1 << order); | 1205 | return pfn + (1 << order); |
| 1204 | } | 1206 | } |
| 1205 | 1207 | ||
| 1206 | return page + pageblock_nr_pages; | 1208 | return pfn + pageblock_nr_pages; |
| 1207 | } | 1209 | } |
| 1208 | 1210 | ||
| 1209 | static bool is_pageblock_removable_nolock(struct page *page) | 1211 | static bool is_pageblock_removable_nolock(unsigned long pfn) |
| 1210 | { | 1212 | { |
| 1213 | struct page *page = pfn_to_page(pfn); | ||
| 1211 | struct zone *zone; | 1214 | struct zone *zone; |
| 1212 | unsigned long pfn; | ||
| 1213 | 1215 | ||
| 1214 | /* | 1216 | /* |
| 1215 | * We have to be careful here because we are iterating over memory | 1217 | * We have to be careful here because we are iterating over memory |
| @@ -1232,13 +1234,14 @@ static bool is_pageblock_removable_nolock(struct page *page) · 16 unmodified lines | |||
| 1232 | /* Checks if this range of memory is likely to be hot-removable. */ | 1234 | /* Checks if this range of memory is likely to be hot-removable. */ |
| 1233 | bool is_mem_section_removable(unsigned long start_pfn, unsigned long nr_pages) | 1235 | bool is_mem_section_removable(unsigned long start_pfn, unsigned long nr_pages) |
| 1234 | { | 1236 | { |
| 1235 | struct page *page = pfn_to_page(start_pfn); | 1237 | unsigned long end_pfn, pfn; |
| 1236 | unsigned long end_pfn = min(start_pfn + nr_pages, zone_end_pfn(page_zone(page))); | 1238 | |
| 1237 | struct page *end_page = pfn_to_page(end_pfn); | 1239 | end_pfn = min(start_pfn + nr_pages, |
| 1240 | zone_end_pfn(page_zone(pfn_to_page(start_pfn)))); | ||
| 1238 | 1241 | ||
| 1239 | /* Check the starting page of each pageblock within the range */ | 1242 | /* Check the starting page of each pageblock within the range */ |
| 1240 | for (; page < end_page; page = next_active_pageblock(page)) { | 1243 | for (pfn = start_pfn; pfn < end_pfn; pfn = next_active_pageblock(pfn)) { |
| 1241 | if (!is_pageblock_removable_nolock(page)) | 1244 | if (!is_pageblock_removable_nolock(pfn)) |
| 1242 | return false; | 1245 | return false; |
| 1243 | cond_resched(); | 1246 | cond_resched(); |
| 1244 | } | 1247 | } |
| @@ -1314,7 +1314,7 @@ static int copy_nodes_to_user(unsigned long __user *mask, unsigned long maxnode, · 1313 unmodified lines | |||
| 1314 | nodemask_t *nodes) | 1314 | nodemask_t *nodes) |
| 1315 | { | 1315 | { |
| 1316 | unsigned long copy = ALIGN(maxnode-1, 64) / 8; | 1316 | unsigned long copy = ALIGN(maxnode-1, 64) / 8; |
| 1317 | const int nbytes = BITS_TO_LONGS(MAX_NUMNODES) * sizeof(long); | 1317 | unsigned int nbytes = BITS_TO_LONGS(nr_node_ids) * sizeof(long); |
| 1318 | 1318 | ||
| 1319 | if (copy > nbytes) { | 1319 | if (copy > nbytes) { |
| 1320 | if (copy > PAGE_SIZE) | 1320 | if (copy > PAGE_SIZE) |
| @@ -1491,7 +1491,7 @@ static int kernel_get_mempolicy(int __user *policy, · 170 unmodified lines | |||
| 1491 | int uninitialized_var(pval); | 1491 | int uninitialized_var(pval); |
| 1492 | nodemask_t nodes; | 1492 | nodemask_t nodes; |
| 1493 | 1493 | ||
| 1494 | if (nmask != NULL && maxnode < MAX_NUMNODES) | 1494 | if (nmask != NULL && maxnode < nr_node_ids) |
| 1495 | return -EINVAL; | 1495 | return -EINVAL; |
| 1496 | 1496 | ||
| 1497 | err = do_get_mempolicy(&pval, &nodes, addr, flags); | 1497 | err = do_get_mempolicy(&pval, &nodes, addr, flags); |
| @@ -1527,7 +1527,7 @@ COMPAT_SYSCALL_DEFINE5(get_mempolicy, int __user *, policy, · 29 unmodified lines | |||
| 1527 | unsigned long nr_bits, alloc_size; | 1527 | unsigned long nr_bits, alloc_size; |
| 1528 | DECLARE_BITMAP(bm, MAX_NUMNODES); | 1528 | DECLARE_BITMAP(bm, MAX_NUMNODES); |
| 1529 | 1529 | ||
| 1530 | nr_bits = min_t(unsigned long, maxnode-1, MAX_NUMNODES); | 1530 | nr_bits = min_t(unsigned long, maxnode-1, nr_node_ids); |
| 1531 | alloc_size = ALIGN(nr_bits, BITS_PER_LONG) / 8; | 1531 | alloc_size = ALIGN(nr_bits, BITS_PER_LONG) / 8; |
| 1532 | 1532 | ||
| 1533 | if (nmask) | 1533 | if (nmask) |
| @@ -1315,6 +1315,16 @@ static int unmap_and_move_huge_page(new_page_t get_new_page, · 1314 unmodified lines | |||
| 1315 | lock_page(hpage); | 1315 | lock_page(hpage); |
| 1316 | } | 1316 | } |
| 1317 | 1317 | ||
| 1318 | /* | ||
| 1319 | * Check for pages which are in the process of being freed. Without | ||
| 1320 | * page_mapping() set, hugetlbfs specific move page routine will not | ||
| 1321 | * be called and we could leak usage counts for subpools. | ||
| 1322 | */ | ||
| 1323 | if (page_private(hpage) && !page_mapping(hpage)) { | ||
| 1324 | rc = -EBUSY; | ||
| 1325 | goto out_unlock; | ||
| 1326 | } | ||
| 1327 | |||
| 1318 | if (PageAnon(hpage)) | 1328 | if (PageAnon(hpage)) |
| 1319 | anon_vma = page_get_anon_vma(hpage); | 1329 | anon_vma = page_get_anon_vma(hpage); |
| 1320 | 1330 | ||
| @@ -1345,6 +1355,7 @@ put_anon: · 24 unmodified lines | |||
| 1345 | put_new_page = NULL; | 1355 | put_new_page = NULL; |
| 1346 | } | 1356 | } |
| 1347 | 1357 | ||
| 1358 | out_unlock: | ||
| 1348 | unlock_page(hpage); | 1359 | unlock_page(hpage); |
| 1349 | out: | 1360 | out: |
| 1350 | if (rc != -EAGAIN) | 1361 | if (rc != -EAGAIN) |
| @@ -2426,12 +2426,11 @@ int expand_downwards(struct vm_area_struct *vma, · 2425 unmodified lines | |||
| 2426 | { | 2426 | { |
| 2427 | struct mm_struct *mm = vma->vm_mm; | 2427 | struct mm_struct *mm = vma->vm_mm; |
| 2428 | struct vm_area_struct *prev; | 2428 | struct vm_area_struct *prev; |
| 2429 | int error; | 2429 | int error = 0; |
| 2430 | 2430 | ||
| 2431 | address &= PAGE_MASK; | 2431 | address &= PAGE_MASK; |
| 2432 | error = security_mmap_addr(address); | 2432 | if (address < mmap_min_addr) |
| 2433 | if (error) | 2433 | return -EPERM; |
| 2434 | return error; | ||
| 2435 | 2434 | ||
| 2436 | /* Enforce stack_guard_gap */ | 2435 | /* Enforce stack_guard_gap */ |
| 2437 | prev = vma->vm_prev; | 2436 | prev = vma->vm_prev; |
| @@ -2170,6 +2170,18 @@ static inline void boost_watermark(struct zone *zone) · 2169 unmodified lines | |||
| 2170 | 2170 | ||
| 2171 | max_boost = mult_frac(zone->_watermark[WMARK_HIGH], | 2171 | max_boost = mult_frac(zone->_watermark[WMARK_HIGH], |
| 2172 | watermark_boost_factor, 10000); | 2172 | watermark_boost_factor, 10000); |
| 2173 | |||
| 2174 | /* | ||
| 2175 | * high watermark may be uninitialised if fragmentation occurs | ||
| 2176 | * very early in boot so do not boost. We do not fall | ||
| 2177 | * through and boost by pageblock_nr_pages as failing | ||
| 2178 | * allocations that early means that reclaim is not going | ||
| 2179 | * to help and it may even be impossible to reclaim the | ||
| 2180 | * boosted watermark resulting in a hang. | ||
| 2181 | */ | ||
| 2182 | if (!max_boost) | ||
| 2183 | return; | ||
| 2184 | |||
| 2173 | max_boost = max(pageblock_nr_pages, max_boost); | 2185 | max_boost = max(pageblock_nr_pages, max_boost); |
| 2174 | 2186 | ||
| 2175 | zone->watermark_boost = min(zone->watermark_boost + pageblock_nr_pages, | 2187 | zone->watermark_boost = min(zone->watermark_boost + pageblock_nr_pages, |
| @@ -4675,11 +4687,11 @@ refill: · 2499 unmodified lines | |||
| 4675 | /* Even if we own the page, we do not use atomic_set(). | 4687 | /* Even if we own the page, we do not use atomic_set(). |
| 4676 | * This would break get_page_unless_zero() users. | 4688 | * This would break get_page_unless_zero() users. |
| 4677 | */ | 4689 | */ |
| 4678 | page_ref_add(page, size); | 4690 | page_ref_add(page, PAGE_FRAG_CACHE_MAX_SIZE); |
| 4679 | 4691 | ||
| 4680 | /* reset page count bias and offset to start of new frag */ | 4692 | /* reset page count bias and offset to start of new frag */ |
| 4681 | nc->pfmemalloc = page_is_pfmemalloc(page); | 4693 | nc->pfmemalloc = page_is_pfmemalloc(page); |
| 4682 | nc->pagecnt_bias = size + 1; | 4694 | nc->pagecnt_bias = PAGE_FRAG_CACHE_MAX_SIZE + 1; |
| 4683 | nc->offset = size; | 4695 | nc->offset = size; |
| 4684 | } | 4696 | } |
| 4685 | 4697 | ||
| @@ -4695,10 +4707,10 @@ refill: · 9 unmodified lines | |||
| 4695 | size = nc->size; | 4707 | size = nc->size; |
| 4696 | #endif | 4708 | #endif |
| 4697 | /* OK, page count is 0, we can safely set it */ | 4709 | /* OK, page count is 0, we can safely set it */ |
| 4698 | set_page_count(page, size + 1); | 4710 | set_page_count(page, PAGE_FRAG_CACHE_MAX_SIZE + 1); |
| 4699 | 4711 | ||
| 4700 | /* reset page count bias and offset to start of new frag */ | 4712 | /* reset page count bias and offset to start of new frag */ |
| 4701 | nc->pagecnt_bias = size + 1; | 4713 | nc->pagecnt_bias = PAGE_FRAG_CACHE_MAX_SIZE + 1; |
| 4702 | offset = size - fragsz; | 4714 | offset = size - fragsz; |
| 4703 | } | 4715 | } |
| 4704 | 4716 | ||
| @@ -2848,16 +2848,20 @@ static int shmem_create(struct inode *dir, struct dentry *dentry, umode_t mode, · 2847 unmodified lines | |||
| 2848 | static int shmem_link(struct dentry *old_dentry, struct inode *dir, struct dentry *dentry) | 2848 | static int shmem_link(struct dentry *old_dentry, struct inode *dir, struct dentry *dentry) |
| 2849 | { | 2849 | { |
| 2850 | struct inode *inode = d_inode(old_dentry); | 2850 | struct inode *inode = d_inode(old_dentry); |
| 2851 | int ret; | 2851 | int ret = 0; |
| 2852 | 2852 | ||
| 2853 | /* | 2853 | /* |
| 2854 | * No ordinary (disk based) filesystem counts links as inodes; | 2854 | * No ordinary (disk based) filesystem counts links as inodes; |
| 2855 | * but each new link needs a new dentry, pinning lowmem, and | 2855 | * but each new link needs a new dentry, pinning lowmem, and |
| 2856 | * tmpfs dentries cannot be pruned until they are unlinked. | 2856 | * tmpfs dentries cannot be pruned until they are unlinked. |
| 2857 | * But if an O_TMPFILE file is linked into the tmpfs, the | ||
| 2858 | * first link must skip that, to get the accounting right. | ||
| 2857 | */ | 2859 | */ |
| 2858 | ret = shmem_reserve_inode(inode->i_sb); | 2860 | if (inode->i_nlink) { |
| 2859 | if (ret) | 2861 | ret = shmem_reserve_inode(inode->i_sb); |
| 2860 | goto out; | 2862 | if (ret) |
| 2863 | goto out; | ||
| 2864 | } | ||
| 2861 | 2865 | ||
| 2862 | dir->i_size += BOGO_DIRENT_SIZE; | 2866 | dir->i_size += BOGO_DIRENT_SIZE; |
| 2863 | inode->i_ctime = dir->i_ctime = dir->i_mtime = current_time(inode); | 2867 | inode->i_ctime = dir->i_ctime = dir->i_mtime = current_time(inode); |
| @@ -2359,7 +2359,7 @@ static void *alloc_slabmgmt(struct kmem_cache *cachep, · 2358 unmodified lines | |||
| 2359 | void *freelist; | 2359 | void *freelist; |
| 2360 | void *addr = page_address(page); | 2360 | void *addr = page_address(page); |
| 2361 | 2361 | ||
| 2362 | page->s_mem = kasan_reset_tag(addr) + colour_off; | 2362 | page->s_mem = addr + colour_off; |
| 2363 | page->active = 0; | 2363 | page->active = 0; |
| 2364 | 2364 | ||
| 2365 | if (OBJFREELIST_SLAB(cachep)) | 2365 | if (OBJFREELIST_SLAB(cachep)) |
| @@ -2368,6 +2368,7 @@ static void *alloc_slabmgmt(struct kmem_cache *cachep, · 2 unmodified lines | |||
| 2368 | /* Slab management obj is off-slab. */ | 2368 | /* Slab management obj is off-slab. */ |
| 2369 | freelist = kmem_cache_alloc_node(cachep->freelist_cache, | 2369 | freelist = kmem_cache_alloc_node(cachep->freelist_cache, |
| 2370 | local_flags, nodeid); | 2370 | local_flags, nodeid); |
| 2371 | freelist = kasan_reset_tag(freelist); | ||
| 2371 | if (!freelist) | 2372 | if (!freelist) |
| 2372 | return NULL; | 2373 | return NULL; |
| 2373 | } else { | 2374 | } else { |
| @@ -2681,6 +2682,13 @@ static struct page *cache_grow_begin(struct kmem_cache *cachep, · 307 unmodified lines | |||
| 2681 | 2682 | ||
| 2682 | offset *= cachep->colour_off; | 2683 | offset *= cachep->colour_off; |
| 2683 | 2684 | ||
| 2685 | /* | ||
| 2686 | * Call kasan_poison_slab() before calling alloc_slabmgmt(), so | ||
| 2687 | * page_address() in the latter returns a non-tagged pointer, | ||
| 2688 | * as it should be for slab pages. | ||
| 2689 | */ | ||
| 2690 | kasan_poison_slab(page); | ||
| 2691 | |||
| 2684 | /* Get slab management. */ | 2692 | /* Get slab management. */ |
| 2685 | freelist = alloc_slabmgmt(cachep, page, offset, | 2693 | freelist = alloc_slabmgmt(cachep, page, offset, |
| 2686 | local_flags & ~GFP_CONSTRAINT_MASK, page_node); | 2694 | local_flags & ~GFP_CONSTRAINT_MASK, page_node); |
| @@ -2689,7 +2697,6 @@ static struct page *cache_grow_begin(struct kmem_cache *cachep, · 2 unmodified lines | |||
| 2689 | 2697 | ||
| 2690 | slab_map_pages(cachep, page, freelist); | 2698 | slab_map_pages(cachep, page, freelist); |
| 2691 | 2699 | ||
| 2692 | kasan_poison_slab(page); | ||
| 2693 | cache_init_objs(cachep, page); | 2700 | cache_init_objs(cachep, page); |
| 2694 | 2701 | ||
| 2695 | if (gfpflags_allow_blocking(local_flags)) | 2702 | if (gfpflags_allow_blocking(local_flags)) |
| @@ -3540,7 +3547,6 @@ void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags) · 844 unmodified lines | |||
| 3540 | { | 3547 | { |
| 3541 | void *ret = slab_alloc(cachep, flags, _RET_IP_); | 3548 | void *ret = slab_alloc(cachep, flags, _RET_IP_); |
| 3542 | 3549 | ||
| 3543 | ret = kasan_slab_alloc(cachep, ret, flags); | ||
| 3544 | trace_kmem_cache_alloc(_RET_IP_, ret, | 3550 | trace_kmem_cache_alloc(_RET_IP_, ret, |
| 3545 | cachep->object_size, cachep->size, flags); | 3551 | cachep->object_size, cachep->size, flags); |
| 3546 | 3552 | ||
| @@ -3630,7 +3636,6 @@ void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid) · 83 unmodified lines | |||
| 3630 | { | 3636 | { |
| 3631 | void *ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); | 3637 | void *ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); |
| 3632 | 3638 | ||
| 3633 | ret = kasan_slab_alloc(cachep, ret, flags); | ||
| 3634 | trace_kmem_cache_alloc_node(_RET_IP_, ret, | 3639 | trace_kmem_cache_alloc_node(_RET_IP_, ret, |
| 3635 | cachep->object_size, cachep->size, | 3640 | cachep->object_size, cachep->size, |
| 3636 | flags, nodeid); | 3641 | flags, nodeid); |
| @@ -4408,6 +4413,8 @@ void __check_heap_object(const void *ptr, unsigned long n, struct page *page, · 771 unmodified lines | |||
| 4408 | unsigned int objnr; | 4413 | unsigned int objnr; |
| 4409 | unsigned long offset; | 4414 | unsigned long offset; |
| 4410 | 4415 | ||
| 4416 | ptr = kasan_reset_tag(ptr); | ||
| 4417 | |||
| 4411 | /* Find and validate object. */ | 4418 | /* Find and validate object. */ |
| 4412 | cachep = page->slab_cache; | 4419 | cachep = page->slab_cache; |
| 4413 | objnr = obj_to_index(cachep, page, (void *)ptr); | 4420 | objnr = obj_to_index(cachep, page, (void *)ptr); |
| @@ -437,11 +437,10 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags, · 436 unmodified lines | |||
| 437 | 437 | ||
| 438 | flags &= gfp_allowed_mask; | 438 | flags &= gfp_allowed_mask; |
| 439 | for (i = 0; i < size; i++) { | 439 | for (i = 0; i < size; i++) { |
| 440 | void *object = p[i]; | 440 | p[i] = kasan_slab_alloc(s, p[i], flags); |
| 441 | 441 | /* As p[i] might get tagged, call kmemleak hook after KASAN. */ | |
| 442 | kmemleak_alloc_recursive(object, s->object_size, 1, | 442 | kmemleak_alloc_recursive(p[i], s->object_size, 1, |
| 443 | s->flags, flags); | 443 | s->flags, flags); |
| 444 | p[i] = kasan_slab_alloc(s, object, flags); | ||
| 445 | } | 444 | } |
| 446 | 445 | ||
| 447 | if (memcg_kmem_enabled()) | 446 | if (memcg_kmem_enabled()) |
| @@ -1228,8 +1228,9 @@ void *kmalloc_order(size_t size, gfp_t flags, unsigned int order) · 1227 unmodified lines | |||
| 1228 | flags |= __GFP_COMP; | 1228 | flags |= __GFP_COMP; |
| 1229 | page = alloc_pages(flags, order); | 1229 | page = alloc_pages(flags, order); |
| 1230 | ret = page ? page_address(page) : NULL; | 1230 | ret = page ? page_address(page) : NULL; |
| 1231 | kmemleak_alloc(ret, size, 1, flags); | ||
| 1232 | ret = kasan_kmalloc_large(ret, size, flags); | 1231 | ret = kasan_kmalloc_large(ret, size, flags); |
| 1232 | /* As ret might get tagged, call kmemleak hook after KASAN. */ | ||
| 1233 | kmemleak_alloc(ret, size, 1, flags); | ||
| 1233 | return ret; | 1234 | return ret; |
| 1234 | } | 1235 | } |
| 1235 | EXPORT_SYMBOL(kmalloc_order); | 1236 | EXPORT_SYMBOL(kmalloc_order); |
| @@ -249,7 +249,18 @@ static inline void *freelist_ptr(const struct kmem_cache *s, void *ptr, · 248 unmodified lines | |||
| 249 | unsigned long ptr_addr) | 249 | unsigned long ptr_addr) |
| 250 | { | 250 | { |
| 251 | #ifdef CONFIG_SLAB_FREELIST_HARDENED | 251 | #ifdef CONFIG_SLAB_FREELIST_HARDENED |
| 252 | return (void *)((unsigned long)ptr ^ s->random ^ ptr_addr); | 252 | /* |
| 253 | * When CONFIG_KASAN_SW_TAGS is enabled, ptr_addr might be tagged. | ||
| 254 | * Normally, this doesn't cause any issues, as both set_freepointer() | ||
| 255 | * and get_freepointer() are called with a pointer with the same tag. | ||
| 256 | * However, there are some issues with CONFIG_SLUB_DEBUG code. For | ||
| 257 | * example, when __free_slub() iterates over objects in a cache, it | ||
| 258 | * passes untagged pointers to check_object(). check_object() in turns | ||
| 259 | * calls get_freepointer() with an untagged pointer, which causes the | ||
| 260 | * freepointer to be restored incorrectly. | ||
| 261 | */ | ||
| 262 | return (void *)((unsigned long)ptr ^ s->random ^ | ||
| 263 | (unsigned long)kasan_reset_tag((void *)ptr_addr)); | ||
| 253 | #else | 264 | #else |
| 254 | return ptr; | 265 | return ptr; |
| 255 | #endif | 266 | #endif |
| @@ -303,15 +314,10 @@ static inline void set_freepointer(struct kmem_cache *s, void *object, void *fp) · 47 unmodified lines | |||
| 303 | __p < (__addr) + (__objects) * (__s)->size; \ | 314 | __p < (__addr) + (__objects) * (__s)->size; \ |
| 304 | __p += (__s)->size) | 315 | __p += (__s)->size) |
| 305 | 316 | ||
| 306 | #define for_each_object_idx(__p, __idx, __s, __addr, __objects) \ | ||
| 307 | for (__p = fixup_red_left(__s, __addr), __idx = 1; \ | ||
| 308 | __idx <= __objects; \ | ||
| 309 | __p += (__s)->size, __idx++) | ||
| 310 | |||
| 311 | /* Determine object index from a given position */ | 317 | /* Determine object index from a given position */ |
| 312 | static inline unsigned int slab_index(void *p, struct kmem_cache *s, void *addr) | 318 | static inline unsigned int slab_index(void *p, struct kmem_cache *s, void *addr) |
| 313 | { | 319 | { |
| 314 | return (p - addr) / s->size; | 320 | return (kasan_reset_tag(p) - addr) / s->size; |
| 315 | } | 321 | } |
| 316 | 322 | ||
| 317 | static inline unsigned int order_objects(unsigned int order, unsigned int size) | 323 | static inline unsigned int order_objects(unsigned int order, unsigned int size) |
| @@ -507,6 +513,7 @@ static inline int check_valid_pointer(struct kmem_cache *s, · 189 unmodified lines | |||
| 507 | return 1; | 513 | return 1; |
| 508 | 514 | ||
| 509 | base = page_address(page); | 515 | base = page_address(page); |
| 516 | object = kasan_reset_tag(object); | ||
| 510 | object = restore_red_left(s, object); | 517 | object = restore_red_left(s, object); |
| 511 | if (object < base || object >= base + page->objects * s->size || | 518 | if (object < base || object >= base + page->objects * s->size || |
| 512 | (object - base) % s->size) { | 519 | (object - base) % s->size) { |
| @@ -1075,6 +1082,16 @@ static void setup_object_debug(struct kmem_cache *s, struct page *page, · 562 unmodified lines | |||
| 1075 | init_tracking(s, object); | 1082 | init_tracking(s, object); |
| 1076 | } | 1083 | } |
| 1077 | 1084 | ||
| 1085 | static void setup_page_debug(struct kmem_cache *s, void *addr, int order) | ||
| 1086 | { | ||
| 1087 | if (!(s->flags & SLAB_POISON)) | ||
| 1088 | return; | ||
| 1089 | |||
| 1090 | metadata_access_enable(); | ||
| 1091 | memset(addr, POISON_INUSE, PAGE_SIZE << order); | ||
| 1092 | metadata_access_disable(); | ||
| 1093 | } | ||
| 1094 | |||
| 1078 | static inline int alloc_consistency_checks(struct kmem_cache *s, | 1095 | static inline int alloc_consistency_checks(struct kmem_cache *s, |
| 1079 | struct page *page, | 1096 | struct page *page, |
| 1080 | void *object, unsigned long addr) | 1097 | void *object, unsigned long addr) |
| @@ -1330,6 +1347,8 @@ slab_flags_t kmem_cache_flags(unsigned int object_size, · 249 unmodified lines | |||
| 1330 | #else /* !CONFIG_SLUB_DEBUG */ | 1347 | #else /* !CONFIG_SLUB_DEBUG */ |
| 1331 | static inline void setup_object_debug(struct kmem_cache *s, | 1348 | static inline void setup_object_debug(struct kmem_cache *s, |
| 1332 | struct page *page, void *object) {} | 1349 | struct page *page, void *object) {} |
| 1350 | static inline void setup_page_debug(struct kmem_cache *s, | ||
| 1351 | void *addr, int order) {} | ||
| 1333 | 1352 | ||
| 1334 | static inline int alloc_debug_processing(struct kmem_cache *s, | 1353 | static inline int alloc_debug_processing(struct kmem_cache *s, |
| 1335 | struct page *page, void *object, unsigned long addr) { return 0; } | 1354 | struct page *page, void *object, unsigned long addr) { return 0; } |
| @@ -1374,8 +1393,10 @@ static inline void dec_slabs_node(struct kmem_cache *s, int node, · 38 unmodified lines | |||
| 1374 | */ | 1393 | */ |
| 1375 | static inline void *kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) | 1394 | static inline void *kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) |
| 1376 | { | 1395 | { |
| 1396 | ptr = kasan_kmalloc_large(ptr, size, flags); | ||
| 1397 | /* As ptr might get tagged, call kmemleak hook after KASAN. */ | ||
| 1377 | kmemleak_alloc(ptr, size, 1, flags); | 1398 | kmemleak_alloc(ptr, size, 1, flags); |
| 1378 | return kasan_kmalloc_large(ptr, size, flags); | 1399 | return ptr; |
| 1379 | } | 1400 | } |
| 1380 | 1401 | ||
| 1381 | static __always_inline void kfree_hook(void *x) | 1402 | static __always_inline void kfree_hook(void *x) |
| @@ -1641,27 +1662,25 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) · 259 unmodified lines | |||
| 1641 | if (page_is_pfmemalloc(page)) | 1662 | if (page_is_pfmemalloc(page)) |
| 1642 | SetPageSlabPfmemalloc(page); | 1663 | SetPageSlabPfmemalloc(page); |
| 1643 | 1664 | ||
| 1665 | kasan_poison_slab(page); | ||
| 1666 | |||
| 1644 | start = page_address(page); | 1667 | start = page_address(page); |
| 1645 | 1668 | ||
| 1646 | if (unlikely(s->flags & SLAB_POISON)) | 1669 | setup_page_debug(s, start, order); |
| 1647 | memset(start, POISON_INUSE, PAGE_SIZE << order); | ||
| 1648 | |||
| 1649 | kasan_poison_slab(page); | ||
| 1650 | 1670 | ||
| 1651 | shuffle = shuffle_freelist(s, page); | 1671 | shuffle = shuffle_freelist(s, page); |
| 1652 | 1672 | ||
| 1653 | if (!shuffle) { | 1673 | if (!shuffle) { |
| 1654 | for_each_object_idx(p, idx, s, start, page->objects) { | ||
| 1655 | if (likely(idx < page->objects)) { | ||
| 1656 | next = p + s->size; | ||
| 1657 | next = setup_object(s, page, next); | ||
| 1658 | set_freepointer(s, p, next); | ||
| 1659 | } else | ||
| 1660 | set_freepointer(s, p, NULL); | ||
| 1661 | } | ||
| 1662 | start = fixup_red_left(s, start); | 1674 | start = fixup_red_left(s, start); |
| 1663 | start = setup_object(s, page, start); | 1675 | start = setup_object(s, page, start); |
| 1664 | page->freelist = start; | 1676 | page->freelist = start; |
| 1677 | for (idx = 0, p = start; idx < page->objects - 1; idx++) { | ||
| 1678 | next = p + s->size; | ||
| 1679 | next = setup_object(s, page, next); | ||
| 1680 | set_freepointer(s, p, next); | ||
| 1681 | p = next; | ||
| 1682 | } | ||
| 1683 | set_freepointer(s, p, NULL); | ||
| 1665 | } | 1684 | } |
| 1666 | 1685 | ||
| 1667 | page->inuse = page->objects; | 1686 | page->inuse = page->objects; |
| @@ -320,11 +320,6 @@ static inline void activate_page_drain(int cpu) · 319 unmodified lines | |||
| 320 | { | 320 | { |
| 321 | } | 321 | } |
| 322 | 322 | ||
| 323 | static bool need_activate_page_drain(int cpu) | ||
| 324 | { | ||
| 325 | return false; | ||
| 326 | } | ||
| 327 | |||
| 328 | void activate_page(struct page *page) | 323 | void activate_page(struct page *page) |
| 329 | { | 324 | { |
| 330 | struct zone *zone = page_zone(page); | 325 | struct zone *zone = page_zone(page); |
| @@ -653,13 +648,15 @@ void lru_add_drain(void) · 322 unmodified lines | |||
| 653 | put_cpu(); | 648 | put_cpu(); |
| 654 | } | 649 | } |
| 655 | 650 | ||
| 651 | #ifdef CONFIG_SMP | ||
| 652 | |||
| 653 | static DEFINE_PER_CPU(struct work_struct, lru_add_drain_work); | ||
| 654 | |||
| 656 | static void lru_add_drain_per_cpu(struct work_struct *dummy) | 655 | static void lru_add_drain_per_cpu(struct work_struct *dummy) |
| 657 | { | 656 | { |
| 658 | lru_add_drain(); | 657 | lru_add_drain(); |
| 659 | } | 658 | } |
| 660 | 659 | ||
| 661 | static DEFINE_PER_CPU(struct work_struct, lru_add_drain_work); | ||
| 662 | |||
| 663 | /* | 660 | /* |
| 664 | * Doesn't need any cpu hotplug locking because we do rely on per-cpu | 661 | * Doesn't need any cpu hotplug locking because we do rely on per-cpu |
| 665 | * kworkers being shut down before our page_alloc_cpu_dead callback is | 662 | * kworkers being shut down before our page_alloc_cpu_dead callback is |
| @@ -702,6 +699,12 @@ void lru_add_drain_all(void) · 36 unmodified lines | |||
| 702 | 699 | ||
| 703 | mutex_unlock(&lock); | 700 | mutex_unlock(&lock); |
| 704 | } | 701 | } |
| 702 | #else | ||
| 703 | void lru_add_drain_all(void) | ||
| 704 | { | ||
| 705 | lru_add_drain(); | ||
| 706 | } | ||
| 707 | #endif | ||
| 705 | 708 | ||
| 706 | /** | 709 | /** |
| 707 | * release_pages - batched put_page() | 710 | * release_pages - batched put_page() |
| @@ -150,7 +150,7 @@ void *memdup_user(const void __user *src, size_t len) · 149 unmodified lines | |||
| 150 | { | 150 | { |
| 151 | void *p; | 151 | void *p; |
| 152 | 152 | ||
| 153 | p = kmalloc_track_caller(len, GFP_USER); | 153 | p = kmalloc_track_caller(len, GFP_USER | __GFP_NOWARN); |
| 154 | if (!p) | 154 | if (!p) |
| 155 | return ERR_PTR(-ENOMEM); | 155 | return ERR_PTR(-ENOMEM); |
| 156 | 156 | ||
| @@ -13,27 +13,13 @@ · 12 unmodified lines | |||
| 13 | #include <net/sock.h> | 13 | #include <net/sock.h> |
| 14 | #include <net/tcp.h> | 14 | #include <net/tcp.h> |
| 15 | 15 | ||
| 16 | static __always_inline u32 bpf_test_run_one(struct bpf_prog *prog, void *ctx, | 16 | static int bpf_test_run(struct bpf_prog *prog, void *ctx, u32 repeat, |
| 17 | struct bpf_cgroup_storage *storage[MAX_BPF_CGROUP_STORAGE_TYPE]) | 17 | u32 *retval, u32 *time) |
| 18 | { | ||
| 19 | u32 ret; | ||
| 20 | |||
| 21 | preempt_disable(); | ||
| 22 | rcu_read_lock(); | ||
| 23 | bpf_cgroup_storage_set(storage); | ||
| 24 | ret = BPF_PROG_RUN(prog, ctx); | ||
| 25 | rcu_read_unlock(); | ||
| 26 | preempt_enable(); | ||
| 27 | |||
| 28 | return ret; | ||
| 29 | } | ||
| 30 | |||
| 31 | static int bpf_test_run(struct bpf_prog *prog, void *ctx, u32 repeat, u32 *ret, | ||
| 32 | u32 *time) | ||
| 33 | { | 18 | { |
| 34 | struct bpf_cgroup_storage *storage[MAX_BPF_CGROUP_STORAGE_TYPE] = { 0 }; | 19 | struct bpf_cgroup_storage *storage[MAX_BPF_CGROUP_STORAGE_TYPE] = { 0 }; |
| 35 | enum bpf_cgroup_storage_type stype; | 20 | enum bpf_cgroup_storage_type stype; |
| 36 | u64 time_start, time_spent = 0; | 21 | u64 time_start, time_spent = 0; |
| 22 | int ret = 0; | ||
| 37 | u32 i; | 23 | u32 i; |
| 38 | 24 | ||
| 39 | for_each_cgroup_storage_type(stype) { | 25 | for_each_cgroup_storage_type(stype) { |
| @@ -48,25 +34,42 @@ static int bpf_test_run(struct bpf_prog *prog, void *ctx, u32 repeat, u32 *ret, · 8 unmodified lines | |||
| 48 | 34 | ||
| 49 | if (!repeat) | 35 | if (!repeat) |
| 50 | repeat = 1; | 36 | repeat = 1; |
| 37 | |||
| 38 | rcu_read_lock(); | ||
| 39 | preempt_disable(); | ||
| 51 | time_start = ktime_get_ns(); | 40 | time_start = ktime_get_ns(); |
| 52 | for (i = 0; i < repeat; i++) { | 41 | for (i = 0; i < repeat; i++) { |
| 53 | *ret = bpf_test_run_one(prog, ctx, storage); | 42 | bpf_cgroup_storage_set(storage); |
| 43 | *retval = BPF_PROG_RUN(prog, ctx); | ||
| 44 | |||
| 45 | if (signal_pending(current)) { | ||
| 46 | ret = -EINTR; | ||
| 47 | break; | ||
| 48 | } | ||
| 49 | |||
| 54 | if (need_resched()) { | 50 | if (need_resched()) { |
| 55 | if (signal_pending(current)) | ||
| 56 | break; | ||
| 57 | time_spent += ktime_get_ns() - time_start; | 51 | time_spent += ktime_get_ns() - time_start; |
| 52 | preempt_enable(); | ||
| 53 | rcu_read_unlock(); | ||
| 54 | |||
| 58 | cond_resched(); | 55 | cond_resched(); |
| 56 | |||
| 57 | rcu_read_lock(); | ||
| 58 | preempt_disable(); | ||
| 59 | time_start = ktime_get_ns(); | 59 | time_start = ktime_get_ns(); |
| 60 | } | 60 | } |
| 61 | } | 61 | } |
| 62 | time_spent += ktime_get_ns() - time_start; | 62 | time_spent += ktime_get_ns() - time_start; |
| 63 | preempt_enable(); | ||
| 64 | rcu_read_unlock(); | ||
| 65 | |||
| 63 | do_div(time_spent, repeat); | 66 | do_div(time_spent, repeat); |
| 64 | *time = time_spent > U32_MAX ? U32_MAX : (u32)time_spent; | 67 | *time = time_spent > U32_MAX ? U32_MAX : (u32)time_spent; |
| 65 | 68 | ||
| 66 | for_each_cgroup_storage_type(stype) | 69 | for_each_cgroup_storage_type(stype) |
| 67 | bpf_cgroup_storage_free(storage[stype]); | 70 | bpf_cgroup_storage_free(storage[stype]); |
| 68 | 71 | ||
| 69 | return 0; | 72 | return ret; |
| 70 | } | 73 | } |
| 71 | 74 | ||
| 72 | static int bpf_test_finish(const union bpf_attr *kattr, | 75 | static int bpf_test_finish(const union bpf_attr *kattr, |
| @@ -1204,14 +1204,7 @@ static void br_multicast_query_received(struct net_bridge *br, · 1203 unmodified lines | |||
| 1204 | return; | 1204 | return; |
| 1205 | 1205 | ||
| 1206 | br_multicast_update_query_timer(br, query, max_delay); | 1206 | br_multicast_update_query_timer(br, query, max_delay); |
| 1207 | 1207 | br_multicast_mark_router(br, port); | |
| 1208 | /* Based on RFC4541, section 2.1.1 IGMP Forwarding Rules, | ||
| 1209 | * the arrival port for IGMP Queries where the source address | ||
| 1210 | * is 0.0.0.0 should not be added to router port list. | ||
| 1211 | */ | ||
| 1212 | if ((saddr->proto == htons(ETH_P_IP) && saddr->u.ip4) || | ||
| 1213 | saddr->proto == htons(ETH_P_IPV6)) | ||
| 1214 | br_multicast_mark_router(br, port); | ||
| 1215 | } | 1208 | } |
| 1216 | 1209 | ||
| 1217 | static void br_ip4_multicast_query(struct net_bridge *br, | 1210 | static void br_ip4_multicast_query(struct net_bridge *br, |
| @@ -2058,6 +2058,8 @@ static int process_connect(struct ceph_connection *con) · 2057 unmodified lines | |||
| 2058 | dout("process_connect on %p tag %d\n", con, (int)con->in_tag); | 2058 | dout("process_connect on %p tag %d\n", con, (int)con->in_tag); |
| 2059 | 2059 | ||
| 2060 | if (con->auth) { | 2060 | if (con->auth) { |
| 2061 | int len = le32_to_cpu(con->in_reply.authorizer_len); | ||
| 2062 | |||
| 2061 | /* | 2063 | /* |
| 2062 | * Any connection that defines ->get_authorizer() | 2064 | * Any connection that defines ->get_authorizer() |
| 2063 | * should also define ->add_authorizer_challenge() and | 2065 | * should also define ->add_authorizer_challenge() and |
| @@ -2067,8 +2069,7 @@ static int process_connect(struct ceph_connection *con) · 3 unmodified lines | |||
| 2067 | */ | 2069 | */ |
| 2068 | if (con->in_reply.tag == CEPH_MSGR_TAG_CHALLENGE_AUTHORIZER) { | 2070 | if (con->in_reply.tag == CEPH_MSGR_TAG_CHALLENGE_AUTHORIZER) { |
| 2069 | ret = con->ops->add_authorizer_challenge( | 2071 | ret = con->ops->add_authorizer_challenge( |
| 2070 | con, con->auth->authorizer_reply_buf, | 2072 | con, con->auth->authorizer_reply_buf, len); |
| 2071 | le32_to_cpu(con->in_reply.authorizer_len)); | ||
| 2072 | if (ret < 0) | 2073 | if (ret < 0) |
| 2073 | return ret; | 2074 | return ret; |
| 2074 | 2075 | ||
| @@ -2078,10 +2079,12 @@ static int process_connect(struct ceph_connection *con) · 3 unmodified lines | |||
| 2078 | return 0; | 2079 | return 0; |
| 2079 | } | 2080 | } |
| 2080 | 2081 | ||
| 2081 | ret = con->ops->verify_authorizer_reply(con); | 2082 | if (len) { |
| 2082 | if (ret < 0) { | 2083 | ret = con->ops->verify_authorizer_reply(con); |
| 2083 | con->error_msg = "bad authorize reply"; | 2084 | if (ret < 0) { |
| 2084 | return ret; | 2085 | con->error_msg = "bad authorize reply"; |
| 2086 | return ret; | ||
| 2087 | } | ||
| 2085 | } | 2088 | } |
| 2086 | } | 2089 | } |
| 2087 | 2090 | ||
| @@ -388,8 +388,12 @@ static int __compat_sys_setsockopt(int fd, int level, int optname, · 387 unmodified lines | |||
| 388 | char __user *optval, unsigned int optlen) | 388 | char __user *optval, unsigned int optlen) |
| 389 | { | 389 | { |
| 390 | int err; | 390 | int err; |
| 391 | struct socket *sock = sockfd_lookup(fd, &err); | 391 | struct socket *sock; |
| 392 | |||
| 393 | if (optlen > INT_MAX) | ||
| 394 | return -EINVAL; | ||
| 392 | 395 | ||
| 396 | sock = sockfd_lookup(fd, &err); | ||
| 393 | if (sock) { | 397 | if (sock) { |
| 394 | err = security_socket_setsockopt(sock, level, optname); | 398 | err = security_socket_setsockopt(sock, level, optname); |
| 395 | if (err) { | 399 | if (err) { |
| @@ -8152,7 +8152,7 @@ static netdev_features_t netdev_sync_upper_features(struct net_device *lower, · 8151 unmodified lines | |||
| 8152 | netdev_features_t feature; | 8152 | netdev_features_t feature; |
| 8153 | int feature_bit; | 8153 | int feature_bit; |
| 8154 | 8154 | ||
| 8155 | for_each_netdev_feature(&upper_disables, feature_bit) { | 8155 | for_each_netdev_feature(upper_disables, feature_bit) { |
| 8156 | feature = __NETIF_F_BIT(feature_bit); | 8156 | feature = __NETIF_F_BIT(feature_bit); |
| 8157 | if (!(upper->wanted_features & feature) | 8157 | if (!(upper->wanted_features & feature) |
| 8158 | && (features & feature)) { | 8158 | && (features & feature)) { |
| @@ -8172,7 +8172,7 @@ static void netdev_sync_lower_features(struct net_device *upper, · 13 unmodified lines | |||
| 8172 | netdev_features_t feature; | 8172 | netdev_features_t feature; |
| 8173 | int feature_bit; | 8173 | int feature_bit; |
| 8174 | 8174 | ||
| 8175 | for_each_netdev_feature(&upper_disables, feature_bit) { | 8175 | for_each_netdev_feature(upper_disables, feature_bit) { |
| 8176 | feature = __NETIF_F_BIT(feature_bit); | 8176 | feature = __NETIF_F_BIT(feature_bit); |
| 8177 | if (!(features & feature) && (lower->features & feature)) { | 8177 | if (!(features & feature) && (lower->features & feature)) { |
| 8178 | netdev_dbg(upper, "Disabling feature %pNF on lower dev %s.\n", | 8178 | netdev_dbg(upper, "Disabling feature %pNF on lower dev %s.\n", |
| @@ -2789,8 +2789,7 @@ static int bpf_skb_proto_4_to_6(struct sk_buff *skb) · 2788 unmodified lines | |||
| 2789 | u32 off = skb_mac_header_len(skb); | 2789 | u32 off = skb_mac_header_len(skb); |
| 2790 | int ret; | 2790 | int ret; |
| 2791 | 2791 | ||
| 2792 | /* SCTP uses GSO_BY_FRAGS, thus cannot adjust it. */ | 2792 | if (!skb_is_gso_tcp(skb)) |
| 2793 | if (skb_is_gso(skb) && unlikely(skb_is_gso_sctp(skb))) | ||
| 2794 | return -ENOTSUPP; | 2793 | return -ENOTSUPP; |
| 2795 | 2794 | ||
| 2796 | ret = skb_cow(skb, len_diff); | 2795 | ret = skb_cow(skb, len_diff); |
| @@ -2831,8 +2830,7 @@ static int bpf_skb_proto_6_to_4(struct sk_buff *skb) · 34 unmodified lines | |||
| 2831 | u32 off = skb_mac_header_len(skb); | 2830 | u32 off = skb_mac_header_len(skb); |
| 2832 | int ret; | 2831 | int ret; |
| 2833 | 2832 | ||
| 2834 | /* SCTP uses GSO_BY_FRAGS, thus cannot adjust it. */ | 2833 | if (!skb_is_gso_tcp(skb)) |
| 2835 | if (skb_is_gso(skb) && unlikely(skb_is_gso_sctp(skb))) | ||
| 2836 | return -ENOTSUPP; | 2834 | return -ENOTSUPP; |
| 2837 | 2835 | ||
| 2838 | ret = skb_unclone(skb, GFP_ATOMIC); | 2836 | ret = skb_unclone(skb, GFP_ATOMIC); |
| @@ -2957,8 +2955,7 @@ static int bpf_skb_net_grow(struct sk_buff *skb, u32 len_diff) · 118 unmodified lines | |||
| 2957 | u32 off = skb_mac_header_len(skb) + bpf_skb_net_base_len(skb); | 2955 | u32 off = skb_mac_header_len(skb) + bpf_skb_net_base_len(skb); |
| 2958 | int ret; | 2956 | int ret; |
| 2959 | 2957 | ||
| 2960 | /* SCTP uses GSO_BY_FRAGS, thus cannot adjust it. */ | 2958 | if (!skb_is_gso_tcp(skb)) |
| 2961 | if (skb_is_gso(skb) && unlikely(skb_is_gso_sctp(skb))) | ||
| 2962 | return -ENOTSUPP; | 2959 | return -ENOTSUPP; |
| 2963 | 2960 | ||
| 2964 | ret = skb_cow(skb, len_diff); | 2961 | ret = skb_cow(skb, len_diff); |
| @@ -2987,8 +2984,7 @@ static int bpf_skb_net_shrink(struct sk_buff *skb, u32 len_diff) · 22 unmodified lines | |||
| 2987 | u32 off = skb_mac_header_len(skb) + bpf_skb_net_base_len(skb); | 2984 | u32 off = skb_mac_header_len(skb) + bpf_skb_net_base_len(skb); |
| 2988 | int ret; | 2985 | int ret; |
| 2989 | 2986 | ||
| 2990 | /* SCTP uses GSO_BY_FRAGS, thus cannot adjust it. */ | 2987 | if (!skb_is_gso_tcp(skb)) |
| 2991 | if (skb_is_gso(skb) && unlikely(skb_is_gso_sctp(skb))) | ||
| 2992 | return -ENOTSUPP; | 2988 | return -ENOTSUPP; |
| 2993 | 2989 | ||
| 2994 | ret = skb_unclone(skb, GFP_ATOMIC); | 2990 | ret = skb_unclone(skb, GFP_ATOMIC); |
| @@ -356,6 +356,8 @@ static void *__netdev_alloc_frag(unsigned int fragsz, gfp_t gfp_mask) · 355 unmodified lines | |||
| 356 | */ | 356 | */ |
| 357 | void *netdev_alloc_frag(unsigned int fragsz) | 357 | void *netdev_alloc_frag(unsigned int fragsz) |
| 358 | { | 358 | { |
| 359 | fragsz = SKB_DATA_ALIGN(fragsz); | ||
| 360 | |||
| 359 | return __netdev_alloc_frag(fragsz, GFP_ATOMIC); | 361 | return __netdev_alloc_frag(fragsz, GFP_ATOMIC); |
| 360 | } | 362 | } |
| 361 | EXPORT_SYMBOL(netdev_alloc_frag); | 363 | EXPORT_SYMBOL(netdev_alloc_frag); |
| @@ -369,6 +371,8 @@ static void *__napi_alloc_frag(unsigned int fragsz, gfp_t gfp_mask) · 7 unmodified lines | |||
| 369 | 371 | ||
| 370 | void *napi_alloc_frag(unsigned int fragsz) | 372 | void *napi_alloc_frag(unsigned int fragsz) |
| 371 | { | 373 | { |
| 374 | fragsz = SKB_DATA_ALIGN(fragsz); | ||
| 375 | |||
| 372 | return __napi_alloc_frag(fragsz, GFP_ATOMIC); | 376 | return __napi_alloc_frag(fragsz, GFP_ATOMIC); |
| 373 | } | 377 | } |
| 374 | EXPORT_SYMBOL(napi_alloc_frag); | 378 | EXPORT_SYMBOL(napi_alloc_frag); |
| @@ -612,8 +612,8 @@ static int dsa_switch_parse_ports_of(struct dsa_switch *ds, · 611 unmodified lines | |||
| 612 | { | 612 | { |
| 613 | struct device_node *ports, *port; | 613 | struct device_node *ports, *port; |
| 614 | struct dsa_port *dp; | 614 | struct dsa_port *dp; |
| 615 | int err = 0; | ||
| 615 | u32 reg; | 616 | u32 reg; |
| 616 | int err; | ||
| 617 | 617 | ||
| 618 | ports = of_get_child_by_name(dn, "ports"); | 618 | ports = of_get_child_by_name(dn, "ports"); |
| 619 | if (!ports) { | 619 | if (!ports) { |
| @@ -624,19 +624,23 @@ static int dsa_switch_parse_ports_of(struct dsa_switch *ds, · 4 unmodified lines | |||
| 624 | for_each_available_child_of_node(ports, port) { | 624 | for_each_available_child_of_node(ports, port) { |
| 625 | err = of_property_read_u32(port, "reg", ®); | 625 | err = of_property_read_u32(port, "reg", ®); |
| 626 | if (err) | 626 | if (err) |
| 627 | return err; | 627 | goto out_put_node; |
| 628 | 628 | ||
| 629 | if (reg >= ds->num_ports) | 629 | if (reg >= ds->num_ports) { |
| 630 | return -EINVAL; | 630 | err = -EINVAL; |
| 631 | goto out_put_node; | ||
| 632 | } | ||
| 631 | 633 | ||
| 632 | dp = &ds->ports[reg]; | 634 | dp = &ds->ports[reg]; |
| 633 | 635 | ||
| 634 | err = dsa_port_parse_of(dp, port); | 636 | err = dsa_port_parse_of(dp, port); |
| 635 | if (err) | 637 | if (err) |
| 636 | return err; | 638 | goto out_put_node; |
| 637 | } | 639 | } |
| 638 | 640 | ||
| 639 | return 0; | 641 | out_put_node: |
| 642 | of_node_put(ports); | ||
| 643 | return err; | ||
| 640 | } | 644 | } |
| 641 | 645 | ||
| 642 | static int dsa_switch_parse_member_of(struct dsa_switch *ds, | 646 | static int dsa_switch_parse_member_of(struct dsa_switch *ds, |
| @@ -69,7 +69,6 @@ static void dsa_port_set_state_now(struct dsa_port *dp, u8 state) · 68 unmodified lines | |||
| 69 | 69 | ||
| 70 | int dsa_port_enable(struct dsa_port *dp, struct phy_device *phy) | 70 | int dsa_port_enable(struct dsa_port *dp, struct phy_device *phy) |
| 71 | { | 71 | { |
| 72 | u8 stp_state = dp->bridge_dev ? BR_STATE_BLOCKING : BR_STATE_FORWARDING; | ||
| 73 | struct dsa_switch *ds = dp->ds; | 72 | struct dsa_switch *ds = dp->ds; |
| 74 | int port = dp->index; | 73 | int port = dp->index; |
| 75 | int err; | 74 | int err; |
| @@ -80,7 +79,8 @@ int dsa_port_enable(struct dsa_port *dp, struct phy_device *phy) · 4 unmodified lines | |||
| 80 | return err; | 79 | return err; |
| 81 | } | 80 | } |
| 82 | 81 | ||
| 83 | dsa_port_set_state_now(dp, stp_state); | 82 | if (!dp->bridge_dev) |
| 83 | dsa_port_set_state_now(dp, BR_STATE_FORWARDING); | ||
| 84 | 84 | ||
| 85 | return 0; | 85 | return 0; |
| 86 | } | 86 | } |
| @@ -90,7 +90,8 @@ void dsa_port_disable(struct dsa_port *dp, struct phy_device *phy) · 3 unmodified lines | |||
| 90 | struct dsa_switch *ds = dp->ds; | 90 | struct dsa_switch *ds = dp->ds; |
| 91 | int port = dp->index; | 91 | int port = dp->index; |
| 92 | 92 | ||
| 93 | dsa_port_set_state_now(dp, BR_STATE_DISABLED); | 93 | if (!dp->bridge_dev) |
| 94 | dsa_port_set_state_now(dp, BR_STATE_DISABLED); | ||
| 94 | 95 | ||
| 95 | if (ds->ops->port_disable) | 96 | if (ds->ops->port_disable) |
| 96 | ds->ops->port_disable(ds, port, phy); | 97 | ds->ops->port_disable(ds, port, phy); |
| @@ -291,6 +292,7 @@ static struct phy_device *dsa_port_get_phy_device(struct dsa_port *dp) · 194 unmodified lines | |||
| 291 | return ERR_PTR(-EPROBE_DEFER); | 292 | return ERR_PTR(-EPROBE_DEFER); |
| 292 | } | 293 | } |
| 293 | 294 | ||
| 295 | of_node_put(phy_dn); | ||
| 294 | return phydev; | 296 | return phydev; |
| 295 | } | 297 | } |
| 296 | 298 | ||
| @@ -667,7 +667,8 @@ static int cipso_v4_map_lvl_valid(const struct cipso_v4_doi *doi_def, u8 level) · 666 unmodified lines | |||
| 667 | case CIPSO_V4_MAP_PASS: | 667 | case CIPSO_V4_MAP_PASS: |
| 668 | return 0; | 668 | return 0; |
| 669 | case CIPSO_V4_MAP_TRANS: | 669 | case CIPSO_V4_MAP_TRANS: |
| 670 | if (doi_def->map.std->lvl.cipso[level] < CIPSO_V4_INV_LVL) | 670 | if ((level < doi_def->map.std->lvl.cipso_size) && |
| 671 | (doi_def->map.std->lvl.cipso[level] < CIPSO_V4_INV_LVL)) | ||
| 671 | return 0; | 672 | return 0; |
| 672 | break; | 673 | break; |
| 673 | } | 674 | } |
| @@ -1735,13 +1736,26 @@ validate_return: · 1061 unmodified lines | |||
| 1735 | */ | 1736 | */ |
| 1736 | void cipso_v4_error(struct sk_buff *skb, int error, u32 gateway) | 1737 | void cipso_v4_error(struct sk_buff *skb, int error, u32 gateway) |
| 1737 | { | 1738 | { |
| 1739 | unsigned char optbuf[sizeof(struct ip_options) + 40]; | ||
| 1740 | struct ip_options *opt = (struct ip_options *)optbuf; | ||
| 1741 | |||
| 1738 | if (ip_hdr(skb)->protocol == IPPROTO_ICMP || error != -EACCES) | 1742 | if (ip_hdr(skb)->protocol == IPPROTO_ICMP || error != -EACCES) |
| 1739 | return; | 1743 | return; |
| 1740 | 1744 | ||
| 1745 | /* | ||
| 1746 | * We might be called above the IP layer, | ||
| 1747 | * so we can not use icmp_send and IPCB here. | ||
| 1748 | */ | ||
| 1749 | |||
| 1750 | memset(opt, 0, sizeof(struct ip_options)); | ||
| 1751 | opt->optlen = ip_hdr(skb)->ihl*4 - sizeof(struct iphdr); | ||
12 more lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 122 lines aren’t shown here.
This diff is large: its 116 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 285 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 129 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 83 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 158 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 199 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 120 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.