We need the USB fixes in here as well to build on top of.
Unified · Split
| @@ -72,6 +72,7 @@ Alice Mikityanska <alice.kernel@fastmail.im> <maximmi@nvidia.com> · 71 unmodified lines | ||
| 72 | 72 | Alice Mikityanska <alice.kernel@fastmail.im> <maxim@isovalent.com> |
| 73 | 73 | Alice Mikityanska <alice.kernel@fastmail.im> <alice@isovalent.com> |
| 74 | 74 | Aloka Dixit <quic_alokad@quicinc.com> <alokad@codeaurora.org> |
| 75 | +Alvin Šipraga <alvin.sipraga@analog.com> <alsi@bang-olufsen.dk> | |
| 75 | 76 | Al Viro <viro@ftp.linux.org.uk> |
| 76 | 77 | Al Viro <viro@zenIV.linux.org.uk> |
| 77 | 78 | Amit Blay <quic_ablay@quicinc.com> <ablay@codeaurora.org> |
| @@ -170,12 +171,14 @@ Boris Brezillon <bbrezillon@kernel.org> <b.brezillon@overkiz.com> · 92 unmodified lines | ||
| 170 | 171 | Boris Brezillon <bbrezillon@kernel.org> <boris.brezillon@bootlin.com> |
| 171 | 172 | Boris Brezillon <bbrezillon@kernel.org> <boris.brezillon@free-electrons.com> |
| 172 | 173 | Brendan Higgins <brendan.higgins@linux.dev> <brendanhiggins@google.com> |
| 174 | +Brendan Jackman <brendan.jackman@linux.dev> <jackmanb@google.com> | |
| 173 | 175 | Brian Avery <b.avery@hp.com> |
| 174 | 176 | Brian Cain <bcain@kernel.org> <brian.cain@oss.qualcomm.com> |
| 175 | 177 | Brian Cain <bcain@kernel.org> <bcain@quicinc.com> |
| 176 | 178 | Brian King <brking@us.ibm.com> |
| 177 | 179 | Brian Silverman <bsilver16384@gmail.com> <brian.silverman@bluerivertech.com> |
| 178 | 180 | Bryan Tan <bryan-bt.tan@broadcom.com> <bryantan@vmware.com> |
| 181 | +Burak Emir <burak.emir@gmail.com> <bqe@google.com> | |
| 179 | 182 | Cai Huoqing <cai.huoqing@linux.dev> <caihuoqing@baidu.com> |
| 180 | 183 | Casey Connolly <casey.connolly@linaro.org> <caleb.connolly@linaro.org> |
| 181 | 184 | Casey Connolly <casey.connolly@linaro.org> <caleb@connolly.tech> |
| @@ -231,6 +234,8 @@ Daniel Lezcano <daniel.lezcano@kernel.org> <daniel.lezcano@linexp.org> · 49 unmodified lines | ||
| 231 | 234 | Daniel Lezcano <daniel.lezcano@kernel.org> <dlezcano@fr.ibm.com> |
| 232 | 235 | Daniel Thompson <danielt@kernel.org> <daniel.thompson@linaro.org> |
| 233 | 236 | Daniele Alessandrelli <daniele.alessandrelli@gmail.com> <daniele.alessandrelli@intel.com> |
| 237 | +Danila Tikhonov <danila@mainlining.org> <danila@jiaxyga.com> | |
| 238 | +Danila Tikhonov <danila@mainlining.org> <JIaxyga@protonmail.com> | |
| 234 | 239 | Danilo Krummrich <dakr@kernel.org> <dakr@redhat.com> |
| 235 | 240 | David Brownell <david-b@pacbell.net> |
| 236 | 241 | David Collins <quic_collinsd@quicinc.com> <collinsd@codeaurora.org> |
| @@ -296,6 +301,7 @@ Frank Rowand <frowand.list@gmail.com> <frank.rowand@sony.com> · 59 unmodified lines | ||
| 296 | 301 | Frank Rowand <frowand.list@gmail.com> <frank.rowand@sonymobile.com> |
| 297 | 302 | Frank Rowand <frowand.list@gmail.com> <frowand@mvista.com> |
| 298 | 303 | Frank Zago <fzago@systemfabricworks.com> |
| 304 | +Fuad Tabba <fuad.tabba@linux.dev> <tabba@google.com> | |
| 299 | 305 | Gao Xiang <xiang@kernel.org> <gaoxiang25@huawei.com> |
| 300 | 306 | Gao Xiang <xiang@kernel.org> <hsiangkao@aol.com> |
| 301 | 307 | Gao Xiang <xiang@kernel.org> <hsiangkao@linux.alibaba.com> |
| @@ -648,6 +654,7 @@ Nicholas Piggin <npiggin@gmail.com> <npiggin@suse.de> · 346 unmodified lines | ||
| 648 | 654 | Nicholas Piggin <npiggin@gmail.com> <nickpiggin@yahoo.com.au> |
| 649 | 655 | Nicholas Piggin <npiggin@gmail.com> <piggin@cyberone.com.au> |
| 650 | 656 | Nicolas Ferre <nicolas.ferre@microchip.com> <nicolas.ferre@atmel.com> |
| 657 | +Nico Pache <nico.pache@linux.dev> <npache@redhat.com> | |
| 651 | 658 | Nicolas Pitre <nico@fluxnic.net> <nicolas.pitre@linaro.org> |
| 652 | 659 | Nicolas Pitre <nico@fluxnic.net> <nico@linaro.org> |
| 653 | 660 | Nicolas Saenz Julienne <nsaenz@kernel.org> <nsaenzjulienne@suse.de> |
| @@ -695,6 +702,7 @@ Paulo Alcantara <pc@manguebit.org> <palcantara@suse.com> · 41 unmodified lines | ||
| 695 | 702 | Paulo Alcantara <pc@manguebit.org> <pc@manguebit.com> |
| 696 | 703 | Pavankumar Kondeti <quic_pkondeti@quicinc.com> <pkondeti@codeaurora.org> |
| 697 | 704 | Peter A Jonsson <pj@ludd.ltu.se> |
| 705 | +Peter Collingbourne <peter@pcc.me.uk> <pcc@google.com> | |
| 698 | 706 | Peter Hilber <peter.hilber@oss.qualcomm.com> <quic_philber@quicinc.com> |
| 699 | 707 | Peter Oruba <peter.oruba@amd.com> |
| 700 | 708 | Peter Oruba <peter@oruba.de> |
| @@ -10,3 +10,4 @@ Description: · 9 unmodified lines | ||
| 10 | 10 | 0 no adjustment of input current limit. This |
| 11 | 11 | helps for more unusual power sources like |
| 12 | 12 | solar modules. |
| 13 | + ============ =========================================== | |
| @@ -2239,9 +2239,12 @@ IO Latency · 2238 unmodified lines | ||
| 2239 | 2239 | ~~~~~~~~~~ |
| 2240 | 2240 | |
| 2241 | 2241 | This is a cgroup v2 controller for IO workload protection. You provide a group |
| 2242 | −with a latency target, and if the average latency exceeds that target the | |
| 2243 | −controller will throttle any peers that have a lower latency target than the | |
| 2244 | −protected workload. | |
| 2242 | +with a latency target, and if the group misses its target the controller will | |
| 2243 | +throttle any peers that have a lower latency target than the protected | |
| 2244 | +workload. How a miss is detected depends on the device: on rotational devices | |
| 2245 | +the average latency over the window must exceed the target, while on | |
| 2246 | +non-rotational devices a miss is counted once enough of the IOs in the window | |
| 2247 | +individually exceed the target. | |
| 2245 | 2248 | |
| 2246 | 2249 | The limits are only applied at the peer level in the hierarchy. This means that |
| 2247 | 2250 | in the diagram below, only groups A, B, and C will influence each other, and |
| @@ -2258,10 +2261,12 @@ So the ideal way to configure this is to set io.latency in groups A, B, and C. · 10 unmodified lines | ||
| 2258 | 2261 | Generally you do not want to set a value lower than the latency your device |
| 2259 | 2262 | supports. Experiment to find the value that works best for your workload. |
| 2260 | 2263 | Start at higher than the expected latency for your device and, with |
| 2261 | −blkcg_debug_stats enabled, watch the avg_lat value in io.stat for your | |
| 2262 | −workload group to get an idea of the latency you see during normal operation. | |
| 2263 | −Use the avg_lat value as a basis for your real setting, setting at 10-15% | |
| 2264 | −higher than the value in io.stat. | |
| 2264 | +blkcg_debug_stats enabled, observe io.stat for your workload group to get an | |
| 2265 | +idea of the latency you see during normal operation. On rotational devices, | |
| 2266 | +use the avg_lat value as a basis for your real setting, setting it 10-15% | |
| 2267 | +higher. On non-rotational devices io.stat reports no average latency; set | |
| 2268 | +the target based on your device and use the missed/total fields to verify it | |
| 2269 | +is being met. | |
| 2265 | 2270 | |
| 2266 | 2271 | How IO Latency Throttling Works |
| 2267 | 2272 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| @@ -2303,19 +2308,36 @@ IO Latency Interface Files · 35 unmodified lines | ||
| 2303 | 2308 | the blkcg_debug_stats module parameter is enabled (it is disabled by |
| 2304 | 2309 | default). |
| 2305 | 2310 | |
| 2311 | + The reported latency fields depend on the device. Rotational devices | |
| 2312 | + report avg_lat and win; non-rotational devices report missed and total | |
| 2313 | + instead. missed and total are live counters for the current window and | |
| 2314 | + may change between reads. | |
| 2315 | + | |
| 2306 | 2316 | depth |
| 2307 | 2317 | This is the current queue depth for the group. |
| 2308 | 2318 | |
| 2309 | 2319 | avg_lat |
| 2310 | − This is an exponential moving average with a decay rate of 1/exp | |
| 2311 | − bound by the sampling interval. The decay rate interval can be | |
| 2312 | − calculated by multiplying the win value in io.stat by the | |
| 2313 | − corresponding number of samples based on the win value. | |
| 2320 | + (Rotational devices only.) This is an exponential moving | |
| 2321 | + average with a decay rate of 1/exp bound by the sampling | |
| 2322 | + interval. The decay rate interval can be calculated by | |
| 2323 | + multiplying the win value in io.stat by the corresponding number | |
| 2324 | + of samples based on the win value. | |
| 2314 | 2325 | |
| 2315 | 2326 | win |
| 2316 | − The sampling window size in milliseconds. This is the minimum | |
| 2317 | − duration of time between evaluation events. Windows only elapse | |
| 2318 | − with IO activity. Idle periods extend the most recent window. | |
| 2327 | + (Rotational devices only.) The sampling window size in | |
| 2328 | + milliseconds. This is the minimum duration of time between | |
| 2329 | + evaluation events. Windows only elapse with IO activity. Idle | |
| 2330 | + periods extend the most recent window. | |
| 2331 | + | |
| 2332 | + missed | |
| 2333 | + (Non-rotational devices only.) The number of IOs in the | |
| 2334 | + current window whose latency exceeded the target. A group is | |
| 2335 | + considered to be missing its target once missed reaches a | |
| 2336 | + certain ratio of total. | |
| 2337 | + | |
| 2338 | + total | |
| 2339 | + (Non-rotational devices only.) The total number of IOs | |
| 2340 | + accounted in the current window. | |
| 2319 | 2341 | |
| 2320 | 2342 | IO Priority |
| 2321 | 2343 | ~~~~~~~~~~~ |
| @@ -8169,6 +8169,11 @@ Kernel parameters · 8168 unmodified lines | ||
| 8169 | 8169 | q = USB_QUIRK_FORCE_ONE_CONFIG (Device |
| 8170 | 8170 | claims zero configurations, |
| 8171 | 8171 | forcing to 1); |
| 8172 | + r = USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE (Device | |
| 8173 | + fails during initialization when asked for | |
| 8174 | + 9-bytes configuration descriptor request. | |
| 8175 | + Ask for 255-bytes request instead to mirror | |
| 8176 | + Windows' behavior); | |
| 8172 | 8177 | Example: quirks=0781:5580:bk,0a5c:5834:gij |
| 8173 | 8178 | |
| 8174 | 8179 | usbhid.mousepoll= |
| @@ -55,10 +55,14 @@ stable kernels. · 54 unmodified lines | ||
| 55 | 55 | +----------------+-----------------+-----------------+-----------------------------+ |
| 56 | 56 | | Ampere | AmpereOne | AC03_CPU_38 | AMPERE_ERRATUM_AC03_CPU_38 | |
| 57 | 57 | +----------------+-----------------+-----------------+-----------------------------+ |
| 58 | +| Ampere | AmpereOne | AC03_CPU_57 | N/A | | |
| 59 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 58 | 60 | | Ampere | AmpereOne AC04 | AC04_CPU_10 | AMPERE_ERRATUM_AC03_CPU_38 | |
| 59 | 61 | +----------------+-----------------+-----------------+-----------------------------+ |
| 60 | 62 | | Ampere | AmpereOne AC04 | AC04_CPU_23 | AMPERE_ERRATUM_AC04_CPU_23 | |
| 61 | 63 | +----------------+-----------------+-----------------+-----------------------------+ |
| 64 | +| Ampere | AmpereOne AC04 | AC04_CPU_29 | N/A | | |
| 65 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 62 | 66 | +----------------+-----------------+-----------------+-----------------------------+ |
| 63 | 67 | | ARM | Cortex-A510 | #2457168 | ARM64_ERRATUM_2457168 | |
| 64 | 68 | +----------------+-----------------+-----------------+-----------------------------+ |
| @@ -135,7 +135,6 @@ properties: · 134 unmodified lines | ||
| 135 | 135 | required: |
| 136 | 136 | - compatible |
| 137 | 137 | - reg |
| 138 | − - interrupts | |
| 139 | 138 | |
| 140 | 139 | unevaluatedProperties: false |
| 141 | 140 | |
| @@ -18,7 +18,14 @@ properties: · 17 unmodified lines | ||
| 18 | 18 | - const: qcom,sa8255p-ufshc |
| 19 | 19 | |
| 20 | 20 | reg: |
| 21 | − maxItems: 1 | |
| 21 | + minItems: 1 | |
| 22 | + maxItems: 2 | |
| 23 | + | |
| 24 | + reg-names: | |
| 25 | + minItems: 1 | |
| 26 | + items: | |
| 27 | + - const: std | |
| 28 | + - const: mcq | |
| 22 | 29 | |
| 23 | 30 | interrupts: |
| 24 | 31 | maxItems: 1 |
| @@ -118,12 +118,16 @@ attribute-sets: · 117 unmodified lines | ||
| 118 | 118 | doc: >- |
| 119 | 119 | The number of seconds after which a keep alive message is sent to the |
| 120 | 120 | peer |
| 121 | + checks: | |
| 122 | + max: 86400 | |
| 121 | 123 | - |
| 122 | 124 | name: keepalive-timeout |
| 123 | 125 | type: u32 |
| 124 | 126 | doc: >- |
| 125 | 127 | The number of seconds from the last activity after which the peer is |
| 126 | 128 | assumed dead |
| 129 | + checks: | |
| 130 | + max: 86400 | |
| 127 | 131 | - |
| 128 | 132 | name: del-reason |
| 129 | 133 | type: u32 |
| @@ -43,12 +43,13 @@ UMEM also has two rings: the FILL ring and the COMPLETION ring. The · 42 unmodified lines | ||
| 43 | 43 | FILL ring is used by the application to send down addr for the kernel |
| 44 | 44 | to fill in with RX packet data. References to these frames will then |
| 45 | 45 | appear in the RX ring once each packet has been received. The |
| 46 | −COMPLETION ring, on the other hand, contains frame addr that the | |
| 47 | −kernel has transmitted completely and can now be used again by user | |
| 48 | −space, for either TX or RX. Thus, the frame addrs appearing in the | |
| 49 | −COMPLETION ring are addrs that were previously transmitted using the | |
| 50 | −TX ring. In summary, the RX and FILL rings are used for the RX path | |
| 51 | −and the TX and COMPLETION rings are used for the TX path. | |
| 46 | +COMPLETION ring, on the other hand, contains frame addresses from Tx | |
| 47 | +descriptors that the kernel has finished processing and that can now be | |
| 48 | +used again by user space, for either Tx or Rx. This includes frames whose | |
| 49 | +transmission has completed as well as frames referenced by invalid Tx | |
| 50 | +descriptors rejected by the kernel. A completion therefore returns | |
| 51 | +ownership of a frame to user space, but does not by itself guarantee that | |
| 52 | +the packet was successfully transmitted. | |
| 52 | 53 | |
| 53 | 54 | The socket is then finally bound with a bind() call to a device and a |
| 54 | 55 | specific queue id on that device, and it is not until bind is |
| @@ -169,14 +170,15 @@ chunks mode, then the incoming addr will be left untouched. · 114 unmodified lines | ||
| 169 | 170 | UMEM Completion Ring |
| 170 | 171 | ~~~~~~~~~~~~~~~~~~~~ |
| 171 | 172 | |
| 172 | −The COMPLETION Ring is used transfer ownership of UMEM frames from | |
| 173 | +The COMPLETION Ring is used to transfer ownership of UMEM frames from | |
| 173 | 174 | kernel-space to user-space. Just like the FILL ring, UMEM indices are |
| 174 | −used. | |
| 175 | − | |
| 176 | −Frames passed from the kernel to user-space are frames that has been | |
| 177 | −sent (TX ring) and can be used by user-space again. | |
| 178 | − | |
| 179 | −The user application consumes UMEM addrs from this ring. | |
| 175 | +used. Frames passed from the kernel to user-space are frames referenced | |
| 176 | +by Tx descriptors that the kernel has finished processing and can be | |
| 177 | +used by user-space again. This includes both frames whose transmission | |
| 178 | +has completed and frames referenced by invalid Tx descriptors that were | |
| 179 | +rejected and reclaimed by the kernel. A completion entry does not | |
| 180 | +guarantee successful packet transmission. The user application consumes | |
| 181 | +UMEM addrs from this ring. | |
| 180 | 182 | |
| 181 | 183 | |
| 182 | 184 | RX Ring |
| @@ -504,21 +506,25 @@ will be treated as an invalid descriptor. · 321 unmodified lines | ||
| 504 | 506 | These are the semantics for producing packets onto AF_XDP Tx ring |
| 505 | 507 | consisting of multiple frames: |
| 506 | 508 | |
| 507 | −* When an invalid descriptor is found, all the other | |
| 508 | − descriptors/frames of this packet are marked as invalid and not | |
| 509 | − completed. The next descriptor is treated as the start of a new | |
| 510 | − packet, even if this was not the intent (because we cannot guess | |
| 511 | − the intent). As before, if your program is producing invalid | |
| 512 | − descriptors you have a bug that must be fixed. | |
| 509 | +* When an invalid descriptor is found, the complete packet is treated as | |
| 510 | + invalid. The kernel consumes descriptors through the descriptor marking | |
| 511 | + the end of the packet and returns all their frame addresses through the | |
| 512 | + COMPLETION ring. A standalone invalid descriptor is treated as a | |
| 513 | + one-descriptor invalid packet. The descriptor following the end of the | |
| 514 | + invalid packet is treated as the start of a new packet. As before, if | |
| 515 | + your program is producing invalid descriptors you have a bug that must | |
| 516 | + be fixed. Rejected descriptors are reported in the ``tx_invalid_descs`` | |
| 517 | + statistic. | |
| 513 | 518 | |
| 514 | 519 | * Zero length descriptors are treated as invalid descriptors. |
| 515 | 520 | |
| 516 | 521 | * For copy mode, the maximum supported number of frames in a packet is |
| 517 | − equal to CONFIG_MAX_SKB_FRAGS + 1. If it is exceeded, all | |
| 518 | − descriptors accumulated so far are dropped and treated as | |
| 519 | − invalid. To produce an application that will work on any system | |
| 520 | − regardless of this config setting, limit the number of frags to 18, | |
| 521 | − as the minimum value of the config is 17. | |
| 522 | + equal to CONFIG_MAX_SKB_FRAGS + 1. If it is exceeded, all descriptors | |
| 523 | + through the end of the oversized packet are consumed, treated as invalid, | |
| 524 | + and their frame addresses are returned through the COMPLETION ring. To | |
| 525 | + produce an application that will work on any system regardless of this | |
| 526 | + config setting, limit the number of frags to 18, as the minimum value of | |
| 527 | + the config is 17. | |
| 522 | 528 | |
| 523 | 529 | * For zero-copy mode, the limit is up to what the NIC HW |
| 524 | 530 | supports. Usually at least five on the NICs we have checked. We |
| @@ -57,3 +57,40 @@ Basic development tools (git, gcc, make, editors) should not be listed. · 56 unmodified lines | ||
| 57 | 57 | Example:: |
| 58 | 58 | |
| 59 | 59 | Assisted-by: Claude:claude-3-opus coccinelle sparse |
| 60 | + | |
| 61 | +Procedure for finding and fixing bugs | |
| 62 | +===================================== | |
| 63 | + | |
| 64 | +When an AI assistant is used to find and fix bugs, it **MUST** follow at least | |
| 65 | +these steps: | |
| 66 | + | |
| 67 | +1. Before starting, read the whole process documentation listed above, as well | |
| 68 | + as any other document mentioned in the request. Do not rely on isolated | |
| 69 | + parts found by keyword search. | |
| 70 | +2. Note the commit ID and Locate a bug as instructed. | |
| 71 | +3. For any bug found that is not trivial, verify that it looks real by | |
| 72 | + attempting to create a reproducer to demonstrate it. Lacking it may cause | |
| 73 | + the report to be ignored, as many unverified bug reports sent to maintainers | |
| 74 | + happen to be invalid. Stop here if it finally looks wrong. | |
| 75 | +4. Write a fix for the bug. This part is not optional: except in a few very | |
| 76 | + rare cases, an AI assistant able to find a bug is able to fix it. Note that | |
| 77 | + fixes written in the same session as used to find the bug will generally | |
| 78 | + lead to better and more accurate fixes as the LLM's reasoning context | |
| 79 | + remains present. | |
| 80 | +5. Build and verify that the fix works either using the reproducer or by | |
| 81 | + re-running a complete analysis; drop any fix that doesn't work and try | |
| 82 | + another one. The fix must not add build warnings and must pass the | |
| 83 | + checkpatch.pl checks (see submitting-patches.rst). | |
| 84 | +6. Commit the working fix with a detailed message describing the problem, the | |
| 85 | + solution and a Fixes tag. Do not add a Signed-off-by tag, and add an | |
| 86 | + Assisted-by tag, as described above. | |
| 87 | +7. Identify the maintainers and lists using scripts/get_maintainer.pl. | |
| 88 | + Documentation/process/security-bugs.rst shows how to do that. | |
| 89 | +8. Indicate what could not be done. If the fix could not be built or tested, or | |
| 90 | + if no reproducer could be produced, say so explicitly: maintainers currently | |
| 91 | + waste too much time analyzing unverified reports and untested fixes. | |
| 92 | +9. Read Documentation/process/threat-model.rst to determine whether the bug is | |
| 93 | + a vulnerability or a regular bug, and leave the result to the reporter for | |
| 94 | + review (the assistant must never send anything itself). Regular bugs are | |
| 95 | + submitted as described in Documentation/process/submitting-patches.rst, | |
| 96 | + vulnerabilities as described in Documentation/process/security-bugs.rst. | |
| @@ -18,6 +18,10 @@ information is helpful. The following information are absolutely necessary in · 17 unmodified lines | ||
| 18 | 18 | **any** security bug report: |
| 19 | 19 | |
| 20 | 20 | * **affected kernel version range**: with no version indication, your report |
| 21 | + will not be processed. Note that a time-dependent version such as "latest | |
| 22 | + mainline" is not acceptable. A stable identifier such as a commit ID or an | |
| 23 | + exact version is required. Versions designating kernels not coming from | |
| 24 | + kernel.org (such as distro kernels) are meaningless to maintainers and | |
| 21 | 25 | will not be processed. A significant part of reports are for bugs that |
| 22 | 26 | have already been fixed, so it is extremely important that vulnerabilities |
| 23 | 27 | are verified on recent versions (development tree or latest stable |
| @@ -225,6 +229,28 @@ there is no need to consume a maintainer's time with an unimportant report. If · 201 unmodified lines | ||
| 225 | 229 | the issue is clearly trivial and publicly discoverable, you should report it |
| 226 | 230 | directly to the public mailing lists. |
| 227 | 231 | |
| 232 | +At the very least, when using an AI assistant to find and report bugs, make the | |
| 233 | +assistant read this file and threat-model.rst before proceeding, and have it | |
| 234 | +follow the procedure described in coding-assistants.rst: work on an up-to-date | |
| 235 | +mainline tree and note the commit ID, verify the bug is real, write a fix, | |
| 236 | +build it warning-free and checkpatch-clean, commit it with a Fixes tag, and | |
| 237 | +identify the maintainers with get_maintainer.pl. | |
| 238 | + | |
| 239 | +On top of that procedure, the AI assistant **MUST**: | |
| 240 | + | |
| 241 | +1. Prepare a plain-text report explaining the problem. It must contain the | |
| 242 | + four items listed at the top of this file as absolutely necessary: the | |
| 243 | + affected version or commit ID noted while following the procedure above, | |
| 244 | + the description of the problem, the reproducer or its status, and the | |
| 245 | + triggering conditions. | |
| 246 | +2. Start the report with a temporary section listing the recipients' addresses | |
| 247 | + (maintainers+list for the patch, maintainers only for the report and | |
| 248 | + reproducer), and with instructions reminding the reporter to check that | |
| 249 | + their email client is properly setup (see email-clients.rst), and leave it | |
| 250 | + to the reporter to remove that temporary section. | |
| 251 | + | |
| 252 | +A more detailed process is covered at https://github.com/masoncl/kres.git. | |
| 253 | + | |
| 228 | 254 | Sending the report |
| 229 | 255 | ------------------ |
| 230 | 256 | |
| @@ -98,11 +98,11 @@ measures whose purpose is to avoid crossing a security boundary when certain · 97 unmodified lines | ||
| 98 | 98 | classes of bugs are found, but a failure of these extra protections do not |
| 99 | 99 | constitute a vulnerability alone. |
| 100 | 100 | |
| 101 | −What does not constitute a security bug | |
| 102 | −--------------------------------------- | |
| 101 | +What classes of problems are not considered vulnerabilities | |
| 102 | +----------------------------------------------------------- | |
| 103 | 103 | |
| 104 | 104 | In the Linux kernel's threat model, the following classes of problems are |
| 105 | −**NOT** considered as Linux Kernel security bugs. However, when it is believed | |
| 105 | +**NOT** considered Linux Kernel vulnerabilities. However, when it is believed | |
| 106 | 106 | that the kernel could do better, they should be reported, so that they can be |
| 107 | 107 | reviewed and fixed where reasonably possible, but they will be handled as any |
| 108 | 108 | regular bug: |
| @@ -111,8 +111,8 @@ regular bug: · 2 unmodified lines | ||
| 111 | 111 | |
| 112 | 112 | * outdated kernels and particularly end-of-life branches are out of the scope |
| 113 | 113 | of the kernel's threat model: administrators are responsible for keeping |
| 114 | − their system up to date. For a bug to qualify as a security bug, it must be | |
| 115 | − demonstrated that it affects actively maintained versions. | |
| 114 | + their system up to date. For a bug to qualify as a vulnerability, it must | |
| 115 | + be demonstrated that it affects actively maintained versions. | |
| 116 | 116 | |
| 117 | 117 | * build-level: changes to the kernel configuration that are explicitly |
| 118 | 118 | documented as lowering the security level (e.g. ``CONFIG_NOMMU``), or |
| @@ -178,9 +178,6 @@ regular bug: · 59 unmodified lines | ||
| 178 | 178 | involving tens of millions of threads, tens of thousands of CPUs, |
| 179 | 179 | unrealistic CPU frequencies, RAM sizes or disk capacities, network speeds). |
| 180 | 180 | |
| 181 | − * issues whose reproduction requires hardware modification or emulation, | |
| 182 | − including fake USB devices that pretend to be another one. | |
| 183 | − | |
| 184 | 181 | * as well as issues that can be triggered at a cost that is orders of |
| 185 | 182 | magnitude higher than the expected benefits (e.g. fully functional keyboard |
| 186 | 183 | emulator only to retrieve 7 uninitialized bytes in a structure, or |
| @@ -208,19 +205,27 @@ regular bug: · 21 unmodified lines | ||
| 208 | 205 | messages. |
| 209 | 206 | |
| 210 | 207 | * Leaks of kernel memory addresses/pointers do not constitute an immediately |
| 211 | − exploitable vector and are not security bugs, though they must be reported | |
| 212 | − and fixed. | |
| 208 | + exploitable vector and are not vulnerabilities, though they must be | |
| 209 | + reported and fixed. | |
| 213 | 210 | |
| 214 | −* **Crafted file system images**: | |
| 211 | +* **Non-conforming devices and media**: | |
| 215 | 212 | |
| 216 | − * bugs triggered by mounting a corrupted or maliciously crafted file system | |
| 217 | − image are generally not security bugs, as the kernel assumes the underlying | |
| 218 | − storage media is under the administrator's control, unless the filesystem | |
| 219 | − driver is specifically documented as being hardened against untrusted media. | |
| 213 | + Drivers are implemented against a specification. When a device or a storage | |
| 214 | + medium violates the specification its driver was written against, the | |
| 215 | + resulting misbehaviour is a regular bug to be fixed, not a vulnerability, | |
| 216 | + unless the driver is specifically documented as being hardened against | |
| 217 | + hostile inputs. The following are therefore not considered vulnerabilities: | |
| 220 | 218 | |
| 221 | − * issues that are resolved, mitigated, or detected by running a filesystem | |
| 219 | + * bugs triggered by mounting a corrupted or maliciously crafted file system | |
| 220 | + image: mounting a block device is a privileged operation (see above), and | |
| 221 | + the administrator is responsible for the media they mount. This includes | |
| 222 | + issues that are resolved, mitigated, or detected by running a filesystem | |
| 222 | 223 | consistency check (fsck) on the image prior to mounting. |
| 223 | 224 | |
| 225 | + * bugs whose reproduction requires hardware modification or emulation, | |
| 226 | + including fake USB devices that pretend to be another one, or devices | |
| 227 | + reporting values outside their documented ranges. | |
| 228 | + | |
| 224 | 229 | * **Physical access**: |
| 225 | 230 | |
| 226 | 231 | Issues that require physical access to the machine, hardware modification, or |
| @@ -232,4 +237,4 @@ regular bug: · 5 unmodified lines | ||
| 232 | 237 | * **Functional and performance regressions**: |
| 233 | 238 | |
| 234 | 239 | Any issue that can be mitigated by setting proper permissions and limits |
| 235 | − doesn't qualify as a security bug. | |
| 240 | + doesn't qualify as a vulnerability. | |
| @@ -8414,6 +8414,12 @@ When this capability is enabled all memory in memslots must be mapped as · 8413 unmodified lines | ||
| 8414 | 8414 | attempts to create a memslot with an invalid mmap will result in an |
| 8415 | 8415 | -EINVAL return. |
| 8416 | 8416 | |
| 8417 | +``guest_memfd``, even though it is an anonymous file, is not supported with MTE. | |
| 8418 | +Attempting to create a memslot backed by ``guest_memfd`` when the MTE capability | |
| 8419 | +is enabled, or attempting to enable the MTE capability after | |
| 8420 | +``guest_memfd``-backed memslots have been created, will result in an -EINVAL | |
| 8421 | +return. | |
| 8422 | + | |
| 8417 | 8423 | When enabled the VMM may make use of the ``KVM_ARM_MTE_COPY_TAGS`` ioctl to |
| 8418 | 8424 | perform a bulk copy of tags to/from the guest. |
| 8419 | 8425 | |
| @@ -112,9 +112,20 @@ Groups: · 111 unmodified lines | ||
| 112 | 112 | mask or unmask the adapter, as specified in mask |
| 113 | 113 | |
| 114 | 114 | KVM_S390_IO_ADAPTER_MAP |
| 115 | − This is now a no-op. The mapping is purely done by the irq route. | |
| 115 | + Map an adapter indicator or summary page for long-term pinning so that | |
| 116 | + interrupt injection can be performed in atomic context. If long-term | |
| 117 | + pinning is not possible (e.g. file-backed memory), the page is verified | |
| 118 | + via a short-term pin and the ioctl returns success; interrupt injection | |
| 119 | + will use the non-atomic irqfd path with short-term pinning on each | |
| 120 | + interrupt. In Secure Execution mode this is a no-op and the ioctl | |
| 121 | + returns success. | |
| 122 | + | |
| 116 | 123 | KVM_S390_IO_ADAPTER_UNMAP |
| 117 | − This is now a no-op. The mapping is purely done by the irq route. | |
| 124 | + Unmap a previously mapped adapter indicator or summary page and release | |
| 125 | + the long-term pin. If the page was not long-term pinned (e.g. file-backed | |
| 126 | + memory), the map entry is removed and success is returned; if no prior | |
| 127 | + map entry exists, -ENOENT is returned. In Secure Execution mode this is | |
| 128 | + a no-op and the ioctl returns success. | |
| 118 | 129 | |
| 119 | 130 | KVM_DEV_FLIC_AISM |
| 120 | 131 | modify the adapter-interruption-suppression mode for a given isc if the |
| @@ -2676,6 +2676,8 @@ F: drivers/irqchip/irq-aspeed-i2c-ic.c · 2675 unmodified lines | ||
| 2676 | 2676 | ARM/ASPEED MACHINE SUPPORT |
| 2677 | 2677 | M: Joel Stanley <joel@jms.id.au> |
| 2678 | 2678 | M: Andrew Jeffery <andrew@codeconstruct.com.au> |
| 2679 | +R: Ryan Chen <ryan_chen@aspeedtech.com> | |
| 2680 | +R: Billy Tsai <billy_tsai@aspeedtech.com> | |
| 2679 | 2681 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 2680 | 2682 | L: linux-aspeed@lists.ozlabs.org (moderated for non-subscribers) |
| 2681 | 2683 | S: Supported |
| @@ -2881,7 +2883,7 @@ W: http://www.armlinux.org.uk/ · 199 unmodified lines | ||
| 2881 | 2883 | F: arch/arm/include/asm/hardware/dec21285.h |
| 2882 | 2884 | F: arch/arm/mach-footbridge/ |
| 2883 | 2885 | |
| 2884 | −ARM/FREESCALE IMX / MXC ARM ARCHITECTURE | |
| 2886 | +ARM/FREESCALE IMX / MXC / LAYERSCAPE ARM ARCHITECTURE | |
| 2885 | 2887 | M: Frank Li <Frank.Li@nxp.com> |
| 2886 | 2888 | M: Sascha Hauer <s.hauer@pengutronix.de> |
| 2887 | 2889 | R: Pengutronix Kernel Team <kernel@pengutronix.de> |
| @@ -2895,22 +2897,11 @@ F: Documentation/devicetree/bindings/firmware/nxp* · 7 unmodified lines | ||
| 2895 | 2897 | F: arch/arm/boot/dts/nxp/ |
| 2896 | 2898 | F: arch/arm64/boot/dts/freescale/ |
| 2897 | 2899 | X: Documentation/devicetree/bindings/media/i2c/ |
| 2898 | −X: arch/arm64/boot/dts/freescale/fsl-* | |
| 2899 | −X: arch/arm64/boot/dts/freescale/qoriq-* | |
| 2900 | 2900 | X: drivers/media/i2c/ |
| 2901 | 2901 | N: imx |
| 2902 | 2902 | N: mxs |
| 2903 | 2903 | N: \bmxc[^\d] |
| 2904 | 2904 | |
| 2905 | −ARM/FREESCALE LAYERSCAPE ARM ARCHITECTURE | |
| 2906 | −M: Frank Li <Frank.Li@nxp.com> | |
| 2907 | −L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) | |
| 2908 | −S: Maintained | |
| 2909 | −T: git git://git.kernel.org/pub/scm/linux/kernel/git/frank.li/linux.git | |
| 2910 | −F: arch/arm/boot/dts/nxp/ls/ | |
| 2911 | −F: arch/arm64/boot/dts/freescale/fsl-* | |
| 2912 | −F: arch/arm64/boot/dts/freescale/qoriq-* | |
| 2913 | − | |
| 2914 | 2905 | ARM/FREESCALE VYBRID ARM ARCHITECTURE |
| 2915 | 2906 | M: Frank Li <Frank.Li@nxp.com> |
| 2916 | 2907 | M: Sascha Hauer <s.hauer@pengutronix.de> |
| @@ -4634,7 +4625,7 @@ F: rust/helpers/cpumask.c · 1717 unmodified lines | ||
| 4634 | 4625 | |
| 4635 | 4626 | BITMAP API [RUST] |
| 4636 | 4627 | M: Alice Ryhl <aliceryhl@google.com> |
| 4637 | −M: Burak Emir <bqe@google.com> | |
| 4628 | +M: Burak Emir <burak.emir@gmail.com> | |
| 4638 | 4629 | R: Yury Norov <yury.norov@gmail.com> |
| 4639 | 4630 | S: Maintained |
| 4640 | 4631 | F: lib/find_bit_benchmark_rust.rs |
| @@ -9617,7 +9608,7 @@ M: Chao Yu <chao@kernel.org> · 4976 unmodified lines | ||
| 9617 | 9608 | R: Yue Hu <zbestahu@gmail.com> |
| 9618 | 9609 | R: Jeffle Xu <jefflexu@linux.alibaba.com> |
| 9619 | 9610 | R: Sandeep Dhavale <dhavale@google.com> |
| 9620 | −R: Hongbo Li <lihongbo22@huawei.com> | |
| 9611 | +R: Hongbo Li <hongbohbli@tencent.com> | |
| 9621 | 9612 | R: Chunhai Guo <guochunhai@vivo.com> |
| 9622 | 9613 | L: linux-erofs@lists.ozlabs.org |
| 9623 | 9614 | S: Maintained |
| @@ -10134,6 +10125,11 @@ F: drivers/base/firmware_loader/ · 510 unmodified lines | ||
| 10134 | 10125 | F: rust/kernel/firmware.rs |
| 10135 | 10126 | F: include/linux/firmware.h |
| 10136 | 10127 | |
| 10128 | +FIXED-LAYOUT NVMEM LAYOUT DRIVER | |
| 10129 | +M: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> | |
| 10130 | +S: Maintained | |
| 10131 | +F: drivers/nvmem/layouts/fixed-layout.c | |
| 10132 | + | |
| 10137 | 10133 | FLEXTIMER FTM-QUADDEC DRIVER |
| 10138 | 10134 | M: Patrick Havelange <patrick.havelange@essensium.com> |
| 10139 | 10135 | L: linux-iio@vger.kernel.org |
| @@ -11743,6 +11739,7 @@ F: drivers/net/ethernet/hisilicon/hibmcge/ · 1603 unmodified lines | ||
| 11743 | 11739 | |
| 11744 | 11740 | HISILICON NETWORK SUBSYSTEM DRIVER |
| 11745 | 11741 | M: Jian Shen <shenjian15@huawei.com> |
| 11742 | +M: Jijie Shao <shaojijie@huawei.com> | |
| 11746 | 11743 | L: netdev@vger.kernel.org |
| 11747 | 11744 | S: Maintained |
| 11748 | 11745 | W: http://www.hisilicon.com |
| @@ -14193,7 +14190,7 @@ F: virt/kvm/* · 2444 unmodified lines | ||
| 14193 | 14190 | KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64) |
| 14194 | 14191 | M: Marc Zyngier <maz@kernel.org> |
| 14195 | 14192 | M: Oliver Upton <oupton@kernel.org> |
| 14196 | −R: Fuad Tabba <tabba@google.com> | |
| 14193 | +R: Fuad Tabba <fuad.tabba@linux.dev> | |
| 14197 | 14194 | R: Joey Gouly <joey.gouly@arm.com> |
| 14198 | 14195 | R: Steffen Eiden <seiden@linux.ibm.com> |
| 14199 | 14196 | R: Suzuki K Poulose <suzuki.poulose@arm.com> |
| @@ -14887,7 +14884,7 @@ X: drivers/macintosh/via-macii.c · 687 unmodified lines | ||
| 14887 | 14884 | |
| 14888 | 14885 | LINUX FOR POWERPC (32-BIT AND 64-BIT) |
| 14889 | 14886 | M: Madhavan Srinivasan <maddy@linux.ibm.com> |
| 14890 | −M: Michael Ellerman <mpe@ellerman.id.au> | |
| 14887 | +R: Michael Ellerman <mpe@ellerman.id.au> | |
| 14891 | 14888 | R: Nicholas Piggin <npiggin@gmail.com> |
| 14892 | 14889 | R: Christophe Leroy (CS GROUP) <chleroy@kernel.org> |
| 14893 | 14890 | L: linuxppc-dev@lists.ozlabs.org |
| @@ -17160,7 +17157,7 @@ M: Andrew Morton <akpm@linux-foundation.org> · 2266 unmodified lines | ||
| 17160 | 17157 | M: Vlastimil Babka <vbabka@kernel.org> |
| 17161 | 17158 | R: Suren Baghdasaryan <surenb@google.com> |
| 17162 | 17159 | R: Michal Hocko <mhocko@suse.com> |
| 17163 | −R: Brendan Jackman <jackmanb@google.com> | |
| 17160 | +R: Brendan Jackman <brendan.jackman@linux.dev> | |
| 17164 | 17161 | R: Johannes Weiner <hannes@cmpxchg.org> |
| 17165 | 17162 | R: Zi Yan <ziy@nvidia.com> |
| 17166 | 17163 | L: linux-mm@kvack.org |
| @@ -17253,7 +17250,7 @@ M: Lorenzo Stoakes <ljs@kernel.org> · 86 unmodified lines | ||
| 17253 | 17250 | R: Zi Yan <ziy@nvidia.com> |
| 17254 | 17251 | R: Baolin Wang <baolin.wang@linux.alibaba.com> |
| 17255 | 17252 | R: Liam R. Howlett <liam@infradead.org> |
| 17256 | −R: Nico Pache <npache@redhat.com> | |
| 17253 | +R: Nico Pache <nico.pache@linux.dev> | |
| 17257 | 17254 | R: Ryan Roberts <ryan.roberts@arm.com> |
| 17258 | 17255 | R: Dev Jain <dev.jain@arm.com> |
| 17259 | 17256 | R: Barry Song <baohua@kernel.org> |
| @@ -17846,7 +17843,7 @@ F: drivers/net/wireless/microchip/ · 586 unmodified lines | ||
| 17846 | 17843 | |
| 17847 | 17844 | MICROCHIP ZL3073X DRIVER |
| 17848 | 17845 | M: Ivan Vecera <ivecera@redhat.com> |
| 17849 | −M: Prathosh Satish <Prathosh.Satish@microchip.com> | |
| 17846 | +M: Min Li <min.li@microchip.com> | |
| 17850 | 17847 | L: netdev@vger.kernel.org |
| 17851 | 17848 | S: Supported |
| 17852 | 17849 | F: Documentation/devicetree/bindings/dpll/microchip,zl30731.yaml |
| @@ -20692,7 +20689,6 @@ F: include/linux/switchtec.h · 2839 unmodified lines | ||
| 20692 | 20689 | F: include/uapi/linux/switchtec_ioctl.h |
| 20693 | 20690 | |
| 20694 | 20691 | PCI DRIVER FOR MOBIVEIL PCIE IP |
| 20695 | −M: Karthikeyan Mitran <m.karthikeyan@mobiveil.co.in> | |
| 20696 | 20692 | M: Hou Zhiqiang <Zhiqiang.Hou@nxp.com> |
| 20697 | 20693 | L: linux-pci@vger.kernel.org |
| 20698 | 20694 | S: Supported |
| @@ -22736,7 +22732,8 @@ F: drivers/watchdog/realtek_otto_wdt.c · 2037 unmodified lines | ||
| 22736 | 22732 | |
| 22737 | 22733 | REALTEK RTL83xx SMI DSA ROUTER CHIPS |
| 22738 | 22734 | M: Linus Walleij <linusw@kernel.org> |
| 22739 | −M: Alvin Šipraga <alsi@bang-olufsen.dk> | |
| 22735 | +M: Luiz Angelo Daros de Luca <luizluca@gmail.com> | |
| 22736 | +R: Alvin Šipraga <alvin.sipraga@analog.com> | |
| 22740 | 22737 | S: Maintained |
| 22741 | 22738 | F: Documentation/devicetree/bindings/net/dsa/realtek.yaml |
| 22742 | 22739 | F: drivers/net/dsa/realtek/* |
| @@ -24633,8 +24630,7 @@ SHARED MEMORY COMMUNICATIONS (SMC) SOCKETS · 1890 unmodified lines | ||
| 24633 | 24630 | M: D. Wythe <alibuda@linux.alibaba.com> |
| 24634 | 24631 | M: Dust Li <dust.li@linux.alibaba.com> |
| 24635 | 24632 | M: Sidraya Jayagond <sidraya@linux.ibm.com> |
| 24636 | −M: Wenjia Zhang <wenjia@linux.ibm.com> | |
| 24637 | −R: Mahanta Jambigi <mjambigi@linux.ibm.com> | |
| 24633 | +M: Mahanta Jambigi <mjambigi@linux.ibm.com> | |
| 24638 | 24634 | R: Tony Lu <tonylu@linux.alibaba.com> |
| 24639 | 24635 | R: Wen Gu <guwen@linux.alibaba.com> |
| 24640 | 24636 | L: linux-rdma@vger.kernel.org |
| @@ -25430,6 +25426,12 @@ S: Maintained · 789 unmodified lines | ||
| 25430 | 25426 | F: Documentation/devicetree/bindings/i2c/spacemit,k1-i2c.yaml |
| 25431 | 25427 | F: drivers/i2c/busses/i2c-k1.c |
| 25432 | 25428 | |
| 25429 | +SPACEMIT K1/K3 I2S DRIVER | |
| 25430 | +M: Troy Mitchell <troy.mitchell@linux.spacemit.com> | |
| 25431 | +S: Maintained | |
| 25432 | +F: Documentation/devicetree/bindings/sound/spacemit,k1-i2s.yaml | |
| 25433 | +F: sound/soc/spacemit/k1_i2s.c | |
| 25434 | + | |
| 25433 | 25435 | SPANISH DOCUMENTATION |
| 25434 | 25436 | M: Carlos Bilbao <carlos.bilbao@kernel.org> |
| 25435 | 25437 | R: Avadhut Naik <avadhut.naik@amd.com> |
| @@ -2,7 +2,7 @@ · 1 unmodified line | ||
| 2 | 2 | VERSION = 7 |
| 3 | 3 | PATCHLEVEL = 2 |
| 4 | 4 | SUBLEVEL = 0 |
| 5 | −EXTRAVERSION = -rc5 | |
| 5 | +EXTRAVERSION = -rc7 | |
| 6 | 6 | NAME = Baby Opossum Posse |
| 7 | 7 | |
| 8 | 8 | # *DOCUMENTATION* |
| @@ -700,13 +700,11 @@ filechk_makefile = { \ · 691 unmodified lines | ||
| 700 | 700 | echo "include $(abs_srctree)/Makefile"; \ |
| 701 | 701 | } |
| 702 | 702 | |
| 703 | −$(objtree)/Makefile: FORCE | |
| 703 | +PHONY += $(CURDIR)/Makefile | |
| 704 | +$(CURDIR)/Makefile: FORCE | |
| 704 | 705 | $(call filechk,makefile) |
| 705 | 706 | |
| 706 | −# Prevent $(srcroot)/Makefile from inhibiting the rule to run. | |
| 707 | −PHONY += $(objtree)/Makefile | |
| 708 | − | |
| 709 | −outputmakefile: $(objtree)/Makefile | |
| 707 | +outputmakefile: $(CURDIR)/Makefile | |
| 710 | 708 | ifeq ($(KBUILD_EXTMOD),) |
| 711 | 709 | @if [ -f $(srctree)/.config -o \ |
| 712 | 710 | -d $(srctree)/include/config -o \ |
| @@ -141,7 +141,7 @@ · 140 unmodified lines | ||
| 141 | 141 | |
| 142 | 142 | /* PCIe Controller 2 */ |
| 143 | 143 | <0x00014000 0 &gic GIC_SPI 138 IRQ_TYPE_LEVEL_HIGH>, |
| 144 | − <0x00014000 1 &gic GIC_SPI 138 IRQ_TYPE_LEVEL_HIGH>, | |
| 144 | + <0x00014000 1 &gic GIC_SPI 139 IRQ_TYPE_LEVEL_HIGH>, | |
| 145 | 145 | <0x00014000 2 &gic GIC_SPI 140 IRQ_TYPE_LEVEL_HIGH>, |
| 146 | 146 | <0x00014000 3 &gic GIC_SPI 141 IRQ_TYPE_LEVEL_HIGH>, |
| 147 | 147 | <0x00014000 4 &gic GIC_SPI 142 IRQ_TYPE_LEVEL_HIGH>, |
| @@ -26,7 +26,7 @@ · 25 unmodified lines | ||
| 26 | 26 | |
| 27 | 27 | nvram@1c080000 { |
| 28 | 28 | compatible = "brcm,nvram"; |
| 29 | − reg = <0x1c080000 0x180000>; | |
| 29 | + reg = <0x1c080000 0x100000>; | |
| 30 | 30 | |
| 31 | 31 | et2macaddr: et2macaddr { |
| 32 | 32 | #nvmem-cell-cells = <1>; |
| @@ -32,6 +32,7 @@ static int npcm7xx_smp_boot_secondary(unsigned int cpu, · 31 unmodified lines | ||
| 32 | 32 | goto out; |
| 33 | 33 | } |
| 34 | 34 | gcr_base = of_iomap(gcr_np, 0); |
| 35 | + of_node_put(gcr_np); | |
| 35 | 36 | if (!gcr_base) { |
| 36 | 37 | pr_err("could not iomap gcr"); |
| 37 | 38 | ret = -ENOMEM; |
| @@ -60,6 +61,7 @@ static void __init npcm7xx_smp_prepare_cpus(unsigned int max_cpus) · 22 unmodified lines | ||
| 60 | 61 | return; |
| 61 | 62 | } |
| 62 | 63 | scu_base = of_iomap(scu_np, 0); |
| 64 | + of_node_put(scu_np); | |
| 63 | 65 | if (!scu_base) { |
| 64 | 66 | pr_err("could not iomap scu"); |
| 65 | 67 | return; |
| @@ -678,8 +678,6 @@ · 677 unmodified lines | ||
| 678 | 678 | <GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(4) | |
| 679 | 679 | IRQ_TYPE_LEVEL_LOW)>, |
| 680 | 680 | <GIC_PPI 10 (GIC_CPU_MASK_SIMPLE(4) | |
| 681 | − IRQ_TYPE_LEVEL_LOW)>, | |
| 682 | − <GIC_PPI 12 (GIC_CPU_MASK_SIMPLE(4) | | |
| 683 | 681 | IRQ_TYPE_LEVEL_LOW)>; |
| 684 | 682 | }; |
| 685 | 683 | |
| @@ -2559,7 +2559,7 @@ · 2558 unmodified lines | ||
| 2559 | 2559 | "dsi_pll"; |
| 2560 | 2560 | |
| 2561 | 2561 | clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>, |
| 2562 | − <&rpmhcc RPMH_CXO_CLK>; | |
| 2562 | + <&bi_tcxo_div2>; | |
| 2563 | 2563 | clock-names = "iface", |
| 2564 | 2564 | "ref"; |
| 2565 | 2565 | |
| @@ -1876,7 +1876,7 @@ · 1875 unmodified lines | ||
| 1876 | 1876 | spi0: spi@b80000 { |
| 1877 | 1877 | compatible = "qcom,geni-spi"; |
| 1878 | 1878 | reg = <0x0 0x00b80000 0x0 0x4000>; |
| 1879 | − interrupts = <GIC_SPI 1052 IRQ_TYPE_LEVEL_HIGH>; | |
| 1879 | + interrupts = <GIC_ESPI 92 IRQ_TYPE_LEVEL_HIGH>; | |
| 1880 | 1880 | clocks = <&gcc GCC_QUPV3_WRAP0_S0_CLK>; |
| 1881 | 1881 | clock-names = "se"; |
| 1882 | 1882 | interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS |
| @@ -1903,7 +1903,7 @@ · 20 unmodified lines | ||
| 1903 | 1903 | i2c1: i2c@b84000 { |
| 1904 | 1904 | compatible = "qcom,geni-i2c"; |
| 1905 | 1905 | reg = <0x0 0x00b84000 0x0 0x4000>; |
| 1906 | − interrupts = <GIC_SPI 1053 IRQ_TYPE_LEVEL_HIGH>; | |
| 1906 | + interrupts = <GIC_ESPI 93 IRQ_TYPE_LEVEL_HIGH>; | |
| 1907 | 1907 | clocks = <&gcc GCC_QUPV3_WRAP0_S1_CLK>; |
| 1908 | 1908 | clock-names = "se"; |
| 1909 | 1909 | interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS |
| @@ -1930,7 +1930,7 @@ · 20 unmodified lines | ||
| 1930 | 1930 | spi1: spi@b84000 { |
| 1931 | 1931 | compatible = "qcom,geni-spi"; |
| 1932 | 1932 | reg = <0x0 0x00b84000 0x0 0x4000>; |
| 1933 | − interrupts = <GIC_SPI 1053 IRQ_TYPE_LEVEL_HIGH>; | |
| 1933 | + interrupts = <GIC_ESPI 93 IRQ_TYPE_LEVEL_HIGH>; | |
| 1934 | 1934 | clocks = <&gcc GCC_QUPV3_WRAP0_S1_CLK>; |
| 1935 | 1935 | clock-names = "se"; |
| 1936 | 1936 | interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS |
| @@ -1957,7 +1957,7 @@ · 20 unmodified lines | ||
| 1957 | 1957 | i2c2: i2c@b88000 { |
| 1958 | 1958 | compatible = "qcom,geni-i2c"; |
| 1959 | 1959 | reg = <0x0 0x00b88000 0x0 0x4000>; |
| 1960 | − interrupts = <GIC_SPI 1054 IRQ_TYPE_LEVEL_HIGH>; | |
| 1960 | + interrupts = <GIC_ESPI 94 IRQ_TYPE_LEVEL_HIGH>; | |
| 1961 | 1961 | clocks = <&gcc GCC_QUPV3_WRAP0_S2_CLK>; |
| 1962 | 1962 | clock-names = "se"; |
| 1963 | 1963 | interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS |
| @@ -1984,7 +1984,7 @@ · 20 unmodified lines | ||
| 1984 | 1984 | spi2: spi@b88000 { |
| 1985 | 1985 | compatible = "qcom,geni-spi"; |
| 1986 | 1986 | reg = <0x0 0x00b88000 0x0 0x4000>; |
| 1987 | − interrupts = <GIC_SPI 1054 IRQ_TYPE_LEVEL_HIGH>; | |
| 1987 | + interrupts = <GIC_ESPI 94 IRQ_TYPE_LEVEL_HIGH>; | |
| 1988 | 1988 | clocks = <&gcc GCC_QUPV3_WRAP0_S2_CLK>; |
| 1989 | 1989 | clock-names = "se"; |
| 1990 | 1990 | interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS |
| @@ -2011,7 +2011,7 @@ · 20 unmodified lines | ||
| 2011 | 2011 | uart2: serial@b88000 { |
| 2012 | 2012 | compatible = "qcom,geni-uart"; |
| 2013 | 2013 | reg = <0x0 0x00b88000 0x0 0x4000>; |
| 2014 | − interrupts = <GIC_SPI 1054 IRQ_TYPE_LEVEL_HIGH>; | |
| 2014 | + interrupts = <GIC_ESPI 94 IRQ_TYPE_LEVEL_HIGH>; | |
| 2015 | 2015 | clocks = <&gcc GCC_QUPV3_WRAP0_S2_CLK>; |
| 2016 | 2016 | clock-names = "se"; |
| 2017 | 2017 | interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS |
| @@ -2056,7 +2056,7 @@ · 38 unmodified lines | ||
| 2056 | 2056 | spi3: spi@b8c000 { |
| 2057 | 2057 | compatible = "qcom,geni-spi"; |
| 2058 | 2058 | reg = <0x0 0x00b8c000 0x0 0x4000>; |
| 2059 | − interrupts = <GIC_SPI 1055 IRQ_TYPE_LEVEL_HIGH>; | |
| 2059 | + interrupts = <GIC_ESPI 95 IRQ_TYPE_LEVEL_HIGH>; | |
| 2060 | 2060 | clocks = <&gcc GCC_QUPV3_WRAP0_S3_CLK>; |
| 2061 | 2061 | clock-names = "se"; |
| 2062 | 2062 | interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS |
| @@ -2110,7 +2110,7 @@ · 47 unmodified lines | ||
| 2110 | 2110 | spi4: spi@b90000 { |
| 2111 | 2111 | compatible = "qcom,geni-spi"; |
| 2112 | 2112 | reg = <0x0 0x00b90000 0x0 0x4000>; |
| 2113 | − interrupts = <GIC_SPI 1056 IRQ_TYPE_LEVEL_HIGH>; | |
| 2113 | + interrupts = <GIC_ESPI 96 IRQ_TYPE_LEVEL_HIGH>; | |
| 2114 | 2114 | clocks = <&gcc GCC_QUPV3_WRAP0_S4_CLK>; |
| 2115 | 2115 | clock-names = "se"; |
| 2116 | 2116 | interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS |
| @@ -2164,7 +2164,7 @@ · 47 unmodified lines | ||
| 2164 | 2164 | spi5: spi@b94000 { |
| 2165 | 2165 | compatible = "qcom,geni-spi"; |
| 2166 | 2166 | reg = <0x0 0x00b94000 0x0 0x4000>; |
| 2167 | − interrupts = <GIC_SPI 1057 IRQ_TYPE_LEVEL_HIGH>; | |
| 2167 | + interrupts = <GIC_ESPI 97 IRQ_TYPE_LEVEL_HIGH>; | |
| 2168 | 2168 | clocks = <&gcc GCC_QUPV3_WRAP0_S5_CLK>; |
| 2169 | 2169 | clock-names = "se"; |
| 2170 | 2170 | interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS |
| @@ -2191,7 +2191,7 @@ · 20 unmodified lines | ||
| 2191 | 2191 | i2c6: i2c@b98000 { |
| 2192 | 2192 | compatible = "qcom,geni-i2c"; |
| 2193 | 2193 | reg = <0x0 0x00b98000 0x0 0x4000>; |
| 2194 | − interrupts = <GIC_SPI 1058 IRQ_TYPE_LEVEL_HIGH>; | |
| 2194 | + interrupts = <GIC_ESPI 98 IRQ_TYPE_LEVEL_HIGH>; | |
| 2195 | 2195 | clocks = <&gcc GCC_QUPV3_WRAP0_S6_CLK>; |
| 2196 | 2196 | clock-names = "se"; |
| 2197 | 2197 | interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS |
| @@ -2218,7 +2218,7 @@ · 20 unmodified lines | ||
| 2218 | 2218 | spi6: spi@b98000 { |
| 2219 | 2219 | compatible = "qcom,geni-spi"; |
| 2220 | 2220 | reg = <0x0 0x00b98000 0x0 0x4000>; |
| 2221 | − interrupts = <GIC_SPI 1058 IRQ_TYPE_LEVEL_HIGH>; | |
| 2221 | + interrupts = <GIC_ESPI 98 IRQ_TYPE_LEVEL_HIGH>; | |
| 2222 | 2222 | clocks = <&gcc GCC_QUPV3_WRAP0_S6_CLK>; |
| 2223 | 2223 | clock-names = "se"; |
| 2224 | 2224 | interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS |
| @@ -2245,7 +2245,7 @@ · 20 unmodified lines | ||
| 2245 | 2245 | i2c7: i2c@b9c000 { |
| 2246 | 2246 | compatible = "qcom,geni-i2c"; |
| 2247 | 2247 | reg = <0x0 0x00b9c000 0x0 0x4000>; |
| 2248 | − interrupts = <GIC_SPI 1059 IRQ_TYPE_LEVEL_HIGH>; | |
| 2248 | + interrupts = <GIC_ESPI 99 IRQ_TYPE_LEVEL_HIGH>; | |
| 2249 | 2249 | clocks = <&gcc GCC_QUPV3_WRAP0_S7_CLK>; |
| 2250 | 2250 | clock-names = "se"; |
| 2251 | 2251 | interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS |
| @@ -2272,7 +2272,7 @@ · 20 unmodified lines | ||
| 2272 | 2272 | spi7: spi@b9c000 { |
| 2273 | 2273 | compatible = "qcom,geni-spi"; |
| 2274 | 2274 | reg = <0x0 0x00b9c000 0x0 0x4000>; |
| 2275 | − interrupts = <GIC_SPI 1059 IRQ_TYPE_LEVEL_HIGH>; | |
| 2275 | + interrupts = <GIC_ESPI 99 IRQ_TYPE_LEVEL_HIGH>; | |
| 2276 | 2276 | clocks = <&gcc GCC_QUPV3_WRAP0_S7_CLK>; |
| 2277 | 2277 | clock-names = "se"; |
| 2278 | 2278 | interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS |
| @@ -6874,9 +6874,9 @@ · 4595 unmodified lines | ||
| 6874 | 6874 | pcie_smmu: iommu@15480000 { |
| 6875 | 6875 | compatible = "arm,smmu-v3"; |
| 6876 | 6876 | reg = <0x0 0x15480000 0x0 0x20000>; |
| 6877 | − interrupts = <GIC_SPI 964 IRQ_TYPE_LEVEL_HIGH>, | |
| 6878 | − <GIC_SPI 962 IRQ_TYPE_LEVEL_HIGH>, | |
| 6879 | − <GIC_SPI 960 IRQ_TYPE_LEVEL_HIGH>; | |
| 6877 | + interrupts = <GIC_ESPI 4 IRQ_TYPE_LEVEL_HIGH>, | |
| 6878 | + <GIC_ESPI 2 IRQ_TYPE_LEVEL_HIGH>, | |
| 6879 | + <GIC_ESPI 0 IRQ_TYPE_LEVEL_HIGH>; | |
| 6880 | 6880 | interrupt-names = "eventq", "cmdq-sync", "gerror"; |
| 6881 | 6881 | dma-coherent; |
| 6882 | 6882 | #iommu-cells = <1>; |
| @@ -7384,6 +7384,7 @@ · 7383 unmodified lines | ||
| 7384 | 7384 | interrupt-controller; |
| 7385 | 7385 | #redistributor-regions = <1>; |
| 7386 | 7386 | redistributor-stride = <0x0 0x20000>; |
| 7387 | + #address-cells = <0>; | |
| 7387 | 7388 | }; |
| 7388 | 7389 | |
| 7389 | 7390 | watchdog@17c10000 { |
| @@ -47,6 +47,8 @@ · 46 unmodified lines | ||
| 47 | 47 | &gpu { |
| 48 | 48 | compatible = "qcom,adreno-43030c00", "qcom,adreno"; |
| 49 | 49 | |
| 50 | + iommus = <&adreno_smmu 0 0x0>; | |
| 51 | + | |
| 50 | 52 | nvmem-cells = <&gpu_speed_bin>; |
| 51 | 53 | nvmem-cell-names = "speed_bin"; |
| 52 | 54 | |
| @@ -637,7 +637,7 @@ · 636 unmodified lines | ||
| 637 | 637 | compatible = "huawei,gaokun3-ec"; |
| 638 | 638 | reg = <0x38>; |
| 639 | 639 | |
| 640 | − interrupts-extended = <&tlmm 107 IRQ_TYPE_LEVEL_LOW>; | |
| 640 | + interrupts-extended = <&tlmm 103 IRQ_TYPE_LEVEL_LOW>; | |
| 641 | 641 | |
| 642 | 642 | #address-cells = <1>; |
| 643 | 643 | #size-cells = <0>; |
| @@ -5356,7 +5356,7 @@ · 5355 unmodified lines | ||
| 5356 | 5356 | <211 699 1>, |
| 5357 | 5357 | <212 705 1>, |
| 5358 | 5358 | <213 450 1>, |
| 5359 | − <214 643 1>, | |
| 5359 | + <214 643 2>, | |
| 5360 | 5360 | <216 646 5>, |
| 5361 | 5361 | <221 390 5>, |
| 5362 | 5362 | <226 700 3>, |
| @@ -5379,7 +5379,7 @@ · 16 unmodified lines | ||
| 5379 | 5379 | <252 798 1>, |
| 5380 | 5380 | <253 765 1>, |
| 5381 | 5381 | <254 763 1>, |
| 5382 | − <255 454 1>, | |
| 5382 | + <255 454 3>, | |
| 5383 | 5383 | <258 139 1>, |
| 5384 | 5384 | <259 786 2>, |
| 5385 | 5385 | <261 370 2>, |
| @@ -347,6 +347,10 @@ · 346 unmodified lines | ||
| 347 | 347 | status = "okay"; |
| 348 | 348 | }; |
| 349 | 349 | |
| 350 | +&cluster_sleep_0 { | |
| 351 | + arm,psci-suspend-param = <0x41008244>; | |
| 352 | +}; | |
| 353 | + | |
| 350 | 354 | &gcc { |
| 351 | 355 | protected-clocks = <GCC_QSPI_CORE_CLK>, |
| 352 | 356 | <GCC_QSPI_CORE_CLK_SRC>, |
| @@ -7091,8 +7091,8 @@ · 7090 unmodified lines | ||
| 7091 | 7091 | |
| 7092 | 7092 | ranges = <0 0 0x14680000 0x2c000>; |
| 7093 | 7093 | |
| 7094 | − ipa_modem_tables: modem-tables@8000 { | |
| 7095 | − reg = <0x8000 0x2000>; | |
| 7094 | + ipa_modem_tables: modem-tables@3000 { | |
| 7095 | + reg = <0x3000 0x2000>; | |
| 7096 | 7096 | }; |
| 7097 | 7097 | }; |
| 7098 | 7098 | |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | enum hyp_enter_exit_reason { |
| 13 | 13 | HYP_REASON_SMC, |
| 14 | 14 | HYP_REASON_HVC, |
| 15 | + HYP_REASON_SYS, | |
| 15 | 16 | HYP_REASON_PSCI, |
| 16 | 17 | HYP_REASON_HOST_ABORT, |
| 17 | 18 | HYP_REASON_GUEST_EXIT, |
| @@ -7,8 +7,6 @@ · 6 unmodified lines | ||
| 7 | 7 | |
| 8 | 8 | #include <linux/ptdump.h> |
| 9 | 9 | |
| 10 | −DECLARE_STATIC_KEY_FALSE(arm64_ptdump_lock_key); | |
| 11 | − | |
| 12 | 10 | #ifdef CONFIG_PTDUMP |
| 13 | 11 | |
| 14 | 12 | #include <linux/mm_types.h> |
| @@ -1,7 +1,7 @@ | ||
| 1 | 1 | // SPDX-License-Identifier: GPL-2.0-only |
| 2 | 2 | // Copyright 2023 Google LLC |
| 3 | 3 | // Authors: Ard Biesheuvel <ardb@google.com> |
| 4 | −// Peter Collingbourne <pcc@google.com> | |
| 4 | +// Peter Collingbourne <peter@pcc.me.uk> | |
| 5 | 5 | |
| 6 | 6 | #include <linux/elf.h> |
| 7 | 7 | #include <linux/init.h> |
| @@ -149,14 +149,27 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, · 148 unmodified lines | ||
| 149 | 149 | set_bit(KVM_ARCH_FLAG_RETURN_NISV_IO_ABORT_TO_USER, |
| 150 | 150 | &kvm->arch.flags); |
| 151 | 151 | break; |
| 152 | − case KVM_CAP_ARM_MTE: | |
| 153 | − mutex_lock(&kvm->lock); | |
| 154 | − if (system_supports_mte() && !kvm->created_vcpus) { | |
| 155 | − r = 0; | |
| 156 | − set_bit(KVM_ARCH_FLAG_MTE_ENABLED, &kvm->arch.flags); | |
| 152 | + case KVM_CAP_ARM_MTE: { | |
| 153 | + struct kvm_memory_slot *memslot; | |
| 154 | + int bkt; | |
| 155 | + | |
| 156 | + guard(mutex)(&kvm->lock); | |
| 157 | + if (!system_supports_mte() || kvm->created_vcpus) | |
| 158 | + break; | |
| 159 | + | |
| 160 | + r = 0; | |
| 161 | + guard(mutex)(&kvm->slots_lock); | |
| 162 | + kvm_for_each_memslot(memslot, bkt, kvm_memslots(kvm)) { | |
| 163 | + if (kvm_slot_has_gmem(memslot)) { | |
| 164 | + r = -EINVAL; | |
| 165 | + break; | |
| 166 | + } | |
| 157 | 167 | } |
| 158 | − mutex_unlock(&kvm->lock); | |
| 168 | + if (r == 0) | |
| 169 | + set_bit(KVM_ARCH_FLAG_MTE_ENABLED, &kvm->arch.flags); | |
| 159 | 170 | break; |
| 171 | + | |
| 172 | + } | |
| 160 | 173 | case KVM_CAP_ARM_SYSTEM_SUSPEND: |
| 161 | 174 | r = 0; |
| 162 | 175 | set_bit(KVM_ARCH_FLAG_SYSTEM_SUSPEND_ENABLED, &kvm->arch.flags); |
| @@ -932,6 +932,7 @@ void handle_trap(struct kvm_cpu_context *host_ctxt) · 931 unmodified lines | ||
| 932 | 932 | handle_host_mem_abort(host_ctxt); |
| 933 | 933 | break; |
| 934 | 934 | case ESR_ELx_EC_SYS64: |
| 935 | + trace_hyp_enter(host_ctxt, HYP_REASON_SYS); | |
| 935 | 936 | if (handle_host_mte(esr)) |
| 936 | 937 | break; |
| 937 | 938 | fallthrough; |
| @@ -37,8 +37,6 @@ static struct hyp_trace_clock { · 36 unmodified lines | ||
| 37 | 37 | u32 shift; |
| 38 | 38 | struct delayed_work work; |
| 39 | 39 | struct completion ready; |
| 40 | − struct mutex lock; | |
| 41 | − bool running; | |
| 42 | 40 | } hyp_clock; |
| 43 | 41 | |
| 44 | 42 | static void __hyp_clock_work(struct work_struct *work) |
| @@ -110,12 +108,9 @@ static void hyp_trace_clock_enable(struct hyp_trace_clock *hyp_clock, bool enabl · 65 unmodified lines | ||
| 110 | 108 | { |
| 111 | 109 | struct system_time_snapshot snap; |
| 112 | 110 | |
| 113 | − if (hyp_clock->running == enable) | |
| 114 | − return; | |
| 115 | − | |
| 116 | 111 | if (!enable) { |
| 117 | 112 | cancel_delayed_work_sync(&hyp_clock->work); |
| 118 | − hyp_clock->running = false; | |
| 113 | + return; | |
| 119 | 114 | } |
| 120 | 115 | |
| 121 | 116 | ktime_get_snapshot_id(CLOCK_BOOTTIME, &snap); |
| @@ -128,7 +123,6 @@ static void hyp_trace_clock_enable(struct hyp_trace_clock *hyp_clock, bool enabl · 6 unmodified lines | ||
| 128 | 123 | INIT_DELAYED_WORK(&hyp_clock->work, __hyp_clock_work); |
| 129 | 124 | schedule_delayed_work(&hyp_clock->work, msecs_to_jiffies(CLOCK_INIT_MS)); |
| 130 | 125 | wait_for_completion(&hyp_clock->ready); |
| 131 | − hyp_clock->running = true; | |
| 132 | 126 | } |
| 133 | 127 | |
| 134 | 128 | /* Access to this struct within the trace_remote_callbacks are protected by the trace_remote lock */ |
| @@ -160,6 +154,7 @@ static int hyp_trace_buffer_alloc_bpages_backing(struct hyp_trace_buffer *trace_ · 25 unmodified lines | ||
| 160 | 154 | int nr_bpages = (PAGE_ALIGN(size) / PAGE_SIZE) + 1; |
| 161 | 155 | size_t backing_size; |
| 162 | 156 | void *start; |
| 157 | + int ret; | |
| 163 | 158 | |
| 164 | 159 | backing_size = PAGE_ALIGN(sizeof(struct simple_buffer_page) * nr_bpages * |
| 165 | 160 | num_possible_cpus()); |
| @@ -168,10 +163,16 @@ static int hyp_trace_buffer_alloc_bpages_backing(struct hyp_trace_buffer *trace_ · 2 unmodified lines | ||
| 168 | 163 | if (!start) |
| 169 | 164 | return -ENOMEM; |
| 170 | 165 | |
| 166 | + ret = __map_hyp(start, backing_size); | |
| 167 | + if (ret) { | |
| 168 | + free_pages_exact(start, backing_size); | |
| 169 | + return ret; | |
| 170 | + } | |
| 171 | + | |
| 171 | 172 | trace_buffer->desc->bpages_backing_start = (unsigned long)start; |
| 172 | 173 | trace_buffer->desc->bpages_backing_size = backing_size; |
| 173 | 174 | |
| 174 | − return __map_hyp(start, backing_size); | |
| 175 | + return ret; | |
| 175 | 176 | } |
| 176 | 177 | |
| 177 | 178 | static void hyp_trace_buffer_free_bpages_backing(struct hyp_trace_buffer *trace_buffer) |
| @@ -228,18 +229,22 @@ static int hyp_trace_buffer_share_hyp(struct hyp_trace_buffer *trace_buffer) · 50 unmodified lines | ||
| 228 | 229 | static struct trace_buffer_desc *hyp_trace_load(unsigned long size, void *priv) |
| 229 | 230 | { |
| 230 | 231 | struct hyp_trace_buffer *trace_buffer = priv; |
| 232 | + size_t desc_size, tb_desc_size; | |
| 231 | 233 | struct hyp_trace_desc *desc; |
| 232 | − size_t desc_size; | |
| 233 | 234 | int ret; |
| 234 | 235 | |
| 235 | 236 | if (WARN_ON(trace_buffer->desc)) |
| 236 | 237 | return ERR_PTR(-EINVAL); |
| 237 | 238 | |
| 238 | − desc_size = trace_buffer_desc_size(size, num_possible_cpus()); | |
| 239 | + tb_desc_size = trace_buffer_desc_size(size, num_possible_cpus()); | |
| 240 | + desc_size = size_add(tb_desc_size, offsetof(struct hyp_trace_desc, trace_buffer_desc)); | |
| 239 | 241 | if (desc_size == SIZE_MAX) |
| 240 | 242 | return ERR_PTR(-E2BIG); |
| 241 | 243 | |
| 242 | 244 | desc_size = PAGE_ALIGN(desc_size); |
| 245 | + if (!desc_size) | |
| 246 | + return ERR_PTR(-E2BIG); | |
| 247 | + | |
| 243 | 248 | desc = (struct hyp_trace_desc *)alloc_pages_exact(desc_size, GFP_KERNEL); |
| 244 | 249 | if (!desc) |
| 245 | 250 | return ERR_PTR(-ENOMEM); |
| @@ -255,7 +260,7 @@ static struct trace_buffer_desc *hyp_trace_load(unsigned long size, void *priv) · 9 unmodified lines | ||
| 255 | 260 | if (ret) |
| 256 | 261 | goto err_free_desc; |
| 257 | 262 | |
| 258 | − ret = trace_remote_alloc_buffer(&desc->trace_buffer_desc, desc_size, size, | |
| 263 | + ret = trace_remote_alloc_buffer(&desc->trace_buffer_desc, tb_desc_size, size, | |
| 259 | 264 | cpu_possible_mask); |
| 260 | 265 | if (ret) |
| 261 | 266 | goto err_free_backing; |
| @@ -304,9 +309,15 @@ static void hyp_trace_unload(struct trace_buffer_desc *desc, void *priv) · 42 unmodified lines | ||
| 304 | 309 | |
| 305 | 310 | static int hyp_trace_enable_tracing(bool enable, void *priv) |
| 306 | 311 | { |
| 312 | + int ret; | |
| 313 | + | |
| 307 | 314 | hyp_trace_clock_enable(&hyp_clock, enable); |
| 308 | 315 | |
| 309 | − return kvm_call_hyp_nvhe(__tracing_enable, enable); | |
| 316 | + ret = kvm_call_hyp_nvhe(__tracing_enable, enable); | |
| 317 | + if (ret) | |
| 318 | + hyp_trace_clock_enable(&hyp_clock, !enable); | |
| 319 | + | |
| 320 | + return ret; | |
| 310 | 321 | } |
| 311 | 322 | |
| 312 | 323 | static int hyp_trace_swap_reader_page(unsigned int cpu, void *priv) |
| @@ -398,6 +409,7 @@ static const char *__hyp_enter_exit_reason_str(u8 reason) · 85 unmodified lines | ||
| 398 | 409 | static const char strs[][12] = { |
| 399 | 410 | "smc", |
| 400 | 411 | "hvc", |
| 412 | + "sys", | |
| 401 | 413 | "psci", |
| 402 | 414 | "host_abort", |
| 403 | 415 | "guest_exit", |
| @@ -2652,6 +2652,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, · 2651 unmodified lines | ||
| 2652 | 2652 | if (kvm_slot_has_gmem(new) && !kvm_memslot_is_gmem_only(new)) |
| 2653 | 2653 | return -EINVAL; |
| 2654 | 2654 | |
| 2655 | + /* guest_memfd is incompatible with MTE. */ | |
| 2656 | + if (kvm_slot_has_gmem(new) && kvm_has_mte(kvm)) | |
| 2657 | + return -EINVAL; | |
| 2658 | + | |
| 2655 | 2659 | hva = new->userspace_addr; |
| 2656 | 2660 | reg_end = hva + (new->npages << PAGE_SHIFT); |
| 2657 | 2661 | |
| @@ -116,18 +116,27 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, · 115 unmodified lines | ||
| 116 | 116 | kfree(irq); |
| 117 | 117 | irq = oldirq; |
| 118 | 118 | } else { |
| 119 | − ret = xa_err(__xa_store(&dist->lpi_xa, intid, irq, 0)); | |
| 119 | + /* | |
| 120 | + * The entry is either empty or contains a dead LPI (refcount=0) | |
| 121 | + * from the deferred release path, pending cleanup by | |
| 122 | + * vgic_release_deleted_lpis(). Evict and free it if present. | |
| 123 | + */ | |
| 124 | + oldirq = __xa_store(&dist->lpi_xa, intid, irq, | |
| 125 | + GFP_NOWAIT | __GFP_ACCOUNT); | |
| 126 | + ret = xa_err(oldirq); | |
| 127 | + if (ret) { | |
| 128 | + xa_unlock_irqrestore(&dist->lpi_xa, flags); | |
| 129 | + kfree(irq); | |
| 130 | + | |
| 131 | + return ERR_PTR(ret); | |
| 132 | + } | |
| 133 | + | |
| 134 | + if (oldirq && !WARN_ON_ONCE(refcount_read(&oldirq->refcount))) | |
| 135 | + kfree_rcu(oldirq, rcu); | |
| 120 | 136 | } |
| 121 | 137 | |
| 122 | 138 | xa_unlock_irqrestore(&dist->lpi_xa, flags); |
| 123 | 139 | |
| 124 | − if (ret) { | |
| 125 | − xa_release(&dist->lpi_xa, intid); | |
| 126 | − kfree(irq); | |
| 127 | − | |
| 128 | − return ERR_PTR(ret); | |
| 129 | − } | |
| 130 | − | |
| 131 | 140 | /* |
| 132 | 141 | * We "cache" the configuration table entries in our struct vgic_irq's. |
| 133 | 142 | * However we only have those structs for mapped IRQs, so we read in |
| @@ -275,7 +275,13 @@ void vgic_v3_deactivate(struct kvm_vcpu *vcpu, u64 val) · 274 unmodified lines | ||
| 275 | 275 | lr = vgic_v3_compute_lr(vcpu, irq) & ~ICH_LR_ACTIVE_BIT; |
| 276 | 276 | } |
| 277 | 277 | |
| 278 | − if (lr & ICH_LR_HW) | |
| 278 | + /* | |
| 279 | + * In the nested state, the irq has already been deactivated via the HW | |
| 280 | + * bit in the LR. Deactivating again would be harmless except AmpereOne | |
| 281 | + * errata AC03_CPU_57, AC04_CPU_29 could cause irq delivery to break if | |
| 282 | + * the deactivation hits the highest priority pending irq. | |
| 283 | + */ | |
| 284 | + if ((lr & ICH_LR_HW) && !vgic_state_is_nested(vcpu)) | |
| 279 | 285 | vgic_v3_deactivate_phys(FIELD_GET(ICH_LR_PHYS_ID_MASK, lr)); |
| 280 | 286 | |
| 281 | 287 | vgic_v3_fold_lr(vcpu, lr); |
| @@ -146,11 +146,7 @@ static __must_check bool __vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) · 145 unmodified lines | ||
| 146 | 146 | |
| 147 | 147 | static __must_check bool vgic_put_irq_norelease(struct kvm *kvm, struct vgic_irq *irq) |
| 148 | 148 | { |
| 149 | − if (!__vgic_put_irq(kvm, irq)) | |
| 150 | − return false; | |
| 151 | − | |
| 152 | − irq->pending_release = true; | |
| 153 | − return true; | |
| 149 | + return __vgic_put_irq(kvm, irq); | |
| 154 | 150 | } |
| 155 | 151 | |
| 156 | 152 | void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) |
| @@ -167,12 +163,14 @@ void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) · 10 unmodified lines | ||
| 167 | 163 | guard(spinlock_irqsave)(&dist->lpi_xa.xa_lock); |
| 168 | 164 | } |
| 169 | 165 | |
| 170 | − if (!__vgic_put_irq(kvm, irq)) | |
| 166 | + if (!irq_is_lpi(kvm, irq->intid)) | |
| 171 | 167 | return; |
| 172 | 168 | |
| 173 | − xa_lock_irqsave(&dist->lpi_xa, flags); | |
| 174 | − vgic_release_lpi_locked(dist, irq); | |
| 175 | − xa_unlock_irqrestore(&dist->lpi_xa, flags); | |
| 169 | + if (refcount_dec_and_lock_irqsave(&irq->refcount, | |
| 170 | + &dist->lpi_xa.xa_lock, &flags)) { | |
| 171 | + vgic_release_lpi_locked(dist, irq); | |
| 172 | + xa_unlock_irqrestore(&dist->lpi_xa, flags); | |
| 173 | + } | |
| 176 | 174 | } |
| 177 | 175 | |
| 178 | 176 | static void vgic_release_deleted_lpis(struct kvm *kvm) |
| @@ -184,7 +182,7 @@ static void vgic_release_deleted_lpis(struct kvm *kvm) · 5 unmodified lines | ||
| 184 | 182 | xa_lock_irqsave(&dist->lpi_xa, flags); |
| 185 | 183 | |
| 186 | 184 | xa_for_each(&dist->lpi_xa, intid, irq) { |
| 187 | − if (irq->pending_release) | |
| 185 | + if (!refcount_read(&irq->refcount)) | |
| 188 | 186 | vgic_release_lpi_locked(dist, irq); |
| 189 | 187 | } |
| 190 | 188 | |
| @@ -49,8 +49,6 @@ · 48 unmodified lines | ||
| 49 | 49 | #define NO_CONT_MAPPINGS BIT(1) |
| 50 | 50 | #define NO_EXEC_MAPPINGS BIT(2) /* assumes FEAT_HPDS is not used */ |
| 51 | 51 | |
| 52 | −DEFINE_STATIC_KEY_FALSE(arm64_ptdump_lock_key); | |
| 53 | − | |
| 54 | 52 | u64 kimage_voffset __ro_after_init; |
| 55 | 53 | EXPORT_SYMBOL(kimage_voffset); |
| 56 | 54 | |
| @@ -1864,8 +1862,7 @@ int pmd_clear_huge(pmd_t *pmdp) · 1807 unmodified lines | ||
| 1864 | 1862 | return 1; |
| 1865 | 1863 | } |
| 1866 | 1864 | |
| 1867 | −static int __pmd_free_pte_page(pmd_t *pmdp, unsigned long addr, | |
| 1868 | − bool acquire_mmap_lock) | |
| 1865 | +int pmd_free_pte_page(pmd_t *pmdp, unsigned long addr) | |
| 1869 | 1866 | { |
| 1870 | 1867 | pte_t *table; |
| 1871 | 1868 | pmd_t pmd; |
| @@ -1877,25 +1874,13 @@ static int __pmd_free_pte_page(pmd_t *pmdp, unsigned long addr, · 5 unmodified lines | ||
| 1877 | 1874 | return 1; |
| 1878 | 1875 | } |
| 1879 | 1876 | |
| 1880 | − /* See comment in pud_free_pmd_page for static key logic */ | |
| 1881 | 1877 | table = pte_offset_kernel(pmdp, addr); |
| 1882 | 1878 | pmd_clear(pmdp); |
| 1883 | 1879 | __flush_tlb_kernel_pgtable(addr); |
| 1884 | − if (static_branch_unlikely(&arm64_ptdump_lock_key) && acquire_mmap_lock) { | |
| 1885 | − mmap_read_lock(&init_mm); | |
| 1886 | − mmap_read_unlock(&init_mm); | |
| 1887 | − } | |
| 1888 | − | |
| 1889 | 1880 | pte_free_kernel(NULL, table); |
| 1890 | 1881 | return 1; |
| 1891 | 1882 | } |
| 1892 | 1883 | |
| 1893 | −int pmd_free_pte_page(pmd_t *pmdp, unsigned long addr) | |
| 1894 | −{ | |
| 1895 | − /* If ptdump is walking the pagetables, acquire init_mm.mmap_lock */ | |
| 1896 | − return __pmd_free_pte_page(pmdp, addr, /* acquire_mmap_lock = */ true); | |
| 1897 | −} | |
| 1898 | − | |
| 1899 | 1884 | int pud_free_pmd_page(pud_t *pudp, unsigned long addr) |
| 1900 | 1885 | { |
| 1901 | 1886 | pmd_t *table; |
| @@ -1911,36 +1896,16 @@ int pud_free_pmd_page(pud_t *pudp, unsigned long addr) · 9 unmodified lines | ||
| 1911 | 1896 | } |
| 1912 | 1897 | |
| 1913 | 1898 | table = pmd_offset(pudp, addr); |
| 1914 | − | |
| 1915 | − /* | |
| 1916 | − * Our objective is to prevent ptdump from reading a PMD table which has | |
| 1917 | − * been freed. In this race, if pud_free_pmd_page observes the key on | |
| 1918 | − * (which got flipped by ptdump) then the mmap lock sequence here will, | |
| 1919 | − * as a result of the mmap write lock/unlock sequence in ptdump, give | |
| 1920 | − * us the correct synchronization. If not, this means that ptdump has | |
| 1921 | − * yet not started walking the pagetables - the sequence of barriers | |
| 1922 | − * issued by __flush_tlb_kernel_pgtable() guarantees that ptdump will | |
| 1923 | − * observe an empty PUD. | |
| 1924 | − */ | |
| 1925 | − pud_clear(pudp); | |
| 1926 | − __flush_tlb_kernel_pgtable(addr); | |
| 1927 | − if (static_branch_unlikely(&arm64_ptdump_lock_key)) { | |
| 1928 | − mmap_read_lock(&init_mm); | |
| 1929 | − mmap_read_unlock(&init_mm); | |
| 1930 | − } | |
| 1931 | − | |
| 1932 | 1899 | pmdp = table; |
| 1933 | 1900 | next = addr; |
| 1934 | 1901 | end = addr + PUD_SIZE; |
| 1935 | 1902 | do { |
| 1936 | 1903 | if (pmd_present(pmdp_get(pmdp))) |
| 1937 | − /* | |
| 1938 | − * PMD has been isolated, so ptdump won't see it. No | |
| 1939 | − * need to acquire init_mm.mmap_lock. | |
| 1940 | − */ | |
| 1941 | − __pmd_free_pte_page(pmdp, next, /* acquire_mmap_lock = */ false); | |
| 1904 | + pmd_free_pte_page(pmdp, next); | |
| 1942 | 1905 | } while (pmdp++, next += PMD_SIZE, next != end); |
| 1943 | 1906 | |
| 1907 | + pud_clear(pudp); | |
| 1908 | + __flush_tlb_kernel_pgtable(addr); | |
| 1944 | 1909 | pmd_free(NULL, table); |
| 1945 | 1910 | return 1; |
| 1946 | 1911 | } |
| @@ -283,13 +283,6 @@ void note_page_flush(struct ptdump_state *pt_st) · 282 unmodified lines | ||
| 283 | 283 | note_page(pt_st, 0, -1, pte_val(pte_zero)); |
| 284 | 284 | } |
| 285 | 285 | |
| 286 | −static void arm64_ptdump_walk_pgd(struct ptdump_state *st, struct mm_struct *mm) | |
| 287 | −{ | |
| 288 | − static_branch_inc(&arm64_ptdump_lock_key); | |
| 289 | − ptdump_walk_pgd(st, mm, NULL); | |
| 290 | − static_branch_dec(&arm64_ptdump_lock_key); | |
| 291 | −} | |
| 292 | − | |
| 293 | 286 | void ptdump_walk(struct seq_file *s, struct ptdump_info *info) |
| 294 | 287 | { |
| 295 | 288 | unsigned long end = ~0UL; |
| @@ -318,7 +311,7 @@ void ptdump_walk(struct seq_file *s, struct ptdump_info *info) · 22 unmodified lines | ||
| 318 | 311 | } |
| 319 | 312 | }; |
| 320 | 313 | |
| 321 | − arm64_ptdump_walk_pgd(&st.ptdump, info->mm); | |
| 314 | + ptdump_walk_pgd(&st.ptdump, info->mm, NULL); | |
| 322 | 315 | } |
| 323 | 316 | |
| 324 | 317 | static void __init ptdump_initialize(void) |
| @@ -360,7 +353,7 @@ bool ptdump_check_wx(void) · 35 unmodified lines | ||
| 360 | 353 | } |
| 361 | 354 | }; |
| 362 | 355 | |
| 363 | − arm64_ptdump_walk_pgd(&st.ptdump, &init_mm); | |
| 356 | + ptdump_walk_pgd(&st.ptdump, &init_mm, NULL); | |
| 364 | 357 | |
| 365 | 358 | if (st.wx_pages || st.uxn_pages) { |
| 366 | 359 | pr_warn("Checked W+X mappings: FAILED, %lu W+X pages found, %lu non-UXN pages found\n", |
| @@ -39,6 +39,8 @@ · 38 unmodified lines | ||
| 39 | 39 | #include <asm/processor.h> |
| 40 | 40 | |
| 41 | 41 | .section .data |
| 42 | +/* The MMU requires a page aligned page directory. */ | |
| 43 | +.align 12 | |
| 42 | 44 | .global swapper_pg_dir |
| 43 | 45 | swapper_pg_dir: |
| 44 | 46 | .space PAGE_SIZE |
| @@ -68,7 +68,7 @@ void platform_init(unsigned long r3, unsigned long r4, unsigned long r5, · 67 unmodified lines | ||
| 68 | 68 | /* finally, setup the timebase */ |
| 69 | 69 | node = fdt_node_offset_by_prop_value(_dtb_start, -1, "device_type", |
| 70 | 70 | "cpu", sizeof("cpu")); |
| 71 | − if (!node) | |
| 71 | + if (node < 0) | |
| 72 | 72 | fatal("Cannot find cpu node\n"); |
| 73 | 73 | timebase = fdt_getprop(_dtb_start, node, "timebase-frequency", &size); |
| 74 | 74 | if (timebase && (size == 4)) |
| @@ -146,7 +146,7 @@ void platform_init(char *userdata) · 145 unmodified lines | ||
| 146 | 146 | |
| 147 | 147 | node = fdt_node_offset_by_prop_value(_dtb_start, -1, "device_type", |
| 148 | 148 | "cpu", sizeof("cpu")); |
| 149 | − if (!node) | |
| 149 | + if (node < 0) | |
| 150 | 150 | fatal("Cannot find cpu node\n"); |
| 151 | 151 | timebase = fdt_getprop(_dtb_start, node, "timebase-frequency", &size); |
| 152 | 152 | if (timebase && (size == 4)) |
| @@ -102,7 +102,7 @@ void platform_init(void) · 101 unmodified lines | ||
| 102 | 102 | |
| 103 | 103 | node = fdt_node_offset_by_prop_value(_dtb_start, -1, "device_type", |
| 104 | 104 | "cpu", sizeof("cpu")); |
| 105 | − if (!node) | |
| 105 | + if (node < 0) | |
| 106 | 106 | fatal("Cannot find cpu node\n"); |
| 107 | 107 | timebase = fdt_getprop(_dtb_start, node, "timebase-frequency", &size); |
| 108 | 108 | if (timebase && (size == 4)) |
| @@ -66,6 +66,13 @@ static inline void srr_regs_clobbered(void) · 65 unmodified lines | ||
| 66 | 66 | static inline void nap_adjust_return(struct pt_regs *regs) |
| 67 | 67 | { |
| 68 | 68 | #ifdef CONFIG_PPC_970_NAP |
| 69 | + /* | |
| 70 | + * Adjust the nap return address before irq_exit_rcu(). irq_exit_rcu() | |
| 71 | + * may invoke softirqs with interrupts re-enabled, allowing a nested | |
| 72 | + * async interrupt to arrive. If _TLF_NAPPING is still set at that | |
| 73 | + * point, the nested interrupt would erroneously redirect its own | |
| 74 | + * return address to power4_idle_nap_return, corrupting the stack. | |
| 75 | + */ | |
| 69 | 76 | if (unlikely(test_thread_local_flags(_TLF_NAPPING))) { |
| 70 | 77 | /* Can avoid a test-and-clear because NMIs do not call this */ |
| 71 | 78 | clear_thread_local_flags(_TLF_NAPPING); |
| @@ -286,14 +293,6 @@ static inline void arch_interrupt_async_enter_prepare(struct pt_regs *regs) · 214 unmodified lines | ||
| 286 | 293 | |
| 287 | 294 | static inline void arch_interrupt_async_exit_prepare(struct pt_regs *regs) |
| 288 | 295 | { |
| 289 | − /* | |
| 290 | − * Adjust at exit so the main handler sees the true NIA. This must | |
| 291 | − * come before irq_exit() because irq_exit can enable interrupts, and | |
| 292 | − * if another interrupt is taken before nap_adjust_return has run | |
| 293 | − * here, then that interrupt would return directly to idle nap return. | |
| 294 | − */ | |
| 295 | − nap_adjust_return(regs); | |
| 296 | − | |
| 297 | 296 | arch_interrupt_exit_prepare(regs); |
| 298 | 297 | } |
| 299 | 298 | |
| @@ -246,6 +246,7 @@ interrupt_handler void func(struct pt_regs *regs) \ · 245 unmodified lines | ||
| 246 | 246 | instrumentation_begin(); \ |
| 247 | 247 | irq_enter_rcu(); \ |
| 248 | 248 | ____##func (regs); \ |
| 249 | + nap_adjust_return(regs); \ | |
| 249 | 250 | irq_exit_rcu(); \ |
| 250 | 251 | instrumentation_end(); \ |
| 251 | 252 | arch_interrupt_async_exit_prepare(regs); \ |
| @@ -53,9 +53,6 @@ struct pt_regs · 52 unmodified lines | ||
| 53 | 53 | unsigned long esr; |
| 54 | 54 | }; |
| 55 | 55 | unsigned long result; |
| 56 | − unsigned long exit_flags; | |
| 57 | − /* Maintain 16 byte interrupt stack alignment */ | |
| 58 | − unsigned long __pt_regs_pad[3]; | |
| 59 | 56 | }; |
| 60 | 57 | }; |
| 61 | 58 | #if defined(CONFIG_PPC64) || defined(CONFIG_PPC_KUAP) |
| @@ -1,6 +1,4 @@ | ||
| 1 | 1 | /* SPDX-License-Identifier: GPL-2.0-or-later */ |
| 2 | −/* | |
| 3 | − */ | |
| 4 | 2 | #ifndef _ASM_POWERPC_SERIAL_H |
| 5 | 3 | #define _ASM_POWERPC_SERIAL_H |
| 6 | 4 | |
| @@ -18,4 +16,4 @@ extern void find_legacy_serial_ports(void); · 11 unmodified lines | ||
| 18 | 16 | #define find_legacy_serial_ports() do { } while (0) |
| 19 | 17 | #endif |
| 20 | 18 | |
| 21 | −#endif /* _PPC64_SERIAL_H */ | |
| 19 | +#endif /* _ASM_POWERPC_SERIAL_H */ | |
| @@ -57,6 +57,7 @@ struct thread_info { · 56 unmodified lines | ||
| 57 | 57 | #ifdef CONFIG_SMP |
| 58 | 58 | unsigned int cpu; |
| 59 | 59 | #endif |
| 60 | + unsigned long exit_flags; /* Exit Flags for entry/exit */ | |
| 60 | 61 | unsigned long syscall_work; /* SYSCALL_WORK_ flags */ |
| 61 | 62 | unsigned long local_flags; /* private flags for thread */ |
| 62 | 63 | #ifdef CONFIG_LIVEPATCH_64 |
| @@ -55,8 +55,6 @@ struct pt_regs · 54 unmodified lines | ||
| 55 | 55 | unsigned long dar; /* Fault registers */ |
| 56 | 56 | unsigned long dsisr; /* on 4xx/Book-E used for ESR */ |
| 57 | 57 | unsigned long result; /* Result of a system call */ |
| 58 | − unsigned long exit_flags; /* System call exit flags */ | |
| 59 | − unsigned long __pt_regs_pad[3]; /* Maintain 16 byte interrupt stack alignment */ | |
| 60 | 58 | }; |
| 61 | 59 | |
| 62 | 60 | #endif /* __ASSEMBLER__ */ |
| @@ -116,12 +114,10 @@ struct pt_regs · 53 unmodified lines | ||
| 116 | 114 | #define PT_DAR 41 |
| 117 | 115 | #define PT_DSISR 42 |
| 118 | 116 | #define PT_RESULT 43 |
| 119 | −#define PT_EXIT_FLAGS 44 | |
| 120 | −#define PT_PAD 47 /* 3 times */ | |
| 121 | −#define PT_DSCR 48 | |
| 122 | −#define PT_REGS_COUNT 48 | |
| 117 | +#define PT_DSCR 44 | |
| 118 | +#define PT_REGS_COUNT 44 | |
| 123 | 119 | |
| 124 | −#define PT_FPR0 (PT_REGS_COUNT + 4) /* each FP reg occupies 2 slots in this space */ | |
| 120 | +#define PT_FPR0 48 /* each FP reg occupies 2 slots in this space */ | |
| 125 | 121 | |
| 126 | 122 | #ifndef __powerpc64__ |
| 127 | 123 | |
| @@ -133,7 +129,7 @@ struct pt_regs · 5 unmodified lines | ||
| 133 | 129 | #define PT_FPSCR (PT_FPR0 + 32) /* each FP reg occupies 1 slot in 64-bit space */ |
| 134 | 130 | |
| 135 | 131 | |
| 136 | −#define PT_VR0 (PT_FPSCR + 2) /* <82> each Vector reg occupies 2 slots in 64-bit */ | |
| 132 | +#define PT_VR0 82 /* each Vector reg occupies 2 slots in 64-bit */ | |
| 137 | 133 | #define PT_VSCR (PT_VR0 + 32*2 + 1) |
| 138 | 134 | #define PT_VRSAVE (PT_VR0 + 33*2) |
| 139 | 135 | |
| @@ -141,7 +137,7 @@ struct pt_regs · 1 unmodified line | ||
| 141 | 137 | /* |
| 142 | 138 | * Only store first 32 VSRs here. The second 32 VSRs in VR0-31 |
| 143 | 139 | */ |
| 144 | −#define PT_VSR0 (PT_VRSAVE + 2) /* each VSR reg occupies 2 slots in 64-bit */ | |
| 140 | +#define PT_VSR0 150 /* each VSR reg occupies 2 slots in 64-bit */ | |
| 145 | 141 | #define PT_VSR31 (PT_VSR0 + 2*31) |
| 146 | 142 | #endif /* __powerpc64__ */ |
| 147 | 143 | |
| @@ -89,15 +89,17 @@ notrace unsigned long syscall_exit_prepare(unsigned long r3, · 88 unmodified lines | ||
| 89 | 89 | long scv) |
| 90 | 90 | { |
| 91 | 91 | unsigned long ti_flags; |
| 92 | + unsigned long ret = 0; | |
| 92 | 93 | bool is_not_scv = !IS_ENABLED(CONFIG_PPC_BOOK3S_64) || !scv; |
| 93 | 94 | |
| 94 | 95 | kuap_assert_locked(); |
| 95 | 96 | |
| 96 | 97 | regs->result = r3; |
| 97 | − regs->exit_flags = 0; | |
| 98 | 98 | |
| 99 | − ti_flags = read_thread_flags(); | |
| 99 | + /* Clear exit_flags so only flags set during this exit are visible */ | |
| 100 | + current->thread_info.exit_flags = 0; | |
| 100 | 101 | |
| 102 | + ti_flags = read_thread_flags(); | |
| 101 | 103 | if (unlikely(r3 >= (unsigned long)-MAX_ERRNO) && is_not_scv) { |
| 102 | 104 | if (likely(!(ti_flags & (_TIF_NOERROR | _TIF_RESTOREALL)))) { |
| 103 | 105 | r3 = -r3; |
| @@ -107,7 +109,7 @@ notrace unsigned long syscall_exit_prepare(unsigned long r3, · 3 unmodified lines | ||
| 107 | 109 | |
| 108 | 110 | if (unlikely(ti_flags & _TIF_PERSYSCALL_MASK)) { |
| 109 | 111 | if (ti_flags & _TIF_RESTOREALL) |
| 110 | − regs->exit_flags = _TIF_RESTOREALL; | |
| 112 | + ret = _TIF_RESTOREALL; | |
| 111 | 113 | else |
| 112 | 114 | regs->gpr[3] = r3; |
| 113 | 115 | clear_bits(_TIF_PERSYSCALL_MASK, ¤t_thread_info()->flags); |
| @@ -116,7 +118,7 @@ notrace unsigned long syscall_exit_prepare(unsigned long r3, · 2 unmodified lines | ||
| 116 | 118 | } |
| 117 | 119 | |
| 118 | 120 | if (unlikely(ti_flags & _TIF_SYSCALL_DOTRACE)) { |
| 119 | − regs->exit_flags |= _TIF_RESTOREALL; | |
| 121 | + ret |= _TIF_RESTOREALL; | |
| 120 | 122 | } |
| 121 | 123 | |
| 122 | 124 | syscall_exit_to_user_mode(regs); |
| @@ -132,17 +134,19 @@ again: · 9 unmodified lines | ||
| 132 | 134 | |
| 133 | 135 | /* Restore user access locks last */ |
| 134 | 136 | kuap_user_restore(regs); |
| 135 | − | |
| 137 | + ret |= current->thread_info.exit_flags; | |
| 136 | 138 | #ifdef CONFIG_PPC64 |
| 137 | − regs->exit_result = regs->exit_flags; | |
| 139 | + regs->exit_result = ret; | |
| 138 | 140 | #endif |
| 139 | 141 | |
| 140 | − return regs->exit_flags; | |
| 142 | + return ret; | |
| 141 | 143 | } |
| 142 | 144 | |
| 143 | 145 | #ifdef CONFIG_PPC64 |
| 144 | 146 | notrace unsigned long syscall_exit_restart(unsigned long r3, struct pt_regs *regs) |
| 145 | 147 | { |
| 148 | + unsigned long ret; | |
| 149 | + | |
| 146 | 150 | /* |
| 147 | 151 | * This is called when detecting a soft-pending interrupt as well as |
| 148 | 152 | * an alternate-return interrupt. So we can't just have the alternate |
| @@ -167,9 +171,11 @@ again: · 18 unmodified lines | ||
| 167 | 171 | } |
| 168 | 172 | |
| 169 | 173 | kuap_user_restore(regs); |
| 170 | − regs->exit_result |= regs->exit_flags; | |
| 174 | + ret = current_thread_info()->exit_flags & _TIF_RESTOREALL; | |
| 175 | + current_thread_info()->exit_flags &= ~_TIF_RESTOREALL; | |
| 176 | + regs->exit_result |= ret; | |
| 171 | 177 | |
| 172 | − return regs->exit_result; | |
| 178 | + return ret; | |
| 173 | 179 | } |
| 174 | 180 | #endif |
| 175 | 181 | |
| @@ -186,8 +192,10 @@ notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs) · 10 unmodified lines | ||
| 186 | 192 | */ |
| 187 | 193 | kuap_assert_locked(); |
| 188 | 194 | |
| 195 | + /* Clear exit_flags so only flags set during this exit are visible */ | |
| 196 | + current_thread_info()->exit_flags = 0; | |
| 197 | + | |
| 189 | 198 | local_irq_disable(); |
| 190 | − regs->exit_flags = 0; | |
| 191 | 199 | again: |
| 192 | 200 | check_return_regs_valid(regs); |
| 193 | 201 | user_enter_irqoff(); |
| @@ -200,9 +208,7 @@ again: · 6 unmodified lines | ||
| 200 | 208 | |
| 201 | 209 | /* Restore user access locks last */ |
| 202 | 210 | kuap_user_restore(regs); |
| 203 | − | |
| 204 | − ret = regs->exit_flags; | |
| 205 | − | |
| 211 | + ret = current_thread_info()->exit_flags & _TIF_RESTOREALL; | |
| 206 | 212 | #ifdef CONFIG_PPC64 |
| 207 | 213 | regs->exit_result = ret; |
| 208 | 214 | #endif |
| @@ -291,7 +291,6 @@ void __init pt_regs_check(void) · 290 unmodified lines | ||
| 291 | 291 | CHECK_REG(PT_DAR, dar); |
| 292 | 292 | CHECK_REG(PT_DSISR, dsisr); |
| 293 | 293 | CHECK_REG(PT_RESULT, result); |
| 294 | − CHECK_REG(PT_EXIT_FLAGS, exit_flags); | |
| 295 | 294 | #undef CHECK_REG |
| 296 | 295 | |
| 297 | 296 | BUILD_BUG_ON(PT_REGS_COUNT != sizeof(struct user_pt_regs) / sizeof(unsigned long)); |
| @@ -356,6 +356,6 @@ void signal_fault(struct task_struct *tsk, struct pt_regs *regs, · 355 unmodified lines | ||
| 356 | 356 | void arch_do_signal_or_restart(struct pt_regs *regs) |
| 357 | 357 | { |
| 358 | 358 | BUG_ON(regs != current->thread.regs); |
| 359 | − regs->exit_flags |= _TIF_RESTOREALL; | |
| 359 | + current_thread_info()->exit_flags |= _TIF_RESTOREALL; | |
| 360 | 360 | do_signal(current); |
| 361 | 361 | } |
| @@ -1023,10 +1023,7 @@ static int thread_imc_event_init(struct perf_event *event) · 1022 unmodified lines | ||
| 1023 | 1023 | |
| 1024 | 1024 | static bool is_thread_imc_pmu(struct perf_event *event) |
| 1025 | 1025 | { |
| 1026 | − if (!strncmp(event->pmu->name, "thread_imc", strlen("thread_imc"))) | |
| 1027 | − return true; | |
| 1028 | − | |
| 1029 | − return false; | |
| 1026 | + return strstarts(event->pmu->name, "thread_imc"); | |
| 1030 | 1027 | } |
| 1031 | 1028 | |
| 1032 | 1029 | static __be64 *get_event_base_addr(struct perf_event *event) |
| @@ -615,6 +615,7 @@ static int dma_ioc0_map_pages(struct ps3_dma_region *r, unsigned long phys_addr, · 614 unmodified lines | ||
| 615 | 615 | |
| 616 | 616 | fail_map: |
| 617 | 617 | for (iopage--; 0 <= iopage; iopage--) { |
| 618 | + offset = (1 << r->page_size) * iopage; | |
| 618 | 619 | lv1_put_iopte(0, |
| 619 | 620 | c->bus_addr + offset, |
| 620 | 621 | c->lpar_addr + offset, |
| @@ -191,8 +191,6 @@ static void spu_unmap(struct spu *spu) · 190 unmodified lines | ||
| 191 | 191 | |
| 192 | 192 | static int __init setup_areas(struct spu *spu) |
| 193 | 193 | { |
| 194 | − struct table {char* name; unsigned long addr; unsigned long size;}; | |
| 195 | − | |
| 196 | 194 | spu_pdata(spu)->shadow = ioremap_prot(spu_pdata(spu)->shadow_addr, |
| 197 | 195 | sizeof(struct spe_shadow), |
| 198 | 196 | pgprot_noncached_wc(PAGE_KERNEL_RO)); |
| @@ -20,12 +20,15 @@ · 19 unmodified lines | ||
| 20 | 20 | void pseries_kexec_cpu_down(int crash_shutdown, int secondary) |
| 21 | 21 | { |
| 22 | 22 | /* |
| 23 | − * Don't risk a hypervisor call if we're crashing | |
| 24 | − * XXX: Why? The hypervisor is not crashing. It might be better | |
| 25 | − * to at least attempt unregister to avoid the hypervisor stepping | |
| 26 | − * on our memory. | |
| 23 | + * Ensure vpa/slb_shadow/dtl cleanup even while we are crashing. | |
| 24 | + * Why? The hypervisor is not crashing so at least attempt unregister to | |
| 25 | + * avoid the hypervisor stepping on our memory. If hypervisor or kexec | |
| 26 | + * kernel steps on the old memory allocated to these areas before the | |
| 27 | + * new kexec-kernel happens to allocate and register new areas, | |
| 28 | + * the hypervisor will see invalid content which may cause | |
| 29 | + * unexpected behavior. | |
| 27 | 30 | */ |
| 28 | − if (firmware_has_feature(FW_FEATURE_SPLPAR) && !crash_shutdown) { | |
| 31 | + if (firmware_has_feature(FW_FEATURE_SPLPAR)) { | |
| 29 | 32 | int ret; |
| 30 | 33 | int cpu = smp_processor_id(); |
| 31 | 34 | int hwcpu = hard_smp_processor_id(); |
| @@ -699,7 +699,7 @@ static ssize_t lparcfg_write(struct file *file, const char __user * buf, · 698 unmodified lines | ||
| 699 | 699 | if (!firmware_has_feature(FW_FEATURE_SPLPAR)) |
| 700 | 700 | return -EINVAL; |
| 701 | 701 | |
| 702 | − if (count > sizeof(kbuf)) | |
| 702 | + if (count == 0 || count > sizeof(kbuf)) | |
| 703 | 703 | return -EINVAL; |
| 704 | 704 | |
| 705 | 705 | if (copy_from_user(kbuf, buf, count)) |
| @@ -230,10 +230,17 @@ static long papr_phy_attest_create_handle(struct papr_phy_attest_io_block __user · 229 unmodified lines | ||
| 230 | 230 | return -ENOMEM; |
| 231 | 231 | |
| 232 | 232 | if (copy_from_user(¶ms->cmd, ulc, |
| 233 | − sizeof(struct papr_phy_attest_io_block))) | |
| 233 | + sizeof(struct papr_phy_attest_io_block))) { | |
| 234 | + kfree(params); | |
| 234 | 235 | return -EFAULT; |
| 236 | + } | |
| 235 | 237 | |
| 236 | 238 | params->cmd_len = be32_to_cpu(params->cmd.length); |
| 239 | + if (params->cmd_len == 0 || params->cmd_len > sizeof(params->cmd)) { | |
| 240 | + kfree(params); | |
| 241 | + return -EINVAL; | |
| 242 | + } | |
| 243 | + | |
| 237 | 244 | seq = (struct papr_rtas_sequence) { |
| 238 | 245 | .begin = phy_attest_sequence_begin, |
| 239 | 246 | .end = phy_attest_sequence_end, |
| @@ -246,6 +253,9 @@ static long papr_phy_attest_create_handle(struct papr_phy_attest_io_block __user · 6 unmodified lines | ||
| 246 | 253 | &papr_phy_attest_handle_ops, |
| 247 | 254 | "[papr-physical-attestation]"); |
| 248 | 255 | |
| 256 | + if (fd < 0) | |
| 257 | + kfree(params); | |
| 258 | + | |
| 249 | 259 | return fd; |
| 250 | 260 | } |
| 251 | 261 | |
| @@ -132,7 +132,7 @@ static int pseries_pci_sriov_enable(struct pci_dev *pdev, u16 num_vfs) · 131 unmodified lines | ||
| 132 | 132 | |
| 133 | 133 | /* First integer stores max config */ |
| 134 | 134 | max_config_vfs = of_read_number(&max_vfs[0], 1); |
| 135 | − if (max_config_vfs < num_vfs && num_vfs > MAX_VFS_FOR_MAP_PE) { | |
| 135 | + if (max_config_vfs < num_vfs || num_vfs > MAX_VFS_FOR_MAP_PE) { | |
| 136 | 136 | dev_err(&pdev->dev, |
| 137 | 137 | "Num VFs %x > %x Configurable VFs\n", |
| 138 | 138 | num_vfs, (num_vfs > MAX_VFS_FOR_MAP_PE) ? |
| @@ -128,7 +128,12 @@ static void smp_setup_cpu(int cpu) · 127 unmodified lines | ||
| 128 | 128 | else if (cpu != boot_cpuid) |
| 129 | 129 | xics_setup_cpu(); |
| 130 | 130 | |
| 131 | − if (firmware_has_feature(FW_FEATURE_SPLPAR)) | |
| 131 | + /* | |
| 132 | + * Initialize VPA on non-boot cpus since boot-cpu vpa was | |
| 133 | + * already initialized in pSeries_setup_arch() | |
| 134 | + */ | |
| 135 | + if (firmware_has_feature(FW_FEATURE_SPLPAR) && | |
| 136 | + cpu != boot_cpuid) | |
| 132 | 137 | vpa_init(cpu); |
| 133 | 138 | |
| 134 | 139 | cpumask_clear_cpu(cpu, of_spin_mask); |
| @@ -168,7 +168,7 @@ vdso_prepare: prepare0 · 167 unmodified lines | ||
| 168 | 168 | endif |
| 169 | 169 | endif |
| 170 | 170 | |
| 171 | −vdso-install-y += arch/riscv/kernel/vdso/vdso.so.dbg | |
| 171 | +vdso-install-$(CONFIG_MMU) += arch/riscv/kernel/vdso/vdso.so.dbg | |
| 172 | 172 | vdso-install-$(CONFIG_RISCV_USER_CFI) += arch/riscv/kernel/vdso_cfi/vdso-cfi.so.dbg |
| 173 | 173 | vdso-install-$(CONFIG_COMPAT) += arch/riscv/kernel/compat_vdso/compat_vdso.so.dbg |
| 174 | 174 | |
| @@ -93,10 +93,8 @@ void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end, · 92 unmodified lines | ||
| 93 | 93 | for (alt = begin; alt < end; alt++) { |
| 94 | 94 | if (alt->vendor_id != SIFIVE_VENDOR_ID) |
| 95 | 95 | continue; |
| 96 | − if (alt->patch_id >= ERRATA_SIFIVE_NUMBER) { | |
| 97 | − WARN(1, "This errata id:%d is not in kernel errata list", alt->patch_id); | |
| 96 | + if (alt->patch_id >= ERRATA_SIFIVE_NUMBER) | |
| 98 | 97 | continue; |
| 99 | − } | |
| 100 | 98 | |
| 101 | 99 | tmp = (1U << alt->patch_id); |
| 102 | 100 | if (cpu_req_errata & tmp) { |
| @@ -22,13 +22,13 @@ static inline cycles_t get_cycles(void) · 21 unmodified lines | ||
| 22 | 22 | #else /* !CONFIG_64BIT */ |
| 23 | 23 | static inline u32 get_cycles(void) |
| 24 | 24 | { |
| 25 | − return readl_relaxed(((u32 *)clint_time_val)); | |
| 25 | + return readl_relaxed(((u32 __iomem *)clint_time_val)); | |
| 26 | 26 | } |
| 27 | 27 | #define get_cycles get_cycles |
| 28 | 28 | |
| 29 | 29 | static inline u32 get_cycles_hi(void) |
| 30 | 30 | { |
| 31 | − return readl_relaxed(((u32 *)clint_time_val) + 1); | |
| 31 | + return readl_relaxed(((u32 __iomem *)clint_time_val) + 1); | |
| 32 | 32 | } |
| 33 | 33 | #define get_cycles_hi get_cycles_hi |
| 34 | 34 | #endif /* CONFIG_64BIT */ |
| @@ -289,7 +289,7 @@ free: · 288 unmodified lines | ||
| 289 | 289 | } |
| 290 | 290 | |
| 291 | 291 | /* Measure unaligned access speed on all CPUs present at boot in parallel. */ |
| 292 | −static int __init vec_check_unaligned_access_speed_all_cpus(void *unused __always_unused) | |
| 292 | +static int vec_check_unaligned_access_speed_all_cpus(void *unused __always_unused) | |
| 293 | 293 | { |
| 294 | 294 | schedule_on_each_cpu(check_vector_unaligned_access); |
| 295 | 295 | riscv_hwprobe_complete_async_probe(); |
| @@ -297,7 +297,7 @@ static int __init vec_check_unaligned_access_speed_all_cpus(void *unused __alway · 1 unmodified line | ||
| 297 | 297 | return 0; |
| 298 | 298 | } |
| 299 | 299 | #else /* CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS */ |
| 300 | −static int __init vec_check_unaligned_access_speed_all_cpus(void *unused __always_unused) | |
| 300 | +static int vec_check_unaligned_access_speed_all_cpus(void *unused __always_unused) | |
| 301 | 301 | { |
| 302 | 302 | return 0; |
| 303 | 303 | } |
| @@ -63,7 +63,8 @@ EXPORT_SYMBOL(phys_ram_base); · 62 unmodified lines | ||
| 63 | 63 | |
| 64 | 64 | #ifdef CONFIG_SPARSEMEM_VMEMMAP |
| 65 | 65 | #define VMEMMAP_ADDR_ALIGN max(1ULL << SECTION_SIZE_BITS, \ |
| 66 | − MAX_FOLIO_VMEMMAP_ALIGN) | |
| 66 | + PFN_PHYS(MAX_FOLIO_VMEMMAP_ALIGN / \ | |
| 67 | + sizeof(struct page))) | |
| 67 | 68 | |
| 68 | 69 | unsigned long vmemmap_start_pfn __ro_after_init; |
| 69 | 70 | EXPORT_SYMBOL(vmemmap_start_pfn); |
| @@ -164,7 +165,9 @@ static void print_vm_layout(void) { } · 94 unmodified lines | ||
| 164 | 165 | |
| 165 | 166 | void __init arch_mm_preinit(void) |
| 166 | 167 | { |
| 167 | − bool swiotlb = max_pfn > PFN_DOWN(dma32_phys_limit); | |
| 168 | + bool swiotlb = max_pfn > PFN_DOWN(dma32_phys_limit) && | |
| 169 | + memblock_start_of_DRAM() < dma32_phys_limit; | |
| 170 | + unsigned int swiotlb_flags = SWIOTLB_VERBOSE; | |
| 168 | 171 | #ifdef CONFIG_FLATMEM |
| 169 | 172 | BUG_ON(!mem_map); |
| 170 | 173 | #endif /* CONFIG_FLATMEM */ |
| @@ -172,17 +175,22 @@ void __init arch_mm_preinit(void) · 1 unmodified line | ||
| 172 | 175 | if (IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC) && !swiotlb && |
| 173 | 176 | dma_cache_alignment != 1) { |
| 174 | 177 | /* |
| 175 | − * If no bouncing needed for ZONE_DMA, allocate 1MB swiotlb | |
| 176 | − * buffer per 1GB of RAM for kmalloc() bouncing on | |
| 177 | − * non-coherent platforms. | |
| 178 | + * No 32-bit DMA bouncing needed (either all DRAM is within | |
| 179 | + * the 32-bit limit, or it all starts above it), but | |
| 180 | + * kmalloc() buffers whose sizes are not cache-line-aligned | |
| 181 | + * still require bouncing for non-coherent DMA. Use | |
| 182 | + * SWIOTLB_ANY so that the buffer can be allocated from high | |
| 183 | + * memory when DRAM starts above dma32_phys_limit. Allocate | |
| 184 | + * ~1 MB per 1 GB of RAM. | |
| 178 | 185 | */ |
| 179 | 186 | unsigned long size = |
| 180 | 187 | DIV_ROUND_UP(memblock_phys_mem_size(), 1024); |
| 181 | 188 | swiotlb_adjust_size(min(swiotlb_size_or_default(), size)); |
| 182 | 189 | swiotlb = true; |
| 190 | + swiotlb_flags |= SWIOTLB_ANY; | |
| 183 | 191 | } |
| 184 | 192 | |
| 185 | − swiotlb_init(swiotlb, SWIOTLB_VERBOSE); | |
| 193 | + swiotlb_init(swiotlb, swiotlb_flags); | |
| 186 | 194 | |
| 187 | 195 | print_vm_layout(); |
| 188 | 196 | } |
| @@ -1618,7 +1626,7 @@ static void __meminit remove_pud_mapping(pud_t *pud_base, unsigned long addr, un · 1429 unmodified lines | ||
| 1618 | 1626 | |
| 1619 | 1627 | for (; addr < end; addr = next) { |
| 1620 | 1628 | next = pud_addr_end(addr, end); |
| 1621 | − pudp = pud_base + pud_index(addr); | |
| 1629 | + pudp = pgtable_l4_enabled ? pud_base + pud_index(addr) : pud_base; | |
| 1622 | 1630 | pud = pudp_get(pudp); |
| 1623 | 1631 | if (!pud_present(pud)) |
| 1624 | 1632 | continue; |
| @@ -1649,7 +1657,7 @@ static void __meminit remove_p4d_mapping(p4d_t *p4d_base, unsigned long addr, un · 24 unmodified lines | ||
| 1649 | 1657 | |
| 1650 | 1658 | for (; addr < end; addr = next) { |
| 1651 | 1659 | next = p4d_addr_end(addr, end); |
| 1652 | − p4dp = p4d_base + p4d_index(addr); | |
| 1660 | + p4dp = pgtable_l5_enabled ? p4d_base + p4d_index(addr) : p4d_base; | |
| 1653 | 1661 | p4d = p4dp_get(p4dp); |
| 1654 | 1662 | if (!p4d_present(p4d)) |
| 1655 | 1663 | continue; |
| @@ -440,6 +440,7 @@ struct kvm_vcpu_arch { · 439 unmodified lines | ||
| 440 | 440 | bool skey_enabled; |
| 441 | 441 | /* Indicator if the access registers have been loaded from guest */ |
| 442 | 442 | bool acrs_loaded; |
| 443 | + bool initialized; | |
| 443 | 444 | struct kvm_s390_pv_vcpu pv; |
| 444 | 445 | union diag318_info diag318_info; |
| 445 | 446 | struct kvm_s390_mmu_cache *mc; |
| @@ -476,6 +477,11 @@ struct s390_map_info { · 30 unmodified lines | ||
| 476 | 477 | __u64 guest_addr; |
| 477 | 478 | __u64 addr; |
| 478 | 479 | struct page *page; |
| 480 | + /* | |
| 481 | + * True if the page is long-term pinned. False if long-term pinning | |
| 482 | + * failed and this entry exists only to preserve MAP/UNMAP symmetry. | |
| 483 | + */ | |
| 484 | + bool pinned; | |
| 479 | 485 | }; |
| 480 | 486 | |
| 481 | 487 | struct s390_io_adapter { |
| @@ -755,13 +755,15 @@ int dat_cond_set_storage_key(struct kvm_s390_mmu_cache *mmc, union asce asce, gf · 754 unmodified lines | ||
| 755 | 755 | return rc; |
| 756 | 756 | } |
| 757 | 757 | |
| 758 | −int dat_reset_reference_bit(union asce asce, gfn_t gfn) | |
| 758 | +int dat_reset_reference_bit(union asce asce, gfn_t gfn, union skey *skey) | |
| 759 | 759 | { |
| 760 | 760 | union pgste pgste, old; |
| 761 | 761 | union crste *crstep; |
| 762 | 762 | union pte *ptep; |
| 763 | 763 | int rc; |
| 764 | 764 | |
| 765 | + skey->skey = 0; | |
| 766 | + | |
| 765 | 767 | rc = dat_entry_walk(NULL, gfn, asce, DAT_WALK_ANY, TABLE_TYPE_PAGE_TABLE, &crstep, &ptep); |
| 766 | 768 | if (rc) |
| 767 | 769 | return rc; |
| @@ -771,21 +773,23 @@ int dat_reset_reference_bit(union asce asce, gfn_t gfn) · 3 unmodified lines | ||
| 771 | 773 | |
| 772 | 774 | if (!crste.h.fc || !crste.s.fc1.pr) |
| 773 | 775 | return 0; |
| 774 | − return page_reset_referenced(large_crste_to_phys(*crstep, gfn)); | |
| 776 | + skey->skey = page_reset_referenced(large_crste_to_phys(*crstep, gfn)) << 1; | |
| 777 | + return 0; | |
| 775 | 778 | } |
| 776 | 779 | old = pgste_get_lock(ptep); |
| 777 | 780 | pgste = old; |
| 778 | 781 | |
| 779 | 782 | if (!ptep->h.i) { |
| 780 | − rc = page_reset_referenced(pte_origin(*ptep)); | |
| 781 | − pgste.hr = rc >> 1; | |
| 783 | + skey->skey = page_reset_referenced(pte_origin(*ptep)) << 1; | |
| 784 | + pgste.hr = skey->r; | |
| 782 | 785 | } |
| 783 | − rc |= (pgste.gr << 1) | pgste.gc; | |
| 786 | + skey->r |= pgste.gr; | |
| 787 | + skey->c |= pgste.gc; | |
| 784 | 788 | pgste.gr = 0; |
| 785 | 789 | |
| 786 | 790 | dat_update_ptep_sd(old, pgste, ptep); |
| 787 | 791 | pgste_set_unlock(ptep, pgste); |
| 788 | − return rc; | |
| 792 | + return 0; | |
| 789 | 793 | } |
| 790 | 794 | |
| 791 | 795 | static long dat_reset_skeys_pte(union pte *ptep, gfn_t gfn, gfn_t next, struct dat_walk *walk) |
| @@ -846,6 +850,7 @@ static long _dat_slot_pte(union pte *ptep, gfn_t gfn, gfn_t next, struct dat_wal · 54 unmodified lines | ||
| 846 | 850 | struct slot_priv *p = walk->priv; |
| 847 | 851 | union crste dummy = { .val = p->token }; |
| 848 | 852 | union pte new_pte, pte = READ_ONCE(*ptep); |
| 853 | + union pgste pgste; | |
| 849 | 854 | |
| 850 | 855 | new_pte = _PTE_TOK(dummy.tok.type, dummy.tok.par); |
| 851 | 856 | |
| @@ -853,7 +858,11 @@ static long _dat_slot_pte(union pte *ptep, gfn_t gfn, gfn_t next, struct dat_wal · 1 unmodified line | ||
| 853 | 858 | if (pte.val == new_pte.val) |
| 854 | 859 | return 0; |
| 855 | 860 | |
| 856 | − dat_ptep_xchg(ptep, new_pte, gfn, walk->asce, false); | |
| 861 | + pgste = pgste_get_lock(ptep); | |
| 862 | + pgste = __dat_ptep_xchg(ptep, pgste, new_pte, gfn, walk->asce, false); | |
| 863 | + pgste.cmma_d = 0; | |
| 864 | + pgste_set_unlock(ptep, pgste); | |
| 865 | + | |
| 857 | 866 | return 0; |
| 858 | 867 | } |
| 859 | 868 | |
| @@ -537,7 +537,7 @@ int dat_set_storage_key(struct kvm_s390_mmu_cache *mc, union asce asce, gfn_t gf · 536 unmodified lines | ||
| 537 | 537 | union skey skey, bool nq); |
| 538 | 538 | int dat_cond_set_storage_key(struct kvm_s390_mmu_cache *mmc, union asce asce, gfn_t gfn, |
| 539 | 539 | union skey skey, union skey *oldkey, bool nq, bool mr, bool mc); |
| 540 | −int dat_reset_reference_bit(union asce asce, gfn_t gfn); | |
| 540 | +int dat_reset_reference_bit(union asce asce, gfn_t gfn, union skey *skey); | |
| 541 | 541 | long dat_reset_skeys(union asce asce, gfn_t start); |
| 542 | 542 | |
| 543 | 543 | unsigned long dat_get_ptval(struct page_table *table, struct ptval_param param); |
| @@ -91,9 +91,9 @@ int kvm_s390_faultin_gfn(struct kvm_vcpu *vcpu, struct kvm *kvm, struct guest_fa · 90 unmodified lines | ||
| 91 | 91 | /* Access outside memory, addressing exception. */ |
| 92 | 92 | if (is_noslot_pfn(f->pfn)) |
| 93 | 93 | return PGM_ADDRESSING; |
| 94 | − /* Signal pending: try again. */ | |
| 95 | − if (f->pfn == KVM_PFN_ERR_SIGPENDING) | |
| 96 | − return -EAGAIN; | |
| 94 | + /* Fatal signal pending: bail out. */ | |
| 95 | + if (is_sigpending_pfn(f->pfn)) | |
| 96 | + return -EINTR; | |
| 97 | 97 | /* Check if it's read-only memory; don't try to actually handle that case. */ |
| 98 | 98 | if (f->pfn == KVM_PFN_ERR_RO_FAULT) |
| 99 | 99 | return -EOPNOTSUPP; |
| @@ -45,13 +45,16 @@ static struct kvm_s390_gib *gib; · 44 unmodified lines | ||
| 45 | 45 | static int sca_ext_call_pending(struct kvm_vcpu *vcpu, int *src_id) |
| 46 | 46 | { |
| 47 | 47 | struct esca_block *sca = vcpu->kvm->arch.sca; |
| 48 | − union esca_sigp_ctrl sigp_ctrl = sca->cpu[vcpu->vcpu_id].sigp_ctrl; | |
| 48 | + union esca_sigp_ctrl sigp_ctrl; | |
| 49 | 49 | |
| 50 | 50 | if (!kvm_s390_test_cpuflags(vcpu, CPUSTAT_ECALL_PEND)) |
| 51 | 51 | return 0; |
| 52 | + if (kvm_is_ucontrol(vcpu->kvm)) | |
| 53 | + return 0; | |
| 52 | 54 | |
| 53 | 55 | BUG_ON(!kvm_s390_use_sca_entries()); |
| 54 | 56 | |
| 57 | + sigp_ctrl = sca->cpu[vcpu->vcpu_id].sigp_ctrl; | |
| 55 | 58 | if (src_id) |
| 56 | 59 | *src_id = sigp_ctrl.scn; |
| 57 | 60 | |
| @@ -60,13 +63,16 @@ static int sca_ext_call_pending(struct kvm_vcpu *vcpu, int *src_id) · 2 unmodified lines | ||
| 60 | 63 | |
| 61 | 64 | static int sca_inject_ext_call(struct kvm_vcpu *vcpu, int src_id) |
| 62 | 65 | { |
| 63 | − struct esca_block *sca = vcpu->kvm->arch.sca; | |
| 64 | − union esca_sigp_ctrl *sigp_ctrl = &sca->cpu[vcpu->vcpu_id].sigp_ctrl; | |
| 65 | 66 | union esca_sigp_ctrl old_val, new_val = {.scn = src_id, .c = 1}; |
| 67 | + struct esca_block *sca = vcpu->kvm->arch.sca; | |
| 68 | + union esca_sigp_ctrl *sigp_ctrl; | |
| 66 | 69 | int expect, rc; |
| 67 | 70 | |
| 68 | 71 | BUG_ON(!kvm_s390_use_sca_entries()); |
| 72 | + if (kvm_is_ucontrol(vcpu->kvm)) | |
| 73 | + return -EINVAL; | |
| 69 | 74 | |
| 75 | + sigp_ctrl = &sca->cpu[vcpu->vcpu_id].sigp_ctrl; | |
| 70 | 76 | old_val = READ_ONCE(*sigp_ctrl); |
| 71 | 77 | old_val.c = 0; |
| 72 | 78 | |
| @@ -84,10 +90,13 @@ static int sca_inject_ext_call(struct kvm_vcpu *vcpu, int src_id) · 11 unmodified lines | ||
| 84 | 90 | static void sca_clear_ext_call(struct kvm_vcpu *vcpu) |
| 85 | 91 | { |
| 86 | 92 | struct esca_block *sca = vcpu->kvm->arch.sca; |
| 87 | − union esca_sigp_ctrl *sigp_ctrl = &sca->cpu[vcpu->vcpu_id].sigp_ctrl; | |
| 93 | + union esca_sigp_ctrl *sigp_ctrl; | |
| 88 | 94 | |
| 89 | − if (!kvm_s390_use_sca_entries()) | |
| 95 | + if (!kvm_s390_use_sca_entries() || !vcpu->arch.initialized || kvm_is_ucontrol(vcpu->kvm)) | |
| 90 | 96 | return; |
| 97 | + | |
| 98 | + /* Initialize after the above check, to prevent going out of bounds */ | |
| 99 | + sigp_ctrl = &sca->cpu[vcpu->vcpu_id].sigp_ctrl; | |
| 91 | 100 | kvm_s390_clear_cpuflags(vcpu, CPUSTAT_ECALL_PEND); |
| 92 | 101 | |
| 93 | 102 | WRITE_ONCE(sigp_ctrl->value, 0); |
| @@ -2520,8 +2529,22 @@ static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 addr) · 2426 unmodified lines | ||
| 2520 | 2529 | map->addr = host_addr; |
| 2521 | 2530 | map->page = pin_map_page(kvm, host_addr, FOLL_LONGTERM); |
| 2522 | 2531 | if (!map->page) { |
| 2523 | − ret = -EINVAL; | |
| 2524 | − goto out; | |
| 2532 | + /* | |
| 2533 | + * Long-term pinning may fail for memory types such as file-backed | |
| 2534 | + * memory. Verify that short-term pinning succeeds so that the | |
| 2535 | + * non-atomic irqfd path can handle interrupt injection. | |
| 2536 | + */ | |
| 2537 | + map->page = pin_map_page(kvm, host_addr, 0); | |
| 2538 | + if (!map->page) { | |
| 2539 | + ret = -EINVAL; | |
| 2540 | + goto out; | |
| 2541 | + } | |
| 2542 | + unpin_user_page(map->page); | |
| 2543 | + map->page = NULL; | |
| 2544 | + map->pinned = false; | |
| 2545 | + /* Add an entry to preserve MAP/UNMAP symmetry. */ | |
| 2546 | + } else { | |
| 2547 | + map->pinned = true; | |
| 2525 | 2548 | } |
| 2526 | 2549 | spin_lock_irqsave(&adapter->maps_lock, flags); |
| 2527 | 2550 | if (adapter->nr_maps < MAX_S390_ADAPTER_MAPS) { |
| @@ -2532,7 +2555,7 @@ static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 addr) · 4 unmodified lines | ||
| 2532 | 2555 | ret = -EINVAL; |
| 2533 | 2556 | } |
| 2534 | 2557 | spin_unlock_irqrestore(&adapter->maps_lock, flags); |
| 2535 | − if (ret) | |
| 2558 | + if (ret && map->page) | |
| 2536 | 2559 | unpin_user_page(map->page); |
| 2537 | 2560 | out: |
| 2538 | 2561 | if (ret) |
| @@ -2546,6 +2569,7 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr) · 7 unmodified lines | ||
| 2546 | 2569 | struct s390_map_info *map, *tmp, *map_to_free; |
| 2547 | 2570 | struct page *map_page_to_put = NULL; |
| 2548 | 2571 | u64 map_addr_to_mark = 0; |
| 2572 | + bool map_pinned = false; | |
| 2549 | 2573 | unsigned long flags; |
| 2550 | 2574 | int found = 0, idx; |
| 2551 | 2575 | |
| @@ -2560,6 +2584,7 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr) · 8 unmodified lines | ||
| 2560 | 2584 | list_del(&map->list); |
| 2561 | 2585 | map_page_to_put = map->page; |
| 2562 | 2586 | map_addr_to_mark = map->guest_addr; |
| 2587 | + map_pinned = map->pinned; | |
| 2563 | 2588 | map_to_free = map; |
| 2564 | 2589 | break; |
| 2565 | 2590 | } |
| @@ -2568,11 +2593,18 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr) · 2 unmodified lines | ||
| 2568 | 2593 | |
| 2569 | 2594 | if (found) { |
| 2570 | 2595 | kfree(map_to_free); |
| 2571 | − idx = srcu_read_lock(&kvm->srcu); | |
| 2572 | − mark_page_dirty(kvm, map_addr_to_mark >> PAGE_SHIFT); | |
| 2573 | − set_page_dirty_lock(map_page_to_put); | |
| 2574 | − srcu_read_unlock(&kvm->srcu, idx); | |
| 2575 | − unpin_user_page(map_page_to_put); | |
| 2596 | + if (map_pinned) { | |
| 2597 | + /* | |
| 2598 | + * Only long-term pinned pages need to be marked dirty | |
| 2599 | + * and released. Fallback entries exist only for | |
| 2600 | + * MAP/UNMAP symmetry. | |
| 2601 | + */ | |
| 2602 | + idx = srcu_read_lock(&kvm->srcu); | |
| 2603 | + mark_page_dirty(kvm, map_addr_to_mark >> PAGE_SHIFT); | |
| 2604 | + set_page_dirty_lock(map_page_to_put); | |
| 2605 | + srcu_read_unlock(&kvm->srcu, idx); | |
| 2606 | + unpin_user_page(map_page_to_put); | |
| 2607 | + } | |
| 2576 | 2608 | } |
| 2577 | 2609 | |
| 2578 | 2610 | return found ? 0 : -ENOENT; |
| @@ -2598,11 +2630,13 @@ void kvm_s390_unmap_all_adapters(struct kvm *kvm) · 19 unmodified lines | ||
| 2598 | 2630 | |
| 2599 | 2631 | list_for_each_entry_safe(map, tmp, &local_list, list) { |
| 2600 | 2632 | list_del(&map->list); |
| 2601 | − idx = srcu_read_lock(&kvm->srcu); | |
| 2602 | − mark_page_dirty(kvm, map->guest_addr >> PAGE_SHIFT); | |
| 2603 | − set_page_dirty_lock(map->page); | |
| 2604 | − srcu_read_unlock(&kvm->srcu, idx); | |
| 2605 | − unpin_user_page(map->page); | |
| 2633 | + if (map->pinned) { | |
| 2634 | + idx = srcu_read_lock(&kvm->srcu); | |
| 2635 | + mark_page_dirty(kvm, map->guest_addr >> PAGE_SHIFT); | |
| 2636 | + set_page_dirty_lock(map->page); | |
| 2637 | + srcu_read_unlock(&kvm->srcu, idx); | |
| 2638 | + unpin_user_page(map->page); | |
| 2639 | + } | |
| 2606 | 2640 | kfree(map); |
| 2607 | 2641 | } |
| 2608 | 2642 | } |
| @@ -2929,8 +2963,11 @@ static struct s390_map_info *get_map_info(struct s390_io_adapter *adapter, · 320 unmodified lines | ||
| 2929 | 2963 | return NULL; |
| 2930 | 2964 | |
| 2931 | 2965 | list_for_each_entry(map, &adapter->maps, list) { |
| 2932 | − if (map->addr == addr) | |
| 2966 | + if (map->addr == addr) { | |
| 2967 | + if (!map->pinned) | |
| 2968 | + return NULL; | |
| 2933 | 2969 | return map; |
| 2970 | + } | |
| 2934 | 2971 | } |
| 2935 | 2972 | return NULL; |
| 2936 | 2973 | } |
| @@ -571,7 +571,7 @@ static int kvm_s390_keyop(struct kvm_s390_mmu_cache *mc, struct kvm *kvm, int op · 570 unmodified lines | ||
| 571 | 571 | switch (op) { |
| 572 | 572 | case KVM_S390_KEYOP_SSKE: |
| 573 | 573 | r = dat_cond_set_storage_key(mc, asce, gfn, skey, &skey, 0, 0, 0); |
| 574 | − if (r >= 0) | |
| 574 | + if (r == 0 || r == 1) | |
| 575 | 575 | return skey.skey; |
| 576 | 576 | break; |
| 577 | 577 | case KVM_S390_KEYOP_ISKE: |
| @@ -580,14 +580,14 @@ static int kvm_s390_keyop(struct kvm_s390_mmu_cache *mc, struct kvm *kvm, int op · 2 unmodified lines | ||
| 580 | 580 | return skey.skey; |
| 581 | 581 | break; |
| 582 | 582 | case KVM_S390_KEYOP_RRBE: |
| 583 | − r = dat_reset_reference_bit(asce, gfn); | |
| 584 | − if (r > 0) | |
| 585 | − return r << 1; | |
| 583 | + r = dat_reset_reference_bit(asce, gfn, &skey); | |
| 584 | + if (!r) | |
| 585 | + return skey.skey; | |
| 586 | 586 | break; |
| 587 | 587 | default: |
| 588 | 588 | return -EINVAL; |
| 589 | 589 | } |
| 590 | − return r; | |
| 590 | + return r > 0 ? -EFAULT : r; | |
| 591 | 591 | } |
| 592 | 592 | |
| 593 | 593 | /* Section: device related */ |
| @@ -1219,8 +1219,8 @@ static void kvm_s390_sync_request_broadcast(struct kvm *kvm, int req) · 625 unmodified lines | ||
| 1219 | 1219 | |
| 1220 | 1220 | /* |
| 1221 | 1221 | * Must be called with kvm->srcu held to avoid races on memslots, and with |
| 1222 | − * kvm->slots_lock to avoid races with ourselves, kvm_s390_vm_stop_migration(), | |
| 1223 | − * and kvm_s390_get_cmma_bits(). | |
| 1222 | + * kvm->slots_arch_lock to avoid races with ourselves, | |
| 1223 | + * kvm_s390_vm_stop_migration(), and kvm_s390_get_cmma_bits(). | |
| 1224 | 1224 | */ |
| 1225 | 1225 | static int kvm_s390_vm_start_migration(struct kvm *kvm) |
| 1226 | 1226 | { |
| @@ -1265,7 +1265,7 @@ static int kvm_s390_vm_start_migration(struct kvm *kvm) · 38 unmodified lines | ||
| 1265 | 1265 | } |
| 1266 | 1266 | |
| 1267 | 1267 | /* |
| 1268 | − * Must be called with kvm->slots_lock to avoid races with ourselves, | |
| 1268 | + * Must be called with kvm->slots_arch_lock to avoid races with ourselves, | |
| 1269 | 1269 | * kvm_s390_vm_start_migration() and kvm_s390_get_cmma_bits(). |
| 1270 | 1270 | */ |
| 1271 | 1271 | static int kvm_s390_vm_stop_migration(struct kvm *kvm) |
| @@ -1300,7 +1300,9 @@ static int kvm_s390_vm_set_migration(struct kvm *kvm, · 28 unmodified lines | ||
| 1300 | 1300 | { |
| 1301 | 1301 | int res = -ENXIO; |
| 1302 | 1302 | |
| 1303 | − mutex_lock(&kvm->slots_lock); | |
| 1303 | + guard(srcu)(&kvm->srcu); | |
| 1304 | + guard(mutex)(&kvm->slots_arch_lock); | |
| 1305 | + | |
| 1304 | 1306 | switch (attr->attr) { |
| 1305 | 1307 | case KVM_S390_VM_MIGRATION_START: |
| 1306 | 1308 | res = kvm_s390_vm_start_migration(kvm); |
| @@ -1311,7 +1313,6 @@ static int kvm_s390_vm_set_migration(struct kvm *kvm, · 4 unmodified lines | ||
| 1311 | 1313 | default: |
| 1312 | 1314 | break; |
| 1313 | 1315 | } |
| 1314 | − mutex_unlock(&kvm->slots_lock); | |
| 1315 | 1316 | |
| 1316 | 1317 | return res; |
| 1317 | 1318 | } |
| @@ -2214,7 +2215,7 @@ static int kvm_s390_get_skeys(struct kvm *kvm, struct kvm_s390_skeys *args) · 896 unmodified lines | ||
| 2214 | 2215 | } |
| 2215 | 2216 | |
| 2216 | 2217 | kvfree(keys); |
| 2217 | − return r; | |
| 2218 | + return r <= 0 ? r : -EFAULT; | |
| 2218 | 2219 | } |
| 2219 | 2220 | |
| 2220 | 2221 | static int kvm_s390_set_skeys(struct kvm *kvm, struct kvm_s390_skeys *args) |
| @@ -2276,7 +2277,7 @@ static int kvm_s390_set_skeys(struct kvm *kvm, struct kvm_s390_skeys *args) · 55 unmodified lines | ||
| 2276 | 2277 | kvm_s390_free_mmu_cache(mc); |
| 2277 | 2278 | out: |
| 2278 | 2279 | kvfree(keys); |
| 2279 | − return r; | |
| 2280 | + return r <= 0 ? r : -EFAULT; | |
| 2280 | 2281 | } |
| 2281 | 2282 | |
| 2282 | 2283 | /* |
| @@ -2386,7 +2387,7 @@ static int kvm_s390_set_cmma_bits(struct kvm *kvm, · 103 unmodified lines | ||
| 2386 | 2387 | |
| 2387 | 2388 | set_bit(GMAP_FLAG_USES_CMM, &kvm->arch.gmap->flags); |
| 2388 | 2389 | |
| 2389 | − return r; | |
| 2390 | + return r <= 0 ? r : -EFAULT; | |
| 2390 | 2391 | } |
| 2391 | 2392 | |
| 2392 | 2393 | /** |
| @@ -2934,6 +2935,9 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · 541 unmodified lines | ||
| 2934 | 2935 | case KVM_S390_INTERRUPT: { |
| 2935 | 2936 | struct kvm_s390_interrupt s390int; |
| 2936 | 2937 | |
| 2938 | + r = -EINVAL; | |
| 2939 | + if (kvm_is_ucontrol(kvm)) | |
| 2940 | + break; | |
| 2937 | 2941 | r = -EFAULT; |
| 2938 | 2942 | if (copy_from_user(&s390int, argp, sizeof(s390int))) |
| 2939 | 2943 | break; |
| @@ -2998,9 +3002,8 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · 58 unmodified lines | ||
| 2998 | 3002 | r = -EFAULT; |
| 2999 | 3003 | if (copy_from_user(&args, argp, sizeof(args))) |
| 3000 | 3004 | break; |
| 3001 | − mutex_lock(&kvm->slots_lock); | |
| 3002 | − r = kvm_s390_get_cmma_bits(kvm, &args); | |
| 3003 | − mutex_unlock(&kvm->slots_lock); | |
| 3005 | + scoped_guard(mutex, &kvm->slots_arch_lock) | |
| 3006 | + r = kvm_s390_get_cmma_bits(kvm, &args); | |
| 3004 | 3007 | if (!r) { |
| 3005 | 3008 | r = copy_to_user(argp, &args, sizeof(args)); |
| 3006 | 3009 | if (r) |
| @@ -3014,9 +3017,9 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · 7 unmodified lines | ||
| 3014 | 3017 | r = -EFAULT; |
| 3015 | 3018 | if (copy_from_user(&args, argp, sizeof(args))) |
| 3016 | 3019 | break; |
| 3017 | − mutex_lock(&kvm->slots_lock); | |
| 3020 | + mutex_lock(&kvm->slots_arch_lock); | |
| 3018 | 3021 | r = kvm_s390_set_cmma_bits(kvm, &args); |
| 3019 | − mutex_unlock(&kvm->slots_lock); | |
| 3022 | + mutex_unlock(&kvm->slots_arch_lock); | |
| 3020 | 3023 | break; |
| 3021 | 3024 | } |
| 3022 | 3025 | case KVM_S390_PV_COMMAND: { |
| @@ -3247,7 +3250,8 @@ static void kvm_s390_crypto_init(struct kvm *kvm) · 224 unmodified lines | ||
| 3247 | 3250 | |
| 3248 | 3251 | static void sca_dispose(struct kvm *kvm) |
| 3249 | 3252 | { |
| 3250 | − free_pages_exact(kvm->arch.sca, sizeof(*kvm->arch.sca)); | |
| 3253 | + if (kvm->arch.sca) | |
| 3254 | + free_pages_exact(kvm->arch.sca, sizeof(*kvm->arch.sca)); | |
| 3251 | 3255 | kvm->arch.sca = NULL; |
| 3252 | 3256 | } |
| 3253 | 3257 | |
| @@ -3461,7 +3465,7 @@ static void sca_del_vcpu(struct kvm_vcpu *vcpu) · 207 unmodified lines | ||
| 3461 | 3465 | { |
| 3462 | 3466 | struct esca_block *sca = vcpu->kvm->arch.sca; |
| 3463 | 3467 | |
| 3464 | − if (!kvm_s390_use_sca_entries()) | |
| 3468 | + if (!kvm_s390_use_sca_entries() || !vcpu->arch.initialized) | |
| 3465 | 3469 | return; |
| 3466 | 3470 | |
| 3467 | 3471 | clear_bit_inv(vcpu->vcpu_id, (unsigned long *)sca->mcn); |
| @@ -3481,8 +3485,8 @@ static void sca_add_vcpu(struct kvm_vcpu *vcpu) · 13 unmodified lines | ||
| 3481 | 3485 | if (!kvm_s390_use_sca_entries()) |
| 3482 | 3486 | return; |
| 3483 | 3487 | |
| 3488 | + WRITE_ONCE(sca->cpu[vcpu->vcpu_id].sda, virt_to_phys(vcpu->arch.sie_block)); | |
| 3484 | 3489 | set_bit_inv(vcpu->vcpu_id, (unsigned long *)sca->mcn); |
| 3485 | − sca->cpu[vcpu->vcpu_id].sda = virt_to_phys(vcpu->arch.sie_block); | |
| 3486 | 3490 | } |
| 3487 | 3491 | |
| 3488 | 3492 | static int sca_can_add_vcpu(struct kvm *kvm, unsigned int id) |
| @@ -3613,6 +3617,9 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu) · 124 unmodified lines | ||
| 3613 | 3617 | if (test_kvm_facility(vcpu->kvm, 74) || vcpu->kvm->arch.user_instr0 || |
| 3614 | 3618 | vcpu->kvm->arch.user_operexec) |
| 3615 | 3619 | vcpu->arch.sie_block->ictl |= ICTL_OPEREXC; |
| 3620 | + | |
| 3621 | + /* Pairs with smp_load_acquire() in kvm_arch_vcpu_ioctl_run() and kvm_arch_vcpu_ioctl() */ | |
| 3622 | + smp_store_release(&vcpu->arch.initialized, true); | |
| 3616 | 3623 | } |
| 3617 | 3624 | |
| 3618 | 3625 | static bool kvm_has_pckmo_subfunc(struct kvm *kvm, unsigned long nr) |
| @@ -3674,7 +3681,8 @@ static void kvm_s390_vcpu_crypto_setup(struct kvm_vcpu *vcpu) · 55 unmodified lines | ||
| 3674 | 3681 | |
| 3675 | 3682 | void kvm_s390_vcpu_unsetup_cmma(struct kvm_vcpu *vcpu) |
| 3676 | 3683 | { |
| 3677 | − free_page((unsigned long)phys_to_virt(vcpu->arch.sie_block->cbrlo)); | |
| 3684 | + if (vcpu->arch.sie_block->cbrlo) | |
| 3685 | + free_page((unsigned long)phys_to_virt(vcpu->arch.sie_block->cbrlo)); | |
| 3678 | 3686 | vcpu->arch.sie_block->cbrlo = 0; |
| 3679 | 3687 | } |
| 3680 | 3688 | |
| @@ -3792,21 +3800,21 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id) · 111 unmodified lines | ||
| 3792 | 3800 | return 0; |
| 3793 | 3801 | } |
| 3794 | 3802 | |
| 3803 | +DEFINE_FREE(sie_page, struct sie_page *, if (_T) free_page((unsigned long)(_T))) | |
| 3804 | + | |
| 3795 | 3805 | int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) |
| 3796 | 3806 | { |
| 3797 | − struct sie_page *sie_page; | |
| 3807 | + struct kvm_s390_mmu_cache *mc __free(kvm_s390_mmu_cache) = NULL; | |
| 3808 | + struct sie_page *sie_page __free(sie_page) = NULL; | |
| 3798 | 3809 | int rc; |
| 3799 | 3810 | |
| 3800 | 3811 | BUILD_BUG_ON(sizeof(struct sie_page) != 4096); |
| 3801 | − vcpu->arch.mc = kvm_s390_new_mmu_cache(); | |
| 3802 | − if (!vcpu->arch.mc) | |
| 3812 | + mc = kvm_s390_new_mmu_cache(); | |
| 3813 | + if (!mc) | |
| 3803 | 3814 | return -ENOMEM; |
| 3804 | 3815 | sie_page = (struct sie_page *) get_zeroed_page(GFP_KERNEL_ACCOUNT); |
| 3805 | − if (!sie_page) { | |
| 3806 | − kvm_s390_free_mmu_cache(vcpu->arch.mc); | |
| 3807 | − vcpu->arch.mc = NULL; | |
| 3816 | + if (!sie_page) | |
| 3808 | 3817 | return -ENOMEM; |
| 3809 | − } | |
| 3810 | 3818 | |
| 3811 | 3819 | vcpu->arch.sie_block = &sie_page->sie_block; |
| 3812 | 3820 | vcpu->arch.sie_block->itdba = virt_to_phys(&sie_page->itdb); |
| @@ -3848,10 +3856,9 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) · 35 unmodified lines | ||
| 3848 | 3856 | vcpu->run->kvm_valid_regs |= KVM_SYNC_FPRS; |
| 3849 | 3857 | |
| 3850 | 3858 | if (kvm_is_ucontrol(vcpu->kvm)) { |
| 3851 | − rc = -ENOMEM; | |
| 3852 | 3859 | vcpu->arch.gmap = gmap_new_child(vcpu->kvm->arch.gmap, -1UL); |
| 3853 | 3860 | if (!vcpu->arch.gmap) |
| 3854 | − goto out_free_sie_block; | |
| 3861 | + return -ENOMEM; | |
| 3855 | 3862 | } |
| 3856 | 3863 | |
| 3857 | 3864 | VM_EVENT(vcpu->kvm, 3, "create cpu %d at 0x%p, sie block at 0x%p", |
| @@ -3859,20 +3866,19 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) · 1 unmodified line | ||
| 3859 | 3866 | trace_kvm_s390_create_vcpu(vcpu->vcpu_id, vcpu, vcpu->arch.sie_block); |
| 3860 | 3867 | |
| 3861 | 3868 | rc = kvm_s390_vcpu_setup(vcpu); |
| 3862 | − if (rc) | |
| 3863 | − goto out_ucontrol_uninit; | |
| 3869 | + if (rc) { | |
| 3870 | + if (kvm_is_ucontrol(vcpu->kvm)) { | |
| 3871 | + scoped_guard(spinlock, &vcpu->kvm->arch.gmap->children_lock) | |
| 3872 | + gmap_remove_child(vcpu->arch.gmap); | |
| 3873 | + vcpu->arch.gmap = gmap_put(vcpu->arch.gmap); | |
| 3874 | + } | |
| 3875 | + return rc; | |
| 3876 | + } | |
| 3864 | 3877 | |
| 3878 | + vcpu->arch.mc = no_free_ptr(mc); | |
| 3879 | + sie_page = NULL; | |
| 3865 | 3880 | kvm_s390_update_topology_change_report(vcpu->kvm, 1); |
| 3866 | 3881 | return 0; |
| 3867 | − | |
| 3868 | −out_ucontrol_uninit: | |
| 3869 | − if (kvm_is_ucontrol(vcpu->kvm)) { | |
| 3870 | − gmap_remove_child(vcpu->arch.gmap); | |
| 3871 | − vcpu->arch.gmap = gmap_put(vcpu->arch.gmap); | |
| 3872 | − } | |
| 3873 | −out_free_sie_block: | |
| 3874 | − free_page((unsigned long)(vcpu->arch.sie_block)); | |
| 3875 | − return rc; | |
| 3876 | 3882 | } |
| 3877 | 3883 | |
| 3878 | 3884 | int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu) |
| @@ -5039,6 +5045,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 1160 unmodified lines | ||
| 5039 | 5045 | kvm_run->kvm_dirty_regs & ~KVM_SYNC_S390_VALID_FIELDS) |
| 5040 | 5046 | return -EINVAL; |
| 5041 | 5047 | |
| 5048 | + /* Pairs with smp_store_release() in kvm_arch_vcpu_postcreate() */ | |
| 5049 | + if (!smp_load_acquire(&vcpu->arch.initialized)) | |
| 5050 | + return -EINVAL; | |
| 5051 | + | |
| 5042 | 5052 | vcpu_load(vcpu); |
| 5043 | 5053 | |
| 5044 | 5054 | if (guestdbg_exit_pending(vcpu)) { |
| @@ -5447,6 +5457,8 @@ long kvm_arch_vcpu_unlocked_ioctl(struct file *filp, unsigned int ioctl, · 402 unmodified lines | ||
| 5447 | 5457 | struct kvm_s390_interrupt s390int; |
| 5448 | 5458 | struct kvm_s390_irq s390irq = {}; |
| 5449 | 5459 | |
| 5460 | + if (kvm_is_ucontrol(vcpu->kvm)) | |
| 5461 | + return -EINVAL; | |
| 5450 | 5462 | if (copy_from_user(&s390int, argp, sizeof(s390int))) |
| 5451 | 5463 | return -EFAULT; |
| 5452 | 5464 | if (s390int_to_s390irq(&s390int, &s390irq)) |
| @@ -5523,6 +5535,10 @@ long kvm_arch_vcpu_ioctl(struct file *filp, · 70 unmodified lines | ||
| 5523 | 5535 | long r; |
| 5524 | 5536 | u16 rc, rrc; |
| 5525 | 5537 | |
| 5538 | + /* Pairs with smp_store_release() in kvm_arch_vcpu_postcreate() */ | |
| 5539 | + if (!smp_load_acquire(&vcpu->arch.initialized)) | |
| 5540 | + return -EINVAL; | |
| 5541 | + | |
| 5526 | 5542 | vcpu_load(vcpu); |
| 5527 | 5543 | |
| 5528 | 5544 | switch (ioctl) { |
| @@ -5794,14 +5810,30 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, · 265 unmodified lines | ||
| 5794 | 5810 | return 0; |
| 5795 | 5811 | } |
| 5796 | 5812 | |
| 5813 | +static long cmma_d_count_pte(union pte *ptep, gfn_t gfn, gfn_t next, struct dat_walk *walk) | |
| 5814 | +{ | |
| 5815 | + union pgste pgste; | |
| 5816 | + | |
| 5817 | + pgste = pgste_get_lock(ptep); | |
| 5818 | + if (pgste.cmma_d) { | |
| 5819 | + pgste.cmma_d = 0; | |
| 5820 | + atomic64_dec(walk->priv); | |
| 5821 | + } | |
| 5822 | + pgste_set_unlock(ptep, pgste); | |
| 5823 | + return 0; | |
| 5824 | +} | |
| 5825 | + | |
| 5797 | 5826 | void kvm_arch_commit_memory_region(struct kvm *kvm, |
| 5798 | 5827 | struct kvm_memory_slot *old, |
| 5799 | 5828 | const struct kvm_memory_slot *new, |
| 5800 | 5829 | enum kvm_mr_change change) |
| 5801 | 5830 | { |
| 5802 | − struct kvm_s390_mmu_cache *mc = NULL; | |
| 5831 | + const struct dat_walk_ops ops = { .pte_entry = cmma_d_count_pte, }; | |
| 5832 | + struct kvm_s390_mmu_cache *mc __free(kvm_s390_mmu_cache) = NULL; | |
| 5803 | 5833 | int rc = 0; |
| 5804 | 5834 | |
| 5835 | + guard(mutex)(&kvm->slots_arch_lock); | |
| 5836 | + | |
| 5805 | 5837 | if (change == KVM_MR_FLAGS_ONLY) |
| 5806 | 5838 | return; |
| 5807 | 5839 | |
| @@ -5812,6 +5844,12 @@ void kvm_arch_commit_memory_region(struct kvm *kvm, · 4 unmodified lines | ||
| 5812 | 5844 | } |
| 5813 | 5845 | |
| 5814 | 5846 | scoped_guard(write_lock, &kvm->mmu_lock) { |
| 5847 | + if (kvm->arch.migration_mode && kvm->arch.use_cmma && old) { | |
| 5848 | + _dat_walk_gfn_range(old->base_gfn, old->base_gfn + old->npages, | |
| 5849 | + kvm->arch.gmap->asce, &ops, DAT_WALK_IGN_HOLES, | |
| 5850 | + &kvm->arch.cmma_dirty_pages); | |
| 5851 | + } | |
| 5852 | + | |
| 5815 | 5853 | switch (change) { |
| 5816 | 5854 | case KVM_MR_DELETE: |
| 5817 | 5855 | rc = dat_delete_slot(mc, kvm->arch.gmap->asce, old->base_gfn, old->npages); |
| @@ -5833,7 +5871,6 @@ void kvm_arch_commit_memory_region(struct kvm *kvm, · 15 unmodified lines | ||
| 5833 | 5871 | out: |
| 5834 | 5872 | if (rc) |
| 5835 | 5873 | pr_warn("failed to commit memory region\n"); |
| 5836 | − kvm_s390_free_mmu_cache(mc); | |
| 5837 | 5874 | return; |
| 5838 | 5875 | } |
| 5839 | 5876 | |
| @@ -190,40 +190,61 @@ static int kvm_zpci_clear_airq(struct zpci_dev *zdev) · 189 unmodified lines | ||
| 190 | 190 | return cc ? -EIO : 0; |
| 191 | 191 | } |
| 192 | 192 | |
| 193 | −static inline void unaccount_mem(unsigned long nr_pages) | |
| 193 | +static inline void unaccount_mem(struct kvm_zdev *kzdev, unsigned long nr_pages) | |
| 194 | 194 | { |
| 195 | − struct user_struct *user = get_uid(current_user()); | |
| 195 | + struct user_struct *user = kzdev->user_account; | |
| 196 | + struct mm_struct *mm_account = kzdev->mm_account; | |
| 196 | 197 | |
| 197 | − if (user) | |
| 198 | + if (user) { | |
| 198 | 199 | atomic_long_sub(nr_pages, &user->locked_vm); |
| 199 | − if (current->mm) | |
| 200 | − atomic64_sub(nr_pages, ¤t->mm->pinned_vm); | |
| 200 | + free_uid(user); | |
| 201 | + kzdev->user_account = NULL; | |
| 202 | + } | |
| 203 | + | |
| 204 | + if (mm_account) { | |
| 205 | + atomic64_sub(nr_pages, &mm_account->pinned_vm); | |
| 206 | + mmdrop(mm_account); | |
| 207 | + kzdev->mm_account = NULL; | |
| 208 | + } | |
| 201 | 209 | } |
| 202 | 210 | |
| 203 | −static inline int account_mem(unsigned long nr_pages) | |
| 211 | +static inline int account_mem(struct kvm_zdev *kzdev, unsigned long nr_pages) | |
| 204 | 212 | { |
| 205 | 213 | struct user_struct *user = get_uid(current_user()); |
| 206 | 214 | unsigned long page_limit, cur_pages, new_pages; |
| 215 | + int rc = 0; | |
| 207 | 216 | |
| 208 | 217 | page_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; |
| 209 | 218 | |
| 210 | 219 | cur_pages = atomic_long_read(&user->locked_vm); |
| 211 | 220 | do { |
| 212 | 221 | new_pages = cur_pages + nr_pages; |
| 213 | − if (new_pages > page_limit) | |
| 214 | − return -ENOMEM; | |
| 222 | + if (new_pages > page_limit) { | |
| 223 | + rc = -ENOMEM; | |
| 224 | + goto out; | |
| 225 | + } | |
| 215 | 226 | } while (!atomic_long_try_cmpxchg(&user->locked_vm, &cur_pages, new_pages)); |
| 216 | 227 | |
| 217 | − atomic64_add(nr_pages, ¤t->mm->pinned_vm); | |
| 228 | + if (current->mm) { | |
| 229 | + mmgrab(current->mm); | |
| 230 | + atomic64_add(nr_pages, ¤t->mm->pinned_vm); | |
| 231 | + } | |
| 232 | + | |
| 233 | + kzdev->user_account = user; | |
| 234 | + kzdev->mm_account = current->mm; | |
| 218 | 235 | |
| 219 | 236 | return 0; |
| 237 | + | |
| 238 | +out: | |
| 239 | + free_uid(user); | |
| 240 | + return rc; | |
| 220 | 241 | } |
| 221 | 242 | |
| 222 | 243 | static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, |
| 223 | 244 | bool assist) |
| 224 | 245 | { |
| 225 | 246 | struct page *pages[1], *aibv_page, *aisb_page = NULL; |
| 226 | − unsigned int msi_vecs, idx; | |
| 247 | + unsigned int msi_vecs, idx, size; | |
| 227 | 248 | struct zpci_gaite *gaite; |
| 228 | 249 | unsigned long hva, bit; |
| 229 | 250 | struct kvm *kvm; |
| @@ -237,6 +258,10 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 7 unmodified lines | ||
| 237 | 258 | if (zdev->gisa == 0) |
| 238 | 259 | return -EINVAL; |
| 239 | 260 | |
| 261 | + /* AIF already enabled for the device */ | |
| 262 | + if (zdev->kzdev->fib.fmt0.aibv != 0) | |
| 263 | + return -EINVAL; | |
| 264 | + | |
| 240 | 265 | kvm = zdev->kzdev->kvm; |
| 241 | 266 | msi_vecs = min_t(unsigned int, fib->fmt0.noi, zdev->max_msi); |
| 242 | 267 | |
| @@ -246,6 +271,14 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 3 unmodified lines | ||
| 246 | 271 | return gisc; |
| 247 | 272 | |
| 248 | 273 | /* Replace AIBV address */ |
| 274 | + size = BITS_TO_LONGS(msi_vecs + fib->fmt0.aibvo) * sizeof(unsigned long); | |
| 275 | + npages = DIV_ROUND_UP((fib->fmt0.aibv & ~PAGE_MASK) + size, PAGE_SIZE); | |
| 276 | + /* AIBV cannot span more than 1 page */ | |
| 277 | + if (npages > 1) { | |
| 278 | + rc = -EINVAL; | |
| 279 | + goto out; | |
| 280 | + } | |
| 281 | + | |
| 249 | 282 | idx = srcu_read_lock(&kvm->srcu); |
| 250 | 283 | hva = gfn_to_hva(kvm, gpa_to_gfn((gpa_t)fib->fmt0.aibv)); |
| 251 | 284 | npages = pin_user_pages_fast(hva, 1, FOLL_WRITE | FOLL_LONGTERM, pages); |
| @@ -261,6 +294,12 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 9 unmodified lines | ||
| 261 | 294 | |
| 262 | 295 | /* Pin the guest AISB if one was specified */ |
| 263 | 296 | if (fib->fmt0.sum == 1) { |
| 297 | + /* AISB must be dword aligned */ | |
| 298 | + if (fib->fmt0.aisb & 0x7) { | |
| 299 | + rc = -EINVAL; | |
| 300 | + goto unpin1; | |
| 301 | + } | |
| 302 | + | |
| 264 | 303 | idx = srcu_read_lock(&kvm->srcu); |
| 265 | 304 | hva = gfn_to_hva(kvm, gpa_to_gfn((gpa_t)fib->fmt0.aisb)); |
| 266 | 305 | npages = pin_user_pages_fast(hva, 1, FOLL_WRITE | FOLL_LONGTERM, |
| @@ -275,20 +314,28 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 8 unmodified lines | ||
| 275 | 314 | } |
| 276 | 315 | |
| 277 | 316 | /* Account for pinned pages, roll back on failure */ |
| 278 | − if (account_mem(pcount)) | |
| 317 | + rc = account_mem(zdev->kzdev, pcount); | |
| 318 | + if (rc) | |
| 279 | 319 | goto unpin2; |
| 280 | 320 | |
| 281 | 321 | /* AISB must be allocated before we can fill in GAITE */ |
| 282 | 322 | mutex_lock(&aift->aift_lock); |
| 283 | 323 | bit = airq_iv_alloc_bit(aift->sbv); |
| 284 | − if (bit == -1UL) | |
| 324 | + if (bit == -1UL) { | |
| 325 | + rc = -ENOMEM; | |
| 285 | 326 | goto unlock; |
| 327 | + } | |
| 286 | 328 | zdev->aisb = bit; /* store the summary bit number */ |
| 287 | 329 | zdev->aibv = airq_iv_create(msi_vecs, AIRQ_IV_DATA | |
| 288 | 330 | AIRQ_IV_BITLOCK | |
| 289 | 331 | AIRQ_IV_GUESTVEC, |
| 290 | 332 | phys_to_virt(fib->fmt0.aibv)); |
| 291 | 333 | |
| 334 | + if (!zdev->aibv) { | |
| 335 | + rc = -ENOMEM; | |
| 336 | + goto free_aisb; | |
| 337 | + } | |
| 338 | + | |
| 292 | 339 | spin_lock_irq(&aift->gait_lock); |
| 293 | 340 | gaite = aift->gait + zdev->aisb; |
| 294 | 341 | |
| @@ -311,21 +358,39 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, · 16 unmodified lines | ||
| 311 | 358 | aift->kzdev[zdev->aisb] = zdev->kzdev; |
| 312 | 359 | spin_unlock_irq(&aift->gait_lock); |
| 313 | 360 | |
| 314 | − /* Update guest FIB for re-issue */ | |
| 315 | − fib->fmt0.aisbo = zdev->aisb & 63; | |
| 316 | − fib->fmt0.aisb = virt_to_phys(aift->sbv->vector) + (zdev->aisb / 64) * 8; | |
| 317 | − fib->fmt0.isc = gisc; | |
| 318 | − | |
| 319 | 361 | /* Save some guest fib values in the host for later use */ |
| 320 | − zdev->kzdev->fib.fmt0.isc = fib->fmt0.isc; | |
| 362 | + zdev->kzdev->fib.fmt0.isc = gisc; | |
| 321 | 363 | zdev->kzdev->fib.fmt0.aibv = fib->fmt0.aibv; |
| 322 | − mutex_unlock(&aift->aift_lock); | |
| 323 | 364 | |
| 324 | 365 | /* Issue the clp to setup the irq now */ |
| 325 | 366 | rc = kvm_zpci_set_airq(zdev); |
| 326 | − return rc; | |
| 367 | + if (!rc) { | |
| 368 | + mutex_unlock(&aift->aift_lock); | |
| 369 | + return rc; | |
| 370 | + } | |
| 371 | + | |
| 372 | + /* Start cleanup */ | |
| 373 | + zdev->kzdev->fib.fmt0.isc = 0; | |
| 374 | + zdev->kzdev->fib.fmt0.aibv = 0; | |
| 327 | 375 | |
| 376 | + spin_lock_irq(&aift->gait_lock); | |
| 377 | + gaite->count--; | |
| 378 | + gaite->aisb = 0; | |
| 379 | + gaite->gisc = 0; | |
| 380 | + gaite->aisbo = 0; | |
| 381 | + gaite->gisa = 0; | |
| 382 | + aift->kzdev[zdev->aisb] = NULL; | |
| 383 | + spin_unlock_irq(&aift->gait_lock); | |
| 384 | + | |
| 385 | + airq_iv_release(zdev->aibv); | |
| 386 | + zdev->aibv = NULL; | |
| 387 | + | |
| 388 | +free_aisb: | |
| 389 | + airq_iv_free_bit(aift->sbv, zdev->aisb); | |
| 390 | + zdev->aisb = 0; | |
| 328 | 391 | unlock: |
| 392 | + if (pcount > 0) | |
| 393 | + unaccount_mem(zdev->kzdev, pcount); | |
| 329 | 394 | mutex_unlock(&aift->aift_lock); |
| 330 | 395 | unpin2: |
| 331 | 396 | if (fib->fmt0.sum == 1) |
| @@ -396,7 +461,7 @@ static int kvm_s390_pci_aif_disable(struct zpci_dev *zdev, bool force) · 64 unmodified lines | ||
| 396 | 461 | pcount++; |
| 397 | 462 | } |
| 398 | 463 | if (pcount > 0) |
| 399 | − unaccount_mem(pcount); | |
| 464 | + unaccount_mem(kzdev, pcount); | |
| 400 | 465 | out: |
| 401 | 466 | mutex_unlock(&aift->aift_lock); |
| 402 | 467 | |
| @@ -22,6 +22,8 @@ struct kvm_zdev { · 21 unmodified lines | ||
| 22 | 22 | struct kvm *kvm; |
| 23 | 23 | struct zpci_fib fib; |
| 24 | 24 | struct list_head entry; |
| 25 | + struct user_struct *user_account; | |
| 26 | + struct mm_struct *mm_account; | |
| 25 | 27 | }; |
| 26 | 28 | |
| 27 | 29 | struct zpci_gaite { |
| @@ -289,6 +289,7 @@ static int handle_iske(struct kvm_vcpu *vcpu) · 288 unmodified lines | ||
| 289 | 289 | static int handle_rrbe(struct kvm_vcpu *vcpu) |
| 290 | 290 | { |
| 291 | 291 | unsigned long gaddr; |
| 292 | + union skey skey; | |
| 292 | 293 | int reg1, reg2; |
| 293 | 294 | int rc; |
| 294 | 295 | |
| @@ -307,12 +308,12 @@ static int handle_rrbe(struct kvm_vcpu *vcpu) · 12 unmodified lines | ||
| 307 | 308 | gaddr = kvm_s390_logical_to_effective(vcpu, gaddr); |
| 308 | 309 | gaddr = kvm_s390_real_to_abs(vcpu, gaddr); |
| 309 | 310 | scoped_guard(read_lock, &vcpu->kvm->mmu_lock) |
| 310 | − rc = dat_reset_reference_bit(vcpu->arch.gmap->asce, gpa_to_gfn(gaddr)); | |
| 311 | + rc = dat_reset_reference_bit(vcpu->arch.gmap->asce, gpa_to_gfn(gaddr), &skey); | |
| 311 | 312 | if (rc > 0) |
| 312 | 313 | return kvm_s390_inject_program_int(vcpu, rc); |
| 313 | 314 | if (rc < 0) |
| 314 | 315 | return rc; |
| 315 | − kvm_s390_set_psw_cc(vcpu, rc); | |
| 316 | + kvm_s390_set_psw_cc(vcpu, (skey.skey >> 1) & 3); | |
| 316 | 317 | return 0; |
| 317 | 318 | } |
| 318 | 319 | |
| @@ -1260,8 +1261,9 @@ static int handle_essa(struct kvm_vcpu *vcpu) · 941 unmodified lines | ||
| 1260 | 1261 | /* Retry the ESSA instruction */ |
| 1261 | 1262 | kvm_s390_retry_instr(vcpu); |
| 1262 | 1263 | } else { |
| 1263 | − scoped_guard(read_lock, &vcpu->kvm->mmu_lock) | |
| 1264 | − i = __do_essa(vcpu, orc); | |
| 1264 | + scoped_guard(mutex, &vcpu->kvm->slots_arch_lock) | |
| 1265 | + scoped_guard(read_lock, &vcpu->kvm->mmu_lock) | |
| 1266 | + i = __do_essa(vcpu, orc); | |
| 1265 | 1267 | if (i < 0) |
| 1266 | 1268 | return i; |
| 1267 | 1269 | /* Account for the possible extra cbrl entry */ |
| @@ -244,6 +244,24 @@ static void kvm_s390_clear_pv_state(struct kvm *kvm) · 243 unmodified lines | ||
| 244 | 244 | kvm->arch.pv.stor_var = NULL; |
| 245 | 245 | } |
| 246 | 246 | |
| 247 | +static void kvm_s390_pv_dispose_cpu(struct kvm_vcpu *vcpu, bool free_stor_base) | |
| 248 | +{ | |
| 249 | + if (free_stor_base) | |
| 250 | + free_pages(vcpu->arch.pv.stor_base, get_order(uv_info.guest_cpu_stor_len)); | |
| 251 | + free_page((unsigned long)sida_addr(vcpu->arch.sie_block)); | |
| 252 | + vcpu->arch.sie_block->pv_handle_cpu = 0; | |
| 253 | + vcpu->arch.sie_block->pv_handle_config = 0; | |
| 254 | + memset(&vcpu->arch.pv, 0, sizeof(vcpu->arch.pv)); | |
| 255 | + vcpu->arch.sie_block->sdf = 0; | |
| 256 | + /* | |
| 257 | + * The sidad field (for sdf == 2) is now the gbea field (for sdf == 0). | |
| 258 | + * Use the reset value of gbea to avoid leaking the kernel pointer of | |
| 259 | + * the just freed sida. | |
| 260 | + */ | |
| 261 | + vcpu->arch.sie_block->gbea = 1; | |
| 262 | + kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu); | |
| 263 | +} | |
| 264 | + | |
| 247 | 265 | int kvm_s390_pv_destroy_cpu(struct kvm_vcpu *vcpu, u16 *rc, u16 *rrc) |
| 248 | 266 | { |
| 249 | 267 | int cc; |
| @@ -258,24 +276,9 @@ int kvm_s390_pv_destroy_cpu(struct kvm_vcpu *vcpu, u16 *rc, u16 *rrc) · 8 unmodified lines | ||
| 258 | 276 | WARN_ONCE(cc, "protvirt destroy cpu failed rc %x rrc %x", *rc, *rrc); |
| 259 | 277 | |
| 260 | 278 | /* Intended memory leak for something that should never happen. */ |
| 261 | − if (!cc) | |
| 262 | − free_pages(vcpu->arch.pv.stor_base, | |
| 263 | − get_order(uv_info.guest_cpu_stor_len)); | |
| 264 | − | |
| 265 | − free_page((unsigned long)sida_addr(vcpu->arch.sie_block)); | |
| 266 | − vcpu->arch.sie_block->pv_handle_cpu = 0; | |
| 267 | − vcpu->arch.sie_block->pv_handle_config = 0; | |
| 268 | − memset(&vcpu->arch.pv, 0, sizeof(vcpu->arch.pv)); | |
| 269 | − vcpu->arch.sie_block->sdf = 0; | |
| 270 | − /* | |
| 271 | − * The sidad field (for sdf == 2) is now the gbea field (for sdf == 0). | |
| 272 | − * Use the reset value of gbea to avoid leaking the kernel pointer of | |
| 273 | − * the just freed sida. | |
| 274 | − */ | |
| 275 | − vcpu->arch.sie_block->gbea = 1; | |
| 276 | − kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu); | |
| 279 | + kvm_s390_pv_dispose_cpu(vcpu, !cc); | |
| 277 | 280 | |
| 278 | − return cc ? EIO : 0; | |
| 281 | + return cc ? -EIO : 0; | |
| 279 | 282 | } |
| 280 | 283 | |
| 281 | 284 | int kvm_s390_pv_create_cpu(struct kvm_vcpu *vcpu, u16 *rc, u16 *rrc) |
| @@ -319,9 +322,7 @@ int kvm_s390_pv_create_cpu(struct kvm_vcpu *vcpu, u16 *rc, u16 *rrc) · 37 unmodified lines | ||
| 319 | 322 | uvcb.header.rrc); |
| 320 | 323 | |
| 321 | 324 | if (cc) { |
| 322 | − u16 dummy; | |
| 323 | − | |
| 324 | − kvm_s390_pv_destroy_cpu(vcpu, &dummy, &dummy); | |
| 325 | + kvm_s390_pv_dispose_cpu(vcpu, true); | |
| 325 | 326 | return -EIO; |
| 326 | 327 | } |
| 327 | 328 | |
| @@ -809,7 +810,7 @@ static int unpack_one(struct kvm *kvm, unsigned long addr, u64 tweak, · 481 unmodified lines | ||
| 809 | 810 | return -EAGAIN; |
| 810 | 811 | } |
| 811 | 812 | |
| 812 | − if (ret && ret != -EAGAIN) | |
| 813 | + if (ret && ret != -EAGAIN && ret != -EINTR) | |
| 813 | 814 | KVM_UV_EVENT(kvm, 3, "PROTVIRT VM UNPACK: failed addr %llx with rc %x rrc %x", |
| 814 | 815 | uvcb.gaddr, *rc, *rrc); |
| 815 | 816 | return ret; |
| @@ -188,6 +188,7 @@ SYSCALL_DEFINE3(s390_pci_mmio_write, unsigned long, mmio_addr, · 187 unmodified lines | ||
| 188 | 188 | goto out_unlock_mmap; |
| 189 | 189 | } |
| 190 | 190 | |
| 191 | + ret = -EFAULT; | |
| 191 | 192 | io_addr = (void __iomem *)((args.pfn << PAGE_SHIFT) | |
| 192 | 193 | (mmio_addr & ~PAGE_MASK)); |
| 193 | 194 | |
| @@ -995,6 +995,9 @@ static int vector_mmsg_rx(struct vector_private *vp, int budget) · 994 unmodified lines | ||
| 995 | 995 | */ |
| 996 | 996 | dev_kfree_skb_irq(skb); |
| 997 | 997 | vp->estats.rx_encaps_errors++; |
| 998 | + (*skbuff_vector) = NULL; | |
| 999 | + mmsg_vector++; | |
| 1000 | + skbuff_vector++; | |
| 998 | 1001 | continue; |
| 999 | 1002 | } |
| 1000 | 1003 | if (header_check > 0) { |
| @@ -32,8 +32,15 @@ · 31 unmodified lines | ||
| 32 | 32 | int memcmp(const void *s1, const void *s2, size_t len) |
| 33 | 33 | { |
| 34 | 34 | bool diff; |
| 35 | − asm("repe cmpsb" | |
| 36 | − : "=@ccnz" (diff), "+D" (s1), "+S" (s2), "+c" (len)); | |
| 35 | + | |
| 36 | + /* | |
| 37 | + * Make sure ZF is properly set in the len==0 case because in it, | |
| 38 | + * RCX==0 and the REPE; CMPSB won't get executed. | |
| 39 | + */ | |
| 40 | + asm volatile("test %3, %3\n\t" | |
| 41 | + "repe cmpsb" | |
| 42 | + : "=@ccnz" (diff), "+D" (s1), "+S" (s2), "+c" (len) | |
| 43 | + : : "cc", "memory"); | |
| 37 | 44 | return diff; |
| 38 | 45 | } |
| 39 | 46 | |
| @@ -937,6 +937,8 @@ SYM_CODE_START(paranoid_entry) · 936 unmodified lines | ||
| 937 | 937 | IBRS_ENTER save_reg=%r15 |
| 938 | 938 | UNTRAIN_RET_FROM_CALL |
| 939 | 939 | |
| 940 | + HANDLE_INTR_SAFERET 8(%rsp) | |
| 941 | + | |
| 940 | 942 | RET |
| 941 | 943 | SYM_CODE_END(paranoid_entry) |
| 942 | 944 | |
| @@ -1039,6 +1041,11 @@ SYM_CODE_START(error_entry) · 96 unmodified lines | ||
| 1039 | 1041 | movl %ecx, %eax /* zero extend */ |
| 1040 | 1042 | cmpq %rax, RIP+8(%rsp) |
| 1041 | 1043 | je .Lbstep_iret |
| 1044 | + | |
| 1045 | + VALIDATE_UNRET_END | |
| 1046 | + | |
| 1047 | + HANDLE_INTR_SAFERET 8(%rsp) | |
| 1048 | + | |
| 1042 | 1049 | cmpq $.Lgs_change, RIP+8(%rsp) |
| 1043 | 1050 | jne .Lerror_entry_done_lfence |
| 1044 | 1051 | |
| @@ -1057,7 +1064,6 @@ SYM_CODE_START(error_entry) · 12 unmodified lines | ||
| 1057 | 1064 | FENCE_SWAPGS_KERNEL_ENTRY |
| 1058 | 1065 | CALL_DEPTH_ACCOUNT |
| 1059 | 1066 | leaq 8(%rsp), %rax /* return pt_regs pointer */ |
| 1060 | − VALIDATE_UNRET_END | |
| 1061 | 1067 | RET |
| 1062 | 1068 | |
| 1063 | 1069 | .Lbstep_iret: |
| @@ -1858,7 +1858,7 @@ struct kvm_x86_ops { · 1857 unmodified lines | ||
| 1858 | 1858 | * Can potentially get non-canonical addresses through INVLPGs, which |
| 1859 | 1859 | * the implementation may choose to ignore if appropriate. |
| 1860 | 1860 | */ |
| 1861 | − void (*flush_tlb_gva)(struct kvm_vcpu *vcpu, gva_t addr); | |
| 1861 | + void (*flush_tlb_gva)(struct kvm_vcpu *vcpu, gva_t addr, bool *full); | |
| 1862 | 1862 | |
| 1863 | 1863 | /* |
| 1864 | 1864 | * Flush any TLB entries created by the guest. Like tlb_flush_gva(), |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | #include <asm/msr-index.h> |
| 13 | 13 | #include <asm/unwind_hints.h> |
| 14 | 14 | #include <asm/percpu.h> |
| 15 | +#include <asm/ptrace-abi.h> | |
| 15 | 16 | |
| 16 | 17 | /* |
| 17 | 18 | * Call depth tracking for Intel SKL CPUs to address the RSB underflow |
| @@ -176,6 +177,50 @@ · 158 unmodified lines | ||
| 176 | 177 | add $(BITS_PER_LONG/8), %_ASM_SP; \ |
| 177 | 178 | lfence; |
| 178 | 179 | |
| 180 | +/* | |
| 181 | + * Helper for detecting if an interrupt occurred at an unsafe location within | |
| 182 | + * Safe-RET. If Safe-RET is interrupted after the CALL or LEA the RSB may get | |
| 183 | + * poisoned by the interrupt handler. | |
| 184 | + * | |
| 185 | + * The Safe-RET sequence is: | |
| 186 | + * | |
| 187 | + * CALL | |
| 188 | + * LEA 8(%RSP), %RSP | |
| 189 | + * RET | |
| 190 | + * | |
| 191 | + * The two CMPs below check whether RIP points to after the CALL or after the | |
| 192 | + * LEA. | |
| 193 | + * | |
| 194 | + * The LFENCE below is to address this particular speculation case: | |
| 195 | + * | |
| 196 | + * 1. Userspace runs and poisons the BTB around the safe-RET routine | |
| 197 | + * | |
| 198 | + * 2. Userspace triggers some kind of exception | |
| 199 | + * | |
| 200 | + * 3. Kernel executes error_entry() and mis-speculates the branch into thinking | |
| 201 | + * it actually came from kernel space | |
| 202 | + * | |
| 203 | + * 4. The kernel then further mis-speculates that the exception occurred due | |
| 204 | + * to an interrupted safe-RET | |
| 205 | + * | |
| 206 | + * 5. The handle_interrupted_saferet() routine speculatively executes and | |
| 207 | + * speculatively does a safe-RET. But this is unsafe since it was never | |
| 208 | + * untrained. | |
| 209 | + * | |
| 210 | + * The LFENCE fixes this by ensuring step 5 is never reached speculatively. | |
| 211 | + * Note that this LFENCE only occurs if safe-RET was actually interrupted (so | |
| 212 | + * it's outside of the normal path). | |
| 213 | + */ | |
| 214 | +#define __HANDLE_INTR_SAFERET(name, pt_regs) \ | |
| 215 | + cmpq $(name), RIP+pt_regs; \ | |
| 216 | + jb 1f; \ | |
| 217 | + cmpq $(name)+5, RIP+pt_regs; \ | |
| 218 | + ja 1f; \ | |
| 219 | + lfence; \ | |
| 220 | + leaq pt_regs, %rdi; \ | |
| 221 | + call handle_interrupted_saferet; \ | |
| 222 | + 1: | |
| 223 | + | |
| 179 | 224 | #ifdef __ASSEMBLER__ |
| 180 | 225 | |
| 181 | 226 | /* |
| @@ -293,6 +338,14 @@ · 111 unmodified lines | ||
| 293 | 338 | #define UNTRAIN_RET_FROM_CALL \ |
| 294 | 339 | __UNTRAIN_RET X86_FEATURE_ENTRY_IBPB, __stringify(RESET_CALL_DEPTH_FROM_CALL) |
| 295 | 340 | |
| 341 | +.macro HANDLE_INTR_SAFERET pt_regs | |
| 342 | +#ifdef CONFIG_MITIGATION_SRSO | |
| 343 | + ALTERNATIVE_2 "", \ | |
| 344 | + __stringify(__HANDLE_INTR_SAFERET(srso_safe_ret, \pt_regs)), X86_FEATURE_SRSO, \ | |
| 345 | + __stringify(__HANDLE_INTR_SAFERET(srso_alias_safe_ret, \pt_regs)), X86_FEATURE_SRSO_ALIAS | |
| 346 | + | |
| 347 | +#endif | |
| 348 | +.endm | |
| 296 | 349 | |
| 297 | 350 | .macro CALL_DEPTH_ACCOUNT |
| 298 | 351 | #ifdef CONFIG_MITIGATION_CALL_DEPTH_TRACKING |
| @@ -625,6 +678,10 @@ static __always_inline void x86_idle_clear_cpu_buffers(void) · 326 unmodified lines | ||
| 625 | 678 | x86_clear_cpu_buffers(); |
| 626 | 679 | } |
| 627 | 680 | |
| 681 | +void srso_safe_ret(void); | |
| 682 | +void srso_alias_safe_ret(void); | |
| 683 | +void handle_interrupted_saferet(struct pt_regs *regs); | |
| 684 | + | |
| 628 | 685 | #endif /* __ASSEMBLER__ */ |
| 629 | 686 | |
| 630 | 687 | #endif /* _ASM_X86_NOSPEC_BRANCH_H_ */ |
| @@ -515,11 +515,13 @@ static void bsp_init_amd(struct cpuinfo_x86 *c) · 514 unmodified lines | ||
| 515 | 515 | case 0x00 ... 0x2f: |
| 516 | 516 | case 0x40 ... 0x4f: |
| 517 | 517 | case 0x60 ... 0x7f: |
| 518 | + case 0xd0 ... 0xd7: | |
| 518 | 519 | setup_force_cpu_cap(X86_FEATURE_ZEN5); |
| 519 | 520 | break; |
| 520 | 521 | case 0x50 ... 0x5f: |
| 521 | 522 | case 0x80 ... 0xaf: |
| 522 | − case 0xc0 ... 0xef: | |
| 523 | + case 0xc0 ... 0xcf: | |
| 524 | + case 0xd8 ... 0xef: | |
| 523 | 525 | setup_force_cpu_cap(X86_FEATURE_ZEN6); |
| 524 | 526 | break; |
| 525 | 527 | default: |
| @@ -3775,3 +3775,42 @@ void __warn_thunk(void) · 3774 unmodified lines | ||
| 3775 | 3775 | { |
| 3776 | 3776 | WARN_ONCE(1, "Unpatched return thunk in use. This should not happen!\n"); |
| 3777 | 3777 | } |
| 3778 | + | |
| 3779 | +#ifdef CONFIG_MITIGATION_SRSO | |
| 3780 | +/* | |
| 3781 | + * Called during exception/interrupt entry if interrupted during the | |
| 3782 | + * safe-RET sequence. The safe-RET sequence consists of 3 instructions: | |
| 3783 | + * | |
| 3784 | + * CALL | |
| 3785 | + * LEA 8(%RSP), %RSP | |
| 3786 | + * RET | |
| 3787 | + * | |
| 3788 | + * An interrupt after the CALL or after the LEA could potentially lead | |
| 3789 | + * to branch predictor poisoning and results in the sequence not being | |
| 3790 | + * able to be safely resumed. | |
| 3791 | + * | |
| 3792 | + * Therefore, modify the regs state as if the remaining part of the | |
| 3793 | + * safe-RET sequence executed so the interrupt returns back to the | |
| 3794 | + * desired return target, instead of the to the safe-RET sequence. | |
| 3795 | + */ | |
| 3796 | +void noinstr handle_interrupted_saferet(struct pt_regs *regs) | |
| 3797 | +{ | |
| 3798 | + unsigned long rip = regs->ip; | |
| 3799 | + | |
| 3800 | + if (rip == (unsigned long) srso_safe_ret || | |
| 3801 | + rip == (unsigned long) srso_alias_safe_ret) { | |
| 3802 | + /* Modify stack pointer as if LEA executed: */ | |
| 3803 | + regs->sp += 8; | |
| 3804 | + } | |
| 3805 | + | |
| 3806 | + /* | |
| 3807 | + * Adjust registers as if RET executed: | |
| 3808 | + * | |
| 3809 | + * 1. Read the return address off the stack and into rIP: | |
| 3810 | + */ | |
| 3811 | + regs->ip = *(unsigned long *)(regs->sp); | |
| 3812 | + | |
| 3813 | + /* 2. Pop rIP off the stack: */ | |
| 3814 | + regs->sp += 8; | |
| 3815 | +} | |
| 3816 | +#endif /* CONFIG_MITIGATION_SRSO */ | |
| @@ -2266,10 +2266,10 @@ void mcheck_cpu_init(struct cpuinfo_x86 *c) · 2265 unmodified lines | ||
| 2266 | 2266 | |
| 2267 | 2267 | mca_cfg.initialized = 1; |
| 2268 | 2268 | |
| 2269 | + __mcheck_cpu_setup_timer(); | |
| 2269 | 2270 | __mcheck_cpu_init_generic(); |
| 2270 | 2271 | __mcheck_cpu_init_vendor(c); |
| 2271 | 2272 | __mcheck_cpu_init_prepare_banks(); |
| 2272 | − __mcheck_cpu_setup_timer(); | |
| 2273 | 2273 | cr4_set_bits(X86_CR4_MCE); |
| 2274 | 2274 | } |
| 2275 | 2275 | |
| @@ -1974,6 +1974,7 @@ int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu) · 1973 unmodified lines | ||
| 1974 | 1974 | u64 entries[KVM_HV_TLB_FLUSH_FIFO_SIZE]; |
| 1975 | 1975 | int i, j, count; |
| 1976 | 1976 | gva_t gva; |
| 1977 | + bool full = false; | |
| 1977 | 1978 | |
| 1978 | 1979 | if (!tdp_enabled || !hv_vcpu) |
| 1979 | 1980 | return -EINVAL; |
| @@ -1982,7 +1983,7 @@ int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu) · 2 unmodified lines | ||
| 1982 | 1983 | |
| 1983 | 1984 | count = kfifo_out(&tlb_flush_fifo->entries, entries, KVM_HV_TLB_FLUSH_FIFO_SIZE); |
| 1984 | 1985 | |
| 1985 | − for (i = 0; i < count; i++) { | |
| 1986 | + for (i = 0; i < count && !full; i++) { | |
| 1986 | 1987 | if (entries[i] == KVM_HV_TLB_FLUSHALL_ENTRY) |
| 1987 | 1988 | goto out_flush_all; |
| 1988 | 1989 | |
| @@ -1991,11 +1992,11 @@ int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu) · 2 unmodified lines | ||
| 1991 | 1992 | * pages to flush. |
| 1992 | 1993 | */ |
| 1993 | 1994 | gva = entries[i] & PAGE_MASK; |
| 1994 | − for (j = 0; j < (entries[i] & ~PAGE_MASK) + 1; j++) { | |
| 1995 | + for (j = 0; j < (entries[i] & ~PAGE_MASK) + 1 && !full; j++) { | |
| 1995 | 1996 | if (is_noncanonical_invlpg_address(gva + j * PAGE_SIZE, vcpu)) |
| 1996 | 1997 | continue; |
| 1997 | 1998 | |
| 1998 | − kvm_x86_call(flush_tlb_gva)(vcpu, gva + j * PAGE_SIZE); | |
| 1999 | + kvm_x86_call(flush_tlb_gva)(vcpu, gva + j * PAGE_SIZE, &full); | |
| 1999 | 2000 | } |
| 2000 | 2001 | |
| 2001 | 2002 | ++vcpu->stat.tlb_flush; |
| @@ -722,6 +722,26 @@ static struct kvm_lpage_info *lpage_info_slot(gfn_t gfn, · 721 unmodified lines | ||
| 722 | 722 | return &slot->arch.lpage_info[level - 2][idx]; |
| 723 | 723 | } |
| 724 | 724 | |
| 725 | +static bool kvm_gfn_is_lpage_allowed(struct kvm *kvm, | |
| 726 | + const struct kvm_memory_slot *slot, | |
| 727 | + gfn_t gfn, int level) | |
| 728 | +{ | |
| 729 | + const struct kvm_memory_slot *other_slot; | |
| 730 | + | |
| 731 | + BUILD_BUG_ON(KVM_MAX_NR_ADDRESS_SPACES > 2); | |
| 732 | + | |
| 733 | + if (lpage_info_slot(gfn, slot, level)->disallow_lpage) | |
| 734 | + return false; | |
| 735 | + | |
| 736 | + if (kvm_arch_nr_memslot_as_ids(kvm) > 1) { | |
| 737 | + other_slot = __gfn_to_memslot(__kvm_memslots(kvm, slot->as_id ^ 1), gfn); | |
| 738 | + if (other_slot && lpage_info_slot(gfn, other_slot, level)->disallow_lpage) | |
| 739 | + return false; | |
| 740 | + } | |
| 741 | + | |
| 742 | + return true; | |
| 743 | +} | |
| 744 | + | |
| 725 | 745 | /* |
| 726 | 746 | * The most significant bit in disallow_lpage tracks whether or not memory |
| 727 | 747 | * attributes are mixed, i.e. not identical for all gfns at the current level. |
| @@ -2422,6 +2442,9 @@ static union kvm_mmu_page_role kvm_mmu_child_role(u64 *sptep, bool direct, · 1694 unmodified lines | ||
| 2422 | 2442 | role.direct = direct; |
| 2423 | 2443 | role.passthrough = 0; |
| 2424 | 2444 | |
| 2445 | + WARN_ON_ONCE(role.invalid); | |
| 2446 | + role.invalid = 0; | |
| 2447 | + | |
| 2425 | 2448 | /* |
| 2426 | 2449 | * If the guest has 4-byte PTEs then that means it's using 32-bit, |
| 2427 | 2450 | * 2-level, non-PAE paging. KVM shadows such guests with PAE paging |
| @@ -2968,7 +2991,7 @@ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot, · 540 unmodified lines | ||
| 2968 | 2991 | * write-protected (see above), thus if the gfn can be mapped with a |
| 2969 | 2992 | * hugepage and isn't write-tracked, it can't have a shadow page. |
| 2970 | 2993 | */ |
| 2971 | − if (!lpage_info_slot(gfn, slot, PG_LEVEL_2M)->disallow_lpage) | |
| 2994 | + if (kvm_gfn_is_lpage_allowed(kvm, slot, gfn, PG_LEVEL_2M)) | |
| 2972 | 2995 | return 0; |
| 2973 | 2996 | |
| 2974 | 2997 | /* |
| @@ -6632,7 +6655,7 @@ void kvm_mmu_invalidate_addr(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, · 3657 unmodified lines | ||
| 6632 | 6655 | if (is_noncanonical_invlpg_address(addr, vcpu)) |
| 6633 | 6656 | return; |
| 6634 | 6657 | |
| 6635 | − kvm_x86_call(flush_tlb_gva)(vcpu, addr); | |
| 6658 | + kvm_x86_call(flush_tlb_gva)(vcpu, addr, NULL); | |
| 6636 | 6659 | } |
| 6637 | 6660 | |
| 6638 | 6661 | if (!mmu->sync_spte) |
| @@ -130,13 +130,23 @@ void __kvm_write_track_remove_gfn(struct kvm *kvm, · 129 unmodified lines | ||
| 130 | 130 | kvm_mmu_gfn_allow_lpage(slot, gfn); |
| 131 | 131 | } |
| 132 | 132 | |
| 133 | −/* | |
| 134 | − * check if the corresponding access on the specified guest page is tracked. | |
| 135 | − */ | |
| 133 | +static bool __kvm_gfn_is_write_tracked(const struct kvm_memory_slot *slot, | |
| 134 | + gfn_t gfn) | |
| 135 | +{ | |
| 136 | + int index; | |
| 137 | + | |
| 138 | + if (!slot) | |
| 139 | + return false; | |
| 140 | + | |
| 141 | + index = gfn_to_index(gfn, slot->base_gfn, PG_LEVEL_4K); | |
| 142 | + return !!READ_ONCE(slot->arch.gfn_write_track[index]); | |
| 143 | +} | |
| 144 | + | |
| 145 | +/* check if write access is tracked on the specified guest page. */ | |
| 136 | 146 | bool kvm_gfn_is_write_tracked(struct kvm *kvm, |
| 137 | 147 | const struct kvm_memory_slot *slot, gfn_t gfn) |
| 138 | 148 | { |
| 139 | − int index; | |
| 149 | + const struct kvm_memory_slot *other_slot; | |
| 140 | 150 | |
| 141 | 151 | if (!slot) |
| 142 | 152 | return false; |
| @@ -144,8 +154,18 @@ bool kvm_gfn_is_write_tracked(struct kvm *kvm, · 1 unmodified line | ||
| 144 | 154 | if (!kvm_page_track_write_tracking_enabled(kvm)) |
| 145 | 155 | return false; |
| 146 | 156 | |
| 147 | − index = gfn_to_index(gfn, slot->base_gfn, PG_LEVEL_4K); | |
| 148 | − return !!READ_ONCE(slot->arch.gfn_write_track[index]); | |
| 157 | + BUILD_BUG_ON(KVM_MAX_NR_ADDRESS_SPACES > 2); | |
| 158 | + | |
| 159 | + if (__kvm_gfn_is_write_tracked(slot, gfn)) | |
| 160 | + return true; | |
| 161 | + | |
| 162 | + if (kvm_arch_nr_memslot_as_ids(kvm) > 1) { | |
| 163 | + other_slot = __gfn_to_memslot(__kvm_memslots(kvm, slot->as_id ^ 1), gfn); | |
| 164 | + if (__kvm_gfn_is_write_tracked(other_slot, gfn)) | |
| 165 | + return true; | |
| 166 | + } | |
| 167 | + | |
| 168 | + return false; | |
| 149 | 169 | } |
| 150 | 170 | |
| 151 | 171 | #ifdef CONFIG_KVM_EXTERNAL_WRITE_TRACKING |
| @@ -240,14 +240,6 @@ static void avic_deactivate_vmcb(struct vcpu_svm *svm) · 239 unmodified lines | ||
| 240 | 240 | if (!is_sev_es_guest(&svm->vcpu)) |
| 241 | 241 | svm_set_intercept(svm, INTERCEPT_CR8_WRITE); |
| 242 | 242 | |
| 243 | − /* | |
| 244 | − * If running nested and the guest uses its own MSR bitmap, there | |
| 245 | − * is no need to update L0's msr bitmap | |
| 246 | − */ | |
| 247 | − if (is_guest_mode(&svm->vcpu) && | |
| 248 | − vmcb12_is_intercept(&svm->nested.ctl, INTERCEPT_MSR_PROT)) | |
| 249 | − return; | |
| 250 | − | |
| 251 | 243 | /* Enabling MSR intercept for x2APIC registers */ |
| 252 | 244 | avic_set_x2apic_msr_interception(svm, true); |
| 253 | 245 | } |
| @@ -97,6 +97,8 @@ static u64 sev_supported_vmsa_features __ro_after_init; · 96 unmodified lines | ||
| 97 | 97 | static u8 sev_enc_bit; |
| 98 | 98 | static DECLARE_RWSEM(sev_deactivate_lock); |
| 99 | 99 | static DEFINE_MUTEX(sev_bitmap_lock); |
| 100 | +/* Protects kvm_sev_info's enc_context_owner, mirror_vms and mirror_entry. */ | |
| 101 | +static DEFINE_MUTEX(sev_mirror_lock); | |
| 100 | 102 | unsigned int max_sev_asid; |
| 101 | 103 | static unsigned int min_sev_asid; |
| 102 | 104 | static unsigned int max_sev_es_asid; |
| @@ -2018,7 +2020,6 @@ static void sev_migrate_from(struct kvm *dst_kvm, struct kvm *src_kvm) · 1915 unmodified lines | ||
| 2018 | 2020 | dst->asid = src->asid; |
| 2019 | 2021 | dst->handle = src->handle; |
| 2020 | 2022 | dst->pages_locked = src->pages_locked; |
| 2021 | − dst->enc_context_owner = src->enc_context_owner; | |
| 2022 | 2023 | dst->es_active = src->es_active; |
| 2023 | 2024 | dst->vmsa_features = src->vmsa_features; |
| 2024 | 2025 | |
| @@ -2026,11 +2027,12 @@ static void sev_migrate_from(struct kvm *dst_kvm, struct kvm *src_kvm) · 1 unmodified line | ||
| 2026 | 2027 | src->active = false; |
| 2027 | 2028 | src->handle = 0; |
| 2028 | 2029 | src->pages_locked = 0; |
| 2029 | − src->enc_context_owner = NULL; | |
| 2030 | 2030 | src->es_active = false; |
| 2031 | 2031 | |
| 2032 | 2032 | list_cut_before(&dst->regions_list, &src->regions_list, &src->regions_list); |
| 2033 | 2033 | |
| 2034 | + mutex_lock(&sev_mirror_lock); | |
| 2035 | + | |
| 2034 | 2036 | /* |
| 2035 | 2037 | * If this VM has mirrors, "transfer" each mirror's refcount of the |
| 2036 | 2038 | * source to the destination (this KVM). The caller holds a reference |
| @@ -2047,12 +2049,15 @@ static void sev_migrate_from(struct kvm *dst_kvm, struct kvm *src_kvm) · 10 unmodified lines | ||
| 2047 | 2049 | * If this VM is a mirror, remove the old mirror from the owners list |
| 2048 | 2050 | * and add the new mirror to the list. |
| 2049 | 2051 | */ |
| 2050 | − if (is_mirroring_enc_context(dst_kvm)) { | |
| 2051 | − struct kvm_sev_info *owner_sev_info = to_kvm_sev_info(dst->enc_context_owner); | |
| 2052 | + if (is_mirroring_enc_context(src_kvm)) { | |
| 2053 | + struct kvm_sev_info *owner_sev_info = to_kvm_sev_info(src->enc_context_owner); | |
| 2052 | 2054 | |
| 2055 | + dst->enc_context_owner = src->enc_context_owner; | |
| 2056 | + src->enc_context_owner = NULL; | |
| 2053 | 2057 | list_del(&src->mirror_entry); |
| 2054 | 2058 | list_add_tail(&dst->mirror_entry, &owner_sev_info->mirror_vms); |
| 2055 | 2059 | } |
| 2060 | + mutex_unlock(&sev_mirror_lock); | |
| 2056 | 2061 | |
| 2057 | 2062 | kvm_for_each_vcpu(i, dst_vcpu, dst_kvm) { |
| 2058 | 2063 | dst_svm = to_svm(dst_vcpu); |
| @@ -2871,11 +2876,14 @@ int sev_vm_copy_enc_context_from(struct kvm *kvm, unsigned int source_fd) · 812 unmodified lines | ||
| 2871 | 2876 | * disappear until we're done with it |
| 2872 | 2877 | */ |
| 2873 | 2878 | source_sev = to_kvm_sev_info(source_kvm); |
| 2874 | − kvm_get_kvm(source_kvm); | |
| 2875 | − list_add_tail(&mirror_sev->mirror_entry, &source_sev->mirror_vms); | |
| 2876 | 2879 | |
| 2877 | 2880 | /* Set enc_context_owner and copy its encryption context over */ |
| 2881 | + mutex_lock(&sev_mirror_lock); | |
| 2882 | + kvm_get_kvm(source_kvm); | |
| 2883 | + list_add_tail(&mirror_sev->mirror_entry, &source_sev->mirror_vms); | |
| 2878 | 2884 | mirror_sev->enc_context_owner = source_kvm; |
| 2885 | + mutex_unlock(&sev_mirror_lock); | |
| 2886 | + | |
| 2879 | 2887 | mirror_sev->active = true; |
| 2880 | 2888 | mirror_sev->asid = source_sev->asid; |
| 2881 | 2889 | mirror_sev->fd = source_sev->fd; |
| @@ -2963,11 +2971,19 @@ void sev_vm_destroy(struct kvm *kvm) · 81 unmodified lines | ||
| 2963 | 2971 | * Note, mirror VMs don't support registering encrypted regions. |
| 2964 | 2972 | */ |
| 2965 | 2973 | if (is_mirroring_enc_context(kvm)) { |
| 2966 | − struct kvm *owner_kvm = sev->enc_context_owner; | |
| 2974 | + struct kvm *owner_kvm; | |
| 2967 | 2975 | |
| 2968 | − mutex_lock(&owner_kvm->lock); | |
| 2976 | + mutex_lock(&sev_mirror_lock); | |
| 2977 | + owner_kvm = sev->enc_context_owner; | |
| 2969 | 2978 | list_del(&sev->mirror_entry); |
| 2970 | − mutex_unlock(&owner_kvm->lock); | |
| 2979 | + sev->enc_context_owner = NULL; | |
| 2980 | + | |
| 2981 | + /* | |
| 2982 | + * The reference to owner_kvm cannot move after sev_mirror_lock is | |
| 2983 | + * released. Release it before kvm_put_kvm() so that owner_kvm is | |
| 2984 | + * never destroyed inside sev_mirror_lock. | |
| 2985 | + */ | |
| 2986 | + mutex_unlock(&sev_mirror_lock); | |
| 2971 | 2987 | kvm_put_kvm(owner_kvm); |
| 2972 | 2988 | return; |
| 2973 | 2989 | } |
| @@ -4227,18 +4227,31 @@ static void svm_flush_tlb_all(struct kvm_vcpu *vcpu) · 4226 unmodified lines | ||
| 4227 | 4227 | svm_flush_tlb_asid(vcpu); |
| 4228 | 4228 | } |
| 4229 | 4229 | |
| 4230 | −static void svm_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t gva) | |
| 4230 | +static void svm_flush_tlb_guest(struct kvm_vcpu *vcpu) | |
| 4231 | 4231 | { |
| 4232 | − struct vcpu_svm *svm = to_svm(vcpu); | |
| 4232 | + kvm_register_mark_dirty(vcpu, VCPU_REG_ERAPS); | |
| 4233 | 4233 | |
| 4234 | − invlpga(gva, svm->vmcb->control.asid); | |
| 4234 | + svm_flush_tlb_asid(vcpu); | |
| 4235 | 4235 | } |
| 4236 | 4236 | |
| 4237 | −static void svm_flush_tlb_guest(struct kvm_vcpu *vcpu) | |
| 4237 | +static void svm_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t gva, bool *full) | |
| 4238 | 4238 | { |
| 4239 | − kvm_register_mark_dirty(vcpu, VCPU_REG_ERAPS); | |
| 4239 | + struct vcpu_svm *svm = to_svm(vcpu); | |
| 4240 | 4240 | |
| 4241 | − svm_flush_tlb_asid(vcpu); | |
| 4241 | + /* | |
| 4242 | + * INVLPGA has had errata on Genoa and Turin, and even on older | |
| 4243 | + * generations there were reports of Windows BSODs if INVLPGA | |
| 4244 | + * was used for Hyper-V tlbflush. Use it only for shadow paging | |
| 4245 | + * where it seems to be okay. | |
| 4246 | + */ | |
| 4247 | + if (!npt_enabled) { | |
| 4248 | + invlpga(gva, svm->vmcb->control.asid); | |
| 4249 | + return; | |
| 4250 | + } | |
| 4251 | + | |
| 4252 | + svm_flush_tlb_guest(vcpu); | |
| 4253 | + if (full) | |
| 4254 | + *full = true; | |
| 4242 | 4255 | } |
| 4243 | 4256 | |
| 4244 | 4257 | static inline void sync_cr8_to_lapic(struct kvm_vcpu *vcpu) |
| @@ -109,6 +109,7 @@ struct kvm_sev_info { · 108 unmodified lines | ||
| 109 | 109 | u64 ap_jump_table; /* SEV-ES AP Jump Table address */ |
| 110 | 110 | u64 vmsa_features; |
| 111 | 111 | u16 ghcb_version; /* Highest guest GHCB protocol version allowed */ |
| 112 | + /* The three fields below are protected by sev_mirror_lock */ | |
| 112 | 113 | struct kvm *enc_context_owner; /* Owner of copied encryption context */ |
| 113 | 114 | struct list_head mirror_vms; /* List of VMs mirroring */ |
| 114 | 115 | struct list_head mirror_entry; /* Use as a list entry of mirrors */ |
| @@ -535,12 +535,12 @@ static void vt_flush_tlb_current(struct kvm_vcpu *vcpu) · 534 unmodified lines | ||
| 535 | 535 | vmx_flush_tlb_current(vcpu); |
| 536 | 536 | } |
| 537 | 537 | |
| 538 | −static void vt_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr) | |
| 538 | +static void vt_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr, bool *full) | |
| 539 | 539 | { |
| 540 | 540 | if (is_td_vcpu(vcpu)) |
| 541 | 541 | return; |
| 542 | 542 | |
| 543 | − vmx_flush_tlb_gva(vcpu, addr); | |
| 543 | + vmx_flush_tlb_gva(vcpu, addr, full); | |
| 544 | 544 | } |
| 545 | 545 | |
| 546 | 546 | static void vt_flush_tlb_guest(struct kvm_vcpu *vcpu) |
| @@ -3361,7 +3361,7 @@ void vmx_flush_tlb_current(struct kvm_vcpu *vcpu) · 3360 unmodified lines | ||
| 3361 | 3361 | vpid_sync_context(vmx_get_current_vpid(vcpu)); |
| 3362 | 3362 | } |
| 3363 | 3363 | |
| 3364 | −void vmx_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr) | |
| 3364 | +void vmx_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr, bool *full) | |
| 3365 | 3365 | { |
| 3366 | 3366 | /* |
| 3367 | 3367 | * vpid_sync_vcpu_addr() is a nop if vpid==0, see the comment in |
| @@ -101,7 +101,7 @@ static __always_inline unsigned long __vmcs_readl(unsigned long field) · 100 unmodified lines | ||
| 101 | 101 | |
| 102 | 102 | : [output] "=r" (value) |
| 103 | 103 | : [field] "r" (field) |
| 104 | − : "cc" | |
| 104 | + : "cc", "memory" | |
| 105 | 105 | : do_fail, do_exception); |
| 106 | 106 | |
| 107 | 107 | return value; |
| @@ -145,7 +145,7 @@ do_exception: · 37 unmodified lines | ||
| 145 | 145 | |
| 146 | 146 | : ASM_CALL_CONSTRAINT, [output] "=&r" (value) |
| 147 | 147 | : [field] "r" (field) |
| 148 | − : "cc"); | |
| 148 | + : "cc", "memory"); | |
| 149 | 149 | return value; |
| 150 | 150 | |
| 151 | 151 | #endif /* CONFIG_CC_HAS_ASM_GOTO_OUTPUT */ |
| @@ -192,7 +192,7 @@ do { \ · 40 unmodified lines | ||
| 192 | 192 | asm goto("1: " __stringify(insn) " %0\n\t" \ |
| 193 | 193 | "jna %l[error]\n\t" \ |
| 194 | 194 | _ASM_EXTABLE(1b, %l[fault]) \ |
| 195 | − : : op1 : "cc" : error, fault); \ | |
| 195 | + : : op1 : "cc", "memory" : error, fault); \ | |
| 196 | 196 | return; \ |
| 197 | 197 | error: \ |
| 198 | 198 | instrumentation_begin(); \ |
| @@ -208,7 +208,7 @@ do { \ · 9 unmodified lines | ||
| 208 | 208 | asm goto("1: " __stringify(insn) " %1, %0\n\t" \ |
| 209 | 209 | "jna %l[error]\n\t" \ |
| 210 | 210 | _ASM_EXTABLE(1b, %l[fault]) \ |
| 211 | − : : op1, op2 : "cc" : error, fault); \ | |
| 211 | + : : op1, op2 : "cc", "memory" : error, fault);\ | |
| 212 | 212 | return; \ |
| 213 | 213 | error: \ |
| 214 | 214 | instrumentation_begin(); \ |
| @@ -82,7 +82,7 @@ void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags); · 81 unmodified lines | ||
| 82 | 82 | bool vmx_get_if_flag(struct kvm_vcpu *vcpu); |
| 83 | 83 | void vmx_flush_tlb_all(struct kvm_vcpu *vcpu); |
| 84 | 84 | void vmx_flush_tlb_current(struct kvm_vcpu *vcpu); |
| 85 | −void vmx_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr); | |
| 85 | +void vmx_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr, bool *full); | |
| 86 | 86 | void vmx_flush_tlb_guest(struct kvm_vcpu *vcpu); |
| 87 | 87 | void vmx_set_interrupt_shadow(struct kvm_vcpu *vcpu, int mask); |
| 88 | 88 | u32 vmx_get_interrupt_shadow(struct kvm_vcpu *vcpu); |
| @@ -13429,9 +13429,15 @@ void kvm_arch_pre_destroy_vm(struct kvm *kvm) · 13428 unmodified lines | ||
| 13429 | 13429 | * iterating over vCPUs in a different task while vCPUs are being freed |
| 13430 | 13430 | * is unsafe, i.e. will lead to use-after-free. The PIT also needs to |
| 13431 | 13431 | * be stopped before IRQ routing is freed. |
| 13432 | + * | |
| 13433 | + * Do NOT free the in-kernel PIC or I/O APIC here (but as above, make | |
| 13434 | + * sure to flush any background work), as KVM expects interrupt routing | |
| 13435 | + * structures to be valid until vCPUs are destroyed. | |
| 13432 | 13436 | */ |
| 13433 | 13437 | #ifdef CONFIG_KVM_IOAPIC |
| 13434 | 13438 | kvm_free_pit(kvm); |
| 13439 | + if (kvm->arch.vioapic) | |
| 13440 | + cancel_delayed_work_sync(&kvm->arch.vioapic->eoi_inject); | |
| 13435 | 13441 | #endif |
| 13436 | 13442 | |
| 13437 | 13443 | kvm_mmu_pre_destroy_vm(kvm); |
| @@ -207,10 +207,24 @@ __EXPORT_THUNK(srso_alias_untrain_ret) · 206 unmodified lines | ||
| 207 | 207 | |
| 208 | 208 | .pushsection .text..__x86.rethunk_safe |
| 209 | 209 | SYM_CODE_START_NOALIGN(srso_alias_safe_ret) |
| 210 | + | |
| 211 | + /* | |
| 212 | + * Tell objtool that those are not function pointers referenced by | |
| 213 | + * __HANDLE_INTR_SAFERET(). Below too. | |
| 214 | + */ | |
| 215 | + ANNOTATE_NOENDBR | |
| 216 | + | |
| 217 | + /* | |
| 218 | + * Safe-RET sequence. If you need to change it, adjust | |
| 219 | + * handle_interrupted_saferet() too. | |
| 220 | + */ | |
| 210 | 221 | lea 8(%_ASM_SP), %_ASM_SP |
| 211 | 222 | UNWIND_HINT_FUNC |
| 223 | + | |
| 224 | + ANNOTATE_NOENDBR | |
| 212 | 225 | ANNOTATE_UNRET_SAFE |
| 213 | 226 | ret |
| 227 | + /* End of Safe-RET sequence */ | |
| 214 | 228 | int3 |
| 215 | 229 | SYM_FUNC_END(srso_alias_safe_ret) |
| 216 | 230 | |
| @@ -245,8 +259,14 @@ SYM_CODE_START_LOCAL_NOALIGN(srso_untrain_ret) · 28 unmodified lines | ||
| 245 | 259 | * the stack. |
| 246 | 260 | */ |
| 247 | 261 | SYM_INNER_LABEL(srso_safe_ret, SYM_L_GLOBAL) |
| 262 | + /* | |
| 263 | + * Safe-RET sequence. If you need to change it, adjust | |
| 264 | + * handle_interrupted_saferet() too. | |
| 265 | + */ | |
| 248 | 266 | lea 8(%_ASM_SP), %_ASM_SP |
| 249 | 267 | ret |
| 268 | + /* End of Safe-RET sequence */ | |
| 269 | + | |
| 250 | 270 | int3 |
| 251 | 271 | int3 |
| 252 | 272 | /* end of movabs */ |
| @@ -1281,14 +1281,18 @@ static void disk_release(struct device *dev) · 1280 unmodified lines | ||
| 1281 | 1281 | /* |
| 1282 | 1282 | * To undo the all initialization from blk_mq_init_allocated_queue in |
| 1283 | 1283 | * case of a probe failure where add_disk is never called we have to |
| 1284 | − * call blk_mq_exit_queue here. We can't do this for the more common | |
| 1285 | − * teardown case (yet) as the tagset can be gone by the time the disk | |
| 1286 | − * is released once it was added. | |
| 1284 | + * call blk_mq_exit_queue here, after stopping the timer and work items | |
| 1285 | + * that I/O issued before add_disk may have left pending. We can't do | |
| 1286 | + * this for the more common teardown case (yet) as the tagset can be | |
| 1287 | + * gone by the time the disk is released once it was added. | |
| 1287 | 1288 | */ |
| 1288 | 1289 | if (queue_is_mq(disk->queue) && |
| 1289 | 1290 | test_bit(GD_OWNS_QUEUE, &disk->state) && |
| 1290 | − !test_bit(GD_ADDED, &disk->state)) | |
| 1291 | + !test_bit(GD_ADDED, &disk->state)) { | |
| 1292 | + blk_sync_queue(disk->queue); | |
| 1293 | + blk_mq_cancel_work_sync(disk->queue); | |
| 1291 | 1294 | blk_mq_exit_queue(disk->queue); |
| 1295 | + } | |
| 1292 | 1296 | |
| 1293 | 1297 | blkcg_exit_disk(disk); |
| 1294 | 1298 | |
| @@ -436,6 +436,23 @@ static void amdxdna_gem_dev_obj_free(struct drm_gem_object *gobj) · 435 unmodified lines | ||
| 436 | 436 | amdxdna_gem_destroy_obj(abo); |
| 437 | 437 | } |
| 438 | 438 | |
| 439 | +static void amdxdna_mark_mapp_invalid(struct amdxdna_gem_obj *abo, | |
| 440 | + struct vm_area_struct *vma) | |
| 441 | +{ | |
| 442 | + struct amdxdna_dev *xdna = to_xdna_dev(to_gobj(abo)->dev); | |
| 443 | + struct amdxdna_umap *mapp; | |
| 444 | + | |
| 445 | + down_write(&xdna->notifier_lock); | |
| 446 | + abo->mem.map_invalid = true; | |
| 447 | + list_for_each_entry(mapp, &abo->mem.umap_list, node) { | |
| 448 | + if (compare_range(mapp, vma->vm_mm, vma->vm_start, vma->vm_end)) { | |
| 449 | + mapp->invalid = true; | |
| 450 | + break; | |
| 451 | + } | |
| 452 | + } | |
| 453 | + up_write(&xdna->notifier_lock); | |
| 454 | +} | |
| 455 | + | |
| 439 | 456 | static int amdxdna_insert_pages(struct amdxdna_gem_obj *abo, |
| 440 | 457 | struct vm_area_struct *vma) |
| 441 | 458 | { |
| @@ -450,26 +467,17 @@ static int amdxdna_insert_pages(struct amdxdna_gem_obj *abo, · 8 unmodified lines | ||
| 450 | 467 | XDNA_ERR(xdna, "Failed shmem mmap %d", ret); |
| 451 | 468 | return ret; |
| 452 | 469 | } |
| 453 | − | |
| 454 | − /* The buffer is based on memory pages. Fix the flag. */ | |
| 455 | − vm_flags_mod(vma, VM_MIXEDMAP, VM_PFNMAP); | |
| 456 | − ret = vm_insert_pages(vma, vma->vm_start, abo->base.pages, | |
| 457 | − &num_pages); | |
| 470 | + } else { | |
| 471 | + vma->vm_private_data = NULL; | |
| 472 | + vma->vm_ops = NULL; | |
| 473 | + ret = dma_buf_mmap(abo->dma_buf, vma, 0); | |
| 458 | 474 | if (ret) { |
| 459 | − XDNA_ERR(xdna, "Failed insert pages %d", ret); | |
| 460 | − vma->vm_ops->close(vma); | |
| 475 | + XDNA_ERR(xdna, "Failed to mmap dma buf %d", ret); | |
| 461 | 476 | return ret; |
| 462 | 477 | } |
| 463 | 478 | |
| 464 | − return 0; | |
| 465 | − } | |
| 466 | − | |
| 467 | − vma->vm_private_data = NULL; | |
| 468 | − vma->vm_ops = NULL; | |
| 469 | − ret = dma_buf_mmap(abo->dma_buf, vma, 0); | |
| 470 | − if (ret) { | |
| 471 | − XDNA_ERR(xdna, "Failed to mmap dma buf %d", ret); | |
| 472 | − return ret; | |
| 479 | + /* Drop the reference drm_gem_mmap_obj() acquired.*/ | |
| 480 | + drm_gem_object_put(to_gobj(abo)); | |
| 473 | 481 | } |
| 474 | 482 | |
| 475 | 483 | do { |
| @@ -478,17 +486,14 @@ static int amdxdna_insert_pages(struct amdxdna_gem_obj *abo, · 2 unmodified lines | ||
| 478 | 486 | fault_ret = handle_mm_fault(vma, vma->vm_start + offset, |
| 479 | 487 | FAULT_FLAG_WRITE, NULL); |
| 480 | 488 | if (fault_ret & VM_FAULT_ERROR) { |
| 481 | − vma->vm_ops->close(vma); | |
| 482 | 489 | XDNA_ERR(xdna, "Fault in page failed"); |
| 483 | − return -EFAULT; | |
| 490 | + amdxdna_mark_mapp_invalid(abo, vma); | |
| 491 | + break; | |
| 484 | 492 | } |
| 485 | 493 | |
| 486 | 494 | offset += PAGE_SIZE; |
| 487 | 495 | } while (--num_pages); |
| 488 | 496 | |
| 489 | − /* Drop the reference drm_gem_mmap_obj() acquired.*/ | |
| 490 | − drm_gem_object_put(to_gobj(abo)); | |
| 491 | − | |
| 492 | 497 | return 0; |
| 493 | 498 | } |
| 494 | 499 | |
| @@ -786,7 +786,7 @@ static int encode_message(struct qaic_device *qdev, struct manage_msg *user_msg, · 785 unmodified lines | ||
| 786 | 786 | break; |
| 787 | 787 | } |
| 788 | 788 | trans_hdr = (struct qaic_manage_trans_hdr *)(user_msg->data + user_len); |
| 789 | − if (trans_hdr->len < sizeof(trans_hdr) || | |
| 789 | + if (trans_hdr->len < sizeof(*trans_hdr) || | |
| 790 | 790 | size_add(user_len, trans_hdr->len) > user_msg->len) { |
| 791 | 791 | ret = -EINVAL; |
| 792 | 792 | break; |
| @@ -475,17 +475,29 @@ bool acpi_cpc_valid(void) · 474 unmodified lines | ||
| 475 | 475 | } |
| 476 | 476 | EXPORT_SYMBOL_GPL(acpi_cpc_valid); |
| 477 | 477 | |
| 478 | −bool cppc_allow_fast_switch(void) | |
| 478 | +bool cppc_allow_fast_switch(const struct cpumask *cpus) | |
| 479 | 479 | { |
| 480 | − struct cpc_register_resource *desired_reg; | |
| 480 | + struct cpc_register_resource *desired_reg, *min_reg, *max_reg; | |
| 481 | 481 | struct cpc_desc *cpc_ptr; |
| 482 | 482 | int cpu; |
| 483 | 483 | |
| 484 | − for_each_online_cpu(cpu) { | |
| 484 | + for_each_cpu(cpu, cpus) { | |
| 485 | 485 | cpc_ptr = per_cpu(cpc_desc_ptr, cpu); |
| 486 | + if (!cpc_ptr) | |
| 487 | + return false; | |
| 486 | 488 | desired_reg = &cpc_ptr->cpc_regs[DESIRED_PERF]; |
| 487 | − if (!CPC_IN_SYSTEM_MEMORY(desired_reg) && | |
| 488 | − !CPC_IN_SYSTEM_IO(desired_reg)) | |
| 489 | + min_reg = &cpc_ptr->cpc_regs[MIN_PERF]; | |
| 490 | + max_reg = &cpc_ptr->cpc_regs[MAX_PERF]; | |
| 491 | + | |
| 492 | + if (!CPC_SUPPORTED(desired_reg) || | |
| 493 | + (!CPC_IN_SYSTEM_MEMORY(desired_reg) && | |
| 494 | + !CPC_IN_SYSTEM_IO(desired_reg)) || | |
| 495 | + (CPC_SUPPORTED(min_reg) && | |
| 496 | + !CPC_IN_SYSTEM_MEMORY(min_reg) && | |
| 497 | + !CPC_IN_SYSTEM_IO(min_reg)) || | |
| 498 | + (CPC_SUPPORTED(max_reg) && | |
| 499 | + !CPC_IN_SYSTEM_MEMORY(max_reg) && | |
| 500 | + !CPC_IN_SYSTEM_IO(max_reg))) | |
| 489 | 501 | return false; |
| 490 | 502 | } |
| 491 | 503 | |
| @@ -1951,16 +1963,17 @@ int cppc_set_perf(int cpu, struct cppc_perf_ctrls *perf_ctrls) · 1459 unmodified lines | ||
| 1951 | 1963 | cpc_desc->write_cmd_status = 0; |
| 1952 | 1964 | } |
| 1953 | 1965 | |
| 1954 | − cpc_write(cpu, desired_reg, perf_ctrls->desired_perf); | |
| 1966 | + if (CPC_SUPPORTED(desired_reg)) | |
| 1967 | + cpc_write(cpu, desired_reg, perf_ctrls->desired_perf); | |
| 1955 | 1968 | |
| 1956 | 1969 | /* |
| 1957 | 1970 | * Only write if min_perf and max_perf not zero. Some drivers pass zero |
| 1958 | 1971 | * value to min and max perf, but they don't mean to set the zero value, |
| 1959 | 1972 | * they just don't want to write to those registers. |
| 1960 | 1973 | */ |
| 1961 | − if (perf_ctrls->min_perf) | |
| 1974 | + if (perf_ctrls->min_perf && CPC_SUPPORTED(min_perf_reg)) | |
| 1962 | 1975 | cpc_write(cpu, min_perf_reg, perf_ctrls->min_perf); |
| 1963 | − if (perf_ctrls->max_perf) | |
| 1976 | + if (perf_ctrls->max_perf && CPC_SUPPORTED(max_perf_reg)) | |
| 1964 | 1977 | cpc_write(cpu, max_perf_reg, perf_ctrls->max_perf); |
| 1965 | 1978 | |
| 1966 | 1979 | if (CPC_IN_PCC(desired_reg) || CPC_IN_PCC(min_perf_reg) || CPC_IN_PCC(max_perf_reg)) |
| @@ -1586,6 +1586,10 @@ impl Process { · 1585 unmodified lines | ||
| 1586 | 1586 | cmd: u32, |
| 1587 | 1587 | reader: &mut UserSliceReader, |
| 1588 | 1588 | ) -> Result { |
| 1589 | + if cmd == uapi::BINDER_FREEZE { | |
| 1590 | + return ioctl_freeze(reader); | |
| 1591 | + } | |
| 1592 | + | |
| 1589 | 1593 | let thread = this.get_current_thread()?; |
| 1590 | 1594 | match cmd { |
| 1591 | 1595 | uapi::BINDER_SET_MAX_THREADS => this.set_max_threads(reader.read()?), |
| @@ -1597,7 +1601,6 @@ impl Process { · 5 unmodified lines | ||
| 1597 | 1601 | uapi::BINDER_ENABLE_ONEWAY_SPAM_DETECTION => { |
| 1598 | 1602 | this.set_oneway_spam_detection_enabled(reader.read()?) |
| 1599 | 1603 | } |
| 1600 | − uapi::BINDER_FREEZE => ioctl_freeze(reader)?, | |
| 1601 | 1604 | _ => return Err(EINVAL), |
| 1602 | 1605 | } |
| 1603 | 1606 | Ok(()) |
| @@ -1612,15 +1615,16 @@ impl Process { · 8 unmodified lines | ||
| 1612 | 1615 | cmd: u32, |
| 1613 | 1616 | data: UserSlice, |
| 1614 | 1617 | ) -> Result { |
| 1615 | − let thread = this.get_current_thread()?; | |
| 1616 | 1618 | let blocking = (file.flags() & file::flags::O_NONBLOCK) == 0; |
| 1617 | 1619 | match cmd { |
| 1618 | − uapi::BINDER_WRITE_READ => thread.write_read(data, blocking)?, | |
| 1620 | + uapi::BINDER_WRITE_READ => this.get_current_thread()?.write_read(data, blocking)?, | |
| 1619 | 1621 | uapi::BINDER_GET_NODE_DEBUG_INFO => this.get_node_debug_info(data)?, |
| 1620 | 1622 | uapi::BINDER_GET_NODE_INFO_FOR_REF => this.get_node_info_from_ref(data)?, |
| 1621 | 1623 | uapi::BINDER_VERSION => this.version(data)?, |
| 1622 | 1624 | uapi::BINDER_GET_FROZEN_INFO => get_frozen_status(data)?, |
| 1623 | − uapi::BINDER_GET_EXTENDED_ERROR => thread.get_extended_error(data)?, | |
| 1625 | + uapi::BINDER_GET_EXTENDED_ERROR => { | |
| 1626 | + this.get_current_thread()?.get_extended_error(data)? | |
| 1627 | + } | |
| 1624 | 1628 | _ => return Err(EINVAL), |
| 1625 | 1629 | } |
| 1626 | 1630 | Ok(()) |
| @@ -211,7 +211,7 @@ static int ceva_ahci_platform_enable_resources(struct ahci_host_priv *hpriv) · 210 unmodified lines | ||
| 211 | 211 | |
| 212 | 212 | rc = phy_init(hpriv->phys[i]); |
| 213 | 213 | if (rc) |
| 214 | − goto disable_rsts; | |
| 214 | + goto exit_phys; | |
| 215 | 215 | } |
| 216 | 216 | |
| 217 | 217 | /* De-assert the controller reset */ |
| @@ -230,14 +230,24 @@ static int ceva_ahci_platform_enable_resources(struct ahci_host_priv *hpriv) · 12 unmodified lines | ||
| 230 | 230 | |
| 231 | 231 | return 0; |
| 232 | 232 | |
| 233 | −disable_rsts: | |
| 234 | − ahci_platform_deassert_rsts(hpriv); | |
| 235 | − | |
| 236 | 233 | disable_phys: |
| 237 | 234 | while (--i >= 0) { |
| 235 | + if (ahci_ignore_port(hpriv, i)) | |
| 236 | + continue; | |
| 237 | + | |
| 238 | 238 | phy_power_off(hpriv->phys[i]); |
| 239 | 239 | phy_exit(hpriv->phys[i]); |
| 240 | 240 | } |
| 241 | + ahci_platform_assert_rsts(hpriv); | |
| 242 | + goto disable_clks; | |
| 243 | + | |
| 244 | +exit_phys: | |
| 245 | + while (--i >= 0) { | |
| 246 | + if (ahci_ignore_port(hpriv, i)) | |
| 247 | + continue; | |
| 248 | + | |
| 249 | + phy_exit(hpriv->phys[i]); | |
| 250 | + } | |
| 241 | 251 | |
| 242 | 252 | disable_clks: |
| 243 | 253 | ahci_platform_disable_clks(hpriv); |
| @@ -4413,6 +4413,16 @@ static const struct ata_dev_quirks_entry __ata_dev_quirks[] = { · 4412 unmodified lines | ||
| 4413 | 4413 | { "WDC WD3000JD-*", NULL, ATA_QUIRK_WD_BROKEN_LPM }, |
| 4414 | 4414 | { "WDC WD3200JD-*", NULL, ATA_QUIRK_WD_BROKEN_LPM }, |
| 4415 | 4415 | |
| 4416 | + /* | |
| 4417 | + * WD drives with LPM issues (irrespective of supported SATA speeds). | |
| 4418 | + * (Unlike ATA_QUIRK_WD_BROKEN_LPM, which is only applied if the drive | |
| 4419 | + * exposes SATA Gen1 speed support, and SATA Gen1 speed support only.) | |
| 4420 | + */ | |
| 4421 | + { "WDC WD100EFGX-68CPLN0", NULL, ATA_QUIRK_NOLPM }, | |
| 4422 | + { "WDC WD102KFBX-68M95N0", NULL, ATA_QUIRK_NOLPM }, | |
| 4423 | + { "WDC WD141KFGX-68FH9N0", NULL, ATA_QUIRK_NOLPM }, | |
| 4424 | + { "WD Green 2.5 480GB", NULL, ATA_QUIRK_NOLPM }, | |
| 4425 | + | |
| 4416 | 4426 | /* |
| 4417 | 4427 | * This sata dom device goes on a walkabout when the ATA_LOG_DIRECTORY |
| 4418 | 4428 | * log page is accessed. Ensure we never ask for this log page with |
| @@ -106,6 +106,12 @@ static const unsigned int ata_eh_flush_timeouts[] = { · 105 unmodified lines | ||
| 106 | 106 | UINT_MAX, |
| 107 | 107 | }; |
| 108 | 108 | |
| 109 | +static const unsigned int ata_eh_standby_timeouts[] = { | |
| 110 | + 15000, /* Some drives may be slow to standby */ | |
| 111 | + /* but don't hold up a suspend too long waiting for them */ | |
| 112 | + UINT_MAX, | |
| 113 | +}; | |
| 114 | + | |
| 109 | 115 | static const unsigned int ata_eh_other_timeouts[] = { |
| 110 | 116 | 5000, /* same rationale as identify timeout */ |
| 111 | 117 | 10000, /* ditto */ |
| @@ -147,6 +153,8 @@ ata_eh_cmd_timeout_table[ATA_EH_CMD_TIMEOUT_TABLE_SIZE] = { · 35 unmodified lines | ||
| 147 | 153 | .timeouts = ata_eh_other_timeouts, }, |
| 148 | 154 | { .commands = CMDS(ATA_CMD_FLUSH, ATA_CMD_FLUSH_EXT), |
| 149 | 155 | .timeouts = ata_eh_flush_timeouts }, |
| 156 | + { .commands = CMDS(ATA_CMD_STANDBYNOW1), | |
| 157 | + .timeouts = ata_eh_standby_timeouts }, | |
| 150 | 158 | { .commands = CMDS(ATA_CMD_VERIFY), |
| 151 | 159 | .timeouts = ata_eh_reset_timeouts }, |
| 152 | 160 | }; |
| @@ -650,29 +658,12 @@ int ata_scsi_cmd_error_handler(struct Scsi_Host *host, struct ata_port *ap, · 497 unmodified lines | ||
| 650 | 658 | set_host_byte(scmd, DID_OK); |
| 651 | 659 | |
| 652 | 660 | ata_qc_for_each_raw(ap, qc, i) { |
| 653 | − if (qc->scsicmd != scmd) | |
| 654 | − continue; | |
| 655 | − if ((qc->flags & ATA_QCFLAG_ACTIVE) || | |
| 656 | − qc == qc->dev->link->deferred_qc) | |
| 661 | + if (qc->scsicmd == scmd && | |
| 662 | + qc->flags & ATA_QCFLAG_ACTIVE) | |
| 657 | 663 | break; |
| 658 | 664 | } |
| 659 | 665 | |
| 660 | − if (i < ATA_MAX_QUEUE && qc == qc->dev->link->deferred_qc) { | |
| 661 | − /* | |
| 662 | − * This is a deferred command that timed out while | |
| 663 | − * waiting for the command queue to drain. Since the qc | |
| 664 | − * is not active yet (deferred_qc is still set, so the | |
| 665 | − * deferred qc work has not issued the command yet), | |
| 666 | − * simply signal the timeout by finishing the SCSI | |
| 667 | − * command and clear the deferred qc to prevent the | |
| 668 | − * deferred qc work from issuing this qc. | |
| 669 | − */ | |
| 670 | − WARN_ON_ONCE(qc->flags & ATA_QCFLAG_ACTIVE); | |
| 671 | − qc->dev->link->deferred_qc = NULL; | |
| 672 | − cancel_work(&qc->dev->link->deferred_qc_work); | |
| 673 | − set_host_byte(scmd, DID_TIME_OUT); | |
| 674 | − scsi_eh_finish_cmd(scmd, &ap->eh_done_q); | |
| 675 | − } else if (i < ATA_MAX_QUEUE) { | |
| 666 | + if (i < ATA_MAX_QUEUE) { | |
| 676 | 667 | /* the scmd has an associated qc */ |
| 677 | 668 | if (!(qc->flags & ATA_QCFLAG_EH)) { |
| 678 | 669 | /* which hasn't failed yet, timeout */ |
| @@ -948,10 +939,10 @@ static void ata_eh_set_pending(struct ata_port *ap, bool fastdrain) · 269 unmodified lines | ||
| 948 | 939 | ap->pflags |= ATA_PFLAG_EH_PENDING; |
| 949 | 940 | |
| 950 | 941 | /* |
| 951 | − * If we have a deferred qc, requeue it so that it is retried once EH | |
| 952 | − * completes. | |
| 942 | + * If we have deferred QCs, requeue them so that the SCSI EH task can | |
| 943 | + * run. | |
| 953 | 944 | */ |
| 954 | − ata_scsi_requeue_deferred_qc(ap); | |
| 945 | + ata_scsi_requeue_deferred_qc(ap, NULL); | |
| 955 | 946 | |
| 956 | 947 | if (!fastdrain) |
| 957 | 948 | return; |
| @@ -913,7 +913,7 @@ static bool ata_scsi_lpm_supported(struct ata_port *ap) · 912 unmodified lines | ||
| 913 | 913 | return false; |
| 914 | 914 | |
| 915 | 915 | ata_for_each_link(link, ap, EDGE) { |
| 916 | − ata_for_each_dev(dev, &ap->link, ENABLED) { | |
| 916 | + ata_for_each_dev(dev, link, ENABLED) { | |
| 917 | 917 | if (dev->quirks & ATA_QUIRK_NOLPM) |
| 918 | 918 | return false; |
| 919 | 919 | } |
| @@ -1685,26 +1685,80 @@ void ata_scsi_deferred_qc_work(struct work_struct *work) · 1684 unmodified lines | ||
| 1685 | 1685 | spin_unlock_irqrestore(ap->lock, flags); |
| 1686 | 1686 | } |
| 1687 | 1687 | |
| 1688 | −void ata_scsi_requeue_deferred_qc(struct ata_port *ap) | |
| 1688 | +enum scsi_timeout_action ata_scsi_requeue_deferred_qc(struct ata_port *ap, | |
| 1689 | + struct scsi_cmnd *timedout_scmd) | |
| 1689 | 1690 | { |
| 1691 | + enum scsi_timeout_action action = SCSI_EH_NOT_HANDLED; | |
| 1692 | + struct ata_queued_cmd *qc; | |
| 1690 | 1693 | struct ata_link *link; |
| 1694 | + u32 host_byte; | |
| 1691 | 1695 | |
| 1692 | 1696 | lockdep_assert_held(ap->lock); |
| 1693 | 1697 | |
| 1694 | 1698 | /* |
| 1695 | − * If we have a deferred qc when a reset occurs or NCQ commands fail, | |
| 1696 | − * do not try to be smart about what to do with this deferred command | |
| 1697 | − * and simply requeue it by completing it with DID_REQUEUE. | |
| 1699 | + * If we have deferred QCs when a reset, a timeout or an NCQ command | |
| 1700 | + * fails, do not try to be smart about what to do with the deferred | |
| 1701 | + * commands and simply terminate them and let the SCSI layer decide | |
| 1702 | + * what to do. | |
| 1698 | 1703 | */ |
| 1699 | 1704 | ata_for_each_link(link, ap, PMP_FIRST) { |
| 1700 | − struct ata_queued_cmd *qc = link->deferred_qc; | |
| 1705 | + qc = link->deferred_qc; | |
| 1706 | + if (!qc) | |
| 1707 | + continue; | |
| 1701 | 1708 | |
| 1702 | − if (qc) { | |
| 1703 | − link->deferred_qc = NULL; | |
| 1704 | − cancel_work(&link->deferred_qc_work); | |
| 1705 | − ata_scsi_qc_done(qc, true, DID_REQUEUE << 16); | |
| 1709 | + /* | |
| 1710 | + * Clear the deferred QC so that the deferred work does not try | |
| 1711 | + * to issue it. | |
| 1712 | + */ | |
| 1713 | + link->deferred_qc = NULL; | |
| 1714 | + cancel_work(&link->deferred_qc_work); | |
| 1715 | + | |
| 1716 | + /* | |
| 1717 | + * We are going to complete some scsi command, either with | |
| 1718 | + * DID_TIME_OUT if the command timed out while waiting for being | |
| 1719 | + * issued, or with DID_REQUEUE if another command timed out or | |
| 1720 | + * we had a failed command. However, the block layer may re-issue | |
| 1721 | + * these commands immediately, keeping the scsi host busy and | |
| 1722 | + * thus preventing the SCSI EH task from running. | |
| 1723 | + * So schedule EH on the port to prevent accepting new commands | |
| 1724 | + * until everything is sorted out with the error or timeout that | |
| 1725 | + * got us here in the first place. Note that we set EH pending | |
| 1726 | + * on the port before calling ata_port_schedule_eh() so that we | |
| 1727 | + * do not reenter this function from ata_eh_set_pending() with | |
| 1728 | + * timedout_scmd being NULL and erroneously retry deferred QCs | |
| 1729 | + * that have timed out on other links. | |
| 1730 | + */ | |
| 1731 | + if (!ata_port_eh_scheduled(ap)) { | |
| 1732 | + ap->pflags |= ATA_PFLAG_EH_PENDING; | |
| 1733 | + ata_port_schedule_eh(ap); | |
| 1706 | 1734 | } |
| 1735 | + | |
| 1736 | + /* | |
| 1737 | + * If we are being called from scsi_timeout(), then we have a | |
| 1738 | + * non-NULL timedout_scmd. If the timed out command is for a | |
| 1739 | + * deferred QC, terminate that deferred QC with DID_TIME_OUT and | |
| 1740 | + * requeue all other deferred QCs. In this case we need to | |
| 1741 | + * return SCSI_EH_DONE, because the timed out command was | |
| 1742 | + * handled. | |
| 1743 | + * If the timed out command is not for a deferred QC, we need to | |
| 1744 | + * requeue all deferred QCs, and return SCSI_EH_NOT_HANDLED so | |
| 1745 | + * that the timed out command gets added to the EH work queue | |
| 1746 | + * with scsi_eh_scmd_add(), for later handling with libata EH | |
| 1747 | + * ata_scsi_cmd_error_handler(). | |
| 1748 | + * If timedout_scmd is NULL, we simply need to requeue all | |
| 1749 | + * deferred QCs and the return value does not matter as we were | |
| 1750 | + * not called from scsi_timeout(). | |
| 1751 | + */ | |
| 1752 | + if (timedout_scmd && qc->scsicmd == timedout_scmd) { | |
| 1753 | + host_byte = DID_TIME_OUT; | |
| 1754 | + action = SCSI_EH_DONE; | |
| 1755 | + } else { | |
| 1756 | + host_byte = DID_REQUEUE; | |
| 1757 | + } | |
| 1758 | + ata_scsi_qc_done(qc, true, host_byte << 16); | |
| 1707 | 1759 | } |
| 1760 | + | |
| 1761 | + return action; | |
| 1708 | 1762 | } |
| 1709 | 1763 | |
| 1710 | 1764 | static void ata_scsi_schedule_deferred_qc(struct ata_link *link) |
| @@ -1723,13 +1777,42 @@ static void ata_scsi_schedule_deferred_qc(struct ata_link *link) · 12 unmodified lines | ||
| 1723 | 1777 | return; |
| 1724 | 1778 | |
| 1725 | 1779 | if (ata_port_eh_scheduled(ap)) { |
| 1726 | − ata_scsi_requeue_deferred_qc(ap); | |
| 1780 | + ata_scsi_requeue_deferred_qc(ap, NULL); | |
| 1727 | 1781 | return; |
| 1728 | 1782 | } |
| 1729 | 1783 | if (!ap->ops->qc_defer(qc)) |
| 1730 | 1784 | queue_work(system_highpri_wq, &link->deferred_qc_work); |
| 1731 | 1785 | } |
| 1732 | 1786 | |
| 1787 | +enum scsi_timeout_action ata_scsi_retry_deferred_qc(struct ata_port *ap, | |
| 1788 | + struct scsi_cmnd *scmd) | |
| 1789 | +{ | |
| 1790 | + enum scsi_timeout_action action; | |
| 1791 | + unsigned long flags; | |
| 1792 | + | |
| 1793 | + spin_lock_irqsave(ap->lock, flags); | |
| 1794 | + action = ata_scsi_requeue_deferred_qc(ap, scmd); | |
| 1795 | + spin_unlock_irqrestore(ap->lock, flags); | |
| 1796 | + | |
| 1797 | + return action; | |
| 1798 | +} | |
| 1799 | +EXPORT_SYMBOL_GPL(ata_scsi_retry_deferred_qc); | |
| 1800 | + | |
| 1801 | +enum scsi_timeout_action ata_scsi_eh_timed_out(struct scsi_cmnd *scmd) | |
| 1802 | +{ | |
| 1803 | + struct ata_port *ap = ata_shost_to_port(scmd->device->host); | |
| 1804 | + | |
| 1805 | + /* | |
| 1806 | + * ata_scsi_cmd_error_handler() takes care of commands that timed out | |
| 1807 | + * while executing. However, if we have deferred QCs while a timeout | |
| 1808 | + * triggers, we must requeue these commands for retry so that we do not | |
| 1809 | + * unnecessarily delay starting the SCSI EH task until these deferred | |
| 1810 | + * commands also time out. | |
| 1811 | + */ | |
| 1812 | + return ata_scsi_retry_deferred_qc(ap, scmd); | |
| 1813 | +} | |
| 1814 | +EXPORT_SYMBOL_GPL(ata_scsi_eh_timed_out); | |
| 1815 | + | |
| 1733 | 1816 | static void ata_scsi_qc_complete(struct ata_queued_cmd *qc) |
| 1734 | 1817 | { |
| 1735 | 1818 | struct ata_link *link = qc->dev->link; |
| @@ -2911,6 +2994,7 @@ static void atapi_fixup_inquiry(struct scsi_cmnd *cmd) · 1175 unmodified lines | ||
| 2911 | 2994 | |
| 2912 | 2995 | static void atapi_qc_complete(struct ata_queued_cmd *qc) |
| 2913 | 2996 | { |
| 2997 | + struct ata_link *link = qc->dev->link; | |
| 2914 | 2998 | struct scsi_cmnd *cmd = qc->scsicmd; |
| 2915 | 2999 | unsigned int err_mask = qc->err_mask; |
| 2916 | 3000 | |
| @@ -2936,8 +3020,16 @@ static void atapi_qc_complete(struct ata_queued_cmd *qc) · 19 unmodified lines | ||
| 2936 | 3020 | if (qc->cdb[0] == ALLOW_MEDIUM_REMOVAL && qc->dev->sdev) |
| 2937 | 3021 | qc->dev->sdev->locked = 0; |
| 2938 | 3022 | |
| 2939 | − ata_scsi_qc_done(qc, true, SAM_STAT_CHECK_CONDITION); | |
| 2940 | − return; | |
| 3023 | + if (cmd->result) | |
| 3024 | + ata_scsi_qc_done(qc, false, 0); | |
| 3025 | + else | |
| 3026 | + ata_scsi_qc_done(qc, true, SAM_STAT_CHECK_CONDITION); | |
| 3027 | + goto schedule_deferred; | |
| 3028 | + } | |
| 3029 | + | |
| 3030 | + if (cmd->result) { | |
| 3031 | + ata_scsi_qc_done(qc, false, 0); | |
| 3032 | + goto schedule_deferred; | |
| 2941 | 3033 | } |
| 2942 | 3034 | |
| 2943 | 3035 | /* successful completion path */ |
| @@ -2945,6 +3037,9 @@ static void atapi_qc_complete(struct ata_queued_cmd *qc) · 1 unmodified line | ||
| 2945 | 3037 | atapi_fixup_inquiry(cmd); |
| 2946 | 3038 | |
| 2947 | 3039 | ata_scsi_qc_done(qc, true, SAM_STAT_GOOD); |
| 3040 | + | |
| 3041 | +schedule_deferred: | |
| 3042 | + ata_scsi_schedule_deferred_qc(link); | |
| 2948 | 3043 | } |
| 2949 | 3044 | /** |
| 2950 | 3045 | * atapi_xlat - Initialize PACKET taskfile |
| @@ -180,7 +180,8 @@ enum scsi_qc_status __ata_scsi_queuecmd(struct scsi_cmnd *scmd, · 179 unmodified lines | ||
| 180 | 180 | struct ata_port *ap) |
| 181 | 181 | __must_hold(ap->lock); |
| 182 | 182 | void ata_scsi_deferred_qc_work(struct work_struct *work); |
| 183 | −void ata_scsi_requeue_deferred_qc(struct ata_port *ap); | |
| 183 | +enum scsi_timeout_action ata_scsi_requeue_deferred_qc(struct ata_port *ap, | |
| 184 | + struct scsi_cmnd *scmd); | |
| 184 | 185 | |
| 185 | 186 | /* libata-eh.c */ |
| 186 | 187 | extern unsigned int ata_internal_cmd_timeout(struct ata_device *dev, u8 cmd); |
| @@ -264,6 +264,7 @@ static struct ata_port_operations sl82c105_port_ops = { · 263 unmodified lines | ||
| 264 | 264 | static int sl82c105_bridge_revision(struct pci_dev *pdev) |
| 265 | 265 | { |
| 266 | 266 | struct pci_dev *bridge; |
| 267 | + u8 revision; | |
| 267 | 268 | |
| 268 | 269 | /* |
| 269 | 270 | * The bridge should be part of the same device, but function 0. |
| @@ -285,8 +286,9 @@ static int sl82c105_bridge_revision(struct pci_dev *pdev) · 15 unmodified lines | ||
| 285 | 286 | /* |
| 286 | 287 | * We need to find function 0's revision, not function 1 |
| 287 | 288 | */ |
| 289 | + revision = bridge->revision; | |
| 288 | 290 | pci_dev_put(bridge); |
| 289 | − return bridge->revision; | |
| 291 | + return revision; | |
| 290 | 292 | } |
| 291 | 293 | |
| 292 | 294 | static void sl82c105_fixup(struct pci_dev *pdev) |
| @@ -4027,7 +4027,7 @@ static int mv_platform_probe(struct platform_device *pdev) · 4026 unmodified lines | ||
| 4027 | 4027 | /* |
| 4028 | 4028 | * Simple resource validation .. |
| 4029 | 4029 | */ |
| 4030 | − if (unlikely(pdev->num_resources != 1)) { | |
| 4030 | + if (unlikely(pdev->num_resources != 1 && pdev->num_resources != 2)) { | |
| 4031 | 4031 | dev_err(&pdev->dev, "invalid number of resources\n"); |
| 4032 | 4032 | return -EINVAL; |
| 4033 | 4033 | } |
| @@ -4764,6 +4764,15 @@ static int ublk_ctrl_add_dev(const struct ublksrv_ctrl_cmd *header) · 4763 unmodified lines | ||
| 4764 | 4764 | /* update device id */ |
| 4765 | 4765 | ub->dev_info.dev_id = ub->ub_number; |
| 4766 | 4766 | |
| 4767 | + /* | |
| 4768 | + * ->state and ->ublksrv_pid are owned by the driver and only read back | |
| 4769 | + * by userspace, but they come from the copied-in dev_info, so reset | |
| 4770 | + * them. Otherwise a device added with ->state != DEAD looks live while | |
| 4771 | + * ->ub_disk is still NULL. | |
| 4772 | + */ | |
| 4773 | + ub->dev_info.state = UBLK_S_DEV_DEAD; | |
| 4774 | + ub->dev_info.ublksrv_pid = -1; | |
| 4775 | + | |
| 4767 | 4776 | /* |
| 4768 | 4777 | * 64bit flags will be copied back to userspace as feature |
| 4769 | 4778 | * negotiation result, so have to clear flags which driver |
| @@ -3771,6 +3771,9 @@ static int btintel_diagnostics(struct hci_dev *hdev, struct sk_buff *skb) · 3770 unmodified lines | ||
| 3771 | 3771 | { |
| 3772 | 3772 | struct intel_tlv *tlv = (void *)&skb->data[5]; |
| 3773 | 3773 | |
| 3774 | + if (skb->len < 5 + sizeof(*tlv) + sizeof(tlv->val[0])) | |
| 3775 | + goto recv_frame; | |
| 3776 | + | |
| 3774 | 3777 | /* The first event is always an event type TLV */ |
| 3775 | 3778 | if (tlv->type != INTEL_TLV_TYPE_ID) |
| 3776 | 3779 | goto recv_frame; |
| @@ -804,56 +804,44 @@ static int btmtk_usb_uhw_reg_write(struct hci_dev *hdev, u32 reg, u32 val) · 803 unmodified lines | ||
| 804 | 804 | static int btmtk_usb_uhw_reg_read(struct hci_dev *hdev, u32 reg, u32 *val) |
| 805 | 805 | { |
| 806 | 806 | struct btmtk_data *data = hci_get_priv(hdev); |
| 807 | − int pipe, err; | |
| 808 | − void *buf; | |
| 809 | − | |
| 810 | − buf = kzalloc(4, GFP_KERNEL); | |
| 811 | − if (!buf) | |
| 812 | − return -ENOMEM; | |
| 807 | + u8 buf[sizeof(u32)]; | |
| 808 | + int err; | |
| 813 | 809 | |
| 814 | − pipe = usb_rcvctrlpipe(data->udev, 0); | |
| 815 | − err = usb_control_msg(data->udev, pipe, 0x01, | |
| 816 | − 0xDE, | |
| 817 | − reg >> 16, reg & 0xffff, | |
| 818 | − buf, 4, USB_CTRL_GET_TIMEOUT); | |
| 819 | − if (err < 0) { | |
| 810 | + *val = 0; | |
| 811 | + err = usb_control_msg_recv(data->udev, 0, 0x01, | |
| 812 | + 0xDE, | |
| 813 | + reg >> 16, reg & 0xffff, | |
| 814 | + buf, sizeof(buf), USB_CTRL_GET_TIMEOUT, | |
| 815 | + GFP_KERNEL); | |
| 816 | + if (err) { | |
| 820 | 817 | bt_dev_err(hdev, "Failed to read uhw reg(%d)", err); |
| 821 | − goto err_free_buf; | |
| 818 | + return err; | |
| 822 | 819 | } |
| 823 | 820 | |
| 824 | 821 | *val = get_unaligned_le32(buf); |
| 825 | 822 | bt_dev_dbg(hdev, "reg=%x, value=0x%08x", reg, *val); |
| 826 | 823 | |
| 827 | −err_free_buf: | |
| 828 | − kfree(buf); | |
| 829 | − | |
| 830 | − return err; | |
| 824 | + return 0; | |
| 831 | 825 | } |
| 832 | 826 | |
| 833 | 827 | static int btmtk_usb_reg_read(struct hci_dev *hdev, u32 reg, u32 *val) |
| 834 | 828 | { |
| 835 | 829 | struct btmtk_data *data = hci_get_priv(hdev); |
| 836 | − int pipe, err, size = sizeof(u32); | |
| 837 | − void *buf; | |
| 838 | − | |
| 839 | − buf = kzalloc(size, GFP_KERNEL); | |
| 840 | − if (!buf) | |
| 841 | − return -ENOMEM; | |
| 830 | + u8 buf[sizeof(u32)]; | |
| 831 | + int err; | |
| 842 | 832 | |
| 843 | − pipe = usb_rcvctrlpipe(data->udev, 0); | |
| 844 | − err = usb_control_msg(data->udev, pipe, 0x63, | |
| 845 | − USB_TYPE_VENDOR | USB_DIR_IN, | |
| 846 | − reg >> 16, reg & 0xffff, | |
| 847 | − buf, size, USB_CTRL_GET_TIMEOUT); | |
| 833 | + *val = 0; | |
| 834 | + err = usb_control_msg_recv(data->udev, 0, 0x63, | |
| 835 | + USB_TYPE_VENDOR | USB_DIR_IN, | |
| 836 | + reg >> 16, reg & 0xffff, | |
| 837 | + buf, sizeof(buf), USB_CTRL_GET_TIMEOUT, | |
| 838 | + GFP_KERNEL); | |
| 848 | 839 | if (err < 0) |
| 849 | − goto err_free_buf; | |
| 840 | + return err; | |
| 850 | 841 | |
| 851 | 842 | *val = get_unaligned_le32(buf); |
| 852 | 843 | |
| 853 | −err_free_buf: | |
| 854 | − kfree(buf); | |
| 855 | − | |
| 856 | − return err; | |
| 844 | + return 0; | |
| 857 | 845 | } |
| 858 | 846 | |
| 859 | 847 | static int btmtk_usb_id_get(struct hci_dev *hdev, u32 reg, u32 *id) |
| @@ -877,7 +865,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 17 unmodified lines | ||
| 877 | 865 | |
| 878 | 866 | if (dev_id == 0x7922) { |
| 879 | 867 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_SUBSYS_RST, &val); |
| 880 | − if (err < 0) | |
| 868 | + if (err) | |
| 881 | 869 | return err; |
| 882 | 870 | val |= 0x00002020; |
| 883 | 871 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_SUBSYS_RST, val); |
| @@ -887,7 +875,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 3 unmodified lines | ||
| 887 | 875 | if (err < 0) |
| 888 | 876 | return err; |
| 889 | 877 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_SUBSYS_RST, &val); |
| 890 | − if (err < 0) | |
| 878 | + if (err) | |
| 891 | 879 | return err; |
| 892 | 880 | val |= BIT(0); |
| 893 | 881 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_SUBSYS_RST, val); |
| @@ -896,14 +884,14 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 2 unmodified lines | ||
| 896 | 884 | msleep(100); |
| 897 | 885 | } else if (dev_id == 0x7925 || dev_id == 0x6639) { |
| 898 | 886 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val); |
| 899 | − if (err < 0) | |
| 887 | + if (err) | |
| 900 | 888 | return err; |
| 901 | 889 | val |= (1 << 5); |
| 902 | 890 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_RESET_REG_CONNV3, val); |
| 903 | 891 | if (err < 0) |
| 904 | 892 | return err; |
| 905 | 893 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val); |
| 906 | − if (err < 0) | |
| 894 | + if (err) | |
| 907 | 895 | return err; |
| 908 | 896 | val &= 0xFFFF00FF; |
| 909 | 897 | val |= (1 << 13); |
| @@ -914,7 +902,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 4 unmodified lines | ||
| 914 | 902 | if (err < 0) |
| 915 | 903 | return err; |
| 916 | 904 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val); |
| 917 | − if (err < 0) | |
| 905 | + if (err) | |
| 918 | 906 | return err; |
| 919 | 907 | val |= (1 << 0); |
| 920 | 908 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_RESET_REG_CONNV3, val); |
| @@ -924,13 +912,13 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 3 unmodified lines | ||
| 924 | 912 | if (err < 0) |
| 925 | 913 | return err; |
| 926 | 914 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT, &val); |
| 927 | − if (err < 0) | |
| 915 | + if (err) | |
| 928 | 916 | return err; |
| 929 | 917 | err = btmtk_usb_uhw_reg_write(hdev, MTK_UDMA_INT_STA_BT1, 0x000000FF); |
| 930 | 918 | if (err < 0) |
| 931 | 919 | return err; |
| 932 | 920 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT1, &val); |
| 933 | − if (err < 0) | |
| 921 | + if (err) | |
| 934 | 922 | return err; |
| 935 | 923 | msleep(100); |
| 936 | 924 | } else { |
| @@ -940,7 +928,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 3 unmodified lines | ||
| 940 | 928 | if (err < 0) |
| 941 | 929 | return err; |
| 942 | 930 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_WDT_STATUS, &val); |
| 943 | − if (err < 0) | |
| 931 | + if (err) | |
| 944 | 932 | return err; |
| 945 | 933 | /* Reset the bluetooth chip via USB interface. */ |
| 946 | 934 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_SUBSYS_RST, 1); |
| @@ -950,13 +938,13 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 3 unmodified lines | ||
| 950 | 938 | if (err < 0) |
| 951 | 939 | return err; |
| 952 | 940 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT, &val); |
| 953 | − if (err < 0) | |
| 941 | + if (err) | |
| 954 | 942 | return err; |
| 955 | 943 | err = btmtk_usb_uhw_reg_write(hdev, MTK_UDMA_INT_STA_BT1, 0x000000FF); |
| 956 | 944 | if (err < 0) |
| 957 | 945 | return err; |
| 958 | 946 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT1, &val); |
| 959 | − if (err < 0) | |
| 947 | + if (err) | |
| 960 | 948 | return err; |
| 961 | 949 | /* MT7921 need to delay 20ms between toggle reset bit */ |
| 962 | 950 | msleep(20); |
| @@ -964,7 +952,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 1 unmodified line | ||
| 964 | 952 | if (err < 0) |
| 965 | 953 | return err; |
| 966 | 954 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_SUBSYS_RST, &val); |
| 967 | − if (err < 0) | |
| 955 | + if (err) | |
| 968 | 956 | return err; |
| 969 | 957 | } |
| 970 | 958 | |
| @@ -980,7 +968,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 9 unmodified lines | ||
| 980 | 968 | } |
| 981 | 969 | |
| 982 | 970 | err = btmtk_usb_id_get(hdev, 0x70010200, &val); |
| 983 | − if (err < 0 || (!val && dev_id != 0x6639)) | |
| 971 | + if (err || (!val && dev_id != 0x6639)) | |
| 984 | 972 | bt_dev_err(hdev, "Can't get device id, subsys reset fail."); |
| 985 | 973 | |
| 986 | 974 | return err; |
| @@ -1324,24 +1312,24 @@ int btmtk_usb_setup(struct hci_dev *hdev) · 337 unmodified lines | ||
| 1324 | 1312 | calltime = ktime_get(); |
| 1325 | 1313 | |
| 1326 | 1314 | err = btmtk_usb_id_get(hdev, 0x80000008, &dev_id); |
| 1327 | − if (err < 0) { | |
| 1315 | + if (err) { | |
| 1328 | 1316 | bt_dev_err(hdev, "Failed to get device id (%d)", err); |
| 1329 | 1317 | return err; |
| 1330 | 1318 | } |
| 1331 | 1319 | |
| 1332 | 1320 | if (!dev_id || dev_id != 0x7663) { |
| 1333 | 1321 | err = btmtk_usb_id_get(hdev, 0x70010200, &dev_id); |
| 1334 | − if (err < 0) { | |
| 1322 | + if (err) { | |
| 1335 | 1323 | bt_dev_err(hdev, "Failed to get device id (%d)", err); |
| 1336 | 1324 | return err; |
| 1337 | 1325 | } |
| 1338 | 1326 | err = btmtk_usb_id_get(hdev, 0x80021004, &fw_version); |
| 1339 | − if (err < 0) { | |
| 1327 | + if (err) { | |
| 1340 | 1328 | bt_dev_err(hdev, "Failed to get fw version (%d)", err); |
| 1341 | 1329 | return err; |
| 1342 | 1330 | } |
| 1343 | 1331 | err = btmtk_usb_id_get(hdev, 0x70010020, &fw_flavor); |
| 1344 | − if (err < 0) { | |
| 1332 | + if (err) { | |
| 1345 | 1333 | bt_dev_err(hdev, "Failed to get fw flavor (%d)", err); |
| 1346 | 1334 | return err; |
| 1347 | 1335 | } |
| @@ -3424,28 +3424,16 @@ static const char *qca_get_fw_subdirectory(const struct qca_version *ver) · 3423 unmodified lines | ||
| 3424 | 3424 | static int btusb_qca_send_vendor_req(struct usb_device *udev, u8 request, |
| 3425 | 3425 | void *data, u16 size) |
| 3426 | 3426 | { |
| 3427 | − int pipe, err; | |
| 3428 | − u8 *buf; | |
| 3429 | − | |
| 3430 | − buf = kmalloc(size, GFP_KERNEL); | |
| 3431 | − if (!buf) | |
| 3432 | − return -ENOMEM; | |
| 3427 | + int err; | |
| 3433 | 3428 | |
| 3434 | 3429 | /* Found some of USB hosts have IOT issues with ours so that we should |
| 3435 | 3430 | * not wait until HCI layer is ready. |
| 3436 | 3431 | */ |
| 3437 | − pipe = usb_rcvctrlpipe(udev, 0); | |
| 3438 | − err = usb_control_msg(udev, pipe, request, USB_TYPE_VENDOR | USB_DIR_IN, | |
| 3439 | − 0, 0, buf, size, USB_CTRL_GET_TIMEOUT); | |
| 3440 | − if (err < 0) { | |
| 3432 | + err = usb_control_msg_recv(udev, 0, request, USB_TYPE_VENDOR | USB_DIR_IN, | |
| 3433 | + 0, 0, data, size, USB_CTRL_GET_TIMEOUT, | |
| 3434 | + GFP_KERNEL); | |
| 3435 | + if (err) | |
| 3441 | 3436 | dev_err(&udev->dev, "Failed to access otp area (%d)", err); |
| 3442 | − goto done; | |
| 3443 | − } | |
| 3444 | − | |
| 3445 | − memcpy(data, buf, size); | |
| 3446 | − | |
| 3447 | −done: | |
| 3448 | − kfree(buf); | |
| 3449 | 3437 | |
| 3450 | 3438 | return err; |
| 3451 | 3439 | } |
| @@ -3652,7 +3640,7 @@ static bool btusb_qca_need_patch(struct usb_device *udev) · 200 unmodified lines | ||
| 3652 | 3640 | struct qca_version ver; |
| 3653 | 3641 | |
| 3654 | 3642 | if (btusb_qca_send_vendor_req(udev, QCA_GET_TARGET_VERSION, &ver, |
| 3655 | − sizeof(ver)) < 0) | |
| 3643 | + sizeof(ver))) | |
| 3656 | 3644 | return false; |
| 3657 | 3645 | /* only low ROM versions need patches */ |
| 3658 | 3646 | return !(le32_to_cpu(ver.rom_version) & ~0xffffU); |
| @@ -3670,7 +3658,7 @@ static int btusb_setup_qca(struct hci_dev *hdev) · 11 unmodified lines | ||
| 3670 | 3658 | |
| 3671 | 3659 | err = btusb_qca_send_vendor_req(udev, QCA_GET_TARGET_VERSION, &ver, |
| 3672 | 3660 | sizeof(ver)); |
| 3673 | − if (err < 0) | |
| 3661 | + if (err) | |
| 3674 | 3662 | return err; |
| 3675 | 3663 | |
| 3676 | 3664 | ver_rom = le32_to_cpu(ver.rom_version); |
| @@ -3693,7 +3681,7 @@ static int btusb_setup_qca(struct hci_dev *hdev) · 16 unmodified lines | ||
| 3693 | 3681 | |
| 3694 | 3682 | err = btusb_qca_send_vendor_req(udev, QCA_CHECK_STATUS, &status, |
| 3695 | 3683 | sizeof(status)); |
| 3696 | − if (err < 0) | |
| 3684 | + if (err) | |
| 3697 | 3685 | return err; |
| 3698 | 3686 | |
| 3699 | 3687 | if (!(status & QCA_PATCH_UPDATED)) { |
| @@ -3704,7 +3692,7 @@ static int btusb_setup_qca(struct hci_dev *hdev) · 4 unmodified lines | ||
| 3704 | 3692 | |
| 3705 | 3693 | err = btusb_qca_send_vendor_req(udev, QCA_GET_TARGET_VERSION, &ver, |
| 3706 | 3694 | sizeof(ver)); |
| 3707 | − if (err < 0) | |
| 3695 | + if (err) | |
| 3708 | 3696 | return err; |
| 3709 | 3697 | |
| 3710 | 3698 | btdata->qca_dump.fw_version = le32_to_cpu(ver.patch_version); |
| @@ -483,7 +483,7 @@ static int mchp_tc_probe(struct platform_device *pdev) · 482 unmodified lines | ||
| 483 | 483 | char clk_name[7]; |
| 484 | 484 | struct regmap *regmap; |
| 485 | 485 | struct clk *clk[3]; |
| 486 | − int channel; | |
| 486 | + u32 channel; | |
| 487 | 487 | int ret, i; |
| 488 | 488 | |
| 489 | 489 | counter = devm_counter_alloc(&pdev->dev, sizeof(*priv)); |
| @@ -517,7 +517,7 @@ static int mchp_tc_probe(struct platform_device *pdev) · 27 unmodified lines | ||
| 517 | 517 | |
| 518 | 518 | priv->channel[i] = channel; |
| 519 | 519 | |
| 520 | − snprintf(clk_name, sizeof(clk_name), "t%d_clk", channel); | |
| 520 | + snprintf(clk_name, sizeof(clk_name), "t%u_clk", channel); | |
| 521 | 521 | |
| 522 | 522 | clk[i] = of_clk_get_by_name(np->parent, clk_name); |
| 523 | 523 | if (IS_ERR(clk[i])) { |
| @@ -1031,7 +1031,7 @@ static int amd_pstate_init_freq(struct amd_cpudata *cpudata) · 1030 unmodified lines | ||
| 1031 | 1031 | return -EINVAL; |
| 1032 | 1032 | } |
| 1033 | 1033 | |
| 1034 | − if (lowest_nonlinear_freq <= min_freq || lowest_nonlinear_freq > nominal_freq) { | |
| 1034 | + if (lowest_nonlinear_freq < min_freq || lowest_nonlinear_freq > nominal_freq) { | |
| 1035 | 1035 | pr_err("lowest_nonlinear_freq(%d) value is out of range [min_freq(%d), nominal_freq(%d)]\n", |
| 1036 | 1036 | lowest_nonlinear_freq, min_freq, nominal_freq); |
| 1037 | 1037 | return -EINVAL; |
| @@ -2167,6 +2167,7 @@ static struct cpufreq_driver amd_pstate_epp_driver = { · 1129 unmodified lines | ||
| 2167 | 2167 | }; |
| 2168 | 2168 | |
| 2169 | 2169 | /* |
| 2170 | + * Processors without frequency scaling support can't do CPPC. | |
| 2170 | 2171 | * CPPC function is not supported for family ID 17H with model_ID ranging from 0x10 to 0x2F. |
| 2171 | 2172 | * show the debug message that helps to check if the CPU has CPPC support for loading issue. |
| 2172 | 2173 | */ |
| @@ -2175,6 +2176,11 @@ static bool amd_cppc_supported(void) · 2 unmodified lines | ||
| 2175 | 2176 | struct cpuinfo_x86 *c = &cpu_data(0); |
| 2176 | 2177 | bool warn = false; |
| 2177 | 2178 | |
| 2179 | + if (!cpu_feature_enabled(X86_FEATURE_HW_PSTATE)) { | |
| 2180 | + pr_debug_once("frequency scaling is not supported by the processor\n"); | |
| 2181 | + return false; | |
| 2182 | + } | |
| 2183 | + | |
| 2178 | 2184 | if ((boot_cpu_data.x86 == 0x17) && (boot_cpu_data.x86_model < 0x30)) { |
| 2179 | 2185 | pr_debug_once("CPPC feature is not supported by the processor\n"); |
| 2180 | 2186 | return false; |
| @@ -290,19 +290,32 @@ static inline void cppc_freq_invariance_exit(void) · 289 unmodified lines | ||
| 290 | 290 | } |
| 291 | 291 | #endif /* CONFIG_ACPI_CPPC_CPUFREQ_FIE */ |
| 292 | 292 | |
| 293 | −static void cppc_cpufreq_update_perf_limits(struct cppc_cpudata *cpu_data, | |
| 294 | − struct cpufreq_policy *policy) | |
| 293 | +static void cppc_cpufreq_get_perf_limits(struct cppc_cpudata *cpu_data, | |
| 294 | + struct cpufreq_policy *policy, | |
| 295 | + u32 *min_perf, u32 *max_perf) | |
| 295 | 296 | { |
| 296 | 297 | struct cppc_perf_caps *caps = &cpu_data->perf_caps; |
| 297 | − u32 min_perf, max_perf; | |
| 298 | + unsigned int min_freq, max_freq; | |
| 299 | + u32 min, max; | |
| 300 | + | |
| 301 | + min_freq = READ_ONCE(policy->min); | |
| 302 | + max_freq = READ_ONCE(policy->max); | |
| 303 | + if (unlikely(min_freq > max_freq)) | |
| 304 | + min_freq = max_freq; | |
| 305 | + | |
| 306 | + min = cppc_khz_to_perf(caps, min_freq); | |
| 307 | + max = cppc_khz_to_perf(caps, max_freq); | |
| 298 | 308 | |
| 299 | − min_perf = cppc_khz_to_perf(caps, policy->min); | |
| 300 | − max_perf = cppc_khz_to_perf(caps, policy->max); | |
| 309 | + *min_perf = clamp_t(u32, min, caps->lowest_perf, caps->highest_perf); | |
| 310 | + *max_perf = clamp_t(u32, max, caps->lowest_perf, caps->highest_perf); | |
| 311 | +} | |
| 301 | 312 | |
| 302 | − cpu_data->perf_ctrls.min_perf = | |
| 303 | − clamp_t(u32, min_perf, caps->lowest_perf, caps->highest_perf); | |
| 304 | − cpu_data->perf_ctrls.max_perf = | |
| 305 | − clamp_t(u32, max_perf, caps->lowest_perf, caps->highest_perf); | |
| 313 | +static void cppc_cpufreq_update_perf_limits(struct cppc_cpudata *cpu_data, | |
| 314 | + struct cpufreq_policy *policy) | |
| 315 | +{ | |
| 316 | + cppc_cpufreq_get_perf_limits(cpu_data, policy, | |
| 317 | + &cpu_data->perf_ctrls.min_perf, | |
| 318 | + &cpu_data->perf_ctrls.max_perf); | |
| 306 | 319 | } |
| 307 | 320 | |
| 308 | 321 | static int cppc_cpufreq_set_target(struct cpufreq_policy *policy, |
| @@ -693,7 +706,7 @@ static int cppc_cpufreq_cpu_init(struct cpufreq_policy *policy) · 384 unmodified lines | ||
| 693 | 706 | goto out; |
| 694 | 707 | } |
| 695 | 708 | |
| 696 | − policy->fast_switch_possible = cppc_allow_fast_switch(); | |
| 709 | + policy->fast_switch_possible = cppc_allow_fast_switch(policy->cpus); | |
| 697 | 710 | policy->dvfs_possible_from_any_cpu = true; |
| 698 | 711 | |
| 699 | 712 | /* |
| @@ -1084,6 +1084,7 @@ static int powernowk8_cpu_init(struct cpufreq_policy *pol) · 1083 unmodified lines | ||
| 1084 | 1084 | |
| 1085 | 1085 | err_out_exit_acpi: |
| 1086 | 1086 | powernow_k8_cpu_exit_acpi(data); |
| 1087 | + kfree(data->powernow_table); | |
| 1087 | 1088 | |
| 1088 | 1089 | err_out: |
| 1089 | 1090 | kfree(data); |
| @@ -118,14 +118,9 @@ err_bit: · 117 unmodified lines | ||
| 118 | 118 | return rc; |
| 119 | 119 | } |
| 120 | 120 | |
| 121 | −static void __dibs_lo_unregister_dmb(struct dibs_lo_dev *ldev, | |
| 122 | − struct dibs_lo_dmb_node *dmb_node) | |
| 121 | +static void dibs_lo_free_dmb(struct dibs_lo_dev *ldev, | |
| 122 | + struct dibs_lo_dmb_node *dmb_node) | |
| 123 | 123 | { |
| 124 | − /* remove dmb from hash table */ | |
| 125 | − write_lock_bh(&ldev->dmb_ht_lock); | |
| 126 | − hash_del(&dmb_node->list); | |
| 127 | − write_unlock_bh(&ldev->dmb_ht_lock); | |
| 128 | − | |
| 129 | 124 | clear_bit(dmb_node->sba_idx, ldev->sba_idx_mask); |
| 130 | 125 | folio_put(virt_to_folio(dmb_node->cpu_addr)); |
| 131 | 126 | kfree(dmb_node); |
| @@ -139,27 +134,33 @@ static int dibs_lo_unregister_dmb(struct dibs_dev *dibs, struct dibs_dmb *dmb) · 7 unmodified lines | ||
| 139 | 134 | struct dibs_lo_dmb_node *dmb_node = NULL, *tmp_node; |
| 140 | 135 | struct dibs_lo_dev *ldev; |
| 141 | 136 | unsigned long flags; |
| 137 | + bool last; | |
| 142 | 138 | |
| 143 | 139 | ldev = dibs->drv_priv; |
| 144 | 140 | |
| 145 | 141 | /* find dmb from hash table */ |
| 146 | − read_lock_bh(&ldev->dmb_ht_lock); | |
| 142 | + write_lock_bh(&ldev->dmb_ht_lock); | |
| 147 | 143 | hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb->dmb_tok) { |
| 148 | 144 | if (tmp_node->token == dmb->dmb_tok) { |
| 149 | 145 | dmb_node = tmp_node; |
| 150 | 146 | break; |
| 151 | 147 | } |
| 152 | 148 | } |
| 153 | − read_unlock_bh(&ldev->dmb_ht_lock); | |
| 154 | − if (!dmb_node) | |
| 149 | + if (!dmb_node) { | |
| 150 | + write_unlock_bh(&ldev->dmb_ht_lock); | |
| 155 | 151 | return -EINVAL; |
| 152 | + } | |
| 153 | + last = refcount_dec_and_test(&dmb_node->refcnt); | |
| 154 | + if (last) | |
| 155 | + hash_del(&dmb_node->list); | |
| 156 | + write_unlock_bh(&ldev->dmb_ht_lock); | |
| 156 | 157 | |
| 157 | − if (refcount_dec_and_test(&dmb_node->refcnt)) { | |
| 158 | + if (last) { | |
| 158 | 159 | spin_lock_irqsave(&dibs->lock, flags); |
| 159 | 160 | dibs->dmb_clientid_arr[dmb_node->sba_idx] = NO_DIBS_CLIENT; |
| 160 | 161 | spin_unlock_irqrestore(&dibs->lock, flags); |
| 161 | 162 | |
| 162 | − __dibs_lo_unregister_dmb(ldev, dmb_node); | |
| 163 | + dibs_lo_free_dmb(ldev, dmb_node); | |
| 163 | 164 | } |
| 164 | 165 | return 0; |
| 165 | 166 | } |
| @@ -188,14 +189,9 @@ static int dibs_lo_attach_dmb(struct dibs_dev *dibs, struct dibs_dmb *dmb) · 22 unmodified lines | ||
| 188 | 189 | read_unlock_bh(&ldev->dmb_ht_lock); |
| 189 | 190 | return -EINVAL; |
| 190 | 191 | } |
| 192 | + refcount_inc(&dmb_node->refcnt); | |
| 191 | 193 | read_unlock_bh(&ldev->dmb_ht_lock); |
| 192 | 194 | |
| 193 | − if (!refcount_inc_not_zero(&dmb_node->refcnt)) | |
| 194 | − /* the dmb is being unregistered, but has | |
| 195 | − * not been removed from the hash table. | |
| 196 | − */ | |
| 197 | − return -EINVAL; | |
| 198 | − | |
| 199 | 195 | /* provide dmb information */ |
| 200 | 196 | dmb->idx = dmb_node->sba_idx; |
| 201 | 197 | dmb->dmb_tok = dmb_node->token; |
| @@ -209,11 +205,12 @@ static int dibs_lo_detach_dmb(struct dibs_dev *dibs, u64 token) · 7 unmodified lines | ||
| 209 | 205 | { |
| 210 | 206 | struct dibs_lo_dmb_node *dmb_node = NULL, *tmp_node; |
| 211 | 207 | struct dibs_lo_dev *ldev; |
| 208 | + bool last; | |
| 212 | 209 | |
| 213 | 210 | ldev = dibs->drv_priv; |
| 214 | 211 | |
| 215 | 212 | /* find dmb_node according to dmb->dmb_tok */ |
| 216 | − read_lock_bh(&ldev->dmb_ht_lock); | |
| 213 | + write_lock_bh(&ldev->dmb_ht_lock); | |
| 217 | 214 | hash_for_each_possible(ldev->dmb_ht, tmp_node, list, token) { |
| 218 | 215 | if (tmp_node->token == token) { |
| 219 | 216 | dmb_node = tmp_node; |
| @@ -221,13 +218,17 @@ static int dibs_lo_detach_dmb(struct dibs_dev *dibs, u64 token) · 1 unmodified line | ||
| 221 | 218 | } |
| 222 | 219 | } |
| 223 | 220 | if (!dmb_node) { |
| 224 | − read_unlock_bh(&ldev->dmb_ht_lock); | |
| 221 | + write_unlock_bh(&ldev->dmb_ht_lock); | |
| 225 | 222 | return -EINVAL; |
| 226 | 223 | } |
| 227 | − read_unlock_bh(&ldev->dmb_ht_lock); | |
| 224 | + last = refcount_dec_and_test(&dmb_node->refcnt); | |
| 225 | + if (last) | |
| 226 | + hash_del(&dmb_node->list); | |
| 227 | + write_unlock_bh(&ldev->dmb_ht_lock); | |
| 228 | + | |
| 229 | + if (last) | |
| 230 | + dibs_lo_free_dmb(ldev, dmb_node); | |
| 228 | 231 | |
| 229 | − if (refcount_dec_and_test(&dmb_node->refcnt)) | |
| 230 | − __dibs_lo_unregister_dmb(ldev, dmb_node); | |
| 231 | 232 | return 0; |
| 232 | 233 | } |
| 233 | 234 | |
| @@ -138,6 +138,7 @@ struct dibs_dev *dibs_dev_alloc(void) · 137 unmodified lines | ||
| 138 | 138 | dibs = kzalloc_obj(*dibs); |
| 139 | 139 | if (!dibs) |
| 140 | 140 | return dibs; |
| 141 | + spin_lock_init(&dibs->lock); | |
| 141 | 142 | dibs->dev.release = dibs_dev_release; |
| 142 | 143 | dibs->dev.class = &dibs_class; |
| 143 | 144 | device_initialize(&dibs->dev); |
| @@ -186,7 +187,6 @@ int dibs_dev_add(struct dibs_dev *dibs) · 42 unmodified lines | ||
| 186 | 187 | int i, ret; |
| 187 | 188 | |
| 188 | 189 | max_dmbs = dibs->ops->max_dmbs(); |
| 189 | − spin_lock_init(&dibs->lock); | |
| 190 | 190 | dibs->dmb_clientid_arr = kzalloc(max_dmbs, GFP_KERNEL); |
| 191 | 191 | if (!dibs->dmb_clientid_arr) |
| 192 | 192 | return -ENOMEM; |
| @@ -288,6 +288,7 @@ static int idxd_cdev_open(struct inode *inode, struct file *filp) · 287 unmodified lines | ||
| 288 | 288 | fdev->parent = cdev_dev(idxd_cdev); |
| 289 | 289 | fdev->bus = &dsa_bus_type; |
| 290 | 290 | fdev->type = &idxd_cdev_file_type; |
| 291 | + idxd_wq_get(wq); | |
| 291 | 292 | |
| 292 | 293 | rc = dev_set_name(fdev, "file%d", ctx->id); |
| 293 | 294 | if (rc < 0) { |
| @@ -301,13 +302,14 @@ static int idxd_cdev_open(struct inode *inode, struct file *filp) · 7 unmodified lines | ||
| 301 | 302 | goto failed_dev_add; |
| 302 | 303 | } |
| 303 | 304 | |
| 304 | − idxd_wq_get(wq); | |
| 305 | 305 | mutex_unlock(&wq->wq_lock); |
| 306 | 306 | return 0; |
| 307 | 307 | |
| 308 | 308 | failed_dev_add: |
| 309 | 309 | failed_dev_name: |
| 310 | + mutex_unlock(&wq->wq_lock); | |
| 310 | 311 | put_device(fdev); |
| 312 | + return rc; | |
| 311 | 313 | failed_ida: |
| 312 | 314 | failed_set_pasid: |
| 313 | 315 | if (device_user_pasid_enabled(idxd)) |
| @@ -159,18 +159,12 @@ static void idxd_cleanup_interrupts(struct idxd_device *idxd) · 158 unmodified lines | ||
| 159 | 159 | |
| 160 | 160 | static void idxd_clean_wqs(struct idxd_device *idxd) |
| 161 | 161 | { |
| 162 | − struct idxd_wq *wq; | |
| 163 | 162 | struct device *conf_dev; |
| 164 | 163 | int i; |
| 165 | 164 | |
| 166 | 165 | for (i = 0; i < idxd->max_wqs; i++) { |
| 167 | − wq = idxd->wqs[i]; | |
| 168 | − if (idxd->hw.wq_cap.op_config) | |
| 169 | − bitmap_free(wq->opcap_bmap); | |
| 170 | − kfree(wq->wqcfg); | |
| 171 | − conf_dev = wq_confdev(wq); | |
| 166 | + conf_dev = wq_confdev(idxd->wqs[i]); | |
| 172 | 167 | put_device(conf_dev); |
| 173 | − kfree(wq); | |
| 174 | 168 | } |
| 175 | 169 | bitmap_free(idxd->wq_enable_map); |
| 176 | 170 | kfree(idxd->wqs); |
| @@ -212,7 +206,6 @@ static int idxd_setup_wqs(struct idxd_device *idxd) · 35 unmodified lines | ||
| 212 | 206 | rc = dev_set_name(conf_dev, "wq%d.%d", idxd->id, wq->id); |
| 213 | 207 | if (rc < 0) { |
| 214 | 208 | put_device(conf_dev); |
| 215 | − kfree(wq); | |
| 216 | 209 | goto err_unwind; |
| 217 | 210 | } |
| 218 | 211 | |
| @@ -227,7 +220,6 @@ static int idxd_setup_wqs(struct idxd_device *idxd) · 8 unmodified lines | ||
| 227 | 220 | wq->wqcfg = kzalloc_node(idxd->wqcfg_size, GFP_KERNEL, dev_to_node(dev)); |
| 228 | 221 | if (!wq->wqcfg) { |
| 229 | 222 | put_device(conf_dev); |
| 230 | − kfree(wq); | |
| 231 | 223 | rc = -ENOMEM; |
| 232 | 224 | goto err_unwind; |
| 233 | 225 | } |
| @@ -235,9 +227,7 @@ static int idxd_setup_wqs(struct idxd_device *idxd) · 1 unmodified line | ||
| 235 | 227 | if (idxd->hw.wq_cap.op_config) { |
| 236 | 228 | wq->opcap_bmap = bitmap_zalloc(IDXD_MAX_OPCAP_BITS, GFP_KERNEL); |
| 237 | 229 | if (!wq->opcap_bmap) { |
| 238 | − kfree(wq->wqcfg); | |
| 239 | 230 | put_device(conf_dev); |
| 240 | − kfree(wq); | |
| 241 | 231 | rc = -ENOMEM; |
| 242 | 232 | goto err_unwind; |
| 243 | 233 | } |
| @@ -252,13 +242,8 @@ static int idxd_setup_wqs(struct idxd_device *idxd) · 8 unmodified lines | ||
| 252 | 242 | |
| 253 | 243 | err_unwind: |
| 254 | 244 | while (--i >= 0) { |
| 255 | − wq = idxd->wqs[i]; | |
| 256 | − if (idxd->hw.wq_cap.op_config) | |
| 257 | − bitmap_free(wq->opcap_bmap); | |
| 258 | − kfree(wq->wqcfg); | |
| 259 | − conf_dev = wq_confdev(wq); | |
| 245 | + conf_dev = wq_confdev(idxd->wqs[i]); | |
| 260 | 246 | put_device(conf_dev); |
| 261 | − kfree(wq); | |
| 262 | 247 | } |
| 263 | 248 | bitmap_free(idxd->wq_enable_map); |
| 264 | 249 | |
| @@ -270,15 +255,12 @@ err_free_wqs: · 5 unmodified lines | ||
| 270 | 255 | |
| 271 | 256 | static void idxd_clean_engines(struct idxd_device *idxd) |
| 272 | 257 | { |
| 273 | − struct idxd_engine *engine; | |
| 274 | 258 | struct device *conf_dev; |
| 275 | 259 | int i; |
| 276 | 260 | |
| 277 | 261 | for (i = 0; i < idxd->max_engines; i++) { |
| 278 | − engine = idxd->engines[i]; | |
| 279 | − conf_dev = engine_confdev(engine); | |
| 262 | + conf_dev = engine_confdev(idxd->engines[i]); | |
| 280 | 263 | put_device(conf_dev); |
| 281 | − kfree(engine); | |
| 282 | 264 | } |
| 283 | 265 | kfree(idxd->engines); |
| 284 | 266 | } |
| @@ -313,7 +295,6 @@ static int idxd_setup_engines(struct idxd_device *idxd) · 28 unmodified lines | ||
| 313 | 295 | rc = dev_set_name(conf_dev, "engine%d.%d", idxd->id, engine->id); |
| 314 | 296 | if (rc < 0) { |
| 315 | 297 | put_device(conf_dev); |
| 316 | − kfree(engine); | |
| 317 | 298 | goto err; |
| 318 | 299 | } |
| 319 | 300 | |
| @@ -324,10 +305,8 @@ static int idxd_setup_engines(struct idxd_device *idxd) · 4 unmodified lines | ||
| 324 | 305 | |
| 325 | 306 | err: |
| 326 | 307 | while (--i >= 0) { |
| 327 | − engine = idxd->engines[i]; | |
| 328 | − conf_dev = engine_confdev(engine); | |
| 308 | + conf_dev = engine_confdev(idxd->engines[i]); | |
| 329 | 309 | put_device(conf_dev); |
| 330 | − kfree(engine); | |
| 331 | 310 | } |
| 332 | 311 | kfree(idxd->engines); |
| 333 | 312 | |
| @@ -336,13 +315,10 @@ static int idxd_setup_engines(struct idxd_device *idxd) · 2 unmodified lines | ||
| 336 | 315 | |
| 337 | 316 | static void idxd_clean_groups(struct idxd_device *idxd) |
| 338 | 317 | { |
| 339 | − struct idxd_group *group; | |
| 340 | 318 | int i; |
| 341 | 319 | |
| 342 | 320 | for (i = 0; i < idxd->max_groups; i++) { |
| 343 | − group = idxd->groups[i]; | |
| 344 | − put_device(group_confdev(group)); | |
| 345 | − kfree(group); | |
| 321 | + put_device(group_confdev(idxd->groups[i])); | |
| 346 | 322 | } |
| 347 | 323 | kfree(idxd->groups); |
| 348 | 324 | } |
| @@ -377,7 +353,6 @@ static int idxd_setup_groups(struct idxd_device *idxd) · 28 unmodified lines | ||
| 377 | 353 | rc = dev_set_name(conf_dev, "group%d.%d", idxd->id, group->id); |
| 378 | 354 | if (rc < 0) { |
| 379 | 355 | put_device(conf_dev); |
| 380 | − kfree(group); | |
| 381 | 356 | goto err; |
| 382 | 357 | } |
| 383 | 358 | |
| @@ -402,7 +377,6 @@ static int idxd_setup_groups(struct idxd_device *idxd) · 18 unmodified lines | ||
| 402 | 377 | while (--i >= 0) { |
| 403 | 378 | group = idxd->groups[i]; |
| 404 | 379 | put_device(group_confdev(group)); |
| 405 | − kfree(group); | |
| 406 | 380 | } |
| 407 | 381 | kfree(idxd->groups); |
| 408 | 382 | |
| @@ -945,16 +945,13 @@ static int sun6i_dma_terminate_all(struct dma_chan *chan) · 944 unmodified lines | ||
| 945 | 945 | |
| 946 | 946 | spin_lock_irqsave(&vchan->vc.lock, flags); |
| 947 | 947 | |
| 948 | − if (vchan->cyclic) { | |
| 949 | − vchan->cyclic = false; | |
| 950 | − if (pchan && pchan->desc) { | |
| 951 | − struct virt_dma_desc *vd = &pchan->desc->vd; | |
| 952 | − struct virt_dma_chan *vc = &vchan->vc; | |
| 948 | + if (pchan && pchan->desc && pchan->desc != pchan->done) { | |
| 949 | + struct virt_dma_desc *vd = &pchan->desc->vd; | |
| 953 | 950 | |
| 954 | − list_add_tail(&vd->node, &vc->desc_completed); | |
| 955 | − } | |
| 951 | + vchan_terminate_vdesc(vd); | |
| 956 | 952 | } |
| 957 | 953 | |
| 954 | + vchan->cyclic = false; | |
| 958 | 955 | vchan_get_all_descriptors(&vchan->vc, &head); |
| 959 | 956 | |
| 960 | 957 | if (pchan) { |
| @@ -1099,7 +1099,7 @@ static int switchtec_dma_chan_init(struct switchtec_dma_dev *swdma_dev, · 1098 unmodified lines | ||
| 1099 | 1099 | dev_dbg(&pdev->dev, "Channel %d: SE buffer count %d\n", i, se_buf_len); |
| 1100 | 1100 | |
| 1101 | 1101 | thresh = se_buf_len / 2; |
| 1102 | − valid_en_se |= FIELD_GET(SE_THRESH_MASK, thresh); | |
| 1102 | + valid_en_se |= FIELD_PREP(SE_THRESH_MASK, thresh); | |
| 1103 | 1103 | writel(valid_en_se, &swdma_chan->mmio_chan_fw->valid_en_se); |
| 1104 | 1104 | |
| 1105 | 1105 | /* request irqs */ |
| @@ -1142,6 +1142,33 @@ void *dpll_pin_on_pin_priv(struct dpll_pin *parent, · 1141 unmodified lines | ||
| 1142 | 1142 | return reg->priv; |
| 1143 | 1143 | } |
| 1144 | 1144 | |
| 1145 | +/** | |
| 1146 | + * dpll_pin_own_dpll_ref_first - find the first owner dpll ref of a pin | |
| 1147 | + * @pin: pointer to a dpll pin | |
| 1148 | + * | |
| 1149 | + * Search pin's dpll_refs for a ref whose dpll matches the pin's | |
| 1150 | + * (module, clock_id) tuple, i.e. the dpll registered by the driver | |
| 1151 | + * that created the pin. This ensures pin-level attributes are | |
| 1152 | + * reported and modified using the owner's ops even when the pin is | |
| 1153 | + * also registered with dplls from other drivers. | |
| 1154 | + * | |
| 1155 | + * Return: pointer to the owner's dpll_pin_ref, or NULL if no | |
| 1156 | + * owner ref is found. | |
| 1157 | + */ | |
| 1158 | +struct dpll_pin_ref *dpll_pin_own_dpll_ref_first(struct dpll_pin *pin) | |
| 1159 | +{ | |
| 1160 | + struct dpll_pin_ref *ref; | |
| 1161 | + unsigned long i; | |
| 1162 | + | |
| 1163 | + xa_for_each(&pin->dpll_refs, i, ref) { | |
| 1164 | + if (ref->dpll->module == pin->module && | |
| 1165 | + ref->dpll->clock_id == pin->clock_id) | |
| 1166 | + return ref; | |
| 1167 | + } | |
| 1168 | + | |
| 1169 | + return NULL; | |
| 1170 | +} | |
| 1171 | + | |
| 1145 | 1172 | const struct dpll_pin_ops *dpll_pin_ops(struct dpll_pin_ref *ref) |
| 1146 | 1173 | { |
| 1147 | 1174 | struct dpll_pin_registration *reg; |
| @@ -93,6 +93,7 @@ void *dpll_pin_on_pin_priv(struct dpll_pin *parent, struct dpll_pin *pin); · 92 unmodified lines | ||
| 93 | 93 | |
| 94 | 94 | const struct dpll_device_ops *dpll_device_ops(struct dpll_device *dpll); |
| 95 | 95 | struct dpll_device *dpll_device_get_by_id(int id); |
| 96 | +struct dpll_pin_ref *dpll_pin_own_dpll_ref_first(struct dpll_pin *pin); | |
| 96 | 97 | const struct dpll_pin_ops *dpll_pin_ops(struct dpll_pin_ref *ref); |
| 97 | 98 | struct dpll_pin_ref *dpll_xa_ref_dpll_first(struct xarray *xa_refs); |
| 98 | 99 | extern struct xarray dpll_device_xa; |
| @@ -699,7 +699,9 @@ dpll_cmd_pin_get_one(struct sk_buff *msg, struct dpll_pin *pin, · 698 unmodified lines | ||
| 699 | 699 | struct dpll_pin_ref *ref; |
| 700 | 700 | int ret; |
| 701 | 701 | |
| 702 | − ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | |
| 702 | + ref = dpll_pin_own_dpll_ref_first(pin); | |
| 703 | + if (!ref) | |
| 704 | + ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | |
| 703 | 705 | ASSERT_NOT_NULL(ref); |
| 704 | 706 | |
| 705 | 707 | ret = dpll_msg_add_pin_handle(msg, pin); |
| @@ -1090,12 +1092,19 @@ dpll_pin_freq_set(struct dpll_pin *pin, struct nlattr *a, · 384 unmodified lines | ||
| 1090 | 1092 | |
| 1091 | 1093 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1092 | 1094 | ops = dpll_pin_ops(ref); |
| 1093 | − if (!ops->frequency_set || !ops->frequency_get) { | |
| 1094 | − NL_SET_ERR_MSG(extack, "frequency set not supported by the device"); | |
| 1095 | + if ((!ops->frequency_set || !ops->frequency_get) && | |
| 1096 | + ref->dpll->module == pin->module && | |
| 1097 | + ref->dpll->clock_id == pin->clock_id) { | |
| 1098 | + NL_SET_ERR_MSG(extack, | |
| 1099 | + "frequency set not supported by the device"); | |
| 1095 | 1100 | return -EOPNOTSUPP; |
| 1096 | 1101 | } |
| 1097 | 1102 | } |
| 1098 | − ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | |
| 1103 | + ref = dpll_pin_own_dpll_ref_first(pin); | |
| 1104 | + if (!ref) { | |
| 1105 | + NL_SET_ERR_MSG(extack, "pin owner dpll not found"); | |
| 1106 | + return -ENODEV; | |
| 1107 | + } | |
| 1099 | 1108 | ops = dpll_pin_ops(ref); |
| 1100 | 1109 | dpll = ref->dpll; |
| 1101 | 1110 | ret = ops->frequency_get(pin, dpll_pin_on_dpll_priv(dpll, pin), dpll, |
| @@ -1109,6 +1118,8 @@ dpll_pin_freq_set(struct dpll_pin *pin, struct nlattr *a, · 7 unmodified lines | ||
| 1109 | 1118 | |
| 1110 | 1119 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1111 | 1120 | ops = dpll_pin_ops(ref); |
| 1121 | + if (!ops->frequency_set) | |
| 1122 | + continue; | |
| 1112 | 1123 | dpll = ref->dpll; |
| 1113 | 1124 | ret = ops->frequency_set(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| 1114 | 1125 | dpll, dpll_priv(dpll), freq, extack); |
| @@ -1128,6 +1139,8 @@ rollback: · 13 unmodified lines | ||
| 1128 | 1139 | if (ref == failed) |
| 1129 | 1140 | break; |
| 1130 | 1141 | ops = dpll_pin_ops(ref); |
| 1142 | + if (!ops->frequency_set) | |
| 1143 | + continue; | |
| 1131 | 1144 | dpll = ref->dpll; |
| 1132 | 1145 | if (ops->frequency_set(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| 1133 | 1146 | dpll, dpll_priv(dpll), old_freq, extack)) |
| @@ -1151,13 +1164,19 @@ dpll_pin_esync_set(struct dpll_pin *pin, struct nlattr *a, · 17 unmodified lines | ||
| 1151 | 1164 | |
| 1152 | 1165 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1153 | 1166 | ops = dpll_pin_ops(ref); |
| 1154 | − if (!ops->esync_set || !ops->esync_get) { | |
| 1167 | + if ((!ops->esync_set || !ops->esync_get) && | |
| 1168 | + ref->dpll->module == pin->module && | |
| 1169 | + ref->dpll->clock_id == pin->clock_id) { | |
| 1155 | 1170 | NL_SET_ERR_MSG(extack, |
| 1156 | 1171 | "embedded sync feature is not supported by this device"); |
| 1157 | 1172 | return -EOPNOTSUPP; |
| 1158 | 1173 | } |
| 1159 | 1174 | } |
| 1160 | − ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | |
| 1175 | + ref = dpll_pin_own_dpll_ref_first(pin); | |
| 1176 | + if (!ref) { | |
| 1177 | + NL_SET_ERR_MSG(extack, "pin owner dpll not found"); | |
| 1178 | + return -ENODEV; | |
| 1179 | + } | |
| 1161 | 1180 | ops = dpll_pin_ops(ref); |
| 1162 | 1181 | dpll = ref->dpll; |
| 1163 | 1182 | ret = ops->esync_get(pin, dpll_pin_on_dpll_priv(dpll, pin), dpll, |
| @@ -1181,6 +1200,8 @@ dpll_pin_esync_set(struct dpll_pin *pin, struct nlattr *a, · 17 unmodified lines | ||
| 1181 | 1200 | void *pin_dpll_priv; |
| 1182 | 1201 | |
| 1183 | 1202 | ops = dpll_pin_ops(ref); |
| 1203 | + if (!ops->esync_set) | |
| 1204 | + continue; | |
| 1184 | 1205 | dpll = ref->dpll; |
| 1185 | 1206 | pin_dpll_priv = dpll_pin_on_dpll_priv(dpll, pin); |
| 1186 | 1207 | ret = ops->esync_set(pin, pin_dpll_priv, dpll, dpll_priv(dpll), |
| @@ -1204,6 +1225,8 @@ rollback: · 17 unmodified lines | ||
| 1204 | 1225 | if (ref == failed) |
| 1205 | 1226 | break; |
| 1206 | 1227 | ops = dpll_pin_ops(ref); |
| 1228 | + if (!ops->esync_set) | |
| 1229 | + continue; | |
| 1207 | 1230 | dpll = ref->dpll; |
| 1208 | 1231 | pin_dpll_priv = dpll_pin_on_dpll_priv(dpll, pin); |
| 1209 | 1232 | if (ops->esync_set(pin, pin_dpll_priv, dpll, dpll_priv(dpll), |
| @@ -1238,8 +1261,11 @@ dpll_pin_ref_sync_state_set(struct dpll_pin *pin, · 28 unmodified lines | ||
| 1238 | 1261 | NL_SET_ERR_MSG(extack, "reference sync pin not available"); |
| 1239 | 1262 | return -EINVAL; |
| 1240 | 1263 | } |
| 1241 | − ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | |
| 1242 | − ASSERT_NOT_NULL(ref); | |
| 1264 | + ref = dpll_pin_own_dpll_ref_first(pin); | |
| 1265 | + if (!ref) { | |
| 1266 | + NL_SET_ERR_MSG(extack, "pin owner dpll not found"); | |
| 1267 | + return -ENODEV; | |
| 1268 | + } | |
| 1243 | 1269 | ops = dpll_pin_ops(ref); |
| 1244 | 1270 | if (!ops->ref_sync_set || !ops->ref_sync_get) { |
| 1245 | 1271 | NL_SET_ERR_MSG(extack, "reference sync not supported by this pin"); |
| @@ -1258,6 +1284,8 @@ dpll_pin_ref_sync_state_set(struct dpll_pin *pin, · 12 unmodified lines | ||
| 1258 | 1284 | return 0; |
| 1259 | 1285 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1260 | 1286 | ops = dpll_pin_ops(ref); |
| 1287 | + if (!ops->ref_sync_set) | |
| 1288 | + continue; | |
| 1261 | 1289 | dpll = ref->dpll; |
| 1262 | 1290 | ret = ops->ref_sync_set(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| 1263 | 1291 | ref_sync_pin, |
| @@ -1280,6 +1308,8 @@ rollback: · 16 unmodified lines | ||
| 1280 | 1308 | if (ref == failed) |
| 1281 | 1309 | break; |
| 1282 | 1310 | ops = dpll_pin_ops(ref); |
| 1311 | + if (!ops->ref_sync_set) | |
| 1312 | + continue; | |
| 1283 | 1313 | dpll = ref->dpll; |
| 1284 | 1314 | if (ops->ref_sync_set(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| 1285 | 1315 | ref_sync_pin, |
| @@ -1471,12 +1501,18 @@ dpll_pin_phase_adj_set(struct dpll_pin *pin, struct nlattr *phase_adj_attr, · 185 unmodified lines | ||
| 1471 | 1501 | |
| 1472 | 1502 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1473 | 1503 | ops = dpll_pin_ops(ref); |
| 1474 | − if (!ops->phase_adjust_set || !ops->phase_adjust_get) { | |
| 1504 | + if ((!ops->phase_adjust_set || !ops->phase_adjust_get) && | |
| 1505 | + ref->dpll->module == pin->module && | |
| 1506 | + ref->dpll->clock_id == pin->clock_id) { | |
| 1475 | 1507 | NL_SET_ERR_MSG(extack, "phase adjust not supported"); |
| 1476 | 1508 | return -EOPNOTSUPP; |
| 1477 | 1509 | } |
| 1478 | 1510 | } |
| 1479 | − ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | |
| 1511 | + ref = dpll_pin_own_dpll_ref_first(pin); | |
| 1512 | + if (!ref) { | |
| 1513 | + NL_SET_ERR_MSG(extack, "pin owner dpll not found"); | |
| 1514 | + return -ENODEV; | |
| 1515 | + } | |
| 1480 | 1516 | ops = dpll_pin_ops(ref); |
| 1481 | 1517 | dpll = ref->dpll; |
| 1482 | 1518 | ret = ops->phase_adjust_get(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| @@ -1491,6 +1527,8 @@ dpll_pin_phase_adj_set(struct dpll_pin *pin, struct nlattr *phase_adj_attr, · 8 unmodified lines | ||
| 1491 | 1527 | |
| 1492 | 1528 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1493 | 1529 | ops = dpll_pin_ops(ref); |
| 1530 | + if (!ops->phase_adjust_set) | |
| 1531 | + continue; | |
| 1494 | 1532 | dpll = ref->dpll; |
| 1495 | 1533 | ret = ops->phase_adjust_set(pin, |
| 1496 | 1534 | dpll_pin_on_dpll_priv(dpll, pin), |
| @@ -1513,6 +1551,8 @@ rollback: · 16 unmodified lines | ||
| 1513 | 1551 | if (ref == failed) |
| 1514 | 1552 | break; |
| 1515 | 1553 | ops = dpll_pin_ops(ref); |
| 1554 | + if (!ops->phase_adjust_set) | |
| 1555 | + continue; | |
| 1516 | 1556 | dpll = ref->dpll; |
| 1517 | 1557 | if (ops->phase_adjust_set(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| 1518 | 1558 | dpll, dpll_priv(dpll), old_phase_adj, |
| @@ -27,12 +27,6 @@ static int pin_control_gpio_get_direction(struct gpio_chip *gc, unsigned int off · 26 unmodified lines | ||
| 27 | 27 | return GPIO_LINE_DIRECTION_IN; |
| 28 | 28 | } |
| 29 | 29 | |
| 30 | −static int pin_control_gpio_direction_output(struct gpio_chip *chip, | |
| 31 | − unsigned int offset, int val) | |
| 32 | −{ | |
| 33 | − return pinctrl_gpio_direction_output(chip, offset); | |
| 34 | −} | |
| 35 | − | |
| 36 | 30 | static int pin_control_gpio_get(struct gpio_chip *chip, unsigned int offset) |
| 37 | 31 | { |
| 38 | 32 | unsigned long config; |
| @@ -55,6 +49,18 @@ static int pin_control_gpio_set(struct gpio_chip *chip, unsigned int offset, · 16 unmodified lines | ||
| 55 | 49 | return pinctrl_gpio_set_config(chip, offset, config); |
| 56 | 50 | } |
| 57 | 51 | |
| 52 | +static int pin_control_gpio_direction_output(struct gpio_chip *chip, | |
| 53 | + unsigned int offset, int val) | |
| 54 | +{ | |
| 55 | + int ret; | |
| 56 | + | |
| 57 | + ret = pinctrl_gpio_direction_output(chip, offset); | |
| 58 | + if (ret) | |
| 59 | + return ret; | |
| 60 | + | |
| 61 | + return pin_control_gpio_set(chip, offset, val); | |
| 62 | +} | |
| 63 | + | |
| 58 | 64 | static int pin_control_gpio_probe(struct platform_device *pdev) |
| 59 | 65 | { |
| 60 | 66 | struct device *dev = &pdev->dev; |
| @@ -604,20 +604,28 @@ static int pca953x_read_regs(struct pca953x_chip *chip, int reg, unsigned long * · 603 unmodified lines | ||
| 604 | 604 | return 0; |
| 605 | 605 | } |
| 606 | 606 | |
| 607 | −static int pca953x_gpio_direction_input(struct gpio_chip *gc, unsigned off) | |
| 607 | +static int pca953x_gpio_direction_input_unlocked(struct gpio_chip *gc, | |
| 608 | + unsigned int off) | |
| 608 | 609 | { |
| 609 | 610 | struct pca953x_chip *chip = gpiochip_get_data(gc); |
| 610 | 611 | u8 dirreg = chip->recalc_addr(chip, chip->regs->direction, off); |
| 611 | 612 | u8 bit = pca953x_get_bit_mask(chip, off); |
| 612 | 613 | |
| 613 | − guard(mutex)(&chip->i2c_lock); | |
| 614 | − | |
| 615 | 614 | if (PCA_CHIP_TYPE(chip->driver_data) == TCA6418_TYPE) |
| 616 | 615 | return regmap_update_bits(chip->regmap, dirreg, bit, 0); |
| 617 | 616 | |
| 618 | 617 | return regmap_update_bits(chip->regmap, dirreg, bit, bit); |
| 619 | 618 | } |
| 620 | 619 | |
| 620 | +static int pca953x_gpio_direction_input(struct gpio_chip *gc, unsigned int off) | |
| 621 | +{ | |
| 622 | + struct pca953x_chip *chip = gpiochip_get_data(gc); | |
| 623 | + | |
| 624 | + guard(mutex)(&chip->i2c_lock); | |
| 625 | + | |
| 626 | + return pca953x_gpio_direction_input_unlocked(gc, off); | |
| 627 | +} | |
| 628 | + | |
| 621 | 629 | static int pca953x_gpio_direction_output(struct gpio_chip *gc, |
| 622 | 630 | unsigned off, int val) |
| 623 | 631 | { |
| @@ -855,9 +863,10 @@ static void pca953x_irq_bus_sync_unlock(struct irq_data *d) · 231 unmodified lines | ||
| 855 | 863 | DECLARE_BITMAP(reg_direction, MAX_LINE); |
| 856 | 864 | int level; |
| 857 | 865 | |
| 866 | + guard(mutex)(&chip->i2c_lock); | |
| 867 | + | |
| 858 | 868 | if (chip->driver_data & PCA_PCAL) { |
| 859 | 869 | DECLARE_BITMAP(latched_inputs, MAX_LINE); |
| 860 | − guard(mutex)(&chip->i2c_lock); | |
| 861 | 870 | |
| 862 | 871 | /* Enable latch on edge-triggered interrupt-enabled inputs */ |
| 863 | 872 | bitmap_or(latched_inputs, chip->irq_trig_fall, chip->irq_trig_raise, gc->ngpio); |
| @@ -879,7 +888,7 @@ static void pca953x_irq_bus_sync_unlock(struct irq_data *d) · 15 unmodified lines | ||
| 879 | 888 | |
| 880 | 889 | /* Look for any newly setup interrupt */ |
| 881 | 890 | for_each_andnot_bit(level, irq_mask, reg_direction, gc->ngpio) |
| 882 | − pca953x_gpio_direction_input(&chip->gpio_chip, level); | |
| 891 | + pca953x_gpio_direction_input_unlocked(&chip->gpio_chip, level); | |
| 883 | 892 | |
| 884 | 893 | mutex_unlock(&chip->irq_lock); |
| 885 | 894 | } |
| @@ -1369,9 +1378,20 @@ static int pca953x_restore_context(struct pca953x_chip *chip) · 483 unmodified lines | ||
| 1369 | 1378 | regcache_mark_dirty(chip->regmap); |
| 1370 | 1379 | ret = pca953x_regcache_sync(chip); |
| 1371 | 1380 | if (ret) |
| 1372 | − return ret; | |
| 1381 | + goto err; | |
| 1382 | + | |
| 1383 | + ret = regcache_sync(chip->regmap); | |
| 1384 | + if (ret) | |
| 1385 | + goto err; | |
| 1386 | + | |
| 1387 | + return 0; | |
| 1373 | 1388 | |
| 1374 | − return regcache_sync(chip->regmap); | |
| 1389 | +err: | |
| 1390 | + if (chip->client->irq > 0) | |
| 1391 | + disable_irq(chip->client->irq); | |
| 1392 | + regcache_cache_only(chip->regmap, true); | |
| 1393 | + | |
| 1394 | + return ret; | |
| 1375 | 1395 | } |
| 1376 | 1396 | |
| 1377 | 1397 | static void pca953x_save_context(struct pca953x_chip *chip) |
| @@ -96,7 +96,7 @@ struct pch_gpio { · 95 unmodified lines | ||
| 96 | 96 | struct pch_gpio_reg_data pch_gpio_reg; |
| 97 | 97 | int irq_base; |
| 98 | 98 | enum pch_type_t ioh; |
| 99 | − spinlock_t spinlock; | |
| 99 | + raw_spinlock_t spinlock; | |
| 100 | 100 | }; |
| 101 | 101 | |
| 102 | 102 | static int pch_gpio_set(struct gpio_chip *gpio, unsigned int nr, int val) |
| @@ -105,7 +105,7 @@ static int pch_gpio_set(struct gpio_chip *gpio, unsigned int nr, int val) · 2 unmodified lines | ||
| 105 | 105 | struct pch_gpio *chip = gpiochip_get_data(gpio); |
| 106 | 106 | unsigned long flags; |
| 107 | 107 | |
| 108 | − spin_lock_irqsave(&chip->spinlock, flags); | |
| 108 | + raw_spin_lock_irqsave(&chip->spinlock, flags); | |
| 109 | 109 | reg_val = ioread32(&chip->reg->po); |
| 110 | 110 | if (val) |
| 111 | 111 | reg_val |= BIT(nr); |
| @@ -113,7 +113,7 @@ static int pch_gpio_set(struct gpio_chip *gpio, unsigned int nr, int val) · 1 unmodified line | ||
| 113 | 113 | reg_val &= ~BIT(nr); |
| 114 | 114 | |
| 115 | 115 | iowrite32(reg_val, &chip->reg->po); |
| 116 | − spin_unlock_irqrestore(&chip->spinlock, flags); | |
| 116 | + raw_spin_unlock_irqrestore(&chip->spinlock, flags); | |
| 117 | 117 | |
| 118 | 118 | return 0; |
| 119 | 119 | } |
| @@ -133,7 +133,7 @@ static int pch_gpio_direction_output(struct gpio_chip *gpio, unsigned int nr, · 13 unmodified lines | ||
| 133 | 133 | u32 reg_val; |
| 134 | 134 | unsigned long flags; |
| 135 | 135 | |
| 136 | − spin_lock_irqsave(&chip->spinlock, flags); | |
| 136 | + raw_spin_lock_irqsave(&chip->spinlock, flags); | |
| 137 | 137 | |
| 138 | 138 | reg_val = ioread32(&chip->reg->po); |
| 139 | 139 | if (val) |
| @@ -147,7 +147,7 @@ static int pch_gpio_direction_output(struct gpio_chip *gpio, unsigned int nr, · 7 unmodified lines | ||
| 147 | 147 | pm |= BIT(nr); |
| 148 | 148 | iowrite32(pm, &chip->reg->pm); |
| 149 | 149 | |
| 150 | − spin_unlock_irqrestore(&chip->spinlock, flags); | |
| 150 | + raw_spin_unlock_irqrestore(&chip->spinlock, flags); | |
| 151 | 151 | |
| 152 | 152 | return 0; |
| 153 | 153 | } |
| @@ -158,12 +158,12 @@ static int pch_gpio_direction_input(struct gpio_chip *gpio, unsigned int nr) · 4 unmodified lines | ||
| 158 | 158 | u32 pm; |
| 159 | 159 | unsigned long flags; |
| 160 | 160 | |
| 161 | − spin_lock_irqsave(&chip->spinlock, flags); | |
| 161 | + raw_spin_lock_irqsave(&chip->spinlock, flags); | |
| 162 | 162 | pm = ioread32(&chip->reg->pm); |
| 163 | 163 | pm &= BIT(gpio_pins[chip->ioh]) - 1; |
| 164 | 164 | pm &= ~BIT(nr); |
| 165 | 165 | iowrite32(pm, &chip->reg->pm); |
| 166 | − spin_unlock_irqrestore(&chip->spinlock, flags); | |
| 166 | + raw_spin_unlock_irqrestore(&chip->spinlock, flags); | |
| 167 | 167 | |
| 168 | 168 | return 0; |
| 169 | 169 | } |
| @@ -265,7 +265,7 @@ static int pch_irq_type(struct irq_data *d, unsigned int type) · 95 unmodified lines | ||
| 265 | 265 | return 0; |
| 266 | 266 | } |
| 267 | 267 | |
| 268 | − spin_lock_irqsave(&chip->spinlock, flags); | |
| 268 | + raw_spin_lock_irqsave(&chip->spinlock, flags); | |
| 269 | 269 | |
| 270 | 270 | /* Set interrupt mode */ |
| 271 | 271 | im = ioread32(im_reg) & ~(PCH_IM_MASK << (im_pos * 4)); |
| @@ -277,7 +277,7 @@ static int pch_irq_type(struct irq_data *d, unsigned int type) · 5 unmodified lines | ||
| 277 | 277 | else if (type & IRQ_TYPE_EDGE_BOTH) |
| 278 | 278 | irq_set_handler_locked(d, handle_edge_irq); |
| 279 | 279 | |
| 280 | − spin_unlock_irqrestore(&chip->spinlock, flags); | |
| 280 | + raw_spin_unlock_irqrestore(&chip->spinlock, flags); | |
| 281 | 281 | return 0; |
| 282 | 282 | } |
| 283 | 283 | |
| @@ -374,7 +374,7 @@ static int pch_gpio_probe(struct pci_dev *pdev, · 90 unmodified lines | ||
| 374 | 374 | chip->ioh = id->driver_data; |
| 375 | 375 | chip->reg = chip->base; |
| 376 | 376 | pci_set_drvdata(pdev, chip); |
| 377 | − spin_lock_init(&chip->spinlock); | |
| 377 | + raw_spin_lock_init(&chip->spinlock); | |
| 378 | 378 | pch_gpio_setup(chip); |
| 379 | 379 | |
| 380 | 380 | ret = devm_gpiochip_add_data(dev, &chip->gpio, chip); |
| @@ -407,9 +407,9 @@ static int pch_gpio_suspend(struct device *dev) · 26 unmodified lines | ||
| 407 | 407 | struct pch_gpio *chip = dev_get_drvdata(dev); |
| 408 | 408 | unsigned long flags; |
| 409 | 409 | |
| 410 | − spin_lock_irqsave(&chip->spinlock, flags); | |
| 410 | + raw_spin_lock_irqsave(&chip->spinlock, flags); | |
| 411 | 411 | pch_gpio_save_reg_conf(chip); |
| 412 | − spin_unlock_irqrestore(&chip->spinlock, flags); | |
| 412 | + raw_spin_unlock_irqrestore(&chip->spinlock, flags); | |
| 413 | 413 | |
| 414 | 414 | return 0; |
| 415 | 415 | } |
| @@ -419,11 +419,11 @@ static int pch_gpio_resume(struct device *dev) · 3 unmodified lines | ||
| 419 | 419 | struct pch_gpio *chip = dev_get_drvdata(dev); |
| 420 | 420 | unsigned long flags; |
| 421 | 421 | |
| 422 | − spin_lock_irqsave(&chip->spinlock, flags); | |
| 422 | + raw_spin_lock_irqsave(&chip->spinlock, flags); | |
| 423 | 423 | iowrite32(0x01, &chip->reg->reset); |
| 424 | 424 | iowrite32(0x00, &chip->reg->reset); |
| 425 | 425 | pch_gpio_restore_reg_conf(chip); |
| 426 | − spin_unlock_irqrestore(&chip->spinlock, flags); | |
| 426 | + raw_spin_unlock_irqrestore(&chip->spinlock, flags); | |
| 427 | 427 | |
| 428 | 428 | return 0; |
| 429 | 429 | } |
| @@ -160,6 +160,13 @@ static int fops_buf_size_get(void *data, u64 *val) · 159 unmodified lines | ||
| 160 | 160 | return 0; |
| 161 | 161 | } |
| 162 | 162 | |
| 163 | +static void fops_buf_release(void *data) | |
| 164 | +{ | |
| 165 | + struct gpio_la_poll_priv *priv = data; | |
| 166 | + | |
| 167 | + vfree(priv->blob.data); | |
| 168 | +} | |
| 169 | + | |
| 163 | 170 | static int fops_buf_size_set(void *data, u64 val) |
| 164 | 171 | { |
| 165 | 172 | struct gpio_la_poll_priv *priv = data; |
| @@ -238,6 +245,9 @@ static int gpio_la_poll_probe(struct platform_device *pdev) · 72 unmodified lines | ||
| 238 | 245 | return ret; |
| 239 | 246 | |
| 240 | 247 | fops_buf_size_set(priv, GPIO_LA_DEFAULT_BUF_SIZE); |
| 248 | + ret = devm_add_action_or_reset(dev, fops_buf_release, priv); | |
| 249 | + if (ret) | |
| 250 | + return ret; | |
| 241 | 251 | |
| 242 | 252 | priv->descs = devm_gpiod_get_array(dev, "probe", GPIOD_IN); |
| 243 | 253 | if (IS_ERR(priv->descs)) |
| @@ -999,14 +999,17 @@ int gpiochip_add_hog(struct gpio_chip *gc, struct fwnode_handle *fwnode) · 998 unmodified lines | ||
| 999 | 999 | if (ret < 0) |
| 1000 | 1000 | return ret; |
| 1001 | 1001 | |
| 1002 | − if (fwnode_property_present(fwnode, "input")) | |
| 1002 | + if (fwnode_property_present(fwnode, "input")) { | |
| 1003 | 1003 | dflags |= GPIOD_IN; |
| 1004 | − else if (fwnode_property_present(fwnode, "output-low")) | |
| 1004 | + } else if (fwnode_property_present(fwnode, "output-low")) { | |
| 1005 | 1005 | dflags |= GPIOD_OUT_LOW; |
| 1006 | − else if (fwnode_property_present(fwnode, "output-high")) | |
| 1006 | + } else if (fwnode_property_present(fwnode, "output-high")) { | |
| 1007 | 1007 | dflags |= GPIOD_OUT_HIGH; |
| 1008 | − else | |
| 1009 | − return -EINVAL; | |
| 1008 | + } else { | |
| 1009 | + gpiochip_warn(gc, "%pfwP: no hogging state specified, bailing out\n", | |
| 1010 | + fwnode); | |
| 1011 | + return 0; | |
| 1012 | + } | |
| 1010 | 1013 | |
| 1011 | 1014 | fwnode_property_read_string(fwnode, "line-name", &name); |
| 1012 | 1015 | |
| @@ -92,7 +92,7 @@ struct amdgpu_amdkfd_fence *to_amdgpu_amdkfd_fence(struct dma_fence *f) · 91 unmodified lines | ||
| 92 | 92 | return NULL; |
| 93 | 93 | |
| 94 | 94 | fence = container_of(f, struct amdgpu_amdkfd_fence, base); |
| 95 | − if (f->ops == &amdkfd_fence_ops) | |
| 95 | + if (rcu_access_pointer(f->ops) == &amdkfd_fence_ops) | |
| 96 | 96 | return fence; |
| 97 | 97 | |
| 98 | 98 | return NULL; |
| @@ -3096,7 +3096,7 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence __rcu * · 3095 unmodified lines | ||
| 3096 | 3096 | process_info->eviction_fence = new_fence; |
| 3097 | 3097 | replace_eviction_fence(ef, dma_fence_get(&new_fence->base)); |
| 3098 | 3098 | } else { |
| 3099 | − WARN_ONCE(*ef != &process_info->eviction_fence->base, | |
| 3099 | + WARN_ONCE(rcu_access_pointer(*ef) != &process_info->eviction_fence->base, | |
| 3100 | 3100 | "KFD eviction fence doesn't match KGD process_info"); |
| 3101 | 3101 | } |
| 3102 | 3102 | |
| @@ -42,6 +42,26 @@ · 41 unmodified lines | ||
| 42 | 42 | #include "amdgpu_ras.h" |
| 43 | 43 | #include "amdgpu_hmm.h" |
| 44 | 44 | |
| 45 | +/* | |
| 46 | + * Maximum IB length (dwords) for rings whose emit_ib packet format | |
| 47 | + * documents a 20-bit size field. | |
| 48 | + */ | |
| 49 | +#define AMDGPU_GFX_SDMA_IB_PACKET_SIZE_MAX_DW 0xFFFFF | |
| 50 | +#define AMDGPU_MM_IB_PACKET_SIZE_MAX_DW 0x7FFFF0 | |
| 51 | + | |
| 52 | +static u32 amdgpu_cs_ib_packet_size_max_dw(enum amdgpu_ring_type type) | |
| 53 | +{ | |
| 54 | + switch (type) { | |
| 55 | + case AMDGPU_RING_TYPE_GFX: | |
| 56 | + case AMDGPU_RING_TYPE_COMPUTE: | |
| 57 | + case AMDGPU_RING_TYPE_SDMA: | |
| 58 | + case AMDGPU_RING_TYPE_VPE: | |
| 59 | + return AMDGPU_GFX_SDMA_IB_PACKET_SIZE_MAX_DW; | |
| 60 | + default: | |
| 61 | + return AMDGPU_MM_IB_PACKET_SIZE_MAX_DW; | |
| 62 | + } | |
| 63 | +} | |
| 64 | + | |
| 45 | 65 | static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, |
| 46 | 66 | struct amdgpu_device *adev, |
| 47 | 67 | struct drm_file *filp, |
| @@ -345,7 +365,6 @@ static int amdgpu_cs_p2_ib(struct amdgpu_cs_parser *p, · 297 unmodified lines | ||
| 345 | 365 | |
| 346 | 366 | job = p->jobs[r]; |
| 347 | 367 | ring = amdgpu_job_ring(job); |
| 348 | − ib = &job->ibs[job->num_ibs++]; | |
| 349 | 368 | |
| 350 | 369 | /* submissions to kernel queues are disabled */ |
| 351 | 370 | if (ring->no_user_submission) |
| @@ -374,6 +393,12 @@ static int amdgpu_cs_p2_ib(struct amdgpu_cs_parser *p, · 22 unmodified lines | ||
| 374 | 393 | return -EINVAL; |
| 375 | 394 | } |
| 376 | 395 | |
| 396 | + if (chunk_ib->ib_bytes / 4 > | |
| 397 | + amdgpu_cs_ib_packet_size_max_dw(ring->funcs->type)) | |
| 398 | + return -EINVAL; | |
| 399 | + | |
| 400 | + ib = &job->ibs[job->num_ibs++]; | |
| 401 | + | |
| 377 | 402 | if (chunk_ib->flags & AMDGPU_IB_FLAG_PREAMBLE) |
| 378 | 403 | job->preamble_status |= AMDGPU_PREAMBLE_IB_PRESENT; |
| 379 | 404 | |
| @@ -1177,19 +1202,6 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p) · 797 unmodified lines | ||
| 1177 | 1202 | job->vm_pd_addr = amdgpu_gmc_pd_addr(vm->root.bo); |
| 1178 | 1203 | } |
| 1179 | 1204 | |
| 1180 | − if (adev->debug_vm) { | |
| 1181 | − /* Invalidate all BOs to test for userspace bugs */ | |
| 1182 | − amdgpu_bo_list_for_each_entry(e, p->bo_list) { | |
| 1183 | − struct amdgpu_bo *bo = e->bo; | |
| 1184 | − | |
| 1185 | − /* ignore duplicates */ | |
| 1186 | − if (!bo) | |
| 1187 | − continue; | |
| 1188 | − | |
| 1189 | − amdgpu_vm_bo_invalidate(bo, false); | |
| 1190 | − } | |
| 1191 | − } | |
| 1192 | − | |
| 1193 | 1205 | return 0; |
| 1194 | 1206 | } |
| 1195 | 1207 | |
| @@ -1378,6 +1390,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p, · 182 unmodified lines | ||
| 1378 | 1390 | /* Cleanup the parser structure */ |
| 1379 | 1391 | static void amdgpu_cs_parser_fini(struct amdgpu_cs_parser *parser) |
| 1380 | 1392 | { |
| 1393 | + struct amdgpu_device *adev = parser->adev; | |
| 1394 | + struct amdgpu_bo_list_entry *e; | |
| 1381 | 1395 | unsigned int i; |
| 1382 | 1396 | |
| 1383 | 1397 | amdgpu_sync_free(&parser->sync); |
| @@ -1393,8 +1407,21 @@ static void amdgpu_cs_parser_fini(struct amdgpu_cs_parser *parser) · 9 unmodified lines | ||
| 1393 | 1407 | |
| 1394 | 1408 | if (parser->ctx) |
| 1395 | 1409 | amdgpu_ctx_put(parser->ctx); |
| 1396 | − if (parser->bo_list) | |
| 1410 | + if (parser->bo_list) { | |
| 1411 | + if (adev->debug_vm) { | |
| 1412 | + /* Invalidate all BOs to test for userspace bugs */ | |
| 1413 | + amdgpu_bo_list_for_each_entry(e, parser->bo_list) { | |
| 1414 | + struct amdgpu_bo *bo = e->bo; | |
| 1415 | + | |
| 1416 | + /* ignore duplicates */ | |
| 1417 | + if (!bo) | |
| 1418 | + continue; | |
| 1419 | + | |
| 1420 | + amdgpu_vm_bo_invalidate(bo, false); | |
| 1421 | + } | |
| 1422 | + } | |
| 1397 | 1423 | amdgpu_bo_list_put(parser->bo_list); |
| 1424 | + } | |
| 1398 | 1425 | |
| 1399 | 1426 | for (i = 0; i < parser->nchunks; i++) |
| 1400 | 1427 | kvfree(parser->chunks[i].kdata); |
| @@ -43,6 +43,11 @@ · 42 unmodified lines | ||
| 43 | 43 | |
| 44 | 44 | #if defined(CONFIG_DEBUG_FS) |
| 45 | 45 | |
| 46 | +/* Encode milliwatts in the raw Q24.8 sensor report format used by UMR. */ | |
| 47 | +#define AMDGPU_DEBUGFS_PWR_MW_TO_Q24_8(power_mw) \ | |
| 48 | + DIV_ROUND_CLOSEST_ULL((u64)(power_mw) * BIT(8), \ | |
| 49 | + MILLIWATT_PER_WATT) | |
| 50 | + | |
| 46 | 51 | /** |
| 47 | 52 | * amdgpu_debugfs_process_reg_op - Handle MMIO register reads/writes |
| 48 | 53 | * |
| @@ -1104,6 +1109,10 @@ static ssize_t amdgpu_debugfs_sensor_read(struct file *f, char __user *buf, · 1055 unmodified lines | ||
| 1104 | 1109 | return r; |
| 1105 | 1110 | } |
| 1106 | 1111 | |
| 1112 | + if (idx == AMDGPU_PP_SENSOR_GPU_AVG_POWER || | |
| 1113 | + idx == AMDGPU_PP_SENSOR_GPU_INPUT_POWER) | |
| 1114 | + values[0] = AMDGPU_DEBUGFS_PWR_MW_TO_Q24_8(values[0]); | |
| 1115 | + | |
| 1107 | 1116 | if (size > valuesize) { |
| 1108 | 1117 | amdgpu_virt_disable_access_debugfs(adev); |
| 1109 | 1118 | return -EINVAL; |
| @@ -1319,8 +1328,8 @@ err: · 209 unmodified lines | ||
| 1319 | 1328 | * @size: Number of bytes to read |
| 1320 | 1329 | * @pos: Offset to seek to |
| 1321 | 1330 | * |
| 1322 | − * Read the last residency value logged. It doesn't auto update, one needs to | |
| 1323 | − * stop logging before getting the current value. | |
| 1331 | + * Read a live GFXOFF residency sample from firmware. One needs to start logging | |
| 1332 | + * before getting the current value. | |
| 1324 | 1333 | */ |
| 1325 | 1334 | static ssize_t amdgpu_debugfs_gfxoff_residency_read(struct file *f, char __user *buf, |
| 1326 | 1335 | size_t size, loff_t *pos) |
| @@ -342,7 +342,7 @@ amdgpu_devcoredump_format(char *buffer, size_t count, struct amdgpu_coredump_inf · 341 unmodified lines | ||
| 342 | 342 | struct amdgpu_ip_block *ip_block; |
| 343 | 343 | struct amdgpu_ring *ring; |
| 344 | 344 | int ver, i, j; |
| 345 | − u32 ring_idx, off; | |
| 345 | + u32 ring_idx; | |
| 346 | 346 | bool sizing_pass; |
| 347 | 347 | |
| 348 | 348 | sizing_pass = buffer == NULL; |
| @@ -442,7 +442,6 @@ amdgpu_devcoredump_format(char *buffer, size_t count, struct amdgpu_coredump_inf · 93 unmodified lines | ||
| 442 | 442 | for (i = 0; i < coredump->num_rings; i++) { |
| 443 | 443 | ring_idx = coredump->rings[i].ring_index; |
| 444 | 444 | ring = coredump->adev->rings[ring_idx]; |
| 445 | − off = coredump->rings[i].offset; | |
| 446 | 445 | |
| 447 | 446 | drm_printf(&p, "ring name: %s\n", ring->name); |
| 448 | 447 | drm_printf(&p, "Rptr: 0x%llx Wptr: 0x%llx RB mask: %x\n", |
| @@ -451,12 +450,18 @@ amdgpu_devcoredump_format(char *buffer, size_t count, struct amdgpu_coredump_inf · 2 unmodified lines | ||
| 451 | 450 | ring->buf_mask); |
| 452 | 451 | drm_printf(&p, "Ring size in dwords: %d\n", |
| 453 | 452 | ring->ring_size / 4); |
| 453 | + | |
| 454 | + if (!coredump->rings[i].ring_dw) { | |
| 455 | + drm_printf(&p, "Ring contents unavailable\n"); | |
| 456 | + continue; | |
| 457 | + } | |
| 458 | + | |
| 454 | 459 | drm_printf(&p, "Ring contents\n"); |
| 455 | 460 | drm_printf(&p, "Offset \t Value\n"); |
| 456 | 461 | |
| 457 | 462 | for (j = 0; j < ring->ring_size; j += 4) |
| 458 | 463 | drm_printf(&p, "0x%x \t 0x%x\n", j, |
| 459 | − coredump->rings_dw[off + j / 4]); | |
| 464 | + coredump->rings[i].ring_dw[j / 4]); | |
| 460 | 465 | } |
| 461 | 466 | } |
| 462 | 467 | |
| @@ -497,10 +502,12 @@ amdgpu_devcoredump_read(char *buffer, loff_t offset, size_t count, · 34 unmodified lines | ||
| 497 | 502 | static void amdgpu_devcoredump_free(void *data) |
| 498 | 503 | { |
| 499 | 504 | struct amdgpu_coredump_info *coredump = data; |
| 505 | + u32 i; | |
| 500 | 506 | |
| 501 | 507 | kvfree(coredump->formatted); |
| 508 | + for (i = 0; i < coredump->num_rings; i++) | |
| 509 | + kvfree(coredump->rings[i].ring_dw); | |
| 502 | 510 | kvfree(coredump->rings); |
| 503 | − kvfree(coredump->rings_dw); | |
| 504 | 511 | kvfree(data); |
| 505 | 512 | } |
| 506 | 513 | |
| @@ -542,9 +549,9 @@ void amdgpu_coredump(struct amdgpu_device *adev, bool skip_vram_check, · 35 unmodified lines | ||
| 542 | 549 | struct amdgpu_coredump_info *coredump; |
| 543 | 550 | size_t size = sizeof(*coredump); |
| 544 | 551 | struct drm_sched_job *s_job; |
| 545 | − u64 total_ring_size, ring_count; | |
| 552 | + u64 ring_count; | |
| 546 | 553 | struct amdgpu_ring *ring; |
| 547 | − int i, off, idx; | |
| 554 | + int i, idx; | |
| 548 | 555 | |
| 549 | 556 | /* No need to generate a new coredump if there's one in progress already. */ |
| 550 | 557 | if (work_busy(&adev->coredump_work)) |
| @@ -553,7 +560,7 @@ void amdgpu_coredump(struct amdgpu_device *adev, bool skip_vram_check, · 2 unmodified lines | ||
| 553 | 560 | if (job && job->pasid) |
| 554 | 561 | size += sizeof(struct amdgpu_coredump_ib_info) * job->num_ibs; |
| 555 | 562 | |
| 556 | − coredump = kzalloc(size, GFP_NOWAIT); | |
| 563 | + coredump = kvzalloc(size, GFP_NOWAIT); | |
| 557 | 564 | if (!coredump) |
| 558 | 565 | return; |
| 559 | 566 | |
| @@ -584,7 +591,6 @@ void amdgpu_coredump(struct amdgpu_device *adev, bool skip_vram_check, · 24 unmodified lines | ||
| 584 | 591 | |
| 585 | 592 | /* Dump ring content if memory allocation succeeds. */ |
| 586 | 593 | ring_count = 0; |
| 587 | − total_ring_size = 0; | |
| 588 | 594 | for (i = 0; i < adev->num_rings; i++) { |
| 589 | 595 | ring = adev->rings[i]; |
| 590 | 596 | |
| @@ -593,34 +599,34 @@ void amdgpu_coredump(struct amdgpu_device *adev, bool skip_vram_check, · 2 unmodified lines | ||
| 593 | 599 | coredump->ring != ring) |
| 594 | 600 | continue; |
| 595 | 601 | |
| 596 | − total_ring_size += ring->ring_size; | |
| 597 | 602 | ring_count++; |
| 598 | 603 | } |
| 599 | − coredump->rings_dw = kzalloc(total_ring_size, GFP_NOWAIT); | |
| 600 | − coredump->rings = kcalloc(ring_count, sizeof(struct amdgpu_coredump_ring), GFP_NOWAIT); | |
| 601 | − if (coredump->rings && coredump->rings_dw) { | |
| 602 | − for (i = 0, off = 0, idx = 0; i < adev->num_rings && idx < ring_count; i++) { | |
| 604 | + if (ring_count) | |
| 605 | + coredump->rings = kvcalloc(ring_count, | |
| 606 | + sizeof(struct amdgpu_coredump_ring), | |
| 607 | + GFP_NOWAIT); | |
| 608 | + if (coredump->rings) { | |
| 609 | + for (i = 0, idx = 0; i < adev->num_rings && idx < ring_count; i++) { | |
| 610 | + struct amdgpu_coredump_ring *cdump_ring; | |
| 611 | + | |
| 603 | 612 | ring = adev->rings[i]; |
| 604 | 613 | |
| 605 | 614 | if (atomic_read(&ring->fence_drv.last_seq) == ring->fence_drv.sync_seq && |
| 606 | 615 | coredump->ring != ring) |
| 607 | 616 | continue; |
| 608 | 617 | |
| 609 | − coredump->rings[idx].ring_index = ring->idx; | |
| 610 | − coredump->rings[idx].rptr = amdgpu_ring_get_rptr(ring); | |
| 611 | − coredump->rings[idx].wptr = amdgpu_ring_get_wptr(ring); | |
| 612 | − coredump->rings[idx].offset = off; | |
| 618 | + cdump_ring = &coredump->rings[idx]; | |
| 619 | + | |
| 620 | + cdump_ring->ring_dw = kvzalloc(ring->ring_size, GFP_NOWAIT); | |
| 621 | + if (cdump_ring->ring_dw) | |
| 622 | + memcpy(cdump_ring->ring_dw, ring->ring, ring->ring_size); | |
| 613 | 623 | |
| 614 | − memcpy(&coredump->rings_dw[off], ring->ring, ring->ring_size); | |
| 615 | − off += ring->ring_size / 4; | |
| 624 | + cdump_ring->ring_index = ring->idx; | |
| 625 | + cdump_ring->rptr = amdgpu_ring_get_rptr(ring); | |
| 626 | + cdump_ring->wptr = amdgpu_ring_get_wptr(ring); | |
| 616 | 627 | idx++; |
| 617 | 628 | } |
| 618 | 629 | coredump->num_rings = idx; |
| 619 | − } else { | |
| 620 | − kvfree(coredump->rings_dw); | |
| 621 | − kvfree(coredump->rings); | |
| 622 | − coredump->rings_dw = NULL; | |
| 623 | − coredump->rings = NULL; | |
| 624 | 630 | } |
| 625 | 631 | |
| 626 | 632 | coredump->adev = adev; |
| @@ -34,8 +34,8 @@ · 33 unmodified lines | ||
| 34 | 34 | struct amdgpu_coredump_ring { |
| 35 | 35 | u64 rptr; |
| 36 | 36 | u64 wptr; |
| 37 | + u32 *ring_dw; | |
| 37 | 38 | u32 ring_index; |
| 38 | − u32 offset; | |
| 39 | 39 | }; |
| 40 | 40 | |
| 41 | 41 | struct amdgpu_coredump_ib_info { |
| @@ -53,7 +53,6 @@ struct amdgpu_coredump_info { · 11 unmodified lines | ||
| 53 | 53 | struct amdgpu_ring *ring; |
| 54 | 54 | |
| 55 | 55 | struct amdgpu_coredump_ring *rings; |
| 56 | − u32 *rings_dw; | |
| 57 | 56 | u32 num_rings; |
| 58 | 57 | |
| 59 | 58 | /* Readable form of coredevdump, generate once to speed up |
| @@ -4194,6 +4194,8 @@ static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) · 4193 unmodified lines | ||
| 4194 | 4194 | |
| 4195 | 4195 | iounmap(adev->rmmio); |
| 4196 | 4196 | adev->rmmio = NULL; |
| 4197 | + if (adev->mman.aper_base_kaddr) | |
| 4198 | + iounmap(adev->mman.aper_base_kaddr); | |
| 4197 | 4199 | adev->mman.aper_base_kaddr = NULL; |
| 4198 | 4200 | |
| 4199 | 4201 | /* Memory manager related */ |
| @@ -2304,6 +2304,7 @@ static int amdgpu_discovery_set_psp_ip_blocks(struct amdgpu_device *adev) · 2303 unmodified lines | ||
| 2304 | 2304 | amdgpu_device_ip_block_add(adev, &psp_v14_0_ip_block); |
| 2305 | 2305 | break; |
| 2306 | 2306 | case IP_VERSION(15, 0, 0): |
| 2307 | + case IP_VERSION(15, 0, 5): | |
| 2307 | 2308 | case IP_VERSION(15, 0, 9): |
| 2308 | 2309 | amdgpu_device_ip_block_add(adev, &psp_v15_0_ip_block); |
| 2309 | 2310 | break; |
| @@ -2908,6 +2908,19 @@ static int amdgpu_pmops_runtime_suspend(struct device *dev) · 2907 unmodified lines | ||
| 2908 | 2908 | return 0; |
| 2909 | 2909 | } |
| 2910 | 2910 | |
| 2911 | +static void amdgpu_restore_umd_profile_pstate_after_runpm(struct amdgpu_device *adev) | |
| 2912 | +{ | |
| 2913 | + enum amd_dpm_forced_level level; | |
| 2914 | + uint32_t profile_mode_mask = AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD | | |
| 2915 | + AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK | | |
| 2916 | + AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK | | |
| 2917 | + AMD_DPM_FORCED_LEVEL_PROFILE_PEAK; | |
| 2918 | + | |
| 2919 | + level = amdgpu_dpm_get_performance_level(adev); | |
| 2920 | + if (level & profile_mode_mask) | |
| 2921 | + amdgpu_asic_update_umd_stable_pstate(adev, true); | |
| 2922 | +} | |
| 2923 | + | |
| 2911 | 2924 | static int amdgpu_pmops_runtime_resume(struct device *dev) |
| 2912 | 2925 | { |
| 2913 | 2926 | struct pci_dev *pdev = to_pci_dev(dev); |
| @@ -2952,6 +2965,8 @@ static int amdgpu_pmops_runtime_resume(struct device *dev) · 38 unmodified lines | ||
| 2952 | 2965 | |
| 2953 | 2966 | if (adev->pm.rpm_mode == AMDGPU_RUNPM_PX) |
| 2954 | 2967 | drm_dev->switch_power_state = DRM_SWITCH_POWER_ON; |
| 2968 | + | |
| 2969 | + amdgpu_restore_umd_profile_pstate_after_runpm(adev); | |
| 2955 | 2970 | adev->in_runpm = false; |
| 2956 | 2971 | return 0; |
| 2957 | 2972 | } |
| @@ -1210,7 +1210,7 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) · 1209 unmodified lines | ||
| 1210 | 1210 | return -EINVAL; |
| 1211 | 1211 | } |
| 1212 | 1212 | } |
| 1213 | − ui32 >>= 8; | |
| 1213 | + ui32 /= MILLIWATT_PER_WATT; | |
| 1214 | 1214 | break; |
| 1215 | 1215 | case AMDGPU_INFO_SENSOR_GPU_INPUT_POWER: |
| 1216 | 1216 | /* get input GPU power */ |
| @@ -1219,7 +1219,7 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) · 2 unmodified lines | ||
| 1219 | 1219 | (void *)&ui32, &ui32_size)) { |
| 1220 | 1220 | return -EINVAL; |
| 1221 | 1221 | } |
| 1222 | − ui32 >>= 8; | |
| 1222 | + ui32 /= MILLIWATT_PER_WATT; | |
| 1223 | 1223 | break; |
| 1224 | 1224 | case AMDGPU_INFO_SENSOR_VDDNB: |
| 1225 | 1225 | /* get VDDNB in millivolts */ |
| @@ -135,6 +135,21 @@ int amdgpu_lockdep_init(void) · 134 unmodified lines | ||
| 135 | 135 | lockdep_set_class(&locks->srbm_mutex, &amdgpu_srbm_lock_key); |
| 136 | 136 | lockdep_set_class(&locks->grbm_idx_mutex, &amdgpu_grbm_lock_key); |
| 137 | 137 | lockdep_set_class(&locks->mmio_idx_lock, &amdgpu_mmio_lock_key); |
| 138 | + | |
| 139 | + /* | |
| 140 | + * Register fs_reclaim lock class FIRST, before taking any locks. | |
| 141 | + * | |
| 142 | + * This acquire/release pair does NOT create a static lockdep edge | |
| 143 | + * (no locks are held between acquire and release). It only registers | |
| 144 | + * the fs_reclaim lock class with lockdep. | |
| 145 | + * | |
| 146 | + * The actual fs_reclaim -> notifier_lock dependency is established at | |
| 147 | + * RUNTIME when memory reclaim invokes MMU notifiers: | |
| 148 | + * fs_reclaim (held by reclaim) -> notifier_lock (acquired in callback) | |
| 149 | + */ | |
| 150 | + fs_reclaim_acquire(GFP_KERNEL); | |
| 151 | + fs_reclaim_release(GFP_KERNEL); | |
| 152 | + | |
| 138 | 153 | /* |
| 139 | 154 | * Take locks in the correct order to train lockdep. |
| 140 | 155 | * This establishes the dependency chain. |
| @@ -154,11 +169,6 @@ int amdgpu_lockdep_init(void) · 13 unmodified lines | ||
| 154 | 169 | |
| 155 | 170 | /* Level 6: Reset control lock */ |
| 156 | 171 | mutex_lock(&locks->reset_lock); |
| 157 | − /* | |
| 158 | − * Mark potential memory reclaim boundary. | |
| 159 | − * GPU operations might trigger memory allocation/reclaim. | |
| 160 | − */ | |
| 161 | − fs_reclaim_acquire(GFP_KERNEL); | |
| 162 | 172 | |
| 163 | 173 | /* Level 7: SRBM register access */ |
| 164 | 174 | mutex_lock(&locks->srbm_mutex); |
| @@ -176,7 +186,6 @@ int amdgpu_lockdep_init(void) · 11 unmodified lines | ||
| 176 | 186 | spin_unlock_irqrestore(&locks->mmio_idx_lock, flags); |
| 177 | 187 | mutex_unlock(&locks->grbm_idx_mutex); |
| 178 | 188 | mutex_unlock(&locks->srbm_mutex); |
| 179 | − fs_reclaim_release(GFP_KERNEL); | |
| 180 | 189 | |
| 181 | 190 | mutex_unlock(&locks->reset_lock); |
| 182 | 191 | up_read(&reset_domain->sem); |
| @@ -275,6 +275,7 @@ static int psp_early_init(struct amdgpu_ip_block *ip_block) · 274 unmodified lines | ||
| 275 | 275 | psp->boot_time_tmr = false; |
| 276 | 276 | break; |
| 277 | 277 | case IP_VERSION(15, 0, 0): |
| 278 | + case IP_VERSION(15, 0, 5): | |
| 278 | 279 | case IP_VERSION(15, 0, 9): |
| 279 | 280 | psp_v15_0_0_set_psp_funcs(psp); |
| 280 | 281 | psp->boot_time_tmr = false; |
| @@ -1222,6 +1223,33 @@ int psp_memory_partition(struct psp_context *psp, int mode) · 941 unmodified lines | ||
| 1222 | 1223 | return ret; |
| 1223 | 1224 | } |
| 1224 | 1225 | |
| 1226 | +int psp_set_mmhub_eco_sec_level(struct amdgpu_device *adev) | |
| 1227 | +{ | |
| 1228 | + int ret; | |
| 1229 | + struct psp_context *psp = &adev->psp; | |
| 1230 | + struct psp_gfx_cmd_resp *cmd = acquire_psp_cmd_buf(psp); | |
| 1231 | + | |
| 1232 | + cmd->cmd_id = GFX_CMD_ID_SET_MMHUB_ECO_SEC_LEVEL; | |
| 1233 | + | |
| 1234 | + ret = psp_cmd_submit_buf(psp, NULL, cmd, psp->fence_buf_mc_addr); | |
| 1235 | + if (ret) { | |
| 1236 | + dev_err(psp->adev->dev, | |
| 1237 | + "PSP request failed to set mmuhub eco sec level with ret=%d\n", ret); | |
| 1238 | + release_psp_cmd_buf(psp); | |
| 1239 | + return ret; | |
| 1240 | + } | |
| 1241 | + | |
| 1242 | + if (cmd->resp.status) { | |
| 1243 | + dev_err(psp->adev->dev, | |
| 1244 | + "MMHUB ECO SEC LEVEL command 0x%x failed, PSP response status: 0x%X\n", | |
| 1245 | + cmd->cmd_id, cmd->resp.status); | |
| 1246 | + ret = -EIO; | |
| 1247 | + } | |
| 1248 | + release_psp_cmd_buf(psp); | |
| 1249 | + | |
| 1250 | + return ret; | |
| 1251 | +} | |
| 1252 | + | |
| 1225 | 1253 | static int psp_ptl_fmt_verify(struct psp_context *psp, enum amdgpu_ptl_fmt fmt, |
| 1226 | 1254 | uint32_t *ptl_fmt) |
| 1227 | 1255 | { |
| @@ -3476,8 +3504,10 @@ static int psp_load_non_psp_fw(struct psp_context *psp) · 2248 unmodified lines | ||
| 3476 | 3504 | amdgpu_ip_version(adev, MP0_HWIP, 0) == |
| 3477 | 3505 | IP_VERSION(15, 0, 0) || |
| 3478 | 3506 | amdgpu_ip_version(adev, MP0_HWIP, 0) == |
| 3479 | − IP_VERSION(15, 0, 8) || | |
| 3507 | + IP_VERSION(15, 0, 5) || | |
| 3480 | 3508 | amdgpu_ip_version(adev, MP0_HWIP, 0) == |
| 3509 | + IP_VERSION(15, 0, 8) || | |
| 3510 | + amdgpu_ip_version(adev, MP0_HWIP, 0) == | |
| 3481 | 3511 | IP_VERSION(15, 0, 9)) && |
| 3482 | 3512 | (ucode->ucode_id == AMDGPU_UCODE_ID_SDMA1 || |
| 3483 | 3513 | ucode->ucode_id == AMDGPU_UCODE_ID_SDMA2 || |
| @@ -656,5 +656,6 @@ int amdgpu_psp_reg_program_no_ring(struct psp_context *psp, uint32_t val, · 655 unmodified lines | ||
| 656 | 656 | void amdgpu_psp_debugfs_init(struct amdgpu_device *adev); |
| 657 | 657 | int amdgpu_psp_get_fw_type(struct amdgpu_firmware_info *ucode, |
| 658 | 658 | enum psp_gfx_fw_type *type); |
| 659 | +int psp_set_mmhub_eco_sec_level(struct amdgpu_device *adev); | |
| 659 | 660 | |
| 660 | 661 | #endif |
| @@ -2120,23 +2120,17 @@ int amdgpu_ttm_init(struct amdgpu_device *adev) · 2119 unmodified lines | ||
| 2120 | 2120 | /* Change the size here instead of the init above so only lpfn is affected */ |
| 2121 | 2121 | amdgpu_ttm_disable_buffer_funcs(adev); |
| 2122 | 2122 | #ifdef CONFIG_64BIT |
| 2123 | − if (adev->gmc.xgmi.connected_to_cpu) { | |
| 2124 | − void *kaddr = devm_memremap(adev->dev, adev->gmc.aper_base, | |
| 2125 | − adev->gmc.visible_vram_size, | |
| 2126 | − MEMREMAP_WB); | |
| 2127 | − if (IS_ERR(kaddr)) | |
| 2128 | − return PTR_ERR(kaddr); | |
| 2129 | − adev->mman.aper_base_kaddr = (__force void __iomem *)kaddr; | |
| 2130 | − } else if (adev->gmc.is_app_apu) { | |
| 2123 | +#ifdef CONFIG_X86 | |
| 2124 | + if (adev->gmc.xgmi.connected_to_cpu) | |
| 2125 | + adev->mman.aper_base_kaddr = ioremap_cache(adev->gmc.aper_base, | |
| 2126 | + adev->gmc.visible_vram_size); | |
| 2127 | + else if (adev->gmc.is_app_apu) | |
| 2131 | 2128 | DRM_DEBUG_DRIVER( |
| 2132 | 2129 | "No need to ioremap when real vram size is 0\n"); |
| 2133 | − } else { | |
| 2134 | − adev->mman.aper_base_kaddr = devm_ioremap_wc(adev->dev, | |
| 2135 | − adev->gmc.aper_base, | |
| 2136 | − adev->gmc.visible_vram_size); | |
| 2137 | − if (!adev->mman.aper_base_kaddr) | |
| 2138 | − return -ENOMEM; | |
| 2139 | − } | |
| 2130 | + else | |
| 2131 | +#endif | |
| 2132 | + adev->mman.aper_base_kaddr = ioremap_wc(adev->gmc.aper_base, | |
| 2133 | + adev->gmc.visible_vram_size); | |
| 2140 | 2134 | #endif |
| 2141 | 2135 | |
| 2142 | 2136 | amdgpu_ttm_init_vram_resv_regions(adev); |
| @@ -2173,6 +2167,18 @@ int amdgpu_ttm_init(struct amdgpu_device *adev) · 30 unmodified lines | ||
| 2173 | 2167 | gtt_size = configured_size; |
| 2174 | 2168 | } |
| 2175 | 2169 | |
| 2170 | + /* Cap GTT so that it does not exceed total physical RAM. */ | |
| 2171 | + if (adev->flags & AMD_IS_APU) { | |
| 2172 | + u64 phys_ram = (u64)totalram_pages() << PAGE_SHIFT; | |
| 2173 | + | |
| 2174 | + if (gtt_size > phys_ram) { | |
| 2175 | + gtt_size = phys_ram; | |
| 2176 | + dev_info(adev->dev, | |
| 2177 | + "Capping GTT to %uM to not exceed available system memory\n", | |
| 2178 | + (unsigned int)(gtt_size / (1024 * 1024))); | |
| 2179 | + } | |
| 2180 | + } | |
| 2181 | + | |
| 2176 | 2182 | /* Initialize GTT memory pool */ |
| 2177 | 2183 | r = amdgpu_gtt_mgr_init(adev, gtt_size); |
| 2178 | 2184 | if (r) { |
| @@ -2275,7 +2281,10 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev) · 96 unmodified lines | ||
| 2275 | 2281 | amdgpu_ttm_unmark_vram_reserved(adev, AMDGPU_RESV_FW_VRAM_USAGE); |
| 2276 | 2282 | amdgpu_ttm_unmark_vram_reserved(adev, AMDGPU_RESV_DRV_VRAM_USAGE); |
| 2277 | 2283 | |
| 2278 | − adev->mman.aper_base_kaddr = NULL; | |
| 2284 | + if (adev->mman.aper_base_kaddr) { | |
| 2285 | + iounmap(adev->mman.aper_base_kaddr); | |
| 2286 | + adev->mman.aper_base_kaddr = NULL; | |
| 2287 | + } | |
| 2279 | 2288 | |
| 2280 | 2289 | if (!adev->gmc.is_app_apu) |
| 2281 | 2290 | amdgpu_vram_mgr_fini(adev); |
| @@ -700,7 +700,12 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args) · 699 unmodified lines | ||
| 700 | 700 | if (!adev->userq_halt_for_enforce_isolation || |
| 701 | 701 | ((queue->queue_type != AMDGPU_HW_IP_GFX) && |
| 702 | 702 | (queue->queue_type != AMDGPU_HW_IP_COMPUTE))) { |
| 703 | + /* Serialize the map against an in-progress GPU reset (MES is | |
| 704 | + * unresponsive during recovery), matching amdgpu_userq_cleanup(). | |
| 705 | + */ | |
| 706 | + down_read(&adev->reset_domain->sem); | |
| 703 | 707 | r = amdgpu_userq_map_helper(queue); |
| 708 | + up_read(&adev->reset_domain->sem); | |
| 704 | 709 | if (r) { |
| 705 | 710 | drm_file_err(uq_mgr->file, "Failed to map Queue\n"); |
| 706 | 711 | mutex_unlock(&uq_mgr->userq_mutex); |
| @@ -965,6 +970,7 @@ amdgpu_userq_vm_validate(struct amdgpu_userq_mgr *uq_mgr) · 258 unmodified lines | ||
| 965 | 970 | struct amdgpu_vm *vm = &fpriv->vm; |
| 966 | 971 | unsigned long key, tmp_key; |
| 967 | 972 | struct amdgpu_bo_va *bo_va; |
| 973 | + struct amdgpu_usermode_queue *queue; | |
| 968 | 974 | struct amdgpu_bo *bo; |
| 969 | 975 | struct drm_exec exec; |
| 970 | 976 | struct xarray xa; |
| @@ -1080,6 +1086,24 @@ retry_lock: · 109 unmodified lines | ||
| 1080 | 1086 | dma_fence_wait(bo_va->last_pt_update, false); |
| 1081 | 1087 | dma_fence_wait(vm->last_update, false); |
| 1082 | 1088 | |
| 1089 | + xa_for_each(&uq_mgr->userq_xa, tmp_key, queue) { | |
| 1090 | + bo = queue->wptr_obj.obj; | |
| 1091 | + if (!bo) { | |
| 1092 | + ret = -EINVAL; | |
| 1093 | + goto unlock_all; | |
| 1094 | + } | |
| 1095 | + | |
| 1096 | + ret = amdgpu_ttm_alloc_gart(&bo->tbo); | |
| 1097 | + if (unlikely(ret)) { | |
| 1098 | + drm_file_err(uq_mgr->file, | |
| 1099 | + "failed to bind wptr bo to gart on resume, qid=%lu ret=%d\n", | |
| 1100 | + tmp_key, ret); | |
| 1101 | + goto unlock_all; | |
| 1102 | + } | |
| 1103 | + | |
| 1104 | + queue->wptr_obj.gpu_addr = amdgpu_bo_gpu_offset(bo); | |
| 1105 | + } | |
| 1106 | + | |
| 1083 | 1107 | ret = amdgpu_evf_mgr_rearm(&fpriv->evf_mgr, &exec); |
| 1084 | 1108 | if (ret) |
| 1085 | 1109 | drm_file_err(uq_mgr->file, "Failed to replace eviction fence\n"); |
| @@ -1553,6 +1553,8 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev, · 1552 unmodified lines | ||
| 1553 | 1553 | struct amdgpu_sync sync; |
| 1554 | 1554 | int r; |
| 1555 | 1555 | |
| 1556 | + if (list_empty(&vm->freed)) | |
| 1557 | + return 0; | |
| 1556 | 1558 | |
| 1557 | 1559 | /* |
| 1558 | 1560 | * Implicitly sync to command submissions in the same VM before |
| @@ -379,7 +379,7 @@ struct amd_sriov_msg_vf2pf_info { · 378 unmodified lines | ||
| 379 | 379 | struct { |
| 380 | 380 | uint8_t id; |
| 381 | 381 | uint32_t version; |
| 382 | − } ucode_info[AMD_SRIOV_MSG_RESERVE_UCODE]; | |
| 382 | + } __packed ucode_info[AMD_SRIOV_MSG_RESERVE_UCODE]; | |
| 383 | 383 | uint64_t dummy_page_addr; |
| 384 | 384 | /* FB allocated for guest MES to record UQ info */ |
| 385 | 385 | uint64_t mes_info_addr; |
| @@ -1823,6 +1823,11 @@ static void gfx_v12_0_constants_init(struct amdgpu_device *adev) · 1822 unmodified lines | ||
| 1823 | 1823 | gfx_v12_0_get_tcc_info(adev); |
| 1824 | 1824 | adev->gfx.config.pa_sc_tile_steering_override = 0; |
| 1825 | 1825 | |
| 1826 | + /* Set whether texture coordinate truncation is conformant. */ | |
| 1827 | + tmp = RREG32_SOC15(GC, 0, regTA_CNTL2); | |
| 1828 | + adev->gfx.config.ta_cntl2_truncate_coord_mode = | |
| 1829 | + REG_GET_FIELD(tmp, TA_CNTL2, TRUNCATE_COORD_MODE); | |
| 1830 | + | |
| 1826 | 1831 | /* XXX SH_MEM regs */ |
| 1827 | 1832 | /* where to put LDS, scratch, GPUVM in FSA64 space */ |
| 1828 | 1833 | mutex_lock(&adev->srbm_mutex); |
| @@ -267,9 +267,24 @@ static bool gmc_v12_1_get_vmid_pasid_mapping_info(struct amdgpu_device *adev, · 266 unmodified lines | ||
| 267 | 267 | * by the amdgpu vm/hsa code. |
| 268 | 268 | */ |
| 269 | 269 | |
| 270 | +/** | |
| 271 | + * gmc_v12_1_use_invalidate_semaphore - judge whether to use semaphore | |
| 272 | + * | |
| 273 | + * @adev: amdgpu_device pointer | |
| 274 | + * @vmhub: vmhub type | |
| 275 | + * | |
| 276 | + */ | |
| 277 | +static bool gmc_v12_1_use_invalidate_semaphore(struct amdgpu_device *adev, | |
| 278 | + uint32_t vmhub) | |
| 279 | +{ | |
| 280 | + return ((!AMDGPU_IS_GFXHUB(vmhub)) && | |
| 281 | + (!amdgpu_sriov_vf(adev))); | |
| 282 | +} | |
| 283 | + | |
| 270 | 284 | static void gmc_v12_1_flush_vm_hub(struct amdgpu_device *adev, uint32_t vmid, |
| 271 | 285 | unsigned int vmhub, uint32_t flush_type) |
| 272 | 286 | { |
| 287 | + bool use_semaphore = gmc_v12_1_use_invalidate_semaphore(adev, vmhub); | |
| 273 | 288 | struct amdgpu_vmhub *hub = &adev->vmhub[vmhub]; |
| 274 | 289 | u32 inv_req = hub->vmhub_funcs->get_invalidate_req(vmid, flush_type); |
| 275 | 290 | u32 tmp; |
| @@ -283,6 +298,19 @@ static void gmc_v12_1_flush_vm_hub(struct amdgpu_device *adev, uint32_t vmid, · 7 unmodified lines | ||
| 283 | 298 | |
| 284 | 299 | spin_lock(&adev->gmc.invalidate_lock); |
| 285 | 300 | |
| 301 | + if (use_semaphore) { | |
| 302 | + for (i = 0; i < adev->usec_timeout; i++) { | |
| 303 | + /* a read return value of 1 means semaphore acuqire */ | |
| 304 | + tmp = RREG32_RLC_NO_KIQ(hub->vm_inv_eng0_sem + hub->eng_distance * eng, hub_ip); | |
| 305 | + if (tmp & 0x1) | |
| 306 | + break; | |
| 307 | + udelay(1); | |
| 308 | + } | |
| 309 | + | |
| 310 | + if (i >= adev->usec_timeout) | |
| 311 | + DRM_ERROR("Timeout waiting for sem acquire in VM flush!\n"); | |
| 312 | + } | |
| 313 | + | |
| 286 | 314 | WREG32_RLC_NO_KIQ(hub->vm_inv_eng0_req + hub->eng_distance * eng, inv_req, hub_ip); |
| 287 | 315 | |
| 288 | 316 | /* Wait for ACK with a delay.*/ |
| @@ -296,6 +324,9 @@ static void gmc_v12_1_flush_vm_hub(struct amdgpu_device *adev, uint32_t vmid, · 7 unmodified lines | ||
| 296 | 324 | udelay(1); |
| 297 | 325 | } |
| 298 | 326 | |
| 327 | + if (use_semaphore) | |
| 328 | + WREG32_RLC_NO_KIQ(hub->vm_inv_eng0_sem + hub->eng_distance * eng, 0, hub_ip); | |
| 329 | + | |
| 299 | 330 | /* Issue additional private vm invalidation to MMHUB */ |
| 300 | 331 | if (!AMDGPU_IS_GFXHUB(vmhub) && |
| 301 | 332 | (hub->vm_l2_bank_select_reserved_cid2) && |
| @@ -396,7 +427,7 @@ static void gmc_v12_1_flush_gpu_tlb_pasid(struct amdgpu_device *adev, · 94 unmodified lines | ||
| 396 | 427 | |
| 397 | 428 | if (all_hub) { |
| 398 | 429 | /* invalidate mm_hub */ |
| 399 | − if (test_bit(AMDGPU_MMHUB1(0), adev->vmhubs_mask)) { | |
| 430 | + if (test_bit(AMDGPU_MMHUB0(0), adev->vmhubs_mask)) { | |
| 400 | 431 | input.hub_id = AMDGPU_MMHUB0(0); |
| 401 | 432 | adev->mes.funcs->invalidate_tlbs_pasid(&adev->mes, &input); |
| 402 | 433 | } |
| @@ -431,10 +462,17 @@ static void gmc_v12_1_flush_gpu_tlb_pasid(struct amdgpu_device *adev, · 28 unmodified lines | ||
| 431 | 462 | static uint64_t gmc_v12_1_emit_flush_gpu_tlb(struct amdgpu_ring *ring, |
| 432 | 463 | unsigned vmid, uint64_t pd_addr) |
| 433 | 464 | { |
| 465 | + bool use_semaphore = gmc_v12_1_use_invalidate_semaphore(ring->adev, ring->vm_hub); | |
| 434 | 466 | struct amdgpu_vmhub *hub = &ring->adev->vmhub[ring->vm_hub]; |
| 435 | 467 | uint32_t req = hub->vmhub_funcs->get_invalidate_req(vmid, 0); |
| 436 | 468 | unsigned eng = ring->vm_inv_eng; |
| 437 | 469 | |
| 470 | + if (use_semaphore) | |
| 471 | + /* a read return value of 1 means semaphore acuqire */ | |
| 472 | + amdgpu_ring_emit_reg_wait(ring, | |
| 473 | + hub->vm_inv_eng0_sem + | |
| 474 | + hub->eng_distance * eng, 0x1, 0x1); | |
| 475 | + | |
| 438 | 476 | amdgpu_ring_emit_wreg(ring, hub->ctx0_ptb_addr_lo32 + |
| 439 | 477 | (hub->ctx_addr_distance * vmid), |
| 440 | 478 | lower_32_bits(pd_addr)); |
| @@ -449,6 +487,14 @@ static uint64_t gmc_v12_1_emit_flush_gpu_tlb(struct amdgpu_ring *ring, · 8 unmodified lines | ||
| 449 | 487 | hub->eng_distance * eng, |
| 450 | 488 | req, 1 << vmid); |
| 451 | 489 | |
| 490 | + if (use_semaphore) | |
| 491 | + /* | |
| 492 | + * add semaphore release after invalidation, | |
| 493 | + * write with 0 means semaphore release | |
| 494 | + */ | |
| 495 | + amdgpu_ring_emit_wreg(ring, hub->vm_inv_eng0_sem + | |
| 496 | + hub->eng_distance * eng, 0); | |
| 497 | + | |
| 452 | 498 | return pd_addr; |
| 453 | 499 | } |
| 454 | 500 | |
| @@ -772,15 +772,28 @@ static int jpeg_v4_0_5_ring_reset(struct amdgpu_ring *ring, · 771 unmodified lines | ||
| 772 | 772 | unsigned int vmid, |
| 773 | 773 | struct amdgpu_fence *timedout_fence) |
| 774 | 774 | { |
| 775 | + struct amdgpu_device *adev = ring->adev; | |
| 776 | + u32 pg_flags = adev->pg_flags; | |
| 775 | 777 | int r; |
| 776 | 778 | |
| 777 | 779 | amdgpu_ring_reset_helper_begin(ring, timedout_fence); |
| 778 | − r = jpeg_v4_0_5_stop(ring->adev); | |
| 779 | − if (r) | |
| 780 | − return r; | |
| 781 | − r = jpeg_v4_0_5_start(ring->adev); | |
| 780 | + | |
| 781 | + /* | |
| 782 | + * The DPG stop path only clears the JPEG_PG_MODE bit and never resets a | |
| 783 | + * hung JRBC, so the post-reset ring test times out and the driver falls | |
| 784 | + * back to a full MODE1 reset. Temporarily force the static power-gating | |
| 785 | + * path so the stop/start sequence actually power-cycles the JPEG block | |
| 786 | + * (JMI soft reset + static power off/on), matching the working jpeg_v4_0 | |
| 787 | + * reset. | |
| 788 | + */ | |
| 789 | + adev->pg_flags &= ~AMD_PG_SUPPORT_JPEG_DPG; | |
| 790 | + r = jpeg_v4_0_5_stop(adev); | |
| 791 | + if (!r) | |
| 792 | + r = jpeg_v4_0_5_start(adev); | |
| 793 | + adev->pg_flags = pg_flags; | |
| 782 | 794 | if (r) |
| 783 | 795 | return r; |
| 796 | + | |
| 784 | 797 | return amdgpu_ring_reset_helper_end(ring, timedout_fence); |
| 785 | 798 | } |
| 786 | 799 | |
| @@ -648,15 +648,28 @@ static int jpeg_v5_0_0_ring_reset(struct amdgpu_ring *ring, · 647 unmodified lines | ||
| 648 | 648 | unsigned int vmid, |
| 649 | 649 | struct amdgpu_fence *timedout_fence) |
| 650 | 650 | { |
| 651 | + struct amdgpu_device *adev = ring->adev; | |
| 652 | + u32 pg_flags = adev->pg_flags; | |
| 651 | 653 | int r; |
| 652 | 654 | |
| 653 | 655 | amdgpu_ring_reset_helper_begin(ring, timedout_fence); |
| 654 | − r = jpeg_v5_0_0_stop(ring->adev); | |
| 655 | − if (r) | |
| 656 | − return r; | |
| 657 | − r = jpeg_v5_0_0_start(ring->adev); | |
| 656 | + | |
| 657 | + /* | |
| 658 | + * The DPG stop path only clears the JPEG_PG_MODE bit and never resets a | |
| 659 | + * hung JRBC, so the post-reset ring test times out and the driver falls | |
| 660 | + * back to a full MODE1 reset. Temporarily force the static power-gating | |
| 661 | + * path so the stop/start sequence actually power-cycles the JPEG block | |
| 662 | + * (JMI soft reset + ONO1 power off/on), matching the working jpeg_v4_0 | |
| 663 | + * reset. | |
| 664 | + */ | |
| 665 | + adev->pg_flags &= ~AMD_PG_SUPPORT_JPEG_DPG; | |
| 666 | + r = jpeg_v5_0_0_stop(adev); | |
| 667 | + if (!r) | |
| 668 | + r = jpeg_v5_0_0_start(adev); | |
| 669 | + adev->pg_flags = pg_flags; | |
| 658 | 670 | if (r) |
| 659 | 671 | return r; |
| 672 | + | |
| 660 | 673 | return amdgpu_ring_reset_helper_end(ring, timedout_fence); |
| 661 | 674 | } |
| 662 | 675 | |
| @@ -398,6 +398,25 @@ static void jpeg_v5_3_0_stop_dpg_mode(struct amdgpu_device *adev, int inst_idx) · 397 unmodified lines | ||
| 398 | 398 | WREG32_SOC15(JPEG, inst_idx, regUVD_JPEG_POWER_STATUS, reg_data); |
| 399 | 399 | } |
| 400 | 400 | |
| 401 | +/** | |
| 402 | + * jpeg_v5_3_0_set_mmhub_eco_sec_level - set jpeg sec lvl reg | |
| 403 | + * | |
| 404 | + * @adev: amdgpu_device pointer | |
| 405 | + * | |
| 406 | + * request psp to set secure lvl | |
| 407 | + */ | |
| 408 | +static int jpeg_v5_3_0_set_mmhub_eco_sec_level(struct amdgpu_device *adev) | |
| 409 | +{ | |
| 410 | + int r = 0; | |
| 411 | + | |
| 412 | + if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { | |
| 413 | + /* Request to PSP to program JPEG secure lvl */ | |
| 414 | + r = psp_set_mmhub_eco_sec_level(adev); | |
| 415 | + } | |
| 416 | + | |
| 417 | + return r; | |
| 418 | +} | |
| 419 | + | |
| 401 | 420 | /** |
| 402 | 421 | * jpeg_v5_3_0_start - start JPEG block |
| 403 | 422 | * |
| @@ -423,6 +442,11 @@ static int jpeg_v5_3_0_start(struct amdgpu_device *adev) · 19 unmodified lines | ||
| 423 | 442 | if (r) |
| 424 | 443 | return r; |
| 425 | 444 | |
| 445 | + /* program JPEG secure lvl register */ | |
| 446 | + r = jpeg_v5_3_0_set_mmhub_eco_sec_level(adev); | |
| 447 | + if (r) | |
| 448 | + return r; | |
| 449 | + | |
| 426 | 450 | /* JPEG disable CGC */ |
| 427 | 451 | jpeg_v5_3_0_disable_clock_gating(adev); |
| 428 | 452 | |
| @@ -631,15 +655,28 @@ static int jpeg_v5_3_0_ring_reset(struct amdgpu_ring *ring, · 202 unmodified lines | ||
| 631 | 655 | unsigned int vmid, |
| 632 | 656 | struct amdgpu_fence *timedout_fence) |
| 633 | 657 | { |
| 658 | + struct amdgpu_device *adev = ring->adev; | |
| 659 | + u32 pg_flags = adev->pg_flags; | |
| 634 | 660 | int r; |
| 635 | 661 | |
| 636 | 662 | amdgpu_ring_reset_helper_begin(ring, timedout_fence); |
| 637 | − r = jpeg_v5_3_0_stop(ring->adev); | |
| 638 | − if (r) | |
| 639 | − return r; | |
| 640 | − r = jpeg_v5_3_0_start(ring->adev); | |
| 663 | + | |
| 664 | + /* | |
| 665 | + * The DPG stop path only clears the JPEG_PG_MODE bit and never resets a | |
| 666 | + * hung JRBC, so the post-reset ring test times out and the driver falls | |
| 667 | + * back to a full MODE1 reset. Temporarily force the static power-gating | |
| 668 | + * path so the stop/start sequence actually power-cycles the JPEG block | |
| 669 | + * (JMI soft reset + static power off/on), matching the working jpeg_v4_0 | |
| 670 | + * reset. | |
| 671 | + */ | |
| 672 | + adev->pg_flags &= ~AMD_PG_SUPPORT_JPEG_DPG; | |
| 673 | + r = jpeg_v5_3_0_stop(adev); | |
| 674 | + if (!r) | |
| 675 | + r = jpeg_v5_3_0_start(adev); | |
| 676 | + adev->pg_flags = pg_flags; | |
| 641 | 677 | if (r) |
| 642 | 678 | return r; |
| 679 | + | |
| 643 | 680 | return amdgpu_ring_reset_helper_end(ring, timedout_fence); |
| 644 | 681 | } |
| 645 | 682 | |
| @@ -70,27 +70,23 @@ mes_userq_create_wptr_mapping(struct amdgpu_device *adev, · 69 unmodified lines | ||
| 70 | 70 | ret = -EINVAL; |
| 71 | 71 | goto fail_map; |
| 72 | 72 | } |
| 73 | − | |
| 74 | − /* TODO use eviction fence instead of pinning. */ | |
| 75 | − ret = amdgpu_bo_pin(wptr_obj->obj, AMDGPU_GEM_DOMAIN_GTT); | |
| 73 | + /* Keep WPTR BO under eviction-fence control instead of pinning. */ | |
| 74 | + ret = amdgpu_evf_mgr_attach_fence(&uq_mgr_to_fpriv(uq_mgr)->evf_mgr, wptr_obj->obj); | |
| 76 | 75 | if (ret) { |
| 77 | − DRM_ERROR("Failed to pin wptr bo. ret %d\n", ret); | |
| 76 | + DRM_ERROR("Failed to attach eviction fence to wptr bo. ret %d\n", ret); | |
| 78 | 77 | goto fail_map; |
| 79 | 78 | } |
| 80 | 79 | |
| 81 | 80 | ret = amdgpu_ttm_alloc_gart(&wptr_obj->obj->tbo); |
| 82 | 81 | if (ret) { |
| 83 | − DRM_ERROR("Failed to bind bo to GART. ret %d\n", ret); | |
| 84 | − goto fail_alloc_gart; | |
| 82 | + DRM_ERROR("Failed to bind wptr bo to GART. ret %d\n", ret); | |
| 83 | + goto fail_map; | |
| 85 | 84 | } |
| 86 | 85 | |
| 87 | 86 | queue->wptr_obj.gpu_addr = amdgpu_bo_gpu_offset(wptr_obj->obj); |
| 88 | 87 | |
| 89 | 88 | drm_exec_fini(&exec); |
| 90 | 89 | return 0; |
| 91 | − | |
| 92 | −fail_alloc_gart: | |
| 93 | − amdgpu_bo_unpin(wptr_obj->obj); | |
| 94 | 90 | fail_map: |
| 95 | 91 | amdgpu_bo_unref(&wptr_obj->obj); |
| 96 | 92 | fail_lock: |
| @@ -468,9 +464,6 @@ static void mes_userq_mqd_destroy(struct amdgpu_usermode_queue *queue) · 371 unmodified lines | ||
| 468 | 464 | amdgpu_bo_free_kernel(&queue->mqd.obj, &queue->mqd.gpu_addr, |
| 469 | 465 | &queue->mqd.cpu_ptr); |
| 470 | 466 | |
| 471 | − amdgpu_bo_reserve(queue->wptr_obj.obj, true); | |
| 472 | − amdgpu_bo_unpin(queue->wptr_obj.obj); | |
| 473 | − amdgpu_bo_unreserve(queue->wptr_obj.obj); | |
| 474 | 467 | amdgpu_bo_unref(&queue->wptr_obj.obj); |
| 475 | 468 | } |
| 476 | 469 | |
| @@ -30,70 +30,76 @@ · 29 unmodified lines | ||
| 30 | 30 | #include "ivsrcid/nbio/irqsrcs_nbif_7_4.h" |
| 31 | 31 | #include <uapi/linux/kfd_ioctl.h> |
| 32 | 32 | |
| 33 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_0_CTRL_nbif_4_10 0x4f0aeb | |
| 34 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_0_CTRL_nbif_4_10_BASE_IDX 3 | |
| 35 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_0_CTRL1_nbif_4_10 0x4f0aec | |
| 36 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_0_CTRL1_nbif_4_10_BASE_IDX 3 | |
| 37 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_1_CTRL_nbif_4_10 0x4f0aed | |
| 38 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_1_CTRL_nbif_4_10_BASE_IDX 3 | |
| 39 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_1_CTRL1_nbif_4_10 0x4f0aee | |
| 40 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_1_CTRL1_nbif_4_10_BASE_IDX 3 | |
| 41 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_2_CTRL_nbif_4_10 0x4f0aef | |
| 42 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_2_CTRL_nbif_4_10_BASE_IDX 3 | |
| 43 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_2_CTRL1_nbif_4_10 0x4f0af0 | |
| 44 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_2_CTRL1_nbif_4_10_BASE_IDX 3 | |
| 45 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_3_CTRL_nbif_4_10 0x4f0af1 | |
| 46 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_3_CTRL_nbif_4_10_BASE_IDX 3 | |
| 47 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_3_CTRL1_nbif_4_10 0x4f0af2 | |
| 48 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_3_CTRL1_nbif_4_10_BASE_IDX 3 | |
| 49 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_4_CTRL_nbif_4_10 0x4f0af3 | |
| 50 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_4_CTRL_nbif_4_10_BASE_IDX 3 | |
| 51 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_4_CTRL1_nbif_4_10 0x4f0af4 | |
| 52 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_4_CTRL1_nbif_4_10_BASE_IDX 3 | |
| 53 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_5_CTRL_nbif_4_10 0x4f0af5 | |
| 54 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_5_CTRL_nbif_4_10_BASE_IDX 3 | |
| 55 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_5_CTRL1_nbif_4_10 0x4f0af6 | |
| 56 | −#define regGDC_S2A0_S2A_DOORBELL_ENTRY_5_CTRL1_nbif_4_10_BASE_IDX 3 | |
| 57 | −#define regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0_nbif_4_10 0x0021 | |
| 58 | −#define regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0_nbif_4_10_BASE_IDX 2 | |
| 59 | − | |
| 60 | −#define regBIF_BX_PF0_DOORBELL_SELFRING_GPA_APER_BASE_HIGH_nbio_7_11_5 0x8e13 | |
| 61 | −#define regBIF_BX_PF0_DOORBELL_SELFRING_GPA_APER_BASE_HIGH_nbio_7_11_5_BASE_IDX 5 | |
| 62 | −#define regBIF_BX_PF0_DOORBELL_SELFRING_GPA_APER_BASE_LOW_nbio_7_11_5 0x8e14 | |
| 63 | −#define regBIF_BX_PF0_DOORBELL_SELFRING_GPA_APER_BASE_LOW_nbio_7_11_5_BASE_IDX 5 | |
| 64 | −#define regBIF_BX_PF0_DOORBELL_SELFRING_GPA_APER_CNTL_nbio_7_11_5 0x8e15 | |
| 65 | −#define regBIF_BX_PF0_DOORBELL_SELFRING_GPA_APER_CNTL_nbio_7_11_5_BASE_IDX 5 | |
| 66 | − | |
| 67 | −#define regBIF_BX0_REMAP_HDP_MEM_FLUSH_CNTL_nbio_7_11_5 0x8e4d | |
| 68 | −#define regBIF_BX0_REMAP_HDP_MEM_FLUSH_CNTL_nbio_7_11_5_BASE_IDX 5 | |
| 69 | −#define regBIF_BX0_REMAP_HDP_REG_FLUSH_CNTL_nbio_7_11_5 0x8e4e | |
| 70 | −#define regBIF_BX0_REMAP_HDP_REG_FLUSH_CNTL_nbio_7_11_5_BASE_IDX 5 | |
| 71 | − | |
| 72 | −#define regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0_nbio_7_11_5 0xd000 | |
| 73 | −#define regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0_nbio_7_11_5_BASE_IDX 5 | |
| 74 | − | |
| 75 | −#define regBIF_BX0_BIF_FB_EN_nbio_7_11_5 0x8e20 | |
| 76 | −#define regBIF_BX0_BIF_FB_EN_nbio_7_11_5_BASE_IDX 5 | |
| 77 | − | |
| 78 | −#define regBIF_BX0_INTERRUPT_CNTL_nbio_7_11_5 0x8e11 | |
| 79 | −#define regBIF_BX0_INTERRUPT_CNTL_nbio_7_11_5_BASE_IDX 5 | |
| 80 | −#define regBIF_BX0_INTERRUPT_CNTL2_nbio_7_11_5 0x8e12 | |
| 81 | −#define regBIF_BX0_INTERRUPT_CNTL2_nbio_7_11_5_BASE_IDX 5 | |
| 82 | − | |
| 83 | −#define regBIF_BX_PF0_GPU_HDP_FLUSH_REQ_nbio_7_11_5 0x8e26 | |
| 84 | −#define regBIF_BX_PF0_GPU_HDP_FLUSH_REQ_nbio_7_11_5_BASE_IDX 5 | |
| 85 | −#define regBIF_BX_PF0_GPU_HDP_FLUSH_DONE_nbio_7_11_5 0x8e27 | |
| 86 | −#define regBIF_BX_PF0_GPU_HDP_FLUSH_DONE_nbio_7_11_5_BASE_IDX 5 | |
| 87 | − | |
| 88 | −#define regBIF_BX_PF0_HDP_MEM_COHERENCY_FLUSH_CNTL_nbio_7_11_5 0x8e17 | |
| 89 | −#define regBIF_BX_PF0_HDP_MEM_COHERENCY_FLUSH_CNTL_nbio_7_11_5_BASE_IDX 5 | |
| 33 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_0_CTRL_nbif_4_10 0x4f0aeb | |
| 34 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_0_CTRL_nbif_4_10_BASE_IDX 3 | |
| 35 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_0_CTRL1_nbif_4_10 0x4f0aec | |
| 36 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_0_CTRL1_nbif_4_10_BASE_IDX 3 | |
| 37 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_1_CTRL_nbif_4_10 0x4f0aed | |
| 38 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_1_CTRL_nbif_4_10_BASE_IDX 3 | |
| 39 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_1_CTRL1_nbif_4_10 0x4f0aee | |
| 40 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_1_CTRL1_nbif_4_10_BASE_IDX 3 | |
| 41 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_2_CTRL_nbif_4_10 0x4f0aef | |
| 42 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_2_CTRL_nbif_4_10_BASE_IDX 3 | |
| 43 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_2_CTRL1_nbif_4_10 0x4f0af0 | |
| 44 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_2_CTRL1_nbif_4_10_BASE_IDX 3 | |
| 45 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_3_CTRL_nbif_4_10 0x4f0af1 | |
| 46 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_3_CTRL_nbif_4_10_BASE_IDX 3 | |
| 47 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_3_CTRL1_nbif_4_10 0x4f0af2 | |
| 48 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_3_CTRL1_nbif_4_10_BASE_IDX 3 | |
| 49 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_4_CTRL_nbif_4_10 0x4f0af3 | |
| 50 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_4_CTRL_nbif_4_10_BASE_IDX 3 | |
| 51 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_4_CTRL1_nbif_4_10 0x4f0af4 | |
| 52 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_4_CTRL1_nbif_4_10_BASE_IDX 3 | |
| 53 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_5_CTRL_nbif_4_10 0x4f0af5 | |
| 54 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_5_CTRL_nbif_4_10_BASE_IDX 3 | |
| 55 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_5_CTRL1_nbif_4_10 0x4f0af6 | |
| 56 | +#define regGDC_S2A0_S2A_DOORBELL_ENTRY_5_CTRL1_nbif_4_10_BASE_IDX 3 | |
| 57 | +#define regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0_nbif_4_10 0x0021 | |
| 58 | +#define regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0_nbif_4_10_BASE_IDX 2 | |
| 59 | + | |
| 60 | +#define regBIF_BX_PF0_DOORBELL_SELFRING_GPA_APER_BASE_HIGH_nbio_7_11_5 0x8e13 | |
| 61 | +#define regBIF_BX_PF0_DOORBELL_SELFRING_GPA_APER_BASE_HIGH_nbio_7_11_5_BASE_IDX 5 | |
| 62 | +#define regBIF_BX_PF0_DOORBELL_SELFRING_GPA_APER_BASE_LOW_nbio_7_11_5 0x8e14 | |
| 63 | +#define regBIF_BX_PF0_DOORBELL_SELFRING_GPA_APER_BASE_LOW_nbio_7_11_5_BASE_IDX 5 | |
| 64 | +#define regBIF_BX_PF0_DOORBELL_SELFRING_GPA_APER_CNTL_nbio_7_11_5 0x8e15 | |
| 65 | +#define regBIF_BX_PF0_DOORBELL_SELFRING_GPA_APER_CNTL_nbio_7_11_5_BASE_IDX 5 | |
| 66 | + | |
| 67 | +#define regBIF_BX1_REMAP_HDP_MEM_FLUSH_CNTL_nbio_7_11_5 0x012d | |
| 68 | +#define regBIF_BX1_REMAP_HDP_MEM_FLUSH_CNTL_nbio_7_11_5_BASE_IDX 2 | |
| 69 | +#define regBIF_BX1_REMAP_HDP_REG_FLUSH_CNTL_nbio_7_11_5 0x012e | |
| 70 | +#define regBIF_BX1_REMAP_HDP_REG_FLUSH_CNTL_nbio_7_11_5_BASE_IDX 2 | |
| 71 | + | |
| 72 | +#define regRCC_STRAP1_RCC_DEV0_EPF0_STRAP0_nbio_7_11_5 0x0021 | |
| 73 | +#define regRCC_STRAP1_RCC_DEV0_EPF0_STRAP0_nbio_7_11_5_BASE_IDX 2 | |
| 74 | + | |
| 75 | +#define regBIF_BX1_BIF_FB_EN_nbio_7_11_5 0x0100 | |
| 76 | +#define regBIF_BX1_BIF_FB_EN_nbio_7_11_5_BASE_IDX 2 | |
| 77 | + | |
| 78 | +#define regBIF_BX1_INTERRUPT_CNTL_nbio_7_11_5 0x00f1 | |
| 79 | +#define regBIF_BX1_INTERRUPT_CNTL_nbio_7_11_5_BASE_IDX 2 | |
| 80 | +#define regBIF_BX1_INTERRUPT_CNTL2_nbio_7_11_5 0x00f2 | |
| 81 | +#define regBIF_BX1_INTERRUPT_CNTL2_nbio_7_11_5_BASE_IDX 2 | |
| 82 | + | |
| 83 | +#define regBIF_BX_PF1_GPU_HDP_FLUSH_REQ_nbio_7_11_5 0x0106 | |
| 84 | +#define regBIF_BX_PF1_GPU_HDP_FLUSH_REQ_nbio_7_11_5_BASE_IDX 2 | |
| 85 | +#define regBIF_BX_PF1_GPU_HDP_FLUSH_DONE_nbio_7_11_5 0x0107 | |
| 86 | +#define regBIF_BX_PF1_GPU_HDP_FLUSH_DONE_nbio_7_11_5_BASE_IDX 2 | |
| 87 | + | |
| 88 | +#define regBIF_BX_PF1_HDP_MEM_COHERENCY_FLUSH_CNTL_nbio_7_11_5 0x00f7 | |
| 89 | +#define regBIF_BX_PF1_HDP_MEM_COHERENCY_FLUSH_CNTL_nbio_7_11_5_BASE_IDX 2 | |
| 90 | + | |
| 91 | +//BIF_BX1_BIF_FB_EN | |
| 92 | +#define BIF_BX1_BIF_FB_EN__FB_READ_EN__SHIFT_nbio_7_11_5 0x0 | |
| 93 | +#define BIF_BX1_BIF_FB_EN__FB_WRITE_EN__SHIFT_nbio_7_11_5 0x1 | |
| 94 | +#define BIF_BX1_BIF_FB_EN__FB_READ_EN_MASK_nbio_7_11_5 0x00000001L | |
| 95 | +#define BIF_BX1_BIF_FB_EN__FB_WRITE_EN_MASK_nbio_7_11_5 0x00000002L | |
| 90 | 96 | |
| 91 | 97 | static void nbif_v6_3_1_remap_hdp_registers(struct amdgpu_device *adev) |
| 92 | 98 | { |
| 93 | 99 | if (amdgpu_ip_version(adev, NBIO_HWIP, 0) == IP_VERSION(7, 11, 5)) { |
| 94 | − WREG32_SOC15(NBIO, 0, regBIF_BX0_REMAP_HDP_MEM_FLUSH_CNTL_nbio_7_11_5, | |
| 100 | + WREG32_SOC15(NBIO, 0, regBIF_BX1_REMAP_HDP_MEM_FLUSH_CNTL_nbio_7_11_5, | |
| 95 | 101 | adev->rmmio_remap.reg_offset + KFD_MMIO_REMAP_HDP_MEM_FLUSH_CNTL); |
| 96 | − WREG32_SOC15(NBIO, 0, regBIF_BX0_REMAP_HDP_REG_FLUSH_CNTL_nbio_7_11_5, | |
| 102 | + WREG32_SOC15(NBIO, 0, regBIF_BX1_REMAP_HDP_REG_FLUSH_CNTL_nbio_7_11_5, | |
| 97 | 103 | adev->rmmio_remap.reg_offset + KFD_MMIO_REMAP_HDP_REG_FLUSH_CNTL); |
| 98 | 104 | } else { |
| 99 | 105 | WREG32_SOC15(NBIO, 0, regBIF_BX0_REMAP_HDP_MEM_FLUSH_CNTL, |
| @@ -110,7 +116,7 @@ static u32 nbif_v6_3_1_get_rev_id(struct amdgpu_device *adev) · 10 unmodified lines | ||
| 110 | 116 | if (amdgpu_ip_version(adev, NBIO_HWIP, 0) == IP_VERSION(7, 11, 4)) |
| 111 | 117 | tmp = RREG32_SOC15(NBIO, 0, regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0_nbif_4_10); |
| 112 | 118 | else if (amdgpu_ip_version(adev, NBIO_HWIP, 0) == IP_VERSION(7, 11, 5)) |
| 113 | − tmp = RREG32_SOC15(NBIO, 0, regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0_nbio_7_11_5); | |
| 119 | + tmp = RREG32_SOC15(NBIO, 0, regRCC_STRAP1_RCC_DEV0_EPF0_STRAP0_nbio_7_11_5); | |
| 114 | 120 | else |
| 115 | 121 | tmp = RREG32_SOC15(NBIO, 0, regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0); |
| 116 | 122 | |
| @@ -124,11 +130,11 @@ static void nbif_v6_3_1_mc_access_enable(struct amdgpu_device *adev, bool enable · 7 unmodified lines | ||
| 124 | 130 | { |
| 125 | 131 | if (amdgpu_ip_version(adev, NBIO_HWIP, 0) == IP_VERSION(7, 11, 5)) { |
| 126 | 132 | if (enable) |
| 127 | − WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN_nbio_7_11_5, | |
| 128 | − BIF_BX0_BIF_FB_EN__FB_READ_EN_MASK | | |
| 129 | − BIF_BX0_BIF_FB_EN__FB_WRITE_EN_MASK); | |
| 133 | + WREG32_SOC15(NBIO, 0, regBIF_BX1_BIF_FB_EN_nbio_7_11_5, | |
| 134 | + BIF_BX1_BIF_FB_EN__FB_READ_EN_MASK_nbio_7_11_5 | | |
| 135 | + BIF_BX1_BIF_FB_EN__FB_WRITE_EN_MASK_nbio_7_11_5); | |
| 130 | 136 | else |
| 131 | − WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN_nbio_7_11_5, 0); | |
| 137 | + WREG32_SOC15(NBIO, 0, regBIF_BX1_BIF_FB_EN_nbio_7_11_5, 0); | |
| 132 | 138 | } else { |
| 133 | 139 | if (enable) |
| 134 | 140 | WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN, |
| @@ -383,12 +389,12 @@ static void nbif_v6_3_1_ih_control(struct amdgpu_device *adev) · 248 unmodified lines | ||
| 383 | 389 | |
| 384 | 390 | /* setup interrupt control */ |
| 385 | 391 | if (amdgpu_ip_version(adev, NBIO_HWIP, 0) == IP_VERSION(7, 11, 5)) |
| 386 | − WREG32_SOC15(NBIO, 0, regBIF_BX0_INTERRUPT_CNTL2_nbio_7_11_5, | |
| 392 | + WREG32_SOC15(NBIO, 0, regBIF_BX1_INTERRUPT_CNTL2_nbio_7_11_5, | |
| 387 | 393 | adev->dummy_page_addr >> 8); |
| 388 | 394 | else |
| 389 | 395 | WREG32_SOC15(NBIO, 0, regBIF_BX0_INTERRUPT_CNTL2, adev->dummy_page_addr >> 8); |
| 390 | 396 | |
| 391 | − interrupt_cntl = RREG32_SOC15(NBIO, 0, regBIF_BX0_INTERRUPT_CNTL_nbio_7_11_5); | |
| 397 | + interrupt_cntl = RREG32_SOC15(NBIO, 0, regBIF_BX1_INTERRUPT_CNTL_nbio_7_11_5); | |
| 392 | 398 | /* |
| 393 | 399 | * BIF_BX0_INTERRUPT_CNTL__IH_DUMMY_RD_OVERRIDE_MASK=0 - dummy read disabled with msi, enabled without msi |
| 394 | 400 | * BIF_BX0_INTERRUPT_CNTL__IH_DUMMY_RD_OVERRIDE_MASK=1 - dummy read controlled by IH_DUMMY_RD_EN |
| @@ -401,7 +407,7 @@ static void nbif_v6_3_1_ih_control(struct amdgpu_device *adev) · 6 unmodified lines | ||
| 401 | 407 | IH_REQ_NONSNOOP_EN, 0); |
| 402 | 408 | |
| 403 | 409 | if (amdgpu_ip_version(adev, NBIO_HWIP, 0) == IP_VERSION(7, 11, 5)) |
| 404 | − WREG32_SOC15(NBIO, 0, regBIF_BX0_INTERRUPT_CNTL_nbio_7_11_5, interrupt_cntl); | |
| 410 | + WREG32_SOC15(NBIO, 0, regBIF_BX1_INTERRUPT_CNTL_nbio_7_11_5, interrupt_cntl); | |
| 405 | 411 | else |
| 406 | 412 | WREG32_SOC15(NBIO, 0, regBIF_BX0_INTERRUPT_CNTL, interrupt_cntl); |
| 407 | 413 | } |
| @@ -427,7 +433,7 @@ nbif_v6_3_1_get_clockgating_state(struct amdgpu_device *adev, · 19 unmodified lines | ||
| 427 | 433 | static u32 nbif_v6_3_1_get_hdp_flush_req_offset(struct amdgpu_device *adev) |
| 428 | 434 | { |
| 429 | 435 | if (amdgpu_ip_version(adev, NBIO_HWIP, 0) == IP_VERSION(7, 11, 5)) |
| 430 | − return SOC15_REG_OFFSET(NBIO, 0, regBIF_BX_PF0_GPU_HDP_FLUSH_REQ_nbio_7_11_5); | |
| 436 | + return SOC15_REG_OFFSET(NBIO, 0, regBIF_BX_PF1_GPU_HDP_FLUSH_REQ_nbio_7_11_5); | |
| 431 | 437 | else |
| 432 | 438 | return SOC15_REG_OFFSET(NBIO, 0, regBIF_BX_PF0_GPU_HDP_FLUSH_REQ); |
| 433 | 439 | } |
| @@ -435,7 +441,7 @@ static u32 nbif_v6_3_1_get_hdp_flush_req_offset(struct amdgpu_device *adev) · 1 unmodified line | ||
| 435 | 441 | static u32 nbif_v6_3_1_get_hdp_flush_done_offset(struct amdgpu_device *adev) |
| 436 | 442 | { |
| 437 | 443 | if (amdgpu_ip_version(adev, NBIO_HWIP, 0) == IP_VERSION(7, 11, 5)) |
| 438 | − return SOC15_REG_OFFSET(NBIO, 0, regBIF_BX_PF0_GPU_HDP_FLUSH_DONE_nbio_7_11_5); | |
| 444 | + return SOC15_REG_OFFSET(NBIO, 0, regBIF_BX_PF1_GPU_HDP_FLUSH_DONE_nbio_7_11_5); | |
| 439 | 445 | else |
| 440 | 446 | return SOC15_REG_OFFSET(NBIO, 0, regBIF_BX_PF0_GPU_HDP_FLUSH_DONE); |
| 441 | 447 | } |
| @@ -622,7 +628,7 @@ static void nbif_v6_3_1_set_reg_remap(struct amdgpu_device *adev) · 180 unmodified lines | ||
| 622 | 628 | } else { |
| 623 | 629 | if (amdgpu_ip_version(adev, NBIO_HWIP, 0) == IP_VERSION(7, 11, 5)) |
| 624 | 630 | adev->rmmio_remap.reg_offset = SOC15_REG_OFFSET(NBIO, 0, |
| 625 | − regBIF_BX_PF0_HDP_MEM_COHERENCY_FLUSH_CNTL_nbio_7_11_5) << 2; | |
| 631 | + regBIF_BX_PF1_HDP_MEM_COHERENCY_FLUSH_CNTL_nbio_7_11_5) << 2; | |
| 626 | 632 | else |
| 627 | 633 | adev->rmmio_remap.reg_offset = SOC15_REG_OFFSET(NBIO, 0, |
| 628 | 634 | regBIF_BX_PF0_HDP_MEM_COHERENCY_FLUSH_CNTL) << 2; |
| @@ -110,6 +110,7 @@ enum psp_gfx_cmd_id · 109 unmodified lines | ||
| 110 | 110 | GFX_CMD_ID_PERF_HW = 0x0000004C, /* performance monitor */ |
| 111 | 111 | GFX_CMD_ID_FB_FW_RESERV_ADDR = 0x00000050, /* Query FW reservation addr */ |
| 112 | 112 | GFX_CMD_ID_FB_FW_RESERV_EXT_ADDR = 0x00000051, /* Query FW reservation extended addr */ |
| 113 | + GFX_CMD_ID_SET_MMHUB_ECO_SEC_LEVEL = 0x0000005D, /* Set MMHUB ECO sec lvls on VCN block */ | |
| 113 | 114 | }; |
| 114 | 115 | |
| 115 | 116 | /* PSP boot config sub-commands */ |
| @@ -33,9 +33,22 @@ · 32 unmodified lines | ||
| 33 | 33 | |
| 34 | 34 | MODULE_FIRMWARE("amdgpu/psp_15_0_0_toc.bin"); |
| 35 | 35 | MODULE_FIRMWARE("amdgpu/psp_15_0_0_ta.bin"); |
| 36 | +MODULE_FIRMWARE("amdgpu/psp_15_0_5_toc.bin"); | |
| 37 | +MODULE_FIRMWARE("amdgpu/psp_15_0_5_ta.bin"); | |
| 36 | 38 | MODULE_FIRMWARE("amdgpu/psp_15_0_9_toc.bin"); |
| 37 | 39 | MODULE_FIRMWARE("amdgpu/psp_15_0_9_ta.bin"); |
| 38 | 40 | |
| 41 | +#define regMPASP_PCRU0_MPASP_C2PMSG_64 0x4280 | |
| 42 | +#define regMPASP_PCRU0_MPASP_C2PMSG_64_BASE_IDX 2 | |
| 43 | +#define regMPASP_PCRU0_MPASP_C2PMSG_67 0x4283 | |
| 44 | +#define regMPASP_PCRU0_MPASP_C2PMSG_67_BASE_IDX 2 | |
| 45 | +#define regMPASP_PCRU0_MPASP_C2PMSG_69 0x4285 | |
| 46 | +#define regMPASP_PCRU0_MPASP_C2PMSG_69_BASE_IDX 2 | |
| 47 | +#define regMPASP_PCRU0_MPASP_C2PMSG_70 0x4286 | |
| 48 | +#define regMPASP_PCRU0_MPASP_C2PMSG_70_BASE_IDX 2 | |
| 49 | +#define regMPASP_PCRU0_MPASP_C2PMSG_71 0x4287 | |
| 50 | +#define regMPASP_PCRU0_MPASP_C2PMSG_71_BASE_IDX 2 | |
| 51 | + | |
| 39 | 52 | static int psp_v15_0_0_init_microcode(struct psp_context *psp) |
| 40 | 53 | { |
| 41 | 54 | struct amdgpu_device *adev = psp->adev; |
| @@ -71,14 +84,25 @@ static int psp_v15_0_0_ring_stop(struct psp_context *psp, · 29 unmodified lines | ||
| 71 | 84 | ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_SMN_C2PMSG_101), |
| 72 | 85 | 0x80000000, 0x80000000, false); |
| 73 | 86 | } else { |
| 74 | − /* Write the ring destroy command*/ | |
| 75 | − WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64, | |
| 76 | − GFX_CTRL_CMD_ID_DESTROY_RINGS); | |
| 77 | − /* there might be handshake issue with hardware which needs delay */ | |
| 78 | − mdelay(20); | |
| 79 | − /* Wait for response flag (bit 31) */ | |
| 80 | − ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64), | |
| 81 | − 0x80000000, 0x80000000, false); | |
| 87 | + if (amdgpu_ip_version(adev, MP0_HWIP, 0) == IP_VERSION(15, 0, 5)) { | |
| 88 | + /* Write the ring destroy command*/ | |
| 89 | + WREG32_SOC15(MP0, 0, regMPASP_PCRU0_MPASP_C2PMSG_64, | |
| 90 | + GFX_CTRL_CMD_ID_DESTROY_RINGS); | |
| 91 | + /* there might be handshake issue with hardware which needs delay */ | |
| 92 | + mdelay(20); | |
| 93 | + /* Wait for response flag (bit 31) */ | |
| 94 | + ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_PCRU0_MPASP_C2PMSG_64), | |
| 95 | + 0x80000000, 0x80000000, false); | |
| 96 | + } else { | |
| 97 | + /* Write the ring destroy command*/ | |
| 98 | + WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64, | |
| 99 | + GFX_CTRL_CMD_ID_DESTROY_RINGS); | |
| 100 | + /* there might be handshake issue with hardware which needs delay */ | |
| 101 | + mdelay(20); | |
| 102 | + /* Wait for response flag (bit 31) */ | |
| 103 | + ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64), | |
| 104 | + 0x80000000, 0x80000000, false); | |
| 105 | + } | |
| 82 | 106 | } |
| 83 | 107 | |
| 84 | 108 | return ret; |
| @@ -118,14 +142,44 @@ static int psp_v15_0_0_ring_create(struct psp_context *psp, · 33 unmodified lines | ||
| 118 | 142 | 0x80000000, 0x8000FFFF, false); |
| 119 | 143 | |
| 120 | 144 | } else { |
| 121 | − /* Wait for sOS ready for ring creation */ | |
| 122 | − ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64), | |
| 123 | − 0x80000000, 0x80000000, false); | |
| 145 | + if (amdgpu_ip_version(adev, MP0_HWIP, 0) == IP_VERSION(15, 0, 5)) { | |
| 146 | + /* Wait for sOS ready for ring creation */ | |
| 147 | + ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_PCRU0_MPASP_C2PMSG_64), | |
| 148 | + 0x80000000, 0x80000000, false); | |
| 124 | 149 | if (ret) { |
| 125 | 150 | DRM_ERROR("Failed to wait for trust OS ready for ring creation\n"); |
| 126 | 151 | return ret; |
| 127 | 152 | } |
| 128 | 153 | |
| 154 | + /* Write low address of the ring to C2PMSG_69 */ | |
| 155 | + psp_ring_reg = lower_32_bits(ring->ring_mem_mc_addr); | |
| 156 | + WREG32_SOC15(MP0, 0, regMPASP_PCRU0_MPASP_C2PMSG_69, psp_ring_reg); | |
| 157 | + /* Write high address of the ring to C2PMSG_70 */ | |
| 158 | + psp_ring_reg = upper_32_bits(ring->ring_mem_mc_addr); | |
| 159 | + WREG32_SOC15(MP0, 0, regMPASP_PCRU0_MPASP_C2PMSG_70, psp_ring_reg); | |
| 160 | + /* Write size of ring to C2PMSG_71 */ | |
| 161 | + psp_ring_reg = ring->ring_size; | |
| 162 | + WREG32_SOC15(MP0, 0, regMPASP_PCRU0_MPASP_C2PMSG_71, psp_ring_reg); | |
| 163 | + /* Write the ring initialization command to C2PMSG_64 */ | |
| 164 | + psp_ring_reg = ring_type; | |
| 165 | + psp_ring_reg = psp_ring_reg << 16; | |
| 166 | + WREG32_SOC15(MP0, 0, regMPASP_PCRU0_MPASP_C2PMSG_64, psp_ring_reg); | |
| 167 | + | |
| 168 | + /* there might be handshake issue with hardware which needs delay */ | |
| 169 | + mdelay(20); | |
| 170 | + | |
| 171 | + /* Wait for response flag (bit 31) in C2PMSG_64 */ | |
| 172 | + ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_PCRU0_MPASP_C2PMSG_64), | |
| 173 | + 0x80000000, 0x8000FFFF, false); | |
| 174 | + } else { | |
| 175 | + /* Wait for sOS ready for ring creation */ | |
| 176 | + ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64), | |
| 177 | + 0x80000000, 0x80000000, false); | |
| 178 | + if (ret) { | |
| 179 | + DRM_ERROR("Failed to wait for trust OS ready for ring creation\n"); | |
| 180 | + return ret; | |
| 181 | + } | |
| 182 | + | |
| 129 | 183 | /* Write low address of the ring to C2PMSG_69 */ |
| 130 | 184 | psp_ring_reg = lower_32_bits(ring->ring_mem_mc_addr); |
| 131 | 185 | WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_69, psp_ring_reg); |
| @@ -146,6 +200,7 @@ static int psp_v15_0_0_ring_create(struct psp_context *psp, · 14 unmodified lines | ||
| 146 | 200 | /* Wait for response flag (bit 31) in C2PMSG_64 */ |
| 147 | 201 | ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64), |
| 148 | 202 | 0x80000000, 0x8000FFFF, false); |
| 203 | + } | |
| 149 | 204 | } |
| 150 | 205 | |
| 151 | 206 | return ret; |
| @@ -176,8 +231,12 @@ static uint32_t psp_v15_0_0_ring_get_wptr(struct psp_context *psp) · 24 unmodified lines | ||
| 176 | 231 | |
| 177 | 232 | if (amdgpu_sriov_vf(adev)) |
| 178 | 233 | data = RREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_102); |
| 179 | − else | |
| 180 | − data = RREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_67); | |
| 234 | + else { | |
| 235 | + if (amdgpu_ip_version(adev, MP0_HWIP, 0) == IP_VERSION(15, 0, 5)) | |
| 236 | + data = RREG32_SOC15(MP0, 0, regMPASP_PCRU0_MPASP_C2PMSG_67); | |
| 237 | + else | |
| 238 | + data = RREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_67); | |
| 239 | + } | |
| 181 | 240 | |
| 182 | 241 | return data; |
| 183 | 242 | } |
| @@ -190,8 +249,12 @@ static void psp_v15_0_0_ring_set_wptr(struct psp_context *psp, uint32_t value) · 6 unmodified lines | ||
| 190 | 249 | WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_102, value); |
| 191 | 250 | WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_101, |
| 192 | 251 | GFX_CTRL_CMD_ID_CONSUME_CMD); |
| 193 | − } else | |
| 194 | − WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_67, value); | |
| 252 | + } else { | |
| 253 | + if (amdgpu_ip_version(adev, MP0_HWIP, 0) == IP_VERSION(15, 0, 5)) | |
| 254 | + WREG32_SOC15(MP0, 0, regMPASP_PCRU0_MPASP_C2PMSG_67, value); | |
| 255 | + else | |
| 256 | + WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_67, value); | |
| 257 | + } | |
| 195 | 258 | } |
| 196 | 259 | |
| 197 | 260 | static const struct psp_funcs psp_v15_0_0_funcs = { |
| @@ -406,6 +406,7 @@ soc21_asic_reset_method(struct amdgpu_device *adev) · 405 unmodified lines | ||
| 406 | 406 | case IP_VERSION(14, 0, 4): |
| 407 | 407 | case IP_VERSION(14, 0, 5): |
| 408 | 408 | case IP_VERSION(15, 0, 0): |
| 409 | + case IP_VERSION(15, 0, 5): | |
| 409 | 410 | case IP_VERSION(15, 0, 9): |
| 410 | 411 | return AMD_RESET_METHOD_MODE2; |
| 411 | 412 | default: |
| @@ -795,6 +795,27 @@ static int vcn_v5_0_0_start_dpg_mode(struct amdgpu_vcn_inst *vinst, · 794 unmodified lines | ||
| 795 | 795 | return 0; |
| 796 | 796 | } |
| 797 | 797 | |
| 798 | +/** | |
| 799 | + * vcn_v5_0_0_set_mmhub_eco_sec_level - set vcn sec lvl reg | |
| 800 | + * | |
| 801 | + * @adev: amdgpu_device pointer | |
| 802 | + * | |
| 803 | + * request psp to set sec lvl | |
| 804 | + */ | |
| 805 | +static int vcn_v5_0_0_set_mmhub_eco_sec_level(struct amdgpu_device *adev) | |
| 806 | +{ | |
| 807 | + int r = 0; | |
| 808 | + | |
| 809 | + if (amdgpu_ip_version(adev, VCN_HWIP, 0) == IP_VERSION(5, 3, 0)) { | |
| 810 | + if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { | |
| 811 | + /* Request to PSP to program VCN secure lvl */ | |
| 812 | + r = psp_set_mmhub_eco_sec_level(adev); | |
| 813 | + } | |
| 814 | + } | |
| 815 | + | |
| 816 | + return r; | |
| 817 | +} | |
| 818 | + | |
| 798 | 819 | /** |
| 799 | 820 | * vcn_v5_0_0_start - VCN start |
| 800 | 821 | * |
| @@ -819,6 +840,11 @@ static int vcn_v5_0_0_start(struct amdgpu_vcn_inst *vinst) · 18 unmodified lines | ||
| 819 | 840 | |
| 820 | 841 | fw_shared = adev->vcn.inst[i].fw_shared.cpu_addr; |
| 821 | 842 | |
| 843 | + /* program VCN secure lvl register */ | |
| 844 | + r = vcn_v5_0_0_set_mmhub_eco_sec_level(adev); | |
| 845 | + if (r) | |
| 846 | + return r; | |
| 847 | + | |
| 822 | 848 | if (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG) |
| 823 | 849 | return vcn_v5_0_0_start_dpg_mode(vinst, adev->vcn.inst[i].indirect_sram); |
| 824 | 850 | |
| @@ -3109,10 +3109,14 @@ static int kfd_ioctl_set_debug_trap(struct file *filep, struct kfd_process *p, v · 3108 unmodified lines | ||
| 3109 | 3109 | goto out; |
| 3110 | 3110 | } |
| 3111 | 3111 | |
| 3112 | − /* Check if target is still PTRACED. */ | |
| 3112 | + /* | |
| 3113 | + * Verify debugger has permission to debug target process. | |
| 3114 | + * For cross-process debugging, require active ptrace relationship. | |
| 3115 | + * This applies to ALL operations to prevent unauthorized interference. | |
| 3116 | + */ | |
| 3113 | 3117 | rcu_read_lock(); |
| 3114 | − if (target != p && args->op != KFD_IOC_DBG_TRAP_DISABLE | |
| 3115 | − && ptrace_parent(target->lead_thread) != current) { | |
| 3118 | + if (target != p && ptrace_parent(target->lead_thread) != current | |
| 3119 | + && target->debugger_process != p) { | |
| 3116 | 3120 | pr_err("PID %i is not PTRACED and cannot be debugged\n", args->pid); |
| 3117 | 3121 | r = -EPERM; |
| 3118 | 3122 | } |
| @@ -1412,6 +1412,15 @@ int kfd_parse_crat_table(void *crat_image, struct list_head *device_list, · 1411 unmodified lines | ||
| 1412 | 1412 | break; |
| 1413 | 1413 | } |
| 1414 | 1414 | |
| 1415 | + /* Validate subtype fits within remaining image */ | |
| 1416 | + if ((char *)sub_type_hdr + sub_type_hdr->length > | |
| 1417 | + (char *)crat_image + image_len) { | |
| 1418 | + pr_warn("CRAT subtype length %u exceeds image bounds\n", | |
| 1419 | + sub_type_hdr->length); | |
| 1420 | + ret = -EINVAL; | |
| 1421 | + break; | |
| 1422 | + } | |
| 1423 | + | |
| 1415 | 1424 | if (sub_type_hdr->flags & CRAT_SUBTYPE_FLAGS_ENABLED) { |
| 1416 | 1425 | ret = kfd_parse_subtype(sub_type_hdr, device_list); |
| 1417 | 1426 | if (ret) |
| @@ -524,6 +524,9 @@ int kfd_criu_restore_event(struct file *devkfd, · 523 unmodified lines | ||
| 524 | 524 | |
| 525 | 525 | ret = create_other_event(p, ev, &ev_priv->event_id); |
| 526 | 526 | break; |
| 527 | + default: | |
| 528 | + ret = -EINVAL; | |
| 529 | + break; | |
| 527 | 530 | } |
| 528 | 531 | mutex_unlock(&p->event_mutex); |
| 529 | 532 | |
| @@ -545,15 +548,27 @@ int kfd_criu_checkpoint_events(struct kfd_process *p, · 15 unmodified lines | ||
| 545 | 548 | int ret = 0; |
| 546 | 549 | struct kfd_event *ev; |
| 547 | 550 | uint32_t ev_id; |
| 551 | + uint32_t num_events; | |
| 548 | 552 | |
| 549 | − uint32_t num_events = kfd_get_num_events(p); | |
| 553 | + /* Serialize the count and the walk below against concurrent event | |
| 554 | + * create/destroy. Those paths take only p->event_mutex, not the | |
| 555 | + * p->mutex held by the CRIU checkpoint caller, so without this the | |
| 556 | + * event_idr can grow between kfd_get_num_events() and the loop and the | |
| 557 | + * walk writes past the ev_privs allocation. | |
| 558 | + */ | |
| 559 | + mutex_lock(&p->event_mutex); | |
| 550 | 560 | |
| 551 | − if (!num_events) | |
| 561 | + num_events = kfd_get_num_events(p); | |
| 562 | + if (!num_events) { | |
| 563 | + mutex_unlock(&p->event_mutex); | |
| 552 | 564 | return 0; |
| 565 | + } | |
| 553 | 566 | |
| 554 | 567 | ev_privs = kvzalloc(num_events * sizeof(*ev_privs), GFP_KERNEL); |
| 555 | − if (!ev_privs) | |
| 568 | + if (!ev_privs) { | |
| 569 | + mutex_unlock(&p->event_mutex); | |
| 556 | 570 | return -ENOMEM; |
| 571 | + } | |
| 557 | 572 | |
| 558 | 573 | |
| 559 | 574 | idr_for_each_entry(&p->event_idr, ev, ev_id) { |
| @@ -594,6 +609,8 @@ int kfd_criu_checkpoint_events(struct kfd_process *p, · 34 unmodified lines | ||
| 594 | 609 | i++; |
| 595 | 610 | } |
| 596 | 611 | |
| 612 | + mutex_unlock(&p->event_mutex); | |
| 613 | + | |
| 597 | 614 | ret = copy_to_user(user_priv_data + *priv_data_offset, |
| 598 | 615 | ev_privs, num_events * sizeof(*ev_privs)); |
| 599 | 616 | if (ret) { |
| @@ -383,7 +383,7 @@ int pqm_create_queue(struct process_queue_manager *pqm, · 382 unmodified lines | ||
| 383 | 383 | false); |
| 384 | 384 | if (retval) { |
| 385 | 385 | dev_err(dev->adev->dev, "failed to allocate process context bo\n"); |
| 386 | − return retval; | |
| 386 | + goto err_allocate_pqn; | |
| 387 | 387 | } |
| 388 | 388 | memset(pdd->proc_ctx_cpu_ptr, 0, AMDGPU_MES_PROC_CTX_SIZE); |
| 389 | 389 | } |
| @@ -288,7 +288,7 @@ int kfd_queue_acquire_buffers(struct kfd_process_device *pdd, struct queue_prope · 287 unmodified lines | ||
| 288 | 288 | } |
| 289 | 289 | err = kfd_queue_buffer_get(vm, (void *)properties->eop_ring_buffer_address, |
| 290 | 290 | &properties->eop_buf_bo, |
| 291 | − ALIGN(properties->eop_ring_buffer_size, PAGE_SIZE)); | |
| 291 | + ALIGN((u64)properties->eop_ring_buffer_size, PAGE_SIZE)); | |
| 292 | 292 | if (err) |
| 293 | 293 | goto out_err_unreserve; |
| 294 | 294 | } |
| @@ -2189,6 +2189,11 @@ static int amdgpu_dm_init(struct amdgpu_device *adev) · 2188 unmodified lines | ||
| 2189 | 2189 | init_data.flags.unify_link_enc_assignment = true; |
| 2190 | 2190 | init_data.flags.usb4_bw_alloc_support = true; |
| 2191 | 2191 | } |
| 2192 | + | |
| 2193 | + /* DCN201 audio desyncs using DP SS */ | |
| 2194 | + if (adev->apu_flags & AMD_APU_IS_CYAN_SKILLFISH2) | |
| 2195 | + init_data.flags.ignore_dpref_ss = true; | |
| 2196 | + | |
| 2192 | 2197 | retrieve_dmi_info(&adev->dm); |
| 2193 | 2198 | if (adev->dm.edp0_on_dp1_quirk) |
| 2194 | 2199 | init_data.flags.support_edp0_on_dp1 = true; |
| @@ -9903,25 +9908,6 @@ static void remove_stream(struct amdgpu_device *adev, · 7708 unmodified lines | ||
| 9903 | 9908 | acrtc->enabled = false; |
| 9904 | 9909 | } |
| 9905 | 9910 | |
| 9906 | −static void prepare_flip_isr(struct amdgpu_crtc *acrtc) | |
| 9907 | −{ | |
| 9908 | − | |
| 9909 | − assert_spin_locked(&acrtc->base.dev->event_lock); | |
| 9910 | − WARN_ON(acrtc->event); | |
| 9911 | − | |
| 9912 | − acrtc->event = acrtc->base.state->event; | |
| 9913 | − | |
| 9914 | − /* Set the flip status */ | |
| 9915 | − acrtc->pflip_status = AMDGPU_FLIP_SUBMITTED; | |
| 9916 | − | |
| 9917 | − /* Mark this event as consumed */ | |
| 9918 | − acrtc->base.state->event = NULL; | |
| 9919 | − | |
| 9920 | − drm_dbg_state(acrtc->base.dev, | |
| 9921 | − "crtc:%d, pflip_stat:AMDGPU_FLIP_SUBMITTED\n", | |
| 9922 | − acrtc->crtc_id); | |
| 9923 | −} | |
| 9924 | − | |
| 9925 | 9911 | static void update_freesync_state_on_stream( |
| 9926 | 9912 | struct amdgpu_display_manager *dm, |
| 9927 | 9913 | struct dm_crtc_state *new_crtc_state, |
| @@ -10274,17 +10260,47 @@ static void dm_arm_vblank_event(struct amdgpu_crtc *acrtc, · 346 unmodified lines | ||
| 10274 | 10260 | return; |
| 10275 | 10261 | |
| 10276 | 10262 | if (pflip_update) { |
| 10277 | − drm_crtc_vblank_get(&acrtc->base); | |
| 10278 | 10263 | WARN_ON(acrtc->pflip_status != AMDGPU_FLIP_NONE); |
| 10279 | − /* Arm flip completion handling and event delivery after programming. */ | |
| 10280 | − prepare_flip_isr(acrtc); | |
| 10264 | + WARN_ON(acrtc->event); | |
| 10265 | + | |
| 10266 | + acrtc->pflip_status = AMDGPU_FLIP_SUBMITTED; | |
| 10267 | + acrtc->event = acrtc->base.state->event; | |
| 10268 | + acrtc->base.state->event = NULL; | |
| 10269 | + | |
| 10270 | + drm_dbg_state(acrtc->base.dev, | |
| 10271 | + "crtc:%d, pflip_stat:AMDGPU_FLIP_SUBMITTED\n", | |
| 10272 | + acrtc->crtc_id); | |
| 10281 | 10273 | } else if (cursor_update) { |
| 10282 | − drm_crtc_vblank_get(&acrtc->base); | |
| 10283 | 10274 | acrtc->event = acrtc->base.state->event; |
| 10284 | 10275 | acrtc->base.state->event = NULL; |
| 10285 | 10276 | } |
| 10286 | 10277 | } |
| 10287 | 10278 | |
| 10279 | +/** | |
| 10280 | + * dm_arm_vblank_event_pre_programming - Prepare for programming | |
| 10281 | + * @acrtc: The amdgpu CRTC to prepare | |
| 10282 | + * @acrtc_state: The new CRTC state | |
| 10283 | + * @pflip_update: Whether a page flip is being programmed | |
| 10284 | + * @cursor_update: Whether a cursor update is being programmed | |
| 10285 | + * | |
| 10286 | + * Grab a reference on the vblank counter if a page flip or cursor update is to | |
| 10287 | + * be programmed. Do this before programming so the HW is not in any | |
| 10288 | + * idle-optimized state (such as PSR). | |
| 10289 | + */ | |
| 10290 | +static void dm_arm_vblank_event_pre_programming(struct amdgpu_crtc *acrtc, | |
| 10291 | + struct dm_crtc_state *acrtc_state, | |
| 10292 | + bool pflip_update, | |
| 10293 | + bool cursor_update) | |
| 10294 | +{ | |
| 10295 | + assert_spin_locked(&acrtc->base.dev->event_lock); | |
| 10296 | + | |
| 10297 | + if (!acrtc->base.state->event || acrtc_state->active_planes == 0) | |
| 10298 | + return; | |
| 10299 | + | |
| 10300 | + if (pflip_update || cursor_update) | |
| 10301 | + drm_crtc_vblank_get(&acrtc->base); | |
| 10302 | +} | |
| 10303 | + | |
| 10288 | 10304 | static void amdgpu_dm_commit_planes(struct drm_atomic_commit *state, |
| 10289 | 10305 | struct drm_device *dev, |
| 10290 | 10306 | struct amdgpu_display_manager *dm, |
| @@ -10550,16 +10566,19 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_commit *state, · 259 unmodified lines | ||
| 10550 | 10566 | } |
| 10551 | 10567 | } |
| 10552 | 10568 | |
| 10553 | − /* | |
| 10554 | − * DCE depends on a combination of GRPH_FLIP, VLINE0, and VUPDATE for | |
| 10555 | − * event delivery. Only GRPH_FLIP handler can send pflip events, and it | |
| 10556 | − * only fires if HW latched to the flip. Maintain legacy behavior by | |
| 10557 | − * arming event before programming. | |
| 10558 | − */ | |
| 10559 | − if (amdgpu_ip_version(dm->adev, DCE_HWIP, 0) == 0) { | |
| 10560 | − scoped_guard(spinlock_irqsave, &pcrtc->dev->event_lock) { | |
| 10569 | + scoped_guard(spinlock_irqsave, &pcrtc->dev->event_lock) { | |
| 10570 | + dm_arm_vblank_event_pre_programming(acrtc_attach, acrtc_state, | |
| 10571 | + pflip_present, | |
| 10572 | + cursor_update); | |
| 10573 | + /* | |
| 10574 | + * DCE depends on a combination of GRPH_FLIP, VLINE0, and | |
| 10575 | + * VUPDATE for event delivery. Only GRPH_FLIP handler can send | |
| 10576 | + * pflip events, and it only fires if HW latched to the flip. | |
| 10577 | + * Maintain legacy behavior by arming event before programming. | |
| 10578 | + */ | |
| 10579 | + if (amdgpu_ip_version(dm->adev, DCE_HWIP, 0) == 0) { | |
| 10561 | 10580 | dm_arm_vblank_event(acrtc_attach, acrtc_state, |
| 10562 | − pflip_present, cursor_update); | |
| 10581 | + pflip_present, cursor_update); | |
| 10563 | 10582 | } |
| 10564 | 10583 | } |
| 10565 | 10584 | |
| @@ -119,7 +119,11 @@ void amdgpu_dm_crtc_set_static_screen_optimze( · 118 unmodified lines | ||
| 119 | 119 | struct dc_link *link = stream->link; |
| 120 | 120 | bool set_vsync_event = !sso_enable; |
| 121 | 121 | |
| 122 | − if (!allow_sr_entry) | |
| 122 | + /* | |
| 123 | + * allow_sr_entry gates only entry. A disable request must still set | |
| 124 | + * the vsync events to force Replay and PSR1 out and keep them blocked. | |
| 125 | + */ | |
| 126 | + if (sso_enable && !allow_sr_entry) | |
| 123 | 127 | return; |
| 124 | 128 | |
| 125 | 129 | amdgpu_dm_replay_set_event(dm, stream, |
| @@ -45,9 +45,7 @@ · 44 unmodified lines | ||
| 45 | 45 | clk_src->base.ctx |
| 46 | 46 | |
| 47 | 47 | #define DC_LOGGER \ |
| 48 | − calc_pll_cs->ctx->logger | |
| 49 | −#define DC_LOGGER_INIT() \ | |
| 50 | − struct calc_pll_clock_source *calc_pll_cs = &clk_src->calc_pll | |
| 48 | + CTX->logger | |
| 51 | 49 | |
| 52 | 50 | #undef FN |
| 53 | 51 | #define FN(reg_name, field_name) \ |
| @@ -287,6 +285,7 @@ static bool calc_pll_dividers_in_range( · 233 unmodified lines | ||
| 287 | 285 | } |
| 288 | 286 | |
| 289 | 287 | static uint32_t calculate_pixel_clock_pll_dividers( |
| 288 | + struct dce110_clk_src *clk_src, | |
| 290 | 289 | struct calc_pll_clock_source *calc_pll_cs, |
| 291 | 290 | struct pll_settings *pll_settings) |
| 292 | 291 | { |
| @@ -474,7 +473,7 @@ static uint32_t dce110_get_pix_clk_dividers_helper ( · 181 unmodified lines | ||
| 474 | 473 | { |
| 475 | 474 | uint32_t field = 0; |
| 476 | 475 | uint32_t pll_calc_error = MAX_PLL_CALC_ERROR; |
| 477 | − DC_LOGGER_INIT(); | |
| 476 | + | |
| 478 | 477 | /* Check if reference clock is external (not pcie/xtalin) |
| 479 | 478 | * HW Dce80 spec: |
| 480 | 479 | * 00 - PCIE_REFCLK, 01 - XTALIN, 02 - GENERICA, 03 - GENERICB |
| @@ -517,12 +516,14 @@ static uint32_t dce110_get_pix_clk_dividers_helper ( · 36 unmodified lines | ||
| 517 | 516 | /*Calculate Dividers by HDMI object, no SS case or SS case */ |
| 518 | 517 | pll_calc_error = |
| 519 | 518 | calculate_pixel_clock_pll_dividers( |
| 519 | + clk_src, | |
| 520 | 520 | &clk_src->calc_pll_hdmi, |
| 521 | 521 | pll_settings); |
| 522 | 522 | else |
| 523 | 523 | /*Calculate Dividers by default object, no SS case or SS case */ |
| 524 | 524 | pll_calc_error = |
| 525 | 525 | calculate_pixel_clock_pll_dividers( |
| 526 | + clk_src, | |
| 526 | 527 | &clk_src->calc_pll, |
| 527 | 528 | pll_settings); |
| 528 | 529 | |
| @@ -568,7 +569,6 @@ static uint32_t dce110_get_pix_clk_dividers( · 39 unmodified lines | ||
| 568 | 569 | { |
| 569 | 570 | struct dce110_clk_src *clk_src = TO_DCE110_CLK_SRC(cs); |
| 570 | 571 | uint32_t pll_calc_error = MAX_PLL_CALC_ERROR; |
| 571 | − DC_LOGGER_INIT(); | |
| 572 | 572 | |
| 573 | 573 | if (pix_clk_params == NULL || pll_settings == NULL |
| 574 | 574 | || pix_clk_params->requested_pix_clk_100hz == 0) { |
| @@ -600,7 +600,6 @@ static uint32_t dce112_get_pix_clk_dividers( · 25 unmodified lines | ||
| 600 | 600 | struct pll_settings *pll_settings) |
| 601 | 601 | { |
| 602 | 602 | struct dce110_clk_src *clk_src = TO_DCE110_CLK_SRC(cs); |
| 603 | − DC_LOGGER_INIT(); | |
| 604 | 603 | |
| 605 | 604 | if (pix_clk_params == NULL || pll_settings == NULL |
| 606 | 605 | || pix_clk_params->requested_pix_clk_100hz == 0) { |
| @@ -1442,8 +1441,6 @@ static uint32_t dcn3_get_pix_clk_dividers( · 835 unmodified lines | ||
| 1442 | 1441 | unsigned long long actual_pix_clk_100Hz = pix_clk_params ? pix_clk_params->requested_pix_clk_100hz : 0; |
| 1443 | 1442 | struct dce110_clk_src *clk_src = TO_DCE110_CLK_SRC(cs); |
| 1444 | 1443 | |
| 1445 | − DC_LOGGER_INIT(); | |
| 1446 | − | |
| 1447 | 1444 | if (pix_clk_params == NULL || pll_settings == NULL |
| 1448 | 1445 | || pix_clk_params->requested_pix_clk_100hz == 0) { |
| 1449 | 1446 | DC_LOG_ERROR( |
| @@ -1513,7 +1510,6 @@ static const struct clock_source_funcs dce110_clk_src_funcs = { · 63 unmodified lines | ||
| 1513 | 1510 | .get_dp_dto_frequency_100hz = get_dp_dto_frequency_100hz |
| 1514 | 1511 | }; |
| 1515 | 1512 | |
| 1516 | − | |
| 1517 | 1513 | static void get_ss_info_from_atombios( |
| 1518 | 1514 | struct dce110_clk_src *clk_src, |
| 1519 | 1515 | enum as_signal_type as_signal, |
| @@ -1526,7 +1522,7 @@ static void get_ss_info_from_atombios( · 6 unmodified lines | ||
| 1526 | 1522 | struct spread_spectrum_info *ss_info_cur; |
| 1527 | 1523 | struct spread_spectrum_data *ss_data_cur; |
| 1528 | 1524 | uint32_t i; |
| 1529 | − DC_LOGGER_INIT(); | |
| 1525 | + | |
| 1530 | 1526 | if (ss_entries_num == NULL) { |
| 1531 | 1527 | DC_LOG_SYNC( |
| 1532 | 1528 | "Invalid entry !!!\n"); |
| @@ -1657,6 +1653,7 @@ static void ss_info_from_atombios_create( · 124 unmodified lines | ||
| 1657 | 1653 | } |
| 1658 | 1654 | |
| 1659 | 1655 | static bool calc_pll_max_vco_construct( |
| 1656 | + struct dce110_clk_src *clk_src, | |
| 1660 | 1657 | struct calc_pll_clock_source *calc_pll_cs, |
| 1661 | 1658 | struct calc_pll_clock_source_init_data *init_data) |
| 1662 | 1659 | { |
| @@ -1808,6 +1805,7 @@ bool dce110_clk_src_construct( · 145 unmodified lines | ||
| 1808 | 1805 | ss_info_from_atombios_create(clk_src); |
| 1809 | 1806 | |
| 1810 | 1807 | if (!calc_pll_max_vco_construct( |
| 1808 | + clk_src, | |
| 1811 | 1809 | &clk_src->calc_pll, |
| 1812 | 1810 | &calc_pll_cs_init_data)) { |
| 1813 | 1811 | ASSERT_CRITICAL(false); |
| @@ -1822,7 +1820,7 @@ bool dce110_clk_src_construct( · 8 unmodified lines | ||
| 1822 | 1820 | |
| 1823 | 1821 | |
| 1824 | 1822 | if (!calc_pll_max_vco_construct( |
| 1825 | − &clk_src->calc_pll_hdmi, &calc_pll_cs_init_data_hdmi)) { | |
| 1823 | + clk_src, &clk_src->calc_pll_hdmi, &calc_pll_cs_init_data_hdmi)) { | |
| 1826 | 1824 | ASSERT_CRITICAL(false); |
| 1827 | 1825 | goto unexpected_failure; |
| 1828 | 1826 | } |
| @@ -359,14 +359,13 @@ void dml21_handle_phantom_streams_planes(const struct dc *dc, struct dc_state *c · 358 unmodified lines | ||
| 359 | 359 | main_plane = main_stream_status->plane_states[dc_plane_index]; |
| 360 | 360 | |
| 361 | 361 | /* create phantom planes for subvp enabled plane */ |
| 362 | − dml21_add_phantom_plane(dml_ctx, | |
| 363 | − dc, | |
| 364 | − context, | |
| 365 | − phantom_stream, | |
| 366 | − main_plane, | |
| 367 | − &dml_ctx->v21.mode_programming.programming->plane_programming[dml_plane_index]); | |
| 368 | − | |
| 369 | − phantoms_added = true; | |
| 362 | + if (dml21_add_phantom_plane(dml_ctx, | |
| 363 | + dc, | |
| 364 | + context, | |
| 365 | + phantom_stream, | |
| 366 | + main_plane, | |
| 367 | + &dml_ctx->v21.mode_programming.programming->plane_programming[dml_plane_index])) | |
| 368 | + phantoms_added = true; | |
| 370 | 369 | } |
| 371 | 370 | } |
| 372 | 371 | } |
| @@ -2452,7 +2452,7 @@ static void calculate_mcache_row_bytes( · 2451 unmodified lines | ||
| 2452 | 2452 | DML_ASSERT(*p->num_mcaches > 0); |
| 2453 | 2453 | } |
| 2454 | 2454 | |
| 2455 | −static void calculate_mcache_setting( | |
| 2455 | +static bool calculate_mcache_setting( | |
| 2456 | 2456 | struct dml2_core_internal_scratch *scratch, |
| 2457 | 2457 | struct dml2_core_calcs_calculate_mcache_setting_params *p) |
| 2458 | 2458 | { |
| @@ -2478,7 +2478,7 @@ static void calculate_mcache_setting( · 19 unmodified lines | ||
| 2478 | 2478 | *p->lc_comb_mcache = 0; |
| 2479 | 2479 | |
| 2480 | 2480 | if (!p->dcc_enable) |
| 2481 | − return; | |
| 2481 | + return true; | |
| 2482 | 2482 | |
| 2483 | 2483 | l->is_dual_plane = dml_is_420(p->source_format) || p->source_format == dml2_rgbe_alpha; |
| 2484 | 2484 | |
| @@ -2515,7 +2515,14 @@ static void calculate_mcache_setting( · 30 unmodified lines | ||
| 2515 | 2515 | l->l_p.mvmpg_per_mcache_lb = &l->mvmpg_per_mcache_lb_l; |
| 2516 | 2516 | |
| 2517 | 2517 | calculate_mcache_row_bytes(scratch, &l->l_p); |
| 2518 | − DML_ASSERT(*p->num_mcaches_l > 0); | |
| 2518 | + if (*p->num_mcaches_l == 0 || | |
| 2519 | + (p->surf_vert ? l->mvmpg_height_l : l->mvmpg_width_l) == 0) { | |
| 2520 | + DML_LOG_VERBOSE("DML::%s: degenerate luma viewport (num_mcaches_l=%u mvmpg_%s_l=%u) — mode not supported\n", | |
| 2521 | + __func__, *p->num_mcaches_l, | |
| 2522 | + p->surf_vert ? "height" : "width", | |
| 2523 | + p->surf_vert ? l->mvmpg_height_l : l->mvmpg_width_l); | |
| 2524 | + return false; | |
| 2525 | + } | |
| 2519 | 2526 | |
| 2520 | 2527 | if (l->is_dual_plane) { |
| 2521 | 2528 | l->c_p.num_chans = p->num_chans; |
| @@ -2551,7 +2558,14 @@ static void calculate_mcache_setting( · 29 unmodified lines | ||
| 2551 | 2558 | l->c_p.mvmpg_per_mcache_lb = &l->mvmpg_per_mcache_lb_c; |
| 2552 | 2559 | |
| 2553 | 2560 | calculate_mcache_row_bytes(scratch, &l->c_p); |
| 2554 | − DML_ASSERT(*p->num_mcaches_c > 0); | |
| 2561 | + if (*p->num_mcaches_c == 0 || | |
| 2562 | + (p->surf_vert ? l->mvmpg_height_c : l->mvmpg_width_c) == 0) { | |
| 2563 | + DML_LOG_VERBOSE("DML::%s: degenerate chroma viewport (num_mcaches_c=%u mvmpg_%s_c=%u) — mode not supported\n", | |
| 2564 | + __func__, *p->num_mcaches_c, | |
| 2565 | + p->surf_vert ? "height" : "width", | |
| 2566 | + p->surf_vert ? l->mvmpg_height_c : l->mvmpg_width_c); | |
| 2567 | + return false; | |
| 2568 | + } | |
| 2555 | 2569 | } |
| 2556 | 2570 | |
| 2557 | 2571 | // Sharing for iMALL access |
| @@ -2661,6 +2675,7 @@ static void calculate_mcache_setting( · 103 unmodified lines | ||
| 2661 | 2675 | |
| 2662 | 2676 | *p->mcache_shift_granularity_l = l->mvmpg_access_width_l; |
| 2663 | 2677 | *p->mcache_shift_granularity_c = l->mvmpg_access_width_c; |
| 2678 | + return true; | |
| 2664 | 2679 | } |
| 2665 | 2680 | |
| 2666 | 2681 | static void calculate_mall_bw_overhead_factor( |
| @@ -9457,7 +9472,10 @@ static bool dml_core_mode_support(struct dml2_core_calcs_mode_support_ex *in_out · 6790 unmodified lines | ||
| 9457 | 9472 | calculate_mcache_setting_params->mall_comb_mcache_c = &mode_lib->ms.mall_comb_mcache_c[k]; |
| 9458 | 9473 | calculate_mcache_setting_params->lc_comb_mcache = &mode_lib->ms.lc_comb_mcache[k]; |
| 9459 | 9474 | |
| 9460 | − calculate_mcache_setting(&mode_lib->scratch, calculate_mcache_setting_params); | |
| 9475 | + if (!calculate_mcache_setting(&mode_lib->scratch, calculate_mcache_setting_params)) { | |
| 9476 | + mode_lib->ms.support.ModeSupport = false; | |
| 9477 | + return false; | |
| 9478 | + } | |
| 9461 | 9479 | } |
| 9462 | 9480 | |
| 9463 | 9481 | calculate_mall_bw_overhead_factor( |
| @@ -10933,7 +10951,8 @@ static bool dml_core_mode_programming(struct dml2_core_calcs_mode_programming_ex · 1469 unmodified lines | ||
| 10933 | 10951 | calculate_mcache_setting_params->mall_comb_mcache_l = &mode_lib->mp.mall_comb_mcache_l[k]; |
| 10934 | 10952 | calculate_mcache_setting_params->mall_comb_mcache_c = &mode_lib->mp.mall_comb_mcache_c[k]; |
| 10935 | 10953 | calculate_mcache_setting_params->lc_comb_mcache = &mode_lib->mp.lc_comb_mcache[k]; |
| 10936 | − calculate_mcache_setting(&mode_lib->scratch, calculate_mcache_setting_params); | |
| 10954 | + if (!calculate_mcache_setting(&mode_lib->scratch, calculate_mcache_setting_params)) | |
| 10955 | + return false; | |
| 10937 | 10956 | } |
| 10938 | 10957 | |
| 10939 | 10958 | calculate_mall_bw_overhead_factor( |
| @@ -1337,8 +1337,30 @@ void dce110_blank_stream(struct pipe_ctx *pipe_ctx) · 1336 unmodified lines | ||
| 1337 | 1337 | |
| 1338 | 1338 | void dce110_set_avmute(struct pipe_ctx *pipe_ctx, bool enable) |
| 1339 | 1339 | { |
| 1340 | − if (pipe_ctx != NULL && pipe_ctx->stream_res.stream_enc != NULL) | |
| 1340 | + if (pipe_ctx == NULL || pipe_ctx->stream_res.stream_enc == NULL) | |
| 1341 | + return; | |
| 1342 | + | |
| 1343 | + if (dc_is_hdmi_signal(pipe_ctx->stream->signal)) { | |
| 1341 | 1344 | pipe_ctx->stream_res.stream_enc->funcs->set_avmute(pipe_ctx->stream_res.stream_enc, enable); |
| 1345 | + | |
| 1346 | + /* Wait for three frames to make sure AV mute is sent out. | |
| 1347 | + * Some HDMI sinks need additional GCP packets to properly | |
| 1348 | + * process the mute state, especially after link re-establishment | |
| 1349 | + * with HDMI 2.0 scrambling enabled. | |
| 1350 | + */ | |
| 1351 | + if (enable && pipe_ctx->stream_res.tg && | |
| 1352 | + pipe_ctx->stream_res.tg->funcs->is_tg_enabled && | |
| 1353 | + pipe_ctx->stream_res.tg->funcs->wait_for_state && | |
| 1354 | + pipe_ctx->stream_res.tg->funcs->is_tg_enabled(pipe_ctx->stream_res.tg)) { | |
| 1355 | + int i; | |
| 1356 | + | |
| 1357 | + pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VACTIVE); | |
| 1358 | + for (i = 0; i < 3; i++) { | |
| 1359 | + pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VBLANK); | |
| 1360 | + pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VACTIVE); | |
| 1361 | + } | |
| 1362 | + } | |
| 1363 | + } | |
| 1342 | 1364 | } |
| 1343 | 1365 | |
| 1344 | 1366 | enum audio_dto_source translate_to_dto_source(enum controller_id crtc_id) |
| @@ -847,13 +847,19 @@ void dcn30_set_avmute(struct pipe_ctx *pipe_ctx, bool enable) · 846 unmodified lines | ||
| 847 | 847 | pipe_ctx->stream_res.stream_enc, |
| 848 | 848 | enable); |
| 849 | 849 | |
| 850 | − /* Wait for two frame to make sure AV mute is sent out */ | |
| 850 | + /* Wait for three frames to make sure AV mute is sent out. | |
| 851 | + * Some HDMI sinks need additional GCP packets to properly | |
| 852 | + * process the mute state, especially after link re-establishment | |
| 853 | + * with HDMI 2.0 scrambling enabled. | |
| 854 | + */ | |
| 851 | 855 | if (enable && pipe_ctx->stream_res.tg->funcs->is_tg_enabled(pipe_ctx->stream_res.tg)) { |
| 856 | + int i; | |
| 857 | + | |
| 852 | 858 | pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VACTIVE); |
| 853 | − pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VBLANK); | |
| 854 | − pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VACTIVE); | |
| 855 | − pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VBLANK); | |
| 856 | − pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VACTIVE); | |
| 859 | + for (i = 0; i < 3; i++) { | |
| 860 | + pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VBLANK); | |
| 861 | + pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VACTIVE); | |
| 862 | + } | |
| 857 | 863 | } |
| 858 | 864 | } |
| 859 | 865 | } |
| @@ -384,7 +384,7 @@ static bool write_i2c_retimer_vga( · 383 unmodified lines | ||
| 384 | 384 | |
| 385 | 385 | for (size_t i = 0; i < ARRAY_SIZE(vga_data); i++) { |
| 386 | 386 | if (!write_i2c_retimer_offset_value(link, address, vga_data[i][0], vga_data[i][1])) { |
| 387 | − DC_LOG_ERROR("Set retimer failed, vga index: %zu\n", i); | |
| 387 | + DC_LOG_DEBUG("Set retimer failed, vga index: %zu\n", i); | |
| 388 | 388 | return false; |
| 389 | 389 | } |
| 390 | 390 | } |
| @@ -405,7 +405,7 @@ static bool write_i2c_retimer_byte( · 14 unmodified lines | ||
| 405 | 405 | return true; |
| 406 | 406 | |
| 407 | 407 | if (!write_i2c_retimer_offset_value(link, address, index, value)) { |
| 408 | − DC_LOG_ERROR("Set retimer failed, 3g index: 0x%x, value: 0x%x\n", index, value); | |
| 408 | + DC_LOG_DEBUG("Set retimer failed, 3g index: 0x%x, value: 0x%x\n", index, value); | |
| 409 | 409 | return false; |
| 410 | 410 | } |
| 411 | 411 | |
| @@ -421,14 +421,14 @@ static bool write_i2c_retimer_byte( · 9 unmodified lines | ||
| 421 | 421 | if (!link_query_ddc_data( |
| 422 | 422 | link->ddc, address, &offset, 1, &value, 1 |
| 423 | 423 | )) { |
| 424 | − DC_LOG_ERROR("Set retimer failed, link_query_ddc_data\n"); | |
| 424 | + DC_LOG_DEBUG("Set retimer failed, link_query_ddc_data\n"); | |
| 425 | 425 | return false; |
| 426 | 426 | } |
| 427 | 427 | } |
| 428 | 428 | |
| 429 | 429 | value |= apply_rx_tx_change; |
| 430 | 430 | if (!write_i2c_retimer_offset_value(link, address, offset, value)) { |
| 431 | − DC_LOG_ERROR("Set retimer failed, 3g offset: 0x%x, value: 0x%x\n", offset, value); | |
| 431 | + DC_LOG_DEBUG("Set retimer failed, 3g offset: 0x%x, value: 0x%x\n", offset, value); | |
| 432 | 432 | return false; |
| 433 | 433 | } |
| 434 | 434 | } |
| @@ -449,7 +449,7 @@ static bool write_i2c_retimer_setting( · 14 unmodified lines | ||
| 449 | 449 | uint8_t value = settings->reg_settings[i].i2c_reg_val; |
| 450 | 450 | |
| 451 | 451 | if (!write_i2c_retimer_byte(link, address, index, value)) { |
| 452 | − DC_LOG_ERROR("Set retimer failed, index: %zu\n", i); | |
| 452 | + DC_LOG_DEBUG("Set retimer failed, index: %zu\n", i); | |
| 453 | 453 | return false; |
| 454 | 454 | } |
| 455 | 455 | } |
| @@ -460,7 +460,7 @@ static bool write_i2c_retimer_setting( · 4 unmodified lines | ||
| 460 | 460 | uint8_t value = settings->reg_settings_6g[i].i2c_reg_val; |
| 461 | 461 | |
| 462 | 462 | if (!write_i2c_retimer_byte(link, address, index, value)) { |
| 463 | − DC_LOG_ERROR("Set retimer failed, 6g index: %zu\n", i); | |
| 463 | + DC_LOG_DEBUG("Set retimer failed, 6g index: %zu\n", i); | |
| 464 | 464 | return false; |
| 465 | 465 | } |
| 466 | 466 | } |
| @@ -492,7 +492,7 @@ static bool write_i2c_default_retimer_setting( · 25 unmodified lines | ||
| 492 | 492 | |
| 493 | 493 | for (size_t i = 0; i < ARRAY_SIZE(data); i++) { |
| 494 | 494 | if (!write_i2c_retimer_offset_value(link, address, data[i][0], data[i][1])) { |
| 495 | − DC_LOG_ERROR("Set default retimer failed, index: %zu\n", i); | |
| 495 | + DC_LOG_DEBUG("Set default retimer failed, index: %zu\n", i); | |
| 496 | 496 | return false; |
| 497 | 497 | } |
| 498 | 498 | } |
| @@ -524,7 +524,7 @@ static bool write_i2c_redriver_setting( · 25 unmodified lines | ||
| 524 | 524 | ); |
| 525 | 525 | |
| 526 | 526 | if (!success) |
| 527 | − DC_LOG_ERROR("Set redriver failed"); | |
| 527 | + DC_LOG_DEBUG("Set redriver failed"); | |
| 528 | 528 | return success; |
| 529 | 529 | } |
| 530 | 530 | |
| @@ -24,6 +24,8 @@ · 23 unmodified lines | ||
| 24 | 24 | #ifndef __KGD_PP_INTERFACE_H__ |
| 25 | 25 | #define __KGD_PP_INTERFACE_H__ |
| 26 | 26 | |
| 27 | +#include <linux/units.h> | |
| 28 | + | |
| 27 | 29 | extern const struct amdgpu_ip_block_version pp_smu_ip_block; |
| 28 | 30 | extern const struct amdgpu_ip_block_version smu_v11_0_ip_block; |
| 29 | 31 | extern const struct amdgpu_ip_block_version smu_v12_0_ip_block; |
| @@ -150,8 +152,8 @@ enum amd_pp_sensors { · 120 unmodified lines | ||
| 150 | 152 | AMDGPU_PP_SENSOR_MEM_TEMP, |
| 151 | 153 | AMDGPU_PP_SENSOR_VCE_POWER, |
| 152 | 154 | AMDGPU_PP_SENSOR_UVD_POWER, |
| 153 | − AMDGPU_PP_SENSOR_GPU_AVG_POWER, | |
| 154 | − AMDGPU_PP_SENSOR_GPU_INPUT_POWER, | |
| 155 | + AMDGPU_PP_SENSOR_GPU_AVG_POWER, /* milliwatts */ | |
| 156 | + AMDGPU_PP_SENSOR_GPU_INPUT_POWER, /* milliwatts */ | |
| 155 | 157 | AMDGPU_PP_SENSOR_SS_APU_SHARE, |
| 156 | 158 | AMDGPU_PP_SENSOR_SS_DGPU_SHARE, |
| 157 | 159 | AMDGPU_PP_SENSOR_STABLE_PSTATE_SCLK, |
| @@ -1183,20 +1183,36 @@ int amdgpu_dpm_dispatch_task(struct amdgpu_device *adev, · 1182 unmodified lines | ||
| 1183 | 1183 | return ret; |
| 1184 | 1184 | } |
| 1185 | 1185 | |
| 1186 | −int amdgpu_dpm_get_pp_table(struct amdgpu_device *adev, char **table) | |
| 1186 | +static bool amdgpu_dpm_is_pp_table_allowed(struct amdgpu_device *adev) | |
| 1187 | +{ | |
| 1188 | + return !amdgpu_sriov_vf(adev) && | |
| 1189 | + !(adev->flags & AMD_IS_APU) && | |
| 1190 | + !adev->scpm_enabled; | |
| 1191 | +} | |
| 1192 | + | |
| 1193 | +int amdgpu_dpm_get_pp_table(struct amdgpu_device *adev, char *table, | |
| 1194 | + size_t size) | |
| 1187 | 1195 | { |
| 1188 | 1196 | const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs; |
| 1197 | + char *pptable = NULL; | |
| 1189 | 1198 | int ret = 0; |
| 1190 | 1199 | |
| 1191 | − if (!table) | |
| 1200 | + if ((!table && size) || (table && !size)) | |
| 1192 | 1201 | return -EINVAL; |
| 1193 | 1202 | |
| 1194 | − if (amdgpu_sriov_vf(adev) || !pp_funcs->get_pp_table || adev->scpm_enabled) | |
| 1203 | + if (!amdgpu_dpm_is_pp_table_allowed(adev) || | |
| 1204 | + !pp_funcs->get_pp_table) | |
| 1195 | 1205 | return -EOPNOTSUPP; |
| 1196 | 1206 | |
| 1197 | 1207 | mutex_lock(&adev->pm.mutex); |
| 1198 | 1208 | ret = pp_funcs->get_pp_table(adev->powerplay.pp_handle, |
| 1199 | − table); | |
| 1209 | + &pptable); | |
| 1210 | + if (ret > 0 && !pptable) { | |
| 1211 | + ret = -EINVAL; | |
| 1212 | + } else if (ret > 0 && table) { | |
| 1213 | + ret = min_t(size_t, ret, size); | |
| 1214 | + memcpy(table, pptable, ret); | |
| 1215 | + } | |
| 1200 | 1216 | mutex_unlock(&adev->pm.mutex); |
| 1201 | 1217 | |
| 1202 | 1218 | return ret; |
| @@ -1426,17 +1442,23 @@ int amdgpu_dpm_set_power_profile_mode(struct amdgpu_device *adev, · 223 unmodified lines | ||
| 1426 | 1442 | return ret; |
| 1427 | 1443 | } |
| 1428 | 1444 | |
| 1429 | −int amdgpu_dpm_get_gpu_metrics(struct amdgpu_device *adev, void **table) | |
| 1445 | +ssize_t amdgpu_dpm_get_gpu_metrics(struct amdgpu_device *adev, void *buf, | |
| 1446 | + size_t size) | |
| 1430 | 1447 | { |
| 1431 | 1448 | const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs; |
| 1432 | − int ret = 0; | |
| 1449 | + void *table; | |
| 1450 | + ssize_t ret; | |
| 1433 | 1451 | |
| 1434 | 1452 | if (!pp_funcs->get_gpu_metrics) |
| 1435 | 1453 | return 0; |
| 1436 | 1454 | |
| 1437 | 1455 | mutex_lock(&adev->pm.mutex); |
| 1438 | 1456 | ret = pp_funcs->get_gpu_metrics(adev->powerplay.pp_handle, |
| 1439 | − table); | |
| 1457 | + &table); | |
| 1458 | + if (ret > 0) { | |
| 1459 | + ret = min_t(ssize_t, ret, size); | |
| 1460 | + memcpy(buf, table, ret); | |
| 1461 | + } | |
| 1440 | 1462 | mutex_unlock(&adev->pm.mutex); |
| 1441 | 1463 | |
| 1442 | 1464 | return ret; |
| @@ -1703,7 +1725,8 @@ int amdgpu_dpm_set_pp_table(struct amdgpu_device *adev, · 260 unmodified lines | ||
| 1703 | 1725 | if (!buf || !size) |
| 1704 | 1726 | return -EINVAL; |
| 1705 | 1727 | |
| 1706 | − if (amdgpu_sriov_vf(adev) || !pp_funcs->set_pp_table || adev->scpm_enabled) | |
| 1728 | + if (!amdgpu_dpm_is_pp_table_allowed(adev) || | |
| 1729 | + !pp_funcs->set_pp_table) | |
| 1707 | 1730 | return -EOPNOTSUPP; |
| 1708 | 1731 | |
| 1709 | 1732 | mutex_lock(&adev->pm.mutex); |
| @@ -41,8 +41,6 @@ · 40 unmodified lines | ||
| 41 | 41 | |
| 42 | 42 | #define DEVICE_ATTR_IS(_name) (attr_id == device_attr_id__##_name) |
| 43 | 43 | |
| 44 | −#define power_2_mwatt(power) (((power) >> 8) * 1000 + ((power) & 0xff)) | |
| 45 | − | |
| 46 | 44 | struct od_attribute { |
| 47 | 45 | struct kobj_attribute attribute; |
| 48 | 46 | struct list_head entry; |
| @@ -564,25 +562,19 @@ static ssize_t amdgpu_get_pp_table(struct device *dev, · 515 unmodified lines | ||
| 564 | 562 | { |
| 565 | 563 | struct drm_device *ddev = dev_get_drvdata(dev); |
| 566 | 564 | struct amdgpu_device *adev = drm_to_adev(ddev); |
| 567 | − char *table = NULL; | |
| 568 | 565 | int size, ret; |
| 569 | 566 | |
| 570 | 567 | ret = amdgpu_pm_get_access_if_active(adev); |
| 571 | 568 | if (ret) |
| 572 | 569 | return ret; |
| 573 | 570 | |
| 574 | − size = amdgpu_dpm_get_pp_table(adev, &table); | |
| 571 | + size = amdgpu_dpm_get_pp_table(adev, buf, PAGE_SIZE - 1); | |
| 575 | 572 | |
| 576 | 573 | amdgpu_pm_put_access(adev); |
| 577 | 574 | |
| 578 | 575 | if (size <= 0) |
| 579 | 576 | return size; |
| 580 | 577 | |
| 581 | − if (size >= PAGE_SIZE) | |
| 582 | − size = PAGE_SIZE - 1; | |
| 583 | − | |
| 584 | − memcpy(buf, table, size); | |
| 585 | − | |
| 586 | 578 | return size; |
| 587 | 579 | } |
| 588 | 580 | |
| @@ -1780,7 +1772,6 @@ static ssize_t amdgpu_get_gpu_metrics(struct device *dev, · 1191 unmodified lines | ||
| 1780 | 1772 | { |
| 1781 | 1773 | struct drm_device *ddev = dev_get_drvdata(dev); |
| 1782 | 1774 | struct amdgpu_device *adev = drm_to_adev(ddev); |
| 1783 | − void *gpu_metrics; | |
| 1784 | 1775 | ssize_t size = 0; |
| 1785 | 1776 | int ret; |
| 1786 | 1777 | |
| @@ -1788,15 +1779,10 @@ static ssize_t amdgpu_get_gpu_metrics(struct device *dev, · 1 unmodified line | ||
| 1788 | 1779 | if (ret) |
| 1789 | 1780 | return ret; |
| 1790 | 1781 | |
| 1791 | − size = amdgpu_dpm_get_gpu_metrics(adev, &gpu_metrics); | |
| 1782 | + size = amdgpu_dpm_get_gpu_metrics(adev, buf, PAGE_SIZE - 1); | |
| 1792 | 1783 | if (size <= 0) |
| 1793 | 1784 | goto out; |
| 1794 | 1785 | |
| 1795 | − if (size >= PAGE_SIZE) | |
| 1796 | − size = PAGE_SIZE - 1; | |
| 1797 | − | |
| 1798 | − memcpy(buf, gpu_metrics, size); | |
| 1799 | − | |
| 1800 | 1786 | out: |
| 1801 | 1787 | amdgpu_pm_put_access(adev); |
| 1802 | 1788 | |
| @@ -2726,10 +2712,9 @@ static int default_attr_update(struct amdgpu_device *adev, struct amdgpu_device_ · 923 unmodified lines | ||
| 2726 | 2712 | *states = ATTR_STATE_UNSUPPORTED; |
| 2727 | 2713 | } else if (DEVICE_ATTR_IS(pp_table)) { |
| 2728 | 2714 | int ret; |
| 2729 | − char *tmp = NULL; | |
| 2730 | 2715 | |
| 2731 | − ret = amdgpu_dpm_get_pp_table(adev, &tmp); | |
| 2732 | − if (ret == -EOPNOTSUPP || !tmp) | |
| 2716 | + ret = amdgpu_dpm_get_pp_table(adev, NULL, 0); | |
| 2717 | + if (ret <= 0) | |
| 2733 | 2718 | *states = ATTR_STATE_UNSUPPORTED; |
| 2734 | 2719 | else |
| 2735 | 2720 | *states = ATTR_STATE_SUPPORTED; |
| @@ -3364,7 +3349,7 @@ static int amdgpu_hwmon_get_power(struct device *dev, · 628 unmodified lines | ||
| 3364 | 3349 | return r; |
| 3365 | 3350 | |
| 3366 | 3351 | /* convert to microwatts */ |
| 3367 | − return power_2_mwatt(query) * 1000; | |
| 3352 | + return query * 1000; | |
| 3368 | 3353 | } |
| 3369 | 3354 | |
| 3370 | 3355 | static ssize_t amdgpu_hwmon_show_power_avg(struct device *dev, |
| @@ -4932,7 +4917,7 @@ static int amdgpu_debugfs_pm_info_pp(struct seq_file *m, struct amdgpu_device *a · 1561 unmodified lines | ||
| 4932 | 4917 | seq_printf(m, "\t%u mV (VDDNB)\n", value); |
| 4933 | 4918 | size = sizeof(uint32_t); |
| 4934 | 4919 | if (!amdgpu_dpm_read_sensor(adev, AMDGPU_PP_SENSOR_GPU_AVG_POWER, (void *)&query, &size)) { |
| 4935 | − mwatt = power_2_mwatt(query); | |
| 4920 | + mwatt = query; | |
| 4936 | 4921 | centiwatt = DIV_ROUND_CLOSEST(mwatt, 10); |
| 4937 | 4922 | if (adev->flags & AMD_IS_APU) |
| 4938 | 4923 | seq_printf(m, "\t%u.%02u W (average SoC including CPU)\n", centiwatt / 100, centiwatt % 100); |
| @@ -4941,7 +4926,7 @@ static int amdgpu_debugfs_pm_info_pp(struct seq_file *m, struct amdgpu_device *a · 2 unmodified lines | ||
| 4941 | 4926 | } |
| 4942 | 4927 | size = sizeof(uint32_t); |
| 4943 | 4928 | if (!amdgpu_dpm_read_sensor(adev, AMDGPU_PP_SENSOR_GPU_INPUT_POWER, (void *)&query, &size)) { |
| 4944 | − mwatt = power_2_mwatt(query); | |
| 4929 | + mwatt = query; | |
| 4945 | 4930 | centiwatt = DIV_ROUND_CLOSEST(mwatt, 10); |
| 4946 | 4931 | if (adev->flags & AMD_IS_APU) |
| 4947 | 4932 | seq_printf(m, "\t%u.%02u W (current SoC including CPU)\n", centiwatt / 100, centiwatt % 100); |
| @@ -487,7 +487,8 @@ int amdgpu_dpm_get_pp_num_states(struct amdgpu_device *adev, · 486 unmodified lines | ||
| 487 | 487 | int amdgpu_dpm_dispatch_task(struct amdgpu_device *adev, |
| 488 | 488 | enum amd_pp_task task_id, |
| 489 | 489 | enum amd_pm_state_type *user_state); |
| 490 | −int amdgpu_dpm_get_pp_table(struct amdgpu_device *adev, char **table); | |
| 490 | +int amdgpu_dpm_get_pp_table(struct amdgpu_device *adev, char *table, | |
| 491 | + size_t size); | |
| 491 | 492 | int amdgpu_dpm_set_fine_grain_clk_vol(struct amdgpu_device *adev, |
| 492 | 493 | uint32_t type, |
| 493 | 494 | long *input, |
| @@ -517,7 +518,8 @@ int amdgpu_dpm_get_power_profile_mode(struct amdgpu_device *adev, · 23 unmodified lines | ||
| 517 | 518 | char *buf); |
| 518 | 519 | int amdgpu_dpm_set_power_profile_mode(struct amdgpu_device *adev, |
| 519 | 520 | long *input, uint32_t size); |
| 520 | −int amdgpu_dpm_get_gpu_metrics(struct amdgpu_device *adev, void **table); | |
| 521 | +ssize_t amdgpu_dpm_get_gpu_metrics(struct amdgpu_device *adev, void *buf, | |
| 522 | + size_t size); | |
| 521 | 523 | ssize_t amdgpu_dpm_get_xcp_metrics(struct amdgpu_device *adev, int xcp_id, |
| 522 | 524 | void *table); |
| 523 | 525 | ssize_t amdgpu_dpm_get_temp_metrics(struct amdgpu_device *adev, |
| @@ -4062,7 +4062,7 @@ static int smu7_get_gpu_power(struct pp_hwmgr *hwmgr, u32 *query) · 4061 unmodified lines | ||
| 4062 | 4062 | (adev->asic_type != CHIP_FIJI) && |
| 4063 | 4063 | (adev->asic_type != CHIP_TONGA)) { |
| 4064 | 4064 | smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_GetCurrPkgPwr, 0, &tmp); |
| 4065 | − *query = tmp; | |
| 4065 | + *query = PP_PWR_Q24_8_TO_MW(tmp); | |
| 4066 | 4066 | |
| 4067 | 4067 | if (tmp != 0) |
| 4068 | 4068 | return 0; |
| @@ -4081,7 +4081,7 @@ static int smu7_get_gpu_power(struct pp_hwmgr *hwmgr, u32 *query) · 12 unmodified lines | ||
| 4081 | 4081 | if (tmp != 0) |
| 4082 | 4082 | break; |
| 4083 | 4083 | } |
| 4084 | − *query = tmp; | |
| 4084 | + *query = PP_PWR_Q24_8_TO_MW(tmp); | |
| 4085 | 4085 | |
| 4086 | 4086 | return 0; |
| 4087 | 4087 | } |
| @@ -3934,8 +3934,8 @@ static int vega10_get_gpu_power(struct pp_hwmgr *hwmgr, · 3933 unmodified lines | ||
| 3934 | 3934 | if (ret) |
| 3935 | 3935 | return ret; |
| 3936 | 3936 | |
| 3937 | − /* SMC returning actual watts, keep consistent with legacy asics, low 8 bit as 8 fractional bits */ | |
| 3938 | − *query = value << 8; | |
| 3937 | + /* SMC returns whole Watts, while power sensors use milliwatts. */ | |
| 3938 | + *query = value * MILLIWATT_PER_WATT; | |
| 3939 | 3939 | |
| 3940 | 3940 | return 0; |
| 3941 | 3941 | } |
| @@ -1419,7 +1419,7 @@ static int vega12_get_gpu_power(struct pp_hwmgr *hwmgr, uint32_t *query) · 1418 unmodified lines | ||
| 1419 | 1419 | if (ret) |
| 1420 | 1420 | return ret; |
| 1421 | 1421 | |
| 1422 | − *query = metrics_table.CurrSocketPower << 8; | |
| 1422 | + *query = metrics_table.CurrSocketPower * MILLIWATT_PER_WATT; | |
| 1423 | 1423 | |
| 1424 | 1424 | return ret; |
| 1425 | 1425 | } |
| @@ -2154,12 +2154,13 @@ static int vega20_get_gpu_power(struct pp_hwmgr *hwmgr, int idx, · 2153 unmodified lines | ||
| 2154 | 2154 | switch (idx) { |
| 2155 | 2155 | case AMDGPU_PP_SENSOR_GPU_AVG_POWER: |
| 2156 | 2156 | if (hwmgr->smu_version == 0x282e00) |
| 2157 | − *query = metrics_table.AverageSocketPower << 8; | |
| 2157 | + *query = metrics_table.AverageSocketPower * | |
| 2158 | + MILLIWATT_PER_WATT; | |
| 2158 | 2159 | else |
| 2159 | 2160 | ret = -EOPNOTSUPP; |
| 2160 | 2161 | break; |
| 2161 | 2162 | case AMDGPU_PP_SENSOR_GPU_INPUT_POWER: |
| 2162 | − *query = metrics_table.CurrSocketPower << 8; | |
| 2163 | + *query = metrics_table.CurrSocketPower * MILLIWATT_PER_WATT; | |
| 2163 | 2164 | break; |
| 2164 | 2165 | } |
| 2165 | 2166 | |
| @@ -35,6 +35,11 @@ struct pp_hwmgr; · 34 unmodified lines | ||
| 35 | 35 | struct phm_fan_speed_info; |
| 36 | 36 | struct pp_atomctrl_voltage_table; |
| 37 | 37 | |
| 38 | +/* Decode legacy unsigned Q24.8 watts to internal milliwatts. */ | |
| 39 | +#define PP_PWR_Q24_8_TO_MW(power) \ | |
| 40 | + DIV_ROUND_CLOSEST_ULL((u64)(power) * MILLIWATT_PER_WATT, \ | |
| 41 | + BIT(8)) | |
| 42 | + | |
| 38 | 43 | #define VOLTAGE_SCALE 4 |
| 39 | 44 | #define VOLTAGE_VID_OFFSET_SCALE1 625 |
| 40 | 45 | #define VOLTAGE_VID_OFFSET_SCALE2 100 |
| @@ -1348,7 +1348,7 @@ struct pptable_funcs { · 1347 unmodified lines | ||
| 1348 | 1348 | u32 (*set_gfx_off_residency)(struct smu_context *smu, bool start); |
| 1349 | 1349 | |
| 1350 | 1350 | /** |
| 1351 | − * @get_gfx_off_residency: Average GFXOFF residency % during the logging interval | |
| 1351 | + * @get_gfx_off_residency: Live GFXOFF residency percentage | |
| 1352 | 1352 | */ |
| 1353 | 1353 | u32 (*get_gfx_off_residency)(struct smu_context *smu, uint32_t *residency); |
| 1354 | 1354 | |
| @@ -144,6 +144,44 @@ typedef struct { · 143 unmodified lines | ||
| 144 | 144 | uint32_t MaxGfxClk; |
| 145 | 145 | } DpmClocks_t; |
| 146 | 146 | |
| 147 | +//Freq in MHz | |
| 148 | +//Voltage in milli volts with 2 fractional bits | |
| 149 | +typedef struct { | |
| 150 | + uint32_t DcfClocks[NUM_DCFCLK_DPM_LEVELS]; | |
| 151 | + uint32_t DispClocks[NUM_DISPCLK_DPM_LEVELS]; | |
| 152 | + uint32_t DppClocks[NUM_DPPCLK_DPM_LEVELS]; | |
| 153 | + uint32_t SocClocks[NUM_SOCCLK_DPM_LEVELS]; | |
| 154 | + | |
| 155 | + uint32_t VPEClocks[NUM_VPE_DPM_LEVELS]; | |
| 156 | + uint32_t FclkClocks_Freq[NUM_FCLK_DPM_LEVELS]; | |
| 157 | + uint32_t FclkClocks_Voltage[NUM_FCLK_DPM_LEVELS]; | |
| 158 | + uint32_t SocVoltage[NUM_SOC_VOLTAGE_LEVELS]; | |
| 159 | + MemPstateTable_t MemPstateTable[NUM_MEM_PSTATE_LEVELS]; | |
| 160 | + | |
| 161 | + uint8_t NumDcfClkLevelsEnabled; | |
| 162 | + uint8_t NumDispClkLevelsEnabled; //Applies to both Dispclk and Dppclk | |
| 163 | + uint8_t NumSocClkLevelsEnabled; | |
| 164 | + | |
| 165 | + uint8_t VpeClkLevelsEnabled; | |
| 166 | + | |
| 167 | + uint8_t NumMemPstatesEnabled; | |
| 168 | + uint8_t NumFclkLevelsEnabled; | |
| 169 | + uint8_t Spare1; | |
| 170 | + uint8_t Spare2; | |
| 171 | + | |
| 172 | + uint8_t Spare3; | |
| 173 | + uint8_t Spare4; | |
| 174 | + uint8_t Spare5[2]; | |
| 175 | + | |
| 176 | + uint32_t MinGfxClk; | |
| 177 | + uint32_t MaxGfxClk; | |
| 178 | + | |
| 179 | + uint32_t Spare6[8]; | |
| 180 | + uint32_t Spare7[8]; | |
| 181 | + uint32_t Spare8[8]; | |
| 182 | + uint32_t Spare9[8]; | |
| 183 | +} DpmClocks_t_v15_0_5; | |
| 184 | + | |
| 147 | 185 | typedef struct { |
| 148 | 186 | uint16_t CoreFrequency[16]; //Target core frequency [MHz] |
| 149 | 187 | uint16_t CorePower[16]; //CAC calculated core power [mW] |
| @@ -110,8 +110,10 @@ · 109 unmodified lines | ||
| 110 | 110 | #define PPSMC_MSG_GetSlowPPTLimit 0x4C |
| 111 | 111 | #define PPSMC_MSG_GetGfxOffStatus 0x50 |
| 112 | 112 | #define PPSMC_MSG_GetGfxOffEntryCount 0x51 |
| 113 | −#define PPSMC_MSG_LogGfxOffResidency 0x52 | |
| 114 | −#define PPSMC_Message_Count 0x53 | |
| 113 | +#define PPSMC_MSG_GfxOffResidencyLogReadSample 0x52 | |
| 114 | +#define PPSMC_MSG_StopGfxOffResidencyLogging 0x53 | |
| 115 | +#define PPSMC_MSG_StartGfxOffResidencyLogging 0x56 | |
| 116 | +#define PPSMC_Message_Count 0x57 | |
| 115 | 117 | |
| 116 | 118 | //Argument for PPSMC_MSG_GfxDeviceDriverReset |
| 117 | 119 | enum { |
| @@ -252,7 +252,9 @@ · 251 unmodified lines | ||
| 252 | 252 | __SMU_DUMMY_MAP(DriverMode2Reset), \ |
| 253 | 253 | __SMU_DUMMY_MAP(GetGfxOffStatus), \ |
| 254 | 254 | __SMU_DUMMY_MAP(GetGfxOffEntryCount), \ |
| 255 | − __SMU_DUMMY_MAP(LogGfxOffResidency), \ | |
| 255 | + __SMU_DUMMY_MAP(StartGfxOffResidencyLogging), \ | |
| 256 | + __SMU_DUMMY_MAP(GfxOffResidencyLogReadSample), \ | |
| 257 | + __SMU_DUMMY_MAP(StopGfxOffResidencyLogging), \ | |
| 256 | 258 | __SMU_DUMMY_MAP(SetNumBadMemoryPagesRetired), \ |
| 257 | 259 | __SMU_DUMMY_MAP(SetBadMemoryPagesRetiredFlagsPerChannel), \ |
| 258 | 260 | __SMU_DUMMY_MAP(AllowGpo), \ |
| @@ -26,7 +26,7 @@ · 25 unmodified lines | ||
| 26 | 26 | #include "amdgpu_smu.h" |
| 27 | 27 | |
| 28 | 28 | #define SMU15_DRIVER_IF_VERSION_INV 0xFFFFFFFF |
| 29 | −#define SMU15_DRIVER_IF_VERSION_SMU_V15_0 0x7 | |
| 29 | +#define SMU15_DRIVER_IF_VERSION_SMU_V15_0 0x9 | |
| 30 | 30 | |
| 31 | 31 | |
| 32 | 32 | #define FEATURE_MASK(feature) (1ULL << feature) |
| @@ -658,7 +658,7 @@ static int arcturus_get_smu_metrics_data(struct smu_context *smu, · 657 unmodified lines | ||
| 658 | 658 | *value = metrics->VcnActivityPercentage; |
| 659 | 659 | break; |
| 660 | 660 | case METRICS_AVERAGE_SOCKETPOWER: |
| 661 | − *value = metrics->AverageSocketPower << 8; | |
| 661 | + *value = metrics->AverageSocketPower * MILLIWATT_PER_WATT; | |
| 662 | 662 | break; |
| 663 | 663 | case METRICS_TEMPERATURE_EDGE: |
| 664 | 664 | *value = metrics->TemperatureEdge * |
| @@ -158,12 +158,10 @@ cyan_skillfish_get_smu_metrics_data(struct smu_context *smu, · 157 unmodified lines | ||
| 158 | 158 | *value = metrics->Current.MemclkFrequency; |
| 159 | 159 | break; |
| 160 | 160 | case METRICS_CURR_SOCKETPOWER: |
| 161 | − *value = (metrics->Current.CurrentSocketPower << 8) / | |
| 162 | − 1000; | |
| 161 | + *value = metrics->Current.CurrentSocketPower; | |
| 163 | 162 | break; |
| 164 | 163 | case METRICS_AVERAGE_SOCKETPOWER: |
| 165 | − *value = (metrics->Average.CurrentSocketPower << 8) / | |
| 166 | − 1000; | |
| 164 | + *value = metrics->Average.CurrentSocketPower; | |
| 167 | 165 | break; |
| 168 | 166 | case METRICS_TEMPERATURE_EDGE: |
| 169 | 167 | *value = metrics->Current.GfxTemperature / 100 * |
| @@ -602,7 +602,7 @@ static int navi10_get_legacy_smu_metrics_data(struct smu_context *smu, · 601 unmodified lines | ||
| 602 | 602 | *value = metrics->AverageUclkActivity; |
| 603 | 603 | break; |
| 604 | 604 | case METRICS_AVERAGE_SOCKETPOWER: |
| 605 | − *value = metrics->AverageSocketPower << 8; | |
| 605 | + *value = metrics->AverageSocketPower * MILLIWATT_PER_WATT; | |
| 606 | 606 | break; |
| 607 | 607 | case METRICS_TEMPERATURE_EDGE: |
| 608 | 608 | *value = metrics->TemperatureEdge * |
| @@ -691,7 +691,7 @@ static int navi10_get_smu_metrics_data(struct smu_context *smu, · 82 unmodified lines | ||
| 691 | 691 | *value = metrics->AverageUclkActivity; |
| 692 | 692 | break; |
| 693 | 693 | case METRICS_AVERAGE_SOCKETPOWER: |
| 694 | − *value = metrics->AverageSocketPower << 8; | |
| 694 | + *value = metrics->AverageSocketPower * MILLIWATT_PER_WATT; | |
| 695 | 695 | break; |
| 696 | 696 | case METRICS_TEMPERATURE_EDGE: |
| 697 | 697 | *value = metrics->TemperatureEdge * |
| @@ -777,7 +777,7 @@ static int navi12_get_legacy_smu_metrics_data(struct smu_context *smu, · 79 unmodified lines | ||
| 777 | 777 | *value = metrics->AverageUclkActivity; |
| 778 | 778 | break; |
| 779 | 779 | case METRICS_AVERAGE_SOCKETPOWER: |
| 780 | − *value = metrics->AverageSocketPower << 8; | |
| 780 | + *value = metrics->AverageSocketPower * MILLIWATT_PER_WATT; | |
| 781 | 781 | break; |
| 782 | 782 | case METRICS_TEMPERATURE_EDGE: |
| 783 | 783 | *value = metrics->TemperatureEdge * |
| @@ -866,7 +866,7 @@ static int navi12_get_smu_metrics_data(struct smu_context *smu, · 82 unmodified lines | ||
| 866 | 866 | *value = metrics->AverageUclkActivity; |
| 867 | 867 | break; |
| 868 | 868 | case METRICS_AVERAGE_SOCKETPOWER: |
| 869 | − *value = metrics->AverageSocketPower << 8; | |
| 869 | + *value = metrics->AverageSocketPower * MILLIWATT_PER_WATT; | |
| 870 | 870 | break; |
| 871 | 871 | case METRICS_TEMPERATURE_EDGE: |
| 872 | 872 | *value = metrics->TemperatureEdge * |
| @@ -855,9 +855,10 @@ static int sienna_cichlid_get_smu_metrics_data(struct smu_context *smu, · 854 unmodified lines | ||
| 855 | 855 | metrics->AverageUclkActivity; |
| 856 | 856 | break; |
| 857 | 857 | case METRICS_AVERAGE_SOCKETPOWER: |
| 858 | − *value = use_metrics_v3 ? metrics_v3->AverageSocketPower << 8 : | |
| 859 | − use_metrics_v2 ? metrics_v2->AverageSocketPower << 8 : | |
| 860 | − metrics->AverageSocketPower << 8; | |
| 858 | + *value = use_metrics_v3 ? metrics_v3->AverageSocketPower : | |
| 859 | + use_metrics_v2 ? metrics_v2->AverageSocketPower : | |
| 860 | + metrics->AverageSocketPower; | |
| 861 | + *value *= MILLIWATT_PER_WATT; | |
| 861 | 862 | break; |
| 862 | 863 | case METRICS_TEMPERATURE_EDGE: |
| 863 | 864 | *value = (use_metrics_v3 ? metrics_v3->TemperatureEdge : |
| @@ -72,6 +72,12 @@ static const struct smu_feature_bits vangogh_dpm_features = { · 71 unmodified lines | ||
| 72 | 72 | } |
| 73 | 73 | }; |
| 74 | 74 | |
| 75 | +/* | |
| 76 | + * SMU support new GFXOFF residency log interface since version 4.63.62.00, | |
| 77 | + * use this to get live readings of GFXOFF residency | |
| 78 | + */ | |
| 79 | +#define SUPPORT_LIVE_RESIDENCY_MSG_VERSION 0x043f3e00 | |
| 80 | + | |
| 75 | 81 | static struct cmn2asic_msg_mapping vangogh_message_map[SMU_MSG_MAX_COUNT] = { |
| 76 | 82 | MSG_MAP(TestMessage, PPSMC_MSG_TestMessage, 0), |
| 77 | 83 | MSG_MAP(GetSmuVersion, PPSMC_MSG_GetSmuVersion, 0), |
| @@ -142,7 +148,9 @@ static struct cmn2asic_msg_mapping vangogh_message_map[SMU_MSG_MAX_COUNT] = { · 64 unmodified lines | ||
| 142 | 148 | MSG_MAP(GetSlowPPTLimit, PPSMC_MSG_GetSlowPPTLimit, 0), |
| 143 | 149 | MSG_MAP(GetGfxOffStatus, PPSMC_MSG_GetGfxOffStatus, 0), |
| 144 | 150 | MSG_MAP(GetGfxOffEntryCount, PPSMC_MSG_GetGfxOffEntryCount, 0), |
| 145 | − MSG_MAP(LogGfxOffResidency, PPSMC_MSG_LogGfxOffResidency, 0), | |
| 151 | + MSG_MAP(StartGfxOffResidencyLogging, PPSMC_MSG_StartGfxOffResidencyLogging, 0), | |
| 152 | + MSG_MAP(GfxOffResidencyLogReadSample, PPSMC_MSG_GfxOffResidencyLogReadSample, 0), | |
| 153 | + MSG_MAP(StopGfxOffResidencyLogging, PPSMC_MSG_StopGfxOffResidencyLogging, 0), | |
| 146 | 154 | }; |
| 147 | 155 | |
| 148 | 156 | static struct cmn2asic_mapping vangogh_feature_mask_map[SMU_FEATURE_COUNT] = { |
| @@ -310,8 +318,7 @@ static int vangogh_get_legacy_smu_metrics_data(struct smu_context *smu, · 161 unmodified lines | ||
| 310 | 318 | *value = metrics->UvdActivity / 100; |
| 311 | 319 | break; |
| 312 | 320 | case METRICS_AVERAGE_SOCKETPOWER: |
| 313 | − *value = (metrics->CurrentSocketPower << 8) / | |
| 314 | − 1000 ; | |
| 321 | + *value = metrics->CurrentSocketPower; | |
| 315 | 322 | break; |
| 316 | 323 | case METRICS_TEMPERATURE_EDGE: |
| 317 | 324 | *value = metrics->GfxTemperature / 100 * |
| @@ -379,12 +386,10 @@ static int vangogh_get_smu_metrics_data(struct smu_context *smu, · 61 unmodified lines | ||
| 379 | 386 | *value = metrics->Current.UvdActivity; |
| 380 | 387 | break; |
| 381 | 388 | case METRICS_AVERAGE_SOCKETPOWER: |
| 382 | − *value = (metrics->Average.CurrentSocketPower << 8) / | |
| 383 | − 1000; | |
| 389 | + *value = metrics->Average.CurrentSocketPower; | |
| 384 | 390 | break; |
| 385 | 391 | case METRICS_CURR_SOCKETPOWER: |
| 386 | − *value = (metrics->Current.CurrentSocketPower << 8) / | |
| 387 | − 1000; | |
| 392 | + *value = metrics->Current.CurrentSocketPower; | |
| 388 | 393 | break; |
| 389 | 394 | case METRICS_TEMPERATURE_EDGE: |
| 390 | 395 | *value = metrics->Current.GfxTemperature / 100 * |
| @@ -2449,19 +2454,32 @@ static int vangogh_set_power_limit(struct smu_context *smu, · 2058 unmodified lines | ||
| 2449 | 2454 | static u32 vangogh_set_gfxoff_residency(struct smu_context *smu, bool start) |
| 2450 | 2455 | { |
| 2451 | 2456 | int ret = 0; |
| 2452 | − u32 residency; | |
| 2453 | 2457 | struct amdgpu_device *adev = smu->adev; |
| 2454 | 2458 | |
| 2455 | 2459 | if (!(adev->pm.pp_feature & PP_GFXOFF_MASK)) |
| 2456 | 2460 | return 0; |
| 2457 | 2461 | |
| 2458 | − ret = smu_cmn_send_smc_msg_with_param(smu, SMU_MSG_LogGfxOffResidency, | |
| 2459 | − start, &residency); | |
| 2460 | − if (ret) | |
| 2461 | − return ret; | |
| 2462 | + if (smu->smc_fw_version < SUPPORT_LIVE_RESIDENCY_MSG_VERSION) { | |
| 2463 | + u32 residency; | |
| 2464 | + | |
| 2465 | + ret = smu_cmn_send_smc_msg_with_param(smu, SMU_MSG_GfxOffResidencyLogReadSample, | |
| 2466 | + start, &residency); | |
| 2467 | + if (ret) | |
| 2468 | + return ret; | |
| 2462 | 2469 | |
| 2463 | − if (!start) | |
| 2464 | − adev->gfx.gfx_off_residency = residency; | |
| 2470 | + if (!start) | |
| 2471 | + adev->gfx.gfx_off_residency = residency; | |
| 2472 | + } else { | |
| 2473 | + if (start) { | |
| 2474 | + ret = smu_cmn_send_smc_msg(smu, SMU_MSG_StartGfxOffResidencyLogging, NULL); | |
| 2475 | + if (ret) | |
| 2476 | + return ret; | |
| 2477 | + } else { | |
| 2478 | + ret = smu_cmn_send_smc_msg(smu, SMU_MSG_StopGfxOffResidencyLogging, NULL); | |
| 2479 | + if (ret) | |
| 2480 | + return ret; | |
| 2481 | + } | |
| 2482 | + } | |
| 2465 | 2483 | |
| 2466 | 2484 | return ret; |
| 2467 | 2485 | } |
| @@ -2478,11 +2496,20 @@ static u32 vangogh_set_gfxoff_residency(struct smu_context *smu, bool start) · 10 unmodified lines | ||
| 2478 | 2496 | */ |
| 2479 | 2497 | static u32 vangogh_get_gfxoff_residency(struct smu_context *smu, uint32_t *residency) |
| 2480 | 2498 | { |
| 2499 | + int ret = 0; | |
| 2481 | 2500 | struct amdgpu_device *adev = smu->adev; |
| 2482 | 2501 | |
| 2483 | − *residency = adev->gfx.gfx_off_residency; | |
| 2502 | + if (!(adev->pm.pp_feature & PP_GFXOFF_MASK)) | |
| 2503 | + return 0; | |
| 2504 | + | |
| 2505 | + if (smu->smc_fw_version < SUPPORT_LIVE_RESIDENCY_MSG_VERSION) { | |
| 2506 | + *residency = adev->gfx.gfx_off_residency; | |
| 2507 | + } else { | |
| 2508 | + ret = smu_cmn_send_smc_msg(smu, SMU_MSG_GfxOffResidencyLogReadSample, | |
| 2509 | + residency); | |
| 2510 | + } | |
| 2484 | 2511 | |
| 2485 | − return 0; | |
| 2512 | + return ret; | |
| 2486 | 2513 | } |
| 2487 | 2514 | |
| 2488 | 2515 | /** |
| @@ -1215,9 +1215,10 @@ static int renoir_get_smu_metrics_data(struct smu_context *smu, · 1214 unmodified lines | ||
| 1215 | 1215 | ((amdgpu_ip_version(adev, MP1_HWIP, 0) == |
| 1216 | 1216 | IP_VERSION(12, 0, 0)) && |
| 1217 | 1217 | (adev->pm.fw_version >= 0x373200))) |
| 1218 | − *value = metrics->CurrentSocketPower << 8; | |
| 1218 | + *value = metrics->CurrentSocketPower * | |
| 1219 | + MILLIWATT_PER_WATT; | |
| 1219 | 1220 | else |
| 1220 | − *value = (metrics->CurrentSocketPower << 8) / 1000; | |
| 1221 | + *value = metrics->CurrentSocketPower; | |
| 1221 | 1222 | break; |
| 1222 | 1223 | case METRICS_TEMPERATURE_EDGE: |
| 1223 | 1224 | *value = (metrics->GfxTemperature / 100) * |
| @@ -677,7 +677,8 @@ static int aldebaran_get_smu_metrics_data(struct smu_context *smu, · 676 unmodified lines | ||
| 677 | 677 | case METRICS_AVERAGE_SOCKETPOWER: |
| 678 | 678 | /* Valid power data is available only from primary die */ |
| 679 | 679 | if (aldebaran_is_primary(smu)) |
| 680 | − *value = metrics->AverageSocketPower << 8; | |
| 680 | + *value = metrics->AverageSocketPower * | |
| 681 | + MILLIWATT_PER_WATT; | |
| 681 | 682 | else |
| 682 | 683 | ret = -EOPNOTSUPP; |
| 683 | 684 | break; |
| @@ -807,7 +807,7 @@ static int smu_v13_0_0_get_smu_metrics_data(struct smu_context *smu, · 806 unmodified lines | ||
| 807 | 807 | metrics->Vcn1ActivityPercentage); |
| 808 | 808 | break; |
| 809 | 809 | case METRICS_AVERAGE_SOCKETPOWER: |
| 810 | − *value = metrics->AverageSocketPower << 8; | |
| 810 | + *value = metrics->AverageSocketPower * MILLIWATT_PER_WATT; | |
| 811 | 811 | break; |
| 812 | 812 | case METRICS_TEMPERATURE_EDGE: |
| 813 | 813 | *value = metrics->AvgTemperature[TEMP_EDGE] * |
| @@ -434,7 +434,8 @@ int smu_v13_0_12_get_smu_metrics_data(struct smu_context *smu, · 433 unmodified lines | ||
| 434 | 434 | *value = SMUQ10_ROUND(metrics->DramBandwidthUtilization); |
| 435 | 435 | break; |
| 436 | 436 | case METRICS_CURR_SOCKETPOWER: |
| 437 | − *value = SMUQ10_ROUND(metrics->SocketPower) << 8; | |
| 437 | + *value = SMUQ10_ROUND(metrics->SocketPower) * | |
| 438 | + MILLIWATT_PER_WATT; | |
| 438 | 439 | break; |
| 439 | 440 | case METRICS_TEMPERATURE_HOTSPOT: |
| 440 | 441 | *value = SMUQ10_ROUND(metrics->MaxSocketTemperature) * |
| @@ -338,10 +338,10 @@ static int smu_v13_0_4_get_smu_metrics_data(struct smu_context *smu, · 337 unmodified lines | ||
| 338 | 338 | *value = metrics->UvdActivity / 100; |
| 339 | 339 | break; |
| 340 | 340 | case METRICS_AVERAGE_SOCKETPOWER: |
| 341 | − *value = (metrics->AverageSocketPower << 8) / 1000; | |
| 341 | + *value = metrics->AverageSocketPower; | |
| 342 | 342 | break; |
| 343 | 343 | case METRICS_CURR_SOCKETPOWER: |
| 344 | − *value = (metrics->CurrentSocketPower << 8) / 1000; | |
| 344 | + *value = metrics->CurrentSocketPower; | |
| 345 | 345 | break; |
| 346 | 346 | case METRICS_TEMPERATURE_EDGE: |
| 347 | 347 | *value = metrics->GfxTemperature / 100 * |
| @@ -295,7 +295,7 @@ static int smu_v13_0_5_get_smu_metrics_data(struct smu_context *smu, · 294 unmodified lines | ||
| 295 | 295 | *value = metrics->UvdActivity / 100; |
| 296 | 296 | break; |
| 297 | 297 | case METRICS_CURR_SOCKETPOWER: |
| 298 | − *value = (metrics->CurrentSocketPower << 8) / 1000; | |
| 298 | + *value = metrics->CurrentSocketPower; | |
| 299 | 299 | break; |
| 300 | 300 | case METRICS_TEMPERATURE_EDGE: |
| 301 | 301 | *value = metrics->GfxTemperature / 100 * |
| @@ -1321,7 +1321,8 @@ static int smu_v13_0_6_get_smu_metrics_data(struct smu_context *smu, · 1320 unmodified lines | ||
| 1321 | 1321 | *value = SMUQ10_ROUND(GET_METRIC_FIELD(DramBandwidthUtilization, version)); |
| 1322 | 1322 | break; |
| 1323 | 1323 | case METRICS_CURR_SOCKETPOWER: |
| 1324 | − *value = SMUQ10_ROUND(GET_METRIC_FIELD(SocketPower, version)) << 8; | |
| 1324 | + *value = SMUQ10_ROUND(GET_METRIC_FIELD(SocketPower, version)) * | |
| 1325 | + MILLIWATT_PER_WATT; | |
| 1325 | 1326 | break; |
| 1326 | 1327 | case METRICS_TEMPERATURE_HOTSPOT: |
| 1327 | 1328 | *value = SMUQ10_ROUND(GET_METRIC_FIELD(MaxSocketTemperature, version)) * |
| @@ -817,7 +817,7 @@ static int smu_v13_0_7_get_smu_metrics_data(struct smu_context *smu, · 816 unmodified lines | ||
| 817 | 817 | *value = smu_safe_u16_nn(metrics->AverageUclkActivity); |
| 818 | 818 | break; |
| 819 | 819 | case METRICS_AVERAGE_SOCKETPOWER: |
| 820 | − *value = metrics->AverageSocketPower << 8; | |
| 820 | + *value = metrics->AverageSocketPower * MILLIWATT_PER_WATT; | |
| 821 | 821 | break; |
| 822 | 822 | case METRICS_TEMPERATURE_EDGE: |
| 823 | 823 | *value = metrics->AvgTemperature[TEMP_EDGE] * |
| @@ -372,7 +372,7 @@ static int yellow_carp_get_smu_metrics_data(struct smu_context *smu, · 371 unmodified lines | ||
| 372 | 372 | *value = metrics->UvdActivity / 100; |
| 373 | 373 | break; |
| 374 | 374 | case METRICS_CURR_SOCKETPOWER: |
| 375 | − *value = (metrics->CurrentSocketPower << 8) / 1000; | |
| 375 | + *value = metrics->CurrentSocketPower; | |
| 376 | 376 | break; |
| 377 | 377 | case METRICS_TEMPERATURE_EDGE: |
| 378 | 378 | *value = metrics->GfxTemperature / 100 * |
| @@ -312,8 +312,7 @@ static int smu_v14_0_0_get_smu_metrics_data(struct smu_context *smu, · 311 unmodified lines | ||
| 312 | 312 | break; |
| 313 | 313 | case METRICS_AVERAGE_SOCKETPOWER: |
| 314 | 314 | case METRICS_CURR_SOCKETPOWER: |
| 315 | − *value = (metrics->SocketPower / 1000 << 8) + | |
| 316 | − (metrics->SocketPower % 1000 / 10); | |
| 315 | + *value = metrics->SocketPower; | |
| 317 | 316 | break; |
| 318 | 317 | case METRICS_TEMPERATURE_EDGE: |
| 319 | 318 | *value = metrics->GfxTemperature / 100 * |
| @@ -695,7 +695,7 @@ static int smu_v14_0_2_get_smu_metrics_data(struct smu_context *smu, · 694 unmodified lines | ||
| 695 | 695 | metrics->Vcn1ActivityPercentage); |
| 696 | 696 | break; |
| 697 | 697 | case METRICS_AVERAGE_SOCKETPOWER: |
| 698 | − *value = metrics->AverageSocketPower << 8; | |
| 698 | + *value = metrics->AverageSocketPower * MILLIWATT_PER_WATT; | |
| 699 | 699 | break; |
| 700 | 700 | case METRICS_TEMPERATURE_EDGE: |
| 701 | 701 | *value = metrics->AvgTemperature[TEMP_EDGE] * |
| @@ -168,7 +168,7 @@ static int smu_v15_0_0_init_smc_tables(struct smu_context *smu) · 167 unmodified lines | ||
| 168 | 168 | |
| 169 | 169 | SMU_TABLE_INIT(tables, SMU_TABLE_WATERMARKS, sizeof(Watermarks_t), |
| 170 | 170 | PAGE_SIZE, AMDGPU_GEM_DOMAIN_VRAM); |
| 171 | − SMU_TABLE_INIT(tables, SMU_TABLE_DPMCLOCKS, sizeof(DpmClocks_t), | |
| 171 | + SMU_TABLE_INIT(tables, SMU_TABLE_DPMCLOCKS, sizeof(DpmClocks_t_v15_0_5), | |
| 172 | 172 | PAGE_SIZE, AMDGPU_GEM_DOMAIN_VRAM); |
| 173 | 173 | SMU_TABLE_INIT(tables, SMU_TABLE_SMU_METRICS, sizeof(SmuMetrics_t), |
| 174 | 174 | PAGE_SIZE, AMDGPU_GEM_DOMAIN_VRAM); |
| @@ -178,7 +178,7 @@ static int smu_v15_0_0_init_smc_tables(struct smu_context *smu) · 3 unmodified lines | ||
| 178 | 178 | goto err0_out; |
| 179 | 179 | smu_table->metrics_time = 0; |
| 180 | 180 | |
| 181 | − smu_table->clocks_table = kzalloc_obj(DpmClocks_t); | |
| 181 | + smu_table->clocks_table = kzalloc_obj(DpmClocks_t_v15_0_5); | |
| 182 | 182 | if (!smu_table->clocks_table) |
| 183 | 183 | goto err1_out; |
| 184 | 184 | |
| @@ -378,8 +378,7 @@ static int smu_v15_0_0_get_smu_metrics_data(struct smu_context *smu, · 193 unmodified lines | ||
| 378 | 378 | break; |
| 379 | 379 | case METRICS_AVERAGE_SOCKETPOWER: |
| 380 | 380 | case METRICS_CURR_SOCKETPOWER: |
| 381 | − *value = (metrics->SocketPower / 1000 << 8) + | |
| 382 | − (metrics->SocketPower % 1000 / 10); | |
| 381 | + *value = metrics->SocketPower; | |
| 383 | 382 | break; |
| 384 | 383 | case METRICS_TEMPERATURE_EDGE: |
| 385 | 384 | *value = metrics->GfxTemperature / 100 * |
| @@ -752,11 +751,48 @@ static int smu_v15_0_0_get_dpm_freq_by_index(struct smu_context *smu, · 366 unmodified lines | ||
| 752 | 751 | return 0; |
| 753 | 752 | } |
| 754 | 753 | |
| 754 | +static int smu_v15_0_5_get_dpm_freq_by_index(struct smu_context *smu, | |
| 755 | + enum smu_clk_type clk_type, | |
| 756 | + uint32_t dpm_level, | |
| 757 | + uint32_t *freq) | |
| 758 | +{ | |
| 759 | + DpmClocks_t_v15_0_5 *clk_table = smu->smu_table.clocks_table; | |
| 760 | + | |
| 761 | + if (!clk_table || clk_type >= SMU_CLK_COUNT) | |
| 762 | + return -EINVAL; | |
| 763 | + | |
| 764 | + switch (clk_type) { | |
| 765 | + case SMU_SOCCLK: | |
| 766 | + if (dpm_level >= clk_table->NumSocClkLevelsEnabled) | |
| 767 | + return -EINVAL; | |
| 768 | + *freq = clk_table->SocClocks[dpm_level]; | |
| 769 | + break; | |
| 770 | + case SMU_UCLK: | |
| 771 | + case SMU_MCLK: | |
| 772 | + if (dpm_level >= clk_table->NumMemPstatesEnabled) | |
| 773 | + return -EINVAL; | |
| 774 | + *freq = clk_table->MemPstateTable[dpm_level].MemClk; | |
| 775 | + break; | |
| 776 | + case SMU_FCLK: | |
| 777 | + if (dpm_level >= clk_table->NumFclkLevelsEnabled) | |
| 778 | + return -EINVAL; | |
| 779 | + *freq = clk_table->FclkClocks_Freq[dpm_level]; | |
| 780 | + break; | |
| 781 | + default: | |
| 782 | + return -EINVAL; | |
| 783 | + } | |
| 784 | + | |
| 785 | + return 0; | |
| 786 | +} | |
| 787 | + | |
| 755 | 788 | static int smu_v15_0_common_get_dpm_freq_by_index(struct smu_context *smu, |
| 756 | 789 | enum smu_clk_type clk_type, |
| 757 | 790 | uint32_t dpm_level, |
| 758 | 791 | uint32_t *freq) |
| 759 | 792 | { |
| 793 | + if (amdgpu_ip_version(smu->adev, MP1_HWIP, 0) == IP_VERSION(15, 0, 5)) | |
| 794 | + smu_v15_0_5_get_dpm_freq_by_index(smu, clk_type, dpm_level, freq); | |
| 795 | + else | |
| 760 | 796 | smu_v15_0_0_get_dpm_freq_by_index(smu, clk_type, dpm_level, freq); |
| 761 | 797 | |
| 762 | 798 | return 0; |
| @@ -907,12 +943,116 @@ failed: · 144 unmodified lines | ||
| 907 | 943 | return ret; |
| 908 | 944 | } |
| 909 | 945 | |
| 946 | +static int smu_v15_0_5_get_dpm_ultimate_freq(struct smu_context *smu, | |
| 947 | + enum smu_clk_type clk_type, | |
| 948 | + uint32_t *min, | |
| 949 | + uint32_t *max) | |
| 950 | +{ | |
| 951 | + DpmClocks_t_v15_0_5 *clk_table = smu->smu_table.clocks_table; | |
| 952 | + uint32_t clock_limit; | |
| 953 | + uint32_t max_dpm_level, min_dpm_level; | |
| 954 | + int ret = 0; | |
| 955 | + | |
| 956 | + if (!smu_v15_0_0_clk_dpm_is_enabled(smu, clk_type)) { | |
| 957 | + switch (clk_type) { | |
| 958 | + case SMU_MCLK: | |
| 959 | + case SMU_UCLK: | |
| 960 | + clock_limit = smu->smu_table.boot_values.uclk; | |
| 961 | + break; | |
| 962 | + case SMU_FCLK: | |
| 963 | + clock_limit = smu->smu_table.boot_values.fclk; | |
| 964 | + break; | |
| 965 | + case SMU_GFXCLK: | |
| 966 | + case SMU_SCLK: | |
| 967 | + clock_limit = smu->smu_table.boot_values.gfxclk; | |
| 968 | + break; | |
| 969 | + case SMU_SOCCLK: | |
| 970 | + clock_limit = smu->smu_table.boot_values.socclk; | |
| 971 | + break; | |
| 972 | + default: | |
| 973 | + clock_limit = 0; | |
| 974 | + break; | |
| 975 | + } | |
| 976 | + | |
| 977 | + /* clock in Mhz unit */ | |
| 978 | + if (min) | |
| 979 | + *min = clock_limit / 100; | |
| 980 | + if (max) | |
| 981 | + *max = clock_limit / 100; | |
| 982 | + | |
| 983 | + return 0; | |
| 984 | + } | |
| 985 | + | |
| 986 | + if (max) { | |
| 987 | + switch (clk_type) { | |
| 988 | + case SMU_GFXCLK: | |
| 989 | + case SMU_SCLK: | |
| 990 | + *max = clk_table->MaxGfxClk; | |
| 991 | + break; | |
| 992 | + case SMU_MCLK: | |
| 993 | + case SMU_UCLK: | |
| 994 | + max_dpm_level = 0; | |
| 995 | + break; | |
| 996 | + case SMU_FCLK: | |
| 997 | + max_dpm_level = clk_table->NumFclkLevelsEnabled - 1; | |
| 998 | + break; | |
| 999 | + case SMU_SOCCLK: | |
| 1000 | + max_dpm_level = clk_table->NumSocClkLevelsEnabled - 1; | |
| 1001 | + break; | |
| 1002 | + default: | |
| 1003 | + ret = -EINVAL; | |
| 1004 | + goto failed; | |
| 1005 | + } | |
| 1006 | + | |
| 1007 | + if (clk_type != SMU_GFXCLK && clk_type != SMU_SCLK) { | |
| 1008 | + ret = smu_v15_0_common_get_dpm_freq_by_index(smu, clk_type, | |
| 1009 | + max_dpm_level, max); | |
| 1010 | + if (ret) | |
| 1011 | + goto failed; | |
| 1012 | + } | |
| 1013 | + } | |
| 1014 | + | |
| 1015 | + if (min) { | |
| 1016 | + switch (clk_type) { | |
| 1017 | + case SMU_GFXCLK: | |
| 1018 | + case SMU_SCLK: | |
| 1019 | + *min = clk_table->MinGfxClk; | |
| 1020 | + break; | |
| 1021 | + case SMU_MCLK: | |
| 1022 | + case SMU_UCLK: | |
| 1023 | + min_dpm_level = clk_table->NumMemPstatesEnabled - 1; | |
| 1024 | + break; | |
| 1025 | + case SMU_FCLK: | |
| 1026 | + min_dpm_level = 0; | |
| 1027 | + break; | |
| 1028 | + case SMU_SOCCLK: | |
| 1029 | + min_dpm_level = 0; | |
| 1030 | + break; | |
| 1031 | + default: | |
| 1032 | + ret = -EINVAL; | |
| 1033 | + goto failed; | |
| 1034 | + } | |
| 1035 | + | |
| 1036 | + if (clk_type != SMU_GFXCLK && clk_type != SMU_SCLK) { | |
| 1037 | + ret = smu_v15_0_common_get_dpm_freq_by_index(smu, clk_type, | |
| 1038 | + min_dpm_level, min); | |
| 1039 | + if (ret) | |
| 1040 | + goto failed; | |
| 1041 | + } | |
| 1042 | + } | |
| 1043 | + | |
| 1044 | +failed: | |
| 1045 | + return ret; | |
| 1046 | +} | |
| 1047 | + | |
| 910 | 1048 | static int smu_v15_0_common_get_dpm_ultimate_freq(struct smu_context *smu, |
| 911 | 1049 | enum smu_clk_type clk_type, |
| 912 | 1050 | uint32_t *min, |
| 913 | 1051 | uint32_t *max) |
| 914 | 1052 | { |
| 915 | − if (clk_type != SMU_VCLK1 && clk_type != SMU_DCLK1) | |
| 1053 | + if (amdgpu_ip_version(smu->adev, MP1_HWIP, 0) == IP_VERSION(15, 0, 5)) | |
| 1054 | + smu_v15_0_5_get_dpm_ultimate_freq(smu, clk_type, min, max); | |
| 1055 | + else if (clk_type != SMU_VCLK1 && clk_type != SMU_DCLK1) | |
| 916 | 1056 | smu_v15_0_0_get_dpm_ultimate_freq(smu, clk_type, min, max); |
| 917 | 1057 | |
| 918 | 1058 | return 0; |
| @@ -986,11 +1126,36 @@ static int smu_v15_0_0_get_dpm_level_count(struct smu_context *smu, · 67 unmodified lines | ||
| 986 | 1126 | return 0; |
| 987 | 1127 | } |
| 988 | 1128 | |
| 1129 | +static int smu_v15_0_5_get_dpm_level_count(struct smu_context *smu, | |
| 1130 | + enum smu_clk_type clk_type, | |
| 1131 | + uint32_t *count) | |
| 1132 | +{ | |
| 1133 | + DpmClocks_t_v15_0_5 *clk_table = smu->smu_table.clocks_table; | |
| 1134 | + | |
| 1135 | + switch (clk_type) { | |
| 1136 | + case SMU_SOCCLK: | |
| 1137 | + *count = clk_table->NumSocClkLevelsEnabled; | |
| 1138 | + break; | |
| 1139 | + case SMU_MCLK: | |
| 1140 | + *count = clk_table->NumMemPstatesEnabled; | |
| 1141 | + break; | |
| 1142 | + case SMU_FCLK: | |
| 1143 | + *count = clk_table->NumFclkLevelsEnabled; | |
| 1144 | + break; | |
| 1145 | + default: | |
| 1146 | + break; | |
| 1147 | + } | |
| 1148 | + | |
| 1149 | + return 0; | |
| 1150 | +} | |
| 1151 | + | |
| 989 | 1152 | static int smu_v15_0_common_get_dpm_level_count(struct smu_context *smu, |
| 990 | 1153 | enum smu_clk_type clk_type, |
| 991 | 1154 | uint32_t *count) |
| 992 | 1155 | { |
| 993 | − if (clk_type != SMU_VCLK1 && clk_type != SMU_DCLK1) | |
| 1156 | + if (amdgpu_ip_version(smu->adev, MP1_HWIP, 0) == IP_VERSION(15, 0, 5)) | |
| 1157 | + smu_v15_0_5_get_dpm_level_count(smu, clk_type, count); | |
| 1158 | + else if (clk_type != SMU_VCLK1 && clk_type != SMU_DCLK1) | |
| 994 | 1159 | smu_v15_0_0_get_dpm_level_count(smu, clk_type, count); |
| 995 | 1160 | |
| 996 | 1161 | return 0; |
| @@ -1363,9 +1528,24 @@ static int smu_v15_0_0_set_fine_grain_gfx_freq_parameters(struct smu_context *sm · 366 unmodified lines | ||
| 1363 | 1528 | return 0; |
| 1364 | 1529 | } |
| 1365 | 1530 | |
| 1531 | +static int smu_v15_0_5_set_fine_grain_gfx_freq_parameters(struct smu_context *smu) | |
| 1532 | +{ | |
| 1533 | + DpmClocks_t_v15_0_5 *clk_table = smu->smu_table.clocks_table; | |
| 1534 | + | |
| 1535 | + smu->gfx_default_hard_min_freq = clk_table->MinGfxClk; | |
| 1536 | + smu->gfx_default_soft_max_freq = clk_table->MaxGfxClk; | |
| 1537 | + smu->gfx_actual_hard_min_freq = 0; | |
| 1538 | + smu->gfx_actual_soft_max_freq = 0; | |
| 1539 | + | |
| 1540 | + return 0; | |
| 1541 | +} | |
| 1542 | + | |
| 1366 | 1543 | static int smu_v15_0_common_set_fine_grain_gfx_freq_parameters(struct smu_context *smu) |
| 1367 | 1544 | { |
| 1368 | − smu_v15_0_0_set_fine_grain_gfx_freq_parameters(smu); | |
| 1545 | + if (amdgpu_ip_version(smu->adev, MP1_HWIP, 0) == IP_VERSION(15, 0, 5)) | |
| 1546 | + smu_v15_0_5_set_fine_grain_gfx_freq_parameters(smu); | |
| 1547 | + else | |
| 1548 | + smu_v15_0_0_set_fine_grain_gfx_freq_parameters(smu); | |
| 1369 | 1549 | |
| 1370 | 1550 | return 0; |
| 1371 | 1551 | } |
| @@ -1405,9 +1585,36 @@ static int smu_v15_0_0_get_dpm_table(struct smu_context *smu, struct dpm_clocks · 33 unmodified lines | ||
| 1405 | 1585 | return 0; |
| 1406 | 1586 | } |
| 1407 | 1587 | |
| 1588 | +static int smu_v15_0_5_get_dpm_table(struct smu_context *smu, struct dpm_clocks *clock_table) | |
| 1589 | +{ | |
| 1590 | + DpmClocks_t_v15_0_5 *clk_table = smu->smu_table.clocks_table; | |
| 1591 | + uint8_t idx; | |
| 1592 | + | |
| 1593 | + /* | |
| 1594 | + * Only the Clock information of SOC and | |
| 1595 | + * VPE is copied to provide VPE DPM settings for use. | |
| 1596 | + */ | |
| 1597 | + for (idx = 0; idx < NUM_SOCCLK_DPM_LEVELS; idx++) { | |
| 1598 | + clock_table->SocClocks[idx].Freq = | |
| 1599 | + (idx < clk_table->NumSocClkLevelsEnabled) ? clk_table->SocClocks[idx]:0; | |
| 1600 | + clock_table->SocClocks[idx].Vol = 0; | |
| 1601 | + } | |
| 1602 | + | |
| 1603 | + for (idx = 0; idx < NUM_VPE_DPM_LEVELS; idx++) { | |
| 1604 | + clock_table->VPEClocks[idx].Freq = | |
| 1605 | + (idx < clk_table->VpeClkLevelsEnabled) ? clk_table->VPEClocks[idx]:0; | |
| 1606 | + clock_table->VPEClocks[idx].Vol = 0; | |
| 1607 | + } | |
| 1608 | + | |
| 1609 | + return 0; | |
| 1610 | +} | |
| 1611 | + | |
| 1408 | 1612 | static int smu_v15_0_common_get_dpm_table(struct smu_context *smu, struct dpm_clocks *clock_table) |
| 1409 | 1613 | { |
| 1410 | − smu_v15_0_0_get_dpm_table(smu, clock_table); | |
| 1614 | + if (amdgpu_ip_version(smu->adev, MP1_HWIP, 0) == IP_VERSION(15, 0, 5)) | |
| 1615 | + smu_v15_0_5_get_dpm_table(smu, clock_table); | |
| 1616 | + else | |
| 1617 | + smu_v15_0_0_get_dpm_table(smu, clock_table); | |
| 1411 | 1618 | |
| 1412 | 1619 | return 0; |
| 1413 | 1620 | } |
| @@ -411,7 +411,8 @@ static int smu_v15_0_8_get_smu_metrics_data(struct smu_context *smu, · 410 unmodified lines | ||
| 411 | 411 | *value = SMUQ10_ROUND(metrics->DramBandwidthUtilization); |
| 412 | 412 | break; |
| 413 | 413 | case METRICS_CURR_SOCKETPOWER: |
| 414 | − *value = SMUQ10_ROUND(metrics->SocketPower) << 8; | |
| 414 | + *value = SMUQ10_ROUND(metrics->SocketPower) * | |
| 415 | + MILLIWATT_PER_WATT; | |
| 415 | 416 | break; |
| 416 | 417 | case METRICS_TEMPERATURE_HOTSPOT: |
| 417 | 418 | *value = SMUQ10_ROUND(metrics->MaxSocketTemperature) * |
| @@ -40,6 +40,13 @@ static int display_connector_attach(struct drm_bridge *bridge, · 39 unmodified lines | ||
| 40 | 40 | return flags & DRM_BRIDGE_ATTACH_NO_CONNECTOR ? 0 : -EINVAL; |
| 41 | 41 | } |
| 42 | 42 | |
| 43 | +static void display_connector_destroy(struct drm_bridge *bridge) | |
| 44 | +{ | |
| 45 | + struct display_connector *conn = to_display_connector(bridge); | |
| 46 | + | |
| 47 | + i2c_put_adapter(conn->bridge.ddc); | |
| 48 | +} | |
| 49 | + | |
| 43 | 50 | static enum drm_connector_status display_connector_detect(struct drm_bridge *bridge) |
| 44 | 51 | { |
| 45 | 52 | struct display_connector *conn = to_display_connector(bridge); |
| @@ -177,6 +184,7 @@ static u32 *display_connector_get_input_bus_fmts(struct drm_bridge *bridge, · 131 unmodified lines | ||
| 177 | 184 | |
| 178 | 185 | static const struct drm_bridge_funcs display_connector_bridge_funcs = { |
| 179 | 186 | .attach = display_connector_attach, |
| 187 | + .destroy = display_connector_destroy, | |
| 180 | 188 | .detect = display_connector_bridge_detect, |
| 181 | 189 | .edid_read = display_connector_edid_read, |
| 182 | 190 | .atomic_get_output_bus_fmts = display_connector_get_output_bus_fmts, |
| @@ -403,9 +411,6 @@ static void display_connector_remove(struct platform_device *pdev) · 220 unmodified lines | ||
| 403 | 411 | regulator_disable(conn->supply); |
| 404 | 412 | |
| 405 | 413 | drm_bridge_remove(&conn->bridge); |
| 406 | − | |
| 407 | − if (!IS_ERR(conn->bridge.ddc)) | |
| 408 | − i2c_put_adapter(conn->bridge.ddc); | |
| 409 | 414 | } |
| 410 | 415 | |
| 411 | 416 | static const struct of_device_id display_connector_match[] = { |
| @@ -257,8 +257,14 @@ static ssize_t ps8640_aux_transfer_msg(struct drm_dp_aux *aux, · 256 unmodified lines | ||
| 257 | 257 | addr_len[PAGE0_SWAUX_LENGTH - base] = (len == 0) ? SWAUX_NO_PAYLOAD : |
| 258 | 258 | ((len - 1) & SWAUX_LENGTH_MASK); |
| 259 | 259 | |
| 260 | − regmap_bulk_write(map, PAGE0_SWAUX_ADDR_7_0, addr_len, | |
| 261 | − ARRAY_SIZE(addr_len)); | |
| 260 | + ret = regmap_bulk_write(map, PAGE0_SWAUX_ADDR_7_0, addr_len, | |
| 261 | + ARRAY_SIZE(addr_len)); | |
| 262 | + if (ret) { | |
| 263 | + DRM_DEV_ERROR(dev, | |
| 264 | + "failed to write AUX address %#x, len %zu: %d\n", | |
| 265 | + msg->address, len, ret); | |
| 266 | + return ret; | |
| 267 | + } | |
| 262 | 268 | |
| 263 | 269 | if (len && (request == DP_AUX_NATIVE_WRITE || |
| 264 | 270 | request == DP_AUX_I2C_WRITE)) { |
| @@ -274,13 +280,22 @@ static ssize_t ps8640_aux_transfer_msg(struct drm_dp_aux *aux, · 9 unmodified lines | ||
| 274 | 280 | } |
| 275 | 281 | } |
| 276 | 282 | |
| 277 | − regmap_write(map, PAGE0_SWAUX_CTRL, SWAUX_SEND); | |
| 283 | + ret = regmap_write(map, PAGE0_SWAUX_CTRL, SWAUX_SEND); | |
| 284 | + if (ret) { | |
| 285 | + DRM_DEV_ERROR(dev, "failed to start AUX transfer: %d\n", ret); | |
| 286 | + return ret; | |
| 287 | + } | |
| 278 | 288 | |
| 279 | 289 | /* Zero delay loop because i2c transactions are slow already */ |
| 280 | − regmap_read_poll_timeout(map, PAGE0_SWAUX_CTRL, data, | |
| 281 | − !(data & SWAUX_SEND), 0, 50 * 1000); | |
| 290 | + ret = regmap_read_poll_timeout(map, PAGE0_SWAUX_CTRL, data, | |
| 291 | + !(data & SWAUX_SEND), 0, 50 * 1000); | |
| 292 | + if (ret) { | |
| 293 | + DRM_DEV_ERROR(dev, "failed to complete AUX transfer: %d\n", | |
| 294 | + ret); | |
| 295 | + return ret; | |
| 296 | + } | |
| 282 | 297 | |
| 283 | − regmap_read(map, PAGE0_SWAUX_STATUS, &data); | |
| 298 | + ret = regmap_read(map, PAGE0_SWAUX_STATUS, &data); | |
| 284 | 299 | if (ret) { |
| 285 | 300 | DRM_DEV_ERROR(dev, "failed to read PAGE0_SWAUX_STATUS: %d\n", |
| 286 | 301 | ret); |
| @@ -3686,6 +3686,18 @@ int drm_dp_get_pcon_max_frl_bw(const u8 dpcd[DP_RECEIVER_CAP_SIZE], · 3685 unmodified lines | ||
| 3686 | 3686 | int bw; |
| 3687 | 3687 | u8 buf; |
| 3688 | 3688 | |
| 3689 | + if (!drm_dp_is_branch(dpcd)) | |
| 3690 | + return 0; | |
| 3691 | + | |
| 3692 | + if (dpcd[DP_DPCD_REV] < 0x11) | |
| 3693 | + return 0; | |
| 3694 | + | |
| 3695 | + if ((dpcd[DP_DOWNSTREAMPORT_PRESENT] & DP_DETAILED_CAP_INFO_AVAILABLE) == 0) | |
| 3696 | + return 0; | |
| 3697 | + | |
| 3698 | + if ((port_cap[0] & DP_DS_PORT_TYPE_MASK) != DP_DS_PORT_TYPE_HDMI) | |
| 3699 | + return 0; | |
| 3700 | + | |
| 3689 | 3701 | buf = port_cap[2]; |
| 3690 | 3702 | bw = buf & DP_PCON_MAX_FRL_BW; |
| 3691 | 3703 | |
| @@ -597,9 +597,13 @@ static vm_fault_t try_insert_pfn(struct vm_fault *vmf, unsigned int order, · 596 unmodified lines | ||
| 597 | 597 | #ifdef CONFIG_ARCH_SUPPORTS_PMD_PFNMAP |
| 598 | 598 | } else if (order == PMD_ORDER) { |
| 599 | 599 | unsigned long paddr = pfn << PAGE_SHIFT; |
| 600 | + struct vm_area_struct *vma = vmf->vma; | |
| 601 | + unsigned long start = ALIGN_DOWN(vmf->address, PMD_SIZE); | |
| 602 | + unsigned long end = start + PMD_SIZE; | |
| 603 | + bool in_range = vma->vm_start <= start && end <= vma->vm_end; | |
| 600 | 604 | bool aligned = (vmf->address & ~PMD_MASK) == (paddr & ~PMD_MASK); |
| 601 | 605 | |
| 602 | − if (aligned && | |
| 606 | + if (aligned && in_range && | |
| 603 | 607 | folio_test_pmd_mappable(page_folio(pfn_to_page(pfn)))) { |
| 604 | 608 | vm_fault_t ret; |
| 605 | 609 | |
| @@ -3501,6 +3501,8 @@ static void intel_ddi_enable_hdmi(struct intel_atomic_state *state, · 3500 unmodified lines | ||
| 3501 | 3501 | } |
| 3502 | 3502 | |
| 3503 | 3503 | intel_ddi_buf_enable(encoder, buf_ctl); |
| 3504 | + | |
| 3505 | + intel_hdmi_poll_for_scrambling_enable(crtc_state, connector); | |
| 3504 | 3506 | } |
| 3505 | 3507 | |
| 3506 | 3508 | static void intel_ddi_enable(struct intel_atomic_state *state, |
| @@ -4242,7 +4242,14 @@ static int intel_dp_hdmi_sink_max_frl(struct intel_dp *intel_dp) · 4241 unmodified lines | ||
| 4242 | 4242 | rate_per_lane = info->hdmi.max_frl_rate_per_lane; |
| 4243 | 4243 | max_frl_rate = max_lanes * rate_per_lane; |
| 4244 | 4244 | |
| 4245 | − if (info->hdmi.dsc_cap.v_1p2) { | |
| 4245 | + /* | |
| 4246 | + * The sink's DSC max FRL rate only applies to compressed video | |
| 4247 | + * transport, which requires a DSC 1.2 encoder in the PCON. Without | |
| 4248 | + * one the HDMI link always carries uncompressed video, for which | |
| 4249 | + * the regular max FRL rate is the limit. | |
| 4250 | + */ | |
| 4251 | + if (drm_dp_pcon_enc_is_dsc_1_2(intel_dp->pcon_dsc_dpcd) && | |
| 4252 | + info->hdmi.dsc_cap.v_1p2) { | |
| 4246 | 4253 | max_dsc_lanes = info->hdmi.dsc_cap.max_lanes; |
| 4247 | 4254 | dsc_rate_per_lane = info->hdmi.dsc_cap.max_frl_rate_per_lane; |
| 4248 | 4255 | if (max_dsc_lanes && dsc_rate_per_lane) |
| @@ -2728,6 +2728,32 @@ intel_hdmi_add_properties(struct intel_hdmi *intel_hdmi, struct drm_connector *_ · 2727 unmodified lines | ||
| 2728 | 2728 | drm_connector_attach_max_bpc_property(&connector->base, 8, 12); |
| 2729 | 2729 | } |
| 2730 | 2730 | |
| 2731 | +/* | |
| 2732 | + * HDMI 2.0 spec, section 6.1.3.1 (Scrambling Control): after | |
| 2733 | + * enabling Scrambling_Enable and starting scrambled video | |
| 2734 | + * transmission, poll Scrambling_Status for up to 200 ms. | |
| 2735 | + */ | |
| 2736 | +void | |
| 2737 | +intel_hdmi_poll_for_scrambling_enable(const struct intel_crtc_state *crtc_state, | |
| 2738 | + struct drm_connector *_connector) | |
| 2739 | +{ | |
| 2740 | + struct intel_connector *connector = to_intel_connector(_connector); | |
| 2741 | + struct intel_display *display = to_intel_display(crtc_state); | |
| 2742 | + bool scrambling_enabled = false; | |
| 2743 | + int ret; | |
| 2744 | + | |
| 2745 | + if (!crtc_state->hdmi_scrambling) | |
| 2746 | + return; | |
| 2747 | + | |
| 2748 | + /* Poll for a max of 200 msec as per HDMI spec */ | |
| 2749 | + ret = poll_timeout_us(scrambling_enabled = drm_scdc_get_scrambling_status(&connector->base), | |
| 2750 | + scrambling_enabled, 1000, 200 * 1000, false); | |
| 2751 | + if (ret) | |
| 2752 | + drm_dbg_kms(display->drm, | |
| 2753 | + "[CONNECTOR:%d:%s] Timed out waiting for scrambling enable\n", | |
| 2754 | + connector->base.base.id, connector->base.name); | |
| 2755 | +} | |
| 2756 | + | |
| 2731 | 2757 | /* |
| 2732 | 2758 | * intel_hdmi_handle_sink_scrambling: handle sink scrambling/clock ratio setup |
| 2733 | 2759 | * @encoder: intel_encoder |
| @@ -70,5 +70,7 @@ void hsw_read_infoframe(struct intel_encoder *encoder, · 69 unmodified lines | ||
| 70 | 70 | const struct intel_crtc_state *crtc_state, |
| 71 | 71 | unsigned int type, |
| 72 | 72 | void *frame, ssize_t len); |
| 73 | +void intel_hdmi_poll_for_scrambling_enable(const struct intel_crtc_state *crtc_state, | |
| 74 | + struct drm_connector *_connector); | |
| 73 | 75 | |
| 74 | 76 | #endif /* __INTEL_HDMI_H__ */ |
| @@ -726,8 +726,6 @@ static void pvr_queue_submit_job_to_cccb(struct pvr_job *job) · 725 unmodified lines | ||
| 726 | 726 | cmd->partial_render_geom_frag_fence.value = job->done_fence->seqno - 1; |
| 727 | 727 | } |
| 728 | 728 | |
| 729 | − trace_pvr_job_submit_fw(job); | |
| 730 | − | |
| 731 | 729 | /* Submit job to FW */ |
| 732 | 730 | pvr_cccb_write_command_with_header(cccb, job->fw_ccb_cmd_type, job->cmd_len, job->cmd, |
| 733 | 731 | job->id, job->id); |
| @@ -802,6 +800,9 @@ static struct dma_fence *pvr_queue_run_job(struct drm_sched_job *sched_job) · 68 unmodified lines | ||
| 802 | 800 | job->hwrt, |
| 803 | 801 | frag_job->fw_ccb_cmd_type == |
| 804 | 802 | ROGUE_FWIF_CCB_CMD_TYPE_FRAG_PR); |
| 803 | + | |
| 804 | + trace_pvr_job_submit_fw(geom_job); | |
| 805 | + trace_pvr_job_submit_fw(frag_job); | |
| 805 | 806 | } else { |
| 806 | 807 | struct pvr_queue *queue = container_of(job->base.sched, |
| 807 | 808 | struct pvr_queue, scheduler); |
| @@ -809,6 +810,8 @@ static struct dma_fence *pvr_queue_run_job(struct drm_sched_job *sched_job) · 1 unmodified line | ||
| 809 | 810 | pvr_cccb_send_kccb_kick(pvr_dev, &queue->cccb, |
| 810 | 811 | pvr_context_get_fw_addr(job->ctx) + queue->ctx_offset, |
| 811 | 812 | job->hwrt); |
| 813 | + | |
| 814 | + trace_pvr_job_submit_fw(job); | |
| 812 | 815 | } |
| 813 | 816 | |
| 814 | 817 | return dma_fence_get(job->done_fence); |
| @@ -41,6 +41,17 @@ TRACE_EVENT(pvr_job_submit_ioctl, · 40 unmodified lines | ||
| 41 | 41 | __entry->count) |
| 42 | 42 | ); |
| 43 | 43 | |
| 44 | +#define PVR_JOB_GET_HWRT_FW_ADDR(job) \ | |
| 45 | + ({ \ | |
| 46 | + struct pvr_job *_job = (job); \ | |
| 47 | + u32 _hwrt_fw_addr = 0; \ | |
| 48 | + \ | |
| 49 | + if (_job && _job->hwrt) \ | |
| 50 | + pvr_fw_object_get_fw_addr(_job->hwrt->fw_obj, &_hwrt_fw_addr); \ | |
| 51 | + \ | |
| 52 | + _hwrt_fw_addr; \ | |
| 53 | + }) | |
| 54 | + | |
| 44 | 55 | #define PVR_JOB_TYPE_TO_STR(val) \ |
| 45 | 56 | __print_symbolic(val, \ |
| 46 | 57 | { DRM_PVR_JOB_TYPE_GEOMETRY, "geometry" }, \ |
| @@ -64,9 +75,7 @@ TRACE_EVENT(pvr_job_create, · 17 unmodified lines | ||
| 64 | 75 | __entry->ctx = job->ctx; |
| 65 | 76 | __entry->fw_obj = job->ctx->fw_obj; |
| 66 | 77 | pvr_fw_object_get_fw_addr(job->ctx->fw_obj, &__entry->fw_addr); |
| 67 | − __entry->hwrt_addr = job->hwrt ? | |
| 68 | − job->hwrt->fw_obj->fw_addr_offset : | |
| 69 | − 0; | |
| 78 | + __entry->hwrt_addr = PVR_JOB_GET_HWRT_FW_ADDR(job); | |
| 70 | 79 | __entry->job = job; |
| 71 | 80 | __entry->job_type = job->type; |
| 72 | 81 | __entry->sync_op_count = sync_op_count;), |
| @@ -82,6 +91,7 @@ TRACE_EVENT(pvr_job_create, · 9 unmodified lines | ||
| 82 | 91 | ); |
| 83 | 92 | |
| 84 | 93 | #undef PVR_JOB_TYPE_TO_STR |
| 94 | +#undef PVR_JOB_GET_HWRT_FW_ADDR | |
| 85 | 95 | |
| 86 | 96 | TRACE_EVENT(pvr_job_submit_fw, |
| 87 | 97 | TP_PROTO(struct pvr_job *job), |
| @@ -154,10 +154,10 @@ static void mtk_crtc_reset(struct drm_crtc *crtc) · 153 unmodified lines | ||
| 154 | 154 | { |
| 155 | 155 | struct mtk_crtc_state *state; |
| 156 | 156 | |
| 157 | − if (crtc->state) | |
| 157 | + if (crtc->state) { | |
| 158 | 158 | __drm_atomic_helper_crtc_destroy_state(crtc->state); |
| 159 | − | |
| 160 | − kfree(to_mtk_crtc_state(crtc->state)); | |
| 159 | + kfree(to_mtk_crtc_state(crtc->state)); | |
| 160 | + } | |
| 161 | 161 | crtc->state = NULL; |
| 162 | 162 | |
| 163 | 163 | state = kzalloc_obj(*state); |
| @@ -625,6 +625,7 @@ static void mtk_disp_ovl_adaptor_master_unbind(struct device *dev) · 624 unmodified lines | ||
| 625 | 625 | struct mtk_disp_ovl_adaptor *priv = dev_get_drvdata(dev); |
| 626 | 626 | |
| 627 | 627 | priv->children_bound = false; |
| 628 | + component_unbind_all(dev, priv->mmsys_dev); | |
| 628 | 629 | } |
| 629 | 630 | |
| 630 | 631 | static const struct component_master_ops mtk_disp_ovl_adaptor_master_ops = { |
| @@ -651,12 +652,15 @@ static int mtk_disp_ovl_adaptor_probe(struct platform_device *pdev) · 20 unmodified lines | ||
| 651 | 652 | |
| 652 | 653 | priv->mmsys_dev = pdev->dev.platform_data; |
| 653 | 654 | |
| 654 | − component_master_add_with_match(dev, &mtk_disp_ovl_adaptor_master_ops, match); | |
| 655 | + ret = component_master_add_with_match(dev, &mtk_disp_ovl_adaptor_master_ops, match); | |
| 656 | + if (ret) | |
| 657 | + return dev_err_probe(dev, ret, "Failed to add component master\n"); | |
| 655 | 658 | |
| 656 | 659 | pm_runtime_enable(dev); |
| 657 | 660 | |
| 658 | 661 | ret = component_add(dev, &mtk_disp_ovl_adaptor_comp_ops); |
| 659 | 662 | if (ret != 0) { |
| 663 | + component_master_del(dev, &mtk_disp_ovl_adaptor_master_ops); | |
| 660 | 664 | pm_runtime_disable(dev); |
| 661 | 665 | return dev_err_probe(dev, ret, "Failed to add component\n"); |
| 662 | 666 | } |
| @@ -666,6 +670,7 @@ static int mtk_disp_ovl_adaptor_probe(struct platform_device *pdev) · 3 unmodified lines | ||
| 666 | 670 | |
| 667 | 671 | static void mtk_disp_ovl_adaptor_remove(struct platform_device *pdev) |
| 668 | 672 | { |
| 673 | + component_del(&pdev->dev, &mtk_disp_ovl_adaptor_comp_ops); | |
| 669 | 674 | component_master_del(&pdev->dev, &mtk_disp_ovl_adaptor_master_ops); |
| 670 | 675 | pm_runtime_disable(&pdev->dev); |
| 671 | 676 | } |
| @@ -743,8 +743,6 @@ static int mtk_dsi_poweron(struct mtk_dsi *dsi) · 742 unmodified lines | ||
| 743 | 743 | mtk_dsi_set_vm_cmd(dsi); |
| 744 | 744 | mtk_dsi_config_vdo_timing(dsi); |
| 745 | 745 | mtk_dsi_set_interrupt_enable(dsi); |
| 746 | − mtk_dsi_lane_ready(dsi); | |
| 747 | − mtk_dsi_clk_hs_mode(dsi, 1); | |
| 748 | 746 | |
| 749 | 747 | return 0; |
| 750 | 748 | err_disable_engine_clk: |
| @@ -858,6 +856,9 @@ static void mtk_dsi_bridge_atomic_pre_enable(struct drm_bridge *bridge, · 107 unmodified lines | ||
| 858 | 856 | ret = mtk_dsi_poweron(dsi); |
| 859 | 857 | if (ret < 0) |
| 860 | 858 | drm_err(drm, "failed to power on dsi\n"); |
| 859 | + | |
| 860 | + mtk_dsi_lane_ready(dsi); | |
| 861 | + mtk_dsi_clk_hs_mode(dsi, 1); | |
| 861 | 862 | } |
| 862 | 863 | |
| 863 | 864 | static void mtk_dsi_bridge_atomic_post_disable(struct drm_bridge *bridge, |
| @@ -290,6 +290,13 @@ static int mtk_hdmi_get_cec_dev(struct mtk_hdmi *hdmi, struct device *dev, struc · 289 unmodified lines | ||
| 290 | 290 | return 0; |
| 291 | 291 | } |
| 292 | 292 | |
| 293 | +static void mtk_hdmi_put_adapter(void *_adap) | |
| 294 | +{ | |
| 295 | + struct i2c_adapter *adap = _adap; | |
| 296 | + | |
| 297 | + i2c_put_adapter(adap); | |
| 298 | +} | |
| 299 | + | |
| 293 | 300 | static int mtk_hdmi_dt_parse_pdata(struct mtk_hdmi *hdmi, struct platform_device *pdev, |
| 294 | 301 | const char * const *clk_names, size_t num_clocks) |
| 295 | 302 | { |
| @@ -328,12 +335,12 @@ static int mtk_hdmi_dt_parse_pdata(struct mtk_hdmi *hdmi, struct platform_device · 32 unmodified lines | ||
| 328 | 335 | if (!i2c_np) |
| 329 | 336 | return dev_err_probe(dev, -EINVAL, "No ddc-i2c-bus in connector\n"); |
| 330 | 337 | |
| 331 | − hdmi->ddc_adpt = of_find_i2c_adapter_by_node(i2c_np); | |
| 338 | + hdmi->ddc_adpt = of_get_i2c_adapter_by_node(i2c_np); | |
| 332 | 339 | of_node_put(i2c_np); |
| 333 | 340 | if (!hdmi->ddc_adpt) |
| 334 | 341 | return dev_err_probe(dev, -EPROBE_DEFER, "Failed to get ddc i2c adapter by node\n"); |
| 335 | 342 | |
| 336 | − ret = devm_add_action_or_reset(dev, mtk_hdmi_put_device, &hdmi->ddc_adpt->dev); | |
| 343 | + ret = devm_add_action_or_reset(dev, mtk_hdmi_put_adapter, hdmi->ddc_adpt); | |
| 337 | 344 | if (ret) |
| 338 | 345 | return ret; |
| 339 | 346 | |
| @@ -1499,13 +1499,6 @@ static int mtk_hdmi_v2_probe(struct platform_device *pdev) · 1498 unmodified lines | ||
| 1499 | 1499 | return 0; |
| 1500 | 1500 | } |
| 1501 | 1501 | |
| 1502 | −static void mtk_hdmi_v2_remove(struct platform_device *pdev) | |
| 1503 | −{ | |
| 1504 | − struct mtk_hdmi *hdmi = platform_get_drvdata(pdev); | |
| 1505 | − | |
| 1506 | − i2c_put_adapter(hdmi->ddc_adpt); | |
| 1507 | −} | |
| 1508 | − | |
| 1509 | 1502 | static const struct of_device_id mtk_drm_hdmi_v2_of_ids[] = { |
| 1510 | 1503 | { .compatible = "mediatek,mt8188-hdmi-tx", .data = &mtk_hdmi_conf_mt8188 }, |
| 1511 | 1504 | { .compatible = "mediatek,mt8195-hdmi-tx", .data = &mtk_hdmi_conf_mt8195 }, |
| @@ -1515,7 +1508,6 @@ MODULE_DEVICE_TABLE(of, mtk_drm_hdmi_v2_of_ids); · 3 unmodified lines | ||
| 1515 | 1508 | |
| 1516 | 1509 | static struct platform_driver mtk_hdmi_v2_driver = { |
| 1517 | 1510 | .probe = mtk_hdmi_v2_probe, |
| 1518 | − .remove = mtk_hdmi_v2_remove, | |
| 1519 | 1511 | .driver = { |
| 1520 | 1512 | .name = "mediatek-drm-hdmi-v2", |
| 1521 | 1513 | .of_match_table = mtk_drm_hdmi_v2_of_ids, |
| @@ -545,6 +545,7 @@ static int panthor_fw_load_section_entry(struct panthor_device *ptdev, · 544 unmodified lines | ||
| 545 | 545 | struct panthor_fw_binary_section_entry_hdr hdr; |
| 546 | 546 | struct panthor_fw_section *section; |
| 547 | 547 | u32 section_size; |
| 548 | + u32 data_size; | |
| 548 | 549 | u32 name_len; |
| 549 | 550 | int ret; |
| 550 | 551 | |
| @@ -595,6 +596,16 @@ static int panthor_fw_load_section_entry(struct panthor_device *ptdev, · 44 unmodified lines | ||
| 595 | 596 | return -EINVAL; |
| 596 | 597 | } |
| 597 | 598 | |
| 599 | + section_size = hdr.va.end - hdr.va.start; | |
| 600 | + data_size = hdr.data.end - hdr.data.start; | |
| 601 | + if (data_size > section_size) { | |
| 602 | + drm_err(&ptdev->base, "Firmware corrupted, section data exceeds section size\n"); | |
| 603 | + return -EINVAL; | |
| 604 | + } | |
| 605 | + | |
| 606 | + if (!section_size) | |
| 607 | + return 0; | |
| 608 | + | |
| 598 | 609 | name_len = iter->size - iter->offset; |
| 599 | 610 | |
| 600 | 611 | section = drmm_kzalloc(&ptdev->base, sizeof(*section), GFP_KERNEL); |
| @@ -603,7 +614,7 @@ static int panthor_fw_load_section_entry(struct panthor_device *ptdev, · 2 unmodified lines | ||
| 603 | 614 | |
| 604 | 615 | list_add_tail(§ion->node, &ptdev->fw->sections); |
| 605 | 616 | section->flags = hdr.flags; |
| 606 | − section->data.size = hdr.data.end - hdr.data.start; | |
| 617 | + section->data.size = data_size; | |
| 607 | 618 | |
| 608 | 619 | if (section->data.size > 0) { |
| 609 | 620 | void *data = drmm_kmalloc(&ptdev->base, section->data.size, GFP_KERNEL); |
| @@ -626,7 +637,6 @@ static int panthor_fw_load_section_entry(struct panthor_device *ptdev, · 16 unmodified lines | ||
| 626 | 637 | section->name = name; |
| 627 | 638 | } |
| 628 | 639 | |
| 629 | − section_size = hdr.va.end - hdr.va.start; | |
| 630 | 640 | if (section_size) { |
| 631 | 641 | u32 cache_mode = hdr.flags & CSF_FW_BINARY_IFACE_ENTRY_CACHE_MODE_MASK; |
| 632 | 642 | struct panthor_gem_object *bo; |
| @@ -857,18 +867,24 @@ out: · 224 unmodified lines | ||
| 857 | 867 | * iface_fw_to_cpu_addr() - Turn an MCU address into a CPU address |
| 858 | 868 | * @ptdev: Device. |
| 859 | 869 | * @mcu_va: MCU address. |
| 870 | + * @size: Size of the object pointed to by @mcu_va. | |
| 860 | 871 | * |
| 861 | − * Return: NULL if the address is not part of the shared section, non-NULL otherwise. | |
| 872 | + * Return: NULL if the object is not part of the shared section, non-NULL otherwise. | |
| 862 | 873 | */ |
| 863 | −static void *iface_fw_to_cpu_addr(struct panthor_device *ptdev, u32 mcu_va) | |
| 874 | +static void *iface_fw_to_cpu_addr(struct panthor_device *ptdev, u32 mcu_va, size_t size) | |
| 864 | 875 | { |
| 865 | 876 | u64 shared_mem_start = panthor_kernel_bo_gpuva(ptdev->fw->shared_section->mem); |
| 866 | − u64 shared_mem_end = shared_mem_start + | |
| 867 | − panthor_kernel_bo_size(ptdev->fw->shared_section->mem); | |
| 868 | − if (mcu_va < shared_mem_start || mcu_va >= shared_mem_end) | |
| 877 | + size_t shared_mem_size = panthor_kernel_bo_size(ptdev->fw->shared_section->mem); | |
| 878 | + u64 offset; | |
| 879 | + | |
| 880 | + if (mcu_va < shared_mem_start) | |
| 881 | + return NULL; | |
| 882 | + | |
| 883 | + offset = mcu_va - shared_mem_start; | |
| 884 | + if (offset > shared_mem_size || size > shared_mem_size - offset) | |
| 869 | 885 | return NULL; |
| 870 | 886 | |
| 871 | − return ptdev->fw->shared_section->mem->kmap + (mcu_va - shared_mem_start); | |
| 887 | + return ptdev->fw->shared_section->mem->kmap + offset; | |
| 872 | 888 | } |
| 873 | 889 | |
| 874 | 890 | static int panthor_init_cs_iface(struct panthor_device *ptdev, |
| @@ -890,8 +906,10 @@ static int panthor_init_cs_iface(struct panthor_device *ptdev, · 15 unmodified lines | ||
| 890 | 906 | |
| 891 | 907 | spin_lock_init(&cs_iface->lock); |
| 892 | 908 | cs_iface->control = ptdev->fw->shared_section->mem->kmap + iface_offset; |
| 893 | − cs_iface->input = iface_fw_to_cpu_addr(ptdev, cs_iface->control->input_va); | |
| 894 | − cs_iface->output = iface_fw_to_cpu_addr(ptdev, cs_iface->control->output_va); | |
| 909 | + cs_iface->input = iface_fw_to_cpu_addr(ptdev, cs_iface->control->input_va, | |
| 910 | + sizeof(*cs_iface->input)); | |
| 911 | + cs_iface->output = iface_fw_to_cpu_addr(ptdev, cs_iface->control->output_va, | |
| 912 | + sizeof(*cs_iface->output)); | |
| 895 | 913 | |
| 896 | 914 | if (!cs_iface->input || !cs_iface->output) { |
| 897 | 915 | drm_err(&ptdev->base, "Invalid stream control interface input/output VA"); |
| @@ -941,8 +959,10 @@ static int panthor_init_csg_iface(struct panthor_device *ptdev, · 43 unmodified lines | ||
| 941 | 959 | |
| 942 | 960 | spin_lock_init(&csg_iface->lock); |
| 943 | 961 | csg_iface->control = ptdev->fw->shared_section->mem->kmap + iface_offset; |
| 944 | − csg_iface->input = iface_fw_to_cpu_addr(ptdev, csg_iface->control->input_va); | |
| 945 | − csg_iface->output = iface_fw_to_cpu_addr(ptdev, csg_iface->control->output_va); | |
| 962 | + csg_iface->input = iface_fw_to_cpu_addr(ptdev, csg_iface->control->input_va, | |
| 963 | + sizeof(*csg_iface->input)); | |
| 964 | + csg_iface->output = iface_fw_to_cpu_addr(ptdev, csg_iface->control->output_va, | |
| 965 | + sizeof(*csg_iface->output)); | |
| 946 | 966 | |
| 947 | 967 | if (csg_iface->control->stream_num < MIN_CS_PER_CSG || |
| 948 | 968 | csg_iface->control->stream_num > MAX_CS_PER_CSG) |
| @@ -999,8 +1019,10 @@ static int panthor_fw_init_ifaces(struct panthor_device *ptdev) · 50 unmodified lines | ||
| 999 | 1019 | return -EINVAL; |
| 1000 | 1020 | } |
| 1001 | 1021 | |
| 1002 | − glb_iface->input = iface_fw_to_cpu_addr(ptdev, glb_iface->control->input_va); | |
| 1003 | − glb_iface->output = iface_fw_to_cpu_addr(ptdev, glb_iface->control->output_va); | |
| 1022 | + glb_iface->input = iface_fw_to_cpu_addr(ptdev, glb_iface->control->input_va, | |
| 1023 | + sizeof(*glb_iface->input)); | |
| 1024 | + glb_iface->output = iface_fw_to_cpu_addr(ptdev, glb_iface->control->output_va, | |
| 1025 | + sizeof(*glb_iface->output)); | |
| 1004 | 1026 | if (!glb_iface->input || !glb_iface->output) { |
| 1005 | 1027 | drm_err(&ptdev->base, "Invalid global control interface input/output VA"); |
| 1006 | 1028 | return -EINVAL; |
| @@ -802,9 +802,13 @@ static vm_fault_t insert_page(struct vm_fault *vmf, unsigned int order, struct p · 801 unmodified lines | ||
| 802 | 802 | } else if (order == PMD_ORDER) { |
| 803 | 803 | unsigned long pfn = page_to_pfn(page); |
| 804 | 804 | unsigned long paddr = pfn << PAGE_SHIFT; |
| 805 | + struct vm_area_struct *vma = vmf->vma; | |
| 806 | + unsigned long start = ALIGN_DOWN(vmf->address, PMD_SIZE); | |
| 807 | + unsigned long end = start + PMD_SIZE; | |
| 808 | + bool in_range = vma->vm_start <= start && end <= vma->vm_end; | |
| 805 | 809 | bool aligned = (vmf->address & ~PMD_MASK) == (paddr & ~PMD_MASK); |
| 806 | 810 | |
| 807 | − if (aligned && | |
| 811 | + if (aligned && in_range && | |
| 808 | 812 | folio_test_pmd_mappable(page_folio(page))) { |
| 809 | 813 | pfn &= PMD_MASK >> PAGE_SHIFT; |
| 810 | 814 | return vmf_insert_pfn_pmd(vmf, pfn, vmf->flags & FAULT_FLAG_WRITE); |
| @@ -360,6 +360,13 @@ static bool radeon_fence_is_signaled(struct dma_fence *f) · 359 unmodified lines | ||
| 360 | 360 | if (atomic64_read(&rdev->fence_drv[ring].last_seq) >= seq) |
| 361 | 361 | return true; |
| 362 | 362 | |
| 363 | + if (down_read_trylock(&rdev->exclusive_lock)) { | |
| 364 | + radeon_fence_activity(rdev, ring); | |
| 365 | + up_read(&rdev->exclusive_lock); | |
| 366 | + | |
| 367 | + if (atomic64_read(&rdev->fence_drv[ring].last_seq) >= seq) | |
| 368 | + return true; | |
| 369 | + } | |
| 363 | 370 | return false; |
| 364 | 371 | } |
| 365 | 372 | |
| @@ -182,6 +182,12 @@ struct v3d_dev { · 181 unmodified lines | ||
| 182 | 182 | */ |
| 183 | 183 | struct mutex reset_lock; |
| 184 | 184 | |
| 185 | + /* Ordered workqueue shared by every queue's scheduler timeout work. | |
| 186 | + * V3D reset is global to all queues, so the timeout handlers must not | |
| 187 | + * run concurrently. | |
| 188 | + */ | |
| 189 | + struct workqueue_struct *reset_wq; | |
| 190 | + | |
| 185 | 191 | /* Lock taken when creating and pushing the GPU scheduler |
| 186 | 192 | * jobs, to keep the sched-fence seqnos in order. |
| 187 | 193 | */ |
| @@ -837,6 +837,7 @@ v3d_queue_sched_init(struct v3d_dev *v3d, const struct drm_sched_backend_ops *op · 836 unmodified lines | ||
| 837 | 837 | struct drm_sched_init_args args = { |
| 838 | 838 | .credit_limit = 1, |
| 839 | 839 | .timeout = msecs_to_jiffies(500), |
| 840 | + .timeout_wq = v3d->reset_wq, | |
| 840 | 841 | .dev = v3d->drm.dev, |
| 841 | 842 | }; |
| 842 | 843 | |
| @@ -851,9 +852,13 @@ v3d_sched_init(struct v3d_dev *v3d) · 8 unmodified lines | ||
| 851 | 852 | { |
| 852 | 853 | int ret; |
| 853 | 854 | |
| 855 | + v3d->reset_wq = alloc_ordered_workqueue("v3d_reset", 0); | |
| 856 | + if (!v3d->reset_wq) | |
| 857 | + return -ENOMEM; | |
| 858 | + | |
| 854 | 859 | ret = v3d_queue_sched_init(v3d, &v3d_bin_sched_ops, V3D_BIN, "v3d_bin"); |
| 855 | 860 | if (ret) |
| 856 | − return ret; | |
| 861 | + goto fail; | |
| 857 | 862 | |
| 858 | 863 | ret = v3d_queue_sched_init(v3d, &v3d_render_sched_ops, V3D_RENDER, |
| 859 | 864 | "v3d_render"); |
| @@ -896,4 +901,6 @@ v3d_sched_fini(struct v3d_dev *v3d) · 36 unmodified lines | ||
| 896 | 901 | if (v3d->queue[q].sched.ready) |
| 897 | 902 | drm_sched_fini(&v3d->queue[q].sched); |
| 898 | 903 | } |
| 904 | + | |
| 905 | + destroy_workqueue(v3d->reset_wq); | |
| 899 | 906 | } |
| @@ -104,7 +104,7 @@ vc4_overflow_mem_work(struct work_struct *work) · 103 unmodified lines | ||
| 104 | 104 | vc4->bin_alloc_overflow = BIT(bin_bo_slot); |
| 105 | 105 | |
| 106 | 106 | V3D_WRITE(V3D_BPOA, bo->base.dma_addr + bin_bo_slot * vc4->bin_alloc_size); |
| 107 | − V3D_WRITE(V3D_BPOS, bo->base.base.size); | |
| 107 | + V3D_WRITE(V3D_BPOS, vc4->bin_alloc_size); | |
| 108 | 108 | V3D_WRITE(V3D_INTCTL, V3D_INT_OUTOMEM); |
| 109 | 109 | V3D_WRITE(V3D_INTENA, V3D_INT_OUTOMEM); |
| 110 | 110 | spin_unlock_irqrestore(&vc4->job_lock, irqflags); |
| @@ -385,6 +385,23 @@ validate_tile_binning_config(VALIDATE_ARGS) · 384 unmodified lines | ||
| 385 | 385 | return -EINVAL; |
| 386 | 386 | } |
| 387 | 387 | |
| 388 | + /* The tile state data array is 48 bytes per tile, and we put it at | |
| 389 | + * the start of a BO containing both it and the tile alloc. | |
| 390 | + */ | |
| 391 | + tile_state_size = 48 * tile_count; | |
| 392 | + | |
| 393 | + /* Since the tile alloc array will follow us, align. */ | |
| 394 | + tile_state_size = roundup(tile_state_size, 4096); | |
| 395 | + | |
| 396 | + /* Reject configurations whose tile state would leave no room for | |
| 397 | + * the tile alloc pool that follows it in the slot. | |
| 398 | + */ | |
| 399 | + if (tile_state_size >= vc4->bin_alloc_size) { | |
| 400 | + DRM_DEBUG("Tile binning config of %dx%d too large\n", | |
| 401 | + exec->bin_tiles_x, exec->bin_tiles_y); | |
| 402 | + return -EINVAL; | |
| 403 | + } | |
| 404 | + | |
| 388 | 405 | bin_slot = vc4_v3d_get_bin_slot(vc4); |
| 389 | 406 | if (bin_slot < 0) { |
| 390 | 407 | if (bin_slot != -EINTR && bin_slot != -ERESTARTSYS) { |
| @@ -400,13 +417,13 @@ validate_tile_binning_config(VALIDATE_ARGS) · 9 unmodified lines | ||
| 400 | 417 | exec->bin_slots |= BIT(bin_slot); |
| 401 | 418 | bin_addr = vc4->bin_bo->base.dma_addr + bin_slot * vc4->bin_alloc_size; |
| 402 | 419 | |
| 403 | − /* The tile state data array is 48 bytes per tile, and we put it at | |
| 404 | − * the start of a BO containing both it and the tile alloc. | |
| 405 | − */ | |
| 406 | − tile_state_size = 48 * tile_count; | |
| 420 | + exec->tile_alloc_offset = bin_addr + tile_state_size; | |
| 407 | 421 | |
| 408 | − /* Since the tile alloc array will follow us, align. */ | |
| 409 | − exec->tile_alloc_offset = bin_addr + roundup(tile_state_size, 4096); | |
| 422 | + /* The TSDA area must be zeroed out before use, otherwise the PTB might | |
| 423 | + * consume a stale tile state. | |
| 424 | + */ | |
| 425 | + memset(vc4->bin_bo->base.vaddr + bin_slot * vc4->bin_alloc_size, 0, | |
| 426 | + tile_state_size); | |
| 410 | 427 | |
| 411 | 428 | *(uint8_t *)(validated + 14) = |
| 412 | 429 | ((flags & ~(VC4_BIN_CONFIG_ALLOC_INIT_BLOCK_SIZE_MASK | |
| @@ -547,14 +547,17 @@ int ttm_prime_fd_to_handle(struct ttm_object_file *tfile, · 546 unmodified lines | ||
| 547 | 547 | if (IS_ERR(dma_buf)) |
| 548 | 548 | return PTR_ERR(dma_buf); |
| 549 | 549 | |
| 550 | − if (dma_buf->ops != &tdev->ops) | |
| 551 | − return -ENOSYS; | |
| 550 | + if (dma_buf->ops != &tdev->ops) { | |
| 551 | + ret = -ENOSYS; | |
| 552 | + goto out; | |
| 553 | + } | |
| 552 | 554 | |
| 553 | 555 | prime = (struct ttm_prime_object *) dma_buf->priv; |
| 554 | 556 | base = &prime->base; |
| 555 | 557 | *handle = base->handle; |
| 556 | 558 | ret = ttm_ref_object_add(tfile, base, NULL, false); |
| 557 | 559 | |
| 560 | +out: | |
| 558 | 561 | dma_buf_put(dma_buf); |
| 559 | 562 | |
| 560 | 563 | return ret; |
| @@ -30,6 +30,7 @@ · 29 unmodified lines | ||
| 30 | 30 | |
| 31 | 31 | #include "vmwgfx_bo.h" |
| 32 | 32 | #include <linux/highmem.h> |
| 33 | +#include <linux/overflow.h> | |
| 33 | 34 | |
| 34 | 35 | /* |
| 35 | 36 | * Template that implements find_first_diff() for a generic |
| @@ -463,19 +464,42 @@ static int vmw_external_bo_copy(struct vmw_bo *dst, u32 dst_offset, · 427 unmodified lines | ||
| 463 | 464 | container_of(dst->tbo.bdev, struct vmw_private, bdev); |
| 464 | 465 | size_t dst_size = dst->tbo.resource->size; |
| 465 | 466 | size_t src_size = src->tbo.resource->size; |
| 467 | + size_t dst_end, src_end; | |
| 466 | 468 | struct iosys_map dst_map = {0}; |
| 467 | 469 | struct iosys_map src_map = {0}; |
| 470 | + bool dst_mapped = false; | |
| 471 | + bool src_mapped = false; | |
| 468 | 472 | int ret, i; |
| 469 | 473 | int x_in_bytes; |
| 470 | 474 | u8 *vsrc; |
| 471 | 475 | u8 *vdst; |
| 472 | 476 | |
| 477 | + if (!height || !width_in_bytes) | |
| 478 | + return 0; | |
| 479 | + | |
| 480 | + if (!dst_stride || !src_stride) | |
| 481 | + return -EINVAL; | |
| 482 | + if (dst_stride < width_in_bytes || src_stride < width_in_bytes) | |
| 483 | + return -EINVAL; | |
| 484 | + if (check_mul_overflow((size_t)dst_stride, (size_t)height - 1, &dst_end) || | |
| 485 | + check_add_overflow(dst_end, (size_t)width_in_bytes, &dst_end) || | |
| 486 | + check_add_overflow((size_t)dst_offset, dst_end, &dst_end) || | |
| 487 | + dst_end > dst_size || | |
| 488 | + check_mul_overflow((size_t)src_stride, (size_t)height - 1, &src_end) || | |
| 489 | + check_add_overflow(src_end, (size_t)width_in_bytes, &src_end) || | |
| 490 | + check_add_overflow((size_t)src_offset, src_end, &src_end) || | |
| 491 | + src_end > src_size) { | |
| 492 | + drm_dbg_driver(&vmw->drm, "Out-of-bounds external BO copy\n"); | |
| 493 | + return -EINVAL; | |
| 494 | + } | |
| 495 | + | |
| 473 | 496 | vsrc = map_external(src, &src_map); |
| 474 | 497 | if (!vsrc) { |
| 475 | 498 | drm_dbg_driver(&vmw->drm, "Wasn't able to map src\n"); |
| 476 | 499 | ret = -ENOMEM; |
| 477 | 500 | goto out; |
| 478 | 501 | } |
| 502 | + src_mapped = true; | |
| 479 | 503 | |
| 480 | 504 | vdst = map_external(dst, &dst_map); |
| 481 | 505 | if (!vdst) { |
| @@ -483,16 +507,13 @@ static int vmw_external_bo_copy(struct vmw_bo *dst, u32 dst_offset, · 1 unmodified line | ||
| 483 | 507 | ret = -ENOMEM; |
| 484 | 508 | goto out; |
| 485 | 509 | } |
| 510 | + dst_mapped = true; | |
| 486 | 511 | |
| 487 | 512 | vsrc += src_offset; |
| 488 | 513 | vdst += dst_offset; |
| 489 | − if (src_stride == dst_stride) { | |
| 490 | − dst_size -= dst_offset; | |
| 491 | − src_size -= src_offset; | |
| 492 | − memcpy(vdst, vsrc, | |
| 493 | − min(dst_stride * height, min(dst_size, src_size))); | |
| 514 | + if (src_stride == dst_stride && width_in_bytes == dst_stride) { | |
| 515 | + memcpy(vdst, vsrc, dst_stride * (size_t)height); | |
| 494 | 516 | } else { |
| 495 | − WARN_ON(dst_stride < width_in_bytes); | |
| 496 | 517 | for (i = 0; i < height; ++i) { |
| 497 | 518 | memcpy(vdst, vsrc, width_in_bytes); |
| 498 | 519 | vsrc += src_stride; |
| @@ -508,8 +529,10 @@ static int vmw_external_bo_copy(struct vmw_bo *dst, u32 dst_offset, · 9 unmodified lines | ||
| 508 | 529 | |
| 509 | 530 | ret = 0; |
| 510 | 531 | out: |
| 511 | − unmap_external(src, &src_map); | |
| 512 | − unmap_external(dst, &dst_map); | |
| 532 | + if (src_mapped) | |
| 533 | + unmap_external(src, &src_map); | |
| 534 | + if (dst_mapped) | |
| 535 | + unmap_external(dst, &dst_map); | |
| 513 | 536 | |
| 514 | 537 | return ret; |
| 515 | 538 | } |
| @@ -432,6 +432,7 @@ vmw_cursor_mob_map(struct vmw_plane_state *vps) · 431 unmodified lines | ||
| 432 | 432 | u32 size = vmw_cursor_mob_size(vps->cursor.update_type, |
| 433 | 433 | vps->base.crtc_w, vps->base.crtc_h); |
| 434 | 434 | struct vmw_bo *vbo = vps->cursor.mob; |
| 435 | + void *map; | |
| 435 | 436 | |
| 436 | 437 | if (!vbo) |
| 437 | 438 | return -EINVAL; |
| @@ -446,11 +447,15 @@ vmw_cursor_mob_map(struct vmw_plane_state *vps) · 8 unmodified lines | ||
| 446 | 447 | if (unlikely(ret != 0)) |
| 447 | 448 | return -ENOMEM; |
| 448 | 449 | |
| 449 | − vmw_bo_map_and_cache(vbo); | |
| 450 | + map = vmw_bo_map_and_cache(vbo); | |
| 451 | + if (!map) { | |
| 452 | + vmw_bo_unmap(vbo); | |
| 453 | + ret = -ENOMEM; | |
| 454 | + } | |
| 450 | 455 | |
| 451 | 456 | ttm_bo_unreserve(&vbo->tbo); |
| 452 | 457 | |
| 453 | − return 0; | |
| 458 | + return ret; | |
| 454 | 459 | } |
| 455 | 460 | |
| 456 | 461 | /** |
| @@ -663,9 +668,15 @@ int vmw_cursor_plane_prepare_fb(struct drm_plane *plane, · 206 unmodified lines | ||
| 663 | 668 | !vmw_cursor_buffer_changed(vps, old_vps)) { |
| 664 | 669 | vps->cursor.update_type = |
| 665 | 670 | VMW_CURSOR_UPDATE_NONE; |
| 666 | − } else { | |
| 667 | − vmw_cursor_mob_get(vcp, vps); | |
| 668 | − vmw_cursor_mob_map(vps); | |
| 671 | + } else if (vps->cursor.update_type == | |
| 672 | + VMW_CURSOR_UPDATE_MOB && | |
| 673 | + (vmw_cursor_mob_get(vcp, vps) || | |
| 674 | + vmw_cursor_mob_map(vps))) { | |
| 675 | + /* | |
| 676 | + * Reset the cursor to avoid crashes later. | |
| 677 | + */ | |
| 678 | + vps->cursor.update_type = | |
| 679 | + VMW_CURSOR_UPDATE_NONE; | |
| 669 | 680 | } |
| 670 | 681 | } |
| 671 | 682 | } |
| @@ -732,6 +743,34 @@ int vmw_cursor_plane_atomic_check(struct drm_plane *plane, · 60 unmodified lines | ||
| 732 | 743 | "surface not suitable for cursor\n"); |
| 733 | 744 | return -EINVAL; |
| 734 | 745 | } |
| 746 | + } else if (update_type == VMW_CURSOR_UPDATE_GB_ONLY || | |
| 747 | + update_type == VMW_CURSOR_UPDATE_MOB) { | |
| 748 | + u32 cursor_max_dim = | |
| 749 | + vmw_read(vmw, SVGA_REG_CURSOR_MAX_DIMENSION); | |
| 750 | + | |
| 751 | + if (new_state->crtc_w > cursor_max_dim || | |
| 752 | + new_state->crtc_h > cursor_max_dim) { | |
| 753 | + drm_warn(&vmw->drm, | |
| 754 | + "Cursor dimensions (%d, %d) exceed device max %u\n", | |
| 755 | + new_state->crtc_w, new_state->crtc_h, | |
| 756 | + cursor_max_dim); | |
| 757 | + return -EINVAL; | |
| 758 | + } | |
| 759 | + | |
| 760 | + if (update_type == VMW_CURSOR_UPDATE_MOB) { | |
| 761 | + u32 mob_max_size = | |
| 762 | + vmw_read(vmw, SVGA_REG_MOB_MAX_SIZE); | |
| 763 | + u64 mob_size = (u64)new_state->crtc_w * | |
| 764 | + new_state->crtc_h * sizeof(u32) + | |
| 765 | + sizeof(SVGAGBCursorHeader); | |
| 766 | + | |
| 767 | + if (mob_size > mob_max_size) { | |
| 768 | + drm_warn(&vmw->drm, | |
| 769 | + "Cursor MOB size %llu exceeds device max %u\n", | |
| 770 | + mob_size, mob_max_size); | |
| 771 | + return -EINVAL; | |
| 772 | + } | |
| 773 | + } | |
| 735 | 774 | } |
| 736 | 775 | |
| 737 | 776 | return 0; |
| @@ -1272,9 +1272,13 @@ static int vmw_cmd_dx_bind_query(struct vmw_private *dev_priv, · 1271 unmodified lines | ||
| 1272 | 1272 | SVGA3dCmdHeader *header) |
| 1273 | 1273 | { |
| 1274 | 1274 | VMW_DECLARE_CMD_VAR(*cmd, SVGA3dCmdDXBindQuery); |
| 1275 | + struct vmw_ctx_validation_info *ctx_node = VMW_GET_CTX_NODE(sw_context); | |
| 1275 | 1276 | struct vmw_bo *vmw_bo; |
| 1276 | 1277 | int ret; |
| 1277 | 1278 | |
| 1279 | + if (!ctx_node) | |
| 1280 | + return -EINVAL; | |
| 1281 | + | |
| 1278 | 1282 | cmd = container_of(header, typeof(*cmd), header); |
| 1279 | 1283 | |
| 1280 | 1284 | /* |
| @@ -1288,7 +1292,7 @@ static int vmw_cmd_dx_bind_query(struct vmw_private *dev_priv, · 7 unmodified lines | ||
| 1288 | 1292 | return ret; |
| 1289 | 1293 | |
| 1290 | 1294 | sw_context->dx_query_mob = vmw_bo; |
| 1291 | − sw_context->dx_query_ctx = sw_context->dx_ctx_node->ctx; | |
| 1295 | + sw_context->dx_query_ctx = ctx_node->ctx; | |
| 1292 | 1296 | return 0; |
| 1293 | 1297 | } |
| 1294 | 1298 | |
| @@ -1506,6 +1510,12 @@ static int vmw_cmd_dma(struct vmw_private *dev_priv, · 211 unmodified lines | ||
| 1506 | 1510 | bool dirty; |
| 1507 | 1511 | |
| 1508 | 1512 | cmd = container_of(header, typeof(*cmd), header); |
| 1513 | + | |
| 1514 | + if (unlikely(header->size < sizeof(cmd->body) + sizeof(*suffix))) { | |
| 1515 | + VMW_DEBUG_USER("Illegal SVGA_3D_CMD_SURFACE_DMA size.\n"); | |
| 1516 | + return -EINVAL; | |
| 1517 | + } | |
| 1518 | + | |
| 1509 | 1519 | suffix = (SVGA3dCmdSurfaceDMASuffix *)((unsigned long) &cmd->body + |
| 1510 | 1520 | header->size - sizeof(*suffix)); |
| 1511 | 1521 | |
| @@ -1567,11 +1577,17 @@ static int vmw_cmd_draw(struct vmw_private *dev_priv, · 55 unmodified lines | ||
| 1567 | 1577 | uint32_t maxnum; |
| 1568 | 1578 | int ret; |
| 1569 | 1579 | |
| 1580 | + cmd = container_of(header, typeof(*cmd), header); | |
| 1581 | + | |
| 1582 | + if (unlikely(header->size < sizeof(cmd->body))) { | |
| 1583 | + VMW_DEBUG_USER("Illegal DRAW_PRIMITIVES header size.\n"); | |
| 1584 | + return -EINVAL; | |
| 1585 | + } | |
| 1586 | + | |
| 1570 | 1587 | ret = vmw_cmd_cid_check(dev_priv, sw_context, header); |
| 1571 | 1588 | if (unlikely(ret != 0)) |
| 1572 | 1589 | return ret; |
| 1573 | 1590 | |
| 1574 | − cmd = container_of(header, typeof(*cmd), header); | |
| 1575 | 1591 | maxnum = (header->size - sizeof(cmd->body)) / sizeof(*decl); |
| 1576 | 1592 | |
| 1577 | 1593 | if (unlikely(cmd->body.numVertexDecls > maxnum)) { |
| @@ -367,13 +367,24 @@ void vmw_fence_fifo_down(struct vmw_fence_manager *fman) · 366 unmodified lines | ||
| 367 | 367 | ret = vmw_fence_obj_wait(fence, false, false, |
| 368 | 368 | VMW_FENCE_WAIT_TIMEOUT); |
| 369 | 369 | |
| 370 | + spin_lock(&fman->lock); | |
| 370 | 371 | if (unlikely(ret != 0)) { |
| 372 | + bool cookie = dma_fence_begin_signalling(); | |
| 373 | + | |
| 371 | 374 | list_del_init(&fence->head); |
| 372 | − dma_fence_signal(&fence->base); | |
| 375 | + if (fence->waiter_added) { | |
| 376 | + vmw_seqno_waiter_remove(fman->dev_priv); | |
| 377 | + fence->waiter_added = false; | |
| 378 | + } | |
| 379 | + dma_fence_signal_locked(&fence->base); | |
| 380 | + dma_fence_end_signalling(cookie); | |
| 373 | 381 | } |
| 374 | 382 | |
| 375 | 383 | BUG_ON(!list_empty(&fence->head)); |
| 384 | + spin_unlock(&fman->lock); | |
| 385 | + | |
| 376 | 386 | dma_fence_put(&fence->base); |
| 387 | + | |
| 377 | 388 | spin_lock(&fman->lock); |
| 378 | 389 | } |
| 379 | 390 | spin_unlock(&fman->lock); |
| @@ -311,7 +311,7 @@ void vmw_bo_dirty_transfer_to_res(struct vmw_resource *res) · 310 unmodified lines | ||
| 311 | 311 | return; |
| 312 | 312 | |
| 313 | 313 | cur = max(res_start, dirty->start); |
| 314 | − res_end = max(res_end, dirty->end); | |
| 314 | + res_end = min(res_end, dirty->end); | |
| 315 | 315 | while (cur < res_end) { |
| 316 | 316 | unsigned long num; |
| 317 | 317 | |
| @@ -347,7 +347,7 @@ void vmw_bo_dirty_clear(struct vmw_bo *vbo) · 29 unmodified lines | ||
| 347 | 347 | return; |
| 348 | 348 | |
| 349 | 349 | cur = max(res_start, dirty->start); |
| 350 | − res_end = max(res_end, dirty->end); | |
| 350 | + res_end = min(res_end, dirty->end); | |
| 351 | 351 | while (cur < res_end) { |
| 352 | 352 | unsigned long num; |
| 353 | 353 | |
| @@ -136,7 +136,7 @@ static void vmw_resource_release(struct kref *kref) · 135 unmodified lines | ||
| 136 | 136 | val_buf.num_shared = 0; |
| 137 | 137 | res->func->unbind(res, false, &val_buf); |
| 138 | 138 | } |
| 139 | − res->guest_memory_size = false; | |
| 139 | + res->guest_memory_dirty = false; | |
| 140 | 140 | vmw_resource_mob_detach(res); |
| 141 | 141 | if (res->dirty) |
| 142 | 142 | res->func->dirty_free(res); |
| @@ -773,7 +773,7 @@ void vmw_resource_unbind_list(struct vmw_bo *vbo) · 630 unmodified lines | ||
| 773 | 773 | if (!WARN_ON_ONCE(!res->func->unbind)) |
| 774 | 774 | (void) res->func->unbind(res, res->res_dirty, &val_buf); |
| 775 | 775 | |
| 776 | − res->guest_memory_size = true; | |
| 776 | + res->guest_memory_dirty = true; | |
| 777 | 777 | res->res_dirty = false; |
| 778 | 778 | vmw_resource_mob_detach(res); |
| 779 | 779 | } |
| @@ -25,6 +25,8 @@ · 24 unmodified lines | ||
| 25 | 25 | * |
| 26 | 26 | **************************************************************************/ |
| 27 | 27 | |
| 28 | +#include <linux/overflow.h> | |
| 29 | + | |
| 28 | 30 | #include <drm/ttm/ttm_placement.h> |
| 29 | 31 | |
| 30 | 32 | #include "vmwgfx_binding.h" |
| @@ -685,7 +687,7 @@ int vmw_shader_destroy_ioctl(struct drm_device *dev, void *data, · 654 unmodified lines | ||
| 685 | 687 | static int vmw_user_shader_alloc(struct vmw_private *dev_priv, |
| 686 | 688 | struct vmw_bo *buffer, |
| 687 | 689 | size_t shader_size, |
| 688 | − size_t offset, | |
| 690 | + u64 offset, | |
| 689 | 691 | SVGA3dShaderType shader_type, |
| 690 | 692 | uint8_t num_input_sig, |
| 691 | 693 | uint8_t num_output_sig, |
| @@ -739,7 +741,7 @@ out: · 47 unmodified lines | ||
| 739 | 741 | static struct vmw_resource *vmw_shader_alloc(struct vmw_private *dev_priv, |
| 740 | 742 | struct vmw_bo *buffer, |
| 741 | 743 | size_t shader_size, |
| 742 | − size_t offset, | |
| 744 | + u64 offset, | |
| 743 | 745 | SVGA3dShaderType shader_type) |
| 744 | 746 | { |
| 745 | 747 | struct vmw_shader *shader; |
| @@ -768,7 +770,7 @@ out_err: · 22 unmodified lines | ||
| 768 | 770 | |
| 769 | 771 | static int vmw_shader_define(struct drm_device *dev, struct drm_file *file_priv, |
| 770 | 772 | enum drm_vmw_shader_type shader_type_drm, |
| 771 | − u32 buffer_handle, size_t size, size_t offset, | |
| 773 | + u32 buffer_handle, size_t size, u64 offset, | |
| 772 | 774 | uint8_t num_input_sig, uint8_t num_output_sig, |
| 773 | 775 | uint32_t *shader_handle) |
| 774 | 776 | { |
| @@ -779,13 +781,16 @@ static int vmw_shader_define(struct drm_device *dev, struct drm_file *file_priv, · 4 unmodified lines | ||
| 779 | 781 | int ret; |
| 780 | 782 | |
| 781 | 783 | if (buffer_handle != SVGA3D_INVALID_ID) { |
| 784 | + u64 end; | |
| 785 | + | |
| 782 | 786 | ret = vmw_user_bo_lookup(file_priv, buffer_handle, &buffer); |
| 783 | 787 | if (unlikely(ret != 0)) { |
| 784 | 788 | VMW_DEBUG_USER("Couldn't find buffer for shader creation.\n"); |
| 785 | 789 | return ret; |
| 786 | 790 | } |
| 787 | 791 | |
| 788 | − if ((u64)buffer->tbo.base.size < (u64)size + (u64)offset) { | |
| 792 | + if (check_add_overflow((u64)size, (u64)offset, &end) || | |
| 793 | + end > buffer->tbo.base.size) { | |
| 789 | 794 | VMW_DEBUG_USER("Illegal buffer- or shader size.\n"); |
| 790 | 795 | ret = -EINVAL; |
| 791 | 796 | goto out_bad_arg; |
| @@ -309,7 +309,8 @@ int vmw_validation_add_resource(struct vmw_validation_context *ctx, · 308 unmodified lines | ||
| 309 | 309 | } |
| 310 | 310 | node->res = vmw_resource_reference_unless_doomed(res); |
| 311 | 311 | if (!node->res) { |
| 312 | − hash_del_rcu(&node->hash.head); | |
| 312 | + if (ctx->sw_context) | |
| 313 | + hash_del_rcu(&node->hash.head); | |
| 313 | 314 | return -ESRCH; |
| 314 | 315 | } |
| 315 | 316 | |
| @@ -206,14 +206,14 @@ vmw_vkms_init(struct vmw_private *vmw) · 205 unmodified lines | ||
| 206 | 206 | vmw->vkms_enabled = false; |
| 207 | 207 | |
| 208 | 208 | ret = vmw_host_get_guestinfo(GUESTINFO_VBLANK, buffer, &buf_len); |
| 209 | − if (ret || buf_len > max_buf_len) | |
| 210 | − return; | |
| 211 | − buffer[buf_len] = '\0'; | |
| 209 | + if (!ret && buf_len <= max_buf_len) { | |
| 210 | + buffer[buf_len] = '\0'; | |
| 212 | 211 | |
| 213 | − ret = kstrtobool(buffer, &vmw->vkms_enabled); | |
| 214 | − if (!ret && vmw->vkms_enabled) { | |
| 215 | − ret = drm_vblank_init(&vmw->drm, VMWGFX_NUM_DISPLAY_UNITS); | |
| 216 | − vmw->vkms_enabled = (ret == 0); | |
| 212 | + ret = kstrtobool(buffer, &vmw->vkms_enabled); | |
| 213 | + if (!ret && vmw->vkms_enabled) { | |
| 214 | + ret = drm_vblank_init(&vmw->drm, VMWGFX_NUM_DISPLAY_UNITS); | |
| 215 | + vmw->vkms_enabled = (ret == 0); | |
| 216 | + } | |
| 217 | 217 | } |
| 218 | 218 | |
| 219 | 219 | vmw->crc_workq = alloc_ordered_workqueue("vmwgfx_crc_generator", 0); |
| @@ -228,7 +228,8 @@ vmw_vkms_init(struct vmw_private *vmw) · 8 unmodified lines | ||
| 228 | 228 | void |
| 229 | 229 | vmw_vkms_cleanup(struct vmw_private *vmw) |
| 230 | 230 | { |
| 231 | − destroy_workqueue(vmw->crc_workq); | |
| 231 | + if (vmw->crc_workq) | |
| 232 | + destroy_workqueue(vmw->crc_workq); | |
| 232 | 233 | } |
| 233 | 234 | |
| 234 | 235 | bool |
| @@ -805,6 +805,9 @@ static int exec_queue_set_hang_replay_state(struct xe_device *xe, · 804 unmodified lines | ||
| 805 | 805 | u64 __user *address = u64_to_user_ptr(value); |
| 806 | 806 | void *ptr; |
| 807 | 807 | |
| 808 | + if (q->replay_state) | |
| 809 | + return -EINVAL; | |
| 810 | + | |
| 808 | 811 | ptr = vmemdup_user(address, size); |
| 809 | 812 | if (XE_IOCTL_DBG(xe, IS_ERR(ptr))) |
| 810 | 813 | return PTR_ERR(ptr); |
| @@ -443,10 +443,6 @@ static bool xe_pt_hugepte_possible(u64 addr, u64 next, unsigned int level, · 442 unmodified lines | ||
| 443 | 443 | if (!xe_pt_covers(addr, next, level, &xe_walk->base)) |
| 444 | 444 | return false; |
| 445 | 445 | |
| 446 | − /* Does the DMA segment cover the whole pte? */ | |
| 447 | − if (next - xe_walk->va_curs_start > xe_walk->curs->size) | |
| 448 | − return false; | |
| 449 | − | |
| 450 | 446 | /* null VMA's and purged BO's do not have dma addresses */ |
| 451 | 447 | if (xe_vma_is_null(xe_walk->vma) || (bo && xe_bo_is_purged(bo))) |
| 452 | 448 | return true; |
| @@ -455,6 +451,10 @@ static bool xe_pt_hugepte_possible(u64 addr, u64 next, unsigned int level, · 2 unmodified lines | ||
| 455 | 451 | if (xe_walk->clear_pt) |
| 456 | 452 | return true; |
| 457 | 453 | |
| 454 | + /* Does the DMA segment cover the whole pte? */ | |
| 455 | + if (next - xe_walk->va_curs_start > xe_walk->curs->size) | |
| 456 | + return false; | |
| 457 | + | |
| 458 | 458 | /* Is the DMA address huge PTE size aligned? */ |
| 459 | 459 | size = next - addr; |
| 460 | 460 | dma = addr - xe_walk->va_curs_start + xe_res_dma(xe_walk->curs); |
| @@ -775,8 +775,11 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma, · 314 unmodified lines | ||
| 775 | 775 | } |
| 776 | 776 | |
| 777 | 777 | xe_walk.needs_64K = (vm->flags & XE_VM_FLAG_64K); |
| 778 | − if (clear_pt) | |
| 778 | + if (clear_pt) { | |
| 779 | + xe_assert(xe, !range); | |
| 780 | + curs.size = xe_vma_size(vma); | |
| 779 | 781 | goto walk_pt; |
| 782 | + } | |
| 780 | 783 | |
| 781 | 784 | if (vma->gpuva.flags & XE_VMA_ATOMIC_PTE_BIT) { |
| 782 | 785 | xe_walk.default_vram_pte = xe_atomic_for_vram(vm, vma) ? XE_USM_PPGTT_PTE_AE : 0; |
| @@ -15,6 +15,7 @@ · 14 unmodified lines | ||
| 15 | 15 | #include "xe_guc_pc.h" |
| 16 | 16 | #include "xe_guc_rc.h" |
| 17 | 17 | #include "xe_guc_engine_activity.h" |
| 18 | +#include "xe_guc_submit.h" | |
| 18 | 19 | #include "xe_huc.h" |
| 19 | 20 | #include "xe_sriov.h" |
| 20 | 21 | #include "xe_wopcm.h" |
| @@ -159,12 +160,14 @@ static int vf_uc_load_hw(struct xe_uc *uc) · 138 unmodified lines | ||
| 159 | 160 | if (err) |
| 160 | 161 | return err; |
| 161 | 162 | |
| 162 | − uc->guc.submission_state.enabled = true; | |
| 163 | − | |
| 164 | 163 | err = xe_guc_opt_in_features_enable(&uc->guc); |
| 165 | 164 | if (err) |
| 166 | 165 | return err; |
| 167 | 166 | |
| 167 | + err = xe_guc_submit_enable(&uc->guc); | |
| 168 | + if (err) | |
| 169 | + return err; | |
| 170 | + | |
| 168 | 171 | err = xe_gt_record_default_lrcs(uc_to_gt(uc)); |
| 169 | 172 | if (err) |
| 170 | 173 | return err; |
| @@ -72,7 +72,8 @@ module_param(max_version, uint, S_IRUGO); · 71 unmodified lines | ||
| 72 | 72 | MODULE_PARM_DESC(max_version, |
| 73 | 73 | "Maximal VMBus protocol version which can be negotiated"); |
| 74 | 74 | |
| 75 | −int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version) | |
| 75 | +static int vmbus_try_connection_id(struct vmbus_channel_msginfo *msginfo, | |
| 76 | + u32 version, u32 connection_id) | |
| 76 | 77 | { |
| 77 | 78 | int ret = 0; |
| 78 | 79 | struct vmbus_channel_initiate_contact *msg; |
| @@ -87,20 +88,20 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version) · 8 unmodified lines | ||
| 87 | 88 | msg->vmbus_version_requested = version; |
| 88 | 89 | |
| 89 | 90 | /* |
| 90 | − * VMBus protocol 5.0 (VERSION_WIN10_V5) and higher require that we must | |
| 91 | − * use VMBUS_MESSAGE_CONNECTION_ID_4 for the Initiate Contact Message, | |
| 92 | − * and for subsequent messages, we must use the Message Connection ID | |
| 93 | − * field in the host-returned Version Response Message. And, with | |
| 94 | − * VERSION_WIN10_V5 and higher, we don't use msg->interrupt_page, but we | |
| 95 | − * tell the host explicitly that we still use VMBUS_MESSAGE_SINT(2) for | |
| 96 | − * compatibility. | |
| 91 | + * For VMBus protocol 5.0 (VERSION_WIN10_V5) and higher, use the | |
| 92 | + * caller-supplied connection_id for the Initiate Contact message so | |
| 93 | + * the caller can implement the required retry scheme. For subsequent | |
| 94 | + * messages, use the Message Connection ID field in the host-returned | |
| 95 | + * Version Response message. With VERSION_WIN10_V5 and higher, we don't | |
| 96 | + * use msg->interrupt_page, but tell the host explicitly that we still | |
| 97 | + * use VMBUS_MESSAGE_SINT(2) for compatibility. | |
| 97 | 98 | * |
| 98 | 99 | * On old hosts, we should always use VMBUS_MESSAGE_CONNECTION_ID (1). |
| 99 | 100 | */ |
| 100 | 101 | if (version >= VERSION_WIN10_V5) { |
| 101 | 102 | msg->msg_sint = VMBUS_MESSAGE_SINT; |
| 102 | 103 | msg->msg_vtl = ms_hyperv.vtl; |
| 103 | − vmbus_connection.msg_conn_id = VMBUS_MESSAGE_CONNECTION_ID_4; | |
| 104 | + vmbus_connection.msg_conn_id = connection_id; | |
| 104 | 105 | } else { |
| 105 | 106 | msg->interrupt_page = virt_to_phys(vmbus_connection.int_page); |
| 106 | 107 | vmbus_connection.msg_conn_id = VMBUS_MESSAGE_CONNECTION_ID; |
| @@ -165,6 +166,22 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version) · 58 unmodified lines | ||
| 165 | 166 | return ret; |
| 166 | 167 | } |
| 167 | 168 | |
| 169 | +int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version) | |
| 170 | +{ | |
| 171 | + int ret; | |
| 172 | + | |
| 173 | + /* Try the redirect ID first for VTL2 with VMBus protocol 5.0+. */ | |
| 174 | + if (version >= VERSION_WIN10_V5 && ms_hyperv.vtl == 2) { | |
| 175 | + ret = vmbus_try_connection_id(msginfo, version, | |
| 176 | + VMBUS_MESSAGE_CONNECTION_ID_REDIRECT); | |
| 177 | + if (ret != -ENXIO) | |
| 178 | + return ret; | |
| 179 | + } | |
| 180 | + | |
| 181 | + return vmbus_try_connection_id(msginfo, version, | |
| 182 | + VMBUS_MESSAGE_CONNECTION_ID_4); | |
| 183 | +} | |
| 184 | + | |
| 168 | 185 | /* |
| 169 | 186 | * vmbus_connect - Sends a connect request on the partition service connection |
| 170 | 187 | */ |
| @@ -457,18 +474,10 @@ int vmbus_post_msg(void *buffer, size_t buflen, bool can_sleep) · 286 unmodified lines | ||
| 457 | 474 | |
| 458 | 475 | switch (ret) { |
| 459 | 476 | case HV_STATUS_INVALID_CONNECTION_ID: |
| 460 | − /* | |
| 461 | − * See vmbus_negotiate_version(): VMBus protocol 5.0 | |
| 462 | − * and higher require that we must use | |
| 463 | − * VMBUS_MESSAGE_CONNECTION_ID_4 for the Initiate | |
| 464 | − * Contact message, but on old hosts that only | |
| 465 | − * support VMBus protocol 4.0 or lower, here we get | |
| 466 | − * HV_STATUS_INVALID_CONNECTION_ID and we should | |
| 467 | − * return an error immediately without retrying. | |
| 468 | − */ | |
| 477 | + /* Allow INITIATE_CONTACT to try another connection ID. */ | |
| 469 | 478 | hdr = buffer; |
| 470 | 479 | if (hdr->msgtype == CHANNELMSG_INITIATE_CONTACT) |
| 471 | − return -EINVAL; | |
| 480 | + return -ENXIO; | |
| 472 | 481 | /* |
| 473 | 482 | * We could get this if we send messages too |
| 474 | 483 | * frequently. |
| @@ -110,6 +110,8 @@ struct hv_input_post_message { · 109 unmodified lines | ||
| 110 | 110 | enum { |
| 111 | 111 | VMBUS_MESSAGE_CONNECTION_ID = 1, |
| 112 | 112 | VMBUS_MESSAGE_CONNECTION_ID_4 = 4, |
| 113 | + /* VTL2 redirect connection ID for INITIATE_CONTACT. */ | |
| 114 | + VMBUS_MESSAGE_CONNECTION_ID_REDIRECT = 0x800074, | |
| 113 | 115 | VMBUS_MESSAGE_PORT_ID = 1, |
| 114 | 116 | VMBUS_EVENT_CONNECTION_ID = 2, |
| 115 | 117 | VMBUS_EVENT_PORT_ID = 2, |
| @@ -169,7 +169,14 @@ static int mshv_try_assert_irq_fast(struct mshv_irqfd *irqfd) · 168 unmodified lines | ||
| 169 | 169 | return -EOPNOTSUPP; |
| 170 | 170 | #endif |
| 171 | 171 | |
| 172 | − vp = partition->pt_vp_array[irq->lapic_apic_id]; | |
| 172 | + /* | |
| 173 | + * Pairs with smp_store_release() in mshv_partition_ioctl_create_vp(). | |
| 174 | + * MSHV_IRQFD does not require the target lapic_apic_id to refer to an | |
| 175 | + * existing VP, so this read can race a concurrent VP creation; the | |
| 176 | + * acquire ensures that a non-NULL pointer implies the VP's | |
| 177 | + * initialising stores are visible. | |
| 178 | + */ | |
| 179 | + vp = smp_load_acquire(&partition->pt_vp_array[irq->lapic_apic_id]); | |
| 173 | 180 | |
| 174 | 181 | if (!vp->vp_register_page) |
| 175 | 182 | return -EOPNOTSUPP; |
| @@ -284,7 +291,7 @@ static void mshv_irqfd_deactivate(struct mshv_irqfd *irqfd) · 108 unmodified lines | ||
| 284 | 291 | if (!mshv_irqfd_is_active(irqfd)) |
| 285 | 292 | return; |
| 286 | 293 | |
| 287 | − hlist_del(&irqfd->irqfd_hnode); | |
| 294 | + hlist_del_init(&irqfd->irqfd_hnode); | |
| 288 | 295 | |
| 289 | 296 | queue_work(irqfd_cleanup_wq, &irqfd->irqfd_shutdown); |
| 290 | 297 | } |
| @@ -473,6 +480,19 @@ static int mshv_irqfd_assign(struct mshv_partition *pt, · 182 unmodified lines | ||
| 473 | 480 | init_poll_funcptr(&irqfd->irqfd_polltbl, mshv_irqfd_queue_proc); |
| 474 | 481 | |
| 475 | 482 | spin_lock_irq(&pt->pt_irqfds_lock); |
| 483 | + ret = 0; | |
| 484 | + hlist_for_each_entry(tmp, &pt->pt_irqfds_list, irqfd_hnode) { | |
| 485 | + if (irqfd->irqfd_eventfd_ctx != tmp->irqfd_eventfd_ctx) | |
| 486 | + continue; | |
| 487 | + /* This fd is used for another irq already. */ | |
| 488 | + ret = -EBUSY; | |
| 489 | + spin_unlock_irq(&pt->pt_irqfds_lock); | |
| 490 | + goto fail; | |
| 491 | + } | |
| 492 | + | |
| 493 | + idx = srcu_read_lock(&pt->pt_irq_srcu); | |
| 494 | + mshv_irqfd_update(pt, irqfd); | |
| 495 | + | |
| 476 | 496 | #if IS_ENABLED(CONFIG_X86) |
| 477 | 497 | if (args->flags & BIT(MSHV_IRQFD_BIT_RESAMPLE) && |
| 478 | 498 | !irqfd->irqfd_lapic_irq.lapic_control.level_triggered) { |
| @@ -481,22 +501,12 @@ static int mshv_irqfd_assign(struct mshv_partition *pt, · 2 unmodified lines | ||
| 481 | 501 | * Otherwise return with failure |
| 482 | 502 | */ |
| 483 | 503 | spin_unlock_irq(&pt->pt_irqfds_lock); |
| 504 | + srcu_read_unlock(&pt->pt_irq_srcu, idx); | |
| 484 | 505 | ret = -EINVAL; |
| 485 | 506 | goto fail; |
| 486 | 507 | } |
| 487 | 508 | #endif |
| 488 | − ret = 0; | |
| 489 | − hlist_for_each_entry(tmp, &pt->pt_irqfds_list, irqfd_hnode) { | |
| 490 | − if (irqfd->irqfd_eventfd_ctx != tmp->irqfd_eventfd_ctx) | |
| 491 | − continue; | |
| 492 | − /* This fd is used for another irq already. */ | |
| 493 | − ret = -EBUSY; | |
| 494 | − spin_unlock_irq(&pt->pt_irqfds_lock); | |
| 495 | − goto fail; | |
| 496 | − } | |
| 497 | 509 | |
| 498 | − idx = srcu_read_lock(&pt->pt_irq_srcu); | |
| 499 | − mshv_irqfd_update(pt, irqfd); | |
| 500 | 510 | hlist_add_head(&irqfd->irqfd_hnode, &pt->pt_irqfds_list); |
| 501 | 511 | spin_unlock_irq(&pt->pt_irqfds_lock); |
| 502 | 512 | |
| @@ -541,13 +551,14 @@ static int mshv_irqfd_deassign(struct mshv_partition *pt, · 38 unmodified lines | ||
| 541 | 551 | if (IS_ERR(eventfd)) |
| 542 | 552 | return PTR_ERR(eventfd); |
| 543 | 553 | |
| 554 | + spin_lock_irq(&pt->pt_irqfds_lock); | |
| 544 | 555 | hlist_for_each_entry_safe(irqfd, n, &pt->pt_irqfds_list, |
| 545 | 556 | irqfd_hnode) { |
| 546 | 557 | if (irqfd->irqfd_eventfd_ctx == eventfd && |
| 547 | 558 | irqfd->irqfd_irqnum == args->gsi) |
| 548 | − | |
| 549 | 559 | mshv_irqfd_deactivate(irqfd); |
| 550 | 560 | } |
| 561 | + spin_unlock_irq(&pt->pt_irqfds_lock); | |
| 551 | 562 | |
| 552 | 563 | eventfd_ctx_put(eventfd); |
| 553 | 564 | |
| @@ -51,7 +51,7 @@ int mshv_update_routing_table(struct mshv_partition *partition, · 50 unmodified lines | ||
| 51 | 51 | /* |
| 52 | 52 | * Allow only one to one mapping between GSI and MSI routing. |
| 53 | 53 | */ |
| 54 | − if (girq->guest_irq_num != 0) { | |
| 54 | + if (girq->girq_entry_valid) { | |
| 55 | 55 | r = -EINVAL; |
| 56 | 56 | goto out; |
| 57 | 57 | } |
| @@ -40,12 +40,14 @@ mshv_port_table_fini(void) · 39 unmodified lines | ||
| 40 | 40 | int |
| 41 | 41 | mshv_portid_alloc(struct port_table_info *info) |
| 42 | 42 | { |
| 43 | − int ret = 0; | |
| 43 | + int ret; | |
| 44 | 44 | |
| 45 | + idr_preload(GFP_KERNEL); | |
| 45 | 46 | idr_lock(&port_table_idr); |
| 46 | 47 | ret = idr_alloc(&port_table_idr, info, PORTID_MIN, |
| 47 | − PORTID_MAX, GFP_KERNEL); | |
| 48 | + PORTID_MAX, GFP_NOWAIT); | |
| 48 | 49 | idr_unlock(&port_table_idr); |
| 50 | + idr_preload_end(); | |
| 49 | 51 | |
| 50 | 52 | return ret; |
| 51 | 53 | } |
| @@ -60,8 +62,7 @@ mshv_portid_free(int port_id) · 8 unmodified lines | ||
| 60 | 62 | WARN_ON(!info); |
| 61 | 63 | idr_unlock(&port_table_idr); |
| 62 | 64 | |
| 63 | − synchronize_rcu(); | |
| 64 | − kfree(info); | |
| 65 | + kfree_rcu(info, portbl_rcu); | |
| 65 | 66 | } |
| 66 | 67 | |
| 67 | 68 | int |
| @@ -1072,6 +1072,8 @@ mshv_partition_ioctl_create_vp(struct mshv_partition *partition, · 1071 unmodified lines | ||
| 1072 | 1072 | struct mshv_vp *vp; |
| 1073 | 1073 | struct page *intercept_msg_page, *register_page, *ghcb_page; |
| 1074 | 1074 | struct hv_stats_page *stats_pages[2]; |
| 1075 | + struct file *file; | |
| 1076 | + int fd; | |
| 1075 | 1077 | long ret; |
| 1076 | 1078 | |
| 1077 | 1079 | if (copy_from_user(&args, arg, sizeof(args))) |
| @@ -1117,8 +1119,10 @@ mshv_partition_ioctl_create_vp(struct mshv_partition *partition, · 39 unmodified lines | ||
| 1117 | 1119 | goto unmap_ghcb_page; |
| 1118 | 1120 | |
| 1119 | 1121 | vp = kzalloc_obj(*vp); |
| 1120 | − if (!vp) | |
| 1122 | + if (!vp) { | |
| 1123 | + ret = -ENOMEM; | |
| 1121 | 1124 | goto unmap_stats_pages; |
| 1125 | + } | |
| 1122 | 1126 | |
| 1123 | 1127 | vp->vp_partition = mshv_partition_get(partition); |
| 1124 | 1128 | if (!vp->vp_partition) { |
| @@ -1144,21 +1148,40 @@ mshv_partition_ioctl_create_vp(struct mshv_partition *partition, · 19 unmodified lines | ||
| 1144 | 1148 | if (ret) |
| 1145 | 1149 | goto put_partition; |
| 1146 | 1150 | |
| 1147 | − /* | |
| 1148 | − * Keep anon_inode_getfd last: it installs fd in the file struct and | |
| 1149 | − * thus makes the state accessible in user space. | |
| 1150 | − */ | |
| 1151 | − ret = anon_inode_getfd("mshv_vp", &mshv_vp_fops, vp, | |
| 1152 | − O_RDWR | O_CLOEXEC); | |
| 1153 | − if (ret < 0) | |
| 1151 | + fd = get_unused_fd_flags(O_RDWR | O_CLOEXEC); | |
| 1152 | + if (fd < 0) { | |
| 1153 | + ret = fd; | |
| 1154 | 1154 | goto remove_debugfs_vp; |
| 1155 | + } | |
| 1156 | + | |
| 1157 | + file = anon_inode_getfile("mshv_vp", &mshv_vp_fops, vp, | |
| 1158 | + O_RDWR | O_CLOEXEC); | |
| 1159 | + if (IS_ERR(file)) { | |
| 1160 | + ret = PTR_ERR(file); | |
| 1161 | + goto put_unused_vp_fd; | |
| 1162 | + } | |
| 1155 | 1163 | |
| 1156 | 1164 | /* already exclusive with the partition mutex for all ioctls */ |
| 1157 | 1165 | partition->pt_vp_count++; |
| 1158 | − partition->pt_vp_array[args.vp_index] = vp; | |
| 1166 | + /* | |
| 1167 | + * Pairs with smp_load_acquire() in mshv_try_assert_irq_fast(), which | |
| 1168 | + * can run concurrently from an irqfd waker without holding pt_mutex. | |
| 1169 | + * The release ensures the VP's initialising stores are visible to any | |
| 1170 | + * reader that observes a non-NULL pointer in pt_vp_array. | |
| 1171 | + */ | |
| 1172 | + smp_store_release(&partition->pt_vp_array[args.vp_index], vp); | |
| 1173 | + | |
| 1174 | + /* | |
| 1175 | + * fd_install() is the userspace-visibility commit point. Must be the | |
| 1176 | + * last operation that can fail or be observed. | |
| 1177 | + */ | |
| 1178 | + fd_install(fd, file); | |
| 1179 | + ret = fd; | |
| 1159 | 1180 | |
| 1160 | 1181 | goto out; |
| 1161 | 1182 | |
| 1183 | +put_unused_vp_fd: | |
| 1184 | + put_unused_fd(fd); | |
| 1162 | 1185 | remove_debugfs_vp: |
| 1163 | 1186 | mshv_debugfs_vp_remove(vp); |
| 1164 | 1187 | put_partition: |
| @@ -129,6 +129,7 @@ mshv_ioctl_create_vtl(void __user *user_arg, struct device *module_dev) · 128 unmodified lines | ||
| 129 | 129 | file = anon_inode_getfile("mshv_vtl", &mshv_vtl_fops, |
| 130 | 130 | vtl, O_RDWR); |
| 131 | 131 | if (IS_ERR(file)) { |
| 132 | + put_unused_fd(fd); | |
| 132 | 133 | kfree(vtl); |
| 133 | 134 | return PTR_ERR(file); |
| 134 | 135 | } |
| @@ -801,7 +802,7 @@ static vm_fault_t mshv_vtl_fault(struct vm_fault *vmf) · 666 unmodified lines | ||
| 801 | 802 | int cpu = vmf->pgoff & MSHV_PG_OFF_CPU_MASK; |
| 802 | 803 | int real_off = vmf->pgoff >> MSHV_REAL_OFF_SHIFT; |
| 803 | 804 | |
| 804 | − if (!cpu_online(cpu)) | |
| 805 | + if (cpu >= nr_cpu_ids || !cpu_online(cpu)) | |
| 805 | 806 | return VM_FAULT_SIGBUS; |
| 806 | 807 | /* |
| 807 | 808 | * CPU Hotplug is not supported in VTL2 in OpenHCL, where this kernel driver exists. |
| @@ -1148,12 +1149,22 @@ static int mshv_vtl_hvcall_call(struct mshv_vtl_hvcall_fd *fd, · 340 unmodified lines | ||
| 1148 | 1149 | */ |
| 1149 | 1150 | in = (void *)__get_free_page(GFP_KERNEL); |
| 1150 | 1151 | out = (void *)__get_free_page(GFP_KERNEL); |
| 1152 | + if (!in || !out) { | |
| 1153 | + ret = -ENOMEM; | |
| 1154 | + goto free_pages; | |
| 1155 | + } | |
| 1151 | 1156 | |
| 1152 | 1157 | if (copy_from_user(in, (void __user *)hvcall.input_ptr, hvcall.input_size)) { |
| 1153 | 1158 | ret = -EFAULT; |
| 1154 | 1159 | goto free_pages; |
| 1155 | 1160 | } |
| 1156 | 1161 | |
| 1162 | + /* | |
| 1163 | + * The caller supplies output_size, so clear the range copied back to | |
| 1164 | + * userspace in case the hypercall writes fewer bytes than requested. | |
| 1165 | + */ | |
| 1166 | + memset(out, 0, hvcall.output_size); | |
| 1167 | + | |
| 1157 | 1168 | hvcall.status = hv_do_hypercall(hvcall.control, in, out); |
| 1158 | 1169 | |
| 1159 | 1170 | if (copy_to_user((void __user *)hvcall.output_ptr, out, hvcall.output_size)) { |
| @@ -1369,8 +1369,6 @@ static void run_vmbus_irqd(unsigned int cpu) · 1368 unmodified lines | ||
| 1369 | 1369 | __vmbus_isr(); |
| 1370 | 1370 | } |
| 1371 | 1371 | |
| 1372 | −static bool vmbus_irq_initialized; | |
| 1373 | − | |
| 1374 | 1372 | static struct smp_hotplug_thread vmbus_irq_threads = { |
| 1375 | 1373 | .store = &vmbus_irqd, |
| 1376 | 1374 | .setup = vmbus_irqd_setup, |
| @@ -1384,8 +1382,19 @@ void vmbus_isr(void) · 7 unmodified lines | ||
| 1384 | 1382 | if (IS_ENABLED(CONFIG_PREEMPT_RT)) { |
| 1385 | 1383 | vmbus_irqd_wake(); |
| 1386 | 1384 | } else { |
| 1387 | − lockdep_hardirq_threaded(); | |
| 1385 | + static DEFINE_WAIT_OVERRIDE_MAP(vmbus_map, LD_WAIT_CONFIG); | |
| 1386 | + | |
| 1387 | + /* | |
| 1388 | + * vmbus_isr is never force-threaded and always invoked at hard | |
| 1389 | + * IRQ level. __vmbus_isr() below can acquire a spinlock_t | |
| 1390 | + * which becomes a sleeping lock and must not be acquired in | |
| 1391 | + * this context. Therefore on PREEMPT_RT this will be threaded | |
| 1392 | + * via vmbus_irqd_wake(). On non-PREEMPT the annotation lets | |
| 1393 | + * lockdep know that acquiring a spinlock_t is not an issue. | |
| 1394 | + */ | |
| 1395 | + lock_map_acquire_try(&vmbus_map); | |
| 1388 | 1396 | __vmbus_isr(); |
| 1397 | + lock_map_release(&vmbus_map); | |
| 1389 | 1398 | } |
| 1390 | 1399 | } |
| 1391 | 1400 | EXPORT_SYMBOL_FOR_MODULES(vmbus_isr, "mshv_vtl"); |
| @@ -1486,11 +1495,10 @@ static int vmbus_bus_init(void) · 94 unmodified lines | ||
| 1486 | 1495 | * the VMbus interrupt handler. |
| 1487 | 1496 | */ |
| 1488 | 1497 | |
| 1489 | − if (IS_ENABLED(CONFIG_PREEMPT_RT) && !vmbus_irq_initialized) { | |
| 1498 | + if (IS_ENABLED(CONFIG_PREEMPT_RT)) { | |
| 1490 | 1499 | ret = smpboot_register_percpu_thread(&vmbus_irq_threads); |
| 1491 | 1500 | if (ret) |
| 1492 | 1501 | goto err_kthread; |
| 1493 | − vmbus_irq_initialized = true; | |
| 1494 | 1502 | } |
| 1495 | 1503 | |
| 1496 | 1504 | if (vmbus_irq == -1) { |
| @@ -1534,10 +1542,8 @@ err_connect: · 37 unmodified lines | ||
| 1534 | 1542 | else |
| 1535 | 1543 | free_percpu_irq(vmbus_irq, &vmbus_evt); |
| 1536 | 1544 | err_setup: |
| 1537 | − if (IS_ENABLED(CONFIG_PREEMPT_RT) && vmbus_irq_initialized) { | |
| 1545 | + if (IS_ENABLED(CONFIG_PREEMPT_RT)) | |
| 1538 | 1546 | smpboot_unregister_percpu_thread(&vmbus_irq_threads); |
| 1539 | − vmbus_irq_initialized = false; | |
| 1540 | − } | |
| 1541 | 1547 | err_kthread: |
| 1542 | 1548 | bus_unregister(&hv_bus); |
| 1543 | 1549 | return ret; |
| @@ -2169,6 +2175,7 @@ int vmbus_device_register(struct hv_device *child_device_obj) · 625 unmodified lines | ||
| 2169 | 2175 | child_device_obj->device.dma_parms = &child_device_obj->dma_parms; |
| 2170 | 2176 | child_device_obj->device.dma_mask = &child_device_obj->dma_mask; |
| 2171 | 2177 | dma_set_mask(&child_device_obj->device, DMA_BIT_MASK(64)); |
| 2178 | + dma_set_coherent_mask(&child_device_obj->device, DMA_BIT_MASK(64)); | |
| 2172 | 2179 | |
| 2173 | 2180 | /* |
| 2174 | 2181 | * Register with the LDM. This will kick off the driver/device |
| @@ -3034,10 +3041,9 @@ static void __exit vmbus_exit(void) · 859 unmodified lines | ||
| 3034 | 3041 | hv_remove_vmbus_handler(); |
| 3035 | 3042 | else |
| 3036 | 3043 | free_percpu_irq(vmbus_irq, &vmbus_evt); |
| 3037 | − if (IS_ENABLED(CONFIG_PREEMPT_RT) && vmbus_irq_initialized) { | |
| 3044 | + if (IS_ENABLED(CONFIG_PREEMPT_RT)) | |
| 3038 | 3045 | smpboot_unregister_percpu_thread(&vmbus_irq_threads); |
| 3039 | − vmbus_irq_initialized = false; | |
| 3040 | − } | |
| 3046 | + | |
| 3041 | 3047 | for_each_online_cpu(cpu) { |
| 3042 | 3048 | struct hv_per_cpu_context *hv_cpu |
| 3043 | 3049 | = per_cpu_ptr(hv_context.cpu_context, cpu); |
| @@ -106,12 +106,11 @@ static int ads7828_probe(struct i2c_client *client) · 105 unmodified lines | ||
| 106 | 106 | struct ads7828_data *data; |
| 107 | 107 | struct device *hwmon_dev; |
| 108 | 108 | unsigned int vref_mv = ADS7828_INT_VREF_MV; |
| 109 | − unsigned int vref_uv; | |
| 109 | + int vref_uv; | |
| 110 | 110 | bool diff_input = false; |
| 111 | 111 | bool ext_vref = false; |
| 112 | 112 | unsigned int regval; |
| 113 | 113 | enum ads7828_chips chip; |
| 114 | − struct regulator *reg; | |
| 115 | 114 | |
| 116 | 115 | data = devm_kzalloc(dev, sizeof(struct ads7828_data), GFP_KERNEL); |
| 117 | 116 | if (!data) |
| @@ -125,9 +124,11 @@ static int ads7828_probe(struct i2c_client *client) · 7 unmodified lines | ||
| 125 | 124 | } else if (dev->of_node) { |
| 126 | 125 | diff_input = of_property_read_bool(dev->of_node, |
| 127 | 126 | "ti,differential-input"); |
| 128 | − reg = devm_regulator_get_optional(dev, "vref"); | |
| 129 | − if (!IS_ERR(reg)) { | |
| 130 | − vref_uv = regulator_get_voltage(reg); | |
| 127 | + vref_uv = devm_regulator_get_enable_read_voltage(dev, "vref"); | |
| 128 | + if (vref_uv < 0) { | |
| 129 | + if (vref_uv != -ENODEV) | |
| 130 | + return vref_uv; | |
| 131 | + } else { | |
| 131 | 132 | vref_mv = DIV_ROUND_CLOSEST(vref_uv, 1000); |
| 132 | 133 | if (vref_mv < ADS7828_EXT_VREF_MV_MIN || |
| 133 | 134 | vref_mv > ADS7828_EXT_VREF_MV_MAX) |
| @@ -70,8 +70,8 @@ static const unsigned short normal_i2c[] = { 0x2C, 0x2E, 0x2F, I2C_CLIENT_END }; · 69 unmodified lines | ||
| 70 | 70 | #define ADT7470_PWM1_AUTO_MASK 0x80 |
| 71 | 71 | #define ADT7470_PWM_AUTO_MASK 0xC0 |
| 72 | 72 | #define ADT7470_REG_PWM34_CFG 0x69 |
| 73 | −#define ADT7470_PWM3_AUTO_MASK 0x40 | |
| 74 | −#define ADT7470_PWM4_AUTO_MASK 0x80 | |
| 73 | +#define ADT7470_PWM4_AUTO_MASK 0x40 | |
| 74 | +#define ADT7470_PWM3_AUTO_MASK 0x80 | |
| 75 | 75 | #define ADT7470_REG_PWM_MIN_BASE_ADDR 0x6A |
| 76 | 76 | #define ADT7470_REG_PWM_MIN_MAX_ADDR 0x6D |
| 77 | 77 | #define ADT7470_REG_PWM_TEMP_MIN_BASE_ADDR 0x6E |
| @@ -110,6 +110,21 @@ static const unsigned short normal_i2c[] = { 0x2C, 0x2E, 0x2F, I2C_CLIENT_END }; · 32 unmodified lines | ||
| 110 | 110 | |
| 111 | 111 | #define ALARM2(x) ((x) << 8) |
| 112 | 112 | |
| 113 | +/* TEMP1..TEMP7 (ch 0..6) are, respectively BIT(0)..BIT(6) of reg 0x41 and | |
| 114 | + * 0x72, or BIT(0)..BIT(6) of data->alarm. | |
| 115 | + * TEMP8..TEMP9 (ch 7..9) are, respectively BIT(0)..BIT(2) of reg 0x42 and | |
| 116 | + * 0x73, or BIT(8)..BIT(10) of data->alarm. | |
| 117 | + */ | |
| 118 | +#define TEMP_ALARM_BIT(ch) ({ \ | |
| 119 | + typeof(ch) _ch = (ch); \ | |
| 120 | + (1 << (_ch < 7 ? _ch : _ch + 1)); \ | |
| 121 | +}) | |
| 122 | + | |
| 123 | +/* FAN1..FAN4 (ch 0..3) are respectively BIT(4)..BIT(7) in | |
| 124 | + * reg 0x42 and 0x73 or BIT(12)..BIT(15) in data->alarm. | |
| 125 | + */ | |
| 126 | +#define FAN_ALARM_BIT(ch) (1 << (12 + (ch))) | |
| 127 | + | |
| 113 | 128 | #define ADT7470_VENDOR 0x41 |
| 114 | 129 | #define ADT7470_DEVICE 0x70 |
| 115 | 130 | /* datasheet only mentions a revision 2 */ |
| @@ -167,6 +182,7 @@ struct adt7470_data { · 51 unmodified lines | ||
| 167 | 182 | u8 pwm_min[ADT7470_PWM_COUNT]; |
| 168 | 183 | s8 pwm_tmin[ADT7470_PWM_COUNT]; |
| 169 | 184 | u8 pwm_auto_temp[ADT7470_PWM_COUNT]; |
| 185 | + u32 pwm_freq; | |
| 170 | 186 | |
| 171 | 187 | struct task_struct *auto_update; |
| 172 | 188 | unsigned int auto_update_interval; |
| @@ -205,11 +221,12 @@ static inline int adt7470_write_word_data(struct adt7470_data *data, unsigned in · 32 unmodified lines | ||
| 205 | 221 | /* Probe for temperature sensors. Assumes lock is held */ |
| 206 | 222 | static int adt7470_read_temperatures(struct adt7470_data *data) |
| 207 | 223 | { |
| 208 | − unsigned long res; | |
| 224 | + struct device *dev = regmap_get_device(data->regmap); | |
| 225 | + u8 pwm[ADT7470_FAN_COUNT]; | |
| 209 | 226 | unsigned int pwm_cfg[2]; |
| 210 | − int err; | |
| 227 | + unsigned long res; | |
| 228 | + int err, err2; | |
| 211 | 229 | int i; |
| 212 | − u8 pwm[ADT7470_FAN_COUNT]; | |
| 213 | 230 | |
| 214 | 231 | /* save pwm[1-4] config register */ |
| 215 | 232 | err = regmap_read(data->regmap, ADT7470_REG_PWM_CFG(0), &pwm_cfg[0]); |
| @@ -233,19 +250,19 @@ static int adt7470_read_temperatures(struct adt7470_data *data) · 17 unmodified lines | ||
| 233 | 250 | err = regmap_update_bits(data->regmap, ADT7470_REG_PWM_CFG(2), |
| 234 | 251 | ADT7470_PWM_AUTO_MASK, 0); |
| 235 | 252 | if (err < 0) |
| 236 | − return err; | |
| 253 | + goto out_restore; | |
| 237 | 254 | |
| 238 | 255 | /* write pwm control to whatever it was */ |
| 239 | 256 | err = regmap_bulk_write(data->regmap, ADT7470_REG_PWM(0), &pwm[0], |
| 240 | 257 | ADT7470_PWM_COUNT); |
| 241 | 258 | if (err < 0) |
| 242 | − return err; | |
| 259 | + goto out_restore; | |
| 243 | 260 | |
| 244 | 261 | /* start reading temperature sensors */ |
| 245 | 262 | err = regmap_update_bits(data->regmap, ADT7470_REG_CFG, |
| 246 | 263 | ADT7470_T05_STB_MASK, ADT7470_T05_STB_MASK); |
| 247 | 264 | if (err < 0) |
| 248 | − return err; | |
| 265 | + goto out_restore; | |
| 249 | 266 | |
| 250 | 267 | /* Delay is 200ms * number of temp sensors. */ |
| 251 | 268 | res = msleep_interruptible((data->num_temp_sensors >= 0 ? |
| @@ -256,13 +273,30 @@ static int adt7470_read_temperatures(struct adt7470_data *data) · 4 unmodified lines | ||
| 256 | 273 | err = regmap_update_bits(data->regmap, ADT7470_REG_CFG, |
| 257 | 274 | ADT7470_T05_STB_MASK, 0); |
| 258 | 275 | if (err < 0) |
| 259 | − return err; | |
| 276 | + goto out_restore; | |
| 260 | 277 | |
| 278 | +out_restore: | |
| 261 | 279 | /* restore pwm[1-4] config registers */ |
| 262 | − err = regmap_write(data->regmap, ADT7470_REG_PWM_CFG(0), pwm_cfg[0]); | |
| 263 | − if (err < 0) | |
| 264 | − return err; | |
| 265 | − err = regmap_write(data->regmap, ADT7470_REG_PWM_CFG(2), pwm_cfg[1]); | |
| 280 | + err2 = regmap_write(data->regmap, ADT7470_REG_PWM_CFG(0), pwm_cfg[0]); | |
| 281 | + if (err2 < 0) { | |
| 282 | + dev_warn_ratelimited(dev, | |
| 283 | + "failed to restore PWM{1,2} config (%d)\n", | |
| 284 | + err2); | |
| 285 | + | |
| 286 | + if (!err) | |
| 287 | + err = err2; | |
| 288 | + } | |
| 289 | + | |
| 290 | + err2 = regmap_write(data->regmap, ADT7470_REG_PWM_CFG(2), pwm_cfg[1]); | |
| 291 | + if (err2 < 0) { | |
| 292 | + dev_warn_ratelimited(dev, | |
| 293 | + "failed to restore PWM{3,4} config (%d)\n", | |
| 294 | + err2); | |
| 295 | + | |
| 296 | + if (!err) | |
| 297 | + err = err2; | |
| 298 | + } | |
| 299 | + | |
| 266 | 300 | if (err < 0) |
| 267 | 301 | return err; |
| 268 | 302 | |
| @@ -491,7 +525,7 @@ static ssize_t auto_update_interval_store(struct device *dev, · 222 unmodified lines | ||
| 491 | 525 | if (kstrtol(buf, 10, &temp)) |
| 492 | 526 | return -EINVAL; |
| 493 | 527 | |
| 494 | − temp = clamp_val(temp, 0, 60000); | |
| 528 | + temp = clamp_val(temp, 500, 60000); | |
| 495 | 529 | |
| 496 | 530 | mutex_lock(&data->lock); |
| 497 | 531 | data->auto_update_interval = temp; |
| @@ -551,7 +585,7 @@ static int adt7470_temp_read(struct device *dev, u32 attr, int channel, long *va · 53 unmodified lines | ||
| 551 | 585 | *val = 1000 * data->temp_max[channel]; |
| 552 | 586 | break; |
| 553 | 587 | case hwmon_temp_alarm: |
| 554 | − *val = !!(data->alarm & channel); | |
| 588 | + *val = !!(data->alarm & TEMP_ALARM_BIT(channel)); | |
| 555 | 589 | break; |
| 556 | 590 | default: |
| 557 | 591 | return -EOPNOTSUPP; |
| @@ -571,14 +605,16 @@ static int adt7470_temp_write(struct device *dev, u32 attr, int channel, long va · 13 unmodified lines | ||
| 571 | 605 | switch (attr) { |
| 572 | 606 | case hwmon_temp_min: |
| 573 | 607 | mutex_lock(&data->lock); |
| 574 | − data->temp_min[channel] = val; | |
| 575 | 608 | err = regmap_write(data->regmap, ADT7470_TEMP_MIN_REG(channel), val); |
| 609 | + if (!err) | |
| 610 | + data->temp_min[channel] = val; | |
| 576 | 611 | mutex_unlock(&data->lock); |
| 577 | 612 | break; |
| 578 | 613 | case hwmon_temp_max: |
| 579 | 614 | mutex_lock(&data->lock); |
| 580 | − data->temp_max[channel] = val; | |
| 581 | 615 | err = regmap_write(data->regmap, ADT7470_TEMP_MAX_REG(channel), val); |
| 616 | + if (!err) | |
| 617 | + data->temp_max[channel] = val; | |
| 582 | 618 | mutex_unlock(&data->lock); |
| 583 | 619 | break; |
| 584 | 620 | default: |
| @@ -624,36 +660,33 @@ static ssize_t alarm_mask_store(struct device *dev, · 39 unmodified lines | ||
| 624 | 660 | static int adt7470_fan_read(struct device *dev, u32 attr, int channel, long *val) |
| 625 | 661 | { |
| 626 | 662 | struct adt7470_data *data = adt7470_update_device(dev); |
| 663 | + u16 fan_data; | |
| 627 | 664 | |
| 628 | 665 | if (IS_ERR(data)) |
| 629 | 666 | return PTR_ERR(data); |
| 630 | 667 | |
| 631 | 668 | switch (attr) { |
| 632 | 669 | case hwmon_fan_input: |
| 633 | − if (FAN_DATA_VALID(data->fan[channel])) | |
| 634 | − *val = FAN_PERIOD_TO_RPM(data->fan[channel]); | |
| 635 | − else | |
| 636 | − *val = 0; | |
| 670 | + fan_data = READ_ONCE(data->fan[channel]); | |
| 637 | 671 | break; |
| 638 | 672 | case hwmon_fan_min: |
| 639 | − if (FAN_DATA_VALID(data->fan_min[channel])) | |
| 640 | − *val = FAN_PERIOD_TO_RPM(data->fan_min[channel]); | |
| 641 | − else | |
| 642 | − *val = 0; | |
| 673 | + fan_data = READ_ONCE(data->fan_min[channel]); | |
| 643 | 674 | break; |
| 644 | 675 | case hwmon_fan_max: |
| 645 | − if (FAN_DATA_VALID(data->fan_max[channel])) | |
| 646 | − *val = FAN_PERIOD_TO_RPM(data->fan_max[channel]); | |
| 647 | − else | |
| 648 | − *val = 0; | |
| 676 | + fan_data = READ_ONCE(data->fan_max[channel]); | |
| 649 | 677 | break; |
| 650 | 678 | case hwmon_fan_alarm: |
| 651 | − *val = !!(data->alarm & (1 << (12 + channel))); | |
| 652 | − break; | |
| 679 | + *val = !!(data->alarm & FAN_ALARM_BIT(channel)); | |
| 680 | + return 0; | |
| 653 | 681 | default: |
| 654 | 682 | return -EOPNOTSUPP; |
| 655 | 683 | } |
| 656 | 684 | |
| 685 | + if (FAN_DATA_VALID(fan_data)) | |
| 686 | + *val = FAN_PERIOD_TO_RPM(fan_data); | |
| 687 | + else | |
| 688 | + *val = 0; | |
| 689 | + | |
| 657 | 690 | return 0; |
| 658 | 691 | } |
| 659 | 692 | |
| @@ -721,7 +754,7 @@ static ssize_t force_pwm_max_store(struct device *dev, · 61 unmodified lines | ||
| 721 | 754 | } |
| 722 | 755 | |
| 723 | 756 | /* These are the valid PWM frequencies to the nearest Hz */ |
| 724 | −static const int adt7470_freq_map[] = { | |
| 757 | +static const u32 adt7470_freq_map[] = { | |
| 725 | 758 | 11, 15, 22, 29, 35, 44, 59, 88, 1400, 22500 |
| 726 | 759 | }; |
| 727 | 760 | |
| @@ -761,7 +794,7 @@ static int adt7470_pwm_read(struct device *dev, u32 attr, int channel, long *val · 33 unmodified lines | ||
| 761 | 794 | *val = 1 + data->pwm_automatic[channel]; |
| 762 | 795 | break; |
| 763 | 796 | case hwmon_pwm_freq: |
| 764 | − *val = pwm1_freq_get(dev); | |
| 797 | + *val = data->pwm_freq; | |
| 765 | 798 | break; |
| 766 | 799 | default: |
| 767 | 800 | return -EOPNOTSUPP; |
| @@ -774,12 +807,14 @@ static int pwm1_freq_set(struct device *dev, long freq) · 6 unmodified lines | ||
| 774 | 807 | { |
| 775 | 808 | struct adt7470_data *data = dev_get_drvdata(dev); |
| 776 | 809 | unsigned int low_freq = ADT7470_CFG_LF; |
| 810 | + u32 closest_freq; | |
| 777 | 811 | int index; |
| 778 | 812 | int err; |
| 779 | 813 | |
| 780 | 814 | /* Round the user value given to the closest available frequency */ |
| 781 | 815 | index = find_closest(freq, adt7470_freq_map, |
| 782 | 816 | ARRAY_SIZE(adt7470_freq_map)); |
| 817 | + closest_freq = adt7470_freq_map[index]; | |
| 783 | 818 | |
| 784 | 819 | if (index >= 8) { |
| 785 | 820 | index -= 8; |
| @@ -797,6 +832,10 @@ static int pwm1_freq_set(struct device *dev, long freq) · 11 unmodified lines | ||
| 797 | 832 | err = regmap_update_bits(data->regmap, ADT7470_REG_CFG_2, |
| 798 | 833 | ADT7470_FREQ_MASK, |
| 799 | 834 | index << ADT7470_FREQ_SHIFT); |
| 835 | + if (err < 0) | |
| 836 | + goto out; | |
| 837 | + | |
| 838 | + data->pwm_freq = closest_freq; | |
| 800 | 839 | out: |
| 801 | 840 | mutex_unlock(&data->lock); |
| 802 | 841 | |
| @@ -813,9 +852,10 @@ static int adt7470_pwm_write(struct device *dev, u32 attr, int channel, long val · 10 unmodified lines | ||
| 813 | 852 | case hwmon_pwm_input: |
| 814 | 853 | val = clamp_val(val, 0, 255); |
| 815 | 854 | mutex_lock(&data->lock); |
| 816 | − data->pwm[channel] = val; | |
| 817 | 855 | err = regmap_write(data->regmap, ADT7470_REG_PWM(channel), |
| 818 | − data->pwm[channel]); | |
| 856 | + val); | |
| 857 | + if (!err) | |
| 858 | + data->pwm[channel] = val; | |
| 819 | 859 | mutex_unlock(&data->lock); |
| 820 | 860 | break; |
| 821 | 861 | case hwmon_pwm_enable: |
| @@ -829,10 +869,11 @@ static int adt7470_pwm_write(struct device *dev, u32 attr, int channel, long val · 7 unmodified lines | ||
| 829 | 869 | val--; |
| 830 | 870 | |
| 831 | 871 | mutex_lock(&data->lock); |
| 832 | − data->pwm_automatic[channel] = val; | |
| 833 | 872 | err = regmap_update_bits(data->regmap, ADT7470_REG_PWM_CFG(channel), |
| 834 | 873 | pwm_auto_reg_mask, |
| 835 | 874 | val ? pwm_auto_reg_mask : 0); |
| 875 | + if (!err) | |
| 876 | + data->pwm_automatic[channel] = val; | |
| 836 | 877 | mutex_unlock(&data->lock); |
| 837 | 878 | break; |
| 838 | 879 | case hwmon_pwm_freq: |
| @@ -1008,8 +1049,10 @@ static ssize_t pwm_auto_temp_store(struct device *dev, · 169 unmodified lines | ||
| 1008 | 1049 | if (temp < 0) |
| 1009 | 1050 | return temp; |
| 1010 | 1051 | |
| 1052 | + if (temp > 0xF) | |
| 1053 | + return -EINVAL; | |
| 1054 | + | |
| 1011 | 1055 | mutex_lock(&data->lock); |
| 1012 | − data->pwm_automatic[attr->index] = temp; | |
| 1013 | 1056 | |
| 1014 | 1057 | if (!(attr->index % 2)) { |
| 1015 | 1058 | mask = 0xF0; |
| @@ -1020,6 +1063,9 @@ static ssize_t pwm_auto_temp_store(struct device *dev, · 4 unmodified lines | ||
| 1020 | 1063 | } |
| 1021 | 1064 | |
| 1022 | 1065 | err = regmap_update_bits(data->regmap, pwm_auto_reg, mask, val); |
| 1066 | + if (!err) | |
| 1067 | + data->pwm_auto_temp[attr->index] = temp; | |
| 1068 | + | |
| 1023 | 1069 | mutex_unlock(&data->lock); |
| 1024 | 1070 | |
| 1025 | 1071 | return err < 0 ? err : count; |
| @@ -1248,6 +1294,7 @@ static int adt7470_probe(struct i2c_client *client) · 222 unmodified lines | ||
| 1248 | 1294 | struct device *dev = &client->dev; |
| 1249 | 1295 | struct adt7470_data *data; |
| 1250 | 1296 | struct device *hwmon_dev; |
| 1297 | + int freq_val; | |
| 1251 | 1298 | int err; |
| 1252 | 1299 | |
| 1253 | 1300 | data = devm_kzalloc(dev, sizeof(struct adt7470_data), GFP_KERNEL); |
| @@ -1272,6 +1319,14 @@ static int adt7470_probe(struct i2c_client *client) · 18 unmodified lines | ||
| 1272 | 1319 | if (err < 0) |
| 1273 | 1320 | return err; |
| 1274 | 1321 | |
| 1322 | + freq_val = pwm1_freq_get(dev); | |
| 1323 | + if (freq_val <= 0) { | |
| 1324 | + err = freq_val < 0 ? freq_val : -EINVAL; | |
| 1325 | + return err; | |
| 1326 | + } | |
| 1327 | + | |
| 1328 | + data->pwm_freq = (u32)freq_val; | |
| 1329 | + | |
| 1275 | 1330 | /* Register sysfs hooks */ |
| 1276 | 1331 | hwmon_dev = devm_hwmon_device_register_with_info(dev, client->name, data, |
| 1277 | 1332 | &adt7470_chip_info, |
| @@ -137,13 +137,18 @@ struct corsairpsu_data { · 136 unmodified lines | ||
| 137 | 137 | }; |
| 138 | 138 | |
| 139 | 139 | /* some values are SMBus LINEAR11 data which need a conversion */ |
| 140 | −static int corsairpsu_linear11_to_int(const u16 val, const int scale) | |
| 140 | +static long corsairpsu_linear11_to_long(const u16 val, const int scale) | |
| 141 | 141 | { |
| 142 | 142 | const int exp = ((s16)val) >> 11; |
| 143 | − const int mant = (((s16)(val & 0x7ff)) << 5) >> 5; | |
| 144 | − const int result = mant * scale; | |
| 143 | + const int mant = ((s16)((val & 0x7ff) << 5)) >> 5; | |
| 144 | + s64 result = mant * scale; | |
| 145 | 145 | |
| 146 | − return (exp >= 0) ? (result << exp) : (result >> -exp); | |
| 146 | + if (exp >= 0) | |
| 147 | + result *= (int)(1UL << exp); | |
| 148 | + else | |
| 149 | + result >>= -exp; | |
| 150 | + | |
| 151 | + return clamp(result, LONG_MIN, LONG_MAX); | |
| 147 | 152 | } |
| 148 | 153 | |
| 149 | 154 | /* the micro-controller uses percentage values to control pwm */ |
| @@ -263,13 +268,13 @@ static int corsairpsu_get_value(struct corsairpsu_data *priv, u8 cmd, u8 rail, l · 113 unmodified lines | ||
| 263 | 268 | case PSU_CMD_RAIL_AMPS: |
| 264 | 269 | case PSU_CMD_TEMP0: |
| 265 | 270 | case PSU_CMD_TEMP1: |
| 266 | − *val = corsairpsu_linear11_to_int(tmp & 0xFFFF, 1000); | |
| 271 | + *val = corsairpsu_linear11_to_long(tmp & 0xFFFF, 1000); | |
| 267 | 272 | break; |
| 268 | 273 | case PSU_CMD_FAN: |
| 269 | − *val = corsairpsu_linear11_to_int(tmp & 0xFFFF, 1); | |
| 274 | + *val = corsairpsu_linear11_to_long(tmp & 0xFFFF, 1); | |
| 270 | 275 | break; |
| 271 | 276 | case PSU_CMD_FAN_PWM_ENABLE: |
| 272 | − *val = corsairpsu_linear11_to_int(tmp & 0xFFFF, 1); | |
| 277 | + *val = corsairpsu_linear11_to_long(tmp & 0xFFFF, 1); | |
| 273 | 278 | /* |
| 274 | 279 | * 0 = automatic mode, means the micro-controller controls the fan using a plan |
| 275 | 280 | * which can be modified, but changing this plan is not supported by this |
| @@ -283,12 +288,12 @@ static int corsairpsu_get_value(struct corsairpsu_data *priv, u8 cmd, u8 rail, l · 7 unmodified lines | ||
| 283 | 288 | *val = 2; |
| 284 | 289 | break; |
| 285 | 290 | case PSU_CMD_FAN_PWM: |
| 286 | − *val = corsairpsu_linear11_to_int(tmp & 0xFFFF, 1); | |
| 291 | + *val = corsairpsu_linear11_to_long(tmp & 0xFFFF, 1); | |
| 287 | 292 | *val = corsairpsu_dutycycle_to_pwm(*val); |
| 288 | 293 | break; |
| 289 | 294 | case PSU_CMD_RAIL_WATTS: |
| 290 | 295 | case PSU_CMD_TOTAL_WATTS: |
| 291 | − *val = corsairpsu_linear11_to_int(tmp & 0xFFFF, 1000000); | |
| 296 | + *val = corsairpsu_linear11_to_long(tmp & 0xFFFF, 1000000); | |
| 292 | 297 | break; |
| 293 | 298 | case PSU_CMD_TOTAL_UPTIME: |
| 294 | 299 | case PSU_CMD_UPTIME: |
| @@ -664,6 +669,8 @@ static void print_uptime(struct seq_file *seqf, u8 cmd) · 369 unmodified lines | ||
| 664 | 669 | long val; |
| 665 | 670 | int ret; |
| 666 | 671 | |
| 672 | + guard(hwmon_lock)(priv->hwmon_dev); | |
| 673 | + | |
| 667 | 674 | ret = corsairpsu_get_value(priv, cmd, 0, &val); |
| 668 | 675 | if (ret < 0) { |
| 669 | 676 | seq_puts(seqf, "N/A\n"); |
| @@ -701,7 +708,7 @@ static int vendor_show(struct seq_file *seqf, void *unused) · 31 unmodified lines | ||
| 701 | 708 | { |
| 702 | 709 | struct corsairpsu_data *priv = seqf->private; |
| 703 | 710 | |
| 704 | − seq_printf(seqf, "%s\n", priv->vendor); | |
| 711 | + seq_printf(seqf, "%.*s\n", REPLY_SIZE, priv->vendor); | |
| 705 | 712 | |
| 706 | 713 | return 0; |
| 707 | 714 | } |
| @@ -711,7 +718,7 @@ static int product_show(struct seq_file *seqf, void *unused) · 3 unmodified lines | ||
| 711 | 718 | { |
| 712 | 719 | struct corsairpsu_data *priv = seqf->private; |
| 713 | 720 | |
| 714 | − seq_printf(seqf, "%s\n", priv->product); | |
| 721 | + seq_printf(seqf, "%.*s\n", REPLY_SIZE, priv->product); | |
| 715 | 722 | |
| 716 | 723 | return 0; |
| 717 | 724 | } |
| @@ -723,6 +730,8 @@ static int ocpmode_show(struct seq_file *seqf, void *unused) · 5 unmodified lines | ||
| 723 | 730 | long val; |
| 724 | 731 | int ret; |
| 725 | 732 | |
| 733 | + guard(hwmon_lock)(priv->hwmon_dev); | |
| 734 | + | |
| 726 | 735 | /* |
| 727 | 736 | * The rail mode is switchable on the fly. The RAW interface can be used for this. But it |
| 728 | 737 | * will not be included here, because I consider it somewhat dangerous for the health of the |
| @@ -1083,7 +1083,6 @@ EXPORT_SYMBOL_GPL(hwmon_device_register_with_info); · 1082 unmodified lines | ||
| 1083 | 1083 | * @dev: the parent device |
| 1084 | 1084 | * @name: hwmon name attribute |
| 1085 | 1085 | * @drvdata: driver data to attach to created device |
| 1086 | − * @extra_groups: pointer to list of additional non-standard attribute groups | |
| 1087 | 1086 | * |
| 1088 | 1087 | * The use of this function is restricted. It is provided for legacy reasons |
| 1089 | 1088 | * and must only be called from the thermal subsystem. |
| @@ -1095,13 +1094,12 @@ EXPORT_SYMBOL_GPL(hwmon_device_register_with_info); · 5 unmodified lines | ||
| 1095 | 1094 | */ |
| 1096 | 1095 | struct device * |
| 1097 | 1096 | hwmon_device_register_for_thermal(struct device *dev, const char *name, |
| 1098 | − void *drvdata, | |
| 1099 | − const struct attribute_group **extra_groups) | |
| 1097 | + void *drvdata) | |
| 1100 | 1098 | { |
| 1101 | 1099 | if (!name || !dev) |
| 1102 | 1100 | return ERR_PTR(-EINVAL); |
| 1103 | 1101 | |
| 1104 | − return __hwmon_device_register(dev, name, drvdata, NULL, extra_groups); | |
| 1102 | + return __hwmon_device_register(dev, name, drvdata, NULL, NULL); | |
| 1105 | 1103 | } |
| 1106 | 1104 | EXPORT_SYMBOL_NS_GPL(hwmon_device_register_for_thermal, "HWMON_THERMAL"); |
| 1107 | 1105 | |
| @@ -16,6 +16,7 @@ · 15 unmodified lines | ||
| 16 | 16 | #include <linux/i2c.h> |
| 17 | 17 | #include <linux/init.h> |
| 18 | 18 | #include <linux/kernel.h> |
| 19 | +#include <linux/limits.h> | |
| 19 | 20 | #include <linux/module.h> |
| 20 | 21 | #include <linux/property.h> |
| 21 | 22 | #include <linux/regmap.h> |
| @@ -266,30 +267,34 @@ static u16 ina226_interval_to_reg(long interval) · 244 unmodified lines | ||
| 266 | 267 | return FIELD_PREP(INA226_AVG_RD_MASK, avg_bits); |
| 267 | 268 | } |
| 268 | 269 | |
| 269 | −static int ina2xx_get_value(struct ina2xx_data *data, u8 reg, | |
| 270 | − unsigned int regval) | |
| 270 | +static long ina2xx_get_value(struct ina2xx_data *data, u8 reg, | |
| 271 | + unsigned int regval) | |
| 271 | 272 | { |
| 272 | − int val; | |
| 273 | + s64 val64; | |
| 274 | + long val; | |
| 273 | 275 | |
| 274 | 276 | switch (reg) { |
| 275 | 277 | case INA2XX_SHUNT_VOLTAGE: |
| 276 | 278 | /* signed register */ |
| 277 | − val = (s16)regval >> data->config->shunt_voltage_shift; | |
| 278 | − val = DIV_ROUND_CLOSEST(val, data->config->shunt_div); | |
| 279 | + val = DIV_ROUND_CLOSEST((s16)regval >> data->config->shunt_voltage_shift, | |
| 280 | + data->config->shunt_div); | |
| 279 | 281 | break; |
| 280 | 282 | case INA2XX_BUS_VOLTAGE: |
| 281 | − val = (regval >> data->config->bus_voltage_shift) * | |
| 282 | − data->config->bus_voltage_lsb; | |
| 283 | − val = DIV_ROUND_CLOSEST(val, 1000); | |
| 283 | + val = DIV_ROUND_CLOSEST((regval >> data->config->bus_voltage_shift) * | |
| 284 | + data->config->bus_voltage_lsb, 1000); | |
| 284 | 285 | break; |
| 285 | 286 | case INA2XX_POWER: |
| 286 | − val = regval * data->power_lsb_uW; | |
| 287 | + val = min_t(u64, (u64)regval * data->power_lsb_uW, LONG_MAX); | |
| 287 | 288 | break; |
| 288 | 289 | case INA2XX_CURRENT: |
| 289 | 290 | /* signed register, result in mA */ |
| 290 | − val = ((s16)regval >> data->config->current_shift) * | |
| 291 | + val64 = (s64)((s16)regval >> data->config->current_shift) * | |
| 291 | 292 | data->current_lsb_uA; |
| 292 | − val = DIV_ROUND_CLOSEST(val, 1000); | |
| 293 | + if (val64 < 0) | |
| 294 | + val64 = -DIV_ROUND_CLOSEST_ULL(-val64, 1000); | |
| 295 | + else | |
| 296 | + val64 = DIV_ROUND_CLOSEST_ULL(val64, 1000); | |
| 297 | + val = clamp_val(val64, LONG_MIN, LONG_MAX); | |
| 293 | 298 | break; |
| 294 | 299 | case INA2XX_CALIBRATION: |
| 295 | 300 | val = regval; |
| @@ -378,27 +383,29 @@ static int ina2xx_read_init(struct device *dev, int reg, long *val) · 82 unmodified lines | ||
| 378 | 383 | */ |
| 379 | 384 | static u16 ina226_alert_to_reg(struct ina2xx_data *data, int reg, long val) |
| 380 | 385 | { |
| 386 | + long limit; | |
| 387 | + | |
| 381 | 388 | switch (reg) { |
| 382 | 389 | case INA2XX_SHUNT_VOLTAGE: |
| 383 | − val = clamp_val(val, 0, SHRT_MAX * data->config->shunt_div); | |
| 384 | − val *= data->config->shunt_div; | |
| 385 | − val <<= data->config->shunt_voltage_shift; | |
| 386 | − return clamp_val(val, 0, SHRT_MAX); | |
| 390 | + val = min_t(long, val, DIV_ROUND_CLOSEST(SHRT_MAX, data->config->shunt_div)); | |
| 391 | + return min_t(long, (val * data->config->shunt_div) << data->config->shunt_voltage_shift, | |
| 392 | + SHRT_MAX); | |
| 387 | 393 | case INA2XX_BUS_VOLTAGE: |
| 388 | − val = clamp_val(val, 0, 200000); | |
| 389 | − val = (val * 1000) << data->config->bus_voltage_shift; | |
| 390 | − val = DIV_ROUND_CLOSEST(val, data->config->bus_voltage_lsb); | |
| 391 | − return clamp_val(val, 0, USHRT_MAX); | |
| 394 | + val = min_t(long, val, 130000); | |
| 395 | + return min_t(long, | |
| 396 | + DIV_ROUND_CLOSEST((val * 1000) << data->config->bus_voltage_shift, | |
| 397 | + data->config->bus_voltage_lsb), | |
| 398 | + USHRT_MAX); | |
| 392 | 399 | case INA2XX_POWER: |
| 393 | − val = clamp_val(val, 0, UINT_MAX - data->power_lsb_uW); | |
| 394 | − val = DIV_ROUND_CLOSEST(val, data->power_lsb_uW); | |
| 395 | − return clamp_val(val, 0, USHRT_MAX); | |
| 400 | + val = min_t(long, val, LONG_MAX - data->power_lsb_uW); | |
| 401 | + return min_t(long, DIV_ROUND_CLOSEST(val, data->power_lsb_uW), USHRT_MAX); | |
| 396 | 402 | case INA2XX_CURRENT: |
| 397 | − val = clamp_val(val, INT_MIN / 1000, INT_MAX / 1000); | |
| 403 | + limit = (LONG_MAX - (data->current_lsb_uA / 2)) / 1000; | |
| 404 | + val = min_t(long, val, limit); | |
| 398 | 405 | /* signed register, result in mA */ |
| 399 | 406 | val = DIV_ROUND_CLOSEST(val * 1000, data->current_lsb_uA); |
| 400 | − val <<= data->config->current_shift; | |
| 401 | − return clamp_val(val, SHRT_MIN, SHRT_MAX); | |
| 407 | + limit = SHRT_MAX >> data->config->current_shift; | |
| 408 | + return (u16)(min_t(long, val, limit) << data->config->current_shift); | |
| 402 | 409 | default: |
| 403 | 410 | /* programmer goofed */ |
| 404 | 411 | WARN_ON_ONCE(1); |
| @@ -537,6 +544,7 @@ static int sy24655_average_power_read(struct ina2xx_data *data, u8 reg, long *va · 132 unmodified lines | ||
| 537 | 544 | u8 template[6]; |
| 538 | 545 | int ret; |
| 539 | 546 | long accumulator_24, sample_count; |
| 547 | + u64 val64; | |
| 540 | 548 | |
| 541 | 549 | /* 48-bit register read */ |
| 542 | 550 | ret = i2c_smbus_read_i2c_block_data(data->client, reg, 6, template); |
| @@ -555,7 +563,8 @@ static int sy24655_average_power_read(struct ina2xx_data *data, u8 reg, long *va · 12 unmodified lines | ||
| 555 | 563 | return 0; |
| 556 | 564 | } |
| 557 | 565 | |
| 558 | − *val = DIV_ROUND_CLOSEST(accumulator_24, sample_count) * data->power_lsb_uW; | |
| 566 | + val64 = (u64)DIV_ROUND_CLOSEST(accumulator_24, sample_count) * data->power_lsb_uW; | |
| 567 | + *val = min_t(u64, val64, LONG_MAX); | |
| 559 | 568 | |
| 560 | 569 | return 0; |
| 561 | 570 | } |
| @@ -247,8 +247,7 @@ static struct lm63_data *lm63_update_device(struct device *dev) · 246 unmodified lines | ||
| 247 | 247 | LM63_REG_TACH_LIMIT_MSB) << 8); |
| 248 | 248 | } |
| 249 | 249 | |
| 250 | − data->pwm1_freq = i2c_smbus_read_byte_data(client, | |
| 251 | − LM63_REG_PWM_FREQ); | |
| 250 | + data->pwm1_freq = i2c_smbus_read_byte_data(client, LM63_REG_PWM_FREQ) & 0x1f; | |
| 252 | 251 | if (data->pwm1_freq == 0) |
| 253 | 252 | data->pwm1_freq = 1; |
| 254 | 253 | data->pwm1[0] = i2c_smbus_read_byte_data(client, |
| @@ -1187,7 +1186,7 @@ static void lm63_init_client(struct lm63_data *data) · 932 unmodified lines | ||
| 1187 | 1186 | data->config |= 0x04; |
| 1188 | 1187 | |
| 1189 | 1188 | /* We may need pwm1_freq before ever updating the client data */ |
| 1190 | − data->pwm1_freq = i2c_smbus_read_byte_data(client, LM63_REG_PWM_FREQ); | |
| 1189 | + data->pwm1_freq = i2c_smbus_read_byte_data(client, LM63_REG_PWM_FREQ) & 0x1f; | |
| 1191 | 1190 | if (data->pwm1_freq == 0) |
| 1192 | 1191 | data->pwm1_freq = 1; |
| 1193 | 1192 | |
| @@ -1194,7 +1194,7 @@ static int lm90_update_alarms_locked(struct lm90_data *data, bool force) · 1193 unmodified lines | ||
| 1194 | 1194 | check_enable = (client->irq || !(data->config_orig & 0x80)) && |
| 1195 | 1195 | (data->config & 0x80); |
| 1196 | 1196 | |
| 1197 | − if (force || check_enable) | |
| 1197 | + if (data->hwmon_dev && (force || check_enable)) | |
| 1198 | 1198 | schedule_work(&data->report_work); |
| 1199 | 1199 | |
| 1200 | 1200 | /* |
| @@ -1202,7 +1202,7 @@ static int lm90_update_alarms_locked(struct lm90_data *data, bool force) · 1 unmodified line | ||
| 1202 | 1202 | * alarms are all clear, and alerts are currently disabled. |
| 1203 | 1203 | * Otherwise (re)schedule worker if needed. |
| 1204 | 1204 | */ |
| 1205 | − if (check_enable) { | |
| 1205 | + if (check_enable && data->hwmon_dev) { | |
| 1206 | 1206 | if (!(data->current_alarms & data->alert_alarms)) { |
| 1207 | 1207 | dev_dbg(&client->dev, "Re-enabling ALERT#\n"); |
| 1208 | 1208 | lm90_update_confreg(data, data->config & ~0x80); |
| @@ -14,6 +14,7 @@ · 13 unmodified lines | ||
| 14 | 14 | #include <linux/hwmon.h> |
| 15 | 15 | #include <linux/i2c.h> |
| 16 | 16 | #include <linux/math.h> |
| 17 | +#include <linux/math64.h> | |
| 17 | 18 | #include <linux/minmax.h> |
| 18 | 19 | #include <linux/module.h> |
| 19 | 20 | #include <linux/regmap.h> |
| @@ -137,7 +138,7 @@ struct ltc4282_state { · 117 unmodified lines | ||
| 137 | 138 | */ |
| 138 | 139 | struct ltc4282_cache in0_1_cache[LTC4282_CHAN_VGPIO]; |
| 139 | 140 | u32 vsense_max; |
| 140 | − long power_max; | |
| 141 | + s64 power_max; | |
| 141 | 142 | u32 rsense; |
| 142 | 143 | u16 vdd; |
| 143 | 144 | u16 vfs_out; |
| @@ -374,8 +375,8 @@ static int ltc4282_read_in(struct ltc4282_state *st, u32 attr, long *val, · 230 unmodified lines | ||
| 374 | 375 | channel, val); |
| 375 | 376 | case hwmon_in_min_alarm: |
| 376 | 377 | if (channel == LTC4282_CHAN_VGPIO) |
| 377 | − ltc4282_read_alarm(st, LTC4282_ADC_ALERT_LOG, | |
| 378 | − LTC4282_GPIO_ALARM_L_MASK, val); | |
| 378 | + return ltc4282_read_alarm(st, LTC4282_ADC_ALERT_LOG, | |
| 379 | + LTC4282_GPIO_ALARM_L_MASK, val); | |
| 379 | 380 | |
| 380 | 381 | return ltc4282_vdd_source_read_alm(st, |
| 381 | 382 | LTC4282_VSOURCE_ALARM_L_MASK, |
| @@ -613,13 +614,12 @@ static int ltc4282_read(struct device *dev, enum hwmon_sensor_types type, · 231 unmodified lines | ||
| 613 | 614 | } |
| 614 | 615 | |
| 615 | 616 | static int ltc4282_write_power_byte(const struct ltc4282_state *st, u32 reg, |
| 616 | − long val) | |
| 617 | + s64 val) | |
| 617 | 618 | { |
| 618 | 619 | u32 power; |
| 619 | 620 | u64 temp; |
| 620 | 621 | |
| 621 | − if (val > st->power_max) | |
| 622 | − val = st->power_max; | |
| 622 | + val = clamp(val, 0, st->power_max); | |
| 623 | 623 | |
| 624 | 624 | temp = val * int_pow(U8_MAX, 2) * st->rsense; |
| 625 | 625 | power = DIV64_U64_ROUND_CLOSEST(temp, |
| @@ -629,7 +629,7 @@ static int ltc4282_write_power_byte(const struct ltc4282_state *st, u32 reg, · 3 unmodified lines | ||
| 629 | 629 | } |
| 630 | 630 | |
| 631 | 631 | static int ltc4282_write_power_word(const struct ltc4282_state *st, u32 reg, |
| 632 | − long val) | |
| 632 | + u64 val) | |
| 633 | 633 | { |
| 634 | 634 | u64 temp = int_pow(U16_MAX, 2) * st->rsense, temp_2; |
| 635 | 635 | __be16 __raw; |
| @@ -930,8 +930,11 @@ static int ltc4282_curr_reset_hist(struct ltc4282_state *st) · 294 unmodified lines | ||
| 930 | 930 | static int ltc4282_write_curr(struct ltc4282_state *st, u32 attr, |
| 931 | 931 | long val) |
| 932 | 932 | { |
| 933 | + s32 ulimit = min_t(u64, INT_MAX, | |
| 934 | + div_u64((u64)INT_MAX * DECA * MICRO, st->rsense)); | |
| 935 | + u64 val64 = clamp(val, 0, ulimit); | |
| 933 | 936 | /* need to pass it in millivolt */ |
| 934 | − u32 in = DIV_ROUND_CLOSEST_ULL((u64)val * st->rsense, DECA * MICRO); | |
| 937 | + u32 in = DIV_ROUND_CLOSEST_ULL(val64 * st->rsense, DECA * MICRO); | |
| 935 | 938 | |
| 936 | 939 | switch (attr) { |
| 937 | 940 | case hwmon_curr_max: |
| @@ -1222,7 +1225,8 @@ static int ltc4282_set_max_limits(struct ltc4282_state *st) · 284 unmodified lines | ||
| 1222 | 1225 | return ret; |
| 1223 | 1226 | |
| 1224 | 1227 | /* Power is given by ISENSE * Vout. */ |
| 1225 | − st->power_max = DIV_ROUND_CLOSEST(st->vsense_max * DECA * MILLI, st->rsense) * st->vfs_out; | |
| 1228 | + st->power_max = DIV_ROUND_CLOSEST_ULL((u64)st->vsense_max * DECA * MILLI, | |
| 1229 | + st->rsense) * st->vfs_out; | |
| 1226 | 1230 | ret = ltc4282_write_power_byte(st, LTC4282_POWER_MAX, st->power_max); |
| 1227 | 1231 | if (ret) |
| 1228 | 1232 | return ret; |
| @@ -1390,7 +1394,7 @@ static int ltc4282_setup(struct ltc4282_state *st, struct device *dev) · 161 unmodified lines | ||
| 1390 | 1394 | if (!ret) { |
| 1391 | 1395 | int reg_val; |
| 1392 | 1396 | |
| 1393 | − switch (val) { | |
| 1397 | + switch (st->vsense_max) { | |
| 1394 | 1398 | case 12500: |
| 1395 | 1399 | reg_val = 0; |
| 1396 | 1400 | break; |
| @@ -791,12 +791,12 @@ static const u16 NCT6106_REG_TOLERANCE_H[] = { 0x112, 0x122, 0x132 }; · 790 unmodified lines | ||
| 791 | 791 | |
| 792 | 792 | static const u16 NCT6106_REG_TARGET[] = { 0x111, 0x121, 0x131 }; |
| 793 | 793 | |
| 794 | −static const u16 NCT6106_REG_WEIGHT_TEMP_SEL[] = { 0x168, 0x178, 0x188 }; | |
| 795 | −static const u16 NCT6106_REG_WEIGHT_TEMP_STEP[] = { 0x169, 0x179, 0x189 }; | |
| 796 | −static const u16 NCT6106_REG_WEIGHT_TEMP_STEP_TOL[] = { 0x16a, 0x17a, 0x18a }; | |
| 797 | −static const u16 NCT6106_REG_WEIGHT_DUTY_STEP[] = { 0x16b, 0x17b, 0x18b }; | |
| 798 | −static const u16 NCT6106_REG_WEIGHT_TEMP_BASE[] = { 0x16c, 0x17c, 0x18c }; | |
| 799 | −static const u16 NCT6106_REG_WEIGHT_DUTY_BASE[] = { 0x16d, 0x17d, 0x18d }; | |
| 794 | +static const u16 NCT6106_REG_WEIGHT_TEMP_SEL[] = { 0x168, 0x178, 0x188, 0, 0 }; | |
| 795 | +static const u16 NCT6106_REG_WEIGHT_TEMP_STEP[] = { 0x169, 0x179, 0x189, 0, 0 }; | |
| 796 | +static const u16 NCT6106_REG_WEIGHT_TEMP_STEP_TOL[] = { 0x16a, 0x17a, 0x18a, 0, 0 }; | |
| 797 | +static const u16 NCT6106_REG_WEIGHT_DUTY_STEP[] = { 0x16b, 0x17b, 0x18b, 0, 0 }; | |
| 798 | +static const u16 NCT6106_REG_WEIGHT_TEMP_BASE[] = { 0x16c, 0x17c, 0x18c, 0, 0 }; | |
| 799 | +static const u16 NCT6106_REG_WEIGHT_DUTY_BASE[] = { 0x16d, 0x17d, 0x18d, 0, 0 }; | |
| 800 | 800 | |
| 801 | 801 | static const u16 NCT6106_REG_AUTO_TEMP[] = { 0x160, 0x170, 0x180 }; |
| 802 | 802 | static const u16 NCT6106_REG_AUTO_PWM[] = { 0x164, 0x174, 0x184 }; |
| @@ -846,8 +846,6 @@ static const u16 NCT6116_FAN_PULSE_SHIFT[] = { 0, 2, 4, 6, 6 }; · 43 unmodified lines | ||
| 846 | 846 | static const u16 NCT6116_REG_PWM[] = { 0x119, 0x129, 0x139, 0x199, 0x1a9 }; |
| 847 | 847 | static const u16 NCT6116_REG_FAN_MODE[] = { 0x113, 0x123, 0x133, 0x193, 0x1a3 }; |
| 848 | 848 | static const u16 NCT6116_REG_TEMP_SEL[] = { 0x110, 0x120, 0x130, 0x190, 0x1a0 }; |
| 849 | −static const u16 NCT6116_REG_TEMP_SOURCE[] = { | |
| 850 | − 0xb0, 0xb1, 0xb2 }; | |
| 851 | 849 | |
| 852 | 850 | static const u16 NCT6116_REG_CRITICAL_TEMP[] = { |
| 853 | 851 | 0x11a, 0x12a, 0x13a, 0x19a, 0x1aa }; |
| @@ -3652,7 +3650,7 @@ int nct6775_probe(struct device *dev, struct nct6775_data *data, · 2798 unmodified lines | ||
| 3652 | 3650 | = NCT6106_CRITICAL_PWM_ENABLE_MASK; |
| 3653 | 3651 | data->REG_CRITICAL_PWM = NCT6116_REG_CRITICAL_PWM; |
| 3654 | 3652 | data->REG_TEMP_OFFSET = NCT6106_REG_TEMP_OFFSET; |
| 3655 | − data->REG_TEMP_SOURCE = NCT6116_REG_TEMP_SOURCE; | |
| 3653 | + data->REG_TEMP_SOURCE = NCT6106_REG_TEMP_SOURCE; | |
| 3656 | 3654 | data->REG_TEMP_SEL = NCT6116_REG_TEMP_SEL; |
| 3657 | 3655 | data->REG_WEIGHT_TEMP_SEL = NCT6106_REG_WEIGHT_TEMP_SEL; |
| 3658 | 3656 | data->REG_WEIGHT_TEMP[0] = NCT6106_REG_WEIGHT_TEMP_STEP; |
| @@ -3666,13 +3664,13 @@ int nct6775_probe(struct device *dev, struct nct6775_data *data, · 7 unmodified lines | ||
| 3666 | 3664 | |
| 3667 | 3665 | reg_temp = NCT6106_REG_TEMP; |
| 3668 | 3666 | reg_temp_mon = NCT6106_REG_TEMP_MON; |
| 3669 | − num_reg_temp = ARRAY_SIZE(NCT6106_REG_TEMP); | |
| 3667 | + num_reg_temp = 3; | |
| 3670 | 3668 | num_reg_temp_mon = ARRAY_SIZE(NCT6106_REG_TEMP_MON); |
| 3671 | 3669 | num_reg_tsi_temp = ARRAY_SIZE(NCT6116_REG_TSI_TEMP); |
| 3672 | 3670 | reg_temp_over = NCT6106_REG_TEMP_OVER; |
| 3673 | 3671 | reg_temp_hyst = NCT6106_REG_TEMP_HYST; |
| 3674 | 3672 | reg_temp_config = NCT6106_REG_TEMP_CONFIG; |
| 3675 | − num_reg_temp_config = ARRAY_SIZE(NCT6106_REG_TEMP_CONFIG); | |
| 3673 | + num_reg_temp_config = 3; | |
| 3676 | 3674 | reg_temp_alternate = NCT6106_REG_TEMP_ALTERNATE; |
| 3677 | 3675 | reg_temp_crit = NCT6106_REG_TEMP_CRIT; |
| 3678 | 3676 | reg_temp_crit_l = NCT6106_REG_TEMP_CRIT_L; |
| @@ -358,6 +358,11 @@ static void npcm7xx_fan_polling(struct timer_list *t) · 357 unmodified lines | ||
| 358 | 358 | add_timer(&data->fan_timer); |
| 359 | 359 | } |
| 360 | 360 | |
| 361 | +static void npcm7xx_fan_cleanup(void *timer) | |
| 362 | +{ | |
| 363 | + timer_shutdown_sync(timer); | |
| 364 | +} | |
| 365 | + | |
| 361 | 366 | static inline void npcm7xx_fan_compute(struct npcm7xx_pwm_fan_data *data, |
| 362 | 367 | u8 fan, u8 cmp, u8 fan_id, u8 flag_int, |
| 363 | 368 | u8 flag_mode, u8 flag_clear) |
| @@ -1020,6 +1025,12 @@ static int npcm7xx_pwm_fan_probe(struct platform_device *pdev) · 656 unmodified lines | ||
| 1020 | 1025 | msecs_to_jiffies(NPCM7XX_FAN_POLL_TIMER_200MS); |
| 1021 | 1026 | timer_setup(&data->fan_timer, |
| 1022 | 1027 | npcm7xx_fan_polling, 0); |
| 1028 | + ret = devm_add_action_or_reset(dev, | |
| 1029 | + npcm7xx_fan_cleanup, | |
| 1030 | + &data->fan_timer); | |
| 1031 | + if (ret) | |
| 1032 | + return ret; | |
| 1033 | + | |
| 1023 | 1034 | add_timer(&data->fan_timer); |
| 1024 | 1035 | break; |
| 1025 | 1036 | } |
| @@ -203,7 +203,7 @@ struct drvdata { · 202 unmodified lines | ||
| 203 | 203 | */ |
| 204 | 204 | struct mutex mutex; |
| 205 | 205 | long update_interval; |
| 206 | − u8 output_buffer[OUTPUT_REPORT_SIZE]; | |
| 206 | + u8 output_buffer[OUTPUT_REPORT_SIZE] __aligned(ARCH_DMA_MINALIGN); | |
| 207 | 207 | }; |
| 208 | 208 | |
| 209 | 209 | static long scale_pwm_value(long val, long orig_max, long new_max) |
| @@ -754,7 +754,11 @@ static int nzxt_smart2_hid_probe(struct hid_device *hdev, · 544 unmodified lines | ||
| 754 | 754 | |
| 755 | 755 | hid_device_io_start(hdev); |
| 756 | 756 | |
| 757 | − init_device(drvdata, UPDATE_INTERVAL_DEFAULT_MS); | |
| 757 | + ret = init_device(drvdata, UPDATE_INTERVAL_DEFAULT_MS); | |
| 758 | + if (ret) { | |
| 759 | + dev_err(&hdev->dev, "init_device failed: %d\n", ret); | |
| 760 | + goto out_hw_close; | |
| 761 | + } | |
| 758 | 762 | |
| 759 | 763 | drvdata->hwmon = |
| 760 | 764 | hwmon_device_register_with_info(&hdev->dev, "nzxtsmart2", drvdata, |
| @@ -14,6 +14,7 @@ · 13 unmodified lines | ||
| 14 | 14 | #include <linux/slab.h> |
| 15 | 15 | #include <linux/i2c.h> |
| 16 | 16 | #include <linux/log2.h> |
| 17 | +#include <linux/math.h> | |
| 17 | 18 | #include <linux/of.h> |
| 18 | 19 | #include "pmbus.h" |
| 19 | 20 | |
| @@ -542,8 +543,8 @@ static int lm25066_probe(struct i2c_client *client) · 522 unmodified lines | ||
| 542 | 543 | if (of_property_read_u32(client->dev.of_node, "shunt-resistor-micro-ohms", &shunt)) |
| 543 | 544 | shunt = 1000; |
| 544 | 545 | |
| 545 | − info->m[PSC_CURRENT_IN] = info->m[PSC_CURRENT_IN] * shunt / 1000; | |
| 546 | − info->m[PSC_POWER] = info->m[PSC_POWER] * shunt / 1000; | |
| 546 | + info->m[PSC_CURRENT_IN] = DIV_ROUND_CLOSEST_ULL((u64)info->m[PSC_CURRENT_IN] * shunt, 1000); | |
| 547 | + info->m[PSC_POWER] = DIV_ROUND_CLOSEST_ULL((u64)info->m[PSC_POWER] * shunt, 1000); | |
| 547 | 548 | |
| 548 | 549 | #if IS_ENABLED(CONFIG_SENSORS_LM25066_REGULATOR) |
| 549 | 550 | /* LM25056 doesn't support OPERATION */ |
| @@ -45,7 +45,7 @@ module_param(wp, int, 0444); · 44 unmodified lines | ||
| 45 | 45 | struct pmbus_sensor { |
| 46 | 46 | struct pmbus_sensor *next; |
| 47 | 47 | char name[PMBUS_NAME_SIZE]; /* sysfs sensor name */ |
| 48 | − struct device_attribute attribute; | |
| 48 | + struct sensor_device_attribute attribute; | |
| 49 | 49 | u8 page; /* page number */ |
| 50 | 50 | u8 phase; /* phase number, 0xff for all phases */ |
| 51 | 51 | u16 reg; /* register */ |
| @@ -68,7 +68,7 @@ struct pmbus_boolean { · 16 unmodified lines | ||
| 68 | 68 | |
| 69 | 69 | struct pmbus_label { |
| 70 | 70 | char name[PMBUS_NAME_SIZE]; /* sysfs label name */ |
| 71 | − struct device_attribute attribute; | |
| 71 | + struct sensor_device_attribute attribute; | |
| 72 | 72 | char label[PMBUS_NAME_SIZE]; /* label */ |
| 73 | 73 | }; |
| 74 | 74 | #define to_pmbus_label(_attr) \ |
| @@ -513,7 +513,7 @@ int pmbus_update_byte_data(struct i2c_client *client, int page, u8 reg, · 438 unmodified lines | ||
| 513 | 513 | if (tmp != rv) |
| 514 | 514 | rv = _pmbus_write_byte_data(client, page, reg, tmp); |
| 515 | 515 | |
| 516 | − return rv; | |
| 516 | + return rv < 0 ? rv : 0; | |
| 517 | 517 | } |
| 518 | 518 | EXPORT_SYMBOL_NS_GPL(pmbus_update_byte_data, "PMBUS"); |
| 519 | 519 | |
| @@ -1241,7 +1241,8 @@ static ssize_t pmbus_show_sensor(struct device *dev, · 721 unmodified lines | ||
| 1241 | 1241 | struct device_attribute *devattr, char *buf) |
| 1242 | 1242 | { |
| 1243 | 1243 | struct i2c_client *client = to_i2c_client(dev->parent); |
| 1244 | − struct pmbus_sensor *sensor = to_pmbus_sensor(devattr); | |
| 1244 | + struct sensor_device_attribute *attr = to_sensor_dev_attr(devattr); | |
| 1245 | + struct pmbus_sensor *sensor = to_pmbus_sensor(attr); | |
| 1245 | 1246 | struct pmbus_data *data = i2c_get_clientdata(client); |
| 1246 | 1247 | s64 val; |
| 1247 | 1248 | |
| @@ -1261,7 +1262,8 @@ static ssize_t pmbus_set_sensor(struct device *dev, · 13 unmodified lines | ||
| 1261 | 1262 | { |
| 1262 | 1263 | struct i2c_client *client = to_i2c_client(dev->parent); |
| 1263 | 1264 | struct pmbus_data *data = i2c_get_clientdata(client); |
| 1264 | − struct pmbus_sensor *sensor = to_pmbus_sensor(devattr); | |
| 1265 | + struct sensor_device_attribute *attr = to_sensor_dev_attr(devattr); | |
| 1266 | + struct pmbus_sensor *sensor = to_pmbus_sensor(attr); | |
| 1265 | 1267 | s64 val; |
| 1266 | 1268 | int ret; |
| 1267 | 1269 | u16 regval; |
| @@ -1283,7 +1285,8 @@ static ssize_t pmbus_set_sensor(struct device *dev, · 15 unmodified lines | ||
| 1283 | 1285 | static ssize_t pmbus_show_label(struct device *dev, |
| 1284 | 1286 | struct device_attribute *da, char *buf) |
| 1285 | 1287 | { |
| 1286 | − struct pmbus_label *label = to_pmbus_label(da); | |
| 1288 | + struct sensor_device_attribute *attr = to_sensor_dev_attr(da); | |
| 1289 | + struct pmbus_label *label = to_pmbus_label(attr); | |
| 1287 | 1290 | |
| 1288 | 1291 | return sysfs_emit(buf, "%s\n", label->label); |
| 1289 | 1292 | } |
| @@ -1436,8 +1439,8 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data, · 146 unmodified lines | ||
| 1436 | 1439 | bool update, bool readonly, |
| 1437 | 1440 | bool writeonly, bool convert) |
| 1438 | 1441 | { |
| 1442 | + struct sensor_device_attribute *a; | |
| 1439 | 1443 | struct pmbus_sensor *sensor; |
| 1440 | − struct device_attribute *a; | |
| 1441 | 1444 | |
| 1442 | 1445 | sensor = devm_kzalloc(data->dev, sizeof(*sensor), GFP_KERNEL); |
| 1443 | 1446 | if (!sensor) |
| @@ -1461,12 +1464,11 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data, · 17 unmodified lines | ||
| 1461 | 1464 | sensor->update = update; |
| 1462 | 1465 | sensor->convert = convert; |
| 1463 | 1466 | sensor->data = -ENODATA; |
| 1464 | − pmbus_dev_attr_init(a, sensor->name, | |
| 1465 | − readonly ? 0444 : 0644, | |
| 1466 | − writeonly ? pmbus_show_zero : pmbus_show_sensor, | |
| 1467 | − pmbus_set_sensor); | |
| 1467 | + pmbus_attr_init(a, sensor->name, readonly ? 0444 : 0644, | |
| 1468 | + writeonly ? pmbus_show_zero : pmbus_show_sensor, | |
| 1469 | + pmbus_set_sensor, -1); | |
| 1468 | 1470 | |
| 1469 | − if (pmbus_add_attribute(data, &a->attr)) | |
| 1471 | + if (pmbus_add_attribute(data, &a->dev_attr.attr)) | |
| 1470 | 1472 | return NULL; |
| 1471 | 1473 | |
| 1472 | 1474 | sensor->next = data->sensors; |
| @@ -1483,8 +1485,8 @@ static int pmbus_add_label(struct pmbus_data *data, · 10 unmodified lines | ||
| 1483 | 1485 | const char *name, int seq, |
| 1484 | 1486 | const char *lstring, int index, int phase) |
| 1485 | 1487 | { |
| 1488 | + struct sensor_device_attribute *a; | |
| 1486 | 1489 | struct pmbus_label *label; |
| 1487 | − struct device_attribute *a; | |
| 1488 | 1490 | |
| 1489 | 1491 | label = devm_kzalloc(data->dev, sizeof(*label), GFP_KERNEL); |
| 1490 | 1492 | if (!label) |
| @@ -1508,8 +1510,8 @@ static int pmbus_add_label(struct pmbus_data *data, · 17 unmodified lines | ||
| 1508 | 1510 | lstring, index, phase); |
| 1509 | 1511 | } |
| 1510 | 1512 | |
| 1511 | − pmbus_dev_attr_init(a, label->name, 0444, pmbus_show_label, NULL); | |
| 1512 | − return pmbus_add_attribute(data, &a->attr); | |
| 1513 | + pmbus_attr_init(a, label->name, 0444, pmbus_show_label, NULL, -1); | |
| 1514 | + return pmbus_add_attribute(data, &a->dev_attr.attr); | |
| 1513 | 1515 | } |
| 1514 | 1516 | |
| 1515 | 1517 | /* |
| @@ -2397,7 +2399,7 @@ struct pmbus_samples_attr { · 881 unmodified lines | ||
| 2397 | 2399 | struct pmbus_samples_reg { |
| 2398 | 2400 | int page; |
| 2399 | 2401 | struct pmbus_samples_attr *attr; |
| 2400 | − struct device_attribute dev_attr; | |
| 2402 | + struct sensor_device_attribute attribute; | |
| 2401 | 2403 | }; |
| 2402 | 2404 | |
| 2403 | 2405 | static struct pmbus_samples_attr pmbus_samples_registers[] = { |
| @@ -2419,14 +2421,15 @@ static struct pmbus_samples_attr pmbus_samples_registers[] = { · 15 unmodified lines | ||
| 2419 | 2421 | } |
| 2420 | 2422 | }; |
| 2421 | 2423 | |
| 2422 | −#define to_samples_reg(x) container_of(x, struct pmbus_samples_reg, dev_attr) | |
| 2424 | +#define to_samples_reg(x) container_of(x, struct pmbus_samples_reg, attribute) | |
| 2423 | 2425 | |
| 2424 | 2426 | static ssize_t pmbus_show_samples(struct device *dev, |
| 2425 | 2427 | struct device_attribute *devattr, char *buf) |
| 2426 | 2428 | { |
| 2427 | 2429 | int val; |
| 2428 | 2430 | struct i2c_client *client = to_i2c_client(dev->parent); |
| 2429 | − struct pmbus_samples_reg *reg = to_samples_reg(devattr); | |
| 2431 | + struct sensor_device_attribute *attr = to_sensor_dev_attr(devattr); | |
| 2432 | + struct pmbus_samples_reg *reg = to_samples_reg(attr); | |
| 2430 | 2433 | |
| 2431 | 2434 | scoped_guard(pmbus_lock, client) { |
| 2432 | 2435 | val = _pmbus_read_word_data(client, reg->page, 0xff, reg->attr->reg); |
| @@ -2444,7 +2447,8 @@ static ssize_t pmbus_set_samples(struct device *dev, · 11 unmodified lines | ||
| 2444 | 2447 | int ret; |
| 2445 | 2448 | long val; |
| 2446 | 2449 | struct i2c_client *client = to_i2c_client(dev->parent); |
| 2447 | − struct pmbus_samples_reg *reg = to_samples_reg(devattr); | |
| 2450 | + struct sensor_device_attribute *attr = to_sensor_dev_attr(devattr); | |
| 2451 | + struct pmbus_samples_reg *reg = to_samples_reg(attr); | |
| 2448 | 2452 | |
| 2449 | 2453 | if (kstrtol(buf, 0, &val) < 0) |
| 2450 | 2454 | return -EINVAL; |
| @@ -2459,6 +2463,7 @@ static ssize_t pmbus_set_samples(struct device *dev, · 8 unmodified lines | ||
| 2459 | 2463 | static int pmbus_add_samples_attr(struct pmbus_data *data, int page, |
| 2460 | 2464 | struct pmbus_samples_attr *attr) |
| 2461 | 2465 | { |
| 2466 | + struct sensor_device_attribute *a; | |
| 2462 | 2467 | struct pmbus_samples_reg *reg; |
| 2463 | 2468 | |
| 2464 | 2469 | reg = devm_kzalloc(data->dev, sizeof(*reg), GFP_KERNEL); |
| @@ -2468,10 +2473,12 @@ static int pmbus_add_samples_attr(struct pmbus_data *data, int page, · 3 unmodified lines | ||
| 2468 | 2473 | reg->attr = attr; |
| 2469 | 2474 | reg->page = page; |
| 2470 | 2475 | |
| 2471 | − pmbus_dev_attr_init(®->dev_attr, attr->name, 0644, | |
| 2472 | − pmbus_show_samples, pmbus_set_samples); | |
| 2476 | + a = ®->attribute; | |
| 2473 | 2477 | |
| 2474 | − return pmbus_add_attribute(data, ®->dev_attr.attr); | |
| 2478 | + pmbus_attr_init(a, attr->name, 0644, | |
| 2479 | + pmbus_show_samples, pmbus_set_samples, -1); | |
| 2480 | + | |
| 2481 | + return pmbus_add_attribute(data, &a->dev_attr.attr); | |
| 2475 | 2482 | } |
| 2476 | 2483 | |
| 2477 | 2484 | static int pmbus_add_samples_attributes(struct i2c_client *client, |
| @@ -2979,14 +2986,21 @@ static void pmbus_notify(struct pmbus_data *data, int page, int reg, int flags) · 501 unmodified lines | ||
| 2979 | 2986 | struct device_attribute *da = to_dev_attr(data->group.attrs[i]); |
| 2980 | 2987 | struct sensor_device_attribute *attr = to_sensor_dev_attr(da); |
| 2981 | 2988 | int index = attr->index; |
| 2982 | − u16 smask = pb_index_to_mask(index); | |
| 2983 | − u8 spage = pb_index_to_page(index); | |
| 2984 | − u16 sreg = pb_index_to_reg(index); | |
| 2989 | + u16 smask, sreg; | |
| 2990 | + u8 spage; | |
| 2991 | + | |
| 2992 | + if (index == -1) | |
| 2993 | + continue; | |
| 2994 | + | |
| 2995 | + smask = pb_index_to_mask(index); | |
| 2996 | + spage = pb_index_to_page(index); | |
| 2997 | + sreg = pb_index_to_reg(index); | |
| 2985 | 2998 | |
| 2986 | 2999 | if (reg == sreg && page == spage && (smask & flags)) { |
| 2987 | 3000 | dev_dbg(data->dev, "sysfs notify: %s", da->attr.name); |
| 2988 | − sysfs_notify(&data->dev->kobj, NULL, da->attr.name); | |
| 2989 | − kobject_uevent(&data->dev->kobj, KOBJ_CHANGE); | |
| 3001 | + sysfs_notify(&data->hwmon_dev->kobj, NULL, | |
| 3002 | + da->attr.name); | |
| 3003 | + kobject_uevent(&data->hwmon_dev->kobj, KOBJ_CHANGE); | |
| 2990 | 3004 | flags &= ~smask; |
| 2991 | 3005 | } |
| 2992 | 3006 | |
| @@ -3426,6 +3440,8 @@ static int pmbus_write_smbalert_mask(struct i2c_client *client, u8 page, u8 reg, · 433 unmodified lines | ||
| 3426 | 3440 | { |
| 3427 | 3441 | int ret; |
| 3428 | 3442 | |
| 3443 | + guard(pmbus_lock)(client); | |
| 3444 | + | |
| 3429 | 3445 | ret = _pmbus_write_word_data(client, page, PMBUS_SMBALERT_MASK, reg | (val << 8)); |
| 3430 | 3446 | |
| 3431 | 3447 | /* |
| @@ -3661,6 +3677,8 @@ static void pmbus_init_debugfs(struct i2c_client *client, · 229 unmodified lines | ||
| 3661 | 3677 | if (!entries) |
| 3662 | 3678 | return; |
| 3663 | 3679 | |
| 3680 | + guard(pmbus_lock)(client); | |
| 3681 | + | |
| 3664 | 3682 | /* |
| 3665 | 3683 | * Add device-specific entries. |
| 3666 | 3684 | * Please note that the PMBUS standard allows all registers to be |
| @@ -21,6 +21,7 @@ · 20 unmodified lines | ||
| 21 | 21 | #include <linux/module.h> |
| 22 | 22 | #include <linux/slab.h> |
| 23 | 23 | #include <linux/jiffies.h> |
| 24 | +#include <linux/unaligned.h> | |
| 24 | 25 | |
| 25 | 26 | /* commands (high repeatability mode) */ |
| 26 | 27 | static const unsigned char sht3x_cmd_measure_single_hpm[] = { 0x24, 0x00 }; |
| @@ -276,9 +277,9 @@ static struct sht3x_data *sht3x_update_client(struct device *dev) · 249 unmodified lines | ||
| 276 | 277 | if (ret) |
| 277 | 278 | goto out; |
| 278 | 279 | |
| 279 | − val = be16_to_cpup((__be16 *)buf); | |
| 280 | + val = get_unaligned_be16(buf); | |
| 280 | 281 | data->temperature = sht3x_extract_temperature(val); |
| 281 | − val = be16_to_cpup((__be16 *)(buf + 3)); | |
| 282 | + val = get_unaligned_be16(buf + 3); | |
| 282 | 283 | data->humidity = sht3x_extract_humidity(val); |
| 283 | 284 | data->last_update = jiffies; |
| 284 | 285 | } |
| @@ -336,7 +337,7 @@ static int limits_update(struct sht3x_data *data) · 51 unmodified lines | ||
| 336 | 337 | if (ret) |
| 337 | 338 | return ret; |
| 338 | 339 | |
| 339 | − raw = be16_to_cpup((__be16 *)buffer); | |
| 340 | + raw = get_unaligned_be16(buffer); | |
| 340 | 341 | temperature = sht3x_extract_temperature((raw & 0x01ff) << 7); |
| 341 | 342 | humidity = sht3x_extract_humidity(raw & 0xfe00); |
| 342 | 343 | data->temperature_limits[index] = temperature; |
| @@ -389,7 +390,7 @@ static size_t limit_write(struct device *dev, · 46 unmodified lines | ||
| 389 | 390 | raw = ((u32)(temperature + 45000) * 24543) >> (16 + 7); |
| 390 | 391 | raw |= ((humidity * 42950) >> 16) & 0xfe00; |
| 391 | 392 | |
| 392 | − *((__be16 *)position) = cpu_to_be16(raw); | |
| 393 | + put_unaligned_be16(raw, position); | |
| 393 | 394 | position += SHT3X_WORD_LEN; |
| 394 | 395 | *position = crc8(sht3x_crc8_table, |
| 395 | 396 | position - SHT3X_WORD_LEN, |
| @@ -316,8 +316,10 @@ static int i2c_amd_probe(struct platform_device *pdev) · 315 unmodified lines | ||
| 316 | 316 | |
| 317 | 317 | amd_mp2_pm_runtime_put(mp2_dev); |
| 318 | 318 | |
| 319 | − if (ret < 0) | |
| 319 | + if (ret < 0) { | |
| 320 | 320 | dev_err(&pdev->dev, "i2c add adapter failed = %d\n", ret); |
| 321 | + amd_mp2_unregister_cb(&i2c_dev->common); | |
| 322 | + } | |
| 321 | 323 | |
| 322 | 324 | return ret; |
| 323 | 325 | } |
| @@ -803,6 +803,17 @@ static int bcm_iproc_i2c_xfer_wait(struct bcm_iproc_i2c_dev *iproc_i2c, · 802 unmodified lines | ||
| 803 | 803 | } |
| 804 | 804 | |
| 805 | 805 | if (!time_left && !iproc_i2c->xfer_is_done) { |
| 806 | + /* | |
| 807 | + * The controller may fail to clear START_BUSY after a timeout, | |
| 808 | + * reset the controller to recover in that case. | |
| 809 | + */ | |
| 810 | + if (!!(iproc_i2c_rd_reg(iproc_i2c, M_CMD_OFFSET) & | |
| 811 | + BIT(M_CMD_START_BUSY_SHIFT))) { | |
| 812 | + bcm_iproc_i2c_enable_disable(iproc_i2c, false); | |
| 813 | + bcm_iproc_i2c_init(iproc_i2c); | |
| 814 | + bcm_iproc_i2c_enable_disable(iproc_i2c, true); | |
| 815 | + } | |
| 816 | + | |
| 806 | 817 | /* flush both TX/RX FIFOs */ |
| 807 | 818 | val = BIT(M_FIFO_RX_FLUSH_SHIFT) | BIT(M_FIFO_TX_FLUSH_SHIFT); |
| 808 | 819 | iproc_i2c_wr_reg(iproc_i2c, M_FIFO_CTRL_OFFSET, val); |
| @@ -8,12 +8,14 @@ · 7 unmodified lines | ||
| 8 | 8 | * Copyright (C) 2007 MontaVista Software Inc. |
| 9 | 9 | * Copyright (C) 2009 Provigent Ltd. |
| 10 | 10 | */ |
| 11 | +#include <linux/acpi.h> | |
| 11 | 12 | #include <linux/clk-provider.h> |
| 12 | 13 | #include <linux/clk.h> |
| 13 | 14 | #include <linux/delay.h> |
| 14 | 15 | #include <linux/dmi.h> |
| 15 | 16 | #include <linux/err.h> |
| 16 | 17 | #include <linux/errno.h> |
| 18 | +#include <linux/gpio/driver.h> | |
| 17 | 19 | #include <linux/i2c.h> |
| 18 | 20 | #include <linux/interrupt.h> |
| 19 | 21 | #include <linux/io.h> |
| @@ -130,6 +132,80 @@ static int i2c_dw_probe_lock_support(struct dw_i2c_dev *dev) · 110 unmodified lines | ||
| 130 | 132 | return 0; |
| 131 | 133 | } |
| 132 | 134 | |
| 135 | +#if defined(CONFIG_ACPI) && defined(CONFIG_GPIOLIB) | |
| 136 | +/* | |
| 137 | + * Check whether an ACPI GpioInt resource's referenced GPIO controller | |
| 138 | + * has finished probing. Resources with no named controller (resource | |
| 139 | + * source string) are skipped, since they can't be resolved to a | |
| 140 | + * struct device. | |
| 141 | + */ | |
| 142 | +static int check_gpioint_resource(struct acpi_resource *ares, void *data) | |
| 143 | +{ | |
| 144 | + struct acpi_resource_gpio *agpio; | |
| 145 | + struct acpi_device *gpio_adev; | |
| 146 | + struct device *gpio_dev; | |
| 147 | + acpi_handle handle; | |
| 148 | + acpi_status status; | |
| 149 | + | |
| 150 | + if (!acpi_gpio_get_irq_resource(ares, &agpio)) | |
| 151 | + return 1; /* not a GpioInt resource, skip */ | |
| 152 | + | |
| 153 | + if (!agpio->resource_source.string_length) | |
| 154 | + return 1; /* no named controller, skip */ | |
| 155 | + | |
| 156 | + status = acpi_get_handle(NULL, agpio->resource_source.string_ptr, &handle); | |
| 157 | + if (ACPI_FAILURE(status)) | |
| 158 | + return 1; | |
| 159 | + | |
| 160 | + gpio_adev = acpi_fetch_acpi_dev(handle); | |
| 161 | + if (!gpio_adev) | |
| 162 | + return 1; | |
| 163 | + | |
| 164 | + struct gpio_device *gdev __free(gpio_device_put) = | |
| 165 | + gpio_device_find_by_fwnode(acpi_fwnode_handle(gpio_adev)); | |
| 166 | + if (!gdev) | |
| 167 | + return -EPROBE_DEFER; /* controller not registered yet: abort walk */ | |
| 168 | + | |
| 169 | + gpio_dev = gpio_device_to_device(gdev)->parent; | |
| 170 | + | |
| 171 | + guard(device)(gpio_dev); | |
| 172 | + if (!device_is_bound(gpio_dev)) | |
| 173 | + return -EPROBE_DEFER; /* controller not bound yet: abort walk */ | |
| 174 | + | |
| 175 | + return 1; /* bound, skip adding to resource list, continue walk */ | |
| 176 | +} | |
| 177 | + | |
| 178 | +static int check_child_gpioint(struct acpi_device *adev, void *data) | |
| 179 | +{ | |
| 180 | + LIST_HEAD(res_list); | |
| 181 | + int ret; | |
| 182 | + | |
| 183 | + ret = acpi_dev_get_resources(adev, &res_list, check_gpioint_resource, NULL); | |
| 184 | + if (ret < 0) | |
| 185 | + return ret; | |
| 186 | + | |
| 187 | + acpi_dev_free_resource_list(&res_list); | |
| 188 | + | |
| 189 | + return 0; | |
| 190 | +} | |
| 191 | + | |
| 192 | +static int i2c_dw_check_gpio_dependencies(struct device *dev) | |
| 193 | +{ | |
| 194 | + struct acpi_device *adev; | |
| 195 | + | |
| 196 | + adev = ACPI_COMPANION(dev); | |
| 197 | + if (!adev) | |
| 198 | + return 0; | |
| 199 | + | |
| 200 | + return acpi_dev_for_each_child(adev, check_child_gpioint, NULL); | |
| 201 | +} | |
| 202 | +#else | |
| 203 | +static int i2c_dw_check_gpio_dependencies(struct device *dev) | |
| 204 | +{ | |
| 205 | + return 0; | |
| 206 | +} | |
| 207 | +#endif /* CONFIG_ACPI && CONFIG_GPIOLIB */ | |
| 208 | + | |
| 133 | 209 | static int dw_i2c_plat_probe(struct platform_device *pdev) |
| 134 | 210 | { |
| 135 | 211 | u32 flags = (uintptr_t)device_get_match_data(&pdev->dev); |
| @@ -138,6 +214,10 @@ static int dw_i2c_plat_probe(struct platform_device *pdev) · 2 unmodified lines | ||
| 138 | 214 | struct dw_i2c_dev *dev; |
| 139 | 215 | int irq, ret; |
| 140 | 216 | |
| 217 | + ret = i2c_dw_check_gpio_dependencies(device); | |
| 218 | + if (ret) | |
| 219 | + return ret; | |
| 220 | + | |
| 141 | 221 | irq = platform_get_irq_optional(pdev, 0); |
| 142 | 222 | if (irq == -ENXIO) |
| 143 | 223 | flags |= ACCESS_POLLING; |
| @@ -930,9 +930,6 @@ static int i2c_imx_reg_slave(struct i2c_client *client) · 929 unmodified lines | ||
| 930 | 930 | if (i2c_imx->slave) |
| 931 | 931 | return -EBUSY; |
| 932 | 932 | |
| 933 | − i2c_imx->slave = client; | |
| 934 | − i2c_imx->last_slave_event = I2C_SLAVE_STOP; | |
| 935 | − | |
| 936 | 933 | /* Resume */ |
| 937 | 934 | ret = pm_runtime_resume_and_get(i2c_imx->adapter.dev.parent); |
| 938 | 935 | if (ret < 0) { |
| @@ -940,6 +937,11 @@ static int i2c_imx_reg_slave(struct i2c_client *client) · 1 unmodified line | ||
| 940 | 937 | return ret; |
| 941 | 938 | } |
| 942 | 939 | |
| 940 | + scoped_guard(spinlock_irqsave, &i2c_imx->slave_lock) { | |
| 941 | + i2c_imx->slave = client; | |
| 942 | + i2c_imx->last_slave_event = I2C_SLAVE_STOP; | |
| 943 | + } | |
| 944 | + | |
| 943 | 945 | i2c_imx_slave_init(i2c_imx); |
| 944 | 946 | |
| 945 | 947 | return 0; |
| @@ -958,6 +960,7 @@ static int i2c_imx_unreg_slave(struct i2c_client *client) · 12 unmodified lines | ||
| 958 | 960 | |
| 959 | 961 | i2c_imx_reset_regs(i2c_imx); |
| 960 | 962 | |
| 963 | + hrtimer_cancel(&i2c_imx->slave_timer); | |
| 961 | 964 | i2c_imx->slave = NULL; |
| 962 | 965 | |
| 963 | 966 | /* Suspend */ |
| @@ -1952,6 +1955,47 @@ static int i2c_imx_runtime_resume(struct device *dev) · 988 unmodified lines | ||
| 1952 | 1955 | return 0; |
| 1953 | 1956 | } |
| 1954 | 1957 | |
| 1958 | +static int __maybe_unused i2c_imx_suspend_noirq(struct device *dev) | |
| 1959 | +{ | |
| 1960 | + struct imx_i2c_struct *i2c_imx = dev_get_drvdata(dev); | |
| 1961 | + int ret; | |
| 1962 | + | |
| 1963 | + i2c_mark_adapter_suspended(&i2c_imx->adapter); | |
| 1964 | + | |
| 1965 | + /* | |
| 1966 | + * Cancel the slave timer before powering down to prevent | |
| 1967 | + * i2c_imx_slave_timeout() from accessing hardware registers | |
| 1968 | + * while the clock is disabled. | |
| 1969 | + */ | |
| 1970 | + hrtimer_cancel(&i2c_imx->slave_timer); | |
| 1971 | + | |
| 1972 | + ret = pm_runtime_force_suspend(dev); | |
| 1973 | + if (ret) { | |
| 1974 | + i2c_mark_adapter_resumed(&i2c_imx->adapter); | |
| 1975 | + if (i2c_imx->slave) { | |
| 1976 | + hrtimer_forward_now(&i2c_imx->slave_timer, I2C_IMX_CHECK_DELAY); | |
| 1977 | + hrtimer_restart(&i2c_imx->slave_timer); | |
| 1978 | + } | |
| 1979 | + return ret; | |
| 1980 | + } | |
| 1981 | + | |
| 1982 | + return 0; | |
| 1983 | +} | |
| 1984 | + | |
| 1985 | +static int __maybe_unused i2c_imx_resume_noirq(struct device *dev) | |
| 1986 | +{ | |
| 1987 | + struct imx_i2c_struct *i2c_imx = dev_get_drvdata(dev); | |
| 1988 | + int ret; | |
| 1989 | + | |
| 1990 | + ret = pm_runtime_force_resume(dev); | |
| 1991 | + if (ret) | |
| 1992 | + return ret; | |
| 1993 | + | |
| 1994 | + i2c_mark_adapter_resumed(&i2c_imx->adapter); | |
| 1995 | + | |
| 1996 | + return 0; | |
| 1997 | +} | |
| 1998 | + | |
| 1955 | 1999 | static int i2c_imx_suspend(struct device *dev) |
| 1956 | 2000 | { |
| 1957 | 2001 | /* |
| @@ -1985,8 +2029,8 @@ static int i2c_imx_resume(struct device *dev) · 27 unmodified lines | ||
| 1985 | 2029 | } |
| 1986 | 2030 | |
| 1987 | 2031 | static const struct dev_pm_ops i2c_imx_pm_ops = { |
| 1988 | − NOIRQ_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, | |
| 1989 | − pm_runtime_force_resume) | |
| 2032 | + NOIRQ_SYSTEM_SLEEP_PM_OPS(i2c_imx_suspend_noirq, | |
| 2033 | + i2c_imx_resume_noirq) | |
| 1990 | 2034 | SYSTEM_SLEEP_PM_OPS(i2c_imx_suspend, i2c_imx_resume) |
| 1991 | 2035 | RUNTIME_PM_OPS(i2c_imx_runtime_suspend, i2c_imx_runtime_resume, NULL) |
| 1992 | 2036 | }; |
| @@ -141,6 +141,7 @@ struct jz4780_i2c { · 140 unmodified lines | ||
| 141 | 141 | void __iomem *iomem; |
| 142 | 142 | int irq; |
| 143 | 143 | struct clk *clk; |
| 144 | + unsigned long clk_rate_khz; | |
| 144 | 145 | struct i2c_adapter adap; |
| 145 | 146 | const struct ingenic_i2c_config *cdata; |
| 146 | 147 | |
| @@ -246,7 +247,7 @@ static int jz4780_i2c_set_target(struct jz4780_i2c *i2c, unsigned char address) · 99 unmodified lines | ||
| 246 | 247 | |
| 247 | 248 | static int jz4780_i2c_set_speed(struct jz4780_i2c *i2c) |
| 248 | 249 | { |
| 249 | − int dev_clk_khz = clk_get_rate(i2c->clk) / 1000; | |
| 250 | + int dev_clk_khz = i2c->clk_rate_khz; | |
| 250 | 251 | int cnt_high = 0; /* HIGH period count of the SCL clock */ |
| 251 | 252 | int cnt_low = 0; /* LOW period count of the SCL clock */ |
| 252 | 253 | int cnt_period = 0; /* period count of the SCL clock */ |
| @@ -796,6 +797,8 @@ static int jz4780_i2c_probe(struct platform_device *pdev) · 543 unmodified lines | ||
| 796 | 797 | if (IS_ERR(i2c->clk)) |
| 797 | 798 | return PTR_ERR(i2c->clk); |
| 798 | 799 | |
| 800 | + i2c->clk_rate_khz = clk_get_rate(i2c->clk) / 1000; | |
| 801 | + | |
| 799 | 802 | ret = of_property_read_u32(pdev->dev.of_node, "clock-frequency", |
| 800 | 803 | &clk_freq); |
| 801 | 804 | if (ret) { |
| @@ -723,11 +723,6 @@ static int spacemit_i2c_probe(struct platform_device *pdev) · 722 unmodified lines | ||
| 723 | 723 | if (i2c->irq < 0) |
| 724 | 724 | return dev_err_probe(dev, i2c->irq, "failed to get irq resource"); |
| 725 | 725 | |
| 726 | − ret = devm_request_irq(i2c->dev, i2c->irq, spacemit_i2c_irq_handler, | |
| 727 | − IRQF_NO_SUSPEND, dev_name(i2c->dev), i2c); | |
| 728 | − if (ret) | |
| 729 | − return dev_err_probe(dev, ret, "failed to request irq"); | |
| 730 | − | |
| 731 | 726 | clk = devm_clk_get_enabled(dev, "func"); |
| 732 | 727 | if (IS_ERR(clk)) |
| 733 | 728 | return dev_err_probe(dev, PTR_ERR(clk), "failed to enable func clock"); |
| @@ -755,6 +750,11 @@ static int spacemit_i2c_probe(struct platform_device *pdev) · 21 unmodified lines | ||
| 755 | 750 | |
| 756 | 751 | init_completion(&i2c->complete); |
| 757 | 752 | |
| 753 | + ret = devm_request_irq(i2c->dev, i2c->irq, spacemit_i2c_irq_handler, | |
| 754 | + IRQF_NO_SUSPEND, dev_name(i2c->dev), i2c); | |
| 755 | + if (ret) | |
| 756 | + return dev_err_probe(dev, ret, "failed to request irq"); | |
| 757 | + | |
| 758 | 758 | platform_set_drvdata(pdev, i2c); |
| 759 | 759 | |
| 760 | 760 | ret = i2c_add_numbered_adapter(&i2c->adapt); |
| @@ -492,24 +492,8 @@ static int __maybe_unused cci_resume_runtime(struct device *dev) · 491 unmodified lines | ||
| 492 | 492 | return 0; |
| 493 | 493 | } |
| 494 | 494 | |
| 495 | −static int __maybe_unused cci_suspend(struct device *dev) | |
| 496 | −{ | |
| 497 | − if (!pm_runtime_suspended(dev)) | |
| 498 | − return cci_suspend_runtime(dev); | |
| 499 | − | |
| 500 | − return 0; | |
| 501 | −} | |
| 502 | − | |
| 503 | −static int __maybe_unused cci_resume(struct device *dev) | |
| 504 | −{ | |
| 505 | − cci_resume_runtime(dev); | |
| 506 | − pm_request_autosuspend(dev); | |
| 507 | − | |
| 508 | − return 0; | |
| 509 | −} | |
| 510 | − | |
| 511 | 495 | static const struct dev_pm_ops qcom_cci_pm = { |
| 512 | − SET_SYSTEM_SLEEP_PM_OPS(cci_suspend, cci_resume) | |
| 496 | + SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, pm_runtime_force_resume) | |
| 513 | 497 | SET_RUNTIME_PM_OPS(cci_suspend_runtime, cci_resume_runtime, NULL) |
| 514 | 498 | }; |
| 515 | 499 | |
| @@ -21,6 +21,7 @@ · 20 unmodified lines | ||
| 21 | 21 | #include <linux/init.h> |
| 22 | 22 | #include <linux/input/mt.h> |
| 23 | 23 | #include <linux/major.h> |
| 24 | +#include <linux/nospec.h> | |
| 24 | 25 | #include <linux/device.h> |
| 25 | 26 | #include <linux/cdev.h> |
| 26 | 27 | #include "input-compat.h" |
| @@ -67,8 +68,10 @@ static size_t evdev_get_mask_cnt(unsigned int type) · 40 unmodified lines | ||
| 67 | 68 | [EV_SND] = SND_CNT, |
| 68 | 69 | [EV_FF] = FF_CNT, |
| 69 | 70 | }; |
| 71 | + unsigned long mask = array_index_mask_nospec(type, EV_CNT); | |
| 70 | 72 | |
| 71 | − return (type < EV_CNT) ? counts[type] : 0; | |
| 73 | + /* Returns 0 for out-of-bounds types, including speculatively */ | |
| 74 | + return counts[type & mask] & mask; | |
| 72 | 75 | } |
| 73 | 76 | |
| 74 | 77 | /* requires the buffer lock to be held */ |
| @@ -146,11 +149,11 @@ static void __evdev_queue_syn_dropped(struct evdev_client *client) · 71 unmodified lines | ||
| 146 | 149 | struct timespec64 ts = ktime_to_timespec64(ev_time[client->clk_type]); |
| 147 | 150 | struct input_event ev; |
| 148 | 151 | |
| 152 | + memset(&ev, 0, sizeof(ev)); | |
| 149 | 153 | ev.input_event_sec = ts.tv_sec; |
| 150 | 154 | ev.input_event_usec = ts.tv_nsec / NSEC_PER_USEC; |
| 151 | 155 | ev.type = EV_SYN; |
| 152 | 156 | ev.code = SYN_DROPPED; |
| 153 | − ev.value = 0; | |
| 154 | 157 | |
| 155 | 158 | client->buffer[client->head++] = ev; |
| 156 | 159 | client->head &= client->bufsize - 1; |
| @@ -218,20 +221,20 @@ static void __pass_event(struct evdev_client *client, · 61 unmodified lines | ||
| 218 | 221 | client->head &= client->bufsize - 1; |
| 219 | 222 | |
| 220 | 223 | if (unlikely(client->head == client->tail)) { |
| 224 | + struct input_event ev; | |
| 225 | + | |
| 226 | + memset(&ev, 0, sizeof(ev)); | |
| 227 | + ev.input_event_sec = event->input_event_sec; | |
| 228 | + ev.input_event_usec = event->input_event_usec; | |
| 229 | + ev.type = EV_SYN; | |
| 230 | + ev.code = SYN_DROPPED; | |
| 231 | + | |
| 221 | 232 | /* |
| 222 | 233 | * This effectively "drops" all unconsumed events, leaving |
| 223 | 234 | * EV_SYN/SYN_DROPPED plus the newest event in the queue. |
| 224 | 235 | */ |
| 225 | 236 | client->tail = (client->head - 2) & (client->bufsize - 1); |
| 226 | − | |
| 227 | − client->buffer[client->tail] = (struct input_event) { | |
| 228 | − .input_event_sec = event->input_event_sec, | |
| 229 | − .input_event_usec = event->input_event_usec, | |
| 230 | − .type = EV_SYN, | |
| 231 | − .code = SYN_DROPPED, | |
| 232 | − .value = 0, | |
| 233 | − }; | |
| 234 | − | |
| 237 | + client->buffer[client->tail] = ev; | |
| 235 | 238 | client->packet_head = client->tail; |
| 236 | 239 | } |
| 237 | 240 | |
| @@ -253,6 +256,8 @@ static void evdev_pass_values(struct evdev_client *client, · 15 unmodified lines | ||
| 253 | 256 | if (client->revoked) |
| 254 | 257 | return; |
| 255 | 258 | |
| 259 | + memset(&event, 0, sizeof(event)); | |
| 260 | + | |
| 256 | 261 | ts = ktime_to_timespec64(ev_time[client->clk_type]); |
| 257 | 262 | event.input_event_sec = ts.tv_sec; |
| 258 | 263 | event.input_event_usec = ts.tv_nsec / NSEC_PER_USEC; |
| @@ -155,6 +155,9 @@ void iforce_process_packet(struct iforce *iforce, · 154 unmodified lines | ||
| 155 | 155 | switch (packet_id) { |
| 156 | 156 | |
| 157 | 157 | case 0x01: /* joystick position data */ |
| 158 | + if (len < 7) | |
| 159 | + break; | |
| 160 | + | |
| 158 | 161 | input_report_abs(dev, ABS_X, |
| 159 | 162 | (__s16) get_unaligned_le16(data)); |
| 160 | 163 | input_report_abs(dev, ABS_Y, |
| @@ -170,6 +173,9 @@ void iforce_process_packet(struct iforce *iforce, · 9 unmodified lines | ||
| 170 | 173 | break; |
| 171 | 174 | |
| 172 | 175 | case 0x03: /* wheel position data */ |
| 176 | + if (len < 7) | |
| 177 | + break; | |
| 178 | + | |
| 173 | 179 | input_report_abs(dev, ABS_WHEEL, |
| 174 | 180 | (__s16) get_unaligned_le16(data)); |
| 175 | 181 | input_report_abs(dev, ABS_GAS, 255 - data[2]); |
| @@ -181,6 +187,9 @@ void iforce_process_packet(struct iforce *iforce, · 5 unmodified lines | ||
| 181 | 187 | break; |
| 182 | 188 | |
| 183 | 189 | case 0x02: /* status report */ |
| 190 | + if (len < 2) | |
| 191 | + break; | |
| 192 | + | |
| 184 | 193 | input_report_key(dev, BTN_DEAD, data[0] & 0x02); |
| 185 | 194 | input_sync(dev); |
| 186 | 195 | |
| @@ -200,7 +209,7 @@ void iforce_process_packet(struct iforce *iforce, · 13 unmodified lines | ||
| 200 | 209 | } |
| 201 | 210 | } |
| 202 | 211 | |
| 203 | − for (j = 3; j < len; j += 2) | |
| 212 | + for (j = 3; j + sizeof(u16) <= len; j += sizeof(u16)) | |
| 204 | 213 | mark_core_as_ready(iforce, get_unaligned_le16(data + j)); |
| 205 | 214 | |
| 206 | 215 | break; |
| @@ -158,6 +158,9 @@ static void iforce_usb_irq(struct urb *urb) · 157 unmodified lines | ||
| 158 | 158 | goto exit; |
| 159 | 159 | } |
| 160 | 160 | |
| 161 | + if (!urb->actual_length) | |
| 162 | + goto exit; | |
| 163 | + | |
| 161 | 164 | iforce_process_packet(iforce, iforce_usb->data_in[0], |
| 162 | 165 | iforce_usb->data_in + 1, urb->actual_length - 1); |
| 163 | 166 | |
| @@ -373,6 +373,7 @@ static int psxpad_spi_probe(struct spi_device *spi) · 372 unmodified lines | ||
| 373 | 373 | return err; |
| 374 | 374 | } |
| 375 | 375 | |
| 376 | + spi_set_drvdata(spi, pad); | |
| 376 | 377 | pm_runtime_enable(&spi->dev); |
| 377 | 378 | |
| 378 | 379 | return 0; |
| @@ -394,6 +394,7 @@ static const struct xpad_device { · 393 unmodified lines | ||
| 394 | 394 | { 0x3285, 0x0646, "Nacon Pro Compact", 0, XTYPE_XBOXONE }, |
| 395 | 395 | { 0x3285, 0x0662, "Nacon Revolution5 Pro", 0, XTYPE_XBOX360 }, |
| 396 | 396 | { 0x3285, 0x0663, "Nacon Evol-X", 0, XTYPE_XBOXONE }, |
| 397 | + { 0x3507, 0x000b, "ZENAIM LEVERLESS", 0, XTYPE_XBOX360 }, | |
| 397 | 398 | { 0x3537, 0x1004, "GameSir T4 Kaleid", 0, XTYPE_XBOX360 }, |
| 398 | 399 | { 0x3537, 0x100f, "GameSir Nova 2 Lite", 0, XTYPE_XBOX360 }, |
| 399 | 400 | { 0x3537, 0x1010, "GameSir G7 SE", 0, XTYPE_XBOXONE }, |
| @@ -557,6 +558,7 @@ static const struct usb_device_id xpad_table[] = { · 157 unmodified lines | ||
| 557 | 558 | XPAD_XBOX360_VENDOR(0x31e3), /* Wooting Keyboards */ |
| 558 | 559 | XPAD_XBOX360_VENDOR(0x3285), /* Nacon GC-100 */ |
| 559 | 560 | XPAD_XBOXONE_VENDOR(0x3285), /* Nacon Evol-X */ |
| 561 | + XPAD_XBOX360_VENDOR(0x3507), /* ZENAIM Controllers */ | |
| 560 | 562 | XPAD_XBOX360_VENDOR(0x3537), /* GameSir Controllers */ |
| 561 | 563 | XPAD_XBOXONE_VENDOR(0x3537), /* GameSir Controllers */ |
| 562 | 564 | XPAD_XBOX360_VENDOR(0x3651), /* CRKD Controllers */ |
| @@ -1916,6 +1916,13 @@ static const struct dmi_system_id atkbd_dmi_quirk_table[] __initconst = { · 1915 unmodified lines | ||
| 1916 | 1916 | }, |
| 1917 | 1917 | .callback = atkbd_deactivate_fixup, |
| 1918 | 1918 | }, |
| 1919 | + { | |
| 1920 | + .matches = { | |
| 1921 | + DMI_MATCH(DMI_SYS_VENDOR, "HONOR"), | |
| 1922 | + DMI_MATCH(DMI_PRODUCT_NAME, "BCC-N"), | |
| 1923 | + }, | |
| 1924 | + .callback = atkbd_deactivate_fixup, | |
| 1925 | + }, | |
| 1919 | 1926 | { |
| 1920 | 1927 | .matches = { |
| 1921 | 1928 | DMI_MATCH(DMI_SYS_VENDOR, "HONOR"), |
| @@ -1923,6 +1930,14 @@ static const struct dmi_system_id atkbd_dmi_quirk_table[] __initconst = { · 1 unmodified line | ||
| 1923 | 1930 | }, |
| 1924 | 1931 | .callback = atkbd_deactivate_fixup, |
| 1925 | 1932 | }, |
| 1933 | + { | |
| 1934 | + /* HONOR MagicBook Pro 14 2026 */ | |
| 1935 | + .matches = { | |
| 1936 | + DMI_MATCH(DMI_SYS_VENDOR, "HONOR"), | |
| 1937 | + DMI_MATCH(DMI_PRODUCT_NAME, "ZQC-P"), | |
| 1938 | + }, | |
| 1939 | + .callback = atkbd_deactivate_fixup, | |
| 1940 | + }, | |
| 1926 | 1941 | { |
| 1927 | 1942 | /* Lenovo Yoga Air 14 (83QK) */ |
| 1928 | 1943 | .matches = { |
| @@ -1932,9 +1947,10 @@ static const struct dmi_system_id atkbd_dmi_quirk_table[] __initconst = { · 3 unmodified lines | ||
| 1932 | 1947 | .callback = atkbd_deactivate_fixup, |
| 1933 | 1948 | }, |
| 1934 | 1949 | { |
| 1950 | + /* Xiaomi Book Pro 14 (TM2424) */ | |
| 1935 | 1951 | .matches = { |
| 1936 | − DMI_MATCH(DMI_SYS_VENDOR, "HONOR"), | |
| 1937 | − DMI_MATCH(DMI_PRODUCT_NAME, "BCC-N"), | |
| 1952 | + DMI_MATCH(DMI_SYS_VENDOR, "XIAOMI"), | |
| 1953 | + DMI_MATCH(DMI_PRODUCT_NAME, "Xiaomi Book Pro 14"), | |
| 1938 | 1954 | }, |
| 1939 | 1955 | .callback = atkbd_deactivate_fixup, |
| 1940 | 1956 | }, |
| @@ -139,10 +139,10 @@ static struct cs40l50_effect *cs40l50_find_effect(int id, struct list_head *effe · 138 unmodified lines | ||
| 139 | 139 | static int cs40l50_effect_bank_set(struct cs40l50_work *work_data, |
| 140 | 140 | struct cs40l50_effect *effect) |
| 141 | 141 | { |
| 142 | − s16 bank_type = work_data->custom_data[0] & CS40L50_CUSTOM_DATA_MASK; | |
| 142 | + u32 bank_type = work_data->custom_data[0] & CS40L50_CUSTOM_DATA_MASK; | |
| 143 | 143 | |
| 144 | 144 | if (bank_type >= CS40L50_WVFRM_BANK_NUM) { |
| 145 | − dev_err(work_data->vib->dev, "Invalid bank (%d)\n", bank_type); | |
| 145 | + dev_err(work_data->vib->dev, "Invalid bank (%u)\n", bank_type); | |
| 146 | 146 | return -EINVAL; |
| 147 | 147 | } |
| 148 | 148 | |
| @@ -326,6 +326,12 @@ static int cs40l50_add(struct input_dev *dev, struct ff_effect *effect, · 177 unmodified lines | ||
| 326 | 326 | return -EINVAL; |
| 327 | 327 | } |
| 328 | 328 | |
| 329 | + if (periodic->custom_len < CS40L50_OWT_CUSTOM_DATA_SIZE) { | |
| 330 | + dev_err(vib->dev, "Invalid custom data length (%u)\n", | |
| 331 | + periodic->custom_len); | |
| 332 | + return -EINVAL; | |
| 333 | + } | |
| 334 | + | |
| 329 | 335 | work_data.custom_data = memdup_array_user(effect->u.periodic.custom_data, |
| 330 | 336 | effect->u.periodic.custom_len, |
| 331 | 337 | sizeof(s16)); |
| @@ -423,7 +423,7 @@ static void byd_disconnect(struct psmouse *psmouse) · 422 unmodified lines | ||
| 423 | 423 | struct byd_data *priv = psmouse->private; |
| 424 | 424 | |
| 425 | 425 | if (priv) { |
| 426 | − timer_delete(&priv->timer); | |
| 426 | + timer_shutdown_sync(&priv->timer); | |
| 427 | 427 | kfree(psmouse->private); |
| 428 | 428 | psmouse->private = NULL; |
| 429 | 429 | } |
| @@ -197,7 +197,7 @@ static void focaltech_process_rel_packet(struct psmouse *psmouse, · 196 unmodified lines | ||
| 197 | 197 | { |
| 198 | 198 | struct focaltech_data *priv = psmouse->private; |
| 199 | 199 | struct focaltech_hw_state *state = &priv->state; |
| 200 | − int finger1, finger2; | |
| 200 | + unsigned int finger1, finger2; | |
| 201 | 201 | |
| 202 | 202 | state->pressed = packet[0] >> 7; |
| 203 | 203 | finger1 = ((packet[0] >> 4) & 0x7) - 1; |
| @@ -104,7 +104,9 @@ struct f54_data { · 103 unmodified lines | ||
| 104 | 104 | |
| 105 | 105 | enum rmi_f54_report_type report_type; |
| 106 | 106 | u8 *report_data; |
| 107 | + size_t max_report_size; | |
| 107 | 108 | int report_size; |
| 109 | + int report_error; | |
| 108 | 110 | |
| 109 | 111 | bool is_busy; |
| 110 | 112 | struct mutex status_mutex; |
| @@ -339,6 +341,12 @@ static void rmi_f54_buffer_queue(struct vb2_buffer *vb) · 228 unmodified lines | ||
| 339 | 341 | mutex_lock(&f54->data_mutex); |
| 340 | 342 | } |
| 341 | 343 | |
| 344 | + if (f54->report_error) { | |
| 345 | + dev_err(&f54->fn->dev, "Error acquiring report: %d\n", f54->report_error); | |
| 346 | + state = VB2_BUF_STATE_ERROR; | |
| 347 | + goto data_done; | |
| 348 | + } | |
| 349 | + | |
| 342 | 350 | ptr = vb2_plane_vaddr(vb, 0); |
| 343 | 351 | if (!ptr) { |
| 344 | 352 | dev_err(&f54->fn->dev, "Error acquiring frame ptr\n"); |
| @@ -444,7 +452,12 @@ static int rmi_f54_set_input(struct f54_data *f54, unsigned int i) · 99 unmodified lines | ||
| 444 | 452 | |
| 445 | 453 | static int rmi_f54_vidioc_s_input(struct file *file, void *priv, unsigned int i) |
| 446 | 454 | { |
| 447 | − return rmi_f54_set_input(video_drvdata(file), i); | |
| 455 | + struct f54_data *f54 = video_drvdata(file); | |
| 456 | + | |
| 457 | + if (vb2_is_busy(&f54->queue)) | |
| 458 | + return -EBUSY; | |
| 459 | + | |
| 460 | + return rmi_f54_set_input(f54, i); | |
| 448 | 461 | } |
| 449 | 462 | |
| 450 | 463 | static int rmi_f54_vidioc_g_input(struct file *file, void *priv, |
| @@ -545,7 +558,14 @@ static void rmi_f54_work(struct work_struct *work) · 94 unmodified lines | ||
| 545 | 558 | dev_err(&fn->dev, "Bad report size, report type=%d\n", |
| 546 | 559 | f54->report_type); |
| 547 | 560 | error = -EINVAL; |
| 548 | − goto error; /* retry won't help */ | |
| 561 | + goto out; /* retry won't help */ | |
| 562 | + } | |
| 563 | + | |
| 564 | + if (report_size > f54->max_report_size) { | |
| 565 | + dev_err(&fn->dev, "Report size %d exceeds buffer size %zu\n", | |
| 566 | + report_size, f54->max_report_size); | |
| 567 | + error = -EINVAL; | |
| 568 | + goto out; | |
| 549 | 569 | } |
| 550 | 570 | |
| 551 | 571 | /* |
| @@ -556,7 +576,7 @@ static void rmi_f54_work(struct work_struct *work) · 4 unmodified lines | ||
| 556 | 576 | &command); |
| 557 | 577 | if (error) { |
| 558 | 578 | dev_err(&fn->dev, "Failed to read back command\n"); |
| 559 | − goto error; | |
| 579 | + goto out; | |
| 560 | 580 | } |
| 561 | 581 | if (command & F54_GET_REPORT) { |
| 562 | 582 | if (time_after(jiffies, f54->timeout)) { |
| @@ -564,7 +584,7 @@ static void rmi_f54_work(struct work_struct *work) · 1 unmodified line | ||
| 564 | 584 | error = -ETIMEDOUT; |
| 565 | 585 | } |
| 566 | 586 | report_size = 0; |
| 567 | − goto error; | |
| 587 | + goto out; | |
| 568 | 588 | } |
| 569 | 589 | |
| 570 | 590 | rmi_dbg(RMI_DEBUG_FN, &fn->dev, "Get report command completed, reading data\n"); |
| @@ -579,7 +599,7 @@ static void rmi_f54_work(struct work_struct *work) · 8 unmodified lines | ||
| 579 | 599 | fifo, sizeof(fifo)); |
| 580 | 600 | if (error) { |
| 581 | 601 | dev_err(&fn->dev, "Failed to set fifo start offset\n"); |
| 582 | − goto abort; | |
| 602 | + goto out; | |
| 583 | 603 | } |
| 584 | 604 | |
| 585 | 605 | error = rmi_read_block(fn->rmi_dev, fn->fd.data_base_addr + |
| @@ -588,16 +608,17 @@ static void rmi_f54_work(struct work_struct *work) · 2 unmodified lines | ||
| 588 | 608 | if (error) { |
| 589 | 609 | dev_err(&fn->dev, "%s: read [%d bytes] returned %d\n", |
| 590 | 610 | __func__, size, error); |
| 591 | − goto abort; | |
| 611 | + goto out; | |
| 592 | 612 | } |
| 593 | 613 | } |
| 594 | 614 | |
| 595 | −abort: | |
| 596 | − f54->report_size = error ? 0 : report_size; | |
| 597 | −error: | |
| 615 | +out: | |
| 598 | 616 | if (error) |
| 599 | 617 | report_size = 0; |
| 600 | 618 | |
| 619 | + f54->report_size = report_size; | |
| 620 | + f54->report_error = error; | |
| 621 | + | |
| 601 | 622 | if (report_size == 0 && !error) { |
| 602 | 623 | queue_delayed_work(f54->workqueue, &f54->work, |
| 603 | 624 | msecs_to_jiffies(1)); |
| @@ -678,8 +699,8 @@ static int rmi_f54_probe(struct rmi_function *fn) · 74 unmodified lines | ||
| 678 | 699 | |
| 679 | 700 | rx = f54->num_rx_electrodes; |
| 680 | 701 | tx = f54->num_tx_electrodes; |
| 681 | − f54->report_data = devm_kzalloc(&fn->dev, | |
| 682 | − array3_size(tx, rx, sizeof(u16)), | |
| 702 | + f54->max_report_size = array3_size(tx, rx, sizeof(u16)); | |
| 703 | + f54->report_data = devm_kzalloc(&fn->dev, f54->max_report_size, | |
| 683 | 704 | GFP_KERNEL); |
| 684 | 705 | if (f54->report_data == NULL) |
| 685 | 706 | return -ENOMEM; |
| @@ -54,10 +54,10 @@ static int rmi_f55_detect(struct rmi_function *fn) · 53 unmodified lines | ||
| 54 | 54 | f55->num_tx_electrodes = f55->qry[F55_NUM_TX_OFFSET]; |
| 55 | 55 | |
| 56 | 56 | f55->cfg_num_rx_electrodes = f55->num_rx_electrodes; |
| 57 | − f55->cfg_num_tx_electrodes = f55->num_rx_electrodes; | |
| 57 | + f55->cfg_num_tx_electrodes = f55->num_tx_electrodes; | |
| 58 | 58 | |
| 59 | 59 | drv_data->num_rx_electrodes = f55->cfg_num_rx_electrodes; |
| 60 | − drv_data->num_tx_electrodes = f55->cfg_num_rx_electrodes; | |
| 60 | + drv_data->num_tx_electrodes = f55->cfg_num_tx_electrodes; | |
| 61 | 61 | |
| 62 | 62 | if (f55->qry[F55_PHYS_CHAR_OFFSET] & F55_CAP_SENSOR_ASSIGN) { |
| 63 | 63 | int i, total; |
| @@ -331,6 +331,8 @@ static irqreturn_t edt_ft5x06_ts_isr(int irq, void *dev_id) · 330 unmodified lines | ||
| 331 | 331 | swap(x, y); |
| 332 | 332 | |
| 333 | 333 | id = (buf[2] >> 4) & 0x0f; |
| 334 | + if (id >= tsdata->max_support_points) | |
| 335 | + continue; | |
| 334 | 336 | |
| 335 | 337 | input_mt_slot(tsdata->input, id); |
| 336 | 338 | if (input_mt_report_slot_state(tsdata->input, MT_TOOL_FINGER, |
| @@ -312,6 +312,12 @@ static void cst3xx_touch_report(struct i2c_client *client) · 311 unmodified lines | ||
| 312 | 312 | return; |
| 313 | 313 | |
| 314 | 314 | touch_cnt = buf[5] & CST3XX_TOUCH_COUNT_MASK; |
| 315 | + if (touch_cnt > ts_data->chip->max_touch_num) { | |
| 316 | + dev_err(&client->dev, "cst3xx invalid touch count (%d vs %d max)\n", | |
| 317 | + touch_cnt, ts_data->chip->max_touch_num); | |
| 318 | + return; | |
| 319 | + } | |
| 320 | + | |
| 315 | 321 | /* |
| 316 | 322 | * Check the check bit of the last touch slot. The check bit is |
| 317 | 323 | * always present after touch point 1 for valid data, and then |
| @@ -334,9 +340,10 @@ static void cst3xx_touch_report(struct i2c_client *client) · 16 unmodified lines | ||
| 334 | 340 | finger_id = (buf[idx] >> 4) & 0x0f; |
| 335 | 341 | |
| 336 | 342 | /* Sanity check we don't have more fingers than we expect */ |
| 337 | − if (ts_data->chip->max_touch_num < finger_id) { | |
| 338 | − dev_err(&client->dev, "cst3xx touch read failure\n"); | |
| 339 | − break; | |
| 343 | + if (finger_id >= ts_data->chip->max_touch_num) { | |
| 344 | + dev_err(&client->dev, | |
| 345 | + "cst3xx invalid finger id %d\n", finger_id); | |
| 346 | + return; | |
| 340 | 347 | } |
| 341 | 348 | |
| 342 | 349 | /* sw value of 0 means no touch, 0x03 means touch */ |
| @@ -297,6 +297,20 @@ unlock: · 296 unmodified lines | ||
| 297 | 297 | return ret; |
| 298 | 298 | } |
| 299 | 299 | |
| 300 | +static int arm_vsmmu_vdevice_init(struct iommufd_vdevice *vdev) | |
| 301 | +{ | |
| 302 | + struct device *dev = iommufd_vdevice_to_device(vdev); | |
| 303 | + struct arm_smmu_master *master = dev_iommu_priv_get(dev); | |
| 304 | + | |
| 305 | + /* | |
| 306 | + * arm_vsmmu_vsid_to_sid() maps a vSID to master->streams[0] alone, so | |
| 307 | + * more streams would leave the rest stale and none reads out of bounds. | |
| 308 | + */ | |
| 309 | + if (master->num_streams != 1) | |
| 310 | + return -EOPNOTSUPP; | |
| 311 | + return 0; | |
| 312 | +} | |
| 313 | + | |
| 300 | 314 | /* This is basically iommu_viommu_arm_smmuv3_invalidate in u64 for conversion */ |
| 301 | 315 | struct arm_vsmmu_invalidation_cmd { |
| 302 | 316 | union { |
| @@ -403,6 +417,7 @@ out: · 100 unmodified lines | ||
| 403 | 417 | static const struct iommufd_viommu_ops arm_vsmmu_ops = { |
| 404 | 418 | .alloc_domain_nested = arm_vsmmu_alloc_domain_nested, |
| 405 | 419 | .cache_invalidate = arm_vsmmu_cache_invalidate, |
| 420 | + .vdevice_init = arm_vsmmu_vdevice_init, | |
| 406 | 421 | }; |
| 407 | 422 | |
| 408 | 423 | size_t arm_smmu_get_viommu_size(struct device *dev, |
| @@ -332,16 +332,30 @@ void iopf_group_response(struct iopf_group *group, · 331 unmodified lines | ||
| 332 | 332 | .code = status, |
| 333 | 333 | }; |
| 334 | 334 | |
| 335 | − /* Only send response if there is a fault report pending */ | |
| 336 | 335 | mutex_lock(&fault_param->lock); |
| 337 | − if (!list_empty(&group->pending_node)) { | |
| 338 | − ops->page_response(dev, &group->last_fault, &resp); | |
| 339 | − list_del_init(&group->pending_node); | |
| 340 | − } | |
| 336 | + ops->page_response(dev, &group->last_fault, &resp); | |
| 337 | + list_del_init(&group->pending_node); | |
| 341 | 338 | mutex_unlock(&fault_param->lock); |
| 342 | 339 | } |
| 343 | 340 | EXPORT_SYMBOL_GPL(iopf_group_response); |
| 344 | 341 | |
| 342 | +/** | |
| 343 | + * iopf_group_dequeue - Dequeue a page fault group from the pending list | |
| 344 | + * @group: the group to dequeue | |
| 345 | + * | |
| 346 | + * The fault handler is responsible for responding to the group after | |
| 347 | + * this function returns. | |
| 348 | + */ | |
| 349 | +void iopf_group_dequeue(struct iopf_group *group) | |
| 350 | +{ | |
| 351 | + struct iommu_fault_param *fault_param = group->fault_param; | |
| 352 | + | |
| 353 | + mutex_lock(&fault_param->lock); | |
| 354 | + list_del_init(&group->pending_node); | |
| 355 | + mutex_unlock(&fault_param->lock); | |
| 356 | +} | |
| 357 | +EXPORT_SYMBOL_GPL(iopf_group_dequeue); | |
| 358 | + | |
| 345 | 359 | /** |
| 346 | 360 | * iopf_queue_discard_partial - Remove all pending partial fault |
| 347 | 361 | * @queue: the queue whose partial faults need to be discarded |
| @@ -589,7 +589,7 @@ static int iommufd_hwpt_replace_device(struct iommufd_device *idev, · 588 unmodified lines | ||
| 589 | 589 | if (rc) |
| 590 | 590 | goto out_free_handle; |
| 591 | 591 | |
| 592 | − iommufd_auto_response_faults(hwpt, old_handle); | |
| 592 | + iommufd_auto_response_faults(old, old_handle); | |
| 593 | 593 | kfree(old_handle); |
| 594 | 594 | |
| 595 | 595 | return 0; |
| @@ -484,6 +484,8 @@ int iommufd_fault_iopf_handler(struct iopf_group *group) · 483 unmodified lines | ||
| 484 | 484 | hwpt = group->attach_handle->domain->iommufd_hwpt; |
| 485 | 485 | fault = hwpt->fault; |
| 486 | 486 | |
| 487 | + iopf_group_dequeue(group); | |
| 488 | + | |
| 487 | 489 | spin_lock(&fault->common.lock); |
| 488 | 490 | list_add_tail(&group->node, &fault->common.deliver); |
| 489 | 491 | spin_unlock(&fault->common.lock); |
| @@ -2451,6 +2451,9 @@ int iopt_area_add_access(struct iopt_area *area, unsigned long start_index, · 2450 unmodified lines | ||
| 2451 | 2451 | if ((flags & IOMMUFD_ACCESS_RW_WRITE) && !pages->writable) |
| 2452 | 2452 | return -EPERM; |
| 2453 | 2453 | |
| 2454 | + if (iopt_is_dmabuf(pages)) | |
| 2455 | + return -EINVAL; | |
| 2456 | + | |
| 2454 | 2457 | mutex_lock(&pages->mutex); |
| 2455 | 2458 | access = iopt_pages_get_exact_access(pages, start_index, last_index); |
| 2456 | 2459 | if (access) { |
| @@ -143,7 +143,7 @@ void iommufd_vdevice_destroy(struct iommufd_object *obj) · 142 unmodified lines | ||
| 143 | 143 | int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *ucmd) |
| 144 | 144 | { |
| 145 | 145 | struct iommu_vdevice_alloc *cmd = ucmd->cmd; |
| 146 | − struct iommufd_vdevice *vdev, *curr; | |
| 146 | + struct iommufd_vdevice *vdev; | |
| 147 | 147 | size_t vdev_size = sizeof(*vdev); |
| 148 | 148 | struct iommufd_viommu *viommu; |
| 149 | 149 | struct iommufd_device *idev; |
| @@ -189,7 +189,7 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *ucmd) · 39 unmodified lines | ||
| 189 | 189 | if (WARN_ON_ONCE(viommu->ops->vdevice_size < vdev_size || |
| 190 | 190 | !viommu->ops->vdevice_init)) { |
| 191 | 191 | rc = -EOPNOTSUPP; |
| 192 | − goto out_put_idev; | |
| 192 | + goto out_unlock_igroup; | |
| 193 | 193 | } |
| 194 | 194 | vdev_size = viommu->ops->vdevice_size; |
| 195 | 195 | } |
| @@ -218,18 +218,28 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *ucmd) · 22 unmodified lines | ||
| 218 | 218 | */ |
| 219 | 219 | idev->vdev = vdev; |
| 220 | 220 | |
| 221 | − curr = xa_cmpxchg(&viommu->vdevs, virt_id, NULL, vdev, GFP_KERNEL); | |
| 222 | − if (curr) { | |
| 223 | − rc = xa_err(curr) ?: -EEXIST; | |
| 221 | + /* | |
| 222 | + * Reserve the slot with a zero entry (reads back as NULL) until the | |
| 223 | + * vdevice_init() op accepts the vDEVICE. Only the xa_* helpers hide a | |
| 224 | + * reserved entry, so never use a raw xas_* iterator on this xarray. | |
| 225 | + */ | |
| 226 | + rc = xa_insert(&viommu->vdevs, virt_id, NULL, GFP_KERNEL); | |
| 227 | + if (rc) { | |
| 228 | + if (rc == -EBUSY) | |
| 229 | + rc = -EEXIST; | |
| 224 | 230 | goto out_abort; |
| 225 | 231 | } |
| 226 | 232 | |
| 227 | 233 | if (viommu->ops && viommu->ops->vdevice_init) { |
| 228 | 234 | rc = viommu->ops->vdevice_init(vdev); |
| 229 | − if (rc) | |
| 235 | + if (rc) { | |
| 236 | + xa_release(&viommu->vdevs, virt_id); | |
| 230 | 237 | goto out_abort; |
| 238 | + } | |
| 231 | 239 | } |
| 232 | 240 | |
| 241 | + xa_store(&viommu->vdevs, virt_id, vdev, GFP_KERNEL); | |
| 242 | + | |
| 233 | 243 | cmd->out_vdevice_id = vdev->obj.id; |
| 234 | 244 | rc = iommufd_ucmd_respond(ucmd, sizeof(*cmd)); |
| 235 | 245 | if (rc) |
| @@ -492,6 +492,7 @@ static void fastrpc_channel_ctx_free(struct kref *ref) · 491 unmodified lines | ||
| 492 | 492 | |
| 493 | 493 | cctx = container_of(ref, struct fastrpc_channel_ctx, refcount); |
| 494 | 494 | |
| 495 | + idr_destroy(&cctx->ctx_idr); | |
| 495 | 496 | kfree(cctx); |
| 496 | 497 | } |
| 497 | 498 | |
| @@ -1305,10 +1306,12 @@ bail: · 807 unmodified lines | ||
| 1305 | 1306 | } |
| 1306 | 1307 | |
| 1307 | 1308 | if (err == -ERESTARTSYS) { |
| 1309 | + spin_lock(&fl->lock); | |
| 1308 | 1310 | list_for_each_entry_safe(buf, b, &fl->mmaps, node) { |
| 1309 | 1311 | list_del(&buf->node); |
| 1310 | 1312 | list_add_tail(&buf->node, &fl->cctx->invoke_interrupted_mmaps); |
| 1311 | 1313 | } |
| 1314 | + spin_unlock(&fl->lock); | |
| 1312 | 1315 | } |
| 1313 | 1316 | |
| 1314 | 1317 | if (err) |
| @@ -1370,7 +1373,9 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl, · 55 unmodified lines | ||
| 1370 | 1373 | err = PTR_ERR(name); |
| 1371 | 1374 | goto err; |
| 1372 | 1375 | } |
| 1373 | − | |
| 1376 | + inbuf.client_id = fl->client_id; | |
| 1377 | + inbuf.namelen = init.namelen; | |
| 1378 | + inbuf.pageslen = 0; | |
| 1374 | 1379 | if (!fl->cctx->remote_heap) { |
| 1375 | 1380 | err = fastrpc_remote_heap_alloc(fl, fl->sctx->dev, init.memlen, |
| 1376 | 1381 | &fl->cctx->remote_heap); |
| @@ -1393,12 +1398,10 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl, · 16 unmodified lines | ||
| 1393 | 1398 | goto err_map; |
| 1394 | 1399 | } |
| 1395 | 1400 | scm_done = true; |
| 1401 | + inbuf.pageslen = 1; | |
| 1396 | 1402 | } |
| 1397 | 1403 | } |
| 1398 | 1404 | |
| 1399 | − inbuf.client_id = fl->client_id; | |
| 1400 | − inbuf.namelen = init.namelen; | |
| 1401 | − inbuf.pageslen = 0; | |
| 1402 | 1405 | fl->pd = USER_PD; |
| 1403 | 1406 | |
| 1404 | 1407 | args[0].ptr = (u64)(uintptr_t)&inbuf; |
| @@ -1673,7 +1676,7 @@ static int fastrpc_device_open(struct inode *inode, struct file *filp) · 268 unmodified lines | ||
| 1673 | 1676 | dev_err(&cctx->rpdev->dev, "No session available\n"); |
| 1674 | 1677 | mutex_destroy(&fl->mutex); |
| 1675 | 1678 | kfree(fl); |
| 1676 | − | |
| 1679 | + fastrpc_channel_ctx_put(cctx); | |
| 1677 | 1680 | return -EBUSY; |
| 1678 | 1681 | } |
| 1679 | 1682 | |
| @@ -1889,9 +1892,6 @@ static int fastrpc_req_munmap_impl(struct fastrpc_user *fl, struct fastrpc_buf * · 209 unmodified lines | ||
| 1889 | 1892 | &args[0]); |
| 1890 | 1893 | if (!err) { |
| 1891 | 1894 | dev_dbg(dev, "unmmap\tpt 0x%09lx OK\n", buf->raddr); |
| 1892 | − spin_lock(&fl->lock); | |
| 1893 | − list_del(&buf->node); | |
| 1894 | − spin_unlock(&fl->lock); | |
| 1895 | 1895 | fastrpc_buf_free(buf); |
| 1896 | 1896 | } else { |
| 1897 | 1897 | dev_err(dev, "unmmap\tpt 0x%09lx ERROR\n", buf->raddr); |
| @@ -1905,6 +1905,7 @@ static int fastrpc_req_munmap(struct fastrpc_user *fl, char __user *argp) · 7 unmodified lines | ||
| 1905 | 1905 | struct fastrpc_buf *buf = NULL, *iter, *b; |
| 1906 | 1906 | struct fastrpc_req_munmap req; |
| 1907 | 1907 | struct device *dev = fl->sctx->dev; |
| 1908 | + int err; | |
| 1908 | 1909 | |
| 1909 | 1910 | if (copy_from_user(&req, argp, sizeof(req))) |
| 1910 | 1911 | return -EFAULT; |
| @@ -1912,6 +1913,7 @@ static int fastrpc_req_munmap(struct fastrpc_user *fl, char __user *argp) · 1 unmodified line | ||
| 1912 | 1913 | spin_lock(&fl->lock); |
| 1913 | 1914 | list_for_each_entry_safe(iter, b, &fl->mmaps, node) { |
| 1914 | 1915 | if ((iter->raddr == req.vaddrout) && (iter->size == req.size)) { |
| 1916 | + list_del(&iter->node); | |
| 1915 | 1917 | buf = iter; |
| 1916 | 1918 | break; |
| 1917 | 1919 | } |
| @@ -1924,7 +1926,14 @@ static int fastrpc_req_munmap(struct fastrpc_user *fl, char __user *argp) · 6 unmodified lines | ||
| 1924 | 1926 | return -EINVAL; |
| 1925 | 1927 | } |
| 1926 | 1928 | |
| 1927 | − return fastrpc_req_munmap_impl(fl, buf); | |
| 1929 | + err = fastrpc_req_munmap_impl(fl, buf); | |
| 1930 | + if (err) { | |
| 1931 | + spin_lock(&fl->lock); | |
| 1932 | + list_add_tail(&buf->node, &fl->mmaps); | |
| 1933 | + spin_unlock(&fl->lock); | |
| 1934 | + } | |
| 1935 | + | |
| 1936 | + return err; | |
| 1928 | 1937 | } |
| 1929 | 1938 | |
| 1930 | 1939 | static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp) |
| @@ -425,18 +425,24 @@ static void mei_io_tx_list_free_cl(struct list_head *head, · 424 unmodified lines | ||
| 425 | 425 | } |
| 426 | 426 | |
| 427 | 427 | /** |
| 428 | − * mei_io_list_free_fp - free cb from a list that matches file pointer | |
| 428 | + * mei_io_rd_list_free_fp - free cb from a rd_completed list that matches file pointer | |
| 429 | 429 | * |
| 430 | − * @head: io list | |
| 430 | + * @cl: host client | |
| 431 | 431 | * @fp: file pointer (matching cb file object), may be NULL |
| 432 | 432 | */ |
| 433 | −static void mei_io_list_free_fp(struct list_head *head, const struct file *fp) | |
| 433 | +static void mei_io_rd_list_free_fp(struct mei_cl *cl, const struct file *fp) | |
| 434 | 434 | { |
| 435 | 435 | struct mei_cl_cb *cb, *next; |
| 436 | + LIST_HEAD(cmpl_list); | |
| 436 | 437 | |
| 437 | − list_for_each_entry_safe(cb, next, head, list) | |
| 438 | + spin_lock(&cl->rd_completed_lock); | |
| 439 | + list_for_each_entry_safe(cb, next, &cl->rd_completed, list) | |
| 438 | 440 | if (!fp || fp == cb->fp) |
| 439 | − mei_io_cb_free(cb); | |
| 441 | + list_move(&cb->list, &cmpl_list); | |
| 442 | + spin_unlock(&cl->rd_completed_lock); | |
| 443 | + | |
| 444 | + list_for_each_entry_safe(cb, next, &cmpl_list, list) | |
| 445 | + mei_io_cb_free(cb); | |
| 440 | 446 | } |
| 441 | 447 | |
| 442 | 448 | /** |
| @@ -565,9 +571,7 @@ int mei_cl_flush_queues(struct mei_cl *cl, const struct file *fp) · 122 unmodified lines | ||
| 565 | 571 | mei_io_list_flush_cl(&cl->dev->ctrl_rd_list, cl); |
| 566 | 572 | mei_cl_free_pending(cl); |
| 567 | 573 | } |
| 568 | − spin_lock(&cl->rd_completed_lock); | |
| 569 | − mei_io_list_free_fp(&cl->rd_completed, fp); | |
| 570 | − spin_unlock(&cl->rd_completed_lock); | |
| 574 | + mei_io_rd_list_free_fp(cl, fp); | |
| 571 | 575 | |
| 572 | 576 | return 0; |
| 573 | 577 | } |
| @@ -1401,7 +1405,7 @@ void mei_cl_add_rd_completed(struct mei_cl *cl, struct mei_cl_cb *cb) · 827 unmodified lines | ||
| 1401 | 1405 | } |
| 1402 | 1406 | |
| 1403 | 1407 | /** |
| 1404 | − * mei_cl_del_rd_completed - free read completed callback with lock | |
| 1408 | + * mei_cl_del_rd_completed - unlink read completed callback with lock and free it | |
| 1405 | 1409 | * |
| 1406 | 1410 | * @cl: host client |
| 1407 | 1411 | * @cb: callback block |
| @@ -1410,8 +1414,9 @@ void mei_cl_add_rd_completed(struct mei_cl *cl, struct mei_cl_cb *cb) · 2 unmodified lines | ||
| 1410 | 1414 | void mei_cl_del_rd_completed(struct mei_cl *cl, struct mei_cl_cb *cb) |
| 1411 | 1415 | { |
| 1412 | 1416 | spin_lock(&cl->rd_completed_lock); |
| 1413 | − mei_io_cb_free(cb); | |
| 1417 | + list_del_init(&cb->list); | |
| 1414 | 1418 | spin_unlock(&cl->rd_completed_lock); |
| 1419 | + mei_io_cb_free(cb); | |
| 1415 | 1420 | } |
| 1416 | 1421 | |
| 1417 | 1422 | /** |
| @@ -1534,8 +1534,8 @@ void bond_alb_monitor(struct work_struct *work) · 1533 unmodified lines | ||
| 1534 | 1534 | struct bonding *bond = container_of(work, struct bonding, |
| 1535 | 1535 | alb_work.work); |
| 1536 | 1536 | struct alb_bond_info *bond_info = &(BOND_ALB_INFO(bond)); |
| 1537 | + struct slave *slave, *curr; | |
| 1537 | 1538 | struct list_head *iter; |
| 1538 | − struct slave *slave; | |
| 1539 | 1539 | |
| 1540 | 1540 | if (!bond_has_slaves(bond)) { |
| 1541 | 1541 | atomic_set(&bond_info->tx_rebalance_counter, 0); |
| @@ -1597,9 +1597,11 @@ void bond_alb_monitor(struct work_struct *work) · 55 unmodified lines | ||
| 1597 | 1597 | * because a slave was disabled then |
| 1598 | 1598 | * it can now leave promiscuous mode. |
| 1599 | 1599 | */ |
| 1600 | − dev_set_promiscuity(rtnl_dereference(bond->curr_active_slave)->dev, | |
| 1601 | − -1); | |
| 1602 | − bond_info->primary_is_promisc = 0; | |
| 1600 | + curr = rtnl_dereference(bond->curr_active_slave); | |
| 1601 | + if (bond_info->primary_is_promisc && curr) { | |
| 1602 | + dev_set_promiscuity(curr->dev, -1); | |
| 1603 | + bond_info->primary_is_promisc = 0; | |
| 1604 | + } | |
| 1603 | 1605 | |
| 1604 | 1606 | rtnl_unlock(); |
| 1605 | 1607 | rcu_read_lock(); |
| @@ -597,20 +597,20 @@ static int c_can_chip_config(struct net_device *dev) · 596 unmodified lines | ||
| 597 | 597 | return err; |
| 598 | 598 | |
| 599 | 599 | /* enable automatic retransmission */ |
| 600 | − priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_ENABLE_AR); | |
| 600 | + priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_ENABLE_AR | CONTROL_INIT); | |
| 601 | 601 | |
| 602 | 602 | if ((priv->can.ctrlmode & CAN_CTRLMODE_LISTENONLY) && |
| 603 | 603 | (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK)) { |
| 604 | 604 | /* loopback + silent mode : useful for hot self-test */ |
| 605 | − priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST); | |
| 605 | + priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST | CONTROL_INIT); | |
| 606 | 606 | priv->write_reg(priv, C_CAN_TEST_REG, TEST_LBACK | TEST_SILENT); |
| 607 | 607 | } else if (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK) { |
| 608 | 608 | /* loopback mode : useful for self-test function */ |
| 609 | − priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST); | |
| 609 | + priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST | CONTROL_INIT); | |
| 610 | 610 | priv->write_reg(priv, C_CAN_TEST_REG, TEST_LBACK); |
| 611 | 611 | } else if (priv->can.ctrlmode & CAN_CTRLMODE_LISTENONLY) { |
| 612 | 612 | /* silent mode : bus-monitoring mode */ |
| 613 | − priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST); | |
| 613 | + priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST | CONTROL_INIT); | |
| 614 | 614 | priv->write_reg(priv, C_CAN_TEST_REG, TEST_SILENT); |
| 615 | 615 | } |
| 616 | 616 | |
| @@ -340,8 +340,8 @@ static void ctucan_set_mode(struct ctucan_priv *priv, const struct can_ctrlmode · 339 unmodified lines | ||
| 340 | 340 | (mode_reg & ~REG_MODE_FDE); |
| 341 | 341 | |
| 342 | 342 | mode_reg = (mode->flags & CAN_CTRLMODE_PRESUME_ACK) ? |
| 343 | − (mode_reg | REG_MODE_ACF) : | |
| 344 | − (mode_reg & ~REG_MODE_ACF); | |
| 343 | + (mode_reg | REG_MODE_STM) : | |
| 344 | + (mode_reg & ~REG_MODE_STM); | |
| 345 | 345 | |
| 346 | 346 | mode_reg = (mode->flags & CAN_CTRLMODE_FD_NON_ISO) ? |
| 347 | 347 | (mode_reg | REG_MODE_NISOFD) : |
| @@ -869,7 +869,7 @@ static void ctucan_err_interrupt(struct net_device *ndev, u32 isr) · 521 unmodified lines | ||
| 869 | 869 | break; |
| 870 | 870 | case CAN_STATE_ERROR_ACTIVE: |
| 871 | 871 | if (skb) { |
| 872 | − cf->can_id |= CAN_ERR_CNT; | |
| 872 | + cf->can_id |= CAN_ERR_CRTL | CAN_ERR_CNT; | |
| 873 | 873 | cf->data[1] = CAN_ERR_CRTL_ACTIVE; |
| 874 | 874 | cf->data[6] = bec.txerr; |
| 875 | 875 | cf->data[7] = bec.rxerr; |
| @@ -1136,8 +1136,12 @@ static irqreturn_t ctucan_interrupt(int irq, void *dev_id) · 260 unmodified lines | ||
| 1136 | 1136 | /* Error interrupts */ |
| 1137 | 1137 | if (FIELD_GET(REG_INT_STAT_EWLI, isr) || |
| 1138 | 1138 | FIELD_GET(REG_INT_STAT_FCSI, isr) || |
| 1139 | − FIELD_GET(REG_INT_STAT_ALI, isr)) { | |
| 1140 | − icr = isr & (REG_INT_STAT_EWLI | REG_INT_STAT_FCSI | REG_INT_STAT_ALI); | |
| 1139 | + FIELD_GET(REG_INT_STAT_ALI, isr) || | |
| 1140 | + FIELD_GET(REG_INT_STAT_BEI, isr)) { | |
| 1141 | + icr = isr & (REG_INT_STAT_EWLI | | |
| 1142 | + REG_INT_STAT_FCSI | | |
| 1143 | + REG_INT_STAT_ALI | | |
| 1144 | + REG_INT_STAT_BEI); | |
| 1141 | 1145 | |
| 1142 | 1146 | ctucan_netdev_dbg(ndev, "some ERR interrupt: clearing 0x%08x\n", icr); |
| 1143 | 1147 | ctucan_write32(priv, CTUCANFD_INT_STAT, icr); |
| @@ -194,7 +194,7 @@ err_free_board: · 193 unmodified lines | ||
| 194 | 194 | pci_set_drvdata(pdev, NULL); |
| 195 | 195 | kfree(bdata); |
| 196 | 196 | err_pci_iounmap_bar0: |
| 197 | − pci_iounmap(pdev, cra_addr); | |
| 197 | + pci_iounmap(pdev, bar0_base); | |
| 198 | 198 | err_pci_iounmap_bar1: |
| 199 | 199 | pci_iounmap(pdev, addr); |
| 200 | 200 | err_release_regions: |
| @@ -266,6 +266,7 @@ static const struct pci_device_id ctucan_pci_tbl[] = { · 65 unmodified lines | ||
| 266 | 266 | CTUCAN_WITH_CTUCAN_ID)}, |
| 267 | 267 | {}, |
| 268 | 268 | }; |
| 269 | +MODULE_DEVICE_TABLE(pci, ctucan_pci_tbl); | |
| 269 | 270 | |
| 270 | 271 | static struct pci_driver ctucan_pci_driver = { |
| 271 | 272 | .name = KBUILD_MODNAME, |
| @@ -2003,20 +2003,12 @@ static int rcar_canfd_global_init(struct rcar_canfd_global *gpriv) · 2002 unmodified lines | ||
| 2003 | 2003 | u32 ch, sts; |
| 2004 | 2004 | int err; |
| 2005 | 2005 | |
| 2006 | − err = reset_control_reset(gpriv->rstc1); | |
| 2007 | − if (err) | |
| 2008 | − return err; | |
| 2009 | − | |
| 2010 | − err = reset_control_reset(gpriv->rstc2); | |
| 2011 | − if (err) | |
| 2012 | − goto fail_reset1; | |
| 2013 | − | |
| 2014 | 2006 | /* Enable peripheral clock for register access */ |
| 2015 | 2007 | err = clk_prepare_enable(gpriv->clkp); |
| 2016 | 2008 | if (err) { |
| 2017 | 2009 | dev_err(dev, "failed to enable peripheral clock: %pe\n", |
| 2018 | 2010 | ERR_PTR(err)); |
| 2019 | − goto fail_reset2; | |
| 2011 | + return err; | |
| 2020 | 2012 | } |
| 2021 | 2013 | |
| 2022 | 2014 | /* Enable RAM clock */ |
| @@ -2027,10 +2019,18 @@ static int rcar_canfd_global_init(struct rcar_canfd_global *gpriv) · 4 unmodified lines | ||
| 2027 | 2019 | goto fail_clk; |
| 2028 | 2020 | } |
| 2029 | 2021 | |
| 2022 | + err = reset_control_reset(gpriv->rstc1); | |
| 2023 | + if (err) | |
| 2024 | + goto fail_ram_clk; | |
| 2025 | + | |
| 2026 | + err = reset_control_reset(gpriv->rstc2); | |
| 2027 | + if (err) | |
| 2028 | + goto fail_reset1; | |
| 2029 | + | |
| 2030 | 2030 | err = rcar_canfd_reset_controller(gpriv); |
| 2031 | 2031 | if (err) { |
| 2032 | 2032 | dev_err(dev, "reset controller failed: %pe\n", ERR_PTR(err)); |
| 2033 | − goto fail_ram_clk; | |
| 2033 | + goto fail_reset2; | |
| 2034 | 2034 | } |
| 2035 | 2035 | |
| 2036 | 2036 | /* Controller in Global reset & Channel reset mode */ |
| @@ -2068,14 +2068,14 @@ static int rcar_canfd_global_init(struct rcar_canfd_global *gpriv) · 31 unmodified lines | ||
| 2068 | 2068 | |
| 2069 | 2069 | fail_mode: |
| 2070 | 2070 | rcar_canfd_disable_global_interrupts(gpriv); |
| 2071 | −fail_ram_clk: | |
| 2072 | − clk_disable_unprepare(gpriv->clk_ram); | |
| 2073 | −fail_clk: | |
| 2074 | − clk_disable_unprepare(gpriv->clkp); | |
| 2075 | 2071 | fail_reset2: |
| 2076 | 2072 | reset_control_assert(gpriv->rstc2); |
| 2077 | 2073 | fail_reset1: |
| 2078 | 2074 | reset_control_assert(gpriv->rstc1); |
| 2075 | +fail_ram_clk: | |
| 2076 | + clk_disable_unprepare(gpriv->clk_ram); | |
| 2077 | +fail_clk: | |
| 2078 | + clk_disable_unprepare(gpriv->clkp); | |
| 2079 | 2079 | return err; |
| 2080 | 2080 | } |
| 2081 | 2081 | |
| @@ -2090,10 +2090,10 @@ static void rcar_canfd_global_deinit(struct rcar_canfd_global *gpriv, bool full) · 8 unmodified lines | ||
| 2090 | 2090 | rcar_canfd_set_bit(gpriv->base, RCANFD_GCTR, RCANFD_GCTR_GSLPR); |
| 2091 | 2091 | } |
| 2092 | 2092 | |
| 2093 | − clk_disable_unprepare(gpriv->clk_ram); | |
| 2094 | − clk_disable_unprepare(gpriv->clkp); | |
| 2095 | 2093 | reset_control_assert(gpriv->rstc2); |
| 2096 | 2094 | reset_control_assert(gpriv->rstc1); |
| 2095 | + clk_disable_unprepare(gpriv->clk_ram); | |
| 2096 | + clk_disable_unprepare(gpriv->clkp); | |
| 2097 | 2097 | } |
| 2098 | 2098 | |
| 2099 | 2099 | static int rcar_canfd_probe(struct platform_device *pdev) |
| @@ -91,12 +91,12 @@ int softing_bootloader_command(struct softing *card, int16_t cmd, · 90 unmodified lines | ||
| 91 | 91 | return ret; |
| 92 | 92 | } |
| 93 | 93 | |
| 94 | −static int fw_parse(const uint8_t **pmem, uint16_t *ptype, uint32_t *paddr, | |
| 95 | − uint16_t *plen, const uint8_t **pdat) | |
| 94 | +static int fw_parse(const u8 **pmem, const u8 *limit, u16 *ptype, | |
| 95 | + u32 *paddr, u16 *plen, const u8 **pdat) | |
| 96 | 96 | { |
| 97 | 97 | uint16_t checksum[2]; |
| 98 | − const uint8_t *mem; | |
| 99 | − const uint8_t *end; | |
| 98 | + const u8 *mem; | |
| 99 | + const u8 *record_end; | |
| 100 | 100 | |
| 101 | 101 | /* |
| 102 | 102 | * firmware records are a binary, unaligned stream composed of: |
| @@ -114,14 +114,21 @@ static int fw_parse(const uint8_t **pmem, uint16_t *ptype, uint32_t *paddr, · 11 unmodified lines | ||
| 114 | 114 | * endianness & alignment. |
| 115 | 115 | */ |
| 116 | 116 | mem = *pmem; |
| 117 | + /* A record needs an 8-byte prefix and a 2-byte checksum. */ | |
| 118 | + if (mem > limit || limit - mem < 10) | |
| 119 | + return -EINVAL; | |
| 120 | + | |
| 117 | 121 | *ptype = le16_to_cpup((void *)&mem[0]); |
| 118 | 122 | *paddr = le32_to_cpup((void *)&mem[2]); |
| 119 | 123 | *plen = le16_to_cpup((void *)&mem[6]); |
| 124 | + if (*plen > limit - mem - 10) | |
| 125 | + return -EINVAL; | |
| 126 | + | |
| 120 | 127 | *pdat = &mem[8]; |
| 121 | 128 | /* verify checksum */ |
| 122 | − end = &mem[8 + *plen]; | |
| 123 | − checksum[0] = le16_to_cpup((void *)end); | |
| 124 | − for (checksum[1] = 0; mem < end; ++mem) | |
| 129 | + record_end = &mem[8 + *plen]; | |
| 130 | + checksum[0] = le16_to_cpup((void *)record_end); | |
| 131 | + for (checksum[1] = 0; mem < record_end; ++mem) | |
| 125 | 132 | checksum[1] += *mem; |
| 126 | 133 | if (checksum[0] != checksum[1]) |
| 127 | 134 | return -EINVAL; |
| @@ -139,6 +146,7 @@ int softing_load_fw(const char *file, struct softing *card, · 11 unmodified lines | ||
| 139 | 146 | uint16_t type, len; |
| 140 | 147 | uint32_t addr; |
| 141 | 148 | uint8_t *buf = NULL, *new_buf; |
| 149 | + s64 dpram_offset; | |
| 142 | 150 | int buflen = 0; |
| 143 | 151 | int8_t type_end = 0; |
| 144 | 152 | |
| @@ -153,7 +161,7 @@ int softing_load_fw(const char *file, struct softing *card, · 8 unmodified lines | ||
| 153 | 161 | mem = fw->data; |
| 154 | 162 | end = &mem[fw->size]; |
| 155 | 163 | /* look for header record */ |
| 156 | − ret = fw_parse(&mem, &type, &addr, &len, &dat); | |
| 164 | + ret = fw_parse(&mem, end, &type, &addr, &len, &dat); | |
| 157 | 165 | if (ret < 0) |
| 158 | 166 | goto failed; |
| 159 | 167 | if (type != 0xffff) |
| @@ -164,7 +172,7 @@ int softing_load_fw(const char *file, struct softing *card, · 4 unmodified lines | ||
| 164 | 172 | } |
| 165 | 173 | /* ok, we had a header */ |
| 166 | 174 | while (mem < end) { |
| 167 | − ret = fw_parse(&mem, &type, &addr, &len, &dat); | |
| 175 | + ret = fw_parse(&mem, end, &type, &addr, &len, &dat); | |
| 168 | 176 | if (ret < 0) |
| 169 | 177 | goto failed; |
| 170 | 178 | if (type == 3) { |
| @@ -179,9 +187,13 @@ int softing_load_fw(const char *file, struct softing *card, · 8 unmodified lines | ||
| 179 | 187 | goto failed; |
| 180 | 188 | } |
| 181 | 189 | |
| 182 | − if ((addr + len + offset) > size) | |
| 190 | + dpram_offset = (s64)addr + offset; | |
| 191 | + if (dpram_offset < 0 || dpram_offset > size || | |
| 192 | + len > size - dpram_offset) { | |
| 193 | + ret = -EINVAL; | |
| 183 | 194 | goto failed; |
| 184 | − memcpy_toio(&dpram[addr + offset], dat, len); | |
| 195 | + } | |
| 196 | + memcpy_toio(&dpram[dpram_offset], dat, len); | |
| 185 | 197 | /* be sure to flush caches from IO space */ |
| 186 | 198 | mb(); |
| 187 | 199 | if (len > buflen) { |
| @@ -195,7 +207,7 @@ int softing_load_fw(const char *file, struct softing *card, · 7 unmodified lines | ||
| 195 | 207 | buf = new_buf; |
| 196 | 208 | } |
| 197 | 209 | /* verify record data */ |
| 198 | − memcpy_fromio(buf, &dpram[addr + offset], len); | |
| 210 | + memcpy_fromio(buf, &dpram[dpram_offset], len); | |
| 199 | 211 | if (memcmp(buf, dat, len)) { |
| 200 | 212 | /* is not ok */ |
| 201 | 213 | dev_alert(&card->pdev->dev, "DPRAM readback failed\n"); |
| @@ -237,7 +249,7 @@ int softing_load_app_fw(const char *file, struct softing *card) · 35 unmodified lines | ||
| 237 | 249 | mem = fw->data; |
| 238 | 250 | end = &mem[fw->size]; |
| 239 | 251 | /* look for header record */ |
| 240 | − ret = fw_parse(&mem, &type, &addr, &len, &dat); | |
| 252 | + ret = fw_parse(&mem, end, &type, &addr, &len, &dat); | |
| 241 | 253 | if (ret) |
| 242 | 254 | goto failed; |
| 243 | 255 | ret = -EINVAL; |
| @@ -253,7 +265,7 @@ int softing_load_app_fw(const char *file, struct softing *card) · 9 unmodified lines | ||
| 253 | 265 | } |
| 254 | 266 | /* ok, we had a header */ |
| 255 | 267 | while (mem < end) { |
| 256 | − ret = fw_parse(&mem, &type, &addr, &len, &dat); | |
| 268 | + ret = fw_parse(&mem, end, &type, &addr, &len, &dat); | |
| 257 | 269 | if (ret) |
| 258 | 270 | goto failed; |
| 259 | 271 | |
| @@ -279,6 +291,12 @@ int softing_load_app_fw(const char *file, struct softing *card) · 19 unmodified lines | ||
| 279 | 291 | /* work in 16bit (target) */ |
| 280 | 292 | sum &= 0xffff; |
| 281 | 293 | |
| 294 | + if (card->pdat->app.offs > card->dpram_size || | |
| 295 | + len > card->dpram_size - card->pdat->app.offs) { | |
| 296 | + ret = -EINVAL; | |
| 297 | + goto failed; | |
| 298 | + } | |
| 299 | + | |
| 282 | 300 | memcpy_toio(&card->dpram[card->pdat->app.offs], dat, len); |
| 283 | 301 | iowrite32(card->pdat->app.offs + card->pdat->app.addr, |
| 284 | 302 | &card->dpram[DPRAM_COMMAND + 2]); |
| @@ -409,6 +409,40 @@ static void ems_usb_rx_err(struct ems_usb *dev, struct ems_cpc_msg *msg) · 408 unmodified lines | ||
| 409 | 409 | netif_rx(skb); |
| 410 | 410 | } |
| 411 | 411 | |
| 412 | +static bool ems_usb_rx_msg_len_valid(struct ems_cpc_msg *msg) | |
| 413 | +{ | |
| 414 | + size_t len = msg->length; | |
| 415 | + size_t can_len; | |
| 416 | + | |
| 417 | + switch (msg->type) { | |
| 418 | + case CPC_MSG_TYPE_CAN_STATE: | |
| 419 | + return len >= sizeof(msg->msg.can_state); | |
| 420 | + | |
| 421 | + case CPC_MSG_TYPE_CAN_FRAME: | |
| 422 | + case CPC_MSG_TYPE_EXT_CAN_FRAME: | |
| 423 | + case CPC_MSG_TYPE_RTR_FRAME: | |
| 424 | + case CPC_MSG_TYPE_EXT_RTR_FRAME: | |
| 425 | + if (len < CPC_CAN_MSG_MIN_SIZE) | |
| 426 | + return false; | |
| 427 | + | |
| 428 | + if (msg->type == CPC_MSG_TYPE_RTR_FRAME || | |
| 429 | + msg->type == CPC_MSG_TYPE_EXT_RTR_FRAME) | |
| 430 | + return true; | |
| 431 | + | |
| 432 | + can_len = can_cc_dlc2len(msg->msg.can_msg.length & 0xf); | |
| 433 | + return len >= CPC_CAN_MSG_MIN_SIZE + can_len; | |
| 434 | + | |
| 435 | + case CPC_MSG_TYPE_CAN_FRAME_ERROR: | |
| 436 | + return len >= sizeof(msg->msg.error); | |
| 437 | + | |
| 438 | + case CPC_MSG_TYPE_OVERRUN: | |
| 439 | + return len >= sizeof(msg->msg.overrun); | |
| 440 | + | |
| 441 | + default: | |
| 442 | + return true; | |
| 443 | + } | |
| 444 | +} | |
| 445 | + | |
| 412 | 446 | /* |
| 413 | 447 | * callback for bulk IN urb |
| 414 | 448 | */ |
| @@ -451,6 +485,15 @@ static void ems_usb_read_bulk_callback(struct urb *urb) · 36 unmodified lines | ||
| 451 | 485 | } |
| 452 | 486 | |
| 453 | 487 | msg = (struct ems_cpc_msg *)&ibuf[start]; |
| 488 | + if (msg->length > | |
| 489 | + urb->actual_length - start - CPC_MSG_HEADER_LEN) { | |
| 490 | + netdev_err(netdev, "format error\n"); | |
| 491 | + break; | |
| 492 | + } | |
| 493 | + if (!ems_usb_rx_msg_len_valid(msg)) { | |
| 494 | + netdev_err(netdev, "format error\n"); | |
| 495 | + break; | |
| 496 | + } | |
| 454 | 497 | |
| 455 | 498 | switch (msg->type) { |
| 456 | 499 | case CPC_MSG_TYPE_CAN_STATE: |
| @@ -1476,7 +1476,6 @@ static void es58x_read_bulk_callback(struct urb *urb) · 1475 unmodified lines | ||
| 1476 | 1476 | dev_err_ratelimited(dev, |
| 1477 | 1477 | "Failed resubmitting read bulk urb: %pe\n", |
| 1478 | 1478 | ERR_PTR(ret)); |
| 1479 | − return; | |
| 1480 | 1479 | |
| 1481 | 1480 | free_urb: |
| 1482 | 1481 | usb_free_coherent(urb->dev, urb->transfer_buffer_length, |
| @@ -674,7 +674,7 @@ static void gs_usb_receive_bulk_callback(struct urb *urb) · 673 unmodified lines | ||
| 674 | 674 | if (hf->flags & GS_CAN_FLAG_FD) { |
| 675 | 675 | skb = alloc_canfd_skb(netdev, &cfd); |
| 676 | 676 | if (!skb) |
| 677 | − return; | |
| 677 | + goto resubmit_urb; | |
| 678 | 678 | |
| 679 | 679 | cfd->can_id = le32_to_cpu(hf->can_id); |
| 680 | 680 | cfd->len = data_length; |
| @@ -687,7 +687,7 @@ static void gs_usb_receive_bulk_callback(struct urb *urb) · 6 unmodified lines | ||
| 687 | 687 | } else { |
| 688 | 688 | skb = alloc_can_skb(netdev, &cf); |
| 689 | 689 | if (!skb) |
| 690 | − return; | |
| 690 | + goto resubmit_urb; | |
| 691 | 691 | |
| 692 | 692 | cf->can_id = le32_to_cpu(hf->can_id); |
| 693 | 693 | can_frame_set_cc_len(cf, hf->can_dlc, dev->can.ctrlmode); |
| @@ -1626,6 +1626,7 @@ static int kvaser_usb_hydra_get_busparams(struct kvaser_usb_net_priv *priv, · 1625 unmodified lines | ||
| 1626 | 1626 | reinit_completion(&priv->get_busparams_comp); |
| 1627 | 1627 | |
| 1628 | 1628 | err = kvaser_usb_send_cmd(dev, cmd, cmd_len); |
| 1629 | + kfree(cmd); | |
| 1629 | 1630 | if (err) |
| 1630 | 1631 | return err; |
| 1631 | 1632 | |
| @@ -691,13 +691,22 @@ static int kvaser_usb_leaf_wait_cmd(const struct kvaser_usb *dev, u8 id, · 690 unmodified lines | ||
| 691 | 691 | continue; |
| 692 | 692 | } |
| 693 | 693 | |
| 694 | − if (pos + tmp->len > actual_len) { | |
| 694 | + if (tmp->len < CMD_HEADER_LEN || | |
| 695 | + tmp->len > actual_len - pos) { | |
| 695 | 696 | dev_err_ratelimited(&dev->intf->dev, |
| 696 | 697 | "Format error\n"); |
| 697 | 698 | break; |
| 698 | 699 | } |
| 699 | 700 | |
| 700 | 701 | if (tmp->id == id) { |
| 702 | + if (tmp->len > sizeof(*cmd)) { | |
| 703 | + dev_err_ratelimited(&dev->intf->dev, | |
| 704 | + "Received command %u too large (%u)\n", | |
| 705 | + tmp->id, tmp->len); | |
| 706 | + err = -EIO; | |
| 707 | + goto end; | |
| 708 | + } | |
| 709 | + | |
| 701 | 710 | memcpy(cmd, tmp, tmp->len); |
| 702 | 711 | goto end; |
| 703 | 712 | } |
| @@ -1737,7 +1746,7 @@ static void kvaser_usb_leaf_read_bulk_callback(struct kvaser_usb *dev, · 1033 unmodified lines | ||
| 1737 | 1746 | continue; |
| 1738 | 1747 | } |
| 1739 | 1748 | |
| 1740 | − if (pos + cmd->len > len) { | |
| 1749 | + if (cmd->len < CMD_HEADER_LEN || cmd->len > len - pos) { | |
| 1741 | 1750 | dev_err_ratelimited(&dev->intf->dev, "Format error\n"); |
| 1742 | 1751 | break; |
| 1743 | 1752 | } |
| @@ -470,7 +470,6 @@ static int peak_usb_start(struct peak_usb_device *dev) · 469 unmodified lines | ||
| 470 | 470 | netif_device_detach(dev->netdev); |
| 471 | 471 | |
| 472 | 472 | usb_unanchor_urb(urb); |
| 473 | − kfree(buf); | |
| 474 | 473 | usb_free_urb(urb); |
| 475 | 474 | break; |
| 476 | 475 | } |
| @@ -566,6 +566,13 @@ static int pcan_usb_fd_decode_canmsg(struct pcan_usb_fd_if *usb_if, · 565 unmodified lines | ||
| 566 | 566 | dev->can.ctrlmode); |
| 567 | 567 | } |
| 568 | 568 | |
| 569 | + if (!(rx_msg_flags & PUCAN_MSG_RTR) && | |
| 570 | + le16_to_cpu(rx_msg->size) - offsetof(struct pucan_rx_msg, d) < | |
| 571 | + cfd->len) { | |
| 572 | + kfree_skb(skb); | |
| 573 | + return -EBADMSG; | |
| 574 | + } | |
| 575 | + | |
| 569 | 576 | cfd->can_id = le32_to_cpu(rm->can_id); |
| 570 | 577 | |
| 571 | 578 | if (rx_msg_flags & PUCAN_MSG_EXT_ID) |
| @@ -714,6 +721,24 @@ static void pcan_usb_fd_decode_ts(struct pcan_usb_fd_if *usb_if, · 142 unmodified lines | ||
| 714 | 721 | peak_usb_set_ts_now(&usb_if->time_ref, le32_to_cpu(ts->ts_low)); |
| 715 | 722 | } |
| 716 | 723 | |
| 724 | +static size_t pcan_usb_fd_rx_msg_min_size(u16 rx_msg_type) | |
| 725 | +{ | |
| 726 | + switch (rx_msg_type) { | |
| 727 | + case PUCAN_MSG_CAN_RX: | |
| 728 | + return offsetof(struct pucan_rx_msg, d); | |
| 729 | + case PCAN_UFD_MSG_CALIBRATION: | |
| 730 | + return sizeof(struct pcan_ufd_ts_msg); | |
| 731 | + case PUCAN_MSG_ERROR: | |
| 732 | + return sizeof(struct pucan_error_msg); | |
| 733 | + case PUCAN_MSG_STATUS: | |
| 734 | + return sizeof(struct pucan_status_msg); | |
| 735 | + case PCAN_UFD_MSG_OVERRUN: | |
| 736 | + return sizeof(struct pcan_ufd_ovr_msg); | |
| 737 | + default: | |
| 738 | + return sizeof(struct pucan_msg); | |
| 739 | + } | |
| 740 | +} | |
| 741 | + | |
| 717 | 742 | /* callback for bulk IN urb */ |
| 718 | 743 | static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) |
| 719 | 744 | { |
| @@ -728,6 +753,12 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) · 8 unmodified lines | ||
| 728 | 753 | msg_end = urb->transfer_buffer + urb->actual_length; |
| 729 | 754 | for (; msg_ptr < msg_end;) { |
| 730 | 755 | u16 rx_msg_type, rx_msg_size; |
| 756 | + size_t rx_msg_min_size; | |
| 757 | + | |
| 758 | + if (msg_end - msg_ptr < sizeof(*rx_msg)) { | |
| 759 | + err = -EBADMSG; | |
| 760 | + break; | |
| 761 | + } | |
| 731 | 762 | |
| 732 | 763 | rx_msg = (struct pucan_msg *)msg_ptr; |
| 733 | 764 | if (!rx_msg->size) { |
| @@ -739,13 +770,20 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) · 5 unmodified lines | ||
| 739 | 770 | rx_msg_type = le16_to_cpu(rx_msg->type); |
| 740 | 771 | |
| 741 | 772 | /* check if the record goes out of current packet */ |
| 742 | − if (msg_ptr + rx_msg_size > msg_end) { | |
| 773 | + if (rx_msg_size > msg_end - msg_ptr) { | |
| 743 | 774 | netdev_err(netdev, |
| 744 | 775 | "got frag rec: should inc usb rx buf sze\n"); |
| 745 | 776 | err = -EBADMSG; |
| 746 | 777 | break; |
| 747 | 778 | } |
| 748 | 779 | |
| 780 | + rx_msg_min_size = pcan_usb_fd_rx_msg_min_size(rx_msg_type); | |
| 781 | + if (rx_msg_size < rx_msg_min_size) { | |
| 782 | + netdev_err(netdev, "got short rec\n"); | |
| 783 | + err = -EBADMSG; | |
| 784 | + break; | |
| 785 | + } | |
| 786 | + | |
| 749 | 787 | switch (rx_msg_type) { |
| 750 | 788 | case PUCAN_MSG_CAN_RX: |
| 751 | 789 | err = pcan_usb_fd_decode_canmsg(usb_if, rx_msg); |
| @@ -534,12 +534,18 @@ static int pcan_usb_pro_handle_canmsg(struct pcan_usb_pro_interface *usb_if, · 533 unmodified lines | ||
| 534 | 534 | struct pcan_usb_pro_rxmsg *rx) |
| 535 | 535 | { |
| 536 | 536 | const unsigned int ctrl_idx = (rx->len >> 4) & 0x0f; |
| 537 | − struct peak_usb_device *dev = usb_if->dev[ctrl_idx]; | |
| 538 | − struct net_device *netdev = dev->netdev; | |
| 537 | + struct peak_usb_device *dev; | |
| 538 | + struct net_device *netdev; | |
| 539 | 539 | struct can_frame *can_frame; |
| 540 | 540 | struct sk_buff *skb; |
| 541 | 541 | struct skb_shared_hwtstamps *hwts; |
| 542 | 542 | |
| 543 | + if (ctrl_idx >= ARRAY_SIZE(usb_if->dev)) | |
| 544 | + return -EINVAL; | |
| 545 | + | |
| 546 | + dev = usb_if->dev[ctrl_idx]; | |
| 547 | + netdev = dev->netdev; | |
| 548 | + | |
| 543 | 549 | skb = alloc_can_skb(netdev, &can_frame); |
| 544 | 550 | if (!skb) |
| 545 | 551 | return -ENOMEM; |
| @@ -573,14 +579,20 @@ static int pcan_usb_pro_handle_error(struct pcan_usb_pro_interface *usb_if, · 27 unmodified lines | ||
| 573 | 579 | { |
| 574 | 580 | const u16 raw_status = le16_to_cpu(er->status); |
| 575 | 581 | const unsigned int ctrl_idx = (er->channel >> 4) & 0x0f; |
| 576 | − struct peak_usb_device *dev = usb_if->dev[ctrl_idx]; | |
| 577 | − struct net_device *netdev = dev->netdev; | |
| 582 | + struct peak_usb_device *dev; | |
| 583 | + struct net_device *netdev; | |
| 578 | 584 | struct can_frame *can_frame; |
| 579 | 585 | enum can_state new_state = CAN_STATE_ERROR_ACTIVE; |
| 580 | 586 | u8 err_mask = 0; |
| 581 | 587 | struct sk_buff *skb; |
| 582 | 588 | struct skb_shared_hwtstamps *hwts; |
| 583 | 589 | |
| 590 | + if (ctrl_idx >= ARRAY_SIZE(usb_if->dev)) | |
| 591 | + return -EINVAL; | |
| 592 | + | |
| 593 | + dev = usb_if->dev[ctrl_idx]; | |
| 594 | + netdev = dev->netdev; | |
| 595 | + | |
| 584 | 596 | /* nothing should be sent while in BUS_OFF state */ |
| 585 | 597 | if (dev->can.state == CAN_STATE_BUS_OFF) |
| 586 | 598 | return 0; |
| @@ -55,8 +55,15 @@ mt7530_regmap_read(void *context, unsigned int reg, unsigned int *val) · 54 unmodified lines | ||
| 55 | 55 | if (ret < 0) |
| 56 | 56 | return ret; |
| 57 | 57 | |
| 58 | − lo = bus->read(bus, priv->mdiodev->addr, r); | |
| 59 | − hi = bus->read(bus, priv->mdiodev->addr, 0x10); | |
| 58 | + ret = bus->read(bus, priv->mdiodev->addr, r); | |
| 59 | + if (ret < 0) | |
| 60 | + return ret; | |
| 61 | + lo = ret; | |
| 62 | + | |
| 63 | + ret = bus->read(bus, priv->mdiodev->addr, 0x10); | |
| 64 | + if (ret < 0) | |
| 65 | + return ret; | |
| 66 | + hi = ret; | |
| 60 | 67 | |
| 61 | 68 | *val = (hi << 16) | (lo & 0xffff); |
| 62 | 69 | |
| @@ -191,12 +191,6 @@ mt7530_write(struct mt7530_priv *priv, u32 reg, u32 val) · 190 unmodified lines | ||
| 191 | 191 | mt7530_mutex_unlock(priv); |
| 192 | 192 | } |
| 193 | 193 | |
| 194 | −static u32 | |
| 195 | −_mt7530_unlocked_read(struct mt7530_dummy_poll *p) | |
| 196 | −{ | |
| 197 | − return mt7530_mii_read(p->priv, p->reg); | |
| 198 | −} | |
| 199 | − | |
| 200 | 194 | static u32 |
| 201 | 195 | _mt7530_read(struct mt7530_dummy_poll *p) |
| 202 | 196 | { |
| @@ -248,15 +242,20 @@ mt7530_fdb_cmd(struct mt7530_priv *priv, enum mt7530_fdb_cmd cmd, u32 *rsp) · 45 unmodified lines | ||
| 248 | 242 | { |
| 249 | 243 | u32 val; |
| 250 | 244 | int ret; |
| 251 | − struct mt7530_dummy_poll p; | |
| 252 | 245 | |
| 253 | 246 | /* Set the command operating upon the MAC address entries */ |
| 254 | 247 | val = ATC_BUSY | ATC_MAT(0) | cmd; |
| 255 | 248 | mt7530_write(priv, MT7530_ATC, val); |
| 256 | 249 | |
| 257 | − INIT_MT7530_DUMMY_POLL(&p, priv, MT7530_ATC); | |
| 258 | − ret = readx_poll_timeout(_mt7530_read, &p, val, | |
| 259 | − !(val & ATC_BUSY), 20, 20000); | |
| 250 | + mt7530_mutex_lock(priv); | |
| 251 | + | |
| 252 | + ret = regmap_read_poll_timeout(priv->regmap, MT7530_ATC, val, | |
| 253 | + !(val & ATC_BUSY), 20, 20000); | |
| 254 | + if (!ret) | |
| 255 | + ret = regmap_read(priv->regmap, MT7530_ATC, &val); | |
| 256 | + | |
| 257 | + mt7530_mutex_unlock(priv); | |
| 258 | + | |
| 260 | 259 | if (ret < 0) { |
| 261 | 260 | dev_err(priv->dev, "reset timeout\n"); |
| 262 | 261 | return ret; |
| @@ -265,7 +264,6 @@ mt7530_fdb_cmd(struct mt7530_priv *priv, enum mt7530_fdb_cmd cmd, u32 *rsp) · 2 unmodified lines | ||
| 265 | 264 | /* Additional sanity for read command if the specified |
| 266 | 265 | * entry is invalid |
| 267 | 266 | */ |
| 268 | − val = mt7530_read(priv, MT7530_ATC); | |
| 269 | 267 | if ((cmd == MT7530_FDB_READ) && (val & ATC_INVALID)) |
| 270 | 268 | return -EINVAL; |
| 271 | 269 | |
| @@ -549,16 +547,13 @@ static int · 277 unmodified lines | ||
| 549 | 547 | mt7531_ind_c45_phy_read(struct mt7530_priv *priv, int port, int devad, |
| 550 | 548 | int regnum) |
| 551 | 549 | { |
| 552 | − struct mt7530_dummy_poll p; | |
| 553 | 550 | u32 reg, val; |
| 554 | 551 | int ret; |
| 555 | 552 | |
| 556 | − INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); | |
| 557 | − | |
| 558 | 553 | mt7530_mutex_lock(priv); |
| 559 | 554 | |
| 560 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 561 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 555 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 556 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 562 | 557 | if (ret < 0) { |
| 563 | 558 | dev_err(priv->dev, "poll timeout\n"); |
| 564 | 559 | goto out; |
| @@ -568,8 +563,8 @@ mt7531_ind_c45_phy_read(struct mt7530_priv *priv, int port, int devad, · 3 unmodified lines | ||
| 568 | 563 | MT7531_MDIO_DEV_ADDR(devad) | regnum; |
| 569 | 564 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); |
| 570 | 565 | |
| 571 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 572 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 566 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 567 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 573 | 568 | if (ret < 0) { |
| 574 | 569 | dev_err(priv->dev, "poll timeout\n"); |
| 575 | 570 | goto out; |
| @@ -579,8 +574,8 @@ mt7531_ind_c45_phy_read(struct mt7530_priv *priv, int port, int devad, · 3 unmodified lines | ||
| 579 | 574 | MT7531_MDIO_DEV_ADDR(devad); |
| 580 | 575 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); |
| 581 | 576 | |
| 582 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 583 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 577 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 578 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 584 | 579 | if (ret < 0) { |
| 585 | 580 | dev_err(priv->dev, "poll timeout\n"); |
| 586 | 581 | goto out; |
| @@ -597,16 +592,13 @@ static int · 10 unmodified lines | ||
| 597 | 592 | mt7531_ind_c45_phy_write(struct mt7530_priv *priv, int port, int devad, |
| 598 | 593 | int regnum, u16 data) |
| 599 | 594 | { |
| 600 | − struct mt7530_dummy_poll p; | |
| 601 | 595 | u32 val, reg; |
| 602 | 596 | int ret; |
| 603 | 597 | |
| 604 | − INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); | |
| 605 | − | |
| 606 | 598 | mt7530_mutex_lock(priv); |
| 607 | 599 | |
| 608 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 609 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 600 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 601 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 610 | 602 | if (ret < 0) { |
| 611 | 603 | dev_err(priv->dev, "poll timeout\n"); |
| 612 | 604 | goto out; |
| @@ -616,8 +608,8 @@ mt7531_ind_c45_phy_write(struct mt7530_priv *priv, int port, int devad, · 3 unmodified lines | ||
| 616 | 608 | MT7531_MDIO_DEV_ADDR(devad) | regnum; |
| 617 | 609 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); |
| 618 | 610 | |
| 619 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 620 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 611 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 612 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 621 | 613 | if (ret < 0) { |
| 622 | 614 | dev_err(priv->dev, "poll timeout\n"); |
| 623 | 615 | goto out; |
| @@ -627,8 +619,8 @@ mt7531_ind_c45_phy_write(struct mt7530_priv *priv, int port, int devad, · 3 unmodified lines | ||
| 627 | 619 | MT7531_MDIO_DEV_ADDR(devad) | data; |
| 628 | 620 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); |
| 629 | 621 | |
| 630 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 631 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 622 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 623 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 632 | 624 | if (ret < 0) { |
| 633 | 625 | dev_err(priv->dev, "poll timeout\n"); |
| 634 | 626 | goto out; |
| @@ -643,16 +635,13 @@ out: · 8 unmodified lines | ||
| 643 | 635 | static int |
| 644 | 636 | mt7531_ind_c22_phy_read(struct mt7530_priv *priv, int port, int regnum) |
| 645 | 637 | { |
| 646 | − struct mt7530_dummy_poll p; | |
| 647 | 638 | int ret; |
| 648 | 639 | u32 val; |
| 649 | 640 | |
| 650 | − INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); | |
| 651 | − | |
| 652 | 641 | mt7530_mutex_lock(priv); |
| 653 | 642 | |
| 654 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 655 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 643 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 644 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 656 | 645 | if (ret < 0) { |
| 657 | 646 | dev_err(priv->dev, "poll timeout\n"); |
| 658 | 647 | goto out; |
| @@ -663,8 +652,8 @@ mt7531_ind_c22_phy_read(struct mt7530_priv *priv, int port, int regnum) · 4 unmodified lines | ||
| 663 | 652 | |
| 664 | 653 | mt7530_mii_write(priv, MT7531_PHY_IAC, val | MT7531_PHY_ACS_ST); |
| 665 | 654 | |
| 666 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 667 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 655 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 656 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 668 | 657 | if (ret < 0) { |
| 669 | 658 | dev_err(priv->dev, "poll timeout\n"); |
| 670 | 659 | goto out; |
| @@ -681,16 +670,13 @@ static int · 10 unmodified lines | ||
| 681 | 670 | mt7531_ind_c22_phy_write(struct mt7530_priv *priv, int port, int regnum, |
| 682 | 671 | u16 data) |
| 683 | 672 | { |
| 684 | − struct mt7530_dummy_poll p; | |
| 685 | 673 | int ret; |
| 686 | 674 | u32 reg; |
| 687 | 675 | |
| 688 | − INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); | |
| 689 | − | |
| 690 | 676 | mt7530_mutex_lock(priv); |
| 691 | 677 | |
| 692 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, reg, | |
| 693 | − !(reg & MT7531_PHY_ACS_ST), 20, 100000); | |
| 678 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, reg, | |
| 679 | + !(reg & MT7531_PHY_ACS_ST), 20, 100000); | |
| 694 | 680 | if (ret < 0) { |
| 695 | 681 | dev_err(priv->dev, "poll timeout\n"); |
| 696 | 682 | goto out; |
| @@ -701,8 +687,8 @@ mt7531_ind_c22_phy_write(struct mt7530_priv *priv, int port, int regnum, · 4 unmodified lines | ||
| 701 | 687 | |
| 702 | 688 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); |
| 703 | 689 | |
| 704 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, reg, | |
| 705 | − !(reg & MT7531_PHY_ACS_ST), 20, 100000); | |
| 690 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, reg, | |
| 691 | + !(reg & MT7531_PHY_ACS_ST), 20, 100000); | |
| 706 | 692 | if (ret < 0) { |
| 707 | 693 | dev_err(priv->dev, "poll timeout\n"); |
| 708 | 694 | goto out; |
| @@ -1626,22 +1612,26 @@ mt7530_port_bridge_join(struct dsa_switch *ds, int port, · 917 unmodified lines | ||
| 1626 | 1612 | static int |
| 1627 | 1613 | mt7530_vlan_cmd(struct mt7530_priv *priv, enum mt7530_vlan_cmd cmd, u16 vid) |
| 1628 | 1614 | { |
| 1629 | − struct mt7530_dummy_poll p; | |
| 1630 | 1615 | u32 val; |
| 1631 | 1616 | int ret; |
| 1632 | 1617 | |
| 1633 | 1618 | val = VTCR_BUSY | VTCR_FUNC(cmd) | vid; |
| 1634 | 1619 | mt7530_write(priv, MT7530_VTCR, val); |
| 1635 | 1620 | |
| 1636 | − INIT_MT7530_DUMMY_POLL(&p, priv, MT7530_VTCR); | |
| 1637 | − ret = readx_poll_timeout(_mt7530_read, &p, val, | |
| 1638 | − !(val & VTCR_BUSY), 20, 20000); | |
| 1621 | + mt7530_mutex_lock(priv); | |
| 1622 | + | |
| 1623 | + ret = regmap_read_poll_timeout(priv->regmap, MT7530_VTCR, val, | |
| 1624 | + !(val & VTCR_BUSY), 20, 20000); | |
| 1625 | + if (!ret) | |
| 1626 | + ret = regmap_read(priv->regmap, MT7530_VTCR, &val); | |
| 1627 | + | |
| 1628 | + mt7530_mutex_unlock(priv); | |
| 1629 | + | |
| 1639 | 1630 | if (ret < 0) { |
| 1640 | 1631 | dev_err(priv->dev, "poll timeout\n"); |
| 1641 | 1632 | return ret; |
| 1642 | 1633 | } |
| 1643 | 1634 | |
| 1644 | − val = mt7530_read(priv, MT7530_VTCR); | |
| 1645 | 1635 | if (val & VTCR_INVALID) { |
| 1646 | 1636 | dev_err(priv->dev, "read VTCR invalid\n"); |
| 1647 | 1637 | return -EINVAL; |
| @@ -1988,16 +1988,19 @@ static void rtl8365mb_get_stats64(struct dsa_switch *ds, int port, · 1987 unmodified lines | ||
| 1988 | 1988 | spin_unlock(&p->stats_lock); |
| 1989 | 1989 | } |
| 1990 | 1990 | |
| 1991 | −static void rtl8365mb_stats_setup(struct realtek_priv *priv) | |
| 1991 | +static int rtl8365mb_stats_setup(struct realtek_priv *priv) | |
| 1992 | 1992 | { |
| 1993 | 1993 | struct rtl8365mb *mb = priv->chip_data; |
| 1994 | 1994 | struct dsa_switch *ds = &priv->ds; |
| 1995 | 1995 | struct dsa_port *dp; |
| 1996 | + int ret; | |
| 1996 | 1997 | |
| 1997 | 1998 | /* Per-chip global mutex to protect MIB counter access, since doing |
| 1998 | 1999 | * so requires accessing a series of registers in a particular order. |
| 1999 | 2000 | */ |
| 2000 | − mutex_init(&mb->mib_lock); | |
| 2001 | + ret = devm_mutex_init(priv->dev, &mb->mib_lock); | |
| 2002 | + if (ret) | |
| 2003 | + return ret; | |
| 2001 | 2004 | |
| 2002 | 2005 | dsa_switch_for_each_available_port(dp, ds) { |
| 2003 | 2006 | struct rtl8365mb_port *p = &mb->ports[dp->index]; |
| @@ -2010,6 +2013,8 @@ static void rtl8365mb_stats_setup(struct realtek_priv *priv) · 6 unmodified lines | ||
| 2010 | 2013 | */ |
| 2011 | 2014 | INIT_DELAYED_WORK(&p->mib_work, rtl8365mb_stats_poll); |
| 2012 | 2015 | } |
| 2016 | + | |
| 2017 | + return 0; | |
| 2013 | 2018 | } |
| 2014 | 2019 | |
| 2015 | 2020 | static void rtl8365mb_stats_teardown(struct realtek_priv *priv) |
| @@ -2567,7 +2572,12 @@ static int rtl8365mb_setup(struct dsa_switch *ds) · 551 unmodified lines | ||
| 2567 | 2572 | } |
| 2568 | 2573 | |
| 2569 | 2574 | /* Start statistics counter polling */ |
| 2570 | − rtl8365mb_stats_setup(priv); | |
| 2575 | + ret = rtl8365mb_stats_setup(priv); | |
| 2576 | + if (ret) { | |
| 2577 | + dev_err(priv->dev, "failed to setup stats: %pe\n", | |
| 2578 | + ERR_PTR(ret)); | |
| 2579 | + goto out_teardown_irq; | |
| 2580 | + } | |
| 2571 | 2581 | |
| 2572 | 2582 | return 0; |
| 2573 | 2583 | |
| @@ -156,9 +156,17 @@ rtl83xx_probe(struct device *dev, · 155 unmodified lines | ||
| 156 | 156 | if (!priv) |
| 157 | 157 | return ERR_PTR(-ENOMEM); |
| 158 | 158 | |
| 159 | − mutex_init(&priv->map_lock); | |
| 160 | − mutex_init(&priv->vlan_lock); | |
| 161 | − mutex_init(&priv->l2_lock); | |
| 159 | + ret = devm_mutex_init(dev, &priv->map_lock); | |
| 160 | + if (ret) | |
| 161 | + return ERR_PTR(ret); | |
| 162 | + | |
| 163 | + ret = devm_mutex_init(dev, &priv->vlan_lock); | |
| 164 | + if (ret) | |
| 165 | + return ERR_PTR(ret); | |
| 166 | + | |
| 167 | + ret = devm_mutex_init(dev, &priv->l2_lock); | |
| 168 | + if (ret) | |
| 169 | + return ERR_PTR(ret); | |
| 162 | 170 | |
| 163 | 171 | rc.lock_arg = priv; |
| 164 | 172 | priv->map = devm_regmap_init(dev, NULL, priv, &rc); |
| @@ -171,6 +171,7 @@ struct pdsc { · 170 unmodified lines | ||
| 171 | 171 | struct timer_list wdtimer; |
| 172 | 172 | unsigned int wdtimer_period; |
| 173 | 173 | struct work_struct health_work; |
| 174 | + bool health_stopped; | |
| 174 | 175 | struct devlink_health_reporter *fw_reporter; |
| 175 | 176 | u32 fw_recoveries; |
| 176 | 177 | |
| @@ -470,8 +470,10 @@ static void pdsc_stop_health_thread(struct pdsc *pdsc) · 469 unmodified lines | ||
| 470 | 470 | return; |
| 471 | 471 | |
| 472 | 472 | timer_shutdown_sync(&pdsc->wdtimer); |
| 473 | − if (pdsc->health_work.func) | |
| 474 | − cancel_work_sync(&pdsc->health_work); | |
| 473 | + if (pdsc->health_work.func && !pdsc->health_stopped) { | |
| 474 | + disable_work_sync(&pdsc->health_work); | |
| 475 | + pdsc->health_stopped = true; | |
| 476 | + } | |
| 475 | 477 | } |
| 476 | 478 | |
| 477 | 479 | static void pdsc_restart_health_thread(struct pdsc *pdsc) |
| @@ -479,6 +481,10 @@ static void pdsc_restart_health_thread(struct pdsc *pdsc) · 1 unmodified line | ||
| 479 | 481 | if (pdsc->pdev->is_virtfn) |
| 480 | 482 | return; |
| 481 | 483 | |
| 484 | + if (pdsc->health_stopped) { | |
| 485 | + enable_work(&pdsc->health_work); | |
| 486 | + pdsc->health_stopped = false; | |
| 487 | + } | |
| 482 | 488 | timer_setup(&pdsc->wdtimer, pdsc_wdtimer_cb, 0); |
| 483 | 489 | mod_timer(&pdsc->wdtimer, jiffies + 1); |
| 484 | 490 | } |
| @@ -555,7 +561,11 @@ static pci_ers_result_t pdsc_pci_error_detected(struct pci_dev *pdev, · 70 unmodified lines | ||
| 555 | 561 | pci_channel_state_t error) |
| 556 | 562 | { |
| 557 | 563 | if (error == pci_channel_io_frozen) { |
| 564 | + struct pdsc *pdsc = pci_get_drvdata(pdev); | |
| 565 | + | |
| 558 | 566 | pdsc_reset_prepare(pdev); |
| 567 | + if (!pdev->is_virtfn) | |
| 568 | + cancel_work_sync(&pdsc->pci_reset_work); | |
| 559 | 569 | return PCI_ERS_RESULT_NEED_RESET; |
| 560 | 570 | } |
| 561 | 571 | |
| @@ -360,6 +360,35 @@ out: · 359 unmodified lines | ||
| 360 | 360 | return !!budget; |
| 361 | 361 | } |
| 362 | 362 | |
| 363 | +void aq_ring_tx_deinit(struct aq_ring_s *self) | |
| 364 | +{ | |
| 365 | + if (!self) | |
| 366 | + return; | |
| 367 | + | |
| 368 | + for (; self->sw_head != self->sw_tail; | |
| 369 | + self->sw_head = aq_ring_next_dx(self, self->sw_head)) { | |
| 370 | + struct aq_ring_buff_s *buff = &self->buff_ring[self->sw_head]; | |
| 371 | + struct device *ndev = aq_nic_get_dev(self->aq_nic); | |
| 372 | + | |
| 373 | + if (buff->is_mapped) { | |
| 374 | + if (buff->is_sop) { | |
| 375 | + dma_unmap_single(ndev, buff->pa, buff->len, | |
| 376 | + DMA_TO_DEVICE); | |
| 377 | + } else { | |
| 378 | + dma_unmap_page(ndev, buff->pa, buff->len, | |
| 379 | + DMA_TO_DEVICE); | |
| 380 | + } | |
| 381 | + } | |
| 382 | + | |
| 383 | + if (buff->is_eop) { | |
| 384 | + if (buff->skb) | |
| 385 | + dev_kfree_skb_any(buff->skb); | |
| 386 | + else if (buff->xdpf) | |
| 387 | + xdp_return_frame(buff->xdpf); | |
| 388 | + } | |
| 389 | + } | |
| 390 | +} | |
| 391 | + | |
| 363 | 392 | static void aq_rx_checksum(struct aq_ring_s *self, |
| 364 | 393 | struct aq_ring_buff_s *buff, |
| 365 | 394 | struct sk_buff *skb) |
| @@ -921,15 +950,29 @@ err_exit: · 555 unmodified lines | ||
| 921 | 950 | |
| 922 | 951 | void aq_ring_rx_deinit(struct aq_ring_s *self) |
| 923 | 952 | { |
| 924 | − if (!self) | |
| 953 | + unsigned int i; | |
| 954 | + | |
| 955 | + if (!self || !self->buff_ring) | |
| 925 | 956 | return; |
| 926 | 957 | |
| 927 | − for (; self->sw_head != self->sw_tail; | |
| 928 | − self->sw_head = aq_ring_next_dx(self, self->sw_head)) { | |
| 929 | − struct aq_ring_buff_s *buff = &self->buff_ring[self->sw_head]; | |
| 958 | + /* Release every page still owned by the ring. | |
| 959 | + * | |
| 960 | + * Walking [sw_head, sw_tail) is not enough: refill is batched | |
| 961 | + * (aq_ring_rx_fill() waits for AQ_CFG_RX_REFILL_THRES free slots), | |
| 962 | + * so slots that were cleaned but not yet reposted accumulate in the | |
| 963 | + * [sw_tail, sw_head) gap, and they keep their page for reuse. Walk | |
| 964 | + * the whole ring and release whatever is left. | |
| 965 | + */ | |
| 966 | + for (i = 0; i < self->size; i++) { | |
| 967 | + struct aq_ring_buff_s *buff = &self->buff_ring[i]; | |
| 968 | + | |
| 969 | + if (!buff->rxdata.page) | |
| 970 | + continue; | |
| 930 | 971 | |
| 931 | 972 | aq_free_rxpage(&buff->rxdata, aq_nic_get_dev(self->aq_nic)); |
| 932 | 973 | } |
| 974 | + | |
| 975 | + self->sw_head = self->sw_tail; | |
| 933 | 976 | } |
| 934 | 977 | |
| 935 | 978 | void aq_ring_free(struct aq_ring_s *self) |
| @@ -202,6 +202,7 @@ void aq_ring_update_queue_state(struct aq_ring_s *ring); · 201 unmodified lines | ||
| 202 | 202 | void aq_ring_queue_wake(struct aq_ring_s *ring); |
| 203 | 203 | void aq_ring_queue_stop(struct aq_ring_s *ring); |
| 204 | 204 | bool aq_ring_tx_clean(struct aq_ring_s *self); |
| 205 | +void aq_ring_tx_deinit(struct aq_ring_s *self); | |
| 205 | 206 | int aq_xdp_xmit(struct net_device *dev, int num_frames, |
| 206 | 207 | struct xdp_frame **frames, u32 flags); |
| 207 | 208 | int aq_ring_rx_clean(struct aq_ring_s *self, |
| @@ -275,7 +275,7 @@ void aq_vec_deinit(struct aq_vec_s *self) · 274 unmodified lines | ||
| 275 | 275 | |
| 276 | 276 | for (i = 0U; self->tx_rings > i; ++i) { |
| 277 | 277 | ring = self->ring[i]; |
| 278 | − aq_ring_tx_clean(&ring[AQ_VEC_TX_ID]); | |
| 278 | + aq_ring_tx_deinit(&ring[AQ_VEC_TX_ID]); | |
| 279 | 279 | aq_ring_rx_deinit(&ring[AQ_VEC_RX_ID]); |
| 280 | 280 | } |
| 281 | 281 | |
| @@ -141,12 +141,15 @@ static void bnge_aux_dev_release(struct device *dev) · 140 unmodified lines | ||
| 141 | 141 | { |
| 142 | 142 | struct bnge_auxr_priv *aux_priv = |
| 143 | 143 | container_of(dev, struct bnge_auxr_priv, aux_dev.dev); |
| 144 | − struct bnge_dev *bd = pci_get_drvdata(aux_priv->auxr_dev->pdev); | |
| 144 | + struct bnge_auxr_dev *auxr_dev = aux_priv->auxr_dev; | |
| 145 | + struct bnge_dev *bd = pci_get_drvdata(to_pci_dev(dev->parent)); | |
| 145 | 146 | |
| 146 | 147 | ida_free(&bnge_aux_dev_ids, aux_priv->id); |
| 147 | − kfree(aux_priv->auxr_dev->auxr_info); | |
| 148 | + if (auxr_dev) { | |
| 149 | + kfree(auxr_dev->auxr_info); | |
| 150 | + kfree(auxr_dev); | |
| 151 | + } | |
| 148 | 152 | bd->auxr_dev = NULL; |
| 149 | − kfree(aux_priv->auxr_dev); | |
| 150 | 153 | kfree(aux_priv); |
| 151 | 154 | bd->aux_priv = NULL; |
| 152 | 155 | } |
| @@ -2768,8 +2768,6 @@ static int bnge_init_nic(struct bnge_net *bn) · 2767 unmodified lines | ||
| 2768 | 2768 | |
| 2769 | 2769 | err_free_ring_grps: |
| 2770 | 2770 | bnge_free_ring_grps(bn); |
| 2771 | − return rc; | |
| 2772 | − | |
| 2773 | 2771 | err_free_rx_ring_pair_bufs: |
| 2774 | 2772 | bnge_free_rx_ring_pair_bufs(bn); |
| 2775 | 2773 | return rc; |
| @@ -163,7 +163,8 @@ static int bnge_adjust_rings(struct bnge_dev *bd, u16 *rx, · 162 unmodified lines | ||
| 163 | 163 | u16 tx_chunks = bnge_num_tx_to_cp(bd, *tx); |
| 164 | 164 | |
| 165 | 165 | if (tx_chunks != *tx) { |
| 166 | − u16 tx_saved = tx_chunks, rc; | |
| 166 | + u16 tx_saved = tx_chunks; | |
| 167 | + int rc; | |
| 167 | 168 | |
| 168 | 169 | rc = bnge_fix_rings_count(rx, &tx_chunks, max_nq, sh); |
| 169 | 170 | if (rc) |
| @@ -4611,11 +4611,14 @@ static void bnxt_init_one_rx_agg_ring_rxbd(struct bnxt *bp, · 4610 unmodified lines | ||
| 4611 | 4611 | type = ((u32)rxr->rx_page_size << RX_BD_LEN_SHIFT) | |
| 4612 | 4612 | RX_BD_TYPE_RX_AGG_BD; |
| 4613 | 4613 | |
| 4614 | − /* On P7, setting EOP will cause the chip to disable | |
| 4615 | − * Relaxed Ordering (RO) for TPA data. Disable EOP for | |
| 4616 | − * potentially higher performance with RO. | |
| 4614 | + /* Disable EOP if TPA is enabled to prevent overlapping zero | |
| 4615 | + * padding with the next segment's data. On P7_PLUS, EOP will | |
| 4616 | + * automatically disable Relaxed Ordering (RO) to prevent | |
| 4617 | + * potential data corruption (and may degrade performance). On | |
| 4618 | + * older chips, RO will not be automatically disabled and may | |
| 4619 | + * cause corruption. | |
| 4617 | 4620 | */ |
| 4618 | − if (BNXT_CHIP_P5_AND_MINUS(bp) || !(bp->flags & BNXT_FLAG_TPA)) | |
| 4621 | + if (!(bp->flags & BNXT_FLAG_TPA)) | |
| 4619 | 4622 | type |= RX_BD_FLAGS_AGG_EOP; |
| 4620 | 4623 | |
| 4621 | 4624 | bnxt_init_rxbd_pages(ring, type); |
| @@ -6704,22 +6707,36 @@ int bnxt_get_nr_rss_ctxs(struct bnxt *bp, int rx_rings) · 2082 unmodified lines | ||
| 6704 | 6707 | static void bnxt_fill_hw_rss_tbl(struct bnxt *bp, struct bnxt_vnic_info *vnic) |
| 6705 | 6708 | { |
| 6706 | 6709 | bool no_rss = !(vnic->flags & BNXT_VNIC_RSS_FLAG); |
| 6707 | − u16 i, j; | |
| 6710 | + u16 i, j, min_j = bp->rx_nr_rings - 1; | |
| 6711 | + | |
| 6712 | + if (!vnic->rss_table) | |
| 6713 | + goto skip_rss_tbl; | |
| 6708 | 6714 | |
| 6709 | 6715 | /* Fill the RSS indirection table with ring group ids */ |
| 6710 | 6716 | for (i = 0, j = 0; i < HW_HASH_INDEX_SIZE; i++) { |
| 6711 | 6717 | if (!no_rss) |
| 6712 | 6718 | j = bp->rss_indir_tbl[i]; |
| 6719 | + min_j = min(j, min_j); | |
| 6713 | 6720 | vnic->rss_table[i] = cpu_to_le16(vnic->fw_grp_ids[j]); |
| 6714 | 6721 | } |
| 6722 | + | |
| 6723 | +skip_rss_tbl: | |
| 6724 | + if (vnic->rss_table && !no_rss) | |
| 6725 | + vnic->default_rx_ring = min_j; | |
| 6726 | + else if (vnic->flags & BNXT_VNIC_RFS_FLAG) | |
| 6727 | + vnic->default_rx_ring = vnic->vnic_id - 1; | |
| 6728 | + else if ((vnic->vnic_id == 1) && BNXT_CHIP_TYPE_NITRO_A0(bp)) | |
| 6729 | + vnic->default_rx_ring = bp->rx_nr_rings - 1; | |
| 6730 | + else | |
| 6731 | + vnic->default_rx_ring = 0; | |
| 6715 | 6732 | } |
| 6716 | 6733 | |
| 6717 | 6734 | static void bnxt_fill_hw_rss_tbl_p5(struct bnxt *bp, |
| 6718 | 6735 | struct bnxt_vnic_info *vnic) |
| 6719 | 6736 | { |
| 6737 | + u16 tbl_size, i, min_j = bp->rx_nr_rings - 1; | |
| 6720 | 6738 | __le16 *ring_tbl = vnic->rss_table; |
| 6721 | 6739 | struct bnxt_rx_ring_info *rxr; |
| 6722 | − u16 tbl_size, i; | |
| 6723 | 6740 | |
| 6724 | 6741 | tbl_size = bnxt_get_rxfh_indir_size(bp->dev); |
| 6725 | 6742 | |
| @@ -6732,6 +6749,7 @@ static void bnxt_fill_hw_rss_tbl_p5(struct bnxt *bp, · 6 unmodified lines | ||
| 6732 | 6749 | j = ethtool_rxfh_context_indir(vnic->rss_ctx)[i]; |
| 6733 | 6750 | else |
| 6734 | 6751 | j = bp->rss_indir_tbl[i]; |
| 6752 | + min_j = min(j, min_j); | |
| 6735 | 6753 | rxr = &bp->rx_ring[j]; |
| 6736 | 6754 | |
| 6737 | 6755 | ring_id = rxr->rx_ring_struct.fw_ring_id; |
| @@ -6739,19 +6757,15 @@ static void bnxt_fill_hw_rss_tbl_p5(struct bnxt *bp, · 1 unmodified line | ||
| 6739 | 6757 | ring_id = bnxt_cp_ring_for_rx(bp, rxr); |
| 6740 | 6758 | *ring_tbl++ = cpu_to_le16(ring_id); |
| 6741 | 6759 | } |
| 6760 | + vnic->default_rx_ring = min_j; | |
| 6742 | 6761 | } |
| 6743 | 6762 | |
| 6744 | 6763 | static void |
| 6745 | 6764 | __bnxt_hwrm_vnic_set_rss(struct bnxt *bp, struct hwrm_vnic_rss_cfg_input *req, |
| 6746 | 6765 | struct bnxt_vnic_info *vnic) |
| 6747 | 6766 | { |
| 6748 | − if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) { | |
| 6749 | − bnxt_fill_hw_rss_tbl_p5(bp, vnic); | |
| 6750 | − if (bp->flags & BNXT_FLAG_CHIP_P7) | |
| 6751 | − req->flags |= VNIC_RSS_CFG_REQ_FLAGS_IPSEC_HASH_TYPE_CFG_SUPPORT; | |
| 6752 | − } else { | |
| 6753 | − bnxt_fill_hw_rss_tbl(bp, vnic); | |
| 6754 | − } | |
| 6767 | + if (bp->flags & BNXT_FLAG_CHIP_P7) | |
| 6768 | + req->flags |= VNIC_RSS_CFG_REQ_FLAGS_IPSEC_HASH_TYPE_CFG_SUPPORT; | |
| 6755 | 6769 | |
| 6756 | 6770 | if (bp->rss_hash_delta) { |
| 6757 | 6771 | req->hash_type = cpu_to_le32(bp->rss_hash_delta); |
| @@ -6803,6 +6817,7 @@ static int bnxt_hwrm_vnic_set_rss_p5(struct bnxt *bp, · 45 unmodified lines | ||
| 6803 | 6817 | if (!set_rss) |
| 6804 | 6818 | return hwrm_req_send(bp, req); |
| 6805 | 6819 | |
| 6820 | + bnxt_fill_hw_rss_tbl_p5(bp, vnic); | |
| 6806 | 6821 | __bnxt_hwrm_vnic_set_rss(bp, req, vnic); |
| 6807 | 6822 | ring_tbl_map = vnic->rss_table_dma_addr; |
| 6808 | 6823 | nr_ctxs = bnxt_get_nr_rss_ctxs(bp, bp->rx_nr_rings); |
| @@ -6939,8 +6954,9 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic) · 130 unmodified lines | ||
| 6939 | 6954 | return rc; |
| 6940 | 6955 | |
| 6941 | 6956 | if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) { |
| 6942 | − struct bnxt_rx_ring_info *rxr = &bp->rx_ring[0]; | |
| 6957 | + struct bnxt_rx_ring_info *rxr; | |
| 6943 | 6958 | |
| 6959 | + rxr = &bp->rx_ring[vnic->default_rx_ring]; | |
| 6944 | 6960 | req->default_rx_ring_id = |
| 6945 | 6961 | cpu_to_le16(rxr->rx_ring_struct.fw_ring_id); |
| 6946 | 6962 | req->default_cmpl_ring_id = |
| @@ -6973,13 +6989,7 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic) · 26 unmodified lines | ||
| 6973 | 6989 | req->cos_rule = cpu_to_le16(0xffff); |
| 6974 | 6990 | } |
| 6975 | 6991 | |
| 6976 | − if (vnic->flags & BNXT_VNIC_RSS_FLAG) | |
| 6977 | − ring = 0; | |
| 6978 | − else if (vnic->flags & BNXT_VNIC_RFS_FLAG) | |
| 6979 | − ring = vnic->vnic_id - 1; | |
| 6980 | − else if ((vnic->vnic_id == 1) && BNXT_CHIP_TYPE_NITRO_A0(bp)) | |
| 6981 | − ring = bp->rx_nr_rings - 1; | |
| 6982 | − | |
| 6992 | + ring = vnic->default_rx_ring; | |
| 6983 | 6993 | grp_idx = bp->rx_ring[ring].bnapi->index; |
| 6984 | 6994 | req->dflt_ring_grp = cpu_to_le16(bp->grp_info[grp_idx].fw_grp_id); |
| 6985 | 6995 | req->lb_rule = cpu_to_le16(0xffff); |
| @@ -10866,6 +10876,7 @@ static int __bnxt_setup_vnic(struct bnxt *bp, struct bnxt_vnic_info *vnic) · 3880 unmodified lines | ||
| 10866 | 10876 | } |
| 10867 | 10877 | |
| 10868 | 10878 | skip_rss_ctx: |
| 10879 | + bnxt_fill_hw_rss_tbl(bp, vnic); | |
| 10869 | 10880 | /* configure default vnic, ring grp */ |
| 10870 | 10881 | rc = bnxt_hwrm_vnic_cfg(bp, vnic); |
| 10871 | 10882 | if (rc) { |
| @@ -11090,6 +11101,11 @@ static int bnxt_set_vnic_mru_p5(struct bnxt *bp, struct bnxt_vnic_info *vnic, · 218 unmodified lines | ||
| 11090 | 11101 | vnic->vnic_id, rc); |
| 11091 | 11102 | return rc; |
| 11092 | 11103 | } |
| 11104 | + if (rxr_id == vnic->default_rx_ring) { | |
| 11105 | + rc = bnxt_hwrm_vnic_cfg(bp, vnic); | |
| 11106 | + if (rc) | |
| 11107 | + return rc; | |
| 11108 | + } | |
| 11093 | 11109 | } |
| 11094 | 11110 | vnic->mru = mru; |
| 11095 | 11111 | bnxt_hwrm_vnic_update(bp, vnic, |
| @@ -11171,6 +11187,9 @@ static int bnxt_setup_nitroa0_vnic(struct bnxt *bp) · 75 unmodified lines | ||
| 11171 | 11187 | return rc; |
| 11172 | 11188 | } |
| 11173 | 11189 | |
| 11190 | + /* Setup the proper default RX ring */ | |
| 11191 | + bnxt_fill_hw_rss_tbl(bp, vnic); | |
| 11192 | + | |
| 11174 | 11193 | rc = bnxt_hwrm_vnic_cfg(bp, vnic); |
| 11175 | 11194 | if (rc) { |
| 11176 | 11195 | netdev_err(bp->dev, "Cannot allocate special vnic for NS2 A0: %x\n", |
| @@ -16217,6 +16236,7 @@ static int bnxt_queue_mem_alloc(struct net_device *dev, · 5040 unmodified lines | ||
| 16217 | 16236 | clone->rx_next_cons = 0; |
| 16218 | 16237 | clone->need_head_pool = false; |
| 16219 | 16238 | clone->rx_page_size = qcfg->rx_page_size; |
| 16239 | + clone->rx_agg_bmap = NULL; | |
| 16220 | 16240 | |
| 16221 | 16241 | rc = bnxt_alloc_rx_page_pool(bp, clone, rxr->page_pool->p.nid); |
| 16222 | 16242 | if (rc) |
| @@ -16269,6 +16289,8 @@ err_free_tpa_info: · 46 unmodified lines | ||
| 16269 | 16289 | bnxt_free_one_tpa_info(bp, clone); |
| 16270 | 16290 | err_free_rx_agg_ring: |
| 16271 | 16291 | bnxt_free_ring(bp, &clone->rx_agg_ring_struct.ring_mem); |
| 16292 | + kfree(clone->rx_agg_bmap); | |
| 16293 | + clone->rx_agg_bmap = NULL; | |
| 16272 | 16294 | err_free_rx_ring: |
| 16273 | 16295 | bnxt_free_ring(bp, &clone->rx_ring_struct.ring_mem); |
| 16274 | 16296 | err_rxq_info_unreg: |
| @@ -1334,6 +1334,7 @@ struct bnxt_vnic_info { · 1333 unmodified lines | ||
| 1334 | 1334 | #define BNXT_VNIC_RSSCTX_FLAG 0x40 |
| 1335 | 1335 | struct ethtool_rxfh_context *rss_ctx; |
| 1336 | 1336 | u32 vnic_id; |
| 1337 | + u16 default_rx_ring; | |
| 1337 | 1338 | }; |
| 1338 | 1339 | |
| 1339 | 1340 | struct bnxt_rss_ctx { |
| @@ -495,12 +495,15 @@ static int bnxt_ptp_enable(struct ptp_clock_info *ptp_info, · 494 unmodified lines | ||
| 495 | 495 | return rc; |
| 496 | 496 | case PTP_CLK_REQ_PPS: |
| 497 | 497 | /* Configure PHC PPS IN */ |
| 498 | − rc = bnxt_ptp_cfg_pin(bp, 0, BNXT_PPS_PIN_PPS_IN); | |
| 498 | + pin_id = 0; | |
| 499 | + if (!on) | |
| 500 | + break; | |
| 501 | + rc = bnxt_ptp_cfg_pin(bp, pin_id, BNXT_PPS_PIN_PPS_IN); | |
| 499 | 502 | if (rc) |
| 500 | 503 | return rc; |
| 501 | 504 | rc = bnxt_ptp_cfg_event(bp, BNXT_PPS_EVENT_INTERNAL); |
| 502 | 505 | if (!rc) |
| 503 | − ptp->pps_info.pins[0].event = BNXT_PPS_EVENT_INTERNAL; | |
| 506 | + ptp->pps_info.pins[pin_id].event = BNXT_PPS_EVENT_INTERNAL; | |
| 504 | 507 | return rc; |
| 505 | 508 | default: |
| 506 | 509 | netdev_err(ptp->bp->dev, "Unrecognized PIN function\n"); |
| @@ -3011,8 +3011,9 @@ static void enic_remove(struct pci_dev *pdev) · 3010 unmodified lines | ||
| 3011 | 3011 | if (netdev) { |
| 3012 | 3012 | struct enic *enic = netdev_priv(netdev); |
| 3013 | 3013 | |
| 3014 | − cancel_work_sync(&enic->reset); | |
| 3015 | − cancel_work_sync(&enic->change_mtu_work); | |
| 3014 | + disable_work_sync(&enic->reset); | |
| 3015 | + disable_work_sync(&enic->tx_hang_reset); | |
| 3016 | + disable_work_sync(&enic->change_mtu_work); | |
| 3016 | 3017 | unregister_netdev(netdev); |
| 3017 | 3018 | enic_dev_deinit(enic); |
| 3018 | 3019 | vnic_dev_close(enic->vdev); |
| @@ -1282,7 +1282,6 @@ static void hix5hd2_dev_remove(struct platform_device *pdev) · 1281 unmodified lines | ||
| 1282 | 1282 | struct net_device *ndev = platform_get_drvdata(pdev); |
| 1283 | 1283 | struct hix5hd2_priv *priv = netdev_priv(ndev); |
| 1284 | 1284 | |
| 1285 | − netif_napi_del(&priv->napi); | |
| 1286 | 1285 | unregister_netdev(ndev); |
| 1287 | 1286 | mdiobus_unregister(priv->bus); |
| 1288 | 1287 | mdiobus_free(priv->bus); |
| @@ -9498,12 +9498,8 @@ static int hclge_init_ae_dev(struct hnae3_ae_dev *ae_dev) · 9497 unmodified lines | ||
| 9498 | 9498 | if (ret) |
| 9499 | 9499 | goto err_ptp_uninit; |
| 9500 | 9500 | |
| 9501 | − if (hdev->hw.mac.media_type != HNAE3_MEDIA_TYPE_COPPER) { | |
| 9501 | + if (hdev->hw.mac.media_type != HNAE3_MEDIA_TYPE_COPPER) | |
| 9502 | 9502 | hdev->hw.mac.req_autoneg = hdev->hw.mac.autoneg; |
| 9503 | − if (hdev->hw.mac.autoneg == AUTONEG_DISABLE && | |
| 9504 | − hdev->hw.mac.speed != SPEED_UNKNOWN) | |
| 9505 | − hdev->hw.mac.req_speed = hdev->hw.mac.speed; | |
| 9506 | − } | |
| 9507 | 9503 | |
| 9508 | 9504 | ret = hclge_set_autoneg_speed_dup(hdev); |
| 9509 | 9505 | if (ret) { |
| @@ -1222,11 +1222,11 @@ err_eeprom: · 1221 unmodified lines | ||
| 1222 | 1222 | |
| 1223 | 1223 | if (hw->flash_address) |
| 1224 | 1224 | iounmap(hw->flash_address); |
| 1225 | +err_mdio_ioremap: | |
| 1225 | 1226 | kfree(adapter->tx_ring); |
| 1226 | 1227 | kfree(adapter->rx_ring); |
| 1227 | 1228 | err_dma: |
| 1228 | 1229 | err_sw_init: |
| 1229 | −err_mdio_ioremap: | |
| 1230 | 1230 | iounmap(hw->ce4100_gbe_mdio_base_virt); |
| 1231 | 1231 | iounmap(hw->hw_addr); |
| 1232 | 1232 | err_ioremap: |
| @@ -793,7 +793,7 @@ err: · 792 unmodified lines | ||
| 793 | 793 | ret, |
| 794 | 794 | libie_aq_str(pf->hw.adminq.sq_last_status), |
| 795 | 795 | pin_type_name[pin_type], pin->idx); |
| 796 | − else | |
| 796 | + else if (pf->hw.adminq.sq_last_status != LIBIE_AQ_RC_EBUSY) | |
| 797 | 797 | dev_err_ratelimited(ice_pf_to_dev(pf), |
| 798 | 798 | "err:%d %s failed to update %s pin:%u\n", |
| 799 | 799 | ret, |
| @@ -3024,7 +3024,8 @@ static int ice_dpll_pps_update_phase_offsets(struct ice_pf *pf, · 2224 unmodified lines | ||
| 3024 | 3024 | *phase_offset_pins_updated = 0; |
| 3025 | 3025 | ret = ice_aq_get_cgu_input_pin_measure(&pf->hw, DPLL_TYPE_PPS, meas, |
| 3026 | 3026 | ARRAY_SIZE(meas)); |
| 3027 | − if (ret && pf->hw.adminq.sq_last_status == LIBIE_AQ_RC_EAGAIN) { | |
| 3027 | + if (ret && (pf->hw.adminq.sq_last_status == LIBIE_AQ_RC_EAGAIN || | |
| 3028 | + pf->hw.adminq.sq_last_status == LIBIE_AQ_RC_EBUSY)) { | |
| 3028 | 3029 | return 0; |
| 3029 | 3030 | } else if (ret) { |
| 3030 | 3031 | dev_err(ice_pf_to_dev(pf), |
| @@ -3086,10 +3087,12 @@ ice_dpll_update_state(struct ice_pf *pf, struct ice_dpll *d, bool init) · 55 unmodified lines | ||
| 3086 | 3087 | d->dpll_idx, d->prev_input_idx, d->input_idx, |
| 3087 | 3088 | d->dpll_state, d->prev_dpll_state, d->mode); |
| 3088 | 3089 | if (ret) { |
| 3089 | − dev_err(ice_pf_to_dev(pf), | |
| 3090 | − "update dpll=%d state failed, ret=%d %s\n", | |
| 3091 | − d->dpll_idx, ret, | |
| 3092 | − libie_aq_str(pf->hw.adminq.sq_last_status)); | |
| 3090 | + /* EBUSY is expected during reset recovery, don't log error */ | |
| 3091 | + if (pf->hw.adminq.sq_last_status != LIBIE_AQ_RC_EBUSY) | |
| 3092 | + dev_err(ice_pf_to_dev(pf), | |
| 3093 | + "update dpll=%d state failed, ret=%d %s\n", | |
| 3094 | + d->dpll_idx, ret, | |
| 3095 | + libie_aq_str(pf->hw.adminq.sq_last_status)); | |
| 3093 | 3096 | return ret; |
| 3094 | 3097 | } |
| 3095 | 3098 | if (init) { |
| @@ -3158,7 +3161,9 @@ static void ice_dpll_periodic_work(struct kthread_work *work) · 62 unmodified lines | ||
| 3158 | 3161 | d->periodic_counter % dp->phase_offset_monitor_period == 0) |
| 3159 | 3162 | ret = ice_dpll_pps_update_phase_offsets(pf, &phase_offset_ntf); |
| 3160 | 3163 | if (ret) { |
| 3161 | − d->cgu_state_acq_err_num++; | |
| 3164 | + /* EBUSY is expected during reset recovery */ | |
| 3165 | + if (pf->hw.adminq.sq_last_status != LIBIE_AQ_RC_EBUSY) | |
| 3166 | + d->cgu_state_acq_err_num++; | |
| 3162 | 3167 | /* stop rescheduling this worker */ |
| 3163 | 3168 | if (d->cgu_state_acq_err_num > |
| 3164 | 3169 | ICE_CGU_STATE_ACQ_ERR_THRESHOLD) { |
| @@ -1069,18 +1069,18 @@ static int ice_lbtest_prepare_rings(struct ice_vsi *vsi) · 1068 unmodified lines | ||
| 1069 | 1069 | |
| 1070 | 1070 | status = ice_vsi_cfg_lan(vsi); |
| 1071 | 1071 | if (status) |
| 1072 | − goto err_setup_rx_ring; | |
| 1072 | + goto err_cfg_lan; | |
| 1073 | 1073 | |
| 1074 | 1074 | status = ice_vsi_start_all_rx_rings(vsi); |
| 1075 | 1075 | if (status) |
| 1076 | − goto err_start_rx_ring; | |
| 1076 | + goto err_cfg_lan; | |
| 1077 | 1077 | |
| 1078 | 1078 | return 0; |
| 1079 | 1079 | |
| 1080 | −err_start_rx_ring: | |
| 1081 | − ice_vsi_free_rx_rings(vsi); | |
| 1082 | −err_setup_rx_ring: | |
| 1080 | +err_cfg_lan: | |
| 1083 | 1081 | ice_vsi_stop_lan_tx_rings(vsi, ICE_NO_RESET, 0); |
| 1082 | +err_setup_rx_ring: | |
| 1083 | + ice_vsi_free_rx_rings(vsi); | |
| 1084 | 1084 | err_setup_tx_ring: |
| 1085 | 1085 | ice_vsi_free_tx_rings(vsi); |
| 1086 | 1086 | |
| @@ -5637,6 +5637,16 @@ static int ice_resume(struct device *dev) · 5636 unmodified lines | ||
| 5637 | 5637 | /* Restart the service task */ |
| 5638 | 5638 | mod_timer(&pf->serv_tmr, round_jiffies(jiffies + pf->serv_tmr_period)); |
| 5639 | 5639 | |
| 5640 | + /* Best-effort wait for the scheduled reset to finish so that the | |
| 5641 | + * device is operational before returning. Without this, userspace | |
| 5642 | + * (e.g. NetworkManager) may try to open the net device while the | |
| 5643 | + * asynchronous reset is still in progress, hitting -EBUSY. | |
| 5644 | + */ | |
| 5645 | + ret = ice_wait_for_reset(pf, secs_to_jiffies(10)); | |
| 5646 | + if (ret) | |
| 5647 | + dev_err(dev, "Wait for reset timed out (10s) during resume: %d\n", | |
| 5648 | + ret); | |
| 5649 | + | |
| 5640 | 5650 | return 0; |
| 5641 | 5651 | } |
| 5642 | 5652 | |
| @@ -848,6 +848,30 @@ static void ice_notify_vf_reset(struct ice_vf *vf) · 847 unmodified lines | ||
| 848 | 848 | NULL); |
| 849 | 849 | } |
| 850 | 850 | |
| 851 | +/** | |
| 852 | + * ice_reset_interrupts - clear all queue interrupt configuration for a VSI | |
| 853 | + * @vsi: the VSI whose interrupt registers should be cleared | |
| 854 | + * | |
| 855 | + * Zero the QINT_RQCTL and QINT_TQCTL registers for all allocated queues | |
| 856 | + * in the VSI. This clears the entire register including MSIX_INDX, ITR_INDX, | |
| 857 | + * CAUSE_ENA and NEXTQ fields, unlike ice_vf_dis_rxq_interrupt() which only | |
| 858 | + * clears the CAUSE_ENA bit. | |
| 859 | + */ | |
| 860 | +void ice_reset_interrupts(struct ice_vsi *vsi) | |
| 861 | +{ | |
| 862 | + struct ice_pf *pf = vsi->back; | |
| 863 | + struct ice_hw *hw = &pf->hw; | |
| 864 | + int i; | |
| 865 | + | |
| 866 | + ice_for_each_alloc_rxq(vsi, i) | |
| 867 | + wr32(hw, QINT_RQCTL(vsi->rxq_map[i]), 0); | |
| 868 | + | |
| 869 | + ice_for_each_alloc_txq(vsi, i) | |
| 870 | + wr32(hw, QINT_TQCTL(vsi->txq_map[i]), 0); | |
| 871 | + | |
| 872 | + ice_flush(hw); | |
| 873 | +} | |
| 874 | + | |
| 851 | 875 | /** |
| 852 | 876 | * ice_reset_vf - Reset a particular VF |
| 853 | 877 | * @vf: pointer to the VF structure |
| @@ -919,6 +943,9 @@ int ice_reset_vf(struct ice_vf *vf, u32 flags) · 65 unmodified lines | ||
| 919 | 943 | |
| 920 | 944 | ice_dis_vf_qs(vf); |
| 921 | 945 | |
| 946 | + /* cleanup interrupt registers */ | |
| 947 | + ice_reset_interrupts(vsi); | |
| 948 | + | |
| 922 | 949 | /* Call Disable LAN Tx queue AQ whether or not queues are |
| 923 | 950 | * enabled. This is needed for successful completion of VFR. |
| 924 | 951 | */ |
| @@ -26,6 +26,7 @@ · 25 unmodified lines | ||
| 26 | 26 | void ice_initialize_vf_entry(struct ice_vf *vf); |
| 27 | 27 | void ice_deinitialize_vf_entry(struct ice_vf *vf); |
| 28 | 28 | void ice_dis_vf_qs(struct ice_vf *vf); |
| 29 | +void ice_reset_interrupts(struct ice_vsi *vsi); | |
| 29 | 30 | int ice_check_vf_init(struct ice_vf *vf); |
| 30 | 31 | enum virtchnl_status_code ice_err_to_virt_err(int err); |
| 31 | 32 | struct ice_port_info *ice_vf_get_port_info(struct ice_vf *vf); |
| @@ -224,6 +224,24 @@ void ice_vf_ena_rxq_interrupt(struct ice_vsi *vsi, u32 q_idx) · 223 unmodified lines | ||
| 224 | 224 | wr32(hw, QINT_RQCTL(pfq), reg | QINT_RQCTL_CAUSE_ENA_M); |
| 225 | 225 | } |
| 226 | 226 | |
| 227 | +/** | |
| 228 | + * ice_vf_dis_rxq_interrupt - disable Rx queue interrupt via QINT_RQCTL | |
| 229 | + * @vsi: VSI of the VF to configure | |
| 230 | + * @q_idx: VF queue index used to determine the queue in the PF's space | |
| 231 | + */ | |
| 232 | +static void ice_vf_dis_rxq_interrupt(struct ice_vsi *vsi, u32 q_idx) | |
| 233 | +{ | |
| 234 | + struct ice_hw *hw = &vsi->back->hw; | |
| 235 | + u32 pfq = vsi->rxq_map[q_idx]; | |
| 236 | + u32 reg; | |
| 237 | + | |
| 238 | + reg = rd32(hw, QINT_RQCTL(pfq)); | |
| 239 | + reg &= ~QINT_RQCTL_CAUSE_ENA_M; | |
| 240 | + wr32(hw, QINT_RQCTL(pfq), reg); | |
| 241 | + | |
| 242 | + ice_flush(hw); | |
| 243 | +} | |
| 244 | + | |
| 227 | 245 | /** |
| 228 | 246 | * ice_vc_ena_qs_msg |
| 229 | 247 | * @vf: pointer to the VF info |
| @@ -416,6 +434,8 @@ int ice_vc_dis_qs_msg(struct ice_vf *vf, u8 *msg) · 186 unmodified lines | ||
| 416 | 434 | goto error_param; |
| 417 | 435 | } |
| 418 | 436 | |
| 437 | + for_each_set_bit(vf_q_id, &q_map, ICE_MAX_RSS_QS_PER_VF) | |
| 438 | + ice_vf_dis_rxq_interrupt(vsi, vf_q_id); | |
| 419 | 439 | bitmap_zero(vf->rxq_ena, ICE_MAX_RSS_QS_PER_VF); |
| 420 | 440 | } else if (q_map) { |
| 421 | 441 | for_each_set_bit(vf_q_id, &q_map, ICE_MAX_RSS_QS_PER_VF) { |
| @@ -436,6 +456,7 @@ int ice_vc_dis_qs_msg(struct ice_vf *vf, u8 *msg) · 14 unmodified lines | ||
| 436 | 456 | goto error_param; |
| 437 | 457 | } |
| 438 | 458 | |
| 459 | + ice_vf_dis_rxq_interrupt(vsi, vf_q_id); | |
| 439 | 460 | /* Clear enabled queues flag */ |
| 440 | 461 | clear_bit(vf_q_id, vf->rxq_ena); |
| 441 | 462 | } |
| @@ -87,7 +87,7 @@ static int idpf_intr_reg_init(struct idpf_vport *vport, · 86 unmodified lines | ||
| 87 | 87 | if (!reg_vals) |
| 88 | 88 | return -ENOMEM; |
| 89 | 89 | |
| 90 | − num_regs = idpf_get_reg_intr_vecs(adapter, reg_vals); | |
| 90 | + num_regs = idpf_get_reg_intr_vecs(adapter, reg_vals, total_vecs); | |
| 91 | 91 | if (num_regs < num_vecs) { |
| 92 | 92 | err = -EINVAL; |
| 93 | 93 | goto free_reg_vals; |
| @@ -139,7 +139,7 @@ static int idpf_mb_intr_req_irq(struct idpf_adapter *adapter) · 138 unmodified lines | ||
| 139 | 139 | if (err) { |
| 140 | 140 | dev_err(&adapter->pdev->dev, |
| 141 | 141 | "IRQ request for mailbox failed, error: %d\n", err); |
| 142 | − | |
| 142 | + kfree(name); | |
| 143 | 143 | return err; |
| 144 | 144 | } |
| 145 | 145 | |
| @@ -3097,10 +3097,7 @@ static netdev_tx_t idpf_tx_splitq_frame(struct sk_buff *skb, · 3096 unmodified lines | ||
| 3097 | 3097 | |
| 3098 | 3098 | tx_params.dtype = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE; |
| 3099 | 3099 | tx_params.eop_cmd = IDPF_TXD_FLEX_FLOW_CMD_EOP; |
| 3100 | − /* Set the RE bit to periodically "clean" the descriptor ring. | |
| 3101 | − * MIN_GAP is set to MIN_RING size to ensure it will be set at | |
| 3102 | − * least once each time around the ring. | |
| 3103 | − */ | |
| 3100 | + /* Set the RE bit periodically to "clean" the descriptor ring */ | |
| 3104 | 3101 | if (idpf_tx_splitq_need_re(tx_q)) { |
| 3105 | 3102 | tx_params.eop_cmd |= IDPF_TXD_FLEX_FLOW_CMD_RE; |
| 3106 | 3103 | tx_q->txq_grp->num_completions_pending++; |
| @@ -21,7 +21,7 @@ · 20 unmodified lines | ||
| 21 | 21 | /* Mailbox Queue */ |
| 22 | 22 | #define IDPF_MAX_MBXQ 1 |
| 23 | 23 | |
| 24 | −#define IDPF_MIN_TXQ_DESC 64 | |
| 24 | +#define IDPF_MIN_TXQ_DESC 128 | |
| 25 | 25 | #define IDPF_MIN_RXQ_DESC 64 |
| 26 | 26 | #define IDPF_MIN_TXQ_COMPLQ_DESC 256 |
| 27 | 27 | #define IDPF_MAX_QIDS 256 |
| @@ -86,7 +86,7 @@ static int idpf_vf_intr_reg_init(struct idpf_vport *vport, · 85 unmodified lines | ||
| 86 | 86 | if (!reg_vals) |
| 87 | 87 | return -ENOMEM; |
| 88 | 88 | |
| 89 | − num_regs = idpf_get_reg_intr_vecs(adapter, reg_vals); | |
| 89 | + num_regs = idpf_get_reg_intr_vecs(adapter, reg_vals, total_vecs); | |
| 90 | 90 | if (num_regs < num_vecs) { |
| 91 | 91 | err = -EINVAL; |
| 92 | 92 | goto free_reg_vals; |
| @@ -1318,11 +1318,12 @@ idpf_vport_init_queue_reg_chunks(struct idpf_vport_config *vport_config, · 1317 unmodified lines | ||
| 1318 | 1318 | * idpf_get_reg_intr_vecs - Get vector queue register offset |
| 1319 | 1319 | * @adapter: adapter structure to get the vector chunks |
| 1320 | 1320 | * @reg_vals: Register offsets to store in |
| 1321 | + * @num_vecs: number of entries the @reg_vals array can hold | |
| 1321 | 1322 | * |
| 1322 | 1323 | * Return: number of registers that got populated |
| 1323 | 1324 | */ |
| 1324 | 1325 | int idpf_get_reg_intr_vecs(struct idpf_adapter *adapter, |
| 1325 | − struct idpf_vec_regs *reg_vals) | |
| 1326 | + struct idpf_vec_regs *reg_vals, int num_vecs) | |
| 1326 | 1327 | { |
| 1327 | 1328 | struct virtchnl2_vector_chunks *chunks; |
| 1328 | 1329 | struct idpf_vec_regs reg_val; |
| @@ -1346,7 +1347,7 @@ int idpf_get_reg_intr_vecs(struct idpf_adapter *adapter, · 17 unmodified lines | ||
| 1346 | 1347 | dynctl_reg_spacing = le32_to_cpu(chunk->dynctl_reg_spacing); |
| 1347 | 1348 | itrn_reg_spacing = le32_to_cpu(chunk->itrn_reg_spacing); |
| 1348 | 1349 | |
| 1349 | − for (i = 0; i < num_vec; i++) { | |
| 1350 | + for (i = 0; i < num_vec && num_regs < num_vecs; i++) { | |
| 1350 | 1351 | reg_vals[num_regs].dyn_ctl_reg = reg_val.dyn_ctl_reg; |
| 1351 | 1352 | reg_vals[num_regs].itrn_reg = reg_val.itrn_reg; |
| 1352 | 1353 | reg_vals[num_regs].itrn_index_spacing = |
| @@ -104,7 +104,7 @@ int idpf_vc_core_init(struct idpf_adapter *adapter); · 103 unmodified lines | ||
| 104 | 104 | void idpf_vc_core_deinit(struct idpf_adapter *adapter); |
| 105 | 105 | |
| 106 | 106 | int idpf_get_reg_intr_vecs(struct idpf_adapter *adapter, |
| 107 | − struct idpf_vec_regs *reg_vals); | |
| 107 | + struct idpf_vec_regs *reg_vals, int num_vecs); | |
| 108 | 108 | int idpf_queue_reg_init(struct idpf_vport *vport, |
| 109 | 109 | struct idpf_q_vec_rsrc *rsrc, |
| 110 | 110 | struct idpf_queue_id_reg_info *chunks); |
| @@ -2190,8 +2190,6 @@ dma_error: · 2189 unmodified lines | ||
| 2190 | 2190 | buffer_info->time_stamp = 0; |
| 2191 | 2191 | buffer_info->length = 0; |
| 2192 | 2192 | buffer_info->mapped_as_page = false; |
| 2193 | − if (count) | |
| 2194 | − count--; | |
| 2195 | 2193 | |
| 2196 | 2194 | /* clear timestamp and dma mappings for remaining portion of packet */ |
| 2197 | 2195 | while (count--) { |
| @@ -3082,7 +3082,7 @@ static void igc_xdp_xmit_zc(struct igc_ring *ring) · 3081 unmodified lines | ||
| 3082 | 3082 | meta_req.tx_buffer = bi; |
| 3083 | 3083 | meta_req.meta = meta; |
| 3084 | 3084 | meta_req.used_desc = 0; |
| 3085 | − xsk_tx_metadata_request(meta, &igc_xsk_tx_metadata_ops, | |
| 3085 | + xsk_tx_metadata_request(pool, &meta, &igc_xsk_tx_metadata_ops, | |
| 3086 | 3086 | &meta_req); |
| 3087 | 3087 | |
| 3088 | 3088 | /* xsk_tx_metadata_request() may have updated next_to_use */ |
| @@ -5352,9 +5352,8 @@ void igc_down(struct igc_adapter *adapter) · 2263 unmodified lines | ||
| 5352 | 5352 | |
| 5353 | 5353 | for (i = 0; i < adapter->num_q_vectors; i++) { |
| 5354 | 5354 | if (adapter->q_vector[i]) { |
| 5355 | − napi_synchronize(&adapter->q_vector[i]->napi); | |
| 5356 | − igc_set_queue_napi(adapter, i, NULL); | |
| 5357 | 5355 | napi_disable(&adapter->q_vector[i]->napi); |
| 5356 | + igc_set_queue_napi(adapter, i, NULL); | |
| 5358 | 5357 | } |
| 5359 | 5358 | } |
| 5360 | 5359 | |
| @@ -7586,11 +7585,13 @@ static int __igc_resume(struct device *dev, bool rpm) · 2225 unmodified lines | ||
| 7586 | 7585 | err = __igc_open(netdev, true); |
| 7587 | 7586 | if (!rpm) |
| 7588 | 7587 | rtnl_unlock(); |
| 7589 | − if (!err) | |
| 7590 | − netif_device_attach(netdev); | |
| 7588 | + if (err) | |
| 7589 | + return err; | |
| 7591 | 7590 | } |
| 7592 | 7591 | |
| 7593 | − return err; | |
| 7592 | + netif_device_attach(netdev); | |
| 7593 | + | |
| 7594 | + return 0; | |
| 7594 | 7595 | } |
| 7595 | 7596 | |
| 7596 | 7597 | static int igc_resume(struct device *dev) |
| @@ -518,6 +518,18 @@ int cgx_set_pkind(void *cgxd, u8 lmac_id, int pkind) · 517 unmodified lines | ||
| 518 | 518 | return 0; |
| 519 | 519 | } |
| 520 | 520 | |
| 521 | +int cgx_get_pkind(void *cgxd, u8 lmac_id, int *pkind) | |
| 522 | +{ | |
| 523 | + struct cgx *cgx = cgxd; | |
| 524 | + | |
| 525 | + if (!is_lmac_valid(cgx, lmac_id)) | |
| 526 | + return -ENODEV; | |
| 527 | + | |
| 528 | + *pkind = cgx_read(cgx, lmac_id, cgx->mac_ops->rxid_map_offset); | |
| 529 | + *pkind = *pkind & 0x3F; | |
| 530 | + return 0; | |
| 531 | +} | |
| 532 | + | |
| 521 | 533 | static u8 cgx_get_lmac_type(void *cgxd, int lmac_id) |
| 522 | 534 | { |
| 523 | 535 | struct cgx *cgx = cgxd; |
| @@ -141,6 +141,7 @@ int cgx_get_cgxid(void *cgxd); · 140 unmodified lines | ||
| 141 | 141 | int cgx_get_lmac_cnt(void *cgxd); |
| 142 | 142 | void *cgx_get_pdata(int cgx_id); |
| 143 | 143 | int cgx_set_pkind(void *cgxd, u8 lmac_id, int pkind); |
| 144 | +int cgx_get_pkind(void *cgxd, u8 lmac_id, int *pkind); | |
| 144 | 145 | int cgx_lmac_evh_register(struct cgx_event_cb *cb, void *cgxd, int lmac_id); |
| 145 | 146 | int cgx_lmac_evh_unregister(void *cgxd, int lmac_id); |
| 146 | 147 | int cgx_get_tx_stats(void *cgxd, int lmac_id, int idx, u64 *tx_stat); |
| @@ -1115,6 +1115,8 @@ void npc_read_mcam_entry(struct rvu *rvu, struct npc_mcam *mcam, · 1114 unmodified lines | ||
| 1115 | 1115 | u8 *intf, u8 *ena); |
| 1116 | 1116 | int npc_config_cntr_default_entries(struct rvu *rvu, bool enable); |
| 1117 | 1117 | bool is_cgx_config_permitted(struct rvu *rvu, u16 pcifunc); |
| 1118 | +bool rvu_cgx_check_permission_and_set_pkind(struct rvu *rvu, u16 pcifunc, int pkind); | |
| 1119 | +bool rvu_cgx_is_pkind_config_permitted(struct rvu *rvu, u16 pcifunc); | |
| 1118 | 1120 | bool is_mac_feature_supported(struct rvu *rvu, int pf, int feature); |
| 1119 | 1121 | u32 rvu_cgx_get_fifolen(struct rvu *rvu); |
| 1120 | 1122 | void *rvu_first_cgx_pdata(struct rvu *rvu); |
| @@ -1355,3 +1355,82 @@ void rvu_mac_reset(struct rvu *rvu, u16 pcifunc) · 1354 unmodified lines | ||
| 1355 | 1355 | if (mac_ops->mac_reset(cgxd, lmac, !is_vf(pcifunc))) |
| 1356 | 1356 | dev_err(rvu->dev, "Failed to reset MAC\n"); |
| 1357 | 1357 | } |
| 1358 | + | |
| 1359 | +/* Do not allow CGX-mapped VFs to overwrite PKIND when special parse kinds | |
| 1360 | + * (HiGig, EDSA, etc.) are in use on the shared LMAC. VFs must not program | |
| 1361 | + * NPC_TX_DEF_PKIND on NIX_AF_LFX_TX_PARSE_CFG in that case: the PF owns | |
| 1362 | + * parse mode and no separate NPC_TX_HIGIG_PKIND is installed on the VF LF. | |
| 1363 | + * TX-parse callers skip the write when denied; rvu_lf_reset() clears each LF | |
| 1364 | + * before alloc so the next permitted owner programs NPC_TX_DEF_PKIND. | |
| 1365 | + */ | |
| 1366 | +bool rvu_cgx_is_pkind_config_permitted(struct rvu *rvu, u16 pcifunc) | |
| 1367 | +{ | |
| 1368 | + int pf, err, rxpkind; | |
| 1369 | + u8 cgx_id, lmac_id; | |
| 1370 | + void *cgxd; | |
| 1371 | + | |
| 1372 | + pf = rvu_get_pf(rvu->pdev, pcifunc); | |
| 1373 | + | |
| 1374 | + if (!(pcifunc & RVU_PFVF_FUNC_MASK)) | |
| 1375 | + return true; | |
| 1376 | + | |
| 1377 | + if (!is_pf_cgxmapped(rvu, pf)) | |
| 1378 | + return true; | |
| 1379 | + | |
| 1380 | + rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, &lmac_id); | |
| 1381 | + cgxd = rvu_cgx_pdata(cgx_id, rvu); | |
| 1382 | + err = cgx_get_pkind(cgxd, lmac_id, &rxpkind); | |
| 1383 | + if (err) | |
| 1384 | + return false; | |
| 1385 | + | |
| 1386 | + switch (rxpkind) { | |
| 1387 | + case NPC_RX_HIGIG_PKIND: | |
| 1388 | + case NPC_RX_EDSA_PKIND: | |
| 1389 | + return false; | |
| 1390 | + default: | |
| 1391 | + return true; | |
| 1392 | + } | |
| 1393 | +} | |
| 1394 | + | |
| 1395 | +/* Do not allow CGX-mapped VFs to overwrite PKIND when special parse kinds | |
| 1396 | + * (HiGig, EDSA, etc.) are in use on the shared LMAC. | |
| 1397 | + */ | |
| 1398 | +bool rvu_cgx_check_permission_and_set_pkind(struct rvu *rvu, u16 pcifunc, int pkind) | |
| 1399 | +{ | |
| 1400 | + int pf, err, rxpkind; | |
| 1401 | + u8 cgx_id, lmac_id; | |
| 1402 | + struct cgx *cgxd; | |
| 1403 | + | |
| 1404 | + pf = rvu_get_pf(rvu->pdev, pcifunc); | |
| 1405 | + | |
| 1406 | + if (!is_pf_cgxmapped(rvu, pf)) | |
| 1407 | + return false; | |
| 1408 | + | |
| 1409 | + rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, &lmac_id); | |
| 1410 | + cgxd = rvu_cgx_pdata(cgx_id, rvu); | |
| 1411 | + | |
| 1412 | + mutex_lock(&cgxd->lock); | |
| 1413 | + if (!is_vf(pcifunc)) | |
| 1414 | + goto set; | |
| 1415 | + | |
| 1416 | + err = cgx_get_pkind(cgxd, lmac_id, &rxpkind); | |
| 1417 | + if (err) | |
| 1418 | + goto err; | |
| 1419 | + | |
| 1420 | + switch (rxpkind) { | |
| 1421 | + case NPC_RX_HIGIG_PKIND: | |
| 1422 | + case NPC_RX_EDSA_PKIND: | |
| 1423 | + goto err; | |
| 1424 | + default: | |
| 1425 | + break; | |
| 1426 | + } | |
| 1427 | + | |
| 1428 | +set: | |
| 1429 | + cgx_set_pkind(rvu_cgx_pdata(cgx_id, rvu), lmac_id, pkind); | |
| 1430 | + mutex_unlock(&cgxd->lock); | |
| 1431 | + return true; | |
| 1432 | + | |
| 1433 | +err: | |
| 1434 | + mutex_unlock(&cgxd->lock); | |
| 1435 | + return false; | |
| 1436 | +} | |
| @@ -363,8 +363,8 @@ static int nix_interface_init(struct rvu *rvu, u16 pcifunc, int type, int nixlf, · 362 unmodified lines | ||
| 363 | 363 | pfvf->tx_chan_cnt = 1; |
| 364 | 364 | rsp->tx_link = cgx_id * hw->lmac_per_cgx + lmac_id; |
| 365 | 365 | |
| 366 | − cgx_set_pkind(rvu_cgx_pdata(cgx_id, rvu), lmac_id, pkind); | |
| 367 | − rvu_npc_set_pkind(rvu, pkind, pfvf); | |
| 366 | + if (rvu_cgx_check_permission_and_set_pkind(rvu, pcifunc, pkind)) | |
| 367 | + rvu_npc_set_pkind(rvu, pkind, pfvf); | |
| 368 | 368 | break; |
| 369 | 369 | case NIX_INTF_TYPE_LBK: |
| 370 | 370 | vf = (pcifunc & RVU_PFVF_FUNC_MASK) - 1; |
| @@ -1505,13 +1505,15 @@ int rvu_mbox_handler_nix_lf_alloc(struct rvu *rvu, · 1134 unmodified lines | ||
| 1505 | 1505 | struct nix_lf_alloc_req *req, |
| 1506 | 1506 | struct nix_lf_alloc_rsp *rsp) |
| 1507 | 1507 | { |
| 1508 | − int nixlf, qints, hwctx_size, intf, rc = 0; | |
| 1508 | + int nixlf, qints, hwctx_size, intf, rc = 0, pf; | |
| 1509 | 1509 | u16 bcast, mcast, promisc, ucast; |
| 1510 | 1510 | struct rvu_hwinfo *hw = rvu->hw; |
| 1511 | 1511 | u16 pcifunc = req->hdr.pcifunc; |
| 1512 | + u8 cgx_id = 0, lmac_id = 0; | |
| 1512 | 1513 | bool rules_created = false; |
| 1513 | 1514 | struct rvu_block *block; |
| 1514 | 1515 | struct rvu_pfvf *pfvf; |
| 1516 | + struct cgx *cgxd; | |
| 1515 | 1517 | u64 cfg, ctx_cfg; |
| 1516 | 1518 | int blkaddr; |
| 1517 | 1519 | |
| @@ -1685,8 +1687,20 @@ int rvu_mbox_handler_nix_lf_alloc(struct rvu *rvu, · 167 unmodified lines | ||
| 1685 | 1687 | rvu_write64(rvu, blkaddr, NIX_AF_LFX_RX_CFG(nixlf), req->rx_cfg); |
| 1686 | 1688 | |
| 1687 | 1689 | /* Configure pkind for TX parse config */ |
| 1690 | + | |
| 1691 | + pf = rvu_get_pf(rvu->pdev, pcifunc); | |
| 1688 | 1692 | cfg = NPC_TX_DEF_PKIND; |
| 1689 | − rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), cfg); | |
| 1693 | + | |
| 1694 | + if (is_pf_cgxmapped(rvu, pf) && is_vf(pcifunc)) { | |
| 1695 | + rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, &lmac_id); | |
| 1696 | + cgxd = rvu_cgx_pdata(cgx_id, rvu); | |
| 1697 | + mutex_lock(&cgxd->lock); | |
| 1698 | + if (rvu_cgx_is_pkind_config_permitted(rvu, pcifunc)) | |
| 1699 | + rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), cfg); | |
| 1700 | + mutex_unlock(&cgxd->lock); | |
| 1701 | + } else { | |
| 1702 | + rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), cfg); | |
| 1703 | + } | |
| 1690 | 1704 | |
| 1691 | 1705 | if (is_rep_dev(rvu, pcifunc)) { |
| 1692 | 1706 | pfvf->tx_chan_base = RVU_SWITCH_LBK_CHAN; |
| @@ -19,6 +19,7 @@ · 18 unmodified lines | ||
| 19 | 19 | #include "cn20k/npc.h" |
| 20 | 20 | #include "rvu_npc.h" |
| 21 | 21 | #include "cn20k/reg.h" |
| 22 | +#include "lmac_common.h" | |
| 22 | 23 | |
| 23 | 24 | #define RSVD_MCAM_ENTRIES_PER_PF 3 /* Broadcast, Promisc and AllMulticast */ |
| 24 | 25 | #define RSVD_MCAM_ENTRIES_PER_NIXLF 1 /* Ucast for LFs */ |
| @@ -4200,10 +4201,11 @@ int rvu_npc_set_parse_mode(struct rvu *rvu, u16 pcifunc, u64 mode, u8 dir, · 4175 unmodified lines | ||
| 4200 | 4201 | |
| 4201 | 4202 | { |
| 4202 | 4203 | struct rvu_pfvf *pfvf = rvu_get_pfvf(rvu, pcifunc); |
| 4203 | − int blkaddr, nixlf, rc, intf_mode; | |
| 4204 | 4204 | int pf = rvu_get_pf(rvu->pdev, pcifunc); |
| 4205 | + int blkaddr, nixlf, rc, intf_mode; | |
| 4206 | + u8 cgx_id = 0, lmac_id = 0; | |
| 4205 | 4207 | u64 rxpkind, txpkind; |
| 4206 | − u8 cgx_id, lmac_id; | |
| 4208 | + struct cgx *cgxd; | |
| 4207 | 4209 | |
| 4208 | 4210 | /* use default pkind to disable edsa/higig */ |
| 4209 | 4211 | rxpkind = rvu_npc_get_pkind(rvu, pf); |
| @@ -4227,12 +4229,8 @@ int rvu_npc_set_parse_mode(struct rvu *rvu, u16 pcifunc, u64 mode, u8 dir, · 17 unmodified lines | ||
| 4227 | 4229 | /* rx pkind set req valid only for cgx mapped PFs */ |
| 4228 | 4230 | if (!is_cgx_config_permitted(rvu, pcifunc)) |
| 4229 | 4231 | return 0; |
| 4230 | − rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, &lmac_id); | |
| 4231 | − | |
| 4232 | − rc = cgx_set_pkind(rvu_cgx_pdata(cgx_id, rvu), lmac_id, | |
| 4233 | − rxpkind); | |
| 4234 | − if (rc) | |
| 4235 | − return rc; | |
| 4232 | + if (!rvu_cgx_check_permission_and_set_pkind(rvu, pcifunc, rxpkind)) | |
| 4233 | + return -EINVAL; | |
| 4236 | 4234 | } |
| 4237 | 4235 | |
| 4238 | 4236 | if (dir & PKIND_TX) { |
| @@ -4241,8 +4239,19 @@ int rvu_npc_set_parse_mode(struct rvu *rvu, u16 pcifunc, u64 mode, u8 dir, · 2 unmodified lines | ||
| 4241 | 4239 | if (rc) |
| 4242 | 4240 | return rc; |
| 4243 | 4241 | |
| 4244 | − rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), | |
| 4245 | − txpkind); | |
| 4242 | + if (is_pf_cgxmapped(rvu, pf) && is_vf(pcifunc)) { | |
| 4243 | + rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, | |
| 4244 | + &lmac_id); | |
| 4245 | + cgxd = rvu_cgx_pdata(cgx_id, rvu); | |
| 4246 | + mutex_lock(&cgxd->lock); | |
| 4247 | + if (rvu_cgx_is_pkind_config_permitted(rvu, pcifunc)) | |
| 4248 | + rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), | |
| 4249 | + txpkind); | |
| 4250 | + mutex_unlock(&cgxd->lock); | |
| 4251 | + } else { | |
| 4252 | + rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), | |
| 4253 | + txpkind); | |
| 4254 | + } | |
| 4246 | 4255 | } |
| 4247 | 4256 | |
| 4248 | 4257 | pfvf->intf_mode = intf_mode; |
| @@ -889,8 +889,8 @@ static void otx2_handle_link_event(struct otx2_nic *pf) · 888 unmodified lines | ||
| 889 | 889 | netif_carrier_on(netdev); |
| 890 | 890 | netif_tx_start_all_queues(netdev); |
| 891 | 891 | } else { |
| 892 | − netif_tx_stop_all_queues(netdev); | |
| 893 | 892 | netif_carrier_off(netdev); |
| 893 | + netif_tx_stop_all_queues(netdev); | |
| 894 | 894 | } |
| 895 | 895 | } |
| 896 | 896 | |
| @@ -54,10 +54,12 @@ static void otx2_get_egress_burst_cfg(struct otx2_nic *nic, u32 burst, · 53 unmodified lines | ||
| 54 | 54 | if (burst) { |
| 55 | 55 | *burst_exp = ilog2(burst) ? ilog2(burst) - 1 : 0; |
| 56 | 56 | tmp = burst - rounddown_pow_of_two(burst); |
| 57 | − if (burst < max_mantissa) | |
| 57 | + if (burst <= max_mantissa) { | |
| 58 | 58 | *burst_mantissa = tmp * 2; |
| 59 | − else | |
| 59 | + } else { | |
| 60 | + WARN_ON(*burst_exp < 7); | |
| 60 | 61 | *burst_mantissa = tmp / (1ULL << (*burst_exp - 7)); |
| 62 | + } | |
| 61 | 63 | } else { |
| 62 | 64 | *burst_exp = MAX_BURST_EXPONENT; |
| 63 | 65 | *burst_mantissa = max_mantissa; |
| @@ -684,6 +684,9 @@ static int prestera_fw_hdr_parse(struct prestera_fw *fw) · 683 unmodified lines | ||
| 684 | 684 | struct prestera_fw_header *hdr; |
| 685 | 685 | u32 magic; |
| 686 | 686 | |
| 687 | + if (fw->bin->size < sizeof(*hdr)) | |
| 688 | + return -EINVAL; | |
| 689 | + | |
| 687 | 690 | hdr = (struct prestera_fw_header *)fw->bin->data; |
| 688 | 691 | |
| 689 | 692 | magic = be32_to_cpu(hdr->magic_number); |
| @@ -3467,7 +3467,7 @@ static void mtk_poll_controller(struct net_device *dev) · 3466 unmodified lines | ||
| 3467 | 3467 | |
| 3468 | 3468 | mtk_tx_irq_disable(eth, MTK_TX_DONE_INT); |
| 3469 | 3469 | mtk_rx_irq_disable(eth, eth->soc->rx.irq_done_mask); |
| 3470 | − mtk_handle_irq_rx(eth->irq[MTK_FE_IRQ_RX], dev); | |
| 3470 | + mtk_handle_irq_rx(eth->irq[MTK_FE_IRQ_RX], eth); | |
| 3471 | 3471 | mtk_tx_irq_enable(eth, MTK_TX_DONE_INT); |
| 3472 | 3472 | mtk_rx_irq_enable(eth, eth->soc->rx.irq_done_mask); |
| 3473 | 3473 | } |
| @@ -1025,13 +1025,11 @@ struct mlx5_fw_tracer *mlx5_fw_tracer_create(struct mlx5_core_dev *dev) · 1024 unmodified lines | ||
| 1025 | 1025 | |
| 1026 | 1026 | tracer = kvzalloc_obj(*tracer); |
| 1027 | 1027 | if (!tracer) |
| 1028 | − return ERR_PTR(-ENOMEM); | |
| 1028 | + return NULL; | |
| 1029 | 1029 | |
| 1030 | 1030 | tracer->work_queue = create_singlethread_workqueue("mlx5_fw_tracer"); |
| 1031 | − if (!tracer->work_queue) { | |
| 1032 | − err = -ENOMEM; | |
| 1031 | + if (!tracer->work_queue) | |
| 1033 | 1032 | goto free_tracer; |
| 1034 | − } | |
| 1035 | 1033 | |
| 1036 | 1034 | tracer->dev = dev; |
| 1037 | 1035 | |
| @@ -1073,7 +1071,7 @@ destroy_workqueue: · 35 unmodified lines | ||
| 1073 | 1071 | destroy_workqueue(tracer->work_queue); |
| 1074 | 1072 | free_tracer: |
| 1075 | 1073 | kvfree(tracer); |
| 1076 | − return ERR_PTR(err); | |
| 1074 | + return NULL; | |
| 1077 | 1075 | } |
| 1078 | 1076 | |
| 1079 | 1077 | static int fw_tracer_event(struct notifier_block *nb, unsigned long action, void *data); |
| @@ -1084,7 +1082,7 @@ int mlx5_fw_tracer_init(struct mlx5_fw_tracer *tracer) · 4 unmodified lines | ||
| 1084 | 1082 | struct mlx5_core_dev *dev; |
| 1085 | 1083 | int err; |
| 1086 | 1084 | |
| 1087 | − if (IS_ERR_OR_NULL(tracer)) | |
| 1085 | + if (!tracer) | |
| 1088 | 1086 | return 0; |
| 1089 | 1087 | |
| 1090 | 1088 | if (!tracer->str_db.loaded) |
| @@ -1134,7 +1132,7 @@ err_cancel_work: · 43 unmodified lines | ||
| 1134 | 1132 | /* Stop tracer + Cleanup HW resources */ |
| 1135 | 1133 | void mlx5_fw_tracer_cleanup(struct mlx5_fw_tracer *tracer) |
| 1136 | 1134 | { |
| 1137 | − if (IS_ERR_OR_NULL(tracer)) | |
| 1135 | + if (!tracer) | |
| 1138 | 1136 | return; |
| 1139 | 1137 | |
| 1140 | 1138 | mutex_lock(&tracer->state_lock); |
| @@ -1163,7 +1161,7 @@ unlock: · 22 unmodified lines | ||
| 1163 | 1161 | /* Free software resources (Buffers, etc ..) */ |
| 1164 | 1162 | void mlx5_fw_tracer_destroy(struct mlx5_fw_tracer *tracer) |
| 1165 | 1163 | { |
| 1166 | − if (IS_ERR_OR_NULL(tracer)) | |
| 1164 | + if (!tracer) | |
| 1167 | 1165 | return; |
| 1168 | 1166 | |
| 1169 | 1167 | mlx5_core_dbg(tracer->dev, "FWTracer: Destroy\n"); |
| @@ -1215,7 +1213,7 @@ int mlx5_fw_tracer_reload(struct mlx5_fw_tracer *tracer) · 45 unmodified lines | ||
| 1215 | 1213 | struct mlx5_core_dev *dev; |
| 1216 | 1214 | int err; |
| 1217 | 1215 | |
| 1218 | − if (IS_ERR_OR_NULL(tracer)) | |
| 1216 | + if (!tracer) | |
| 1219 | 1217 | return 0; |
| 1220 | 1218 | |
| 1221 | 1219 | dev = tracer->dev; |
| @@ -483,7 +483,7 @@ typedef int (*mlx5e_fp_xmit_xdp_frame_check)(struct mlx5e_xdpsq *); · 482 unmodified lines | ||
| 483 | 483 | typedef bool (*mlx5e_fp_xmit_xdp_frame)(struct mlx5e_xdpsq *, |
| 484 | 484 | struct mlx5e_xmit_data *, |
| 485 | 485 | int, |
| 486 | − struct xsk_tx_metadata *); | |
| 486 | + struct xsk_tx_metadata **); | |
| 487 | 487 | |
| 488 | 488 | struct mlx5e_xdpsq { |
| 489 | 489 | /* data path */ |
| @@ -984,7 +984,6 @@ struct mlx5e_priv { · 494 unmodified lines | ||
| 984 | 984 | struct mlx5e_mqprio_rl *mqprio_rl; |
| 985 | 985 | struct dentry *dfs_root; |
| 986 | 986 | struct mlx5_devcom_comp_dev *devcom; |
| 987 | − struct ethtool_fec_hist_range *fec_ranges; | |
| 988 | 987 | }; |
| 989 | 988 | |
| 990 | 989 | static inline u16 mlx5e_stats_nch_read(const struct mlx5e_priv *priv) |
| @@ -30,6 +30,7 @@ enum { · 29 unmodified lines | ||
| 30 | 30 | MLX5E_TC_FLOW_FLAG_FAILED = MLX5E_TC_FLOW_BASE + 9, |
| 31 | 31 | MLX5E_TC_FLOW_FLAG_SAMPLE = MLX5E_TC_FLOW_BASE + 10, |
| 32 | 32 | MLX5E_TC_FLOW_FLAG_USE_ACT_STATS = MLX5E_TC_FLOW_BASE + 11, |
| 33 | + MLX5E_TC_FLOW_FLAG_PEER = MLX5E_TC_FLOW_BASE + 12, | |
| 33 | 34 | }; |
| 34 | 35 | |
| 35 | 36 | struct mlx5e_tc_flow_parse_attr { |
| @@ -452,11 +452,11 @@ INDIRECT_CALLABLE_SCOPE int mlx5e_xmit_xdp_frame_check_mpwqe(struct mlx5e_xdpsq · 451 unmodified lines | ||
| 452 | 452 | |
| 453 | 453 | INDIRECT_CALLABLE_SCOPE bool |
| 454 | 454 | mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptxd, |
| 455 | − int check_result, struct xsk_tx_metadata *meta); | |
| 455 | + int check_result, struct xsk_tx_metadata **meta); | |
| 456 | 456 | |
| 457 | 457 | INDIRECT_CALLABLE_SCOPE bool |
| 458 | 458 | mlx5e_xmit_xdp_frame_mpwqe(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptxd, |
| 459 | − int check_result, struct xsk_tx_metadata *meta) | |
| 459 | + int check_result, struct xsk_tx_metadata **meta) | |
| 460 | 460 | { |
| 461 | 461 | struct mlx5e_tx_mpwqe *session = &sq->mpwqe; |
| 462 | 462 | struct mlx5e_xdpsq_stats *stats = sq->stats; |
| @@ -504,7 +504,10 @@ mlx5e_xmit_xdp_frame_mpwqe(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptx · 41 unmodified lines | ||
| 504 | 504 | * and it's safe to complete it at any time. |
| 505 | 505 | */ |
| 506 | 506 | mlx5e_xdp_mpwqe_session_start(sq); |
| 507 | − xsk_tx_metadata_request(meta, &mlx5e_xsk_tx_metadata_ops, &session->wqe->eth); | |
| 507 | + if (meta) | |
| 508 | + xsk_tx_metadata_request(sq->xsk_pool, meta, | |
| 509 | + &mlx5e_xsk_tx_metadata_ops, | |
| 510 | + &session->wqe->eth); | |
| 508 | 511 | } |
| 509 | 512 | |
| 510 | 513 | mlx5e_xdp_mpwqe_add_dseg(sq, p, stats); |
| @@ -535,7 +538,7 @@ INDIRECT_CALLABLE_SCOPE int mlx5e_xmit_xdp_frame_check(struct mlx5e_xdpsq *sq) · 24 unmodified lines | ||
| 535 | 538 | |
| 536 | 539 | INDIRECT_CALLABLE_SCOPE bool |
| 537 | 540 | mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptxd, |
| 538 | − int check_result, struct xsk_tx_metadata *meta) | |
| 541 | + int check_result, struct xsk_tx_metadata **meta) | |
| 539 | 542 | { |
| 540 | 543 | struct mlx5e_xmit_data_frags *xdptxdf = |
| 541 | 544 | container_of(xdptxd, struct mlx5e_xmit_data_frags, xd); |
| @@ -649,7 +652,9 @@ mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptxd, · 107 unmodified lines | ||
| 649 | 652 | |
| 650 | 653 | sq->pc += num_wqebbs; |
| 651 | 654 | |
| 652 | − xsk_tx_metadata_request(meta, &mlx5e_xsk_tx_metadata_ops, eseg); | |
| 655 | + if (meta) | |
| 656 | + xsk_tx_metadata_request(sq->xsk_pool, meta, | |
| 657 | + &mlx5e_xsk_tx_metadata_ops, eseg); | |
| 653 | 658 | |
| 654 | 659 | sq->doorbell_cseg = cseg; |
| 655 | 660 | |
| @@ -114,11 +114,11 @@ extern const struct xsk_tx_metadata_ops mlx5e_xsk_tx_metadata_ops; · 113 unmodified lines | ||
| 114 | 114 | INDIRECT_CALLABLE_DECLARE(bool mlx5e_xmit_xdp_frame_mpwqe(struct mlx5e_xdpsq *sq, |
| 115 | 115 | struct mlx5e_xmit_data *xdptxd, |
| 116 | 116 | int check_result, |
| 117 | − struct xsk_tx_metadata *meta)); | |
| 117 | + struct xsk_tx_metadata **meta)); | |
| 118 | 118 | INDIRECT_CALLABLE_DECLARE(bool mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq, |
| 119 | 119 | struct mlx5e_xmit_data *xdptxd, |
| 120 | 120 | int check_result, |
| 121 | − struct xsk_tx_metadata *meta)); | |
| 121 | + struct xsk_tx_metadata **meta)); | |
| 122 | 122 | INDIRECT_CALLABLE_DECLARE(int mlx5e_xmit_xdp_frame_check_mpwqe(struct mlx5e_xdpsq *sq)); |
| 123 | 123 | INDIRECT_CALLABLE_DECLARE(int mlx5e_xmit_xdp_frame_check(struct mlx5e_xdpsq *sq)); |
| 124 | 124 | |
| @@ -105,7 +105,7 @@ bool mlx5e_xsk_tx(struct mlx5e_xdpsq *sq, unsigned int budget) · 104 unmodified lines | ||
| 105 | 105 | |
| 106 | 106 | ret = INDIRECT_CALL_2(sq->xmit_xdp_frame, mlx5e_xmit_xdp_frame_mpwqe, |
| 107 | 107 | mlx5e_xmit_xdp_frame, sq, &xdptxd, |
| 108 | − check_result, meta); | |
| 108 | + check_result, &meta); | |
| 109 | 109 | if (unlikely(!ret)) { |
| 110 | 110 | if (sq->mpwqe.wqe) |
| 111 | 111 | mlx5e_xdp_mpwqe_complete(sq); |
| @@ -1939,8 +1939,10 @@ err_free_txqsq: · 1938 unmodified lines | ||
| 1939 | 1939 | void mlx5e_activate_txqsq(struct mlx5e_txqsq *sq) |
| 1940 | 1940 | { |
| 1941 | 1941 | sq->txq = netdev_get_tx_queue(sq->netdev, sq->txq_ix); |
| 1942 | + /* Reset BQL only when the SQ has no bytes in flight. */ | |
| 1943 | + if (sq->cc == sq->pc) | |
| 1944 | + netdev_tx_reset_queue(sq->txq); | |
| 1942 | 1945 | set_bit(MLX5E_SQ_STATE_ENABLED, &sq->state); |
| 1943 | − netdev_tx_reset_queue(sq->txq); | |
| 1944 | 1946 | netif_tx_start_queue(sq->txq); |
| 1945 | 1947 | netif_queue_set_napi(sq->netdev, sq->txq_ix, NETDEV_QUEUE_TYPE_TX, sq->cq.napi); |
| 1946 | 1948 | } |
| @@ -6415,14 +6417,8 @@ int mlx5e_priv_init(struct mlx5e_priv *priv, · 4468 unmodified lines | ||
| 6415 | 6417 | if (!priv->channel_stats) |
| 6416 | 6418 | goto err_free_tx_rates; |
| 6417 | 6419 | |
| 6418 | − priv->fec_ranges = kzalloc_objs(*priv->fec_ranges, ETHTOOL_FEC_HIST_MAX); | |
| 6419 | − if (!priv->fec_ranges) | |
| 6420 | − goto err_free_channel_stats; | |
| 6421 | − | |
| 6422 | 6420 | return 0; |
| 6423 | 6421 | |
| 6424 | −err_free_channel_stats: | |
| 6425 | − kfree(priv->channel_stats); | |
| 6426 | 6422 | err_free_tx_rates: |
| 6427 | 6423 | kfree(priv->tx_rates); |
| 6428 | 6424 | err_free_txq2sq_stats: |
| @@ -6447,7 +6443,6 @@ void mlx5e_priv_cleanup(struct mlx5e_priv *priv) · 18 unmodified lines | ||
| 6447 | 6443 | if (!priv->mdev) |
| 6448 | 6444 | return; |
| 6449 | 6445 | |
| 6450 | − kfree(priv->fec_ranges); | |
| 6451 | 6446 | for (i = 0; i < priv->stats_nch; i++) |
| 6452 | 6447 | kvfree(priv->channel_stats[i]); |
| 6453 | 6448 | kfree(priv->channel_stats); |
| @@ -1550,7 +1550,7 @@ static bool fec_rs_validate_hist_type(int mode, int hist_type) · 1549 unmodified lines | ||
| 1550 | 1550 | |
| 1551 | 1551 | static u8 |
| 1552 | 1552 | fec_rs_histogram_fill_ranges(struct mlx5e_priv *priv, int mode, |
| 1553 | − const struct ethtool_fec_hist_range **ranges) | |
| 1553 | + struct ethtool_fec_hist_range *ranges) | |
| 1554 | 1554 | { |
| 1555 | 1555 | struct mlx5_core_dev *mdev = priv->mdev; |
| 1556 | 1556 | u32 out[MLX5_ST_SZ_DW(pphcr_reg)] = {0}; |
| @@ -1558,8 +1558,6 @@ fec_rs_histogram_fill_ranges(struct mlx5e_priv *priv, int mode, · 1 unmodified line | ||
| 1558 | 1558 | int sz = MLX5_ST_SZ_BYTES(pphcr_reg); |
| 1559 | 1559 | u8 hist_type, num_of_bins; |
| 1560 | 1560 | |
| 1561 | − memset(priv->fec_ranges, 0, | |
| 1562 | − ETHTOOL_FEC_HIST_MAX * sizeof(*priv->fec_ranges)); | |
| 1563 | 1561 | MLX5_SET(pphcr_reg, in, local_port, 1); |
| 1564 | 1562 | if (mlx5_core_access_reg(mdev, in, sz, out, sz, MLX5_REG_PPHCR, 0, 0)) |
| 1565 | 1563 | return 0; |
| @@ -1575,12 +1573,11 @@ fec_rs_histogram_fill_ranges(struct mlx5e_priv *priv, int mode, · 9 unmodified lines | ||
| 1575 | 1573 | for (int i = 0; i < num_of_bins; i++) { |
| 1576 | 1574 | void *bin_range = MLX5_ADDR_OF(pphcr_reg, out, bin_range[i]); |
| 1577 | 1575 | |
| 1578 | − priv->fec_ranges[i].high = MLX5_GET(bin_range_layout, bin_range, | |
| 1579 | − high_val); | |
| 1580 | − priv->fec_ranges[i].low = MLX5_GET(bin_range_layout, bin_range, | |
| 1581 | − low_val); | |
| 1576 | + ranges[i].high = MLX5_GET(bin_range_layout, bin_range, | |
| 1577 | + high_val); | |
| 1578 | + ranges[i].low = MLX5_GET(bin_range_layout, bin_range, | |
| 1579 | + low_val); | |
| 1582 | 1580 | } |
| 1583 | − *ranges = priv->fec_ranges; | |
| 1584 | 1581 | |
| 1585 | 1582 | return num_of_bins; |
| 1586 | 1583 | } |
| @@ -1622,10 +1619,12 @@ static void fec_set_histograms_stats(struct mlx5e_priv *priv, int mode, · 35 unmodified lines | ||
| 1622 | 1619 | case MLX5E_FEC_LLRS_272_257_1: |
| 1623 | 1620 | case MLX5E_FEC_RS_544_514_INTERLEAVED_QUAD: |
| 1624 | 1621 | num_of_bins = |
| 1625 | − fec_rs_histogram_fill_ranges(priv, mode, &hist->ranges); | |
| 1626 | − if (num_of_bins) | |
| 1622 | + fec_rs_histogram_fill_ranges(priv, mode, hist->ranges_buf); | |
| 1623 | + if (num_of_bins) { | |
| 1624 | + hist->ranges = hist->ranges_buf; | |
| 1627 | 1625 | return fec_rs_histogram_fill_stats(priv, num_of_bins, |
| 1628 | 1626 | hist); |
| 1627 | + } | |
| 1629 | 1628 | break; |
| 1630 | 1629 | default: |
| 1631 | 1630 | return; |
| @@ -2161,7 +2161,8 @@ static void mlx5e_tc_del_flow(struct mlx5e_priv *priv, · 2160 unmodified lines | ||
| 2161 | 2161 | if (mlx5e_is_eswitch_flow(flow)) { |
| 2162 | 2162 | struct mlx5_devcom_comp_dev *devcom = flow->priv->mdev->priv.eswitch->devcom; |
| 2163 | 2163 | |
| 2164 | − if (!mlx5_devcom_for_each_peer_begin(devcom)) { | |
| 2164 | + if (flow_flag_test(flow, PEER) || | |
| 2165 | + !mlx5_devcom_for_each_peer_begin(devcom)) { | |
| 2165 | 2166 | mlx5e_tc_del_fdb_flow(priv, flow); |
| 2166 | 2167 | return; |
| 2167 | 2168 | } |
| @@ -4628,6 +4629,7 @@ static int mlx5e_tc_add_fdb_peer_flow(struct flow_cls_offload *f, · 2460 unmodified lines | ||
| 4628 | 4629 | else |
| 4629 | 4630 | in_mdev = priv->mdev; |
| 4630 | 4631 | |
| 4632 | + flow_flags |= BIT(MLX5E_TC_FLOW_FLAG_PEER); | |
| 4631 | 4633 | parse_attr = flow->attr->parse_attr; |
| 4632 | 4634 | peer_flow = __mlx5e_add_fdb_flow(peer_priv, f, flow_flags, |
| 4633 | 4635 | parse_attr->filter_dev, |
| @@ -3986,7 +3986,7 @@ static void esw_offloads_steering_cleanup(struct mlx5_eswitch *esw) · 3985 unmodified lines | ||
| 3986 | 3986 | mutex_destroy(&esw->fdb_table.offloads.vports.lock); |
| 3987 | 3987 | } |
| 3988 | 3988 | |
| 3989 | −static void esw_vfs_changed_event_handler(struct mlx5_eswitch *esw) | |
| 3989 | +static void esw_changed_event_handler(struct mlx5_eswitch *esw) | |
| 3990 | 3990 | { |
| 3991 | 3991 | struct mlx5_esw_pf_info host_pf_info; |
| 3992 | 3992 | u16 new_num_vfs; |
| @@ -3999,6 +3999,11 @@ static void esw_vfs_changed_event_handler(struct mlx5_eswitch *esw) · 6 unmodified lines | ||
| 3999 | 3999 | host_pf_info = mlx5_esw_get_host_pf_info(esw->dev, out); |
| 4000 | 4000 | new_num_vfs = host_pf_info.num_of_vfs; |
| 4001 | 4001 | |
| 4002 | + if (host_pf_info.pf_disabled) { | |
| 4003 | + mlx5_sf_table_esw_changed_event_handler(esw->dev); | |
| 4004 | + mlx5_sf_hw_table_esw_changed_event_handler(esw->dev); | |
| 4005 | + } | |
| 4006 | + | |
| 4002 | 4007 | if (new_num_vfs == esw->esw_funcs.num_vfs || host_pf_info.pf_disabled) |
| 4003 | 4008 | goto free; |
| 4004 | 4009 | |
| @@ -4091,8 +4096,7 @@ int mlx5_esw_funcs_changed_handler(struct notifier_block *nb, · 86 unmodified lines | ||
| 4091 | 4096 | esw_funcs = mlx5_nb_cof(nb, struct mlx5_esw_functions, nb); |
| 4092 | 4097 | esw = container_of(esw_funcs, struct mlx5_eswitch, esw_funcs); |
| 4093 | 4098 | |
| 4094 | − ret = mlx5_esw_add_work(esw, esw_vfs_changed_event_handler, | |
| 4095 | − GFP_ATOMIC); | |
| 4099 | + ret = mlx5_esw_add_work(esw, esw_changed_event_handler, GFP_ATOMIC); | |
| 4096 | 4100 | if (ret) |
| 4097 | 4101 | return NOTIFY_DONE; |
| 4098 | 4102 | |
| @@ -561,3 +561,32 @@ bool mlx5_sf_table_empty(const struct mlx5_core_dev *dev) · 560 unmodified lines | ||
| 561 | 561 | |
| 562 | 562 | return xa_empty(&table->function_ids); |
| 563 | 563 | } |
| 564 | + | |
| 565 | +void mlx5_sf_table_esw_changed_event_handler(struct mlx5_core_dev *dev) | |
| 566 | +{ | |
| 567 | + struct mlx5_sf_table *table = dev->priv.sf_table; | |
| 568 | + unsigned long index; | |
| 569 | + struct mlx5_sf *sf; | |
| 570 | + | |
| 571 | + trace_mlx5_sf_host_pf_disabled(dev); | |
| 572 | + | |
| 573 | + if (!table) | |
| 574 | + return; | |
| 575 | + | |
| 576 | + mutex_lock(&table->sf_state_lock); | |
| 577 | + xa_for_each(&table->function_ids, index, sf) { | |
| 578 | + if (!sf->controller) | |
| 579 | + continue; | |
| 580 | + | |
| 581 | + if (sf->hw_state == MLX5_VHCA_STATE_IN_USE) | |
| 582 | + sf->hw_state = MLX5_VHCA_STATE_ACTIVE; | |
| 583 | + else if (sf->hw_state == MLX5_VHCA_STATE_TEARDOWN_REQUEST) | |
| 584 | + sf->hw_state = MLX5_VHCA_STATE_ALLOCATED; | |
| 585 | + else | |
| 586 | + continue; | |
| 587 | + trace_mlx5_sf_update_state(table->dev, sf->port_index, | |
| 588 | + sf->controller, sf->hw_fn_id, | |
| 589 | + sf->hw_state); | |
| 590 | + } | |
| 591 | + mutex_unlock(&table->sf_state_lock); | |
| 592 | +} | |
| @@ -11,6 +11,14 @@ · 10 unmodified lines | ||
| 11 | 11 | #include <linux/mlx5/driver.h> |
| 12 | 12 | #include "sf/vhca_event.h" |
| 13 | 13 | |
| 14 | +TRACE_EVENT(mlx5_sf_host_pf_disabled, | |
| 15 | + TP_PROTO(const struct mlx5_core_dev *dev), | |
| 16 | + TP_ARGS(dev), | |
| 17 | + TP_STRUCT__entry(__string(devname, dev_name(dev->device))), | |
| 18 | + TP_fast_assign(__assign_str(devname);), | |
| 19 | + TP_printk("(%s)\n", __get_str(devname)) | |
| 20 | +); | |
| 21 | + | |
| 14 | 22 | TRACE_EVENT(mlx5_sf_add, |
| 15 | 23 | TP_PROTO(const struct mlx5_core_dev *dev, |
| 16 | 24 | unsigned int port_index, |
| @@ -459,3 +459,25 @@ bool mlx5_sf_hw_table_supported(const struct mlx5_core_dev *dev) · 458 unmodified lines | ||
| 459 | 459 | { |
| 460 | 460 | return !!dev->priv.sf_hw_table; |
| 461 | 461 | } |
| 462 | + | |
| 463 | +void mlx5_sf_hw_table_esw_changed_event_handler(struct mlx5_core_dev *dev) | |
| 464 | +{ | |
| 465 | + struct mlx5_sf_hw_table *table; | |
| 466 | + struct mlx5_sf_hwc_table *hwc; | |
| 467 | + int i; | |
| 468 | + | |
| 469 | + table = dev->priv.sf_hw_table; | |
| 470 | + if (!table) | |
| 471 | + return; | |
| 472 | + | |
| 473 | + mutex_lock(&table->table_lock); | |
| 474 | + hwc = &table->hwc[MLX5_SF_HWC_EXT_HOST]; | |
| 475 | + for (i = 0; i < hwc->max_fn; i++) { | |
| 476 | + struct mlx5_sf_hw *sf_hw; | |
| 477 | + | |
| 478 | + sf_hw = &hwc->sfs[i]; | |
| 479 | + if (sf_hw->allocated && sf_hw->pending_delete) | |
| 480 | + mlx5_sf_hw_table_hwc_sf_free(dev, hwc, i); | |
| 481 | + } | |
| 482 | + mutex_unlock(&table->table_lock); | |
| 483 | +} | |
| @@ -15,12 +15,14 @@ void mlx5_sf_hw_table_cleanup(struct mlx5_core_dev *dev); · 14 unmodified lines | ||
| 15 | 15 | int mlx5_sf_hw_notifier_init(struct mlx5_core_dev *dev); |
| 16 | 16 | void mlx5_sf_hw_notifier_cleanup(struct mlx5_core_dev *dev); |
| 17 | 17 | void mlx5_sf_hw_table_destroy(struct mlx5_core_dev *dev); |
| 18 | +void mlx5_sf_hw_table_esw_changed_event_handler(struct mlx5_core_dev *dev); | |
| 18 | 19 | |
| 19 | 20 | int mlx5_sf_notifiers_init(struct mlx5_core_dev *dev); |
| 20 | 21 | int mlx5_sf_table_init(struct mlx5_core_dev *dev); |
| 21 | 22 | void mlx5_sf_notifiers_cleanup(struct mlx5_core_dev *dev); |
| 22 | 23 | void mlx5_sf_table_cleanup(struct mlx5_core_dev *dev); |
| 23 | 24 | bool mlx5_sf_table_empty(const struct mlx5_core_dev *dev); |
| 25 | +void mlx5_sf_table_esw_changed_event_handler(struct mlx5_core_dev *dev); | |
| 24 | 26 | |
| 25 | 27 | int mlx5_devlink_sf_port_new(struct devlink *devlink, |
| 26 | 28 | const struct devlink_port_new_attrs *add_attr, |
| @@ -60,6 +62,11 @@ static inline void mlx5_sf_hw_table_destroy(struct mlx5_core_dev *dev) · 33 unmodified lines | ||
| 60 | 62 | { |
| 61 | 63 | } |
| 62 | 64 | |
| 65 | +static inline void | |
| 66 | +mlx5_sf_hw_table_esw_changed_event_handler(struct mlx5_core_dev *dev) | |
| 67 | +{ | |
| 68 | +} | |
| 69 | + | |
| 63 | 70 | static inline int mlx5_sf_notifiers_init(struct mlx5_core_dev *dev) |
| 64 | 71 | { |
| 65 | 72 | return 0; |
| @@ -83,6 +90,11 @@ static inline bool mlx5_sf_table_empty(const struct mlx5_core_dev *dev) · 17 unmodified lines | ||
| 83 | 90 | return true; |
| 84 | 91 | } |
| 85 | 92 | |
| 93 | +static inline void | |
| 94 | +mlx5_sf_table_esw_changed_event_handler(struct mlx5_core_dev *dev) | |
| 95 | +{ | |
| 96 | +} | |
| 97 | + | |
| 86 | 98 | #endif |
| 87 | 99 | |
| 88 | 100 | #endif |
| @@ -2829,7 +2829,7 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc, · 2828 unmodified lines | ||
| 2829 | 2829 | |
| 2830 | 2830 | rxq = kvzalloc_flex(*rxq, rx_oobs, apc->rx_queue_size); |
| 2831 | 2831 | if (!rxq) |
| 2832 | − return NULL; | |
| 2832 | + return ERR_PTR(-ENOMEM); | |
| 2833 | 2833 | |
| 2834 | 2834 | rxq->ndev = ndev; |
| 2835 | 2835 | rxq->num_rx_buf = apc->rx_queue_size; |
| @@ -2930,7 +2930,7 @@ out: · 94 unmodified lines | ||
| 2930 | 2930 | |
| 2931 | 2931 | mana_destroy_rxq(apc, rxq, false); |
| 2932 | 2932 | |
| 2933 | − return NULL; | |
| 2933 | + return ERR_PTR(err); | |
| 2934 | 2934 | } |
| 2935 | 2935 | |
| 2936 | 2936 | static void mana_create_rxq_debugfs(struct mana_port_context *apc, int idx) |
| @@ -2964,8 +2964,8 @@ static int mana_add_rx_queues(struct mana_port_context *apc, · 27 unmodified lines | ||
| 2964 | 2964 | |
| 2965 | 2965 | for (i = 0; i < apc->num_queues; i++) { |
| 2966 | 2966 | rxq = mana_create_rxq(apc, i, &apc->eqs[i], ndev); |
| 2967 | − if (!rxq) { | |
| 2968 | − err = -ENOMEM; | |
| 2967 | + if (IS_ERR(rxq)) { | |
| 2968 | + err = PTR_ERR(rxq); | |
| 2969 | 2969 | netdev_err(ndev, "Failed to create rxq %d : %d\n", i, err); |
| 2970 | 2970 | goto out; |
| 2971 | 2971 | } |
| @@ -6187,10 +6187,10 @@ static void nv_remove(struct pci_dev *pci_dev) · 6186 unmodified lines | ||
| 6187 | 6187 | struct net_device *dev = pci_get_drvdata(pci_dev); |
| 6188 | 6188 | struct fe_priv *np = netdev_priv(dev); |
| 6189 | 6189 | |
| 6190 | − free_percpu(np->txrx_stats); | |
| 6191 | − | |
| 6192 | 6190 | unregister_netdev(dev); |
| 6193 | 6191 | |
| 6192 | + free_percpu(np->txrx_stats); | |
| 6193 | + | |
| 6194 | 6194 | nv_restore_mac_addr(pci_dev); |
| 6195 | 6195 | |
| 6196 | 6196 | /* restore any phy related changes */ |
| @@ -107,7 +107,7 @@ static void qede_remove(struct pci_dev *pdev); · 106 unmodified lines | ||
| 107 | 107 | static void qede_shutdown(struct pci_dev *pdev); |
| 108 | 108 | static void qede_link_update(void *dev, struct qed_link_output *link); |
| 109 | 109 | static void qede_schedule_recovery_handler(void *dev); |
| 110 | −static void qede_recovery_handler(struct qede_dev *edev); | |
| 110 | +static bool qede_recovery_handler(struct qede_dev *edev); | |
| 111 | 111 | static void qede_schedule_hw_err_handler(void *dev, |
| 112 | 112 | enum qed_hw_err_type err_type); |
| 113 | 113 | static void qede_get_eth_tlv_data(void *edev, void *data); |
| @@ -1043,21 +1043,6 @@ void __qede_unlock(struct qede_dev *edev) · 929 unmodified lines | ||
| 1043 | 1043 | mutex_unlock(&edev->qede_lock); |
| 1044 | 1044 | } |
| 1045 | 1045 | |
| 1046 | −/* This version of the lock should be used when acquiring the RTNL lock is also | |
| 1047 | − * needed in addition to the internal qede lock. | |
| 1048 | − */ | |
| 1049 | −static void qede_lock(struct qede_dev *edev) | |
| 1050 | −{ | |
| 1051 | − rtnl_lock(); | |
| 1052 | − __qede_lock(edev); | |
| 1053 | −} | |
| 1054 | − | |
| 1055 | −static void qede_unlock(struct qede_dev *edev) | |
| 1056 | −{ | |
| 1057 | − __qede_unlock(edev); | |
| 1058 | − rtnl_unlock(); | |
| 1059 | −} | |
| 1060 | − | |
| 1061 | 1046 | static void qede_periodic_task(struct work_struct *work) |
| 1062 | 1047 | { |
| 1063 | 1048 | struct qede_dev *edev = container_of(work, struct qede_dev, |
| @@ -1094,6 +1079,8 @@ static void qede_sp_task(struct work_struct *work) · 30 unmodified lines | ||
| 1094 | 1079 | */ |
| 1095 | 1080 | |
| 1096 | 1081 | if (test_and_clear_bit(QEDE_SP_RECOVERY, &edev->sp_flags)) { |
| 1082 | + bool reloaded; | |
| 1083 | + | |
| 1097 | 1084 | cancel_delayed_work_sync(&edev->periodic_task); |
| 1098 | 1085 | #ifdef CONFIG_QED_SRIOV |
| 1099 | 1086 | /* SRIOV must be disabled outside the lock to avoid a deadlock. |
| @@ -1102,9 +1089,17 @@ static void qede_sp_task(struct work_struct *work) · 2 unmodified lines | ||
| 1102 | 1089 | if (pci_num_vf(edev->pdev)) |
| 1103 | 1090 | qede_sriov_configure(edev->pdev, 0); |
| 1104 | 1091 | #endif |
| 1105 | − qede_lock(edev); | |
| 1106 | − qede_recovery_handler(edev); | |
| 1107 | − qede_unlock(edev); | |
| 1092 | + rtnl_lock(); | |
| 1093 | + __qede_lock(edev); | |
| 1094 | + reloaded = qede_recovery_handler(edev); | |
| 1095 | + __qede_unlock(edev); | |
| 1096 | + | |
| 1097 | + /* The udp_tunnel core synchronously calls back into | |
| 1098 | + * qede_udp_tunnel_sync(), which takes the qede lock. | |
| 1099 | + */ | |
| 1100 | + if (reloaded) | |
| 1101 | + udp_tunnel_nic_reset_ntf(edev->ndev); | |
| 1102 | + rtnl_unlock(); | |
| 1108 | 1103 | } |
| 1109 | 1104 | |
| 1110 | 1105 | __qede_lock(edev); |
| @@ -2645,9 +2640,13 @@ static void qede_recovery_failed(struct qede_dev *edev) · 1534 unmodified lines | ||
| 2645 | 2640 | edev->ops->common->set_power_state(edev->cdev, PCI_D3hot); |
| 2646 | 2641 | } |
| 2647 | 2642 | |
| 2648 | −static void qede_recovery_handler(struct qede_dev *edev) | |
| 2643 | +/* Returns true if an open device was successfully reloaded and its | |
| 2644 | + * udp_tunnel ports need to be re-synced by the caller. | |
| 2645 | + */ | |
| 2646 | +static bool qede_recovery_handler(struct qede_dev *edev) | |
| 2649 | 2647 | { |
| 2650 | 2648 | u32 curr_state = edev->state; |
| 2649 | + bool reloaded = false; | |
| 2651 | 2650 | int rc; |
| 2652 | 2651 | |
| 2653 | 2652 | DP_NOTICE(edev, "Starting a recovery process\n"); |
| @@ -2677,17 +2676,18 @@ static void qede_recovery_handler(struct qede_dev *edev) · 23 unmodified lines | ||
| 2677 | 2676 | goto err; |
| 2678 | 2677 | |
| 2679 | 2678 | qede_config_rx_mode(edev->ndev); |
| 2680 | − udp_tunnel_nic_reset_ntf(edev->ndev); | |
| 2679 | + reloaded = true; | |
| 2681 | 2680 | } |
| 2682 | 2681 | |
| 2683 | 2682 | edev->state = curr_state; |
| 2684 | 2683 | |
| 2685 | 2684 | DP_NOTICE(edev, "Recovery handling is done\n"); |
| 2686 | 2685 | |
| 2687 | − return; | |
| 2686 | + return reloaded; | |
| 2688 | 2687 | |
| 2689 | 2688 | err: |
| 2690 | 2689 | qede_recovery_failed(edev); |
| 2690 | + return false; | |
| 2691 | 2691 | } |
| 2692 | 2692 | |
| 2693 | 2693 | static void qede_atomic_hw_err_handler(struct qede_dev *edev) |
| @@ -1623,6 +1623,9 @@ static netdev_tx_t rtase_start_xmit(struct sk_buff *skb, · 1622 unmodified lines | ||
| 1623 | 1623 | err_dma_1: |
| 1624 | 1624 | ring->skbuff[entry] = NULL; |
| 1625 | 1625 | rtase_tx_clear_range(ring, ring->cur_idx + 1, frags); |
| 1626 | + if (frags) | |
| 1627 | + /* the frags were cleared above, along with the skb */ | |
| 1628 | + return NETDEV_TX_OK; | |
| 1626 | 1629 | |
| 1627 | 1630 | err_dma_0: |
| 1628 | 1631 | tp->stats.tx_dropped++; |
| @@ -597,14 +597,13 @@ static int init_dma_desc_rings(struct net_device *netd) · 596 unmodified lines | ||
| 597 | 597 | |
| 598 | 598 | return 0; |
| 599 | 599 | |
| 600 | −txalloc_err: | |
| 601 | − while (queue_num--) | |
| 602 | − free_tx_ring(priv->device, priv->txq[queue_num], tx_rsize); | |
| 603 | − return ret; | |
| 604 | − | |
| 605 | 600 | rxalloc_err: |
| 606 | 601 | while (queue_num--) |
| 607 | 602 | free_rx_ring(priv->device, priv->rxq[queue_num], rx_rsize); |
| 603 | + queue_num = SXGBE_TX_QUEUES; | |
| 604 | +txalloc_err: | |
| 605 | + while (queue_num--) | |
| 606 | + free_tx_ring(priv->device, priv->txq[queue_num], tx_rsize); | |
| 608 | 607 | return ret; |
| 609 | 608 | } |
| 610 | 609 | |
| @@ -1079,7 +1078,9 @@ static int sxgbe_open(struct net_device *dev) · 468 unmodified lines | ||
| 1079 | 1078 | priv->dma_buf_sz = SXGBE_ALIGN(DMA_BUFFER_SIZE); |
| 1080 | 1079 | priv->tx_tc = TC_DEFAULT; |
| 1081 | 1080 | priv->rx_tc = TC_DEFAULT; |
| 1082 | − init_dma_desc_rings(dev); | |
| 1081 | + ret = init_dma_desc_rings(dev); | |
| 1082 | + if (ret) | |
| 1083 | + goto init_phy_error; | |
| 1083 | 1084 | |
| 1084 | 1085 | /* DMA initialization and SW reset */ |
| 1085 | 1086 | ret = sxgbe_init_dma_engine(priv); |
| @@ -1188,6 +1189,7 @@ static int sxgbe_open(struct net_device *dev) · 102 unmodified lines | ||
| 1188 | 1189 | |
| 1189 | 1190 | init_error: |
| 1190 | 1191 | free_dma_desc_resources(priv); |
| 1192 | +init_phy_error: | |
| 1191 | 1193 | if (dev->phydev) |
| 1192 | 1194 | phy_disconnect(dev->phydev); |
| 1193 | 1195 | phy_error: |
| @@ -755,7 +755,8 @@ static int stmmac_hwtstamp_set(struct net_device *dev, · 754 unmodified lines | ||
| 755 | 755 | config->rx_filter = HWTSTAMP_FILTER_PTP_V2_EVENT; |
| 756 | 756 | ptp_v2 = PTP_TCR_TSVER2ENA; |
| 757 | 757 | snap_type_sel = PTP_TCR_SNAPTYPSEL_1; |
| 758 | − if (priv->synopsys_id < DWMAC_CORE_4_10) | |
| 758 | + if (priv->synopsys_id < DWMAC_CORE_3_70 && | |
| 759 | + priv->plat->core_type != DWMAC_CORE_XGMAC) | |
| 759 | 760 | ts_event_en = PTP_TCR_TSEVNTENA; |
| 760 | 761 | ptp_over_ipv4_udp = PTP_TCR_TSIPV4ENA; |
| 761 | 762 | ptp_over_ipv6_udp = PTP_TCR_TSIPV6ENA; |
| @@ -2747,8 +2748,8 @@ static bool stmmac_xdp_xmit_zc(struct stmmac_priv *priv, u32 queue, u32 budget) · 1985 unmodified lines | ||
| 2747 | 2748 | meta_req.set_ic = &set_ic; |
| 2748 | 2749 | meta_req.tbs = tx_q->tbs; |
| 2749 | 2750 | meta_req.edesc = &tx_q->dma_entx[entry]; |
| 2750 | − xsk_tx_metadata_request(meta, &stmmac_xsk_tx_metadata_ops, | |
| 2751 | − &meta_req); | |
| 2751 | + xsk_tx_metadata_request(pool, &meta, | |
| 2752 | + &stmmac_xsk_tx_metadata_ops, &meta_req); | |
| 2752 | 2753 | if (set_ic) { |
| 2753 | 2754 | tx_q->tx_count_frames = 0; |
| 2754 | 2755 | stmmac_set_tx_ic(priv, tx_desc); |
| @@ -4133,6 +4134,15 @@ static int __stmmac_open(struct net_device *dev, · 1378 unmodified lines | ||
| 4133 | 4134 | dma_conf->tx_queue[i].tbs = priv->dma_conf.tx_queue[i].tbs; |
| 4134 | 4135 | memcpy(&priv->dma_conf, dma_conf, sizeof(*dma_conf)); |
| 4135 | 4136 | |
| 4137 | + /* The PHY is suspended when the interface is reopened without | |
| 4138 | + * disconnecting the PHY, e.g. on MTU change. IEEE 802.3 allows PHYs | |
| 4139 | + * to stop their receive clock while powered down, but the DMA | |
| 4140 | + * software reset in stmmac_hw_setup() requires a running receive | |
| 4141 | + * clock, and phylink_start() below resumes the PHY only after the | |
| 4142 | + * hardware setup. Resume a suspended PHY here first. | |
| 4143 | + */ | |
| 4144 | + phylink_prepare_resume(priv->phylink); | |
| 4145 | + | |
| 4136 | 4146 | stmmac_reset_queues_param(priv); |
| 4137 | 4147 | |
| 4138 | 4148 | ret = stmmac_hw_setup(dev); |
| @@ -1606,6 +1606,8 @@ static netdev_tx_t wx_xmit_frame_ring(struct sk_buff *skb, · 1605 unmodified lines | ||
| 1606 | 1606 | if (skb_vlan_tag_present(skb)) { |
| 1607 | 1607 | tx_flags |= skb_vlan_tag_get(skb) << WX_TX_FLAGS_VLAN_SHIFT; |
| 1608 | 1608 | tx_flags |= WX_TX_FLAGS_HW_VLAN; |
| 1609 | + } else if (eth_type_vlan(skb->protocol)) { | |
| 1610 | + tx_flags |= WX_TX_FLAGS_SW_VLAN; | |
| 1609 | 1611 | } |
| 1610 | 1612 | |
| 1611 | 1613 | if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) && |
| @@ -35,25 +35,11 @@ static void ovpn_priv_free(struct net_device *net) · 34 unmodified lines | ||
| 35 | 35 | |
| 36 | 36 | static int ovpn_mp_alloc(struct ovpn_priv *ovpn) |
| 37 | 37 | { |
| 38 | − struct in_device *dev_v4; | |
| 39 | 38 | int i; |
| 40 | 39 | |
| 41 | 40 | if (ovpn->mode != OVPN_MODE_MP) |
| 42 | 41 | return 0; |
| 43 | 42 | |
| 44 | − dev_v4 = __in_dev_get_rtnl(ovpn->dev); | |
| 45 | − if (dev_v4) { | |
| 46 | − /* disable redirects as Linux gets confused by ovpn | |
| 47 | − * handling same-LAN routing. | |
| 48 | − * This happens because a multipeer interface is used as | |
| 49 | − * relay point between hosts in the same subnet, while | |
| 50 | − * in a classic LAN this would not be needed because the | |
| 51 | − * two hosts would be able to talk directly. | |
| 52 | − */ | |
| 53 | − IN_DEV_CONF_SET(dev_v4, SEND_REDIRECTS, false); | |
| 54 | − IPV4_DEVCONF_ALL(dev_net(ovpn->dev), SEND_REDIRECTS) = false; | |
| 55 | − } | |
| 56 | − | |
| 57 | 43 | /* the peer container is fairly large, therefore we allocate it only in |
| 58 | 44 | * MP mode |
| 59 | 45 | */ |
| @@ -97,9 +83,38 @@ static void ovpn_net_uninit(struct net_device *dev) · 37 unmodified lines | ||
| 97 | 83 | gro_cells_destroy(&ovpn->gro_cells); |
| 98 | 84 | } |
| 99 | 85 | |
| 86 | +static int ovpn_net_open(struct net_device *dev) | |
| 87 | +{ | |
| 88 | + struct ovpn_priv *ovpn = netdev_priv(dev); | |
| 89 | + struct in_device *dev_v4; | |
| 90 | + | |
| 91 | + /* the IPv4 in_device (and thus its config) is recreated whenever the | |
| 92 | + * interface is moved to a new netns, so redirects must be disabled on | |
| 93 | + * every bring-up rather than once at creation time, otherwise the | |
| 94 | + * setting is silently lost after such a move | |
| 95 | + */ | |
| 96 | + if (ovpn->mode == OVPN_MODE_MP) { | |
| 97 | + dev_v4 = __in_dev_get_rtnl(dev); | |
| 98 | + if (dev_v4) { | |
| 99 | + /* disable redirects as Linux gets confused by ovpn | |
| 100 | + * handling same-LAN routing. | |
| 101 | + * This happens because a multipeer interface is used as | |
| 102 | + * relay point between hosts in the same subnet, while | |
| 103 | + * in a classic LAN this would not be needed because the | |
| 104 | + * two hosts would be able to talk directly. | |
| 105 | + */ | |
| 106 | + IN_DEV_CONF_SET(dev_v4, SEND_REDIRECTS, false); | |
| 107 | + IPV4_DEVCONF_ALL(dev_net(dev), SEND_REDIRECTS) = false; | |
| 108 | + } | |
| 109 | + } | |
| 110 | + | |
| 111 | + return 0; | |
| 112 | +} | |
| 113 | + | |
| 100 | 114 | static const struct net_device_ops ovpn_netdev_ops = { |
| 101 | 115 | .ndo_init = ovpn_net_init, |
| 102 | 116 | .ndo_uninit = ovpn_net_uninit, |
| 117 | + .ndo_open = ovpn_net_open, | |
| 103 | 118 | .ndo_start_xmit = ovpn_net_xmit, |
| 104 | 119 | }; |
| 105 | 120 | |
| @@ -183,6 +198,7 @@ static int ovpn_newlink(struct net_device *dev, · 77 unmodified lines | ||
| 183 | 198 | struct ovpn_priv *ovpn = netdev_priv(dev); |
| 184 | 199 | struct nlattr **data = params->data; |
| 185 | 200 | enum ovpn_mode mode = OVPN_MODE_P2P; |
| 201 | + int ret; | |
| 186 | 202 | |
| 187 | 203 | if (data && data[IFLA_OVPN_MODE]) { |
| 188 | 204 | mode = nla_get_u8(data[IFLA_OVPN_MODE]); |
| @@ -207,7 +223,17 @@ static int ovpn_newlink(struct net_device *dev, · 18 unmodified lines | ||
| 207 | 223 | else |
| 208 | 224 | netif_carrier_off(dev); |
| 209 | 225 | |
| 210 | − return register_netdevice(dev); | |
| 226 | + ret = register_netdevice(dev); | |
| 227 | + if (ret < 0) | |
| 228 | + return ret; | |
| 229 | + | |
| 230 | + return 0; | |
| 231 | +} | |
| 232 | + | |
| 233 | +static size_t ovpn_get_size(const struct net_device *dev) | |
| 234 | +{ | |
| 235 | + /* IFLA_OVPN_MODE */ | |
| 236 | + return nla_total_size(sizeof(u8)); | |
| 211 | 237 | } |
| 212 | 238 | |
| 213 | 239 | static int ovpn_fill_info(struct sk_buff *skb, const struct net_device *dev) |
| @@ -228,13 +254,17 @@ static struct rtnl_link_ops ovpn_link_ops = { · 14 unmodified lines | ||
| 228 | 254 | .policy = ovpn_policy, |
| 229 | 255 | .maxtype = IFLA_OVPN_MAX, |
| 230 | 256 | .newlink = ovpn_newlink, |
| 257 | + .get_size = ovpn_get_size, | |
| 231 | 258 | .fill_info = ovpn_fill_info, |
| 232 | 259 | }; |
| 233 | 260 | |
| 234 | 261 | static int __init ovpn_init(void) |
| 235 | 262 | { |
| 236 | − int err = rtnl_link_register(&ovpn_link_ops); | |
| 263 | + int err; | |
| 264 | + | |
| 265 | + ovpn_tcp_init(); | |
| 237 | 266 | |
| 267 | + err = rtnl_link_register(&ovpn_link_ops); | |
| 238 | 268 | if (err) { |
| 239 | 269 | pr_err("ovpn: can't register rtnl link ops: %d\n", err); |
| 240 | 270 | return err; |
| @@ -246,8 +276,6 @@ static int __init ovpn_init(void) · 5 unmodified lines | ||
| 246 | 276 | goto unreg_rtnl; |
| 247 | 277 | } |
| 248 | 278 | |
| 249 | − ovpn_tcp_init(); | |
| 250 | − | |
| 251 | 279 | return 0; |
| 252 | 280 | |
| 253 | 281 | unreg_rtnl: |
| @@ -16,6 +16,14 @@ static const struct netlink_range_validation ovpn_a_peer_id_range = { · 15 unmodified lines | ||
| 16 | 16 | .max = 16777215ULL, |
| 17 | 17 | }; |
| 18 | 18 | |
| 19 | +static const struct netlink_range_validation ovpn_a_peer_keepalive_interval_range = { | |
| 20 | + .max = 86400ULL, | |
| 21 | +}; | |
| 22 | + | |
| 23 | +static const struct netlink_range_validation ovpn_a_peer_keepalive_timeout_range = { | |
| 24 | + .max = 86400ULL, | |
| 25 | +}; | |
| 26 | + | |
| 19 | 27 | static const struct netlink_range_validation ovpn_a_peer_tx_id_range = { |
| 20 | 28 | .max = 16777215ULL, |
| 21 | 29 | }; |
| @@ -68,8 +76,8 @@ const struct nla_policy ovpn_peer_nl_policy[OVPN_A_PEER_TX_ID + 1] = { · 46 unmodified lines | ||
| 68 | 76 | [OVPN_A_PEER_LOCAL_IPV4] = { .type = NLA_BE32, }, |
| 69 | 77 | [OVPN_A_PEER_LOCAL_IPV6] = NLA_POLICY_EXACT_LEN(16), |
| 70 | 78 | [OVPN_A_PEER_LOCAL_PORT] = NLA_POLICY_MIN(NLA_BE16, 1), |
| 71 | − [OVPN_A_PEER_KEEPALIVE_INTERVAL] = { .type = NLA_U32, }, | |
| 72 | − [OVPN_A_PEER_KEEPALIVE_TIMEOUT] = { .type = NLA_U32, }, | |
| 79 | + [OVPN_A_PEER_KEEPALIVE_INTERVAL] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_keepalive_interval_range), | |
| 80 | + [OVPN_A_PEER_KEEPALIVE_TIMEOUT] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_keepalive_timeout_range), | |
| 73 | 81 | [OVPN_A_PEER_DEL_REASON] = NLA_POLICY_MAX(NLA_U32, 4), |
| 74 | 82 | [OVPN_A_PEER_VPN_RX_BYTES] = { .type = NLA_UINT, }, |
| 75 | 83 | [OVPN_A_PEER_VPN_TX_BYTES] = { .type = NLA_UINT, }, |
| @@ -97,8 +105,8 @@ const struct nla_policy ovpn_peer_new_input_nl_policy[OVPN_A_PEER_TX_ID + 1] = { · 21 unmodified lines | ||
| 97 | 105 | [OVPN_A_PEER_VPN_IPV6] = NLA_POLICY_EXACT_LEN(16), |
| 98 | 106 | [OVPN_A_PEER_LOCAL_IPV4] = { .type = NLA_BE32, }, |
| 99 | 107 | [OVPN_A_PEER_LOCAL_IPV6] = NLA_POLICY_EXACT_LEN(16), |
| 100 | − [OVPN_A_PEER_KEEPALIVE_INTERVAL] = { .type = NLA_U32, }, | |
| 101 | − [OVPN_A_PEER_KEEPALIVE_TIMEOUT] = { .type = NLA_U32, }, | |
| 108 | + [OVPN_A_PEER_KEEPALIVE_INTERVAL] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_keepalive_interval_range), | |
| 109 | + [OVPN_A_PEER_KEEPALIVE_TIMEOUT] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_keepalive_timeout_range), | |
| 102 | 110 | [OVPN_A_PEER_TX_ID] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_tx_id_range), |
| 103 | 111 | }; |
| 104 | 112 | |
| @@ -112,8 +120,8 @@ const struct nla_policy ovpn_peer_set_input_nl_policy[OVPN_A_PEER_TX_ID + 1] = { · 7 unmodified lines | ||
| 112 | 120 | [OVPN_A_PEER_VPN_IPV6] = NLA_POLICY_EXACT_LEN(16), |
| 113 | 121 | [OVPN_A_PEER_LOCAL_IPV4] = { .type = NLA_BE32, }, |
| 114 | 122 | [OVPN_A_PEER_LOCAL_IPV6] = NLA_POLICY_EXACT_LEN(16), |
| 115 | − [OVPN_A_PEER_KEEPALIVE_INTERVAL] = { .type = NLA_U32, }, | |
| 116 | − [OVPN_A_PEER_KEEPALIVE_TIMEOUT] = { .type = NLA_U32, }, | |
| 123 | + [OVPN_A_PEER_KEEPALIVE_INTERVAL] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_keepalive_interval_range), | |
| 124 | + [OVPN_A_PEER_KEEPALIVE_TIMEOUT] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_keepalive_timeout_range), | |
| 117 | 125 | [OVPN_A_PEER_TX_ID] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_tx_id_range), |
| 118 | 126 | }; |
| 119 | 127 | |
| @@ -534,6 +534,12 @@ int ovpn_nl_peer_set_doit(struct sk_buff *skb, struct genl_info *info) · 533 unmodified lines | ||
| 534 | 534 | */ |
| 535 | 535 | if (ret > 0) |
| 536 | 536 | ovpn_peer_hash_vpn_ip(peer); |
| 537 | + /* if the remote endpoint was updated, the by_transp_addr hash bucket | |
| 538 | + * also needs to be refreshed, otherwise incoming packets from the new | |
| 539 | + * remote address would fail the lockless lookup | |
| 540 | + */ | |
| 541 | + if (attrs[OVPN_A_PEER_REMOTE_IPV4] || attrs[OVPN_A_PEER_REMOTE_IPV6]) | |
| 542 | + ovpn_peer_hash_transp_addr(peer); | |
| 537 | 543 | spin_unlock_bh(&ovpn->lock); |
| 538 | 544 | ovpn_peer_put(peer); |
| 539 | 545 | |
| @@ -189,6 +189,9 @@ int ovpn_peer_reset_sockaddr(struct ovpn_peer *peer, · 188 unmodified lines | ||
| 189 | 189 | &(*__tbl1)[ovpn_get_hash_slot(*__tbl1, _key, _key_len)];\ |
| 190 | 190 | }) |
| 191 | 191 | |
| 192 | +static void __ovpn_peer_hash_transp_addr(struct ovpn_peer *peer, | |
| 193 | + const struct ovpn_bind *bind); | |
| 194 | + | |
| 192 | 195 | /** |
| 193 | 196 | * ovpn_peer_endpoints_update - update remote or local endpoint for peer |
| 194 | 197 | * @peer: peer to update the remote endpoint for |
| @@ -196,7 +199,6 @@ int ovpn_peer_reset_sockaddr(struct ovpn_peer *peer, · 1 unmodified line | ||
| 196 | 199 | */ |
| 197 | 200 | void ovpn_peer_endpoints_update(struct ovpn_peer *peer, struct sk_buff *skb) |
| 198 | 201 | { |
| 199 | − struct hlist_nulls_head *nhead; | |
| 200 | 202 | struct sockaddr_storage ss; |
| 201 | 203 | struct sockaddr_in6 *sa6; |
| 202 | 204 | bool reset_cache = false; |
| @@ -220,9 +222,16 @@ void ovpn_peer_endpoints_update(struct ovpn_peer *peer, struct sk_buff *skb) · 17 unmodified lines | ||
| 220 | 222 | */ |
| 221 | 223 | local_ip = &ip_hdr(skb)->daddr; |
| 222 | 224 | sa = (struct sockaddr_in *)&ss; |
| 223 | − sa->sin_family = AF_INET; | |
| 224 | − sa->sin_addr.s_addr = ip_hdr(skb)->saddr; | |
| 225 | − sa->sin_port = udp_hdr(skb)->source; | |
| 225 | + /* use a designated initializer so the sin_zero padding | |
| 226 | + * is zeroed (it ends up in the by_transp_addr hash key) | |
| 227 | + * without memset-ing the whole sockaddr_storage on the | |
| 228 | + * RX fast path | |
| 229 | + */ | |
| 230 | + *sa = (struct sockaddr_in) { | |
| 231 | + .sin_family = AF_INET, | |
| 232 | + .sin_addr.s_addr = ip_hdr(skb)->saddr, | |
| 233 | + .sin_port = udp_hdr(skb)->source, | |
| 234 | + }; | |
| 226 | 235 | salen = sizeof(*sa); |
| 227 | 236 | reset_cache = true; |
| 228 | 237 | break; |
| @@ -248,11 +257,19 @@ void ovpn_peer_endpoints_update(struct ovpn_peer *peer, struct sk_buff *skb) · 19 unmodified lines | ||
| 248 | 257 | */ |
| 249 | 258 | local_ip = &ipv6_hdr(skb)->daddr; |
| 250 | 259 | sa6 = (struct sockaddr_in6 *)&ss; |
| 251 | − sa6->sin6_family = AF_INET6; | |
| 252 | − sa6->sin6_addr = ipv6_hdr(skb)->saddr; | |
| 253 | − sa6->sin6_port = udp_hdr(skb)->source; | |
| 254 | − sa6->sin6_scope_id = ipv6_iface_scope_id(&ipv6_hdr(skb)->saddr, | |
| 255 | − skb->skb_iif); | |
| 260 | + /* use a designated initializer so the sin6_flowinfo | |
| 261 | + * padding is zeroed (it ends up in the by_transp_addr | |
| 262 | + * hash key) without memset-ing the whole | |
| 263 | + * sockaddr_storage on the RX fast path | |
| 264 | + */ | |
| 265 | + *sa6 = (struct sockaddr_in6) { | |
| 266 | + .sin6_family = AF_INET6, | |
| 267 | + .sin6_addr = ipv6_hdr(skb)->saddr, | |
| 268 | + .sin6_port = udp_hdr(skb)->source, | |
| 269 | + .sin6_scope_id = | |
| 270 | + ipv6_iface_scope_id(&ipv6_hdr(skb)->saddr, | |
| 271 | + skb->skb_iif), | |
| 272 | + }; | |
| 256 | 273 | salen = sizeof(*sa6); |
| 257 | 274 | reset_cache = true; |
| 258 | 275 | break; |
| @@ -295,42 +312,25 @@ void ovpn_peer_endpoints_update(struct ovpn_peer *peer, struct sk_buff *skb) · 36 unmodified lines | ||
| 295 | 312 | ovpn_nl_peer_float_notify(peer, &ss); |
| 296 | 313 | |
| 297 | 314 | /* rehashing is required only in MP mode as P2P has one peer |
| 298 | − * only and thus there is no hashtable | |
| 315 | + * only and thus there is no hashtable. | |
| 316 | + * | |
| 317 | + * This function may be invoked concurrently, so re-read peer->bind | |
| 318 | + * under the proper locks and rehash against its current value. | |
| 299 | 319 | */ |
| 300 | − if (peer->ovpn->mode == OVPN_MODE_MP) { | |
| 301 | − spin_lock_bh(&peer->ovpn->lock); | |
| 302 | − spin_lock_bh(&peer->lock); | |
| 303 | − bind = rcu_dereference_protected(peer->bind, | |
| 304 | − lockdep_is_held(&peer->lock)); | |
| 305 | − if (unlikely(!bind)) { | |
| 306 | − spin_unlock_bh(&peer->lock); | |
| 307 | − spin_unlock_bh(&peer->ovpn->lock); | |
| 308 | − return; | |
| 309 | − } | |
| 310 | − | |
| 311 | − /* This function may be invoked concurrently, therefore another | |
| 312 | − * float may have happened in parallel: perform rehashing | |
| 313 | − * using the peer->bind->remote directly as key | |
| 314 | − */ | |
| 315 | − | |
| 316 | − switch (bind->remote.in4.sin_family) { | |
| 317 | − case AF_INET: | |
| 318 | − salen = sizeof(*sa); | |
| 319 | − break; | |
| 320 | − case AF_INET6: | |
| 321 | − salen = sizeof(*sa6); | |
| 322 | − break; | |
| 323 | − } | |
| 320 | + if (peer->ovpn->mode != OVPN_MODE_MP) | |
| 321 | + return; | |
| 324 | 322 | |
| 325 | − /* remove old hashing */ | |
| 326 | − hlist_nulls_del_init_rcu(&peer->hash_entry_transp_addr); | |
| 327 | − /* re-add with new transport address */ | |
| 328 | − nhead = ovpn_get_hash_head(peer->ovpn->peers->by_transp_addr, | |
| 329 | − &bind->remote, salen); | |
| 330 | − hlist_nulls_add_head_rcu(&peer->hash_entry_transp_addr, nhead); | |
| 331 | − spin_unlock_bh(&peer->lock); | |
| 332 | − spin_unlock_bh(&peer->ovpn->lock); | |
| 333 | − } | |
| 323 | + /* This function may be invoked concurrently, therefore another | |
| 324 | + * float may have happened in parallel: re-acquire the locks and | |
| 325 | + * rehash using the peer->bind->remote directly as key | |
| 326 | + */ | |
| 327 | + spin_lock_bh(&peer->ovpn->lock); | |
| 328 | + spin_lock_bh(&peer->lock); | |
| 329 | + bind = rcu_dereference_protected(peer->bind, | |
| 330 | + lockdep_is_held(&peer->lock)); | |
| 331 | + __ovpn_peer_hash_transp_addr(peer, bind); | |
| 332 | + spin_unlock_bh(&peer->lock); | |
| 333 | + spin_unlock_bh(&peer->ovpn->lock); | |
| 334 | 334 | return; |
| 335 | 335 | unlock: |
| 336 | 336 | spin_unlock_bh(&peer->lock); |
| @@ -896,6 +896,83 @@ bool ovpn_peer_check_by_src(struct ovpn_priv *ovpn, struct sk_buff *skb, · 559 unmodified lines | ||
| 896 | 896 | return match; |
| 897 | 897 | } |
| 898 | 898 | |
| 899 | +/* Move @peer to the by_transp_addr bucket matching its current bind. | |
| 900 | + * | |
| 901 | + * Caller must hold both peer->ovpn->lock and peer->lock, and must have | |
| 902 | + * already dereferenced a valid (non-NULL) peer->bind, passed in as @bind. | |
| 903 | + */ | |
| 904 | +static void __ovpn_peer_hash_transp_addr(struct ovpn_peer *peer, | |
| 905 | + const struct ovpn_bind *bind) | |
| 906 | +{ | |
| 907 | + struct sockaddr_storage sa = {}; | |
| 908 | + struct hlist_nulls_head *nhead; | |
| 909 | + struct sockaddr_in6 *sa6; | |
| 910 | + struct sockaddr_in *sa4; | |
| 911 | + size_t salen; | |
| 912 | + | |
| 913 | + lockdep_assert_held(&peer->ovpn->lock); | |
| 914 | + lockdep_assert_held(&peer->lock); | |
| 915 | + | |
| 916 | + if (WARN_ON_ONCE(!bind)) | |
| 917 | + return; | |
| 918 | + | |
| 919 | + /* peer may have been concurrently removed between the caller's | |
| 920 | + * initial lookup and our acquisition of ovpn->lock; skip the | |
| 921 | + * rehash so we don't re-insert a removed peer | |
| 922 | + */ | |
| 923 | + if (unlikely(hlist_unhashed(&peer->hash_entry_id))) | |
| 924 | + return; | |
| 925 | + | |
| 926 | + /* Build the hash key from the transport identity only | |
| 927 | + * (family/address/port), matching ovpn_peer_add_mp() and the lookup | |
| 928 | + * in ovpn_peer_get_by_transp_addr(). Hashing bind->remote directly | |
| 929 | + * would fold in sin6_scope_id (set on the float path but never by the | |
| 930 | + * lookup), scattering the peer into a bucket lookups cannot reach. | |
| 931 | + */ | |
| 932 | + switch (bind->remote.in4.sin_family) { | |
| 933 | + case AF_INET: | |
| 934 | + sa4 = (struct sockaddr_in *)&sa; | |
| 935 | + sa4->sin_family = AF_INET; | |
| 936 | + sa4->sin_addr.s_addr = bind->remote.in4.sin_addr.s_addr; | |
| 937 | + sa4->sin_port = bind->remote.in4.sin_port; | |
| 938 | + salen = sizeof(*sa4); | |
| 939 | + break; | |
| 940 | + case AF_INET6: | |
| 941 | + sa6 = (struct sockaddr_in6 *)&sa; | |
| 942 | + sa6->sin6_family = AF_INET6; | |
| 943 | + sa6->sin6_addr = bind->remote.in6.sin6_addr; | |
| 944 | + sa6->sin6_port = bind->remote.in6.sin6_port; | |
| 945 | + salen = sizeof(*sa6); | |
| 946 | + break; | |
| 947 | + default: | |
| 948 | + return; | |
| 949 | + } | |
| 950 | + | |
| 951 | + /* remove old hashing (no-op if entry is not currently linked) */ | |
| 952 | + hlist_nulls_del_init_rcu(&peer->hash_entry_transp_addr); | |
| 953 | + /* re-add with current transport address */ | |
| 954 | + nhead = ovpn_get_hash_head(peer->ovpn->peers->by_transp_addr, &sa, | |
| 955 | + salen); | |
| 956 | + hlist_nulls_add_head_rcu(&peer->hash_entry_transp_addr, nhead); | |
| 957 | +} | |
| 958 | + | |
| 959 | +void ovpn_peer_hash_transp_addr(struct ovpn_peer *peer) | |
| 960 | +{ | |
| 961 | + struct ovpn_bind *bind; | |
| 962 | + | |
| 963 | + lockdep_assert_held(&peer->ovpn->lock); | |
| 964 | + | |
| 965 | + /* rehashing makes sense only in multipeer mode */ | |
| 966 | + if (peer->ovpn->mode != OVPN_MODE_MP) | |
| 967 | + return; | |
| 968 | + | |
| 969 | + spin_lock_bh(&peer->lock); | |
| 970 | + bind = rcu_dereference_protected(peer->bind, | |
| 971 | + lockdep_is_held(&peer->lock)); | |
| 972 | + __ovpn_peer_hash_transp_addr(peer, bind); | |
| 973 | + spin_unlock_bh(&peer->lock); | |
| 974 | +} | |
| 975 | + | |
| 899 | 976 | void ovpn_peer_hash_vpn_ip(struct ovpn_peer *peer) |
| 900 | 977 | { |
| 901 | 978 | struct hlist_nulls_head *nhead; |
| @@ -906,6 +983,13 @@ void ovpn_peer_hash_vpn_ip(struct ovpn_peer *peer) · 4 unmodified lines | ||
| 906 | 983 | if (peer->ovpn->mode != OVPN_MODE_MP) |
| 907 | 984 | return; |
| 908 | 985 | |
| 986 | + /* peer may have been concurrently removed between the caller's | |
| 987 | + * initial lookup and our acquisition of ovpn->lock; skip the | |
| 988 | + * rehash so we don't re-insert a removed peer | |
| 989 | + */ | |
| 990 | + if (hlist_unhashed(&peer->hash_entry_id)) | |
| 991 | + return; | |
| 992 | + | |
| 909 | 993 | if (peer->vpn_addrs.ipv4.s_addr != htonl(INADDR_ANY)) { |
| 910 | 994 | /* remove potential old hashing */ |
| 911 | 995 | hlist_nulls_del_init_rcu(&peer->hash_entry_addr4); |
| @@ -1165,7 +1249,7 @@ static void ovpn_peer_release_p2p(struct ovpn_priv *ovpn, struct sock *sk, · 253 unmodified lines | ||
| 1165 | 1249 | } |
| 1166 | 1250 | |
| 1167 | 1251 | if (sk) { |
| 1168 | − ovpn_sock = rcu_access_pointer(peer->sock); | |
| 1252 | + ovpn_sock = rcu_dereference_bh(peer->sock); | |
| 1169 | 1253 | if (!ovpn_sock || ovpn_sock->sk != sk) { |
| 1170 | 1254 | spin_unlock_bh(&ovpn->lock); |
| 1171 | 1255 | return; |
| @@ -150,6 +150,7 @@ struct ovpn_peer *ovpn_peer_get_by_id(struct ovpn_priv *ovpn, u32 peer_id); · 149 unmodified lines | ||
| 150 | 150 | struct ovpn_peer *ovpn_peer_get_by_dst(struct ovpn_priv *ovpn, |
| 151 | 151 | struct sk_buff *skb); |
| 152 | 152 | void ovpn_peer_hash_vpn_ip(struct ovpn_peer *peer); |
| 153 | +void ovpn_peer_hash_transp_addr(struct ovpn_peer *peer); | |
| 153 | 154 | bool ovpn_peer_check_by_src(struct ovpn_priv *ovpn, struct sk_buff *skb, |
| 154 | 155 | struct ovpn_peer *peer); |
| 155 | 156 | |
| @@ -162,6 +162,15 @@ struct ovpn_socket *ovpn_socket_new(struct socket *sock, struct ovpn_peer *peer) · 161 unmodified lines | ||
| 162 | 162 | rcu_read_lock(); |
| 163 | 163 | ovpn_sock = rcu_dereference_sk_user_data(sk); |
| 164 | 164 | if (ovpn_sock) { |
| 165 | + /* something else filled the sk_user_data without | |
| 166 | + * setting the encap_type. Reject the socket. | |
| 167 | + */ | |
| 168 | + if (!type) { | |
| 169 | + ovpn_sock = ERR_PTR(-EBUSY); | |
| 170 | + rcu_read_unlock(); | |
| 171 | + goto sock_release; | |
| 172 | + } | |
| 173 | + | |
| 165 | 174 | /* socket owned by another ovpn instance, we can't use it */ |
| 166 | 175 | if (ovpn_sock->ovpn != peer->ovpn) { |
| 167 | 176 | ovpn_sock = ERR_PTR(-EBUSY); |
| @@ -53,15 +53,15 @@ · 52 unmodified lines | ||
| 53 | 53 | #define MTK_GPHY_LED_RX_BLINK_SET (MTK_PHY_LED_BLINK_1000RX | \ |
| 54 | 54 | MTK_PHY_LED_BLINK_100RX | \ |
| 55 | 55 | MTK_PHY_LED_BLINK_10RX) |
| 56 | −#define MTK_GPHY_LED_TX_BLINK_SET (MTK_PHY_LED_BLINK_1000RX | \ | |
| 57 | − MTK_PHY_LED_BLINK_100RX | \ | |
| 58 | − MTK_PHY_LED_BLINK_10RX) | |
| 56 | +#define MTK_GPHY_LED_TX_BLINK_SET (MTK_PHY_LED_BLINK_1000TX | \ | |
| 57 | + MTK_PHY_LED_BLINK_100TX | \ | |
| 58 | + MTK_PHY_LED_BLINK_10TX) | |
| 59 | 59 | |
| 60 | 60 | #define MTK_2P5GPHY_LED_ON_SET (MTK_PHY_LED_ON_LINK2500 | \ |
| 61 | 61 | MTK_GPHY_LED_ON_SET) |
| 62 | 62 | #define MTK_2P5GPHY_LED_RX_BLINK_SET (MTK_PHY_LED_BLINK_2500RX | \ |
| 63 | 63 | MTK_GPHY_LED_RX_BLINK_SET) |
| 64 | −#define MTK_2P5GPHY_LED_TX_BLINK_SET (MTK_PHY_LED_BLINK_2500RX | \ | |
| 64 | +#define MTK_2P5GPHY_LED_TX_BLINK_SET (MTK_PHY_LED_BLINK_2500TX | \ | |
| 65 | 65 | MTK_GPHY_LED_TX_BLINK_SET) |
| 66 | 66 | |
| 67 | 67 | #define MTK_PHY_LED_STATE_FORCE_ON 0 |
| @@ -1875,8 +1875,8 @@ struct phylink *phylink_create(struct phylink_config *config, · 1874 unmodified lines | ||
| 1875 | 1875 | } else if (config->type == PHYLINK_DEV) { |
| 1876 | 1876 | pl->dev = config->dev; |
| 1877 | 1877 | } else { |
| 1878 | − kfree(pl); | |
| 1879 | − return ERR_PTR(-EINVAL); | |
| 1878 | + ret = -EINVAL; | |
| 1879 | + goto free_pl; | |
| 1880 | 1880 | } |
| 1881 | 1881 | |
| 1882 | 1882 | pl->mac_supports_eee_ops = phylink_mac_implements_lpi(mac_ops); |
| @@ -1909,28 +1909,29 @@ struct phylink *phylink_create(struct phylink_config *config, · 26 unmodified lines | ||
| 1909 | 1909 | phylink_validate(pl, pl->supported, &pl->link_config); |
| 1910 | 1910 | |
| 1911 | 1911 | ret = phylink_parse_mode(pl, fwnode); |
| 1912 | − if (ret < 0) { | |
| 1913 | − kfree(pl); | |
| 1914 | − return ERR_PTR(ret); | |
| 1915 | − } | |
| 1912 | + if (ret < 0) | |
| 1913 | + goto free_pl; | |
| 1916 | 1914 | |
| 1917 | 1915 | if (pl->cfg_link_an_mode == MLO_AN_FIXED) { |
| 1918 | 1916 | ret = phylink_parse_fixedlink(pl, fwnode); |
| 1919 | − if (ret < 0) { | |
| 1920 | − kfree(pl); | |
| 1921 | − return ERR_PTR(ret); | |
| 1922 | − } | |
| 1917 | + if (ret < 0) | |
| 1918 | + goto release_link_gpio; | |
| 1923 | 1919 | } |
| 1924 | 1920 | |
| 1925 | 1921 | pl->req_link_an_mode = pl->cfg_link_an_mode; |
| 1926 | 1922 | |
| 1927 | 1923 | ret = phylink_register_sfp(pl, fwnode); |
| 1928 | − if (ret < 0) { | |
| 1929 | − kfree(pl); | |
| 1930 | − return ERR_PTR(ret); | |
| 1931 | − } | |
| 1924 | + if (ret < 0) | |
| 1925 | + goto release_link_gpio; | |
| 1932 | 1926 | |
| 1933 | 1927 | return pl; |
| 1928 | + | |
| 1929 | +release_link_gpio: | |
| 1930 | + if (pl->link_gpio) | |
| 1931 | + gpiod_put(pl->link_gpio); | |
| 1932 | +free_pl: | |
| 1933 | + kfree(pl); | |
| 1934 | + return ERR_PTR(ret); | |
| 1934 | 1935 | } |
| 1935 | 1936 | EXPORT_SYMBOL_GPL(phylink_create); |
| 1936 | 1937 | |
| @@ -1074,10 +1074,21 @@ static int tap_get_user_xdp(struct tap_queue *q, struct xdp_buff *xdp) · 1073 unmodified lines | ||
| 1074 | 1074 | skb_reset_mac_header(skb); |
| 1075 | 1075 | skb->protocol = eth_hdr(skb)->h_proto; |
| 1076 | 1076 | |
| 1077 | + rcu_read_lock(); | |
| 1078 | + tap = rcu_dereference(q->tap); | |
| 1079 | + if (!tap) { | |
| 1080 | + kfree_skb(skb); | |
| 1081 | + rcu_read_unlock(); | |
| 1082 | + return 0; | |
| 1083 | + } | |
| 1084 | + skb->dev = tap->dev; | |
| 1085 | + | |
| 1077 | 1086 | if (vnet_hdr_len) { |
| 1078 | 1087 | err = tun_vnet_hdr_to_skb(q->flags, skb, gso); |
| 1079 | − if (err) | |
| 1088 | + if (err) { | |
| 1089 | + rcu_read_unlock(); | |
| 1080 | 1090 | goto err_kfree; |
| 1091 | + } | |
| 1081 | 1092 | } |
| 1082 | 1093 | |
| 1083 | 1094 | /* Move network header to the right position for VLAN tagged packets */ |
| @@ -1085,15 +1096,8 @@ static int tap_get_user_xdp(struct tap_queue *q, struct xdp_buff *xdp) · 1 unmodified line | ||
| 1085 | 1096 | vlan_get_protocol_and_depth(skb, skb->protocol, &depth) != 0) |
| 1086 | 1097 | skb_set_network_header(skb, depth); |
| 1087 | 1098 | |
| 1088 | − rcu_read_lock(); | |
| 1089 | − tap = rcu_dereference(q->tap); | |
| 1090 | − if (tap) { | |
| 1091 | − skb->dev = tap->dev; | |
| 1092 | − skb_probe_transport_header(skb); | |
| 1093 | − dev_queue_xmit(skb); | |
| 1094 | − } else { | |
| 1095 | − kfree_skb(skb); | |
| 1096 | − } | |
| 1099 | + skb_probe_transport_header(skb); | |
| 1100 | + dev_queue_xmit(skb); | |
| 1097 | 1101 | rcu_read_unlock(); |
| 1098 | 1102 | |
| 1099 | 1103 | return 0; |
| @@ -386,11 +386,16 @@ static void tbnet_tear_down(struct tbnet *net, bool send_logout) · 385 unmodified lines | ||
| 386 | 386 | break; |
| 387 | 387 | } |
| 388 | 388 | |
| 389 | − tb_ring_stop(net->rx_ring.ring); | |
| 390 | − tb_ring_stop(net->tx_ring.ring); | |
| 391 | − tbnet_free_buffers(&net->rx_ring); | |
| 392 | − tbnet_free_buffers(&net->tx_ring); | |
| 393 | − | |
| 389 | + /* Tear the paths down before stopping the rings. This mirrors | |
| 390 | + * tbnet_connected_work(), which enables the paths last so the | |
| 391 | + * Rx ring is primed before packets can arrive. Stopping a | |
| 392 | + * ring zeroes its descriptor base and tbnet_free_buffers() | |
| 393 | + * unmaps and frees the frame buffers, leaving anything still | |
| 394 | + * in flight with nowhere to drain to; | |
| 395 | + * __tb_path_deactivate_hop() then waits for the hop's | |
| 396 | + * 'pending' bit, which on some host routers never clears in | |
| 397 | + * that state. | |
| 398 | + */ | |
| 394 | 399 | ret = tb_xdomain_disable_paths(net->xd, |
| 395 | 400 | net->local_transmit_path, |
| 396 | 401 | net->tx_ring.ring->hop, |
| @@ -399,6 +404,11 @@ static void tbnet_tear_down(struct tbnet *net, bool send_logout) · 2 unmodified lines | ||
| 399 | 404 | if (ret) |
| 400 | 405 | netdev_warn(net->dev, "failed to disable DMA paths\n"); |
| 401 | 406 | |
| 407 | + tb_ring_stop(net->rx_ring.ring); | |
| 408 | + tb_ring_stop(net->tx_ring.ring); | |
| 409 | + tbnet_free_buffers(&net->rx_ring); | |
| 410 | + tbnet_free_buffers(&net->tx_ring); | |
| 411 | + | |
| 402 | 412 | tb_xdomain_release_in_hopid(net->xd, net->remote_transmit_path); |
| 403 | 413 | net->remote_transmit_path = 0; |
| 404 | 414 | } |
| @@ -925,12 +935,8 @@ static int tbnet_open(struct net_device *dev) · 520 unmodified lines | ||
| 925 | 935 | |
| 926 | 936 | netif_carrier_off(dev); |
| 927 | 937 | |
| 928 | − flags = RING_FLAG_FRAME; | |
| 929 | − /* Only enable full E2E if the other end supports it too */ | |
| 930 | − if (tbnet_e2e && net->svc->prtcstns & TBNET_E2E) | |
| 931 | − flags |= RING_FLAG_E2E; | |
| 932 | − | |
| 933 | − ring = tb_ring_alloc_tx(xd->tb->nhi, -1, TBNET_RING_SIZE, flags); | |
| 938 | + ring = tb_ring_alloc_tx(xd->tb->nhi, -1, TBNET_RING_SIZE, | |
| 939 | + RING_FLAG_FRAME); | |
| 934 | 940 | if (!ring) { |
| 935 | 941 | netdev_err(dev, "failed to allocate Tx ring\n"); |
| 936 | 942 | return -ENOMEM; |
| @@ -949,6 +955,11 @@ static int tbnet_open(struct net_device *dev) · 12 unmodified lines | ||
| 949 | 955 | sof_mask = BIT(TBIP_PDF_FRAME_START); |
| 950 | 956 | eof_mask = BIT(TBIP_PDF_FRAME_END); |
| 951 | 957 | |
| 958 | + flags = RING_FLAG_FRAME; | |
| 959 | + /* Only enable full E2E if the other end supports it too */ | |
| 960 | + if (tbnet_e2e && net->svc->prtcstns & TBNET_E2E) | |
| 961 | + flags |= RING_FLAG_E2E; | |
| 962 | + | |
| 952 | 963 | ring = tb_ring_alloc_rx(xd->tb->nhi, -1, TBNET_RING_SIZE, flags, |
| 953 | 964 | net->tx_ring.ring->hop, sof_mask, |
| 954 | 965 | eof_mask, tbnet_start_poll, net); |
| @@ -145,8 +145,6 @@ struct tun_file { · 144 unmodified lines | ||
| 145 | 145 | struct list_head next; |
| 146 | 146 | struct tun_struct *detached; |
| 147 | 147 | struct ptr_ring tx_ring; |
| 148 | − /* Protected by tx_ring.consumer_lock */ | |
| 149 | − int cons_cnt; | |
| 150 | 148 | struct xdp_rxq_info xdp_rxq; |
| 151 | 149 | }; |
| 152 | 150 | |
| @@ -590,13 +588,8 @@ static void __tun_detach(struct tun_file *tfile, bool clean) · 437 unmodified lines | ||
| 590 | 588 | rcu_assign_pointer(tun->tfiles[index], |
| 591 | 589 | tun->tfiles[tun->numqueues - 1]); |
| 592 | 590 | ntfile = rtnl_dereference(tun->tfiles[index]); |
| 593 | − spin_lock(&ntfile->tx_ring.consumer_lock); | |
| 594 | 591 | ntfile->queue_index = index; |
| 595 | 592 | ntfile->xdp_rxq.queue_index = index; |
| 596 | − ntfile->cons_cnt = 0; | |
| 597 | − if (__ptr_ring_empty(&ntfile->tx_ring)) | |
| 598 | − netif_wake_subqueue(tun->dev, index); | |
| 599 | − spin_unlock(&ntfile->tx_ring.consumer_lock); | |
| 600 | 593 | rcu_assign_pointer(tun->tfiles[tun->numqueues - 1], |
| 601 | 594 | NULL); |
| 602 | 595 | |
| @@ -737,9 +730,6 @@ static int tun_attach(struct tun_struct *tun, struct file *file, · 134 unmodified lines | ||
| 737 | 730 | goto out; |
| 738 | 731 | } |
| 739 | 732 | |
| 740 | − spin_lock(&tfile->tx_ring.consumer_lock); | |
| 741 | − tfile->cons_cnt = 0; | |
| 742 | − spin_unlock(&tfile->tx_ring.consumer_lock); | |
| 743 | 733 | tfile->queue_index = tun->numqueues; |
| 744 | 734 | tfile->socket.sk->sk_shutdown &= ~RCV_SHUTDOWN; |
| 745 | 735 | |
| @@ -1018,7 +1008,6 @@ static netdev_tx_t tun_net_xmit(struct sk_buff *skb, struct net_device *dev) · 272 unmodified lines | ||
| 1018 | 1008 | struct netdev_queue *queue; |
| 1019 | 1009 | struct tun_file *tfile; |
| 1020 | 1010 | int len = skb->len; |
| 1021 | − int ret; | |
| 1022 | 1011 | |
| 1023 | 1012 | rcu_read_lock(); |
| 1024 | 1013 | tfile = rcu_dereference(tun->tfiles[txq]); |
| @@ -1073,33 +1062,13 @@ static netdev_tx_t tun_net_xmit(struct sk_buff *skb, struct net_device *dev) · 48 unmodified lines | ||
| 1073 | 1062 | |
| 1074 | 1063 | nf_reset_ct(skb); |
| 1075 | 1064 | |
| 1076 | − queue = netdev_get_tx_queue(dev, txq); | |
| 1077 | − | |
| 1078 | − spin_lock(&tfile->tx_ring.producer_lock); | |
| 1079 | − ret = __ptr_ring_produce(&tfile->tx_ring, skb); | |
| 1080 | − if (!qdisc_txq_has_no_queue(queue) && | |
| 1081 | − __ptr_ring_check_produce(&tfile->tx_ring) == -ENOSPC) { | |
| 1082 | − netif_tx_stop_queue(queue); | |
| 1083 | − /* Paired with smp_mb() in __tun_wake_queue() */ | |
| 1084 | − smp_mb__after_atomic(); | |
| 1085 | − if (!__ptr_ring_check_produce(&tfile->tx_ring)) | |
| 1086 | − netif_tx_wake_queue(queue); | |
| 1087 | − } | |
| 1088 | − spin_unlock(&tfile->tx_ring.producer_lock); | |
| 1089 | − | |
| 1090 | − if (ret) { | |
| 1091 | − /* This should be a rare case if a qdisc is present, but | |
| 1092 | − * can happen due to lltx. | |
| 1093 | − * Since skb_tx_timestamp(), skb_orphan(), | |
| 1094 | − * run_ebpf_filter() and pskb_trim() could have tinkered | |
| 1095 | − * with the SKB, returning NETDEV_TX_BUSY is unsafe and | |
| 1096 | − * we must drop instead. | |
| 1097 | − */ | |
| 1065 | + if (ptr_ring_produce(&tfile->tx_ring, skb)) { | |
| 1098 | 1066 | drop_reason = SKB_DROP_REASON_FULL_RING; |
| 1099 | 1067 | goto drop; |
| 1100 | 1068 | } |
| 1101 | 1069 | |
| 1102 | 1070 | /* dev->lltx requires to do our own update of trans_start */ |
| 1071 | + queue = netdev_get_tx_queue(dev, txq); | |
| 1103 | 1072 | txq_trans_cond_update(queue); |
| 1104 | 1073 | |
| 1105 | 1074 | /* Notify and wake up reader process */ |
| @@ -2147,46 +2116,13 @@ done: · 1041 unmodified lines | ||
| 2147 | 2116 | return total; |
| 2148 | 2117 | } |
| 2149 | 2118 | |
| 2150 | −/* Callers must hold ring.consumer_lock */ | |
| 2151 | −static void __tun_wake_queue(struct tun_struct *tun, | |
| 2152 | − struct tun_file *tfile, int consumed) | |
| 2153 | −{ | |
| 2154 | − struct netdev_queue *txq = netdev_get_tx_queue(tun->dev, | |
| 2155 | − tfile->queue_index); | |
| 2156 | − | |
| 2157 | − /* Paired with smp_mb__after_atomic() in tun_net_xmit() */ | |
| 2158 | − smp_mb(); | |
| 2159 | − if (netif_tx_queue_stopped(txq)) { | |
| 2160 | − tfile->cons_cnt += consumed; | |
| 2161 | − if (tfile->cons_cnt >= tfile->tx_ring.size / 2 || | |
| 2162 | − __ptr_ring_empty(&tfile->tx_ring)) { | |
| 2163 | − netif_tx_wake_queue(txq); | |
| 2164 | − tfile->cons_cnt = 0; | |
| 2165 | − } | |
| 2166 | − } | |
| 2167 | −} | |
| 2168 | − | |
| 2169 | −static void *tun_ring_consume(struct tun_struct *tun, struct tun_file *tfile) | |
| 2170 | −{ | |
| 2171 | − void *ptr; | |
| 2172 | − | |
| 2173 | − spin_lock(&tfile->tx_ring.consumer_lock); | |
| 2174 | − ptr = __ptr_ring_consume(&tfile->tx_ring); | |
| 2175 | − if (ptr) | |
| 2176 | − __tun_wake_queue(tun, tfile, 1); | |
| 2177 | − | |
| 2178 | − spin_unlock(&tfile->tx_ring.consumer_lock); | |
| 2179 | − return ptr; | |
| 2180 | −} | |
| 2181 | − | |
| 2182 | −static void *tun_ring_recv(struct tun_struct *tun, struct tun_file *tfile, | |
| 2183 | − int noblock, int *err) | |
| 2119 | +static void *tun_ring_recv(struct tun_file *tfile, int noblock, int *err) | |
| 2184 | 2120 | { |
| 2185 | 2121 | DECLARE_WAITQUEUE(wait, current); |
| 2186 | 2122 | void *ptr = NULL; |
| 2187 | 2123 | int error = 0; |
| 2188 | 2124 | |
| 2189 | − ptr = tun_ring_consume(tun, tfile); | |
| 2125 | + ptr = ptr_ring_consume(&tfile->tx_ring); | |
| 2190 | 2126 | if (ptr) |
| 2191 | 2127 | goto out; |
| 2192 | 2128 | if (noblock) { |
| @@ -2198,7 +2134,7 @@ static void *tun_ring_recv(struct tun_struct *tun, struct tun_file *tfile, · 5 unmodified lines | ||
| 2198 | 2134 | |
| 2199 | 2135 | while (1) { |
| 2200 | 2136 | set_current_state(TASK_INTERRUPTIBLE); |
| 2201 | − ptr = tun_ring_consume(tun, tfile); | |
| 2137 | + ptr = ptr_ring_consume(&tfile->tx_ring); | |
| 2202 | 2138 | if (ptr) |
| 2203 | 2139 | break; |
| 2204 | 2140 | if (signal_pending(current)) { |
| @@ -2235,7 +2171,7 @@ static ssize_t tun_do_read(struct tun_struct *tun, struct tun_file *tfile, · 30 unmodified lines | ||
| 2235 | 2171 | |
| 2236 | 2172 | if (!ptr) { |
| 2237 | 2173 | /* Read frames from ring */ |
| 2238 | − ptr = tun_ring_recv(tun, tfile, noblock, &err); | |
| 2174 | + ptr = tun_ring_recv(tfile, noblock, &err); | |
| 2239 | 2175 | if (!ptr) |
| 2240 | 2176 | return err; |
| 2241 | 2177 | } |
| @@ -3690,16 +3626,6 @@ static int tun_queue_resize(struct tun_struct *tun) · 1448 unmodified lines | ||
| 3690 | 3626 | dev->tx_queue_len, GFP_KERNEL, |
| 3691 | 3627 | tun_ptr_free); |
| 3692 | 3628 | |
| 3693 | − if (!ret) { | |
| 3694 | − for (i = 0; i < tun->numqueues; i++) { | |
| 3695 | − tfile = rtnl_dereference(tun->tfiles[i]); | |
| 3696 | − spin_lock(&tfile->tx_ring.consumer_lock); | |
| 3697 | − netif_wake_subqueue(tun->dev, tfile->queue_index); | |
| 3698 | − tfile->cons_cnt = 0; | |
| 3699 | − spin_unlock(&tfile->tx_ring.consumer_lock); | |
| 3700 | − } | |
| 3701 | − } | |
| 3702 | − | |
| 3703 | 3629 | kfree(rings); |
| 3704 | 3630 | return ret; |
| 3705 | 3631 | } |
| @@ -3808,29 +3734,6 @@ struct ptr_ring *tun_get_tx_ring(struct file *file) · 102 unmodified lines | ||
| 3808 | 3734 | } |
| 3809 | 3735 | EXPORT_SYMBOL_GPL(tun_get_tx_ring); |
| 3810 | 3736 | |
| 3811 | −/* Callers must hold ring.consumer_lock */ | |
| 3812 | −void tun_wake_queue(struct file *file, int consumed) | |
| 3813 | −{ | |
| 3814 | − struct tun_file *tfile; | |
| 3815 | − struct tun_struct *tun; | |
| 3816 | − | |
| 3817 | − if (file->f_op != &tun_fops) | |
| 3818 | − return; | |
| 3819 | − | |
| 3820 | − tfile = file->private_data; | |
| 3821 | − if (!tfile) | |
| 3822 | − return; | |
| 3823 | − | |
| 3824 | − rcu_read_lock(); | |
| 3825 | − | |
| 3826 | − tun = rcu_dereference(tfile->tun); | |
| 3827 | − if (tun) | |
| 3828 | − __tun_wake_queue(tun, tfile, consumed); | |
| 3829 | − | |
| 3830 | − rcu_read_unlock(); | |
| 3831 | −} | |
| 3832 | −EXPORT_SYMBOL_GPL(tun_wake_queue); | |
| 3833 | − | |
| 3834 | 3737 | module_init(tun_init); |
| 3835 | 3738 | module_exit(tun_cleanup); |
| 3836 | 3739 | MODULE_DESCRIPTION(DRV_DESCRIPTION); |
| @@ -1487,8 +1487,10 @@ ax88179_tx_fixup(struct usbnet *dev, struct sk_buff *skb, gfp_t flags) · 1486 unmodified lines | ||
| 1487 | 1487 | |
| 1488 | 1488 | headroom = skb_headroom(skb) - 8; |
| 1489 | 1489 | |
| 1490 | − if ((dev->net->features & NETIF_F_SG) && skb_linearize(skb)) | |
| 1490 | + if ((dev->net->features & NETIF_F_SG) && skb_linearize(skb)) { | |
| 1491 | + dev_kfree_skb_any(skb); | |
| 1491 | 1492 | return NULL; |
| 1493 | + } | |
| 1492 | 1494 | |
| 1493 | 1495 | if ((skb_header_cloned(skb) || headroom < 0) && |
| 1494 | 1496 | pskb_expand_head(skb, headroom < 0 ? 8 : 0, 0, GFP_ATOMIC)) { |
| @@ -490,6 +490,7 @@ static int ipheth_open(struct net_device *net) · 489 unmodified lines | ||
| 490 | 490 | if (retval) |
| 491 | 491 | return retval; |
| 492 | 492 | |
| 493 | + enable_delayed_work(&dev->carrier_work); | |
| 493 | 494 | schedule_delayed_work(&dev->carrier_work, IPHETH_CARRIER_CHECK_TIMEOUT); |
| 494 | 495 | return retval; |
| 495 | 496 | } |
| @@ -499,7 +500,11 @@ static int ipheth_close(struct net_device *net) · 3 unmodified lines | ||
| 499 | 500 | struct ipheth_device *dev = netdev_priv(net); |
| 500 | 501 | |
| 501 | 502 | netif_stop_queue(net); |
| 502 | − cancel_delayed_work_sync(&dev->carrier_work); | |
| 503 | + /* A TX URB can still complete with an error after this point and | |
| 504 | + * try to re-arm the carrier work. Disable it instead of cancelling | |
| 505 | + * it, so that such a schedule_delayed_work() is a no-op. | |
| 506 | + */ | |
| 507 | + disable_delayed_work_sync(&dev->carrier_work); | |
| 503 | 508 | return 0; |
| 504 | 509 | } |
| 505 | 510 | |
| @@ -629,6 +634,10 @@ static int ipheth_probe(struct usb_interface *intf, · 123 unmodified lines | ||
| 629 | 634 | } |
| 630 | 635 | |
| 631 | 636 | INIT_DELAYED_WORK(&dev->carrier_work, ipheth_carrier_check_work); |
| 637 | + /* Armed only between ipheth_open() and ipheth_close(). Start out | |
| 638 | + * disabled so the enable/disable counts balance from the first open. | |
| 639 | + */ | |
| 640 | + disable_delayed_work(&dev->carrier_work); | |
| 632 | 641 | |
| 633 | 642 | retval = ipheth_alloc_urbs(dev); |
| 634 | 643 | if (retval) { |
| @@ -1794,7 +1794,7 @@ usbnet_probe(struct usb_interface *udev, const struct usb_device_id *prod) · 1793 unmodified lines | ||
| 1794 | 1794 | */ |
| 1795 | 1795 | dev->hard_mtu = net->mtu + net->hard_header_len; |
| 1796 | 1796 | net->min_mtu = 0; |
| 1797 | − net->max_mtu = ETH_MAX_MTU; | |
| 1797 | + net->max_mtu = net->mtu; | |
| 1798 | 1798 | |
| 1799 | 1799 | net->netdev_ops = &usbnet_netdev_ops; |
| 1800 | 1800 | net->watchdog_timeo = TX_TIMEOUT_JIFFIES; |
| @@ -1804,6 +1804,7 @@ usbnet_probe(struct usb_interface *udev, const struct usb_device_id *prod) · 3 unmodified lines | ||
| 1804 | 1804 | // allow device-specific bind/init procedures |
| 1805 | 1805 | // NOTE net->name still not usable ... |
| 1806 | 1806 | if (info->bind) { |
| 1807 | + net->max_mtu = ETH_MAX_MTU; | |
| 1807 | 1808 | status = info->bind(dev, udev); |
| 1808 | 1809 | if (status < 0) |
| 1809 | 1810 | goto out1; |
| @@ -756,7 +756,7 @@ static int veth_convert_skb_to_xdp_buff(struct veth_rq *rq, · 755 unmodified lines | ||
| 756 | 756 | u32 frame_sz; |
| 757 | 757 | |
| 758 | 758 | if (skb_shared(skb) || skb_head_is_locked(skb) || |
| 759 | − skb_shinfo(skb)->nr_frags || | |
| 759 | + skb_is_nonlinear(skb) || | |
| 760 | 760 | skb_headroom(skb) < XDP_PACKET_HEADROOM) { |
| 761 | 761 | if (skb_pp_cow_data(rq->page_pool, pskb, XDP_PACKET_HEADROOM)) |
| 762 | 762 | goto drop; |
| @@ -771,7 +771,7 @@ static int veth_convert_skb_to_xdp_buff(struct veth_rq *rq, · 8 unmodified lines | ||
| 771 | 771 | xdp_prepare_buff(xdp, skb->head, skb_headroom(skb), |
| 772 | 772 | skb_headlen(skb), true); |
| 773 | 773 | |
| 774 | − if (skb_is_nonlinear(skb)) { | |
| 774 | + if (skb_shinfo(skb)->nr_frags) { | |
| 775 | 775 | skb_shinfo(skb)->xdp_frags_size = skb->data_len; |
| 776 | 776 | xdp_buff_set_frags_flag(xdp); |
| 777 | 777 | } else { |
| @@ -1850,7 +1850,7 @@ static int arp_reduce(struct net_device *dev, struct sk_buff *skb, __be32 vni) · 1849 unmodified lines | ||
| 1850 | 1850 | if (dev->flags & IFF_NOARP) |
| 1851 | 1851 | goto out; |
| 1852 | 1852 | |
| 1853 | − if (!pskb_may_pull(skb, arp_hdr_len(dev))) { | |
| 1853 | + if (!pskb_network_may_pull(skb, arp_hdr_len(dev))) { | |
| 1854 | 1854 | dev_dstats_tx_dropped(dev); |
| 1855 | 1855 | vxlan_vnifilter_count(vxlan, vni, NULL, |
| 1856 | 1856 | VXLAN_VNI_STATS_TX_DROPS, 0); |
| @@ -2111,7 +2111,7 @@ static bool route_shortcircuit(struct net_device *dev, struct sk_buff *skb) · 254 unmodified lines | ||
| 2111 | 2111 | { |
| 2112 | 2112 | struct iphdr *pip; |
| 2113 | 2113 | |
| 2114 | − if (!pskb_may_pull(skb, sizeof(struct iphdr))) | |
| 2114 | + if (!pskb_network_may_pull(skb, sizeof(struct iphdr))) | |
| 2115 | 2115 | return false; |
| 2116 | 2116 | pip = ip_hdr(skb); |
| 2117 | 2117 | n = neigh_lookup(&arp_tbl, &pip->daddr, dev); |
| @@ -2137,7 +2137,7 @@ static bool route_shortcircuit(struct net_device *dev, struct sk_buff *skb) · 19 unmodified lines | ||
| 2137 | 2137 | */ |
| 2138 | 2138 | if (!ipv6_mod_enabled()) |
| 2139 | 2139 | return false; |
| 2140 | − if (!pskb_may_pull(skb, sizeof(struct ipv6hdr))) | |
| 2140 | + if (!pskb_network_may_pull(skb, sizeof(struct ipv6hdr))) | |
| 2141 | 2141 | return false; |
| 2142 | 2142 | pip6 = ipv6_hdr(skb); |
| 2143 | 2143 | n = neigh_lookup(&nd_tbl, &pip6->daddr, dev); |
| @@ -2159,13 +2159,19 @@ static bool route_shortcircuit(struct net_device *dev, struct sk_buff *skb) · 15 unmodified lines | ||
| 2159 | 2159 | } |
| 2160 | 2160 | |
| 2161 | 2161 | if (n) { |
| 2162 | + u8 haddr[ETH_ALEN]; | |
| 2162 | 2163 | bool diff; |
| 2163 | 2164 | |
| 2164 | − diff = !ether_addr_equal(eth_hdr(skb)->h_dest, n->ha); | |
| 2165 | + neigh_ha_snapshot(haddr, n, dev); | |
| 2166 | + diff = !ether_addr_equal_unaligned(eth_hdr(skb)->h_dest, haddr); | |
| 2165 | 2167 | if (diff) { |
| 2168 | + if (skb_cow_head(skb, 0)) { | |
| 2169 | + neigh_release(n); | |
| 2170 | + return false; | |
| 2171 | + } | |
| 2166 | 2172 | memcpy(eth_hdr(skb)->h_source, eth_hdr(skb)->h_dest, |
| 2167 | 2173 | dev->addr_len); |
| 2168 | − memcpy(eth_hdr(skb)->h_dest, n->ha, dev->addr_len); | |
| 2174 | + memcpy(eth_hdr(skb)->h_dest, haddr, dev->addr_len); | |
| 2169 | 2175 | } |
| 2170 | 2176 | neigh_release(n); |
| 2171 | 2177 | return diff; |
| @@ -2757,8 +2763,8 @@ static netdev_tx_t vxlan_xmit(struct sk_buff *skb, struct net_device *dev) · 585 unmodified lines | ||
| 2757 | 2763 | return arp_reduce(dev, skb, vni); |
| 2758 | 2764 | #if IS_ENABLED(CONFIG_IPV6) |
| 2759 | 2765 | else if (ntohs(eth->h_proto) == ETH_P_IPV6 && |
| 2760 | − pskb_may_pull(skb, sizeof(struct ipv6hdr) + | |
| 2761 | − sizeof(struct nd_msg)) && | |
| 2766 | + pskb_network_may_pull(skb, sizeof(struct ipv6hdr) + | |
| 2767 | + sizeof(struct nd_msg)) && | |
| 2762 | 2768 | ipv6_hdr(skb)->nexthdr == IPPROTO_ICMPV6) { |
| 2763 | 2769 | struct nd_msg *m = (struct nd_msg *)(ipv6_hdr(skb) + 1); |
| 2764 | 2770 | |
| @@ -2796,6 +2802,7 @@ static netdev_tx_t vxlan_xmit(struct sk_buff *skb, struct net_device *dev) · 31 unmodified lines | ||
| 2796 | 2802 | (ntohs(eth->h_proto) == ETH_P_IP || |
| 2797 | 2803 | ntohs(eth->h_proto) == ETH_P_IPV6)) { |
| 2798 | 2804 | did_rsc = route_shortcircuit(dev, skb); |
| 2805 | + eth = eth_hdr(skb); | |
| 2799 | 2806 | if (did_rsc) |
| 2800 | 2807 | f = vxlan_find_mac_tx(vxlan, eth->h_dest, vni); |
| 2801 | 2808 | } |
| @@ -1631,7 +1631,7 @@ struct vxlan_mdb_entry *vxlan_mdb_entry_skb_get(struct vxlan_dev *vxlan, · 1630 unmodified lines | ||
| 1631 | 1631 | |
| 1632 | 1632 | switch (skb->protocol) { |
| 1633 | 1633 | case htons(ETH_P_IP): |
| 1634 | − if (!pskb_may_pull(skb, sizeof(struct iphdr))) | |
| 1634 | + if (!pskb_network_may_pull(skb, sizeof(struct iphdr))) | |
| 1635 | 1635 | return NULL; |
| 1636 | 1636 | group.dst.sa.sa_family = AF_INET; |
| 1637 | 1637 | group.dst.sin.sin_addr.s_addr = ip_hdr(skb)->daddr; |
| @@ -1640,7 +1640,7 @@ struct vxlan_mdb_entry *vxlan_mdb_entry_skb_get(struct vxlan_dev *vxlan, · 2 unmodified lines | ||
| 1640 | 1640 | break; |
| 1641 | 1641 | #if IS_ENABLED(CONFIG_IPV6) |
| 1642 | 1642 | case htons(ETH_P_IPV6): |
| 1643 | − if (!pskb_may_pull(skb, sizeof(struct ipv6hdr))) | |
| 1643 | + if (!pskb_network_may_pull(skb, sizeof(struct ipv6hdr))) | |
| 1644 | 1644 | return NULL; |
| 1645 | 1645 | group.dst.sa.sa_family = AF_INET6; |
| 1646 | 1646 | group.dst.sin6.sin6_addr = ipv6_hdr(skb)->daddr; |
| @@ -1544,6 +1544,8 @@ static void ath12k_core_pre_reconfigure_recovery(struct ath12k_base *ab) · 1543 unmodified lines | ||
| 1544 | 1544 | } |
| 1545 | 1545 | |
| 1546 | 1546 | wiphy_unlock(ah->hw->wiphy); |
| 1547 | + | |
| 1548 | + complete(&ah->peer_ml_id_done); | |
| 1547 | 1549 | } |
| 1548 | 1550 | |
| 1549 | 1551 | wake_up(&ab->wmi_ab.tx_credits_wq); |
| @@ -72,6 +72,7 @@ · 71 unmodified lines | ||
| 72 | 72 | |
| 73 | 73 | #define ATH12K_MAX_MLO_PEERS 256 |
| 74 | 74 | #define ATH12K_MLO_PEER_ID_INVALID 0xFFFF |
| 75 | +#define ATH12K_MLO_PEER_ID_PENDING 0xFFFE | |
| 75 | 76 | |
| 76 | 77 | #define ATH12K_INVALID_RSSI_FULL -1 |
| 77 | 78 | #define ATH12K_INVALID_RSSI_EMPTY -128 |
| @@ -793,6 +794,8 @@ struct ath12k_hw { · 715 unmodified lines | ||
| 793 | 794 | enum ath12k_hw_state state; |
| 794 | 795 | bool regd_updated; |
| 795 | 796 | bool use_6ghz_regd; |
| 797 | + bool host_alloc_ml_id; | |
| 798 | + struct completion peer_ml_id_done; | |
| 796 | 799 | |
| 797 | 800 | u8 num_radio; |
| 798 | 801 | |
| @@ -6,6 +6,7 @@ · 5 unmodified lines | ||
| 6 | 6 | |
| 7 | 7 | #include "core.h" |
| 8 | 8 | #include "peer.h" |
| 9 | +#include "dp_peer.h" | |
| 9 | 10 | #include "htc.h" |
| 10 | 11 | #include "dp_htt.h" |
| 11 | 12 | #include "debugfs_htt_stats.h" |
| @@ -575,6 +576,51 @@ exit: · 563 unmodified lines | ||
| 575 | 576 | rcu_read_unlock(); |
| 576 | 577 | } |
| 577 | 578 | |
| 579 | +static void ath12k_dp_htt_mlo_peer_map_handler(struct ath12k_base *ab, | |
| 580 | + struct sk_buff *skb) | |
| 581 | +{ | |
| 582 | + struct htt_resp_msg *resp = (struct htt_resp_msg *)skb->data; | |
| 583 | + struct htt_t2h_mlo_peer_map_event *ev = &resp->mlo_peer_map_ev; | |
| 584 | + u16 raw_peer_id, peer_id, addr_h16; | |
| 585 | + u8 peer_addr[ETH_ALEN]; | |
| 586 | + int ret; | |
| 587 | + | |
| 588 | + if (skb->len < sizeof(*ev)) { | |
| 589 | + ath12k_warn(ab, "unexpected htt mlo peer map event len %u\n", | |
| 590 | + skb->len); | |
| 591 | + return; | |
| 592 | + } | |
| 593 | + | |
| 594 | + raw_peer_id = le32_get_bits(ev->info0, | |
| 595 | + HTT_T2H_MLO_PEER_MAP_INFO0_MLO_PEER_ID); | |
| 596 | + peer_id = raw_peer_id | ATH12K_PEER_ML_ID_VALID; | |
| 597 | + | |
| 598 | + addr_h16 = le32_get_bits(ev->info1, | |
| 599 | + HTT_T2H_MLO_PEER_MAP_INFO1_MAC_ADDR_H16); | |
| 600 | + ath12k_dp_get_mac_addr(le32_to_cpu(ev->mac_addr_l32), addr_h16, | |
| 601 | + peer_addr); | |
| 602 | + | |
| 603 | + ath12k_dbg(ab, ATH12K_DBG_DP_HTT, "htt mlo peer map peer %pM id %u\n", | |
| 604 | + peer_addr, peer_id); | |
| 605 | + | |
| 606 | + /* | |
| 607 | + * Fix up the dp_peer entry created with ATH12K_MLO_PEER_ID_PENDING | |
| 608 | + * earlier; on chips with host_alloc_ml_id == false this is the only | |
| 609 | + * point at which the host learns the firmware-assigned ID. Chips | |
| 610 | + * that allocate the ID on the host also receive this event but the | |
| 611 | + * firmware-reported ID matches the host-allocated one, so there is | |
| 612 | + * nothing to fix up. | |
| 613 | + */ | |
| 614 | + if (!ab->hw_params->host_alloc_ml_id) { | |
| 615 | + ret = ath12k_dp_peer_fixup_peer_id(ab, peer_addr, | |
| 616 | + peer_id); | |
| 617 | + if (ret) | |
| 618 | + ath12k_warn(ab, | |
| 619 | + "failed to fix up peer id %u for dp peer %pM: %d\n", | |
| 620 | + peer_id, peer_addr, ret); | |
| 621 | + } | |
| 622 | +} | |
| 623 | + | |
| 578 | 624 | void ath12k_dp_htt_htc_t2h_msg_handler(struct ath12k_base *ab, |
| 579 | 625 | struct sk_buff *skb) |
| 580 | 626 | { |
| @@ -659,6 +705,9 @@ void ath12k_dp_htt_htc_t2h_msg_handler(struct ath12k_base *ab, · 78 unmodified lines | ||
| 659 | 705 | case HTT_T2H_MSG_TYPE_MLO_TIMESTAMP_OFFSET_IND: |
| 660 | 706 | ath12k_htt_mlo_offset_event_handler(ab, skb); |
| 661 | 707 | break; |
| 708 | + case HTT_T2H_MSG_TYPE_MLO_RX_PEER_MAP: | |
| 709 | + ath12k_dp_htt_mlo_peer_map_handler(ab, skb); | |
| 710 | + break; | |
| 662 | 711 | default: |
| 663 | 712 | ath12k_dbg(ab, ATH12K_DBG_DP_HTT, "dp_htt event %d not handled\n", |
| 664 | 713 | type); |
| @@ -930,6 +930,7 @@ enum htt_t2h_msg_type { · 929 unmodified lines | ||
| 930 | 930 | HTT_T2H_MSG_TYPE_EXT_STATS_CONF = 0x1c, |
| 931 | 931 | HTT_T2H_MSG_TYPE_BKPRESSURE_EVENT_IND = 0x24, |
| 932 | 932 | HTT_T2H_MSG_TYPE_MLO_TIMESTAMP_OFFSET_IND = 0x28, |
| 933 | + HTT_T2H_MSG_TYPE_MLO_RX_PEER_MAP = 0x29, | |
| 933 | 934 | HTT_T2H_MSG_TYPE_PEER_MAP3 = 0x2b, |
| 934 | 935 | HTT_T2H_MSG_TYPE_VDEV_TXRX_STATS_PERIODIC_IND = 0x2c, |
| 935 | 936 | }; |
| @@ -974,11 +975,22 @@ struct htt_t2h_peer_unmap_event { · 38 unmodified lines | ||
| 974 | 975 | __le32 info1; |
| 975 | 976 | } __packed; |
| 976 | 977 | |
| 978 | +#define HTT_T2H_MLO_PEER_MAP_INFO0_MLO_PEER_ID GENMASK(23, 8) | |
| 979 | +#define HTT_T2H_MLO_PEER_MAP_INFO1_MAC_ADDR_H16 GENMASK(15, 0) | |
| 980 | + | |
| 981 | +struct htt_t2h_mlo_peer_map_event { | |
| 982 | + __le32 info0; | |
| 983 | + __le32 mac_addr_l32; | |
| 984 | + __le32 info1; | |
| 985 | + __le32 reserved[5]; | |
| 986 | +} __packed; | |
| 987 | + | |
| 977 | 988 | struct htt_resp_msg { |
| 978 | 989 | union { |
| 979 | 990 | struct htt_t2h_version_conf_msg version_msg; |
| 980 | 991 | struct htt_t2h_peer_map_event peer_map_ev; |
| 981 | 992 | struct htt_t2h_peer_unmap_event peer_unmap_ev; |
| 993 | + struct htt_t2h_mlo_peer_map_event mlo_peer_map_ev; | |
| 982 | 994 | }; |
| 983 | 995 | } __packed; |
| 984 | 996 | |
| @@ -475,7 +475,9 @@ int ath12k_dp_peer_create(struct ath12k_dp_hw *dp_hw, u8 *addr, · 474 unmodified lines | ||
| 475 | 475 | dp_peer->is_mlo = params->is_mlo; |
| 476 | 476 | |
| 477 | 477 | /* |
| 478 | − * For MLO client, the host assigns the ML peer ID, so set peer_id in dp_peer | |
| 478 | + * For MLO client, the ML peer ID, either known or PENDING, needs to be | |
| 479 | + * initialized here since the following logic depends on it. | |
| 480 | + * | |
| 479 | 481 | * For non-MLO client, host gets link peer ID from firmware and will be |
| 480 | 482 | * assigned at the time of link peer creation |
| 481 | 483 | */ |
| @@ -491,13 +493,17 @@ int ath12k_dp_peer_create(struct ath12k_dp_hw *dp_hw, u8 *addr, · 9 unmodified lines | ||
| 491 | 493 | list_add(&dp_peer->list, &dp_hw->dp_peers_list); |
| 492 | 494 | |
| 493 | 495 | /* |
| 494 | − * For MLO client, the peer_id for ath12k_dp_peer is allocated by host | |
| 495 | − * and that peer_id is known at this point, and hence this ath12k_dp_peer | |
| 496 | − * can be added to the RCU table using the peer_id. | |
| 497 | − * For non-MLO client, this addition to RCU table shall be done at the | |
| 498 | − * time of assignment of ath12k_dp_link_peer to ath12k_dp_peer. | |
| 496 | + * For an MLO client whose ML peer ID is allocated by the host, the | |
| 497 | + * peer_id is known here and the dp_peer can be added to the RCU | |
| 498 | + * table using it. For an MLO client on chips where the firmware | |
| 499 | + * allocates the ID, peer_id is ATH12K_MLO_PEER_ID_PENDING and the | |
| 500 | + * RCU table publish is deferred to the | |
| 501 | + * HTT_T2H_MSG_TYPE_MLO_RX_PEER_MAP handler. For a non-MLO client | |
| 502 | + * the publish happens later, at the time of assignment of | |
| 503 | + * ath12k_dp_link_peer to ath12k_dp_peer. | |
| 499 | 504 | */ |
| 500 | − if (dp_peer->is_mlo) | |
| 505 | + if (dp_peer->is_mlo && | |
| 506 | + dp_peer->peer_id != ATH12K_MLO_PEER_ID_PENDING) | |
| 501 | 507 | rcu_assign_pointer(dp_hw->dp_peers[dp_peer->peer_id], dp_peer); |
| 502 | 508 | |
| 503 | 509 | spin_unlock_bh(&dp_hw->peer_lock); |
| @@ -518,7 +524,8 @@ void ath12k_dp_peer_delete(struct ath12k_dp_hw *dp_hw, u8 *addr, · 14 unmodified lines | ||
| 518 | 524 | return; |
| 519 | 525 | } |
| 520 | 526 | |
| 521 | − if (dp_peer->is_mlo) | |
| 527 | + if (dp_peer->is_mlo && | |
| 528 | + dp_peer->peer_id != ATH12K_MLO_PEER_ID_PENDING) | |
| 522 | 529 | rcu_assign_pointer(dp_hw->dp_peers[dp_peer->peer_id], NULL); |
| 523 | 530 | |
| 524 | 531 | list_del(&dp_peer->list); |
| @@ -695,3 +702,55 @@ void ath12k_dp_link_peer_reset_rx_stats(struct ath12k_dp *dp, const u8 *addr) · 170 unmodified lines | ||
| 695 | 702 | if (rx_stats) |
| 696 | 703 | memset(rx_stats, 0, sizeof(*rx_stats)); |
| 697 | 704 | } |
| 705 | + | |
| 706 | +int ath12k_dp_peer_fixup_peer_id(struct ath12k_base *ab, | |
| 707 | + const u8 *peer_addr, u16 peer_id) | |
| 708 | +{ | |
| 709 | + struct ath12k_dp_link_peer *link_peer; | |
| 710 | + struct ath12k_dp_peer *dp_peer = NULL; | |
| 711 | + struct ath12k_hw_group *ag = ab->ag; | |
| 712 | + struct ath12k_dp_hw *dp_hw = NULL; | |
| 713 | + struct ath12k_hw *ah; | |
| 714 | + int i; | |
| 715 | + | |
| 716 | + if (peer_id >= (ATH12K_PEER_ML_ID_VALID | ATH12K_MAX_MLO_PEERS)) | |
| 717 | + return -EINVAL; | |
| 718 | + | |
| 719 | + for (i = 0; i < ag->num_hw; i++) { | |
| 720 | + ah = ag->ah[i]; | |
| 721 | + if (!ah) | |
| 722 | + continue; | |
| 723 | + | |
| 724 | + spin_lock_bh(&ah->dp_hw.peer_lock); | |
| 725 | + dp_peer = ath12k_dp_peer_find_by_addr(&ah->dp_hw, | |
| 726 | + (u8 *)peer_addr); | |
| 727 | + if (dp_peer) { | |
| 728 | + dp_hw = &ah->dp_hw; | |
| 729 | + break; | |
| 730 | + } | |
| 731 | + spin_unlock_bh(&ah->dp_hw.peer_lock); | |
| 732 | + } | |
| 733 | + | |
| 734 | + if (!dp_peer) | |
| 735 | + return -ENOENT; | |
| 736 | + | |
| 737 | + /* dp_hw->peer_lock is held */ | |
| 738 | + | |
| 739 | + dp_peer->peer_id = peer_id; | |
| 740 | + rcu_assign_pointer(dp_hw->dp_peers[peer_id], dp_peer); | |
| 741 | + | |
| 742 | + for (i = 0; i < ATH12K_NUM_MAX_LINKS; i++) { | |
| 743 | + link_peer = rcu_dereference_protected(dp_peer->link_peers[i], | |
| 744 | + lockdep_is_held(&dp_hw->peer_lock)); | |
| 745 | + if (link_peer) | |
| 746 | + link_peer->ml_id = peer_id; | |
| 747 | + } | |
| 748 | + | |
| 749 | + ath12k_sta_to_ahsta(dp_peer->sta)->ml_peer_id = peer_id; | |
| 750 | + | |
| 751 | + spin_unlock_bh(&dp_hw->peer_lock); | |
| 752 | + | |
| 753 | + complete(&ah->peer_ml_id_done); | |
| 754 | + | |
| 755 | + return 0; | |
| 756 | +} | |
| @@ -181,4 +181,6 @@ struct ath12k_dp_peer *ath12k_dp_peer_find_by_peerid(struct ath12k_pdev_dp *dp_p · 180 unmodified lines | ||
| 181 | 181 | struct ath12k_dp_link_peer * |
| 182 | 182 | ath12k_dp_link_peer_find_by_peerid(struct ath12k_pdev_dp *dp_pdev, u16 peer_id); |
| 183 | 183 | void ath12k_dp_link_peer_free(struct ath12k_dp_link_peer *peer); |
| 184 | +int ath12k_dp_peer_fixup_peer_id(struct ath12k_base *ab, const u8 *peer_addr, | |
| 185 | + u16 peer_id); | |
| 184 | 186 | #endif |
| @@ -236,6 +236,8 @@ struct ath12k_hw_params { · 235 unmodified lines | ||
| 236 | 236 | u32 max_client_dbs; |
| 237 | 237 | u32 max_client_dbs_sbs; |
| 238 | 238 | } client; |
| 239 | + | |
| 240 | + bool host_alloc_ml_id; | |
| 239 | 241 | }; |
| 240 | 242 | |
| 241 | 243 | struct ath12k_hw_ops { |
| @@ -1282,13 +1282,17 @@ void ath12k_mac_dp_peer_cleanup(struct ath12k_hw *ah) · 1281 unmodified lines | ||
| 1282 | 1282 | struct ath12k_dp_peer *dp_peer, *tmp; |
| 1283 | 1283 | struct ath12k_dp_hw *dp_hw = &ah->dp_hw; |
| 1284 | 1284 | |
| 1285 | + lockdep_assert_wiphy(ah->hw->wiphy); | |
| 1286 | + | |
| 1285 | 1287 | INIT_LIST_HEAD(&peers); |
| 1286 | 1288 | |
| 1287 | 1289 | spin_lock_bh(&dp_hw->peer_lock); |
| 1288 | 1290 | list_for_each_entry_safe(dp_peer, tmp, &dp_hw->dp_peers_list, list) { |
| 1289 | 1291 | if (dp_peer->is_mlo) { |
| 1290 | − rcu_assign_pointer(dp_hw->dp_peers[dp_peer->peer_id], NULL); | |
| 1291 | − clear_bit(dp_peer->peer_id, ah->free_ml_peer_id_map); | |
| 1292 | + if (dp_peer->peer_id != ATH12K_MLO_PEER_ID_PENDING) | |
| 1293 | + rcu_assign_pointer(dp_hw->dp_peers[dp_peer->peer_id], | |
| 1294 | + NULL); | |
| 1295 | + ath12k_peer_ml_free(ah, ath12k_sta_to_ahsta(dp_peer->sta)); | |
| 1292 | 1296 | } |
| 1293 | 1297 | |
| 1294 | 1298 | list_move(&dp_peer->list, &peers); |
| @@ -3531,11 +3535,16 @@ static void ath12k_peer_assoc_h_mlo(struct ath12k_link_sta *arsta, · 2236 unmodified lines | ||
| 3531 | 3535 | struct ath12k_sta *ahsta = arsta->ahsta; |
| 3532 | 3536 | struct ath12k_link_sta *arsta_p; |
| 3533 | 3537 | struct ath12k_link_vif *arvif; |
| 3538 | + struct ath12k_hw *ah = arsta->arvif->ar->ah; | |
| 3534 | 3539 | unsigned long links; |
| 3535 | 3540 | u8 link_id; |
| 3536 | 3541 | int i; |
| 3537 | 3542 | |
| 3538 | − if (!sta->mlo || ahsta->ml_peer_id == ATH12K_MLO_PEER_ID_INVALID) | |
| 3543 | + if (!sta->mlo) | |
| 3544 | + return; | |
| 3545 | + | |
| 3546 | + if (ah->host_alloc_ml_id && | |
| 3547 | + ahsta->ml_peer_id == ATH12K_MLO_PEER_ID_INVALID) | |
| 3539 | 3548 | return; |
| 3540 | 3549 | |
| 3541 | 3550 | ml->enabled = true; |
| @@ -3543,12 +3552,25 @@ static void ath12k_peer_assoc_h_mlo(struct ath12k_link_sta *arsta, · 1 unmodified line | ||
| 3543 | 3552 | |
| 3544 | 3553 | /* For now considering the primary umac based on assoc link */ |
| 3545 | 3554 | ml->primary_umac = arsta->is_assoc_link; |
| 3546 | − ml->peer_id_valid = true; | |
| 3555 | + /* | |
| 3556 | + * Only chips that allocate the MLD peer ID on the host send a valid | |
| 3557 | + * ml_peer_id in WMI_PEER_ASSOC_CMDID. For chips where the firmware | |
| 3558 | + * picks the ID, leave peer_id_valid false to avoid unexpected issues. | |
| 3559 | + */ | |
| 3560 | + ml->peer_id_valid = ah->host_alloc_ml_id; | |
| 3547 | 3561 | ml->logical_link_idx_valid = true; |
| 3548 | 3562 | |
| 3549 | 3563 | ether_addr_copy(ml->mld_addr, sta->addr); |
| 3550 | 3564 | ml->logical_link_idx = arsta->link_idx; |
| 3551 | − ml->ml_peer_id = ahsta->ml_peer_id; | |
| 3565 | + /* | |
| 3566 | + * WMI_MLO_PEER_ASSOC_PARAMS expects the raw ML peer ID without | |
| 3567 | + * the host-side ATH12K_PEER_ML_ID_VALID bookkeeping bit. For chips | |
| 3568 | + * where the firmware allocates the ID, the field is unused (the | |
| 3569 | + * firmware always allocates regardless of the value here); send 0 | |
| 3570 | + * to make that intent explicit. | |
| 3571 | + */ | |
| 3572 | + ml->ml_peer_id = ah->host_alloc_ml_id ? | |
| 3573 | + (ahsta->ml_peer_id & ~ATH12K_PEER_ML_ID_VALID) : 0; | |
| 3552 | 3574 | ml->ieee_link_id = arsta->link_id; |
| 3553 | 3575 | ml->num_partner_links = 0; |
| 3554 | 3576 | ml->eml_cap = sta->eml_cap; |
| @@ -3595,8 +3617,6 @@ static void ath12k_peer_assoc_prepare(struct ath12k *ar, · 40 unmodified lines | ||
| 3595 | 3617 | |
| 3596 | 3618 | memset(arg, 0, sizeof(*arg)); |
| 3597 | 3619 | |
| 3598 | − reinit_completion(&ar->peer_assoc_done); | |
| 3599 | − | |
| 3600 | 3620 | arg->peer_new_assoc = !reassoc; |
| 3601 | 3621 | ath12k_peer_assoc_h_basic(ar, arvif, arsta, arg); |
| 3602 | 3622 | ath12k_peer_assoc_h_crypto(ar, arvif, arsta, arg); |
| @@ -3836,6 +3856,52 @@ static u32 ath12k_mac_ieee80211_sta_bw_to_wmi(struct ath12k *ar, · 233 unmodified lines | ||
| 3836 | 3856 | return bw; |
| 3837 | 3857 | } |
| 3838 | 3858 | |
| 3859 | +static int ath12k_mac_peer_assoc(struct ath12k *ar, | |
| 3860 | + struct ath12k_wmi_peer_assoc_arg *peer_arg) | |
| 3861 | +{ | |
| 3862 | + struct ath12k_hw *ah = ath12k_ar_to_ah(ar); | |
| 3863 | + int ret; | |
| 3864 | + | |
| 3865 | + reinit_completion(&ar->peer_assoc_done); | |
| 3866 | + reinit_completion(&ah->peer_ml_id_done); | |
| 3867 | + | |
| 3868 | + ret = ath12k_wmi_send_peer_assoc_cmd(ar, peer_arg); | |
| 3869 | + if (ret) { | |
| 3870 | + ath12k_warn(ar->ab, "failed to run peer assoc for %pM vdev %i: %d\n", | |
| 3871 | + peer_arg->peer_mac, peer_arg->vdev_id, ret); | |
| 3872 | + return ret; | |
| 3873 | + } | |
| 3874 | + | |
| 3875 | + if (!wait_for_completion_timeout(&ar->peer_assoc_done, 1 * HZ)) { | |
| 3876 | + ath12k_warn(ar->ab, "failed to get peer assoc conf event for %pM vdev %i\n", | |
| 3877 | + peer_arg->peer_mac, peer_arg->vdev_id); | |
| 3878 | + return -ETIMEDOUT; | |
| 3879 | + } | |
| 3880 | + | |
| 3881 | + /* | |
| 3882 | + * For devices where the firmware allocates the MLD peer ID, the host | |
| 3883 | + * learns the real ID only from the MLO_RX_PEER_MAP HTT event, which is | |
| 3884 | + * handled in a softirq (BH workqueue) context that cannot take the | |
| 3885 | + * wiphy lock. Block here, while still holding the wiphy lock, until | |
| 3886 | + * that event has fixed up the ID. This serialises the fixup against | |
| 3887 | + * all other wiphy-locked ml_peer_id accesses. | |
| 3888 | + * | |
| 3889 | + * The firmware sends the event only once, in response to the assoc-link | |
| 3890 | + * peer assoc, so block only for that link. | |
| 3891 | + */ | |
| 3892 | + if (!ah->host_alloc_ml_id && | |
| 3893 | + peer_arg->is_assoc && | |
| 3894 | + peer_arg->ml.enabled && | |
| 3895 | + peer_arg->ml.assoc_link && | |
| 3896 | + !wait_for_completion_timeout(&ah->peer_ml_id_done, 1 * HZ)) { | |
| 3897 | + ath12k_warn(ar->ab, "failed to get MLO peer map event for %pM vdev %i\n", | |
| 3898 | + peer_arg->peer_mac, peer_arg->vdev_id); | |
| 3899 | + return -ETIMEDOUT; | |
| 3900 | + } | |
| 3901 | + | |
| 3902 | + return 0; | |
| 3903 | +} | |
| 3904 | + | |
| 3839 | 3905 | static void ath12k_bss_assoc(struct ath12k *ar, |
| 3840 | 3906 | struct ath12k_link_vif *arvif, |
| 3841 | 3907 | struct ieee80211_bss_conf *bss_conf) |
| @@ -3916,18 +3982,10 @@ static void ath12k_bss_assoc(struct ath12k *ar, · 74 unmodified lines | ||
| 3916 | 3982 | } |
| 3917 | 3983 | |
| 3918 | 3984 | peer_arg->is_assoc = true; |
| 3919 | − ret = ath12k_wmi_send_peer_assoc_cmd(ar, peer_arg); | |
| 3920 | − if (ret) { | |
| 3921 | − ath12k_warn(ar->ab, "failed to run peer assoc for %pM vdev %i: %d\n", | |
| 3922 | − bss_conf->bssid, arvif->vdev_id, ret); | |
| 3923 | − return; | |
| 3924 | − } | |
| 3925 | 3985 | |
| 3926 | − if (!wait_for_completion_timeout(&ar->peer_assoc_done, 1 * HZ)) { | |
| 3927 | − ath12k_warn(ar->ab, "failed to get peer assoc conf event for %pM vdev %i\n", | |
| 3928 | − bss_conf->bssid, arvif->vdev_id); | |
| 3986 | + ret = ath12k_mac_peer_assoc(ar, peer_arg); | |
| 3987 | + if (ret) | |
| 3929 | 3988 | return; |
| 3930 | − } | |
| 3931 | 3989 | |
| 3932 | 3990 | ret = ath12k_setup_peer_smps(ar, arvif, bss_conf->bssid, |
| 3933 | 3991 | &link_sta->ht_cap, &link_sta->he_6ghz_capa); |
| @@ -6481,18 +6539,10 @@ static int ath12k_mac_station_assoc(struct ath12k *ar, · 2547 unmodified lines | ||
| 6481 | 6539 | } |
| 6482 | 6540 | |
| 6483 | 6541 | peer_arg->is_assoc = true; |
| 6484 | − ret = ath12k_wmi_send_peer_assoc_cmd(ar, peer_arg); | |
| 6485 | − if (ret) { | |
| 6486 | − ath12k_warn(ar->ab, "failed to run peer assoc for STA %pM vdev %i: %d\n", | |
| 6487 | − arsta->addr, arvif->vdev_id, ret); | |
| 6488 | − return ret; | |
| 6489 | − } | |
| 6490 | 6542 | |
| 6491 | − if (!wait_for_completion_timeout(&ar->peer_assoc_done, 1 * HZ)) { | |
| 6492 | − ath12k_warn(ar->ab, "failed to get peer assoc conf event for %pM vdev %i\n", | |
| 6493 | − arsta->addr, arvif->vdev_id); | |
| 6494 | − return -ETIMEDOUT; | |
| 6495 | − } | |
| 6543 | + ret = ath12k_mac_peer_assoc(ar, peer_arg); | |
| 6544 | + if (ret) | |
| 6545 | + return ret; | |
| 6496 | 6546 | |
| 6497 | 6547 | num_vht_rates = ath12k_mac_bitrate_mask_num_vht_rates(ar, band, mask); |
| 6498 | 6548 | num_he_rates = ath12k_mac_bitrate_mask_num_he_rates(ar, band, mask); |
| @@ -6841,14 +6891,8 @@ static void ath12k_sta_rc_update_wk(struct wiphy *wiphy, struct wiphy_work *wk) · 342 unmodified lines | ||
| 6841 | 6891 | peer_arg, true); |
| 6842 | 6892 | |
| 6843 | 6893 | peer_arg->is_assoc = false; |
| 6844 | − err = ath12k_wmi_send_peer_assoc_cmd(ar, peer_arg); | |
| 6845 | − if (err) | |
| 6846 | − ath12k_warn(ar->ab, "failed to run peer assoc for STA %pM vdev %i: %d\n", | |
| 6847 | − arsta->addr, arvif->vdev_id, err); | |
| 6848 | 6894 | |
| 6849 | − if (!wait_for_completion_timeout(&ar->peer_assoc_done, 1 * HZ)) | |
| 6850 | − ath12k_warn(ar->ab, "failed to get peer assoc conf event for %pM vdev %i\n", | |
| 6851 | − arsta->addr, arvif->vdev_id); | |
| 6895 | + ath12k_mac_peer_assoc(ar, peer_arg); | |
| 6852 | 6896 | } |
| 6853 | 6897 | } |
| 6854 | 6898 | } |
| @@ -7266,10 +7310,8 @@ static void ath12k_mac_ml_station_remove(struct ath12k_vif *ahvif, · 411 unmodified lines | ||
| 7266 | 7310 | ath12k_mac_free_unassign_link_sta(ah, ahsta, link_id); |
| 7267 | 7311 | } |
| 7268 | 7312 | |
| 7269 | − if (sta->mlo) { | |
| 7270 | − clear_bit(ahsta->ml_peer_id, ah->free_ml_peer_id_map); | |
| 7271 | − ahsta->ml_peer_id = ATH12K_MLO_PEER_ID_INVALID; | |
| 7272 | − } | |
| 7313 | + if (sta->mlo) | |
| 7314 | + ath12k_peer_ml_free(ah, ahsta); | |
| 7273 | 7315 | } |
| 7274 | 7316 | |
| 7275 | 7317 | static int ath12k_mac_handle_link_sta_state(struct ieee80211_hw *hw, |
| @@ -7733,15 +7775,23 @@ int ath12k_mac_op_sta_state(struct ieee80211_hw *hw, · 457 unmodified lines | ||
| 7733 | 7775 | /* ML sta */ |
| 7734 | 7776 | if (sta->mlo && !ahsta->links_map && |
| 7735 | 7777 | (hweight16(sta->valid_links) == 1)) { |
| 7736 | − ahsta->ml_peer_id = ath12k_peer_ml_alloc(ah); | |
| 7737 | − if (ahsta->ml_peer_id == ATH12K_MLO_PEER_ID_INVALID) { | |
| 7738 | − ath12k_hw_warn(ah, "unable to allocate ML peer id for sta %pM", | |
| 7739 | − sta->addr); | |
| 7740 | − goto exit; | |
| 7778 | + if (ah->host_alloc_ml_id) { | |
| 7779 | + ahsta->ml_peer_id = ath12k_peer_ml_alloc(ah); | |
| 7780 | + if (ahsta->ml_peer_id == ATH12K_MLO_PEER_ID_INVALID) { | |
| 7781 | + ath12k_hw_warn(ah, "unable to allocate ML peer id for sta %pM", | |
| 7782 | + sta->addr); | |
| 7783 | + goto exit; | |
| 7784 | + } | |
| 7785 | + } else { | |
| 7786 | + /* | |
| 7787 | + * firmware allocates the ML peer ID and notifies | |
| 7788 | + * the host via HTT_T2H_MSG_TYPE_MLO_RX_PEER_MAP | |
| 7789 | + */ | |
| 7790 | + ahsta->ml_peer_id = ATH12K_MLO_PEER_ID_PENDING; | |
| 7741 | 7791 | } |
| 7742 | 7792 | |
| 7743 | 7793 | dp_params.is_mlo = true; |
| 7744 | − dp_params.peer_id = ahsta->ml_peer_id | ATH12K_PEER_ML_ID_VALID; | |
| 7794 | + dp_params.peer_id = ahsta->ml_peer_id; | |
| 7745 | 7795 | } |
| 7746 | 7796 | |
| 7747 | 7797 | dp_params.sta = sta; |
| @@ -7878,10 +7928,8 @@ int ath12k_mac_op_sta_state(struct ieee80211_hw *hw, · 130 unmodified lines | ||
| 7878 | 7928 | peer_delete: |
| 7879 | 7929 | ath12k_dp_peer_delete(&ah->dp_hw, sta->addr, sta); |
| 7880 | 7930 | ml_peer_id_clear: |
| 7881 | − if (sta->mlo) { | |
| 7882 | − clear_bit(ahsta->ml_peer_id, ah->free_ml_peer_id_map); | |
| 7883 | − ahsta->ml_peer_id = ATH12K_MLO_PEER_ID_INVALID; | |
| 7884 | − } | |
| 7931 | + if (sta->mlo) | |
| 7932 | + ath12k_peer_ml_free(ah, ahsta); | |
| 7885 | 7933 | exit: |
| 7886 | 7934 | /* update the state if everything went well */ |
| 7887 | 7935 | if (!ret) |
| @@ -15310,6 +15358,7 @@ static struct ath12k_hw *ath12k_mac_hw_allocate(struct ath12k_hw_group *ag, · 7422 unmodified lines | ||
| 15310 | 15358 | ah->num_radio = num_pdev_map; |
| 15311 | 15359 | |
| 15312 | 15360 | mutex_init(&ah->hw_mutex); |
| 15361 | + init_completion(&ah->peer_ml_id_done); | |
| 15313 | 15362 | |
| 15314 | 15363 | spin_lock_init(&ah->dp_hw.peer_lock); |
| 15315 | 15364 | INIT_LIST_HEAD(&ah->dp_hw.dp_peers_list); |
| @@ -15384,8 +15433,9 @@ int ath12k_mac_allocate(struct ath12k_hw_group *ag) · 68 unmodified lines | ||
| 15384 | 15433 | int mac_id, device_id, total_radio, num_hw; |
| 15385 | 15434 | struct ath12k_base *ab; |
| 15386 | 15435 | struct ath12k_hw *ah; |
| 15387 | − int ret, i, j; | |
| 15436 | + bool conf = false; | |
| 15388 | 15437 | u8 radio_per_hw; |
| 15438 | + int ret, i, j; | |
| 15389 | 15439 | |
| 15390 | 15440 | total_radio = 0; |
| 15391 | 15441 | for (i = 0; i < ag->num_devices; i++) { |
| @@ -15425,6 +15475,20 @@ int ath12k_mac_allocate(struct ath12k_hw_group *ag) · 33 unmodified lines | ||
| 15425 | 15475 | } |
| 15426 | 15476 | |
| 15427 | 15477 | ab = ag->ab[device_id]; |
| 15478 | + | |
| 15479 | + /* | |
| 15480 | + * the assumption is all devices within an ah | |
| 15481 | + * share the same host_alloc_ml_id configuration | |
| 15482 | + */ | |
| 15483 | + if (j == 0) { | |
| 15484 | + conf = ab->hw_params->host_alloc_ml_id; | |
| 15485 | + } else if (conf != ab->hw_params->host_alloc_ml_id) { | |
| 15486 | + ath12k_warn(ab, "inconsistent ML ID config within ah, device 0 uses %s allocated ID, while device %u doesn't\n", | |
| 15487 | + conf ? "host" : "firmware", device_id); | |
| 15488 | + ret = -EINVAL; | |
| 15489 | + goto err; | |
| 15490 | + } | |
| 15491 | + | |
| 15428 | 15492 | pdev_map[j].ab = ab; |
| 15429 | 15493 | pdev_map[j].pdev_idx = mac_id; |
| 15430 | 15494 | mac_id++; |
| @@ -15449,6 +15513,7 @@ int ath12k_mac_allocate(struct ath12k_hw_group *ag) · 18 unmodified lines | ||
| 15449 | 15513 | } |
| 15450 | 15514 | |
| 15451 | 15515 | ah->dev = ab->dev; |
| 15516 | + ah->host_alloc_ml_id = conf; | |
| 15452 | 15517 | |
| 15453 | 15518 | ag->ah[i] = ah; |
| 15454 | 15519 | ag->num_hw++; |
| @@ -230,7 +230,16 @@ int ath12k_peer_create(struct ath12k *ar, struct ath12k_link_vif *arvif, · 229 unmodified lines | ||
| 230 | 230 | /* Fill ML info into created peer */ |
| 231 | 231 | if (sta->mlo) { |
| 232 | 232 | ml_peer_id = ahsta->ml_peer_id; |
| 233 | − peer->ml_id = ml_peer_id | ATH12K_PEER_ML_ID_VALID; | |
| 233 | + /* | |
| 234 | + * For chips where firmware allocates the ML peer ID, | |
| 235 | + * ml_peer_id is ATH12K_MLO_PEER_ID_PENDING here. The | |
| 236 | + * MLO_RX_PEER_MAP HTT event handler fixes up | |
| 237 | + * peer->ml_id once the ID is known. | |
| 238 | + */ | |
| 239 | + if (ml_peer_id == ATH12K_MLO_PEER_ID_PENDING) | |
| 240 | + peer->ml_id = ATH12K_MLO_PEER_ID_INVALID; | |
| 241 | + else | |
| 242 | + peer->ml_id = ml_peer_id; | |
| 234 | 243 | ether_addr_copy(peer->ml_addr, sta->addr); |
| 235 | 244 | |
| 236 | 245 | /* the assoc link is considered primary for now */ |
| @@ -276,9 +285,25 @@ u16 ath12k_peer_ml_alloc(struct ath12k_hw *ah) · 39 unmodified lines | ||
| 276 | 285 | } |
| 277 | 286 | |
| 278 | 287 | if (ml_peer_id == ATH12K_MAX_MLO_PEERS) |
| 279 | − ml_peer_id = ATH12K_MLO_PEER_ID_INVALID; | |
| 288 | + return ATH12K_MLO_PEER_ID_INVALID; | |
| 280 | 289 | |
| 281 | − return ml_peer_id; | |
| 290 | + return ml_peer_id | ATH12K_PEER_ML_ID_VALID; | |
| 291 | +} | |
| 292 | + | |
| 293 | +void ath12k_peer_ml_free(struct ath12k_hw *ah, struct ath12k_sta *ahsta) | |
| 294 | +{ | |
| 295 | + lockdep_assert_wiphy(ah->hw->wiphy); | |
| 296 | + | |
| 297 | + /* | |
| 298 | + * Only devices that allocate the ID on the host own a slot in | |
| 299 | + * free_ml_peer_id_map. | |
| 300 | + */ | |
| 301 | + if (ah->host_alloc_ml_id && | |
| 302 | + (ahsta->ml_peer_id < | |
| 303 | + (ATH12K_MAX_MLO_PEERS | ATH12K_PEER_ML_ID_VALID))) | |
| 304 | + clear_bit(ahsta->ml_peer_id & ~ATH12K_PEER_ML_ID_VALID, | |
| 305 | + ah->free_ml_peer_id_map); | |
| 306 | + ahsta->ml_peer_id = ATH12K_MLO_PEER_ID_INVALID; | |
| 282 | 307 | } |
| 283 | 308 | |
| 284 | 309 | int ath12k_peer_mlo_link_peers_delete(struct ath12k_vif *ahvif, struct ath12k_sta *ahsta) |
| @@ -26,4 +26,5 @@ int ath12k_link_sta_rhash_add(struct ath12k_base *ab, struct ath12k_link_sta *ar · 25 unmodified lines | ||
| 26 | 26 | struct ath12k_link_sta *ath12k_link_sta_find_by_addr(struct ath12k_base *ab, |
| 27 | 27 | const u8 *addr); |
| 28 | 28 | u16 ath12k_peer_ml_alloc(struct ath12k_hw *ah); |
| 29 | +void ath12k_peer_ml_free(struct ath12k_hw *ah, struct ath12k_sta *ahsta); | |
| 29 | 30 | #endif /* _PEER_H_ */ |
| @@ -442,6 +442,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 441 unmodified lines | ||
| 442 | 442 | .max_client_dbs = 128, |
| 443 | 443 | .max_client_dbs_sbs = 128, |
| 444 | 444 | }, |
| 445 | + | |
| 446 | + .host_alloc_ml_id = true, | |
| 445 | 447 | }, |
| 446 | 448 | { |
| 447 | 449 | .name = "wcn7850 hw2.0", |
| @@ -533,6 +535,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 85 unmodified lines | ||
| 533 | 535 | .max_client_dbs = 128, |
| 534 | 536 | .max_client_dbs_sbs = 128, |
| 535 | 537 | }, |
| 538 | + | |
| 539 | + .host_alloc_ml_id = false, | |
| 536 | 540 | }, |
| 537 | 541 | { |
| 538 | 542 | .name = "qcn9274 hw2.0", |
| @@ -620,6 +624,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 81 unmodified lines | ||
| 620 | 624 | .max_client_dbs = 128, |
| 621 | 625 | .max_client_dbs_sbs = 128, |
| 622 | 626 | }, |
| 627 | + | |
| 628 | + .host_alloc_ml_id = true, | |
| 623 | 629 | }, |
| 624 | 630 | { |
| 625 | 631 | .name = "ipq5332 hw1.0", |
| @@ -700,6 +706,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 74 unmodified lines | ||
| 700 | 706 | .max_client_dbs = 128, |
| 701 | 707 | .max_client_dbs_sbs = 128, |
| 702 | 708 | }, |
| 709 | + | |
| 710 | + .host_alloc_ml_id = true, | |
| 703 | 711 | }, |
| 704 | 712 | { |
| 705 | 713 | .name = "qcc2072 hw1.0", |
| @@ -792,6 +800,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 86 unmodified lines | ||
| 792 | 800 | .max_client_dbs = 128, |
| 793 | 801 | .max_client_dbs_sbs = 128, |
| 794 | 802 | }, |
| 803 | + | |
| 804 | + .host_alloc_ml_id = false, | |
| 795 | 805 | }, |
| 796 | 806 | { |
| 797 | 807 | .name = "ipq5424 hw1.0", |
| @@ -876,6 +886,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 78 unmodified lines | ||
| 876 | 886 | .max_client_dbs = 128, |
| 877 | 887 | .max_client_dbs_sbs = 128, |
| 878 | 888 | }, |
| 889 | + | |
| 890 | + .host_alloc_ml_id = true, | |
| 879 | 891 | }, |
| 880 | 892 | }; |
| 881 | 893 | |
| @@ -2179,8 +2179,8 @@ il_remove_station(struct il_priv *il, const u8 sta_id, const u8 * addr) · 2178 unmodified lines | ||
| 2179 | 2179 | il->stations[sta_id].used &= ~IL_STA_DRIVER_ACTIVE; |
| 2180 | 2180 | |
| 2181 | 2181 | il->num_stations--; |
| 2182 | − | |
| 2183 | − BUG_ON(il->num_stations < 0); | |
| 2182 | + if (WARN_ON(il->num_stations < 0)) | |
| 2183 | + il->num_stations = 0; | |
| 2184 | 2184 | |
| 2185 | 2185 | spin_unlock_irqrestore(&il->sta_lock, flags); |
| 2186 | 2186 | |
| @@ -2328,7 +2328,8 @@ il_dealloc_bcast_stations(struct il_priv *il) · 141 unmodified lines | ||
| 2328 | 2328 | |
| 2329 | 2329 | il->stations[i].used &= ~IL_STA_UCODE_ACTIVE; |
| 2330 | 2330 | il->num_stations--; |
| 2331 | − BUG_ON(il->num_stations < 0); | |
| 2331 | + if (WARN_ON(il->num_stations < 0)) | |
| 2332 | + il->num_stations = 0; | |
| 2332 | 2333 | kfree(il->stations[i].lq); |
| 2333 | 2334 | il->stations[i].lq = NULL; |
| 2334 | 2335 | } |
| @@ -396,7 +396,7 @@ il_dbgfs_stations_read(struct file *file, char __user *user_buf, size_t count, · 395 unmodified lines | ||
| 396 | 396 | int i, j, pos = 0; |
| 397 | 397 | ssize_t ret; |
| 398 | 398 | /* Add 30 for initial string */ |
| 399 | − const size_t bufsz = 30 + sizeof(char) * 500 * (il->num_stations); | |
| 399 | + const size_t bufsz = 30 + sizeof(char) * 500 * max_sta; | |
| 400 | 400 | |
| 401 | 401 | buf = kmalloc(bufsz, GFP_KERNEL); |
| 402 | 402 | if (!buf) |
| @@ -44,7 +44,7 @@ static int mwifiex_11n_dispatch_amsdu_pkt(struct mwifiex_private *priv, · 43 unmodified lines | ||
| 44 | 44 | ntohs(rx_hdr->eth803_hdr.h_proto) == ETH_P_TDLS) { |
| 45 | 45 | mwifiex_process_tdls_action_frame(priv, |
| 46 | 46 | (u8 *)rx_hdr, |
| 47 | − skb->len); | |
| 47 | + rx_skb->len); | |
| 48 | 48 | } |
| 49 | 49 | |
| 50 | 50 | if (priv->bss_role == MWIFIEX_BSS_ROLE_UAP) |
| @@ -18,6 +18,22 @@ static const struct regmap_config apple_spmi_regmap_config = { · 17 unmodified lines | ||
| 18 | 18 | .max_register = 0xffff, |
| 19 | 19 | }; |
| 20 | 20 | |
| 21 | +static int apple_spmi_nvmem_read(void *priv, unsigned int offset, void *val, | |
| 22 | + size_t bytes) | |
| 23 | +{ | |
| 24 | + struct regmap *map = priv; | |
| 25 | + | |
| 26 | + return regmap_bulk_read(map, offset, val, bytes); | |
| 27 | +} | |
| 28 | + | |
| 29 | +static int apple_spmi_nvmem_write(void *priv, unsigned int offset, void *val, | |
| 30 | + size_t bytes) | |
| 31 | +{ | |
| 32 | + struct regmap *map = priv; | |
| 33 | + | |
| 34 | + return regmap_bulk_write(map, offset, val, bytes); | |
| 35 | +} | |
| 36 | + | |
| 21 | 37 | static int apple_spmi_nvmem_probe(struct spmi_device *sdev) |
| 22 | 38 | { |
| 23 | 39 | struct regmap *regmap; |
| @@ -28,8 +44,8 @@ static int apple_spmi_nvmem_probe(struct spmi_device *sdev) · 4 unmodified lines | ||
| 28 | 44 | .word_size = 1, |
| 29 | 45 | .stride = 1, |
| 30 | 46 | .size = 0xffff, |
| 31 | − .reg_read = (void *)regmap_bulk_read, | |
| 32 | − .reg_write = (void *)regmap_bulk_write, | |
| 47 | + .reg_read = apple_spmi_nvmem_read, | |
| 48 | + .reg_write = apple_spmi_nvmem_write, | |
| 33 | 49 | }; |
| 34 | 50 | |
| 35 | 51 | regmap = devm_regmap_init_spmi_ext(sdev, &apple_spmi_regmap_config); |
| @@ -786,7 +786,7 @@ static int nvmem_validate_keepouts(struct nvmem_device *nvmem) · 785 unmodified lines | ||
| 786 | 786 | return 0; |
| 787 | 787 | } |
| 788 | 788 | |
| 789 | −static int nvmem_add_cells_from_dt(struct nvmem_device *nvmem, struct device_node *np) | |
| 789 | +int nvmem_add_cells_from_dt(struct nvmem_device *nvmem, struct device_node *np) | |
| 790 | 790 | { |
| 791 | 791 | struct device *dev = &nvmem->dev; |
| 792 | 792 | const __be32 *addr; |
| @@ -834,29 +834,13 @@ static int nvmem_add_cells_from_dt(struct nvmem_device *nvmem, struct device_nod · 41 unmodified lines | ||
| 834 | 834 | |
| 835 | 835 | return 0; |
| 836 | 836 | } |
| 837 | +EXPORT_SYMBOL_GPL(nvmem_add_cells_from_dt); | |
| 837 | 838 | |
| 838 | 839 | static int nvmem_add_cells_from_legacy_of(struct nvmem_device *nvmem) |
| 839 | 840 | { |
| 840 | 841 | return nvmem_add_cells_from_dt(nvmem, nvmem->dev.of_node); |
| 841 | 842 | } |
| 842 | 843 | |
| 843 | −static int nvmem_add_cells_from_fixed_layout(struct nvmem_device *nvmem) | |
| 844 | −{ | |
| 845 | − struct device_node *layout_np; | |
| 846 | − int err = 0; | |
| 847 | − | |
| 848 | − layout_np = of_nvmem_layout_get_container(nvmem); | |
| 849 | − if (!layout_np) | |
| 850 | − return 0; | |
| 851 | − | |
| 852 | − if (of_device_is_compatible(layout_np, "fixed-layout")) | |
| 853 | − err = nvmem_add_cells_from_dt(nvmem, layout_np); | |
| 854 | − | |
| 855 | − of_node_put(layout_np); | |
| 856 | − | |
| 857 | − return err; | |
| 858 | −} | |
| 859 | − | |
| 860 | 844 | int nvmem_layout_register(struct nvmem_layout *layout) |
| 861 | 845 | { |
| 862 | 846 | int ret; |
| @@ -1005,10 +989,6 @@ struct nvmem_device *nvmem_register(const struct nvmem_config *config) · 142 unmodified lines | ||
| 1005 | 989 | goto err_remove_cells; |
| 1006 | 990 | } |
| 1007 | 991 | |
| 1008 | − rval = nvmem_add_cells_from_fixed_layout(nvmem); | |
| 1009 | − if (rval) | |
| 1010 | − goto err_remove_cells; | |
| 1011 | − | |
| 1012 | 992 | dev_dbg(&nvmem->dev, "Registering nvmem device %s\n", config->name); |
| 1013 | 993 | |
| 1014 | 994 | rval = device_add(&nvmem->dev); |
| @@ -35,6 +35,8 @@ struct nvmem_device { · 34 unmodified lines | ||
| 35 | 35 | bool sysfs_cells_populated; |
| 36 | 36 | }; |
| 37 | 37 | |
| 38 | +int nvmem_add_cells_from_dt(struct nvmem_device *nvmem, struct device_node *np); | |
| 39 | + | |
| 38 | 40 | #if IS_ENABLED(CONFIG_OF) |
| 39 | 41 | int nvmem_layout_bus_register(void); |
| 40 | 42 | void nvmem_layout_bus_unregister(void); |
| @@ -125,11 +125,6 @@ static int nvmem_layout_create_device(struct nvmem_device *nvmem, · 124 unmodified lines | ||
| 125 | 125 | return 0; |
| 126 | 126 | } |
| 127 | 127 | |
| 128 | −static const struct of_device_id of_nvmem_layout_skip_table[] = { | |
| 129 | − { .compatible = "fixed-layout", }, | |
| 130 | − {} | |
| 131 | −}; | |
| 132 | − | |
| 133 | 128 | static int nvmem_layout_bus_populate(struct nvmem_device *nvmem, |
| 134 | 129 | struct device_node *layout_dn) |
| 135 | 130 | { |
| @@ -142,12 +137,6 @@ static int nvmem_layout_bus_populate(struct nvmem_device *nvmem, · 6 unmodified lines | ||
| 142 | 137 | return 0; |
| 143 | 138 | } |
| 144 | 139 | |
| 145 | − /* Fixed layouts are parsed manually somewhere else for now */ | |
| 146 | − if (of_match_node(of_nvmem_layout_skip_table, layout_dn)) { | |
| 147 | − pr_debug("%s() - skipping %pOF node\n", __func__, layout_dn); | |
| 148 | − return 0; | |
| 149 | − } | |
| 150 | − | |
| 151 | 140 | if (of_node_check_flag(layout_dn, OF_POPULATED_BUS)) { |
| 152 | 141 | pr_debug("%s() - skipping %pOF, already populated\n", |
| 153 | 142 | __func__, layout_dn); |
| @@ -3,6 +3,7 @@ · 2 unmodified lines | ||
| 3 | 3 | # Makefile for nvmem layouts. |
| 4 | 4 | # |
| 5 | 5 | |
| 6 | +obj-$(CONFIG_NVMEM_LAYOUTS) += fixed-layout.o | |
| 6 | 7 | obj-$(CONFIG_NVMEM_LAYOUT_SL28_VPD) += sl28vpd.o |
| 7 | 8 | obj-$(CONFIG_NVMEM_LAYOUT_ONIE_TLV) += onie-tlv.o |
| 8 | 9 | obj-$(CONFIG_NVMEM_LAYOUT_U_BOOT_ENV) += u-boot-env.o |
| @@ -0,0 +1,58 @@ | ||
| 1 | +// SPDX-License-Identifier: GPL-2.0-only | |
| 2 | +/* | |
| 3 | + * Copyright 2026 Bootlin | |
| 4 | + * | |
| 5 | + * Authors: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> | |
| 6 | + */ | |
| 7 | + | |
| 8 | +#include <linux/nvmem-provider.h> | |
| 9 | +#include <linux/of.h> | |
| 10 | + | |
| 11 | +#include "../internals.h" | |
| 12 | + | |
| 13 | +static int fixed_layout_add_cells(struct nvmem_layout *layout) | |
| 14 | +{ | |
| 15 | + struct device_node *np; | |
| 16 | + int ret; | |
| 17 | + | |
| 18 | + np = of_nvmem_layout_get_container(layout->nvmem); | |
| 19 | + if (!np) | |
| 20 | + return -ENOENT; | |
| 21 | + | |
| 22 | + ret = nvmem_add_cells_from_dt(layout->nvmem, np); | |
| 23 | + of_node_put(np); | |
| 24 | + | |
| 25 | + return ret; | |
| 26 | +} | |
| 27 | + | |
| 28 | +static int fixed_layout_probe(struct nvmem_layout *layout) | |
| 29 | +{ | |
| 30 | + layout->add_cells = fixed_layout_add_cells; | |
| 31 | + | |
| 32 | + return nvmem_layout_register(layout); | |
| 33 | +} | |
| 34 | + | |
| 35 | +static void fixed_layout_remove(struct nvmem_layout *layout) | |
| 36 | +{ | |
| 37 | + nvmem_layout_unregister(layout); | |
| 38 | +} | |
| 39 | + | |
| 40 | +static const struct of_device_id fixed_layout_of_match_table[] = { | |
| 41 | + { .compatible = "fixed-layout", }, | |
| 42 | + {}, | |
| 43 | +}; | |
| 44 | + | |
| 45 | +static struct nvmem_layout_driver fixed_layout_layout = { | |
| 46 | + .driver = { | |
| 47 | + .name = "fixed-layout", | |
| 48 | + .of_match_table = fixed_layout_of_match_table, | |
| 49 | + }, | |
| 50 | + .probe = fixed_layout_probe, | |
| 51 | + .remove = fixed_layout_remove, | |
| 52 | +}; | |
| 53 | +module_nvmem_layout_driver(fixed_layout_layout); | |
| 54 | + | |
| 55 | +MODULE_AUTHOR("Mathieu Dubois-Briand"); | |
| 56 | +MODULE_LICENSE("GPL"); | |
| 57 | +MODULE_DEVICE_TABLE(of, fixed_layout_of_match_table); | |
| 58 | +MODULE_DESCRIPTION("NVMEM fixed-layout driver"); | |
| @@ -753,6 +753,7 @@ EXPORT_SYMBOL(of_property_read_reg); · 752 unmodified lines | ||
| 753 | 753 | static int parser_init(struct of_pci_range_parser *parser, |
| 754 | 754 | struct device_node *node, const char *name) |
| 755 | 755 | { |
| 756 | + const __be32 *range; | |
| 756 | 757 | int rlen; |
| 757 | 758 | |
| 758 | 759 | parser->node = node; |
| @@ -761,12 +762,20 @@ static int parser_init(struct of_pci_range_parser *parser, · 2 unmodified lines | ||
| 761 | 762 | parser->ns = of_bus_n_size_cells(node); |
| 762 | 763 | parser->dma = !strcmp(name, "dma-ranges"); |
| 763 | 764 | parser->bus = of_match_bus(node); |
| 765 | + parser->range = NULL; | |
| 766 | + parser->end = NULL; | |
| 764 | 767 | |
| 765 | − parser->range = of_get_property(node, name, &rlen); | |
| 766 | − if (parser->range == NULL) | |
| 768 | + range = of_get_property(node, name, &rlen); | |
| 769 | + if (!range) | |
| 767 | 770 | return -ENOENT; |
| 768 | 771 | |
| 769 | − parser->end = parser->range + rlen / sizeof(__be32); | |
| 772 | + if (!parser->bus || | |
| 773 | + !OF_CHECK_COUNTS(parser->na, parser->ns) || | |
| 774 | + !OF_CHECK_ADDR_COUNT(parser->pna)) | |
| 775 | + return -EINVAL; | |
| 776 | + | |
| 777 | + parser->range = range; | |
| 778 | + parser->end = range + rlen / sizeof(__be32); | |
| 770 | 779 | |
| 771 | 780 | return 0; |
| 772 | 781 | } |
| @@ -792,7 +801,7 @@ struct of_pci_range *of_pci_range_parser_one(struct of_pci_range_parser *parser, · 19 unmodified lines | ||
| 792 | 801 | int na = parser->na; |
| 793 | 802 | int ns = parser->ns; |
| 794 | 803 | int np = parser->pna + na + ns; |
| 795 | − int busflag_na = parser->bus->flag_cells; | |
| 804 | + int busflag_na; | |
| 796 | 805 | |
| 797 | 806 | if (!range) |
| 798 | 807 | return NULL; |
| @@ -800,6 +809,8 @@ struct of_pci_range *of_pci_range_parser_one(struct of_pci_range_parser *parser, · 1 unmodified line | ||
| 800 | 809 | if (!parser->range || parser->range + np > parser->end) |
| 801 | 810 | return NULL; |
| 802 | 811 | |
| 812 | + busflag_na = parser->bus->flag_cells; | |
| 813 | + | |
| 803 | 814 | range->flags = parser->bus->get_flags(parser->range); |
| 804 | 815 | |
| 805 | 816 | range->bus_addr = of_read_number(parser->range + busflag_na, na - busflag_na); |
| @@ -976,8 +987,7 @@ phys_addr_t __init of_dma_get_max_cpu_address(struct device_node *np) · 170 unmodified lines | ||
| 976 | 987 | np = of_root; |
| 977 | 988 | |
| 978 | 989 | ranges = of_get_property(np, "dma-ranges", &len); |
| 979 | − if (ranges && len) { | |
| 980 | − of_dma_range_parser_init(&parser, np); | |
| 990 | + if (ranges && len && !of_dma_range_parser_init(&parser, np)) { | |
| 981 | 991 | for_each_of_range(&parser, &range) |
| 982 | 992 | if (range.cpu_addr + range.size > cpu_end) |
| 983 | 993 | cpu_end = range.cpu_addr + range.size - 1; |
| @@ -359,6 +359,7 @@ int __init fdt_scan_reserved_mem(void) · 358 unmodified lines | ||
| 359 | 359 | err = __reserved_mem_reserve_reg(child, uname); |
| 360 | 360 | if (!err) |
| 361 | 361 | count++; |
| 362 | + | |
| 362 | 363 | /* |
| 363 | 364 | * Save the nodes for the dynamically-placed regions |
| 364 | 365 | * into an array which will be used for allocation right |
| @@ -366,10 +367,17 @@ int __init fdt_scan_reserved_mem(void) · 1 unmodified line | ||
| 366 | 367 | * or marked as no-map. This is done to avoid dynamically |
| 367 | 368 | * allocating from one of the statically-placed regions. |
| 368 | 369 | */ |
| 369 | − if (err == -ENOENT && of_get_flat_dt_prop(child, "size", NULL)) { | |
| 370 | − dynamic_nodes[dynamic_nodes_cnt] = child; | |
| 371 | − dynamic_nodes_cnt++; | |
| 370 | + if (err != -ENOENT || !of_get_flat_dt_prop(child, "size", NULL)) | |
| 371 | + continue; | |
| 372 | + | |
| 373 | + if (dynamic_nodes_cnt == MAX_RESERVED_REGIONS) { | |
| 374 | + pr_err("too many defined dynamic regions, skip '%s'\n", | |
| 375 | + uname); | |
| 376 | + continue; | |
| 372 | 377 | } |
| 378 | + | |
| 379 | + dynamic_nodes[dynamic_nodes_cnt] = child; | |
| 380 | + dynamic_nodes_cnt++; | |
| 373 | 381 | } |
| 374 | 382 | for (int i = 0; i < dynamic_nodes_cnt; i++) { |
| 375 | 383 | const char *uname; |
| @@ -680,21 +680,12 @@ static int imx_pcie_attach_pd(struct device *dev) · 679 unmodified lines | ||
| 680 | 680 | |
| 681 | 681 | static int imx6q_pcie_enable_ref_clk(struct imx_pcie *imx_pcie, bool enable) |
| 682 | 682 | { |
| 683 | − if (enable) { | |
| 684 | − /* power up core phy and enable ref clock */ | |
| 685 | − regmap_clear_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, IMX6Q_GPR1_PCIE_TEST_PD); | |
| 686 | − /* | |
| 687 | − * The async reset input need ref clock to sync internally, | |
| 688 | − * when the ref clock comes after reset, internal synced | |
| 689 | − * reset time is too short, cannot meet the requirement. | |
| 690 | − * Add a ~10us delay here. | |
| 691 | − */ | |
| 692 | − usleep_range(10, 100); | |
| 693 | − regmap_set_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, IMX6Q_GPR1_PCIE_REF_CLK_EN); | |
| 694 | − } else { | |
| 695 | − regmap_clear_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, IMX6Q_GPR1_PCIE_REF_CLK_EN); | |
| 696 | − regmap_set_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, IMX6Q_GPR1_PCIE_TEST_PD); | |
| 697 | − } | |
| 683 | + if (enable) | |
| 684 | + regmap_set_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, | |
| 685 | + IMX6Q_GPR1_PCIE_REF_CLK_EN); | |
| 686 | + else | |
| 687 | + regmap_clear_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, | |
| 688 | + IMX6Q_GPR1_PCIE_REF_CLK_EN); | |
| 698 | 689 | |
| 699 | 690 | return 0; |
| 700 | 691 | } |
| @@ -825,8 +816,16 @@ static int imx6sx_pcie_core_reset(struct imx_pcie *imx_pcie, bool assert) · 124 unmodified lines | ||
| 825 | 816 | |
| 826 | 817 | static int imx6qp_pcie_core_reset(struct imx_pcie *imx_pcie, bool assert) |
| 827 | 818 | { |
| 819 | + if (assert) | |
| 820 | + regmap_set_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, | |
| 821 | + IMX6Q_GPR1_PCIE_TEST_PD); | |
| 822 | + else | |
| 823 | + regmap_clear_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, | |
| 824 | + IMX6Q_GPR1_PCIE_TEST_PD); | |
| 825 | + | |
| 828 | 826 | regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, IMX6Q_GPR1_PCIE_SW_RST, |
| 829 | 827 | assert ? IMX6Q_GPR1_PCIE_SW_RST : 0); |
| 828 | + | |
| 830 | 829 | if (!assert) |
| 831 | 830 | usleep_range(200, 500); |
| 832 | 831 | |
| @@ -835,11 +834,15 @@ static int imx6qp_pcie_core_reset(struct imx_pcie *imx_pcie, bool assert) · 2 unmodified lines | ||
| 835 | 834 | |
| 836 | 835 | static int imx6q_pcie_core_reset(struct imx_pcie *imx_pcie, bool assert) |
| 837 | 836 | { |
| 838 | − if (!assert) | |
| 839 | − return 0; | |
| 837 | + if (assert) | |
| 838 | + regmap_set_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, | |
| 839 | + IMX6Q_GPR1_PCIE_TEST_PD); | |
| 840 | + else | |
| 841 | + regmap_clear_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, | |
| 842 | + IMX6Q_GPR1_PCIE_TEST_PD); | |
| 840 | 843 | |
| 841 | − regmap_set_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, IMX6Q_GPR1_PCIE_TEST_PD); | |
| 842 | − regmap_set_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, IMX6Q_GPR1_PCIE_REF_CLK_EN); | |
| 844 | + if (!assert) | |
| 845 | + usleep_range(200, 500); | |
| 843 | 846 | |
| 844 | 847 | return 0; |
| 845 | 848 | } |
| @@ -1445,6 +1448,7 @@ static int imx_pcie_host_init(struct dw_pcie_rp *pp) · 599 unmodified lines | ||
| 1445 | 1448 | return 0; |
| 1446 | 1449 | |
| 1447 | 1450 | err_phy_off: |
| 1451 | + imx_pcie_assert_core_reset(imx_pcie); | |
| 1448 | 1452 | phy_power_off(imx_pcie->phy); |
| 1449 | 1453 | err_phy_exit: |
| 1450 | 1454 | phy_exit(imx_pcie->phy); |
| @@ -1466,6 +1470,7 @@ static void imx_pcie_host_exit(struct dw_pcie_rp *pp) · 15 unmodified lines | ||
| 1466 | 1470 | struct dw_pcie *pci = to_dw_pcie_from_pp(pp); |
| 1467 | 1471 | struct imx_pcie *imx_pcie = to_imx_pcie(pci); |
| 1468 | 1472 | |
| 1473 | + imx_pcie_assert_core_reset(imx_pcie); | |
| 1469 | 1474 | if (imx_pcie->phy) { |
| 1470 | 1475 | if (phy_power_off(imx_pcie->phy)) |
| 1471 | 1476 | dev_err(pci->dev, "unable to power off PHY\n"); |
| @@ -1995,7 +2000,8 @@ static const struct imx_pcie_drvdata drvdata[] = { · 523 unmodified lines | ||
| 1995 | 2000 | .flags = IMX_PCIE_FLAG_IMX_PHY | |
| 1996 | 2001 | IMX_PCIE_FLAG_SPEED_CHANGE_WORKAROUND | |
| 1997 | 2002 | IMX_PCIE_FLAG_BROKEN_SUSPEND | |
| 1998 | − IMX_PCIE_FLAG_SUPPORTS_SUSPEND, | |
| 2003 | + IMX_PCIE_FLAG_SUPPORTS_SUSPEND | | |
| 2004 | + IMX_PCIE_FLAG_KEEP_MSI_CAP, | |
| 1999 | 2005 | .dbi_length = 0x200, |
| 2000 | 2006 | .gpr = "fsl,imx6q-iomuxc-gpr", |
| 2001 | 2007 | .ltssm_off = IOMUXC_GPR12, |
| @@ -2011,7 +2017,8 @@ static const struct imx_pcie_drvdata drvdata[] = { · 9 unmodified lines | ||
| 2011 | 2017 | .flags = IMX_PCIE_FLAG_IMX_PHY | |
| 2012 | 2018 | IMX_PCIE_FLAG_SPEED_CHANGE_WORKAROUND | |
| 2013 | 2019 | IMX_PCIE_FLAG_SKIP_L23_READY | |
| 2014 | − IMX_PCIE_FLAG_SUPPORTS_SUSPEND, | |
| 2020 | + IMX_PCIE_FLAG_SUPPORTS_SUSPEND | | |
| 2021 | + IMX_PCIE_FLAG_KEEP_MSI_CAP, | |
| 2015 | 2022 | .gpr = "fsl,imx6q-iomuxc-gpr", |
| 2016 | 2023 | .ltssm_off = IOMUXC_GPR12, |
| 2017 | 2024 | .ltssm_mask = IMX6Q_GPR12_PCIE_CTL_2, |
| @@ -2026,7 +2033,8 @@ static const struct imx_pcie_drvdata drvdata[] = { · 8 unmodified lines | ||
| 2026 | 2033 | .flags = IMX_PCIE_FLAG_IMX_PHY | |
| 2027 | 2034 | IMX_PCIE_FLAG_SPEED_CHANGE_WORKAROUND | |
| 2028 | 2035 | IMX_PCIE_FLAG_SKIP_L23_READY | |
| 2029 | − IMX_PCIE_FLAG_SUPPORTS_SUSPEND, | |
| 2036 | + IMX_PCIE_FLAG_SUPPORTS_SUSPEND | | |
| 2037 | + IMX_PCIE_FLAG_KEEP_MSI_CAP, | |
| 2030 | 2038 | .dbi_length = 0x200, |
| 2031 | 2039 | .gpr = "fsl,imx6q-iomuxc-gpr", |
| 2032 | 2040 | .ltssm_off = IOMUXC_GPR12, |
| @@ -221,7 +221,7 @@ disable_repeater: · 220 unmodified lines | ||
| 221 | 221 | disable_vreg: |
| 222 | 222 | regulator_bulk_disable(M31_EUSB_NUM_VREGS, phy->vregs); |
| 223 | 223 | |
| 224 | − return 0; | |
| 224 | + return ret; | |
| 225 | 225 | } |
| 226 | 226 | |
| 227 | 227 | static int m31eusb2_phy_exit(struct phy *uphy) |
| @@ -452,9 +452,6 @@ static void rk_combphy_common_cfg_ssc(struct rockchip_combphy_priv *priv, unsign · 451 unmodified lines | ||
| 452 | 452 | struct device_node *np = priv->dev->of_node; |
| 453 | 453 | u32 val; |
| 454 | 454 | |
| 455 | − if (!priv->enable_ssc) | |
| 456 | − return; | |
| 457 | − | |
| 458 | 455 | /* Set SSC downward spread spectrum for PCIe and USB3 */ |
| 459 | 456 | if (priv->type == PHY_TYPE_PCIE || priv->type == PHY_TYPE_USB3) { |
| 460 | 457 | val = FIELD_PREP(RK3568_PHYREG32_SSC_MASK, RK3568_PHYREG32_SSC_DOWNWARD); |
| @@ -471,6 +468,9 @@ static void rk_combphy_common_cfg_ssc(struct rockchip_combphy_priv *priv, unsign · 10 unmodified lines | ||
| 471 | 468 | RK3568_PHYREG32); |
| 472 | 469 | } |
| 473 | 470 | |
| 471 | + if (!priv->enable_ssc) | |
| 472 | + return; | |
| 473 | + | |
| 474 | 474 | /* Enable SSC */ |
| 475 | 475 | val = readl(priv->mmio + RK3568_PHYREG8); |
| 476 | 476 | val |= RK3568_PHYREG8_SSC_EN; |
| @@ -53,7 +53,7 @@ · 52 unmodified lines | ||
| 53 | 53 | #define L0_TM_DIG_6 0x106c |
| 54 | 54 | #define L0_TM_DIS_DESCRAMBLE_DECODER 0x0f |
| 55 | 55 | #define L0_TX_DIG_61 0x00f4 |
| 56 | −#define L0_TM_DISABLE_SCRAMBLE_ENCODER 0x0f | |
| 56 | +#define L0_TM_DISABLE_SCRAMBLE_ENCODER (BIT(3) | GENMASK(1, 0)) | |
| 57 | 57 | |
| 58 | 58 | /* PLL Test Mode register parameters */ |
| 59 | 59 | #define L0_TM_PLL_DIG_37 0x2094 |
| @@ -502,11 +502,30 @@ static void xpsgtr_lane_set_protocol(struct xpsgtr_phy *gtr_phy) · 442 unmodified lines | ||
| 502 | 502 | } |
| 503 | 503 | } |
| 504 | 504 | |
| 505 | −/* Bypass (de)scrambler and 8b/10b decoder and encoder. */ | |
| 506 | −static void xpsgtr_bypass_scrambler_8b10b(struct xpsgtr_phy *gtr_phy) | |
| 505 | +/** | |
| 506 | + * xpsgtr_bypass_scrambler_8b10b - Configure scrambler/encoder behavior | |
| 507 | + * @gtr_phy: pointer to lane context | |
| 508 | + * @bypass: true to enable scrambler/encoder bypass (SATA/SGMII), | |
| 509 | + * false to disable scrambler/encoder bypass (USB3) | |
| 510 | + * | |
| 511 | + * Uses RMW to preserve reserved and unrelated register fields. | |
| 512 | + */ | |
| 513 | +static void xpsgtr_bypass_scrambler_8b10b(struct xpsgtr_phy *gtr_phy, | |
| 514 | + bool bypass) | |
| 507 | 515 | { |
| 508 | − xpsgtr_write_phy(gtr_phy, L0_TM_DIG_6, L0_TM_DIS_DESCRAMBLE_DECODER); | |
| 509 | − xpsgtr_write_phy(gtr_phy, L0_TX_DIG_61, L0_TM_DISABLE_SCRAMBLE_ENCODER); | |
| 516 | + if (bypass) { | |
| 517 | + xpsgtr_clr_set_phy(gtr_phy, L0_TM_DIG_6, | |
| 518 | + L0_TM_DIS_DESCRAMBLE_DECODER, | |
| 519 | + L0_TM_DIS_DESCRAMBLE_DECODER); | |
| 520 | + xpsgtr_clr_set_phy(gtr_phy, L0_TX_DIG_61, | |
| 521 | + L0_TM_DISABLE_SCRAMBLE_ENCODER, | |
| 522 | + L0_TM_DISABLE_SCRAMBLE_ENCODER); | |
| 523 | + } else { | |
| 524 | + xpsgtr_clr_set_phy(gtr_phy, L0_TM_DIG_6, | |
| 525 | + L0_TM_DIS_DESCRAMBLE_DECODER, 0); | |
| 526 | + xpsgtr_clr_set_phy(gtr_phy, L0_TX_DIG_61, | |
| 527 | + L0_TM_DISABLE_SCRAMBLE_ENCODER, 0); | |
| 528 | + } | |
| 510 | 529 | } |
| 511 | 530 | |
| 512 | 531 | /* DP-specific initialization. */ |
| @@ -527,7 +546,7 @@ static void xpsgtr_phy_init_sata(struct xpsgtr_phy *gtr_phy) · 14 unmodified lines | ||
| 527 | 546 | { |
| 528 | 547 | struct xpsgtr_dev *gtr_dev = gtr_phy->dev; |
| 529 | 548 | |
| 530 | − xpsgtr_bypass_scrambler_8b10b(gtr_phy); | |
| 549 | + xpsgtr_bypass_scrambler_8b10b(gtr_phy, true); | |
| 531 | 550 | |
| 532 | 551 | writel(gtr_phy->lane, gtr_dev->siou + SATA_CONTROL_OFFSET); |
| 533 | 552 | } |
| @@ -543,7 +562,7 @@ static void xpsgtr_phy_init_sgmii(struct xpsgtr_phy *gtr_phy) · 9 unmodified lines | ||
| 543 | 562 | xpsgtr_clr_set(gtr_dev, TX_PROT_BUS_WIDTH, mask, val); |
| 544 | 563 | xpsgtr_clr_set(gtr_dev, RX_PROT_BUS_WIDTH, mask, val); |
| 545 | 564 | |
| 546 | − xpsgtr_bypass_scrambler_8b10b(gtr_phy); | |
| 565 | + xpsgtr_bypass_scrambler_8b10b(gtr_phy, true); | |
| 547 | 566 | } |
| 548 | 567 | |
| 549 | 568 | /* Configure TX de-emphasis and margining for DP. */ |
| @@ -658,12 +677,13 @@ static int xpsgtr_phy_init(struct phy *phy) · 108 unmodified lines | ||
| 658 | 677 | { |
| 659 | 678 | struct xpsgtr_phy *gtr_phy = phy_get_drvdata(phy); |
| 660 | 679 | struct xpsgtr_dev *gtr_dev = gtr_phy->dev; |
| 661 | − int ret = 0; | |
| 680 | + int ret; | |
| 662 | 681 | |
| 663 | 682 | mutex_lock(>r_dev->gtr_mutex); |
| 664 | 683 | |
| 665 | 684 | /* Configure and enable the clock when peripheral phy_init call */ |
| 666 | − if (clk_prepare_enable(gtr_dev->clk[gtr_phy->refclk])) | |
| 685 | + ret = clk_prepare_enable(gtr_dev->clk[gtr_phy->refclk]); | |
| 686 | + if (ret) | |
| 667 | 687 | goto out; |
| 668 | 688 | |
| 669 | 689 | /* Skip initialization if not required. */ |
| @@ -673,7 +693,7 @@ static int xpsgtr_phy_init(struct phy *phy) · 3 unmodified lines | ||
| 673 | 693 | if (gtr_dev->tx_term_fix) { |
| 674 | 694 | ret = xpsgtr_phy_tx_term_fix(gtr_phy); |
| 675 | 695 | if (ret < 0) |
| 676 | − goto out; | |
| 696 | + goto out_disable_clk; | |
| 677 | 697 | |
| 678 | 698 | gtr_dev->tx_term_fix = false; |
| 679 | 699 | } |
| @@ -687,7 +707,7 @@ static int xpsgtr_phy_init(struct phy *phy) · 7 unmodified lines | ||
| 687 | 707 | */ |
| 688 | 708 | ret = xpsgtr_configure_pll(gtr_phy); |
| 689 | 709 | if (ret) |
| 690 | − goto out; | |
| 710 | + goto out_disable_clk; | |
| 691 | 711 | |
| 692 | 712 | xpsgtr_lane_set_protocol(gtr_phy); |
| 693 | 713 | |
| @@ -703,8 +723,16 @@ static int xpsgtr_phy_init(struct phy *phy) · 9 unmodified lines | ||
| 703 | 723 | case ICM_PROTOCOL_SGMII: |
| 704 | 724 | xpsgtr_phy_init_sgmii(gtr_phy); |
| 705 | 725 | break; |
| 726 | + | |
| 727 | + case ICM_PROTOCOL_USB: | |
| 728 | + xpsgtr_bypass_scrambler_8b10b(gtr_phy, false); | |
| 729 | + break; | |
| 706 | 730 | } |
| 707 | 731 | |
| 732 | + goto out; | |
| 733 | + | |
| 734 | +out_disable_clk: | |
| 735 | + clk_disable_unprepare(gtr_dev->clk[gtr_phy->refclk]); | |
| 708 | 736 | out: |
| 709 | 737 | mutex_unlock(>r_dev->gtr_mutex); |
| 710 | 738 | return ret; |
| @@ -1039,6 +1067,12 @@ static int xpsgtr_probe(struct platform_device *pdev) · 328 unmodified lines | ||
| 1039 | 1067 | return PTR_ERR(provider); |
| 1040 | 1068 | } |
| 1041 | 1069 | |
| 1070 | + gtr_dev->saved_regs = devm_kmalloc(gtr_dev->dev, | |
| 1071 | + sizeof(save_reg_address), | |
| 1072 | + GFP_KERNEL); | |
| 1073 | + if (!gtr_dev->saved_regs) | |
| 1074 | + return -ENOMEM; | |
| 1075 | + | |
| 1042 | 1076 | pm_runtime_set_active(gtr_dev->dev); |
| 1043 | 1077 | pm_runtime_enable(gtr_dev->dev); |
| 1044 | 1078 | |
| @@ -1048,12 +1082,6 @@ static int xpsgtr_probe(struct platform_device *pdev) · 3 unmodified lines | ||
| 1048 | 1082 | return ret; |
| 1049 | 1083 | } |
| 1050 | 1084 | |
| 1051 | − gtr_dev->saved_regs = devm_kmalloc(gtr_dev->dev, | |
| 1052 | − sizeof(save_reg_address), | |
| 1053 | − GFP_KERNEL); | |
| 1054 | − if (!gtr_dev->saved_regs) | |
| 1055 | − return -ENOMEM; | |
| 1056 | − | |
| 1057 | 1085 | return 0; |
| 1058 | 1086 | } |
| 1059 | 1087 | |
| @@ -166,6 +166,7 @@ config PINCTRL_BM1880 · 165 unmodified lines | ||
| 166 | 166 | depends on OF && (ARCH_BITMAIN || COMPILE_TEST) |
| 167 | 167 | default ARCH_BITMAIN |
| 168 | 168 | select PINMUX |
| 169 | + select GENERIC_PINCONF | |
| 169 | 170 | help |
| 170 | 171 | Pinctrl driver for Bitmain BM1880 SoC. |
| 171 | 172 | |
| @@ -426,6 +427,7 @@ config PINCTRL_MICROCHIP_SGPIO · 254 unmodified lines | ||
| 426 | 427 | select GENERIC_PINCONF |
| 427 | 428 | select GENERIC_PINCTRL_GROUPS |
| 428 | 429 | select GENERIC_PINMUX_FUNCTIONS |
| 430 | + select REGMAP_MMIO | |
| 429 | 431 | help |
| 430 | 432 | Support for the serial GPIO interface used on Microsemi and |
| 431 | 433 | Microchip SoCs. By using a serial interface, the SIO |
| @@ -69,6 +69,10 @@ static int dt_remember_or_free_map(struct pinctrl *p, const char *statename, · 68 unmodified lines | ||
| 69 | 69 | int i; |
| 70 | 70 | struct pinctrl_dt_map *dt_map; |
| 71 | 71 | |
| 72 | + /* Initialize dev_name before any allocation can fail */ | |
| 73 | + for (i = 0; i < num_maps; i++) | |
| 74 | + map[i].dev_name = NULL; | |
| 75 | + | |
| 72 | 76 | /* Initialize common mapping table entry fields */ |
| 73 | 77 | for (i = 0; i < num_maps; i++) { |
| 74 | 78 | const char *devname; |
| @@ -884,8 +884,7 @@ static void amd_gpio_irq_init(struct amd_gpio *gpio_dev) · 883 unmodified lines | ||
| 884 | 884 | u32 pin_reg, mask; |
| 885 | 885 | int i; |
| 886 | 886 | |
| 887 | − mask = BIT(WAKE_CNTRL_OFF_S0I3) | BIT(WAKE_CNTRL_OFF_S3) | | |
| 888 | − BIT(WAKE_CNTRL_OFF_S4); | |
| 887 | + mask = BIT(WAKE_CNTRL_OFF_S0I3) | BIT(WAKE_CNTRL_OFF_S3); | |
| 889 | 888 | |
| 890 | 889 | for (i = 0; i < desc->npins; i++) { |
| 891 | 890 | int pin = desc->pins[i].number; |
| @@ -480,7 +480,7 @@ static const char * const ps_hold_groups[] = { · 479 unmodified lines | ||
| 480 | 480 | }; |
| 481 | 481 | |
| 482 | 482 | static const struct pinfunction ipq8064_functions[] = { |
| 483 | − IPQ_PIN_FUNCTION(gpio), | |
| 483 | + IPQ_GPIO_PIN_FUNCTION(gpio), | |
| 484 | 484 | IPQ_PIN_FUNCTION(mdio), |
| 485 | 485 | IPQ_PIN_FUNCTION(ssbi), |
| 486 | 486 | IPQ_PIN_FUNCTION(spmi), |
| @@ -507,19 +507,19 @@ static const struct pinfunction ipq8064_functions[] = { · 20 unmodified lines | ||
| 507 | 507 | IPQ_PIN_FUNCTION(usb2_hsic), |
| 508 | 508 | IPQ_PIN_FUNCTION(rgmii2), |
| 509 | 509 | IPQ_PIN_FUNCTION(sata), |
| 510 | − IPQ_PIN_FUNCTION(pcie1_rst), | |
| 510 | + IPQ_GPIO_PIN_FUNCTION(pcie1_rst), | |
| 511 | 511 | IPQ_PIN_FUNCTION(pcie1_prsnt), |
| 512 | 512 | IPQ_PIN_FUNCTION(pcie1_pwren_n), |
| 513 | 513 | IPQ_PIN_FUNCTION(pcie1_pwren), |
| 514 | 514 | IPQ_PIN_FUNCTION(pcie1_pwrflt), |
| 515 | 515 | IPQ_PIN_FUNCTION(pcie1_clk_req), |
| 516 | − IPQ_PIN_FUNCTION(pcie2_rst), | |
| 516 | + IPQ_GPIO_PIN_FUNCTION(pcie2_rst), | |
| 517 | 517 | IPQ_PIN_FUNCTION(pcie2_prsnt), |
| 518 | 518 | IPQ_PIN_FUNCTION(pcie2_pwren_n), |
| 519 | 519 | IPQ_PIN_FUNCTION(pcie2_pwren), |
| 520 | 520 | IPQ_PIN_FUNCTION(pcie2_pwrflt), |
| 521 | 521 | IPQ_PIN_FUNCTION(pcie2_clk_req), |
| 522 | − IPQ_PIN_FUNCTION(pcie3_rst), | |
| 522 | + IPQ_GPIO_PIN_FUNCTION(pcie3_rst), | |
| 523 | 523 | IPQ_PIN_FUNCTION(pcie3_prsnt), |
| 524 | 524 | IPQ_PIN_FUNCTION(pcie3_pwren_n), |
| 525 | 525 | IPQ_PIN_FUNCTION(pcie3_pwren), |
| @@ -310,11 +310,11 @@ static const char *const audio_sec_mclk_out0_groups[] = { · 309 unmodified lines | ||
| 310 | 310 | }; |
| 311 | 311 | |
| 312 | 312 | static const char *const audio_sec_mclk_in1_groups[] = { |
| 313 | − "gpio37", | |
| 313 | + "gpio39", | |
| 314 | 314 | }; |
| 315 | 315 | |
| 316 | 316 | static const char *const audio_sec_mclk_out1_groups[] = { |
| 317 | − "gpio37", | |
| 317 | + "gpio39", | |
| 318 | 318 | }; |
| 319 | 319 | |
| 320 | 320 | static const char *const audio_sec_groups[] = { |
| @@ -1240,12 +1240,12 @@ static int msm_gpio_irq_reqres(struct irq_data *d) · 1239 unmodified lines | ||
| 1240 | 1240 | /* |
| 1241 | 1241 | * If the wakeup_enable bit is present and marked as available for the |
| 1242 | 1242 | * requested GPIO, it should be enabled when the GPIO is marked as |
| 1243 | − * wake irq in order to allow the interrupt event to be transfered to | |
| 1244 | − * the PDC HW. | |
| 1243 | + * wake irq in order to allow the interrupt event to be transferred to | |
| 1244 | + * the PDC/MPM HW. | |
| 1245 | 1245 | * While the name implies only the wakeup event, it's also required for |
| 1246 | 1246 | * the interrupt event. |
| 1247 | 1247 | */ |
| 1248 | − if (test_bit(d->hwirq, pctrl->skip_wake_irqs) && g->intr_wakeup_present_bit) { | |
| 1248 | + if (g->intr_wakeup_present_bit) { | |
| 1249 | 1249 | u32 intr_cfg; |
| 1250 | 1250 | |
| 1251 | 1251 | raw_spin_lock_irqsave(&pctrl->lock, flags); |
| @@ -1273,7 +1273,7 @@ static void msm_gpio_irq_relres(struct irq_data *d) · 21 unmodified lines | ||
| 1273 | 1273 | unsigned long flags; |
| 1274 | 1274 | |
| 1275 | 1275 | /* Disable the wakeup_enable bit if it has been set in msm_gpio_irq_reqres() */ |
| 1276 | − if (test_bit(d->hwirq, pctrl->skip_wake_irqs) && g->intr_wakeup_present_bit) { | |
| 1276 | + if (g->intr_wakeup_present_bit) { | |
| 1277 | 1277 | u32 intr_cfg; |
| 1278 | 1278 | |
| 1279 | 1279 | raw_spin_lock_irqsave(&pctrl->lock, flags); |
| @@ -24,6 +24,11 @@ struct pinctrl_pin_desc; · 23 unmodified lines | ||
| 24 | 24 | fname##_groups, \ |
| 25 | 25 | ARRAY_SIZE(fname##_groups)) |
| 26 | 26 | |
| 27 | +#define IPQ_GPIO_PIN_FUNCTION(fname) \ | |
| 28 | + [IPQ_MUX_##fname] = PINCTRL_GPIO_PINFUNCTION(#fname, \ | |
| 29 | + fname##_groups, \ | |
| 30 | + ARRAY_SIZE(fname##_groups)) | |
| 31 | + | |
| 27 | 32 | #define MSM_PIN_FUNCTION(fname) \ |
| 28 | 33 | [msm_mux_##fname] = PINCTRL_PINFUNCTION(#fname, \ |
| 29 | 34 | fname##_groups, \ |
| @@ -1881,16 +1881,17 @@ static const struct msm_gpio_wakeirq_map sc8280xp_pdc_map[] = { · 1880 unmodified lines | ||
| 1881 | 1881 | { 126, 200 }, { 127, 225 }, { 128, 262 }, { 129, 201 }, |
| 1882 | 1882 | { 130, 209 }, { 131, 173 }, { 132, 202 }, { 136, 210 }, |
| 1883 | 1883 | { 138, 171 }, { 139, 226 }, { 140, 227 }, { 142, 228 }, |
| 1884 | − { 144, 229 }, { 145, 230 }, { 146, 231 }, { 148, 232 }, | |
| 1885 | − { 149, 233 }, { 150, 234 }, { 152, 235 }, { 154, 212 }, | |
| 1886 | − { 157, 213 }, { 161, 219 }, { 170, 236 }, { 171, 221 }, | |
| 1887 | − { 174, 222 }, { 175, 237 }, { 176, 223 }, { 177, 170 }, | |
| 1888 | − { 180, 238 }, { 181, 239 }, { 182, 240 }, { 183, 241 }, | |
| 1889 | − { 184, 242 }, { 185, 243 }, { 190, 178 }, { 193, 184 }, | |
| 1890 | − { 196, 185 }, { 198, 186 }, { 200, 174 }, { 201, 175 }, | |
| 1891 | − { 205, 176 }, { 206, 177 }, { 208, 187 }, { 210, 198 }, | |
| 1892 | − { 211, 199 }, { 212, 204 }, { 215, 205 }, { 220, 188 }, | |
| 1893 | − { 221, 194 }, { 223, 195 }, { 225, 196 }, { 227, 197 }, | |
| 1884 | + { 143, 261 }, { 144, 229 }, { 145, 230 }, { 146, 231 }, | |
| 1885 | + { 148, 232 }, { 149, 233 }, { 150, 234 }, { 151, 264 }, | |
| 1886 | + { 152, 235 }, { 154, 212 }, { 157, 213 }, { 161, 219 }, | |
| 1887 | + { 170, 236 }, { 171, 221 }, { 174, 222 }, { 175, 237 }, | |
| 1888 | + { 176, 223 }, { 177, 170 }, { 180, 238 }, { 181, 239 }, | |
| 1889 | + { 182, 240 }, { 183, 241 }, { 184, 242 }, { 185, 243 }, | |
| 1890 | + { 190, 178 }, { 193, 184 }, { 196, 185 }, { 198, 186 }, | |
| 1891 | + { 200, 174 }, { 201, 175 }, { 205, 176 }, { 206, 177 }, | |
| 1892 | + { 208, 187 }, { 210, 198 }, { 211, 199 }, { 212, 204 }, | |
| 1893 | + { 215, 205 }, { 220, 188 }, { 221, 194 }, { 223, 195 }, | |
| 1894 | + { 225, 196 }, { 227, 197 }, | |
| 1894 | 1895 | }; |
| 1895 | 1896 | |
| 1896 | 1897 | static struct msm_pinctrl_soc_data sc8280xp_pinctrl = { |
| @@ -13,7 +13,7 @@ · 12 unmodified lines | ||
| 13 | 13 | #include <linux/types.h> |
| 14 | 14 | #include <linux/uuid.h> |
| 15 | 15 | |
| 16 | −#define PALC_DSM_FN_TRIGGER_PLDR BIT(1) | |
| 16 | +#define PALC_DSM_FN_TRIGGER_PLDR 1 | |
| 17 | 17 | |
| 18 | 18 | static guid_t palc_dsm_guid = |
| 19 | 19 | GUID_INIT(0x5a1a4bba, 0x8006, 0x487e, 0xbe, 0x0a, 0xac, 0xf5, 0xd8, 0xfd, 0xfe, 0x59); |
| @@ -66,7 +66,7 @@ static int palc_probe(struct platform_device *pdev) · 46 unmodified lines | ||
| 66 | 66 | if (!handle) |
| 67 | 67 | return -ENODEV; |
| 68 | 68 | |
| 69 | − if (!acpi_check_dsm(handle, &palc_dsm_guid, 1, PALC_DSM_FN_TRIGGER_PLDR)) | |
| 69 | + if (!acpi_check_dsm(handle, &palc_dsm_guid, 1, BIT(PALC_DSM_FN_TRIGGER_PLDR))) | |
| 70 | 70 | return -ENODEV; |
| 71 | 71 | |
| 72 | 72 | return 0; |
| @@ -320,7 +320,7 @@ static const u32 bq25890_tspct_tbl[] = { · 319 unmodified lines | ||
| 320 | 320 | 145, 140, 130, 120, 115, 110, 100, 90, |
| 321 | 321 | 80, 70, 60, 50, 40, 30, 20, 10, |
| 322 | 322 | 0, -10, -20, -30, -40, -60, -70, -80, |
| 323 | − -90, -10, -120, -140, -150, -170, -190, -210, | |
| 323 | + -90, -100, -120, -140, -150, -170, -190, -210, | |
| 324 | 324 | }; |
| 325 | 325 | |
| 326 | 326 | #define BQ25890_TSPCT_TBL_SIZE ARRAY_SIZE(bq25890_tspct_tbl) |
| @@ -86,6 +86,11 @@ struct macsmc_power { · 85 unmodified lines | ||
| 86 | 86 | bool has_ch0i; /* Force discharge (Older firmware) */ |
| 87 | 87 | bool has_ch0c; /* Inhibit charge (Older firmware) */ |
| 88 | 88 | bool has_chte; /* Inhibit charge (Modern firmware) */ |
| 89 | + /* | |
| 90 | + * Battery critical key is 1 byte and charge key is little endian | |
| 91 | + * (Modern firmware) | |
| 92 | + */ | |
| 93 | + bool fw_ge_27; | |
| 89 | 94 | |
| 90 | 95 | u8 num_cells; |
| 91 | 96 | int nominal_voltage_mv; |
| @@ -273,6 +278,20 @@ static int macsmc_battery_get_date(const char *s, int *out) · 181 unmodified lines | ||
| 273 | 278 | return 0; |
| 274 | 279 | } |
| 275 | 280 | |
| 281 | +static int macsmc_battery_read_bcf0(struct macsmc_power *power, u32 *val) | |
| 282 | +{ | |
| 283 | + u8 tval = 0; | |
| 284 | + int ret; | |
| 285 | + | |
| 286 | + if (power->fw_ge_27) { | |
| 287 | + ret = apple_smc_read_u8(power->smc, SMC_KEY(BCF0), &tval); | |
| 288 | + *val = tval; | |
| 289 | + return ret; | |
| 290 | + } | |
| 291 | + | |
| 292 | + return apple_smc_read_u32(power->smc, SMC_KEY(BCF0), val); | |
| 293 | +} | |
| 294 | + | |
| 276 | 295 | static int macsmc_battery_get_capacity_level(struct macsmc_power *power) |
| 277 | 296 | { |
| 278 | 297 | bool flag; |
| @@ -280,7 +299,7 @@ static int macsmc_battery_get_capacity_level(struct macsmc_power *power) · 1 unmodified line | ||
| 280 | 299 | int ret; |
| 281 | 300 | |
| 282 | 301 | /* Check for emergency shutdown condition */ |
| 283 | − if (apple_smc_read_u32(power->smc, SMC_KEY(BCF0), &val) >= 0 && val) | |
| 302 | + if (macsmc_battery_read_bcf0(power, &val) >= 0 && val) | |
| 284 | 303 | return POWER_SUPPLY_CAPACITY_LEVEL_CRITICAL; |
| 285 | 304 | |
| 286 | 305 | /* Check AC status for whether we could boot in this state */ |
| @@ -303,6 +322,12 @@ static int macsmc_battery_get_capacity_level(struct macsmc_power *power) · 16 unmodified lines | ||
| 303 | 322 | return POWER_SUPPLY_CAPACITY_LEVEL_NORMAL; |
| 304 | 323 | } |
| 305 | 324 | |
| 325 | +static s16 macsmc_swap_b0rm(struct macsmc_power *power, s16 b0rm) | |
| 326 | +{ | |
| 327 | + /* B0RM was Big Endian, likely pass through from TI gas gauge */ | |
| 328 | + return power->fw_ge_27 ? b0rm : (s16)swab16(b0rm); | |
| 329 | +} | |
| 330 | + | |
| 306 | 331 | static int macsmc_battery_get_property(struct power_supply *psy, |
| 307 | 332 | enum power_supply_property psp, |
| 308 | 333 | union power_supply_propval *val) |
| @@ -397,8 +422,7 @@ static int macsmc_battery_get_property(struct power_supply *psy, · 88 unmodified lines | ||
| 397 | 422 | break; |
| 398 | 423 | case POWER_SUPPLY_PROP_CHARGE_NOW: |
| 399 | 424 | ret = apple_smc_read_u16(power->smc, SMC_KEY(B0RM), &vu16); |
| 400 | − /* B0RM is Big Endian, likely pass through from TI gas gauge */ | |
| 401 | − val->intval = (s16)swab16(vu16) * 1000; | |
| 425 | + val->intval = macsmc_swap_b0rm(power, vu16) * 1000; | |
| 402 | 426 | break; |
| 403 | 427 | case POWER_SUPPLY_PROP_ENERGY_FULL_DESIGN: |
| 404 | 428 | ret = apple_smc_read_u16(power->smc, SMC_KEY(B0DC), &vu16); |
| @@ -410,8 +434,7 @@ static int macsmc_battery_get_property(struct power_supply *psy, · 5 unmodified lines | ||
| 410 | 434 | break; |
| 411 | 435 | case POWER_SUPPLY_PROP_ENERGY_NOW: |
| 412 | 436 | ret = apple_smc_read_u16(power->smc, SMC_KEY(B0RM), &vu16); |
| 413 | − /* B0RM is Big Endian, likely pass through from TI gas gauge */ | |
| 414 | − val->intval = (s16)swab16(vu16) * power->nominal_voltage_mv; | |
| 437 | + val->intval = macsmc_swap_b0rm(power, vu16) * power->nominal_voltage_mv; | |
| 415 | 438 | break; |
| 416 | 439 | case POWER_SUPPLY_PROP_TEMP: |
| 417 | 440 | ret = apple_smc_read_u16(power->smc, SMC_KEY(B0AT), &vu16); |
| @@ -577,7 +600,7 @@ static void macsmc_power_critical_work(struct work_struct *wrk) · 159 unmodified lines | ||
| 577 | 600 | * Check if SMC flagged the battery as empty. |
| 578 | 601 | * We trigger a graceful shutdown to let the OS save data. |
| 579 | 602 | */ |
| 580 | − if (apple_smc_read_u32(power->smc, SMC_KEY(BCF0), &bcf0) == 0 && bcf0 != 0) { | |
| 603 | + if (macsmc_battery_read_bcf0(power, &bcf0) == 0 && bcf0 != 0) { | |
| 581 | 604 | power->orderly_shutdown_triggered = true; |
| 582 | 605 | dev_crit(power->dev, "Battery critical (empty flag set). Triggering orderly shutdown.\n"); |
| 583 | 606 | orderly_poweroff(true); |
| @@ -616,6 +639,7 @@ static int macsmc_power_probe(struct platform_device *pdev) · 32 unmodified lines | ||
| 616 | 639 | struct device *dev = &pdev->dev; |
| 617 | 640 | struct apple_smc *smc = dev_get_drvdata(pdev->dev.parent); |
| 618 | 641 | struct power_supply_config psy_cfg = {}; |
| 642 | + struct apple_smc_key_info info; | |
| 619 | 643 | struct macsmc_power *power; |
| 620 | 644 | bool has_battery = false; |
| 621 | 645 | bool has_ac_adapter = false; |
| @@ -714,6 +738,20 @@ static int macsmc_power_probe(struct platform_device *pdev) · 92 unmodified lines | ||
| 714 | 738 | if (apple_smc_key_exists(smc, SMC_KEY(CH0I))) |
| 715 | 739 | power->has_ch0i = true; |
| 716 | 740 | |
| 741 | + ret = apple_smc_get_key_info(power->smc, SMC_KEY(BCF0), &info); | |
| 742 | + if (ret) { | |
| 743 | + dev_err(&pdev->dev, "Failed to determine BCF0 key size\n"); | |
| 744 | + return ret; | |
| 745 | + } | |
| 746 | + if (info.size == 1) | |
| 747 | + power->fw_ge_27 = true; | |
| 748 | + else if (info.size == 4) | |
| 749 | + power->fw_ge_27 = false; | |
| 750 | + else { | |
| 751 | + dev_err(&pdev->dev, "Unexpected BCF0 key size %d\n", info.size); | |
| 752 | + return -EIO; | |
| 753 | + } | |
| 754 | + | |
| 717 | 755 | /* Reset "Optimised Battery Charging" flags to default state */ |
| 718 | 756 | if (power->has_chte) |
| 719 | 757 | apple_smc_write_u32(smc, SMC_KEY(CHTE), 0); |
| @@ -766,7 +804,7 @@ static int macsmc_power_probe(struct platform_device *pdev) · 46 unmodified lines | ||
| 766 | 804 | power->nominal_voltage_mv = MACSMC_NOMINAL_CELL_VOLTAGE_MV * power->num_cells; |
| 767 | 805 | |
| 768 | 806 | /* Enable critical shutdown notifications by reading status once */ |
| 769 | − apple_smc_read_u32(power->smc, SMC_KEY(BCF0), &val32); | |
| 807 | + macsmc_battery_read_bcf0(power, &val32); | |
| 770 | 808 | |
| 771 | 809 | psy_cfg.drv_data = power; |
| 772 | 810 | power->batt = devm_power_supply_register(dev, &power->batt_desc, &psy_cfg); |
| @@ -405,7 +405,11 @@ static int max17040_get_property(struct power_supply *psy, · 404 unmodified lines | ||
| 405 | 405 | val->intval = chip->low_soc_alert; |
| 406 | 406 | break; |
| 407 | 407 | case POWER_SUPPLY_PROP_STATUS: |
| 408 | − power_supply_get_property_from_supplier(psy, psp, val); | |
| 408 | + ret = power_supply_get_property_from_supplier(psy, psp, val); | |
| 409 | + if (ret == -ENODEV) | |
| 410 | + val->intval = POWER_SUPPLY_STATUS_UNKNOWN; | |
| 411 | + else if (ret) | |
| 412 | + return ret; | |
| 409 | 413 | break; |
| 410 | 414 | case POWER_SUPPLY_PROP_TEMP: |
| 411 | 415 | if (!chip->channel_temp) |
| @@ -769,6 +769,7 @@ static void netc_timer_init(struct netc_timer *priv) · 768 unmodified lines | ||
| 769 | 769 | TMR_CTRL_TE | TMR_CTRL_FS; |
| 770 | 770 | netc_timer_wr(priv, NETC_TMR_CTRL, tmr_ctrl); |
| 771 | 771 | netc_timer_wr(priv, NETC_TMR_PRSC, priv->oclk_prsc); |
| 772 | + netc_timer_wr(priv, NETC_TMR_TEMASK, 0); | |
| 772 | 773 | |
| 773 | 774 | /* Disable FIPER by default */ |
| 774 | 775 | fiper_ctrl = netc_timer_rd(priv, NETC_TMR_FIPER_CTRL); |
| @@ -901,6 +902,11 @@ static irqreturn_t netc_timer_isr(int irq, void *data) · 126 unmodified lines | ||
| 901 | 902 | /* Clear interrupts status */ |
| 902 | 903 | netc_timer_wr(priv, NETC_TMR_TEVENT, tmr_event); |
| 903 | 904 | |
| 905 | + if (!tmr_event) { | |
| 906 | + spin_unlock(&priv->lock); | |
| 907 | + return IRQ_NONE; | |
| 908 | + } | |
| 909 | + | |
| 904 | 910 | if (tmr_event & TMR_TEVENT_ALMEN(0)) |
| 905 | 911 | netc_timer_alarm_write(priv, NETC_TMR_DEFAULT_ALARM, 0); |
| 906 | 912 | |
| @@ -936,7 +942,8 @@ static int netc_timer_init_msix_irq(struct netc_timer *priv) · 29 unmodified lines | ||
| 936 | 942 | } |
| 937 | 943 | |
| 938 | 944 | priv->irq = pci_irq_vector(pdev, 0); |
| 939 | − err = request_irq(priv->irq, netc_timer_isr, 0, priv->irq_name, priv); | |
| 945 | + err = request_irq(priv->irq, netc_timer_isr, IRQF_NO_AUTOEN, | |
| 946 | + priv->irq_name, priv); | |
| 940 | 947 | if (err) { |
| 941 | 948 | dev_err(&pdev->dev, "request_irq() failed\n"); |
| 942 | 949 | pci_free_irq_vectors(pdev); |
| @@ -951,7 +958,6 @@ static void netc_timer_free_msix_irq(struct netc_timer *priv) · 8 unmodified lines | ||
| 951 | 958 | { |
| 952 | 959 | struct pci_dev *pdev = priv->pdev; |
| 953 | 960 | |
| 954 | − disable_irq(priv->irq); | |
| 955 | 961 | free_irq(priv->irq, priv); |
| 956 | 962 | pci_free_irq_vectors(pdev); |
| 957 | 963 | } |
| @@ -1005,6 +1011,8 @@ static int netc_timer_probe(struct pci_dev *pdev, · 47 unmodified lines | ||
| 1005 | 1011 | goto free_msix_irq; |
| 1006 | 1012 | } |
| 1007 | 1013 | |
| 1014 | + enable_irq(priv->irq); | |
| 1015 | + | |
| 1008 | 1016 | return 0; |
| 1009 | 1017 | |
| 1010 | 1018 | free_msix_irq: |
| @@ -1019,9 +1027,10 @@ static void netc_timer_remove(struct pci_dev *pdev) · 8 unmodified lines | ||
| 1019 | 1027 | { |
| 1020 | 1028 | struct netc_timer *priv = pci_get_drvdata(pdev); |
| 1021 | 1029 | |
| 1030 | + disable_irq(priv->irq); | |
| 1031 | + ptp_clock_unregister(priv->clock); | |
| 1022 | 1032 | netc_timer_wr(priv, NETC_TMR_TEMASK, 0); |
| 1023 | 1033 | netc_timer_wr(priv, NETC_TMR_CTRL, 0); |
| 1024 | − ptp_clock_unregister(priv->clock); | |
| 1025 | 1034 | netc_timer_free_msix_irq(priv); |
| 1026 | 1035 | netc_timer_pci_remove(pdev); |
| 1027 | 1036 | } |
| @@ -2177,9 +2177,11 @@ ptp_ocp_devlink_info_get(struct devlink *devlink, struct devlink_info_req *req, · 2176 unmodified lines | ||
| 2177 | 2177 | if (err) |
| 2178 | 2178 | return err; |
| 2179 | 2179 | |
| 2180 | + snprintf(buf, sizeof(buf), "%.*s", OCP_BOARD_ID_LEN, | |
| 2181 | + (const char *)bp->board_id); | |
| 2180 | 2182 | err = devlink_info_version_fixed_put(req, |
| 2181 | 2183 | DEVLINK_INFO_VERSION_GENERIC_BOARD_ID, |
| 2182 | − bp->board_id); | |
| 2184 | + buf); | |
| 2183 | 2185 | if (err) |
| 2184 | 2186 | return err; |
| 2185 | 2187 | |
| @@ -21,6 +21,7 @@ · 20 unmodified lines | ||
| 21 | 21 | #include <linux/debugfs.h> |
| 22 | 22 | #include <linux/seq_file.h> |
| 23 | 23 | #include <linux/vmalloc.h> |
| 24 | +#include <linux/delay.h> | |
| 24 | 25 | |
| 25 | 26 | #include <asm/machine.h> |
| 26 | 27 | #include <asm/ccwdev.h> |
| @@ -2511,6 +2512,13 @@ static inline int _dasd_term_running_cqr(struct dasd_device *device) · 2484 unmodified lines | ||
| 2511 | 2512 | if (list_empty(&device->ccw_queue)) |
| 2512 | 2513 | return 0; |
| 2513 | 2514 | cqr = list_entry(device->ccw_queue.next, struct dasd_ccw_req, devlist); |
| 2515 | + /* | |
| 2516 | + * Path verification requests must not be terminated. They are critical | |
| 2517 | + * for bringing paths back online. Terminating them would cause rc=-EIO | |
| 2518 | + * because CLEARED requests skip the retry path. | |
| 2519 | + */ | |
| 2520 | + if (test_bit(DASD_CQR_VERIFY_PATH, &cqr->flags)) | |
| 2521 | + return -EAGAIN; | |
| 2514 | 2522 | rc = device->discipline->term_IO(cqr); |
| 2515 | 2523 | if (!rc) |
| 2516 | 2524 | /* |
| @@ -2535,7 +2543,11 @@ int dasd_sleep_on_immediatly(struct dasd_ccw_req *cqr) · 18 unmodified lines | ||
| 2535 | 2543 | return -EIO; |
| 2536 | 2544 | } |
| 2537 | 2545 | spin_lock_irq(get_ccwdev_lock(device->cdev)); |
| 2538 | − rc = _dasd_term_running_cqr(device); | |
| 2546 | + while ((rc = _dasd_term_running_cqr(device)) == -EAGAIN) { | |
| 2547 | + spin_unlock_irq(get_ccwdev_lock(device->cdev)); | |
| 2548 | + msleep(1); | |
| 2549 | + spin_lock_irq(get_ccwdev_lock(device->cdev)); | |
| 2550 | + } | |
| 2539 | 2551 | if (rc) { |
| 2540 | 2552 | spin_unlock_irq(get_ccwdev_lock(device->cdev)); |
| 2541 | 2553 | return rc; |
| @@ -20,6 +20,7 @@ · 19 unmodified lines | ||
| 20 | 20 | #include <linux/seq_file.h> |
| 21 | 21 | #include <linux/uaccess.h> |
| 22 | 22 | #include <linux/io.h> |
| 23 | +#include <linux/overflow.h> | |
| 23 | 24 | |
| 24 | 25 | #include <asm/css_chars.h> |
| 25 | 26 | #include <asm/machine.h> |
| @@ -3475,11 +3476,11 @@ static int dasd_eckd_check_device_format(struct dasd_device *base, · 3449 unmodified lines | ||
| 3475 | 3476 | { |
| 3476 | 3477 | struct dasd_eckd_private *private = base->private; |
| 3477 | 3478 | struct eckd_count *fmt_buffer; |
| 3478 | − struct irb irb; | |
| 3479 | + size_t fmt_buffer_size; | |
| 3480 | + unsigned int trkcount; | |
| 3479 | 3481 | int rpt_max, rpt_exp; |
| 3480 | − int fmt_buffer_size; | |
| 3482 | + struct irb irb; | |
| 3481 | 3483 | int trk_per_cyl; |
| 3482 | − int trkcount; | |
| 3483 | 3484 | int tpm = 0; |
| 3484 | 3485 | int rc; |
| 3485 | 3486 | |
| @@ -3490,7 +3491,9 @@ static int dasd_eckd_check_device_format(struct dasd_device *base, · 4 unmodified lines | ||
| 3490 | 3491 | rpt_exp = recs_per_track(&private->rdc_data, 0, cdata->expect.blksize); |
| 3491 | 3492 | |
| 3492 | 3493 | trkcount = cdata->expect.stop_unit - cdata->expect.start_unit + 1; |
| 3493 | − fmt_buffer_size = trkcount * rpt_max * sizeof(struct eckd_count); | |
| 3494 | + if (check_mul_overflow(trkcount, rpt_max, &fmt_buffer_size) || | |
| 3495 | + check_mul_overflow(fmt_buffer_size, sizeof(struct eckd_count), &fmt_buffer_size)) | |
| 3496 | + return -EINVAL; | |
| 3494 | 3497 | |
| 3495 | 3498 | fmt_buffer = kzalloc(fmt_buffer_size, GFP_KERNEL | GFP_DMA); |
| 3496 | 3499 | if (!fmt_buffer) |
| @@ -324,7 +324,7 @@ out_err: · 323 unmodified lines | ||
| 324 | 324 | static int dasd_release_space(struct dasd_device *device, |
| 325 | 325 | struct format_data_t *rdata) |
| 326 | 326 | { |
| 327 | − if (!device->discipline->is_ese && !device->discipline->is_ese(device)) | |
| 327 | + if (!device->discipline->is_ese || !device->discipline->is_ese(device)) | |
| 328 | 328 | return -ENOTSUPP; |
| 329 | 329 | if (!device->discipline->release_space) |
| 330 | 330 | return -ENOTSUPP; |
| @@ -8,6 +8,7 @@ · 7 unmodified lines | ||
| 8 | 8 | */ |
| 9 | 9 | |
| 10 | 10 | #include <linux/vfio.h> |
| 11 | +#include <linux/nospec.h> | |
| 11 | 12 | |
| 12 | 13 | #include "vfio_ccw_private.h" |
| 13 | 14 | |
| @@ -24,11 +25,20 @@ static ssize_t vfio_ccw_async_region_read(struct vfio_ccw_private *private, · 10 unmodified lines | ||
| 24 | 25 | return -EINVAL; |
| 25 | 26 | |
| 26 | 27 | mutex_lock(&private->io_mutex); |
| 28 | + | |
| 29 | + if (i >= private->num_regions) { | |
| 30 | + ret = -EINVAL; | |
| 31 | + goto out_unlock; | |
| 32 | + } | |
| 33 | + | |
| 34 | + i = array_index_nospec(i, private->num_regions); | |
| 27 | 35 | region = private->region[i].data; |
| 28 | 36 | if (copy_to_user(buf, (void *)region + pos, count)) |
| 29 | 37 | ret = -EFAULT; |
| 30 | 38 | else |
| 31 | 39 | ret = count; |
| 40 | + | |
| 41 | +out_unlock: | |
| 32 | 42 | mutex_unlock(&private->io_mutex); |
| 33 | 43 | return ret; |
| 34 | 44 | } |
| @@ -48,6 +58,12 @@ static ssize_t vfio_ccw_async_region_write(struct vfio_ccw_private *private, · 13 unmodified lines | ||
| 48 | 58 | if (!mutex_trylock(&private->io_mutex)) |
| 49 | 59 | return -EAGAIN; |
| 50 | 60 | |
| 61 | + if (i >= private->num_regions) { | |
| 62 | + ret = -EINVAL; | |
| 63 | + goto out_unlock; | |
| 64 | + } | |
| 65 | + | |
| 66 | + i = array_index_nospec(i, private->num_regions); | |
| 51 | 67 | region = private->region[i].data; |
| 52 | 68 | if (copy_from_user((void *)region + pos, buf, count)) { |
| 53 | 69 | ret = -EFAULT; |
| @@ -9,6 +9,7 @@ · 8 unmodified lines | ||
| 9 | 9 | */ |
| 10 | 10 | |
| 11 | 11 | #include <linux/slab.h> |
| 12 | +#include <linux/nospec.h> | |
| 12 | 13 | #include <linux/vfio.h> |
| 13 | 14 | #include "vfio_ccw_private.h" |
| 14 | 15 | |
| @@ -26,6 +27,13 @@ static ssize_t vfio_ccw_schib_region_read(struct vfio_ccw_private *private, · 11 unmodified lines | ||
| 26 | 27 | return -EINVAL; |
| 27 | 28 | |
| 28 | 29 | mutex_lock(&private->io_mutex); |
| 30 | + | |
| 31 | + if (i >= private->num_regions) { | |
| 32 | + ret = -EINVAL; | |
| 33 | + goto out; | |
| 34 | + } | |
| 35 | + | |
| 36 | + i = array_index_nospec(i, private->num_regions); | |
| 29 | 37 | region = private->region[i].data; |
| 30 | 38 | |
| 31 | 39 | if (cio_update_schib(sch)) { |
| @@ -85,19 +93,30 @@ static ssize_t vfio_ccw_crw_region_read(struct vfio_ccw_private *private, · 53 unmodified lines | ||
| 85 | 93 | loff_t pos = *ppos & VFIO_CCW_OFFSET_MASK; |
| 86 | 94 | struct ccw_crw_region *region; |
| 87 | 95 | struct vfio_ccw_crw *crw; |
| 96 | + unsigned long flags; | |
| 88 | 97 | int ret; |
| 89 | 98 | |
| 90 | 99 | if (pos + count > sizeof(*region)) |
| 91 | 100 | return -EINVAL; |
| 92 | 101 | |
| 102 | + mutex_lock(&private->io_mutex); | |
| 103 | + if (i >= private->num_regions) { | |
| 104 | + ret = -EINVAL; | |
| 105 | + goto out; | |
| 106 | + } | |
| 107 | + | |
| 108 | + i = array_index_nospec(i, private->num_regions); | |
| 109 | + region = private->region[i].data; | |
| 110 | + | |
| 111 | + spin_lock_irqsave(&private->crw_lock, flags); | |
| 93 | 112 | crw = list_first_entry_or_null(&private->crw, |
| 94 | 113 | struct vfio_ccw_crw, next); |
| 95 | 114 | |
| 96 | 115 | if (crw) |
| 97 | 116 | list_del(&crw->next); |
| 98 | 117 | |
| 99 | − mutex_lock(&private->io_mutex); | |
| 100 | − region = private->region[i].data; | |
| 118 | + /* Drop CRW lock while copying to userspace */ | |
| 119 | + spin_unlock_irqrestore(&private->crw_lock, flags); | |
| 101 | 120 | |
| 102 | 121 | if (crw) |
| 103 | 122 | memcpy(®ion->crw, &crw->crw, sizeof(region->crw)); |
| @@ -108,14 +127,16 @@ static ssize_t vfio_ccw_crw_region_read(struct vfio_ccw_private *private, · 4 unmodified lines | ||
| 108 | 127 | ret = count; |
| 109 | 128 | |
| 110 | 129 | region->crw = 0; |
| 111 | − | |
| 112 | − mutex_unlock(&private->io_mutex); | |
| 113 | − | |
| 114 | 130 | kfree(crw); |
| 115 | 131 | |
| 116 | 132 | /* Notify the guest if more CRWs are on our queue */ |
| 133 | + spin_lock_irqsave(&private->crw_lock, flags); | |
| 117 | 134 | if (!list_empty(&private->crw) && private->crw_trigger) |
| 118 | 135 | eventfd_signal(private->crw_trigger); |
| 136 | + spin_unlock_irqrestore(&private->crw_lock, flags); | |
| 137 | + | |
| 138 | +out: | |
| 139 | + mutex_unlock(&private->io_mutex); | |
| 119 | 140 | |
| 120 | 141 | return ret; |
| 121 | 142 | } |
| @@ -233,6 +233,7 @@ static void convert_ccw0_to_ccw1(struct ccw1 *source, unsigned long len) · 232 unmodified lines | ||
| 233 | 233 | } |
| 234 | 234 | |
| 235 | 235 | #define idal_is_2k(_cp) (!(_cp)->orb.cmd.c64 || (_cp)->orb.cmd.i2k) |
| 236 | +#define get_idaw_size(_cp) ((_cp)->orb.cmd.c64 ? sizeof(u64) : sizeof(u32)) | |
| 236 | 237 | |
| 237 | 238 | /* |
| 238 | 239 | * Helpers to operate ccwchain. |
| @@ -332,6 +333,7 @@ static struct ccwchain *ccwchain_alloc(struct channel_program *cp, int len) · 93 unmodified lines | ||
| 332 | 333 | goto out_err; |
| 333 | 334 | |
| 334 | 335 | list_add_tail(&chain->next, &cp->ccwchain_list); |
| 336 | + cp->ccwchain_count++; | |
| 335 | 337 | |
| 336 | 338 | return chain; |
| 337 | 339 | |
| @@ -376,11 +378,9 @@ static void ccwchain_cda_free(struct ccwchain *chain, int idx) · 38 unmodified lines | ||
| 376 | 378 | static int ccwchain_calc_length(u64 iova, struct channel_program *cp) |
| 377 | 379 | { |
| 378 | 380 | struct ccw1 *ccw = cp->guest_cp; |
| 379 | − int cnt = 0; | |
| 380 | − | |
| 381 | − do { | |
| 382 | − cnt++; | |
| 381 | + int cnt; | |
| 383 | 382 | |
| 383 | + for (cnt = 1; cnt <= CCWCHAIN_LEN_MAX; cnt++, ccw++) { | |
| 384 | 384 | /* |
| 385 | 385 | * We want to keep counting if the current CCW has the |
| 386 | 386 | * command-chaining flag enabled, or if it is a TIC CCW |
| @@ -390,15 +390,10 @@ static int ccwchain_calc_length(u64 iova, struct channel_program *cp) · 3 unmodified lines | ||
| 390 | 390 | * after the TIC, depending on the results of its operation. |
| 391 | 391 | */ |
| 392 | 392 | if (!ccw_is_chain(ccw) && !is_tic_within_range(ccw, iova, cnt)) |
| 393 | − break; | |
| 394 | − | |
| 395 | − ccw++; | |
| 396 | − } while (cnt < CCWCHAIN_LEN_MAX + 1); | |
| 397 | − | |
| 398 | − if (cnt == CCWCHAIN_LEN_MAX + 1) | |
| 399 | − cnt = -EINVAL; | |
| 393 | + return cnt; | |
| 394 | + } | |
| 400 | 395 | |
| 401 | − return cnt; | |
| 396 | + return -EINVAL; | |
| 402 | 397 | } |
| 403 | 398 | |
| 404 | 399 | static int tic_target_chain_exists(struct ccw1 *tic, struct channel_program *cp) |
| @@ -441,6 +436,10 @@ static int ccwchain_handle_ccw(dma32_t cda, struct channel_program *cp) · 36 unmodified lines | ||
| 441 | 436 | if (len < 0) |
| 442 | 437 | return len; |
| 443 | 438 | |
| 439 | + /* Limit number of chains in a single channel program */ | |
| 440 | + if (cp->ccwchain_count >= CCWCHAIN_COUNT_MAX) | |
| 441 | + return -EINVAL; | |
| 442 | + | |
| 444 | 443 | /* Need alloc a new chain for this one. */ |
| 445 | 444 | chain = ccwchain_alloc(cp, len); |
| 446 | 445 | if (!chain) |
| @@ -455,9 +454,6 @@ static int ccwchain_handle_ccw(dma32_t cda, struct channel_program *cp) · 8 unmodified lines | ||
| 455 | 454 | /* Loop for tics on this new chain. */ |
| 456 | 455 | ret = ccwchain_loop_tic(chain, cp); |
| 457 | 456 | |
| 458 | − if (ret) | |
| 459 | − ccwchain_free(chain); | |
| 460 | − | |
| 461 | 457 | return ret; |
| 462 | 458 | } |
| 463 | 459 | |
| @@ -486,6 +482,23 @@ static int ccwchain_loop_tic(struct ccwchain *chain, struct channel_program *cp) · 22 unmodified lines | ||
| 486 | 482 | return 0; |
| 487 | 483 | } |
| 488 | 484 | |
| 485 | +static int ccwchain_build_ccws(dma32_t cda, struct channel_program *cp) | |
| 486 | +{ | |
| 487 | + struct ccwchain *chain, *temp; | |
| 488 | + int ret; | |
| 489 | + | |
| 490 | + ret = ccwchain_handle_ccw(cda, cp); | |
| 491 | + | |
| 492 | + if (ret) { | |
| 493 | + /* Cleanup if an error occurred */ | |
| 494 | + list_for_each_entry_safe(chain, temp, &cp->ccwchain_list, next) { | |
| 495 | + ccwchain_free(chain); | |
| 496 | + } | |
| 497 | + } | |
| 498 | + | |
| 499 | + return ret; | |
| 500 | +} | |
| 501 | + | |
| 489 | 502 | static int ccwchain_fetch_tic(struct ccw1 *ccw, |
| 490 | 503 | struct channel_program *cp) |
| 491 | 504 | { |
| @@ -511,7 +524,8 @@ static dma64_t *get_guest_idal(struct ccw1 *ccw, struct channel_program *cp, int · 19 unmodified lines | ||
| 511 | 524 | &container_of(cp, struct vfio_ccw_private, cp)->vdev; |
| 512 | 525 | dma64_t *idaws; |
| 513 | 526 | dma32_t *idaws_f1; |
| 514 | − int idal_len = idaw_nr * sizeof(*idaws); | |
| 527 | + u64 first_idaw; | |
| 528 | + int idal_len = idaw_nr * get_idaw_size(cp); | |
| 515 | 529 | int idaw_size = idal_is_2k(cp) ? PAGE_SIZE / 2 : PAGE_SIZE; |
| 516 | 530 | int idaw_mask = ~(idaw_size - 1); |
| 517 | 531 | int i, ret; |
| @@ -527,6 +541,18 @@ static dma64_t *get_guest_idal(struct ccw1 *ccw, struct channel_program *cp, int · 9 unmodified lines | ||
| 527 | 541 | kfree(idaws); |
| 528 | 542 | return ERR_PTR(ret); |
| 529 | 543 | } |
| 544 | + | |
| 545 | + idaws_f1 = (dma32_t *)idaws; | |
| 546 | + if (cp->orb.cmd.c64) | |
| 547 | + first_idaw = dma64_to_u64(idaws[0]); | |
| 548 | + else | |
| 549 | + first_idaw = dma32_to_u32(idaws_f1[0]); | |
| 550 | + | |
| 551 | + /* Unexpected mismatch from earlier read */ | |
| 552 | + if (first_idaw != cp->guest_iova) { | |
| 553 | + kfree(idaws); | |
| 554 | + return ERR_PTR(-EINVAL); | |
| 555 | + } | |
| 530 | 556 | } else { |
| 531 | 557 | /* Fabricate an IDAL based off CCW data address */ |
| 532 | 558 | if (cp->orb.cmd.c64) { |
| @@ -568,7 +594,7 @@ static int ccw_count_idaws(struct ccw1 *ccw, · 35 unmodified lines | ||
| 568 | 594 | struct vfio_device *vdev = |
| 569 | 595 | &container_of(cp, struct vfio_ccw_private, cp)->vdev; |
| 570 | 596 | u64 iova; |
| 571 | − int size = cp->orb.cmd.c64 ? sizeof(u64) : sizeof(u32); | |
| 597 | + int size = get_idaw_size(cp); | |
| 572 | 598 | int ret; |
| 573 | 599 | int bytes = 1; |
| 574 | 600 | |
| @@ -592,6 +618,9 @@ static int ccw_count_idaws(struct ccw1 *ccw, · 17 unmodified lines | ||
| 592 | 618 | iova = dma32_to_u32(ccw->cda); |
| 593 | 619 | } |
| 594 | 620 | |
| 621 | + /* Save the read address for later */ | |
| 622 | + cp->guest_iova = iova; | |
| 623 | + | |
| 595 | 624 | /* Format-1 IDAWs operate on 2K each */ |
| 596 | 625 | if (!cp->orb.cmd.c64) |
| 597 | 626 | return idal_2k_nr_words((void *)iova, bytes); |
| @@ -731,11 +760,12 @@ int cp_init(struct channel_program *cp, union orb *orb) · 133 unmodified lines | ||
| 731 | 760 | vdev->dev, |
| 732 | 761 | "Prefetching channel program even though prefetch not specified in ORB"); |
| 733 | 762 | |
| 763 | + cp->ccwchain_count = 0; | |
| 734 | 764 | INIT_LIST_HEAD(&cp->ccwchain_list); |
| 735 | 765 | memcpy(&cp->orb, orb, sizeof(*orb)); |
| 736 | 766 | |
| 737 | 767 | /* Build a ccwchain for the first CCW segment */ |
| 738 | − ret = ccwchain_handle_ccw(orb->cmd.cpa, cp); | |
| 768 | + ret = ccwchain_build_ccws(orb->cmd.cpa, cp); | |
| 739 | 769 | |
| 740 | 770 | if (!ret) |
| 741 | 771 | cp->initialized = true; |
| @@ -947,17 +977,23 @@ void cp_update_scsw(struct channel_program *cp, union scsw *scsw) · 205 unmodified lines | ||
| 947 | 977 | */ |
| 948 | 978 | bool cp_iova_pinned(struct channel_program *cp, u64 iova, u64 length) |
| 949 | 979 | { |
| 980 | + struct vfio_ccw_private *private = | |
| 981 | + container_of(cp, struct vfio_ccw_private, cp); | |
| 950 | 982 | struct ccwchain *chain; |
| 951 | 983 | int i; |
| 952 | 984 | |
| 953 | 985 | if (!cp->initialized) |
| 954 | 986 | return false; |
| 955 | 987 | |
| 988 | + mutex_lock(&private->io_mutex); | |
| 956 | 989 | list_for_each_entry(chain, &cp->ccwchain_list, next) { |
| 957 | 990 | for (i = 0; i < chain->ch_len; i++) |
| 958 | − if (page_array_iova_pinned(&chain->ch_pa[i], iova, length)) | |
| 991 | + if (page_array_iova_pinned(&chain->ch_pa[i], iova, length)) { | |
| 992 | + mutex_unlock(&private->io_mutex); | |
| 959 | 993 | return true; |
| 994 | + } | |
| 960 | 995 | } |
| 996 | + mutex_unlock(&private->io_mutex); | |
| 961 | 997 | |
| 962 | 998 | return false; |
| 963 | 999 | } |
| @@ -23,11 +23,19 @@ · 22 unmodified lines | ||
| 23 | 23 | */ |
| 24 | 24 | #define CCWCHAIN_LEN_MAX 256 |
| 25 | 25 | |
| 26 | +/* | |
| 27 | + * Maximum number of chains | |
| 28 | + */ | |
| 29 | +#define CCWCHAIN_COUNT_MAX 16 | |
| 30 | + | |
| 26 | 31 | /** |
| 27 | 32 | * struct channel_program - manage information for channel program |
| 28 | 33 | * @ccwchain_list: list head of ccwchains |
| 29 | 34 | * @orb: orb for the currently processed ssch request |
| 30 | 35 | * @initialized: whether this instance is actually initialized |
| 36 | + * @guest_cp: copy of guest channel program | |
| 37 | + * @ccwchain_count: number of channel program segments (linked by TIC) | |
| 38 | + * @guest_iova: first data address of a guest channel program | |
| 31 | 39 | * |
| 32 | 40 | * @ccwchain_list is the head of a ccwchain list, that contents the |
| 33 | 41 | * translated result of the guest channel program that pointed out by |
| @@ -38,6 +46,8 @@ struct channel_program { · 4 unmodified lines | ||
| 38 | 46 | union orb orb; |
| 39 | 47 | bool initialized; |
| 40 | 48 | struct ccw1 *guest_cp; |
| 49 | + unsigned int ccwchain_count; | |
| 50 | + u64 guest_iova; | |
| 41 | 51 | }; |
| 42 | 52 | |
| 43 | 53 | int cp_init(struct channel_program *cp, union orb *orb); |
| @@ -91,6 +91,7 @@ void vfio_ccw_sch_io_todo(struct work_struct *work) · 90 unmodified lines | ||
| 91 | 91 | |
| 92 | 92 | is_final = !(scsw_actl(&irb->scsw) & |
| 93 | 93 | (SCSW_ACTL_DEVACT | SCSW_ACTL_SCHACT)); |
| 94 | + mutex_lock(&private->io_mutex); | |
| 94 | 95 | if (scsw_is_solicited(&irb->scsw)) { |
| 95 | 96 | cp_update_scsw(&private->cp, &irb->scsw); |
| 96 | 97 | if (is_final && private->state == VFIO_CCW_STATE_CP_PENDING) { |
| @@ -98,9 +99,7 @@ void vfio_ccw_sch_io_todo(struct work_struct *work) · 1 unmodified line | ||
| 98 | 99 | cp_is_finished = true; |
| 99 | 100 | } |
| 100 | 101 | } |
| 101 | − mutex_lock(&private->io_mutex); | |
| 102 | 102 | memcpy(private->io_region->irb_area, irb, sizeof(*irb)); |
| 103 | − mutex_unlock(&private->io_mutex); | |
| 104 | 103 | |
| 105 | 104 | /* |
| 106 | 105 | * Reset to IDLE only if processing of a channel program |
| @@ -110,6 +109,7 @@ void vfio_ccw_sch_io_todo(struct work_struct *work) · 3 unmodified lines | ||
| 110 | 109 | */ |
| 111 | 110 | if (cp_is_finished) |
| 112 | 111 | private->state = VFIO_CCW_STATE_IDLE; |
| 112 | + mutex_unlock(&private->io_mutex); | |
| 113 | 113 | |
| 114 | 114 | if (private->io_trigger) |
| 115 | 115 | eventfd_signal(private->io_trigger); |
| @@ -118,11 +118,25 @@ void vfio_ccw_sch_io_todo(struct work_struct *work) · 2 unmodified lines | ||
| 118 | 118 | void vfio_ccw_crw_todo(struct work_struct *work) |
| 119 | 119 | { |
| 120 | 120 | struct vfio_ccw_private *private; |
| 121 | + unsigned long flags; | |
| 121 | 122 | |
| 122 | 123 | private = container_of(work, struct vfio_ccw_private, crw_work); |
| 123 | 124 | |
| 125 | + spin_lock_irqsave(&private->crw_lock, flags); | |
| 124 | 126 | if (!list_empty(&private->crw) && private->crw_trigger) |
| 125 | 127 | eventfd_signal(private->crw_trigger); |
| 128 | + spin_unlock_irqrestore(&private->crw_lock, flags); | |
| 129 | +} | |
| 130 | + | |
| 131 | +void vfio_ccw_notoper_todo(struct work_struct *work) | |
| 132 | +{ | |
| 133 | + struct vfio_ccw_private *private; | |
| 134 | + | |
| 135 | + private = container_of(work, struct vfio_ccw_private, notoper_work); | |
| 136 | + | |
| 137 | + mutex_lock(&private->io_mutex); | |
| 138 | + cp_free(&private->cp); | |
| 139 | + mutex_unlock(&private->io_mutex); | |
| 126 | 140 | } |
| 127 | 141 | |
| 128 | 142 | /* |
| @@ -275,6 +289,7 @@ static void vfio_ccw_queue_crw(struct vfio_ccw_private *private, · 146 unmodified lines | ||
| 275 | 289 | unsigned int rsid) |
| 276 | 290 | { |
| 277 | 291 | struct vfio_ccw_crw *crw; |
| 292 | + unsigned long flags; | |
| 278 | 293 | |
| 279 | 294 | /* |
| 280 | 295 | * If unable to allocate a CRW, just drop the event and |
| @@ -292,7 +307,9 @@ static void vfio_ccw_queue_crw(struct vfio_ccw_private *private, · 11 unmodified lines | ||
| 292 | 307 | crw->crw.erc = erc; |
| 293 | 308 | crw->crw.rsid = rsid; |
| 294 | 309 | |
| 310 | + spin_lock_irqsave(&private->crw_lock, flags); | |
| 295 | 311 | list_add_tail(&crw->next, &private->crw); |
| 312 | + spin_unlock_irqrestore(&private->crw_lock, flags); | |
| 296 | 313 | queue_work(vfio_ccw_work_q, &private->crw_work); |
| 297 | 314 | } |
| 298 | 315 | |
| @@ -170,8 +170,8 @@ static void fsm_notoper(struct vfio_ccw_private *private, · 169 unmodified lines | ||
| 170 | 170 | css_sched_sch_todo(sch, SCH_TODO_UNREG); |
| 171 | 171 | private->state = VFIO_CCW_STATE_NOT_OPER; |
| 172 | 172 | |
| 173 | − /* This is usually handled during CLOSE event */ | |
| 174 | − cp_free(&private->cp); | |
| 173 | + /* This routine could be called from IRQ context, so defer */ | |
| 174 | + queue_work(vfio_ccw_work_q, &private->notoper_work); | |
| 175 | 175 | } |
| 176 | 176 | |
| 177 | 177 | /* |
| @@ -410,7 +410,11 @@ static void fsm_close(struct vfio_ccw_private *private, · 232 unmodified lines | ||
| 410 | 410 | |
| 411 | 411 | private->state = VFIO_CCW_STATE_STANDBY; |
| 412 | 412 | spin_unlock_irq(&sch->lock); |
| 413 | + | |
| 414 | + mutex_lock(&private->io_mutex); | |
| 413 | 415 | cp_free(&private->cp); |
| 416 | + mutex_unlock(&private->io_mutex); | |
| 417 | + | |
| 414 | 418 | return; |
| 415 | 419 | |
| 416 | 420 | err_unlock: |
| @@ -54,6 +54,8 @@ static int vfio_ccw_mdev_init_dev(struct vfio_device *vdev) · 53 unmodified lines | ||
| 54 | 54 | INIT_LIST_HEAD(&private->crw); |
| 55 | 55 | INIT_WORK(&private->io_work, vfio_ccw_sch_io_todo); |
| 56 | 56 | INIT_WORK(&private->crw_work, vfio_ccw_crw_todo); |
| 57 | + INIT_WORK(&private->notoper_work, vfio_ccw_notoper_todo); | |
| 58 | + spin_lock_init(&private->crw_lock); | |
| 57 | 59 | |
| 58 | 60 | private->cp.guest_cp = kzalloc_objs(struct ccw1, CCWCHAIN_LEN_MAX); |
| 59 | 61 | if (!private->cp.guest_cp) |
| @@ -130,11 +132,28 @@ static void vfio_ccw_mdev_release_dev(struct vfio_device *vdev) · 70 unmodified lines | ||
| 130 | 132 | struct vfio_ccw_private *private = |
| 131 | 133 | container_of(vdev, struct vfio_ccw_private, vdev); |
| 132 | 134 | struct vfio_ccw_crw *crw, *temp; |
| 135 | + unsigned long flags; | |
| 133 | 136 | |
| 137 | + /* | |
| 138 | + * Ensure these work items are fully drained, so none can | |
| 139 | + * fire after being released. | |
| 140 | + * | |
| 141 | + * notoper_work should have nothing to do here, because only | |
| 142 | + * open devices could have channel_program resources in use | |
| 143 | + * and those would be released during close. Nevertheless, | |
| 144 | + * call flush here as well to be certain anything that was | |
| 145 | + * allocated is freed. | |
| 146 | + */ | |
| 147 | + cancel_work_sync(&private->io_work); | |
| 148 | + cancel_work_sync(&private->crw_work); | |
| 149 | + flush_work(&private->notoper_work); | |
| 150 | + | |
| 151 | + spin_lock_irqsave(&private->crw_lock, flags); | |
| 134 | 152 | list_for_each_entry_safe(crw, temp, &private->crw, next) { |
| 135 | 153 | list_del(&crw->next); |
| 136 | 154 | kfree(crw); |
| 137 | 155 | } |
| 156 | + spin_unlock_irqrestore(&private->crw_lock, flags); | |
| 138 | 157 | |
| 139 | 158 | kmem_cache_free(vfio_ccw_crw_region, private->crw_region); |
| 140 | 159 | kmem_cache_free(vfio_ccw_schib_region, private->schib_region); |
| @@ -202,6 +221,19 @@ static void vfio_ccw_mdev_close_device(struct vfio_device *vdev) · 61 unmodified lines | ||
| 202 | 221 | container_of(vdev, struct vfio_ccw_private, vdev); |
| 203 | 222 | |
| 204 | 223 | vfio_ccw_fsm_event(private, VFIO_CCW_EVENT_CLOSE); |
| 224 | + | |
| 225 | + /* | |
| 226 | + * Ensure these work items are drained, in the event the | |
| 227 | + * device is re-opened instead of released. | |
| 228 | + * | |
| 229 | + * notoper_work needs to be given a chance to run if it | |
| 230 | + * is queued, so any memory associated with the channel | |
| 231 | + * program can be returned. | |
| 232 | + */ | |
| 233 | + cancel_work_sync(&private->io_work); | |
| 234 | + cancel_work_sync(&private->crw_work); | |
| 235 | + flush_work(&private->notoper_work); | |
| 236 | + | |
| 205 | 237 | vfio_ccw_unregister_dev_regions(private); |
| 206 | 238 | } |
| 207 | 239 | |
| @@ -243,6 +275,7 @@ static ssize_t vfio_ccw_mdev_read(struct vfio_device *vdev, · 35 unmodified lines | ||
| 243 | 275 | return vfio_ccw_mdev_read_io_region(private, buf, count, ppos); |
| 244 | 276 | default: |
| 245 | 277 | index -= VFIO_CCW_NUM_REGIONS; |
| 278 | + index = array_index_nospec(index, private->num_regions); | |
| 246 | 279 | return private->region[index].ops->read(private, buf, count, |
| 247 | 280 | ppos); |
| 248 | 281 | } |
| @@ -295,6 +328,7 @@ static ssize_t vfio_ccw_mdev_write(struct vfio_device *vdev, · 46 unmodified lines | ||
| 295 | 328 | return vfio_ccw_mdev_write_io_region(private, buf, count, ppos); |
| 296 | 329 | default: |
| 297 | 330 | index -= VFIO_CCW_NUM_REGIONS; |
| 331 | + index = array_index_nospec(index, private->num_regions); | |
| 298 | 332 | return private->region[index].ops->write(private, buf, count, |
| 299 | 333 | ppos); |
| 300 | 334 | } |
| @@ -338,11 +372,8 @@ static int vfio_ccw_mdev_ioctl_get_region_info(struct vfio_device *vdev, · 37 unmodified lines | ||
| 338 | 372 | VFIO_CCW_NUM_REGIONS + private->num_regions) |
| 339 | 373 | return -EINVAL; |
| 340 | 374 | |
| 341 | − info->index = array_index_nospec(info->index, | |
| 342 | − VFIO_CCW_NUM_REGIONS + | |
| 343 | − private->num_regions); | |
| 344 | − | |
| 345 | 375 | i = info->index - VFIO_CCW_NUM_REGIONS; |
| 376 | + i = array_index_nospec(i, private->num_regions); | |
| 346 | 377 | |
| 347 | 378 | info->offset = VFIO_CCW_INDEX_TO_OFFSET(info->index); |
| 348 | 379 | info->size = private->region[i].size; |
| @@ -88,7 +88,8 @@ struct vfio_ccw_parent { · 87 unmodified lines | ||
| 88 | 88 | * @state: internal state of the device |
| 89 | 89 | * @completion: synchronization helper of the I/O completion |
| 90 | 90 | * @io_region: MMIO region to input/output I/O arguments/results |
| 91 | − * @io_mutex: protect against concurrent update of I/O regions | |
| 91 | + * @io_mutex: protect against concurrent update of I/O resources | |
| 92 | + * and @cp lifecycle | |
| 92 | 93 | * @region: additional regions for other subchannel operations |
| 93 | 94 | * @cmd_region: MMIO region for asynchronous I/O commands other than START |
| 94 | 95 | * @schib_region: MMIO region for SCHIB information |
| @@ -97,11 +98,14 @@ struct vfio_ccw_parent { · 2 unmodified lines | ||
| 97 | 98 | * @cp: channel program for the current I/O operation |
| 98 | 99 | * @irb: irb info received from interrupt |
| 99 | 100 | * @scsw: scsw info |
| 101 | + * @crw_lock: serialization of CRW list information | |
| 102 | + * @crw: list of Channel Report Word elements | |
| 100 | 103 | * @io_trigger: eventfd ctx for signaling userspace I/O results |
| 101 | 104 | * @crw_trigger: eventfd ctx for signaling userspace CRW information |
| 102 | 105 | * @req_trigger: eventfd ctx for signaling userspace to return device |
| 103 | 106 | * @io_work: work for deferral process of I/O handling |
| 104 | 107 | * @crw_work: work for deferral process of CRW handling |
| 108 | + * @notoper_work: work for deferred processing in not-operational state | |
| 105 | 109 | */ |
| 106 | 110 | struct vfio_ccw_private { |
| 107 | 111 | struct vfio_device vdev; |
| @@ -118,6 +122,8 @@ struct vfio_ccw_private { · 10 unmodified lines | ||
| 118 | 122 | struct channel_program cp; |
| 119 | 123 | struct irb irb; |
| 120 | 124 | union scsw scsw; |
| 125 | + | |
| 126 | + spinlock_t crw_lock; | |
| 121 | 127 | struct list_head crw; |
| 122 | 128 | |
| 123 | 129 | struct eventfd_ctx *io_trigger; |
| @@ -125,11 +131,13 @@ struct vfio_ccw_private { · 1 unmodified line | ||
| 125 | 131 | struct eventfd_ctx *req_trigger; |
| 126 | 132 | struct work_struct io_work; |
| 127 | 133 | struct work_struct crw_work; |
| 134 | + struct work_struct notoper_work; | |
| 128 | 135 | } __aligned(8); |
| 129 | 136 | |
| 130 | 137 | int vfio_ccw_sch_quiesce(struct subchannel *sch); |
| 131 | 138 | void vfio_ccw_sch_io_todo(struct work_struct *work); |
| 132 | 139 | void vfio_ccw_crw_todo(struct work_struct *work); |
| 140 | +void vfio_ccw_notoper_todo(struct work_struct *work); | |
| 133 | 141 | |
| 134 | 142 | extern struct mdev_driver vfio_ccw_mdev_driver; |
| 135 | 143 | |
| @@ -236,22 +236,16 @@ static int cca_key2protkey(const struct pkey_apqn *apqns, size_t nr_apqns, · 235 unmodified lines | ||
| 236 | 236 | if (hdr->type == TOKTYPE_CCA_INTERNAL && |
| 237 | 237 | hdr->version == TOKVER_CCA_AES) { |
| 238 | 238 | /* CCA AES data key */ |
| 239 | − if (keylen < sizeof(struct secaeskeytoken)) | |
| 240 | − return -EINVAL; | |
| 241 | − if (cca_check_secaeskeytoken(pkey_dbf_info, 3, key, 0)) | |
| 239 | + if (cca_check_secaeskeytoken(pkey_dbf_info, 3, key, keylen, 0)) | |
| 242 | 240 | return -EINVAL; |
| 243 | 241 | } else if (hdr->type == TOKTYPE_CCA_INTERNAL && |
| 244 | 242 | hdr->version == TOKVER_CCA_VLSC) { |
| 245 | 243 | /* CCA AES cipher key */ |
| 246 | − if (keylen < hdr->len) | |
| 247 | − return -EINVAL; | |
| 248 | 244 | if (cca_check_secaescipherkey(pkey_dbf_info, |
| 249 | − 3, key, 0, 1)) | |
| 245 | + 3, key, keylen, 0, 1)) | |
| 250 | 246 | return -EINVAL; |
| 251 | 247 | } else if (hdr->type == TOKTYPE_CCA_INTERNAL_PKA) { |
| 252 | 248 | /* CCA ECC (private) key */ |
| 253 | − if (keylen < sizeof(struct eccprivkeytoken)) | |
| 254 | − return -EINVAL; | |
| 255 | 249 | if (cca_check_sececckeytoken(pkey_dbf_info, 3, key, keylen, 1)) |
| 256 | 250 | return -EINVAL; |
| 257 | 251 | } else { |
| @@ -484,7 +478,7 @@ static int cca_verifykey(const u8 *key, u32 keylen, · 226 unmodified lines | ||
| 484 | 478 | hdr->version == TOKVER_CCA_AES) { |
| 485 | 479 | struct secaeskeytoken *t = (struct secaeskeytoken *)key; |
| 486 | 480 | |
| 487 | − rc = cca_check_secaeskeytoken(pkey_dbf_info, 3, key, 0); | |
| 481 | + rc = cca_check_secaeskeytoken(pkey_dbf_info, 3, key, keylen, 0); | |
| 488 | 482 | if (rc) |
| 489 | 483 | goto out; |
| 490 | 484 | *keytype = PKEY_TYPE_CCA_DATA; |
| @@ -512,7 +506,8 @@ static int cca_verifykey(const u8 *key, u32 keylen, · 21 unmodified lines | ||
| 512 | 506 | hdr->version == TOKVER_CCA_VLSC) { |
| 513 | 507 | struct cipherkeytoken *t = (struct cipherkeytoken *)key; |
| 514 | 508 | |
| 515 | − rc = cca_check_secaescipherkey(pkey_dbf_info, 3, key, 0, 1); | |
| 509 | + rc = cca_check_secaescipherkey(pkey_dbf_info, 3, | |
| 510 | + key, keylen, 0, 1); | |
| 516 | 511 | if (rc) |
| 517 | 512 | goto out; |
| 518 | 513 | *keytype = PKEY_TYPE_CCA_CIPHER; |
| @@ -879,6 +879,7 @@ static long _zcrypt_send_cprb(u32 xflags, struct ap_perms *perms, · 878 unmodified lines | ||
| 879 | 879 | |
| 880 | 880 | if (perms != &ap_perms && domain < AP_DOMAINS) { |
| 881 | 881 | if (ap_msg.flags & AP_MSG_FLAG_ADMIN) { |
| 882 | + domain = array_index_nospec(domain, AP_DOMAINS); | |
| 882 | 883 | if (!test_bit_inv(domain, perms->adm)) { |
| 883 | 884 | rc = -ENODEV; |
| 884 | 885 | goto out; |
| @@ -1077,8 +1078,9 @@ static long _zcrypt_send_ep11_cprb(u32 xflags, struct ap_perms *perms, · 192 unmodified lines | ||
| 1077 | 1078 | print_hex_dump_debug("ep11req: ", DUMP_PREFIX_ADDRESS, 16, 1, |
| 1078 | 1079 | ap_msg.msg, ap_msg.len, false); |
| 1079 | 1080 | |
| 1080 | − if (perms != &ap_perms && domain < AUTOSEL_DOM) { | |
| 1081 | + if (perms != &ap_perms && domain < AP_DOMAINS) { | |
| 1081 | 1082 | if (ap_msg.flags & AP_MSG_FLAG_ADMIN) { |
| 1083 | + domain = array_index_nospec(domain, AP_DOMAINS); | |
| 1082 | 1084 | if (!test_bit_inv(domain, perms->adm)) { |
| 1083 | 1085 | rc = -ENODEV; |
| 1084 | 1086 | goto out; |
| @@ -15,6 +15,7 @@ · 14 unmodified lines | ||
| 15 | 15 | #include <linux/module.h> |
| 16 | 16 | #include <linux/slab.h> |
| 17 | 17 | #include <linux/random.h> |
| 18 | +#include <linux/align.h> | |
| 18 | 19 | #include <asm/zcrypt.h> |
| 19 | 20 | #include <asm/pkey.h> |
| 20 | 21 | |
| @@ -62,12 +63,18 @@ static DEFINE_MUTEX(dev_status_mem_mutex); · 41 unmodified lines | ||
| 62 | 63 | * also checked. Returns 0 on success or errno value on failure. |
| 63 | 64 | */ |
| 64 | 65 | int cca_check_secaeskeytoken(debug_info_t *dbg, int dbflvl, |
| 65 | − const u8 *token, int keybitsize) | |
| 66 | + const u8 *token, u32 keysize, int keybitsize) | |
| 66 | 67 | { |
| 67 | 68 | struct secaeskeytoken *t = (struct secaeskeytoken *)token; |
| 68 | 69 | |
| 69 | 70 | #define DBF(...) debug_sprintf_event(dbg, dbflvl, ##__VA_ARGS__) |
| 70 | 71 | |
| 72 | + if (keysize < sizeof(*t)) { | |
| 73 | + if (dbg) | |
| 74 | + DBF("%s keysize %u < min token size %zu\n", | |
| 75 | + __func__, keysize, sizeof(*t)); | |
| 76 | + return -EINVAL; | |
| 77 | + } | |
| 71 | 78 | if (t->type != TOKTYPE_CCA_INTERNAL) { |
| 72 | 79 | if (dbg) |
| 73 | 80 | DBF("%s token check failed, type 0x%02x != 0x%02x\n", |
| @@ -101,14 +108,20 @@ EXPORT_SYMBOL(cca_check_secaeskeytoken); · 27 unmodified lines | ||
| 101 | 108 | * Returns 0 on success or errno value on failure. |
| 102 | 109 | */ |
| 103 | 110 | int cca_check_secaescipherkey(debug_info_t *dbg, int dbflvl, |
| 104 | − const u8 *token, int keybitsize, | |
| 105 | − int checkcpacfexport) | |
| 111 | + const u8 *token, u32 keysize, | |
| 112 | + int keybitsize, int checkcpacfexport) | |
| 106 | 113 | { |
| 107 | 114 | struct cipherkeytoken *t = (struct cipherkeytoken *)token; |
| 108 | 115 | bool keybitsizeok = true; |
| 109 | 116 | |
| 110 | 117 | #define DBF(...) debug_sprintf_event(dbg, dbflvl, ##__VA_ARGS__) |
| 111 | 118 | |
| 119 | + if (keysize < sizeof(*t)) { | |
| 120 | + if (dbg) | |
| 121 | + DBF("%s keysize %u < min token size %zu\n", | |
| 122 | + __func__, keysize, sizeof(*t)); | |
| 123 | + return -EINVAL; | |
| 124 | + } | |
| 112 | 125 | if (t->type != TOKTYPE_CCA_INTERNAL) { |
| 113 | 126 | if (dbg) |
| 114 | 127 | DBF("%s token check failed, type 0x%02x != 0x%02x\n", |
| @@ -121,6 +134,18 @@ int cca_check_secaescipherkey(debug_info_t *dbg, int dbflvl, · 6 unmodified lines | ||
| 121 | 134 | __func__, (int)t->version, TOKVER_CCA_VLSC); |
| 122 | 135 | return -EINVAL; |
| 123 | 136 | } |
| 137 | + if (t->len > keysize) { | |
| 138 | + if (dbg) | |
| 139 | + DBF("%s token check failed, len %d > keysize %u\n", | |
| 140 | + __func__, (int)t->len, keysize); | |
| 141 | + return -EINVAL; | |
| 142 | + } | |
| 143 | + if (t->len < sizeof(*t)) { | |
| 144 | + if (dbg) | |
| 145 | + DBF("%s token check failed, len %d < min token size %zu\n", | |
| 146 | + __func__, (int)t->len, sizeof(*t)); | |
| 147 | + return -EINVAL; | |
| 148 | + } | |
| 124 | 149 | if (t->algtype != 0x02) { |
| 125 | 150 | if (dbg) |
| 126 | 151 | DBF("%s token check failed, algtype 0x%02x != 0x02\n", |
| @@ -195,6 +220,12 @@ int cca_check_sececckeytoken(debug_info_t *dbg, int dbflvl, · 68 unmodified lines | ||
| 195 | 220 | |
| 196 | 221 | #define DBF(...) debug_sprintf_event(dbg, dbflvl, ##__VA_ARGS__) |
| 197 | 222 | |
| 223 | + if (keysize < sizeof(*t)) { | |
| 224 | + if (dbg) | |
| 225 | + DBF("%s keysize %u < min token size %zu\n", | |
| 226 | + __func__, keysize, sizeof(*t)); | |
| 227 | + return -EINVAL; | |
| 228 | + } | |
| 198 | 229 | if (t->type != TOKTYPE_CCA_INTERNAL_PKA) { |
| 199 | 230 | if (dbg) |
| 200 | 231 | DBF("%s token check failed, type 0x%02x != 0x%02x\n", |
| @@ -207,6 +238,12 @@ int cca_check_sececckeytoken(debug_info_t *dbg, int dbflvl, · 6 unmodified lines | ||
| 207 | 238 | __func__, (int)t->len, keysize); |
| 208 | 239 | return -EINVAL; |
| 209 | 240 | } |
| 241 | + if (t->len < sizeof(*t)) { | |
| 242 | + if (dbg) | |
| 243 | + DBF("%s token check failed, len %d < min token size %zu\n", | |
| 244 | + __func__, (int)t->len, sizeof(*t)); | |
| 245 | + return -EINVAL; | |
| 246 | + } | |
| 210 | 247 | if (t->secid != 0x20) { |
| 211 | 248 | if (dbg) |
| 212 | 249 | DBF("%s token check failed, secid 0x%02x != 0x20\n", |
| @@ -231,6 +268,10 @@ EXPORT_SYMBOL(cca_check_sececckeytoken); · 18 unmodified lines | ||
| 231 | 268 | * block, reply CPRB and reply param block and fill in values |
| 232 | 269 | * for the common fields. Returns 0 on success or errno value |
| 233 | 270 | * on failure. |
| 271 | + * It is guaranteed that request and a possible param block | |
| 272 | + * are aligned to a 4 byte boundary. Furthermore if a param | |
| 273 | + * block is used, the memory allocated for this is rounded up to | |
| 274 | + * the next multiple of 4 bytes. | |
| 234 | 275 | */ |
| 235 | 276 | static int alloc_and_prep_cprbmem(size_t paramblen, |
| 236 | 277 | u8 **p_cprb_mem, |
| @@ -239,7 +280,8 @@ static int alloc_and_prep_cprbmem(size_t paramblen, · 2 unmodified lines | ||
| 239 | 280 | u32 xflags) |
| 240 | 281 | { |
| 241 | 282 | u8 *cprbmem = NULL; |
| 242 | − size_t cprbplusparamblen = sizeof(struct CPRBX) + paramblen; | |
| 283 | + size_t cprbplusparamblen = | |
| 284 | + ALIGN(sizeof(struct CPRBX), 4) + ALIGN(paramblen, 4); | |
| 243 | 285 | size_t len = 2 * cprbplusparamblen; |
| 244 | 286 | struct CPRBX *preqcblk, *prepcblk; |
| 245 | 287 | |
| @@ -266,10 +308,10 @@ static int alloc_and_prep_cprbmem(size_t paramblen, · 20 unmodified lines | ||
| 266 | 308 | memcpy(preqcblk->func_id, "T2", 2); |
| 267 | 309 | preqcblk->rpl_msgbl = cprbplusparamblen; |
| 268 | 310 | if (paramblen) { |
| 269 | − preqcblk->req_parmb = | |
| 270 | − ((u8 __user *)preqcblk) + sizeof(struct CPRBX); | |
| 271 | − preqcblk->rpl_parmb = | |
| 272 | − ((u8 __user *)prepcblk) + sizeof(struct CPRBX); | |
| 311 | + preqcblk->req_parmb = ((u8 __user *)preqcblk) + | |
| 312 | + ALIGN(sizeof(struct CPRBX), 4); | |
| 313 | + preqcblk->rpl_parmb = ((u8 __user *)prepcblk) + | |
| 314 | + ALIGN(sizeof(struct CPRBX), 4); | |
| 273 | 315 | } |
| 274 | 316 | |
| 275 | 317 | *p_cprb_mem = cprbmem; |
| @@ -287,8 +329,10 @@ static int alloc_and_prep_cprbmem(size_t paramblen, · 11 unmodified lines | ||
| 287 | 329 | */ |
| 288 | 330 | static void free_cprbmem(void *mem, size_t paramblen, bool scrub, u32 xflags) |
| 289 | 331 | { |
| 332 | + size_t cprblen = ALIGN(sizeof(struct CPRBX), 4) + ALIGN(paramblen, 4); | |
| 333 | + | |
| 290 | 334 | if (mem && scrub) |
| 291 | − memzero_explicit(mem, 2 * (sizeof(struct CPRBX) + paramblen)); | |
| 335 | + memzero_explicit(mem, 2 * cprblen); | |
| 292 | 336 | |
| 293 | 337 | if (xflags & ZCRYPT_XFLAG_NOMEMALLOC) |
| 294 | 338 | mempool_free(mem, cprb_mempool); |
| @@ -443,7 +487,8 @@ int cca_genseckey(u16 cardnr, u16 domain, · 148 unmodified lines | ||
| 443 | 487 | |
| 444 | 488 | /* check secure key token */ |
| 445 | 489 | rc = cca_check_secaeskeytoken(zcrypt_dbf_info, DBF_ERR, |
| 446 | − prepparm->lv3.keyblock.tok, 8 * keysize); | |
| 490 | + prepparm->lv3.keyblock.tok, | |
| 491 | + seckeysize, 8 * keysize); | |
| 447 | 492 | if (rc) { |
| 448 | 493 | rc = -EIO; |
| 449 | 494 | goto out; |
| @@ -582,7 +627,8 @@ int cca_clr2seckey(u16 cardnr, u16 domain, u32 keybitsize, · 132 unmodified lines | ||
| 582 | 627 | |
| 583 | 628 | /* check secure key token */ |
| 584 | 629 | rc = cca_check_secaeskeytoken(zcrypt_dbf_info, DBF_ERR, |
| 585 | − prepparm->lv3.keyblock.tok, 8 * keysize); | |
| 630 | + prepparm->lv3.keyblock.tok, | |
| 631 | + seckeysize, 8 * keysize); | |
| 586 | 632 | if (rc) { |
| 587 | 633 | rc = -EIO; |
| 588 | 634 | goto out; |
| @@ -842,6 +888,7 @@ int cca_gencipherkey(u16 cardnr, u16 domain, u32 keybitsize, u32 keygenflags, · 253 unmodified lines | ||
| 842 | 888 | } kb; |
| 843 | 889 | } __packed * prepparm; |
| 844 | 890 | struct cipherkeytoken *t; |
| 891 | + u32 keybuflen; | |
| 845 | 892 | |
| 846 | 893 | /* get already prepared memory for 2 cprbs with param block each */ |
| 847 | 894 | rc = alloc_and_prep_cprbmem(PARMBSIZE, &mem, |
| @@ -936,23 +983,28 @@ int cca_gencipherkey(u16 cardnr, u16 domain, u32 keybitsize, u32 keygenflags, · 88 unmodified lines | ||
| 936 | 983 | } |
| 937 | 984 | |
| 938 | 985 | /* and some checks on the generated key */ |
| 986 | + t = (struct cipherkeytoken *)prepparm->kb.tlv1.gen_key; | |
| 987 | + if (prepparm->kb.tlv1.len < 2 * sizeof(uint16_t) + sizeof(*t)) { | |
| 988 | + rc = -EIO; | |
| 989 | + goto out; | |
| 990 | + } | |
| 991 | + keybuflen = prepparm->kb.tlv1.len - 2 * sizeof(uint16_t); | |
| 939 | 992 | rc = cca_check_secaescipherkey(zcrypt_dbf_info, DBF_ERR, |
| 940 | 993 | prepparm->kb.tlv1.gen_key, |
| 941 | − keybitsize, 1); | |
| 994 | + keybuflen, keybitsize, 1); | |
| 942 | 995 | if (rc) { |
| 943 | 996 | rc = -EIO; |
| 944 | 997 | goto out; |
| 945 | 998 | } |
| 946 | 999 | |
| 947 | 1000 | /* copy the generated vlsc key token */ |
| 948 | − t = (struct cipherkeytoken *)prepparm->kb.tlv1.gen_key; | |
| 949 | 1001 | if (keybuf) { |
| 950 | − if (*keybufsize >= t->len) | |
| 951 | − memcpy(keybuf, t, t->len); | |
| 1002 | + if (*keybufsize >= keybuflen) | |
| 1003 | + memcpy(keybuf, t, keybuflen); | |
| 952 | 1004 | else |
| 953 | 1005 | rc = -EINVAL; |
| 954 | 1006 | } |
| 955 | − *keybufsize = t->len; | |
| 1007 | + *keybufsize = keybuflen; | |
| 956 | 1008 | |
| 957 | 1009 | out: |
| 958 | 1010 | free_cprbmem(mem, PARMBSIZE, false, xflags); |
| @@ -971,7 +1023,8 @@ static int _ip_cprb_helper(u16 cardnr, u16 domain, · 12 unmodified lines | ||
| 971 | 1023 | int clr_key_bit_size, |
| 972 | 1024 | u8 *key_token, |
| 973 | 1025 | int *key_token_size, |
| 974 | − u32 xflags) | |
| 1026 | + u32 xflags, | |
| 1027 | + bool scrub) | |
| 975 | 1028 | { |
| 976 | 1029 | int rc, n; |
| 977 | 1030 | u8 *mem, *ptr; |
| @@ -1111,7 +1164,7 @@ static int _ip_cprb_helper(u16 cardnr, u16 domain, · 133 unmodified lines | ||
| 1111 | 1164 | *key_token_size = t->len; |
| 1112 | 1165 | |
| 1113 | 1166 | out: |
| 1114 | − free_cprbmem(mem, PARMBSIZE, false, xflags); | |
| 1167 | + free_cprbmem(mem, PARMBSIZE, scrub, xflags); | |
| 1115 | 1168 | return rc; |
| 1116 | 1169 | } |
| 1117 | 1170 | |
| @@ -1162,28 +1215,32 @@ int cca_clr2cipherkey(u16 card, u16 dom, u32 keybitsize, u32 keygenflags, · 44 unmodified lines | ||
| 1162 | 1215 | * 4/4 COMPLETE the secure cipher key import |
| 1163 | 1216 | */ |
| 1164 | 1217 | rc = _ip_cprb_helper(card, dom, "AES ", "FIRST ", "MIN3PART", |
| 1165 | − exorbuf, keybitsize, token, &tokensize, xflags); | |
| 1218 | + exorbuf, keybitsize, token, &tokensize, | |
| 1219 | + xflags, true); | |
| 1166 | 1220 | if (rc) { |
| 1167 | 1221 | ZCRYPT_DBF_ERR("%s clear key import 1/4 with CSNBKPI2 failed, rc=%d\n", |
| 1168 | 1222 | __func__, rc); |
| 1169 | 1223 | goto out; |
| 1170 | 1224 | } |
| 1171 | 1225 | rc = _ip_cprb_helper(card, dom, "AES ", "ADD-PART", NULL, |
| 1172 | − clrkey, keybitsize, token, &tokensize, xflags); | |
| 1226 | + clrkey, keybitsize, token, &tokensize, | |
| 1227 | + xflags, true); | |
| 1173 | 1228 | if (rc) { |
| 1174 | 1229 | ZCRYPT_DBF_ERR("%s clear key import 2/4 with CSNBKPI2 failed, rc=%d\n", |
| 1175 | 1230 | __func__, rc); |
| 1176 | 1231 | goto out; |
| 1177 | 1232 | } |
| 1178 | 1233 | rc = _ip_cprb_helper(card, dom, "AES ", "ADD-PART", NULL, |
| 1179 | − exorbuf, keybitsize, token, &tokensize, xflags); | |
| 1234 | + exorbuf, keybitsize, token, &tokensize, | |
| 1235 | + xflags, true); | |
| 1180 | 1236 | if (rc) { |
| 1181 | 1237 | ZCRYPT_DBF_ERR("%s clear key import 3/4 with CSNBKPI2 failed, rc=%d\n", |
| 1182 | 1238 | __func__, rc); |
| 1183 | 1239 | goto out; |
| 1184 | 1240 | } |
| 1185 | 1241 | rc = _ip_cprb_helper(card, dom, "AES ", "COMPLETE", NULL, |
| 1186 | − NULL, keybitsize, token, &tokensize, xflags); | |
| 1242 | + NULL, keybitsize, token, &tokensize, | |
| 1243 | + xflags, true); | |
| 1187 | 1244 | if (rc) { |
| 1188 | 1245 | ZCRYPT_DBF_ERR("%s clear key import 4/4 with CSNBKPI2 failed, rc=%d\n", |
| 1189 | 1246 | __func__, rc); |
| @@ -1200,6 +1257,8 @@ int cca_clr2cipherkey(u16 card, u16 dom, u32 keybitsize, u32 keygenflags, · 10 unmodified lines | ||
| 1200 | 1257 | *keybufsize = tokensize; |
| 1201 | 1258 | |
| 1202 | 1259 | out: |
| 1260 | + memzero_explicit(exorbuf, sizeof(exorbuf)); | |
| 1261 | + memzero_explicit(mem, CPRB_MEMPOOL_ITEM_SIZE); | |
| 1203 | 1262 | mempool_free(mem, cprb_mempool); |
| 1204 | 1263 | return rc; |
| 1205 | 1264 | } |
| @@ -1261,6 +1320,9 @@ int cca_cipher2protkey(u16 cardnr, u16 domain, const u8 *ckey, · 55 unmodified lines | ||
| 1261 | 1320 | } __packed * prepparm; |
| 1262 | 1321 | int keytoklen = ((struct cipherkeytoken *)ckey)->len; |
| 1263 | 1322 | |
| 1323 | + if (keytoklen > PARMBSIZE - sizeof(struct aureqparm)) | |
| 1324 | + return -EINVAL; | |
| 1325 | + | |
| 1264 | 1326 | /* get already prepared memory for 2 cprbs with param block each */ |
| 1265 | 1327 | rc = alloc_and_prep_cprbmem(PARMBSIZE, &mem, |
| 1266 | 1328 | &preqcblk, &prepcblk, xflags); |
| @@ -1425,6 +1487,9 @@ int cca_ecc2protkey(u16 cardnr, u16 domain, const u8 *key, · 158 unmodified lines | ||
| 1425 | 1487 | } __packed * prepparm; |
| 1426 | 1488 | int keylen = ((struct eccprivkeytoken *)key)->len; |
| 1427 | 1489 | |
| 1490 | + if (keylen > PARMBSIZE - sizeof(struct aureqparm)) | |
| 1491 | + return -EINVAL; | |
| 1492 | + | |
| 1428 | 1493 | /* get already prepared memory for 2 cprbs with param block each */ |
| 1429 | 1494 | rc = alloc_and_prep_cprbmem(PARMBSIZE, &mem, |
| 1430 | 1495 | &preqcblk, &prepcblk, xflags); |
| @@ -136,7 +136,7 @@ struct eccprivkeytoken { · 135 unmodified lines | ||
| 136 | 136 | * also checked. Returns 0 on success or errno value on failure. |
| 137 | 137 | */ |
| 138 | 138 | int cca_check_secaeskeytoken(debug_info_t *dbg, int dbflvl, |
| 139 | − const u8 *token, int keybitsize); | |
| 139 | + const u8 *token, u32 keysize, int keybitsize); | |
| 140 | 140 | |
| 141 | 141 | /* |
| 142 | 142 | * Simple check if the token is a valid CCA secure AES cipher key |
| @@ -146,8 +146,8 @@ int cca_check_secaeskeytoken(debug_info_t *dbg, int dbflvl, · 3 unmodified lines | ||
| 146 | 146 | * Returns 0 on success or errno value on failure. |
| 147 | 147 | */ |
| 148 | 148 | int cca_check_secaescipherkey(debug_info_t *dbg, int dbflvl, |
| 149 | − const u8 *token, int keybitsize, | |
| 150 | − int checkcpacfexport); | |
| 149 | + const u8 *token, u32 keysize, | |
| 150 | + int keybitsize, int checkcpacfexport); | |
| 151 | 151 | |
| 152 | 152 | /* |
| 153 | 153 | * Simple check if the token is a valid CCA secure ECC private |
| @@ -14,6 +14,7 @@ · 13 unmodified lines | ||
| 14 | 14 | #include <linux/module.h> |
| 15 | 15 | #include <linux/random.h> |
| 16 | 16 | #include <linux/slab.h> |
| 17 | +#include <linux/align.h> | |
| 17 | 18 | #include <asm/zcrypt.h> |
| 18 | 19 | #include <asm/pkey.h> |
| 19 | 20 | #include <crypto/aes.h> |
| @@ -355,21 +356,24 @@ EXPORT_SYMBOL(ep11_check_aes_key); · 335 unmodified lines | ||
| 355 | 356 | |
| 356 | 357 | /* |
| 357 | 358 | * Allocate and prepare ep11 cprb plus additional payload. |
| 359 | + * It is guaranteed that the memory is aligned to a 4 byte boundary. | |
| 360 | + * Furthermore the memory allocation is rounded up to the next | |
| 361 | + * multiple of 4 bytes (with taking the payload_len into account). | |
| 358 | 362 | */ |
| 359 | 363 | static void *alloc_cprbmem(size_t payload_len, u32 xflags) |
| 360 | 364 | { |
| 361 | − size_t len = sizeof(struct ep11_cprb) + payload_len; | |
| 365 | + size_t memlen = ALIGN(sizeof(struct ep11_cprb) + payload_len, 4); | |
| 362 | 366 | struct ep11_cprb *cprb = NULL; |
| 363 | 367 | |
| 364 | 368 | if (xflags & ZCRYPT_XFLAG_NOMEMALLOC) { |
| 365 | − if (len <= CPRB_MEMPOOL_ITEM_SIZE) | |
| 369 | + if (memlen <= CPRB_MEMPOOL_ITEM_SIZE) | |
| 366 | 370 | cprb = mempool_alloc_preallocated(cprb_mempool); |
| 367 | 371 | } else { |
| 368 | − cprb = kmalloc(len, GFP_KERNEL); | |
| 372 | + cprb = kmalloc(memlen, GFP_KERNEL); | |
| 369 | 373 | } |
| 370 | 374 | if (!cprb) |
| 371 | 375 | return NULL; |
| 372 | − memset(cprb, 0, len); | |
| 376 | + memset(cprb, 0, memlen); | |
| 373 | 377 | |
| 374 | 378 | cprb->cprb_len = sizeof(struct ep11_cprb); |
| 375 | 379 | cprb->cprb_ver_id = 0x04; |
| @@ -385,8 +389,10 @@ static void *alloc_cprbmem(size_t payload_len, u32 xflags) · 9 unmodified lines | ||
| 385 | 389 | */ |
| 386 | 390 | static void free_cprbmem(void *mem, size_t payload_len, bool scrub, u32 xflags) |
| 387 | 391 | { |
| 392 | + size_t memlen = ALIGN(sizeof(struct ep11_cprb) + payload_len, 4); | |
| 393 | + | |
| 388 | 394 | if (mem && scrub) |
| 389 | − memzero_explicit(mem, sizeof(struct ep11_cprb) + payload_len); | |
| 395 | + memzero_explicit(mem, memlen); | |
| 390 | 396 | |
| 391 | 397 | if (xflags & ZCRYPT_XFLAG_NOMEMALLOC) |
| 392 | 398 | mempool_free(mem, cprb_mempool); |
| @@ -19,6 +19,7 @@ · 18 unmodified lines | ||
| 19 | 19 | #include <linux/slab.h> |
| 20 | 20 | #include <linux/atomic.h> |
| 21 | 21 | #include <linux/uaccess.h> |
| 22 | +#include <linux/unaligned.h> | |
| 22 | 23 | |
| 23 | 24 | #include "ap_bus.h" |
| 24 | 25 | #include "zcrypt_api.h" |
| @@ -34,6 +35,9 @@ · 9 unmodified lines | ||
| 34 | 35 | #define CEXXC_RESPONSE_TYPE_XCRB 1 |
| 35 | 36 | #define CEXXC_RESPONSE_TYPE_EP11 2 |
| 36 | 37 | |
| 38 | +/* smallest possible EP11 payload size */ | |
| 39 | +#define MIN_EP11_PAYLOAD_SIZE 5 | |
| 40 | + | |
| 37 | 41 | MODULE_AUTHOR("IBM Corporation"); |
| 38 | 42 | MODULE_DESCRIPTION("Cryptographic Coprocessor (message type 6), " \ |
| 39 | 43 | "Copyright IBM Corp. 2001, 2023"); |
| @@ -342,49 +346,39 @@ static int xcrb_msg_to_type6cprb_msgx(bool userspace, struct ap_message *ap_msg, · 302 unmodified lines | ||
| 342 | 346 | }; |
| 343 | 347 | } __packed * msg = ap_msg->msg; |
| 344 | 348 | |
| 345 | − int rcblen = CEIL4(xcrb->request_control_blk_length); | |
| 346 | − int req_sumlen, resp_sumlen; | |
| 347 | − char *req_data = ap_msg->msg + sizeof(struct type6_hdr) + rcblen; | |
| 348 | − char *function_code; | |
| 349 | − | |
| 350 | − if (CEIL4(xcrb->request_control_blk_length) < | |
| 351 | − xcrb->request_control_blk_length) | |
| 352 | − return -EINVAL; /* overflow after alignment*/ | |
| 349 | + size_t req_cblen, rep_cblen, req_sumlen, rep_sumlen; | |
| 350 | + char *function_code, *req_data; | |
| 353 | 351 | |
| 354 | − /* length checks */ | |
| 355 | − ap_msg->len = sizeof(struct type6_hdr) + | |
| 356 | − CEIL4(xcrb->request_control_blk_length) + | |
| 357 | − xcrb->request_data_length; | |
| 352 | + /* request length and overflow checks */ | |
| 353 | + if (xcrb->request_control_blk_length < sizeof(struct CPRBX)) | |
| 354 | + return -EINVAL; | |
| 355 | + req_cblen = CEIL4((size_t)xcrb->request_control_blk_length); | |
| 356 | + if (req_cblen > U32_MAX) | |
| 357 | + return -EINVAL; | |
| 358 | + req_sumlen = req_cblen + xcrb->request_data_length; | |
| 359 | + if (req_sumlen > U32_MAX) | |
| 360 | + return -EINVAL; | |
| 361 | + ap_msg->len = sizeof(struct type6_hdr) + req_sumlen; | |
| 358 | 362 | if (ap_msg->len > ap_msg->bufsize) |
| 359 | 363 | return -EINVAL; |
| 360 | − | |
| 361 | − /* | |
| 362 | − * Overflow check | |
| 363 | − * sum must be greater (or equal) than the largest operand | |
| 364 | − */ | |
| 365 | − req_sumlen = CEIL4(xcrb->request_control_blk_length) + | |
| 366 | − xcrb->request_data_length; | |
| 367 | − if ((CEIL4(xcrb->request_control_blk_length) <= | |
| 368 | − xcrb->request_data_length) ? | |
| 364 | + if (req_cblen <= xcrb->request_data_length ? | |
| 369 | 365 | req_sumlen < xcrb->request_data_length : |
| 370 | − req_sumlen < CEIL4(xcrb->request_control_blk_length)) { | |
| 366 | + req_sumlen < req_cblen) { | |
| 371 | 367 | return -EINVAL; |
| 372 | 368 | } |
| 373 | 369 | |
| 374 | − if (CEIL4(xcrb->reply_control_blk_length) < | |
| 375 | − xcrb->reply_control_blk_length) | |
| 376 | − return -EINVAL; /* overflow after alignment*/ | |
| 377 | − | |
| 378 | − /* | |
| 379 | − * Overflow check | |
| 380 | − * sum must be greater (or equal) than the largest operand | |
| 381 | − */ | |
| 382 | − resp_sumlen = CEIL4(xcrb->reply_control_blk_length) + | |
| 383 | − xcrb->reply_data_length; | |
| 384 | − if ((CEIL4(xcrb->reply_control_blk_length) <= | |
| 385 | − xcrb->reply_data_length) ? | |
| 386 | − resp_sumlen < xcrb->reply_data_length : | |
| 387 | − resp_sumlen < CEIL4(xcrb->reply_control_blk_length)) { | |
| 370 | + /* reply length and overflow checks */ | |
| 371 | + if (xcrb->reply_control_blk_length < sizeof(struct CPRBX)) | |
| 372 | + return -EINVAL; | |
| 373 | + rep_cblen = CEIL4((size_t)xcrb->reply_control_blk_length); | |
| 374 | + if (rep_cblen > U32_MAX) | |
| 375 | + return -EINVAL; | |
| 376 | + rep_sumlen = rep_cblen + xcrb->reply_data_length; | |
| 377 | + if (rep_sumlen > U32_MAX) | |
| 378 | + return -EINVAL; | |
| 379 | + if (rep_cblen <= xcrb->reply_data_length ? | |
| 380 | + rep_sumlen < xcrb->reply_data_length : | |
| 381 | + rep_sumlen < rep_cblen) { | |
| 388 | 382 | return -EINVAL; |
| 389 | 383 | } |
| 390 | 384 | |
| @@ -393,7 +387,7 @@ static int xcrb_msg_to_type6cprb_msgx(bool userspace, struct ap_message *ap_msg, · 2 unmodified lines | ||
| 393 | 387 | memcpy(msg->hdr.agent_id, &xcrb->agent_ID, sizeof(xcrb->agent_ID)); |
| 394 | 388 | msg->hdr.tocardlen1 = xcrb->request_control_blk_length; |
| 395 | 389 | if (xcrb->request_data_length) { |
| 396 | − msg->hdr.offset2 = msg->hdr.offset1 + rcblen; | |
| 390 | + msg->hdr.offset2 = msg->hdr.offset1 + req_cblen; | |
| 397 | 391 | msg->hdr.tocardlen2 = xcrb->request_data_length; |
| 398 | 392 | } |
| 399 | 393 | msg->hdr.fromcardlen1 = xcrb->reply_control_blk_length; |
| @@ -404,8 +398,12 @@ static int xcrb_msg_to_type6cprb_msgx(bool userspace, struct ap_message *ap_msg, · 4 unmodified lines | ||
| 404 | 398 | xcrb->request_control_blk_addr, |
| 405 | 399 | xcrb->request_control_blk_length)) |
| 406 | 400 | return -EFAULT; |
| 407 | − if (msg->cprbx.cprb_len + sizeof(msg->hdr.function_code) > | |
| 408 | − xcrb->request_control_blk_length) | |
| 401 | + /* pad tail with 0 up to req_cblen */ | |
| 402 | + if (xcrb->request_control_blk_length < req_cblen) | |
| 403 | + memset(msg->userdata + xcrb->request_control_blk_length, | |
| 404 | + 0, req_cblen - xcrb->request_control_blk_length); | |
| 405 | + /* copy subfunction code into AP msg type 6 function code field */ | |
| 406 | + if (msg->cprbx.cprb_len > req_cblen - sizeof(msg->hdr.function_code)) | |
| 409 | 407 | return -EINVAL; |
| 410 | 408 | function_code = ((unsigned char *)&msg->cprbx) + msg->cprbx.cprb_len; |
| 411 | 409 | memcpy(msg->hdr.function_code, function_code, |
| @@ -437,20 +435,70 @@ static int xcrb_msg_to_type6cprb_msgx(bool userspace, struct ap_message *ap_msg, · 25 unmodified lines | ||
| 437 | 435 | } |
| 438 | 436 | |
| 439 | 437 | /* copy data block */ |
| 440 | − if (xcrb->request_data_length && | |
| 441 | − z_copy_from_user(userspace, req_data, xcrb->request_data_address, | |
| 442 | − xcrb->request_data_length)) | |
| 443 | − return -EFAULT; | |
| 438 | + if (xcrb->request_data_length) { | |
| 439 | + req_data = ap_msg->msg + sizeof(struct type6_hdr) + req_cblen; | |
| 440 | + if (z_copy_from_user(userspace, req_data, | |
| 441 | + xcrb->request_data_address, | |
| 442 | + xcrb->request_data_length)) | |
| 443 | + return -EFAULT; | |
| 444 | + } | |
| 444 | 445 | |
| 445 | 446 | return 0; |
| 446 | 447 | } |
| 447 | 448 | |
| 449 | +/* | |
| 450 | + * Simple asn1 int reader/decoder helper function | |
| 451 | + * Returns number of bytes processed or < 0 on failure | |
| 452 | + * Only accepts int length values of 1, 2 or 4. | |
| 453 | + */ | |
| 454 | +static inline int asn1_int_decode(const u8 *buf, size_t intlen, u32 *u) | |
| 455 | +{ | |
| 456 | + switch (intlen) { | |
| 457 | + case 1: | |
| 458 | + *u = (u32)(*buf); | |
| 459 | + return 1; | |
| 460 | + case 2: | |
| 461 | + *u = (u32)get_unaligned_be16(buf); | |
| 462 | + return 2; | |
| 463 | + case 4: | |
| 464 | + *u = (u32)get_unaligned_be32(buf); | |
| 465 | + return 4; | |
| 466 | + default: | |
| 467 | + return -EINVAL; | |
| 468 | + } | |
| 469 | +} | |
| 470 | + | |
| 471 | +/* | |
| 472 | + * Simple asn1 length parse helper function | |
| 473 | + * Returns number of bytes processed or < 0 on failure | |
| 474 | + * Only accepts length encoded within the length octet | |
| 475 | + * or for long form 1, 2 or 4 octet length bytes. | |
| 476 | + */ | |
| 477 | +static inline int asn1_length_decode(const u8 *buf, size_t buflen, u32 *u) | |
| 478 | +{ | |
| 479 | + int i; | |
| 480 | + | |
| 481 | + if (buflen < 1) | |
| 482 | + return -EINVAL; | |
| 483 | + | |
| 484 | + if (*buf < 128) { | |
| 485 | + *u = (u32)(*buf & 0x7F); | |
| 486 | + return 1; | |
| 487 | + } | |
| 488 | + | |
| 489 | + i = *buf & 0x7F; | |
| 490 | + if (--buflen < i) | |
| 491 | + return -EINVAL; | |
| 492 | + i = asn1_int_decode(++buf, i, u); | |
| 493 | + | |
| 494 | + return i < 0 ? i : i + 1; | |
| 495 | +} | |
| 496 | + | |
| 448 | 497 | static int xcrb_msg_to_type6_ep11cprb_msgx(bool userspace, struct ap_message *ap_msg, |
| 449 | 498 | struct ep11_urb *xcrb, |
| 450 | 499 | unsigned int *fcode, |
| 451 | 500 | unsigned int *domain) |
| 452 | 501 | { |
| 453 | − unsigned int lfmt; | |
| 454 | 502 | static struct type6_hdr static_type6_ep11_hdr = { |
| 455 | 503 | .type = 0x06, |
| 456 | 504 | .rqid = {0x00, 0x01}, |
| @@ -462,34 +510,32 @@ static int xcrb_msg_to_type6_ep11cprb_msgx(bool userspace, struct ap_message *ap · 5 unmodified lines | ||
| 462 | 510 | struct { |
| 463 | 511 | struct type6_hdr hdr; |
| 464 | 512 | union { |
| 465 | − struct { | |
| 466 | − struct ep11_cprb cprbx; | |
| 467 | − unsigned char pld_tag; /* fixed value 0x30 */ | |
| 468 | − unsigned char pld_lenfmt; /* length format */ | |
| 469 | − } __packed; | |
| 513 | + struct ep11_cprb cprbx; | |
| 470 | 514 | DECLARE_FLEX_ARRAY(u8, userdata); |
| 471 | 515 | }; |
| 472 | 516 | } __packed * msg = ap_msg->msg; |
| 473 | 517 | |
| 474 | − struct pld_hdr { | |
| 475 | − unsigned char func_tag; /* fixed value 0x4 */ | |
| 476 | − unsigned char func_len; /* fixed value 0x4 */ | |
| 477 | − unsigned int func_val; /* function ID */ | |
| 478 | − unsigned char dom_tag; /* fixed value 0x4 */ | |
| 479 | − unsigned char dom_len; /* fixed value 0x4 */ | |
| 480 | − unsigned int dom_val; /* domain id */ | |
| 481 | − } __packed * payload_hdr = NULL; | |
| 518 | + size_t req_len, rep_len, pld_len; | |
| 519 | + unsigned char *pld; | |
| 520 | + int offs = 0, i; | |
| 521 | + unsigned int u; | |
| 482 | 522 | |
| 483 | − if (CEIL4(xcrb->req_len) < xcrb->req_len) | |
| 484 | − return -EINVAL; /* overflow after alignment*/ | |
| 485 | − | |
| 486 | − /* length checks */ | |
| 487 | − ap_msg->len = sizeof(struct type6_hdr) + CEIL4(xcrb->req_len); | |
| 523 | + /* request length and overflow checks */ | |
| 524 | + if (xcrb->req_len < sizeof(struct ep11_cprb) + MIN_EP11_PAYLOAD_SIZE) | |
| 525 | + return -EINVAL; | |
| 526 | + req_len = CEIL4(xcrb->req_len); | |
| 527 | + if (req_len < xcrb->req_len || req_len > U32_MAX) | |
| 528 | + return -EINVAL; | |
| 529 | + ap_msg->len = sizeof(struct type6_hdr) + req_len; | |
| 488 | 530 | if (ap_msg->len > ap_msg->bufsize) |
| 489 | 531 | return -EINVAL; |
| 490 | 532 | |
| 491 | − if (CEIL4(xcrb->resp_len) < xcrb->resp_len) | |
| 492 | − return -EINVAL; /* overflow after alignment*/ | |
| 533 | + /* reply length and overflow checks */ | |
| 534 | + if (xcrb->resp_len < sizeof(struct ep11_cprb)) | |
| 535 | + return -EINVAL; | |
| 536 | + rep_len = CEIL4(xcrb->resp_len); | |
| 537 | + if (rep_len < xcrb->resp_len || rep_len > U32_MAX) | |
| 538 | + return -EINVAL; | |
| 493 | 539 | |
| 494 | 540 | /* prepare type6 header */ |
| 495 | 541 | msg->hdr = static_type6_ep11_hdr; |
| @@ -498,26 +544,55 @@ static int xcrb_msg_to_type6_ep11cprb_msgx(bool userspace, struct ap_message *ap · 2 unmodified lines | ||
| 498 | 544 | |
| 499 | 545 | /* Import CPRB data from the ioctl input parameter */ |
| 500 | 546 | if (z_copy_from_user(userspace, msg->userdata, |
| 501 | − (char __force __user *)xcrb->req, xcrb->req_len)) { | |
| 547 | + (char __force __user *)xcrb->req, xcrb->req_len)) | |
| 502 | 548 | return -EFAULT; |
| 503 | − } | |
| 549 | + /* pad tail with 0 up to req_len */ | |
| 550 | + if (xcrb->req_len < req_len) | |
| 551 | + memset(msg->userdata + xcrb->req_len, 0, | |
| 552 | + req_len - xcrb->req_len); | |
| 553 | + | |
| 554 | + pld = msg->userdata + sizeof(struct ep11_cprb); | |
| 555 | + pld_len = msg->cprbx.payload_len; | |
| 556 | + if (pld_len != xcrb->req_len - sizeof(struct ep11_cprb)) | |
| 557 | + return -EINVAL; | |
| 558 | + /* | |
| 559 | + * At this point pld_len is always >= MIN_EP11_PAYLOAD_SIZE | |
| 560 | + * and the smallest supported asn1 payload is: | |
| 561 | + * payload tag (1 octet) | |
| 562 | + * payload length (1-5 octets) | |
| 563 | + * function tag (1 octet) | |
| 564 | + * function length (1-5 octets) | |
| 565 | + * function value (1-4 octets) | |
| 566 | + */ | |
| 504 | 567 | |
| 505 | − if ((msg->pld_lenfmt & 0x80) == 0x80) { /*ext.len.fmt 2 or 3*/ | |
| 506 | − switch (msg->pld_lenfmt & 0x03) { | |
| 507 | − case 1: | |
| 508 | − lfmt = 2; | |
| 509 | − break; | |
| 510 | − case 2: | |
| 511 | − lfmt = 3; | |
| 512 | − break; | |
| 513 | − default: | |
| 514 | − return -EINVAL; | |
| 515 | − } | |
| 516 | − } else { | |
| 517 | − lfmt = 1; /* length format #1 */ | |
| 518 | − } | |
| 519 | − payload_hdr = (struct pld_hdr *)((&msg->pld_lenfmt) + lfmt); | |
| 520 | − *fcode = payload_hdr->func_val & 0xFFFF; | |
| 568 | + /* payload tag */ | |
| 569 | + if (pld[offs++] != 0x30) | |
| 570 | + return -EINVAL; | |
| 571 | + /* payload length field */ | |
| 572 | + i = asn1_length_decode(pld + offs, pld_len - offs, &u); | |
| 573 | + if (i < 0) | |
| 574 | + return -EINVAL; | |
| 575 | + offs += i; | |
| 576 | + if (offs >= pld_len || u > pld_len - offs) | |
| 577 | + return -EINVAL; | |
| 578 | + /* function tag */ | |
| 579 | + if (pld[offs++] != 0x04) | |
| 580 | + return -EINVAL; | |
| 581 | + /* function length */ | |
| 582 | + if (offs >= pld_len) | |
| 583 | + return -EINVAL; | |
| 584 | + i = asn1_length_decode(pld + offs, pld_len - offs, &u); | |
| 585 | + if (i < 0) | |
| 586 | + return -EINVAL; | |
| 587 | + offs += i; | |
| 588 | + if (offs >= pld_len || u > pld_len - offs) | |
| 589 | + return -EINVAL; | |
| 590 | + /* function value */ | |
| 591 | + i = asn1_int_decode(pld + offs, u, &u); | |
| 592 | + if (i < 0) | |
| 593 | + return -EINVAL; | |
| 594 | + offs += i; | |
| 595 | + *fcode = 0xFFFF & u; | |
| 521 | 596 | |
| 522 | 597 | /* enable special processing based on the cprbs flags special bit */ |
| 523 | 598 | if (msg->cprbx.flags & 0x20) |
| @@ -1159,6 +1234,28 @@ int prep_ep11_ap_msg(bool userspace, struct ep11_urb *xcrb, · 635 unmodified lines | ||
| 1159 | 1234 | func_code, domain); |
| 1160 | 1235 | } |
| 1161 | 1236 | |
| 1237 | +/* | |
| 1238 | + * Simple asn1 int writer/encoder helper function | |
| 1239 | + * Returns number of bytes processed or < 0 on failure | |
| 1240 | + * Only accepts int length values of 1, 2 or 4. | |
| 1241 | + */ | |
| 1242 | +static inline int asn1_int_encode(u8 *buf, size_t intlen, u32 u) | |
| 1243 | +{ | |
| 1244 | + switch (intlen) { | |
| 1245 | + case 1: | |
| 1246 | + *buf = (u8)u; | |
| 1247 | + return 1; | |
| 1248 | + case 2: | |
| 1249 | + put_unaligned_be16((u16)u, buf); | |
| 1250 | + return 2; | |
| 1251 | + case 4: | |
| 1252 | + put_unaligned_be32((u32)u, buf); | |
| 1253 | + return 4; | |
| 1254 | + default: | |
| 1255 | + return -EINVAL; | |
| 1256 | + } | |
| 1257 | +} | |
| 1258 | + | |
| 1162 | 1259 | /* |
| 1163 | 1260 | * The request distributor calls this function if it picked the CEX4P |
| 1164 | 1261 | * device to handle a send_ep11_cprb request. |
| @@ -1171,51 +1268,94 @@ static long zcrypt_msgtype6_send_ep11_cprb(bool userspace, struct zcrypt_queue * · 6 unmodified lines | ||
| 1171 | 1268 | struct ap_message *ap_msg) |
| 1172 | 1269 | { |
| 1173 | 1270 | int rc; |
| 1174 | − unsigned int lfmt; | |
| 1175 | 1271 | struct ap_response_type *resp_type = &ap_msg->response; |
| 1176 | 1272 | struct { |
| 1177 | 1273 | struct type6_hdr hdr; |
| 1178 | 1274 | struct ep11_cprb cprbx; |
| 1179 | − unsigned char pld_tag; /* fixed value 0x30 */ | |
| 1180 | − unsigned char pld_lenfmt; /* payload length format */ | |
| 1181 | 1275 | } __packed * msg = ap_msg->msg; |
| 1182 | − struct pld_hdr { | |
| 1183 | − unsigned char func_tag; /* fixed value 0x4 */ | |
| 1184 | − unsigned char func_len; /* fixed value 0x4 */ | |
| 1185 | − unsigned int func_val; /* function ID */ | |
| 1186 | − unsigned char dom_tag; /* fixed value 0x4 */ | |
| 1187 | − unsigned char dom_len; /* fixed value 0x4 */ | |
| 1188 | − unsigned int dom_val; /* domain id */ | |
| 1189 | − } __packed * payload_hdr = NULL; | |
| 1190 | 1276 | |
| 1191 | 1277 | /* |
| 1192 | 1278 | * The target domain field within the cprb body/payload block will be |
| 1193 | 1279 | * replaced by the usage domain for non-management commands only. |
| 1194 | 1280 | * Therefore we check the first bit of the 'flags' parameter for |
| 1195 | 1281 | * management command indication. |
| 1196 | − * 0 - non management command | |
| 1197 | − * 1 - management command | |
| 1198 | 1282 | */ |
| 1199 | − if (!((msg->cprbx.flags & 0x80) == 0x80)) { | |
| 1200 | − msg->cprbx.target_id = (unsigned int) | |
| 1201 | − AP_QID_QUEUE(zq->queue->qid); | |
| 1202 | − | |
| 1203 | − if ((msg->pld_lenfmt & 0x80) == 0x80) { /*ext.len.fmt 2 or 3*/ | |
| 1204 | − switch (msg->pld_lenfmt & 0x03) { | |
| 1205 | − case 1: | |
| 1206 | − lfmt = 2; | |
| 1207 | − break; | |
| 1208 | − case 2: | |
| 1209 | − lfmt = 3; | |
| 1210 | − break; | |
| 1211 | − default: | |
| 1283 | + if (!(msg->cprbx.flags & 0x80)) { | |
| 1284 | + int i, offs = 0; | |
| 1285 | + size_t pld_len; | |
| 1286 | + u8 *pld; | |
| 1287 | + u32 u; | |
| 1288 | + | |
| 1289 | + /* update target field in ep11_cprb */ | |
| 1290 | + msg->cprbx.target_id = (u32)AP_QID_QUEUE(zq->queue->qid); | |
| 1291 | + | |
| 1292 | + /* ptr and length to payload */ | |
| 1293 | + pld = ap_msg->msg + | |
| 1294 | + sizeof(struct type6_hdr) + sizeof(struct ep11_cprb); | |
| 1295 | + pld_len = msg->cprbx.payload_len; | |
| 1296 | + if (pld_len < MIN_EP11_PAYLOAD_SIZE) | |
| 1297 | + return -EINVAL; | |
| 1298 | + | |
| 1299 | + /* | |
| 1300 | + * Parse the asn1 payload, at least we have | |
| 1301 | + * pld tag (1 octet) | |
| 1302 | + * payload length (1-5 octets) | |
| 1303 | + * function tag (1 octet) | |
| 1304 | + * function length (1-5 octets) | |
| 1305 | + * function value (1-4 octets) | |
| 1306 | + * ----- optional fields ----- | |
| 1307 | + * domain tag (1 octet) | |
| 1308 | + * domain length (1-5 octets) | |
| 1309 | + * domain value (1-4 octets) | |
| 1310 | + * ... maybe much more data ... | |
| 1311 | + */ | |
| 1312 | + | |
| 1313 | + /* payload tag */ | |
| 1314 | + if (pld[offs++] != 0x30) | |
| 1315 | + return -EINVAL; | |
| 1316 | + /* payload length field */ | |
| 1317 | + i = asn1_length_decode(pld + offs, pld_len - offs, &u); | |
| 1318 | + if (i < 0) | |
| 1319 | + return -EINVAL; | |
| 1320 | + offs += i; | |
| 1321 | + if (offs >= pld_len || u > pld_len - offs) | |
| 1322 | + return -EINVAL; | |
| 1323 | + /* function tag */ | |
| 1324 | + if (pld[offs++] != 0x04) | |
| 1325 | + return -EINVAL; | |
| 1326 | + /* function length */ | |
| 1327 | + if (offs >= pld_len) | |
| 1328 | + return -EINVAL; | |
| 1329 | + i = asn1_length_decode(pld + offs, pld_len - offs, &u); | |
| 1330 | + if (i < 0) | |
| 1331 | + return -EINVAL; | |
| 1332 | + offs += i; | |
| 1333 | + if (u > pld_len - offs) | |
| 1334 | + return -EINVAL; | |
| 1335 | + /* skip over the function value */ | |
| 1336 | + offs += u; | |
| 1337 | + /* is there some payload left which could hold a domain value ? */ | |
| 1338 | + if (offs < pld_len && pld_len - offs >= 3) { | |
| 1339 | + /* domain tag */ | |
| 1340 | + if (pld[offs++] != 0x04) | |
| 1212 | 1341 | return -EINVAL; |
| 1213 | − } | |
| 1214 | − } else { | |
| 1215 | − lfmt = 1; /* length format #1 */ | |
| 1342 | + /* domain length */ | |
| 1343 | + i = asn1_length_decode(pld + offs, pld_len - offs, &u); | |
| 1344 | + if (i < 0) | |
| 1345 | + return -EINVAL; | |
| 1346 | + offs += i; | |
| 1347 | + if (offs >= pld_len || u > pld_len - offs) | |
| 1348 | + return -EINVAL; | |
| 1349 | + /* | |
| 1350 | + * pld[offs] is now at the start of the domain value | |
| 1351 | + * with the value sprawled in u octets. | |
| 1352 | + */ | |
| 1353 | + i = asn1_int_encode(pld + offs, u, | |
| 1354 | + AP_QID_QUEUE(zq->queue->qid)); | |
| 1355 | + if (i < 0) | |
| 1356 | + return -EINVAL; | |
| 1357 | + offs += i; | |
| 1216 | 1358 | } |
| 1217 | − payload_hdr = (struct pld_hdr *)((&msg->pld_lenfmt) + lfmt); | |
| 1218 | − payload_hdr->dom_val = AP_QID_QUEUE(zq->queue->qid); | |
| 1219 | 1359 | } |
| 1220 | 1360 | |
| 1221 | 1361 | /* |
| @@ -148,13 +148,16 @@ static int unregister_sba(struct ism_dev *ism) · 147 unmodified lines | ||
| 148 | 148 | if (ret && ret != ISM_ERROR) |
| 149 | 149 | return -EIO; |
| 150 | 150 | |
| 151 | + return 0; | |
| 152 | +} | |
| 153 | + | |
| 154 | +static void ism_free_sba(struct ism_dev *ism) | |
| 155 | +{ | |
| 151 | 156 | dma_free_coherent(&ism->pdev->dev, PAGE_SIZE, |
| 152 | 157 | ism->sba, ism->sba_dma_addr); |
| 153 | 158 | |
| 154 | 159 | ism->sba = NULL; |
| 155 | 160 | ism->sba_dma_addr = 0; |
| 156 | − | |
| 157 | − return 0; | |
| 158 | 161 | } |
| 159 | 162 | |
| 160 | 163 | static int unregister_ieq(struct ism_dev *ism) |
| @@ -168,13 +171,16 @@ static int unregister_ieq(struct ism_dev *ism) · 7 unmodified lines | ||
| 168 | 171 | if (ret && ret != ISM_ERROR) |
| 169 | 172 | return -EIO; |
| 170 | 173 | |
| 174 | + return 0; | |
| 175 | +} | |
| 176 | + | |
| 177 | +static void ism_free_ieq(struct ism_dev *ism) | |
| 178 | +{ | |
| 171 | 179 | dma_free_coherent(&ism->pdev->dev, PAGE_SIZE, |
| 172 | 180 | ism->ieq, ism->ieq_dma_addr); |
| 173 | 181 | |
| 174 | 182 | ism->ieq = NULL; |
| 175 | 183 | ism->ieq_dma_addr = 0; |
| 176 | − | |
| 177 | − return 0; | |
| 178 | 184 | } |
| 179 | 185 | |
| 180 | 186 | static int ism_read_local_gid(struct dibs_dev *dibs) |
| @@ -573,6 +579,7 @@ static int ism_dev_init(struct ism_dev *ism) · 392 unmodified lines | ||
| 573 | 579 | |
| 574 | 580 | unreg_sba: |
| 575 | 581 | unregister_sba(ism); |
| 582 | + ism_free_sba(ism); | |
| 576 | 583 | free_irq: |
| 577 | 584 | free_irq(pci_irq_vector(pdev, 0), ism); |
| 578 | 585 | free_vectors: |
| @@ -585,9 +592,13 @@ static void ism_dev_exit(struct ism_dev *ism) · 6 unmodified lines | ||
| 585 | 592 | { |
| 586 | 593 | struct pci_dev *pdev = ism->pdev; |
| 587 | 594 | |
| 595 | + /* ism will only generate new IRQs while ieq & sba are registered */ | |
| 588 | 596 | unregister_ieq(ism); |
| 589 | 597 | unregister_sba(ism); |
| 598 | + /* drain ongoing irpt handlers */ | |
| 590 | 599 | free_irq(pci_irq_vector(pdev, 0), ism); |
| 600 | + ism_free_ieq(ism); | |
| 601 | + ism_free_sba(ism); | |
| 591 | 602 | pci_free_irq_vectors(pdev); |
| 592 | 603 | } |
| 593 | 604 | |
| @@ -4710,6 +4710,9 @@ static int qeth_snmp_command(struct qeth_card *card, char __user *udata) · 4709 unmodified lines | ||
| 4710 | 4710 | if (req_len > QETH_BUFSIZE) |
| 4711 | 4711 | return -EINVAL; |
| 4712 | 4712 | |
| 4713 | + if (qinfo.udata_len < sizeof(struct qeth_snmp_ureq_hdr)) | |
| 4714 | + return -EINVAL; | |
| 4715 | + | |
| 4713 | 4716 | iob = qeth_get_adapter_cmd(card, IPA_SETADP_SET_SNMP_CONTROL, req_len); |
| 4714 | 4717 | if (!iob) |
| 4715 | 4718 | return -ENOMEM; |
| @@ -6525,6 +6528,9 @@ int qeth_siocdevprivate(struct net_device *dev, struct ifreq *rq, void __user *d · 1809 unmodified lines | ||
| 6525 | 6528 | struct qeth_card *card = dev->ml_priv; |
| 6526 | 6529 | int rc = 0; |
| 6527 | 6530 | |
| 6531 | + if (!capable(CAP_NET_ADMIN)) | |
| 6532 | + return -EPERM; | |
| 6533 | + | |
| 6528 | 6534 | switch (cmd) { |
| 6529 | 6535 | case SIOC_QETH_ADP_SET_SNMP_CONTROL: |
| 6530 | 6536 | rc = qeth_snmp_command(card, data); |
| @@ -1415,6 +1415,11 @@ static int qeth_l3_arp_query(struct qeth_card *card, char __user *udata) · 1414 unmodified lines | ||
| 1415 | 1415 | rc = -EFAULT; |
| 1416 | 1416 | goto out; |
| 1417 | 1417 | } |
| 1418 | + | |
| 1419 | + if (qinfo.udata_len < QETH_QARP_ENTRIES_OFFSET) { | |
| 1420 | + rc = -EINVAL; | |
| 1421 | + goto out; | |
| 1422 | + } | |
| 1418 | 1423 | qinfo.udata = kzalloc(qinfo.udata_len, GFP_KERNEL); |
| 1419 | 1424 | if (!qinfo.udata) { |
| 1420 | 1425 | rc = -ENOMEM; |
| @@ -253,6 +253,7 @@ static int zfcp_allocate_low_mem_buffers(struct zfcp_adapter *adapter) · 252 unmodified lines | ||
| 253 | 253 | static void zfcp_free_low_mem_buffers(struct zfcp_adapter *adapter) |
| 254 | 254 | { |
| 255 | 255 | mempool_destroy(adapter->pool.erp_req); |
| 256 | + mempool_destroy(adapter->pool.gid_pn_req); | |
| 256 | 257 | mempool_destroy(adapter->pool.scsi_req); |
| 257 | 258 | mempool_destroy(adapter->pool.scsi_abort); |
| 258 | 259 | mempool_destroy(adapter->pool.qtcb_pool); |
| @@ -5261,15 +5261,7 @@ static int _resume_v3_hw(struct device *device) · 5260 unmodified lines | ||
| 5261 | 5261 | return rc; |
| 5262 | 5262 | } |
| 5263 | 5263 | phys_init_v3_hw(hisi_hba); |
| 5264 | − | |
| 5265 | − /* | |
| 5266 | − * If a directly-attached disk is removed during suspend, a deadlock | |
| 5267 | − * may occur, as the PHYE_RESUME_TIMEOUT processing will require the | |
| 5268 | − * hisi_hba->device to be active, which can only happen when resume | |
| 5269 | − * completes. So don't wait for the HA event workqueue to drain upon | |
| 5270 | − * resume. | |
| 5271 | − */ | |
| 5272 | − sas_resume_ha_no_sync(sha); | |
| 5264 | + sas_resume_ha(sha); | |
| 5273 | 5265 | clear_bit(HISI_SAS_RESETTING_BIT, &hisi_hba->flags); |
| 5274 | 5266 | |
| 5275 | 5267 | dev_warn(dev, "end of resuming controller\n"); |
| @@ -918,7 +918,7 @@ invalid_datalen: · 917 unmodified lines | ||
| 918 | 918 | } |
| 919 | 919 | |
| 920 | 920 | senselen = get_unaligned_be16(data); |
| 921 | − if (datalen < senselen) | |
| 921 | + if (datalen < senselen + 2) | |
| 922 | 922 | goto invalid_datalen; |
| 923 | 923 | |
| 924 | 924 | memcpy(sc->sense_buffer, data + 2, |
| @@ -752,13 +752,6 @@ iscsi_tcp_hdr_dissect(struct iscsi_conn *conn, struct iscsi_hdr *hdr) · 751 unmodified lines | ||
| 752 | 752 | rc = __iscsi_complete_pdu(conn, hdr, NULL, 0); |
| 753 | 753 | spin_unlock(&conn->session->back_lock); |
| 754 | 754 | break; |
| 755 | − case ISCSI_OP_SCSI_CMD_RSP: | |
| 756 | − if (tcp_conn->in.datalen) { | |
| 757 | − iscsi_tcp_data_recv_prep(tcp_conn); | |
| 758 | − return 0; | |
| 759 | − } | |
| 760 | − rc = iscsi_complete_pdu(conn, hdr, NULL, 0); | |
| 761 | − break; | |
| 762 | 755 | case ISCSI_OP_R2T: |
| 763 | 756 | if (ahslen) { |
| 764 | 757 | rc = ISCSI_ERR_AHSLEN; |
| @@ -766,6 +759,7 @@ iscsi_tcp_hdr_dissect(struct iscsi_conn *conn, struct iscsi_hdr *hdr) · 1 unmodified line | ||
| 766 | 759 | } |
| 767 | 760 | rc = iscsi_tcp_r2t_rsp(conn, hdr); |
| 768 | 761 | break; |
| 762 | + case ISCSI_OP_SCSI_CMD_RSP: | |
| 769 | 763 | case ISCSI_OP_LOGIN_RSP: |
| 770 | 764 | case ISCSI_OP_TEXT_RSP: |
| 771 | 765 | case ISCSI_OP_REJECT: |
| @@ -410,7 +410,7 @@ static void sas_resume_insert_broadcast_ha(struct sas_ha_struct *ha) · 409 unmodified lines | ||
| 410 | 410 | } |
| 411 | 411 | } |
| 412 | 412 | |
| 413 | −static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain) | |
| 413 | +void sas_resume_ha(struct sas_ha_struct *ha) | |
| 414 | 414 | { |
| 415 | 415 | const unsigned long tmo = msecs_to_jiffies(25000); |
| 416 | 416 | int i; |
| @@ -426,6 +426,23 @@ static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain) · 9 unmodified lines | ||
| 426 | 426 | dev_info(ha->dev, "waiting up to 25 seconds for %d phy%s to resume\n", |
| 427 | 427 | i, i > 1 ? "s" : ""); |
| 428 | 428 | wait_event_timeout(ha->eh_wait_q, phys_suspended(ha) == 0, tmo); |
| 429 | + | |
| 430 | + /* | |
| 431 | + * All phys are back up or timed out. Turn on I/O and drain | |
| 432 | + * pending work. | |
| 433 | + */ | |
| 434 | + scsi_unblock_requests(ha->shost); | |
| 435 | + sas_drain_work(ha); | |
| 436 | + | |
| 437 | + /* | |
| 438 | + * Send PHYE_RESUME_TIMEOUT after sas_drain_work(). The handler | |
| 439 | + * calls sas_deform_port() -> sas_destruct_devices(), which removes | |
| 440 | + * SCSI devices and, for LLDDs using device_link() PM sync, waits | |
| 441 | + * for the host to be runtime-active. Sending it before the drain | |
| 442 | + * would deadlock: the drain waits for the handler, the handler | |
| 443 | + * waits for host resume, and host resume waits for the drain to | |
| 444 | + * finish. | |
| 445 | + */ | |
| 429 | 446 | for (i = 0; i < ha->num_phys; i++) { |
| 430 | 447 | struct asd_sas_phy *phy = ha->sas_phy[i]; |
| 431 | 448 | |
| @@ -436,12 +453,6 @@ static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain) · 4 unmodified lines | ||
| 436 | 453 | } |
| 437 | 454 | } |
| 438 | 455 | |
| 439 | − /* all phys are back up or timed out, turn on i/o so we can | |
| 440 | − * flush out disks that did not return | |
| 441 | − */ | |
| 442 | − scsi_unblock_requests(ha->shost); | |
| 443 | − if (drain) | |
| 444 | − sas_drain_work(ha); | |
| 445 | 456 | clear_bit(SAS_HA_RESUMING, &ha->state); |
| 446 | 457 | |
| 447 | 458 | sas_queue_deferred_work(ha); |
| @@ -450,20 +461,8 @@ static void _sas_resume_ha(struct sas_ha_struct *ha, bool drain) · 2 unmodified lines | ||
| 450 | 461 | */ |
| 451 | 462 | sas_resume_insert_broadcast_ha(ha); |
| 452 | 463 | } |
| 453 | − | |
| 454 | −void sas_resume_ha(struct sas_ha_struct *ha) | |
| 455 | −{ | |
| 456 | − _sas_resume_ha(ha, true); | |
| 457 | −} | |
| 458 | 464 | EXPORT_SYMBOL(sas_resume_ha); |
| 459 | 465 | |
| 460 | −/* A no-sync variant, which does not call sas_drain_ha(). */ | |
| 461 | −void sas_resume_ha_no_sync(struct sas_ha_struct *ha) | |
| 462 | −{ | |
| 463 | − _sas_resume_ha(ha, false); | |
| 464 | −} | |
| 465 | −EXPORT_SYMBOL(sas_resume_ha_no_sync); | |
| 466 | − | |
| 467 | 466 | void sas_suspend_ha(struct sas_ha_struct *ha) |
| 468 | 467 | { |
| 469 | 468 | int i; |
| @@ -502,6 +502,21 @@ int sas_eh_target_reset_handler(struct scsi_cmnd *cmd) · 501 unmodified lines | ||
| 502 | 502 | } |
| 503 | 503 | EXPORT_SYMBOL_GPL(sas_eh_target_reset_handler); |
| 504 | 504 | |
| 505 | +/* | |
| 506 | + * Handle deferred QCs in case of a command timeout. | |
| 507 | + * See ata_scsi_eh_timed_out() for details. | |
| 508 | + */ | |
| 509 | +enum scsi_timeout_action sas_eh_timed_out(struct scsi_cmnd *cmd) | |
| 510 | +{ | |
| 511 | + struct domain_device *dev = cmd_to_domain_dev(cmd); | |
| 512 | + | |
| 513 | + if (dev_is_sata(dev)) | |
| 514 | + return ata_scsi_retry_deferred_qc(dev->sata_dev.ap, cmd); | |
| 515 | + | |
| 516 | + return SCSI_EH_NOT_HANDLED; | |
| 517 | +} | |
| 518 | +EXPORT_SYMBOL_GPL(sas_eh_timed_out); | |
| 519 | + | |
| 505 | 520 | /* Try to reset a device */ |
| 506 | 521 | static int try_to_reset_cmd_device(struct scsi_cmnd *cmd) |
| 507 | 522 | { |
| @@ -9,6 +9,7 @@ · 8 unmodified lines | ||
| 9 | 9 | |
| 10 | 10 | #include "mpi3mr.h" |
| 11 | 11 | #include <linux/io-64-nonatomic-lo-hi.h> |
| 12 | +#include <linux/sched/mm.h> | |
| 12 | 13 | |
| 13 | 14 | static int |
| 14 | 15 | mpi3mr_issue_reset(struct mpi3mr_ioc *mrioc, u16 reset_type, u16 reset_reason); |
| @@ -1287,11 +1288,14 @@ out_failed: · 1272 unmodified lines | ||
| 1287 | 1288 | static void mpi3mr_fault_uevent_emit(struct mpi3mr_ioc *mrioc) |
| 1288 | 1289 | { |
| 1289 | 1290 | struct kobj_uevent_env *env; |
| 1291 | + unsigned int noio_flag; | |
| 1290 | 1292 | int ret; |
| 1291 | 1293 | |
| 1294 | + noio_flag = memalloc_noio_save(); | |
| 1295 | + | |
| 1292 | 1296 | env = kzalloc_obj(*env); |
| 1293 | 1297 | if (!env) |
| 1294 | − return; | |
| 1298 | + goto out_restore; | |
| 1295 | 1299 | |
| 1296 | 1300 | ret = add_uevent_var(env, "DRIVER=%s", mrioc->driver_name); |
| 1297 | 1301 | if (ret) |
| @@ -1326,7 +1330,8 @@ static void mpi3mr_fault_uevent_emit(struct mpi3mr_ioc *mrioc) · 28 unmodified lines | ||
| 1326 | 1330 | |
| 1327 | 1331 | out_free: |
| 1328 | 1332 | kfree(env); |
| 1329 | − | |
| 1333 | +out_restore: | |
| 1334 | + memalloc_noio_restore(noio_flag); | |
| 1330 | 1335 | } |
| 1331 | 1336 | |
| 1332 | 1337 | /** |
| @@ -5890,6 +5890,7 @@ static int resp_report_zones(struct scsi_cmnd *scp, · 5889 unmodified lines | ||
| 5890 | 5890 | u32 alloc_len, rep_opts, rep_len; |
| 5891 | 5891 | bool partial; |
| 5892 | 5892 | u64 lba, zs_lba; |
| 5893 | + u64 arr_len; | |
| 5893 | 5894 | u8 *arr = NULL, *desc; |
| 5894 | 5895 | u8 *cmd = scp->cmnd; |
| 5895 | 5896 | struct sdeb_zone_state *zsp = NULL; |
| @@ -5911,9 +5912,12 @@ static int resp_report_zones(struct scsi_cmnd *scp, · 15 unmodified lines | ||
| 5911 | 5912 | return check_condition_result; |
| 5912 | 5913 | } |
| 5913 | 5914 | |
| 5914 | − rep_max_zones = (alloc_len - 64) >> ilog2(RZONES_DESC_HD); | |
| 5915 | + rep_max_zones = (ALIGN((u64)alloc_len, RZONES_DESC_HD) - RZONES_DESC_HD) >> | |
| 5916 | + ilog2(RZONES_DESC_HD); | |
| 5917 | + rep_max_zones = min_t(unsigned int, rep_max_zones, devip->nr_zones); | |
| 5918 | + arr_len = (u64)RZONES_DESC_HD * (rep_max_zones + 1); | |
| 5915 | 5919 | |
| 5916 | − arr = kzalloc(alloc_len, GFP_ATOMIC | __GFP_NOWARN); | |
| 5920 | + arr = kzalloc(arr_len, GFP_ATOMIC | __GFP_NOWARN); | |
| 5917 | 5921 | if (!arr) { |
| 5918 | 5922 | mk_sense_buffer(scp, ILLEGAL_REQUEST, INSUFF_RES_ASC, |
| 5919 | 5923 | INSUFF_RES_ASCQ); |
| @@ -4,3 +4,4 @@ obj-$(CONFIG_ASPEED_LPC_SNOOP) += aspeed-lpc-snoop.o · 3 unmodified lines | ||
| 4 | 4 | obj-$(CONFIG_ASPEED_UART_ROUTING) += aspeed-uart-routing.o |
| 5 | 5 | obj-$(CONFIG_ASPEED_P2A_CTRL) += aspeed-p2a-ctrl.o |
| 6 | 6 | obj-$(CONFIG_ASPEED_SOCINFO) += aspeed-socinfo.o |
| 7 | +CONTEXT_ANALYSIS_aspeed-lpc-snoop.o := y | |
| @@ -11,6 +11,7 @@ · 10 unmodified lines | ||
| 11 | 11 | */ |
| 12 | 12 | |
| 13 | 13 | #include <linux/bitops.h> |
| 14 | +#include <linux/cleanup.h> | |
| 14 | 15 | #include <linux/clk.h> |
| 15 | 16 | #include <linux/dev_printk.h> |
| 16 | 17 | #include <linux/interrupt.h> |
| @@ -74,7 +75,8 @@ struct aspeed_lpc_snoop_channel_cfg { · 57 unmodified lines | ||
| 74 | 75 | struct aspeed_lpc_snoop_channel { |
| 75 | 76 | const struct aspeed_lpc_snoop_channel_cfg *cfg; |
| 76 | 77 | bool enabled; |
| 77 | − struct kfifo fifo; | |
| 78 | + spinlock_t lock; | |
| 79 | + struct kfifo fifo __guarded_by(&lock); | |
| 78 | 80 | wait_queue_head_t wq; |
| 79 | 81 | struct miscdevice miscdev; |
| 80 | 82 | }; |
| @@ -114,6 +116,7 @@ static ssize_t snoop_file_read(struct file *file, char __user *buffer, · 33 unmodified lines | ||
| 114 | 116 | size_t count, loff_t *ppos) |
| 115 | 117 | { |
| 116 | 118 | struct aspeed_lpc_snoop_channel *chan = snoop_file_to_chan(file); |
| 119 | + u8 *buf __free(kfree) = NULL; | |
| 117 | 120 | unsigned int copied; |
| 118 | 121 | int ret = 0; |
| 119 | 122 | |
| @@ -125,9 +128,16 @@ static ssize_t snoop_file_read(struct file *file, char __user *buffer, · 5 unmodified lines | ||
| 125 | 128 | if (ret == -ERESTARTSYS) |
| 126 | 129 | return -EINTR; |
| 127 | 130 | } |
| 128 | − ret = kfifo_to_user(&chan->fifo, buffer, count, &copied); | |
| 129 | − if (ret) | |
| 130 | − return ret; | |
| 131 | + | |
| 132 | + count = min_t(size_t, count, SNOOP_FIFO_SIZE); | |
| 133 | + | |
| 134 | + buf = kmalloc(count, GFP_KERNEL); | |
| 135 | + if (!buf) | |
| 136 | + return -ENOMEM; | |
| 137 | + | |
| 138 | + copied = kfifo_out_spinlocked(&chan->fifo, buf, count, &chan->lock); | |
| 139 | + if (copied && copy_to_user(buffer, buf, copied)) | |
| 140 | + return -EFAULT; | |
| 131 | 141 | |
| 132 | 142 | return copied; |
| 133 | 143 | } |
| @@ -151,11 +161,13 @@ static const struct file_operations snoop_fops = { · 17 unmodified lines | ||
| 151 | 161 | /* Save a byte to a FIFO and discard the oldest byte if FIFO is full */ |
| 152 | 162 | static void put_fifo_with_discard(struct aspeed_lpc_snoop_channel *chan, u8 val) |
| 153 | 163 | { |
| 154 | − if (!kfifo_initialized(&chan->fifo)) | |
| 155 | − return; | |
| 156 | − if (kfifo_is_full(&chan->fifo)) | |
| 157 | − kfifo_skip(&chan->fifo); | |
| 158 | − kfifo_put(&chan->fifo, val); | |
| 164 | + scoped_guard(spinlock, &chan->lock) { | |
| 165 | + if (!kfifo_initialized(&chan->fifo)) | |
| 166 | + return; | |
| 167 | + if (kfifo_is_full(&chan->fifo)) | |
| 168 | + kfifo_skip(&chan->fifo); | |
| 169 | + kfifo_put(&chan->fifo, val); | |
| 170 | + } | |
| 159 | 171 | wake_up_interruptible(&chan->wq); |
| 160 | 172 | } |
| 161 | 173 | |
| @@ -239,9 +251,11 @@ static int aspeed_lpc_enable_snoop(struct device *dev, · 77 unmodified lines | ||
| 239 | 251 | if (!channel->miscdev.name) |
| 240 | 252 | return -ENOMEM; |
| 241 | 253 | |
| 242 | − rc = kfifo_alloc(&channel->fifo, SNOOP_FIFO_SIZE, GFP_KERNEL); | |
| 243 | − if (rc) | |
| 244 | − return rc; | |
| 254 | + scoped_guard(spinlock_init, &channel->lock) { | |
| 255 | + rc = kfifo_alloc(&channel->fifo, SNOOP_FIFO_SIZE, GFP_KERNEL); | |
| 256 | + if (rc) | |
| 257 | + return rc; | |
| 258 | + } | |
| 245 | 259 | |
| 246 | 260 | rc = misc_register(&channel->miscdev); |
| 247 | 261 | if (rc) |
| @@ -261,6 +261,7 @@ static int intel_ace2x_bpt_open_stream(struct sdw_intel *sdw, struct sdw_slave * · 260 unmodified lines | ||
| 261 | 261 | __func__, str_read_write(command), ret); |
| 262 | 262 | |
| 263 | 263 | ret1 = hda_sdw_bpt_close(cdns->dev->parent, /* PCI device */ |
| 264 | + sdw->instance, | |
| 264 | 265 | sdw->bpt_ctx.bpt_tx_stream, &sdw->bpt_ctx.dmab_tx_bdl, |
| 265 | 266 | sdw->bpt_ctx.bpt_rx_stream, &sdw->bpt_ctx.dmab_rx_bdl); |
| 266 | 267 | if (ret1 < 0) |
| @@ -295,7 +296,8 @@ static void intel_ace2x_bpt_close_stream(struct sdw_intel *sdw, struct sdw_slave · 28 unmodified lines | ||
| 295 | 296 | struct sdw_cdns *cdns = &sdw->cdns; |
| 296 | 297 | int ret; |
| 297 | 298 | |
| 298 | − ret = hda_sdw_bpt_close(cdns->dev->parent /* PCI device */, sdw->bpt_ctx.bpt_tx_stream, | |
| 299 | + ret = hda_sdw_bpt_close(cdns->dev->parent /* PCI device */, sdw->instance, | |
| 300 | + sdw->bpt_ctx.bpt_tx_stream, | |
| 299 | 301 | &sdw->bpt_ctx.dmab_tx_bdl, sdw->bpt_ctx.bpt_rx_stream, |
| 300 | 302 | &sdw->bpt_ctx.dmab_rx_bdl); |
| 301 | 303 | if (ret < 0) |
| @@ -388,11 +388,13 @@ static inline void cdns_spi_writer(struct cdns_spi *xspi) · 387 unmodified lines | ||
| 388 | 388 | |
| 389 | 389 | /** |
| 390 | 390 | * cdns_spi_process_fifo - Fills the TX FIFO, and drain the RX FIFO |
| 391 | + * @ctlr: Pointer to the spi_controller structure | |
| 391 | 392 | * @xspi: Pointer to the cdns_spi structure |
| 392 | 393 | * @ntx: Number of bytes to pack into the TX FIFO |
| 393 | 394 | * @nrx: Number of bytes to drain from the RX FIFO |
| 394 | 395 | */ |
| 395 | −static void cdns_spi_process_fifo(struct cdns_spi *xspi, int ntx, int nrx) | |
| 396 | +static void cdns_spi_process_fifo(struct spi_controller *ctlr, | |
| 397 | + struct cdns_spi *xspi, int ntx, int nrx) | |
| 396 | 398 | { |
| 397 | 399 | ntx = clamp(ntx, 0, xspi->tx_bytes); |
| 398 | 400 | nrx = clamp(nrx, 0, xspi->rx_bytes); |
| @@ -407,6 +409,16 @@ static void cdns_spi_process_fifo(struct cdns_spi *xspi, int ntx, int nrx) · 8 unmodified lines | ||
| 407 | 409 | } |
| 408 | 410 | |
| 409 | 411 | if (ntx) { |
| 412 | + /* When xspi in busy condition, bytes may send failed, | |
| 413 | + * then spi control didn't work thoroughly, add one byte | |
| 414 | + * delay. Only in host mode; in target mode this delay | |
| 415 | + * causes data corruption as the target fails to prepare | |
| 416 | + * data in time. | |
| 417 | + */ | |
| 418 | + if (!spi_controller_is_target(ctlr) && | |
| 419 | + (cdns_spi_read(xspi, CDNS_SPI_ISR) & CDNS_SPI_IXR_TXFULL)) | |
| 420 | + udelay(10); | |
| 421 | + | |
| 410 | 422 | cdns_spi_writer(xspi); |
| 411 | 423 | ntx--; |
| 412 | 424 | } |
| @@ -460,14 +472,14 @@ static irqreturn_t cdns_spi_irq(int irq, void *dev_id) · 47 unmodified lines | ||
| 460 | 472 | cdns_spi_write(xspi, CDNS_SPI_THLD, 1); |
| 461 | 473 | |
| 462 | 474 | if (xspi->tx_bytes) { |
| 463 | − cdns_spi_process_fifo(xspi, trans_cnt, trans_cnt); | |
| 475 | + cdns_spi_process_fifo(ctlr, xspi, trans_cnt, trans_cnt); | |
| 464 | 476 | } else { |
| 465 | 477 | /* Fixed delay due to controller limitation with |
| 466 | 478 | * RX_NEMPTY incorrect status |
| 467 | 479 | * Xilinx AR:65885 contains more details |
| 468 | 480 | */ |
| 469 | 481 | udelay(10); |
| 470 | − cdns_spi_process_fifo(xspi, 0, trans_cnt); | |
| 482 | + cdns_spi_process_fifo(ctlr, xspi, 0, trans_cnt); | |
| 471 | 483 | cdns_spi_write(xspi, CDNS_SPI_IDR, |
| 472 | 484 | CDNS_SPI_IXR_DEFAULT); |
| 473 | 485 | spi_finalize_current_transfer(ctlr); |
| @@ -520,17 +532,11 @@ static int cdns_transfer_one(struct spi_controller *ctlr, · 46 unmodified lines | ||
| 520 | 532 | cdns_spi_write(xspi, CDNS_SPI_THLD, xspi->tx_fifo_depth >> 1); |
| 521 | 533 | } |
| 522 | 534 | |
| 523 | − /* When xspi in busy condition, bytes may send failed, | |
| 524 | − * then spi control didn't work thoroughly, add one byte delay | |
| 525 | − */ | |
| 526 | − if (cdns_spi_read(xspi, CDNS_SPI_ISR) & CDNS_SPI_IXR_TXFULL) | |
| 527 | − udelay(10); | |
| 528 | − | |
| 529 | 535 | xspi->n_bytes = cdns_spi_n_bytes(transfer); |
| 530 | 536 | xspi->tx_bytes = DIV_ROUND_UP(xspi->tx_bytes, xspi->n_bytes); |
| 531 | 537 | xspi->rx_bytes = DIV_ROUND_UP(xspi->rx_bytes, xspi->n_bytes); |
| 532 | 538 | |
| 533 | − cdns_spi_process_fifo(xspi, xspi->tx_fifo_depth, 0); | |
| 539 | + cdns_spi_process_fifo(ctlr, xspi, xspi->tx_fifo_depth, 0); | |
| 534 | 540 | |
| 535 | 541 | cdns_spi_write(xspi, CDNS_SPI_IER, CDNS_SPI_IXR_DEFAULT); |
| 536 | 542 | return transfer->len; |
| @@ -340,6 +340,18 @@ struct nxp_fspi_devtype_data { · 339 unmodified lines | ||
| 340 | 340 | unsigned int quirks; |
| 341 | 341 | unsigned int lut_num; |
| 342 | 342 | bool little_endian; |
| 343 | + /* | |
| 344 | + * The max clock rate (Hz) that FlexSPI can output to the device | |
| 345 | + * in SDR mode (RXCLKSRC=0). Defaults to 66MHz if zero. | |
| 346 | + * Some SoCs (e.g. LX2160A) support up to 100MHz in SDR mode. | |
| 347 | + */ | |
| 348 | + unsigned long max_sdr_rate; | |
| 349 | + /* | |
| 350 | + * The max clock rate (Hz) that FlexSPI can output to the device | |
| 351 | + * in DTR mode (RXCLKSRC=3). Defaults to 166MHz if zero. | |
| 352 | + * Some SoCs (e.g. i.MX95, i.MX8QM, i.MX8DXL) support up to 200MHz. | |
| 353 | + */ | |
| 354 | + unsigned long max_dtr_rate; | |
| 343 | 355 | }; |
| 344 | 356 | |
| 345 | 357 | static struct nxp_fspi_devtype_data lx2160a_data = { |
| @@ -349,6 +361,10 @@ static struct nxp_fspi_devtype_data lx2160a_data = { · 3 unmodified lines | ||
| 349 | 361 | .quirks = FSPI_QUIRK_DISABLE_DTR, |
| 350 | 362 | .lut_num = 32, |
| 351 | 363 | .little_endian = true, /* little-endian */ |
| 364 | + /* | |
| 365 | + * LX2160ACEC: SDR RXCLKSRC=0 max 100MHz, DTR disabled via quirk. | |
| 366 | + */ | |
| 367 | + .max_sdr_rate = 100000000, | |
| 352 | 368 | }; |
| 353 | 369 | |
| 354 | 370 | static struct nxp_fspi_devtype_data imx8mm_data = { |
| @@ -358,6 +374,21 @@ static struct nxp_fspi_devtype_data imx8mm_data = { · 3 unmodified lines | ||
| 358 | 374 | .quirks = 0, |
| 359 | 375 | .lut_num = 32, |
| 360 | 376 | .little_endian = true, /* little-endian */ |
| 377 | + /* IMX8MMCEC §3.9.10: SDR RXCLKSRC=0 max 66MHz, DDR RXCLKSRC=3 max 166MHz */ | |
| 378 | + .max_sdr_rate = 66000000, | |
| 379 | + .max_dtr_rate = 166000000, | |
| 380 | +}; | |
| 381 | + | |
| 382 | +static struct nxp_fspi_devtype_data imx8mp_data = { | |
| 383 | + .rxfifo = SZ_512, /* (64 * 64 bits) */ | |
| 384 | + .txfifo = SZ_1K, /* (128 * 64 bits) */ | |
| 385 | + .ahb_buf_size = SZ_2K, /* (256 * 64 bits) */ | |
| 386 | + .quirks = 0, | |
| 387 | + .lut_num = 32, | |
| 388 | + .little_endian = true, /* little-endian */ | |
| 389 | + /* IMX8MPCEC: SDR RXCLKSRC=0 max 66MHz, DDR RXCLKSRC=3 max 166MHz */ | |
| 390 | + .max_sdr_rate = 66000000, | |
| 391 | + .max_dtr_rate = 166000000, | |
| 361 | 392 | }; |
| 362 | 393 | |
| 363 | 394 | static struct nxp_fspi_devtype_data imx8qxp_data = { |
| @@ -367,6 +398,12 @@ static struct nxp_fspi_devtype_data imx8qxp_data = { · 3 unmodified lines | ||
| 367 | 398 | .quirks = 0, |
| 368 | 399 | .lut_num = 32, |
| 369 | 400 | .little_endian = true, /* little-endian */ |
| 401 | + /* | |
| 402 | + * IMX8QXPCEC: SDR RXCLKSRC=0 max 60MHz, DDR RXCLKSRC=3 max 200MHz. | |
| 403 | + * i.MX8QM and i.MX8DXL share the same FlexSPI IP and limits. | |
| 404 | + */ | |
| 405 | + .max_sdr_rate = 60000000, | |
| 406 | + .max_dtr_rate = 200000000, | |
| 370 | 407 | }; |
| 371 | 408 | |
| 372 | 409 | static struct nxp_fspi_devtype_data imx8dxl_data = { |
| @@ -376,6 +413,12 @@ static struct nxp_fspi_devtype_data imx8dxl_data = { · 3 unmodified lines | ||
| 376 | 413 | .quirks = FSPI_QUIRK_USE_IP_ONLY, |
| 377 | 414 | .lut_num = 32, |
| 378 | 415 | .little_endian = true, /* little-endian */ |
| 416 | + /* | |
| 417 | + * IMX8DXLCEC (i.MX 8XLite): SDR RXCLKSRC=0 max 60MHz, | |
| 418 | + * DDR RXCLKSRC=3 max 200MHz. | |
| 419 | + */ | |
| 420 | + .max_sdr_rate = 60000000, | |
| 421 | + .max_dtr_rate = 200000000, | |
| 379 | 422 | }; |
| 380 | 423 | |
| 381 | 424 | static struct nxp_fspi_devtype_data imx8ulp_data = { |
| @@ -385,6 +428,29 @@ static struct nxp_fspi_devtype_data imx8ulp_data = { · 3 unmodified lines | ||
| 385 | 428 | .quirks = 0, |
| 386 | 429 | .lut_num = 16, |
| 387 | 430 | .little_endian = true, /* little-endian */ |
| 431 | + /* | |
| 432 | + * IMX8ULPCEC §7.3.1, Normal Drive (ND, 1.0V) mode: | |
| 433 | + * SDR RXCLKSRC=0 max 60MHz, DDR RXCLKSRC=3 max 166MHz. | |
| 434 | + * Note: Overdrive (OD, 1.05V) allows up to 180MHz DTR | |
| 435 | + * but is not the default use case. | |
| 436 | + */ | |
| 437 | + .max_sdr_rate = 60000000, | |
| 438 | + .max_dtr_rate = 166000000, | |
| 439 | +}; | |
| 440 | + | |
| 441 | +static struct nxp_fspi_devtype_data imx95_data = { | |
| 442 | + .rxfifo = SZ_512, /* (64 * 64 bits) */ | |
| 443 | + .txfifo = SZ_1K, /* (128 * 64 bits) */ | |
| 444 | + .ahb_buf_size = SZ_2K, /* (256 * 64 bits) */ | |
| 445 | + .quirks = 0, | |
| 446 | + .lut_num = 32, | |
| 447 | + .little_endian = true, /* little-endian */ | |
| 448 | + /* | |
| 449 | + * IMX95CEC Rev.8 §4.11.7: SDR RXCLKSRC=0 max 66MHz, | |
| 450 | + * DDR RXCLKSRC=3 max 200MHz (Nominal/Overdrive mode). | |
| 451 | + */ | |
| 452 | + .max_sdr_rate = 66000000, | |
| 453 | + .max_dtr_rate = 200000000, | |
| 388 | 454 | }; |
| 389 | 455 | |
| 390 | 456 | struct nxp_fspi { |
| @@ -691,10 +757,20 @@ static void nxp_fspi_select_rx_sample_clk_source(struct nxp_fspi *f, · 300 unmodified lines | ||
| 691 | 757 | reg = fspi_readl(f, f->iobase + FSPI_MCR0); |
| 692 | 758 | if (op_is_dtr) { |
| 693 | 759 | reg |= FSPI_MCR0_RXCLKSRC(3); |
| 694 | − f->max_rate = 166000000; | |
| 760 | + /* | |
| 761 | + * Use the SoC-specific DTR max rate if provided, otherwise | |
| 762 | + * fall back to 166MHz (limit from IMX8MN datasheet §3.9.9). | |
| 763 | + */ | |
| 764 | + f->max_rate = f->devtype_data->max_dtr_rate ? | |
| 765 | + f->devtype_data->max_dtr_rate : 166000000; | |
| 695 | 766 | } else { /*select mode 0 */ |
| 696 | 767 | reg &= ~FSPI_MCR0_RXCLKSRC(3); |
| 697 | − f->max_rate = 66000000; | |
| 768 | + /* | |
| 769 | + * Use the SoC-specific SDR max rate if provided, otherwise | |
| 770 | + * fall back to 66MHz (limit from IMX8MN datasheet §3.9.9). | |
| 771 | + */ | |
| 772 | + f->max_rate = f->devtype_data->max_sdr_rate ? | |
| 773 | + f->devtype_data->max_sdr_rate : 66000000; | |
| 698 | 774 | } |
| 699 | 775 | fspi_writel(f, reg, f->iobase + FSPI_MCR0); |
| 700 | 776 | } |
| @@ -783,14 +859,15 @@ static void nxp_fspi_dll_override(struct nxp_fspi *f) · 82 unmodified lines | ||
| 783 | 859 | * Value for rest of the CS FLSHxxCR0 register would be zero. |
| 784 | 860 | * |
| 785 | 861 | */ |
| 786 | −static void nxp_fspi_select_mem(struct nxp_fspi *f, struct spi_device *spi, | |
| 787 | − const struct spi_mem_op *op) | |
| 862 | +static int nxp_fspi_select_mem(struct nxp_fspi *f, struct spi_device *spi, | |
| 863 | + const struct spi_mem_op *op) | |
| 788 | 864 | { |
| 789 | 865 | /* flexspi only support one DTR mode: 8D-8D-8D */ |
| 790 | 866 | bool op_is_dtr = op->cmd.dtr && op->addr.dtr && op->dummy.dtr && op->data.dtr; |
| 791 | 867 | unsigned long rate = op->max_freq; |
| 792 | 868 | int ret; |
| 793 | 869 | uint64_t size_kb; |
| 870 | + u32 reg; | |
| 794 | 871 | |
| 795 | 872 | /* |
| 796 | 873 | * Return when following condition all meet, |
| @@ -804,7 +881,7 @@ static void nxp_fspi_select_mem(struct nxp_fspi *f, struct spi_device *spi, · 7 unmodified lines | ||
| 804 | 881 | if ((f->selected == spi_get_chipselect(spi, 0)) && |
| 805 | 882 | (!!(f->flags & FSPI_DTR_MODE) == op_is_dtr) && |
| 806 | 883 | (f->pre_op_rate == op->max_freq)) |
| 807 | − return; | |
| 884 | + return 0; | |
| 808 | 885 | |
| 809 | 886 | /* Reset FLSHxxCR0 registers */ |
| 810 | 887 | fspi_writel(f, 0, f->iobase + FSPI_FLSHA1CR0); |
| @@ -820,6 +897,15 @@ static void nxp_fspi_select_mem(struct nxp_fspi *f, struct spi_device *spi, · 9 unmodified lines | ||
| 820 | 897 | |
| 821 | 898 | dev_dbg(f->dev, "Target device [CS:%x] selected\n", spi_get_chipselect(spi, 0)); |
| 822 | 899 | |
| 900 | + /* | |
| 901 | + * Per the FlexSPI reference manual (initialization sequence), MCR0 and | |
| 902 | + * the DLL control registers should be configured while the module is in | |
| 903 | + * stop mode (MCR0[MDIS] = 1). Enter stop mode before reconfiguring the | |
| 904 | + * RX sample clock source and the DLL, then exit stop mode afterwards. | |
| 905 | + */ | |
| 906 | + reg = fspi_readl(f, f->iobase + FSPI_MCR0); | |
| 907 | + fspi_writel(f, reg | FSPI_MCR0_MDIS, f->iobase + FSPI_MCR0); | |
| 908 | + | |
| 823 | 909 | nxp_fspi_select_rx_sample_clk_source(f, op_is_dtr); |
| 824 | 910 | rate = min(f->max_rate, op->max_freq); |
| 825 | 911 | |
| @@ -836,12 +922,19 @@ static void nxp_fspi_select_mem(struct nxp_fspi *f, struct spi_device *spi, · 10 unmodified lines | ||
| 836 | 922 | nxp_fspi_clk_disable_unprep(f); |
| 837 | 923 | |
| 838 | 924 | ret = clk_set_rate(f->clk, rate); |
| 839 | − if (ret) | |
| 840 | − return; | |
| 925 | + if (ret) { | |
| 926 | + /* | |
| 927 | + * clk_set_rate() failed with the clocks already disabled. | |
| 928 | + * Re-enable them so the enable count matches what the caller's | |
| 929 | + * pm_runtime_put() (runtime_suspend) will drop. | |
| 930 | + */ | |
| 931 | + nxp_fspi_clk_prep_enable(f); | |
| 932 | + return ret; | |
| 933 | + } | |
| 841 | 934 | |
| 842 | 935 | ret = nxp_fspi_clk_prep_enable(f); |
| 843 | 936 | if (ret) |
| 844 | − return; | |
| 937 | + return ret; | |
| 845 | 938 | |
| 846 | 939 | /* |
| 847 | 940 | * If clock rate > 100MHz, then switch from DLL override mode to |
| @@ -852,9 +945,15 @@ static void nxp_fspi_select_mem(struct nxp_fspi *f, struct spi_device *spi, · 4 unmodified lines | ||
| 852 | 945 | else |
| 853 | 946 | nxp_fspi_dll_override(f); |
| 854 | 947 | |
| 948 | + /* Exit stop mode now that MCR0 and the DLL have been reconfigured. */ | |
| 949 | + reg = fspi_readl(f, f->iobase + FSPI_MCR0); | |
| 950 | + fspi_writel(f, reg & ~FSPI_MCR0_MDIS, f->iobase + FSPI_MCR0); | |
| 951 | + | |
| 855 | 952 | f->pre_op_rate = op->max_freq; |
| 856 | 953 | |
| 857 | 954 | f->selected = spi_get_chipselect(spi, 0); |
| 955 | + | |
| 956 | + return 0; | |
| 858 | 957 | } |
| 859 | 958 | |
| 860 | 959 | static int nxp_fspi_read_ahb(struct nxp_fspi *f, const struct spi_mem_op *op) |
| @@ -1042,7 +1141,16 @@ static int nxp_fspi_exec_op(struct spi_mem *mem, const struct spi_mem_op *op) · 181 unmodified lines | ||
| 1042 | 1141 | FSPI_STS0_ARB_IDLE, 1, POLL_TOUT, true); |
| 1043 | 1142 | WARN_ON(err); |
| 1044 | 1143 | |
| 1045 | − nxp_fspi_select_mem(f, mem->spi, op); | |
| 1144 | + err = nxp_fspi_select_mem(f, mem->spi, op); | |
| 1145 | + if (err) { | |
| 1146 | + /* | |
| 1147 | + * On failure the FlexSPI clock may be left disabled, so avoid | |
| 1148 | + * any further register access (which would trigger a synchronous | |
| 1149 | + * external abort) and bail out. | |
| 1150 | + */ | |
| 1151 | + pm_runtime_put_autosuspend(f->dev); | |
| 1152 | + return err; | |
| 1153 | + } | |
| 1046 | 1154 | |
| 1047 | 1155 | nxp_fspi_prepare_lut(f, op); |
| 1048 | 1156 | /* |
| @@ -1444,10 +1552,11 @@ static const struct dev_pm_ops nxp_fspi_pm_ops = { · 395 unmodified lines | ||
| 1444 | 1552 | static const struct of_device_id nxp_fspi_dt_ids[] = { |
| 1445 | 1553 | { .compatible = "nxp,lx2160a-fspi", .data = (void *)&lx2160a_data, }, |
| 1446 | 1554 | { .compatible = "nxp,imx8mm-fspi", .data = (void *)&imx8mm_data, }, |
| 1447 | − { .compatible = "nxp,imx8mp-fspi", .data = (void *)&imx8mm_data, }, | |
| 1555 | + { .compatible = "nxp,imx8mp-fspi", .data = (void *)&imx8mp_data, }, | |
| 1448 | 1556 | { .compatible = "nxp,imx8qxp-fspi", .data = (void *)&imx8qxp_data, }, |
| 1449 | 1557 | { .compatible = "nxp,imx8dxl-fspi", .data = (void *)&imx8dxl_data, }, |
| 1450 | 1558 | { .compatible = "nxp,imx8ulp-fspi", .data = (void *)&imx8ulp_data, }, |
| 1559 | + { .compatible = "nxp,imx95-fspi", .data = (void *)&imx95_data, }, | |
| 1451 | 1560 | { /* sentinel */ } |
| 1452 | 1561 | }; |
| 1453 | 1562 | MODULE_DEVICE_TABLE(of, nxp_fspi_dt_ids); |
| @@ -775,7 +775,8 @@ static int qcom_qspi_probe(struct platform_device *pdev) · 774 unmodified lines | ||
| 775 | 775 | return dev_err_probe(dev, ret, "could not set DMA mask\n"); |
| 776 | 776 | |
| 777 | 777 | host->max_speed_hz = 300000000; |
| 778 | − host->max_dma_len = 65536; /* as per HPG */ | |
| 778 | + /* as per HPG, it is 64KB, limit to 60KB to avoid boundary condition failures */ | |
| 779 | + host->max_dma_len = 0xf000; | |
| 779 | 780 | host->dma_alignment = QSPI_ALIGN_REQ; |
| 780 | 781 | host->num_chipselect = QSPI_NUM_CS; |
| 781 | 782 | host->bus_num = -1; |
| @@ -1358,6 +1358,22 @@ static int qcom_spi_send_cmdaddr(struct qcom_nand_controller *snandc, · 1357 unmodified lines | ||
| 1358 | 1358 | snandc->regs->addr0 = cpu_to_le32(op->addr.val); |
| 1359 | 1359 | snandc->regs->addr1 = cpu_to_le32(0); |
| 1360 | 1360 | |
| 1361 | + /* | |
| 1362 | + * The feature value has to reach NAND_FLASH_FEATURES before the | |
| 1363 | + * command is executed, otherwise the controller programs the chip | |
| 1364 | + * with whatever the register happened to hold from a previous | |
| 1365 | + * operation. | |
| 1366 | + */ | |
| 1367 | + if (opcode == SPINAND_SET_FEATURE) { | |
| 1368 | + u32 ftr = 0; | |
| 1369 | + | |
| 1370 | + memcpy(&ftr, op->data.buf.out, | |
| 1371 | + min_t(size_t, op->data.nbytes, sizeof(ftr))); | |
| 1372 | + snandc->regs->flash_feature = cpu_to_le32(ftr); | |
| 1373 | + qcom_write_reg_dma(snandc, &snandc->regs->flash_feature, | |
| 1374 | + NAND_FLASH_FEATURES, 1, NAND_BAM_NEXT_SGL); | |
| 1375 | + } | |
| 1376 | + | |
| 1361 | 1377 | qcom_write_reg_dma(snandc, &snandc->regs->cmd, NAND_FLASH_CMD, 3, NAND_BAM_NEXT_SGL); |
| 1362 | 1378 | qcom_write_reg_dma(snandc, &snandc->regs->exec, NAND_EXEC_CMD, 1, NAND_BAM_NEXT_SGL); |
| 1363 | 1379 | |
| @@ -1395,10 +1411,8 @@ static int qcom_spi_io_op(struct qcom_nand_controller *snandc, const struct spi_ · 31 unmodified lines | ||
| 1395 | 1411 | copy_ftr = true; |
| 1396 | 1412 | break; |
| 1397 | 1413 | case SPINAND_SET_FEATURE: |
| 1398 | − snandc->regs->flash_feature = cpu_to_le32(*(u32 *)op->data.buf.out); | |
| 1399 | − qcom_write_reg_dma(snandc, &snandc->regs->flash_feature, | |
| 1400 | − NAND_FLASH_FEATURES, 1, NAND_BAM_NEXT_SGL); | |
| 1401 | − break; | |
| 1414 | + /* fully handled by qcom_spi_send_cmdaddr() */ | |
| 1415 | + return 0; | |
| 1402 | 1416 | case SPINAND_PROGRAM_EXECUTE: |
| 1403 | 1417 | case SPINAND_WRITE_EN: |
| 1404 | 1418 | case SPINAND_RESET: |
| @@ -278,25 +278,25 @@ static int k1_spi_dma_one(struct spi_controller *host, struct spi_device *spi, · 277 unmodified lines | ||
| 278 | 278 | struct spi_transfer *transfer) |
| 279 | 279 | { |
| 280 | 280 | struct k1_spi_driver_data *drv_data = spi_controller_get_devdata(host); |
| 281 | − struct dma_async_tx_descriptor *desc; | |
| 281 | + struct dma_async_tx_descriptor *txdesc, *rxdesc; | |
| 282 | 282 | u32 val; |
| 283 | 283 | |
| 284 | − /* Prepare the TX descriptor and submit it */ | |
| 285 | − desc = k1_spi_dma_prep(drv_data, transfer, true); | |
| 286 | − if (!desc) | |
| 284 | + /* Prepare the TX descriptor */ | |
| 285 | + txdesc = k1_spi_dma_prep(drv_data, transfer, true); | |
| 286 | + if (!txdesc) | |
| 287 | 287 | goto fallback; |
| 288 | − dmaengine_submit(desc); | |
| 289 | 288 | |
| 290 | − /* Prepare the RX descriptor and submit it */ | |
| 291 | − desc = k1_spi_dma_prep(drv_data, transfer, false); | |
| 292 | − if (!desc) | |
| 289 | + /* Prepare the RX descriptor */ | |
| 290 | + rxdesc = k1_spi_dma_prep(drv_data, transfer, false); | |
| 291 | + if (!rxdesc) | |
| 293 | 292 | goto fallback; |
| 294 | 293 | |
| 295 | 294 | /* When RX is complete we also know TX has completed */ |
| 296 | − desc->callback = k1_spi_dma_callback; | |
| 297 | − desc->callback_param = drv_data; | |
| 295 | + rxdesc->callback = k1_spi_dma_callback; | |
| 296 | + rxdesc->callback_param = drv_data; | |
| 298 | 297 | |
| 299 | − dmaengine_submit(desc); | |
| 298 | + dmaengine_submit(txdesc); | |
| 299 | + dmaengine_submit(rxdesc); | |
| 300 | 300 | |
| 301 | 301 | val = readl(drv_data->base + SSP_TOP_CTRL); |
| 302 | 302 | val |= TOP_TRAIL; /* Trailing bytes handled by DMA */ |
| @@ -371,6 +371,9 @@ unsigned char *rtw_get_wpa_ie(unsigned char *pie, int *wpa_ie_len, int limit) · 370 unmodified lines | ||
| 371 | 371 | pbuf = rtw_get_ie(pbuf, WLAN_EID_VENDOR_SPECIFIC, &len, limit_new); |
| 372 | 372 | |
| 373 | 373 | if (pbuf) { |
| 374 | + if (len < 6) | |
| 375 | + goto check_next_ie; | |
| 376 | + | |
| 374 | 377 | /* check if oui matches... */ |
| 375 | 378 | if (memcmp((pbuf + 2), wpa_oui_type, sizeof(wpa_oui_type))) |
| 376 | 379 | goto check_next_ie; |
| @@ -879,7 +879,7 @@ unsigned int OnAuthClient(struct adapter *padapter, union recv_frame *precv_fram · 878 unmodified lines | ||
| 879 | 879 | p = rtw_get_ie(pframe + WLAN_HDR_A3_LEN + _AUTH_IE_OFFSET_, WLAN_EID_CHALLENGE, (int *)&len, |
| 880 | 880 | pkt_len - WLAN_HDR_A3_LEN - _AUTH_IE_OFFSET_); |
| 881 | 881 | |
| 882 | − if (!p) | |
| 882 | + if (!p || len != WLAN_AUTH_CHALLENGE_LEN) | |
| 883 | 883 | goto authclnt_fail; |
| 884 | 884 | |
| 885 | 885 | memcpy(pmlmeinfo->chg_txt, p + 2, len); |
| @@ -666,6 +666,9 @@ int WMM_param_handler(struct adapter *padapter, struct ndis_80211_var_ie *pIE) · 665 unmodified lines | ||
| 666 | 666 | return false; |
| 667 | 667 | } |
| 668 | 668 | |
| 669 | + if (pIE->length != WLAN_WMM_LEN) | |
| 670 | + return false; | |
| 671 | + | |
| 669 | 672 | if (!memcmp(&(pmlmeinfo->WMM_param), (pIE->data + 6), sizeof(struct WMM_para_element))) |
| 670 | 673 | return false; |
| 671 | 674 | |
| @@ -2033,6 +2033,8 @@ static netdev_tx_t rtw_cfg80211_monitor_if_xmit_entry(struct sk_buff *skb, struc · 2032 unmodified lines | ||
| 2033 | 2033 | |
| 2034 | 2034 | /* Skip the ratio tap header */ |
| 2035 | 2035 | skb_pull(skb, rtap_len); |
| 2036 | + if (skb->len < dot11_hdr_len) | |
| 2037 | + goto fail; | |
| 2036 | 2038 | |
| 2037 | 2039 | dot11_hdr = (struct ieee80211_hdr *)skb->data; |
| 2038 | 2040 | frame_control = le16_to_cpu(dot11_hdr->frame_control); |
| @@ -2045,6 +2047,8 @@ static netdev_tx_t rtw_cfg80211_monitor_if_xmit_entry(struct sk_buff *skb, struc · 6 unmodified lines | ||
| 2045 | 2047 | qos_len = 2; |
| 2046 | 2048 | if ((frame_control & 0x0300) == 0x0300) |
| 2047 | 2049 | dot11_hdr_len += 6; |
| 2050 | + if (skb->len < dot11_hdr_len + qos_len + snap_len) | |
| 2051 | + goto fail; | |
| 2048 | 2052 | |
| 2049 | 2053 | memcpy(dst_mac_addr, dot11_hdr->addr1, sizeof(dst_mac_addr)); |
| 2050 | 2054 | memcpy(src_mac_addr, dot11_hdr->addr2, sizeof(src_mac_addr)); |
| @@ -906,7 +906,7 @@ static sense_reason_t iblock_execute_pr_out(struct se_cmd *cmd, u8 sa, u64 key, · 905 unmodified lines | ||
| 906 | 906 | break; |
| 907 | 907 | case PRO_PREEMPT: |
| 908 | 908 | case PRO_PREEMPT_AND_ABORT: |
| 909 | − if (!ops->pr_clear) { | |
| 909 | + if (!ops->pr_preempt) { | |
| 910 | 910 | pr_err("block_device does not support pr_preempt.\n"); |
| 911 | 911 | return TCM_UNSUPPORTED_SCSI_OPCODE; |
| 912 | 912 | } |
| @@ -916,8 +916,8 @@ static sense_reason_t iblock_execute_pr_out(struct se_cmd *cmd, u8 sa, u64 key, · 3 unmodified lines | ||
| 916 | 916 | sa == PRO_PREEMPT_AND_ABORT); |
| 917 | 917 | break; |
| 918 | 918 | case PRO_RELEASE: |
| 919 | − if (!ops->pr_clear) { | |
| 920 | − pr_err("block_device does not support pr_pclear.\n"); | |
| 919 | + if (!ops->pr_release) { | |
| 920 | + pr_err("block_device does not support pr_release.\n"); | |
| 921 | 921 | return TCM_UNSUPPORTED_SCSI_OPCODE; |
| 922 | 922 | } |
| 923 | 923 | |
| @@ -1734,6 +1734,7 @@ int target_init_cmd(struct se_cmd *se_cmd, struct se_session *se_sess, · 1733 unmodified lines | ||
| 1734 | 1734 | u32 data_length, int task_attr, int data_dir, int flags) |
| 1735 | 1735 | { |
| 1736 | 1736 | struct se_portal_group *se_tpg; |
| 1737 | + int ret; | |
| 1737 | 1738 | |
| 1738 | 1739 | se_tpg = se_sess->se_tpg; |
| 1739 | 1740 | BUG_ON(!se_tpg); |
| @@ -1763,7 +1764,11 @@ int target_init_cmd(struct se_cmd *se_cmd, struct se_session *se_sess, · 23 unmodified lines | ||
| 1763 | 1764 | * necessary for fabrics using TARGET_SCF_ACK_KREF that expect a second |
| 1764 | 1765 | * kref_put() to happen during fabric packet acknowledgement. |
| 1765 | 1766 | */ |
| 1766 | − return target_get_sess_cmd(se_cmd, flags & TARGET_SCF_ACK_KREF); | |
| 1767 | + ret = target_get_sess_cmd(se_cmd, flags & TARGET_SCF_ACK_KREF); | |
| 1768 | + if (ret) | |
| 1769 | + se_cmd->cmd_cnt = NULL; | |
| 1770 | + | |
| 1771 | + return ret; | |
| 1767 | 1772 | } |
| 1768 | 1773 | EXPORT_SYMBOL_GPL(target_init_cmd); |
| 1769 | 1774 | |
| @@ -2039,8 +2044,10 @@ int target_submit_tmr(struct se_cmd *se_cmd, struct se_session *se_sess, · 269 unmodified lines | ||
| 2039 | 2044 | * allocation failure. |
| 2040 | 2045 | */ |
| 2041 | 2046 | ret = core_tmr_alloc_req(se_cmd, fabric_tmr_ptr, tm_type, gfp); |
| 2042 | − if (ret < 0) | |
| 2047 | + if (ret < 0) { | |
| 2048 | + se_cmd->cmd_cnt = NULL; | |
| 2043 | 2049 | return -ENOMEM; |
| 2050 | + } | |
| 2044 | 2051 | |
| 2045 | 2052 | if (tm_type == TMR_ABORT_TASK) |
| 2046 | 2053 | se_cmd->se_tmr_req->ref_task_tag = tag; |
| @@ -2048,6 +2055,7 @@ int target_submit_tmr(struct se_cmd *se_cmd, struct se_session *se_sess, · 1 unmodified line | ||
| 2048 | 2055 | /* See target_submit_cmd for commentary */ |
| 2049 | 2056 | ret = target_get_sess_cmd(se_cmd, flags & TARGET_SCF_ACK_KREF); |
| 2050 | 2057 | if (ret) { |
| 2058 | + se_cmd->cmd_cnt = NULL; | |
| 2051 | 2059 | core_tmr_release_req(se_cmd->se_tmr_req); |
| 2052 | 2060 | return ret; |
| 2053 | 2061 | } |
| @@ -19,19 +19,28 @@ · 18 unmodified lines | ||
| 19 | 19 | #include "thermal_hwmon.h" |
| 20 | 20 | #include "thermal_core.h" |
| 21 | 21 | |
| 22 | −/* | |
| 23 | − * Needs to be large enough to hold a thermal zone type string followed by an | |
| 24 | − * underline character and a 32-bit integer in decimal representation. | |
| 25 | − */ | |
| 26 | −#define THERMAL_HWMON_NAME_LENGTH (THERMAL_NAME_LENGTH + 11) | |
| 27 | − | |
| 28 | 22 | /* hwmon sys I/F */ |
| 29 | 23 | /* thermal zone devices with the same type share one hwmon device */ |
| 30 | 24 | struct thermal_hwmon_device { |
| 31 | − char name[THERMAL_HWMON_NAME_LENGTH]; | |
| 25 | + char type[THERMAL_NAME_LENGTH]; | |
| 32 | 26 | struct device *device; |
| 27 | + int count; | |
| 28 | + struct list_head tz_list; | |
| 33 | 29 | struct list_head node; |
| 30 | +}; | |
| 31 | + | |
| 32 | +struct thermal_hwmon_attr { | |
| 33 | + struct device_attribute attr; | |
| 34 | + char name[16]; | |
| 35 | +}; | |
| 36 | + | |
| 37 | +/* one temperature input for each thermal zone */ | |
| 38 | +struct thermal_hwmon_temp { | |
| 39 | + struct list_head hwmon_node; | |
| 34 | 40 | struct thermal_zone_device *tz; |
| 41 | + struct thermal_hwmon_attr temp_input; /* hwmon sys attr */ | |
| 42 | + struct thermal_hwmon_attr temp_crit; /* hwmon sys attr */ | |
| 43 | + bool temp_crit_present; | |
| 35 | 44 | }; |
| 36 | 45 | |
| 37 | 46 | static LIST_HEAD(thermal_hwmon_list); |
| @@ -39,14 +48,19 @@ static LIST_HEAD(thermal_hwmon_list); · 1 unmodified line | ||
| 39 | 48 | static DEFINE_MUTEX(thermal_hwmon_list_lock); |
| 40 | 49 | |
| 41 | 50 | static ssize_t |
| 42 | −temp1_input_show(struct device *dev, struct device_attribute *attr, char *buf) | |
| 51 | +temp_input_show(struct device *dev, struct device_attribute *attr, char *buf) | |
| 43 | 52 | { |
| 44 | − struct thermal_hwmon_device *hwmon = dev_get_drvdata(dev); | |
| 45 | − struct thermal_zone_device *tz = hwmon->tz; | |
| 46 | 53 | int temperature; |
| 47 | 54 | int ret; |
| 55 | + struct thermal_hwmon_attr *hwmon_attr | |
| 56 | + = container_of(attr, struct thermal_hwmon_attr, attr); | |
| 57 | + struct thermal_hwmon_temp *temp | |
| 58 | + = container_of(hwmon_attr, struct thermal_hwmon_temp, | |
| 59 | + temp_input); | |
| 60 | + struct thermal_zone_device *tz = temp->tz; | |
| 48 | 61 | |
| 49 | 62 | ret = thermal_zone_get_temp(tz, &temperature); |
| 63 | + | |
| 50 | 64 | if (ret) |
| 51 | 65 | return ret; |
| 52 | 66 | |
| @@ -54,10 +68,14 @@ temp1_input_show(struct device *dev, struct device_attribute *attr, char *buf) · 1 unmodified line | ||
| 54 | 68 | } |
| 55 | 69 | |
| 56 | 70 | static ssize_t |
| 57 | −temp1_crit_show(struct device *dev, struct device_attribute *attr, char *buf) | |
| 71 | +temp_crit_show(struct device *dev, struct device_attribute *attr, char *buf) | |
| 58 | 72 | { |
| 59 | − struct thermal_hwmon_device *hwmon = dev_get_drvdata(dev); | |
| 60 | − struct thermal_zone_device *tz = hwmon->tz; | |
| 73 | + struct thermal_hwmon_attr *hwmon_attr | |
| 74 | + = container_of(attr, struct thermal_hwmon_attr, attr); | |
| 75 | + struct thermal_hwmon_temp *temp | |
| 76 | + = container_of(hwmon_attr, struct thermal_hwmon_temp, | |
| 77 | + temp_crit); | |
| 78 | + struct thermal_zone_device *tz = temp->tz; | |
| 61 | 79 | int temperature; |
| 62 | 80 | int ret; |
| 63 | 81 | |
| @@ -70,97 +88,169 @@ temp1_crit_show(struct device *dev, struct device_attribute *attr, char *buf) · 6 unmodified lines | ||
| 70 | 88 | return sysfs_emit(buf, "%d\n", temperature); |
| 71 | 89 | } |
| 72 | 90 | |
| 73 | −static DEVICE_ATTR_RO(temp1_input); | |
| 74 | −static DEVICE_ATTR_RO(temp1_crit); | |
| 75 | − | |
| 76 | −static struct attribute *thermal_hwmon_attrs[] = { | |
| 77 | − &dev_attr_temp1_input.attr, | |
| 78 | − &dev_attr_temp1_crit.attr, | |
| 79 | − NULL, | |
| 80 | −}; | |
| 81 | 91 | |
| 82 | −static umode_t thermal_hwmon_attr_is_visible(struct kobject *kobj, | |
| 83 | − struct attribute *a, int n) | |
| 92 | +static struct thermal_hwmon_device * | |
| 93 | +thermal_hwmon_lookup_by_type(const struct thermal_zone_device *tz) | |
| 84 | 94 | { |
| 85 | − if (a == &dev_attr_temp1_input.attr) | |
| 86 | − return a->mode; | |
| 87 | − | |
| 88 | − if (a == &dev_attr_temp1_crit.attr) { | |
| 89 | − struct thermal_hwmon_device *hwmon = dev_get_drvdata(kobj_to_dev(kobj)); | |
| 90 | − struct thermal_zone_device *tz = hwmon->tz; | |
| 91 | − int dummy; | |
| 95 | + struct thermal_hwmon_device *hwmon; | |
| 96 | + char type[THERMAL_NAME_LENGTH]; | |
| 92 | 97 | |
| 93 | − if (tz->ops.get_crit_temp && !tz->ops.get_crit_temp(tz, &dummy)) | |
| 94 | − return a->mode; | |
| 98 | + mutex_lock(&thermal_hwmon_list_lock); | |
| 99 | + list_for_each_entry(hwmon, &thermal_hwmon_list, node) { | |
| 100 | + strscpy(type, tz->type); | |
| 101 | + strreplace(type, '-', '_'); | |
| 102 | + if (!strcmp(hwmon->type, type)) { | |
| 103 | + mutex_unlock(&thermal_hwmon_list_lock); | |
| 104 | + return hwmon; | |
| 105 | + } | |
| 95 | 106 | } |
| 107 | + mutex_unlock(&thermal_hwmon_list_lock); | |
| 96 | 108 | |
| 97 | − return 0; | |
| 109 | + return NULL; | |
| 98 | 110 | } |
| 99 | 111 | |
| 100 | −static const struct attribute_group thermal_hwmon_group = { | |
| 101 | − .attrs = thermal_hwmon_attrs, | |
| 102 | − .is_visible = thermal_hwmon_attr_is_visible, | |
| 103 | −}; | |
| 112 | +/* Find the temperature input matching a given thermal zone */ | |
| 113 | +static struct thermal_hwmon_temp * | |
| 114 | +thermal_hwmon_lookup_temp(const struct thermal_hwmon_device *hwmon, | |
| 115 | + const struct thermal_zone_device *tz) | |
| 116 | +{ | |
| 117 | + struct thermal_hwmon_temp *temp; | |
| 104 | 118 | |
| 105 | −__ATTRIBUTE_GROUPS(thermal_hwmon); | |
| 119 | + mutex_lock(&thermal_hwmon_list_lock); | |
| 120 | + list_for_each_entry(temp, &hwmon->tz_list, hwmon_node) | |
| 121 | + if (temp->tz == tz) { | |
| 122 | + mutex_unlock(&thermal_hwmon_list_lock); | |
| 123 | + return temp; | |
| 124 | + } | |
| 125 | + mutex_unlock(&thermal_hwmon_list_lock); | |
| 126 | + | |
| 127 | + return NULL; | |
| 128 | +} | |
| 129 | + | |
| 130 | +static bool thermal_zone_crit_temp_valid(struct thermal_zone_device *tz) | |
| 131 | +{ | |
| 132 | + int temp; | |
| 133 | + return tz->ops.get_crit_temp && !tz->ops.get_crit_temp(tz, &temp); | |
| 134 | +} | |
| 106 | 135 | |
| 107 | 136 | int thermal_add_hwmon_sysfs(struct thermal_zone_device *tz) |
| 108 | 137 | { |
| 109 | 138 | struct thermal_hwmon_device *hwmon; |
| 139 | + struct thermal_hwmon_temp *temp; | |
| 140 | + int new_hwmon_device = 1; | |
| 141 | + int result; | |
| 142 | + | |
| 143 | + hwmon = thermal_hwmon_lookup_by_type(tz); | |
| 144 | + if (hwmon) { | |
| 145 | + new_hwmon_device = 0; | |
| 146 | + goto register_sys_interface; | |
| 147 | + } | |
| 110 | 148 | |
| 111 | 149 | hwmon = kzalloc_obj(*hwmon); |
| 112 | 150 | if (!hwmon) |
| 113 | 151 | return -ENOMEM; |
| 114 | 152 | |
| 115 | − hwmon->tz = tz; | |
| 116 | − /* | |
| 117 | − * Append the thermal zone ID preceded by an underline character to the | |
| 118 | − * type to disambiguate the sensors command output. | |
| 119 | − */ | |
| 120 | − scnprintf(hwmon->name, THERMAL_HWMON_NAME_LENGTH, "%s_%d", tz->type, tz->id); | |
| 121 | − strreplace(hwmon->name, '-', '_'); | |
| 153 | + INIT_LIST_HEAD(&hwmon->tz_list); | |
| 154 | + strscpy(hwmon->type, tz->type, THERMAL_NAME_LENGTH); | |
| 155 | + strreplace(hwmon->type, '-', '_'); | |
| 122 | 156 | hwmon->device = hwmon_device_register_for_thermal(&tz->device, |
| 123 | − hwmon->name, hwmon, | |
| 124 | − thermal_hwmon_groups); | |
| 157 | + hwmon->type, hwmon); | |
| 125 | 158 | if (IS_ERR(hwmon->device)) { |
| 126 | − int result = PTR_ERR(hwmon->device); | |
| 159 | + result = PTR_ERR(hwmon->device); | |
| 160 | + goto free_mem; | |
| 161 | + } | |
| 127 | 162 | |
| 128 | − kfree(hwmon); | |
| 129 | − return result; | |
| 163 | + register_sys_interface: | |
| 164 | + temp = kzalloc_obj(*temp); | |
| 165 | + if (!temp) { | |
| 166 | + result = -ENOMEM; | |
| 167 | + goto unregister_name; | |
| 168 | + } | |
| 169 | + | |
| 170 | + temp->tz = tz; | |
| 171 | + hwmon->count++; | |
| 172 | + | |
| 173 | + snprintf(temp->temp_input.name, sizeof(temp->temp_input.name), | |
| 174 | + "temp%d_input", hwmon->count); | |
| 175 | + temp->temp_input.attr.attr.name = temp->temp_input.name; | |
| 176 | + temp->temp_input.attr.attr.mode = 0444; | |
| 177 | + temp->temp_input.attr.show = temp_input_show; | |
| 178 | + sysfs_attr_init(&temp->temp_input.attr.attr); | |
| 179 | + result = device_create_file(hwmon->device, &temp->temp_input.attr); | |
| 180 | + if (result) | |
| 181 | + goto free_temp_mem; | |
| 182 | + | |
| 183 | + if (thermal_zone_crit_temp_valid(tz)) { | |
| 184 | + snprintf(temp->temp_crit.name, | |
| 185 | + sizeof(temp->temp_crit.name), | |
| 186 | + "temp%d_crit", hwmon->count); | |
| 187 | + temp->temp_crit.attr.attr.name = temp->temp_crit.name; | |
| 188 | + temp->temp_crit.attr.attr.mode = 0444; | |
| 189 | + temp->temp_crit.attr.show = temp_crit_show; | |
| 190 | + sysfs_attr_init(&temp->temp_crit.attr.attr); | |
| 191 | + result = device_create_file(hwmon->device, | |
| 192 | + &temp->temp_crit.attr); | |
| 193 | + if (result) | |
| 194 | + goto unregister_input; | |
| 195 | + | |
| 196 | + temp->temp_crit_present = true; | |
| 130 | 197 | } |
| 131 | 198 | |
| 132 | − /* The list is needed for hwmon lookup during removal. */ | |
| 133 | 199 | mutex_lock(&thermal_hwmon_list_lock); |
| 134 | − list_add_tail(&hwmon->node, &thermal_hwmon_list); | |
| 200 | + if (new_hwmon_device) | |
| 201 | + list_add_tail(&hwmon->node, &thermal_hwmon_list); | |
| 202 | + list_add_tail(&temp->hwmon_node, &hwmon->tz_list); | |
| 135 | 203 | mutex_unlock(&thermal_hwmon_list_lock); |
| 136 | 204 | |
| 137 | 205 | return 0; |
| 206 | + | |
| 207 | + unregister_input: | |
| 208 | + device_remove_file(hwmon->device, &temp->temp_input.attr); | |
| 209 | + free_temp_mem: | |
| 210 | + kfree(temp); | |
| 211 | + unregister_name: | |
| 212 | + if (new_hwmon_device) | |
| 213 | + hwmon_device_unregister(hwmon->device); | |
| 214 | + free_mem: | |
| 215 | + if (new_hwmon_device) | |
| 216 | + kfree(hwmon); | |
| 217 | + | |
| 218 | + return result; | |
| 138 | 219 | } |
| 139 | 220 | EXPORT_SYMBOL_GPL(thermal_add_hwmon_sysfs); |
| 140 | 221 | |
| 141 | −static struct thermal_hwmon_device * | |
| 142 | −thermal_hwmon_lookup(const struct thermal_zone_device *tz) | |
| 222 | +void thermal_remove_hwmon_sysfs(struct thermal_zone_device *tz) | |
| 143 | 223 | { |
| 144 | 224 | struct thermal_hwmon_device *hwmon; |
| 225 | + struct thermal_hwmon_temp *temp; | |
| 145 | 226 | |
| 146 | − list_for_each_entry(hwmon, &thermal_hwmon_list, node) { | |
| 147 | − if (hwmon->tz == tz) | |
| 148 | − return hwmon; | |
| 227 | + hwmon = thermal_hwmon_lookup_by_type(tz); | |
| 228 | + if (unlikely(!hwmon)) { | |
| 229 | + /* Should never happen... */ | |
| 230 | + dev_dbg(&tz->device, "hwmon device lookup failed!\n"); | |
| 231 | + return; | |
| 149 | 232 | } |
| 150 | − return NULL; | |
| 151 | −} | |
| 152 | 233 | |
| 153 | −void thermal_remove_hwmon_sysfs(struct thermal_zone_device *tz) | |
| 154 | −{ | |
| 155 | − struct thermal_hwmon_device *hwmon; | |
| 234 | + temp = thermal_hwmon_lookup_temp(hwmon, tz); | |
| 235 | + if (unlikely(!temp)) { | |
| 236 | + /* Should never happen... */ | |
| 237 | + dev_dbg(&tz->device, "temperature input lookup failed!\n"); | |
| 238 | + return; | |
| 239 | + } | |
| 156 | 240 | |
| 157 | − scoped_guard(mutex, &thermal_hwmon_list_lock) { | |
| 158 | − hwmon = thermal_hwmon_lookup(tz); | |
| 159 | − if (!hwmon) | |
| 160 | − return; | |
| 241 | + device_remove_file(hwmon->device, &temp->temp_input.attr); | |
| 242 | + if (temp->temp_crit_present) | |
| 243 | + device_remove_file(hwmon->device, &temp->temp_crit.attr); | |
| 161 | 244 | |
| 162 | − list_del(&hwmon->node); | |
| 245 | + mutex_lock(&thermal_hwmon_list_lock); | |
| 246 | + list_del(&temp->hwmon_node); | |
| 247 | + kfree(temp); | |
| 248 | + if (!list_empty(&hwmon->tz_list)) { | |
| 249 | + mutex_unlock(&thermal_hwmon_list_lock); | |
| 250 | + return; | |
| 163 | 251 | } |
| 252 | + list_del(&hwmon->node); | |
| 253 | + mutex_unlock(&thermal_hwmon_list_lock); | |
| 164 | 254 | |
| 165 | 255 | hwmon_device_unregister(hwmon->device); |
| 166 | 256 | kfree(hwmon); |
| @@ -394,9 +394,16 @@ static int tb_drom_parse_entry_port(struct tb_switch *sw, · 393 unmodified lines | ||
| 394 | 394 | return -EIO; |
| 395 | 395 | } |
| 396 | 396 | port->link_nr = entry->link_nr; |
| 397 | − if (entry->has_dual_link_port) | |
| 397 | + if (entry->has_dual_link_port) { | |
| 398 | + if (entry->dual_link_port_nr > sw->config.max_port_number) { | |
| 399 | + tb_sw_warn(sw, | |
| 400 | + "port entry has invalid dual link port number %u\n", | |
| 401 | + entry->dual_link_port_nr); | |
| 402 | + return -EIO; | |
| 403 | + } | |
| 398 | 404 | port->dual_link_port = |
| 399 | 405 | &port->sw->ports[entry->dual_link_port_nr]; |
| 406 | + } | |
| 400 | 407 | } |
| 401 | 408 | return 0; |
| 402 | 409 | } |
| @@ -2341,7 +2341,7 @@ static int icm_usb4_switch_op(struct tb_switch *sw, u16 opcode, u32 *metadata, · 2340 unmodified lines | ||
| 2341 | 2341 | if (tx_data_len) { |
| 2342 | 2342 | request.data_len_valid |= ICM_USB4_SWITCH_DATA_VALID; |
| 2343 | 2343 | if (tx_data_len < ARRAY_SIZE(request.data)) |
| 2344 | − request.data_len_valid = | |
| 2344 | + request.data_len_valid |= | |
| 2345 | 2345 | tx_data_len & ICM_USB4_SWITCH_DATA_LEN_MASK; |
| 2346 | 2346 | memcpy(request.data, tx_data, tx_data_len * sizeof(u32)); |
| 2347 | 2347 | } |
| @@ -1226,6 +1226,8 @@ int nhi_probe(struct tb_nhi *nhi) · 1225 unmodified lines | ||
| 1226 | 1226 | return dev_err_probe(dev, res, "NHI specific init failed\n"); |
| 1227 | 1227 | } |
| 1228 | 1228 | |
| 1229 | + init_completion(&nhi->domain_released); | |
| 1230 | + | |
| 1229 | 1231 | tb = nhi_select_cm(nhi); |
| 1230 | 1232 | if (!tb) |
| 1231 | 1233 | return dev_err_probe(dev, -ENODEV, |
| @@ -1233,8 +1235,6 @@ int nhi_probe(struct tb_nhi *nhi) · 1 unmodified line | ||
| 1233 | 1235 | |
| 1234 | 1236 | dev_dbg(dev, "NHI initialized, starting thunderbolt\n"); |
| 1235 | 1237 | |
| 1236 | − init_completion(&nhi->domain_released); | |
| 1237 | − | |
| 1238 | 1238 | res = tb_domain_add(tb, host_reset); |
| 1239 | 1239 | if (res) { |
| 1240 | 1240 | /* |
| @@ -257,7 +257,7 @@ static void tbstream_ring_free(struct tbstream_ring *ring) · 256 unmodified lines | ||
| 257 | 257 | |
| 258 | 258 | if (sf->frame.buffer_phy) |
| 259 | 259 | dma_unmap_page(dma_dev, sf->frame.buffer_phy, |
| 260 | − tb_ring_frame_size(&sf->frame), dir); | |
| 260 | + TB_MAX_FRAME_SIZE, dir); | |
| 261 | 261 | sf->frame.buffer_phy = 0; |
| 262 | 262 | if (sf->page) |
| 263 | 263 | __free_page(sf->page); |
| @@ -609,7 +609,7 @@ static int tb_consumed_dp_bandwidth(struct tb *tb, · 608 unmodified lines | ||
| 609 | 609 | int *consumed_up, |
| 610 | 610 | int *consumed_down) |
| 611 | 611 | { |
| 612 | − int group_reserved[MAX_GROUPS] = {}; | |
| 612 | + int group_reserved[MAX_GROUPS + 1] = {}; | |
| 613 | 613 | struct tb_cm *tcm = tb_priv(tb); |
| 614 | 614 | struct tb_tunnel *tunnel; |
| 615 | 615 | bool downstream; |
| @@ -211,11 +211,12 @@ void serial8250_rx_dma_flush(struct uart_8250_port *p) · 210 unmodified lines | ||
| 211 | 211 | { |
| 212 | 212 | struct uart_8250_dma *dma = p->dma; |
| 213 | 213 | |
| 214 | − if (dma->rx_running) { | |
| 215 | − dmaengine_pause(dma->rxchan); | |
| 216 | − __dma_rx_complete(p); | |
| 217 | − dmaengine_terminate_async(dma->rxchan); | |
| 218 | − } | |
| 214 | + if (!dma || !dma->rxchan || !dma->rx_running) | |
| 215 | + return; | |
| 216 | + | |
| 217 | + dmaengine_pause(dma->rxchan); | |
| 218 | + __dma_rx_complete(p); | |
| 219 | + dmaengine_terminate_async(dma->rxchan); | |
| 219 | 220 | } |
| 220 | 221 | EXPORT_SYMBOL_GPL(serial8250_rx_dma_flush); |
| 221 | 222 | |
| @@ -324,6 +325,7 @@ void serial8250_release_dma(struct uart_8250_port *p) · 102 unmodified lines | ||
| 324 | 325 | |
| 325 | 326 | /* Release RX resources */ |
| 326 | 327 | dmaengine_terminate_sync(dma->rxchan); |
| 328 | + dma->rx_running = 0; | |
| 327 | 329 | dma_free_coherent(dma->rxchan->device->dev, dma->rx_size, dma->rx_buf, |
| 328 | 330 | dma->rx_addr); |
| 329 | 331 | dma_release_channel(dma->rxchan); |
| @@ -81,6 +81,40 @@ static int of_platform_serial_clk_notifier_cb(struct notifier_block *nb, unsigne · 80 unmodified lines | ||
| 81 | 81 | return NOTIFY_DONE; |
| 82 | 82 | } |
| 83 | 83 | |
| 84 | +static int lpc32xx_handle_irq(struct uart_port *port) | |
| 85 | +{ | |
| 86 | + struct uart_8250_port *up = up_to_u8250p(port); | |
| 87 | + unsigned int iir; | |
| 88 | + u16 status; | |
| 89 | + | |
| 90 | + guard(serial8250_rpm)(up); | |
| 91 | + | |
| 92 | + iir = serial_port_in(port, UART_IIR); | |
| 93 | + if (iir & UART_IIR_NO_INT) | |
| 94 | + return 0; | |
| 95 | + | |
| 96 | + guard(uart_port_lock_check_sysrq_irqsave)(port); | |
| 97 | + | |
| 98 | + /* | |
| 99 | + * The LPC32xx UART can assert an RX character-timeout interrupt while | |
| 100 | + * the RX FIFO is empty: IIR reports UART_IIR_RX_TIMEOUT but LSR.DR is | |
| 101 | + * clear. The timeout is only cleared by reading RHR, but the core RX | |
| 102 | + * path skips that read when the FIFO is empty, so the level-triggered | |
| 103 | + * IRQ re-fires forever and livelocks this single-core SoC. Do one | |
| 104 | + * throwaway RHR read to clear it; a healthy UART never reports a | |
| 105 | + * timeout with DR/BI clear, so no received data is ever discarded. | |
| 106 | + */ | |
| 107 | + if ((iir & 0x3f) == UART_IIR_RX_TIMEOUT) { | |
| 108 | + status = serial_lsr_in(up); | |
| 109 | + if (!(status & (UART_LSR_DR | UART_LSR_BI))) | |
| 110 | + serial_port_in(port, UART_RX); | |
| 111 | + } | |
| 112 | + | |
| 113 | + serial8250_handle_irq_locked(port, iir); | |
| 114 | + | |
| 115 | + return 1; | |
| 116 | +} | |
| 117 | + | |
| 84 | 118 | /* |
| 85 | 119 | * Fill a struct uart_port for a given device node |
| 86 | 120 | */ |
| @@ -173,6 +207,9 @@ static int of_platform_serial_setup(struct platform_device *ofdev, · 86 unmodified lines | ||
| 173 | 207 | case PORT_NPCM: |
| 174 | 208 | ret = npcm_setup(port); |
| 175 | 209 | break; |
| 210 | + case PORT_LPC3220: | |
| 211 | + port->handle_irq = lpc32xx_handle_irq; | |
| 212 | + break; | |
| 176 | 213 | default: |
| 177 | 214 | /* Nothing to do */ |
| 178 | 215 | ret = 0; |
| @@ -369,6 +406,7 @@ static struct platform_driver of_platform_serial_driver = { · 190 unmodified lines | ||
| 369 | 406 | |
| 370 | 407 | module_platform_driver(of_platform_serial_driver); |
| 371 | 408 | |
| 409 | +MODULE_IMPORT_NS("SERIAL_8250"); | |
| 372 | 410 | MODULE_AUTHOR("Arnd Bergmann <arnd@arndb.de>"); |
| 373 | 411 | MODULE_LICENSE("GPL"); |
| 374 | 412 | MODULE_DESCRIPTION("Serial Port driver for Open Firmware platform devices"); |
| @@ -309,6 +309,7 @@ enum pl011_rs485_tx_state { · 308 unmodified lines | ||
| 309 | 309 | WAIT_AFTER_RTS, |
| 310 | 310 | SEND, |
| 311 | 311 | WAIT_AFTER_SEND, |
| 312 | + WAIT_AFTER_SEND_DELAY, | |
| 312 | 313 | }; |
| 313 | 314 | |
| 314 | 315 | /* |
| @@ -1246,7 +1247,7 @@ static void pl011_dma_shutdown(struct uart_amba_port *uap) · 931 unmodified lines | ||
| 1246 | 1247 | |
| 1247 | 1248 | if (uap->using_tx_dma) { |
| 1248 | 1249 | /* In theory, this should already be done by pl011_dma_flush_buffer */ |
| 1249 | − dmaengine_terminate_all(uap->dmatx.chan); | |
| 1250 | + dmaengine_terminate_sync(uap->dmatx.chan); | |
| 1250 | 1251 | if (uap->dmatx.queued) { |
| 1251 | 1252 | dma_unmap_single(uap->dmatx.chan->device->dev, |
| 1252 | 1253 | uap->dmatx.dma, uap->dmatx.len, |
| @@ -1259,12 +1260,12 @@ static void pl011_dma_shutdown(struct uart_amba_port *uap) · 6 unmodified lines | ||
| 1259 | 1260 | } |
| 1260 | 1261 | |
| 1261 | 1262 | if (uap->using_rx_dma) { |
| 1262 | − dmaengine_terminate_all(uap->dmarx.chan); | |
| 1263 | + if (uap->dmarx.poll_rate) | |
| 1264 | + timer_delete_sync(&uap->dmarx.timer); | |
| 1265 | + dmaengine_terminate_sync(uap->dmarx.chan); | |
| 1263 | 1266 | /* Clean up the RX DMA */ |
| 1264 | 1267 | pl011_dmabuf_free(uap->dmarx.chan, &uap->dmarx.dbuf_a, DMA_FROM_DEVICE); |
| 1265 | 1268 | pl011_dmabuf_free(uap->dmarx.chan, &uap->dmarx.dbuf_b, DMA_FROM_DEVICE); |
| 1266 | − if (uap->dmarx.poll_rate) | |
| 1267 | − timer_delete_sync(&uap->dmarx.timer); | |
| 1268 | 1269 | uap->using_rx_dma = false; |
| 1269 | 1270 | } |
| 1270 | 1271 | } |
| @@ -1333,11 +1334,30 @@ static inline bool pl011_dma_rx_running(struct uart_amba_port *uap) · 62 unmodified lines | ||
| 1333 | 1334 | #define pl011_dma_flush_buffer NULL |
| 1334 | 1335 | #endif |
| 1335 | 1336 | |
| 1336 | −static void pl011_rs485_tx_stop(struct uart_amba_port *uap) | |
| 1337 | +static void pl011_rs485_tx_stop_now(struct uart_amba_port *uap) | |
| 1337 | 1338 | { |
| 1338 | 1339 | struct uart_port *port = &uap->port; |
| 1339 | 1340 | u32 cr; |
| 1340 | 1341 | |
| 1342 | + cr = pl011_read(uap, REG_CR); | |
| 1343 | + | |
| 1344 | + if (port->rs485.flags & SER_RS485_RTS_AFTER_SEND) | |
| 1345 | + cr &= ~UART011_CR_RTS; | |
| 1346 | + else | |
| 1347 | + cr |= UART011_CR_RTS; | |
| 1348 | + | |
| 1349 | + /* Disable the transmitter and reenable the transceiver */ | |
| 1350 | + cr &= ~UART011_CR_TXE; | |
| 1351 | + cr |= UART011_CR_RXE; | |
| 1352 | + pl011_write(cr, uap, REG_CR); | |
| 1353 | + | |
| 1354 | + uap->rs485_tx_state = OFF; | |
| 1355 | +} | |
| 1356 | + | |
| 1357 | +static void pl011_rs485_tx_stop(struct uart_amba_port *uap) | |
| 1358 | +{ | |
| 1359 | + struct uart_port *port = &uap->port; | |
| 1360 | + | |
| 1341 | 1361 | if (uap->rs485_tx_state == SEND) |
| 1342 | 1362 | uap->rs485_tx_state = WAIT_AFTER_SEND; |
| 1343 | 1363 | |
| @@ -1350,6 +1370,7 @@ static void pl011_rs485_tx_stop(struct uart_amba_port *uap) · 6 unmodified lines | ||
| 1350 | 1370 | return; |
| 1351 | 1371 | } |
| 1352 | 1372 | if (port->rs485.delay_rts_after_send > 0) { |
| 1373 | + uap->rs485_tx_state = WAIT_AFTER_SEND_DELAY; | |
| 1353 | 1374 | hrtimer_start(&uap->trigger_stop_tx, |
| 1354 | 1375 | ms_to_ktime(port->rs485.delay_rts_after_send), |
| 1355 | 1376 | HRTIMER_MODE_REL); |
| @@ -1360,19 +1381,7 @@ static void pl011_rs485_tx_stop(struct uart_amba_port *uap) · 4 unmodified lines | ||
| 1360 | 1381 | hrtimer_try_to_cancel(&uap->trigger_start_tx); |
| 1361 | 1382 | } |
| 1362 | 1383 | |
| 1363 | − cr = pl011_read(uap, REG_CR); | |
| 1364 | − | |
| 1365 | − if (port->rs485.flags & SER_RS485_RTS_AFTER_SEND) | |
| 1366 | − cr &= ~UART011_CR_RTS; | |
| 1367 | − else | |
| 1368 | − cr |= UART011_CR_RTS; | |
| 1369 | − | |
| 1370 | − /* Disable the transmitter and reenable the transceiver */ | |
| 1371 | − cr &= ~UART011_CR_TXE; | |
| 1372 | − cr |= UART011_CR_RXE; | |
| 1373 | − pl011_write(cr, uap, REG_CR); | |
| 1374 | − | |
| 1375 | − uap->rs485_tx_state = OFF; | |
| 1384 | + pl011_rs485_tx_stop_now(uap); | |
| 1376 | 1385 | } |
| 1377 | 1386 | |
| 1378 | 1387 | static void pl011_stop_tx(struct uart_port *port) |
| @@ -1415,7 +1424,8 @@ static void pl011_rs485_tx_start(struct uart_amba_port *uap) · 36 unmodified lines | ||
| 1415 | 1424 | uap->rs485_tx_state = SEND; |
| 1416 | 1425 | return; |
| 1417 | 1426 | } |
| 1418 | − if (uap->rs485_tx_state == WAIT_AFTER_SEND) { | |
| 1427 | + if (uap->rs485_tx_state == WAIT_AFTER_SEND || | |
| 1428 | + uap->rs485_tx_state == WAIT_AFTER_SEND_DELAY) { | |
| 1419 | 1429 | hrtimer_try_to_cancel(&uap->trigger_stop_tx); |
| 1420 | 1430 | uap->rs485_tx_state = SEND; |
| 1421 | 1431 | return; |
| @@ -1482,7 +1492,8 @@ static enum hrtimer_restart pl011_trigger_stop_tx(struct hrtimer *t) · 60 unmodified lines | ||
| 1482 | 1492 | unsigned long flags; |
| 1483 | 1493 | |
| 1484 | 1494 | uart_port_lock_irqsave(&uap->port, &flags); |
| 1485 | − if (uap->rs485_tx_state == WAIT_AFTER_SEND) | |
| 1495 | + if (uap->rs485_tx_state == WAIT_AFTER_SEND || | |
| 1496 | + uap->rs485_tx_state == WAIT_AFTER_SEND_DELAY) | |
| 1486 | 1497 | pl011_rs485_tx_stop(uap); |
| 1487 | 1498 | uart_port_unlock_irqrestore(&uap->port, flags); |
| 1488 | 1499 | |
| @@ -2080,11 +2091,20 @@ static void pl011_shutdown(struct uart_port *port) · 591 unmodified lines | ||
| 2080 | 2091 | |
| 2081 | 2092 | pl011_dma_shutdown(uap); |
| 2082 | 2093 | |
| 2083 | − if ((port->rs485.flags & SER_RS485_ENABLED && uap->rs485_tx_state != OFF)) | |
| 2084 | − pl011_rs485_tx_stop(uap); | |
| 2085 | − | |
| 2086 | 2094 | free_irq(uap->port.irq, uap); |
| 2087 | 2095 | |
| 2096 | + /* | |
| 2097 | + * free_irq() drains the UART interrupt handler, which can arm either | |
| 2098 | + * timer. Cancel the timers afterwards to drain their callbacks too. | |
| 2099 | + */ | |
| 2100 | + hrtimer_cancel(&uap->trigger_start_tx); | |
| 2101 | + hrtimer_cancel(&uap->trigger_stop_tx); | |
| 2102 | + | |
| 2103 | + uart_port_lock_irq(port); | |
| 2104 | + if (uap->rs485_tx_state != OFF) | |
| 2105 | + pl011_rs485_tx_stop_now(uap); | |
| 2106 | + uart_port_unlock_irq(port); | |
| 2107 | + | |
| 2088 | 2108 | pl011_disable_uart(uap); |
| 2089 | 2109 | |
| 2090 | 2110 | /* |
| @@ -3063,6 +3083,8 @@ static void pl011_remove(struct amba_device *dev) · 972 unmodified lines | ||
| 3063 | 3083 | struct uart_amba_port *uap = amba_get_drvdata(dev); |
| 3064 | 3084 | |
| 3065 | 3085 | uart_remove_one_port(&amba_reg, &uap->port); |
| 3086 | + hrtimer_cancel(&uap->trigger_start_tx); | |
| 3087 | + hrtimer_cancel(&uap->trigger_stop_tx); | |
| 3066 | 3088 | pl011_unregister_port(uap); |
| 3067 | 3089 | } |
| 3068 | 3090 | |
| @@ -158,6 +158,7 @@ static const struct uart_ops qcom_geni_uart_pops; · 157 unmodified lines | ||
| 158 | 158 | static struct uart_driver qcom_geni_console_driver; |
| 159 | 159 | static struct uart_driver qcom_geni_uart_driver; |
| 160 | 160 | |
| 161 | +static void qcom_geni_serial_stop_tx_dma(struct uart_port *uport); | |
| 161 | 162 | static void __qcom_geni_serial_cancel_tx_cmd(struct uart_port *uport); |
| 162 | 163 | static void qcom_geni_serial_cancel_tx_cmd(struct uart_port *uport); |
| 163 | 164 | static int qcom_geni_serial_port_setup(struct uart_port *uport); |
| @@ -636,35 +637,34 @@ static unsigned int qcom_geni_serial_tx_empty(struct uart_port *uport) · 472 unmodified lines | ||
| 636 | 637 | return !readl(uport->membase + SE_GENI_TX_FIFO_STATUS); |
| 637 | 638 | } |
| 638 | 639 | |
| 640 | +static void qcom_geni_serial_flush_buffer_dma(struct uart_port *uport) | |
| 641 | +{ | |
| 642 | + struct qcom_geni_serial_port *port = to_dev_port(uport); | |
| 643 | + | |
| 644 | + qcom_geni_serial_stop_tx_dma(uport); | |
| 645 | + port->tx_remaining = 0; | |
| 646 | + port->tx_queued = 0; | |
| 647 | +} | |
| 648 | + | |
| 639 | 649 | static void qcom_geni_serial_stop_tx_dma(struct uart_port *uport) |
| 640 | 650 | { |
| 641 | 651 | struct qcom_geni_serial_port *port = to_dev_port(uport); |
| 642 | − bool done; | |
| 643 | 652 | |
| 644 | − if (!qcom_geni_serial_main_active(uport)) | |
| 645 | − return; | |
| 653 | + if (qcom_geni_serial_main_active(uport)) | |
| 654 | + __qcom_geni_serial_cancel_tx_cmd(uport); | |
| 646 | 655 | |
| 647 | 656 | if (port->tx_dma_addr) { |
| 657 | + writel(1, uport->membase + SE_DMA_TX_FSM_RST); | |
| 658 | + if (!qcom_geni_serial_poll_bit(uport, SE_DMA_TX_IRQ_STAT, | |
| 659 | + TX_RESET_DONE, true)) | |
| 660 | + dev_err_ratelimited(uport->dev, "TX DMA reset failed"); | |
| 661 | + writel(TX_RESET_DONE | TX_DMA_DONE, | |
| 662 | + uport->membase + SE_DMA_TX_IRQ_CLR); | |
| 663 | + | |
| 648 | 664 | geni_se_tx_dma_unprep(&port->se, port->tx_dma_addr, |
| 649 | 665 | port->tx_remaining); |
| 650 | 666 | port->tx_dma_addr = 0; |
| 651 | − port->tx_remaining = 0; | |
| 652 | 667 | } |
| 653 | − | |
| 654 | − geni_se_cancel_m_cmd(&port->se); | |
| 655 | − | |
| 656 | − done = qcom_geni_serial_poll_bit(uport, SE_GENI_M_IRQ_STATUS, | |
| 657 | − M_CMD_CANCEL_EN, true); | |
| 658 | − if (!done) { | |
| 659 | − geni_se_abort_m_cmd(&port->se); | |
| 660 | − done = qcom_geni_serial_poll_bit(uport, SE_GENI_M_IRQ_STATUS, | |
| 661 | − M_CMD_ABORT_EN, true); | |
| 662 | − if (!done) | |
| 663 | − dev_err_ratelimited(uport->dev, "M_CMD_ABORT_EN not set"); | |
| 664 | − writel(M_CMD_ABORT_EN, uport->membase + SE_GENI_M_IRQ_CLEAR); | |
| 665 | − } | |
| 666 | − | |
| 667 | − writel(M_CMD_CANCEL_EN, uport->membase + SE_GENI_M_IRQ_CLEAR); | |
| 668 | 668 | } |
| 669 | 669 | |
| 670 | 670 | static void qcom_geni_serial_start_tx_dma(struct uart_port *uport) |
| @@ -1180,7 +1180,7 @@ static void qcom_geni_serial_shutdown(struct uart_port *uport) · 509 unmodified lines | ||
| 1180 | 1180 | uart_port_unlock_irq(uport); |
| 1181 | 1181 | } |
| 1182 | 1182 | |
| 1183 | −static void qcom_geni_serial_flush_buffer(struct uart_port *uport) | |
| 1183 | +static void qcom_geni_serial_flush_buffer_fifo(struct uart_port *uport) | |
| 1184 | 1184 | { |
| 1185 | 1185 | qcom_geni_serial_cancel_tx_cmd(uport); |
| 1186 | 1186 | } |
| @@ -1769,7 +1769,7 @@ static const struct uart_ops qcom_geni_console_pops = { · 582 unmodified lines | ||
| 1769 | 1769 | .request_port = qcom_geni_serial_request_port, |
| 1770 | 1770 | .config_port = qcom_geni_serial_config_port, |
| 1771 | 1771 | .shutdown = qcom_geni_serial_shutdown, |
| 1772 | − .flush_buffer = qcom_geni_serial_flush_buffer, | |
| 1772 | + .flush_buffer = qcom_geni_serial_flush_buffer_fifo, | |
| 1773 | 1773 | .type = qcom_geni_serial_get_type, |
| 1774 | 1774 | .set_mctrl = qcom_geni_serial_set_mctrl, |
| 1775 | 1775 | .get_mctrl = qcom_geni_serial_get_mctrl, |
| @@ -1792,6 +1792,7 @@ static const struct uart_ops qcom_geni_uart_pops = { · 16 unmodified lines | ||
| 1792 | 1792 | .request_port = qcom_geni_serial_request_port, |
| 1793 | 1793 | .config_port = qcom_geni_serial_config_port, |
| 1794 | 1794 | .shutdown = qcom_geni_serial_shutdown, |
| 1795 | + .flush_buffer = qcom_geni_serial_flush_buffer_dma, | |
| 1795 | 1796 | .type = qcom_geni_serial_get_type, |
| 1796 | 1797 | .set_mctrl = qcom_geni_serial_set_mctrl, |
| 1797 | 1798 | .get_mctrl = qcom_geni_serial_get_mctrl, |
| @@ -827,6 +827,9 @@ static void sc16is7xx_tx_proc(struct kthread_work *ws) · 826 unmodified lines | ||
| 827 | 827 | msleep(port->rs485.delay_rts_before_send); |
| 828 | 828 | |
| 829 | 829 | guard(mutex)(&one->lock); |
| 830 | + sc16is7xx_port_update(port, SC16IS7XX_IER_REG, | |
| 831 | + SC16IS7XX_IER_THRI_BIT, | |
| 832 | + SC16IS7XX_IER_THRI_BIT); | |
| 830 | 833 | sc16is7xx_handle_tx(port); |
| 831 | 834 | } |
| 832 | 835 | |
| @@ -1437,7 +1437,7 @@ static void kbd_keycode(unsigned int keycode, int down, bool hw_raw) · 1436 unmodified lines | ||
| 1437 | 1437 | struct keyboard_notifier_param param = { .vc = vc, .value = keycode, .down = down }; |
| 1438 | 1438 | int rc; |
| 1439 | 1439 | |
| 1440 | − tty = vc->port.tty; | |
| 1440 | + tty = tty_port_tty_get(&vc->port); | |
| 1441 | 1441 | |
| 1442 | 1442 | if (tty && (!tty->driver_data)) { |
| 1443 | 1443 | /* No driver data? Strange. Okay we fix it then. */ |
| @@ -1497,9 +1497,12 @@ static void kbd_keycode(unsigned int keycode, int down, bool hw_raw) · 53 unmodified lines | ||
| 1497 | 1497 | * characters get aren't echoed locally. This makes key repeat |
| 1498 | 1498 | * usable with slow applications and under heavy loads. |
| 1499 | 1499 | */ |
| 1500 | + tty_kref_put(tty); | |
| 1500 | 1501 | return; |
| 1501 | 1502 | } |
| 1502 | 1503 | |
| 1504 | + tty_kref_put(tty); | |
| 1505 | + | |
| 1503 | 1506 | param.shift = shift_final = (shift_state | kbd->slockstate) ^ kbd->lockstate; |
| 1504 | 1507 | param.ledstate = kbd->ledflagstate; |
| 1505 | 1508 | key_map = key_maps[shift_final]; |
| @@ -406,6 +406,8 @@ static int vt_k_ioctl(struct tty_struct *tty, unsigned int cmd, · 405 unmodified lines | ||
| 406 | 406 | /* this could be folded into KDSKBMODE, but for compatibility |
| 407 | 407 | reasons it is not so easy to fold KDGKBMETA into KDGKBMODE */ |
| 408 | 408 | case KDSKBMETA: |
| 409 | + if (!perm) | |
| 410 | + return -EPERM; | |
| 409 | 411 | return vt_do_kdskbmeta(console, arg); |
| 410 | 412 | |
| 411 | 413 | case KDGKBMETA: |
| @@ -152,8 +152,6 @@ int ufs_rpmb_probe(struct ufs_hba *hba) · 151 unmodified lines | ||
| 152 | 152 | return -EINVAL; |
| 153 | 153 | } |
| 154 | 154 | |
| 155 | − INIT_LIST_HEAD(&hba->rpmbs); | |
| 156 | − | |
| 157 | 155 | struct rpmb_descr descr = { |
| 158 | 156 | .type = RPMB_TYPE_UFS, |
| 159 | 157 | .route_frames = ufs_rpmb_route_frames, |
| @@ -7357,7 +7357,7 @@ static irqreturn_t ufshcd_sl_intr(struct ufs_hba *hba, u32 intr_status) · 7356 unmodified lines | ||
| 7357 | 7357 | } |
| 7358 | 7358 | |
| 7359 | 7359 | /** |
| 7360 | − * ufshcd_threaded_intr - Threaded interrupt service routine | |
| 7360 | + * ufshcd_intr - Main interrupt service routine | |
| 7361 | 7361 | * @irq: irq number |
| 7362 | 7362 | * @__hba: pointer to adapter instance |
| 7363 | 7363 | * |
| @@ -7365,7 +7365,7 @@ static irqreturn_t ufshcd_sl_intr(struct ufs_hba *hba, u32 intr_status) · 1 unmodified line | ||
| 7365 | 7365 | * IRQ_HANDLED - If interrupt is valid |
| 7366 | 7366 | * IRQ_NONE - If invalid interrupt |
| 7367 | 7367 | */ |
| 7368 | −static irqreturn_t ufshcd_threaded_intr(int irq, void *__hba) | |
| 7368 | +static irqreturn_t ufshcd_intr(int irq, void *__hba) | |
| 7369 | 7369 | { |
| 7370 | 7370 | u32 last_intr_status, intr_status, enabled_intr_status = 0; |
| 7371 | 7371 | irqreturn_t retval = IRQ_NONE; |
| @@ -7404,38 +7404,6 @@ static irqreturn_t ufshcd_threaded_intr(int irq, void *__hba) · 32 unmodified lines | ||
| 7404 | 7404 | return retval; |
| 7405 | 7405 | } |
| 7406 | 7406 | |
| 7407 | −/** | |
| 7408 | − * ufshcd_intr - Main interrupt service routine | |
| 7409 | − * @irq: irq number | |
| 7410 | − * @__hba: pointer to adapter instance | |
| 7411 | − * | |
| 7412 | − * Return: | |
| 7413 | − * IRQ_HANDLED - If interrupt is valid | |
| 7414 | − * IRQ_WAKE_THREAD - If handling is moved to threaded handled | |
| 7415 | − * IRQ_NONE - If invalid interrupt | |
| 7416 | − */ | |
| 7417 | −static irqreturn_t ufshcd_intr(int irq, void *__hba) | |
| 7418 | −{ | |
| 7419 | − struct ufs_hba *hba = __hba; | |
| 7420 | − u32 intr_status, enabled_intr_status; | |
| 7421 | − | |
| 7422 | − /* | |
| 7423 | − * Handle interrupt in thread if MCQ or ESI is disabled, | |
| 7424 | − * and no active UIC command. | |
| 7425 | − */ | |
| 7426 | − if ((!hba->mcq_enabled || !hba->mcq_esi_enabled) && | |
| 7427 | − !hba->active_uic_cmd) | |
| 7428 | − return IRQ_WAKE_THREAD; | |
| 7429 | − | |
| 7430 | − intr_status = ufshcd_readl(hba, REG_INTERRUPT_STATUS); | |
| 7431 | − enabled_intr_status = intr_status & ufshcd_readl(hba, REG_INTERRUPT_ENABLE); | |
| 7432 | − | |
| 7433 | − ufshcd_writel(hba, intr_status, REG_INTERRUPT_STATUS); | |
| 7434 | − | |
| 7435 | − /* Directly handle interrupts since MCQ ESI handlers does the hard job */ | |
| 7436 | − return ufshcd_sl_intr(hba, enabled_intr_status); | |
| 7437 | −} | |
| 7438 | − | |
| 7439 | 7407 | static int ufshcd_clear_tm_cmd(struct ufs_hba *hba, int tag) |
| 7440 | 7408 | { |
| 7441 | 7409 | int err = 0; |
| @@ -10269,6 +10237,7 @@ static int __ufshcd_wl_suspend(struct ufs_hba *hba, enum ufs_pm_op pm_op) · 2827 unmodified lines | ||
| 10269 | 10237 | req_link_state == UIC_LINK_ACTIVE_STATE) { |
| 10270 | 10238 | ufshcd_disable_auto_bkops(hba); |
| 10271 | 10239 | flush_work(&hba->eeh_work); |
| 10240 | + cancel_delayed_work_sync(&hba->ufs_rtc_update_work); | |
| 10272 | 10241 | goto vops_suspend; |
| 10273 | 10242 | } |
| 10274 | 10243 | |
| @@ -10478,10 +10447,11 @@ static int __ufshcd_wl_resume(struct ufs_hba *hba, enum ufs_pm_op pm_op) · 203 unmodified lines | ||
| 10478 | 10447 | if (ret) |
| 10479 | 10448 | goto set_old_link_state; |
| 10480 | 10449 | ufshcd_set_timestamp_attr(hba); |
| 10481 | − schedule_delayed_work(&hba->ufs_rtc_update_work, | |
| 10482 | − msecs_to_jiffies(UFS_RTC_UPDATE_INTERVAL_MS)); | |
| 10483 | 10450 | } |
| 10484 | 10451 | |
| 10452 | + schedule_delayed_work(&hba->ufs_rtc_update_work, | |
| 10453 | + msecs_to_jiffies(UFS_RTC_UPDATE_INTERVAL_MS)); | |
| 10454 | + | |
| 10485 | 10455 | if (ufshcd_keep_autobkops_enabled_except_suspend(hba)) |
| 10486 | 10456 | ufshcd_enable_auto_bkops(hba); |
| 10487 | 10457 | else |
| @@ -10988,6 +10958,7 @@ int ufshcd_alloc_host(struct device *dev, struct ufs_hba **hba_handle) · 500 unmodified lines | ||
| 10988 | 10958 | hba->nop_out_timeout = NOP_OUT_TIMEOUT; |
| 10989 | 10959 | ufshcd_set_sg_entry_size(hba, sizeof(struct ufshcd_sg_entry)); |
| 10990 | 10960 | INIT_LIST_HEAD(&hba->clk_list_head); |
| 10961 | + INIT_LIST_HEAD(&hba->rpmbs); | |
| 10991 | 10962 | spin_lock_init(&hba->outstanding_lock); |
| 10992 | 10963 | |
| 10993 | 10964 | *hba_handle = hba; |
| @@ -11236,8 +11207,7 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq) · 242 unmodified lines | ||
| 11236 | 11207 | ufshcd_readl(hba, REG_INTERRUPT_ENABLE); |
| 11237 | 11208 | |
| 11238 | 11209 | /* IRQ registration */ |
| 11239 | − err = devm_request_threaded_irq(dev, irq, ufshcd_intr, ufshcd_threaded_intr, | |
| 11240 | − IRQF_ONESHOT | IRQF_SHARED, UFSHCD, hba); | |
| 11210 | + err = devm_request_irq(dev, irq, ufshcd_intr, IRQF_SHARED, UFSHCD, hba); | |
| 11241 | 11211 | if (err) { |
| 11242 | 11212 | dev_err(hba->dev, "request irq failed\n"); |
| 11243 | 11213 | goto out_disable; |
| @@ -700,6 +700,8 @@ static int cxacru_cm(struct cxacru_data *instance, enum cxacru_cm_request cm, · 699 unmodified lines | ||
| 700 | 700 | ret = offd; |
| 701 | 701 | usb_dbg(instance->usbatm, "cm %#x\n", cm); |
| 702 | 702 | fail: |
| 703 | + if (ret < 0) | |
| 704 | + usb_kill_urb(instance->rcv_urb); | |
| 703 | 705 | mutex_unlock(&instance->cm_serialize); |
| 704 | 706 | err: |
| 705 | 707 | return ret; |
| @@ -163,9 +163,9 @@ static void cdnsp_set_apb_timeout_value(struct cdnsp_device *pdev) · 162 unmodified lines | ||
| 163 | 163 | offset = cdnsp_find_next_ext_cap(base, offset, D_XEC_PRE_REGS_CAP); |
| 164 | 164 | reg = base + offset + REG_CHICKEN_BITS_3_OFFSET; |
| 165 | 165 | |
| 166 | − val = le32_to_cpu(readl(reg)); | |
| 166 | + val = readl(reg); | |
| 167 | 167 | val = CHICKEN_APB_TIMEOUT_SET(val, cdns->override_apb_timeout); |
| 168 | − writel(cpu_to_le32(val), reg); | |
| 168 | + writel(val, reg); | |
| 169 | 169 | } |
| 170 | 170 | |
| 171 | 171 | static void cdnsp_set_chicken_bits_2(struct cdnsp_device *pdev, u32 bit) |
| @@ -912,6 +912,18 @@ int usb_get_configuration(struct usb_device *dev) · 911 unmodified lines | ||
| 912 | 912 | unsigned char *bigbuffer; |
| 913 | 913 | struct usb_config_descriptor *desc; |
| 914 | 914 | int result; |
| 915 | + size_t usb_config_req_size; | |
| 916 | + | |
| 917 | + /* | |
| 918 | + * We usually start by grabbing the first 9-bytes descriptor so we know | |
| 919 | + * how long the whole configuration is. Some devices with quirky | |
| 920 | + * firmware will fail enumeration, so if the quirk is set, use 255 instead, | |
| 921 | + * mirroring the behavior of Windows. | |
| 922 | + */ | |
| 923 | + if (dev->quirks & USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE) | |
| 924 | + usb_config_req_size = 255; | |
| 925 | + else | |
| 926 | + usb_config_req_size = USB_DT_CONFIG_SIZE; | |
| 915 | 927 | |
| 916 | 928 | if (ncfg > USB_MAXCONFIG) { |
| 917 | 929 | dev_notice(ddev, "too many configurations: %d, " |
| @@ -938,15 +950,13 @@ int usb_get_configuration(struct usb_device *dev) · 20 unmodified lines | ||
| 938 | 950 | if (!dev->rawdescriptors) |
| 939 | 951 | return -ENOMEM; |
| 940 | 952 | |
| 941 | − desc = kmalloc(USB_DT_CONFIG_SIZE, GFP_KERNEL); | |
| 953 | + desc = kmalloc(usb_config_req_size, GFP_KERNEL); | |
| 942 | 954 | if (!desc) |
| 943 | 955 | return -ENOMEM; |
| 944 | 956 | |
| 945 | 957 | for (cfgno = 0; cfgno < ncfg; cfgno++) { |
| 946 | − /* We grab just the first descriptor so we know how long | |
| 947 | − * the whole configuration is */ | |
| 948 | 958 | result = usb_get_descriptor(dev, USB_DT_CONFIG, cfgno, |
| 949 | − desc, USB_DT_CONFIG_SIZE); | |
| 959 | + desc, usb_config_req_size); | |
| 950 | 960 | if (result < 0) { |
| 951 | 961 | dev_err(ddev, "unable to read config index %d " |
| 952 | 962 | "descriptor/%s: %d\n", cfgno, "start", result); |
| @@ -956,16 +966,14 @@ int usb_get_configuration(struct usb_device *dev) · 3 unmodified lines | ||
| 956 | 966 | dev->descriptor.bNumConfigurations = cfgno; |
| 957 | 967 | break; |
| 958 | 968 | } else if (result < 4) { |
| 959 | − dev_err(ddev, "config index %d descriptor too short " | |
| 960 | − "(expected %i, got %i)\n", cfgno, | |
| 961 | − USB_DT_CONFIG_SIZE, result); | |
| 969 | + dev_err(ddev, "config index %d descriptor too short (asked for %zu, got %i)\n", | |
| 970 | + cfgno, usb_config_req_size, result); | |
| 962 | 971 | result = -EINVAL; |
| 963 | 972 | goto err; |
| 964 | 973 | } |
| 965 | 974 | length = max_t(int, le16_to_cpu(desc->wTotalLength), |
| 966 | 975 | USB_DT_CONFIG_SIZE); |
| 967 | 976 | |
| 968 | − /* Now that we know the length, get the whole thing */ | |
| 969 | 977 | bigbuffer = kmalloc(length, GFP_KERNEL); |
| 970 | 978 | if (!bigbuffer) { |
| 971 | 979 | result = -ENOMEM; |
| @@ -975,6 +983,13 @@ int usb_get_configuration(struct usb_device *dev) · 3 unmodified lines | ||
| 975 | 983 | if (dev->quirks & USB_QUIRK_DELAY_INIT) |
| 976 | 984 | msleep(200); |
| 977 | 985 | |
| 986 | + /* Skip the second read if we already got everything */ | |
| 987 | + if (result >= length) { | |
| 988 | + memcpy(bigbuffer, desc, length); | |
| 989 | + goto store_and_parse; | |
| 990 | + } | |
| 991 | + | |
| 992 | + /* Get the whole thing */ | |
| 978 | 993 | result = usb_get_descriptor(dev, USB_DT_CONFIG, cfgno, |
| 979 | 994 | bigbuffer, length); |
| 980 | 995 | if (result < 0) { |
| @@ -989,6 +1004,7 @@ int usb_get_configuration(struct usb_device *dev) · 8 unmodified lines | ||
| 989 | 1004 | length = result; |
| 990 | 1005 | } |
| 991 | 1006 | |
| 1007 | +store_and_parse: | |
| 992 | 1008 | dev->rawdescriptors[cfgno] = bigbuffer; |
| 993 | 1009 | |
| 994 | 1010 | result = usb_parse_configuration(dev, cfgno, |
| @@ -2406,7 +2406,7 @@ static void show_string(struct usb_device *udev, char *id, char *string) · 2405 unmodified lines | ||
| 2406 | 2406 | dev_info(&udev->dev, "%s: %s\n", id, string); |
| 2407 | 2407 | } |
| 2408 | 2408 | |
| 2409 | −static void announce_device(struct usb_device *udev) | |
| 2409 | +static void announce_device_ids(struct usb_device *udev) | |
| 2410 | 2410 | { |
| 2411 | 2411 | u16 bcdDevice = le16_to_cpu(udev->descriptor.bcdDevice); |
| 2412 | 2412 | |
| @@ -2415,6 +2415,10 @@ static void announce_device(struct usb_device *udev) · 2 unmodified lines | ||
| 2415 | 2415 | le16_to_cpu(udev->descriptor.idVendor), |
| 2416 | 2416 | le16_to_cpu(udev->descriptor.idProduct), |
| 2417 | 2417 | bcdDevice >> 8, bcdDevice & 0xff); |
| 2418 | +} | |
| 2419 | + | |
| 2420 | +static void announce_device_strings(struct usb_device *udev) | |
| 2421 | +{ | |
| 2418 | 2422 | dev_info(&udev->dev, |
| 2419 | 2423 | "New USB device strings: Mfr=%d, Product=%d, SerialNumber=%d\n", |
| 2420 | 2424 | udev->descriptor.iManufacturer, |
| @@ -2425,7 +2429,8 @@ static void announce_device(struct usb_device *udev) · 4 unmodified lines | ||
| 2425 | 2429 | show_string(udev, "SerialNumber", udev->serial); |
| 2426 | 2430 | } |
| 2427 | 2431 | #else |
| 2428 | −static inline void announce_device(struct usb_device *udev) { } | |
| 2432 | +static inline void announce_device_ids(struct usb_device *udev) { } | |
| 2433 | +static inline void announce_device_strings(struct usb_device *udev) { } | |
| 2429 | 2434 | #endif |
| 2430 | 2435 | |
| 2431 | 2436 | |
| @@ -2656,6 +2661,9 @@ int usb_new_device(struct usb_device *udev) · 224 unmodified lines | ||
| 2656 | 2661 | device_init_wakeup(&udev->dev, 0); |
| 2657 | 2662 | } |
| 2658 | 2663 | |
| 2664 | + /* Announce the device identity */ | |
| 2665 | + announce_device_ids(udev); | |
| 2666 | + | |
| 2659 | 2667 | /* Tell the runtime-PM framework the device is active */ |
| 2660 | 2668 | pm_runtime_set_active(&udev->dev); |
| 2661 | 2669 | pm_runtime_get_noresume(&udev->dev); |
| @@ -2677,8 +2685,8 @@ int usb_new_device(struct usb_device *udev) · 15 unmodified lines | ||
| 2677 | 2685 | udev->dev.devt = MKDEV(USB_DEVICE_MAJOR, |
| 2678 | 2686 | (((udev->bus->busnum-1) * 128) + (udev->devnum-1))); |
| 2679 | 2687 | |
| 2680 | − /* Tell the world! */ | |
| 2681 | − announce_device(udev); | |
| 2688 | + /* Announce the device's product, manufacturer and serial number */ | |
| 2689 | + announce_device_strings(udev); | |
| 2682 | 2690 | |
| 2683 | 2691 | if (udev->serial) |
| 2684 | 2692 | add_device_randomness(udev->serial, strlen(udev->serial)); |
| @@ -142,6 +142,10 @@ static int quirks_param_set(const char *value, const struct kernel_param *kp) · 141 unmodified lines | ||
| 142 | 142 | break; |
| 143 | 143 | case 'q': |
| 144 | 144 | flags |= USB_QUIRK_FORCE_ONE_CONFIG; |
| 145 | + break; | |
| 146 | + case 'r': | |
| 147 | + flags |= USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE; | |
| 148 | + break; | |
| 145 | 149 | /* Ignore unrecognized flag characters */ |
| 146 | 150 | } |
| 147 | 151 | } |
| @@ -589,6 +593,9 @@ static const struct usb_device_id usb_quirk_list[] = { · 441 unmodified lines | ||
| 589 | 593 | |
| 590 | 594 | { USB_DEVICE(0x2386, 0x350e), .driver_info = USB_QUIRK_NO_LPM }, |
| 591 | 595 | |
| 596 | + /* ShanWan Wireless Gamepad */ | |
| 597 | + { USB_DEVICE(0x2563, 0x0575), .driver_info = USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE }, | |
| 598 | + | |
| 592 | 599 | /* UGREEN 35871 - BOS descriptor fetch hangs at SuperSpeed Plus */ |
| 593 | 600 | { USB_DEVICE(0x2b89, 0x5871), .driver_info = USB_QUIRK_NO_BOS }, |
| 594 | 601 | |
| @@ -1171,7 +1171,7 @@ static int ncm_unwrap_ntb(struct gether *port, · 1170 unmodified lines | ||
| 1171 | 1171 | unsigned char *ntb_ptr = skb->data; |
| 1172 | 1172 | __le16 *tmp; |
| 1173 | 1173 | unsigned index, index2; |
| 1174 | − int ndp_index; | |
| 1174 | + unsigned int ndp_index; | |
| 1175 | 1175 | unsigned dg_len, dg_len2; |
| 1176 | 1176 | unsigned ndp_len; |
| 1177 | 1177 | unsigned block_len; |
| @@ -190,13 +190,13 @@ struct xhci_op_regs { · 189 unmodified lines | ||
| 190 | 190 | |
| 191 | 191 | /* CRCR - Command Ring Control Register - cmd_ring bitmasks */ |
| 192 | 192 | /* bit 0 - Cycle bit indicates the ownership of the command ring */ |
| 193 | −#define CMD_RING_CYCLE BIT(0) | |
| 193 | +#define CMD_RING_CYCLE BIT_ULL(0) | |
| 194 | 194 | /* stop ring operation after completion of the currently executing command */ |
| 195 | −#define CMD_RING_PAUSE BIT(1) | |
| 195 | +#define CMD_RING_PAUSE BIT_ULL(1) | |
| 196 | 196 | /* stop ring immediately - abort the currently executing command */ |
| 197 | −#define CMD_RING_ABORT BIT(2) | |
| 197 | +#define CMD_RING_ABORT BIT_ULL(2) | |
| 198 | 198 | /* true: command ring is running */ |
| 199 | −#define CMD_RING_RUNNING BIT(3) | |
| 199 | +#define CMD_RING_RUNNING BIT_ULL(3) | |
| 200 | 200 | /* bits 63:6 - Command Ring pointer */ |
| 201 | 201 | #define CMD_RING_PTR_MASK GENMASK_ULL(63, 6) |
19 more lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 126 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 266 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 253 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 119 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 138 lines aren’t shown here.
This diff is large: its 357 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 130 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 111 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 129 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 116 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 130 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 120 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 304 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 497 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 657 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 199 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 82 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 706 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 145 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 570 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 311 lines aren’t shown here.
This diff is large: its 573 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 102 lines aren’t shown here.
This diff is large: its 116 lines aren’t shown here.
This diff is large: its 146 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 141 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 224 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 417 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 194 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 148 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 259 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 123 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 188 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.