Linux 7.0-rc4 Required for the ds4422 series which is build upon; 5187e03b817c ("iio: dac: ds4424: reject -128 RAW value")
Unified · Split
| @@ -210,10 +210,16 @@ Daniel Borkmann <daniel@iogearbox.net> <daniel.borkmann@tik.ee.ethz.ch> · 209 unmodified lines | ||
| 210 | 210 | Daniel Borkmann <daniel@iogearbox.net> <dborkmann@redhat.com> |
| 211 | 211 | Daniel Borkmann <daniel@iogearbox.net> <dborkman@redhat.com> |
| 212 | 212 | Daniel Borkmann <daniel@iogearbox.net> <dxchgb@gmail.com> |
| 213 | +Daniel Lezcano <daniel.lezcano@kernel.org> <daniel.lezcano@linaro.org> | |
| 214 | +Daniel Lezcano <daniel.lezcano@kernel.org> <daniel.lezcano@free.fr> | |
| 215 | +Daniel Lezcano <daniel.lezcano@kernel.org> <daniel.lezcano@linexp.org> | |
| 216 | +Daniel Lezcano <daniel.lezcano@kernel.org> <dlezcano@fr.ibm.com> | |
| 213 | 217 | Daniel Thompson <danielt@kernel.org> <daniel.thompson@linaro.org> |
| 218 | +Daniele Alessandrelli <daniele.alessandrelli@gmail.com> <daniele.alessandrelli@intel.com> | |
| 214 | 219 | Danilo Krummrich <dakr@kernel.org> <dakr@redhat.com> |
| 215 | 220 | David Brownell <david-b@pacbell.net> |
| 216 | 221 | David Collins <quic_collinsd@quicinc.com> <collinsd@codeaurora.org> |
| 222 | +David Gow <david@davidgow.net> <davidgow@google.com> | |
| 217 | 223 | David Heidelberg <david@ixit.cz> <d.okias@gmail.com> |
| 218 | 224 | David Hildenbrand <david@kernel.org> <david@redhat.com> |
| 219 | 225 | David Rheinsberg <david@readahead.eu> <dh.herrmann@gmail.com> |
| @@ -348,6 +354,7 @@ Jarkko Sakkinen <jarkko@kernel.org> <jarkko.sakkinen@opinsys.com> · 128 unmodified lines | ||
| 348 | 354 | Jason Gunthorpe <jgg@ziepe.ca> <jgg@mellanox.com> |
| 349 | 355 | Jason Gunthorpe <jgg@ziepe.ca> <jgg@nvidia.com> |
| 350 | 356 | Jason Gunthorpe <jgg@ziepe.ca> <jgunthorpe@obsidianresearch.com> |
| 357 | +Jason Xing <kerneljasonxing@gmail.com> <kernelxing@tencent.com> | |
| 351 | 358 | <javier@osg.samsung.com> <javier.martinez@collabora.co.uk> |
| 352 | 359 | Javi Merino <javi.merino@kernel.org> <javi.merino@arm.com> |
| 353 | 360 | Jayachandran C <c.jayachandran@gmail.com> <jayachandranc@netlogicmicro.com> |
| @@ -396,6 +403,7 @@ Jiri Slaby <jirislaby@kernel.org> <xslaby@fi.muni.cz> · 42 unmodified lines | ||
| 396 | 403 | Jisheng Zhang <jszhang@kernel.org> <jszhang@marvell.com> |
| 397 | 404 | Jisheng Zhang <jszhang@kernel.org> <Jisheng.Zhang@synaptics.com> |
| 398 | 405 | Jishnu Prakash <quic_jprakash@quicinc.com> <jprakash@codeaurora.org> |
| 406 | +Joe Damato <joe@dama.to> <jdamato@fastly.com> | |
| 399 | 407 | Joel Granados <joel.granados@kernel.org> <j.granados@samsung.com> |
| 400 | 408 | Johan Hovold <johan@kernel.org> <jhovold@gmail.com> |
| 401 | 409 | Johan Hovold <johan@kernel.org> <johan@hovoldconsulting.com> |
| @@ -490,7 +498,8 @@ Lior David <quic_liord@quicinc.com> <liord@codeaurora.org> · 88 unmodified lines | ||
| 490 | 498 | Loic Poulain <loic.poulain@oss.qualcomm.com> <loic.poulain@linaro.org> |
| 491 | 499 | Loic Poulain <loic.poulain@oss.qualcomm.com> <loic.poulain@intel.com> |
| 492 | 500 | Lorenzo Pieralisi <lpieralisi@kernel.org> <lorenzo.pieralisi@arm.com> |
| 493 | −Lorenzo Stoakes <lorenzo.stoakes@oracle.com> <lstoakes@gmail.com> | |
| 501 | +Lorenzo Stoakes <ljs@kernel.org> <lstoakes@gmail.com> | |
| 502 | +Lorenzo Stoakes <ljs@kernel.org> <lorenzo.stoakes@oracle.com> | |
| 494 | 503 | Luca Ceresoli <luca.ceresoli@bootlin.com> <luca@lucaceresoli.net> |
| 495 | 504 | Luca Weiss <luca@lucaweiss.eu> <luca@z3ntu.xyz> |
| 496 | 505 | Lucas De Marchi <demarchi@kernel.org> <lucas.demarchi@intel.com> |
| @@ -876,6 +885,7 @@ Vivien Didelot <vivien.didelot@gmail.com> <vivien.didelot@savoirfairelinux.com> · 379 unmodified lines | ||
| 876 | 885 | Vlad Dogaru <ddvlad@gmail.com> <vlad.dogaru@intel.com> |
| 877 | 886 | Vladimir Davydov <vdavydov.dev@gmail.com> <vdavydov@parallels.com> |
| 878 | 887 | Vladimir Davydov <vdavydov.dev@gmail.com> <vdavydov@virtuozzo.com> |
| 888 | +Vlastimil Babka <vbabka@kernel.org> <vbabka@suse.cz> | |
| 879 | 889 | WangYuli <wangyuli@aosc.io> <wangyl5933@chinaunicom.cn> |
| 880 | 890 | WangYuli <wangyuli@aosc.io> <wangyuli@deepin.org> |
| 881 | 891 | Weiwen Hu <huweiwen@linux.alibaba.com> <sehuww@mail.scut.edu.cn> |
| @@ -890,7 +900,8 @@ Yanteng Si <si.yanteng@linux.dev> <siyanteng@loongson.cn> · 8 unmodified lines | ||
| 890 | 900 | Ying Huang <huang.ying.caritas@gmail.com> <ying.huang@intel.com> |
| 891 | 901 | Yixun Lan <dlan@kernel.org> <dlan@gentoo.org> |
| 892 | 902 | Yixun Lan <dlan@kernel.org> <yixun.lan@amlogic.com> |
| 893 | −Yosry Ahmed <yosry.ahmed@linux.dev> <yosryahmed@google.com> | |
| 903 | +Yosry Ahmed <yosry@kernel.org> <yosryahmed@google.com> | |
| 904 | +Yosry Ahmed <yosry@kernel.org> <yosry.ahmed@linux.dev> | |
| 894 | 905 | Yu-Chun Lin <eleanor.lin@realtek.com> <eleanor15x@gmail.com> |
| 895 | 906 | Yusuke Goda <goda.yusuke@renesas.com> |
| 896 | 907 | Zack Rusin <zack.rusin@broadcom.com> <zackr@vmware.com> |
| @@ -1242,6 +1242,10 @@ N: Veaceslav Falico · 1241 unmodified lines | ||
| 1242 | 1242 | E: vfalico@gmail.com |
| 1243 | 1243 | D: Co-maintainer and co-author of the network bonding driver. |
| 1244 | 1244 | |
| 1245 | +N: Thomas Falcon | |
| 1246 | +E: tlfalcon@linux.ibm.com | |
| 1247 | +D: Initial author of the IBM ibmvnic network driver | |
| 1248 | + | |
| 1245 | 1249 | N: János Farkas |
| 1246 | 1250 | E: chexum@shadow.banki.hu |
| 1247 | 1251 | D: romfs, various (mostly networking) fixes |
| @@ -2415,6 +2419,10 @@ S: Am Muehlenweg 38 · 1167 unmodified lines | ||
| 2415 | 2419 | S: D53424 Remagen |
| 2416 | 2420 | S: Germany |
| 2417 | 2421 | |
| 2422 | +N: Jonathan Lemon | |
| 2423 | +E: jonathan.lemon@gmail.com | |
| 2424 | +D: OpenCompute PTP clock driver (ptp_ocp) | |
| 2425 | + | |
| 2418 | 2426 | N: Colin Leroy |
| 2419 | 2427 | E: colin@colino.net |
| 2420 | 2428 | W: http://www.geekounet.org/ |
| @@ -151,11 +151,11 @@ Description: · 150 unmodified lines | ||
| 151 | 151 | The algorithm_params file is write-only and is used to setup |
| 152 | 152 | compression algorithm parameters. |
| 153 | 153 | |
| 154 | −What: /sys/block/zram<id>/writeback_compressed | |
| 154 | +What: /sys/block/zram<id>/compressed_writeback | |
| 155 | 155 | Date: Decemeber 2025 |
| 156 | 156 | Contact: Richard Chang <richardycc@google.com> |
| 157 | 157 | Description: |
| 158 | − The writeback_compressed device atrribute toggles compressed | |
| 158 | + The compressed_writeback device atrribute toggles compressed | |
| 159 | 159 | writeback feature. |
| 160 | 160 | |
| 161 | 161 | What: /sys/block/zram<id>/writeback_batch_size |
| @@ -1,4 +1,4 @@ | ||
| 1 | −What: /sys/bus/platform/devices/INOU0000:XX/fn_lock_toggle_enable | |
| 1 | +What: /sys/bus/platform/devices/INOU0000:XX/fn_lock | |
| 2 | 2 | Date: November 2025 |
| 3 | 3 | KernelVersion: 6.19 |
| 4 | 4 | Contact: Armin Wolf <W_Armin@gmx.de> |
| @@ -8,15 +8,15 @@ Description: · 3 unmodified lines | ||
| 8 | 8 | |
| 9 | 9 | Reading this file returns the current enable status of the FN lock functionality. |
| 10 | 10 | |
| 11 | −What: /sys/bus/platform/devices/INOU0000:XX/super_key_toggle_enable | |
| 11 | +What: /sys/bus/platform/devices/INOU0000:XX/super_key_enable | |
| 12 | 12 | Date: November 2025 |
| 13 | 13 | KernelVersion: 6.19 |
| 14 | 14 | Contact: Armin Wolf <W_Armin@gmx.de> |
| 15 | 15 | Description: |
| 16 | − Allows userspace applications to enable/disable the super key functionality | |
| 17 | − of the integrated keyboard by writing "1"/"0" into this file. | |
| 16 | + Allows userspace applications to enable/disable the super key of the integrated | |
| 17 | + keyboard by writing "1"/"0" into this file. | |
| 18 | 18 | |
| 19 | − Reading this file returns the current enable status of the super key functionality. | |
| 19 | + Reading this file returns the current enable status of the super key. | |
| 20 | 20 | |
| 21 | 21 | What: /sys/bus/platform/devices/INOU0000:XX/touchpad_toggle_enable |
| 22 | 22 | Date: November 2025 |
| @@ -216,7 +216,7 @@ writeback_limit WO specifies the maximum amount of write IO zram · 215 unmodified lines | ||
| 216 | 216 | writeback_limit_enable RW show and set writeback_limit feature |
| 217 | 217 | writeback_batch_size RW show and set maximum number of in-flight |
| 218 | 218 | writeback operations |
| 219 | −writeback_compressed RW show and set compressed writeback feature | |
| 219 | +compressed_writeback RW show and set compressed writeback feature | |
| 220 | 220 | comp_algorithm RW show and change the compression algorithm |
| 221 | 221 | algorithm_params WO setup compression algorithm parameters |
| 222 | 222 | compact WO trigger memory compaction |
| @@ -439,11 +439,11 @@ budget in next setting is user's job. · 216 unmodified lines | ||
| 439 | 439 | By default zram stores written back pages in decompressed (raw) form, which |
| 440 | 440 | means that writeback operation involves decompression of the page before |
| 441 | 441 | writing it to the backing device. This behavior can be changed by enabling |
| 442 | −`writeback_compressed` feature, which causes zram to write compressed pages | |
| 442 | +`compressed_writeback` feature, which causes zram to write compressed pages | |
| 443 | 443 | to the backing device, thus avoiding decompression overhead. To enable |
| 444 | 444 | this feature, execute:: |
| 445 | 445 | |
| 446 | − $ echo yes > /sys/block/zramX/writeback_compressed | |
| 446 | + $ echo yes > /sys/block/zramX/compressed_writeback | |
| 447 | 447 | |
| 448 | 448 | Note that this feature should be configured before the `zramX` device is |
| 449 | 449 | initialized. |
| @@ -74,6 +74,7 @@ · 73 unmodified lines | ||
| 74 | 74 | TPM TPM drivers are enabled. |
| 75 | 75 | UMS USB Mass Storage support is enabled. |
| 76 | 76 | USB USB support is enabled. |
| 77 | + NVME NVMe support is enabled | |
| 77 | 78 | USBHID USB Human Interface Device support is enabled. |
| 78 | 79 | V4L Video For Linux support is enabled. |
| 79 | 80 | VGA The VGA console has been enabled. |
| @@ -4787,6 +4788,18 @@ Kernel parameters · 4707 unmodified lines | ||
| 4787 | 4788 | This can be set from sysctl after boot. |
| 4788 | 4789 | See Documentation/admin-guide/sysctl/vm.rst for details. |
| 4789 | 4790 | |
| 4791 | + nvme.quirks= [NVME] A list of quirk entries to augment the built-in | |
| 4792 | + nvme quirk list. List entries are separated by a | |
| 4793 | + '-' character. | |
| 4794 | + Each entry has the form VendorID:ProductID:quirk_names. | |
| 4795 | + The IDs are 4-digits hex numbers and quirk_names is a | |
| 4796 | + list of quirk names separated by commas. A quirk name | |
| 4797 | + can be prefixed by '^', meaning that the specified | |
| 4798 | + quirk must be disabled. | |
| 4799 | + | |
| 4800 | + Example: | |
| 4801 | + nvme.quirks=7710:2267:bogus_nid,^identify_cns-9900:7711:broken_msi | |
| 4802 | + | |
| 4790 | 4803 | ohci1394_dma=early [HW,EARLY] enable debugging via the ohci1394 driver. |
| 4791 | 4804 | See Documentation/core-api/debugging-via-ohci1394.rst for more |
| 4792 | 4805 | info. |
| @@ -8183,6 +8196,9 @@ Kernel parameters · 3390 unmodified lines | ||
| 8183 | 8196 | p = USB_QUIRK_SHORT_SET_ADDRESS_REQ_TIMEOUT |
| 8184 | 8197 | (Reduce timeout of the SET_ADDRESS |
| 8185 | 8198 | request from 5000 ms to 500 ms); |
| 8199 | + q = USB_QUIRK_FORCE_ONE_CONFIG (Device | |
| 8200 | + claims zero configurations, | |
| 8201 | + forcing to 1); | |
| 8186 | 8202 | Example: quirks=0781:5580:bk,0a5c:5834:gij |
| 8187 | 8203 | |
| 8188 | 8204 | usbhid.mousepoll= |
| @@ -24,7 +24,7 @@ Keyboard settings · 23 unmodified lines | ||
| 24 | 24 | |
| 25 | 25 | The ``uniwill-laptop`` driver allows the user to enable/disable: |
| 26 | 26 | |
| 27 | − - the FN and super key lock functionality of the integrated keyboard | |
| 27 | + - the FN lock and super key of the integrated keyboard | |
| 28 | 28 | - the touchpad toggle functionality of the integrated touchpad |
| 29 | 29 | |
| 30 | 30 | See Documentation/ABI/testing/sysfs-driver-uniwill-laptop for details. |
| @@ -594,6 +594,9 @@ Values: · 593 unmodified lines | ||
| 594 | 594 | their sockets will only be able to connect within their own |
| 595 | 595 | namespace. |
| 596 | 596 | |
| 597 | +The first write to ``child_ns_mode`` locks its value. Subsequent writes of the | |
| 598 | +same value succeed, but writing a different value returns ``-EBUSY``. | |
| 599 | + | |
| 597 | 600 | Changing ``child_ns_mode`` only affects namespaces created after the change; |
| 598 | 601 | it does not modify the current namespace or any existing children. |
| 599 | 602 | |
| @@ -253,7 +253,6 @@ allOf: · 252 unmodified lines | ||
| 253 | 253 | enum: |
| 254 | 254 | # these platforms support 2 streams MST on some interfaces, |
| 255 | 255 | # others are SST only |
| 256 | − - qcom,glymur-dp | |
| 257 | 256 | - qcom,sc8280xp-dp |
| 258 | 257 | - qcom,x1e80100-dp |
| 259 | 258 | then: |
| @@ -310,6 +309,26 @@ allOf: · 50 unmodified lines | ||
| 310 | 309 | minItems: 6 |
| 311 | 310 | maxItems: 8 |
| 312 | 311 | |
| 312 | + - if: | |
| 313 | + properties: | |
| 314 | + compatible: | |
| 315 | + contains: | |
| 316 | + enum: | |
| 317 | + # these platforms support 2 streams MST on some interfaces, | |
| 318 | + # others are SST only, but all controllers have 4 ports | |
| 319 | + - qcom,glymur-dp | |
| 320 | + then: | |
| 321 | + properties: | |
| 322 | + reg: | |
| 323 | + minItems: 9 | |
| 324 | + maxItems: 9 | |
| 325 | + clocks: | |
| 326 | + minItems: 5 | |
| 327 | + maxItems: 6 | |
| 328 | + clocks-names: | |
| 329 | + minItems: 5 | |
| 330 | + maxItems: 6 | |
| 331 | + | |
| 313 | 332 | unevaluatedProperties: false |
| 314 | 333 | |
| 315 | 334 | examples: |
| @@ -176,13 +176,17 @@ examples: · 175 unmodified lines | ||
| 176 | 176 | }; |
| 177 | 177 | }; |
| 178 | 178 | |
| 179 | − displayport-controller@ae90000 { | |
| 179 | + displayport-controller@af54000 { | |
| 180 | 180 | compatible = "qcom,glymur-dp"; |
| 181 | − reg = <0xae90000 0x200>, | |
| 182 | − <0xae90200 0x200>, | |
| 183 | − <0xae90400 0x600>, | |
| 184 | − <0xae91000 0x400>, | |
| 185 | − <0xae91400 0x400>; | |
| 181 | + reg = <0xaf54000 0x200>, | |
| 182 | + <0xaf54200 0x200>, | |
| 183 | + <0xaf55000 0xc00>, | |
| 184 | + <0xaf56000 0x400>, | |
| 185 | + <0xaf57000 0x400>, | |
| 186 | + <0xaf58000 0x400>, | |
| 187 | + <0xaf59000 0x400>, | |
| 188 | + <0xaf5a000 0x600>, | |
| 189 | + <0xaf5b000 0x600>; | |
| 186 | 190 | |
| 187 | 191 | interrupt-parent = <&mdss>; |
| 188 | 192 | interrupts = <12>; |
| @@ -10,7 +10,7 @@ maintainers: · 9 unmodified lines | ||
| 10 | 10 | - Krzysztof Kozlowski <krzk@kernel.org> |
| 11 | 11 | |
| 12 | 12 | description: |
| 13 | − SM8650 MSM Mobile Display Subsystem(MDSS), which encapsulates sub-blocks like | |
| 13 | + SM8750 MSM Mobile Display Subsystem(MDSS), which encapsulates sub-blocks like | |
| 14 | 14 | DPU display controller, DSI and DP interfaces etc. |
| 15 | 15 | |
| 16 | 16 | $ref: /schemas/display/msm/mdss-common.yaml# |
| @@ -16,7 +16,6 @@ description: | · 15 unmodified lines | ||
| 16 | 16 | properties: |
| 17 | 17 | compatible: |
| 18 | 18 | enum: |
| 19 | − - kontron,sa67mcu-hwmon | |
| 20 | 19 | - kontron,sl28cpld-fan |
| 21 | 20 | |
| 22 | 21 | reg: |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Synopsys DesignWare APB I2C Controller |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Jarkko Nikula <jarkko.nikula@linux.intel.com> | |
| 10 | + - Mika Westerberg <mika.westerberg@linux.intel.com> | |
| 11 | 11 | |
| 12 | 12 | allOf: |
| 13 | 13 | - $ref: /schemas/i2c/i2c-controller.yaml# |
| @@ -87,6 +87,7 @@ required: · 86 unmodified lines | ||
| 87 | 87 | |
| 88 | 88 | allOf: |
| 89 | 89 | - $ref: can-controller.yaml# |
| 90 | + - $ref: /schemas/memory-controllers/mc-peripheral-props.yaml | |
| 90 | 91 | - if: |
| 91 | 92 | properties: |
| 92 | 93 | compatible: |
| @@ -0,0 +1,93 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/powerpc/fsl/fsl,mpc83xx.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Freescale PowerQUICC II Pro (MPC83xx) platforms | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - J. Neuschäfer <j.ne@posteo.net> | |
| 11 | + | |
| 12 | +properties: | |
| 13 | + $nodename: | |
| 14 | + const: '/' | |
| 15 | + compatible: | |
| 16 | + oneOf: | |
| 17 | + - description: MPC83xx Reference Design Boards | |
| 18 | + items: | |
| 19 | + - enum: | |
| 20 | + - fsl,mpc8308rdb | |
| 21 | + - fsl,mpc8315erdb | |
| 22 | + - fsl,mpc8360rdk | |
| 23 | + - fsl,mpc8377rdb | |
| 24 | + - fsl,mpc8377wlan | |
| 25 | + - fsl,mpc8378rdb | |
| 26 | + - fsl,mpc8379rdb | |
| 27 | + | |
| 28 | + - description: MPC8313E Reference Design Board | |
| 29 | + items: | |
| 30 | + - const: MPC8313ERDB | |
| 31 | + - const: MPC831xRDB | |
| 32 | + - const: MPC83xxRDB | |
| 33 | + | |
| 34 | + - description: MPC8323E Reference Design Board | |
| 35 | + items: | |
| 36 | + - const: MPC8323ERDB | |
| 37 | + - const: MPC832xRDB | |
| 38 | + - const: MPC83xxRDB | |
| 39 | + | |
| 40 | + - description: MPC8349E-mITX(-GP) Reference Design Platform | |
| 41 | + items: | |
| 42 | + - enum: | |
| 43 | + - MPC8349EMITX | |
| 44 | + - MPC8349EMITXGP | |
| 45 | + - const: MPC834xMITX | |
| 46 | + - const: MPC83xxMITX | |
| 47 | + | |
| 48 | + - description: Keymile KMETER1 board | |
| 49 | + const: keymile,KMETER1 | |
| 50 | + | |
| 51 | + - description: MPC8308 P1M board | |
| 52 | + const: denx,mpc8308_p1m | |
| 53 | + | |
| 54 | +patternProperties: | |
| 55 | + "^soc@.*$": | |
| 56 | + type: object | |
| 57 | + properties: | |
| 58 | + compatible: | |
| 59 | + oneOf: | |
| 60 | + - items: | |
| 61 | + - enum: | |
| 62 | + - fsl,mpc8315-immr | |
| 63 | + - fsl,mpc8308-immr | |
| 64 | + - const: simple-bus | |
| 65 | + - items: | |
| 66 | + - const: fsl,mpc8360-immr | |
| 67 | + - const: fsl,immr | |
| 68 | + - const: fsl,soc | |
| 69 | + - const: simple-bus | |
| 70 | + - const: simple-bus | |
| 71 | + | |
| 72 | +additionalProperties: true | |
| 73 | + | |
| 74 | +examples: | |
| 75 | + - | | |
| 76 | + / { | |
| 77 | + compatible = "fsl,mpc8315erdb"; | |
| 78 | + model = "MPC8315E-RDB"; | |
| 79 | + #address-cells = <1>; | |
| 80 | + #size-cells = <1>; | |
| 81 | + | |
| 82 | + soc@e0000000 { | |
| 83 | + compatible = "fsl,mpc8315-immr", "simple-bus"; | |
| 84 | + reg = <0xe0000000 0x00000200>; | |
| 85 | + #address-cells = <1>; | |
| 86 | + #size-cells = <1>; | |
| 87 | + device_type = "soc"; | |
| 88 | + ranges = <0 0xe0000000 0x00100000>; | |
| 89 | + bus-frequency = <0>; | |
| 90 | + }; | |
| 91 | + }; | |
| 92 | + | |
| 93 | +... | |
| @@ -287,7 +287,7 @@ examples: · 286 unmodified lines | ||
| 287 | 287 | regulator-max-microvolt = <1700000>; |
| 288 | 288 | }; |
| 289 | 289 | mt6359_vrfck_1_ldo_reg: ldo_vrfck_1 { |
| 290 | − regulator-name = "vrfck"; | |
| 290 | + regulator-name = "vrfck_1"; | |
| 291 | 291 | regulator-min-microvolt = <1240000>; |
| 292 | 292 | regulator-max-microvolt = <1600000>; |
| 293 | 293 | }; |
| @@ -309,7 +309,7 @@ examples: · 15 unmodified lines | ||
| 309 | 309 | regulator-max-microvolt = <3300000>; |
| 310 | 310 | }; |
| 311 | 311 | mt6359_vemc_1_ldo_reg: ldo_vemc_1 { |
| 312 | − regulator-name = "vemc"; | |
| 312 | + regulator-name = "vemc_1"; | |
| 313 | 313 | regulator-min-microvolt = <2500000>; |
| 314 | 314 | regulator-max-microvolt = <3300000>; |
| 315 | 315 | }; |
| @@ -23,6 +23,7 @@ properties: · 22 unmodified lines | ||
| 23 | 23 | enum: |
| 24 | 24 | - nvidia,tegra210-audio-graph-card |
| 25 | 25 | - nvidia,tegra186-audio-graph-card |
| 26 | + - nvidia,tegra238-audio-graph-card | |
| 26 | 27 | - nvidia,tegra264-audio-graph-card |
| 27 | 28 | |
| 28 | 29 | clocks: |
| @@ -20,6 +20,7 @@ properties: · 19 unmodified lines | ||
| 20 | 20 | - renesas,r9a07g044-ssi # RZ/G2{L,LC} |
| 21 | 21 | - renesas,r9a07g054-ssi # RZ/V2L |
| 22 | 22 | - renesas,r9a08g045-ssi # RZ/G3S |
| 23 | + - renesas,r9a08g046-ssi # RZ/G3L | |
| 23 | 24 | - const: renesas,rz-ssi |
| 24 | 25 | |
| 25 | 26 | reg: |
| @@ -6,9 +6,6 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 5 unmodified lines | ||
| 6 | 6 | |
| 7 | 7 | title: Allwinner A31 SPI Controller |
| 8 | 8 | |
| 9 | −allOf: | |
| 10 | − - $ref: spi-controller.yaml | |
| 11 | − | |
| 12 | 9 | maintainers: |
| 13 | 10 | - Chen-Yu Tsai <wens@csie.org> |
| 14 | 11 | - Maxime Ripard <mripard@kernel.org> |
| @@ -82,11 +79,11 @@ patternProperties: · 67 unmodified lines | ||
| 82 | 79 | |
| 83 | 80 | spi-rx-bus-width: |
| 84 | 81 | items: |
| 85 | − - const: 1 | |
| 82 | + enum: [0, 1, 2, 4] | |
| 86 | 83 | |
| 87 | 84 | spi-tx-bus-width: |
| 88 | 85 | items: |
| 89 | − - const: 1 | |
| 86 | + enum: [0, 1, 2, 4] | |
| 90 | 87 | |
| 91 | 88 | required: |
| 92 | 89 | - compatible |
| @@ -95,6 +92,28 @@ required: · 2 unmodified lines | ||
| 95 | 92 | - clocks |
| 96 | 93 | - clock-names |
| 97 | 94 | |
| 95 | +allOf: | |
| 96 | + - $ref: spi-controller.yaml | |
| 97 | + - if: | |
| 98 | + not: | |
| 99 | + properties: | |
| 100 | + compatible: | |
| 101 | + contains: | |
| 102 | + enum: | |
| 103 | + - allwinner,sun50i-r329-spi | |
| 104 | + - allwinner,sun55i-a523-spi | |
| 105 | + then: | |
| 106 | + patternProperties: | |
| 107 | + "^.*@[0-9a-f]+": | |
| 108 | + properties: | |
| 109 | + spi-rx-bus-width: | |
| 110 | + items: | |
| 111 | + enum: [0, 1] | |
| 112 | + | |
| 113 | + spi-tx-bus-width: | |
| 114 | + items: | |
| 115 | + enum: [0, 1] | |
| 116 | + | |
| 98 | 117 | unevaluatedProperties: false |
| 99 | 118 | |
| 100 | 119 | examples: |
| @@ -22,21 +22,6 @@ allOf: · 21 unmodified lines | ||
| 22 | 22 | properties: |
| 23 | 23 | reg: |
| 24 | 24 | minItems: 2 |
| 25 | − - if: | |
| 26 | − properties: | |
| 27 | − compatible: | |
| 28 | − contains: | |
| 29 | − enum: | |
| 30 | − - baikal,bt1-sys-ssi | |
| 31 | − then: | |
| 32 | − properties: | |
| 33 | − mux-controls: | |
| 34 | − maxItems: 1 | |
| 35 | − required: | |
| 36 | − - mux-controls | |
| 37 | − else: | |
| 38 | − required: | |
| 39 | − - interrupts | |
| 40 | 25 | - if: |
| 41 | 26 | properties: |
| 42 | 27 | compatible: |
| @@ -75,10 +60,6 @@ properties: · 32 unmodified lines | ||
| 75 | 60 | const: intel,mountevans-imc-ssi |
| 76 | 61 | - description: AMD Pensando Elba SoC SPI Controller |
| 77 | 62 | const: amd,pensando-elba-spi |
| 78 | − - description: Baikal-T1 SPI Controller | |
| 79 | − const: baikal,bt1-ssi | |
| 80 | − - description: Baikal-T1 System Boot SPI Controller | |
| 81 | − const: baikal,bt1-sys-ssi | |
| 82 | 63 | - description: Canaan Kendryte K210 SoS SPI Controller |
| 83 | 64 | const: canaan,k210-spi |
| 84 | 65 | - description: Renesas RZ/N1 SPI Controller |
| @@ -170,6 +151,7 @@ required: · 85 unmodified lines | ||
| 170 | 151 | - "#address-cells" |
| 171 | 152 | - "#size-cells" |
| 172 | 153 | - clocks |
| 154 | + - interrupts | |
| 173 | 155 | |
| 174 | 156 | examples: |
| 175 | 157 | - | |
| @@ -190,15 +172,4 @@ examples: · 14 unmodified lines | ||
| 190 | 172 | rx-sample-delay-ns = <7>; |
| 191 | 173 | }; |
| 192 | 174 | }; |
| 193 | − - | | |
| 194 | − spi@1f040100 { | |
| 195 | − compatible = "baikal,bt1-sys-ssi"; | |
| 196 | − reg = <0x1f040100 0x900>, | |
| 197 | − <0x1c000000 0x1000000>; | |
| 198 | − #address-cells = <1>; | |
| 199 | − #size-cells = <0>; | |
| 200 | − mux-controls = <&boot_mux>; | |
| 201 | − clocks = <&ccu_sys>; | |
| 202 | − clock-names = "ssi_clk"; | |
| 203 | − }; | |
| 204 | 175 | ... |
| @@ -57,7 +57,7 @@ Supported chips: · 56 unmodified lines | ||
| 57 | 57 | - https://ww1.microchip.com/downloads/en/DeviceDoc/EMC1438%20DS%20Rev.%201.0%20(04-29-10).pdf |
| 58 | 58 | |
| 59 | 59 | Author: |
| 60 | − Kalhan Trisal <kalhan.trisal@intel.com | |
| 60 | + Kalhan Trisal <kalhan.trisal@intel.com> | |
| 61 | 61 | |
| 62 | 62 | |
| 63 | 63 | Description |
| @@ -220,7 +220,6 @@ Hardware Monitoring Kernel Drivers · 219 unmodified lines | ||
| 220 | 220 | q54sj108a2 |
| 221 | 221 | qnap-mcu-hwmon |
| 222 | 222 | raspberrypi-hwmon |
| 223 | − sa67 | |
| 224 | 223 | sbrmi |
| 225 | 224 | sbtsi_temp |
| 226 | 225 | sch5627 |
| @@ -1,41 +0,0 @@ | ||
| 1 | −.. SPDX-License-Identifier: GPL-2.0-only | |
| 2 | − | |
| 3 | −Kernel driver sa67mcu | |
| 4 | −===================== | |
| 5 | − | |
| 6 | −Supported chips: | |
| 7 | − | |
| 8 | − * Kontron sa67mcu | |
| 9 | − | |
| 10 | − Prefix: 'sa67mcu' | |
| 11 | − | |
| 12 | − Datasheet: not available | |
| 13 | − | |
| 14 | −Authors: Michael Walle <mwalle@kernel.org> | |
| 15 | − | |
| 16 | −Description | |
| 17 | −----------- | |
| 18 | − | |
| 19 | −The sa67mcu is a board management controller which also exposes a hardware | |
| 20 | −monitoring controller. | |
| 21 | − | |
| 22 | −The controller has two voltage and one temperature sensor. The values are | |
| 23 | −hold in two 8 bit registers to form one 16 bit value. Reading the lower byte | |
| 24 | −will also capture the high byte to make the access atomic. The unit of the | |
| 25 | −volatge sensors are 1mV and the unit of the temperature sensor is 0.1degC. | |
| 26 | − | |
| 27 | −Sysfs entries | |
| 28 | −------------- | |
| 29 | − | |
| 30 | −The following attributes are supported. | |
| 31 | − | |
| 32 | −======================= ======================================================== | |
| 33 | −in0_label "VDDIN" | |
| 34 | −in0_input Measured VDDIN voltage. | |
| 35 | − | |
| 36 | −in1_label "VDD_RTC" | |
| 37 | −in1_input Measured VDD_RTC voltage. | |
| 38 | − | |
| 39 | −temp1_input MCU temperature. Roughly the board temperature. | |
| 40 | −======================= ======================================================== | |
| 41 | − | |
| @@ -152,7 +152,7 @@ operations: · 151 unmodified lines | ||
| 152 | 152 | - compound-ops |
| 153 | 153 | - |
| 154 | 154 | name: threads-set |
| 155 | − doc: set the number of running threads | |
| 155 | + doc: set the maximum number of running threads | |
| 156 | 156 | attribute-set: server |
| 157 | 157 | flags: [admin-perm] |
| 158 | 158 | do: |
| @@ -165,7 +165,7 @@ operations: · 6 unmodified lines | ||
| 165 | 165 | - min-threads |
| 166 | 166 | - |
| 167 | 167 | name: threads-get |
| 168 | − doc: get the number of running threads | |
| 168 | + doc: get the maximum number of running threads | |
| 169 | 169 | attribute-set: server |
| 170 | 170 | do: |
| 171 | 171 | reply: |
| @@ -43,7 +43,6 @@ options should be enabled to use sched_ext: · 42 unmodified lines | ||
| 43 | 43 | CONFIG_DEBUG_INFO_BTF=y |
| 44 | 44 | CONFIG_BPF_JIT_ALWAYS_ON=y |
| 45 | 45 | CONFIG_BPF_JIT_DEFAULT_ON=y |
| 46 | − CONFIG_PAHOLE_HAS_BTF_TAG=y | |
| 47 | 46 | |
| 48 | 47 | sched_ext is used only when the BPF scheduler is loaded and running. |
| 49 | 48 | |
| @@ -58,7 +57,8 @@ in ``ops->flags``, all ``SCHED_NORMAL``, ``SCHED_BATCH``, ``SCHED_IDLE``, and · 8 unmodified lines | ||
| 58 | 57 | However, when the BPF scheduler is loaded and ``SCX_OPS_SWITCH_PARTIAL`` is |
| 59 | 58 | set in ``ops->flags``, only tasks with the ``SCHED_EXT`` policy are scheduled |
| 60 | 59 | by sched_ext, while tasks with ``SCHED_NORMAL``, ``SCHED_BATCH`` and |
| 61 | −``SCHED_IDLE`` policies are scheduled by the fair-class scheduler. | |
| 60 | +``SCHED_IDLE`` policies are scheduled by the fair-class scheduler which has | |
| 61 | +higher sched_class precedence than ``SCHED_EXT``. | |
| 62 | 62 | |
| 63 | 63 | Terminating the sched_ext scheduler program, triggering `SysRq-S`, or |
| 64 | 64 | detection of any internal error including stalled runnable tasks aborts the |
| @@ -345,6 +345,8 @@ Where to Look · 280 unmodified lines | ||
| 345 | 345 | The functions prefixed with ``scx_bpf_`` can be called from the BPF |
| 346 | 346 | scheduler. |
| 347 | 347 | |
| 348 | +* ``kernel/sched/ext_idle.c`` contains the built-in idle CPU selection policy. | |
| 349 | + | |
| 348 | 350 | * ``tools/sched_ext/`` hosts example BPF scheduler implementations. |
| 349 | 351 | |
| 350 | 352 | * ``scx_simple[.bpf].c``: Minimal global FIFO scheduler example using a |
| @@ -353,13 +355,35 @@ Where to Look · 2 unmodified lines | ||
| 353 | 355 | * ``scx_qmap[.bpf].c``: A multi-level FIFO scheduler supporting five |
| 354 | 356 | levels of priority implemented with ``BPF_MAP_TYPE_QUEUE``. |
| 355 | 357 | |
| 358 | + * ``scx_central[.bpf].c``: A central FIFO scheduler where all scheduling | |
| 359 | + decisions are made on one CPU, demonstrating ``LOCAL_ON`` dispatching, | |
| 360 | + tickless operation, and kthread preemption. | |
| 361 | + | |
| 362 | + * ``scx_cpu0[.bpf].c``: A scheduler that queues all tasks to a shared DSQ | |
| 363 | + and only dispatches them on CPU0 in FIFO order. Useful for testing bypass | |
| 364 | + behavior. | |
| 365 | + | |
| 366 | + * ``scx_flatcg[.bpf].c``: A flattened cgroup hierarchy scheduler | |
| 367 | + implementing hierarchical weight-based cgroup CPU control by compounding | |
| 368 | + each cgroup's share at every level into a single flat scheduling layer. | |
| 369 | + | |
| 370 | + * ``scx_pair[.bpf].c``: A core-scheduling example that always makes | |
| 371 | + sibling CPU pairs execute tasks from the same CPU cgroup. | |
| 372 | + | |
| 373 | + * ``scx_sdt[.bpf].c``: A variation of ``scx_simple`` demonstrating BPF | |
| 374 | + arena memory management for per-task data. | |
| 375 | + | |
| 376 | + * ``scx_userland[.bpf].c``: A minimal scheduler demonstrating user space | |
| 377 | + scheduling. Tasks with CPU affinity are direct-dispatched in FIFO order; | |
| 378 | + all others are scheduled in user space by a simple vruntime scheduler. | |
| 379 | + | |
| 356 | 380 | ABI Instability |
| 357 | 381 | =============== |
| 358 | 382 | |
| 359 | 383 | The APIs provided by sched_ext to BPF schedulers programs have no stability |
| 360 | 384 | guarantees. This includes the ops table callbacks and constants defined in |
| 361 | 385 | ``include/linux/sched/ext.h``, as well as the ``scx_bpf_`` kfuncs defined in |
| 362 | −``kernel/sched/ext.c``. | |
| 386 | +``kernel/sched/ext.c`` and ``kernel/sched/ext_idle.c``. | |
| 363 | 387 | |
| 364 | 388 | While we will attempt to provide a relatively stable API surface when |
| 365 | 389 | possible, they are subject to change without warning between kernel |
| @@ -2372,6 +2372,10 @@ quirk_flags · 2371 unmodified lines | ||
| 2372 | 2372 | audible volume |
| 2373 | 2373 | * bit 25: ``mixer_capture_min_mute`` |
| 2374 | 2374 | Similar to bit 24 but for capture streams |
| 2375 | + * bit 26: ``skip_iface_setup`` | |
| 2376 | + Skip the probe-time interface setup (usb_set_interface, | |
| 2377 | + init_pitch, init_sample_rate); redundant with | |
| 2378 | + snd_usb_endpoint_prepare() at stream-open time | |
| 2375 | 2379 | |
| 2376 | 2380 | This module supports multiple devices, autoprobe and hotplugging. |
| 2377 | 2381 | |
| @@ -1396,7 +1396,10 @@ or its flags may be modified, but it may not be resized. · 1395 unmodified lines | ||
| 1396 | 1396 | Memory for the region is taken starting at the address denoted by the |
| 1397 | 1397 | field userspace_addr, which must point at user addressable memory for |
| 1398 | 1398 | the entire memory slot size. Any object may back this memory, including |
| 1399 | −anonymous memory, ordinary files, and hugetlbfs. | |
| 1399 | +anonymous memory, ordinary files, and hugetlbfs. Changes in the backing | |
| 1400 | +of the memory region are automatically reflected into the guest. | |
| 1401 | +For example, an mmap() that affects the region will be made visible | |
| 1402 | +immediately. Another example is madvise(MADV_DROP). | |
| 1400 | 1403 | |
| 1401 | 1404 | On architectures that support a form of address tagging, userspace_addr must |
| 1402 | 1405 | be an untagged address. |
| @@ -1412,11 +1415,6 @@ use it. The latter can be set, if KVM_CAP_READONLY_MEM capability allows it, · 9 unmodified lines | ||
| 1412 | 1415 | to make a new slot read-only. In this case, writes to this memory will be |
| 1413 | 1416 | posted to userspace as KVM_EXIT_MMIO exits. |
| 1414 | 1417 | |
| 1415 | −When the KVM_CAP_SYNC_MMU capability is available, changes in the backing of | |
| 1416 | −the memory region are automatically reflected into the guest. For example, an | |
| 1417 | −mmap() that affects the region will be made visible immediately. Another | |
| 1418 | −example is madvise(MADV_DROP). | |
| 1419 | − | |
| 1420 | 1418 | For TDX guest, deleting/moving memory region loses guest memory contents. |
| 1421 | 1419 | Read only region isn't supported. Only as-id 0 is supported. |
| 1422 | 1420 | |
| @@ -8437,115 +8435,123 @@ KVM_CHECK_EXTENSION. · 7014 unmodified lines | ||
| 8437 | 8435 | |
| 8438 | 8436 | The valid bits in cap.args[0] are: |
| 8439 | 8437 | |
| 8440 | −=================================== ============================================ | |
| 8441 | − KVM_X86_QUIRK_LINT0_REENABLED By default, the reset value for the LVT | |
| 8442 | − LINT0 register is 0x700 (APIC_MODE_EXTINT). | |
| 8443 | − When this quirk is disabled, the reset value | |
| 8444 | − is 0x10000 (APIC_LVT_MASKED). | |
| 8445 | − | |
| 8446 | − KVM_X86_QUIRK_CD_NW_CLEARED By default, KVM clears CR0.CD and CR0.NW on | |
| 8447 | − AMD CPUs to workaround buggy guest firmware | |
| 8448 | − that runs in perpetuity with CR0.CD, i.e. | |
| 8449 | − with caches in "no fill" mode. | |
| 8450 | − | |
| 8451 | − When this quirk is disabled, KVM does not | |
| 8452 | − change the value of CR0.CD and CR0.NW. | |
| 8453 | − | |
| 8454 | − KVM_X86_QUIRK_LAPIC_MMIO_HOLE By default, the MMIO LAPIC interface is | |
| 8455 | − available even when configured for x2APIC | |
| 8456 | − mode. When this quirk is disabled, KVM | |
| 8457 | − disables the MMIO LAPIC interface if the | |
| 8458 | − LAPIC is in x2APIC mode. | |
| 8459 | − | |
| 8460 | − KVM_X86_QUIRK_OUT_7E_INC_RIP By default, KVM pre-increments %rip before | |
| 8461 | − exiting to userspace for an OUT instruction | |
| 8462 | − to port 0x7e. When this quirk is disabled, | |
| 8463 | − KVM does not pre-increment %rip before | |
| 8464 | − exiting to userspace. | |
| 8465 | − | |
| 8466 | − KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT When this quirk is disabled, KVM sets | |
| 8467 | − CPUID.01H:ECX[bit 3] (MONITOR/MWAIT) if | |
| 8468 | − IA32_MISC_ENABLE[bit 18] (MWAIT) is set. | |
| 8469 | − Additionally, when this quirk is disabled, | |
| 8470 | − KVM clears CPUID.01H:ECX[bit 3] if | |
| 8471 | − IA32_MISC_ENABLE[bit 18] is cleared. | |
| 8472 | − | |
| 8473 | − KVM_X86_QUIRK_FIX_HYPERCALL_INSN By default, KVM rewrites guest | |
| 8474 | − VMMCALL/VMCALL instructions to match the | |
| 8475 | − vendor's hypercall instruction for the | |
| 8476 | − system. When this quirk is disabled, KVM | |
| 8477 | − will no longer rewrite invalid guest | |
| 8478 | − hypercall instructions. Executing the | |
| 8479 | − incorrect hypercall instruction will | |
| 8480 | − generate a #UD within the guest. | |
| 8481 | − | |
| 8482 | −KVM_X86_QUIRK_MWAIT_NEVER_UD_FAULTS By default, KVM emulates MONITOR/MWAIT (if | |
| 8483 | − they are intercepted) as NOPs regardless of | |
| 8484 | − whether or not MONITOR/MWAIT are supported | |
| 8485 | − according to guest CPUID. When this quirk | |
| 8486 | − is disabled and KVM_X86_DISABLE_EXITS_MWAIT | |
| 8487 | − is not set (MONITOR/MWAIT are intercepted), | |
| 8488 | − KVM will inject a #UD on MONITOR/MWAIT if | |
| 8489 | − they're unsupported per guest CPUID. Note, | |
| 8490 | − KVM will modify MONITOR/MWAIT support in | |
| 8491 | − guest CPUID on writes to MISC_ENABLE if | |
| 8492 | − KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT is | |
| 8493 | − disabled. | |
| 8494 | − | |
| 8495 | −KVM_X86_QUIRK_SLOT_ZAP_ALL By default, for KVM_X86_DEFAULT_VM VMs, KVM | |
| 8496 | − invalidates all SPTEs in all memslots and | |
| 8497 | − address spaces when a memslot is deleted or | |
| 8498 | − moved. When this quirk is disabled (or the | |
| 8499 | − VM type isn't KVM_X86_DEFAULT_VM), KVM only | |
| 8500 | − ensures the backing memory of the deleted | |
| 8501 | − or moved memslot isn't reachable, i.e KVM | |
| 8502 | − _may_ invalidate only SPTEs related to the | |
| 8503 | − memslot. | |
| 8504 | − | |
| 8505 | −KVM_X86_QUIRK_STUFF_FEATURE_MSRS By default, at vCPU creation, KVM sets the | |
| 8506 | − vCPU's MSR_IA32_PERF_CAPABILITIES (0x345), | |
| 8507 | − MSR_IA32_ARCH_CAPABILITIES (0x10a), | |
| 8508 | − MSR_PLATFORM_INFO (0xce), and all VMX MSRs | |
| 8509 | − (0x480..0x492) to the maximal capabilities | |
| 8510 | − supported by KVM. KVM also sets | |
| 8511 | − MSR_IA32_UCODE_REV (0x8b) to an arbitrary | |
| 8512 | − value (which is different for Intel vs. | |
| 8513 | − AMD). Lastly, when guest CPUID is set (by | |
| 8514 | − userspace), KVM modifies select VMX MSR | |
| 8515 | − fields to force consistency between guest | |
| 8516 | − CPUID and L2's effective ISA. When this | |
| 8517 | − quirk is disabled, KVM zeroes the vCPU's MSR | |
| 8518 | − values (with two exceptions, see below), | |
| 8519 | − i.e. treats the feature MSRs like CPUID | |
| 8520 | − leaves and gives userspace full control of | |
| 8521 | − the vCPU model definition. This quirk does | |
| 8522 | − not affect VMX MSRs CR0/CR4_FIXED1 (0x487 | |
| 8523 | − and 0x489), as KVM does now allow them to | |
| 8524 | − be set by userspace (KVM sets them based on | |
| 8525 | − guest CPUID, for safety purposes). | |
| 8526 | − | |
| 8527 | −KVM_X86_QUIRK_IGNORE_GUEST_PAT By default, on Intel platforms, KVM ignores | |
| 8528 | − guest PAT and forces the effective memory | |
| 8529 | − type to WB in EPT. The quirk is not available | |
| 8530 | − on Intel platforms which are incapable of | |
| 8531 | − safely honoring guest PAT (i.e., without CPU | |
| 8532 | − self-snoop, KVM always ignores guest PAT and | |
| 8533 | − forces effective memory type to WB). It is | |
| 8534 | − also ignored on AMD platforms or, on Intel, | |
| 8535 | − when a VM has non-coherent DMA devices | |
| 8536 | − assigned; KVM always honors guest PAT in | |
| 8537 | − such case. The quirk is needed to avoid | |
| 8538 | − slowdowns on certain Intel Xeon platforms | |
| 8539 | − (e.g. ICX, SPR) where self-snoop feature is | |
| 8540 | − supported but UC is slow enough to cause | |
| 8541 | − issues with some older guests that use | |
| 8542 | − UC instead of WC to map the video RAM. | |
| 8543 | − Userspace can disable the quirk to honor | |
| 8544 | − guest PAT if it knows that there is no such | |
| 8545 | − guest software, for example if it does not | |
| 8546 | − expose a bochs graphics device (which is | |
| 8547 | − known to have had a buggy driver). | |
| 8548 | −=================================== ============================================ | |
| 8438 | +======================================== ================================================ | |
| 8439 | +KVM_X86_QUIRK_LINT0_REENABLED By default, the reset value for the LVT | |
| 8440 | + LINT0 register is 0x700 (APIC_MODE_EXTINT). | |
| 8441 | + When this quirk is disabled, the reset value | |
| 8442 | + is 0x10000 (APIC_LVT_MASKED). | |
| 8443 | + | |
| 8444 | +KVM_X86_QUIRK_CD_NW_CLEARED By default, KVM clears CR0.CD and CR0.NW on | |
| 8445 | + AMD CPUs to workaround buggy guest firmware | |
| 8446 | + that runs in perpetuity with CR0.CD, i.e. | |
| 8447 | + with caches in "no fill" mode. | |
| 8448 | + | |
| 8449 | + When this quirk is disabled, KVM does not | |
| 8450 | + change the value of CR0.CD and CR0.NW. | |
| 8451 | + | |
| 8452 | +KVM_X86_QUIRK_LAPIC_MMIO_HOLE By default, the MMIO LAPIC interface is | |
| 8453 | + available even when configured for x2APIC | |
| 8454 | + mode. When this quirk is disabled, KVM | |
| 8455 | + disables the MMIO LAPIC interface if the | |
| 8456 | + LAPIC is in x2APIC mode. | |
| 8457 | + | |
| 8458 | +KVM_X86_QUIRK_OUT_7E_INC_RIP By default, KVM pre-increments %rip before | |
| 8459 | + exiting to userspace for an OUT instruction | |
| 8460 | + to port 0x7e. When this quirk is disabled, | |
| 8461 | + KVM does not pre-increment %rip before | |
| 8462 | + exiting to userspace. | |
| 8463 | + | |
| 8464 | +KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT When this quirk is disabled, KVM sets | |
| 8465 | + CPUID.01H:ECX[bit 3] (MONITOR/MWAIT) if | |
| 8466 | + IA32_MISC_ENABLE[bit 18] (MWAIT) is set. | |
| 8467 | + Additionally, when this quirk is disabled, | |
| 8468 | + KVM clears CPUID.01H:ECX[bit 3] if | |
| 8469 | + IA32_MISC_ENABLE[bit 18] is cleared. | |
| 8470 | + | |
| 8471 | +KVM_X86_QUIRK_FIX_HYPERCALL_INSN By default, KVM rewrites guest | |
| 8472 | + VMMCALL/VMCALL instructions to match the | |
| 8473 | + vendor's hypercall instruction for the | |
| 8474 | + system. When this quirk is disabled, KVM | |
| 8475 | + will no longer rewrite invalid guest | |
| 8476 | + hypercall instructions. Executing the | |
| 8477 | + incorrect hypercall instruction will | |
| 8478 | + generate a #UD within the guest. | |
| 8479 | + | |
| 8480 | +KVM_X86_QUIRK_MWAIT_NEVER_UD_FAULTS By default, KVM emulates MONITOR/MWAIT (if | |
| 8481 | + they are intercepted) as NOPs regardless of | |
| 8482 | + whether or not MONITOR/MWAIT are supported | |
| 8483 | + according to guest CPUID. When this quirk | |
| 8484 | + is disabled and KVM_X86_DISABLE_EXITS_MWAIT | |
| 8485 | + is not set (MONITOR/MWAIT are intercepted), | |
| 8486 | + KVM will inject a #UD on MONITOR/MWAIT if | |
| 8487 | + they're unsupported per guest CPUID. Note, | |
| 8488 | + KVM will modify MONITOR/MWAIT support in | |
| 8489 | + guest CPUID on writes to MISC_ENABLE if | |
| 8490 | + KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT is | |
| 8491 | + disabled. | |
| 8492 | + | |
| 8493 | +KVM_X86_QUIRK_SLOT_ZAP_ALL By default, for KVM_X86_DEFAULT_VM VMs, KVM | |
| 8494 | + invalidates all SPTEs in all memslots and | |
| 8495 | + address spaces when a memslot is deleted or | |
| 8496 | + moved. When this quirk is disabled (or the | |
| 8497 | + VM type isn't KVM_X86_DEFAULT_VM), KVM only | |
| 8498 | + ensures the backing memory of the deleted | |
| 8499 | + or moved memslot isn't reachable, i.e KVM | |
| 8500 | + _may_ invalidate only SPTEs related to the | |
| 8501 | + memslot. | |
| 8502 | + | |
| 8503 | +KVM_X86_QUIRK_STUFF_FEATURE_MSRS By default, at vCPU creation, KVM sets the | |
| 8504 | + vCPU's MSR_IA32_PERF_CAPABILITIES (0x345), | |
| 8505 | + MSR_IA32_ARCH_CAPABILITIES (0x10a), | |
| 8506 | + MSR_PLATFORM_INFO (0xce), and all VMX MSRs | |
| 8507 | + (0x480..0x492) to the maximal capabilities | |
| 8508 | + supported by KVM. KVM also sets | |
| 8509 | + MSR_IA32_UCODE_REV (0x8b) to an arbitrary | |
| 8510 | + value (which is different for Intel vs. | |
| 8511 | + AMD). Lastly, when guest CPUID is set (by | |
| 8512 | + userspace), KVM modifies select VMX MSR | |
| 8513 | + fields to force consistency between guest | |
| 8514 | + CPUID and L2's effective ISA. When this | |
| 8515 | + quirk is disabled, KVM zeroes the vCPU's MSR | |
| 8516 | + values (with two exceptions, see below), | |
| 8517 | + i.e. treats the feature MSRs like CPUID | |
| 8518 | + leaves and gives userspace full control of | |
| 8519 | + the vCPU model definition. This quirk does | |
| 8520 | + not affect VMX MSRs CR0/CR4_FIXED1 (0x487 | |
| 8521 | + and 0x489), as KVM does now allow them to | |
| 8522 | + be set by userspace (KVM sets them based on | |
| 8523 | + guest CPUID, for safety purposes). | |
| 8524 | + | |
| 8525 | +KVM_X86_QUIRK_IGNORE_GUEST_PAT By default, on Intel platforms, KVM ignores | |
| 8526 | + guest PAT and forces the effective memory | |
| 8527 | + type to WB in EPT. The quirk is not available | |
| 8528 | + on Intel platforms which are incapable of | |
| 8529 | + safely honoring guest PAT (i.e., without CPU | |
| 8530 | + self-snoop, KVM always ignores guest PAT and | |
| 8531 | + forces effective memory type to WB). It is | |
| 8532 | + also ignored on AMD platforms or, on Intel, | |
| 8533 | + when a VM has non-coherent DMA devices | |
| 8534 | + assigned; KVM always honors guest PAT in | |
| 8535 | + such case. The quirk is needed to avoid | |
| 8536 | + slowdowns on certain Intel Xeon platforms | |
| 8537 | + (e.g. ICX, SPR) where self-snoop feature is | |
| 8538 | + supported but UC is slow enough to cause | |
| 8539 | + issues with some older guests that use | |
| 8540 | + UC instead of WC to map the video RAM. | |
| 8541 | + Userspace can disable the quirk to honor | |
| 8542 | + guest PAT if it knows that there is no such | |
| 8543 | + guest software, for example if it does not | |
| 8544 | + expose a bochs graphics device (which is | |
| 8545 | + known to have had a buggy driver). | |
| 8546 | + | |
| 8547 | +KVM_X86_QUIRK_VMCS12_ALLOW_FREEZE_IN_SMM By default, KVM relaxes the consistency | |
| 8548 | + check for GUEST_IA32_DEBUGCTL in vmcs12 | |
| 8549 | + to allow FREEZE_IN_SMM to be set. When | |
| 8550 | + this quirk is disabled, KVM requires this | |
| 8551 | + bit to be cleared. Note that the vmcs02 | |
| 8552 | + bit is still completely controlled by the | |
| 8553 | + host, regardless of the quirk setting. | |
| 8554 | +======================================== ================================================ | |
| 8549 | 8555 | |
| 8550 | 8556 | 7.32 KVM_CAP_MAX_VCPU_ID |
| 8551 | 8557 | ------------------------ |
| @@ -17,6 +17,8 @@ The acquisition orders for mutexes are as follows: · 16 unmodified lines | ||
| 17 | 17 | |
| 18 | 18 | - kvm->lock is taken outside kvm->slots_lock and kvm->irq_lock |
| 19 | 19 | |
| 20 | +- vcpu->mutex is taken outside kvm->slots_lock and kvm->slots_arch_lock | |
| 21 | + | |
| 20 | 22 | - kvm->slots_lock is taken outside kvm->irq_lock, though acquiring |
| 21 | 23 | them together is quite rare. |
| 22 | 24 | |
| @@ -996,10 +996,8 @@ F: Documentation/devicetree/bindings/thermal/amazon,al-thermal.yaml · 995 unmodified lines | ||
| 996 | 996 | F: drivers/thermal/thermal_mmio.c |
| 997 | 997 | |
| 998 | 998 | AMAZON ETHERNET DRIVERS |
| 999 | −M: Shay Agroskin <shayagr@amazon.com> | |
| 1000 | 999 | M: Arthur Kiyanovski <akiyano@amazon.com> |
| 1001 | −R: David Arinzon <darinzon@amazon.com> | |
| 1002 | −R: Saeed Bishara <saeedb@amazon.com> | |
| 1000 | +M: David Arinzon <darinzon@amazon.com> | |
| 1003 | 1001 | L: netdev@vger.kernel.org |
| 1004 | 1002 | S: Maintained |
| 1005 | 1003 | F: Documentation/networking/device_drivers/ethernet/amazon/ena.rst |
| @@ -1295,7 +1293,6 @@ F: include/trace/events/amdxdna.h · 289 unmodified lines | ||
| 1295 | 1293 | F: include/uapi/drm/amdxdna_accel.h |
| 1296 | 1294 | |
| 1297 | 1295 | AMD XGBE DRIVER |
| 1298 | −M: "Shyam Sundar S K" <Shyam-sundar.S-k@amd.com> | |
| 1299 | 1296 | M: Raju Rangoju <Raju.Rangoju@amd.com> |
| 1300 | 1297 | L: netdev@vger.kernel.org |
| 1301 | 1298 | S: Maintained |
| @@ -4638,7 +4635,6 @@ F: drivers/bluetooth/ · 3336 unmodified lines | ||
| 4638 | 4635 | |
| 4639 | 4636 | BLUETOOTH SUBSYSTEM |
| 4640 | 4637 | M: Marcel Holtmann <marcel@holtmann.org> |
| 4641 | −M: Johan Hedberg <johan.hedberg@gmail.com> | |
| 4642 | 4638 | M: Luiz Augusto von Dentz <luiz.dentz@gmail.com> |
| 4643 | 4639 | L: linux-bluetooth@vger.kernel.org |
| 4644 | 4640 | S: Supported |
| @@ -6233,20 +6229,20 @@ F: drivers/scsi/fnic/ · 1588 unmodified lines | ||
| 6233 | 6229 | |
| 6234 | 6230 | CISCO SCSI HBA DRIVER |
| 6235 | 6231 | M: Karan Tilak Kumar <kartilak@cisco.com> |
| 6232 | +M: Narsimhulu Musini <nmusini@cisco.com> | |
| 6236 | 6233 | M: Sesidhar Baddela <sebaddel@cisco.com> |
| 6237 | 6234 | L: linux-scsi@vger.kernel.org |
| 6238 | 6235 | S: Supported |
| 6239 | 6236 | F: drivers/scsi/snic/ |
| 6240 | 6237 | |
| 6241 | 6238 | CISCO VIC ETHERNET NIC DRIVER |
| 6242 | −M: Christian Benvenuti <benve@cisco.com> | |
| 6243 | 6239 | M: Satish Kharat <satishkh@cisco.com> |
| 6244 | 6240 | S: Maintained |
| 6245 | 6241 | F: drivers/net/ethernet/cisco/enic/ |
| 6246 | 6242 | |
| 6247 | 6243 | CISCO VIC LOW LATENCY NIC DRIVER |
| 6248 | −M: Christian Benvenuti <benve@cisco.com> | |
| 6249 | 6244 | M: Nelson Escobar <neescoba@cisco.com> |
| 6245 | +M: Satish Kharat <satishkh@cisco.com> | |
| 6250 | 6246 | S: Supported |
| 6251 | 6247 | F: drivers/infiniband/hw/usnic/ |
| 6252 | 6248 | |
| @@ -6300,7 +6296,7 @@ S: Maintained · 47 unmodified lines | ||
| 6300 | 6296 | F: include/linux/clk.h |
| 6301 | 6297 | |
| 6302 | 6298 | CLOCKSOURCE, CLOCKEVENT DRIVERS |
| 6303 | −M: Daniel Lezcano <daniel.lezcano@linaro.org> | |
| 6299 | +M: Daniel Lezcano <daniel.lezcano@kernel.org> | |
| 6304 | 6300 | M: Thomas Gleixner <tglx@kernel.org> |
| 6305 | 6301 | L: linux-kernel@vger.kernel.org |
| 6306 | 6302 | S: Supported |
| @@ -6689,7 +6685,7 @@ F: rust/kernel/cpu.rs · 382 unmodified lines | ||
| 6689 | 6685 | |
| 6690 | 6686 | CPU IDLE TIME MANAGEMENT FRAMEWORK |
| 6691 | 6687 | M: "Rafael J. Wysocki" <rafael@kernel.org> |
| 6692 | −M: Daniel Lezcano <daniel.lezcano@linaro.org> | |
| 6688 | +M: Daniel Lezcano <daniel.lezcano@kernel.org> | |
| 6693 | 6689 | R: Christian Loehle <christian.loehle@arm.com> |
| 6694 | 6690 | L: linux-pm@vger.kernel.org |
| 6695 | 6691 | S: Maintained |
| @@ -6719,7 +6715,7 @@ F: arch/x86/kernel/msr.c · 23 unmodified lines | ||
| 6719 | 6715 | |
| 6720 | 6716 | CPUIDLE DRIVER - ARM BIG LITTLE |
| 6721 | 6717 | M: Lorenzo Pieralisi <lpieralisi@kernel.org> |
| 6722 | −M: Daniel Lezcano <daniel.lezcano@linaro.org> | |
| 6718 | +M: Daniel Lezcano <daniel.lezcano@kernel.org> | |
| 6723 | 6719 | L: linux-pm@vger.kernel.org |
| 6724 | 6720 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 6725 | 6721 | S: Maintained |
| @@ -6727,7 +6723,7 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm.git · 1 unmodified line | ||
| 6727 | 6723 | F: drivers/cpuidle/cpuidle-big_little.c |
| 6728 | 6724 | |
| 6729 | 6725 | CPUIDLE DRIVER - ARM EXYNOS |
| 6730 | −M: Daniel Lezcano <daniel.lezcano@linaro.org> | |
| 6726 | +M: Daniel Lezcano <daniel.lezcano@kernel.org> | |
| 6731 | 6727 | M: Kukjin Kim <kgene@kernel.org> |
| 6732 | 6728 | R: Krzysztof Kozlowski <krzk@kernel.org> |
| 6733 | 6729 | L: linux-pm@vger.kernel.org |
| @@ -8650,9 +8646,8 @@ F: drivers/gpu/drm/lima/ · 1916 unmodified lines | ||
| 8650 | 8646 | F: include/uapi/drm/lima_drm.h |
| 8651 | 8647 | |
| 8652 | 8648 | DRM DRIVERS FOR LOONGSON |
| 8653 | −M: Sui Jingfeng <suijingfeng@loongson.cn> | |
| 8654 | 8649 | L: dri-devel@lists.freedesktop.org |
| 8655 | −S: Supported | |
| 8650 | +S: Orphan | |
| 8656 | 8651 | T: git https://gitlab.freedesktop.org/drm/misc/kernel.git |
| 8657 | 8652 | F: drivers/gpu/drm/loongson/ |
| 8658 | 8653 | |
| @@ -10192,8 +10187,8 @@ F: drivers/i2c/busses/i2c-cpm.c · 1533 unmodified lines | ||
| 10192 | 10187 | |
| 10193 | 10188 | FREESCALE IMX / MXC FEC DRIVER |
| 10194 | 10189 | M: Wei Fang <wei.fang@nxp.com> |
| 10190 | +R: Frank Li <frank.li@nxp.com> | |
| 10195 | 10191 | R: Shenwei Wang <shenwei.wang@nxp.com> |
| 10196 | −R: Clark Wang <xiaoning.wang@nxp.com> | |
| 10197 | 10192 | L: imx@lists.linux.dev |
| 10198 | 10193 | L: netdev@vger.kernel.org |
| 10199 | 10194 | S: Maintained |
| @@ -10505,7 +10500,7 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git · 305 unmodified lines | ||
| 10505 | 10500 | F: Documentation/trace/ftrace* |
| 10506 | 10501 | F: arch/*/*/*/*ftrace* |
| 10507 | 10502 | F: arch/*/*/*ftrace* |
| 10508 | −F: include/*/ftrace.h | |
| 10503 | +F: include/*/*ftrace* | |
| 10509 | 10504 | F: kernel/trace/fgraph.c |
| 10510 | 10505 | F: kernel/trace/ftrace* |
| 10511 | 10506 | F: samples/ftrace |
| @@ -12237,7 +12232,6 @@ IBM Power SRIOV Virtual NIC Device Driver · 1725 unmodified lines | ||
| 12237 | 12232 | M: Haren Myneni <haren@linux.ibm.com> |
| 12238 | 12233 | M: Rick Lindsley <ricklind@linux.ibm.com> |
| 12239 | 12234 | R: Nick Child <nnac123@linux.ibm.com> |
| 12240 | −R: Thomas Falcon <tlfalcon@linux.ibm.com> | |
| 12241 | 12235 | L: netdev@vger.kernel.org |
| 12242 | 12236 | S: Maintained |
| 12243 | 12237 | F: drivers/net/ethernet/ibm/ibmvnic.* |
| @@ -13963,7 +13957,7 @@ F: fs/smb/server/ · 1719 unmodified lines | ||
| 13963 | 13957 | |
| 13964 | 13958 | KERNEL UNIT TESTING FRAMEWORK (KUnit) |
| 13965 | 13959 | M: Brendan Higgins <brendan.higgins@linux.dev> |
| 13966 | −M: David Gow <davidgow@google.com> | |
| 13960 | +M: David Gow <david@davidgow.net> | |
| 13967 | 13961 | R: Rae Moar <raemoar63@gmail.com> |
| 13968 | 13962 | L: linux-kselftest@vger.kernel.org |
| 13969 | 13963 | L: kunit-dev@googlegroups.com |
| @@ -14432,9 +14426,9 @@ LANTIQ PEF2256 DRIVER · 462 unmodified lines | ||
| 14432 | 14426 | M: Herve Codina <herve.codina@bootlin.com> |
| 14433 | 14427 | S: Maintained |
| 14434 | 14428 | F: Documentation/devicetree/bindings/net/lantiq,pef2256.yaml |
| 14435 | −F: drivers/net/wan/framer/pef2256/ | |
| 14429 | +F: drivers/net/wan/framer/ | |
| 14436 | 14430 | F: drivers/pinctrl/pinctrl-pef2256.c |
| 14437 | −F: include/linux/framer/pef2256.h | |
| 14431 | +F: include/linux/framer/ | |
| 14438 | 14432 | |
| 14439 | 14433 | LASI 53c700 driver for PARISC |
| 14440 | 14434 | M: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com> |
| @@ -14783,7 +14777,7 @@ F: drivers/misc/lis3lv02d/ · 342 unmodified lines | ||
| 14783 | 14777 | F: drivers/platform/x86/hp/hp_accel.c |
| 14784 | 14778 | |
| 14785 | 14779 | LIST KUNIT TEST |
| 14786 | −M: David Gow <davidgow@google.com> | |
| 14780 | +M: David Gow <david@davidgow.net> | |
| 14787 | 14781 | L: linux-kselftest@vger.kernel.org |
| 14788 | 14782 | L: kunit-dev@googlegroups.com |
| 14789 | 14783 | S: Maintained |
| @@ -15396,10 +15390,8 @@ F: drivers/crypto/marvell/ · 606 unmodified lines | ||
| 15396 | 15390 | F: include/linux/soc/marvell/octeontx2/ |
| 15397 | 15391 | |
| 15398 | 15392 | MARVELL GIGABIT ETHERNET DRIVERS (skge/sky2) |
| 15399 | −M: Mirko Lindner <mlindner@marvell.com> | |
| 15400 | −M: Stephen Hemminger <stephen@networkplumber.org> | |
| 15401 | 15393 | L: netdev@vger.kernel.org |
| 15402 | −S: Odd fixes | |
| 15394 | +S: Orphan | |
| 15403 | 15395 | F: drivers/net/ethernet/marvell/sk* |
| 15404 | 15396 | |
| 15405 | 15397 | MARVELL LIBERTAS WIRELESS DRIVER |
| @@ -15496,7 +15488,6 @@ MARVELL OCTEONTX2 RVU ADMIN FUNCTION DRIVER · 90 unmodified lines | ||
| 15496 | 15488 | M: Sunil Goutham <sgoutham@marvell.com> |
| 15497 | 15489 | M: Linu Cherian <lcherian@marvell.com> |
| 15498 | 15490 | M: Geetha sowjanya <gakula@marvell.com> |
| 15499 | −M: Jerin Jacob <jerinj@marvell.com> | |
| 15500 | 15491 | M: hariprasad <hkelam@marvell.com> |
| 15501 | 15492 | M: Subbaraya Sundeep <sbhatta@marvell.com> |
| 15502 | 15493 | L: netdev@vger.kernel.org |
| @@ -15511,7 +15502,7 @@ S: Supported · 8 unmodified lines | ||
| 15511 | 15502 | F: drivers/perf/marvell_pem_pmu.c |
| 15512 | 15503 | |
| 15513 | 15504 | MARVELL PRESTERA ETHERNET SWITCH DRIVER |
| 15514 | −M: Taras Chornyi <taras.chornyi@plvision.eu> | |
| 15505 | +M: Elad Nachman <enachman@marvell.com> | |
| 15515 | 15506 | S: Supported |
| 15516 | 15507 | W: https://github.com/Marvell-switching/switchdev-prestera |
| 15517 | 15508 | F: drivers/net/ethernet/marvell/prestera/ |
| @@ -16185,7 +16176,6 @@ F: drivers/dma/mediatek/ · 667 unmodified lines | ||
| 16185 | 16176 | |
| 16186 | 16177 | MEDIATEK ETHERNET DRIVER |
| 16187 | 16178 | M: Felix Fietkau <nbd@nbd.name> |
| 16188 | −M: Sean Wang <sean.wang@mediatek.com> | |
| 16189 | 16179 | M: Lorenzo Bianconi <lorenzo@kernel.org> |
| 16190 | 16180 | L: netdev@vger.kernel.org |
| 16191 | 16181 | S: Maintained |
| @@ -16378,8 +16368,6 @@ F: include/soc/mediatek/smi.h · 186 unmodified lines | ||
| 16378 | 16368 | MEDIATEK SWITCH DRIVER |
| 16379 | 16369 | M: Chester A. Unal <chester.a.unal@arinc9.com> |
| 16380 | 16370 | M: Daniel Golle <daniel@makrotopia.org> |
| 16381 | −M: DENG Qingfang <dqfext@gmail.com> | |
| 16382 | −M: Sean Wang <sean.wang@mediatek.com> | |
| 16383 | 16371 | L: netdev@vger.kernel.org |
| 16384 | 16372 | S: Maintained |
| 16385 | 16373 | F: drivers/net/dsa/mt7530-mdio.c |
| @@ -16389,7 +16377,6 @@ F: net/dsa/tag_mtk.c · 3 unmodified lines | ||
| 16389 | 16377 | |
| 16390 | 16378 | MEDIATEK T7XX 5G WWAN MODEM DRIVER |
| 16391 | 16379 | M: Chandrashekar Devegowda <chandrashekar.devegowda@intel.com> |
| 16392 | −R: Chiranjeevi Rapolu <chiranjeevi.rapolu@linux.intel.com> | |
| 16393 | 16380 | R: Liu Haijun <haijun.liu@mediatek.com> |
| 16394 | 16381 | R: Ricardo Martinez <ricardo.martinez@linux.intel.com> |
| 16395 | 16382 | L: netdev@vger.kernel.org |
| @@ -16674,9 +16661,9 @@ F: mm/balloon.c · 278 unmodified lines | ||
| 16674 | 16661 | MEMORY MANAGEMENT - CORE |
| 16675 | 16662 | M: Andrew Morton <akpm@linux-foundation.org> |
| 16676 | 16663 | M: David Hildenbrand <david@kernel.org> |
| 16677 | −R: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | |
| 16664 | +R: Lorenzo Stoakes <ljs@kernel.org> | |
| 16678 | 16665 | R: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 16679 | −R: Vlastimil Babka <vbabka@suse.cz> | |
| 16666 | +R: Vlastimil Babka <vbabka@kernel.org> | |
| 16680 | 16667 | R: Mike Rapoport <rppt@kernel.org> |
| 16681 | 16668 | R: Suren Baghdasaryan <surenb@google.com> |
| 16682 | 16669 | R: Michal Hocko <mhocko@suse.com> |
| @@ -16804,9 +16791,9 @@ F: mm/workingset.c · 121 unmodified lines | ||
| 16804 | 16791 | MEMORY MANAGEMENT - MISC |
| 16805 | 16792 | M: Andrew Morton <akpm@linux-foundation.org> |
| 16806 | 16793 | M: David Hildenbrand <david@kernel.org> |
| 16807 | −R: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | |
| 16794 | +R: Lorenzo Stoakes <ljs@kernel.org> | |
| 16808 | 16795 | R: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 16809 | −R: Vlastimil Babka <vbabka@suse.cz> | |
| 16796 | +R: Vlastimil Babka <vbabka@kernel.org> | |
| 16810 | 16797 | R: Mike Rapoport <rppt@kernel.org> |
| 16811 | 16798 | R: Suren Baghdasaryan <surenb@google.com> |
| 16812 | 16799 | R: Michal Hocko <mhocko@suse.com> |
| @@ -16861,7 +16848,7 @@ F: mm/oom_kill.c · 48 unmodified lines | ||
| 16861 | 16848 | |
| 16862 | 16849 | MEMORY MANAGEMENT - PAGE ALLOCATOR |
| 16863 | 16850 | M: Andrew Morton <akpm@linux-foundation.org> |
| 16864 | −M: Vlastimil Babka <vbabka@suse.cz> | |
| 16851 | +M: Vlastimil Babka <vbabka@kernel.org> | |
| 16865 | 16852 | R: Suren Baghdasaryan <surenb@google.com> |
| 16866 | 16853 | R: Michal Hocko <mhocko@suse.com> |
| 16867 | 16854 | R: Brendan Jackman <jackmanb@google.com> |
| @@ -16895,7 +16882,7 @@ R: David Hildenbrand <david@kernel.org> · 27 unmodified lines | ||
| 16895 | 16882 | R: Michal Hocko <mhocko@kernel.org> |
| 16896 | 16883 | R: Qi Zheng <zhengqi.arch@bytedance.com> |
| 16897 | 16884 | R: Shakeel Butt <shakeel.butt@linux.dev> |
| 16898 | −R: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | |
| 16885 | +R: Lorenzo Stoakes <ljs@kernel.org> | |
| 16899 | 16886 | L: linux-mm@kvack.org |
| 16900 | 16887 | S: Maintained |
| 16901 | 16888 | F: mm/vmscan.c |
| @@ -16904,10 +16891,10 @@ F: mm/workingset.c · 2 unmodified lines | ||
| 16904 | 16891 | MEMORY MANAGEMENT - RMAP (REVERSE MAPPING) |
| 16905 | 16892 | M: Andrew Morton <akpm@linux-foundation.org> |
| 16906 | 16893 | M: David Hildenbrand <david@kernel.org> |
| 16907 | −M: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | |
| 16894 | +M: Lorenzo Stoakes <ljs@kernel.org> | |
| 16908 | 16895 | R: Rik van Riel <riel@surriel.com> |
| 16909 | 16896 | R: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 16910 | −R: Vlastimil Babka <vbabka@suse.cz> | |
| 16897 | +R: Vlastimil Babka <vbabka@kernel.org> | |
| 16911 | 16898 | R: Harry Yoo <harry.yoo@oracle.com> |
| 16912 | 16899 | R: Jann Horn <jannh@google.com> |
| 16913 | 16900 | L: linux-mm@kvack.org |
| @@ -16949,7 +16936,7 @@ F: mm/swapfile.c · 35 unmodified lines | ||
| 16949 | 16936 | MEMORY MANAGEMENT - THP (TRANSPARENT HUGE PAGE) |
| 16950 | 16937 | M: Andrew Morton <akpm@linux-foundation.org> |
| 16951 | 16938 | M: David Hildenbrand <david@kernel.org> |
| 16952 | −M: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | |
| 16939 | +M: Lorenzo Stoakes <ljs@kernel.org> | |
| 16953 | 16940 | R: Zi Yan <ziy@nvidia.com> |
| 16954 | 16941 | R: Baolin Wang <baolin.wang@linux.alibaba.com> |
| 16955 | 16942 | R: Liam R. Howlett <Liam.Howlett@oracle.com> |
| @@ -16989,7 +16976,7 @@ F: tools/testing/selftests/mm/uffd-*.[ch] · 33 unmodified lines | ||
| 16989 | 16976 | |
| 16990 | 16977 | MEMORY MANAGEMENT - RUST |
| 16991 | 16978 | M: Alice Ryhl <aliceryhl@google.com> |
| 16992 | −R: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | |
| 16979 | +R: Lorenzo Stoakes <ljs@kernel.org> | |
| 16993 | 16980 | R: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 16994 | 16981 | L: linux-mm@kvack.org |
| 16995 | 16982 | L: rust-for-linux@vger.kernel.org |
| @@ -17005,8 +16992,8 @@ F: rust/kernel/page.rs · 9 unmodified lines | ||
| 17005 | 16992 | MEMORY MAPPING |
| 17006 | 16993 | M: Andrew Morton <akpm@linux-foundation.org> |
| 17007 | 16994 | M: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 17008 | −M: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | |
| 17009 | −R: Vlastimil Babka <vbabka@suse.cz> | |
| 16995 | +M: Lorenzo Stoakes <ljs@kernel.org> | |
| 16996 | +R: Vlastimil Babka <vbabka@kernel.org> | |
| 17010 | 16997 | R: Jann Horn <jannh@google.com> |
| 17011 | 16998 | R: Pedro Falcato <pfalcato@suse.de> |
| 17012 | 16999 | L: linux-mm@kvack.org |
| @@ -17035,8 +17022,8 @@ MEMORY MAPPING - LOCKING · 22 unmodified lines | ||
| 17035 | 17022 | M: Andrew Morton <akpm@linux-foundation.org> |
| 17036 | 17023 | M: Suren Baghdasaryan <surenb@google.com> |
| 17037 | 17024 | M: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 17038 | −M: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | |
| 17039 | −R: Vlastimil Babka <vbabka@suse.cz> | |
| 17025 | +M: Lorenzo Stoakes <ljs@kernel.org> | |
| 17026 | +R: Vlastimil Babka <vbabka@kernel.org> | |
| 17040 | 17027 | R: Shakeel Butt <shakeel.butt@linux.dev> |
| 17041 | 17028 | L: linux-mm@kvack.org |
| 17042 | 17029 | S: Maintained |
| @@ -17050,9 +17037,9 @@ F: mm/mmap_lock.c · 7 unmodified lines | ||
| 17050 | 17037 | MEMORY MAPPING - MADVISE (MEMORY ADVICE) |
| 17051 | 17038 | M: Andrew Morton <akpm@linux-foundation.org> |
| 17052 | 17039 | M: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 17053 | −M: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | |
| 17040 | +M: Lorenzo Stoakes <ljs@kernel.org> | |
| 17054 | 17041 | M: David Hildenbrand <david@kernel.org> |
| 17055 | −R: Vlastimil Babka <vbabka@suse.cz> | |
| 17042 | +R: Vlastimil Babka <vbabka@kernel.org> | |
| 17056 | 17043 | R: Jann Horn <jannh@google.com> |
| 17057 | 17044 | L: linux-mm@kvack.org |
| 17058 | 17045 | S: Maintained |
| @@ -19247,8 +19234,6 @@ F: tools/objtool/ · 2188 unmodified lines | ||
| 19247 | 19234 | |
| 19248 | 19235 | OCELOT ETHERNET SWITCH DRIVER |
| 19249 | 19236 | M: Vladimir Oltean <vladimir.oltean@nxp.com> |
| 19250 | −M: Claudiu Manoil <claudiu.manoil@nxp.com> | |
| 19251 | −M: Alexandre Belloni <alexandre.belloni@bootlin.com> | |
| 19252 | 19237 | M: UNGLinuxDriver@microchip.com |
| 19253 | 19238 | L: netdev@vger.kernel.org |
| 19254 | 19239 | S: Supported |
| @@ -19834,7 +19819,6 @@ F: arch/*/boot/dts/ · 579 unmodified lines | ||
| 19834 | 19819 | F: include/dt-bindings/ |
| 19835 | 19820 | |
| 19836 | 19821 | OPENCOMPUTE PTP CLOCK DRIVER |
| 19837 | −M: Jonathan Lemon <jonathan.lemon@gmail.com> | |
| 19838 | 19822 | M: Vadim Fedorenko <vadim.fedorenko@linux.dev> |
| 19839 | 19823 | L: netdev@vger.kernel.org |
| 19840 | 19824 | S: Maintained |
| @@ -20142,9 +20126,8 @@ F: Documentation/devicetree/bindings/pci/marvell,armada-3700-pcie.yaml · 301 unmodified lines | ||
| 20142 | 20126 | F: drivers/pci/controller/pci-aardvark.c |
| 20143 | 20127 | |
| 20144 | 20128 | PCI DRIVER FOR ALTERA PCIE IP |
| 20145 | −M: Joyce Ooi <joyce.ooi@intel.com> | |
| 20146 | 20129 | L: linux-pci@vger.kernel.org |
| 20147 | −S: Supported | |
| 20130 | +S: Orphan | |
| 20148 | 20131 | F: Documentation/devicetree/bindings/pci/altr,pcie-root-port.yaml |
| 20149 | 20132 | F: drivers/pci/controller/pcie-altera.c |
| 20150 | 20133 | |
| @@ -20389,9 +20372,8 @@ S: Supported · 238 unmodified lines | ||
| 20389 | 20372 | F: Documentation/PCI/pci-error-recovery.rst |
| 20390 | 20373 | |
| 20391 | 20374 | PCI MSI DRIVER FOR ALTERA MSI IP |
| 20392 | −M: Joyce Ooi <joyce.ooi@intel.com> | |
| 20393 | 20375 | L: linux-pci@vger.kernel.org |
| 20394 | −S: Supported | |
| 20376 | +S: Orphan | |
| 20395 | 20377 | F: Documentation/devicetree/bindings/interrupt-controller/altr,msi-controller.yaml |
| 20396 | 20378 | F: drivers/pci/controller/pcie-altera-msi.c |
| 20397 | 20379 | |
| @@ -20529,7 +20511,7 @@ F: Documentation/devicetree/bindings/pci/hisilicon,kirin-pcie.yaml · 131 unmodified lines | ||
| 20529 | 20511 | F: drivers/pci/controller/dwc/pcie-kirin.c |
| 20530 | 20512 | |
| 20531 | 20513 | PCIE DRIVER FOR HISILICON STB |
| 20532 | −M: Shawn Guo <shawn.guo@linaro.org> | |
| 20514 | +M: Shawn Guo <shawnguo@kernel.org> | |
| 20533 | 20515 | L: linux-pci@vger.kernel.org |
| 20534 | 20516 | S: Maintained |
| 20535 | 20517 | F: Documentation/devicetree/bindings/pci/hisilicon-histb-pcie.txt |
| @@ -21478,9 +21460,8 @@ S: Supported · 942 unmodified lines | ||
| 21478 | 21460 | F: drivers/scsi/qedi/ |
| 21479 | 21461 | |
| 21480 | 21462 | QLOGIC QL4xxx ETHERNET DRIVER |
| 21481 | −M: Manish Chopra <manishc@marvell.com> | |
| 21482 | 21463 | L: netdev@vger.kernel.org |
| 21483 | −S: Maintained | |
| 21464 | +S: Orphan | |
| 21484 | 21465 | F: drivers/net/ethernet/qlogic/qed/ |
| 21485 | 21466 | F: drivers/net/ethernet/qlogic/qede/ |
| 21486 | 21467 | F: include/linux/qed/ |
| @@ -21715,7 +21696,7 @@ S: Maintained · 228 unmodified lines | ||
| 21715 | 21696 | F: drivers/net/ethernet/qualcomm/emac/ |
| 21716 | 21697 | |
| 21717 | 21698 | QUALCOMM ETHQOS ETHERNET DRIVER |
| 21718 | −M: Vinod Koul <vkoul@kernel.org> | |
| 21699 | +M: Mohd Ayaan Anwar <mohd.anwar@oss.qualcomm.com> | |
| 21719 | 21700 | L: netdev@vger.kernel.org |
| 21720 | 21701 | L: linux-arm-msm@vger.kernel.org |
| 21721 | 21702 | S: Maintained |
| @@ -21975,7 +21956,7 @@ F: drivers/media/radio/radio-tea5777.c · 253 unmodified lines | ||
| 21975 | 21956 | |
| 21976 | 21957 | RADOS BLOCK DEVICE (RBD) |
| 21977 | 21958 | M: Ilya Dryomov <idryomov@gmail.com> |
| 21978 | −R: Dongsheng Yang <dongsheng.yang@easystack.cn> | |
| 21959 | +R: Dongsheng Yang <dongsheng.yang@linux.dev> | |
| 21979 | 21960 | L: ceph-devel@vger.kernel.org |
| 21980 | 21961 | S: Supported |
| 21981 | 21962 | W: http://ceph.com/ |
| @@ -22304,6 +22285,16 @@ L: linux-wireless@vger.kernel.org · 322 unmodified lines | ||
| 22304 | 22285 | S: Orphan |
| 22305 | 22286 | F: drivers/net/wireless/rsi/ |
| 22306 | 22287 | |
| 22288 | +RELAY | |
| 22289 | +M: Andrew Morton <akpm@linux-foundation.org> | |
| 22290 | +M: Jens Axboe <axboe@kernel.dk> | |
| 22291 | +M: Jason Xing <kernelxing@tencent.com> | |
| 22292 | +L: linux-kernel@vger.kernel.org | |
| 22293 | +S: Maintained | |
| 22294 | +F: Documentation/filesystems/relay.rst | |
| 22295 | +F: include/linux/relay.h | |
| 22296 | +F: kernel/relay.c | |
| 22297 | + | |
| 22307 | 22298 | REGISTER MAP ABSTRACTION |
| 22308 | 22299 | M: Mark Brown <broonie@kernel.org> |
| 22309 | 22300 | L: linux-kernel@vger.kernel.org |
| @@ -23193,8 +23184,8 @@ K: \b(?i:rust)\b · 883 unmodified lines | ||
| 23193 | 23184 | |
| 23194 | 23185 | RUST [ALLOC] |
| 23195 | 23186 | M: Danilo Krummrich <dakr@kernel.org> |
| 23196 | −R: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> | |
| 23197 | −R: Vlastimil Babka <vbabka@suse.cz> | |
| 23187 | +R: Lorenzo Stoakes <ljs@kernel.org> | |
| 23188 | +R: Vlastimil Babka <vbabka@kernel.org> | |
| 23198 | 23189 | R: Liam R. Howlett <Liam.Howlett@oracle.com> |
| 23199 | 23190 | R: Uladzislau Rezki <urezki@gmail.com> |
| 23200 | 23191 | L: rust-for-linux@vger.kernel.org |
| @@ -24357,7 +24348,6 @@ F: Documentation/devicetree/bindings/interrupt-controller/kontron,sl28cpld-intc. · 1156 unmodified lines | ||
| 24357 | 24348 | F: Documentation/devicetree/bindings/pwm/kontron,sl28cpld-pwm.yaml |
| 24358 | 24349 | F: Documentation/devicetree/bindings/watchdog/kontron,sl28cpld-wdt.yaml |
| 24359 | 24350 | F: drivers/gpio/gpio-sl28cpld.c |
| 24360 | −F: drivers/hwmon/sa67mcu-hwmon.c | |
| 24361 | 24351 | F: drivers/hwmon/sl28cpld-hwmon.c |
| 24362 | 24352 | F: drivers/irqchip/irq-sl28cpld.c |
| 24363 | 24353 | F: drivers/pwm/pwm-sl28cpld.c |
| @@ -24370,12 +24360,13 @@ F: Documentation/devicetree/bindings/nvmem/layouts/kontron,sl28-vpd.yaml · 6 unmodified lines | ||
| 24370 | 24360 | F: drivers/nvmem/layouts/sl28vpd.c |
| 24371 | 24361 | |
| 24372 | 24362 | SLAB ALLOCATOR |
| 24373 | −M: Vlastimil Babka <vbabka@suse.cz> | |
| 24363 | +M: Vlastimil Babka <vbabka@kernel.org> | |
| 24364 | +M: Harry Yoo <harry.yoo@oracle.com> | |
| 24374 | 24365 | M: Andrew Morton <akpm@linux-foundation.org> |
| 24366 | +R: Hao Li <hao.li@linux.dev> | |
| 24375 | 24367 | R: Christoph Lameter <cl@gentwo.org> |
| 24376 | 24368 | R: David Rientjes <rientjes@google.com> |
| 24377 | 24369 | R: Roman Gushchin <roman.gushchin@linux.dev> |
| 24378 | −R: Harry Yoo <harry.yoo@oracle.com> | |
| 24379 | 24370 | L: linux-mm@kvack.org |
| 24380 | 24371 | S: Maintained |
| 24381 | 24372 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab.git |
| @@ -25786,6 +25777,7 @@ F: include/net/pkt_cls.h · 1404 unmodified lines | ||
| 25786 | 25777 | F: include/net/pkt_sched.h |
| 25787 | 25778 | F: include/net/sch_priv.h |
| 25788 | 25779 | F: include/net/tc_act/ |
| 25780 | +F: include/net/tc_wrapper.h | |
| 25789 | 25781 | F: include/uapi/linux/pkt_cls.h |
| 25790 | 25782 | F: include/uapi/linux/pkt_sched.h |
| 25791 | 25783 | F: include/uapi/linux/tc_act/ |
| @@ -26237,7 +26229,7 @@ F: drivers/media/radio/radio-raremono.c · 445 unmodified lines | ||
| 26237 | 26229 | |
| 26238 | 26230 | THERMAL |
| 26239 | 26231 | M: Rafael J. Wysocki <rafael@kernel.org> |
| 26240 | −M: Daniel Lezcano <daniel.lezcano@linaro.org> | |
| 26232 | +M: Daniel Lezcano <daniel.lezcano@kernel.org> | |
| 26241 | 26233 | R: Zhang Rui <rui.zhang@intel.com> |
| 26242 | 26234 | R: Lukasz Luba <lukasz.luba@arm.com> |
| 26243 | 26235 | L: linux-pm@vger.kernel.org |
| @@ -26267,7 +26259,7 @@ F: drivers/thermal/amlogic_thermal.c · 23 unmodified lines | ||
| 26267 | 26259 | |
| 26268 | 26260 | THERMAL/CPU_COOLING |
| 26269 | 26261 | M: Amit Daniel Kachhap <amit.kachhap@gmail.com> |
| 26270 | −M: Daniel Lezcano <daniel.lezcano@linaro.org> | |
| 26262 | +M: Daniel Lezcano <daniel.lezcano@kernel.org> | |
| 26271 | 26263 | M: Viresh Kumar <viresh.kumar@linaro.org> |
| 26272 | 26264 | R: Lukasz Luba <lukasz.luba@arm.com> |
| 26273 | 26265 | L: linux-pm@vger.kernel.org |
| @@ -29206,7 +29198,7 @@ K: zstd · 2932 unmodified lines | ||
| 29206 | 29198 | |
| 29207 | 29199 | ZSWAP COMPRESSED SWAP CACHING |
| 29208 | 29200 | M: Johannes Weiner <hannes@cmpxchg.org> |
| 29209 | −M: Yosry Ahmed <yosry.ahmed@linux.dev> | |
| 29201 | +M: Yosry Ahmed <yosry@kernel.org> | |
| 29210 | 29202 | M: Nhat Pham <nphamcs@gmail.com> |
| 29211 | 29203 | R: Chengming Zhou <chengming.zhou@linux.dev> |
| 29212 | 29204 | L: linux-mm@kvack.org |
| @@ -2,7 +2,7 @@ · 1 unmodified line | ||
| 2 | 2 | VERSION = 7 |
| 3 | 3 | PATCHLEVEL = 0 |
| 4 | 4 | SUBLEVEL = 0 |
| 5 | −EXTRAVERSION = -rc1 | |
| 5 | +EXTRAVERSION = -rc4 | |
| 6 | 6 | NAME = Baby Opossum Posse |
| 7 | 7 | |
| 8 | 8 | # *DOCUMENTATION* |
| @@ -476,6 +476,7 @@ KBUILD_USERLDFLAGS := $(USERLDFLAGS) · 467 unmodified lines | ||
| 476 | 476 | export rust_common_flags := --edition=2021 \ |
| 477 | 477 | -Zbinary_dep_depinfo=y \ |
| 478 | 478 | -Astable_features \ |
| 479 | + -Aunused_features \ | |
| 479 | 480 | -Dnon_ascii_idents \ |
| 480 | 481 | -Dunsafe_op_in_unsafe_fn \ |
| 481 | 482 | -Wmissing_docs \ |
| @@ -1113,6 +1114,9 @@ KBUILD_CFLAGS += -fno-builtin-wcslen · 631 unmodified lines | ||
| 1113 | 1114 | # change __FILE__ to the relative path to the source directory |
| 1114 | 1115 | ifdef building_out_of_srctree |
| 1115 | 1116 | KBUILD_CPPFLAGS += -fmacro-prefix-map=$(srcroot)/= |
| 1117 | +ifeq ($(call rustc-option-yn, --remap-path-scope=macro),y) | |
| 1118 | +KBUILD_RUSTFLAGS += --remap-path-prefix=$(srcroot)/= --remap-path-scope=macro | |
| 1119 | +endif | |
| 1116 | 1120 | endif |
| 1117 | 1121 | |
| 1118 | 1122 | # include additional Makefiles when needed |
| @@ -1497,13 +1501,13 @@ ifneq ($(wildcard $(resolve_btfids_O)),) · 378 unmodified lines | ||
| 1497 | 1501 | $(Q)$(MAKE) -sC $(srctree)/tools/bpf/resolve_btfids O=$(resolve_btfids_O) clean |
| 1498 | 1502 | endif |
| 1499 | 1503 | |
| 1500 | −PHONY += objtool_clean | |
| 1504 | +PHONY += objtool_clean objtool_mrproper | |
| 1501 | 1505 | |
| 1502 | 1506 | objtool_O = $(abspath $(objtree))/tools/objtool |
| 1503 | 1507 | |
| 1504 | −objtool_clean: | |
| 1508 | +objtool_clean objtool_mrproper: | |
| 1505 | 1509 | ifneq ($(wildcard $(objtool_O)),) |
| 1506 | − $(Q)$(MAKE) -sC $(abs_srctree)/tools/objtool O=$(objtool_O) srctree=$(abs_srctree) clean | |
| 1510 | + $(Q)$(MAKE) -sC $(abs_srctree)/tools/objtool O=$(objtool_O) srctree=$(abs_srctree) $(patsubst objtool_%,%,$@) | |
| 1507 | 1511 | endif |
| 1508 | 1512 | |
| 1509 | 1513 | tools/: FORCE |
| @@ -1686,7 +1690,7 @@ PHONY += $(mrproper-dirs) mrproper · 176 unmodified lines | ||
| 1686 | 1690 | $(mrproper-dirs): |
| 1687 | 1691 | $(Q)$(MAKE) $(clean)=$(patsubst _mrproper_%,%,$@) |
| 1688 | 1692 | |
| 1689 | −mrproper: clean $(mrproper-dirs) | |
| 1693 | +mrproper: clean objtool_mrproper $(mrproper-dirs) | |
| 1690 | 1694 | $(call cmd,rmfiles) |
| 1691 | 1695 | @find . $(RCS_FIND_IGNORE) \ |
| 1692 | 1696 | \( -name '*.rmeta' \) \ |
| @@ -71,6 +71,7 @@ SECTIONS · 70 unmodified lines | ||
| 71 | 71 | |
| 72 | 72 | STABS_DEBUG |
| 73 | 73 | DWARF_DEBUG |
| 74 | + MODINFO | |
| 74 | 75 | ELF_DETAILS |
| 75 | 76 | |
| 76 | 77 | DISCARDS |
| @@ -123,6 +123,7 @@ SECTIONS · 122 unmodified lines | ||
| 123 | 123 | _end = . ; |
| 124 | 124 | |
| 125 | 125 | STABS_DEBUG |
| 126 | + MODINFO | |
| 126 | 127 | ELF_DETAILS |
| 127 | 128 | DISCARDS |
| 128 | 129 | |
| @@ -21,6 +21,7 @@ SECTIONS · 20 unmodified lines | ||
| 21 | 21 | COMMON_DISCARDS |
| 22 | 22 | *(.ARM.exidx*) |
| 23 | 23 | *(.ARM.extab*) |
| 24 | + *(.modinfo) | |
| 24 | 25 | *(.note.*) |
| 25 | 26 | *(.rel.*) |
| 26 | 27 | *(.printk_index) |
| @@ -154,6 +154,7 @@ SECTIONS · 153 unmodified lines | ||
| 154 | 154 | |
| 155 | 155 | STABS_DEBUG |
| 156 | 156 | DWARF_DEBUG |
| 157 | + MODINFO | |
| 157 | 158 | ARM_DETAILS |
| 158 | 159 | |
| 159 | 160 | ARM_ASSERTS |
| @@ -153,6 +153,7 @@ SECTIONS · 152 unmodified lines | ||
| 153 | 153 | |
| 154 | 154 | STABS_DEBUG |
| 155 | 155 | DWARF_DEBUG |
| 156 | + MODINFO | |
| 156 | 157 | ARM_DETAILS |
| 157 | 158 | |
| 158 | 159 | ARM_ASSERTS |
| @@ -91,8 +91,9 @@ __XCHG_GEN(_mb) · 90 unmodified lines | ||
| 91 | 91 | #define __xchg_wrapper(sfx, ptr, x) \ |
| 92 | 92 | ({ \ |
| 93 | 93 | __typeof__(*(ptr)) __ret; \ |
| 94 | − __ret = (__typeof__(*(ptr))) \ | |
| 95 | − __arch_xchg##sfx((unsigned long)(x), (ptr), sizeof(*(ptr))); \ | |
| 94 | + __ret = (__force __typeof__(*(ptr))) \ | |
| 95 | + __arch_xchg##sfx((__force unsigned long)(x), (ptr), \ | |
| 96 | + sizeof(*(ptr))); \ | |
| 96 | 97 | __ret; \ |
| 97 | 98 | }) |
| 98 | 99 | |
| @@ -175,9 +176,10 @@ __CMPXCHG_GEN(_mb) · 76 unmodified lines | ||
| 175 | 176 | #define __cmpxchg_wrapper(sfx, ptr, o, n) \ |
| 176 | 177 | ({ \ |
| 177 | 178 | __typeof__(*(ptr)) __ret; \ |
| 178 | − __ret = (__typeof__(*(ptr))) \ | |
| 179 | − __cmpxchg##sfx((ptr), (unsigned long)(o), \ | |
| 180 | − (unsigned long)(n), sizeof(*(ptr))); \ | |
| 179 | + __ret = (__force __typeof__(*(ptr))) \ | |
| 180 | + __cmpxchg##sfx((ptr), (__force unsigned long)(o), \ | |
| 181 | + (__force unsigned long)(n), \ | |
| 182 | + sizeof(*(ptr))); \ | |
| 181 | 183 | __ret; \ |
| 182 | 184 | }) |
| 183 | 185 | |
| @@ -264,19 +264,33 @@ __iowrite64_copy(void __iomem *to, const void *from, size_t count) · 263 unmodified lines | ||
| 264 | 264 | typedef int (*ioremap_prot_hook_t)(phys_addr_t phys_addr, size_t size, |
| 265 | 265 | pgprot_t *prot); |
| 266 | 266 | int arm64_ioremap_prot_hook_register(const ioremap_prot_hook_t hook); |
| 267 | +void __iomem *__ioremap_prot(phys_addr_t phys, size_t size, pgprot_t prot); | |
| 267 | 268 | |
| 268 | −#define ioremap_prot ioremap_prot | |
| 269 | +static inline void __iomem *ioremap_prot(phys_addr_t phys, size_t size, | |
| 270 | + pgprot_t user_prot) | |
| 271 | +{ | |
| 272 | + pgprot_t prot; | |
| 273 | + ptdesc_t user_prot_val = pgprot_val(user_prot); | |
| 274 | + | |
| 275 | + if (WARN_ON_ONCE(!(user_prot_val & PTE_USER))) | |
| 276 | + return NULL; | |
| 269 | 277 | |
| 270 | −#define _PAGE_IOREMAP PROT_DEVICE_nGnRE | |
| 278 | + prot = __pgprot_modify(PAGE_KERNEL, PTE_ATTRINDX_MASK, | |
| 279 | + user_prot_val & PTE_ATTRINDX_MASK); | |
| 280 | + return __ioremap_prot(phys, size, prot); | |
| 281 | +} | |
| 282 | +#define ioremap_prot ioremap_prot | |
| 271 | 283 | |
| 284 | +#define ioremap(addr, size) \ | |
| 285 | + __ioremap_prot((addr), (size), __pgprot(PROT_DEVICE_nGnRE)) | |
| 272 | 286 | #define ioremap_wc(addr, size) \ |
| 273 | − ioremap_prot((addr), (size), __pgprot(PROT_NORMAL_NC)) | |
| 287 | + __ioremap_prot((addr), (size), __pgprot(PROT_NORMAL_NC)) | |
| 274 | 288 | #define ioremap_np(addr, size) \ |
| 275 | − ioremap_prot((addr), (size), __pgprot(PROT_DEVICE_nGnRnE)) | |
| 289 | + __ioremap_prot((addr), (size), __pgprot(PROT_DEVICE_nGnRnE)) | |
| 276 | 290 | |
| 277 | 291 | |
| 278 | 292 | #define ioremap_encrypted(addr, size) \ |
| 279 | − ioremap_prot((addr), (size), PAGE_KERNEL) | |
| 293 | + __ioremap_prot((addr), (size), PAGE_KERNEL) | |
| 280 | 294 | |
| 281 | 295 | /* |
| 282 | 296 | * io{read,write}{16,32,64}be() macros |
| @@ -297,7 +311,7 @@ static inline void __iomem *ioremap_cache(phys_addr_t addr, size_t size) · 14 unmodified lines | ||
| 297 | 311 | if (pfn_is_map_memory(__phys_to_pfn(addr))) |
| 298 | 312 | return (void __iomem *)__phys_to_virt(addr); |
| 299 | 313 | |
| 300 | − return ioremap_prot(addr, size, __pgprot(PROT_NORMAL)); | |
| 314 | + return __ioremap_prot(addr, size, __pgprot(PROT_NORMAL)); | |
| 301 | 315 | } |
| 302 | 316 | |
| 303 | 317 | /* |
| @@ -784,6 +784,9 @@ struct kvm_host_data { · 783 unmodified lines | ||
| 784 | 784 | /* Number of debug breakpoints/watchpoints for this CPU (minus 1) */ |
| 785 | 785 | unsigned int debug_brps; |
| 786 | 786 | unsigned int debug_wrps; |
| 787 | + | |
| 788 | + /* Last vgic_irq part of the AP list recorded in an LR */ | |
| 789 | + struct vgic_irq *last_lr_irq; | |
| 787 | 790 | }; |
| 788 | 791 | |
| 789 | 792 | struct kvm_host_psci_config { |
| @@ -1616,7 +1619,8 @@ void kvm_set_vm_id_reg(struct kvm *kvm, u32 reg, u64 val); · 826 unmodified lines | ||
| 1616 | 1619 | (kvm_has_feat((k), ID_AA64MMFR3_EL1, S1PIE, IMP)) |
| 1617 | 1620 | |
| 1618 | 1621 | #define kvm_has_s1poe(k) \ |
| 1619 | − (kvm_has_feat((k), ID_AA64MMFR3_EL1, S1POE, IMP)) | |
| 1622 | + (system_supports_poe() && \ | |
| 1623 | + kvm_has_feat((k), ID_AA64MMFR3_EL1, S1POE, IMP)) | |
| 1620 | 1624 | |
| 1621 | 1625 | #define kvm_has_ras(k) \ |
| 1622 | 1626 | (kvm_has_feat((k), ID_AA64PFR0_EL1, RAS, IMP)) |
| @@ -397,6 +397,8 @@ int kvm_vcpu_allocate_vncr_tlb(struct kvm_vcpu *vcpu); · 396 unmodified lines | ||
| 397 | 397 | int kvm_handle_vncr_abort(struct kvm_vcpu *vcpu); |
| 398 | 398 | void kvm_handle_s1e2_tlbi(struct kvm_vcpu *vcpu, u32 inst, u64 val); |
| 399 | 399 | |
| 400 | +u16 get_asid_by_regime(struct kvm_vcpu *vcpu, enum trans_regime regime); | |
| 401 | + | |
| 400 | 402 | #define vncr_fixmap(c) \ |
| 401 | 403 | ({ \ |
| 402 | 404 | u32 __c = (c); \ |
| @@ -50,11 +50,11 @@ · 49 unmodified lines | ||
| 50 | 50 | |
| 51 | 51 | #define _PAGE_DEFAULT (_PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL)) |
| 52 | 52 | |
| 53 | −#define _PAGE_KERNEL (PROT_NORMAL) | |
| 54 | −#define _PAGE_KERNEL_RO ((PROT_NORMAL & ~PTE_WRITE) | PTE_RDONLY) | |
| 55 | −#define _PAGE_KERNEL_ROX ((PROT_NORMAL & ~(PTE_WRITE | PTE_PXN)) | PTE_RDONLY) | |
| 56 | −#define _PAGE_KERNEL_EXEC (PROT_NORMAL & ~PTE_PXN) | |
| 57 | −#define _PAGE_KERNEL_EXEC_CONT ((PROT_NORMAL & ~PTE_PXN) | PTE_CONT) | |
| 53 | +#define _PAGE_KERNEL (PROT_NORMAL | PTE_DIRTY) | |
| 54 | +#define _PAGE_KERNEL_RO ((PROT_NORMAL & ~PTE_WRITE) | PTE_RDONLY | PTE_DIRTY) | |
| 55 | +#define _PAGE_KERNEL_ROX ((PROT_NORMAL & ~(PTE_WRITE | PTE_PXN)) | PTE_RDONLY | PTE_DIRTY) | |
| 56 | +#define _PAGE_KERNEL_EXEC ((PROT_NORMAL & ~PTE_PXN) | PTE_DIRTY) | |
| 57 | +#define _PAGE_KERNEL_EXEC_CONT ((PROT_NORMAL & ~PTE_PXN) | PTE_CONT | PTE_DIRTY) | |
| 58 | 58 | |
| 59 | 59 | #define _PAGE_SHARED (_PAGE_DEFAULT | PTE_USER | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_UXN | PTE_WRITE) |
| 60 | 60 | #define _PAGE_SHARED_EXEC (_PAGE_DEFAULT | PTE_USER | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_WRITE) |
| @@ -164,9 +164,6 @@ static inline bool __pure lpa2_is_enabled(void) · 103 unmodified lines | ||
| 164 | 164 | #define _PAGE_GCS (_PAGE_DEFAULT | PTE_NG | PTE_UXN | PTE_WRITE | PTE_USER) |
| 165 | 165 | #define _PAGE_GCS_RO (_PAGE_DEFAULT | PTE_NG | PTE_UXN | PTE_USER) |
| 166 | 166 | |
| 167 | −#define PAGE_GCS __pgprot(_PAGE_GCS) | |
| 168 | −#define PAGE_GCS_RO __pgprot(_PAGE_GCS_RO) | |
| 169 | − | |
| 170 | 167 | #define PIE_E0 ( \ |
| 171 | 168 | PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_GCS), PIE_GCS) | \ |
| 172 | 169 | PIRx_ELx_PERM_PREP(pte_pi_index(_PAGE_GCS_RO), PIE_R) | \ |
| @@ -2,6 +2,10 @@ · 1 unmodified line | ||
| 2 | 2 | #ifndef _ASM_RUNTIME_CONST_H |
| 3 | 3 | #define _ASM_RUNTIME_CONST_H |
| 4 | 4 | |
| 5 | +#ifdef MODULE | |
| 6 | + #error "Cannot use runtime-const infrastructure from modules" | |
| 7 | +#endif | |
| 8 | + | |
| 5 | 9 | #include <asm/cacheflush.h> |
| 6 | 10 | |
| 7 | 11 | /* Sigh. You can still run arm64 in BE mode */ |
| @@ -31,19 +31,11 @@ · 30 unmodified lines | ||
| 31 | 31 | */ |
| 32 | 32 | #define __TLBI_0(op, arg) asm (ARM64_ASM_PREAMBLE \ |
| 33 | 33 | "tlbi " #op "\n" \ |
| 34 | − ALTERNATIVE("nop\n nop", \ | |
| 35 | − "dsb ish\n tlbi " #op, \ | |
| 36 | − ARM64_WORKAROUND_REPEAT_TLBI, \ | |
| 37 | − CONFIG_ARM64_WORKAROUND_REPEAT_TLBI) \ | |
| 38 | 34 | : : ) |
| 39 | 35 | |
| 40 | 36 | #define __TLBI_1(op, arg) asm (ARM64_ASM_PREAMBLE \ |
| 41 | − "tlbi " #op ", %0\n" \ | |
| 42 | − ALTERNATIVE("nop\n nop", \ | |
| 43 | − "dsb ish\n tlbi " #op ", %0", \ | |
| 44 | − ARM64_WORKAROUND_REPEAT_TLBI, \ | |
| 45 | − CONFIG_ARM64_WORKAROUND_REPEAT_TLBI) \ | |
| 46 | − : : "r" (arg)) | |
| 37 | + "tlbi " #op ", %x0\n" \ | |
| 38 | + : : "rZ" (arg)) | |
| 47 | 39 | |
| 48 | 40 | #define __TLBI_N(op, arg, n, ...) __TLBI_##n(op, arg) |
| 49 | 41 | |
| @@ -181,6 +173,34 @@ static inline unsigned long get_trans_granule(void) · 131 unmodified lines | ||
| 181 | 173 | (__pages >> (5 * (scale) + 1)) - 1; \ |
| 182 | 174 | }) |
| 183 | 175 | |
| 176 | +#define __repeat_tlbi_sync(op, arg...) \ | |
| 177 | +do { \ | |
| 178 | + if (!alternative_has_cap_unlikely(ARM64_WORKAROUND_REPEAT_TLBI)) \ | |
| 179 | + break; \ | |
| 180 | + __tlbi(op, ##arg); \ | |
| 181 | + dsb(ish); \ | |
| 182 | +} while (0) | |
| 183 | + | |
| 184 | +/* | |
| 185 | + * Complete broadcast TLB maintenance issued by the host which invalidates | |
| 186 | + * stage 1 information in the host's own translation regime. | |
| 187 | + */ | |
| 188 | +static inline void __tlbi_sync_s1ish(void) | |
| 189 | +{ | |
| 190 | + dsb(ish); | |
| 191 | + __repeat_tlbi_sync(vale1is, 0); | |
| 192 | +} | |
| 193 | + | |
| 194 | +/* | |
| 195 | + * Complete broadcast TLB maintenance issued by hyp code which invalidates | |
| 196 | + * stage 1 translation information in any translation regime. | |
| 197 | + */ | |
| 198 | +static inline void __tlbi_sync_s1ish_hyp(void) | |
| 199 | +{ | |
| 200 | + dsb(ish); | |
| 201 | + __repeat_tlbi_sync(vale2is, 0); | |
| 202 | +} | |
| 203 | + | |
| 184 | 204 | /* |
| 185 | 205 | * TLB Invalidation |
| 186 | 206 | * ================ |
| @@ -279,7 +299,7 @@ static inline void flush_tlb_all(void) · 92 unmodified lines | ||
| 279 | 299 | { |
| 280 | 300 | dsb(ishst); |
| 281 | 301 | __tlbi(vmalle1is); |
| 282 | − dsb(ish); | |
| 302 | + __tlbi_sync_s1ish(); | |
| 283 | 303 | isb(); |
| 284 | 304 | } |
| 285 | 305 | |
| @@ -291,7 +311,7 @@ static inline void flush_tlb_mm(struct mm_struct *mm) · 5 unmodified lines | ||
| 291 | 311 | asid = __TLBI_VADDR(0, ASID(mm)); |
| 292 | 312 | __tlbi(aside1is, asid); |
| 293 | 313 | __tlbi_user(aside1is, asid); |
| 294 | − dsb(ish); | |
| 314 | + __tlbi_sync_s1ish(); | |
| 295 | 315 | mmu_notifier_arch_invalidate_secondary_tlbs(mm, 0, -1UL); |
| 296 | 316 | } |
| 297 | 317 | |
| @@ -345,20 +365,11 @@ static inline void flush_tlb_page(struct vm_area_struct *vma, · 47 unmodified lines | ||
| 345 | 365 | unsigned long uaddr) |
| 346 | 366 | { |
| 347 | 367 | flush_tlb_page_nosync(vma, uaddr); |
| 348 | − dsb(ish); | |
| 368 | + __tlbi_sync_s1ish(); | |
| 349 | 369 | } |
| 350 | 370 | |
| 351 | 371 | static inline bool arch_tlbbatch_should_defer(struct mm_struct *mm) |
| 352 | 372 | { |
| 353 | − /* | |
| 354 | − * TLB flush deferral is not required on systems which are affected by | |
| 355 | − * ARM64_WORKAROUND_REPEAT_TLBI, as __tlbi()/__tlbi_user() implementation | |
| 356 | − * will have two consecutive TLBI instructions with a dsb(ish) in between | |
| 357 | − * defeating the purpose (i.e save overall 'dsb ish' cost). | |
| 358 | − */ | |
| 359 | − if (alternative_has_cap_unlikely(ARM64_WORKAROUND_REPEAT_TLBI)) | |
| 360 | − return false; | |
| 361 | − | |
| 362 | 373 | return true; |
| 363 | 374 | } |
| 364 | 375 | |
| @@ -374,7 +385,7 @@ static inline bool arch_tlbbatch_should_defer(struct mm_struct *mm) · 9 unmodified lines | ||
| 374 | 385 | */ |
| 375 | 386 | static inline void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch) |
| 376 | 387 | { |
| 377 | − dsb(ish); | |
| 388 | + __tlbi_sync_s1ish(); | |
| 378 | 389 | } |
| 379 | 390 | |
| 380 | 391 | /* |
| @@ -509,7 +520,7 @@ static inline void __flush_tlb_range(struct vm_area_struct *vma, · 128 unmodified lines | ||
| 509 | 520 | { |
| 510 | 521 | __flush_tlb_range_nosync(vma->vm_mm, start, end, stride, |
| 511 | 522 | last_level, tlb_level); |
| 512 | − dsb(ish); | |
| 523 | + __tlbi_sync_s1ish(); | |
| 513 | 524 | } |
| 514 | 525 | |
| 515 | 526 | static inline void local_flush_tlb_contpte(struct vm_area_struct *vma, |
| @@ -557,7 +568,7 @@ static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end · 41 unmodified lines | ||
| 557 | 568 | dsb(ishst); |
| 558 | 569 | __flush_tlb_range_op(vaale1is, start, pages, stride, 0, |
| 559 | 570 | TLBI_TTL_UNKNOWN, false, lpa2_is_enabled()); |
| 560 | − dsb(ish); | |
| 571 | + __tlbi_sync_s1ish(); | |
| 561 | 572 | isb(); |
| 562 | 573 | } |
| 563 | 574 | |
| @@ -571,7 +582,7 @@ static inline void __flush_tlb_kernel_pgtable(unsigned long kaddr) · 7 unmodified lines | ||
| 571 | 582 | |
| 572 | 583 | dsb(ishst); |
| 573 | 584 | __tlbi(vaae1is, addr); |
| 574 | − dsb(ish); | |
| 585 | + __tlbi_sync_s1ish(); | |
| 575 | 586 | isb(); |
| 576 | 587 | } |
| 577 | 588 | |
| @@ -377,7 +377,7 @@ void __iomem *acpi_os_ioremap(acpi_physical_address phys, acpi_size size) · 376 unmodified lines | ||
| 377 | 377 | prot = __acpi_get_writethrough_mem_attribute(); |
| 378 | 378 | } |
| 379 | 379 | } |
| 380 | − return ioremap_prot(phys, size, prot); | |
| 380 | + return __ioremap_prot(phys, size, prot); | |
| 381 | 381 | } |
| 382 | 382 | |
| 383 | 383 | /* |
| @@ -2345,6 +2345,15 @@ static bool can_trap_icv_dir_el1(const struct arm64_cpu_capabilities *entry, · 2344 unmodified lines | ||
| 2345 | 2345 | !is_midr_in_range_list(has_vgic_v3)) |
| 2346 | 2346 | return false; |
| 2347 | 2347 | |
| 2348 | + /* | |
| 2349 | + * pKVM prevents late onlining of CPUs. This means that whatever | |
| 2350 | + * state the capability is in after deprivilege cannot be affected | |
| 2351 | + * by a new CPU booting -- this is garanteed to be a CPU we have | |
| 2352 | + * already seen, and the cap is therefore unchanged. | |
| 2353 | + */ | |
| 2354 | + if (system_capabilities_finalized() && is_protected_kvm_enabled()) | |
| 2355 | + return cpus_have_final_cap(ARM64_HAS_ICH_HCR_EL2_TDIR); | |
| 2356 | + | |
| 2348 | 2357 | if (is_kernel_in_hyp_mode()) |
| 2349 | 2358 | res.a1 = read_sysreg_s(SYS_ICH_VTR_EL2); |
| 2350 | 2359 | else |
| @@ -37,7 +37,7 @@ __do_compat_cache_op(unsigned long start, unsigned long end) · 36 unmodified lines | ||
| 37 | 37 | * We pick the reserved-ASID to minimise the impact. |
| 38 | 38 | */ |
| 39 | 39 | __tlbi(aside1is, __TLBI_VADDR(0, 0)); |
| 40 | − dsb(ish); | |
| 40 | + __tlbi_sync_s1ish(); | |
| 41 | 41 | } |
| 42 | 42 | |
| 43 | 43 | ret = caches_clean_inval_user_pou(start, start + chunk); |
| @@ -400,16 +400,25 @@ static inline · 399 unmodified lines | ||
| 400 | 400 | int counters_read_on_cpu(int cpu, smp_call_func_t func, u64 *val) |
| 401 | 401 | { |
| 402 | 402 | /* |
| 403 | − * Abort call on counterless CPU or when interrupts are | |
| 404 | − * disabled - can lead to deadlock in smp sync call. | |
| 403 | + * Abort call on counterless CPU. | |
| 405 | 404 | */ |
| 406 | 405 | if (!cpu_has_amu_feat(cpu)) |
| 407 | 406 | return -EOPNOTSUPP; |
| 408 | 407 | |
| 409 | − if (WARN_ON_ONCE(irqs_disabled())) | |
| 410 | − return -EPERM; | |
| 411 | − | |
| 412 | − smp_call_function_single(cpu, func, val, 1); | |
| 408 | + if (irqs_disabled()) { | |
| 409 | + /* | |
| 410 | + * When IRQs are disabled (tick path: sched_tick -> | |
| 411 | + * topology_scale_freq_tick or cppc_scale_freq_tick), only local | |
| 412 | + * CPU counter reads are allowed. Remote CPU counter read would | |
| 413 | + * require smp_call_function_single() which is unsafe with IRQs | |
| 414 | + * disabled. | |
| 415 | + */ | |
| 416 | + if (WARN_ON_ONCE(cpu != smp_processor_id())) | |
| 417 | + return -EPERM; | |
| 418 | + func(val); | |
| 419 | + } else { | |
| 420 | + smp_call_function_single(cpu, func, val, 1); | |
| 421 | + } | |
| 413 | 422 | |
| 414 | 423 | return 0; |
| 415 | 424 | } |
| @@ -349,6 +349,7 @@ SECTIONS · 348 unmodified lines | ||
| 349 | 349 | |
| 350 | 350 | STABS_DEBUG |
| 351 | 351 | DWARF_DEBUG |
| 352 | + MODINFO | |
| 352 | 353 | ELF_DETAILS |
| 353 | 354 | |
| 354 | 355 | HEAD_SYMBOLS |
| @@ -21,7 +21,6 @@ menuconfig KVM · 20 unmodified lines | ||
| 21 | 21 | bool "Kernel-based Virtual Machine (KVM) support" |
| 22 | 22 | select KVM_COMMON |
| 23 | 23 | select KVM_GENERIC_HARDWARE_ENABLING |
| 24 | − select KVM_GENERIC_MMU_NOTIFIER | |
| 25 | 24 | select HAVE_KVM_CPU_RELAX_INTERCEPT |
| 26 | 25 | select KVM_MMIO |
| 27 | 26 | select KVM_GENERIC_DIRTYLOG_READ_PROTECT |
| @@ -358,7 +358,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 357 unmodified lines | ||
| 358 | 358 | break; |
| 359 | 359 | case KVM_CAP_IOEVENTFD: |
| 360 | 360 | case KVM_CAP_USER_MEMORY: |
| 361 | − case KVM_CAP_SYNC_MMU: | |
| 362 | 361 | case KVM_CAP_DESTROY_MEMORY_REGION_WORKS: |
| 363 | 362 | case KVM_CAP_ONE_REG: |
| 364 | 363 | case KVM_CAP_ARM_PSCI: |
| @@ -540,31 +540,8 @@ static int walk_s1(struct kvm_vcpu *vcpu, struct s1_walk_info *wi, · 539 unmodified lines | ||
| 540 | 540 | wr->pa |= va & GENMASK_ULL(va_bottom - 1, 0); |
| 541 | 541 | |
| 542 | 542 | wr->nG = (wi->regime != TR_EL2) && (desc & PTE_NG); |
| 543 | − if (wr->nG) { | |
| 544 | − u64 asid_ttbr, tcr; | |
| 545 | − | |
| 546 | − switch (wi->regime) { | |
| 547 | − case TR_EL10: | |
| 548 | − tcr = vcpu_read_sys_reg(vcpu, TCR_EL1); | |
| 549 | − asid_ttbr = ((tcr & TCR_A1) ? | |
| 550 | − vcpu_read_sys_reg(vcpu, TTBR1_EL1) : | |
| 551 | − vcpu_read_sys_reg(vcpu, TTBR0_EL1)); | |
| 552 | − break; | |
| 553 | − case TR_EL20: | |
| 554 | − tcr = vcpu_read_sys_reg(vcpu, TCR_EL2); | |
| 555 | − asid_ttbr = ((tcr & TCR_A1) ? | |
| 556 | − vcpu_read_sys_reg(vcpu, TTBR1_EL2) : | |
| 557 | − vcpu_read_sys_reg(vcpu, TTBR0_EL2)); | |
| 558 | − break; | |
| 559 | − default: | |
| 560 | − BUG(); | |
| 561 | − } | |
| 562 | − | |
| 563 | − wr->asid = FIELD_GET(TTBR_ASID_MASK, asid_ttbr); | |
| 564 | − if (!kvm_has_feat_enum(vcpu->kvm, ID_AA64MMFR0_EL1, ASIDBITS, 16) || | |
| 565 | − !(tcr & TCR_ASID16)) | |
| 566 | − wr->asid &= GENMASK(7, 0); | |
| 567 | − } | |
| 543 | + if (wr->nG) | |
| 544 | + wr->asid = get_asid_by_regime(vcpu, wi->regime); | |
| 568 | 545 | |
| 569 | 546 | return 0; |
| 570 | 547 | |
| @@ -1527,8 +1504,6 @@ int __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) · 956 unmodified lines | ||
| 1527 | 1504 | fail = true; |
| 1528 | 1505 | } |
| 1529 | 1506 | |
| 1530 | − isb(); | |
| 1531 | − | |
| 1532 | 1507 | if (!fail) |
| 1533 | 1508 | par = read_sysreg_par(); |
| 1534 | 1509 | |
| @@ -29,7 +29,7 @@ · 28 unmodified lines | ||
| 29 | 29 | |
| 30 | 30 | #include "trace.h" |
| 31 | 31 | |
| 32 | −const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 32 | +const struct kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 33 | 33 | KVM_GENERIC_VM_STATS() |
| 34 | 34 | }; |
| 35 | 35 | |
| @@ -42,7 +42,7 @@ const struct kvm_stats_header kvm_vm_stats_header = { · 6 unmodified lines | ||
| 42 | 42 | sizeof(kvm_vm_stats_desc), |
| 43 | 43 | }; |
| 44 | 44 | |
| 45 | −const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 45 | +const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 46 | 46 | KVM_GENERIC_VCPU_STATS(), |
| 47 | 47 | STATS_DESC_COUNTER(VCPU, hvc_exit_stat), |
| 48 | 48 | STATS_DESC_COUNTER(VCPU, wfe_exit_stat), |
| @@ -518,7 +518,7 @@ static int host_stage2_adjust_range(u64 addr, struct kvm_mem_range *range) · 517 unmodified lines | ||
| 518 | 518 | granule = kvm_granule_size(level); |
| 519 | 519 | cur.start = ALIGN_DOWN(addr, granule); |
| 520 | 520 | cur.end = cur.start + granule; |
| 521 | − if (!range_included(&cur, range)) | |
| 521 | + if (!range_included(&cur, range) && level < KVM_PGTABLE_LAST_LEVEL) | |
| 522 | 522 | continue; |
| 523 | 523 | *range = cur; |
| 524 | 524 | return 0; |
| @@ -271,7 +271,7 @@ static void fixmap_clear_slot(struct hyp_fixmap_slot *slot) · 270 unmodified lines | ||
| 271 | 271 | */ |
| 272 | 272 | dsb(ishst); |
| 273 | 273 | __tlbi_level(vale2is, __TLBI_VADDR(addr, 0), level); |
| 274 | − dsb(ish); | |
| 274 | + __tlbi_sync_s1ish_hyp(); | |
| 275 | 275 | isb(); |
| 276 | 276 | } |
| 277 | 277 | |
| @@ -342,6 +342,7 @@ static void pkvm_init_features_from_host(struct pkvm_hyp_vm *hyp_vm, const struc · 341 unmodified lines | ||
| 342 | 342 | /* No restrictions for non-protected VMs. */ |
| 343 | 343 | if (!kvm_vm_is_protected(kvm)) { |
| 344 | 344 | hyp_vm->kvm.arch.flags = host_arch_flags; |
| 345 | + hyp_vm->kvm.arch.flags &= ~BIT_ULL(KVM_ARCH_FLAG_ID_REGS_INITIALIZED); | |
| 345 | 346 | |
| 346 | 347 | bitmap_copy(kvm->arch.vcpu_features, |
| 347 | 348 | host_kvm->arch.vcpu_features, |
| @@ -391,7 +392,7 @@ static void unpin_host_sve_state(struct pkvm_hyp_vcpu *hyp_vcpu) · 43 unmodified lines | ||
| 391 | 392 | if (!vcpu_has_feature(&hyp_vcpu->vcpu, KVM_ARM_VCPU_SVE)) |
| 392 | 393 | return; |
| 393 | 394 | |
| 394 | − sve_state = kern_hyp_va(hyp_vcpu->vcpu.arch.sve_state); | |
| 395 | + sve_state = hyp_vcpu->vcpu.arch.sve_state; | |
| 395 | 396 | hyp_unpin_shared_mem(sve_state, |
| 396 | 397 | sve_state + vcpu_sve_state_size(&hyp_vcpu->vcpu)); |
| 397 | 398 | } |
| @@ -471,6 +472,35 @@ err: · 73 unmodified lines | ||
| 471 | 472 | return ret; |
| 472 | 473 | } |
| 473 | 474 | |
| 475 | +static int vm_copy_id_regs(struct pkvm_hyp_vcpu *hyp_vcpu) | |
| 476 | +{ | |
| 477 | + struct pkvm_hyp_vm *hyp_vm = pkvm_hyp_vcpu_to_hyp_vm(hyp_vcpu); | |
| 478 | + const struct kvm *host_kvm = hyp_vm->host_kvm; | |
| 479 | + struct kvm *kvm = &hyp_vm->kvm; | |
| 480 | + | |
| 481 | + if (!test_bit(KVM_ARCH_FLAG_ID_REGS_INITIALIZED, &host_kvm->arch.flags)) | |
| 482 | + return -EINVAL; | |
| 483 | + | |
| 484 | + if (test_and_set_bit(KVM_ARCH_FLAG_ID_REGS_INITIALIZED, &kvm->arch.flags)) | |
| 485 | + return 0; | |
| 486 | + | |
| 487 | + memcpy(kvm->arch.id_regs, host_kvm->arch.id_regs, sizeof(kvm->arch.id_regs)); | |
| 488 | + | |
| 489 | + return 0; | |
| 490 | +} | |
| 491 | + | |
| 492 | +static int pkvm_vcpu_init_sysregs(struct pkvm_hyp_vcpu *hyp_vcpu) | |
| 493 | +{ | |
| 494 | + int ret = 0; | |
| 495 | + | |
| 496 | + if (pkvm_hyp_vcpu_is_protected(hyp_vcpu)) | |
| 497 | + kvm_init_pvm_id_regs(&hyp_vcpu->vcpu); | |
| 498 | + else | |
| 499 | + ret = vm_copy_id_regs(hyp_vcpu); | |
| 500 | + | |
| 501 | + return ret; | |
| 502 | +} | |
| 503 | + | |
| 474 | 504 | static int init_pkvm_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu, |
| 475 | 505 | struct pkvm_hyp_vm *hyp_vm, |
| 476 | 506 | struct kvm_vcpu *host_vcpu) |
| @@ -490,8 +520,9 @@ static int init_pkvm_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu, · 13 unmodified lines | ||
| 490 | 520 | hyp_vcpu->vcpu.arch.cflags = READ_ONCE(host_vcpu->arch.cflags); |
| 491 | 521 | hyp_vcpu->vcpu.arch.mp_state.mp_state = KVM_MP_STATE_STOPPED; |
| 492 | 522 | |
| 493 | − if (pkvm_hyp_vcpu_is_protected(hyp_vcpu)) | |
| 494 | − kvm_init_pvm_id_regs(&hyp_vcpu->vcpu); | |
| 523 | + ret = pkvm_vcpu_init_sysregs(hyp_vcpu); | |
| 524 | + if (ret) | |
| 525 | + goto done; | |
| 495 | 526 | |
| 496 | 527 | ret = pkvm_vcpu_init_traps(hyp_vcpu); |
| 497 | 528 | if (ret) |
| @@ -169,7 +169,7 @@ void __kvm_tlb_flush_vmid_ipa(struct kvm_s2_mmu *mmu, · 168 unmodified lines | ||
| 169 | 169 | */ |
| 170 | 170 | dsb(ish); |
| 171 | 171 | __tlbi(vmalle1is); |
| 172 | − dsb(ish); | |
| 172 | + __tlbi_sync_s1ish_hyp(); | |
| 173 | 173 | isb(); |
| 174 | 174 | |
| 175 | 175 | exit_vmid_context(&cxt); |
| @@ -226,7 +226,7 @@ void __kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, · 50 unmodified lines | ||
| 226 | 226 | |
| 227 | 227 | dsb(ish); |
| 228 | 228 | __tlbi(vmalle1is); |
| 229 | − dsb(ish); | |
| 229 | + __tlbi_sync_s1ish_hyp(); | |
| 230 | 230 | isb(); |
| 231 | 231 | |
| 232 | 232 | exit_vmid_context(&cxt); |
| @@ -240,7 +240,7 @@ void __kvm_tlb_flush_vmid(struct kvm_s2_mmu *mmu) · 7 unmodified lines | ||
| 240 | 240 | enter_vmid_context(mmu, &cxt, false); |
| 241 | 241 | |
| 242 | 242 | __tlbi(vmalls12e1is); |
| 243 | − dsb(ish); | |
| 243 | + __tlbi_sync_s1ish_hyp(); | |
| 244 | 244 | isb(); |
| 245 | 245 | |
| 246 | 246 | exit_vmid_context(&cxt); |
| @@ -266,5 +266,5 @@ void __kvm_flush_vm_context(void) · 19 unmodified lines | ||
| 266 | 266 | /* Same remark as in enter_vmid_context() */ |
| 267 | 267 | dsb(ish); |
| 268 | 268 | __tlbi(alle1is); |
| 269 | − dsb(ish); | |
| 269 | + __tlbi_sync_s1ish_hyp(); | |
| 270 | 270 | } |
| @@ -501,7 +501,7 @@ static int hyp_unmap_walker(const struct kvm_pgtable_visit_ctx *ctx, · 500 unmodified lines | ||
| 501 | 501 | *unmapped += granule; |
| 502 | 502 | } |
| 503 | 503 | |
| 504 | − dsb(ish); | |
| 504 | + __tlbi_sync_s1ish_hyp(); | |
| 505 | 505 | isb(); |
| 506 | 506 | mm_ops->put_page(ctx->ptep); |
| 507 | 507 | |
| @@ -115,7 +115,7 @@ void __kvm_tlb_flush_vmid_ipa(struct kvm_s2_mmu *mmu, · 114 unmodified lines | ||
| 115 | 115 | */ |
| 116 | 116 | dsb(ish); |
| 117 | 117 | __tlbi(vmalle1is); |
| 118 | − dsb(ish); | |
| 118 | + __tlbi_sync_s1ish_hyp(); | |
| 119 | 119 | isb(); |
| 120 | 120 | |
| 121 | 121 | exit_vmid_context(&cxt); |
| @@ -176,7 +176,7 @@ void __kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, · 54 unmodified lines | ||
| 176 | 176 | |
| 177 | 177 | dsb(ish); |
| 178 | 178 | __tlbi(vmalle1is); |
| 179 | − dsb(ish); | |
| 179 | + __tlbi_sync_s1ish_hyp(); | |
| 180 | 180 | isb(); |
| 181 | 181 | |
| 182 | 182 | exit_vmid_context(&cxt); |
| @@ -192,7 +192,7 @@ void __kvm_tlb_flush_vmid(struct kvm_s2_mmu *mmu) · 9 unmodified lines | ||
| 192 | 192 | enter_vmid_context(mmu, &cxt); |
| 193 | 193 | |
| 194 | 194 | __tlbi(vmalls12e1is); |
| 195 | − dsb(ish); | |
| 195 | + __tlbi_sync_s1ish_hyp(); | |
| 196 | 196 | isb(); |
| 197 | 197 | |
| 198 | 198 | exit_vmid_context(&cxt); |
| @@ -217,7 +217,7 @@ void __kvm_flush_vm_context(void) · 18 unmodified lines | ||
| 217 | 217 | { |
| 218 | 218 | dsb(ishst); |
| 219 | 219 | __tlbi(alle1is); |
| 220 | − dsb(ish); | |
| 220 | + __tlbi_sync_s1ish_hyp(); | |
| 221 | 221 | } |
| 222 | 222 | |
| 223 | 223 | /* |
| @@ -358,7 +358,7 @@ int __kvm_tlbi_s1e2(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding) · 134 unmodified lines | ||
| 358 | 358 | default: |
| 359 | 359 | ret = -EINVAL; |
| 360 | 360 | } |
| 361 | − dsb(ish); | |
| 361 | + __tlbi_sync_s1ish_hyp(); | |
| 362 | 362 | isb(); |
| 363 | 363 | |
| 364 | 364 | if (mmu) |
| @@ -1751,17 +1751,16 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, · 1750 unmodified lines | ||
| 1751 | 1751 | |
| 1752 | 1752 | force_pte = (max_map_size == PAGE_SIZE); |
| 1753 | 1753 | vma_pagesize = min_t(long, vma_pagesize, max_map_size); |
| 1754 | + vma_shift = __ffs(vma_pagesize); | |
| 1754 | 1755 | } |
| 1755 | 1756 | |
| 1756 | 1757 | /* |
| 1757 | − * Both the canonical IPA and fault IPA must be hugepage-aligned to | |
| 1758 | − * ensure we find the right PFN and lay down the mapping in the right | |
| 1759 | − * place. | |
| 1758 | + * Both the canonical IPA and fault IPA must be aligned to the | |
| 1759 | + * mapping size to ensure we find the right PFN and lay down the | |
| 1760 | + * mapping in the right place. | |
| 1760 | 1761 | */ |
| 1761 | − if (vma_pagesize == PMD_SIZE || vma_pagesize == PUD_SIZE) { | |
| 1762 | − fault_ipa &= ~(vma_pagesize - 1); | |
| 1763 | − ipa &= ~(vma_pagesize - 1); | |
| 1764 | − } | |
| 1762 | + fault_ipa = ALIGN_DOWN(fault_ipa, vma_pagesize); | |
| 1763 | + ipa = ALIGN_DOWN(ipa, vma_pagesize); | |
| 1765 | 1764 | |
| 1766 | 1765 | gfn = ipa >> PAGE_SHIFT; |
| 1767 | 1766 | mte_allowed = kvm_vma_mte_allowed(vma); |
| @@ -1839,10 +1838,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, · 71 unmodified lines | ||
| 1839 | 1838 | if (exec_fault && s2_force_noncacheable) |
| 1840 | 1839 | ret = -ENOEXEC; |
| 1841 | 1840 | |
| 1842 | − if (ret) { | |
| 1843 | − kvm_release_page_unused(page); | |
| 1844 | − return ret; | |
| 1845 | − } | |
| 1841 | + if (ret) | |
| 1842 | + goto out_put_page; | |
| 1846 | 1843 | |
| 1847 | 1844 | /* |
| 1848 | 1845 | * Guest performs atomic/exclusive operations on memory with unsupported |
| @@ -1852,7 +1849,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, · 3 unmodified lines | ||
| 1852 | 1849 | */ |
| 1853 | 1850 | if (esr_fsc_is_excl_atomic_fault(kvm_vcpu_get_esr(vcpu))) { |
| 1854 | 1851 | kvm_inject_dabt_excl_atomic(vcpu, kvm_vcpu_get_hfar(vcpu)); |
| 1855 | − return 1; | |
| 1852 | + ret = 1; | |
| 1853 | + goto out_put_page; | |
| 1856 | 1854 | } |
| 1857 | 1855 | |
| 1858 | 1856 | if (nested) |
| @@ -1938,6 +1936,10 @@ out_unlock: · 79 unmodified lines | ||
| 1938 | 1936 | mark_page_dirty_in_slot(kvm, memslot, gfn); |
| 1939 | 1937 | |
| 1940 | 1938 | return ret != -EAGAIN ? ret : 0; |
| 1939 | + | |
| 1940 | +out_put_page: | |
| 1941 | + kvm_release_page_unused(page); | |
| 1942 | + return ret; | |
| 1941 | 1943 | } |
| 1942 | 1944 | |
| 1943 | 1945 | /* Resolve the access fault by making the page young again. */ |
| @@ -152,31 +152,31 @@ static int get_ia_size(struct s2_walk_info *wi) · 151 unmodified lines | ||
| 152 | 152 | return 64 - wi->t0sz; |
| 153 | 153 | } |
| 154 | 154 | |
| 155 | −static int check_base_s2_limits(struct s2_walk_info *wi, | |
| 155 | +static int check_base_s2_limits(struct kvm_vcpu *vcpu, struct s2_walk_info *wi, | |
| 156 | 156 | int level, int input_size, int stride) |
| 157 | 157 | { |
| 158 | − int start_size, ia_size; | |
| 158 | + int start_size, pa_max; | |
| 159 | 159 | |
| 160 | − ia_size = get_ia_size(wi); | |
| 160 | + pa_max = kvm_get_pa_bits(vcpu->kvm); | |
| 161 | 161 | |
| 162 | 162 | /* Check translation limits */ |
| 163 | 163 | switch (BIT(wi->pgshift)) { |
| 164 | 164 | case SZ_64K: |
| 165 | − if (level == 0 || (level == 1 && ia_size <= 42)) | |
| 165 | + if (level == 0 || (level == 1 && pa_max <= 42)) | |
| 166 | 166 | return -EFAULT; |
| 167 | 167 | break; |
| 168 | 168 | case SZ_16K: |
| 169 | − if (level == 0 || (level == 1 && ia_size <= 40)) | |
| 169 | + if (level == 0 || (level == 1 && pa_max <= 40)) | |
| 170 | 170 | return -EFAULT; |
| 171 | 171 | break; |
| 172 | 172 | case SZ_4K: |
| 173 | − if (level < 0 || (level == 0 && ia_size <= 42)) | |
| 173 | + if (level < 0 || (level == 0 && pa_max <= 42)) | |
| 174 | 174 | return -EFAULT; |
| 175 | 175 | break; |
| 176 | 176 | } |
| 177 | 177 | |
| 178 | 178 | /* Check input size limits */ |
| 179 | − if (input_size > ia_size) | |
| 179 | + if (input_size > pa_max) | |
| 180 | 180 | return -EFAULT; |
| 181 | 181 | |
| 182 | 182 | /* Check number of entries in starting level table */ |
| @@ -269,16 +269,19 @@ static int walk_nested_s2_pgd(struct kvm_vcpu *vcpu, phys_addr_t ipa, · 86 unmodified lines | ||
| 269 | 269 | if (input_size > 48 || input_size < 25) |
| 270 | 270 | return -EFAULT; |
| 271 | 271 | |
| 272 | − ret = check_base_s2_limits(wi, level, input_size, stride); | |
| 273 | − if (WARN_ON(ret)) | |
| 272 | + ret = check_base_s2_limits(vcpu, wi, level, input_size, stride); | |
| 273 | + if (WARN_ON(ret)) { | |
| 274 | + out->esr = compute_fsc(0, ESR_ELx_FSC_FAULT); | |
| 274 | 275 | return ret; |
| 276 | + } | |
| 275 | 277 | |
| 276 | 278 | base_lower_bound = 3 + input_size - ((3 - level) * stride + |
| 277 | 279 | wi->pgshift); |
| 278 | 280 | base_addr = wi->baddr & GENMASK_ULL(47, base_lower_bound); |
| 279 | 281 | |
| 280 | 282 | if (check_output_size(wi, base_addr)) { |
| 281 | − out->esr = compute_fsc(level, ESR_ELx_FSC_ADDRSZ); | |
| 283 | + /* R_BFHQH */ | |
| 284 | + out->esr = compute_fsc(0, ESR_ELx_FSC_ADDRSZ); | |
| 282 | 285 | return 1; |
| 283 | 286 | } |
| 284 | 287 | |
| @@ -293,8 +296,10 @@ static int walk_nested_s2_pgd(struct kvm_vcpu *vcpu, phys_addr_t ipa, · 8 unmodified lines | ||
| 293 | 296 | |
| 294 | 297 | paddr = base_addr | index; |
| 295 | 298 | ret = read_guest_s2_desc(vcpu, paddr, &desc, wi); |
| 296 | − if (ret < 0) | |
| 299 | + if (ret < 0) { | |
| 300 | + out->esr = ESR_ELx_FSC_SEA_TTW(level); | |
| 297 | 301 | return ret; |
| 302 | + } | |
| 298 | 303 | |
| 299 | 304 | new_desc = desc; |
| 300 | 305 | |
| @@ -854,6 +859,33 @@ int kvm_inject_s2_fault(struct kvm_vcpu *vcpu, u64 esr_el2) · 553 unmodified lines | ||
| 854 | 859 | return kvm_inject_nested_sync(vcpu, esr_el2); |
| 855 | 860 | } |
| 856 | 861 | |
| 862 | +u16 get_asid_by_regime(struct kvm_vcpu *vcpu, enum trans_regime regime) | |
| 863 | +{ | |
| 864 | + enum vcpu_sysreg ttbr_elx; | |
| 865 | + u64 tcr; | |
| 866 | + u16 asid; | |
| 867 | + | |
| 868 | + switch (regime) { | |
| 869 | + case TR_EL10: | |
| 870 | + tcr = vcpu_read_sys_reg(vcpu, TCR_EL1); | |
| 871 | + ttbr_elx = (tcr & TCR_A1) ? TTBR1_EL1 : TTBR0_EL1; | |
| 872 | + break; | |
| 873 | + case TR_EL20: | |
| 874 | + tcr = vcpu_read_sys_reg(vcpu, TCR_EL2); | |
| 875 | + ttbr_elx = (tcr & TCR_A1) ? TTBR1_EL2 : TTBR0_EL2; | |
| 876 | + break; | |
| 877 | + default: | |
| 878 | + BUG(); | |
| 879 | + } | |
| 880 | + | |
| 881 | + asid = FIELD_GET(TTBRx_EL1_ASID, vcpu_read_sys_reg(vcpu, ttbr_elx)); | |
| 882 | + if (!kvm_has_feat_enum(vcpu->kvm, ID_AA64MMFR0_EL1, ASIDBITS, 16) || | |
| 883 | + !(tcr & TCR_ASID16)) | |
| 884 | + asid &= GENMASK(7, 0); | |
| 885 | + | |
| 886 | + return asid; | |
| 887 | +} | |
| 888 | + | |
| 857 | 889 | static void invalidate_vncr(struct vncr_tlb *vt) |
| 858 | 890 | { |
| 859 | 891 | vt->valid = false; |
| @@ -1154,9 +1186,6 @@ void kvm_arch_flush_shadow_all(struct kvm *kvm) · 294 unmodified lines | ||
| 1154 | 1186 | { |
| 1155 | 1187 | int i; |
| 1156 | 1188 | |
| 1157 | − if (!kvm->arch.nested_mmus_size) | |
| 1158 | − return; | |
| 1159 | − | |
| 1160 | 1189 | for (i = 0; i < kvm->arch.nested_mmus_size; i++) { |
| 1161 | 1190 | struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i]; |
| 1162 | 1191 | |
| @@ -1336,20 +1365,8 @@ static bool kvm_vncr_tlb_lookup(struct kvm_vcpu *vcpu) · 173 unmodified lines | ||
| 1336 | 1365 | if (read_vncr_el2(vcpu) != vt->gva) |
| 1337 | 1366 | return false; |
| 1338 | 1367 | |
| 1339 | − if (vt->wr.nG) { | |
| 1340 | − u64 tcr = vcpu_read_sys_reg(vcpu, TCR_EL2); | |
| 1341 | − u64 ttbr = ((tcr & TCR_A1) ? | |
| 1342 | − vcpu_read_sys_reg(vcpu, TTBR1_EL2) : | |
| 1343 | − vcpu_read_sys_reg(vcpu, TTBR0_EL2)); | |
| 1344 | − u16 asid; | |
| 1345 | − | |
| 1346 | − asid = FIELD_GET(TTBR_ASID_MASK, ttbr); | |
| 1347 | − if (!kvm_has_feat_enum(vcpu->kvm, ID_AA64MMFR0_EL1, ASIDBITS, 16) || | |
| 1348 | − !(tcr & TCR_ASID16)) | |
| 1349 | − asid &= GENMASK(7, 0); | |
| 1350 | − | |
| 1351 | − return asid == vt->wr.asid; | |
| 1352 | − } | |
| 1368 | + if (vt->wr.nG) | |
| 1369 | + return get_asid_by_regime(vcpu, TR_EL20) == vt->wr.asid; | |
| 1353 | 1370 | |
| 1354 | 1371 | return true; |
| 1355 | 1372 | } |
| @@ -1452,21 +1469,8 @@ static void kvm_map_l1_vncr(struct kvm_vcpu *vcpu) · 96 unmodified lines | ||
| 1452 | 1469 | if (read_vncr_el2(vcpu) != vt->gva) |
| 1453 | 1470 | return; |
| 1454 | 1471 | |
| 1455 | − if (vt->wr.nG) { | |
| 1456 | − u64 tcr = vcpu_read_sys_reg(vcpu, TCR_EL2); | |
| 1457 | − u64 ttbr = ((tcr & TCR_A1) ? | |
| 1458 | − vcpu_read_sys_reg(vcpu, TTBR1_EL2) : | |
| 1459 | − vcpu_read_sys_reg(vcpu, TTBR0_EL2)); | |
| 1460 | − u16 asid; | |
| 1461 | − | |
| 1462 | − asid = FIELD_GET(TTBR_ASID_MASK, ttbr); | |
| 1463 | − if (!kvm_has_feat_enum(vcpu->kvm, ID_AA64MMFR0_EL1, ASIDBITS, 16) || | |
| 1464 | − !(tcr & TCR_ASID16)) | |
| 1465 | − asid &= GENMASK(7, 0); | |
| 1466 | − | |
| 1467 | − if (asid != vt->wr.asid) | |
| 1468 | − return; | |
| 1469 | − } | |
| 1472 | + if (vt->wr.nG && get_asid_by_regime(vcpu, TR_EL20) != vt->wr.asid) | |
| 1473 | + return; | |
| 1470 | 1474 | |
| 1471 | 1475 | vt->cpu = smp_processor_id(); |
| 1472 | 1476 | |
| @@ -1816,6 +1816,9 @@ static u64 __kvm_read_sanitised_id_reg(const struct kvm_vcpu *vcpu, · 1815 unmodified lines | ||
| 1816 | 1816 | ID_AA64MMFR3_EL1_SCTLRX | |
| 1817 | 1817 | ID_AA64MMFR3_EL1_S1POE | |
| 1818 | 1818 | ID_AA64MMFR3_EL1_S1PIE; |
| 1819 | + | |
| 1820 | + if (!system_supports_poe()) | |
| 1821 | + val &= ~ID_AA64MMFR3_EL1_S1POE; | |
| 1819 | 1822 | break; |
| 1820 | 1823 | case SYS_ID_MMFR4_EL1: |
| 1821 | 1824 | val &= ~ID_MMFR4_EL1_CCIDX; |
| @@ -143,6 +143,21 @@ int kvm_vgic_create(struct kvm *kvm, u32 type) · 142 unmodified lines | ||
| 143 | 143 | kvm->arch.vgic.in_kernel = true; |
| 144 | 144 | kvm->arch.vgic.vgic_model = type; |
| 145 | 145 | kvm->arch.vgic.implementation_rev = KVM_VGIC_IMP_REV_LATEST; |
| 146 | + kvm->arch.vgic.vgic_dist_base = VGIC_ADDR_UNDEF; | |
| 147 | + | |
| 148 | + aa64pfr0 = kvm_read_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1) & ~ID_AA64PFR0_EL1_GIC; | |
| 149 | + pfr1 = kvm_read_vm_id_reg(kvm, SYS_ID_PFR1_EL1) & ~ID_PFR1_EL1_GIC; | |
| 150 | + | |
| 151 | + if (type == KVM_DEV_TYPE_ARM_VGIC_V2) { | |
| 152 | + kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF; | |
| 153 | + } else { | |
| 154 | + INIT_LIST_HEAD(&kvm->arch.vgic.rd_regions); | |
| 155 | + aa64pfr0 |= SYS_FIELD_PREP_ENUM(ID_AA64PFR0_EL1, GIC, IMP); | |
| 156 | + pfr1 |= SYS_FIELD_PREP_ENUM(ID_PFR1_EL1, GIC, GICv3); | |
| 157 | + } | |
| 158 | + | |
| 159 | + kvm_set_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1, aa64pfr0); | |
| 160 | + kvm_set_vm_id_reg(kvm, SYS_ID_PFR1_EL1, pfr1); | |
| 146 | 161 | |
| 147 | 162 | kvm_for_each_vcpu(i, vcpu, kvm) { |
| 148 | 163 | ret = vgic_allocate_private_irqs_locked(vcpu, type); |
| @@ -157,25 +172,10 @@ int kvm_vgic_create(struct kvm *kvm, u32 type) · 8 unmodified lines | ||
| 157 | 172 | vgic_cpu->private_irqs = NULL; |
| 158 | 173 | } |
| 159 | 174 | |
| 175 | + kvm->arch.vgic.vgic_model = 0; | |
| 160 | 176 | goto out_unlock; |
| 161 | 177 | } |
| 162 | 178 | |
| 163 | − kvm->arch.vgic.vgic_dist_base = VGIC_ADDR_UNDEF; | |
| 164 | − | |
| 165 | − aa64pfr0 = kvm_read_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1) & ~ID_AA64PFR0_EL1_GIC; | |
| 166 | − pfr1 = kvm_read_vm_id_reg(kvm, SYS_ID_PFR1_EL1) & ~ID_PFR1_EL1_GIC; | |
| 167 | − | |
| 168 | − if (type == KVM_DEV_TYPE_ARM_VGIC_V2) { | |
| 169 | − kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF; | |
| 170 | − } else { | |
| 171 | − INIT_LIST_HEAD(&kvm->arch.vgic.rd_regions); | |
| 172 | − aa64pfr0 |= SYS_FIELD_PREP_ENUM(ID_AA64PFR0_EL1, GIC, IMP); | |
| 173 | − pfr1 |= SYS_FIELD_PREP_ENUM(ID_PFR1_EL1, GIC, GICv3); | |
| 174 | − } | |
| 175 | − | |
| 176 | − kvm_set_vm_id_reg(kvm, SYS_ID_AA64PFR0_EL1, aa64pfr0); | |
| 177 | − kvm_set_vm_id_reg(kvm, SYS_ID_PFR1_EL1, pfr1); | |
| 178 | − | |
| 179 | 179 | if (type == KVM_DEV_TYPE_ARM_VGIC_V3) |
| 180 | 180 | kvm->arch.vgic.nassgicap = system_supports_direct_sgis(); |
| 181 | 181 | |
| @@ -115,7 +115,7 @@ void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu) · 114 unmodified lines | ||
| 115 | 115 | struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; |
| 116 | 116 | struct vgic_v2_cpu_if *cpuif = &vgic_cpu->vgic_v2; |
| 117 | 117 | u32 eoicount = FIELD_GET(GICH_HCR_EOICOUNT, cpuif->vgic_hcr); |
| 118 | − struct vgic_irq *irq; | |
| 118 | + struct vgic_irq *irq = *host_data_ptr(last_lr_irq); | |
| 119 | 119 | |
| 120 | 120 | DEBUG_SPINLOCK_BUG_ON(!irqs_disabled()); |
| 121 | 121 | |
| @@ -123,7 +123,7 @@ void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu) · 1 unmodified line | ||
| 123 | 123 | vgic_v2_fold_lr(vcpu, cpuif->vgic_lr[lr]); |
| 124 | 124 | |
| 125 | 125 | /* See the GICv3 equivalent for the EOIcount handling rationale */ |
| 126 | − list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) { | |
| 126 | + list_for_each_entry_continue(irq, &vgic_cpu->ap_list_head, ap_list) { | |
| 127 | 127 | u32 lr; |
| 128 | 128 | |
| 129 | 129 | if (!eoicount) { |
| @@ -148,7 +148,7 @@ void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu) · 147 unmodified lines | ||
| 148 | 148 | struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; |
| 149 | 149 | struct vgic_v3_cpu_if *cpuif = &vgic_cpu->vgic_v3; |
| 150 | 150 | u32 eoicount = FIELD_GET(ICH_HCR_EL2_EOIcount, cpuif->vgic_hcr); |
| 151 | − struct vgic_irq *irq; | |
| 151 | + struct vgic_irq *irq = *host_data_ptr(last_lr_irq); | |
| 152 | 152 | |
| 153 | 153 | DEBUG_SPINLOCK_BUG_ON(!irqs_disabled()); |
| 154 | 154 | |
| @@ -158,12 +158,12 @@ void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu) · 3 unmodified lines | ||
| 158 | 158 | /* |
| 159 | 159 | * EOIMode=0: use EOIcount to emulate deactivation. We are |
| 160 | 160 | * guaranteed to deactivate in reverse order of the activation, so |
| 161 | − * just pick one active interrupt after the other in the ap_list, | |
| 162 | − * and replay the deactivation as if the CPU was doing it. We also | |
| 163 | − * rely on priority drop to have taken place, and the list to be | |
| 164 | − * sorted by priority. | |
| 161 | + * just pick one active interrupt after the other in the tail part | |
| 162 | + * of the ap_list, past the LRs, and replay the deactivation as if | |
| 163 | + * the CPU was doing it. We also rely on priority drop to have taken | |
| 164 | + * place, and the list to be sorted by priority. | |
| 165 | 165 | */ |
| 166 | − list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) { | |
| 166 | + list_for_each_entry_continue(irq, &vgic_cpu->ap_list_head, ap_list) { | |
| 167 | 167 | u64 lr; |
| 168 | 168 | |
| 169 | 169 | /* |
| @@ -814,6 +814,9 @@ retry: · 813 unmodified lines | ||
| 814 | 814 | |
| 815 | 815 | static inline void vgic_fold_lr_state(struct kvm_vcpu *vcpu) |
| 816 | 816 | { |
| 817 | + if (!*host_data_ptr(last_lr_irq)) | |
| 818 | + return; | |
| 819 | + | |
| 817 | 820 | if (kvm_vgic_global_state.type == VGIC_V2) |
| 818 | 821 | vgic_v2_fold_lr_state(vcpu); |
| 819 | 822 | else |
| @@ -960,10 +963,13 @@ static void vgic_flush_lr_state(struct kvm_vcpu *vcpu) · 140 unmodified lines | ||
| 960 | 963 | if (irqs_outside_lrs(&als)) |
| 961 | 964 | vgic_sort_ap_list(vcpu); |
| 962 | 965 | |
| 966 | + *host_data_ptr(last_lr_irq) = NULL; | |
| 967 | + | |
| 963 | 968 | list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) { |
| 964 | 969 | scoped_guard(raw_spinlock, &irq->irq_lock) { |
| 965 | 970 | if (likely(vgic_target_oracle(irq) == vcpu)) { |
| 966 | 971 | vgic_populate_lr(vcpu, irq, count++); |
| 972 | + *host_data_ptr(last_lr_irq) = irq; | |
| 967 | 973 | } |
| 968 | 974 | } |
| 969 | 975 | |
| @@ -32,7 +32,11 @@ static inline unsigned long xloops_to_cycles(unsigned long xloops) · 31 unmodified lines | ||
| 32 | 32 | * Note that userspace cannot change the offset behind our back either, |
| 33 | 33 | * as the vcpu mutex is held as long as KVM_RUN is in progress. |
| 34 | 34 | */ |
| 35 | −#define __delay_cycles() __arch_counter_get_cntvct_stable() | |
| 35 | +static cycles_t notrace __delay_cycles(void) | |
| 36 | +{ | |
| 37 | + guard(preempt_notrace)(); | |
| 38 | + return __arch_counter_get_cntvct_stable(); | |
| 39 | +} | |
| 36 | 40 | |
| 37 | 41 | void __delay(unsigned long cycles) |
| 38 | 42 | { |
| @@ -599,6 +599,27 @@ void contpte_clear_young_dirty_ptes(struct vm_area_struct *vma, · 598 unmodified lines | ||
| 599 | 599 | } |
| 600 | 600 | EXPORT_SYMBOL_GPL(contpte_clear_young_dirty_ptes); |
| 601 | 601 | |
| 602 | +static bool contpte_all_subptes_match_access_flags(pte_t *ptep, pte_t entry) | |
| 603 | +{ | |
| 604 | + pte_t *cont_ptep = contpte_align_down(ptep); | |
| 605 | + /* | |
| 606 | + * PFNs differ per sub-PTE. Match only bits consumed by | |
| 607 | + * __ptep_set_access_flags(): AF, DIRTY and write permission. | |
| 608 | + */ | |
| 609 | + const pteval_t cmp_mask = PTE_RDONLY | PTE_AF | PTE_WRITE | PTE_DIRTY; | |
| 610 | + pteval_t entry_cmp = pte_val(entry) & cmp_mask; | |
| 611 | + int i; | |
| 612 | + | |
| 613 | + for (i = 0; i < CONT_PTES; i++) { | |
| 614 | + pteval_t pte_cmp = pte_val(__ptep_get(cont_ptep + i)) & cmp_mask; | |
| 615 | + | |
| 616 | + if (pte_cmp != entry_cmp) | |
| 617 | + return false; | |
| 618 | + } | |
| 619 | + | |
| 620 | + return true; | |
| 621 | +} | |
| 622 | + | |
| 602 | 623 | int contpte_ptep_set_access_flags(struct vm_area_struct *vma, |
| 603 | 624 | unsigned long addr, pte_t *ptep, |
| 604 | 625 | pte_t entry, int dirty) |
| @@ -608,13 +629,37 @@ int contpte_ptep_set_access_flags(struct vm_area_struct *vma, · 3 unmodified lines | ||
| 608 | 629 | int i; |
| 609 | 630 | |
| 610 | 631 | /* |
| 611 | − * Gather the access/dirty bits for the contiguous range. If nothing has | |
| 612 | − * changed, its a noop. | |
| 632 | + * Check whether all sub-PTEs in the CONT block already match the | |
| 633 | + * requested access flags/write permission, using raw per-PTE values | |
| 634 | + * rather than the gathered ptep_get() view. | |
| 635 | + * | |
| 636 | + * __ptep_set_access_flags() can update AF, dirty and write | |
| 637 | + * permission, but only to make the mapping more permissive. | |
| 638 | + * | |
| 639 | + * ptep_get() gathers AF/dirty state across the whole CONT block, | |
| 640 | + * which is correct for a CPU with FEAT_HAFDBS. But page-table | |
| 641 | + * walkers that evaluate each descriptor individually (e.g. a CPU | |
| 642 | + * without DBM support, or an SMMU without HTTU, or with HA/HD | |
| 643 | + * disabled in CD.TCR) can keep faulting on the target sub-PTE if | |
| 644 | + * only a sibling has been updated. Gathering can therefore cause | |
| 645 | + * false no-ops when only a sibling has been updated: | |
| 646 | + * - write faults: target still has PTE_RDONLY (needs PTE_RDONLY cleared) | |
| 647 | + * - read faults: target still lacks PTE_AF | |
| 648 | + * | |
| 649 | + * Per Arm ARM (DDI 0487) D8.7.1, any sub-PTE in a CONT range may | |
| 650 | + * become the effective cached translation, so all entries must have | |
| 651 | + * consistent attributes. Check the full CONT block before returning | |
| 652 | + * no-op, and when any sub-PTE mismatches, proceed to update the whole | |
| 653 | + * range. | |
| 613 | 654 | */ |
| 614 | − orig_pte = pte_mknoncont(ptep_get(ptep)); | |
| 615 | − if (pte_val(orig_pte) == pte_val(entry)) | |
| 655 | + if (contpte_all_subptes_match_access_flags(ptep, entry)) | |
| 616 | 656 | return 0; |
| 617 | 657 | |
| 658 | + /* | |
| 659 | + * Use raw target pte (not gathered) for write-bit unfold decision. | |
| 660 | + */ | |
| 661 | + orig_pte = pte_mknoncont(__ptep_get(ptep)); | |
| 662 | + | |
| 618 | 663 | /* |
| 619 | 664 | * We can fix up access/dirty bits without having to unfold the contig |
| 620 | 665 | * range. But if the write bit is changing, we must unfold. |
| @@ -14,8 +14,8 @@ int arm64_ioremap_prot_hook_register(ioremap_prot_hook_t hook) · 13 unmodified lines | ||
| 14 | 14 | return 0; |
| 15 | 15 | } |
| 16 | 16 | |
| 17 | −void __iomem *ioremap_prot(phys_addr_t phys_addr, size_t size, | |
| 18 | − pgprot_t pgprot) | |
| 17 | +void __iomem *__ioremap_prot(phys_addr_t phys_addr, size_t size, | |
| 18 | + pgprot_t pgprot) | |
| 19 | 19 | { |
| 20 | 20 | unsigned long last_addr = phys_addr + size - 1; |
| 21 | 21 | |
| @@ -39,7 +39,7 @@ void __iomem *ioremap_prot(phys_addr_t phys_addr, size_t size, · 17 unmodified lines | ||
| 39 | 39 | |
| 40 | 40 | return generic_ioremap_prot(phys_addr, size, pgprot); |
| 41 | 41 | } |
| 42 | −EXPORT_SYMBOL(ioremap_prot); | |
| 42 | +EXPORT_SYMBOL(__ioremap_prot); | |
| 43 | 43 | |
| 44 | 44 | /* |
| 45 | 45 | * Must be called after early_fixmap_init |
| @@ -34,6 +34,8 @@ static pgprot_t protection_map[16] __ro_after_init = { · 33 unmodified lines | ||
| 34 | 34 | [VM_SHARED | VM_EXEC | VM_WRITE | VM_READ] = PAGE_SHARED_EXEC |
| 35 | 35 | }; |
| 36 | 36 | |
| 37 | +static ptdesc_t gcs_page_prot __ro_after_init = _PAGE_GCS_RO; | |
| 38 | + | |
| 37 | 39 | /* |
| 38 | 40 | * You really shouldn't be using read() or write() on /dev/mem. This might go |
| 39 | 41 | * away in the future. |
| @@ -73,9 +75,11 @@ static int __init adjust_protection_map(void) · 33 unmodified lines | ||
| 73 | 75 | protection_map[VM_EXEC | VM_SHARED] = PAGE_EXECONLY; |
| 74 | 76 | } |
| 75 | 77 | |
| 76 | − if (lpa2_is_enabled()) | |
| 78 | + if (lpa2_is_enabled()) { | |
| 77 | 79 | for (int i = 0; i < ARRAY_SIZE(protection_map); i++) |
| 78 | 80 | pgprot_val(protection_map[i]) &= ~PTE_SHARED; |
| 81 | + gcs_page_prot &= ~PTE_SHARED; | |
| 82 | + } | |
| 79 | 83 | |
| 80 | 84 | return 0; |
| 81 | 85 | } |
| @@ -87,7 +91,11 @@ pgprot_t vm_get_page_prot(vm_flags_t vm_flags) · 5 unmodified lines | ||
| 87 | 91 | |
| 88 | 92 | /* Short circuit GCS to avoid bloating the table. */ |
| 89 | 93 | if (system_supports_gcs() && (vm_flags & VM_SHADOW_STACK)) { |
| 90 | − prot = _PAGE_GCS_RO; | |
| 94 | + /* Honour mprotect(PROT_NONE) on shadow stack mappings */ | |
| 95 | + if (vm_flags & VM_ACCESS_FLAGS) | |
| 96 | + prot = gcs_page_prot; | |
| 97 | + else | |
| 98 | + prot = pgprot_val(protection_map[VM_NONE]); | |
| 91 | 99 | } else { |
| 92 | 100 | prot = pgprot_val(protection_map[vm_flags & |
| 93 | 101 | (VM_READ|VM_WRITE|VM_EXEC|VM_SHARED)]); |
| @@ -2119,7 +2119,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) · 2118 unmodified lines | ||
| 2119 | 2119 | extable_offset = round_up(prog_size + PLT_TARGET_SIZE, extable_align); |
| 2120 | 2120 | image_size = extable_offset + extable_size; |
| 2121 | 2121 | ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr, |
| 2122 | − sizeof(u32), &header, &image_ptr, | |
| 2122 | + sizeof(u64), &header, &image_ptr, | |
| 2123 | 2123 | jit_fill_hole); |
| 2124 | 2124 | if (!ro_header) { |
| 2125 | 2125 | prog = orig_prog; |
| @@ -109,6 +109,7 @@ SECTIONS · 108 unmodified lines | ||
| 109 | 109 | |
| 110 | 110 | STABS_DEBUG |
| 111 | 111 | DWARF_DEBUG |
| 112 | + MODINFO | |
| 112 | 113 | ELF_DETAILS |
| 113 | 114 | |
| 114 | 115 | DISCARDS |
| @@ -62,6 +62,7 @@ SECTIONS · 61 unmodified lines | ||
| 62 | 62 | |
| 63 | 63 | STABS_DEBUG |
| 64 | 64 | DWARF_DEBUG |
| 65 | + MODINFO | |
| 65 | 66 | ELF_DETAILS |
| 66 | 67 | .hexagon.attributes 0 : { *(.hexagon.attributes) } |
| 67 | 68 | |
| @@ -147,6 +147,7 @@ SECTIONS · 146 unmodified lines | ||
| 147 | 147 | |
| 148 | 148 | STABS_DEBUG |
| 149 | 149 | DWARF_DEBUG |
| 150 | + MODINFO | |
| 150 | 151 | ELF_DETAILS |
| 151 | 152 | |
| 152 | 153 | #ifdef CONFIG_EFI_STUB |
| @@ -28,7 +28,6 @@ config KVM · 27 unmodified lines | ||
| 28 | 28 | select KVM_COMMON |
| 29 | 29 | select KVM_GENERIC_DIRTYLOG_READ_PROTECT |
| 30 | 30 | select KVM_GENERIC_HARDWARE_ENABLING |
| 31 | − select KVM_GENERIC_MMU_NOTIFIER | |
| 32 | 31 | select KVM_MMIO |
| 33 | 32 | select VIRT_XFER_TO_GUEST_WORK |
| 34 | 33 | select SCHED_INFO |
| @@ -14,7 +14,7 @@ · 13 unmodified lines | ||
| 14 | 14 | #define CREATE_TRACE_POINTS |
| 15 | 15 | #include "trace.h" |
| 16 | 16 | |
| 17 | −const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 17 | +const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 18 | 18 | KVM_GENERIC_VCPU_STATS(), |
| 19 | 19 | STATS_DESC_COUNTER(VCPU, int_exits), |
| 20 | 20 | STATS_DESC_COUNTER(VCPU, idle_exits), |
| @@ -10,7 +10,7 @@ · 9 unmodified lines | ||
| 10 | 10 | #include <asm/kvm_eiointc.h> |
| 11 | 11 | #include <asm/kvm_pch_pic.h> |
| 12 | 12 | |
| 13 | −const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 13 | +const struct kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 14 | 14 | KVM_GENERIC_VM_STATS(), |
| 15 | 15 | STATS_DESC_ICOUNTER(VM, pages), |
| 16 | 16 | STATS_DESC_ICOUNTER(VM, hugepages), |
| @@ -118,7 +118,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 101 unmodified lines | ||
| 118 | 118 | case KVM_CAP_ONE_REG: |
| 119 | 119 | case KVM_CAP_ENABLE_CAP: |
| 120 | 120 | case KVM_CAP_READONLY_MEM: |
| 121 | − case KVM_CAP_SYNC_MMU: | |
| 122 | 121 | case KVM_CAP_IMMEDIATE_EXIT: |
| 123 | 122 | case KVM_CAP_IOEVENTFD: |
| 124 | 123 | case KVM_CAP_MP_STATE: |
| @@ -85,6 +85,7 @@ SECTIONS { · 84 unmodified lines | ||
| 85 | 85 | _end = .; |
| 86 | 86 | |
| 87 | 87 | STABS_DEBUG |
| 88 | + MODINFO | |
| 88 | 89 | ELF_DETAILS |
| 89 | 90 | |
| 90 | 91 | /* Sections to be discarded */ |
| @@ -58,6 +58,7 @@ SECTIONS · 57 unmodified lines | ||
| 58 | 58 | _end = . ; |
| 59 | 59 | |
| 60 | 60 | STABS_DEBUG |
| 61 | + MODINFO | |
| 61 | 62 | ELF_DETAILS |
| 62 | 63 | |
| 63 | 64 | /* Sections to be discarded */ |
| @@ -51,6 +51,7 @@ __init_begin = .; · 50 unmodified lines | ||
| 51 | 51 | _end = . ; |
| 52 | 52 | |
| 53 | 53 | STABS_DEBUG |
| 54 | + MODINFO | |
| 54 | 55 | ELF_DETAILS |
| 55 | 56 | |
| 56 | 57 | /* Sections to be discarded */ |
| @@ -217,6 +217,7 @@ SECTIONS · 216 unmodified lines | ||
| 217 | 217 | |
| 218 | 218 | STABS_DEBUG |
| 219 | 219 | DWARF_DEBUG |
| 220 | + MODINFO | |
| 220 | 221 | ELF_DETAILS |
| 221 | 222 | |
| 222 | 223 | /* These must appear regardless of . */ |
| @@ -23,7 +23,6 @@ config KVM · 22 unmodified lines | ||
| 23 | 23 | select KVM_COMMON |
| 24 | 24 | select KVM_GENERIC_DIRTYLOG_READ_PROTECT |
| 25 | 25 | select KVM_MMIO |
| 26 | − select KVM_GENERIC_MMU_NOTIFIER | |
| 27 | 26 | select KVM_GENERIC_HARDWARE_ENABLING |
| 28 | 27 | select HAVE_KVM_READONLY_MEM |
| 29 | 28 | help |
| @@ -38,7 +38,7 @@ · 37 unmodified lines | ||
| 38 | 38 | #define VECTORSPACING 0x100 /* for EI/VI mode */ |
| 39 | 39 | #endif |
| 40 | 40 | |
| 41 | −const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 41 | +const struct kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 42 | 42 | KVM_GENERIC_VM_STATS() |
| 43 | 43 | }; |
| 44 | 44 | |
| @@ -51,7 +51,7 @@ const struct kvm_stats_header kvm_vm_stats_header = { · 6 unmodified lines | ||
| 51 | 51 | sizeof(kvm_vm_stats_desc), |
| 52 | 52 | }; |
| 53 | 53 | |
| 54 | −const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 54 | +const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 55 | 55 | KVM_GENERIC_VCPU_STATS(), |
| 56 | 56 | STATS_DESC_COUNTER(VCPU, wait_exits), |
| 57 | 57 | STATS_DESC_COUNTER(VCPU, cache_exits), |
| @@ -1035,7 +1035,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 977 unmodified lines | ||
| 1035 | 1035 | case KVM_CAP_ONE_REG: |
| 1036 | 1036 | case KVM_CAP_ENABLE_CAP: |
| 1037 | 1037 | case KVM_CAP_READONLY_MEM: |
| 1038 | − case KVM_CAP_SYNC_MMU: | |
| 1039 | 1038 | case KVM_CAP_IMMEDIATE_EXIT: |
| 1040 | 1039 | r = 1; |
| 1041 | 1040 | break; |
| @@ -57,6 +57,7 @@ SECTIONS · 56 unmodified lines | ||
| 57 | 57 | |
| 58 | 58 | STABS_DEBUG |
| 59 | 59 | DWARF_DEBUG |
| 60 | + MODINFO | |
| 60 | 61 | ELF_DETAILS |
| 61 | 62 | |
| 62 | 63 | DISCARDS |
| @@ -101,6 +101,7 @@ SECTIONS · 100 unmodified lines | ||
| 101 | 101 | /* Throw in the debugging sections */ |
| 102 | 102 | STABS_DEBUG |
| 103 | 103 | DWARF_DEBUG |
| 104 | + MODINFO | |
| 104 | 105 | ELF_DETAILS |
| 105 | 106 | |
| 106 | 107 | /* Sections to be discarded -- must be last */ |
| @@ -90,6 +90,7 @@ SECTIONS · 89 unmodified lines | ||
| 90 | 90 | /* Sections to be discarded */ |
| 91 | 91 | DISCARDS |
| 92 | 92 | /DISCARD/ : { |
| 93 | + *(.modinfo) | |
| 93 | 94 | #ifdef CONFIG_64BIT |
| 94 | 95 | /* temporary hack until binutils is fixed to not emit these |
| 95 | 96 | * for static binaries |
| @@ -85,7 +85,7 @@ extern void __update_cache(pte_t pte); · 84 unmodified lines | ||
| 85 | 85 | printk("%s:%d: bad pgd %08lx.\n", __FILE__, __LINE__, (unsigned long)pgd_val(e)) |
| 86 | 86 | |
| 87 | 87 | /* This is the size of the initially mapped kernel memory */ |
| 88 | −#if defined(CONFIG_64BIT) | |
| 88 | +#if defined(CONFIG_64BIT) || defined(CONFIG_KALLSYMS) | |
| 89 | 89 | #define KERNEL_INITIAL_ORDER 26 /* 1<<26 = 64MB */ |
| 90 | 90 | #else |
| 91 | 91 | #define KERNEL_INITIAL_ORDER 25 /* 1<<25 = 32MB */ |
| @@ -56,6 +56,7 @@ ENTRY(parisc_kernel_start) · 55 unmodified lines | ||
| 56 | 56 | |
| 57 | 57 | .import __bss_start,data |
| 58 | 58 | .import __bss_stop,data |
| 59 | + .import __end,data | |
| 59 | 60 | |
| 60 | 61 | load32 PA(__bss_start),%r3 |
| 61 | 62 | load32 PA(__bss_stop),%r4 |
| @@ -149,7 +150,11 @@ $cpu_ok: · 87 unmodified lines | ||
| 149 | 150 | * everything ... it will get remapped correctly later */ |
| 150 | 151 | ldo 0+_PAGE_KERNEL_RWX(%r0),%r3 /* Hardwired 0 phys addr start */ |
| 151 | 152 | load32 (1<<(KERNEL_INITIAL_ORDER-PAGE_SHIFT)),%r11 /* PFN count */ |
| 152 | − load32 PA(pg0),%r1 | |
| 153 | + load32 PA(_end),%r1 | |
| 154 | + SHRREG %r1,PAGE_SHIFT,%r1 /* %r1 is PFN count for _end symbol */ | |
| 155 | + cmpb,<<,n %r11,%r1,1f | |
| 156 | + copy %r1,%r11 /* %r1 PFN count smaller than %r11 */ | |
| 157 | +1: load32 PA(pg0),%r1 | |
| 153 | 158 | |
| 154 | 159 | $pgt_fill_loop: |
| 155 | 160 | STREGM %r3,ASM_PTE_ENTRY_SIZE(%r1) |
| @@ -120,14 +120,6 @@ void __init setup_arch(char **cmdline_p) · 119 unmodified lines | ||
| 120 | 120 | #endif |
| 121 | 121 | printk(KERN_CONT ".\n"); |
| 122 | 122 | |
| 123 | − /* | |
| 124 | − * Check if initial kernel page mappings are sufficient. | |
| 125 | − * panic early if not, else we may access kernel functions | |
| 126 | − * and variables which can't be reached. | |
| 127 | − */ | |
| 128 | − if (__pa((unsigned long) &_end) >= KERNEL_INITIAL_SIZE) | |
| 129 | − panic("KERNEL_INITIAL_ORDER too small!"); | |
| 130 | − | |
| 131 | 123 | #ifdef CONFIG_64BIT |
| 132 | 124 | if(parisc_narrow_firmware) { |
| 133 | 125 | printk(KERN_INFO "Kernel is using PDC in 32-bit mode.\n"); |
| @@ -279,6 +271,18 @@ void __init start_parisc(void) · 145 unmodified lines | ||
| 279 | 271 | int ret, cpunum; |
| 280 | 272 | struct pdc_coproc_cfg coproc_cfg; |
| 281 | 273 | |
| 274 | + /* | |
| 275 | + * Check if initial kernel page mapping is sufficient. | |
| 276 | + * Print warning if not, because we may access kernel functions and | |
| 277 | + * variables which can't be reached yet through the initial mappings. | |
| 278 | + * Note that the panic() and printk() functions are not functional | |
| 279 | + * yet, so we need to use direct iodc() firmware calls instead. | |
| 280 | + */ | |
| 281 | + const char warn1[] = "CRITICAL: Kernel may crash because " | |
| 282 | + "KERNEL_INITIAL_ORDER is too small.\n"; | |
| 283 | + if (__pa((unsigned long) &_end) >= KERNEL_INITIAL_SIZE) | |
| 284 | + pdc_iodc_print(warn1, sizeof(warn1) - 1); | |
| 285 | + | |
| 282 | 286 | /* check QEMU/SeaBIOS marker in PAGE0 */ |
| 283 | 287 | running_on_qemu = (memcmp(&PAGE0->pad0, "SeaBIOS", 8) == 0); |
| 284 | 288 | |
| @@ -165,6 +165,7 @@ SECTIONS · 164 unmodified lines | ||
| 165 | 165 | _end = . ; |
| 166 | 166 | |
| 167 | 167 | STABS_DEBUG |
| 168 | + MODINFO | |
| 168 | 169 | ELF_DETAILS |
| 169 | 170 | .note 0 : { *(.note) } |
| 170 | 171 | |
| @@ -573,8 +573,8 @@ config ARCH_USING_PATCHABLE_FUNCTION_ENTRY · 572 unmodified lines | ||
| 573 | 573 | depends on FUNCTION_TRACER && (PPC32 || PPC64_ELF_ABI_V2) |
| 574 | 574 | depends on $(cc-option,-fpatchable-function-entry=2) |
| 575 | 575 | def_bool y if PPC32 |
| 576 | − def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh $(CC) -mlittle-endian) if PPC64 && CPU_LITTLE_ENDIAN | |
| 577 | − def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh $(CC) -mbig-endian) if PPC64 && CPU_BIG_ENDIAN | |
| 576 | + def_bool $(success,$(srctree)/arch/powerpc/tools/check-fpatchable-function-entry.sh $(CC) $(CLANG_FLAGS) -mlittle-endian) if PPC64 && CPU_LITTLE_ENDIAN | |
| 577 | + def_bool $(success,$(srctree)/arch/powerpc/tools/check-fpatchable-function-entry.sh $(CC) -mbig-endian) if PPC64 && CPU_BIG_ENDIAN | |
| 578 | 578 | |
| 579 | 579 | config PPC_FTRACE_OUT_OF_LINE |
| 580 | 580 | def_bool PPC64 && ARCH_USING_PATCHABLE_FUNCTION_ENTRY |
| @@ -37,7 +37,7 @@ · 36 unmodified lines | ||
| 37 | 37 | }; |
| 38 | 38 | }; |
| 39 | 39 | |
| 40 | − memory { | |
| 40 | + memory@0 { | |
| 41 | 41 | device_type = "memory"; |
| 42 | 42 | reg = <0x00000000 0x8000000>; // 128MB at 0 |
| 43 | 43 | }; |
| @@ -1,156 +0,0 @@ | ||
| 1 | −/* T4240 Interlaken LAC Portal device tree stub with 24 portals. | |
| 2 | − * | |
| 3 | − * Copyright 2012 Freescale Semiconductor Inc. | |
| 4 | − * | |
| 5 | − * Redistribution and use in source and binary forms, with or without | |
| 6 | − * modification, are permitted provided that the following conditions are met: | |
| 7 | − * * Redistributions of source code must retain the above copyright | |
| 8 | − * notice, this list of conditions and the following disclaimer. | |
| 9 | − * * Redistributions in binary form must reproduce the above copyright | |
| 10 | − * notice, this list of conditions and the following disclaimer in the | |
| 11 | − * documentation and/or other materials provided with the distribution. | |
| 12 | − * * Neither the name of Freescale Semiconductor nor the | |
| 13 | − * names of its contributors may be used to endorse or promote products | |
| 14 | − * derived from this software without specific prior written permission. | |
| 15 | − * | |
| 16 | − * | |
| 17 | − * ALTERNATIVELY, this software may be distributed under the terms of the | |
| 18 | − * GNU General Public License ("GPL") as published by the Free Software | |
| 19 | − * Foundation, either version 2 of that License or (at your option) any | |
| 20 | − * later version. | |
| 21 | − * | |
| 22 | − * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY | |
| 23 | − * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED | |
| 24 | − * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE | |
| 25 | − * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY | |
| 26 | − * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES | |
| 27 | − * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; | |
| 28 | − * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND | |
| 29 | − * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | |
| 30 | − * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS | |
| 31 | − * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | |
| 32 | − */ | |
| 33 | − | |
| 34 | −#address-cells = <0x1>; | |
| 35 | −#size-cells = <0x1>; | |
| 36 | −compatible = "fsl,interlaken-lac-portals"; | |
| 37 | − | |
| 38 | −lportal0: lac-portal@0 { | |
| 39 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 40 | − reg = <0x0 0x1000>; | |
| 41 | −}; | |
| 42 | − | |
| 43 | −lportal1: lac-portal@1000 { | |
| 44 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 45 | − reg = <0x1000 0x1000>; | |
| 46 | −}; | |
| 47 | − | |
| 48 | −lportal2: lac-portal@2000 { | |
| 49 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 50 | − reg = <0x2000 0x1000>; | |
| 51 | −}; | |
| 52 | − | |
| 53 | −lportal3: lac-portal@3000 { | |
| 54 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 55 | − reg = <0x3000 0x1000>; | |
| 56 | −}; | |
| 57 | − | |
| 58 | −lportal4: lac-portal@4000 { | |
| 59 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 60 | − reg = <0x4000 0x1000>; | |
| 61 | −}; | |
| 62 | − | |
| 63 | −lportal5: lac-portal@5000 { | |
| 64 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 65 | − reg = <0x5000 0x1000>; | |
| 66 | −}; | |
| 67 | − | |
| 68 | −lportal6: lac-portal@6000 { | |
| 69 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 70 | − reg = <0x6000 0x1000>; | |
| 71 | −}; | |
| 72 | − | |
| 73 | −lportal7: lac-portal@7000 { | |
| 74 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 75 | − reg = <0x7000 0x1000>; | |
| 76 | −}; | |
| 77 | − | |
| 78 | −lportal8: lac-portal@8000 { | |
| 79 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 80 | − reg = <0x8000 0x1000>; | |
| 81 | −}; | |
| 82 | − | |
| 83 | −lportal9: lac-portal@9000 { | |
| 84 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 85 | − reg = <0x9000 0x1000>; | |
| 86 | −}; | |
| 87 | − | |
| 88 | −lportal10: lac-portal@A000 { | |
| 89 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 90 | − reg = <0xA000 0x1000>; | |
| 91 | −}; | |
| 92 | − | |
| 93 | −lportal11: lac-portal@B000 { | |
| 94 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 95 | − reg = <0xB000 0x1000>; | |
| 96 | −}; | |
| 97 | − | |
| 98 | −lportal12: lac-portal@C000 { | |
| 99 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 100 | − reg = <0xC000 0x1000>; | |
| 101 | −}; | |
| 102 | − | |
| 103 | −lportal13: lac-portal@D000 { | |
| 104 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 105 | − reg = <0xD000 0x1000>; | |
| 106 | −}; | |
| 107 | − | |
| 108 | −lportal14: lac-portal@E000 { | |
| 109 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 110 | − reg = <0xE000 0x1000>; | |
| 111 | −}; | |
| 112 | − | |
| 113 | −lportal15: lac-portal@F000 { | |
| 114 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 115 | − reg = <0xF000 0x1000>; | |
| 116 | −}; | |
| 117 | − | |
| 118 | −lportal16: lac-portal@10000 { | |
| 119 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 120 | − reg = <0x10000 0x1000>; | |
| 121 | −}; | |
| 122 | − | |
| 123 | −lportal17: lac-portal@11000 { | |
| 124 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 125 | − reg = <0x11000 0x1000>; | |
| 126 | −}; | |
| 127 | − | |
| 128 | −lportal18: lac-portal@1200 { | |
| 129 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 130 | − reg = <0x12000 0x1000>; | |
| 131 | −}; | |
| 132 | − | |
| 133 | −lportal19: lac-portal@13000 { | |
| 134 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 135 | − reg = <0x13000 0x1000>; | |
| 136 | −}; | |
| 137 | − | |
| 138 | −lportal20: lac-portal@14000 { | |
| 139 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 140 | − reg = <0x14000 0x1000>; | |
| 141 | −}; | |
| 142 | − | |
| 143 | −lportal21: lac-portal@15000 { | |
| 144 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 145 | − reg = <0x15000 0x1000>; | |
| 146 | −}; | |
| 147 | − | |
| 148 | −lportal22: lac-portal@16000 { | |
| 149 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 150 | − reg = <0x16000 0x1000>; | |
| 151 | −}; | |
| 152 | − | |
| 153 | −lportal23: lac-portal@17000 { | |
| 154 | − compatible = "fsl,interlaken-lac-portal-v1.0"; | |
| 155 | − reg = <0x17000 0x1000>; | |
| 156 | −}; | |
| @@ -1,45 +0,0 @@ | ||
| 1 | −/* | |
| 2 | − * T4 Interlaken Look-aside Controller (LAC) device tree stub | |
| 3 | − * | |
| 4 | − * Copyright 2012 Freescale Semiconductor Inc. | |
| 5 | − * | |
| 6 | − * Redistribution and use in source and binary forms, with or without | |
| 7 | − * modification, are permitted provided that the following conditions are met: | |
| 8 | − * * Redistributions of source code must retain the above copyright | |
| 9 | − * notice, this list of conditions and the following disclaimer. | |
| 10 | − * * Redistributions in binary form must reproduce the above copyright | |
| 11 | − * notice, this list of conditions and the following disclaimer in the | |
| 12 | − * documentation and/or other materials provided with the distribution. | |
| 13 | − * * Neither the name of Freescale Semiconductor nor the | |
| 14 | − * names of its contributors may be used to endorse or promote products | |
| 15 | − * derived from this software without specific prior written permission. | |
| 16 | − * | |
| 17 | − * | |
| 18 | − * ALTERNATIVELY, this software may be distributed under the terms of the | |
| 19 | − * GNU General Public License ("GPL") as published by the Free Software | |
| 20 | − * Foundation, either version 2 of that License or (at your option) any | |
| 21 | − * later version. | |
| 22 | − * | |
| 23 | − * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY | |
| 24 | − * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED | |
| 25 | − * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE | |
| 26 | − * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY | |
| 27 | − * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES | |
| 28 | − * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; | |
| 29 | − * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND | |
| 30 | − * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | |
| 31 | − * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS | |
| 32 | − * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | |
| 33 | − */ | |
| 34 | − | |
| 35 | −lac: lac@229000 { | |
| 36 | − compatible = "fsl,interlaken-lac"; | |
| 37 | − reg = <0x229000 0x1000>; | |
| 38 | − interrupts = <16 2 1 18>; | |
| 39 | −}; | |
| 40 | − | |
| 41 | −lac-hv@228000 { | |
| 42 | − compatible = "fsl,interlaken-lac-hv"; | |
| 43 | − reg = <0x228000 0x1000>; | |
| 44 | − fsl,non-hv-node = <&lac>; | |
| 45 | −}; | |
| @@ -1,43 +0,0 @@ | ||
| 1 | −/* | |
| 2 | − * PQ3 MPIC Message (Group B) device tree stub [ controller @ offset 0x42400 ] | |
| 3 | − * | |
| 4 | − * Copyright 2012 Freescale Semiconductor Inc. | |
| 5 | − * | |
| 6 | − * Redistribution and use in source and binary forms, with or without | |
| 7 | − * modification, are permitted provided that the following conditions are met: | |
| 8 | − * * Redistributions of source code must retain the above copyright | |
| 9 | − * notice, this list of conditions and the following disclaimer. | |
| 10 | − * * Redistributions in binary form must reproduce the above copyright | |
| 11 | − * notice, this list of conditions and the following disclaimer in the | |
| 12 | − * documentation and/or other materials provided with the distribution. | |
| 13 | − * * Neither the name of Freescale Semiconductor nor the | |
| 14 | − * names of its contributors may be used to endorse or promote products | |
| 15 | − * derived from this software without specific prior written permission. | |
| 16 | − * | |
| 17 | − * | |
| 18 | − * ALTERNATIVELY, this software may be distributed under the terms of the | |
| 19 | − * GNU General Public License ("GPL") as published by the Free Software | |
| 20 | − * Foundation, either version 2 of that License or (at your option) any | |
| 21 | − * later version. | |
| 22 | − * | |
| 23 | − * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY | |
| 24 | − * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED | |
| 25 | − * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE | |
| 26 | − * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY | |
| 27 | − * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES | |
| 28 | − * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; | |
| 29 | − * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND | |
| 30 | − * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | |
| 31 | − * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS | |
| 32 | − * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | |
| 33 | − */ | |
| 34 | − | |
| 35 | −message@42400 { | |
| 36 | − compatible = "fsl,mpic-v3.1-msgr"; | |
| 37 | − reg = <0x42400 0x200>; | |
| 38 | − interrupts = < | |
| 39 | − 0xb4 2 0 0 | |
| 40 | − 0xb5 2 0 0 | |
| 41 | − 0xb6 2 0 0 | |
| 42 | − 0xb7 2 0 0>; | |
| 43 | −}; | |
| @@ -1,80 +0,0 @@ | ||
| 1 | −/* | |
| 2 | − * QorIQ FMan v3 1g port #1 device tree stub [ controller @ offset 0x400000 ] | |
| 3 | − * | |
| 4 | − * Copyright 2012 - 2015 Freescale Semiconductor Inc. | |
| 5 | − * | |
| 6 | − * Redistribution and use in source and binary forms, with or without | |
| 7 | − * modification, are permitted provided that the following conditions are met: | |
| 8 | − * * Redistributions of source code must retain the above copyright | |
| 9 | − * notice, this list of conditions and the following disclaimer. | |
| 10 | − * * Redistributions in binary form must reproduce the above copyright | |
| 11 | − * notice, this list of conditions and the following disclaimer in the | |
| 12 | − * documentation and/or other materials provided with the distribution. | |
| 13 | − * * Neither the name of Freescale Semiconductor nor the | |
| 14 | − * names of its contributors may be used to endorse or promote products | |
| 15 | − * derived from this software without specific prior written permission. | |
| 16 | − * | |
| 17 | − * | |
| 18 | − * ALTERNATIVELY, this software may be distributed under the terms of the | |
| 19 | − * GNU General Public License ("GPL") as published by the Free Software | |
| 20 | − * Foundation, either version 2 of that License or (at your option) any | |
| 21 | − * later version. | |
| 22 | − * | |
| 23 | − * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor ``AS IS'' AND ANY | |
| 24 | − * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED | |
| 25 | − * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE | |
| 26 | − * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY | |
| 27 | − * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES | |
| 28 | − * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; | |
| 29 | − * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND | |
| 30 | − * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | |
| 31 | − * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS | |
| 32 | − * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | |
| 33 | − */ | |
| 34 | − | |
| 35 | −fman@400000 { | |
| 36 | − fman0_rx_0x09: port@89000 { | |
| 37 | − cell-index = <0x9>; | |
| 38 | − compatible = "fsl,fman-v3-port-rx"; | |
| 39 | − reg = <0x89000 0x1000>; | |
| 40 | − fsl,fman-10g-port; | |
| 41 | − fsl,fman-best-effort-port; | |
| 42 | − }; | |
| 43 | − | |
| 44 | − fman0_tx_0x29: port@a9000 { | |
| 45 | − cell-index = <0x29>; | |
| 46 | − compatible = "fsl,fman-v3-port-tx"; | |
| 47 | − reg = <0xa9000 0x1000>; | |
| 48 | − fsl,fman-10g-port; | |
| 49 | − fsl,fman-best-effort-port; | |
| 50 | − }; | |
| 51 | − | |
| 52 | − ethernet@e2000 { | |
| 53 | − cell-index = <1>; | |
| 54 | − compatible = "fsl,fman-memac"; | |
| 55 | − reg = <0xe2000 0x1000>; | |
| 56 | − fsl,fman-ports = <&fman0_rx_0x09 &fman0_tx_0x29>; | |
| 57 | − ptp-timer = <&ptp_timer0>; | |
| 58 | − pcsphy-handle = <&pcsphy1>, <&qsgmiia_pcs1>; | |
| 59 | − pcs-handle-names = "sgmii", "qsgmii"; | |
| 60 | − }; | |
| 61 | − | |
| 62 | − mdio@e1000 { | |
| 63 | − qsgmiia_pcs1: ethernet-pcs@1 { | |
| 64 | − compatible = "fsl,lynx-pcs"; | |
| 65 | − reg = <1>; | |
| 66 | − }; | |
| 67 | − }; | |
| 68 | − | |
| 69 | − mdio@e3000 { | |
| 70 | − #address-cells = <1>; | |
| 71 | − #size-cells = <0>; | |
| 72 | − compatible = "fsl,fman-memac-mdio", "fsl,fman-xmdio"; | |
| 73 | − reg = <0xe3000 0x1000>; | |
| 74 | − fsl,erratum-a011043; /* must ignore read errors */ | |
| 75 | − | |
| 76 | − pcsphy1: ethernet-phy@0 { | |
| 77 | − reg = <0x0>; | |
| 78 | − }; | |
| 79 | − }; | |
| 80 | −}; | |
| @@ -37,7 +37,7 @@ · 36 unmodified lines | ||
| 37 | 37 | }; |
| 38 | 38 | }; |
| 39 | 39 | |
| 40 | − memory { | |
| 40 | + memory@0 { | |
| 41 | 41 | device_type = "memory"; |
| 42 | 42 | reg = <0x00000000 0x08000000>; // 128MB at 0 |
| 43 | 43 | }; |
| @@ -38,7 +38,7 @@ · 37 unmodified lines | ||
| 38 | 38 | }; |
| 39 | 39 | }; |
| 40 | 40 | |
| 41 | − memory { | |
| 41 | + memory@0 { | |
| 42 | 42 | device_type = "memory"; |
| 43 | 43 | reg = <0x00000000 0x08000000>; // 128MB at 0 |
| 44 | 44 | }; |
| @@ -6,6 +6,7 @@ · 5 unmodified lines | ||
| 6 | 6 | */ |
| 7 | 7 | |
| 8 | 8 | /dts-v1/; |
| 9 | +#include <dt-bindings/interrupt-controller/irq.h> | |
| 9 | 10 | |
| 10 | 11 | / { |
| 11 | 12 | model = "MPC8313ERDB"; |
| @@ -38,7 +39,7 @@ · 26 unmodified lines | ||
| 38 | 39 | }; |
| 39 | 40 | }; |
| 40 | 41 | |
| 41 | − memory { | |
| 42 | + memory@0 { | |
| 42 | 43 | device_type = "memory"; |
| 43 | 44 | reg = <0x00000000 0x08000000>; // 128MB at 0 |
| 44 | 45 | }; |
| @@ -48,7 +49,7 @@ · 3 unmodified lines | ||
| 48 | 49 | #size-cells = <1>; |
| 49 | 50 | compatible = "fsl,mpc8313-elbc", "fsl,elbc", "simple-bus"; |
| 50 | 51 | reg = <0xe0005000 0x1000>; |
| 51 | − interrupts = <77 0x8>; | |
| 52 | + interrupts = <77 IRQ_TYPE_LEVEL_LOW>; | |
| 52 | 53 | interrupt-parent = <&ipic>; |
| 53 | 54 | |
| 54 | 55 | // CS0 and CS1 are swapped when |
| @@ -118,7 +119,7 @@ · 63 unmodified lines | ||
| 118 | 119 | cell-index = <0>; |
| 119 | 120 | compatible = "fsl-i2c"; |
| 120 | 121 | reg = <0x3000 0x100>; |
| 121 | − interrupts = <14 0x8>; | |
| 122 | + interrupts = <14 IRQ_TYPE_LEVEL_LOW>; | |
| 122 | 123 | interrupt-parent = <&ipic>; |
| 123 | 124 | dfsrr; |
| 124 | 125 | rtc@68 { |
| @@ -131,7 +132,7 @@ · 6 unmodified lines | ||
| 131 | 132 | compatible = "fsl,sec2.2", "fsl,sec2.1", |
| 132 | 133 | "fsl,sec2.0"; |
| 133 | 134 | reg = <0x30000 0x10000>; |
| 134 | − interrupts = <11 0x8>; | |
| 135 | + interrupts = <11 IRQ_TYPE_LEVEL_LOW>; | |
| 135 | 136 | interrupt-parent = <&ipic>; |
| 136 | 137 | fsl,num-channels = <1>; |
| 137 | 138 | fsl,channel-fifo-len = <24>; |
| @@ -146,7 +147,7 @@ · 8 unmodified lines | ||
| 146 | 147 | cell-index = <1>; |
| 147 | 148 | compatible = "fsl-i2c"; |
| 148 | 149 | reg = <0x3100 0x100>; |
| 149 | − interrupts = <15 0x8>; | |
| 150 | + interrupts = <15 IRQ_TYPE_LEVEL_LOW>; | |
| 150 | 151 | interrupt-parent = <&ipic>; |
| 151 | 152 | dfsrr; |
| 152 | 153 | }; |
| @@ -155,7 +156,7 @@ · 2 unmodified lines | ||
| 155 | 156 | cell-index = <0>; |
| 156 | 157 | compatible = "fsl,spi"; |
| 157 | 158 | reg = <0x7000 0x1000>; |
| 158 | − interrupts = <16 0x8>; | |
| 159 | + interrupts = <16 IRQ_TYPE_LEVEL_LOW>; | |
| 159 | 160 | interrupt-parent = <&ipic>; |
| 160 | 161 | mode = "cpu"; |
| 161 | 162 | }; |
| @@ -167,7 +168,7 @@ · 5 unmodified lines | ||
| 167 | 168 | #address-cells = <1>; |
| 168 | 169 | #size-cells = <0>; |
| 169 | 170 | interrupt-parent = <&ipic>; |
| 170 | − interrupts = <38 0x8>; | |
| 171 | + interrupts = <38 IRQ_TYPE_LEVEL_LOW>; | |
| 171 | 172 | phy_type = "utmi_wide"; |
| 172 | 173 | sleep = <&pmc 0x00300000>; |
| 173 | 174 | }; |
| @@ -175,7 +176,8 @@ · 1 unmodified line | ||
| 175 | 176 | ptp_clock@24E00 { |
| 176 | 177 | compatible = "fsl,etsec-ptp"; |
| 177 | 178 | reg = <0x24E00 0xB0>; |
| 178 | − interrupts = <12 0x8 13 0x8>; | |
| 179 | + interrupts = <12 IRQ_TYPE_LEVEL_LOW>, | |
| 180 | + <13 IRQ_TYPE_LEVEL_LOW>; | |
| 179 | 181 | interrupt-parent = < &ipic >; |
| 180 | 182 | fsl,tclk-period = <10>; |
| 181 | 183 | fsl,tmr-prsc = <100>; |
| @@ -197,7 +199,9 @@ · 15 unmodified lines | ||
| 197 | 199 | compatible = "gianfar"; |
| 198 | 200 | reg = <0x24000 0x1000>; |
| 199 | 201 | local-mac-address = [ 00 00 00 00 00 00 ]; |
| 200 | − interrupts = <37 0x8 36 0x8 35 0x8>; | |
| 202 | + interrupts = <37 IRQ_TYPE_LEVEL_LOW>, | |
| 203 | + <36 IRQ_TYPE_LEVEL_LOW>, | |
| 204 | + <35 IRQ_TYPE_LEVEL_LOW>; | |
| 201 | 205 | interrupt-parent = <&ipic>; |
| 202 | 206 | tbi-handle = < &tbi0 >; |
| 203 | 207 | /* Vitesse 7385 isn't on the MDIO bus */ |
| @@ -211,7 +215,7 @@ · 7 unmodified lines | ||
| 211 | 215 | reg = <0x520 0x20>; |
| 212 | 216 | phy4: ethernet-phy@4 { |
| 213 | 217 | interrupt-parent = <&ipic>; |
| 214 | − interrupts = <20 0x8>; | |
| 218 | + interrupts = <20 IRQ_TYPE_LEVEL_LOW>; | |
| 215 | 219 | reg = <0x4>; |
| 216 | 220 | }; |
| 217 | 221 | tbi0: tbi-phy@11 { |
| @@ -231,7 +235,9 @@ · 13 unmodified lines | ||
| 231 | 235 | reg = <0x25000 0x1000>; |
| 232 | 236 | ranges = <0x0 0x25000 0x1000>; |
| 233 | 237 | local-mac-address = [ 00 00 00 00 00 00 ]; |
| 234 | − interrupts = <34 0x8 33 0x8 32 0x8>; | |
| 238 | + interrupts = <34 IRQ_TYPE_LEVEL_LOW>, | |
| 239 | + <33 IRQ_TYPE_LEVEL_LOW>, | |
| 240 | + <32 IRQ_TYPE_LEVEL_LOW>; | |
| 235 | 241 | interrupt-parent = <&ipic>; |
| 236 | 242 | tbi-handle = < &tbi1 >; |
| 237 | 243 | phy-handle = < &phy4 >; |
| @@ -259,7 +265,7 @@ · 21 unmodified lines | ||
| 259 | 265 | compatible = "fsl,ns16550", "ns16550"; |
| 260 | 266 | reg = <0x4500 0x100>; |
| 261 | 267 | clock-frequency = <0>; |
| 262 | − interrupts = <9 0x8>; | |
| 268 | + interrupts = <9 IRQ_TYPE_LEVEL_LOW>; | |
| 263 | 269 | interrupt-parent = <&ipic>; |
| 264 | 270 | }; |
| 265 | 271 | |
| @@ -269,15 +275,12 @@ · 3 unmodified lines | ||
| 269 | 275 | compatible = "fsl,ns16550", "ns16550"; |
| 270 | 276 | reg = <0x4600 0x100>; |
| 271 | 277 | clock-frequency = <0>; |
| 272 | − interrupts = <10 0x8>; | |
| 278 | + interrupts = <10 IRQ_TYPE_LEVEL_LOW>; | |
| 273 | 279 | interrupt-parent = <&ipic>; |
| 274 | 280 | }; |
| 275 | 281 | |
| 276 | 282 | /* IPIC |
| 277 | − * interrupts cell = <intr #, sense> | |
| 278 | − * sense values match linux IORESOURCE_IRQ_* defines: | |
| 279 | − * sense == 8: Level, low assertion | |
| 280 | − * sense == 2: Edge, high-to-low change | |
| 283 | + * interrupts cell = <intr #, type> | |
| 281 | 284 | */ |
| 282 | 285 | ipic: pic@700 { |
| 283 | 286 | interrupt-controller; |
| @@ -290,7 +293,7 @@ · 6 unmodified lines | ||
| 290 | 293 | pmc: power@b00 { |
| 291 | 294 | compatible = "fsl,mpc8313-pmc", "fsl,mpc8349-pmc"; |
| 292 | 295 | reg = <0xb00 0x100 0xa00 0x100>; |
| 293 | − interrupts = <80 8>; | |
| 296 | + interrupts = <80 IRQ_TYPE_LEVEL_LOW>; | |
| 294 | 297 | interrupt-parent = <&ipic>; |
| 295 | 298 | fsl,mpc8313-wakeup-timer = <>m1>; |
| 296 | 299 | |
| @@ -306,14 +309,20 @@ · 9 unmodified lines | ||
| 306 | 309 | gtm1: timer@500 { |
| 307 | 310 | compatible = "fsl,mpc8313-gtm", "fsl,gtm"; |
| 308 | 311 | reg = <0x500 0x100>; |
| 309 | − interrupts = <90 8 78 8 84 8 72 8>; | |
| 312 | + interrupts = <90 IRQ_TYPE_LEVEL_LOW>, | |
| 313 | + <78 IRQ_TYPE_LEVEL_LOW>, | |
| 314 | + <84 IRQ_TYPE_LEVEL_LOW>, | |
| 315 | + <72 IRQ_TYPE_LEVEL_LOW>; | |
| 310 | 316 | interrupt-parent = <&ipic>; |
| 311 | 317 | }; |
| 312 | 318 | |
| 313 | 319 | timer@600 { |
| 314 | 320 | compatible = "fsl,mpc8313-gtm", "fsl,gtm"; |
| 315 | 321 | reg = <0x600 0x100>; |
| 316 | − interrupts = <91 8 79 8 85 8 73 8>; | |
| 322 | + interrupts = <91 IRQ_TYPE_LEVEL_LOW>, | |
| 323 | + <79 IRQ_TYPE_LEVEL_LOW>, | |
| 324 | + <85 IRQ_TYPE_LEVEL_LOW>, | |
| 325 | + <73 IRQ_TYPE_LEVEL_LOW>; | |
| 317 | 326 | interrupt-parent = <&ipic>; |
| 318 | 327 | }; |
| 319 | 328 | }; |
| @@ -341,7 +350,7 @@ · 21 unmodified lines | ||
| 341 | 350 | 0x7800 0x0 0x0 0x3 &ipic 17 0x8 |
| 342 | 351 | 0x7800 0x0 0x0 0x4 &ipic 18 0x8>; |
| 343 | 352 | interrupt-parent = <&ipic>; |
| 344 | − interrupts = <66 0x8>; | |
| 353 | + interrupts = <66 IRQ_TYPE_LEVEL_LOW>; | |
| 345 | 354 | bus-range = <0x0 0x0>; |
| 346 | 355 | ranges = <0x02000000 0x0 0x90000000 0x90000000 0x0 0x10000000 |
| 347 | 356 | 0x42000000 0x0 0x80000000 0x80000000 0x0 0x10000000 |
| @@ -363,14 +372,14 @@ · 15 unmodified lines | ||
| 363 | 372 | reg = <0xe00082a8 4>; |
| 364 | 373 | ranges = <0 0xe0008100 0x1a8>; |
| 365 | 374 | interrupt-parent = <&ipic>; |
| 366 | − interrupts = <71 8>; | |
| 375 | + interrupts = <71 IRQ_TYPE_LEVEL_LOW>; | |
| 367 | 376 | |
| 368 | 377 | dma-channel@0 { |
| 369 | 378 | compatible = "fsl,mpc8313-dma-channel", |
| 370 | 379 | "fsl,elo-dma-channel"; |
| 371 | 380 | reg = <0 0x28>; |
| 372 | 381 | interrupt-parent = <&ipic>; |
| 373 | − interrupts = <71 8>; | |
| 382 | + interrupts = <71 IRQ_TYPE_LEVEL_LOW>; | |
| 374 | 383 | cell-index = <0>; |
| 375 | 384 | }; |
| 376 | 385 | |
| @@ -379,7 +388,7 @@ · 2 unmodified lines | ||
| 379 | 388 | "fsl,elo-dma-channel"; |
| 380 | 389 | reg = <0x80 0x28>; |
| 381 | 390 | interrupt-parent = <&ipic>; |
| 382 | − interrupts = <71 8>; | |
| 391 | + interrupts = <71 IRQ_TYPE_LEVEL_LOW>; | |
| 383 | 392 | cell-index = <1>; |
| 384 | 393 | }; |
| 385 | 394 | |
| @@ -388,7 +397,7 @@ · 2 unmodified lines | ||
| 388 | 397 | "fsl,elo-dma-channel"; |
| 389 | 398 | reg = <0x100 0x28>; |
| 390 | 399 | interrupt-parent = <&ipic>; |
| 391 | − interrupts = <71 8>; | |
| 400 | + interrupts = <71 IRQ_TYPE_LEVEL_LOW>; | |
| 392 | 401 | cell-index = <2>; |
| 393 | 402 | }; |
| 394 | 403 | |
| @@ -397,7 +406,7 @@ · 2 unmodified lines | ||
| 397 | 406 | "fsl,elo-dma-channel"; |
| 398 | 407 | reg = <0x180 0x28>; |
| 399 | 408 | interrupt-parent = <&ipic>; |
| 400 | − interrupts = <71 8>; | |
| 409 | + interrupts = <71 IRQ_TYPE_LEVEL_LOW>; | |
| 401 | 410 | cell-index = <3>; |
| 402 | 411 | }; |
| 403 | 412 | }; |
| @@ -40,7 +40,7 @@ · 39 unmodified lines | ||
| 40 | 40 | }; |
| 41 | 41 | }; |
| 42 | 42 | |
| 43 | − memory { | |
| 43 | + memory@0 { | |
| 44 | 44 | device_type = "memory"; |
| 45 | 45 | reg = <0x00000000 0x08000000>; // 128MB at 0 |
| 46 | 46 | }; |
| @@ -50,7 +50,7 @@ · 3 unmodified lines | ||
| 50 | 50 | #size-cells = <1>; |
| 51 | 51 | compatible = "fsl,mpc8315-elbc", "fsl,elbc", "simple-bus"; |
| 52 | 52 | reg = <0xe0005000 0x1000>; |
| 53 | − interrupts = <77 0x8>; | |
| 53 | + interrupts = <77 IRQ_TYPE_LEVEL_LOW>; | |
| 54 | 54 | interrupt-parent = <&ipic>; |
| 55 | 55 | |
| 56 | 56 | // CS0 and CS1 are swapped when |
| @@ -112,7 +112,7 @@ · 55 unmodified lines | ||
| 112 | 112 | cell-index = <0>; |
| 113 | 113 | compatible = "fsl-i2c"; |
| 114 | 114 | reg = <0x3000 0x100>; |
| 115 | − interrupts = <14 0x8>; | |
| 115 | + interrupts = <14 IRQ_TYPE_LEVEL_LOW>; | |
| 116 | 116 | interrupt-parent = <&ipic>; |
| 117 | 117 | dfsrr; |
| 118 | 118 | rtc@68 { |
| @@ -133,8 +133,10 @@ · 14 unmodified lines | ||
| 133 | 133 | cell-index = <0>; |
| 134 | 134 | compatible = "fsl,spi"; |
| 135 | 135 | reg = <0x7000 0x1000>; |
| 136 | − interrupts = <16 0x8>; | |
| 136 | + interrupts = <16 IRQ_TYPE_LEVEL_LOW>; | |
| 137 | 137 | interrupt-parent = <&ipic>; |
| 138 | + #address-cells = <1>; | |
| 139 | + #size-cells = <0>; | |
| 138 | 140 | mode = "cpu"; |
| 139 | 141 | }; |
| 140 | 142 | |
| @@ -145,35 +147,35 @@ · 4 unmodified lines | ||
| 145 | 147 | reg = <0x82a8 4>; |
| 146 | 148 | ranges = <0 0x8100 0x1a8>; |
| 147 | 149 | interrupt-parent = <&ipic>; |
| 148 | − interrupts = <71 8>; | |
| 150 | + interrupts = <71 IRQ_TYPE_LEVEL_LOW>; | |
| 149 | 151 | cell-index = <0>; |
| 150 | 152 | dma-channel@0 { |
| 151 | 153 | compatible = "fsl,mpc8315-dma-channel", "fsl,elo-dma-channel"; |
| 152 | 154 | reg = <0 0x80>; |
| 153 | 155 | cell-index = <0>; |
| 154 | 156 | interrupt-parent = <&ipic>; |
| 155 | − interrupts = <71 8>; | |
| 157 | + interrupts = <71 IRQ_TYPE_LEVEL_LOW>; | |
| 156 | 158 | }; |
| 157 | 159 | dma-channel@80 { |
| 158 | 160 | compatible = "fsl,mpc8315-dma-channel", "fsl,elo-dma-channel"; |
| 159 | 161 | reg = <0x80 0x80>; |
| 160 | 162 | cell-index = <1>; |
| 161 | 163 | interrupt-parent = <&ipic>; |
| 162 | − interrupts = <71 8>; | |
| 164 | + interrupts = <71 IRQ_TYPE_LEVEL_LOW>; | |
| 163 | 165 | }; |
| 164 | 166 | dma-channel@100 { |
| 165 | 167 | compatible = "fsl,mpc8315-dma-channel", "fsl,elo-dma-channel"; |
| 166 | 168 | reg = <0x100 0x80>; |
| 167 | 169 | cell-index = <2>; |
| 168 | 170 | interrupt-parent = <&ipic>; |
| 169 | − interrupts = <71 8>; | |
| 171 | + interrupts = <71 IRQ_TYPE_LEVEL_LOW>; | |
| 170 | 172 | }; |
| 171 | 173 | dma-channel@180 { |
| 172 | 174 | compatible = "fsl,mpc8315-dma-channel", "fsl,elo-dma-channel"; |
| 173 | 175 | reg = <0x180 0x28>; |
| 174 | 176 | cell-index = <3>; |
| 175 | 177 | interrupt-parent = <&ipic>; |
| 176 | − interrupts = <71 8>; | |
| 178 | + interrupts = <71 IRQ_TYPE_LEVEL_LOW>; | |
| 177 | 179 | }; |
| 178 | 180 | }; |
| 179 | 181 | |
| @@ -183,7 +185,7 @@ · 3 unmodified lines | ||
| 183 | 185 | #address-cells = <1>; |
| 184 | 186 | #size-cells = <0>; |
| 185 | 187 | interrupt-parent = <&ipic>; |
| 186 | − interrupts = <38 0x8>; | |
| 188 | + interrupts = <38 IRQ_TYPE_LEVEL_LOW>; | |
| 187 | 189 | phy_type = "utmi"; |
| 188 | 190 | }; |
| 189 | 191 | |
| @@ -197,7 +199,9 @@ · 7 unmodified lines | ||
| 197 | 199 | reg = <0x24000 0x1000>; |
| 198 | 200 | ranges = <0x0 0x24000 0x1000>; |
| 199 | 201 | local-mac-address = [ 00 00 00 00 00 00 ]; |
| 200 | − interrupts = <32 0x8 33 0x8 34 0x8>; | |
| 202 | + interrupts = <32 IRQ_TYPE_LEVEL_LOW>, | |
| 203 | + <33 IRQ_TYPE_LEVEL_LOW>, | |
| 204 | + <34 IRQ_TYPE_LEVEL_LOW>; | |
| 201 | 205 | interrupt-parent = <&ipic>; |
| 202 | 206 | tbi-handle = <&tbi0>; |
| 203 | 207 | phy-handle = < &phy0 >; |
| @@ -238,7 +242,9 @@ · 34 unmodified lines | ||
| 238 | 242 | reg = <0x25000 0x1000>; |
| 239 | 243 | ranges = <0x0 0x25000 0x1000>; |
| 240 | 244 | local-mac-address = [ 00 00 00 00 00 00 ]; |
| 241 | − interrupts = <35 0x8 36 0x8 37 0x8>; | |
| 245 | + interrupts = <35 IRQ_TYPE_LEVEL_LOW>, | |
| 246 | + <36 IRQ_TYPE_LEVEL_LOW>, | |
| 247 | + <37 IRQ_TYPE_LEVEL_LOW>; | |
| 242 | 248 | interrupt-parent = <&ipic>; |
| 243 | 249 | tbi-handle = <&tbi1>; |
| 244 | 250 | phy-handle = < &phy1 >; |
| @@ -263,7 +269,7 @@ · 18 unmodified lines | ||
| 263 | 269 | compatible = "fsl,ns16550", "ns16550"; |
| 264 | 270 | reg = <0x4500 0x100>; |
| 265 | 271 | clock-frequency = <133333333>; |
| 266 | − interrupts = <9 0x8>; | |
| 272 | + interrupts = <9 IRQ_TYPE_LEVEL_LOW>; | |
| 267 | 273 | interrupt-parent = <&ipic>; |
| 268 | 274 | }; |
| 269 | 275 | |
| @@ -273,7 +279,7 @@ · 3 unmodified lines | ||
| 273 | 279 | compatible = "fsl,ns16550", "ns16550"; |
| 274 | 280 | reg = <0x4600 0x100>; |
| 275 | 281 | clock-frequency = <133333333>; |
| 276 | − interrupts = <10 0x8>; | |
| 282 | + interrupts = <10 IRQ_TYPE_LEVEL_LOW>; | |
| 277 | 283 | interrupt-parent = <&ipic>; |
| 278 | 284 | }; |
| 279 | 285 | |
| @@ -282,7 +288,7 @@ · 2 unmodified lines | ||
| 282 | 288 | "fsl,sec2.4", "fsl,sec2.2", "fsl,sec2.1", |
| 283 | 289 | "fsl,sec2.0"; |
| 284 | 290 | reg = <0x30000 0x10000>; |
| 285 | − interrupts = <11 0x8>; | |
| 291 | + interrupts = <11 IRQ_TYPE_LEVEL_LOW>; | |
| 286 | 292 | interrupt-parent = <&ipic>; |
| 287 | 293 | fsl,num-channels = <4>; |
| 288 | 294 | fsl,channel-fifo-len = <24>; |
| @@ -294,7 +300,7 @@ · 5 unmodified lines | ||
| 294 | 300 | compatible = "fsl,mpc8315-sata", "fsl,pq-sata"; |
| 295 | 301 | reg = <0x18000 0x1000>; |
| 296 | 302 | cell-index = <1>; |
| 297 | − interrupts = <44 0x8>; | |
| 303 | + interrupts = <44 IRQ_TYPE_LEVEL_LOW>; | |
| 298 | 304 | interrupt-parent = <&ipic>; |
| 299 | 305 | }; |
| 300 | 306 | |
| @@ -302,14 +308,17 @@ · 1 unmodified line | ||
| 302 | 308 | compatible = "fsl,mpc8315-sata", "fsl,pq-sata"; |
| 303 | 309 | reg = <0x19000 0x1000>; |
| 304 | 310 | cell-index = <2>; |
| 305 | − interrupts = <45 0x8>; | |
| 311 | + interrupts = <45 IRQ_TYPE_LEVEL_LOW>; | |
| 306 | 312 | interrupt-parent = <&ipic>; |
| 307 | 313 | }; |
| 308 | 314 | |
| 309 | 315 | gtm1: timer@500 { |
| 310 | 316 | compatible = "fsl,mpc8315-gtm", "fsl,gtm"; |
| 311 | 317 | reg = <0x500 0x100>; |
| 312 | − interrupts = <90 8 78 8 84 8 72 8>; | |
| 318 | + interrupts = <90 IRQ_TYPE_LEVEL_LOW>, | |
| 319 | + <78 IRQ_TYPE_LEVEL_LOW>, | |
| 320 | + <84 IRQ_TYPE_LEVEL_LOW>, | |
| 321 | + <72 IRQ_TYPE_LEVEL_LOW>; | |
| 313 | 322 | interrupt-parent = <&ipic>; |
| 314 | 323 | clock-frequency = <133333333>; |
| 315 | 324 | }; |
| @@ -317,16 +326,16 @@ · 1 unmodified line | ||
| 317 | 326 | timer@600 { |
| 318 | 327 | compatible = "fsl,mpc8315-gtm", "fsl,gtm"; |
| 319 | 328 | reg = <0x600 0x100>; |
| 320 | − interrupts = <91 8 79 8 85 8 73 8>; | |
| 329 | + interrupts = <91 IRQ_TYPE_LEVEL_LOW>, | |
| 330 | + <79 IRQ_TYPE_LEVEL_LOW>, | |
| 331 | + <85 IRQ_TYPE_LEVEL_LOW>, | |
| 332 | + <73 IRQ_TYPE_LEVEL_LOW>; | |
| 321 | 333 | interrupt-parent = <&ipic>; |
| 322 | 334 | clock-frequency = <133333333>; |
| 323 | 335 | }; |
| 324 | 336 | |
| 325 | 337 | /* IPIC |
| 326 | − * interrupts cell = <intr #, sense> | |
| 327 | − * sense values match linux IORESOURCE_IRQ_* defines: | |
| 328 | − * sense == 8: Level, low assertion | |
| 329 | − * sense == 2: Edge, high-to-low change | |
| 338 | + * interrupts cell = <intr #, type> | |
| 330 | 339 | */ |
| 331 | 340 | ipic: interrupt-controller@700 { |
| 332 | 341 | interrupt-controller; |
| @@ -340,14 +349,14 @@ · 7 unmodified lines | ||
| 340 | 349 | compatible = "fsl,ipic-msi"; |
| 341 | 350 | reg = <0x7c0 0x40>; |
| 342 | 351 | msi-available-ranges = <0 0x100>; |
| 343 | − interrupts = <0x43 0x8 | |
| 344 | − 0x4 0x8 | |
| 345 | − 0x51 0x8 | |
| 346 | − 0x52 0x8 | |
| 347 | − 0x56 0x8 | |
| 348 | − 0x57 0x8 | |
| 349 | − 0x58 0x8 | |
| 350 | − 0x59 0x8>; | |
| 352 | + interrupts = <0x43 IRQ_TYPE_LEVEL_LOW | |
| 353 | + 0x4 IRQ_TYPE_LEVEL_LOW | |
| 354 | + 0x51 IRQ_TYPE_LEVEL_LOW | |
| 355 | + 0x52 IRQ_TYPE_LEVEL_LOW | |
| 356 | + 0x56 IRQ_TYPE_LEVEL_LOW | |
| 357 | + 0x57 IRQ_TYPE_LEVEL_LOW | |
| 358 | + 0x58 IRQ_TYPE_LEVEL_LOW | |
| 359 | + 0x59 IRQ_TYPE_LEVEL_LOW>; | |
| 351 | 360 | interrupt-parent = < &ipic >; |
| 352 | 361 | }; |
| 353 | 362 | |
| @@ -355,7 +364,7 @@ · 1 unmodified line | ||
| 355 | 364 | compatible = "fsl,mpc8315-pmc", "fsl,mpc8313-pmc", |
| 356 | 365 | "fsl,mpc8349-pmc"; |
| 357 | 366 | reg = <0xb00 0x100 0xa00 0x100>; |
| 358 | − interrupts = <80 8>; | |
| 367 | + interrupts = <80 IRQ_TYPE_LEVEL_LOW>; | |
| 359 | 368 | interrupt-parent = <&ipic>; |
| 360 | 369 | fsl,mpc8313-wakeup-timer = <>m1>; |
| 361 | 370 | }; |
| @@ -374,24 +383,24 @@ · 12 unmodified lines | ||
| 374 | 383 | interrupt-map-mask = <0xf800 0x0 0x0 0x7>; |
| 375 | 384 | interrupt-map = < |
| 376 | 385 | /* IDSEL 0x0E -mini PCI */ |
| 377 | − 0x7000 0x0 0x0 0x1 &ipic 18 0x8 | |
| 378 | − 0x7000 0x0 0x0 0x2 &ipic 18 0x8 | |
| 379 | − 0x7000 0x0 0x0 0x3 &ipic 18 0x8 | |
| 380 | − 0x7000 0x0 0x0 0x4 &ipic 18 0x8 | |
| 386 | + 0x7000 0x0 0x0 0x1 &ipic 18 IRQ_TYPE_LEVEL_LOW | |
| 387 | + 0x7000 0x0 0x0 0x2 &ipic 18 IRQ_TYPE_LEVEL_LOW | |
| 388 | + 0x7000 0x0 0x0 0x3 &ipic 18 IRQ_TYPE_LEVEL_LOW | |
| 389 | + 0x7000 0x0 0x0 0x4 &ipic 18 IRQ_TYPE_LEVEL_LOW | |
| 381 | 390 | |
| 382 | 391 | /* IDSEL 0x0F -mini PCI */ |
| 383 | − 0x7800 0x0 0x0 0x1 &ipic 17 0x8 | |
| 384 | − 0x7800 0x0 0x0 0x2 &ipic 17 0x8 | |
| 385 | − 0x7800 0x0 0x0 0x3 &ipic 17 0x8 | |
| 386 | − 0x7800 0x0 0x0 0x4 &ipic 17 0x8 | |
| 392 | + 0x7800 0x0 0x0 0x1 &ipic 17 IRQ_TYPE_LEVEL_LOW | |
| 393 | + 0x7800 0x0 0x0 0x2 &ipic 17 IRQ_TYPE_LEVEL_LOW | |
| 394 | + 0x7800 0x0 0x0 0x3 &ipic 17 IRQ_TYPE_LEVEL_LOW | |
| 395 | + 0x7800 0x0 0x0 0x4 &ipic 17 IRQ_TYPE_LEVEL_LOW | |
| 387 | 396 | |
| 388 | 397 | /* IDSEL 0x10 - PCI slot */ |
| 389 | − 0x8000 0x0 0x0 0x1 &ipic 48 0x8 | |
| 390 | − 0x8000 0x0 0x0 0x2 &ipic 17 0x8 | |
| 391 | − 0x8000 0x0 0x0 0x3 &ipic 48 0x8 | |
| 392 | − 0x8000 0x0 0x0 0x4 &ipic 17 0x8>; | |
| 398 | + 0x8000 0x0 0x0 0x1 &ipic 48 IRQ_TYPE_LEVEL_LOW | |
| 399 | + 0x8000 0x0 0x0 0x2 &ipic 17 IRQ_TYPE_LEVEL_LOW | |
| 400 | + 0x8000 0x0 0x0 0x3 &ipic 48 IRQ_TYPE_LEVEL_LOW | |
| 401 | + 0x8000 0x0 0x0 0x4 &ipic 17 IRQ_TYPE_LEVEL_LOW>; | |
| 393 | 402 | interrupt-parent = <&ipic>; |
| 394 | − interrupts = <66 0x8>; | |
| 403 | + interrupts = <66 IRQ_TYPE_LEVEL_LOW>; | |
| 395 | 404 | bus-range = <0x0 0x0>; |
| 396 | 405 | ranges = <0x02000000 0 0x90000000 0x90000000 0 0x10000000 |
| 397 | 406 | 0x42000000 0 0x80000000 0x80000000 0 0x10000000 |
| @@ -417,10 +426,10 @@ · 19 unmodified lines | ||
| 417 | 426 | 0x01000000 0 0x00000000 0xb1000000 0 0x00800000>; |
| 418 | 427 | bus-range = <0 255>; |
| 419 | 428 | interrupt-map-mask = <0xf800 0 0 7>; |
| 420 | − interrupt-map = <0 0 0 1 &ipic 1 8 | |
| 421 | − 0 0 0 2 &ipic 1 8 | |
| 422 | − 0 0 0 3 &ipic 1 8 | |
| 423 | − 0 0 0 4 &ipic 1 8>; | |
| 429 | + interrupt-map = <0 0 0 1 &ipic 1 IRQ_TYPE_LEVEL_LOW | |
| 430 | + 0 0 0 2 &ipic 1 IRQ_TYPE_LEVEL_LOW | |
| 431 | + 0 0 0 3 &ipic 1 IRQ_TYPE_LEVEL_LOW | |
| 432 | + 0 0 0 4 &ipic 1 IRQ_TYPE_LEVEL_LOW>; | |
| 424 | 433 | clock-frequency = <0>; |
| 425 | 434 | |
| 426 | 435 | pcie@0 { |
| @@ -448,10 +457,10 @@ · 21 unmodified lines | ||
| 448 | 457 | 0x01000000 0 0x00000000 0xd1000000 0 0x00800000>; |
| 449 | 458 | bus-range = <0 255>; |
| 450 | 459 | interrupt-map-mask = <0xf800 0 0 7>; |
| 451 | − interrupt-map = <0 0 0 1 &ipic 2 8 | |
| 452 | − 0 0 0 2 &ipic 2 8 | |
| 453 | − 0 0 0 3 &ipic 2 8 | |
| 454 | − 0 0 0 4 &ipic 2 8>; | |
| 460 | + interrupt-map = <0 0 0 1 &ipic 2 IRQ_TYPE_LEVEL_LOW | |
| 461 | + 0 0 0 2 &ipic 2 IRQ_TYPE_LEVEL_LOW | |
| 462 | + 0 0 0 3 &ipic 2 IRQ_TYPE_LEVEL_LOW | |
| 463 | + 0 0 0 4 &ipic 2 IRQ_TYPE_LEVEL_LOW>; | |
| 455 | 464 | clock-frequency = <0>; |
| 456 | 465 | |
| 457 | 466 | pcie@0 { |
| @@ -471,12 +480,12 @@ · 13 unmodified lines | ||
| 471 | 480 | leds { |
| 472 | 481 | compatible = "gpio-leds"; |
| 473 | 482 | |
| 474 | − pwr { | |
| 483 | + led-pwr { | |
| 475 | 484 | gpios = <&mcu_pio 0 0>; |
| 476 | 485 | default-state = "on"; |
| 477 | 486 | }; |
| 478 | 487 | |
| 479 | − hdd { | |
| 488 | + led-hdd { | |
| 480 | 489 | gpios = <&mcu_pio 1 0>; |
| 481 | 490 | linux,default-trigger = "disk-activity"; |
| 482 | 491 | }; |
| @@ -38,7 +38,7 @@ · 37 unmodified lines | ||
| 38 | 38 | }; |
| 39 | 39 | }; |
| 40 | 40 | |
| 41 | − memory { | |
| 41 | + memory@0 { | |
| 42 | 42 | device_type = "memory"; |
| 43 | 43 | reg = <0x00000000 0x04000000>; |
| 44 | 44 | }; |
| @@ -39,7 +39,7 @@ · 38 unmodified lines | ||
| 39 | 39 | }; |
| 40 | 40 | }; |
| 41 | 41 | |
| 42 | − memory { | |
| 42 | + memory@0 { | |
| 43 | 43 | device_type = "memory"; |
| 44 | 44 | reg = <0x00000000 0x10000000>; |
| 45 | 45 | }; |
| @@ -37,7 +37,7 @@ · 36 unmodified lines | ||
| 37 | 37 | }; |
| 38 | 38 | }; |
| 39 | 39 | |
| 40 | − memory { | |
| 40 | + memory@0 { | |
| 41 | 41 | device_type = "memory"; |
| 42 | 42 | reg = <0x00000000 0x10000000>; |
| 43 | 43 | }; |
| @@ -39,7 +39,7 @@ · 38 unmodified lines | ||
| 39 | 39 | }; |
| 40 | 40 | }; |
| 41 | 41 | |
| 42 | − memory { | |
| 42 | + memory@0 { | |
| 43 | 43 | device_type = "memory"; |
| 44 | 44 | reg = <0x00000000 0x10000000>; // 256MB at 0 |
| 45 | 45 | }; |
| @@ -40,7 +40,7 @@ · 39 unmodified lines | ||
| 40 | 40 | }; |
| 41 | 41 | }; |
| 42 | 42 | |
| 43 | − memory { | |
| 43 | + memory@0 { | |
| 44 | 44 | device_type = "memory"; |
| 45 | 45 | reg = <0x00000000 0x20000000>; // 512MB at 0 |
| 46 | 46 | }; |
| @@ -39,7 +39,7 @@ · 38 unmodified lines | ||
| 39 | 39 | }; |
| 40 | 40 | }; |
| 41 | 41 | |
| 42 | − memory { | |
| 42 | + memory@0 { | |
| 43 | 43 | device_type = "memory"; |
| 44 | 44 | reg = <0x00000000 0x10000000>; // 256MB at 0 |
| 45 | 45 | }; |
| @@ -37,7 +37,7 @@ · 36 unmodified lines | ||
| 37 | 37 | }; |
| 38 | 38 | }; |
| 39 | 39 | |
| 40 | − memory { | |
| 40 | + memory@0 { | |
| 41 | 41 | device_type = "memory"; |
| 42 | 42 | reg = <0x00000000 0x10000000>; // 256MB at 0 |
| 43 | 43 | }; |
| @@ -120,10 +120,8 @@ · 119 unmodified lines | ||
| 120 | 120 | |
| 121 | 121 | #if defined(CONFIG_44x) |
| 122 | 122 | #include <asm/nohash/32/pte-44x.h> |
| 123 | −#elif defined(CONFIG_PPC_85xx) && defined(CONFIG_PTE_64BIT) | |
| 124 | −#include <asm/nohash/pte-e500.h> | |
| 125 | 123 | #elif defined(CONFIG_PPC_85xx) |
| 126 | −#include <asm/nohash/32/pte-85xx.h> | |
| 124 | +#include <asm/nohash/pte-e500.h> | |
| 127 | 125 | #elif defined(CONFIG_PPC_8xx) |
| 128 | 126 | #include <asm/nohash/32/pte-8xx.h> |
| 129 | 127 | #endif |
| @@ -1,59 +0,0 @@ | ||
| 1 | −/* SPDX-License-Identifier: GPL-2.0 */ | |
| 2 | −#ifndef _ASM_POWERPC_NOHASH_32_PTE_85xx_H | |
| 3 | −#define _ASM_POWERPC_NOHASH_32_PTE_85xx_H | |
| 4 | −#ifdef __KERNEL__ | |
| 5 | − | |
| 6 | −/* PTE bit definitions for Freescale BookE SW loaded TLB MMU based | |
| 7 | − * processors | |
| 8 | − * | |
| 9 | − MMU Assist Register 3: | |
| 10 | − | |
| 11 | − 32 33 34 35 36 ... 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | |
| 12 | − RPN...................... 0 0 U0 U1 U2 U3 UX SX UW SW UR SR | |
| 13 | − | |
| 14 | − - PRESENT *must* be in the bottom two bits because swap PTEs use | |
| 15 | − the top 30 bits. | |
| 16 | − | |
| 17 | −*/ | |
| 18 | − | |
| 19 | −/* Definitions for FSL Book-E Cores */ | |
| 20 | −#define _PAGE_READ 0x00001 /* H: Read permission (SR) */ | |
| 21 | −#define _PAGE_PRESENT 0x00002 /* S: PTE contains a translation */ | |
| 22 | −#define _PAGE_WRITE 0x00004 /* S: Write permission (SW) */ | |
| 23 | −#define _PAGE_DIRTY 0x00008 /* S: Page dirty */ | |
| 24 | −#define _PAGE_EXEC 0x00010 /* H: SX permission */ | |
| 25 | −#define _PAGE_ACCESSED 0x00020 /* S: Page referenced */ | |
| 26 | − | |
| 27 | −#define _PAGE_ENDIAN 0x00040 /* H: E bit */ | |
| 28 | −#define _PAGE_GUARDED 0x00080 /* H: G bit */ | |
| 29 | −#define _PAGE_COHERENT 0x00100 /* H: M bit */ | |
| 30 | −#define _PAGE_NO_CACHE 0x00200 /* H: I bit */ | |
| 31 | −#define _PAGE_WRITETHRU 0x00400 /* H: W bit */ | |
| 32 | −#define _PAGE_SPECIAL 0x00800 /* S: Special page */ | |
| 33 | − | |
| 34 | −#define _PMD_PRESENT 0 | |
| 35 | −#define _PMD_PRESENT_MASK (PAGE_MASK) | |
| 36 | −#define _PMD_BAD (~PAGE_MASK) | |
| 37 | −#define _PMD_USER 0 | |
| 38 | − | |
| 39 | −#define _PTE_NONE_MASK 0 | |
| 40 | − | |
| 41 | −#define PTE_WIMGE_SHIFT (6) | |
| 42 | − | |
| 43 | −/* | |
| 44 | − * We define 2 sets of base prot bits, one for basic pages (ie, | |
| 45 | − * cacheable kernel and user pages) and one for non cacheable | |
| 46 | − * pages. We always set _PAGE_COHERENT when SMP is enabled or | |
| 47 | − * the processor might need it for DMA coherency. | |
| 48 | − */ | |
| 49 | −#define _PAGE_BASE_NC (_PAGE_PRESENT | _PAGE_ACCESSED) | |
| 50 | −#if defined(CONFIG_SMP) || defined(CONFIG_PPC_E500MC) | |
| 51 | −#define _PAGE_BASE (_PAGE_BASE_NC | _PAGE_COHERENT) | |
| 52 | −#else | |
| 53 | −#define _PAGE_BASE (_PAGE_BASE_NC) | |
| 54 | −#endif | |
| 55 | − | |
| 56 | −#include <asm/pgtable-masks.h> | |
| 57 | − | |
| 58 | −#endif /* __KERNEL__ */ | |
| 59 | −#endif /* _ASM_POWERPC_NOHASH_32_PTE_FSL_85xx_H */ | |
| @@ -49,7 +49,7 @@ static inline unsigned long pud_val(pud_t x) · 48 unmodified lines | ||
| 49 | 49 | #endif /* CONFIG_PPC64 */ |
| 50 | 50 | |
| 51 | 51 | /* PGD level */ |
| 52 | −#if defined(CONFIG_PPC_85xx) && defined(CONFIG_PTE_64BIT) | |
| 52 | +#if defined(CONFIG_PPC_85xx) | |
| 53 | 53 | typedef struct { unsigned long long pgd; } pgd_t; |
| 54 | 54 | |
| 55 | 55 | static inline unsigned long long pgd_val(pgd_t x) |
| @@ -15,6 +15,9 @@ · 14 unmodified lines | ||
| 15 | 15 | #define TASK_SIZE_MAX TASK_SIZE_USER64 |
| 16 | 16 | #endif |
| 17 | 17 | |
| 18 | +/* Threshold above which VMX copy path is used */ | |
| 19 | +#define VMX_COPY_THRESHOLD 3328 | |
| 20 | + | |
| 18 | 21 | #include <asm-generic/access_ok.h> |
| 19 | 22 | |
| 20 | 23 | /* |
| @@ -255,7 +258,7 @@ __gus_failed: \ · 234 unmodified lines | ||
| 255 | 258 | ".section .fixup,\"ax\"\n" \ |
| 256 | 259 | "4: li %0,%3\n" \ |
| 257 | 260 | " li %1,0\n" \ |
| 258 | − " li %1+1,0\n" \ | |
| 261 | + " li %L1,0\n" \ | |
| 259 | 262 | " b 3b\n" \ |
| 260 | 263 | ".previous\n" \ |
| 261 | 264 | EX_TABLE(1b, 4b) \ |
| @@ -326,40 +329,62 @@ do { \ · 64 unmodified lines | ||
| 326 | 329 | extern unsigned long __copy_tofrom_user(void __user *to, |
| 327 | 330 | const void __user *from, unsigned long size); |
| 328 | 331 | |
| 329 | −#ifdef __powerpc64__ | |
| 330 | −static inline unsigned long | |
| 331 | −raw_copy_in_user(void __user *to, const void __user *from, unsigned long n) | |
| 332 | +unsigned long __copy_tofrom_user_base(void __user *to, | |
| 333 | + const void __user *from, unsigned long size); | |
| 334 | + | |
| 335 | +unsigned long __copy_tofrom_user_power7_vmx(void __user *to, | |
| 336 | + const void __user *from, unsigned long size); | |
| 337 | + | |
| 338 | +static __always_inline bool will_use_vmx(unsigned long n) | |
| 339 | +{ | |
| 340 | + return IS_ENABLED(CONFIG_ALTIVEC) && cpu_has_feature(CPU_FTR_VMX_COPY) && | |
| 341 | + n > VMX_COPY_THRESHOLD; | |
| 342 | +} | |
| 343 | + | |
| 344 | +static __always_inline unsigned long | |
| 345 | +raw_copy_tofrom_user(void __user *to, const void __user *from, | |
| 346 | + unsigned long n, unsigned long dir) | |
| 332 | 347 | { |
| 333 | 348 | unsigned long ret; |
| 334 | 349 | |
| 335 | − barrier_nospec(); | |
| 336 | − allow_user_access(to, KUAP_READ_WRITE); | |
| 350 | + if (will_use_vmx(n) && enter_vmx_usercopy()) { | |
| 351 | + allow_user_access(to, dir); | |
| 352 | + ret = __copy_tofrom_user_power7_vmx(to, from, n); | |
| 353 | + prevent_user_access(dir); | |
| 354 | + exit_vmx_usercopy(); | |
| 355 | + | |
| 356 | + if (unlikely(ret)) { | |
| 357 | + allow_user_access(to, dir); | |
| 358 | + ret = __copy_tofrom_user_base(to, from, n); | |
| 359 | + prevent_user_access(dir); | |
| 360 | + } | |
| 361 | + return ret; | |
| 362 | + } | |
| 363 | + | |
| 364 | + allow_user_access(to, dir); | |
| 337 | 365 | ret = __copy_tofrom_user(to, from, n); |
| 338 | − prevent_user_access(KUAP_READ_WRITE); | |
| 366 | + prevent_user_access(dir); | |
| 339 | 367 | return ret; |
| 340 | 368 | } |
| 341 | −#endif /* __powerpc64__ */ | |
| 342 | 369 | |
| 343 | −static inline unsigned long raw_copy_from_user(void *to, | |
| 344 | − const void __user *from, unsigned long n) | |
| 370 | +#ifdef CONFIG_PPC64 | |
| 371 | +static inline unsigned long | |
| 372 | +raw_copy_in_user(void __user *to, const void __user *from, unsigned long n) | |
| 345 | 373 | { |
| 346 | − unsigned long ret; | |
| 374 | + barrier_nospec(); | |
| 375 | + return raw_copy_tofrom_user(to, from, n, KUAP_READ_WRITE); | |
| 376 | +} | |
| 377 | +#endif /* CONFIG_PPC64 */ | |
| 347 | 378 | |
| 348 | − allow_user_access(NULL, KUAP_READ); | |
| 349 | − ret = __copy_tofrom_user((__force void __user *)to, from, n); | |
| 350 | − prevent_user_access(KUAP_READ); | |
| 351 | − return ret; | |
| 379 | +static inline unsigned long raw_copy_from_user(void *to, const void __user *from, unsigned long n) | |
| 380 | +{ | |
| 381 | + return raw_copy_tofrom_user((__force void __user *)to, from, n, KUAP_READ); | |
| 352 | 382 | } |
| 353 | 383 | |
| 354 | 384 | static inline unsigned long |
| 355 | 385 | raw_copy_to_user(void __user *to, const void *from, unsigned long n) |
| 356 | 386 | { |
| 357 | − unsigned long ret; | |
| 358 | − | |
| 359 | − allow_user_access(to, KUAP_WRITE); | |
| 360 | − ret = __copy_tofrom_user(to, (__force const void __user *)from, n); | |
| 361 | − prevent_user_access(KUAP_WRITE); | |
| 362 | − return ret; | |
| 387 | + return raw_copy_tofrom_user(to, (__force const void __user *)from, n, KUAP_WRITE); | |
| 363 | 388 | } |
| 364 | 389 | |
| 365 | 390 | unsigned long __arch_clear_user(void __user *addr, unsigned long size); |
| @@ -305,7 +305,6 @@ set_ivor: · 304 unmodified lines | ||
| 305 | 305 | * r12 is pointer to the pte |
| 306 | 306 | * r10 is the pshift from the PGD, if we're a hugepage |
| 307 | 307 | */ |
| 308 | −#ifdef CONFIG_PTE_64BIT | |
| 309 | 308 | #ifdef CONFIG_HUGETLB_PAGE |
| 310 | 309 | #define FIND_PTE \ |
| 311 | 310 | rlwinm r12, r13, 14, 18, 28; /* Compute pgdir/pmd offset */ \ |
| @@ -329,15 +328,6 @@ set_ivor: · 17 unmodified lines | ||
| 329 | 328 | rlwimi r12, r13, 23, 20, 28; /* Compute pte address */ \ |
| 330 | 329 | lwz r11, 4(r12); /* Get pte entry */ |
| 331 | 330 | #endif /* HUGEPAGE */ |
| 332 | −#else /* !PTE_64BIT */ | |
| 333 | −#define FIND_PTE \ | |
| 334 | − rlwimi r11, r13, 12, 20, 29; /* Create L1 (pgdir/pmd) address */ \ | |
| 335 | − lwz r11, 0(r11); /* Get L1 entry */ \ | |
| 336 | − rlwinm. r12, r11, 0, 0, 19; /* Extract L2 (pte) base address */ \ | |
| 337 | − beq 2f; /* Bail if no table */ \ | |
| 338 | − rlwimi r12, r13, 22, 20, 29; /* Compute PTE address */ \ | |
| 339 | − lwz r11, 0(r12); /* Get Linux PTE */ | |
| 340 | −#endif | |
| 341 | 331 | |
| 342 | 332 | /* |
| 343 | 333 | * Interrupt vector entry code |
| @@ -473,21 +463,15 @@ END_BTB_FLUSH_SECTION · 129 unmodified lines | ||
| 473 | 463 | 4: |
| 474 | 464 | FIND_PTE |
| 475 | 465 | |
| 476 | −#ifdef CONFIG_PTE_64BIT | |
| 477 | 466 | li r13,_PAGE_PRESENT|_PAGE_BAP_SR |
| 478 | 467 | oris r13,r13,_PAGE_ACCESSED@h |
| 479 | −#else | |
| 480 | − li r13,_PAGE_PRESENT|_PAGE_READ|_PAGE_ACCESSED | |
| 481 | −#endif | |
| 482 | 468 | andc. r13,r13,r11 /* Check permission */ |
| 483 | 469 | |
| 484 | −#ifdef CONFIG_PTE_64BIT | |
| 485 | 470 | #ifdef CONFIG_SMP |
| 486 | 471 | subf r13,r11,r12 /* create false data dep */ |
| 487 | 472 | lwzx r13,r11,r13 /* Get upper pte bits */ |
| 488 | 473 | #else |
| 489 | 474 | lwz r13,0(r12) /* Get upper pte bits */ |
| 490 | −#endif | |
| 491 | 475 | #endif |
| 492 | 476 | |
| 493 | 477 | bne 2f /* Bail if permission/valid mismatch */ |
| @@ -552,12 +536,8 @@ END_BTB_FLUSH_SECTION · 58 unmodified lines | ||
| 552 | 536 | |
| 553 | 537 | FIND_PTE |
| 554 | 538 | /* Make up the required permissions for kernel code */ |
| 555 | −#ifdef CONFIG_PTE_64BIT | |
| 556 | 539 | li r13,_PAGE_PRESENT | _PAGE_BAP_SX |
| 557 | 540 | oris r13,r13,_PAGE_ACCESSED@h |
| 558 | −#else | |
| 559 | − li r13,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC | |
| 560 | −#endif | |
| 561 | 541 | b 4f |
| 562 | 542 | |
| 563 | 543 | /* Get the PGD for the current thread */ |
| @@ -573,23 +553,17 @@ END_BTB_FLUSH_SECTION · 9 unmodified lines | ||
| 573 | 553 | |
| 574 | 554 | FIND_PTE |
| 575 | 555 | /* Make up the required permissions for user code */ |
| 576 | −#ifdef CONFIG_PTE_64BIT | |
| 577 | 556 | li r13,_PAGE_PRESENT | _PAGE_BAP_UX |
| 578 | 557 | oris r13,r13,_PAGE_ACCESSED@h |
| 579 | −#else | |
| 580 | − li r13,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC | |
| 581 | −#endif | |
| 582 | 558 | |
| 583 | 559 | 4: |
| 584 | 560 | andc. r13,r13,r11 /* Check permission */ |
| 585 | 561 | |
| 586 | −#ifdef CONFIG_PTE_64BIT | |
| 587 | 562 | #ifdef CONFIG_SMP |
| 588 | 563 | subf r13,r11,r12 /* create false data dep */ |
| 589 | 564 | lwzx r13,r11,r13 /* Get upper pte bits */ |
| 590 | 565 | #else |
| 591 | 566 | lwz r13,0(r12) /* Get upper pte bits */ |
| 592 | −#endif | |
| 593 | 567 | #endif |
| 594 | 568 | |
| 595 | 569 | bne 2f /* Bail if permission mismatch */ |
| @@ -683,7 +657,7 @@ interrupt_end: · 87 unmodified lines | ||
| 683 | 657 | * r10 - tsize encoding (if HUGETLB_PAGE) or available to use |
| 684 | 658 | * r11 - TLB (info from Linux PTE) |
| 685 | 659 | * r12 - available to use |
| 686 | − * r13 - upper bits of PTE (if PTE_64BIT) or available to use | |
| 660 | + * r13 - upper bits of PTE | |
| 687 | 661 | * CR5 - results of addr >= PAGE_OFFSET |
| 688 | 662 | * MAS0, MAS1 - loaded with proper value when we get here |
| 689 | 663 | * MAS2, MAS3 - will need additional info from Linux PTE |
| @@ -751,7 +725,6 @@ finish_tlb_load: · 61 unmodified lines | ||
| 751 | 725 | * here we (properly should) assume have the appropriate value. |
| 752 | 726 | */ |
| 753 | 727 | finish_tlb_load_cont: |
| 754 | −#ifdef CONFIG_PTE_64BIT | |
| 755 | 728 | rlwinm r12, r11, 32-2, 26, 31 /* Move in perm bits */ |
| 756 | 729 | andi. r10, r11, _PAGE_DIRTY |
| 757 | 730 | bne 1f |
| @@ -764,26 +737,9 @@ BEGIN_MMU_FTR_SECTION · 6 unmodified lines | ||
| 764 | 737 | srwi r10, r13, 12 /* grab RPN[12:31] */ |
| 765 | 738 | mtspr SPRN_MAS7, r10 |
| 766 | 739 | END_MMU_FTR_SECTION_IFSET(MMU_FTR_BIG_PHYS) |
| 767 | −#else | |
| 768 | − li r10, (_PAGE_EXEC | _PAGE_READ) | |
| 769 | − mr r13, r11 | |
| 770 | − rlwimi r10, r11, 31, 29, 29 /* extract _PAGE_DIRTY into SW */ | |
| 771 | − and r12, r11, r10 | |
| 772 | − mcrf cr0, cr5 /* Test for user page */ | |
| 773 | − slwi r10, r12, 1 | |
| 774 | − or r10, r10, r12 | |
| 775 | − rlwinm r10, r10, 0, ~_PAGE_EXEC /* Clear SX on user pages */ | |
| 776 | − isellt r12, r10, r12 | |
| 777 | − rlwimi r13, r12, 0, 20, 31 /* Get RPN from PTE, merge w/ perms */ | |
| 778 | − mtspr SPRN_MAS3, r13 | |
| 779 | −#endif | |
| 780 | 740 | |
| 781 | 741 | mfspr r12, SPRN_MAS2 |
| 782 | −#ifdef CONFIG_PTE_64BIT | |
| 783 | 742 | rlwimi r12, r11, 32-19, 27, 31 /* extract WIMGE from pte */ |
| 784 | −#else | |
| 785 | − rlwimi r12, r11, 26, 27, 31 /* extract WIMGE from pte */ | |
| 786 | −#endif | |
| 787 | 743 | #ifdef CONFIG_HUGETLB_PAGE |
| 788 | 744 | beq 6, 3f /* don't mask if page isn't huge */ |
| 789 | 745 | li r13, 1 |
| @@ -1159,7 +1159,7 @@ spapr_tce_platform_iommu_attach_dev(struct iommu_domain *platform_domain, · 1158 unmodified lines | ||
| 1159 | 1159 | struct device *dev, |
| 1160 | 1160 | struct iommu_domain *old) |
| 1161 | 1161 | { |
| 1162 | − struct iommu_domain *domain = iommu_get_domain_for_dev(dev); | |
| 1162 | + struct iommu_domain *domain = iommu_driver_get_domain_for_dev(dev); | |
| 1163 | 1163 | struct iommu_table_group *table_group; |
| 1164 | 1164 | struct iommu_group *grp; |
| 1165 | 1165 | |
| @@ -212,6 +212,13 @@ struct pci_dev *of_create_pci_dev(struct device_node *node, · 211 unmodified lines | ||
| 212 | 212 | dev->error_state = pci_channel_io_normal; |
| 213 | 213 | dev->dma_mask = 0xffffffff; |
| 214 | 214 | |
| 215 | + /* | |
| 216 | + * Assume 64-bit addresses for MSI initially. Will be changed to 32-bit | |
| 217 | + * if MSI (rather than MSI-X) capability does not have | |
| 218 | + * PCI_MSI_FLAGS_64BIT. Can also be overridden by driver. | |
| 219 | + */ | |
| 220 | + dev->msi_addr_mask = DMA_BIT_MASK(64); | |
| 221 | + | |
| 215 | 222 | /* Early fixups, before probing the BARs */ |
| 216 | 223 | pci_fixup_device(pci_fixup_early, dev); |
| 217 | 224 | |
| @@ -2893,7 +2893,8 @@ static void __init fixup_device_tree_pmac(void) · 2892 unmodified lines | ||
| 2893 | 2893 | for (node = 0; prom_next_node(&node); ) { |
| 2894 | 2894 | type[0] = '\0'; |
| 2895 | 2895 | prom_getprop(node, "device_type", type, sizeof(type)); |
| 2896 | − if (prom_strcmp(type, "escc") && prom_strcmp(type, "i2s")) | |
| 2896 | + if (prom_strcmp(type, "escc") && prom_strcmp(type, "i2s") && | |
| 2897 | + prom_strcmp(type, "media-bay")) | |
| 2897 | 2898 | continue; |
| 2898 | 2899 | |
| 2899 | 2900 | if (prom_getproplen(node, "#size-cells") != PROM_ERROR) |
| @@ -35,7 +35,6 @@ · 34 unmodified lines | ||
| 35 | 35 | #include <linux/of_irq.h> |
| 36 | 36 | #include <linux/hugetlb.h> |
| 37 | 37 | #include <linux/pgtable.h> |
| 38 | −#include <asm/kexec.h> | |
| 39 | 38 | #include <asm/io.h> |
| 40 | 39 | #include <asm/paca.h> |
| 41 | 40 | #include <asm/processor.h> |
| @@ -995,15 +994,6 @@ void __init setup_arch(char **cmdline_p) · 953 unmodified lines | ||
| 995 | 994 | |
| 996 | 995 | initmem_init(); |
| 997 | 996 | |
| 998 | − /* | |
| 999 | − * Reserve large chunks of memory for use by CMA for kdump, fadump, KVM and | |
| 1000 | − * hugetlb. These must be called after initmem_init(), so that | |
| 1001 | − * pageblock_order is initialised. | |
| 1002 | − */ | |
| 1003 | − fadump_cma_init(); | |
| 1004 | − kdump_cma_reserve(); | |
| 1005 | − kvm_cma_reserve(); | |
| 1006 | − | |
| 1007 | 997 | early_memtest(min_low_pfn << PAGE_SHIFT, max_low_pfn << PAGE_SHIFT); |
| 1008 | 998 | |
| 1009 | 999 | if (ppc_md.setup_arch) |
| @@ -37,11 +37,29 @@ unsigned long ftrace_call_adjust(unsigned long addr) · 36 unmodified lines | ||
| 37 | 37 | if (addr >= (unsigned long)__exittext_begin && addr < (unsigned long)__exittext_end) |
| 38 | 38 | return 0; |
| 39 | 39 | |
| 40 | − if (IS_ENABLED(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY) && | |
| 41 | − !IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) { | |
| 42 | − addr += MCOUNT_INSN_SIZE; | |
| 43 | − if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS)) | |
| 40 | + if (IS_ENABLED(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY)) { | |
| 41 | + if (!IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) { | |
| 44 | 42 | addr += MCOUNT_INSN_SIZE; |
| 43 | + if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS)) | |
| 44 | + addr += MCOUNT_INSN_SIZE; | |
| 45 | + } else if (IS_ENABLED(CONFIG_CC_IS_CLANG) && IS_ENABLED(CONFIG_PPC64)) { | |
| 46 | + /* | |
| 47 | + * addr points to global entry point though the NOP was emitted at local | |
| 48 | + * entry point due to https://github.com/llvm/llvm-project/issues/163706 | |
| 49 | + * Handle that here with ppc_function_entry() for kernel symbols while | |
| 50 | + * adjusting module addresses in the else case, by looking for the below | |
| 51 | + * module global entry point sequence: | |
| 52 | + * ld r2, -8(r12) | |
| 53 | + * add r2, r2, r12 | |
| 54 | + */ | |
| 55 | + if (is_kernel_text(addr) || is_kernel_inittext(addr)) | |
| 56 | + addr = ppc_function_entry((void *)addr); | |
| 57 | + else if ((ppc_inst_val(ppc_inst_read((u32 *)addr)) == | |
| 58 | + PPC_RAW_LD(_R2, _R12, -8)) && | |
| 59 | + (ppc_inst_val(ppc_inst_read((u32 *)(addr+4))) == | |
| 60 | + PPC_RAW_ADD(_R2, _R2, _R12))) | |
| 61 | + addr += 8; | |
| 62 | + } | |
| 45 | 63 | } |
| 46 | 64 | |
| 47 | 65 | return addr; |
| @@ -397,6 +397,7 @@ SECTIONS · 396 unmodified lines | ||
| 397 | 397 | _end = . ; |
| 398 | 398 | |
| 399 | 399 | DWARF_DEBUG |
| 400 | + MODINFO | |
| 400 | 401 | ELF_DETAILS |
| 401 | 402 | |
| 402 | 403 | DISCARDS |
| @@ -23,6 +23,7 @@ · 22 unmodified lines | ||
| 23 | 23 | #include <asm/firmware.h> |
| 24 | 24 | |
| 25 | 25 | #define cpu_to_be_ulong __PASTE(cpu_to_be, BITS_PER_LONG) |
| 26 | +#define __be_word __PASTE(__be, BITS_PER_LONG) | |
| 26 | 27 | |
| 27 | 28 | #ifdef CONFIG_CRASH_DUMP |
| 28 | 29 | void machine_crash_shutdown(struct pt_regs *regs) |
| @@ -146,25 +147,25 @@ int __init overlaps_crashkernel(unsigned long start, unsigned long size) · 117 unmodified lines | ||
| 146 | 147 | } |
| 147 | 148 | |
| 148 | 149 | /* Values we need to export to the second kernel via the device tree. */ |
| 149 | −static phys_addr_t crashk_base; | |
| 150 | −static phys_addr_t crashk_size; | |
| 151 | −static unsigned long long mem_limit; | |
| 150 | +static __be_word crashk_base; | |
| 151 | +static __be_word crashk_size; | |
| 152 | +static __be_word mem_limit; | |
| 152 | 153 | |
| 153 | 154 | static struct property crashk_base_prop = { |
| 154 | 155 | .name = "linux,crashkernel-base", |
| 155 | − .length = sizeof(phys_addr_t), | |
| 156 | + .length = sizeof(__be_word), | |
| 156 | 157 | .value = &crashk_base |
| 157 | 158 | }; |
| 158 | 159 | |
| 159 | 160 | static struct property crashk_size_prop = { |
| 160 | 161 | .name = "linux,crashkernel-size", |
| 161 | − .length = sizeof(phys_addr_t), | |
| 162 | + .length = sizeof(__be_word), | |
| 162 | 163 | .value = &crashk_size, |
| 163 | 164 | }; |
| 164 | 165 | |
| 165 | 166 | static struct property memory_limit_prop = { |
| 166 | 167 | .name = "linux,memory-limit", |
| 167 | − .length = sizeof(unsigned long long), | |
| 168 | + .length = sizeof(__be_word), | |
| 168 | 169 | .value = &mem_limit, |
| 169 | 170 | }; |
| 170 | 171 | |
| @@ -193,11 +194,11 @@ static void __init export_crashk_values(struct device_node *node) · 22 unmodified lines | ||
| 193 | 194 | } |
| 194 | 195 | #endif /* CONFIG_CRASH_RESERVE */ |
| 195 | 196 | |
| 196 | −static phys_addr_t kernel_end; | |
| 197 | +static __be_word kernel_end; | |
| 197 | 198 | |
| 198 | 199 | static struct property kernel_end_prop = { |
| 199 | 200 | .name = "linux,kernel-end", |
| 200 | − .length = sizeof(phys_addr_t), | |
| 201 | + .length = sizeof(__be_word), | |
| 201 | 202 | .value = &kernel_end, |
| 202 | 203 | }; |
| 203 | 204 | |
| @@ -450,6 +450,11 @@ static int load_elfcorehdr_segment(struct kimage *image, struct kexec_buf *kbuf) · 449 unmodified lines | ||
| 450 | 450 | kbuf->buffer = headers; |
| 451 | 451 | kbuf->mem = KEXEC_BUF_MEM_UNKNOWN; |
| 452 | 452 | kbuf->bufsz = headers_sz; |
| 453 | + | |
| 454 | + /* | |
| 455 | + * Account for extra space required to accommodate additional memory | |
| 456 | + * ranges in elfcorehdr due to memory hotplug events. | |
| 457 | + */ | |
| 453 | 458 | kbuf->memsz = headers_sz + kdump_extra_elfcorehdr_size(cmem); |
| 454 | 459 | kbuf->top_down = false; |
| 455 | 460 | |
| @@ -460,7 +465,14 @@ static int load_elfcorehdr_segment(struct kimage *image, struct kexec_buf *kbuf) · 4 unmodified lines | ||
| 460 | 465 | } |
| 461 | 466 | |
| 462 | 467 | image->elf_load_addr = kbuf->mem; |
| 463 | − image->elf_headers_sz = headers_sz; | |
| 468 | + | |
| 469 | + /* | |
| 470 | + * If CONFIG_CRASH_HOTPLUG is enabled, the elfcorehdr kexec segment | |
| 471 | + * memsz can be larger than bufsz. Always initialize elf_headers_sz | |
| 472 | + * with memsz. This ensures the correct size is reserved for elfcorehdr | |
| 473 | + * memory in the FDT prepared for kdump. | |
| 474 | + */ | |
| 475 | + image->elf_headers_sz = kbuf->memsz; | |
| 464 | 476 | image->elf_headers = headers; |
| 465 | 477 | out: |
| 466 | 478 | kfree(cmem); |
| @@ -38,7 +38,6 @@ config KVM_BOOK3S_64_HANDLER · 37 unmodified lines | ||
| 38 | 38 | config KVM_BOOK3S_PR_POSSIBLE |
| 39 | 39 | bool |
| 40 | 40 | select KVM_MMIO |
| 41 | − select KVM_GENERIC_MMU_NOTIFIER | |
| 42 | 41 | |
| 43 | 42 | config KVM_BOOK3S_HV_POSSIBLE |
| 44 | 43 | bool |
| @@ -81,7 +80,6 @@ config KVM_BOOK3S_64_HV · 36 unmodified lines | ||
| 81 | 80 | tristate "KVM for POWER7 and later using hypervisor mode in host" |
| 82 | 81 | depends on KVM_BOOK3S_64 && PPC_POWERNV |
| 83 | 82 | select KVM_BOOK3S_HV_POSSIBLE |
| 84 | − select KVM_GENERIC_MMU_NOTIFIER | |
| 85 | 83 | select KVM_BOOK3S_HV_PMU |
| 86 | 84 | select CMA |
| 87 | 85 | help |
| @@ -203,7 +201,6 @@ config KVM_E500V2 · 115 unmodified lines | ||
| 203 | 201 | depends on !CONTEXT_TRACKING_USER |
| 204 | 202 | select KVM |
| 205 | 203 | select KVM_MMIO |
| 206 | − select KVM_GENERIC_MMU_NOTIFIER | |
| 207 | 204 | help |
| 208 | 205 | Support running unmodified E500 guest kernels in virtual machines on |
| 209 | 206 | E500v2 host processors. |
| @@ -220,7 +217,6 @@ config KVM_E500MC · 10 unmodified lines | ||
| 220 | 217 | select KVM |
| 221 | 218 | select KVM_MMIO |
| 222 | 219 | select KVM_BOOKE_HV |
| 223 | − select KVM_GENERIC_MMU_NOTIFIER | |
| 224 | 220 | help |
| 225 | 221 | Support running unmodified E500MC/E5500/E6500 guest kernels in |
| 226 | 222 | virtual machines on E500MC/E5500/E6500 host processors. |
| @@ -38,7 +38,7 @@ · 37 unmodified lines | ||
| 38 | 38 | |
| 39 | 39 | /* #define EXIT_DEBUG */ |
| 40 | 40 | |
| 41 | −const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 41 | +const struct kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 42 | 42 | KVM_GENERIC_VM_STATS(), |
| 43 | 43 | STATS_DESC_ICOUNTER(VM, num_2M_pages), |
| 44 | 44 | STATS_DESC_ICOUNTER(VM, num_1G_pages) |
| @@ -53,7 +53,7 @@ const struct kvm_stats_header kvm_vm_stats_header = { · 8 unmodified lines | ||
| 53 | 53 | sizeof(kvm_vm_stats_desc), |
| 54 | 54 | }; |
| 55 | 55 | |
| 56 | −const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 56 | +const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 57 | 57 | KVM_GENERIC_VCPU_STATS(), |
| 58 | 58 | STATS_DESC_COUNTER(VCPU, sum_exits), |
| 59 | 59 | STATS_DESC_COUNTER(VCPU, mmio_exits), |
| @@ -36,7 +36,7 @@ · 35 unmodified lines | ||
| 36 | 36 | |
| 37 | 37 | unsigned long kvmppc_booke_handlers; |
| 38 | 38 | |
| 39 | −const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 39 | +const struct kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 40 | 40 | KVM_GENERIC_VM_STATS(), |
| 41 | 41 | STATS_DESC_ICOUNTER(VM, num_2M_pages), |
| 42 | 42 | STATS_DESC_ICOUNTER(VM, num_1G_pages) |
| @@ -51,7 +51,7 @@ const struct kvm_stats_header kvm_vm_stats_header = { · 8 unmodified lines | ||
| 51 | 51 | sizeof(kvm_vm_stats_desc), |
| 52 | 52 | }; |
| 53 | 53 | |
| 54 | −const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 54 | +const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 55 | 55 | KVM_GENERIC_VCPU_STATS(), |
| 56 | 56 | STATS_DESC_COUNTER(VCPU, sum_exits), |
| 57 | 57 | STATS_DESC_COUNTER(VCPU, mmio_exits), |
| @@ -39,15 +39,11 @@ enum vcpu_ftr { · 38 unmodified lines | ||
| 39 | 39 | /* bits [6-5] MAS2_X1 and MAS2_X0 and [4-0] bits for WIMGE */ |
| 40 | 40 | #define E500_TLB_MAS2_ATTR (0x7f) |
| 41 | 41 | |
| 42 | −struct tlbe_ref { | |
| 42 | +struct tlbe_priv { | |
| 43 | 43 | kvm_pfn_t pfn; /* valid only for TLB0, except briefly */ |
| 44 | 44 | unsigned int flags; /* E500_TLB_* */ |
| 45 | 45 | }; |
| 46 | 46 | |
| 47 | −struct tlbe_priv { | |
| 48 | − struct tlbe_ref ref; | |
| 49 | −}; | |
| 50 | − | |
| 51 | 47 | #ifdef CONFIG_KVM_E500V2 |
| 52 | 48 | struct vcpu_id_table; |
| 53 | 49 | #endif |
| @@ -920,12 +920,12 @@ int kvmppc_e500_tlb_init(struct kvmppc_vcpu_e500 *vcpu_e500) · 919 unmodified lines | ||
| 920 | 920 | vcpu_e500->gtlb_offset[0] = 0; |
| 921 | 921 | vcpu_e500->gtlb_offset[1] = KVM_E500_TLB0_SIZE; |
| 922 | 922 | |
| 923 | − vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_ref, | |
| 923 | + vcpu_e500->gtlb_priv[0] = kzalloc_objs(struct tlbe_priv, | |
| 924 | 924 | vcpu_e500->gtlb_params[0].entries); |
| 925 | 925 | if (!vcpu_e500->gtlb_priv[0]) |
| 926 | 926 | goto free_vcpu; |
| 927 | 927 | |
| 928 | − vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_ref, | |
| 928 | + vcpu_e500->gtlb_priv[1] = kzalloc_objs(struct tlbe_priv, | |
| 929 | 929 | vcpu_e500->gtlb_params[1].entries); |
| 930 | 930 | if (!vcpu_e500->gtlb_priv[1]) |
| 931 | 931 | goto free_vcpu; |
| @@ -189,16 +189,16 @@ void inval_gtlbe_on_host(struct kvmppc_vcpu_e500 *vcpu_e500, int tlbsel, · 188 unmodified lines | ||
| 189 | 189 | { |
| 190 | 190 | struct kvm_book3e_206_tlb_entry *gtlbe = |
| 191 | 191 | get_entry(vcpu_e500, tlbsel, esel); |
| 192 | − struct tlbe_ref *ref = &vcpu_e500->gtlb_priv[tlbsel][esel].ref; | |
| 192 | + struct tlbe_priv *tlbe = &vcpu_e500->gtlb_priv[tlbsel][esel]; | |
| 193 | 193 | |
| 194 | 194 | /* Don't bother with unmapped entries */ |
| 195 | − if (!(ref->flags & E500_TLB_VALID)) { | |
| 196 | − WARN(ref->flags & (E500_TLB_BITMAP | E500_TLB_TLB0), | |
| 197 | − "%s: flags %x\n", __func__, ref->flags); | |
| 195 | + if (!(tlbe->flags & E500_TLB_VALID)) { | |
| 196 | + WARN(tlbe->flags & (E500_TLB_BITMAP | E500_TLB_TLB0), | |
| 197 | + "%s: flags %x\n", __func__, tlbe->flags); | |
| 198 | 198 | WARN_ON(tlbsel == 1 && vcpu_e500->g2h_tlb1_map[esel]); |
| 199 | 199 | } |
| 200 | 200 | |
| 201 | − if (tlbsel == 1 && ref->flags & E500_TLB_BITMAP) { | |
| 201 | + if (tlbsel == 1 && tlbe->flags & E500_TLB_BITMAP) { | |
| 202 | 202 | u64 tmp = vcpu_e500->g2h_tlb1_map[esel]; |
| 203 | 203 | int hw_tlb_indx; |
| 204 | 204 | unsigned long flags; |
| @@ -216,28 +216,28 @@ void inval_gtlbe_on_host(struct kvmppc_vcpu_e500 *vcpu_e500, int tlbsel, · 11 unmodified lines | ||
| 216 | 216 | } |
| 217 | 217 | mb(); |
| 218 | 218 | vcpu_e500->g2h_tlb1_map[esel] = 0; |
| 219 | − ref->flags &= ~(E500_TLB_BITMAP | E500_TLB_VALID); | |
| 219 | + tlbe->flags &= ~(E500_TLB_BITMAP | E500_TLB_VALID); | |
| 220 | 220 | local_irq_restore(flags); |
| 221 | 221 | } |
| 222 | 222 | |
| 223 | − if (tlbsel == 1 && ref->flags & E500_TLB_TLB0) { | |
| 223 | + if (tlbsel == 1 && tlbe->flags & E500_TLB_TLB0) { | |
| 224 | 224 | /* |
| 225 | 225 | * TLB1 entry is backed by 4k pages. This should happen |
| 226 | 226 | * rarely and is not worth optimizing. Invalidate everything. |
| 227 | 227 | */ |
| 228 | 228 | kvmppc_e500_tlbil_all(vcpu_e500); |
| 229 | − ref->flags &= ~(E500_TLB_TLB0 | E500_TLB_VALID); | |
| 229 | + tlbe->flags &= ~(E500_TLB_TLB0 | E500_TLB_VALID); | |
| 230 | 230 | } |
| 231 | 231 | |
| 232 | 232 | /* |
| 233 | 233 | * If TLB entry is still valid then it's a TLB0 entry, and thus |
| 234 | 234 | * backed by at most one host tlbe per shadow pid |
| 235 | 235 | */ |
| 236 | − if (ref->flags & E500_TLB_VALID) | |
| 236 | + if (tlbe->flags & E500_TLB_VALID) | |
| 237 | 237 | kvmppc_e500_tlbil_one(vcpu_e500, gtlbe); |
| 238 | 238 | |
| 239 | 239 | /* Mark the TLB as not backed by the host anymore */ |
| 240 | − ref->flags = 0; | |
| 240 | + tlbe->flags = 0; | |
| 241 | 241 | } |
| 242 | 242 | |
| 243 | 243 | static inline int tlbe_is_writable(struct kvm_book3e_206_tlb_entry *tlbe) |
| @@ -245,26 +245,26 @@ static inline int tlbe_is_writable(struct kvm_book3e_206_tlb_entry *tlbe) · 1 unmodified line | ||
| 245 | 245 | return tlbe->mas7_3 & (MAS3_SW|MAS3_UW); |
| 246 | 246 | } |
| 247 | 247 | |
| 248 | −static inline void kvmppc_e500_ref_setup(struct tlbe_ref *ref, | |
| 249 | − struct kvm_book3e_206_tlb_entry *gtlbe, | |
| 250 | − kvm_pfn_t pfn, unsigned int wimg, | |
| 251 | − bool writable) | |
| 248 | +static inline void kvmppc_e500_tlbe_setup(struct tlbe_priv *tlbe, | |
| 249 | + struct kvm_book3e_206_tlb_entry *gtlbe, | |
| 250 | + kvm_pfn_t pfn, unsigned int wimg, | |
| 251 | + bool writable) | |
| 252 | 252 | { |
| 253 | − ref->pfn = pfn; | |
| 254 | − ref->flags = E500_TLB_VALID; | |
| 253 | + tlbe->pfn = pfn; | |
| 254 | + tlbe->flags = E500_TLB_VALID; | |
| 255 | 255 | if (writable) |
| 256 | − ref->flags |= E500_TLB_WRITABLE; | |
| 256 | + tlbe->flags |= E500_TLB_WRITABLE; | |
| 257 | 257 | |
| 258 | 258 | /* Use guest supplied MAS2_G and MAS2_E */ |
| 259 | − ref->flags |= (gtlbe->mas2 & MAS2_ATTRIB_MASK) | wimg; | |
| 259 | + tlbe->flags |= (gtlbe->mas2 & MAS2_ATTRIB_MASK) | wimg; | |
| 260 | 260 | } |
| 261 | 261 | |
| 262 | −static inline void kvmppc_e500_ref_release(struct tlbe_ref *ref) | |
| 262 | +static inline void kvmppc_e500_tlbe_release(struct tlbe_priv *tlbe) | |
| 263 | 263 | { |
| 264 | − if (ref->flags & E500_TLB_VALID) { | |
| 264 | + if (tlbe->flags & E500_TLB_VALID) { | |
| 265 | 265 | /* FIXME: don't log bogus pfn for TLB1 */ |
| 266 | − trace_kvm_booke206_ref_release(ref->pfn, ref->flags); | |
| 267 | − ref->flags = 0; | |
| 266 | + trace_kvm_booke206_ref_release(tlbe->pfn, tlbe->flags); | |
| 267 | + tlbe->flags = 0; | |
| 268 | 268 | } |
| 269 | 269 | } |
| 270 | 270 | |
| @@ -284,11 +284,8 @@ static void clear_tlb_privs(struct kvmppc_vcpu_e500 *vcpu_e500) · 13 unmodified lines | ||
| 284 | 284 | int i; |
| 285 | 285 | |
| 286 | 286 | for (tlbsel = 0; tlbsel <= 1; tlbsel++) { |
| 287 | − for (i = 0; i < vcpu_e500->gtlb_params[tlbsel].entries; i++) { | |
| 288 | − struct tlbe_ref *ref = | |
| 289 | − &vcpu_e500->gtlb_priv[tlbsel][i].ref; | |
| 290 | − kvmppc_e500_ref_release(ref); | |
| 291 | − } | |
| 287 | + for (i = 0; i < vcpu_e500->gtlb_params[tlbsel].entries; i++) | |
| 288 | + kvmppc_e500_tlbe_release(&vcpu_e500->gtlb_priv[tlbsel][i]); | |
| 292 | 289 | } |
| 293 | 290 | } |
| 294 | 291 | |
| @@ -304,18 +301,18 @@ void kvmppc_core_flush_tlb(struct kvm_vcpu *vcpu) · 9 unmodified lines | ||
| 304 | 301 | static void kvmppc_e500_setup_stlbe( |
| 305 | 302 | struct kvm_vcpu *vcpu, |
| 306 | 303 | struct kvm_book3e_206_tlb_entry *gtlbe, |
| 307 | − int tsize, struct tlbe_ref *ref, u64 gvaddr, | |
| 304 | + int tsize, struct tlbe_priv *tlbe, u64 gvaddr, | |
| 308 | 305 | struct kvm_book3e_206_tlb_entry *stlbe) |
| 309 | 306 | { |
| 310 | − kvm_pfn_t pfn = ref->pfn; | |
| 307 | + kvm_pfn_t pfn = tlbe->pfn; | |
| 311 | 308 | u32 pr = vcpu->arch.shared->msr & MSR_PR; |
| 312 | − bool writable = !!(ref->flags & E500_TLB_WRITABLE); | |
| 309 | + bool writable = !!(tlbe->flags & E500_TLB_WRITABLE); | |
| 313 | 310 | |
| 314 | − BUG_ON(!(ref->flags & E500_TLB_VALID)); | |
| 311 | + BUG_ON(!(tlbe->flags & E500_TLB_VALID)); | |
| 315 | 312 | |
| 316 | 313 | /* Force IPROT=0 for all guest mappings. */ |
| 317 | 314 | stlbe->mas1 = MAS1_TSIZE(tsize) | get_tlb_sts(gtlbe) | MAS1_VALID; |
| 318 | − stlbe->mas2 = (gvaddr & MAS2_EPN) | (ref->flags & E500_TLB_MAS2_ATTR); | |
| 315 | + stlbe->mas2 = (gvaddr & MAS2_EPN) | (tlbe->flags & E500_TLB_MAS2_ATTR); | |
| 319 | 316 | stlbe->mas7_3 = ((u64)pfn << PAGE_SHIFT) | |
| 320 | 317 | e500_shadow_mas3_attrib(gtlbe->mas7_3, writable, pr); |
| 321 | 318 | } |
| @@ -323,7 +320,7 @@ static void kvmppc_e500_setup_stlbe( · 1 unmodified line | ||
| 323 | 320 | static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, |
| 324 | 321 | u64 gvaddr, gfn_t gfn, struct kvm_book3e_206_tlb_entry *gtlbe, |
| 325 | 322 | int tlbsel, struct kvm_book3e_206_tlb_entry *stlbe, |
| 326 | − struct tlbe_ref *ref) | |
| 323 | + struct tlbe_priv *tlbe) | |
| 327 | 324 | { |
| 328 | 325 | struct kvm_memory_slot *slot; |
| 329 | 326 | unsigned int psize; |
| @@ -455,9 +452,9 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, · 125 unmodified lines | ||
| 455 | 452 | } |
| 456 | 453 | } |
| 457 | 454 | |
| 458 | − kvmppc_e500_ref_setup(ref, gtlbe, pfn, wimg, writable); | |
| 455 | + kvmppc_e500_tlbe_setup(tlbe, gtlbe, pfn, wimg, writable); | |
| 459 | 456 | kvmppc_e500_setup_stlbe(&vcpu_e500->vcpu, gtlbe, tsize, |
| 460 | − ref, gvaddr, stlbe); | |
| 457 | + tlbe, gvaddr, stlbe); | |
| 461 | 458 | writable = tlbe_is_writable(stlbe); |
| 462 | 459 | |
| 463 | 460 | /* Clear i-cache for new pages */ |
| @@ -474,17 +471,17 @@ static int kvmppc_e500_tlb0_map(struct kvmppc_vcpu_e500 *vcpu_e500, int esel, · 10 unmodified lines | ||
| 474 | 471 | struct kvm_book3e_206_tlb_entry *stlbe) |
| 475 | 472 | { |
| 476 | 473 | struct kvm_book3e_206_tlb_entry *gtlbe; |
| 477 | − struct tlbe_ref *ref; | |
| 474 | + struct tlbe_priv *tlbe; | |
| 478 | 475 | int stlbsel = 0; |
| 479 | 476 | int sesel = 0; |
| 480 | 477 | int r; |
| 481 | 478 | |
| 482 | 479 | gtlbe = get_entry(vcpu_e500, 0, esel); |
| 483 | − ref = &vcpu_e500->gtlb_priv[0][esel].ref; | |
| 480 | + tlbe = &vcpu_e500->gtlb_priv[0][esel]; | |
| 484 | 481 | |
| 485 | 482 | r = kvmppc_e500_shadow_map(vcpu_e500, get_tlb_eaddr(gtlbe), |
| 486 | 483 | get_tlb_raddr(gtlbe) >> PAGE_SHIFT, |
| 487 | − gtlbe, 0, stlbe, ref); | |
| 484 | + gtlbe, 0, stlbe, tlbe); | |
| 488 | 485 | if (r) |
| 489 | 486 | return r; |
| 490 | 487 | |
| @@ -494,7 +491,7 @@ static int kvmppc_e500_tlb0_map(struct kvmppc_vcpu_e500 *vcpu_e500, int esel, · 3 unmodified lines | ||
| 494 | 491 | } |
| 495 | 492 | |
| 496 | 493 | static int kvmppc_e500_tlb1_map_tlb1(struct kvmppc_vcpu_e500 *vcpu_e500, |
| 497 | − struct tlbe_ref *ref, | |
| 494 | + struct tlbe_priv *tlbe, | |
| 498 | 495 | int esel) |
| 499 | 496 | { |
| 500 | 497 | unsigned int sesel = vcpu_e500->host_tlb1_nv++; |
| @@ -507,10 +504,10 @@ static int kvmppc_e500_tlb1_map_tlb1(struct kvmppc_vcpu_e500 *vcpu_e500, · 6 unmodified lines | ||
| 507 | 504 | vcpu_e500->g2h_tlb1_map[idx] &= ~(1ULL << sesel); |
| 508 | 505 | } |
| 509 | 506 | |
| 510 | − vcpu_e500->gtlb_priv[1][esel].ref.flags |= E500_TLB_BITMAP; | |
| 507 | + vcpu_e500->gtlb_priv[1][esel].flags |= E500_TLB_BITMAP; | |
| 511 | 508 | vcpu_e500->g2h_tlb1_map[esel] |= (u64)1 << sesel; |
| 512 | 509 | vcpu_e500->h2g_tlb1_rmap[sesel] = esel + 1; |
| 513 | − WARN_ON(!(ref->flags & E500_TLB_VALID)); | |
| 510 | + WARN_ON(!(tlbe->flags & E500_TLB_VALID)); | |
| 514 | 511 | |
| 515 | 512 | return sesel; |
| 516 | 513 | } |
| @@ -522,24 +519,24 @@ static int kvmppc_e500_tlb1_map(struct kvmppc_vcpu_e500 *vcpu_e500, · 5 unmodified lines | ||
| 522 | 519 | u64 gvaddr, gfn_t gfn, struct kvm_book3e_206_tlb_entry *gtlbe, |
| 523 | 520 | struct kvm_book3e_206_tlb_entry *stlbe, int esel) |
| 524 | 521 | { |
| 525 | − struct tlbe_ref *ref = &vcpu_e500->gtlb_priv[1][esel].ref; | |
| 522 | + struct tlbe_priv *tlbe = &vcpu_e500->gtlb_priv[1][esel]; | |
| 526 | 523 | int sesel; |
| 527 | 524 | int r; |
| 528 | 525 | |
| 529 | 526 | r = kvmppc_e500_shadow_map(vcpu_e500, gvaddr, gfn, gtlbe, 1, stlbe, |
| 530 | − ref); | |
| 527 | + tlbe); | |
| 531 | 528 | if (r) |
| 532 | 529 | return r; |
| 533 | 530 | |
| 534 | 531 | /* Use TLB0 when we can only map a page with 4k */ |
| 535 | 532 | if (get_tlb_tsize(stlbe) == BOOK3E_PAGESZ_4K) { |
| 536 | − vcpu_e500->gtlb_priv[1][esel].ref.flags |= E500_TLB_TLB0; | |
| 533 | + vcpu_e500->gtlb_priv[1][esel].flags |= E500_TLB_TLB0; | |
| 537 | 534 | write_stlbe(vcpu_e500, gtlbe, stlbe, 0, 0); |
| 538 | 535 | return 0; |
| 539 | 536 | } |
| 540 | 537 | |
| 541 | 538 | /* Otherwise map into TLB1 */ |
| 542 | − sesel = kvmppc_e500_tlb1_map_tlb1(vcpu_e500, ref, esel); | |
| 539 | + sesel = kvmppc_e500_tlb1_map_tlb1(vcpu_e500, tlbe, esel); | |
| 543 | 540 | write_stlbe(vcpu_e500, gtlbe, stlbe, 1, sesel); |
| 544 | 541 | |
| 545 | 542 | return 0; |
| @@ -561,11 +558,11 @@ void kvmppc_mmu_map(struct kvm_vcpu *vcpu, u64 eaddr, gpa_t gpaddr, · 15 unmodified lines | ||
| 561 | 558 | priv = &vcpu_e500->gtlb_priv[tlbsel][esel]; |
| 562 | 559 | |
| 563 | 560 | /* Triggers after clear_tlb_privs or on initial mapping */ |
| 564 | − if (!(priv->ref.flags & E500_TLB_VALID)) { | |
| 561 | + if (!(priv->flags & E500_TLB_VALID)) { | |
| 565 | 562 | kvmppc_e500_tlb0_map(vcpu_e500, esel, &stlbe); |
| 566 | 563 | } else { |
| 567 | 564 | kvmppc_e500_setup_stlbe(vcpu, gtlbe, BOOK3E_PAGESZ_4K, |
| 568 | − &priv->ref, eaddr, &stlbe); | |
| 565 | + priv, eaddr, &stlbe); | |
| 569 | 566 | write_stlbe(vcpu_e500, gtlbe, &stlbe, 0, 0); |
| 570 | 567 | } |
| 571 | 568 | break; |
| @@ -623,12 +623,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 622 unmodified lines | ||
| 623 | 623 | r = !!(hv_enabled && kvmppc_hv_ops->enable_nested && |
| 624 | 624 | !kvmppc_hv_ops->enable_nested(NULL)); |
| 625 | 625 | break; |
| 626 | −#endif | |
| 627 | − case KVM_CAP_SYNC_MMU: | |
| 628 | − BUILD_BUG_ON(!IS_ENABLED(CONFIG_KVM_GENERIC_MMU_NOTIFIER)); | |
| 629 | − r = 1; | |
| 630 | − break; | |
| 631 | −#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE | |
| 632 | 626 | case KVM_CAP_PPC_HTAB_FD: |
| 633 | 627 | r = hv_enabled; |
| 634 | 628 | break; |
| @@ -562,3 +562,4 @@ exc; std r10,32(3) · 561 unmodified lines | ||
| 562 | 562 | li r5,4096 |
| 563 | 563 | b .Ldst_aligned |
| 564 | 564 | EXPORT_SYMBOL(__copy_tofrom_user) |
| 565 | +EXPORT_SYMBOL(__copy_tofrom_user_base) | |
| @@ -5,13 +5,9 @@ · 4 unmodified lines | ||
| 5 | 5 | * |
| 6 | 6 | * Author: Anton Blanchard <anton@au.ibm.com> |
| 7 | 7 | */ |
| 8 | +#include <linux/export.h> | |
| 8 | 9 | #include <asm/ppc_asm.h> |
| 9 | 10 | |
| 10 | −#ifndef SELFTEST_CASE | |
| 11 | −/* 0 == don't use VMX, 1 == use VMX */ | |
| 12 | −#define SELFTEST_CASE 0 | |
| 13 | −#endif | |
| 14 | − | |
| 15 | 11 | #ifdef __BIG_ENDIAN__ |
| 16 | 12 | #define LVS(VRT,RA,RB) lvsl VRT,RA,RB |
| 17 | 13 | #define VPERM(VRT,VRA,VRB,VRC) vperm VRT,VRA,VRB,VRC |
| @@ -47,10 +43,14 @@ · 29 unmodified lines | ||
| 47 | 43 | ld r15,STK_REG(R15)(r1) |
| 48 | 44 | ld r14,STK_REG(R14)(r1) |
| 49 | 45 | .Ldo_err3: |
| 50 | − bl CFUNC(exit_vmx_usercopy) | |
| 46 | + ld r6,STK_REG(R31)(r1) /* original destination pointer */ | |
| 47 | + ld r5,STK_REG(R29)(r1) /* original number of bytes */ | |
| 48 | + subf r7,r6,r3 /* #bytes copied */ | |
| 49 | + subf r3,r7,r5 /* #bytes not copied in r3 */ | |
| 51 | 50 | ld r0,STACKFRAMESIZE+16(r1) |
| 52 | 51 | mtlr r0 |
| 53 | − b .Lexit | |
| 52 | + addi r1,r1,STACKFRAMESIZE | |
| 53 | + blr | |
| 54 | 54 | #endif /* CONFIG_ALTIVEC */ |
| 55 | 55 | |
| 56 | 56 | .Ldo_err2: |
| @@ -74,7 +74,6 @@ · 17 unmodified lines | ||
| 74 | 74 | |
| 75 | 75 | _GLOBAL(__copy_tofrom_user_power7) |
| 76 | 76 | cmpldi r5,16 |
| 77 | − cmpldi cr1,r5,3328 | |
| 78 | 77 | |
| 79 | 78 | std r3,-STACKFRAMESIZE+STK_REG(R31)(r1) |
| 80 | 79 | std r4,-STACKFRAMESIZE+STK_REG(R30)(r1) |
| @@ -82,12 +81,6 @@ _GLOBAL(__copy_tofrom_user_power7) · 1 unmodified line | ||
| 82 | 81 | |
| 83 | 82 | blt .Lshort_copy |
| 84 | 83 | |
| 85 | −#ifdef CONFIG_ALTIVEC | |
| 86 | −test_feature = SELFTEST_CASE | |
| 87 | −BEGIN_FTR_SECTION | |
| 88 | − bgt cr1,.Lvmx_copy | |
| 89 | −END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC) | |
| 90 | −#endif | |
| 91 | 84 | |
| 92 | 85 | .Lnonvmx_copy: |
| 93 | 86 | /* Get the source 8B aligned */ |
| @@ -263,23 +256,14 @@ err1; stb r0,0(r3) · 169 unmodified lines | ||
| 263 | 256 | 15: li r3,0 |
| 264 | 257 | blr |
| 265 | 258 | |
| 266 | −.Lunwind_stack_nonvmx_copy: | |
| 267 | − addi r1,r1,STACKFRAMESIZE | |
| 268 | − b .Lnonvmx_copy | |
| 269 | − | |
| 270 | −.Lvmx_copy: | |
| 271 | 259 | #ifdef CONFIG_ALTIVEC |
| 260 | +_GLOBAL(__copy_tofrom_user_power7_vmx) | |
| 272 | 261 | mflr r0 |
| 273 | 262 | std r0,16(r1) |
| 274 | 263 | stdu r1,-STACKFRAMESIZE(r1) |
| 275 | − bl CFUNC(enter_vmx_usercopy) | |
| 276 | − cmpwi cr1,r3,0 | |
| 277 | − ld r0,STACKFRAMESIZE+16(r1) | |
| 278 | − ld r3,STK_REG(R31)(r1) | |
| 279 | − ld r4,STK_REG(R30)(r1) | |
| 280 | − ld r5,STK_REG(R29)(r1) | |
| 281 | − mtlr r0 | |
| 282 | 264 | |
| 265 | + std r3,STK_REG(R31)(r1) | |
| 266 | + std r5,STK_REG(R29)(r1) | |
| 283 | 267 | /* |
| 284 | 268 | * We prefetch both the source and destination using enhanced touch |
| 285 | 269 | * instructions. We use a stream ID of 0 for the load side and |
| @@ -300,8 +284,6 @@ err1; stb r0,0(r3) · 14 unmodified lines | ||
| 300 | 284 | |
| 301 | 285 | DCBT_SETUP_STREAMS(r6, r7, r9, r10, r8) |
| 302 | 286 | |
| 303 | − beq cr1,.Lunwind_stack_nonvmx_copy | |
| 304 | − | |
| 305 | 287 | /* |
| 306 | 288 | * If source and destination are not relatively aligned we use a |
| 307 | 289 | * slower permute loop. |
| @@ -478,7 +460,8 @@ err3; lbz r0,0(r4) · 170 unmodified lines | ||
| 478 | 460 | err3; stb r0,0(r3) |
| 479 | 461 | |
| 480 | 462 | 15: addi r1,r1,STACKFRAMESIZE |
| 481 | − b CFUNC(exit_vmx_usercopy) /* tail call optimise */ | |
| 463 | + li r3,0 | |
| 464 | + blr | |
| 482 | 465 | |
| 483 | 466 | .Lvmx_unaligned_copy: |
| 484 | 467 | /* Get the destination 16B aligned */ |
| @@ -681,5 +664,7 @@ err3; lbz r0,0(r4) · 196 unmodified lines | ||
| 681 | 664 | err3; stb r0,0(r3) |
| 682 | 665 | |
| 683 | 666 | 15: addi r1,r1,STACKFRAMESIZE |
| 684 | − b CFUNC(exit_vmx_usercopy) /* tail call optimise */ | |
| 667 | + li r3,0 | |
| 668 | + blr | |
| 669 | +EXPORT_SYMBOL(__copy_tofrom_user_power7_vmx) | |
| 685 | 670 | #endif /* CONFIG_ALTIVEC */ |
| @@ -27,6 +27,7 @@ int enter_vmx_usercopy(void) · 26 unmodified lines | ||
| 27 | 27 | |
| 28 | 28 | return 1; |
| 29 | 29 | } |
| 30 | +EXPORT_SYMBOL(enter_vmx_usercopy); | |
| 30 | 31 | |
| 31 | 32 | /* |
| 32 | 33 | * This function must return 0 because we tail call optimise when calling |
| @@ -49,6 +50,7 @@ int exit_vmx_usercopy(void) · 16 unmodified lines | ||
| 49 | 50 | set_dec(1); |
| 50 | 51 | return 0; |
| 51 | 52 | } |
| 53 | +EXPORT_SYMBOL(exit_vmx_usercopy); | |
| 52 | 54 | |
| 53 | 55 | int enter_vmx_ops(void) |
| 54 | 56 | { |
| @@ -30,6 +30,10 @@ · 29 unmodified lines | ||
| 30 | 30 | #include <asm/setup.h> |
| 31 | 31 | #include <asm/fixmap.h> |
| 32 | 32 | |
| 33 | +#include <asm/fadump.h> | |
| 34 | +#include <asm/kexec.h> | |
| 35 | +#include <asm/kvm_ppc.h> | |
| 36 | + | |
| 33 | 37 | #include <mm/mmu_decl.h> |
| 34 | 38 | |
| 35 | 39 | unsigned long long memory_limit __initdata; |
| @@ -268,6 +272,16 @@ void __init paging_init(void) · 232 unmodified lines | ||
| 268 | 272 | |
| 269 | 273 | void __init arch_mm_preinit(void) |
| 270 | 274 | { |
| 275 | + | |
| 276 | + /* | |
| 277 | + * Reserve large chunks of memory for use by CMA for kdump, fadump, KVM | |
| 278 | + * and hugetlb. These must be called after pageblock_order is | |
| 279 | + * initialised. | |
| 280 | + */ | |
| 281 | + fadump_cma_init(); | |
| 282 | + kdump_cma_reserve(); | |
| 283 | + kvm_cma_reserve(); | |
| 284 | + | |
| 271 | 285 | /* |
| 272 | 286 | * book3s is limited to 16 page sizes due to encoding this in |
| 273 | 287 | * a 4-bit field for slices. |
| @@ -81,9 +81,6 @@ · 80 unmodified lines | ||
| 81 | 81 | |
| 82 | 82 | #ifdef CONFIG_PPC64 |
| 83 | 83 | |
| 84 | −/* for gpr non volatile registers BPG_REG_6 to 10 */ | |
| 85 | −#define BPF_PPC_STACK_SAVE (6 * 8) | |
| 86 | − | |
| 87 | 84 | /* If dummy pass (!image), account for maximum possible instructions */ |
| 88 | 85 | #define PPC_LI64(d, i) do { \ |
| 89 | 86 | if (!image) \ |
| @@ -219,8 +216,6 @@ int bpf_jit_emit_exit_insn(u32 *image, struct codegen_context *ctx, int tmp_reg, · 129 unmodified lines | ||
| 219 | 216 | int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, u32 *fimage, int pass, |
| 220 | 217 | struct codegen_context *ctx, int insn_idx, |
| 221 | 218 | int jmp_off, int dst_reg, u32 code); |
| 222 | − | |
| 223 | −int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx); | |
| 224 | 219 | #endif |
| 225 | 220 | |
| 226 | 221 | #endif |
| @@ -450,7 +450,7 @@ bool bpf_jit_supports_subprog_tailcalls(void) · 449 unmodified lines | ||
| 450 | 450 | |
| 451 | 451 | bool bpf_jit_supports_kfunc_call(void) |
| 452 | 452 | { |
| 453 | − return true; | |
| 453 | + return IS_ENABLED(CONFIG_PPC64); | |
| 454 | 454 | } |
| 455 | 455 | |
| 456 | 456 | bool bpf_jit_supports_arena(void) |
| @@ -638,19 +638,12 @@ static int invoke_bpf_mod_ret(u32 *image, u32 *ro_image, struct codegen_context · 181 unmodified lines | ||
| 638 | 638 | * for the traced function (BPF subprog/callee) to fetch it. |
| 639 | 639 | */ |
| 640 | 640 | static void bpf_trampoline_setup_tail_call_info(u32 *image, struct codegen_context *ctx, |
| 641 | − int func_frame_offset, | |
| 642 | − int bpf_dummy_frame_size, int r4_off) | |
| 641 | + int bpf_frame_size, int r4_off) | |
| 643 | 642 | { |
| 644 | 643 | if (IS_ENABLED(CONFIG_PPC64)) { |
| 645 | − /* See Generated stack layout */ | |
| 646 | − int tailcallinfo_offset = BPF_PPC_TAILCALL; | |
| 647 | − | |
| 648 | − /* | |
| 649 | − * func_frame_offset = ...(1) | |
| 650 | − * bpf_dummy_frame_size + trampoline_frame_size | |
| 651 | − */ | |
| 652 | − EMIT(PPC_RAW_LD(_R4, _R1, func_frame_offset)); | |
| 653 | − EMIT(PPC_RAW_LD(_R3, _R4, -tailcallinfo_offset)); | |
| 644 | + EMIT(PPC_RAW_LD(_R4, _R1, bpf_frame_size)); | |
| 645 | + /* Refer to trampoline's Generated stack layout */ | |
| 646 | + EMIT(PPC_RAW_LD(_R3, _R4, -BPF_PPC_TAILCALL)); | |
| 654 | 647 | |
| 655 | 648 | /* |
| 656 | 649 | * Setting the tail_call_info in trampoline's frame |
| @@ -658,22 +651,14 @@ static void bpf_trampoline_setup_tail_call_info(u32 *image, struct codegen_conte · 1 unmodified line | ||
| 658 | 651 | */ |
| 659 | 652 | EMIT(PPC_RAW_CMPLWI(_R3, MAX_TAIL_CALL_CNT)); |
| 660 | 653 | PPC_BCC_CONST_SHORT(COND_GT, 8); |
| 661 | − EMIT(PPC_RAW_ADDI(_R3, _R4, bpf_jit_stack_tailcallinfo_offset(ctx))); | |
| 654 | + EMIT(PPC_RAW_ADDI(_R3, _R4, -BPF_PPC_TAILCALL)); | |
| 655 | + | |
| 662 | 656 | /* |
| 663 | − * From ...(1) above: | |
| 664 | − * trampoline_frame_bottom = ...(2) | |
| 665 | − * func_frame_offset - bpf_dummy_frame_size | |
| 666 | − * | |
| 667 | − * Using ...(2) derived above: | |
| 668 | − * trampoline_tail_call_info_offset = ...(3) | |
| 669 | − * trampoline_frame_bottom - tailcallinfo_offset | |
| 670 | − * | |
| 671 | − * From ...(3): | |
| 672 | − * Use trampoline_tail_call_info_offset to write reference of main's | |
| 673 | − * tail_call_info in trampoline frame. | |
| 657 | + * Trampoline's tail_call_info is at the same offset, as that of | |
| 658 | + * any bpf program, with reference to previous frame. Update the | |
| 659 | + * address of main's tail_call_info in trampoline frame. | |
| 674 | 660 | */ |
| 675 | − EMIT(PPC_RAW_STL(_R3, _R1, (func_frame_offset - bpf_dummy_frame_size) | |
| 676 | − - tailcallinfo_offset)); | |
| 661 | + EMIT(PPC_RAW_STL(_R3, _R1, bpf_frame_size - BPF_PPC_TAILCALL)); | |
| 677 | 662 | } else { |
| 678 | 663 | /* See bpf_jit_stack_offsetof() and BPF_PPC_TC */ |
| 679 | 664 | EMIT(PPC_RAW_LL(_R4, _R1, r4_off)); |
| @@ -681,7 +666,7 @@ static void bpf_trampoline_setup_tail_call_info(u32 *image, struct codegen_conte · 1 unmodified line | ||
| 681 | 666 | } |
| 682 | 667 | |
| 683 | 668 | static void bpf_trampoline_restore_tail_call_cnt(u32 *image, struct codegen_context *ctx, |
| 684 | − int func_frame_offset, int r4_off) | |
| 669 | + int bpf_frame_size, int r4_off) | |
| 685 | 670 | { |
| 686 | 671 | if (IS_ENABLED(CONFIG_PPC32)) { |
| 687 | 672 | /* |
| @@ -692,12 +677,12 @@ static void bpf_trampoline_restore_tail_call_cnt(u32 *image, struct codegen_cont · 4 unmodified lines | ||
| 692 | 677 | } |
| 693 | 678 | } |
| 694 | 679 | |
| 695 | −static void bpf_trampoline_save_args(u32 *image, struct codegen_context *ctx, int func_frame_offset, | |
| 696 | − int nr_regs, int regs_off) | |
| 680 | +static void bpf_trampoline_save_args(u32 *image, struct codegen_context *ctx, | |
| 681 | + int bpf_frame_size, int nr_regs, int regs_off) | |
| 697 | 682 | { |
| 698 | 683 | int param_save_area_offset; |
| 699 | 684 | |
| 700 | − param_save_area_offset = func_frame_offset; /* the two frames we alloted */ | |
| 685 | + param_save_area_offset = bpf_frame_size; | |
| 701 | 686 | param_save_area_offset += STACK_FRAME_MIN_SIZE; /* param save area is past frame header */ |
| 702 | 687 | |
| 703 | 688 | for (int i = 0; i < nr_regs; i++) { |
| @@ -720,11 +705,11 @@ static void bpf_trampoline_restore_args_regs(u32 *image, struct codegen_context · 16 unmodified lines | ||
| 720 | 705 | |
| 721 | 706 | /* Used when we call into the traced function. Replicate parameter save area */ |
| 722 | 707 | static void bpf_trampoline_restore_args_stack(u32 *image, struct codegen_context *ctx, |
| 723 | − int func_frame_offset, int nr_regs, int regs_off) | |
| 708 | + int bpf_frame_size, int nr_regs, int regs_off) | |
| 724 | 709 | { |
| 725 | 710 | int param_save_area_offset; |
| 726 | 711 | |
| 727 | − param_save_area_offset = func_frame_offset; /* the two frames we alloted */ | |
| 712 | + param_save_area_offset = bpf_frame_size; | |
| 728 | 713 | param_save_area_offset += STACK_FRAME_MIN_SIZE; /* param save area is past frame header */ |
| 729 | 714 | |
| 730 | 715 | for (int i = 8; i < nr_regs; i++) { |
| @@ -741,10 +726,10 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 10 unmodified lines | ||
| 741 | 726 | void *func_addr) |
| 742 | 727 | { |
| 743 | 728 | int regs_off, nregs_off, ip_off, run_ctx_off, retval_off, nvr_off, alt_lr_off, r4_off = 0; |
| 744 | − int i, ret, nr_regs, bpf_frame_size = 0, bpf_dummy_frame_size = 0, func_frame_offset; | |
| 745 | 729 | struct bpf_tramp_links *fmod_ret = &tlinks[BPF_TRAMP_MODIFY_RETURN]; |
| 746 | 730 | struct bpf_tramp_links *fentry = &tlinks[BPF_TRAMP_FENTRY]; |
| 747 | 731 | struct bpf_tramp_links *fexit = &tlinks[BPF_TRAMP_FEXIT]; |
| 732 | + int i, ret, nr_regs, retaddr_off, bpf_frame_size = 0; | |
| 748 | 733 | struct codegen_context codegen_ctx, *ctx; |
| 749 | 734 | u32 *image = (u32 *)rw_image; |
| 750 | 735 | ppc_inst_t branch_insn; |
| @@ -770,24 +755,19 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 19 unmodified lines | ||
| 770 | 755 | * Generated stack layout: |
| 771 | 756 | * |
| 772 | 757 | * func prev back chain [ back chain ] |
| 773 | − * [ ] | |
| 774 | − * bpf prog redzone/tailcallcnt [ ... ] 64 bytes (64-bit powerpc) | |
| 775 | − * [ ] -- | |
| 776 | − * LR save area [ r0 save (64-bit) ] | header | |
| 777 | − * [ r0 save (32-bit) ] | | |
| 778 | − * dummy frame for unwind [ back chain 1 ] -- | |
| 779 | 758 | * [ tail_call_info ] optional - 64-bit powerpc |
| 780 | 759 | * [ padding ] align stack frame |
| 781 | 760 | * r4_off [ r4 (tailcallcnt) ] optional - 32-bit powerpc |
| 782 | 761 | * alt_lr_off [ real lr (ool stub)] optional - actual lr |
| 762 | + * retaddr_off [ return address ] | |
| 783 | 763 | * [ r26 ] |
| 784 | 764 | * nvr_off [ r25 ] nvr save area |
| 785 | 765 | * retval_off [ return value ] |
| 786 | 766 | * [ reg argN ] |
| 787 | 767 | * [ ... ] |
| 788 | − * regs_off [ reg_arg1 ] prog ctx context | |
| 789 | − * nregs_off [ args count ] | |
| 790 | − * ip_off [ traced function ] | |
| 768 | + * regs_off [ reg_arg1 ] prog_ctx | |
| 769 | + * nregs_off [ args count ] ((u64 *)prog_ctx)[-1] | |
| 770 | + * ip_off [ traced function ] ((u64 *)prog_ctx)[-2] | |
| 791 | 771 | * [ ... ] |
| 792 | 772 | * run_ctx_off [ bpf_tramp_run_ctx ] |
| 793 | 773 | * [ reg argN ] |
| @@ -843,6 +823,10 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 49 unmodified lines | ||
| 843 | 823 | nvr_off = bpf_frame_size; |
| 844 | 824 | bpf_frame_size += 2 * SZL; |
| 845 | 825 | |
| 826 | + /* Save area for return address */ | |
| 827 | + retaddr_off = bpf_frame_size; | |
| 828 | + bpf_frame_size += SZL; | |
| 829 | + | |
| 846 | 830 | /* Optional save area for actual LR in case of ool ftrace */ |
| 847 | 831 | if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) { |
| 848 | 832 | alt_lr_off = bpf_frame_size; |
| @@ -869,16 +853,8 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 20 unmodified lines | ||
| 869 | 853 | /* Padding to align stack frame, if any */ |
| 870 | 854 | bpf_frame_size = round_up(bpf_frame_size, SZL * 2); |
| 871 | 855 | |
| 872 | − /* Dummy frame size for proper unwind - includes 64-bytes red zone for 64-bit powerpc */ | |
| 873 | − bpf_dummy_frame_size = STACK_FRAME_MIN_SIZE + 64; | |
| 874 | − | |
| 875 | − /* Offset to the traced function's stack frame */ | |
| 876 | − func_frame_offset = bpf_dummy_frame_size + bpf_frame_size; | |
| 877 | − | |
| 878 | − /* Create dummy frame for unwind, store original return value */ | |
| 856 | + /* Store original return value */ | |
| 879 | 857 | EMIT(PPC_RAW_STL(_R0, _R1, PPC_LR_STKOFF)); |
| 880 | − /* Protect red zone where tail call count goes */ | |
| 881 | − EMIT(PPC_RAW_STLU(_R1, _R1, -bpf_dummy_frame_size)); | |
| 882 | 858 | |
| 883 | 859 | /* Create our stack frame */ |
| 884 | 860 | EMIT(PPC_RAW_STLU(_R1, _R1, -bpf_frame_size)); |
| @@ -893,34 +869,44 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 8 unmodified lines | ||
| 893 | 869 | if (IS_ENABLED(CONFIG_PPC32) && nr_regs < 2) |
| 894 | 870 | EMIT(PPC_RAW_STL(_R4, _R1, r4_off)); |
| 895 | 871 | |
| 896 | − bpf_trampoline_save_args(image, ctx, func_frame_offset, nr_regs, regs_off); | |
| 872 | + bpf_trampoline_save_args(image, ctx, bpf_frame_size, nr_regs, regs_off); | |
| 897 | 873 | |
| 898 | − /* Save our return address */ | |
| 874 | + /* Save our LR/return address */ | |
| 899 | 875 | EMIT(PPC_RAW_MFLR(_R3)); |
| 900 | 876 | if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) |
| 901 | 877 | EMIT(PPC_RAW_STL(_R3, _R1, alt_lr_off)); |
| 902 | 878 | else |
| 903 | − EMIT(PPC_RAW_STL(_R3, _R1, bpf_frame_size + PPC_LR_STKOFF)); | |
| 879 | + EMIT(PPC_RAW_STL(_R3, _R1, retaddr_off)); | |
| 904 | 880 | |
| 905 | 881 | /* |
| 906 | − * Save ip address of the traced function. | |
| 907 | − * We could recover this from LR, but we will need to address for OOL trampoline, | |
| 908 | − * and optional GEP area. | |
| 882 | + * Derive IP address of the traced function. | |
| 883 | + * In case of CONFIG_PPC_FTRACE_OUT_OF_LINE or BPF program, LR points to the instruction | |
| 884 | + * after the 'bl' instruction in the OOL stub. Refer to ftrace_init_ool_stub() and | |
| 885 | + * bpf_arch_text_poke() for OOL stub of kernel functions and bpf programs respectively. | |
| 886 | + * Relevant stub sequence: | |
| 887 | + * | |
| 888 | + * bl <tramp> | |
| 889 | + * LR (R3) => mtlr r0 | |
| 890 | + * b <func_addr+4> | |
| 891 | + * | |
| 892 | + * Recover kernel function/bpf program address from the unconditional | |
| 893 | + * branch instruction at the end of OOL stub. | |
| 909 | 894 | */ |
| 910 | 895 | if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE) || flags & BPF_TRAMP_F_IP_ARG) { |
| 911 | 896 | EMIT(PPC_RAW_LWZ(_R4, _R3, 4)); |
| 912 | 897 | EMIT(PPC_RAW_SLWI(_R4, _R4, 6)); |
| 913 | 898 | EMIT(PPC_RAW_SRAWI(_R4, _R4, 6)); |
| 914 | 899 | EMIT(PPC_RAW_ADD(_R3, _R3, _R4)); |
| 915 | − EMIT(PPC_RAW_ADDI(_R3, _R3, 4)); | |
| 916 | 900 | } |
| 917 | 901 | |
| 918 | 902 | if (flags & BPF_TRAMP_F_IP_ARG) |
| 919 | 903 | EMIT(PPC_RAW_STL(_R3, _R1, ip_off)); |
| 920 | 904 | |
| 921 | − if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) | |
| 922 | − /* Fake our LR for unwind */ | |
| 923 | − EMIT(PPC_RAW_STL(_R3, _R1, bpf_frame_size + PPC_LR_STKOFF)); | |
| 905 | + if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) { | |
| 906 | + /* Fake our LR for BPF_TRAMP_F_CALL_ORIG case */ | |
| 907 | + EMIT(PPC_RAW_ADDI(_R3, _R3, 4)); | |
| 908 | + EMIT(PPC_RAW_STL(_R3, _R1, retaddr_off)); | |
| 909 | + } | |
| 924 | 910 | |
| 925 | 911 | /* Save function arg count -- see bpf_get_func_arg_cnt() */ |
| 926 | 912 | EMIT(PPC_RAW_LI(_R3, nr_regs)); |
| @@ -958,20 +944,19 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 31 unmodified lines | ||
| 958 | 944 | /* Call the traced function */ |
| 959 | 945 | if (flags & BPF_TRAMP_F_CALL_ORIG) { |
| 960 | 946 | /* |
| 961 | − * The address in LR save area points to the correct point in the original function | |
| 947 | + * retaddr on trampoline stack points to the correct point in the original function | |
| 962 | 948 | * with both PPC_FTRACE_OUT_OF_LINE as well as with traditional ftrace instruction |
| 963 | 949 | * sequence |
| 964 | 950 | */ |
| 965 | − EMIT(PPC_RAW_LL(_R3, _R1, bpf_frame_size + PPC_LR_STKOFF)); | |
| 951 | + EMIT(PPC_RAW_LL(_R3, _R1, retaddr_off)); | |
| 966 | 952 | EMIT(PPC_RAW_MTCTR(_R3)); |
| 967 | 953 | |
| 968 | 954 | /* Replicate tail_call_cnt before calling the original BPF prog */ |
| 969 | 955 | if (flags & BPF_TRAMP_F_TAIL_CALL_CTX) |
| 970 | − bpf_trampoline_setup_tail_call_info(image, ctx, func_frame_offset, | |
| 971 | − bpf_dummy_frame_size, r4_off); | |
| 956 | + bpf_trampoline_setup_tail_call_info(image, ctx, bpf_frame_size, r4_off); | |
| 972 | 957 | |
| 973 | 958 | /* Restore args */ |
| 974 | − bpf_trampoline_restore_args_stack(image, ctx, func_frame_offset, nr_regs, regs_off); | |
| 959 | + bpf_trampoline_restore_args_stack(image, ctx, bpf_frame_size, nr_regs, regs_off); | |
| 975 | 960 | |
| 976 | 961 | /* Restore TOC for 64-bit */ |
| 977 | 962 | if (IS_ENABLED(CONFIG_PPC64_ELF_ABI_V2) && !IS_ENABLED(CONFIG_PPC_KERNEL_PCREL)) |
| @@ -985,7 +970,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 7 unmodified lines | ||
| 985 | 970 | |
| 986 | 971 | /* Restore updated tail_call_cnt */ |
| 987 | 972 | if (flags & BPF_TRAMP_F_TAIL_CALL_CTX) |
| 988 | − bpf_trampoline_restore_tail_call_cnt(image, ctx, func_frame_offset, r4_off); | |
| 973 | + bpf_trampoline_restore_tail_call_cnt(image, ctx, bpf_frame_size, r4_off); | |
| 989 | 974 | |
| 990 | 975 | /* Reserve space to patch branch instruction to skip fexit progs */ |
| 991 | 976 | if (ro_image) /* image is NULL for dummy pass */ |
| @@ -1037,7 +1022,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 45 unmodified lines | ||
| 1037 | 1022 | EMIT(PPC_RAW_LD(_R2, _R1, 24)); |
| 1038 | 1023 | if (flags & BPF_TRAMP_F_SKIP_FRAME) { |
| 1039 | 1024 | /* Skip the traced function and return to parent */ |
| 1040 | − EMIT(PPC_RAW_ADDI(_R1, _R1, func_frame_offset)); | |
| 1025 | + EMIT(PPC_RAW_ADDI(_R1, _R1, bpf_frame_size)); | |
| 1041 | 1026 | EMIT(PPC_RAW_LL(_R0, _R1, PPC_LR_STKOFF)); |
| 1042 | 1027 | EMIT(PPC_RAW_MTLR(_R0)); |
| 1043 | 1028 | EMIT(PPC_RAW_BLR()); |
| @@ -1045,13 +1030,13 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · 1 unmodified line | ||
| 1045 | 1030 | if (IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE)) { |
| 1046 | 1031 | EMIT(PPC_RAW_LL(_R0, _R1, alt_lr_off)); |
| 1047 | 1032 | EMIT(PPC_RAW_MTLR(_R0)); |
| 1048 | − EMIT(PPC_RAW_ADDI(_R1, _R1, func_frame_offset)); | |
| 1033 | + EMIT(PPC_RAW_ADDI(_R1, _R1, bpf_frame_size)); | |
| 1049 | 1034 | EMIT(PPC_RAW_LL(_R0, _R1, PPC_LR_STKOFF)); |
| 1050 | 1035 | EMIT(PPC_RAW_BLR()); |
| 1051 | 1036 | } else { |
| 1052 | − EMIT(PPC_RAW_LL(_R0, _R1, bpf_frame_size + PPC_LR_STKOFF)); | |
| 1037 | + EMIT(PPC_RAW_LL(_R0, _R1, retaddr_off)); | |
| 1053 | 1038 | EMIT(PPC_RAW_MTCTR(_R0)); |
| 1054 | − EMIT(PPC_RAW_ADDI(_R1, _R1, func_frame_offset)); | |
| 1039 | + EMIT(PPC_RAW_ADDI(_R1, _R1, bpf_frame_size)); | |
| 1055 | 1040 | EMIT(PPC_RAW_LL(_R0, _R1, PPC_LR_STKOFF)); |
| 1056 | 1041 | EMIT(PPC_RAW_MTLR(_R0)); |
| 1057 | 1042 | EMIT(PPC_RAW_BCTR()); |
| @@ -32,23 +32,27 @@ · 31 unmodified lines | ||
| 32 | 32 | * |
| 33 | 33 | * [ prev sp ] <------------- |
| 34 | 34 | * [ tail_call_info ] 8 | |
| 35 | − * [ nv gpr save area ] 6*8 + (12*8) | | |
| 35 | + * [ nv gpr save area ] (6 * 8) | | |
| 36 | + * [ addl. nv gpr save area] (12 * 8) | <--- exception boundary/callback program | |
| 36 | 37 | * [ local_tmp_var ] 24 | |
| 37 | 38 | * fp (r31) --> [ ebpf stack space ] upto 512 | |
| 38 | 39 | * [ frame header ] 32/112 | |
| 39 | 40 | * sp (r1) ---> [ stack pointer ] -------------- |
| 40 | 41 | * |
| 41 | − * Additional (12*8) in 'nv gpr save area' only in case of | |
| 42 | − * exception boundary. | |
| 42 | + * Additional (12 * 8) in 'nv gpr save area' only in case of | |
| 43 | + * exception boundary/callback. | |
| 43 | 44 | */ |
| 44 | 45 | |
| 46 | +/* BPF non-volatile registers save area size */ | |
| 47 | +#define BPF_PPC_STACK_SAVE (6 * 8) | |
| 48 | + | |
| 45 | 49 | /* for bpf JIT code internal usage */ |
| 46 | 50 | #define BPF_PPC_STACK_LOCALS 24 |
| 47 | 51 | /* |
| 48 | 52 | * for additional non volatile registers(r14-r25) to be saved |
| 49 | 53 | * at exception boundary |
| 50 | 54 | */ |
| 51 | −#define BPF_PPC_EXC_STACK_SAVE (12*8) | |
| 55 | +#define BPF_PPC_EXC_STACK_SAVE (12 * 8) | |
| 52 | 56 | |
| 53 | 57 | /* stack frame excluding BPF stack, ensure this is quadword aligned */ |
| 54 | 58 | #define BPF_PPC_STACKFRAME (STACK_FRAME_MIN_SIZE + \ |
| @@ -125,12 +129,13 @@ static inline bool bpf_has_stack_frame(struct codegen_context *ctx) · 70 unmodified lines | ||
| 125 | 129 | * [ ... ] | |
| 126 | 130 | * sp (r1) ---> [ stack pointer ] -------------- |
| 127 | 131 | * [ tail_call_info ] 8 |
| 128 | − * [ nv gpr save area ] 6*8 + (12*8) | |
| 132 | + * [ nv gpr save area ] (6 * 8) | |
| 133 | + * [ addl. nv gpr save area] (12 * 8) <--- exception boundary/callback program | |
| 129 | 134 | * [ local_tmp_var ] 24 |
| 130 | 135 | * [ unused red zone ] 224 |
| 131 | 136 | * |
| 132 | − * Additional (12*8) in 'nv gpr save area' only in case of | |
| 133 | − * exception boundary. | |
| 137 | + * Additional (12 * 8) in 'nv gpr save area' only in case of | |
| 138 | + * exception boundary/callback. | |
| 134 | 139 | */ |
| 135 | 140 | static int bpf_jit_stack_local(struct codegen_context *ctx) |
| 136 | 141 | { |
| @@ -148,7 +153,7 @@ static int bpf_jit_stack_local(struct codegen_context *ctx) · 11 unmodified lines | ||
| 148 | 153 | } |
| 149 | 154 | } |
| 150 | 155 | |
| 151 | −int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx) | |
| 156 | +static int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx) | |
| 152 | 157 | { |
| 153 | 158 | return bpf_jit_stack_local(ctx) + BPF_PPC_STACK_LOCALS + BPF_PPC_STACK_SAVE; |
| 154 | 159 | } |
| @@ -237,10 +242,6 @@ void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx) · 82 unmodified lines | ||
| 237 | 242 | |
| 238 | 243 | if (bpf_has_stack_frame(ctx) && !ctx->exception_cb) { |
| 239 | 244 | /* |
| 240 | − * exception_cb uses boundary frame after stack walk. | |
| 241 | − * It can simply use redzone, this optimization reduces | |
| 242 | − * stack walk loop by one level. | |
| 243 | − * | |
| 244 | 245 | * We need a stack frame, but we don't necessarily need to |
| 245 | 246 | * save/restore LR unless we call other functions |
| 246 | 247 | */ |
| @@ -284,6 +285,22 @@ void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx) · 37 unmodified lines | ||
| 284 | 285 | * program(main prog) as third arg |
| 285 | 286 | */ |
| 286 | 287 | EMIT(PPC_RAW_MR(_R1, _R5)); |
| 288 | + /* | |
| 289 | + * Exception callback reuses the stack frame of exception boundary. | |
| 290 | + * But BPF stack depth of exception callback and exception boundary | |
| 291 | + * don't have to be same. If BPF stack depth is different, adjust the | |
| 292 | + * stack frame size considering BPF stack depth of exception callback. | |
| 293 | + * The non-volatile register save area remains unchanged. These non- | |
| 294 | + * volatile registers are restored in exception callback's epilogue. | |
| 295 | + */ | |
| 296 | + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), _R5, 0)); | |
| 297 | + EMIT(PPC_RAW_SUB(bpf_to_ppc(TMP_REG_2), bpf_to_ppc(TMP_REG_1), _R1)); | |
| 298 | + EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_2), bpf_to_ppc(TMP_REG_2), | |
| 299 | + -BPF_PPC_EXC_STACKFRAME)); | |
| 300 | + EMIT(PPC_RAW_CMPLDI(bpf_to_ppc(TMP_REG_2), ctx->stack_size)); | |
| 301 | + PPC_BCC_CONST_SHORT(COND_EQ, 12); | |
| 302 | + EMIT(PPC_RAW_MR(_R1, bpf_to_ppc(TMP_REG_1))); | |
| 303 | + EMIT(PPC_RAW_STDU(_R1, _R1, -(BPF_PPC_EXC_STACKFRAME + ctx->stack_size))); | |
| 287 | 304 | } |
| 288 | 305 | |
| 289 | 306 | /* |
| @@ -482,6 +499,83 @@ int bpf_jit_emit_func_call_rel(u32 *image, u32 *fimage, struct codegen_context * · 192 unmodified lines | ||
| 482 | 499 | return 0; |
| 483 | 500 | } |
| 484 | 501 | |
| 502 | +static int zero_extend(u32 *image, struct codegen_context *ctx, u32 src_reg, u32 dst_reg, u32 size) | |
| 503 | +{ | |
| 504 | + switch (size) { | |
| 505 | + case 1: | |
| 506 | + /* zero-extend 8 bits into 64 bits */ | |
| 507 | + EMIT(PPC_RAW_RLDICL(dst_reg, src_reg, 0, 56)); | |
| 508 | + return 0; | |
| 509 | + case 2: | |
| 510 | + /* zero-extend 16 bits into 64 bits */ | |
| 511 | + EMIT(PPC_RAW_RLDICL(dst_reg, src_reg, 0, 48)); | |
| 512 | + return 0; | |
| 513 | + case 4: | |
| 514 | + /* zero-extend 32 bits into 64 bits */ | |
| 515 | + EMIT(PPC_RAW_RLDICL(dst_reg, src_reg, 0, 32)); | |
| 516 | + fallthrough; | |
| 517 | + case 8: | |
| 518 | + /* Nothing to do */ | |
| 519 | + return 0; | |
| 520 | + default: | |
| 521 | + return -1; | |
| 522 | + } | |
| 523 | +} | |
| 524 | + | |
| 525 | +static int sign_extend(u32 *image, struct codegen_context *ctx, u32 src_reg, u32 dst_reg, u32 size) | |
| 526 | +{ | |
| 527 | + switch (size) { | |
| 528 | + case 1: | |
| 529 | + /* sign-extend 8 bits into 64 bits */ | |
| 530 | + EMIT(PPC_RAW_EXTSB(dst_reg, src_reg)); | |
| 531 | + return 0; | |
| 532 | + case 2: | |
| 533 | + /* sign-extend 16 bits into 64 bits */ | |
| 534 | + EMIT(PPC_RAW_EXTSH(dst_reg, src_reg)); | |
| 535 | + return 0; | |
| 536 | + case 4: | |
| 537 | + /* sign-extend 32 bits into 64 bits */ | |
| 538 | + EMIT(PPC_RAW_EXTSW(dst_reg, src_reg)); | |
| 539 | + fallthrough; | |
| 540 | + case 8: | |
| 541 | + /* Nothing to do */ | |
| 542 | + return 0; | |
| 543 | + default: | |
| 544 | + return -1; | |
| 545 | + } | |
| 546 | +} | |
| 547 | + | |
| 548 | +/* | |
| 549 | + * Handle powerpc ABI expectations from caller: | |
| 550 | + * - Unsigned arguments are zero-extended. | |
| 551 | + * - Signed arguments are sign-extended. | |
| 552 | + */ | |
| 553 | +static int prepare_for_kfunc_call(const struct bpf_prog *fp, u32 *image, | |
| 554 | + struct codegen_context *ctx, | |
| 555 | + const struct bpf_insn *insn) | |
| 556 | +{ | |
| 557 | + const struct btf_func_model *m = bpf_jit_find_kfunc_model(fp, insn); | |
| 558 | + int i; | |
| 559 | + | |
| 560 | + if (!m) | |
| 561 | + return -1; | |
| 562 | + | |
| 563 | + for (i = 0; i < m->nr_args; i++) { | |
| 564 | + /* Note that BPF ABI only allows up to 5 args for kfuncs */ | |
| 565 | + u32 reg = bpf_to_ppc(BPF_REG_1 + i), size = m->arg_size[i]; | |
| 566 | + | |
| 567 | + if (!(m->arg_flags[i] & BTF_FMODEL_SIGNED_ARG)) { | |
| 568 | + if (zero_extend(image, ctx, reg, reg, size)) | |
| 569 | + return -1; | |
| 570 | + } else { | |
| 571 | + if (sign_extend(image, ctx, reg, reg, size)) | |
| 572 | + return -1; | |
| 573 | + } | |
| 574 | + } | |
| 575 | + | |
| 576 | + return 0; | |
| 577 | +} | |
| 578 | + | |
| 485 | 579 | static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 out) |
| 486 | 580 | { |
| 487 | 581 | /* |
| @@ -522,9 +616,30 @@ static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 o · 34 unmodified lines | ||
| 522 | 616 | |
| 523 | 617 | /* |
| 524 | 618 | * tail_call_info++; <- Actual value of tcc here |
| 619 | + * Writeback this updated value only if tailcall succeeds. | |
| 525 | 620 | */ |
| 526 | 621 | EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), 1)); |
| 527 | 622 | |
| 623 | + /* prog = array->ptrs[index]; */ | |
| 624 | + EMIT(PPC_RAW_MULI(bpf_to_ppc(TMP_REG_2), b2p_index, 8)); | |
| 625 | + EMIT(PPC_RAW_ADD(bpf_to_ppc(TMP_REG_2), bpf_to_ppc(TMP_REG_2), b2p_bpf_array)); | |
| 626 | + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_2), bpf_to_ppc(TMP_REG_2), | |
| 627 | + offsetof(struct bpf_array, ptrs))); | |
| 628 | + | |
| 629 | + /* | |
| 630 | + * if (prog == NULL) | |
| 631 | + * goto out; | |
| 632 | + */ | |
| 633 | + EMIT(PPC_RAW_CMPLDI(bpf_to_ppc(TMP_REG_2), 0)); | |
| 634 | + PPC_BCC_SHORT(COND_EQ, out); | |
| 635 | + | |
| 636 | + /* goto *(prog->bpf_func + prologue_size); */ | |
| 637 | + EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_2), bpf_to_ppc(TMP_REG_2), | |
| 638 | + offsetof(struct bpf_prog, bpf_func))); | |
| 639 | + EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_2), bpf_to_ppc(TMP_REG_2), | |
| 640 | + FUNCTION_DESCR_SIZE + bpf_tailcall_prologue_size)); | |
| 641 | + EMIT(PPC_RAW_MTCTR(bpf_to_ppc(TMP_REG_2))); | |
| 642 | + | |
| 528 | 643 | /* |
| 529 | 644 | * Before writing updated tail_call_info, distinguish if current frame |
| 530 | 645 | * is storing a reference to tail_call_info or actual tcc value in |
| @@ -539,24 +654,6 @@ static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 o · 8 unmodified lines | ||
| 539 | 654 | /* Writeback updated value to tail_call_info */ |
| 540 | 655 | EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), 0)); |
| 541 | 656 | |
| 542 | − /* prog = array->ptrs[index]; */ | |
| 543 | − EMIT(PPC_RAW_MULI(bpf_to_ppc(TMP_REG_1), b2p_index, 8)); | |
| 544 | − EMIT(PPC_RAW_ADD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), b2p_bpf_array)); | |
| 545 | − EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), offsetof(struct bpf_array, ptrs))); | |
| 546 | − | |
| 547 | − /* | |
| 548 | − * if (prog == NULL) | |
| 549 | − * goto out; | |
| 550 | − */ | |
| 551 | − EMIT(PPC_RAW_CMPLDI(bpf_to_ppc(TMP_REG_1), 0)); | |
| 552 | − PPC_BCC_SHORT(COND_EQ, out); | |
| 553 | − | |
| 554 | − /* goto *(prog->bpf_func + prologue_size); */ | |
| 555 | − EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), offsetof(struct bpf_prog, bpf_func))); | |
| 556 | − EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_1), | |
| 557 | − FUNCTION_DESCR_SIZE + bpf_tailcall_prologue_size)); | |
| 558 | − EMIT(PPC_RAW_MTCTR(bpf_to_ppc(TMP_REG_1))); | |
| 559 | − | |
| 560 | 657 | /* tear down stack, restore NVRs, ... */ |
| 561 | 658 | bpf_jit_emit_common_epilogue(image, ctx); |
| 562 | 659 | |
| @@ -1123,14 +1220,16 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct code · 560 unmodified lines | ||
| 1123 | 1220 | /* special mov32 for zext */ |
| 1124 | 1221 | EMIT(PPC_RAW_RLWINM(dst_reg, dst_reg, 0, 0, 31)); |
| 1125 | 1222 | break; |
| 1126 | − } else if (off == 8) { | |
| 1127 | − EMIT(PPC_RAW_EXTSB(dst_reg, src_reg)); | |
| 1128 | − } else if (off == 16) { | |
| 1129 | − EMIT(PPC_RAW_EXTSH(dst_reg, src_reg)); | |
| 1130 | − } else if (off == 32) { | |
| 1131 | − EMIT(PPC_RAW_EXTSW(dst_reg, src_reg)); | |
| 1132 | − } else if (dst_reg != src_reg) | |
| 1133 | − EMIT(PPC_RAW_MR(dst_reg, src_reg)); | |
| 1223 | + } | |
| 1224 | + if (off == 0) { | |
| 1225 | + /* MOV */ | |
| 1226 | + if (dst_reg != src_reg) | |
| 1227 | + EMIT(PPC_RAW_MR(dst_reg, src_reg)); | |
| 1228 | + } else { | |
| 1229 | + /* MOVSX: dst = (s8,s16,s32)src (off = 8,16,32) */ | |
| 1230 | + if (sign_extend(image, ctx, src_reg, dst_reg, off / 8)) | |
| 1231 | + return -1; | |
| 1232 | + } | |
| 1134 | 1233 | goto bpf_alu32_trunc; |
| 1135 | 1234 | case BPF_ALU | BPF_MOV | BPF_K: /* (u32) dst = imm */ |
| 1136 | 1235 | case BPF_ALU64 | BPF_MOV | BPF_K: /* dst = (s64) imm */ |
| @@ -1598,6 +1697,12 @@ emit_clear: · 461 unmodified lines | ||
| 1598 | 1697 | if (ret < 0) |
| 1599 | 1698 | return ret; |
| 1600 | 1699 | |
| 1700 | + /* Take care of powerpc ABI requirements before kfunc call */ | |
| 1701 | + if (insn[i].src_reg == BPF_PSEUDO_KFUNC_CALL) { | |
| 1702 | + if (prepare_for_kfunc_call(fp, image, ctx, &insn[i])) | |
| 1703 | + return -1; | |
| 1704 | + } | |
| 1705 | + | |
| 1601 | 1706 | ret = bpf_jit_emit_func_call_rel(image, fimage, ctx, func_addr); |
| 1602 | 1707 | if (ret) |
| 1603 | 1708 | return ret; |
| @@ -103,6 +103,11 @@ perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, struct pt_regs *re · 102 unmodified lines | ||
| 103 | 103 | void |
| 104 | 104 | perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs) |
| 105 | 105 | { |
| 106 | + perf_callchain_store(entry, perf_arch_instruction_pointer(regs)); | |
| 107 | + | |
| 108 | + if (!current->mm) | |
| 109 | + return; | |
| 110 | + | |
| 106 | 111 | if (!is_32bit_task()) |
| 107 | 112 | perf_callchain_user_64(entry, regs); |
| 108 | 113 | else |
| @@ -142,7 +142,6 @@ void perf_callchain_user_32(struct perf_callchain_entry_ctx *entry, · 141 unmodified lines | ||
| 142 | 142 | next_ip = perf_arch_instruction_pointer(regs); |
| 143 | 143 | lr = regs->link; |
| 144 | 144 | sp = regs->gpr[1]; |
| 145 | − perf_callchain_store(entry, next_ip); | |
| 146 | 145 | |
| 147 | 146 | while (entry->nr < entry->max_stack) { |
| 148 | 147 | fp = (unsigned int __user *) (unsigned long) sp; |
| @@ -77,7 +77,6 @@ void perf_callchain_user_64(struct perf_callchain_entry_ctx *entry, · 76 unmodified lines | ||
| 77 | 77 | next_ip = perf_arch_instruction_pointer(regs); |
| 78 | 78 | lr = regs->link; |
| 79 | 79 | sp = regs->gpr[1]; |
| 80 | − perf_callchain_store(entry, next_ip); | |
| 81 | 80 | |
| 82 | 81 | while (entry->nr < entry->max_stack) { |
| 83 | 82 | fp = (unsigned long __user *) sp; |
| @@ -155,8 +155,8 @@ machine_device_initcall(mpc83xx_km, mpc83xx_declare_of_platform_devices); · 154 unmodified lines | ||
| 155 | 155 | |
| 156 | 156 | /* list of the supported boards */ |
| 157 | 157 | static char *board[] __initdata = { |
| 158 | − "Keymile,KMETER1", | |
| 159 | − "Keymile,kmpbec8321", | |
| 158 | + "keymile,KMETER1", | |
| 159 | + "keymile,kmpbec8321", | |
| 160 | 160 | NULL |
| 161 | 161 | }; |
| 162 | 162 | |
| @@ -276,7 +276,7 @@ config PPC_BOOK3S · 275 unmodified lines | ||
| 276 | 276 | config PPC_E500 |
| 277 | 277 | select FSL_EMB_PERFMON |
| 278 | 278 | bool |
| 279 | − select ARCH_SUPPORTS_HUGETLBFS if PHYS_64BIT || PPC64 | |
| 279 | + select ARCH_SUPPORTS_HUGETLBFS | |
| 280 | 280 | select PPC_SMP_MUXED_IPI |
| 281 | 281 | select PPC_DOORBELL |
| 282 | 282 | select PPC_KUEP |
| @@ -337,7 +337,7 @@ config BOOKE · 54 unmodified lines | ||
| 337 | 337 | config PTE_64BIT |
| 338 | 338 | bool |
| 339 | 339 | depends on 44x || PPC_E500 || PPC_86xx |
| 340 | − default y if PHYS_64BIT | |
| 340 | + default y if PPC_E500 || PHYS_64BIT | |
| 341 | 341 | |
| 342 | 342 | config PHYS_64BIT |
| 343 | 343 | bool 'Large physical address support' if PPC_E500 || PPC_86xx |
| @@ -605,7 +605,7 @@ static int pseries_irq_domain_alloc(struct irq_domain *domain, unsigned int virq · 604 unmodified lines | ||
| 605 | 605 | &pseries_msi_irq_chip, pseries_dev); |
| 606 | 606 | } |
| 607 | 607 | |
| 608 | − pseries_dev->msi_used++; | |
| 608 | + pseries_dev->msi_used += nr_irqs; | |
| 609 | 609 | return 0; |
| 610 | 610 | |
| 611 | 611 | out: |
File renamed without changes.
| @@ -15,9 +15,9 @@ if [ -z "$is_64bit" ]; then · 14 unmodified lines | ||
| 15 | 15 | RELOCATION=R_PPC_ADDR32 |
| 16 | 16 | fi |
| 17 | 17 | |
| 18 | −num_ool_stubs_total=$($objdump -r -j __patchable_function_entries "$vmlinux_o" | | |
| 18 | +num_ool_stubs_total=$($objdump -r -j __patchable_function_entries -d "$vmlinux_o" | | |
| 19 | 19 | grep -c "$RELOCATION") |
| 20 | −num_ool_stubs_inittext=$($objdump -r -j __patchable_function_entries "$vmlinux_o" | | |
| 20 | +num_ool_stubs_inittext=$($objdump -r -j __patchable_function_entries -d "$vmlinux_o" | | |
| 21 | 21 | grep -e ".init.text" -e ".text.startup" | grep -c "$RELOCATION") |
| 22 | 22 | num_ool_stubs_text=$((num_ool_stubs_total - num_ool_stubs_inittext)) |
| 23 | 23 | |
| @@ -170,6 +170,7 @@ SECTIONS · 169 unmodified lines | ||
| 170 | 170 | |
| 171 | 171 | STABS_DEBUG |
| 172 | 172 | DWARF_DEBUG |
| 173 | + MODINFO | |
| 173 | 174 | ELF_DETAILS |
| 174 | 175 | .riscv.attributes 0 : { *(.riscv.attributes) } |
| 175 | 176 | |
| @@ -30,7 +30,6 @@ config KVM · 29 unmodified lines | ||
| 30 | 30 | select KVM_GENERIC_HARDWARE_ENABLING |
| 31 | 31 | select KVM_MMIO |
| 32 | 32 | select VIRT_XFER_TO_GUEST_WORK |
| 33 | − select KVM_GENERIC_MMU_NOTIFIER | |
| 34 | 33 | select SCHED_INFO |
| 35 | 34 | select GUEST_PERF_EVENTS if PERF_EVENTS |
| 36 | 35 | help |
| @@ -13,6 +13,7 @@ · 12 unmodified lines | ||
| 13 | 13 | #include <linux/irqchip/riscv-imsic.h> |
| 14 | 14 | #include <linux/irqdomain.h> |
| 15 | 15 | #include <linux/kvm_host.h> |
| 16 | +#include <linux/nospec.h> | |
| 16 | 17 | #include <linux/percpu.h> |
| 17 | 18 | #include <linux/spinlock.h> |
| 18 | 19 | #include <asm/cpufeature.h> |
| @@ -182,9 +183,14 @@ int kvm_riscv_vcpu_aia_get_csr(struct kvm_vcpu *vcpu, · 163 unmodified lines | ||
| 182 | 183 | unsigned long *out_val) |
| 183 | 184 | { |
| 184 | 185 | struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr; |
| 186 | + unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long); | |
| 185 | 187 | |
| 186 | − if (reg_num >= sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long)) | |
| 188 | + if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA)) | |
| 187 | 189 | return -ENOENT; |
| 190 | + if (reg_num >= regs_max) | |
| 191 | + return -ENOENT; | |
| 192 | + | |
| 193 | + reg_num = array_index_nospec(reg_num, regs_max); | |
| 188 | 194 | |
| 189 | 195 | *out_val = 0; |
| 190 | 196 | if (kvm_riscv_aia_available()) |
| @@ -198,9 +204,14 @@ int kvm_riscv_vcpu_aia_set_csr(struct kvm_vcpu *vcpu, · 7 unmodified lines | ||
| 198 | 204 | unsigned long val) |
| 199 | 205 | { |
| 200 | 206 | struct kvm_vcpu_aia_csr *csr = &vcpu->arch.aia_context.guest_csr; |
| 207 | + unsigned long regs_max = sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long); | |
| 201 | 208 | |
| 202 | − if (reg_num >= sizeof(struct kvm_riscv_aia_csr) / sizeof(unsigned long)) | |
| 209 | + if (!riscv_isa_extension_available(vcpu->arch.isa, SSAIA)) | |
| 203 | 210 | return -ENOENT; |
| 211 | + if (reg_num >= regs_max) | |
| 212 | + return -ENOENT; | |
| 213 | + | |
| 214 | + reg_num = array_index_nospec(reg_num, regs_max); | |
| 204 | 215 | |
| 205 | 216 | if (kvm_riscv_aia_available()) { |
| 206 | 217 | ((unsigned long *)csr)[reg_num] = val; |
| @@ -10,6 +10,7 @@ · 9 unmodified lines | ||
| 10 | 10 | #include <linux/irqchip/riscv-aplic.h> |
| 11 | 11 | #include <linux/kvm_host.h> |
| 12 | 12 | #include <linux/math.h> |
| 13 | +#include <linux/nospec.h> | |
| 13 | 14 | #include <linux/spinlock.h> |
| 14 | 15 | #include <linux/swab.h> |
| 15 | 16 | #include <kvm/iodev.h> |
| @@ -45,7 +46,7 @@ static u32 aplic_read_sourcecfg(struct aplic *aplic, u32 irq) · 29 unmodified lines | ||
| 45 | 46 | |
| 46 | 47 | if (!irq || aplic->nr_irqs <= irq) |
| 47 | 48 | return 0; |
| 48 | − irqd = &aplic->irqs[irq]; | |
| 49 | + irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; | |
| 49 | 50 | |
| 50 | 51 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 51 | 52 | ret = irqd->sourcecfg; |
| @@ -61,7 +62,7 @@ static void aplic_write_sourcecfg(struct aplic *aplic, u32 irq, u32 val) · 9 unmodified lines | ||
| 61 | 62 | |
| 62 | 63 | if (!irq || aplic->nr_irqs <= irq) |
| 63 | 64 | return; |
| 64 | − irqd = &aplic->irqs[irq]; | |
| 65 | + irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; | |
| 65 | 66 | |
| 66 | 67 | if (val & APLIC_SOURCECFG_D) |
| 67 | 68 | val = 0; |
| @@ -81,7 +82,7 @@ static u32 aplic_read_target(struct aplic *aplic, u32 irq) · 13 unmodified lines | ||
| 81 | 82 | |
| 82 | 83 | if (!irq || aplic->nr_irqs <= irq) |
| 83 | 84 | return 0; |
| 84 | − irqd = &aplic->irqs[irq]; | |
| 85 | + irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; | |
| 85 | 86 | |
| 86 | 87 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 87 | 88 | ret = irqd->target; |
| @@ -97,7 +98,7 @@ static void aplic_write_target(struct aplic *aplic, u32 irq, u32 val) · 9 unmodified lines | ||
| 97 | 98 | |
| 98 | 99 | if (!irq || aplic->nr_irqs <= irq) |
| 99 | 100 | return; |
| 100 | − irqd = &aplic->irqs[irq]; | |
| 101 | + irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; | |
| 101 | 102 | |
| 102 | 103 | val &= APLIC_TARGET_EIID_MASK | |
| 103 | 104 | (APLIC_TARGET_HART_IDX_MASK << APLIC_TARGET_HART_IDX_SHIFT) | |
| @@ -116,7 +117,7 @@ static bool aplic_read_pending(struct aplic *aplic, u32 irq) · 12 unmodified lines | ||
| 116 | 117 | |
| 117 | 118 | if (!irq || aplic->nr_irqs <= irq) |
| 118 | 119 | return false; |
| 119 | − irqd = &aplic->irqs[irq]; | |
| 120 | + irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; | |
| 120 | 121 | |
| 121 | 122 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 122 | 123 | ret = (irqd->state & APLIC_IRQ_STATE_PENDING) ? true : false; |
| @@ -132,7 +133,7 @@ static void aplic_write_pending(struct aplic *aplic, u32 irq, bool pending) · 9 unmodified lines | ||
| 132 | 133 | |
| 133 | 134 | if (!irq || aplic->nr_irqs <= irq) |
| 134 | 135 | return; |
| 135 | − irqd = &aplic->irqs[irq]; | |
| 136 | + irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; | |
| 136 | 137 | |
| 137 | 138 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 138 | 139 | |
| @@ -170,7 +171,7 @@ static bool aplic_read_enabled(struct aplic *aplic, u32 irq) · 31 unmodified lines | ||
| 170 | 171 | |
| 171 | 172 | if (!irq || aplic->nr_irqs <= irq) |
| 172 | 173 | return false; |
| 173 | − irqd = &aplic->irqs[irq]; | |
| 174 | + irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; | |
| 174 | 175 | |
| 175 | 176 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 176 | 177 | ret = (irqd->state & APLIC_IRQ_STATE_ENABLED) ? true : false; |
| @@ -186,7 +187,7 @@ static void aplic_write_enabled(struct aplic *aplic, u32 irq, bool enabled) · 9 unmodified lines | ||
| 186 | 187 | |
| 187 | 188 | if (!irq || aplic->nr_irqs <= irq) |
| 188 | 189 | return; |
| 189 | − irqd = &aplic->irqs[irq]; | |
| 190 | + irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; | |
| 190 | 191 | |
| 191 | 192 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 192 | 193 | if (enabled) |
| @@ -205,7 +206,7 @@ static bool aplic_read_input(struct aplic *aplic, u32 irq) · 12 unmodified lines | ||
| 205 | 206 | |
| 206 | 207 | if (!irq || aplic->nr_irqs <= irq) |
| 207 | 208 | return false; |
| 208 | − irqd = &aplic->irqs[irq]; | |
| 209 | + irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; | |
| 209 | 210 | |
| 210 | 211 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 211 | 212 | |
| @@ -254,7 +255,7 @@ static void aplic_update_irq_range(struct kvm *kvm, u32 first, u32 last) · 42 unmodified lines | ||
| 254 | 255 | for (irq = first; irq <= last; irq++) { |
| 255 | 256 | if (!irq || aplic->nr_irqs <= irq) |
| 256 | 257 | continue; |
| 257 | − irqd = &aplic->irqs[irq]; | |
| 258 | + irqd = &aplic->irqs[array_index_nospec(irq, aplic->nr_irqs)]; | |
| 258 | 259 | |
| 259 | 260 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| 260 | 261 | |
| @@ -283,7 +284,7 @@ int kvm_riscv_aia_aplic_inject(struct kvm *kvm, u32 source, bool level) · 22 unmodified lines | ||
| 283 | 284 | |
| 284 | 285 | if (!aplic || !source || (aplic->nr_irqs <= source)) |
| 285 | 286 | return -ENODEV; |
| 286 | − irqd = &aplic->irqs[source]; | |
| 287 | + irqd = &aplic->irqs[array_index_nospec(source, aplic->nr_irqs)]; | |
| 287 | 288 | ie = (aplic->domaincfg & APLIC_DOMAINCFG_IE) ? true : false; |
| 288 | 289 | |
| 289 | 290 | raw_spin_lock_irqsave(&irqd->lock, flags); |
| @@ -11,6 +11,7 @@ · 10 unmodified lines | ||
| 11 | 11 | #include <linux/irqchip/riscv-imsic.h> |
| 12 | 12 | #include <linux/kvm_host.h> |
| 13 | 13 | #include <linux/uaccess.h> |
| 14 | +#include <linux/cpufeature.h> | |
| 14 | 15 | |
| 15 | 16 | static int aia_create(struct kvm_device *dev, u32 type) |
| 16 | 17 | { |
| @@ -22,6 +23,9 @@ static int aia_create(struct kvm_device *dev, u32 type) · 5 unmodified lines | ||
| 22 | 23 | if (irqchip_in_kernel(kvm)) |
| 23 | 24 | return -EEXIST; |
| 24 | 25 | |
| 26 | + if (!riscv_isa_extension_available(NULL, SSAIA)) | |
| 27 | + return -ENODEV; | |
| 28 | + | |
| 25 | 29 | ret = -EBUSY; |
| 26 | 30 | if (kvm_trylock_all_vcpus(kvm)) |
| 27 | 31 | return ret; |
| @@ -437,7 +441,7 @@ static int aia_get_attr(struct kvm_device *dev, struct kvm_device_attr *attr) · 409 unmodified lines | ||
| 437 | 441 | |
| 438 | 442 | static int aia_has_attr(struct kvm_device *dev, struct kvm_device_attr *attr) |
| 439 | 443 | { |
| 440 | − int nr_vcpus; | |
| 444 | + int nr_vcpus, r = -ENXIO; | |
| 441 | 445 | |
| 442 | 446 | switch (attr->group) { |
| 443 | 447 | case KVM_DEV_RISCV_AIA_GRP_CONFIG: |
| @@ -466,12 +470,18 @@ static int aia_has_attr(struct kvm_device *dev, struct kvm_device_attr *attr) · 22 unmodified lines | ||
| 466 | 470 | } |
| 467 | 471 | break; |
| 468 | 472 | case KVM_DEV_RISCV_AIA_GRP_APLIC: |
| 469 | − return kvm_riscv_aia_aplic_has_attr(dev->kvm, attr->attr); | |
| 473 | + mutex_lock(&dev->kvm->lock); | |
| 474 | + r = kvm_riscv_aia_aplic_has_attr(dev->kvm, attr->attr); | |
| 475 | + mutex_unlock(&dev->kvm->lock); | |
| 476 | + break; | |
| 470 | 477 | case KVM_DEV_RISCV_AIA_GRP_IMSIC: |
| 471 | − return kvm_riscv_aia_imsic_has_attr(dev->kvm, attr->attr); | |
| 478 | + mutex_lock(&dev->kvm->lock); | |
| 479 | + r = kvm_riscv_aia_imsic_has_attr(dev->kvm, attr->attr); | |
| 480 | + mutex_unlock(&dev->kvm->lock); | |
| 481 | + break; | |
| 472 | 482 | } |
| 473 | 483 | |
| 474 | − return -ENXIO; | |
| 484 | + return r; | |
| 475 | 485 | } |
| 476 | 486 | |
| 477 | 487 | struct kvm_device_ops kvm_riscv_aia_device_ops = { |
| @@ -908,6 +908,10 @@ int kvm_riscv_vcpu_aia_imsic_rmw(struct kvm_vcpu *vcpu, unsigned long isel, · 907 unmodified lines | ||
| 908 | 908 | int r, rc = KVM_INSN_CONTINUE_NEXT_SEPC; |
| 909 | 909 | struct imsic *imsic = vcpu->arch.aia_context.imsic_state; |
| 910 | 910 | |
| 911 | + /* If IMSIC vCPU state not initialized then forward to user space */ | |
| 912 | + if (!imsic) | |
| 913 | + return KVM_INSN_EXIT_TO_USER_SPACE; | |
| 914 | + | |
| 911 | 915 | if (isel == KVM_RISCV_AIA_IMSIC_TOPEI) { |
| 912 | 916 | /* Read pending and enabled interrupt with highest priority */ |
| 913 | 917 | topei = imsic_mrif_topei(imsic->swfile, imsic->nr_eix, |
| @@ -245,6 +245,7 @@ out: · 244 unmodified lines | ||
| 245 | 245 | bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range) |
| 246 | 246 | { |
| 247 | 247 | struct kvm_gstage gstage; |
| 248 | + bool mmu_locked; | |
| 248 | 249 | |
| 249 | 250 | if (!kvm->arch.pgd) |
| 250 | 251 | return false; |
| @@ -253,9 +254,12 @@ bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range) · 2 unmodified lines | ||
| 253 | 254 | gstage.flags = 0; |
| 254 | 255 | gstage.vmid = READ_ONCE(kvm->arch.vmid.vmid); |
| 255 | 256 | gstage.pgd = kvm->arch.pgd; |
| 257 | + mmu_locked = spin_trylock(&kvm->mmu_lock); | |
| 256 | 258 | kvm_riscv_gstage_unmap_range(&gstage, range->start << PAGE_SHIFT, |
| 257 | 259 | (range->end - range->start) << PAGE_SHIFT, |
| 258 | 260 | range->may_block); |
| 261 | + if (mmu_locked) | |
| 262 | + spin_unlock(&kvm->mmu_lock); | |
| 259 | 263 | return false; |
| 260 | 264 | } |
| 261 | 265 | |
| @@ -535,7 +539,7 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot, · 273 unmodified lines | ||
| 535 | 539 | goto out_unlock; |
| 536 | 540 | |
| 537 | 541 | /* Check if we are backed by a THP and thus use block mapping if possible */ |
| 538 | − if (vma_pagesize == PAGE_SIZE) | |
| 542 | + if (!logging && (vma_pagesize == PAGE_SIZE)) | |
| 539 | 543 | vma_pagesize = transparent_hugepage_adjust(kvm, memslot, hva, &hfn, &gpa); |
| 540 | 544 | |
| 541 | 545 | if (writable) { |
| @@ -24,7 +24,7 @@ · 23 unmodified lines | ||
| 24 | 24 | #define CREATE_TRACE_POINTS |
| 25 | 25 | #include "trace.h" |
| 26 | 26 | |
| 27 | −const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 27 | +const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 28 | 28 | KVM_GENERIC_VCPU_STATS(), |
| 29 | 29 | STATS_DESC_COUNTER(VCPU, ecall_exit_stat), |
| 30 | 30 | STATS_DESC_COUNTER(VCPU, wfi_exit_stat), |
| @@ -10,6 +10,7 @@ · 9 unmodified lines | ||
| 10 | 10 | #include <linux/errno.h> |
| 11 | 11 | #include <linux/err.h> |
| 12 | 12 | #include <linux/kvm_host.h> |
| 13 | +#include <linux/nospec.h> | |
| 13 | 14 | #include <linux/uaccess.h> |
| 14 | 15 | #include <asm/cpufeature.h> |
| 15 | 16 | |
| @@ -93,9 +94,11 @@ int kvm_riscv_vcpu_get_reg_fp(struct kvm_vcpu *vcpu, · 77 unmodified lines | ||
| 93 | 94 | if (reg_num == KVM_REG_RISCV_FP_F_REG(fcsr)) |
| 94 | 95 | reg_val = &cntx->fp.f.fcsr; |
| 95 | 96 | else if ((KVM_REG_RISCV_FP_F_REG(f[0]) <= reg_num) && |
| 96 | − reg_num <= KVM_REG_RISCV_FP_F_REG(f[31])) | |
| 97 | + reg_num <= KVM_REG_RISCV_FP_F_REG(f[31])) { | |
| 98 | + reg_num = array_index_nospec(reg_num, | |
| 99 | + ARRAY_SIZE(cntx->fp.f.f)); | |
| 97 | 100 | reg_val = &cntx->fp.f.f[reg_num]; |
| 98 | − else | |
| 101 | + } else | |
| 99 | 102 | return -ENOENT; |
| 100 | 103 | } else if ((rtype == KVM_REG_RISCV_FP_D) && |
| 101 | 104 | riscv_isa_extension_available(vcpu->arch.isa, d)) { |
| @@ -107,6 +110,8 @@ int kvm_riscv_vcpu_get_reg_fp(struct kvm_vcpu *vcpu, · 5 unmodified lines | ||
| 107 | 110 | reg_num <= KVM_REG_RISCV_FP_D_REG(f[31])) { |
| 108 | 111 | if (KVM_REG_SIZE(reg->id) != sizeof(u64)) |
| 109 | 112 | return -EINVAL; |
| 113 | + reg_num = array_index_nospec(reg_num, | |
| 114 | + ARRAY_SIZE(cntx->fp.d.f)); | |
| 110 | 115 | reg_val = &cntx->fp.d.f[reg_num]; |
| 111 | 116 | } else |
| 112 | 117 | return -ENOENT; |
| @@ -138,9 +143,11 @@ int kvm_riscv_vcpu_set_reg_fp(struct kvm_vcpu *vcpu, · 25 unmodified lines | ||
| 138 | 143 | if (reg_num == KVM_REG_RISCV_FP_F_REG(fcsr)) |
| 139 | 144 | reg_val = &cntx->fp.f.fcsr; |
| 140 | 145 | else if ((KVM_REG_RISCV_FP_F_REG(f[0]) <= reg_num) && |
| 141 | − reg_num <= KVM_REG_RISCV_FP_F_REG(f[31])) | |
| 146 | + reg_num <= KVM_REG_RISCV_FP_F_REG(f[31])) { | |
| 147 | + reg_num = array_index_nospec(reg_num, | |
| 148 | + ARRAY_SIZE(cntx->fp.f.f)); | |
| 142 | 149 | reg_val = &cntx->fp.f.f[reg_num]; |
| 143 | − else | |
| 150 | + } else | |
| 144 | 151 | return -ENOENT; |
| 145 | 152 | } else if ((rtype == KVM_REG_RISCV_FP_D) && |
| 146 | 153 | riscv_isa_extension_available(vcpu->arch.isa, d)) { |
| @@ -152,6 +159,8 @@ int kvm_riscv_vcpu_set_reg_fp(struct kvm_vcpu *vcpu, · 5 unmodified lines | ||
| 152 | 159 | reg_num <= KVM_REG_RISCV_FP_D_REG(f[31])) { |
| 153 | 160 | if (KVM_REG_SIZE(reg->id) != sizeof(u64)) |
| 154 | 161 | return -EINVAL; |
| 162 | + reg_num = array_index_nospec(reg_num, | |
| 163 | + ARRAY_SIZE(cntx->fp.d.f)); | |
| 155 | 164 | reg_val = &cntx->fp.d.f[reg_num]; |
| 156 | 165 | } else |
| 157 | 166 | return -ENOENT; |
| @@ -10,6 +10,7 @@ · 9 unmodified lines | ||
| 10 | 10 | #include <linux/bitops.h> |
| 11 | 11 | #include <linux/errno.h> |
| 12 | 12 | #include <linux/err.h> |
| 13 | +#include <linux/nospec.h> | |
| 13 | 14 | #include <linux/uaccess.h> |
| 14 | 15 | #include <linux/kvm_host.h> |
| 15 | 16 | #include <asm/cacheflush.h> |
| @@ -127,6 +128,7 @@ static int kvm_riscv_vcpu_isa_check_host(unsigned long kvm_ext, unsigned long *g · 111 unmodified lines | ||
| 127 | 128 | kvm_ext >= ARRAY_SIZE(kvm_isa_ext_arr)) |
| 128 | 129 | return -ENOENT; |
| 129 | 130 | |
| 131 | + kvm_ext = array_index_nospec(kvm_ext, ARRAY_SIZE(kvm_isa_ext_arr)); | |
| 130 | 132 | *guest_ext = kvm_isa_ext_arr[kvm_ext]; |
| 131 | 133 | switch (*guest_ext) { |
| 132 | 134 | case RISCV_ISA_EXT_SMNPM: |
| @@ -443,13 +445,16 @@ static int kvm_riscv_vcpu_get_reg_core(struct kvm_vcpu *vcpu, · 310 unmodified lines | ||
| 443 | 445 | unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK | |
| 444 | 446 | KVM_REG_SIZE_MASK | |
| 445 | 447 | KVM_REG_RISCV_CORE); |
| 448 | + unsigned long regs_max = sizeof(struct kvm_riscv_core) / sizeof(unsigned long); | |
| 446 | 449 | unsigned long reg_val; |
| 447 | 450 | |
| 448 | 451 | if (KVM_REG_SIZE(reg->id) != sizeof(unsigned long)) |
| 449 | 452 | return -EINVAL; |
| 450 | − if (reg_num >= sizeof(struct kvm_riscv_core) / sizeof(unsigned long)) | |
| 453 | + if (reg_num >= regs_max) | |
| 451 | 454 | return -ENOENT; |
| 452 | 455 | |
| 456 | + reg_num = array_index_nospec(reg_num, regs_max); | |
| 457 | + | |
| 453 | 458 | if (reg_num == KVM_REG_RISCV_CORE_REG(regs.pc)) |
| 454 | 459 | reg_val = cntx->sepc; |
| 455 | 460 | else if (KVM_REG_RISCV_CORE_REG(regs.pc) < reg_num && |
| @@ -476,13 +481,16 @@ static int kvm_riscv_vcpu_set_reg_core(struct kvm_vcpu *vcpu, · 20 unmodified lines | ||
| 476 | 481 | unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK | |
| 477 | 482 | KVM_REG_SIZE_MASK | |
| 478 | 483 | KVM_REG_RISCV_CORE); |
| 484 | + unsigned long regs_max = sizeof(struct kvm_riscv_core) / sizeof(unsigned long); | |
| 479 | 485 | unsigned long reg_val; |
| 480 | 486 | |
| 481 | 487 | if (KVM_REG_SIZE(reg->id) != sizeof(unsigned long)) |
| 482 | 488 | return -EINVAL; |
| 483 | − if (reg_num >= sizeof(struct kvm_riscv_core) / sizeof(unsigned long)) | |
| 489 | + if (reg_num >= regs_max) | |
| 484 | 490 | return -ENOENT; |
| 485 | 491 | |
| 492 | + reg_num = array_index_nospec(reg_num, regs_max); | |
| 493 | + | |
| 486 | 494 | if (copy_from_user(®_val, uaddr, KVM_REG_SIZE(reg->id))) |
| 487 | 495 | return -EFAULT; |
| 488 | 496 | |
| @@ -507,10 +515,13 @@ static int kvm_riscv_vcpu_general_get_csr(struct kvm_vcpu *vcpu, · 18 unmodified lines | ||
| 507 | 515 | unsigned long *out_val) |
| 508 | 516 | { |
| 509 | 517 | struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr; |
| 518 | + unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long); | |
| 510 | 519 | |
| 511 | − if (reg_num >= sizeof(struct kvm_riscv_csr) / sizeof(unsigned long)) | |
| 520 | + if (reg_num >= regs_max) | |
| 512 | 521 | return -ENOENT; |
| 513 | 522 | |
| 523 | + reg_num = array_index_nospec(reg_num, regs_max); | |
| 524 | + | |
| 514 | 525 | if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) { |
| 515 | 526 | kvm_riscv_vcpu_flush_interrupts(vcpu); |
| 516 | 527 | *out_val = (csr->hvip >> VSIP_TO_HVIP_SHIFT) & VSIP_VALID_MASK; |
| @@ -526,10 +537,13 @@ static int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu, · 9 unmodified lines | ||
| 526 | 537 | unsigned long reg_val) |
| 527 | 538 | { |
| 528 | 539 | struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr; |
| 540 | + unsigned long regs_max = sizeof(struct kvm_riscv_csr) / sizeof(unsigned long); | |
| 529 | 541 | |
| 530 | − if (reg_num >= sizeof(struct kvm_riscv_csr) / sizeof(unsigned long)) | |
| 542 | + if (reg_num >= regs_max) | |
| 531 | 543 | return -ENOENT; |
| 532 | 544 | |
| 545 | + reg_num = array_index_nospec(reg_num, regs_max); | |
| 546 | + | |
| 533 | 547 | if (reg_num == KVM_REG_RISCV_CSR_REG(sip)) { |
| 534 | 548 | reg_val &= VSIP_VALID_MASK; |
| 535 | 549 | reg_val <<= VSIP_TO_HVIP_SHIFT; |
| @@ -548,10 +562,15 @@ static inline int kvm_riscv_vcpu_smstateen_set_csr(struct kvm_vcpu *vcpu, · 12 unmodified lines | ||
| 548 | 562 | unsigned long reg_val) |
| 549 | 563 | { |
| 550 | 564 | struct kvm_vcpu_smstateen_csr *csr = &vcpu->arch.smstateen_csr; |
| 565 | + unsigned long regs_max = sizeof(struct kvm_riscv_smstateen_csr) / | |
| 566 | + sizeof(unsigned long); | |
| 551 | 567 | |
| 552 | − if (reg_num >= sizeof(struct kvm_riscv_smstateen_csr) / | |
| 553 | − sizeof(unsigned long)) | |
| 554 | − return -EINVAL; | |
| 568 | + if (!riscv_isa_extension_available(vcpu->arch.isa, SMSTATEEN)) | |
| 569 | + return -ENOENT; | |
| 570 | + if (reg_num >= regs_max) | |
| 571 | + return -ENOENT; | |
| 572 | + | |
| 573 | + reg_num = array_index_nospec(reg_num, regs_max); | |
| 555 | 574 | |
| 556 | 575 | ((unsigned long *)csr)[reg_num] = reg_val; |
| 557 | 576 | return 0; |
| @@ -562,10 +581,15 @@ static int kvm_riscv_vcpu_smstateen_get_csr(struct kvm_vcpu *vcpu, · 4 unmodified lines | ||
| 562 | 581 | unsigned long *out_val) |
| 563 | 582 | { |
| 564 | 583 | struct kvm_vcpu_smstateen_csr *csr = &vcpu->arch.smstateen_csr; |
| 584 | + unsigned long regs_max = sizeof(struct kvm_riscv_smstateen_csr) / | |
| 585 | + sizeof(unsigned long); | |
| 565 | 586 | |
| 566 | − if (reg_num >= sizeof(struct kvm_riscv_smstateen_csr) / | |
| 567 | − sizeof(unsigned long)) | |
| 568 | − return -EINVAL; | |
| 587 | + if (!riscv_isa_extension_available(vcpu->arch.isa, SMSTATEEN)) | |
| 588 | + return -ENOENT; | |
| 589 | + if (reg_num >= regs_max) | |
| 590 | + return -ENOENT; | |
| 591 | + | |
| 592 | + reg_num = array_index_nospec(reg_num, regs_max); | |
| 569 | 593 | |
| 570 | 594 | *out_val = ((unsigned long *)csr)[reg_num]; |
| 571 | 595 | return 0; |
| @@ -595,10 +619,7 @@ static int kvm_riscv_vcpu_get_reg_csr(struct kvm_vcpu *vcpu, · 23 unmodified lines | ||
| 595 | 619 | rc = kvm_riscv_vcpu_aia_get_csr(vcpu, reg_num, ®_val); |
| 596 | 620 | break; |
| 597 | 621 | case KVM_REG_RISCV_CSR_SMSTATEEN: |
| 598 | − rc = -EINVAL; | |
| 599 | − if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SMSTATEEN)) | |
| 600 | − rc = kvm_riscv_vcpu_smstateen_get_csr(vcpu, reg_num, | |
| 601 | − ®_val); | |
| 622 | + rc = kvm_riscv_vcpu_smstateen_get_csr(vcpu, reg_num, ®_val); | |
| 602 | 623 | break; |
| 603 | 624 | default: |
| 604 | 625 | rc = -ENOENT; |
| @@ -640,10 +661,7 @@ static int kvm_riscv_vcpu_set_reg_csr(struct kvm_vcpu *vcpu, · 35 unmodified lines | ||
| 640 | 661 | rc = kvm_riscv_vcpu_aia_set_csr(vcpu, reg_num, reg_val); |
| 641 | 662 | break; |
| 642 | 663 | case KVM_REG_RISCV_CSR_SMSTATEEN: |
| 643 | − rc = -EINVAL; | |
| 644 | − if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SMSTATEEN)) | |
| 645 | − rc = kvm_riscv_vcpu_smstateen_set_csr(vcpu, reg_num, | |
| 646 | − reg_val); | |
| 664 | + rc = kvm_riscv_vcpu_smstateen_set_csr(vcpu, reg_num, reg_val); | |
| 647 | 665 | break; |
| 648 | 666 | default: |
| 649 | 667 | rc = -ENOENT; |
| @@ -10,6 +10,7 @@ · 9 unmodified lines | ||
| 10 | 10 | #include <linux/errno.h> |
| 11 | 11 | #include <linux/err.h> |
| 12 | 12 | #include <linux/kvm_host.h> |
| 13 | +#include <linux/nospec.h> | |
| 13 | 14 | #include <linux/perf/riscv_pmu.h> |
| 14 | 15 | #include <asm/csr.h> |
| 15 | 16 | #include <asm/kvm_vcpu_sbi.h> |
| @@ -87,7 +88,8 @@ static void kvm_pmu_release_perf_event(struct kvm_pmc *pmc) · 71 unmodified lines | ||
| 87 | 88 | |
| 88 | 89 | static u64 kvm_pmu_get_perf_event_hw_config(u32 sbi_event_code) |
| 89 | 90 | { |
| 90 | − return hw_event_perf_map[sbi_event_code]; | |
| 91 | + return hw_event_perf_map[array_index_nospec(sbi_event_code, | |
| 92 | + SBI_PMU_HW_GENERAL_MAX)]; | |
| 91 | 93 | } |
| 92 | 94 | |
| 93 | 95 | static u64 kvm_pmu_get_perf_event_cache_config(u32 sbi_event_code) |
| @@ -218,6 +220,7 @@ static int pmu_fw_ctr_read_hi(struct kvm_vcpu *vcpu, unsigned long cidx, · 124 unmodified lines | ||
| 218 | 220 | return -EINVAL; |
| 219 | 221 | } |
| 220 | 222 | |
| 223 | + cidx = array_index_nospec(cidx, RISCV_KVM_MAX_COUNTERS); | |
| 221 | 224 | pmc = &kvpmu->pmc[cidx]; |
| 222 | 225 | |
| 223 | 226 | if (pmc->cinfo.type != SBI_PMU_CTR_TYPE_FW) |
| @@ -244,6 +247,7 @@ static int pmu_ctr_read(struct kvm_vcpu *vcpu, unsigned long cidx, · 20 unmodified lines | ||
| 244 | 247 | return -EINVAL; |
| 245 | 248 | } |
| 246 | 249 | |
| 250 | + cidx = array_index_nospec(cidx, RISCV_KVM_MAX_COUNTERS); | |
| 247 | 251 | pmc = &kvpmu->pmc[cidx]; |
| 248 | 252 | |
| 249 | 253 | if (pmc->cinfo.type == SBI_PMU_CTR_TYPE_FW) { |
| @@ -520,11 +524,12 @@ int kvm_riscv_vcpu_pmu_ctr_info(struct kvm_vcpu *vcpu, unsigned long cidx, · 270 unmodified lines | ||
| 520 | 524 | { |
| 521 | 525 | struct kvm_pmu *kvpmu = vcpu_to_pmu(vcpu); |
| 522 | 526 | |
| 523 | − if (cidx > RISCV_KVM_MAX_COUNTERS || cidx == 1) { | |
| 527 | + if (cidx >= RISCV_KVM_MAX_COUNTERS || cidx == 1) { | |
| 524 | 528 | retdata->err_val = SBI_ERR_INVALID_PARAM; |
| 525 | 529 | return 0; |
| 526 | 530 | } |
| 527 | 531 | |
| 532 | + cidx = array_index_nospec(cidx, RISCV_KVM_MAX_COUNTERS); | |
| 528 | 533 | retdata->out_val = kvpmu->pmc[cidx].cinfo.value; |
| 529 | 534 | |
| 530 | 535 | return 0; |
| @@ -559,7 +564,8 @@ int kvm_riscv_vcpu_pmu_ctr_start(struct kvm_vcpu *vcpu, unsigned long ctr_base, · 28 unmodified lines | ||
| 559 | 564 | } |
| 560 | 565 | /* Start the counters that have been configured and requested by the guest */ |
| 561 | 566 | for_each_set_bit(i, &ctr_mask, RISCV_MAX_COUNTERS) { |
| 562 | − pmc_index = i + ctr_base; | |
| 567 | + pmc_index = array_index_nospec(i + ctr_base, | |
| 568 | + RISCV_KVM_MAX_COUNTERS); | |
| 563 | 569 | if (!test_bit(pmc_index, kvpmu->pmc_in_use)) |
| 564 | 570 | continue; |
| 565 | 571 | /* The guest started the counter again. Reset the overflow status */ |
| @@ -630,7 +636,8 @@ int kvm_riscv_vcpu_pmu_ctr_stop(struct kvm_vcpu *vcpu, unsigned long ctr_base, · 64 unmodified lines | ||
| 630 | 636 | |
| 631 | 637 | /* Stop the counters that have been configured and requested by the guest */ |
| 632 | 638 | for_each_set_bit(i, &ctr_mask, RISCV_MAX_COUNTERS) { |
| 633 | − pmc_index = i + ctr_base; | |
| 639 | + pmc_index = array_index_nospec(i + ctr_base, | |
| 640 | + RISCV_KVM_MAX_COUNTERS); | |
| 634 | 641 | if (!test_bit(pmc_index, kvpmu->pmc_in_use)) |
| 635 | 642 | continue; |
| 636 | 643 | pmc = &kvpmu->pmc[pmc_index]; |
| @@ -761,6 +768,7 @@ int kvm_riscv_vcpu_pmu_ctr_cfg_match(struct kvm_vcpu *vcpu, unsigned long ctr_ba · 124 unmodified lines | ||
| 761 | 768 | } |
| 762 | 769 | } |
| 763 | 770 | |
| 771 | + ctr_idx = array_index_nospec(ctr_idx, RISCV_KVM_MAX_COUNTERS); | |
| 764 | 772 | pmc = &kvpmu->pmc[ctr_idx]; |
| 765 | 773 | pmc->idx = ctr_idx; |
| 766 | 774 | |
| @@ -13,7 +13,7 @@ · 12 unmodified lines | ||
| 13 | 13 | #include <linux/kvm_host.h> |
| 14 | 14 | #include <asm/kvm_mmu.h> |
| 15 | 15 | |
| 16 | −const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 16 | +const struct kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 17 | 17 | KVM_GENERIC_VM_STATS() |
| 18 | 18 | }; |
| 19 | 19 | static_assert(ARRAY_SIZE(kvm_vm_stats_desc) == |
| @@ -181,7 +181,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 161 unmodified lines | ||
| 181 | 181 | break; |
| 182 | 182 | case KVM_CAP_IOEVENTFD: |
| 183 | 183 | case KVM_CAP_USER_MEMORY: |
| 184 | − case KVM_CAP_SYNC_MMU: | |
| 185 | 184 | case KVM_CAP_DESTROY_MEMORY_REGION_WORKS: |
| 186 | 185 | case KVM_CAP_ONE_REG: |
| 187 | 186 | case KVM_CAP_READONLY_MEM: |
| @@ -19,9 +19,9 @@ struct s390_idle_data { · 18 unmodified lines | ||
| 19 | 19 | unsigned long mt_cycles_enter[8]; |
| 20 | 20 | }; |
| 21 | 21 | |
| 22 | +DECLARE_PER_CPU(struct s390_idle_data, s390_idle); | |
| 23 | + | |
| 22 | 24 | extern struct device_attribute dev_attr_idle_count; |
| 23 | 25 | extern struct device_attribute dev_attr_idle_time_us; |
| 24 | 26 | |
| 25 | −void psw_idle(struct s390_idle_data *data, unsigned long psw_mask); | |
| 26 | − | |
| 27 | 27 | #endif /* _S390_IDLE_H */ |
| @@ -159,7 +159,7 @@ static __always_inline void __stackleak_poison(unsigned long erase_low, · 158 unmodified lines | ||
| 159 | 159 | " j 4f\n" |
| 160 | 160 | "3: mvc 8(1,%[addr]),0(%[addr])\n" |
| 161 | 161 | "4:" |
| 162 | − : [addr] "+&a" (erase_low), [count] "+&d" (count), [tmp] "=&a" (tmp) | |
| 162 | + : [addr] "+&a" (erase_low), [count] "+&a" (count), [tmp] "=&a" (tmp) | |
| 163 | 163 | : [poison] "d" (poison) |
| 164 | 164 | : "memory", "cc" |
| 165 | 165 | ); |
| @@ -2,6 +2,12 @@ · 1 unmodified line | ||
| 2 | 2 | #ifndef _S390_VTIME_H |
| 3 | 3 | #define _S390_VTIME_H |
| 4 | 4 | |
| 5 | +#include <asm/lowcore.h> | |
| 6 | +#include <asm/cpu_mf.h> | |
| 7 | +#include <asm/idle.h> | |
| 8 | + | |
| 9 | +DECLARE_PER_CPU(u64, mt_cycles[8]); | |
| 10 | + | |
| 5 | 11 | static inline void update_timer_sys(void) |
| 6 | 12 | { |
| 7 | 13 | struct lowcore *lc = get_lowcore(); |
| @@ -20,4 +26,32 @@ static inline void update_timer_mcck(void) · 12 unmodified lines | ||
| 20 | 26 | lc->last_update_timer = lc->mcck_enter_timer; |
| 21 | 27 | } |
| 22 | 28 | |
| 29 | +static inline void update_timer_idle(void) | |
| 30 | +{ | |
| 31 | + struct s390_idle_data *idle = this_cpu_ptr(&s390_idle); | |
| 32 | + struct lowcore *lc = get_lowcore(); | |
| 33 | + u64 cycles_new[8]; | |
| 34 | + int i, mtid; | |
| 35 | + | |
| 36 | + mtid = smp_cpu_mtid; | |
| 37 | + if (mtid) { | |
| 38 | + stcctm(MT_DIAG, mtid, cycles_new); | |
| 39 | + for (i = 0; i < mtid; i++) | |
| 40 | + __this_cpu_add(mt_cycles[i], cycles_new[i] - idle->mt_cycles_enter[i]); | |
| 41 | + } | |
| 42 | + /* | |
| 43 | + * This is a bit subtle: Forward last_update_clock so it excludes idle | |
| 44 | + * time. For correct steal time calculation in do_account_vtime() add | |
| 45 | + * passed wall time before idle_enter to steal_timer: | |
| 46 | + * During the passed wall time before idle_enter CPU time may have | |
| 47 | + * been accounted to system, hardirq, softirq, etc. lowcore fields. | |
| 48 | + * The accounted CPU times will be subtracted again from steal_timer | |
| 49 | + * when accumulated steal time is calculated in do_account_vtime(). | |
| 50 | + */ | |
| 51 | + lc->steal_timer += idle->clock_idle_enter - lc->last_update_clock; | |
| 52 | + lc->last_update_clock = lc->int_clock; | |
| 53 | + lc->system_timer += lc->last_update_timer - idle->timer_idle_enter; | |
| 54 | + lc->last_update_timer = lc->sys_enter_timer; | |
| 55 | +} | |
| 56 | + | |
| 23 | 57 | #endif /* _S390_VTIME_H */ |
| @@ -56,8 +56,6 @@ long sys_s390_pci_mmio_write(unsigned long, const void __user *, size_t); · 55 unmodified lines | ||
| 56 | 56 | long sys_s390_pci_mmio_read(unsigned long, void __user *, size_t); |
| 57 | 57 | long sys_s390_sthyi(unsigned long function_code, void __user *buffer, u64 __user *return_code, unsigned long flags); |
| 58 | 58 | |
| 59 | −DECLARE_PER_CPU(u64, mt_cycles[8]); | |
| 60 | − | |
| 61 | 59 | unsigned long stack_alloc(void); |
| 62 | 60 | void stack_free(unsigned long stack); |
| 63 | 61 | |
| @@ -15,37 +15,22 @@ · 14 unmodified lines | ||
| 15 | 15 | #include <trace/events/power.h> |
| 16 | 16 | #include <asm/cpu_mf.h> |
| 17 | 17 | #include <asm/cputime.h> |
| 18 | +#include <asm/idle.h> | |
| 18 | 19 | #include <asm/nmi.h> |
| 19 | 20 | #include <asm/smp.h> |
| 20 | −#include "entry.h" | |
| 21 | 21 | |
| 22 | −static DEFINE_PER_CPU(struct s390_idle_data, s390_idle); | |
| 22 | +DEFINE_PER_CPU(struct s390_idle_data, s390_idle); | |
| 23 | 23 | |
| 24 | 24 | void account_idle_time_irq(void) |
| 25 | 25 | { |
| 26 | 26 | struct s390_idle_data *idle = this_cpu_ptr(&s390_idle); |
| 27 | − struct lowcore *lc = get_lowcore(); | |
| 28 | 27 | unsigned long idle_time; |
| 29 | − u64 cycles_new[8]; | |
| 30 | − int i; | |
| 31 | 28 | |
| 32 | − if (smp_cpu_mtid) { | |
| 33 | − stcctm(MT_DIAG, smp_cpu_mtid, cycles_new); | |
| 34 | − for (i = 0; i < smp_cpu_mtid; i++) | |
| 35 | − this_cpu_add(mt_cycles[i], cycles_new[i] - idle->mt_cycles_enter[i]); | |
| 36 | − } | |
| 37 | − | |
| 38 | − idle_time = lc->int_clock - idle->clock_idle_enter; | |
| 39 | − | |
| 40 | − lc->steal_timer += idle->clock_idle_enter - lc->last_update_clock; | |
| 41 | − lc->last_update_clock = lc->int_clock; | |
| 42 | − | |
| 43 | − lc->system_timer += lc->last_update_timer - idle->timer_idle_enter; | |
| 44 | − lc->last_update_timer = lc->sys_enter_timer; | |
| 29 | + idle_time = get_lowcore()->int_clock - idle->clock_idle_enter; | |
| 45 | 30 | |
| 46 | 31 | /* Account time spent with enabled wait psw loaded as idle time. */ |
| 47 | − WRITE_ONCE(idle->idle_time, READ_ONCE(idle->idle_time) + idle_time); | |
| 48 | − WRITE_ONCE(idle->idle_count, READ_ONCE(idle->idle_count) + 1); | |
| 32 | + __atomic64_add(idle_time, &idle->idle_time); | |
| 33 | + __atomic64_add_const(1, &idle->idle_count); | |
| 49 | 34 | account_idle_time(cputime_to_nsecs(idle_time)); |
| 50 | 35 | } |
| 51 | 36 | |
| @@ -2377,7 +2377,7 @@ void __init setup_ipl(void) · 2376 unmodified lines | ||
| 2377 | 2377 | atomic_notifier_chain_register(&panic_notifier_list, &on_panic_nb); |
| 2378 | 2378 | } |
| 2379 | 2379 | |
| 2380 | −void s390_reset_system(void) | |
| 2380 | +void __no_stack_protector s390_reset_system(void) | |
| 2381 | 2381 | { |
| 2382 | 2382 | /* Disable prefixing */ |
| 2383 | 2383 | set_prefix(0); |
| @@ -146,6 +146,10 @@ void noinstr do_io_irq(struct pt_regs *regs) · 145 unmodified lines | ||
| 146 | 146 | struct pt_regs *old_regs = set_irq_regs(regs); |
| 147 | 147 | bool from_idle; |
| 148 | 148 | |
| 149 | + from_idle = test_and_clear_cpu_flag(CIF_ENABLED_WAIT); | |
| 150 | + if (from_idle) | |
| 151 | + update_timer_idle(); | |
| 152 | + | |
| 149 | 153 | irq_enter_rcu(); |
| 150 | 154 | |
| 151 | 155 | if (user_mode(regs)) { |
| @@ -154,7 +158,6 @@ void noinstr do_io_irq(struct pt_regs *regs) · 2 unmodified lines | ||
| 154 | 158 | current->thread.last_break = regs->last_break; |
| 155 | 159 | } |
| 156 | 160 | |
| 157 | − from_idle = test_and_clear_cpu_flag(CIF_ENABLED_WAIT); | |
| 158 | 161 | if (from_idle) |
| 159 | 162 | account_idle_time_irq(); |
| 160 | 163 | |
| @@ -182,6 +185,10 @@ void noinstr do_ext_irq(struct pt_regs *regs) · 21 unmodified lines | ||
| 182 | 185 | struct pt_regs *old_regs = set_irq_regs(regs); |
| 183 | 186 | bool from_idle; |
| 184 | 187 | |
| 188 | + from_idle = test_and_clear_cpu_flag(CIF_ENABLED_WAIT); | |
| 189 | + if (from_idle) | |
| 190 | + update_timer_idle(); | |
| 191 | + | |
| 185 | 192 | irq_enter_rcu(); |
| 186 | 193 | |
| 187 | 194 | if (user_mode(regs)) { |
| @@ -194,7 +201,6 @@ void noinstr do_ext_irq(struct pt_regs *regs) · 6 unmodified lines | ||
| 194 | 201 | regs->int_parm = get_lowcore()->ext_params; |
| 195 | 202 | regs->int_parm_long = get_lowcore()->ext_params2; |
| 196 | 203 | |
| 197 | − from_idle = test_and_clear_cpu_flag(CIF_ENABLED_WAIT); | |
| 198 | 204 | if (from_idle) |
| 199 | 205 | account_idle_time_irq(); |
| 200 | 206 | |
| @@ -221,6 +221,7 @@ SECTIONS · 220 unmodified lines | ||
| 221 | 221 | /* Debugging sections. */ |
| 222 | 222 | STABS_DEBUG |
| 223 | 223 | DWARF_DEBUG |
| 224 | + MODINFO | |
| 224 | 225 | ELF_DETAILS |
| 225 | 226 | |
| 226 | 227 | /* |
| @@ -48,8 +48,7 @@ static inline void set_vtimer(u64 expires) · 47 unmodified lines | ||
| 48 | 48 | |
| 49 | 49 | static inline int virt_timer_forward(u64 elapsed) |
| 50 | 50 | { |
| 51 | − BUG_ON(!irqs_disabled()); | |
| 52 | − | |
| 51 | + lockdep_assert_irqs_disabled(); | |
| 53 | 52 | if (list_empty(&virt_timer_list)) |
| 54 | 53 | return 0; |
| 55 | 54 | elapsed = atomic64_add_return(elapsed, &virt_timer_elapsed); |
| @@ -137,23 +136,16 @@ static int do_account_vtime(struct task_struct *tsk) · 81 unmodified lines | ||
| 137 | 136 | lc->system_timer += timer; |
| 138 | 137 | |
| 139 | 138 | /* Update MT utilization calculation */ |
| 140 | − if (smp_cpu_mtid && | |
| 141 | − time_after64(jiffies_64, this_cpu_read(mt_scaling_jiffies))) | |
| 139 | + if (smp_cpu_mtid && time_after64(jiffies_64, __this_cpu_read(mt_scaling_jiffies))) | |
| 142 | 140 | update_mt_scaling(); |
| 143 | 141 | |
| 144 | 142 | /* Calculate cputime delta */ |
| 145 | − user = update_tsk_timer(&tsk->thread.user_timer, | |
| 146 | − READ_ONCE(lc->user_timer)); | |
| 147 | − guest = update_tsk_timer(&tsk->thread.guest_timer, | |
| 148 | − READ_ONCE(lc->guest_timer)); | |
| 149 | − system = update_tsk_timer(&tsk->thread.system_timer, | |
| 150 | − READ_ONCE(lc->system_timer)); | |
| 151 | − hardirq = update_tsk_timer(&tsk->thread.hardirq_timer, | |
| 152 | − READ_ONCE(lc->hardirq_timer)); | |
| 153 | − softirq = update_tsk_timer(&tsk->thread.softirq_timer, | |
| 154 | − READ_ONCE(lc->softirq_timer)); | |
| 155 | − lc->steal_timer += | |
| 156 | − clock - user - guest - system - hardirq - softirq; | |
| 143 | + user = update_tsk_timer(&tsk->thread.user_timer, lc->user_timer); | |
| 144 | + guest = update_tsk_timer(&tsk->thread.guest_timer, lc->guest_timer); | |
| 145 | + system = update_tsk_timer(&tsk->thread.system_timer, lc->system_timer); | |
| 146 | + hardirq = update_tsk_timer(&tsk->thread.hardirq_timer, lc->hardirq_timer); | |
| 147 | + softirq = update_tsk_timer(&tsk->thread.softirq_timer, lc->softirq_timer); | |
| 148 | + lc->steal_timer += clock - user - guest - system - hardirq - softirq; | |
| 157 | 149 | |
| 158 | 150 | /* Push account value */ |
| 159 | 151 | if (user) { |
| @@ -225,10 +217,6 @@ static u64 vtime_delta(void) · 65 unmodified lines | ||
| 225 | 217 | return timer - lc->last_update_timer; |
| 226 | 218 | } |
| 227 | 219 | |
| 228 | −/* | |
| 229 | − * Update process times based on virtual cpu times stored by entry.S | |
| 230 | − * to the lowcore fields user_timer, system_timer & steal_clock. | |
| 231 | − */ | |
| 232 | 220 | void vtime_account_kernel(struct task_struct *tsk) |
| 233 | 221 | { |
| 234 | 222 | struct lowcore *lc = get_lowcore(); |
| @@ -238,27 +226,17 @@ void vtime_account_kernel(struct task_struct *tsk) · 3 unmodified lines | ||
| 238 | 226 | lc->guest_timer += delta; |
| 239 | 227 | else |
| 240 | 228 | lc->system_timer += delta; |
| 241 | − | |
| 242 | − virt_timer_forward(delta); | |
| 243 | 229 | } |
| 244 | 230 | EXPORT_SYMBOL_GPL(vtime_account_kernel); |
| 245 | 231 | |
| 246 | 232 | void vtime_account_softirq(struct task_struct *tsk) |
| 247 | 233 | { |
| 248 | − u64 delta = vtime_delta(); | |
| 249 | − | |
| 250 | − get_lowcore()->softirq_timer += delta; | |
| 251 | − | |
| 252 | − virt_timer_forward(delta); | |
| 234 | + get_lowcore()->softirq_timer += vtime_delta(); | |
| 253 | 235 | } |
| 254 | 236 | |
| 255 | 237 | void vtime_account_hardirq(struct task_struct *tsk) |
| 256 | 238 | { |
| 257 | − u64 delta = vtime_delta(); | |
| 258 | − | |
| 259 | − get_lowcore()->hardirq_timer += delta; | |
| 260 | − | |
| 261 | − virt_timer_forward(delta); | |
| 239 | + get_lowcore()->hardirq_timer += vtime_delta(); | |
| 262 | 240 | } |
| 263 | 241 | |
| 264 | 242 | /* |
| @@ -28,9 +28,7 @@ config KVM · 27 unmodified lines | ||
| 28 | 28 | select HAVE_KVM_INVALID_WAKEUPS |
| 29 | 29 | select HAVE_KVM_NO_POLL |
| 30 | 30 | select KVM_VFIO |
| 31 | − select MMU_NOTIFIER | |
| 32 | 31 | select VIRT_XFER_TO_GUEST_WORK |
| 33 | − select KVM_GENERIC_MMU_NOTIFIER | |
| 34 | 32 | select KVM_MMU_LOCKLESS_AGING |
| 35 | 33 | help |
| 36 | 34 | Support hosting paravirtualized guest machines using the SIE |
| @@ -65,7 +65,7 @@ · 64 unmodified lines | ||
| 65 | 65 | #define VCPU_IRQS_MAX_BUF (sizeof(struct kvm_s390_irq) * \ |
| 66 | 66 | (KVM_MAX_VCPUS + LOCAL_IRQS)) |
| 67 | 67 | |
| 68 | −const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 68 | +const struct kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 69 | 69 | KVM_GENERIC_VM_STATS(), |
| 70 | 70 | STATS_DESC_COUNTER(VM, inject_io), |
| 71 | 71 | STATS_DESC_COUNTER(VM, inject_float_mchk), |
| @@ -91,7 +91,7 @@ const struct kvm_stats_header kvm_vm_stats_header = { · 19 unmodified lines | ||
| 91 | 91 | sizeof(kvm_vm_stats_desc), |
| 92 | 92 | }; |
| 93 | 93 | |
| 94 | −const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 94 | +const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 95 | 95 | KVM_GENERIC_VCPU_STATS(), |
| 96 | 96 | STATS_DESC_COUNTER(VCPU, exit_userspace), |
| 97 | 97 | STATS_DESC_COUNTER(VCPU, exit_null), |
| @@ -601,7 +601,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 503 unmodified lines | ||
| 601 | 601 | switch (ext) { |
| 602 | 602 | case KVM_CAP_S390_PSW: |
| 603 | 603 | case KVM_CAP_S390_GMAP: |
| 604 | − case KVM_CAP_SYNC_MMU: | |
| 605 | 604 | #ifdef CONFIG_KVM_S390_UCONTROL |
| 606 | 605 | case KVM_CAP_S390_UCONTROL: |
| 607 | 606 | #endif |
| @@ -28,8 +28,8 @@ static void xor_xc_2(unsigned long bytes, unsigned long * __restrict p1, · 27 unmodified lines | ||
| 28 | 28 | " j 3f\n" |
| 29 | 29 | "2: xc 0(1,%1),0(%2)\n" |
| 30 | 30 | "3:" |
| 31 | − : : "d" (bytes), "a" (p1), "a" (p2) | |
| 32 | − : "0", "cc", "memory"); | |
| 31 | + : "+a" (bytes), "+a" (p1), "+a" (p2) | |
| 32 | + : : "0", "cc", "memory"); | |
| 33 | 33 | } |
| 34 | 34 | |
| 35 | 35 | static void xor_xc_3(unsigned long bytes, unsigned long * __restrict p1, |
| @@ -54,7 +54,7 @@ static void xor_xc_3(unsigned long bytes, unsigned long * __restrict p1, · 18 unmodified lines | ||
| 54 | 54 | "2: xc 0(1,%1),0(%2)\n" |
| 55 | 55 | "3: xc 0(1,%1),0(%3)\n" |
| 56 | 56 | "4:" |
| 57 | − : "+d" (bytes), "+a" (p1), "+a" (p2), "+a" (p3) | |
| 57 | + : "+a" (bytes), "+a" (p1), "+a" (p2), "+a" (p3) | |
| 58 | 58 | : : "0", "cc", "memory"); |
| 59 | 59 | } |
| 60 | 60 | |
| @@ -85,7 +85,7 @@ static void xor_xc_4(unsigned long bytes, unsigned long * __restrict p1, · 24 unmodified lines | ||
| 85 | 85 | "3: xc 0(1,%1),0(%3)\n" |
| 86 | 86 | "4: xc 0(1,%1),0(%4)\n" |
| 87 | 87 | "5:" |
| 88 | − : "+d" (bytes), "+a" (p1), "+a" (p2), "+a" (p3), "+a" (p4) | |
| 88 | + : "+a" (bytes), "+a" (p1), "+a" (p2), "+a" (p3), "+a" (p4) | |
| 89 | 89 | : : "0", "cc", "memory"); |
| 90 | 90 | } |
| 91 | 91 | |
| @@ -96,7 +96,6 @@ static void xor_xc_5(unsigned long bytes, unsigned long * __restrict p1, · 4 unmodified lines | ||
| 96 | 96 | const unsigned long * __restrict p5) |
| 97 | 97 | { |
| 98 | 98 | asm volatile( |
| 99 | − " larl 1,2f\n" | |
| 100 | 99 | " aghi %0,-1\n" |
| 101 | 100 | " jm 6f\n" |
| 102 | 101 | " srlg 0,%0,8\n" |
| @@ -122,7 +121,7 @@ static void xor_xc_5(unsigned long bytes, unsigned long * __restrict p1, · 19 unmodified lines | ||
| 122 | 121 | "4: xc 0(1,%1),0(%4)\n" |
| 123 | 122 | "5: xc 0(1,%1),0(%5)\n" |
| 124 | 123 | "6:" |
| 125 | − : "+d" (bytes), "+a" (p1), "+a" (p2), "+a" (p3), "+a" (p4), | |
| 124 | + : "+a" (bytes), "+a" (p1), "+a" (p2), "+a" (p3), "+a" (p4), | |
| 126 | 125 | "+a" (p5) |
| 127 | 126 | : : "0", "cc", "memory"); |
| 128 | 127 | } |
| @@ -62,7 +62,7 @@ int __pfault_init(void) · 61 unmodified lines | ||
| 62 | 62 | "0: nopr %%r7\n" |
| 63 | 63 | EX_TABLE(0b, 0b) |
| 64 | 64 | : [rc] "+d" (rc) |
| 65 | − : [refbk] "a" (&pfault_init_refbk), "m" (pfault_init_refbk) | |
| 65 | + : [refbk] "a" (virt_to_phys(&pfault_init_refbk)), "m" (pfault_init_refbk) | |
| 66 | 66 | : "cc"); |
| 67 | 67 | return rc; |
| 68 | 68 | } |
| @@ -84,7 +84,7 @@ void __pfault_fini(void) · 15 unmodified lines | ||
| 84 | 84 | "0: nopr %%r7\n" |
| 85 | 85 | EX_TABLE(0b, 0b) |
| 86 | 86 | : |
| 87 | − : [refbk] "a" (&pfault_fini_refbk), "m" (pfault_fini_refbk) | |
| 87 | + : [refbk] "a" (virt_to_phys(&pfault_fini_refbk)), "m" (pfault_fini_refbk) | |
| 88 | 88 | : "cc"); |
| 89 | 89 | } |
| 90 | 90 | |
| @@ -89,6 +89,7 @@ SECTIONS · 88 unmodified lines | ||
| 89 | 89 | |
| 90 | 90 | STABS_DEBUG |
| 91 | 91 | DWARF_DEBUG |
| 92 | + MODINFO | |
| 92 | 93 | ELF_DETAILS |
| 93 | 94 | |
| 94 | 95 | DISCARDS |
| @@ -312,6 +312,8 @@ static dma_addr_t dma_4u_map_phys(struct device *dev, phys_addr_t phys, · 311 unmodified lines | ||
| 312 | 312 | if (direction != DMA_TO_DEVICE) |
| 313 | 313 | iopte_protection |= IOPTE_WRITE; |
| 314 | 314 | |
| 315 | + phys &= IO_PAGE_MASK; | |
| 316 | + | |
| 315 | 317 | for (i = 0; i < npages; i++, base++, phys += IO_PAGE_SIZE) |
| 316 | 318 | iopte_val(*base) = iopte_protection | phys; |
| 317 | 319 | |
| @@ -355,6 +355,13 @@ static struct pci_dev *of_create_pci_dev(struct pci_pbm_info *pbm, · 354 unmodified lines | ||
| 355 | 355 | dev->error_state = pci_channel_io_normal; |
| 356 | 356 | dev->dma_mask = 0xffffffff; |
| 357 | 357 | |
| 358 | + /* | |
| 359 | + * Assume 64-bit addresses for MSI initially. Will be changed to 32-bit | |
| 360 | + * if MSI (rather than MSI-X) capability does not have | |
| 361 | + * PCI_MSI_FLAGS_64BIT. Can also be overridden by driver. | |
| 362 | + */ | |
| 363 | + dev->msi_addr_mask = DMA_BIT_MASK(64); | |
| 364 | + | |
| 358 | 365 | if (of_node_name_eq(node, "pci")) { |
| 359 | 366 | /* a PCI-PCI bridge */ |
| 360 | 367 | dev->hdr_type = PCI_HEADER_TYPE_BRIDGE; |
| @@ -410,6 +410,8 @@ static dma_addr_t dma_4v_map_phys(struct device *dev, phys_addr_t phys, · 409 unmodified lines | ||
| 410 | 410 | |
| 411 | 411 | iommu_batch_start(dev, prot, entry); |
| 412 | 412 | |
| 413 | + phys &= IO_PAGE_MASK; | |
| 414 | + | |
| 413 | 415 | for (i = 0; i < npages; i++, phys += IO_PAGE_SIZE) { |
| 414 | 416 | long err = iommu_batch_add(phys, mask); |
| 415 | 417 | if (unlikely(err < 0L)) |
| @@ -191,6 +191,7 @@ SECTIONS · 190 unmodified lines | ||
| 191 | 191 | |
| 192 | 192 | STABS_DEBUG |
| 193 | 193 | DWARF_DEBUG |
| 194 | + MODINFO | |
| 194 | 195 | ELF_DETAILS |
| 195 | 196 | |
| 196 | 197 | DISCARDS |
| @@ -69,11 +69,11 @@ struct io_thread_req { · 68 unmodified lines | ||
| 69 | 69 | }; |
| 70 | 70 | |
| 71 | 71 | |
| 72 | −static struct io_thread_req * (*irq_req_buffer)[]; | |
| 72 | +static struct io_thread_req **irq_req_buffer; | |
| 73 | 73 | static struct io_thread_req *irq_remainder; |
| 74 | 74 | static int irq_remainder_size; |
| 75 | 75 | |
| 76 | −static struct io_thread_req * (*io_req_buffer)[]; | |
| 76 | +static struct io_thread_req **io_req_buffer; | |
| 77 | 77 | static struct io_thread_req *io_remainder; |
| 78 | 78 | static int io_remainder_size; |
| 79 | 79 | |
| @@ -398,7 +398,7 @@ static int thread_fd = -1; · 318 unmodified lines | ||
| 398 | 398 | |
| 399 | 399 | static int bulk_req_safe_read( |
| 400 | 400 | int fd, |
| 401 | − struct io_thread_req * (*request_buffer)[], | |
| 401 | + struct io_thread_req **request_buffer, | |
| 402 | 402 | struct io_thread_req **remainder, |
| 403 | 403 | int *remainder_size, |
| 404 | 404 | int max_recs |
| @@ -465,7 +465,7 @@ static irqreturn_t ubd_intr(int irq, void *dev) · 60 unmodified lines | ||
| 465 | 465 | &irq_remainder, &irq_remainder_size, |
| 466 | 466 | UBD_REQ_BUFFER_SIZE)) >= 0) { |
| 467 | 467 | for (i = 0; i < len / sizeof(struct io_thread_req *); i++) |
| 468 | − ubd_end_request((*irq_req_buffer)[i]); | |
| 468 | + ubd_end_request(irq_req_buffer[i]); | |
| 469 | 469 | } |
| 470 | 470 | |
| 471 | 471 | if (len < 0 && len != -EAGAIN) |
| @@ -1512,7 +1512,7 @@ void *io_thread(void *arg) · 1040 unmodified lines | ||
| 1512 | 1512 | } |
| 1513 | 1513 | |
| 1514 | 1514 | for (count = 0; count < n/sizeof(struct io_thread_req *); count++) { |
| 1515 | − struct io_thread_req *req = (*io_req_buffer)[count]; | |
| 1515 | + struct io_thread_req *req = io_req_buffer[count]; | |
| 1516 | 1516 | int i; |
| 1517 | 1517 | |
| 1518 | 1518 | io_count++; |
| @@ -172,6 +172,7 @@ SECTIONS · 171 unmodified lines | ||
| 172 | 172 | |
| 173 | 173 | STABS_DEBUG |
| 174 | 174 | DWARF_DEBUG |
| 175 | + MODINFO | |
| 175 | 176 | ELF_DETAILS |
| 176 | 177 | |
| 177 | 178 | DISCARDS |
| @@ -113,6 +113,7 @@ SECTIONS · 112 unmodified lines | ||
| 113 | 113 | |
| 114 | 114 | STABS_DEBUG |
| 115 | 115 | DWARF_DEBUG |
| 116 | + MODINFO | |
| 116 | 117 | ELF_DETAILS |
| 117 | 118 | |
| 118 | 119 | DISCARDS |
| @@ -113,6 +113,7 @@ vmlinux-objs-$(CONFIG_EFI_SBAT) += $(obj)/sbat.o · 112 unmodified lines | ||
| 113 | 113 | |
| 114 | 114 | ifdef CONFIG_EFI_SBAT |
| 115 | 115 | $(obj)/sbat.o: $(CONFIG_EFI_SBAT_FILE) |
| 116 | +AFLAGS_sbat.o += -I $(srctree) | |
| 116 | 117 | endif |
| 117 | 118 | |
| 118 | 119 | $(obj)/vmlinux: $(vmlinux-objs-y) $(vmlinux-libs-y) FORCE |
| @@ -28,17 +28,17 @@ · 27 unmodified lines | ||
| 28 | 28 | #include "sev.h" |
| 29 | 29 | |
| 30 | 30 | static struct ghcb boot_ghcb_page __aligned(PAGE_SIZE); |
| 31 | −struct ghcb *boot_ghcb; | |
| 31 | +struct ghcb *boot_ghcb __section(".data"); | |
| 32 | 32 | |
| 33 | 33 | #undef __init |
| 34 | 34 | #define __init |
| 35 | 35 | |
| 36 | 36 | #define __BOOT_COMPRESSED |
| 37 | 37 | |
| 38 | −u8 snp_vmpl; | |
| 39 | −u16 ghcb_version; | |
| 38 | +u8 snp_vmpl __section(".data"); | |
| 39 | +u16 ghcb_version __section(".data"); | |
| 40 | 40 | |
| 41 | −u64 boot_svsm_caa_pa; | |
| 41 | +u64 boot_svsm_caa_pa __section(".data"); | |
| 42 | 42 | |
| 43 | 43 | /* Include code for early handlers */ |
| 44 | 44 | #include "../../boot/startup/sev-shared.c" |
| @@ -188,6 +188,7 @@ bool sev_es_check_ghcb_fault(unsigned long address) · 143 unmodified lines | ||
| 188 | 188 | MSR_AMD64_SNP_RESERVED_BIT13 | \ |
| 189 | 189 | MSR_AMD64_SNP_RESERVED_BIT15 | \ |
| 190 | 190 | MSR_AMD64_SNP_SECURE_AVIC | \ |
| 191 | + MSR_AMD64_SNP_RESERVED_BITS19_22 | \ | |
| 191 | 192 | MSR_AMD64_SNP_RESERVED_MASK) |
| 192 | 193 | |
| 193 | 194 | #ifdef CONFIG_AMD_SECURE_AVIC |
| @@ -88,7 +88,7 @@ SECTIONS · 87 unmodified lines | ||
| 88 | 88 | /DISCARD/ : { |
| 89 | 89 | *(.dynamic) *(.dynsym) *(.dynstr) *(.dynbss) |
| 90 | 90 | *(.hash) *(.gnu.hash) |
| 91 | − *(.note.*) | |
| 91 | + *(.note.*) *(.modinfo) | |
| 92 | 92 | } |
| 93 | 93 | |
| 94 | 94 | .got.plt (INFO) : { |
| @@ -31,7 +31,7 @@ static u32 cpuid_std_range_max __ro_after_init; · 30 unmodified lines | ||
| 31 | 31 | static u32 cpuid_hyp_range_max __ro_after_init; |
| 32 | 32 | static u32 cpuid_ext_range_max __ro_after_init; |
| 33 | 33 | |
| 34 | −bool sev_snp_needs_sfw; | |
| 34 | +bool sev_snp_needs_sfw __section(".data"); | |
| 35 | 35 | |
| 36 | 36 | void __noreturn |
| 37 | 37 | sev_es_terminate(unsigned int set, unsigned int reason) |
| @@ -89,6 +89,7 @@ static const char * const sev_status_feat_names[] = { · 88 unmodified lines | ||
| 89 | 89 | [MSR_AMD64_SNP_VMSA_REG_PROT_BIT] = "VMSARegProt", |
| 90 | 90 | [MSR_AMD64_SNP_SMT_PROT_BIT] = "SMTProt", |
| 91 | 91 | [MSR_AMD64_SNP_SECURE_AVIC_BIT] = "SecureAVIC", |
| 92 | + [MSR_AMD64_SNP_IBPB_ON_ENTRY_BIT] = "IBPBOnEntry", | |
| 92 | 93 | }; |
| 93 | 94 | |
| 94 | 95 | /* |
| @@ -160,8 +160,6 @@ void __init fred_complete_exception_setup(void) · 159 unmodified lines | ||
| 160 | 160 | static noinstr void fred_extint(struct pt_regs *regs) |
| 161 | 161 | { |
| 162 | 162 | unsigned int vector = regs->fred_ss.vector; |
| 163 | − unsigned int index = array_index_nospec(vector - FIRST_SYSTEM_VECTOR, | |
| 164 | − NR_SYSTEM_VECTORS); | |
| 165 | 163 | |
| 166 | 164 | if (WARN_ON_ONCE(vector < FIRST_EXTERNAL_VECTOR)) |
| 167 | 165 | return; |
| @@ -170,7 +168,8 @@ static noinstr void fred_extint(struct pt_regs *regs) · 2 unmodified lines | ||
| 170 | 168 | irqentry_state_t state = irqentry_enter(regs); |
| 171 | 169 | |
| 172 | 170 | instrumentation_begin(); |
| 173 | − sysvec_table[index](regs); | |
| 171 | + sysvec_table[array_index_nospec(vector - FIRST_SYSTEM_VECTOR, | |
| 172 | + NR_SYSTEM_VECTORS)](regs); | |
| 174 | 173 | instrumentation_end(); |
| 175 | 174 | irqentry_exit(regs, state); |
| 176 | 175 | } else { |
| @@ -35,9 +35,38 @@ · 34 unmodified lines | ||
| 35 | 35 | #endif |
| 36 | 36 | .endm |
| 37 | 37 | |
| 38 | +/* | |
| 39 | + * WARNING: | |
| 40 | + * | |
| 41 | + * A bug in the libgcc unwinder as of at least gcc 15.2 (2026) means that | |
| 42 | + * the unwinder fails to recognize the signal frame flag. | |
| 43 | + * | |
| 44 | + * There is a hacky legacy fallback path in libgcc which ends up | |
| 45 | + * getting invoked instead. It happens to work as long as BOTH of the | |
| 46 | + * following conditions are true: | |
| 47 | + * | |
| 48 | + * 1. There is at least one byte before the each of the sigreturn | |
| 49 | + * functions which falls outside any function. This is enforced by | |
| 50 | + * an explicit nop instruction before the ALIGN. | |
| 51 | + * 2. The code sequences between the entry point up to and including | |
| 52 | + * the int $0x80 below need to match EXACTLY. Do not change them | |
| 53 | + * in any way. The exact byte sequences are: | |
| 54 | + * | |
| 55 | + * __kernel_sigreturn: | |
| 56 | + * 0: 58 pop %eax | |
| 57 | + * 1: b8 77 00 00 00 mov $0x77,%eax | |
| 58 | + * 6: cd 80 int $0x80 | |
| 59 | + * | |
| 60 | + * __kernel_rt_sigreturn: | |
| 61 | + * 0: b8 ad 00 00 00 mov $0xad,%eax | |
| 62 | + * 5: cd 80 int $0x80 | |
| 63 | + * | |
| 64 | + * For details, see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=124050 | |
| 65 | + */ | |
| 38 | 66 | .text |
| 39 | 67 | .globl __kernel_sigreturn |
| 40 | 68 | .type __kernel_sigreturn,@function |
| 69 | + nop /* libgcc hack: see comment above */ | |
| 41 | 70 | ALIGN |
| 42 | 71 | __kernel_sigreturn: |
| 43 | 72 | STARTPROC_SIGNAL_FRAME IA32_SIGFRAME_sigcontext |
| @@ -52,6 +81,7 @@ SYM_INNER_LABEL(vdso32_sigreturn_landing_pad, SYM_L_GLOBAL) · 8 unmodified lines | ||
| 52 | 81 | |
| 53 | 82 | .globl __kernel_rt_sigreturn |
| 54 | 83 | .type __kernel_rt_sigreturn,@function |
| 84 | + nop /* libgcc hack: see comment above */ | |
| 55 | 85 | ALIGN |
| 56 | 86 | __kernel_rt_sigreturn: |
| 57 | 87 | STARTPROC_SIGNAL_FRAME IA32_RT_SIGFRAME_sigcontext |
| @@ -6497,6 +6497,32 @@ static struct intel_uncore_type gnr_uncore_ubox = { · 6496 unmodified lines | ||
| 6497 | 6497 | .attr_update = uncore_alias_groups, |
| 6498 | 6498 | }; |
| 6499 | 6499 | |
| 6500 | +static struct uncore_event_desc gnr_uncore_imc_events[] = { | |
| 6501 | + INTEL_UNCORE_EVENT_DESC(clockticks, "event=0x01,umask=0x00"), | |
| 6502 | + INTEL_UNCORE_EVENT_DESC(cas_count_read_sch0, "event=0x05,umask=0xcf"), | |
| 6503 | + INTEL_UNCORE_EVENT_DESC(cas_count_read_sch0.scale, "6.103515625e-5"), | |
| 6504 | + INTEL_UNCORE_EVENT_DESC(cas_count_read_sch0.unit, "MiB"), | |
| 6505 | + INTEL_UNCORE_EVENT_DESC(cas_count_read_sch1, "event=0x06,umask=0xcf"), | |
| 6506 | + INTEL_UNCORE_EVENT_DESC(cas_count_read_sch1.scale, "6.103515625e-5"), | |
| 6507 | + INTEL_UNCORE_EVENT_DESC(cas_count_read_sch1.unit, "MiB"), | |
| 6508 | + INTEL_UNCORE_EVENT_DESC(cas_count_write_sch0, "event=0x05,umask=0xf0"), | |
| 6509 | + INTEL_UNCORE_EVENT_DESC(cas_count_write_sch0.scale, "6.103515625e-5"), | |
| 6510 | + INTEL_UNCORE_EVENT_DESC(cas_count_write_sch0.unit, "MiB"), | |
| 6511 | + INTEL_UNCORE_EVENT_DESC(cas_count_write_sch1, "event=0x06,umask=0xf0"), | |
| 6512 | + INTEL_UNCORE_EVENT_DESC(cas_count_write_sch1.scale, "6.103515625e-5"), | |
| 6513 | + INTEL_UNCORE_EVENT_DESC(cas_count_write_sch1.unit, "MiB"), | |
| 6514 | + { /* end: all zeroes */ }, | |
| 6515 | +}; | |
| 6516 | + | |
| 6517 | +static struct intel_uncore_type gnr_uncore_imc = { | |
| 6518 | + SPR_UNCORE_MMIO_COMMON_FORMAT(), | |
| 6519 | + .name = "imc", | |
| 6520 | + .fixed_ctr_bits = 48, | |
| 6521 | + .fixed_ctr = SNR_IMC_MMIO_PMON_FIXED_CTR, | |
| 6522 | + .fixed_ctl = SNR_IMC_MMIO_PMON_FIXED_CTL, | |
| 6523 | + .event_descs = gnr_uncore_imc_events, | |
| 6524 | +}; | |
| 6525 | + | |
| 6500 | 6526 | static struct intel_uncore_type gnr_uncore_pciex8 = { |
| 6501 | 6527 | SPR_UNCORE_PCI_COMMON_FORMAT(), |
| 6502 | 6528 | .name = "pciex8", |
| @@ -6544,7 +6570,7 @@ static struct intel_uncore_type *gnr_uncores[UNCORE_GNR_NUM_UNCORE_TYPES] = { · 41 unmodified lines | ||
| 6544 | 6570 | NULL, |
| 6545 | 6571 | &spr_uncore_pcu, |
| 6546 | 6572 | &gnr_uncore_ubox, |
| 6547 | − &spr_uncore_imc, | |
| 6573 | + &gnr_uncore_imc, | |
| 6548 | 6574 | NULL, |
| 6549 | 6575 | &gnr_uncore_upi, |
| 6550 | 6576 | NULL, |
| @@ -7,7 +7,7 @@ · 6 unmodified lines | ||
| 7 | 7 | #include <linux/objtool.h> |
| 8 | 8 | #include <asm/asm.h> |
| 9 | 9 | |
| 10 | −#ifndef __ASSEMBLY__ | |
| 10 | +#ifndef __ASSEMBLER__ | |
| 11 | 11 | struct bug_entry; |
| 12 | 12 | extern void __WARN_trap(struct bug_entry *bug, ...); |
| 13 | 13 | #endif |
| @@ -137,7 +137,7 @@ do { \ · 123 unmodified lines | ||
| 137 | 137 | |
| 138 | 138 | #ifdef HAVE_ARCH_BUG_FORMAT_ARGS |
| 139 | 139 | |
| 140 | −#ifndef __ASSEMBLY__ | |
| 140 | +#ifndef __ASSEMBLER__ | |
| 141 | 141 | #include <linux/static_call_types.h> |
| 142 | 142 | DECLARE_STATIC_CALL(WARN_trap, __WARN_trap); |
| 143 | 143 | |
| @@ -153,7 +153,7 @@ struct arch_va_list { · 9 unmodified lines | ||
| 153 | 153 | struct sysv_va_list args; |
| 154 | 154 | }; |
| 155 | 155 | extern void *__warn_args(struct arch_va_list *args, struct pt_regs *regs); |
| 156 | −#endif /* __ASSEMBLY__ */ | |
| 156 | +#endif /* __ASSEMBLER__ */ | |
| 157 | 157 | |
| 158 | 158 | #define __WARN_bug_entry(flags, format) ({ \ |
| 159 | 159 | struct bug_entry *bug; \ |
| @@ -111,6 +111,12 @@ extern bhi_thunk __bhi_args_end[]; · 110 unmodified lines | ||
| 111 | 111 | |
| 112 | 112 | struct pt_regs; |
| 113 | 113 | |
| 114 | +#ifdef CONFIG_CALL_PADDING | |
| 115 | +#define CFI_OFFSET (CONFIG_FUNCTION_PADDING_CFI+5) | |
| 116 | +#else | |
| 117 | +#define CFI_OFFSET 5 | |
| 118 | +#endif | |
| 119 | + | |
| 114 | 120 | #ifdef CONFIG_CFI |
| 115 | 121 | enum bug_trap_type handle_cfi_failure(struct pt_regs *regs); |
| 116 | 122 | #define __bpfcall |
| @@ -119,11 +125,9 @@ static inline int cfi_get_offset(void) · 2 unmodified lines | ||
| 119 | 125 | { |
| 120 | 126 | switch (cfi_mode) { |
| 121 | 127 | case CFI_FINEIBT: |
| 122 | − return 16; | |
| 128 | + return /* fineibt_prefix_size */ 16; | |
| 123 | 129 | case CFI_KCFI: |
| 124 | − if (IS_ENABLED(CONFIG_CALL_PADDING)) | |
| 125 | − return 16; | |
| 126 | − return 5; | |
| 130 | + return CFI_OFFSET; | |
| 127 | 131 | default: |
| 128 | 132 | return 0; |
| 129 | 133 | } |
| @@ -138,7 +138,7 @@ extern void __init efi_apply_memmap_quirks(void); · 137 unmodified lines | ||
| 138 | 138 | extern int __init efi_reuse_config(u64 tables, int nr_tables); |
| 139 | 139 | extern void efi_delete_dummy_variable(void); |
| 140 | 140 | extern void efi_crash_gracefully_on_page_fault(unsigned long phys_addr); |
| 141 | −extern void efi_free_boot_services(void); | |
| 141 | +extern void efi_unmap_boot_services(void); | |
| 142 | 142 | |
| 143 | 143 | void arch_efi_call_virt_setup(void); |
| 144 | 144 | void arch_efi_call_virt_teardown(void); |
| @@ -77,7 +77,7 @@ static __always_inline void native_local_irq_restore(unsigned long flags) · 76 unmodified lines | ||
| 77 | 77 | #endif |
| 78 | 78 | |
| 79 | 79 | #ifndef CONFIG_PARAVIRT |
| 80 | −#ifndef __ASSEMBLY__ | |
| 80 | +#ifndef __ASSEMBLER__ | |
| 81 | 81 | /* |
| 82 | 82 | * Used in the idle loop; sti takes one instruction cycle |
| 83 | 83 | * to complete: |
| @@ -95,7 +95,7 @@ static __always_inline void halt(void) · 11 unmodified lines | ||
| 95 | 95 | { |
| 96 | 96 | native_halt(); |
| 97 | 97 | } |
| 98 | −#endif /* __ASSEMBLY__ */ | |
| 98 | +#endif /* __ASSEMBLER__ */ | |
| 99 | 99 | #endif /* CONFIG_PARAVIRT */ |
| 100 | 100 | |
| 101 | 101 | #ifdef CONFIG_PARAVIRT_XXL |
| @@ -2485,7 +2485,8 @@ int memslot_rmap_alloc(struct kvm_memory_slot *slot, unsigned long npages); · 2484 unmodified lines | ||
| 2485 | 2485 | KVM_X86_QUIRK_MWAIT_NEVER_UD_FAULTS | \ |
| 2486 | 2486 | KVM_X86_QUIRK_SLOT_ZAP_ALL | \ |
| 2487 | 2487 | KVM_X86_QUIRK_STUFF_FEATURE_MSRS | \ |
| 2488 | − KVM_X86_QUIRK_IGNORE_GUEST_PAT) | |
| 2488 | + KVM_X86_QUIRK_IGNORE_GUEST_PAT | \ | |
| 2489 | + KVM_X86_QUIRK_VMCS12_ALLOW_FREEZE_IN_SMM) | |
| 2489 | 2490 | |
| 2490 | 2491 | #define KVM_X86_CONDITIONAL_QUIRKS \ |
| 2491 | 2492 | (KVM_X86_QUIRK_CD_NW_CLEARED | \ |
| @@ -68,7 +68,7 @@ · 67 unmodified lines | ||
| 68 | 68 | * Depending on -fpatchable-function-entry=N,N usage (CONFIG_CALL_PADDING) the |
| 69 | 69 | * CFI symbol layout changes. |
| 70 | 70 | * |
| 71 | − * Without CALL_THUNKS: | |
| 71 | + * Without CALL_PADDING: | |
| 72 | 72 | * |
| 73 | 73 | * .align FUNCTION_ALIGNMENT |
| 74 | 74 | * __cfi_##name: |
| @@ -77,7 +77,7 @@ · 2 unmodified lines | ||
| 77 | 77 | * .long __kcfi_typeid_##name |
| 78 | 78 | * name: |
| 79 | 79 | * |
| 80 | − * With CALL_THUNKS: | |
| 80 | + * With CALL_PADDING: | |
| 81 | 81 | * |
| 82 | 82 | * .align FUNCTION_ALIGNMENT |
| 83 | 83 | * __cfi_##name: |
| @@ -740,7 +740,10 @@ · 739 unmodified lines | ||
| 740 | 740 | #define MSR_AMD64_SNP_SMT_PROT BIT_ULL(MSR_AMD64_SNP_SMT_PROT_BIT) |
| 741 | 741 | #define MSR_AMD64_SNP_SECURE_AVIC_BIT 18 |
| 742 | 742 | #define MSR_AMD64_SNP_SECURE_AVIC BIT_ULL(MSR_AMD64_SNP_SECURE_AVIC_BIT) |
| 743 | −#define MSR_AMD64_SNP_RESV_BIT 19 | |
| 743 | +#define MSR_AMD64_SNP_RESERVED_BITS19_22 GENMASK_ULL(22, 19) | |
| 744 | +#define MSR_AMD64_SNP_IBPB_ON_ENTRY_BIT 23 | |
| 745 | +#define MSR_AMD64_SNP_IBPB_ON_ENTRY BIT_ULL(MSR_AMD64_SNP_IBPB_ON_ENTRY_BIT) | |
| 746 | +#define MSR_AMD64_SNP_RESV_BIT 24 | |
| 744 | 747 | #define MSR_AMD64_SNP_RESERVED_MASK GENMASK_ULL(63, MSR_AMD64_SNP_RESV_BIT) |
| 745 | 748 | #define MSR_AMD64_SAVIC_CONTROL 0xc0010138 |
| 746 | 749 | #define MSR_AMD64_SAVIC_EN_BIT 0 |
| @@ -22,6 +22,7 @@ extern int numa_off; · 21 unmodified lines | ||
| 22 | 22 | */ |
| 23 | 23 | extern s16 __apicid_to_node[MAX_LOCAL_APIC]; |
| 24 | 24 | extern nodemask_t numa_nodes_parsed __initdata; |
| 25 | +extern nodemask_t numa_phys_nodes_parsed __initdata; | |
| 25 | 26 | |
| 26 | 27 | static inline void set_apicid_to_node(int apicid, s16 node) |
| 27 | 28 | { |
| @@ -48,6 +49,7 @@ extern void __init init_cpu_to_node(void); · 20 unmodified lines | ||
| 48 | 49 | extern void numa_add_cpu(unsigned int cpu); |
| 49 | 50 | extern void numa_remove_cpu(unsigned int cpu); |
| 50 | 51 | extern void init_gi_nodes(void); |
| 52 | +extern int num_phys_nodes(void); | |
| 51 | 53 | #else /* CONFIG_NUMA */ |
| 52 | 54 | static inline void numa_set_node(int cpu, int node) { } |
| 53 | 55 | static inline void numa_clear_node(int cpu) { } |
| @@ -55,6 +57,10 @@ static inline void init_cpu_to_node(void) { } · 1 unmodified line | ||
| 55 | 57 | static inline void numa_add_cpu(unsigned int cpu) { } |
| 56 | 58 | static inline void numa_remove_cpu(unsigned int cpu) { } |
| 57 | 59 | static inline void init_gi_nodes(void) { } |
| 60 | +static inline int num_phys_nodes(void) | |
| 61 | +{ | |
| 62 | + return 1; | |
| 63 | +} | |
| 58 | 64 | #endif /* CONFIG_NUMA */ |
| 59 | 65 | |
| 60 | 66 | #ifdef CONFIG_DEBUG_PER_CPU_MAPS |
| @@ -20,7 +20,7 @@ · 19 unmodified lines | ||
| 20 | 20 | |
| 21 | 21 | #define PER_CPU_VAR(var) __percpu(var)__percpu_rel |
| 22 | 22 | |
| 23 | −#else /* !__ASSEMBLY__: */ | |
| 23 | +#else /* !__ASSEMBLER__: */ | |
| 24 | 24 | |
| 25 | 25 | #include <linux/args.h> |
| 26 | 26 | #include <linux/bits.h> |
| @@ -19,10 +19,8 @@ · 18 unmodified lines | ||
| 19 | 19 | extern p4d_t level4_kernel_pgt[512]; |
| 20 | 20 | extern p4d_t level4_ident_pgt[512]; |
| 21 | 21 | extern pud_t level3_kernel_pgt[512]; |
| 22 | −extern pud_t level3_ident_pgt[512]; | |
| 23 | 22 | extern pmd_t level2_kernel_pgt[512]; |
| 24 | 23 | extern pmd_t level2_fixmap_pgt[512]; |
| 25 | −extern pmd_t level2_ident_pgt[512]; | |
| 26 | 24 | extern pte_t level1_fixmap_pgt[512 * FIXMAP_PMD_NUM]; |
| 27 | 25 | extern pgd_t init_top_pgt[]; |
| 28 | 26 | |
| @@ -6,7 +6,7 @@ · 5 unmodified lines | ||
| 6 | 6 | #error "Cannot use runtime-const infrastructure from modules" |
| 7 | 7 | #endif |
| 8 | 8 | |
| 9 | −#ifdef __ASSEMBLY__ | |
| 9 | +#ifdef __ASSEMBLER__ | |
| 10 | 10 | |
| 11 | 11 | .macro RUNTIME_CONST_PTR sym reg |
| 12 | 12 | movq $0x0123456789abcdef, %\reg |
| @@ -16,7 +16,7 @@ · 3 unmodified lines | ||
| 16 | 16 | .popsection |
| 17 | 17 | .endm |
| 18 | 18 | |
| 19 | −#else /* __ASSEMBLY__ */ | |
| 19 | +#else /* __ASSEMBLER__ */ | |
| 20 | 20 | |
| 21 | 21 | #define runtime_const_ptr(sym) ({ \ |
| 22 | 22 | typeof(sym) __ret; \ |
| @@ -74,5 +74,5 @@ static inline void runtime_const_fixup(void (*fn)(void *, unsigned long), · 51 unmodified lines | ||
| 74 | 74 | } |
| 75 | 75 | } |
| 76 | 76 | |
| 77 | −#endif /* __ASSEMBLY__ */ | |
| 77 | +#endif /* __ASSEMBLER__ */ | |
| 78 | 78 | #endif |
| @@ -155,6 +155,7 @@ extern unsigned int __max_logical_packages; · 154 unmodified lines | ||
| 155 | 155 | extern unsigned int __max_threads_per_core; |
| 156 | 156 | extern unsigned int __num_threads_per_package; |
| 157 | 157 | extern unsigned int __num_cores_per_package; |
| 158 | +extern unsigned int __num_nodes_per_package; | |
| 158 | 159 | |
| 159 | 160 | const char *get_topology_cpu_type_name(struct cpuinfo_x86 *c); |
| 160 | 161 | enum x86_topology_cpu_type get_topology_cpu_type(struct cpuinfo_x86 *c); |
| @@ -179,6 +180,11 @@ static inline unsigned int topology_num_threads_per_package(void) · 18 unmodified lines | ||
| 179 | 180 | return __num_threads_per_package; |
| 180 | 181 | } |
| 181 | 182 | |
| 183 | +static inline unsigned int topology_num_nodes_per_package(void) | |
| 184 | +{ | |
| 185 | + return __num_nodes_per_package; | |
| 186 | +} | |
| 187 | + | |
| 182 | 188 | #ifdef CONFIG_X86_LOCAL_APIC |
| 183 | 189 | int topology_get_logical_id(u32 apicid, enum x86_topology_domains at_level); |
| 184 | 190 | #else |
| @@ -25,6 +25,8 @@ extern int ibt_selftest_noendbr(void); · 24 unmodified lines | ||
| 25 | 25 | void handle_invalid_op(struct pt_regs *regs); |
| 26 | 26 | #endif |
| 27 | 27 | |
| 28 | +noinstr bool handle_bug(struct pt_regs *regs); | |
| 29 | + | |
| 28 | 30 | static inline int get_si_code(unsigned long condition) |
| 29 | 31 | { |
| 30 | 32 | if (condition & DR_STEP) |
| @@ -476,6 +476,7 @@ struct kvm_sync_regs { · 475 unmodified lines | ||
| 476 | 476 | #define KVM_X86_QUIRK_SLOT_ZAP_ALL (1 << 7) |
| 477 | 477 | #define KVM_X86_QUIRK_STUFF_FEATURE_MSRS (1 << 8) |
| 478 | 478 | #define KVM_X86_QUIRK_IGNORE_GUEST_PAT (1 << 9) |
| 479 | +#define KVM_X86_QUIRK_VMCS12_ALLOW_FREEZE_IN_SMM (1 << 10) | |
| 479 | 480 | |
| 480 | 481 | #define KVM_STATE_NESTED_FORMAT_VMX 0 |
| 481 | 482 | #define KVM_STATE_NESTED_FORMAT_SVM 1 |
| @@ -1182,7 +1182,7 @@ void __init_or_module noinline apply_seal_endbr(s32 *start, s32 *end) · 1181 unmodified lines | ||
| 1182 | 1182 | |
| 1183 | 1183 | poison_endbr(addr); |
| 1184 | 1184 | if (IS_ENABLED(CONFIG_FINEIBT)) |
| 1185 | − poison_cfi(addr - 16); | |
| 1185 | + poison_cfi(addr - CFI_OFFSET); | |
| 1186 | 1186 | } |
| 1187 | 1187 | } |
| 1188 | 1188 | |
| @@ -1389,6 +1389,8 @@ extern u8 fineibt_preamble_end[]; · 200 unmodified lines | ||
| 1389 | 1389 | #define fineibt_preamble_ud 0x13 |
| 1390 | 1390 | #define fineibt_preamble_hash 5 |
| 1391 | 1391 | |
| 1392 | +#define fineibt_prefix_size (fineibt_preamble_size - ENDBR_INSN_SIZE) | |
| 1393 | + | |
| 1392 | 1394 | /* |
| 1393 | 1395 | * <fineibt_caller_start>: |
| 1394 | 1396 | * 0: b8 78 56 34 12 mov $0x12345678, %eax |
| @@ -1634,7 +1636,7 @@ static int cfi_rewrite_preamble(s32 *start, s32 *end) · 239 unmodified lines | ||
| 1634 | 1636 | * have determined there are no indirect calls to it and we |
| 1635 | 1637 | * don't need no CFI either. |
| 1636 | 1638 | */ |
| 1637 | − if (!is_endbr(addr + 16)) | |
| 1639 | + if (!is_endbr(addr + CFI_OFFSET)) | |
| 1638 | 1640 | continue; |
| 1639 | 1641 | |
| 1640 | 1642 | hash = decode_preamble_hash(addr, &arity); |
| @@ -1642,6 +1644,15 @@ static int cfi_rewrite_preamble(s32 *start, s32 *end) · 1 unmodified line | ||
| 1642 | 1644 | addr, addr, 5, addr)) |
| 1643 | 1645 | return -EINVAL; |
| 1644 | 1646 | |
| 1647 | + /* | |
| 1648 | + * FineIBT relies on being at func-16, so if the preamble is | |
| 1649 | + * actually larger than that, place it the tail end. | |
| 1650 | + * | |
| 1651 | + * NOTE: this is possible with things like DEBUG_CALL_THUNKS | |
| 1652 | + * and DEBUG_FORCE_FUNCTION_ALIGN_64B. | |
| 1653 | + */ | |
| 1654 | + addr += CFI_OFFSET - fineibt_prefix_size; | |
| 1655 | + | |
| 1645 | 1656 | text_poke_early(addr, fineibt_preamble_start, fineibt_preamble_size); |
| 1646 | 1657 | WARN_ON(*(u32 *)(addr + fineibt_preamble_hash) != 0x12345678); |
| 1647 | 1658 | text_poke_early(addr + fineibt_preamble_hash, &hash, 4); |
| @@ -1664,10 +1675,10 @@ static void cfi_rewrite_endbr(s32 *start, s32 *end) · 16 unmodified lines | ||
| 1664 | 1675 | for (s = start; s < end; s++) { |
| 1665 | 1676 | void *addr = (void *)s + *s; |
| 1666 | 1677 | |
| 1667 | − if (!exact_endbr(addr + 16)) | |
| 1678 | + if (!exact_endbr(addr + CFI_OFFSET)) | |
| 1668 | 1679 | continue; |
| 1669 | 1680 | |
| 1670 | − poison_endbr(addr + 16); | |
| 1681 | + poison_endbr(addr + CFI_OFFSET); | |
| 1671 | 1682 | } |
| 1672 | 1683 | } |
| 1673 | 1684 | |
| @@ -1772,7 +1783,8 @@ static void __apply_fineibt(s32 *start_retpoline, s32 *end_retpoline, · 98 unmodified lines | ||
| 1772 | 1783 | if (FINEIBT_WARN(fineibt_preamble_size, 20) || |
| 1773 | 1784 | FINEIBT_WARN(fineibt_preamble_bhi + fineibt_bhi1_size, 20) || |
| 1774 | 1785 | FINEIBT_WARN(fineibt_caller_size, 14) || |
| 1775 | − FINEIBT_WARN(fineibt_paranoid_size, 20)) | |
| 1786 | + FINEIBT_WARN(fineibt_paranoid_size, 20) || | |
| 1787 | + WARN_ON_ONCE(CFI_OFFSET < fineibt_prefix_size)) | |
| 1776 | 1788 | return; |
| 1777 | 1789 | |
| 1778 | 1790 | if (cfi_mode == CFI_AUTO) { |
| @@ -1885,6 +1897,11 @@ static void poison_cfi(void *addr) · 106 unmodified lines | ||
| 1885 | 1897 | */ |
| 1886 | 1898 | switch (cfi_mode) { |
| 1887 | 1899 | case CFI_FINEIBT: |
| 1900 | + /* | |
| 1901 | + * FineIBT preamble is at func-16. | |
| 1902 | + */ | |
| 1903 | + addr += CFI_OFFSET - fineibt_prefix_size; | |
| 1904 | + | |
| 1888 | 1905 | /* |
| 1889 | 1906 | * FineIBT prefix should start with an ENDBR. |
| 1890 | 1907 | */ |
| @@ -1923,8 +1940,6 @@ static void poison_cfi(void *addr) · 32 unmodified lines | ||
| 1923 | 1940 | } |
| 1924 | 1941 | } |
| 1925 | 1942 | |
| 1926 | −#define fineibt_prefix_size (fineibt_preamble_size - ENDBR_INSN_SIZE) | |
| 1927 | − | |
| 1928 | 1943 | /* |
| 1929 | 1944 | * When regs->ip points to a 0xD6 byte in the FineIBT preamble, |
| 1930 | 1945 | * return true and fill out target and type. |
| @@ -1894,6 +1894,7 @@ void __init check_x2apic(void) · 1893 unmodified lines | ||
| 1894 | 1894 | |
| 1895 | 1895 | static inline void try_to_enable_x2apic(int remap_mode) { } |
| 1896 | 1896 | static inline void __x2apic_enable(void) { } |
| 1897 | +static inline void __x2apic_disable(void) { } | |
| 1897 | 1898 | #endif /* !CONFIG_X86_X2APIC */ |
| 1898 | 1899 | |
| 1899 | 1900 | void __init enable_IR_x2apic(void) |
| @@ -2456,6 +2457,11 @@ static void lapic_resume(void *data) · 556 unmodified lines | ||
| 2456 | 2457 | if (x2apic_mode) { |
| 2457 | 2458 | __x2apic_enable(); |
| 2458 | 2459 | } else { |
| 2460 | + if (x2apic_enabled()) { | |
| 2461 | + pr_warn_once("x2apic: re-enabled by firmware during resume. Disabling\n"); | |
| 2462 | + __x2apic_disable(); | |
| 2463 | + } | |
| 2464 | + | |
| 2459 | 2465 | /* |
| 2460 | 2466 | * Make sure the APICBASE points to the right address |
| 2461 | 2467 | * |
| @@ -95,6 +95,9 @@ EXPORT_SYMBOL(__max_dies_per_package); · 94 unmodified lines | ||
| 95 | 95 | unsigned int __max_logical_packages __ro_after_init = 1; |
| 96 | 96 | EXPORT_SYMBOL(__max_logical_packages); |
| 97 | 97 | |
| 98 | +unsigned int __num_nodes_per_package __ro_after_init = 1; | |
| 99 | +EXPORT_SYMBOL(__num_nodes_per_package); | |
| 100 | + | |
| 98 | 101 | unsigned int __num_cores_per_package __ro_after_init = 1; |
| 99 | 102 | EXPORT_SYMBOL(__num_cores_per_package); |
| 100 | 103 | |
| @@ -364,7 +364,7 @@ void arch_mon_domain_online(struct rdt_resource *r, struct rdt_l3_mon_domain *d) · 363 unmodified lines | ||
| 364 | 364 | msr_clear_bit(MSR_RMID_SNC_CONFIG, 0); |
| 365 | 365 | } |
| 366 | 366 | |
| 367 | −/* CPU models that support MSR_RMID_SNC_CONFIG */ | |
| 367 | +/* CPU models that support SNC and MSR_RMID_SNC_CONFIG */ | |
| 368 | 368 | static const struct x86_cpu_id snc_cpu_ids[] __initconst = { |
| 369 | 369 | X86_MATCH_VFM(INTEL_ICELAKE_X, 0), |
| 370 | 370 | X86_MATCH_VFM(INTEL_SAPPHIRERAPIDS_X, 0), |
| @@ -375,40 +375,14 @@ static const struct x86_cpu_id snc_cpu_ids[] __initconst = { · 4 unmodified lines | ||
| 375 | 375 | {} |
| 376 | 376 | }; |
| 377 | 377 | |
| 378 | −/* | |
| 379 | − * There isn't a simple hardware bit that indicates whether a CPU is running | |
| 380 | − * in Sub-NUMA Cluster (SNC) mode. Infer the state by comparing the | |
| 381 | − * number of CPUs sharing the L3 cache with CPU0 to the number of CPUs in | |
| 382 | − * the same NUMA node as CPU0. | |
| 383 | − * It is not possible to accurately determine SNC state if the system is | |
| 384 | − * booted with a maxcpus=N parameter. That distorts the ratio of SNC nodes | |
| 385 | − * to L3 caches. It will be OK if system is booted with hyperthreading | |
| 386 | − * disabled (since this doesn't affect the ratio). | |
| 387 | − */ | |
| 388 | 378 | static __init int snc_get_config(void) |
| 389 | 379 | { |
| 390 | − struct cacheinfo *ci = get_cpu_cacheinfo_level(0, RESCTRL_L3_CACHE); | |
| 391 | − const cpumask_t *node0_cpumask; | |
| 392 | − int cpus_per_node, cpus_per_l3; | |
| 393 | − int ret; | |
| 394 | − | |
| 395 | − if (!x86_match_cpu(snc_cpu_ids) || !ci) | |
| 396 | − return 1; | |
| 380 | + int ret = topology_num_nodes_per_package(); | |
| 397 | 381 | |
| 398 | − cpus_read_lock(); | |
| 399 | − if (num_online_cpus() != num_present_cpus()) | |
| 400 | − pr_warn("Some CPUs offline, SNC detection may be incorrect\n"); | |
| 401 | − cpus_read_unlock(); | |
| 402 | − | |
| 403 | − node0_cpumask = cpumask_of_node(cpu_to_node(0)); | |
| 404 | − | |
| 405 | − cpus_per_node = cpumask_weight(node0_cpumask); | |
| 406 | − cpus_per_l3 = cpumask_weight(&ci->shared_cpu_map); | |
| 407 | − | |
| 408 | − if (!cpus_per_node || !cpus_per_l3) | |
| 382 | + if (ret > 1 && !x86_match_cpu(snc_cpu_ids)) { | |
| 383 | + pr_warn("CoD enabled system? Resctrl not supported\n"); | |
| 409 | 384 | return 1; |
| 410 | − | |
| 411 | − ret = cpus_per_l3 / cpus_per_node; | |
| 385 | + } | |
| 412 | 386 | |
| 413 | 387 | /* sanity check: Only valid results are 1, 2, 3, 4, 6 */ |
| 414 | 388 | switch (ret) { |
| @@ -31,6 +31,7 @@ · 30 unmodified lines | ||
| 31 | 31 | #include <asm/mpspec.h> |
| 32 | 32 | #include <asm/msr.h> |
| 33 | 33 | #include <asm/smp.h> |
| 34 | +#include <asm/numa.h> | |
| 34 | 35 | |
| 35 | 36 | #include "cpu.h" |
| 36 | 37 | |
| @@ -492,11 +493,19 @@ void __init topology_init_possible_cpus(void) · 455 unmodified lines | ||
| 492 | 493 | set_nr_cpu_ids(allowed); |
| 493 | 494 | |
| 494 | 495 | cnta = domain_weight(TOPO_PKG_DOMAIN); |
| 495 | − cntb = domain_weight(TOPO_DIE_DOMAIN); | |
| 496 | 496 | __max_logical_packages = cnta; |
| 497 | + | |
| 498 | + pr_info("Max. logical packages: %3u\n", __max_logical_packages); | |
| 499 | + | |
| 500 | + cntb = num_phys_nodes(); | |
| 501 | + __num_nodes_per_package = DIV_ROUND_UP(cntb, cnta); | |
| 502 | + | |
| 503 | + pr_info("Max. logical nodes: %3u\n", cntb); | |
| 504 | + pr_info("Num. nodes per package:%3u\n", __num_nodes_per_package); | |
| 505 | + | |
| 506 | + cntb = domain_weight(TOPO_DIE_DOMAIN); | |
| 497 | 507 | __max_dies_per_package = 1U << (get_count_order(cntb) - get_count_order(cnta)); |
| 498 | 508 | |
| 499 | − pr_info("Max. logical packages: %3u\n", cnta); | |
| 500 | 509 | pr_info("Max. logical dies: %3u\n", cntb); |
| 501 | 510 | pr_info("Max. dies per package: %3u\n", __max_dies_per_package); |
| 502 | 511 | |
| @@ -616,38 +616,10 @@ SYM_DATA(early_recursion_flag, .long 0) · 615 unmodified lines | ||
| 616 | 616 | |
| 617 | 617 | .data |
| 618 | 618 | |
| 619 | −#if defined(CONFIG_XEN_PV) || defined(CONFIG_PVH) | |
| 620 | −SYM_DATA_START_PTI_ALIGNED(init_top_pgt) | |
| 621 | − .quad level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC | |
| 622 | − .org init_top_pgt + L4_PAGE_OFFSET*8, 0 | |
| 623 | − .quad level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC | |
| 624 | − .org init_top_pgt + L4_START_KERNEL*8, 0 | |
| 625 | − /* (2^48-(2*1024*1024*1024))/(2^39) = 511 */ | |
| 626 | − .quad level3_kernel_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC | |
| 627 | − .fill PTI_USER_PGD_FILL,8,0 | |
| 628 | −SYM_DATA_END(init_top_pgt) | |
| 629 | − | |
| 630 | −SYM_DATA_START_PAGE_ALIGNED(level3_ident_pgt) | |
| 631 | − .quad level2_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC | |
| 632 | − .fill 511, 8, 0 | |
| 633 | −SYM_DATA_END(level3_ident_pgt) | |
| 634 | −SYM_DATA_START_PAGE_ALIGNED(level2_ident_pgt) | |
| 635 | − /* | |
| 636 | − * Since I easily can, map the first 1G. | |
| 637 | − * Don't set NX because code runs from these pages. | |
| 638 | − * | |
| 639 | − * Note: This sets _PAGE_GLOBAL despite whether | |
| 640 | − * the CPU supports it or it is enabled. But, | |
| 641 | − * the CPU should ignore the bit. | |
| 642 | − */ | |
| 643 | − PMDS(0, __PAGE_KERNEL_IDENT_LARGE_EXEC, PTRS_PER_PMD) | |
| 644 | −SYM_DATA_END(level2_ident_pgt) | |
| 645 | −#else | |
| 646 | 619 | SYM_DATA_START_PTI_ALIGNED(init_top_pgt) |
| 647 | 620 | .fill 512,8,0 |
| 648 | 621 | .fill PTI_USER_PGD_FILL,8,0 |
| 649 | 622 | SYM_DATA_END(init_top_pgt) |
| 650 | −#endif | |
| 651 | 623 | |
| 652 | 624 | SYM_DATA_START_PAGE_ALIGNED(level4_kernel_pgt) |
| 653 | 625 | .fill 511,8,0 |
| @@ -468,13 +468,6 @@ static int x86_cluster_flags(void) · 467 unmodified lines | ||
| 468 | 468 | } |
| 469 | 469 | #endif |
| 470 | 470 | |
| 471 | −/* | |
| 472 | − * Set if a package/die has multiple NUMA nodes inside. | |
| 473 | − * AMD Magny-Cours, Intel Cluster-on-Die, and Intel | |
| 474 | − * Sub-NUMA Clustering have this. | |
| 475 | − */ | |
| 476 | −static bool x86_has_numa_in_package; | |
| 477 | − | |
| 478 | 471 | static struct sched_domain_topology_level x86_topology[] = { |
| 479 | 472 | SDTL_INIT(tl_smt_mask, cpu_smt_flags, SMT), |
| 480 | 473 | #ifdef CONFIG_SCHED_CLUSTER |
| @@ -496,7 +489,7 @@ static void __init build_sched_topology(void) · 15 unmodified lines | ||
| 496 | 489 | * PKG domain since the NUMA domains will auto-magically create the |
| 497 | 490 | * right spanning domains based on the SLIT. |
| 498 | 491 | */ |
| 499 | − if (x86_has_numa_in_package) { | |
| 492 | + if (topology_num_nodes_per_package() > 1) { | |
| 500 | 493 | unsigned int pkgdom = ARRAY_SIZE(x86_topology) - 2; |
| 501 | 494 | |
| 502 | 495 | memset(&x86_topology[pkgdom], 0, sizeof(x86_topology[pkgdom])); |
| @@ -513,33 +506,149 @@ static void __init build_sched_topology(void) · 10 unmodified lines | ||
| 513 | 506 | } |
| 514 | 507 | |
| 515 | 508 | #ifdef CONFIG_NUMA |
| 516 | −static int sched_avg_remote_distance; | |
| 517 | −static int avg_remote_numa_distance(void) | |
| 509 | +/* | |
| 510 | + * Test if the on-trace cluster at (N,N) is symmetric. | |
| 511 | + * Uses upper triangle iteration to avoid obvious duplicates. | |
| 512 | + */ | |
| 513 | +static bool slit_cluster_symmetric(int N) | |
| 518 | 514 | { |
| 519 | − int i, j; | |
| 520 | − int distance, nr_remote, total_distance; | |
| 521 | − | |
| 522 | − if (sched_avg_remote_distance > 0) | |
| 523 | − return sched_avg_remote_distance; | |
| 524 | − | |
| 525 | − nr_remote = 0; | |
| 526 | − total_distance = 0; | |
| 527 | − for_each_node_state(i, N_CPU) { | |
| 528 | − for_each_node_state(j, N_CPU) { | |
| 529 | − distance = node_distance(i, j); | |
| 530 | − | |
| 531 | − if (distance >= REMOTE_DISTANCE) { | |
| 532 | − nr_remote++; | |
| 533 | − total_distance += distance; | |
| 534 | − } | |
| 515 | + int u = topology_num_nodes_per_package(); | |
| 516 | + | |
| 517 | + for (int k = 0; k < u; k++) { | |
| 518 | + for (int l = k; l < u; l++) { | |
| 519 | + if (node_distance(N + k, N + l) != | |
| 520 | + node_distance(N + l, N + k)) | |
| 521 | + return false; | |
| 535 | 522 | } |
| 536 | 523 | } |
| 537 | − if (nr_remote) | |
| 538 | − sched_avg_remote_distance = total_distance / nr_remote; | |
| 539 | − else | |
| 540 | − sched_avg_remote_distance = REMOTE_DISTANCE; | |
| 541 | 524 | |
| 542 | − return sched_avg_remote_distance; | |
| 525 | + return true; | |
| 526 | +} | |
| 527 | + | |
| 528 | +/* | |
| 529 | + * Return the package-id of the cluster, or ~0 if indeterminate. | |
| 530 | + * Each node in the on-trace cluster should have the same package-id. | |
| 531 | + */ | |
| 532 | +static u32 slit_cluster_package(int N) | |
| 533 | +{ | |
| 534 | + int u = topology_num_nodes_per_package(); | |
| 535 | + u32 pkg_id = ~0; | |
| 536 | + | |
| 537 | + for (int n = 0; n < u; n++) { | |
| 538 | + const struct cpumask *cpus = cpumask_of_node(N + n); | |
| 539 | + int cpu; | |
| 540 | + | |
| 541 | + for_each_cpu(cpu, cpus) { | |
| 542 | + u32 id = topology_logical_package_id(cpu); | |
| 543 | + | |
| 544 | + if (pkg_id == ~0) | |
| 545 | + pkg_id = id; | |
| 546 | + if (pkg_id != id) | |
| 547 | + return ~0; | |
| 548 | + } | |
| 549 | + } | |
| 550 | + | |
| 551 | + return pkg_id; | |
| 552 | +} | |
| 553 | + | |
| 554 | +/* | |
| 555 | + * Validate the SLIT table is of the form expected for SNC, specifically: | |
| 556 | + * | |
| 557 | + * - each on-trace cluster should be symmetric, | |
| 558 | + * - each on-trace cluster should have a unique package-id. | |
| 559 | + * | |
| 560 | + * If you NUMA_EMU on top of SNC, you get to keep the pieces. | |
| 561 | + */ | |
| 562 | +static bool slit_validate(void) | |
| 563 | +{ | |
| 564 | + int u = topology_num_nodes_per_package(); | |
| 565 | + u32 pkg_id, prev_pkg_id = ~0; | |
| 566 | + | |
| 567 | + for (int pkg = 0; pkg < topology_max_packages(); pkg++) { | |
| 568 | + int n = pkg * u; | |
| 569 | + | |
| 570 | + /* | |
| 571 | + * Ensure the on-trace cluster is symmetric and each cluster | |
| 572 | + * has a different package id. | |
| 573 | + */ | |
| 574 | + if (!slit_cluster_symmetric(n)) | |
| 575 | + return false; | |
| 576 | + pkg_id = slit_cluster_package(n); | |
| 577 | + if (pkg_id == ~0) | |
| 578 | + return false; | |
| 579 | + if (pkg && pkg_id == prev_pkg_id) | |
| 580 | + return false; | |
| 581 | + | |
| 582 | + prev_pkg_id = pkg_id; | |
| 583 | + } | |
| 584 | + | |
| 585 | + return true; | |
| 586 | +} | |
| 587 | + | |
| 588 | +/* | |
| 589 | + * Compute a sanitized SLIT table for SNC; notably SNC-3 can end up with | |
| 590 | + * asymmetric off-trace clusters, reflecting physical assymmetries. However | |
| 591 | + * this leads to 'unfortunate' sched_domain configurations. | |
| 592 | + * | |
| 593 | + * For example dual socket GNR with SNC-3: | |
| 594 | + * | |
| 595 | + * node distances: | |
| 596 | + * node 0 1 2 3 4 5 | |
| 597 | + * 0: 10 15 17 21 28 26 | |
| 598 | + * 1: 15 10 15 23 26 23 | |
| 599 | + * 2: 17 15 10 26 23 21 | |
| 600 | + * 3: 21 28 26 10 15 17 | |
| 601 | + * 4: 23 26 23 15 10 15 | |
| 602 | + * 5: 26 23 21 17 15 10 | |
| 603 | + * | |
| 604 | + * Fix things up by averaging out the off-trace clusters; resulting in: | |
| 605 | + * | |
| 606 | + * node 0 1 2 3 4 5 | |
| 607 | + * 0: 10 15 17 24 24 24 | |
| 608 | + * 1: 15 10 15 24 24 24 | |
| 609 | + * 2: 17 15 10 24 24 24 | |
| 610 | + * 3: 24 24 24 10 15 17 | |
| 611 | + * 4: 24 24 24 15 10 15 | |
| 612 | + * 5: 24 24 24 17 15 10 | |
| 613 | + */ | |
| 614 | +static int slit_cluster_distance(int i, int j) | |
| 615 | +{ | |
| 616 | + static int slit_valid = -1; | |
| 617 | + int u = topology_num_nodes_per_package(); | |
| 618 | + long d = 0; | |
| 619 | + int x, y; | |
| 620 | + | |
| 621 | + if (slit_valid < 0) { | |
| 622 | + slit_valid = slit_validate(); | |
| 623 | + if (!slit_valid) | |
| 624 | + pr_err(FW_BUG "SLIT table doesn't have the expected form for SNC -- fixup disabled!\n"); | |
| 625 | + else | |
| 626 | + pr_info("Fixing up SNC SLIT table.\n"); | |
| 627 | + } | |
| 628 | + | |
| 629 | + /* | |
| 630 | + * Is this a unit cluster on the trace? | |
| 631 | + */ | |
| 632 | + if ((i / u) == (j / u) || !slit_valid) | |
| 633 | + return node_distance(i, j); | |
| 634 | + | |
| 635 | + /* | |
| 636 | + * Off-trace cluster. | |
| 637 | + * | |
| 638 | + * Notably average out the symmetric pair of off-trace clusters to | |
| 639 | + * ensure the resulting SLIT table is symmetric. | |
| 640 | + */ | |
| 641 | + x = i - (i % u); | |
| 642 | + y = j - (j % u); | |
| 643 | + | |
| 644 | + for (i = x; i < x + u; i++) { | |
| 645 | + for (j = y; j < y + u; j++) { | |
| 646 | + d += node_distance(i, j); | |
| 647 | + d += node_distance(j, i); | |
| 648 | + } | |
| 649 | + } | |
| 650 | + | |
| 651 | + return d / (2*u*u); | |
| 543 | 652 | } |
| 544 | 653 | |
| 545 | 654 | int arch_sched_node_distance(int from, int to) |
| @@ -549,34 +658,14 @@ int arch_sched_node_distance(int from, int to) · 3 unmodified lines | ||
| 549 | 658 | switch (boot_cpu_data.x86_vfm) { |
| 550 | 659 | case INTEL_GRANITERAPIDS_X: |
| 551 | 660 | case INTEL_ATOM_DARKMONT_X: |
| 552 | − | |
| 553 | − if (!x86_has_numa_in_package || topology_max_packages() == 1 || | |
| 554 | − d < REMOTE_DISTANCE) | |
| 661 | + if (topology_max_packages() == 1 || | |
| 662 | + topology_num_nodes_per_package() < 3) | |
| 555 | 663 | return d; |
| 556 | 664 | |
| 557 | 665 | /* |
| 558 | − * With SNC enabled, there could be too many levels of remote | |
| 559 | − * NUMA node distances, creating NUMA domain levels | |
| 560 | − * including local nodes and partial remote nodes. | |
| 561 | − * | |
| 562 | − * Trim finer distance tuning for NUMA nodes in remote package | |
| 563 | − * for the purpose of building sched domains. Group NUMA nodes | |
| 564 | − * in the remote package in the same sched group. | |
| 565 | − * Simplify NUMA domains and avoid extra NUMA levels including | |
| 566 | − * different remote NUMA nodes and local nodes. | |
| 567 | − * | |
| 568 | − * GNR and CWF don't expect systems with more than 2 packages | |
| 569 | − * and more than 2 hops between packages. Single average remote | |
| 570 | − * distance won't be appropriate if there are more than 2 | |
| 571 | − * packages as average distance to different remote packages | |
| 572 | − * could be different. | |
| 666 | + * Handle SNC-3 asymmetries. | |
| 573 | 667 | */ |
| 574 | − WARN_ONCE(topology_max_packages() > 2, | |
| 575 | − "sched: Expect only up to 2 packages for GNR or CWF, " | |
| 576 | − "but saw %d packages when building sched domains.", | |
| 577 | − topology_max_packages()); | |
| 578 | − | |
| 579 | − d = avg_remote_numa_distance(); | |
| 668 | + return slit_cluster_distance(from, to); | |
| 580 | 669 | } |
| 581 | 670 | return d; |
| 582 | 671 | } |
| @@ -606,7 +695,7 @@ void set_cpu_sibling_map(int cpu) · 23 unmodified lines | ||
| 606 | 695 | o = &cpu_data(i); |
| 607 | 696 | |
| 608 | 697 | if (match_pkg(c, o) && !topology_same_node(c, o)) |
| 609 | − x86_has_numa_in_package = true; | |
| 698 | + WARN_ON_ONCE(topology_num_nodes_per_package() == 1); | |
| 610 | 699 | |
| 611 | 700 | if ((i == cpu) || (has_smt && match_smt(c, o))) |
| 612 | 701 | link_mask(topology_sibling_cpumask, cpu, i); |
| @@ -397,7 +397,7 @@ static inline void handle_invalid_op(struct pt_regs *regs) · 396 unmodified lines | ||
| 397 | 397 | ILL_ILLOPN, error_get_trap_addr(regs)); |
| 398 | 398 | } |
| 399 | 399 | |
| 400 | −static noinstr bool handle_bug(struct pt_regs *regs) | |
| 400 | +noinstr bool handle_bug(struct pt_regs *regs) | |
| 401 | 401 | { |
| 402 | 402 | unsigned long addr = regs->ip; |
| 403 | 403 | bool handled = false; |
| @@ -427,6 +427,7 @@ SECTIONS · 426 unmodified lines | ||
| 427 | 427 | .llvm_bb_addr_map : { *(.llvm_bb_addr_map) } |
| 428 | 428 | #endif |
| 429 | 429 | |
| 430 | + MODINFO | |
| 430 | 431 | ELF_DETAILS |
| 431 | 432 | |
| 432 | 433 | DISCARDS |
| @@ -20,7 +20,6 @@ if VIRTUALIZATION · 19 unmodified lines | ||
| 20 | 20 | config KVM_X86 |
| 21 | 21 | def_tristate KVM if (KVM_INTEL != n || KVM_AMD != n) |
| 22 | 22 | select KVM_COMMON |
| 23 | − select KVM_GENERIC_MMU_NOTIFIER | |
| 24 | 23 | select KVM_ELIDE_TLB_FLUSH_IF_YOUNG |
| 25 | 24 | select KVM_MMU_LOCKLESS_AGING |
| 26 | 25 | select HAVE_KVM_IRQCHIP |
| @@ -776,7 +776,10 @@ do { \ · 775 unmodified lines | ||
| 776 | 776 | #define SYNTHESIZED_F(name) \ |
| 777 | 777 | ({ \ |
| 778 | 778 | kvm_cpu_cap_synthesized |= feature_bit(name); \ |
| 779 | − F(name); \ | |
| 779 | + \ | |
| 780 | + BUILD_BUG_ON(X86_FEATURE_##name >= MAX_CPU_FEATURES); \ | |
| 781 | + if (boot_cpu_has(X86_FEATURE_##name)) \ | |
| 782 | + F(name); \ | |
| 780 | 783 | }) |
| 781 | 784 | |
| 782 | 785 | /* |
| @@ -1981,16 +1981,17 @@ int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu) · 1980 unmodified lines | ||
| 1981 | 1981 | if (entries[i] == KVM_HV_TLB_FLUSHALL_ENTRY) |
| 1982 | 1982 | goto out_flush_all; |
| 1983 | 1983 | |
| 1984 | − if (is_noncanonical_invlpg_address(entries[i], vcpu)) | |
| 1985 | − continue; | |
| 1986 | − | |
| 1987 | 1984 | /* |
| 1988 | 1985 | * Lower 12 bits of 'address' encode the number of additional |
| 1989 | 1986 | * pages to flush. |
| 1990 | 1987 | */ |
| 1991 | 1988 | gva = entries[i] & PAGE_MASK; |
| 1992 | − for (j = 0; j < (entries[i] & ~PAGE_MASK) + 1; j++) | |
| 1989 | + for (j = 0; j < (entries[i] & ~PAGE_MASK) + 1; j++) { | |
| 1990 | + if (is_noncanonical_invlpg_address(gva + j * PAGE_SIZE, vcpu)) | |
| 1991 | + continue; | |
| 1992 | + | |
| 1993 | 1993 | kvm_x86_call(flush_tlb_gva)(vcpu, gva + j * PAGE_SIZE); |
| 1994 | + } | |
| 1994 | 1995 | |
| 1995 | 1996 | ++vcpu->stat.tlb_flush; |
| 1996 | 1997 | } |
| @@ -321,7 +321,8 @@ void kvm_fire_mask_notifiers(struct kvm *kvm, unsigned irqchip, unsigned pin, · 320 unmodified lines | ||
| 321 | 321 | idx = srcu_read_lock(&kvm->irq_srcu); |
| 322 | 322 | gsi = kvm_irq_map_chip_pin(kvm, irqchip, pin); |
| 323 | 323 | if (gsi != -1) |
| 324 | − hlist_for_each_entry_rcu(kimn, &ioapic->mask_notifier_list, link) | |
| 324 | + hlist_for_each_entry_srcu(kimn, &ioapic->mask_notifier_list, link, | |
| 325 | + srcu_read_lock_held(&kvm->irq_srcu)) | |
| 325 | 326 | if (kimn->irq == gsi) |
| 326 | 327 | kimn->func(kimn, mask); |
| 327 | 328 | srcu_read_unlock(&kvm->irq_srcu, idx); |
| @@ -189,12 +189,12 @@ static void avic_activate_vmcb(struct vcpu_svm *svm) · 188 unmodified lines | ||
| 189 | 189 | struct kvm_vcpu *vcpu = &svm->vcpu; |
| 190 | 190 | |
| 191 | 191 | vmcb->control.int_ctl &= ~(AVIC_ENABLE_MASK | X2APIC_MODE_MASK); |
| 192 | − | |
| 193 | 192 | vmcb->control.avic_physical_id &= ~AVIC_PHYSICAL_MAX_INDEX_MASK; |
| 194 | 193 | vmcb->control.avic_physical_id |= avic_get_max_physical_id(vcpu); |
| 195 | − | |
| 196 | 194 | vmcb->control.int_ctl |= AVIC_ENABLE_MASK; |
| 197 | 195 | |
| 196 | + svm_clr_intercept(svm, INTERCEPT_CR8_WRITE); | |
| 197 | + | |
| 198 | 198 | /* |
| 199 | 199 | * Note: KVM supports hybrid-AVIC mode, where KVM emulates x2APIC MSR |
| 200 | 200 | * accesses, while interrupt injection to a running vCPU can be |
| @@ -226,6 +226,9 @@ static void avic_deactivate_vmcb(struct vcpu_svm *svm) · 25 unmodified lines | ||
| 226 | 226 | vmcb->control.int_ctl &= ~(AVIC_ENABLE_MASK | X2APIC_MODE_MASK); |
| 227 | 227 | vmcb->control.avic_physical_id &= ~AVIC_PHYSICAL_MAX_INDEX_MASK; |
| 228 | 228 | |
| 229 | + if (!sev_es_guest(svm->vcpu.kvm)) | |
| 230 | + svm_set_intercept(svm, INTERCEPT_CR8_WRITE); | |
| 231 | + | |
| 229 | 232 | /* |
| 230 | 233 | * If running nested and the guest uses its own MSR bitmap, there |
| 231 | 234 | * is no need to update L0's msr bitmap |
| @@ -368,7 +371,7 @@ void avic_init_vmcb(struct vcpu_svm *svm, struct vmcb *vmcb) · 136 unmodified lines | ||
| 368 | 371 | vmcb->control.avic_physical_id = __sme_set(__pa(kvm_svm->avic_physical_id_table)); |
| 369 | 372 | vmcb->control.avic_vapic_bar = APIC_DEFAULT_PHYS_BASE; |
| 370 | 373 | |
| 371 | − if (kvm_apicv_activated(svm->vcpu.kvm)) | |
| 374 | + if (kvm_vcpu_apicv_active(&svm->vcpu)) | |
| 372 | 375 | avic_activate_vmcb(svm); |
| 373 | 376 | else |
| 374 | 377 | avic_deactivate_vmcb(svm); |
| @@ -418,6 +418,15 @@ static bool nested_vmcb_check_controls(struct kvm_vcpu *vcpu) · 417 unmodified lines | ||
| 418 | 418 | return __nested_vmcb_check_controls(vcpu, ctl); |
| 419 | 419 | } |
| 420 | 420 | |
| 421 | +int nested_svm_check_cached_vmcb12(struct kvm_vcpu *vcpu) | |
| 422 | +{ | |
| 423 | + if (!nested_vmcb_check_save(vcpu) || | |
| 424 | + !nested_vmcb_check_controls(vcpu)) | |
| 425 | + return -EINVAL; | |
| 426 | + | |
| 427 | + return 0; | |
| 428 | +} | |
| 429 | + | |
| 421 | 430 | /* |
| 422 | 431 | * If a feature is not advertised to L1, clear the corresponding vmcb12 |
| 423 | 432 | * intercept. |
| @@ -1028,8 +1037,7 @@ int nested_svm_vmrun(struct kvm_vcpu *vcpu) · 604 unmodified lines | ||
| 1028 | 1037 | nested_copy_vmcb_control_to_cache(svm, &vmcb12->control); |
| 1029 | 1038 | nested_copy_vmcb_save_to_cache(svm, &vmcb12->save); |
| 1030 | 1039 | |
| 1031 | − if (!nested_vmcb_check_save(vcpu) || | |
| 1032 | − !nested_vmcb_check_controls(vcpu)) { | |
| 1040 | + if (nested_svm_check_cached_vmcb12(vcpu) < 0) { | |
| 1033 | 1041 | vmcb12->control.exit_code = SVM_EXIT_ERR; |
| 1034 | 1042 | vmcb12->control.exit_info_1 = 0; |
| 1035 | 1043 | vmcb12->control.exit_info_2 = 0; |
| @@ -1077,8 +1077,7 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event) · 1076 unmodified lines | ||
| 1077 | 1077 | svm_set_intercept(svm, INTERCEPT_CR0_WRITE); |
| 1078 | 1078 | svm_set_intercept(svm, INTERCEPT_CR3_WRITE); |
| 1079 | 1079 | svm_set_intercept(svm, INTERCEPT_CR4_WRITE); |
| 1080 | − if (!kvm_vcpu_apicv_active(vcpu)) | |
| 1081 | − svm_set_intercept(svm, INTERCEPT_CR8_WRITE); | |
| 1080 | + svm_set_intercept(svm, INTERCEPT_CR8_WRITE); | |
| 1082 | 1081 | |
| 1083 | 1082 | set_dr_intercepts(svm); |
| 1084 | 1083 | |
| @@ -1189,7 +1188,7 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event) · 104 unmodified lines | ||
| 1189 | 1188 | if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS)) |
| 1190 | 1189 | svm->vmcb->control.erap_ctl |= ERAP_CONTROL_ALLOW_LARGER_RAP; |
| 1191 | 1190 | |
| 1192 | − if (kvm_vcpu_apicv_active(vcpu)) | |
| 1191 | + if (enable_apicv && irqchip_in_kernel(vcpu->kvm)) | |
| 1193 | 1192 | avic_init_vmcb(svm, vmcb); |
| 1194 | 1193 | |
| 1195 | 1194 | if (vnmi) |
| @@ -2674,9 +2673,11 @@ static int dr_interception(struct kvm_vcpu *vcpu) · 1478 unmodified lines | ||
| 2674 | 2673 | |
| 2675 | 2674 | static int cr8_write_interception(struct kvm_vcpu *vcpu) |
| 2676 | 2675 | { |
| 2676 | + u8 cr8_prev = kvm_get_cr8(vcpu); | |
| 2677 | 2677 | int r; |
| 2678 | 2678 | |
| 2679 | − u8 cr8_prev = kvm_get_cr8(vcpu); | |
| 2679 | + WARN_ON_ONCE(kvm_vcpu_apicv_active(vcpu)); | |
| 2680 | + | |
| 2680 | 2681 | /* instruction emulation calls kvm_set_cr8() */ |
| 2681 | 2682 | r = cr_interception(vcpu); |
| 2682 | 2683 | if (lapic_in_kernel(vcpu)) |
| @@ -4879,11 +4880,15 @@ static int svm_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram) · 2196 unmodified lines | ||
| 4879 | 4880 | vmcb12 = map.hva; |
| 4880 | 4881 | nested_copy_vmcb_control_to_cache(svm, &vmcb12->control); |
| 4881 | 4882 | nested_copy_vmcb_save_to_cache(svm, &vmcb12->save); |
| 4882 | − ret = enter_svm_guest_mode(vcpu, smram64->svm_guest_vmcb_gpa, vmcb12, false); | |
| 4883 | 4883 | |
| 4884 | − if (ret) | |
| 4884 | + if (nested_svm_check_cached_vmcb12(vcpu) < 0) | |
| 4885 | + goto unmap_save; | |
| 4886 | + | |
| 4887 | + if (enter_svm_guest_mode(vcpu, smram64->svm_guest_vmcb_gpa, | |
| 4888 | + vmcb12, false) != 0) | |
| 4885 | 4889 | goto unmap_save; |
| 4886 | 4890 | |
| 4891 | + ret = 0; | |
| 4887 | 4892 | svm->nested.nested_run_pending = 1; |
| 4888 | 4893 | |
| 4889 | 4894 | unmap_save: |
| @@ -797,6 +797,7 @@ static inline int nested_svm_simple_vmexit(struct vcpu_svm *svm, u32 exit_code) · 796 unmodified lines | ||
| 797 | 797 | |
| 798 | 798 | int nested_svm_exit_handled(struct vcpu_svm *svm); |
| 799 | 799 | int nested_svm_check_permissions(struct kvm_vcpu *vcpu); |
| 800 | +int nested_svm_check_cached_vmcb12(struct kvm_vcpu *vcpu); | |
| 800 | 801 | int nested_svm_check_exception(struct vcpu_svm *svm, unsigned nr, |
| 801 | 802 | bool has_error_code, u32 error_code); |
| 802 | 803 | int nested_svm_exit_special(struct vcpu_svm *svm); |
| @@ -3300,10 +3300,24 @@ static int nested_vmx_check_guest_state(struct kvm_vcpu *vcpu, · 3299 unmodified lines | ||
| 3300 | 3300 | if (CC(vmcs12->guest_cr4 & X86_CR4_CET && !(vmcs12->guest_cr0 & X86_CR0_WP))) |
| 3301 | 3301 | return -EINVAL; |
| 3302 | 3302 | |
| 3303 | − if ((vmcs12->vm_entry_controls & VM_ENTRY_LOAD_DEBUG_CONTROLS) && | |
| 3304 | − (CC(!kvm_dr7_valid(vmcs12->guest_dr7)) || | |
| 3305 | − CC(!vmx_is_valid_debugctl(vcpu, vmcs12->guest_ia32_debugctl, false)))) | |
| 3306 | − return -EINVAL; | |
| 3303 | + if (vmcs12->vm_entry_controls & VM_ENTRY_LOAD_DEBUG_CONTROLS) { | |
| 3304 | + u64 debugctl = vmcs12->guest_ia32_debugctl; | |
| 3305 | + | |
| 3306 | + /* | |
| 3307 | + * FREEZE_IN_SMM is not virtualized, but allow L1 to set it in | |
| 3308 | + * vmcs12's DEBUGCTL under a quirk for backwards compatibility. | |
| 3309 | + * Note that the quirk only relaxes the consistency check. The | |
| 3310 | + * vmcc02 bit is still under the control of the host. In | |
| 3311 | + * particular, if a host administrator decides to clear the bit, | |
| 3312 | + * then L1 has no say in the matter. | |
| 3313 | + */ | |
| 3314 | + if (kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_VMCS12_ALLOW_FREEZE_IN_SMM)) | |
| 3315 | + debugctl &= ~DEBUGCTLMSR_FREEZE_IN_SMM; | |
| 3316 | + | |
| 3317 | + if (CC(!kvm_dr7_valid(vmcs12->guest_dr7)) || | |
| 3318 | + CC(!vmx_is_valid_debugctl(vcpu, debugctl, false))) | |
| 3319 | + return -EINVAL; | |
| 3320 | + } | |
| 3307 | 3321 | |
| 3308 | 3322 | if ((vmcs12->vm_entry_controls & VM_ENTRY_LOAD_IA32_PAT) && |
| 3309 | 3323 | CC(!kvm_pat_valid(vmcs12->guest_ia32_pat))) |
| @@ -6842,13 +6856,34 @@ void vmx_leave_nested(struct kvm_vcpu *vcpu) · 3532 unmodified lines | ||
| 6842 | 6856 | free_nested(vcpu); |
| 6843 | 6857 | } |
| 6844 | 6858 | |
| 6859 | +int nested_vmx_check_restored_vmcs12(struct kvm_vcpu *vcpu) | |
| 6860 | +{ | |
| 6861 | + enum vm_entry_failure_code ignored; | |
| 6862 | + struct vmcs12 *vmcs12 = get_vmcs12(vcpu); | |
| 6863 | + | |
| 6864 | + if (nested_cpu_has_shadow_vmcs(vmcs12) && | |
| 6865 | + vmcs12->vmcs_link_pointer != INVALID_GPA) { | |
| 6866 | + struct vmcs12 *shadow_vmcs12 = get_shadow_vmcs12(vcpu); | |
| 6867 | + | |
| 6868 | + if (shadow_vmcs12->hdr.revision_id != VMCS12_REVISION || | |
| 6869 | + !shadow_vmcs12->hdr.shadow_vmcs) | |
| 6870 | + return -EINVAL; | |
| 6871 | + } | |
| 6872 | + | |
| 6873 | + if (nested_vmx_check_controls(vcpu, vmcs12) || | |
| 6874 | + nested_vmx_check_host_state(vcpu, vmcs12) || | |
| 6875 | + nested_vmx_check_guest_state(vcpu, vmcs12, &ignored)) | |
| 6876 | + return -EINVAL; | |
| 6877 | + | |
| 6878 | + return 0; | |
| 6879 | +} | |
| 6880 | + | |
| 6845 | 6881 | static int vmx_set_nested_state(struct kvm_vcpu *vcpu, |
| 6846 | 6882 | struct kvm_nested_state __user *user_kvm_nested_state, |
| 6847 | 6883 | struct kvm_nested_state *kvm_state) |
| 6848 | 6884 | { |
| 6849 | 6885 | struct vcpu_vmx *vmx = to_vmx(vcpu); |
| 6850 | 6886 | struct vmcs12 *vmcs12; |
| 6851 | − enum vm_entry_failure_code ignored; | |
| 6852 | 6887 | struct kvm_vmx_nested_state_data __user *user_vmx_nested_state = |
| 6853 | 6888 | &user_kvm_nested_state->data.vmx[0]; |
| 6854 | 6889 | int ret; |
| @@ -6979,25 +7014,20 @@ static int vmx_set_nested_state(struct kvm_vcpu *vcpu, · 124 unmodified lines | ||
| 6979 | 7014 | vmx->nested.mtf_pending = |
| 6980 | 7015 | !!(kvm_state->flags & KVM_STATE_NESTED_MTF_PENDING); |
| 6981 | 7016 | |
| 6982 | − ret = -EINVAL; | |
| 6983 | 7017 | if (nested_cpu_has_shadow_vmcs(vmcs12) && |
| 6984 | 7018 | vmcs12->vmcs_link_pointer != INVALID_GPA) { |
| 6985 | 7019 | struct vmcs12 *shadow_vmcs12 = get_shadow_vmcs12(vcpu); |
| 6986 | 7020 | |
| 7021 | + ret = -EINVAL; | |
| 6987 | 7022 | if (kvm_state->size < |
| 6988 | 7023 | sizeof(*kvm_state) + |
| 6989 | 7024 | sizeof(user_vmx_nested_state->vmcs12) + sizeof(*shadow_vmcs12)) |
| 6990 | 7025 | goto error_guest_mode; |
| 6991 | 7026 | |
| 7027 | + ret = -EFAULT; | |
| 6992 | 7028 | if (copy_from_user(shadow_vmcs12, |
| 6993 | 7029 | user_vmx_nested_state->shadow_vmcs12, |
| 6994 | − sizeof(*shadow_vmcs12))) { | |
| 6995 | − ret = -EFAULT; | |
| 6996 | − goto error_guest_mode; | |
| 6997 | − } | |
| 6998 | − | |
| 6999 | − if (shadow_vmcs12->hdr.revision_id != VMCS12_REVISION || | |
| 7000 | − !shadow_vmcs12->hdr.shadow_vmcs) | |
| 7030 | + sizeof(*shadow_vmcs12))) | |
| 7001 | 7031 | goto error_guest_mode; |
| 7002 | 7032 | } |
| 7003 | 7033 | |
| @@ -7008,9 +7038,8 @@ static int vmx_set_nested_state(struct kvm_vcpu *vcpu, · 4 unmodified lines | ||
| 7008 | 7038 | kvm_state->hdr.vmx.preemption_timer_deadline; |
| 7009 | 7039 | } |
| 7010 | 7040 | |
| 7011 | − if (nested_vmx_check_controls(vcpu, vmcs12) || | |
| 7012 | − nested_vmx_check_host_state(vcpu, vmcs12) || | |
| 7013 | − nested_vmx_check_guest_state(vcpu, vmcs12, &ignored)) | |
| 7041 | + ret = nested_vmx_check_restored_vmcs12(vcpu); | |
| 7042 | + if (ret < 0) | |
| 7014 | 7043 | goto error_guest_mode; |
| 7015 | 7044 | |
| 7016 | 7045 | vmx->nested.dirty_vmcs12 = true; |
| @@ -22,6 +22,7 @@ void nested_vmx_setup_ctls_msrs(struct vmcs_config *vmcs_conf, u32 ept_caps); · 21 unmodified lines | ||
| 22 | 22 | void nested_vmx_hardware_unsetup(void); |
| 23 | 23 | __init int nested_vmx_hardware_setup(int (*exit_handlers[])(struct kvm_vcpu *)); |
| 24 | 24 | void nested_vmx_set_vmcs_shadowing_bitmap(void); |
| 25 | +int nested_vmx_check_restored_vmcs12(struct kvm_vcpu *vcpu); | |
| 25 | 26 | void nested_vmx_free_vcpu(struct kvm_vcpu *vcpu); |
| 26 | 27 | enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu, |
| 27 | 28 | bool from_vmentry); |
| @@ -1149,7 +1149,7 @@ static void add_atomic_switch_msr(struct vcpu_vmx *vmx, unsigned msr, · 1148 unmodified lines | ||
| 1149 | 1149 | } |
| 1150 | 1150 | |
| 1151 | 1151 | vmx_add_auto_msr(&m->guest, msr, guest_val, VM_ENTRY_MSR_LOAD_COUNT, kvm); |
| 1152 | − vmx_add_auto_msr(&m->guest, msr, host_val, VM_EXIT_MSR_LOAD_COUNT, kvm); | |
| 1152 | + vmx_add_auto_msr(&m->host, msr, host_val, VM_EXIT_MSR_LOAD_COUNT, kvm); | |
| 1153 | 1153 | } |
| 1154 | 1154 | |
| 1155 | 1155 | static bool update_transition_efer(struct vcpu_vmx *vmx) |
| @@ -8528,9 +8528,13 @@ int vmx_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram) · 7372 unmodified lines | ||
| 8528 | 8528 | } |
| 8529 | 8529 | |
| 8530 | 8530 | if (vmx->nested.smm.guest_mode) { |
| 8531 | + /* Triple fault if the state is invalid. */ | |
| 8532 | + if (nested_vmx_check_restored_vmcs12(vcpu) < 0) | |
| 8533 | + return 1; | |
| 8534 | + | |
| 8531 | 8535 | ret = nested_vmx_enter_non_root_mode(vcpu, false); |
| 8532 | − if (ret) | |
| 8533 | − return ret; | |
| 8536 | + if (ret != NVMX_VMENTRY_SUCCESS) | |
| 8537 | + return 1; | |
| 8534 | 8538 | |
| 8535 | 8539 | vmx->nested.nested_run_pending = 1; |
| 8536 | 8540 | vmx->nested.smm.guest_mode = false; |
| @@ -243,7 +243,7 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(enable_ipiv); · 242 unmodified lines | ||
| 243 | 243 | bool __read_mostly enable_device_posted_irqs = true; |
| 244 | 244 | EXPORT_SYMBOL_FOR_KVM_INTERNAL(enable_device_posted_irqs); |
| 245 | 245 | |
| 246 | −const struct _kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 246 | +const struct kvm_stats_desc kvm_vm_stats_desc[] = { | |
| 247 | 247 | KVM_GENERIC_VM_STATS(), |
| 248 | 248 | STATS_DESC_COUNTER(VM, mmu_shadow_zapped), |
| 249 | 249 | STATS_DESC_COUNTER(VM, mmu_pte_write), |
| @@ -269,7 +269,7 @@ const struct kvm_stats_header kvm_vm_stats_header = { · 19 unmodified lines | ||
| 269 | 269 | sizeof(kvm_vm_stats_desc), |
| 270 | 270 | }; |
| 271 | 271 | |
| 272 | −const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 272 | +const struct kvm_stats_desc kvm_vcpu_stats_desc[] = { | |
| 273 | 273 | KVM_GENERIC_VCPU_STATS(), |
| 274 | 274 | STATS_DESC_COUNTER(VCPU, pf_taken), |
| 275 | 275 | STATS_DESC_COUNTER(VCPU, pf_fixed), |
| @@ -4805,7 +4805,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 4529 unmodified lines | ||
| 4805 | 4805 | #endif |
| 4806 | 4806 | case KVM_CAP_NOP_IO_DELAY: |
| 4807 | 4807 | case KVM_CAP_MP_STATE: |
| 4808 | − case KVM_CAP_SYNC_MMU: | |
| 4809 | 4808 | case KVM_CAP_USER_NMI: |
| 4810 | 4809 | case KVM_CAP_IRQ_INJECT_STATUS: |
| 4811 | 4810 | case KVM_CAP_IOEVENTFD: |
| @@ -411,14 +411,11 @@ void __init early_fixup_exception(struct pt_regs *regs, int trapnr) · 410 unmodified lines | ||
| 411 | 411 | return; |
| 412 | 412 | |
| 413 | 413 | if (trapnr == X86_TRAP_UD) { |
| 414 | − if (report_bug(regs->ip, regs) == BUG_TRAP_TYPE_WARN) { | |
| 415 | − /* Skip the ud2. */ | |
| 416 | − regs->ip += LEN_UD2; | |
| 414 | + if (handle_bug(regs)) | |
| 417 | 415 | return; |
| 418 | − } | |
| 419 | 416 | |
| 420 | 417 | /* |
| 421 | − * If this was a BUG and report_bug returns or if this | |
| 418 | + * If this was a BUG and handle_bug returns or if this | |
| 422 | 419 | * was just a normal #UD, we want to continue onward and |
| 423 | 420 | * crash. |
| 424 | 421 | */ |
| @@ -48,6 +48,8 @@ s16 __apicid_to_node[MAX_LOCAL_APIC] = { · 47 unmodified lines | ||
| 48 | 48 | [0 ... MAX_LOCAL_APIC-1] = NUMA_NO_NODE |
| 49 | 49 | }; |
| 50 | 50 | |
| 51 | +nodemask_t numa_phys_nodes_parsed __initdata; | |
| 52 | + | |
| 51 | 53 | int numa_cpu_node(int cpu) |
| 52 | 54 | { |
| 53 | 55 | u32 apicid = early_per_cpu(x86_cpu_to_apicid, cpu); |
| @@ -57,6 +59,11 @@ int numa_cpu_node(int cpu) · 3 unmodified lines | ||
| 57 | 59 | return NUMA_NO_NODE; |
| 58 | 60 | } |
| 59 | 61 | |
| 62 | +int __init num_phys_nodes(void) | |
| 63 | +{ | |
| 64 | + return bitmap_weight(numa_phys_nodes_parsed.bits, MAX_NUMNODES); | |
| 65 | +} | |
| 66 | + | |
| 60 | 67 | cpumask_var_t node_to_cpumask_map[MAX_NUMNODES]; |
| 61 | 68 | EXPORT_SYMBOL(node_to_cpumask_map); |
| 62 | 69 | |
| @@ -210,6 +217,7 @@ static int __init dummy_numa_init(void) · 147 unmodified lines | ||
| 210 | 217 | 0LLU, PFN_PHYS(max_pfn) - 1); |
| 211 | 218 | |
| 212 | 219 | node_set(0, numa_nodes_parsed); |
| 220 | + node_set(0, numa_phys_nodes_parsed); | |
| 213 | 221 | numa_add_memblk(0, 0, PFN_PHYS(max_pfn)); |
| 214 | 222 | |
| 215 | 223 | return 0; |
| @@ -57,6 +57,7 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa) · 56 unmodified lines | ||
| 57 | 57 | } |
| 58 | 58 | set_apicid_to_node(apic_id, node); |
| 59 | 59 | node_set(node, numa_nodes_parsed); |
| 60 | + node_set(node, numa_phys_nodes_parsed); | |
| 60 | 61 | pr_debug("SRAT: PXM %u -> APIC 0x%04x -> Node %u\n", pxm, apic_id, node); |
| 61 | 62 | } |
| 62 | 63 | |
| @@ -97,6 +98,7 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa) · 34 unmodified lines | ||
| 97 | 98 | |
| 98 | 99 | set_apicid_to_node(apic_id, node); |
| 99 | 100 | node_set(node, numa_nodes_parsed); |
| 101 | + node_set(node, numa_phys_nodes_parsed); | |
| 100 | 102 | pr_debug("SRAT: PXM %u -> APIC 0x%02x -> Node %u\n", pxm, apic_id, node); |
| 101 | 103 | } |
| 102 | 104 | |
| @@ -438,17 +438,8 @@ static void emit_kcfi(u8 **pprog, u32 hash) · 437 unmodified lines | ||
| 438 | 438 | |
| 439 | 439 | EMIT1_off32(0xb8, hash); /* movl $hash, %eax */ |
| 440 | 440 | #ifdef CONFIG_CALL_PADDING |
| 441 | − EMIT1(0x90); | |
| 442 | − EMIT1(0x90); | |
| 443 | − EMIT1(0x90); | |
| 444 | − EMIT1(0x90); | |
| 445 | − EMIT1(0x90); | |
| 446 | − EMIT1(0x90); | |
| 447 | − EMIT1(0x90); | |
| 448 | − EMIT1(0x90); | |
| 449 | − EMIT1(0x90); | |
| 450 | − EMIT1(0x90); | |
| 451 | − EMIT1(0x90); | |
| 441 | + for (int i = 0; i < CONFIG_FUNCTION_PADDING_CFI; i++) | |
| 442 | + EMIT1(0x90); | |
| 452 | 443 | #endif |
| 453 | 444 | EMIT_ENDBR(); |
| 454 | 445 | |
| @@ -836,7 +836,7 @@ static void __init __efi_enter_virtual_mode(void) · 835 unmodified lines | ||
| 836 | 836 | } |
| 837 | 837 | |
| 838 | 838 | efi_check_for_embedded_firmwares(); |
| 839 | − efi_free_boot_services(); | |
| 839 | + efi_unmap_boot_services(); | |
| 840 | 840 | |
| 841 | 841 | if (!efi_is_mixed()) |
| 842 | 842 | efi_native_runtime_setup(); |
| @@ -341,7 +341,7 @@ void __init efi_reserve_boot_services(void) · 340 unmodified lines | ||
| 341 | 341 | |
| 342 | 342 | /* |
| 343 | 343 | * Because the following memblock_reserve() is paired |
| 344 | − * with memblock_free_late() for this region in | |
| 344 | + * with free_reserved_area() for this region in | |
| 345 | 345 | * efi_free_boot_services(), we must be extremely |
| 346 | 346 | * careful not to reserve, and subsequently free, |
| 347 | 347 | * critical regions of memory (like the kernel image) or |
| @@ -404,17 +404,33 @@ static void __init efi_unmap_pages(efi_memory_desc_t *md) · 56 unmodified lines | ||
| 404 | 404 | pr_err("Failed to unmap VA mapping for 0x%llx\n", va); |
| 405 | 405 | } |
| 406 | 406 | |
| 407 | −void __init efi_free_boot_services(void) | |
| 407 | +struct efi_freeable_range { | |
| 408 | + u64 start; | |
| 409 | + u64 end; | |
| 410 | +}; | |
| 411 | + | |
| 412 | +static struct efi_freeable_range *ranges_to_free; | |
| 413 | + | |
| 414 | +void __init efi_unmap_boot_services(void) | |
| 408 | 415 | { |
| 409 | 416 | struct efi_memory_map_data data = { 0 }; |
| 410 | 417 | efi_memory_desc_t *md; |
| 411 | 418 | int num_entries = 0; |
| 419 | + int idx = 0; | |
| 420 | + size_t sz; | |
| 412 | 421 | void *new, *new_md; |
| 413 | 422 | |
| 414 | 423 | /* Keep all regions for /sys/kernel/debug/efi */ |
| 415 | 424 | if (efi_enabled(EFI_DBG)) |
| 416 | 425 | return; |
| 417 | 426 | |
| 427 | + sz = sizeof(*ranges_to_free) * efi.memmap.nr_map + 1; | |
| 428 | + ranges_to_free = kzalloc(sz, GFP_KERNEL); | |
| 429 | + if (!ranges_to_free) { | |
| 430 | + pr_err("Failed to allocate storage for freeable EFI regions\n"); | |
| 431 | + return; | |
| 432 | + } | |
| 433 | + | |
| 418 | 434 | for_each_efi_memory_desc(md) { |
| 419 | 435 | unsigned long long start = md->phys_addr; |
| 420 | 436 | unsigned long long size = md->num_pages << EFI_PAGE_SHIFT; |
| @@ -471,7 +487,15 @@ void __init efi_free_boot_services(void) · 50 unmodified lines | ||
| 471 | 487 | start = SZ_1M; |
| 472 | 488 | } |
| 473 | 489 | |
| 474 | − memblock_free_late(start, size); | |
| 490 | + /* | |
| 491 | + * With CONFIG_DEFERRED_STRUCT_PAGE_INIT parts of the memory | |
| 492 | + * map are still not initialized and we can't reliably free | |
| 493 | + * memory here. | |
| 494 | + * Queue the ranges to free at a later point. | |
| 495 | + */ | |
| 496 | + ranges_to_free[idx].start = start; | |
| 497 | + ranges_to_free[idx].end = start + size; | |
| 498 | + idx++; | |
| 475 | 499 | } |
| 476 | 500 | |
| 477 | 501 | if (!num_entries) |
| @@ -512,6 +536,31 @@ void __init efi_free_boot_services(void) · 34 unmodified lines | ||
| 512 | 536 | } |
| 513 | 537 | } |
| 514 | 538 | |
| 539 | +static int __init efi_free_boot_services(void) | |
| 540 | +{ | |
| 541 | + struct efi_freeable_range *range = ranges_to_free; | |
| 542 | + unsigned long freed = 0; | |
| 543 | + | |
| 544 | + if (!ranges_to_free) | |
| 545 | + return 0; | |
| 546 | + | |
| 547 | + while (range->start) { | |
| 548 | + void *start = phys_to_virt(range->start); | |
| 549 | + void *end = phys_to_virt(range->end); | |
| 550 | + | |
| 551 | + free_reserved_area(start, end, -1, NULL); | |
| 552 | + freed += (end - start); | |
| 553 | + range++; | |
| 554 | + } | |
| 555 | + kfree(ranges_to_free); | |
| 556 | + | |
| 557 | + if (freed) | |
| 558 | + pr_info("Freeing EFI boot services memory: %ldK\n", freed / SZ_1K); | |
| 559 | + | |
| 560 | + return 0; | |
| 561 | +} | |
| 562 | +arch_initcall(efi_free_boot_services); | |
| 563 | + | |
| 515 | 564 | /* |
| 516 | 565 | * A number of config table entries get remapped to virtual addresses |
| 517 | 566 | * after entering EFI virtual mode. However, the kexec kernel requires |
| @@ -25,11 +25,6 @@ struct hvm_start_info __initdata pvh_start_info; · 24 unmodified lines | ||
| 25 | 25 | |
| 26 | 26 | const unsigned int __initconst pvh_start_info_sz = sizeof(pvh_start_info); |
| 27 | 27 | |
| 28 | −static u64 __init pvh_get_root_pointer(void) | |
| 29 | −{ | |
| 30 | − return pvh_start_info.rsdp_paddr; | |
| 31 | −} | |
| 32 | − | |
| 33 | 28 | /* |
| 34 | 29 | * Xen guests are able to obtain the memory map from the hypervisor via the |
| 35 | 30 | * HYPERVISOR_memory_op hypercall. |
| @@ -95,7 +90,7 @@ static void __init init_pvh_bootparams(bool xen_guest) · 59 unmodified lines | ||
| 95 | 90 | pvh_bootparams.hdr.version = (2 << 8) | 12; |
| 96 | 91 | pvh_bootparams.hdr.type_of_loader = ((xen_guest ? 0x9 : 0xb) << 4) | 0; |
| 97 | 92 | |
| 98 | − x86_init.acpi.get_root_pointer = pvh_get_root_pointer; | |
| 93 | + pvh_bootparams.acpi_rsdp_addr = pvh_start_info.rsdp_paddr; | |
| 99 | 94 | } |
| 100 | 95 | |
| 101 | 96 | /* |
| @@ -392,7 +392,7 @@ static void __init xen_init_capabilities(void) · 391 unmodified lines | ||
| 392 | 392 | |
| 393 | 393 | /* |
| 394 | 394 | * Xen PV would need some work to support PCID: CR3 handling as well |
| 395 | − * as xen_flush_tlb_others() would need updating. | |
| 395 | + * as xen_flush_tlb_multi() would need updating. | |
| 396 | 396 | */ |
| 397 | 397 | setup_clear_cpu_cap(X86_FEATURE_PCID); |
| 398 | 398 | |
| @@ -105,6 +105,9 @@ pte_t xen_make_pte_init(pteval_t pte); · 104 unmodified lines | ||
| 105 | 105 | static pud_t level3_user_vsyscall[PTRS_PER_PUD] __page_aligned_bss; |
| 106 | 106 | #endif |
| 107 | 107 | |
| 108 | +static pud_t level3_ident_pgt[PTRS_PER_PUD] __page_aligned_bss; | |
| 109 | +static pmd_t level2_ident_pgt[PTRS_PER_PMD] __page_aligned_bss; | |
| 110 | + | |
| 108 | 111 | /* |
| 109 | 112 | * Protects atomic reservation decrease/increase against concurrent increases. |
| 110 | 113 | * Also protects non-atomic updates of current_pages and balloon lists. |
| @@ -1777,6 +1780,12 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn) · 1666 unmodified lines | ||
| 1777 | 1780 | /* Zap identity mapping */ |
| 1778 | 1781 | init_top_pgt[0] = __pgd(0); |
| 1779 | 1782 | |
| 1783 | + init_top_pgt[pgd_index(__PAGE_OFFSET_BASE_L4)].pgd = | |
| 1784 | + __pa_symbol(level3_ident_pgt) + _KERNPG_TABLE_NOENC; | |
| 1785 | + init_top_pgt[pgd_index(__START_KERNEL_map)].pgd = | |
| 1786 | + __pa_symbol(level3_kernel_pgt) + _PAGE_TABLE_NOENC; | |
| 1787 | + level3_ident_pgt[0].pud = __pa_symbol(level2_ident_pgt) + _KERNPG_TABLE_NOENC; | |
| 1788 | + | |
| 1780 | 1789 | /* Pre-constructed entries are in pfn, so convert to mfn */ |
| 1781 | 1790 | /* L4[273] -> level3_ident_pgt */ |
| 1782 | 1791 | /* L4[511] -> level3_kernel_pgt */ |
| @@ -398,8 +398,7 @@ static struct bio *bio_copy_kern(struct request *rq, void *data, unsigned int le · 397 unmodified lines | ||
| 398 | 398 | if (op_is_write(op)) |
| 399 | 399 | memcpy(page_address(page), p, bytes); |
| 400 | 400 | |
| 401 | − if (bio_add_page(bio, page, bytes, 0) < bytes) | |
| 402 | − break; | |
| 401 | + __bio_add_page(bio, page, bytes, 0); | |
| 403 | 402 | |
| 404 | 403 | len -= bytes; |
| 405 | 404 | p += bytes; |
| @@ -4793,38 +4793,45 @@ static void blk_mq_update_queue_map(struct blk_mq_tag_set *set) · 4792 unmodified lines | ||
| 4793 | 4793 | } |
| 4794 | 4794 | } |
| 4795 | 4795 | |
| 4796 | −static int blk_mq_realloc_tag_set_tags(struct blk_mq_tag_set *set, | |
| 4797 | − int new_nr_hw_queues) | |
| 4796 | +static struct blk_mq_tags **blk_mq_prealloc_tag_set_tags( | |
| 4797 | + struct blk_mq_tag_set *set, | |
| 4798 | + int new_nr_hw_queues) | |
| 4798 | 4799 | { |
| 4799 | 4800 | struct blk_mq_tags **new_tags; |
| 4800 | 4801 | int i; |
| 4801 | 4802 | |
| 4802 | 4803 | if (set->nr_hw_queues >= new_nr_hw_queues) |
| 4803 | − goto done; | |
| 4804 | + return NULL; | |
| 4804 | 4805 | |
| 4805 | 4806 | new_tags = kcalloc_node(new_nr_hw_queues, sizeof(struct blk_mq_tags *), |
| 4806 | 4807 | GFP_KERNEL, set->numa_node); |
| 4807 | 4808 | if (!new_tags) |
| 4808 | − return -ENOMEM; | |
| 4809 | + return ERR_PTR(-ENOMEM); | |
| 4809 | 4810 | |
| 4810 | 4811 | if (set->tags) |
| 4811 | 4812 | memcpy(new_tags, set->tags, set->nr_hw_queues * |
| 4812 | 4813 | sizeof(*set->tags)); |
| 4813 | − kfree(set->tags); | |
| 4814 | − set->tags = new_tags; | |
| 4815 | 4814 | |
| 4816 | 4815 | for (i = set->nr_hw_queues; i < new_nr_hw_queues; i++) { |
| 4817 | − if (!__blk_mq_alloc_map_and_rqs(set, i)) { | |
| 4818 | − while (--i >= set->nr_hw_queues) | |
| 4819 | − __blk_mq_free_map_and_rqs(set, i); | |
| 4820 | − return -ENOMEM; | |
| 4816 | + if (blk_mq_is_shared_tags(set->flags)) { | |
| 4817 | + new_tags[i] = set->shared_tags; | |
| 4818 | + } else { | |
| 4819 | + new_tags[i] = blk_mq_alloc_map_and_rqs(set, i, | |
| 4820 | + set->queue_depth); | |
| 4821 | + if (!new_tags[i]) | |
| 4822 | + goto out_unwind; | |
| 4821 | 4823 | } |
| 4822 | 4824 | cond_resched(); |
| 4823 | 4825 | } |
| 4824 | 4826 | |
| 4825 | −done: | |
| 4826 | − set->nr_hw_queues = new_nr_hw_queues; | |
| 4827 | − return 0; | |
| 4827 | + return new_tags; | |
| 4828 | +out_unwind: | |
| 4829 | + while (--i >= set->nr_hw_queues) { | |
| 4830 | + if (!blk_mq_is_shared_tags(set->flags)) | |
| 4831 | + blk_mq_free_map_and_rqs(set, new_tags[i], i); | |
| 4832 | + } | |
| 4833 | + kfree(new_tags); | |
| 4834 | + return ERR_PTR(-ENOMEM); | |
| 4828 | 4835 | } |
| 4829 | 4836 | |
| 4830 | 4837 | /* |
| @@ -5113,6 +5120,7 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, · 282 unmodified lines | ||
| 5113 | 5120 | unsigned int memflags; |
| 5114 | 5121 | int i; |
| 5115 | 5122 | struct xarray elv_tbl; |
| 5123 | + struct blk_mq_tags **new_tags; | |
| 5116 | 5124 | bool queues_frozen = false; |
| 5117 | 5125 | |
| 5118 | 5126 | lockdep_assert_held(&set->tag_list_lock); |
| @@ -5147,11 +5155,18 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, · 28 unmodified lines | ||
| 5147 | 5155 | if (blk_mq_elv_switch_none(q, &elv_tbl)) |
| 5148 | 5156 | goto switch_back; |
| 5149 | 5157 | |
| 5158 | + new_tags = blk_mq_prealloc_tag_set_tags(set, nr_hw_queues); | |
| 5159 | + if (IS_ERR(new_tags)) | |
| 5160 | + goto switch_back; | |
| 5161 | + | |
| 5150 | 5162 | list_for_each_entry(q, &set->tag_list, tag_set_list) |
| 5151 | 5163 | blk_mq_freeze_queue_nomemsave(q); |
| 5152 | 5164 | queues_frozen = true; |
| 5153 | − if (blk_mq_realloc_tag_set_tags(set, nr_hw_queues) < 0) | |
| 5154 | − goto switch_back; | |
| 5165 | + if (new_tags) { | |
| 5166 | + kfree(set->tags); | |
| 5167 | + set->tags = new_tags; | |
| 5168 | + } | |
| 5169 | + set->nr_hw_queues = nr_hw_queues; | |
| 5155 | 5170 | |
| 5156 | 5171 | fallback: |
| 5157 | 5172 | blk_mq_update_queue_map(set); |
| @@ -78,8 +78,14 @@ queue_requests_store(struct gendisk *disk, const char *page, size_t count) · 77 unmodified lines | ||
| 78 | 78 | /* |
| 79 | 79 | * Serialize updating nr_requests with concurrent queue_requests_store() |
| 80 | 80 | * and switching elevator. |
| 81 | + * | |
| 82 | + * Use trylock to avoid circular lock dependency with kernfs active | |
| 83 | + * reference during concurrent disk deletion: | |
| 84 | + * update_nr_hwq_lock -> kn->active (via del_gendisk -> kobject_del) | |
| 85 | + * kn->active -> update_nr_hwq_lock (via this sysfs write path) | |
| 81 | 86 | */ |
| 82 | − down_write(&set->update_nr_hwq_lock); | |
| 87 | + if (!down_write_trylock(&set->update_nr_hwq_lock)) | |
| 88 | + return -EBUSY; | |
| 83 | 89 | |
| 84 | 90 | if (nr == q->nr_requests) |
| 85 | 91 | goto unlock; |
| @@ -807,7 +807,16 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf, · 806 unmodified lines | ||
| 807 | 807 | elv_iosched_load_module(ctx.name); |
| 808 | 808 | ctx.type = elevator_find_get(ctx.name); |
| 809 | 809 | |
| 810 | − down_read(&set->update_nr_hwq_lock); | |
| 810 | + /* | |
| 811 | + * Use trylock to avoid circular lock dependency with kernfs active | |
| 812 | + * reference during concurrent disk deletion: | |
| 813 | + * update_nr_hwq_lock -> kn->active (via del_gendisk -> kobject_del) | |
| 814 | + * kn->active -> update_nr_hwq_lock (via this sysfs write path) | |
| 815 | + */ | |
| 816 | + if (!down_read_trylock(&set->update_nr_hwq_lock)) { | |
| 817 | + ret = -EBUSY; | |
| 818 | + goto out; | |
| 819 | + } | |
| 811 | 820 | if (!blk_queue_no_elv_switch(q)) { |
| 812 | 821 | ret = elevator_change(q, &ctx); |
| 813 | 822 | if (!ret) |
| @@ -817,6 +826,7 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf, · 3 unmodified lines | ||
| 817 | 826 | } |
| 818 | 827 | up_read(&set->update_nr_hwq_lock); |
| 819 | 828 | |
| 829 | +out: | |
| 820 | 830 | if (ctx.type) |
| 821 | 831 | elevator_put(ctx.type); |
| 822 | 832 | return ret; |
| @@ -876,8 +876,6 @@ config CRYPTO_BLAKE2B · 875 unmodified lines | ||
| 876 | 876 | - blake2b-384 |
| 877 | 877 | - blake2b-512 |
| 878 | 878 | |
| 879 | − Used by the btrfs filesystem. | |
| 880 | − | |
| 881 | 879 | See https://blake2.net for further information. |
| 882 | 880 | |
| 883 | 881 | config CRYPTO_CMAC |
| @@ -965,7 +963,6 @@ config CRYPTO_SHA256 · 81 unmodified lines | ||
| 965 | 963 | 10118-3), including HMAC support. |
| 966 | 964 | |
| 967 | 965 | This is required for IPsec AH (XFRM_AH) and IPsec ESP (XFRM_ESP). |
| 968 | − Used by the btrfs filesystem, Ceph, NFS, and SMB. | |
| 969 | 966 | |
| 970 | 967 | config CRYPTO_SHA512 |
| 971 | 968 | tristate "SHA-384 and SHA-512" |
| @@ -1039,8 +1036,6 @@ config CRYPTO_XXHASH · 67 unmodified lines | ||
| 1039 | 1036 | |
| 1040 | 1037 | Extremely fast, working at speeds close to RAM limits. |
| 1041 | 1038 | |
| 1042 | − Used by the btrfs filesystem. | |
| 1043 | − | |
| 1044 | 1039 | endmenu |
| 1045 | 1040 | |
| 1046 | 1041 | menu "CRCs (cyclic redundancy checks)" |
| @@ -1058,8 +1053,6 @@ config CRYPTO_CRC32C · 11 unmodified lines | ||
| 1058 | 1053 | on Communications, Vol. 41, No. 6, June 1993, selected for use with |
| 1059 | 1054 | iSCSI. |
| 1060 | 1055 | |
| 1061 | − Used by btrfs, ext4, jbd2, NVMeoF/TCP, and iSCSI. | |
| 1062 | − | |
| 1063 | 1056 | config CRYPTO_CRC32 |
| 1064 | 1057 | tristate "CRC32" |
| 1065 | 1058 | select CRYPTO_HASH |
| @@ -1067,8 +1060,6 @@ config CRYPTO_CRC32 · 1 unmodified line | ||
| 1067 | 1060 | help |
| 1068 | 1061 | CRC32 CRC algorithm (IEEE 802.3) |
| 1069 | 1062 | |
| 1070 | − Used by RoCEv2 and f2fs. | |
| 1071 | − | |
| 1072 | 1063 | endmenu |
| 1073 | 1064 | |
| 1074 | 1065 | menu "Compression" |
| @@ -4132,7 +4132,7 @@ static const struct alg_test_desc alg_test_descs[] = { · 4131 unmodified lines | ||
| 4132 | 4132 | .fips_allowed = 1, |
| 4133 | 4133 | }, { |
| 4134 | 4134 | .alg = "authenc(hmac(sha224),cbc(aes))", |
| 4135 | − .generic_driver = "authenc(hmac-sha224-lib,cbc(aes-generic))", | |
| 4135 | + .generic_driver = "authenc(hmac-sha224-lib,cbc(aes-lib))", | |
| 4136 | 4136 | .test = alg_test_aead, |
| 4137 | 4137 | .suite = { |
| 4138 | 4138 | .aead = __VECS(hmac_sha224_aes_cbc_tv_temp) |
| @@ -4194,7 +4194,7 @@ static const struct alg_test_desc alg_test_descs[] = { · 55 unmodified lines | ||
| 4194 | 4194 | .fips_allowed = 1, |
| 4195 | 4195 | }, { |
| 4196 | 4196 | .alg = "authenc(hmac(sha384),cbc(aes))", |
| 4197 | − .generic_driver = "authenc(hmac-sha384-lib,cbc(aes-generic))", | |
| 4197 | + .generic_driver = "authenc(hmac-sha384-lib,cbc(aes-lib))", | |
| 4198 | 4198 | .test = alg_test_aead, |
| 4199 | 4199 | .suite = { |
| 4200 | 4200 | .aead = __VECS(hmac_sha384_aes_cbc_tv_temp) |
| @@ -23,9 +23,9 @@ · 22 unmodified lines | ||
| 23 | 23 | #include "amdxdna_pci_drv.h" |
| 24 | 24 | #include "amdxdna_pm.h" |
| 25 | 25 | |
| 26 | −static bool force_cmdlist; | |
| 26 | +static bool force_cmdlist = true; | |
| 27 | 27 | module_param(force_cmdlist, bool, 0600); |
| 28 | −MODULE_PARM_DESC(force_cmdlist, "Force use command list (Default false)"); | |
| 28 | +MODULE_PARM_DESC(force_cmdlist, "Force use command list (Default true)"); | |
| 29 | 29 | |
| 30 | 30 | #define HWCTX_MAX_TIMEOUT 60000 /* milliseconds */ |
| 31 | 31 | |
| @@ -53,6 +53,7 @@ static void aie2_hwctx_stop(struct amdxdna_dev *xdna, struct amdxdna_hwctx *hwct · 21 unmodified lines | ||
| 53 | 53 | { |
| 54 | 54 | drm_sched_stop(&hwctx->priv->sched, bad_job); |
| 55 | 55 | aie2_destroy_context(xdna->dev_handle, hwctx); |
| 56 | + drm_sched_start(&hwctx->priv->sched, 0); | |
| 56 | 57 | } |
| 57 | 58 | |
| 58 | 59 | static int aie2_hwctx_restart(struct amdxdna_dev *xdna, struct amdxdna_hwctx *hwctx) |
| @@ -80,7 +81,6 @@ static int aie2_hwctx_restart(struct amdxdna_dev *xdna, struct amdxdna_hwctx *hw · 21 unmodified lines | ||
| 80 | 81 | } |
| 81 | 82 | |
| 82 | 83 | out: |
| 83 | − drm_sched_start(&hwctx->priv->sched, 0); | |
| 84 | 84 | XDNA_DBG(xdna, "%s restarted, ret %d", hwctx->name, ret); |
| 85 | 85 | return ret; |
| 86 | 86 | } |
| @@ -165,7 +165,6 @@ aie2_sched_notify(struct amdxdna_sched_job *job) · 78 unmodified lines | ||
| 165 | 165 | |
| 166 | 166 | trace_xdna_job(&job->base, job->hwctx->name, "signaled fence", job->seq); |
| 167 | 167 | |
| 168 | − amdxdna_pm_suspend_put(job->hwctx->client->xdna); | |
| 169 | 168 | job->hwctx->priv->completed++; |
| 170 | 169 | dma_fence_signal(fence); |
| 171 | 170 | |
| @@ -186,13 +185,13 @@ aie2_sched_resp_handler(void *handle, void __iomem *data, size_t size) · 14 unmodified lines | ||
| 186 | 185 | cmd_abo = job->cmd_bo; |
| 187 | 186 | |
| 188 | 187 | if (unlikely(job->job_timeout)) { |
| 189 | − amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_TIMEOUT); | |
| 188 | + amdxdna_cmd_set_error(cmd_abo, job, 0, ERT_CMD_STATE_TIMEOUT); | |
| 190 | 189 | ret = -EINVAL; |
| 191 | 190 | goto out; |
| 192 | 191 | } |
| 193 | 192 | |
| 194 | 193 | if (unlikely(!data) || unlikely(size != sizeof(u32))) { |
| 195 | − amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ABORT); | |
| 194 | + amdxdna_cmd_set_error(cmd_abo, job, 0, ERT_CMD_STATE_ABORT); | |
| 196 | 195 | ret = -EINVAL; |
| 197 | 196 | goto out; |
| 198 | 197 | } |
| @@ -202,7 +201,7 @@ aie2_sched_resp_handler(void *handle, void __iomem *data, size_t size) · 3 unmodified lines | ||
| 202 | 201 | if (status == AIE2_STATUS_SUCCESS) |
| 203 | 202 | amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_COMPLETED); |
| 204 | 203 | else |
| 205 | − amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ERROR); | |
| 204 | + amdxdna_cmd_set_error(cmd_abo, job, 0, ERT_CMD_STATE_ERROR); | |
| 206 | 205 | |
| 207 | 206 | out: |
| 208 | 207 | aie2_sched_notify(job); |
| @@ -244,13 +243,13 @@ aie2_sched_cmdlist_resp_handler(void *handle, void __iomem *data, size_t size) · 35 unmodified lines | ||
| 244 | 243 | cmd_abo = job->cmd_bo; |
| 245 | 244 | |
| 246 | 245 | if (unlikely(job->job_timeout)) { |
| 247 | − amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_TIMEOUT); | |
| 246 | + amdxdna_cmd_set_error(cmd_abo, job, 0, ERT_CMD_STATE_TIMEOUT); | |
| 248 | 247 | ret = -EINVAL; |
| 249 | 248 | goto out; |
| 250 | 249 | } |
| 251 | 250 | |
| 252 | 251 | if (unlikely(!data) || unlikely(size != sizeof(u32) * 3)) { |
| 253 | − amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ABORT); | |
| 252 | + amdxdna_cmd_set_error(cmd_abo, job, 0, ERT_CMD_STATE_ABORT); | |
| 254 | 253 | ret = -EINVAL; |
| 255 | 254 | goto out; |
| 256 | 255 | } |
| @@ -270,19 +269,12 @@ aie2_sched_cmdlist_resp_handler(void *handle, void __iomem *data, size_t size) · 13 unmodified lines | ||
| 270 | 269 | fail_cmd_idx, fail_cmd_status); |
| 271 | 270 | |
| 272 | 271 | if (fail_cmd_status == AIE2_STATUS_SUCCESS) { |
| 273 | − amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ABORT); | |
| 272 | + amdxdna_cmd_set_error(cmd_abo, job, fail_cmd_idx, ERT_CMD_STATE_ABORT); | |
| 274 | 273 | ret = -EINVAL; |
| 275 | − goto out; | |
| 274 | + } else { | |
| 275 | + amdxdna_cmd_set_error(cmd_abo, job, fail_cmd_idx, ERT_CMD_STATE_ERROR); | |
| 276 | 276 | } |
| 277 | − amdxdna_cmd_set_state(cmd_abo, ERT_CMD_STATE_ERROR); | |
| 278 | − | |
| 279 | − if (amdxdna_cmd_get_op(cmd_abo) == ERT_CMD_CHAIN) { | |
| 280 | − struct amdxdna_cmd_chain *cc = amdxdna_cmd_get_payload(cmd_abo, NULL); | |
| 281 | 277 | |
| 282 | − cc->error_index = fail_cmd_idx; | |
| 283 | − if (cc->error_index >= cc->command_count) | |
| 284 | − cc->error_index = 0; | |
| 285 | − } | |
| 286 | 278 | out: |
| 287 | 279 | aie2_sched_notify(job); |
| 288 | 280 | return ret; |
| @@ -306,10 +298,6 @@ aie2_sched_job_run(struct drm_sched_job *sched_job) · 17 unmodified lines | ||
| 306 | 298 | kref_get(&job->refcnt); |
| 307 | 299 | fence = dma_fence_get(job->fence); |
| 308 | 300 | |
| 309 | − ret = amdxdna_pm_resume_get(hwctx->client->xdna); | |
| 310 | − if (ret) | |
| 311 | − goto out; | |
| 312 | − | |
| 313 | 301 | if (job->drv_cmd) { |
| 314 | 302 | switch (job->drv_cmd->opcode) { |
| 315 | 303 | case SYNC_DEBUG_BO: |
| @@ -336,7 +324,6 @@ aie2_sched_job_run(struct drm_sched_job *sched_job) · 20 unmodified lines | ||
| 336 | 324 | |
| 337 | 325 | out: |
| 338 | 326 | if (ret) { |
| 339 | − amdxdna_pm_suspend_put(hwctx->client->xdna); | |
| 340 | 327 | dma_fence_put(job->fence); |
| 341 | 328 | aie2_job_put(job); |
| 342 | 329 | mmput(job->mm); |
| @@ -497,7 +484,7 @@ static void aie2_release_resource(struct amdxdna_hwctx *hwctx) · 154 unmodified lines | ||
| 497 | 484 | |
| 498 | 485 | if (AIE2_FEATURE_ON(xdna->dev_handle, AIE2_TEMPORAL_ONLY)) { |
| 499 | 486 | ret = aie2_destroy_context(xdna->dev_handle, hwctx); |
| 500 | − if (ret) | |
| 487 | + if (ret && ret != -ENODEV) | |
| 501 | 488 | XDNA_ERR(xdna, "Destroy temporal only context failed, ret %d", ret); |
| 502 | 489 | } else { |
| 503 | 490 | ret = xrs_release_resource(xdna->xrs_hdl, (uintptr_t)hwctx); |
| @@ -629,7 +616,7 @@ int aie2_hwctx_init(struct amdxdna_hwctx *hwctx) · 125 unmodified lines | ||
| 629 | 616 | goto free_entity; |
| 630 | 617 | } |
| 631 | 618 | |
| 632 | − ret = amdxdna_pm_resume_get(xdna); | |
| 619 | + ret = amdxdna_pm_resume_get_locked(xdna); | |
| 633 | 620 | if (ret) |
| 634 | 621 | goto free_col_list; |
| 635 | 622 | |
| @@ -760,7 +747,7 @@ static int aie2_hwctx_cu_config(struct amdxdna_hwctx *hwctx, void *buf, u32 size · 124 unmodified lines | ||
| 760 | 747 | if (!hwctx->cus) |
| 761 | 748 | return -ENOMEM; |
| 762 | 749 | |
| 763 | − ret = amdxdna_pm_resume_get(xdna); | |
| 750 | + ret = amdxdna_pm_resume_get_locked(xdna); | |
| 764 | 751 | if (ret) |
| 765 | 752 | goto free_cus; |
| 766 | 753 | |
| @@ -1070,6 +1057,8 @@ void aie2_hmm_invalidate(struct amdxdna_gem_obj *abo, · 303 unmodified lines | ||
| 1070 | 1057 | |
| 1071 | 1058 | ret = dma_resv_wait_timeout(gobj->resv, DMA_RESV_USAGE_BOOKKEEP, |
| 1072 | 1059 | true, MAX_SCHEDULE_TIMEOUT); |
| 1073 | − if (!ret || ret == -ERESTARTSYS) | |
| 1060 | + if (!ret) | |
| 1074 | 1061 | XDNA_ERR(xdna, "Failed to wait for bo, ret %ld", ret); |
| 1062 | + else if (ret == -ERESTARTSYS) | |
| 1063 | + XDNA_DBG(xdna, "Wait for bo interrupted by signal"); | |
| 1075 | 1064 | } |
| @@ -40,11 +40,8 @@ static int aie2_send_mgmt_msg_wait(struct amdxdna_dev_hdl *ndev, · 39 unmodified lines | ||
| 40 | 40 | return -ENODEV; |
| 41 | 41 | |
| 42 | 42 | ret = xdna_send_msg_wait(xdna, ndev->mgmt_chann, msg); |
| 43 | − if (ret == -ETIME) { | |
| 44 | − xdna_mailbox_stop_channel(ndev->mgmt_chann); | |
| 45 | − xdna_mailbox_destroy_channel(ndev->mgmt_chann); | |
| 46 | − ndev->mgmt_chann = NULL; | |
| 47 | − } | |
| 43 | + if (ret == -ETIME) | |
| 44 | + aie2_destroy_mgmt_chann(ndev); | |
| 48 | 45 | |
| 49 | 46 | if (!ret && *hdl->status != AIE2_STATUS_SUCCESS) { |
| 50 | 47 | XDNA_ERR(xdna, "command opcode 0x%x failed, status 0x%x", |
| @@ -216,8 +213,10 @@ static int aie2_destroy_context_req(struct amdxdna_dev_hdl *ndev, u32 id) · 165 unmodified lines | ||
| 216 | 213 | |
| 217 | 214 | req.context_id = id; |
| 218 | 215 | ret = aie2_send_mgmt_msg_wait(ndev, &msg); |
| 219 | − if (ret) | |
| 216 | + if (ret && ret != -ENODEV) | |
| 220 | 217 | XDNA_WARN(xdna, "Destroy context failed, ret %d", ret); |
| 218 | + else if (ret == -ENODEV) | |
| 219 | + XDNA_DBG(xdna, "Destroy context: device already stopped"); | |
| 221 | 220 | |
| 222 | 221 | return ret; |
| 223 | 222 | } |
| @@ -294,13 +293,20 @@ int aie2_create_context(struct amdxdna_dev_hdl *ndev, struct amdxdna_hwctx *hwct · 70 unmodified lines | ||
| 294 | 293 | } |
| 295 | 294 | |
| 296 | 295 | intr_reg = i2x.mb_head_ptr_reg + 4; |
| 297 | − hwctx->priv->mbox_chann = xdna_mailbox_create_channel(ndev->mbox, &x2i, &i2x, | |
| 298 | − intr_reg, ret); | |
| 296 | + hwctx->priv->mbox_chann = xdna_mailbox_alloc_channel(ndev->mbox); | |
| 299 | 297 | if (!hwctx->priv->mbox_chann) { |
| 300 | 298 | XDNA_ERR(xdna, "Not able to create channel"); |
| 301 | 299 | ret = -EINVAL; |
| 302 | 300 | goto del_ctx_req; |
| 303 | 301 | } |
| 302 | + | |
| 303 | + ret = xdna_mailbox_start_channel(hwctx->priv->mbox_chann, &x2i, &i2x, | |
| 304 | + intr_reg, ret); | |
| 305 | + if (ret) { | |
| 306 | + XDNA_ERR(xdna, "Not able to create channel"); | |
| 307 | + ret = -EINVAL; | |
| 308 | + goto free_channel; | |
| 309 | + } | |
| 304 | 310 | ndev->hwctx_num++; |
| 305 | 311 | |
| 306 | 312 | XDNA_DBG(xdna, "Mailbox channel irq: %d, msix_id: %d", ret, resp.msix_id); |
| @@ -308,6 +314,8 @@ int aie2_create_context(struct amdxdna_dev_hdl *ndev, struct amdxdna_hwctx *hwct · 1 unmodified line | ||
| 308 | 314 | |
| 309 | 315 | return 0; |
| 310 | 316 | |
| 317 | +free_channel: | |
| 318 | + xdna_mailbox_free_channel(hwctx->priv->mbox_chann); | |
| 311 | 319 | del_ctx_req: |
| 312 | 320 | aie2_destroy_context_req(ndev, hwctx->fw_ctx_id); |
| 313 | 321 | return ret; |
| @@ -318,9 +326,12 @@ int aie2_destroy_context(struct amdxdna_dev_hdl *ndev, struct amdxdna_hwctx *hwc · 4 unmodified lines | ||
| 318 | 326 | struct amdxdna_dev *xdna = ndev->xdna; |
| 319 | 327 | int ret; |
| 320 | 328 | |
| 329 | + if (!hwctx->priv->mbox_chann) | |
| 330 | + return 0; | |
| 331 | + | |
| 321 | 332 | xdna_mailbox_stop_channel(hwctx->priv->mbox_chann); |
| 322 | 333 | ret = aie2_destroy_context_req(ndev, hwctx->fw_ctx_id); |
| 323 | − xdna_mailbox_destroy_channel(hwctx->priv->mbox_chann); | |
| 334 | + xdna_mailbox_free_channel(hwctx->priv->mbox_chann); | |
| 324 | 335 | XDNA_DBG(xdna, "Destroyed fw ctx %d", hwctx->fw_ctx_id); |
| 325 | 336 | hwctx->priv->mbox_chann = NULL; |
| 326 | 337 | hwctx->fw_ctx_id = -1; |
| @@ -694,11 +705,11 @@ aie2_cmdlist_fill_npu_cf(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *siz · 367 unmodified lines | ||
| 694 | 705 | u32 cmd_len; |
| 695 | 706 | void *cmd; |
| 696 | 707 | |
| 697 | − memset(npu_slot, 0, sizeof(*npu_slot)); | |
| 698 | 708 | cmd = amdxdna_cmd_get_payload(cmd_bo, &cmd_len); |
| 699 | 709 | if (*size < sizeof(*npu_slot) + cmd_len) |
| 700 | 710 | return -EINVAL; |
| 701 | 711 | |
| 712 | + memset(npu_slot, 0, sizeof(*npu_slot)); | |
| 702 | 713 | npu_slot->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo); |
| 703 | 714 | if (npu_slot->cu_idx == INVALID_CU_IDX) |
| 704 | 715 | return -EINVAL; |
| @@ -719,7 +730,6 @@ aie2_cmdlist_fill_npu_dpu(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *si · 14 unmodified lines | ||
| 719 | 730 | u32 cmd_len; |
| 720 | 731 | u32 arg_sz; |
| 721 | 732 | |
| 722 | − memset(npu_slot, 0, sizeof(*npu_slot)); | |
| 723 | 733 | sn = amdxdna_cmd_get_payload(cmd_bo, &cmd_len); |
| 724 | 734 | arg_sz = cmd_len - sizeof(*sn); |
| 725 | 735 | if (cmd_len < sizeof(*sn) || arg_sz > MAX_NPU_ARGS_SIZE) |
| @@ -728,6 +738,7 @@ aie2_cmdlist_fill_npu_dpu(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *si · 2 unmodified lines | ||
| 728 | 738 | if (*size < sizeof(*npu_slot) + arg_sz) |
| 729 | 739 | return -EINVAL; |
| 730 | 740 | |
| 741 | + memset(npu_slot, 0, sizeof(*npu_slot)); | |
| 731 | 742 | npu_slot->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo); |
| 732 | 743 | if (npu_slot->cu_idx == INVALID_CU_IDX) |
| 733 | 744 | return -EINVAL; |
| @@ -751,7 +762,6 @@ aie2_cmdlist_fill_npu_preempt(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t · 17 unmodified lines | ||
| 751 | 762 | u32 cmd_len; |
| 752 | 763 | u32 arg_sz; |
| 753 | 764 | |
| 754 | − memset(npu_slot, 0, sizeof(*npu_slot)); | |
| 755 | 765 | pd = amdxdna_cmd_get_payload(cmd_bo, &cmd_len); |
| 756 | 766 | arg_sz = cmd_len - sizeof(*pd); |
| 757 | 767 | if (cmd_len < sizeof(*pd) || arg_sz > MAX_NPU_ARGS_SIZE) |
| @@ -760,6 +770,7 @@ aie2_cmdlist_fill_npu_preempt(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t · 2 unmodified lines | ||
| 760 | 770 | if (*size < sizeof(*npu_slot) + arg_sz) |
| 761 | 771 | return -EINVAL; |
| 762 | 772 | |
| 773 | + memset(npu_slot, 0, sizeof(*npu_slot)); | |
| 763 | 774 | npu_slot->cu_idx = amdxdna_cmd_get_cu_idx(cmd_bo); |
| 764 | 775 | if (npu_slot->cu_idx == INVALID_CU_IDX) |
| 765 | 776 | return -EINVAL; |
| @@ -787,7 +798,6 @@ aie2_cmdlist_fill_npu_elf(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *si · 21 unmodified lines | ||
| 787 | 798 | u32 cmd_len; |
| 788 | 799 | u32 arg_sz; |
| 789 | 800 | |
| 790 | − memset(npu_slot, 0, sizeof(*npu_slot)); | |
| 791 | 801 | pd = amdxdna_cmd_get_payload(cmd_bo, &cmd_len); |
| 792 | 802 | arg_sz = cmd_len - sizeof(*pd); |
| 793 | 803 | if (cmd_len < sizeof(*pd) || arg_sz > MAX_NPU_ARGS_SIZE) |
| @@ -796,6 +806,7 @@ aie2_cmdlist_fill_npu_elf(struct amdxdna_gem_obj *cmd_bo, void *slot, size_t *si · 2 unmodified lines | ||
| 796 | 806 | if (*size < sizeof(*npu_slot) + arg_sz) |
| 797 | 807 | return -EINVAL; |
| 798 | 808 | |
| 809 | + memset(npu_slot, 0, sizeof(*npu_slot)); | |
| 799 | 810 | npu_slot->type = EXEC_NPU_TYPE_ELF; |
| 800 | 811 | npu_slot->inst_buf_addr = pd->inst_buf; |
| 801 | 812 | npu_slot->save_buf_addr = pd->save_buf; |
| @@ -909,6 +920,20 @@ void aie2_msg_init(struct amdxdna_dev_hdl *ndev) · 107 unmodified lines | ||
| 909 | 920 | ndev->exec_msg_ops = &legacy_exec_message_ops; |
| 910 | 921 | } |
| 911 | 922 | |
| 923 | +void aie2_destroy_mgmt_chann(struct amdxdna_dev_hdl *ndev) | |
| 924 | +{ | |
| 925 | + struct amdxdna_dev *xdna = ndev->xdna; | |
| 926 | + | |
| 927 | + drm_WARN_ON(&xdna->ddev, !mutex_is_locked(&xdna->dev_lock)); | |
| 928 | + | |
| 929 | + if (!ndev->mgmt_chann) | |
| 930 | + return; | |
| 931 | + | |
| 932 | + xdna_mailbox_stop_channel(ndev->mgmt_chann); | |
| 933 | + xdna_mailbox_free_channel(ndev->mgmt_chann); | |
| 934 | + ndev->mgmt_chann = NULL; | |
| 935 | +} | |
| 936 | + | |
| 912 | 937 | static inline struct amdxdna_gem_obj * |
| 913 | 938 | aie2_cmdlist_get_cmd_buf(struct amdxdna_sched_job *job) |
| 914 | 939 | { |
| @@ -32,6 +32,11 @@ static int aie2_max_col = XRS_MAX_COL; · 31 unmodified lines | ||
| 32 | 32 | module_param(aie2_max_col, uint, 0600); |
| 33 | 33 | MODULE_PARM_DESC(aie2_max_col, "Maximum column could be used"); |
| 34 | 34 | |
| 35 | +static char *npu_fw[] = { | |
| 36 | + "npu_7.sbin", | |
| 37 | + "npu.sbin" | |
| 38 | +}; | |
| 39 | + | |
| 35 | 40 | /* |
| 36 | 41 | * The management mailbox channel is allocated by firmware. |
| 37 | 42 | * The related register and ring buffer information is on SRAM BAR. |
| @@ -323,10 +328,9 @@ static void aie2_hw_stop(struct amdxdna_dev *xdna) · 285 unmodified lines | ||
| 323 | 328 | return; |
| 324 | 329 | } |
| 325 | 330 | |
| 331 | + aie2_runtime_cfg(ndev, AIE2_RT_CFG_CLK_GATING, NULL); | |
| 326 | 332 | aie2_mgmt_fw_fini(ndev); |
| 327 | − xdna_mailbox_stop_channel(ndev->mgmt_chann); | |
| 328 | − xdna_mailbox_destroy_channel(ndev->mgmt_chann); | |
| 329 | − ndev->mgmt_chann = NULL; | |
| 333 | + aie2_destroy_mgmt_chann(ndev); | |
| 330 | 334 | drmm_kfree(&xdna->ddev, ndev->mbox); |
| 331 | 335 | ndev->mbox = NULL; |
| 332 | 336 | aie2_psp_stop(ndev->psp_hdl); |
| @@ -357,10 +361,29 @@ static int aie2_hw_start(struct amdxdna_dev *xdna) · 24 unmodified lines | ||
| 357 | 361 | } |
| 358 | 362 | pci_set_master(pdev); |
| 359 | 363 | |
| 364 | + mbox_res.ringbuf_base = ndev->sram_base; | |
| 365 | + mbox_res.ringbuf_size = pci_resource_len(pdev, xdna->dev_info->sram_bar); | |
| 366 | + mbox_res.mbox_base = ndev->mbox_base; | |
| 367 | + mbox_res.mbox_size = MBOX_SIZE(ndev); | |
| 368 | + mbox_res.name = "xdna_mailbox"; | |
| 369 | + ndev->mbox = xdnam_mailbox_create(&xdna->ddev, &mbox_res); | |
| 370 | + if (!ndev->mbox) { | |
| 371 | + XDNA_ERR(xdna, "failed to create mailbox device"); | |
| 372 | + ret = -ENODEV; | |
| 373 | + goto disable_dev; | |
| 374 | + } | |
| 375 | + | |
| 376 | + ndev->mgmt_chann = xdna_mailbox_alloc_channel(ndev->mbox); | |
| 377 | + if (!ndev->mgmt_chann) { | |
| 378 | + XDNA_ERR(xdna, "failed to alloc channel"); | |
| 379 | + ret = -ENODEV; | |
| 380 | + goto disable_dev; | |
| 381 | + } | |
| 382 | + | |
| 360 | 383 | ret = aie2_smu_init(ndev); |
| 361 | 384 | if (ret) { |
| 362 | 385 | XDNA_ERR(xdna, "failed to init smu, ret %d", ret); |
| 363 | − goto disable_dev; | |
| 386 | + goto free_channel; | |
| 364 | 387 | } |
| 365 | 388 | |
| 366 | 389 | ret = aie2_psp_start(ndev->psp_hdl); |
| @@ -375,18 +398,6 @@ static int aie2_hw_start(struct amdxdna_dev *xdna) · 8 unmodified lines | ||
| 375 | 398 | goto stop_psp; |
| 376 | 399 | } |
| 377 | 400 | |
| 378 | − mbox_res.ringbuf_base = ndev->sram_base; | |
| 379 | − mbox_res.ringbuf_size = pci_resource_len(pdev, xdna->dev_info->sram_bar); | |
| 380 | − mbox_res.mbox_base = ndev->mbox_base; | |
| 381 | − mbox_res.mbox_size = MBOX_SIZE(ndev); | |
| 382 | − mbox_res.name = "xdna_mailbox"; | |
| 383 | − ndev->mbox = xdnam_mailbox_create(&xdna->ddev, &mbox_res); | |
| 384 | − if (!ndev->mbox) { | |
| 385 | − XDNA_ERR(xdna, "failed to create mailbox device"); | |
| 386 | − ret = -ENODEV; | |
| 387 | − goto stop_psp; | |
| 388 | − } | |
| 389 | − | |
| 390 | 401 | mgmt_mb_irq = pci_irq_vector(pdev, ndev->mgmt_chan_idx); |
| 391 | 402 | if (mgmt_mb_irq < 0) { |
| 392 | 403 | ret = mgmt_mb_irq; |
| @@ -395,52 +406,55 @@ static int aie2_hw_start(struct amdxdna_dev *xdna) · 2 unmodified lines | ||
| 395 | 406 | } |
| 396 | 407 | |
| 397 | 408 | xdna_mailbox_intr_reg = ndev->mgmt_i2x.mb_head_ptr_reg + 4; |
| 398 | − ndev->mgmt_chann = xdna_mailbox_create_channel(ndev->mbox, | |
| 399 | − &ndev->mgmt_x2i, | |
| 400 | − &ndev->mgmt_i2x, | |
| 401 | − xdna_mailbox_intr_reg, | |
| 402 | − mgmt_mb_irq); | |
| 403 | − if (!ndev->mgmt_chann) { | |
| 404 | − XDNA_ERR(xdna, "failed to create management mailbox channel"); | |
| 409 | + ret = xdna_mailbox_start_channel(ndev->mgmt_chann, | |
| 410 | + &ndev->mgmt_x2i, | |
| 411 | + &ndev->mgmt_i2x, | |
| 412 | + xdna_mailbox_intr_reg, | |
| 413 | + mgmt_mb_irq); | |
| 414 | + if (ret) { | |
| 415 | + XDNA_ERR(xdna, "failed to start management mailbox channel"); | |
| 405 | 416 | ret = -EINVAL; |
| 406 | 417 | goto stop_psp; |
| 407 | 418 | } |
| 408 | 419 | |
| 409 | − ret = aie2_pm_init(ndev); | |
| 420 | + ret = aie2_mgmt_fw_init(ndev); | |
| 410 | 421 | if (ret) { |
| 411 | − XDNA_ERR(xdna, "failed to init pm, ret %d", ret); | |
| 412 | − goto destroy_mgmt_chann; | |
| 422 | + XDNA_ERR(xdna, "initial mgmt firmware failed, ret %d", ret); | |
| 423 | + goto stop_fw; | |
| 413 | 424 | } |
| 414 | 425 | |
| 415 | − ret = aie2_mgmt_fw_init(ndev); | |
| 426 | + ret = aie2_pm_init(ndev); | |
| 416 | 427 | if (ret) { |
| 417 | − XDNA_ERR(xdna, "initial mgmt firmware failed, ret %d", ret); | |
| 418 | − goto destroy_mgmt_chann; | |
| 428 | + XDNA_ERR(xdna, "failed to init pm, ret %d", ret); | |
| 429 | + goto stop_fw; | |
| 419 | 430 | } |
| 420 | 431 | |
| 421 | 432 | ret = aie2_mgmt_fw_query(ndev); |
| 422 | 433 | if (ret) { |
| 423 | 434 | XDNA_ERR(xdna, "failed to query fw, ret %d", ret); |
| 424 | − goto destroy_mgmt_chann; | |
| 435 | + goto stop_fw; | |
| 425 | 436 | } |
| 426 | 437 | |
| 427 | 438 | ret = aie2_error_async_events_alloc(ndev); |
| 428 | 439 | if (ret) { |
| 429 | 440 | XDNA_ERR(xdna, "Allocate async events failed, ret %d", ret); |
| 430 | − goto destroy_mgmt_chann; | |
| 441 | + goto stop_fw; | |
| 431 | 442 | } |
| 432 | 443 | |
| 433 | 444 | ndev->dev_status = AIE2_DEV_START; |
| 434 | 445 | |
| 435 | 446 | return 0; |
| 436 | 447 | |
| 437 | −destroy_mgmt_chann: | |
| 448 | +stop_fw: | |
| 449 | + aie2_suspend_fw(ndev); | |
| 438 | 450 | xdna_mailbox_stop_channel(ndev->mgmt_chann); |
| 439 | − xdna_mailbox_destroy_channel(ndev->mgmt_chann); | |
| 440 | 451 | stop_psp: |
| 441 | 452 | aie2_psp_stop(ndev->psp_hdl); |
| 442 | 453 | fini_smu: |
| 443 | 454 | aie2_smu_fini(ndev); |
| 455 | +free_channel: | |
| 456 | + xdna_mailbox_free_channel(ndev->mgmt_chann); | |
| 457 | + ndev->mgmt_chann = NULL; | |
| 444 | 458 | disable_dev: |
| 445 | 459 | pci_disable_device(pdev); |
| 446 | 460 | |
| @@ -451,7 +465,6 @@ static int aie2_hw_suspend(struct amdxdna_dev *xdna) · 4 unmodified lines | ||
| 451 | 465 | { |
| 452 | 466 | struct amdxdna_client *client; |
| 453 | 467 | |
| 454 | − guard(mutex)(&xdna->dev_lock); | |
| 455 | 468 | list_for_each_entry(client, &xdna->client_list, node) |
| 456 | 469 | aie2_hwctx_suspend(client); |
| 457 | 470 | |
| @@ -489,6 +502,7 @@ static int aie2_init(struct amdxdna_dev *xdna) · 31 unmodified lines | ||
| 489 | 502 | struct psp_config psp_conf; |
| 490 | 503 | const struct firmware *fw; |
| 491 | 504 | unsigned long bars = 0; |
| 505 | + char *fw_full_path; | |
| 492 | 506 | int i, nvec, ret; |
| 493 | 507 | |
| 494 | 508 | if (!hypervisor_is_type(X86_HYPER_NATIVE)) { |
| @@ -503,7 +517,19 @@ static int aie2_init(struct amdxdna_dev *xdna) · 8 unmodified lines | ||
| 503 | 517 | ndev->priv = xdna->dev_info->dev_priv; |
| 504 | 518 | ndev->xdna = xdna; |
| 505 | 519 | |
| 506 | − ret = request_firmware(&fw, ndev->priv->fw_path, &pdev->dev); | |
| 520 | + for (i = 0; i < ARRAY_SIZE(npu_fw); i++) { | |
| 521 | + fw_full_path = kasprintf(GFP_KERNEL, "%s%s", ndev->priv->fw_path, npu_fw[i]); | |
| 522 | + if (!fw_full_path) | |
| 523 | + return -ENOMEM; | |
| 524 | + | |
| 525 | + ret = firmware_request_nowarn(&fw, fw_full_path, &pdev->dev); | |
| 526 | + kfree(fw_full_path); | |
| 527 | + if (!ret) { | |
| 528 | + XDNA_INFO(xdna, "Load firmware %s%s", ndev->priv->fw_path, npu_fw[i]); | |
| 529 | + break; | |
| 530 | + } | |
| 531 | + } | |
| 532 | + | |
| 507 | 533 | if (ret) { |
| 508 | 534 | XDNA_ERR(xdna, "failed to request_firmware %s, ret %d", |
| 509 | 535 | ndev->priv->fw_path, ret); |
| @@ -951,7 +977,7 @@ static int aie2_get_info(struct amdxdna_client *client, struct amdxdna_drm_get_i · 441 unmodified lines | ||
| 951 | 977 | if (!drm_dev_enter(&xdna->ddev, &idx)) |
| 952 | 978 | return -ENODEV; |
| 953 | 979 | |
| 954 | − ret = amdxdna_pm_resume_get(xdna); | |
| 980 | + ret = amdxdna_pm_resume_get_locked(xdna); | |
| 955 | 981 | if (ret) |
| 956 | 982 | goto dev_exit; |
| 957 | 983 | |
| @@ -1044,7 +1070,7 @@ static int aie2_get_array(struct amdxdna_client *client, · 86 unmodified lines | ||
| 1044 | 1070 | if (!drm_dev_enter(&xdna->ddev, &idx)) |
| 1045 | 1071 | return -ENODEV; |
| 1046 | 1072 | |
| 1047 | − ret = amdxdna_pm_resume_get(xdna); | |
| 1073 | + ret = amdxdna_pm_resume_get_locked(xdna); | |
| 1048 | 1074 | if (ret) |
| 1049 | 1075 | goto dev_exit; |
| 1050 | 1076 | |
| @@ -1134,7 +1160,7 @@ static int aie2_set_state(struct amdxdna_client *client, · 83 unmodified lines | ||
| 1134 | 1160 | if (!drm_dev_enter(&xdna->ddev, &idx)) |
| 1135 | 1161 | return -ENODEV; |
| 1136 | 1162 | |
| 1137 | − ret = amdxdna_pm_resume_get(xdna); | |
| 1163 | + ret = amdxdna_pm_resume_get_locked(xdna); | |
| 1138 | 1164 | if (ret) |
| 1139 | 1165 | goto dev_exit; |
| 1140 | 1166 | |
| @@ -303,6 +303,7 @@ int aie2_get_array_async_error(struct amdxdna_dev_hdl *ndev, · 302 unmodified lines | ||
| 303 | 303 | |
| 304 | 304 | /* aie2_message.c */ |
| 305 | 305 | void aie2_msg_init(struct amdxdna_dev_hdl *ndev); |
| 306 | +void aie2_destroy_mgmt_chann(struct amdxdna_dev_hdl *ndev); | |
| 306 | 307 | int aie2_suspend_fw(struct amdxdna_dev_hdl *ndev); |
| 307 | 308 | int aie2_resume_fw(struct amdxdna_dev_hdl *ndev); |
| 308 | 309 | int aie2_set_runtime_cfg(struct amdxdna_dev_hdl *ndev, u32 type, u64 value); |
| @@ -31,7 +31,7 @@ int aie2_pm_set_dpm(struct amdxdna_dev_hdl *ndev, u32 dpm_level) · 30 unmodified lines | ||
| 31 | 31 | { |
| 32 | 32 | int ret; |
| 33 | 33 | |
| 34 | − ret = amdxdna_pm_resume_get(ndev->xdna); | |
| 34 | + ret = amdxdna_pm_resume_get_locked(ndev->xdna); | |
| 35 | 35 | if (ret) |
| 36 | 36 | return ret; |
| 37 | 37 | |
| @@ -17,6 +17,7 @@ · 16 unmodified lines | ||
| 17 | 17 | #include "amdxdna_ctx.h" |
| 18 | 18 | #include "amdxdna_gem.h" |
| 19 | 19 | #include "amdxdna_pci_drv.h" |
| 20 | +#include "amdxdna_pm.h" | |
| 20 | 21 | |
| 21 | 22 | #define MAX_HWCTX_ID 255 |
| 22 | 23 | #define MAX_ARG_COUNT 4095 |
| @@ -104,7 +105,10 @@ void *amdxdna_cmd_get_payload(struct amdxdna_gem_obj *abo, u32 *size) · 81 unmodified lines | ||
| 104 | 105 | |
| 105 | 106 | if (size) { |
| 106 | 107 | count = FIELD_GET(AMDXDNA_CMD_COUNT, cmd->header); |
| 107 | − if (unlikely(count <= num_masks)) { | |
| 108 | + if (unlikely(count <= num_masks || | |
| 109 | + count * sizeof(u32) + | |
| 110 | + offsetof(struct amdxdna_cmd, data[0]) > | |
| 111 | + abo->mem.size)) { | |
| 108 | 112 | *size = 0; |
| 109 | 113 | return NULL; |
| 110 | 114 | } |
| @@ -132,6 +136,33 @@ u32 amdxdna_cmd_get_cu_idx(struct amdxdna_gem_obj *abo) · 21 unmodified lines | ||
| 132 | 136 | return INVALID_CU_IDX; |
| 133 | 137 | } |
| 134 | 138 | |
| 139 | +int amdxdna_cmd_set_error(struct amdxdna_gem_obj *abo, | |
| 140 | + struct amdxdna_sched_job *job, u32 cmd_idx, | |
| 141 | + enum ert_cmd_state error_state) | |
| 142 | +{ | |
| 143 | + struct amdxdna_client *client = job->hwctx->client; | |
| 144 | + struct amdxdna_cmd *cmd = abo->mem.kva; | |
| 145 | + struct amdxdna_cmd_chain *cc = NULL; | |
| 146 | + | |
| 147 | + cmd->header &= ~AMDXDNA_CMD_STATE; | |
| 148 | + cmd->header |= FIELD_PREP(AMDXDNA_CMD_STATE, error_state); | |
| 149 | + | |
| 150 | + if (amdxdna_cmd_get_op(abo) == ERT_CMD_CHAIN) { | |
| 151 | + cc = amdxdna_cmd_get_payload(abo, NULL); | |
| 152 | + cc->error_index = (cmd_idx < cc->command_count) ? cmd_idx : 0; | |
| 153 | + abo = amdxdna_gem_get_obj(client, cc->data[0], AMDXDNA_BO_CMD); | |
| 154 | + if (!abo) | |
| 155 | + return -EINVAL; | |
| 156 | + cmd = abo->mem.kva; | |
| 157 | + } | |
| 158 | + | |
| 159 | + memset(cmd->data, 0xff, abo->mem.size - sizeof(*cmd)); | |
| 160 | + if (cc) | |
| 161 | + amdxdna_gem_put_obj(abo); | |
| 162 | + | |
| 163 | + return 0; | |
| 164 | +} | |
| 165 | + | |
| 135 | 166 | /* |
| 136 | 167 | * This should be called in close() and remove(). DO NOT call in other syscalls. |
| 137 | 168 | * This guarantee that when hwctx and resources will be released, if user |
| @@ -266,9 +297,9 @@ int amdxdna_drm_config_hwctx_ioctl(struct drm_device *dev, void *data, struct dr · 128 unmodified lines | ||
| 266 | 297 | struct amdxdna_drm_config_hwctx *args = data; |
| 267 | 298 | struct amdxdna_dev *xdna = to_xdna_dev(dev); |
| 268 | 299 | struct amdxdna_hwctx *hwctx; |
| 269 | − int ret, idx; | |
| 270 | 300 | u32 buf_size; |
| 271 | 301 | void *buf; |
| 302 | + int ret; | |
| 272 | 303 | u64 val; |
| 273 | 304 | |
| 274 | 305 | if (XDNA_MBZ_DBG(xdna, &args->pad, sizeof(args->pad))) |
| @@ -310,20 +341,17 @@ int amdxdna_drm_config_hwctx_ioctl(struct drm_device *dev, void *data, struct dr · 35 unmodified lines | ||
| 310 | 341 | return -EINVAL; |
| 311 | 342 | } |
| 312 | 343 | |
| 313 | − mutex_lock(&xdna->dev_lock); | |
| 314 | − idx = srcu_read_lock(&client->hwctx_srcu); | |
| 344 | + guard(mutex)(&xdna->dev_lock); | |
| 315 | 345 | hwctx = xa_load(&client->hwctx_xa, args->handle); |
| 316 | 346 | if (!hwctx) { |
| 317 | 347 | XDNA_DBG(xdna, "PID %d failed to get hwctx %d", client->pid, args->handle); |
| 318 | 348 | ret = -EINVAL; |
| 319 | − goto unlock_srcu; | |
| 349 | + goto free_buf; | |
| 320 | 350 | } |
| 321 | 351 | |
| 322 | 352 | ret = xdna->dev_info->ops->hwctx_config(hwctx, args->param_type, val, buf, buf_size); |
| 323 | 353 | |
| 324 | −unlock_srcu: | |
| 325 | − srcu_read_unlock(&client->hwctx_srcu, idx); | |
| 326 | − mutex_unlock(&xdna->dev_lock); | |
| 354 | +free_buf: | |
| 327 | 355 | kfree(buf); |
| 328 | 356 | return ret; |
| 329 | 357 | } |
| @@ -334,7 +362,7 @@ int amdxdna_hwctx_sync_debug_bo(struct amdxdna_client *client, u32 debug_bo_hdl) · 4 unmodified lines | ||
| 334 | 362 | struct amdxdna_hwctx *hwctx; |
| 335 | 363 | struct amdxdna_gem_obj *abo; |
| 336 | 364 | struct drm_gem_object *gobj; |
| 337 | − int ret, idx; | |
| 365 | + int ret; | |
| 338 | 366 | |
| 339 | 367 | if (!xdna->dev_info->ops->hwctx_sync_debug_bo) |
| 340 | 368 | return -EOPNOTSUPP; |
| @@ -345,17 +373,15 @@ int amdxdna_hwctx_sync_debug_bo(struct amdxdna_client *client, u32 debug_bo_hdl) · 4 unmodified lines | ||
| 345 | 373 | |
| 346 | 374 | abo = to_xdna_obj(gobj); |
| 347 | 375 | guard(mutex)(&xdna->dev_lock); |
| 348 | − idx = srcu_read_lock(&client->hwctx_srcu); | |
| 349 | 376 | hwctx = xa_load(&client->hwctx_xa, abo->assigned_hwctx); |
| 350 | 377 | if (!hwctx) { |
| 351 | 378 | ret = -EINVAL; |
| 352 | − goto unlock_srcu; | |
| 379 | + goto put_obj; | |
| 353 | 380 | } |
| 354 | 381 | |
| 355 | 382 | ret = xdna->dev_info->ops->hwctx_sync_debug_bo(hwctx, debug_bo_hdl); |
| 356 | 383 | |
| 357 | −unlock_srcu: | |
| 358 | − srcu_read_unlock(&client->hwctx_srcu, idx); | |
| 384 | +put_obj: | |
| 359 | 385 | drm_gem_object_put(gobj); |
| 360 | 386 | return ret; |
| 361 | 387 | } |
| @@ -420,6 +446,7 @@ put_shmem_bo: · 58 unmodified lines | ||
| 420 | 446 | void amdxdna_sched_job_cleanup(struct amdxdna_sched_job *job) |
| 421 | 447 | { |
| 422 | 448 | trace_amdxdna_debug_point(job->hwctx->name, job->seq, "job release"); |
| 449 | + amdxdna_pm_suspend_put(job->hwctx->client->xdna); | |
| 423 | 450 | amdxdna_arg_bos_put(job); |
| 424 | 451 | amdxdna_gem_put_obj(job->cmd_bo); |
| 425 | 452 | dma_fence_put(job->fence); |
| @@ -457,6 +484,12 @@ int amdxdna_cmd_submit(struct amdxdna_client *client, · 31 unmodified lines | ||
| 457 | 484 | goto cmd_put; |
| 458 | 485 | } |
| 459 | 486 | |
| 487 | + ret = amdxdna_pm_resume_get(xdna); | |
| 488 | + if (ret) { | |
| 489 | + XDNA_ERR(xdna, "Resume failed, ret %d", ret); | |
| 490 | + goto put_bos; | |
| 491 | + } | |
| 492 | + | |
| 460 | 493 | idx = srcu_read_lock(&client->hwctx_srcu); |
| 461 | 494 | hwctx = xa_load(&client->hwctx_xa, hwctx_hdl); |
| 462 | 495 | if (!hwctx) { |
| @@ -497,6 +530,8 @@ put_fence: · 34 unmodified lines | ||
| 497 | 530 | dma_fence_put(job->fence); |
| 498 | 531 | unlock_srcu: |
| 499 | 532 | srcu_read_unlock(&client->hwctx_srcu, idx); |
| 533 | + amdxdna_pm_suspend_put(xdna); | |
| 534 | +put_bos: | |
| 500 | 535 | amdxdna_arg_bos_put(job); |
| 501 | 536 | cmd_put: |
| 502 | 537 | amdxdna_gem_put_obj(job->cmd_bo); |
| @@ -167,6 +167,9 @@ amdxdna_cmd_get_state(struct amdxdna_gem_obj *abo) · 166 unmodified lines | ||
| 167 | 167 | |
| 168 | 168 | void *amdxdna_cmd_get_payload(struct amdxdna_gem_obj *abo, u32 *size); |
| 169 | 169 | u32 amdxdna_cmd_get_cu_idx(struct amdxdna_gem_obj *abo); |
| 170 | +int amdxdna_cmd_set_error(struct amdxdna_gem_obj *abo, | |
| 171 | + struct amdxdna_sched_job *job, u32 cmd_idx, | |
| 172 | + enum ert_cmd_state error_state); | |
| 170 | 173 | |
| 171 | 174 | void amdxdna_sched_job_cleanup(struct amdxdna_sched_job *job); |
| 172 | 175 | void amdxdna_hwctx_remove_all(struct amdxdna_client *client); |
| @@ -21,8 +21,6 @@ · 20 unmodified lines | ||
| 21 | 21 | #include "amdxdna_pci_drv.h" |
| 22 | 22 | #include "amdxdna_ubuf.h" |
| 23 | 23 | |
| 24 | −#define XDNA_MAX_CMD_BO_SIZE SZ_32K | |
| 25 | − | |
| 26 | 24 | MODULE_IMPORT_NS("DMA_BUF"); |
| 27 | 25 | |
| 28 | 26 | static int |
| @@ -745,12 +743,6 @@ amdxdna_drm_create_cmd_bo(struct drm_device *dev, · 716 unmodified lines | ||
| 745 | 743 | { |
| 746 | 744 | struct amdxdna_dev *xdna = to_xdna_dev(dev); |
| 747 | 745 | struct amdxdna_gem_obj *abo; |
| 748 | − int ret; | |
| 749 | − | |
| 750 | − if (args->size > XDNA_MAX_CMD_BO_SIZE) { | |
| 751 | − XDNA_ERR(xdna, "Command bo size 0x%llx too large", args->size); | |
| 752 | − return ERR_PTR(-EINVAL); | |
| 753 | − } | |
| 754 | 746 | |
| 755 | 747 | if (args->size < sizeof(struct amdxdna_cmd)) { |
| 756 | 748 | XDNA_DBG(xdna, "Command BO size 0x%llx too small", args->size); |
| @@ -764,17 +756,7 @@ amdxdna_drm_create_cmd_bo(struct drm_device *dev, · 7 unmodified lines | ||
| 764 | 756 | abo->type = AMDXDNA_BO_CMD; |
| 765 | 757 | abo->client = filp->driver_priv; |
| 766 | 758 | |
| 767 | − ret = amdxdna_gem_obj_vmap(abo, &abo->mem.kva); | |
| 768 | − if (ret) { | |
| 769 | − XDNA_ERR(xdna, "Vmap cmd bo failed, ret %d", ret); | |
| 770 | − goto release_obj; | |
| 771 | − } | |
| 772 | − | |
| 773 | 759 | return abo; |
| 774 | − | |
| 775 | −release_obj: | |
| 776 | − drm_gem_object_put(to_gobj(abo)); | |
| 777 | − return ERR_PTR(ret); | |
| 778 | 760 | } |
| 779 | 761 | |
| 780 | 762 | int amdxdna_drm_create_bo_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) |
| @@ -871,6 +853,7 @@ struct amdxdna_gem_obj *amdxdna_gem_get_obj(struct amdxdna_client *client, · 90 unmodified lines | ||
| 871 | 853 | struct amdxdna_dev *xdna = client->xdna; |
| 872 | 854 | struct amdxdna_gem_obj *abo; |
| 873 | 855 | struct drm_gem_object *gobj; |
| 856 | + int ret; | |
| 874 | 857 | |
| 875 | 858 | gobj = drm_gem_object_lookup(client->filp, bo_hdl); |
| 876 | 859 | if (!gobj) { |
| @@ -879,9 +862,26 @@ struct amdxdna_gem_obj *amdxdna_gem_get_obj(struct amdxdna_client *client, · 2 unmodified lines | ||
| 879 | 862 | } |
| 880 | 863 | |
| 881 | 864 | abo = to_xdna_obj(gobj); |
| 882 | − if (bo_type == AMDXDNA_BO_INVALID || abo->type == bo_type) | |
| 865 | + if (bo_type != AMDXDNA_BO_INVALID && abo->type != bo_type) | |
| 866 | + goto put_obj; | |
| 867 | + | |
| 868 | + if (bo_type != AMDXDNA_BO_CMD || abo->mem.kva) | |
| 883 | 869 | return abo; |
| 884 | 870 | |
| 871 | + if (abo->mem.size > SZ_32K) { | |
| 872 | + XDNA_ERR(xdna, "Cmd bo is too big %ld", abo->mem.size); | |
| 873 | + goto put_obj; | |
| 874 | + } | |
| 875 | + | |
| 876 | + ret = amdxdna_gem_obj_vmap(abo, &abo->mem.kva); | |
| 877 | + if (ret) { | |
| 878 | + XDNA_ERR(xdna, "Vmap cmd bo failed, ret %d", ret); | |
| 879 | + goto put_obj; | |
| 880 | + } | |
| 881 | + | |
| 882 | + return abo; | |
| 883 | + | |
| 884 | +put_obj: | |
| 885 | 885 | drm_gem_object_put(gobj); |
| 886 | 886 | return NULL; |
| 887 | 887 | } |
| @@ -460,26 +460,49 @@ msg_id_failed: · 459 unmodified lines | ||
| 460 | 460 | return ret; |
| 461 | 461 | } |
| 462 | 462 | |
| 463 | −struct mailbox_channel * | |
| 464 | −xdna_mailbox_create_channel(struct mailbox *mb, | |
| 465 | − const struct xdna_mailbox_chann_res *x2i, | |
| 466 | − const struct xdna_mailbox_chann_res *i2x, | |
| 467 | − u32 iohub_int_addr, | |
| 468 | − int mb_irq) | |
| 463 | +struct mailbox_channel *xdna_mailbox_alloc_channel(struct mailbox *mb) | |
| 469 | 464 | { |
| 470 | 465 | struct mailbox_channel *mb_chann; |
| 471 | − int ret; | |
| 472 | − | |
| 473 | − if (!is_power_of_2(x2i->rb_size) || !is_power_of_2(i2x->rb_size)) { | |
| 474 | − pr_err("Ring buf size must be power of 2"); | |
| 475 | − return NULL; | |
| 476 | − } | |
| 477 | 466 | |
| 478 | 467 | mb_chann = kzalloc_obj(*mb_chann); |
| 479 | 468 | if (!mb_chann) |
| 480 | 469 | return NULL; |
| 481 | 470 | |
| 471 | + INIT_WORK(&mb_chann->rx_work, mailbox_rx_worker); | |
| 472 | + mb_chann->work_q = create_singlethread_workqueue(MAILBOX_NAME); | |
| 473 | + if (!mb_chann->work_q) { | |
| 474 | + MB_ERR(mb_chann, "Create workqueue failed"); | |
| 475 | + goto free_chann; | |
| 476 | + } | |
| 482 | 477 | mb_chann->mb = mb; |
| 478 | + | |
| 479 | + return mb_chann; | |
| 480 | + | |
| 481 | +free_chann: | |
| 482 | + kfree(mb_chann); | |
| 483 | + return NULL; | |
| 484 | +} | |
| 485 | + | |
| 486 | +void xdna_mailbox_free_channel(struct mailbox_channel *mb_chann) | |
| 487 | +{ | |
| 488 | + destroy_workqueue(mb_chann->work_q); | |
| 489 | + kfree(mb_chann); | |
| 490 | +} | |
| 491 | + | |
| 492 | +int | |
| 493 | +xdna_mailbox_start_channel(struct mailbox_channel *mb_chann, | |
| 494 | + const struct xdna_mailbox_chann_res *x2i, | |
| 495 | + const struct xdna_mailbox_chann_res *i2x, | |
| 496 | + u32 iohub_int_addr, | |
| 497 | + int mb_irq) | |
| 498 | +{ | |
| 499 | + int ret; | |
| 500 | + | |
| 501 | + if (!is_power_of_2(x2i->rb_size) || !is_power_of_2(i2x->rb_size)) { | |
| 502 | + pr_err("Ring buf size must be power of 2"); | |
| 503 | + return -EINVAL; | |
| 504 | + } | |
| 505 | + | |
| 483 | 506 | mb_chann->msix_irq = mb_irq; |
| 484 | 507 | mb_chann->iohub_int_addr = iohub_int_addr; |
| 485 | 508 | memcpy(&mb_chann->res[CHAN_RES_X2I], x2i, sizeof(*x2i)); |
| @@ -489,61 +512,37 @@ xdna_mailbox_create_channel(struct mailbox *mb, · 3 unmodified lines | ||
| 489 | 512 | mb_chann->x2i_tail = mailbox_get_tailptr(mb_chann, CHAN_RES_X2I); |
| 490 | 513 | mb_chann->i2x_head = mailbox_get_headptr(mb_chann, CHAN_RES_I2X); |
| 491 | 514 | |
| 492 | − INIT_WORK(&mb_chann->rx_work, mailbox_rx_worker); | |
| 493 | − mb_chann->work_q = create_singlethread_workqueue(MAILBOX_NAME); | |
| 494 | − if (!mb_chann->work_q) { | |
| 495 | − MB_ERR(mb_chann, "Create workqueue failed"); | |
| 496 | − goto free_and_out; | |
| 497 | − } | |
| 498 | − | |
| 499 | 515 | /* Everything look good. Time to enable irq handler */ |
| 500 | 516 | ret = request_irq(mb_irq, mailbox_irq_handler, 0, MAILBOX_NAME, mb_chann); |
| 501 | 517 | if (ret) { |
| 502 | 518 | MB_ERR(mb_chann, "Failed to request irq %d ret %d", mb_irq, ret); |
| 503 | − goto destroy_wq; | |
| 519 | + return ret; | |
| 504 | 520 | } |
| 505 | 521 | |
| 506 | 522 | mb_chann->bad_state = false; |
| 507 | 523 | mailbox_reg_write(mb_chann, mb_chann->iohub_int_addr, 0); |
| 508 | 524 | |
| 509 | − MB_DBG(mb_chann, "Mailbox channel created (irq: %d)", mb_chann->msix_irq); | |
| 510 | − return mb_chann; | |
| 511 | − | |
| 512 | −destroy_wq: | |
| 513 | − destroy_workqueue(mb_chann->work_q); | |
| 514 | −free_and_out: | |
| 515 | − kfree(mb_chann); | |
| 516 | − return NULL; | |
| 525 | + MB_DBG(mb_chann, "Mailbox channel started (irq: %d)", mb_chann->msix_irq); | |
| 526 | + return 0; | |
| 517 | 527 | } |
| 518 | 528 | |
| 519 | −int xdna_mailbox_destroy_channel(struct mailbox_channel *mb_chann) | |
| 529 | +void xdna_mailbox_stop_channel(struct mailbox_channel *mb_chann) | |
| 520 | 530 | { |
| 521 | 531 | struct mailbox_msg *mb_msg; |
| 522 | 532 | unsigned long msg_id; |
| 523 | 533 | |
| 524 | − MB_DBG(mb_chann, "IRQ disabled and RX work cancelled"); | |
| 534 | + /* Disable an irq and wait. This might sleep. */ | |
| 525 | 535 | free_irq(mb_chann->msix_irq, mb_chann); |
| 526 | − destroy_workqueue(mb_chann->work_q); | |
| 527 | − /* We can clean up and release resources */ | |
| 528 | 536 | |
| 537 | + /* Cancel RX work and wait for it to finish */ | |
| 538 | + drain_workqueue(mb_chann->work_q); | |
| 539 | + | |
| 540 | + /* We can clean up and release resources */ | |
| 529 | 541 | xa_for_each(&mb_chann->chan_xa, msg_id, mb_msg) |
| 530 | 542 | mailbox_release_msg(mb_chann, mb_msg); |
| 531 | − | |
| 532 | 543 | xa_destroy(&mb_chann->chan_xa); |
| 533 | 544 | |
| 534 | − MB_DBG(mb_chann, "Mailbox channel destroyed, irq: %d", mb_chann->msix_irq); | |
| 535 | − kfree(mb_chann); | |
| 536 | − return 0; | |
| 537 | −} | |
| 538 | − | |
| 539 | −void xdna_mailbox_stop_channel(struct mailbox_channel *mb_chann) | |
| 540 | −{ | |
| 541 | − /* Disable an irq and wait. This might sleep. */ | |
| 542 | − disable_irq(mb_chann->msix_irq); | |
| 543 | − | |
| 544 | − /* Cancel RX work and wait for it to finish */ | |
| 545 | − cancel_work_sync(&mb_chann->rx_work); | |
| 546 | − MB_DBG(mb_chann, "IRQ disabled and RX work cancelled"); | |
| 545 | + MB_DBG(mb_chann, "Mailbox channel stopped, irq: %d", mb_chann->msix_irq); | |
| 547 | 546 | } |
| 548 | 547 | |
| 549 | 548 | struct mailbox *xdnam_mailbox_create(struct drm_device *ddev, |
| @@ -74,9 +74,16 @@ struct mailbox *xdnam_mailbox_create(struct drm_device *ddev, · 73 unmodified lines | ||
| 74 | 74 | const struct xdna_mailbox_res *res); |
| 75 | 75 | |
| 76 | 76 | /* |
| 77 | − * xdna_mailbox_create_channel() -- Create a mailbox channel instance | |
| 77 | + * xdna_mailbox_alloc_channel() -- alloc a mailbox channel | |
| 78 | 78 | * |
| 79 | − * @mailbox: the handle return from xdna_mailbox_create() | |
| 79 | + * @mb: mailbox handle | |
| 80 | + */ | |
| 81 | +struct mailbox_channel *xdna_mailbox_alloc_channel(struct mailbox *mb); | |
| 82 | + | |
| 83 | +/* | |
| 84 | + * xdna_mailbox_start_channel() -- start a mailbox channel instance | |
| 85 | + * | |
| 86 | + * @mb_chann: the handle return from xdna_mailbox_alloc_channel() | |
| 80 | 87 | * @x2i: host to firmware mailbox resources |
| 81 | 88 | * @i2x: firmware to host mailbox resources |
| 82 | 89 | * @xdna_mailbox_intr_reg: register addr of MSI-X interrupt |
| @@ -84,28 +91,24 @@ struct mailbox *xdnam_mailbox_create(struct drm_device *ddev, · 1 unmodified line | ||
| 84 | 91 | * |
| 85 | 92 | * Return: If success, return a handle of mailbox channel. Otherwise, return NULL. |
| 86 | 93 | */ |
| 87 | −struct mailbox_channel * | |
| 88 | −xdna_mailbox_create_channel(struct mailbox *mailbox, | |
| 89 | − const struct xdna_mailbox_chann_res *x2i, | |
| 90 | − const struct xdna_mailbox_chann_res *i2x, | |
| 91 | − u32 xdna_mailbox_intr_reg, | |
| 92 | − int mb_irq); | |
| 94 | +int | |
| 95 | +xdna_mailbox_start_channel(struct mailbox_channel *mb_chann, | |
| 96 | + const struct xdna_mailbox_chann_res *x2i, | |
| 97 | + const struct xdna_mailbox_chann_res *i2x, | |
| 98 | + u32 xdna_mailbox_intr_reg, | |
| 99 | + int mb_irq); | |
| 93 | 100 | |
| 94 | 101 | /* |
| 95 | − * xdna_mailbox_destroy_channel() -- destroy mailbox channel | |
| 102 | + * xdna_mailbox_free_channel() -- free mailbox channel | |
| 96 | 103 | * |
| 97 | 104 | * @mailbox_chann: the handle return from xdna_mailbox_create_channel() |
| 98 | − * | |
| 99 | − * Return: if success, return 0. otherwise return error code | |
| 100 | 105 | */ |
| 101 | −int xdna_mailbox_destroy_channel(struct mailbox_channel *mailbox_chann); | |
| 106 | +void xdna_mailbox_free_channel(struct mailbox_channel *mailbox_chann); | |
| 102 | 107 | |
| 103 | 108 | /* |
| 104 | 109 | * xdna_mailbox_stop_channel() -- stop mailbox channel |
| 105 | 110 | * |
| 106 | 111 | * @mailbox_chann: the handle return from xdna_mailbox_create_channel() |
| 107 | − * | |
| 108 | − * Return: if success, return 0. otherwise return error code | |
| 109 | 112 | */ |
| 110 | 113 | void xdna_mailbox_stop_channel(struct mailbox_channel *mailbox_chann); |
| 111 | 114 | |
| @@ -23,6 +23,9 @@ MODULE_FIRMWARE("amdnpu/1502_00/npu.sbin"); · 22 unmodified lines | ||
| 23 | 23 | MODULE_FIRMWARE("amdnpu/17f0_10/npu.sbin"); |
| 24 | 24 | MODULE_FIRMWARE("amdnpu/17f0_11/npu.sbin"); |
| 25 | 25 | MODULE_FIRMWARE("amdnpu/17f0_20/npu.sbin"); |
| 26 | +MODULE_FIRMWARE("amdnpu/1502_00/npu_7.sbin"); | |
| 27 | +MODULE_FIRMWARE("amdnpu/17f0_10/npu_7.sbin"); | |
| 28 | +MODULE_FIRMWARE("amdnpu/17f0_11/npu_7.sbin"); | |
| 26 | 29 | |
| 27 | 30 | /* |
| 28 | 31 | * 0.0: Initial version |
| @@ -16,6 +16,7 @@ int amdxdna_pm_suspend(struct device *dev) · 15 unmodified lines | ||
| 16 | 16 | struct amdxdna_dev *xdna = to_xdna_dev(dev_get_drvdata(dev)); |
| 17 | 17 | int ret = -EOPNOTSUPP; |
| 18 | 18 | |
| 19 | + guard(mutex)(&xdna->dev_lock); | |
| 19 | 20 | if (xdna->dev_info->ops->suspend) |
| 20 | 21 | ret = xdna->dev_info->ops->suspend(xdna); |
| 21 | 22 | |
| @@ -28,6 +29,7 @@ int amdxdna_pm_resume(struct device *dev) · 6 unmodified lines | ||
| 28 | 29 | struct amdxdna_dev *xdna = to_xdna_dev(dev_get_drvdata(dev)); |
| 29 | 30 | int ret = -EOPNOTSUPP; |
| 30 | 31 | |
| 32 | + guard(mutex)(&xdna->dev_lock); | |
| 31 | 33 | if (xdna->dev_info->ops->resume) |
| 32 | 34 | ret = xdna->dev_info->ops->resume(xdna); |
| 33 | 35 | |
| @@ -15,4 +15,15 @@ void amdxdna_pm_suspend_put(struct amdxdna_dev *xdna); · 14 unmodified lines | ||
| 15 | 15 | void amdxdna_pm_init(struct amdxdna_dev *xdna); |
| 16 | 16 | void amdxdna_pm_fini(struct amdxdna_dev *xdna); |
| 17 | 17 | |
| 18 | +static inline int amdxdna_pm_resume_get_locked(struct amdxdna_dev *xdna) | |
| 19 | +{ | |
| 20 | + int ret; | |
| 21 | + | |
| 22 | + mutex_unlock(&xdna->dev_lock); | |
| 23 | + ret = amdxdna_pm_resume_get(xdna); | |
| 24 | + mutex_lock(&xdna->dev_lock); | |
| 25 | + | |
| 26 | + return ret; | |
| 27 | +} | |
| 28 | + | |
| 18 | 29 | #endif /* _AMDXDNA_PM_H_ */ |
| @@ -7,6 +7,7 @@ · 6 unmodified lines | ||
| 7 | 7 | #include <drm/drm_device.h> |
| 8 | 8 | #include <drm/drm_print.h> |
| 9 | 9 | #include <linux/dma-buf.h> |
| 10 | +#include <linux/overflow.h> | |
| 10 | 11 | #include <linux/pagemap.h> |
| 11 | 12 | #include <linux/vmalloc.h> |
| 12 | 13 | |
| @@ -176,7 +177,10 @@ struct dma_buf *amdxdna_get_ubuf(struct drm_device *dev, · 163 unmodified lines | ||
| 176 | 177 | goto free_ent; |
| 177 | 178 | } |
| 178 | 179 | |
| 179 | − exp_info.size += va_ent[i].len; | |
| 180 | + if (check_add_overflow(exp_info.size, va_ent[i].len, &exp_info.size)) { | |
| 181 | + ret = -EINVAL; | |
| 182 | + goto free_ent; | |
| 183 | + } | |
| 180 | 184 | } |
| 181 | 185 | |
| 182 | 186 | ubuf->nr_pages = exp_info.size >> PAGE_SHIFT; |
| @@ -67,12 +67,12 @@ const struct dpm_clk_freq npu1_dpm_clk_table[] = { · 66 unmodified lines | ||
| 67 | 67 | |
| 68 | 68 | static const struct aie2_fw_feature_tbl npu1_fw_feature_table[] = { |
| 69 | 69 | { .major = 5, .min_minor = 7 }, |
| 70 | − { .features = BIT_U64(AIE2_NPU_COMMAND), .min_minor = 8 }, | |
| 70 | + { .features = BIT_U64(AIE2_NPU_COMMAND), .major = 5, .min_minor = 8 }, | |
| 71 | 71 | { 0 } |
| 72 | 72 | }; |
| 73 | 73 | |
| 74 | 74 | static const struct amdxdna_dev_priv npu1_dev_priv = { |
| 75 | − .fw_path = "amdnpu/1502_00/npu.sbin", | |
| 75 | + .fw_path = "amdnpu/1502_00/", | |
| 76 | 76 | .rt_config = npu1_default_rt_cfg, |
| 77 | 77 | .dpm_clk_tbl = npu1_dpm_clk_table, |
| 78 | 78 | .fw_feature_tbl = npu1_fw_feature_table, |
| @@ -98,7 +98,7 @@ const struct aie2_fw_feature_tbl npu4_fw_feature_table[] = { · 97 unmodified lines | ||
| 98 | 98 | }; |
| 99 | 99 | |
| 100 | 100 | static const struct amdxdna_dev_priv npu4_dev_priv = { |
| 101 | − .fw_path = "amdnpu/17f0_10/npu.sbin", | |
| 101 | + .fw_path = "amdnpu/17f0_10/", | |
| 102 | 102 | .rt_config = npu4_default_rt_cfg, |
| 103 | 103 | .dpm_clk_tbl = npu4_dpm_clk_table, |
| 104 | 104 | .fw_feature_tbl = npu4_fw_feature_table, |
| @@ -63,7 +63,7 @@ · 62 unmodified lines | ||
| 63 | 63 | #define NPU5_SRAM_BAR_BASE MMNPU_APERTURE1_BASE |
| 64 | 64 | |
| 65 | 65 | static const struct amdxdna_dev_priv npu5_dev_priv = { |
| 66 | − .fw_path = "amdnpu/17f0_11/npu.sbin", | |
| 66 | + .fw_path = "amdnpu/17f0_11/", | |
| 67 | 67 | .rt_config = npu4_default_rt_cfg, |
| 68 | 68 | .dpm_clk_tbl = npu4_dpm_clk_table, |
| 69 | 69 | .fw_feature_tbl = npu4_fw_feature_table, |
| @@ -63,7 +63,7 @@ · 62 unmodified lines | ||
| 63 | 63 | #define NPU6_SRAM_BAR_BASE MMNPU_APERTURE1_BASE |
| 64 | 64 | |
| 65 | 65 | static const struct amdxdna_dev_priv npu6_dev_priv = { |
| 66 | − .fw_path = "amdnpu/17f0_10/npu.sbin", | |
| 66 | + .fw_path = "amdnpu/17f0_10/", | |
| 67 | 67 | .rt_config = npu4_default_rt_cfg, |
| 68 | 68 | .dpm_clk_tbl = npu4_dpm_clk_table, |
| 69 | 69 | .fw_feature_tbl = npu4_fw_feature_table, |
| @@ -154,7 +154,7 @@ static void cmd_state_init(struct cmd_state *st) · 153 unmodified lines | ||
| 154 | 154 | |
| 155 | 155 | static u64 cmd_to_addr(u32 *cmd) |
| 156 | 156 | { |
| 157 | − return ((u64)((cmd[0] & 0xff0000) << 16)) | cmd[1]; | |
| 157 | + return (((u64)cmd[0] & 0xff0000) << 16) | cmd[1]; | |
| 158 | 158 | } |
| 159 | 159 | |
| 160 | 160 | static u64 dma_length(struct ethosu_validated_cmdstream_info *info, |
| @@ -245,11 +245,14 @@ static int calc_sizes(struct drm_device *ddev, · 84 unmodified lines | ||
| 245 | 245 | ((st->ifm.stride_kernel >> 1) & 0x1) + 1; |
| 246 | 246 | u32 stride_x = ((st->ifm.stride_kernel >> 5) & 0x2) + |
| 247 | 247 | (st->ifm.stride_kernel & 0x1) + 1; |
| 248 | − u32 ifm_height = st->ofm.height[2] * stride_y + | |
| 248 | + s32 ifm_height = st->ofm.height[2] * stride_y + | |
| 249 | 249 | st->ifm.height[2] - (st->ifm.pad_top + st->ifm.pad_bottom); |
| 250 | − u32 ifm_width = st->ofm.width * stride_x + | |
| 250 | + s32 ifm_width = st->ofm.width * stride_x + | |
| 251 | 251 | st->ifm.width - (st->ifm.pad_left + st->ifm.pad_right); |
| 252 | 252 | |
| 253 | + if (ifm_height < 0 || ifm_width < 0) | |
| 254 | + return -EINVAL; | |
| 255 | + | |
| 253 | 256 | len = feat_matrix_length(info, &st->ifm, ifm_width, |
| 254 | 257 | ifm_height, st->ifm.depth); |
| 255 | 258 | dev_dbg(ddev->dev, "op %d: IFM:%d:0x%llx-0x%llx\n", |
| @@ -417,7 +420,10 @@ static int ethosu_gem_cmdstream_copy_and_validate(struct drm_device *ddev, · 161 unmodified lines | ||
| 417 | 420 | return ret; |
| 418 | 421 | break; |
| 419 | 422 | case NPU_OP_ELEMENTWISE: |
| 420 | − use_ifm2 = !((st.ifm2.broadcast == 8) || (param == 5) || | |
| 423 | + use_scale = ethosu_is_u65(edev) ? | |
| 424 | + (st.ifm2.broadcast & 0x80) : | |
| 425 | + (st.ifm2.broadcast == 8); | |
| 426 | + use_ifm2 = !(use_scale || (param == 5) || | |
| 421 | 427 | (param == 6) || (param == 7) || (param == 0x24)); |
| 422 | 428 | use_ifm = st.ifm.broadcast != 8; |
| 423 | 429 | ret = calc_sizes_elemwise(ddev, info, cmd, &st, use_ifm, use_ifm2); |
| @@ -143,23 +143,29 @@ out: · 142 unmodified lines | ||
| 143 | 143 | return ret; |
| 144 | 144 | } |
| 145 | 145 | |
| 146 | +static void ethosu_job_err_cleanup(struct ethosu_job *job) | |
| 147 | +{ | |
| 148 | + unsigned int i; | |
| 149 | + | |
| 150 | + for (i = 0; i < job->region_cnt; i++) | |
| 151 | + drm_gem_object_put(job->region_bo[i]); | |
| 152 | + | |
| 153 | + drm_gem_object_put(job->cmd_bo); | |
| 154 | + | |
| 155 | + kfree(job); | |
| 156 | +} | |
| 157 | + | |
| 146 | 158 | static void ethosu_job_cleanup(struct kref *ref) |
| 147 | 159 | { |
| 148 | 160 | struct ethosu_job *job = container_of(ref, struct ethosu_job, |
| 149 | 161 | refcount); |
| 150 | − unsigned int i; | |
| 151 | 162 | |
| 152 | 163 | pm_runtime_put_autosuspend(job->dev->base.dev); |
| 153 | 164 | |
| 154 | 165 | dma_fence_put(job->done_fence); |
| 155 | 166 | dma_fence_put(job->inference_done_fence); |
| 156 | 167 | |
| 157 | − for (i = 0; i < job->region_cnt; i++) | |
| 158 | − drm_gem_object_put(job->region_bo[i]); | |
| 159 | − | |
| 160 | − drm_gem_object_put(job->cmd_bo); | |
| 161 | − | |
| 162 | − kfree(job); | |
| 168 | + ethosu_job_err_cleanup(job); | |
| 163 | 169 | } |
| 164 | 170 | |
| 165 | 171 | static void ethosu_job_put(struct ethosu_job *job) |
| @@ -454,12 +460,16 @@ static int ethosu_ioctl_submit_job(struct drm_device *dev, struct drm_file *file · 288 unmodified lines | ||
| 454 | 460 | } |
| 455 | 461 | } |
| 456 | 462 | ret = ethosu_job_push(ejob); |
| 463 | + if (!ret) { | |
| 464 | + ethosu_job_put(ejob); | |
| 465 | + return 0; | |
| 466 | + } | |
| 457 | 467 | |
| 458 | 468 | out_cleanup_job: |
| 459 | 469 | if (ret) |
| 460 | 470 | drm_sched_job_cleanup(&ejob->base); |
| 461 | 471 | out_put_job: |
| 462 | − ethosu_job_put(ejob); | |
| 472 | + ethosu_job_err_cleanup(ejob); | |
| 463 | 473 | |
| 464 | 474 | return ret; |
| 465 | 475 | } |
| @@ -121,12 +121,6 @@ · 120 unmodified lines | ||
| 121 | 121 | #define VPU_50XX_HOST_SS_AON_PWR_ISLAND_STATUS_DLY 0x0003006cu |
| 122 | 122 | #define VPU_50XX_HOST_SS_AON_PWR_ISLAND_STATUS_DLY_STATUS_DLY_MASK GENMASK(7, 0) |
| 123 | 123 | |
| 124 | −#define VPU_40XX_HOST_SS_AON_RETENTION0 0x0003000cu | |
| 125 | −#define VPU_40XX_HOST_SS_AON_RETENTION1 0x00030010u | |
| 126 | −#define VPU_40XX_HOST_SS_AON_RETENTION2 0x00030014u | |
| 127 | −#define VPU_40XX_HOST_SS_AON_RETENTION3 0x00030018u | |
| 128 | −#define VPU_40XX_HOST_SS_AON_RETENTION4 0x0003001cu | |
| 129 | − | |
| 130 | 124 | #define VPU_40XX_HOST_SS_AON_IDLE_GEN 0x00030200u |
| 131 | 125 | #define VPU_40XX_HOST_SS_AON_IDLE_GEN_EN_MASK BIT_MASK(0) |
| 132 | 126 | #define VPU_40XX_HOST_SS_AON_IDLE_GEN_HW_PG_EN_MASK BIT_MASK(1) |
| @@ -931,7 +931,6 @@ static int soc_cpu_boot_40xx(struct ivpu_device *vdev) · 930 unmodified lines | ||
| 931 | 931 | |
| 932 | 932 | static int soc_cpu_boot_60xx(struct ivpu_device *vdev) |
| 933 | 933 | { |
| 934 | − REGV_WR64(VPU_40XX_HOST_SS_AON_RETENTION1, vdev->fw->mem_bp->vpu_addr); | |
| 935 | 934 | soc_cpu_set_entry_point_40xx(vdev, vdev->fw->cold_boot_entry_point); |
| 936 | 935 | |
| 937 | 936 | return 0; |
| @@ -9,6 +9,7 @@ config ARCH_SUPPORTS_ACPI · 8 unmodified lines | ||
| 9 | 9 | menuconfig ACPI |
| 10 | 10 | bool "ACPI (Advanced Configuration and Power Interface) Support" |
| 11 | 11 | depends on ARCH_SUPPORTS_ACPI |
| 12 | + select AUXILIARY_BUS | |
| 12 | 13 | select PNP |
| 13 | 14 | select NLS |
| 14 | 15 | select CRC32 |
| @@ -135,7 +135,7 @@ struct platform_device *acpi_create_platform_device(struct acpi_device *adev, · 134 unmodified lines | ||
| 135 | 135 | } |
| 136 | 136 | } |
| 137 | 137 | |
| 138 | − if (adev->device_type == ACPI_BUS_TYPE_DEVICE && !adev->pnp.type.backlight) { | |
| 138 | + if (adev->device_type == ACPI_BUS_TYPE_DEVICE) { | |
| 139 | 139 | LIST_HEAD(resource_list); |
| 140 | 140 | |
| 141 | 141 | count = acpi_dev_get_resources(adev, &resource_list, NULL, NULL); |
| @@ -9,6 +9,7 @@ · 8 unmodified lines | ||
| 9 | 9 | |
| 10 | 10 | #define pr_fmt(fmt) "ACPI: video: " fmt |
| 11 | 11 | |
| 12 | +#include <linux/auxiliary_bus.h> | |
| 12 | 13 | #include <linux/kernel.h> |
| 13 | 14 | #include <linux/module.h> |
| 14 | 15 | #include <linux/init.h> |
| @@ -21,7 +22,6 @@ · 6 unmodified lines | ||
| 21 | 22 | #include <linux/sort.h> |
| 22 | 23 | #include <linux/pci.h> |
| 23 | 24 | #include <linux/pci_ids.h> |
| 24 | −#include <linux/platform_device.h> | |
| 25 | 25 | #include <linux/slab.h> |
| 26 | 26 | #include <linux/dmi.h> |
| 27 | 27 | #include <linux/suspend.h> |
| @@ -77,8 +77,9 @@ static int register_count; · 49 unmodified lines | ||
| 77 | 77 | static DEFINE_MUTEX(register_count_mutex); |
| 78 | 78 | static DEFINE_MUTEX(video_list_lock); |
| 79 | 79 | static LIST_HEAD(video_bus_head); |
| 80 | −static int acpi_video_bus_probe(struct platform_device *pdev); | |
| 81 | −static void acpi_video_bus_remove(struct platform_device *pdev); | |
| 80 | +static int acpi_video_bus_probe(struct auxiliary_device *aux_dev, | |
| 81 | + const struct auxiliary_device_id *id); | |
| 82 | +static void acpi_video_bus_remove(struct auxiliary_device *aux); | |
| 82 | 83 | static void acpi_video_bus_notify(acpi_handle handle, u32 event, void *data); |
| 83 | 84 | |
| 84 | 85 | /* |
| @@ -93,19 +94,16 @@ enum acpi_video_level_idx { · 8 unmodified lines | ||
| 93 | 94 | ACPI_VIDEO_FIRST_LEVEL, /* actual supported levels begin here */ |
| 94 | 95 | }; |
| 95 | 96 | |
| 96 | −static const struct acpi_device_id video_device_ids[] = { | |
| 97 | − {ACPI_VIDEO_HID, 0}, | |
| 98 | − {"", 0}, | |
| 97 | +static const struct auxiliary_device_id video_bus_auxiliary_id_table[] = { | |
| 98 | + { .name = "acpi.video_bus" }, | |
| 99 | + {}, | |
| 99 | 100 | }; |
| 100 | −MODULE_DEVICE_TABLE(acpi, video_device_ids); | |
| 101 | +MODULE_DEVICE_TABLE(auxiliary, video_bus_auxiliary_id_table); | |
| 101 | 102 | |
| 102 | −static struct platform_driver acpi_video_bus = { | |
| 103 | +static struct auxiliary_driver acpi_video_bus = { | |
| 103 | 104 | .probe = acpi_video_bus_probe, |
| 104 | 105 | .remove = acpi_video_bus_remove, |
| 105 | − .driver = { | |
| 106 | − .name = "acpi-video", | |
| 107 | − .acpi_match_table = video_device_ids, | |
| 108 | − }, | |
| 106 | + .id_table = video_bus_auxiliary_id_table, | |
| 109 | 107 | }; |
| 110 | 108 | |
| 111 | 109 | struct acpi_video_bus_flags { |
| @@ -1885,7 +1883,7 @@ static void acpi_video_dev_add_notify_handler(struct acpi_video_device *device) · 1773 unmodified lines | ||
| 1885 | 1883 | } |
| 1886 | 1884 | |
| 1887 | 1885 | static int acpi_video_bus_add_notify_handler(struct acpi_video_bus *video, |
| 1888 | − struct platform_device *pdev) | |
| 1886 | + struct device *parent) | |
| 1889 | 1887 | { |
| 1890 | 1888 | struct input_dev *input; |
| 1891 | 1889 | struct acpi_video_device *dev; |
| @@ -1908,7 +1906,7 @@ static int acpi_video_bus_add_notify_handler(struct acpi_video_bus *video, · 16 unmodified lines | ||
| 1908 | 1906 | input->phys = video->phys; |
| 1909 | 1907 | input->id.bustype = BUS_HOST; |
| 1910 | 1908 | input->id.product = 0x06; |
| 1911 | − input->dev.parent = &pdev->dev; | |
| 1909 | + input->dev.parent = parent; | |
| 1912 | 1910 | input->evbit[0] = BIT(EV_KEY); |
| 1913 | 1911 | set_bit(KEY_SWITCHVIDEOMODE, input->keybit); |
| 1914 | 1912 | set_bit(KEY_VIDEO_NEXT, input->keybit); |
| @@ -1980,9 +1978,10 @@ static int acpi_video_bus_put_devices(struct acpi_video_bus *video) · 65 unmodified lines | ||
| 1980 | 1978 | |
| 1981 | 1979 | static int instance; |
| 1982 | 1980 | |
| 1983 | −static int acpi_video_bus_probe(struct platform_device *pdev) | |
| 1981 | +static int acpi_video_bus_probe(struct auxiliary_device *aux_dev, | |
| 1982 | + const struct auxiliary_device_id *id_unused) | |
| 1984 | 1983 | { |
| 1985 | − struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | |
| 1984 | + struct acpi_device *device = ACPI_COMPANION(&aux_dev->dev); | |
| 1986 | 1985 | struct acpi_video_bus *video; |
| 1987 | 1986 | bool auto_detect; |
| 1988 | 1987 | int error; |
| @@ -2019,7 +2018,7 @@ static int acpi_video_bus_probe(struct platform_device *pdev) · 30 unmodified lines | ||
| 2019 | 2018 | instance++; |
| 2020 | 2019 | } |
| 2021 | 2020 | |
| 2022 | − platform_set_drvdata(pdev, video); | |
| 2021 | + auxiliary_set_drvdata(aux_dev, video); | |
| 2023 | 2022 | |
| 2024 | 2023 | video->device = device; |
| 2025 | 2024 | strscpy(acpi_device_name(device), ACPI_VIDEO_BUS_NAME); |
| @@ -2068,7 +2067,7 @@ static int acpi_video_bus_probe(struct platform_device *pdev) · 42 unmodified lines | ||
| 2068 | 2067 | !auto_detect) |
| 2069 | 2068 | acpi_video_bus_register_backlight(video); |
| 2070 | 2069 | |
| 2071 | − error = acpi_video_bus_add_notify_handler(video, pdev); | |
| 2070 | + error = acpi_video_bus_add_notify_handler(video, &aux_dev->dev); | |
| 2072 | 2071 | if (error) |
| 2073 | 2072 | goto err_del; |
| 2074 | 2073 | |
| @@ -2096,10 +2095,10 @@ err_free_video: · 21 unmodified lines | ||
| 2096 | 2095 | return error; |
| 2097 | 2096 | } |
| 2098 | 2097 | |
| 2099 | −static void acpi_video_bus_remove(struct platform_device *pdev) | |
| 2098 | +static void acpi_video_bus_remove(struct auxiliary_device *aux_dev) | |
| 2100 | 2099 | { |
| 2101 | − struct acpi_video_bus *video = platform_get_drvdata(pdev); | |
| 2102 | − struct acpi_device *device = ACPI_COMPANION(&pdev->dev); | |
| 2100 | + struct acpi_video_bus *video = auxiliary_get_drvdata(aux_dev); | |
| 2101 | + struct acpi_device *device = ACPI_COMPANION(&aux_dev->dev); | |
| 2103 | 2102 | |
| 2104 | 2103 | acpi_dev_remove_notify_handler(device, ACPI_DEVICE_NOTIFY, |
| 2105 | 2104 | acpi_video_bus_notify); |
| @@ -2163,7 +2162,7 @@ int acpi_video_register(void) · 57 unmodified lines | ||
| 2163 | 2162 | |
| 2164 | 2163 | dmi_check_system(video_dmi_table); |
| 2165 | 2164 | |
| 2166 | − ret = platform_driver_register(&acpi_video_bus); | |
| 2165 | + ret = auxiliary_driver_register(&acpi_video_bus); | |
| 2167 | 2166 | if (ret) |
| 2168 | 2167 | goto leave; |
| 2169 | 2168 | |
| @@ -2183,7 +2182,7 @@ void acpi_video_unregister(void) · 13 unmodified lines | ||
| 2183 | 2182 | { |
| 2184 | 2183 | mutex_lock(®ister_count_mutex); |
| 2185 | 2184 | if (register_count) { |
| 2186 | − platform_driver_unregister(&acpi_video_bus); | |
| 2185 | + auxiliary_driver_unregister(&acpi_video_bus); | |
| 2187 | 2186 | register_count = 0; |
| 2188 | 2187 | may_report_brightness_keys = false; |
| 2189 | 2188 | } |
| @@ -379,8 +379,9 @@ const union acpi_predefined_info acpi_gbl_predefined_methods[] = { · 378 unmodified lines | ||
| 379 | 379 | |
| 380 | 380 | {{"_CPC", METHOD_0ARGS, |
| 381 | 381 | METHOD_RETURNS(ACPI_RTYPE_PACKAGE)}}, /* Variable-length (Ints/Bufs) */ |
| 382 | − PACKAGE_INFO(ACPI_PTYPE1_VAR, ACPI_RTYPE_INTEGER | ACPI_RTYPE_BUFFER, 0, | |
| 383 | − 0, 0, 0), | |
| 382 | + PACKAGE_INFO(ACPI_PTYPE1_VAR, | |
| 383 | + ACPI_RTYPE_INTEGER | ACPI_RTYPE_BUFFER | | |
| 384 | + ACPI_RTYPE_PACKAGE, 0, 0, 0, 0), | |
| 384 | 385 | |
| 385 | 386 | {{"_CR3", METHOD_0ARGS, /* ACPI 6.0 */ |
| 386 | 387 | METHOD_RETURNS(ACPI_RTYPE_INTEGER)}}, |
| @@ -1456,15 +1456,6 @@ int acpi_dev_pm_attach(struct device *dev, bool power_on) · 1455 unmodified lines | ||
| 1456 | 1456 | if (!adev || !acpi_match_device_ids(adev, special_pm_ids)) |
| 1457 | 1457 | return 0; |
| 1458 | 1458 | |
| 1459 | − /* | |
| 1460 | − * Skip devices whose ACPI companions don't support power management and | |
| 1461 | − * don't have a wakeup GPE. | |
| 1462 | − */ | |
| 1463 | − if (!acpi_device_power_manageable(adev) && !acpi_device_can_wakeup(adev)) { | |
| 1464 | − dev_dbg(dev, "No ACPI power management or wakeup GPE\n"); | |
| 1465 | − return 0; | |
| 1466 | − } | |
| 1467 | − | |
| 1468 | 1459 | /* |
| 1469 | 1460 | * Only attach the power domain to the first device if the |
| 1470 | 1461 | * companion is shared by multiple. This is to prevent doing power |
| @@ -389,6 +389,19 @@ static const struct dmi_system_id acpi_osi_dmi_table[] __initconst = { · 388 unmodified lines | ||
| 389 | 389 | }, |
| 390 | 390 | }, |
| 391 | 391 | |
| 392 | + /* | |
| 393 | + * The screen backlight turns off during udev device creation | |
| 394 | + * when returning true for _OSI("Windows 2009") | |
| 395 | + */ | |
| 396 | + { | |
| 397 | + .callback = dmi_disable_osi_win7, | |
| 398 | + .ident = "Acer Aspire One D255", | |
| 399 | + .matches = { | |
| 400 | + DMI_MATCH(DMI_SYS_VENDOR, "Acer"), | |
| 401 | + DMI_MATCH(DMI_PRODUCT_NAME, "AOD255"), | |
| 402 | + }, | |
| 403 | + }, | |
| 404 | + | |
| 392 | 405 | /* |
| 393 | 406 | * The wireless hotkey does not work on those machines when |
| 394 | 407 | * returning true for _OSI("Windows 2012") |
| @@ -1681,7 +1681,7 @@ acpi_status __init acpi_os_initialize(void) · 1680 unmodified lines | ||
| 1681 | 1681 | * Use acpi_os_map_generic_address to pre-map the reset |
| 1682 | 1682 | * register if it's in system memory. |
| 1683 | 1683 | */ |
| 1684 | − void *rv; | |
| 1684 | + void __iomem *rv; | |
| 1685 | 1685 | |
| 1686 | 1686 | rv = acpi_os_map_generic_address(&acpi_gbl_FADT.reset_register); |
| 1687 | 1687 | pr_debug("%s: Reset register mapping %s\n", __func__, |
| @@ -6,6 +6,7 @@ · 5 unmodified lines | ||
| 6 | 6 | #define pr_fmt(fmt) "ACPI: " fmt |
| 7 | 7 | |
| 8 | 8 | #include <linux/async.h> |
| 9 | +#include <linux/auxiliary_bus.h> | |
| 9 | 10 | #include <linux/module.h> |
| 10 | 11 | #include <linux/init.h> |
| 11 | 12 | #include <linux/slab.h> |
| @@ -2192,6 +2193,44 @@ static acpi_status acpi_bus_check_add_2(acpi_handle handle, u32 lvl_not_used, · 2180 unmodified lines | ||
| 2192 | 2193 | return acpi_bus_check_add(handle, false, (struct acpi_device **)ret_p); |
| 2193 | 2194 | } |
| 2194 | 2195 | |
| 2196 | +static void acpi_video_bus_device_release(struct device *dev) | |
| 2197 | +{ | |
| 2198 | + struct auxiliary_device *aux_dev = to_auxiliary_dev(dev); | |
| 2199 | + | |
| 2200 | + kfree(aux_dev); | |
| 2201 | +} | |
| 2202 | + | |
| 2203 | +static void acpi_create_video_bus_device(struct acpi_device *adev, | |
| 2204 | + struct acpi_device *parent) | |
| 2205 | +{ | |
| 2206 | + struct auxiliary_device *aux_dev; | |
| 2207 | + static unsigned int aux_dev_id; | |
| 2208 | + | |
| 2209 | + aux_dev = kzalloc_obj(*aux_dev); | |
| 2210 | + if (!aux_dev) | |
| 2211 | + return; | |
| 2212 | + | |
| 2213 | + aux_dev->id = aux_dev_id++; | |
| 2214 | + aux_dev->name = "video_bus"; | |
| 2215 | + aux_dev->dev.parent = acpi_get_first_physical_node(parent); | |
| 2216 | + if (!aux_dev->dev.parent) | |
| 2217 | + goto err; | |
| 2218 | + | |
| 2219 | + aux_dev->dev.release = acpi_video_bus_device_release; | |
| 2220 | + | |
| 2221 | + if (auxiliary_device_init(aux_dev)) | |
| 2222 | + goto err; | |
| 2223 | + | |
| 2224 | + ACPI_COMPANION_SET(&aux_dev->dev, adev); | |
| 2225 | + if (__auxiliary_device_add(aux_dev, "acpi")) | |
| 2226 | + auxiliary_device_uninit(aux_dev); | |
| 2227 | + | |
| 2228 | + return; | |
| 2229 | + | |
| 2230 | +err: | |
| 2231 | + kfree(aux_dev); | |
| 2232 | +} | |
| 2233 | + | |
| 2195 | 2234 | struct acpi_scan_system_dev { |
| 2196 | 2235 | struct list_head node; |
| 2197 | 2236 | struct acpi_device *adev; |
| @@ -2229,6 +2268,12 @@ static void acpi_default_enumeration(struct acpi_device *device) · 31 unmodified lines | ||
| 2229 | 2268 | sd->adev = device; |
| 2230 | 2269 | list_add_tail(&sd->node, &acpi_scan_system_dev_list); |
| 2231 | 2270 | } |
| 2271 | + } else if (device->pnp.type.backlight) { | |
| 2272 | + struct acpi_device *parent; | |
| 2273 | + | |
| 2274 | + parent = acpi_dev_parent(device); | |
| 2275 | + if (parent) | |
| 2276 | + acpi_create_video_bus_device(device, parent); | |
| 2232 | 2277 | } else { |
| 2233 | 2278 | /* For a regular device object, create a platform device. */ |
| 2234 | 2279 | acpi_create_platform_device(device, NULL); |
| @@ -386,6 +386,14 @@ static const struct dmi_system_id acpisleep_dmi_table[] __initconst = { · 385 unmodified lines | ||
| 386 | 386 | DMI_MATCH(DMI_PRODUCT_NAME, "80E1"), |
| 387 | 387 | }, |
| 388 | 388 | }, |
| 389 | + { | |
| 390 | + .callback = init_nvs_save_s3, | |
| 391 | + .ident = "Lenovo G70-35", | |
| 392 | + .matches = { | |
| 393 | + DMI_MATCH(DMI_SYS_VENDOR, "LENOVO"), | |
| 394 | + DMI_MATCH(DMI_PRODUCT_NAME, "80Q5"), | |
| 395 | + }, | |
| 396 | + }, | |
| 389 | 397 | /* |
| 390 | 398 | * ThinkPad X1 Tablet(2016) cannot do suspend-to-idle using |
| 391 | 399 | * the Low Power S0 Idle firmware interface (see |
| @@ -142,6 +142,30 @@ pub(crate) struct ShrinkablePageRange { · 141 unmodified lines | ||
| 142 | 142 | _pin: PhantomPinned, |
| 143 | 143 | } |
| 144 | 144 | |
| 145 | +// We do not define any ops. For now, used only to check identity of vmas. | |
| 146 | +static BINDER_VM_OPS: bindings::vm_operations_struct = pin_init::zeroed(); | |
| 147 | + | |
| 148 | +// To ensure that we do not accidentally install pages into or zap pages from the wrong vma, we | |
| 149 | +// check its vm_ops and private data before using it. | |
| 150 | +fn check_vma(vma: &virt::VmaRef, owner: *const ShrinkablePageRange) -> Option<&virt::VmaMixedMap> { | |
| 151 | + // SAFETY: Just reading the vm_ops pointer of any active vma is safe. | |
| 152 | + let vm_ops = unsafe { (*vma.as_ptr()).vm_ops }; | |
| 153 | + if !ptr::eq(vm_ops, &BINDER_VM_OPS) { | |
| 154 | + return None; | |
| 155 | + } | |
| 156 | + | |
| 157 | + // SAFETY: Reading the vm_private_data pointer of a binder-owned vma is safe. | |
| 158 | + let vm_private_data = unsafe { (*vma.as_ptr()).vm_private_data }; | |
| 159 | + // The ShrinkablePageRange is only dropped when the Process is dropped, which only happens once | |
| 160 | + // the file's ->release handler is invoked, which means the ShrinkablePageRange outlives any | |
| 161 | + // VMA associated with it, so there can't be any false positives due to pointer reuse here. | |
| 162 | + if !ptr::eq(vm_private_data, owner.cast()) { | |
| 163 | + return None; | |
| 164 | + } | |
| 165 | + | |
| 166 | + vma.as_mixedmap_vma() | |
| 167 | +} | |
| 168 | + | |
| 145 | 169 | struct Inner { |
| 146 | 170 | /// Array of pages. |
| 147 | 171 | /// |
| @@ -308,6 +332,18 @@ impl ShrinkablePageRange { · 160 unmodified lines | ||
| 308 | 332 | inner.size = num_pages; |
| 309 | 333 | inner.vma_addr = vma.start(); |
| 310 | 334 | |
| 335 | + // This pointer is only used for comparison - it's not dereferenced. | |
| 336 | + // | |
| 337 | + // SAFETY: We own the vma, and we don't use any methods on VmaNew that rely on | |
| 338 | + // `vm_private_data`. | |
| 339 | + unsafe { | |
| 340 | + (*vma.as_ptr()).vm_private_data = ptr::from_ref(self).cast_mut().cast::<c_void>() | |
| 341 | + }; | |
| 342 | + | |
| 343 | + // SAFETY: We own the vma, and we don't use any methods on VmaNew that rely on | |
| 344 | + // `vm_ops`. | |
| 345 | + unsafe { (*vma.as_ptr()).vm_ops = &BINDER_VM_OPS }; | |
| 346 | + | |
| 311 | 347 | Ok(num_pages) |
| 312 | 348 | } |
| 313 | 349 | |
| @@ -399,22 +435,25 @@ impl ShrinkablePageRange { · 85 unmodified lines | ||
| 399 | 435 | // |
| 400 | 436 | // Using `mmput_async` avoids this, because then the `mm` cleanup is instead queued to a |
| 401 | 437 | // workqueue. |
| 402 | − MmWithUser::into_mmput_async(self.mm.mmget_not_zero().ok_or(ESRCH)?) | |
| 403 | − .mmap_read_lock() | |
| 404 | − .vma_lookup(vma_addr) | |
| 405 | − .ok_or(ESRCH)? | |
| 406 | − .as_mixedmap_vma() | |
| 407 | − .ok_or(ESRCH)? | |
| 408 | − .vm_insert_page(user_page_addr, &new_page) | |
| 409 | − .inspect_err(|err| { | |
| 410 | − pr_warn!( | |
| 411 | − "Failed to vm_insert_page({}): vma_addr:{} i:{} err:{:?}", | |
| 412 | − user_page_addr, | |
| 413 | − vma_addr, | |
| 414 | − i, | |
| 415 | − err | |
| 416 | − ) | |
| 417 | − })?; | |
| 438 | + let mm = MmWithUser::into_mmput_async(self.mm.mmget_not_zero().ok_or(ESRCH)?); | |
| 439 | + { | |
| 440 | + let vma_read; | |
| 441 | + let mmap_read; | |
| 442 | + let vma = if let Some(ret) = mm.lock_vma_under_rcu(vma_addr) { | |
| 443 | + vma_read = ret; | |
| 444 | + check_vma(&vma_read, self) | |
| 445 | + } else { | |
| 446 | + mmap_read = mm.mmap_read_lock(); | |
| 447 | + mmap_read | |
| 448 | + .vma_lookup(vma_addr) | |
| 449 | + .and_then(|vma| check_vma(vma, self)) | |
| 450 | + }; | |
| 451 | + | |
| 452 | + match vma { | |
| 453 | + Some(vma) => vma.vm_insert_page(user_page_addr, &new_page)?, | |
| 454 | + None => return Err(ESRCH), | |
| 455 | + } | |
| 456 | + } | |
| 418 | 457 | |
| 419 | 458 | let inner = self.lock.lock(); |
| 420 | 459 | |
| @@ -667,12 +706,15 @@ unsafe extern "C" fn rust_shrink_free_page( · 246 unmodified lines | ||
| 667 | 706 | let mmap_read; |
| 668 | 707 | let mm_mutex; |
| 669 | 708 | let vma_addr; |
| 709 | + let range_ptr; | |
| 670 | 710 | |
| 671 | 711 | { |
| 672 | 712 | // CAST: The `list_head` field is first in `PageInfo`. |
| 673 | 713 | let info = item as *mut PageInfo; |
| 674 | 714 | // SAFETY: The `range` field of `PageInfo` is immutable. |
| 675 | − let range = unsafe { &*((*info).range) }; | |
| 715 | + range_ptr = unsafe { (*info).range }; | |
| 716 | + // SAFETY: The `range` outlives its `PageInfo` values. | |
| 717 | + let range = unsafe { &*range_ptr }; | |
| 676 | 718 | |
| 677 | 719 | mm = match range.mm.mmget_not_zero() { |
| 678 | 720 | Some(mm) => MmWithUser::into_mmput_async(mm), |
| @@ -717,9 +759,11 @@ unsafe extern "C" fn rust_shrink_free_page( · 38 unmodified lines | ||
| 717 | 759 | // SAFETY: The lru lock is locked when this method is called. |
| 718 | 760 | unsafe { bindings::spin_unlock(&raw mut (*lru).lock) }; |
| 719 | 761 | |
| 720 | − if let Some(vma) = mmap_read.vma_lookup(vma_addr) { | |
| 721 | − let user_page_addr = vma_addr + (page_index << PAGE_SHIFT); | |
| 722 | − vma.zap_page_range_single(user_page_addr, PAGE_SIZE); | |
| 762 | + if let Some(unchecked_vma) = mmap_read.vma_lookup(vma_addr) { | |
| 763 | + if let Some(vma) = check_vma(unchecked_vma, range_ptr) { | |
| 764 | + let user_page_addr = vma_addr + (page_index << PAGE_SHIFT); | |
| 765 | + vma.zap_page_range_single(user_page_addr, PAGE_SIZE); | |
| 766 | + } | |
| 723 | 767 | } |
| 724 | 768 | |
| 725 | 769 | drop(mmap_read); |
| @@ -1295,7 +1295,8 @@ impl Process { · 1294 unmodified lines | ||
| 1295 | 1295 | } |
| 1296 | 1296 | |
| 1297 | 1297 | pub(crate) fn dead_binder_done(&self, cookie: u64, thread: &Thread) { |
| 1298 | − if let Some(death) = self.inner.lock().pull_delivered_death(cookie) { | |
| 1298 | + let death = self.inner.lock().pull_delivered_death(cookie); | |
| 1299 | + if let Some(death) = death { | |
| 1299 | 1300 | death.set_notification_done(thread); |
| 1300 | 1301 | } |
| 1301 | 1302 | } |
| @@ -118,7 +118,7 @@ impl<T> ArrayRangeAllocator<T> { · 117 unmodified lines | ||
| 118 | 118 | size: usize, |
| 119 | 119 | is_oneway: bool, |
| 120 | 120 | pid: Pid, |
| 121 | − ) -> Result<usize> { | |
| 121 | + ) -> Result<(usize, bool)> { | |
| 122 | 122 | // Compute new value of free_oneway_space, which is set only on success. |
| 123 | 123 | let new_oneway_space = if is_oneway { |
| 124 | 124 | match self.free_oneway_space.checked_sub(size) { |
| @@ -146,7 +146,38 @@ impl<T> ArrayRangeAllocator<T> { · 21 unmodified lines | ||
| 146 | 146 | .ok() |
| 147 | 147 | .unwrap(); |
| 148 | 148 | |
| 149 | − Ok(insert_at_offset) | |
| 149 | + // Start detecting spammers once we have less than 20% | |
| 150 | + // of async space left (which is less than 10% of total | |
| 151 | + // buffer size). | |
| 152 | + // | |
| 153 | + // (This will short-circuit, so `low_oneway_space` is | |
| 154 | + // only called when necessary.) | |
| 155 | + let oneway_spam_detected = | |
| 156 | + is_oneway && new_oneway_space < self.size / 10 && self.low_oneway_space(pid); | |
| 157 | + | |
| 158 | + Ok((insert_at_offset, oneway_spam_detected)) | |
| 159 | + } | |
| 160 | + | |
| 161 | + /// Find the amount and size of buffers allocated by the current caller. | |
| 162 | + /// | |
| 163 | + /// The idea is that once we cross the threshold, whoever is responsible | |
| 164 | + /// for the low async space is likely to try to send another async transaction, | |
| 165 | + /// and at some point we'll catch them in the act. This is more efficient | |
| 166 | + /// than keeping a map per pid. | |
| 167 | + fn low_oneway_space(&self, calling_pid: Pid) -> bool { | |
| 168 | + let mut total_alloc_size = 0; | |
| 169 | + let mut num_buffers = 0; | |
| 170 | + | |
| 171 | + // Warn if this pid has more than 50 transactions, or more than 50% of | |
| 172 | + // async space (which is 25% of total buffer size). Oneway spam is only | |
| 173 | + // detected when the threshold is exceeded. | |
| 174 | + for range in &self.ranges { | |
| 175 | + if range.state.is_oneway() && range.state.pid() == calling_pid { | |
| 176 | + total_alloc_size += range.size; | |
| 177 | + num_buffers += 1; | |
| 178 | + } | |
| 179 | + } | |
| 180 | + num_buffers > 50 || total_alloc_size > self.size / 4 | |
| 150 | 181 | } |
| 151 | 182 | |
| 152 | 183 | pub(crate) fn reservation_abort(&mut self, offset: usize) -> Result<FreedRange> { |
| @@ -188,11 +188,11 @@ impl<T> RangeAllocator<T> { · 187 unmodified lines | ||
| 188 | 188 | self.reserve_new(args) |
| 189 | 189 | } |
| 190 | 190 | Impl::Array(array) => { |
| 191 | − let offset = | |
| 191 | + let (offset, oneway_spam_detected) = | |
| 192 | 192 | array.reserve_new(args.debug_id, args.size, args.is_oneway, args.pid)?; |
| 193 | 193 | Ok(ReserveNew::Success(ReserveNewSuccess { |
| 194 | 194 | offset, |
| 195 | − oneway_spam_detected: false, | |
| 195 | + oneway_spam_detected, | |
| 196 | 196 | _empty_array_alloc: args.empty_array_alloc, |
| 197 | 197 | _new_tree_alloc: args.new_tree_alloc, |
| 198 | 198 | _tree_alloc: args.tree_alloc, |
| @@ -164,15 +164,6 @@ impl<T> TreeRangeAllocator<T> { · 163 unmodified lines | ||
| 164 | 164 | self.free_oneway_space |
| 165 | 165 | }; |
| 166 | 166 | |
| 167 | − // Start detecting spammers once we have less than 20% | |
| 168 | − // of async space left (which is less than 10% of total | |
| 169 | − // buffer size). | |
| 170 | − // | |
| 171 | − // (This will short-circut, so `low_oneway_space` is | |
| 172 | − // only called when necessary.) | |
| 173 | − let oneway_spam_detected = | |
| 174 | − is_oneway && new_oneway_space < self.size / 10 && self.low_oneway_space(pid); | |
| 175 | − | |
| 176 | 167 | let (found_size, found_off, tree_node, free_tree_node) = match self.find_best_match(size) { |
| 177 | 168 | None => { |
| 178 | 169 | pr_warn!("ENOSPC from range_alloc.reserve_new - size: {}", size); |
| @@ -203,6 +194,15 @@ impl<T> TreeRangeAllocator<T> { · 24 unmodified lines | ||
| 203 | 194 | self.free_tree.insert(free_tree_node); |
| 204 | 195 | } |
| 205 | 196 | |
| 197 | + // Start detecting spammers once we have less than 20% | |
| 198 | + // of async space left (which is less than 10% of total | |
| 199 | + // buffer size). | |
| 200 | + // | |
| 201 | + // (This will short-circuit, so `low_oneway_space` is | |
| 202 | + // only called when necessary.) | |
| 203 | + let oneway_spam_detected = | |
| 204 | + is_oneway && new_oneway_space < self.size / 10 && self.low_oneway_space(pid); | |
| 205 | + | |
| 206 | 206 | Ok((found_off, oneway_spam_detected)) |
| 207 | 207 | } |
| 208 | 208 | |
| @@ -1015,12 +1015,9 @@ impl Thread { · 1014 unmodified lines | ||
| 1015 | 1015 | |
| 1016 | 1016 | // Copy offsets if there are any. |
| 1017 | 1017 | if offsets_size > 0 { |
| 1018 | − { | |
| 1019 | − let mut reader = | |
| 1020 | − UserSlice::new(UserPtr::from_addr(trd_data_ptr.offsets as _), offsets_size) | |
| 1021 | − .reader(); | |
| 1022 | − alloc.copy_into(&mut reader, aligned_data_size, offsets_size)?; | |
| 1023 | − } | |
| 1018 | + let mut offsets_reader = | |
| 1019 | + UserSlice::new(UserPtr::from_addr(trd_data_ptr.offsets as _), offsets_size) | |
| 1020 | + .reader(); | |
| 1024 | 1021 | |
| 1025 | 1022 | let offsets_start = aligned_data_size; |
| 1026 | 1023 | let offsets_end = aligned_data_size + offsets_size; |
| @@ -1041,11 +1038,9 @@ impl Thread { · 14 unmodified lines | ||
| 1041 | 1038 | .step_by(size_of::<u64>()) |
| 1042 | 1039 | .enumerate() |
| 1043 | 1040 | { |
| 1044 | − let offset: usize = view | |
| 1045 | − .alloc | |
| 1046 | − .read::<u64>(index_offset)? | |
| 1047 | − .try_into() | |
| 1048 | − .map_err(|_| EINVAL)?; | |
| 1041 | + let offset = offsets_reader.read::<u64>()?; | |
| 1042 | + view.alloc.write(index_offset, &offset)?; | |
| 1043 | + let offset: usize = offset.try_into().map_err(|_| EINVAL)?; | |
| 1049 | 1044 | |
| 1050 | 1045 | if offset < end_of_previous_object || !is_aligned(offset, size_of::<u32>()) { |
| 1051 | 1046 | pr_warn!("Got transaction with invalid offset."); |
| @@ -4189,6 +4189,7 @@ static const struct ata_dev_quirks_entry __ata_dev_quirks[] = { · 4188 unmodified lines | ||
| 4189 | 4189 | ATA_QUIRK_FIRMWARE_WARN }, |
| 4190 | 4190 | |
| 4191 | 4191 | /* Seagate disks with LPM issues */ |
| 4192 | + { "ST1000DM010-2EP102", NULL, ATA_QUIRK_NOLPM }, | |
| 4192 | 4193 | { "ST2000DM008-2FR102", NULL, ATA_QUIRK_NOLPM }, |
| 4193 | 4194 | |
| 4194 | 4195 | /* drives which fail FPDMA_AA activation (some may freeze afterwards) |
| @@ -4231,6 +4232,7 @@ static const struct ata_dev_quirks_entry __ata_dev_quirks[] = { · 36 unmodified lines | ||
| 4231 | 4232 | /* Devices that do not need bridging limits applied */ |
| 4232 | 4233 | { "MTRON MSP-SATA*", NULL, ATA_QUIRK_BRIDGE_OK }, |
| 4233 | 4234 | { "BUFFALO HD-QSU2/R5", NULL, ATA_QUIRK_BRIDGE_OK }, |
| 4235 | + { "QEMU HARDDISK", "2.5+", ATA_QUIRK_BRIDGE_OK }, | |
| 4234 | 4236 | |
| 4235 | 4237 | /* Devices which aren't very happy with higher link speeds */ |
| 4236 | 4238 | { "WD My Book", NULL, ATA_QUIRK_1_5_GBPS }, |
| @@ -6269,10 +6271,6 @@ static void ata_port_detach(struct ata_port *ap) · 2032 unmodified lines | ||
| 6269 | 6271 | } |
| 6270 | 6272 | } |
| 6271 | 6273 | |
| 6272 | − /* Make sure the deferred qc work finished. */ | |
| 6273 | − cancel_work_sync(&ap->deferred_qc_work); | |
| 6274 | − WARN_ON(ap->deferred_qc); | |
| 6275 | − | |
| 6276 | 6274 | /* Tell EH to disable all devices */ |
| 6277 | 6275 | ap->pflags |= ATA_PFLAG_UNLOADING; |
| 6278 | 6276 | ata_port_schedule_eh(ap); |
| @@ -6283,9 +6281,11 @@ static void ata_port_detach(struct ata_port *ap) · 4 unmodified lines | ||
| 6283 | 6281 | /* wait till EH commits suicide */ |
| 6284 | 6282 | ata_port_wait_eh(ap); |
| 6285 | 6283 | |
| 6286 | − /* it better be dead now */ | |
| 6284 | + /* It better be dead now and not have any remaining deferred qc. */ | |
| 6287 | 6285 | WARN_ON(!(ap->pflags & ATA_PFLAG_UNLOADED)); |
| 6286 | + WARN_ON(ap->deferred_qc); | |
| 6288 | 6287 | |
| 6288 | + cancel_work_sync(&ap->deferred_qc_work); | |
| 6289 | 6289 | cancel_delayed_work_sync(&ap->hotplug_task); |
| 6290 | 6290 | cancel_delayed_work_sync(&ap->scsi_rescan_task); |
| 6291 | 6291 | |
| @@ -640,12 +640,29 @@ void ata_scsi_cmd_error_handler(struct Scsi_Host *host, struct ata_port *ap, · 639 unmodified lines | ||
| 640 | 640 | set_host_byte(scmd, DID_OK); |
| 641 | 641 | |
| 642 | 642 | ata_qc_for_each_raw(ap, qc, i) { |
| 643 | − if (qc->flags & ATA_QCFLAG_ACTIVE && | |
| 644 | − qc->scsicmd == scmd) | |
| 643 | + if (qc->scsicmd != scmd) | |
| 644 | + continue; | |
| 645 | + if ((qc->flags & ATA_QCFLAG_ACTIVE) || | |
| 646 | + qc == ap->deferred_qc) | |
| 645 | 647 | break; |
| 646 | 648 | } |
| 647 | 649 | |
| 648 | − if (i < ATA_MAX_QUEUE) { | |
| 650 | + if (i < ATA_MAX_QUEUE && qc == ap->deferred_qc) { | |
| 651 | + /* | |
| 652 | + * This is a deferred command that timed out while | |
| 653 | + * waiting for the command queue to drain. Since the qc | |
| 654 | + * is not active yet (deferred_qc is still set, so the | |
| 655 | + * deferred qc work has not issued the command yet), | |
| 656 | + * simply signal the timeout by finishing the SCSI | |
| 657 | + * command and clear the deferred qc to prevent the | |
| 658 | + * deferred qc work from issuing this qc. | |
| 659 | + */ | |
| 660 | + WARN_ON_ONCE(qc->flags & ATA_QCFLAG_ACTIVE); | |
| 661 | + ap->deferred_qc = NULL; | |
| 662 | + cancel_work(&ap->deferred_qc_work); | |
| 663 | + set_host_byte(scmd, DID_TIME_OUT); | |
| 664 | + scsi_eh_finish_cmd(scmd, &ap->eh_done_q); | |
| 665 | + } else if (i < ATA_MAX_QUEUE) { | |
| 649 | 666 | /* the scmd has an associated qc */ |
| 650 | 667 | if (!(qc->flags & ATA_QCFLAG_EH)) { |
| 651 | 668 | /* which hasn't failed yet, timeout */ |
| @@ -1699,6 +1699,7 @@ void ata_scsi_requeue_deferred_qc(struct ata_port *ap) · 1698 unmodified lines | ||
| 1699 | 1699 | |
| 1700 | 1700 | scmd = qc->scsicmd; |
| 1701 | 1701 | ap->deferred_qc = NULL; |
| 1702 | + cancel_work(&ap->deferred_qc_work); | |
| 1702 | 1703 | ata_qc_free(qc); |
| 1703 | 1704 | scmd->result = (DID_SOFT_ERROR << 16); |
| 1704 | 1705 | scsi_done(scmd); |
| @@ -179,19 +179,10 @@ void device_release_driver_internal(struct device *dev, const struct device_driv · 178 unmodified lines | ||
| 179 | 179 | void driver_detach(const struct device_driver *drv); |
| 180 | 180 | void driver_deferred_probe_del(struct device *dev); |
| 181 | 181 | void device_set_deferred_probe_reason(const struct device *dev, struct va_format *vaf); |
| 182 | −static inline int driver_match_device_locked(const struct device_driver *drv, | |
| 183 | − struct device *dev) | |
| 184 | −{ | |
| 185 | − device_lock_assert(dev); | |
| 186 | − | |
| 187 | − return drv->bus->match ? drv->bus->match(dev, drv) : 1; | |
| 188 | −} | |
| 189 | − | |
| 190 | 182 | static inline int driver_match_device(const struct device_driver *drv, |
| 191 | 183 | struct device *dev) |
| 192 | 184 | { |
| 193 | − guard(device)(dev); | |
| 194 | − return driver_match_device_locked(drv, dev); | |
| 185 | + return drv->bus->match ? drv->bus->match(dev, drv) : 1; | |
| 195 | 186 | } |
| 196 | 187 | |
| 197 | 188 | static inline void dev_sync_state(struct device *dev) |
| @@ -928,7 +928,7 @@ static int __device_attach_driver(struct device_driver *drv, void *_data) · 927 unmodified lines | ||
| 928 | 928 | bool async_allowed; |
| 929 | 929 | int ret; |
| 930 | 930 | |
| 931 | − ret = driver_match_device_locked(drv, dev); | |
| 931 | + ret = driver_match_device(drv, dev); | |
| 932 | 932 | if (ret == 0) { |
| 933 | 933 | /* no match */ |
| 934 | 934 | return 0; |
| @@ -797,7 +797,18 @@ struct fwnode_handle * · 796 unmodified lines | ||
| 797 | 797 | fwnode_get_next_child_node(const struct fwnode_handle *fwnode, |
| 798 | 798 | struct fwnode_handle *child) |
| 799 | 799 | { |
| 800 | − return fwnode_call_ptr_op(fwnode, get_next_child_node, child); | |
| 800 | + struct fwnode_handle *next; | |
| 801 | + | |
| 802 | + if (IS_ERR_OR_NULL(fwnode)) | |
| 803 | + return NULL; | |
| 804 | + | |
| 805 | + /* Try to find a child in primary fwnode */ | |
| 806 | + next = fwnode_call_ptr_op(fwnode, get_next_child_node, child); | |
| 807 | + if (next) | |
| 808 | + return next; | |
| 809 | + | |
| 810 | + /* When no more children in primary, continue with secondary */ | |
| 811 | + return fwnode_call_ptr_op(fwnode->secondary, get_next_child_node, child); | |
| 801 | 812 | } |
| 802 | 813 | EXPORT_SYMBOL_GPL(fwnode_get_next_child_node); |
| 803 | 814 | |
| @@ -841,19 +852,7 @@ EXPORT_SYMBOL_GPL(fwnode_get_next_available_child_node); · 37 unmodified lines | ||
| 841 | 852 | struct fwnode_handle *device_get_next_child_node(const struct device *dev, |
| 842 | 853 | struct fwnode_handle *child) |
| 843 | 854 | { |
| 844 | − const struct fwnode_handle *fwnode = dev_fwnode(dev); | |
| 845 | − struct fwnode_handle *next; | |
| 846 | − | |
| 847 | − if (IS_ERR_OR_NULL(fwnode)) | |
| 848 | − return NULL; | |
| 849 | − | |
| 850 | − /* Try to find a child in primary fwnode */ | |
| 851 | − next = fwnode_get_next_child_node(fwnode, child); | |
| 852 | − if (next) | |
| 853 | − return next; | |
| 854 | − | |
| 855 | − /* When no more children in primary, continue with secondary */ | |
| 856 | − return fwnode_get_next_child_node(fwnode->secondary, child); | |
| 855 | + return fwnode_get_next_child_node(dev_fwnode(dev), child); | |
| 857 | 856 | } |
| 858 | 857 | EXPORT_SYMBOL_GPL(device_get_next_child_node); |
| 859 | 858 | |
| @@ -483,38 +483,20 @@ void drbd_al_begin_io(struct drbd_device *device, struct drbd_interval *i) · 482 unmodified lines | ||
| 483 | 483 | |
| 484 | 484 | int drbd_al_begin_io_nonblock(struct drbd_device *device, struct drbd_interval *i) |
| 485 | 485 | { |
| 486 | − struct lru_cache *al = device->act_log; | |
| 487 | 486 | /* for bios crossing activity log extent boundaries, |
| 488 | 487 | * we may need to activate two extents in one go */ |
| 489 | 488 | unsigned first = i->sector >> (AL_EXTENT_SHIFT-9); |
| 490 | 489 | unsigned last = i->size == 0 ? first : (i->sector + (i->size >> 9) - 1) >> (AL_EXTENT_SHIFT-9); |
| 491 | − unsigned nr_al_extents; | |
| 492 | − unsigned available_update_slots; | |
| 493 | 490 | unsigned enr; |
| 494 | 491 | |
| 495 | − D_ASSERT(device, first <= last); | |
| 496 | − | |
| 497 | − nr_al_extents = 1 + last - first; /* worst case: all touched extends are cold. */ | |
| 498 | − available_update_slots = min(al->nr_elements - al->used, | |
| 499 | − al->max_pending_changes - al->pending_changes); | |
| 500 | − | |
| 501 | − /* We want all necessary updates for a given request within the same transaction | |
| 502 | − * We could first check how many updates are *actually* needed, | |
| 503 | − * and use that instead of the worst-case nr_al_extents */ | |
| 504 | − if (available_update_slots < nr_al_extents) { | |
| 505 | − /* Too many activity log extents are currently "hot". | |
| 506 | − * | |
| 507 | − * If we have accumulated pending changes already, | |
| 508 | − * we made progress. | |
| 509 | − * | |
| 510 | − * If we cannot get even a single pending change through, | |
| 511 | − * stop the fast path until we made some progress, | |
| 512 | − * or requests to "cold" extents could be starved. */ | |
| 513 | − if (!al->pending_changes) | |
| 514 | − __set_bit(__LC_STARVING, &device->act_log->flags); | |
| 515 | − return -ENOBUFS; | |
| 492 | + if (i->partially_in_al_next_enr) { | |
| 493 | + D_ASSERT(device, first < i->partially_in_al_next_enr); | |
| 494 | + D_ASSERT(device, last >= i->partially_in_al_next_enr); | |
| 495 | + first = i->partially_in_al_next_enr; | |
| 516 | 496 | } |
| 517 | 497 | |
| 498 | + D_ASSERT(device, first <= last); | |
| 499 | + | |
| 518 | 500 | /* Is resync active in this area? */ |
| 519 | 501 | for (enr = first; enr <= last; enr++) { |
| 520 | 502 | struct lc_element *tmp; |
| @@ -529,14 +511,21 @@ int drbd_al_begin_io_nonblock(struct drbd_device *device, struct drbd_interval * · 8 unmodified lines | ||
| 529 | 511 | } |
| 530 | 512 | } |
| 531 | 513 | |
| 532 | − /* Checkout the refcounts. | |
| 533 | − * Given that we checked for available elements and update slots above, | |
| 534 | − * this has to be successful. */ | |
| 514 | + /* Try to checkout the refcounts. */ | |
| 535 | 515 | for (enr = first; enr <= last; enr++) { |
| 536 | 516 | struct lc_element *al_ext; |
| 537 | 517 | al_ext = lc_get_cumulative(device->act_log, enr); |
| 538 | − if (!al_ext) | |
| 539 | − drbd_info(device, "LOGIC BUG for enr=%u\n", enr); | |
| 518 | + | |
| 519 | + if (!al_ext) { | |
| 520 | + /* Did not work. We may have exhausted the possible | |
| 521 | + * changes per transaction. Or raced with someone | |
| 522 | + * "locking" it against changes. | |
| 523 | + * Remember where to continue from. | |
| 524 | + */ | |
| 525 | + if (enr > first) | |
| 526 | + i->partially_in_al_next_enr = enr; | |
| 527 | + return -ENOBUFS; | |
| 528 | + } | |
| 540 | 529 | } |
| 541 | 530 | return 0; |
| 542 | 531 | } |
| @@ -556,7 +545,11 @@ void drbd_al_complete_io(struct drbd_device *device, struct drbd_interval *i) · 13 unmodified lines | ||
| 556 | 545 | |
| 557 | 546 | for (enr = first; enr <= last; enr++) { |
| 558 | 547 | extent = lc_find(device->act_log, enr); |
| 559 | − if (!extent) { | |
| 548 | + /* Yes, this masks a bug elsewhere. However, during normal | |
| 549 | + * operation this is harmless, so no need to crash the kernel | |
| 550 | + * by the BUG_ON(refcount == 0) in lc_put(). | |
| 551 | + */ | |
| 552 | + if (!extent || extent->refcnt == 0) { | |
| 560 | 553 | drbd_err(device, "al_complete_io() called on inactive extent %u\n", enr); |
| 561 | 554 | continue; |
| 562 | 555 | } |
| @@ -8,12 +8,15 @@ · 7 unmodified lines | ||
| 8 | 8 | struct drbd_interval { |
| 9 | 9 | struct rb_node rb; |
| 10 | 10 | sector_t sector; /* start sector of the interval */ |
| 11 | − unsigned int size; /* size in bytes */ | |
| 12 | 11 | sector_t end; /* highest interval end in subtree */ |
| 12 | + unsigned int size; /* size in bytes */ | |
| 13 | 13 | unsigned int local:1 /* local or remote request? */; |
| 14 | 14 | unsigned int waiting:1; /* someone is waiting for completion */ |
| 15 | 15 | unsigned int completed:1; /* this has been completed already; |
| 16 | 16 | * ignore for conflict detection */ |
| 17 | + | |
| 18 | + /* to resume a partially successful drbd_al_begin_io_nonblock(); */ | |
| 19 | + unsigned int partially_in_al_next_enr; | |
| 17 | 20 | }; |
| 18 | 21 | |
| 19 | 22 | static inline void drbd_clear_interval(struct drbd_interval *i) |
| @@ -32,6 +32,7 @@ · 31 unmodified lines | ||
| 32 | 32 | #include <linux/memcontrol.h> |
| 33 | 33 | #include <linux/mm_inline.h> |
| 34 | 34 | #include <linux/slab.h> |
| 35 | +#include <linux/string.h> | |
| 35 | 36 | #include <linux/random.h> |
| 36 | 37 | #include <linux/reboot.h> |
| 37 | 38 | #include <linux/notifier.h> |
| @@ -732,9 +733,9 @@ int drbd_send_sync_param(struct drbd_peer_device *peer_device) · 694 unmodified lines | ||
| 732 | 733 | } |
| 733 | 734 | |
| 734 | 735 | if (apv >= 88) |
| 735 | − strcpy(p->verify_alg, nc->verify_alg); | |
| 736 | + strscpy(p->verify_alg, nc->verify_alg); | |
| 736 | 737 | if (apv >= 89) |
| 737 | − strcpy(p->csums_alg, nc->csums_alg); | |
| 738 | + strscpy(p->csums_alg, nc->csums_alg); | |
| 738 | 739 | rcu_read_unlock(); |
| 739 | 740 | |
| 740 | 741 | return drbd_send_command(peer_device, sock, cmd, size, NULL, 0); |
| @@ -745,6 +746,7 @@ int __drbd_send_protocol(struct drbd_connection *connection, enum drbd_packet cm · 4 unmodified lines | ||
| 745 | 746 | struct drbd_socket *sock; |
| 746 | 747 | struct p_protocol *p; |
| 747 | 748 | struct net_conf *nc; |
| 749 | + size_t integrity_alg_len; | |
| 748 | 750 | int size, cf; |
| 749 | 751 | |
| 750 | 752 | sock = &connection->data; |
| @@ -762,8 +764,10 @@ int __drbd_send_protocol(struct drbd_connection *connection, enum drbd_packet cm · 11 unmodified lines | ||
| 762 | 764 | } |
| 763 | 765 | |
| 764 | 766 | size = sizeof(*p); |
| 765 | − if (connection->agreed_pro_version >= 87) | |
| 766 | − size += strlen(nc->integrity_alg) + 1; | |
| 767 | + if (connection->agreed_pro_version >= 87) { | |
| 768 | + integrity_alg_len = strlen(nc->integrity_alg) + 1; | |
| 769 | + size += integrity_alg_len; | |
| 770 | + } | |
| 767 | 771 | |
| 768 | 772 | p->protocol = cpu_to_be32(nc->wire_protocol); |
| 769 | 773 | p->after_sb_0p = cpu_to_be32(nc->after_sb_0p); |
| @@ -778,7 +782,7 @@ int __drbd_send_protocol(struct drbd_connection *connection, enum drbd_packet cm · 8 unmodified lines | ||
| 778 | 782 | p->conn_flags = cpu_to_be32(cf); |
| 779 | 783 | |
| 780 | 784 | if (connection->agreed_pro_version >= 87) |
| 781 | − strcpy(p->integrity_alg, nc->integrity_alg); | |
| 785 | + strscpy(p->integrity_alg, nc->integrity_alg, integrity_alg_len); | |
| 782 | 786 | rcu_read_unlock(); |
| 783 | 787 | |
| 784 | 788 | return __conn_send_command(connection, sock, cmd, size, NULL, 0); |
| @@ -3801,14 +3801,14 @@ static int receive_SyncParam(struct drbd_connection *connection, struct packet_i · 3800 unmodified lines | ||
| 3801 | 3801 | *new_net_conf = *old_net_conf; |
| 3802 | 3802 | |
| 3803 | 3803 | if (verify_tfm) { |
| 3804 | − strcpy(new_net_conf->verify_alg, p->verify_alg); | |
| 3804 | + strscpy(new_net_conf->verify_alg, p->verify_alg); | |
| 3805 | 3805 | new_net_conf->verify_alg_len = strlen(p->verify_alg) + 1; |
| 3806 | 3806 | crypto_free_shash(peer_device->connection->verify_tfm); |
| 3807 | 3807 | peer_device->connection->verify_tfm = verify_tfm; |
| 3808 | 3808 | drbd_info(device, "using verify-alg: \"%s\"\n", p->verify_alg); |
| 3809 | 3809 | } |
| 3810 | 3810 | if (csums_tfm) { |
| 3811 | − strcpy(new_net_conf->csums_alg, p->csums_alg); | |
| 3811 | + strscpy(new_net_conf->csums_alg, p->csums_alg); | |
| 3812 | 3812 | new_net_conf->csums_alg_len = strlen(p->csums_alg) + 1; |
| 3813 | 3813 | crypto_free_shash(peer_device->connection->csums_tfm); |
| 3814 | 3814 | peer_device->connection->csums_tfm = csums_tfm; |
| @@ -621,7 +621,8 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what, · 620 unmodified lines | ||
| 621 | 621 | break; |
| 622 | 622 | |
| 623 | 623 | case READ_COMPLETED_WITH_ERROR: |
| 624 | − drbd_set_out_of_sync(peer_device, req->i.sector, req->i.size); | |
| 624 | + drbd_set_out_of_sync(first_peer_device(device), | |
| 625 | + req->i.sector, req->i.size); | |
| 625 | 626 | drbd_report_io_error(device, req); |
| 626 | 627 | __drbd_chk_io_error(device, DRBD_READ_ERROR); |
| 627 | 628 | fallthrough; |
| @@ -4443,7 +4443,9 @@ static int ublk_ctrl_start_dev(struct ublk_device *ub, · 4442 unmodified lines | ||
| 4443 | 4443 | |
| 4444 | 4444 | /* Skip partition scan if disabled by user */ |
| 4445 | 4445 | if (ub->dev_info.flags & UBLK_F_NO_AUTO_PART_SCAN) { |
| 4446 | − clear_bit(GD_SUPPRESS_PART_SCAN, &disk->state); | |
| 4446 | + /* Not clear for unprivileged daemons, see comment above */ | |
| 4447 | + if (!ub->unprivileged_daemons) | |
| 4448 | + clear_bit(GD_SUPPRESS_PART_SCAN, &disk->state); | |
| 4447 | 4449 | } else { |
| 4448 | 4450 | /* Schedule async partition scan for trusted daemons */ |
| 4449 | 4451 | if (!ub->unprivileged_daemons) |
| @@ -5006,15 +5008,22 @@ static int ublk_ctrl_get_features(const struct ublksrv_ctrl_cmd *header) · 556 unmodified lines | ||
| 5006 | 5008 | return 0; |
| 5007 | 5009 | } |
| 5008 | 5010 | |
| 5009 | −static void ublk_ctrl_set_size(struct ublk_device *ub, const struct ublksrv_ctrl_cmd *header) | |
| 5011 | +static int ublk_ctrl_set_size(struct ublk_device *ub, const struct ublksrv_ctrl_cmd *header) | |
| 5010 | 5012 | { |
| 5011 | 5013 | struct ublk_param_basic *p = &ub->params.basic; |
| 5012 | 5014 | u64 new_size = header->data[0]; |
| 5015 | + int ret = 0; | |
| 5013 | 5016 | |
| 5014 | 5017 | mutex_lock(&ub->mutex); |
| 5018 | + if (!ub->ub_disk) { | |
| 5019 | + ret = -ENODEV; | |
| 5020 | + goto out; | |
| 5021 | + } | |
| 5015 | 5022 | p->dev_sectors = new_size; |
| 5016 | 5023 | set_capacity_and_notify(ub->ub_disk, p->dev_sectors); |
| 5024 | +out: | |
| 5017 | 5025 | mutex_unlock(&ub->mutex); |
| 5026 | + return ret; | |
| 5018 | 5027 | } |
| 5019 | 5028 | |
| 5020 | 5029 | struct count_busy { |
| @@ -5335,8 +5344,7 @@ static int ublk_ctrl_uring_cmd(struct io_uring_cmd *cmd, · 314 unmodified lines | ||
| 5335 | 5344 | ret = ublk_ctrl_end_recovery(ub, &header); |
| 5336 | 5345 | break; |
| 5337 | 5346 | case UBLK_CMD_UPDATE_SIZE: |
| 5338 | − ublk_ctrl_set_size(ub, &header); | |
| 5339 | − ret = 0; | |
| 5347 | + ret = ublk_ctrl_set_size(ub, &header); | |
| 5340 | 5348 | break; |
| 5341 | 5349 | case UBLK_CMD_QUIESCE_DEV: |
| 5342 | 5350 | ret = ublk_ctrl_quiesce_dev(ub, &header); |
| @@ -542,6 +542,21 @@ out: · 541 unmodified lines | ||
| 542 | 542 | zloop_put_cmd(cmd); |
| 543 | 543 | } |
| 544 | 544 | |
| 545 | +/* | |
| 546 | + * Sync the entire FS containing the zone files instead of walking all files. | |
| 547 | + */ | |
| 548 | +static int zloop_flush(struct zloop_device *zlo) | |
| 549 | +{ | |
| 550 | + struct super_block *sb = file_inode(zlo->data_dir)->i_sb; | |
| 551 | + int ret; | |
| 552 | + | |
| 553 | + down_read(&sb->s_umount); | |
| 554 | + ret = sync_filesystem(sb); | |
| 555 | + up_read(&sb->s_umount); | |
| 556 | + | |
| 557 | + return ret; | |
| 558 | +} | |
| 559 | + | |
| 545 | 560 | static void zloop_handle_cmd(struct zloop_cmd *cmd) |
| 546 | 561 | { |
| 547 | 562 | struct request *rq = blk_mq_rq_from_pdu(cmd); |
| @@ -562,11 +577,7 @@ static void zloop_handle_cmd(struct zloop_cmd *cmd) · 14 unmodified lines | ||
| 562 | 577 | zloop_rw(cmd); |
| 563 | 578 | return; |
| 564 | 579 | case REQ_OP_FLUSH: |
| 565 | − /* | |
| 566 | − * Sync the entire FS containing the zone files instead of | |
| 567 | − * walking all files | |
| 568 | − */ | |
| 569 | − cmd->ret = sync_filesystem(file_inode(zlo->data_dir)->i_sb); | |
| 580 | + cmd->ret = zloop_flush(zlo); | |
| 570 | 581 | break; |
| 571 | 582 | case REQ_OP_ZONE_RESET: |
| 572 | 583 | cmd->ret = zloop_reset_zone(zlo, rq_zone_no(rq)); |
| @@ -981,7 +992,8 @@ static int zloop_ctl_add(struct zloop_options *opts) · 408 unmodified lines | ||
| 981 | 992 | struct queue_limits lim = { |
| 982 | 993 | .max_hw_sectors = SZ_1M >> SECTOR_SHIFT, |
| 983 | 994 | .chunk_sectors = opts->zone_size, |
| 984 | − .features = BLK_FEAT_ZONED, | |
| 995 | + .features = BLK_FEAT_ZONED | BLK_FEAT_WRITE_CACHE, | |
| 996 | + | |
| 985 | 997 | }; |
| 986 | 998 | unsigned int nr_zones, i, j; |
| 987 | 999 | struct zloop_device *zlo; |
| @@ -1162,7 +1174,12 @@ static int zloop_ctl_remove(struct zloop_options *opts) · 174 unmodified lines | ||
| 1162 | 1174 | int ret; |
| 1163 | 1175 | |
| 1164 | 1176 | if (!(opts->mask & ZLOOP_OPT_ID)) { |
| 1165 | − pr_err("No ID specified\n"); | |
| 1177 | + pr_err("No ID specified for remove\n"); | |
| 1178 | + return -EINVAL; | |
| 1179 | + } | |
| 1180 | + | |
| 1181 | + if (opts->mask & ~ZLOOP_OPT_ID) { | |
| 1182 | + pr_err("Invalid option specified for remove\n"); | |
| 1166 | 1183 | return -EINVAL; |
| 1167 | 1184 | } |
| 1168 | 1185 | |
| @@ -549,7 +549,7 @@ static ssize_t bd_stat_show(struct device *dev, struct device_attribute *attr, · 548 unmodified lines | ||
| 549 | 549 | return ret; |
| 550 | 550 | } |
| 551 | 551 | |
| 552 | −static ssize_t writeback_compressed_store(struct device *dev, | |
| 552 | +static ssize_t compressed_writeback_store(struct device *dev, | |
| 553 | 553 | struct device_attribute *attr, |
| 554 | 554 | const char *buf, size_t len) |
| 555 | 555 | { |
| @@ -564,12 +564,12 @@ static ssize_t writeback_compressed_store(struct device *dev, · 8 unmodified lines | ||
| 564 | 564 | return -EBUSY; |
| 565 | 565 | } |
| 566 | 566 | |
| 567 | − zram->wb_compressed = val; | |
| 567 | + zram->compressed_wb = val; | |
| 568 | 568 | |
| 569 | 569 | return len; |
| 570 | 570 | } |
| 571 | 571 | |
| 572 | −static ssize_t writeback_compressed_show(struct device *dev, | |
| 572 | +static ssize_t compressed_writeback_show(struct device *dev, | |
| 573 | 573 | struct device_attribute *attr, |
| 574 | 574 | char *buf) |
| 575 | 575 | { |
| @@ -577,7 +577,7 @@ static ssize_t writeback_compressed_show(struct device *dev, · 1 unmodified line | ||
| 577 | 577 | struct zram *zram = dev_to_zram(dev); |
| 578 | 578 | |
| 579 | 579 | guard(rwsem_read)(&zram->dev_lock); |
| 580 | − val = zram->wb_compressed; | |
| 580 | + val = zram->compressed_wb; | |
| 581 | 581 | |
| 582 | 582 | return sysfs_emit(buf, "%d\n", val); |
| 583 | 583 | } |
| @@ -946,7 +946,7 @@ static int zram_writeback_complete(struct zram *zram, struct zram_wb_req *req) · 362 unmodified lines | ||
| 946 | 946 | goto out; |
| 947 | 947 | } |
| 948 | 948 | |
| 949 | − if (zram->wb_compressed) { | |
| 949 | + if (zram->compressed_wb) { | |
| 950 | 950 | /* |
| 951 | 951 | * ZRAM_WB slots get freed, we need to preserve data required |
| 952 | 952 | * for read decompression. |
| @@ -960,7 +960,7 @@ static int zram_writeback_complete(struct zram *zram, struct zram_wb_req *req) · 7 unmodified lines | ||
| 960 | 960 | set_slot_flag(zram, index, ZRAM_WB); |
| 961 | 961 | set_slot_handle(zram, index, req->blk_idx); |
| 962 | 962 | |
| 963 | − if (zram->wb_compressed) { | |
| 963 | + if (zram->compressed_wb) { | |
| 964 | 964 | if (huge) |
| 965 | 965 | set_slot_flag(zram, index, ZRAM_HUGE); |
| 966 | 966 | set_slot_size(zram, index, size); |
| @@ -1100,7 +1100,7 @@ static int zram_writeback_slots(struct zram *zram, · 133 unmodified lines | ||
| 1100 | 1100 | */ |
| 1101 | 1101 | if (!test_slot_flag(zram, index, ZRAM_PP_SLOT)) |
| 1102 | 1102 | goto next; |
| 1103 | − if (zram->wb_compressed) | |
| 1103 | + if (zram->compressed_wb) | |
| 1104 | 1104 | err = read_from_zspool_raw(zram, req->page, index); |
| 1105 | 1105 | else |
| 1106 | 1106 | err = read_from_zspool(zram, req->page, index); |
| @@ -1429,7 +1429,7 @@ static void zram_async_read_endio(struct bio *bio) · 322 unmodified lines | ||
| 1429 | 1429 | * |
| 1430 | 1430 | * Keep the existing behavior for now. |
| 1431 | 1431 | */ |
| 1432 | − if (zram->wb_compressed == false) { | |
| 1432 | + if (zram->compressed_wb == false) { | |
| 1433 | 1433 | /* No decompression needed, complete the parent IO */ |
| 1434 | 1434 | bio_endio(req->parent); |
| 1435 | 1435 | bio_put(bio); |
| @@ -1508,7 +1508,7 @@ static int read_from_bdev_sync(struct zram *zram, struct page *page, u32 index, · 72 unmodified lines | ||
| 1508 | 1508 | flush_work(&req.work); |
| 1509 | 1509 | destroy_work_on_stack(&req.work); |
| 1510 | 1510 | |
| 1511 | − if (req.error || zram->wb_compressed == false) | |
| 1511 | + if (req.error || zram->compressed_wb == false) | |
| 1512 | 1512 | return req.error; |
| 1513 | 1513 | |
| 1514 | 1514 | return decompress_bdev_page(zram, page, index); |
| @@ -3007,7 +3007,7 @@ static DEVICE_ATTR_WO(writeback); · 1492 unmodified lines | ||
| 3007 | 3007 | static DEVICE_ATTR_RW(writeback_limit); |
| 3008 | 3008 | static DEVICE_ATTR_RW(writeback_limit_enable); |
| 3009 | 3009 | static DEVICE_ATTR_RW(writeback_batch_size); |
| 3010 | −static DEVICE_ATTR_RW(writeback_compressed); | |
| 3010 | +static DEVICE_ATTR_RW(compressed_writeback); | |
| 3011 | 3011 | #endif |
| 3012 | 3012 | #ifdef CONFIG_ZRAM_MULTI_COMP |
| 3013 | 3013 | static DEVICE_ATTR_RW(recomp_algorithm); |
| @@ -3031,7 +3031,7 @@ static struct attribute *zram_disk_attrs[] = { · 17 unmodified lines | ||
| 3031 | 3031 | &dev_attr_writeback_limit.attr, |
| 3032 | 3032 | &dev_attr_writeback_limit_enable.attr, |
| 3033 | 3033 | &dev_attr_writeback_batch_size.attr, |
| 3034 | − &dev_attr_writeback_compressed.attr, | |
| 3034 | + &dev_attr_compressed_writeback.attr, | |
| 3035 | 3035 | #endif |
| 3036 | 3036 | &dev_attr_io_stat.attr, |
| 3037 | 3037 | &dev_attr_mm_stat.attr, |
| @@ -3091,7 +3091,7 @@ static int zram_add(void) · 53 unmodified lines | ||
| 3091 | 3091 | init_rwsem(&zram->dev_lock); |
| 3092 | 3092 | #ifdef CONFIG_ZRAM_WRITEBACK |
| 3093 | 3093 | zram->wb_batch_size = 32; |
| 3094 | − zram->wb_compressed = false; | |
| 3094 | + zram->compressed_wb = false; | |
| 3095 | 3095 | #endif |
| 3096 | 3096 | |
| 3097 | 3097 | /* gendisk structure */ |
| @@ -133,7 +133,7 @@ struct zram { · 132 unmodified lines | ||
| 133 | 133 | #ifdef CONFIG_ZRAM_WRITEBACK |
| 134 | 134 | struct file *backing_dev; |
| 135 | 135 | bool wb_limit_enable; |
| 136 | − bool wb_compressed; | |
| 136 | + bool compressed_wb; | |
| 137 | 137 | u32 wb_batch_size; |
| 138 | 138 | u64 bd_wb_limit; |
| 139 | 139 | struct block_device *bdev; |
| @@ -2046,19 +2046,23 @@ retry: · 2045 unmodified lines | ||
| 2046 | 2046 | } |
| 2047 | 2047 | |
| 2048 | 2048 | out: |
| 2049 | − if (ret && retries < MAX_INIT_RETRIES) { | |
| 2050 | − bt_dev_warn(hdev, "Retry BT power ON:%d", retries); | |
| 2049 | + if (ret) { | |
| 2051 | 2050 | qca_power_shutdown(hu); |
| 2052 | − if (hu->serdev) { | |
| 2053 | − serdev_device_close(hu->serdev); | |
| 2054 | − ret = serdev_device_open(hu->serdev); | |
| 2055 | − if (ret) { | |
| 2056 | − bt_dev_err(hdev, "failed to open port"); | |
| 2057 | − return ret; | |
| 2051 | + | |
| 2052 | + if (retries < MAX_INIT_RETRIES) { | |
| 2053 | + bt_dev_warn(hdev, "Retry BT power ON:%d", retries); | |
| 2054 | + if (hu->serdev) { | |
| 2055 | + serdev_device_close(hu->serdev); | |
| 2056 | + ret = serdev_device_open(hu->serdev); | |
| 2057 | + if (ret) { | |
| 2058 | + bt_dev_err(hdev, "failed to open port"); | |
| 2059 | + return ret; | |
| 2060 | + } | |
| 2058 | 2061 | } |
| 2062 | + retries++; | |
| 2063 | + goto retry; | |
| 2059 | 2064 | } |
| 2060 | − retries++; | |
| 2061 | − goto retry; | |
| 2065 | + return ret; | |
| 2062 | 2066 | } |
| 2063 | 2067 | |
| 2064 | 2068 | /* Setup bdaddr */ |
| @@ -202,11 +202,16 @@ static int ipmi_ipmb_slave_cb(struct i2c_client *client, · 201 unmodified lines | ||
| 202 | 202 | break; |
| 203 | 203 | |
| 204 | 204 | case I2C_SLAVE_READ_REQUESTED: |
| 205 | + *val = 0xff; | |
| 206 | + ipmi_ipmb_check_msg_done(iidev); | |
| 207 | + break; | |
| 208 | + | |
| 205 | 209 | case I2C_SLAVE_STOP: |
| 206 | 210 | ipmi_ipmb_check_msg_done(iidev); |
| 207 | 211 | break; |
| 208 | 212 | |
| 209 | 213 | case I2C_SLAVE_READ_PROCESSED: |
| 214 | + *val = 0xff; | |
| 210 | 215 | break; |
| 211 | 216 | } |
| 212 | 217 | |
| @@ -602,6 +602,22 @@ static int __ipmi_bmc_register(struct ipmi_smi *intf, · 601 unmodified lines | ||
| 602 | 602 | static int __scan_channels(struct ipmi_smi *intf, |
| 603 | 603 | struct ipmi_device_id *id, bool rescan); |
| 604 | 604 | |
| 605 | +static void ipmi_lock_xmit_msgs(struct ipmi_smi *intf, int run_to_completion, | |
| 606 | + unsigned long *flags) | |
| 607 | +{ | |
| 608 | + if (run_to_completion) | |
| 609 | + return; | |
| 610 | + spin_lock_irqsave(&intf->xmit_msgs_lock, *flags); | |
| 611 | +} | |
| 612 | + | |
| 613 | +static void ipmi_unlock_xmit_msgs(struct ipmi_smi *intf, int run_to_completion, | |
| 614 | + unsigned long *flags) | |
| 615 | +{ | |
| 616 | + if (run_to_completion) | |
| 617 | + return; | |
| 618 | + spin_unlock_irqrestore(&intf->xmit_msgs_lock, *flags); | |
| 619 | +} | |
| 620 | + | |
| 605 | 621 | static void free_ipmi_user(struct kref *ref) |
| 606 | 622 | { |
| 607 | 623 | struct ipmi_user *user = container_of(ref, struct ipmi_user, refcount); |
| @@ -1869,21 +1885,32 @@ static struct ipmi_smi_msg *smi_add_send_msg(struct ipmi_smi *intf, · 1261 unmodified lines | ||
| 1869 | 1885 | return smi_msg; |
| 1870 | 1886 | } |
| 1871 | 1887 | |
| 1872 | −static void smi_send(struct ipmi_smi *intf, | |
| 1888 | +static int smi_send(struct ipmi_smi *intf, | |
| 1873 | 1889 | const struct ipmi_smi_handlers *handlers, |
| 1874 | 1890 | struct ipmi_smi_msg *smi_msg, int priority) |
| 1875 | 1891 | { |
| 1876 | 1892 | int run_to_completion = READ_ONCE(intf->run_to_completion); |
| 1877 | 1893 | unsigned long flags = 0; |
| 1894 | + int rv = 0; | |
| 1878 | 1895 | |
| 1879 | − if (!run_to_completion) | |
| 1880 | − spin_lock_irqsave(&intf->xmit_msgs_lock, flags); | |
| 1896 | + ipmi_lock_xmit_msgs(intf, run_to_completion, &flags); | |
| 1881 | 1897 | smi_msg = smi_add_send_msg(intf, smi_msg, priority); |
| 1882 | − if (!run_to_completion) | |
| 1883 | − spin_unlock_irqrestore(&intf->xmit_msgs_lock, flags); | |
| 1898 | + ipmi_unlock_xmit_msgs(intf, run_to_completion, &flags); | |
| 1884 | 1899 | |
| 1885 | − if (smi_msg) | |
| 1886 | − handlers->sender(intf->send_info, smi_msg); | |
| 1900 | + if (smi_msg) { | |
| 1901 | + rv = handlers->sender(intf->send_info, smi_msg); | |
| 1902 | + if (rv) { | |
| 1903 | + ipmi_lock_xmit_msgs(intf, run_to_completion, &flags); | |
| 1904 | + intf->curr_msg = NULL; | |
| 1905 | + ipmi_unlock_xmit_msgs(intf, run_to_completion, &flags); | |
| 1906 | + /* | |
| 1907 | + * Something may have been added to the transmit | |
| 1908 | + * queue, so schedule a check for that. | |
| 1909 | + */ | |
| 1910 | + queue_work(system_wq, &intf->smi_work); | |
| 1911 | + } | |
| 1912 | + } | |
| 1913 | + return rv; | |
| 1887 | 1914 | } |
| 1888 | 1915 | |
| 1889 | 1916 | static bool is_maintenance_mode_cmd(struct kernel_ipmi_msg *msg) |
| @@ -2296,6 +2323,7 @@ static int i_ipmi_request(struct ipmi_user *user, · 406 unmodified lines | ||
| 2296 | 2323 | struct ipmi_recv_msg *recv_msg; |
| 2297 | 2324 | int run_to_completion = READ_ONCE(intf->run_to_completion); |
| 2298 | 2325 | int rv = 0; |
| 2326 | + bool in_seq_table = false; | |
| 2299 | 2327 | |
| 2300 | 2328 | if (supplied_recv) { |
| 2301 | 2329 | recv_msg = supplied_recv; |
| @@ -2349,33 +2377,50 @@ static int i_ipmi_request(struct ipmi_user *user, · 47 unmodified lines | ||
| 2349 | 2377 | rv = i_ipmi_req_ipmb(intf, addr, msgid, msg, smi_msg, recv_msg, |
| 2350 | 2378 | source_address, source_lun, |
| 2351 | 2379 | retries, retry_time_ms); |
| 2380 | + in_seq_table = true; | |
| 2352 | 2381 | } else if (is_ipmb_direct_addr(addr)) { |
| 2353 | 2382 | rv = i_ipmi_req_ipmb_direct(intf, addr, msgid, msg, smi_msg, |
| 2354 | 2383 | recv_msg, source_lun); |
| 2355 | 2384 | } else if (is_lan_addr(addr)) { |
| 2356 | 2385 | rv = i_ipmi_req_lan(intf, addr, msgid, msg, smi_msg, recv_msg, |
| 2357 | 2386 | source_lun, retries, retry_time_ms); |
| 2387 | + in_seq_table = true; | |
| 2358 | 2388 | } else { |
| 2359 | − /* Unknown address type. */ | |
| 2389 | + /* Unknown address type. */ | |
| 2360 | 2390 | ipmi_inc_stat(intf, sent_invalid_commands); |
| 2361 | 2391 | rv = -EINVAL; |
| 2362 | 2392 | } |
| 2363 | 2393 | |
| 2364 | − if (rv) { | |
| 2365 | −out_err: | |
| 2366 | − if (!supplied_smi) | |
| 2367 | − ipmi_free_smi_msg(smi_msg); | |
| 2368 | − if (!supplied_recv) | |
| 2369 | − ipmi_free_recv_msg(recv_msg); | |
| 2370 | − } else { | |
| 2394 | + if (!rv) { | |
| 2371 | 2395 | dev_dbg(intf->si_dev, "Send: %*ph\n", |
| 2372 | 2396 | smi_msg->data_size, smi_msg->data); |
| 2373 | 2397 | |
| 2374 | − smi_send(intf, intf->handlers, smi_msg, priority); | |
| 2398 | + rv = smi_send(intf, intf->handlers, smi_msg, priority); | |
| 2399 | + if (rv != IPMI_CC_NO_ERROR) | |
| 2400 | + /* smi_send() returns an IPMI err, return a Linux one. */ | |
| 2401 | + rv = -EIO; | |
| 2402 | + if (rv && in_seq_table) { | |
| 2403 | + /* | |
| 2404 | + * If it's in the sequence table, it will be | |
| 2405 | + * retried later, so ignore errors. | |
| 2406 | + */ | |
| 2407 | + rv = 0; | |
| 2408 | + /* But we need to fix the timeout. */ | |
| 2409 | + intf_start_seq_timer(intf, smi_msg->msgid); | |
| 2410 | + ipmi_free_smi_msg(smi_msg); | |
| 2411 | + smi_msg = NULL; | |
| 2412 | + } | |
| 2375 | 2413 | } |
| 2414 | +out_err: | |
| 2376 | 2415 | if (!run_to_completion) |
| 2377 | 2416 | mutex_unlock(&intf->users_mutex); |
| 2378 | 2417 | |
| 2418 | + if (rv) { | |
| 2419 | + if (!supplied_smi) | |
| 2420 | + ipmi_free_smi_msg(smi_msg); | |
| 2421 | + if (!supplied_recv) | |
| 2422 | + ipmi_free_recv_msg(recv_msg); | |
| 2423 | + } | |
| 2379 | 2424 | return rv; |
| 2380 | 2425 | } |
| 2381 | 2426 | |
| @@ -3949,12 +3994,12 @@ static int handle_ipmb_get_msg_cmd(struct ipmi_smi *intf, · 1567 unmodified lines | ||
| 3949 | 3994 | dev_dbg(intf->si_dev, "Invalid command: %*ph\n", |
| 3950 | 3995 | msg->data_size, msg->data); |
| 3951 | 3996 | |
| 3952 | − smi_send(intf, intf->handlers, msg, 0); | |
| 3953 | − /* | |
| 3954 | − * We used the message, so return the value that | |
| 3955 | − * causes it to not be freed or queued. | |
| 3956 | − */ | |
| 3957 | − rv = -1; | |
| 3997 | + if (smi_send(intf, intf->handlers, msg, 0) == IPMI_CC_NO_ERROR) | |
| 3998 | + /* | |
| 3999 | + * We used the message, so return the value that | |
| 4000 | + * causes it to not be freed or queued. | |
| 4001 | + */ | |
| 4002 | + rv = -1; | |
| 3958 | 4003 | } else if (!IS_ERR(recv_msg)) { |
| 3959 | 4004 | /* Extract the source address from the data. */ |
| 3960 | 4005 | ipmb_addr = (struct ipmi_ipmb_addr *) &recv_msg->addr; |
| @@ -4028,12 +4073,12 @@ static int handle_ipmb_direct_rcv_cmd(struct ipmi_smi *intf, · 67 unmodified lines | ||
| 4028 | 4073 | msg->data[4] = IPMI_INVALID_CMD_COMPLETION_CODE; |
| 4029 | 4074 | msg->data_size = 5; |
| 4030 | 4075 | |
| 4031 | − smi_send(intf, intf->handlers, msg, 0); | |
| 4032 | − /* | |
| 4033 | − * We used the message, so return the value that | |
| 4034 | − * causes it to not be freed or queued. | |
| 4035 | − */ | |
| 4036 | − rv = -1; | |
| 4076 | + if (smi_send(intf, intf->handlers, msg, 0) == IPMI_CC_NO_ERROR) | |
| 4077 | + /* | |
| 4078 | + * We used the message, so return the value that | |
| 4079 | + * causes it to not be freed or queued. | |
| 4080 | + */ | |
| 4081 | + rv = -1; | |
| 4037 | 4082 | } else if (!IS_ERR(recv_msg)) { |
| 4038 | 4083 | /* Extract the source address from the data. */ |
| 4039 | 4084 | daddr = (struct ipmi_ipmb_direct_addr *)&recv_msg->addr; |
| @@ -4173,7 +4218,7 @@ static int handle_lan_get_msg_cmd(struct ipmi_smi *intf, · 133 unmodified lines | ||
| 4173 | 4218 | struct ipmi_smi_msg *msg) |
| 4174 | 4219 | { |
| 4175 | 4220 | struct cmd_rcvr *rcvr; |
| 4176 | − int rv = 0; | |
| 4221 | + int rv = 0; /* Free by default */ | |
| 4177 | 4222 | unsigned char netfn; |
| 4178 | 4223 | unsigned char cmd; |
| 4179 | 4224 | unsigned char chan; |
| @@ -4226,12 +4271,12 @@ static int handle_lan_get_msg_cmd(struct ipmi_smi *intf, · 46 unmodified lines | ||
| 4226 | 4271 | dev_dbg(intf->si_dev, "Invalid command: %*ph\n", |
| 4227 | 4272 | msg->data_size, msg->data); |
| 4228 | 4273 | |
| 4229 | − smi_send(intf, intf->handlers, msg, 0); | |
| 4230 | − /* | |
| 4231 | − * We used the message, so return the value that | |
| 4232 | − * causes it to not be freed or queued. | |
| 4233 | − */ | |
| 4234 | − rv = -1; | |
| 4274 | + if (smi_send(intf, intf->handlers, msg, 0) == IPMI_CC_NO_ERROR) | |
| 4275 | + /* | |
| 4276 | + * We used the message, so return the value that | |
| 4277 | + * causes it to not be freed or queued. | |
| 4278 | + */ | |
| 4279 | + rv = -1; | |
| 4235 | 4280 | } else if (!IS_ERR(recv_msg)) { |
| 4236 | 4281 | /* Extract the source address from the data. */ |
| 4237 | 4282 | lan_addr = (struct ipmi_lan_addr *) &recv_msg->addr; |
| @@ -4824,8 +4869,7 @@ static void smi_work(struct work_struct *t) · 586 unmodified lines | ||
| 4824 | 4869 | * message delivery. |
| 4825 | 4870 | */ |
| 4826 | 4871 | restart: |
| 4827 | − if (!run_to_completion) | |
| 4828 | − spin_lock_irqsave(&intf->xmit_msgs_lock, flags); | |
| 4872 | + ipmi_lock_xmit_msgs(intf, run_to_completion, &flags); | |
| 4829 | 4873 | if (intf->curr_msg == NULL && !intf->in_shutdown) { |
| 4830 | 4874 | struct list_head *entry = NULL; |
| 4831 | 4875 | |
| @@ -4841,8 +4885,7 @@ restart: · 9 unmodified lines | ||
| 4841 | 4885 | intf->curr_msg = newmsg; |
| 4842 | 4886 | } |
| 4843 | 4887 | } |
| 4844 | − if (!run_to_completion) | |
| 4845 | − spin_unlock_irqrestore(&intf->xmit_msgs_lock, flags); | |
| 4888 | + ipmi_unlock_xmit_msgs(intf, run_to_completion, &flags); | |
| 4846 | 4889 | |
| 4847 | 4890 | if (newmsg) { |
| 4848 | 4891 | cc = intf->handlers->sender(intf->send_info, newmsg); |
| @@ -4850,8 +4893,11 @@ restart: · 1 unmodified line | ||
| 4850 | 4893 | if (newmsg->recv_msg) |
| 4851 | 4894 | deliver_err_response(intf, |
| 4852 | 4895 | newmsg->recv_msg, cc); |
| 4853 | − else | |
| 4854 | − ipmi_free_smi_msg(newmsg); | |
| 4896 | + ipmi_lock_xmit_msgs(intf, run_to_completion, &flags); | |
| 4897 | + intf->curr_msg = NULL; | |
| 4898 | + ipmi_unlock_xmit_msgs(intf, run_to_completion, &flags); | |
| 4899 | + ipmi_free_smi_msg(newmsg); | |
| 4900 | + newmsg = NULL; | |
| 4855 | 4901 | goto restart; |
| 4856 | 4902 | } |
| 4857 | 4903 | } |
| @@ -4919,16 +4965,14 @@ void ipmi_smi_msg_received(struct ipmi_smi *intf, · 61 unmodified lines | ||
| 4919 | 4965 | spin_unlock_irqrestore(&intf->waiting_rcv_msgs_lock, |
| 4920 | 4966 | flags); |
| 4921 | 4967 | |
| 4922 | − if (!run_to_completion) | |
| 4923 | − spin_lock_irqsave(&intf->xmit_msgs_lock, flags); | |
| 4968 | + ipmi_lock_xmit_msgs(intf, run_to_completion, &flags); | |
| 4924 | 4969 | /* |
| 4925 | 4970 | * We can get an asynchronous event or receive message in addition |
| 4926 | 4971 | * to commands we send. |
| 4927 | 4972 | */ |
| 4928 | 4973 | if (msg == intf->curr_msg) |
| 4929 | 4974 | intf->curr_msg = NULL; |
| 4930 | − if (!run_to_completion) | |
| 4931 | − spin_unlock_irqrestore(&intf->xmit_msgs_lock, flags); | |
| 4975 | + ipmi_unlock_xmit_msgs(intf, run_to_completion, &flags); | |
| 4932 | 4976 | |
| 4933 | 4977 | if (run_to_completion) |
| 4934 | 4978 | smi_work(&intf->smi_work); |
| @@ -5041,7 +5085,12 @@ static void check_msg_timeout(struct ipmi_smi *intf, struct seq_table *ent, · 106 unmodified lines | ||
| 5041 | 5085 | ipmi_inc_stat(intf, |
| 5042 | 5086 | retransmitted_ipmb_commands); |
| 5043 | 5087 | |
| 5044 | − smi_send(intf, intf->handlers, smi_msg, 0); | |
| 5088 | + /* If this fails we'll retry later or timeout. */ | |
| 5089 | + if (smi_send(intf, intf->handlers, smi_msg, 0) != IPMI_CC_NO_ERROR) { | |
| 5090 | + /* But fix the timeout. */ | |
| 5091 | + intf_start_seq_timer(intf, smi_msg->msgid); | |
| 5092 | + ipmi_free_smi_msg(smi_msg); | |
| 5093 | + } | |
| 5045 | 5094 | } else |
| 5046 | 5095 | ipmi_free_smi_msg(smi_msg); |
| 5047 | 5096 | |
| @@ -809,6 +809,12 @@ restart: · 808 unmodified lines | ||
| 809 | 809 | */ |
| 810 | 810 | return_hosed_msg(smi_info, IPMI_BUS_ERR); |
| 811 | 811 | } |
| 812 | + if (smi_info->waiting_msg != NULL) { | |
| 813 | + /* Also handle if there was a message waiting. */ | |
| 814 | + smi_info->curr_msg = smi_info->waiting_msg; | |
| 815 | + smi_info->waiting_msg = NULL; | |
| 816 | + return_hosed_msg(smi_info, IPMI_BUS_ERR); | |
| 817 | + } | |
| 812 | 818 | smi_mod_timer(smi_info, jiffies + SI_TIMEOUT_HOSED); |
| 813 | 819 | goto out; |
| 814 | 820 | } |
| @@ -918,9 +924,14 @@ static int sender(void *send_info, struct ipmi_smi_msg *msg) · 103 unmodified lines | ||
| 918 | 924 | { |
| 919 | 925 | struct smi_info *smi_info = send_info; |
| 920 | 926 | unsigned long flags; |
| 927 | + int rv = IPMI_CC_NO_ERROR; | |
| 921 | 928 | |
| 922 | 929 | debug_timestamp(smi_info, "Enqueue"); |
| 923 | 930 | |
| 931 | + /* | |
| 932 | + * Check here for run to completion mode. A check under lock is | |
| 933 | + * later. | |
| 934 | + */ | |
| 924 | 935 | if (smi_info->si_state == SI_HOSED) |
| 925 | 936 | return IPMI_BUS_ERR; |
| 926 | 937 | |
| @@ -934,18 +945,15 @@ static int sender(void *send_info, struct ipmi_smi_msg *msg) · 7 unmodified lines | ||
| 934 | 945 | } |
| 935 | 946 | |
| 936 | 947 | spin_lock_irqsave(&smi_info->si_lock, flags); |
| 937 | − /* | |
| 938 | − * The following two lines don't need to be under the lock for | |
| 939 | − * the lock's sake, but they do need SMP memory barriers to | |
| 940 | − * avoid getting things out of order. We are already claiming | |
| 941 | − * the lock, anyway, so just do it under the lock to avoid the | |
| 942 | − * ordering problem. | |
| 943 | − */ | |
| 944 | − BUG_ON(smi_info->waiting_msg); | |
| 945 | − smi_info->waiting_msg = msg; | |
| 946 | − check_start_timer_thread(smi_info); | |
| 948 | + if (smi_info->si_state == SI_HOSED) { | |
| 949 | + rv = IPMI_BUS_ERR; | |
| 950 | + } else { | |
| 951 | + BUG_ON(smi_info->waiting_msg); | |
| 952 | + smi_info->waiting_msg = msg; | |
| 953 | + check_start_timer_thread(smi_info); | |
| 954 | + } | |
| 947 | 955 | spin_unlock_irqrestore(&smi_info->si_lock, flags); |
| 948 | − return IPMI_CC_NO_ERROR; | |
| 956 | + return rv; | |
| 949 | 957 | } |
| 950 | 958 | |
| 951 | 959 | static void set_run_to_completion(void *send_info, bool i_run_to_completion) |
| @@ -1113,7 +1121,9 @@ static void smi_timeout(struct timer_list *t) · 161 unmodified lines | ||
| 1113 | 1121 | * SI_USEC_PER_JIFFY); |
| 1114 | 1122 | smi_result = smi_event_handler(smi_info, time_diff); |
| 1115 | 1123 | |
| 1116 | − if ((smi_info->io.irq) && (!smi_info->interrupt_disabled)) { | |
| 1124 | + if (smi_info->si_state == SI_HOSED) { | |
| 1125 | + timeout = jiffies + SI_TIMEOUT_HOSED; | |
| 1126 | + } else if ((smi_info->io.irq) && (!smi_info->interrupt_disabled)) { | |
| 1117 | 1127 | /* Running with interrupts, only do long timeouts. */ |
| 1118 | 1128 | timeout = jiffies + SI_TIMEOUT_JIFFIES; |
| 1119 | 1129 | smi_inc_stat(smi_info, long_timeouts); |
| @@ -2226,7 +2236,8 @@ static void wait_msg_processed(struct smi_info *smi_info) · 1106 unmodified lines | ||
| 2226 | 2236 | unsigned long jiffies_now; |
| 2227 | 2237 | long time_diff; |
| 2228 | 2238 | |
| 2229 | − while (smi_info->curr_msg || (smi_info->si_state != SI_NORMAL)) { | |
| 2239 | + while (smi_info->si_state != SI_HOSED && | |
| 2240 | + (smi_info->curr_msg || (smi_info->si_state != SI_NORMAL))) { | |
| 2230 | 2241 | jiffies_now = jiffies; |
| 2231 | 2242 | time_diff = (((long)jiffies_now - (long)smi_info->last_timeout_jiffies) |
| 2232 | 2243 | * SI_USEC_PER_JIFFY); |
| @@ -168,7 +168,7 @@ static void ipmi_ls2k_remove(struct platform_device *pdev) · 167 unmodified lines | ||
| 168 | 168 | ipmi_si_remove_by_dev(&pdev->dev); |
| 169 | 169 | } |
| 170 | 170 | |
| 171 | −struct platform_driver ipmi_ls2k_platform_driver = { | |
| 171 | +static struct platform_driver ipmi_ls2k_platform_driver = { | |
| 172 | 172 | .driver = { |
| 173 | 173 | .name = "ls2k-ipmi-si", |
| 174 | 174 | }, |
| @@ -96,8 +96,7 @@ static ATOMIC_NOTIFIER_HEAD(random_ready_notifier); · 95 unmodified lines | ||
| 96 | 96 | /* Control how we warn userspace. */ |
| 97 | 97 | static struct ratelimit_state urandom_warning = |
| 98 | 98 | RATELIMIT_STATE_INIT_FLAGS("urandom_warning", HZ, 3, RATELIMIT_MSG_ON_RELEASE); |
| 99 | −static int ratelimit_disable __read_mostly = | |
| 100 | − IS_ENABLED(CONFIG_WARN_ALL_UNSEEDED_RANDOM); | |
| 99 | +static int ratelimit_disable __read_mostly = 0; | |
| 101 | 100 | module_param_named(ratelimit_disable, ratelimit_disable, int, 0644); |
| 102 | 101 | MODULE_PARM_DESC(ratelimit_disable, "Disable random ratelimit suppression"); |
| 103 | 102 | |
| @@ -168,12 +167,6 @@ int __cold execute_with_initialized_rng(struct notifier_block *nb) · 64 unmodified lines | ||
| 168 | 167 | return ret; |
| 169 | 168 | } |
| 170 | 169 | |
| 171 | −#define warn_unseeded_randomness() \ | |
| 172 | − if (IS_ENABLED(CONFIG_WARN_ALL_UNSEEDED_RANDOM) && !crng_ready()) \ | |
| 173 | − printk_deferred(KERN_NOTICE "random: %s called from %pS with crng_init=%d\n", \ | |
| 174 | − __func__, (void *)_RET_IP_, crng_init) | |
| 175 | − | |
| 176 | − | |
| 177 | 170 | /********************************************************************* |
| 178 | 171 | * |
| 179 | 172 | * Fast key erasure RNG, the "crng". |
| @@ -434,7 +427,6 @@ static void _get_random_bytes(void *buf, size_t len) · 254 unmodified lines | ||
| 434 | 427 | */ |
| 435 | 428 | void get_random_bytes(void *buf, size_t len) |
| 436 | 429 | { |
| 437 | − warn_unseeded_randomness(); | |
| 438 | 430 | _get_random_bytes(buf, len); |
| 439 | 431 | } |
| 440 | 432 | EXPORT_SYMBOL(get_random_bytes); |
| @@ -523,8 +515,6 @@ type get_random_ ##type(void) \ · 82 unmodified lines | ||
| 523 | 515 | struct batch_ ##type *batch; \ |
| 524 | 516 | unsigned long next_gen; \ |
| 525 | 517 | \ |
| 526 | − warn_unseeded_randomness(); \ | |
| 527 | − \ | |
| 528 | 518 | if (!crng_ready()) { \ |
| 529 | 519 | _get_random_bytes(&ret, sizeof(ret)); \ |
| 530 | 520 | return ret; \ |
| @@ -346,7 +346,29 @@ static struct platform_driver imx8qxp_clk_driver = { · 345 unmodified lines | ||
| 346 | 346 | }, |
| 347 | 347 | .probe = imx8qxp_clk_probe, |
| 348 | 348 | }; |
| 349 | −module_platform_driver(imx8qxp_clk_driver); | |
| 349 | + | |
| 350 | +static int __init imx8qxp_clk_init(void) | |
| 351 | +{ | |
| 352 | + int ret; | |
| 353 | + | |
| 354 | + ret = platform_driver_register(&imx8qxp_clk_driver); | |
| 355 | + if (ret) | |
| 356 | + return ret; | |
| 357 | + | |
| 358 | + ret = imx_clk_scu_module_init(); | |
| 359 | + if (ret) | |
| 360 | + platform_driver_unregister(&imx8qxp_clk_driver); | |
| 361 | + | |
| 362 | + return ret; | |
| 363 | +} | |
| 364 | +module_init(imx8qxp_clk_init); | |
| 365 | + | |
| 366 | +static void __exit imx8qxp_clk_exit(void) | |
| 367 | +{ | |
| 368 | + imx_clk_scu_module_exit(); | |
| 369 | + platform_driver_unregister(&imx8qxp_clk_driver); | |
| 370 | +} | |
| 371 | +module_exit(imx8qxp_clk_exit); | |
| 350 | 372 | |
| 351 | 373 | MODULE_AUTHOR("Aisheng Dong <aisheng.dong@nxp.com>"); |
| 352 | 374 | MODULE_DESCRIPTION("NXP i.MX8QXP clock driver"); |
| @@ -191,6 +191,16 @@ static bool imx_scu_clk_is_valid(u32 rsrc_id) · 190 unmodified lines | ||
| 191 | 191 | return p != NULL; |
| 192 | 192 | } |
| 193 | 193 | |
| 194 | +int __init imx_clk_scu_module_init(void) | |
| 195 | +{ | |
| 196 | + return platform_driver_register(&imx_clk_scu_driver); | |
| 197 | +} | |
| 198 | + | |
| 199 | +void __exit imx_clk_scu_module_exit(void) | |
| 200 | +{ | |
| 201 | + return platform_driver_unregister(&imx_clk_scu_driver); | |
| 202 | +} | |
| 203 | + | |
| 194 | 204 | int imx_clk_scu_init(struct device_node *np, |
| 195 | 205 | const struct imx_clk_scu_rsrc_table *data) |
| 196 | 206 | { |
| @@ -215,7 +225,7 @@ int imx_clk_scu_init(struct device_node *np, · 18 unmodified lines | ||
| 215 | 225 | rsrc_table = data; |
| 216 | 226 | } |
| 217 | 227 | |
| 218 | − return platform_driver_register(&imx_clk_scu_driver); | |
| 228 | + return 0; | |
| 219 | 229 | } |
| 220 | 230 | |
| 221 | 231 | /* |
| @@ -25,6 +25,8 @@ extern const struct imx_clk_scu_rsrc_table imx_clk_scu_rsrc_imx8dxl; · 24 unmodified lines | ||
| 25 | 25 | extern const struct imx_clk_scu_rsrc_table imx_clk_scu_rsrc_imx8qxp; |
| 26 | 26 | extern const struct imx_clk_scu_rsrc_table imx_clk_scu_rsrc_imx8qm; |
| 27 | 27 | |
| 28 | +int __init imx_clk_scu_module_init(void); | |
| 29 | +void __exit imx_clk_scu_module_exit(void); | |
| 28 | 30 | int imx_clk_scu_init(struct device_node *np, |
| 29 | 31 | const struct imx_clk_scu_rsrc_table *data); |
| 30 | 32 | struct clk_hw *imx_scu_of_clk_src_get(struct of_phandle_args *clkspec, |
| @@ -1476,13 +1476,13 @@ static void __intel_pstate_update_max_freq(struct cpufreq_policy *policy, · 1475 unmodified lines | ||
| 1476 | 1476 | refresh_frequency_limits(policy); |
| 1477 | 1477 | } |
| 1478 | 1478 | |
| 1479 | −static bool intel_pstate_update_max_freq(struct cpudata *cpudata) | |
| 1479 | +static bool intel_pstate_update_max_freq(int cpu) | |
| 1480 | 1480 | { |
| 1481 | − struct cpufreq_policy *policy __free(put_cpufreq_policy) = cpufreq_cpu_get(cpudata->cpu); | |
| 1481 | + struct cpufreq_policy *policy __free(put_cpufreq_policy) = cpufreq_cpu_get(cpu); | |
| 1482 | 1482 | if (!policy) |
| 1483 | 1483 | return false; |
| 1484 | 1484 | |
| 1485 | − __intel_pstate_update_max_freq(policy, cpudata); | |
| 1485 | + __intel_pstate_update_max_freq(policy, all_cpu_data[cpu]); | |
| 1486 | 1486 | |
| 1487 | 1487 | return true; |
| 1488 | 1488 | } |
| @@ -1501,7 +1501,7 @@ static void intel_pstate_update_limits_for_all(void) · 12 unmodified lines | ||
| 1501 | 1501 | int cpu; |
| 1502 | 1502 | |
| 1503 | 1503 | for_each_possible_cpu(cpu) |
| 1504 | − intel_pstate_update_max_freq(all_cpu_data[cpu]); | |
| 1504 | + intel_pstate_update_max_freq(cpu); | |
| 1505 | 1505 | |
| 1506 | 1506 | mutex_lock(&hybrid_capacity_lock); |
| 1507 | 1507 | |
| @@ -1647,8 +1647,8 @@ static ssize_t store_no_turbo(struct kobject *a, struct kobj_attribute *b, · 139 unmodified lines | ||
| 1647 | 1647 | static void update_cpu_qos_request(int cpu, enum freq_qos_req_type type) |
| 1648 | 1648 | { |
| 1649 | 1649 | struct cpudata *cpudata = all_cpu_data[cpu]; |
| 1650 | − unsigned int freq = cpudata->pstate.turbo_freq; | |
| 1651 | 1650 | struct freq_qos_request *req; |
| 1651 | + unsigned int freq; | |
| 1652 | 1652 | |
| 1653 | 1653 | struct cpufreq_policy *policy __free(put_cpufreq_policy) = cpufreq_cpu_get(cpu); |
| 1654 | 1654 | if (!policy) |
| @@ -1661,6 +1661,8 @@ static void update_cpu_qos_request(int cpu, enum freq_qos_req_type type) · 6 unmodified lines | ||
| 1661 | 1661 | if (hwp_active) |
| 1662 | 1662 | intel_pstate_get_hwp_cap(cpudata); |
| 1663 | 1663 | |
| 1664 | + freq = cpudata->pstate.turbo_freq; | |
| 1665 | + | |
| 1664 | 1666 | if (type == FREQ_QOS_MIN) { |
| 1665 | 1667 | freq = DIV_ROUND_UP(freq * global.min_perf_pct, 100); |
| 1666 | 1668 | } else { |
| @@ -1908,7 +1910,7 @@ static void intel_pstate_notify_work(struct work_struct *work) · 241 unmodified lines | ||
| 1908 | 1910 | struct cpudata *cpudata = |
| 1909 | 1911 | container_of(to_delayed_work(work), struct cpudata, hwp_notify_work); |
| 1910 | 1912 | |
| 1911 | − if (intel_pstate_update_max_freq(cpudata)) { | |
| 1913 | + if (intel_pstate_update_max_freq(cpudata->cpu)) { | |
| 1912 | 1914 | /* |
| 1913 | 1915 | * The driver will not be unregistered while this function is |
| 1914 | 1916 | * running, so update the capacity without acquiring the driver |
| @@ -359,16 +359,6 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev, · 358 unmodified lines | ||
| 359 | 359 | int cpuidle_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, |
| 360 | 360 | bool *stop_tick) |
| 361 | 361 | { |
| 362 | − /* | |
| 363 | − * If there is only a single idle state (or none), there is nothing | |
| 364 | − * meaningful for the governor to choose. Skip the governor and | |
| 365 | − * always use state 0 with the tick running. | |
| 366 | − */ | |
| 367 | − if (drv->state_count <= 1) { | |
| 368 | − *stop_tick = false; | |
| 369 | − return 0; | |
| 370 | − } | |
| 371 | − | |
| 372 | 362 | return cpuidle_curr_governor->select(drv, dev, stop_tick); |
| 373 | 363 | } |
| 374 | 364 | |
| @@ -52,9 +52,10 @@ static int atmel_sha204a_rng_read_nonblocking(struct hwrng *rng, void *data, · 51 unmodified lines | ||
| 52 | 52 | rng->priv = 0; |
| 53 | 53 | } else { |
| 54 | 54 | work_data = kmalloc_obj(*work_data, GFP_ATOMIC); |
| 55 | − if (!work_data) | |
| 55 | + if (!work_data) { | |
| 56 | + atomic_dec(&i2c_priv->tfm_count); | |
| 56 | 57 | return -ENOMEM; |
| 57 | − | |
| 58 | + } | |
| 58 | 59 | work_data->ctx = i2c_priv; |
| 59 | 60 | work_data->client = i2c_priv->client; |
| 60 | 61 | |
| @@ -378,9 +378,9 @@ void sev_tsm_init_locked(struct sev_device *sev, void *tio_status_page) · 377 unmodified lines | ||
| 378 | 378 | return; |
| 379 | 379 | |
| 380 | 380 | error_exit: |
| 381 | − kfree(t); | |
| 382 | 381 | pr_err("Failed to enable SEV-TIO: ret=%d en=%d initdone=%d SEV=%d\n", |
| 383 | 382 | ret, t->tio_en, t->tio_init_done, boot_cpu_has(X86_FEATURE_SEV)); |
| 383 | + kfree(t); | |
| 384 | 384 | } |
| 385 | 385 | |
| 386 | 386 | void sev_tsm_uninit(struct sev_device *sev) |
| @@ -1105,15 +1105,12 @@ struct page *snp_alloc_hv_fixed_pages(unsigned int num_2mb_pages) · 1104 unmodified lines | ||
| 1105 | 1105 | { |
| 1106 | 1106 | struct psp_device *psp_master = psp_get_master_device(); |
| 1107 | 1107 | struct snp_hv_fixed_pages_entry *entry; |
| 1108 | − struct sev_device *sev; | |
| 1109 | 1108 | unsigned int order; |
| 1110 | 1109 | struct page *page; |
| 1111 | 1110 | |
| 1112 | − if (!psp_master || !psp_master->sev_data) | |
| 1111 | + if (!psp_master) | |
| 1113 | 1112 | return NULL; |
| 1114 | 1113 | |
| 1115 | − sev = psp_master->sev_data; | |
| 1116 | − | |
| 1117 | 1114 | order = get_order(PMD_SIZE * num_2mb_pages); |
| 1118 | 1115 | |
| 1119 | 1116 | /* |
| @@ -1126,7 +1123,8 @@ struct page *snp_alloc_hv_fixed_pages(unsigned int num_2mb_pages) · 6 unmodified lines | ||
| 1126 | 1123 | * This API uses SNP_INIT_EX to transition allocated pages to HV_Fixed |
| 1127 | 1124 | * page state, fail if SNP is already initialized. |
| 1128 | 1125 | */ |
| 1129 | − if (sev->snp_initialized) | |
| 1126 | + if (psp_master->sev_data && | |
| 1127 | + ((struct sev_device *)psp_master->sev_data)->snp_initialized) | |
| 1130 | 1128 | return NULL; |
| 1131 | 1129 | |
| 1132 | 1130 | /* Re-use freed pages that match the request */ |
| @@ -1162,7 +1160,7 @@ void snp_free_hv_fixed_pages(struct page *page) · 29 unmodified lines | ||
| 1162 | 1160 | struct psp_device *psp_master = psp_get_master_device(); |
| 1163 | 1161 | struct snp_hv_fixed_pages_entry *entry, *nentry; |
| 1164 | 1162 | |
| 1165 | − if (!psp_master || !psp_master->sev_data) | |
| 1163 | + if (!psp_master) | |
| 1166 | 1164 | return; |
| 1167 | 1165 | |
| 1168 | 1166 | /* |
| @@ -152,6 +152,24 @@ int cxl_pci_get_bandwidth(struct pci_dev *pdev, struct access_coordinate *c); · 151 unmodified lines | ||
| 152 | 152 | int cxl_port_get_switch_dport_bandwidth(struct cxl_port *port, |
| 153 | 153 | struct access_coordinate *c); |
| 154 | 154 | |
| 155 | +static inline struct device *port_to_host(struct cxl_port *port) | |
| 156 | +{ | |
| 157 | + struct cxl_port *parent = is_cxl_root(port) ? NULL : | |
| 158 | + to_cxl_port(port->dev.parent); | |
| 159 | + | |
| 160 | + /* | |
| 161 | + * The host of CXL root port and the first level of ports is | |
| 162 | + * the platform firmware device, the host of all other ports | |
| 163 | + * is their parent port. | |
| 164 | + */ | |
| 165 | + if (!parent) | |
| 166 | + return port->uport_dev; | |
| 167 | + else if (is_cxl_root(parent)) | |
| 168 | + return parent->uport_dev; | |
| 169 | + else | |
| 170 | + return &parent->dev; | |
| 171 | +} | |
| 172 | + | |
| 155 | 173 | static inline struct device *dport_to_host(struct cxl_dport *dport) |
| 156 | 174 | { |
| 157 | 175 | struct cxl_port *port = dport->port; |
| @@ -904,7 +904,7 @@ static void cxl_decoder_reset(struct cxl_decoder *cxld) · 903 unmodified lines | ||
| 904 | 904 | if ((cxld->flags & CXL_DECODER_F_ENABLE) == 0) |
| 905 | 905 | return; |
| 906 | 906 | |
| 907 | − if (test_bit(CXL_DECODER_F_LOCK, &cxld->flags)) | |
| 907 | + if (cxld->flags & CXL_DECODER_F_LOCK) | |
| 908 | 908 | return; |
| 909 | 909 | |
| 910 | 910 | if (port->commit_end == id) |
| @@ -311,6 +311,7 @@ static bool cxl_mem_raw_command_allowed(u16 opcode) · 310 unmodified lines | ||
| 311 | 311 | * cxl_payload_from_user_allowed() - Check contents of in_payload. |
| 312 | 312 | * @opcode: The mailbox command opcode. |
| 313 | 313 | * @payload_in: Pointer to the input payload passed in from user space. |
| 314 | + * @in_size: Size of @payload_in in bytes. | |
| 314 | 315 | * |
| 315 | 316 | * Return: |
| 316 | 317 | * * true - payload_in passes check for @opcode. |
| @@ -325,12 +326,15 @@ static bool cxl_mem_raw_command_allowed(u16 opcode) · 8 unmodified lines | ||
| 325 | 326 | * |
| 326 | 327 | * The specific checks are determined by the opcode. |
| 327 | 328 | */ |
| 328 | −static bool cxl_payload_from_user_allowed(u16 opcode, void *payload_in) | |
| 329 | +static bool cxl_payload_from_user_allowed(u16 opcode, void *payload_in, | |
| 330 | + size_t in_size) | |
| 329 | 331 | { |
| 330 | 332 | switch (opcode) { |
| 331 | 333 | case CXL_MBOX_OP_SET_PARTITION_INFO: { |
| 332 | 334 | struct cxl_mbox_set_partition_info *pi = payload_in; |
| 333 | 335 | |
| 336 | + if (in_size < sizeof(*pi)) | |
| 337 | + return false; | |
| 334 | 338 | if (pi->flags & CXL_SET_PARTITION_IMMEDIATE_FLAG) |
| 335 | 339 | return false; |
| 336 | 340 | break; |
| @@ -338,6 +342,8 @@ static bool cxl_payload_from_user_allowed(u16 opcode, void *payload_in) · 1 unmodified line | ||
| 338 | 342 | case CXL_MBOX_OP_CLEAR_LOG: { |
| 339 | 343 | const uuid_t *uuid = (uuid_t *)payload_in; |
| 340 | 344 | |
| 345 | + if (in_size < sizeof(uuid_t)) | |
| 346 | + return false; | |
| 341 | 347 | /* |
| 342 | 348 | * Restrict the ‘Clear log’ action to only apply to |
| 343 | 349 | * Vendor debug logs. |
| @@ -365,7 +371,8 @@ static int cxl_mbox_cmd_ctor(struct cxl_mbox_cmd *mbox_cmd, · 21 unmodified lines | ||
| 365 | 371 | if (IS_ERR(mbox_cmd->payload_in)) |
| 366 | 372 | return PTR_ERR(mbox_cmd->payload_in); |
| 367 | 373 | |
| 368 | − if (!cxl_payload_from_user_allowed(opcode, mbox_cmd->payload_in)) { | |
| 374 | + if (!cxl_payload_from_user_allowed(opcode, mbox_cmd->payload_in, | |
| 375 | + in_size)) { | |
| 369 | 376 | dev_dbg(cxl_mbox->host, "%s: input payload not allowed\n", |
| 370 | 377 | cxl_mem_opcode_to_name(opcode)); |
| 371 | 378 | kvfree(mbox_cmd->payload_in); |
| @@ -1089,10 +1089,8 @@ static int cxlmd_add(struct cxl_memdev *cxlmd, struct cxl_dev_state *cxlds) · 1088 unmodified lines | ||
| 1089 | 1089 | DEFINE_FREE(put_cxlmd, struct cxl_memdev *, |
| 1090 | 1090 | if (!IS_ERR_OR_NULL(_T)) put_device(&_T->dev)) |
| 1091 | 1091 | |
| 1092 | −static struct cxl_memdev *cxl_memdev_autoremove(struct cxl_memdev *cxlmd) | |
| 1092 | +static bool cxl_memdev_attach_failed(struct cxl_memdev *cxlmd) | |
| 1093 | 1093 | { |
| 1094 | − int rc; | |
| 1095 | − | |
| 1096 | 1094 | /* |
| 1097 | 1095 | * If @attach is provided fail if the driver is not attached upon |
| 1098 | 1096 | * return. Note that failure here could be the result of a race to |
| @@ -1100,7 +1098,14 @@ static struct cxl_memdev *cxl_memdev_autoremove(struct cxl_memdev *cxlmd) · 1 unmodified line | ||
| 1100 | 1098 | * succeeded and then cxl_mem unbound before the lock is acquired. |
| 1101 | 1099 | */ |
| 1102 | 1100 | guard(device)(&cxlmd->dev); |
| 1103 | − if (cxlmd->attach && !cxlmd->dev.driver) { | |
| 1101 | + return (cxlmd->attach && !cxlmd->dev.driver); | |
| 1102 | +} | |
| 1103 | + | |
| 1104 | +static struct cxl_memdev *cxl_memdev_autoremove(struct cxl_memdev *cxlmd) | |
| 1105 | +{ | |
| 1106 | + int rc; | |
| 1107 | + | |
| 1108 | + if (cxl_memdev_attach_failed(cxlmd)) { | |
| 1104 | 1109 | cxl_memdev_unregister(cxlmd); |
| 1105 | 1110 | return ERR_PTR(-ENXIO); |
| 1106 | 1111 | } |
| @@ -115,15 +115,17 @@ static void unregister_nvb(void *_cxl_nvb) · 114 unmodified lines | ||
| 115 | 115 | device_unregister(&cxl_nvb->dev); |
| 116 | 116 | } |
| 117 | 117 | |
| 118 | −/** | |
| 119 | − * devm_cxl_add_nvdimm_bridge() - add the root of a LIBNVDIMM topology | |
| 120 | − * @host: platform firmware root device | |
| 121 | − * @port: CXL port at the root of a CXL topology | |
| 122 | − * | |
| 123 | − * Return: bridge device that can host cxl_nvdimm objects | |
| 124 | − */ | |
| 125 | −struct cxl_nvdimm_bridge *devm_cxl_add_nvdimm_bridge(struct device *host, | |
| 126 | − struct cxl_port *port) | |
| 118 | +static bool cxl_nvdimm_bridge_failed_attach(struct cxl_nvdimm_bridge *cxl_nvb) | |
| 119 | +{ | |
| 120 | + struct device *dev = &cxl_nvb->dev; | |
| 121 | + | |
| 122 | + guard(device)(dev); | |
| 123 | + /* If the device has no driver, then it failed to attach. */ | |
| 124 | + return dev->driver == NULL; | |
| 125 | +} | |
| 126 | + | |
| 127 | +struct cxl_nvdimm_bridge *__devm_cxl_add_nvdimm_bridge(struct device *host, | |
| 128 | + struct cxl_port *port) | |
| 127 | 129 | { |
| 128 | 130 | struct cxl_nvdimm_bridge *cxl_nvb; |
| 129 | 131 | struct device *dev; |
| @@ -145,6 +147,11 @@ struct cxl_nvdimm_bridge *devm_cxl_add_nvdimm_bridge(struct device *host, · 15 unmodified lines | ||
| 145 | 147 | if (rc) |
| 146 | 148 | goto err; |
| 147 | 149 | |
| 150 | + if (cxl_nvdimm_bridge_failed_attach(cxl_nvb)) { | |
| 151 | + unregister_nvb(cxl_nvb); | |
| 152 | + return ERR_PTR(-ENODEV); | |
| 153 | + } | |
| 154 | + | |
| 148 | 155 | rc = devm_add_action_or_reset(host, unregister_nvb, cxl_nvb); |
| 149 | 156 | if (rc) |
| 150 | 157 | return ERR_PTR(rc); |
| @@ -155,7 +162,7 @@ err: · 4 unmodified lines | ||
| 155 | 162 | put_device(dev); |
| 156 | 163 | return ERR_PTR(rc); |
| 157 | 164 | } |
| 158 | −EXPORT_SYMBOL_NS_GPL(devm_cxl_add_nvdimm_bridge, "CXL"); | |
| 165 | +EXPORT_SYMBOL_FOR_MODULES(__devm_cxl_add_nvdimm_bridge, "cxl_pmem"); | |
| 159 | 166 | |
| 160 | 167 | static void cxl_nvdimm_release(struct device *dev) |
| 161 | 168 | { |
| @@ -255,6 +262,21 @@ int devm_cxl_add_nvdimm(struct device *host, struct cxl_port *port, · 93 unmodified lines | ||
| 255 | 262 | if (!cxl_nvb) |
| 256 | 263 | return -ENODEV; |
| 257 | 264 | |
| 265 | + /* | |
| 266 | + * Take the uport_dev lock to guard against race of nvdimm_bus object. | |
| 267 | + * cxl_acpi_probe() registers the nvdimm_bus and is done under the | |
| 268 | + * root port uport_dev lock. | |
| 269 | + * | |
| 270 | + * Take the cxl_nvb device lock to ensure that cxl_nvb driver is in a | |
| 271 | + * consistent state. And the driver registers nvdimm_bus. | |
| 272 | + */ | |
| 273 | + guard(device)(cxl_nvb->port->uport_dev); | |
| 274 | + guard(device)(&cxl_nvb->dev); | |
| 275 | + if (!cxl_nvb->nvdimm_bus) { | |
| 276 | + rc = -ENODEV; | |
| 277 | + goto err_alloc; | |
| 278 | + } | |
| 279 | + | |
| 258 | 280 | cxl_nvd = cxl_nvdimm_alloc(cxl_nvb, cxlmd); |
| 259 | 281 | if (IS_ERR(cxl_nvd)) { |
| 260 | 282 | rc = PTR_ERR(cxl_nvd); |
| @@ -615,22 +615,8 @@ struct cxl_port *parent_port_of(struct cxl_port *port) · 614 unmodified lines | ||
| 615 | 615 | static void unregister_port(void *_port) |
| 616 | 616 | { |
| 617 | 617 | struct cxl_port *port = _port; |
| 618 | − struct cxl_port *parent = parent_port_of(port); | |
| 619 | − struct device *lock_dev; | |
| 620 | 618 | |
| 621 | − /* | |
| 622 | − * CXL root port's and the first level of ports are unregistered | |
| 623 | − * under the platform firmware device lock, all other ports are | |
| 624 | − * unregistered while holding their parent port lock. | |
| 625 | − */ | |
| 626 | − if (!parent) | |
| 627 | − lock_dev = port->uport_dev; | |
| 628 | − else if (is_cxl_root(parent)) | |
| 629 | − lock_dev = parent->uport_dev; | |
| 630 | − else | |
| 631 | − lock_dev = &parent->dev; | |
| 632 | − | |
| 633 | − device_lock_assert(lock_dev); | |
| 619 | + device_lock_assert(port_to_host(port)); | |
| 634 | 620 | port->dead = true; |
| 635 | 621 | device_unregister(&port->dev); |
| 636 | 622 | } |
| @@ -1427,20 +1413,11 @@ static struct device *grandparent(struct device *dev) · 790 unmodified lines | ||
| 1427 | 1413 | return NULL; |
| 1428 | 1414 | } |
| 1429 | 1415 | |
| 1430 | −static struct device *endpoint_host(struct cxl_port *endpoint) | |
| 1431 | −{ | |
| 1432 | − struct cxl_port *port = to_cxl_port(endpoint->dev.parent); | |
| 1433 | − | |
| 1434 | − if (is_cxl_root(port)) | |
| 1435 | − return port->uport_dev; | |
| 1436 | − return &port->dev; | |
| 1437 | −} | |
| 1438 | − | |
| 1439 | 1416 | static void delete_endpoint(void *data) |
| 1440 | 1417 | { |
| 1441 | 1418 | struct cxl_memdev *cxlmd = data; |
| 1442 | 1419 | struct cxl_port *endpoint = cxlmd->endpoint; |
| 1443 | − struct device *host = endpoint_host(endpoint); | |
| 1420 | + struct device *host = port_to_host(endpoint); | |
| 1444 | 1421 | |
| 1445 | 1422 | scoped_guard(device, host) { |
| 1446 | 1423 | if (host->driver && !endpoint->dead) { |
| @@ -1456,7 +1433,7 @@ static void delete_endpoint(void *data) · 9 unmodified lines | ||
| 1456 | 1433 | |
| 1457 | 1434 | int cxl_endpoint_autoremove(struct cxl_memdev *cxlmd, struct cxl_port *endpoint) |
| 1458 | 1435 | { |
| 1459 | − struct device *host = endpoint_host(endpoint); | |
| 1436 | + struct device *host = port_to_host(endpoint); | |
| 1460 | 1437 | struct device *dev = &cxlmd->dev; |
| 1461 | 1438 | |
| 1462 | 1439 | get_device(host); |
| @@ -1790,7 +1767,16 @@ static struct cxl_dport *find_or_add_dport(struct cxl_port *port, · 327 unmodified lines | ||
| 1790 | 1767 | { |
| 1791 | 1768 | struct cxl_dport *dport; |
| 1792 | 1769 | |
| 1793 | − device_lock_assert(&port->dev); | |
| 1770 | + /* | |
| 1771 | + * The port is already visible in CXL hierarchy, but it may still | |
| 1772 | + * be in the process of binding to the CXL port driver at this point. | |
| 1773 | + * | |
| 1774 | + * port creation and driver binding are protected by the port's host | |
| 1775 | + * lock, so acquire the host lock here to ensure the port has completed | |
| 1776 | + * driver binding before proceeding with dport addition. | |
| 1777 | + */ | |
| 1778 | + guard(device)(port_to_host(port)); | |
| 1779 | + guard(device)(&port->dev); | |
| 1794 | 1780 | dport = cxl_find_dport_by_dev(port, dport_dev); |
| 1795 | 1781 | if (!dport) { |
| 1796 | 1782 | dport = probe_dport(port, dport_dev); |
| @@ -1857,13 +1843,11 @@ retry: · 60 unmodified lines | ||
| 1857 | 1843 | * RP port enumerated by cxl_acpi without dport will |
| 1858 | 1844 | * have the dport added here. |
| 1859 | 1845 | */ |
| 1860 | − scoped_guard(device, &port->dev) { | |
| 1861 | − dport = find_or_add_dport(port, dport_dev); | |
| 1862 | − if (IS_ERR(dport)) { | |
| 1863 | − if (PTR_ERR(dport) == -EAGAIN) | |
| 1864 | − goto retry; | |
| 1865 | − return PTR_ERR(dport); | |
| 1866 | − } | |
| 1846 | + dport = find_or_add_dport(port, dport_dev); | |
| 1847 | + if (IS_ERR(dport)) { | |
| 1848 | + if (PTR_ERR(dport) == -EAGAIN) | |
| 1849 | + goto retry; | |
| 1850 | + return PTR_ERR(dport); | |
| 1867 | 1851 | } |
| 1868 | 1852 | |
| 1869 | 1853 | rc = cxl_add_ep(dport, &cxlmd->dev); |
| @@ -1100,12 +1100,12 @@ static int cxl_rr_assign_decoder(struct cxl_port *port, struct cxl_region *cxlr, · 1099 unmodified lines | ||
| 1100 | 1100 | static void cxl_region_setup_flags(struct cxl_region *cxlr, |
| 1101 | 1101 | struct cxl_decoder *cxld) |
| 1102 | 1102 | { |
| 1103 | − if (test_bit(CXL_DECODER_F_LOCK, &cxld->flags)) { | |
| 1103 | + if (cxld->flags & CXL_DECODER_F_LOCK) { | |
| 1104 | 1104 | set_bit(CXL_REGION_F_LOCK, &cxlr->flags); |
| 1105 | 1105 | clear_bit(CXL_REGION_F_NEEDS_RESET, &cxlr->flags); |
| 1106 | 1106 | } |
| 1107 | 1107 | |
| 1108 | − if (test_bit(CXL_DECODER_F_NORMALIZED_ADDRESSING, &cxld->flags)) | |
| 1108 | + if (cxld->flags & CXL_DECODER_F_NORMALIZED_ADDRESSING) | |
| 1109 | 1109 | set_bit(CXL_REGION_F_NORMALIZED_ADDRESSING, &cxlr->flags); |
| 1110 | 1110 | } |
| 1111 | 1111 | |
| @@ -574,11 +574,16 @@ struct cxl_nvdimm_bridge { · 573 unmodified lines | ||
| 574 | 574 | |
| 575 | 575 | #define CXL_DEV_ID_LEN 19 |
| 576 | 576 | |
| 577 | +enum { | |
| 578 | + CXL_NVD_F_INVALIDATED = 0, | |
| 579 | +}; | |
| 580 | + | |
| 577 | 581 | struct cxl_nvdimm { |
| 578 | 582 | struct device dev; |
| 579 | 583 | struct cxl_memdev *cxlmd; |
| 580 | 584 | u8 dev_id[CXL_DEV_ID_LEN]; /* for nvdimm, string of 'serial' */ |
| 581 | 585 | u64 dirty_shutdowns; |
| 586 | + unsigned long flags; | |
| 582 | 587 | }; |
| 583 | 588 | |
| 584 | 589 | struct cxl_pmem_region_mapping { |
| @@ -920,6 +925,8 @@ void cxl_driver_unregister(struct cxl_driver *cxl_drv); · 335 unmodified lines | ||
| 920 | 925 | struct cxl_nvdimm_bridge *to_cxl_nvdimm_bridge(struct device *dev); |
| 921 | 926 | struct cxl_nvdimm_bridge *devm_cxl_add_nvdimm_bridge(struct device *host, |
| 922 | 927 | struct cxl_port *port); |
| 928 | +struct cxl_nvdimm_bridge *__devm_cxl_add_nvdimm_bridge(struct device *host, | |
| 929 | + struct cxl_port *port); | |
| 923 | 930 | struct cxl_nvdimm *to_cxl_nvdimm(struct device *dev); |
| 924 | 931 | bool is_cxl_nvdimm(struct device *dev); |
| 925 | 932 | int devm_cxl_add_nvdimm(struct device *host, struct cxl_port *port, |
| @@ -13,6 +13,20 @@ · 12 unmodified lines | ||
| 13 | 13 | |
| 14 | 14 | static __read_mostly DECLARE_BITMAP(exclusive_cmds, CXL_MEM_COMMAND_ID_MAX); |
| 15 | 15 | |
| 16 | +/** | |
| 17 | + * devm_cxl_add_nvdimm_bridge() - add the root of a LIBNVDIMM topology | |
| 18 | + * @host: platform firmware root device | |
| 19 | + * @port: CXL port at the root of a CXL topology | |
| 20 | + * | |
| 21 | + * Return: bridge device that can host cxl_nvdimm objects | |
| 22 | + */ | |
| 23 | +struct cxl_nvdimm_bridge *devm_cxl_add_nvdimm_bridge(struct device *host, | |
| 24 | + struct cxl_port *port) | |
| 25 | +{ | |
| 26 | + return __devm_cxl_add_nvdimm_bridge(host, port); | |
| 27 | +} | |
| 28 | +EXPORT_SYMBOL_NS_GPL(devm_cxl_add_nvdimm_bridge, "CXL"); | |
| 29 | + | |
| 16 | 30 | static void clear_exclusive(void *mds) |
| 17 | 31 | { |
| 18 | 32 | clear_exclusive_cxl_commands(mds, exclusive_cmds); |
| @@ -129,6 +143,9 @@ static int cxl_nvdimm_probe(struct device *dev) · 110 unmodified lines | ||
| 129 | 143 | struct nvdimm *nvdimm; |
| 130 | 144 | int rc; |
| 131 | 145 | |
| 146 | + if (test_bit(CXL_NVD_F_INVALIDATED, &cxl_nvd->flags)) | |
| 147 | + return -EBUSY; | |
| 148 | + | |
| 132 | 149 | set_exclusive_cxl_commands(mds, exclusive_cmds); |
| 133 | 150 | rc = devm_add_action_or_reset(dev, clear_exclusive, mds); |
| 134 | 151 | if (rc) |
| @@ -309,8 +326,10 @@ static int detach_nvdimm(struct device *dev, void *data) · 174 unmodified lines | ||
| 309 | 326 | scoped_guard(device, dev) { |
| 310 | 327 | if (dev->driver) { |
| 311 | 328 | cxl_nvd = to_cxl_nvdimm(dev); |
| 312 | − if (cxl_nvd->cxlmd && cxl_nvd->cxlmd->cxl_nvb == data) | |
| 329 | + if (cxl_nvd->cxlmd && cxl_nvd->cxlmd->cxl_nvb == data) { | |
| 313 | 330 | release = true; |
| 331 | + set_bit(CXL_NVD_F_INVALIDATED, &cxl_nvd->flags); | |
| 332 | + } | |
| 314 | 333 | } |
| 315 | 334 | } |
| 316 | 335 | if (release) |
| @@ -353,6 +372,7 @@ static struct cxl_driver cxl_nvdimm_bridge_driver = { · 36 unmodified lines | ||
| 353 | 372 | .probe = cxl_nvdimm_bridge_probe, |
| 354 | 373 | .id = CXL_DEVICE_NVDIMM_BRIDGE, |
| 355 | 374 | .drv = { |
| 375 | + .probe_type = PROBE_FORCE_SYNCHRONOUS, | |
| 356 | 376 | .suppress_bind_attrs = true, |
| 357 | 377 | }, |
| 358 | 378 | }; |
| @@ -981,11 +981,7 @@ zl3073x_devm_dpll_init(struct zl3073x_dev *zldev, u8 num_dplls) · 980 unmodified lines | ||
| 981 | 981 | } |
| 982 | 982 | |
| 983 | 983 | /* Add devres action to release DPLL related resources */ |
| 984 | − rc = devm_add_action_or_reset(zldev->dev, zl3073x_dev_dpll_fini, zldev); | |
| 985 | − if (rc) | |
| 986 | − goto error; | |
| 987 | − | |
| 988 | − return 0; | |
| 984 | + return devm_add_action_or_reset(zldev->dev, zl3073x_dev_dpll_fini, zldev); | |
| 989 | 985 | |
| 990 | 986 | error: |
| 991 | 987 | zl3073x_dev_dpll_fini(zldev); |
| @@ -1026,6 +1022,7 @@ int zl3073x_dev_probe(struct zl3073x_dev *zldev, · 34 unmodified lines | ||
| 1026 | 1022 | "Unknown or non-match chip ID: 0x%0x\n", |
| 1027 | 1023 | id); |
| 1028 | 1024 | } |
| 1025 | + zldev->chip_id = id; | |
| 1029 | 1026 | |
| 1030 | 1027 | /* Read revision, firmware version and custom config version */ |
| 1031 | 1028 | rc = zl3073x_read_u16(zldev, ZL_REG_REVISION, &revision); |
| @@ -35,6 +35,7 @@ struct zl3073x_dpll; · 34 unmodified lines | ||
| 35 | 35 | * @dev: pointer to device |
| 36 | 36 | * @regmap: regmap to access device registers |
| 37 | 37 | * @multiop_lock: to serialize multiple register operations |
| 38 | + * @chip_id: chip ID read from hardware | |
| 38 | 39 | * @ref: array of input references' invariants |
| 39 | 40 | * @out: array of outs' invariants |
| 40 | 41 | * @synth: array of synths' invariants |
| @@ -48,6 +49,7 @@ struct zl3073x_dev { · 7 unmodified lines | ||
| 48 | 49 | struct device *dev; |
| 49 | 50 | struct regmap *regmap; |
| 50 | 51 | struct mutex multiop_lock; |
| 52 | + u16 chip_id; | |
| 51 | 53 | |
| 52 | 54 | /* Invariants */ |
| 53 | 55 | struct zl3073x_ref ref[ZL3073X_NUM_REFS]; |
| @@ -144,6 +146,32 @@ int zl3073x_write_hwreg_seq(struct zl3073x_dev *zldev, · 90 unmodified lines | ||
| 144 | 146 | |
| 145 | 147 | int zl3073x_ref_phase_offsets_update(struct zl3073x_dev *zldev, int channel); |
| 146 | 148 | |
| 149 | +/** | |
| 150 | + * zl3073x_dev_is_ref_phase_comp_32bit - check ref phase comp register size | |
| 151 | + * @zldev: pointer to zl3073x device | |
| 152 | + * | |
| 153 | + * Some chip IDs have a 32-bit wide ref_phase_offset_comp register instead | |
| 154 | + * of the default 48-bit. | |
| 155 | + * | |
| 156 | + * Return: true if the register is 32-bit, false if 48-bit | |
| 157 | + */ | |
| 158 | +static inline bool | |
| 159 | +zl3073x_dev_is_ref_phase_comp_32bit(struct zl3073x_dev *zldev) | |
| 160 | +{ | |
| 161 | + switch (zldev->chip_id) { | |
| 162 | + case 0x0E30: | |
| 163 | + case 0x0E93: | |
| 164 | + case 0x0E94: | |
| 165 | + case 0x0E95: | |
| 166 | + case 0x0E96: | |
| 167 | + case 0x0E97: | |
| 168 | + case 0x1F60: | |
| 169 | + return true; | |
| 170 | + default: | |
| 171 | + return false; | |
| 172 | + } | |
| 173 | +} | |
| 174 | + | |
| 147 | 175 | static inline bool |
| 148 | 176 | zl3073x_is_n_pin(u8 id) |
| 149 | 177 | { |
| @@ -475,8 +475,11 @@ zl3073x_dpll_input_pin_phase_adjust_get(const struct dpll_pin *dpll_pin, · 474 unmodified lines | ||
| 475 | 475 | ref_id = zl3073x_input_pin_ref_get(pin->id); |
| 476 | 476 | ref = zl3073x_ref_state_get(zldev, ref_id); |
| 477 | 477 | |
| 478 | − /* Perform sign extension for 48bit signed value */ | |
| 479 | − phase_comp = sign_extend64(ref->phase_comp, 47); | |
| 478 | + /* Perform sign extension based on register width */ | |
| 479 | + if (zl3073x_dev_is_ref_phase_comp_32bit(zldev)) | |
| 480 | + phase_comp = sign_extend64(ref->phase_comp, 31); | |
| 481 | + else | |
| 482 | + phase_comp = sign_extend64(ref->phase_comp, 47); | |
| 480 | 483 | |
| 481 | 484 | /* Reverse two's complement negation applied during set and convert |
| 482 | 485 | * to 32bit signed int |
| @@ -121,8 +121,16 @@ int zl3073x_ref_state_fetch(struct zl3073x_dev *zldev, u8 index) · 120 unmodified lines | ||
| 121 | 121 | return rc; |
| 122 | 122 | |
| 123 | 123 | /* Read phase compensation register */ |
| 124 | − rc = zl3073x_read_u48(zldev, ZL_REG_REF_PHASE_OFFSET_COMP, | |
| 125 | − &ref->phase_comp); | |
| 124 | + if (zl3073x_dev_is_ref_phase_comp_32bit(zldev)) { | |
| 125 | + u32 val; | |
| 126 | + | |
| 127 | + rc = zl3073x_read_u32(zldev, ZL_REG_REF_PHASE_OFFSET_COMP_32, | |
| 128 | + &val); | |
| 129 | + ref->phase_comp = val; | |
| 130 | + } else { | |
| 131 | + rc = zl3073x_read_u48(zldev, ZL_REG_REF_PHASE_OFFSET_COMP, | |
| 132 | + &ref->phase_comp); | |
| 133 | + } | |
| 126 | 134 | if (rc) |
| 127 | 135 | return rc; |
| 128 | 136 | |
| @@ -179,9 +187,16 @@ int zl3073x_ref_state_set(struct zl3073x_dev *zldev, u8 index, · 50 unmodified lines | ||
| 179 | 187 | if (!rc && dref->sync_ctrl != ref->sync_ctrl) |
| 180 | 188 | rc = zl3073x_write_u8(zldev, ZL_REG_REF_SYNC_CTRL, |
| 181 | 189 | ref->sync_ctrl); |
| 182 | − if (!rc && dref->phase_comp != ref->phase_comp) | |
| 183 | − rc = zl3073x_write_u48(zldev, ZL_REG_REF_PHASE_OFFSET_COMP, | |
| 184 | − ref->phase_comp); | |
| 190 | + if (!rc && dref->phase_comp != ref->phase_comp) { | |
| 191 | + if (zl3073x_dev_is_ref_phase_comp_32bit(zldev)) | |
| 192 | + rc = zl3073x_write_u32(zldev, | |
| 193 | + ZL_REG_REF_PHASE_OFFSET_COMP_32, | |
| 194 | + ref->phase_comp); | |
| 195 | + else | |
| 196 | + rc = zl3073x_write_u48(zldev, | |
| 197 | + ZL_REG_REF_PHASE_OFFSET_COMP, | |
| 198 | + ref->phase_comp); | |
| 199 | + } | |
| 185 | 200 | if (rc) |
| 186 | 201 | return rc; |
| 187 | 202 | |
| @@ -194,6 +194,7 @@ · 193 unmodified lines | ||
| 194 | 194 | #define ZL_REF_CONFIG_DIFF_EN BIT(2) |
| 195 | 195 | |
| 196 | 196 | #define ZL_REG_REF_PHASE_OFFSET_COMP ZL_REG(10, 0x28, 6) |
| 197 | +#define ZL_REG_REF_PHASE_OFFSET_COMP_32 ZL_REG(10, 0x28, 4) | |
| 197 | 198 | |
| 198 | 199 | #define ZL_REG_REF_SYNC_CTRL ZL_REG(10, 0x2e, 1) |
| 199 | 200 | #define ZL_REF_SYNC_CTRL_MODE GENMASK(2, 0) |
| @@ -848,7 +848,7 @@ static int ar_context_init(struct ar_context *ctx, struct fw_ohci *ohci, · 847 unmodified lines | ||
| 848 | 848 | { |
| 849 | 849 | struct device *dev = ohci->card.device; |
| 850 | 850 | unsigned int i; |
| 851 | − struct page *pages[AR_BUFFERS + AR_WRAPAROUND_PAGES]; | |
| 851 | + struct page *pages[AR_BUFFERS + AR_WRAPAROUND_PAGES] = { NULL }; | |
| 852 | 852 | dma_addr_t dma_addrs[AR_BUFFERS]; |
| 853 | 853 | void *vaddr; |
| 854 | 854 | struct descriptor *d; |
| @@ -1610,11 +1610,17 @@ static int cs_dsp_load(struct cs_dsp *dsp, const struct firmware *firmware, · 1609 unmodified lines | ||
| 1610 | 1610 | region_name); |
| 1611 | 1611 | |
| 1612 | 1612 | if (reg) { |
| 1613 | + /* | |
| 1614 | + * Although we expect the underlying bus does not require | |
| 1615 | + * physically-contiguous buffers, we pessimistically use | |
| 1616 | + * a temporary buffer instead of trusting that the | |
| 1617 | + * alignment of region->data is ok. | |
| 1618 | + */ | |
| 1613 | 1619 | region_len = le32_to_cpu(region->len); |
| 1614 | 1620 | if (region_len > buf_len) { |
| 1615 | 1621 | buf_len = round_up(region_len, PAGE_SIZE); |
| 1616 | − kfree(buf); | |
| 1617 | − buf = kmalloc(buf_len, GFP_KERNEL | GFP_DMA); | |
| 1622 | + vfree(buf); | |
| 1623 | + buf = vmalloc(buf_len); | |
| 1618 | 1624 | if (!buf) { |
| 1619 | 1625 | ret = -ENOMEM; |
| 1620 | 1626 | goto out_fw; |
| @@ -1643,7 +1649,7 @@ static int cs_dsp_load(struct cs_dsp *dsp, const struct firmware *firmware, · 22 unmodified lines | ||
| 1643 | 1649 | |
| 1644 | 1650 | ret = 0; |
| 1645 | 1651 | out_fw: |
| 1646 | − kfree(buf); | |
| 1652 | + vfree(buf); | |
| 1647 | 1653 | |
| 1648 | 1654 | if (ret == -EOVERFLOW) |
| 1649 | 1655 | cs_dsp_err(dsp, "%s: file content overflows file data\n", file); |
| @@ -2331,11 +2337,17 @@ static int cs_dsp_load_coeff(struct cs_dsp *dsp, const struct firmware *firmware · 681 unmodified lines | ||
| 2331 | 2337 | } |
| 2332 | 2338 | |
| 2333 | 2339 | if (reg) { |
| 2340 | + /* | |
| 2341 | + * Although we expect the underlying bus does not require | |
| 2342 | + * physically-contiguous buffers, we pessimistically use | |
| 2343 | + * a temporary buffer instead of trusting that the | |
| 2344 | + * alignment of blk->data is ok. | |
| 2345 | + */ | |
| 2334 | 2346 | region_len = le32_to_cpu(blk->len); |
| 2335 | 2347 | if (region_len > buf_len) { |
| 2336 | 2348 | buf_len = round_up(region_len, PAGE_SIZE); |
| 2337 | − kfree(buf); | |
| 2338 | − buf = kmalloc(buf_len, GFP_KERNEL | GFP_DMA); | |
| 2349 | + vfree(buf); | |
| 2350 | + buf = vmalloc(buf_len); | |
| 2339 | 2351 | if (!buf) { |
| 2340 | 2352 | ret = -ENOMEM; |
| 2341 | 2353 | goto out_fw; |
| @@ -2366,7 +2378,7 @@ static int cs_dsp_load_coeff(struct cs_dsp *dsp, const struct firmware *firmware · 24 unmodified lines | ||
| 2366 | 2378 | |
| 2367 | 2379 | ret = 0; |
| 2368 | 2380 | out_fw: |
| 2369 | − kfree(buf); | |
| 2381 | + vfree(buf); | |
| 2370 | 2382 | |
| 2371 | 2383 | if (ret == -EOVERFLOW) |
| 2372 | 2384 | cs_dsp_err(dsp, "%s: file content overflows file data\n", file); |
| @@ -85,7 +85,7 @@ static struct kobject *mokvar_kobj; · 84 unmodified lines | ||
| 85 | 85 | * as an alternative to ordinary EFI variables, due to platform-dependent |
| 86 | 86 | * limitations. The memory occupied by this table is marked as reserved. |
| 87 | 87 | * |
| 88 | − * This routine must be called before efi_free_boot_services() in order | |
| 88 | + * This routine must be called before efi_unmap_boot_services() in order | |
| 89 | 89 | * to guarantee that it can mark the table as reserved. |
| 90 | 90 | * |
| 91 | 91 | * Implicit inputs: |
| @@ -768,7 +768,9 @@ static int stratix10_rsu_probe(struct platform_device *pdev) · 767 unmodified lines | ||
| 768 | 768 | rsu_async_status_callback); |
| 769 | 769 | if (ret) { |
| 770 | 770 | dev_err(dev, "Error, getting RSU status %i\n", ret); |
| 771 | + stratix10_svc_remove_async_client(priv->chan); | |
| 771 | 772 | stratix10_svc_free_channel(priv->chan); |
| 773 | + return ret; | |
| 772 | 774 | } |
| 773 | 775 | |
| 774 | 776 | /* get DCMF version from firmware */ |
| @@ -37,15 +37,14 @@ · 36 unmodified lines | ||
| 37 | 37 | * service layer will return error to FPGA manager when timeout occurs, |
| 38 | 38 | * timeout is set to 30 seconds (30 * 1000) at Intel Stratix10 SoC. |
| 39 | 39 | */ |
| 40 | −#define SVC_NUM_DATA_IN_FIFO 32 | |
| 40 | +#define SVC_NUM_DATA_IN_FIFO 8 | |
| 41 | 41 | #define SVC_NUM_CHANNEL 4 |
| 42 | −#define FPGA_CONFIG_DATA_CLAIM_TIMEOUT_MS 200 | |
| 42 | +#define FPGA_CONFIG_DATA_CLAIM_TIMEOUT_MS 2000 | |
| 43 | 43 | #define FPGA_CONFIG_STATUS_TIMEOUT_SEC 30 |
| 44 | 44 | #define BYTE_TO_WORD_SIZE 4 |
| 45 | 45 | |
| 46 | 46 | /* stratix10 service layer clients */ |
| 47 | 47 | #define STRATIX10_RSU "stratix10-rsu" |
| 48 | −#define INTEL_FCS "intel-fcs" | |
| 49 | 48 | |
| 50 | 49 | /* Maximum number of SDM client IDs. */ |
| 51 | 50 | #define MAX_SDM_CLIENT_IDS 16 |
| @@ -105,11 +104,9 @@ struct stratix10_svc_chan; · 53 unmodified lines | ||
| 105 | 104 | /** |
| 106 | 105 | * struct stratix10_svc - svc private data |
| 107 | 106 | * @stratix10_svc_rsu: pointer to stratix10 RSU device |
| 108 | − * @intel_svc_fcs: pointer to the FCS device | |
| 109 | 107 | */ |
| 110 | 108 | struct stratix10_svc { |
| 111 | 109 | struct platform_device *stratix10_svc_rsu; |
| 112 | − struct platform_device *intel_svc_fcs; | |
| 113 | 110 | }; |
| 114 | 111 | |
| 115 | 112 | /** |
| @@ -251,12 +248,10 @@ struct stratix10_async_ctrl { · 135 unmodified lines | ||
| 251 | 248 | * @num_active_client: number of active service client |
| 252 | 249 | * @node: list management |
| 253 | 250 | * @genpool: memory pool pointing to the memory region |
| 254 | − * @task: pointer to the thread task which handles SMC or HVC call | |
| 255 | − * @svc_fifo: a queue for storing service message data | |
| 256 | 251 | * @complete_status: state for completion |
| 257 | − * @svc_fifo_lock: protect access to service message data queue | |
| 258 | 252 | * @invoke_fn: function to issue secure monitor call or hypervisor call |
| 259 | 253 | * @svc: manages the list of client svc drivers |
| 254 | + * @sdm_lock: only allows a single command single response to SDM | |
| 260 | 255 | * @actrl: async control structure |
| 261 | 256 | * |
| 262 | 257 | * This struct is used to create communication channels for service clients, to |
| @@ -269,12 +264,10 @@ struct stratix10_svc_controller { · 6 unmodified lines | ||
| 269 | 264 | int num_active_client; |
| 270 | 265 | struct list_head node; |
| 271 | 266 | struct gen_pool *genpool; |
| 272 | − struct task_struct *task; | |
| 273 | − struct kfifo svc_fifo; | |
| 274 | 267 | struct completion complete_status; |
| 275 | − spinlock_t svc_fifo_lock; | |
| 276 | 268 | svc_invoke_fn *invoke_fn; |
| 277 | 269 | struct stratix10_svc *svc; |
| 270 | + struct mutex sdm_lock; | |
| 278 | 271 | struct stratix10_async_ctrl actrl; |
| 279 | 272 | }; |
| 280 | 273 | |
| @@ -283,6 +276,9 @@ struct stratix10_svc_controller { · 2 unmodified lines | ||
| 283 | 276 | * @ctrl: pointer to service controller which is the provider of this channel |
| 284 | 277 | * @scl: pointer to service client which owns the channel |
| 285 | 278 | * @name: service client name associated with the channel |
| 279 | + * @task: pointer to the thread task which handles SMC or HVC call | |
| 280 | + * @svc_fifo: a queue for storing service message data (separate fifo for every channel) | |
| 281 | + * @svc_fifo_lock: protect access to service message data queue (locking pending fifo) | |
| 286 | 282 | * @lock: protect access to the channel |
| 287 | 283 | * @async_chan: reference to asynchronous channel object for this channel |
| 288 | 284 | * |
| @@ -293,6 +289,9 @@ struct stratix10_svc_chan { · 4 unmodified lines | ||
| 293 | 289 | struct stratix10_svc_controller *ctrl; |
| 294 | 290 | struct stratix10_svc_client *scl; |
| 295 | 291 | char *name; |
| 292 | + struct task_struct *task; | |
| 293 | + struct kfifo svc_fifo; | |
| 294 | + spinlock_t svc_fifo_lock; | |
| 296 | 295 | spinlock_t lock; |
| 297 | 296 | struct stratix10_async_chan *async_chan; |
| 298 | 297 | }; |
| @@ -527,10 +526,10 @@ static void svc_thread_recv_status_ok(struct stratix10_svc_data *p_data, · 228 unmodified lines | ||
| 527 | 526 | */ |
| 528 | 527 | static int svc_normal_to_secure_thread(void *data) |
| 529 | 528 | { |
| 530 | − struct stratix10_svc_controller | |
| 531 | − *ctrl = (struct stratix10_svc_controller *)data; | |
| 532 | − struct stratix10_svc_data *pdata; | |
| 533 | − struct stratix10_svc_cb_data *cbdata; | |
| 529 | + struct stratix10_svc_chan *chan = (struct stratix10_svc_chan *)data; | |
| 530 | + struct stratix10_svc_controller *ctrl = chan->ctrl; | |
| 531 | + struct stratix10_svc_data *pdata = NULL; | |
| 532 | + struct stratix10_svc_cb_data *cbdata = NULL; | |
| 534 | 533 | struct arm_smccc_res res; |
| 535 | 534 | unsigned long a0, a1, a2, a3, a4, a5, a6, a7; |
| 536 | 535 | int ret_fifo = 0; |
| @@ -555,12 +554,12 @@ static int svc_normal_to_secure_thread(void *data) · 18 unmodified lines | ||
| 555 | 554 | a6 = 0; |
| 556 | 555 | a7 = 0; |
| 557 | 556 | |
| 558 | − pr_debug("smc_hvc_shm_thread is running\n"); | |
| 557 | + pr_debug("%s: %s: Thread is running!\n", __func__, chan->name); | |
| 559 | 558 | |
| 560 | 559 | while (!kthread_should_stop()) { |
| 561 | − ret_fifo = kfifo_out_spinlocked(&ctrl->svc_fifo, | |
| 560 | + ret_fifo = kfifo_out_spinlocked(&chan->svc_fifo, | |
| 562 | 561 | pdata, sizeof(*pdata), |
| 563 | − &ctrl->svc_fifo_lock); | |
| 562 | + &chan->svc_fifo_lock); | |
| 564 | 563 | |
| 565 | 564 | if (!ret_fifo) |
| 566 | 565 | continue; |
| @@ -569,9 +568,25 @@ static int svc_normal_to_secure_thread(void *data) · 2 unmodified lines | ||
| 569 | 568 | (unsigned int)pdata->paddr, pdata->command, |
| 570 | 569 | (unsigned int)pdata->size); |
| 571 | 570 | |
| 571 | + /* SDM can only process one command at a time */ | |
| 572 | + pr_debug("%s: %s: Thread is waiting for mutex!\n", | |
| 573 | + __func__, chan->name); | |
| 574 | + if (mutex_lock_interruptible(&ctrl->sdm_lock)) { | |
| 575 | + /* item already dequeued; notify client to unblock it */ | |
| 576 | + cbdata->status = BIT(SVC_STATUS_ERROR); | |
| 577 | + cbdata->kaddr1 = NULL; | |
| 578 | + cbdata->kaddr2 = NULL; | |
| 579 | + cbdata->kaddr3 = NULL; | |
| 580 | + if (pdata->chan->scl) | |
| 581 | + pdata->chan->scl->receive_cb(pdata->chan->scl, | |
| 582 | + cbdata); | |
| 583 | + break; | |
| 584 | + } | |
| 585 | + | |
| 572 | 586 | switch (pdata->command) { |
| 573 | 587 | case COMMAND_RECONFIG_DATA_CLAIM: |
| 574 | 588 | svc_thread_cmd_data_claim(ctrl, pdata, cbdata); |
| 589 | + mutex_unlock(&ctrl->sdm_lock); | |
| 575 | 590 | continue; |
| 576 | 591 | case COMMAND_RECONFIG: |
| 577 | 592 | a0 = INTEL_SIP_SMC_FPGA_CONFIG_START; |
| @@ -700,10 +715,11 @@ static int svc_normal_to_secure_thread(void *data) · 122 unmodified lines | ||
| 700 | 715 | break; |
| 701 | 716 | default: |
| 702 | 717 | pr_warn("it shouldn't happen\n"); |
| 703 | − break; | |
| 718 | + mutex_unlock(&ctrl->sdm_lock); | |
| 719 | + continue; | |
| 704 | 720 | } |
| 705 | − pr_debug("%s: before SMC call -- a0=0x%016x a1=0x%016x", | |
| 706 | − __func__, | |
| 721 | + pr_debug("%s: %s: before SMC call -- a0=0x%016x a1=0x%016x", | |
| 722 | + __func__, chan->name, | |
| 707 | 723 | (unsigned int)a0, |
| 708 | 724 | (unsigned int)a1); |
| 709 | 725 | pr_debug(" a2=0x%016x\n", (unsigned int)a2); |
| @@ -712,8 +728,8 @@ static int svc_normal_to_secure_thread(void *data) · 2 unmodified lines | ||
| 712 | 728 | pr_debug(" a5=0x%016x\n", (unsigned int)a5); |
| 713 | 729 | ctrl->invoke_fn(a0, a1, a2, a3, a4, a5, a6, a7, &res); |
| 714 | 730 | |
| 715 | − pr_debug("%s: after SMC call -- res.a0=0x%016x", | |
| 716 | − __func__, (unsigned int)res.a0); | |
| 731 | + pr_debug("%s: %s: after SMC call -- res.a0=0x%016x", | |
| 732 | + __func__, chan->name, (unsigned int)res.a0); | |
| 717 | 733 | pr_debug(" res.a1=0x%016x, res.a2=0x%016x", |
| 718 | 734 | (unsigned int)res.a1, (unsigned int)res.a2); |
| 719 | 735 | pr_debug(" res.a3=0x%016x\n", (unsigned int)res.a3); |
| @@ -728,6 +744,7 @@ static int svc_normal_to_secure_thread(void *data) · 8 unmodified lines | ||
| 728 | 744 | cbdata->kaddr2 = NULL; |
| 729 | 745 | cbdata->kaddr3 = NULL; |
| 730 | 746 | pdata->chan->scl->receive_cb(pdata->chan->scl, cbdata); |
| 747 | + mutex_unlock(&ctrl->sdm_lock); | |
| 731 | 748 | continue; |
| 732 | 749 | } |
| 733 | 750 | |
| @@ -801,6 +818,8 @@ static int svc_normal_to_secure_thread(void *data) · 67 unmodified lines | ||
| 801 | 818 | break; |
| 802 | 819 | |
| 803 | 820 | } |
| 821 | + | |
| 822 | + mutex_unlock(&ctrl->sdm_lock); | |
| 804 | 823 | } |
| 805 | 824 | |
| 806 | 825 | kfree(cbdata); |
| @@ -1696,22 +1715,33 @@ int stratix10_svc_send(struct stratix10_svc_chan *chan, void *msg) · 889 unmodified lines | ||
| 1696 | 1715 | if (!p_data) |
| 1697 | 1716 | return -ENOMEM; |
| 1698 | 1717 | |
| 1699 | − /* first client will create kernel thread */ | |
| 1700 | − if (!chan->ctrl->task) { | |
| 1701 | − chan->ctrl->task = | |
| 1702 | − kthread_run_on_cpu(svc_normal_to_secure_thread, | |
| 1703 | − (void *)chan->ctrl, | |
| 1704 | − cpu, "svc_smc_hvc_thread"); | |
| 1705 | − if (IS_ERR(chan->ctrl->task)) { | |
| 1718 | + /* first caller creates the per-channel kthread */ | |
| 1719 | + if (!chan->task) { | |
| 1720 | + struct task_struct *task; | |
| 1721 | + | |
| 1722 | + task = kthread_run_on_cpu(svc_normal_to_secure_thread, | |
| 1723 | + (void *)chan, | |
| 1724 | + cpu, "svc_smc_hvc_thread"); | |
| 1725 | + if (IS_ERR(task)) { | |
| 1706 | 1726 | dev_err(chan->ctrl->dev, |
| 1707 | 1727 | "failed to create svc_smc_hvc_thread\n"); |
| 1708 | 1728 | kfree(p_data); |
| 1709 | 1729 | return -EINVAL; |
| 1710 | 1730 | } |
| 1731 | + | |
| 1732 | + spin_lock(&chan->lock); | |
| 1733 | + if (chan->task) { | |
| 1734 | + /* another caller won the race; discard our thread */ | |
| 1735 | + spin_unlock(&chan->lock); | |
| 1736 | + kthread_stop(task); | |
| 1737 | + } else { | |
| 1738 | + chan->task = task; | |
| 1739 | + spin_unlock(&chan->lock); | |
| 1740 | + } | |
| 1711 | 1741 | } |
| 1712 | 1742 | |
| 1713 | − pr_debug("%s: sent P-va=%p, P-com=%x, P-size=%u\n", __func__, | |
| 1714 | − p_msg->payload, p_msg->command, | |
| 1743 | + pr_debug("%s: %s: sent P-va=%p, P-com=%x, P-size=%u\n", __func__, | |
| 1744 | + chan->name, p_msg->payload, p_msg->command, | |
| 1715 | 1745 | (unsigned int)p_msg->payload_length); |
| 1716 | 1746 | |
| 1717 | 1747 | if (list_empty(&svc_data_mem)) { |
| @@ -1747,12 +1777,16 @@ int stratix10_svc_send(struct stratix10_svc_chan *chan, void *msg) · 29 unmodified lines | ||
| 1747 | 1777 | p_data->arg[2] = p_msg->arg[2]; |
| 1748 | 1778 | p_data->size = p_msg->payload_length; |
| 1749 | 1779 | p_data->chan = chan; |
| 1750 | − pr_debug("%s: put to FIFO pa=0x%016x, cmd=%x, size=%u\n", __func__, | |
| 1751 | − (unsigned int)p_data->paddr, p_data->command, | |
| 1752 | − (unsigned int)p_data->size); | |
| 1753 | − ret = kfifo_in_spinlocked(&chan->ctrl->svc_fifo, p_data, | |
| 1780 | + pr_debug("%s: %s: put to FIFO pa=0x%016x, cmd=%x, size=%u\n", | |
| 1781 | + __func__, | |
| 1782 | + chan->name, | |
| 1783 | + (unsigned int)p_data->paddr, | |
| 1784 | + p_data->command, | |
| 1785 | + (unsigned int)p_data->size); | |
| 1786 | + | |
| 1787 | + ret = kfifo_in_spinlocked(&chan->svc_fifo, p_data, | |
| 1754 | 1788 | sizeof(*p_data), |
| 1755 | − &chan->ctrl->svc_fifo_lock); | |
| 1789 | + &chan->svc_fifo_lock); | |
| 1756 | 1790 | |
| 1757 | 1791 | kfree(p_data); |
| 1758 | 1792 | |
| @@ -1773,11 +1807,12 @@ EXPORT_SYMBOL_GPL(stratix10_svc_send); · 14 unmodified lines | ||
| 1773 | 1807 | */ |
| 1774 | 1808 | void stratix10_svc_done(struct stratix10_svc_chan *chan) |
| 1775 | 1809 | { |
| 1776 | − /* stop thread when thread is running AND only one active client */ | |
| 1777 | − if (chan->ctrl->task && chan->ctrl->num_active_client <= 1) { | |
| 1778 | − pr_debug("svc_smc_hvc_shm_thread is stopped\n"); | |
| 1779 | − kthread_stop(chan->ctrl->task); | |
| 1780 | − chan->ctrl->task = NULL; | |
| 1810 | + /* stop thread when thread is running */ | |
| 1811 | + if (chan->task) { | |
| 1812 | + pr_debug("%s: %s: svc_smc_hvc_shm_thread is stopping\n", | |
| 1813 | + __func__, chan->name); | |
| 1814 | + kthread_stop(chan->task); | |
| 1815 | + chan->task = NULL; | |
| 1781 | 1816 | } |
| 1782 | 1817 | } |
| 1783 | 1818 | EXPORT_SYMBOL_GPL(stratix10_svc_done); |
| @@ -1817,8 +1852,8 @@ void *stratix10_svc_allocate_memory(struct stratix10_svc_chan *chan, · 33 unmodified lines | ||
| 1817 | 1852 | pmem->paddr = pa; |
| 1818 | 1853 | pmem->size = s; |
| 1819 | 1854 | list_add_tail(&pmem->node, &svc_data_mem); |
| 1820 | − pr_debug("%s: va=%p, pa=0x%016x\n", __func__, | |
| 1821 | − pmem->vaddr, (unsigned int)pmem->paddr); | |
| 1855 | + pr_debug("%s: %s: va=%p, pa=0x%016x\n", __func__, | |
| 1856 | + chan->name, pmem->vaddr, (unsigned int)pmem->paddr); | |
| 1822 | 1857 | |
| 1823 | 1858 | return (void *)va; |
| 1824 | 1859 | } |
| @@ -1855,6 +1890,13 @@ static const struct of_device_id stratix10_svc_drv_match[] = { · 30 unmodified lines | ||
| 1855 | 1890 | {}, |
| 1856 | 1891 | }; |
| 1857 | 1892 | |
| 1893 | +static const char * const chan_names[SVC_NUM_CHANNEL] = { | |
| 1894 | + SVC_CLIENT_FPGA, | |
| 1895 | + SVC_CLIENT_RSU, | |
| 1896 | + SVC_CLIENT_FCS, | |
| 1897 | + SVC_CLIENT_HWMON | |
| 1898 | +}; | |
| 1899 | + | |
| 1858 | 1900 | static int stratix10_svc_drv_probe(struct platform_device *pdev) |
| 1859 | 1901 | { |
| 1860 | 1902 | struct device *dev = &pdev->dev; |
| @@ -1862,11 +1904,11 @@ static int stratix10_svc_drv_probe(struct platform_device *pdev) · 1 unmodified line | ||
| 1862 | 1904 | struct stratix10_svc_chan *chans; |
| 1863 | 1905 | struct gen_pool *genpool; |
| 1864 | 1906 | struct stratix10_svc_sh_memory *sh_memory; |
| 1865 | − struct stratix10_svc *svc; | |
| 1907 | + struct stratix10_svc *svc = NULL; | |
| 1866 | 1908 | |
| 1867 | 1909 | svc_invoke_fn *invoke_fn; |
| 1868 | 1910 | size_t fifo_size; |
| 1869 | − int ret; | |
| 1911 | + int ret, i = 0; | |
| 1870 | 1912 | |
| 1871 | 1913 | /* get SMC or HVC function */ |
| 1872 | 1914 | invoke_fn = get_invoke_func(dev); |
| @@ -1905,8 +1947,8 @@ static int stratix10_svc_drv_probe(struct platform_device *pdev) · 32 unmodified lines | ||
| 1905 | 1947 | controller->num_active_client = 0; |
| 1906 | 1948 | controller->chans = chans; |
| 1907 | 1949 | controller->genpool = genpool; |
| 1908 | − controller->task = NULL; | |
| 1909 | 1950 | controller->invoke_fn = invoke_fn; |
| 1951 | + INIT_LIST_HEAD(&controller->node); | |
| 1910 | 1952 | init_completion(&controller->complete_status); |
| 1911 | 1953 | |
| 1912 | 1954 | ret = stratix10_svc_async_init(controller); |
| @@ -1917,32 +1959,20 @@ static int stratix10_svc_drv_probe(struct platform_device *pdev) · 4 unmodified lines | ||
| 1917 | 1959 | } |
| 1918 | 1960 | |
| 1919 | 1961 | fifo_size = sizeof(struct stratix10_svc_data) * SVC_NUM_DATA_IN_FIFO; |
| 1920 | − ret = kfifo_alloc(&controller->svc_fifo, fifo_size, GFP_KERNEL); | |
| 1921 | − if (ret) { | |
| 1922 | − dev_err(dev, "failed to allocate FIFO\n"); | |
| 1923 | − goto err_async_exit; | |
| 1924 | − } | |
| 1925 | − spin_lock_init(&controller->svc_fifo_lock); | |
| 1926 | − | |
| 1927 | − chans[0].scl = NULL; | |
| 1928 | − chans[0].ctrl = controller; | |
| 1929 | − chans[0].name = SVC_CLIENT_FPGA; | |
| 1930 | − spin_lock_init(&chans[0].lock); | |
| 1962 | + mutex_init(&controller->sdm_lock); | |
| 1931 | 1963 | |
| 1932 | − chans[1].scl = NULL; | |
| 1933 | − chans[1].ctrl = controller; | |
| 1934 | − chans[1].name = SVC_CLIENT_RSU; | |
| 1935 | − spin_lock_init(&chans[1].lock); | |
| 1936 | − | |
| 1937 | − chans[2].scl = NULL; | |
| 1938 | − chans[2].ctrl = controller; | |
| 1939 | − chans[2].name = SVC_CLIENT_FCS; | |
| 1940 | − spin_lock_init(&chans[2].lock); | |
| 1941 | − | |
| 1942 | − chans[3].scl = NULL; | |
| 1943 | − chans[3].ctrl = controller; | |
| 1944 | − chans[3].name = SVC_CLIENT_HWMON; | |
| 1945 | − spin_lock_init(&chans[3].lock); | |
| 1964 | + for (i = 0; i < SVC_NUM_CHANNEL; i++) { | |
| 1965 | + chans[i].scl = NULL; | |
| 1966 | + chans[i].ctrl = controller; | |
| 1967 | + chans[i].name = (char *)chan_names[i]; | |
| 1968 | + spin_lock_init(&chans[i].lock); | |
| 1969 | + ret = kfifo_alloc(&chans[i].svc_fifo, fifo_size, GFP_KERNEL); | |
| 1970 | + if (ret) { | |
| 1971 | + dev_err(dev, "failed to allocate FIFO %d\n", i); | |
| 1972 | + goto err_free_fifos; | |
| 1973 | + } | |
| 1974 | + spin_lock_init(&chans[i].svc_fifo_lock); | |
| 1975 | + } | |
| 1946 | 1976 | |
| 1947 | 1977 | list_add_tail(&controller->node, &svc_ctrl); |
| 1948 | 1978 | platform_set_drvdata(pdev, controller); |
| @@ -1951,7 +1981,7 @@ static int stratix10_svc_drv_probe(struct platform_device *pdev) · 2 unmodified lines | ||
| 1951 | 1981 | svc = devm_kzalloc(dev, sizeof(*svc), GFP_KERNEL); |
| 1952 | 1982 | if (!svc) { |
| 1953 | 1983 | ret = -ENOMEM; |
| 1954 | − goto err_free_kfifo; | |
| 1984 | + goto err_free_fifos; | |
| 1955 | 1985 | } |
| 1956 | 1986 | controller->svc = svc; |
| 1957 | 1987 | |
| @@ -1959,51 +1989,43 @@ static int stratix10_svc_drv_probe(struct platform_device *pdev) · 1 unmodified line | ||
| 1959 | 1989 | if (!svc->stratix10_svc_rsu) { |
| 1960 | 1990 | dev_err(dev, "failed to allocate %s device\n", STRATIX10_RSU); |
| 1961 | 1991 | ret = -ENOMEM; |
| 1962 | − goto err_free_kfifo; | |
| 1992 | + goto err_free_fifos; | |
| 1963 | 1993 | } |
| 1964 | 1994 | |
| 1965 | 1995 | ret = platform_device_add(svc->stratix10_svc_rsu); |
| 1966 | − if (ret) { | |
| 1967 | − platform_device_put(svc->stratix10_svc_rsu); | |
| 1968 | − goto err_free_kfifo; | |
| 1969 | − } | |
| 1970 | − | |
| 1971 | − svc->intel_svc_fcs = platform_device_alloc(INTEL_FCS, 1); | |
| 1972 | − if (!svc->intel_svc_fcs) { | |
| 1973 | − dev_err(dev, "failed to allocate %s device\n", INTEL_FCS); | |
| 1974 | − ret = -ENOMEM; | |
| 1975 | − goto err_unregister_rsu_dev; | |
| 1976 | − } | |
| 1977 | − | |
| 1978 | − ret = platform_device_add(svc->intel_svc_fcs); | |
| 1979 | − if (ret) { | |
| 1980 | − platform_device_put(svc->intel_svc_fcs); | |
| 1981 | − goto err_unregister_rsu_dev; | |
| 1982 | − } | |
| 1996 | + if (ret) | |
| 1997 | + goto err_put_device; | |
| 1983 | 1998 | |
| 1984 | 1999 | ret = of_platform_default_populate(dev_of_node(dev), NULL, dev); |
| 1985 | 2000 | if (ret) |
| 1986 | − goto err_unregister_fcs_dev; | |
| 2001 | + goto err_unregister_rsu_dev; | |
| 1987 | 2002 | |
| 1988 | 2003 | pr_info("Intel Service Layer Driver Initialized\n"); |
| 1989 | 2004 | |
| 1990 | 2005 | return 0; |
| 1991 | 2006 | |
| 1992 | −err_unregister_fcs_dev: | |
| 1993 | − platform_device_unregister(svc->intel_svc_fcs); | |
| 1994 | 2007 | err_unregister_rsu_dev: |
| 1995 | 2008 | platform_device_unregister(svc->stratix10_svc_rsu); |
| 1996 | −err_free_kfifo: | |
| 1997 | − kfifo_free(&controller->svc_fifo); | |
| 1998 | −err_async_exit: | |
| 2009 | + goto err_free_fifos; | |
| 2010 | +err_put_device: | |
| 2011 | + platform_device_put(svc->stratix10_svc_rsu); | |
| 2012 | +err_free_fifos: | |
| 2013 | + /* only remove from list if list_add_tail() was reached */ | |
| 2014 | + if (!list_empty(&controller->node)) | |
| 2015 | + list_del(&controller->node); | |
| 2016 | + /* free only the FIFOs that were successfully allocated */ | |
| 2017 | + while (i--) | |
| 2018 | + kfifo_free(&chans[i].svc_fifo); | |
| 1999 | 2019 | stratix10_svc_async_exit(controller); |
| 2000 | 2020 | err_destroy_pool: |
| 2001 | 2021 | gen_pool_destroy(genpool); |
| 2022 | + | |
| 2002 | 2023 | return ret; |
| 2003 | 2024 | } |
| 2004 | 2025 | |
| 2005 | 2026 | static void stratix10_svc_drv_remove(struct platform_device *pdev) |
| 2006 | 2027 | { |
| 2028 | + int i; | |
| 2007 | 2029 | struct stratix10_svc_controller *ctrl = platform_get_drvdata(pdev); |
| 2008 | 2030 | struct stratix10_svc *svc = ctrl->svc; |
| 2009 | 2031 | |
| @@ -2011,14 +2033,16 @@ static void stratix10_svc_drv_remove(struct platform_device *pdev) · 1 unmodified line | ||
| 2011 | 2033 | |
| 2012 | 2034 | of_platform_depopulate(ctrl->dev); |
| 2013 | 2035 | |
| 2014 | − platform_device_unregister(svc->intel_svc_fcs); | |
| 2015 | 2036 | platform_device_unregister(svc->stratix10_svc_rsu); |
| 2016 | 2037 | |
| 2017 | − kfifo_free(&ctrl->svc_fifo); | |
| 2018 | − if (ctrl->task) { | |
| 2019 | − kthread_stop(ctrl->task); | |
| 2020 | − ctrl->task = NULL; | |
| 2038 | + for (i = 0; i < SVC_NUM_CHANNEL; i++) { | |
| 2039 | + if (ctrl->chans[i].task) { | |
| 2040 | + kthread_stop(ctrl->chans[i].task); | |
| 2041 | + ctrl->chans[i].task = NULL; | |
| 2042 | + } | |
| 2043 | + kfifo_free(&ctrl->chans[i].svc_fifo); | |
| 2021 | 2044 | } |
| 2045 | + | |
| 2022 | 2046 | if (ctrl->genpool) |
| 2023 | 2047 | gen_pool_destroy(ctrl->genpool); |
| 2024 | 2048 | list_del(&ctrl->node); |
| @@ -38,8 +38,10 @@ MODULE_DESCRIPTION("GPIB driver for LPVO usb devices"); · 37 unmodified lines | ||
| 38 | 38 | /* |
| 39 | 39 | * Table of devices that work with this driver. |
| 40 | 40 | * |
| 41 | − * Currently, only one device is known to be used in the | |
| 42 | − * lpvo_usb_gpib adapter (FTDI 0403:6001). | |
| 41 | + * Currently, only one device is known to be used in the lpvo_usb_gpib | |
| 42 | + * adapter (FTDI 0403:6001) but as this device id is already handled by the | |
| 43 | + * ftdi_sio USB serial driver the LPVO driver must not bind to it by default. | |
| 44 | + * | |
| 43 | 45 | * If your adapter uses a different chip, insert a line |
| 44 | 46 | * in the following table with proper <Vendor-id>, <Product-id>. |
| 45 | 47 | * |
| @@ -50,7 +52,6 @@ MODULE_DESCRIPTION("GPIB driver for LPVO usb devices"); · 4 unmodified lines | ||
| 50 | 52 | */ |
| 51 | 53 | |
| 52 | 54 | static const struct usb_device_id skel_table[] = { |
| 53 | − { USB_DEVICE(0x0403, 0x6001) }, | |
| 54 | 55 | { } /* Terminating entry */ |
| 55 | 56 | }; |
| 56 | 57 | MODULE_DEVICE_TABLE(usb, skel_table); |
| @@ -748,14 +748,14 @@ static bool gpio_shared_entry_is_really_shared(struct gpio_shared_entry *entry) · 747 unmodified lines | ||
| 748 | 748 | static void gpio_shared_free_exclusive(void) |
| 749 | 749 | { |
| 750 | 750 | struct gpio_shared_entry *entry, *epos; |
| 751 | + struct gpio_shared_ref *ref, *rpos; | |
| 751 | 752 | |
| 752 | 753 | list_for_each_entry_safe(entry, epos, &gpio_shared_list, list) { |
| 753 | 754 | if (gpio_shared_entry_is_really_shared(entry)) |
| 754 | 755 | continue; |
| 755 | 756 | |
| 756 | − gpio_shared_drop_ref(list_first_entry(&entry->refs, | |
| 757 | − struct gpio_shared_ref, | |
| 758 | − list)); | |
| 757 | + list_for_each_entry_safe(ref, rpos, &entry->refs, list) | |
| 758 | + gpio_shared_drop_ref(ref); | |
| 759 | 759 | gpio_shared_drop_entry(entry); |
| 760 | 760 | } |
| 761 | 761 | } |
| @@ -3267,8 +3267,12 @@ static int gpiochip_get(struct gpio_chip *gc, unsigned int offset) · 3266 unmodified lines | ||
| 3267 | 3267 | |
| 3268 | 3268 | /* Make sure this is called after checking for gc->get(). */ |
| 3269 | 3269 | ret = gc->get(gc, offset); |
| 3270 | − if (ret > 1) | |
| 3271 | − ret = -EBADE; | |
| 3270 | + if (ret > 1) { | |
| 3271 | + gpiochip_warn(gc, | |
| 3272 | + "invalid return value from gc->get(): %d, consider fixing the driver\n", | |
| 3273 | + ret); | |
| 3274 | + ret = !!ret; | |
| 3275 | + } | |
| 3272 | 3276 | |
| 3273 | 3277 | return ret; |
| 3274 | 3278 | } |
| @@ -641,6 +641,7 @@ static void aca_error_fini(struct aca_error *aerr) · 640 unmodified lines | ||
| 641 | 641 | aca_bank_error_remove(aerr, bank_error); |
| 642 | 642 | |
| 643 | 643 | out_unlock: |
| 644 | + mutex_unlock(&aerr->lock); | |
| 644 | 645 | mutex_destroy(&aerr->lock); |
| 645 | 646 | } |
| 646 | 647 | |
| @@ -1439,7 +1439,10 @@ static int init_kfd_vm(struct amdgpu_vm *vm, void **process_info, · 1438 unmodified lines | ||
| 1439 | 1439 | *process_info = info; |
| 1440 | 1440 | } |
| 1441 | 1441 | |
| 1442 | − vm->process_info = *process_info; | |
| 1442 | + if (cmpxchg(&vm->process_info, NULL, *process_info) != NULL) { | |
| 1443 | + ret = -EINVAL; | |
| 1444 | + goto already_acquired; | |
| 1445 | + } | |
| 1443 | 1446 | |
| 1444 | 1447 | /* Validate page directory and attach eviction fence */ |
| 1445 | 1448 | ret = amdgpu_bo_reserve(vm->root.bo, true); |
| @@ -1479,6 +1482,7 @@ validate_pd_fail: · 33 unmodified lines | ||
| 1479 | 1482 | amdgpu_bo_unreserve(vm->root.bo); |
| 1480 | 1483 | reserve_pd_fail: |
| 1481 | 1484 | vm->process_info = NULL; |
| 1485 | +already_acquired: | |
| 1482 | 1486 | if (info) { |
| 1483 | 1487 | dma_fence_put(&info->eviction_fence->base); |
| 1484 | 1488 | *process_info = NULL; |
| @@ -2690,8 +2690,10 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) · 2689 unmodified lines | ||
| 2690 | 2690 | break; |
| 2691 | 2691 | default: |
| 2692 | 2692 | r = amdgpu_discovery_set_ip_blocks(adev); |
| 2693 | − if (r) | |
| 2693 | + if (r) { | |
| 2694 | + adev->num_ip_blocks = 0; | |
| 2694 | 2695 | return r; |
| 2696 | + } | |
| 2695 | 2697 | break; |
| 2696 | 2698 | } |
| 2697 | 2699 | |
| @@ -3247,6 +3249,8 @@ int amdgpu_device_set_cg_state(struct amdgpu_device *adev, · 549 unmodified lines | ||
| 3247 | 3249 | i = state == AMD_CG_STATE_GATE ? j : adev->num_ip_blocks - j - 1; |
| 3248 | 3250 | if (!adev->ip_blocks[i].status.late_initialized) |
| 3249 | 3251 | continue; |
| 3252 | + if (!adev->ip_blocks[i].version) | |
| 3253 | + continue; | |
| 3250 | 3254 | /* skip CG for GFX, SDMA on S0ix */ |
| 3251 | 3255 | if (adev->in_s0ix && |
| 3252 | 3256 | (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GFX || |
| @@ -3286,6 +3290,8 @@ int amdgpu_device_set_pg_state(struct amdgpu_device *adev, · 33 unmodified lines | ||
| 3286 | 3290 | i = state == AMD_PG_STATE_GATE ? j : adev->num_ip_blocks - j - 1; |
| 3287 | 3291 | if (!adev->ip_blocks[i].status.late_initialized) |
| 3288 | 3292 | continue; |
| 3293 | + if (!adev->ip_blocks[i].version) | |
| 3294 | + continue; | |
| 3289 | 3295 | /* skip PG for GFX, SDMA on S0ix */ |
| 3290 | 3296 | if (adev->in_s0ix && |
| 3291 | 3297 | (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GFX || |
| @@ -3493,6 +3499,8 @@ static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) · 201 unmodified lines | ||
| 3493 | 3499 | int i, r; |
| 3494 | 3500 | |
| 3495 | 3501 | for (i = 0; i < adev->num_ip_blocks; i++) { |
| 3502 | + if (!adev->ip_blocks[i].version) | |
| 3503 | + continue; | |
| 3496 | 3504 | if (!adev->ip_blocks[i].version->funcs->early_fini) |
| 3497 | 3505 | continue; |
| 3498 | 3506 | |
| @@ -3570,6 +3578,8 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev) · 71 unmodified lines | ||
| 3570 | 3578 | if (!adev->ip_blocks[i].status.sw) |
| 3571 | 3579 | continue; |
| 3572 | 3580 | |
| 3581 | + if (!adev->ip_blocks[i].version) | |
| 3582 | + continue; | |
| 3573 | 3583 | if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GMC) { |
| 3574 | 3584 | amdgpu_ucode_free_bo(adev); |
| 3575 | 3585 | amdgpu_free_static_csa(&adev->virt.csa_obj); |
| @@ -3596,6 +3606,8 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev) · 20 unmodified lines | ||
| 3596 | 3606 | for (i = adev->num_ip_blocks - 1; i >= 0; i--) { |
| 3597 | 3607 | if (!adev->ip_blocks[i].status.late_initialized) |
| 3598 | 3608 | continue; |
| 3609 | + if (!adev->ip_blocks[i].version) | |
| 3610 | + continue; | |
| 3599 | 3611 | if (adev->ip_blocks[i].version->funcs->late_fini) |
| 3600 | 3612 | adev->ip_blocks[i].version->funcs->late_fini(&adev->ip_blocks[i]); |
| 3601 | 3613 | adev->ip_blocks[i].status.late_initialized = false; |
| @@ -7059,6 +7071,15 @@ pci_ers_result_t amdgpu_pci_slot_reset(struct pci_dev *pdev) · 3457 unmodified lines | ||
| 7059 | 7071 | dev_info(adev->dev, "PCI error: slot reset callback!!\n"); |
| 7060 | 7072 | |
| 7061 | 7073 | memset(&reset_context, 0, sizeof(reset_context)); |
| 7074 | + INIT_LIST_HEAD(&device_list); | |
| 7075 | + hive = amdgpu_get_xgmi_hive(adev); | |
| 7076 | + if (hive) { | |
| 7077 | + mutex_lock(&hive->hive_lock); | |
| 7078 | + list_for_each_entry(tmp_adev, &hive->device_list, gmc.xgmi.head) | |
| 7079 | + list_add_tail(&tmp_adev->reset_list, &device_list); | |
| 7080 | + } else { | |
| 7081 | + list_add_tail(&adev->reset_list, &device_list); | |
| 7082 | + } | |
| 7062 | 7083 | |
| 7063 | 7084 | if (adev->pcie_reset_ctx.swus) |
| 7064 | 7085 | link_dev = adev->pcie_reset_ctx.swus; |
| @@ -7099,19 +7120,13 @@ pci_ers_result_t amdgpu_pci_slot_reset(struct pci_dev *pdev) · 34 unmodified lines | ||
| 7099 | 7120 | reset_context.reset_req_dev = adev; |
| 7100 | 7121 | set_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags); |
| 7101 | 7122 | set_bit(AMDGPU_SKIP_COREDUMP, &reset_context.flags); |
| 7102 | − INIT_LIST_HEAD(&device_list); | |
| 7103 | 7123 | |
| 7104 | − hive = amdgpu_get_xgmi_hive(adev); | |
| 7105 | 7124 | if (hive) { |
| 7106 | − mutex_lock(&hive->hive_lock); | |
| 7107 | 7125 | reset_context.hive = hive; |
| 7108 | − list_for_each_entry(tmp_adev, &hive->device_list, gmc.xgmi.head) { | |
| 7126 | + list_for_each_entry(tmp_adev, &hive->device_list, gmc.xgmi.head) | |
| 7109 | 7127 | tmp_adev->pcie_reset_ctx.in_link_reset = true; |
| 7110 | − list_add_tail(&tmp_adev->reset_list, &device_list); | |
| 7111 | − } | |
| 7112 | 7128 | } else { |
| 7113 | 7129 | set_bit(AMDGPU_SKIP_HW_RESET, &reset_context.flags); |
| 7114 | − list_add_tail(&adev->reset_list, &device_list); | |
| 7115 | 7130 | } |
| 7116 | 7131 | |
| 7117 | 7132 | r = amdgpu_device_asic_reset(adev, &device_list, &reset_context); |
| @@ -83,7 +83,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) · 82 unmodified lines | ||
| 83 | 83 | { |
| 84 | 84 | struct amdgpu_device *adev = drm_to_adev(dev); |
| 85 | 85 | |
| 86 | − if (adev == NULL) | |
| 86 | + if (adev == NULL || !adev->num_ip_blocks) | |
| 87 | 87 | return; |
| 88 | 88 | |
| 89 | 89 | amdgpu_unregister_gpu_instance(adev); |
| @@ -368,15 +368,15 @@ struct amdgpu_mode_info { · 367 unmodified lines | ||
| 368 | 368 | |
| 369 | 369 | struct drm_property *plane_ctm_property; |
| 370 | 370 | /** |
| 371 | − * @shaper_lut_property: Plane property to set pre-blending shaper LUT | |
| 372 | − * that converts color content before 3D LUT. If | |
| 373 | − * plane_shaper_tf_property != Identity TF, AMD color module will | |
| 371 | + * @plane_shaper_lut_property: Plane property to set pre-blending | |
| 372 | + * shaper LUT that converts color content before 3D LUT. | |
| 373 | + * If plane_shaper_tf_property != Identity TF, AMD color module will | |
| 374 | 374 | * combine the user LUT values with pre-defined TF into the LUT |
| 375 | 375 | * parameters to be programmed. |
| 376 | 376 | */ |
| 377 | 377 | struct drm_property *plane_shaper_lut_property; |
| 378 | 378 | /** |
| 379 | − * @shaper_lut_size_property: Plane property for the size of | |
| 379 | + * @plane_shaper_lut_size_property: Plane property for the size of | |
| 380 | 380 | * pre-blending shaper LUT as supported by the driver (read-only). |
| 381 | 381 | */ |
| 382 | 382 | struct drm_property *plane_shaper_lut_size_property; |
| @@ -400,10 +400,10 @@ struct amdgpu_mode_info { · 17 unmodified lines | ||
| 400 | 400 | */ |
| 401 | 401 | struct drm_property *plane_lut3d_property; |
| 402 | 402 | /** |
| 403 | − * @plane_degamma_lut_size_property: Plane property to define the max | |
| 404 | − * size of 3D LUT as supported by the driver (read-only). The max size | |
| 405 | − * is the max size of one dimension and, therefore, the max number of | |
| 406 | − * entries for 3D LUT array is the 3D LUT size cubed; | |
| 403 | + * @plane_lut3d_size_property: Plane property to define the max size | |
| 404 | + * of 3D LUT as supported by the driver (read-only). The max size is | |
| 405 | + * the max size of one dimension and, therefore, the max number of | |
| 406 | + * entries for 3D LUT array is the 3D LUT size cubed. | |
| 407 | 407 | */ |
| 408 | 408 | struct drm_property *plane_lut3d_size_property; |
| 409 | 409 | /** |
| @@ -332,13 +332,13 @@ static ssize_t ta_if_invoke_debugfs_write(struct file *fp, const char *buf, size · 331 unmodified lines | ||
| 332 | 332 | if (!context || !context->initialized) { |
| 333 | 333 | dev_err(adev->dev, "TA is not initialized\n"); |
| 334 | 334 | ret = -EINVAL; |
| 335 | − goto err_free_shared_buf; | |
| 335 | + goto free_shared_buf; | |
| 336 | 336 | } |
| 337 | 337 | |
| 338 | 338 | if (!psp->ta_funcs || !psp->ta_funcs->fn_ta_invoke) { |
| 339 | 339 | dev_err(adev->dev, "Unsupported function to invoke TA\n"); |
| 340 | 340 | ret = -EOPNOTSUPP; |
| 341 | − goto err_free_shared_buf; | |
| 341 | + goto free_shared_buf; | |
| 342 | 342 | } |
| 343 | 343 | |
| 344 | 344 | context->session_id = ta_id; |
| @@ -346,7 +346,7 @@ static ssize_t ta_if_invoke_debugfs_write(struct file *fp, const char *buf, size · 1 unmodified line | ||
| 346 | 346 | mutex_lock(&psp->ras_context.mutex); |
| 347 | 347 | ret = prep_ta_mem_context(&context->mem_context, shared_buf, shared_buf_len); |
| 348 | 348 | if (ret) |
| 349 | − goto err_free_shared_buf; | |
| 349 | + goto unlock; | |
| 350 | 350 | |
| 351 | 351 | ret = psp_fn_ta_invoke(psp, cmd_id); |
| 352 | 352 | if (ret || context->resp_status) { |
| @@ -354,15 +354,17 @@ static ssize_t ta_if_invoke_debugfs_write(struct file *fp, const char *buf, size · 1 unmodified line | ||
| 354 | 354 | ret, context->resp_status); |
| 355 | 355 | if (!ret) { |
| 356 | 356 | ret = -EINVAL; |
| 357 | − goto err_free_shared_buf; | |
| 357 | + goto unlock; | |
| 358 | 358 | } |
| 359 | 359 | } |
| 360 | 360 | |
| 361 | 361 | if (copy_to_user((char *)&buf[copy_pos], context->mem_context.shared_buf, shared_buf_len)) |
| 362 | 362 | ret = -EFAULT; |
| 363 | 363 | |
| 364 | −err_free_shared_buf: | |
| 364 | +unlock: | |
| 365 | 365 | mutex_unlock(&psp->ras_context.mutex); |
| 366 | + | |
| 367 | +free_shared_buf: | |
| 366 | 368 | kfree(shared_buf); |
| 367 | 369 | |
| 368 | 370 | return ret; |
| @@ -446,8 +446,7 @@ static int amdgpu_userq_wait_for_last_fence(struct amdgpu_usermode_queue *queue) · 445 unmodified lines | ||
| 446 | 446 | return ret; |
| 447 | 447 | } |
| 448 | 448 | |
| 449 | −static void amdgpu_userq_cleanup(struct amdgpu_usermode_queue *queue, | |
| 450 | − int queue_id) | |
| 449 | +static void amdgpu_userq_cleanup(struct amdgpu_usermode_queue *queue) | |
| 451 | 450 | { |
| 452 | 451 | struct amdgpu_userq_mgr *uq_mgr = queue->userq_mgr; |
| 453 | 452 | struct amdgpu_device *adev = uq_mgr->adev; |
| @@ -461,7 +460,6 @@ static void amdgpu_userq_cleanup(struct amdgpu_usermode_queue *queue, · 7 unmodified lines | ||
| 461 | 460 | uq_funcs->mqd_destroy(queue); |
| 462 | 461 | amdgpu_userq_fence_driver_free(queue); |
| 463 | 462 | /* Use interrupt-safe locking since IRQ handlers may access these XArrays */ |
| 464 | − xa_erase_irq(&uq_mgr->userq_xa, (unsigned long)queue_id); | |
| 465 | 463 | xa_erase_irq(&adev->userq_doorbell_xa, queue->doorbell_index); |
| 466 | 464 | queue->userq_mgr = NULL; |
| 467 | 465 | list_del(&queue->userq_va_list); |
| @@ -470,12 +468,6 @@ static void amdgpu_userq_cleanup(struct amdgpu_usermode_queue *queue, · 2 unmodified lines | ||
| 470 | 468 | up_read(&adev->reset_domain->sem); |
| 471 | 469 | } |
| 472 | 470 | |
| 473 | −static struct amdgpu_usermode_queue * | |
| 474 | −amdgpu_userq_find(struct amdgpu_userq_mgr *uq_mgr, int qid) | |
| 475 | −{ | |
| 476 | − return xa_load(&uq_mgr->userq_xa, qid); | |
| 477 | −} | |
| 478 | − | |
| 479 | 471 | void |
| 480 | 472 | amdgpu_userq_ensure_ev_fence(struct amdgpu_userq_mgr *uq_mgr, |
| 481 | 473 | struct amdgpu_eviction_fence_mgr *evf_mgr) |
| @@ -625,22 +617,13 @@ unref_bo: · 143 unmodified lines | ||
| 625 | 617 | } |
| 626 | 618 | |
| 627 | 619 | static int |
| 628 | −amdgpu_userq_destroy(struct drm_file *filp, int queue_id) | |
| 620 | +amdgpu_userq_destroy(struct amdgpu_userq_mgr *uq_mgr, struct amdgpu_usermode_queue *queue) | |
| 629 | 621 | { |
| 630 | − struct amdgpu_fpriv *fpriv = filp->driver_priv; | |
| 631 | − struct amdgpu_userq_mgr *uq_mgr = &fpriv->userq_mgr; | |
| 632 | 622 | struct amdgpu_device *adev = uq_mgr->adev; |
| 633 | − struct amdgpu_usermode_queue *queue; | |
| 634 | 623 | int r = 0; |
| 635 | 624 | |
| 636 | 625 | cancel_delayed_work_sync(&uq_mgr->resume_work); |
| 637 | 626 | mutex_lock(&uq_mgr->userq_mutex); |
| 638 | − queue = amdgpu_userq_find(uq_mgr, queue_id); | |
| 639 | − if (!queue) { | |
| 640 | − drm_dbg_driver(adev_to_drm(uq_mgr->adev), "Invalid queue id to destroy\n"); | |
| 641 | − mutex_unlock(&uq_mgr->userq_mutex); | |
| 642 | − return -EINVAL; | |
| 643 | − } | |
| 644 | 627 | amdgpu_userq_wait_for_last_fence(queue); |
| 645 | 628 | /* Cancel any pending hang detection work and cleanup */ |
| 646 | 629 | if (queue->hang_detect_fence) { |
| @@ -672,7 +655,7 @@ amdgpu_userq_destroy(struct drm_file *filp, int queue_id) · 25 unmodified lines | ||
| 672 | 655 | drm_warn(adev_to_drm(uq_mgr->adev), "trying to destroy a HW mapping userq\n"); |
| 673 | 656 | queue->state = AMDGPU_USERQ_STATE_HUNG; |
| 674 | 657 | } |
| 675 | − amdgpu_userq_cleanup(queue, queue_id); | |
| 658 | + amdgpu_userq_cleanup(queue); | |
| 676 | 659 | mutex_unlock(&uq_mgr->userq_mutex); |
| 677 | 660 | |
| 678 | 661 | pm_runtime_put_autosuspend(adev_to_drm(adev)->dev); |
| @@ -680,6 +663,37 @@ amdgpu_userq_destroy(struct drm_file *filp, int queue_id) · 1 unmodified line | ||
| 680 | 663 | return r; |
| 681 | 664 | } |
| 682 | 665 | |
| 666 | +static void amdgpu_userq_kref_destroy(struct kref *kref) | |
| 667 | +{ | |
| 668 | + int r; | |
| 669 | + struct amdgpu_usermode_queue *queue = | |
| 670 | + container_of(kref, struct amdgpu_usermode_queue, refcount); | |
| 671 | + struct amdgpu_userq_mgr *uq_mgr = queue->userq_mgr; | |
| 672 | + | |
| 673 | + r = amdgpu_userq_destroy(uq_mgr, queue); | |
| 674 | + if (r) | |
| 675 | + drm_file_err(uq_mgr->file, "Failed to destroy usermode queue %d\n", r); | |
| 676 | +} | |
| 677 | + | |
| 678 | +struct amdgpu_usermode_queue *amdgpu_userq_get(struct amdgpu_userq_mgr *uq_mgr, u32 qid) | |
| 679 | +{ | |
| 680 | + struct amdgpu_usermode_queue *queue; | |
| 681 | + | |
| 682 | + xa_lock(&uq_mgr->userq_xa); | |
| 683 | + queue = xa_load(&uq_mgr->userq_xa, qid); | |
| 684 | + if (queue) | |
| 685 | + kref_get(&queue->refcount); | |
| 686 | + xa_unlock(&uq_mgr->userq_xa); | |
| 687 | + | |
| 688 | + return queue; | |
| 689 | +} | |
| 690 | + | |
| 691 | +void amdgpu_userq_put(struct amdgpu_usermode_queue *queue) | |
| 692 | +{ | |
| 693 | + if (queue) | |
| 694 | + kref_put(&queue->refcount, amdgpu_userq_kref_destroy); | |
| 695 | +} | |
| 696 | + | |
| 683 | 697 | static int amdgpu_userq_priority_permit(struct drm_file *filp, |
| 684 | 698 | int priority) |
| 685 | 699 | { |
| @@ -834,6 +848,9 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args) · 148 unmodified lines | ||
| 834 | 848 | goto unlock; |
| 835 | 849 | } |
| 836 | 850 | |
| 851 | + /* drop this refcount during queue destroy */ | |
| 852 | + kref_init(&queue->refcount); | |
| 853 | + | |
| 837 | 854 | /* Wait for mode-1 reset to complete */ |
| 838 | 855 | down_read(&adev->reset_domain->sem); |
| 839 | 856 | r = xa_err(xa_store_irq(&adev->userq_doorbell_xa, index, queue, GFP_KERNEL)); |
| @@ -985,7 +1002,9 @@ int amdgpu_userq_ioctl(struct drm_device *dev, void *data, · 145 unmodified lines | ||
| 985 | 1002 | struct drm_file *filp) |
| 986 | 1003 | { |
| 987 | 1004 | union drm_amdgpu_userq *args = data; |
| 988 | − int r; | |
| 1005 | + struct amdgpu_fpriv *fpriv = filp->driver_priv; | |
| 1006 | + struct amdgpu_usermode_queue *queue; | |
| 1007 | + int r = 0; | |
| 989 | 1008 | |
| 990 | 1009 | if (!amdgpu_userq_enabled(dev)) |
| 991 | 1010 | return -ENOTSUPP; |
| @@ -1000,11 +1019,16 @@ int amdgpu_userq_ioctl(struct drm_device *dev, void *data, · 8 unmodified lines | ||
| 1000 | 1019 | drm_file_err(filp, "Failed to create usermode queue\n"); |
| 1001 | 1020 | break; |
| 1002 | 1021 | |
| 1003 | − case AMDGPU_USERQ_OP_FREE: | |
| 1004 | − r = amdgpu_userq_destroy(filp, args->in.queue_id); | |
| 1005 | − if (r) | |
| 1006 | − drm_file_err(filp, "Failed to destroy usermode queue\n"); | |
| 1022 | + case AMDGPU_USERQ_OP_FREE: { | |
| 1023 | + xa_lock(&fpriv->userq_mgr.userq_xa); | |
| 1024 | + queue = __xa_erase(&fpriv->userq_mgr.userq_xa, args->in.queue_id); | |
| 1025 | + xa_unlock(&fpriv->userq_mgr.userq_xa); | |
| 1026 | + if (!queue) | |
| 1027 | + return -ENOENT; | |
| 1028 | + | |
| 1029 | + amdgpu_userq_put(queue); | |
| 1007 | 1030 | break; |
| 1031 | + } | |
| 1008 | 1032 | |
| 1009 | 1033 | default: |
| 1010 | 1034 | drm_dbg_driver(dev, "Invalid user queue op specified: %d\n", args->in.op); |
| @@ -1023,16 +1047,23 @@ amdgpu_userq_restore_all(struct amdgpu_userq_mgr *uq_mgr) · 12 unmodified lines | ||
| 1023 | 1047 | |
| 1024 | 1048 | /* Resume all the queues for this process */ |
| 1025 | 1049 | xa_for_each(&uq_mgr->userq_xa, queue_id, queue) { |
| 1050 | + queue = amdgpu_userq_get(uq_mgr, queue_id); | |
| 1051 | + if (!queue) | |
| 1052 | + continue; | |
| 1053 | + | |
| 1026 | 1054 | if (!amdgpu_userq_buffer_vas_mapped(queue)) { |
| 1027 | 1055 | drm_file_err(uq_mgr->file, |
| 1028 | 1056 | "trying restore queue without va mapping\n"); |
| 1029 | 1057 | queue->state = AMDGPU_USERQ_STATE_INVALID_VA; |
| 1058 | + amdgpu_userq_put(queue); | |
| 1030 | 1059 | continue; |
| 1031 | 1060 | } |
| 1032 | 1061 | |
| 1033 | 1062 | r = amdgpu_userq_restore_helper(queue); |
| 1034 | 1063 | if (r) |
| 1035 | 1064 | ret = r; |
| 1065 | + | |
| 1066 | + amdgpu_userq_put(queue); | |
| 1036 | 1067 | } |
| 1037 | 1068 | |
| 1038 | 1069 | if (ret) |
| @@ -1266,9 +1297,13 @@ amdgpu_userq_evict_all(struct amdgpu_userq_mgr *uq_mgr) · 227 unmodified lines | ||
| 1266 | 1297 | amdgpu_userq_detect_and_reset_queues(uq_mgr); |
| 1267 | 1298 | /* Try to unmap all the queues in this process ctx */ |
| 1268 | 1299 | xa_for_each(&uq_mgr->userq_xa, queue_id, queue) { |
| 1300 | + queue = amdgpu_userq_get(uq_mgr, queue_id); | |
| 1301 | + if (!queue) | |
| 1302 | + continue; | |
| 1269 | 1303 | r = amdgpu_userq_preempt_helper(queue); |
| 1270 | 1304 | if (r) |
| 1271 | 1305 | ret = r; |
| 1306 | + amdgpu_userq_put(queue); | |
| 1272 | 1307 | } |
| 1273 | 1308 | |
| 1274 | 1309 | if (ret) |
| @@ -1301,16 +1336,24 @@ amdgpu_userq_wait_for_signal(struct amdgpu_userq_mgr *uq_mgr) · 26 unmodified lines | ||
| 1301 | 1336 | int ret; |
| 1302 | 1337 | |
| 1303 | 1338 | xa_for_each(&uq_mgr->userq_xa, queue_id, queue) { |
| 1339 | + queue = amdgpu_userq_get(uq_mgr, queue_id); | |
| 1340 | + if (!queue) | |
| 1341 | + continue; | |
| 1342 | + | |
| 1304 | 1343 | struct dma_fence *f = queue->last_fence; |
| 1305 | 1344 | |
| 1306 | − if (!f || dma_fence_is_signaled(f)) | |
| 1345 | + if (!f || dma_fence_is_signaled(f)) { | |
| 1346 | + amdgpu_userq_put(queue); | |
| 1307 | 1347 | continue; |
| 1348 | + } | |
| 1308 | 1349 | ret = dma_fence_wait_timeout(f, true, msecs_to_jiffies(100)); |
| 1309 | 1350 | if (ret <= 0) { |
| 1310 | 1351 | drm_file_err(uq_mgr->file, "Timed out waiting for fence=%llu:%llu\n", |
| 1311 | 1352 | f->context, f->seqno); |
| 1353 | + amdgpu_userq_put(queue); | |
| 1312 | 1354 | return -ETIMEDOUT; |
| 1313 | 1355 | } |
| 1356 | + amdgpu_userq_put(queue); | |
| 1314 | 1357 | } |
| 1315 | 1358 | |
| 1316 | 1359 | return 0; |
| @@ -1361,20 +1404,23 @@ int amdgpu_userq_mgr_init(struct amdgpu_userq_mgr *userq_mgr, struct drm_file *f · 44 unmodified lines | ||
| 1361 | 1404 | void amdgpu_userq_mgr_fini(struct amdgpu_userq_mgr *userq_mgr) |
| 1362 | 1405 | { |
| 1363 | 1406 | struct amdgpu_usermode_queue *queue; |
| 1364 | − unsigned long queue_id; | |
| 1407 | + unsigned long queue_id = 0; | |
| 1408 | + | |
| 1409 | + for (;;) { | |
| 1410 | + xa_lock(&userq_mgr->userq_xa); | |
| 1411 | + queue = xa_find(&userq_mgr->userq_xa, &queue_id, ULONG_MAX, | |
| 1412 | + XA_PRESENT); | |
| 1413 | + if (queue) | |
| 1414 | + __xa_erase(&userq_mgr->userq_xa, queue_id); | |
| 1415 | + xa_unlock(&userq_mgr->userq_xa); | |
| 1365 | 1416 | |
| 1366 | − cancel_delayed_work_sync(&userq_mgr->resume_work); | |
| 1417 | + if (!queue) | |
| 1418 | + break; | |
| 1367 | 1419 | |
| 1368 | − mutex_lock(&userq_mgr->userq_mutex); | |
| 1369 | − amdgpu_userq_detect_and_reset_queues(userq_mgr); | |
| 1370 | − xa_for_each(&userq_mgr->userq_xa, queue_id, queue) { | |
| 1371 | − amdgpu_userq_wait_for_last_fence(queue); | |
| 1372 | − amdgpu_userq_unmap_helper(queue); | |
| 1373 | − amdgpu_userq_cleanup(queue, queue_id); | |
| 1420 | + amdgpu_userq_put(queue); | |
| 1374 | 1421 | } |
| 1375 | 1422 | |
| 1376 | 1423 | xa_destroy(&userq_mgr->userq_xa); |
| 1377 | − mutex_unlock(&userq_mgr->userq_mutex); | |
| 1378 | 1424 | mutex_destroy(&userq_mgr->userq_mutex); |
| 1379 | 1425 | } |
| 1380 | 1426 | |
| @@ -74,6 +74,7 @@ struct amdgpu_usermode_queue { · 73 unmodified lines | ||
| 74 | 74 | struct dentry *debugfs_queue; |
| 75 | 75 | struct delayed_work hang_detect_work; |
| 76 | 76 | struct dma_fence *hang_detect_fence; |
| 77 | + struct kref refcount; | |
| 77 | 78 | |
| 78 | 79 | struct list_head userq_va_list; |
| 79 | 80 | }; |
| @@ -112,6 +113,9 @@ struct amdgpu_db_info { · 32 unmodified lines | ||
| 112 | 113 | struct amdgpu_userq_obj *db_obj; |
| 113 | 114 | }; |
| 114 | 115 | |
| 116 | +struct amdgpu_usermode_queue *amdgpu_userq_get(struct amdgpu_userq_mgr *uq_mgr, u32 qid); | |
| 117 | +void amdgpu_userq_put(struct amdgpu_usermode_queue *queue); | |
| 118 | + | |
| 115 | 119 | int amdgpu_userq_ioctl(struct drm_device *dev, void *data, struct drm_file *filp); |
| 116 | 120 | |
| 117 | 121 | int amdgpu_userq_mgr_init(struct amdgpu_userq_mgr *userq_mgr, struct drm_file *file_priv, |
| @@ -35,6 +35,8 @@ · 34 unmodified lines | ||
| 35 | 35 | static const struct dma_fence_ops amdgpu_userq_fence_ops; |
| 36 | 36 | static struct kmem_cache *amdgpu_userq_fence_slab; |
| 37 | 37 | |
| 38 | +#define AMDGPU_USERQ_MAX_HANDLES (1U << 16) | |
| 39 | + | |
| 38 | 40 | int amdgpu_userq_fence_slab_init(void) |
| 39 | 41 | { |
| 40 | 42 | amdgpu_userq_fence_slab = kmem_cache_create("amdgpu_userq_fence", |
| @@ -464,7 +466,7 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, · 423 unmodified lines | ||
| 464 | 466 | struct drm_amdgpu_userq_signal *args = data; |
| 465 | 467 | struct drm_gem_object **gobj_write = NULL; |
| 466 | 468 | struct drm_gem_object **gobj_read = NULL; |
| 467 | − struct amdgpu_usermode_queue *queue; | |
| 469 | + struct amdgpu_usermode_queue *queue = NULL; | |
| 468 | 470 | struct amdgpu_userq_fence *userq_fence; |
| 469 | 471 | struct drm_syncobj **syncobj = NULL; |
| 470 | 472 | u32 *bo_handles_write, num_write_bo_handles; |
| @@ -478,6 +480,11 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, · 7 unmodified lines | ||
| 478 | 480 | if (!amdgpu_userq_enabled(dev)) |
| 479 | 481 | return -ENOTSUPP; |
| 480 | 482 | |
| 483 | + if (args->num_syncobj_handles > AMDGPU_USERQ_MAX_HANDLES || | |
| 484 | + args->num_bo_write_handles > AMDGPU_USERQ_MAX_HANDLES || | |
| 485 | + args->num_bo_read_handles > AMDGPU_USERQ_MAX_HANDLES) | |
| 486 | + return -EINVAL; | |
| 487 | + | |
| 481 | 488 | num_syncobj_handles = args->num_syncobj_handles; |
| 482 | 489 | syncobj_handles = memdup_user(u64_to_user_ptr(args->syncobj_handles), |
| 483 | 490 | size_mul(sizeof(u32), num_syncobj_handles)); |
| @@ -546,7 +553,7 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, · 62 unmodified lines | ||
| 546 | 553 | } |
| 547 | 554 | |
| 548 | 555 | /* Retrieve the user queue */ |
| 549 | − queue = xa_load(&userq_mgr->userq_xa, args->queue_id); | |
| 556 | + queue = amdgpu_userq_get(userq_mgr, args->queue_id); | |
| 550 | 557 | if (!queue) { |
| 551 | 558 | r = -ENOENT; |
| 552 | 559 | goto put_gobj_write; |
| @@ -641,6 +648,9 @@ free_syncobj: · 88 unmodified lines | ||
| 641 | 648 | free_syncobj_handles: |
| 642 | 649 | kfree(syncobj_handles); |
| 643 | 650 | |
| 651 | + if (queue) | |
| 652 | + amdgpu_userq_put(queue); | |
| 653 | + | |
| 644 | 654 | return r; |
| 645 | 655 | } |
| 646 | 656 | |
| @@ -653,7 +663,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 6 unmodified lines | ||
| 653 | 663 | struct drm_amdgpu_userq_wait *wait_info = data; |
| 654 | 664 | struct amdgpu_fpriv *fpriv = filp->driver_priv; |
| 655 | 665 | struct amdgpu_userq_mgr *userq_mgr = &fpriv->userq_mgr; |
| 656 | − struct amdgpu_usermode_queue *waitq; | |
| 666 | + struct amdgpu_usermode_queue *waitq = NULL; | |
| 657 | 667 | struct drm_gem_object **gobj_write; |
| 658 | 668 | struct drm_gem_object **gobj_read; |
| 659 | 669 | struct dma_fence **fences = NULL; |
| @@ -664,6 +674,11 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 4 unmodified lines | ||
| 664 | 674 | if (!amdgpu_userq_enabled(dev)) |
| 665 | 675 | return -ENOTSUPP; |
| 666 | 676 | |
| 677 | + if (wait_info->num_syncobj_handles > AMDGPU_USERQ_MAX_HANDLES || | |
| 678 | + wait_info->num_bo_write_handles > AMDGPU_USERQ_MAX_HANDLES || | |
| 679 | + wait_info->num_bo_read_handles > AMDGPU_USERQ_MAX_HANDLES) | |
| 680 | + return -EINVAL; | |
| 681 | + | |
| 667 | 682 | num_read_bo_handles = wait_info->num_bo_read_handles; |
| 668 | 683 | bo_handles_read = memdup_user(u64_to_user_ptr(wait_info->bo_read_handles), |
| 669 | 684 | size_mul(sizeof(u32), num_read_bo_handles)); |
| @@ -833,7 +848,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 163 unmodified lines | ||
| 833 | 848 | |
| 834 | 849 | dma_resv_for_each_fence(&resv_cursor, gobj_read[i]->resv, |
| 835 | 850 | DMA_RESV_USAGE_READ, fence) { |
| 836 | − if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) { | |
| 851 | + if (num_fences >= wait_info->num_fences) { | |
| 837 | 852 | r = -EINVAL; |
| 838 | 853 | goto free_fences; |
| 839 | 854 | } |
| @@ -850,7 +865,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 10 unmodified lines | ||
| 850 | 865 | |
| 851 | 866 | dma_resv_for_each_fence(&resv_cursor, gobj_write[i]->resv, |
| 852 | 867 | DMA_RESV_USAGE_WRITE, fence) { |
| 853 | − if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) { | |
| 868 | + if (num_fences >= wait_info->num_fences) { | |
| 854 | 869 | r = -EINVAL; |
| 855 | 870 | goto free_fences; |
| 856 | 871 | } |
| @@ -874,8 +889,9 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 17 unmodified lines | ||
| 874 | 889 | goto free_fences; |
| 875 | 890 | |
| 876 | 891 | dma_fence_unwrap_for_each(f, &iter, fence) { |
| 877 | − if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) { | |
| 892 | + if (num_fences >= wait_info->num_fences) { | |
| 878 | 893 | r = -EINVAL; |
| 894 | + dma_fence_put(fence); | |
| 879 | 895 | goto free_fences; |
| 880 | 896 | } |
| 881 | 897 | |
| @@ -898,8 +914,9 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 16 unmodified lines | ||
| 898 | 914 | if (r) |
| 899 | 915 | goto free_fences; |
| 900 | 916 | |
| 901 | − if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) { | |
| 917 | + if (num_fences >= wait_info->num_fences) { | |
| 902 | 918 | r = -EINVAL; |
| 919 | + dma_fence_put(fence); | |
| 903 | 920 | goto free_fences; |
| 904 | 921 | } |
| 905 | 922 | |
| @@ -912,7 +929,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 6 unmodified lines | ||
| 912 | 929 | */ |
| 913 | 930 | num_fences = dma_fence_dedup_array(fences, num_fences); |
| 914 | 931 | |
| 915 | − waitq = xa_load(&userq_mgr->userq_xa, wait_info->waitq_id); | |
| 932 | + waitq = amdgpu_userq_get(userq_mgr, wait_info->waitq_id); | |
| 916 | 933 | if (!waitq) { |
| 917 | 934 | r = -EINVAL; |
| 918 | 935 | goto free_fences; |
| @@ -969,32 +986,14 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, · 50 unmodified lines | ||
| 969 | 986 | r = -EFAULT; |
| 970 | 987 | goto free_fences; |
| 971 | 988 | } |
| 972 | − | |
| 973 | − kfree(fences); | |
| 974 | − kfree(fence_info); | |
| 975 | 989 | } |
| 976 | 990 | |
| 977 | − drm_exec_fini(&exec); | |
| 978 | − for (i = 0; i < num_read_bo_handles; i++) | |
| 979 | − drm_gem_object_put(gobj_read[i]); | |
| 980 | − kfree(gobj_read); | |
| 981 | − | |
| 982 | − for (i = 0; i < num_write_bo_handles; i++) | |
| 983 | − drm_gem_object_put(gobj_write[i]); | |
| 984 | − kfree(gobj_write); | |
| 985 | − | |
| 986 | − kfree(timeline_points); | |
| 987 | − kfree(timeline_handles); | |
| 988 | − kfree(syncobj_handles); | |
| 989 | − kfree(bo_handles_write); | |
| 990 | − kfree(bo_handles_read); | |
| 991 | − | |
| 992 | − return 0; | |
| 993 | − | |
| 994 | 991 | free_fences: |
| 995 | − while (num_fences-- > 0) | |
| 996 | − dma_fence_put(fences[num_fences]); | |
| 997 | − kfree(fences); | |
| 992 | + if (fences) { | |
| 993 | + while (num_fences-- > 0) | |
| 994 | + dma_fence_put(fences[num_fences]); | |
| 995 | + kfree(fences); | |
| 996 | + } | |
| 998 | 997 | free_fence_info: |
| 999 | 998 | kfree(fence_info); |
| 1000 | 999 | exec_fini: |
| @@ -1018,5 +1017,8 @@ free_bo_handles_write: · 17 unmodified lines | ||
| 1018 | 1017 | free_bo_handles_read: |
| 1019 | 1018 | kfree(bo_handles_read); |
| 1020 | 1019 | |
| 1020 | + if (waitq) | |
| 1021 | + amdgpu_userq_put(waitq); | |
| 1022 | + | |
| 1021 | 1023 | return r; |
| 1022 | 1024 | } |
| @@ -720,11 +720,6 @@ static int mes_v11_0_set_hw_resources(struct amdgpu_mes *mes) · 719 unmodified lines | ||
| 720 | 720 | mes_set_hw_res_pkt.enable_reg_active_poll = 1; |
| 721 | 721 | mes_set_hw_res_pkt.enable_level_process_quantum_check = 1; |
| 722 | 722 | mes_set_hw_res_pkt.oversubscription_timer = 50; |
| 723 | − if ((mes->adev->mes.sched_version & AMDGPU_MES_VERSION_MASK) >= 0x7f) | |
| 724 | − mes_set_hw_res_pkt.enable_lr_compute_wa = 1; | |
| 725 | − else | |
| 726 | − dev_info_once(mes->adev->dev, | |
| 727 | − "MES FW version must be >= 0x7f to enable LR compute workaround.\n"); | |
| 728 | 723 | |
| 729 | 724 | if (amdgpu_mes_log_enable) { |
| 730 | 725 | mes_set_hw_res_pkt.enable_mes_event_int_logging = 1; |
| @@ -731,6 +731,9 @@ static int mes_v12_0_set_hw_resources(struct amdgpu_mes *mes, int pipe) · 730 unmodified lines | ||
| 731 | 731 | int i; |
| 732 | 732 | struct amdgpu_device *adev = mes->adev; |
| 733 | 733 | union MESAPI_SET_HW_RESOURCES mes_set_hw_res_pkt; |
| 734 | + uint32_t mes_rev = (pipe == AMDGPU_MES_SCHED_PIPE) ? | |
| 735 | + (mes->sched_version & AMDGPU_MES_VERSION_MASK) : | |
| 736 | + (mes->kiq_version & AMDGPU_MES_VERSION_MASK); | |
| 734 | 737 | |
| 735 | 738 | memset(&mes_set_hw_res_pkt, 0, sizeof(mes_set_hw_res_pkt)); |
| 736 | 739 | |
| @@ -779,18 +782,13 @@ static int mes_v12_0_set_hw_resources(struct amdgpu_mes *mes, int pipe) · 42 unmodified lines | ||
| 779 | 782 | mes_set_hw_res_pkt.use_different_vmid_compute = 1; |
| 780 | 783 | mes_set_hw_res_pkt.enable_reg_active_poll = 1; |
| 781 | 784 | mes_set_hw_res_pkt.enable_level_process_quantum_check = 1; |
| 782 | − if ((mes->adev->mes.sched_version & AMDGPU_MES_VERSION_MASK) >= 0x82) | |
| 783 | − mes_set_hw_res_pkt.enable_lr_compute_wa = 1; | |
| 784 | − else | |
| 785 | − dev_info_once(adev->dev, | |
| 786 | − "MES FW version must be >= 0x82 to enable LR compute workaround.\n"); | |
| 787 | 785 | |
| 788 | 786 | /* |
| 789 | 787 | * Keep oversubscribe timer for sdma . When we have unmapped doorbell |
| 790 | 788 | * handling support, other queue will not use the oversubscribe timer. |
| 791 | 789 | * handling mode - 0: disabled; 1: basic version; 2: basic+ version |
| 792 | 790 | */ |
| 793 | − mes_set_hw_res_pkt.oversubscription_timer = 50; | |
| 791 | + mes_set_hw_res_pkt.oversubscription_timer = mes_rev < 0x8b ? 0 : 50; | |
| 794 | 792 | mes_set_hw_res_pkt.unmapped_doorbell_handling = 1; |
| 795 | 793 | |
| 796 | 794 | if (amdgpu_mes_log_enable) { |
| @@ -69,12 +69,12 @@ static int psp_v15_0_0_ring_stop(struct psp_context *psp, · 68 unmodified lines | ||
| 69 | 69 | 0x80000000, 0x80000000, false); |
| 70 | 70 | } else { |
| 71 | 71 | /* Write the ring destroy command*/ |
| 72 | − WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_64, | |
| 72 | + WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64, | |
| 73 | 73 | GFX_CTRL_CMD_ID_DESTROY_RINGS); |
| 74 | 74 | /* there might be handshake issue with hardware which needs delay */ |
| 75 | 75 | mdelay(20); |
| 76 | 76 | /* Wait for response flag (bit 31) */ |
| 77 | − ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_SMN_C2PMSG_64), | |
| 77 | + ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64), | |
| 78 | 78 | 0x80000000, 0x80000000, false); |
| 79 | 79 | } |
| 80 | 80 | |
| @@ -116,7 +116,7 @@ static int psp_v15_0_0_ring_create(struct psp_context *psp, · 35 unmodified lines | ||
| 116 | 116 | |
| 117 | 117 | } else { |
| 118 | 118 | /* Wait for sOS ready for ring creation */ |
| 119 | − ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_SMN_C2PMSG_64), | |
| 119 | + ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64), | |
| 120 | 120 | 0x80000000, 0x80000000, false); |
| 121 | 121 | if (ret) { |
| 122 | 122 | DRM_ERROR("Failed to wait for trust OS ready for ring creation\n"); |
| @@ -125,23 +125,23 @@ static int psp_v15_0_0_ring_create(struct psp_context *psp, · 2 unmodified lines | ||
| 125 | 125 | |
| 126 | 126 | /* Write low address of the ring to C2PMSG_69 */ |
| 127 | 127 | psp_ring_reg = lower_32_bits(ring->ring_mem_mc_addr); |
| 128 | − WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_69, psp_ring_reg); | |
| 128 | + WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_69, psp_ring_reg); | |
| 129 | 129 | /* Write high address of the ring to C2PMSG_70 */ |
| 130 | 130 | psp_ring_reg = upper_32_bits(ring->ring_mem_mc_addr); |
| 131 | − WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_70, psp_ring_reg); | |
| 131 | + WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_70, psp_ring_reg); | |
| 132 | 132 | /* Write size of ring to C2PMSG_71 */ |
| 133 | 133 | psp_ring_reg = ring->ring_size; |
| 134 | − WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_71, psp_ring_reg); | |
| 134 | + WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_71, psp_ring_reg); | |
| 135 | 135 | /* Write the ring initialization command to C2PMSG_64 */ |
| 136 | 136 | psp_ring_reg = ring_type; |
| 137 | 137 | psp_ring_reg = psp_ring_reg << 16; |
| 138 | − WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_64, psp_ring_reg); | |
| 138 | + WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64, psp_ring_reg); | |
| 139 | 139 | |
| 140 | 140 | /* there might be handshake issue with hardware which needs delay */ |
| 141 | 141 | mdelay(20); |
| 142 | 142 | |
| 143 | 143 | /* Wait for response flag (bit 31) in C2PMSG_64 */ |
| 144 | − ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_SMN_C2PMSG_64), | |
| 144 | + ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_64), | |
| 145 | 145 | 0x80000000, 0x8000FFFF, false); |
| 146 | 146 | } |
| 147 | 147 | |
| @@ -174,7 +174,7 @@ static uint32_t psp_v15_0_0_ring_get_wptr(struct psp_context *psp) · 26 unmodified lines | ||
| 174 | 174 | if (amdgpu_sriov_vf(adev)) |
| 175 | 175 | data = RREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_102); |
| 176 | 176 | else |
| 177 | − data = RREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_67); | |
| 177 | + data = RREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_67); | |
| 178 | 178 | |
| 179 | 179 | return data; |
| 180 | 180 | } |
| @@ -188,7 +188,7 @@ static void psp_v15_0_0_ring_set_wptr(struct psp_context *psp, uint32_t value) · 7 unmodified lines | ||
| 188 | 188 | WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_101, |
| 189 | 189 | GFX_CTRL_CMD_ID_CONSUME_CMD); |
| 190 | 190 | } else |
| 191 | − WREG32_SOC15(MP0, 0, regMPASP_SMN_C2PMSG_67, value); | |
| 191 | + WREG32_SOC15(MP0, 0, regMPASP_PCRU1_MPASP_C2PMSG_67, value); | |
| 192 | 192 | } |
| 193 | 193 | |
| 194 | 194 | static const struct psp_funcs psp_v15_0_0_funcs = { |
| @@ -858,7 +858,9 @@ static int soc21_common_early_init(struct amdgpu_ip_block *ip_block) · 857 unmodified lines | ||
| 858 | 858 | AMD_CG_SUPPORT_IH_CG | |
| 859 | 859 | AMD_CG_SUPPORT_BIF_MGCG | |
| 860 | 860 | AMD_CG_SUPPORT_BIF_LS; |
| 861 | − adev->pg_flags = AMD_PG_SUPPORT_VCN | | |
| 861 | + adev->pg_flags = AMD_PG_SUPPORT_VCN_DPG | | |
| 862 | + AMD_PG_SUPPORT_VCN | | |
| 863 | + AMD_PG_SUPPORT_JPEG_DPG | | |
| 862 | 864 | AMD_PG_SUPPORT_JPEG | |
| 863 | 865 | AMD_PG_SUPPORT_GFX_PG; |
| 864 | 866 | adev->external_rev_id = adev->rev_id + 0x1; |
| @@ -174,6 +174,10 @@ static int vcn_v5_0_0_sw_init(struct amdgpu_ip_block *ip_block) · 173 unmodified lines | ||
| 174 | 174 | fw_shared->present_flag_0 = cpu_to_le32(AMDGPU_FW_SHARED_FLAG_0_UNIFIED_QUEUE); |
| 175 | 175 | fw_shared->sq.is_enabled = 1; |
| 176 | 176 | |
| 177 | + fw_shared->present_flag_0 |= cpu_to_le32(AMDGPU_VCN_SMU_DPM_INTERFACE_FLAG); | |
| 178 | + fw_shared->smu_dpm_interface.smu_interface_type = (adev->flags & AMD_IS_APU) ? | |
| 179 | + AMDGPU_VCN_SMU_DPM_INTERFACE_APU : AMDGPU_VCN_SMU_DPM_INTERFACE_DGPU; | |
| 180 | + | |
| 177 | 181 | if (amdgpu_vcnfw_log) |
| 178 | 182 | amdgpu_vcn_fwlog_init(&adev->vcn.inst[i]); |
| 179 | 183 | |
| @@ -593,6 +593,7 @@ int pqm_update_queue_properties(struct process_queue_manager *pqm, · 592 unmodified lines | ||
| 593 | 593 | p->queue_size)) { |
| 594 | 594 | pr_debug("ring buf 0x%llx size 0x%llx not mapped on GPU\n", |
| 595 | 595 | p->queue_address, p->queue_size); |
| 596 | + amdgpu_bo_unreserve(vm->root.bo); | |
| 596 | 597 | return -EFAULT; |
| 597 | 598 | } |
| 598 | 599 | |
| @@ -1706,6 +1706,7 @@ __set_dm_plane_colorop_3dlut(struct drm_plane_state *plane_state, · 1705 unmodified lines | ||
| 1706 | 1706 | struct dc_transfer_func *tf = &dc_plane_state->in_shaper_func; |
| 1707 | 1707 | struct drm_atomic_state *state = plane_state->state; |
| 1708 | 1708 | const struct amdgpu_device *adev = drm_to_adev(colorop->dev); |
| 1709 | + bool has_3dlut = adev->dm.dc->caps.color.dpp.hw_3d_lut || adev->dm.dc->caps.color.mpc.preblend; | |
| 1709 | 1710 | const struct drm_device *dev = colorop->dev; |
| 1710 | 1711 | const struct drm_color_lut32 *lut3d; |
| 1711 | 1712 | uint32_t lut3d_size; |
| @@ -1722,7 +1723,7 @@ __set_dm_plane_colorop_3dlut(struct drm_plane_state *plane_state, · 10 unmodified lines | ||
| 1722 | 1723 | } |
| 1723 | 1724 | |
| 1724 | 1725 | if (colorop_state && !colorop_state->bypass && colorop->type == DRM_COLOROP_3D_LUT) { |
| 1725 | − if (!adev->dm.dc->caps.color.dpp.hw_3d_lut) { | |
| 1726 | + if (!has_3dlut) { | |
| 1726 | 1727 | drm_dbg(dev, "3D LUT is not supported by hardware\n"); |
| 1727 | 1728 | return -EINVAL; |
| 1728 | 1729 | } |
| @@ -1875,6 +1876,7 @@ amdgpu_dm_plane_set_colorop_properties(struct drm_plane_state *plane_state, · 146 unmodified lines | ||
| 1875 | 1876 | struct drm_colorop *colorop = plane_state->color_pipeline; |
| 1876 | 1877 | struct drm_device *dev = plane_state->plane->dev; |
| 1877 | 1878 | struct amdgpu_device *adev = drm_to_adev(dev); |
| 1879 | + bool has_3dlut = adev->dm.dc->caps.color.dpp.hw_3d_lut || adev->dm.dc->caps.color.mpc.preblend; | |
| 1878 | 1880 | int ret; |
| 1879 | 1881 | |
| 1880 | 1882 | /* 1D Curve - DEGAM TF */ |
| @@ -1907,7 +1909,7 @@ amdgpu_dm_plane_set_colorop_properties(struct drm_plane_state *plane_state, · 26 unmodified lines | ||
| 1907 | 1909 | if (ret) |
| 1908 | 1910 | return ret; |
| 1909 | 1911 | |
| 1910 | − if (adev->dm.dc->caps.color.dpp.hw_3d_lut) { | |
| 1912 | + if (has_3dlut) { | |
| 1911 | 1913 | /* 1D Curve & LUT - SHAPER TF & LUT */ |
| 1912 | 1914 | colorop = colorop->next; |
| 1913 | 1915 | if (!colorop) { |
| @@ -60,6 +60,7 @@ int amdgpu_dm_initialize_default_pipeline(struct drm_plane *plane, struct drm_pr · 59 unmodified lines | ||
| 60 | 60 | struct drm_colorop *ops[MAX_COLOR_PIPELINE_OPS]; |
| 61 | 61 | struct drm_device *dev = plane->dev; |
| 62 | 62 | struct amdgpu_device *adev = drm_to_adev(dev); |
| 63 | + bool has_3dlut = adev->dm.dc->caps.color.dpp.hw_3d_lut || adev->dm.dc->caps.color.mpc.preblend; | |
| 63 | 64 | int ret; |
| 64 | 65 | int i = 0; |
| 65 | 66 | |
| @@ -112,7 +113,7 @@ int amdgpu_dm_initialize_default_pipeline(struct drm_plane *plane, struct drm_pr · 46 unmodified lines | ||
| 112 | 113 | |
| 113 | 114 | i++; |
| 114 | 115 | |
| 115 | − if (adev->dm.dc->caps.color.dpp.hw_3d_lut) { | |
| 116 | + if (has_3dlut) { | |
| 116 | 117 | /* 1D curve - SHAPER TF */ |
| 117 | 118 | ops[i] = kzalloc_obj(*ops[0]); |
| 118 | 119 | if (!ops[i]) { |
| @@ -765,15 +765,15 @@ int amdgpu_dm_crtc_init(struct amdgpu_display_manager *dm, · 764 unmodified lines | ||
| 765 | 765 | dm->adev->mode_info.crtcs[crtc_index] = acrtc; |
| 766 | 766 | |
| 767 | 767 | /* Don't enable DRM CRTC degamma property for |
| 768 | − * 1. Degamma is replaced by color pipeline. | |
| 769 | − * 2. DCE since it doesn't support programmable degamma anywhere. | |
| 770 | − * 3. DCN401 since pre-blending degamma LUT doesn't apply to cursor. | |
| 768 | + * 1. DCE since it doesn't support programmable degamma anywhere. | |
| 769 | + * 2. DCN401 since pre-blending degamma LUT doesn't apply to cursor. | |
| 770 | + * Note: DEGAMMA properties are created even if the primary plane has the | |
| 771 | + * COLOR_PIPELINE property. User space can use either the DEGAMMA properties | |
| 772 | + * or the COLOR_PIPELINE property. An atomic commit which attempts to enable | |
| 773 | + * both is rejected. | |
| 771 | 774 | */ |
| 772 | − if (plane->color_pipeline_property) | |
| 773 | − has_degamma = false; | |
| 774 | − else | |
| 775 | − has_degamma = dm->adev->dm.dc->caps.color.dpp.dcn_arch && | |
| 776 | − dm->adev->dm.dc->ctx->dce_version != DCN_VERSION_4_01; | |
| 775 | + has_degamma = dm->adev->dm.dc->caps.color.dpp.dcn_arch && | |
| 776 | + dm->adev->dm.dc->ctx->dce_version != DCN_VERSION_4_01; | |
| 777 | 777 | |
| 778 | 778 | drm_crtc_enable_color_mgmt(&acrtc->base, has_degamma ? MAX_COLOR_LUT_ENTRIES : 0, |
| 779 | 779 | true, MAX_COLOR_LUT_ENTRIES); |
| @@ -1256,6 +1256,14 @@ static int amdgpu_dm_plane_atomic_check(struct drm_plane *plane, · 1255 unmodified lines | ||
| 1256 | 1256 | if (ret) |
| 1257 | 1257 | return ret; |
| 1258 | 1258 | |
| 1259 | + /* Reject commits that attempt to use both COLOR_PIPELINE and CRTC DEGAMMA_LUT */ | |
| 1260 | + if (new_plane_state->color_pipeline && new_crtc_state->degamma_lut) { | |
| 1261 | + drm_dbg_atomic(plane->dev, | |
| 1262 | + "[PLANE:%d:%s] COLOR_PIPELINE and CRTC DEGAMMA_LUT cannot be enabled simultaneously\n", | |
| 1263 | + plane->base.id, plane->name); | |
| 1264 | + return -EINVAL; | |
| 1265 | + } | |
| 1266 | + | |
| 1259 | 1267 | ret = amdgpu_dm_plane_fill_dc_scaling_info(adev, new_plane_state, &scaling_info); |
| 1260 | 1268 | if (ret) |
| 1261 | 1269 | return ret; |
| @@ -170,11 +170,11 @@ struct dc_stream_state *dc_create_stream_for_sink( · 169 unmodified lines | ||
| 170 | 170 | if (sink == NULL) |
| 171 | 171 | goto fail; |
| 172 | 172 | |
| 173 | − stream = kzalloc_obj(struct dc_stream_state); | |
| 173 | + stream = kzalloc_obj(struct dc_stream_state, GFP_ATOMIC); | |
| 174 | 174 | if (stream == NULL) |
| 175 | 175 | goto fail; |
| 176 | 176 | |
| 177 | − stream->update_scratch = kzalloc((int32_t) dc_update_scratch_space_size(), GFP_KERNEL); | |
| 177 | + stream->update_scratch = kzalloc((int32_t) dc_update_scratch_space_size(), GFP_ATOMIC); | |
| 178 | 178 | if (stream->update_scratch == NULL) |
| 179 | 179 | goto fail; |
| 180 | 180 | |
| @@ -38,7 +38,11 @@ · 37 unmodified lines | ||
| 38 | 38 | DCCG_SRII(PIXEL_RATE_CNTL, OTG, 0),\ |
| 39 | 39 | DCCG_SRII(PIXEL_RATE_CNTL, OTG, 1),\ |
| 40 | 40 | SR(DISPCLK_FREQ_CHANGE_CNTL),\ |
| 41 | − SR(DC_MEM_GLOBAL_PWR_REQ_CNTL) | |
| 41 | + SR(DC_MEM_GLOBAL_PWR_REQ_CNTL),\ | |
| 42 | + SR(MICROSECOND_TIME_BASE_DIV),\ | |
| 43 | + SR(MILLISECOND_TIME_BASE_DIV),\ | |
| 44 | + SR(DCCG_GATE_DISABLE_CNTL),\ | |
| 45 | + SR(DCCG_GATE_DISABLE_CNTL2) | |
| 42 | 46 | |
| 43 | 47 | #define DCCG_REG_LIST_DCN2() \ |
| 44 | 48 | DCCG_COMMON_REG_LIST_DCN_BASE(),\ |
| @@ -96,6 +96,25 @@ static void dccg21_update_dpp_dto(struct dccg *dccg, int dpp_inst, int req_dppcl · 95 unmodified lines | ||
| 96 | 96 | dccg->pipe_dppclk_khz[dpp_inst] = req_dppclk; |
| 97 | 97 | } |
| 98 | 98 | |
| 99 | +/* | |
| 100 | + * On DCN21 S0i3 resume, BIOS programs MICROSECOND_TIME_BASE_DIV to | |
| 101 | + * 0x00120464 as a marker that golden init has already been done. | |
| 102 | + * dcn21_s0i3_golden_init_wa() reads this marker later in bios_golden_init() | |
| 103 | + * to decide whether to skip golden init. | |
| 104 | + * | |
| 105 | + * dccg2_init() unconditionally overwrites MICROSECOND_TIME_BASE_DIV to | |
| 106 | + * 0x00120264, destroying the marker before it can be read. | |
| 107 | + * | |
| 108 | + * Guard the call: if the S0i3 marker is present, skip dccg2_init() so the | |
| 109 | + * WA can function correctly. bios_golden_init() will handle init in that case. | |
| 110 | + */ | |
| 111 | +static void dccg21_init(struct dccg *dccg) | |
| 112 | +{ | |
| 113 | + if (dccg2_is_s0i3_golden_init_wa_done(dccg)) | |
| 114 | + return; | |
| 115 | + | |
| 116 | + dccg2_init(dccg); | |
| 117 | +} | |
| 99 | 118 | |
| 100 | 119 | static const struct dccg_funcs dccg21_funcs = { |
| 101 | 120 | .update_dpp_dto = dccg21_update_dpp_dto, |
| @@ -103,7 +122,7 @@ static const struct dccg_funcs dccg21_funcs = { · 1 unmodified line | ||
| 103 | 122 | .set_fifo_errdet_ovr_en = dccg2_set_fifo_errdet_ovr_en, |
| 104 | 123 | .otg_add_pixel = dccg2_otg_add_pixel, |
| 105 | 124 | .otg_drop_pixel = dccg2_otg_drop_pixel, |
| 106 | − .dccg_init = dccg2_init, | |
| 125 | + .dccg_init = dccg21_init, | |
| 107 | 126 | .refclk_setup = dccg2_refclk_setup, /* Deprecated - for backward compatibility only */ |
| 108 | 127 | .allow_clock_gating = dccg2_allow_clock_gating, |
| 109 | 128 | .enable_memory_low_power = dccg2_enable_memory_low_power, |
| @@ -34,7 +34,13 @@ · 33 unmodified lines | ||
| 34 | 34 | DCCG_SRII(DTO_PARAM, DPPCLK, 1),\ |
| 35 | 35 | DCCG_SRII(DTO_PARAM, DPPCLK, 2),\ |
| 36 | 36 | DCCG_SRII(DTO_PARAM, DPPCLK, 3),\ |
| 37 | − SR(REFCLK_CNTL) | |
| 37 | + SR(REFCLK_CNTL),\ | |
| 38 | + SR(DISPCLK_FREQ_CHANGE_CNTL),\ | |
| 39 | + SR(DC_MEM_GLOBAL_PWR_REQ_CNTL),\ | |
| 40 | + SR(MICROSECOND_TIME_BASE_DIV),\ | |
| 41 | + SR(MILLISECOND_TIME_BASE_DIV),\ | |
| 42 | + SR(DCCG_GATE_DISABLE_CNTL),\ | |
| 43 | + SR(DCCG_GATE_DISABLE_CNTL2) | |
| 38 | 44 | |
| 39 | 45 | #define DCCG_MASK_SH_LIST_DCN301(mask_sh) \ |
| 40 | 46 | DCCG_SFI(DPPCLK_DTO_CTRL, DTO_ENABLE, DPPCLK, 0, mask_sh),\ |
| @@ -64,9 +64,12 @@ · 63 unmodified lines | ||
| 64 | 64 | SR(DSCCLK1_DTO_PARAM),\ |
| 65 | 65 | SR(DSCCLK2_DTO_PARAM),\ |
| 66 | 66 | SR(DSCCLK_DTO_CTRL),\ |
| 67 | + SR(DCCG_GATE_DISABLE_CNTL),\ | |
| 67 | 68 | SR(DCCG_GATE_DISABLE_CNTL2),\ |
| 68 | 69 | SR(DCCG_GATE_DISABLE_CNTL3),\ |
| 69 | − SR(HDMISTREAMCLK0_DTO_PARAM) | |
| 70 | + SR(HDMISTREAMCLK0_DTO_PARAM),\ | |
| 71 | + SR(DC_MEM_GLOBAL_PWR_REQ_CNTL),\ | |
| 72 | + SR(MICROSECOND_TIME_BASE_DIV) | |
| 70 | 73 | |
| 71 | 74 | |
| 72 | 75 | #define DCCG_MASK_SH_LIST_DCN31(mask_sh) \ |
| @@ -70,11 +70,14 @@ · 69 unmodified lines | ||
| 70 | 70 | SR(DSCCLK2_DTO_PARAM),\ |
| 71 | 71 | SR(DSCCLK3_DTO_PARAM),\ |
| 72 | 72 | SR(DSCCLK_DTO_CTRL),\ |
| 73 | + SR(DCCG_GATE_DISABLE_CNTL),\ | |
| 73 | 74 | SR(DCCG_GATE_DISABLE_CNTL2),\ |
| 74 | 75 | SR(DCCG_GATE_DISABLE_CNTL3),\ |
| 75 | 76 | SR(HDMISTREAMCLK0_DTO_PARAM),\ |
| 76 | 77 | SR(OTG_PIXEL_RATE_DIV),\ |
| 77 | − SR(DTBCLK_P_CNTL) | |
| 78 | + SR(DTBCLK_P_CNTL),\ | |
| 79 | + SR(DC_MEM_GLOBAL_PWR_REQ_CNTL),\ | |
| 80 | + SR(MICROSECOND_TIME_BASE_DIV) | |
| 78 | 81 | |
| 79 | 82 | #define DCCG_MASK_SH_LIST_DCN314_COMMON(mask_sh) \ |
| 80 | 83 | DCCG_SFI(DPPCLK_DTO_CTRL, DTO_DB_EN, DPPCLK, 0, mask_sh),\ |
| @@ -72,7 +72,11 @@ void dcn401_initialize_min_clocks(struct dc *dc) · 71 unmodified lines | ||
| 72 | 72 | * audio corruption. Read current DISPCLK from DENTIST and request the same |
| 73 | 73 | * freq to ensure that the timing is valid and unchanged. |
| 74 | 74 | */ |
| 75 | − clocks->dispclk_khz = dc->clk_mgr->funcs->get_dispclk_from_dentist(dc->clk_mgr); | |
| 75 | + if (dc->clk_mgr->funcs->get_dispclk_from_dentist) { | |
| 76 | + clocks->dispclk_khz = dc->clk_mgr->funcs->get_dispclk_from_dentist(dc->clk_mgr); | |
| 77 | + } else { | |
| 78 | + clocks->dispclk_khz = dc->clk_mgr->boot_snapshot.dispclk * 1000; | |
| 79 | + } | |
| 76 | 80 | } |
| 77 | 81 | clocks->ref_dtbclk_khz = dc->clk_mgr->bw_params->clk_table.entries[0].dtbclk_mhz * 1000; |
| 78 | 82 | clocks->fclk_p_state_change_support = true; |
| @@ -82,6 +82,24 @@ · 81 unmodified lines | ||
| 82 | 82 | #define regMPASP_SMN_IH_SW_INT_CTRL 0x0142 |
| 83 | 83 | #define regMPASP_SMN_IH_SW_INT_CTRL_BASE_IDX 0 |
| 84 | 84 | |
| 85 | +// addressBlock: mp_SmuMpASPPub_PcruDec | |
| 86 | +// base address: 0x3800000 | |
| 87 | +#define regMPASP_PCRU1_MPASP_C2PMSG_64 0x4280 | |
| 88 | +#define regMPASP_PCRU1_MPASP_C2PMSG_64_BASE_IDX 3 | |
| 89 | +#define regMPASP_PCRU1_MPASP_C2PMSG_65 0x4281 | |
| 90 | +#define regMPASP_PCRU1_MPASP_C2PMSG_65_BASE_IDX 3 | |
| 91 | +#define regMPASP_PCRU1_MPASP_C2PMSG_66 0x4282 | |
| 92 | +#define regMPASP_PCRU1_MPASP_C2PMSG_66_BASE_IDX 3 | |
| 93 | +#define regMPASP_PCRU1_MPASP_C2PMSG_67 0x4283 | |
| 94 | +#define regMPASP_PCRU1_MPASP_C2PMSG_67_BASE_IDX 3 | |
| 95 | +#define regMPASP_PCRU1_MPASP_C2PMSG_68 0x4284 | |
| 96 | +#define regMPASP_PCRU1_MPASP_C2PMSG_68_BASE_IDX 3 | |
| 97 | +#define regMPASP_PCRU1_MPASP_C2PMSG_69 0x4285 | |
| 98 | +#define regMPASP_PCRU1_MPASP_C2PMSG_69_BASE_IDX 3 | |
| 99 | +#define regMPASP_PCRU1_MPASP_C2PMSG_70 0x4286 | |
| 100 | +#define regMPASP_PCRU1_MPASP_C2PMSG_70_BASE_IDX 3 | |
| 101 | +#define regMPASP_PCRU1_MPASP_C2PMSG_71 0x4287 | |
| 102 | +#define regMPASP_PCRU1_MPASP_C2PMSG_71_BASE_IDX 3 | |
| 85 | 103 | |
| 86 | 104 | // addressBlock: mp_SmuMp1_SmnDec |
| 87 | 105 | // base address: 0x0 |
| @@ -2034,6 +2034,7 @@ static ssize_t smu_v13_0_0_get_gpu_metrics(struct smu_context *smu, · 2033 unmodified lines | ||
| 2034 | 2034 | smu, SMU_DRIVER_TABLE_GPU_METRICS); |
| 2035 | 2035 | SmuMetricsExternal_t metrics_ext; |
| 2036 | 2036 | SmuMetrics_t *metrics = &metrics_ext.SmuMetrics; |
| 2037 | + uint32_t mp1_ver = amdgpu_ip_version(smu->adev, MP1_HWIP, 0); | |
| 2037 | 2038 | int ret = 0; |
| 2038 | 2039 | |
| 2039 | 2040 | ret = smu_cmn_get_metrics_table(smu, |
| @@ -2058,7 +2059,12 @@ static ssize_t smu_v13_0_0_get_gpu_metrics(struct smu_context *smu, · 18 unmodified lines | ||
| 2058 | 2059 | metrics->Vcn1ActivityPercentage); |
| 2059 | 2060 | |
| 2060 | 2061 | gpu_metrics->average_socket_power = metrics->AverageSocketPower; |
| 2061 | − gpu_metrics->energy_accumulator = metrics->EnergyAccumulator; | |
| 2062 | + | |
| 2063 | + if ((mp1_ver == IP_VERSION(13, 0, 0) && smu->smc_fw_version <= 0x004e1e00) || | |
| 2064 | + (mp1_ver == IP_VERSION(13, 0, 10) && smu->smc_fw_version <= 0x00500800)) | |
| 2065 | + gpu_metrics->energy_accumulator = metrics->EnergyAccumulator; | |
| 2066 | + else | |
| 2067 | + gpu_metrics->energy_accumulator = UINT_MAX; | |
| 2062 | 2068 | |
| 2063 | 2069 | if (metrics->AverageGfxActivity <= SMU_13_0_0_BUSY_THRESHOLD) |
| 2064 | 2070 | gpu_metrics->average_gfxclk_frequency = metrics->AverageGfxclkFrequencyPostDs; |
| @@ -2216,7 +2222,8 @@ static int smu_v13_0_0_restore_user_od_settings(struct smu_context *smu) · 151 unmodified lines | ||
| 2216 | 2222 | user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) | |
| 2217 | 2223 | BIT(PP_OD_FEATURE_UCLK_BIT) | |
| 2218 | 2224 | BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) | |
| 2219 | − BIT(PP_OD_FEATURE_FAN_CURVE_BIT); | |
| 2225 | + BIT(PP_OD_FEATURE_FAN_CURVE_BIT) | | |
| 2226 | + BIT(PP_OD_FEATURE_ZERO_FAN_BIT); | |
| 2220 | 2227 | res = smu_v13_0_0_upload_overdrive_table(smu, user_od_table); |
| 2221 | 2228 | user_od_table->OverDriveTable.FeatureCtrlMask = 0; |
| 2222 | 2229 | if (res == 0) |
| @@ -2065,7 +2065,8 @@ static ssize_t smu_v13_0_7_get_gpu_metrics(struct smu_context *smu, · 2064 unmodified lines | ||
| 2065 | 2065 | metrics->Vcn1ActivityPercentage); |
| 2066 | 2066 | |
| 2067 | 2067 | gpu_metrics->average_socket_power = metrics->AverageSocketPower; |
| 2068 | − gpu_metrics->energy_accumulator = metrics->EnergyAccumulator; | |
| 2068 | + gpu_metrics->energy_accumulator = smu->smc_fw_version <= 0x00521400 ? | |
| 2069 | + metrics->EnergyAccumulator : UINT_MAX; | |
| 2069 | 2070 | |
| 2070 | 2071 | if (metrics->AverageGfxActivity <= SMU_13_0_7_BUSY_THRESHOLD) |
| 2071 | 2072 | gpu_metrics->average_gfxclk_frequency = metrics->AverageGfxclkFrequencyPostDs; |
| @@ -2223,7 +2224,8 @@ static int smu_v13_0_7_restore_user_od_settings(struct smu_context *smu) · 151 unmodified lines | ||
| 2223 | 2224 | user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) | |
| 2224 | 2225 | BIT(PP_OD_FEATURE_UCLK_BIT) | |
| 2225 | 2226 | BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) | |
| 2226 | − BIT(PP_OD_FEATURE_FAN_CURVE_BIT); | |
| 2227 | + BIT(PP_OD_FEATURE_FAN_CURVE_BIT) | | |
| 2228 | + BIT(PP_OD_FEATURE_ZERO_FAN_BIT); | |
| 2227 | 2229 | res = smu_v13_0_7_upload_overdrive_table(smu, user_od_table); |
| 2228 | 2230 | user_od_table->OverDriveTable.FeatureCtrlMask = 0; |
| 2229 | 2231 | if (res == 0) |
| @@ -2311,7 +2311,8 @@ static int smu_v14_0_2_restore_user_od_settings(struct smu_context *smu) · 2310 unmodified lines | ||
| 2311 | 2311 | user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) | |
| 2312 | 2312 | BIT(PP_OD_FEATURE_UCLK_BIT) | |
| 2313 | 2313 | BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) | |
| 2314 | − BIT(PP_OD_FEATURE_FAN_CURVE_BIT); | |
| 2314 | + BIT(PP_OD_FEATURE_FAN_CURVE_BIT) | | |
| 2315 | + BIT(PP_OD_FEATURE_ZERO_FAN_BIT); | |
| 2315 | 2316 | res = smu_v14_0_2_upload_overdrive_table(smu, user_od_table); |
| 2316 | 2317 | user_od_table->OverDriveTable.FeatureCtrlMask = 0; |
| 2317 | 2318 | if (res == 0) |
| @@ -1881,6 +1881,14 @@ static int samsung_dsim_register_te_irq(struct samsung_dsim *dsi, struct device · 1880 unmodified lines | ||
| 1881 | 1881 | return 0; |
| 1882 | 1882 | } |
| 1883 | 1883 | |
| 1884 | +static void samsung_dsim_unregister_te_irq(struct samsung_dsim *dsi) | |
| 1885 | +{ | |
| 1886 | + if (dsi->te_gpio) { | |
| 1887 | + free_irq(gpiod_to_irq(dsi->te_gpio), dsi); | |
| 1888 | + gpiod_put(dsi->te_gpio); | |
| 1889 | + } | |
| 1890 | +} | |
| 1891 | + | |
| 1884 | 1892 | static int samsung_dsim_host_attach(struct mipi_dsi_host *host, |
| 1885 | 1893 | struct mipi_dsi_device *device) |
| 1886 | 1894 | { |
| @@ -1961,7 +1969,7 @@ of_find_panel_or_bridge: · 74 unmodified lines | ||
| 1961 | 1969 | if (!(device->mode_flags & MIPI_DSI_MODE_VIDEO)) { |
| 1962 | 1970 | ret = samsung_dsim_register_te_irq(dsi, &device->dev); |
| 1963 | 1971 | if (ret) |
| 1964 | − return ret; | |
| 1972 | + goto err_remove_bridge; | |
| 1965 | 1973 | } |
| 1966 | 1974 | |
| 1967 | 1975 | // The next bridge can be used by host_ops->attach |
| @@ -1982,15 +1990,12 @@ of_find_panel_or_bridge: · 14 unmodified lines | ||
| 1982 | 1990 | err_release_next_bridge: |
| 1983 | 1991 | drm_bridge_put(dsi->bridge.next_bridge); |
| 1984 | 1992 | dsi->bridge.next_bridge = NULL; |
| 1985 | − return ret; | |
| 1986 | −} | |
| 1987 | 1993 | |
| 1988 | −static void samsung_dsim_unregister_te_irq(struct samsung_dsim *dsi) | |
| 1989 | −{ | |
| 1990 | − if (dsi->te_gpio) { | |
| 1991 | − free_irq(gpiod_to_irq(dsi->te_gpio), dsi); | |
| 1992 | − gpiod_put(dsi->te_gpio); | |
| 1993 | − } | |
| 1994 | + if (!(device->mode_flags & MIPI_DSI_MODE_VIDEO)) | |
| 1995 | + samsung_dsim_unregister_te_irq(dsi); | |
| 1996 | +err_remove_bridge: | |
| 1997 | + drm_bridge_remove(&dsi->bridge); | |
| 1998 | + return ret; | |
| 1994 | 1999 | } |
| 1995 | 2000 | |
| 1996 | 2001 | static int samsung_dsim_host_detach(struct mipi_dsi_host *host, |
| @@ -2049,7 +2049,9 @@ struct dw_dp *dw_dp_bind(struct device *dev, struct drm_encoder *encoder, · 2048 unmodified lines | ||
| 2049 | 2049 | bridge->type = DRM_MODE_CONNECTOR_DisplayPort; |
| 2050 | 2050 | bridge->ycbcr_420_allowed = true; |
| 2051 | 2051 | |
| 2052 | − devm_drm_bridge_add(dev, bridge); | |
| 2052 | + ret = devm_drm_bridge_add(dev, bridge); | |
| 2053 | + if (ret) | |
| 2054 | + return ERR_PTR(ret); | |
| 2053 | 2055 | |
| 2054 | 2056 | dp->aux.dev = dev; |
| 2055 | 2057 | dp->aux.drm_dev = encoder->dev; |
| @@ -351,9 +351,9 @@ static u8 sn65dsi83_get_dsi_range(struct sn65dsi83 *ctx, · 350 unmodified lines | ||
| 351 | 351 | * DSI_CLK = mode clock * bpp / dsi_data_lanes / 2 |
| 352 | 352 | * the 2 is there because the bus is DDR. |
| 353 | 353 | */ |
| 354 | − return DIV_ROUND_UP(clamp((unsigned int)mode->clock * | |
| 355 | − mipi_dsi_pixel_format_to_bpp(ctx->dsi->format) / | |
| 356 | − ctx->dsi->lanes / 2, 40000U, 500000U), 5000U); | |
| 354 | + return clamp((unsigned int)mode->clock * | |
| 355 | + mipi_dsi_pixel_format_to_bpp(ctx->dsi->format) / | |
| 356 | + ctx->dsi->lanes / 2, 40000U, 500000U) / 5000U; | |
| 357 | 357 | } |
| 358 | 358 | |
| 359 | 359 | static u8 sn65dsi83_get_dsi_div(struct sn65dsi83 *ctx) |
| @@ -517,6 +517,7 @@ static void sn65dsi83_atomic_pre_enable(struct drm_bridge *bridge, · 157 unmodified lines | ||
| 517 | 517 | struct drm_atomic_state *state) |
| 518 | 518 | { |
| 519 | 519 | struct sn65dsi83 *ctx = bridge_to_sn65dsi83(bridge); |
| 520 | + const unsigned int dual_factor = ctx->lvds_dual_link ? 2 : 1; | |
| 520 | 521 | const struct drm_bridge_state *bridge_state; |
| 521 | 522 | const struct drm_crtc_state *crtc_state; |
| 522 | 523 | const struct drm_display_mode *mode; |
| @@ -653,18 +654,18 @@ static void sn65dsi83_atomic_pre_enable(struct drm_bridge *bridge, · 130 unmodified lines | ||
| 653 | 654 | /* 32 + 1 pixel clock to ensure proper operation */ |
| 654 | 655 | le16val = cpu_to_le16(32 + 1); |
| 655 | 656 | regmap_bulk_write(ctx->regmap, REG_VID_CHA_SYNC_DELAY_LOW, &le16val, 2); |
| 656 | − le16val = cpu_to_le16(mode->hsync_end - mode->hsync_start); | |
| 657 | + le16val = cpu_to_le16((mode->hsync_end - mode->hsync_start) / dual_factor); | |
| 657 | 658 | regmap_bulk_write(ctx->regmap, REG_VID_CHA_HSYNC_PULSE_WIDTH_LOW, |
| 658 | 659 | &le16val, 2); |
| 659 | 660 | le16val = cpu_to_le16(mode->vsync_end - mode->vsync_start); |
| 660 | 661 | regmap_bulk_write(ctx->regmap, REG_VID_CHA_VSYNC_PULSE_WIDTH_LOW, |
| 661 | 662 | &le16val, 2); |
| 662 | 663 | regmap_write(ctx->regmap, REG_VID_CHA_HORIZONTAL_BACK_PORCH, |
| 663 | − mode->htotal - mode->hsync_end); | |
| 664 | + (mode->htotal - mode->hsync_end) / dual_factor); | |
| 664 | 665 | regmap_write(ctx->regmap, REG_VID_CHA_VERTICAL_BACK_PORCH, |
| 665 | 666 | mode->vtotal - mode->vsync_end); |
| 666 | 667 | regmap_write(ctx->regmap, REG_VID_CHA_HORIZONTAL_FRONT_PORCH, |
| 667 | − mode->hsync_start - mode->hdisplay); | |
| 668 | + (mode->hsync_start - mode->hdisplay) / dual_factor); | |
| 668 | 669 | regmap_write(ctx->regmap, REG_VID_CHA_VERTICAL_FRONT_PORCH, |
| 669 | 670 | mode->vsync_start - mode->vdisplay); |
| 670 | 671 | regmap_write(ctx->regmap, REG_VID_CHA_TEST_PATTERN, 0x00); |
| @@ -1415,6 +1415,7 @@ static int ti_sn_bridge_probe(struct auxiliary_device *adev, · 1414 unmodified lines | ||
| 1415 | 1415 | { |
| 1416 | 1416 | struct ti_sn65dsi86 *pdata = dev_get_drvdata(adev->dev.parent); |
| 1417 | 1417 | struct device_node *np = pdata->dev->of_node; |
| 1418 | + const struct i2c_client *client = to_i2c_client(pdata->dev); | |
| 1418 | 1419 | int ret; |
| 1419 | 1420 | |
| 1420 | 1421 | pdata->next_bridge = devm_drm_of_get_bridge(&adev->dev, np, 1, 0); |
| @@ -1433,8 +1434,9 @@ static int ti_sn_bridge_probe(struct auxiliary_device *adev, · 12 unmodified lines | ||
| 1433 | 1434 | ? DRM_MODE_CONNECTOR_DisplayPort : DRM_MODE_CONNECTOR_eDP; |
| 1434 | 1435 | |
| 1435 | 1436 | if (pdata->bridge.type == DRM_MODE_CONNECTOR_DisplayPort) { |
| 1436 | − pdata->bridge.ops = DRM_BRIDGE_OP_EDID | DRM_BRIDGE_OP_DETECT | | |
| 1437 | − DRM_BRIDGE_OP_HPD; | |
| 1437 | + pdata->bridge.ops = DRM_BRIDGE_OP_EDID | DRM_BRIDGE_OP_DETECT; | |
| 1438 | + if (client->irq) | |
| 1439 | + pdata->bridge.ops |= DRM_BRIDGE_OP_HPD; | |
| 1438 | 1440 | /* |
| 1439 | 1441 | * If comms were already enabled they would have been enabled |
| 1440 | 1442 | * with the wrong value of HPD_DISABLE. Update it now. Comms |
| @@ -930,7 +930,8 @@ int drm_client_modeset_probe(struct drm_client_dev *client, unsigned int width, · 929 unmodified lines | ||
| 930 | 930 | mutex_unlock(&client->modeset_mutex); |
| 931 | 931 | out: |
| 932 | 932 | kfree(crtcs); |
| 933 | − modes_destroy(dev, modes, connector_count); | |
| 933 | + if (modes) | |
| 934 | + modes_destroy(dev, modes, connector_count); | |
| 934 | 935 | kfree(modes); |
| 935 | 936 | kfree(offsets); |
| 936 | 937 | kfree(enabled); |
| @@ -1338,14 +1338,14 @@ bool drm_gpusvm_range_pages_valid(struct drm_gpusvm *gpusvm, · 1337 unmodified lines | ||
| 1338 | 1338 | EXPORT_SYMBOL_GPL(drm_gpusvm_range_pages_valid); |
| 1339 | 1339 | |
| 1340 | 1340 | /** |
| 1341 | − * drm_gpusvm_range_pages_valid_unlocked() - GPU SVM range pages valid unlocked | |
| 1341 | + * drm_gpusvm_pages_valid_unlocked() - GPU SVM pages valid unlocked | |
| 1342 | 1342 | * @gpusvm: Pointer to the GPU SVM structure |
| 1343 | − * @range: Pointer to the GPU SVM range structure | |
| 1343 | + * @svm_pages: Pointer to the GPU SVM pages structure | |
| 1344 | 1344 | * |
| 1345 | − * This function determines if a GPU SVM range pages are valid. Expected be | |
| 1346 | − * called without holding gpusvm->notifier_lock. | |
| 1345 | + * This function determines if a GPU SVM pages are valid. Expected be called | |
| 1346 | + * without holding gpusvm->notifier_lock. | |
| 1347 | 1347 | * |
| 1348 | − * Return: True if GPU SVM range has valid pages, False otherwise | |
| 1348 | + * Return: True if GPU SVM pages are valid, False otherwise | |
| 1349 | 1349 | */ |
| 1350 | 1350 | static bool drm_gpusvm_pages_valid_unlocked(struct drm_gpusvm *gpusvm, |
| 1351 | 1351 | struct drm_gpusvm_pages *svm_pages) |
| @@ -480,18 +480,8 @@ int drm_pagemap_migrate_to_devmem(struct drm_pagemap_devmem *devmem_allocation, · 479 unmodified lines | ||
| 480 | 480 | .start = start, |
| 481 | 481 | .end = end, |
| 482 | 482 | .pgmap_owner = pagemap->owner, |
| 483 | − /* | |
| 484 | − * FIXME: MIGRATE_VMA_SELECT_DEVICE_PRIVATE intermittently | |
| 485 | − * causes 'xe_exec_system_allocator --r *race*no*' to trigger aa | |
| 486 | − * engine reset and a hard hang due to getting stuck on a folio | |
| 487 | − * lock. This should work and needs to be root-caused. The only | |
| 488 | − * downside of not selecting MIGRATE_VMA_SELECT_DEVICE_PRIVATE | |
| 489 | − * is that device-to-device migrations won’t work; instead, | |
| 490 | − * memory will bounce through system memory. This path should be | |
| 491 | − * rare and only occur when the madvise attributes of memory are | |
| 492 | − * changed or atomics are being used. | |
| 493 | − */ | |
| 494 | − .flags = MIGRATE_VMA_SELECT_SYSTEM | MIGRATE_VMA_SELECT_DEVICE_COHERENT, | |
| 483 | + .flags = MIGRATE_VMA_SELECT_SYSTEM | MIGRATE_VMA_SELECT_DEVICE_COHERENT | | |
| 484 | + MIGRATE_VMA_SELECT_DEVICE_PRIVATE, | |
| 495 | 485 | }; |
| 496 | 486 | unsigned long i, npages = npages_in_range(start, end); |
| 497 | 487 | unsigned long own_pages = 0, migrated_pages = 0; |
| @@ -339,7 +339,9 @@ static int gud_stats_debugfs(struct seq_file *m, void *data) · 338 unmodified lines | ||
| 339 | 339 | } |
| 340 | 340 | |
| 341 | 341 | static const struct drm_crtc_helper_funcs gud_crtc_helper_funcs = { |
| 342 | − .atomic_check = drm_crtc_helper_atomic_check | |
| 342 | + .atomic_check = drm_crtc_helper_atomic_check, | |
| 343 | + .atomic_enable = gud_crtc_atomic_enable, | |
| 344 | + .atomic_disable = gud_crtc_atomic_disable, | |
| 343 | 345 | }; |
| 344 | 346 | |
| 345 | 347 | static const struct drm_crtc_funcs gud_crtc_funcs = { |
| @@ -364,6 +366,10 @@ static const struct drm_plane_funcs gud_plane_funcs = { · 18 unmodified lines | ||
| 364 | 366 | DRM_GEM_SHADOW_PLANE_FUNCS, |
| 365 | 367 | }; |
| 366 | 368 | |
| 369 | +static const struct drm_mode_config_helper_funcs gud_mode_config_helpers = { | |
| 370 | + .atomic_commit_tail = drm_atomic_helper_commit_tail_rpm, | |
| 371 | +}; | |
| 372 | + | |
| 367 | 373 | static const struct drm_mode_config_funcs gud_mode_config_funcs = { |
| 368 | 374 | .fb_create = drm_gem_fb_create_with_dirty, |
| 369 | 375 | .atomic_check = drm_atomic_helper_check, |
| @@ -499,6 +505,7 @@ static int gud_probe(struct usb_interface *intf, const struct usb_device_id *id) · 129 unmodified lines | ||
| 499 | 505 | drm->mode_config.min_height = le32_to_cpu(desc.min_height); |
| 500 | 506 | drm->mode_config.max_height = le32_to_cpu(desc.max_height); |
| 501 | 507 | drm->mode_config.funcs = &gud_mode_config_funcs; |
| 508 | + drm->mode_config.helper_private = &gud_mode_config_helpers; | |
| 502 | 509 | |
| 503 | 510 | /* Format init */ |
| 504 | 511 | formats_dev = devm_kmalloc(dev, GUD_FORMATS_MAX_NUM, GFP_KERNEL); |
| @@ -62,6 +62,10 @@ int gud_usb_set_u8(struct gud_device *gdrm, u8 request, u8 val); · 61 unmodified lines | ||
| 62 | 62 | |
| 63 | 63 | void gud_clear_damage(struct gud_device *gdrm); |
| 64 | 64 | void gud_flush_work(struct work_struct *work); |
| 65 | +void gud_crtc_atomic_enable(struct drm_crtc *crtc, | |
| 66 | + struct drm_atomic_state *state); | |
| 67 | +void gud_crtc_atomic_disable(struct drm_crtc *crtc, | |
| 68 | + struct drm_atomic_state *state); | |
| 65 | 69 | int gud_plane_atomic_check(struct drm_plane *plane, |
| 66 | 70 | struct drm_atomic_state *state); |
| 67 | 71 | void gud_plane_atomic_update(struct drm_plane *plane, |
| @@ -580,6 +580,39 @@ out: · 579 unmodified lines | ||
| 580 | 580 | return ret; |
| 581 | 581 | } |
| 582 | 582 | |
| 583 | +void gud_crtc_atomic_enable(struct drm_crtc *crtc, | |
| 584 | + struct drm_atomic_state *state) | |
| 585 | +{ | |
| 586 | + struct drm_device *drm = crtc->dev; | |
| 587 | + struct gud_device *gdrm = to_gud_device(drm); | |
| 588 | + int idx; | |
| 589 | + | |
| 590 | + if (!drm_dev_enter(drm, &idx)) | |
| 591 | + return; | |
| 592 | + | |
| 593 | + gud_usb_set_u8(gdrm, GUD_REQ_SET_CONTROLLER_ENABLE, 1); | |
| 594 | + gud_usb_set(gdrm, GUD_REQ_SET_STATE_COMMIT, 0, NULL, 0); | |
| 595 | + gud_usb_set_u8(gdrm, GUD_REQ_SET_DISPLAY_ENABLE, 1); | |
| 596 | + | |
| 597 | + drm_dev_exit(idx); | |
| 598 | +} | |
| 599 | + | |
| 600 | +void gud_crtc_atomic_disable(struct drm_crtc *crtc, | |
| 601 | + struct drm_atomic_state *state) | |
| 602 | +{ | |
| 603 | + struct drm_device *drm = crtc->dev; | |
| 604 | + struct gud_device *gdrm = to_gud_device(drm); | |
| 605 | + int idx; | |
| 606 | + | |
| 607 | + if (!drm_dev_enter(drm, &idx)) | |
| 608 | + return; | |
| 609 | + | |
| 610 | + gud_usb_set_u8(gdrm, GUD_REQ_SET_DISPLAY_ENABLE, 0); | |
| 611 | + gud_usb_set_u8(gdrm, GUD_REQ_SET_CONTROLLER_ENABLE, 0); | |
| 612 | + | |
| 613 | + drm_dev_exit(idx); | |
| 614 | +} | |
| 615 | + | |
| 583 | 616 | void gud_plane_atomic_update(struct drm_plane *plane, |
| 584 | 617 | struct drm_atomic_state *atomic_state) |
| 585 | 618 | { |
| @@ -607,24 +640,12 @@ void gud_plane_atomic_update(struct drm_plane *plane, · 21 unmodified lines | ||
| 607 | 640 | mutex_unlock(&gdrm->damage_lock); |
| 608 | 641 | } |
| 609 | 642 | |
| 610 | − if (!drm_dev_enter(drm, &idx)) | |
| 643 | + if (!crtc || !drm_dev_enter(drm, &idx)) | |
| 611 | 644 | return; |
| 612 | 645 | |
| 613 | − if (!old_state->fb) | |
| 614 | − gud_usb_set_u8(gdrm, GUD_REQ_SET_CONTROLLER_ENABLE, 1); | |
| 615 | − | |
| 616 | − if (fb && (crtc->state->mode_changed || crtc->state->connectors_changed)) | |
| 617 | − gud_usb_set(gdrm, GUD_REQ_SET_STATE_COMMIT, 0, NULL, 0); | |
| 618 | − | |
| 619 | − if (crtc->state->active_changed) | |
| 620 | − gud_usb_set_u8(gdrm, GUD_REQ_SET_DISPLAY_ENABLE, crtc->state->active); | |
| 621 | − | |
| 622 | − if (!fb) | |
| 623 | − goto ctrl_disable; | |
| 624 | − | |
| 625 | 646 | ret = drm_gem_fb_begin_cpu_access(fb, DMA_FROM_DEVICE); |
| 626 | 647 | if (ret) |
| 627 | − goto ctrl_disable; | |
| 648 | + goto out; | |
| 628 | 649 | |
| 629 | 650 | drm_atomic_helper_damage_iter_init(&iter, old_state, new_state); |
| 630 | 651 | drm_atomic_for_each_plane_damage(&iter, &damage) |
| @@ -632,9 +653,6 @@ void gud_plane_atomic_update(struct drm_plane *plane, · 1 unmodified line | ||
| 632 | 653 | |
| 633 | 654 | drm_gem_fb_end_cpu_access(fb, DMA_FROM_DEVICE); |
| 634 | 655 | |
| 635 | −ctrl_disable: | |
| 636 | − if (!crtc->state->enable) | |
| 637 | − gud_usb_set_u8(gdrm, GUD_REQ_SET_CONTROLLER_ENABLE, 0); | |
| 638 | − | |
| 656 | +out: | |
| 639 | 657 | drm_dev_exit(idx); |
| 640 | 658 | } |
| @@ -43,12 +43,6 @@ bool intel_alpm_is_alpm_aux_less(struct intel_dp *intel_dp, · 42 unmodified lines | ||
| 43 | 43 | |
| 44 | 44 | void intel_alpm_init(struct intel_dp *intel_dp) |
| 45 | 45 | { |
| 46 | − u8 dpcd; | |
| 47 | − | |
| 48 | − if (drm_dp_dpcd_readb(&intel_dp->aux, DP_RECEIVER_ALPM_CAP, &dpcd) < 0) | |
| 49 | − return; | |
| 50 | − | |
| 51 | − intel_dp->alpm_dpcd = dpcd; | |
| 52 | 46 | mutex_init(&intel_dp->alpm.lock); |
| 53 | 47 | } |
| 54 | 48 | |
| @@ -562,12 +556,7 @@ void intel_alpm_disable(struct intel_dp *intel_dp) · 507 unmodified lines | ||
| 562 | 556 | mutex_lock(&intel_dp->alpm.lock); |
| 563 | 557 | |
| 564 | 558 | intel_de_rmw(display, ALPM_CTL(display, cpu_transcoder), |
| 565 | − ALPM_CTL_ALPM_ENABLE | ALPM_CTL_LOBF_ENABLE | | |
| 566 | − ALPM_CTL_ALPM_AUX_LESS_ENABLE, 0); | |
| 567 | − | |
| 568 | − intel_de_rmw(display, | |
| 569 | − PORT_ALPM_CTL(cpu_transcoder), | |
| 570 | − PORT_ALPM_CTL_ALPM_AUX_LESS_ENABLE, 0); | |
| 559 | + ALPM_CTL_ALPM_ENABLE | ALPM_CTL_LOBF_ENABLE, 0); | |
| 571 | 560 | |
| 572 | 561 | drm_dbg_kms(display->drm, "Disabling ALPM\n"); |
| 573 | 562 | mutex_unlock(&intel_dp->alpm.lock); |
| @@ -1614,7 +1614,6 @@ static void hsw_configure_cpu_transcoder(const struct intel_crtc_state *crtc_sta · 1613 unmodified lines | ||
| 1614 | 1614 | } |
| 1615 | 1615 | |
| 1616 | 1616 | intel_set_transcoder_timings(crtc_state); |
| 1617 | − intel_vrr_set_transcoder_timings(crtc_state); | |
| 1618 | 1617 | |
| 1619 | 1618 | if (cpu_transcoder != TRANSCODER_EDP) |
| 1620 | 1619 | intel_de_write(display, TRANS_MULT(display, cpu_transcoder), |
| @@ -4577,6 +4577,7 @@ static bool · 4576 unmodified lines | ||
| 4577 | 4577 | intel_edp_init_dpcd(struct intel_dp *intel_dp, struct intel_connector *connector) |
| 4578 | 4578 | { |
| 4579 | 4579 | struct intel_display *display = to_intel_display(intel_dp); |
| 4580 | + int ret; | |
| 4580 | 4581 | |
| 4581 | 4582 | /* this function is meant to be called only once */ |
| 4582 | 4583 | drm_WARN_ON(display->drm, intel_dp->dpcd[DP_DPCD_REV] != 0); |
| @@ -4616,6 +4617,12 @@ intel_edp_init_dpcd(struct intel_dp *intel_dp, struct intel_connector *connector · 33 unmodified lines | ||
| 4616 | 4617 | */ |
| 4617 | 4618 | intel_dp_init_source_oui(intel_dp); |
| 4618 | 4619 | |
| 4620 | + /* Read the ALPM DPCD caps */ | |
| 4621 | + ret = drm_dp_dpcd_read_byte(&intel_dp->aux, DP_RECEIVER_ALPM_CAP, | |
| 4622 | + &intel_dp->alpm_dpcd); | |
| 4623 | + if (ret < 0) | |
| 4624 | + return false; | |
| 4625 | + | |
| 4619 | 4626 | /* |
| 4620 | 4627 | * This has to be called after intel_dp->edp_dpcd is filled, PSR checks |
| 4621 | 4628 | * for SET_POWER_CAPABLE bit in intel_dp->edp_dpcd[1] |
| @@ -1307,9 +1307,14 @@ static bool psr2_granularity_check(struct intel_crtc_state *crtc_state, · 1306 unmodified lines | ||
| 1307 | 1307 | u16 sink_y_granularity = crtc_state->has_panel_replay ? |
| 1308 | 1308 | connector->dp.panel_replay_caps.su_y_granularity : |
| 1309 | 1309 | connector->dp.psr_caps.su_y_granularity; |
| 1310 | − u16 sink_w_granularity = crtc_state->has_panel_replay ? | |
| 1311 | − connector->dp.panel_replay_caps.su_w_granularity : | |
| 1312 | − connector->dp.psr_caps.su_w_granularity; | |
| 1310 | + u16 sink_w_granularity; | |
| 1311 | + | |
| 1312 | + if (crtc_state->has_panel_replay) | |
| 1313 | + sink_w_granularity = connector->dp.panel_replay_caps.su_w_granularity == | |
| 1314 | + DP_PANEL_REPLAY_FULL_LINE_GRANULARITY ? | |
| 1315 | + crtc_hdisplay : connector->dp.panel_replay_caps.su_w_granularity; | |
| 1316 | + else | |
| 1317 | + sink_w_granularity = connector->dp.psr_caps.su_w_granularity; | |
| 1313 | 1318 | |
| 1314 | 1319 | /* PSR2 HW only send full lines so we only need to validate the width */ |
| 1315 | 1320 | if (crtc_hdisplay % sink_w_granularity) |
| @@ -2614,6 +2619,12 @@ void intel_psr2_program_trans_man_trk_ctl(struct intel_dsb *dsb, · 1298 unmodified lines | ||
| 2614 | 2619 | |
| 2615 | 2620 | intel_de_write_dsb(display, dsb, PIPE_SRCSZ_ERLY_TPT(crtc->pipe), |
| 2616 | 2621 | crtc_state->pipe_srcsz_early_tpt); |
| 2622 | + | |
| 2623 | + if (!crtc_state->dsc.compression_enable) | |
| 2624 | + return; | |
| 2625 | + | |
| 2626 | + intel_dsc_su_et_parameters_configure(dsb, encoder, crtc_state, | |
| 2627 | + drm_rect_height(&crtc_state->psr2_su_area)); | |
| 2617 | 2628 | } |
| 2618 | 2629 | |
| 2619 | 2630 | static void psr2_man_trk_ctl_calc(struct intel_crtc_state *crtc_state, |
| @@ -2684,11 +2695,12 @@ static void clip_area_update(struct drm_rect *overlap_damage_area, · 64 unmodified lines | ||
| 2684 | 2695 | overlap_damage_area->y2 = damage_area->y2; |
| 2685 | 2696 | } |
| 2686 | 2697 | |
| 2687 | −static void intel_psr2_sel_fetch_pipe_alignment(struct intel_crtc_state *crtc_state) | |
| 2698 | +static bool intel_psr2_sel_fetch_pipe_alignment(struct intel_crtc_state *crtc_state) | |
| 2688 | 2699 | { |
| 2689 | 2700 | struct intel_display *display = to_intel_display(crtc_state); |
| 2690 | 2701 | const struct drm_dsc_config *vdsc_cfg = &crtc_state->dsc.config; |
| 2691 | 2702 | u16 y_alignment; |
| 2703 | + bool su_area_changed = false; | |
| 2692 | 2704 | |
| 2693 | 2705 | /* ADLP aligns the SU region to vdsc slice height in case dsc is enabled */ |
| 2694 | 2706 | if (crtc_state->dsc.compression_enable && |
| @@ -2697,10 +2709,18 @@ static void intel_psr2_sel_fetch_pipe_alignment(struct intel_crtc_state *crtc_st · 2 unmodified lines | ||
| 2697 | 2709 | else |
| 2698 | 2710 | y_alignment = crtc_state->su_y_granularity; |
| 2699 | 2711 | |
| 2700 | − crtc_state->psr2_su_area.y1 -= crtc_state->psr2_su_area.y1 % y_alignment; | |
| 2701 | − if (crtc_state->psr2_su_area.y2 % y_alignment) | |
| 2712 | + if (crtc_state->psr2_su_area.y1 % y_alignment) { | |
| 2713 | + crtc_state->psr2_su_area.y1 -= crtc_state->psr2_su_area.y1 % y_alignment; | |
| 2714 | + su_area_changed = true; | |
| 2715 | + } | |
| 2716 | + | |
| 2717 | + if (crtc_state->psr2_su_area.y2 % y_alignment) { | |
| 2702 | 2718 | crtc_state->psr2_su_area.y2 = ((crtc_state->psr2_su_area.y2 / |
| 2703 | 2719 | y_alignment) + 1) * y_alignment; |
| 2720 | + su_area_changed = true; | |
| 2721 | + } | |
| 2722 | + | |
| 2723 | + return su_area_changed; | |
| 2704 | 2724 | } |
| 2705 | 2725 | |
| 2706 | 2726 | /* |
| @@ -2834,7 +2854,7 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state, · 127 unmodified lines | ||
| 2834 | 2854 | struct intel_crtc_state *crtc_state = intel_atomic_get_new_crtc_state(state, crtc); |
| 2835 | 2855 | struct intel_plane_state *new_plane_state, *old_plane_state; |
| 2836 | 2856 | struct intel_plane *plane; |
| 2837 | − bool full_update = false, cursor_in_su_area = false; | |
| 2857 | + bool full_update = false, su_area_changed; | |
| 2838 | 2858 | int i, ret; |
| 2839 | 2859 | |
| 2840 | 2860 | if (!crtc_state->enable_psr2_sel_fetch) |
| @@ -2941,15 +2961,32 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state, · 100 unmodified lines | ||
| 2941 | 2961 | if (ret) |
| 2942 | 2962 | return ret; |
| 2943 | 2963 | |
| 2944 | − /* | |
| 2945 | − * Adjust su area to cover cursor fully as necessary (early | |
| 2946 | − * transport). This needs to be done after | |
| 2947 | − * drm_atomic_add_affected_planes to ensure visible cursor is added into | |
| 2948 | − * affected planes even when cursor is not updated by itself. | |
| 2949 | − */ | |
| 2950 | − intel_psr2_sel_fetch_et_alignment(state, crtc, &cursor_in_su_area); | |
| 2964 | + do { | |
| 2965 | + bool cursor_in_su_area; | |
| 2966 | + | |
| 2967 | + /* | |
| 2968 | + * Adjust su area to cover cursor fully as necessary | |
| 2969 | + * (early transport). This needs to be done after | |
| 2970 | + * drm_atomic_add_affected_planes to ensure visible | |
| 2971 | + * cursor is added into affected planes even when | |
| 2972 | + * cursor is not updated by itself. | |
| 2973 | + */ | |
| 2974 | + intel_psr2_sel_fetch_et_alignment(state, crtc, &cursor_in_su_area); | |
| 2951 | 2975 | |
| 2952 | − intel_psr2_sel_fetch_pipe_alignment(crtc_state); | |
| 2976 | + su_area_changed = intel_psr2_sel_fetch_pipe_alignment(crtc_state); | |
| 2977 | + | |
| 2978 | + /* | |
| 2979 | + * If the cursor was outside the SU area before | |
| 2980 | + * alignment, the alignment step (which only expands | |
| 2981 | + * SU) may pull the cursor partially inside, so we | |
| 2982 | + * must run ET alignment again to fully cover it. But | |
| 2983 | + * if the cursor was already fully inside before | |
| 2984 | + * alignment, expanding the SU area won't change that, | |
| 2985 | + * so no further work is needed. | |
| 2986 | + */ | |
| 2987 | + if (cursor_in_su_area) | |
| 2988 | + break; | |
| 2989 | + } while (su_area_changed); | |
| 2953 | 2990 | |
| 2954 | 2991 | /* |
| 2955 | 2992 | * Now that we have the pipe damaged area check if it intersect with |
| @@ -3009,6 +3046,10 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state, · 53 unmodified lines | ||
| 3009 | 3046 | } |
| 3010 | 3047 | |
| 3011 | 3048 | skip_sel_fetch_set_loop: |
| 3049 | + if (full_update) | |
| 3050 | + clip_area_update(&crtc_state->psr2_su_area, &crtc_state->pipe_src, | |
| 3051 | + &crtc_state->pipe_src); | |
| 3052 | + | |
| 3012 | 3053 | psr2_man_trk_ctl_calc(crtc_state, full_update); |
| 3013 | 3054 | crtc_state->pipe_srcsz_early_tpt = |
| 3014 | 3055 | psr2_pipe_srcsz_early_tpt_calc(crtc_state, full_update); |
| @@ -767,6 +767,29 @@ void intel_dsc_dp_pps_write(struct intel_encoder *encoder, · 766 unmodified lines | ||
| 767 | 767 | sizeof(dp_dsc_pps_sdp)); |
| 768 | 768 | } |
| 769 | 769 | |
| 770 | +void intel_dsc_su_et_parameters_configure(struct intel_dsb *dsb, struct intel_encoder *encoder, | |
| 771 | + const struct intel_crtc_state *crtc_state, int su_lines) | |
| 772 | +{ | |
| 773 | + struct intel_display *display = to_intel_display(crtc_state); | |
| 774 | + struct intel_crtc *crtc = to_intel_crtc(crtc_state->uapi.crtc); | |
| 775 | + const struct drm_dsc_config *vdsc_cfg = &crtc_state->dsc.config; | |
| 776 | + enum pipe pipe = crtc->pipe; | |
| 777 | + int vdsc_instances_per_pipe = intel_dsc_get_vdsc_per_pipe(crtc_state); | |
| 778 | + int slice_row_per_frame = su_lines / vdsc_cfg->slice_height; | |
| 779 | + u32 val; | |
| 780 | + | |
| 781 | + drm_WARN_ON_ONCE(display->drm, su_lines % vdsc_cfg->slice_height); | |
| 782 | + drm_WARN_ON_ONCE(display->drm, vdsc_instances_per_pipe > 2); | |
| 783 | + | |
| 784 | + val = DSC_SUPS0_SU_SLICE_ROW_PER_FRAME(slice_row_per_frame); | |
| 785 | + val |= DSC_SUPS0_SU_PIC_HEIGHT(su_lines); | |
| 786 | + | |
| 787 | + intel_de_write_dsb(display, dsb, LNL_DSC0_SU_PARAMETER_SET_0(pipe), val); | |
| 788 | + | |
| 789 | + if (vdsc_instances_per_pipe == 2) | |
| 790 | + intel_de_write_dsb(display, dsb, LNL_DSC1_SU_PARAMETER_SET_0(pipe), val); | |
| 791 | +} | |
| 792 | + | |
| 770 | 793 | static i915_reg_t dss_ctl1_reg(struct intel_crtc *crtc, enum transcoder cpu_transcoder) |
| 771 | 794 | { |
| 772 | 795 | return is_pipe_dsc(crtc, cpu_transcoder) ? |
| @@ -13,6 +13,7 @@ struct drm_printer; · 12 unmodified lines | ||
| 13 | 13 | enum transcoder; |
| 14 | 14 | struct intel_crtc; |
| 15 | 15 | struct intel_crtc_state; |
| 16 | +struct intel_dsb; | |
| 16 | 17 | struct intel_encoder; |
| 17 | 18 | |
| 18 | 19 | bool intel_dsc_source_support(const struct intel_crtc_state *crtc_state); |
| @@ -31,6 +32,8 @@ void intel_dsc_dsi_pps_write(struct intel_encoder *encoder, · 12 unmodified lines | ||
| 31 | 32 | const struct intel_crtc_state *crtc_state); |
| 32 | 33 | void intel_dsc_dp_pps_write(struct intel_encoder *encoder, |
| 33 | 34 | const struct intel_crtc_state *crtc_state); |
| 35 | +void intel_dsc_su_et_parameters_configure(struct intel_dsb *dsb, struct intel_encoder *encoder, | |
| 36 | + const struct intel_crtc_state *crtc_state, int su_lines); | |
| 34 | 37 | void intel_vdsc_state_dump(struct drm_printer *p, int indent, |
| 35 | 38 | const struct intel_crtc_state *crtc_state); |
| 36 | 39 | int intel_vdsc_min_cdclk(const struct intel_crtc_state *crtc_state); |
| @@ -196,6 +196,18 @@ · 195 unmodified lines | ||
| 196 | 196 | #define DSC_PPS18_NSL_BPG_OFFSET(offset) REG_FIELD_PREP(DSC_PPS18_NSL_BPG_OFFSET_MASK, offset) |
| 197 | 197 | #define DSC_PPS18_SL_OFFSET_ADJ(offset) REG_FIELD_PREP(DSC_PPS18_SL_OFFSET_ADJ_MASK, offset) |
| 198 | 198 | |
| 199 | +#define _LNL_DSC0_SU_PARAMETER_SET_0_PA 0x78064 | |
| 200 | +#define _LNL_DSC1_SU_PARAMETER_SET_0_PA 0x78164 | |
| 201 | +#define _LNL_DSC0_SU_PARAMETER_SET_0_PB 0x78264 | |
| 202 | +#define _LNL_DSC1_SU_PARAMETER_SET_0_PB 0x78364 | |
| 203 | +#define LNL_DSC0_SU_PARAMETER_SET_0(pipe) _MMIO_PIPE((pipe), _LNL_DSC0_SU_PARAMETER_SET_0_PA, _LNL_DSC0_SU_PARAMETER_SET_0_PB) | |
| 204 | +#define LNL_DSC1_SU_PARAMETER_SET_0(pipe) _MMIO_PIPE((pipe), _LNL_DSC1_SU_PARAMETER_SET_0_PA, _LNL_DSC1_SU_PARAMETER_SET_0_PB) | |
| 205 | + | |
| 206 | +#define DSC_SUPS0_SU_SLICE_ROW_PER_FRAME_MASK REG_GENMASK(31, 20) | |
| 207 | +#define DSC_SUPS0_SU_SLICE_ROW_PER_FRAME(rows) REG_FIELD_PREP(DSC_SUPS0_SU_SLICE_ROW_PER_FRAME_MASK, (rows)) | |
| 208 | +#define DSC_SUPS0_SU_PIC_HEIGHT_MASK REG_GENMASK(15, 0) | |
| 209 | +#define DSC_SUPS0_SU_PIC_HEIGHT(h) REG_FIELD_PREP(DSC_SUPS0_SU_PIC_HEIGHT_MASK, (h)) | |
| 210 | + | |
| 199 | 211 | /* Icelake Rate Control Buffer Threshold Registers */ |
| 200 | 212 | #define DSCA_RC_BUF_THRESH_0 _MMIO(0x6B230) |
| 201 | 213 | #define DSCA_RC_BUF_THRESH_0_UDW _MMIO(0x6B230 + 4) |
| @@ -597,6 +597,18 @@ void intel_vrr_set_transcoder_timings(const struct intel_crtc_state *crtc_state) · 596 unmodified lines | ||
| 597 | 597 | if (!HAS_VRR(display)) |
| 598 | 598 | return; |
| 599 | 599 | |
| 600 | + /* | |
| 601 | + * Bspec says: | |
| 602 | + * "(note: VRR needs to be programmed after | |
| 603 | + * TRANS_DDI_FUNC_CTL and before TRANS_CONF)." | |
| 604 | + * | |
| 605 | + * In practice it turns out that ICL can hang if | |
| 606 | + * TRANS_VRR_VMAX/FLIPLINE are written before | |
| 607 | + * enabling TRANS_DDI_FUNC_CTL. | |
| 608 | + */ | |
| 609 | + drm_WARN_ON(display->drm, | |
| 610 | + !(intel_de_read(display, TRANS_DDI_FUNC_CTL(display, cpu_transcoder)) & TRANS_DDI_FUNC_ENABLE)); | |
| 611 | + | |
| 600 | 612 | /* |
| 601 | 613 | * This bit seems to have two meanings depending on the platform: |
| 602 | 614 | * TGL: generate VRR "safe window" for DSB vblank waits |
| @@ -939,6 +951,8 @@ void intel_vrr_transcoder_enable(const struct intel_crtc_state *crtc_state) · 336 unmodified lines | ||
| 939 | 951 | { |
| 940 | 952 | struct intel_display *display = to_intel_display(crtc_state); |
| 941 | 953 | |
| 954 | + intel_vrr_set_transcoder_timings(crtc_state); | |
| 955 | + | |
| 942 | 956 | if (!intel_vrr_possible(crtc_state)) |
| 943 | 957 | return; |
| 944 | 958 | |
| @@ -153,8 +153,12 @@ int shmem_sg_alloc_table(struct drm_i915_private *i915, struct sg_table *st, · 152 unmodified lines | ||
| 153 | 153 | } |
| 154 | 154 | } while (1); |
| 155 | 155 | |
| 156 | − nr_pages = min_t(unsigned long, | |
| 157 | − folio_nr_pages(folio), page_count - i); | |
| 156 | + nr_pages = min_array(((unsigned long[]) { | |
| 157 | + folio_nr_pages(folio), | |
| 158 | + page_count - i, | |
| 159 | + max_segment / PAGE_SIZE, | |
| 160 | + }), 3); | |
| 161 | + | |
| 158 | 162 | if (!i || |
| 159 | 163 | sg->length >= max_segment || |
| 160 | 164 | folio_pfn(folio) != next_pfn) { |
| @@ -164,7 +168,9 @@ int shmem_sg_alloc_table(struct drm_i915_private *i915, struct sg_table *st, · 3 unmodified lines | ||
| 164 | 168 | st->nents++; |
| 165 | 169 | sg_set_folio(sg, folio, nr_pages * PAGE_SIZE, 0); |
| 166 | 170 | } else { |
| 167 | − /* XXX: could overflow? */ | |
| 171 | + nr_pages = min_t(unsigned long, nr_pages, | |
| 172 | + (max_segment - sg->length) / PAGE_SIZE); | |
| 173 | + | |
| 168 | 174 | sg->length += nr_pages * PAGE_SIZE; |
| 169 | 175 | } |
| 170 | 176 | next_pfn = folio_pfn(folio) + nr_pages; |
| @@ -256,7 +256,9 @@ static int imx_pd_probe(struct platform_device *pdev) · 255 unmodified lines | ||
| 256 | 256 | |
| 257 | 257 | platform_set_drvdata(pdev, imxpd); |
| 258 | 258 | |
| 259 | − devm_drm_bridge_add(dev, &imxpd->bridge); | |
| 259 | + ret = devm_drm_bridge_add(dev, &imxpd->bridge); | |
| 260 | + if (ret) | |
| 261 | + return ret; | |
| 260 | 262 | |
| 261 | 263 | return component_add(dev, &imx_pd_ops); |
| 262 | 264 | } |
| @@ -92,7 +92,6 @@ static int logicvc_drm_config_parse(struct logicvc_drm *logicvc) · 91 unmodified lines | ||
| 92 | 92 | struct device *dev = drm_dev->dev; |
| 93 | 93 | struct device_node *of_node = dev->of_node; |
| 94 | 94 | struct logicvc_drm_config *config = &logicvc->config; |
| 95 | − struct device_node *layers_node; | |
| 96 | 95 | int ret; |
| 97 | 96 | |
| 98 | 97 | logicvc_of_property_parse_bool(of_node, LOGICVC_OF_PROPERTY_DITHERING, |
| @@ -128,7 +127,8 @@ static int logicvc_drm_config_parse(struct logicvc_drm *logicvc) · 29 unmodified lines | ||
| 128 | 127 | if (ret) |
| 129 | 128 | return ret; |
| 130 | 129 | |
| 131 | − layers_node = of_get_child_by_name(of_node, "layers"); | |
| 130 | + struct device_node *layers_node __free(device_node) = | |
| 131 | + of_get_child_by_name(of_node, "layers"); | |
| 132 | 132 | if (!layers_node) { |
| 133 | 133 | drm_err(drm_dev, "Missing non-optional layers node\n"); |
| 134 | 134 | return -EINVAL; |
| @@ -78,7 +78,7 @@ static void a2xx_gpummu_destroy(struct msm_mmu *mmu) · 77 unmodified lines | ||
| 78 | 78 | { |
| 79 | 79 | struct a2xx_gpummu *gpummu = to_a2xx_gpummu(mmu); |
| 80 | 80 | |
| 81 | − dma_free_attrs(mmu->dev, TABLE_SIZE, gpummu->table, gpummu->pt_base, | |
| 81 | + dma_free_attrs(mmu->dev, TABLE_SIZE + 32, gpummu->table, gpummu->pt_base, | |
| 82 | 82 | DMA_ATTR_FORCE_CONTIGUOUS); |
| 83 | 83 | |
| 84 | 84 | kfree(gpummu); |
| @@ -1759,7 +1759,7 @@ static const u32 x285_protect_regs[] = { · 1758 unmodified lines | ||
| 1759 | 1759 | A6XX_PROTECT_NORDWR(0x27c06, 0x0000), |
| 1760 | 1760 | }; |
| 1761 | 1761 | |
| 1762 | −DECLARE_ADRENO_PROTECT(x285_protect, 64); | |
| 1762 | +DECLARE_ADRENO_PROTECT(x285_protect, 15); | |
| 1763 | 1763 | |
| 1764 | 1764 | static const struct adreno_reglist_pipe a840_nonctxt_regs[] = { |
| 1765 | 1765 | { REG_A8XX_CP_SMMU_STREAM_ID_LPAC, 0x00000101, BIT(PIPE_NONE) }, |
| @@ -1966,5 +1966,4 @@ static inline __always_unused void __build_asserts(void) · 200 unmodified lines | ||
| 1966 | 1966 | BUILD_BUG_ON(a660_protect.count > a660_protect.count_max); |
| 1967 | 1967 | BUILD_BUG_ON(a690_protect.count > a690_protect.count_max); |
| 1968 | 1968 | BUILD_BUG_ON(a730_protect.count > a730_protect.count_max); |
| 1969 | − BUILD_BUG_ON(a840_protect.count > a840_protect.count_max); | |
| 1970 | 1969 | } |
| @@ -310,11 +310,21 @@ static void a8xx_set_ubwc_config(struct msm_gpu *gpu) · 309 unmodified lines | ||
| 310 | 310 | hbb = cfg->highest_bank_bit - 13; |
| 311 | 311 | hbb_hi = hbb >> 2; |
| 312 | 312 | hbb_lo = hbb & 3; |
| 313 | − a8xx_write_pipe(gpu, PIPE_BV, REG_A8XX_GRAS_NC_MODE_CNTL, hbb << 5); | |
| 314 | − a8xx_write_pipe(gpu, PIPE_BR, REG_A8XX_GRAS_NC_MODE_CNTL, hbb << 5); | |
| 313 | + | |
| 314 | + a8xx_write_pipe(gpu, PIPE_BV, REG_A8XX_GRAS_NC_MODE_CNTL, | |
| 315 | + hbb << 5 | | |
| 316 | + level3_swizzling_dis << 4 | | |
| 317 | + level2_swizzling_dis << 3); | |
| 318 | + | |
| 319 | + a8xx_write_pipe(gpu, PIPE_BR, REG_A8XX_GRAS_NC_MODE_CNTL, | |
| 320 | + hbb << 5 | | |
| 321 | + level3_swizzling_dis << 4 | | |
| 322 | + level2_swizzling_dis << 3); | |
| 315 | 323 | |
| 316 | 324 | a8xx_write_pipe(gpu, PIPE_BR, REG_A8XX_RB_CCU_NC_MODE_CNTL, |
| 317 | 325 | yuvnotcomptofc << 6 | |
| 326 | + level3_swizzling_dis << 5 | | |
| 327 | + level2_swizzling_dis << 4 | | |
| 318 | 328 | hbb_hi << 3 | |
| 319 | 329 | hbb_lo << 1); |
| 320 | 330 | |
| @@ -302,6 +302,7 @@ static const struct of_device_id dt_match[] = { · 301 unmodified lines | ||
| 302 | 302 | { .compatible = "qcom,kgsl-3d0" }, |
| 303 | 303 | {} |
| 304 | 304 | }; |
| 305 | +MODULE_DEVICE_TABLE(of, dt_match); | |
| 305 | 306 | |
| 306 | 307 | static int adreno_runtime_resume(struct device *dev) |
| 307 | 308 | { |
| @@ -133,7 +133,7 @@ static const struct dpu_sspp_cfg sc8280xp_sspp[] = { · 132 unmodified lines | ||
| 133 | 133 | static const struct dpu_lm_cfg sc8280xp_lm[] = { |
| 134 | 134 | { |
| 135 | 135 | .name = "lm_0", .id = LM_0, |
| 136 | − .base = 0x44000, .len = 0x320, | |
| 136 | + .base = 0x44000, .len = 0x400, | |
| 137 | 137 | .features = MIXER_MSM8998_MASK, |
| 138 | 138 | .sblk = &sdm845_lm_sblk, |
| 139 | 139 | .lm_pair = LM_1, |
| @@ -141,7 +141,7 @@ static const struct dpu_lm_cfg sc8280xp_lm[] = { · 1 unmodified line | ||
| 141 | 141 | .dspp = DSPP_0, |
| 142 | 142 | }, { |
| 143 | 143 | .name = "lm_1", .id = LM_1, |
| 144 | − .base = 0x45000, .len = 0x320, | |
| 144 | + .base = 0x45000, .len = 0x400, | |
| 145 | 145 | .features = MIXER_MSM8998_MASK, |
| 146 | 146 | .sblk = &sdm845_lm_sblk, |
| 147 | 147 | .lm_pair = LM_0, |
| @@ -149,7 +149,7 @@ static const struct dpu_lm_cfg sc8280xp_lm[] = { · 1 unmodified line | ||
| 149 | 149 | .dspp = DSPP_1, |
| 150 | 150 | }, { |
| 151 | 151 | .name = "lm_2", .id = LM_2, |
| 152 | − .base = 0x46000, .len = 0x320, | |
| 152 | + .base = 0x46000, .len = 0x400, | |
| 153 | 153 | .features = MIXER_MSM8998_MASK, |
| 154 | 154 | .sblk = &sdm845_lm_sblk, |
| 155 | 155 | .lm_pair = LM_3, |
| @@ -157,7 +157,7 @@ static const struct dpu_lm_cfg sc8280xp_lm[] = { · 1 unmodified line | ||
| 157 | 157 | .dspp = DSPP_2, |
| 158 | 158 | }, { |
| 159 | 159 | .name = "lm_3", .id = LM_3, |
| 160 | − .base = 0x47000, .len = 0x320, | |
| 160 | + .base = 0x47000, .len = 0x400, | |
| 161 | 161 | .features = MIXER_MSM8998_MASK, |
| 162 | 162 | .sblk = &sdm845_lm_sblk, |
| 163 | 163 | .lm_pair = LM_2, |
| @@ -165,14 +165,14 @@ static const struct dpu_lm_cfg sc8280xp_lm[] = { · 1 unmodified line | ||
| 165 | 165 | .dspp = DSPP_3, |
| 166 | 166 | }, { |
| 167 | 167 | .name = "lm_4", .id = LM_4, |
| 168 | − .base = 0x48000, .len = 0x320, | |
| 168 | + .base = 0x48000, .len = 0x400, | |
| 169 | 169 | .features = MIXER_MSM8998_MASK, |
| 170 | 170 | .sblk = &sdm845_lm_sblk, |
| 171 | 171 | .lm_pair = LM_5, |
| 172 | 172 | .pingpong = PINGPONG_4, |
| 173 | 173 | }, { |
| 174 | 174 | .name = "lm_5", .id = LM_5, |
| 175 | − .base = 0x49000, .len = 0x320, | |
| 175 | + .base = 0x49000, .len = 0x400, | |
| 176 | 176 | .features = MIXER_MSM8998_MASK, |
| 177 | 177 | .sblk = &sdm845_lm_sblk, |
| 178 | 178 | .lm_pair = LM_4, |
| @@ -134,7 +134,7 @@ static const struct dpu_sspp_cfg sm8450_sspp[] = { · 133 unmodified lines | ||
| 134 | 134 | static const struct dpu_lm_cfg sm8450_lm[] = { |
| 135 | 135 | { |
| 136 | 136 | .name = "lm_0", .id = LM_0, |
| 137 | − .base = 0x44000, .len = 0x320, | |
| 137 | + .base = 0x44000, .len = 0x400, | |
| 138 | 138 | .features = MIXER_MSM8998_MASK, |
| 139 | 139 | .sblk = &sdm845_lm_sblk, |
| 140 | 140 | .lm_pair = LM_1, |
| @@ -142,7 +142,7 @@ static const struct dpu_lm_cfg sm8450_lm[] = { · 1 unmodified line | ||
| 142 | 142 | .dspp = DSPP_0, |
| 143 | 143 | }, { |
| 144 | 144 | .name = "lm_1", .id = LM_1, |
| 145 | − .base = 0x45000, .len = 0x320, | |
| 145 | + .base = 0x45000, .len = 0x400, | |
| 146 | 146 | .features = MIXER_MSM8998_MASK, |
| 147 | 147 | .sblk = &sdm845_lm_sblk, |
| 148 | 148 | .lm_pair = LM_0, |
| @@ -150,7 +150,7 @@ static const struct dpu_lm_cfg sm8450_lm[] = { · 1 unmodified line | ||
| 150 | 150 | .dspp = DSPP_1, |
| 151 | 151 | }, { |
| 152 | 152 | .name = "lm_2", .id = LM_2, |
| 153 | − .base = 0x46000, .len = 0x320, | |
| 153 | + .base = 0x46000, .len = 0x400, | |
| 154 | 154 | .features = MIXER_MSM8998_MASK, |
| 155 | 155 | .sblk = &sdm845_lm_sblk, |
| 156 | 156 | .lm_pair = LM_3, |
| @@ -158,7 +158,7 @@ static const struct dpu_lm_cfg sm8450_lm[] = { · 1 unmodified line | ||
| 158 | 158 | .dspp = DSPP_2, |
| 159 | 159 | }, { |
| 160 | 160 | .name = "lm_3", .id = LM_3, |
| 161 | − .base = 0x47000, .len = 0x320, | |
| 161 | + .base = 0x47000, .len = 0x400, | |
| 162 | 162 | .features = MIXER_MSM8998_MASK, |
| 163 | 163 | .sblk = &sdm845_lm_sblk, |
| 164 | 164 | .lm_pair = LM_2, |
| @@ -166,14 +166,14 @@ static const struct dpu_lm_cfg sm8450_lm[] = { · 1 unmodified line | ||
| 166 | 166 | .dspp = DSPP_3, |
| 167 | 167 | }, { |
| 168 | 168 | .name = "lm_4", .id = LM_4, |
| 169 | − .base = 0x48000, .len = 0x320, | |
| 169 | + .base = 0x48000, .len = 0x400, | |
| 170 | 170 | .features = MIXER_MSM8998_MASK, |
| 171 | 171 | .sblk = &sdm845_lm_sblk, |
| 172 | 172 | .lm_pair = LM_5, |
| 173 | 173 | .pingpong = PINGPONG_4, |
| 174 | 174 | }, { |
| 175 | 175 | .name = "lm_5", .id = LM_5, |
| 176 | − .base = 0x49000, .len = 0x320, | |
| 176 | + .base = 0x49000, .len = 0x400, | |
| 177 | 177 | .features = MIXER_MSM8998_MASK, |
| 178 | 178 | .sblk = &sdm845_lm_sblk, |
| 179 | 179 | .lm_pair = LM_4, |
| @@ -366,8 +366,8 @@ static const struct dpu_intf_cfg sa8775p_intf[] = { · 365 unmodified lines | ||
| 366 | 366 | .type = INTF_NONE, |
| 367 | 367 | .controller_id = MSM_DP_CONTROLLER_0, /* pair with intf_0 for DP MST */ |
| 368 | 368 | .prog_fetch_lines_worst_case = 24, |
| 369 | − .intr_underrun = DPU_IRQ_IDX(MDP_SSPP_TOP0_INTR, 17), | |
| 370 | − .intr_vsync = DPU_IRQ_IDX(MDP_SSPP_TOP0_INTR, 16), | |
| 369 | + .intr_underrun = DPU_IRQ_IDX(MDP_SSPP_TOP0_INTR, 16), | |
| 370 | + .intr_vsync = DPU_IRQ_IDX(MDP_SSPP_TOP0_INTR, 17), | |
| 371 | 371 | }, { |
| 372 | 372 | .name = "intf_7", .id = INTF_7, |
| 373 | 373 | .base = 0x3b000, .len = 0x280, |
| @@ -131,7 +131,7 @@ static const struct dpu_sspp_cfg sm8550_sspp[] = { · 130 unmodified lines | ||
| 131 | 131 | static const struct dpu_lm_cfg sm8550_lm[] = { |
| 132 | 132 | { |
| 133 | 133 | .name = "lm_0", .id = LM_0, |
| 134 | − .base = 0x44000, .len = 0x320, | |
| 134 | + .base = 0x44000, .len = 0x400, | |
| 135 | 135 | .features = MIXER_MSM8998_MASK, |
| 136 | 136 | .sblk = &sdm845_lm_sblk, |
| 137 | 137 | .lm_pair = LM_1, |
| @@ -139,7 +139,7 @@ static const struct dpu_lm_cfg sm8550_lm[] = { · 1 unmodified line | ||
| 139 | 139 | .dspp = DSPP_0, |
| 140 | 140 | }, { |
| 141 | 141 | .name = "lm_1", .id = LM_1, |
| 142 | − .base = 0x45000, .len = 0x320, | |
| 142 | + .base = 0x45000, .len = 0x400, | |
| 143 | 143 | .features = MIXER_MSM8998_MASK, |
| 144 | 144 | .sblk = &sdm845_lm_sblk, |
| 145 | 145 | .lm_pair = LM_0, |
| @@ -147,7 +147,7 @@ static const struct dpu_lm_cfg sm8550_lm[] = { · 1 unmodified line | ||
| 147 | 147 | .dspp = DSPP_1, |
| 148 | 148 | }, { |
| 149 | 149 | .name = "lm_2", .id = LM_2, |
| 150 | − .base = 0x46000, .len = 0x320, | |
| 150 | + .base = 0x46000, .len = 0x400, | |
| 151 | 151 | .features = MIXER_MSM8998_MASK, |
| 152 | 152 | .sblk = &sdm845_lm_sblk, |
| 153 | 153 | .lm_pair = LM_3, |
| @@ -155,7 +155,7 @@ static const struct dpu_lm_cfg sm8550_lm[] = { · 1 unmodified line | ||
| 155 | 155 | .dspp = DSPP_2, |
| 156 | 156 | }, { |
| 157 | 157 | .name = "lm_3", .id = LM_3, |
| 158 | − .base = 0x47000, .len = 0x320, | |
| 158 | + .base = 0x47000, .len = 0x400, | |
| 159 | 159 | .features = MIXER_MSM8998_MASK, |
| 160 | 160 | .sblk = &sdm845_lm_sblk, |
| 161 | 161 | .lm_pair = LM_2, |
| @@ -163,14 +163,14 @@ static const struct dpu_lm_cfg sm8550_lm[] = { · 1 unmodified line | ||
| 163 | 163 | .dspp = DSPP_3, |
| 164 | 164 | }, { |
| 165 | 165 | .name = "lm_4", .id = LM_4, |
| 166 | − .base = 0x48000, .len = 0x320, | |
| 166 | + .base = 0x48000, .len = 0x400, | |
| 167 | 167 | .features = MIXER_MSM8998_MASK, |
| 168 | 168 | .sblk = &sdm845_lm_sblk, |
| 169 | 169 | .lm_pair = LM_5, |
| 170 | 170 | .pingpong = PINGPONG_4, |
| 171 | 171 | }, { |
| 172 | 172 | .name = "lm_5", .id = LM_5, |
| 173 | − .base = 0x49000, .len = 0x320, | |
| 173 | + .base = 0x49000, .len = 0x400, | |
| 174 | 174 | .features = MIXER_MSM8998_MASK, |
| 175 | 175 | .sblk = &sdm845_lm_sblk, |
| 176 | 176 | .lm_pair = LM_4, |
| @@ -131,7 +131,7 @@ static const struct dpu_sspp_cfg sar2130p_sspp[] = { · 130 unmodified lines | ||
| 131 | 131 | static const struct dpu_lm_cfg sar2130p_lm[] = { |
| 132 | 132 | { |
| 133 | 133 | .name = "lm_0", .id = LM_0, |
| 134 | − .base = 0x44000, .len = 0x320, | |
| 134 | + .base = 0x44000, .len = 0x400, | |
| 135 | 135 | .features = MIXER_MSM8998_MASK, |
| 136 | 136 | .sblk = &sdm845_lm_sblk, |
| 137 | 137 | .lm_pair = LM_1, |
| @@ -139,7 +139,7 @@ static const struct dpu_lm_cfg sar2130p_lm[] = { · 1 unmodified line | ||
| 139 | 139 | .dspp = DSPP_0, |
| 140 | 140 | }, { |
| 141 | 141 | .name = "lm_1", .id = LM_1, |
| 142 | − .base = 0x45000, .len = 0x320, | |
| 142 | + .base = 0x45000, .len = 0x400, | |
| 143 | 143 | .features = MIXER_MSM8998_MASK, |
| 144 | 144 | .sblk = &sdm845_lm_sblk, |
| 145 | 145 | .lm_pair = LM_0, |
| @@ -147,7 +147,7 @@ static const struct dpu_lm_cfg sar2130p_lm[] = { · 1 unmodified line | ||
| 147 | 147 | .dspp = DSPP_1, |
| 148 | 148 | }, { |
| 149 | 149 | .name = "lm_2", .id = LM_2, |
| 150 | − .base = 0x46000, .len = 0x320, | |
| 150 | + .base = 0x46000, .len = 0x400, | |
| 151 | 151 | .features = MIXER_MSM8998_MASK, |
| 152 | 152 | .sblk = &sdm845_lm_sblk, |
| 153 | 153 | .lm_pair = LM_3, |
| @@ -155,7 +155,7 @@ static const struct dpu_lm_cfg sar2130p_lm[] = { · 1 unmodified line | ||
| 155 | 155 | .dspp = DSPP_2, |
| 156 | 156 | }, { |
| 157 | 157 | .name = "lm_3", .id = LM_3, |
| 158 | − .base = 0x47000, .len = 0x320, | |
| 158 | + .base = 0x47000, .len = 0x400, | |
| 159 | 159 | .features = MIXER_MSM8998_MASK, |
| 160 | 160 | .sblk = &sdm845_lm_sblk, |
| 161 | 161 | .lm_pair = LM_2, |
| @@ -163,14 +163,14 @@ static const struct dpu_lm_cfg sar2130p_lm[] = { · 1 unmodified line | ||
| 163 | 163 | .dspp = DSPP_3, |
| 164 | 164 | }, { |
| 165 | 165 | .name = "lm_4", .id = LM_4, |
| 166 | − .base = 0x48000, .len = 0x320, | |
| 166 | + .base = 0x48000, .len = 0x400, | |
| 167 | 167 | .features = MIXER_MSM8998_MASK, |
| 168 | 168 | .sblk = &sdm845_lm_sblk, |
| 169 | 169 | .lm_pair = LM_5, |
| 170 | 170 | .pingpong = PINGPONG_4, |
| 171 | 171 | }, { |
| 172 | 172 | .name = "lm_5", .id = LM_5, |
| 173 | − .base = 0x49000, .len = 0x320, | |
| 173 | + .base = 0x49000, .len = 0x400, | |
| 174 | 174 | .features = MIXER_MSM8998_MASK, |
| 175 | 175 | .sblk = &sdm845_lm_sblk, |
| 176 | 176 | .lm_pair = LM_4, |
| @@ -130,7 +130,7 @@ static const struct dpu_sspp_cfg x1e80100_sspp[] = { · 129 unmodified lines | ||
| 130 | 130 | static const struct dpu_lm_cfg x1e80100_lm[] = { |
| 131 | 131 | { |
| 132 | 132 | .name = "lm_0", .id = LM_0, |
| 133 | − .base = 0x44000, .len = 0x320, | |
| 133 | + .base = 0x44000, .len = 0x400, | |
| 134 | 134 | .features = MIXER_MSM8998_MASK, |
| 135 | 135 | .sblk = &sdm845_lm_sblk, |
| 136 | 136 | .lm_pair = LM_1, |
| @@ -138,7 +138,7 @@ static const struct dpu_lm_cfg x1e80100_lm[] = { · 1 unmodified line | ||
| 138 | 138 | .dspp = DSPP_0, |
| 139 | 139 | }, { |
| 140 | 140 | .name = "lm_1", .id = LM_1, |
| 141 | − .base = 0x45000, .len = 0x320, | |
| 141 | + .base = 0x45000, .len = 0x400, | |
| 142 | 142 | .features = MIXER_MSM8998_MASK, |
| 143 | 143 | .sblk = &sdm845_lm_sblk, |
| 144 | 144 | .lm_pair = LM_0, |
| @@ -146,7 +146,7 @@ static const struct dpu_lm_cfg x1e80100_lm[] = { · 1 unmodified line | ||
| 146 | 146 | .dspp = DSPP_1, |
| 147 | 147 | }, { |
| 148 | 148 | .name = "lm_2", .id = LM_2, |
| 149 | − .base = 0x46000, .len = 0x320, | |
| 149 | + .base = 0x46000, .len = 0x400, | |
| 150 | 150 | .features = MIXER_MSM8998_MASK, |
| 151 | 151 | .sblk = &sdm845_lm_sblk, |
| 152 | 152 | .lm_pair = LM_3, |
| @@ -154,7 +154,7 @@ static const struct dpu_lm_cfg x1e80100_lm[] = { · 1 unmodified line | ||
| 154 | 154 | .dspp = DSPP_2, |
| 155 | 155 | }, { |
| 156 | 156 | .name = "lm_3", .id = LM_3, |
| 157 | − .base = 0x47000, .len = 0x320, | |
| 157 | + .base = 0x47000, .len = 0x400, | |
| 158 | 158 | .features = MIXER_MSM8998_MASK, |
| 159 | 159 | .sblk = &sdm845_lm_sblk, |
| 160 | 160 | .lm_pair = LM_2, |
| @@ -162,14 +162,14 @@ static const struct dpu_lm_cfg x1e80100_lm[] = { · 1 unmodified line | ||
| 162 | 162 | .dspp = DSPP_3, |
| 163 | 163 | }, { |
| 164 | 164 | .name = "lm_4", .id = LM_4, |
| 165 | − .base = 0x48000, .len = 0x320, | |
| 165 | + .base = 0x48000, .len = 0x400, | |
| 166 | 166 | .features = MIXER_MSM8998_MASK, |
| 167 | 167 | .sblk = &sdm845_lm_sblk, |
| 168 | 168 | .lm_pair = LM_5, |
| 169 | 169 | .pingpong = PINGPONG_4, |
| 170 | 170 | }, { |
| 171 | 171 | .name = "lm_5", .id = LM_5, |
| 172 | − .base = 0x49000, .len = 0x320, | |
| 172 | + .base = 0x49000, .len = 0x400, | |
| 173 | 173 | .features = MIXER_MSM8998_MASK, |
| 174 | 174 | .sblk = &sdm845_lm_sblk, |
| 175 | 175 | .lm_pair = LM_4, |
| @@ -89,7 +89,7 @@ static void dpu_setup_dspp_gc(struct dpu_hw_dspp *ctx, · 88 unmodified lines | ||
| 89 | 89 | base = ctx->cap->sblk->gc.base; |
| 90 | 90 | |
| 91 | 91 | if (!base) { |
| 92 | − DRM_ERROR("invalid ctx %pK gc base\n", ctx); | |
| 92 | + DRM_ERROR("invalid ctx %p gc base\n", ctx); | |
| 93 | 93 | return; |
| 94 | 94 | } |
| 95 | 95 | |
| @@ -156,11 +156,13 @@ static void dpu_hw_sspp_setup_pe_config_v13(struct dpu_hw_sspp *ctx, · 155 unmodified lines | ||
| 156 | 156 | u8 color; |
| 157 | 157 | u32 lr_pe[4], tb_pe[4]; |
| 158 | 158 | const u32 bytemask = 0xff; |
| 159 | − u32 offset = ctx->cap->sblk->sspp_rec0_blk.base; | |
| 159 | + u32 offset; | |
| 160 | 160 | |
| 161 | 161 | if (!ctx || !pe_ext) |
| 162 | 162 | return; |
| 163 | 163 | |
| 164 | + offset = ctx->cap->sblk->sspp_rec0_blk.base; | |
| 165 | + | |
| 164 | 166 | c = &ctx->hw; |
| 165 | 167 | /* program SW pixel extension override for all pipes*/ |
| 166 | 168 | for (color = 0; color < DPU_MAX_PLANES; color++) { |
| @@ -350,26 +350,28 @@ static bool _dpu_rm_check_lm_and_get_connected_blks(struct dpu_rm *rm, · 349 unmodified lines | ||
| 350 | 350 | return true; |
| 351 | 351 | } |
| 352 | 352 | |
| 353 | −static bool dpu_rm_find_lms(struct dpu_rm *rm, | |
| 354 | − struct dpu_global_state *global_state, | |
| 355 | − uint32_t crtc_id, bool skip_dspp, | |
| 356 | − struct msm_display_topology *topology, | |
| 357 | − int *lm_idx, int *pp_idx, int *dspp_idx) | |
| 353 | +static int _dpu_rm_reserve_lms(struct dpu_rm *rm, | |
| 354 | + struct dpu_global_state *global_state, | |
| 355 | + uint32_t crtc_id, | |
| 356 | + struct msm_display_topology *topology) | |
| 358 | 357 | |
| 359 | 358 | { |
| 359 | + int lm_idx[MAX_BLOCKS]; | |
| 360 | + int pp_idx[MAX_BLOCKS]; | |
| 361 | + int dspp_idx[MAX_BLOCKS] = {0}; | |
| 360 | 362 | int i, lm_count = 0; |
| 361 | 363 | |
| 364 | + if (!topology->num_lm) { | |
| 365 | + DPU_ERROR("zero LMs in topology\n"); | |
| 366 | + return -EINVAL; | |
| 367 | + } | |
| 368 | + | |
| 362 | 369 | /* Find a primary mixer */ |
| 363 | 370 | for (i = 0; i < ARRAY_SIZE(rm->mixer_blks) && |
| 364 | 371 | lm_count < topology->num_lm; i++) { |
| 365 | 372 | if (!rm->mixer_blks[i]) |
| 366 | 373 | continue; |
| 367 | 374 | |
| 368 | − if (skip_dspp && to_dpu_hw_mixer(rm->mixer_blks[i])->cap->dspp) { | |
| 369 | − DPU_DEBUG("Skipping LM_%d, skipping LMs with DSPPs\n", i); | |
| 370 | − continue; | |
| 371 | − } | |
| 372 | − | |
| 373 | 375 | /* |
| 374 | 376 | * Reset lm_count to an even index. This will drop the previous |
| 375 | 377 | * primary mixer if failed to find its peer. |
| @@ -408,38 +410,12 @@ static bool dpu_rm_find_lms(struct dpu_rm *rm, · 32 unmodified lines | ||
| 408 | 410 | } |
| 409 | 411 | } |
| 410 | 412 | |
| 411 | − return lm_count == topology->num_lm; | |
| 412 | −} | |
| 413 | − | |
| 414 | −static int _dpu_rm_reserve_lms(struct dpu_rm *rm, | |
| 415 | − struct dpu_global_state *global_state, | |
| 416 | − uint32_t crtc_id, | |
| 417 | − struct msm_display_topology *topology) | |
| 418 | − | |
| 419 | −{ | |
| 420 | − int lm_idx[MAX_BLOCKS]; | |
| 421 | − int pp_idx[MAX_BLOCKS]; | |
| 422 | − int dspp_idx[MAX_BLOCKS] = {0}; | |
| 423 | − int i; | |
| 424 | − bool found; | |
| 425 | − | |
| 426 | − if (!topology->num_lm) { | |
| 427 | − DPU_ERROR("zero LMs in topology\n"); | |
| 428 | − return -EINVAL; | |
| 429 | − } | |
| 430 | − | |
| 431 | − /* Try using non-DSPP LM blocks first */ | |
| 432 | − found = dpu_rm_find_lms(rm, global_state, crtc_id, !topology->num_dspp, | |
| 433 | − topology, lm_idx, pp_idx, dspp_idx); | |
| 434 | − if (!found && !topology->num_dspp) | |
| 435 | − found = dpu_rm_find_lms(rm, global_state, crtc_id, false, | |
| 436 | − topology, lm_idx, pp_idx, dspp_idx); | |
| 437 | − if (!found) { | |
| 413 | + if (lm_count != topology->num_lm) { | |
| 438 | 414 | DPU_DEBUG("unable to find appropriate mixers\n"); |
| 439 | 415 | return -ENAVAIL; |
| 440 | 416 | } |
| 441 | 417 | |
| 442 | − for (i = 0; i < topology->num_lm; i++) { | |
| 418 | + for (i = 0; i < lm_count; i++) { | |
| 443 | 419 | global_state->mixer_to_crtc_id[lm_idx[i]] = crtc_id; |
| 444 | 420 | global_state->pingpong_to_crtc_id[pp_idx[i]] = crtc_id; |
| 445 | 421 | global_state->dspp_to_crtc_id[dspp_idx[i]] = |
| @@ -584,13 +584,30 @@ void dsi_link_clk_disable_v2(struct msm_dsi_host *msm_host) · 583 unmodified lines | ||
| 584 | 584 | * FIXME: Reconsider this if/when CMD mode handling is rewritten to use |
| 585 | 585 | * transfer time and data overhead as a starting point of the calculations. |
| 586 | 586 | */ |
| 587 | −static unsigned long dsi_adjust_pclk_for_compression(const struct drm_display_mode *mode, | |
| 588 | − const struct drm_dsc_config *dsc) | |
| 587 | +static unsigned long | |
| 588 | +dsi_adjust_pclk_for_compression(const struct drm_display_mode *mode, | |
| 589 | + const struct drm_dsc_config *dsc, | |
| 590 | + bool is_bonded_dsi) | |
| 589 | 591 | { |
| 590 | − int new_hdisplay = DIV_ROUND_UP(mode->hdisplay * drm_dsc_get_bpp_int(dsc), | |
| 591 | − dsc->bits_per_component * 3); | |
| 592 | + int hdisplay, new_hdisplay, new_htotal; | |
| 592 | 593 | |
| 593 | − int new_htotal = mode->htotal - mode->hdisplay + new_hdisplay; | |
| 594 | + /* | |
| 595 | + * For bonded DSI, split hdisplay across two links and round up each | |
| 596 | + * half separately, passing the full hdisplay would only round up once. | |
| 597 | + * This also aligns with the hdisplay we program later in | |
| 598 | + * dsi_timing_setup() | |
| 599 | + */ | |
| 600 | + hdisplay = mode->hdisplay; | |
| 601 | + if (is_bonded_dsi) | |
| 602 | + hdisplay /= 2; | |
| 603 | + | |
| 604 | + new_hdisplay = DIV_ROUND_UP(hdisplay * drm_dsc_get_bpp_int(dsc), | |
| 605 | + dsc->bits_per_component * 3); | |
| 606 | + | |
| 607 | + if (is_bonded_dsi) | |
| 608 | + new_hdisplay *= 2; | |
| 609 | + | |
| 610 | + new_htotal = mode->htotal - mode->hdisplay + new_hdisplay; | |
| 594 | 611 | |
| 595 | 612 | return mult_frac(mode->clock * 1000u, new_htotal, mode->htotal); |
| 596 | 613 | } |
| @@ -603,7 +620,7 @@ static unsigned long dsi_get_pclk_rate(const struct drm_display_mode *mode, · 6 unmodified lines | ||
| 603 | 620 | pclk_rate = mode->clock * 1000u; |
| 604 | 621 | |
| 605 | 622 | if (dsc) |
| 606 | − pclk_rate = dsi_adjust_pclk_for_compression(mode, dsc); | |
| 623 | + pclk_rate = dsi_adjust_pclk_for_compression(mode, dsc, is_bonded_dsi); | |
| 607 | 624 | |
| 608 | 625 | /* |
| 609 | 626 | * For bonded DSI mode, the current DRM mode has the complete width of the |
| @@ -993,7 +1010,7 @@ static void dsi_timing_setup(struct msm_dsi_host *msm_host, bool is_bonded_dsi) · 383 unmodified lines | ||
| 993 | 1010 | |
| 994 | 1011 | if (msm_host->dsc) { |
| 995 | 1012 | struct drm_dsc_config *dsc = msm_host->dsc; |
| 996 | − u32 bytes_per_pclk; | |
| 1013 | + u32 bits_per_pclk; | |
| 997 | 1014 | |
| 998 | 1015 | /* update dsc params with timing params */ |
| 999 | 1016 | if (!dsc || !mode->hdisplay || !mode->vdisplay) { |
| @@ -1015,7 +1032,9 @@ static void dsi_timing_setup(struct msm_dsi_host *msm_host, bool is_bonded_dsi) · 15 unmodified lines | ||
| 1015 | 1032 | |
| 1016 | 1033 | /* |
| 1017 | 1034 | * DPU sends 3 bytes per pclk cycle to DSI. If widebus is |
| 1018 | − * enabled, bus width is extended to 6 bytes. | |
| 1035 | + * enabled, MDP always sends out 48-bit compressed data per | |
| 1036 | + * pclk and on average, DSI consumes an amount of compressed | |
| 1037 | + * data equivalent to the uncompressed pixel depth per pclk. | |
| 1019 | 1038 | * |
| 1020 | 1039 | * Calculate the number of pclks needed to transmit one line of |
| 1021 | 1040 | * the compressed data. |
| @@ -1027,12 +1046,12 @@ static void dsi_timing_setup(struct msm_dsi_host *msm_host, bool is_bonded_dsi) · 5 unmodified lines | ||
| 1027 | 1046 | * unused anyway. |
| 1028 | 1047 | */ |
| 1029 | 1048 | h_total -= hdisplay; |
| 1030 | − if (wide_bus_enabled && !(msm_host->mode_flags & MIPI_DSI_MODE_VIDEO)) | |
| 1031 | − bytes_per_pclk = 6; | |
| 1049 | + if (wide_bus_enabled) | |
| 1050 | + bits_per_pclk = mipi_dsi_pixel_format_to_bpp(msm_host->format); | |
| 1032 | 1051 | else |
| 1033 | − bytes_per_pclk = 3; | |
| 1052 | + bits_per_pclk = 24; | |
| 1034 | 1053 | |
| 1035 | − hdisplay = DIV_ROUND_UP(msm_dsc_get_bytes_per_line(msm_host->dsc), bytes_per_pclk); | |
| 1054 | + hdisplay = DIV_ROUND_UP(msm_dsc_get_bytes_per_line(msm_host->dsc) * 8, bits_per_pclk); | |
| 1036 | 1055 | |
| 1037 | 1056 | h_total += hdisplay; |
| 1038 | 1057 | ha_end = ha_start + hdisplay; |
| @@ -51,8 +51,8 @@ · 50 unmodified lines | ||
| 51 | 51 | #define DSI_PHY_7NM_QUIRK_V4_3 BIT(3) |
| 52 | 52 | /* Hardware is V5.2 */ |
| 53 | 53 | #define DSI_PHY_7NM_QUIRK_V5_2 BIT(4) |
| 54 | −/* Hardware is V7.0 */ | |
| 55 | −#define DSI_PHY_7NM_QUIRK_V7_0 BIT(5) | |
| 54 | +/* Hardware is V7.2 */ | |
| 55 | +#define DSI_PHY_7NM_QUIRK_V7_2 BIT(5) | |
| 56 | 56 | |
| 57 | 57 | struct dsi_pll_config { |
| 58 | 58 | bool enable_ssc; |
| @@ -143,7 +143,7 @@ static void dsi_pll_calc_dec_frac(struct dsi_pll_7nm *pll, struct dsi_pll_config · 84 unmodified lines | ||
| 143 | 143 | |
| 144 | 144 | if (pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_PRE_V4_1) { |
| 145 | 145 | config->pll_clock_inverters = 0x28; |
| 146 | − } else if ((pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_0)) { | |
| 146 | + } else if ((pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_2)) { | |
| 147 | 147 | if (pll_freq < 163000000ULL) |
| 148 | 148 | config->pll_clock_inverters = 0xa0; |
| 149 | 149 | else if (pll_freq < 175000000ULL) |
| @@ -284,7 +284,7 @@ static void dsi_pll_config_hzindep_reg(struct dsi_pll_7nm *pll) · 134 unmodified lines | ||
| 284 | 284 | } |
| 285 | 285 | |
| 286 | 286 | if ((pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V5_2) || |
| 287 | − (pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_0)) { | |
| 287 | + (pll->phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_2)) { | |
| 288 | 288 | if (pll->vco_current_rate < 1557000000ULL) |
| 289 | 289 | vco_config_1 = 0x08; |
| 290 | 290 | else |
| @@ -699,7 +699,7 @@ static int dsi_7nm_set_usecase(struct msm_dsi_phy *phy) · 408 unmodified lines | ||
| 699 | 699 | case MSM_DSI_PHY_MASTER: |
| 700 | 700 | pll_7nm->slave = pll_7nm_list[(pll_7nm->phy->id + 1) % DSI_MAX]; |
| 701 | 701 | /* v7.0: Enable ATB_EN0 and alternate clock output to external phy */ |
| 702 | − if (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_0) | |
| 702 | + if (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_2) | |
| 703 | 703 | writel(0x07, base + REG_DSI_7nm_PHY_CMN_CTRL_5); |
| 704 | 704 | break; |
| 705 | 705 | case MSM_DSI_PHY_SLAVE: |
| @@ -987,7 +987,7 @@ static int dsi_7nm_phy_enable(struct msm_dsi_phy *phy, · 281 unmodified lines | ||
| 987 | 987 | /* Request for REFGEN READY */ |
| 988 | 988 | if ((phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V4_3) || |
| 989 | 989 | (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V5_2) || |
| 990 | − (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_0)) { | |
| 990 | + (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_2)) { | |
| 991 | 991 | writel(0x1, phy->base + REG_DSI_7nm_PHY_CMN_GLBL_DIGTOP_SPARE10); |
| 992 | 992 | udelay(500); |
| 993 | 993 | } |
| @@ -1021,7 +1021,7 @@ static int dsi_7nm_phy_enable(struct msm_dsi_phy *phy, · 27 unmodified lines | ||
| 1021 | 1021 | lane_ctrl0 = 0x1f; |
| 1022 | 1022 | } |
| 1023 | 1023 | |
| 1024 | − if ((phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_0)) { | |
| 1024 | + if ((phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_2)) { | |
| 1025 | 1025 | if (phy->cphy_mode) { |
| 1026 | 1026 | /* TODO: different for second phy */ |
| 1027 | 1027 | vreg_ctrl_0 = 0x57; |
| @@ -1097,7 +1097,7 @@ static int dsi_7nm_phy_enable(struct msm_dsi_phy *phy, · 69 unmodified lines | ||
| 1097 | 1097 | |
| 1098 | 1098 | /* program CMN_CTRL_4 for minor_ver 2 chipsets*/ |
| 1099 | 1099 | if ((phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V5_2) || |
| 1100 | − (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_0) || | |
| 1100 | + (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_2) || | |
| 1101 | 1101 | (readl(base + REG_DSI_7nm_PHY_CMN_REVISION_ID0) & (0xf0)) == 0x20) |
| 1102 | 1102 | writel(0x04, base + REG_DSI_7nm_PHY_CMN_CTRL_4); |
| 1103 | 1103 | |
| @@ -1213,7 +1213,7 @@ static void dsi_7nm_phy_disable(struct msm_dsi_phy *phy) · 109 unmodified lines | ||
| 1213 | 1213 | /* Turn off REFGEN Vote */ |
| 1214 | 1214 | if ((phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V4_3) || |
| 1215 | 1215 | (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V5_2) || |
| 1216 | − (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_0)) { | |
| 1216 | + (phy->cfg->quirks & DSI_PHY_7NM_QUIRK_V7_2)) { | |
| 1217 | 1217 | writel(0x0, base + REG_DSI_7nm_PHY_CMN_GLBL_DIGTOP_SPARE10); |
| 1218 | 1218 | wmb(); |
| 1219 | 1219 | /* Delay to ensure HW removes vote before PHY shut down */ |
| @@ -1502,7 +1502,7 @@ const struct msm_dsi_phy_cfg dsi_phy_3nm_8750_cfgs = { · 282 unmodified lines | ||
| 1502 | 1502 | #endif |
| 1503 | 1503 | .io_start = { 0xae95000, 0xae97000 }, |
| 1504 | 1504 | .num_dsi_phy = 2, |
| 1505 | − .quirks = DSI_PHY_7NM_QUIRK_V7_0, | |
| 1505 | + .quirks = DSI_PHY_7NM_QUIRK_V7_2, | |
| 1506 | 1506 | }; |
| 1507 | 1507 | |
| 1508 | 1508 | const struct msm_dsi_phy_cfg dsi_phy_3nm_kaanapali_cfgs = { |
| @@ -1525,5 +1525,5 @@ const struct msm_dsi_phy_cfg dsi_phy_3nm_kaanapali_cfgs = { · 16 unmodified lines | ||
| 1525 | 1525 | #endif |
| 1526 | 1526 | .io_start = { 0x9ac1000, 0x9ac4000 }, |
| 1527 | 1527 | .num_dsi_phy = 2, |
| 1528 | − .quirks = DSI_PHY_7NM_QUIRK_V7_0, | |
| 1528 | + .quirks = DSI_PHY_7NM_QUIRK_V7_2, | |
| 1529 | 1529 | }; |
| @@ -1230,6 +1230,9 @@ nouveau_connector_aux_xfer(struct drm_dp_aux *obj, struct drm_dp_aux_msg *msg) · 1229 unmodified lines | ||
| 1230 | 1230 | u8 size = msg->size; |
| 1231 | 1231 | int ret; |
| 1232 | 1232 | |
| 1233 | + if (pm_runtime_suspended(nv_connector->base.dev->dev)) | |
| 1234 | + return -EBUSY; | |
| 1235 | + | |
| 1233 | 1236 | nv_encoder = find_encoder(&nv_connector->base, DCB_OUTPUT_DP); |
| 1234 | 1237 | if (!nv_encoder) |
| 1235 | 1238 | return -ENODEV; |
| @@ -737,8 +737,8 @@ r535_gsp_acpi_caps(acpi_handle handle, CAPS_METHOD_DATA *caps) · 736 unmodified lines | ||
| 737 | 737 | if (!obj) |
| 738 | 738 | goto done; |
| 739 | 739 | |
| 740 | − if (WARN_ON(obj->type != ACPI_TYPE_BUFFER) || | |
| 741 | − WARN_ON(obj->buffer.length != 4)) | |
| 740 | + if (obj->type != ACPI_TYPE_BUFFER || | |
| 741 | + obj->buffer.length != 4) | |
| 742 | 742 | goto done; |
| 743 | 743 | |
| 744 | 744 | caps->status = 0; |
| @@ -773,8 +773,8 @@ r535_gsp_acpi_jt(acpi_handle handle, JT_METHOD_DATA *jt) · 28 unmodified lines | ||
| 773 | 773 | if (!obj) |
| 774 | 774 | goto done; |
| 775 | 775 | |
| 776 | − if (WARN_ON(obj->type != ACPI_TYPE_BUFFER) || | |
| 777 | − WARN_ON(obj->buffer.length != 4)) | |
| 776 | + if (obj->type != ACPI_TYPE_BUFFER || | |
| 777 | + obj->buffer.length != 4) | |
| 778 | 778 | goto done; |
| 779 | 779 | |
| 780 | 780 | jt->status = 0; |
| @@ -861,8 +861,8 @@ r535_gsp_acpi_dod(acpi_handle handle, DOD_METHOD_DATA *dod) · 80 unmodified lines | ||
| 861 | 861 | |
| 862 | 862 | _DOD = output.pointer; |
| 863 | 863 | |
| 864 | − if (WARN_ON(_DOD->type != ACPI_TYPE_PACKAGE) || | |
| 865 | − WARN_ON(_DOD->package.count > ARRAY_SIZE(dod->acpiIdList))) | |
| 864 | + if (_DOD->type != ACPI_TYPE_PACKAGE || | |
| 865 | + _DOD->package.count > ARRAY_SIZE(dod->acpiIdList)) | |
| 866 | 866 | return; |
| 867 | 867 | |
| 868 | 868 | for (int i = 0; i < _DOD->package.count; i++) { |
| @@ -893,14 +893,15 @@ panthor_queue_get_syncwait_obj(struct panthor_group *group, struct panthor_queue · 892 unmodified lines | ||
| 893 | 893 | |
| 894 | 894 | out_sync: |
| 895 | 895 | /* Make sure the CPU caches are invalidated before the seqno is read. |
| 896 | − * drm_gem_shmem_sync() is a NOP if map_wc=true, so no need to check | |
| 896 | + * panthor_gem_sync() is a NOP if map_wc=true, so no need to check | |
| 897 | 897 | * it here. |
| 898 | 898 | */ |
| 899 | − panthor_gem_sync(&bo->base.base, queue->syncwait.offset, | |
| 899 | + panthor_gem_sync(&bo->base.base, | |
| 900 | + DRM_PANTHOR_BO_SYNC_CPU_CACHE_FLUSH_AND_INVALIDATE, | |
| 901 | + queue->syncwait.offset, | |
| 900 | 902 | queue->syncwait.sync64 ? |
| 901 | 903 | sizeof(struct panthor_syncobj_64b) : |
| 902 | − sizeof(struct panthor_syncobj_32b), | |
| 903 | − DRM_PANTHOR_BO_SYNC_CPU_CACHE_FLUSH_AND_INVALIDATE); | |
| 904 | + sizeof(struct panthor_syncobj_32b)); | |
| 904 | 905 | |
| 905 | 906 | return queue->syncwait.kmap + queue->syncwait.offset; |
| 906 | 907 | |
| @@ -1122,6 +1122,7 @@ static int rzg2l_mipi_dsi_host_attach(struct mipi_dsi_host *host, · 1121 unmodified lines | ||
| 1122 | 1122 | struct mipi_dsi_device *device) |
| 1123 | 1123 | { |
| 1124 | 1124 | struct rzg2l_mipi_dsi *dsi = host_to_rzg2l_mipi_dsi(host); |
| 1125 | + int bpp; | |
| 1125 | 1126 | int ret; |
| 1126 | 1127 | |
| 1127 | 1128 | if (device->lanes > dsi->num_data_lanes) { |
| @@ -1131,7 +1132,8 @@ static int rzg2l_mipi_dsi_host_attach(struct mipi_dsi_host *host, · 3 unmodified lines | ||
| 1131 | 1132 | return -EINVAL; |
| 1132 | 1133 | } |
| 1133 | 1134 | |
| 1134 | − switch (mipi_dsi_pixel_format_to_bpp(device->format)) { | |
| 1135 | + bpp = mipi_dsi_pixel_format_to_bpp(device->format); | |
| 1136 | + switch (bpp) { | |
| 1135 | 1137 | case 24: |
| 1136 | 1138 | break; |
| 1137 | 1139 | case 18: |
| @@ -1162,6 +1164,18 @@ static int rzg2l_mipi_dsi_host_attach(struct mipi_dsi_host *host, · 24 unmodified lines | ||
| 1162 | 1164 | |
| 1163 | 1165 | drm_bridge_add(&dsi->bridge); |
| 1164 | 1166 | |
| 1167 | + /* | |
| 1168 | + * Report the required division ratio setting for the MIPI clock dividers. | |
| 1169 | + * | |
| 1170 | + * vclk * bpp = hsclk * 8 * num_lanes | |
| 1171 | + * | |
| 1172 | + * vclk * DSI_AB_divider = hsclk * 16 | |
| 1173 | + * | |
| 1174 | + * which simplifies to... | |
| 1175 | + * DSI_AB_divider = bpp * 2 / num_lanes | |
| 1176 | + */ | |
| 1177 | + rzg2l_cpg_dsi_div_set_divider(bpp * 2 / dsi->lanes, PLL5_TARGET_DSI); | |
| 1178 | + | |
| 1165 | 1179 | return 0; |
| 1166 | 1180 | } |
| 1167 | 1181 | |
| @@ -361,6 +361,7 @@ static void drm_sched_run_free_queue(struct drm_gpu_scheduler *sched) · 360 unmodified lines | ||
| 361 | 361 | /** |
| 362 | 362 | * drm_sched_job_done - complete a job |
| 363 | 363 | * @s_job: pointer to the job which is done |
| 364 | + * @result: 0 on success, -ERRNO on error | |
| 364 | 365 | * |
| 365 | 366 | * Finish the job's fence and resubmit the work items. |
| 366 | 367 | */ |
| @@ -347,6 +347,12 @@ static int st7586_probe(struct spi_device *spi) · 346 unmodified lines | ||
| 347 | 347 | if (ret) |
| 348 | 348 | return ret; |
| 349 | 349 | |
| 350 | + /* | |
| 351 | + * Override value set by mipi_dbi_spi_init(). This driver is a bit | |
| 352 | + * non-standard, so best to set it explicitly here. | |
| 353 | + */ | |
| 354 | + dbi->write_memory_bpw = 8; | |
| 355 | + | |
| 350 | 356 | /* Cannot read from this controller via SPI */ |
| 351 | 357 | dbi->read_commands = NULL; |
| 352 | 358 | |
| @@ -356,15 +362,6 @@ static int st7586_probe(struct spi_device *spi) · 3 unmodified lines | ||
| 356 | 362 | if (ret) |
| 357 | 363 | return ret; |
| 358 | 364 | |
| 359 | − /* | |
| 360 | − * we are using 8-bit data, so we are not actually swapping anything, | |
| 361 | − * but setting mipi->swap_bytes makes mipi_dbi_typec3_command() do the | |
| 362 | − * right thing and not use 16-bit transfers (which results in swapped | |
| 363 | − * bytes on little-endian systems and causes out of order data to be | |
| 364 | − * sent to the display). | |
| 365 | − */ | |
| 366 | − dbi->swap_bytes = true; | |
| 367 | − | |
| 368 | 365 | drm_mode_config_reset(drm); |
| 369 | 366 | |
| 370 | 367 | ret = drm_dev_register(drm, 0); |
| @@ -737,6 +737,7 @@ static int ssd130x_update_rect(struct ssd130x_device *ssd130x, · 736 unmodified lines | ||
| 737 | 737 | unsigned int height = drm_rect_height(rect); |
| 738 | 738 | unsigned int line_length = DIV_ROUND_UP(width, 8); |
| 739 | 739 | unsigned int page_height = SSD130X_PAGE_HEIGHT; |
| 740 | + u8 page_start = ssd130x->page_offset + y / page_height; | |
| 740 | 741 | unsigned int pages = DIV_ROUND_UP(height, page_height); |
| 741 | 742 | struct drm_device *drm = &ssd130x->drm; |
| 742 | 743 | u32 array_idx = 0; |
| @@ -774,14 +775,11 @@ static int ssd130x_update_rect(struct ssd130x_device *ssd130x, · 31 unmodified lines | ||
| 774 | 775 | */ |
| 775 | 776 | |
| 776 | 777 | if (!ssd130x->page_address_mode) { |
| 777 | − u8 page_start; | |
| 778 | − | |
| 779 | 778 | /* Set address range for horizontal addressing mode */ |
| 780 | 779 | ret = ssd130x_set_col_range(ssd130x, ssd130x->col_offset + x, width); |
| 781 | 780 | if (ret < 0) |
| 782 | 781 | return ret; |
| 783 | 782 | |
| 784 | − page_start = ssd130x->page_offset + y / page_height; | |
| 785 | 783 | ret = ssd130x_set_page_range(ssd130x, page_start, pages); |
| 786 | 784 | if (ret < 0) |
| 787 | 785 | return ret; |
| @@ -813,7 +811,7 @@ static int ssd130x_update_rect(struct ssd130x_device *ssd130x, · 25 unmodified lines | ||
| 813 | 811 | */ |
| 814 | 812 | if (ssd130x->page_address_mode) { |
| 815 | 813 | ret = ssd130x_set_page_pos(ssd130x, |
| 816 | − ssd130x->page_offset + i, | |
| 814 | + page_start + i, | |
| 817 | 815 | ssd130x->col_offset + x); |
| 818 | 816 | if (ret < 0) |
| 819 | 817 | return ret; |
| @@ -541,8 +541,8 @@ static int sharp_memory_probe(struct spi_device *spi) · 540 unmodified lines | ||
| 541 | 541 | |
| 542 | 542 | smd = devm_drm_dev_alloc(dev, &sharp_memory_drm_driver, |
| 543 | 543 | struct sharp_memory_device, drm); |
| 544 | − if (!smd) | |
| 545 | − return -ENOMEM; | |
| 544 | + if (IS_ERR(smd)) | |
| 545 | + return PTR_ERR(smd); | |
| 546 | 546 | |
| 547 | 547 | spi_set_drvdata(spi, smd); |
| 548 | 548 | |
| @@ -222,13 +222,13 @@ static void ttm_bo_reserve_interrupted(struct kunit *test) · 221 unmodified lines | ||
| 222 | 222 | KUNIT_FAIL(test, "Couldn't create ttm bo reserve task\n"); |
| 223 | 223 | |
| 224 | 224 | /* Take a lock so the threaded reserve has to wait */ |
| 225 | − mutex_lock(&bo->base.resv->lock.base); | |
| 225 | + dma_resv_lock(bo->base.resv, NULL); | |
| 226 | 226 | |
| 227 | 227 | wake_up_process(task); |
| 228 | 228 | msleep(20); |
| 229 | 229 | err = kthread_stop(task); |
| 230 | 230 | |
| 231 | − mutex_unlock(&bo->base.resv->lock.base); | |
| 231 | + dma_resv_unlock(bo->base.resv); | |
| 232 | 232 | |
| 233 | 233 | KUNIT_ASSERT_EQ(test, err, -ERESTARTSYS); |
| 234 | 234 | } |
| @@ -1107,8 +1107,7 @@ struct ttm_bo_swapout_walk { · 1106 unmodified lines | ||
| 1107 | 1107 | static s64 |
| 1108 | 1108 | ttm_bo_swapout_cb(struct ttm_lru_walk *walk, struct ttm_buffer_object *bo) |
| 1109 | 1109 | { |
| 1110 | − struct ttm_resource *res = bo->resource; | |
| 1111 | − struct ttm_place place = { .mem_type = res->mem_type }; | |
| 1110 | + struct ttm_place place = { .mem_type = bo->resource->mem_type }; | |
| 1112 | 1111 | struct ttm_bo_swapout_walk *swapout_walk = |
| 1113 | 1112 | container_of(walk, typeof(*swapout_walk), walk); |
| 1114 | 1113 | struct ttm_operation_ctx *ctx = walk->arg.ctx; |
| @@ -1148,7 +1147,7 @@ ttm_bo_swapout_cb(struct ttm_lru_walk *walk, struct ttm_buffer_object *bo) · 33 unmodified lines | ||
| 1148 | 1147 | /* |
| 1149 | 1148 | * Move to system cached |
| 1150 | 1149 | */ |
| 1151 | − if (res->mem_type != TTM_PL_SYSTEM) { | |
| 1150 | + if (bo->resource->mem_type != TTM_PL_SYSTEM) { | |
| 1152 | 1151 | struct ttm_resource *evict_mem; |
| 1153 | 1152 | struct ttm_place hop; |
| 1154 | 1153 | |
| @@ -1180,15 +1179,15 @@ ttm_bo_swapout_cb(struct ttm_lru_walk *walk, struct ttm_buffer_object *bo) · 25 unmodified lines | ||
| 1180 | 1179 | |
| 1181 | 1180 | if (ttm_tt_is_populated(tt)) { |
| 1182 | 1181 | spin_lock(&bdev->lru_lock); |
| 1183 | − ttm_resource_del_bulk_move(res, bo); | |
| 1182 | + ttm_resource_del_bulk_move(bo->resource, bo); | |
| 1184 | 1183 | spin_unlock(&bdev->lru_lock); |
| 1185 | 1184 | |
| 1186 | 1185 | ret = ttm_tt_swapout(bdev, tt, swapout_walk->gfp_flags); |
| 1187 | 1186 | |
| 1188 | 1187 | spin_lock(&bdev->lru_lock); |
| 1189 | 1188 | if (ret) |
| 1190 | − ttm_resource_add_bulk_move(res, bo); | |
| 1191 | − ttm_resource_move_to_lru_tail(res); | |
| 1189 | + ttm_resource_add_bulk_move(bo->resource, bo); | |
| 1190 | + ttm_resource_move_to_lru_tail(bo->resource); | |
| 1192 | 1191 | spin_unlock(&bdev->lru_lock); |
| 1193 | 1192 | } |
| 1194 | 1193 | |
| @@ -17,7 +17,7 @@ static inline bool ttm_pool_uses_dma32(struct ttm_pool *pool) · 16 unmodified lines | ||
| 17 | 17 | return pool->alloc_flags & TTM_ALLOCATION_POOL_USE_DMA32; |
| 18 | 18 | } |
| 19 | 19 | |
| 20 | −static inline bool ttm_pool_beneficial_order(struct ttm_pool *pool) | |
| 20 | +static inline unsigned int ttm_pool_beneficial_order(struct ttm_pool *pool) | |
| 21 | 21 | { |
| 22 | 22 | return pool->alloc_flags & 0xff; |
| 23 | 23 | } |
| @@ -105,6 +105,7 @@ struct vmw_cmdbuf_context { · 104 unmodified lines | ||
| 105 | 105 | * @handle: DMA address handle for the command buffer space if @using_mob is |
| 106 | 106 | * false. Immutable. |
| 107 | 107 | * @size: The size of the command buffer space. Immutable. |
| 108 | + * @id: Monotonically increasing ID of the last cmdbuf submitted. | |
| 108 | 109 | * @num_contexts: Number of contexts actually enabled. |
| 109 | 110 | */ |
| 110 | 111 | struct vmw_cmdbuf_man { |
| @@ -132,6 +133,7 @@ struct vmw_cmdbuf_man { · 21 unmodified lines | ||
| 132 | 133 | bool has_pool; |
| 133 | 134 | dma_addr_t handle; |
| 134 | 135 | size_t size; |
| 136 | + u64 id; | |
| 135 | 137 | u32 num_contexts; |
| 136 | 138 | }; |
| 137 | 139 | |
| @@ -303,6 +305,8 @@ static int vmw_cmdbuf_header_submit(struct vmw_cmdbuf_header *header) · 165 unmodified lines | ||
| 303 | 305 | struct vmw_cmdbuf_man *man = header->man; |
| 304 | 306 | u32 val; |
| 305 | 307 | |
| 308 | + header->cb_header->id = man->id++; | |
| 309 | + | |
| 306 | 310 | val = upper_32_bits(header->handle); |
| 307 | 311 | vmw_write(man->dev_priv, SVGA_REG_COMMAND_HIGH, val); |
| 308 | 312 | |
| @@ -1143,7 +1143,7 @@ static int vmw_translate_mob_ptr(struct vmw_private *dev_priv, · 1142 unmodified lines | ||
| 1143 | 1143 | ret = vmw_user_bo_lookup(sw_context->filp, handle, &vmw_bo); |
| 1144 | 1144 | if (ret != 0) { |
| 1145 | 1145 | drm_dbg(&dev_priv->drm, "Could not find or use MOB buffer.\n"); |
| 1146 | − return PTR_ERR(vmw_bo); | |
| 1146 | + return ret; | |
| 1147 | 1147 | } |
| 1148 | 1148 | vmw_bo_placement_set(vmw_bo, VMW_BO_DOMAIN_MOB, VMW_BO_DOMAIN_MOB); |
| 1149 | 1149 | ret = vmw_validation_add_bo(sw_context->ctx, vmw_bo); |
| @@ -1199,7 +1199,7 @@ static int vmw_translate_guest_ptr(struct vmw_private *dev_priv, · 49 unmodified lines | ||
| 1199 | 1199 | ret = vmw_user_bo_lookup(sw_context->filp, handle, &vmw_bo); |
| 1200 | 1200 | if (ret != 0) { |
| 1201 | 1201 | drm_dbg(&dev_priv->drm, "Could not find or use GMR region.\n"); |
| 1202 | − return PTR_ERR(vmw_bo); | |
| 1202 | + return ret; | |
| 1203 | 1203 | } |
| 1204 | 1204 | vmw_bo_placement_set(vmw_bo, VMW_BO_DOMAIN_GMR | VMW_BO_DOMAIN_VRAM, |
| 1205 | 1205 | VMW_BO_DOMAIN_GMR | VMW_BO_DOMAIN_VRAM); |
| @@ -260,6 +260,13 @@ out_no_dirty: · 259 unmodified lines | ||
| 260 | 260 | return ret; |
| 261 | 261 | } |
| 262 | 262 | |
| 263 | +static void vmw_bo_dirty_free(struct kref *kref) | |
| 264 | +{ | |
| 265 | + struct vmw_bo_dirty *dirty = container_of(kref, struct vmw_bo_dirty, ref_count); | |
| 266 | + | |
| 267 | + kvfree(dirty); | |
| 268 | +} | |
| 269 | + | |
| 263 | 270 | /** |
| 264 | 271 | * vmw_bo_dirty_release - Release a dirty-tracking user from a buffer object |
| 265 | 272 | * @vbo: The buffer object |
| @@ -274,7 +281,7 @@ void vmw_bo_dirty_release(struct vmw_bo *vbo) · 8 unmodified lines | ||
| 274 | 281 | { |
| 275 | 282 | struct vmw_bo_dirty *dirty = vbo->dirty; |
| 276 | 283 | |
| 277 | − if (dirty && kref_put(&dirty->ref_count, (void *)kvfree)) | |
| 284 | + if (dirty && kref_put(&dirty->ref_count, vmw_bo_dirty_free)) | |
| 278 | 285 | vbo->dirty = NULL; |
| 279 | 286 | } |
| 280 | 287 | |
| @@ -96,6 +96,12 @@ · 95 unmodified lines | ||
| 96 | 96 | #define ENABLE_SEMAPHORE_POLL_BIT REG_BIT(13) |
| 97 | 97 | |
| 98 | 98 | #define RING_CMD_CCTL(base) XE_REG((base) + 0xc4, XE_REG_OPTION_MASKED) |
| 99 | + | |
| 100 | +#define CS_MMIO_GROUP_INSTANCE_SELECT(base) XE_REG((base) + 0xcc) | |
| 101 | +#define SELECTIVE_READ_ADDRESSING REG_BIT(30) | |
| 102 | +#define SELECTIVE_READ_GROUP REG_GENMASK(29, 23) | |
| 103 | +#define SELECTIVE_READ_INSTANCE REG_GENMASK(22, 16) | |
| 104 | + | |
| 99 | 105 | /* |
| 100 | 106 | * CMD_CCTL read/write fields take a MOCS value and _not_ a table index. |
| 101 | 107 | * The lsb of each can be considered a separate enabling bit for encryption. |
| @@ -830,6 +830,7 @@ static void xe_config_device_release(struct config_item *item) · 829 unmodified lines | ||
| 830 | 830 | |
| 831 | 831 | mutex_destroy(&dev->lock); |
| 832 | 832 | |
| 833 | + kfree(dev->config.ctx_restore_mid_bb[0].cs); | |
| 833 | 834 | kfree(dev->config.ctx_restore_post_bb[0].cs); |
| 834 | 835 | kfree(dev); |
| 835 | 836 | } |
| @@ -266,6 +266,16 @@ static struct xe_exec_queue *__xe_exec_queue_alloc(struct xe_device *xe, · 265 unmodified lines | ||
| 266 | 266 | return q; |
| 267 | 267 | } |
| 268 | 268 | |
| 269 | +static void __xe_exec_queue_fini(struct xe_exec_queue *q) | |
| 270 | +{ | |
| 271 | + int i; | |
| 272 | + | |
| 273 | + q->ops->fini(q); | |
| 274 | + | |
| 275 | + for (i = 0; i < q->width; ++i) | |
| 276 | + xe_lrc_put(q->lrc[i]); | |
| 277 | +} | |
| 278 | + | |
| 269 | 279 | static int __xe_exec_queue_init(struct xe_exec_queue *q, u32 exec_queue_flags) |
| 270 | 280 | { |
| 271 | 281 | int i, err; |
| @@ -320,21 +330,10 @@ static int __xe_exec_queue_init(struct xe_exec_queue *q, u32 exec_queue_flags) · 48 unmodified lines | ||
| 320 | 330 | return 0; |
| 321 | 331 | |
| 322 | 332 | err_lrc: |
| 323 | − for (i = i - 1; i >= 0; --i) | |
| 324 | − xe_lrc_put(q->lrc[i]); | |
| 333 | + __xe_exec_queue_fini(q); | |
| 325 | 334 | return err; |
| 326 | 335 | } |
| 327 | 336 | |
| 328 | −static void __xe_exec_queue_fini(struct xe_exec_queue *q) | |
| 329 | −{ | |
| 330 | − int i; | |
| 331 | − | |
| 332 | − q->ops->fini(q); | |
| 333 | − | |
| 334 | − for (i = 0; i < q->width; ++i) | |
| 335 | − xe_lrc_put(q->lrc[i]); | |
| 336 | −} | |
| 337 | − | |
| 338 | 337 | struct xe_exec_queue *xe_exec_queue_create(struct xe_device *xe, struct xe_vm *vm, |
| 339 | 338 | u32 logical_mask, u16 width, |
| 340 | 339 | struct xe_hw_engine *hwe, u32 flags, |
| @@ -435,15 +435,11 @@ static int proxy_channel_alloc(struct xe_gsc *gsc) · 434 unmodified lines | ||
| 435 | 435 | return 0; |
| 436 | 436 | } |
| 437 | 437 | |
| 438 | −static void xe_gsc_proxy_remove(void *arg) | |
| 438 | +static void xe_gsc_proxy_stop(struct xe_gsc *gsc) | |
| 439 | 439 | { |
| 440 | − struct xe_gsc *gsc = arg; | |
| 441 | 440 | struct xe_gt *gt = gsc_to_gt(gsc); |
| 442 | 441 | struct xe_device *xe = gt_to_xe(gt); |
| 443 | 442 | |
| 444 | − if (!gsc->proxy.component_added) | |
| 445 | − return; | |
| 446 | − | |
| 447 | 443 | /* disable HECI2 IRQs */ |
| 448 | 444 | scoped_guard(xe_pm_runtime, xe) { |
| 449 | 445 | CLASS(xe_force_wake, fw_ref)(gt_to_fw(gt), XE_FW_GSC); |
| @@ -455,6 +451,30 @@ static void xe_gsc_proxy_remove(void *arg) · 5 unmodified lines | ||
| 455 | 451 | } |
| 456 | 452 | |
| 457 | 453 | xe_gsc_wait_for_worker_completion(gsc); |
| 454 | + gsc->proxy.started = false; | |
| 455 | +} | |
| 456 | + | |
| 457 | +static void xe_gsc_proxy_remove(void *arg) | |
| 458 | +{ | |
| 459 | + struct xe_gsc *gsc = arg; | |
| 460 | + struct xe_gt *gt = gsc_to_gt(gsc); | |
| 461 | + struct xe_device *xe = gt_to_xe(gt); | |
| 462 | + | |
| 463 | + if (!gsc->proxy.component_added) | |
| 464 | + return; | |
| 465 | + | |
| 466 | + /* | |
| 467 | + * GSC proxy start is an async process that can be ongoing during | |
| 468 | + * Xe module load/unload. Using devm managed action to register | |
| 469 | + * xe_gsc_proxy_stop could cause issues if Xe module unload has | |
| 470 | + * already started when the action is registered, potentially leading | |
| 471 | + * to the cleanup being called at the wrong time. Therefore, instead | |
| 472 | + * of registering a separate devm action to undo what is done in | |
| 473 | + * proxy start, we call it from here, but only if the start has | |
| 474 | + * completed successfully (tracked with the 'started' flag). | |
| 475 | + */ | |
| 476 | + if (gsc->proxy.started) | |
| 477 | + xe_gsc_proxy_stop(gsc); | |
| 458 | 478 | |
| 459 | 479 | component_del(xe->drm.dev, &xe_gsc_proxy_component_ops); |
| 460 | 480 | gsc->proxy.component_added = false; |
| @@ -510,6 +530,7 @@ int xe_gsc_proxy_init(struct xe_gsc *gsc) · 49 unmodified lines | ||
| 510 | 530 | */ |
| 511 | 531 | int xe_gsc_proxy_start(struct xe_gsc *gsc) |
| 512 | 532 | { |
| 533 | + struct xe_gt *gt = gsc_to_gt(gsc); | |
| 513 | 534 | int err; |
| 514 | 535 | |
| 515 | 536 | /* enable the proxy interrupt in the GSC shim layer */ |
| @@ -521,12 +542,18 @@ int xe_gsc_proxy_start(struct xe_gsc *gsc) · 5 unmodified lines | ||
| 521 | 542 | */ |
| 522 | 543 | err = xe_gsc_proxy_request_handler(gsc); |
| 523 | 544 | if (err) |
| 524 | − return err; | |
| 545 | + goto err_irq_disable; | |
| 525 | 546 | |
| 526 | 547 | if (!xe_gsc_proxy_init_done(gsc)) { |
| 527 | − xe_gt_err(gsc_to_gt(gsc), "GSC FW reports proxy init not completed\n"); | |
| 528 | − return -EIO; | |
| 548 | + xe_gt_err(gt, "GSC FW reports proxy init not completed\n"); | |
| 549 | + err = -EIO; | |
| 550 | + goto err_irq_disable; | |
| 529 | 551 | } |
| 530 | 552 | |
| 553 | + gsc->proxy.started = true; | |
| 531 | 554 | return 0; |
| 555 | + | |
| 556 | +err_irq_disable: | |
| 557 | + gsc_proxy_irq_toggle(gsc, false); | |
| 558 | + return err; | |
| 532 | 559 | } |
| @@ -58,6 +58,8 @@ struct xe_gsc { · 57 unmodified lines | ||
| 58 | 58 | struct mutex mutex; |
| 59 | 59 | /** @proxy.component_added: whether the component has been added */ |
| 60 | 60 | bool component_added; |
| 61 | + /** @proxy.started: whether the proxy has been started */ | |
| 62 | + bool started; | |
| 61 | 63 | /** @proxy.bo: object to store message to and from the GSC */ |
| 62 | 64 | struct xe_bo *bo; |
| 63 | 65 | /** @proxy.to_gsc: map of the memory used to send messages to the GSC */ |
| @@ -210,11 +210,15 @@ static int emit_nop_job(struct xe_gt *gt, struct xe_exec_queue *q) · 209 unmodified lines | ||
| 210 | 210 | return ret; |
| 211 | 211 | } |
| 212 | 212 | |
| 213 | +/* Dwords required to emit a RMW of a register */ | |
| 214 | +#define EMIT_RMW_DW 20 | |
| 215 | + | |
| 213 | 216 | static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) |
| 214 | 217 | { |
| 215 | − struct xe_reg_sr *sr = &q->hwe->reg_lrc; | |
| 218 | + struct xe_hw_engine *hwe = q->hwe; | |
| 219 | + struct xe_reg_sr *sr = &hwe->reg_lrc; | |
| 216 | 220 | struct xe_reg_sr_entry *entry; |
| 217 | − int count_rmw = 0, count = 0, ret; | |
| 221 | + int count_rmw = 0, count_rmw_mcr = 0, count = 0, ret; | |
| 218 | 222 | unsigned long idx; |
| 219 | 223 | struct xe_bb *bb; |
| 220 | 224 | size_t bb_len = 0; |
| @@ -224,6 +228,8 @@ static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) · 3 unmodified lines | ||
| 224 | 228 | xa_for_each(&sr->xa, idx, entry) { |
| 225 | 229 | if (entry->reg.masked || entry->clr_bits == ~0) |
| 226 | 230 | ++count; |
| 231 | + else if (entry->reg.mcr) | |
| 232 | + ++count_rmw_mcr; | |
| 227 | 233 | else |
| 228 | 234 | ++count_rmw; |
| 229 | 235 | } |
| @@ -231,17 +237,35 @@ static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) · 1 unmodified line | ||
| 231 | 237 | if (count) |
| 232 | 238 | bb_len += count * 2 + 1; |
| 233 | 239 | |
| 234 | − if (count_rmw) | |
| 235 | − bb_len += count_rmw * 20 + 7; | |
| 240 | + /* | |
| 241 | + * RMW of MCR registers is the same as a normal RMW, except an | |
| 242 | + * additional LRI (3 dwords) is required per register to steer the read | |
| 243 | + * to a nom-terminated instance. | |
| 244 | + * | |
| 245 | + * We could probably shorten the batch slightly by eliding the | |
| 246 | + * steering for consecutive MCR registers that have the same | |
| 247 | + * group/instance target, but it's not worth the extra complexity to do | |
| 248 | + * so. | |
| 249 | + */ | |
| 250 | + bb_len += count_rmw * EMIT_RMW_DW; | |
| 251 | + bb_len += count_rmw_mcr * (EMIT_RMW_DW + 3); | |
| 252 | + | |
| 253 | + /* | |
| 254 | + * After doing all RMW, we need 7 trailing dwords to clean up, | |
| 255 | + * plus an additional 3 dwords to reset steering if any of the | |
| 256 | + * registers were MCR. | |
| 257 | + */ | |
| 258 | + if (count_rmw || count_rmw_mcr) | |
| 259 | + bb_len += 7 + (count_rmw_mcr ? 3 : 0); | |
| 236 | 260 | |
| 237 | − if (q->hwe->class == XE_ENGINE_CLASS_RENDER) | |
| 261 | + if (hwe->class == XE_ENGINE_CLASS_RENDER) | |
| 238 | 262 | /* |
| 239 | 263 | * Big enough to emit all of the context's 3DSTATE via |
| 240 | 264 | * xe_lrc_emit_hwe_state_instructions() |
| 241 | 265 | */ |
| 242 | − bb_len += xe_gt_lrc_size(gt, q->hwe->class) / sizeof(u32); | |
| 266 | + bb_len += xe_gt_lrc_size(gt, hwe->class) / sizeof(u32); | |
| 243 | 267 | |
| 244 | − xe_gt_dbg(gt, "LRC %s WA job: %zu dwords\n", q->hwe->name, bb_len); | |
| 268 | + xe_gt_dbg(gt, "LRC %s WA job: %zu dwords\n", hwe->name, bb_len); | |
| 245 | 269 | |
| 246 | 270 | bb = xe_bb_new(gt, bb_len, false); |
| 247 | 271 | if (IS_ERR(bb)) |
| @@ -276,13 +300,23 @@ static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) · 28 unmodified lines | ||
| 276 | 300 | } |
| 277 | 301 | } |
| 278 | 302 | |
| 279 | − if (count_rmw) { | |
| 280 | − /* Emit MI_MATH for each RMW reg: 20dw per reg + 7 trailing dw */ | |
| 281 | − | |
| 303 | + if (count_rmw || count_rmw_mcr) { | |
| 282 | 304 | xa_for_each(&sr->xa, idx, entry) { |
| 283 | 305 | if (entry->reg.masked || entry->clr_bits == ~0) |
| 284 | 306 | continue; |
| 285 | 307 | |
| 308 | + if (entry->reg.mcr) { | |
| 309 | + struct xe_reg_mcr reg = { .__reg.raw = entry->reg.raw }; | |
| 310 | + u8 group, instance; | |
| 311 | + | |
| 312 | + xe_gt_mcr_get_nonterminated_steering(gt, reg, &group, &instance); | |
| 313 | + *cs++ = MI_LOAD_REGISTER_IMM | MI_LRI_NUM_REGS(1); | |
| 314 | + *cs++ = CS_MMIO_GROUP_INSTANCE_SELECT(hwe->mmio_base).addr; | |
| 315 | + *cs++ = SELECTIVE_READ_ADDRESSING | | |
| 316 | + REG_FIELD_PREP(SELECTIVE_READ_GROUP, group) | | |
| 317 | + REG_FIELD_PREP(SELECTIVE_READ_INSTANCE, instance); | |
| 318 | + } | |
| 319 | + | |
| 286 | 320 | *cs++ = MI_LOAD_REGISTER_REG | MI_LRR_DST_CS_MMIO; |
| 287 | 321 | *cs++ = entry->reg.addr; |
| 288 | 322 | *cs++ = CS_GPR_REG(0, 0).addr; |
| @@ -308,8 +342,9 @@ static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) · 19 unmodified lines | ||
| 308 | 342 | *cs++ = CS_GPR_REG(0, 0).addr; |
| 309 | 343 | *cs++ = entry->reg.addr; |
| 310 | 344 | |
| 311 | − xe_gt_dbg(gt, "REG[%#x] = ~%#x|%#x\n", | |
| 312 | − entry->reg.addr, entry->clr_bits, entry->set_bits); | |
| 345 | + xe_gt_dbg(gt, "REG[%#x] = ~%#x|%#x%s\n", | |
| 346 | + entry->reg.addr, entry->clr_bits, entry->set_bits, | |
| 347 | + entry->reg.mcr ? " (MCR)" : ""); | |
| 313 | 348 | } |
| 314 | 349 | |
| 315 | 350 | /* reset used GPR */ |
| @@ -321,6 +356,13 @@ static int emit_wa_job(struct xe_gt *gt, struct xe_exec_queue *q) · 5 unmodified lines | ||
| 321 | 356 | *cs++ = 0; |
| 322 | 357 | *cs++ = CS_GPR_REG(0, 2).addr; |
| 323 | 358 | *cs++ = 0; |
| 359 | + | |
| 360 | + /* reset steering */ | |
| 361 | + if (count_rmw_mcr) { | |
| 362 | + *cs++ = MI_LOAD_REGISTER_IMM | MI_LRI_NUM_REGS(1); | |
| 363 | + *cs++ = CS_MMIO_GROUP_INSTANCE_SELECT(q->hwe->mmio_base).addr; | |
| 364 | + *cs++ = 0; | |
| 365 | + } | |
| 324 | 366 | } |
| 325 | 367 | |
| 326 | 368 | cs = xe_lrc_emit_hwe_state_instructions(q, cs); |
| @@ -75,7 +75,8 @@ static inline struct xe_lrc *xe_lrc_get(struct xe_lrc *lrc) · 74 unmodified lines | ||
| 75 | 75 | */ |
| 76 | 76 | static inline void xe_lrc_put(struct xe_lrc *lrc) |
| 77 | 77 | { |
| 78 | − kref_put(&lrc->refcount, xe_lrc_destroy); | |
| 78 | + if (lrc) | |
| 79 | + kref_put(&lrc->refcount, xe_lrc_destroy); | |
| 79 | 80 | } |
| 80 | 81 | |
| 81 | 82 | /** |
| @@ -98,10 +98,12 @@ int xe_reg_sr_add(struct xe_reg_sr *sr, · 97 unmodified lines | ||
| 98 | 98 | *pentry = *e; |
| 99 | 99 | ret = xa_err(xa_store(&sr->xa, idx, pentry, GFP_KERNEL)); |
| 100 | 100 | if (ret) |
| 101 | − goto fail; | |
| 101 | + goto fail_free; | |
| 102 | 102 | |
| 103 | 103 | return 0; |
| 104 | 104 | |
| 105 | +fail_free: | |
| 106 | + kfree(pentry); | |
| 105 | 107 | fail: |
| 106 | 108 | xe_gt_err(gt, |
| 107 | 109 | "discarding save-restore reg %04lx (clear: %08x, set: %08x, masked: %s, mcr: %s): ret=%d\n", |
| @@ -280,6 +280,9 @@ static void __emit_job_gen12_simple(struct xe_sched_job *job, struct xe_lrc *lrc · 279 unmodified lines | ||
| 280 | 280 | |
| 281 | 281 | i = emit_bb_start(batch_addr, ppgtt_flag, dw, i); |
| 282 | 282 | |
| 283 | + /* Don't preempt fence signaling */ | |
| 284 | + dw[i++] = MI_ARB_ON_OFF | MI_ARB_DISABLE; | |
| 285 | + | |
| 283 | 286 | if (job->user_fence.used) { |
| 284 | 287 | i = emit_flush_dw(dw, i); |
| 285 | 288 | i = emit_store_imm_ppgtt_posted(job->user_fence.addr, |
| @@ -345,6 +348,9 @@ static void __emit_job_gen12_video(struct xe_sched_job *job, struct xe_lrc *lrc, · 59 unmodified lines | ||
| 345 | 348 | |
| 346 | 349 | i = emit_bb_start(batch_addr, ppgtt_flag, dw, i); |
| 347 | 350 | |
| 351 | + /* Don't preempt fence signaling */ | |
| 352 | + dw[i++] = MI_ARB_ON_OFF | MI_ARB_DISABLE; | |
| 353 | + | |
| 348 | 354 | if (job->user_fence.used) { |
| 349 | 355 | i = emit_flush_dw(dw, i); |
| 350 | 356 | i = emit_store_imm_ppgtt_posted(job->user_fence.addr, |
| @@ -397,6 +403,9 @@ static void __emit_job_gen12_render_compute(struct xe_sched_job *job, · 46 unmodified lines | ||
| 397 | 403 | |
| 398 | 404 | i = emit_bb_start(batch_addr, ppgtt_flag, dw, i); |
| 399 | 405 | |
| 406 | + /* Don't preempt fence signaling */ | |
| 407 | + dw[i++] = MI_ARB_ON_OFF | MI_ARB_DISABLE; | |
| 408 | + | |
| 400 | 409 | i = emit_render_cache_flush(job, dw, i); |
| 401 | 410 | |
| 402 | 411 | if (job->user_fence.used) |
| @@ -146,8 +146,10 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef, · 145 unmodified lines | ||
| 146 | 146 | |
| 147 | 147 | if (!signal) { |
| 148 | 148 | sync->fence = drm_syncobj_fence_get(sync->syncobj); |
| 149 | − if (XE_IOCTL_DBG(xe, !sync->fence)) | |
| 150 | − return -EINVAL; | |
| 149 | + if (XE_IOCTL_DBG(xe, !sync->fence)) { | |
| 150 | + err = -EINVAL; | |
| 151 | + goto free_sync; | |
| 152 | + } | |
| 151 | 153 | } |
| 152 | 154 | break; |
| 153 | 155 | |
| @@ -167,17 +169,21 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef, · 13 unmodified lines | ||
| 167 | 169 | |
| 168 | 170 | if (signal) { |
| 169 | 171 | sync->chain_fence = dma_fence_chain_alloc(); |
| 170 | − if (!sync->chain_fence) | |
| 171 | − return -ENOMEM; | |
| 172 | + if (!sync->chain_fence) { | |
| 173 | + err = -ENOMEM; | |
| 174 | + goto free_sync; | |
| 175 | + } | |
| 172 | 176 | } else { |
| 173 | 177 | sync->fence = drm_syncobj_fence_get(sync->syncobj); |
| 174 | − if (XE_IOCTL_DBG(xe, !sync->fence)) | |
| 175 | − return -EINVAL; | |
| 178 | + if (XE_IOCTL_DBG(xe, !sync->fence)) { | |
| 179 | + err = -EINVAL; | |
| 180 | + goto free_sync; | |
| 181 | + } | |
| 176 | 182 | |
| 177 | 183 | err = dma_fence_chain_find_seqno(&sync->fence, |
| 178 | 184 | sync_in.timeline_value); |
| 179 | 185 | if (err) |
| 180 | − return err; | |
| 186 | + goto free_sync; | |
| 181 | 187 | } |
| 182 | 188 | break; |
| 183 | 189 | |
| @@ -200,8 +206,10 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef, · 16 unmodified lines | ||
| 200 | 206 | if (XE_IOCTL_DBG(xe, IS_ERR(sync->ufence))) |
| 201 | 207 | return PTR_ERR(sync->ufence); |
| 202 | 208 | sync->ufence_chain_fence = dma_fence_chain_alloc(); |
| 203 | − if (!sync->ufence_chain_fence) | |
| 204 | − return -ENOMEM; | |
| 209 | + if (!sync->ufence_chain_fence) { | |
| 210 | + err = -ENOMEM; | |
| 211 | + goto free_sync; | |
| 212 | + } | |
| 205 | 213 | sync->ufence_syncobj = ufence_syncobj; |
| 206 | 214 | } |
| 207 | 215 | |
| @@ -216,6 +224,10 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef, · 8 unmodified lines | ||
| 216 | 224 | sync->timeline_value = sync_in.timeline_value; |
| 217 | 225 | |
| 218 | 226 | return 0; |
| 227 | + | |
| 228 | +free_sync: | |
| 229 | + xe_sync_entry_cleanup(sync); | |
| 230 | + return err; | |
| 219 | 231 | } |
| 220 | 232 | ALLOW_ERROR_INJECTION(xe_sync_entry_parse, ERRNO); |
| 221 | 233 | |
| @@ -453,7 +453,7 @@ int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · 452 unmodified lines | ||
| 453 | 453 | madvise_range.num_vmas, |
| 454 | 454 | args->atomic.val)) { |
| 455 | 455 | err = -EINVAL; |
| 456 | − goto madv_fini; | |
| 456 | + goto free_vmas; | |
| 457 | 457 | } |
| 458 | 458 | } |
| 459 | 459 | |
| @@ -490,6 +490,7 @@ int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · 30 unmodified lines | ||
| 490 | 490 | err_fini: |
| 491 | 491 | if (madvise_range.has_bo_vmas) |
| 492 | 492 | drm_exec_fini(&exec); |
| 493 | +free_vmas: | |
| 493 | 494 | kfree(madvise_range.vmas); |
| 494 | 495 | madvise_range.vmas = NULL; |
| 495 | 496 | madv_fini: |
| @@ -241,12 +241,13 @@ static const struct xe_rtp_entry_sr gt_was[] = { · 240 unmodified lines | ||
| 241 | 241 | |
| 242 | 242 | { XE_RTP_NAME("16025250150"), |
| 243 | 243 | XE_RTP_RULES(GRAPHICS_VERSION(2001)), |
| 244 | − XE_RTP_ACTIONS(SET(LSN_VC_REG2, | |
| 245 | − LSN_LNI_WGT(1) | | |
| 246 | − LSN_LNE_WGT(1) | | |
| 247 | − LSN_DIM_X_WGT(1) | | |
| 248 | − LSN_DIM_Y_WGT(1) | | |
| 249 | − LSN_DIM_Z_WGT(1))) | |
| 244 | + XE_RTP_ACTIONS(FIELD_SET(LSN_VC_REG2, | |
| 245 | + LSN_LNI_WGT_MASK | LSN_LNE_WGT_MASK | | |
| 246 | + LSN_DIM_X_WGT_MASK | LSN_DIM_Y_WGT_MASK | | |
| 247 | + LSN_DIM_Z_WGT_MASK, | |
| 248 | + LSN_LNI_WGT(1) | LSN_LNE_WGT(1) | | |
| 249 | + LSN_DIM_X_WGT(1) | LSN_DIM_Y_WGT(1) | | |
| 250 | + LSN_DIM_Z_WGT(1))) | |
| 250 | 251 | }, |
| 251 | 252 | |
| 252 | 253 | /* Xe2_HPM */ |
| @@ -47,16 +47,12 @@ struct PteArray<const NUM_ENTRIES: usize>([u64; NUM_ENTRIES]); · 46 unmodified lines | ||
| 47 | 47 | unsafe impl<const NUM_ENTRIES: usize> AsBytes for PteArray<NUM_ENTRIES> {} |
| 48 | 48 | |
| 49 | 49 | impl<const NUM_PAGES: usize> PteArray<NUM_PAGES> { |
| 50 | − /// Creates a new page table array mapping `NUM_PAGES` GSP pages starting at address `start`. | |
| 51 | − fn new(start: DmaAddress) -> Result<Self> { | |
| 52 | − let mut ptes = [0u64; NUM_PAGES]; | |
| 53 | − for (i, pte) in ptes.iter_mut().enumerate() { | |
| 54 | − *pte = start | |
| 55 | − .checked_add(num::usize_as_u64(i) << GSP_PAGE_SHIFT) | |
| 56 | − .ok_or(EOVERFLOW)?; | |
| 57 | − } | |
| 58 | − | |
| 59 | − Ok(Self(ptes)) | |
| 50 | + /// Returns the page table entry for `index`, for a mapping starting at `start`. | |
| 51 | + // TODO: Replace with `IoView` projection once available. | |
| 52 | + fn entry(start: DmaAddress, index: usize) -> Result<u64> { | |
| 53 | + start | |
| 54 | + .checked_add(num::usize_as_u64(index) << GSP_PAGE_SHIFT) | |
| 55 | + .ok_or(EOVERFLOW) | |
| 60 | 56 | } |
| 61 | 57 | } |
| 62 | 58 | |
| @@ -86,16 +82,22 @@ impl LogBuffer { · 23 unmodified lines | ||
| 86 | 82 | NUM_PAGES * GSP_PAGE_SIZE, |
| 87 | 83 | GFP_KERNEL | __GFP_ZERO, |
| 88 | 84 | )?); |
| 89 | − let ptes = PteArray::<NUM_PAGES>::new(obj.0.dma_handle())?; | |
| 85 | + | |
| 86 | + let start_addr = obj.0.dma_handle(); | |
| 90 | 87 | |
| 91 | 88 | // SAFETY: `obj` has just been created and we are its sole user. |
| 92 | − unsafe { | |
| 93 | − // Copy the self-mapping PTE at the expected location. | |
| 89 | + let pte_region = unsafe { | |
| 94 | 90 | obj.0 |
| 95 | − .as_slice_mut(size_of::<u64>(), size_of_val(&ptes))? | |
| 96 | − .copy_from_slice(ptes.as_bytes()) | |
| 91 | + .as_slice_mut(size_of::<u64>(), NUM_PAGES * size_of::<u64>())? | |
| 97 | 92 | }; |
| 98 | 93 | |
| 94 | + // Write values one by one to avoid an on-stack instance of `PteArray`. | |
| 95 | + for (i, chunk) in pte_region.chunks_exact_mut(size_of::<u64>()).enumerate() { | |
| 96 | + let pte_value = PteArray::<0>::entry(start_addr, i)?; | |
| 97 | + | |
| 98 | + chunk.copy_from_slice(&pte_value.to_ne_bytes()); | |
| 99 | + } | |
| 100 | + | |
| 99 | 101 | Ok(obj) |
| 100 | 102 | } |
| 101 | 103 | } |
| @@ -143,14 +145,14 @@ impl Gsp { · 41 unmodified lines | ||
| 143 | 145 | // _kgspInitLibosLoggingStructures (allocates memory for buffers) |
| 144 | 146 | // kgspSetupLibosInitArgs_IMPL (creates pLibosInitArgs[] array) |
| 145 | 147 | dma_write!( |
| 146 | − libos[0] = LibosMemoryRegionInitArgument::new("LOGINIT", &loginit.0) | |
| 147 | − )?; | |
| 148 | + libos, [0]?, LibosMemoryRegionInitArgument::new("LOGINIT", &loginit.0) | |
| 149 | + ); | |
| 148 | 150 | dma_write!( |
| 149 | − libos[1] = LibosMemoryRegionInitArgument::new("LOGINTR", &logintr.0) | |
| 150 | − )?; | |
| 151 | − dma_write!(libos[2] = LibosMemoryRegionInitArgument::new("LOGRM", &logrm.0))?; | |
| 152 | − dma_write!(rmargs[0].inner = fw::GspArgumentsCached::new(cmdq))?; | |
| 153 | − dma_write!(libos[3] = LibosMemoryRegionInitArgument::new("RMARGS", rmargs))?; | |
| 151 | + libos, [1]?, LibosMemoryRegionInitArgument::new("LOGINTR", &logintr.0) | |
| 152 | + ); | |
| 153 | + dma_write!(libos, [2]?, LibosMemoryRegionInitArgument::new("LOGRM", &logrm.0)); | |
| 154 | + dma_write!(rmargs, [0]?.inner, fw::GspArgumentsCached::new(cmdq)); | |
| 155 | + dma_write!(libos, [3]?, LibosMemoryRegionInitArgument::new("RMARGS", rmargs)); | |
| 154 | 156 | }, |
| 155 | 157 | })) |
| 156 | 158 | }) |
| @@ -157,7 +157,7 @@ impl super::Gsp { · 156 unmodified lines | ||
| 157 | 157 | |
| 158 | 158 | let wpr_meta = |
| 159 | 159 | CoherentAllocation::<GspFwWprMeta>::alloc_coherent(dev, 1, GFP_KERNEL | __GFP_ZERO)?; |
| 160 | − dma_write!(wpr_meta[0] = GspFwWprMeta::new(&gsp_fw, &fb_layout))?; | |
| 160 | + dma_write!(wpr_meta, [0]?, GspFwWprMeta::new(&gsp_fw, &fb_layout)); | |
| 161 | 161 | |
| 162 | 162 | self.cmdq |
| 163 | 163 | .send_command(bar, commands::SetSystemInfo::new(pdev))?; |
| @@ -2,11 +2,7 @@ · 1 unmodified line | ||
| 2 | 2 | |
| 3 | 3 | use core::{ |
| 4 | 4 | cmp, |
| 5 | − mem, | |
| 6 | − sync::atomic::{ | |
| 7 | − fence, | |
| 8 | − Ordering, // | |
| 9 | − }, // | |
| 5 | + mem, // | |
| 10 | 6 | }; |
| 11 | 7 | |
| 12 | 8 | use kernel::{ |
| @@ -146,30 +142,36 @@ static_assert!(align_of::<MsgqData>() == GSP_PAGE_SIZE); · 133 unmodified lines | ||
| 146 | 142 | #[repr(C)] |
| 147 | 143 | // There is no struct defined for this in the open-gpu-kernel-source headers. |
| 148 | 144 | // Instead it is defined by code in `GspMsgQueuesInit()`. |
| 149 | −struct Msgq { | |
| 145 | +// TODO: Revert to private once `IoView` projections replace the `gsp_mem` module. | |
| 146 | +pub(super) struct Msgq { | |
| 150 | 147 | /// Header for sending messages, including the write pointer. |
| 151 | − tx: MsgqTxHeader, | |
| 148 | + pub(super) tx: MsgqTxHeader, | |
| 152 | 149 | /// Header for receiving messages, including the read pointer. |
| 153 | − rx: MsgqRxHeader, | |
| 150 | + pub(super) rx: MsgqRxHeader, | |
| 154 | 151 | /// The message queue proper. |
| 155 | 152 | msgq: MsgqData, |
| 156 | 153 | } |
| 157 | 154 | |
| 158 | 155 | /// Structure shared between the driver and the GSP and containing the command and message queues. |
| 159 | 156 | #[repr(C)] |
| 160 | −struct GspMem { | |
| 157 | +// TODO: Revert to private once `IoView` projections replace the `gsp_mem` module. | |
| 158 | +pub(super) struct GspMem { | |
| 161 | 159 | /// Self-mapping page table entries. |
| 162 | − ptes: PteArray<{ GSP_PAGE_SIZE / size_of::<u64>() }>, | |
| 160 | + ptes: PteArray<{ Self::PTE_ARRAY_SIZE }>, | |
| 163 | 161 | /// CPU queue: the driver writes commands here, and the GSP reads them. It also contains the |
| 164 | 162 | /// write and read pointers that the CPU updates. |
| 165 | 163 | /// |
| 166 | 164 | /// This member is read-only for the GSP. |
| 167 | − cpuq: Msgq, | |
| 165 | + pub(super) cpuq: Msgq, | |
| 168 | 166 | /// GSP queue: the GSP writes messages here, and the driver reads them. It also contains the |
| 169 | 167 | /// write and read pointers that the GSP updates. |
| 170 | 168 | /// |
| 171 | 169 | /// This member is read-only for the driver. |
| 172 | − gspq: Msgq, | |
| 170 | + pub(super) gspq: Msgq, | |
| 171 | +} | |
| 172 | + | |
| 173 | +impl GspMem { | |
| 174 | + const PTE_ARRAY_SIZE: usize = GSP_PAGE_SIZE / size_of::<u64>(); | |
| 173 | 175 | } |
| 174 | 176 | |
| 175 | 177 | // SAFETY: These structs don't meet the no-padding requirements of AsBytes but |
| @@ -201,9 +203,19 @@ impl DmaGspMem { · 25 unmodified lines | ||
| 201 | 203 | |
| 202 | 204 | let gsp_mem = |
| 203 | 205 | CoherentAllocation::<GspMem>::alloc_coherent(dev, 1, GFP_KERNEL | __GFP_ZERO)?; |
| 204 | − dma_write!(gsp_mem[0].ptes = PteArray::new(gsp_mem.dma_handle())?)?; | |
| 205 | − dma_write!(gsp_mem[0].cpuq.tx = MsgqTxHeader::new(MSGQ_SIZE, RX_HDR_OFF, MSGQ_NUM_PAGES))?; | |
| 206 | − dma_write!(gsp_mem[0].cpuq.rx = MsgqRxHeader::new())?; | |
| 206 | + | |
| 207 | + let start = gsp_mem.dma_handle(); | |
| 208 | + // Write values one by one to avoid an on-stack instance of `PteArray`. | |
| 209 | + for i in 0..GspMem::PTE_ARRAY_SIZE { | |
| 210 | + dma_write!(gsp_mem, [0]?.ptes.0[i], PteArray::<0>::entry(start, i)?); | |
| 211 | + } | |
| 212 | + | |
| 213 | + dma_write!( | |
| 214 | + gsp_mem, | |
| 215 | + [0]?.cpuq.tx, | |
| 216 | + MsgqTxHeader::new(MSGQ_SIZE, RX_HDR_OFF, MSGQ_NUM_PAGES) | |
| 217 | + ); | |
| 218 | + dma_write!(gsp_mem, [0]?.cpuq.rx, MsgqRxHeader::new()); | |
| 207 | 219 | |
| 208 | 220 | Ok(Self(gsp_mem)) |
| 209 | 221 | } |
| @@ -317,12 +329,7 @@ impl DmaGspMem { · 107 unmodified lines | ||
| 317 | 329 | // |
| 318 | 330 | // - The returned value is between `0` and `MSGQ_NUM_PAGES`. |
| 319 | 331 | fn gsp_write_ptr(&self) -> u32 { |
| 320 | − let gsp_mem = self.0.start_ptr(); | |
| 321 | − | |
| 322 | − // SAFETY: | |
| 323 | − // - The 'CoherentAllocation' contains at least one object. | |
| 324 | − // - By the invariants of `CoherentAllocation` the pointer is valid. | |
| 325 | − (unsafe { (*gsp_mem).gspq.tx.write_ptr() } % MSGQ_NUM_PAGES) | |
| 332 | + super::fw::gsp_mem::gsp_write_ptr(&self.0) | |
| 326 | 333 | } |
| 327 | 334 | |
| 328 | 335 | // Returns the index of the memory page the GSP will read the next command from. |
| @@ -331,12 +338,7 @@ impl DmaGspMem { · 2 unmodified lines | ||
| 331 | 338 | // |
| 332 | 339 | // - The returned value is between `0` and `MSGQ_NUM_PAGES`. |
| 333 | 340 | fn gsp_read_ptr(&self) -> u32 { |
| 334 | − let gsp_mem = self.0.start_ptr(); | |
| 335 | − | |
| 336 | − // SAFETY: | |
| 337 | − // - The 'CoherentAllocation' contains at least one object. | |
| 338 | − // - By the invariants of `CoherentAllocation` the pointer is valid. | |
| 339 | − (unsafe { (*gsp_mem).gspq.rx.read_ptr() } % MSGQ_NUM_PAGES) | |
| 341 | + super::fw::gsp_mem::gsp_read_ptr(&self.0) | |
| 340 | 342 | } |
| 341 | 343 | |
| 342 | 344 | // Returns the index of the memory page the CPU can read the next message from. |
| @@ -345,27 +347,12 @@ impl DmaGspMem { · 2 unmodified lines | ||
| 345 | 347 | // |
| 346 | 348 | // - The returned value is between `0` and `MSGQ_NUM_PAGES`. |
| 347 | 349 | fn cpu_read_ptr(&self) -> u32 { |
| 348 | − let gsp_mem = self.0.start_ptr(); | |
| 349 | − | |
| 350 | − // SAFETY: | |
| 351 | − // - The ['CoherentAllocation'] contains at least one object. | |
| 352 | − // - By the invariants of CoherentAllocation the pointer is valid. | |
| 353 | − (unsafe { (*gsp_mem).cpuq.rx.read_ptr() } % MSGQ_NUM_PAGES) | |
| 350 | + super::fw::gsp_mem::cpu_read_ptr(&self.0) | |
| 354 | 351 | } |
| 355 | 352 | |
| 356 | 353 | // Informs the GSP that it can send `elem_count` new pages into the message queue. |
| 357 | 354 | fn advance_cpu_read_ptr(&mut self, elem_count: u32) { |
| 358 | − let rptr = self.cpu_read_ptr().wrapping_add(elem_count) % MSGQ_NUM_PAGES; | |
| 359 | − | |
| 360 | − // Ensure read pointer is properly ordered. | |
| 361 | − fence(Ordering::SeqCst); | |
| 362 | − | |
| 363 | − let gsp_mem = self.0.start_ptr_mut(); | |
| 364 | − | |
| 365 | − // SAFETY: | |
| 366 | − // - The 'CoherentAllocation' contains at least one object. | |
| 367 | − // - By the invariants of `CoherentAllocation` the pointer is valid. | |
| 368 | − unsafe { (*gsp_mem).cpuq.rx.set_read_ptr(rptr) }; | |
| 355 | + super::fw::gsp_mem::advance_cpu_read_ptr(&self.0, elem_count) | |
| 369 | 356 | } |
| 370 | 357 | |
| 371 | 358 | // Returns the index of the memory page the CPU can write the next command to. |
| @@ -374,26 +361,12 @@ impl DmaGspMem { · 2 unmodified lines | ||
| 374 | 361 | // |
| 375 | 362 | // - The returned value is between `0` and `MSGQ_NUM_PAGES`. |
| 376 | 363 | fn cpu_write_ptr(&self) -> u32 { |
| 377 | − let gsp_mem = self.0.start_ptr(); | |
| 378 | − | |
| 379 | − // SAFETY: | |
| 380 | − // - The 'CoherentAllocation' contains at least one object. | |
| 381 | − // - By the invariants of `CoherentAllocation` the pointer is valid. | |
| 382 | − (unsafe { (*gsp_mem).cpuq.tx.write_ptr() } % MSGQ_NUM_PAGES) | |
| 364 | + super::fw::gsp_mem::cpu_write_ptr(&self.0) | |
| 383 | 365 | } |
| 384 | 366 | |
| 385 | 367 | // Informs the GSP that it can process `elem_count` new pages from the command queue. |
| 386 | 368 | fn advance_cpu_write_ptr(&mut self, elem_count: u32) { |
| 387 | − let wptr = self.cpu_write_ptr().wrapping_add(elem_count) & MSGQ_NUM_PAGES; | |
| 388 | − let gsp_mem = self.0.start_ptr_mut(); | |
| 389 | − | |
| 390 | − // SAFETY: | |
| 391 | − // - The 'CoherentAllocation' contains at least one object. | |
| 392 | − // - By the invariants of `CoherentAllocation` the pointer is valid. | |
| 393 | − unsafe { (*gsp_mem).cpuq.tx.set_write_ptr(wptr) }; | |
| 394 | − | |
| 395 | − // Ensure all command data is visible before triggering the GSP read. | |
| 396 | − fence(Ordering::SeqCst); | |
| 369 | + super::fw::gsp_mem::advance_cpu_write_ptr(&self.0, elem_count) | |
| 397 | 370 | } |
| 398 | 371 | } |
| 399 | 372 | |
| @@ -40,6 +40,75 @@ use crate::{ · 39 unmodified lines | ||
| 40 | 40 | }, |
| 41 | 41 | }; |
| 42 | 42 | |
| 43 | +// TODO: Replace with `IoView` projections once available; the `unwrap()` calls go away once we | |
| 44 | +// switch to the new `dma::Coherent` API. | |
| 45 | +pub(super) mod gsp_mem { | |
| 46 | + use core::sync::atomic::{ | |
| 47 | + fence, | |
| 48 | + Ordering, // | |
| 49 | + }; | |
| 50 | + | |
| 51 | + use kernel::{ | |
| 52 | + dma::CoherentAllocation, | |
| 53 | + dma_read, | |
| 54 | + dma_write, | |
| 55 | + prelude::*, // | |
| 56 | + }; | |
| 57 | + | |
| 58 | + use crate::gsp::cmdq::{ | |
| 59 | + GspMem, | |
| 60 | + MSGQ_NUM_PAGES, // | |
| 61 | + }; | |
| 62 | + | |
| 63 | + pub(in crate::gsp) fn gsp_write_ptr(qs: &CoherentAllocation<GspMem>) -> u32 { | |
| 64 | + // PANIC: A `dma::CoherentAllocation` always contains at least one element. | |
| 65 | + || -> Result<u32> { Ok(dma_read!(qs, [0]?.gspq.tx.0.writePtr) % MSGQ_NUM_PAGES) }().unwrap() | |
| 66 | + } | |
| 67 | + | |
| 68 | + pub(in crate::gsp) fn gsp_read_ptr(qs: &CoherentAllocation<GspMem>) -> u32 { | |
| 69 | + // PANIC: A `dma::CoherentAllocation` always contains at least one element. | |
| 70 | + || -> Result<u32> { Ok(dma_read!(qs, [0]?.gspq.rx.0.readPtr) % MSGQ_NUM_PAGES) }().unwrap() | |
| 71 | + } | |
| 72 | + | |
| 73 | + pub(in crate::gsp) fn cpu_read_ptr(qs: &CoherentAllocation<GspMem>) -> u32 { | |
| 74 | + // PANIC: A `dma::CoherentAllocation` always contains at least one element. | |
| 75 | + || -> Result<u32> { Ok(dma_read!(qs, [0]?.cpuq.rx.0.readPtr) % MSGQ_NUM_PAGES) }().unwrap() | |
| 76 | + } | |
| 77 | + | |
| 78 | + pub(in crate::gsp) fn advance_cpu_read_ptr(qs: &CoherentAllocation<GspMem>, count: u32) { | |
| 79 | + let rptr = cpu_read_ptr(qs).wrapping_add(count) % MSGQ_NUM_PAGES; | |
| 80 | + | |
| 81 | + // Ensure read pointer is properly ordered. | |
| 82 | + fence(Ordering::SeqCst); | |
| 83 | + | |
| 84 | + // PANIC: A `dma::CoherentAllocation` always contains at least one element. | |
| 85 | + || -> Result { | |
| 86 | + dma_write!(qs, [0]?.cpuq.rx.0.readPtr, rptr); | |
| 87 | + Ok(()) | |
| 88 | + }() | |
| 89 | + .unwrap() | |
| 90 | + } | |
| 91 | + | |
| 92 | + pub(in crate::gsp) fn cpu_write_ptr(qs: &CoherentAllocation<GspMem>) -> u32 { | |
| 93 | + // PANIC: A `dma::CoherentAllocation` always contains at least one element. | |
| 94 | + || -> Result<u32> { Ok(dma_read!(qs, [0]?.cpuq.tx.0.writePtr) % MSGQ_NUM_PAGES) }().unwrap() | |
| 95 | + } | |
| 96 | + | |
| 97 | + pub(in crate::gsp) fn advance_cpu_write_ptr(qs: &CoherentAllocation<GspMem>, count: u32) { | |
| 98 | + let wptr = cpu_write_ptr(qs).wrapping_add(count) % MSGQ_NUM_PAGES; | |
| 99 | + | |
| 100 | + // PANIC: A `dma::CoherentAllocation` always contains at least one element. | |
| 101 | + || -> Result { | |
| 102 | + dma_write!(qs, [0]?.cpuq.tx.0.writePtr, wptr); | |
| 103 | + Ok(()) | |
| 104 | + }() | |
| 105 | + .unwrap(); | |
| 106 | + | |
| 107 | + // Ensure all command data is visible before triggering the GSP read. | |
| 108 | + fence(Ordering::SeqCst); | |
| 109 | + } | |
| 110 | +} | |
| 111 | + | |
| 43 | 112 | /// Empty type to group methods related to heap parameters for running the GSP firmware. |
| 44 | 113 | enum GspFwHeapParams {} |
| 45 | 114 | |
| @@ -708,22 +777,6 @@ impl MsgqTxHeader { · 662 unmodified lines | ||
| 708 | 777 | entryOff: num::usize_into_u32::<GSP_PAGE_SIZE>(), |
| 709 | 778 | }) |
| 710 | 779 | } |
| 711 | − | |
| 712 | − /// Returns the value of the write pointer for this queue. | |
| 713 | − pub(crate) fn write_ptr(&self) -> u32 { | |
| 714 | − let ptr = core::ptr::from_ref(&self.0.writePtr); | |
| 715 | − | |
| 716 | − // SAFETY: `ptr` is a valid pointer to a `u32`. | |
| 717 | − unsafe { ptr.read_volatile() } | |
| 718 | − } | |
| 719 | − | |
| 720 | − /// Sets the value of the write pointer for this queue. | |
| 721 | − pub(crate) fn set_write_ptr(&mut self, val: u32) { | |
| 722 | − let ptr = core::ptr::from_mut(&mut self.0.writePtr); | |
| 723 | − | |
| 724 | − // SAFETY: `ptr` is a valid pointer to a `u32`. | |
| 725 | − unsafe { ptr.write_volatile(val) } | |
| 726 | − } | |
| 727 | 780 | } |
| 728 | 781 | |
| 729 | 782 | // SAFETY: Padding is explicit and does not contain uninitialized data. |
| @@ -739,22 +792,6 @@ impl MsgqRxHeader { · 9 unmodified lines | ||
| 739 | 792 | pub(crate) fn new() -> Self { |
| 740 | 793 | Self(Default::default()) |
| 741 | 794 | } |
| 742 | − | |
| 743 | − /// Returns the value of the read pointer for this queue. | |
| 744 | − pub(crate) fn read_ptr(&self) -> u32 { | |
| 745 | − let ptr = core::ptr::from_ref(&self.0.readPtr); | |
| 746 | − | |
| 747 | − // SAFETY: `ptr` is a valid pointer to a `u32`. | |
| 748 | − unsafe { ptr.read_volatile() } | |
| 749 | − } | |
| 750 | − | |
| 751 | − /// Sets the value of the read pointer for this queue. | |
| 752 | − pub(crate) fn set_read_ptr(&mut self, val: u32) { | |
| 753 | − let ptr = core::ptr::from_mut(&mut self.0.readPtr); | |
| 754 | − | |
| 755 | − // SAFETY: `ptr` is a valid pointer to a `u32`. | |
| 756 | − unsafe { ptr.write_volatile(val) } | |
| 757 | − } | |
| 758 | 795 | } |
| 759 | 796 | |
| 760 | 797 | // SAFETY: Padding is explicit and does not contain uninitialized data. |
| @@ -365,6 +365,9 @@ static const struct apple_non_apple_keyboard non_apple_keyboards[] = { · 364 unmodified lines | ||
| 365 | 365 | { "A3R" }, |
| 366 | 366 | { "hfd.cn" }, |
| 367 | 367 | { "WKB603" }, |
| 368 | + { "TH87" }, /* EPOMAKER TH87 BT mode */ | |
| 369 | + { "HFD Epomaker TH87" }, /* EPOMAKER TH87 USB mode */ | |
| 370 | + { "2.4G Wireless Receiver" }, /* EPOMAKER TH87 dongle */ | |
| 368 | 371 | }; |
| 369 | 372 | |
| 370 | 373 | static bool apple_is_non_apple_keyboard(struct hid_device *hdev) |
| @@ -686,9 +689,7 @@ static const __u8 *apple_report_fixup(struct hid_device *hdev, __u8 *rdesc, · 315 unmodified lines | ||
| 686 | 689 | hid_info(hdev, |
| 687 | 690 | "fixing up Magic Keyboard battery report descriptor\n"); |
| 688 | 691 | *rsize = *rsize - 1; |
| 689 | − rdesc = kmemdup(rdesc + 1, *rsize, GFP_KERNEL); | |
| 690 | − if (!rdesc) | |
| 691 | − return NULL; | |
| 692 | + rdesc = rdesc + 1; | |
| 692 | 693 | |
| 693 | 694 | rdesc[0] = 0x05; |
| 694 | 695 | rdesc[1] = 0x01; |
| @@ -1399,14 +1399,21 @@ static const __u8 *asus_report_fixup(struct hid_device *hdev, __u8 *rdesc, · 1398 unmodified lines | ||
| 1399 | 1399 | */ |
| 1400 | 1400 | if (*rsize == rsize_orig && |
| 1401 | 1401 | rdesc[offs] == 0x09 && rdesc[offs + 1] == 0x76) { |
| 1402 | − *rsize = rsize_orig + 1; | |
| 1403 | − rdesc = kmemdup(rdesc, *rsize, GFP_KERNEL); | |
| 1404 | − if (!rdesc) | |
| 1405 | − return NULL; | |
| 1402 | + __u8 *new_rdesc; | |
| 1403 | + | |
| 1404 | + new_rdesc = devm_kzalloc(&hdev->dev, rsize_orig + 1, | |
| 1405 | + GFP_KERNEL); | |
| 1406 | + if (!new_rdesc) | |
| 1407 | + return rdesc; | |
| 1406 | 1408 | |
| 1407 | 1409 | hid_info(hdev, "Fixing up %s keyb report descriptor\n", |
| 1408 | 1410 | drvdata->quirks & QUIRK_T100CHI ? |
| 1409 | 1411 | "T100CHI" : "T90CHI"); |
| 1412 | + | |
| 1413 | + memcpy(new_rdesc, rdesc, rsize_orig); | |
| 1414 | + *rsize = rsize_orig + 1; | |
| 1415 | + rdesc = new_rdesc; | |
| 1416 | + | |
| 1410 | 1417 | memmove(rdesc + offs + 4, rdesc + offs + 2, 12); |
| 1411 | 1418 | rdesc[offs] = 0x19; |
| 1412 | 1419 | rdesc[offs + 1] = 0x00; |
| @@ -1490,6 +1497,9 @@ static const struct hid_device_id asus_devices[] = { · 77 unmodified lines | ||
| 1490 | 1497 | { HID_USB_DEVICE(USB_VENDOR_ID_ASUSTEK, |
| 1491 | 1498 | USB_DEVICE_ID_ASUSTEK_ROG_NKEY_ALLY_X), |
| 1492 | 1499 | QUIRK_USE_KBD_BACKLIGHT | QUIRK_ROG_NKEY_KEYBOARD | QUIRK_ROG_ALLY_XPAD }, |
| 1500 | + { HID_USB_DEVICE(USB_VENDOR_ID_ASUSTEK, | |
| 1501 | + USB_DEVICE_ID_ASUSTEK_XGM_2023), | |
| 1502 | + }, | |
| 1493 | 1503 | { HID_USB_DEVICE(USB_VENDOR_ID_ASUSTEK, |
| 1494 | 1504 | USB_DEVICE_ID_ASUSTEK_ROG_CLAYMORE_II_KEYBOARD), |
| 1495 | 1505 | QUIRK_ROG_CLAYMORE_II_KEYBOARD }, |
| @@ -99,7 +99,7 @@ static int cmhid_raw_event(struct hid_device *hid, struct hid_report *report, · 98 unmodified lines | ||
| 99 | 99 | { |
| 100 | 100 | struct cmhid *cm = hid_get_drvdata(hid); |
| 101 | 101 | |
| 102 | − if (len != CM6533_JD_RAWEV_LEN) | |
| 102 | + if (len != CM6533_JD_RAWEV_LEN || !(hid->claimed & HID_CLAIMED_INPUT)) | |
| 103 | 103 | goto out; |
| 104 | 104 | if (memcmp(data+CM6533_JD_SFX_OFFSET, ji_sfx, sizeof(ji_sfx))) |
| 105 | 105 | goto out; |
| @@ -153,7 +153,7 @@ static int creative_sb0540_raw_event(struct hid_device *hid, · 152 unmodified lines | ||
| 153 | 153 | u64 code, main_code; |
| 154 | 154 | int key; |
| 155 | 155 | |
| 156 | − if (len != 6) | |
| 156 | + if (len != 6 || !(hid->claimed & HID_CLAIMED_INPUT)) | |
| 157 | 157 | return 0; |
| 158 | 158 | |
| 159 | 159 | /* From daemons/hw_hiddev.c sb0540_rec() in lirc */ |
| @@ -229,6 +229,7 @@ · 228 unmodified lines | ||
| 229 | 229 | #define USB_DEVICE_ID_ASUSTEK_ROG_NKEY_ALLY_X 0x1b4c |
| 230 | 230 | #define USB_DEVICE_ID_ASUSTEK_ROG_CLAYMORE_II_KEYBOARD 0x196b |
| 231 | 231 | #define USB_DEVICE_ID_ASUSTEK_FX503VD_KEYBOARD 0x1869 |
| 232 | +#define USB_DEVICE_ID_ASUSTEK_XGM_2023 0x1a9a | |
| 232 | 233 | |
| 233 | 234 | #define USB_VENDOR_ID_ATEN 0x0557 |
| 234 | 235 | #define USB_DEVICE_ID_ATEN_UC100KM 0x2004 |
| @@ -990,13 +990,11 @@ static const __u8 *magicmouse_report_fixup(struct hid_device *hdev, __u8 *rdesc, · 989 unmodified lines | ||
| 990 | 990 | */ |
| 991 | 991 | if ((is_usb_magicmouse2(hdev->vendor, hdev->product) || |
| 992 | 992 | is_usb_magictrackpad2(hdev->vendor, hdev->product)) && |
| 993 | − *rsize == 83 && rdesc[46] == 0x84 && rdesc[58] == 0x85) { | |
| 993 | + *rsize >= 83 && rdesc[46] == 0x84 && rdesc[58] == 0x85) { | |
| 994 | 994 | hid_info(hdev, |
| 995 | 995 | "fixing up magicmouse battery report descriptor\n"); |
| 996 | 996 | *rsize = *rsize - 1; |
| 997 | − rdesc = kmemdup(rdesc + 1, *rsize, GFP_KERNEL); | |
| 998 | − if (!rdesc) | |
| 999 | − return NULL; | |
| 997 | + rdesc = rdesc + 1; | |
| 1000 | 998 | |
| 1001 | 999 | rdesc[0] = 0x05; |
| 1002 | 1000 | rdesc[1] = 0x01; |
| @@ -353,6 +353,8 @@ static int mcp_i2c_smbus_read(struct mcp2221 *mcp, · 352 unmodified lines | ||
| 353 | 353 | usleep_range(90, 100); |
| 354 | 354 | retries++; |
| 355 | 355 | } else { |
| 356 | + usleep_range(980, 1000); | |
| 357 | + mcp_cancel_last_cmd(mcp); | |
| 356 | 358 | return ret; |
| 357 | 359 | } |
| 358 | 360 | } else { |
| @@ -77,6 +77,7 @@ MODULE_LICENSE("GPL"); · 76 unmodified lines | ||
| 77 | 77 | #define MT_QUIRK_ORIENTATION_INVERT BIT(22) |
| 78 | 78 | #define MT_QUIRK_APPLE_TOUCHBAR BIT(23) |
| 79 | 79 | #define MT_QUIRK_YOGABOOK9I BIT(24) |
| 80 | +#define MT_QUIRK_KEEP_LATENCY_ON_CLOSE BIT(25) | |
| 80 | 81 | |
| 81 | 82 | #define MT_INPUTMODE_TOUCHSCREEN 0x02 |
| 82 | 83 | #define MT_INPUTMODE_TOUCHPAD 0x03 |
| @@ -214,6 +215,7 @@ static void mt_post_parse(struct mt_device *td, struct mt_application *app); · 131 unmodified lines | ||
| 214 | 215 | #define MT_CLS_WIN_8_DISABLE_WAKEUP 0x0016 |
| 215 | 216 | #define MT_CLS_WIN_8_NO_STICKY_FINGERS 0x0017 |
| 216 | 217 | #define MT_CLS_WIN_8_FORCE_MULTI_INPUT_NSMU 0x0018 |
| 218 | +#define MT_CLS_WIN_8_KEEP_LATENCY_ON_CLOSE 0x0019 | |
| 217 | 219 | |
| 218 | 220 | /* vendor specific classes */ |
| 219 | 221 | #define MT_CLS_3M 0x0101 |
| @@ -233,6 +235,7 @@ static void mt_post_parse(struct mt_device *td, struct mt_application *app); · 13 unmodified lines | ||
| 233 | 235 | #define MT_CLS_SMART_TECH 0x0113 |
| 234 | 236 | #define MT_CLS_APPLE_TOUCHBAR 0x0114 |
| 235 | 237 | #define MT_CLS_YOGABOOK9I 0x0115 |
| 238 | +#define MT_CLS_EGALAX_P80H84 0x0116 | |
| 236 | 239 | #define MT_CLS_SIS 0x0457 |
| 237 | 240 | |
| 238 | 241 | #define MT_DEFAULT_MAXCONTACT 10 |
| @@ -334,6 +337,15 @@ static const struct mt_class mt_classes[] = { · 95 unmodified lines | ||
| 334 | 337 | MT_QUIRK_CONTACT_CNT_ACCURATE | |
| 335 | 338 | MT_QUIRK_WIN8_PTP_BUTTONS, |
| 336 | 339 | .export_all_inputs = true }, |
| 340 | + { .name = MT_CLS_WIN_8_KEEP_LATENCY_ON_CLOSE, | |
| 341 | + .quirks = MT_QUIRK_ALWAYS_VALID | | |
| 342 | + MT_QUIRK_IGNORE_DUPLICATES | | |
| 343 | + MT_QUIRK_HOVERING | | |
| 344 | + MT_QUIRK_CONTACT_CNT_ACCURATE | | |
| 345 | + MT_QUIRK_STICKY_FINGERS | | |
| 346 | + MT_QUIRK_WIN8_PTP_BUTTONS | | |
| 347 | + MT_QUIRK_KEEP_LATENCY_ON_CLOSE, | |
| 348 | + .export_all_inputs = true }, | |
| 337 | 349 | |
| 338 | 350 | /* |
| 339 | 351 | * vendor specific classes |
| @@ -438,6 +450,11 @@ static const struct mt_class mt_classes[] = { · 98 unmodified lines | ||
| 438 | 450 | MT_QUIRK_YOGABOOK9I, |
| 439 | 451 | .export_all_inputs = true |
| 440 | 452 | }, |
| 453 | + { .name = MT_CLS_EGALAX_P80H84, | |
| 454 | + .quirks = MT_QUIRK_ALWAYS_VALID | | |
| 455 | + MT_QUIRK_IGNORE_DUPLICATES | | |
| 456 | + MT_QUIRK_CONTACT_CNT_ACCURATE, | |
| 457 | + }, | |
| 441 | 458 | { } |
| 442 | 459 | }; |
| 443 | 460 | |
| @@ -849,7 +866,8 @@ static int mt_touch_input_mapping(struct hid_device *hdev, struct hid_input *hi, · 405 unmodified lines | ||
| 849 | 866 | if ((cls->name == MT_CLS_WIN_8 || |
| 850 | 867 | cls->name == MT_CLS_WIN_8_FORCE_MULTI_INPUT || |
| 851 | 868 | cls->name == MT_CLS_WIN_8_FORCE_MULTI_INPUT_NSMU || |
| 852 | − cls->name == MT_CLS_WIN_8_DISABLE_WAKEUP) && | |
| 869 | + cls->name == MT_CLS_WIN_8_DISABLE_WAKEUP || | |
| 870 | + cls->name == MT_CLS_WIN_8_KEEP_LATENCY_ON_CLOSE) && | |
| 853 | 871 | (field->application == HID_DG_TOUCHPAD || |
| 854 | 872 | field->application == HID_DG_TOUCHSCREEN)) |
| 855 | 873 | app->quirks |= MT_QUIRK_CONFIDENCE; |
| @@ -1762,7 +1780,8 @@ static int mt_input_configured(struct hid_device *hdev, struct hid_input *hi) · 906 unmodified lines | ||
| 1762 | 1780 | int ret; |
| 1763 | 1781 | |
| 1764 | 1782 | if (td->is_haptic_touchpad && (td->mtclass.name == MT_CLS_WIN_8 || |
| 1765 | − td->mtclass.name == MT_CLS_WIN_8_FORCE_MULTI_INPUT)) { | |
| 1783 | + td->mtclass.name == MT_CLS_WIN_8_FORCE_MULTI_INPUT || | |
| 1784 | + td->mtclass.name == MT_CLS_WIN_8_KEEP_LATENCY_ON_CLOSE)) { | |
| 1766 | 1785 | if (hid_haptic_input_configured(hdev, td->haptic, hi) == 0) |
| 1767 | 1786 | td->is_haptic_touchpad = false; |
| 1768 | 1787 | } else { |
| @@ -2075,7 +2094,12 @@ static void mt_on_hid_hw_open(struct hid_device *hdev) · 306 unmodified lines | ||
| 2075 | 2094 | |
| 2076 | 2095 | static void mt_on_hid_hw_close(struct hid_device *hdev) |
| 2077 | 2096 | { |
| 2078 | − mt_set_modes(hdev, HID_LATENCY_HIGH, TOUCHPAD_REPORT_NONE); | |
| 2097 | + struct mt_device *td = hid_get_drvdata(hdev); | |
| 2098 | + | |
| 2099 | + if (td->mtclass.quirks & MT_QUIRK_KEEP_LATENCY_ON_CLOSE) | |
| 2100 | + mt_set_modes(hdev, HID_LATENCY_NORMAL, TOUCHPAD_REPORT_NONE); | |
| 2101 | + else | |
| 2102 | + mt_set_modes(hdev, HID_LATENCY_HIGH, TOUCHPAD_REPORT_NONE); | |
| 2079 | 2103 | } |
| 2080 | 2104 | |
| 2081 | 2105 | /* |
| @@ -2215,8 +2239,9 @@ static const struct hid_device_id mt_devices[] = { · 133 unmodified lines | ||
| 2215 | 2239 | { .driver_data = MT_CLS_EGALAX_SERIAL, |
| 2216 | 2240 | MT_USB_DEVICE(USB_VENDOR_ID_DWAV, |
| 2217 | 2241 | USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_C000) }, |
| 2218 | − { .driver_data = MT_CLS_EGALAX, | |
| 2219 | − MT_USB_DEVICE(USB_VENDOR_ID_DWAV, | |
| 2242 | + { .driver_data = MT_CLS_EGALAX_P80H84, | |
| 2243 | + HID_DEVICE(HID_BUS_ANY, HID_GROUP_MULTITOUCH_WIN_8, | |
| 2244 | + USB_VENDOR_ID_DWAV, | |
| 2220 | 2245 | USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_C002) }, |
| 2221 | 2246 | |
| 2222 | 2247 | /* Elan devices */ |
| @@ -2461,6 +2486,14 @@ static const struct hid_device_id mt_devices[] = { · 238 unmodified lines | ||
| 2461 | 2486 | MT_USB_DEVICE(USB_VENDOR_ID_UNITEC, |
| 2462 | 2487 | USB_DEVICE_ID_UNITEC_USB_TOUCH_0A19) }, |
| 2463 | 2488 | |
| 2489 | + /* Uniwill touchpads */ | |
| 2490 | + { .driver_data = MT_CLS_WIN_8_KEEP_LATENCY_ON_CLOSE, | |
| 2491 | + HID_DEVICE(BUS_I2C, HID_GROUP_MULTITOUCH_WIN_8, | |
| 2492 | + USB_VENDOR_ID_PIXART, 0x0255) }, | |
| 2493 | + { .driver_data = MT_CLS_WIN_8_KEEP_LATENCY_ON_CLOSE, | |
| 2494 | + HID_DEVICE(BUS_I2C, HID_GROUP_MULTITOUCH_WIN_8, | |
| 2495 | + USB_VENDOR_ID_PIXART, 0x0274) }, | |
| 2496 | + | |
| 2464 | 2497 | /* VTL panels */ |
| 2465 | 2498 | { .driver_data = MT_CLS_VTL, |
| 2466 | 2499 | MT_USB_DEVICE(USB_VENDOR_ID_VTL, |
| @@ -114,7 +114,7 @@ static int zc_raw_event(struct hid_device *hdev, struct hid_report *report, · 113 unmodified lines | ||
| 114 | 114 | unsigned key; |
| 115 | 115 | unsigned short index; |
| 116 | 116 | |
| 117 | − if (report->id == data[0]) { | |
| 117 | + if (report->id == data[0] && (hdev->claimed & HID_CLAIMED_INPUT)) { | |
| 118 | 118 | |
| 119 | 119 | /* break keys */ |
| 120 | 120 | for (index = 0; index < 4; index++) { |
| @@ -39,6 +39,8 @@ · 38 unmodified lines | ||
| 39 | 39 | #define PCI_DEVICE_ID_INTEL_ISH_PTL_H 0xE345 |
| 40 | 40 | #define PCI_DEVICE_ID_INTEL_ISH_PTL_P 0xE445 |
| 41 | 41 | #define PCI_DEVICE_ID_INTEL_ISH_WCL 0x4D45 |
| 42 | +#define PCI_DEVICE_ID_INTEL_ISH_NVL_H 0xD354 | |
| 43 | +#define PCI_DEVICE_ID_INTEL_ISH_NVL_S 0x6E78 | |
| 42 | 44 | |
| 43 | 45 | #define REVISION_ID_CHT_A0 0x6 |
| 44 | 46 | #define REVISION_ID_CHT_Ax_SI 0x0 |
| @@ -28,11 +28,15 @@ enum ishtp_driver_data_index { · 27 unmodified lines | ||
| 28 | 28 | ISHTP_DRIVER_DATA_LNL_M, |
| 29 | 29 | ISHTP_DRIVER_DATA_PTL, |
| 30 | 30 | ISHTP_DRIVER_DATA_WCL, |
| 31 | + ISHTP_DRIVER_DATA_NVL_H, | |
| 32 | + ISHTP_DRIVER_DATA_NVL_S, | |
| 31 | 33 | }; |
| 32 | 34 | |
| 33 | 35 | #define ISH_FW_GEN_LNL_M "lnlm" |
| 34 | 36 | #define ISH_FW_GEN_PTL "ptl" |
| 35 | 37 | #define ISH_FW_GEN_WCL "wcl" |
| 38 | +#define ISH_FW_GEN_NVL_H "nvlh" | |
| 39 | +#define ISH_FW_GEN_NVL_S "nvls" | |
| 36 | 40 | |
| 37 | 41 | #define ISH_FIRMWARE_PATH(gen) "intel/ish/ish_" gen ".bin" |
| 38 | 42 | #define ISH_FIRMWARE_PATH_ALL "intel/ish/ish_*.bin" |
| @@ -47,6 +51,12 @@ static struct ishtp_driver_data ishtp_driver_data[] = { · 8 unmodified lines | ||
| 47 | 51 | [ISHTP_DRIVER_DATA_WCL] = { |
| 48 | 52 | .fw_generation = ISH_FW_GEN_WCL, |
| 49 | 53 | }, |
| 54 | + [ISHTP_DRIVER_DATA_NVL_H] = { | |
| 55 | + .fw_generation = ISH_FW_GEN_NVL_H, | |
| 56 | + }, | |
| 57 | + [ISHTP_DRIVER_DATA_NVL_S] = { | |
| 58 | + .fw_generation = ISH_FW_GEN_NVL_S, | |
| 59 | + }, | |
| 50 | 60 | }; |
| 51 | 61 | |
| 52 | 62 | static const struct pci_device_id ish_pci_tbl[] = { |
| @@ -76,6 +86,8 @@ static const struct pci_device_id ish_pci_tbl[] = { · 23 unmodified lines | ||
| 76 | 86 | {PCI_DEVICE_DATA(INTEL, ISH_PTL_H, ISHTP_DRIVER_DATA_PTL)}, |
| 77 | 87 | {PCI_DEVICE_DATA(INTEL, ISH_PTL_P, ISHTP_DRIVER_DATA_PTL)}, |
| 78 | 88 | {PCI_DEVICE_DATA(INTEL, ISH_WCL, ISHTP_DRIVER_DATA_WCL)}, |
| 89 | + {PCI_DEVICE_DATA(INTEL, ISH_NVL_H, ISHTP_DRIVER_DATA_NVL_H)}, | |
| 90 | + {PCI_DEVICE_DATA(INTEL, ISH_NVL_S, ISHTP_DRIVER_DATA_NVL_S)}, | |
| 79 | 91 | {} |
| 80 | 92 | }; |
| 81 | 93 | MODULE_DEVICE_TABLE(pci, ish_pci_tbl); |
| @@ -1452,10 +1452,13 @@ static int pidff_init_fields(struct pidff_device *pidff, struct input_dev *dev) · 1451 unmodified lines | ||
| 1452 | 1452 | hid_warn(pidff->hid, "unknown ramp effect layout\n"); |
| 1453 | 1453 | |
| 1454 | 1454 | if (PIDFF_FIND_FIELDS(set_condition, PID_SET_CONDITION, 1)) { |
| 1455 | − if (test_and_clear_bit(FF_SPRING, dev->ffbit) || | |
| 1456 | − test_and_clear_bit(FF_DAMPER, dev->ffbit) || | |
| 1457 | − test_and_clear_bit(FF_FRICTION, dev->ffbit) || | |
| 1458 | − test_and_clear_bit(FF_INERTIA, dev->ffbit)) | |
| 1455 | + bool test = false; | |
| 1456 | + | |
| 1457 | + test |= test_and_clear_bit(FF_SPRING, dev->ffbit); | |
| 1458 | + test |= test_and_clear_bit(FF_DAMPER, dev->ffbit); | |
| 1459 | + test |= test_and_clear_bit(FF_FRICTION, dev->ffbit); | |
| 1460 | + test |= test_and_clear_bit(FF_INERTIA, dev->ffbit); | |
| 1461 | + if (test) | |
| 1459 | 1462 | hid_warn(pidff->hid, "unknown condition effect layout\n"); |
| 1460 | 1463 | } |
| 1461 | 1464 | |
| @@ -1493,8 +1493,7 @@ config SENSORS_LM73 · 1492 unmodified lines | ||
| 1493 | 1493 | |
| 1494 | 1494 | config SENSORS_LM75 |
| 1495 | 1495 | tristate "National Semiconductor LM75 and compatibles" |
| 1496 | − depends on I2C | |
| 1497 | − depends on I3C || !I3C | |
| 1496 | + depends on I3C_OR_I2C | |
| 1498 | 1497 | select REGMAP_I2C |
| 1499 | 1498 | select REGMAP_I3C if I3C |
| 1500 | 1499 | help |
| @@ -1927,16 +1926,6 @@ config SENSORS_RASPBERRYPI_HWMON · 426 unmodified lines | ||
| 1927 | 1926 | This driver can also be built as a module. If so, the module |
| 1928 | 1927 | will be called raspberrypi-hwmon. |
| 1929 | 1928 | |
| 1930 | −config SENSORS_SA67MCU | |
| 1931 | − tristate "Kontron sa67mcu hardware monitoring driver" | |
| 1932 | − depends on MFD_SL28CPLD || COMPILE_TEST | |
| 1933 | − help | |
| 1934 | − If you say yes here you get support for the voltage and temperature | |
| 1935 | − monitor of the sa67 board management controller. | |
| 1936 | − | |
| 1937 | − This driver can also be built as a module. If so, the module | |
| 1938 | − will be called sa67mcu-hwmon. | |
| 1939 | − | |
| 1940 | 1929 | config SENSORS_SL28CPLD |
| 1941 | 1930 | tristate "Kontron sl28cpld hardware monitoring driver" |
| 1942 | 1931 | depends on MFD_SL28CPLD || COMPILE_TEST |
| @@ -2392,8 +2381,7 @@ config SENSORS_TMP103 · 449 unmodified lines | ||
| 2392 | 2381 | |
| 2393 | 2382 | config SENSORS_TMP108 |
| 2394 | 2383 | tristate "Texas Instruments TMP108" |
| 2395 | − depends on I2C | |
| 2396 | − depends on I3C || !I3C | |
| 2384 | + depends on I3C_OR_I2C | |
| 2397 | 2385 | select REGMAP_I2C |
| 2398 | 2386 | select REGMAP_I3C if I3C |
| 2399 | 2387 | help |
| @@ -199,7 +199,6 @@ obj-$(CONFIG_SENSORS_PT5161L) += pt5161l.o · 198 unmodified lines | ||
| 199 | 199 | obj-$(CONFIG_SENSORS_PWM_FAN) += pwm-fan.o |
| 200 | 200 | obj-$(CONFIG_SENSORS_QNAP_MCU_HWMON) += qnap-mcu-hwmon.o |
| 201 | 201 | obj-$(CONFIG_SENSORS_RASPBERRYPI_HWMON) += raspberrypi-hwmon.o |
| 202 | −obj-$(CONFIG_SENSORS_SA67MCU) += sa67mcu-hwmon.o | |
| 203 | 202 | obj-$(CONFIG_SENSORS_SBTSI) += sbtsi_temp.o |
| 204 | 203 | obj-$(CONFIG_SENSORS_SBRMI) += sbrmi.o |
| 205 | 204 | obj-$(CONFIG_SENSORS_SCH56XX_COMMON)+= sch56xx-common.o |
| @@ -37,7 +37,9 @@ · 36 unmodified lines | ||
| 37 | 37 | #define AHT10_CMD_MEAS 0b10101100 |
| 38 | 38 | #define AHT10_CMD_RST 0b10111010 |
| 39 | 39 | |
| 40 | −#define DHT20_CMD_INIT 0x71 | |
| 40 | +#define AHT20_CMD_INIT 0b10111110 | |
| 41 | + | |
| 42 | +#define DHT20_CMD_INIT 0b01110001 | |
| 41 | 43 | |
| 42 | 44 | /* |
| 43 | 45 | * Flags in the answer byte/command |
| @@ -341,7 +343,7 @@ static int aht10_probe(struct i2c_client *client) · 297 unmodified lines | ||
| 341 | 343 | data->meas_size = AHT20_MEAS_SIZE; |
| 342 | 344 | data->crc8 = true; |
| 343 | 345 | crc8_populate_msb(crc8_table, AHT20_CRC8_POLY); |
| 344 | − data->init_cmd = AHT10_CMD_INIT; | |
| 346 | + data->init_cmd = AHT20_CMD_INIT; | |
| 345 | 347 | break; |
| 346 | 348 | case dht20: |
| 347 | 349 | data->meas_size = AHT20_MEAS_SIZE; |
| @@ -3590,10 +3590,13 @@ static int it87_resume(struct device *dev) · 3589 unmodified lines | ||
| 3590 | 3590 | { |
| 3591 | 3591 | struct platform_device *pdev = to_platform_device(dev); |
| 3592 | 3592 | struct it87_data *data = dev_get_drvdata(dev); |
| 3593 | + int err; | |
| 3593 | 3594 | |
| 3594 | 3595 | it87_resume_sio(pdev); |
| 3595 | 3596 | |
| 3596 | − it87_lock(data); | |
| 3597 | + err = it87_lock(data); | |
| 3598 | + if (err) | |
| 3599 | + return err; | |
| 3597 | 3600 | |
| 3598 | 3601 | it87_check_pwm(dev); |
| 3599 | 3602 | it87_check_limit_regs(data); |
| @@ -22,6 +22,7 @@ · 21 unmodified lines | ||
| 22 | 22 | |
| 23 | 23 | #include <linux/bitfield.h> |
| 24 | 24 | #include <linux/hwmon.h> |
| 25 | +#include <linux/math64.h> | |
| 25 | 26 | #include <linux/mfd/macsmc.h> |
| 26 | 27 | #include <linux/module.h> |
| 27 | 28 | #include <linux/of.h> |
| @@ -130,7 +131,7 @@ static int macsmc_hwmon_read_ioft_scaled(struct apple_smc *smc, smc_key key, · 102 unmodified lines | ||
| 130 | 131 | if (ret < 0) |
| 131 | 132 | return ret; |
| 132 | 133 | |
| 133 | − *p = mult_frac(val, scale, 65536); | |
| 134 | + *p = mul_u64_u32_div(val, scale, 65536); | |
| 134 | 135 | |
| 135 | 136 | return 0; |
| 136 | 137 | } |
| @@ -140,7 +141,7 @@ static int macsmc_hwmon_read_ioft_scaled(struct apple_smc *smc, smc_key key, · 3 unmodified lines | ||
| 140 | 141 | * them. |
| 141 | 142 | */ |
| 142 | 143 | static int macsmc_hwmon_read_f32_scaled(struct apple_smc *smc, smc_key key, |
| 143 | − int *p, int scale) | |
| 144 | + long *p, int scale) | |
| 144 | 145 | { |
| 145 | 146 | u32 fval; |
| 146 | 147 | u64 val; |
| @@ -162,21 +163,21 @@ static int macsmc_hwmon_read_f32_scaled(struct apple_smc *smc, smc_key key, · 15 unmodified lines | ||
| 162 | 163 | val = 0; |
| 163 | 164 | else if (exp < 0) |
| 164 | 165 | val >>= -exp; |
| 165 | − else if (exp != 0 && (val & ~((1UL << (64 - exp)) - 1))) /* overflow */ | |
| 166 | + else if (exp != 0 && (val & ~((1ULL << (64 - exp)) - 1))) /* overflow */ | |
| 166 | 167 | val = U64_MAX; |
| 167 | 168 | else |
| 168 | 169 | val <<= exp; |
| 169 | 170 | |
| 170 | 171 | if (fval & FLT_SIGN_MASK) { |
| 171 | − if (val > (-(s64)INT_MIN)) | |
| 172 | − *p = INT_MIN; | |
| 172 | + if (val > (u64)LONG_MAX + 1) | |
| 173 | + *p = LONG_MIN; | |
| 173 | 174 | else |
| 174 | − *p = -val; | |
| 175 | + *p = -(long)val; | |
| 175 | 176 | } else { |
| 176 | − if (val > INT_MAX) | |
| 177 | − *p = INT_MAX; | |
| 177 | + if (val > (u64)LONG_MAX) | |
| 178 | + *p = LONG_MAX; | |
| 178 | 179 | else |
| 179 | − *p = val; | |
| 180 | + *p = (long)val; | |
| 180 | 181 | } |
| 181 | 182 | |
| 182 | 183 | return 0; |
| @@ -195,7 +196,7 @@ static int macsmc_hwmon_read_key(struct apple_smc *smc, · 12 unmodified lines | ||
| 195 | 196 | switch (sensor->info.type_code) { |
| 196 | 197 | /* 32-bit IEEE 754 float */ |
| 197 | 198 | case __SMC_KEY('f', 'l', 't', ' '): { |
| 198 | − u32 flt_ = 0; | |
| 199 | + long flt_ = 0; | |
| 199 | 200 | |
| 200 | 201 | ret = macsmc_hwmon_read_f32_scaled(smc, sensor->macsmc_key, |
| 201 | 202 | &flt_, scale); |
| @@ -214,7 +215,10 @@ static int macsmc_hwmon_read_key(struct apple_smc *smc, · 12 unmodified lines | ||
| 214 | 215 | if (ret) |
| 215 | 216 | return ret; |
| 216 | 217 | |
| 217 | − *val = (long)ioft; | |
| 218 | + if (ioft > LONG_MAX) | |
| 219 | + *val = LONG_MAX; | |
| 220 | + else | |
| 221 | + *val = (long)ioft; | |
| 218 | 222 | break; |
| 219 | 223 | } |
| 220 | 224 | default: |
| @@ -224,29 +228,26 @@ static int macsmc_hwmon_read_key(struct apple_smc *smc, · 3 unmodified lines | ||
| 224 | 228 | return 0; |
| 225 | 229 | } |
| 226 | 230 | |
| 227 | −static int macsmc_hwmon_write_f32(struct apple_smc *smc, smc_key key, int value) | |
| 231 | +static int macsmc_hwmon_write_f32(struct apple_smc *smc, smc_key key, long value) | |
| 228 | 232 | { |
| 229 | 233 | u64 val; |
| 230 | 234 | u32 fval = 0; |
| 231 | − int exp = 0, neg; | |
| 235 | + int exp, neg; | |
| 232 | 236 | |
| 237 | + neg = value < 0; | |
| 233 | 238 | val = abs(value); |
| 234 | − neg = val != value; | |
| 235 | 239 | |
| 236 | 240 | if (val) { |
| 237 | − int msb = __fls(val) - exp; | |
| 238 | − | |
| 239 | − if (msb > 23) { | |
| 240 | − val >>= msb - FLT_MANT_BIAS; | |
| 241 | − exp -= msb - FLT_MANT_BIAS; | |
| 242 | − } else if (msb < 23) { | |
| 243 | − val <<= FLT_MANT_BIAS - msb; | |
| 244 | − exp += msb; | |
| 245 | − } | |
| 241 | + exp = __fls(val); | |
| 242 | + | |
| 243 | + if (exp > 23) | |
| 244 | + val >>= exp - 23; | |
| 245 | + else | |
| 246 | + val <<= 23 - exp; | |
| 246 | 247 | |
| 247 | 248 | fval = FIELD_PREP(FLT_SIGN_MASK, neg) | |
| 248 | 249 | FIELD_PREP(FLT_EXP_MASK, exp + FLT_EXP_BIAS) | |
| 249 | − FIELD_PREP(FLT_MANT_MASK, val); | |
| 250 | + FIELD_PREP(FLT_MANT_MASK, val & FLT_MANT_MASK); | |
| 250 | 251 | } |
| 251 | 252 | |
| 252 | 253 | return apple_smc_write_u32(smc, key, fval); |
| @@ -663,8 +664,8 @@ static int macsmc_hwmon_populate_sensors(struct macsmc_hwmon *hwmon, · 410 unmodified lines | ||
| 663 | 664 | if (!hwmon->volt.sensors) |
| 664 | 665 | return -ENOMEM; |
| 665 | 666 | |
| 666 | − for_each_child_of_node_with_prefix(hwmon_node, key_node, "volt-") { | |
| 667 | − sensor = &hwmon->temp.sensors[hwmon->temp.count]; | |
| 667 | + for_each_child_of_node_with_prefix(hwmon_node, key_node, "voltage-") { | |
| 668 | + sensor = &hwmon->volt.sensors[hwmon->volt.count]; | |
| 668 | 669 | if (!macsmc_hwmon_create_sensor(hwmon->dev, hwmon->smc, key_node, sensor)) { |
| 669 | 670 | sensor->attrs = HWMON_I_INPUT; |
| 670 | 671 | |
| @@ -607,7 +607,7 @@ static int max6639_init_client(struct i2c_client *client, · 606 unmodified lines | ||
| 607 | 607 | return err; |
| 608 | 608 | |
| 609 | 609 | /* Fans PWM polarity high by default */ |
| 610 | − err = regmap_write(data->regmap, MAX6639_REG_FAN_CONFIG2a(i), 0x00); | |
| 610 | + err = regmap_write(data->regmap, MAX6639_REG_FAN_CONFIG2a(i), 0x02); | |
| 611 | 611 | if (err) |
| 612 | 612 | return err; |
| 613 | 613 | |
| @@ -79,7 +79,8 @@ static ssize_t q54sj108a2_debugfs_read(struct file *file, char __user *buf, · 78 unmodified lines | ||
| 79 | 79 | int idx = *idxp; |
| 80 | 80 | struct q54sj108a2_data *psu = to_psu(idxp, idx); |
| 81 | 81 | char data[I2C_SMBUS_BLOCK_MAX + 2] = { 0 }; |
| 82 | − char data_char[I2C_SMBUS_BLOCK_MAX + 2] = { 0 }; | |
| 82 | + char data_char[I2C_SMBUS_BLOCK_MAX * 2 + 2] = { 0 }; | |
| 83 | + char *out = data; | |
| 83 | 84 | char *res; |
| 84 | 85 | |
| 85 | 86 | switch (idx) { |
| @@ -150,27 +151,27 @@ static ssize_t q54sj108a2_debugfs_read(struct file *file, char __user *buf, · 64 unmodified lines | ||
| 150 | 151 | if (rc < 0) |
| 151 | 152 | return rc; |
| 152 | 153 | |
| 153 | − res = bin2hex(data, data_char, 32); | |
| 154 | − rc = res - data; | |
| 155 | − | |
| 154 | + res = bin2hex(data_char, data, rc); | |
| 155 | + rc = res - data_char; | |
| 156 | + out = data_char; | |
| 156 | 157 | break; |
| 157 | 158 | case Q54SJ108A2_DEBUGFS_FLASH_KEY: |
| 158 | 159 | rc = i2c_smbus_read_block_data(psu->client, PMBUS_FLASH_KEY_WRITE, data); |
| 159 | 160 | if (rc < 0) |
| 160 | 161 | return rc; |
| 161 | 162 | |
| 162 | − res = bin2hex(data, data_char, 4); | |
| 163 | − rc = res - data; | |
| 164 | − | |
| 163 | + res = bin2hex(data_char, data, rc); | |
| 164 | + rc = res - data_char; | |
| 165 | + out = data_char; | |
| 165 | 166 | break; |
| 166 | 167 | default: |
| 167 | 168 | return -EINVAL; |
| 168 | 169 | } |
| 169 | 170 | |
| 170 | − data[rc] = '\n'; | |
| 171 | + out[rc] = '\n'; | |
| 171 | 172 | rc += 2; |
| 172 | 173 | |
| 173 | − return simple_read_from_buffer(buf, count, ppos, data, rc); | |
| 174 | + return simple_read_from_buffer(buf, count, ppos, out, rc); | |
| 174 | 175 | } |
| 175 | 176 | |
| 176 | 177 | static ssize_t q54sj108a2_debugfs_write(struct file *file, const char __user *buf, |
| @@ -1,161 +0,0 @@ | ||
| 1 | −// SPDX-License-Identifier: GPL-2.0-only | |
| 2 | −/* | |
| 3 | − * sl67mcu hardware monitoring driver | |
| 4 | − * | |
| 5 | − * Copyright 2025 Kontron Europe GmbH | |
| 6 | − */ | |
| 7 | − | |
| 8 | −#include <linux/bitfield.h> | |
| 9 | −#include <linux/hwmon.h> | |
| 10 | −#include <linux/kernel.h> | |
| 11 | −#include <linux/mod_devicetable.h> | |
| 12 | −#include <linux/module.h> | |
| 13 | −#include <linux/platform_device.h> | |
| 14 | −#include <linux/property.h> | |
| 15 | −#include <linux/regmap.h> | |
| 16 | − | |
| 17 | −#define SA67MCU_VOLTAGE(n) (0x00 + ((n) * 2)) | |
| 18 | −#define SA67MCU_TEMP(n) (0x04 + ((n) * 2)) | |
| 19 | − | |
| 20 | −struct sa67mcu_hwmon { | |
| 21 | − struct regmap *regmap; | |
| 22 | − u32 offset; | |
| 23 | −}; | |
| 24 | − | |
| 25 | −static int sa67mcu_hwmon_read(struct device *dev, | |
| 26 | − enum hwmon_sensor_types type, u32 attr, | |
| 27 | − int channel, long *input) | |
| 28 | −{ | |
| 29 | − struct sa67mcu_hwmon *hwmon = dev_get_drvdata(dev); | |
| 30 | − unsigned int offset; | |
| 31 | − u8 reg[2]; | |
| 32 | − int ret; | |
| 33 | − | |
| 34 | − switch (type) { | |
| 35 | − case hwmon_in: | |
| 36 | − switch (attr) { | |
| 37 | − case hwmon_in_input: | |
| 38 | − offset = hwmon->offset + SA67MCU_VOLTAGE(channel); | |
| 39 | − break; | |
| 40 | − default: | |
| 41 | − return -EOPNOTSUPP; | |
| 42 | − } | |
| 43 | − break; | |
| 44 | − case hwmon_temp: | |
| 45 | − switch (attr) { | |
| 46 | − case hwmon_temp_input: | |
| 47 | − offset = hwmon->offset + SA67MCU_TEMP(channel); | |
| 48 | − break; | |
| 49 | − default: | |
| 50 | − return -EOPNOTSUPP; | |
| 51 | − } | |
| 52 | − break; | |
| 53 | − default: | |
| 54 | − return -EOPNOTSUPP; | |
| 55 | − } | |
| 56 | − | |
| 57 | − /* Reading the low byte will capture the value */ | |
| 58 | − ret = regmap_bulk_read(hwmon->regmap, offset, reg, ARRAY_SIZE(reg)); | |
| 59 | − if (ret) | |
| 60 | − return ret; | |
| 61 | − | |
| 62 | − *input = reg[1] << 8 | reg[0]; | |
| 63 | − | |
| 64 | − /* Temperatures are s16 and in 0.1degC steps. */ | |
| 65 | − if (type == hwmon_temp) | |
| 66 | − *input = sign_extend32(*input, 15) * 100; | |
| 67 | − | |
| 68 | − return 0; | |
| 69 | −} | |
| 70 | − | |
| 71 | −static const struct hwmon_channel_info * const sa67mcu_hwmon_info[] = { | |
| 72 | − HWMON_CHANNEL_INFO(in, | |
| 73 | − HWMON_I_INPUT | HWMON_I_LABEL, | |
| 74 | − HWMON_I_INPUT | HWMON_I_LABEL), | |
| 75 | − HWMON_CHANNEL_INFO(temp, HWMON_T_INPUT), | |
| 76 | − NULL | |
| 77 | −}; | |
| 78 | − | |
| 79 | −static const char *const sa67mcu_hwmon_in_labels[] = { | |
| 80 | − "VDDIN", | |
| 81 | − "VDD_RTC", | |
| 82 | −}; | |
| 83 | − | |
| 84 | −static int sa67mcu_hwmon_read_string(struct device *dev, | |
| 85 | − enum hwmon_sensor_types type, u32 attr, | |
| 86 | − int channel, const char **str) | |
| 87 | −{ | |
| 88 | − switch (type) { | |
| 89 | − case hwmon_in: | |
| 90 | − switch (attr) { | |
| 91 | − case hwmon_in_label: | |
| 92 | − *str = sa67mcu_hwmon_in_labels[channel]; | |
| 93 | − return 0; | |
| 94 | − default: | |
| 95 | − return -EOPNOTSUPP; | |
| 96 | − } | |
| 97 | − default: | |
| 98 | − return -EOPNOTSUPP; | |
| 99 | − } | |
| 100 | −} | |
| 101 | − | |
| 102 | −static const struct hwmon_ops sa67mcu_hwmon_ops = { | |
| 103 | − .visible = 0444, | |
| 104 | − .read = sa67mcu_hwmon_read, | |
| 105 | − .read_string = sa67mcu_hwmon_read_string, | |
| 106 | −}; | |
| 107 | − | |
| 108 | −static const struct hwmon_chip_info sa67mcu_hwmon_chip_info = { | |
| 109 | − .ops = &sa67mcu_hwmon_ops, | |
| 110 | − .info = sa67mcu_hwmon_info, | |
| 111 | −}; | |
| 112 | − | |
| 113 | −static int sa67mcu_hwmon_probe(struct platform_device *pdev) | |
| 114 | −{ | |
| 115 | − struct sa67mcu_hwmon *hwmon; | |
| 116 | − struct device *hwmon_dev; | |
| 117 | − int ret; | |
| 118 | − | |
| 119 | − if (!pdev->dev.parent) | |
| 120 | − return -ENODEV; | |
| 121 | − | |
| 122 | − hwmon = devm_kzalloc(&pdev->dev, sizeof(*hwmon), GFP_KERNEL); | |
| 123 | − if (!hwmon) | |
| 124 | − return -ENOMEM; | |
| 125 | − | |
| 126 | − hwmon->regmap = dev_get_regmap(pdev->dev.parent, NULL); | |
| 127 | − if (!hwmon->regmap) | |
| 128 | − return -ENODEV; | |
| 129 | − | |
| 130 | − ret = device_property_read_u32(&pdev->dev, "reg", &hwmon->offset); | |
| 131 | − if (ret) | |
| 132 | − return -EINVAL; | |
| 133 | − | |
| 134 | − hwmon_dev = devm_hwmon_device_register_with_info(&pdev->dev, | |
| 135 | − "sa67mcu_hwmon", hwmon, | |
| 136 | − &sa67mcu_hwmon_chip_info, | |
| 137 | − NULL); | |
| 138 | − if (IS_ERR(hwmon_dev)) | |
| 139 | − dev_err(&pdev->dev, "failed to register as hwmon device"); | |
| 140 | − | |
| 141 | − return PTR_ERR_OR_ZERO(hwmon_dev); | |
| 142 | −} | |
| 143 | − | |
| 144 | −static const struct of_device_id sa67mcu_hwmon_of_match[] = { | |
| 145 | − { .compatible = "kontron,sa67mcu-hwmon", }, | |
| 146 | − {} | |
| 147 | −}; | |
| 148 | −MODULE_DEVICE_TABLE(of, sa67mcu_hwmon_of_match); | |
| 149 | − | |
| 150 | −static struct platform_driver sa67mcu_hwmon_driver = { | |
| 151 | − .probe = sa67mcu_hwmon_probe, | |
| 152 | − .driver = { | |
| 153 | − .name = "sa67mcu-hwmon", | |
| 154 | − .of_match_table = sa67mcu_hwmon_of_match, | |
| 155 | − }, | |
| 156 | −}; | |
| 157 | −module_platform_driver(sa67mcu_hwmon_driver); | |
| 158 | − | |
| 159 | −MODULE_DESCRIPTION("sa67mcu Hardware Monitoring Driver"); | |
| 160 | −MODULE_AUTHOR("Michael Walle <mwalle@kernel.org>"); | |
| 161 | −MODULE_LICENSE("GPL"); | |
| @@ -310,9 +310,10 @@ struct i801_priv { · 309 unmodified lines | ||
| 310 | 310 | |
| 311 | 311 | /* |
| 312 | 312 | * If set to true the host controller registers are reserved for |
| 313 | − * ACPI AML use. | |
| 313 | + * ACPI AML use. Needs extra protection by acpi_lock. | |
| 314 | 314 | */ |
| 315 | 315 | bool acpi_reserved; |
| 316 | + struct mutex acpi_lock; | |
| 316 | 317 | }; |
| 317 | 318 | |
| 318 | 319 | #define FEATURE_SMBUS_PEC BIT(0) |
| @@ -894,8 +895,11 @@ static s32 i801_access(struct i2c_adapter *adap, u16 addr, · 575 unmodified lines | ||
| 894 | 895 | int hwpec, ret; |
| 895 | 896 | struct i801_priv *priv = i2c_get_adapdata(adap); |
| 896 | 897 | |
| 897 | − if (priv->acpi_reserved) | |
| 898 | + mutex_lock(&priv->acpi_lock); | |
| 899 | + if (priv->acpi_reserved) { | |
| 900 | + mutex_unlock(&priv->acpi_lock); | |
| 898 | 901 | return -EBUSY; |
| 902 | + } | |
| 899 | 903 | |
| 900 | 904 | pm_runtime_get_sync(&priv->pci_dev->dev); |
| 901 | 905 | |
| @@ -935,6 +939,7 @@ out: · 33 unmodified lines | ||
| 935 | 939 | iowrite8(SMBHSTSTS_INUSE_STS | STATUS_FLAGS, SMBHSTSTS(priv)); |
| 936 | 940 | |
| 937 | 941 | pm_runtime_put_autosuspend(&priv->pci_dev->dev); |
| 942 | + mutex_unlock(&priv->acpi_lock); | |
| 938 | 943 | return ret; |
| 939 | 944 | } |
| 940 | 945 | |
| @@ -1465,7 +1470,7 @@ i801_acpi_io_handler(u32 function, acpi_physical_address address, u32 bits, · 524 unmodified lines | ||
| 1465 | 1470 | * further access from the driver itself. This device is now owned |
| 1466 | 1471 | * by the system firmware. |
| 1467 | 1472 | */ |
| 1468 | − i2c_lock_bus(&priv->adapter, I2C_LOCK_SEGMENT); | |
| 1473 | + mutex_lock(&priv->acpi_lock); | |
| 1469 | 1474 | |
| 1470 | 1475 | if (!priv->acpi_reserved && i801_acpi_is_smbus_ioport(priv, address)) { |
| 1471 | 1476 | priv->acpi_reserved = true; |
| @@ -1485,7 +1490,7 @@ i801_acpi_io_handler(u32 function, acpi_physical_address address, u32 bits, · 13 unmodified lines | ||
| 1485 | 1490 | else |
| 1486 | 1491 | status = acpi_os_write_port(address, (u32)*value, bits); |
| 1487 | 1492 | |
| 1488 | − i2c_unlock_bus(&priv->adapter, I2C_LOCK_SEGMENT); | |
| 1493 | + mutex_unlock(&priv->acpi_lock); | |
| 1489 | 1494 | |
| 1490 | 1495 | return status; |
| 1491 | 1496 | } |
| @@ -1545,6 +1550,7 @@ static int i801_probe(struct pci_dev *dev, const struct pci_device_id *id) · 53 unmodified lines | ||
| 1545 | 1550 | priv->adapter.dev.parent = &dev->dev; |
| 1546 | 1551 | acpi_use_parent_companion(&priv->adapter.dev); |
| 1547 | 1552 | priv->adapter.retries = 3; |
| 1553 | + mutex_init(&priv->acpi_lock); | |
| 1548 | 1554 | |
| 1549 | 1555 | priv->pci_dev = dev; |
| 1550 | 1556 | priv->features = id->driver_data; |
| @@ -22,3 +22,15 @@ menuconfig I3C · 21 unmodified lines | ||
| 22 | 22 | if I3C |
| 23 | 23 | source "drivers/i3c/master/Kconfig" |
| 24 | 24 | endif # I3C |
| 25 | + | |
| 26 | +config I3C_OR_I2C | |
| 27 | + tristate | |
| 28 | + default m if I3C=m | |
| 29 | + default I2C | |
| 30 | + help | |
| 31 | + Device drivers using module_i3c_i2c_driver() can use either | |
| 32 | + i2c or i3c hosts, but cannot be built-in for the kernel when | |
| 33 | + CONFIG_I3C=m. | |
| 34 | + | |
| 35 | + Add 'depends on I2C_OR_I3C' in Kconfig for those drivers to | |
| 36 | + get the correct dependencies. | |
| @@ -1024,7 +1024,7 @@ static int dw_i3c_master_reattach_i3c_dev(struct i3c_dev_desc *dev, · 1023 unmodified lines | ||
| 1024 | 1024 | master->free_pos &= ~BIT(pos); |
| 1025 | 1025 | } |
| 1026 | 1026 | |
| 1027 | − writel(DEV_ADDR_TABLE_DYNAMIC_ADDR(dev->info.dyn_addr), | |
| 1027 | + writel(DEV_ADDR_TABLE_DYNAMIC_ADDR(dev->info.dyn_addr) | DEV_ADDR_TABLE_SIR_REJECT, | |
| 1028 | 1028 | master->regs + |
| 1029 | 1029 | DEV_ADDR_TABLE_LOC(master->datstartaddr, data->index)); |
| 1030 | 1030 | |
| @@ -1053,7 +1053,7 @@ static int dw_i3c_master_attach_i3c_dev(struct i3c_dev_desc *dev) · 22 unmodified lines | ||
| 1053 | 1053 | master->free_pos &= ~BIT(pos); |
| 1054 | 1054 | i3c_dev_set_master_data(dev, data); |
| 1055 | 1055 | |
| 1056 | − writel(DEV_ADDR_TABLE_DYNAMIC_ADDR(master->devs[pos].addr), | |
| 1056 | + writel(DEV_ADDR_TABLE_DYNAMIC_ADDR(master->devs[pos].addr) | DEV_ADDR_TABLE_SIR_REJECT, | |
| 1057 | 1057 | master->regs + |
| 1058 | 1058 | DEV_ADDR_TABLE_LOC(master->datstartaddr, data->index)); |
| 1059 | 1059 | |
| @@ -1659,6 +1659,8 @@ int dw_i3c_common_probe(struct dw_i3c_master *master, · 599 unmodified lines | ||
| 1659 | 1659 | pm_runtime_get_noresume(&pdev->dev); |
| 1660 | 1660 | |
| 1661 | 1661 | INIT_WORK(&master->hj_work, dw_i3c_hj_work); |
| 1662 | + | |
| 1663 | + device_set_of_node_from_dev(&master->base.i2c.dev, &pdev->dev); | |
| 1662 | 1664 | ret = i3c_master_register(&master->base, &pdev->dev, |
| 1663 | 1665 | &dw_mipi_i3c_ops, false); |
| 1664 | 1666 | if (ret) |
| @@ -17,6 +17,7 @@ · 16 unmodified lines | ||
| 17 | 17 | #define CMD_0_TOC W0_BIT_(31) |
| 18 | 18 | #define CMD_0_ROC W0_BIT_(30) |
| 19 | 19 | #define CMD_0_ATTR W0_MASK(2, 0) |
| 20 | +#define CMD_0_TID W0_MASK(6, 3) | |
| 20 | 21 | |
| 21 | 22 | /* |
| 22 | 23 | * Response Descriptor Structure |
| @@ -331,12 +331,10 @@ static int hci_cmd_v1_daa(struct i3c_hci *hci) · 330 unmodified lines | ||
| 331 | 331 | CMD_A0_ROC | CMD_A0_TOC; |
| 332 | 332 | xfer->cmd_desc[1] = 0; |
| 333 | 333 | xfer->completion = &done; |
| 334 | − hci->io->queue_xfer(hci, xfer, 1); | |
| 335 | − if (!wait_for_completion_timeout(&done, HZ) && | |
| 336 | − hci->io->dequeue_xfer(hci, xfer, 1)) { | |
| 337 | − ret = -ETIME; | |
| 334 | + xfer->timeout = HZ; | |
| 335 | + ret = i3c_hci_process_xfer(hci, xfer, 1); | |
| 336 | + if (ret) | |
| 338 | 337 | break; |
| 339 | − } | |
| 340 | 338 | if ((RESP_STATUS(xfer->response) == RESP_ERR_ADDR_HEADER || |
| 341 | 339 | RESP_STATUS(xfer->response) == RESP_ERR_NACK) && |
| 342 | 340 | RESP_DATA_LENGTH(xfer->response) == 1) { |
| @@ -253,6 +253,7 @@ static int hci_cmd_v2_daa(struct i3c_hci *hci) · 252 unmodified lines | ||
| 253 | 253 | xfer[0].rnw = true; |
| 254 | 254 | xfer[0].cmd_desc[1] = CMD_A1_DATA_LENGTH(8); |
| 255 | 255 | xfer[1].completion = &done; |
| 256 | + xfer[1].timeout = HZ; | |
| 256 | 257 | |
| 257 | 258 | for (;;) { |
| 258 | 259 | ret = i3c_master_get_free_addr(&hci->master, next_addr); |
| @@ -272,12 +273,9 @@ static int hci_cmd_v2_daa(struct i3c_hci *hci) · 13 unmodified lines | ||
| 272 | 273 | CMD_A0_ASSIGN_ADDRESS(next_addr) | |
| 273 | 274 | CMD_A0_ROC | |
| 274 | 275 | CMD_A0_TOC; |
| 275 | − hci->io->queue_xfer(hci, xfer, 2); | |
| 276 | − if (!wait_for_completion_timeout(&done, HZ) && | |
| 277 | − hci->io->dequeue_xfer(hci, xfer, 2)) { | |
| 278 | − ret = -ETIME; | |
| 276 | + ret = i3c_hci_process_xfer(hci, xfer, 2); | |
| 277 | + if (ret) | |
| 279 | 278 | break; |
| 280 | − } | |
| 281 | 279 | if (RESP_STATUS(xfer[0].response) != RESP_SUCCESS) { |
| 282 | 280 | ret = 0; /* no more devices to be assigned */ |
| 283 | 281 | break; |
| @@ -152,7 +152,11 @@ static int i3c_hci_bus_init(struct i3c_master_controller *m) · 151 unmodified lines | ||
| 152 | 152 | if (hci->quirks & HCI_QUIRK_RESP_BUF_THLD) |
| 153 | 153 | amd_set_resp_buf_thld(hci); |
| 154 | 154 | |
| 155 | − reg_set(HC_CONTROL, HC_CONTROL_BUS_ENABLE); | |
| 155 | + scoped_guard(spinlock_irqsave, &hci->lock) | |
| 156 | + hci->irq_inactive = false; | |
| 157 | + | |
| 158 | + /* Enable bus with Hot-Join disabled */ | |
| 159 | + reg_set(HC_CONTROL, HC_CONTROL_BUS_ENABLE | HC_CONTROL_HOT_JOIN_CTRL); | |
| 156 | 160 | dev_dbg(&hci->master.dev, "HC_CONTROL = %#x", reg_read(HC_CONTROL)); |
| 157 | 161 | |
| 158 | 162 | return 0; |
| @@ -177,21 +181,51 @@ static int i3c_hci_bus_disable(struct i3c_hci *hci) · 18 unmodified lines | ||
| 177 | 181 | return ret; |
| 178 | 182 | } |
| 179 | 183 | |
| 184 | +static int i3c_hci_software_reset(struct i3c_hci *hci) | |
| 185 | +{ | |
| 186 | + u32 regval; | |
| 187 | + int ret; | |
| 188 | + | |
| 189 | + /* | |
| 190 | + * SOFT_RST must be clear before we write to it. | |
| 191 | + * Then we must wait until it clears again. | |
| 192 | + */ | |
| 193 | + ret = readx_poll_timeout(reg_read, RESET_CONTROL, regval, | |
| 194 | + !(regval & SOFT_RST), 0, 10 * USEC_PER_MSEC); | |
| 195 | + if (ret) { | |
| 196 | + dev_err(&hci->master.dev, "%s: Software reset stuck\n", __func__); | |
| 197 | + return ret; | |
| 198 | + } | |
| 199 | + | |
| 200 | + reg_write(RESET_CONTROL, SOFT_RST); | |
| 201 | + | |
| 202 | + ret = readx_poll_timeout(reg_read, RESET_CONTROL, regval, | |
| 203 | + !(regval & SOFT_RST), 0, 10 * USEC_PER_MSEC); | |
| 204 | + if (ret) { | |
| 205 | + dev_err(&hci->master.dev, "%s: Software reset failed\n", __func__); | |
| 206 | + return ret; | |
| 207 | + } | |
| 208 | + | |
| 209 | + return 0; | |
| 210 | +} | |
| 211 | + | |
| 180 | 212 | void i3c_hci_sync_irq_inactive(struct i3c_hci *hci) |
| 181 | 213 | { |
| 182 | 214 | struct platform_device *pdev = to_platform_device(hci->master.dev.parent); |
| 183 | 215 | int irq = platform_get_irq(pdev, 0); |
| 184 | 216 | |
| 185 | 217 | reg_write(INTR_SIGNAL_ENABLE, 0x0); |
| 186 | − hci->irq_inactive = true; | |
| 187 | 218 | synchronize_irq(irq); |
| 219 | + scoped_guard(spinlock_irqsave, &hci->lock) | |
| 220 | + hci->irq_inactive = true; | |
| 188 | 221 | } |
| 189 | 222 | |
| 190 | 223 | static void i3c_hci_bus_cleanup(struct i3c_master_controller *m) |
| 191 | 224 | { |
| 192 | 225 | struct i3c_hci *hci = to_i3c_hci(m); |
| 193 | 226 | |
| 194 | − i3c_hci_bus_disable(hci); | |
| 227 | + if (i3c_hci_bus_disable(hci)) | |
| 228 | + i3c_hci_software_reset(hci); | |
| 195 | 229 | hci->io->cleanup(hci); |
| 196 | 230 | } |
| 197 | 231 | |
| @@ -212,6 +246,36 @@ void mipi_i3c_hci_dct_index_reset(struct i3c_hci *hci) · 14 unmodified lines | ||
| 212 | 246 | reg_write(DCT_SECTION, FIELD_PREP(DCT_TABLE_INDEX, 0)); |
| 213 | 247 | } |
| 214 | 248 | |
| 249 | +int i3c_hci_process_xfer(struct i3c_hci *hci, struct hci_xfer *xfer, int n) | |
| 250 | +{ | |
| 251 | + struct completion *done = xfer[n - 1].completion; | |
| 252 | + unsigned long timeout = xfer[n - 1].timeout; | |
| 253 | + int ret; | |
| 254 | + | |
| 255 | + ret = hci->io->queue_xfer(hci, xfer, n); | |
| 256 | + if (ret) | |
| 257 | + return ret; | |
| 258 | + | |
| 259 | + if (!wait_for_completion_timeout(done, timeout)) { | |
| 260 | + if (hci->io->dequeue_xfer(hci, xfer, n)) { | |
| 261 | + dev_err(&hci->master.dev, "%s: timeout error\n", __func__); | |
| 262 | + return -ETIMEDOUT; | |
| 263 | + } | |
| 264 | + return 0; | |
| 265 | + } | |
| 266 | + | |
| 267 | + if (hci->io->handle_error) { | |
| 268 | + bool error = false; | |
| 269 | + | |
| 270 | + for (int i = 0; i < n && !error; i++) | |
| 271 | + error = RESP_STATUS(xfer[i].response); | |
| 272 | + if (error) | |
| 273 | + return hci->io->handle_error(hci, xfer, n); | |
| 274 | + } | |
| 275 | + | |
| 276 | + return 0; | |
| 277 | +} | |
| 278 | + | |
| 215 | 279 | static int i3c_hci_send_ccc_cmd(struct i3c_master_controller *m, |
| 216 | 280 | struct i3c_ccc_cmd *ccc) |
| 217 | 281 | { |
| @@ -252,18 +316,14 @@ static int i3c_hci_send_ccc_cmd(struct i3c_master_controller *m, · 34 unmodified lines | ||
| 252 | 316 | last = i - 1; |
| 253 | 317 | xfer[last].cmd_desc[0] |= CMD_0_TOC; |
| 254 | 318 | xfer[last].completion = &done; |
| 319 | + xfer[last].timeout = HZ; | |
| 255 | 320 | |
| 256 | 321 | if (prefixed) |
| 257 | 322 | xfer--; |
| 258 | 323 | |
| 259 | − ret = hci->io->queue_xfer(hci, xfer, nxfers); | |
| 324 | + ret = i3c_hci_process_xfer(hci, xfer, nxfers); | |
| 260 | 325 | if (ret) |
| 261 | 326 | goto out; |
| 262 | − if (!wait_for_completion_timeout(&done, HZ) && | |
| 263 | − hci->io->dequeue_xfer(hci, xfer, nxfers)) { | |
| 264 | − ret = -ETIME; | |
| 265 | − goto out; | |
| 266 | − } | |
| 267 | 327 | for (i = prefixed; i < nxfers; i++) { |
| 268 | 328 | if (ccc->rnw) |
| 269 | 329 | ccc->dests[i - prefixed].payload.len = |
| @@ -334,15 +394,11 @@ static int i3c_hci_i3c_xfers(struct i3c_dev_desc *dev, · 64 unmodified lines | ||
| 334 | 394 | last = i - 1; |
| 335 | 395 | xfer[last].cmd_desc[0] |= CMD_0_TOC; |
| 336 | 396 | xfer[last].completion = &done; |
| 397 | + xfer[last].timeout = HZ; | |
| 337 | 398 | |
| 338 | − ret = hci->io->queue_xfer(hci, xfer, nxfers); | |
| 399 | + ret = i3c_hci_process_xfer(hci, xfer, nxfers); | |
| 339 | 400 | if (ret) |
| 340 | 401 | goto out; |
| 341 | − if (!wait_for_completion_timeout(&done, HZ) && | |
| 342 | − hci->io->dequeue_xfer(hci, xfer, nxfers)) { | |
| 343 | − ret = -ETIME; | |
| 344 | − goto out; | |
| 345 | − } | |
| 346 | 402 | for (i = 0; i < nxfers; i++) { |
| 347 | 403 | if (i3c_xfers[i].rnw) |
| 348 | 404 | i3c_xfers[i].len = RESP_DATA_LENGTH(xfer[i].response); |
| @@ -382,15 +438,11 @@ static int i3c_hci_i2c_xfers(struct i2c_dev_desc *dev, · 33 unmodified lines | ||
| 382 | 438 | last = i - 1; |
| 383 | 439 | xfer[last].cmd_desc[0] |= CMD_0_TOC; |
| 384 | 440 | xfer[last].completion = &done; |
| 441 | + xfer[last].timeout = m->i2c.timeout; | |
| 385 | 442 | |
| 386 | − ret = hci->io->queue_xfer(hci, xfer, nxfers); | |
| 443 | + ret = i3c_hci_process_xfer(hci, xfer, nxfers); | |
| 387 | 444 | if (ret) |
| 388 | 445 | goto out; |
| 389 | − if (!wait_for_completion_timeout(&done, m->i2c.timeout) && | |
| 390 | − hci->io->dequeue_xfer(hci, xfer, nxfers)) { | |
| 391 | − ret = -ETIME; | |
| 392 | − goto out; | |
| 393 | − } | |
| 394 | 446 | for (i = 0; i < nxfers; i++) { |
| 395 | 447 | if (RESP_STATUS(xfer[i].response) != RESP_SUCCESS) { |
| 396 | 448 | ret = -EIO; |
| @@ -566,6 +618,8 @@ static irqreturn_t i3c_hci_irq_handler(int irq, void *dev_id) · 169 unmodified lines | ||
| 566 | 618 | irqreturn_t result = IRQ_NONE; |
| 567 | 619 | u32 val; |
| 568 | 620 | |
| 621 | + guard(spinlock)(&hci->lock); | |
| 622 | + | |
| 569 | 623 | /* |
| 570 | 624 | * The IRQ can be shared, so the handler may be called when the IRQ is |
| 571 | 625 | * due to a different device. That could happen when runtime suspended, |
| @@ -601,34 +655,6 @@ static irqreturn_t i3c_hci_irq_handler(int irq, void *dev_id) · 29 unmodified lines | ||
| 601 | 655 | return result; |
| 602 | 656 | } |
| 603 | 657 | |
| 604 | −static int i3c_hci_software_reset(struct i3c_hci *hci) | |
| 605 | −{ | |
| 606 | − u32 regval; | |
| 607 | − int ret; | |
| 608 | − | |
| 609 | − /* | |
| 610 | − * SOFT_RST must be clear before we write to it. | |
| 611 | − * Then we must wait until it clears again. | |
| 612 | − */ | |
| 613 | − ret = readx_poll_timeout(reg_read, RESET_CONTROL, regval, | |
| 614 | − !(regval & SOFT_RST), 0, 10 * USEC_PER_MSEC); | |
| 615 | − if (ret) { | |
| 616 | − dev_err(&hci->master.dev, "%s: Software reset stuck\n", __func__); | |
| 617 | − return ret; | |
| 618 | − } | |
| 619 | − | |
| 620 | − reg_write(RESET_CONTROL, SOFT_RST); | |
| 621 | − | |
| 622 | − ret = readx_poll_timeout(reg_read, RESET_CONTROL, regval, | |
| 623 | − !(regval & SOFT_RST), 0, 10 * USEC_PER_MSEC); | |
| 624 | − if (ret) { | |
| 625 | − dev_err(&hci->master.dev, "%s: Software reset failed\n", __func__); | |
| 626 | − return ret; | |
| 627 | − } | |
| 628 | − | |
| 629 | − return 0; | |
| 630 | −} | |
| 631 | − | |
| 632 | 658 | static inline bool is_version_1_1_or_newer(struct i3c_hci *hci) |
| 633 | 659 | { |
| 634 | 660 | return hci->version_major > 1 || (hci->version_major == 1 && hci->version_minor > 0); |
| @@ -739,8 +765,12 @@ static int i3c_hci_runtime_suspend(struct device *dev) · 104 unmodified lines | ||
| 739 | 765 | int ret; |
| 740 | 766 | |
| 741 | 767 | ret = i3c_hci_bus_disable(hci); |
| 742 | − if (ret) | |
| 768 | + if (ret) { | |
| 769 | + /* Fall back to software reset to disable the bus */ | |
| 770 | + ret = i3c_hci_software_reset(hci); | |
| 771 | + i3c_hci_sync_irq_inactive(hci); | |
| 743 | 772 | return ret; |
| 773 | + } | |
| 744 | 774 | |
| 745 | 775 | hci->io->suspend(hci); |
| 746 | 776 | |
| @@ -760,11 +790,13 @@ static int i3c_hci_runtime_resume(struct device *dev) · 13 unmodified lines | ||
| 760 | 790 | |
| 761 | 791 | mipi_i3c_hci_dat_v1.restore(hci); |
| 762 | 792 | |
| 763 | − hci->irq_inactive = false; | |
| 764 | − | |
| 765 | 793 | hci->io->resume(hci); |
| 766 | 794 | |
| 767 | − reg_set(HC_CONTROL, HC_CONTROL_BUS_ENABLE); | |
| 795 | + scoped_guard(spinlock_irqsave, &hci->lock) | |
| 796 | + hci->irq_inactive = false; | |
| 797 | + | |
| 798 | + /* Enable bus with Hot-Join disabled */ | |
| 799 | + reg_set(HC_CONTROL, HC_CONTROL_BUS_ENABLE | HC_CONTROL_HOT_JOIN_CTRL); | |
| 768 | 800 | |
| 769 | 801 | return 0; |
| 770 | 802 | } |
| @@ -924,6 +956,9 @@ static int i3c_hci_probe(struct platform_device *pdev) · 153 unmodified lines | ||
| 924 | 956 | if (!hci) |
| 925 | 957 | return -ENOMEM; |
| 926 | 958 | |
| 959 | + spin_lock_init(&hci->lock); | |
| 960 | + mutex_init(&hci->control_mutex); | |
| 961 | + | |
| 927 | 962 | /* |
| 928 | 963 | * Multi-bus instances share the same MMIO address range, but not |
| 929 | 964 | * necessarily in separate contiguous sub-ranges. To avoid overlapping |
| @@ -950,6 +985,8 @@ static int i3c_hci_probe(struct platform_device *pdev) · 20 unmodified lines | ||
| 950 | 985 | if (ret) |
| 951 | 986 | return ret; |
| 952 | 987 | |
| 988 | + hci->irq_inactive = true; | |
| 989 | + | |
| 953 | 990 | irq = platform_get_irq(pdev, 0); |
| 954 | 991 | ret = devm_request_irq(&pdev->dev, irq, i3c_hci_irq_handler, |
| 955 | 992 | IRQF_SHARED, NULL, hci); |
| @@ -129,9 +129,8 @@ struct hci_rh_data { · 128 unmodified lines | ||
| 129 | 129 | dma_addr_t xfer_dma, resp_dma, ibi_status_dma, ibi_data_dma; |
| 130 | 130 | unsigned int xfer_entries, ibi_status_entries, ibi_chunks_total; |
| 131 | 131 | unsigned int xfer_struct_sz, resp_struct_sz, ibi_status_sz, ibi_chunk_sz; |
| 132 | − unsigned int done_ptr, ibi_chunk_ptr; | |
| 132 | + unsigned int done_ptr, ibi_chunk_ptr, xfer_space; | |
| 133 | 133 | struct hci_xfer **src_xfers; |
| 134 | − spinlock_t lock; | |
| 135 | 134 | struct completion op_done; |
| 136 | 135 | }; |
| 137 | 136 | |
| @@ -261,6 +260,7 @@ ring_ready: · 123 unmodified lines | ||
| 261 | 260 | |
| 262 | 261 | rh->done_ptr = 0; |
| 263 | 262 | rh->ibi_chunk_ptr = 0; |
| 263 | + rh->xfer_space = rh->xfer_entries; | |
| 264 | 264 | } |
| 265 | 265 | |
| 266 | 266 | static void hci_dma_init_rings(struct i3c_hci *hci) |
| @@ -344,7 +344,6 @@ static int hci_dma_init(struct i3c_hci *hci) · 77 unmodified lines | ||
| 344 | 344 | goto err_out; |
| 345 | 345 | rh = &rings->headers[i]; |
| 346 | 346 | rh->regs = hci->base_regs + offset; |
| 347 | − spin_lock_init(&rh->lock); | |
| 348 | 347 | init_completion(&rh->op_done); |
| 349 | 348 | |
| 350 | 349 | rh->xfer_entries = XFER_RING_ENTRIES; |
| @@ -439,26 +438,63 @@ static void hci_dma_unmap_xfer(struct i3c_hci *hci, · 88 unmodified lines | ||
| 439 | 438 | } |
| 440 | 439 | } |
| 441 | 440 | |
| 441 | +static struct i3c_dma *hci_dma_map_xfer(struct device *dev, struct hci_xfer *xfer) | |
| 442 | +{ | |
| 443 | + enum dma_data_direction dir = xfer->rnw ? DMA_FROM_DEVICE : DMA_TO_DEVICE; | |
| 444 | + bool need_bounce = device_iommu_mapped(dev) && xfer->rnw && (xfer->data_len & 3); | |
| 445 | + | |
| 446 | + return i3c_master_dma_map_single(dev, xfer->data, xfer->data_len, need_bounce, dir); | |
| 447 | +} | |
| 448 | + | |
| 449 | +static int hci_dma_map_xfer_list(struct i3c_hci *hci, struct device *dev, | |
| 450 | + struct hci_xfer *xfer_list, int n) | |
| 451 | +{ | |
| 452 | + for (int i = 0; i < n; i++) { | |
| 453 | + struct hci_xfer *xfer = xfer_list + i; | |
| 454 | + | |
| 455 | + if (!xfer->data) | |
| 456 | + continue; | |
| 457 | + | |
| 458 | + xfer->dma = hci_dma_map_xfer(dev, xfer); | |
| 459 | + if (!xfer->dma) { | |
| 460 | + hci_dma_unmap_xfer(hci, xfer_list, i); | |
| 461 | + return -ENOMEM; | |
| 462 | + } | |
| 463 | + } | |
| 464 | + | |
| 465 | + return 0; | |
| 466 | +} | |
| 467 | + | |
| 442 | 468 | static int hci_dma_queue_xfer(struct i3c_hci *hci, |
| 443 | 469 | struct hci_xfer *xfer_list, int n) |
| 444 | 470 | { |
| 445 | 471 | struct hci_rings_data *rings = hci->io_data; |
| 446 | 472 | struct hci_rh_data *rh; |
| 447 | 473 | unsigned int i, ring, enqueue_ptr; |
| 448 | − u32 op1_val, op2_val; | |
| 474 | + u32 op1_val; | |
| 475 | + int ret; | |
| 476 | + | |
| 477 | + ret = hci_dma_map_xfer_list(hci, rings->sysdev, xfer_list, n); | |
| 478 | + if (ret) | |
| 479 | + return ret; | |
| 449 | 480 | |
| 450 | 481 | /* For now we only use ring 0 */ |
| 451 | 482 | ring = 0; |
| 452 | 483 | rh = &rings->headers[ring]; |
| 453 | 484 | |
| 485 | + spin_lock_irq(&hci->lock); | |
| 486 | + | |
| 487 | + if (n > rh->xfer_space) { | |
| 488 | + spin_unlock_irq(&hci->lock); | |
| 489 | + hci_dma_unmap_xfer(hci, xfer_list, n); | |
| 490 | + return -EBUSY; | |
| 491 | + } | |
| 492 | + | |
| 454 | 493 | op1_val = rh_reg_read(RING_OPERATION1); |
| 455 | 494 | enqueue_ptr = FIELD_GET(RING_OP1_CR_ENQ_PTR, op1_val); |
| 456 | 495 | for (i = 0; i < n; i++) { |
| 457 | 496 | struct hci_xfer *xfer = xfer_list + i; |
| 458 | 497 | u32 *ring_data = rh->xfer + rh->xfer_struct_sz * enqueue_ptr; |
| 459 | − enum dma_data_direction dir = xfer->rnw ? DMA_FROM_DEVICE : | |
| 460 | − DMA_TO_DEVICE; | |
| 461 | − bool need_bounce; | |
| 462 | 498 | |
| 463 | 499 | /* store cmd descriptor */ |
| 464 | 500 | *ring_data++ = xfer->cmd_desc[0]; |
| @@ -477,18 +513,6 @@ static int hci_dma_queue_xfer(struct i3c_hci *hci, · 12 unmodified lines | ||
| 477 | 513 | |
| 478 | 514 | /* 2nd and 3rd words of Data Buffer Descriptor Structure */ |
| 479 | 515 | if (xfer->data) { |
| 480 | − need_bounce = device_iommu_mapped(rings->sysdev) && | |
| 481 | − xfer->rnw && | |
| 482 | − xfer->data_len != ALIGN(xfer->data_len, 4); | |
| 483 | − xfer->dma = i3c_master_dma_map_single(rings->sysdev, | |
| 484 | − xfer->data, | |
| 485 | − xfer->data_len, | |
| 486 | − need_bounce, | |
| 487 | − dir); | |
| 488 | − if (!xfer->dma) { | |
| 489 | − hci_dma_unmap_xfer(hci, xfer_list, i); | |
| 490 | − return -ENOMEM; | |
| 491 | − } | |
| 492 | 516 | *ring_data++ = lower_32_bits(xfer->dma->addr); |
| 493 | 517 | *ring_data++ = upper_32_bits(xfer->dma->addr); |
| 494 | 518 | } else { |
| @@ -503,26 +527,14 @@ static int hci_dma_queue_xfer(struct i3c_hci *hci, · 8 unmodified lines | ||
| 503 | 527 | xfer->ring_entry = enqueue_ptr; |
| 504 | 528 | |
| 505 | 529 | enqueue_ptr = (enqueue_ptr + 1) % rh->xfer_entries; |
| 506 | − | |
| 507 | − /* | |
| 508 | − * We may update the hardware view of the enqueue pointer | |
| 509 | − * only if we didn't reach its dequeue pointer. | |
| 510 | − */ | |
| 511 | − op2_val = rh_reg_read(RING_OPERATION2); | |
| 512 | − if (enqueue_ptr == FIELD_GET(RING_OP2_CR_DEQ_PTR, op2_val)) { | |
| 513 | − /* the ring is full */ | |
| 514 | − hci_dma_unmap_xfer(hci, xfer_list, i + 1); | |
| 515 | − return -EBUSY; | |
| 516 | − } | |
| 517 | 530 | } |
| 518 | 531 | |
| 519 | − /* take care to update the hardware enqueue pointer atomically */ | |
| 520 | − spin_lock_irq(&rh->lock); | |
| 521 | − op1_val = rh_reg_read(RING_OPERATION1); | |
| 532 | + rh->xfer_space -= n; | |
| 533 | + | |
| 522 | 534 | op1_val &= ~RING_OP1_CR_ENQ_PTR; |
| 523 | 535 | op1_val |= FIELD_PREP(RING_OP1_CR_ENQ_PTR, enqueue_ptr); |
| 524 | 536 | rh_reg_write(RING_OPERATION1, op1_val); |
| 525 | − spin_unlock_irq(&rh->lock); | |
| 537 | + spin_unlock_irq(&hci->lock); | |
| 526 | 538 | |
| 527 | 539 | return 0; |
| 528 | 540 | } |
| @@ -534,18 +546,29 @@ static bool hci_dma_dequeue_xfer(struct i3c_hci *hci, · 5 unmodified lines | ||
| 534 | 546 | struct hci_rh_data *rh = &rings->headers[xfer_list[0].ring_number]; |
| 535 | 547 | unsigned int i; |
| 536 | 548 | bool did_unqueue = false; |
| 537 | − | |
| 538 | − /* stop the ring */ | |
| 539 | − rh_reg_write(RING_CONTROL, RING_CTRL_ABORT); | |
| 540 | − if (wait_for_completion_timeout(&rh->op_done, HZ) == 0) { | |
| 541 | − /* | |
| 542 | − * We're deep in it if ever this condition is ever met. | |
| 543 | − * Hardware might still be writing to memory, etc. | |
| 544 | − */ | |
| 545 | − dev_crit(&hci->master.dev, "unable to abort the ring\n"); | |
| 546 | − WARN_ON(1); | |
| 549 | + u32 ring_status; | |
| 550 | + | |
| 551 | + guard(mutex)(&hci->control_mutex); | |
| 552 | + | |
| 553 | + ring_status = rh_reg_read(RING_STATUS); | |
| 554 | + if (ring_status & RING_STATUS_RUNNING) { | |
| 555 | + /* stop the ring */ | |
| 556 | + reinit_completion(&rh->op_done); | |
| 557 | + rh_reg_write(RING_CONTROL, RING_CTRL_ENABLE | RING_CTRL_ABORT); | |
| 558 | + wait_for_completion_timeout(&rh->op_done, HZ); | |
| 559 | + ring_status = rh_reg_read(RING_STATUS); | |
| 560 | + if (ring_status & RING_STATUS_RUNNING) { | |
| 561 | + /* | |
| 562 | + * We're deep in it if ever this condition is ever met. | |
| 563 | + * Hardware might still be writing to memory, etc. | |
| 564 | + */ | |
| 565 | + dev_crit(&hci->master.dev, "unable to abort the ring\n"); | |
| 566 | + WARN_ON(1); | |
| 567 | + } | |
| 547 | 568 | } |
| 548 | 569 | |
| 570 | + spin_lock_irq(&hci->lock); | |
| 571 | + | |
| 549 | 572 | for (i = 0; i < n; i++) { |
| 550 | 573 | struct hci_xfer *xfer = xfer_list + i; |
| 551 | 574 | int idx = xfer->ring_entry; |
| @@ -559,7 +582,7 @@ static bool hci_dma_dequeue_xfer(struct i3c_hci *hci, · 7 unmodified lines | ||
| 559 | 582 | u32 *ring_data = rh->xfer + rh->xfer_struct_sz * idx; |
| 560 | 583 | |
| 561 | 584 | /* store no-op cmd descriptor */ |
| 562 | − *ring_data++ = FIELD_PREP(CMD_0_ATTR, 0x7); | |
| 585 | + *ring_data++ = FIELD_PREP(CMD_0_ATTR, 0x7) | FIELD_PREP(CMD_0_TID, xfer->cmd_tid); | |
| 563 | 586 | *ring_data++ = 0; |
| 564 | 587 | if (hci->cmd == &mipi_i3c_hci_cmd_v2) { |
| 565 | 588 | *ring_data++ = 0; |
| @@ -577,15 +600,25 @@ static bool hci_dma_dequeue_xfer(struct i3c_hci *hci, · 11 unmodified lines | ||
| 577 | 600 | } |
| 578 | 601 | |
| 579 | 602 | /* restart the ring */ |
| 603 | + mipi_i3c_hci_resume(hci); | |
| 580 | 604 | rh_reg_write(RING_CONTROL, RING_CTRL_ENABLE); |
| 605 | + rh_reg_write(RING_CONTROL, RING_CTRL_ENABLE | RING_CTRL_RUN_STOP); | |
| 606 | + | |
| 607 | + spin_unlock_irq(&hci->lock); | |
| 581 | 608 | |
| 582 | 609 | return did_unqueue; |
| 583 | 610 | } |
| 584 | 611 | |
| 612 | +static int hci_dma_handle_error(struct i3c_hci *hci, struct hci_xfer *xfer_list, int n) | |
| 613 | +{ | |
| 614 | + return hci_dma_dequeue_xfer(hci, xfer_list, n) ? -EIO : 0; | |
| 615 | +} | |
| 616 | + | |
| 585 | 617 | static void hci_dma_xfer_done(struct i3c_hci *hci, struct hci_rh_data *rh) |
| 586 | 618 | { |
| 587 | 619 | u32 op1_val, op2_val, resp, *ring_resp; |
| 588 | 620 | unsigned int tid, done_ptr = rh->done_ptr; |
| 621 | + unsigned int done_cnt = 0; | |
| 589 | 622 | struct hci_xfer *xfer; |
| 590 | 623 | |
| 591 | 624 | for (;;) { |
| @@ -603,6 +636,7 @@ static void hci_dma_xfer_done(struct i3c_hci *hci, struct hci_rh_data *rh) · 11 unmodified lines | ||
| 603 | 636 | dev_dbg(&hci->master.dev, "orphaned ring entry"); |
| 604 | 637 | } else { |
| 605 | 638 | hci_dma_unmap_xfer(hci, xfer, 1); |
| 639 | + rh->src_xfers[done_ptr] = NULL; | |
| 606 | 640 | xfer->ring_entry = -1; |
| 607 | 641 | xfer->response = resp; |
| 608 | 642 | if (tid != xfer->cmd_tid) { |
| @@ -617,15 +651,14 @@ static void hci_dma_xfer_done(struct i3c_hci *hci, struct hci_rh_data *rh) · 8 unmodified lines | ||
| 617 | 651 | |
| 618 | 652 | done_ptr = (done_ptr + 1) % rh->xfer_entries; |
| 619 | 653 | rh->done_ptr = done_ptr; |
| 654 | + done_cnt += 1; | |
| 620 | 655 | } |
| 621 | 656 | |
| 622 | − /* take care to update the software dequeue pointer atomically */ | |
| 623 | − spin_lock(&rh->lock); | |
| 657 | + rh->xfer_space += done_cnt; | |
| 624 | 658 | op1_val = rh_reg_read(RING_OPERATION1); |
| 625 | 659 | op1_val &= ~RING_OP1_CR_SW_DEQ_PTR; |
| 626 | 660 | op1_val |= FIELD_PREP(RING_OP1_CR_SW_DEQ_PTR, done_ptr); |
| 627 | 661 | rh_reg_write(RING_OPERATION1, op1_val); |
| 628 | − spin_unlock(&rh->lock); | |
| 629 | 662 | } |
| 630 | 663 | |
| 631 | 664 | static int hci_dma_request_ibi(struct i3c_hci *hci, struct i3c_dev_desc *dev, |
| @@ -805,13 +838,10 @@ static void hci_dma_process_ibi(struct i3c_hci *hci, struct hci_rh_data *rh) · 173 unmodified lines | ||
| 805 | 838 | i3c_master_queue_ibi(dev, slot); |
| 806 | 839 | |
| 807 | 840 | done: |
| 808 | − /* take care to update the ibi dequeue pointer atomically */ | |
| 809 | − spin_lock(&rh->lock); | |
| 810 | 841 | op1_val = rh_reg_read(RING_OPERATION1); |
| 811 | 842 | op1_val &= ~RING_OP1_IBI_DEQ_PTR; |
| 812 | 843 | op1_val |= FIELD_PREP(RING_OP1_IBI_DEQ_PTR, deq_ptr); |
| 813 | 844 | rh_reg_write(RING_OPERATION1, op1_val); |
| 814 | − spin_unlock(&rh->lock); | |
| 815 | 845 | |
| 816 | 846 | /* update the chunk pointer */ |
| 817 | 847 | rh->ibi_chunk_ptr += ibi_chunks; |
| @@ -845,29 +875,8 @@ static bool hci_dma_irq_handler(struct i3c_hci *hci) · 27 unmodified lines | ||
| 845 | 875 | hci_dma_xfer_done(hci, rh); |
| 846 | 876 | if (status & INTR_RING_OP) |
| 847 | 877 | complete(&rh->op_done); |
| 848 | − | |
| 849 | − if (status & INTR_TRANSFER_ABORT) { | |
| 850 | − u32 ring_status; | |
| 851 | − | |
| 852 | − dev_notice_ratelimited(&hci->master.dev, | |
| 853 | − "Ring %d: Transfer Aborted\n", i); | |
| 854 | − mipi_i3c_hci_resume(hci); | |
| 855 | − ring_status = rh_reg_read(RING_STATUS); | |
| 856 | − if (!(ring_status & RING_STATUS_RUNNING) && | |
| 857 | − status & INTR_TRANSFER_COMPLETION && | |
| 858 | − status & INTR_TRANSFER_ERR) { | |
| 859 | − /* | |
| 860 | − * Ring stop followed by run is an Intel | |
| 861 | − * specific required quirk after resuming the | |
| 862 | − * halted controller. Do it only when the ring | |
| 863 | − * is not in running state after a transfer | |
| 864 | − * error. | |
| 865 | − */ | |
| 866 | − rh_reg_write(RING_CONTROL, RING_CTRL_ENABLE); | |
| 867 | − rh_reg_write(RING_CONTROL, RING_CTRL_ENABLE | | |
| 868 | − RING_CTRL_RUN_STOP); | |
| 869 | − } | |
| 870 | − } | |
| 878 | + if (status & INTR_TRANSFER_ABORT) | |
| 879 | + dev_dbg(&hci->master.dev, "Ring %d: Transfer Aborted\n", i); | |
| 871 | 880 | if (status & INTR_IBI_RING_FULL) |
| 872 | 881 | dev_err_ratelimited(&hci->master.dev, |
| 873 | 882 | "Ring %d: IBI Ring Full Condition\n", i); |
| @@ -883,6 +892,7 @@ const struct hci_io_ops mipi_i3c_hci_dma = { · 9 unmodified lines | ||
| 883 | 892 | .cleanup = hci_dma_cleanup, |
| 884 | 893 | .queue_xfer = hci_dma_queue_xfer, |
| 885 | 894 | .dequeue_xfer = hci_dma_dequeue_xfer, |
| 895 | + .handle_error = hci_dma_handle_error, | |
| 886 | 896 | .irq_handler = hci_dma_irq_handler, |
| 887 | 897 | .request_ibi = hci_dma_request_ibi, |
| 888 | 898 | .free_ibi = hci_dma_free_ibi, |
| @@ -50,6 +50,8 @@ struct i3c_hci { · 49 unmodified lines | ||
| 50 | 50 | const struct hci_io_ops *io; |
| 51 | 51 | void *io_data; |
| 52 | 52 | const struct hci_cmd_ops *cmd; |
| 53 | + spinlock_t lock; | |
| 54 | + struct mutex control_mutex; | |
| 53 | 55 | atomic_t next_cmd_tid; |
| 54 | 56 | bool irq_inactive; |
| 55 | 57 | u32 caps; |
| @@ -87,6 +89,7 @@ struct hci_xfer { · 31 unmodified lines | ||
| 87 | 89 | unsigned int data_len; |
| 88 | 90 | unsigned int cmd_tid; |
| 89 | 91 | struct completion *completion; |
| 92 | + unsigned long timeout; | |
| 90 | 93 | union { |
| 91 | 94 | struct { |
| 92 | 95 | /* PIO specific */ |
| @@ -120,6 +123,7 @@ struct hci_io_ops { · 27 unmodified lines | ||
| 120 | 123 | bool (*irq_handler)(struct i3c_hci *hci); |
| 121 | 124 | int (*queue_xfer)(struct i3c_hci *hci, struct hci_xfer *xfer, int n); |
| 122 | 125 | bool (*dequeue_xfer)(struct i3c_hci *hci, struct hci_xfer *xfer, int n); |
| 126 | + int (*handle_error)(struct i3c_hci *hci, struct hci_xfer *xfer, int n); | |
| 123 | 127 | int (*request_ibi)(struct i3c_hci *hci, struct i3c_dev_desc *dev, |
| 124 | 128 | const struct i3c_ibi_setup *req); |
| 125 | 129 | void (*free_ibi)(struct i3c_hci *hci, struct i3c_dev_desc *dev); |
| @@ -154,5 +158,6 @@ void mipi_i3c_hci_dct_index_reset(struct i3c_hci *hci); · 28 unmodified lines | ||
| 154 | 158 | void amd_set_od_pp_timing(struct i3c_hci *hci); |
| 155 | 159 | void amd_set_resp_buf_thld(struct i3c_hci *hci); |
| 156 | 160 | void i3c_hci_sync_irq_inactive(struct i3c_hci *hci); |
| 161 | +int i3c_hci_process_xfer(struct i3c_hci *hci, struct hci_xfer *xfer, int n); | |
| 157 | 162 | |
| 158 | 163 | #endif |
| @@ -123,7 +123,6 @@ struct hci_pio_ibi_data { · 122 unmodified lines | ||
| 123 | 123 | }; |
| 124 | 124 | |
| 125 | 125 | struct hci_pio_data { |
| 126 | − spinlock_t lock; | |
| 127 | 126 | struct hci_xfer *curr_xfer, *xfer_queue; |
| 128 | 127 | struct hci_xfer *curr_rx, *rx_queue; |
| 129 | 128 | struct hci_xfer *curr_tx, *tx_queue; |
| @@ -212,7 +211,6 @@ static int hci_pio_init(struct i3c_hci *hci) · 82 unmodified lines | ||
| 212 | 211 | return -ENOMEM; |
| 213 | 212 | |
| 214 | 213 | hci->io_data = pio; |
| 215 | − spin_lock_init(&pio->lock); | |
| 216 | 214 | |
| 217 | 215 | __hci_pio_init(hci, &size_val); |
| 218 | 216 | |
| @@ -631,7 +629,7 @@ static int hci_pio_queue_xfer(struct i3c_hci *hci, struct hci_xfer *xfer, int n) · 412 unmodified lines | ||
| 631 | 629 | xfer[i].data_left = xfer[i].data_len; |
| 632 | 630 | } |
| 633 | 631 | |
| 634 | − spin_lock_irq(&pio->lock); | |
| 632 | + spin_lock_irq(&hci->lock); | |
| 635 | 633 | prev_queue_tail = pio->xfer_queue; |
| 636 | 634 | pio->xfer_queue = &xfer[n - 1]; |
| 637 | 635 | if (pio->curr_xfer) { |
| @@ -645,7 +643,7 @@ static int hci_pio_queue_xfer(struct i3c_hci *hci, struct hci_xfer *xfer, int n) · 7 unmodified lines | ||
| 645 | 643 | pio_reg_read(INTR_STATUS), |
| 646 | 644 | pio_reg_read(INTR_SIGNAL_ENABLE)); |
| 647 | 645 | } |
| 648 | − spin_unlock_irq(&pio->lock); | |
| 646 | + spin_unlock_irq(&hci->lock); | |
| 649 | 647 | return 0; |
| 650 | 648 | } |
| 651 | 649 | |
| @@ -716,14 +714,14 @@ static bool hci_pio_dequeue_xfer(struct i3c_hci *hci, struct hci_xfer *xfer, int · 64 unmodified lines | ||
| 716 | 714 | struct hci_pio_data *pio = hci->io_data; |
| 717 | 715 | int ret; |
| 718 | 716 | |
| 719 | − spin_lock_irq(&pio->lock); | |
| 717 | + spin_lock_irq(&hci->lock); | |
| 720 | 718 | dev_dbg(&hci->master.dev, "n=%d status=%#x/%#x", n, |
| 721 | 719 | pio_reg_read(INTR_STATUS), pio_reg_read(INTR_SIGNAL_ENABLE)); |
| 722 | 720 | dev_dbg(&hci->master.dev, "main_status = %#x/%#x", |
| 723 | 721 | readl(hci->base_regs + 0x20), readl(hci->base_regs + 0x28)); |
| 724 | 722 | |
| 725 | 723 | ret = hci_pio_dequeue_xfer_common(hci, pio, xfer, n); |
| 726 | − spin_unlock_irq(&pio->lock); | |
| 724 | + spin_unlock_irq(&hci->lock); | |
| 727 | 725 | return ret; |
| 728 | 726 | } |
| 729 | 727 | |
| @@ -1016,15 +1014,12 @@ static bool hci_pio_irq_handler(struct i3c_hci *hci) · 286 unmodified lines | ||
| 1016 | 1014 | struct hci_pio_data *pio = hci->io_data; |
| 1017 | 1015 | u32 status; |
| 1018 | 1016 | |
| 1019 | − spin_lock(&pio->lock); | |
| 1020 | 1017 | status = pio_reg_read(INTR_STATUS); |
| 1021 | 1018 | dev_dbg(&hci->master.dev, "PIO_INTR_STATUS %#x/%#x", |
| 1022 | 1019 | status, pio->enabled_irqs); |
| 1023 | 1020 | status &= pio->enabled_irqs | STAT_LATENCY_WARNINGS; |
| 1024 | − if (!status) { | |
| 1025 | − spin_unlock(&pio->lock); | |
| 1021 | + if (!status) | |
| 1026 | 1022 | return false; |
| 1027 | − } | |
| 1028 | 1023 | |
| 1029 | 1024 | if (status & STAT_IBI_STATUS_THLD) |
| 1030 | 1025 | hci_pio_process_ibi(hci, pio); |
| @@ -1058,7 +1053,6 @@ static bool hci_pio_irq_handler(struct i3c_hci *hci) · 27 unmodified lines | ||
| 1058 | 1053 | pio_reg_write(INTR_SIGNAL_ENABLE, pio->enabled_irqs); |
| 1059 | 1054 | dev_dbg(&hci->master.dev, "PIO_INTR_STATUS %#x/%#x", |
| 1060 | 1055 | pio_reg_read(INTR_STATUS), pio_reg_read(INTR_SIGNAL_ENABLE)); |
| 1061 | − spin_unlock(&pio->lock); | |
| 1062 | 1056 | return true; |
| 1063 | 1057 | } |
| 1064 | 1058 | |
| @@ -541,7 +541,7 @@ static int ad7768_reg_access(struct iio_dev *indio_dev, · 540 unmodified lines | ||
| 541 | 541 | return ret; |
| 542 | 542 | } |
| 543 | 543 | |
| 544 | −static void ad7768_fill_scale_tbl(struct iio_dev *dev) | |
| 544 | +static int ad7768_fill_scale_tbl(struct iio_dev *dev) | |
| 545 | 545 | { |
| 546 | 546 | struct ad7768_state *st = iio_priv(dev); |
| 547 | 547 | const struct iio_scan_type *scan_type; |
| @@ -551,6 +551,11 @@ static void ad7768_fill_scale_tbl(struct iio_dev *dev) · 3 unmodified lines | ||
| 551 | 551 | u64 tmp2; |
| 552 | 552 | |
| 553 | 553 | scan_type = iio_get_current_scan_type(dev, &dev->channels[0]); |
| 554 | + if (IS_ERR(scan_type)) { | |
| 555 | + dev_err(&st->spi->dev, "Failed to get scan type.\n"); | |
| 556 | + return PTR_ERR(scan_type); | |
| 557 | + } | |
| 558 | + | |
| 554 | 559 | if (scan_type->sign == 's') |
| 555 | 560 | val2 = scan_type->realbits - 1; |
| 556 | 561 | else |
| @@ -575,6 +580,8 @@ static void ad7768_fill_scale_tbl(struct iio_dev *dev) · 18 unmodified lines | ||
| 575 | 580 | st->scale_tbl[i][0] = tmp0; /* Integer part */ |
| 576 | 581 | st->scale_tbl[i][1] = abs(tmp1); /* Fractional part */ |
| 577 | 582 | } |
| 583 | + | |
| 584 | + return 0; | |
| 578 | 585 | } |
| 579 | 586 | |
| 580 | 587 | static int ad7768_set_sinc3_dec_rate(struct ad7768_state *st, |
| @@ -679,7 +686,9 @@ static int ad7768_configure_dig_fil(struct iio_dev *dev, · 98 unmodified lines | ||
| 679 | 686 | } |
| 680 | 687 | |
| 681 | 688 | /* Update scale table: scale values vary according to the precision */ |
| 682 | − ad7768_fill_scale_tbl(dev); | |
| 689 | + ret = ad7768_fill_scale_tbl(dev); | |
| 690 | + if (ret) | |
| 691 | + return ret; | |
| 683 | 692 | |
| 684 | 693 | ad7768_fill_samp_freq_tbl(st); |
| 685 | 694 | |
| @@ -613,7 +613,7 @@ static int bme680_wait_for_eoc(struct bme680_data *data) · 612 unmodified lines | ||
| 613 | 613 | * + heater duration |
| 614 | 614 | */ |
| 615 | 615 | int wait_eoc_us = ((data->oversampling_temp + data->oversampling_press + |
| 616 | − data->oversampling_humid) * 1936) + (477 * 4) + | |
| 616 | + data->oversampling_humid) * 1963) + (477 * 4) + | |
| 617 | 617 | (477 * 5) + 1000 + (data->heater_dur * 1000); |
| 618 | 618 | |
| 619 | 619 | fsleep(wait_eoc_us); |
| @@ -171,7 +171,7 @@ static int sps30_i2c_read_meas(struct sps30_state *state, __be32 *meas, size_t n · 170 unmodified lines | ||
| 171 | 171 | if (!sps30_i2c_meas_ready(state)) |
| 172 | 172 | return -ETIMEDOUT; |
| 173 | 173 | |
| 174 | − return sps30_i2c_command(state, SPS30_I2C_READ_MEAS, NULL, 0, meas, sizeof(num) * num); | |
| 174 | + return sps30_i2c_command(state, SPS30_I2C_READ_MEAS, NULL, 0, meas, sizeof(*meas) * num); | |
| 175 | 175 | } |
| 176 | 176 | |
| 177 | 177 | static int sps30_i2c_clean_fan(struct sps30_state *state) |
| @@ -303,7 +303,7 @@ static int sps30_serial_read_meas(struct sps30_state *state, __be32 *meas, size_ · 302 unmodified lines | ||
| 303 | 303 | if (msleep_interruptible(1000)) |
| 304 | 304 | return -EINTR; |
| 305 | 305 | |
| 306 | − ret = sps30_serial_command(state, SPS30_SERIAL_READ_MEAS, NULL, 0, meas, num * sizeof(num)); | |
| 306 | + ret = sps30_serial_command(state, SPS30_SERIAL_READ_MEAS, NULL, 0, meas, num * sizeof(*meas)); | |
| 307 | 307 | if (ret < 0) |
| 308 | 308 | return ret; |
| 309 | 309 | /* if measurements aren't ready sensor returns empty frame */ |
| @@ -140,7 +140,7 @@ static int ds4424_write_raw(struct iio_dev *indio_dev, · 139 unmodified lines | ||
| 140 | 140 | |
| 141 | 141 | switch (mask) { |
| 142 | 142 | case IIO_CHAN_INFO_RAW: |
| 143 | − if (val < S8_MIN || val > S8_MAX) | |
| 143 | + if (val <= S8_MIN || val > S8_MAX) | |
| 144 | 144 | return -EINVAL; |
| 145 | 145 | |
| 146 | 146 | if (val > 0) { |
| @@ -508,7 +508,7 @@ static int adf4377_soft_reset(struct adf4377_state *st) · 507 unmodified lines | ||
| 508 | 508 | return ret; |
| 509 | 509 | |
| 510 | 510 | return regmap_read_poll_timeout(st->regmap, 0x0, read_val, |
| 511 | − !(read_val & (ADF4377_0000_SOFT_RESET_R_MSK | | |
| 511 | + !(read_val & (ADF4377_0000_SOFT_RESET_MSK | | |
| 512 | 512 | ADF4377_0000_SOFT_RESET_R_MSK)), 200, 200 * 100); |
| 513 | 513 | } |
| 514 | 514 | |
| @@ -322,7 +322,9 @@ static int mpu3050_read_raw(struct iio_dev *indio_dev, · 321 unmodified lines | ||
| 322 | 322 | } |
| 323 | 323 | case IIO_CHAN_INFO_RAW: |
| 324 | 324 | /* Resume device */ |
| 325 | − pm_runtime_get_sync(mpu3050->dev); | |
| 325 | + ret = pm_runtime_resume_and_get(mpu3050->dev); | |
| 326 | + if (ret) | |
| 327 | + return ret; | |
| 326 | 328 | mutex_lock(&mpu3050->lock); |
| 327 | 329 | |
| 328 | 330 | ret = mpu3050_set_8khz_samplerate(mpu3050); |
| @@ -647,14 +649,20 @@ out_trigger_unlock: · 318 unmodified lines | ||
| 647 | 649 | static int mpu3050_buffer_preenable(struct iio_dev *indio_dev) |
| 648 | 650 | { |
| 649 | 651 | struct mpu3050 *mpu3050 = iio_priv(indio_dev); |
| 652 | + int ret; | |
| 650 | 653 | |
| 651 | − pm_runtime_get_sync(mpu3050->dev); | |
| 654 | + ret = pm_runtime_resume_and_get(mpu3050->dev); | |
| 655 | + if (ret) | |
| 656 | + return ret; | |
| 652 | 657 | |
| 653 | 658 | /* Unless we have OUR trigger active, run at full speed */ |
| 654 | − if (!mpu3050->hw_irq_trigger) | |
| 655 | − return mpu3050_set_8khz_samplerate(mpu3050); | |
| 659 | + if (!mpu3050->hw_irq_trigger) { | |
| 660 | + ret = mpu3050_set_8khz_samplerate(mpu3050); | |
| 661 | + if (ret) | |
| 662 | + pm_runtime_put_autosuspend(mpu3050->dev); | |
| 663 | + } | |
| 656 | 664 | |
| 657 | − return 0; | |
| 665 | + return ret; | |
| 658 | 666 | } |
| 659 | 667 | |
| 660 | 668 | static int mpu3050_buffer_postdisable(struct iio_dev *indio_dev) |
| @@ -19,8 +19,7 @@ static int mpu3050_i2c_bypass_select(struct i2c_mux_core *mux, u32 chan_id) · 18 unmodified lines | ||
| 19 | 19 | struct mpu3050 *mpu3050 = i2c_mux_priv(mux); |
| 20 | 20 | |
| 21 | 21 | /* Just power up the device, that is all that is needed */ |
| 22 | − pm_runtime_get_sync(mpu3050->dev); | |
| 23 | − return 0; | |
| 22 | + return pm_runtime_resume_and_get(mpu3050->dev); | |
| 24 | 23 | } |
| 25 | 24 | |
| 26 | 25 | static int mpu3050_i2c_bypass_deselect(struct i2c_mux_core *mux, u32 chan_id) |
| @@ -526,7 +526,7 @@ int adis_init(struct adis *adis, struct iio_dev *indio_dev, · 525 unmodified lines | ||
| 526 | 526 | |
| 527 | 527 | adis->spi = spi; |
| 528 | 528 | adis->data = data; |
| 529 | − if (!adis->ops->write && !adis->ops->read && !adis->ops->reset) | |
| 529 | + if (!adis->ops) | |
| 530 | 530 | adis->ops = &adis_default_ops; |
| 531 | 531 | else if (!adis->ops->write || !adis->ops->read || !adis->ops->reset) |
| 532 | 532 | return -EINVAL; |
| @@ -651,6 +651,8 @@ static int inv_icm42600_accel_write_odr(struct iio_dev *indio_dev, · 650 unmodified lines | ||
| 651 | 651 | return -EINVAL; |
| 652 | 652 | |
| 653 | 653 | conf.odr = inv_icm42600_accel_odr_conv[idx / 2]; |
| 654 | + if (conf.odr == st->conf.accel.odr) | |
| 655 | + return 0; | |
| 654 | 656 | |
| 655 | 657 | pm_runtime_get_sync(dev); |
| 656 | 658 | mutex_lock(&st->lock); |
| @@ -371,6 +371,8 @@ static int inv_icm42600_buffer_predisable(struct iio_dev *indio_dev) · 370 unmodified lines | ||
| 371 | 371 | static int inv_icm42600_buffer_postdisable(struct iio_dev *indio_dev) |
| 372 | 372 | { |
| 373 | 373 | struct inv_icm42600_state *st = iio_device_get_drvdata(indio_dev); |
| 374 | + struct inv_icm42600_sensor_state *sensor_st = iio_priv(indio_dev); | |
| 375 | + struct inv_sensors_timestamp *ts = &sensor_st->ts; | |
| 374 | 376 | struct device *dev = regmap_get_device(st->map); |
| 375 | 377 | unsigned int sensor; |
| 376 | 378 | unsigned int *watermark; |
| @@ -392,6 +394,8 @@ static int inv_icm42600_buffer_postdisable(struct iio_dev *indio_dev) · 15 unmodified lines | ||
| 392 | 394 | |
| 393 | 395 | mutex_lock(&st->lock); |
| 394 | 396 | |
| 397 | + inv_sensors_timestamp_apply_odr(ts, 0, 0, 0); | |
| 398 | + | |
| 395 | 399 | ret = inv_icm42600_buffer_set_fifo_en(st, st->fifo.en & ~sensor); |
| 396 | 400 | if (ret) |
| 397 | 401 | goto out_unlock; |
| @@ -358,6 +358,8 @@ static int inv_icm42600_gyro_write_odr(struct iio_dev *indio_dev, · 357 unmodified lines | ||
| 358 | 358 | return -EINVAL; |
| 359 | 359 | |
| 360 | 360 | conf.odr = inv_icm42600_gyro_odr_conv[idx / 2]; |
| 361 | + if (conf.odr == st->conf.gyro.odr) | |
| 362 | + return 0; | |
| 361 | 363 | |
| 362 | 364 | pm_runtime_get_sync(dev); |
| 363 | 365 | mutex_lock(&st->lock); |
| @@ -205,7 +205,7 @@ struct inv_icm45600_sensor_state { · 204 unmodified lines | ||
| 205 | 205 | #define INV_ICM45600_SPI_SLEW_RATE_38NS 0 |
| 206 | 206 | |
| 207 | 207 | #define INV_ICM45600_REG_INT1_CONFIG2 0x0018 |
| 208 | −#define INV_ICM45600_INT1_CONFIG2_PUSH_PULL BIT(2) | |
| 208 | +#define INV_ICM45600_INT1_CONFIG2_OPEN_DRAIN BIT(2) | |
| 209 | 209 | #define INV_ICM45600_INT1_CONFIG2_LATCHED BIT(1) |
| 210 | 210 | #define INV_ICM45600_INT1_CONFIG2_ACTIVE_HIGH BIT(0) |
| 211 | 211 | #define INV_ICM45600_INT1_CONFIG2_ACTIVE_LOW 0x00 |
| @@ -637,8 +637,8 @@ static int inv_icm45600_irq_init(struct inv_icm45600_state *st, int irq, · 636 unmodified lines | ||
| 637 | 637 | break; |
| 638 | 638 | } |
| 639 | 639 | |
| 640 | − if (!open_drain) | |
| 641 | − val |= INV_ICM45600_INT1_CONFIG2_PUSH_PULL; | |
| 640 | + if (open_drain) | |
| 641 | + val |= INV_ICM45600_INT1_CONFIG2_OPEN_DRAIN; | |
| 642 | 642 | |
| 643 | 643 | ret = regmap_write(st->map, INV_ICM45600_REG_INT1_CONFIG2, val); |
| 644 | 644 | if (ret) |
| @@ -744,6 +744,11 @@ int inv_icm45600_core_probe(struct regmap *regmap, const struct inv_icm45600_chi · 99 unmodified lines | ||
| 744 | 744 | */ |
| 745 | 745 | fsleep(5 * USEC_PER_MSEC); |
| 746 | 746 | |
| 747 | + /* set pm_runtime active early for disable vddio resource cleanup */ | |
| 748 | + ret = pm_runtime_set_active(dev); | |
| 749 | + if (ret) | |
| 750 | + return ret; | |
| 751 | + | |
| 747 | 752 | ret = inv_icm45600_enable_regulator_vddio(st); |
| 748 | 753 | if (ret) |
| 749 | 754 | return ret; |
| @@ -776,7 +781,7 @@ int inv_icm45600_core_probe(struct regmap *regmap, const struct inv_icm45600_chi · 26 unmodified lines | ||
| 776 | 781 | if (ret) |
| 777 | 782 | return ret; |
| 778 | 783 | |
| 779 | − ret = devm_pm_runtime_set_active_enabled(dev); | |
| 784 | + ret = devm_pm_runtime_enable(dev); | |
| 780 | 785 | if (ret) |
| 781 | 786 | return ret; |
| 782 | 787 | |
| @@ -1943,6 +1943,14 @@ int inv_mpu_core_probe(struct regmap *regmap, int irq, const char *name, · 1942 unmodified lines | ||
| 1943 | 1943 | irq_type); |
| 1944 | 1944 | return -EINVAL; |
| 1945 | 1945 | } |
| 1946 | + | |
| 1947 | + /* | |
| 1948 | + * Acking interrupts by status register does not work reliably | |
| 1949 | + * but seem to work when this bit is set. | |
| 1950 | + */ | |
| 1951 | + if (st->chip_type == INV_MPU9150) | |
| 1952 | + st->irq_mask |= INV_MPU6050_INT_RD_CLEAR; | |
| 1953 | + | |
| 1946 | 1954 | device_set_wakeup_capable(dev, true); |
| 1947 | 1955 | |
| 1948 | 1956 | st->vdd_supply = devm_regulator_get(dev, "vdd"); |
| @@ -390,6 +390,8 @@ struct inv_mpu6050_state { · 389 unmodified lines | ||
| 390 | 390 | /* enable level triggering */ |
| 391 | 391 | #define INV_MPU6050_LATCH_INT_EN 0x20 |
| 392 | 392 | #define INV_MPU6050_BIT_BYPASS_EN 0x2 |
| 393 | +/* allow acking interrupts by any register read */ | |
| 394 | +#define INV_MPU6050_INT_RD_CLEAR 0x10 | |
| 393 | 395 | |
| 394 | 396 | /* Allowed timestamp period jitter in percent */ |
| 395 | 397 | #define INV_MPU6050_TS_PERIOD_JITTER 4 |
| @@ -248,7 +248,6 @@ static irqreturn_t inv_mpu6050_interrupt_handle(int irq, void *p) · 247 unmodified lines | ||
| 248 | 248 | switch (st->chip_type) { |
| 249 | 249 | case INV_MPU6000: |
| 250 | 250 | case INV_MPU6050: |
| 251 | − case INV_MPU9150: | |
| 252 | 251 | /* |
| 253 | 252 | * WoM is not supported and interrupt status read seems to be broken for |
| 254 | 253 | * some chips. Since data ready is the only interrupt, bypass interrupt |
| @@ -257,6 +256,10 @@ static irqreturn_t inv_mpu6050_interrupt_handle(int irq, void *p) · 2 unmodified lines | ||
| 257 | 256 | wom_bits = 0; |
| 258 | 257 | int_status = INV_MPU6050_BIT_RAW_DATA_RDY_INT; |
| 259 | 258 | goto data_ready_interrupt; |
| 259 | + case INV_MPU9150: | |
| 260 | + /* IRQ needs to be acked */ | |
| 261 | + wom_bits = 0; | |
| 262 | + break; | |
| 260 | 263 | case INV_MPU6500: |
| 261 | 264 | case INV_MPU6515: |
| 262 | 265 | case INV_MPU6880: |
| @@ -228,8 +228,10 @@ static ssize_t iio_buffer_write(struct file *filp, const char __user *buf, · 227 unmodified lines | ||
| 228 | 228 | written = 0; |
| 229 | 229 | add_wait_queue(&rb->pollq, &wait); |
| 230 | 230 | do { |
| 231 | − if (!indio_dev->info) | |
| 232 | − return -ENODEV; | |
| 231 | + if (!indio_dev->info) { | |
| 232 | + ret = -ENODEV; | |
| 233 | + break; | |
| 234 | + } | |
| 233 | 235 | |
| 234 | 236 | if (!iio_buffer_space_available(rb)) { |
| 235 | 237 | if (signal_pending(current)) { |
| @@ -109,9 +109,9 @@ static int bh1780_read_raw(struct iio_dev *indio_dev, · 108 unmodified lines | ||
| 109 | 109 | case IIO_LIGHT: |
| 110 | 110 | pm_runtime_get_sync(&bh1780->client->dev); |
| 111 | 111 | value = bh1780_read_word(bh1780, BH1780_REG_DLOW); |
| 112 | + pm_runtime_put_autosuspend(&bh1780->client->dev); | |
| 112 | 113 | if (value < 0) |
| 113 | 114 | return value; |
| 114 | − pm_runtime_put_autosuspend(&bh1780->client->dev); | |
| 115 | 115 | *val = value; |
| 116 | 116 | |
| 117 | 117 | return IIO_VAL_INT; |
| @@ -143,8 +143,7 @@ config MMC5633 · 142 unmodified lines | ||
| 143 | 143 | tristate "MEMSIC MMC5633 3-axis magnetic sensor" |
| 144 | 144 | select REGMAP_I2C |
| 145 | 145 | select REGMAP_I3C if I3C |
| 146 | − depends on I2C | |
| 147 | − depends on I3C || !I3C | |
| 146 | + depends on I3C_OR_I2C | |
| 148 | 147 | help |
| 149 | 148 | Say yes here to build support for the MEMSIC MMC5633 3-axis |
| 150 | 149 | magnetic sensor. |
| @@ -171,7 +171,7 @@ static s16 tlv493d_get_channel_data(u8 *b, enum tlv493d_channels ch) · 170 unmodified lines | ||
| 171 | 171 | switch (ch) { |
| 172 | 172 | case TLV493D_AXIS_X: |
| 173 | 173 | val = FIELD_GET(TLV493D_BX_MAG_X_AXIS_MSB, b[TLV493D_RD_REG_BX]) << 4 | |
| 174 | − FIELD_GET(TLV493D_BX2_MAG_X_AXIS_LSB, b[TLV493D_RD_REG_BX2]) >> 4; | |
| 174 | + FIELD_GET(TLV493D_BX2_MAG_X_AXIS_LSB, b[TLV493D_RD_REG_BX2]); | |
| 175 | 175 | break; |
| 176 | 176 | case TLV493D_AXIS_Y: |
| 177 | 177 | val = FIELD_GET(TLV493D_BY_MAG_Y_AXIS_MSB, b[TLV493D_RD_REG_BY]) << 4 | |
| @@ -221,7 +221,7 @@ static int mcp4131_write_raw(struct iio_dev *indio_dev, · 220 unmodified lines | ||
| 221 | 221 | |
| 222 | 222 | mutex_lock(&data->lock); |
| 223 | 223 | |
| 224 | − data->buf[0] = address << MCP4131_WIPER_SHIFT; | |
| 224 | + data->buf[0] = address; | |
| 225 | 225 | data->buf[0] |= MCP4131_WRITE | (val >> 8); |
| 226 | 226 | data->buf[1] = val & 0xFF; /* 8 bits here */ |
| 227 | 227 | |
| @@ -719,6 +719,9 @@ static int hx9023s_set_samp_freq(struct hx9023s_data *data, int val, int val2) · 718 unmodified lines | ||
| 719 | 719 | struct device *dev = regmap_get_device(data->regmap); |
| 720 | 720 | unsigned int i, period_ms; |
| 721 | 721 | |
| 722 | + if (!val && !val2) | |
| 723 | + return -EINVAL; | |
| 724 | + | |
| 722 | 725 | period_ms = div_u64(NANO, (val * MEGA + val2)); |
| 723 | 726 | |
| 724 | 727 | for (i = 0; i < ARRAY_SIZE(hx9023s_samp_freq_table); i++) { |
| @@ -1034,9 +1037,8 @@ static int hx9023s_send_cfg(const struct firmware *fw, struct hx9023s_data *data · 309 unmodified lines | ||
| 1034 | 1037 | if (!bin) |
| 1035 | 1038 | return -ENOMEM; |
| 1036 | 1039 | |
| 1037 | − memcpy(bin->data, fw->data, fw->size); | |
| 1038 | − | |
| 1039 | 1040 | bin->fw_size = fw->size; |
| 1041 | + memcpy(bin->data, fw->data, bin->fw_size); | |
| 1040 | 1042 | bin->fw_ver = bin->data[FW_VER_OFFSET]; |
| 1041 | 1043 | bin->reg_count = get_unaligned_le16(bin->data + FW_REG_CNT_OFFSET); |
| 1042 | 1044 | |
| @@ -6,6 +6,7 @@ menuconfig INFINIBAND · 5 unmodified lines | ||
| 6 | 6 | depends on INET |
| 7 | 7 | depends on m || IPV6 != m |
| 8 | 8 | depends on !ALPHA |
| 9 | + select DMA_SHARED_BUFFER | |
| 9 | 10 | select IRQ_POLL |
| 10 | 11 | select DIMLIB |
| 11 | 12 | help |
| @@ -926,6 +926,13 @@ static int gid_table_setup_one(struct ib_device *ib_dev) · 925 unmodified lines | ||
| 926 | 926 | if (err) |
| 927 | 927 | return err; |
| 928 | 928 | |
| 929 | + /* | |
| 930 | + * Mark the device as ready for GID cache updates. This allows netdev | |
| 931 | + * event handlers to update the GID cache even before the device is | |
| 932 | + * fully registered. | |
| 933 | + */ | |
| 934 | + ib_device_enable_gid_updates(ib_dev); | |
| 935 | + | |
| 929 | 936 | rdma_roce_rescan_device(ib_dev); |
| 930 | 937 | |
| 931 | 938 | return err; |
| @@ -1637,6 +1644,12 @@ void ib_cache_release_one(struct ib_device *device) · 705 unmodified lines | ||
| 1637 | 1644 | |
| 1638 | 1645 | void ib_cache_cleanup_one(struct ib_device *device) |
| 1639 | 1646 | { |
| 1647 | + /* | |
| 1648 | + * Clear the GID updates mark first to prevent event handlers from | |
| 1649 | + * accessing the device while it's being torn down. | |
| 1650 | + */ | |
| 1651 | + ib_device_disable_gid_updates(device); | |
| 1652 | + | |
| 1640 | 1653 | /* The cleanup function waits for all in-progress workqueue |
| 1641 | 1654 | * elements and cleans up the GID cache. This function should be |
| 1642 | 1655 | * called after the device was removed from the devices list and |
| @@ -2729,6 +2729,9 @@ static int cma_listen_on_dev(struct rdma_id_private *id_priv, · 2728 unmodified lines | ||
| 2729 | 2729 | *to_destroy = NULL; |
| 2730 | 2730 | if (cma_family(id_priv) == AF_IB && !rdma_cap_ib_cm(cma_dev->device, 1)) |
| 2731 | 2731 | return 0; |
| 2732 | + if (id_priv->restricted_node_type != RDMA_NODE_UNSPECIFIED && | |
| 2733 | + id_priv->restricted_node_type != cma_dev->device->node_type) | |
| 2734 | + return 0; | |
| 2732 | 2735 | |
| 2733 | 2736 | dev_id_priv = |
| 2734 | 2737 | __rdma_create_id(net, cma_listen_handler, id_priv, |
| @@ -2736,6 +2739,7 @@ static int cma_listen_on_dev(struct rdma_id_private *id_priv, · 1 unmodified line | ||
| 2736 | 2739 | if (IS_ERR(dev_id_priv)) |
| 2737 | 2740 | return PTR_ERR(dev_id_priv); |
| 2738 | 2741 | |
| 2742 | + dev_id_priv->restricted_node_type = id_priv->restricted_node_type; | |
| 2739 | 2743 | dev_id_priv->state = RDMA_CM_ADDR_BOUND; |
| 2740 | 2744 | memcpy(cma_src_addr(dev_id_priv), cma_src_addr(id_priv), |
| 2741 | 2745 | rdma_addr_size(cma_src_addr(id_priv))); |
| @@ -4194,7 +4198,7 @@ int rdma_restrict_node_type(struct rdma_cm_id *id, u8 node_type) · 1452 unmodified lines | ||
| 4194 | 4198 | } |
| 4195 | 4199 | |
| 4196 | 4200 | mutex_lock(&lock); |
| 4197 | − if (id_priv->cma_dev) | |
| 4201 | + if (READ_ONCE(id_priv->state) != RDMA_CM_IDLE) | |
| 4198 | 4202 | ret = -EALREADY; |
| 4199 | 4203 | else |
| 4200 | 4204 | id_priv->restricted_node_type = node_type; |
| @@ -100,6 +100,9 @@ void ib_enum_all_roce_netdevs(roce_netdev_filter filter, · 99 unmodified lines | ||
| 100 | 100 | roce_netdev_callback cb, |
| 101 | 101 | void *cookie); |
| 102 | 102 | |
| 103 | +void ib_device_enable_gid_updates(struct ib_device *device); | |
| 104 | +void ib_device_disable_gid_updates(struct ib_device *device); | |
| 105 | + | |
| 103 | 106 | typedef int (*nldev_callback)(struct ib_device *device, |
| 104 | 107 | struct sk_buff *skb, |
| 105 | 108 | struct netlink_callback *cb, |
| @@ -93,6 +93,7 @@ static struct workqueue_struct *ib_unreg_wq; · 92 unmodified lines | ||
| 93 | 93 | static DEFINE_XARRAY_FLAGS(devices, XA_FLAGS_ALLOC); |
| 94 | 94 | static DECLARE_RWSEM(devices_rwsem); |
| 95 | 95 | #define DEVICE_REGISTERED XA_MARK_1 |
| 96 | +#define DEVICE_GID_UPDATES XA_MARK_2 | |
| 96 | 97 | |
| 97 | 98 | static u32 highest_client_id; |
| 98 | 99 | #define CLIENT_REGISTERED XA_MARK_1 |
| @@ -2412,11 +2413,42 @@ void ib_enum_all_roce_netdevs(roce_netdev_filter filter, · 2313 unmodified lines | ||
| 2412 | 2413 | unsigned long index; |
| 2413 | 2414 | |
| 2414 | 2415 | down_read(&devices_rwsem); |
| 2415 | − xa_for_each_marked (&devices, index, dev, DEVICE_REGISTERED) | |
| 2416 | + xa_for_each_marked(&devices, index, dev, DEVICE_GID_UPDATES) | |
| 2416 | 2417 | ib_enum_roce_netdev(dev, filter, filter_cookie, cb, cookie); |
| 2417 | 2418 | up_read(&devices_rwsem); |
| 2418 | 2419 | } |
| 2419 | 2420 | |
| 2421 | +/** | |
| 2422 | + * ib_device_enable_gid_updates - Mark device as ready for GID cache updates | |
| 2423 | + * @device: Device to mark | |
| 2424 | + * | |
| 2425 | + * Called after GID table is allocated and initialized. After this mark is set, | |
| 2426 | + * netdevice event handlers can update the device's GID cache. This allows | |
| 2427 | + * events that arrive during device registration to be processed, avoiding | |
| 2428 | + * stale GID entries when netdev properties change during the device | |
| 2429 | + * registration process. | |
| 2430 | + */ | |
| 2431 | +void ib_device_enable_gid_updates(struct ib_device *device) | |
| 2432 | +{ | |
| 2433 | + down_write(&devices_rwsem); | |
| 2434 | + xa_set_mark(&devices, device->index, DEVICE_GID_UPDATES); | |
| 2435 | + up_write(&devices_rwsem); | |
| 2436 | +} | |
| 2437 | + | |
| 2438 | +/** | |
| 2439 | + * ib_device_disable_gid_updates - Clear the GID updates mark | |
| 2440 | + * @device: Device to unmark | |
| 2441 | + * | |
| 2442 | + * Called before GID table cleanup to prevent event handlers from accessing | |
| 2443 | + * the device while it's being torn down. | |
| 2444 | + */ | |
| 2445 | +void ib_device_disable_gid_updates(struct ib_device *device) | |
| 2446 | +{ | |
| 2447 | + down_write(&devices_rwsem); | |
| 2448 | + xa_clear_mark(&devices, device->index, DEVICE_GID_UPDATES); | |
| 2449 | + up_write(&devices_rwsem); | |
| 2450 | +} | |
| 2451 | + | |
| 2420 | 2452 | /* |
| 2421 | 2453 | * ib_enum_all_devs - enumerate all ib_devices |
| 2422 | 2454 | * @cb: Callback to call for each found ib_device |
| @@ -218,13 +218,11 @@ ib_umem_dmabuf_get_pinned_with_dma_device(struct ib_device *device, · 217 unmodified lines | ||
| 218 | 218 | |
| 219 | 219 | err = ib_umem_dmabuf_map_pages(umem_dmabuf); |
| 220 | 220 | if (err) |
| 221 | − goto err_unpin; | |
| 221 | + goto err_release; | |
| 222 | 222 | dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv); |
| 223 | 223 | |
| 224 | 224 | return umem_dmabuf; |
| 225 | 225 | |
| 226 | −err_unpin: | |
| 227 | − dma_buf_unpin(umem_dmabuf->attach); | |
| 228 | 226 | err_release: |
| 229 | 227 | dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv); |
| 230 | 228 | ib_umem_release(&umem_dmabuf->umem); |
| @@ -10,6 +10,8 @@ · 9 unmodified lines | ||
| 10 | 10 | #include "rdma_core.h" |
| 11 | 11 | #include "uverbs.h" |
| 12 | 12 | |
| 13 | +MODULE_IMPORT_NS("DMA_BUF"); | |
| 14 | + | |
| 13 | 15 | static int uverbs_dmabuf_attach(struct dma_buf *dmabuf, |
| 14 | 16 | struct dma_buf_attachment *attachment) |
| 15 | 17 | { |
| @@ -54,9 +54,6 @@ static void bng_re_destroy_chip_ctx(struct bng_re_dev *rdev) · 53 unmodified lines | ||
| 54 | 54 | { |
| 55 | 55 | struct bng_re_chip_ctx *chip_ctx; |
| 56 | 56 | |
| 57 | − if (!rdev->chip_ctx) | |
| 58 | − return; | |
| 59 | − | |
| 60 | 57 | kfree(rdev->dev_attr); |
| 61 | 58 | rdev->dev_attr = NULL; |
| 62 | 59 | |
| @@ -124,12 +121,6 @@ static int bng_re_net_ring_free(struct bng_re_dev *rdev, · 61 unmodified lines | ||
| 124 | 121 | struct bnge_fw_msg fw_msg = {}; |
| 125 | 122 | int rc = -EINVAL; |
| 126 | 123 | |
| 127 | − if (!rdev) | |
| 128 | − return rc; | |
| 129 | − | |
| 130 | − if (!aux_dev) | |
| 131 | − return rc; | |
| 132 | − | |
| 133 | 124 | bng_re_init_hwrm_hdr((void *)&req, HWRM_RING_FREE); |
| 134 | 125 | req.ring_type = type; |
| 135 | 126 | req.ring_id = cpu_to_le16(fw_ring_id); |
| @@ -150,10 +141,7 @@ static int bng_re_net_ring_alloc(struct bng_re_dev *rdev, · 14 unmodified lines | ||
| 150 | 141 | struct hwrm_ring_alloc_input req = {}; |
| 151 | 142 | struct hwrm_ring_alloc_output resp; |
| 152 | 143 | struct bnge_fw_msg fw_msg = {}; |
| 153 | − int rc = -EINVAL; | |
| 154 | − | |
| 155 | − if (!aux_dev) | |
| 156 | − return rc; | |
| 144 | + int rc; | |
| 157 | 145 | |
| 158 | 146 | bng_re_init_hwrm_hdr((void *)&req, HWRM_RING_ALLOC); |
| 159 | 147 | req.enables = 0; |
| @@ -184,10 +172,7 @@ static int bng_re_stats_ctx_free(struct bng_re_dev *rdev) · 24 unmodified lines | ||
| 184 | 172 | struct hwrm_stat_ctx_free_input req = {}; |
| 185 | 173 | struct hwrm_stat_ctx_free_output resp = {}; |
| 186 | 174 | struct bnge_fw_msg fw_msg = {}; |
| 187 | − int rc = -EINVAL; | |
| 188 | − | |
| 189 | − if (!aux_dev) | |
| 190 | − return rc; | |
| 175 | + int rc; | |
| 191 | 176 | |
| 192 | 177 | bng_re_init_hwrm_hdr((void *)&req, HWRM_STAT_CTX_FREE); |
| 193 | 178 | req.stat_ctx_id = cpu_to_le32(rdev->stats_ctx.fw_id); |
| @@ -208,13 +193,10 @@ static int bng_re_stats_ctx_alloc(struct bng_re_dev *rdev) · 14 unmodified lines | ||
| 208 | 193 | struct hwrm_stat_ctx_alloc_output resp = {}; |
| 209 | 194 | struct hwrm_stat_ctx_alloc_input req = {}; |
| 210 | 195 | struct bnge_fw_msg fw_msg = {}; |
| 211 | − int rc = -EINVAL; | |
| 196 | + int rc; | |
| 212 | 197 | |
| 213 | 198 | stats->fw_id = BNGE_INVALID_STATS_CTX_ID; |
| 214 | 199 | |
| 215 | − if (!aux_dev) | |
| 216 | − return rc; | |
| 217 | − | |
| 218 | 200 | bng_re_init_hwrm_hdr((void *)&req, HWRM_STAT_CTX_ALLOC); |
| 219 | 201 | req.update_period_ms = cpu_to_le32(1000); |
| 220 | 202 | req.stats_dma_addr = cpu_to_le64(stats->dma_map); |
| @@ -303,7 +285,7 @@ static int bng_re_dev_init(struct bng_re_dev *rdev) · 82 unmodified lines | ||
| 303 | 285 | if (rc) { |
| 304 | 286 | ibdev_err(&rdev->ibdev, |
| 305 | 287 | "Failed to register with netedev: %#x\n", rc); |
| 306 | − return -EINVAL; | |
| 288 | + goto reg_netdev_fail; | |
| 307 | 289 | } |
| 308 | 290 | |
| 309 | 291 | set_bit(BNG_RE_FLAG_NETDEV_REGISTERED, &rdev->flags); |
| @@ -312,19 +294,16 @@ static int bng_re_dev_init(struct bng_re_dev *rdev) · 2 unmodified lines | ||
| 312 | 294 | ibdev_err(&rdev->ibdev, |
| 313 | 295 | "RoCE requires minimum 2 MSI-X vectors, but only %d reserved\n", |
| 314 | 296 | rdev->aux_dev->auxr_info->msix_requested); |
| 315 | − bnge_unregister_dev(rdev->aux_dev); | |
| 316 | − clear_bit(BNG_RE_FLAG_NETDEV_REGISTERED, &rdev->flags); | |
| 317 | − return -EINVAL; | |
| 297 | + rc = -EINVAL; | |
| 298 | + goto msix_ctx_fail; | |
| 318 | 299 | } |
| 319 | 300 | ibdev_dbg(&rdev->ibdev, "Got %d MSI-X vectors\n", |
| 320 | 301 | rdev->aux_dev->auxr_info->msix_requested); |
| 321 | 302 | |
| 322 | 303 | rc = bng_re_setup_chip_ctx(rdev); |
| 323 | 304 | if (rc) { |
| 324 | − bnge_unregister_dev(rdev->aux_dev); | |
| 325 | − clear_bit(BNG_RE_FLAG_NETDEV_REGISTERED, &rdev->flags); | |
| 326 | 305 | ibdev_err(&rdev->ibdev, "Failed to get chip context\n"); |
| 327 | − return -EINVAL; | |
| 306 | + goto msix_ctx_fail; | |
| 328 | 307 | } |
| 329 | 308 | |
| 330 | 309 | bng_re_query_hwrm_version(rdev); |
| @@ -333,16 +312,14 @@ static int bng_re_dev_init(struct bng_re_dev *rdev) · 2 unmodified lines | ||
| 333 | 312 | if (rc) { |
| 334 | 313 | ibdev_err(&rdev->ibdev, |
| 335 | 314 | "Failed to allocate RCFW Channel: %#x\n", rc); |
| 336 | − goto fail; | |
| 315 | + goto alloc_fw_chl_fail; | |
| 337 | 316 | } |
| 338 | 317 | |
| 339 | 318 | /* Allocate nq record memory */ |
| 340 | 319 | rdev->nqr = kzalloc_obj(*rdev->nqr); |
| 341 | 320 | if (!rdev->nqr) { |
| 342 | − bng_re_destroy_chip_ctx(rdev); | |
| 343 | − bnge_unregister_dev(rdev->aux_dev); | |
| 344 | − clear_bit(BNG_RE_FLAG_NETDEV_REGISTERED, &rdev->flags); | |
| 345 | − return -ENOMEM; | |
| 321 | + rc = -ENOMEM; | |
| 322 | + goto nq_alloc_fail; | |
| 346 | 323 | } |
| 347 | 324 | |
| 348 | 325 | rdev->nqr->num_msix = rdev->aux_dev->auxr_info->msix_requested; |
| @@ -411,9 +388,15 @@ disable_rcfw: · 62 unmodified lines | ||
| 411 | 388 | free_ring: |
| 412 | 389 | bng_re_net_ring_free(rdev, rdev->rcfw.creq.ring_id, type); |
| 413 | 390 | free_rcfw: |
| 391 | + kfree(rdev->nqr); | |
| 392 | +nq_alloc_fail: | |
| 414 | 393 | bng_re_free_rcfw_channel(&rdev->rcfw); |
| 415 | −fail: | |
| 416 | − bng_re_dev_uninit(rdev); | |
| 394 | +alloc_fw_chl_fail: | |
| 395 | + bng_re_destroy_chip_ctx(rdev); | |
| 396 | +msix_ctx_fail: | |
| 397 | + bnge_unregister_dev(rdev->aux_dev); | |
| 398 | + clear_bit(BNG_RE_FLAG_NETDEV_REGISTERED, &rdev->flags); | |
| 399 | +reg_netdev_fail: | |
| 417 | 400 | return rc; |
| 418 | 401 | } |
| 419 | 402 | |
| @@ -486,8 +469,7 @@ static void bng_re_remove(struct auxiliary_device *adev) · 66 unmodified lines | ||
| 486 | 469 | |
| 487 | 470 | rdev = dev_info->rdev; |
| 488 | 471 | |
| 489 | − if (rdev) | |
| 490 | − bng_re_remove_device(rdev, adev); | |
| 472 | + bng_re_remove_device(rdev, adev); | |
| 491 | 473 | kfree(dev_info); |
| 492 | 474 | } |
| 493 | 475 | |
| @@ -1661,7 +1661,7 @@ static struct efa_mr *efa_alloc_mr(struct ib_pd *ibpd, int access_flags, · 1660 unmodified lines | ||
| 1661 | 1661 | struct efa_mr *mr; |
| 1662 | 1662 | |
| 1663 | 1663 | if (udata && udata->inlen && |
| 1664 | − !ib_is_udata_cleared(udata, 0, sizeof(udata->inlen))) { | |
| 1664 | + !ib_is_udata_cleared(udata, 0, udata->inlen)) { | |
| 1665 | 1665 | ibdev_dbg(&dev->ibdev, |
| 1666 | 1666 | "Incompatible ABI params, udata not cleared\n"); |
| 1667 | 1667 | return ERR_PTR(-EINVAL); |
| @@ -1218,7 +1218,7 @@ int ionic_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, · 1217 unmodified lines | ||
| 1218 | 1218 | rdma_udata_to_drv_context(udata, struct ionic_ctx, ibctx); |
| 1219 | 1219 | struct ionic_vcq *vcq = to_ionic_vcq(ibcq); |
| 1220 | 1220 | struct ionic_tbl_buf buf = {}; |
| 1221 | − struct ionic_cq_resp resp; | |
| 1221 | + struct ionic_cq_resp resp = {}; | |
| 1222 | 1222 | struct ionic_cq_req req; |
| 1223 | 1223 | int udma_idx = 0, rc; |
| 1224 | 1224 | |
| @@ -81,6 +81,8 @@ static int ionic_query_port(struct ib_device *ibdev, u32 port, · 80 unmodified lines | ||
| 81 | 81 | return -EINVAL; |
| 82 | 82 | |
| 83 | 83 | ndev = ib_device_get_netdev(ibdev, port); |
| 84 | + if (!ndev) | |
| 85 | + return -ENODEV; | |
| 84 | 86 | |
| 85 | 87 | if (netif_running(ndev) && netif_carrier_ok(ndev)) { |
| 86 | 88 | attr->state = IB_PORT_ACTIVE; |
| @@ -5212,7 +5212,7 @@ static int irdma_create_user_ah(struct ib_ah *ibah, · 5211 unmodified lines | ||
| 5212 | 5212 | #define IRDMA_CREATE_AH_MIN_RESP_LEN offsetofend(struct irdma_create_ah_resp, rsvd) |
| 5213 | 5213 | struct irdma_ah *ah = container_of(ibah, struct irdma_ah, ibah); |
| 5214 | 5214 | struct irdma_device *iwdev = to_iwdev(ibah->pd->device); |
| 5215 | − struct irdma_create_ah_resp uresp; | |
| 5215 | + struct irdma_create_ah_resp uresp = {}; | |
| 5216 | 5216 | struct irdma_ah *parent_ah; |
| 5217 | 5217 | int err; |
| 5218 | 5218 | |
| @@ -428,6 +428,8 @@ static int mthca_create_srq(struct ib_srq *ibsrq, · 427 unmodified lines | ||
| 428 | 428 | |
| 429 | 429 | if (context && ib_copy_to_udata(udata, &srq->srqn, sizeof(__u32))) { |
| 430 | 430 | mthca_free_srq(to_mdev(ibsrq->device), srq); |
| 431 | + mthca_unmap_user_db(to_mdev(ibsrq->device), &context->uar, | |
| 432 | + context->db_tab, ucmd.db_index); | |
| 431 | 433 | return -EFAULT; |
| 432 | 434 | } |
| 433 | 435 | |
| @@ -436,6 +438,7 @@ static int mthca_create_srq(struct ib_srq *ibsrq, · 2 unmodified lines | ||
| 436 | 438 | |
| 437 | 439 | static int mthca_destroy_srq(struct ib_srq *srq, struct ib_udata *udata) |
| 438 | 440 | { |
| 441 | + mthca_free_srq(to_mdev(srq->device), to_msrq(srq)); | |
| 439 | 442 | if (udata) { |
| 440 | 443 | struct mthca_ucontext *context = |
| 441 | 444 | rdma_udata_to_drv_context( |
| @@ -446,8 +449,6 @@ static int mthca_destroy_srq(struct ib_srq *srq, struct ib_udata *udata) · 4 unmodified lines | ||
| 446 | 449 | mthca_unmap_user_db(to_mdev(srq->device), &context->uar, |
| 447 | 450 | context->db_tab, to_msrq(srq)->db_index); |
| 448 | 451 | } |
| 449 | − | |
| 450 | − mthca_free_srq(to_mdev(srq->device), to_msrq(srq)); | |
| 451 | 452 | return 0; |
| 452 | 453 | } |
| 453 | 454 | |
| @@ -3474,6 +3474,7 @@ static struct its_device *its_create_device(struct its_node *its, u32 dev_id, · 3473 unmodified lines | ||
| 3474 | 3474 | int lpi_base; |
| 3475 | 3475 | int nr_lpis; |
| 3476 | 3476 | int nr_ites; |
| 3477 | + int id_bits; | |
| 3477 | 3478 | int sz; |
| 3478 | 3479 | |
| 3479 | 3480 | if (!its_alloc_device_table(its, dev_id)) |
| @@ -3485,7 +3486,10 @@ static struct its_device *its_create_device(struct its_node *its, u32 dev_id, · 5 unmodified lines | ||
| 3485 | 3486 | /* |
| 3486 | 3487 | * Even if the device wants a single LPI, the ITT must be |
| 3487 | 3488 | * sized as a power of two (and you need at least one bit...). |
| 3489 | + * Also honor the ITS's own EID limit. | |
| 3488 | 3490 | */ |
| 3491 | + id_bits = FIELD_GET(GITS_TYPER_IDBITS, its->typer) + 1; | |
| 3492 | + nvecs = min_t(unsigned int, nvecs, BIT(id_bits)); | |
| 3489 | 3493 | nr_ites = max(2, nvecs); |
| 3490 | 3494 | sz = nr_ites * (FIELD_GET(GITS_TYPER_ITT_ENTRY_SIZE, its->typer) + 1); |
| 3491 | 3495 | sz = max(sz, ITS_ITT_ALIGN); |
| @@ -699,7 +699,7 @@ static int __init gicv5_irs_init(struct gicv5_irs_chip_data *irs_data) · 698 unmodified lines | ||
| 699 | 699 | */ |
| 700 | 700 | if (list_empty(&irs_nodes)) { |
| 701 | 701 | idr = irs_readl_relaxed(irs_data, GICV5_IRS_IDR0); |
| 702 | − gicv5_global_data.virt_capable = !FIELD_GET(GICV5_IRS_IDR0_VIRT, idr); | |
| 702 | + gicv5_global_data.virt_capable = !!FIELD_GET(GICV5_IRS_IDR0_VIRT, idr); | |
| 703 | 703 | |
| 704 | 704 | idr = irs_readl_relaxed(irs_data, GICV5_IRS_IDR1); |
| 705 | 705 | irs_setup_pri_bits(idr); |
| @@ -125,32 +125,45 @@ static const struct irq_domain_ops extirq_domain_ops = { · 124 unmodified lines | ||
| 125 | 125 | static int |
| 126 | 126 | ls_extirq_parse_map(struct ls_extirq_data *priv, struct device_node *node) |
| 127 | 127 | { |
| 128 | − struct of_imap_parser imap_parser; | |
| 129 | − struct of_imap_item imap_item; | |
| 128 | + const __be32 *map; | |
| 129 | + u32 mapsize; | |
| 130 | 130 | int ret; |
| 131 | 131 | |
| 132 | − ret = of_imap_parser_init(&imap_parser, node, &imap_item); | |
| 133 | − if (ret) | |
| 134 | − return ret; | |
| 132 | + map = of_get_property(node, "interrupt-map", &mapsize); | |
| 133 | + if (!map) | |
| 134 | + return -ENOENT; | |
| 135 | + if (mapsize % sizeof(*map)) | |
| 136 | + return -EINVAL; | |
| 137 | + mapsize /= sizeof(*map); | |
| 135 | 138 | |
| 136 | − for_each_of_imap_item(&imap_parser, &imap_item) { | |
| 139 | + while (mapsize) { | |
| 137 | 140 | struct device_node *ipar; |
| 138 | − u32 hwirq; | |
| 139 | − int i; | |
| 141 | + u32 hwirq, intsize, j; | |
| 140 | 142 | |
| 141 | − hwirq = imap_item.child_imap[0]; | |
| 142 | − if (hwirq >= MAXIRQ) { | |
| 143 | − of_node_put(imap_item.parent_args.np); | |
| 143 | + if (mapsize < 3) | |
| 144 | + return -EINVAL; | |
| 145 | + hwirq = be32_to_cpup(map); | |
| 146 | + if (hwirq >= MAXIRQ) | |
| 144 | 147 | return -EINVAL; |
| 145 | − } | |
| 146 | 148 | priv->nirq = max(priv->nirq, hwirq + 1); |
| 147 | 149 | |
| 148 | − ipar = of_node_get(imap_item.parent_args.np); | |
| 149 | − priv->map[hwirq].fwnode = of_fwnode_handle(ipar); | |
| 150 | + ipar = of_find_node_by_phandle(be32_to_cpup(map + 2)); | |
| 151 | + map += 3; | |
| 152 | + mapsize -= 3; | |
| 153 | + if (!ipar) | |
| 154 | + return -EINVAL; | |
| 155 | + priv->map[hwirq].fwnode = &ipar->fwnode; | |
| 156 | + ret = of_property_read_u32(ipar, "#interrupt-cells", &intsize); | |
| 157 | + if (ret) | |
| 158 | + return ret; | |
| 150 | 159 | |
| 151 | − priv->map[hwirq].param_count = imap_item.parent_args.args_count; | |
| 152 | − for (i = 0; i < priv->map[hwirq].param_count; i++) | |
| 153 | − priv->map[hwirq].param[i] = imap_item.parent_args.args[i]; | |
| 160 | + if (intsize > mapsize) | |
| 161 | + return -EINVAL; | |
| 162 | + | |
| 163 | + priv->map[hwirq].param_count = intsize; | |
| 164 | + for (j = 0; j < intsize; ++j) | |
| 165 | + priv->map[hwirq].param[j] = be32_to_cpup(map++); | |
| 166 | + mapsize -= intsize; | |
| 154 | 167 | } |
| 155 | 168 | return 0; |
| 156 | 169 | } |
| @@ -177,8 +190,10 @@ static int ls_extirq_probe(struct platform_device *pdev) · 20 unmodified lines | ||
| 177 | 190 | return dev_err_probe(dev, -ENOMEM, "Failed to allocate memory\n"); |
| 178 | 191 | |
| 179 | 192 | priv->intpcr = devm_of_iomap(dev, node, 0, NULL); |
| 180 | − if (!priv->intpcr) | |
| 181 | − return dev_err_probe(dev, -ENOMEM, "Cannot ioremap OF node %pOF\n", node); | |
| 193 | + if (IS_ERR(priv->intpcr)) { | |
| 194 | + return dev_err_probe(dev, PTR_ERR(priv->intpcr), | |
| 195 | + "Cannot ioremap OF node %pOF\n", node); | |
| 196 | + } | |
| 182 | 197 | |
| 183 | 198 | ret = ls_extirq_parse_map(priv, node); |
| 184 | 199 | if (ret) |
| @@ -136,7 +136,7 @@ static void icu_unmask_irq(struct irq_data *d) · 135 unmodified lines | ||
| 136 | 136 | } |
| 137 | 137 | } |
| 138 | 138 | |
| 139 | −struct irq_chip icu_irq_chip = { | |
| 139 | +static const struct irq_chip icu_irq_chip = { | |
| 140 | 140 | .name = "icu_irq", |
| 141 | 141 | .irq_mask = icu_mask_irq, |
| 142 | 142 | .irq_mask_ack = icu_mask_ack_irq, |
| @@ -116,6 +116,16 @@ static struct syscore aplic_syscore = { · 115 unmodified lines | ||
| 116 | 116 | .ops = &aplic_syscore_ops, |
| 117 | 117 | }; |
| 118 | 118 | |
| 119 | +static bool aplic_syscore_registered __ro_after_init; | |
| 120 | + | |
| 121 | +static void aplic_syscore_init(void) | |
| 122 | +{ | |
| 123 | + if (!aplic_syscore_registered) { | |
| 124 | + register_syscore(&aplic_syscore); | |
| 125 | + aplic_syscore_registered = true; | |
| 126 | + } | |
| 127 | +} | |
| 128 | + | |
| 119 | 129 | static int aplic_pm_notifier(struct notifier_block *nb, unsigned long action, void *data) |
| 120 | 130 | { |
| 121 | 131 | struct aplic_priv *priv = container_of(nb, struct aplic_priv, genpd_nb); |
| @@ -372,18 +382,21 @@ static int aplic_probe(struct platform_device *pdev) · 250 unmodified lines | ||
| 372 | 382 | rc = aplic_msi_setup(dev, regs); |
| 373 | 383 | else |
| 374 | 384 | rc = aplic_direct_setup(dev, regs); |
| 375 | − if (rc) | |
| 385 | + | |
| 386 | + if (rc) { | |
| 376 | 387 | dev_err_probe(dev, rc, "failed to setup APLIC in %s mode\n", |
| 377 | 388 | msi_mode ? "MSI" : "direct"); |
| 378 | − else | |
| 379 | − register_syscore(&aplic_syscore); | |
| 389 | + return rc; | |
| 390 | + } | |
| 391 | + | |
| 392 | + aplic_syscore_init(); | |
| 380 | 393 | |
| 381 | 394 | #ifdef CONFIG_ACPI |
| 382 | 395 | if (!acpi_disabled) |
| 383 | 396 | acpi_dev_clear_dependencies(ACPI_COMPANION(dev)); |
| 384 | 397 | #endif |
| 385 | 398 | |
| 386 | − return rc; | |
| 399 | + return 0; | |
| 387 | 400 | } |
| 388 | 401 | |
| 389 | 402 | static const struct of_device_id aplic_match[] = { |
| @@ -172,8 +172,13 @@ static void plic_irq_disable(struct irq_data *d) · 171 unmodified lines | ||
| 172 | 172 | static void plic_irq_eoi(struct irq_data *d) |
| 173 | 173 | { |
| 174 | 174 | struct plic_handler *handler = this_cpu_ptr(&plic_handlers); |
| 175 | + u32 __iomem *reg; | |
| 176 | + bool enabled; | |
| 177 | + | |
| 178 | + reg = handler->enable_base + (d->hwirq / 32) * sizeof(u32); | |
| 179 | + enabled = readl(reg) & BIT(d->hwirq % 32); | |
| 175 | 180 | |
| 176 | − if (unlikely(irqd_irq_disabled(d))) { | |
| 181 | + if (unlikely(!enabled)) { | |
| 177 | 182 | plic_toggle(handler, d->hwirq, 1); |
| 178 | 183 | writel(d->hwirq, handler->hart_base + CONTEXT_CLAIM); |
| 179 | 184 | plic_toggle(handler, d->hwirq, 0); |
| @@ -168,7 +168,9 @@ static int dvb_dvr_open(struct inode *inode, struct file *file) · 167 unmodified lines | ||
| 168 | 168 | mutex_unlock(&dmxdev->mutex); |
| 169 | 169 | return -ENOMEM; |
| 170 | 170 | } |
| 171 | − dvb_ringbuffer_init(&dmxdev->dvr_buffer, mem, DVR_BUFFER_SIZE); | |
| 171 | + dmxdev->dvr_buffer.data = mem; | |
| 172 | + dmxdev->dvr_buffer.size = DVR_BUFFER_SIZE; | |
| 173 | + dvb_ringbuffer_reset(&dmxdev->dvr_buffer); | |
| 172 | 174 | if (dmxdev->may_do_mmap) |
| 173 | 175 | dvb_vb2_init(&dmxdev->dvr_vb2_ctx, "dvr", |
| 174 | 176 | &dmxdev->mutex, |
| @@ -228,6 +228,9 @@ static int handle_one_ule_extension( struct dvb_net_priv *p ) · 227 unmodified lines | ||
| 228 | 228 | unsigned char hlen = (p->ule_sndu_type & 0x0700) >> 8; |
| 229 | 229 | unsigned char htype = p->ule_sndu_type & 0x00FF; |
| 230 | 230 | |
| 231 | + if (htype >= ARRAY_SIZE(ule_mandatory_ext_handlers)) | |
| 232 | + return -1; | |
| 233 | + | |
| 231 | 234 | /* Discriminate mandatory and optional extension headers. */ |
| 232 | 235 | if (hlen == 0) { |
| 233 | 236 | /* Mandatory extension header */ |
| @@ -1,10 +1,9 @@ | ||
| 1 | 1 | # SPDX-License-Identifier: GPL-2.0-only |
| 2 | 2 | config AMD_SBRMI_I2C |
| 3 | 3 | tristate "AMD side band RMI support" |
| 4 | − depends on I2C | |
| 4 | + depends on I3C_OR_I2C | |
| 5 | 5 | depends on ARM || ARM64 || COMPILE_TEST |
| 6 | 6 | select REGMAP_I2C |
| 7 | − depends on I3C || !I3C | |
| 8 | 7 | select REGMAP_I3C if I3C |
| 9 | 8 | help |
| 10 | 9 | Side band RMI over I2C/I3C support for AMD out of band management. |
| @@ -36,6 +36,8 @@ struct dw_mci_rockchip_priv_data { · 35 unmodified lines | ||
| 36 | 36 | int default_sample_phase; |
| 37 | 37 | int num_phases; |
| 38 | 38 | bool internal_phase; |
| 39 | + int sample_phase; | |
| 40 | + int drv_phase; | |
| 39 | 41 | }; |
| 40 | 42 | |
| 41 | 43 | /* |
| @@ -573,9 +575,43 @@ static void dw_mci_rockchip_remove(struct platform_device *pdev) · 531 unmodified lines | ||
| 573 | 575 | dw_mci_pltfm_remove(pdev); |
| 574 | 576 | } |
| 575 | 577 | |
| 578 | +static int dw_mci_rockchip_runtime_suspend(struct device *dev) | |
| 579 | +{ | |
| 580 | + struct platform_device *pdev = to_platform_device(dev); | |
| 581 | + struct dw_mci *host = platform_get_drvdata(pdev); | |
| 582 | + struct dw_mci_rockchip_priv_data *priv = host->priv; | |
| 583 | + | |
| 584 | + if (priv->internal_phase) { | |
| 585 | + priv->sample_phase = rockchip_mmc_get_phase(host, true); | |
| 586 | + priv->drv_phase = rockchip_mmc_get_phase(host, false); | |
| 587 | + } | |
| 588 | + | |
| 589 | + return dw_mci_runtime_suspend(dev); | |
| 590 | +} | |
| 591 | + | |
| 592 | +static int dw_mci_rockchip_runtime_resume(struct device *dev) | |
| 593 | +{ | |
| 594 | + struct platform_device *pdev = to_platform_device(dev); | |
| 595 | + struct dw_mci *host = platform_get_drvdata(pdev); | |
| 596 | + struct dw_mci_rockchip_priv_data *priv = host->priv; | |
| 597 | + int ret; | |
| 598 | + | |
| 599 | + ret = dw_mci_runtime_resume(dev); | |
| 600 | + if (ret) | |
| 601 | + return ret; | |
| 602 | + | |
| 603 | + if (priv->internal_phase) { | |
| 604 | + rockchip_mmc_set_phase(host, true, priv->sample_phase); | |
| 605 | + rockchip_mmc_set_phase(host, false, priv->drv_phase); | |
| 606 | + mci_writel(host, MISC_CON, MEM_CLK_AUTOGATE_ENABLE); | |
| 607 | + } | |
| 608 | + | |
| 609 | + return ret; | |
| 610 | +} | |
| 611 | + | |
| 576 | 612 | static const struct dev_pm_ops dw_mci_rockchip_dev_pm_ops = { |
| 577 | 613 | SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, pm_runtime_force_resume) |
| 578 | − RUNTIME_PM_OPS(dw_mci_runtime_suspend, dw_mci_runtime_resume, NULL) | |
| 614 | + RUNTIME_PM_OPS(dw_mci_rockchip_runtime_suspend, dw_mci_rockchip_runtime_resume, NULL) | |
| 579 | 615 | }; |
| 580 | 616 | |
| 581 | 617 | static struct platform_driver dw_mci_rockchip_pltfm_driver = { |
| @@ -109,6 +109,7 @@ static int of_get_dml_pipe_index(struct device_node *np, const char *name) · 108 unmodified lines | ||
| 109 | 109 | &dma_spec)) |
| 110 | 110 | return -ENODEV; |
| 111 | 111 | |
| 112 | + of_node_put(dma_spec.np); | |
| 112 | 113 | if (dma_spec.args_count) |
| 113 | 114 | return dma_spec.args[0]; |
| 114 | 115 | |
| @@ -116,7 +116,7 @@ static void sdhci_brcmstb_restore_regs(struct mmc_host *mmc, enum cfg_core_ver v · 115 unmodified lines | ||
| 116 | 116 | writel(sr->boot_main_ctl, priv->boot_regs + SDIO_BOOT_MAIN_CTL); |
| 117 | 117 | |
| 118 | 118 | if (ver == SDIO_CFG_CORE_V1) { |
| 119 | − writel(sr->sd_pin_sel, cr + SDIO_CFG_SD_PIN_SEL); | |
| 119 | + writel(sr->sd_pin_sel, cr + SDIO_CFG_V1_SD_PIN_SEL); | |
| 120 | 120 | return; |
| 121 | 121 | } |
| 122 | 122 | |
| @@ -324,7 +324,7 @@ static bool bond_sk_check(struct bonding *bond) · 323 unmodified lines | ||
| 324 | 324 | } |
| 325 | 325 | } |
| 326 | 326 | |
| 327 | −bool bond_xdp_check(struct bonding *bond, int mode) | |
| 327 | +bool __bond_xdp_check(int mode, int xmit_policy) | |
| 328 | 328 | { |
| 329 | 329 | switch (mode) { |
| 330 | 330 | case BOND_MODE_ROUNDROBIN: |
| @@ -335,7 +335,7 @@ bool bond_xdp_check(struct bonding *bond, int mode) · 4 unmodified lines | ||
| 335 | 335 | /* vlan+srcmac is not supported with XDP as in most cases the 802.1q |
| 336 | 336 | * payload is not in the packet due to hardware offload. |
| 337 | 337 | */ |
| 338 | − if (bond->params.xmit_policy != BOND_XMIT_POLICY_VLAN_SRCMAC) | |
| 338 | + if (xmit_policy != BOND_XMIT_POLICY_VLAN_SRCMAC) | |
| 339 | 339 | return true; |
| 340 | 340 | fallthrough; |
| 341 | 341 | default: |
| @@ -343,6 +343,11 @@ bool bond_xdp_check(struct bonding *bond, int mode) · 1 unmodified line | ||
| 343 | 343 | } |
| 344 | 344 | } |
| 345 | 345 | |
| 346 | +bool bond_xdp_check(struct bonding *bond, int mode) | |
| 347 | +{ | |
| 348 | + return __bond_xdp_check(mode, bond->params.xmit_policy); | |
| 349 | +} | |
| 350 | + | |
| 346 | 351 | /*---------------------------------- VLAN -----------------------------------*/ |
| 347 | 352 | |
| 348 | 353 | /* In the following 2 functions, bond_vlan_rx_add_vid and bond_vlan_rx_kill_vid, |
| @@ -1504,6 +1509,50 @@ static netdev_features_t bond_fix_features(struct net_device *dev, · 1155 unmodified lines | ||
| 1504 | 1509 | return features; |
| 1505 | 1510 | } |
| 1506 | 1511 | |
| 1512 | +static int bond_header_create(struct sk_buff *skb, struct net_device *bond_dev, | |
| 1513 | + unsigned short type, const void *daddr, | |
| 1514 | + const void *saddr, unsigned int len) | |
| 1515 | +{ | |
| 1516 | + struct bonding *bond = netdev_priv(bond_dev); | |
| 1517 | + const struct header_ops *slave_ops; | |
| 1518 | + struct slave *slave; | |
| 1519 | + int ret = 0; | |
| 1520 | + | |
| 1521 | + rcu_read_lock(); | |
| 1522 | + slave = rcu_dereference(bond->curr_active_slave); | |
| 1523 | + if (slave) { | |
| 1524 | + slave_ops = READ_ONCE(slave->dev->header_ops); | |
| 1525 | + if (slave_ops && slave_ops->create) | |
| 1526 | + ret = slave_ops->create(skb, slave->dev, | |
| 1527 | + type, daddr, saddr, len); | |
| 1528 | + } | |
| 1529 | + rcu_read_unlock(); | |
| 1530 | + return ret; | |
| 1531 | +} | |
| 1532 | + | |
| 1533 | +static int bond_header_parse(const struct sk_buff *skb, unsigned char *haddr) | |
| 1534 | +{ | |
| 1535 | + struct bonding *bond = netdev_priv(skb->dev); | |
| 1536 | + const struct header_ops *slave_ops; | |
| 1537 | + struct slave *slave; | |
| 1538 | + int ret = 0; | |
| 1539 | + | |
| 1540 | + rcu_read_lock(); | |
| 1541 | + slave = rcu_dereference(bond->curr_active_slave); | |
| 1542 | + if (slave) { | |
| 1543 | + slave_ops = READ_ONCE(slave->dev->header_ops); | |
| 1544 | + if (slave_ops && slave_ops->parse) | |
| 1545 | + ret = slave_ops->parse(skb, haddr); | |
| 1546 | + } | |
| 1547 | + rcu_read_unlock(); | |
| 1548 | + return ret; | |
| 1549 | +} | |
| 1550 | + | |
| 1551 | +static const struct header_ops bond_header_ops = { | |
| 1552 | + .create = bond_header_create, | |
| 1553 | + .parse = bond_header_parse, | |
| 1554 | +}; | |
| 1555 | + | |
| 1507 | 1556 | static void bond_setup_by_slave(struct net_device *bond_dev, |
| 1508 | 1557 | struct net_device *slave_dev) |
| 1509 | 1558 | { |
| @@ -1511,7 +1560,8 @@ static void bond_setup_by_slave(struct net_device *bond_dev, · 1 unmodified line | ||
| 1511 | 1560 | |
| 1512 | 1561 | dev_close(bond_dev); |
| 1513 | 1562 | |
| 1514 | − bond_dev->header_ops = slave_dev->header_ops; | |
| 1563 | + bond_dev->header_ops = slave_dev->header_ops ? | |
| 1564 | + &bond_header_ops : NULL; | |
| 1515 | 1565 | |
| 1516 | 1566 | bond_dev->type = slave_dev->type; |
| 1517 | 1567 | bond_dev->hard_header_len = slave_dev->hard_header_len; |
| @@ -2796,8 +2846,14 @@ static void bond_miimon_commit(struct bonding *bond) · 1278 unmodified lines | ||
| 2796 | 2846 | |
| 2797 | 2847 | continue; |
| 2798 | 2848 | |
| 2849 | + case BOND_LINK_FAIL: | |
| 2850 | + case BOND_LINK_BACK: | |
| 2851 | + slave_dbg(bond->dev, slave->dev, "link_new_state %d on slave\n", | |
| 2852 | + slave->link_new_state); | |
| 2853 | + continue; | |
| 2854 | + | |
| 2799 | 2855 | default: |
| 2800 | − slave_err(bond->dev, slave->dev, "invalid new link %d on slave\n", | |
| 2856 | + slave_err(bond->dev, slave->dev, "invalid link_new_state %d on slave\n", | |
| 2801 | 2857 | slave->link_new_state); |
| 2802 | 2858 | bond_propose_link_state(slave, BOND_LINK_NOCHANGE); |
| 2803 | 2859 | |
| @@ -3372,7 +3428,7 @@ int bond_rcv_validate(const struct sk_buff *skb, struct bonding *bond, · 568 unmodified lines | ||
| 3372 | 3428 | } else if (is_arp) { |
| 3373 | 3429 | return bond_arp_rcv(skb, bond, slave); |
| 3374 | 3430 | #if IS_ENABLED(CONFIG_IPV6) |
| 3375 | − } else if (is_ipv6) { | |
| 3431 | + } else if (is_ipv6 && likely(ipv6_mod_enabled())) { | |
| 3376 | 3432 | return bond_na_rcv(skb, bond, slave); |
| 3377 | 3433 | #endif |
| 3378 | 3434 | } else { |
| @@ -5064,13 +5120,18 @@ static void bond_set_slave_arr(struct bonding *bond, · 1685 unmodified lines | ||
| 5064 | 5120 | { |
| 5065 | 5121 | struct bond_up_slave *usable, *all; |
| 5066 | 5122 | |
| 5067 | − usable = rtnl_dereference(bond->usable_slaves); | |
| 5068 | − rcu_assign_pointer(bond->usable_slaves, usable_slaves); | |
| 5069 | − kfree_rcu(usable, rcu); | |
| 5070 | − | |
| 5071 | 5123 | all = rtnl_dereference(bond->all_slaves); |
| 5072 | 5124 | rcu_assign_pointer(bond->all_slaves, all_slaves); |
| 5073 | 5125 | kfree_rcu(all, rcu); |
| 5126 | + | |
| 5127 | + if (BOND_MODE(bond) == BOND_MODE_BROADCAST) { | |
| 5128 | + kfree_rcu(usable_slaves, rcu); | |
| 5129 | + return; | |
| 5130 | + } | |
| 5131 | + | |
| 5132 | + usable = rtnl_dereference(bond->usable_slaves); | |
| 5133 | + rcu_assign_pointer(bond->usable_slaves, usable_slaves); | |
| 5134 | + kfree_rcu(usable, rcu); | |
| 5074 | 5135 | } |
| 5075 | 5136 | |
| 5076 | 5137 | static void bond_reset_slave_arr(struct bonding *bond) |
| @@ -1575,6 +1575,8 @@ static int bond_option_fail_over_mac_set(struct bonding *bond, · 1574 unmodified lines | ||
| 1575 | 1575 | static int bond_option_xmit_hash_policy_set(struct bonding *bond, |
| 1576 | 1576 | const struct bond_opt_value *newval) |
| 1577 | 1577 | { |
| 1578 | + if (bond->xdp_prog && !__bond_xdp_check(BOND_MODE(bond), newval->value)) | |
| 1579 | + return -EOPNOTSUPP; | |
| 1578 | 1580 | netdev_dbg(bond->dev, "Setting xmit hash policy to %s (%llu)\n", |
| 1579 | 1581 | newval->string, newval->value); |
| 1580 | 1582 | bond->params.xmit_policy = newval->value; |
| @@ -297,6 +297,7 @@ static void ser_release(struct work_struct *work) · 296 unmodified lines | ||
| 297 | 297 | dev_close(ser->dev); |
| 298 | 298 | unregister_netdevice(ser->dev); |
| 299 | 299 | debugfs_deinit(ser); |
| 300 | + tty_kref_put(tty->link); | |
| 300 | 301 | tty_kref_put(tty); |
| 301 | 302 | } |
| 302 | 303 | rtnl_unlock(); |
| @@ -331,6 +332,7 @@ static int ldisc_open(struct tty_struct *tty) · 28 unmodified lines | ||
| 331 | 332 | |
| 332 | 333 | ser = netdev_priv(dev); |
| 333 | 334 | ser->tty = tty_kref_get(tty); |
| 335 | + tty_kref_get(tty->link); | |
| 334 | 336 | ser->dev = dev; |
| 335 | 337 | debugfs_init(ser, tty); |
| 336 | 338 | tty->receive_room = 4096; |
| @@ -339,6 +341,7 @@ static int ldisc_open(struct tty_struct *tty) · 2 unmodified lines | ||
| 339 | 341 | rtnl_lock(); |
| 340 | 342 | result = register_netdevice(dev); |
| 341 | 343 | if (result) { |
| 344 | + tty_kref_put(tty->link); | |
| 342 | 345 | tty_kref_put(tty); |
| 343 | 346 | rtnl_unlock(); |
| 344 | 347 | free_netdev(dev); |
| @@ -8,7 +8,7 @@ · 7 unmodified lines | ||
| 8 | 8 | #include <linux/units.h> |
| 9 | 9 | #include <linux/can/dev.h> |
| 10 | 10 | |
| 11 | −#define CAN_CALC_MAX_ERROR 50 /* in one-tenth of a percent */ | |
| 11 | +#define CAN_CALC_MAX_ERROR 500 /* max error 5% */ | |
| 12 | 12 | |
| 13 | 13 | /* CiA recommended sample points for Non Return to Zero encoding. */ |
| 14 | 14 | static int can_calc_sample_point_nrz(const struct can_bittiming *bt) |
| @@ -241,6 +241,7 @@ static int __init dummy_can_init(void) · 240 unmodified lines | ||
| 241 | 241 | |
| 242 | 242 | dev->netdev_ops = &dummy_can_netdev_ops; |
| 243 | 243 | dev->ethtool_ops = &dummy_can_ethtool_ops; |
| 244 | + dev->flags |= IFF_ECHO; /* enable echo handling */ | |
| 244 | 245 | priv = netdev_priv(dev); |
| 245 | 246 | priv->can.bittiming_const = &dummy_can_bittiming_const; |
| 246 | 247 | priv->can.bitrate_max = 20 * MEGA /* BPS */; |
| @@ -755,7 +755,9 @@ static int hi3110_open(struct net_device *net) · 754 unmodified lines | ||
| 755 | 755 | return ret; |
| 756 | 756 | |
| 757 | 757 | mutex_lock(&priv->hi3110_lock); |
| 758 | − hi3110_power_enable(priv->transceiver, 1); | |
| 758 | + ret = hi3110_power_enable(priv->transceiver, 1); | |
| 759 | + if (ret) | |
| 760 | + goto out_close_candev; | |
| 759 | 761 | |
| 760 | 762 | priv->force_quit = 0; |
| 761 | 763 | priv->tx_skb = NULL; |
| @@ -790,6 +792,7 @@ static int hi3110_open(struct net_device *net) · 28 unmodified lines | ||
| 790 | 792 | hi3110_hw_sleep(spi); |
| 791 | 793 | out_close: |
| 792 | 794 | hi3110_power_enable(priv->transceiver, 0); |
| 795 | + out_close_candev: | |
| 793 | 796 | close_candev(net); |
| 794 | 797 | mutex_unlock(&priv->hi3110_lock); |
| 795 | 798 | return ret; |
| @@ -1214,6 +1214,7 @@ static int mcp251x_open(struct net_device *net) · 1213 unmodified lines | ||
| 1214 | 1214 | { |
| 1215 | 1215 | struct mcp251x_priv *priv = netdev_priv(net); |
| 1216 | 1216 | struct spi_device *spi = priv->spi; |
| 1217 | + bool release_irq = false; | |
| 1217 | 1218 | unsigned long flags = 0; |
| 1218 | 1219 | int ret; |
| 1219 | 1220 | |
| @@ -1257,12 +1258,24 @@ static int mcp251x_open(struct net_device *net) · 37 unmodified lines | ||
| 1257 | 1258 | return 0; |
| 1258 | 1259 | |
| 1259 | 1260 | out_free_irq: |
| 1260 | − free_irq(spi->irq, priv); | |
| 1261 | + /* The IRQ handler might be running, and if so it will be waiting | |
| 1262 | + * for the lock. But free_irq() must wait for the handler to finish | |
| 1263 | + * so calling it here would deadlock. | |
| 1264 | + * | |
| 1265 | + * Setting priv->force_quit will let the handler exit right away | |
| 1266 | + * without any access to the hardware. This make it safe to call | |
| 1267 | + * free_irq() after the lock is released. | |
| 1268 | + */ | |
| 1269 | + priv->force_quit = 1; | |
| 1270 | + release_irq = true; | |
| 1271 | + | |
| 1261 | 1272 | mcp251x_hw_sleep(spi); |
| 1262 | 1273 | out_close: |
| 1263 | 1274 | mcp251x_power_enable(priv->transceiver, 0); |
| 1264 | 1275 | close_candev(net); |
| 1265 | 1276 | mutex_unlock(&priv->mcp_lock); |
| 1277 | + if (release_irq) | |
| 1278 | + free_irq(spi->irq, priv); | |
| 1266 | 1279 | return ret; |
| 1267 | 1280 | } |
| 1268 | 1281 | |
| @@ -445,6 +445,11 @@ static void ems_usb_read_bulk_callback(struct urb *urb) · 444 unmodified lines | ||
| 445 | 445 | start = CPC_HEADER_SIZE; |
| 446 | 446 | |
| 447 | 447 | while (msg_count) { |
| 448 | + if (start + CPC_MSG_HEADER_LEN > urb->actual_length) { | |
| 449 | + netdev_err(netdev, "format error\n"); | |
| 450 | + break; | |
| 451 | + } | |
| 452 | + | |
| 448 | 453 | msg = (struct ems_cpc_msg *)&ibuf[start]; |
| 449 | 454 | |
| 450 | 455 | switch (msg->type) { |
| @@ -474,7 +479,7 @@ static void ems_usb_read_bulk_callback(struct urb *urb) · 23 unmodified lines | ||
| 474 | 479 | start += CPC_MSG_HEADER_LEN + msg->length; |
| 475 | 480 | msg_count--; |
| 476 | 481 | |
| 477 | − if (start > urb->transfer_buffer_length) { | |
| 482 | + if (start > urb->actual_length) { | |
| 478 | 483 | netdev_err(netdev, "format error\n"); |
| 479 | 484 | break; |
| 480 | 485 | } |
| @@ -272,6 +272,9 @@ struct esd_usb { · 271 unmodified lines | ||
| 272 | 272 | |
| 273 | 273 | struct usb_anchor rx_submitted; |
| 274 | 274 | |
| 275 | + unsigned int rx_pipe; | |
| 276 | + unsigned int tx_pipe; | |
| 277 | + | |
| 275 | 278 | int net_count; |
| 276 | 279 | u32 version; |
| 277 | 280 | int rxinitdone; |
| @@ -537,7 +540,7 @@ static void esd_usb_read_bulk_callback(struct urb *urb) · 259 unmodified lines | ||
| 537 | 540 | } |
| 538 | 541 | |
| 539 | 542 | resubmit_urb: |
| 540 | − usb_fill_bulk_urb(urb, dev->udev, usb_rcvbulkpipe(dev->udev, 1), | |
| 543 | + usb_fill_bulk_urb(urb, dev->udev, dev->rx_pipe, | |
| 541 | 544 | urb->transfer_buffer, ESD_USB_RX_BUFFER_SIZE, |
| 542 | 545 | esd_usb_read_bulk_callback, dev); |
| 543 | 546 | |
| @@ -626,9 +629,7 @@ static int esd_usb_send_msg(struct esd_usb *dev, union esd_usb_msg *msg) · 82 unmodified lines | ||
| 626 | 629 | { |
| 627 | 630 | int actual_length; |
| 628 | 631 | |
| 629 | − return usb_bulk_msg(dev->udev, | |
| 630 | − usb_sndbulkpipe(dev->udev, 2), | |
| 631 | − msg, | |
| 632 | + return usb_bulk_msg(dev->udev, dev->tx_pipe, msg, | |
| 632 | 633 | msg->hdr.len * sizeof(u32), /* convert to # of bytes */ |
| 633 | 634 | &actual_length, |
| 634 | 635 | 1000); |
| @@ -639,12 +640,8 @@ static int esd_usb_wait_msg(struct esd_usb *dev, · 4 unmodified lines | ||
| 639 | 640 | { |
| 640 | 641 | int actual_length; |
| 641 | 642 | |
| 642 | − return usb_bulk_msg(dev->udev, | |
| 643 | − usb_rcvbulkpipe(dev->udev, 1), | |
| 644 | − msg, | |
| 645 | − sizeof(*msg), | |
| 646 | − &actual_length, | |
| 647 | − 1000); | |
| 643 | + return usb_bulk_msg(dev->udev, dev->rx_pipe, msg, | |
| 644 | + sizeof(*msg), &actual_length, 1000); | |
| 648 | 645 | } |
| 649 | 646 | |
| 650 | 647 | static int esd_usb_setup_rx_urbs(struct esd_usb *dev) |
| @@ -677,8 +674,7 @@ static int esd_usb_setup_rx_urbs(struct esd_usb *dev) · 26 unmodified lines | ||
| 677 | 674 | |
| 678 | 675 | urb->transfer_dma = buf_dma; |
| 679 | 676 | |
| 680 | − usb_fill_bulk_urb(urb, dev->udev, | |
| 681 | − usb_rcvbulkpipe(dev->udev, 1), | |
| 677 | + usb_fill_bulk_urb(urb, dev->udev, dev->rx_pipe, | |
| 682 | 678 | buf, ESD_USB_RX_BUFFER_SIZE, |
| 683 | 679 | esd_usb_read_bulk_callback, dev); |
| 684 | 680 | urb->transfer_flags |= URB_NO_TRANSFER_DMA_MAP; |
| @@ -903,7 +899,7 @@ static netdev_tx_t esd_usb_start_xmit(struct sk_buff *skb, · 218 unmodified lines | ||
| 903 | 899 | /* hnd must not be 0 - MSB is stripped in txdone handling */ |
| 904 | 900 | msg->tx.hnd = BIT(31) | i; /* returned in TX done message */ |
| 905 | 901 | |
| 906 | − usb_fill_bulk_urb(urb, dev->udev, usb_sndbulkpipe(dev->udev, 2), buf, | |
| 902 | + usb_fill_bulk_urb(urb, dev->udev, dev->tx_pipe, buf, | |
| 907 | 903 | msg->hdr.len * sizeof(u32), /* convert to # of bytes */ |
| 908 | 904 | esd_usb_write_bulk_callback, context); |
| 909 | 905 | |
| @@ -1298,10 +1294,16 @@ done: · 388 unmodified lines | ||
| 1298 | 1294 | static int esd_usb_probe(struct usb_interface *intf, |
| 1299 | 1295 | const struct usb_device_id *id) |
| 1300 | 1296 | { |
| 1297 | + struct usb_endpoint_descriptor *ep_in, *ep_out; | |
| 1301 | 1298 | struct esd_usb *dev; |
| 1302 | 1299 | union esd_usb_msg *msg; |
| 1303 | 1300 | int i, err; |
| 1304 | 1301 | |
| 1302 | + err = usb_find_common_endpoints(intf->cur_altsetting, &ep_in, &ep_out, | |
| 1303 | + NULL, NULL); | |
| 1304 | + if (err) | |
| 1305 | + return err; | |
| 1306 | + | |
| 1305 | 1307 | dev = kzalloc_obj(*dev); |
| 1306 | 1308 | if (!dev) { |
| 1307 | 1309 | err = -ENOMEM; |
| @@ -1309,6 +1311,8 @@ static int esd_usb_probe(struct usb_interface *intf, · 1 unmodified line | ||
| 1309 | 1311 | } |
| 1310 | 1312 | |
| 1311 | 1313 | dev->udev = interface_to_usbdev(intf); |
| 1314 | + dev->rx_pipe = usb_rcvbulkpipe(dev->udev, ep_in->bEndpointAddress); | |
| 1315 | + dev->tx_pipe = usb_sndbulkpipe(dev->udev, ep_out->bEndpointAddress); | |
| 1312 | 1316 | |
| 1313 | 1317 | init_usb_anchor(&dev->rx_submitted); |
| 1314 | 1318 | |
| @@ -1461,12 +1461,18 @@ static void es58x_read_bulk_callback(struct urb *urb) · 1460 unmodified lines | ||
| 1461 | 1461 | } |
| 1462 | 1462 | |
| 1463 | 1463 | resubmit_urb: |
| 1464 | + usb_anchor_urb(urb, &es58x_dev->rx_urbs); | |
| 1464 | 1465 | ret = usb_submit_urb(urb, GFP_ATOMIC); |
| 1466 | + if (!ret) | |
| 1467 | + return; | |
| 1468 | + | |
| 1469 | + usb_unanchor_urb(urb); | |
| 1470 | + | |
| 1465 | 1471 | if (ret == -ENODEV) { |
| 1466 | 1472 | for (i = 0; i < es58x_dev->num_can_ch; i++) |
| 1467 | 1473 | if (es58x_dev->netdev[i]) |
| 1468 | 1474 | netif_device_detach(es58x_dev->netdev[i]); |
| 1469 | − } else if (ret) | |
| 1475 | + } else | |
| 1470 | 1476 | dev_err_ratelimited(dev, |
| 1471 | 1477 | "Failed resubmitting read bulk urb: %pe\n", |
| 1472 | 1478 | ERR_PTR(ret)); |
| @@ -413,6 +413,7 @@ static void f81604_read_bulk_callback(struct urb *urb) · 412 unmodified lines | ||
| 413 | 413 | { |
| 414 | 414 | struct f81604_can_frame *frame = urb->transfer_buffer; |
| 415 | 415 | struct net_device *netdev = urb->context; |
| 416 | + struct f81604_port_priv *priv = netdev_priv(netdev); | |
| 416 | 417 | int ret; |
| 417 | 418 | |
| 418 | 419 | if (!netif_device_present(netdev)) |
| @@ -445,10 +446,15 @@ static void f81604_read_bulk_callback(struct urb *urb) · 26 unmodified lines | ||
| 445 | 446 | f81604_process_rx_packet(netdev, frame); |
| 446 | 447 | |
| 447 | 448 | resubmit_urb: |
| 449 | + usb_anchor_urb(urb, &priv->urbs_anchor); | |
| 448 | 450 | ret = usb_submit_urb(urb, GFP_ATOMIC); |
| 451 | + if (!ret) | |
| 452 | + return; | |
| 453 | + usb_unanchor_urb(urb); | |
| 454 | + | |
| 449 | 455 | if (ret == -ENODEV) |
| 450 | 456 | netif_device_detach(netdev); |
| 451 | − else if (ret) | |
| 457 | + else | |
| 452 | 458 | netdev_err(netdev, |
| 453 | 459 | "%s: failed to resubmit read bulk urb: %pe\n", |
| 454 | 460 | __func__, ERR_PTR(ret)); |
| @@ -620,6 +626,12 @@ static void f81604_read_int_callback(struct urb *urb) · 165 unmodified lines | ||
| 620 | 626 | netdev_info(netdev, "%s: Int URB aborted: %pe\n", __func__, |
| 621 | 627 | ERR_PTR(urb->status)); |
| 622 | 628 | |
| 629 | + if (urb->actual_length < sizeof(*data)) { | |
| 630 | + netdev_warn(netdev, "%s: short int URB: %u < %zu\n", | |
| 631 | + __func__, urb->actual_length, sizeof(*data)); | |
| 632 | + goto resubmit_urb; | |
| 633 | + } | |
| 634 | + | |
| 623 | 635 | switch (urb->status) { |
| 624 | 636 | case 0: /* success */ |
| 625 | 637 | break; |
| @@ -646,10 +658,15 @@ static void f81604_read_int_callback(struct urb *urb) · 20 unmodified lines | ||
| 646 | 658 | f81604_handle_tx(priv, data); |
| 647 | 659 | |
| 648 | 660 | resubmit_urb: |
| 661 | + usb_anchor_urb(urb, &priv->urbs_anchor); | |
| 649 | 662 | ret = usb_submit_urb(urb, GFP_ATOMIC); |
| 663 | + if (!ret) | |
| 664 | + return; | |
| 665 | + usb_unanchor_urb(urb); | |
| 666 | + | |
| 650 | 667 | if (ret == -ENODEV) |
| 651 | 668 | netif_device_detach(netdev); |
| 652 | − else if (ret) | |
| 669 | + else | |
| 653 | 670 | netdev_err(netdev, "%s: failed to resubmit int urb: %pe\n", |
| 654 | 671 | __func__, ERR_PTR(ret)); |
| 655 | 672 | } |
| @@ -874,9 +891,27 @@ static void f81604_write_bulk_callback(struct urb *urb) · 218 unmodified lines | ||
| 874 | 891 | if (!netif_device_present(netdev)) |
| 875 | 892 | return; |
| 876 | 893 | |
| 877 | − if (urb->status) | |
| 878 | − netdev_info(netdev, "%s: Tx URB error: %pe\n", __func__, | |
| 879 | − ERR_PTR(urb->status)); | |
| 894 | + if (!urb->status) | |
| 895 | + return; | |
| 896 | + | |
| 897 | + switch (urb->status) { | |
| 898 | + case -ENOENT: | |
| 899 | + case -ECONNRESET: | |
| 900 | + case -ESHUTDOWN: | |
| 901 | + return; | |
| 902 | + default: | |
| 903 | + break; | |
| 904 | + } | |
| 905 | + | |
| 906 | + if (net_ratelimit()) | |
| 907 | + netdev_err(netdev, "%s: Tx URB error: %pe\n", __func__, | |
| 908 | + ERR_PTR(urb->status)); | |
| 909 | + | |
| 910 | + can_free_echo_skb(netdev, 0, NULL); | |
| 911 | + netdev->stats.tx_dropped++; | |
| 912 | + netdev->stats.tx_errors++; | |
| 913 | + | |
| 914 | + netif_wake_queue(netdev); | |
| 880 | 915 | } |
| 881 | 916 | |
| 882 | 917 | static void f81604_clear_reg_work(struct work_struct *work) |
| @@ -772,9 +772,8 @@ device_detach: · 771 unmodified lines | ||
| 772 | 772 | } |
| 773 | 773 | } |
| 774 | 774 | |
| 775 | −static int gs_usb_set_bittiming(struct net_device *netdev) | |
| 775 | +static int gs_usb_set_bittiming(struct gs_can *dev) | |
| 776 | 776 | { |
| 777 | − struct gs_can *dev = netdev_priv(netdev); | |
| 778 | 777 | struct can_bittiming *bt = &dev->can.bittiming; |
| 779 | 778 | struct gs_device_bittiming dbt = { |
| 780 | 779 | .prop_seg = cpu_to_le32(bt->prop_seg), |
| @@ -791,9 +790,8 @@ static int gs_usb_set_bittiming(struct net_device *netdev) · 10 unmodified lines | ||
| 791 | 790 | GFP_KERNEL); |
| 792 | 791 | } |
| 793 | 792 | |
| 794 | −static int gs_usb_set_data_bittiming(struct net_device *netdev) | |
| 793 | +static int gs_usb_set_data_bittiming(struct gs_can *dev) | |
| 795 | 794 | { |
| 796 | − struct gs_can *dev = netdev_priv(netdev); | |
| 797 | 795 | struct can_bittiming *bt = &dev->can.fd.data_bittiming; |
| 798 | 796 | struct gs_device_bittiming dbt = { |
| 799 | 797 | .prop_seg = cpu_to_le32(bt->prop_seg), |
| @@ -1057,6 +1055,20 @@ static int gs_can_open(struct net_device *netdev) · 257 unmodified lines | ||
| 1057 | 1055 | if (dev->feature & GS_CAN_FEATURE_HW_TIMESTAMP) |
| 1058 | 1056 | flags |= GS_CAN_MODE_HW_TIMESTAMP; |
| 1059 | 1057 | |
| 1058 | + rc = gs_usb_set_bittiming(dev); | |
| 1059 | + if (rc) { | |
| 1060 | + netdev_err(netdev, "failed to set bittiming: %pe\n", ERR_PTR(rc)); | |
| 1061 | + goto out_usb_kill_anchored_urbs; | |
| 1062 | + } | |
| 1063 | + | |
| 1064 | + if (ctrlmode & CAN_CTRLMODE_FD) { | |
| 1065 | + rc = gs_usb_set_data_bittiming(dev); | |
| 1066 | + if (rc) { | |
| 1067 | + netdev_err(netdev, "failed to set data bittiming: %pe\n", ERR_PTR(rc)); | |
| 1068 | + goto out_usb_kill_anchored_urbs; | |
| 1069 | + } | |
| 1070 | + } | |
| 1071 | + | |
| 1060 | 1072 | /* finally start device */ |
| 1061 | 1073 | dev->can.state = CAN_STATE_ERROR_ACTIVE; |
| 1062 | 1074 | dm.flags = cpu_to_le32(flags); |
| @@ -1370,7 +1382,6 @@ static struct gs_can *gs_make_candev(unsigned int channel, · 307 unmodified lines | ||
| 1370 | 1382 | dev->can.state = CAN_STATE_STOPPED; |
| 1371 | 1383 | dev->can.clock.freq = le32_to_cpu(bt_const.fclk_can); |
| 1372 | 1384 | dev->can.bittiming_const = &dev->bt_const; |
| 1373 | − dev->can.do_set_bittiming = gs_usb_set_bittiming; | |
| 1374 | 1385 | |
| 1375 | 1386 | dev->can.ctrlmode_supported = CAN_CTRLMODE_CC_LEN8_DLC; |
| 1376 | 1387 | |
| @@ -1394,7 +1405,6 @@ static struct gs_can *gs_make_candev(unsigned int channel, · 17 unmodified lines | ||
| 1394 | 1405 | * GS_CAN_FEATURE_BT_CONST_EXT is set. |
| 1395 | 1406 | */ |
| 1396 | 1407 | dev->can.fd.data_bittiming_const = &dev->bt_const; |
| 1397 | − dev->can.fd.do_set_data_bittiming = gs_usb_set_data_bittiming; | |
| 1398 | 1408 | } |
| 1399 | 1409 | |
| 1400 | 1410 | if (feature & GS_CAN_FEATURE_TERMINATION) { |
| @@ -748,7 +748,7 @@ static void ucan_read_bulk_callback(struct urb *urb) · 747 unmodified lines | ||
| 748 | 748 | len = le16_to_cpu(m->len); |
| 749 | 749 | |
| 750 | 750 | /* check sanity (length of content) */ |
| 751 | − if (urb->actual_length - pos < len) { | |
| 751 | + if ((len == 0) || (urb->actual_length - pos < len)) { | |
| 752 | 752 | netdev_warn(up->netdev, |
| 753 | 753 | "invalid message (short; no data; l:%d)\n", |
| 754 | 754 | urb->actual_length); |
| @@ -1108,6 +1108,7 @@ static int ksz_ptp_msg_irq_setup(struct ksz_port *port, u8 n) · 1107 unmodified lines | ||
| 1108 | 1108 | const struct ksz_dev_ops *ops = port->ksz_dev->dev_ops; |
| 1109 | 1109 | struct ksz_irq *ptpirq = &port->ptpirq; |
| 1110 | 1110 | struct ksz_ptp_irq *ptpmsg_irq; |
| 1111 | + int ret; | |
| 1111 | 1112 | |
| 1112 | 1113 | ptpmsg_irq = &port->ptpmsg_irq[n]; |
| 1113 | 1114 | ptpmsg_irq->num = irq_create_mapping(ptpirq->domain, n); |
| @@ -1119,9 +1120,13 @@ static int ksz_ptp_msg_irq_setup(struct ksz_port *port, u8 n) · 5 unmodified lines | ||
| 1119 | 1120 | |
| 1120 | 1121 | strscpy(ptpmsg_irq->name, name[n]); |
| 1121 | 1122 | |
| 1122 | − return request_threaded_irq(ptpmsg_irq->num, NULL, | |
| 1123 | − ksz_ptp_msg_thread_fn, IRQF_ONESHOT, | |
| 1124 | − ptpmsg_irq->name, ptpmsg_irq); | |
| 1123 | + ret = request_threaded_irq(ptpmsg_irq->num, NULL, | |
| 1124 | + ksz_ptp_msg_thread_fn, IRQF_ONESHOT, | |
| 1125 | + ptpmsg_irq->name, ptpmsg_irq); | |
| 1126 | + if (ret) | |
| 1127 | + irq_dispose_mapping(ptpmsg_irq->num); | |
| 1128 | + | |
| 1129 | + return ret; | |
| 1125 | 1130 | } |
| 1126 | 1131 | |
| 1127 | 1132 | int ksz_ptp_irq_setup(struct dsa_switch *ds, u8 p) |
| @@ -149,7 +149,6 @@ static int mxl862xx_setup_mdio(struct dsa_switch *ds) · 148 unmodified lines | ||
| 149 | 149 | return -ENOMEM; |
| 150 | 150 | |
| 151 | 151 | bus->priv = priv; |
| 152 | − ds->user_mii_bus = bus; | |
| 153 | 152 | bus->name = KBUILD_MODNAME "-mii"; |
| 154 | 153 | snprintf(bus->id, MII_BUS_ID_SIZE, "%s-mii", dev_name(dev)); |
| 155 | 154 | bus->read_c45 = mxl862xx_phy_read_c45_mii_bus; |
| @@ -769,7 +769,7 @@ static int rtl8365mb_phy_ocp_write(struct realtek_priv *priv, int phy, · 768 unmodified lines | ||
| 769 | 769 | out: |
| 770 | 770 | rtl83xx_unlock(priv); |
| 771 | 771 | |
| 772 | − return 0; | |
| 772 | + return ret; | |
| 773 | 773 | } |
| 774 | 774 | |
| 775 | 775 | static int rtl8365mb_phy_read(struct realtek_priv *priv, int phy, int regnum) |
| @@ -1480,8 +1480,7 @@ static void rtl8365mb_stats_update(struct realtek_priv *priv, int port) · 704 unmodified lines | ||
| 1480 | 1480 | |
| 1481 | 1481 | stats->rx_packets = cnt[RTL8365MB_MIB_ifInUcastPkts] + |
| 1482 | 1482 | cnt[RTL8365MB_MIB_ifInMulticastPkts] + |
| 1483 | − cnt[RTL8365MB_MIB_ifInBroadcastPkts] - | |
| 1484 | − cnt[RTL8365MB_MIB_ifOutDiscards]; | |
| 1483 | + cnt[RTL8365MB_MIB_ifInBroadcastPkts]; | |
| 1485 | 1484 | |
| 1486 | 1485 | stats->tx_packets = cnt[RTL8365MB_MIB_ifOutUcastPkts] + |
| 1487 | 1486 | cnt[RTL8365MB_MIB_ifOutMulticastPkts] + |
| @@ -12,11 +12,11 @@ static inline u32 rtl8366rb_led_group_port_mask(u8 led_group, u8 port) · 11 unmodified lines | ||
| 12 | 12 | case 0: |
| 13 | 13 | return FIELD_PREP(RTL8366RB_LED_0_X_CTRL_MASK, BIT(port)); |
| 14 | 14 | case 1: |
| 15 | − return FIELD_PREP(RTL8366RB_LED_0_X_CTRL_MASK, BIT(port)); | |
| 15 | + return FIELD_PREP(RTL8366RB_LED_X_1_CTRL_MASK, BIT(port)); | |
| 16 | 16 | case 2: |
| 17 | − return FIELD_PREP(RTL8366RB_LED_0_X_CTRL_MASK, BIT(port)); | |
| 17 | + return FIELD_PREP(RTL8366RB_LED_2_X_CTRL_MASK, BIT(port)); | |
| 18 | 18 | case 3: |
| 19 | − return FIELD_PREP(RTL8366RB_LED_0_X_CTRL_MASK, BIT(port)); | |
| 19 | + return FIELD_PREP(RTL8366RB_LED_X_3_CTRL_MASK, BIT(port)); | |
| 20 | 20 | default: |
| 21 | 21 | return 0; |
| 22 | 22 | } |
| @@ -2278,6 +2278,12 @@ int sja1105_static_config_reload(struct sja1105_private *priv, · 2277 unmodified lines | ||
| 2278 | 2278 | * change it through the dynamic interface later. |
| 2279 | 2279 | */ |
| 2280 | 2280 | dsa_switch_for_each_available_port(dp, ds) { |
| 2281 | + /* May be called during unbind when we unoffload a VLAN-aware | |
| 2282 | + * bridge from port 1 while port 0 was already torn down | |
| 2283 | + */ | |
| 2284 | + if (!dp->pl) | |
| 2285 | + continue; | |
| 2286 | + | |
| 2281 | 2287 | phylink_replay_link_begin(dp->pl); |
| 2282 | 2288 | mac[dp->index].speed = priv->info->port_speed[SJA1105_SPEED_AUTO]; |
| 2283 | 2289 | } |
| @@ -2333,13 +2339,13 @@ int sja1105_static_config_reload(struct sja1105_private *priv, · 49 unmodified lines | ||
| 2333 | 2339 | goto out; |
| 2334 | 2340 | } |
| 2335 | 2341 | |
| 2336 | − dsa_switch_for_each_available_port(dp, ds) | |
| 2337 | − phylink_replay_link_end(dp->pl); | |
| 2338 | − | |
| 2339 | 2342 | rc = sja1105_reload_cbs(priv); |
| 2340 | − if (rc < 0) | |
| 2341 | − goto out; | |
| 2343 | + | |
| 2342 | 2344 | out: |
| 2345 | + dsa_switch_for_each_available_port(dp, ds) | |
| 2346 | + if (dp->pl) | |
| 2347 | + phylink_replay_link_end(dp->pl); | |
| 2348 | + | |
| 2343 | 2349 | mutex_unlock(&priv->mgmt_lock); |
| 2344 | 2350 | mutex_unlock(&priv->fdb_lock); |
| 2345 | 2351 | |
| @@ -431,7 +431,7 @@ · 430 unmodified lines | ||
| 431 | 431 | #define MAC_SSIR_SSINC_INDEX 16 |
| 432 | 432 | #define MAC_SSIR_SSINC_WIDTH 8 |
| 433 | 433 | #define MAC_TCR_SS_INDEX 29 |
| 434 | −#define MAC_TCR_SS_WIDTH 2 | |
| 434 | +#define MAC_TCR_SS_WIDTH 3 | |
| 435 | 435 | #define MAC_TCR_TE_INDEX 0 |
| 436 | 436 | #define MAC_TCR_TE_WIDTH 1 |
| 437 | 437 | #define MAC_TCR_VNE_INDEX 24 |
| @@ -1120,7 +1120,6 @@ int xgbe_powerdown(struct net_device *netdev, unsigned int caller) · 1119 unmodified lines | ||
| 1120 | 1120 | { |
| 1121 | 1121 | struct xgbe_prv_data *pdata = netdev_priv(netdev); |
| 1122 | 1122 | struct xgbe_hw_if *hw_if = &pdata->hw_if; |
| 1123 | − unsigned long flags; | |
| 1124 | 1123 | |
| 1125 | 1124 | DBGPR("-->xgbe_powerdown\n"); |
| 1126 | 1125 | |
| @@ -1131,8 +1130,6 @@ int xgbe_powerdown(struct net_device *netdev, unsigned int caller) · 4 unmodified lines | ||
| 1131 | 1130 | return -EINVAL; |
| 1132 | 1131 | } |
| 1133 | 1132 | |
| 1134 | − spin_lock_irqsave(&pdata->lock, flags); | |
| 1135 | − | |
| 1136 | 1133 | if (caller == XGMAC_DRIVER_CONTEXT) |
| 1137 | 1134 | netif_device_detach(netdev); |
| 1138 | 1135 | |
| @@ -1148,8 +1145,6 @@ int xgbe_powerdown(struct net_device *netdev, unsigned int caller) · 9 unmodified lines | ||
| 1148 | 1145 | |
| 1149 | 1146 | pdata->power_down = 1; |
| 1150 | 1147 | |
| 1151 | − spin_unlock_irqrestore(&pdata->lock, flags); | |
| 1152 | − | |
| 1153 | 1148 | DBGPR("<--xgbe_powerdown\n"); |
| 1154 | 1149 | |
| 1155 | 1150 | return 0; |
| @@ -1159,7 +1154,6 @@ int xgbe_powerup(struct net_device *netdev, unsigned int caller) · 3 unmodified lines | ||
| 1159 | 1154 | { |
| 1160 | 1155 | struct xgbe_prv_data *pdata = netdev_priv(netdev); |
| 1161 | 1156 | struct xgbe_hw_if *hw_if = &pdata->hw_if; |
| 1162 | − unsigned long flags; | |
| 1163 | 1157 | |
| 1164 | 1158 | DBGPR("-->xgbe_powerup\n"); |
| 1165 | 1159 | |
| @@ -1170,8 +1164,6 @@ int xgbe_powerup(struct net_device *netdev, unsigned int caller) · 4 unmodified lines | ||
| 1170 | 1164 | return -EINVAL; |
| 1171 | 1165 | } |
| 1172 | 1166 | |
| 1173 | − spin_lock_irqsave(&pdata->lock, flags); | |
| 1174 | − | |
| 1175 | 1167 | pdata->power_down = 0; |
| 1176 | 1168 | |
| 1177 | 1169 | xgbe_napi_enable(pdata, 0); |
| @@ -1186,8 +1178,6 @@ int xgbe_powerup(struct net_device *netdev, unsigned int caller) · 8 unmodified lines | ||
| 1186 | 1178 | |
| 1187 | 1179 | xgbe_start_timers(pdata); |
| 1188 | 1180 | |
| 1189 | − spin_unlock_irqrestore(&pdata->lock, flags); | |
| 1190 | − | |
| 1191 | 1181 | DBGPR("<--xgbe_powerup\n"); |
| 1192 | 1182 | |
| 1193 | 1183 | return 0; |
| @@ -1281,20 +1271,25 @@ static int xgbe_start(struct xgbe_prv_data *pdata) · 87 unmodified lines | ||
| 1281 | 1271 | if (ret) |
| 1282 | 1272 | goto err_napi; |
| 1283 | 1273 | |
| 1274 | + /* Reset the phy settings */ | |
| 1275 | + ret = xgbe_phy_reset(pdata); | |
| 1276 | + if (ret) | |
| 1277 | + goto err_irqs; | |
| 1278 | + | |
| 1279 | + /* Start the phy */ | |
| 1284 | 1280 | ret = phy_if->phy_start(pdata); |
| 1285 | 1281 | if (ret) |
| 1286 | 1282 | goto err_irqs; |
| 1287 | 1283 | |
| 1288 | 1284 | hw_if->enable_tx(pdata); |
| 1289 | 1285 | hw_if->enable_rx(pdata); |
| 1286 | + /* Synchronize flag with hardware state after enabling TX/RX. | |
| 1287 | + * This prevents stale state after device restart cycles. | |
| 1288 | + */ | |
| 1289 | + pdata->data_path_stopped = false; | |
| 1290 | 1290 | |
| 1291 | 1291 | udp_tunnel_nic_reset_ntf(netdev); |
| 1292 | 1292 | |
| 1293 | − /* Reset the phy settings */ | |
| 1294 | − ret = xgbe_phy_reset(pdata); | |
| 1295 | − if (ret) | |
| 1296 | − goto err_txrx; | |
| 1297 | − | |
| 1298 | 1293 | netif_tx_start_all_queues(netdev); |
| 1299 | 1294 | |
| 1300 | 1295 | xgbe_start_timers(pdata); |
| @@ -1304,10 +1299,6 @@ static int xgbe_start(struct xgbe_prv_data *pdata) · 3 unmodified lines | ||
| 1304 | 1299 | |
| 1305 | 1300 | return 0; |
| 1306 | 1301 | |
| 1307 | −err_txrx: | |
| 1308 | − hw_if->disable_rx(pdata); | |
| 1309 | − hw_if->disable_tx(pdata); | |
| 1310 | − | |
| 1311 | 1302 | err_irqs: |
| 1312 | 1303 | xgbe_free_irqs(pdata); |
| 1313 | 1304 | |
| @@ -76,7 +76,6 @@ struct xgbe_prv_data *xgbe_alloc_pdata(struct device *dev) · 75 unmodified lines | ||
| 76 | 76 | pdata->netdev = netdev; |
| 77 | 77 | pdata->dev = dev; |
| 78 | 78 | |
| 79 | − spin_lock_init(&pdata->lock); | |
| 80 | 79 | spin_lock_init(&pdata->xpcs_lock); |
| 81 | 80 | mutex_init(&pdata->rss_mutex); |
| 82 | 81 | spin_lock_init(&pdata->tstamp_lock); |
| @@ -1942,7 +1942,7 @@ static void xgbe_set_rx_adap_mode(struct xgbe_prv_data *pdata, · 1941 unmodified lines | ||
| 1942 | 1942 | static void xgbe_rx_adaptation(struct xgbe_prv_data *pdata) |
| 1943 | 1943 | { |
| 1944 | 1944 | struct xgbe_phy_data *phy_data = pdata->phy_data; |
| 1945 | − unsigned int reg; | |
| 1945 | + int reg; | |
| 1946 | 1946 | |
| 1947 | 1947 | /* step 2: force PCS to send RX_ADAPT Req to PHY */ |
| 1948 | 1948 | XMDIO_WRITE_BITS(pdata, MDIO_MMD_PMAPMD, MDIO_PMA_RX_EQ_CTRL4, |
| @@ -1964,11 +1964,20 @@ static void xgbe_rx_adaptation(struct xgbe_prv_data *pdata) · 15 unmodified lines | ||
| 1964 | 1964 | |
| 1965 | 1965 | /* Step 4: Check for Block lock */ |
| 1966 | 1966 | |
| 1967 | − /* Link status is latched low, so read once to clear | |
| 1968 | − * and then read again to get current state | |
| 1969 | − */ | |
| 1970 | − reg = XMDIO_READ(pdata, MDIO_MMD_PCS, MDIO_STAT1); | |
| 1971 | 1967 | reg = XMDIO_READ(pdata, MDIO_MMD_PCS, MDIO_STAT1); |
| 1968 | + if (reg < 0) | |
| 1969 | + goto set_mode; | |
| 1970 | + | |
| 1971 | + /* Link status is latched low so that momentary link drops | |
| 1972 | + * can be detected. If link was already down read again | |
| 1973 | + * to get the latest state. | |
| 1974 | + */ | |
| 1975 | + if (!pdata->phy.link && !(reg & MDIO_STAT1_LSTATUS)) { | |
| 1976 | + reg = XMDIO_READ(pdata, MDIO_MMD_PCS, MDIO_STAT1); | |
| 1977 | + if (reg < 0) | |
| 1978 | + goto set_mode; | |
| 1979 | + } | |
| 1980 | + | |
| 1972 | 1981 | if (reg & MDIO_STAT1_LSTATUS) { |
| 1973 | 1982 | /* If the block lock is found, update the helpers |
| 1974 | 1983 | * and declare the link up |
| @@ -2008,6 +2017,48 @@ rx_adapt_reinit: · 33 unmodified lines | ||
| 2008 | 2017 | xgbe_rx_adaptation(pdata); |
| 2009 | 2018 | } |
| 2010 | 2019 | |
| 2020 | +/* | |
| 2021 | + * xgbe_phy_stop_data_path - Stop TX/RX to prevent packet corruption | |
| 2022 | + * @pdata: driver private data | |
| 2023 | + * | |
| 2024 | + * This function stops the data path (TX and RX) to prevent packet | |
| 2025 | + * corruption during critical PHY operations like RX adaptation. | |
| 2026 | + * Must be called before initiating RX adaptation when link goes down. | |
| 2027 | + */ | |
| 2028 | +static void xgbe_phy_stop_data_path(struct xgbe_prv_data *pdata) | |
| 2029 | +{ | |
| 2030 | + if (pdata->data_path_stopped) | |
| 2031 | + return; | |
| 2032 | + | |
| 2033 | + /* Stop TX/RX to prevent packet corruption during RX adaptation */ | |
| 2034 | + pdata->hw_if.disable_tx(pdata); | |
| 2035 | + pdata->hw_if.disable_rx(pdata); | |
| 2036 | + pdata->data_path_stopped = true; | |
| 2037 | + | |
| 2038 | + netif_dbg(pdata, link, pdata->netdev, | |
| 2039 | + "stopping data path for RX adaptation\n"); | |
| 2040 | +} | |
| 2041 | + | |
| 2042 | +/* | |
| 2043 | + * xgbe_phy_start_data_path - Re-enable TX/RX after RX adaptation | |
| 2044 | + * @pdata: driver private data | |
| 2045 | + * | |
| 2046 | + * This function re-enables the data path (TX and RX) after RX adaptation | |
| 2047 | + * has completed successfully. Only called when link is confirmed up. | |
| 2048 | + */ | |
| 2049 | +static void xgbe_phy_start_data_path(struct xgbe_prv_data *pdata) | |
| 2050 | +{ | |
| 2051 | + if (!pdata->data_path_stopped) | |
| 2052 | + return; | |
| 2053 | + | |
| 2054 | + pdata->hw_if.enable_rx(pdata); | |
| 2055 | + pdata->hw_if.enable_tx(pdata); | |
| 2056 | + pdata->data_path_stopped = false; | |
| 2057 | + | |
| 2058 | + netif_dbg(pdata, link, pdata->netdev, | |
| 2059 | + "restarting data path after RX adaptation\n"); | |
| 2060 | +} | |
| 2061 | + | |
| 2011 | 2062 | static void xgbe_phy_rx_reset(struct xgbe_prv_data *pdata) |
| 2012 | 2063 | { |
| 2013 | 2064 | int reg; |
| @@ -2801,13 +2852,27 @@ static int xgbe_phy_link_status(struct xgbe_prv_data *pdata, int *an_restart) · 787 unmodified lines | ||
| 2801 | 2852 | if (pdata->en_rx_adap) { |
| 2802 | 2853 | /* if the link is available and adaptation is done, |
| 2803 | 2854 | * declare link up |
| 2855 | + * | |
| 2856 | + * Note: When link is up and adaptation is done, we can | |
| 2857 | + * safely re-enable the data path if it was stopped | |
| 2858 | + * for adaptation. | |
| 2804 | 2859 | */ |
| 2805 | − if ((reg & MDIO_STAT1_LSTATUS) && pdata->rx_adapt_done) | |
| 2860 | + if ((reg & MDIO_STAT1_LSTATUS) && pdata->rx_adapt_done) { | |
| 2861 | + xgbe_phy_start_data_path(pdata); | |
| 2806 | 2862 | return 1; |
| 2863 | + } | |
| 2807 | 2864 | /* If either link is not available or adaptation is not done, |
| 2808 | 2865 | * retrigger the adaptation logic. (if the mode is not set, |
| 2809 | 2866 | * then issue mailbox command first) |
| 2810 | 2867 | */ |
| 2868 | + | |
| 2869 | + /* CRITICAL: Stop data path BEFORE triggering RX adaptation | |
| 2870 | + * to prevent CRC errors from packets corrupted during | |
| 2871 | + * the adaptation process. This is especially important | |
| 2872 | + * when AN is OFF in 10G KR mode. | |
| 2873 | + */ | |
| 2874 | + xgbe_phy_stop_data_path(pdata); | |
| 2875 | + | |
| 2811 | 2876 | if (pdata->mode_set) { |
| 2812 | 2877 | xgbe_phy_rx_adaptation(pdata); |
| 2813 | 2878 | } else { |
| @@ -2815,8 +2880,11 @@ static int xgbe_phy_link_status(struct xgbe_prv_data *pdata, int *an_restart) · 1 unmodified line | ||
| 2815 | 2880 | xgbe_phy_set_mode(pdata, phy_data->cur_mode); |
| 2816 | 2881 | } |
| 2817 | 2882 | |
| 2818 | − if (pdata->rx_adapt_done) | |
| 2883 | + if (pdata->rx_adapt_done) { | |
| 2884 | + /* Adaptation complete, safe to re-enable data path */ | |
| 2885 | + xgbe_phy_start_data_path(pdata); | |
| 2819 | 2886 | return 1; |
| 2887 | + } | |
| 2820 | 2888 | } else if (reg & MDIO_STAT1_LSTATUS) |
| 2821 | 2889 | return 1; |
| 2822 | 2890 | |
| @@ -1004,9 +1004,6 @@ struct xgbe_prv_data { · 1003 unmodified lines | ||
| 1004 | 1004 | unsigned int pp3; |
| 1005 | 1005 | unsigned int pp4; |
| 1006 | 1006 | |
| 1007 | − /* Overall device lock */ | |
| 1008 | − spinlock_t lock; | |
| 1009 | − | |
| 1010 | 1007 | /* XPCS indirect addressing lock */ |
| 1011 | 1008 | spinlock_t xpcs_lock; |
| 1012 | 1009 | unsigned int xpcs_window_def_reg; |
| @@ -1246,6 +1243,10 @@ struct xgbe_prv_data { · 233 unmodified lines | ||
| 1246 | 1243 | bool en_rx_adap; |
| 1247 | 1244 | int rx_adapt_retries; |
| 1248 | 1245 | bool rx_adapt_done; |
| 1246 | + /* Flag to track if data path (TX/RX) was stopped for RX adaptation. | |
| 1247 | + * This prevents packet corruption during the adaptation window. | |
| 1248 | + */ | |
| 1249 | + bool data_path_stopped; | |
| 1249 | 1250 | bool mode_set; |
| 1250 | 1251 | bool sph; |
| 1251 | 1252 | }; |
| @@ -934,6 +934,17 @@ int arc_emac_probe(struct net_device *ndev, int interface) · 933 unmodified lines | ||
| 934 | 934 | /* Set poll rate so that it polls every 1 ms */ |
| 935 | 935 | arc_reg_set(priv, R_POLLRATE, clock_frequency / 1000000); |
| 936 | 936 | |
| 937 | + /* | |
| 938 | + * Put the device into a known quiescent state before requesting | |
| 939 | + * the IRQ. Clear only EMAC interrupt status bits here; leave the | |
| 940 | + * MDIO completion bit alone and avoid writing TXPL_MASK, which is | |
| 941 | + * used to force TX polling rather than acknowledge interrupts. | |
| 942 | + */ | |
| 943 | + arc_reg_set(priv, R_ENABLE, 0); | |
| 944 | + arc_reg_set(priv, R_STATUS, RXINT_MASK | TXINT_MASK | ERR_MASK | | |
| 945 | + TXCH_MASK | MSER_MASK | RXCR_MASK | | |
| 946 | + RXFR_MASK | RXFL_MASK); | |
| 947 | + | |
| 937 | 948 | ndev->irq = irq; |
| 938 | 949 | dev_info(dev, "IRQ is %d\n", ndev->irq); |
| 939 | 950 | |
| @@ -6232,6 +6232,9 @@ int bnxt_hwrm_cfa_ntuple_filter_free(struct bnxt *bp, · 6231 unmodified lines | ||
| 6232 | 6232 | int rc; |
| 6233 | 6233 | |
| 6234 | 6234 | set_bit(BNXT_FLTR_FW_DELETED, &fltr->base.state); |
| 6235 | + if (!test_bit(BNXT_STATE_OPEN, &bp->state)) | |
| 6236 | + return 0; | |
| 6237 | + | |
| 6235 | 6238 | rc = hwrm_req_init(bp, req, HWRM_CFA_NTUPLE_FILTER_FREE); |
| 6236 | 6239 | if (rc) |
| 6237 | 6240 | return rc; |
| @@ -10879,12 +10882,10 @@ void bnxt_del_one_rss_ctx(struct bnxt *bp, struct bnxt_rss_ctx *rss_ctx, · 4641 unmodified lines | ||
| 10879 | 10882 | struct bnxt_ntuple_filter *ntp_fltr; |
| 10880 | 10883 | int i; |
| 10881 | 10884 | |
| 10882 | − if (netif_running(bp->dev)) { | |
| 10883 | − bnxt_hwrm_vnic_free_one(bp, &rss_ctx->vnic); | |
| 10884 | − for (i = 0; i < BNXT_MAX_CTX_PER_VNIC; i++) { | |
| 10885 | − if (vnic->fw_rss_cos_lb_ctx[i] != INVALID_HW_RING_ID) | |
| 10886 | − bnxt_hwrm_vnic_ctx_free_one(bp, vnic, i); | |
| 10887 | − } | |
| 10885 | + bnxt_hwrm_vnic_free_one(bp, &rss_ctx->vnic); | |
| 10886 | + for (i = 0; i < BNXT_MAX_CTX_PER_VNIC; i++) { | |
| 10887 | + if (vnic->fw_rss_cos_lb_ctx[i] != INVALID_HW_RING_ID) | |
| 10888 | + bnxt_hwrm_vnic_ctx_free_one(bp, vnic, i); | |
| 10888 | 10889 | } |
| 10889 | 10890 | if (!all) |
| 10890 | 10891 | return; |
| @@ -979,8 +979,8 @@ static int bnxt_set_channels(struct net_device *dev, · 978 unmodified lines | ||
| 979 | 979 | |
| 980 | 980 | if (bnxt_get_nr_rss_ctxs(bp, req_rx_rings) != |
| 981 | 981 | bnxt_get_nr_rss_ctxs(bp, bp->rx_nr_rings) && |
| 982 | − netif_is_rxfh_configured(dev)) { | |
| 983 | − netdev_warn(dev, "RSS table size change required, RSS table entries must be default to proceed\n"); | |
| 982 | + (netif_is_rxfh_configured(dev) || bp->num_rss_ctx)) { | |
| 983 | + netdev_warn(dev, "RSS table size change required, RSS table entries must be default (with no additional RSS contexts present) to proceed\n"); | |
| 984 | 984 | return -EINVAL; |
| 985 | 985 | } |
| 986 | 986 | |
| @@ -1342,8 +1342,7 @@ static void bcmgenet_get_ethtool_stats(struct net_device *dev, · 1341 unmodified lines | ||
| 1342 | 1342 | } |
| 1343 | 1343 | } |
| 1344 | 1344 | |
| 1345 | −void bcmgenet_eee_enable_set(struct net_device *dev, bool enable, | |
| 1346 | − bool tx_lpi_enabled) | |
| 1345 | +void bcmgenet_eee_enable_set(struct net_device *dev, bool enable) | |
| 1347 | 1346 | { |
| 1348 | 1347 | struct bcmgenet_priv *priv = netdev_priv(dev); |
| 1349 | 1348 | u32 off = priv->hw_params->tbuf_offset + TBUF_ENERGY_CTRL; |
| @@ -1363,7 +1362,7 @@ void bcmgenet_eee_enable_set(struct net_device *dev, bool enable, · 13 unmodified lines | ||
| 1363 | 1362 | |
| 1364 | 1363 | /* Enable EEE and switch to a 27Mhz clock automatically */ |
| 1365 | 1364 | reg = bcmgenet_readl(priv->base + off); |
| 1366 | − if (tx_lpi_enabled) | |
| 1365 | + if (enable) | |
| 1367 | 1366 | reg |= TBUF_EEE_EN | TBUF_PM_EN; |
| 1368 | 1367 | else |
| 1369 | 1368 | reg &= ~(TBUF_EEE_EN | TBUF_PM_EN); |
| @@ -1382,14 +1381,12 @@ void bcmgenet_eee_enable_set(struct net_device *dev, bool enable, · 12 unmodified lines | ||
| 1382 | 1381 | priv->clk_eee_enabled = false; |
| 1383 | 1382 | } |
| 1384 | 1383 | |
| 1385 | − priv->eee.eee_enabled = enable; | |
| 1386 | − priv->eee.tx_lpi_enabled = tx_lpi_enabled; | |
| 1387 | 1384 | } |
| 1388 | 1385 | |
| 1389 | 1386 | static int bcmgenet_get_eee(struct net_device *dev, struct ethtool_keee *e) |
| 1390 | 1387 | { |
| 1391 | 1388 | struct bcmgenet_priv *priv = netdev_priv(dev); |
| 1392 | − struct ethtool_keee *p = &priv->eee; | |
| 1389 | + int ret; | |
| 1393 | 1390 | |
| 1394 | 1391 | if (GENET_IS_V1(priv)) |
| 1395 | 1392 | return -EOPNOTSUPP; |
| @@ -1397,17 +1394,21 @@ static int bcmgenet_get_eee(struct net_device *dev, struct ethtool_keee *e) · 1 unmodified line | ||
| 1397 | 1394 | if (!dev->phydev) |
| 1398 | 1395 | return -ENODEV; |
| 1399 | 1396 | |
| 1400 | − e->tx_lpi_enabled = p->tx_lpi_enabled; | |
| 1397 | + ret = phy_ethtool_get_eee(dev->phydev, e); | |
| 1398 | + if (ret) | |
| 1399 | + return ret; | |
| 1400 | + | |
| 1401 | + /* tx_lpi_timer is maintained by the MAC hardware register; the | |
| 1402 | + * PHY-level eee_cfg timer is not set for GENET. | |
| 1403 | + */ | |
| 1401 | 1404 | e->tx_lpi_timer = bcmgenet_umac_readl(priv, UMAC_EEE_LPI_TIMER); |
| 1402 | 1405 | |
| 1403 | − return phy_ethtool_get_eee(dev->phydev, e); | |
| 1406 | + return 0; | |
| 1404 | 1407 | } |
| 1405 | 1408 | |
| 1406 | 1409 | static int bcmgenet_set_eee(struct net_device *dev, struct ethtool_keee *e) |
| 1407 | 1410 | { |
| 1408 | 1411 | struct bcmgenet_priv *priv = netdev_priv(dev); |
| 1409 | − struct ethtool_keee *p = &priv->eee; | |
| 1410 | − bool active; | |
| 1411 | 1412 | |
| 1412 | 1413 | if (GENET_IS_V1(priv)) |
| 1413 | 1414 | return -EOPNOTSUPP; |
| @@ -1415,15 +1416,7 @@ static int bcmgenet_set_eee(struct net_device *dev, struct ethtool_keee *e) · 1 unmodified line | ||
| 1415 | 1416 | if (!dev->phydev) |
| 1416 | 1417 | return -ENODEV; |
| 1417 | 1418 | |
| 1418 | − p->eee_enabled = e->eee_enabled; | |
| 1419 | − | |
| 1420 | − if (!p->eee_enabled) { | |
| 1421 | − bcmgenet_eee_enable_set(dev, false, false); | |
| 1422 | − } else { | |
| 1423 | − active = phy_init_eee(dev->phydev, false) >= 0; | |
| 1424 | − bcmgenet_umac_writel(priv, e->tx_lpi_timer, UMAC_EEE_LPI_TIMER); | |
| 1425 | − bcmgenet_eee_enable_set(dev, active, e->tx_lpi_enabled); | |
| 1426 | − } | |
| 1419 | + bcmgenet_umac_writel(priv, e->tx_lpi_timer, UMAC_EEE_LPI_TIMER); | |
| 1427 | 1420 | |
| 1428 | 1421 | return phy_ethtool_set_eee(dev->phydev, e); |
| 1429 | 1422 | } |
| @@ -665,8 +665,6 @@ struct bcmgenet_priv { · 664 unmodified lines | ||
| 665 | 665 | u8 sopass[SOPASS_MAX]; |
| 666 | 666 | |
| 667 | 667 | struct bcmgenet_mib_counters mib; |
| 668 | − | |
| 669 | − struct ethtool_keee eee; | |
| 670 | 668 | }; |
| 671 | 669 | |
| 672 | 670 | static inline bool bcmgenet_has_40bits(struct bcmgenet_priv *priv) |
| @@ -749,7 +747,6 @@ int bcmgenet_wol_power_down_cfg(struct bcmgenet_priv *priv, · 76 unmodified lines | ||
| 749 | 747 | int bcmgenet_wol_power_up_cfg(struct bcmgenet_priv *priv, |
| 750 | 748 | enum bcmgenet_power_mode mode); |
| 751 | 749 | |
| 752 | −void bcmgenet_eee_enable_set(struct net_device *dev, bool enable, | |
| 753 | − bool tx_lpi_enabled); | |
| 750 | +void bcmgenet_eee_enable_set(struct net_device *dev, bool enable); | |
| 754 | 751 | |
| 755 | 752 | #endif /* __BCMGENET_H__ */ |
| @@ -29,7 +29,6 @@ static void bcmgenet_mac_config(struct net_device *dev) · 28 unmodified lines | ||
| 29 | 29 | struct bcmgenet_priv *priv = netdev_priv(dev); |
| 30 | 30 | struct phy_device *phydev = dev->phydev; |
| 31 | 31 | u32 reg, cmd_bits = 0; |
| 32 | − bool active; | |
| 33 | 32 | |
| 34 | 33 | /* speed */ |
| 35 | 34 | if (phydev->speed == SPEED_1000) |
| @@ -90,10 +89,6 @@ static void bcmgenet_mac_config(struct net_device *dev) · 54 unmodified lines | ||
| 90 | 89 | bcmgenet_umac_writel(priv, reg, UMAC_CMD); |
| 91 | 90 | spin_unlock_bh(&priv->reg_lock); |
| 92 | 91 | |
| 93 | − active = phy_init_eee(phydev, 0) >= 0; | |
| 94 | − bcmgenet_eee_enable_set(dev, | |
| 95 | − priv->eee.eee_enabled && active, | |
| 96 | − priv->eee.tx_lpi_enabled); | |
| 97 | 92 | } |
| 98 | 93 | |
| 99 | 94 | /* setup netdev link state when PHY link status change and |
| @@ -113,6 +108,8 @@ void bcmgenet_mii_setup(struct net_device *dev) · 13 unmodified lines | ||
| 113 | 108 | bcmgenet_ext_writel(priv, reg, EXT_RGMII_OOB_CTRL); |
| 114 | 109 | } |
| 115 | 110 | |
| 111 | + bcmgenet_eee_enable_set(dev, phydev->enable_tx_lpi); | |
| 112 | + | |
| 116 | 113 | phy_print_status(phydev); |
| 117 | 114 | } |
| 118 | 115 | |
| @@ -412,6 +409,9 @@ int bcmgenet_mii_probe(struct net_device *dev) · 293 unmodified lines | ||
| 412 | 409 | /* Indicate that the MAC is responsible for PHY PM */ |
| 413 | 410 | dev->phydev->mac_managed_pm = true; |
| 414 | 411 | |
| 412 | + if (!GENET_IS_V1(priv)) | |
| 413 | + phy_support_eee(dev->phydev); | |
| 414 | + | |
| 415 | 415 | return 0; |
| 416 | 416 | } |
| 417 | 417 | |
| @@ -36,6 +36,7 @@ · 35 unmodified lines | ||
| 36 | 36 | #include <linux/tcp.h> |
| 37 | 37 | #include <linux/types.h> |
| 38 | 38 | #include <linux/udp.h> |
| 39 | +#include <linux/gcd.h> | |
| 39 | 40 | #include <net/pkt_sched.h> |
| 40 | 41 | #include "macb.h" |
| 41 | 42 | |
| @@ -668,6 +669,97 @@ static void macb_mac_link_down(struct phylink_config *config, unsigned int mode, · 626 unmodified lines | ||
| 668 | 669 | netif_tx_stop_all_queues(ndev); |
| 669 | 670 | } |
| 670 | 671 | |
| 672 | +/* Use juggling algorithm to left rotate tx ring and tx skb array */ | |
| 673 | +static void gem_shuffle_tx_one_ring(struct macb_queue *queue) | |
| 674 | +{ | |
| 675 | + unsigned int head, tail, count, ring_size, desc_size; | |
| 676 | + struct macb_tx_skb tx_skb, *skb_curr, *skb_next; | |
| 677 | + struct macb_dma_desc *desc_curr, *desc_next; | |
| 678 | + unsigned int i, cycles, shift, curr, next; | |
| 679 | + struct macb *bp = queue->bp; | |
| 680 | + unsigned char desc[24]; | |
| 681 | + unsigned long flags; | |
| 682 | + | |
| 683 | + desc_size = macb_dma_desc_get_size(bp); | |
| 684 | + | |
| 685 | + if (WARN_ON_ONCE(desc_size > ARRAY_SIZE(desc))) | |
| 686 | + return; | |
| 687 | + | |
| 688 | + spin_lock_irqsave(&queue->tx_ptr_lock, flags); | |
| 689 | + head = queue->tx_head; | |
| 690 | + tail = queue->tx_tail; | |
| 691 | + ring_size = bp->tx_ring_size; | |
| 692 | + count = CIRC_CNT(head, tail, ring_size); | |
| 693 | + | |
| 694 | + if (!(tail % ring_size)) | |
| 695 | + goto unlock; | |
| 696 | + | |
| 697 | + if (!count) { | |
| 698 | + queue->tx_head = 0; | |
| 699 | + queue->tx_tail = 0; | |
| 700 | + goto unlock; | |
| 701 | + } | |
| 702 | + | |
| 703 | + shift = tail % ring_size; | |
| 704 | + cycles = gcd(ring_size, shift); | |
| 705 | + | |
| 706 | + for (i = 0; i < cycles; i++) { | |
| 707 | + memcpy(&desc, macb_tx_desc(queue, i), desc_size); | |
| 708 | + memcpy(&tx_skb, macb_tx_skb(queue, i), | |
| 709 | + sizeof(struct macb_tx_skb)); | |
| 710 | + | |
| 711 | + curr = i; | |
| 712 | + next = (curr + shift) % ring_size; | |
| 713 | + | |
| 714 | + while (next != i) { | |
| 715 | + desc_curr = macb_tx_desc(queue, curr); | |
| 716 | + desc_next = macb_tx_desc(queue, next); | |
| 717 | + | |
| 718 | + memcpy(desc_curr, desc_next, desc_size); | |
| 719 | + | |
| 720 | + if (next == ring_size - 1) | |
| 721 | + desc_curr->ctrl &= ~MACB_BIT(TX_WRAP); | |
| 722 | + if (curr == ring_size - 1) | |
| 723 | + desc_curr->ctrl |= MACB_BIT(TX_WRAP); | |
| 724 | + | |
| 725 | + skb_curr = macb_tx_skb(queue, curr); | |
| 726 | + skb_next = macb_tx_skb(queue, next); | |
| 727 | + memcpy(skb_curr, skb_next, sizeof(struct macb_tx_skb)); | |
| 728 | + | |
| 729 | + curr = next; | |
| 730 | + next = (curr + shift) % ring_size; | |
| 731 | + } | |
| 732 | + | |
| 733 | + desc_curr = macb_tx_desc(queue, curr); | |
| 734 | + memcpy(desc_curr, &desc, desc_size); | |
| 735 | + if (i == ring_size - 1) | |
| 736 | + desc_curr->ctrl &= ~MACB_BIT(TX_WRAP); | |
| 737 | + if (curr == ring_size - 1) | |
| 738 | + desc_curr->ctrl |= MACB_BIT(TX_WRAP); | |
| 739 | + memcpy(macb_tx_skb(queue, curr), &tx_skb, | |
| 740 | + sizeof(struct macb_tx_skb)); | |
| 741 | + } | |
| 742 | + | |
| 743 | + queue->tx_head = count; | |
| 744 | + queue->tx_tail = 0; | |
| 745 | + | |
| 746 | + /* Make descriptor updates visible to hardware */ | |
| 747 | + wmb(); | |
| 748 | + | |
| 749 | +unlock: | |
| 750 | + spin_unlock_irqrestore(&queue->tx_ptr_lock, flags); | |
| 751 | +} | |
| 752 | + | |
| 753 | +/* Rotate the queue so that the tail is at index 0 */ | |
| 754 | +static void gem_shuffle_tx_rings(struct macb *bp) | |
| 755 | +{ | |
| 756 | + struct macb_queue *queue; | |
| 757 | + int q; | |
| 758 | + | |
| 759 | + for (q = 0, queue = bp->queues; q < bp->num_queues; q++, queue++) | |
| 760 | + gem_shuffle_tx_one_ring(queue); | |
| 761 | +} | |
| 762 | + | |
| 671 | 763 | static void macb_mac_link_up(struct phylink_config *config, |
| 672 | 764 | struct phy_device *phy, |
| 673 | 765 | unsigned int mode, phy_interface_t interface, |
| @@ -706,8 +798,6 @@ static void macb_mac_link_up(struct phylink_config *config, · 32 unmodified lines | ||
| 706 | 798 | ctrl |= MACB_BIT(PAE); |
| 707 | 799 | |
| 708 | 800 | for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { |
| 709 | − queue->tx_head = 0; | |
| 710 | − queue->tx_tail = 0; | |
| 711 | 801 | queue_writel(queue, IER, |
| 712 | 802 | bp->rx_intr_mask | MACB_TX_INT_FLAGS | MACB_BIT(HRESP)); |
| 713 | 803 | } |
| @@ -721,8 +811,10 @@ static void macb_mac_link_up(struct phylink_config *config, · 7 unmodified lines | ||
| 721 | 811 | |
| 722 | 812 | spin_unlock_irqrestore(&bp->lock, flags); |
| 723 | 813 | |
| 724 | − if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC)) | |
| 814 | + if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC)) { | |
| 725 | 815 | macb_set_tx_clk(bp, speed); |
| 816 | + gem_shuffle_tx_rings(bp); | |
| 817 | + } | |
| 726 | 818 | |
| 727 | 819 | /* Enable Rx and Tx; Enable PTP unicast */ |
| 728 | 820 | ctrl = macb_readl(bp, NCR); |
| @@ -1533,7 +1533,7 @@ static irqreturn_t dpaa2_switch_irq0_handler_thread(int irq_num, void *arg) · 1532 unmodified lines | ||
| 1533 | 1533 | if_id = (status & 0xFFFF0000) >> 16; |
| 1534 | 1534 | if (if_id >= ethsw->sw_attr.num_ifs) { |
| 1535 | 1535 | dev_err(dev, "Invalid if_id %d in IRQ status\n", if_id); |
| 1536 | − goto out; | |
| 1536 | + goto out_clear; | |
| 1537 | 1537 | } |
| 1538 | 1538 | port_priv = ethsw->ports[if_id]; |
| 1539 | 1539 | |
| @@ -1553,6 +1553,7 @@ static irqreturn_t dpaa2_switch_irq0_handler_thread(int irq_num, void *arg) · 13 unmodified lines | ||
| 1553 | 1553 | dpaa2_switch_port_connect_mac(port_priv); |
| 1554 | 1554 | } |
| 1555 | 1555 | |
| 1556 | +out_clear: | |
| 1556 | 1557 | err = dpsw_clear_irq_status(ethsw->mc_io, 0, ethsw->dpsw_handle, |
| 1557 | 1558 | DPSW_IRQ_INDEX_IF, status); |
| 1558 | 1559 | if (err) |
| @@ -3034,6 +3035,13 @@ static int dpaa2_switch_init(struct fsl_mc_device *sw_dev) · 1475 unmodified lines | ||
| 3034 | 3035 | goto err_close; |
| 3035 | 3036 | } |
| 3036 | 3037 | |
| 3038 | + if (ethsw->sw_attr.num_ifs >= DPSW_MAX_IF) { | |
| 3039 | + dev_err(dev, "DPSW num_ifs %u exceeds max %u\n", | |
| 3040 | + ethsw->sw_attr.num_ifs, DPSW_MAX_IF); | |
| 3041 | + err = -EINVAL; | |
| 3042 | + goto err_close; | |
| 3043 | + } | |
| 3044 | + | |
| 3037 | 3045 | err = dpsw_get_api_version(ethsw->mc_io, 0, |
| 3038 | 3046 | ðsw->major, |
| 3039 | 3047 | ðsw->minor); |
| @@ -3467,7 +3467,7 @@ static int enetc_int_vector_init(struct enetc_ndev_priv *priv, int i, · 3466 unmodified lines | ||
| 3467 | 3467 | priv->rx_ring[i] = bdr; |
| 3468 | 3468 | |
| 3469 | 3469 | err = __xdp_rxq_info_reg(&bdr->xdp.rxq, priv->ndev, i, 0, |
| 3470 | − ENETC_RXB_DMA_SIZE_XDP); | |
| 3470 | + ENETC_RXB_TRUESIZE); | |
| 3471 | 3471 | if (err) |
| 3472 | 3472 | goto free_vector; |
| 3473 | 3473 | |
| @@ -333,11 +333,13 @@ static int netc_get_phy_addr(struct device_node *np) · 332 unmodified lines | ||
| 333 | 333 | |
| 334 | 334 | mdio_node = of_get_child_by_name(np, "mdio"); |
| 335 | 335 | if (!mdio_node) |
| 336 | − return 0; | |
| 336 | + return -ENODEV; | |
| 337 | 337 | |
| 338 | 338 | phy_node = of_get_next_child(mdio_node, NULL); |
| 339 | − if (!phy_node) | |
| 339 | + if (!phy_node) { | |
| 340 | + err = -ENODEV; | |
| 340 | 341 | goto of_put_mdio_node; |
| 342 | + } | |
| 341 | 343 | |
| 342 | 344 | err = of_property_read_u32(phy_node, "reg", &addr); |
| 343 | 345 | if (err) |
| @@ -423,6 +425,9 @@ static int imx95_enetc_mdio_phyaddr_config(struct platform_device *pdev) · 79 unmodified lines | ||
| 423 | 425 | |
| 424 | 426 | addr = netc_get_phy_addr(gchild); |
| 425 | 427 | if (addr < 0) { |
| 428 | + if (addr == -ENODEV) | |
| 429 | + continue; | |
| 430 | + | |
| 426 | 431 | dev_err(dev, "Failed to get PHY address\n"); |
| 427 | 432 | return addr; |
| 428 | 433 | } |
| @@ -433,12 +438,6 @@ static int imx95_enetc_mdio_phyaddr_config(struct platform_device *pdev) · 4 unmodified lines | ||
| 433 | 438 | return -EINVAL; |
| 434 | 439 | } |
| 435 | 440 | |
| 436 | − /* The default value of LaBCR[MDIO_PHYAD_PRTAD ] is | |
| 437 | − * 0, so no need to set the register. | |
| 438 | − */ | |
| 439 | − if (!addr) | |
| 440 | − continue; | |
| 441 | − | |
| 442 | 441 | switch (bus_devfn) { |
| 443 | 442 | case IMX95_ENETC0_BUS_DEVFN: |
| 444 | 443 | netc_reg_write(priv->ierb, IERB_LBCR(0), |
| @@ -578,16 +577,13 @@ static int imx94_enetc_mdio_phyaddr_config(struct netc_blk_ctrl *priv, · 133 unmodified lines | ||
| 578 | 577 | |
| 579 | 578 | addr = netc_get_phy_addr(np); |
| 580 | 579 | if (addr < 0) { |
| 580 | + if (addr == -ENODEV) | |
| 581 | + return 0; | |
| 582 | + | |
| 581 | 583 | dev_err(dev, "Failed to get PHY address\n"); |
| 582 | 584 | return addr; |
| 583 | 585 | } |
| 584 | 586 | |
| 585 | − /* The default value of LaBCR[MDIO_PHYAD_PRTAD] is 0, | |
| 586 | − * so no need to set the register. | |
| 587 | − */ | |
| 588 | − if (!addr) | |
| 589 | − return 0; | |
| 590 | − | |
| 591 | 587 | if (phy_mask & BIT(addr)) { |
| 592 | 588 | dev_err(dev, |
| 593 | 589 | "Find same PHY address in EMDIO and ENETC node\n"); |
| @@ -167,6 +167,25 @@ gve_free_pending_packet(struct gve_tx_ring *tx, · 166 unmodified lines | ||
| 167 | 167 | } |
| 168 | 168 | } |
| 169 | 169 | |
| 170 | +static void gve_unmap_packet(struct device *dev, | |
| 171 | + struct gve_tx_pending_packet_dqo *pkt) | |
| 172 | +{ | |
| 173 | + int i; | |
| 174 | + | |
| 175 | + if (!pkt->num_bufs) | |
| 176 | + return; | |
| 177 | + | |
| 178 | + /* SKB linear portion is guaranteed to be mapped */ | |
| 179 | + dma_unmap_single(dev, dma_unmap_addr(pkt, dma[0]), | |
| 180 | + dma_unmap_len(pkt, len[0]), DMA_TO_DEVICE); | |
| 181 | + for (i = 1; i < pkt->num_bufs; i++) { | |
| 182 | + netmem_dma_unmap_page_attrs(dev, dma_unmap_addr(pkt, dma[i]), | |
| 183 | + dma_unmap_len(pkt, len[i]), | |
| 184 | + DMA_TO_DEVICE, 0); | |
| 185 | + } | |
| 186 | + pkt->num_bufs = 0; | |
| 187 | +} | |
| 188 | + | |
| 170 | 189 | /* gve_tx_free_desc - Cleans up all pending tx requests and buffers. |
| 171 | 190 | */ |
| 172 | 191 | static void gve_tx_clean_pending_packets(struct gve_tx_ring *tx) |
| @@ -176,21 +195,12 @@ static void gve_tx_clean_pending_packets(struct gve_tx_ring *tx) · 3 unmodified lines | ||
| 176 | 195 | for (i = 0; i < tx->dqo.num_pending_packets; i++) { |
| 177 | 196 | struct gve_tx_pending_packet_dqo *cur_state = |
| 178 | 197 | &tx->dqo.pending_packets[i]; |
| 179 | − int j; | |
| 180 | − | |
| 181 | − for (j = 0; j < cur_state->num_bufs; j++) { | |
| 182 | − if (j == 0) { | |
| 183 | − dma_unmap_single(tx->dev, | |
| 184 | − dma_unmap_addr(cur_state, dma[j]), | |
| 185 | − dma_unmap_len(cur_state, len[j]), | |
| 186 | − DMA_TO_DEVICE); | |
| 187 | − } else { | |
| 188 | − dma_unmap_page(tx->dev, | |
| 189 | − dma_unmap_addr(cur_state, dma[j]), | |
| 190 | − dma_unmap_len(cur_state, len[j]), | |
| 191 | − DMA_TO_DEVICE); | |
| 192 | − } | |
| 193 | − } | |
| 198 | + | |
| 199 | + if (tx->dqo.qpl) | |
| 200 | + gve_free_tx_qpl_bufs(tx, cur_state); | |
| 201 | + else | |
| 202 | + gve_unmap_packet(tx->dev, cur_state); | |
| 203 | + | |
| 194 | 204 | if (cur_state->skb) { |
| 195 | 205 | dev_consume_skb_any(cur_state->skb); |
| 196 | 206 | cur_state->skb = NULL; |
| @@ -1154,22 +1164,6 @@ static void remove_from_list(struct gve_tx_ring *tx, · 957 unmodified lines | ||
| 1154 | 1164 | } |
| 1155 | 1165 | } |
| 1156 | 1166 | |
| 1157 | −static void gve_unmap_packet(struct device *dev, | |
| 1158 | − struct gve_tx_pending_packet_dqo *pkt) | |
| 1159 | −{ | |
| 1160 | − int i; | |
| 1161 | − | |
| 1162 | − /* SKB linear portion is guaranteed to be mapped */ | |
| 1163 | − dma_unmap_single(dev, dma_unmap_addr(pkt, dma[0]), | |
| 1164 | − dma_unmap_len(pkt, len[0]), DMA_TO_DEVICE); | |
| 1165 | − for (i = 1; i < pkt->num_bufs; i++) { | |
| 1166 | − netmem_dma_unmap_page_attrs(dev, dma_unmap_addr(pkt, dma[i]), | |
| 1167 | − dma_unmap_len(pkt, len[i]), | |
| 1168 | − DMA_TO_DEVICE, 0); | |
| 1169 | − } | |
| 1170 | − pkt->num_bufs = 0; | |
| 1171 | −} | |
| 1172 | − | |
| 1173 | 1167 | /* Completion types and expected behavior: |
| 1174 | 1168 | * No Miss compl + Packet compl = Packet completed normally. |
| 1175 | 1169 | * Miss compl + Re-inject compl = Packet completed normally. |
| @@ -2952,8 +2952,6 @@ static int e1000_tx_map(struct e1000_adapter *adapter, · 2951 unmodified lines | ||
| 2952 | 2952 | dma_error: |
| 2953 | 2953 | dev_err(&pdev->dev, "TX DMA map failed\n"); |
| 2954 | 2954 | buffer_info->dma = 0; |
| 2955 | − if (count) | |
| 2956 | − count--; | |
| 2957 | 2955 | |
| 2958 | 2956 | while (count--) { |
| 2959 | 2957 | if (i == 0) |
| @@ -33,6 +33,7 @@ · 32 unmodified lines | ||
| 33 | 33 | |
| 34 | 34 | /* Extended Device Control */ |
| 35 | 35 | #define E1000_CTRL_EXT_LPCD 0x00000004 /* LCD Power Cycle Done */ |
| 36 | +#define E1000_CTRL_EXT_DPG_EN 0x00000008 /* Dynamic Power Gating Enable */ | |
| 36 | 37 | #define E1000_CTRL_EXT_SDP3_DATA 0x00000080 /* Value of SW Definable Pin 3 */ |
| 37 | 38 | #define E1000_CTRL_EXT_FORCE_SMBUS 0x00000800 /* Force SMBus mode */ |
| 38 | 39 | #define E1000_CTRL_EXT_EE_RST 0x00002000 /* Reinitialize from EEPROM */ |
| @@ -117,7 +117,8 @@ enum e1000_boards { · 116 unmodified lines | ||
| 117 | 117 | board_pch_cnp, |
| 118 | 118 | board_pch_tgp, |
| 119 | 119 | board_pch_adp, |
| 120 | − board_pch_mtp | |
| 120 | + board_pch_mtp, | |
| 121 | + board_pch_ptp | |
| 121 | 122 | }; |
| 122 | 123 | |
| 123 | 124 | struct e1000_ps_page { |
| @@ -527,6 +528,7 @@ extern const struct e1000_info e1000_pch_cnp_info; · 403 unmodified lines | ||
| 527 | 528 | extern const struct e1000_info e1000_pch_tgp_info; |
| 528 | 529 | extern const struct e1000_info e1000_pch_adp_info; |
| 529 | 530 | extern const struct e1000_info e1000_pch_mtp_info; |
| 531 | +extern const struct e1000_info e1000_pch_ptp_info; | |
| 530 | 532 | extern const struct e1000_info e1000_es2_info; |
| 531 | 533 | |
| 532 | 534 | void e1000e_ptp_init(struct e1000_adapter *adapter); |
| @@ -118,8 +118,6 @@ struct e1000_hw; · 117 unmodified lines | ||
| 118 | 118 | #define E1000_DEV_ID_PCH_ARL_I219_V24 0x57A1 |
| 119 | 119 | #define E1000_DEV_ID_PCH_PTP_I219_LM25 0x57B3 |
| 120 | 120 | #define E1000_DEV_ID_PCH_PTP_I219_V25 0x57B4 |
| 121 | −#define E1000_DEV_ID_PCH_PTP_I219_LM26 0x57B5 | |
| 122 | −#define E1000_DEV_ID_PCH_PTP_I219_V26 0x57B6 | |
| 123 | 121 | #define E1000_DEV_ID_PCH_PTP_I219_LM27 0x57B7 |
| 124 | 122 | #define E1000_DEV_ID_PCH_PTP_I219_V27 0x57B8 |
| 125 | 123 | #define E1000_DEV_ID_PCH_NVL_I219_LM29 0x57B9 |
| @@ -528,7 +528,7 @@ static s32 e1000_init_phy_params_pchlan(struct e1000_hw *hw) · 527 unmodified lines | ||
| 528 | 528 | |
| 529 | 529 | phy->id = e1000_phy_unknown; |
| 530 | 530 | |
| 531 | − if (hw->mac.type == e1000_pch_mtp) { | |
| 531 | + if (hw->mac.type == e1000_pch_mtp || hw->mac.type == e1000_pch_ptp) { | |
| 532 | 532 | phy->retry_count = 2; |
| 533 | 533 | e1000e_enable_phy_retry(hw); |
| 534 | 534 | } |
| @@ -4932,6 +4932,15 @@ static s32 e1000_reset_hw_ich8lan(struct e1000_hw *hw) · 4397 unmodified lines | ||
| 4932 | 4932 | reg |= E1000_KABGTXD_BGSQLBIAS; |
| 4933 | 4933 | ew32(KABGTXD, reg); |
| 4934 | 4934 | |
| 4935 | + /* The hardware reset value of the DPG_EN bit is 1. | |
| 4936 | + * Clear DPG_EN to prevent unexpected autonomous power gating. | |
| 4937 | + */ | |
| 4938 | + if (hw->mac.type >= e1000_pch_ptp) { | |
| 4939 | + reg = er32(CTRL_EXT); | |
| 4940 | + reg &= ~E1000_CTRL_EXT_DPG_EN; | |
| 4941 | + ew32(CTRL_EXT, reg); | |
| 4942 | + } | |
| 4943 | + | |
| 4935 | 4944 | return 0; |
| 4936 | 4945 | } |
| 4937 | 4946 | |
| @@ -6208,3 +6217,23 @@ const struct e1000_info e1000_pch_mtp_info = { · 1270 unmodified lines | ||
| 6208 | 6217 | .phy_ops = &ich8_phy_ops, |
| 6209 | 6218 | .nvm_ops = &spt_nvm_ops, |
| 6210 | 6219 | }; |
| 6220 | + | |
| 6221 | +const struct e1000_info e1000_pch_ptp_info = { | |
| 6222 | + .mac = e1000_pch_ptp, | |
| 6223 | + .flags = FLAG_IS_ICH | |
| 6224 | + | FLAG_HAS_WOL | |
| 6225 | + | FLAG_HAS_HW_TIMESTAMP | |
| 6226 | + | FLAG_HAS_CTRLEXT_ON_LOAD | |
| 6227 | + | FLAG_HAS_AMT | |
| 6228 | + | FLAG_HAS_FLASH | |
| 6229 | + | FLAG_HAS_JUMBO_FRAMES | |
| 6230 | + | FLAG_APME_IN_WUC, | |
| 6231 | + .flags2 = FLAG2_HAS_PHY_STATS | |
| 6232 | + | FLAG2_HAS_EEE, | |
| 6233 | + .pba = 26, | |
| 6234 | + .max_hw_frame_size = 9022, | |
| 6235 | + .get_variants = e1000_get_variants_ich8lan, | |
| 6236 | + .mac_ops = &ich8_mac_ops, | |
| 6237 | + .phy_ops = &ich8_phy_ops, | |
| 6238 | + .nvm_ops = &spt_nvm_ops, | |
| 6239 | +}; | |
| @@ -55,6 +55,7 @@ static const struct e1000_info *e1000_info_tbl[] = { · 54 unmodified lines | ||
| 55 | 55 | [board_pch_tgp] = &e1000_pch_tgp_info, |
| 56 | 56 | [board_pch_adp] = &e1000_pch_adp_info, |
| 57 | 57 | [board_pch_mtp] = &e1000_pch_mtp_info, |
| 58 | + [board_pch_ptp] = &e1000_pch_ptp_info, | |
| 58 | 59 | }; |
| 59 | 60 | |
| 60 | 61 | struct e1000_reg_info { |
| @@ -5651,8 +5652,6 @@ static int e1000_tx_map(struct e1000_ring *tx_ring, struct sk_buff *skb, · 5590 unmodified lines | ||
| 5651 | 5652 | dma_error: |
| 5652 | 5653 | dev_err(&pdev->dev, "Tx DMA map failed\n"); |
| 5653 | 5654 | buffer_info->dma = 0; |
| 5654 | − if (count) | |
| 5655 | − count--; | |
| 5656 | 5655 | |
| 5657 | 5656 | while (count--) { |
| 5658 | 5657 | if (i == 0) |
| @@ -7922,14 +7921,12 @@ static const struct pci_device_id e1000_pci_tbl[] = { · 2263 unmodified lines | ||
| 7922 | 7921 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_LNP_I219_V21), board_pch_mtp }, |
| 7923 | 7922 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_ARL_I219_LM24), board_pch_mtp }, |
| 7924 | 7923 | { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_ARL_I219_V24), board_pch_mtp }, |
| 7925 | − { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_LM25), board_pch_mtp }, | |
| 7926 | − { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_V25), board_pch_mtp }, | |
| 7927 | − { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_LM26), board_pch_mtp }, | |
| 7928 | − { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_V26), board_pch_mtp }, | |
| 7929 | − { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_LM27), board_pch_mtp }, | |
| 7930 | − { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_V27), board_pch_mtp }, | |
| 7931 | − { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_NVL_I219_LM29), board_pch_mtp }, | |
| 7932 | − { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_NVL_I219_V29), board_pch_mtp }, | |
| 7924 | + { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_LM25), board_pch_ptp }, | |
| 7925 | + { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_V25), board_pch_ptp }, | |
| 7926 | + { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_LM27), board_pch_ptp }, | |
| 7927 | + { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_PTP_I219_V27), board_pch_ptp }, | |
| 7928 | + { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_NVL_I219_LM29), board_pch_ptp }, | |
| 7929 | + { PCI_VDEVICE(INTEL, E1000_DEV_ID_PCH_NVL_I219_V29), board_pch_ptp }, | |
| 7933 | 7930 | |
| 7934 | 7931 | { 0, 0, 0, 0, 0, 0, 0 } /* terminate list */ |
| 7935 | 7932 | }; |
| @@ -3569,6 +3569,7 @@ static int i40e_configure_rx_ring(struct i40e_ring *ring) · 3568 unmodified lines | ||
| 3569 | 3569 | u16 pf_q = vsi->base_queue + ring->queue_index; |
| 3570 | 3570 | struct i40e_hw *hw = &vsi->back->hw; |
| 3571 | 3571 | struct i40e_hmc_obj_rxq rx_ctx; |
| 3572 | + u32 xdp_frame_sz; | |
| 3572 | 3573 | int err = 0; |
| 3573 | 3574 | bool ok; |
| 3574 | 3575 | |
| @@ -3578,49 +3579,47 @@ static int i40e_configure_rx_ring(struct i40e_ring *ring) · 3 unmodified lines | ||
| 3578 | 3579 | memset(&rx_ctx, 0, sizeof(rx_ctx)); |
| 3579 | 3580 | |
| 3580 | 3581 | ring->rx_buf_len = vsi->rx_buf_len; |
| 3582 | + xdp_frame_sz = i40e_rx_pg_size(ring) / 2; | |
| 3581 | 3583 | |
| 3582 | 3584 | /* XDP RX-queue info only needed for RX rings exposed to XDP */ |
| 3583 | 3585 | if (ring->vsi->type != I40E_VSI_MAIN) |
| 3584 | 3586 | goto skip; |
| 3585 | 3587 | |
| 3586 | − if (!xdp_rxq_info_is_reg(&ring->xdp_rxq)) { | |
| 3587 | − err = __xdp_rxq_info_reg(&ring->xdp_rxq, ring->netdev, | |
| 3588 | − ring->queue_index, | |
| 3589 | − ring->q_vector->napi.napi_id, | |
| 3590 | − ring->rx_buf_len); | |
| 3591 | − if (err) | |
| 3592 | − return err; | |
| 3593 | − } | |
| 3594 | − | |
| 3595 | 3588 | ring->xsk_pool = i40e_xsk_pool(ring); |
| 3596 | 3589 | if (ring->xsk_pool) { |
| 3597 | − xdp_rxq_info_unreg(&ring->xdp_rxq); | |
| 3590 | + xdp_frame_sz = xsk_pool_get_rx_frag_step(ring->xsk_pool); | |
| 3598 | 3591 | ring->rx_buf_len = xsk_pool_get_rx_frame_size(ring->xsk_pool); |
| 3599 | 3592 | err = __xdp_rxq_info_reg(&ring->xdp_rxq, ring->netdev, |
| 3600 | 3593 | ring->queue_index, |
| 3601 | 3594 | ring->q_vector->napi.napi_id, |
| 3602 | − ring->rx_buf_len); | |
| 3595 | + xdp_frame_sz); | |
| 3603 | 3596 | if (err) |
| 3604 | 3597 | return err; |
| 3605 | 3598 | err = xdp_rxq_info_reg_mem_model(&ring->xdp_rxq, |
| 3606 | 3599 | MEM_TYPE_XSK_BUFF_POOL, |
| 3607 | 3600 | NULL); |
| 3608 | 3601 | if (err) |
| 3609 | − return err; | |
| 3602 | + goto unreg_xdp; | |
| 3610 | 3603 | dev_info(&vsi->back->pdev->dev, |
| 3611 | 3604 | "Registered XDP mem model MEM_TYPE_XSK_BUFF_POOL on Rx ring %d\n", |
| 3612 | 3605 | ring->queue_index); |
| 3613 | 3606 | |
| 3614 | 3607 | } else { |
| 3608 | + err = __xdp_rxq_info_reg(&ring->xdp_rxq, ring->netdev, | |
| 3609 | + ring->queue_index, | |
| 3610 | + ring->q_vector->napi.napi_id, | |
| 3611 | + xdp_frame_sz); | |
| 3612 | + if (err) | |
| 3613 | + return err; | |
| 3615 | 3614 | err = xdp_rxq_info_reg_mem_model(&ring->xdp_rxq, |
| 3616 | 3615 | MEM_TYPE_PAGE_SHARED, |
| 3617 | 3616 | NULL); |
| 3618 | 3617 | if (err) |
| 3619 | − return err; | |
| 3618 | + goto unreg_xdp; | |
| 3620 | 3619 | } |
| 3621 | 3620 | |
| 3622 | 3621 | skip: |
| 3623 | − xdp_init_buff(&ring->xdp, i40e_rx_pg_size(ring) / 2, &ring->xdp_rxq); | |
| 3622 | + xdp_init_buff(&ring->xdp, xdp_frame_sz, &ring->xdp_rxq); | |
| 3624 | 3623 | |
| 3625 | 3624 | rx_ctx.dbuff = DIV_ROUND_UP(ring->rx_buf_len, |
| 3626 | 3625 | BIT_ULL(I40E_RXQ_CTX_DBUFF_SHIFT)); |
| @@ -3654,7 +3653,8 @@ skip: · 27 unmodified lines | ||
| 3654 | 3653 | dev_info(&vsi->back->pdev->dev, |
| 3655 | 3654 | "Failed to clear LAN Rx queue context on Rx ring %d (pf_q %d), error: %d\n", |
| 3656 | 3655 | ring->queue_index, pf_q, err); |
| 3657 | − return -ENOMEM; | |
| 3656 | + err = -ENOMEM; | |
| 3657 | + goto unreg_xdp; | |
| 3658 | 3658 | } |
| 3659 | 3659 | |
| 3660 | 3660 | /* set the context in the HMC */ |
| @@ -3663,7 +3663,8 @@ skip: · 2 unmodified lines | ||
| 3663 | 3663 | dev_info(&vsi->back->pdev->dev, |
| 3664 | 3664 | "Failed to set LAN Rx queue context on Rx ring %d (pf_q %d), error: %d\n", |
| 3665 | 3665 | ring->queue_index, pf_q, err); |
| 3666 | − return -ENOMEM; | |
| 3666 | + err = -ENOMEM; | |
| 3667 | + goto unreg_xdp; | |
| 3667 | 3668 | } |
| 3668 | 3669 | |
| 3669 | 3670 | /* configure Rx buffer alignment */ |
| @@ -3671,7 +3672,8 @@ skip: · 1 unmodified line | ||
| 3671 | 3672 | if (I40E_2K_TOO_SMALL_WITH_PADDING) { |
| 3672 | 3673 | dev_info(&vsi->back->pdev->dev, |
| 3673 | 3674 | "2k Rx buffer is too small to fit standard MTU and skb_shared_info\n"); |
| 3674 | − return -EOPNOTSUPP; | |
| 3675 | + err = -EOPNOTSUPP; | |
| 3676 | + goto unreg_xdp; | |
| 3675 | 3677 | } |
| 3676 | 3678 | clear_ring_build_skb_enabled(ring); |
| 3677 | 3679 | } else { |
| @@ -3701,6 +3703,11 @@ skip: · 23 unmodified lines | ||
| 3701 | 3703 | } |
| 3702 | 3704 | |
| 3703 | 3705 | return 0; |
| 3706 | +unreg_xdp: | |
| 3707 | + if (ring->vsi->type == I40E_VSI_MAIN) | |
| 3708 | + xdp_rxq_info_unreg(&ring->xdp_rxq); | |
| 3709 | + | |
| 3710 | + return err; | |
| 3704 | 3711 | } |
| 3705 | 3712 | |
| 3706 | 3713 | /** |
| @@ -88,7 +88,7 @@ TRACE_EVENT(i40e_napi_poll, · 87 unmodified lines | ||
| 88 | 88 | __entry->rx_clean_complete = rx_clean_complete; |
| 89 | 89 | __entry->tx_clean_complete = tx_clean_complete; |
| 90 | 90 | __entry->irq_num = q->irq_num; |
| 91 | − __entry->curr_cpu = get_cpu(); | |
| 91 | + __entry->curr_cpu = smp_processor_id(); | |
| 92 | 92 | __assign_str(qname); |
| 93 | 93 | __assign_str(dev_name); |
| 94 | 94 | __assign_bitmask(irq_affinity, cpumask_bits(&q->affinity_mask), |
| @@ -1470,6 +1470,9 @@ void i40e_clean_rx_ring(struct i40e_ring *rx_ring) · 1469 unmodified lines | ||
| 1470 | 1470 | if (!rx_ring->rx_bi) |
| 1471 | 1471 | return; |
| 1472 | 1472 | |
| 1473 | + if (xdp_rxq_info_is_reg(&rx_ring->xdp_rxq)) | |
| 1474 | + xdp_rxq_info_unreg(&rx_ring->xdp_rxq); | |
| 1475 | + | |
| 1473 | 1476 | if (rx_ring->xsk_pool) { |
| 1474 | 1477 | i40e_xsk_clean_rx_ring(rx_ring); |
| 1475 | 1478 | goto skip_free; |
| @@ -1527,8 +1530,6 @@ skip_free: · 51 unmodified lines | ||
| 1527 | 1530 | void i40e_free_rx_resources(struct i40e_ring *rx_ring) |
| 1528 | 1531 | { |
| 1529 | 1532 | i40e_clean_rx_ring(rx_ring); |
| 1530 | − if (rx_ring->vsi->type == I40E_VSI_MAIN) | |
| 1531 | − xdp_rxq_info_unreg(&rx_ring->xdp_rxq); | |
| 1532 | 1533 | rx_ring->xdp_prog = NULL; |
| 1533 | 1534 | kfree(rx_ring->rx_bi); |
| 1534 | 1535 | rx_ring->rx_bi = NULL; |
| @@ -3833,10 +3833,10 @@ static int i40e_vc_del_cloud_filter(struct i40e_vf *vf, u8 *msg) · 3832 unmodified lines | ||
| 3833 | 3833 | cfilter.n_proto = ETH_P_IP; |
| 3834 | 3834 | if (mask.dst_ip[0] & tcf.dst_ip[0]) |
| 3835 | 3835 | memcpy(&cfilter.ip.v4.dst_ip, tcf.dst_ip, |
| 3836 | − ARRAY_SIZE(tcf.dst_ip)); | |
| 3837 | − else if (mask.src_ip[0] & tcf.dst_ip[0]) | |
| 3836 | + sizeof(cfilter.ip.v4.dst_ip)); | |
| 3837 | + else if (mask.src_ip[0] & tcf.src_ip[0]) | |
| 3838 | 3838 | memcpy(&cfilter.ip.v4.src_ip, tcf.src_ip, |
| 3839 | − ARRAY_SIZE(tcf.dst_ip)); | |
| 3839 | + sizeof(cfilter.ip.v4.src_ip)); | |
| 3840 | 3840 | break; |
| 3841 | 3841 | case VIRTCHNL_TCP_V6_FLOW: |
| 3842 | 3842 | cfilter.n_proto = ETH_P_IPV6; |
| @@ -3891,7 +3891,7 @@ static int i40e_vc_del_cloud_filter(struct i40e_vf *vf, u8 *msg) · 48 unmodified lines | ||
| 3891 | 3891 | /* for ipv6, mask is set for all sixteen bytes (4 words) */ |
| 3892 | 3892 | if (cfilter.n_proto == ETH_P_IPV6 && mask.dst_ip[3]) |
| 3893 | 3893 | if (memcmp(&cfilter.ip.v6.dst_ip6, &cf->ip.v6.dst_ip6, |
| 3894 | − sizeof(cfilter.ip.v6.src_ip6))) | |
| 3894 | + sizeof(cfilter.ip.v6.dst_ip6))) | |
| 3895 | 3895 | continue; |
| 3896 | 3896 | if (mask.vlan_id) |
| 3897 | 3897 | if (cfilter.vlan_id != cf->vlan_id) |
| @@ -3979,10 +3979,10 @@ static int i40e_vc_add_cloud_filter(struct i40e_vf *vf, u8 *msg) · 81 unmodified lines | ||
| 3979 | 3979 | cfilter->n_proto = ETH_P_IP; |
| 3980 | 3980 | if (mask.dst_ip[0] & tcf.dst_ip[0]) |
| 3981 | 3981 | memcpy(&cfilter->ip.v4.dst_ip, tcf.dst_ip, |
| 3982 | − ARRAY_SIZE(tcf.dst_ip)); | |
| 3983 | − else if (mask.src_ip[0] & tcf.dst_ip[0]) | |
| 3982 | + sizeof(cfilter->ip.v4.dst_ip)); | |
| 3983 | + else if (mask.src_ip[0] & tcf.src_ip[0]) | |
| 3984 | 3984 | memcpy(&cfilter->ip.v4.src_ip, tcf.src_ip, |
| 3985 | − ARRAY_SIZE(tcf.dst_ip)); | |
| 3985 | + sizeof(cfilter->ip.v4.src_ip)); | |
| 3986 | 3986 | break; |
| 3987 | 3987 | case VIRTCHNL_TCP_V6_FLOW: |
| 3988 | 3988 | cfilter->n_proto = ETH_P_IPV6; |
| @@ -260,7 +260,6 @@ struct iavf_adapter { · 259 unmodified lines | ||
| 260 | 260 | struct work_struct adminq_task; |
| 261 | 261 | struct work_struct finish_config; |
| 262 | 262 | wait_queue_head_t down_waitqueue; |
| 263 | − wait_queue_head_t reset_waitqueue; | |
| 264 | 263 | wait_queue_head_t vc_waitqueue; |
| 265 | 264 | struct iavf_q_vector *q_vectors; |
| 266 | 265 | struct list_head vlan_filter_list; |
| @@ -626,5 +625,5 @@ void iavf_add_adv_rss_cfg(struct iavf_adapter *adapter); · 359 unmodified lines | ||
| 626 | 625 | void iavf_del_adv_rss_cfg(struct iavf_adapter *adapter); |
| 627 | 626 | struct iavf_mac_filter *iavf_add_filter(struct iavf_adapter *adapter, |
| 628 | 627 | const u8 *macaddr); |
| 629 | −int iavf_wait_for_reset(struct iavf_adapter *adapter); | |
| 628 | +void iavf_reset_step(struct iavf_adapter *adapter); | |
| 630 | 629 | #endif /* _IAVF_H_ */ |
| @@ -492,7 +492,6 @@ static int iavf_set_ringparam(struct net_device *netdev, · 491 unmodified lines | ||
| 492 | 492 | { |
| 493 | 493 | struct iavf_adapter *adapter = netdev_priv(netdev); |
| 494 | 494 | u32 new_rx_count, new_tx_count; |
| 495 | − int ret = 0; | |
| 496 | 495 | |
| 497 | 496 | if ((ring->rx_mini_pending) || (ring->rx_jumbo_pending)) |
| 498 | 497 | return -EINVAL; |
| @@ -537,13 +536,11 @@ static int iavf_set_ringparam(struct net_device *netdev, · 38 unmodified lines | ||
| 537 | 536 | } |
| 538 | 537 | |
| 539 | 538 | if (netif_running(netdev)) { |
| 540 | − iavf_schedule_reset(adapter, IAVF_FLAG_RESET_NEEDED); | |
| 541 | − ret = iavf_wait_for_reset(adapter); | |
| 542 | − if (ret) | |
| 543 | − netdev_warn(netdev, "Changing ring parameters timeout or interrupted waiting for reset"); | |
| 539 | + adapter->flags |= IAVF_FLAG_RESET_NEEDED; | |
| 540 | + iavf_reset_step(adapter); | |
| 544 | 541 | } |
| 545 | 542 | |
| 546 | − return ret; | |
| 543 | + return 0; | |
| 547 | 544 | } |
| 548 | 545 | |
| 549 | 546 | /** |
| @@ -1723,7 +1720,6 @@ static int iavf_set_channels(struct net_device *netdev, · 1173 unmodified lines | ||
| 1723 | 1720 | { |
| 1724 | 1721 | struct iavf_adapter *adapter = netdev_priv(netdev); |
| 1725 | 1722 | u32 num_req = ch->combined_count; |
| 1726 | − int ret = 0; | |
| 1727 | 1723 | |
| 1728 | 1724 | if ((adapter->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_ADQ) && |
| 1729 | 1725 | adapter->num_tc) { |
| @@ -1745,13 +1741,10 @@ static int iavf_set_channels(struct net_device *netdev, · 15 unmodified lines | ||
| 1745 | 1741 | |
| 1746 | 1742 | adapter->num_req_queues = num_req; |
| 1747 | 1743 | adapter->flags |= IAVF_FLAG_REINIT_ITR_NEEDED; |
| 1748 | − iavf_schedule_reset(adapter, IAVF_FLAG_RESET_NEEDED); | |
| 1744 | + adapter->flags |= IAVF_FLAG_RESET_NEEDED; | |
| 1745 | + iavf_reset_step(adapter); | |
| 1749 | 1746 | |
| 1750 | − ret = iavf_wait_for_reset(adapter); | |
| 1751 | − if (ret) | |
| 1752 | − netdev_warn(netdev, "Changing channel count timeout or interrupted waiting for reset"); | |
| 1753 | − | |
| 1754 | − return ret; | |
| 1747 | + return 0; | |
| 1755 | 1748 | } |
| 1756 | 1749 | |
| 1757 | 1750 | /** |
| @@ -185,31 +185,6 @@ static bool iavf_is_reset_in_progress(struct iavf_adapter *adapter) · 184 unmodified lines | ||
| 185 | 185 | return false; |
| 186 | 186 | } |
| 187 | 187 | |
| 188 | −/** | |
| 189 | − * iavf_wait_for_reset - Wait for reset to finish. | |
| 190 | − * @adapter: board private structure | |
| 191 | − * | |
| 192 | − * Returns 0 if reset finished successfully, negative on timeout or interrupt. | |
| 193 | − */ | |
| 194 | −int iavf_wait_for_reset(struct iavf_adapter *adapter) | |
| 195 | −{ | |
| 196 | − int ret = wait_event_interruptible_timeout(adapter->reset_waitqueue, | |
| 197 | − !iavf_is_reset_in_progress(adapter), | |
| 198 | − msecs_to_jiffies(5000)); | |
| 199 | − | |
| 200 | − /* If ret < 0 then it means wait was interrupted. | |
| 201 | − * If ret == 0 then it means we got a timeout while waiting | |
| 202 | − * for reset to finish. | |
| 203 | − * If ret > 0 it means reset has finished. | |
| 204 | − */ | |
| 205 | − if (ret > 0) | |
| 206 | − return 0; | |
| 207 | − else if (ret < 0) | |
| 208 | − return -EINTR; | |
| 209 | − else | |
| 210 | − return -EBUSY; | |
| 211 | −} | |
| 212 | − | |
| 213 | 188 | /** |
| 214 | 189 | * iavf_allocate_dma_mem_d - OS specific memory alloc for shared code |
| 215 | 190 | * @hw: pointer to the HW structure |
| @@ -2793,7 +2768,22 @@ static void iavf_init_config_adapter(struct iavf_adapter *adapter) · 2577 unmodified lines | ||
| 2793 | 2768 | netdev->watchdog_timeo = 5 * HZ; |
| 2794 | 2769 | |
| 2795 | 2770 | netdev->min_mtu = ETH_MIN_MTU; |
| 2796 | − netdev->max_mtu = LIBIE_MAX_MTU; | |
| 2771 | + | |
| 2772 | + /* PF/VF API: vf_res->max_mtu is max frame size (not MTU). | |
| 2773 | + * Convert to MTU. | |
| 2774 | + */ | |
| 2775 | + if (!adapter->vf_res->max_mtu) { | |
| 2776 | + netdev->max_mtu = LIBIE_MAX_MTU; | |
| 2777 | + } else if (adapter->vf_res->max_mtu < LIBETH_RX_LL_LEN + ETH_MIN_MTU || | |
| 2778 | + adapter->vf_res->max_mtu > | |
| 2779 | + LIBETH_RX_LL_LEN + LIBIE_MAX_MTU) { | |
| 2780 | + netdev_warn_once(adapter->netdev, | |
| 2781 | + "invalid max frame size %d from PF, using default MTU %d", | |
| 2782 | + adapter->vf_res->max_mtu, LIBIE_MAX_MTU); | |
| 2783 | + netdev->max_mtu = LIBIE_MAX_MTU; | |
| 2784 | + } else { | |
| 2785 | + netdev->max_mtu = adapter->vf_res->max_mtu - LIBETH_RX_LL_LEN; | |
| 2786 | + } | |
| 2797 | 2787 | |
| 2798 | 2788 | if (!is_valid_ether_addr(adapter->hw.mac.addr)) { |
| 2799 | 2789 | dev_info(&pdev->dev, "Invalid MAC address %pM, using random\n", |
| @@ -3021,6 +3011,8 @@ static void iavf_disable_vf(struct iavf_adapter *adapter) · 221 unmodified lines | ||
| 3021 | 3011 | |
| 3022 | 3012 | adapter->flags |= IAVF_FLAG_PF_COMMS_FAILED; |
| 3023 | 3013 | |
| 3014 | + iavf_ptp_release(adapter); | |
| 3015 | + | |
| 3024 | 3016 | /* We don't use netif_running() because it may be true prior to |
| 3025 | 3017 | * ndo_open() returning, so we can't assume it means all our open |
| 3026 | 3018 | * tasks have finished, since we're not holding the rtnl_lock here. |
| @@ -3096,18 +3088,16 @@ static void iavf_reconfig_qs_bw(struct iavf_adapter *adapter) · 69 unmodified lines | ||
| 3096 | 3088 | } |
| 3097 | 3089 | |
| 3098 | 3090 | /** |
| 3099 | − * iavf_reset_task - Call-back task to handle hardware reset | |
| 3100 | − * @work: pointer to work_struct | |
| 3091 | + * iavf_reset_step - Perform the VF reset sequence | |
| 3092 | + * @adapter: board private structure | |
| 3101 | 3093 | * |
| 3102 | − * During reset we need to shut down and reinitialize the admin queue | |
| 3103 | − * before we can use it to communicate with the PF again. We also clear | |
| 3104 | − * and reinit the rings because that context is lost as well. | |
| 3105 | − **/ | |
| 3106 | −static void iavf_reset_task(struct work_struct *work) | |
| 3094 | + * Requests a reset from PF, polls for completion, and reconfigures | |
| 3095 | + * the driver. Caller must hold the netdev instance lock. | |
| 3096 | + * | |
| 3097 | + * This can sleep for several seconds while polling HW registers. | |
| 3098 | + */ | |
| 3099 | +void iavf_reset_step(struct iavf_adapter *adapter) | |
| 3107 | 3100 | { |
| 3108 | − struct iavf_adapter *adapter = container_of(work, | |
| 3109 | − struct iavf_adapter, | |
| 3110 | − reset_task); | |
| 3111 | 3101 | struct virtchnl_vf_resource *vfres = adapter->vf_res; |
| 3112 | 3102 | struct net_device *netdev = adapter->netdev; |
| 3113 | 3103 | struct iavf_hw *hw = &adapter->hw; |
| @@ -3118,7 +3108,7 @@ static void iavf_reset_task(struct work_struct *work) · 4 unmodified lines | ||
| 3118 | 3108 | int i = 0, err; |
| 3119 | 3109 | bool running; |
| 3120 | 3110 | |
| 3121 | − netdev_lock(netdev); | |
| 3111 | + netdev_assert_locked(netdev); | |
| 3122 | 3112 | |
| 3123 | 3113 | iavf_misc_irq_disable(adapter); |
| 3124 | 3114 | if (adapter->flags & IAVF_FLAG_RESET_NEEDED) { |
| @@ -3163,7 +3153,6 @@ static void iavf_reset_task(struct work_struct *work) · 38 unmodified lines | ||
| 3163 | 3153 | dev_err(&adapter->pdev->dev, "Reset never finished (%x)\n", |
| 3164 | 3154 | reg_val); |
| 3165 | 3155 | iavf_disable_vf(adapter); |
| 3166 | − netdev_unlock(netdev); | |
| 3167 | 3156 | return; /* Do not attempt to reinit. It's dead, Jim. */ |
| 3168 | 3157 | } |
| 3169 | 3158 | |
| @@ -3175,7 +3164,6 @@ continue_reset: · 5 unmodified lines | ||
| 3175 | 3164 | iavf_startup(adapter); |
| 3176 | 3165 | queue_delayed_work(adapter->wq, &adapter->watchdog_task, |
| 3177 | 3166 | msecs_to_jiffies(30)); |
| 3178 | − netdev_unlock(netdev); | |
| 3179 | 3167 | return; |
| 3180 | 3168 | } |
| 3181 | 3169 | |
| @@ -3196,6 +3184,8 @@ continue_reset: · 14 unmodified lines | ||
| 3196 | 3184 | iavf_change_state(adapter, __IAVF_RESETTING); |
| 3197 | 3185 | adapter->flags &= ~IAVF_FLAG_RESET_PENDING; |
| 3198 | 3186 | |
| 3187 | + iavf_ptp_release(adapter); | |
| 3188 | + | |
| 3199 | 3189 | /* free the Tx/Rx rings and descriptors, might be better to just |
| 3200 | 3190 | * re-use them sometime in the future |
| 3201 | 3191 | */ |
| @@ -3316,9 +3306,6 @@ continue_reset: · 114 unmodified lines | ||
| 3316 | 3306 | |
| 3317 | 3307 | adapter->flags &= ~IAVF_FLAG_REINIT_ITR_NEEDED; |
| 3318 | 3308 | |
| 3319 | − wake_up(&adapter->reset_waitqueue); | |
| 3320 | − netdev_unlock(netdev); | |
| 3321 | − | |
| 3322 | 3309 | return; |
| 3323 | 3310 | reset_err: |
| 3324 | 3311 | if (running) { |
| @@ -3327,10 +3314,21 @@ reset_err: · 2 unmodified lines | ||
| 3327 | 3314 | } |
| 3328 | 3315 | iavf_disable_vf(adapter); |
| 3329 | 3316 | |
| 3330 | − netdev_unlock(netdev); | |
| 3331 | 3317 | dev_err(&adapter->pdev->dev, "failed to allocate resources during reinit\n"); |
| 3332 | 3318 | } |
| 3333 | 3319 | |
| 3320 | +static void iavf_reset_task(struct work_struct *work) | |
| 3321 | +{ | |
| 3322 | + struct iavf_adapter *adapter = container_of(work, | |
| 3323 | + struct iavf_adapter, | |
| 3324 | + reset_task); | |
| 3325 | + struct net_device *netdev = adapter->netdev; | |
| 3326 | + | |
| 3327 | + netdev_lock(netdev); | |
| 3328 | + iavf_reset_step(adapter); | |
| 3329 | + netdev_unlock(netdev); | |
| 3330 | +} | |
| 3331 | + | |
| 3334 | 3332 | /** |
| 3335 | 3333 | * iavf_adminq_task - worker thread to clean the admin queue |
| 3336 | 3334 | * @work: pointer to work_struct containing our data |
| @@ -4596,22 +4594,17 @@ static int iavf_close(struct net_device *netdev) · 1259 unmodified lines | ||
| 4596 | 4594 | static int iavf_change_mtu(struct net_device *netdev, int new_mtu) |
| 4597 | 4595 | { |
| 4598 | 4596 | struct iavf_adapter *adapter = netdev_priv(netdev); |
| 4599 | − int ret = 0; | |
| 4600 | 4597 | |
| 4601 | 4598 | netdev_dbg(netdev, "changing MTU from %d to %d\n", |
| 4602 | 4599 | netdev->mtu, new_mtu); |
| 4603 | 4600 | WRITE_ONCE(netdev->mtu, new_mtu); |
| 4604 | 4601 | |
| 4605 | 4602 | if (netif_running(netdev)) { |
| 4606 | − iavf_schedule_reset(adapter, IAVF_FLAG_RESET_NEEDED); | |
| 4607 | − ret = iavf_wait_for_reset(adapter); | |
| 4608 | − if (ret < 0) | |
| 4609 | − netdev_warn(netdev, "MTU change interrupted waiting for reset"); | |
| 4610 | − else if (ret) | |
| 4611 | − netdev_warn(netdev, "MTU change timed out waiting for reset"); | |
| 4603 | + adapter->flags |= IAVF_FLAG_RESET_NEEDED; | |
| 4604 | + iavf_reset_step(adapter); | |
| 4612 | 4605 | } |
| 4613 | 4606 | |
| 4614 | − return ret; | |
| 4607 | + return 0; | |
| 4615 | 4608 | } |
| 4616 | 4609 | |
| 4617 | 4610 | /** |
| @@ -5416,9 +5409,6 @@ static int iavf_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 798 unmodified lines | ||
| 5416 | 5409 | /* Setup the wait queue for indicating transition to down status */ |
| 5417 | 5410 | init_waitqueue_head(&adapter->down_waitqueue); |
| 5418 | 5411 | |
| 5419 | − /* Setup the wait queue for indicating transition to running state */ | |
| 5420 | − init_waitqueue_head(&adapter->reset_waitqueue); | |
| 5421 | − | |
| 5422 | 5412 | /* Setup the wait queue for indicating virtchannel events */ |
| 5423 | 5413 | init_waitqueue_head(&adapter->vc_waitqueue); |
| 5424 | 5414 | |
| @@ -2736,7 +2736,6 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter, · 2735 unmodified lines | ||
| 2736 | 2736 | case VIRTCHNL_OP_ENABLE_QUEUES: |
| 2737 | 2737 | /* enable transmits */ |
| 2738 | 2738 | iavf_irq_enable(adapter, true); |
| 2739 | − wake_up(&adapter->reset_waitqueue); | |
| 2740 | 2739 | adapter->flags &= ~IAVF_FLAG_QUEUES_DISABLED; |
| 2741 | 2740 | break; |
| 2742 | 2741 | case VIRTCHNL_OP_DISABLE_QUEUES: |
| @@ -1360,7 +1360,7 @@ ice_devlink_enable_roce_get(struct devlink *devlink, u32 id, · 1359 unmodified lines | ||
| 1360 | 1360 | |
| 1361 | 1361 | cdev = pf->cdev_info; |
| 1362 | 1362 | if (!cdev) |
| 1363 | − return -ENODEV; | |
13 more lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 141 lines aren’t shown here.
This diff is large: its 120 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 82 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 123 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 269 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 259 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 119 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 174 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 301 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 189 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 129 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 150 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 83 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 237 lines aren’t shown here.
This diff is large: its 255 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 236 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 253 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 150 lines aren’t shown here.
This diff is large: its 560 lines aren’t shown here.
This diff is large: its 245 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 476 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 126 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 130 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 82 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 151 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 147 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 128 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 116 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 420 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 142 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 121 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 154 lines aren’t shown here.
This diff is large: its 275 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 331 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 163 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 295 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 309 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 197 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 186 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 152 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 193 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 125 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 159 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 398 lines aren’t shown here.
This diff is large: its 315 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 172 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 163 lines aren’t shown here.
This diff is large: its 217 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 306 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 154 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 2 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 548 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 162 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 139 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 434 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 122 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 106 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 174 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 2 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 142 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 90 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 866 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 4 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 219 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 117 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 198 lines aren’t shown here.
This diff is large: its 188 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 166 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 427 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 151 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 205 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 165 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 82 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 137 lines aren’t shown here.